YL Blog #6: India Must Take the Lead in the AI Norms Race

July 26, 2019

There appears to be a growing consensus in global policy circles that military use of artificial intelligence (AI) technologies is “inevitable.” This perceived inevitability is largely a product of the P-5 – China, the United States, United Kingdom, France and Russia – actively pursuing military applications of AI while simultaneously opposing a ban on their development and production. The rationale for the pursuit of such technologies is simple: the advantages in terms of speed, lethality, and effectiveness provided by AI systems will make them an essential element of military dominance in the coming decades.

AI is therefore a crucial part of China’s strategy to displace the United States as the dominant power in the Indo-Pacific region. As Beijing sets its sights on dominance of the seas, it is also working to enhance its deep sea capabilities through
“intelligentized” (智能) military systems, which in turn will have broader implications for the Indo-Pacific strategic nuclear balance. China is rapidly spreading its maritime military footprint through its outposts in the South China Sea and its broader maritime influence across the Indian Ocean. China’s ambitions extend to the oceanic depths. In the past few years, China has reportedly placed acoustic sensors in the Mariana Trench, begun development on unmanned AI submarines and is in the planning stages of an AI-driven deep sea base in the South China Sea.

There is informed analysis in policy circles that China’s underwater capabilities serve a couple of strategic aims: first, they enable China to track foreign vessels belonging to adversaries, and second, they serve to strengthen China’s nuclear deterrent. The primary driver of Beijing’s dive into deep sea AI capabilities is a preoccupation with accurate detection of incoming nuclear strikes. At the same time, a fundamental dilemma of nuclear strategy, especially in the era of multiple nuclear actors, is that attempts to secure one’s own nuclear arsenal against a stronger adversary – in this case the United States – will create first strike instability with other actors in the system. In other words, China’s attempts to create a survivable nuclear arsenal by going to the deep seas will bring instability – intended or unintended – into its nuclear balance with India. India has already taken note of China’s burgeoning investment in AI and other dual-use emerging technologies. As General Bipin Rawat has said, “Our adversary on the northern border (China) is spending huge amounts of money on Artificial Intelligence and cyber warfare. We cannot be left behind.”

Yet India’s fundamental concern with regard to China’s deep sea AI capabilities should be in the apparent absence of concern in Chinese policy circles on over-reliance on technology, especially in an area as sensitive as nuclear systems. While China aggressively pursues a policy of “civil-military fusion” to rapidly develop and integrate emerging technologies into the military, it is not clear whether such integration is being undertaken with consideration of the fact that technologies can fail. A 2017 report by the Center for New American Security indicated that Chinese strategic thinkers believe it is “inevitable” that AI will replace humans, including in command decision-making. A review by the Bulletin of Atomic Scientists of Chinese literature on AI and autonomy similarly points to a greater concern about being unable to detect nuclear attacks as opposed to false warnings and inadvertent escalation.

While China aggressively pursues a policy of “civil-military fusion” to rapidly develop and integrate emerging technologies into the military, it is not clear whether such integration is being undertaken with consideration of the fact that technologies can fail.

There are numerous Cold War tales of individuals like Stanislav Petrov and Vasili Arkhipov whose tact helped avert all-out nuclear war between the Soviet Union and the US. There are equally numerous instances in recent decades of catastrophic machine and computer failures with lethal consequences. The simple fact is that although we call AI “intelligent,” this intelligence is highly domain specific, i.e., it functions best within narrow parameters and given limited goals. However, the realm of nuclear deterrence is mind-bogglingly complex: not only are there multiple actors with their own unique nuclear policy and strategic context, but deterrence is borne of the interplay of military, diplomatic, and economic components. Without a human in-the-loop who understands these considerations, an AI-driven nuclear deterrent is destined for catastrophe.

The seemingly unstoppable momentum of China’s military AI R&D has elicited concern within Chinese academia as well. A 2019 CNAS report on Chinese thinking on AI and national security highlights the following core concerns: First, the lack of casualties brought about by AI deployment would embolden military decision-makers to take risky actions they would otherwise avoid. Secondly, the lack of norms surrounding the use of AI systems could result in misperception and unintended escalation.

India must take advantage of this open door by leading rather than following. India’s leadership in norms-building exercises on emerging technologies, such as those within the aegis of the UN Convention on Certain Conventional Weapons (CCW), is a positive first step.

India must take advantage of this open door by leading rather than following.

At the normative level, Indian diplomats in key norms-making bodies should emphasize that AI cannot take the role of nuclear commander. Accordingly, no nuclear system should be completely autonomous and all decisions regarding use of force must involve a human chain of command. India should also prioritize the creation of international parameters for assessing transparency and explainability of AI algorithms.

The creation of norms in the AI space is a race in its own right, one against the rapid deployment of these technologies in potentially destabilizing roles in the deep seas, driven by Beijing’s desire for overwhelming naval dominance of the Indo-Pacific.

Trisha Ray is a Junior Fellow with the Cyber Initiative at the Observer Research Foundation. A version of this article was originally published by the Observer Research Foundation.

Disclaimer: All opinions in this article are solely those of the author and do not represent any organization.