![](\"https:\/\/media.wired.com\/photos\/5df994d491848b0008e657c6\/master\/pass\/Security_Shanahan_RTC9J6.jpg\"\/)
<\/p>\n<\/p>\n
Credit to Author: Elias Groll| Date: Wed, 18 Dec 2019 20:34:49 +0000<\/strong><\/p>\n Elias Groll<\/span><\/a><\/span> <\/span><\/p>\n Lt. Gen. Jack Shanahan doesn't want killer robots—but he does want artificial intelligence to occupy a central role in warfighting.<\/p>\n Nearly every day, in war zones around the world, American military forces request fire support. By radioing coordinates to a howitzer miles away, infantrymen can deliver the awful ruin of a 155-mm artillery shell on opposing forces. If defense officials in Washington have their way, artificial intelligence is about to make<\/a> that process a whole lot faster.<\/p>\n The effort to speed up fire support is one of a handful initiatives that Lt. Gen. Jack Shanahan describes as the \u201clower consequence missions\u201d that the Pentagon is using to demonstrate how it can integrate artificial intelligence into its weapons systems. As the head of the Joint Artificial Intelligence Center, a 140-person clearinghouse within the Department of Defense focused on speeding up AI adoption, Shanahan and his team are building applications in well-established AI domains\u2014tools for predictive maintenance and health record analysis\u2014but also venturing into the more exotic, pursuing AI capabilities that would make the technology a centerpiece of American warfighting.<\/p>\n Shanahan envisions an American military that uses AI to move much faster. Where once human intelligence analysts might have stared at a screen to identify and track a target, a computer would do that task. Today, a human officer might present options for what weapons to employ against an enemy; within 20 years or so, a computer could present \u201crecommendations as fast as possible to a human to make decisions about employing weapons,\u201d Shanahan told WIRED<\/em> in an interview this month. Multiple command and control systems that track battlefield conditions are to be unified into one.<\/p>\n It\u2019s not a vision for killer robots deciding who lives and dies. It\u2019s more like Waze, but for war. Or as Shanahan put it: \u201cAs much machine-to-machine interaction as is possible to allow humans to be presented with various courses of actions for decision.\u201d<\/p>\n The hurdles for implementing that plan are legion. The massive data sets needed to build those computer vision and decisionmaking algorithms are rarely of the necessary quality. And algorithms are only as good as the data sets upon which they are built.<\/p>\n Perhaps more profoundly, the military integration of intelligent computer systems raises questions about whether some realms of human life, such as the violent taking of it, should be computer-enabled. \u201cThat loss of human control moves us into questions of authorization and accountability we haven't worked out yet,\u201d says Peter Singer, a defense analyst and coauthor of the forthcoming techno-thriller Burn-In<\/a><\/em>.<\/p>\n "Twenty years from now we'll be looking at algorithms versus algorithms."<\/p>\n Jack Shanahan, JAIC<\/p>\n These ethical questions have exposed a divide within Silicon Valley about working with the Pentagon on artificial intelligence initiatives. Before he headed up the JAIC, Shanahan ran Project Maven<\/a>, the computer vision project that aimed to take reams of aerial surveillance footage and automate the detection of enemy forces. Facing an employee uproar, Google pulled out of that project in 2018, but that hasn\u2019t stopped the initiative from moving forward. Just last week, Business Insider reported<\/a> that Palantir, Peter Thiel\u2019s data analytics company, has taken over the contract.<\/p>\n The sheer size of Pentagon spending on AI\u2014difficult to determine exactly but estimated<\/a> at $4 billion for fiscal year 2020\u2014makes it unlikely any of the tech giants will stay away for long. Despite having pulled out of Maven, Google executives maintain that their company would very much like to work with the Pentagon. \u201cWe are eager to do more,\u201d Google senior vice president Kent Walker told<\/a> a National Security Commission on Artificial Intelligence conference last month. Meanwhile, Amazon CEO Jeff Bezos is using the issue to distinguish his company as one that won\u2019t shy from the controversy of taking on military work. \u201cIf Big Tech is going to turn their backs on the Department of Defense, this country is in trouble,\u201d he said<\/a> during remarks at the Reagan National Defense Forum earlier this month.<\/p>\n Bezos\u2019s public embrace of the Pentagon comes as Amazon is challenging the award of a $10 billion cloud computing contract<\/a> called JEDI, or the Joint Enterprise Defense Infrastructure, to Microsoft. That system will be key to Shanahan\u2019s AI ambitions, giving him the computing power and the shared infrastructure to crunch massive data sets and unify disparate systems.<\/p>\n It was the lack of such a cloud system that convinced Shanahan of its importance. When he ran Maven, he couldn\u2019t digitally access the surveillance footage he needed, instead having to dispatch his subordinates to fetch it. \u201cWe had cases where we had trucks going around and picking up tapes of full-motion video,\u201d Shanahan says. \u201cThat would have been a hell of a lot easier had there been an enterprise cloud solution.\u201d<\/p>\n To push updates to the system, Shanahan\u2019s team similarly had to travel to physically install newer versions at military installations. Today, Maven is getting software updates every month or so\u2014fast for government work, but still not fast enough, he adds.<\/p>\n But JEDI isn\u2019t going to solve all of Shanahan\u2019s problems, chief among them the poor quality of data. Take just one JAIC project, a predictive maintenance tool for the military\u2019s ubiquitous UH-60 Black Hawk helicopter that tries to figure out when key components are about to break. When they started collecting data from across the various branches, Shanahan\u2019s team discovered that the Army\u2019s Black Hawk was instrumented slightly differently than a version used by Special Operations Command, generating different data for machines that are essentially identical.<\/p>\n \u201cIn every single instance the data is never quite in the quality that you\u2019re looking for,\u201d he says. \u201cIf it exists, I have not seen a pristine set of data yet.\u201d<\/p>\n Data quality is one of the chief pitfalls in applying artificial intelligence to military systems; a computer will never know what it doesn\u2019t know. \u201cThere are risks that algorithms trained on historical data might face battlefield conditions that are different than the one it trained on,\u201d says Michael Horowitz, a professor at the University of Pennsylvania.<\/p>\n Shanahan argues a rigorous testing and evaluation program will mitigate that risk, and it might very well be manageable when trying to predict the moment an engine blade will crack. But it becomes a different question entirely in a shooting war at the scale and speed of which the AI has never seen.<\/p>\n The at times unpredictable nature of computer reasoning reasoning presents a thorny problem when paired with the mind of a human being. A computer may reach a baffling conclusion, one that the human who has been teamed with it has to decide whether to trust. When Google\u2019s AlphaGo defeated Lee Sedol<\/a>, the world\u2019s best Go player, in 2016, there was a moment in the match when Lee simply stood up from his chair and left the room. His computer adversary had made such an ingenious and unexpected move (from a human perspective) that Lee was flummoxed. \u201cI\u2019ve never seen a human play this move,\u201d one observer said<\/a> of the move. \u201cSo beautiful.\u201d<\/p>\n Imagine a weapons system giving a human commander a similarly incomprehensible course of action in the heat of a high-stakes conflict. It\u2019s a problem the US military is actively working on, but one for which it doesn\u2019t have a ready solution. The Defense Advanced Research Projects Agency is working on a program<\/a> to come up with \u201cexplainable AI,\u201d which aims to turn the black box of a machine-learning system into one that can provide the reasoning for the decisions it makes.<\/p>\n To build that trust, Shanahan notes commanders need to be educated in the technology early on. Projects using computer vision and satellite imagery to understand flooding and wildfire risks allow his team to learn by doing and build up expertise. \u201cYou have to understand the art of the possible or else it's all science fiction,\u201d he says.<\/p>\n But key bureaucratic hurdles also stand in Shanahan\u2019s way. A congressionally mandated report on the Pentagon\u2019s AI initiatives released this week finds that the DoD lacks \u201cbaselines and metrics\u201d to assess progress, that the JAIC\u2019s role within the DoD ecosystem remains unclear, and that the JAIC lacks the authority to deliver on its goals. It also offers a dismal assessment of the Pentagon\u2019s testing and verification regime as \u201cnowhere close to ensuring the performance and safety of AI applications, particularly where safety-critical systems are concerned.\u201d<\/p>\n In a statement, the Pentagon welcomed the report, which speaks to the immense challenges facing the US military in embracing a technology that it sees as integral to a possible conflict with Russia or China. \u201cThe speed, the op tempo of that conflict will be so fast,\u201d Shanahan says. \u201cTwenty years from now we'll be looking at algorithms versus algorithms.\u201d<\/p>\n The US response to Beijing relies in part on automation. The Army is testing<\/a> an automated gun turret. The Air Force is developing<\/a> a drone wingman. The Navy\u2019s \u201cGhost Fleet\u201d concept<\/a> is looking into unmanned surface vessels. To get faster, the Pentagon is once again turning to computers.<\/p>\n \u201cThe ultimate question we have to ask ourselves is what level of accuracy is acceptable for software,\u201d says Martijn Rasser, a former CIA analyst and a fellow at the Center for a New American Security. \u201cLet\u2019s say a human being is correct 99.99 percent of the time. Is it fine for the software to be the same, or does it need to be an order of magnitude better?\u201d<\/p>\n These are questions the Pentagon is exploring. An October report from the Defense Innovation Board laid out<\/a> a series of principles for how the military<\/a> might ethically adopt AI. Shanahan wants to hire an ethicist to join the JAIC, and he is at pains to emphasize that he is tuned into the ethical debates around military AI. He says he remains fundamentally opposed to what would be popularly thought of as \u201ckiller robots\u201d and what he calls \u201can unsupervised independent self-targeting system making life-or-death decisions.\u201d<\/p>\n He remains an optimist. \u201cHumans make mistakes in combat every single day. Bad things happen. It's chaotic. Emotions run high. Friends are dying. We make mistakes,\u201d Shanahan says. \u201cI am in the camp that says we can do a lot to help reduce the potential for those mistakes with AI enabled capabilities\u2014never eliminate."<\/p>\n