GENEVA, 17 MAY 2021 — Today the Security and Technology Programme at the United Nations Institute for Disarmament Research (UNIDIR) published “Known Unknowns: Data Issues and Military Autonomous Systems,” a comprehensive research report on the challenges that data issues pose to the safe, legal and responsible use of autonomous systems in conflict.
In order to perform as desired, autonomous systems must collect data from their environment that are complete, relevant, accurate, and aligned with the data for which the system was developed and tested. But the harsh, dynamic, complex and adversarial nature of conflict environments poses a wide range of obstacles to the collection of such data. As a result, autonomous systems cannot always be expected to achieve the exact same performance in real-world use that they demonstrated in development or testing. And crucially, they will be liable to failures that are both inevitable and impossible to anticipate: “known unknowns.”
“All complex weapon systems fail, but autonomous systems are unique in the sense that their failure modes are wide, inevitable and unforeseeable,” said Arthur Holland Michel, an associate researcher at UNIDIR and the author of the report. “Figuring out how to characterize, respond to, and address such ‘known unknowns’ is going to be a fundamental task for the international community moving forward.”
This new report describes common data issues for autonomous systems and explains how they give rise to “known unknown” failures. It then explores the legal and operational implications of such failures, and considers a range of potential policy and technical solutions by which they could be addressed.
“This report will be a valuable resource for all stakeholders working on issues of autonomy and AI in conflict, particularly with respect to reviews and risk assessments of such systems,” said UNIDIR Director Robin Geiss. “It covers a wide range of concrete ideas for action.”
ABOUT UNIDIR
UNIDIR is a voluntarily funded, autonomous institute within the United Nations. One of the few policy institutes worldwide focusing on disarmament, UNIDIR generates knowledge and promotes dialogue and action on disarmament and security. Based in Geneva, UNIDIR assists the international community to develop the practical, innovative ideas needed to find solutions to critical security problems. For more information about UNIDIR, visit: www.unidir.org.
Follow UNIDIR on Facebook, Instagram, and Twitter.
MEDIA CONTACT
Aaron J. Buckley
Strategic Communications Officer
+41 76 691 10 74
buckleya@un.org
Arthur Holland
Associate Ressearcher
+41 22 917 13 56