Skip to main content
default

Publication
Known Unknowns

Known Unknowns

17 May 2021

Arthur Holland

41 Pages

In order to perform as desired, autonomous systems must collect data that are complete, relevant, accurate, and aligned with the data for which the system was developed and tested. But the harsh, dynamic, complex and adversarial nature of conflict environments poses a wide range of obstacles to the collection of such data. As a result, autonomous systems cannot always be expected to achieve the exact same performance in real-world use that they demonstrated in development or testing. And crucially, they will be liable to failures that are both inevitable and impossible to anticipate: “known unknowns.” 

Data and its vagaries therefore have significant implications for the application of international humanitarian law and other rules of war. This report describes common data issues for autonomous systems and explains how they give rise to "known unknown" failures. It then explores the legal and operational implications of such failures, and considers a range of potential policy and technical solutions by which they could be addressed.

Holland Michel, Arthur. 2021. Known Unknowns: Data Issues and Military Autonomous Systems. Geneva: United Nations Institute for Disarmament Research. https://doi.org/10.37559/SecTec/21/AI1

EXECUTIVE SUMMARY

Known Unknowns Executive Summary Preview

This publication is an executive summary for UNIDIR’s report, Known Unknowns: Data Issues and Military Autonomous Systems.

To read the executive summary click here.

linked programmes

Security and Technology

sponsor organization

The Governments of Germany, the Netherlands, Norway and Switzerland, and Microsoft

Share
Download the publication
Cite this publication