Predictability and understandability are widely held to be vital characteristics of artificially intelligent systems. Put simply: AI should do what we expect it to do, and it must do so for intelligible reasons. This consideration stands at the heart of the ongoing discussion about lethal autonomous weapon systems and other forms of military AI. But what does it mean for an intelligent system to be "predictable" and "understandable" (or, conversely, unpredictable and unintelligible)? What is the role of predictability and understandability in the development, use, and assessment of military AI? What is the appropriate level of predictability and understandability for AI weapons in any given instance of use? And how can these thresholds be assured?
This study provides a clear, comprehensive introduction to these questions, and proposes a range of avenues for action by which they may be addressed.
Citation: Arthur Holland Michel (2020) "The Black Box, Unlocked", UNIDIR, Geneva. doi: 10.37559/SecTec/20/AI1
Teaser: Predictability and Understandability in Military AI