Exploring the Feasibility and Utility of Machine Learning-Assisted Command and Control

Volume 1, Findings and Recommendations

by Matthew Walsh, Lance Menthe, Edward Geist, Eric Hastings, Joshua Kerrigan, Jasmin Léveillé, Joshua Margolis, Nicholas Martin, Brian P. Donnelly

更多

This report concerns the potential for artificial intelligence (AI) systems to assist in Air Force command and control (C2) from a technical perspective. The authors present an analytical framework for assessing the suitability of a given AI system for a given C2 problem. The purpose of the framework is to identify AI systems that address the distinct needs of different C2 problems and to identify the technical gaps that remain. Although the authors focus on C2, the analytical framework applies to other warfighting functions and services as well.

The goal of C2 is to enable what is operationally possible by planning, synchronizing, and integrating forces in time and purpose. The authors first present a taxonomy of problem characteristics and apply them to numerous games and C2 processes. Recent commercial applications of AI systems underscore that AI offers real-world value and can function successfully as components of larger human-machine teams. The authors outline a taxonomy of solution capabilities and apply them to numerous AI systems.

While primarily focusing on determining alignment between AI systems and C2 processes, the report's analysis of C2 processes is also informative with respect to pervasive technological capabilities that will be required of Department of Defense (DoD) AI systems. Finally, the authors develop metrics — based on measures of performance, effectiveness, and suitability — that can be used to evaluate AI systems, once implemented, and to demonstrate and socialize their utility.

Key Findings

C2 processes are very different from many of the games and environments used to develop and demonstrate AI systems

Game-playing algorithms exploit regularity to achieve superhuman performance, but nature and the adversary intervene to break this simplifying assumption in military tasks.

Characterizing and developing representative problems and environments will enable research, development, testing, and evaluation of AI systems under conditions representative of DoD problem sets, thereby increasing transferability to operational environments.

The distinctive nature of C2 processes calls for AI systems different from those optimized for game play

Understanding the capabilities and limitations of existing AI systems will allow the Air Force to identify systems that are suitable for different C2 processes.

Choosing the right approach at problem outset can substantially reduce application development time, increase solution quality, and decrease risk associated with transitioning the solution.

New guidance, infrastructure, and metrics are needed to evaluate applications of AI to C2

Establishing and operationalizing measures of merit will enable the evaluation and comparison of potential AI systems.

Additionally, measures of merit provide a way to communicate the return on investment of AI-enabled C2.

Hybrid approaches are needed to deal with the multitude of problem characteristics that are present in C2 processes

Given the generality of the analytical framework and the emergence of Joint All-Domain C2, all these conclusions and recommendations extend to the pursuit of AI across DoD.

Recommendations

Use the structured method described in this report to systematically analyze the characteristics of games, problems, and C2 processes to determine where existing AI test beds are representative and nonrepresentative of C2 tasks.

Develop new AI test beds with problem characteristics that are representative of C2 tasks in kind and in intensity.

Use the structured method described in this report to identify and invest in high-priority solution capabilities called for across a wide range of C2 processes and not currently available (e.g., robustness and assuredness).

Use the structured method described in this report to evaluate alignment between the characteristics of potential AI systems and particular C2 processes to prioritize which systems to develop.

Develop metrics for AI solutions that assess capabilities beyond algorithm soundness and optimality (e.g., robustness and explainability).

Use the structured method described in this report to identify key measures of performance, effectiveness, and suitability for a given C2 process.

Perform a comprehensive assessment of AI systems for a given C2 process based on identified measures of merit.

Identify, reuse, and combine algorithmic solutions that confer critical AI system capabilities.

收起