Air Dominance Through Machine Learning

A Preliminary Exploration of Artificial Intelligence–Assisted Mission Planning

by Li Ang Zhang, Jia Xu, Dara Gold, Jeff Hagen, Ajay K. Kochhar, Andrew J. Lohn, Osonde A. Osoba

更多

U.S. air superiority, a cornerstone of U.S. deterrence efforts, is being challenged by competitors—most notably, China. The spread of machine learning (ML) is only enhancing that threat. One potential approach to combat this challenge is to more effectively use automation to enable new approaches to mission planning.

The authors of this report demonstrate a prototype of a proof-of-concept artificial intelligence (AI) system to help develop and evaluate new concepts of operations for the air domain. The prototype platform integrates open-source deep learning frameworks, contemporary algorithms, and the Advanced Framework for Simulation, Integration, and Modeling—a U.S. Department of Defense–standard combat simulation tool. The goal is to exploit AI systems' ability to learn through replay at scale, generalize from experience, and improve over repetitions to accelerate and enrich operational concept development.

In this report, the authors discuss collaborative behavior orchestrated by AI agents in highly simplified versions of suppression of enemy air defenses missions. The initial findings highlight both the potential of reinforcement learning (RL) to tackle complex, collaborative air mission planning problems, and some significant challenges facing this approach.

Key Findings

RL can tackle complex planning problems but still has limitations, and there are still challenges to this approach

Pure RL algorithms can be inefficient and prone to learning collapse.

Proximal policy optimization is a recent step in the right direction for addressing the learning collapse issue: It has built-in constraints preventing the network parameters from changing too much in each iteration.

ML agents are capable of learning cooperative strategies. In simulations, the strike aircraft synergized with jammer or decoy effects on a SAM.

Trained algorithms should be able to deal with changes in mission parameters (number and locations of assets) fairly easily.

Few real-world data exist on successful and unsuccessful missions. Compared with the volumes of data used to train contemporary ML systems, very few real missions have been flown against air defenses, and virtually all of them were successful.

For analyses involving the use of large simulations in place of large datasets, the required computational burden will continue to be a significant challenge. The scaling of computational power and time required to train realistic sets of capabilities (dozens of platforms) against realistic threats (dozens of SAMs) remains unclear.

Developing trust in AI algorithms will require more-exhaustive testing and fundamental advances in algorithm verifiability, and safety and boundary assurances.

Recommendations

Future work on automated mission planning should focus on developing robust multiagent algorithms. Reward functions in RL problems can drastically change AI behavior in often unexpected ways. Care must be taken in designing such functions to accurately capture risk and intent.

Although simulation environments are crucial in data-scarce problems, simulations should be tuned to balance speed (lower computational requirements) versus accuracy (real-world transferability).

Table of Contents

Chapter One

Introduction

Chapter Two

One-Dimensional Problem

Chapter Three

Two-Dimensional Problem

Chapter Four

Computational Infrastructure

Chapter Five

Conclusions

Appendix A

2-D Problem State Vector Normalization

Appendix B

Containerization and ML Infrastructure

Appendix C

Managing Agent-Simulation Interaction in the 2-D Problem

Appendix D

Overview of Learning Algorithms

2020年5月31日，美国兰德公司发布《通过机器学习实现空中优势：对人工智能辅助任务规划的初步探索》（Air Dominance Through Machine Learning：A Preliminary Exploration of Artificial Intelligence–Assisted Mission Planning）研究报告，该研究为人工智能原型系统在空战环境中开发和评估新型作战概念的潜力提供了证据支撑。

研究团队测试了几种学习技术和算法来训练能够在模拟环境中进行空战规划的智能代理，目标是利用人工智能系统的能力大规模地反复模拟和持续改进，从而加速并丰富作战概念的发展。报告指出，人工智能任务规划工具相比现有的人工或自动规划技术将具有极大的速度优势。

该研究最大的目的是在简化的仿真模拟环境中验证人工智能系统进行空战任务规划进而创新战法的应用潜力，值得关注的亮点和结论有：1）假想的场景是给定一组配置不同传感器、武器、诱饵和电子战载荷的无人机对孤立防空体系进行进攻，场景的选取贴合蜂群、“马赛克战”等的基本构想；2）将开源深度学习框架与国防部标准作战模拟工具“仿真、集成与建模高级框架”（AFSIM）集成，分别测试了生成对抗网络（GAN）、Q学习、异步优势动作评价（A3C）、近端策略优化（PPO）等当前最新的开源算法，结果显示只有PPO算法可以在一组变化的复杂场景下满足任务规划需求；3）人工智能算法在任务规划中展现了一定优势和潜力，倘若在算法、训练和部署中进一步深化研究，有望在速度上相比现有的人工或自动任务规划带来巨大优势；4）强化学习是在数据资源稀缺情况下利用人工智能实现作战概念和战术创新最有力的工具，回报函数的设置是决定人工智能演进策略的关键所在，是作战智慧的核心涌现；5）在真实场景和关键任务中使用基于统计学习方法的人工智能系统必须经过充分的试验、测试和认证。因此必须重视和加快航空可信人工智能框架方面的工作，其重要性不亚于其他航空人工智能应用探索工作。

收起