Operational Feasibility of Adversarial Attacks Against Artificial Intelligence

by Li Ang Zhang, Gavin S. Hartnett, Jair Aguirre, Andrew J. Lohn, Inez Khan, Marissa Herron, Caolionn O'Connell

A large body of academic literature describes myriad attack vectors and suggests that most of the U.S. Department of Defense's (DoD's) artificial intelligence (AI) systems are in constant peril. However, RAND researchers investigated adversarial attacks designed to hide objects (causing algorithmic false negatives) and found that many attacks are operationally infeasible to design and deploy because of high knowledge requirements and impractical attack vectors. As the researchers discuss in this report, there are tried-and-true nonadversarial techniques that can be less expensive, more practical, and often more effective. Thus, adversarial attacks against AI pose less risk to DoD applications than academic research currently implies. Nevertheless, well-designed AI systems, as well as mitigation strategies, can further weaken the risks of such attacks.

更多

Key Findings

Adversarial attacks designed to hide objects from AI pose less risk to DoD applications than academic research currently implies.

In the real world, such adversarial attacks are difficult to design and deploy because of high knowledge requirements and infeasible attack vectors; there are often less expensive, more practical, and more effective nonadversarial techniques available.

Fusing data and predictions across sensor modalities, signal-sampling rates, and image resolution can further mitigate the risk of adversarial attacks against AI.

Recommendations

DoD should assess how at-risk its AI models are to adversarial attacks by considering how adversaries can feasibly influence models. It should also assess how knowledge leaks about models can affect attack efficacy and estimate the costs associated with adversary actions.

DoD should maintain situational awareness of academic state-of-the-art techniques to attack AI in real-world scenarios and understand how these technologies feasibly affect concepts of operation for both itself and its adversaries.

DoD should develop robust AI models, preprocessing techniques, and proper data fusion systems to vastly increase the resource costs to an adversary to perform an attack.

DoD should invest in responsive support teams for AI systems to quickly detect, identify, and respond to adversarial threats.

收起