Air Force Capability Development Planning

Analytical Methods to Support Investment Decisions

by James A. Leftwich, Debra Knopman, Jordan R. Fischbach, Michael J. D. Vermeer, Kristin Van Abel, Nidhi Kalra

更多

In view of uncertainties about the future threat environment, trajectories of technological development, and shifting budgeting priorities, RAND Project AIR FORCE examined analytical methods that would best guide the recently established Air Force Warfighting Integration Capability in capability development planning (CDP), as well as concept development and future force design. In their review, researchers found that specific methods of decisionmaking under deep uncertainty (DMDU) can provide the most suitable means for arriving at solutions that are flexible, adaptable, and robust and guiding investment pathways and modernization efforts.

The method highlighted, Robust Decisionmaking, rests on a simple concept. Rather than using models and data to assess policies under a single set of assumptions, RDM runs models over hundreds to thousands of different sets of assumptions about the problem space with the aim of understanding how plans perform under many plausible conditions. Each of the four steps of RDM — decision-framing, case generation, vulnerability assessment and scenario discovery, and trade-off analysis — feeds into the next, providing stakeholders and decisionmakers with a more informed tradespace. By exposing vulnerabilities of different options under different scenarios, RDM can enable stakeholders to engage in a rich dialogue about which risks or vulnerabilities are acceptable, as well as to review assumptions made in framing the problem and make adjustments as needed.

Key Findings

Compared with existing methods currently in use for CDP, DMDU methods — and specifically the RDM paradigm — could support investment decisionmaking in situations when evidence is lacking to quantify the probability of future threats and risks and in the face of deep uncertainties.

A workshop in which RAND experts and Air Force stakeholders explored ways to frame a decision problem revealed the layers of complexity embedded within the CDP challenge to produce decision-quality analysis to support design and planning choices.

Workshop participants identified technical limitations to implementing RDM, including availability of models and their fidelity.

Participants also noted that the practicalities of developing competencies in a new approach for which the capabilities may need to be built rather than simply adapted will be a challenge. A related issue is the adequacy of internal staffing and capacity to support CDP.

RDM and similar methods represent a fundamental shift in problem-solving from prediction-then-act analysis and decisionmaking to a risk management approach of seeking robust solutions across a range of future threat environments and other uncertainties. The key output of this type of approach is not a single answer but rather trade-off curves or other visualizations that compare performance of alternative strategies or portfolios of options with the status quo or other baseline in terms of metrics that reflect what is most important to the Air Force.

Recommendations

A first step in strategic developing planning and experimentation would be to choose a timely and relevant investment decision problem and demonstrate the robustness of a number of alternative investment portfolios using a mostly qualitative approach that would alleviate the need for sophisticated — or nonexistent — models capable of encompassing a range of emerging technologies, such as quantum computing, artificial intelligence, hypersonics, or directed energy. The analysis should consider performance under a wide range of future conditions and visualize trade-offs among these alternatives across the performance metrics relevant to Air Force decisionmakers.

To be workable, a test application will need to be amenable to analysis. Models should be understandable and appropriate for exploring broad-ranging uncertainty analysis; in the absence of numerical models, conceptual models may be used.

In the test application, there should be basic access to data that could be used to establish performance under baseline or status-quo concepts of operation and existing capabilities.

In the test application, classification levels should be managed to enable serious analysis and reveal key developments underway but not overly restrict the necessary audience of key analysts and decisionmakers.

Air Force stakeholders should be willing to engage in an iterative process in developing a decision-framing matrix, including the identification of performance metrics, key uncertainties, modeling tools, and investment/divestment strategies (levers).

The problem can be scoped to accommodate a qualitative analysis within 8–12 months of initiation, including all relevant data collection, modeling, analysis, visualization, and interactive review of and response to results.

Table of Contents

Chapter One

Introduction

Chapter Two

Capability Development Planning in the Air Force

Chapter Three

Current Assessment and Aspirational View of Capability Development Planning

Chapter Four

Analytic Methods for Capability Development Planning

Chapter Five

Decision-Framing Exercise

Chapter Six

Recommendations and Next Steps

Appendix

Background Information on Capability Development Planning and Strategy, Planning, and Programming Process

收起