A Framework for Assessing the Costs and Benefits of Digital Engineering
A Systems Approach
Published Mar 13, 2024
更多
by N. Peter Whitehead, Thomas Light, Adrian Luna, Jim Mignano
RAND researchers worked to understand the costs and benefits of digital engineering in the U.S. Department of Defense (DoD) and develop a decision support framework for digital engineering activities in weapon system programs. To prepare, the authors reviewed the literature and interviewed stakeholders to understand the current state of digital engineering practice and prior efforts to assess the costs and benefits of digital engineering and model-based systems engineering. They then developed decision support frameworks incorporating (1) established DoD cost-benefit analysis approaches and (2) established systems engineering decision methodologies. Along the way, the authors noted critical issues with rigor and risks in the practice of DoD digital engineering and added that aspect to the study.
This research suggests that cost-benefit decision support for digital engineering is possible at any stage of a weapon system program life cycle if program data have been collected accordingly or if goal-based systems engineering principles are leveraged. Calculating definitive costs and benefits of digital engineering is imperfect because no analyst will have access to an identical weapon system program developed without digital engineering — the counterfactual scenario.
Key Findings
Absent a consensus about what constitutes digital engineering, scholars and practitioners lack a common foundation on which to conduct analysis of costs and benefits.
Tracking digital engineering investments and distinguishing them from other costs (e.g., systems engineering, multipurpose IT investments) does not appear to be a standard practice.
Basing program decisions for digital engineering resource allocation on a systems thinking approach might produce optimal program outcomes.
Much effort has been devoted to making the case for digital engineering by projecting possible benefits without, at the same time, evaluating costs.
Some authors — and some services — are looking to a digital engineering maturity model as a path to improving outcomes without any cost data, causality map, or risk analysis to support that conclusion.
Recommendations
Develop consistency and goal-focused consensus in what digital engineering is (and is not).
Establish clear digital engineering program goals and system boundaries to eliminate pretending and to eliminate applying software principles and standards to complex systems.
Collect program goal-derived data to support assessment of digital engineering costs and benefits.
Eliminate aspirational and general guidance (top-down direction) for implementing digital engineering in such terms as to the maximum extent practical.
Promote an environment in which digital engineering is a useful tool among many others to be used in good systems engineering practice and weapon system project management.
Objectively analyze enterprise digital engineering tools in their operational context.
Promote digital engineering practice and culture where learning from mistakes is as important as achieving success.
Establish policy whereby understanding and mitigating risks is a key facet of digital engineering and MBSE practice.
Establish a roadmap for identifying and achieving absorptive capacity needs across DoD and the respective services in conjunction with the respective digital transformations.
Develop a framework for establishing the hierarchical goals for leveraging data and intellectual property in a weapon system program.
Establish DoD weapon system program life cycle decision and milestone gate criteria for effectively leveraging modeling and simulation based on established systems engineering practice.
Objectively study and develop recommendations for the long-term life cycles of multi-level classification systems, data, and infrastructures as required.
收起
