Using the National Training Center Instrumentation System to Aid Simulation-Based Acquisition

by Andrew Cady

Though current data sources for simulation models used in the United States Department of Defense (DoD) acquisition process are many and varied, none adequately represents how weapon systems behave in combat in a robust, quantifiable manner, leading to uncertainty in the acquisition decision making process. The objective of this dissertation is to improve this process by developing empirically derived measures of direct fire behaviors from U.S. Army National Training Center (NTC) data and by demonstrating how these measures can be used to support acquisition decisions based on the output of simulation-based modeling. To accomplish this, I employ a three-part methodology.

更多

First, I identify the current data sources for models and simulations used in the defense acquisition process. Of these four data sources — historical combat, operational testing, other simulations, and subject matter expert (SME) judgment — no single source can adequately describe combat behaviors of weapon systems across a wide range of operational environments.

Second, I turn to the NTC data as a potential solution to this gap in data sources. I first examine prior NTC-based research and lessons this literature holds for current and future research. I examine the doctrinal underpinnings of maneuver combat behaviors, deriving five important aspects of direct fire — two of which are operationalized in this dissertation: direct fire engagement and movement and maneuver. I examine the NTC instrumentation system data generation process, strengths, and drawbacks to determine if it could measure these. Finally, I describe four measures that I derive to describe maneuver combat behavior — weapon system probability of hit, weapon system rate of fire, unit dispersion, and unit speed.

Third, I compare the measures derived in this dissertation against baseline measures from the Joint Conflict and Tactical Simulation (JCATS) simulation model to determine the difference the two sources of measures — actual NTC behavior and JCATS baseline — make in three outcomes: exchange ratio, drawdown of forces rate, and volume of fire. To perform this comparison, I create a scenario based on prior simulation studies along with four excursions to test the influence of changes in mission, enemy, and terrain on the impact of data source. I analyze the results of 300 runs of the JCATS model using a series of linear regressions. For each excursion, regression models indicate a highly significant effect of data source on each model outcome.

I conclude this dissertation with a recommendation that the measures described herein form the basis for a larger system of NTC-based behavioral measurement for modeling and simulation (M&S) data. I also recommend several software and hardware improvements to the NTC instrumentation system that could improve its utility as both a data source and a training resource. As future research, I recommend applying advanced analytic techniques to these data, applying these methods to other combat training centers and applying these measures to training and tactics development.

Table of Contents

Chapter One

Introduction

Chapter Two

Current Sources of Data for Combat Simulation Modeling

Chapter Three

Deriving Behavioral Combat Measures from NTC-IS

Chapter Four

Testing the Difference in JCATS Model Outcomes from Using NTC-based Data

Chapter Five

Conclusions and Policy Recommendations

Appendix A

Pairing Fires and Hits in NTC-IS

Appendix B

Line of Sight Algorithm Description and Assumptions

Appendix C

Additional NTC-IS Measures Not Tested in this Research

Appendix D

Verification and Validation of the NTC-IS Data

Appendix E

Full Results of NTC-IS Analysis

Appendix F

Regression Model Specifications and Diagnostics

收起