9.5.4. Test and Evaluation Strategy (Milestone A)
9.5.4.1. Description
The TES describes the concept for tests and evaluations throughout the program life cycle, starting with Technology Development and continuing through EMD into Production and Deployment. The TES evolves into the TEMP at Milestone B. Development of a TES requires early involvement of testers, evaluators, and others as a program conducts pre-system acquisition activities. These personnel provide the necessary technical, operational, and programmatic expertise to ensure nothing is overlooked in laying out a complete strategy. The TES approval process is explained in 9.5.4.3.
The TES must remain consistent with the Technology Development Strategy (TDS) and Initial Capabilities Document (ICD). The TES should address the identification and management of technology risk, the evaluation of system design concepts against the preliminary mission and sustainment requirements resulting from the analysis of alternatives, competitive prototyping, early demonstration of technologies in operationally relevant environments, and the development of an integrated test approach. The TES also satisfies the TDS test plan to ensure the completion of goals and exit criteria for the technology demonstrations in a relevant environment in accordance with section 2359a of title 10 USC. It also provides a road map for evaluations, integrated test plans, and resource requirements necessary to accomplish the TD phase objectives.
The TES begins by focusing on TD phase activities, and describes the demonstration of component technologies under development in an operationally relevant environment to support the program's transition into the EMD Phase. It contains hardware and software maturity success criteria used to assess key technology maturity for entry into EMD. For programs following an evolutionary acquisition strategy with more than one developmental increment, the TES describes the application of T&E and M&S to each planned increment to provide the required operational effectiveness, suitability, and survivability or operational security, as would be required of a program containing only one increment. TES development supports the initial Milestone A decision. The TEMP subsumes the TES for all increments thereafter, unless a follow-on increment requires a new Milestone A decision. TES development establishes an early consensus among T&E WIPT member organizations on the program’s scope for testing and evaluation, with particular consideration given to needed resources to support PPB&E process activities. The TES requires the inclusion of cost estimates beginning with program initiation and continuing through development and production, including nonrecurring and recurring research and development (R&D) costs for prototypes, engineering development equipment and/or test hardware (and major components thereof). Additionally, the TES fully identifies and estimates contractor T&E and Government support to the test program. Estimate any support, such as support equipment, training, data, and military construction. Include the cost of all related R&D (such as redesign and test efforts necessary to install equipment or software into existing platforms). See DoD 5000.4-M, "Cost Analysis Guidance Procedures," Table C2.T2, "Defense Acquisition Program Life-Cycle Cost Categories Research and Development," for a more specific list of R&D costs. The basis for the T&E resources required in the Cost Analysis Requirements Description comes from the TES’ cost information.
9.5.4.2. TES Content and Format
The following content and format provides all necessary information for a TES, and assists in the transition to a TEMP at Milestone B.
PART I – INTRODUCTION
1.1. Purpose. State the purpose of the TES. Reference the documentation initiating the TES (i.e., ICD, AoA, CONOPS).
1.2. Mission Description. Briefly summarize the mission need described in the capability requirements documents in terms of the capability it will provide to the Joint Forces Commander. Briefly summarize the CONOPS, and include a high level operational concept graphic (OV-1) or similar diagram.
1.3. System Description. Describe the system or prototype configurations. Identify key features, technologies, and components, both hardware and software for the planned Technology Development phase.
1.3.1. System Threat Assessment. Succinctly summarize the threat environment in which the system or components will operate. Reference the appropriate DIA- or DoD Component-validated threat documents.
1.3.2. Program Background. Briefly discuss any background information. Reference the AoA, the materiel development decision, and any previous tests or evaluations that have an effect on the T&E strategy.
1.3.3. Key Capabilities. Identify the system attributes that support key capabilities from the ICD. Identify the T&E-related TD Phase exit criteria.
1.3.3.1. Key Interfaces. Identify interfaces with existing or planned systems’ architectures (to the extent known at Milestone A) that are required for mission accomplishment.
1.3.3.2. Special Test Requirements. Identify unique system characteristics or support concepts that will necessitate development of special test and evaluation assets or techniques.
1.3.3.3. SE Requirements. Summarize SE-based information driving the Technology Development phase and prototype development. Reference the SEP and other applicable source documents.
PART II – TEST and EVALUATION PROGRAM MANAGEMENT AND SCHEDULE
2.1. T&E Management. Discuss the test and evaluation role of participating organizations. Describe the role of contractor and governmental personnel. Provide organizational construct that includes organizations such as the T&E WIPT or Service equivalent.
2.2. T&E Data Strategy. Describe the strategy and methods for collecting, validating, and sharing data as it becomes available from the contractors, DT&E, and oversight organizations.
2.3. Integrated Test Program Schedule. Provide the overall time sequencing of the major events with an emphasis on the TD phase. Include event dates such as major decision points, preliminary design reviews, prototypes and test article availability, and phases of DT&E.
PART III – TEST AND EVALUATION STRATEGY
3.1. T&E Strategy Introduction. This section should summarize an effective and efficient approach to the T&E program.
3.2. Evaluation Framework. Describe the overall concept of the T&E program with an emphasis on decisions in the Technology Development phase and information required to draft the CDD. Specific areas of evaluation should include Technology Readiness Level (TRL) and prototype testing. Include a Top-Level Evaluation Framework matrix that shows the correlation between decisions, the primary capabilities, critical technologies, critical technical parameters, and other key test measures.
3.3. Developmental Evaluation Approach. The discussion should be related to the TD phase, including a focus on ICD issues. If applicable, discuss the T&E supporting the reliability growth approach.
3.3.1. Developmental Test Objectives. Summarize the planned objectives and state the methodology to test the technology attributes defined by the TDS.
3.3.2. Modeling & Simulation. Describe the key models and simulations and their intended use. Identify who will perform M&S verification, validation, and accreditation.
3.3.3. Test Limitations. Discuss any test limitations that may significantly affect the evaluator's ability to draw conclusions about the TRL and capabilities.
3.4. Operational Evaluation Approach. Discuss the approach during the TD phase to providing operational insights from the user perspective, including resolution of the ICD issues. Include reliability growth testing, if appropriate.
3.4.1. Mission-Oriented Approach. Describe the approach to evaluate the system performance at the appropriate TRLs.
3.4.2. Operational Test Objectives. Summarize the planned objectives and state the methodology to test the technology attributes defined by the TDS.
3.4.3. M&S. Describe the key models and simulations and their intended use. Identify who will perform M&S verification, validation, and accreditation.
3.4.4. Test Limitations. Discuss any test limitations that may significantly affect the evaluator's ability to draw conclusions about the TRL and capabilities.
3.5. Future Test and Evaluation. Summarize all remaining significant T&E that has not been discussed yet, extending through the acquisition life cycle. Test events after Milestone B will be described in detail in the Milestone B TEMP update.
PART IV – RESOURCE SUMMARY
4.1. Introduction. Testing will be planned and conducted to take full advantage of existing DoD investment in ranges, facilities, and other resources wherever practical. Describe all key test and evaluation resources, both government and contractor, that will be used during the course of the TD phase. Include long-lead items for the next phase, if known.
4.1.1. Test Articles. Identify the prototypes and test articles.
4.1.2. Test Sites and Instrumentation. Identify the test ranges and facilities to be used for testing.
4.1.3. Test Support Equipment. Identify test support, analysis equipment, and personnel required to conduct testing.
4.1.4. Threat Representation. Identify the type, number, availability, fidelity requirements, and schedule for representations of the threat (to include threat targets) to be used in testing.
4.1.5. Test Targets and Expendables. Specify the type, number, availability, and schedule for test targets and expendables, (e.g. targets, weapons, flares, chaff, sonobuoys, countermeasures).
4.1.6. Operational Force Test Support. Specify the type and timing of aircraft flying hours, ship steaming days, and on-orbit satellite contacts/coverage, and other operational force support.
4.1.7. Simulations, Models and Testbeds. Specify the models and simulations to be used. Identify opportunities to simulate any of the required support. Identify the resources required to validate and accredit their usage, responsible agency, and timeframe.
4.1.8. Joint Mission Environment. Describe the live, virtual, or constructive components or assets necessary to create an acceptable environment to evaluate TRLs and mission capabilities.
4.1.9. Special Requirements. Identify requirements for non-instrumentation capabilities or instrumentation and analysis tools that require development or upgrades.
4.2. Test and Evaluation Funding Summary. Provide initial estimates of DT&E, OT&E, and LFT&E costs.
9.5.4.3. TES Approval Process
For programs under OSD T&E oversight, the PM or leader of the concept development team, with the T&E WIPT providing support, submits the DoD Component/Defense Agency-approved TES to OSD for staffing and approval before Milestone A. The PM should submit the TES at least 45 days prior to Milestone A to support the decision. The DOT&E and the DASD(DT&E) approve the TES for all programs on the OSD T&E Oversight List. For programs not on the OSD T&E Oversight List, the CAE, or designated representative, approves the TES.
DAG Tutorial 
