Ultimately, management responsibility for an acquisition program’s T&E resides with the PM. However, the planning, executing, and reporting of T&E involves interactions, support, and oversight from other organizations within OSD, the Services, Defense Agencies, and in some cases, other government agencies; as well as the system contractor(s). The PM charters a T&E WIPT early in the acquisition model to support development of test strategies and estimates of resource requirements, strengthening the overall input to the program’s integrated product team (IPT). For additional information, consult “Rules of the Road – A Guide for Leading a Successful Integrated Product Team,” October 1999.

The PM, in concert with the user and the T&E community, coordinates DT&E,

OT&E, LFT&E, family-of-systems (FoS) interoperability testing, IA testing, reliability and maintainability (RAM) growth testing (DTM 11–003, “Reliability Analysis, Planning, Tracking, and Reporting,” dated December 2, 2011), and modeling and simulation (M&S) activities, into an efficient continuum, closely integrated with requirements definition and systems design and development. The PM has responsibility for the development and final approval of the TEMP that effectively describes the overall strategy for T&E supporting the program’s acquisition strategy and Systems Engineering Plan (SEP), and the resources necessary to execute the test program. MDAP/MAIS programs and programs identified as being on OSD T&E Oversight List require Component level approval and OSD approval by DASD DT&E for programs on DT&E oversight and DOT&E for programs on OT&E and/or LFT&E oversight. For a program requiring LFT&E in accordance with section 2366 of title 10 USC, the PM must ensure timely submission of waivers and alternative plans to meet SecDef obligations to advise Congress of any deviations from full up, system level (FUSL) LFT&E requirements. All MDAP/MAIS programs should identify key leadership positions (KLPs) early in the acquisition process. An early charter for a T&E WIPT proves essential to the success of a test and evaluation program.

9.2.2. T&E Board of Directors (BoD(ES))

Acting as the agent for the Service Vice Chiefs and equivalent OUSD and Defense Agency representatives with T&E management responsibilities is the BOD Executive Secretariat (BOD(ES)), consisting of the Service T&E principals and equivalent OUSD and Defense Agency representatives with T&E infrastructure management responsibilities. The BOD(ES):

Endorses guidance and policy for T&E infrastructure and investment management to ensure a disciplined test process that supports weapon, IT & NSS system acquisition and operational, safety, suitability, and effectiveness assessments with a cost-effective infrastructure.
Supports program review and advocacy for T&E capabilities and requisite infrastructure to OSD and Congress.
Endorses the T&E Executive Agent Test Resources Master Plan.
Approves and directs studies in support of T&E infrastructure management, standards, policy, configuration and investments.
Endorses T&E infrastructure standards that promote interoperability and commonality among test centers and ranges.
Endorses processes for workload measurement, forecasting, utilization, and full cost visibility application to T&E infrastructure investments and other related decisions.
Endorses principles of T&E Reliance (joint OSD and individual Services’ efforts to maximize commonality, interoperability, and effective utilization of products and services in support of the T&E infrastructure).
Approves joint T&E requirements and recommends solutions from the needs and solutions process for the Central T&E Investment Program (CTEIP) consideration.
Serves as the T&E representatives on the OSD chartered Defense Test and Training Steering Group (DTTSG).

9.2.3. Component and Joint T&E Organizations

9.2.3.1. Defense Information Systems Agency (DISA) T&E Executive (TEO)

The DISA T&E Executive serves as the Test, Evaluation, and Certification (TE&C) subject matter expert and Special Advisor to the DISA Director, DISA, and Senior Executive Leadership. The DISA T&E Executive duties and responsibilities include:

Establishing and providing oversight of DISA’s overarching TE&C strategies, policies, and procedures as well as missions and functions.
Coordinating accomplishment of TE&C goals and investment strategies with DISA’s Joint Interoperability Test Command (JITC), program executive officer (PEOs), and PMO’s for the development and management of the DISA T&E Resource Management Plan.
Providing oversight of DISA TE&C missions and functions, to include formulation of overarching T&E strategies, policies, and program direction.
Providing policy oversight and resource management.
Publishing and enforcing TE&C policies and guidance related to agency acquisition programs and projects, examines TE&C strategies to ensure consistent application of sound agile TE&C strategies, methodologies, and processes.
Providing TE&C oversight and support for the agency in the development of program documentation (e.g., TES and TEMP) to ensure governance, construct, infrastructure, and operations satisfy legal and regulatory requirements for adequate TE&C. Functions as the final TE&C review authority and signatory for TEMPs prior to Component Acquisition Executive (CAE) and OSD approval and signature.
Leading internal and external transitional TE&C concepts and methodologies to ensure agile, mission capabilities-based, and Warfighter-relevant processes for IT Systems and Services for the agency and DoD.
Representing the agency to the DoD T&E community, ensuring alignment with the OSD and Joint Staff as a member of the T&E BoD(ES) and as a voting member of the Military Communications-Electronics Board (MCEB) Interoperability Policy & Certification Panels (IP/ICP) as well as other OSD TE&C advisory working groups.
Providing oversight and development of Agency’s TE&C career management plan for recruiting, training, and retaining a professional TE&C workforce. Serves as the track manager for the DAWIA T&E component.

9.2.3.2. Assistant Deputy Under Secretary of the Army for Test & Evaluation (ADUSA(T&E))

Within the Army, the T&E Executive is the Director, T&E Office under the authority, direction, and control of the Deputy Under Secretary of the Army. Key Army T&E Executive duties and responsibilities include:

Serving as the senior advisor to the Secretary of the Army and the Chief of Staff, Army, on all Army T&E matters.
Advising the Army Systems Acquisition Review Council (ASARC), the Army Requirements Oversight Council (AROC), and OIPTs on T&E matters.
Approving test-related documentation for the Secretary of the Army and forwards, as appropriate, to OSD.
Coordinating T&E matters with the Joint Staff and OSD, to include serving as principal Army interface on matters of T&E with the USD(AT&L) and DOT&E.
Overseeing all Army T&E missions and functions, to include formulating overarching Army T&E strategy, policy, and program direction, providing policy oversight, and managing resources.
Providing HQDA oversight on the funding of the Army Threat Simulator Program, Army Targets Program, and Army Instrumentation Program; and coordinate with the Project Manager for Instrumentation, Targets, and Threat Simulators (PM ITTS).
Overseeing Army responsibilities in Joint T&E, Foreign Comparative Testing (FCT), and multi-Service and multinational T&E acquisition programs.
Serving as the Acquisition Workforce Functional Chief for the T&E acquisition workforce Career Field.

9.2.3.3. Director, Air Force Test & Evaluation (AF/TE)

The Air Force T&E Executive serves as the Director, Air Force Test and Evaluation (AF/TE), who serves under the authority and direction of the Secretary of the Air Force (SECAF) and the Chief of Staff of the Air Force (CSAF). In this capacity, the AF/TE:

Functions as the sole focal point for Air Force T&E policy, guidance, direction, and oversight for the formulation, review, and execution of T&E plans, programs, and budgets.
Functions as the chief T&E advisor to senior Air Force leadership on T&E processes; DT&E, including contractor testing and LFT&E; OT&E; and the use of M&S in T&E.
Functions as the final T&E review authority and signatory for TEMPs prior to CAE and OSD approval and signature.
Collaborates with requirements sponsors and system developers to improve operational requirements, system development, and the fielding of operationally effective, suitable, safe, and survivable systems.
Reviews and/or prepares T&E information for timely release to OSD, Congress, and decision makers.
Oversees the Air Force T&E infrastructure by determining the adequacy of T&E resources required to support system acquisition activities. Administers various T&E resource processes and chairs or serves on various committees, boards, and groups supporting T&E activities.
Acts as the single point of entry for the Air Force Foreign Materiel Program.
Manages the Air Force Joint Test & Evaluation Program according to DoDI 5010.41 “Joint Test and Evaluation (JT&E) Program,” dated September 12, 2005.
Functions as the certifying authority for T&E personnel for T&E Level 3 in the Acquisition Professional Development Program (APDP) when not delegated to the Major Commands (MAJCOMs).

9.2.3.4. Department of the Navy Test & Evaluation Executive (OPNAV N091)

The Director, Test and Evaluation and Technology Requirements (OPNAV N091) serves as the Department of Navy (DON) T&E Executive. The DON T&E Executive reports to the Chief of Naval Operations (CNO), the Commandant of the Marine Corps (CMC), and the Principle Military Deputy to the Assistant Secretary of the Navy for Research, Development, and Acquisition (PMD ASN (RDA)) on all matters pertaining to test and evaluation.

The DON T&E Executive supports and advises the Vice Chief of Naval Operations (VCNO) regarding the VCNO’s role on the T&E BOD and serves as the Navy representative on the T&E BOD Executive Secretariat.

The Director, Test and Evaluation and Technology Requirements (N091):

Approves all Navy Test and Evaluation Master Plans for CNO.
Establishes Navy T&E requirements and promulgates policy, regulation, and procedures governing Navy T&E.
Acts for CNO in resolving T&E requirements.

9.2.4. Office of the Secretary of Defense T&E Management

9.2.4.1. Developmental Test & Evaluation

Statute and policy prescribes the management of DT by the DASD(DT&E), who, for all programs on DT oversight, acts as the final approval authority for DT planning in the TEMP. ODASD(DT&E) staff representatives actively participate in acquisition program T&E WIPTs and provide advice to the T&E WIPT and PM; as well as providing independent assessments to DASD(DT&E) on progress of performance of the test program and overall performance of the system. By statute, the DASD(DT&E) has access to all test data and program information relevant to the execution of testing and fulfillment of the ODASD(DT&E) responsibilities. As a member of the OIPT, the DASD(DT&E) provides advice and recommendations at Defense Acquisition Board (DAB), reviews and submits an independent Assessment for Operational Test Readiness (AOTR) to the Component Acquisition Executive (CAE) and USD(AT&L) for all programs on DT oversight prior to the CAE decision on material readiness for initial operational test and evaluation (IOT&E).

The PM should initiate early engagement with the ODASD(DT&E) and charter a T&E WIPT to aid in development of test strategies and building a TEMP. Given that DT spans the entire lifecycle of an acquisition program and remains a vital part of all levels in the work structure of the systems engineering process, DASD(DT&E) expects due diligence from the PMs to ensure they base program and design decisions on test results conducted and reported as independent verification steps in the process, and not simply pulled from design and test learning processes. This effort requires close and continuous coordination with the SEP, Information Support Plan (ISP), and developing activity engineering and test activities to ensure test plans and reports reflect independent evaluation of the test data from the engineering staff vested in the development activities.

Ideally, the PM bases all development decisions on test events and not schedules or costs; but in the pragmatic environment of developing systems for the Warfighter, time and cost prove significant drivers in pressuring test activities. Therefore, DT activities must provide realistic T&E schedules to PMs during the establishment of the programs integrated management schedule. This effort ensures the effective management of the overall progress and cost of the program; particularly with complex systems that have a number of dependent sub systems and technologies requiring efficient integration as an end product.

As such, the DASD(DT&E):

Develops policies and guidance for the planning, execution, and reporting of DT&E in the DoD, according to section 139b of title 10 USC.
Develops policies and guidance for the integration of DT and OT, in coordination with DOT&E.
Publishes, in conjunction with DOT&E, a combined list of OSD T&E Oversight programs for DT&E, OT&E, and LFT&E.
Monitors and reviews the DT&E activities of MDAPs and other programs.
Periodically conducts AOTRs.
Provides advocacy, oversight, and guidance to the acquisition workforce responsible for test and evaluation.
Reviews and approves TES/TEMPs and selected DT&E plans.
Periodically reviews the Services’ organizational DT&E capabilities to identify needed changes or improvements.

9.2.4.2. Operational Test & Evaluation

By law, DOT&E prescribes policies and procedures for the conduct of OT&E in the Department of Defense. For programs on DOT&E OT oversight, DOT&E serves as the final approval authority for OT&E planning to include approval of the TEMP. DOT&E staff representatives actively participates in acquisition program T&E WIPTs and provide advice to the T&E WIPT and PM; as well as providing independent assessments to the DOT&E on progress of performance of the test program and overall performance of the system. By law, DOT&E has access to all data and records DOT&E considers necessary to review in fulfillment of DOT&E OT&E responsibilities. DOT&E serves as a member of both the Joint Requirements Oversight Council and the OIPT, providing advice and recommendations at DAB reviews; and has direct access to both USD(AT&L) and the SecDef, on all matters relating to operational test and evaluation.

The PM should initiate early engagement with DOT&E through the Service and Defense Agency T&E Executive and independent OTA and charter a T&E WIPT to aid in development of T&E strategies and the TEMP. Since OT&E generally acts as the validation process in SE, early engagement of the OTA and DOT&E, as early as the Analysis of Alternatives and requirements development, ensures a comprehensive assessment of measurability and testability of requirements; and the associated implications to cost and schedule to effectively evaluate the system capabilities and limitations. This requires close and continuous coordination with users, sponsors, developers, and all test activities to ensure understanding and articulation of end-game expectations during program planning and documentation.

Per section 2399 of title 10 USC, an MDAP must complete IOT&E before proceeding beyond full-rate production (FRP). Law also requires DOT&E to provide a Beyond Low-Rate Initial Production (BLRIP) report to the SecDef, USD(AT&L), and congressional defense committees on the adequacy of OT&E conducted; as well as the results of T&E to confirm effectiveness and suitability for combat. Additionally, DoDI 5000.02 charges DOT&E with completing the section 2366 of title 10 USC LFT&E report requirement for submission to the congressional defense committees, SecDef, and USD(AT&L) before the system may proceed to FRP. For purposes of compliance with completion of IOT&E, the PM must ensure the system under test reflect production configured or representative systems, preferably Low Rate Initial Production (LRIP) systems. Title 10 requires DOT&E to determine the number of LRIP systems for all operational testing of programs on DOT&E’s OT&E oversight and the Service OTA to determine LRIP requirements for non-OSD T&E oversight programs. DOT&E and the OTAs routinely engage the PM in those decisions. For programs not on the OSD T&E Oversight List, the Service or Defense Agency OTA will work with the PMs for OT&E, including planning, applicable oversight, execution and reporting. Service or Defense Agency OTAs may delegate the responsibilities to other responsible DoD test agencies.

DOT&E approves all OT&E plans, to include early operational assessments (EOAs), OAs, Limited User Tests (LUTs), IOT&E, and Follow-on Operational Test & Evaluation (FOT&E). DOT&E requires the OTAs to provide plans to assess adequacy of data collection and analysis planning to support the operational evaluation of a systems operational effectiveness and operational suitability, since integrated test concepts aid in generating test efficiencies and reduced development time. OTAs must schedule test concept briefings 180 days prior to an operational test. PMs must provide OT&E plans for DOT&E approval 60 days prior to test events.

In addition to OT&E oversight, the SecDef charges DOT&E with approving waivers to full up system level (FUSL) LFT&E and approval of required alternative LFT&E plans prior to Milestone B.

For programs to effectively track through the complex acquisition process and meet their cost, schedule, and performance goals, it remains essential to engage OSD early, continuously, and to quickly resolve working issues presenting obstacles to any of the T&E stakeholders’ duties. Service T&E Executives must establish clear issue resolution processes to resolve issues in a timely fashion.

As such, the DOT&E:

Prescribes OT&E and LFT&E policies for the DoD according to sections 139, 2366, 2399, and 2400 of title 10; and DoDD 5141.2, “Director of Operational Test and Evaluation (DOT&E),” dated February 2, 2009.
Exercises oversight responsibility for ACAT I or other programs in which the SecDef has special interest. Monitors and reviews OT and LF activities in the DoD.
Participates in integrated test teams and test integrated product teams to foster program success.
Publishes, in conjunction with the DASD(DT&E), a combined list of OSD T&E Oversight programs for DT, OT, and LF.
Approves, in writing, the adequacy of operational test plans for those programs on OSD OT&E Oversight prior to the commencement of operational testing. Approves the operational test portions of integrated test plans. Approves the quantity of test articles required for operational testing of major defense acquisition programs (MDAP).
Approves TEMP and T&E strategies for OSD T&E Oversight programs in conjunction with the DASD(DT&E) and DoD Chief Information Officer (CIO).
Approves LFT&E strategies and waivers prior to commencement of LFT&E activities.
Submits a report to SecDef and Congress before systems on OSD OT&E Oversight may proceed BLRIP.

9.2.5. Major Range and Test Facility Base

The DoD, through the TRMC, oversees sustainment of twenty-four T&E organizations or activities with a skilled workforce and T&E technical capabilities and processes, and available to all components under a common charge policy. In accordance with DoDD 3200.11 “Major Range and Test Facility Base MRTFB,” dated December 27, 2007 and DoDI 3200.18 “Management and Operation of the Major Range Test Facility Base (MRTFB),” dated February 1, 2010, TRMC manages the following activities:

ARMY ACTIVITIES

White Sands Test Center

High Energy Laser Systems Test Facility

U.S. Army Kwajalein Atoll (Ronald Reagan Ballistic Missile Defense Test Site)

Yuma Test Center

Cold Regions Test Center

Tropic Regions Test Center

West Desert Test Center

Aberdeen Test Center

Electronic Proving Ground

NAVY ACTIVITIES

Naval Air Warfare Center-Weapons Division, Point Mugu