Test and Evaluation Process Guidelines

Test and Evaluation Process Guidelines

Revised 4/2009

COVER PAGE

LIST OF FIGURES

LIST OF TABLES

BACKGROUND

1 : PURPOSE

2 : TEST AND EVALUATION GUIDING PRINCIPLES

3 : TEST AND EVALUATION DURING THE ACQUISITION MANAGEMENT PHASES - OVERVIEW
3.1 : INVESTMENT ANALYSIS TEST AND EVALUATION
3.1.1 : Initial Investment Analysis
3.1.2 : Final Investment Analysis
3.2 : SI AND ISM TEST AND EVALUATION
3.2.1 : Needs and Requirements
3.2.1.1 : SI Needs and Requirements
3.2.1.2 : ISM Needs and Requirements (TEGS Phase 1:
3.2.2 : Design and Development
3.2.2.1 : SI Design and Development
3.2.2.2 : ISM Design and Development (TEGS Phase 2):
3.2.3 : Development Test
3.2.3.1 : SI Development Test
3.2.3.2 : ISM Development Test (TEGS Phase 3:
3.2.4 : SI Operational Test and ISM System Test
3.2.4.1 : SI Operational Test
3.2.4.2 : ISM System Test (TEGS Phase 4:
3.2.5 : Site Tests
3.2.5.1 : SI Site Tests
3.2.5.2 : ISM Key Site Test (TEGS Phase 5):
3.2.6 : Independent Operational Test and Evaluation
3.2.7 : Field Acceptance and Field Familiarization Tests
3.2.7.1 : SI Field Familiarization Tests
3.2.7.2 : ISM Field Acceptance Test and Field Familiarization
3.2.8 : SI to ISM Transition
3.3 : TEST AND EVALUATION FOR PRE-PLANNED PRODUCT IMPROVEMENTS

4 : RESEARCH AND DEVELOPMENT PROTOTYPE TEST ACTIVITIES
4.1 : KEY TEST AND EVALUATION ORGANIZATIONS

5 : KEY TEST AND EVALUATION ORGANIZATIONS
5.1 : SERVICE ORGANIZATION
5.2 : NEXTGEN AND OPERATIONS PLANNING (ATO-P)
5.3 : OFFICE OF INDEPENDENT OPERATIONAL TEST AND EVALUATION
5.4 : SITE TEST ORGANIZATION
5.5 : SECOND-LEVEL SUPPORT
5.6 : TECHNICAL OPERATIONS

6 : TEST AND EVALUATION DOCUMENTATION
6.1 : EXHIBIT 300 PROGRAM BASELINE
6.1.1 : Attachment 1: Program Requirements
6.1.2 : Implementation Strategy and Planning
6.2 : SI TEST DOCUMENTATION
6.2.1 : Development Test Documentation
6.2.2 : Operational Test Documentation
6.2.3 : Field Familiarization Documentation
6.2.4 : Site Acceptance Test Plan
6.2.5 : Field Familiarization Test Plan
6.2.6 : Test Procedures
6.2.7 : Quick-Look Test Reports
6.2.8 : Test Reports
6.3 : ISM DOCUMENTATION
6.3.1 : Description of Modification
6.3.2 : System/Project Development Plan
6.3.3 : Test Tool and Test Environment Specification
6.3.4 : Development Test Activities Planned and Documented
6.3.5 : Failure/Discrepancy Reports
6.3.6 : Development Test Report
6.3.7 : System/Project Test Plan
6.3.8 : Baseline Established/Development Documentation
6.3.9 : Develop Key Site Test Plan
6.3.10 : Baseline Performance/Functional Verification
6.3.11 : System/Project Release Documents
6.3.12 : System/Project Test Report
6.3.13 : Key Site Package (if applicable)
6.3.14 : Finalize Field Familiarization Plan and Generic Upgrade Schedule
6.3.15 : Key Site Test Report
6.3.16 : Discrepancies/Issues/Actions Documented

7 : RECOMMENDED TEST AND EVALUATION PROCESSES
7.1 : SI SYSTEM TEST PLANNING PROCESS
7.2 : SI DEVELOPMENT TEST
7.3 : OPERATIONAL TEST

APPENDIX A - BEST PRACTICES

APPENDIX B - SI LESSONS LEARNED

APPENDIX C - FORMAT AND CONTENT GUIDANCE FOR T&E DOCUMENTS
C-1: CONTENT AND FORMAT OF FAA OPERATIONAL TEST (OT) PLANS
C-2: CONTENT AND FORMAT OF OPERATIONAL TEST (OT) PROCEDURES
C-3: CONTENT AND FORMAT OF OPERATIONAL TEST (OT) REPORTS
C-4: CONTENT AND FORMAT OF QUICK LOOK REPORTS
C-5: CONTENT AND FORMAT OF DEVELOPMENT TEST (DT) AND PRODUCTION ACCEPTANCE TEST (PAT) PLANS
C-6: CONTENT AND FORMAT OF DEVELOPMENT TEST (DT) AND PERFORMANCE ACCEPTANCE TEST (PAT) PROCEDURES
C-7: CONTENT AND FORMAT OF DEVELOPMENT TEST (DT) AND PERFORMANCE ACCEPTANCE TEST (PAT) REPORTS
C-8: CONTENT AND FORMAT OF FIELD FAMILIARIZATION TEST PLANS
C-9: FORMAT REQUIREMENTS FOR FIELD FAMILIARIZATION TEST REPORTS
C-10: CONTENT AND FORMAT OF SYSTEM PERFORMANCE ANALYSIS REPORTS
C-11: SAMPLE IOTRD FORMAT
C-12: CONTENT AND FORMAT OF CONTRACTOR MASTER TEST PLAN (CMTP)
C-13: CONTENT AND FORMAT OF DEVELOPMENT TEST PLAN (ISM)
C-14: CONTENT AND FORMAT OF FAILURE REPORTS FOR ISM
C-15: CONTENT AND FORMAT OF DEVELOPMENT TEST REPORT FOR ISM
C-16: CONTENT AND FORMAT OF SYSTEM TEST PLAN FOR ISM
C-17: CONTENT AND FORMAT OF SYSTEM TEST REPORT FOR ISM
C-18: CONTENT AND FORMAT OF KEY SITE TEST PLAN FOR ISM
C-19: CONTENT AND FORMAT OF KEY SITE TEST REPORT FOR ISM

APPENDIX D - IOT&E DETAILS
D.1 IOT&E Documentation
D2 IOT&E Team
D.3 Relationship with Service Organizations
D.4 IOT&E Designation Process
D.5 IOT&E Method of System Assessment

APPENDIX E - ACRONYMS AND DEFINITIONS
E1.0 Overview
E1.1 List of Acronyms
E1.3 Definitions