Systems Engineering Technical Review Timing

seal

Overview of Techncial Reviews Key Points Lessons Learned Feedback

IBR

Review: Integrated Baseline Review (IBR)
Purpose:Establishes the Performance Measurement Baseline (PMB), validates the scope of work, ensures adequate resources, utilizes correct processes, etc.
Timing: Utilized by Program Managers and their technical staffs throughout the life of programs with contracts requiring Earned Value (EV) Management. The IBR is intended to provide a foundation for understanding of project risks.
Entry Criteria: The Performance Management Baseline should be established by the performing organization (Contractor or Government) and should reflect the entire scope of work documented at the appropriate level of detail.
More Information: Integrated Baseline Review (IBR)

IBR

Review: Integrated Baseline Review (IBR)
Purpose:Establishes the Performance Measurement Baseline (PMB), validates the scope of work, ensures adequate resources, utilizes correct processes, etc.
Timing: Utilized by Program Managers and their technical staffs throughout the life of programs with contracts requiring Earned Value (EV) Management. The IBR is intended to provide a foundation for understanding of project risks.
Entry Criteria: The Performance Management Baseline should be established by the performing organization (Contractor or Government) and should reflect the entire scope of work documented at the appropriate level of detail.
More Information: Integrated Baseline Review (IBR)

TRA

Review: Technology Readiness Assessment (TRA)
Purpose: Regulatory requirement per DoDI 5000.02. TRA is a systematic, metrics-based process to assess maturity of Critical Technology Elements. Technology Readiness Level(TRL)-6 equiv is required at MS B, and a TRL-9 target at MS C.
Timing: Conducted prior to MS B and updated for MS C, per DODI 5000.02. May be specified as CDR entry criterion. A TRA may also be required by the MDA at any other time. Should be aligned with the appropriate technical review for synergy.
Entry Criteria: Pre-MS B MDAPs require both a TRA and successful system-level PDR. Non-MDAP pre-MS B TRA should align with completion of SRR or SFR. MS C requires defined capability needs (signed ICD/CDD/CPD); updated TRA; etc.
More Information: Technology Readiness Assessment (TRA)

IBR

Review: Integrated Baseline Review (IBR)
Purpose:Regulatory requirement per DoDI 5000.02 for cost incentive contracts = $20M. Establishes the Performance Measurement Baseline (PMB), validates the scope of work, ensures adequate resources, utilizes correct processes, etc.
Timing: Utilized by Program Managers and their technical staffs throughout the life of programs with contracts requiring Earned Value Management (EVM). The IBR is intended to provide a foundation for understanding of project risks.
Entry Criteria: The Performance Management Baseline should be established by the performing organization and should reflect the entire scope of work documented at the appropriate level of detail.
More Information: Integrated Baseline Review (IBR)

TRA

Review: Technology Readiness Assessment (TRA)
Purpose: Regulatory requirement per DoDI 5000.02. TRA is a systematic, metrics-based process to assess maturity of Critical Technology Elements. Technology Readiness Level(TRL)-6 equiv is required at MS B, and a TRL-9 target at MS C.
Timing: Conducted prior to MS B and updated for MS C, per DODI 5000.02. May be specified as CDR entry criterion. A TRA may also be required by the MDA at any other time. Should be aligned with the appropriate technical review for synergy.
Entry Criteria: Pre-MS B MDAPs require both a TRA and successful system-level PDR. Non-MDAP pre-MS B TRA should align with completion of SRR or SFR. MS C requires defined capability needs (signed ICD/CDD/CPD); updated TRA; etc.
More Information: Technology Readiness Assessment (TRA)

AOTR

Key: Assessment of Operational Test Readiness (AOTR)
Purpose: An independent Assessment of Operational Test Readiness (AOTR) that considers the risks associated with the system's ability to meet operational suitability and effectiveness goals described in the TEMP.
Timing: Conducted prior to OTRR to ensure system is adequately prepared for OTRR and OT&E.
Entry Criteria: This assessment shall be based on capabilities demonstrated in DT&E and Operational Assessments against criteria described in the TEMP.
More Information: Assessment of Operational Test Readiness (AOTR)

OTRR

Review: Operational Test Readiness Review (OTRR)
Purpose:A multi-disciplined product and process assessment to ensure the system under review can proceed into Operational Test and Evaluation (OT&E) with a high probability it will be deemed effective and suitable.
Timing: Conducted sufficiently in advance of the OT&E and with adequate maturity in the areas to be operationally evaluated.
Entry Criteria: Specific Component directives establish the minimum criteria required for the determination of readiness to commence OT&E and for the conduct of OTRRs.
More Information: Operational Test Readiness Review (OTRR)

ITR

Review: Initial Technical Review (ITR)
Purpose: Supports the technical basis for initial cost estimate and budget submissions for the POM.
Timing: Normally done at the earliest stages of program inception. Should precede submission of POM inputs for the program.
Entry Criteria: A CARD-like document with cost-estimating relationship assumptions documented for each element of estimated cost.
More Information: Initial Technical Review (ITR)

ASR

Review: Alternative System Review (ASR)
Purpose:Reviews the results and assesses preferred system concept and identifies key system elements to be prototyped.
Timing: After completion of Materiel Solution Analysis and prior to MS A and initiation of the Technology Development phase.
Entry Criteria: AoA complete, preferred system concept fully developed, technology development plan complete. Includes initial planning estimates for EMD.
More Information: Alternative System Review (ASR)

Prototype Technical Reviews

Review: Prototype Technical Reviews
Purpose: Technical reviews in support of competitive prototyping and systems engineering technical reviews such as Prototype Functional Review, Prototype Preliminary Design Review or Prototype Critical Design Review, can be beneficial to support the technical approach and program plans. Application of these reviews are at the discretion of the program office and prototype developer and should follow the guidance herein as presented for these reviews, which are discussed only in the context of full system development.
Timing: As needed to support prototype decisions during the Prototyping work efforts of the Technology Development Phase.
Entry Criteria: As appropriate and defined in the prototype development plan and/or SEP.

SRR

Review: System Requirements Review (SRR)
Purpose: Assesses if the system under review can proceed into initial system development.
Timing: Precedes and supports MS B. A second SRR after contract award may be used to assess contractor's readiness to begin development.
Entry Criteria: Preliminary system design specification, technology development activities, and Technology Development Strategy completed/documented.
More Information: System Requirements Review (SRR)

SFR

Review: System Functional Review (SFR)
Purpose: Assesses the system functional baseline and readiness for full functional allocation.
Timing: At completion of system-wide functional analysis and prior to functional allocation to configuration items.
Entry Criteria: Updated system design specification, preliminary system functional baseline complete, and preliminary software functional requirements documented.
More Information: System Functional Review (SFR)

PDR

Review: Preliminary Design Review (PDR)
Purpose: A mandatory review that assesses the system allocated baseline and readiness for full detailed design.
Timing: At completion of functional allocation activities and prior to completion of detailed design.
Entry Criteria: Completed allocated baseline as documented in design specification for each hardware and software configuration item.
More Information: Preliminary Design Review(PDR)

CDR

Review: Critical Design Review (CDR)
Purpose:A mandatory review that assesses the system product baseline and supports the program's full commitment to fabrication.
Timing: At completion of detailed design and prior to full fabrication of hardware and final coding of software modules.
Entry Criteria: Completed initial product baseline (hardware and software) and updates to other baselines, as required.
More Information: Critical Design Review (CDR)

TRR

Review: Test Readiness Review (TRR)
Purpose: Assesses the readiness to begin developmental test and technical evaluation. Sub-system TRRs may be conducted prior to sub-system testing.
Timing: At completion of system fabrication and prior to initiation of developmental test.
Entry Criteria: System is under configuration control; all test plans have been approved; and test assets/resources have been identified and are available.
More Information: Test Readiness Review (TRR)

SFR

Review: System Verification Review (SVR)
Purpose: An audit trail from the current functional baseline that assesses system functionality and determines if it meets functional requirements.
Timing: Often conducted concurrently with PRR and/or FCA, prior to any production decision.
Entry Criteria: Evidence indicates the EMD product can satisfy the CDD/draft CPD.
More Information: System Verification Review (SVR)
Review: Functional Configuration Audit (FCA)
Purpose: Review of test/analysis data of each CI (hardware and software) to validate intended function or specification performance.
Timing: Often conducted concurrently with SVR and/or PRR, prior to any production decision.
Entry Criteria: After EMD test/analysis data available, and evidence EMD product is sufficiently mature for entrance into LRIP.
More Information: Functional Configuration Audit (FCA)
Review: Production Readiness Review (PRR)
Purpose: Assesses system's design readiness to begin production, at an acceptable level of risk.
Timing: At completion of Developmental Testing (DT) and following demonstrated acceptable performance in Operational Assessment (supporting AOTR), and prior to MS C Decision.
Entry Criteria: Manufacturing processes, quality management system, production plans and transition to production risk assessment complete.
More Information: Production Readiness Review (PRR)

PCA

Review: Physical Configuration Audit (PCA)
Purpose: Assesses the as-delivered system's compliance with the product baseline and supports the full-rate production decision.
Timing: After low-rate initial production (LRIP), and prior to full-rate production.
Entry Criteria: Technical data package complete, quality control results available, manufacturing and quality control plans complete.
More Information: Physical Configuration Audit (PCA)

ISR

Review: In-Service Review (ISR)
Purpose: Assesses the in-service technical health of a fielded system from a risk, readiness, and resources perspective.
Timing: Performed annually following Initial Operational Capability (IOC), in advance and in support of the operations and support/maintenance budget requirements definition process.
Entry Criteria: Resolution of deficiency reports, hazard risk assessments for all known system hazards, readiness metrics and trends, and funding status.
More Information: In-Service Review (ISR)
Engineering Technical Reviews are an integral part of the systems engineering process and life cycle management of acquisition programs. Technical reviews are a primary method for assessing the technical health of a program at key points in its life cycle, and provide the program manager with an assessment to the readiness to enter the next technical phase of effort. Engineering rigor, interdisciplinary communications, and technical discipline insight are applied to the maturing design in the assessment of requirements traceability, product metrics, and decision rationale. Technical reviews bring to bear additional non-advocate subject matter expertise. The overarching objective of these reviews is a well-managed technical effort leading to successful developmental and operational testing, and the fielding of an effective and suitable system for the warfighter.
Key Points:
Timing - Technical reviews of program progress shall be event-driven (vice schedule driven) and conducted when the system under development meets review entrance criteria as documented in the approved Systems Engineering Plan (SEP).
Chairing - The Chairperson should be independent of the program and a "non-advocate." The Chairperson should be collaboratively assigned through agreement between program management (PEO or PM) and the engineering technical authority.
Participation - Proper attendance / participation of all program stakeholders is critical. Technical reviews shall include participation by subject matter experts (SMEs) who are independent of the program unless specifically waived in the approved SEP.
Integrated Assessment - A major value of the technical review process is an integrated assessment across subject areas by multiple technical disciplines. If partitioning of the review is necessary, it should be partitioned by product (e.g., conduct sub-system reviews) vice having separate reviews by technical disciplines.
Technical Review Checklists - These checklists should be used as the primary guide for technical risk assessment during the conduct of technical reviews. They are available at https://acc.dau.mil/TechRevChklst. The checklist can be both an input to and an output of the review.
Review Products - A primary product of each technical review is a technical assessment for the Program Manager on the readiness to proceed to the next phase of work. Specific products of the review should include: Technical Review report (including minutes, recommendations, and attendee list), Requests For Action (RFAs), and an updated risk assessment.
Lessons Learned:
  • Technical reviews shall be conducted when the SEP-approved entry criteria are achieved (event-driven review). Review timing based only on schedule (schedule-based), and not meeting the entry criteria, will not be able to cover the needed material, will be unable to assess technical risk against the proper criteria, and will waste valuable resources (time/money).
  • Ensure that all stakeholders are included in the review. Service field users and program sponsors are important participants. Failure to include key stakeholders can result in divergent program expectations.
  • Ensure that appropriate, non-advocate SMEs are included as review participants. For example, flight clearance, software, cost analysis, and other SMEs for high-risk areas are essential for assessing technical risk at each review, and should be included as participants. The program engineers, logisticians, and technical review chairperson should ensure that all technical areas and risks are covered.
  • Make program technical data for the review available to the participants well in advance of the review.
  • Sub-system technical reviews ideally should be conducted prior to system level reviews.
  • Technical reviews are not the place for problem solving, but to verify that problem solving has occurred.
See Acquipedia Article: We welcome your feedback on this page. Please email
se-tools@dau.mil with relevant comments or questions.
See ACQuipedia Article, Milestone (MS) A.
See ACQuipedia Article, Milestone (MS) B.
See ACQuipedia Article, Milestone (MS) C.
See ACQuipedia Article, Materiel Solution Analysis.
See ACQuipedia Article, Materiel Development Decision.
See ACQuipedia Article, Materiel Development Decision.
See ACQuipedia Article, Technology Development.
See ACQuipedia Article, Operations & Sustainment.
See ACQuipedia Article, Disposal.
ACQuipedia: Milestone A ACQuipedia: Milestone B ACQuipedia: Milestone C
Phases ACQuipedia: Materiel Solution Analysis ACQuipedia: Technology Development ACQuipedia: Enginerring and Manufacturing Development ACQuipedia: Production and Deployment ACQuipedia: Operations and Support
ACQuipedia: Prototyping System Design ACQuipedia: FRP Decision Review ACQuipedia: Sustainment
ACQuipedia: Materiel Development Decision ACQuipedia: Disposal
Integrated Baseline Review (IBR) Integrated Baseline Review (IBR) Technology Readiness Assessment (TRA) Integrated Baseline Review (IBR) Technology Readiness Assessment (TRA) Assessment of Operationa Test Readiness (AOTR) Operational Test Readiness Review (OTRR)
Initial Technical Review (ITR) Alternative System Review (ASR) System Requirements Review (SRR) System Functional Review (SFR) Preliminary Design Review (PDR) Critical Design Review (CDR) Test Readiness Review (TRR) System Verification Review (SVR) - Functional Configuration Audit (FCA) - Production Readiness Review (PRR) Physical Configuration Audit (PCA) In-Service Review (ISR)
technical Review
space
Technology Readiness Assessment Technology Readiness Assessment
Technical Reviews Technical Reviews and Audits
Program Reviews Program Reviews
Decision Point Decision Point
Milestone Review Milestone Review
AOTR -  Assessment of Operational Test Readiness PCA  -  Physical Configuration Audit
ASR -  Alternative System Review PDR  -  Preliminary Design Review
CDR - Critical Design Review PRR  -  Production Readiness Review
FCA -  Functional Configuration Audit SFR  -  System Functional Review
FRP -  Full Rate Production SRR  -  System Requirements Review
IBR -  Integrated Baseline Review SVR  -  System Verification Review
ISR -  In-Service Review TRR  -  Test Readiness Review
ITR -  Initial Technical Review  
OTRR -  Operational Test Readiness Review  
Sliding version (requires Flash Player)     SE Community of Practice         Risk Report Matrix
"SYSTEM ENGINEERING FOR MISSION SUCCESS"