Detailed Information on the
Constellation Systems Assessment

Conduct planned internal reviews.

Plan and conduct comprehensive external program review.

Develop and baseline metrics for transition of activities and assets from Space Operations to Constellations Systems.

Measure: Extend Human presence across the solar system.

Explanation:The President's Vision for Space Exploration states that the United States will "Extend human presence across the solar system, starting with a human return to the Moon before the year 2020, in preparation for human exploration of Mars and other destinations." The baseline is that humans have not traveled more than a few hundred miles from Earth since 1972.

Measure: Demonstrate an operational capability to support human exploration missions.

Explanation:The Vision for Space Exploration requires that the U.S. develop a new crew exploration vehicle and "conduct the initial test flight before the end of this decade in order to provide an operational capability to support human exploration missions no later than 2014." The baseline is that there is no current operational capability to support exploration beyond low Earth orbit. NASA cannot credibly promise that the Orion CEV could come on-line operationally any sooner than March 2015 (with a 65% cost confidence level and appx. 4 month variance in the confidence interval), although the Constellation Program is trying to meet earlier schedule milestones.

Measure: Demonstrate one or more commercial space services for International Space Station cargo and/or crew transport.

Explanation:The Vision for Space Exploration encourages entrepreneurial and other commercial launch companies in the pursuit of opportunities to providing transportation and other services supporting the International Space Station. The Commercial Crew and Cargo Program Office (C3PO) signed two Space Act Agreements to provide funding to entrepreneurial launch companies in 2006, and five Space Act Agreements without funding in 2007. The Agreements lay out a mutually negotiated set of technical and financial milestones that would culminate in demonstration flights before 2010. There are planned milestones/deliverables for FY2008 through FY2010 that provide specific performance targets. These targets must be achieved to successfully meet this metric. These deliverables are as follows. In FY2008: complete the Flight Demonstration 1 Readiness Review, complete the Flight Demonstration 2 Preliminary Design Review (PDR), complete the Flight Demonstration 3 System Requirements Review (SRR). At least three funded and unfunded Partners are receiving technical assistance through the COTS Assistance Team (CAT) and making progress toward orbital demonstrations of commercial crew and cargo systems. Specifically, In FY2009, at least one Partner completes at least one orbital demonstration flight. At least three funded and unfunded Partners are receiving technical assistance through the COTS Assistance Team (CAT) and making progress toward orbital demonstrations of commercial crew and cargo systems. In FY2010, at least one Partner completes at least one demonstration flight to the ISS. At least three funded and unfunded Partners are receiving technical assistance through the COTS Assistance Team (CAT) and making progress toward orbital demonstrations of commercial crew and cargo systems. ore information on the COTS program can be found at http://procurement.jsc.nasa.gov/cots/.

Measure: Transition applicable Shuttle components, infrastructure, and workforce to the Constellation program

Explanation:NASA will identify Shuttle capabilities required for new exploration systems and preserve them for future use. Since parts of the new transportation system are Shuttle-derived, NASA will find ways to use existing Shuttle hardware, infrastructure, and workforce in Constellation efficiently and effectively. Specific targets will be identified by September 2006.

Measure: Accomplish key Crew Exploration Vehicle (CEV) and Crew Launch Vehicle (CLV) development and technology activities.

Explanation:FY 2006 - Competitively awarded contract for Phase A & Phase B design & flight demo of the CEV; FY 2007 - Completed CEV and CLV Systems Requirements Review (SRR) project boards; FY 2008 - Complete System Design Review (SDR) for CEV & CLV; complete Pre-Non-Advocate Review (PNAR) for CEV and CLV; Complete Preliminary Design Review (PDR) for CEV & CLV; FY 2009 Complete Critical Design Review (CDR) for CEV; complete the first development flight test Ares I-X; ; FY 2010 - Complete Critical Design Review (CDR) for CLV and perform Orion Abort test

Measure: Cumulative percentage of baseline cost overrun for projects under development.

Explanation:On average, the cumlative estimate to complete and completion cost of all the Constellation projects under development will not exceed 10% of the baseline cost.

Measure: Cumulative percentage of baseline schedule slip for Constellation projects under development.

Explanation:The slip of key milestones of all the Constellation projects under development, on average, will not exceed the baseline schedule, by more than 10% cumulatively.

Measure: Reduction in ground operations cost of the Constellation systems based on original estimates in the Exploration Systems Architecture Study. Value represents the ground operations cost through 2012.

Explanation:This measure's purpose is to drive the Constellation systems' designs for greater operational efficiencies during the early design phases of the program. This efficiency measurement will be used until the program baseline is established in FY 2008. Constellation is currently in the formulation stage of development and so does not have a baseline ground operations cost.

Measure: Cycle time reduction between requirement identification and final disposition for Program level requirements incorporation.

Explanation:This measure identifies a critical factor in the early years of a program to identify the processes that need to be improved in the requirements development/revision cycles. The current design analysis cycles take on the order of 20 weeks to complete leading to the Systems Requirements Review (SRR) in September 2006. Reducing this time after SRR is critical to stabilize the program requirements before establishing a baseline in 2008.

Is the program purpose clear?

Explanation: The Constellation Systems Program supports the President's Vision for Space Exploration by developing systems necessary to enable safe, affordable, human exploration to the Moon, Mars, and beyond.

Evidence: Constellation programs include the Crew Exploration Vehicle and Crew Launch Vehicle which will provide crew transportation for missions beyond Low Earth Orbit. The Commercial Crew Cargo project will provide transportation and other services supporting the International Space Station. These goals are set out by the President's Vision for Space Exploration and the NASA Authorization Act of 2005. Supporting Documentation: President's Vision for Space Exploration: http://www.whitehouse.gov/space/renewed_spirit.pdf; Public Law 109-155, NASA Authorization Act of 2005: http://thomas.loc.gov/cgi-bin/query/D?c109:5:./temp/~c109cIdB1s::

Does the program address a specific and existing problem, interest, or need?

Explanation: The Constellation Program is responsible for developing capabilities essential to making the Vision for Space Exploration a reality. The Crew Exploration Vehicle and Crew Launch Vehicle will provide transportation to the Moon, Mars, and beyond. The ISS Cargo Crew Services project will ensure that NASA can support its needs and commitments for transporting cargo and crew to and from the International Space Station (ISS).

Evidence: The President's Vision for Space Exploration and the NASA Authorization Act of 2005 both direct NASA to conduct a program of exploration. New vehicles are required to execute this program because existing vehicles do not have the capability to support many exploration activities. Alternate U.S. capabilities to service the Space Station will also be required to supplement and then replace the Space Shuttle. Supporting Documentation: President's Vision for Space Exploration (VSE); Public Law 109-155, NASA Authorization Act of 2005; FY 2007 Integrated Budget and Performance Document

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: No other U.S. human spaceflight system exists for exploration beyond Low Earth Orbit. The Commercial Crew/Cargo project is needed to resupply the International Space Station (ISS) because the Space Shuttle will be retired in 2010 and legislation may prevent the United States from purchasing Russian capabilities to support human spaceflight activities after 2012.

Evidence: Supporting Documentation: President's Vision for Space Exploration (VSE); FY 2007 Integrated Budget and Performance Document; AIAA International Reference Guide to Space Launch Systems; Iran Nonproliferation Act of 2000 (as amended)

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The Exploration Systems Architecture Study (ESAS) evaluated alternate approaches to meeting the program's goals and recommended an initial architecture that appeared to be more efficient and effective than other approahces. The Constellation architecture is being further refined in the program concept development phase which culminates with the System Requirements Review (SRR) and Pre-Non-Advocate Review (PNAR). By rigorously following the systems engineering best practices, the program is expected to move towards the best architecture consistent with available budget, schedule, and risk management. Development of the Constellation System Architecture will follow the NASA Systems Engineering Procedural Requirements embodied in NASA Procedural Requirements Document NPR-7123.1. This systems engineering process was developed by incorporating the best systems engineering approaches currently being practiced by the U.S. government and industry. To ensure effective management and program control, the program office team includes an integrated program planning and controls function that includes resource management, performance assessments, and integrated schedules management. This function is responsible for developing the processes, reports, and measures needed to ensure the development of an effective, efficient, and affordable program which is consistent with the Capital Programming Guide for Capital Assets and Service Acquisition Programs.

Evidence: Supporting Documentation: Constellation Program Management Plan (currently in review, available in Spring 2007); Constellation Architecture Requirements Document (CARD) (currently in review, available September 2006); ESAS Report: http://www.nasa.gov/mission_pages/exploration/news/ESAS_report.html; Systems Engineering Procedural Requirements, NPR 7123.2: http://nodis3.gsfc.nasa.gov/displayDir.cfm?Internal_ID=N_PR_7123_0001_&page_name=main

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: The purpose of the Nation's Space Exploration Program and its intended benefits are to advance U.S. scientific, economic, and security interests through a robust exploration program. The Constellation Program supports this policy goal by developing the Crew Exploration Vehicle, Crew Launch vehicle, and other systems necessary to provide for a crew transportation system. These systems are based on the Exploration Systems Architecture Study (ESAS), which analyzed competing architectures and determined the cost and technical objectives necessary for each design. The Commercial Orbital Transportation Services (COTS) project will ensure a domestic solution to resupplying the International Space Station. Such a capability does not now exist. There are no unintended subsidies from this program.

Evidence: ESAS laid out the exploration and scientific objectives by which NASA will address the Nation's Vision for Space Exploration. Subsequently there have been several additional Design Analysis Cycles that have matured the architecture design. The COTS demonstration is underway with a wide variety of organizations submitting proposals for Earth-to-orbit space flight demonstrations for crew and cargo delivery requirements to the ISS. Additionally, NASA Administrator Mike Griffin explained in his April 2006 speech how space exploration serves the nation's essential interests. Supporting Documentation: Public Law 109-155, NASA Authorization Act of 2005; Constellation Architecture Requirements Document (currently in review, available September 2006); SRR Package (currently in review, available September 2006); COTS Awards Announcement (Later in 2006); ESAS final report: http://www.nasa.gov/mission_pages/exploration/news/ESAS_report.html; "Remarks by NASA Administrator Michael Griffin at the National Space Symposium" 6 April 2006: http://www.spaceref.com/news/viewsr.html?pid=20189

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The Constellation program has three long-term performance measures that focus on outcomes and are consistent with the Vision for Space Exploration. The two long-term outcome measures are to extend human presence across the solar system and demonstrate flight capability. These goals are central to the purpose of the Constellation program, which is to support the Vision for Space Exploration by developing systems which enable human space exploration. Transitioning Shuttle components to Constellation is essential in reducing costs and maintaining the workforce.

Evidence: The long-term performance goals and associated outcomes are documented in the measures tab.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: NASA's goals to achieve a lunar mission by 2020 and fly the Crew Exploration Vehicle (CEV) by 2014 are ambitious given that it has been over 30 years since the nation has developed a new human space transportation system. Although the Apollo program was developed more quickly, the CEV will have greater capabilities than Apollo, including higher safety and mission reliability and allowance of more crew to the lunar surface, and will require a far lower percentage of the federal budget than did Apollo.

Evidence: Long-term performance measures are documented in the measures tab. Constellation's high level requirements are documented in the Constellation Architecture Requirements Document, which is currently under review and will be baselined at the Systems Requirements Review in Sept. 2006.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: NASA has established a set of annual outcome and output performance goals for FY07-FY11 for the Constellation Program tied to the Vision for Space Exploration, which are included in the measures tab. The annual measures will track progress towards achieving the long term goals of Crew Exploration Vehicle development and exploration. Constellation has two annual efficiency measures, which are both under development at this time.

Evidence: Annual performance goals for FY07-FY11 documented in the measures tab.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: Of NASA's annual measures, one (regarding Crew Exploration Vehicle (CEV) development) has a baseline and ambitious targets. The others are under development and lack either baselines or ambitious targets at this time.

Evidence: The Constellation Program has targeted completion of the preliminary design for the CEV and Crew Launch Vehicle less than 18 months after the System Requirements are baselined. This is sooner than in past programs of this complexity and size and makes for ambitious but realistic targets for the annual measures. Supporting Documentation: ESAS Report; Constellation Architecture Requirements Document (currently in review, available September 2006); Constellation Systems Engineering Management Plan (currently in review, available September 2006); Constellation Program Management Plan (includes the IMS) (currently in review, available Spring 2007)

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: NASA's senior managers have annual performance goals outlined in their personnel performance plans. NASA's performance based contracting approach requires the contractors to propose milestones and performance metrics against which NASA evaluates their award fee. As part of their proposal, the contractors are required to submit an Integrated Management Plan and an Integrated Master Schedule that outline their milestones and annual performance goals. The Exploration Systems Mission Directorate and Space Operations Mission Directorate (which is responsible for the Space Shuttle and the Space Station) are involved in joint planning and have adopted shared goals as part of a shared mission to meet the President's Vision for Space Exploration. Both Directorates participate in a joint control board to further enhance coordination and are both accountable to agency senior management for successful performance. The Commercial Orbital Transportation Systems program is negotiating demonstration milestones and performance metrics with potential partners. Accomplishing demonstration milestones provide the basis for assessing continued commitment to the long-range program goals. Additionally, partners will culminate their demonstration plan by conducting an orbital cargo flight.

Evidence: The two CEV contractors both provided an Integrated Management Plan and an Integrated Master Schedule against which their phase one performance is being assessed. Additionally, future contracts for Constellation components will render contractual requirements to commit partners to programmatic goals. The Research & Technology programs have performance goals to support the development of technology in support of the Constellation Program. COTS partner demonstration milestones will be identified in the individual Space Act Agreements to be finalized this summer. NASA Integrated Budget and Performance Document: http://www.nasa.gov/pdf/142458main_FY07_budget_full.pdf

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: There are currently no external bodies outside the agency that review Constellation. However, NASA requires all programs and projects to undergo a series of independent programmatic and cost evaluations.

Evidence: Per NPR 7120.5C, the Constellation Program is collaborating with the agency's Program Analysis and Evaluation Directorate to plan the independent reviews of the Constellation Program and Projects. The first of these reviews (PNAR) is scheduled for Spring 2007. Constellation's process and procedures for complying with the NPR 7120.5c requirements will be detailed in the Constellation Program Management Plan which will be baselined at the PNAR. Supporting Documentation: NPR 7120.5C; Constellation Program Management Plan (currently in review, available Spring 2007)

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: Budget and performance planning processes are integrated for the Constellation theme in NASA's Integrated Budget & Performance Document (IBPD). This document presents the annual performance goals and the relevant budget elements for the program. The FY 2007 President's Budget Submit includes all direct and indirect full cost elements from FY 2007 to 2011. As the program matures, the tie between budget requests and performance goals will become better defined.

Evidence: The FY 2007 IBPD is available at http://www.nasa.gov/pdf/142458main_FY07_budget_full.pdf

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The agency has undertaken a series of strategic level initiatives which will result in a significantly improved planning and implementation process for managing the Space Exploration Program. These initiatives have come to fruition following the completion of the Exploration Systems Architecture Study (ESAS) and will be vital to successfully implementing the guidelines established during ESAS. First, NASA is revitalizing its systems engineering policies, similar to work underway in the DoD. Secondly, the advent of measures, such as the new systems engineering requirements, will strengthen the role of NASA engineers, resulting in better management of contractors, and, hence, better control of program expenditures, development of an improved NASA workforce, and increased retention of a trained technical staff. Thirdly, following ESAS, the decision was made to move much of the Constellation program to the NASA Centers of Excellence, increasing the role of the centers in space exploration. As the work is distributed across the NASA centers, it is hoped that a breakdown of organizational stove pipes will result, leading to a more effective program.

Evidence: The structure and responsibilities of the Program Office are documented in the Formulation Authorization Document and the Program Management Plan (PMP). The PMP requires the application of the Systems Engineering NPR (7123.1) and the Program and Project Management Process (NPR 7120.5) as well as the Program Organizational Structure. The results of the ESAS study are documented in the ESAS Report. Supporting Documentation: NPR 7123.1; NPR 7120.5; ESAS Report; System Requirements Review Package (currently in review, available September 2006)

Has the agency/program conducted a recent, meaningful, credible analysis of alternatives that includes trade-offs between cost, schedule, risk, and performance goals, and used the results to guide the resulting activity?

Explanation: The Constellation Program has conducted a series of Analysis of Alternatives studies. These studies were, and continue to be, the framework for the baseline architecture. In 2005, NASA completed an early launch vehicle Analysis of Alternatives that compared the cost, schedule, and mission risk of 7 different approaches for launch vehicle configurations. NASA followed this with the Exploration Systems Architecture Study (ESAS), which performed a suite of in-depth trade studies of possible concepts including launch configurations, propulsion systems, design reference missions, crew size, vehicle shapes, and operations concepts. The ESAS trade studies examined costs, risks, and performance in order to select a baseline architecture. Implementation of the ESAS baseline architecture initiated additional Ground Operations studies to focus on infrastructure trades which minimize lifecycle costs while leveraging existing capabilities.

Evidence: Supporting Documentation: ESAS Report (www.nasa.gov/mission_pages/exploration/news/ESAS_report.html)

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: The Constellation Program is being managed in accordance with NPR 7120.5, NASA Program and Project Management Processes and Requirements. Program progress will be measured by the Earned Value Measurement System (EVMS) as mandated by NPR 7120.5 and applied across all Constellation Projects. Contractors will also be required to have an EVMS and submit monthly EVM data. In addition, program performance information is reviewed through monthly and quarterly progress reports, agency level non-advocate programmatic reviews, and program and project level technical reviews. Constellation will collect a wide range of cost and schedule data in addition to these reviews to assess whether projects are able to meet milestones and deliverables, and make management decisions as appropriate to meet performance goals. Constellation also collects performance data from the Space Operations Mission Directorate as it relates to the human spaceflight transition plan. Commercial Orbital Transportation Services Program partners' performance is verified through accomplishing predefined and negotiated demonstration milestones.

Evidence: Preliminary drafts of the program plan and the Constellation Systems Engineering Management Plan will be available at the preliminary non-advocate review. Final documents will be available at the non-advocate programmatic reviews. Exploration Systems Mission Directorate is implementing an EVMS across the Constellation Program to manage technical cost and schedule data as part of its management control system. Commercial Orbital Transportation Services partner milestones will be identified in the individual Space Act Agreements to be finalized this summer. Supporting Documentation: Constellation Systems Engineering Management Plan (currently in review, available September 2006); Constellation Program Management Plan (currently in review, available Spring 2007); NPR 7120.5C; NPR 7123.1

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: NASA managers are held accountable for cost, schedule, and performance in accordance with the procedures and processes established by NPR 7120.5 and NPR 7123.1. Per 7120.5C, managers responsible for achieving key program results are defined in program products, including the program plans. Contract incentives are planned for Constellation contracts related to cost and schedule performance. Exploration Systems Mission Directorate and Space Operations Mission Directorate are involved in joint planning and have adopted shared goals as part of a shared mission to meet the President's Vision for Space Exploration. Both Directorates participate in a joint control board to further enhance coordination and are both accountable to agency senior management for successful performance. Commercial Orbital Transportation Services partners will be held accountable for cost, schedule, and performance standards by way of negotiated payments based on accomplished milestones. Minimum performance thresholds are controlled by the milestones mentioned above for COTS and are identified in Constellation acquisition products.

Evidence: The draft Constellation Program Management Plan mandates the timely preparation and use of management control documents and reviews. These include the Formulation Authorization Document, program and project internal reviews and non-advocate reviews, and the implementation of an Earned Value Management System (EVMS). EVMS provides the mechanism to enable the program to assess the cost and schedule performance for the contracts. A preliminary version of the Program Management Plan will be available at the Constellation Program PNAR and an updated version will be released for the non-advocate review. Supporting Documentation: Constellation Formulation Authorization Document; Constellation Program Management Plan (currently in review, available Spring 2007); NPR 7120.5C; NPR 7123.1

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: Constellation's annual appropriation is available for obligation for a two year time period. At the beginning of the fiscal year, obligation and cost monthly phasing plans are developed and used as the basis for tracking actual obligations and cost. During the monthly project office reviews, actual obligations and cost are reported against the plans and discussed. Funds not obligated are subject to reprogramming at the Associate Administrator's discretion. Exceptions are granted for problem procurement actions and minor funds cleanup. Unobligated balances are also considered when determining where to make reductions in order to fund Agency or Mission Directorate contingencies.

Evidence: 55.82% of Fiscal Year 2005 funds were obligated by September 30, 2005; and 94.17% by June 14, 2006. Of the 5.83% remaining unobligated, 2.75% is in various corporate accounts, 0.69% is in Center G&A, and 1.74% is in Constellation programmatic procurements. As of June 14, 2006, 17.3% of Fiscal Year 2006 Constellation funds are obligated. Crew Exploration Vehicle (CEV) has obligated 9.3%, and Commercial Orbital Transportation Services (COTS) only a minimal amount. Constellation plans to obligate the majority of its PY 06 funds by the end of FY 2006. The CEV prime contract is planned for award in September, and COTS Space Act Agreements are planned for award in August. Several Crew Launch Vehicle (CLV) contracts are also planned for award this summer. These CEV, COTS, and CLV awards will obligate most of Constellation's PY 06 funding. However, the 2006 funding is available for obligation through the end of FY 2007, so some funds will be kept deliberately unobligated to cover normal operations and contingencies in FY 2007 prior to the receipt of funds from the 2007 appropriation.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The program has two efficiency measures with baselines and targets which are under development at this time. Other than those, the program has established an Integrated Collaborative Environment (ICE) to attain the efficiencies that result from improved communications across the ten NASA Centers and Headquarters. Within ICE, a common set of software tools have been embedded for risk, schedule requirements, and data management. Earned Value Management System (EVMS) is utilized to measure schedule and cost performance. Commonality is applied across the program to achieve cost and schedule savings and risk reduction. For example, the Crew Launch Vehicle will incorporate the Solid Rocket Booster Motors from Space Shuttle and the Crew Exploration Vehicle (CEV) will utilize the Apollo capsule outer mold line in the CEV architecture.

Evidence: The ICE Vision Document for Constellation Systems defines the Program ICE approach. The draft Program Management Plan will require the use of EVMS as the measurement tool to track program cost and schedule performance. A preliminary version of the Program Management Plan will be available at the Constellation Program preliminary non-advocate review (PNAR) and an updated version will be released for the NAR. The Exploration Systems Architecture Study (ESAS) Report and Constellation Architecture Requirements Document incorporate commonality approaches to enhance program cost and schedule effectiveness. Supporting Documents: ICE Vision Document for Constellation Systems; Constellation Program Management Plan (currently in review, available Spring 2007); CARD (currently in review, available September 2006); ESAS Report; Measures Tab

Does the program collaborate and coordinate effectively with related programs?

Explanation: The Constellation Program, by its nature, touches several other programs and government agencies. Programs that interface with Constellation include the International Space Station, Space Shuttle, Air Force Launch Complexes and Vehicle Stages, and the Nuclear Regulatory Commission. Protocols, formal control boards, and working groups have been established to assure that the needs, goals, and objectives of Constellation and companion programs are addressed. The Integration Control Board and Shuttle Transition Board have been chartered to assure a smooth transition from the Space Shuttle to the Crew Exploration Vehicle, with continued support for the International Space Station. This includes securing Shuttle flight system elements and launch facilities that are of value to Constellation. Facility Interfaces with the Air Force will be addressed in the Ground Operations Project Management Plan (preliminary version to be baselined at the preliminary non-advocate review) and the Concept of Operations (to be baselined at the systems requirements review). Plans to incorporate the Air Force's RS 68 liquid rocket engine in the Cargo Launch Vehicle are incorporated in the Constellation SRR documentation package.

Evidence: Supporting Documentation: Constellation Program Management Plan (currently in review, available Spring 2007); Constellation Concept of Operations (currently in review, available September 2006); Ground Operations Project Management Plan (currently in review, available Spring 2007); Constellation SRR Package (currently in review, available September 2006)

Does the program use strong financial management practices?

Explanation: The most recent Independent Auditor report for NASA identified three material weaknesses, all of which are repeats, as well as noncompliance with the Federal Financial Management Improvement Act.

Evidence: NASA's FY 2005 Performance and Accountability Report (available at www.nasa.gov/about/budget/index.html) includes the communication from the NASA Inspector General and the report of the Independent Auditor. In addition, the GAO has published numerous reports identifying shortcominsg in NASA's financial management system as well as its financial management processes (example is GAO-04-754T released on May 19, 2004).

Has the program taken meaningful steps to address its management deficiencies?

Explanation: The need for improved management processes at NASA is documented in the Columbia Accident Investigation Board (CAIB) report among others. Subsequently, the agency has taken several steps to incorporate new management methodologies in its programs. At the agency level, NASA has established a new governance model. The governance model provides a dual path to the NASA Associate Administrator for the technical requirements owners and the program requirements owners. As a result, the technical community now has a mechanism to state its position at the highest levels of NASA and a process has been set in place to adjudicate disputes between the technical and programmatic functions of a program. Other improvements to NASA's management procedures include the upcoming revision of NPR 7120.5 for Program and Project Management and NPR 7123.1 for Systems Engineering (issued March 14, 2006). These processes and procedures result in improved communications, independent technical authorities for technical insight, and reinvigoration of systems engineering processes.

Evidence: The preliminary Program Management Plan will be available at the Constellation Program preliminary non-advocate review and the final Program Management Plan will be released for the non-advocate review. Lessons learned reports have been generated for the Orbital Space Plane and Next Generation Launch Technologies programs in support of the Exploration Systems Mission Directorate. Access to program documents is available to the Constellation team by means of the Constellation Windchill documentation management environment. Supporting Documentation: CAIB Report: http://www.nasa.gov/columbia/home/CAIB_Vol1.html; Constellation Program Management Plan (currently in review, available Spring 2007) ; NPR 7120.5C; NPR 7123.1; NASA Governance Model

Is the program managed by maintaining clearly defined deliverables, capability/performance characteristics, and appropriate, credible cost and schedule goals?

Explanation: NASA regulations (NPR 7120.5C and NPR 7123.1) provide the requirements for program and project management of cost, schedule, and performance goals and objectives. The Constellation Program cost, schedule, and performance requirements will be detailed in the Constellation Program Management Plan and related documents. Configuration and data management processes are used to control changes to the program and project baselines. Budgets and schedules are managed by means of the Planning, Programming, Budgeting, and Execution System which was developed, demonstrated, and successfully employed by the DoD and now adapted by NASA. Constellation uses performance-based contracts, which is consistent with the Capital Programming Guide for Capital Assets and Service Acquisition Programs.

Evidence: The Constellation Program Management Plan describes the organization, projects, budget distribution, schedule of key milestones, programmatic and technical reviews and deliverables throughout the life of the program. The first version (preliminary) will be available at the Constellation Program Preliminary Non-Advocate Review with an updated, final version released for the Non-Advocate Review. The Program Manager is empowered to plan and manage the Program by the Formulation Authorization Document approved by the Associate Administrator for Exploration Systems. Work products are described by the Constellation Work Breakdown Structure and progress will be followed via the EVMS. The process for configuration management will be detailed in the Configuration Management Plan (to be baselined at the SRR in September 2006) and the process for data management will be detailed in the Constellation Data Management Plan (baselined at SRR). Technical Baselines are controlled via the Constellation Margins Management Plan (baselined at SRR). Supporting Documentation NPR 7120.5C; NPR 7123.1; Constellation Program Management Plan (currently in review, available Spring 2007); Constellation Configuration Management Plan (currently in review, available September 2006); Constellation Data Management Plan (currently in review, available September 2006); Constellation Margins Management Plan (currently in review, available September 2006); Constellation Work Breakdown Structure (currently in review, available September 2006)

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: NASA has structured a program to achieve the Constellation program's goals, formed a NASA-wide Space Exploration Organization, developed acquisition strategies to acquire Constellation systems, developed the required program enabling documents in accordance with NASA processes, and started the transition of human and infrastructure assets from Shuttle to Constellation. In addition, the Exploration Systems Architecture Study was completed in November 2005, outlining the initial requirements to implement the Vision. Constellation has contracted with two industry teams to complete Phase 1 studies for the Crew Exploration Vehicle (CEV) and plans to select a prime contractor by Sept. 2006. NASA has also issued a solicitation for Commercial Crew Cargo Services and will sign one or more Space Act agreements by Sept. 2006. Although progress on the Commercial Crew Cargo Services was initially slow, it has met recent schedule goals and good progress has been demonstrated on other areas of the Constellaion program.

Evidence: The Exploration Systems Architecture Study (ESAS) Report contains the architecture for the Constellation Program. This has been updated and documented in the draft Constellation Architecture Requirements Document (CARD). The ESMD and Program Office organizations are contained in the draft Constellation Program Management Plan. A preliminary version of the Program Management Plan (PMP) will be available at the Preliminary Non-Advocate Review (PNAR), in 2007, and an updated version will be released for the NAR, in 2008. The overall Constellation Acquisition Strategy is discussed in the draft PMP. A Formulation Authorization Document (FAD) has been issued which provides a warrant for the Program Manager to proceed with planning his program and projects and setting the required management structure in place. A charter was developed for a joint ESMD-SOMD Directorate Working Group to develop a transition strategy from Space Shuttle to the CEV and senior NASA Transition Managers have been assigned within each project, program, and center to lead, coordinate, and facilitate transition activities. Supporting Documentation: ESAS Report; CARD (currently in review, available September 2006); Constellation FAD; Constellation PMP (currently in review, available Spring 2007); Transition Plan

Does the program (including program partners) achieve its annual performance goals?

Explanation: The Constellation Program is early in its formulation and the program's annual performance goals have been related to getting industry concepts for the Vision along with getting contracts in place for the design of Constellation systems. Constellation is getting ready to award contracts for both the Crew Exploration Vehicle and the Commercial Orbital Transportation System Program. Constellation will apply all annual measures once baselines and targets have been established for those currently under development.

Evidence: Supporting Documentation: Annual performance measures in measures tab; Exploration Systems Architecture Study Report; NASA FY07 Integrated Budget and Performance Document

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The Constellation Program has established several initiatives for improving program efficiency and effectiveness. These include: (1) Chartering a Transition Working Group, co-chaired by the Associate Administrators for Exploration Systems and Space Operations, to oversee the transition of the shuttle workforce, resources, and facilities to Constellation; (2) Leveraging reuse of existing Space Shuttle contracts to expedite the initiation of Constellation work scope and to achieve efficiencies in contract management; and (3) Designing for system commonality and adaptation of existing hardware and infrastructure for the Constellation architecture. The efficiency measures are currently under development, but will be baselined in Sept. 2006.

Evidence: Supporting Documentation: NASA FY2007 Integrated Budget and Performance Document; Shuttle Transition Working Group Charter; Joint Integration Control Board Charter; System Requirements Review Package (currently under review, available in September 2006); Measures tab

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: The Constellation Program is unique in providing a U.S. capability to transport crew and cargo to Low Earth Orbit, the Lunar Surface, and, eventually, to Mars and other destinations. There is no current program with which to compare Constellation.

Evidence:

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: The Constellation Program has not yet undergone an independent evaluation of sufficient scope and quality to determine whether the program is effective and achieving results. The Crew Exploration Vehicle (CEV) and Crew Launch Vehicle (CLV) Projects along with the Constellation Program as a whole are currently being audited by the General Accounting Office and the NASA Inspector General. Internal reviews and the progress of the program indicate that it is achieving results, and major internal reviews are scheduled for 2007.

Evidence: The Constellation program requirements and review plans will be documented in the Constellation Program Management Plan. The first review, the Pre-Non Advocate Review (PNAR), is scheduled for Spring 2007. Results of the PNAR and all subsequent independent reviews will be available in the reports generated following these reviews. The CEV Project was technically evaluated in March 2006 by the NASA engineering and safety center as part of the preparation for selecting and awarding the CEV Phase 2 contract. Supporting Documentation: NPR 7120.5C; Constellation Program Management Plan (currently in review, available Spring 2007); ESAS Report

Were program goals achieved within budgeted costs and established schedules?

Explanation: Constellation does not yet have a full set of annual PART goals with adequate baselines and targets, so measuring progress is difficult. Budget costs and schedule goals have generally been met. Several announcements and solicitations have been issued on schedule. The Commercial Orbit Transportation Services program was slow to start, but has recently been meeting schedule goals and other elements of the Constellation program appear to be on schedule.

Evidence: (1) The Crew Exploration Vehicle Request For Proposals (RFP) was issued as planned; (2) A Broad Agency Announcement for Constellation Concept Refinements was issued in FY 2005 as planned; (3) A Broad Agency Announcement for Constellation Ground Operations was issued in April 2006 to seek industry innovation and improvements to lower development and life cycle costs; (4) A Commercial Orbit Transportation Services (COTS) solicitation was released and proposals were received as planned. COTS schedule goals have been met, and there are currently no threats to planned goals.