Summary
Estimated costs for the Department of Defense's (DOD) major space acquisition programs have increased by about $12.2 billion from initial estimates for fiscal years 2006 through 2011. Cost growth for ongoing Air Force programs above initial estimates accounts for a substantial portion of this 44 percent increase. In light of the role that optimistic estimating is believed to have played in exacerbating space acquisition cost growth, you requested that we examine (1) in what areas space system acquisition cost estimates have been unrealistic and (2) what incentives and pressures have contributed to the quality and usefulness of cost estimates for space system acquisitions.
Costs for DOD space acquisitions over the past several decades have consistently been underestimated--sometimes by billions of dollars. For example, Space Based Infrared System High program costs were originally estimated at $4 billion, but the program is now estimated to cost over $10 billion. Estimated costs for the National Polar-orbiting Operational Satellite System program have grown from almost $6 billion at program start to over $11 billion. For the most part, cost growth has not been caused by poor cost estimating, but rather the tendency to start programs before knowing whether requirements can be achieved within available resources--largely because of pressures to secure funding. At the same time, however, unrealistic program office cost estimates have exacerbated space acquisition problems. Specifically, with budgets originally set at unrealistic amounts, DOD has had to resort to continually shifting funds to and from programs, and such shifts have had costly, reverberating effects. Our analyses of six ongoing space programs found that original cost estimates were particularly unrealistic about the promise of savings from increased contractor program management responsibilities, the constancy and availability of the industrial base, savings that could be accrued from heritage systems, the amount of weight growth that would occur during a program, the availability of mature technology, the stability of funding, the stability of requirements, and the achievability of planned schedules. At times, estimates that were more realistic in these areas were available to the Air Force, but they were not used. Cost-estimating and program officials we spoke with identified a number of factors that have contributed to this condition, in addition to larger pressures to produce low estimates that are more likely to win support for funding. Although the National Security Space Acquisition policy requires that independent cost estimates be prepared by bodies outside the acquisition chain of command, it does not require that they be relied upon to develop program budgets. While the policy requires that cost estimates be updated at major acquisition milestones, significant events, such as changes in the industrial base or funding, have occurred between milestones. Within space system acquisitions, cost-estimating officials believe that their roles and responsibilities are not clear and the cost-estimating function is fragmented. Cost-estimating resources have atrophied over the years because of previous downsizing of the workforce, making resources such as staff and data inadequate and the Air Force more dependent on support contractors for the estimating function.
Recommendations
Our recommendations from this work are listed below with a Contact for more information. Status will change
from "In process" to "Implemented" or "Not implemented" based on our follow up work.
Director:
Team:
Phone:
Cristina T. Chaplain
Government Accountability Office: Acquisition and Sourcing Management
(202) 512-4859
Recommendations for Executive Action
Recommendation: To increase accountability and transparency of decisions in space programs where an independent estimate produced by the Cost Analysis Improvement Group (CAIG) or the Air Force Cost Analysis Agency (AFCAA) is not chosen, the Secretary of Defense should direct the Under Secretary of Defense for Acquisition, Technology and Logistics or the Secretary of the Air Force, as appropriate, to require officials involved in milestone decisions to document and justify the reasons for their choice and the differences between the program cost estimate and the independent cost estimate.
Agency Affected: Department of Defense
Status: Not Implemented
Comments: DOD partially concurred with this recommendation, and stated in its comments that at both the development and the production Milestones for all ACAT I programs U.S.C. Title 10 - Armed Forces requires the Milestone Decision Authority (MDA) [USD(AT&L)] to be informed by an independently developed life cycle cost estimate before making a decision on how to proceed. DOD also stated that the OSD CAIG is charged with developing this estimate and, in practice, present its findings to the Defense Acquisition Board (DAB) when they meet to advise the MDA. According to DOD, the OSD CAIG formally documents its Independent Cost Estimate (ICE) in a report to the MDA. DOD also stated that the complex decision to determine the cost figure used as a basis for funding and to evaluate future program performance must weigh many competing factors that are often qualitative in nature. DOD concluded that as with all other acquisition related decisions, the decision is the MDA's alone, and although thoroughly discussed with the MDA advisers during the DAB meeting and clearly documented in the Acquisition Decision Memorandum, documenting the explicit justification will reduce the MDA's future decision-making flexibility.
Recommendation: To better ensure investment decisions for space programs are knowledge-based, the Secretary of Defense should direct the Under Secretary of Defense for Acquisition, Technology and Logistics or the Secretary of the Air Force, as appropriate, to instill processes and tools necessary to ensure lessons learned are incorporated into future estimates. This could include conducting postmortem reviews of past space program cost estimates (program office and independent cost estimates) to measure cost-estimating effectiveness and to track and record cost-estimating mistakes; developing a centralized cost-estimating database that provides realistic and credible data to cost estimators; establishing protocols by which cost estimators working with the National Reconnaissance Office can share data with the DOD space cost-estimating community while still maintaining appropriate security over classified data; and ensuring estimates are updated as major events occur within a program that could have a material impact on cost, such as budget reductions, integration problems, hardware/software quality problems, and so forth.
Agency Affected: Department of Defense
Status: In process
Comments: In a memo to GAO in April 2007, DOD stated that DOD and the Air Force have taken multiple actions to improve cost estimates for space programs and to combat the decline in the cost estimating approach and management. One major step is the creation of the Aerospace Industries Association Cost Estimating Forum. The Cost Estimating Forum, ongoing since November 2005, is composed of senior government and industry leaders and is tasked with making recommendations to increase the accuracy of space cost estimates through improvements in methodology, policies, processes, data, and tools. The forum members were working on a list of recommendations to be briefed to the corporate Chief Executive Officers and Under Secretary for Acquisition, Technology & Logistics. It is expected that the forum effort will include the agreed-upon corrective actions for this GAO report. The forum was to present its recommendations to Under Secretary for Acquisition, Technology & Logistics in November 2007. DOD will not be implementing the recommendation that cost estimates be updated as major events occur within a program. DOD stated that updating estimates, including independently developed cost estimates, more frequently than only at designated Key Decision Points is clearly helpful to inform budgets and support program resource adjustment decisions, but it is important not to mandate updates as, by their nature, they should be program and program phase dependent.
Recommendation: To optimize analysis and collaboration within the space cost-estimating community, the Secretary of Defense should direct the Under Secretary of Defense for Acquisition, Technology and Logistics or the Secretary of the Air Force, as appropriate, to clearly articulate the roles and responsibilities of the various Air Force cost-estimating organizations, and ensure that space system cost estimators are organized so that the Air Force can gain the most from their knowledge and expertise. In taking these actions for programs for which no independent estimate is developed by the CAIG, consider assigning AFCAA the responsibility for the development of independent cost estimates for space system acquisitions, since it is outside of the acquisition chain of command and therefore likely to be unbiased and not pressured to produce optimistic estimates.
Agency Affected: Department of Defense
Status: Implemented
Comments: On August 8, 2007, the Under Secretary of the Air Force issued a cost estimating guidance memorandum. The memorandum clearly defines the roles and responsibilities of the Air Force Deputy Assistant Secretary for Cost and Economics (which includes the Air Force Cost Analysis Agency) in providing non-advocate cost assessments in support of acquisition and budget decisions. In addition, the memorandum directs the Deputy Assistant Secretary for Cost and Economics to develop and administer cost analysis policy and guidance, standards and templates, training and education, and to maintain an historical cost database. We believe that all of these actions will increase collaboration within the cost-estimating community and allow the Air Force to benefit from the cost estimators' knowledge and expertise.
