Summary
The National Aeronautics and Space Administration (NASA) has struggled to implement a fully integrated financial management system. The lack of such a system has affected the agency's ability to control program costs, raising concerns about the management of its most costly programs, including the space shuttle program and the International Space Station. In April 2000 NASA initiated the Integrated Financial Management Program (IFMP)--its third effort to improve the agencywide management of its resources. Implementation is expected by fiscal year 2006 with an estimated life-cycle cost of nearly $1 billion. This report (1) assesses NASA's methodology for preparing the current life-cycle cost estimate for implementing IFMP, (2) determines whether NASA's current schedule is reasonable, and (3) evaluates NASA's processes for ensuring adequate cost contingencies.
The uncertain reliability of cost estimates, optimistic schedules, and insufficient processes for ensuring adequate funding reserves have put NASA's latest financial management modernization effort at risk. Over the past several years, IFMP's life-cycle cost estimates have fluctuated, and NASA's current estimate is 14 percent greater than the previous estimate. The reliability of these estimates is uncertain because disciplined costestimating processes required by NASA and recognized as best practices were not used in preparing them. For example, IFMP's current life-cycle cost estimate did not include the full cost likely to be incurred during the life of the program, including certain operations costs and costs to retire the system. In addition, NASA did not consistently use breakdowns of work in preparing the cost estimate, as recommended by NASA guidance. In cases where work breakdowns were used, the agency did not always show the connection between the work breakdown estimates and the official program cost estimate. This has been a weakness since the inception. Although more than half of the IFMP modules have been implemented--including the Core Financial module, which is considered the backbone of IFMP--the system may not be fully implemented by the end of fiscal year 2006 as planned. Efforts to complete the integrated system as quickly as possible might have resulted in schedule margins that are insufficient to manage program challenges--such as personnel shortages, uncertainties about software availability, and Office of Management and Budget (OMB) initiatives to implement electronic systems for agency business processes governmentwide. These OMB initiatives have put IFMP in a reactive mode and are already affecting planning for the payroll, procurement, and travel components of the integrated system, which could result in additional schedule delays and cost growth. Finally, reserve funding for IFMP contingencies may be insufficient, which is particularly problematic, given the program's unreliable cost estimates and optimistic schedule. One module--Budget Formulation--is already experiencing potential shortfalls in its reserves, and project officials expressed concerns that the module's functionality may have to be reduced. Yet the program continues to establish funding reserves based on reserve levels set by other high-risk NASA programs, such as NASA's space flight program--not on analyses of the potential cost impact of risks and unknowns specific to IFMP, as required by NASA guidance. Moreover, the program did not quantify the cost impact of high-criticality risks--also required by NASA--or link its risks to funding reserves to help IFMP develop realistic budget estimates.
Recommendations
Our recommendations from this work are listed below with a Contact for more information. Status will change
from "In process" to "Implemented" or "Not implemented" based on our follow up work.
Director:
Team:
Phone:
Allen Li
Government Accountability Office: Acquisition and Sourcing Management
(202) 512-3600
Recommendations for Executive Action
Recommendation: To ensure that IFMP's life-cycle cost estimate conforms to NASA guidance and best practices, the NASA Administrator should direct IFMP to prepare cost estimates by the current Work Breakdown Structure for the remaining modules.
Agency Affected: National Aeronautics and Space Administration
Status: Implemented
Comments: In February 2006, NASA submitted an updated life-cycle cost estimate along with supporting documentation to GAO for review. Upon reviewing the information, we found that NASA had prepared Work Breakdown Structure (WBS) estimates using the current WBS for the remaining modules. Using the WBS as a structured approach to prepare the cost estimate will help ensure that all costs are accounted for and should help improve the reliability of the estimate and help control program costs.
Recommendation: To ensure that IFMP's life-cycle cost estimate conforms to NASA guidance and best practices, the NASA Administrator should direct IFMP to provide a clear audit trail between detailed WBS estimates and the program's cost estimate for the remaining modules.
Agency Affected: National Aeronautics and Space Administration
Status: Implemented
Comments: In February 2006, NASA submitted an updated life-cycle cost estimate along with supporting documentation to GAO for review. Upon reviewing the information, we found that NASA had established a clear audit trail from the detailed Work Breakdown Structure (WBS) estimates to the life-cycle cost estimate. NASA's action should help improve the reliability of the cost estimate by facilitating a comparison between detailed WBS estimates and the official program cost estimate and help control program costs.
Recommendation: To ensure that IFMP's life-cycle cost estimate conforms to NASA guidance and best practices, the NASA Administrator should direct IFMP to prepare a full life-cycle cost estimate for the entire IFMP that meets NASA's life-cycle cost and full cost guidance.
Agency Affected: National Aeronautics and Space Administration
Status: Implemented
Comments: In February 2006, NASA submitted an updated life-cycle cost estimate along with supporting documentation to GAO for review. Upon reviewing the information, we found that NASA had prepared a full-cost life-cycle cost estimate, in accordance with NASA's life-cycle cost and full-cost guidance. NASA's action should help decision makers better assess all the costs associated with operating and implementing the program and help control program costs.
Recommendation: To ensure that contingencies are funded in accordance with NASA guidance and best practices, the NASA Administrator should direct IFMP to utilize a systematic, logical, and comprehensive tool, such as Probabilistic Risk Assessment, in establishing the level of financial reserves for the remaining module projects and tailor the analysis to risks specific to IFMP.
Agency Affected: National Aeronautics and Space Administration
Status: Implemented
Comments: During its NASA IFMP Follow-up assignment (code 192159), GAO found that NASA had made progress toward implementing this recommendation. NASA had established a comprehensive risk evaluation methodology which is used to facilitate the estimation and allocation of financial reserves. A key part of the methodology employs a probabilistic risk assessment (PRA) tool--CrystalBall--for setting IFMP-specific risk-based reserves. Although the new methodology and PRA tool had been used to set reserve levels for the program office, the PRA tool had not been used in setting reserves for the remaining module projects. According to IFMP officials, the tool would be used in setting reserves for all projects during the fiscal year 2007 budget cycle. Our recent analyses of NASA IFMP "project reserves templates" and "reserves by fiscal year templates" show that NASA has used the PRA tool in setting risk reserves for the individual module projects for its fiscal year 2007 budget cycle.
Recommendation: To ensure that contingencies are funded in accordance with NASA guidance and best practices, the NASA Administrator should direct IFMP to quantify the cost impact of at least all risks with a high likelihood of occurrence and a high magnitude of impact to facilitate the continuing analysis necessary to maintain adequate reserve levels.
Agency Affected: National Aeronautics and Space Administration
Status: Implemented
Comments: During its NASA IFMP Follow-up assignment (code 192159), GAO found that NASA had made progress in implementing this recommendation. Specifically, NASA now requires that the cost impact of high severity risks be analyzed more consistently through the use of standardized risk reserves templates and quantified through the use of a more rigorous methodology and probabilistic risk assessment (PRA) tool. However, NASA had only used the PRA tool in calculating reserves for the program office, and not for the individual module projects. IFMP officials stated that NASA planned to use the PRA tool along with the new risk methodology in establishing reserve levels for the individual project modules, as well, during the fiscal year 2007 budget cycle. Our recent analyses of NASA IFMP "project reserves templates" and "reserves by fiscal year templates" show that NASA has used the PRA tool in establishing reserve levels for the individual project modules for its fiscal year 2007 budget cycle.
Recommendation: To ensure that contingencies are funded in accordance with NASA guidance and best practices, the NASA Administrator should direct IFMP to establish a clear link between the program's risk database and financial reserves.
Agency Affected: National Aeronautics and Space Administration
Status: Implemented
Comments: During its NASA IFMP Follow-up assignment (code 192159), GAO found that NASA has successfully established the linkage between the program's risk database and financial reserves. Specifically, NASA's new risk/reserves methodology ensures that traceability is maintained through the use of the risk reserve templates that assist the program and projects in determining the high severity risks and the direct relationship between risks and reserves. GAO also observed this relationship in the IFMP office's reserve template, which included the estimated costs of high severity risks and then NASA's reflection of the estimated costs in the program office's budget submission for the fiscal year 2006 budget cycle.
