Summary
Each year, agencies submit to the Office of Management and Budget (OMB) a Capital Asset Plan and Business Case--the exhibit 300--to justify each request for a major information technology (IT) investment. The exhibit's content should reflect controls that agencies have established to ensure good project management, as well as showing that they have defined cost, schedule, and performance goals. It is thus a tool to help OMB and agencies identify and correct poorly planned or performing investments. In its budget and oversight role, OMB relies on the accuracy and completeness of this information. GAO was asked to determine the extent to which selected agencies have underlying support for the information in their fiscal year 2006 exhibit 300s. From five major departments having over $1 billion in IT expenditures in that year, GAO chose for analysis 29 exhibits for projects that supported a cross section of federal activities.
Underlying support was often inadequate for information provided in the exhibit 300s reviewed. Three general types of weaknesses were evident. All exhibit 300s had documentation weaknesses. Documentation either did not exist or did not fully agree with specific areas of the exhibit 300. For example, both these problems occurred in relation to calculations of financial benefits for most investments. In addition, for 23 of the 29 investments, information on performance goals and measures was not supported by explanations of how agencies had initially measured their baseline levels of performance (from which they determine progress) or how they determined the actual progress reported in the exhibit 300. Agencies did not always demonstrate that they complied with federal or departmental requirements or policies with regard to management and reporting processes. For example, 21 investments were required to use a specific management system as the basis for the cost, schedule, and performance information in the exhibit 300, but only 6 did so following OMB-required standards. Also, none had cost analyses that fully complied with OMB requirements for cost-benefit and cost-effectiveness analyses. In contrast, most investments did demonstrate compliance with information security planning and training requirements. In sections that required actual cost data, these data were unreliable because they were not derived from cost-accounting systems with adequate controls. In the absence of such systems, agencies generally derived cost information from ad hoc processes. Officials from the five agencies (the Departments of Agriculture, Commerce, Energy, Transportation, and the Treasury) attributed these shortcomings in support to lack of understanding of a requirement or how to respond to it. Agency officials mentioned in particular insufficient guidance or training, as well as lack of familiarity with particular requirements. The weaknesses in the 29 exhibit 300s raise questions regarding the sufficiency of the business cases for these major investments and the quality of the projects' management. Without adequate support in key areas, OMB and agency executives may be depending on unreliable information to make critical decisions on IT projects, thus putting at risk millions of dollars. Further, although the 29 examples cannot be directly projected to the over one thousand business cases developed each year across the federal government, the results suggest that the underlying causes for the weaknesses identified need attention. These weaknesses and their causes are also consistent with problems in project and investment management that are pervasive governmentwide, including at such agencies as the Departments of Defense, Health and Human Services, and Homeland Security, as documented in reports by GAO and others.
Recommendations
Our recommendations from this work are listed below with a Contact for more information. Status will change
from "In process" to "Implemented" or "Not implemented" based on our follow up work.
Director:
Team:
Phone:
David A. Powner
Government Accountability Office: Information Technology
No phone on record
Recommendations for Executive Action
Recommendation: Because decision makers should be aware of any weaknesses in the processes used to develop the information in the exhibit 300s, the Director of OMB should direct agencies to determine the extent to which the information contained in each exhibit 300 is accurate and reliable. Where weaknesses in accuracy and reliability are identified, the agency should be required to disclose them and explain the agency's approach to mitigating them.
Agency Affected: Executive Office of the President: Office of Management and Budget
Status: In process
Comments: Since our report, the Exhibit 300 guidance that has been issued by OMB does not fully require agencies to disclose weaknesses in the reported information. For example, to determine the accuracy of cost and schedule performance information, OMB modified the guidance to require agencies to identify whether earned value management data reported in the exhibit 300 was derived from a system compliant with the industry standard. However, other areas of the exhibit 300, such as the analysis of alternatives, does not require agencies to disclose weaknesses in the reported information.
Recommendation: To help ensure that agency personnel completing exhibit 300s better understand their responsibilities, the Director of OMB should, in advance of OMB's next issuance of the Circular A-11 update, develop and promulgate clearer and more explicit guidance for sections of the exhibit 300 business case that cause confusion, including addressing weaknesses identified in this report (as indicated below) and consulting with agency personnel having responsibility for completing exhibit 300s across government to identify other areas of confusion. At a minimum, the guidance should do the following: (1) provide a more detailed description of the requirements for completing an operational analysis, as referred to in the supplement to Part 7 of Circular A-11, the Capital Programming Guide; and (2) address or clarify possible flexibilities and alternative approaches available to agencies in completing their exhibit 300s: for example, whether the analysis of alternatives section of the exhibit 300 needs to be updated every year for steady state investments and whether all risk areas are relevant for all investments.
Agency Affected: Executive Office of the President: Office of Management and Budget
Status: Implemented
Comments: In January 2006, we reported that underlying support was often inadequate for information provided in the business cases (exhibit 300s) reviewed. As a result, we recommended that the Director of the Office of Management and Budget (OMB) develop more explicit guidance for sections of the exhibit 300 that cause confusion, including addressing weaknesses we identified. Consistent with our recommendation, in June 2006, OMB modified exhibit 300 requirements and provided more explicit guidance for key sections. On key areas that we addressed as having weaknesses, OMB removed the risk inventory requirement, clarified guidance on earned value management reporting, and added guidance on operational analysis, among other things. Establishing a new exhibit 300 with clearer guidance will improve the quality of the information reported in the business case.
Recommendation: To help ensure that agency personnel completing exhibit 300s better understand their responsibilities, the Director of OMB should provide for training of agency personnel responsible for completing exhibit 300s. This training should go beyond a description of changes from prior year's guidance and include working through examples for a variety of investments. In developing the training, OMB should consult with agencies to identify deficiencies that the training should address.
Agency Affected: Executive Office of the President: Office of Management and Budget
Status: Implemented
Comments: In January 2006, we reported that underlying support was often inadequate for information in agency IT budget request business cases (exhibit 300s) provided to the Office of Management and Budget (OMB). As a result, we recommended that OMB develop more explicit guidance for sections of the exhibit 300 that cause confusion, including addressing the weaknesses we identified. Consistent with our recommendation, in June 2006, OMB modified exhibit 300 requirements and provided more explicit guidance for key sections. On key areas that we addressed as having weaknesses, OMB clarified guidance on earned value management reporting, and added guidance on operational analysis, among other things. Establishing a new exhibit 300 with clearer guidance should help to improve the quality of the information reported in agency business cases.
Recommendation: In implementing these recommendations, OMB should work with the Chief Information Officers Council to develop the necessary guidance and implement an effective training program to ensure governmentwide acceptance of these changes.
Agency Affected: Executive Office of the President: Office of Management and Budget
Status: In process
Comments: OMB provides training for agency personnel responsible for completing exhibit 300s. We have work planned to evaluate the adequacy of this training to determine whether it goes beyond a description of changes from prior years' guidance and includes case studies from a variety of investments.
