This is the accessible text file for GAO report number GAO-06-257T entitled 'DOD Acquisition Outcomes: A Case for Change' which was released on November 15, 2005. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Testimony: Before the Subcommittee on AirLand, Committee on Armed Services, U.S. Senate: United States Government Accountability Office: GAO: For Release on Delivery Expected at 2:30 p.m. EST: Tuesday, November 15, 2005: DOD Acquisition Outcomes: A Case for Change: Statement of Katherine V. Schinasi, Managing Director: Acquisition and Sourcing Management: GAO-06-257T: GAO Highlights: Highlights of GAO-06-257T, a testimony before the Subcommittee on Airland, Committee on Armed Services, U.S. Senate: Why GAO Did This Study: The Department of Defense (DOD) is shepherding a portfolio of major weapon systems valued at about $1.3 trillion. How DOD is managing this investment has been a matter of concern for some time. Since 1990, GAO has designated DOD’s weapon system acquisitions as a high-risk area for fraud, waste, abuse, and mismanagement. DOD has experienced cost overruns, missed deadlines, performance shortfalls, and persistent management problems. In light of the serious budget pressures facing the nation, such problems are especially troubling. GAO has issued hundreds of reports addressing broad-based issues, such as best practices, as well as reports focusing on individual acquisitions. These reports have included many recommendations. The subcommittee asked GAO to testify on possible problems with and improvements to defense acquisition policy. In doing so, we highlight the risks of conducting business as usual and identify some of the solutions we have found in successful acquisition programs and organizations. What GAO Found: DOD is facing a cascading number of problems in managing its acquisitions. Cost increases incurred while developing new weapon systems mean DOD cannot produce as many of those weapons as intended nor can it be relied on to deliver to the warfighter when promised. Military operations in Afghanistan and Iraq are consuming a large share of DOD resources and causing the department to invest more money sooner than expected to replace or fix existing weapons. Meanwhile, DOD is intent on transforming military operations and has its eye on multiple megasystems that are expected to be the most expensive and complex ever. These costly conditions are running head-on into the nation’s unsustainable fiscal path. DOD knows what to do to achieve more successful outcomes but finds it difficult to apply the necessary discipline and controls or assign much- needed accountability. DOD has written into policy an approach that emphasizes attaining a certain level of knowledge at critical junctures before managers agree to invest more money in the next phase of weapon system development. This knowledge-based approach results in evolutionary—that is, incremental, manageable, predictable—development and inserts several controls to help managers gauge progress in meeting cost, schedule, and performance goals. But DOD is not employing the knowledge-based approach, discipline is lacking, and business cases are weak. Persistent practices show a decided lack of restraint. DOD’s requirements process generates more demand for new programs than fiscal resources can support. DOD compounds the problem by approving so many highly complex and interdependent programs. Once too many programs are approved to start, the budgeting process exacerbates problems. Because programs are funded annually and departmentwide, cross-portfolio priorities have not been established, competition for funding continues over time, forcing programs to view success as the ability to secure the next funding increment rather than delivering capabilities when and as promised. Improving this condition requires discipline in the requirements and budgetary processes. Determining who should be held accountable for deviations and what penalties are needed is crucial. If DOD cannot discipline itself now to execute programs within fiscal realities, then draconian, budget-driven decisions may have to be made later. What GAO Recommends: www.gao.gov/cgi-bin/getrpt?GAO-06-257T. To view the full product, including the scope and methodology, click on the link above. For more information, contact Katherine V. Schinasi, 202-512-4841. [End of section] Mr. Chairman and members of the subcommittee: I am pleased to be here today to discuss why and how to get a better return from the Department of Defense's (DOD) weapon system investments. U.S. weapons are the best in the world, but the programs to acquire them frequently take significantly longer and cost more money than promised and often deliver fewer quantities and other capabilities than planned. It is not unusual for estimates of time and money to be off by 20 to 50 percent. When costs and schedules increase, quantities are cut, and the value for the warfighter--as well as the value of the investment dollar--is reduced. DOD's planned investment in research, development, and procurement of major weapon systems is approximately $1.3 trillion for its current portfolio, with over $800 billion of that investment yet to be made. The planned annual investment is expected to rise from around $149 billion in fiscal year 2005 to $178 billion in fiscal year 2011. Marquee programs include the Army's Future Combat Systems; the Missile Defense Agency's suite of land, sea, air, and space systems; the Navy's advanced ships, such as the DD(X) Destroyer; the Air Force's Transformational Satellite Communications System; and the Joint Strike Fighter. Programs like these--and the Global Information Grid that is designed to interconnect them--are likely to dominate the budget and doctrinal debate well into the next decade. Not only do these programs represent huge technological leaps over their predecessors, DOD is proposing to deliver them faster. The persistent nature of acquisition problems has perhaps made decision makers complacent about cost growth, schedule delays, and quantity reductions in weapon system programs. But fiscal realities, coupled with the larger scale of acquisitions, will not allow budgets to accommodate the typical margins of error. Thus, we must either make tough decisions now to increase the chances for programs to be executable within fiscal realities or brace ourselves for more draconian decisions later driven by those fiscal realities. The means to make the thoughtful decisions are known. My statement today highlights the risks of conducting business as usual and identifies some of the solutions we have found in successful acquisition programs and organizations. The Case for Change: The way DOD develops and produces its major weapons systems has had disappointing outcomes. There is a vast difference between DOD's budgeting plans and the reality of the cost of its systems. Performance, if it is defined as the capability that actually reaches the warfighter, often falls short, as cost increases result in fewer quantities of produced systems and schedule slips. Performance, if it is defined as an acceptable return on investment, has not lived up to promises. Table 1 illustrates seven programs with a significant reduction in buying power; we have reported similar outcomes in many more programs. For example, the Air Force initially planned to buy 648 F/A-22 Raptor tactical aircraft at a program acquisition unit cost of about $125 million (fiscal year 2006 dollars). Technology and design components matured late in the development of the aircraft, which contributed to cost growth and schedule delays. Now, the Air Force plans to buy 181 aircraft at a program acquisition unit cost of about $361 million, an almost 189 percent increase. Table 1: Examples of DOD Programs with Reduced Buying Power: Program: Joint Strike Fighter; Initial investment: $189. 8 billion; Percent of unit cost increase: 26.8%. Program: Future Combat Systems; Initial investment: $82.6 billion; Percent of unit cost increase: 54.4%. Program: F/A-22 Raptor; Initial investment: $81.1 billion; Percent of unit cost increase: 188.7%. Program: Virginia Class Submarine; Initial investment: $53.7 billion; Percent of unit cost increase: 49.7%. Program: Evolved Expendable Launch Vehicle; Initial investment: $15.4 billion; Percent of unit cost increase: 137.8%. Program: Space Based Infrared System High; Initial investment: $4.1 billion; Percent of unit cost increase: 160.2%. Program: Expeditionary Fighting Vehicle; Initial investment: $8.1 billion; Percent of unit cost increase: 35.9%. Source: DOD (data); GAO (analysis and presentation). [End of table] Furthermore, the conventional acquisition process is not agile enough for today's demands. Congress has expressed concern that urgent warfighting requirements are not being met in the most expeditious manner and has put in place several authorities for rapid acquisition to work around the process. The U.S. Joint Forces Command's Limited Acquisition Authority and the Secretary of Defense's Rapid Acquisition Authority seek the ability to get warfighting capability to the field faster. According to U.S. Joint Forces Command officials, it is only through Limited Acquisition Authority that the command has the authority to satisfy the unanticipated, unbudgeted, urgent mission needs of other combatant commands. With a formal process that requires as many as 5, 10, or 15 years to get from program start to production, such experiments are needed to meet the warfighters' needs. Today we are at a crossroad. Our nation is on an unsustainable fiscal path. Long-term budget simulations by GAO, the Congressional Budget Office, and others show that, over the long term, we face a large and growing structural deficit due primarily to known demographic trends and rising health care costs. Continuing on this unsustainable fiscal path will gradually erode, if not suddenly damage, our economy, our standard of living, and ultimately our national security. Federal discretionary spending, along with other federal policies and programs, will face serious budget pressures in the coming years stemming from new budgetary demands and demographic trends. Defense spending falls within the discretionary spending accounts. Further, current military operations, such as those in Afghanistan and Iraq, consume a large share of DOD resources and are causing faster wear on existing weapons. Refurbishment or replacement sooner than planned is putting further pressure on DOD's investment accounts. At the same time DOD is facing these problems, programs are commanding larger budgets. DOD is undertaking new efforts that are expected to be the most expensive and complex ever and on which DOD is heavily relying to fundamentally transform military operations. And it is giving contractors increased program management responsibilities to develop requirements, design products, and select major system and subsystem contractors. Table 2 shows that just 5 years ago, the top five weapon systems cost about $291 billion combined; today, the top five weapon systems cost about $550 billion. Table 2: Total Cost of DOD's Top Five Programs in Fiscal Years 2001 and 2006 (in 2006 dollars): Program: F/A-22 Raptor aircraft; 2001: Cost: $65.0 billion. Program: DDG-51 class destroyer ship; 2001: Cost: $64.4 billion. Program: Virginia class submarine; 2001: Cost: $62.1 billion. Program: C-17 Globemaster airlift aircraft; 2001: Cost: $51.1 billion. Program: F/A-18E/F Super Hornet fighter aircraft; 2001: Cost: $48.2 billion. 2001: Total Cost: $290.8 billion. Program: Joint Strike Fighter; 2006: Cost: $206.3 billion. Program: Future Combat Systems; 2006: Cost: $127.5 billion. Program: Virginia class submarine; 2006: Cost: $80.4 billion. Program: DDG-51 class destroyer ship; 2006: Cost: $70.4 billion. Program: F/A-22 Raptor aircraft; 2006: Cost: $65.4 billion. 2006: Total Cost: $550.0 billion. Source: DOD (data); GAO (analysis and presentation). [End of table] If these megasystems are managed with traditional margins of error, the financial consequences can be dire, especially in light of a constrained discretionary budget. Success for acquisitions means making sound decisions to ensure that program investments are getting promised returns. In the commercial world, successful companies have no choice but to adopt processes and cultures that emphasize basing decisions on knowledge, reducing risks prior to undertaking new efforts, producing realistic cost and schedule estimates, and building-in quality in order to deliver products to customers at the right price, the right time, and the right cost. At first blush, it would seem DOD's definition of success would be very similar: deliver capability to the warfighter at the right price, the right time, and the right cost. However, this is not happening within DOD. In an important sense, success has come to mean starting and continuing programs even when cost, schedule, and quantities must be sacrificed. DOD knows what to do to improve acquisitions but finds it difficult to apply the controls or assign the accountability necessary for successful outcomes. To understand why these problems persist, we must look not just at the product development process but at the underlying requirements and budgeting processes to define problems and find solutions. A Knowledge-Based Process Can Lead to Better Outcomes: Over the last several years, we have undertaken a body of work that examines weapon acquisition issues from a perspective that draws upon lessons learned from best product development practices. Leading commercial firms expect that their program managers will deliver high- quality products on time and within budget. Doing otherwise could result in the customer walking away. Thus, those firms have created an environment and adopted practices that put their program managers in a good position to succeed in meeting these expectations. Collectively, these practices comprise a process that is anchored in knowledge. It is a process in which technology development and product development are treated differently and managed separately. The process of developing technology culminates in discovery--the gathering of knowledge--and must, by its nature, allow room for unexpected results and delays. Leading firms do not ask their product managers to develop technology. Successful programs give responsibility for maturing technologies to a science and technology organizations, rather than the program or product development managers. The process of developing a product culminates in delivery, and, therefore, gives great weight to design and production. The firms demand--and receive--specific knowledge about a new product before production begins. A program does not go forward unless a strong business case on which the program was originally justified continues to hold true. Successful product developers ensure a high level of knowledge is achieved at key junctures in development. We characterize these junctures as knowledge points. These knowledge points and associated indicators are defined as follows: Knowledge point 1: Resources and needs match. This point occurs when a sound business case is made for the product--that is, a match is made between the customer's requirements and the product developer's available resources in terms of knowledge, time, money, and capacity. Achieving a high level of technology maturity at the start of system development is an important indicator of whether this match has been made. This means that the technologies needed to meet essential product requirements have been demonstrated to work in their intended environment. Knowledge point 2: Product design is stable. This point occurs when a program determines that a product's design is stable--that is, it will meet customer requirements, as well as cost, schedule and reliability targets. A best practice is to achieve design stability at the system- level critical design review, usually held midway through development. Completion of at least 90 percent of engineering drawings at the system design review provides tangible evidence that the design is stable. Knowledge point 3: Production processes are mature. This point is achieved when it has been demonstrated that the company can manufacture the product within cost, schedule, and quality targets. A best practice is to ensure that all key manufacturing processes are in statistical control--that is, they are repeatable, sustainable, and capable of consistently producing parts within the product's quality tolerances and standards--at the start of production. A result of this knowledge-based process is evolutionary product development, an incremental approach that enables developers to rely more on available resources rather than making promises about unproven technologies. Predictability is a key to success as successful product developers know that invention cannot be scheduled and its cost is difficult to estimate. They do not bring technology into new product development unless that technology has been demonstrated to meet the user's requirements. Allowing technology development to spill over into product development puts an extra burden on decision makers and provides a weak foundation for making product development estimates. While the user may not initially receive the ultimate capability under this approach, the initial product is available sooner and at a lower, more predictable cost. There is a synergy in this process, as the attainment of each successive knowledge point builds on the preceding one. Metrics gauge when the requisite level of knowledge has been attained. Controls are used to attain a high level of knowledge before making additional significant investments. Controls are considered effective if they are backed by measurable criteria and if decision makers are required to consider them before deciding to advance a program to the next level. Effective controls help decision makers gauge progress in meeting cost, schedule, and performance goals and ensure that managers will (1) conduct activities to capture relevant product development knowledge, (2) provide evidence that knowledge was captured, and (3) hold decision reviews to determine that appropriate knowledge was captured to move to the next phase. The result is a product development process that holds decision makers accountable and delivers excellent results in a predictable manner. A hallmark of an executable program is shorter development cycle times, which allow more systems to enter production more quickly. DOD itself suggests that product development should be limited to about 5 years. Time constraints, such as this, are important because they serve to limit the initial product's requirements. Limiting product development cycle times to 5 years or less would allow for more frequent assimilation of new technologies into weapon systems, speeding new technology to the warfighter, hold program managers accountable, as well as make more frequent and predictable work in production, where contractors and the industrial base can profit by being efficient. Despite Policy, DOD Is Not Employing a Knowledge-Based Process: DOD's policy adopts the knowledge-based, evolutionary approach used by leading commercial companies that enables developers to rely more on available resources rather than making promises about unproven technologies. The policy provides a framework for developers to ask themselves at key decision points whether they have the knowledge they need to move to the next phase of acquisition. For example, DOD Directive 5000.1 states that program managers "shall provide knowledge about key aspects of a system at key points in the acquisition process," such as demonstrating "technologies in a relevant environment … prior to program initiation." This knowledge-based framework can help managers gain the confidence they need to make significant and sound investment decisions for major weapon systems. In placing greater emphasis on evolutionary product development, the policy sets up a more manageable environment for achieving knowledge. However, the longstanding problem of programs beginning development with immature technologies is continuing to be seen on even the newest programs. Several programs approved to begin product development within only the last few years began with most of their technologies immature and have already experienced significant development cost increases. In the case of the Army's Future Combat Systems, nearly 2 years after program launch and with $4.6 billion invested, only 1 out of more than 50 critical technologies is considered mature and the research and development cost estimate has grown by 48 percent. In March 2005, we reported that very few programs--15 percent of the programs we assessed--began development having demonstrated high levels of technology maturity. Acquisition unit costs for programs leveraging mature technologies increased by less than 1 percent, whereas programs that started development with immature technologies experienced an average acquisition unit cost increase of nearly 21 percent over the first full estimate. Establishing a Sound Business Case Depends on Disciplined Requirements and Funding Process: The decision to start a new program is the most highly leveraged point in the product development process. Establishing a sound business case for individual programs depends on disciplined requirements and funding processes. Our work has shown that DOD's requirements process generates more demand for new programs than fiscal resources can support. DOD compounds the problem by approving so many highly complex and interdependent programs. Moreover, once a program is approved, requirements can be added along the way that increases costs and risks. Once too many programs are approved to start, the budgeting process exacerbates problems. Because programs are funded annually and department wide, cross-portfolio priorities have not been established, competition for funding continues over time, forcing programs to view success as the ability to secure the next funding increment rather than delivering capabilities when and as promised. As a result, there is pressure to suppress bad news about programs, which could endanger funding and support, as well as to skip testing because of its high cost. Concurrently, when faced with budget constraints, senior officials tend to make across-the-board cuts to all programs rather than make the hard decisions as to which ones to keep and which ones to cancel or cut back. In many cases, the system delivers less performance than promised when initial investment decisions were made. So, the condition we encounter time after time describes a predictable outcome. The acquisition environment encourages launching product developments that embody more technical unknowns and less knowledge about the performance and production risks they entail. A new weapon system is encouraged to possess performance features that significantly distinguish it from other systems and promises the best capability. A new program will not be approved unless its costs fall within forecasts of available funds and, therefore, looks affordable. Because cost and schedule estimates are comparatively soft at the time, successfully competing for funds encourages the program's estimates to be squeezed into the funds available. Consequently, DOD program managers have incentives to promote performance features and design characteristics that rely on immature technologies and decision makers lack the knowledge they need to make good decisions. The Path to Better Decisions: A path can be laid out to make decisions that will lead to better program choices and better outcomes. Much of this is known and has been recommended by one study or another. GAO itself has issued hundreds of reports. The key recommendations we have made have been focused on the product development process: * constraining individual program requirements by working within available resources and by leveraging systems engineering; * establishing clear business cases for each individual investment; * enabling science and technology organizations to shoulder the technology burden; * ensuring that the workforce is capable of managing requirements trades, source selection, and knowledge-based acquisition strategies; and: * establishing and enforcing controls to ensure that appropriate knowledge is captured and used at critical junctures before moving programs forward and investing more money. As I have outlined above, however, setting the right conditions for successful acquisitions outcomes goes beyond product development. We are currently examining how to bring discipline to the Department's requirements and budgetary process and the role played by the program manager. As we conduct this work, we will be asking: * who is currently accountable for acquisition decisions; * who should be held accountable; * how much deviation from the original business case is allowed before the entire program investment is reconsidered; and: * what is the penalty when investments do not result in meeting promised warfighter needs? We can make hard, but thoughtful, decisions now or postpone them, allowing budgetary realities to force draconian decisions later. Mr. Chairman, this concludes my prepared statement. I would be happy to respond to any questions that you or other members of the subcommittee may have. Contacts and Staff Acknowledgments: For further information regarding this testimony, please contact Katherine V. Schinasi at (202) 512-4841 or schinasik@gao.gov. Individuals making key contributions to this testimony included Paul L. Francis, David B. Best, David J. Hand, Alan R. Frazier, Adam Vodraska, and Lily J. Chin. [End of section] Related GAO Products: Space Acquisitions: Stronger Development Practices and Investment Planning Needed to Address Continuing Problems. GAO-05-891T. Washington, D.C.: July 12, 2005. Air Force Procurement: Protests Challenging Role of Biased Official Sustained. GAO-05-436T. Washington, D.C.: April 14, 2005. Tactical Aircraft: F/A-22 and JSF Acquisition Plans and Implications for Tactical Aircraft Modernization. GAO-05-591T. Washington, D.C.: April 6, 2005. Defense Acquisitions: Assessments of Selected Major Weapon Programs. GAO-05-301. Washington, D.C.: March 31, 2005. Defense Acquisitions: Future Combat Systems Challenges and Prospects for Success. GAO-05-428T. Washington, D.C.: March 16, 2005. Defense Acquisitions: Stronger Management Practices Are Needed to Improve DOD's Software-Intensive Weapon Acquisitions. GAO-04-393. Washington, D.C.: March 1, 2004. Defense Acquisitions: DOD's Revised Policy Emphasizes Best Practices, but More Controls Are Needed. GAO-04-53. Washington, D.C.: November 10, 2003. Best Practices: Setting Requirements Differently Could Reduce Weapon Systems' Total Ownership Costs. GAO-03-57. Washington, D.C.: February 11, 2003. Best Practices: Capturing Design and Manufacturing Knowledge Early Improves Acquisition Outcomes. GAO-02-701. Washington, D.C.: July 15, 2002. Defense Acquisitions: DOD Faces Challenges in Implementing Best Practices. GAO-02-469T. Washington, D.C.: February 27, 2002. Best Practices: Better Matching of Needs and Resources Will Lead to Better Weapon System Outcomes. GAO-01-288. Washington, D.C.: March 8, 2001. Best Practices: A More Constructive Test Approach Is Key to Better Weapon System Outcomes. GAO/NSIAD-00-199. Washington, D.C.: July 31, 2000. Defense Acquisition: Employing Best Practices Can Shape Better Weapon System Decisions. GAO/T-NSIAD-00-137. Washington, D.C.: April 26, 2000. Best Practices: DOD Training Can Do More to Help Weapon System Program Implement Best Practices. GAO/NSIAD-99-206. Washington, D.C.: August 16, 1999. Best Practices: Better Management of Technology Development Can Improve Weapon System Outcomes. GAO/NSIAD-99-162. Washington, D.C.: July 30, 1999. Defense Acquisitions: Best Commercial Practices Can Improve Program Outcomes. GAO/T-NSIAD-99-116. Washington, D.C.: March 17, 1999. Defense Acquisition: Improved Program Outcomes Are Possible. GAO/T- NSIAD-98-123. Washington, D.C.: March 18, 1998. Best Practices: Successful Application to Weapon Acquisition Requires Changes in DOD's Environment. GAO/NSIAD-98-56. Washington, D.C.: February 24, 1998. Major Acquisitions: Significant Changes Underway in DOD's Earned Value Management Process. GAO/NSIAD-97-108. Washington, D.C.: May 5, 1997. Best Practices: Commercial Quality Assurance Practices Offer Improvements for DOD. GAO/NSIAD-96-162. Washington, D.C.: August 26, 1996. GAO's Mission: The Government Accountability Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's Web site (www.gao.gov). Each weekday, GAO posts newly released reports, testimony, and correspondence on its Web site. To have GAO e-mail you a list of newly posted products every afternoon, go to www.gao.gov and select "Subscribe to Updates." Order by Mail or Phone: The first copy of each printed report is free. Additional copies are $2 each. A check or money order should be made out to the Superintendent of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single address are discounted 25 percent. Orders should be sent to: U.S. Government Accountability Office: 441 G Street NW, Room LM Washington, D.C. 20548: To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202) 512-6061: To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov Automated answering system: (800) 424-5454 or (202) 512-7470: Congressional Relations: Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400 U.S. Government Accountability Office, 441 G Street NW, Room 7125 Washington, D.C. 20548: Public Affairs: Paul Anderson, Manager, AndersonP@gao.gov (202) 512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149 Washington, D.C. 20548: