Summary
The Department of Defense (DOD) relies on a relatively small cadre of officials to develop and deliver weapon systems. In view of the importance of DOD's investment in weapon systems, we have undertaken an extensive body of work that examines DOD's acquisition issues from a perspective that draws lessons learned from the best commercial product development efforts to see if they apply to weapon system acquisitions. In response to a request from the Chairman and Ranking Minority Member of the Subcommittee on Readiness and Management Support, Senate Committee on Armed Services, this report assesses (1) how successful commercial companies position their program managers, (2) how DOD positions its program managers, and (3) underlying reasons for the differences. In compiling this report, GAO conducted a survey of program managers. See GAO-06-112SP.
U.S. weapons are among the best in the world, but the programs to acquire them often take significantly longer and cost more money than promised and often deliver fewer quantities and capabilities than planned. It is not unusual for estimates of time and money to be off by 20 to 50 percent. When costs and schedules increase, quantities are cut, and the value for the warfighter--as well as the value of the investment dollar--is reduced. When we examined private sector companies that developed complex and technical products similar to DOD, we found that their success hinged on the tone set by leadership and disciplined, knowledge-based processes for product development and execution. More specifically, long before the initiation of a new program, senior company leaders made critical investment decisions about the firm's mix of products so that they could commit to programs they determined best fit within their overall goals. These decisions considered long-term needs versus wants as well as affordability and sustainability. Once high level investment decisions were made, senior leaders ensured that programs did not begin unless they had a business case that made sure resources were in-hand to execute the program--that is, time, technology, money, and people. Once a business case was established, senior leaders tasked program managers with executing that business case for each new product from initiation to delivery, but required their program managers to use a knowledge-based product development process that demanded appropriate demonstrations of technology, designs, and processes at critical junctures. The program manager was empowered to execute the business case, but also held accountable for delivering the right product at the right time for the right cost. Requiring the program manager to stay throughout the length of a project was a principal means of enforcing accountability. Overall, by providing the right foundation and support for program managers, the companies we visited were able to consistently deliver quality products within targets, and in turn, transform themselves into highly competitive organizations. DOD program managers are put in a very different situation. DOD leadership rarely separates long-term wants from needs based on credible, future threats. As a result, DOD starts many more programs than it can afford--creating a competition for funds that pressures program managers to produce optimistic cost estimates and to overpromise capabilities. Moreover, our work has shown that DOD allows programs to begin without establishing a formal business case. And once they begin, requirements and funding change over time. In fact, program managers personally consider requirements and funding instability--which occur throughout the program--to be their biggest obstacles to success. Program managers also believe that they are not sufficiently empowered to execute their programs, and that because much remains outside of their span of control, they cannot be held accountable.
Recommendations
Our recommendations from this work are listed below with a Contact for more information. Status will change
from "In process" to "Implemented" or "Not implemented" based on our follow up work.
Director:
Team:
Phone:
Michael J. Sullivan
Government Accountability Office: Acquisition and Sourcing Management
(937) 258-7915
Recommendations for Executive Action
Recommendation: To ensure program managers are well positioned to successfully execute and be held accountable for weapon acquisitions, DOD should develop an investment strategy that, at a minimum, determines the priority order of needed capabilities with a corollary assessment of the resources, that is dollars, technologies, time and people needed to achieve these capabilities. The remaining capabilities should be set out separately as desirable, resources permitting.
Agency Affected: Department of Defense
Status: In process
Comments: DOD concurred with this recommendation, but stated in follow-up that the Quadrennial Defense Review of February 2006 would address issues of investment planning. The review proposed actions to improve decision making by implementing more agile, transparent and open processes. Further, the review advocated reaching investment decisions through collaboration among the warfighters, acquisition, and resource communities. The review called for decisions about investments to be informed by the acquisition community's judgment of technological feasibility and return on investment as well as the resource community's assessment of affordability. According to officials, DOD conducted a business process review that resulted in a series of recommended policy initiatives. The initiatives included improvements to DOD governance and changes to tactical management that would improve program stability and transparency. The department planned to assess the initiatives through both executive review and a series of pilot efforts designed to refine the policies. The department believes that these initiatives will respond to all three GAO recommendations. Development and evaluation of the initiatives is still underway and will be completed when the deputy Secretary of Defense has approved the policies. According to a recently published GAO study (GAO-07-388) Best Practices: An Integrated Portfolio Management Approach to Weapon System Investments Could Improve DOD's Acquisition Outcomes) DOD has begun to pilot test initiatives to establish an early decision gate for reviewing proposed programs at the concept stage, testing portfolio management approaches in selected capability areas, and setting up capital accounts for programs in development. In addition DOD is completing a revision of the 5000 series which officials believe will provide a regulatory basis for implementation. However, the revised regulations have not yet been issued. Further, the 2008 National Defense Authorization Act, Section 943 Pub L. No. 110-181 requires DOD, both the Milestone Decision Authority and the Joint Requirement Oversight Council, to certify before beginning technology development that the system fulfills an approved capabilities document, the system is necessary and appropriate and that a cost estimate for the system and the level of resources required to develop and procure the system is consistent with the priority assigned by the Joint Requirements Council.
Recommendation: To ensure program managers are well positioned to successfully execute and be held accountable for weapon acquisitions, DOD should develop an investment strategy that, at a minimum, lays out incremental product development programs for achieving desired capabilities.
Agency Affected: Department of Defense
Status: In process
Comments: The Quadrennial Review of February 2006 stated that the Department is considering adoption of a risk-based source selection process. The Review reports that such a process would require senior leaders to make tradeoffs necessary to balance performance, time, and available resources, and result in a more predictable and stable acquisition that can be measured by cost, schedule and performance. According to a recently published GAO study (GAO-07-388)Best Practices: An Integrated Portfolio Management Approach to Weapon System Investments Could Improve DOD's Acquisition Outcomes) DOD has begun to pilot test initiatives to establish an early decision gate for reviewing proposed programs at the concept stage, testing portfolio management approaches in selected capability areas, and setting up capital accounts for programs in development. These initiatives are intended to operate within DOD's existing organizational and process framework. Two congressional initiatives contained in the National Defense Authorization Acts Pub L. 110-181, section 817 and Pub Law110-181, section 943 require DOD to provide certifications for development. DOD is currently revising its 5000 series regulations with could address some of these issues. However the revision is not yet final.
Recommendation: To ensure program managers are well positioned to successfully execute and be held accountable for weapon acquisitions, DOD should develop an investment strategy that, at a minimum, establishes controls to ensure that requirements, funding, and acquisition processes will work together so that DOD will sustain its commitment to its priority programs.
Agency Affected: Department of Defense
Status: In process
Comments: DOD commented that its processes are expected to become more collaborative, but did not identify specific mechanisms for implementing such collaboration. According to a recently published GAO study (GAO-07-388)Best Practices: An Integrated Portfolio Management Approach to Weapon System Investments Could Improve DOD's Acquisition Outcomes) DOD has begun to pilot test initiatives to establish an early decision gate for reviewing proposed programs at the concept stage, testing portfolio management approaches in selected capability areas, and setting up capital accounts for programs in development. DOD is revising the 5000 series. According to officials the revisions could provide a regulatory basis for implementing this recommendation. The 2008 National Defense Authorization Act, Section 943 Pub L. No. 110-181 requires DOD, both the Milestone Decision Authority and the Joint Requirement Oversight Council, to certify before beginning technology development that the system fulfills an approved capabilities document, the system is necessary and appropriate and that a cost estimate for the system and the level of resources required to develop and procure the system is consistent with the priority assigned by the Joint Requirements Council.
Recommendation: As DOD works to develop the strategy, it should take an interim step by identifying priorities for programs that are already past milestone B (the formal start of development). Once the strategy is complete, it should be used by the Office of the Secretary of Defense to prepare and assess annual budget proposals as well as to balance funding between science and technology efforts and acquisition efforts to ensure that robust technology development efforts are conducted, but outside the acquisition program until reaching maturity.
Agency Affected: Department of Defense
Status: In process
Comments: In follow-up comments to the report, DOD stated that it already had established criteria to be met before a program could be formally initiated, including a joint requirements oversight council validated requirement; an analysis of alternatives; evidence of technology maturity; an approved acquisition plan and program baseline; an independent cost estimate and full funding. They stated that mechanisms to improve the efficiency of these policies were being considered in the revision to the 5000.
Recommendation: For each new major weapons program, DOD should require that senior-level officials from the requirements, science and technology, program management, testing communities as well as the Office of the Comptroller formally commit to a business case prior to approving a program at milestone B. At a minimum, the business case should demonstrate that a requirement exists that warrants a materiel solution consistent with national military strategy.
Agency Affected: Department of Defense
Status: In process
Comments: In follow-up comments, DOD stated that it already had established criteria to be met before a program could be formally initiated, including a joint requirements oversight council validated requirement; an analysis of alternatives; evidence of technology maturity; an approved acquisition plan and program baseline; an independent cost estimate and full funding. DOD officials stated that mechanisms to improve the efficiency of these policies were being considered in the revision of the 5000 series. The unpredictable nature of DOD programs could be traced to instabilities, including funding of the acquisition system. DOD's revision could impact its requirements process. In addition the 2008 National Defense Authorization Act (Pub L. No 110-181 ) requires the Department to conduct a Milestone A certification to begin a technology development including addressing the availability of funding and the relative priority of developments.
Recommendation: For each new major weapons program, DOD should require that senior-level officials from the requirements, science and technology, program management, testing communities as well as the Office of the Comptroller formally commit to a business case prior to approving a program at milestone B. At a minimum, the business case should demonstrate that an independent analysis of alternatives has been conducted.
Agency Affected: Department of Defense
Status: In process
Comments: In follow-up comments, DOD stated that it already had established criteria to be met before a program could be formally initiated, including a joint requirements oversight council validated requirement; an analysis of alternatives; evidence of technology maturity; an approved acquisition plan and program baseline; an independent cost estimate and full funding. They stated that mechanisms to improve the efficiency of these policies were being considered in the revision to the 5000 series.
Recommendation: For each new major weapons program, DOD should require that senior-level officials from the requirements, science and technology, program management, testing communities as well as the Office of the Comptroller formally commit to a business case prior to approving a program at milestone B. At a minimum, the business case should demonstrate that the developer has the requisite technical knowledge to meet the requirement.
Agency Affected: Department of Defense
Status: In process
Comments: In follow-up comments, DOD stated that it already had established criteria to be met before a program could be formally initiated, including a joint requirements oversight council validated requirement; an analysis of alternatives; evidence of technology maturity; an approved acquisition plan and program baseline; an independent cost estimate and full funding. In its 2006 National Defense Authorization Act, the congress provided that before a major defense program could receive approval to begin development (prior to Milestone B) the Milestone Decision Authority must certify the programs affordability, reasonable cost and schedule estimates have been prepared for development and production, market research was conducted to avoid duplication of technologies and that the technologies have been demonstrated in a relevant environment. DOD officials have stated that the planned revision to the 5000 will provide additional metrics and a basis for evaluating the business case.
Recommendation: For each new major weapons program, DOD should require that senior-level officials from the requirements, science and technology, program management, testing communities as well as the Office of the Comptroller formally commit to a business case prior to approving a program at milestone B. At a minimum, the business case should demonstrate that the developer has a knowledge-based product development and production plan that will attain high levels of design and production maturity.
Agency Affected: Department of Defense
Status: In process
Comments: In follow-up comments, DOD stated that it already had established criteria to be met before a program could be formally initiated, including a joint requirements oversight council validated requirement; an analysis of alternatives; evidence of technology maturity; an approved acquisition plan and program baseline; an independent cost estimate and full funding. In follow-up comments, DOD stated that it already had established criteria to be met before a program could be formally initiated, including a joint requirements oversight council validated requirement; an analysis of alternatives; evidence of technology maturity; an approved acquisition plan and program baseline; an independent cost estimate and full funding. In its 2006 National Defense Authorization Act, the congress provided that before a major defense program could receive approval to begin development (prior to Milestone B) the Milestone Decision Authority must certify the programs affordability, reasonable cost and schedule estimates have been prepared for development and production, market research was conducted to avoid duplication of technologies and that the technologies have been demonstrated in a relevant environment. DOD officials have stated that the planned revision to the 5000 will provide additional metrics and a basis for evaluating the business case.
Recommendation: For each new major weapons program, DOD should require that senior-level officials from the requirements, science and technology, program management, testing communities as well as the Office of the Comptroller formally commit to a business case prior to approving a program at milestone B. At a minimum, the business case should demonstrate that reasonable estimates have been developed to execute the product development and production plan.
Agency Affected: Department of Defense
Status: In process
Comments: In follow-up comments, DOD stated that it already had established criteria to be met before a program could be formally initiated, including a joint requirements oversight council validated requirement; an analysis of alternatives; evidence of technology maturity; an approved acquisition plan and program baseline; an independent cost estimate and full funding. In follow-up comments, DOD stated that it already had established criteria to be met before a program could be formally initiated, including a joint requirements oversight council validated requirement; an analysis of alternatives; evidence of technology maturity; an approved acquisition plan and program baseline; an independent cost estimate and full funding. In its 2006 National Defense Authorization Act, the congress provided that before a major defense program could receive approval to begin development (prior to Milestone B) the Milestone Decision Authority must certify the programs affordability, reasonable cost and schedule estimates have been prepared for development and production, market research was conducted to avoid duplication of technologies and that the technologies have been demonstrated in a relevant environment. DOD officials have stated that the planned revision to the 5000 will provide additional metrics and a basis for evaluating the business case.
Recommendation: For each new major weapons program, DOD should require that senior-level officials from the requirements, science and technology, program management, testing communities as well as the Office of the Comptroller formally commit to a business case prior to approving a program at milestone B. At a minimum, the business case should demonstrate that funding is available to execute the plan.
Agency Affected: Department of Defense
Status: In process
Comments: Several pieces of legislation require that DOD ensure that funding is available and cost estimates prepared before proceeding with development. Among these are National Defense Authorization Act for 2006 Pub L. No 109-163, section 801 and National Defense Authorization Act for Fiscal Year 2008 Pub L. No 110-181, section 817. DOD officials have undertaken a revision of the 5000 series, where practices to implement these statutes can be promulgated. However the revisions are still under consideration.
Recommendation: To ensure program managers are well positioned to successfully execute and be held accountable for weapon acquisitions, DOD should develop and implement a process to instill and sustain accountability for successful program outcomes. At a minimum, this should consider matching program manager tenure with delivery of a product or for system design and demonstration.
Agency Affected: Department of Defense
Status: Implemented
Comments: In response to GAO's report, the Authorization Act fiscal year 2007 (Public Law 109-364) Section 853 addressed empowerment and accountability for program managers. The following were recommended: The Act requires the Secretary of Defense to establish a comprehensive strategy to enhance the role of DOD program managers. The strategy was to include enhanced training, mentoring, career path identification, incentives for recruitment and retention, improved resources and technical support, collection and dissemination of best practices and lessons learned, common templates and tools for data gathering and analysis for program management and oversight purposes, increased accountability for results, and enhanced monetary and non monetary incentives. The legislation directed the Secretary of Defense to revise guidance for major defense acquisition programs to address the qualifications, resources, responsibilities, tenure and accountability of program managers for the program development period (pre-milestone B or Key Decision Point B). In response DOD has adopted a policy on tenure for program managers and established a program management agreement as a tool for increasing the accountability of program managers.
Recommendation: To ensure program managers are well positioned to successfully execute and be held accountable for weapon acquisitions, DOD should develop and implement a process to instill and sustain accountability for successful program outcomes. At a minimum, this should consider tailoring career paths and performance management systems to incentivize longer tenures.
Agency Affected: Department of Defense
Status: Implemented
Comments: In response to GAO's report, the Authorization Act fiscal year 2007 (Public Law 109-364) Section 853 addressed empowerment and accountability for program managers. The following were recommended: The Act requires the Secretary of Defense to establish a comprehensive strategy to enhance the role of DOD program managers. The strategy was to include enhanced training, mentoring, career path identification, incentives for recruitment and retention, improved resources and technical support, collection and dissemination of best practices and lessons learned, common templates and tools for data gathering and analysis for program management and oversight purposes, increased accountability for results, and enhanced monetary and non monetary incentives. The legislation directed the Secretary of Defense to revise guidance for major defense acquisition programs to address the qualifications, resources, responsibilities, tenure and accountability of program managers for the program development period (pre-milestone B or Key Decision Point B). In response DOD has adopted a policy on tenure for program managers and established a program management agreement as a tool for increasing the accountability of program managers.
Recommendation: To ensure program managers are well positioned to successfully execute and be held accountable for weapon acquisitions, DOD should develop and implement a process to instill and sustain accountability for successful program outcomes. At a minimum, this should consider empowering program managers to execute their programs, including an examination of whether and how much additional authority can be provided over funding, staffing, and approving requirements proposed after milestone B.
Agency Affected: Department of Defense
Status: Implemented
Comments: In response to GAO's report, the Authorization Act fiscal year 2007 (Public Law 109-364) Section 853 addressed empowerment and accountability for program managers. The following were recommended: The Act requires the Secretary of Defense to establish a comprehensive strategy to enhance the role of DOD program managers. The strategy was to include enhanced training, mentoring, career path identification, incentives for recruitment and retention, improved resources and technical support, collection and dissemination of best practices and lessons learned, common templates and tools for data gathering and analysis for program management and oversight purposes, increased accountability for results, and enhanced monetary and non monetary incentives. The legislation directed the Secretary of Defense to revise guidance for major defense acquisition programs to address the qualifications, resources, responsibilities, tenure and accountability of program managers for the program development period (pre-milestone B or Key Decision Point B). In response DOD has adopted a policy on tenure for program managers and established a program management agreement as a tool for increasing the accountability of program managers.
Recommendation: To ensure program managers are well positioned to successfully execute and be held accountable for weapon acquisitions, DOD should develop and implement a process to instill and sustain accountability for successful program outcomes. At a minimum, this should consider developing and providing automated tools to enhance management and oversight as well as to reduce time required to prepare status information.
Agency Affected: Department of Defense
Status: Implemented
Comments: In response to GAO's report, the Authorization Act fiscal year 2007 (Public Law 109-364) Section 853 addressed empowerment and accountability for program managers. The following were recommended: The Act requires the Secretary of Defense to establish a comprehensive strategy to enhance the role of DOD program managers. The strategy was to include enhanced training, mentoring, career path identification, incentives for recruitment and retention, improved resources and technical support, collection and dissemination of best practices and lessons learned, common templates and tools for data gathering and analysis for program management and oversight purposes, increased accountability for results, and enhanced monetary and non monetary incentives. The legislation directed the Secretary of Defense to revise guidance for major defense acquisition programs to address the qualifications, resources, responsibilities, tenure and accountability of program managers for the program development period (pre-milestone B or Key Decision Point B). In response DOD has adopted a policy on tenure for program managers and established a program management agreement as a tool for increasing the accountability of program managers.
