This is the accessible text file for GAO report number GAO-04-164R entitled 'Military Aircraft: Institute for Defense Analyses Purchase Price Estimate for the Air Force's Aerial Refusing Aircraft Leasing Proposal' which was released on October 14, 2003. This text file was formatted by the U.S. General Accounting Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. GAO-04-164R: United States General Accounting Office: Washington, DC 20548: October 14, 2003: The Honorable John Warner Chairman The Honorable Carl Levin Ranking Member Committee on Armed Services United States Senate: Subject: Military Aircraft: Institute for Defense Analyses Purchase Price Estimate for the Air Force's Aerial Refueling Aircraft Leasing Proposal: On September 4, 2003, we provided the Senate Committee on Armed Services our observations on the Air Force's proposed lease of 100 Boeing 767 aircraft modified for aerial refueling, to be known as the KC-767A.[Footnote 1] At the hearing, the Committee heard testimony from the Institute for Defense Analyses (IDA) concerning the results of a study it did on the estimated acquisition cost of each aircraft.[Footnote 2] IDA concluded that $120.7 million was a reasonable price for a KC-767A aerial refueling aircraft. At the time of the hearing, we had just obtained access to the IDA study and were not in position to comment on it. On September 5, 2003, you asked us to analyze the IDA study and provide you our assessment. This letter responds to that request. Our objectives were to assess the reasonableness of IDA's approach to the issue of pricing the KC-767A aircraft and to provide any comments that we had on its methodology used to estimate the base prices and costs for each aircraft. To analyze the IDA report, we reviewed the report submitted to the Office of the Under Secretary of Defense (Acquisition Technology and Logistics) and the Office of the Director, Program Analysis and Evaluation, Department of Defense (DOD).[Footnote 3] We met with the IDA (Cost Analysis and Research Division) and Air Force acquisition officials from the Office of the Assistant Secretary (Acquisition), Directorate of Global Reach. We also met with Office of Management and Budget officials to discuss the methodology, conclusions, and data used in IDA's analysis. In addition, we examined the Air Force's draft lease (which is still in negotiation and is subject to change) and reviewed documents and briefings from the Office of the Assistant Secretary of the Air Force for Acquisitions, Air Mobility Programs, to identify issues and costs that are related to the IDA study. Finally, we used data gathered for our review of the DOD response to your suggestion that the Air Force lease 25 aircraft and purchase 75[Footnote 4] and additional data gathered for our on-going review of tanker requirements being conducted for the House Armed Services Committee's Subcommittee on Readiness. Summary: We believe the IDA has provided a reasonable and comprehensive estimate within the stated objective and assumptions given by the study authors. The objective of the IDA study was to establish a reasonable direct purchase price for 100 KC-767A aircraft. (Refer to the enclosure for a description of the aircraft specifications as assessed by the IDA.) The IDA was not asked to and did not address or make any assumptions about provisions of the proposed lease, financing, suitability of leasing, or any other acquisition alternatives to the proposal. Further, the IDA did not attempt to reconcile its study results to coincide with the aircraft configuration currently being negotiated between the Air Force and Boeing. Background: The Department of Defense Appropriations Act for fiscal year 2002[Footnote 5] included a provision allowing the Air Force to establish a multiyear pilot program for leasing Boeing 767 commercial aircraft to be used as aerial refueling aircraft. Aerial refueling provides a key capability in enhancing the mobility of U.S. forces and the Air Force is in the process of planning for the replacement of its aging aircraft fleet. As you know, the Air Force is in the final stages of negotiating a lease agreement with Boeing for 100 new 767 aircraft that will be modified for use as refueling aircraft. While recognizing that aerial refueling is a key capability that is essential to the mobility of U.S. forces, the Senate Committee on Armed Services has raised many issues concerning the proposed Air Force lease, including questions about the validity of an independent DOD- commissioned study performed by the Institute for Defense Analyses, a federally funded research and development center. The July 2003 IDA study entitled Purchase Price Estimate for the KC-767A Tanker Aircraft concluded that $120.7 million was a "conservative, robust" estimate of a reasonable purchase price for the proposed KC-767A aircraft. The IDA Study and Our Assessment: To determine the cost of acquiring 100 KC-767A aircraft and to formulate the associated assessment methodology and data sources, IDA first developed specific categories to describe the aircraft elements being priced. These categories,[Footnote 6] reflecting the most significant financial investments to the proposed aircraft purchase, were (a) basic B767-200ER aircraft, (b) enhanced B767-200ER features, (c) combination (so-called "combi") configuration modifications, (d) auxiliary fuel tanks, (e) tanker and other modifications, and (f) development costs. The Table provides a detailed summary of IDA's purchase price analysis. Table 1: Summary of Institute for Defense Analyses' KC-767A Tanker/ Combi Purchase Price Analysis: FY02 Dollars in millions. Basic B767-200ER; IDA unit price estimate: 72.1; Primary analysis technique: Commercial pricing; Primary data sources: Consultants, Department of Transportation data. Enhanced B767-200ER features; IDA unit price estimate: 1.6; Primary analysis technique: Commercial pricing; Primary data sources: Consultants, Boeing, Air Force data, IDA models, vendor quotes. 'Combi' modifications; IDA unit price estimate: 9.5; Primary analysis technique: Commercial pricing; Primary data sources: Consultants, public data. Auxiliary fuel tanks; IDA unit price estimate: 6.3; Primary analysis technique: Cost analysis; Primary data sources: Vendor quotes, IDA models. Tanker and other modifications; IDA unit price estimate: 20.3; Primary analysis technique: Cost analysis; Primary data sources: IDA models, Air Force, Boeing. Development costs; IDA unit price estimate: 10.9; Primary analysis technique: Cost analysis; Primary data sources: Air Force, IDA models. Total; IDA unit price estimate: 120.7. Source: IDA and Air Force. [End of table] IDA separately estimated each of the above categories using a variety of estimating techniques including cost estimating relationships, cost models, and analogous data. In most cases, they used several techniques to generate multiple cost estimates of a single item. The IDA took an average of these estimates as the final estimate value in each category. Likewise, the IDA used commercial pricing techniques and marketing analysis where possible and traditional cost analysis techniques, as dictated by the content of the aircraft, where IDA determined that no significant commercial market existed.[Footnote 7] In addition, IDA's analysis relied on data from a variety of public sources, including other government sources, the analyses of consultant organizations hired by the IDA, data supplied by Boeing, other aerospace suppliers, and the Air Force. The specific categories and our conclusions about the reasonableness of the IDA analysis of each category follow: * Basic B767-200ER: This element represents the cost to acquire the baseline aircraft, the commercial aircraft upon which the KC-767A is based. The necessary modifications and design changes would be applied to this aircraft. To determine the cost, the IDA gathered pricing information from five different commercial and government sources. These prices were then averaged to obtain the price used in the study. We believe this was a reasonable approach. In analytical terms, the coefficient of variance for the five data points used is very small (4 percent), which shows the various estimates are close together and provides confidence in the estimate.[Footnote 8] We also note that data provided by the Air Force in October 2003 showed that they used six different methods to estimate the cost of the basic B767-200ER aircraft. The prices developed under the approaches ranged from $60.0 million to $80.5 million per aircraft for an average of $71.1 million. Air Force officials ultimately used a weighted average and concluded that a reasonable price for the basic aircraft would be about $79.0 million per aircraft. * Enhanced B767-200ER features: This category includes features added to the basic aircraft to create the KC-767A. These items are not normally available on the 200ER variant of the B767 but have been specified by the Air Force for the KC-767A. However, Boeing has still not released its updated version of the system specifications negotiated for the B767-200ER aircraft variant. Each feature (e.g. enhanced cockpit, landing gear, and engines; upgraded power supply, etc.) was estimated separately using either commercially available information or IDA cost models. Again, given the complexity and assumptions associated with these engineering features, we believe the IDA's approach to be reasonable and comprehensive for estimating this element's cost. * 'Combi' modifications: This element contains modifications to the baseline B767-200ER to allow the aircraft to transport passengers or freight, or a combination of the two simultaneously. To calculate this estimate, IDA performed three separate estimates and then took the average. Because this is a well-accepted and standard means to conducting analysis of this type, we believe this methodology represents a sound approach. Also, the coefficient of variation is very small (6 percent). This shows that the various estimates are close together and provides further confidence in the estimate. * Auxiliary fuel tanks: This category provides for extra fuel capacity (i.e., the lower fuselage fuel tanks, pumps, and installation materials required to give the KC-767A additional fuel capacity). IDA solicited quotes from two different vendors to generate the estimate. The two quotes were averaged for the final result. Although, for assessment purposes, the quotes were further apart with a coefficient of variation of 34 percent, we believe they are only slightly above the rule of thumb for the range of reasonableness of about a 30 percent coefficient of variation. * Tanker and other Air Force-unique modifications: This sub-grouping includes modifications associated with the refueling, fuel-receiving and military-unique capabilities. These modifications are associated with the ability to receive and offload fuel, including the installation of the centerline boom, the hose/drogue unit, the fuel receiving receptacle, the remote aerial refueling operator, and the plumbing and electrical changes associated with these items.[Footnote 9] Avionics and miscellaneous Air Force requirements are also included in this category. The IDA separately estimated each modification that would occur. A cost estimating relationship was used to develop the final estimates, which were based on historical data collected by the IDA.[Footnote 10] We believe this to be a reasonable methodology and analytical tool for pricing this element. * Development costs: This element, broken down into three sub- categories, captures any non-recurring investment costs needed to design the KC-767A and represents the investment Boeing would make to create the KC-767A configuration. The IDA accepted Air Force data for most of the costs and used a cost estimating relationship to estimate the remainder. The IDA assumed that a portion of these development costs would be borne by the impending foreign sales to Italy and Japan and adjusted their estimate accordingly. We believe this is a reasonable approach for pricing this element. In addition, as a further check on its assessment approach, the IDA enlisted an expert panel of former and/or retired distinguished Air Force officials to review its analytical approach and study methodology as well as its positions on a range of reasonable prices for the KC- 767A. IDA officials told us that the panel concurred with IDA's final report assessment and reporting results. In its analysis, IDA used a December 19, 2002, system specification document identifying the KC-767A aircraft configuration. The notional aircraft is configured as a combination aerial refueling, cargo, and passenger aircraft designed to permit both freight hauling and passenger transport in the same mission. We provide a detailed description of the notional aircraft in appendix I. To obtain a lower negotiated price on the KC-767A aircraft, the Air Force has changed the specifications to eliminate the requirement that the aircraft be capable of passenger transport and cargo hauling on the same flight while retaining the other planned combi modifications. Air Force and Office of Management and Budget Comments on the IDA Study: Both the Air Force and Office of Management and Budget acknowledge that the IDA study was a useful and valuable tool used in negotiations with Boeing and believe it assisted the Air Force in negotiating a lower price for the 100 KC-767A aircraft leasing package. However, the Air Force believes that the IDA purchase price estimate did not have the "fidelity" or accuracy of its negotiation position with Boeing. Air Force officials told us that any line-by-line comparison of what individual items should cost between the IDA study results and those negotiated by the Air Force and Boeing is of limited value because the contract negotiation price was for 100 aircraft and each party would undoubtedly divide the pieces differently. However, the Air Force did not provide sufficient evidence to explain how negotiating for 100 aircraft would necessarily lead to a higher price than the average unit price for 100 aircraft given that IDA did not assume that a volume discount would be given. In addition, according to the Air Force, even if the IDA estimate proved correct and the price per aircraft were lower than the negotiated price, the government would be protected. Air Force officials pointed out that the contract as currently proposed includes a best price guarantee and a return-on-sales cap that enables the government to receive an equitable adjustment if Boeing's profit exceeds the cap. While the draft contract does include a "best price guarantee," this provision only guarantees that The Boeing Company will not sell comparable KC-767A aircraft for less than the Air Force would pay but it does not address the question of whether the Air Force could have obtained a lower price. Moreover, as we testified before the Senate Committees on Armed Services, and Commerce, Science and Transportation,[Footnote 11] it is not clear to us why the sales cap is 15 percent as negotiated, when a financial analysis has concluded that Boeing's profit on commercial 767 aircraft is in the range of 6 percent.[Footnote 12] The Air Force also raised several other concerns with the IDA estimate surrounding different acquisition strategies used, different bases for estimates, etc. However, after review, we considered most of the differences to be inherent in the nature of IDA's tasking and not attributable to significant discrepancies or voids in information. For example, the IDA was directed to consider only the cost of a direct purchase price, not a leasing arrangement, so the IDA did assume a different acquisition strategy than the Air Force. However, the Air Force did not provide sufficient evidence to explain how a different acquisition strategy would necessarily lead to a different purchase price for the aircraft. Moreover, the cost estimates used by IDA to develop its purchase prices are applicable and also used by DOD cost estimators for both firm fixed price and cost contracts, and the planned lease of KC-767A aircraft is to be done under a firm fixed price contract. Agency Comments and Our Evaluation: In oral comments on a draft of this correspondence, representatives from the Air Force did not disagree with our analysis or our conclusions. However, these officials believe that they negotiated a reasonable price for the aircraft as planned to be configured by the Air Force and in the quantity to be delivered, and that the IDA estimate of $120.7 million per aircraft was not achievable. We do not know whether such a price is achievable because such analysis was outside the scope of our work. In addition, officials from the Office of the Secretary of Defense and IDA generally concurred with our analysis and our report. We conducted this work from September to October 2003 in accordance with generally accepted government auditing standards. Unless you announce its contents earlier, we plan no further distribution of this letter until 10 days from its issue date. At that time, we will send copies of this letter to the chairman and ranking member of the Committee on Armed Services, House of Representatives, and the defense subcommittees of the Senate and House Committees on Appropriations. We will send a copy to the Chairman, Subcommittee on Readiness, House Committee on Armed Services, for whom we are conducting a broader body of work in this area. We will also send copies to the Secretary of Defense, the Director of the Office of Management and Budget, and the President of the Institute for Defense Analyses. We will also make copies available to other interested parties upon request. In addition, the letter will be available at no charge on the GAO Web site at http://www.gao.gov. We appreciate this opportunity to be of assistance. If you or your staffs have any questions regarding this letter, please contact me at (202) 512-4914 or Brian J. Lepore, Assistant Director, at (202) 512- 4523. Other key contributors to this review were Ann M. Dubois, Joseph J. Faley, Jennifer K. Echard, Kenneth W. Newell, Madhav S. Panwar, Charles W. Perdue, Kenneth E. Patton, and Tim F. Stone. Neal P. Curtin, Director Defense Capabilities and Management: Signed by Neal P. Curtin: Enclosure: [End of section] Enclosure I: Description of Proposed Aircraft as Assessed by IDA: The KC-767A tanker/combi aircraft (the aircraft can serve as an air- refueling tanker, can carry freight or passengers, or can combine freight and passengers in the same mission) is to be based on the commercial B767-200ER.[Footnote 13] Modifications would include the addition of features available on other Boeing 767 models, as well as changes required for the military application. In the tanker role, total fuel capacity is to be just over 200,000 pounds, including up to 41,000 pounds carried in added auxiliary fuel tanks. The KC-767A would have the capability to perform refueling by the hose/drogue and boom methods from the aircraft centerline and would also be able to receive fuel from other refueling aircraft. The cabin of the KC-767A would be convertible to three configurations. In the passenger configuration, the KC-767A would accommodate up to 190 passengers and 10 crewmembers. The freight configuration would accommodate 19 cargo pallets and 10 crewmembers. The combination (so-called "Combi") configuration would have the capacity for 10 pallets, 10 crewmembers, and 70 passengers, although this configuration has been dropped by the Air Force to reduce the price of the proposed aircraft. FOOTNOTES [1] Military Aircraft: Observations on the Proposed Lease of Aerial Refueling Aircraft by the Air Force. GAO-03-923T. Washington, D.C.: September 4, 2003. [2] Purchase Price Estimate for the KC-767A Tanker Aircraft. IDA Paper P-3800. Alexandria, Virginia.: July 2003. [3] These offices sponsored the IDA study. [4] Military Aircraft: Observations on DOD's Aerial Refueling Aircraft Acquisition Options. GAO-04-169R. Washington, D.C.: October 14, 2003. [5] Pub. L. No. 107-117, ยง 8159, 115 Stat. 2230, 2284-85. [6] A detailed description of these categories may be found in the IDA report page S-2. [7] The Boeing Company has asserted that a commercial market does exist for its KC-767A aircraft. [8] Coefficient of variation is a measure of dispersion. It is the ratio of the standard deviation to the arithmetic mean expressed as a percent. The smaller the number, the less the variation in the distribution and therefore the closer observations are to the mean. [9] "Hose and drogue" or "probe and drogue" and "centerline boom and receptacle" refer to different types of refueling equipment systems used to refuel different types of aircraft in flight. [10] A cost-estimating relationship is a cost function whose arguments are variables related to the performance of the items or to specific features of their designs. [11] Military Aircraft: Observations on the Air Force's Plan to Lease Aerial Refueling Aircraft. GAO-03-1143T. Washington, D.C.: September 3, 2003; and Military Aircraft: Observations on the Proposed Lease of Aerial Refueling Aircraft by the Air Force. GAO-03-923T. Washington, D.C.: September 4, 2003. [12] Morgan Stanley, Does 767 Tanker Equate to 700+ Comml Orders?, (May 30, 2003). [13] System specification for Air Force KC-767A tanker transport aircraft, Boeing Integrated Defense Systems, December 19, 2002.