This is the accessible text file for GAO report number GAO-09-189R entitled 'Defense Acquisitions: Status of the Safety, Performance, and Reliability of the Expeditionary Fire Support System' which was released on November 18, 2008. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. GAO-09-189R: United States Government Accountability Office: Washington, DC 20548: November 18, 2008: The Honorable Carl Levin: Chairman: Committee on Armed Services: United States Senate: Subject: Defense Acquisitions: Status of the Safety, Performance, and Reliability of the Expeditionary Fire Support System: The Expeditionary Fire Support System (EFSS)--which consists of two kinds of motorized vehicles, a 120-mm mortar, an ammunition trailer, and fire direction equipment--is being developed to meet the United States Marine Corps' need for a weapon system that can be carried inside the MV-22 Osprey to support assault operations. The Marine Corps Operational Test and Evaluation Activity (MCOTEA), the independent test agency for the Marines, conducted initial operational testing and evaluation of the EFSS from May to July 2007, and reported in September 2007, among other things, that it experienced several safety, performance, reliability, and mechanical problems. We briefed you on these and other issues related to the EFSS in September 2007. Subsequently, at your request, the Marine Corps delayed full-rate production of the EFSS until after GAO reported on the system. In December 2007, we issued our report,[Footnote 1] which described the system's safety, performance, reliability, and mechanical problems. MCOTEA retested the system in February and March 2008, focusing on determining whether the problems identified in 2007 were resolved. It reported its analysis of the test results in May 2008. In October 2008, you asked us to provide you with a brief assessment of the Marine Corps' conclusions regarding whether the concerns we reported have been addressed. To do so, we reviewed MCOTEA's May 2008 Independent Evaluation Report from the EFSS' Follow-on Operational Test and Evaluation as well as documentation of the system's insensitive munition certification[Footnote 2], and compared the results with the concerns we reported in 2007. We also reviewed documentation of EFSS' full-rate production decision. We interviewed EFSS program officials, a Marine Corps Combat Development Command official, and the MCOTEA official who oversaw EFSS testing to obtain their perspectives regarding whether and how the previously reported concerns were addressed. We conducted this performance audit from October 2008 through November 2008 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Results in Brief: Based on the May 2008 independent test report, most of the EFSS' safety, performance, reliability, and mechanical issues we reported in 2007 have been addressed through a combination of design changes and increased training. While some issues remain, MCOTEA did not judge them to be serious. The test report indicates that the EFSS' major safety issue we previously reported on--the system's inability to tow the ammunition trailer safely--has been addressed. The 2008 report also indicates that most of the performance issues we reported from the 2007 initial operational testing have been addressed. For example, follow-on testing showed that the system met all requirements associated with timed events, except the maximum rate of fire requirement, and resolved problems with the mortar's sight. Because the maximum rate of fire was frequently, but not consistently achieved, MCOTEA did not report it as a serious shortcoming and a Marine Corps requirements official stated that it was not a problem. EFSS vehicles are still not capable of securely carrying all required equipment, but Marine Corps officials attributed this problem to the space constraints imposed by the need to fit the system inside the V-22 Osprey, rather than to a design problem. The 2008 follow-on testing indicated that all of the reliability issues we reported in 2007 have been addressed, including issues related to the mortar's need for maintenance, transport barrel clamp weakness, and inconsistent vehicle configuration. The mechanical issues we reported on in 2007 appear to have been addressed, although not all of them could be directly tested during the 2008 follow-on test events. Although most of EFSS's earlier problems have been addressed, MCOTEA reiterated in its 2008 test report that the EFSS is a survivable platform provided it is used within its concept of employment and that employing the EFSS outside of the concept of employment would present a significant survivability liability to the operators given its limited protection. Background: The EFSS is designed to use its M327 120-mm rifled mortar to provide mobile fire support for expeditionary forces. The EFSS includes two kinds of prime mover vehicles and a trailer, in addition to the mortar. One vehicle tows the mortar, while the other tows the trailer that carries ammunition for the mortar. MCOTEA reported that the EFSS has a capability that is not currently fielded in the operating forces. It provides greater range and lethality than the mortars currently in use by the Marine Corps and it can be internally transported via assault support aircraft, in particular, the MV-22 Osprey. MCOTEA conducted initial and follow-on operational testing and evaluation for the EFSS to ensure that the system meets its mission essential tasks as well as the criteria of Operational Effectiveness and Operational Suitability by providing effective fire support and proving its reliability, availability, and maintainability. The Initial Operational Test and Evaluation, which we reported on in 2007, was conducted in three phases: amphibious, desert, and high-altitude operations. Follow-on testing was conducted in two phases: cold weather at the Cold Regions Test Center, Fort Greely, Alaska, from February 17, 2008, through March 2, 2008, and desert terrain at the Marine Corps Air Ground Combat Center, Twentynine Palms, California, from March 6-9, 2008. Data from both the initial testing and the follow-on testing were used by the Marine Corps Operational Test and Evaluation Activity to prepare the May 2008 Independent Evaluation Report for the Follow-on Operational Test and Evaluation. On June 2, 2008, the Marine Corps Systems Command approved the EFSS program to move forward with full-rate production of the system. In addition, the Command approved full fielding of the system once certain conditions have been met.[Footnote 3] The program office reported that the request for proposal for full-rate production for fiscal year 2009 has been issued, but they have not yet entered into a contract. Most EFSS Safety, Performance, Reliability, and Mechanical Issues Identified in 2007 Have Been Addressed: In the 2008 follow-on testing, MCOTEA determined that the EFSS is operationally effective and operationally suitable. The test report indicates that most of the safety, performance, and reliability issues identified in 2007 initial operational testing have been addressed. Substantial design changes were made to the system in response to issues identified in initial operational testing and the test unit participants received substantially more training in the use of the system than in previous tests. The Operational Test Project Officer stated that the prime mover was "like a completely different vehicle" and the director of the follow-on testing reported that the system "proved it can operate in extreme conditions with close to normal response times." Although substantial improvements have been made to the EFSS, it remains critical that the system be employed within its concept of employment in order to be survivable. Table 1 summarizes the status of these issues based on the 2008 testing. Table 1: Comparison of Selected Results from 2007 and 2008 EFSS Testing: Issue: Safety; Factors Limiting EFSS Capability as Reported by GAO in 2007: Ammunition trailer risked injury to crewmember; Status Based on 2008 Test Results: Addressed. Issue: Performance; Factors Limiting EFSS Capability as Reported by GAO in 2007: Measures associated with timed events were not met, including: rate of fire, first round response, shift out of traverse, emplacement, and displacement; Status Based on 2008 Test Results: Addressed except maximum rate of fire was not consistently met. Issue: Performance; Factors Limiting EFSS Capability as Reported by GAO in 2007: Mortar sight difficult to use; Status Based on 2008 Test Results: Addressed[A]. Issue: Performance; Factors Limiting EFSS Capability as Reported by GAO in 2007: Shock mount problems; Status Based on 2008 Test Results: Minor problems persist. Issue: Performance; Factors Limiting EFSS Capability as Reported by GAO in 2007: Vehicles carry required equipment, but not securely; Status Based on 2008 Test Results: Not addressed, but officials report this is due to the vehicles' size requirements. Issue: Reliability; Factors Limiting EFSS Capability as Reported by GAO in 2007: The mortar's mean rounds between operational mission failure did not meet the threshold; Status Based on 2008 Test Results: Addressed. Issue: Reliability; Factors Limiting EFSS Capability as Reported by GAO in 2007: Transport barrel clamps contributed to brake damage; Status Based on 2008 Test Results: Addressed. Issue: Reliability; Factors Limiting EFSS Capability as Reported by GAO in 2007: Vehicle physical configuration not considered to be production representative; Status Based on 2008 Test Results: Addressed. Issue: Mechanical failures; Factors Limiting EFSS Capability as Reported by GAO in 2007: Problems with Central Tire Inflation System; Status Based on 2008 Test Results: Addressed. Issue: Mechanical failures; Factors Limiting EFSS Capability as Reported by GAO in 2007: Cooling system insufficient during operations; Status Based on 2008 Test Results: Considered by MCOTEA to be addressed[B]. Issue: Mechanical failures; Factors Limiting EFSS Capability as Reported by GAO in 2007: Vehicles had difficulty starting at high altitude; Status Based on 2008 Test Results: Considered by MCOTEA to be addressed[B]. Source: GAO analysis based on Marine Corps Operational Test and Evaluation Activity's Expeditionary Fire Support System Follow-on Test and Evaluation Independent Evaluation Report (May 2008) and GAO, Defense Acquisitions: Status of the Expeditionary Fire Support System, GAO-08-331R (Washington, D.C., Dec. 21, 2007). [A] MCOTEA reported that the operators could use the mortar sight without significant problem, but the mortar does require additional training for the operators. [B] Although the EFSS was not retested under these conditions in follow- on testing, the Operational Test Project Officer stated that he was satisfied with the changes because testing for a system with the same cooling and starting components resulted in satisfactory operation in high heat and at an elevation of over 8,000 feet. [End of table] Safety: The 2008 follow-on test report indicates that the EFSS's major safety issue identified in initial operational testing--the system's inability to tow the ammunition trailer safely, which posed a risk of injury to the crewmember riding in rear seat of vehicle--has been addressed. Three incidents involving the trailer occurred during the 2007 test events. Prior to the 2008 follow-on test design changes were made, including lengthening the tongue on the ammunition trailer to increase the trailer's stability, redesigning the ammunition trailer's brakes, and lowering tire pressure. In addition, a cage and roll bar were installed on the rear of the prime mover to protect the Marine riding in the third seat and safe towing speeds were emphasized during testing. The test report indicated that throughout the follow-on testing, the Marines operated the prime mover and trailer in accordance with training standards and no incidents occurred to indicate the trailer would pose a hazard to the operators. In addition, in 2007 we reported that the EFSS mortar had not yet completed insensitive munition certification--a certification that a munition will not detonate under any condition other than its intended mission. The program office reported and provided documentation in October 2008, that this certification is now complete. Performance: The report from follow-on testing indicates that most of the performance issues identified in initial operational testing have been addressed. In follow-on testing, the EFSS successfully and consistently met requirements for almost all timed events that MCOTEA retested due to performance deficiencies in initial operational testing. Marine Corps officials attributed this improvement primarily to more thorough training and the test unit's higher level of performance. During initial operational testing, the EFSS did not meet the critical requirement related to maximum rate of fire. It also did not meet some non-critical requirements for timed events, which involved sustained rate of fire, first-round response, shift out of traverse, and emplacement and displacement. Although required times were achieved on occasion for these events, the preponderance of observed times took longer than required. These events were all retested in 2008, during which the test unit successfully and consistently performed those timed events except for the events related to rate of fire. MCOTEA assessed the sustained rate of fire requirement as met.[Footnote 4] However, the maximum rate of fire critical requirement was not met. Although the test unit frequently achieved the maximum rate of fire of 4 rounds per minute, the average for all trials was 3.5 rounds per minute. MCOTEA did not report the rate of fire results as a serious shortcoming and a Marine Corps requirements official stated that he did not see the system's maximum rate of fire as a problem because the maximum rate of fire was shown in testing to be achievable. The 2008 test report also indicates that performance issues associated with the mortar were largely addressed. Initial testing in 2007 found that the mortar sight was difficult for operators to use and its declinations were difficult to read in low light. The program office reported that, in order to help address these difficulties, the sight's eyepiece was lengthened and a better training package was developed to help users learn how to use the sight more effectively. MCOTEA reported in 2008 that operators could use the mortar sight without significant problems. In addition, in initial operational testing, the shock mount used to protect the sight from firing vibration caused issues with boresight verification. The shock mount was redesigned and officials stated that most of the problems associated with it were resolved. However, the test report indicated that the shock mount still posed some challenges with verification of the boresight. The EFSS program manager stated that a modification has been made to the shock mount latching mechanism to address the problems caused by the shock mount. The remaining performance issue is the limited storage space offered by the system, which may not be resolvable given the system's need to be small enough to be transported inside a V-22 tilt-rotor aircraft. Both the 2007 and 2008 test reports found that the EFSS vehicles were capable of carrying all required equipment, but not securely. The reports stated that section and personal equipment were loaded and carried haphazardly due to insufficient storage space. Because space inside the vehicle is limited, the Marines need to tie down their gear on a "gypsy rack" or on other parts of the prime movers. A Marine Corps requirements official stated that this limitation will be a challenge in terms of packing equipment, but several officials, including the Test Director stated that it is a function of the need to fit the system inside the V-22. Reliability: Follow-on testing indicated that all of the reliability issues identified in initial operational testing had been addressed. In initial operational testing, the mortar's mean rounds between operational mission failure did not meet the threshold. However, in follow-on testing the achieved availability for the launcher was met, meaning it did not experience a large amount of down time for maintenance relative to the amount of operating time. In addition, initial operational testing indicated that the EFSS' transport barrel clamps were not stout enough to prevent its mortar tube from rotating, which contributed to brake damage on one mortar during movement. This issue has been addressed--the transport barrel clamp has been augmented with a stop block and follow-on testing indicates that the augmentation effectively prevents the mortar tube from rotating during travel. Lastly, initial operational testing showed that each prime mover vehicle's physical configuration was different--the vehicles were not considered to be production representative. Spare and repair parts were also nonstandard, and the test team could not interchange parts between vehicles on several occasions. MCOTEA's EFSS Operational Test Project Officer stated that in follow-on testing, the vehicles' physical configuration was consistent and the system appeared to be production representative. Although substandard spare and repair parts continued to be an issue in follow-on testing, the program office stated that a complete spare parts package will be produced and will undergo quality inspection for delivery with the full-rate production systems. Mechanical Issues: The mechanical issues we reported on in 2007 appear to have been addressed, although not all of them could be directly tested during the 2008 follow-on test events. Initial testing indicated that the compressor for the air ride system and Central Tire Inflation System was not robust enough to support all the air-powered systems on the vehicle. However, for follow-on testing, the system's operators were properly trained to use the systems, which operated satisfactorily. Substantial engineering changes were made in response to other mechanical issues raised in initial operational testing and the Operational Test Project Officer indicated he was satisfied with those changes. The 2007 testing indicated that EFSS' cooling system was not able to cool the engine and transmission sufficiently during operations. In addition, the same tests indicated that the vehicles did not start reliably in the mornings during high-altitude operations. To address these and other issues, the radiator and cooling system have been redesigned and expanded. The program office described these changes as significant. No cooling issues occurred during follow-on testing although desert temperatures were not considered high enough to challenge the system. The Operational Test Project Officer indicated that he was satisfied with the changes because testing for another Marine Corps system with the same cooling system resulted in satisfactory operation in high heat and at an elevation of over 8,000 feet. Other Considerations: The Marine Corps' 2008 test report pointed out that the EFSS would enhance the capabilities of the Marine Corps' artillery community because it has a capability not currently fielded in the operating forces, but stated that the platform is only survivable[Footnote 5] when used within its concept of employment. MCOTEA reported that the system provides greater range and lethality than the 60 and 81 millimeter mortars currently in use and the entire EFSS can be internally transported via assault support aircraft, particularly the MV-22 Osprey. The EFSS's 2008 follow-on test report, like the initial test report, indicated that the EFSS is a survivable platform provided it is used within its concept of employment. The EFSS concept of employment indicates that the EFSS will be flown in via MV-22 Osprey or helicopter. Once on the ground, most Marines would travel by foot, with the EFSS accompanying them at comparable speeds and at an equivalent distance. The system's concept of employment does not suggest employment in areas with asymmetric threats such as the urban areas of Iraq or Afghanistan. Both test reports stated that employing the EFSS outside of the concept of employment would present a significant survivability liability to the operators given its limited protection. Agency Comments and Our Evaluation: We provided a draft of this report to Department of Defense officials for review and comment and they chose to provide only technical comments, which were incorporated as appropriate. We are sending copies of this report to the Secretary of Defense, the Commandant of the Marine Corps, and interested congressional committees. We will also make copies available to others upon request. In addition, the report will be available at no charge on GAO's Web site at [hyperlink, http://www.gao.gov]. If you have any questions concerning this report, please contact me at (202) 512-4841. Key contributors to this assignment are David Best, Laura Holliday, and Greg Campbell. Signed by: Paul Francis, Director: Acquisition and Sourcing Management: [End of section] Footnotes: [1] GAO, Defense Acquisitions: Status of the Expeditionary Fire Support System, [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-08-331R] (Washington, D.C., Dec. 21, 2007). [2] Certification that a munition will not detonate under any condition other than its intended mission. [3] These conditions include: verification that safety testing has been successfully completed and that manuals and training protocols address all necessary operating cautions and safety procedures, validation of the production process for ammunition manufacturing, and certification that support equipment and special tools have been provided and required technical manuals have been made available. [4] Sustained rate of fire was not directly evaluated because the entire basic load of ammunition was not fired--the assessment of the sustained rate of fire was derived from the maximum rate of fire evaluation. [5] According to DOD, survivability is a concept that includes all aspects of protecting personnel, weapons, and supplies while simultaneously deceiving the enemy. [End of section] GAO's Mission: The Government Accountability Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each weekday, GAO posts newly released reports, testimony, and correspondence on its Web site. To have GAO e-mail you a list of newly posted products every afternoon, go to [hyperlink, http://www.gao.gov] and select "E-mail Updates." Order by Phone: The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO’s Web site, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: E-mail: fraudnet@gao.gov: Automated answering system: (800) 424-5454 or (202) 512-7470: Congressional Relations: Ralph Dawn, Managing Director, dawnr@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, D.C. 20548: Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, D.C. 20548: