Millennium Challenge Corporation: Independent Reviews and Consistent Approaches Will Strengthen Projections of Program Impact

GAO-08-730 June 17, 2008
Highlights Page (PDF)   Full Report (PDF, 46 pages)   Accessible Text   Recommendations (HTML)

Summary

In January 2004, Congress established the Millennium Challenge Corporation (MCC) for foreign assistance. Eligible countries submit compact proposals for MCC funding for projects aimed at reducing poverty through economic growth. To assess the proposed compacts' likely impact, MCC performs economic analyses estimating the compacts' economic rate of return (ERR) and effects on income and poverty as well as the number of compact beneficiaries. MCC uses these analyses to inform its decisions to fund proposed compacts and to inform Congress and the public about its progress in achieving its mission of poverty reduction through economic growth. GAO was asked to examine MCC's projections of (1) ERR and (2) compacts' impact on income and poverty as well as numbers of beneficiaries. GAO reviewed MCC's stated impacts and analyses for four MCC compacts that represented 41 percent of MCC's compact assistance and met with MCC officials.

MCC used different time frames and methods to calculate ERRs for its compacts with Armenia, El Salvador, Lesotho, and Mozambique. In calculating ERR for 20 projects within the compacts, MCC used a 20-year time frame for 9 projects and used different time frames for the other 11. In 2 of the 11 projects, using a 20-year time frame, as MCC used for similar projects, would reduce the ERR below the level MCC set as the minimum acceptable ERR. At the compact level, MCC's use of varying time frames did not affect the ERRs significantly. MCC used varying methods to account for the costs of same-sector projects, although its approaches to determining project benefits were generally similar. MCC also used two different methods to calculate compact-level ERR; however, the choice of method did not reduce it below the minimum ERR. In three of the four compacts that we reviewed, MCC did not retain documentation of the economic analyses used to support the investment decision, but continued to modify the analyses. MCC has recently begun to standardize elements of its economic analyses and centralize its records management. MCC identified and corrected analytic errors in its projections of impact on income and poverty. MCC also used varying methods to project these impacts. In responding to GAO's questions about its published projections of impact on income and poverty, MCC identified analytic errors for three of the four compacts and, in correcting these errors, generally lowered the projected impacts. Correcting these errors raised one projection by 5 percent but reduced others by 3 percent to 96 percent. According to MCC officials, the revised projections would not have affected MCC's decision to recommend signing the compacts. The officials noted that, in the future, compact impact projections will undergo peer review. However, MCC has not documented procedures for these reviews. MCC used varying methods for its projections of impact on income and poverty, limiting comparability and replicability across compacts. To project impact on income for Armenia, El Salvador, and Mozambique, MCC estimated the compacts' impact by summing the total benefits of individual compact projects and adding them to the income that would have prevailed without MCC. However, for Lesotho, MCC estimated the impact on income based on the published results of a World Bank model based on elements different from those of the MCC compact. In response to our questions, MCC revised its initial estimate of the effect of income growth on poverty for Mozambique by presenting two estimates, based on Mozambique-specific and cross-country data, respectively. Although a number of methods for projecting poverty impact are valid, the method chosen can affect the results, and MCC's guidelines do not identify preferred methods for these calculations. MCC also used varying methods to estimate numbers of beneficiaries for the compacts and has not provided specific criteria for defining beneficiaries; however, MCC officials reported they are taking steps to provide more detailed guidance.



Recommendations

Our recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Implemented" or "Not implemented" based on our follow up work.

Director:
Team:
Phone:
David B. Gootnick
Government Accountability Office: International Affairs and Trade
No phone on record


Recommendations for Executive Action


Recommendation: To improve the reliability and comparability of its projected ERR and economic impacts, the CEO of MCC should adopt and implement written procedures for a secondary independent review of the methods and results of its economic analyses.

Agency Affected: Millennium Challenge Corporation

Status: In process

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: To improve the reliability and comparability of its projected ERR and economic impacts, the CEO of MCC should improve MCC's guidelines by identifying a consistent approach with preferred methods for projecting compacts' impact on income and poverty.

Agency Affected: Millennium Challenge Corporation

Status: In process

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.