Detailed Information on the
US Agency for International Development Climate Change Program Assessment

GCC team procurement for FY05 provided new funding for CCTP, CCSP, and Methane to Markets initiatives. USAID issued an Annual Program Statement (a new funding solicitation) for the Climate Change Team for FY06, which prioritizes support for Presidential initiatives. The GCC team also revised its reporting tool to improve data collection from Missions. A plan to address PART concerns will be finalized in FY06; as Agency Strategies are formalized for relevant initiatives.

Focusing funding on priority areas in the short term: specifically, the high priority geographic and programmatic areas that would support the Administration's climate negotiating team.

A plan to improve direction and review of the Agency's GCC program was submitted to EGAT Senior Management in 2006. The plan was put on hold pending completion of the new Foreign Assistance Framework and budgeting. The environment technical working group recommended review of the GCC, Clean Energy and Biodiversity programs as part of the future OP process.

Measure: Total area (hectares) where AID is acting to maintain or increase carbon stocks or reduce their rate of loss (in millions)

Explanation:

Measure: Annual emissions of carbon dioxide equivalents (million metric tons) avoided due to USAID assistance

Explanation:

Measure: Dollars per ton of carbon dioxide equivalents (CO2eq) avoided or reduced across the Program

Explanation:This measure is calculated by dividing the annual total funds attributed to USAID's Global Climate Change Program by the estimated total amount of greenhouse gas emissions reduced or sequestered as a result of activities attributed to that program. There are many types of greenhouse gasses, but we use accepted equations for converting them into the common measure of carbon dioxide equivalents (CO2eq). Dollars/ton CO2eq is an efficiency measure because it shows how many dollars are required to achieve one unit of CO2 emissions reductions or sequestration. However, 98% of the activities that are attributed to the Global Climate Change Program are implemented by the field, are not under the management authority of the Global Climate Change Team, and are seeking other objectives in addition to CO2eq. These factors affect the achievement of greater efficiencies as measured by this indicator.

Is the program purpose clear?

Explanation: AID's Climate Change Program's mission statement is to promote sustainable development that minimizes the associated growth in greenhouse gas emissions and reduces vulnerability to climate change. The program supports activities to decrease the rate of growth in net greenhouse gas (GHG) emissions by decreasing GHG sources and maintaining or increasing GHG sinks; increase developing and transition country participation in the UN Framework Convention on Climate Change (UNFCCC); and decrease developing and transition country vulnerability to the threats posed by climate change.

Evidence: Mission statement and program objectives are outlined in AID's 1998 publication entitled "Climate Change Initiative, 1998-2002." This report framed the climate change issue and defined AID's strategy. AID has not yet completed a new strategy for fiscal year 2003 and beyond. In 1990, Congress directed AID to engage in climate change activities by focusing its global warming activities in "specific key countries which stand to contribute significantly to GHG emissions, and in which actions to promote energy efficiency, reliance on renewable energy resources, and conservation of forest resources, could significantly reduce emissions of greenhouse gases" (P.L. 101-167, section 534).

Does the program address a specific interest, problem or need?

Explanation: AID works with developing and transition countries to incorporate climate change considerations into their economic growth plans. Following U.S. treaty obligations and Bush Administration policy, AID funds projects that help developing countries reduce GHG emissions and avoid and/or adapt to projected global climate changes that may negatively affect them.

Evidence: In 1992, the U.S. ratified the UN Framework Convention on Climate Change (UNFCCC). The treaty's objective is "to achieve??stabilization of greenhouse gas concentrations in the atmosphere at a level that would prevent dangerous anthropogenic interference with the climate system??to ensure food production is not threatened and to enable economic development to proceed in a sustainable manner." The Intergovernmental Panel on Climate Change (IPCC), an international body of 2,500 of the world's leading scientists and technical experts also outline the need to address both sustainable development and climate change in the publication, "Climate Change 2001: Impacts, Adaptation, and Vulnerability; IPCC Working Group II Third Assessment." "U.S. Climate Change Strategy A New Approach" White House, February 14, 2002

Is the program designed to have a significant impact in addressing the interest, problem or need?

Explanation: A case study of the climate change program in Philippines concluded that "Conventional AID environmental programs (forestry and land use, energy technology and management, and urban and industrial pollution) can yield significant ancillary climate change benefits." The program is implemented through regional programs in Central Africa, Central America, and Central Asia, as well as through bilateral assistance to thirty-five countries throughout Africa, Asia, Latin America, Eastern Europe and Eurasia. Across all regions and countries, AID places strong emphasis on leveraging the capabilities and resources of other donors, the private sector, non-governmental organizations (NGO's), and communities.

Evidence: "AID Programs that Respond to the Global Climate Change: Philippines Case Study;" CDIE, DRAFT. To enhance the impact of its activities, the program leverages funding from other sources. While it is difficult to calculate an exact figure that measures AID's ability to leverage funding from other donors and institutions, the Agency nevertheless seeks to assess its overall impact in promoting innovative approaches and activities that address climate change with its partners. In FY 2001, based on data collected from all countries participating in the program, AID leveraged over $1.07 billion, multiplying its investment by almost seven times the amount it obligated for climate-related activities in that year, $156 million.

Is the program designed to make a unique contribution in addressing the interest, problem or need (i.e., not needlessly redundant of any other Federal, state, local or private efforts)?

Explanation: The AID climate change program is the primary vehicle for U.S. assistance to developing countries on climate change. It is comprehensive, not needlessly redundant, and it has more funding than other existing federal programs. However, there are several agencies that have programs with similar goals (DOE, EPA, NOAA, USDA, Treasury). In addition, all parties to the UNFCCC and several private institutions share the stated aim of promoting sustainable development and reducing the intensity of GHG emissions in developing and transitioning economies (World Bank, donor nation development assistance agencies, Global Environment Fund, U.N. Environment Program). AID's challenge is to coordinate its programs with the many federal and non-federal programs conducting similar work.

Evidence: The UNFCC has been ratified by more than 190 nations including many developing nations. World Bank, Global Environment Fund, United Nations Development Program, United Nations Environmental Program and numerous OECD countries have GHG emission reduction, adaptation to climate change, and sustainable economic development as stated goals. EPA is engaged in activities within developing countries and countries undergoing economic transition that address both air quality improvement and climate change mitigation. Other Federal Agencies initiate and develop projects independently of AID. Also, AID collaborates with other US and international government agencies, and the private an non-profit sectors. AID's Climate Change Initiative report acknowledges the contributions of multilateral and bilateral donors, international organizations, the private sector, and other agencies and the departments of the U.S. government.

Is the program optimally designed to address the interest, problem or need?

Explanation: The program seeks to promote sustainable development and address climate change by taking account of climate change concerns in a broad range of development assistance activities. The program is not designed to target only climate change mitigation and adaptation projects; therefore, the climate change objectives are not maximized. However, given that many developing countries that AID works with have not made specific commitments to reduce or control GHG's, it is not feasible to fully maximize climate benefits. AID focuses funding on recipient countries priority sustainable development goals while "building in" climate change benefits. Any AID development assistance project that AID funds that has a corollary climate benefit is counted as part of AID's climate program. AID implements only a small amount of climate-specific funding. AID and many other donors followed the "win-win" approach, whereby funding and policy decisions have primarily focused on traditional development outcomes while capturing climate impacts.

Evidence: AID mainstreams climate change concerns throughout the Agency and with in-country partners by educating participants on the climate-related benefits of long-standing and new development assistance programs. The "win-win" approach is supported by academic work included in the Assessment Report of the Intergovernmental Panel on Climate Change (see evidence for cite) which finds that "Activities requited for enhancement of adaptive capacity [to climate change] are essentially equivalent to those promoting sustainable development." Publication: "Climate Change 2001: Impacts, Adaptation, and Vulnerability". Published for the Intergovernmental Panel on Climate Change. Cambridge University Press.

Does the program have a limited number of specific, ambitious long-term performance goals that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: AID needs to develop new long term goals and indicators that focus on measureable outcomes. The overarching long-term performance goals are clear and AID set proxy measures in 1998 on an annual basis. This question is answered "Yes" because one of the indicators is specific, measurable and had a cumulative target linked to a an outcome (hectares where AID funding increased carbon stocks or reduced their rate of loss). However, another indicator (achieve 300 capacity building activities) is not linked to outcomes at all. The long term goals are: (1) Decrease the rate of growth in net GHG emissions by decreasing GHG sources and maintaining or increasing GHG sinks. (AID has not developed a direct measure for this goal.); (2) Increase developing and transition country participation in the UN Framework Convention on Climate Change (UNFCCC). (Indicator, number of capacity building activities, is poor.); (3) Decrease developing and transition country vulnerability to the threats posed by climate change (No indicator developed).

Evidence: The original long-term goals of the program are outlined in the published document "Climate Change Initiative, 1998-2002." The Performance Monitoring Plan set the annual indicators and targets. One of the goals has a long-term cumulative target for the 5-year period. The program needs to measure climate outcomes and set long term targets that can provide a basis for annual program and budget decisions. The program has begun a review of the 1998-2002 program.

Does the program have a limited number of annual performance goals that demonstrate progress toward achieving the long-term goals?

Explanation: In 1998, baselines and annual targets for the following three indicators were established in a Performance Monitoring Plan (PMP) as proxy measures of the long term goals: 1) area where AID has initiated interventions to maintain or increase carbon stocks or reduce their rate of loss, 2) emissions of carbon dioxide equivalents avoided, due to AID assistance (measuring carbon dioxide, methane, and nitrous oxide, and 3) institutions strengthened in support of the Framework Convention on Climate Change, in the land use/forestry sector, energy sector, industry, or urban areas. Annual results are calculated and compared to the PMP targets for each respective year, as well as to the previous year's performance. This tracking system measures progress toward achieving the longer term goals of the program. The PMP does not capture the program's progress with respect to the third long-term goal, "decreasing developing and transition country vulnerability to the threats posed by climate change," as no quantifiable standard for determining baselines or measuring progress had been established.

Evidence: AID publication: "AID Climate Change Program Special Objective 1: Agency Climate Change Program Effectively Implemented Performance Monitoring Plan (draft), Global Environment Center AID. April 19, 2000

Do all partners (grantees, sub-grantees, contractors, etc.) support program planning efforts by committing to the annual and/or long-term goals of the program?

Explanation: Partners under contract with AID's operating units are required to report results on specific indicators. Under AID's "win-win" approach to climate change operating units do not establish strategic objectives to specifically address climate change. However, the Agency has established uniform criteria for measuring the extent to which its programs are contributing to its long-term climate change goals, and has required mandatory annual reporting by all operating units who are implementing climate-related activities. As activities are planned and implemented, contractors are instructed on how to use these standard indicators for measuring and reporting climate-related benefits.

Evidence: Each year, the Climate Change Team in Washington sends guidance and indicator reporting forms to operating units to collect information about the climate change activities they support with their partners. The team manages, aggregates, and analyzes the information through a central database. AID operating units and their cooperating partners sign a Letter of Understanding (LOU) outlining their interests and planned cooperation toward the strategic objective of the operating unit. In addition, it is standard language in both cooperative agreements and grant agreements with AID for the cooperating partner to submit regular progress reports.

Does the program collaborate and coordinate effectively with related programs that share similar goals and objectives?

Explanation: The Agency coordinates with other Federal agencies that have similar international programs. For example, AID makes use of expertise within EPA, DOE, USDA and often enters into cooperative agreements with NGOs to support their programs that meet USG objectives.

Evidence: One example is AID's support of the US Climate Technology Cooperation (US-CTC), an interagency initiative of DOE, EPA, State, and AID, that encourages climate-friendly technologies and practices. Under US-CTC, AID has co-sponsored the Climate Technology Partnership (CTP), for technical assistance to facilitate the use of clean energy and renewable energy technologies. For example, AID's support focused technology transfer efforts in Mexico, Philippines, South Africa, Brazil, and Egypt, while EPA focused efforts in China and Korea, countries where AID does not have authority to work.

Are independent and quality evaluations of sufficient scope conducted on a regular basis or as needed to fill gaps in performance information to support program improvements and evaluate effectiveness?

Explanation: Current evaluations of program performance are internal AID reports. A case study of AID programs that respond to global climate change in the Philippines was conducted by the Center for Development Information and Evaluation (CDIE) this year. CDIE also did an overview of the the program in 2001. Also, Each year, the AID Washington-based Global Climate Change (GCC) Team aggregates and evaluates performance data submitted by more than 35 operating units. AID considers the GCC to be independent and non-biased. Gaps in performance from year to year are identified and communicated to the appropriate AID operating unit which manages the in-country climate-related activities in question. However, the central GCC Team does not have the ability to link climate-related performance with budgeting for these individual operating units. The Team makes technical suggestions for specific program improvement, but cannot fully control to what degree those suggestions are budgeted for and acted upon.

Evidence: 1) "AID Programs that Respond to the Challenge of Global Climate Change: An Experience Review; CDIE; October 2001" and 2) "AID Programs that Respond to the Global Climate Change: Philippines Case Study;" CDIE, DRAFT.

Is the program budget aligned with the program goals in such a way that the impact of funding, policy, and legislative changes on performance is readily known?

Explanation: AID programs its activities based on annual funding targets. At the end of the year, the level of funding to programs with climate benefit is assessed. AID has exceeded its annual budget target for climate change obligations since the inception of the Climate Change Initiative in FY98.

Evidence: AID staff points out that while the annual program budget is not tied to performance goals, the impact of changes in funding, policy, and legislation on the program volume can be determined. The impact of changes on performance are not known.

Has the program taken meaningful steps to address its strategic planning deficiencies?

Explanation: The effort has begun, but has not affected planning yet. The Climate Change Team started developing a new strategic plan for the GCC program in March 2001. Washington and field-based staff discussed achievements, lessons learned, and ideas for future direction of the program. Information from the workshop and US climate change policy, announced in February 2002, formed the basis for a draft Concept Paper in May 2002. Review of the strategic priorities is currently underway.

Evidence: Workshop report, "Recommendations and Conclusions from the AID Climate Change Training Workshop 2001," and Concept Paper, "Global Climate Change and AID."

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: AID regulations require recipients of funding to manage, monitor, and report project performance information on a regular basis. Operating units collect performance reports from recipients on a quarterly, semi-annual, or annual basis. The Agency's Annual Reporting system collects program performance data. The first year of climate change reporting helped to form the baseline for targets in the Performance Monitoring Plan (PMP).

Evidence: AID abides by 22 CFR 226, "Administration of Assistance Awards to US Non-governmental Organizations", section 226.51, "Monitoring and reporting program performance" to require partners to provide performance reports. In addition, the Climate Change Team houses information collected through indicator reporting by operating units in a database designed specifically for this purpose. FY 2001 indicator reporting documents.

Are Federal managers and program partners (grantees, subgrantees, contractors, etc.) held accountable for cost, schedule and performance results?

Explanation: In the Agency personnel evaluation process, project managers are evaluated for effective management of agreements; staff who serve as point-persons for country programs are assessed according to their contributions toward a mission's strategic objective. Input is sought from supervisors, peers, and clients on performance concerning these criteria as part of the annual evaluation process.

Evidence: Work objectives in employee work plans are tied to the employee's role in the organization. Among other areas, project managers are evaluated against the quality of technical guidance provided to contractors and grantees and their management of financial and human resources. Additionally, staff are assessed against criteria appropriate to the stage of the country program; for example, contribution to a new country strategy or assistance in ensuring that country resources are appropriately allocated to agreements managed by AID/W if the mission chooses to program its funds that way. Project pipelines factor into annual budget decisions.

Are all funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: AID obligates all its climate funds in the same year the funds are made available. The Washington office issues guidance annually on the appropriate use of climate funds to help ensure that funds are spent for their intended purpose. Additionally, the Climate Change Team and PPC jointly review obligation data to ensure the integrity of the data before it is submitted to Congress. The Agency regularly audits contracts and grants to assure the funds are spent for the intended purpose.

Evidence: AID obligation data is provided annually in the Congressionally mandated Federal Report to Congress on Climate Change Expenditures.

Does the program have incentives and procedures (e.g., competitive sourcing/cost comparisons, IT improvements) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: Efficiency and productivity gains are not generally part of contracts and grant agreements. However, the Agency implements much of its program through competitively-awarded mechanisms, and usually makes awards on the basis of "best value" to the government; has empowered its frontline managers to make program decisions; has pursued increased efficiencies by consolidating projects; has focused on improving use of information technology.

Evidence: No evidence was provided that showed that incentives and procedures were in place.

Does the agency estimate and budget for the full annual costs of operating the program (including all administrative costs and allocated overhead) so that program performance changes are identified with changes in funding levels?

Explanation: Although the agency has made progress in the allocation of administrative costs, they are still unable to fully account for headquarters costs or allocate down to the individual program level. In addition, to date, the information has simply been used to highlight missions with either extraordinarily high or low operating expense to program ratios rather than as a direct factor in decisions regarding program funding.

Evidence: Status Report on Operating Expenses, Managerial Cost Accounting Report ( previews/summary pieces) The finals should be available by the end of the month.

Does the program use strong financial management practices?

Explanation: Although the agency has made progress in achieving a qualified financial audit opinion for FY 2001, it continues to not meet basic Federal financial management requirements. In addition, it has not yet implemented a compliant financial management system for its field missions.

Evidence: AID Inspector General's audit report for FY 2001.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: Annual program guidance on appropriate use of climate funds is issued to help ensure the integrity of the program. Country budget targets are jointly established by PPC, the Climate Change Team, and the regional bureaus. At year end, country programs including performance are reviewed jointly by PPC and the Climate Change Team. To more effectively manage the program, the Climate Change Team has increased its level of staffing, increased communication with operating units in the field through regular email news updates and site visits, established an intra-agency team composed of Washington-based staff that meets regularly to facilitate improved understanding of the program.

Evidence: The AID Annual Report process collects program, performance and financial data on the Agency's Climate Change programs. Through this process, deficiencies (if any) are detected and addressed. In addition, in FY 2001, the management of the Economic Growth, Agriculture, and Trade (EGAT) pillar implemented a rigorous portfolio review of the centrally managed Environment and Climate Change programs.

Are grant applications independently reviewed based on clear criteria (rather than earmarked) and are awards made based on results of the peer review process?

Explanation: Agency funding is awarded through a competitive process with few exceptions. Applications are reviewed by technical panels and ranked according to how well they meet the selection criteria outlined in the request for applications. Most awards are evaluated based on the following broad criteria: program design and performance monitoring, institutional capacity and past performance, personnel capability and experience. Applicants are informed of the relative weight of each category and the elements within it that count toward the total score. Applicants' budget proposals are separately evaluated by AID's Office of Procurement. The program office also receives unsolicited proposals which are peer reviewed and awarded based on merit and how well they meet the needs of the operating unit's strategic objectives.

Evidence: In addition to following OMB Circulars (e.g., A-133, A-122, and A-110), AID has its own implementing regulations that are published in Automated Directive System (ADS) Chapter 303 for Grants and Cooperative Agreements to Non-Governmental Organizations.

Does the grant competition encourage the participation of new/first-time grantees through a fair and open application process?

Explanation: AID requests for applications are open and announced on the Agency website. In addition, in the near future, assistance opportunities will be announced on General Services Administration's electronic posting system, Federal Business Opportunities (www.FedBizOpps.gov). Unsolicited proposals are received at any time during the fiscal year from all interested parties. In addition, the Climate Change Team receives unsolicited presentations by potential first-time contractors as well as previous partners for new project ideas.

Evidence: The Agency tracks the number of new awards per year. In FY 2001, 45.6% of new assistance awards included new agreements, first time recipients, and repeat recipients.

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: AID operating units oversee activities of their cooperating partners through annual workplan reviews, management reviews, site visits, briefings, results reporting and financial reviews. The Climate Change Team in Washington relies on results reporting from operating units in the field to monitor and evaluate climate-related activities and expenditures to ensure that they meet the goals of the program.

Evidence: AID's Automated Directive System (ADS) requires Cognizant Technical Officers (CTOs) of cooperative agreements to "monitor and evaluate the recipient's performance [through site visits, reviewing and analyzing performance, assuring compliance with the terms and conditions of the award...]; evaluate...program effectiveness at the end of the program; and perform other duties as may be requested or as delegated by the Agreement Officer." CTO responsibilities for contract oversight are articulated in each award agreement.

Does the program collect performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: The Climate Change Program collects performance data on an annual basis and makes it available to the public.

Evidence: 2002 Report to Congress of Federal Climate Change Expenditures, AID's Annual Performance Report, AID website, Climate Change Initiative Annual Report.

Has the program demonstrated adequate progress in achieving its long-term outcome goal(s)?

Explanation: AID can report on the 5th year (2002) indicator targets and measures set in 1998. Only one of the indicators is measurable and has a cumulative target linked to an outcome (hectares where AID funding increased carbon stocks or reduced their rate of loss). The annual targets and indicators have not changed since the Climate Change Initiative program's inception in 1998. The targets for 2001 set in 1998 were so far exceeded that it raises the question whether the targets were rigorous enough and could have been reevaluated during the 5-year period. AID is currently evaluating the FY 2002 data.

Evidence: The original long-term goals of the program are outlined in the published document "Climate Change Initiative, 1998-2002." The Performance Monitoring Plan set the annual indicators and targets.

Does the program (including program partners) achieve its annual performance goals?

Explanation: Since 1998 when the Agency began tracking performance of its climate change program, it has consistently met its annual targets as outlined in the Performance Monitoring Plan. The Agency could not achieve this level of accomplishment without the cooperation of its program partners, who, as noted in Section II, question 3, contribute fully to the program's objectives and whose performance is evaluated on a regular basis. The annual indicators and results for FY 2001 are reported here.

Evidence: Results are reported annually in the following documents: 1) AID Annual Performance Plan; 2) Agency Annual Performance Report; 3) AID's annual submission to the Federal Climate Change Expenditures Report to Congress.

Does the program demonstrate improved efficiencies and cost effectiveness in achieving program goals each year?

Explanation: The program has not undergone an A-76 competition or other systematic way of improving contracting efficiencies aimed at reducing cost to achieve goals. Also, the central GCC Team does not have the ability to link climate-related performance with budgeting for individual operating units. The Team makes technical suggestions for specific program improvement, but cannot fully control to what degree those suggestions are budgeted for and acted upon.

Evidence: No evidence provided that would justify a yes answer.

Does the performance of this program compare favorably to other programs with similar purpose and goals?

Explanation: We were not able to find data evaluating and comparing the effectiveness of climate change programs.

Evidence:  

Do independent and quality evaluations of this program indicate that the program is effective and achieving results?

Explanation: Current evaluations of program performance are internal AID reports. A case study of AID programs that respond to global climate change in the Philippines was conducted by the Center for Development Information and Evaluation (CDIE) this year. The CDIE report "AID Programs that Reduce the threat of Global Climate Change An experience Review notes that "the Agency's programs affecting global climate change have registered significant outputs. However, there is little documentation on the impact of these programs." The Philippine Case study concluded that "Conventional AID environmental programs (forestry and land use, energy technology and management, and urban and industrial pollution) can yield significant ancillary climate change benefits." There is no independent analysis of the effectiveness of the entire program available.

Evidence: 1) "AID Programs that Respond to the Challenge of Global Climate Change: An Experience Review; CDIE; October 2001" and 2) "AID Programs that Respond to the Global Climate Change: Philippines Case Study;" CDIE, DRAFT.