Detailed Information on the
Development Assistance to Latin America and the Caribbean Assessment

Develop regional performance indicators for the remaining regional bureaus at the agency.

Continue efforts to strengthen budget and performance integration using the new agency-wide and regional performance data.

Continue to refine the analysis of this new performance data to broaden its applications for management decision-making at all levels of the agency.

Measure: Number of USG-supported anti-corruption measures.

Explanation:This indicator captures progress on a broad range of program approaches to fighting corruption. Anti-corruption measures may include new laws, regulations, procedures, consultative mechanisms, oversight mechanisms, investigative/prosecutorial initiatives, public information initiatives, civil society initiatives, and other measures taken (in any sector) with the objective of increasing transparency about public decision making, conflict of interest, resource allocation, etc.; decreasing impunity for corrupt acts; increasing demand for reform or awareness of the problem; increasing knowledge about corruption and its costs; and reducing opportunities for corruption. Implementation requires that the measure be adopted, that organizational arrangements are put in place, financial and human resources allocated, & that observable steps are taken to initiate implementation and repeated, continued or/& expanded to demonstrate that implementation is continuing.

Measure: Number of participants in USG-supported trade, investment environment, and investment capacity building trainings.

Explanation:This is an output measure of training in trade and investment-related areas. As the number of people trained in trade and investment increases, the ability of small businesses to conduct trade increases. This measure supports the outcome measure "Improved trade readiness (i.e., complying with WTO standards and protocols for production and export) of LAC presence countries, as measured by country exports as a percentage of GDP." FY 2009 and FY 2010 targets will be set according to funding levels provided.

Measure: Percentage of a cohort of students enrolled in first grade that are expected to reach grade five

Explanation:The percentage of a cohort of pupils enrolled in the first grade of primary school in a given school-year who are expected to reach grade 5, regardless of repetition. Data is provided by the UNESCO Institute for Statistics and includes only those countries where USAID has either a bilateral education program or provides assistance through the regional Centers of Excellence for Teacher Training initiative. Actual performance data is not available for years 2005-2007 as the data, which is collected and published by UNESCO, is not collected every year. The last year the data was published was 2004.

Measure: Number of hectares under improved natural resource management as a result of USG assistance.

Explanation:"Improved NRM" includes activities that promote enhanced management of natural resources for one or more objectives, such as sustaining soil and/or water resources, mitigating climate change, and/or promoting sustainable agriculture, etc. Management should be guided by a stakeholder-endorsed process following principles of sustainable NRM, improved human and institutional capacity for sustainable NRM, access to better information for decision-making, and/or adoption of sustainable NRM practices. A spatial indicator is an appropriate measure of the scale of impact of NRM interventions. The standard of 'improved' management as defined by implementation of best practices and approaches demonstrates progress and results across a wide range of development programs.

Measure: Percentage of LAC USAID-supported Millennium Challenge Account (MCA) candidate countries that pass at least one-half of the indicators in the "Ruling Justly" policy category, and above the median on the corruption indicator.

Explanation:This indicator measures the percentage of countries that are eligible for MCA compact funds under MCA's "Ruling Justly" criteria. Selection for MCA compact eligibility is based primarily on a country's overall performance in relation to three broad policy categories: 1) Ruling Justly; 2) Encouraging Economic Freedom; and 3) Investing in People. The MCC Board relies upon 17 publicly available and independent indicators to assess policy performance and demonstrated commitment in these three areas, to the maximum extent possible, for determining which countries would be eligible for MCA Compact assistance. In determining eligibility, the Board considers if a country performed above the median in relation to its peers on at least half of the indicators in each of the three policy categories and above the median on the "Control of Corruption" indicator. DA funds spent on democracy and governance programs in Latin America countries should result in an increase in the number of countries that pass the MCA Ruling Justly criteria.

Measure: Number of primary school learners that are direct beneficiaries of USAID programs.

Explanation:This indicator captures the number of individuals enrolled in USG-supported primary schools and equivalent non-school-based settings for the purpose of acquiring academic basic education skills and knowledge. It may include individuals receiving USG-supported educational radio and/or TV programs. It captures only direct beneficiaries and does not measure the impact of policy reform and other initiatives that indirectly impact beneficiaries. The indicator provides an indication of the overall scope of USAID education programming in LAC. Data source: FACTS indicator for Basic Education (number of learners enrolled in USG-supported primary schools or equivalent non-school based settings). Since this indicator is based on a FACTS indicator, the most recent actual data available is for 2007. Realistic targets for this indicator are dependent on funding levels and are set at the operating unit level and rolled up through FACTS. Operational Units will set targets for 2009 in fall, 2008, and for 2010 in fall, 2009.

Measure: Ratio of DA account-attributed Operating Expenses and DA account Program Support funds to total DA Program Funds.

Explanation:Ratio of DA account-attributed OE (operating expenses) and DA account program support funds per total DA program funds

Measure: Improved trade readiness (i.e., complying with WTO standards and protocols for production and export) of LAC presence countries, as measured by country exports as a percentage of GDP.

Explanation:

Is the program purpose clear?

Explanation: The Foreign Assistance Act of 1961 (FAA) does not distinctly authorize Development Assistance (DA), nor does any specific authorization legislation that follows. It is per the direction of annnual Foreign Operations Appropriations legislation, DA is funded to carry out sections 103, 105, 106, and 131 of Chapter 1 and Chapter 10 of Part 1 of the FAA, including activities for free market economic development, agriculture, rural development, literacy and basic education for children and adults, environment, energy, science and technology and other programs related to longer-term development. USAID/LAC's programming of DA funds supports the US foreign policy priority to advance sustainable development and global interests, and contributes to USAID's goals of economic growth and agricultural development, democracy and good governance, human capacity building, and environmental protection.

Evidence: 1) Secretary of State Statement in Bureau Performance Plans (BPPs). 2) State-USAID Strategic Plan for FY 2004-2009. 3) USAID Administrator Natsios testimony to HACFO, April 9, 2003. 4) Assistant Administrator (AA) Franco testimony to SACFO, April 2, 2003. 5) AA Franco testimony to the House Committee on International Relations, February 27, 2003. 6) Congressional Budget Justification (CBJ) Overview. 7) Anderson-Albright memo on State/USAID coordination, dated September 28, 2000. 8) USAID Administrator Natsios testimony to House Appropriations Subcommittee on Foreign Operations, April 9, 2003. 9) LAC Assistant Adminsitrator Franco testimony to the Senate Committee on Foreign Relations on April 2, 2003. 10) AA Franco testimony to the House Committee on International Relations on February 27, 2003. 11) Congressional Budget Justification (CBJ) Overview.

Does the program address a specific and existing problem, interest or need?

Explanation: The program addresses a specific need and specific problems within the region. The DA programs in the LAC region support U.S. foreign policy interests as well as address country level problems: promoting broad-based economic growth, strengthening democratic processes, reducing poverty, protecting and improving health and nutrition, and fostering cooperation on issues such as drug trafficking and crime, and environmental protection. The development challenges in the LAC region include contracting economic growth rates, extensive poverty, unemployment, skewed income distribution, crime and lawlessness, a thriving narcotics industry, and a deteriorating natural resource base. There are also civil unrest due to poor economic conditions, political instability in some countries, and natural disasters. These challenges are documented through LAC's strategic planning processes, in the Budget Justification to Congress (CBJ) and in trend analyses, as well as development statistics from other institutions such as the World Bank and the UN Economic Commission for Latin America and the Caribbean. Additionally, challenges faced by each country are reflected in each Mission Performance Plan (MPP), which encompasses all activities of United State Government (USG) agencies working in the country and in the region.

Evidence: 1) Secretary of State Statement in Bureau Performance Plans (BPPs). 2) State-USAID Strategic Plan for FY 2004-2009. 3) USAID Administrator Natsios testimony to HACFO, April 9, 2003. 4) LAC Assistant Administrator (AA) Franco testimony to SACFO, April 2, 2003. 5) AA Franco testimony to the House Committee on International Relations, February 27, 2003. 6) Congressional Budget Justification (CBJ) Overview. 7) Anderson-Albright memo on State/USAID coordination, dated September 28, 2000. 8) USAID Administrator Natsios testimony to House Appropriations Subcommittee on Foreign Operations, April 9, 2003. 9) LAC Assistant Adminsitrator Franco testimony to the Senate Committee on Foreign Relations on April 2, 2003. 10) AA Franco testimony to the House Committee on International Relations on February 27, 2003. 11) Congressional Budget Justification (CBJ) overview. 12) 12/19/02 Financial Times article by Mark Mulligan, "Early Signs of Recovery in Latin America." 13) LAC Bureau Trends Analysis in Environment, as an example. 14) Budget Justification to Congress, FY2004 (outlines existing problems and challenges on the regional, sub-regional, and country level.) 15) El Salvador Mission Performance Plan (MPP).

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: The program is designed so that is not redundant or unneccessarily duplicative of any other Federal, state, local or private efforts. As one of several development entities in a developing host country, and the lead United States Government (USG) foreign assistance entity, USAID coordinates with all of the other in-country USG entities under the aegis of the U.S. Ambassador, as well as with the host government and other prominent donors, to avoid duplication of assistance efforts and eliminate program redundancies, and promote country program synergies. For example, in El Salvador, the Center for Disease Control and USAID jointly designed and implemented a program to strengthen an integrated public health surveillance system. Preparation of the annual Mission Performance Plan (MPP) is coordinated by the U.S. Embassy and documents all activities of USG agencies that are carried out in that country. Country Team meetings, involving the principals of all USG entities and chaired by the Ambassador, are also held on a weekly basis. Further, USAID and other bilateral and multilateral donors, the relevant host Government Office, and implementing partners including the UN health agencies and the Pan American Health Organization (PAHO) meet regularly to share development experiences and successes, and maximize assistance outcomes. Finally, through the Global Development Alliance (GDA), LAC leverages private resources for jointly funded programs. Preparation of the annual Mission Performance Plan (MPP) is coordinated by the U.S. Embassy and documents all activities of USG agencies that are carried out in that country (see MPPs for Bolivia, El Salvador and El Salvador). Country Team meetings, involving the principals of all USG entities and chaired by the Ambassador are also held on a weekly basis. Further, USAID and other prominent donors, the relevant host Government Office, and implementing partners meet regularly to share development experiences and successes, and maximize assistance outcomes. Finally, through the Global Development Alliance (GDA), LAC leverages private resources for jointly funded programs.

Evidence: 1) Mission Performance Plan (MPP) - Bolivia, El Salvador, and Honduras as examples. 2) CJB--Donor Collaboration section. 3) OU Example: International Assistance section of Brazil strategy (pg. 13-15). 4) Donor Collaboration section of Health Strategic Objective in Peru Strategy (pg. 72-73). 5) Global Development Alliance (GDA) Guidelines. 6) List of Public-Private Alliances with the Gates Foundation in LAC countries. 7) USAID grant with PAHO. 8) The Automated Directives System (ADS) - Sections 201.3.6 and 201.3.7 (pg. 26-38) provides Agency guidance for the preparation of Strategic Plans. Section 201.3.9.2 provides specific guidance on donor coordination analysis in the strategic planning phase. 9) Donor Collaboration section of the CBJ. 10) Global Development Alliance (GDA) Guidelines. 11) GDA initiatives in LAC - as an example: Coffee Alliance in LAC with Proctor and Gamble and Technoserve (GDA info available on USAID website). 12) Examples of Inter-Agency Agreements with other US Government (USG) entities: a. Federal Trade Commission (Andean Community, September 2002); b. State Department PL 106-246 (Chapare region of Bolivia, August 2001); c. National Oceanic and Atmospheric Administration (Amazon, Brazil March 1999). 13) ADS Sections 201.3.6 and 201.3.7 (pg. 26-38) guides the preparation of Strategic Plans. Section 201.3.9.2 provides specific guidance on donor coordination analysis in the strategic planning phase.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The LAC/DA program design is free of major flaws that would limit it's effectiveness or efficiency. As required by the USAID Automated Directives System (ADS), LAC operating units (OUs) must develop a Results Framework (RF) for each strategic objective (SO), linking the development challenge being addressed to program implementation activities and milestones to be achieved. The SO is the most ambitious result in a particular program area that an operating unit (with its partners) can materially affect and for which it is willing to be held accountable. The RF is informed by various technical and resource analyses and include sector assessments, lessons learned, and gender and country conflict vulnerability analysis. All OU strategies are subject to intensive agency-wide reviews. Final agency approval is contingent on the resolution of all technical and policy-related issues and concerns that were raised during the reviews, as well as consistency with the LAC Bureau's stated priorities and U.S. Foreign Policy interests. ADS 201 details the planning/design process. The LAC Bureau's design process is initiated at the Mission level to ensure that the LAC Missions have strategies that are relevant to the specific country needs and circumstances. After the extensive review of the strategy by relevant USAID offices, a management contract is developed between the LAC Bureau and OUs which serves as the basis for approval of the proposed Strategic Plan. The Management Agreement provides a summary of agreements on the OU's program, confirmation of estimated resources, program start and end dates, and additional guidance on any special management concerns. This contract is reviewed on an annual basis and when needed updated through the annual report process and mid cycle intensive reviews that done every three years.

Evidence: 1) ADS 201, pg. 30-33 provides Agency guidance regarding the content of Strategic Plans; ADS 201 pg. 42-48 provide guidance on mandatory and other analyses for developing Strategic Plans. 2) Example of LAC review of OU Strategic Plan: Peru Management Letter and Issues Paper as examples.

Is the program effectively targeted, so that resources will reach intended beneficiaries and/or otherwise address the program's purpose directly?

Explanation: The program is effectively targeted to ensure that resources reach the intended beneficiaries. Regular consultation with prominent partners, customers (including beneficiaries), and stakeholders is critical to the development and implementation of all OUs strategic plans. The results of these consultations are integrated into strategic plans and implementation, (i.e. results and activities) are influenced by feedback from stakeholders. In addition, Country and Assistance Checklists are completed to ensure that USAID assistance is targeted at eligible countries and eligible organizations. To ensure that services reach the intended beneficiaries, USAID independent evaluations (see 2.6) are conducted to validate program approaches and direction and serve as the basis for any mid-course corrections. USAID monitors assistance received by beneficiaries through evaluating data collected in the performance monitoring plan (PMP) as well as grantee/contractor reports.

Evidence: 1) Example of OU Strategy - Peru Mission/OU. 2) ADS 201.3.42. 3) ADS 203.3 prohibition to assistance to Drug traffickers. 4) Evidence of participatory planning: Conducting Service Assessment. 5) Parks in Peril Work Plan. 6) El Salvador PMP. 7) TIPS Number 7: Preparing a PMP. 8) Also look at evaluation and performance monitoring techniques on the following website: www.usaidresults.org . 9) Ecuador FY 2003 Annual Report.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: In response to the need for more coordinated planning of the United States' foreign policy and development assistance programs, the Department of State and USAID jointly developed the Strategic Plan for Fiscal Year 2004 to 2009. The strategic objectives and specific long-term performance goals contained in the Strategic Plan constitute the top-level strategic planning framework for both agencies. To facilitate performance monitoring and reporting under the Joint State/USAID strategies, the LAC Bureau in FY 2004 undertook an extensive effort to align and harmonize a set of contextual and regional performance indicators that would provide valuable performance information to managers both in the field and at Washington Headquarters. USAID's performance management framework provides the vital linkage between joint State/USAID strategic objectives, strategic goals, performance goals, DA contextual and common performance indicators at the regional level, and annualized performance targets. USAID's long-term performance goals are supported by outcome and/or output-related performance measures and regional indicators which are used to assess progress towards completion of the long-term goals, and allow Agency management to re-direct program where necessary.

Evidence: 1) Joint State/USAID Strategic Plan for Fiscal Year 2004 to 2009. 2) USAID FY 2005 Annual Performance Budget, March 25, 2004. 3) LAC Bureau Approved Regional Indicators, May 14, 2004. 4) Agency FY 2003 Performance and Accountability Report. 5) Agency Guidance on Performance Management - specifically ADS 203.2 through 203.3.5. 6) Example of a PMP : El Salvador Performance Management Plan (PMP). 7) Agency Guidance on Performance Management - specifically ADS 203.2 through 203.3.5. 8) Example of an OU Strategy - Peru Mission Strategy. 9 ) Example of an OU Performance Management Plan (PMP) - El Salvador Mission. 10) Ecuador FY 2003 Annual Report.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The LAC/DA program has set ambitious targets and yearly timeframes for it's long-term measures. The two long term measures, % of countries meeting the MCA criteria on ruling justly and corruption, and improvement in trade readiness, consistent with WTO criteria, have targets for the LAC region through 2010 which are based on significant improvement from the baseline data.

Evidence: 1) Joint State/USAID Strategic Plan for Fiscal Year 2004 to 2009. 2) USAID FY 2005 Annual Performance Budget, March 25, 2004. 3) LAC Bureau Approved Regional Indicators, May 14, 2004. 4) OU-level indicators - El Salvador Mission PMP as an example; ADS 203.3.5 for Agency Guidance on Data Quality Assessments. 5) Changes to indicators - PMP for El Salvador, SO 519-004. 6) ADS 203.33. 7) IG Audit of Data Quality for USAID/Guyana's Results Review and Resource Request. 8) IG Report, Data Quality for USAID/Panama SO for Canal Watershed Management.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: USAID's performance management framework is structured around a limited number of challenging but achievable strategic objectives and goals (developed jointly with the Department of State), as well as DA performance goals, targets and indicators that are logically linked to the long-term joint goals and objectives which enables USAID to demonstrate progress toward achievement. The LAC Bureau measures their performance against this structure through a set of discrete, quantifiable, and measurable annual targets and contextual and performance indicators, which are monitored by the LAC Bureau and are reported each year in the annual reporting process. In the FY 2005 Annual Performance Budget, the LAC region also reports regional baselines and targets as a subset of the agency's common indicators.

Evidence: 1) Joint State/USAID Strategic Plan for Fiscal Year 2004 to 2009. 2) USAID FY 2005 Annual Performance Budget, March 25, 2004. 3) LAC Bureau Approved Regional Indicators, May 14, 2004. 4) OU-level indicators - El Salvador PMP as an example. 5) Draft 2005 Annual Reporting Guidance, Section I, Sub-section A, Item 5 (Page 6) and Guidance Annex VIII, Page 29.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The DA program has documented baselines and ambitious targets for its performance measures. Development of targets was based on a review of OU-level indicators and targets for related strategic objectives. Some of these measures have been used in prior years at the OU level, some have not. This is the first year that USAID has developed regional performance measures, and this complements the achievements of the agency in the past year in developing agency-wide performance measures and targets.

Evidence: 1) Peru PMP and Strategy Illustrative Performance Measures (pg. 70 of Peru Strategy) as examples. 2) ADS 203.2, 203.3, 203.4 and 203.5 provide Agency Guidance on Performance Management and Data Quality, performance monitoring plans, performance monitoring, selecting performance indicators, and data quality. 3) El Salvador PMP, as an example. 4) LAC Bureau Approved Regional Indicators, May 14, 2004.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: At the OU-level, Strategic Objective Agreements (SOAGs) with host Governments outline the terms of agreement, including commitment to shared goals of USAID/LAC OUs and counterpart Ministries. Solicitation documents for contractors and grantees such as Request for Proposals (RFP), Request for Applications (RFA), or Annual Program Statements (APS) incorporate SO and/or program objectives. Respondents to the RFP or RFA need to articulate how they plan to contribute to the achievement of LAC Mission Strategic Objectives. The extent to which they would be able to help achieve these shared objectives and goals is incorporated in the technical merit criterion of the selection criteria. Grants and Contract Agreements incorporate or reflect these shared commitments to common goals, typically in the scope of work section. These commitments are likewise reflected in Contractors and Grantees annual work plans. By the terms and conditions of their agreements, contractors and grantees provide quarterly progress reports to OUs that document their results to date. USAID managers verify information in these reports through monitoring, site visits, and meetings. The information in the OU's PMP includes results reported and verified from contractors and grantees. All of these activities are designed and carried out to ensure that USAID partners commit to and work toward the annual and/or long-term goals of the program.

Evidence: 1) Sample SOAG. 2) ADS 350 guidance on SOAGs. 3) ADS 303.5.5 b and c provide Agency guidance on Evaluation Criteria for Grants. 4) Program Description of Brazil RFA. 5) Work Plan of Parks in Peril. 6) RFA for Guatemala Income Generation.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: Independent evalautions of sufficient scope and quality are conducted on a regular basis. Evaluations are intended to look at results achieved vs. targets/goals, program impact and effectiveness, and lessons learned among others. For example, in the past five years approximately 200 evaluations have been conducted for the LAC Bureau's programs, including DA-funded activities. A break-out of evaluations of DA-only activities is not available because evaluations are often cross-sectoral and activities can involve multiple funding sources. Evaluations can be internal, external, collaborative or participatory, and are generally done by entities outside of the LAC Bureau such as independent private firms not directly associated with the activity or program, the Inspector General Office, or USAID's Center for Development Information and Evaluation (CDIE). Mid-term, evaluation recommendations are used to introduce any course corrections and final evaluations are used to determine Bureau decisions on any further assistance to that technical sector. Situations that require evaluations include: input to inform a key management decision; performance data that indicate an unexpected result or finding which cannot be readily explained and therefore, need further analysis, customer or partner feedback that suggest there are implementation problems or unmet needs; the need to re-examine program sustainability; cost-effectiveness or relevance; and to identify lessons learned. One example of this is the July 2001 audit by the Inspector General Office to determine if USAID/Nicaragua's Water and Sanitation activities were on target. The audit concluded that the activities were on target with the exception of the construction of sewage pits. The Mission was advised to construct these pits or reprogram the money.

Evidence: 1) LAC list of evaluations, as compiled by USAID's Center for Development Information and Evaluation (CDIE). 2) Inspector General Audit Reports (see Bolivia Financial Operations audit as example). 3) ADS 203.3.6 provides Agency guidance on evaluations. 4) IG Audit of USAID/Nicaragua's Water and Sanitation Project July 25, 2001.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: At the regional level, USAID/LAC does not yet develop budget requests that are explicitly tied to accomplishment of annual and long-term performance goals. The framework of regional and common agency performance goals and targets is too new for the agency to be able to integrate this performance data into their strategic budgeting. OMB commends USAID for its efforts in developing a strategic budgeting model (which factors in performance data) to support the LAC (and all regional bureaus) in their budget formulation. At the level of the Operating Units (OUs) budget requests are linked somewhat to the Annual Report of the OU, which summarizes program progress toward each Strategic Objective of the OU. Furthermore, the Congressional Budget Justification summarizes program progress by Strategic Objective, for each individual OU, bu this is done in a qualitative, anecdotal manner, and the agency has not yet reached the stage of incorporating their more quantitative performance data into budget presentations. That said, the CBJ is otherwise a transparent and complete presentation of resource needs.

Evidence: 1) OU Annual Report (AR) - Guatemala as an example. 2) LAC Review of AR - Guatemala as an example. 3) FY 2005/2006 Bureau Program and Budget Submission (BPBS) Guidance issued by the Agency's Policy and Program Coordination Office. 4) Congressional Budget Justification (CBJ).

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: USAID has taken extensive action to improve its strategic planning and budgeting processes and tools, and has made sufficient progress on its Budget and Performance Integration PMA scorecard to upgrade its status score to Yellow. For the LAC bureau on Development Assistance, this has resulted in improved linkage between joint State/USAID strategic objectives, strategic goals, performance goals, contextual and common performance indicators at the regional level, and annualized performance targets.

Evidence: 1) Joint State/USAID Strategic Plan for Fiscal Year 2004 to 2009. 2) USAID FY 2005 Annual Performance Budget, March 25, 2004. 3) LAC Bureau Approved Regional Indicators, May 14, 2004. 4) Agency FY 2003 Performance and Accountability Report. 5) Agency Guidance on Performance Management - specifically ADS 203.2 through 203.3.5. 6) LAC Timeline for Mission Program/Strategy Reviews. 7) Intensive Review memos - Jamaica and Haiti as samples. 8) AA Franco-Cronin Memo, re: Central America and Mexico Regional Strategy. 9) ADS 203.3.10 - Planning, pg. 39. 10) Draft 2005 Annual Reporting Guidance, Section I, Sub-section A, Item 5 (Page 6) and Guidance Annex VIII, Page 29.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: USAID regularly collects timely and credible performance information from key program partners. Funding recipients are required to manage, monitor, and report performance information on a regular basis. This information is submitted to OUs on a quarterly basis and feed into the Annual Report (AR) and PMP. OUs indicate in the AR the status of targets for DA. The LAC Bureau, in turn, reviews OU performance via the AR review process. When targets are not met, various options may be considered depending on the specific circumstances and issues involved: the contract or grant agreement may be modified, additional funds may be delayed or not provided, management changes may be introduced, discussions may take place with other partners (other donors or host-country) regarding their commitment, etc. If goals are exceeded the OU will decide whether to set new targets or to focus on other interventions. The AR review process in turn feeds into Bureau decisions about programming, budgeting, and staffing, among others. OUs indicate in the Annual Report whether targets for DA funded programs are met or exceeded. The LAC Bureau, in turn, reviews OU performance via the AR review process. When targets are not met, various options may be considered depending on the specific circumstances and issues involved: the contract or grant agreement may be modified, additional funds may be delayed or not provided, management changes may be introduced, discussions may take place with other partners (other donors or host-country) regarding their commitment, etc. The AR review process in turn feeds into Bureau decisions about programming, budgeting, and staffing, among others.

Evidence: 1) Operating Unit Annual Report (AR), with Guatemala AR as a sample. 2) LAC Bureau Review of Guatemala AR. 3) Operating Unit Performance Management Plan (PMP), with Ecuador as a sample. 4) USAID's ADS 202.3.6; USAID adherence to 22 CRR 226, "Administration of Assistance Awards to US Non-governmental Organizations," Section 226.51, "Monitoring and Reporting Program Performance"; ADS 303.7 Mandatory Reference for Grants. 5) Draft 2005 Annual Reporting Guidance, Section I, Sub-section A, Item 5 (Page 6) and Guidance Annex VIII, Page 29.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: USAID federal managers and program partners are held accountable for cost, schedule, and performance results. A cognizant technical officer (CTO) is designated for each LAC activity to manage contractors and grantees. CTO responsibilities include, among others, the technical review and approval of vouchers, monitoring contractor/grantee performance and financial status, and overseeing sub-agreements. The Agency recently re-established a training program to certify CTOs responsible for managing contracts and grants. The courses are designed to provide CTOs with the basic skills and knowledge to effectively carry out the role of a CTO. Under USAID's personnel evaluation process, CTOs are evaluated for effective management of agreements, including the quality of technical guidance provided to contractors and grantees. The Agency also offers a course for CTO supervisors to enable them to more effectively provide on-the-job training and to better monitor and evaluate the performance of a CTO. Past performance of contractors and grantees is a criteria in the award. Institutional capability and past performance is 25% of the total selection criteria. Performance-based type of contracts and agreements are used to focus contractors on achievement of program results. Mid-term evaluations are also used to review performance to date and identify issues. An example is the August 2001 mid-term evaluation of Nicaragua's Diversification, Sustainability and Social marketing grant wherein program recommendations were outlined by an independent evaluator and carried out by the program partner.

Evidence: 1) CTO responsibilities as defined in ADS 300. 2) Brazil HIV/AIDS RFA, Guatemala G-CAP Performance-Based Statement of Work for its Health SO. 3) August 2001 mid-term evaluation of Nicaragua's Diversification, Sustainability and Social Marketing Grant. 4) Course description of CTO Certification Program. 5) Sample Request for Proposal: Dominican Republic RFP # 517-03-011, pg. 10.

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: USAID funds are obligated in a timely manner and spent for the intended purpose. Obligations, among other financial data, are tracked via Phoenix, an Agency-wide accounting system. LAC/DA funds are provided on an incremental, as-needed basis, with specific amounts determined by the budget and final Operating Year Budget setting processes. Once funds are obligated, OUs and the LAC Bureau monitor disbursements over the life of the program. Based on Agency policy and LAC Bureau's analysis, an OU's program may not receive its full increment of funds in a subsequent year if it is determined that an unacceptable level of funds is still unspent (undisbursed). According to Agency policy, a program can only have unspent funds sufficient to cover expected disbursements for the next 12-18 months. Finally, all OUs are required to prepare procurement plans that outline planned procurements and obligations and help ensure that funds are spent in a timely manner and for the intended purpose.

Evidence: 1) Phoenix Flash report. 2) ADS 602 also provides specific Agency guidance regarding forward funding of programs. 3) LAC Regional Procurement Plan.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: Mission Management Assessments (MMAs) recently undertaken by the LAC Bureau provide a systematic and comprehensive view of the Bureau's OUs as a basis for defining ways to improve operations and to help rationalize the allocation of scarce staff and operating expense (OE) resources. The results of the Mexico, Guatemala, Honduras and Nicaragua MMAs were taken into account during the development of the Central America and Mexico Regional strategy. Building upon favorable experience with regional hubs for service support, the Bureau has agreed to move toward one regional platform over time in Central America, which will be located in El Salvador. Consolidation of accounting functions in the Carribbean missions resulted in the reduction of five positions and a future annual cost reduction of approximately $60,000 in OE. The upgrade in Wide Area Renovation Plan communications capability througout the LAC region, coupled with Virtual Private Network technology, has facilitated secure and extremely rapid transmission of raw accounting and payment information from the serviced post to the regional center and back to the serviced post in the form of usable, almost real-time accounting information. As a result, El Salvador performs payment certification functions for Guatemala, resulting in the reduction of one position and a future annual cost savings of $70,000. Although the above examples demonstrate operational efficiencies, LAC does not have procedures to measure program efficiency in terms of program output or performance as required.LAC Bureau adheres to Federal regulations on competition.

Evidence: 1) ADS 300 provides the overall guidance on procurement including competitive procurements (ADS 302 for Contracts and 303 for Grants). 2) FAR for ADS 302 -Contracts and 22 CFR 226 for ADS 303 - Grants. 3) LAC FY 2005/2006 Bureau Program and Budget Submissions (BPBS) (pg. 26-29 for Management Improvements/Operational Efficiencies). USAID adheres to FAR for contract and to relevant CFR provisions and OMB Circulars for grants.

Does the program collaborate and coordinate effectively with related programs?

Explanation: The program does collaborate and coordinate effectively with related programs. Most notably, the Department of State and USAID have recently developed joint goals at the Department/Agency level via the Joint State-USAID FY 2004-2009 Strategic Plan which formalize the relationship highlighted in 1.3. Historically, State and USAID have planned and implemented programs collaboratively at the country level via the MPP process, which will now influence the joint State/USAID Strategic Plan. Weekly country team meetings are held among USG agencies and regular coordination meetings with other donors in a country. In addition, the LAC Bureau and its OUs collaborate and coordinate with related programs in a leadership or participant role through a variety of mechanisms: close working collaboration with UN health organizations and PAHO, working arrangements with other Federal agencies such as CDC, and alliances with the private sector. Finally, the budget request process is a joint effort between State and USAID and is reflected in the FY 2005/2006 Bureau Program Budget Submission which will be the basis for an integrated budget that is submitted to the White House.State and USAID have joint goals at the Department/Agency level via the Joint State-USAID Strategic Plan and at the country level via the MPP. In addition, the LAC Bureau and its OUs collaborate and coordinate with related programs in a leadership or participant role through a variety of mechanisms: close working collaboration with State counterparts at the Bureau and country levels; memo of understanding with other Federal Agencies; 632, PASA or RSSA arrangements with other USG agencies; InterAgency working groups such as with the US Trade Representative on CAFTA issues; Consultative Group with other donor agencies. Finally, the Bureau actively seeks public-private partner alliances in common development areas. Once these relationships are established they are continued evaluated and monitored through the annual program review process.

Evidence: 1) State-USAID Strategic Plan. 2) Examples of MPPs - El Salvador, Honduras and Bolivia. 3) Example of public-private partner alliance: GDA alliance with the Gates Foundation. 4) Example of grant with PAHO. 5) FY 2005/2006 Bureau Program Budget Submission. 6) Joint Memo dated 9/28/00 from STATE/USAID on guidance and recommendation to improve cooperation and coordination between the agencies.

Does the program use strong financial management practices?

Explanation: Through the Federal Managers Financial IntegrityAct (FMFIA)/Management Control Review Committee process, USAID ensures that resources are protected against fraud, waste and abuse and that they achieve the results for which funds were appropriated. The process requires each OU in the LAC Bureau to do a self-assessment of the adequacy of management controls in all areas of agency operations including program, administrative, and financial management. Each OU submits an FMFIA memo to the LAC Bureau, which in turn submits a Bureau memo to the USAID Administrator. LAC grants and contractors are also subject to audits. Currently, the Phoenix system is used for Washington accounting and the MACS system for field accounting, using two systems has created a disconnect between the two. USAID plans to roll out the Phonenix system to the field, which will resolve the outstanding problems. It is anticipated that a first LAC pilot Mission (USAID/Peru) may begin Phoenix operation in mid-FY 2004. Other pilot Missions will follow a few months later depending on the success of the first pilot. Once Phoneix is deployed to all missions, USAID anticipates getting a "yes" answer for this question.

Evidence: 1) Federal Manager's Financial Integrity Act (FMFIA) checklist. 2) OU completed FMFIA analysis, with Ecuador as an example. 3) LAC Bureau FMFIA memo to USAID Administrator dated October 29, 2002. 4) Report on Material Weaknesses - Guyana and Ecuador as examples. 5) Agency-wide guidelines for Recipient-contracted audits (RCA). The Agency's ADS 620 chapter provides the overarching accounting and financial reporting principles and standards for the Agency. 6) ADS 596 provides Agency guidance regarding the accountability and effectiveness of USAID's programs and operations by establishing, assessing, correcting, and reporting on management controls. These ADS chapters are based on regulations issued by Federal oversight agencies, including OMB, Treasury and GAO.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: The program has taken meaningful steps to address management deficiencies. For example, the FMFIA process requires each OU in the LAC Bureau to do a self-assessment of the adequacy of financial management controls in all areas of agency operations, including management of program assistance. Each OU submits an FMFIA memo to the LAC Bureau, which in turn submits a Bureau memo to the USAID Administrator. At the OU level, LAC missions address deficiencies or weaknesses that could appropriately be resolved at that level. Any significant deficiency or material weakness (whether new or unresolved from previous FMFIA reviews) is reported to the LAC Bureau and included in the Bureau FMFIA memo to the Administrator. Steps for resolving the weakness are identified and a date proposed by which the weakness or deficiency will be corrected. In 2002, missions were also specifically requested to complete an information system security checklist, as part of the FMFIA process. The LAC Bureau has also recently undertaken management assessments of its OUs. Six assessments have been completed since May 2002. The results of the Mexico, Guatemala, Honduras and Nicaragua management assessments were taken into account during the development of the Central America and Mexico (CAM) Regional strategy. The assessments have helped define ways to improve Bureau operations, such as consolidating accounting functions in regional hubs, resulting in significant savings. The El Salvador mission is poised to serve as the Central America regional hub and will provide support services for a number of missions in the region.

Evidence: 1) ADS 620, FMFIA checklist. 2) OU completed FMFIA analysis- with Ecuador as a sample. 3) LAC Bureau FMFIA memo to USAID Administrator, dated October 29, 2002. 4) Report on Material Weaknesses - Guyana and Ecuador as samples. 5) LAC FY 2005 BPBS (pg. 26-29 for Management Improvements/Operational Efficiencies).

Are grants awarded based on a clear competitive process that includes a qualified assessment of merit?

Explanation: LAC Bureau contracts and grants are awarded competitively, except where supported by documentation approved by officials with authority to approve non-competitive awards. For example, in FY 2002 $78.8 million of new assistance in LAC were awarded competitively including a qualified assessment of merit, out of a total of $104.7 million (about 75%). Decisions on grant and contract awards are properly documented (via selection memos, memos of negotiation, etc.). USAID Washington (via the Office of Procurement) reviews and provides oversight to ensure LAC adherence to Federal regulations and Agency guidance on competition.

Evidence: 1) ADS 303.5 for Grants and CBD notices for Contracts. 2) ADS 202.3.9 provides guidance on ensuring procurement integrity and ethics.

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: The LAC program has oversight practices that provide sufficient knowledge of grantee activities. For example, a Cognizant Technical Officer (CTO) is designated for each program to manage contractors and grantees. CTO responsibilities include, among others, technical review and approval of vouchers, monitoring contractor/grantee performance and financial pipelines, and overseeing sub-awards. The Activity Manager and the SO Team Leader, who may or may not be the CTOs, also provide oversight. In addition, mission management is kept informed of program performance, progress, and issues via periodic portfolio reviews, staff meetings, etc. The LAC Bureau, in turn, is informed via the Annual Review process. The OU Agreement Officer is the mandatory control point of record for all official communications and contacts with the recipient that may affect the award budget, the program description or any terms and conditions of the award. Audits per OMB Circular A-133 or recipient-contracted audits are also conducted as required by policy or regulation.

Evidence: 1) Cognizant Technical Officer (CTO) responsibilities as provided in ADS 303.3 for Grants and ADS 302.3 for Contracts. 2) Recipient-Contracted Audit guidelines. 3) USAID FY 2003 Performance and Accountability Report.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: The LAC Bureau collects grantee performance data and makes it available to the public. In preparing USAID's reporting documents, the data supplied by the various grantees and contractors are utilized to support LAC Bureau's analysis and program descriptions. Disclosure to the public is done in a number of different ways. For example, the USAID FY 2003 Performance and Accountability Report discloses program information at the Agency level. The CBJ (available on the USAID web site) provides specific information to the public on LAC programs including information on SO performance and results, by OU. The CBJ also indicates the various grantees and contractors working in the different SOs. USAID's Center for Development Information and Evaluation (CDIE), via the Development Experience Clearinghouse (DEC), makes some evaluations of specific programs available to the public.

Evidence: 1) USAID FY 2003 Performance and Accountability Report. 2) CBJ. 3) Program/Activity evaluations.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: At the regional level, the LAC Bureau's long-term and annual performance measures for DA programs are new. These measures are supplemented by country-level indicators which are contained in OU Performance Monitoring Plans (PMPs). Through review of the history of OU performance, there is sufficient evidence for this review to give a "small extent" rating that the program is on-track to meet its long-term performance goals, including the "top level" strategic goals/objectives within the joint State/AID Strategic Plan. For FY 2003, 93% of the strategic objectives USAID was required to report on met or exceeded their targets. USAID also met the target in the FY 2003 Annual Performance Plan of 90% of strategic objectives met or exceeded.

Evidence: 1) Joint State/USAID Strategic Plan for Fiscal Year 2004 to 2009. 2) USAID FY 2005 Annual Performance Budget, March 25, 2004. 3) LAC Bureau Approved Regional Indicators, May 14, 2004. 4) Agency FY 2003 Performance and Accountability Report. 5) Examples of PMP and Annual Report: El Salvador PMP, Guatemala AR. 6) Draft 2005 Annual Reporting Guidance, Section I, Sub-section A, Item 5 (Page 6) and Guidance Annex VIII, Page 29.

Does the program (including program partners) achieve its annual performance goals?

Explanation: At the regional level, the LAC Bureau's long-term and annual performance measures for DA programs are new. These measures are supplemented by country-level indicators which are contained in OU Performance Monitoring Plans (PMPs). Through review of the history of OU performance, there is sufficient evidence for this review to give a "small extent" rating that the program is on-track to meet its annual performance goals, including the "top level" strategic goals/objectives within the joint State/AID Strategic Plan. For FY 2003, 93% of the strategic objectives USAID was required to report on met or exceeded their targets. USAID also met the target in the FY 2003 Annual Performance Plan of 90% of strategic objectives met or exceeded.

Evidence: 1) Joint State/USAID Strategic Plan for Fiscal Year 2004 to 2009. 2) USAID FY 2005 Annual Performance Budget, March 25, 2004. 3) LAC Bureau Approved Regional Indicators, May 14, 2004. 4) Examples of PMP and Annual Report: El Salvador PMP, Guatemala AR. 5) USAID FY 2003 Performance and Accountability Report.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The LAC program has demonstrated improved efficiencies and cost effectiveness in achieving program goals each year. For example, improvements in technology and consolidation of accounting functions have resulted in major savings. In the Caribbean, results include five less staff positions and an annual OE cost reduction of approximately $60,000. In Guatemala, a reduction of one position meant savings of $70,000 as El Salvador (already poised as a regional support center) assumed payment certification functions. The mission management assessments (MMAs) conducted by LAC since May 2002 provide a systematic and comprehensive view of the Bureau's overseas OUs as a basis for defining ways to improve Bureau operations, and to help rationalize the allocation of scarce staff and operating expense resources. The results of the Mexico, Guatemala, Honduras and Nicaragua MMAs have already been taken into account during the development of the Central America and Mexico Regional strategy. Improvements in technology and consolidation of accounting functions have resulted in major savings.

Evidence: 1) LAC FY 2005/2006 BPBS (pg. 26-29 for Management Improvements/Operational Efficiencies). 2) CTO responsibilities as provided in ADS 303.3 for Grants and ADS 302.3 for Contracts. 3) Guyana and Ecuador FMFIA reports. 4) LAC Bureau's FMFIA memo dated October 29, 2002.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: The performance of the LAC/DA program compares favorably to other programs with similar purposes and goals. Because of its field presence and flexible programming, USAID is often cited by other donors and host governments for its ability to respond quickly, for both disaster assistance and standard international assistance, and for its leadership of country program coordination with other donors. For example, in 2001, the Government of Honduras prepared its Poverty Reduction Strategy Paper (PRSP) in response to the World Bank and the International Monetary Fund's highly indebted poor country (HIPC) initiative. USAID was widely credited for its leadership role among the donors. USAID programming decisions and experience also heavily influenced Nicaragua's PRSP and USAID's involvement facilitated public discussion and debate to a degree that would not have occurred in the past.

Evidence: 1) LAC consultations with the World Bank and IDB on Country Assistance Strategies for Honduras, Bolivia, Nicaragua, and Ecuador, May 2002. 2) Evaluation Brief Number 5, March 2003: USAID's Approach to Poverty Reduction - The Case of Honduras (pg. 7-8). 3) April 24, 2003 letter from Asst. US Trade Representative for Intergovernmental Affairs and Public Liaison Christopher Padilla to USAID G-CAP Deputy Director regarding CAFTA (Central America Free Trade Agreement). 4) Press releases from: a. Washington Post Foreign Service on USAID earthquake reconstruction assistance in El Salvador; b. Tea and Coffee Trade Journal on USAID assistance for coffee growers.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: Independent evaluations have shown that DA programs are achieving results to a large extent as defined by the goals for the specific program or strategic objective. Independent evaluation have been conducted at the OU/Mission level for sector specific or strategic objectives. These evaluations are done by independent consultants, IG or GAO and are of sufficient scope and quality to determine that LAC Mission programs are accomplishing or have achieved their program goals.

Evidence: 1) LAC list of evaluations, as compiled by CDIE. 2) USAID's Approach to Poverty Reduction - The Case of Honduras: http://cdie.usaid.gov/pdf_docs/PNACR351.pdf.