Detailed Information on the
Assistance to Developing Countries Assessment

Identify and track intermediate indicators at the operating unit level that assess whether the program is on track to meet longer-term targets.

Ensure that performance on measures is factored into the decision-making process and presentation of future budget requests.

Develop targets for the efficiency measure and develop a method to identify cost savings or efficiencies from meeting future targets.

Measure: Net enrollment rate in primary school

Explanation:Number of pupils of the theoretical school-age for a given level of education, expressed as a percentage of the total population in that age-group. The rationale for this measure is to show the extent of participation in a given level of education of children and youths belonging to the official age-group corresponding to the given level of education. FY 2004 actual data is an average of FY 2004 data for Egypt, Indonesia, and Pakistan as reported by UNESCO. Egypt, Indonesia, and Pakistan will continue to report on this indicator. The 2015 target is the internationally agreed upon Millennium Development Goal for Education for All. Direction of Change: Higher = Better

Measure: Number of learners enrolled in USG-supported primary schools or equivalent non-school based setting

Explanation: This measure indicates the number of individuals formally enrolled in USG-supported primary schools or equivalent non-school based settings for the purpose of acquiring academic basic education skills or knowledge. This may include individuals receiving USG-supported educational radio and/or TV programs. This indicator is intended to capture direct rather than indirect beneficiaries and is reported on by USAID operating units. Counting the number of learners allows USAID to track the number of direct beneficiaries of education programs. This is not a cumulative indicator. Egypt, Indonesia and Pakistan contributed past performance data and targets for this indicator and will continue to report on this indicator. The FY 07 target is much lower than the FY 06 actual targets mostly because Pakistan is designing a new basic education program that won't be fully ramped up until FY 08. Direction of Change: Higher = Better.

Measure: Number of deaths among children under age five in a given year per 1,000 live births in that same year

Explanation:Under-five mortality rate (U5MR) is the probability (expressed as a rate per 1,000 births) of a child born in a specified year dying before reaching the age of five if subject to current age-specific mortality rates. The under-five mortality rate demonstrates the likelihood that a child will survive past the age of five and is a widely used indicator to measure the overall health in a country. The past performance data is an average of data for Egypt, Indonesia, and Pakistan as reported by UNICEF and the World Health Organization (WHO). Egypt, Indonesia, and Pakistan will continue to report on this indicator. The 2015 target is based on the Millennium Development Goal of reducing the under five mortality rate by two-thirds between 1990 and 2015 and was calculated by taking the average of the 2015 targets for Egypt, Indonesia, and Pakistan. The 2015 targets were calculated by reducing the 1990 baseline for each country as reported by UNICEF and WHO by two-thirds. Direction: Lower = Better

Measure: Number of cases of child diarrhea treated in USG-assisted programs

Explanation:This indicator measures the number of cases of child diarrhea treated through USG-supported programs with oral rehydration therapy or zinc supplements. Diarrheal illness is a major cause of preventable mortality among infants and young children; this indicator provides a measure of the number of children with diarrheal illness receiving required treatment. This is not a cumulative indicator. This is a new indicator, and therefore, there is no specific past performance information although related past performance information is discussed in question 4.2. Indonesia and Pakistan contributed to the FY 07 and FY 08 targets for this indicator and will continue to report on this indicator. Direction of Change: Higher = Better.

Measure: Number of people trained in maternal/newborn health through USG-supported programs

Explanation:This indicator measures the number of people (health professionals, primary health care workers, community health workers, volunteers, non-health personnel) trained in maternal and/or newborn health and nutrition care through USG-supported programs. Development of human capacity through training is a major component of USG-supported health area programs in this element. This is not a cumulative indicator. FY 05 and 06 data does not include Indonesia as they did not report on this indicator in the past. Egypt, Indonesia, and Pakistan contributed to the FY 07 and 08 targets and will continue to report on this indicator. FY 06 performance falls below the target because Egypt experienced problems in the follow-on procurement for their project. As a result, there were significant delays in implementation and almost no training for several months. Direction of Change: Higher = Better.

Measure: Number of children reached by USG-supported nutrition programs

Explanation:This indicator measures the number of children reached by programs that promote good infant and young child feeding and/or growth promotion programs. Promotion of good infant and young child feeding practices, including breastfeeding, and participation in community based growth monitoring and promotion are essential in preventing malnutrition and improving child survival. This is not a cumulative indicator. This is a new indicator, and therefore, there is no specific past performance information although related past performance information is discussed in question 4.2. Egypt, Indonesia, and Pakistan contributed to the FY 07 and FY 08 targets for this indicator and will continue to report on this indicator. Direction of Change: Higher = Better.

Measure: Days to start a business

Explanation:The definition of a business includes the following: it is a limited company and operates in the country's most populous city; it is 100% domestically owned and has 5 owners (none of whom is a legal entity); it performs industrial or commercial activities and does not perform trade activities or handle products subject to special tax regime (e.g. liquor or tobacco); it has up to 50 employees, does not qualify for investment incentives and has a turnover of at least 100 times income per capita. Measuring the days to start a business captures the bureaucratic and legal hurdles an entrepreneur must overcome to incorporate and register a new firm. Relatively fast business start-up shows that there are no unnecessary bureaucratic or legal hurdles or costs that constrain investment, productivity, and growth. The lower the number of days to start a business, the more likely it is that a country will attract foreign direct investment and local business start-up, thus accelerating economic development. It examines the procedures, time, and cost involved in launching a commercial or industrial firm with up to 50 employees and start-up capital of 10 times the economy's per-capita gross national income (GNI). This indicator is one of a set of indicators the Bank tracks for "Starting a Business." The actuals for FY 04, 05, and 06 are an average of the days to start a business in Egypt, Indonesia, Jordan and Pakistan as reported by the World Bank. The 2015 target of 18 days is based on the average number of days it took to start a business in the Organisation for Economic Cooperation and Development (OECD) countries in 2006. The OECD average is extremely ambitious, and has been chosen as the 2015 target because of the rapid improvement the Developing Countries are experiencing in this area. In fact, the average in 2006 for the four countries, 40 days, was nearly the same as the Transforming and Sustaining Country category averages, which were 39.5 and 38.9, respectively.

Measure: Number of the 11 core commercial laws put into place as a result of USG Assistance

Explanation:The eleven core commercial laws relate to legal categories, not individual statutes. They correspond to whether countries have established (i.e. put into place) a functioning legal regime for the following 11 business climate areas: 1. Company Law, 2. Contract Law and Enforcement, 3. Real Property, 4. Mortgage Law, 5. Secured Transactions Law, 6. Bankruptcy Law, 7. Competition Policy, 8. Commercial Dispute Resolution, 9. Foreign Direct Investment, 10. Corporate Governance and 11. International Trade Law. This is a comprehensive set of business climate areas. Demonstrating improvements in any of them indicates systemic changes are afoot. The target is the total number (combining Egypt, Indonesia, and Pakistan) of functioning legal regimes put into place that year out of the 11 major business climate areas. This is a new indicator, and therefore, there is no specific past performance information although related past performance information is discussed in question 4.2. Egypt, Jordan, and Pakistan contributed to the FY 07 and 08 targets and will continue to report on this indicator. Direction of Change: Higher = Better.

Measure: Number of new members in private business associations as a result of U.S. Government assistance

Explanation:This indicator measures the number of new members of private business/trade associations as a result of USG assistance. Firms improve their productivity, and in turn their competitiveness, by accessing capital and increasing investment in productive assets. Members of business associations also pressure the government to improve the business environment, reducing the costs and obstacles to starting a business. This indicator measures the number of new members joining in that year. This is a new indicator, and therefore, there is no specific past performance information although related past performance information is discussed in question 4.2. Jordan and Pakistan contributed to the FY 07 and FY 08 targets and will continue to report on this indicator. Direction of Change: Higher = Better.

Measure: World Bank Government Effectiveness Index

Explanation:The Government Effectiveness Index is one of six indicators utilized by the World Bank's Governance Matters Initiative, as reported by D. Kaufmann, A. Kraay, and M. Mastruzzi. Each indicator is a weighted average of its underlying data, with weights reflecting the precision of the individual data sources. The composite score depends on the available material for that year, for that country. Each indicator score is based on a worldwide average being 0.0, with scores ranging from -2.5 to 2.5, with higher scores corresponding to better outcomes. The methodology generates margins of error for the estimates for each indicator for each country, which needs to be taken into account when making comparisons across countries and over time. The Government Effectiveness Index measures the quality of public services, the quality of the civil service and the degree of its independence from political pressures, the quality of policy formulation and implementation, and the credibility of the government's commitment to such policies. Scholars and researchers have found that a country improving its quality of governance from a low level to an average level can in the long term quadruple the income per capita of its population, and similarly reduce infant mortality and illiteracy. The FY 00 and 05 actuals are an average of the government effectiveness rating for Egypt, Indonesia and Pakistan as reported by the World Bank in their Government Effectiveness Index and the 2015 target represents the average score for countries in the Transforming Country category. Jordan is not included in this indicator as it has already achieved a score of .08, surpassing the average for the next country category (Transforming). These countries will continue to report on this indicator. The direction of causality is clear: it goes from better governance to higher incomes, and not vice versa.

Measure: Number of sub-national government entities receiving USG assistance to improve their performance

Explanation:Entities refers to 'local governments' and their departments and divisions. Services on which they might be working to improve performance include public sanitation, public health, street lighting, regulation and operation of public markets, street or road maintenance, planning and regulation of land use. The quality of delivery of services is a fundamental measure of local government performance and this indicator relates to USG assistance to these entities to improve the quality of their services. This is not a cumulative indicator; actuals represent annual results only. This is a new indicator, and therefore, there is no specific past performance information, although related past performance information is discussed in question 4.2. Egypt, Indonesia, Jordan, and Pakistan contributed to the FY 07 and FY 08 targets for this indicator and will continue to report on this indicator. Direction of Change: Higher = Better

Measure: Number of USG-assisted courts with improved case management

Explanation:Improved case management can include such examples as: case documents made available to parties upon request; routine compilation of statistical data on cases; evidence that the system manages flow of cases through scheduling set by procedural law; coherent and compatible data elements with related information systems such as those of the police, prosecution and corrections agencies; and cases that are uniquely identified, registered, indexed and filed. Improved case management leads to a more effective justice system by decreasing case backlog and case disposition time, reducing administrative burdens on judges, increasing transparency of judicial procedures and improving compliance with procedural law. This is not cumulative because it is the number of USG-assisted courts with improved case management as a result of assistance in that fiscal year. If the assistance is not continued, it will not be counted the following year (although we anticipate improved case management to continue independently of our assistance as with the outcomes of any and all program activities.) This is a new indicator, and therefore, there is no specific past performance information although related past performance information is discussed in question 4.2. Egypt, Indonesia, and Jordan contributed to the FY 07 and 08 targets and will continue to report on this indicator. Direction of Change: Higher = Better.

Measure: % of U.S. Government Management Support Funds to Total U.S. Government Program Funds

Explanation:This efficiency indicator measures the ratio of funds used for management support to the total amount of program funds utilized in Developing Countries. Support functions include personnel, program design and learning activities (i.e. evaluations, assessments, and new activity design), outreach, contracting, and financial management.

Measure: Percentage of Indicative Benchmarks in the Financial Sector Memorandum of Understanding (MOU) for Non-Projectized Assistance met by the Government of Egypt.

Explanation:The U.S. Government and the Government of Egypt (GOE) have a memorandum of understanding concerning the disbursement of $800 million in FY 2003 - 2008 Development Support Program Economic Support Funds with specific performance indicators - called Indicative Benchmarks - linked to specific time frames. There are 19 indicators total, 16 of which were originally supposed to be completed in 2005 and three to be completed in 2008. Due to delays in the negotiating and finalization of the agreement, the target dates were known to be unachievable and were informally extended. Therefore 12 (of the 16) indicators that were scheduled for 2005 but completed in 2006 are considered adequate performance. The remaining four indicators originally scheduled for completion in 2005 are expected to be completed by the end of the MOU timeframe, i.e., 2008.

Measure: Percentage of condition precidents met by the Government of Jordan to receive non-projectized monies

Explanation:The major goals of the cash transfer program are the promotion of economic and political reforms. To accomplish this, the release of the cash transfer is based upon the Government of Jordan's (GOJ) achieving agreed-upon reform measures, expressed as condition precedents (CPs). The CPs support Jordan's economic and political restructuring process. The disbursement of the cash transfer program each year is conditioned on the GOJ meeting all of the CPs for that year. The Mission does not assign dollar figures to each CP, rather the GOJ has to meet all of the CPs and then receives all of the money. If not all of the CPs are met, the GOJ receives nothing USAID/Jordan has instituted a set of procedures to formulate, negotiate, and implement the CPs of the cash transfer program. Several principles guide the formulation of the CPs, as delineated below: ?? The CPs complement the goals of USAID/Jordan's programs and help to expedite implementation of reform agendas in economic opportunities, education, democracy and governance, water resources and the environment, and population and family health. ?? The CPs present difficult but achievable goals for the Government of Jordan within the fiscal year of the program. ?? The conditions do not replicate agreements already signed with other donors or international organizations, such as the IMF. ?? USAID/Jordan's technical assistance programs can be support the implementation of the CPs, if needed.

Is the program purpose clear?

Explanation: Yes. The purpose of providing foreign assistance to Developing Countries (the program) is to "continue progress in expanding and deepening democracy, strengthening public and private institutions, and supporting policies that promote economic growth and poverty reduction." Starting in FY 2006, the Department of State created the Office of U.S. Foreign Assistance in order to clearly outline USG-wide objectives for financial engagement overseas. This office produced a new Strategic Framework for U.S. Foreign Assistance. This framework clearly defines the purpose of assistance to Developing Countries, one of five Country Category designations that group countries according to common characteristics that make common goals clear. Developing Countries are those with low or lower-middle income, not yet meeting governance criteria set by the Millennium Challenge Corporation (MCC). Specific priorities are driven by individual country conditions and special attention is given to countries that are at risk for instability in order to avoid declines that could result in conflict and a less advanced state of development. This particular program is demonstrated in four countries in the Asia and Near East region: Egypt, Indonesia, Jordan and Pakistan. The program focuses on three broad objectives: Investing in People, Economic Growth, and Governing Justly and Democratically. The Assistance to Developing Countries program is administered jointly by the U.S. Agency for International Development (USAID) and the Department of State (State).

Evidence: The new Strategic Framework for U.S. Foreign Assistance is built around five priority objectives that support the overarching goal of helping move countries toward self-sufficiency and strengthened strategic partnerships. The framework includes five country categories based on shared characteristics to make common goals clear. (Foreign Assistance Summary Framework and Foreign Assistance Extended Framework, http://www.state.gov/f/direction/) The Developing Country Guidance includes the following: developing category definitions, goals, issues, and approaches used to help advance those countries that fall into this category along the transformational development trajectory to a Transforming Country. (Developing Country Category Guidance http://f.state.gov/framework.html)

Does the program address a specific and existing problem, interest, or need?

Explanation: Yes. Assistance to countries in the Developing Country category addresses the specific and existing problems, interests and needs that these countries must target in order to progress toward the overarching goal of graduating from a U.S. development assistance program: "sustained, well-governed states that will respond to the needs of their people, reduce widespread poverty and conduct themselves responsibly in the international system." Assistance to Developing Countries specifically addresses low per capita income and marginal economic performance, weak policy performance, corruption, and nascent political rights. Indonesia is a clear example of a developing country with a range of challenges: active international terrorists; ethnic and separatist conflicts; weak institutions; high levels of corruption; poverty and underemployment; and low levels of education and poor health conditions among the Indonesian population. To address these challenges, the U.S. government is engaged in a multi-year effort to enhance economic growth, just and good governance and investments in people, such as in education and health. As President Bush and Secretary Rice have repeatedly stated, our nation's security depends on the stability of other nations. Foreign assistance helps empower citizens worldwide to overcome the poverty and hopelessness our foes seek to exploit, and is therefore, a pillar of the U.S. national security strategy and Global War on Terror.

Evidence: The FY 2008 Congressional Budget Justification reflects a focus on the specific gaps and obstacles countries face in moving from one country category to another, and the identification of the target objectives appropriate to the individual country context. With the proper implementation of financial and human resources, it is believed that the USG goal of moving countries through the transformational development trajectory is possible. (FY 2008 Foreign Operations Congressional Budget Justification http://www.usaid.gov/policy/budget/cbj2008/). Examples of development indicators can be found in each country's "spider graph." Spider graphs provide a current snapshot of country progress across a handful of key dimensions in four of the five objectives in the new foreign assistance framework. The charts are intended to provide an empirical look at country progress to facilitate discussion on country needs across objectives as well as within objectives. Alongside other considerations (including government commitment, U.S. strategic priorities, U.S. assistance effectiveness, other donor activity, and political considerations such as earmarks) this discussion in turn is intended to facilitate USG resource allocation decisions. http://ppc.usaid.gov/esds/tracking_tools.cfm The data, primarily from the World Bank, UN, and Freedom House, were converted to values ranging from a "1" to "5" representing the best score possible worldwide.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: Yes. The new Office of the Director of U.S. Foreign Assistance developed processes to help ensure that there is no duplication of effort within the U.S. Government (USG) and between international donors. This Office has authority over State and USAID foreign assistance program resources and the ability to convene all USG agencies involved in foreign assistance. Using this authority, the Office integrates budgeting, planning, and implementation of assistance programs within the U.S. Government. At the headquarters level, other U.S. Government agencies, including the Millennium Challenge Corporation, Department of Defense, and Treasury, participate in budget allocation meetings and policy reviews to ensure that there is no duplication of effort. The reform process has also addressed redundancy and coordination through the development of integrated Operational Plans, in which State and USAID describe how they collaborate with other USG agencies and international donors. For example, in Pakistan, the Operational Plan process uncovered redundant programs being implemented by two embassy units, leading to better coordination and differentiation. Field offices avoid duplication by organizing country team meetings and donor working groups. In Jordan, the USG meets bi-annually with ten other bilateral and multilateral donors involved in education reform to review progress, identify emerging issues and gaps, and propose ways to overcome challenges.

Evidence: Through collaboration between USAID and the Department of State, guidance was issued to help operating units develop their FY 2007 Operational Plans. The guidance addresses how the units should take into account resources and activities from other U.S. Government agencies and other donors in planning activities. . (FY 2007 Operational Plan Guidance) An Operational Plan is an integral tool designed to link funding to activities and results in a particular country. By their design, they help to strengthen the role of leadership in the field in driving the allocation of foreign assistance; improve coordination, efficiency, and effectiveness; strengthen accountability; help the USG provide more accurate and consistent data on programs so that the USG can efficiently and effectively communicate to various stakeholders, including Congress, OMB, and the American public what funds have been directed to and what we are getting in return for our foreign assistance investments; and identify the essential links between U.S. policy objectives, resource allocation, and results. The Operational Plan for Jordan was reviewed in conjunction with this PART to help identify those areas of focus that will help the country progress to the Transforming Country category. (FY 07 Operational Plan for Jordan) With the creation of the Office of the Director of U.S. Foreign Assistance came the ability to fully integrate into one office all foreign assistance budgeting, planning, and implementation. With the Office of the Director of Foreign Assistance at the helm, there are increased opportunities for those agencies implementing foreign assistance to collaborate in order to accomplish maximum outcomes. (Testimony of Ambassador Randall L. Tobias, Director of U.S. Foreign Assistance and Administrator of USAID, Hearing on Foreign Assistance Reform and FY 2008 Budget, House Foreign Affairs Committee, March 8, 2007)

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: Yes. The Developing Countries assistance program is free of major flaws and is sufficiently flexible to allow a country program to implement activities deemed the most effective and efficient in the specific country context. Prior to foreign assistance reform, the processes related to program design, including analysis, resource availability and priority setting, were often fragmented across operating units and between field missions and Washington headquarters. Field missions submitted design proposals based on a given country context without full benefit of the global overview afforded by headquarters. Absent a uniform system for foreign assistance program analysis and design, critical information to determine the most effective programs and appropriate resource allocations was not always available. The Foreign Assistance Strategic Framework established priorities for Developing Countries, which are translated into annual Operational Plans at the country or other operating unit level. Prior to approval, the Office of the Director of U.S. Foreign Assistance leads an inter-agency technical and programmatic review of each Plan to identify possible design and implementation flaws and to determine whether the proposed activities will move the country along the transformational development and diplomacy path. Senior leaders then review the plan for alignment with overall foreign policy and assistance objectives. Operational Plans are approved by the Director of U.S. Foreign Assistance once all technical, program and policy-related issues and concerns are resolved. While the foreign assistance framework has only been in place for a year, there is already evidence that this new approach will be more effective in achieving the intended results of the program. To ensure that the program continues to be free of flaws throughout the year, inter-agency teams come together periodically to review proposed changes to the Operational Plans or to discuss new problems or unforeseen issues in a specific country or region. In addition, State and USAID also continually evaluate how to address limitations that may be imposed by Congress, such as earmarks and directives, to ensure that the program design continues to be effective. Evaluations and efficiency reviews assess whether other approaches would be more efficient or effective to achieve the ultimate purpose of moving the country to the next category, i.e., Transforming.

Evidence: With the creation of the Office of the Director of U.S. Foreign Assistance (State/F) came the ability to fully integrate into one office all foreign assistance budgeting, planning, and implementation. With State/F at the helm, there are increased opportunities for those agencies implementing foreign assistance to collaborate in order to accomplish maximum outcomes. (Testimony of Ambassador Randall L. Tobias, U.S. Director of Foreign Assistance and Administrator of USAID, Hearing on Foreign Assistance Reform and FY 2008 Budget, House Foreign Affairs Committee, March 8, 2007-http://www.usaid.gov/press/speeches/2007/sp070308.html) The Operational Plan review process assesses whether, within dollar levels provided to the operating unit for the objective, area, and element, the proposed programs maximizes the impacts on that functional objective and reflects best practices. Operational Plan reviews assess whether Bureaus are fulfilling their specific roles such as conducting research, providing technical leadership, and supporting the field. The overall goal of the Plans is to ensure that operating units are planning in accordance with the Secretary's Transformational Diplomacy goals and providing solutions when progress is being impeded. (Operational Plan Review Process Guidance and Templates) Evaluations are conducted to evaluate program effectiveness and to identify areas for improvement. All evaluations conducted at USAID are made available to the public at-large for review. Viewers are able to view how current programs fare and where recommendations are made for improvement. Evaluations and cost effectiveness studies are available at http://dec.usaid.gov/

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: Yes. The Developing Country program targets resources directly on the specific gaps and obstacles countries face in moving to the next country category. Prior to foreign assistance reform, resources were first allocated by account, then by sector and finally by country. Starting with FY 2008 budget development, the inter-agency process reversed that order, beginning with allocations to countries for the most critical interventions, and then to accounts. Teams of experts from USAID and State use independent indicators to plot the development gaps in each country and then crafted the Fiscal Year 2008 budget to address those gaps with the goal of improving the state of Developing Countries. As part of this process, operating units generated detailed Operational Plans that describe the activities that State and USAID will undertake to achieve the objectives of Developing Country programs, including the intended beneficiaries. USAID, as the main implementing agency, ensures that assistance reaches the intended beneficiaries and that the program achieves results through evaluations, routine performance reports submitted by grantees and contractors, audits, and field visits.

Evidence: The FY 2008 Foreign Operations Congressional Budget Justification reflects a focus on the specific gaps and obstacles countries face in moving from one country category to another, and the identification of the target objectives appropriate to the individual country context. "Spider graphs" are graphic illustrations of a country's performance during a particular year within strategic framework objectives, i.e. Governing Justly and Democratically, Investing In People and Economic Growth. Progress or regression on a spider graph provides an indication of whether the program is successful in reaching appropriate beneficiaries and progress is occurring along the transformational diplomacy trajectory (Fact Sheet: New Foreign Assistance Budget Process http://www.state.gov/f/releases/factsheets2006/75017.htm). The FY 2007 Operational Plan Guidance, P. 12, notes "host country needs" as the first critical question to be considered when preparing an Operational Plan. Grant applications require a needs statement to ensure that the activities target the proper beneficiaries. Evaluations are conducted to assess program effectiveness and to identify areas for improvement. All needs assessments and evaluations conducted by USAID are made available to the public at http://dec.usaid.gov.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: Yes. The U.S. Agency for International Development (USAID) and the Department of State use specific long-term measures that focus on outcomes and meaningfully measure both what is being accomplished with U.S. foreign assistance funds and the collective impact of foreign and host-government efforts to advance country development. This assessment includes a representative set of Developing Country measures for the four developing country programs highlighted to demonstrate outcomes that will be achieved over the next five to ten years. The long-term indicators measure progress in the areas where the bulk of the budget for the Developing Country program is invested: Investing in People (education, health), Economic Growth, and Governing Justly and Democratically. Two of the measures - net enrollment rate in primary education and under five mortality rate - are used by the international community to track progress towards achieving the Millennium Development Goals (MDGs). The other two long-term measures - number of days to start a business and the World Bank Government Effectiveness Index - are used by the Millennium Challenge Corporation (MCC), USAID and the Department of State to determine whether countries are moving along the transformational development trajectory. Uniquely, the Pakistan, Egypt, and Jordan programs include sizeable cash transfers to the host country - the objectives of which are high-level, foreign policy goals such as support for the Global War on Terror and regional stability and peace in the Middle East. These goals are not easily captured by standard, development-focused indicators. The impact of Pakistan's cash transfer will be reflected in the four long-term measures discussed above, as the bulk of the funding is going to improving governance, health, education, and private sector competitiveness with specific agreements tying the non-projectized assistance to improvements in these areas. Egypt's cash transfer is tied to specific benchmarks in financial sector reform that have been included as additional long-term performance indicators solely for Egypt in the Program Performance Measures Section. Jordan's assistance is also conditional on the completion of all "condition precedents" negotiated with the Government of Jordan each year. The condition precedents complement the goals of USAID/Jordan's programs and help to expedite implementation of reform agendas in economic opportunities, education, democracy and governance, water resources and the environment, and population and family health. A custom indicator for Jordan's cash assistance is also given in the Program Performance Measures Section.

Evidence: The long term goal for moving towards graduating from the Developing Country Category is described in the Foreign Assistance Framework and Extended Framework, http://www.state.gov/f/direction/, and Foreign Assistance Standardized Program Structure and Definitions, http://www.state.gov/f/direction/. The specific indicators and their explanation can be found at: http://www.state.gov/documents/organization/78558.pdf, http://www.state.gov/documents/organization/78557.pdf, and http://www.state.gov/f/releases/factsheets2007/78450.htm. The Millennium Development Goals are at http://www.undp.org/mdg/goallist.shtml while the Millennium Challenge Account indicators are here: http://www.mcc.gov/selection/indicators/index.php. The impact of Pakistan's cash transfer is demonstrated in the Action Memo for Congressional Notice for Pakistan. Egypt's cash transfer indicators come directly from the Memorandum of Understanding with the Government of Egypt. The condition precedents complement the goals of USAID/Jordan's programs and help to expedite implementation of reform agendas as explained in the Email Explaining Jordan Cash Transfer and the Matrix for Condition Precedents for FY06, FY06 Supplemental and FY07.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: Yes, the U.S. Agency for International Development (USAID) and the Department of State have set ambitious targets and timeframes for the long-term measures that represent very ambitious goals for the Developing Country program over a five to ten year time period. Two of these long-term measures - net enrollment rate in primary school and under five mortality rate - reflect the international community's and host countries' commitment to the Millennium Development Goals (MDGs). State and USAID based the targets for these indicators on the MDG goals of reaching 100 percent enrollment in primary school by 2015 and reducing by two-thirds, between 1990 and 2015, the under five mortality rate. As there is not an MDG goal related to the World Bank's Government Effectiveness indicator, a 2015 target of -.31 has been established as it is the average for countries in the Transforming Country category. The 2015 target for the long-term measure of Days to Start a Business, 18 days, is also ambitious. It reflects the average in 2006 for the Organisation for Economic Cooperation and Development (OECD) countries and has been chosen because of the rapid improvement the Developing Countries are experiencing in this area. In fact, the average in 2006 for the four countries was nearly the same as the Transforming and Sustaining Country category averages, which were 39.5 and 38.9, respectively. Meeting these targets will demonstrate significant progress toward moving away from the definition of a Developing Country and toward graduation from USG development assistance. Although these organizations do not set annual targets prior to 2015, the U.S. Government will track progress towards end targets as data is available on most of these measures annually. Operating units currently report on indicators that demonstrate outcomes attributable to U.S. foreign assistance and show the link between the annual, output measures and the long-term, outcome measures. Program managers use these mid-level indicators to assess whether the program is on track to meet the 2015 targets, making management changes if progress is lagging. The Office of the Director of U.S. Foreign Assistance has convened a State/USAID team to examine the most efficient and accurate way to track each operating unit's specific mid-level indicators within its Operational Plans, which in turn are based on the foreign assistance framework.

Evidence: The long term goal for moving towards graduating from the Developing Country Category is described in the Foreign Assistance Framework and Extended Framework, http://www.state.gov/f/direction/, and Foreign Assistance Standardized Program Structure and Definitions, http://www.state.gov/f/direction/. The specific indicators and their explanation can be found at: http://www.state.gov/documents/organization/78558.pdf, http://www.state.gov/documents/organization/78557.pdf, and http://www.state.gov/f/releases/factsheets2007/78450.htm. The Millennium Development Goals are at http://www.undp.org/mdg/goallist.shtml while the Millennium Challenge Account indicators are here: http://www.mcc.gov/selection/indicators/index.php.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: Yes. The Department of State and the U.S. Agency for International Development (USAID) have selected discrete, quantifiable, and measurable annual performance measures from the list of common indicators developed by the Office of the Director of U.S. Foreign Assistance to assess progress towards achieving the program goals established in the Foreign Assistance Framework. These annual performance measures, when combined with mid-level, impact indicators, indicate whether the Developing Country program is of sufficient scope and focus and on track to achieve the long-term targets discussed in questions 2.1 and 2.2. For example, the annual outcome indicator "number of cases of child diarrhea treated in USAID-assisted programs" and the annual output indicator "number of people trained in maternal and newborn health through USG-supported program" establish the necessary conditions to reach the long-term target of reducing the number of deaths among children under age five. Each country category has benchmarks for these indicators, which when combined with per capita income, determine whether a country is progressing toward graduating from USG development assistance altogether. The annual performance measures are reported on by operating units and measure outcomes and outputs that are directly attributable to the U.S. Government program. In addition to these annual indicators, each USAID Mission tracks mid-level impact indicators that highlight the link between annual and long-term measures. An example of a mid-level indicator is the percentage of USG-assisted mothers breastfeeding until two years of age, which links the annual indicator "number of people trained in maternal/newborn health" with the long-term indicator "number of deaths among children under age five." Although these mid-level indicators are not tracked by the Office of the Director of Foreign Assistance, the Office is examining the most efficient and accurate way to track each operating unit's specific mid-level indicators within its Operational Plans.

Evidence: The long term goal for moving towards graduating from the Developing Country Category is the Foreign Assistance Framework and Extended Framework, http://www.state.gov/f/direction/, and Foreign Assistance Standardized Program Structure and Definitions, http://www.state.gov/f/direction/. The specific indicators and their explanation can be found at: http://www.state.gov/documents/organization/78558.pdf, http://www.state.gov/documents/organization/78557.pdf, and http://www.state.gov/f/releases/factsheets2007/78450.htm. The Millennium Development Goals are at http://www.undp.org/mdg/goallist.shtml while the Millennium Challenge Account indicators are here: http://www.mcc.gov/selection/indicators/index.php.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: No. Most of the annual measures are being tracked for the first time starting in FY 2007. Therefore actual data from previous years is not available to add context to the targets. The Department of State and U.S. Agency for International Development use assessments of need, trusted data from other organizations, and/or performance data from on-going projects to determine USG engagement in an activity and develop programs around the gaps in need. The assessments used to benchmark USG targets for these measures were not provided. Targets also take into account other critical assumptions, contextual factors, past performance, other donors' actions, adequacy of staffing, status and timeline of needed inputs, the units set achievable, yet ambitious targets. To ensure that funding is going to achieve significant results, an inter-agency team of technical, strategy, and policy experts review the targets to ensure they are ambitious yet realistic. Operating units' activities are not approved until they set targets for their activities. In some cases, the baseline for a measure is zero because it is a new indicator reliant on USG attribution and FY 2007 will serve as the first actual data point, as noted in the explanations. For other annual indicators that track USG-supported outputs number of people served is not cumulative from one year to the next and therefore the actual data is a subset of the universe of those to be served. In either case, analysis of the current situation is used to formulate where to aim. Each year, annual indicator targets are established for the next two years. For nine indicators presented here, there is a baseline, other than zero, showing past actual data. With the new common indicators from the foreign assistance reform, six indicators do not have past data as they are new indicators - child diarrhea, child nutrition, core commercial laws, private business associations, sub-national government performance, and court case management.

Evidence: The FY 2007 Operational Plan Guidance describes the baseline and target selection process operating units used and the FY 2007 Operational Plan Review Process Guidance and Templates from Technical and Program Reviews explain how targets were reviewed to be ambitious yet realistic. USAID's Automated Directive System - ADS 200 Planning offers many helpful tools that are currently being re-written to accommodate the new foreign assistance framework however the basic principles of the learning cycle beginning with a needs assessment, target formulation and SMART (Specific, Measurable, Achievable, Realistic, Timely) indicators will remain the same. Related data is available for some of the new indicators with baselines of zero: In FY 05, USAID Indonesia reduced diarrhea - the second highest killer of children under five - by extending clean water and sanitation services to 71,675 people in unserved areas. In FY 06, this number rose to 191,940. In FY 05, food rations provided by USAID helped increase attendance rates at community-based health centers in Indonesia from 47% to 90%. Children's nutrition was given a boost in FY 04, as USAID well exceeded its target of 250,000 beneficiaries of food-for-work and supplemental feeding programs, reaching over 500,000 people. These accomplishments are largely responsible for the drastic reduction in the child mortality rate in Indonesia. While the number of core commercial laws is a new indicator, past performance information shows that the Developing Country program has had a measurable impact on creating an enabling environment for business. In Indonesia, USAID established 15 one-stop service centers for improved business registration, slashing time to register a business from 32 days to 13 and cutting costs by two-thirds. The Government of Jordan has accepted USAID recommendations for simplification of the tax code and fiscal sector reform. These recommendations have been incorporated into Jordan's National Agenda and will be used to guide the country's development strategy over the next ten years. In Egypt, USAID exceeded its FY 04 and 05 targets for World Trade Organization compliance, which is a proxy for economic liberalization and free trade. Performance information submitted in the annual report also shows that case management in Jordan was given a major boost in FY 05 with the establishment of an Arabic language-based court automation system in 60% of the courts in Amman. In addition, USAID exceeded its FY 04 and 05 targets for training judges to use the new system. In Indonesia, USAID had a major impact on government effectiveness by greatly exceeding their FY 04 target for the number of local governments using improved management tools and practices. These results are directly tied to the annual measure of supporting sub-national government entities and the long-term measure of increasing government effectiveness.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: Yes. The execution of a contract, grant, or cooperative agreement, in the Developing Country program requires that all program partners agree to commit to and work toward the specific annual and/or long-term program goals and provide regular performance reports that document their progress toward these goals. These goals directly align to the measures captured in this assessment. A Cognizant Technical Officer (CTO) is assigned for each contract, grant and cooperative agreement to closely monitor partners' performance and activities and ensure that they are committed to the goals of the program. CTOs hold partners accountable for performance and ensure that their activities are contributing to the program purpose, as described in questions 1.1 and 1.2. Results are verified by U.S. Agency for International Development (USAID) and Department of State employees through regular site visits, meetings with partners and stakeholders, evaluations, and audits.

Evidence: Request for Applications: Jordan-05-012 Community Based Initiatives for Water Demand Management articulates the goals the implementing partner agreed to. ADS 300 defines CTO responsibilities (ADS 302 for contracts and ADS 303 for grants), see http://www.usaid.gov/policy/ads/300/, to closely monitor partners' performance and activities to ensure that they are committed to the goals of the program. The Semi-Annual Report from USAID/Pakistan Contractor on Behind the Veil Project available at http://dec.usaid.gov/ is an example of a contracting partner's update which tied performance to the project's goals. The final evaluation from USAID/Indonesia's Biodiversity Project at http://dec.usaid.gov/ and the audit of Egypt's Agricultural Exports and Rural Incomes Project at http://www.usaid.gov/oig/public/aud_usaid.htm, illustrate how these partners' commitment to the program was realized.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: Yes. Independent evaluations of sufficient scope and quality are conducted on a regular basis to support program improvements and evaluate effectiveness and relevance to the needs of the Developing Countries program. As the primary implementing agency, U.S. Agency for International Development (USAID) policy ensures that timely and meaningful information is available to make management decisions. In recognition of the importance of rigorous and comprehensive evaluation, in 2005, USAID launched an initiative to revitalize evaluations. This initiative requires evaluations of all major programs and mandates that all operating units appoint a monitoring and evaluation officer, set aside funding for evaluations, create an annual evaluation plan, provide training in evaluation, and offer incentives to staff that encourage evaluations. Most evaluation work is done at the project and activity level where contracts and grants often require independent evaluations. There are often mid-term evaluations to determine the future course of projects, pointing out gaps or weaknesses in the approach so the program can be adjusted and achieve its objectives efficiently and effectively. Final evaluations assess the impact of a completed project, determine whether there are future opportunities or obstacles for assistance, and articulate lessons learned for future interventions. Evaluations are conducted by entities outside of USAID and State, such as private firms not directly associated with the activity or program. Regular audits by the Government Accountability Office and Office of the Inspector General provide additional, independent information on the impact of the USAID and State programs. In Developing Countries, USAID ensures that independent evaluations of programs across all the objectives in the strategic framework are conducted throughout the lifespan of the programs. For example, in the Governing Justly and Democratically Objective, Management Systems International conducted an independent evaluation of USG-funding for civil society strengthening in Indonesia. Chemonics International was the prime contractor, with four sub-contractors, none of whom were affiliated with the evaluator. The scope of work for the evaluation was ambitious and broad, requiring the evaluator to answer some 113 questions regarding the impact of the program to date, partners' performance, and improvements that should be made as implementation goes forward. In the Investing in People Objective, The Mitchell Group, Inc. conducted an independent final evaluation of USAID/Indonesia's Managing Basic Education program, which was implemented by RTI International. This comprehensive evaluation had the dual mandate of documenting results and impact, as well as informing future programming possibilities. In the Economic Growth Objective, USAID/Egypt hired Emerging Markets Group, Ltd to conduct an independent evaluation of the thirty-year utility assistance program, representing a $5.7 billion investment in Egypt's water and wastewater, power, and telecommunications sectors since 1975. Taken together, these three evaluations cover a large percentage of the Developing Country program and provide critical information of the effectiveness, relevance, and prospects of the program.

Evidence: The recommitment of USAID to evaluation in 2005 is described in Administrator Andrew Natsios' Cable FW: 05070811419/ State 127594 / Actions Required to Implement the Initiative. ADS 203.3.6 on evaluations discusses the importance of evaluations, when to evaluate, what to evaluate, evaluation methodologies, and learning from evaluations at http://www.usaid.gov/policy/ads/200/. ADS 202.3.9.4 on Conducting Audits can be found at http://www.usaid.gov/policy/ads/200/202.pdf. The independent evaluations cited - Mid-term Evaluation of Indonesia's Civil Society Strengthening Program, Final Evaluation of Indonesia's Managing Basic Education Program, and Evaluation of Egypt's Utility Projects - demonstrate that evaluations of sufficient scope and quality are conducted on a regular basis to support program improvements and evaluate the program's relevance.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: Yes. The U.S. Agency for International Development (USAID) is the primary implementing agency for this program, and with the exception of FY 2008, its Congressional Budget Justifications (CBJs) tied budget requests for each of its performance goals to a performance score. The performance scores were calculated utilizing various inputs from an array of disparate and disaggregated sources. Foreign assistance reform introduced a more streamlined, standardized and transparent performance management system to be phased in over a two-year period. This system requires each USAID and Department of State (State) operating unit to prepare an annual Operational Plan (OP) that reflects a uniform system of budget requests linked to standardized performance objectives, goals, measures and targets for the current and future years. The OP data is supported by a budget and performance database tracking system, the Foreign Assistance Comprehensive Tracking System (FACTS). Once aggregated across relevant operating units, the performance measures tied to requested funding reflect both the direct and indirect resources needed to achieve the entire program's performance goals. In addition, each Operational Plan (OP) addresses the impact of a 10% increase or decrease in funding. The transition to the OP system began in FY 2007, and as a result the linkage between performance data and resource needs was not fully available prior to submission of the FY 2008 Foreign Operations Congressional Budget Justification (CBJ).. In lieu of this data, the budget justification drew from performance data contained in the integrated FY 2008 Mission Performance Plans (MPPs) submitted by each of the program's field operating units. As the FY 2007 Operational Plans are now complete, a summary FY 2008 Foreign Assistance Performance Plan is being prepared which will include budget and performance data for this program. With FY 2008 Operational Plans scheduled for submission in early fall, the performance and budget data contained in these Plans will serve as the basis for the FY 2009 Congressional Budget Justification and will be incorporated into requests to the Office of Management and Budget.

Evidence: USAID's FY 2007 Congressional Budget Justification includes Country Profile Tables that tie budget requests for each of its performance goals (listed as "Objectives") to performance scores. FY 2008 Operational Plans for Egypt, Indonesia, Jordan and Pakistan link budget and performance data in a transparent and complete manner. The Foreign Assistance Tracking and Coordination System (FACTS) is the database that houses Operational Plan data for each operating unit, including the specific links between performance measures and budget requests. FY 2008 Mission Performance Plans are comprehensive, integrated planning documents prepared by each overseas U.S. mission.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: Yes. Prior to foreign assistance reform, the program was fragmented across numerous operating units within the U.S. Agency for International Development (USAID) and the Department of State. Assistance efforts lacked the coherence necessary for maximum impact, accountability could be difficult to measure, and resources were not always strategically tied to overarching goals. Under foreign assistance reform, Washington and the field focus on their respective strengths and responsibilities - integrated strategic direction and priorities across agencies set in Washington, and operational plans and tactics for the achievement of results developed and implemented by the field. The new Foreign Assistance Strategic Framework clearly defines the purpose of and priorities for Assistance to Developing Countries, one of five Country Category designations that group countries according to common characteristics that make common goals clear. In the past, State and USAID operating units in the same country submitted their own plans, to their own agencies, for their own operations??often on different timelines. This resulted in inconsistent opportunities, and little motivation, to compare programs across agencies and ensure that foreign assistance programs were comprehensive and coordinated. Under foreign assistance reform, USAID and State have prepared joint operational plans for each of the four countries that comprise this Developing Country program, namely Egypt, Indonesia, Jordan and Pakistan. The plans fully integrate USAID and State activities and resources that in turn link directly to the long-term and annual performance goals of the Assistance to Developing Countries program. Strategic planning deficiencies were corrected through the process of preparing the Plans; for example, in Pakistan the Operational Plan process uncovered redundant programs being implemented by two different embassy units, a discovery that lead to better coordination and differentiation between the two. As identified in 2.2 and 2.3, the Office of the Director of Foreign Assistance does not currently track all mid-level indicators that link annual measure to long-term performance measures. However, the Office of the Director of U.S. Foreign Assistance has convened a State/USAID team to examine the most efficient and accurate way to track each operating unit's specific mid-level indicators within its Operational Plans.

Evidence: The new Strategic Framework for U.S. Foreign Assistance is built around five priority objectives that, if achieved, support the USG overarching goal of helping move countries toward self-sufficiency and strengthening strategic partnerships. The framework includes five country categories based on shared characteristics to make common goals clear. (Foreign Assistance Summary Framework and Foreign Assistance Extended Framework, http://www.state.gov/f/direction/) Through a joint collaboration of efforts between USAID, the Department of State and the Office of the Director of Foreign Assistance a guidance manual was created to assist missions in their FY 2007 Operational Plans. The guidance provided a "how to process" on how missions should complete their operational plans and addressed frequently asked questions. (FY 2007 Operational Plan Guidance) Operational Plan is an integral tool designed to link funding to activities and results in a particular country. By their design, they help to strengthen the role of leadership in the field in driving the allocation of foreign assistance; improve coordination, efficiency, and effectiveness; strengthen accountability; help the USG provide more accurate and consistent data on programs so that the USG can efficiently and effectively communicate to various stakeholders, including Congress, OMB, and the American public what funds have been directed to and what we are getting in return for our foreign assistance investments; and identify the essential links between U.S. policy objectives, resource allocation, and results. (FY 2007 Operational Plans for Egypt, Indonesia, Jordan and Pakistan)

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: Yes. Credible and timely performance information is collected from partners and used to manage the program and improve performance. As specified in their contracts or grants, funding recipients are required to submit performance information to operating units on a regular basis (no more than quarterly, no less than annually). The program manager, the Cognizant Technical Officer (CTO), compares the reported performance results to the baseline data and milestones established in the program's performance management plan. The CTO meets with implementing partners to discuss performance issues and make adjustments as needed. Performance is also assessed periodically during portfolio reviews. For example, in its 2006 economic growth portfolio review, USAID/Pakistan reported on results to date, expected results in 2007, programmatic issues, management issues, and funding pipeline issues for senior management review and input. In addition, operating units conduct program evaluations to identify strengths and weaknesses, assess performance, and recommend improvements to the program's design, approach, or implementation. CTOs assess the credibility of performance information submitted by partners by conducting data quality assessments. In their annual reports, operating units use performance information from their partners to report on progress towards meeting the targets established in their performance management plan. When targets are not met, the contract or grant agreement may be modified, additional funds may be delayed or not provided, or management changes may be introduced.

Evidence: Credible and timely performance information is collected from partners and used to manage the program and improve performance. (Quarterly Report from USAID/Indonesia Contractor on Aceh Technical Assistance Recovery Project http://dec.usaid.gov/) The Operational Plan is an integral tool designed to link funding to activities and results in a particular country. By their design, they help to: strengthen the role of leadership in the field in driving the allocation of foreign assistance; improve coordination, efficiency, and effectiveness; strengthen accountability; help the USG provide more accurate and consistent data on programs so that the USG can efficiently and effectively communicate to various stakeholders, including Congress, OMB, and the American public the programs to which funds have been directed and what we are getting in return for our foreign assistance investments; and identify the essential links between U.S. policy objectives, resource allocation, and results. The Operational Plan for Jordan was reviewed in conjunction with this PART to help identify those areas of focus that will help the country progress to the Transforming Country category. (FY 2007 Jordan Operational Plan) A Mission Performance Management Plan is a critical tool for planning, managing, and documenting how performance data is collected and used. It defines specific performance indicators for each strategic objective, determines baselines, and sets targets. It is also used to plan and manage the Annual Report data collection process to meet quality standards, incorporates relevant data collection requirements into activities and obligation agreements, and communicates expectations to partner institutions responsible for producing the outputs intended to cause measurable changes in performance. (Performance Management Plan for USAID/Jordan's SO 9: Improved Social Sector Development and Governance) Additional evaluations used to assess program performance include: USAID/Pakistan 2006 Portfolio Review Economic Growth Issues Paper; Mid-term evaluation of USAID/Egypt's Information and Communication Technology Project http://dec.usaid.gov/ and Data Quality Assessment checklist (MS Word); An Annual Report assesses the country's performance against targets/goals established in the country's performance plan for a particular fiscal year. (USAID/Pakistan's FY 2006 Annual Report) In order to assess implementing partner performance and overall program performance, CTOs and Strategic Objective Teams routinely gather to discuss performance. ADS 202.3.6 on Monitoring Timeliness of Key Outputs outlines CTO responsibilities as they relate to monitoring contractor/grantee performance. (http://www.usaid.gov/policy/ads/200/202.pdf) Policies used to manage and assess assistance awards to grantees are found at 22 CFR 226 "Administration of Assistance Awards to US Non-governmental organizations," Section 226.51 "Monitoring and Reporting Program Performance" http://a257.g.akamaitech.net/7/257/2422/10apr20061500/edocket.access.gpo.gov/cfr_2006/aprqtr/pdf/22cfr226.51.pdf.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Yes. Project and grant managers are held accountable for program results under their purview through an annual performance evaluation and periodic program reviews and discussions. Agency employees are evaluated annually on their overall performance and skills, which have a direct bearing on the success of their programs. A Cognizant Technical Officer (CTO) is designated for each contract and grant, signing a standard designation letter which outlines their duties and states that failure to discharge these responsibilities can result in disciplinary action. CTOs are certified by passing formal training courses to ensure that they have the knowledge and skills to effectively manage contracts and grants. CTOs are responsible for holding partners accountable for performance and compliance with technical, cost, and schedule provisions of their award. When performance targets are not met or other accountability issues arise, managers and program partners work together to identify the source of the problem and needed remedial action. The contract or grant agreement may be modified, additional funds may be delayed or not provided, and/or management changes may be introduced. CTOs report to USAID Mission Directors, who are responsible for achieving program results in their country. Each Mission Director must complete an annual performance plan with work objectives and performance measures. These annual performance plans must include one work objective and one performance measure related to achieving key program results. The work objective is stated as follows: "Provide leadership and direction to Mission staff to identify and develop technical programs, activities and partners to accomplish foreign policy/transformational diplomacy objectives," and the performance measure must relate to how "Mission program implementation and management of approved activities meet or exceed annual program and management efficiency targets." A Mission Director's performance on this work objective and performance measure forms the basis for their annual performance evaluations which are, in turn, the basis for Performance Board decisions on promotion, limited career extensions, performance pay, Presidential Award and referrals to the Performance Standards Board, which determines whether the employee has failed to meet the standards of class. Findings of the Performance Boards are also one basis for tenure decisions. A Senior Coordinator at State is responsible for overseeing and coordinating assistance for the Developing country program. The Senior Coordinator is measured on his or her ability to coordinate processes with State and USAID regional bureaus to develop foreign assistance priorities and associated foreign assistance funding requirements for the Developing country category throughout the budget development process. Although Senior Coordinators are not responsible for program performance, Deputy Assistant Administrators (DAAs) at USAID are responsible for achieving program results at the regional level instead of the country category level.

Evidence: A Cognizant Technical Officer (CTO) is designated for each contract and grant, signing a standard designation letter that outlines their duties and states that failure to discharge these responsibilities can result in disciplinary action (AAPD 04-10 Standardized Model Letters for Designating the Cognizant Technical Officers for Contracts, Grants and Cooperative Agreements can be found at http://www.usaid.gov/business/business_opportunities/cib/pdf/aapd04_10_Att1.pdf for contracts and http://www.usaid.gov/business/business_opportunities/cib/pdf/aapd04_10_Att2.pdf for grants). Agency managers are required to pass a Cognizant Technical Officer course to ensure that they have the knowledge and skills to effectively manage contracts and grants (USAID's CTO Program Course Description, CTO Certification Requirement Announcement). ADS 303.2 (f) defines CTO responsibilities (http://www.usaid.gov/policy/ads/300/303.pdf). Performance is a major factor in selecting contractors and grantees (See Guidance on Evaluation and Use of Contractor Performance Information and ADS 303.3.6.3 http://www.usaid.gov/policy/ads/300/303.pdf for guidance on the use of past performance information in selecting grantees). Request for Applications Jordan 05-012 Community Based Initiatives for Water Demand Management shows that past performance is 15% of the score used for selection. A USAID Executive Message dated June 14, 2006, entitled "Standard Work Objective for Mission Director's Annual Evaluation Form," outlines the required performance objectives and measures (http://iapp1.usaid.gov/notices/notDetail.cfm?msgid=11509&currmo=6&curryr=2006&prevnext=no). USAID's Automated Directive System (ADS) Chapter 463 describes the foreign service Performance Board system.

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: Yes. As the primary implementing agency, the U.S. Agency for International Development (USAID) has a financial management system, Phoenix, which tracks obligations and enables transparent and accurate reporting of obligations down to the program element level. In Fiscal Year 2006, USAID's Asia and Near East Bureau and its operating units obligated 99 percent of that year's funds. Funds are provided on an incremental, as-needed basis so that operating units and headquarters can monitor disbursements. Agency policy stipulates that the amount of funds in the pipeline (funds obligated into an agreement but not expended by the partner) cannot exceed what is needed for the next 12 - 18 months. Regular portfolio reviews are conducted to ensure that partners are expending funds in a timely manner and for the intended purpose. For example, due to its portfolio review process, USAID/Pakistan established a corrective action plan when it found that a partner implementing a democracy program was expending funds too slowly. Finally, all operating units are required to prepare plans that outline upcoming procurements and obligations to ensure that funds are spent in a timely manner. Awards and modifications are reported in the Federal Procurement Data System - Next Generation.

Evidence: The U.S. Agency for International Development (USAID) has a financial management system, Phoenix, which tracks obligations and enables transparent and accurate reporting of obligations down to the program element level. (Sample Accruals Report Phoenix Report on Commitments and Obligations) The following policies in USAID's Automated Directive System/Employee Handbook provide directives on how to manage obligations and forward funding. (ADS 202.3.8.3 on Obligations Management http://www.usaid.gov/policy/ads/200/202.pdf; ADS 602 provides Agency guidance regarding forward funding of programs http://www.usaid.gov/policy/ads/600/602.pdf) Among other purposes, portfolio reviews are conducted to ensure that partners are expending funds in a timely manner and for the intended purpose. (USAID/Pakistan 2006 Portfolio Review Democracy and Governance Issues Paper) Procurement plans provide an outline for those services/goods a particular country will be seeking to obtain in order to carry out program activities to meet stated objectives. (USAID/Jordan 2006 Procurement Plan)

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: Yes. The foreign assistance reform process has included a rigorous review of the program's measures for achieving efficiencies and cost effectiveness. However, there is no over arching goal and purpose articulated behind this review which would better define the targets to be achieved. As the primary implementing agency for this program, USAID has undertaken a number of initiatives under the reform to improve cost effectiveness. These measures include: 1) improved financial forecasting through more frequent pipeline reviews (pipeline is defined as the calculation of financial obligations less accrued expenditures [accruals plus disbursements]); 2) adopting a new standard of Expanded Object Class Codes (EOCC), which links to performance goals and focuses on data gathering across all funds, providing more detail regarding the agency's administrative support costs than ever before; 3) implementation of the Foreign Assistance Coordination and Tracking System (FACTS) which houses program budget data linked directly to program goals; 4) implementing the Global Acquisition System (GLAS), a web-based system that standardizes and streamlines the agency's business processes by eliminating paper-based manual procedures, thus reducing costs agency-wide and enabling better managing and tracking of procurement decisions and contract supervision; and 5) implementing the Manage to Budget (MTB) initiative which strives to ensure the most efficient and effective administrative operations for the U.S. Agency for International Development. In launching the MTB, USAID conducted a comprehensive review of its program goal support costs, collecting prior year data from all operating units. USAID analyzed this data to determine the current average ratio of administrative costs to program costs of 12%. Prior to this exercise, USAID had not developed an agency-wide standard for administrative cost ratios. As a result of this analysis, USAID was able to set initial guidance that an operating unit's annual combined management support costs cannot exceed the current baseline average of 12% of the annual program budget. As USAID further analyzes this data, this 12% level threshold may need to be adjusted. Under MTB, managers must justify all management support expenses, as opposed to only explaining increases over the prior year's budget, defend any budgets exceeding the established budget ratio guidance, and present a narrative describing the impact of a 10% reduction from it. In the annual Operational Plans, field posts and other operating units must indicate the amount of general program funds associated with management support, a key factor in determining the program's efficiencies and cost effectiveness. As part of the Operational Plan approval process, this cost is reviewed and if necessary, adjusted by the Office of the Director of U.S. Foreign Assistance. Operating units must also provide quarterly reports on their progress toward target ratios, burn rates and significant changes. Managers are held accountable for their decisions through performance evaluations which include a measure that must relate to how Mission program implementation and management of approved activities meets or exceeds annual program and management efficiency targets. USAID also has a decentralized management structure which allows field managers to modify programs as needed in order to respond to rapidly changing circumstances, without headquarter's approval. USAID also uses annual portfolio reviews to identify ways to achieve efficiencies. For example, the October 2004 review of USAID's Asia Near East Bureau Washington-based programs resulted in the transfer of remotely-managed activities to the Bangkok regional office, cutting travel costs and improving oversight.

Evidence: USAID's OMB Presentation: Going to Green - Manage to Budget Implementation (March 28, 2007) and USAID Administrator's Memorandum for Senior Staff, USAID/Washington and Overseas re FY 2007 and FY 2008 Operational Budget Request Guidance (August 1, 2006)) USAID's Getting to Green Plan (March 15, 2007) outlined the steps requiring implementation within the agency in order to move the agency to "Green" status on the Presidential Management Agenda Scorecard and establish a system where managers are held accountable for their decisions and rewarded for managing an efficient operational unit and strategically aligning their program and management support resources. A cable outlined efficiencies to be achieved and duplicative efforts to be eliminated when the Department of State and USAID moved to cost-sharing for administrative support services. (Consolidating and Eliminating Duplication in Shared Administrative Support Services (State Cable, December 2005)) The Operational Plan review process was an evaluation tool used to assess whether, within dollar levels provided to the operating unit for the objective, area, and element, the proposed programs maximize the impacts on that functional objective, and reflects best practices. Operational plan reviews were also used to assess whether Bureaus were fulfilling such specific roles as conducting research, providing technical leadership, and supporting the field. Their overall goal was to ensure that operating units were planning in accordance with the Secretary's Transformational Diplomacy goals and providing solutions when flaws or progress was being impeded. (Egypt, Indonesia, Jordan and Pakistan FY 2007 Operational Plans)

Does the program collaborate and coordinate effectively with related programs?

Explanation: Yes. Since 2004, the U.S. Agency for International Development (USAID) and the Department of State have had a joint strategy that establishes common goals and ensures that all programs are contributing to U.S. foreign policy objectives. In 2006, the process for foreign assistance program coordination and collaboration was greatly enhanced with the creation of the Office of the Director of U.S. Foreign Assistance. To ensure that funds appropriated to USAID and the Department of State are properly targeted and coordinated, the Office instituted a new budget formulation process that resulted in the first joint USAID/Department of State foreign operations budget request to Congress, for Fiscal Year 2008. This led to a more strategic budget request as each agency discussed their plans and uncovered where they were duplicative and where they were complementary before attaching numbers. For example, in interagency discussions on Pakistan, security-related bureaus with rule of law programs met with USAID bureaus in charge of democracy and conflict mitigation and coordinated their FY 2008 requests to ensure their programs were complementary before putting forth their requests. At the country level, the Operational Plan process forced various agencies at post to discuss the right mix of resources and activities needed to achieve U.S. foreign policy objectives, resulting in a more strategic allocation of resources. For example, in Jordan, USAID and the U.S. Geological Survey work together in groundwater management to make the most of limited resources. Following the same principle, USAID divides water supply and sanitation tasks in Jordan with other donors, including Japan, Germany, and Sweden. The inter-agency review of Washington-based and field-based Operational Plans also ensured that the use of FY 07 resources would lead to the best outcomes. During their Operational Plan review, USAID's Asia and Near East Bureau had to prove that their proposed higher education activities will fill a niche that similar programs managed by the State Department do not in order to get funds for this activity.

Evidence: The Joint State-USAID Strategic Plan Fiscal Years 2004 - 2009 is a collaborative effort that sets forth the Secretary of State's direction and priorities for both organizations in the coming years. The Strategic Plan supports the policy positions set forth by President Bush in the 2002 National Security Strategy and presents how the Department and USAID will implement U.S. foreign policy and development assistance. (http://www.state.gov/s/d/rm/rls/dosstrat/2004/) An Operational Plan is an integral tool designed to link funding to activities and results in a particular country. By their design, they help to strengthen the role of leadership in the field in driving the allocation of foreign assistance; improve coordination, efficiency, and effectiveness; strengthen accountability; help the USG provide more accurate and consistent data on programs so that the USG can efficiently and effectively communicate to various stakeholders, including Congress, OMB, and the American public what funds have been directed to and what we are getting in return for our foreign assistance investments; and identify the essential links between U.S. policy objectives, resource allocation, and results. (FY 2007 Operational Plan for Jordan (pg. 131), see "Work of Other Players" and "Narrative of USG Participants" as examples of how USAID and State collaborate with other donors and other USG agencies)

Does the program use strong financial management practices?

Explanation: Yes. Under the Federal Managers Financial Integrity Act (FMFIA) and Management Control Review Committee (MCRC) process, the primary implementing agency, the U.S Agency for International Development (USAID), ensures that resources are protected against fraud, waste, and abuse and that they achieve the results for which funds were appropriated.. The process requires each operating unit to do an assessment of the adequacy of management controls in all areas of agency operations, including program, administrative, and financial management. Each operating unit within USAID's Asia and Near East Bureau (ANE) submits a FMFIA memo to ANE Bureau management, which in turn submits a consolidated memo to the USAID Administrator. Following the review of these memos by the MCRC, USAID's Administrator reports major deficiencies and plans to correct them to Congress and OMB via the Performance and Accountability Report. The Office of the Inspector General audits agencies financial statements every year and conducts financial-related audits of grantees and contractors. For the fourth year in a row, USAID and the Department of State have received clean (unqualified) audit opinions on its financial statements. USAID has significantly improved its accounting practices by rolling out its Phoenix financial management system to field-based missions. Financial data is now available worldwide in real time, allowing USAID to know its financial status at all times.

Evidence: Under the Federal Managers Financial Integrity Act (FMFIA) and Management Control Review Committee (MCRC) process, USAID ensures that resources are protected against fraud, waste, and abuse and that they achieve the results for which funds were appropriated. The following USAID Automated Directive System policies/USAID Employee Handbook outlines policies and procedures instituted to ensure strong financial management practices (ADS 620 chapter on Financial Managements Principles and Standards http://www.usaid.gov/policy/ads/600/620.pdf & ADS 596 Management Accountability and Control http://www.usaid.gov/policy/ads/500/596.pdf). A memo is prepared for USAID's Administrator by the ANE Bureau that provides details on the assessment of the adequacy of management controls in all areas of Agency operations, including program, administrative, and financial management. Following the review of these memos by the Management Control Review Committee, USAID's Administrator reports major deficiencies and plans to correct them to Congress and OMB via the Performance and Accountability Report. (ANE Bureau FMFIA Memo to Ambassador Tobias) Performance accountability reports (PAR) are one of several reporting requirements that are required for all Federal agencies under the Government Performance and Results Act (GPRA). Traditionally, PARs have enabled interested stakeholders such as the President, the Congress and the public at-large to assess the performance of an agency relative to its mission and demonstrate accountability. (USAID FY 2006 Performance and Accountability Report http://www.usaid.gov/policy/par06/) The following reports provide additional evidence of commitment to strong financial management practices and courses of actions taken in order to sustain sound financial management. (All IG audits available at http://www.usaid.gov/oig/public/aud_usaid.htm, for example see Audit of USAID/Indonesia's Banda Aceh-Lamno Road Reconstruction Activities, Phoenix Report on Commitments and Obligations and Report on the Audit of USAID's Financial Statements for Fiscal Years 2006 and 2005 http://www.usaid.gov/oig/public/fy07rpts/0-000-07-001-c.pdf)

Has the program taken meaningful steps to address its management deficiencies?

Explanation: The Department of State/U.S. Agency for International Development (USAID) Joint Management Council is charged with aligning and achieving the diplomatic and development priorities of the President, Secretary of State, and USAID Administrator, as the two agencies come together to build a common management foundation. Regularly bringing State and USAID together, it addresses financial management, human resources, information technology, procurement, and shared services. This body has overseen the consolidation of mission administrative support services, known as ICASS (International Cooperative Administrative Support Services System), leading to improved efficiencies and decreased duplication. In terms of program management, State and USAID use regular evaluations, audits, site visits, and meetings with implementing partners to identify management weaknesses and rectify deficiencies. Additionally, USAID's Business Transformation Executive Committee (BTEC) is made up of senior officials who meet monthly to discuss progress and obstacles to reforming USAID's management systems and improving organizational performance. Through the BTEC's efforts, USAID improved financial management accountability and received its second consecutive unqualified (clean) audit opinion from the Office of the Inspector General in 2004 mentioned in 3.6. The BTEC was the body that led to the Global Acquisition System (GLAS) improvement noted in answer 3.4 due to identified inconsistencies in procurement. In addition, USAID's Asia and Near East Bureau prepares an annual memo for USAID's Administrator that assesses the adequacy of management controls in all areas of Agency operations, including program, administrative and financial management. Following the review of these memos by USAID's Management Control Review Committee, USAID's Administrator reports major deficiencies and plans to correct them to Congress and OMB via the Performance and Accountability Report. The financial management system, Phoenix, mentioned in 3.6 was a direct result of this procedure through which the Agency found that the lack of a common financial management system was a material weakness. USAID is required to report to OMB, on an annual basis, 100% of funding (Program and Operating Expense) that is allocated by OMB mandated class codes. The Agency has never fully complied with this directive, because the Agency did not have an accounting system that could systematically capture the data. Similarly, USAID/Washington had no systematic way of tracking what missions were spending in administrative expenses across both program and OE. This made it difficult to compare agency administrative performance and set targets for administrative cost reductions. With Phoenix deployed worldwide, the Agency can now capture all financial actions in Phoenix using the new code structures, or Expanded Object Class Codes (EOCC), cited in 3.4. While the Phoenix financial management system provides the Agency's financial operating platform, the Agency's overarching goal is to streamline operations and reduce costs throughout the entire Agency. Therefore, USAID is implementing the Global Acquisition System (GLAS), which will bring the Agency another step closer to achieving its goal by providing an end-to-end automated procurement process that will address multiple acquisition and procurement challenges faced by the Agency. USAID has been struggling under ever tightening operating expense budgets that are reducing the ability of USAID to effectively manage program resources and deliver foreign assistance consistent with foreign assistance needs. The Manage To Budget initiative described in 3.4 is intended to permit the Agency to address in part the declining operating resources by devolving responsibility for achieving cost reductions to mission managers, who are then provided with strong incentives to develop innovative solutions.

Evidence: The Joint Management Council regularly brings State and USAID together to address common management issues including financial management, human resources, information technology, procurement, and shared services. Their website is: http://jmc.state.gov/. Evaluations are conducted to evaluate program effectiveness and to identify areas for improvement. All evaluations conducted at USAID are made available to the public at-large for review. Viewers are able to view how current programs fare and where recommendations are made for improvement. Evaluations and cost effectiveness studies are available at http://dec.usaid.gov/ and IG audits are at http://www.usaid.gov/oig/public/aud_usaid.htm. Management deficiencies are also addressed through USAID's Business Transformation Executive Committee (BTEC). This committee, made up of senior officials, meets monthly to discuss progress and obstacles to reforming USAID's management systems and improving organizational performance as noted in the Welcome to the BTEC Website, a message from the Deputy Administrator and Chairman of USAID's Business Transformation Executive Committee, http://inside.usaid.gov/BTEC/misc/welcome.html. The FMFIA Memo submitted by the Asia and Near East Bureau to the USAID Administrator provides details on the assessment of the adequacy of management controls in all areas of Agency operations, including program, administrative, and financial management. Following the review by the Management Control Review Committee of this memo and those submitted by other Bureaus, USAID's Administrator reports major deficiencies and plans to correct them to Congress and OMB via the Performance and Accountability Report, also submitted. Many of the challenges and proposed solutions from question 3.4 are outlined in USAID's Getting to Green Plan, March 15, 2007 include Phoenix, Manage to Budget, and Expanded Object Class Codes.

Are grants awarded based on a clear competitive process that includes a qualified assessment of merit?

Explanation: Yes. Federal and Agency regulations require grants and contracts to be awarded through a clear, competitive process. Requests for proposals are posted at www.fedbizopps.gov. Assistance programs are announced in the Catalog of Federal Domestic Assistance (www.cfda.gov), and requests for applications and annual program statements are posted at www.Grants.gov. All solicitations include selection criteria. USAID is the primary implementing agency for this program, and a significant majority of USAID's grants and contracts (74%) are awarded based on a competitive process through which an independent panel scores and ranks proposals. Exceptions are made according to federal and USAID procurement regulations when full and open competition would impair programs, when urgency is critical, or when an organization is the only one suited to undertake the work. Justifications for Other Than Full and Open Competition and unsolicited proposals demonstrating a unique, innovative, and propriety capability must be documented and approved by officials with the authority to approve non-competitive awards. USAID's Office of Acquisition and Assistance reviews and provides oversight to ensure adherence to federal regulations and Agency guidance on competition. USAID has led a major push in the past few years to reach out to new and unconventional partners, including faith-based organizations, small businesses, and minority serving institutions. In an effort to expand the range of implementing partners, all operating units are required to justify a partner receiving more than 15% of the program budget, which is then approved by the Director of U.S. Foreign Assistance through the Operational Plan approval.

Evidence: Federal and Agency regulations require grants and contracts to be awarded through a clear, competitive process. Requests for proposals are posted at www.fedbizopps.gov. Assistance programs are announced in the Catalog of Federal Domestic Assistance (www.cfda.gov), and requests for applications and annual program statements are posted at www.Grants.gov. All solicitations include selection criteria. The following evidence lays out federal and agency acquisition and assistance processes. ADS 300 provides overall guidance on procurement, including competition requirements (ADS 302 for Contracts and ADS 303 for Grants http://www.usaid.gov/policy/ads/300) A sample request for proposals includes No. 263-06-01, Technical Assistance for Micro enterprise activity in USAID/Egypt at www.fedbizopps.gov. A significant majority of USAID's grants and contracts are awarded based on a competitive process through which an independent panel scores and ranks proposals. (AIDAR Part 706 - Competition Requirements www.usaid.gov/policy/ads/300/aidar.pdf) Exceptions are made according to federal procurement regulations when full and open competition would impair programs, when urgency is critical, or when an organization is the only one suited to undertake the work. For example, USAID/Pakistan's exception from full and open competition was approved for an unsolicited proposal from an organization with a unique ability to improve child health in the FATA. USAID's Office of Acquisition and Assistance reviews and provides oversight to ensure adherence to federal regulations and Agency guidance on competition. (ADS 202.3.9 on Avoiding Conflict of Interest, Ensuring Procurement Integrity, Complying with Ethics Rules, and Meeting Audit Responsibilities http://www.usaid.gov/policy/ads/200/202.pdf) USAID has led a major push in the past few years to reach out to new and unconventional partners, including faith-based organizations, small businesses, and minority serving institutions. (AAPD 03-10, issued October 31, 2003, "Prohibition on Requirement for Prior USAID-Specific Experience in Evaluation Criteria for Award of Agency A&A Instruments" & Creating Opportunities for USAID: http://www.usaid.gov/our_work/global_partnerships/)

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: Yes. A Cognizant Technical Officer (CTO) is designated for each contract and grant to audit grantee performance and assess compliance with the technical, price, and schedule provisions of their award. CTO responsibilities include reviewing and approving vouchers, monitoring contractor/grantee performance, conducting site visits, tracking financial expenditures to ensure that funds are used for their designated purpose, and overseeing sub-awards. Oversight is reinforced through a Strategic Objective Team approach, whereby a team routinely comes together to discuss contractor and grantee performance and resolve performance or technical issues. Operating unit management is kept informed of program performance, progress, and issues via periodic portfolio reviews. For example, during the 2006 annual portfolio review at the U.S. Agency for International Development mission in Pakistan, each Strategic Objective Team presented to senior management the results of their programs, expected results for 2007, programmatic issues, management issues, and funding pipeline issues. Headquarters provides another layer of oversight through the annual report process, while audits, evaluations, and site visits enable sound oversight.

Evidence: Cognizant Technical Officers have a number of duties and responsibilities. These duties and responsibilities are specifically outlined in USAIDs Automated Directive System/Employee Handbook. (ADS 303.3 defines CTO responsibilities http://www.usaid.gov/policy/ads/300/) In order to assess implementing partner performance and overall program performance, CTOs and Strategic Objective Teams routinely gather to discuss performance. ADS 202.3.6 on Monitoring Timeliness of Key Outputs outlines CTO responsibilities as they relate to monitoring contractor/grantee performance. (http://www.usaid.gov/policy/ads/200/202.pdf) Annual Reports assesses the country's performance against targets/goals established in the country's performance plan for a particular fiscal year. (USAID/Indonesia's 2006 Annual Report) Audits conducted by USAID's Office of the Inspector General (OIG) are used to provide insight into how well the agency is operating, making recommendations when problems are found and acknowledging when things are going well. ADS 202.3.9.4 on Conducting Audits provides explanations on the purpose of USAID OIG audits and the types of audits performed. (http://www.usaid.gov/policy/ads/200/202.pdf) Evaluations are available at http://dec.cdie.org, and IG audits are available at http://www.usaid.gov/oig/public/aud_usaid.htm, for example see the Audit of Egypt's Agricultural Exports and Rural Incomes Project.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: Yes. Grantees and other funding recipients are required to submit performance information at least once a year. The U.S. Agency for International Development (USAID) and the Department of State report on performance data in their annual Performance and Accountability Report, which is available to the public in print and on the agencies' internal and public websites. The report is also distributed in hard copy to Congress and the Office of Management and Budget. Program evaluations and contractors' and grantees' quarterly, semi-annual, annual, and final reports are also available to the public through USAID's Development Experience Clearinghouse online database. Inspector General audits are also published, allowing the public to read objective assessments of performance.

Evidence: Performance accountability reports (PAR) are required for all Federal agencies under the Government Performance and Results Act (GPRA). Traditionally, PARs have enabled interested stakeholders such as the President, the Congress and the public at-large to assess the performance of an agency relative to its mission and demonstrate accountability. (USAID FY 2006 Performance and Accountability Report http://www.usaid.gov/policy/par06/) Evaluations are conducted to evaluate program effectiveness and to identify areas for improvement. All evaluations conducted at USAID are made available to the public at-large for review. Viewers are able to view how current programs fare and where recommendations are made for improvement. Evaluations are available to the public on http://dec.usaid.gov/, for example see Assessment of USAID's Global Development Alliances in the Bureau for Asia and the Near East. Additional reports assessing agency's program progress include: Semi-Annual Report from USAID/Pakistan Contractor on Behind the Veil Project available at http://dec.usaid.gov/ and Audit of Egypt's Agricultural Exports and Rural Incomes Project available at http://www.usaid.gov/oig/public/aud_usaid.htm.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: The long-term measures not only show progress towards achieving the 2015 targets but also indicate that the Developing Country program is of sufficient scope and focus to advance Egypt, Indonesia, Jordan, and Pakistan toward graduation from U.S. development assistance. As the criteria for graduation include per capita income and meeting the Millennium Challenge Account thresholds for Ruling Justly, Economic Freedom, and Corruption, making progress towards the 2015 targets shows that a country is making progress towards graduating. Specifically, the Developing Country program is demonstrating notable progress towards achieving three of the four long-term performance targets. The net enrollment rate in primary education has increased from 71% in 1991 to 85% in 2004 and is well on its way to achieving the Millennium Development Goal (MDG) of 100% enrollment by 2015. Under five mortality has declined from 108 per thousand in 1990 to 56 in 2005, indicating that the MDG of reducing under five mortality by two-thirds by 2015 is well within reach. In the area of economic freedom, the business enabling environment is steadily improving as evidenced by the decline in the number of days to start a business from 60 in 2004 to 40 in 2006. The World Bank Government Effectiveness Index shows a slight decline in effectiveness from 2000 to 2004; however, the methodology may be partially the cause as the index was comprised of different surveys between those years. Proof that the Developing Country program is making significant progress toward these long-term targets can also be found in the U.S. Agency for International Development (USAID) award-winning Performance and Accountability Report (PAR). This report shows that USAID consistently meets its annual targets for enrollment rate and child mortality. It also shows great progress in improving the business enabling environment - which would mean a reduction in the number of days to start a business - through indicators that measure the incorporation of Millennium Challenge Account goals into United Nations programs and the number of companies for whom advocacy services are provided. Finally, the PAR demonstrates strong performance in improving democratic principles and institutions in Pakistan and the Near East and reducing corruption, which is directly related to increasing government effectiveness. USAID and State will know whether the Developing Country program is on track to meet its ambitious 2015 targets by assessing mid-level impact measures for which the U.S. Government can claim credit. Operating units establish mid-level indicators which provide the link between the annual output measures and long-term outcome measures. Program managers will use the performance data from these mid-level indicators to make management changes if the Developing Country program is not on track to meet its targets. Although these mid-level indicators are not tracked by the Office of the Director of U.S. Foreign Assistance, the Office is examining the most efficient and accurate way to track each operating unit's specific mid-level indicators within its Operational Plans.

Evidence: Proof that the Developing Country program is making significant progress toward these long-term targets can also be found in the U.S. Agency for International Development (USAID) award-winning Performance and Accountability Report (PAR) http://www.usaid.gov/policy/par06/. This report shows that USAID consistently meets its annual targets for enrollment rate and child mortality. It also shows great progress in improving the business enabling environment - which would mean a reduction in the number of days to start a business - through indicators that measure the incorporation of Millennium Challenge Account goals into United Nations programs and the number of companies for whom advocacy services are provided. Finally, the PAR demonstrates strong performance in improving democratic principles and institutions in Pakistan and the Near East and reducing corruption, which is directly related to increasing government effectiveness.

Does the program (including program partners) achieve its annual performance goals?

Explanation: Small Extent. The annual performance measures are directly linked to the long-term measures, and when combined with the mid-level impact indicators, tell whether the program is on track to meet its long-term, ambitious performance targets. Although most of the annual indicators are new, and therefore do not have past performance data, qualitative and quantitative data from the operating units' annual reports shows strong performance in improving child health, creating a more conducive business enabling environment, and strengthening government effectiveness. The Developing Country program regularly achieved, and often exceeded, its past annual performance targets. For example, the annual measure - number of learners enrolled in USG-supported primary schools - well exceeded its FY 06 target of 733,070 with 1.214 million enrolled. The number of people trained in maternal/newborn health rose from 13,493 in 2005 to 16,707 in 2006, although it did not meet its FY 06 target of 19,291, as Egypt experienced procurement delays. Performance information in the annual report reflects implementing partners' accomplishments. For example, in FY 05, the U.S. Agency for International Development's (USAID) Indonesia program reduced diarrhea - the second highest killer of children under five - by extending clean water and sanitation services to 71,675 people in unserved areas. In FY 06, this number rose to 191,940. In FY 05, food rations provided by USAID helped increase attendance rates at community-based health centers in Indonesia from 47% to 90%. Children's nutrition was given a boost in FY 04, as USAID well exceeded its target of 250,000 beneficiaries of food-for-work and supplemental feeding programs, reaching over 500,000 people. These accomplishments are largely responsible for the drastic reduction in the child mortality rate in Indonesia. While the number of core commercial laws is a new indicator, past performance information shows that the Developing Country program has had a measurable impact on creating an enabling environment for business, which will ultimately reduce the number of days it takes to start a business. In Indonesia, USAID established 15 one-stop service centers for improved business registration, slashing time to register a business from 32 days to 13 and cutting costs by two-thirds. USAID-supported economic reforms in Jordan continue to achieve remarkable results. Thus far, the Government of Jordan has accepted USAID recommendations for simplification of the tax code and fiscal sector reform. These recommendations have been incorporated into Jordan's National Agenda and will be used to guide the country's development strategy over the next ten years. In Egypt, USAID exceeded its FY 04 and 05 targets for WTO compliance, which is a proxy for economic liberalization and free trade. Although the indicator - number of USG-assisted courts with improved case management - is new, performance information submitted in the annual report shows that case management in Jordan was given a major boost in FY 05 with the establishment of an Arabic language-based court automation system in 60% of the courts in Amman. In addition, USAID exceeded its FY 04 and 05 targets for training judges to use the new system. As a result of USAID's technical assistance in Jordan, the Ministry of Industry and Trade won the prestigious King Abdullah II award in 2005 for excellence in government performance. In Indonesia, USAID had a major impact on government effectiveness by greatly exceeding their FY 04 target for the number of local governments using improved management tools and practices. These results are directly tied to the annual measure of supporting sub-national government entities and the long-term measure of increasing government effectiveness.

Evidence: Qualitative and quantitative data from USAID/Indonesia's 2006 Annual Report, USAID/Jordan's 2006 Annual Report and USAID/Egypt's 2006 Annual Report shows strong performance in education, child health, creating a more conducive business enabling environment, and strengthening government effectiveness.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: No. A key objective of foreign assistance reform is to improve efficiencies and cost effectiveness in achieving program goals. Under the processes described in 3.4, a standard administrative cost ratio of 12% has been established as the initial target for each operating unit implementing the program. Because this is a new baseline, USAID is unable to identify cost savings from improved efficiency initiatives at this time.

Evidence: Guidance on the target efficiency ratio is in the FY 2007 Operational Plan Guidance and USAID Administrator's Memorandum for Senior Staff, USAID/Washington and overseas re FY 2007 and FY 2008 Operational Budget Request Guidance (August 1, 2006). Evidence of the program's commitment to improved efficiencies is in USAID's OMB Presentation: Going to Green - Manage to Budget Implementation (March 28, 2007) and USAID's Getting to Green Plan (March 15, 2007). As discussed in 3.4, the Consolidating and Eliminating Duplication in Shared Administrative Support Services State Cable in December 2005 sought to decrease inefficiencies by combining administrative services for the two bodies and USAID's Asia and Near East Bureau's 2004 Portfolio Review Final Narrative and Activity Table documented a transfer of remotely-managed activities to the Bangkok regional office, cutting travel costs and improving oversight.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: Yes. The Developing Countries program performs favorably in comparison to other government and donor programs with similar purposes and goals. Because of its strong field presence and flexible programming, the U.S. Agency for International Development (USAID) is often cited by other donors and host governments for its ability to respond quickly to challenges and opportunities and its leadership of coordination with other donors. The Paris Declaration on Aid Effectiveness highlights the importance of devolving responsibility to field staff, the hallmark of USAID's programs. The United States is also one of the only donors that has a stringent, auditable performance management system that monitors and can report on results with good quality data. In its 1998 peer review, the Development Assistance Committee (DAC)-Organization for Economic Cooperation and Development (OECD) states that, "among DAC Member-agencies, USAID has been one that has placed great emphasis in assessing its aid effectiveness, measuring and evaluating program performance, (including gender issues) and using this information "to manage for results." Most donors are unable to track their performance because they conduct development assistance through budget support. Recognizing USAID's expertise in program management, other U.S. government agencies regularly ask USAID to manage their foreign assistance funds. USAID and the Department of State are often sited for their comparative advantage in implementing programs in high-threat environments because of the security resources at hand and on-the-ground presence. For example, in Pakistan, USAID successfully implements health and education programs in the FATA by using local and international contractors and grantees. This remote area is inaccessible to most donors with less experienced and embedded partners. USAID and the Department of State routinely seek out the best entity to implement foreign assistance programs, recognizing that different organizations have different strengths. Through full and open competition, USAID ensures that the implementing partner is the best-positioned to achieve the program's objectives. USAID also recognizes the private sector's comparative advantage in providing assistance in business-related sectors, and therefore, routinely partners with private sector and non-profit organizations to leverage funds and increase the program's impact. In addition, USAID and State transfer funds to another U.S. Government Agency if that Agency is better positioned to implement the activity, such as to the Army Corps of Engineers for infrastructure programs.

Evidence: The OECD-DAC Peer Review of U.S. Foreign Assistance 2006 (http://www.oecd.org/dataoecd/61/57/37885999.pdf) and 1998 (http://www.oecd.org/document/5/0,2340,en_2649_201185_2094661_1_1_1_1,00.html) cite the U.S. Agency for International Development for its strong field presence and flexible programming. USAID and the Department of State are often sited for their comparative advantage in implementing programs in high-threat environments because of the security resources at hand and on-the-ground presence; see Operating in High-Threat Environments Report.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: Large Extent. Independent evaluations of the Developing Country program that cover a large percentage of the program and the relevant objectives - Governing Justly and Democratically, Investing in People, and Economic Growth - indicate that it is effective and achieving results. For example, in Governing Justly and Democratically, the mid-term evaluation of Indonesia's Civil Strengthening Program (CSSP) found that the program met or exceeded the program's targets, despite the daunting challenges. In addition to enhancing the capacities of Indonesian non-governmental organizations, the program had a measurable impact on improving advocacy - a "significant achievement and one that would not have happened or at best taken a lot longer in the absence of a program like CSSP." The comprehensive evaluation of the Managing Basic Education (MBE) program in Indonesia proves that the program is advancing the goal of Investing in People by improving the quality of basic education. Evaluators found that, "on all accounts, the MBE project has had impressive impacts on district and school management, community involvement, and teaching and learning. When compared to the average Indonesian school, MBE-assisted schools are better managed institutions, with active community participation, richer learning environments, and teachers with a wider variety of teaching approaches and materials." The evaluation documented dramatic, visible change in school facilities; marked increase in energy and enthusiasm in classrooms; improved school management; very active parental participation; and engaged school communities and community leaders. In Economic Growth, an independent evaluation of USAID's thirty-year, $5.7 billion investment in Egypt's infrastructure has had a profound impact on industrial and commercial development, private sector engagement, and public health. The evaluators concluded that this program has benefited nearly the entire Egyptian population across social and economic strata. Taken together, these evaluations demonstrate that the Developing Country program is achieving results effectively and efficiently. These examples indicate the program is effective and achieving results in the five objective areas for these countries. Progress on these objectives will enable these countries to transition to the Transforming Category, and in turn progress towards graduating from U.S. development assistance.

Evidence: The Mid-term Evaluation of USAID/Indonesia's Civil Society Strengthening Program found that the program has met or exceeded the targets set for the strategic objectives and enhanced the capacities of Indonesian non-governmental organizations. The evaluation of the Managing Basic Education (MBE) program in Indonesia shows remarkable progress in improving the quality of basic education, with dramatic changes in the learning environment, teacher preparedness, administration and parental participation, and school facilities. The Evaluation of USAID/Egypt's Utility Projects demonstrates that the U.S. Government's thirty-year investment in water, power, and telecommunications sectors has had a profound impact on economic development in Egypt and the well-being of the majority of Egypt's citizens. These evaluations meet the quality, scope, and independence criterion discussed in question 2.6 and cover the bulk of the funding covered in this assessment.