Detailed Information on the
Africa Child Survival and Health Assessment

Developing an efficiency measure for Africa health programs that demonstrates a commitment to programming funds at a lower cost for either services, commodities, or total overhead.

Planning evaluations that over a 5-8 year time provide a comprehensive picture of the performance of the Africa health programs.

Develop and implement a common framework (set of output, outcomes and impact indicators) for monitoring and reporting on the revitalized maternal and child health initiative.

Putting the new Strategic Framework in place so that better performing countries receive substantially more dollars and human resources than poorer performers.

Measure: DPT 3 Coverage Rate (%).

Explanation:Percentage of children who have received a third dose of DPT by one year of age. Indicates sustainable use of immunivation services. Source: WHO

Measure: Under-five mortality rate.

Explanation:Number of deaths amoung children under age five in a given year per 1,000 live births in that same year. Indicates long-term impact on child survival. Source: DHS and MICS

Measure: Modern contraceptive Prevalence Rate (%).

Explanation:Percentage of all women of reprodcutive age (15-49) using, or whose partner is using, a modern method of contraception at the time of the survey. Indicates use of birth spacing and family size limiting. Sources: DHS and MICS

Measure: HIV Prevalence Rate.

Explanation:Percentage of blood samples taken from women aged 15-24 that test positive for HIV during routine sentinel surveillance at selected antenatal clinics. Indicates progress in preventing transmission. Sources: UNAIDS and DHS

Measure: ITN Coverage Rate (%).

Explanation:Percentage of households in malarial areas that own at least one insecticide treated bednet. Indicates reach of the malaria program. Sources: DHS and USAID program reporting

Measure: The cost in dollars of delivering an impregnated bednet.

Explanation:This indicator looks at an "output efficiency" for the President's Malaria Initiative (PMI) in Africa. Source: PMI program data.

Is the program purpose clear?

Explanation: The Child Survival and Health programs in Africa have a broad-based mission as stated in the new Africa Bureau Framework: to foster a healthier African population and to increase the effectiveness of African institutions. The Africa Bureau has several key priority health areas. These have remained relatively consistent over time, both in the recent past and going forward in the new Africa Framework. They include: HIV/AIDS (61% of FY2004 funding), family planning and reproductive health (15%), child survival and maternal health (13%), malaria and other infectious diseases (10%).

Evidence: "Secretary of State Statement in Bureau Performance Plans (BPPs), p.5-8 2) U.S. Department of State and U.S.Agency for International Development Strategic Plan: Fiscal years 2004-2009. p.18-25 3) Assistant Administrator Pierson testimony to SFRC, March 2, 2005, p.3-5 4) FY 2006 Congressional Budget Justification (CBJ) Africa overview 5) H.R. 4818 Consolidated Appropriations Act, 2005: CSH (entire document) "

Does the program address a specific and existing problem, interest, or need?

Explanation: "The CSH program addresses a specific need and specific problems within the AFR region. The CSH program in the AFR region supports U.S. foreign policy interests while addressing country level needs. Africa is the world's poorest region and the development challenges are diverse. For example, the lifetime risk of maternal mortality is highest in Africa at 1 in 16 compared to 1 in 2,800 in high income countries. 60 percent of women give birth without a skilled birth attendant in sub-Saharan Africa. Of the 20 nations with the highest infant mortality rates, 16 are in Africa. Sub- Saharan Africa is also the epicenter of the HIV-AIDS pandemic. "

Evidence: "The health challenges are documented through AFR's strategic planning processes, in the Budget Justification to Congress (CBJ), in sector summaries of USAID Annual Reports, as well as in development statistics from other institutions such as the World Bank and the UN Economic Commission for Africa. Additionally, challenges faced by each country are reflected in each Mission Performance Plan (MPP), which encompasses all activities of United States Government (USG) agencies working in the country and in the region. "

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: The CSH program is designed so that it is not unnecessarily duplicative of any other Federal, state, local or private efforts. As the lead United States Government (USG) foreign assistance entity in a developing host country, USAID coordinates with all other in-country USG entities (including the Centers for Disease Control and others) under the aegis of the U.S. Ambassador, as well as with the host government and other prominent donors. This coordination serves to eliminate program redundancies and to promote country program synergies. USAID regularly signs inter-Agency agreements to avoid duplication of efforts and to create program synergies among all actors. Preparation of the annual Mission Performance Plan (MPP) is coordinated by the U.S. Embassy and documents all activities of USG agencies that are carried out in that country. Coordination issues are also addressed through Country Team meetings, involving the principals of all USG entities and chaired by the Ambassador, and are held on a weekly basis. Furthermore, USAID and other bilateral and multilateral donors, the relevant host Government Office, and implementing partners including the UN health agencies meet regularly to share development experiences, and maximize assistance outcomes. Finally, the AFR CSH program is able to actively coordinate with the private sector through the Global Development Alliance (GDA) mechanism for public-private partnerships, leveraging private sector resources and eliminating duplication of efforts

Evidence: "1) Mission Performance Plan: Kenya 2006, p.7-38 and DROC 2006, p.8-26 2) Department of State/USAID Joint Management Council Charter 3) Congressional Budget Justification (CBJ) Africa overview, FY 2006 : Other Donor section, p. 5 4) Global Development Alliance Guidelines 5) The Automated Directives System (ADS) - Sections 201.3.6 and 201.3.7 (pg. 26-38) Preparation of Strategic Plans. Section 201.3.9.2 Donor coordination analysis in the strategic planning phase "

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The CSH program is free of major flaws that might affect program effectiveness. As required by the USAID Automated Directives System (ADS), AFR operating units (OUs) must develop a Results Framework (RF) for each strategic objective (SO), linking the development challenge being addressed to program implementation activities and milestones to be achieved. The SO is the most ambitious result in a particular program area that an operating unit (with its partners) can materially affect and for which it is willing to be held accountable. The RF is informed by various technical and resource analyses including evaluations, sector assessments, lessons learned, and gender and country conflict vulnerability analyses.

Evidence: 1) ADS 201, pg. 29-32 Content of Strategic Plans; ADS 201 pg. 45-51 Mandatory and other analyses for developing Strategic Plans. (Note that the ADS 201 is currently under revision) 2) Example of AFR review of OU Strategic Plan: Eritrea Strategy Review Cable 3) Global Development Alliance Guidelines

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: The CSH program operates under several design/legal constraints that might affect the programs effectiveness. The Child Survival and Health account which funds the AFR bureau health programs is almost completely earmarked annually by Congress leaving the program without the flexibility to program funding where priorities might be highest or determine the level of funding appropriate for each health sector. In addition, the programs decentralization to mission-level has resulted in strategic objectives that are not coordinated across the African region or prioritized in a regional manner. The new AFR strategic framework is being developed to address this issue. While strategic planning will still be decentralized to capture the country-level expertise of the OU's, regional prioritization, evaluation, and management will be directed through USAID headquarters. However, once allocated the CSH program is effectively targeted to reach intended beneficiaries. Country strategies in the AFR bureau are developed and implemented by OUs that have constant and direct contact with the program beneficiaries. In addition, Country and Assistance Checklists are completed and cleared by all relevant offices including the General Counsel, Procurement, and Financial Management.

Evidence: "The FY 2005 annual appropriations bill and report language directed 100% of all Child Survival and Health funding to specific earmarks for programs. The USAID White Paper advocates the need for greater flexibility between and within accounts to adequately respond to assistance needs. The new AFR strategic framework is a direct response to the problem of the decentralized strategic planning throughout USAID and its bureaus. " Country and Assistance Checklists ensure that USAID assistance is targeted at eligible countries and cooperating organizations that are registered with USAID and certified as having adequate financial and administrative systems and controls for receiving USAID funding.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The Africa CSH program does have specific long-term performance measures that are consistent across the region. While direct attribution remains impossible, these capture the outcomes/impact that the AFR CSH programs are designed to influence. The measures include: reduced HIV Prevalence Rate, increased Contraceptive Prevalence Rate, and reduced Under-five Mortality Rate. The AFR bureau long-term measures feed into USAID overarching agency-wide long-term strategic goals as well as those articulated in the Department of State-USAID Joint Performance Plan. The AFR CSH goals listed are continuous CSH performance measures that have been fed into the Africa bureau's new Bureau Strategic Framework which will set regional priorities and align and harmonize a set of contextual and regional performance indicators that in order to provide valuable performance information to managers both in the field and at Washington headquarters. These measures have been used for managing the AFR CSH program during the past 10 years and are built into OU's Performance Management Plans often with slight variations in measures.

Evidence: "At the Operating Unit (OU) level, the regional performance indicators and targets, as well as baseline data, are or will be contained in every OU's Performance Management Plan (PMP). At the OU-level, there are identifiable long-term indicators and targets that are linked to the Strategic Objectives related to CSH. These indicators and targets, as well as baseline data are contained in every OU's Performance Management Plan. "

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The Africa Bureau CSH program has ambitious targets and timeframes for its long-term measures. AFR is currently completing a new Strategic Framework for Africa that includes several long term performance indicators for which baseline data has already been collected. Both baseline data and annual targets are included in the Measures tab. Past trend data demonstrate that these targets, while reasonable, are ambitious and represent a significant increase in the rate of positive change anticipated. For example, one of the long term measures, the coverage rate of insecticide treated bed nets, has held steady at 7% for the past few years, yet by 2010, through coordinated, concerted efforts, the program is targeting coverage of 60% and has every expectation of meeting this target.--Need different example, ITN's are an annual measure. A number of the indicators within the new framework specify how Africa CSH activities impact the greater African social and economic trends. All long-term measures targets for AFR CSH programs will be reviewed on a bi-annual basis. Targets are expressed as percentages, where appropriate, or as actual figures in cases where an actual quantified number can be used.

Evidence: "1) U.S. Department of State and U.S.Agency for International Development Strategic Plan: Fiscal years 2004-2009, p.18-25 2) Joint Performance Plan p.40-62, 158-267 3) Selected Mission Annual Report: Uganda FY 2004, p.8-34 4) OU-level indicators - South Africa SO 5 (entire document) 5) ADS 203.3.5 Data Quality Assessments 6) ADS 203.3.3 Performance Management Plans "

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The AFR CSH annual performance goals, targets, and indicators are logically linked to the long-term joint goals and objectives, enabling USAID to demonstrate progress towards achieving them. For example, the annual performance measure of an increased diphtheria-tetanus vaccination rate feeds into the long-term indicator of under-five mortality rate. The AFR Bureau measures its regional performance through a set of discrete, quantifiable, and measurable annual targets and contextual and performance indicators, which are monitored by the AFR Bureau and are reported each year in the annual reporting process. The AFR Bureau uses an efficiency measure calculating the ratio of the Strategic Objective (SO) management costs to SO program dollars (in cents). At the OU level (country-level), the PMP includes annual performance measures which are used to assess progress towards the country/operational program's long-term goals and objectives. Operating Units (OUs) are delegated the authority to define quantitative and verifiable targets and timeframes for their respective Strategic Objectives (SOs, the Bureau's key, ambitious and challenging measures). The PMP provides baseline data, annual targets, and other important information. The OU level annual measures must also feed into the measurement of the regional annual and long-term performance measures highlighted in this assessment.

Evidence: "1) U.S. Department of State and U.S.Agency for International Development Strategic Plan: Fiscal years 2004-2009, p.18-25 2) Joint Performance Plan, p.7-11, p.40-62, p.158-267 3) OU-level indicators - South Africa SO 5 (entire document) 4) 2005 Annual Reporting Guidance, Section I, Sub-section A, Item 5 (Page 6) and Guidance Annex VIII, page 29 "

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The Africa Bureau CSH program has documented baselines and ambitious targets and timeframes for its annual performance measures. Annual performance measures exist at both the OU and the regional level. At the OU level, baselines and targets are documented in each OU's Performance Management Plans and Annual Reports. Baselines and targets are included for measures related to higher level Strategic Objectives (SOs) as well as the incremental Intermediate Results (IRs) that represent milestones towards achieving each SO. Targets are reviewed carefully when new strategies are submitted to AFR/Washington for approval to ensure they are sufficiently ambitious. Data are reported against those targets and used to inform management decisions. Similarly ambitious targets exist for annual performance measures at the regional level. Baselines and targets for regional annual performance measures are contained in the Measures tab of this PART submission.

Evidence: "1) Example of a Performance Management Plan (PMP): South Africa SO 5 (entire document) 2) ADS 203.3.2, 203.3.3, 203.3.4, 203.3.5, 203.3.10 Performance Management, performance management plans, selecting performance indicators, and data quality. 3) Selected Mission Annual Report: Uganda FY 2004 p.8-19 "

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: "At the OU-level, Strategic Objective Agreements (SOAGs) with host Governments outline the terms of agreement, including commitment to shared goals of USAID/AFR OUs and host country counterparts. Solicitation documents for contractors and grantees such as Request for Proposals (RFP), Request for Applications (RFA), or Annual Program Statements (APS) incorporate SO and/or program objectives. Respondents to the RFP or RFA must articulate how they will contribute to the achievement of AFR Mission Strategic Objectives. The extent to which they are able to help achieve these shared objectives and goals is evaluated in the technical merit criterion of the selection criteria. Grants and Contract Agreements reflect these shared commitments to common goals in the scope of work section. These commitments are likewise reflected in Contractors and Grantees annual work plans. By the terms and conditions of their agreements, contractors and grantees provide quarterly progress reports to OUs that document their results to date. USAID managers verify information in these reports through monitoring, site visits, and meetings. The information in the OU's PMP includes results reported and verified from contractors and grantees. All of these activities are designed and carried out to ensure that USAID partners commit to and work toward the annual and/or long-term goals of the program."

Evidence: "1) ADS 350.3.1-350.3.2 SOAGs 2) ADS 303.5.5 b and c - Evaluation criteria for grants 3) Example of a Performance Management Plan (PMP): Ghana Health SO 7 (entire document) 4) Request for Proposal Number 690-04-0027- Malawi (Section C4- Targeted Results, Pages C3, C5-C14 "

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: Evaluations are intended to look at results achieved vs. targets/goals, program impact and effectiveness, and lessons learned. Individual operating units use evaluations for program management and improvement. In the past five years approximately 154 evaluations have been conducted for the AFR Bureau's programs, including CSH-funded activities. A break-out of evaluations of CSH-only activities is not available because evaluations are often cross-sectoral and activities can involve multiple types of funding (Economic Support Funds, SEED funds, etc??). Evaluations can be internal, external, collaborative or participatory, and are generally completed by entities outside of the AFR Bureau including independent private firms not directly associated with the activity or program, the Office of the Inspector General, or USAID's Center for Development Information and Evaluation (CDIE). The AFR bureau uses mid-term evaluation recommendations to introduce course corrections and final evaluations to determine whether further assistance should be directed to that technical sector. Evaluations are used to provide input to inform key management decisions, to evaluate performance data that indicates an unexpected result or finding which cannot be readily explained, to investigate customer or partner feedback that suggests there are implementation problems or unmet needs, to re-examine program sustainability, to assess cost-effectiveness or relevance, and to identify lessons learned.

Evidence: "1) AFR list of evaluations, as compiled by USAID's Center for Development Information and Evaluation 2) Inspector General Audit Reports: Uganda, Zambia, Benin, Guinea, DRC 3) ADS 203.3.6 provides Agency guidance on evaluations 4) Example evaluation "

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: "At the regional level, USAID/AFR does not yet develop budget requests that are explicitly tied to accomplishment of annual and long-term performance goals. Also, this program does not submit a budget request makes clear the impact of funding, policy, or legislative decisions on expected performance and explains why the requested performance/resource mix is appropriate. OMB commends USAID for its efforts in developing a strategic budgeting model (which factors in performance data) to support AFR (and all regional bureaus) in their budget formulation. At the level of the Operating Units (OUs) budget requests are linked somewhat to the Annual Report of the OU, which summarizes program progress toward each Strategic Objective of the OU. Furthermore, the Congressional Budget Justification summarizes program progress by Strategic Objective, for each individual OU, however, this is done in a qualitative, anecdotal manner, and the agency has not yet reached the stage of incorporating their more quantitative performance data into budget presentations. "

Evidence: "1) Congressional Budget Justification 2006 (pending) 2) FY 2006 Annual Budget Submission, p.1-4,8-10 3) AFR review of an Annual Report: TBD 4) Selected Mission Annual Report: Uganda FY 2004 (entire document) 5) Bureau Program Budget Submissions (BPBS) "

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: To facilitate performance monitoring and reporting under the Joint State/USAID strategies, the AFR Bureau is completing an extensive effort to align and harmonize a set of contextual and regional performance indicators that would provide valuable performance information to managers both in the field and at Washington headquarters. The AFR Bureau Framework will provides the vital link among joint State/USAID strategic objectives, strategic goals, performance goals, contextual and common performance indicators at the regional level, and annualized performance targets. USAID's long-term performance goals are supported by outcome and/or output-related performance measures and regional indicators which are used to assess progress towards completion of the long-term goals, and allow Agency management to re-direct programs where necessary. At the OU-level, the regional performance indicators and targets, as well as baseline data, are or will be contained in every OU's Performance Management Plan (PMP). At the OU-level, there are identifiable long-term indicators and targets that are linked to the Strategic Objectives related to CSH. These indicators and targets, as well as baseline data, are contained in every OU's Performance Management Plan (PMP).

Evidence: "3) AFR Bureau Approved Bureau Framework Regional Indicators (pending) 4) Agency FY 2004 Performance and Accountability Report, p.10-12, 24-36, 98-107,122-174 5) ADS 203 Performance Management (emphasis on ADS 203.2 through 203.3.5) 6) Example of AFR review of OU Strategic Plan: Eritrea Strategy Review Cable 7) ADS 203.3.10 Intensive Program Reviews 8) Draft USAID/Guinea Triennial Review Reporting Cable 9) Strategic Management Interim Guidance (entire document) 10) 2005 Annual Reporting Guidance, Section I, Sub-section A, Item 5 (Page 6) and Guidance Annex VIII, page 29 "

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: USAID regularly collects timely and credible performance information from key program partners and from field missions. Funding recipients are required as part of their contractual or grant agreements to manage, monitor, and report performance information on a regular basis (usually quarterly). This information is submitted to Operating Units (OUs) on a quarterly basis and aggregated into the Annual Report (AR) and Performance Management Plans for each activity annually. OUs indicate in the AR the status of targets for programs. The AFR Bureau, in turn, reviews OU performance via the AR review process. When targets are not met, various options may be considered depending on the specific circumstances and issues involved: the contract or grant agreement may be modified, additional funds may be delayed or not provided, management changes may be introduced, discussions may take place with other partners (other donors or host-country) regarding their commitment, etc. If goals are exceeded the OU will decide whether to set new targets or to focus on other interventions. The AR review process in turn feeds into Bureau decisions about programming, budgeting, and staffing during the Bureau Program Budget Submission (BPBS) process.

Evidence: "1) Selected Mission Annual Report: Uganda FY 2004 (entire document) 2) AFR Bureau Annual Report Sector Summary: Health. Population, and Nutrition Results Reporting 3) Operating Unit Performance Management Plan (PMP), with South Africa as a sample (see sections on data analysis, data utilization, and dissemination for each indicator.) 4) USAID's ADS 202.3.6- Monitoring timeliness of key outputs 5) USAID adherence to 22 CFR 226, ""Administration of Assistance Awards to US Non-governmental Organizations,"" Section 226.51, ""Monitoring and Reporting Program Performance""; (from ADS 303.7 Mandatory Reference for Grants.) 6) 2005 Annual Reporting Guidance, Cover Memo and Section I, Sub-section A, Item 5 (Page 6) and Guidance Annex VII, (Pages 34-36) - completing performance measures table, including setting targets 7) DRAFT PPC Guidance on FY 2007 BPBS submissions, (Executive Summary and Overview, Page 3; Part 2- Budget and Performance Integration, Pages 7-8) "

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: At the bureau level, there is no evidence provided to ascertain whether AFR/CSH has identified senior managers at USAID who are responsible for achieving key program results and whether there are established clearly defined or quantifiable performance standards for those managers. At the contract management level, USAID cognizant technical officers (CTO) and program partners are held accountable for cost, schedule, and performance results. A cognizant technical officer (CTO) is designated for each AFR activity to manage contractors and grantees. CTO responsibilities include, among others, the technical review and approval of vouchers, monitoring contractor/grantee performance and financial status, and overseeing sub-agreements. Under USAID's personnel evaluation process, CTOs are evaluated for effective management of agreements, including the quality of technical guidance provided to contractors and grantees. The Agency also offers a course for CTO supervisors to enable them to more effectively provide on-the-job training and to better monitor and evaluate the performance of a CTO. Past performance of contractors and grantees is a criterion for all awards. Institutional capability and past performance is a large part of the selection criteria. Performance-based contracts and agreements are used to focus contractors on achievement of program results. Mid-term evaluations are also used to review performance to date and identify issues.

Evidence: "1) CTO responsibilities as defined in ADS 303.3 (Page 8) 2) DRAFT CTO Certification requirement agency notice (entire document) 3) Request for Proposal Number 690-04-0027- Malawi (Section C4- Targeted Results, Pages C6-C14; Section E3- Monitoring and Evaluation, Page E1; Section F- Deliveries or Performance, Pages F1-F5) 4) Mid-term evaluation: TBD 5) Course description of CTO Certification Programs (entire document) Please provide an example of a performance-based contract"

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: USAID CSH funds are obligated in a timely manner and spent for the intended purpose. Obligations, among other financial data, are tracked via Phoenix, the Washington-based accounting system that is being rolled out worldwide over the next 14 months a few Missions at a time, and the Mission Accounting and Control (MACS) field-based legacy accounting system. AFR/CSH funds are provided on an incremental, as-needed basis, with specific amounts determined by the budget and final Operating Year Budget setting processes. Once funds are obligated, OUs and the AFR Bureau monitor disbursements over the life of the program. Based on Agency policy and AFR Bureau's analysis, an OU's program may not receive its full increment of funds in a subsequent year if it is determined that an unacceptable level of funds is still unspent (undisbursed). According to Agency policy, a program can only have unspent funds sufficient to cover expected disbursements for the next 12-18 months. This policy is strictly enforced by the AFR Bureau Development Planning Office. Additionally, the Cognizant Technical Officer (CTO) of each grant, contract, or cooperating mechanism is responsible for monitoring this process. Finally, all OUs are required to prepare procurement plans that outline planned procurements and obligations and help ensure that funds are spent in a timely manner and for the intended purpose.

Evidence: "1) Phoenix Reports 2) ADS 602 - Agency guidance regarding forward funding of programs. 4) ADS 202.3.8.3- Obligations Management 5) FY 2006 Annual Budget Submission (Page 6) "

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The AFR bureau developed an efficiency measure designed to manage the ratio of Strategic Objective management costs to SO program dollars. This efficiency measure is intended to be applied across the region so that comparisons and analysis can identify questions of efficiency and guide corrections to the CSH structure. However, the baselines and targets proposed for the measure do not reflect any anticipated efficiencies. In addition, Missions undertake periodic portfolio reviews which provide a systematic and comprehensive view of the each activity. Through the strategic planning reform process, AFR is defining ways to improve operations and to help rationalize the allocation of scarce staff and operating expense (OE) resources. Specifically, the Africa Bureau is undertaking a Resource Harmonization exercise that will significantly reallocate staff and operating expenses across Missions in Africa. The results of this exercise are being taken into account during the development of the Agency Strategic Framework for Africa. Finally, AFR Bureau adheres to all Federal regulations on competition which also helps to ensure cost effectiveness and efficiency in procurement.

Evidence: "1) ADS 300 provides the overall guidance on procurement including competitive procurements (ADS 302.5.8 for Contracts and 303.5 for Grants). 2) AIDAR 706.501- Mandatory Reference for Competition 3) FY 2006 Annual Budget Submission (Pages 3-6) "

Does the program collaborate and coordinate effectively with related programs?

Explanation: The program collaborates and coordinates effectively with related programs across the United States Government (USG) and with other international donors. Most notably, the Department of State and USAID have recently developed joint goals at the Department/Agency level via the Joint State-USAID FY 2004-2009 Strategic Plan. In the field, weekly country team meetings are held among USG agencies and regular coordination meetings with other donors in AFR countries with USAID missions. The Annual Budget Submission (ABS) process is a joint effort between State and USAID and is reflected in the FY 2006 ABS. In addition, the AFR Bureau and its OUs collaborate and coordinate with related programs: through participation in the Joint Management Council and Joint Policy Council; close working collaboration with State counterparts at the Bureau and country levels- especially through the MPP review process; memos of understanding with other Federal Agencies; staffing arrangements with other USG agencies; Inter-agency working groups such as with the State Counter-Terrorism task force; Consultative Groups with other donor agencies. Notably, AFR has developed a close working relationship with colleagues at the Office of the Global AIDS Coordinator (O/GAC). Many USAID staff are working at O/GAC, and communication between O/GAC and USAID is constant. Finally, the Bureau actively seeks public-private partner alliances in common development areas. One example of this collaboration is our work with the Gates Foundation. Once these relationships are established they are evaluated and monitored through the annual program review process.

Evidence: "1) U.S. Department of State and U.S.Agency for International Development Strategic Plan: Fiscal years 2004-2009, (Messages from Secretary of State and USAID Administrator, Preface; Mission, Pages 1-4; Achieving Peace and Security, Pages 5-8; Advance Sustainable Development and Global Interests, Pages 18-29; Organizational Impacts, Pages 39-40) 2) Example of State Department Mission Performance Plan (MPP) - Kenya, FY 2006 (Chief of Mission Statement, references to USAID program partnerships throughout the text.) 3) Example of State Department Mission Performance Plan (MPP) - Democratic Republic of Congo, FY 2006 (Chief of Mission Statement, references to USAID program partnerships throughout the text.) 4) Examples of USAID Global Partnerships. For example, AFR/SD polio team works with diverse actors in the polio eradication effort, including U.S. government agencies, the European Union, World Health Organization (WHO), Gates Foundation, UN Foundation and World Bank. 5) State/USAID announcement of Joint Policy and Joint management Councils for cooperation and coordination between the agencies 6) Assistant Administrator (AA) Pierson testimony to SFRC, March 2, 2005 (p. 11) "

Does the program use strong financial management practices?

Explanation: Through the Federal Managers Financial IntegrityAct (FMFIA)/Management Control Review Committee process, USAID ensures that resources are protected against fraud, waste and abuse and that they achieve the results for which funds were appropriated. The process requires each OU in the AFR Bureau to do a self-assessment of the adequacy of management controls in all areas of agency operations including program, administrative, and financial management. Each OU submits an FMFIA memo to the AFR Bureau, which in turn submits a Bureau memo to the USAID Administrator. AFR grants and contractors are also subject to audits. However, currently the Phoenix system is used for Washington accounting and the MACS system for field accounting, using two systems has created a disconnect between the two. USAID plans to roll out the Phonenix system to the field, which will resolve the outstanding problems. Once Phoneix is deployed to all missions, USAID anticipates getting a "yes" answer for this question. During FY 2005, USAID Ghana became the first Phoenix pilot mission in Africa, servicing accounting operations for USAID Ghana, the West Africa Regional Program, Liberia and Nigeria.

Evidence: "1) Federal Manager's Financial Integrity Act (FMFIA) checklist 2) AFR Bureau FMFIA memo to USAID Administrator dated September 3, 2004 3) FMFIA Material Weaknesses reported by USAID AFR Missions for FY 2004 4) ADS 620 chapter - Financial management principles and standards (entire chapter) 5) Agency FY 2004 Performance and Accountability Report (Administrator's Letter, Pages 5-7; Chief Financial Officer's Letter, Pages 8-9; Management's Discussion and Analysis, Pages 72-89) 6) ADS 596 - Management Accountability and Control "

Has the program taken meaningful steps to address its management deficiencies?

Explanation: The program has taken meaningful steps to address management deficiencies. For example, the program is rolling out it's new Phoenix financial system to ensure streamlined accounting operations in the future. The roll out is being handled in a methodical, measured way to ensure that not only are the benefits of the new system fully realized, but that any issues of a too precipitous switch in accounting systems can be avoided. In another example, the FMFIA process requires each OU in the AFR Bureau to do a self-assessment of the adequacy of financial management controls in all areas of agency operations, including management of program assistance. Each OU submits an FMFIA memo to the AFR Bureau, which in turn submits a Bureau memo to the USAID Administrator. At the OU level, AFR missions address deficiencies or weaknesses that could appropriately be resolved at that level. Any significant deficiency or material weakness (whether new or unresolved from previous FMFIA reviews) is reported to the AFR Bureau and included in the Bureau FMFIA memo to the Administrator. Steps for resolving the weakness are identified and a date proposed by which the weakness or deficiency will be corrected. Any deficiencies are also noted publicly in the Agency Performance and Accountability Report each year. As a final example, the Agency engages in rigorous oversight of the performance of its senior managers to ensure that any management deficiencies are identified quickly so they can be immediately resolved. Geographic Directors at AFR-Washington are in contact with each Mission Director several times a week to ensure that priorities remain aligned across the Bureau and program management is consistent and effective.

Evidence: "1) ADS 620, FMFIA checklist. 2) AFR Bureau FMFIA memo to USAID Administrator dated September 3, 2004 3) FMFIA Material Weaknesses reported by USAID AFR Missions for FY 2004 "

Are grants awarded based on a clear competitive process that includes a qualified assessment of merit?

Explanation: The program does not make the overwhelming majority of awards (95 percent or more of funding) are distributed according to a competitive process. The Africa Child Survival and Health program awards _________ percent of funds through a competitive process. These Requests for Proposals (RFPs), Requests for Assistance (RFAs) and Annual Program Statements (APSs) are all posted publicly at www.fedgrants.gov. All decisions on grant awards are properly documented (via selection memos, memos of negotiation, etc.). USAID Washington (via the Office of Acquisition and Assistance) reviews and provides oversight to ensure AFR adherence to Federal regulations and Agency guidance on competition. Procurement integrity is taken very seriously and all CTOs are trained in the specific regulations USAID employees are required to follow during the procurement process. Additionally, all USAID staff are required to take ethics training on a regular basis. This training includes instruction on procurement integrity. A number of mechanisms exist at USAID to specifically do outreach to potential new partners. These include the Global Development Alliance, the Office of Small and Disadvantaged Business Utilization and the Center for Faith Based and Community Initiatives. Most offices also have yearly Annual Program Statements that are designed to find new partners.

Evidence: "1) ADS 300 provides the overall guidance on procurement including competitive procurements (ADS 302.5.8 for Contracts and 303.5 for Grants). 2) AIDAR 706.501- Mandatory Reference for Competition 3) FY 2006 Annual Budget Submission (Pages 3-6) 4) ADS 202.3.9 - Avoiding conflicts of interest, ensuring procurement integrity, complying with ethics rules, and meeting audit responsibilities (entire section) 5) Sample Annual Program Statement: Public-Private Alliances, http://www.fedgrants.gov/Applicants/AID/OP/WAS/GDA-05-001/listing.html

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: The AFR program has significant oversight practices that provide sufficient knowledge of grantee activities. For example, a Cognizant Technical Officer (CTO) is designated for each program to manage contractors and grantees. CTO responsibilities include, among others, technical review and approval of vouchers, monitoring contractor/grantee performance and financial pipelines, and sub-awards. Expenditures are tracked by each CTO and reported to higher level Managers who track expenditures across different activities. The Activity Manager and the SO Team Leader, who may or may not be the CTOs, also provide oversight. Expenditures are tracked by each CTO and reported to higher level Managers who track expenditures across different activities. In addition, mission management is kept informed of program performance, progress, and issues via periodic portfolio reviews, staff meetings, etc. The AFR Bureau, in turn, is informed via the Annual Review process. The OU Agreement Officer is the mandatory control point of record for all official communications and contacts with the recipient that may affect the award budget, the program description or any terms and conditions of the award. Audits per OMB Circular A-133 or recipient-contracted audits are also conducted as required by policy or regulation.

Evidence: "1) CTO responsibilities as defined in ADS 303.3 (Page 8) 2) ADS 202.3.9.4- Conducting Audits 3) USAID FY 2004 Performance and Accountability Report (entire report) "

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: The AFR Bureau collects grantee and compiles performance data. However, it does not disseminates grantee performance information in an accessible manner, such as via a web site or widely available program reports. In preparing USAID's reporting documents, the data supplied by the various grantees and contractors are utilized to support AFR Bureau's analysis and program descriptions. Performance information provided in the Performance and Accoutability Report are aggregated at a high level, reporting on select country achievements. The Congressional Budget Justification (CBJ, available on the USAID web site) provides specific information to the public on AFR programs activities including information on SO performance and results, and by country (OU). The CBJ also indicates the various grantees and contractors working in the different SOs. USAID's Center for Development Information and Evaluation (CDIE), via the Development Experience Clearinghouse (DEC), makes evaluations of specific programs available to the public.

Evidence: "1) USAID FY 2004 Performance and Accountability Report (entire report) 2) Congressional Budget Justification (CBJ) Africa overview, FY 2006 3) List of currently available Africa Bureau Program/Activity evaluations available to public through the Development Experience Clearinghouse (www.dec.org/partners/evalweb/) 4) Evaluation of Recent USAID Evaluation Experience (Pages iii-iv, entire document) "

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: The CSH program is on-track to meet long-term performance goals, including the regional goals and the country-level OU indicators. Trend data demonstrates that CSH is on track to meet the regional goals. Past performance data did not report against targets and baselines but showed sustained improvement in the performance information collected to show achievement toward the program purpose. FY 2006, will be the first year that accomplishment of the new regional measures will be evaluated.

Evidence: "1) U.S. Department of State and U.S.Agency for International Development Strategic Plan: Fiscal years 2004-2009 (entire document) 2) USAID FY 2006 Joint Performance Plan (entire document) 3) USAID FY 2004 Performance and Accountability Report (entire report) 4) Example of a Performance Management Plan (PMP): South Africa SO 5 (see specific updates throughout PMP) 5) Selected Mission Annual Report: Uganda FY 2004 (entire document) 6) USAID/Guinea Triennial Review "

Does the program (including program partners) achieve its annual performance goals?

Explanation: At the regional level, the AFR Bureau has specific, quantifiable annual targets and common regional indicators (contextual and performance as part of the Bureau Framework) for CSH programs that will be evaluated for accomplishment in FY 2006. These measures are supplemented by country-level annual indicators which are defined in OU Performance Monitoring Plans (PMPs) and reported in Annual Reports. Referring to past performance data and past performance results stated in the Joint Performance Plan, the program is on-track to meet of the annualized targets, including those which are joint targets with the Department of State and other program partners. At the country level, the PMP and Annual Report are the primary documents that show whether program performance is on track. These documents are, in large part, based on grantee and contractor (partner) information. In the case of CSH, targets at the OU level have been predominantly met. With very few exceptions, OUs at the country level report verified data that targets are met or exceeded.

Evidence: "1) U.S. Department of State and U.S.Agency for International Development Strategic Plan: Fiscal years 2004-2009 2) USAID FY 2006 Joint Performance Plan (entire document) 3) AFR Bureau Approved Bureau Framework Regional Indicators 4) Example of a Performance Management Plan (PMP): South Africa SO 5 (see specific updates throughout PMP) 5) Selected Mission Annual Report: Uganda FY 2004 (entire document) 6) USAID FY 2004 Performance and Accountability Report (entire report) "

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The AFR program established a baseline for the efficiency measure (SO management costs/SO program dollars) in 2003. However, the targets set for 2004 through 2007 do not anticipate any efficiency gains in the program. AFR is developing a model to rationalize the allocation of scarce staff and operating expense resources. The results of this exercise and other management improvements are being factored into the development of the Framework. Additionally, AFR has significantly reduced the levels of obligated but undisbursed funds for individual OUs over the past several fiscal years. This means that individual OUs are being required to reduce overall pipelines for program efficiency.

Evidence: ") FY 2006 Annual Budget Submission (entire submission) 2) CTO responsibilities as defined in ADS 303.3 (Page 8) 3) AFR Bureau FMFIA memo to USAID Administrator dated September 3, 2004 "

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: USAID is one of the only donors that has a stringent, auditable performance management system that monitors and can report on results with good quality data. Most donors, on the other hand, lack the capability to track their performance because they conduct development assistance through budget support.

Evidence: "1) House of Representatives F76Resolution 120 commending USAID staff on Asian Tsunami Response "

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: The program did not provide information on the quality of the evaluation, their scope, or what were the specific findings in order to receive credit for this answer. There have been very few independent evaluations showing that CSH programs are effective and achieving results. These independent evaluations are conducted at the OU/Mission level for sector specific or strategic objectives. All of the evaluations provided as evidence were USAID IG reports from 2002-2004. The IG reports cited that a program was achieving results if half or more of the performance indicators were being met. Several of the reports cited major deficiencies overall in the ability to effectively monitor performance. The Agency is reinvigorating the evaluation process at USAID through a new evaluations training course and an increased prominence on conducting evaluations.

Evidence: Some specific examples of CSH evaluation summaries can be found at www.poptechproject.com/library/sums.html (Examples of recent evaluations include: "Assessment of Advance Africa and the Catalyst Consortium - November, 2004 ", "Evaluation of the Support for Analysis and Research in Africa II Project - March, 2005", "Mid-term Evaluation of Population Services International's Improved Health for Social Marketing in Malawi - September, 2004", "Family Planning Graduation Experience: Lessons for the Future - October, 2004".) While pointing to some constraints, the majority of these evaluations are favorable. Additional evaluations accessible to the public are on the USAID website at: www.dec.org/partners/eval.cfm