Appendix III: USAID Program Evaluation Plan

Over the past 11 years, USAID has embraced new evaluation and review models to ensure the use of robust and accurate data in performance planning and reporting. USAID has made particular efforts to provide explicit policy guidance, extensive training, and supportive technical assistance to establish these data collection and analysis systems into all aspects of program management. With these new systems, USAID has improved its capacity to assess whether it is achieving its overall objectives. Program evaluation and special studies are important tools used to determine whether its programs have reached their objectives, as well as providing objective inputs to policy and planning decisions.

Evaluation Tools and Methods

USAID will continue to use a variety of methods and tools to evaluate its work. The tools and methods offer varying levels of detail and focus based on their intended use, but each tool or method is critical to ensure USAID evaluates all of its programs and is wisely spending its resources to reach the tenets of this joint Strategic Plan. The following are the different tools and methods USAID will continue to use:

Business Transformation Executive Committee (BTEC): Early in the Bush administration, USAID established the BTEC to evaluate and make recommendations to improve USAID management. This ongoing effort has made many changes in the way the Agency does business. BTEC’s achievements include: clean audit opinions on financial statements that demonstrate transparent and accountable financial practices; the saving of over five million dollars through joint licensing agreements as a result of the Joint Financial Management System (JFMS) collaboration with the Department; and automation of the recruitment process reducing the hiring cycle to less than 45 days from close of job announcement to employee job offer. One of BTEC’s major efforts is the annual Administrator’s Morale and Customer Service Survey, which is widely considered to be a success by employees—as evidenced by a high response rate. The survey provides an objective reference point over time, and it is accepted and used by others external to USAID.
Agency Level Evaluations: In 2005, USAID launched a major effort to revitalize its evaluation system, including the establishment of the Evalweb site where evaluation agendas are posted and findings disseminated widely. The Initiative to Revitalize Evaluations seeks to improve the way evaluations are done and used, as well as increase the number of evaluations conducted by USAID field missions. This has helped to reverse the downward trend with 167 evaluations done in FY 2004, a significant increase from the 79 evaluations completed in 2001. Evaluations are driven by a real need for information which can be used by management to improve programs and policies.
Mission Management Assessments (MMA): In 2004, USAID initiated a system of coordinated MMAs. The purpose of an MMA is to evaluate and improve the effectiveness of mission operations in terms of both its program and internal management. It typically covers the following areas: program rationale, implementation and monitoring, roles of offices and teams, communication, organization and staffing, administrative and financial management, external relationships, and morale and security. Following a pilot assessment of the Regional Center for Southern Africa (RCSA) conducted in October 2004, USAID completed 10 MMAs by 2005, covering all geographic regions. The Bureau for Management and the Cross-Regional Bureau assessments were the first headquarters assessments completed in 2006.
Office of Inspector General (OIG) and Government Accountability Office (GAO) Evaluations: In addition to evaluations that Bureaus undertake independently, OIG and GAO are two independent bodies that also drive evaluation within USAID. These evaluations and reports provided the joint Strategic Plan with an objective assessment of performance and specific recommendations for improvement.
Program Assessment Rating Tool (PART): USAID also utilizes the results from the Office of Management and Budget’s PART to inform how we will improve our performance. Starting in FY 2002, this tool was used to evaluate USAID’s programs on Climate Change, Public Law 480 Title II Food Aid, Population, Development Assistance and Child Survival and Health for Africa, Latin America and the Caribbean, Office of Transition Initiatives, International Disaster and Famine Account, Development Administration and Capital Investment, and Development Credit Authority.
Special Studies: In addition to formal evaluations, USAID conducts a number of special studies to determine program impact. The most well-known of these is the Demographic and Health Survey, which USAID conducts in numerous countries and is often the primary mechanism to measure status and change in a variety of health, child survival, population, and HIV/AIDS indicators.

Collectively, all of USAID’s program evaluation elements enable the Agency to track key program results, aggregate them in a single Performance and Accountability Report (PAR), and disseminate them to internal and external audiences. Except for security or procurement sensitive sections, mission annual reports are published on USAID’s internal and external Web sites. This leads to transparency in reporting, and to making USAID’s lessons learned widely available to all interested parties.