1996, Number 9 Performance Monitoring and Evaluation TIPS USAID Center for Development Information and Evaluation CONDUCTING CUSTOMER SERVICE ASSESSMENTS What Is a Customer Service Assessment? A customer service assessment is a management tool for understanding USAID's programs from the customer's perspective. Most often these assessments seek feedback from customers about a program's service delivery performance. The Agency seeks views from both ultimate customers (the end-users, or beneficiaries, of USAID activities usually disadvantaged groups) and intermediate customers (persons or organizations using USAID resources, services, or products to serve the needs of the ultimate customers). Customer service assessments may also be used to elicit opinions from customers or potential customers about USAID's strategic plans, strategic objectives, or other planning issues. For example, the operating unit may seek their views on development needs and priorities to help identify new, relevant activities. Why Conduct Customer Service Assessments? USAID's reengineered operating system calls for regularly conducting customer service assessments for all program activities. Experience indicates that effective customer feedback on service delivery improves performance, achieves better results, and creates a more participatory working environment for programs, and thus increases sustainability. These assessments provide USAID staff with the information they need for making constructive changes in the design and execution of development programs. This information may also be shared with partners and customers as an element in a collaborative, ongoing relationship. In addition, customer service assessments provide input for reporting on results, allocating resources, and presenting the operating unit's development programs to external audiences. Customer service assessments are relevant not only to program-funded activities directed to customers external to USAID. They can also be very useful in assessing services provided to internal USAID customers. Moreover, customer service assessments are federally mandated. The Government Performance and Results Act of 1993 and Executive Order 12862 of 1993 direct federal agencies to reorient their programs toward achievement of measurable re- sults that reflect customers' needs and to systematically assess those needs. Agencies must report annually to the Administration on customer service performance. Who Does Customer Service Assessments? USAID reengineering guidance specifies that all operating units should develop a customer service plan. The plan should include information about customers' needs, preferences, and reactions as an element in a unit's planning, achieving, performance monitoring and evaluation functions (see box 1). Depending on the scope of its program operations, an operating unit may find it needs to plan several customer service assessments. The various assessments might be tailored to different strategic objectives, program activities and services, or customer groups (differentiated, for example, by gender, ethnicity, or income). Responsibility for designing and managing these assessments typically is assigned to the relevant strategic objective or results package team. How Do Customer Service Assessments Com- plement Performance Monitoring And Evaluation? Performance monitoring and evaluation broadly addresses the results or outcomes of a program. These results reflect objectives chosen by the operating unit (in consultation with partners and customer representatives) and may en- compass several types of results. Often they are medium- to longer-term developmental changes or impacts. Examples: reductions in fertility rates, increases in income, improvements in agricultural yields, reductions in forest land destroyed. Another type of result often included in performance monitoring and evaluation involves customer perceptions and responses to goods or services delivered by a pro- gram for example, the percentage of women satisfied with the maternity care they receive, or the proportion of farmers who have tried a new seed variety and intend to use it again. Customer service assessments look at this type of result customer satisfaction, perceptions, preferences, and related opinions about the operating unit's performance in delivering the program's products and services. Unless the service or product delivery is satisfactory (i.e., timely, relevant, accessible, good quality) from the per- spective of the customers, it is unlikely that the program will achieve its substantive development results, which, after all, ultimately depend on customers' participation and use of the service or product. For example, a family-planning program is unlikely to achieve reduced fertility rates unless customers are satisfied with the contraceptive products it offers and the delivery mechanism it uses to provide them. If not sufficiently satisfied, customers will simply not use them. Customer service assessments thus complement broader performance monitoring and evaluation systems by moni- toring a specific type of result: service delivery perfor- mance from the customer's perspective. By providing managers with information on whether customers are satisfied with and using a program's products and services, these assessments are especially useful for giving early indications of whether longer term substantive development results are likely to be met. Both customer service assessments and performance monitoring and evaluation use the same array of standard social science investigation techniques surveys, rapid and participatory appraisal, document reviews, and the like. In some cases, the same survey or rapid appraisal may even be used to gather both types of information. For example, a survey of customers of an irrigation program might ask questions about service delivery aspects (e.g., access, timeliness, quality, use of irrigation water) and questions concerning longer term development results (e.g., yields, income). Steps In Conducting A Customer Service As- sessment Step 1. Decide when the assessment should be done. Customer service assessments should be conducted whenever the operating unit requires customer information for its management purposes. The general timing and frequency of customer service assessments is typically outlined in the unit's customer service plan. Customer service assessments are likely to be most effective if they are planned to coordinate with critical points in cycles associated with the program being assessed (crop cycles, local school year cycles, host country fiscal year cycles, etc.) as well as with the Agency's own annual reporting and funding cycles. Customer service assessments will be most valuable as management and reporting tools if they are carried out some months in advance of the operating unit's annual planning and reporting process. For example, if a unit's results review and resources request (R4) report is to be completed by February, the customer service assessment might be conducted in November. However, the precise scheduling and execution of assessments is a task appropriate for those responsible for results in a program sector members of the strategic objective or results package team. Step 2. Design the assessment. Depending on the scale of the effort, an operating unit may wish to develop a scope of work for a customer service assessment. At a minimum, planning the assessment should 1) identify the purpose and intended uses of the information, 2) clarify the program products or services being assessed, 3) identify the customer groups involved, and 4) define the issues the study will address. Moreover, the scope of work typically discusses data collection methods, analysis techniques, reporting and dissemination plans, and a budget and time schedule. Specific issues to be assessed will vary with the strategic objective, program activities under way, socioeconomic conditions, and other factors. However, customer service assessments generally aim at understanding þ Customer views regarding the importance of various USAID-provided services (e.g., training, information, commodities, technical assistance) to their own needs and priorities þ Customer judgments, based on measurable service standards, on how well USAID is performing service delivery þ Customer comparisons of USAID service delivery with that of other providers Open-ended inquiry is especially well suited for ad- dressing the first issue. The other two may be measured and analyzed quantitatively or qualitatively by consulting with ultimate or intermediate customers with respect to a number of service delivery attributes or criteria important to customer satisfaction (see box 2). In more formal surveys, for example, customers may be asked to rate services and products on, say, a 1-to-5 scale indicating their level of satisfaction with specific service characteristics or attributes they consider important (e.g., quality, reliability, responsiveness). In addition to rating the actual services, customers may be asked what they would consider "excellent" service, referring to the same service attributes and using the same 5-point scale. Analysis of the gap between what customers expect as an ideal standard and what they perceive they actually receive indicates the areas of service delivery needing improvement. In more qualitative approaches, such as focus groups, customers discuss these issues among themselves while researchers listen carefully to their perspectives. Operating units and teams should design their customer assessments to collect customer feedback on service delivery issues and attributes they believe are most im- portant to achieving sustainable results toward a clearly defined strategic objective. These issues will vary with the nature of the objective and program activity. 3. Conduct the assessment. With its objective clearly in mind, and the information to be collected carefully specified, the operating unit may decide to use in-house resources, external assistance from consultants, or a combination of the two, to conduct the assessment. Select from a broad range of methods. A customer service assessment is not just a survey. It may use a broad repertory of inquiry tools designed to elicit information about the needs, preferences, or reactions of customers regarding a USAID activity, product or service. Methods may include the following: þ Formal customer surveys þ Rapid appraisal methods (e.g., focus groups, town meetings, interviews with key informants) þ Participatory appraisal techniques, in which customers plan, analyze, self-monitor, evaluate or set priorities for activities þ Document reviews, including systematic use of social science research conducted by others Use systematic research methods. A hastily prepared and executed effort does not provide quality customer service assessment information. Sound social science methods are essential. Practice triangulation. To the extent resources and time permit, it is preferable to gather information from several sources and methods, rather than relying on just one. Such triangulation will build confidence in findings and provide adequate depth of information for good decision-making and program management. In particular, quantitative surveys and qualitative studies often complement each other. Whereas a quantitative survey can produce statistical measurements of customer satisfaction (e.g., with quality, timeliness, or other aspects of a program operation) that can be generalized to a whole population, qualitative studies can provide an in-depth understanding and insight into customer perceptions and expectations on these issues. Conduct assessments routinely. Customer service as- sessments are designed to be consciously iterative. In other words, they are undertaken periodically to enable the operating unit to build a foundation of findings over time to inform management of changing customer needs and perceptions. Maintaining an outreach orienta- tion will help the program adapt to changing circum- stances as reflected in customer views. 4. Broadly disseminate and use assessment findings to improve performance. Customer service assessments gain value when broadly disseminated within the operating unit, to other operating units active in similar program sectors, to partners, and more widely within USAID. Sharing this information is also important to maintaining open, transparent relations with customers themselves. Assessment findings provide operating unit managers with insight on what is important to customers and how well the unit is delivering its programs. They also can help identify operations that need quality improvement, provide early detection of problems, and direct attention to areas where remedial action may be taken to improve delivery of services. Customer assessments form the basis for review of and recommitment to service principles. They enable mea- surement of service delivery performance against service standards and encourage closer rapport with customers and partners. Moreover, they encourage a more collaborative, participatory, and effective approach to achievement of objectives. Selected Further Reading Resource Manual for Customer Surveys. Statistical Policy Office, Office of Management and Budget. October 1993. H. S. Plunkett and Elizabeth Baltimore, Customer Focus Cookbook, USAID/M/ROR, August 1996. Zeithaml, Valarie A; A. Parasuraman; and Leonard L.Berry. Delivering Quality Service. New York: Free Press.