Detailed Information on the
Drinking Water Protection Program Assessment

Developing a long-term outcome performance measure to assess the public health impacts of improvements in drinking water compliance.

Revising the current drinking water small system affordability methodology to address negative distributional impacts.

Implementing data quality review recommendations to improve the overall quality of the data in EPA's drinking water compliance reporting system.

The program is developing an efficiency measure that is more useful and meaningful for tracking annual programmatic efficiency.

Measure: Percent of population served by community water systems that meet all applicable health-based drinking water standards through approaches that include effective treatment and source water protection.

Explanation:The performance measure tracks the percent population served by community water systems that receives drinking water in compliance with health-based standards. EPA 2003-2008 Final and 2006-2011 Strategic plans include the measure as a subobjective to track progress for the subobjective "Water Safe to Drink" under Goal 2 "Clean and Safe Water).

Measure: Percent of community water systems for which minimized risk to public health through source water protection is achieved.

Explanation:The performance measure applies to surface and ground water sources. Minimized risk is achieved by the substantial implementation, as determined by the state, of source water actions in a source water protection strategy. Actions may be taken by the state, system, local community, or other stakeholders. EPA's 2006-2011 Strategic Plan includes the measure as a strategic target for the subobjective "Water Safe to Drink".

Measure: Percent of community water systems that meet all applicable health-based standards through approaches that include effective treatment and source water protection.

Explanation:The measure tracks the percent of community water systems in compliance with any health-based standard that is in effect. Achievement ensures that the long-term outcome measure of population served by community water systems in compliance is achieved. The measure is a Strategic Target in the Agency 2003-2008 Strategic Plan, but is separated into two measures addressing regulations issued prior to and after December 2001. For the next Strategic Plan, EPA is going to use the same measure as used in the PART, which does not differentiate between when rules were issued.

Measure: People receiving drinking water that meets all applicable health-based standards per million dollars spent to manage the national drinking water program.

Explanation:Dollars include EPA funds, UIC and PWSS grants (with associated state match), DWSRF grant set-aside funds and DWSRF funds provided for assistance (including state match, bond proceeds, repayments and interest earnings).

Measure: Percent of community water systems that have undergone a sanitary survey within the past three years (five years for outstanding performers).

Explanation:Maintaining a sanitary survey program is a primacy requirement for state drinking water programs. The measure also addresses requirements under the Interim Enhanced and Long-term 1 Surface Water Treatment Rules. States have indicated that sanitary surveys are one of the most important tools they have in overseeing public water systems. The measure has been revised from a measure included in the PWSS PART to look at the percent of community water systems that have had a survey rather than the number of states that have met the requirement.

Measure: Percent of person months during which community water systems provide drinking water that meets all applicable health-based standards.

Explanation:The measure tracks the duration of a population's exposure to violations. For example, some systems may receive a violation for an incident that lasts only one day (e.g., turbidity increase due to storm event). The measure is intended to give a more accurate picture of exposure of the population to contamination. Achievement of the measure will help ensure that the long-term outcome measure of population served by community water systems in compliance at all times is achieved. It is under consideration for inclusion as a strategic target for the 2006-2011 Strategic Plan.

Measure: Percent of data for violations of health based standards at public water systems that are accurate and complete in SDWIS/FED for all MCL and TT rules.

Explanation:This measure of data quality will be based on rolling 3 year data from data verification audits. The measure does not include the Lead and Copper Rule (which is monitored by a separate measure). The measure is being considered for inclusion as program activity measure (indicator or target).

Is the program purpose clear?

Explanation: The purpose of EPA's Drinking Water Protection Program is to protect public health by ensuring that all public water systems provide drinking water to their customers, which meets applicable federal drinking water quality standards designed to protect public health. The program's purpose is derived from the Safe Drinking Water Act (SDWA), which authorizes EPA and States to carry out programs to implement and enforce National Primary Drinking Water Regulations and regulate underground injection of fluids. The Drinking Water Protection program and its State partners also implement SDWA provisions requiring public water systems to inform customers about drinking water quality. In addition, the Program and States assist in locally driven, voluntary source water quality protection programs. The national drinking water program also implements the water security program, whose purpose is to provide tools, training, and technical assistance to reduce the risks from a terrorist or other intentional act.

Evidence: ??Safe Drinking Water Act, as amended 42 U.S.C. 300(f-j). The Act requires EPA to regulate contaminants that (1) "may have an adverse effect on the health of persons," (2) are "known to occur or there is a substantial likelihood that the contaminant[s] will occur in public water systems with a frequency and at levels of public health concern," and (3) "regulation of such contaminant presents a meaningful opportunity for health risk reduction for persons served by public water systems."; ??EPA Strategic Plan - Goal 2: Clean and Safe Water. At http://www.epa.gov/ocfo/; ??Public Health Security and Bioterrorism Preparedness and Response Act of 2002 (Bioterrorism Act) at http://cfpub.epa.gov/safewater/watersecurity/bioterrorism.cfm; ??Homeland Security Presidential Directives at http://cfpub.epa.gov/safewater/watersecurity/legislation.cfm; ??Drinking Water and Underground Injection Control program Regulations in 40 CFR 141-149; ??Fact Sheets developed for the 30th Anniversary of the Safe Drinking Water Act at http://www.epa.gov/safewater/sdwa/30th/index.html .

Does the program address a specific and existing problem, interest, or need?

Explanation: The Drinking Water Protection Program addresses the public need for safe drinking water. Continued disease outbreaks and illnesses caused by contaminated drinking water indicate the ongoing need to protect drinking water quality. With respect to water security, the program is needed to reduce the vulnerabilities of and consequences to the water sector associated with a terrorist or other intentional attack.

Evidence: ??Waterborne Outbreaks Caused by Microbial Agents in Public Water Systems 1991-2000, in Appendix A, Appendices to the Occurrence and Exposure Assessment for the Final Long Term 2 Enhanced Surface Water Treatment Rule, December 2005 at http://www.epa.gov/safewater/disinfection/lt2/pdfs/assessment_lt2_occurance_appendices.pdf ??Outbreaks in drinking-water systems, 1991-1998, Craun, Gunther F; Nwachuku, Nena; Calderon, Rebecca L; Craun, Michael F, Journal of Environmental Health , Volume: 65 , Number: 1 , Page: 16-23 , Jul/Aug 2002 ??CDC Report: Surveillance for water-borne disease outbreaks associated with drinking water - 2001-2002; MMWR Vol. 53 / No. SS-8, October 22, 2004 http://www.cdc.gov/mmwr/PDF/SS/SS5308.pdf ??Milwaukee, WI - Cost of Illness in the 1993 Cryptosporidium Outbreak in Milwaukee, WI, Corso, et al., Emerging Infectious Diseases, 2003: http://www.cdc.gov/ncidod/eid/vol9no4/pdfs/02-0417.pdf ??Bass Island, OH - Report by OH EPA on gastrointestinal illness outbreak in 2004 due to widespread contamination of ground water: http://www.epa.state.oh.us/ddagw/SBIweb/SBImain.htm ??NAS: Public Water Supply Distribution Systems: Assessing and Reducing Risks -- First Report, 2005: http://www.nap.edu/catalog/11262.html (Project site at http://www8.nationalacademies.org/cp/projectview.aspx?key=96) ??Drinking Water: Experts' View on How Federal Funding Can Best Be Spent to Improve Security (GAO-04-29), http://www.gao.gov/docdblite/summary.php?rptno=GAO-04-1098T&accno=A12881. ??Drinking Water Factoids: http://www.epa.gov/safewater/data/getdata.html

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: EPA is the only federal agency responsible for setting national standards for the quality of drinking water provided by public water systems. EPA is also the only federal agency responsible for regulating underground injection practices to protect the safety of underground drinking water sources. In most cases, States directly implement and enforce National Primary Drinking Water Regulations through State drinking water protection programs. In these instances, the national Program provides an oversight and advisory function while working with the States to prevent duplication of efforts. EPA has primary responsibility for facilitating the protection of the water sector (drinking water and wastewater systems). EPA has worked in close collaboration with the Department of Homeland Security to ensure that its efforts are either unique or complement other agency efforts.

Evidence: ??Safe Drinking Water Act 42 U.S.C. 300(f-j), especially §§ 1412; 1413; 1422, 1423. ??State/EPA Guiding Principles for Rule Development. ??PWSS logic model outline of EPA and State activities in support of the long-term outcome of safe drinking water. ??IG report on Response to Katrina in Louisiana and Mississippi demonstrates how EPA and state drinking water programs work together to help water systems. Louisiana report - http://www.epa.gov/oig/reports/2006/20060307-2006-P-00014.pdf. Mississippi report - http://www.epa.gov/oig/reports/2006/20060214-2006-P-00011.pdf. ??Examples of EPA direct implementation activities to protect public health in Wyoming.. ??Map showing how underground injection control programs are managed -http://www.epa.gov/safewater/uic/primacy.html. ??Homeland Security Presidential Directives - http://cfpub.epa.gov/safewater/watersecurity/legislation.cfm ??National Infrastructure Protection Plan - http://www.dhs.gov/nipp. ??National Response Plan - http://www.dhs.gov/interweb/assetlibrary/NRP_FullText.pdf

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The Safe Drinking Water Act (SDWA) requires EPA to consider national costs and benefits when developing drinking water regulations. In general, States have primary responsibility for implementation and enforcement of national drinking water regulations and SDWA provides flexibility in designing these programs. When designing the approach in SDWA, Congress recognized that while it is efficient at a national scale, it has a potential for inefficiencies at "small" systems. Since the majority of systems are "small" systems, SDWA includes a provision to address these potential inefficiencies. The Act requires EPA to identify affordable technologies for "small" systems. When such a system cannot afford to comply with a national drinking water regulation and it installs the affordable treatment technology, the Act allows the state to grant a variance to that system, which assures adequate protection of public health. EPA is currently revising its affordability methodology in order to better address this provision of SDWA.

Evidence: ?? Safe Drinking Water Act 42 U.S.C. 300(f-j) especially 1412, 1412(b)(4)(E), 1412(b)(3)(C), 1415(a)(1)(A), 1413, and 1422. ??Drinking Water Factoids which illustrate the following: In FY 2004, 8% (3,913) of the total (52,838) community water systems (CWS) served greater than 10,000 people, or 81% (220 million) of the total population served by CWSs (272 million). The remaining 19% (52 million) of the population served by CWSs, are served by the 92% (48,925) of systems that reach fewer than 10,000 people. Within this group, over half of the systems (30,006) serve fewer than 500 people. http://www.epa.gov/safewater/data/getdata.html ??Report of the Senate Committee on Environment and Public Works: Safe Drinking Water Act Amendments of 1995 (S. 1316). S. Rept. 106-169. (November 7, 1995). Section 14. ??EPA's July 2006 Federal Register notice to revise affordability methodology - http://www.epa.gov/safewater/smallsys/affordability.html.

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: Beneficiaries of the Drinking Water Protection Program include state drinking water programs, public water systems and consumers. The program is structured so that outputs such as tools and technical assistance are targeted to intended beneficiaries in order to achieve the program outcomes. The program design ensures adequate flexibility to annually adjust resource allocations to address shifting needs in order to effectively meet the program's purpose. Major needs include contaminant identification, regulatory development, and regulatory implementation. As the program identifies contaminants and develops regulations, it shifts resources towards regulatory implementation and support activities. The program has also continued to support core activities related to oversight, on-going regulatory development activities, monitoring for unregulated contaminants and data management. Funds are also made available annually to each Region's drinking water program so that highest priority, regional-specific projects can be carried out. There is no evidence that unintended subsidies are occurring.

Evidence: ??Summary of requirements in the 1996 SDWA amendments. http://www.epa.gov/safewater/sdwa/summ.html ??Annual budget requests and operating plans for the drinking water program between FY 1997 and FY 2005. ??Budget submissions for FY 2006 and 2007 - http://www.epa.gov/ocfo/budget/index.htm ??National Water Program Guidance - http://www.epa.gov/water/waterplan/ ??Public Water System Supervision Program Implementation Logic Model. ??Examples of Regional priority projects - oRegion 1 FY 2004 Regional Strategic Plan - drinking water initiatives described on pages 26-29. http://www.epa.gov/region1/about/pdfs/stratplan2004.pdf . Region 1 websites describing initiatives in their strategic plan: small systems initiative (http://www.epa.gov/region1/eco/drinkwater/small_dw_initiative.html ); schools initiative (http://www.epa.gov/region1/eco/drinkwater/schoolsnh2o.html) ; business partnership initiative (http://www.epa.gov/region1/eco/drinkwater/h2o_supplierbiz.html ) ; and water security initiative (http://www.epa.gov/region1/eco/drinkwater/dw-security.html)

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The program has two long-term performance measures, one outcome based and the other output based. These support the EPA goal of "Water Safe to Drink" by reducing the public's exposure to contaminants in drinking water and encouraging implementation of source water protection actions designed to minimize risks to sources of drinking water. The program has begun preliminary work to develop a high-level outcome measure (based on GAO's Hierarchy of Indicators) to assess the reduction in waterborne disease due to unsafe drinking water. In working to improve the measure, the program is working with several partners including the Centers for Disease Control and Prevention, the National Drinking Water Advisory Council, and other EPA Offices (Office of Research and Development and Office of Environmental Information). The program plans to include a final measure in the strategic plan to be developed in 2009.

Evidence: ??EPA 2003-2008 Final and 2006-2011 Strategic Plans, Goal 2, Objective 2.1, sub-objective 2.1.1. . http://www.epa.gov/ocfo/plan/plan.htm ?? Measure Development and Implementation Plan: Waterborne Disease Measure. ?? Waterborne Disease Measure Development Update (4/06/06). ?? "Managing for Results: EPA Faces Challenges in Developing Results-Oriented Performance Goals and Measures." GAO/RCED-00-77. April, 2000.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The program has clear baselines and targets which are both realistic and ambitious. The previous target for the outcome measure, set for the 2003 - 2008 Strategic Plan, was 95%. The program is proposing a new target of 91% by 2011, which is based on a larger set of regulations and a new, more realistic bottom-up approach to target development based on historic data. Therefore, though compliance appears to have decreased, it is not necessarily indicative of a decrease in drinking water quality because systems are reducing exposure to additional contaminants by implementing new regulations. The waterborne disease measure currently being developed should track the program's effectiveness in reducing waterborne illness.

Evidence: ??FY07 Draft National Water Program Guidance, Part III, "Water Program Management System", pages 56- 61. Available at http://www.epa.gov/water/waterplan/documents/060301FinalFY07NPGPRD.doc ??FY06 National Program Guidance, April 2005, Part III, "Water Program Management System," pp. 55-60; Available at: http://www.epa.gov/water/waterplan/documents/FY06NPGNarrative.pdf

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The program has four annual measures, two outcomes and two outputs, which track progress towards the long-term goal of "Water Safe to Drink.". These include an outcome-based measure that tracks the rate of compliance of the nation's community water systems with health based drinking water standards, which are designed to reduce the population's exposure to contaminants. The second outcome measure is a new measure which begins to incorporate the length of time that customers are potentially exposed to contaminants in water served by community water systems. The first output measure tracks state sanitary surveys. The second output measure is intended to track the quality of data in the program's Safe Drinking Water Information System (SIDWIS). The program is developing a water security measure.

Evidence: ??EPA 2003-2008 Final and 2006-2011 Strategic Plans, Goal 2, Objective 2.1, sub-objective 2.1.1. http://www.epa.gov/ocfo/plan/plan.htm ??National Water Program Guidance Program Activity Measures - FY07 Draft, at http://www.epa.gov/water/waterplan/index.html#VI

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The program has baselines for all of its annual measures as well as specific, ambitious, and realistic targets for three of its 4 annual measures.

Evidence: ??EPA 2003-2008 Final and 2006-2011 Strategic Plans, Goal 2, Objective 2.1, sub-objective 2.1.1. ??FY07 Draft National Water Program Guidance, Part III, "Water Program Management System", pages 56- 61. Available at http://www.epa.gov/water/waterplan/index.html#VI ??FY06 National Program Guidance, http://www.epa.gov/water/waterplan/index.html#nwp06

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: Partners of the National Drinking Water program include states, tribes, regions and other internal and external federal programs. As the Program's primary partners, states commit to the program's annual and long-term goals. The Program has two key mechanisms for involving partners: the "National Water Program Management System" (NWPMS) and the EPA Order on Environmental Results under EPA Assistance Agreements. The program and its partners use the NWPMS to define performance measures, set targets, evaluate interim and end-of-year progress, and revise program operations. The Program uses EPA's Order on Environmental Results to help connect activities funded by grants to Program goals. The program also has measures in place to ensure that work assignments for performance-based contracts include linkages to Program goals.

Evidence: ??FY07 Draft National Water Program Guidance, Part III, "Water Program Management System", pages 56- 61. Available at http://www.epa.gov/water/waterplan/documents/060301FinalFY07NPGPRD.doc ??Performance Measure Template for State Grant Workplans, Water Introduction: http://www.epa.gov/ocfo/npmguidance/pdfs/national_water_program_template_intro.pdf; Measures: http://www.epa.gov/ocfo/npmguidance/pdfs/ow_measures.xls ??FY05 Subobjective Implementation Plan for Safe Drinking Water/Security. Available at: http://www.epa.gov/water/waterplan/documents/drinking-sip.pdf ??Environmental Order No.: 5700.7, Environmental Results under EPA Assistance Agreements, effective: January 1, 2005 ??Environmental Order No. 5700.6 A1, Policy on Compliance, Review and Monitoring, revised January 8, 2004 ??Ohio SFY 2006 Annual Resource Deployment Plan (ARDP) (i.e., workplan). ??Massachusetts DEP Environmental Progress Report: Safe Drinking Water - http://mass.gov/dep/water/priorities/dwhome.htm ??Kansas PWSS FY 2005 Workplan -??Nevada has developed a workplan that integrates activities carried out using PWSS and DWSRF funding.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: Several independent organizations have performed high quality evaluations of various elements of the Drinking Water Protection program on a fairly regular basis. While no one evaluation provides information about the entire program, together the group of evaluations generally covers the entire program. In addition, the EPA Office of Inspector General (IG) plans to consider a near term capping study to summarize and aggregate the findings of recent program evaluations into one report. The IG also supports a risk based approach to selecting future evaluations. While the program does not have a detailed list of future evaluations, the dozens of recent evaluations and IG support of continued evaluations is sufficient to meet the "regular" criteria.

Evidence: ??OIG Memo (9/22/06): Program Evaluations of the National Drinking Water Program, from Wade T. Najjum, Assistant Inspector General for Program Evaluation to Cynthia C. Dougherty, Director, Office of Ground Water and Drinking Water ??OIG Multiyear Plan (2003-2005) - http://www.epa.gov/oig/reports/2003/Multi-YearPlan2003-2005.pdf (drinking water activities on pages 13-14, including EPA's Implementation of the Safe Drinking Water Amendments of 1996 as a specific track). ??OIG Two Year Plan (2004-2006) - http://www.epa.gov/oig/reports/2004/20040930-OIG-Two-Year-Plan.pdf (drinking water activities on page 6). ??External Evaluations of the Drinking Water Program (evaluators include EPA IG, Governmental Accountability Office, Congressional Budget Office and the National Academy of Sciences) ?? IG reports available at http://www.epa.gov/oig/reports/water.htm

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: The Drinking Water Protection Program does not link direct and indirect budgetary resources with annual and/or long-term performance targets. The annual OMB submission and Congressional Justification both present budget and performance information; however, the presentation of these does not make clear the impact of funding, policy or legislative decisions on expected performance.

Evidence: ??FY 2007 Congressional justification at http://www.epa.gov/ocfo/budget/2007/2007cj.htm.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: Past reviews of the major grant programs administered by the Drinking Water Protection Program identified an inability to measure public health benefits of drinking water regulations. These reviews recommended that the program develop an appropriate measure for assessing waterborne illness. The program still has a lot of work to do before it can implement and track a meaningful measure. This is primarily due to the current lack of robust data. However, the program has taken steps towards measure development. EPA is implementing a plan that includes coordinating with CDC to build capacity for improving waterborne illness reporting, identifying mechanisms by which to assess the exposure of people to waterborne contaminants, and developing approaches to estimate rates of waterborne illness in the nation. The program has also made progress on other strategic fronts. For example, the program negotiates with States on annual targets and commitments for performance measures, which link to the Agency strategic plan. In addition, using the National Water Program Guidance, the Program ensures that measures are meaningful while minimizing reporting burdens on states and Tribes.

Evidence: ??PART reviews of the Underground Injection Control program, Public Water System Supervision program and Drinking Water State Revolving Fund program ??Memorandum and guidance from the CFO on Regional Plans, FY 2007 operating year processes, and reporting on FY 2006 performance commitments: http://www.epa.gov/ocfo/npmguidance/final_guidance_12192005.pdf ??Performance Measure Template for State Grant Workplans: http://www.epa.gov/ocfo/npmguidance/template.htm ; Water Introduction: http://www.epa.gov/ocfo/npmguidance/pdfs/national_water_program_template_intro.pdf; Measures: http://www.epa.gov/ocfo/npmguidance/pdfs/ow_measures.xls ??Evidence for efforts to strengthen the strategic plan process related to the national water and drinking water programs are found in the FY 05-07 National Water Program Guidance, including the Appendices describing Program Activity Measures. At http://www.epa.gov/water/waterplan/ ??The PWSS program logic model ??Regional Source Water Protection Work Plan Interim Guidance ?? Waterborne Disease Measure Development Plan Update

Are all regulations issued by the program/agency necessary to meet the stated goals of the program, and do all regulations clearly indicate how the rules contribute to achievement of the goals?

Explanation: Under the structure of the Safe Drinking Water Act (1996) and the Agency's implementation of SDWA, reduction of contaminants in drinking water should lead to reduction in public heath risk from exposure to those contaminants. Therefore, the contaminant level and treatment technique regulations should be addressing the goal of providing clean and safe water. This will be further enhanced by improvement in measuring the actual health benefits from drinking water regulations, as discussed in question 2.1. Preambles to regulations issued by EPA clearly describe how they serve a compelling need to help the Agency achieve the goal of ensuring that water is safe to drink. Since the 1996 SDWA Amendments, the Agency has addressed or plans to address the regulations required in the Statute. In addition, the Agency reviews existing regulations and considers unregulated contaminants for regulation, as required by the 1996 SDWA amendments. EPA uses its regulatory development process to screen all regulations to ensure that they are necessary. When EPA finds that regulations are not likely to contribute to program goals the Agency has chosen not to proceed (e.g., Class V Underground Injection Wells, negative Regulatory Determinations).

Evidence: ??Safe Drinking Water Act, as amended 42 U.S.C. 300(f-j) includes both named contaminants/rules and new frameworks for reviewing and identifying contaminants. ??Stage 2 Disinfectants and Disinfection Byproducts Rule P at http://www.epa.gov/fedrgstr/EPA-WATER/2006/January/Day-04/w03.htm. ??Long Term 2 Enhanced Surface Water Treatment Rule at http://www.epa.gov/fedrgstr/EPA-WATER/2006/January/Day-05/w04a.htm. ??Revised arsenic standardavailable at http://www.epa.gov/safewater/arsenic/regulations.html. ?? ??First review of existing drinking water regulations completed in July of 2003 (http://www.epa.gov/safewater/review.html) ??Contaminant Candidate List 1 (CCL1) and Regulatory Determinations for CCL 1 (http://www.epa.gov/safewater/ccl/ccl1.html) ??Contaminant Candidate List 2 (CCL2) (http://www.epa.gov/safewater/ccl/ccl2.html) ?? Regulatory Actions for Class V wells -determination that additional regulations are not required for sub-classes of Class V wells: http://www.epa.gov/safewater/uic/classv/index.html#three.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: The Program collects performance information and uses it to manage and improve program performance. The program maintains a two-pronged approach to collecting information. The first is through consultation with Regional and state staff. Regions monitor each state's performance and direct technical assistance and guidance to low performing states. In addition, Regions carry out periodic reviews of programs and identify corrective action plans, where appropriate. Grants officers supplement this process by monitoring grants to track progress towards meeting commitments. Through the second prong, the program collects public water system violation data through the Safe Drinking Water Information System (SDWIS). This system does not collect water quality data, which would allow a more complete understanding of partner performance; however, the data in SDWIS are adequate for reporting on overall program performance. In addition to providing a more complete understanding of partner performance, a database of water quality information would greatly improve the national program's understanding of contaminant occurrence, which could be used in a variety of regulatory and non-regulatory applications. The program is encouraged to work towards such a system. In addition, reviews of SDWIS data indicate that it is incomplete; the program is working to address this issue. Finally, the program has additional programmatic data bases, which collect information to help monitor program progress.

Evidence: ??Safe Drinking Water Information System - http://www.epa.gov/safewater/databases.html. ??FACTOIDS: Drinking Water and Ground Water Statistics - available at http://www.epa.gov/safewater/data/getdata.html. ??Annual Commitment System - indicates regional and national targets and actual results ??Protocol for Data Verifications of State PWSS Programs ??List of Data Verifications & Program Review Reports (on file with the Program) ??Example Data Verification report: South Carolina Data Verification Final Report ?? Regional Reviews: Nebraska Annual Program Evaluation for 2005, Kansas Focused Annual Program Evaluation for 2005, Kansas UIC FY 2005 Workplan ??Region 4 review of the Lead and Copper Rule in Georgia ??Series of letters in 2000 to 2001 between EPA Region 1 and the Connecticut Department of Public Health indicate identification of deficiencies in the state's management of non-community public water systems and follow-up actions to address these. ?? Environmental Order No.: 5700.7, Environmental Results under EPA Assistance Agreements, effective: January 1, 2005. ??SDWIS Improvements: See evidence in Question 3.7.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Drinking Water Program managers and staff in Headquarters and Regions are held accountable for program performance, contracts management, and grants management through standards in annual performance appraisals. Additionally, grant and contract project officers are held accountable for ensuring that all policies and procedures of EPA's Grants Administration Division and contract regulations are followed and must be included as standard elements for grant/contract oversight in their performance agreements. Finally, contractors are held accountable for cost, schedule and performance results.

Evidence: ??2004 Senior Executive Service Revised Performance Standards; Memo from Morris Winn, February 19, 2004 ??Linking Performance Standards for Managers and Staff to EPA's Strategic Goals and Mission, Memo from David O'Connor; March 4, 2004 ??Office of Water Response to OARM Request to Review and Update Performance Standards for Grants Management , Memo from Mike Shapiro, December 12, 2003 ??Developing Performance Plans & Goal Cascading Summary Checklist ??OGWDW Performance Appraisal and Recognition System 2006 Template ??National drinking water program director performance standards ??Assessing FY 05 Mid-Year Performance Under the EPA Strategic Plan, Memo from Michael H. Shapiro, DAA to EPA ODs, RWDDs, et al; April 6, 2005. ??All work assignments in the Water Security Division must have language linking to strategic plan, for example: Work Assignment 0-02, Amendment 2 Performance Work Statement shows this linkage. ??Other examples of OGWDW performance-based contracts include the Pocket Guide: National Drinking Water Standards Task Order and the Task Order for production of an educational video about Lead in Drinking Water

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: The program creates an annual operating plan and allocates resources by goal, objective, program and object class. Obligations and expenditures for the entire program are tracked in the Agency-wide Integrated Financial Management System against the operating plan. Limited unobligated balances remain at the end of the year. Resource tracking is also conducted at the sub-program level, though a clear operating plan is not presented in this form of aggregation. EPA ensures that the terms and conditions of contracts and cooperative agreement awards reflect the program's goals and operating plans and the awardees' spending is consistent with the approved workplan. In Headquarters, use of funds and FTE are monitored through monthly status reports (weekly basis the last quarter of the FY), augmented by quarterly Senior Resource Official reports and mid-year and annual program reviews.

Evidence: ??FY 2005 Preliminary Operating Plan guidance memo from Deputy Office Director to Senior Managers - provides information to guide resource planning for the fiscal year. ??OGWDW Budget Structure to track spending- shows funding alignment with the Agency strategic plan. ??End of Year Status of Funds reports for 2006, 2005, 2004, 2003 indicate essentially no unobligated balances. The program also tracks status of funds on a monthly basis - several months in 2006 were provided. ??FY 2006 Budget Object Class Report for Drinking Water Programs program/project from Budget and Accounting System - shows operating plan and actual obligations, which indicate limited unobligated balances at sub-program level ??FY 2005 Mid-year management briefing for Senior Management.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The program has an efficiency measure, which tracks people receiving drinking water in compliance with health-based drinking water standards per million dollars. This measure has baselines and targets and will capture improvements in program performance. Therefore, it is relevant to the program's purpose and useful at a macro scale. The program has some concerns with the measure since due to its strong correlation to population. An increase in population without a proportionate increase in spending will appear as an efficiency improvement, though in reality this would not be a true improvement in programmatic efficiency (i.e. doing more with less). While the measure is has some meaning and utility, the program is working on developing a more useful measure for annual tracking of programmatic efficiency. The office has increased its use of performance-based contracting to improve cost-effectiveness by making contractors more accountable for their actions. Project officers must conduct a cost review of assistance agreements. The program's IT efforts have focused on reducing reporting burden and enhancing productivity. Most of the drinking water program's IT investments go toward its Safe Drinking Water Information System (SDWIS). Over the past several years, the office has been working to modernize this data system, which has resulted in cost savings to the program and its partners.

Evidence: ??Policy for Competition of Assistance Agreements (January 2005) http://www.epa.gov/ogd/competition/5700_5A1.pdf ??Feb 2006 Interim Guidance on the Evaluation of Applicants Past Performance in Reporting on Outputs and Outcomes ??Cost Review Analysis Guide for Office of Water Project Officers ??EPA Order 2100.2A1: Information Technology Capital Planning and Investment Control (CPIC). ??CPIC Lite and E-Government Guidance for the OMB Submission ??Applied Information Economics (AIE) Analysis of SDWIS Plus

Does the program collaborate and coordinate effectively with related programs?

Explanation: The program collaborates and coordinates effectively with other related programs. The program coordinates with EPA Office of Research and Development, the Centers for Disease Control and Prevention (CDC), the United States Geologic Survey (USGS) and other research partners to improve the base of science which support regulatory decisions. The program seeks input from federal advisory committees and state co-regulators to improve the quality of its decisions. The program works with a wide range of partners to ensure that systems have training and technical assistance to improve their ability to provide safe drinking water. The program collaborates with environmental stakeholders, federal agencies, and other EPA programs to ensure that sources of drinking water are protected. To ensure the security of drinking water, the program collaborates with the Department of Homeland Security and serves as the Chair of the Water Government Coordinating Council, consisting of federal (e.g., DHS, CDC, Department of Interior, Food and Drug Administration) and state agencies. EPA regional offices work closely with states, associations, environmental groups and other stakeholders within their regions to address state and local priorities and build support for drinking water protection.

Evidence: ??EPA White Paper: Collaborative Efforts in the National Drinking Water Program - Contains more examples of collaboration ??EPA Drinking Water Research Multi-year plan revision - team rosters including ORD and OW staff, meeting agendas, and communications between ORD and OW. ??2006 HQW Research Advisory Council Workgroup Process ??EPA Meeting with CDC Centers (NCEH and NCID) ??Stakeholder Meetings Concerning the Long-Term 2 Enhanced Surface Water Treatment Rule and Stage 2 Disinfectants and Disinfection Byproducts Rule Proposals; Notice of Public Meetings (http://www.epa.gov/fedrgstr/EPA-WATER/2003/September/Day-22/w24121.htm) ??EPA meeting on January 18, 2006 with the Association of State Drinking Water Agencies - Agenda and Summary ??EPA Region 1/Rhode Island Department of Health Partnership Agreement for Implementation of the Long Term 2 Enhanced Surface Water Treatment Rule and the Stage 2 Disinfectants and Disinfection Byproducts Rule ??Cooperative Agreement with the Association of Metropolitan Water Agencies to develop source water protection tools and training (http://www.cleanwaterfund.org/sourcewater/)

Does the program use strong financial management practices?

Explanation: The program follows EPA's financial management guidelines for committing, obligating, reprogramming, and reconciling appropriated funds. Agency officials have a system of controls and accountability, based on GAO, Treasury and OMB guidance as well as generally accepted accounting practices (GAAP). EPA trains individuals to ensure that they understand their roles and responsibilities for invoice review and for carrying out the financial aspects of program objectives. As part of oversight procedures for the DWSRF program, EPA has measures in place to minimize erroneous payments. EPA received an unqualified audit opinion on its financial statements and had no material weaknesses associated with the audit.

Evidence: ??Audit of EPA's Fiscal 2005 and 2004 Consolidated Financial Statements (2006-1-00015) - http://www.epa.gov/oig/reports/2006/20051114-2006-1-00015.pdf. ??FY 2005 Annual Financial Statement -http://www.epa.gov/ocfo/finstatement/2005par/par05_intro_fin_stmnts.pdf

Has the program taken meaningful steps to address its management deficiencies?

Explanation: The program has a system for identifying and correcting management deficiencies. Internal program reviews and EPA IG reviews have identified data quality as a program deficiency. The drinking water program has put processes in place to address the closeout of the Agency level weakness for the Safe Drinking Water Information System (SDWIS) by the end of 2007. The SDWIS weakness is being addressed by efforts to modernize the data system, most of which were completed by the end of 2005. The program is continuing to work with primacy agencies to address program issues relative to compliance determinations and completeness of reporting violations to SDWIS. The frequency and scope of data verification audits of the state drinking water programs has been increased to ensure that states are reviewed every three years. The program is also on track to address Agency-identified weaknesses associated with the Water Security component of the Agency's Homeland Security responsibilities.

Evidence: ??IG Report: EPA Claims to Meet Drinking Water Goals Despite Persistent Data Quality Shortcomings, at http://www.epa.gov/oig/reports/2004/20040305-2004-P-0008.pdf ??SDWIS - FMFIA status as of February 2006 ??Information Strategic Plan: Modernizing the Safe Drinking Water Information System for the Public Water System Supervision Program http://www.epa.gov/safewater/sdwisfed/modernization/pdfs/strategic-plan_sdwisfed_jan2004.pdf ??Data Analysis and Action Plan (2000 and 2003) - http://www.epa.gov/safewater/data/reports.html ??August 2003 and December 2004 Memos from Cynthia Dougherty to Regions outlining changes to Data Verification audit program. ??March 2006 Corrective Action Plan for Water Security Weaknesses ??Mid-year management briefing for DAA.

Did the program seek and take into account the views of all affected parties (e.g., consumers; large and small businesses; State, local and tribal governments; beneficiaries; and the general public) when developing significant regulations?

Explanation: When developing regulations, the program identifies the affected parties, communicates the context of the rulemaking, shares underlying data and analysis, and solicits comment on regulatory options. This dialogue often goes beyond what is required by law, such as working closely with state co-regulators, convening multi-stakeholder advisory committees or small business advocacy panels. When required or otherwise appropriate, EPA consults with specific groups of stakeholders, such as small business representatives, or elected officials. The Agency reviews all comments and incorporates new information as part of reaching a decision for a final regulation. Preambles to the regulations summarize the analysis behind and consequences of regulatory decisions. Preambles to final rules also summarize the major comments received and discuss why comments were or were not addressed. The program also prepares a complete response to comments document and makes it available to the public through the public docket.

Evidence: The program provided an exhaustive list of evidence indicating their effectiveness in seeking and considering input from affected parties when developing regulations. The following is a sample of items from this list. ??Principles for working with State Co-regulators: State/EPA Guiding Principles for Rule Development ??Summaries of Public Meetings related to Safe Drinking Water Act Implementation http://www.epa.gov/safewater/ndwacsum.html ??Example of use of Federal Advisory Committee to help inform requirements in proposed rule: The Stage 2 Microbial and Disinfection Byproducts (MDBP) FACA process resulted in an Agreement in Principle that provided the Agency with recommendations for requirements that would be included in the Stage 2 Disinfection Byproduct Rule (Stage 2) and Long Term 2 Enhanced Surface Water Treatment Rule (LT2), both of which were issued in December 2005 available at http://www.epa.gov/safewater/disinfection/stage2/regulations.html#history and http://www.epa.gov/safewater/disinfection/st2agreement.html ??Example of stakeholder meetings held to explain proposed rules: Stakeholder Meetings Concerning the LT2 and Stage 2 rules. Notice of public meeting - http://www.epa.gov/fedrgstr/EPA-WATER/2003/September/Day-22/w24121.htm. ??Examples of soliciting comments on proposed rules: Stage 2, Proposal, 68 FR 49565, 49574, August 18 2003. http://www.epa.gov/safewater/disinfection/stage2/regulations.html; LT2, 68 FR 47678, August 11, 2003. Proposal http://www.epa.gov/safewater/disinfection/lt2/regulations.html ??Example of notice providing additional information after proposal and prior to final rule: Notice of Data Availability for the Ground Water Rule - http://www.epa.gov/safewater/disinfection/gwr/regulation.html ??Example of responses to Comments for Final Rules: Stage 2, Final Rule, 2006. http://www.epa.gov/safewater/disinfection/stage2/regulations.html.

Did the program prepare adequate regulatory impact analyses if required by Executive Order 12866, regulatory flexibility analyses if required by the Regulatory Flexibility Act and SBREFA, and cost-benefit analyses if required under the Unfunded Mandates Reform Act; and did those analyses comply with OMB guidelines?

Explanation: The program generally addresses all applicable requirements for regulatory analyses. While there have been concerns with the initial quality of some regulatory analyses, the program has made significant progress in improving the analyses, partly due to the analytical requirements of the 1996 amendments to SDWA. The program follows procedures for rule-making in accordance with the Agency's Action Development Process which include procedures for complying with Executive Orders, other statutory requirements, and OMB circulars. In addition, the SDWA (Sec. 1412(b)(3)(C)) requires that the program prepare Health Risk Reduction Cost Analyses for each regulatory alternative that include consideration of the following: quantifiable and non-quantifiable health risk reduction benefits from reducing the regulated contaminant and co-occurring contaminants, quantifiable and non-quantifiable costs, incremental costs and benefits, effects of the contaminant on the general population and sensitive sub-populations, possible increased health risks, and all uncertainties. To comply with these requirements, the program evaluates cost and benefit estimates of the potential regulatory alternatives against baseline conditions. Incremental costs and benefits are analyzed through an evaluation of successively more stringent regulatory alternatives. Details of how the requirements have been addressed are included in the economic analyses and preambles to proposed and final rules. Economic analyses comply with OMB guidelines (Circular A-4) and examine alternative approaches.

Evidence: ??EPA's Action Development Process Guidance, Guidance for EPA Staff on Developing Quality Actions ??Guidelines for Preparing Economic Analysis, EPA 240-R-00-003, September 2000. ??Examples of RFA, SBREFA, Cost Benefit and UMRA analysis: Stage 2 Disinfectants and Disinfection Byproducts Rule (Stage 2), Proposal, 68 FR 496, August 18 2003. http://www.epa.gov/safewater/disinfection/stage2/regulations.html; Stage 2, Final FR 49623 - 58, http://www.epa.gov/fedrgstr/EPA-WATER/2006/January/Day-04/w03.pdf; Long Term 2 Enhanced Surface Water Treatment Rule (LT2), Proposal 68 FR 47738-66, August 11, 2003. http://www.epa.gov/safewater/disinfection/lt2/regulations.html; LT2, Final 71 FR 730-762, http://www.epa.gov/fedrgstr/EPA-WATER/2006/January/Day-05/w04a.htm; Ground Water Rule (GWR), Proposal, 65 FR 243-263, http://www.epa.gov/safewater/protect/gwrfr.html ??GAO Review of Environmental Protection Agency: National Primary Drinking Water Regulations: Long Term 2 Enhanced Surface Water Treatment Rule GAO-06-354R, January 19, 2006 - http://www.gao.gov/decisions/majrule/d06354r.pdf ?? GAO Review of Environmental Protection Agency: National Primary Drinking Water Regulations: Stage 2 Disinfectants and Disinfection Byproducts Rule, GAO-06-350R, January 13, 2006, http://www.gao.gov/decisions/majrule/d06350r.pdf

Are the regulations designed to achieve program goals, to the extent practicable, by maximizing the net benefits of its regulatory activity?

Explanation: While the program thoroughly considers benefits and costs of regulatory activities, they do not specifically seek to maximize net benefits. In developing drinking water rules, the Safe Drinking Water Act (SDWA) requires that EPA set uniform national maximum contaminant levels (MCLs) as close as close as "feasible" to the level "at which no known or anticipated adverse effects on the health of persons occur," taking costs to large systems into account. Where national benefits of the feasible level do not justify national costs, SDWA Section 1412(b)(6)(A) provides the program with the authority to establish the standard at a level that "maximizes health risk reduction benefits at a cost that is justified by the benefits." EPA has on two occasions (uranium and arsenic) established MCLs at levels higher than the feasible level. Since SDWA requires EPA to set uniform national standards for economically dissimilar systems, it is not conducive to maximizing net benefits. Congress contemplated instances when benefits might not justify costs and provided specific provisions for small systems, for which costs can and often do greatly exceed benefits. However, under the current policy, no small systems have been able to take advantage of this provision and the Agency is in the process of revising its methodology to assess the affordability of regulations to address this difficulty.

Evidence: ??EPA's July 2006 Federal Register notice to revise affordability methodology -http://www.epa.gov/safewater/smallsys/affordability.html. ??Report of the Senate Committee on Environment and Public Works: Safe Drinking Water Act Amendments of 1995 (S. 1316). S. Rept. 106-169. (November 7, 1995). Section 14. ?? Safe Drinking Water Act 42 U.S.C. 300(f-j) ??Stage 2 Disinfectant and Disinfection Byproducts Rule (Stage 2) Preamble: 71 FR 468. http://www.epa.gov/fedrgstr/EPA-WATER/2006/January/Day-04/w03.htm. ??Long Term 2 Enhanced Surface Water Treatment Rule (LT2): 71 FR 754 http://www.epa.gov/fedrgstr/EPA-WATER/2006/January/Day-05/w04c.htm. ??Economic Analysis for the following final rules: Arsenic (2001) - http://www.epa.gov/safewater/arsenic/pdfs/econ_analysis.pdf; Long Term 2 Enhanced Surface Water Treatment Rule (2005) - http://www.epa.gov/safewater/disinfection/lt2/pdfs/anaylis_lt2_economic_main.pdf; Filter Backwash Rule (2001): http://www.epa.gov/safewater/mdbp/fbrr_econ_anal.pdf; Stage 2 Disinfectants and Disinfection Byproducts Rule (2005) - http://www.epa.gov/safewater/disinfection/stage2/pdfs/anaylsis_stage2_ecconomic_main.pdf

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: Nationally, the percent of the population served by community water systems meeting all health-based standards increased steadily from 79% in 1993 to 90% in 1999 and then leveled out, with the exception of 2002 when compliance was 94%. The program initially set targets for the 2003-2008 Strategic Plan based on a baseline of 94% in 2002, which was the latest data available at the time. The targets were 92% for 2003 and 2004 and 93% for 2005, which the program failed to meet. However, over this time, the percent population served by systems meeting all health based standards has stayed fairly level, with a slight decrease to 88.5% in 2005. Preliminary data for the first 3 quarters in 2006 and the last quarter in 2005, indicate approximately 89.3% of the population was served by systems meeting all health based standards. The program's failure to meet initial targets is, in part, a function of the ambitious targets set by the program, based on an anomalous baseline. Additionally, the measure is sensitive to violations of standards at large systems. Overlaying these complexities are the changing standards with which community water systems have to comply. The drinking water program continuously reviews existing regulations and emerging contaminants, which periodically leads to changes in regulations or new regulations. Therefore, the current health based standards are more protective than the 2002 standards. In response to these insights, the program has revised the baseline to a more representative year (2003) and updated the out-year targets to maintain their ambitiousness while remaining realistic. In addition, the program has developed a new annual measure to better capture exposure by incorporating the duration of a violation. For the long-term measure focused on source water protection, the program has achieved the target it set for 2006 that 20% of source water areas minimize risks by substantial implementation of source water actions in a protection strategy. This achievement is notable in that implementation of a protection strategy is not a requirement of the law and reflects voluntary efforts on the part of states and local communities. The program is continuing efforts to develop new waterborne disease measures.

Evidence: Program data show the volatility of the percent population served measure to violations at large systems: In 2005, a system serving 6.5 million people had 2 short-duration violations of the Surface Water Treatment Rule (one was less than one day and the other was 4 months). The population served by the system accounted for 2.3% of the 4.5% difference between the 2005 target (93%) and the actual measure (88.5%). Regional data also show this volatility. The regions with the lowest 2005 values were Region 2 (55.3%) and Region 6 (87.8%). In Region 2, nine systems in New York and Puerto Rico accounted for 10.2 of the 14.2 million people in the region who were served water by a community water system with a violation. ??Status of Progress on Waterborne Disease Measure Development Plan

Does the program (including program partners) achieve its annual performance goals?

Explanation: The program has targets for 3 of its 4 measures and is developing targets for the measure of SDWIS data completeness. In 2005, the program failed to achieve its target associated with the percentage of community of water systems that meet all health-based drinking water standards. This measure is similar to the long-term percent population served measure discussed in question 4.1. Like the long-term measure, the failure to meet targets for this measure is, in part, a function of the ambitious targets set by the program. As new regulations come into effect, there tend to be more non-compliance issues because systems had not been monitoring for a contaminant and were thus unaware that they would exceed a new standard or because systems require time to make the treatment changes needed to meet the new standard. The measure included in the PART covers all rules, and therefore can be negatively affected by new rules. Like the long-term percent population measure, the program has revised the targets for the annual community water systems measure. The new targets are more realistic while maintaining their ambitiousness. The drinking water program also has had a measure focused on ensuring that states are fulfilling their duties in conducting sanitary surveys, with a target of 94% (of 50 states and Puerto Rico). The program met this target for 2005. The program has modified this measure to focus on community water systems with an ultimate target that 100% of systems have a sanitary survey meeting the schedule required by regulation. The final measure will complement the long-term population measure by better capturing exposure by incorporating the duration of a violation. The program has baseline data and will begin collecting actuals in 2007.

Evidence:

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The program's efficiency measure assesses the population served by systems meeting applicable health-based drinking water standards per million dollars spent on managing the national drinking water program as well as applicable grants, state-match funds and applicable drinking water state revolving funds provided for assistance. This measure is a slight revision from a previous measure, which was developed for the Public Water System Supervision, Underground Injection Control, and Drinking Water State Revolving Fund PARTs as well as the Agency Strategic and Annual Plans. The previous measure captured a smaller sub-set of program inputs but showed improving efficiencies from 2003 to 2005 (520,615 people per million dollars to 528,206 people per million dollars). This measure will be revised in all applicable drinking water program PARTs and GPRA documents. The program is also working to develop an annual efficiency measure, which will be more useful and meaningful for assessing programmatic efficiencies. The program continues to identify and improve its efficiency and cost effectiveness. Examples include the modernization of the Safe Drinking Water Information Systems (SDWIS); successful efforts to increase the number of states using SDWIS-STATE; development of electronic sanitary survey and UIC inspection tools, and use of internet-based training to reach states and water systems.

Evidence: ??See evidence under Question 3.4 ??Drinking Water Academy - http://www.epa.gov/safewater/dwa.html ??Electronic Sanitary Survey http://www.epa.gov/safewater/dwa/e-sansurvey.html. ??Fact sheet describing Class V UIC PDA effort

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: As a whole, the national EPA Drinking Water program is unique. There are no other EPA programs, federal agencies, or nongovernmental programs engaged in implementation and enforcement of drinking water protection. State drinking water programs complement EPA's role, generally without duplication, and are supported in part by this program, so a comparison of State and federal programs would not be meaningful. There are some aspects of the national Drinking Water program which could be compared to state programs. In particular, both the national program and state programs engage in direct implementation of rules as well as provide outreach and training. Though these are small portions of the overall national program, comparisons would ensure the national program performs activities similar to state activities in the most efficient and effective way.

Evidence:

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: Evaluations of the drinking water program generally indicate that the program design is sound and that the program is generally effective in developing and implementing drinking water standards designed to protect the nation's drinking water quality and public health. Evaluations have recognized program strengths and remaining challenges. More than 30 studies have been conducted over the past several years by parties including the EPA Inspector General and Government Accountability Office. As previously discussed, the drinking water program measures compliance with drinking water standards, which are designed to protect public health. Therefore, an assumption is made that compliance is equivalent to public health protection. However, the program does not measure the ultimate public health outcomes of its regulations. Though this requires substantial improvements in currently available data, the program is working with partners to develop such a measure.

Evidence: ??IG reports available at http://www.epa.gov/oig/reports/water.htm ?? External Evaluations of the Drinking Water Program (evaluators include EPA IG, Governmental Accountability Office, Congressional Budget Office and the National Academy of Sciences)

Were programmatic goals (and benefits) achieved at the least incremental societal cost and did the program maximize net benefits?

Explanation: There is no evidence of periodic analyses of the actual effects of all significant regulations and therefore it is not possible to determine whether the rules maximized net benefits. In addition, as discussed in 3.RG4, it is likely that net benefits are not maximized under current regulations. There have been some independent retrospective analyses of costs and/or benefits. In addition, there are extensive prospective analyses of expected costs and benefits during rule development.

Evidence: ??Predicting Community Water System Compliance Choices: Lessons from the Past. Final report prepared for US EPA, Office of Policy, Economics, and Innovation. Stedge, G. et al, Abt Associates, Inc., 2000. Other analyses considering post-rule costs and benefits ??Economic Analysis for the following final rules: Arsenic (2001) - http://www.epa.gov/safewater/arsenic/pdfs/econ_analysis.pdf; Long Term 2 Enhanced Surface Water Treatment Rule (2005) - http://www.epa.gov/safewater/disinfection/lt2/pdfs/anaylis_lt2_economic_main.pdf; Filter Backwash Rule (2001): http://www.epa.gov/safewater/mdbp/fbrr_econ_anal.pdf; Stage 2 Disinfectants and Disinfection Byproducts Rule (2005) - http://www.epa.gov/safewater/disinfection/stage2/pdfs/anaylsis_stage2_ecconomic_main.pdf