The UST Program Appraisal Strategy OSWER Directive 9610.6 May 6, 1988

Directive Organization

INTRODUCTION
THE OUST APPROACH: ASSESSING AND IMPROVING PERFORMANCE
1. Taking Stock
2. Improving Performance
ROLES AND RESPONSIBILITIES
PROGRAM PRIORITIES FOR FY 98
THE UST PROGRAM APPRAISAL PROCESS

APPENDIX A: STATE PERFORMANCE MEASURES

State Prevention Program Measures

State Clean-up Program Measures

State Program Priorities Measures

Program Appraisal Roles

APPENDIX B: REGIONAL PERFORMANCE MEASURES

Regional Prevention Program Measures

Regional Clean-up Program Measures

Regional Program Priorities Measures

APPENDIX C: HEADQUARTERS PERFORMANCE MEASURES

Headquarters Prevention Program Measures

Headquarters Clean-up Program Measures

Corrective Action

Headquarters Program Priorities Measures

APPENDIX D: REPORTING REQUIREMENTS/SPMS

NOTE: The document you are viewing is an HTML facsimile of OSWER Directive 9610.6 that has been reformatted for the Internet. This version maintains as much as possible of the original document integrity. Only a couple of non-essential elements are missing, namely facsimiles of the OSWER Directive cover page, and EPA Form 1315-17 (the Directive Initiation Request). Also, the original typed document had the directive number as a header on each page—in this version the directive number appears at the beginning of each new section.

OSWER DIRECTIVE 9610.6

I. INTRODUCTION

The purpose of this document is to explain the system of program appraisal, based on a franchise management approach, that will be used by EPA’s Office of Underground Storage Tanks (OUST) This document should clarify the franchise approach as it applies to program appraisal by:

Describing the nature of the program appraisal process, and how it, works to lift the level of performance by all of us working to protect human health & the environment from potential or existing releases from USTs;
Clarifying the roles and. responsibilities of headquarters, Regions, and States;
Defining clear performance goals and expectations by outlining national priority areas and listing the specific performance measures that will be used to help evaluate States, Regions, and Headquarters programs in FY 89. The performance measures are included in the appendices of this strategy.

Top of page

OSWER DIRECTIVE 9610.6

II. THE OUST APPROACH: ASSESSING AND IMPROVING PERFORMANCE

The goal of the program appraisal process is to ensure protection of human health and the environment from UST releases by helping to build strong and effective State and local programs. This process provides a systematic means for us (1) to take stock of our collective accomplishments and (2) to work together to raise the level of performance by States, Regions, and Headquarters. Through regular communication and feedback on issues, the process will focus on solving or anticipating issues such that the performance of all (States, Regions, and Headquarters) is improved.

1. Taking Stock

The first job of program appraisal is to provide an accurate representation of what we have accomplished and how far we have to go to achieve our long range goals. It is a learning process that is designed to generate facts about current program results that will be useful to States and EPA.

These facts will provide a realistic program baseline from which we can build for success in the long run. Establishing a realistic baseline is critical given the wide range of program development currently, found in the UST community, and the tendency to begin with unrealistically high expectations for start-up programs.

Tracking progress over time is also essential for identifying areas where policies and programs are more and less successful. Less successful areas will call for corrections or new initiatives. In addition, periodic taking stock is necessary so that we can report to Congress and the public on how effectively their environmental and public health goals are being implemented.

2. Improving Performance

States and EPA will constantly work together to identify ways to improve performance - by owners/operators, installers, clean-up contractors, State and local personnel, and by EPA Regional and Headquarters offices. States and Regions will work to identify successful technical and management practices at the State and local level and work with headquarters to make these practices available to other areas of the country. Each State will also work with its Regional office to identify performance improvement areas and assistance needs for its own program. Headquarters will support Regional efforts and work with them to identify system-wide tools such as videos, technical guidance, or management systems.

Top of page

OSWER DIRECTIVE 9610.6

III. ROLES AND RESPONSIBILITIES

1. The State’s Role

States have the lead role in field implementation of UST and LUST Programs and must balance State environmental priorities with national ones. States exercise discretion in how to organize and staff for program implementation, and choose how to best meet federal Subtitle I objectives tailored to State and local conditions.

As primary implementing agencies, States play a major role in the program appraisal process.

Working with Regional offices, States determine goals and priorities for the State program which are reflected in grants and cooperative agreements.
Through timely and accurate quarterly reporting and annual self-appraisals, States provide the raw facts for us to take stock of accomplishments.
By making available information regarding successful efforts, States foster improvement throughout the system.
States also work with Regions to identify areas to improve their program and needs for assistance from EPA.

Two major tools will be available to the States for taking stock of their own programs. First, States will have available from EPA data on national program progress which they can use as a point of reference for their own programs. Second, Regions will ask States to conduct a self-appraisal that will result in States completing a brief State accomplishments report designed to get the State's perspective on their accomplishments and needs.

In what areas was the State most successful and what were the keys to success?
Did the State meet expectations as outlined in grant and cooperative agreement workplans and performance measures?
What challenges did the State face in implementing the program, and how did they address them?
What areas are the State targeting for improvement in the coming year and how do they plan to achieve results?
What assistance does the State need from EPA?

2. The Role Of EPA’S Regional Offices

EPA’s Regional offices are the most important link in the program appraisal process. Key objectives for the Regions are:

to establish a good working relationship with States and develop expert knowledge of State performance and progress in key programmatic areas;
to work with States to identify their needs for improvement and assistance;
provide assistance to States; and
to work with headquarters to identify success stories and problems areas deserving of national attention.

Program appraisal will normally be linked to the ongoing process of negotiating and evaluating performance under grants and cooperative agreements, but will also occur through less formal communications, visits, and efforts at joint problem solving.

During the negotiation of grants and cooperative agreements, Regions and States will agree on performance goals and the means by which the Region will appraise the level of State performance throughout the year. A major element of this appraisal will normally be formal mid-and/or end-of-year reviews. These reviews should be documented in a brief written assessment of State accomplishments, needs, and possible performance improvement areas. Based on discussions with States regarding improvement areas, Regions will target services and resources to assist States in either problem diagnosis or developing solutions. Regional responses could include providing contractor assistance, assigning a Regional staff person on a temporary basis, arranging training, modifying a State workplan, or other creative approaches.

Regional offices will use a variety of tools to assist, in their assessment of State programs. The State performance measures in the appendix provides a useful checklist of program areas to examine. In addition, SPMS data and other quantitative quarterly reports contribute to the baseline of information. Regions will also review the timeliness and accuracy of reports, and oversee State progress in meeting commitments in grant and cooperative agreement workplans.

Regional offices outside of UST (including Grants, Planning and Management, Emergency Response, Regional Council and Financial Management) serve as experts in their functional areas in the State UST program appraisal process. UST Regional Coordinators coordinate with these other offices in their evaluations of UST State program performance.The financial management officer in the Region will be responsible for reviewing State program financial and fiscal accounting procedures.

Regions will also contribute to the appraisal process by conducting a self-appraisal leading to a brief Regional accomplishment report. This report is designed to provide a Regional perspective on program progress and needs in the Region. The report should be based on the Regional performance measures which will focus on:

In what areas was the Region most successful in responding to State needs and assisting States in improving performance? What were the keys to this success?
What major challenges did the Region face in carrying out its responsibilities, and how did they address them?
What areas is the Region targeting for improving its performance in the coming year, and how do they plan to achieve results?
What assistance and support does the Region need from headquarters?

The Regions can also use this opportunity to evaluate Headquarters performance. The measures included in appendix C list measures for Headquarters activities. The Regions should evaluate Headquarters performance according to the definitions of performance provided. These evaluations are to be used during Headquarter's annual review of the Region.

3. The Role Of EPA Headquarters

Headquarters is responsible for supporting Regions and States in implementing the UST program. It issues and revises regulations, develops guidance, establishes national program priorities and policies with significant involvement and input by States and Regions. It provides tools and resources to State and Regional UST programs, and compiles and assesses national information. In the appraisal process, headquarters will work with Regions and States to clearly define agreed upon expectations for Regional performance, including criteria used to assess performance. These expectations and criteria are defined in The Agency Operating Guidance, OUST Grant and Cooperative Agreement Guidance, the Transition Strategy, and Appendix B of this document.

Headquarters will conduct annual reviews of each Region focusing on Appendix B Regional Performance Measures, and will provide feedback to help improve Regional performance. It will offer tools, such as contractor resources, training materials and data management systems to help States and local governments implement UST programs. Headquarters will use reported information to compile statistics Regions and States can use. They will also compile and disseminate information regarding successful Regional practices, particularly ways that Regions have been most successful in improving State performance.

Some issues may need to be explored in greater depth, particularly when issues arise which affect several Regions. To meet this need, Headquarters, with Regional assistance, may initiate detailed studies that focus on issues of concern. The need for these studies may arise from Regional requests for assistance or policy direction, information from quarterly reporting data, or constantly recurring operational issues. An example of a focused study is the cost recovery pilots project.

Top of page

OSWER DIRECTIVE 9610.6

IV. PROGRAM PRIORITIES FOR FY 89

The Agency Operating Guidance for FY 89 states the priority activities for UST programs. Appendix A, B, and C translate FY 89 priorities into specific performance measures for States, Regions, Headquarters. Appendix D lists the State reporting requirements and SPMS measures.

STATE PROGRAM APPROVAL: Develop State authorities and funding mechanisms, meet the "no less stringent federal objectives" as stated in the State Program Approval Handbook, and submit/approve complete applications for State program approval.
ADMINISTERING THE LUST TRUST FUND: Negotiate and oversee Trust Fund cooperative agreements which increase emphasis on site work and oversight of responsible party clean-ups and decrease the emphasis on program development.
PROMOTE COMPLIANCE WITH REGULATIONS: Offer training and technical assistance to States in order to develop State capabilities to implement effective State regulatory and clean-up programs. Provide education and information to the regulated community on their regulatory responsibilities.
TRANSITION STRATEGY: Use existing State activities as a foundation for negotiating formal agreements with States with grants and cooperative agreements to begin implementing the federal regulations during the period after the final Federal regulations are promulgated and before State programs are approved. Work with States to help them develop technical and enforcement expertise as needed.

Top of page

OSWER DIRECTIVE 9610.6

THE UST PROGRAM APPRAISAL PROCESS
TAKING STOCK — IMPROVING PERFORMANCE

State Roles

Work with Regions to determine goals and priorities for grants and cooperative agreements.
Produce Quarterly Reports to define national program progress.
Produce Self-Appraisals to give feedback on State accomplishments and needs.

Regional Roles

Evaluate performance of States under grants and cooperative agreements.
Keep in contact with States to learn State programs.
Perform mid- and/or end-of-year reviews of State programs.
Produce a Brief Written Assessment which outlines State accomplishments needs and possible improvement areas.
Target resources to State needs.
Produce Self-Appraisals to give feedback on program progress and to evaluate Headquarters.