Detailed Information on the
Pesticide Field Programs Assessment

Establish executive leads to provide senior leadership for each of the 3 mission areas in the new Strategic Plan.

The pesticide programs were restructured to match the 2011 Strategic Plan and the current PARTs programs do not exist. The pesticide program is now waiting a rePART to align the PART programs with the current budget and Strategic architecture.

Include a $1 million reduction in funding for the Field Programs WQ program in the FY 2006 President??s Budget. EPA must ensure that WQ program activities affected by this reduction are adequately addressed in the Office of Water??s Surface Water Protection program.

Make the Field Programs budgeting more transparent and more clearly link to adequate and relevant program-specific measures.

Develop and implement annual goals and efficiency measures and continue development of baselines and targets for long-term outcome measures for all Field Programs.

Develop and implement a method of compiling and disseminating Field Programs grantee performance data in a manner easily accessible to the public. EPA worked with states to develop a simplified, electronic, EOY reporting system for worker saftey activities. Will expand to other field programs by EOY 2007.

Implement new strategic plan architecture into FY 08 management activities and day-to-day operations.

Establish executive leads to provide senior leadership for each of the 3 mission areas in the new Strategic Plan.

Brief staff on new Strategic Plan in order to incorporate stronger alignment between Strategic Plan individual Performance Agreement and Recognition System (PARS) agreements.

Executive leads working toward the development and refinement of meaningful outcome oriented measures for each of the three mission areas in the new Strategic Plan.

Independent assessment of the performance measures improvement project by the Federal Consulting Group.

Measure: Percent reduction in concentrations of pesticides detected in general population

Explanation:Current PART measures were based on the 2003-2008 Strategic Plan. With the development of the new Strategic Plan, EPA and OMB emphasized improving the outcome-orientation of pesticide program performance measures that link the pesticide programs and mission, as reflected in the Pesticide Field Program's PART improvement plan. The proposed measure is an environmental outcome (health) indicator (environmental indicator hierarchy level = 5, body burden/exposure). Data are from CDC's National Health and Nutrition Examination Survey (NHANES). Units are ug/L. Baseline Text: According to NHANES data for 1999  2002 the concentration of pesticides residues detected in blood samples from the general population are:Dimethylphosphaste = 0.41 ug/L, Dimethylthiophosphate = 1.06 ug/L, Dimethyldithiophosphate = 0.07 ug/L,Diethylphosphate = 0.78 ug/L,Diethylthiophosphate = 0.5 ug/L,Diethyldithiophosphate = 0.07 ug/L and 3,5,6-Trichloro-2-pyridinol = 1.9 ug/L.

Measure: Incidents per 100,000 potential risk events in population occupationally exposed to pesticides.

Explanation:Current PART measures were based on the 2003-2008 Strategic Plan. With the development of the new Strategic Plan, EPA and OMB emphasized improving the outcome-orientation of pesticide program performance measures that link the pesticide programs and mission, as reflected in the Pesticide Field Program's PART improvement plan. The proposed measure is an environmental end-outcome (health) indicator (environmental indicator hierarchy level = 6, measure of health). Data come from Poison Control Centers that participate in a national data collection system known as the Toxic Exposure Surveillance System (TESS). Among the types of exposures reported are pesticide related exposures. Collected data include date of call, age, gender, location of exposure, route of exposure, substance exposed to, initial symptom assessment, treatment received and an evaluation of the medical outcome. Symptoms are categories as minor, moderate, or major with criteria for each category.

Measure: Percent reduction in moderate to severe incidents for six acutely toxic agricultural pesticides with the highest incident rate

Explanation:The Agency will use Poison Control Center (PCC) incident data (TESS) and EPA risk assessments to calculate the baseline for this measure. The top Six Agricultural Pesticides showing the most agricultural-work related (occupational) incidents covering five years from 1999 to 2003 for which the measure will target for annual 10% reduction for EACH of the pesticides are: chlorpyrifos, 67 incidents; diazinon, 51 incidents; malathion, 36 incidents; pyrethrins, 29 incidents; 2, 4-D, 27 incidents; carbofuran, 24 incidents. A 50% targeted reduction in moderate to severe incidents for six acutely toxic agricultural pesticides with the highest incident rate by 2011.

Measure: Billions of dollars in crop loss avoided by ensuring that effective pesticides are available to address pest infestations

Explanation:Current PART measures were based on the 2003-2008 Strategic Plan. With the development of the new Strategic Plan, EPA and OMB emphasized improving the outcome-orientation of pesticide program performance measures that link the pesticide programs and mission, as reflected in the Pesticide Field Program's PART improvement plan. The proposed measure is an environmental end-outcome (health) indicator (environmental indicator hierarchy level = 6, measure of health). Data come from Poison Control Centers that participate in a national data collection system known as the Toxic Exposure Surveillance System (TESS). Among the types of exposures reported are pesticide related exposures. Collected data include date of call, age, gender, location of exposure, route of exposure, substance exposed to, initial symptom assessment, treatment received and an evaluation of the medical outcome. Symptoms are categories as minor, moderate, or major with criteria for each category.

Measure: Million of dollars in termite structural damage avoided annually by ensuring safe and effective pesticides are registered/re-registered and available for termite treatment

Explanation:Current PART measures were based on the 2003-2008 Strategic Plan. With the development of the new Strategic Plan, EPA and OMB emphasized improving the outcome-orientation of pesticide program performance measures that link the pesticide programs and mission, as reflected in the Pesticide Field Program's PART improvement plan. The proposed measure is an environmental end-outcome (health) indicator (environmental indicator hierarchy level = 6, measure of health). Data come from Poison Control Centers that participate in a national data collection system known as the Toxic Exposure Surveillance System (TESS). Among the types of exposures reported are pesticide related exposures. Collected data include date of call, age, gender, location of exposure, route of exposure, substance exposed to, initial symptom assessment, treatment received and an evaluation of the medical outcome. Symptoms are categories as minor, moderate, or major with criteria for each category.

Measure: Average cost and average time to produce or update an Endangered Species Bulletin

Explanation:Efficiency measure developed under new SP architecture and as part of the OPP Measures Development Group.

Measure: Average cost and average time to produce or update an Endangered Species Bulletin

Explanation:

Measure: Reduced cost per pesticide occupational incident avoided

Explanation:Efficiency measure developed under new SP architecture and as part of the OPP Measures Development Group.

Measure: Reduced cost per pesticide occupational incident avoided

Explanation:

Measure: Reduced cost per acres using reduced risk management practices compared to the grant and/or contract funds on environmental stewardship.

Explanation:Efficiency measure developed under new SP architecture and as part of the OPP Measures Development Group.

Is the program purpose clear?

Explanation: EPA's Pesticide Field Programs comprises three major program areas: Endangered Species (ES), Worker Protection/Certification & Training (WP/CT), Water Quality (WQ). The first two of these three have very clear purposes which can be directly tied to statutory requirements. The ES program directly supports the needs of EPA's pesticide Registration and Reregistration needs by implementing Endangered Species Act consultation requirements. The purpose of the WP/CT programs is to train and certify applicators to protect human health and the environment. The mission of the WQ program is vague - to protect America's water resources from pesticide use - can not be as readily tied to pesticide-related statutory authority. Grants are provided to states for efforts related to each of these programs but clear statutory authority for issuing grants in all areas is not apparent (ES, WQ in particular).

Evidence: The Endangered Species Act (ESA) (16 USC 1531) §7(a)(2) requires agencies to ensure that their actions (e.g., pesticide registration) are not likely to jeopardize the continued existence of any endangered or threatened species. Federal Insecticide, Fungicide, and Rodenticide Act (FIFRA), 7 USC §136i and §136u authorize certification and training of pesticide applicators. FIFRA §136w provides authority to issue worker protection standards (40 CFR 170). The Office of Prevention, Pesticides and Toxic Substances (OPPTS) 2005-07 National Program Guidance from Principal Deputy Assistant Administrator of OPPTS to OPPTS Regional Division Directors, April 21, 2004. The Pesticide Environmental Stewardship Program (PESP) and the Strategic Agricultural Initiative (SAI) (also part of Field Programs) were developed in part to support the requirements in Food Quality Protection Act (FQPA) of 1996 (Public Law 104-170). Grant authority: FIFRA §136r and §136u.

Does the program address a specific and existing problem, interest or need?

Explanation: The Field Programs address specific and ongoing risks posed by the use of registered pesticides in the field. The use of registered pesticides according to their labels can still result in residues in both ground and surface water. The ES program brings EPA into compliance with the Endangered Species Act by protecting listed threatened or endangered species from registered pesticides. The WP/CT program ensures that pesticides are applied safely and worker health is protected. Both the PESP and the SAI encourage pesticide users to move toward safer pest management strategies voluntarily.

Evidence: Endangered Species Act (ESA) (16 USC 1531) §7(a)(2). Final joint counterpart regulations by the U.S. Fish and Wildlife Service (FWS) and the National Marine Fisheries Service of the National Oceanic and Atmospheric Administration (NOAA): 8/5/04 Federal Register Vol 69, No 150, pg 47732 (50 CFR 402). Federal Insecticide, Fungicide, and Rodenticide Act (FIFRA), 7 USC §136i and §136u. Food Quality Protection Act (FQPA) of 1996 (Public Law 104-170).

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: For some elements of the pesticide Field Programs, redundancy is a concern. The ES and WP/CT Field Programs are not redundant of other efforts. It is not evident that the purpose of WQ program is sufficiently independent and not duplicative of EPA's Office of Water (OW). Though the WQ program does work with OW, it is not clear that the WQ program efforts are adequately distinct from OW's activities. Insufficient evidence was provided to prove that the PESP and SAI programs are not redundant or duplicative of other Federal efforts.

Evidence: EPA works closely with public and private sector partners in some areas to ensure that activities which could promote common objectives are designed and carried out effectively and not unnecessarily duplicative: US Dept. of Agriculture (USDA) Cooperative State Research, Education, and Extension Service on Certification and Training Programs; Fish and Wildlife Service (FWS) and the National Marine Fisheries Service on ES program design, operations, and decisions.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: There is no strong evidence that a different approach or mechanism would be more efficient of effective. The programs are generally able respond to changes in the field because their design requires stakeholder involvement. Where problems have been identified, solutions are being implemented. To address design and operational deficiencies identified through litigation, the ES program is implementing comprehensive reforms. No major design flaws have been identified in the WQ or WP/CT programs, other than the potential duplication of the WQ efforts and the efforts of EPA's Office of Water.

Evidence: Industrial Economics, May 2002: Results of Primary Evaluation of the Endangered Species Protection Program's County Bulletin Program. Pacific Northwest Agricultural Safety & Health Center at the University of Washington, April 2004: Evaluation of the Worker Protection Standard, Train-the-Trainer Model Curriculum. GAO report, March 2000: Improvements Needed to Ensure the Safety of Farmworkers and Their Children.

Is the program effectively targeted, so that resources will reach intended beneficiaries and/or otherwise address the program's purpose directly?

Explanation: Grant resources are equitably targeted but it is not clear that definitive authority exists in the Federal Insecticide, Fungicide, and Rodenticide Act (FIFRA) for all the grants issued (water quality and endangered species-related grants in particular). It is also not clear that the pesticide-related water quality issues would or could not be addressed through other EPA grant programs. In general, grants to states and/or regions include base funding and additional funds based on formulas which consider the scope of issues in each state and/or region.

Evidence: EPA's pesticides grant funding formulas and spending. Outline of allocation amounts to state lead agencies. Cooperative agreements and end of year reports.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The program has proposed an adequate long-term outcome measure for the WP/CT program which focuses on greater protection of agricultural workers. Long term measures for the ES and WQ programs were also proposed but are not satisfactory. Some of the goals proposed for the other programs are on the right track and the Field Programs have agreed to work on developing better measures. Measures Development and Implementation Plans have been submitted.

Evidence:  

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: Because a No was given for 2.1, PART guidance requires a No for this question. Further development of targets and timeframes will be included in the programs' measures development work. A baseline and outyear targets are under development of the new WP/CT long-term measure.

Evidence:  

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: No adequate annual performance measures have been proposed for any of the Field Programs. Work will be needed to establish meaningful annual goals that show how Field Programs' activities directly contribute to proposed long term measures. The programs have committed to developing efficiency measures. One efficiency measure is required for each major program evaluated in this PART.

Evidence:  

Does the program have baselines and ambitious targets for its annual measures?

Explanation: Because a No was given for 2.3, PART guidance requires a No for this question. Further development of targets and timeframes will be included in the programs' measures development work.

Evidence:  

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: Since the programs do not have adequate long-term and annual goals and thus received a No for 2.1 and 2.3, PART guidance requires that the programs receive a No for this question. The programs do establish cooperative and other grant agreements with their state/regional partners. Certain individual expectations (some of which align with overarching agency Strategic Plan elements) are set forth in these documents and Year End Reviews are completed which summarize overall progress. This will be a useful mechanism for ensuring grant recipient partners commit to program goals once adequate goals are established. It is not clear how the programs will ensure that other partners, such as USDA's Cooperative State Research, Education, and Extension Service, commit to and work toward the same or substantially similar goals.

Evidence:  

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: While EPA and others have conducted some external and internal program reviews to assess the operation and effectiveness certain elements of some Field Programs, no independent evaluations that meet PART guidance requirements are conducted on a regular basis.

Evidence: In 2004, the Environmental Stewardship program was formally evaluated by an outside source; the results of that evaluation should help improve the program.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: The budgets of the Field Programs taken independently can not tie to program-specific goals because no adequate program-specific goals have been established. The total budget for the Field Programs is tied to the Office of Pesticide Programs (OPP's) goals outlined in the agency's Strategic Plan. This can be seen in OPP's Rainbow Table which is used for budget tracking. Inadequate evidence was provided to explain how the Field Programs manage their budget internally and how they link funding to specific activities.

Evidence:  

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The Field Programs have taken actions to address strategic deficiencies and does do planning on a multi-year schedule. The programs do provide input to the Office of Pesticide Programs (OPP) strategic planning process which is feeds into the agency-wide process. Also, the Field Programs are participating in the OPP-wide effort to improve performance measurement and develop better metrics. It is not clear that deficiencies in the Field Programs are caught and/or addressed early or quickly. Strategic decisions do not seem to be preventative in nature and some were clearly made after issues were brought to the programs' attention by external forces.

Evidence: Field Programs strategic priorities are laid out in the National Program Manager Guidance to Regions which is distributed every 2-3 years. Examples of steps taken to correct strategic deficiencies: As a result of significant losses in court cases, the ES program identified an over-reliance on a voluntary approach for achieving endangered species requirements. EPA and its ES partner agencies have established counterpart regulations that overhaul the EPA's ES review process for pesticides. The WQ program is in transition from a rule-making approach to a more flexible operational approach building on state and tribal experience.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: Data on state and regional Field Program-related activities is collected on an annual basis through End-of-Year grant agreement reviews. Some state's reviews tie back to recommendations made in prior year's or mid-year assessments. Assessments at the regional level are also completed. But, these reports focus on state-specific outputs and other recommendations, many of which don't have quantifiable targets or baselines and most do not tie to outcomes. The reports are not consistent in their content or structure and adequate performance measures are not used across all (or even a majority of) states, tribes, and regions. It is not clear how data from these reports is complied and put in a useful format for making program management or strategic decisions. The reporting structure is well established though and once the Field Programs implement adequate performance goals, they will need to be somehow incorporated into the state's agreements.

Evidence:  

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: EPA management performance standards include links to high-level pesticide-related objectives in the current strategic plan. These performance standards are included in each manager's Annual Performance Agreement which serves as the basis for both the mid-year and end-of-year performance evaluation. States discuss program/grant results with regional offices in mid- and end-of-year grant report meetings. When adequate Field Program-specific performance measures are established, these will need to be reflected in the performance standards specifying performance requirements and mandate performance and progress reporting.

Evidence: EPA Managers Annual Performance Agreements. Annual Grant Guidance. Specific language in grants.

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: Inadequate evidence was provided regarding the Field Programs' obligation tracking and methodolgy for ensuring obligations are consistent with the overall program plan.

Evidence:  

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The program does not have procedures in place to measure and achieve efficiencies. It has, on occasion, done things to improve program efficiencies. An example is the IT improvement work done in collaboration with the US Geological Survey (USGS) to more efficiently create products (e.g., maps) necessary to implement the endangered species program.

Evidence:  

Does the program collaborate and coordinate effectively with related programs?

Explanation: The programs work closely with other programs that have related responsibilities. The water quality field program participates in a cross-program work group with EPA's Office of Water that is overseen by a senior management council that holds quarterly meetings. The reports from this group show significant overlap in responsibilities between the two programs. The workgroup attempts to minimize the duplication of effort. Timely and effective registration and deregistration of pesticides requires that the ES program work closely with the Registration and Reregistration programs as well as consultation with Fish and Wildlife and National Marine Fisheries Services, and the USGS on geospatial data. On worker safety issues, EPA works closely with the USDA to coordinate the worker protection and certification and training programs.

Evidence: Though the Office of Water (OW) and the WQ program overlap in their purpose, mission, and activities, the WQ program is receiving credit in the question for making an effort to coordinate efforts with OW: Summary of meetings of the OW/Office of Pesticides Program Coordination Group (Priority Activities Matrix). ES program's joint Advanced Notice of Proposed Rulemaking (ANPR) (Federal Register, Volume 68, No. 16. January 24, 2003, p 3786) announcing the intention of the Fish and Wildlife Service (FWS), and the National Marine Fisheries Service (NMFS) in cooperation with EPA to promulgate "counterpart regulations" under the Endangered Species Act. For the WP/CT program: Interagency Agreements between EPA and the US Dept. of Agriculture to carry out the certification and training programs.

Does the program use strong financial management practices?

Explanation: The programs are free of material weaknesses. EPA's contract and grant management policies and procedures require stringent financial management practices. All grantees have been subject to grants office and program audits. The Field Programs use the agency-wide Integrated Financial Management System (IFMS).

Evidence: EPA contract and grant policies. Pesticide program's cooperative agreement guidance. Agency Financial Statements and audit results. Agency-wide Integrated Financial Management System (IFMS) reports.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: Some management deficiencies have been identified and addressed but it is not evident that the various Field Programs have a systematic approach to evaluate the effectiveness of their management of these programs and then to address any deficiencies identified in those reviews. For example, in 2003 EPA and the USDA identified a significant backlog in the processing and commitment of training funds. This undercut the timely delivery of training to pesticide users. EPA worked aggressively with the USDA to successfully eliminate this problem. The program took significant actions against a noncompliant grantee by withholding funds and signaling potential debarment. These actions resulted in the grantee fulfilling his grant obligations and commitments.

Evidence:  

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: The Field Programs use a thorough End-of-Year reporting process to document grantee activities and accomplishments. No evidence was provided of Field Programs tracking grantee's actual expenditures and the majority of these annual reports do not tie grant dollars to specific actions. EPA regional offices do conduct mid- and end-of-year reviews with grantees. Grantee WQ efforts are reviewed during the annual reporting process and through monthly calls and quarterly stakeholder meetings. EPA regional offices review the ES performance of each grantee at the end of the year and provide that report to EPA headquarters. WP/CT program activity is conducted through cooperative agreements with special conditions requiring regular interaction and reports between the recipient and the EPA program project officer.

Evidence: State Pesticide End of Year Reports - summary of site visit by EPA regional staff to state offices to partially fulfill EPA's responsibilities to provide oversight and review of the EPA Cooperative Agreement with each state. End-of-Year Evaluations of each state program performed by EPA regional offices.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: The Field Programs do collect general grantee performance data through annual review process and will make it available to the public upon request. This information is not compiled and disseminated in an easily accessible manner and no evidence was provided to show that the data is aggregated in any way.

Evidence:  

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: The program did propose several new long-term measures but only one met the adequacy threshold. The acceptable measure applied to the WP/CT program only. The programs are working to develop adequate long-term performance measures that focus on outcomes. The Field Programs do not currently utilize performance measures that adequately reflect progress for each individual program and therefore there are no consistent data available to measure historical performance.

Evidence:  

Does the program (including program partners) achieve its annual performance goals?

Explanation: A No in response to Question 2.3 requires that the Field Programs receive a No answer for this question. Some existing output annual performance measures have been achieved.

Evidence:  

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: No quantified efficiency or cost-effectiveness improvements were provided. Actions have been taken in some Field Programs that are expected to improve efficiency or cost-effectiveness: Agreements with the Fish and Wildlife Service and the National Marine Fisheries Service to implement the proposed Counterpart regulation established timelines for consultation should improve the ES program. The WP/CT program has worked with USDA to address the grant processing backlog thus providing funding for additional training programs.

Evidence: Advanced Notice of Proposed Rulemaking (Federal Register, Vol 68, No. 16 page 3786) announcing the intent of the FWS, the NMFS ' in cooperation with the EPA to promulgate "counterpart regulations" under the Endangered Species Act. Interagency Agreements (IAGs) between EPA and the USDA.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: No documented thorough comparisons of the Field Programs to other similar programs have been done. Based on the information reviewed in this PART and the level of collaboration between the Field Programs and similar programs, some aspects of the Field Programs do compare favorably.

Evidence: The Field Programs are similar in nature to other programs and thus opportunities exist for comparison but it not clear that it would cost effective to undertake thorough or complete comparisons given the size of these programs.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: While many external and internal evaluations (discussed in Question 2.6) have provided data on progam progress and effectiveness, and while some of these evaluations have resulted in changes in the design and operations of the programs, these evaluations were not independent and of sufficient scope and quality and conducted on a regular basis.

Evidence: EPA should consider contracting with an independent third party to do evaluations of sufficient scope and quality on a regular basis.