Detailed Information on the
Toxic Air Pollutants - Regulations and Federal Support Assessment

Focus on maximizing programmatic net benefits and minimizing the cost per deleterious health effect avoided.

Use the newly developed efficiency measure to demonstrate efficiency improvements.

Increase funding for toxic air pollutant programs by $7 million in State grants for monitoring to help fill data gaps.

By the end of March 2008, brief OMB on proposals for implementing a toxicity-weighted efficiency measure.

Measure: Percentage reduction in tons of toxicity-weighted (for cancer risk) emissions of air toxics, compared to 1993 baseline.

Explanation:Measures percent reduction in the inventory of air toxic emissions (from a 1993 baseline), calculated as tons of emissions and multiplied by a unit risk estimate. All outyears are normalized to the same 1993 non-weighted emission value of 7.24 million tons.

Measure: Percentage reduction in tons of toxicity-weighted (for noncancer risk) emissions of air toxics, compared to 1993 baseline.

Explanation:Measures percent reduction in the inventory of air toxic emissions (from a 1993 baseline), calculated as tons of emissions and divided by the reference concentration to get noncancer tons. All outyears are normalized to the same 1993 non-weighted emission value of 7.24 million tons.

Measure: Cumulative percentage reduction in tons of toxicity-weighted (for cancer risk) emissions of air toxics, compared to 1993 baseline.

Explanation:Measures percent reduction in the inventory of air toxic emissions (from a 1993 baseline), calculated as tons of emissions and multiplied by a unit risk estimate. All outyears are normalized to the same 1993 non-weighted emission value of 7.24 million tons.

Measure: Cumulative percentage reduction in tons of toxicity-weighted (for noncancer risk) emissions of air toxics, compared to 1993 baseline.

Explanation:Measures percent reduction in the inventory of air toxic emissions (from a 1993 baseline), calculated as tons of emissions and divided by the reference concentration to get noncancer tons. All outyears are normalized to the same 1993 non-weighted emission value of 7.24 million tons.

Measure: Tons of toxicity-weighted (for cancer and noncancer risk) emissions reduced per total cost ($). (Targets and baseline under development)

Explanation:Will measure cumulative reduction in toxicity-weighted emissions divided by estimated total dollars spent by the Federal Government and regulated industries.

Is the program purpose clear?

Explanation: The Clean Air Act Amendments (CAAA) of 1990 require regulation of air toxics from motor vehicles under Title II and from stationary sources through the maximum achievable control technology (MACT) program under Title III.

Evidence: Technology-based toxics standards are required for mobile and stationary sources, followed by a residual risk-assessment. EPA issued 96 MACT standards, and a mobile source toxics rule. The residual risk program is underway, though EPA is still addressing some science and data issues.

Does the program address a specific and existing problem, interest or need?

Explanation: People exposed to toxic air pollutants at sufficient concentrations and durations may have an increased chance of getting cancer or experiencing other serious health effects. Prior to the 1990 Amendments, EPA set standards based on risk assessments, but promulgated few of them. Congress then mandated standards based on the best available technology with subsequent evaluation of risk.

Evidence: Acute and chronic health effects of air toxics are detailed on EPA's Air Toxics website: www.epa.gov/ttn/atw/. Without the air toxics programs called for by the 1990 Amendments, EPA estimates that overall emissions would have increased 36% from 1990 levels by 2007.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: EPA is the only agency to develop national regulations for industrial and mobile sources of air toxics. In addition, EPA regulates transportation fuels to ensure nationwide consistency and fungibility.

Evidence: State/local agencies have not developed standards and a State-by-State program could create inconsistency. Federal mobile source regulation helps make more uniform requirements for vehicle manufacturers and oil companies.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: In the 1990 CAAA, Congress provided a two-tiered structure for regulating air toxics emissions from major stationary sources - technology based standards followed by additional standards if the residual risk at these facilities is too high. For mobile sources, Congress instructs EPA to set technology-based standards, taking into consideration feasibility and cost. Though the current structure for stationary source regulation has proven to be more effective than the air toxics program established by the 1970 Clean Air Act (CAA), the underlying statute is not designed to maximize net benefits. Section 112 of the CAA requires technology-based standards that limit flexibility in achieving risk reductions and have generated costs in excess of benefits. EPA has some regulatory flexibility, and the program has attempted to develop regulations that minimize the cost, but it is not possible to maximize net benefits under the current statute.

Evidence: Section 112 CAA.

Is the program effectively targeted, so that resources will reach intended beneficiaries and/or otherwise address the program's purpose directly?

Explanation: Program funds are spent to complete statutory requirements and implement standards designed to reduce emissions of air toxics

Evidence: Section 112 of the CAA.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The program's purpose is to reduce air toxics emissions to reduce the risk to public health and the environment. As a means of measuring the program's progress toward risk reduction, EPA has established a performance measure that tracks toxicity-weighted emissions reductions. Toxicity-weighting evaluates the potential toxicity of each air toxic by its relative carcinogenic and non-carcinogenic toxicity factor. This permits the agency to sum and compare reductions of all HAPs with available health data. EPA performs this assessment for both cancer and noncancer emissions, resulting in two separate performance measures. According to the General Accounting Office (GAO) Hierarchy of Indicators, this is a low-level (level 3) outcome measure. In the near future, EPA will begin to augment this measure with a more direct method for demonstrating risk reduction. This new measure will utilize ambient air toxics data that the Agency began collecting in 2003. A Measure Implementation Plan (MIP) for this ambient data is undergoing OMB evaluation.

Evidence: MIP for Toxicity-weighted emissions and MIP for Ambient air toxics data.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The program has targets for reducing toxicity-weighted air toxics emissions. These targets are achievable, as the reduction will result from rules that have already been promulaged, but the program has not demonstrated that they are ambitious. In fact, according to performance data, these targets have already been achieved. The program's performance measures use a 1993 baseline based on emissions inventory data from 1990 and 1993. Due to changes in measurement methodology, comparisons of current estimates to this baseline are unreliable.

Evidence: MIP for Toxicity-weighted emissions.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: Annual performance will be demonstrated through tons of toxicity-weighted emissions reduced on an annual basis. As an increment of the long term performance measures, this measure can demonstrate progress towards long term goals. As above, emissions reduction targets will be stated both in tons of cancer and non-cancer emissions, and these emissions will cover all HAPs with available health data. In the future, the ambient air toxics data discussed in 2.1 will also be used to measure annual progress. The program is also developing an efficiency measure that will compare toxicity weighted emissions reductions to estimated total dollars (federal & non-federal) spent on air toxics standards. Since it is impossible to link the annual input in this efficiency measure with annual outcomes, this measure will be long term and cumulative.

Evidence: MIP for Toxicity-weighted emissions and MIP for Ambient air toxics data.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: Annual targets are set based on estimates of emissions and risk reductions that will be achieved through implementation of stationary source standards. The program uses targets of toxicity weighted emissions reductions for cancer and non cancer health risks. Both annual targets are achievable, as the reductions will result from rules that have already been promulaged, but the program has not demonstrated that the targets are ambitious. In fact, according to annual performance data, both targets have already been achieved and emissions will actually rise slightly over the next two years. The program's performance measures use a 1993 baseline based on emissions inventory data from 1990 and 1993. Due to changes in measurement methodology and limited data, comparisons of current estimates to this baseline are unreliable.

Evidence: www.epa.gov/ttn/chief/trends for information on the baseline inventory.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: EPA's contract statement of work includes requirements for analytical support for projecting emission inventories, estimating inventory changes, and assessment of health and environmental impacts. Grant workplans for State and local partners include commitments to implement toxics programs. However, sample statements of work and grant workplans do not include performance measures that are clearly aligned with program goals.

Evidence:  

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: Independent evaluations have been conducted, but the program's evaluation history does not meet all criteria. The impact of the program has not been rigouously assessed. The majority of past reviews have focused on specific aspects of program such as monitoring or emissions reporting and are not of sufficient scope. Also, there is no program evaluation plan in place to ensure ongoing reviews of sufficient quality, scope and independence.

Evidence: The Science Advisory Board (SAB) evaluated the design of an air toxics monitoring network in 2000. The SAB also reviewed EPA residual risk methodology and a case study. The National Academy of Sciences (NAS) recently completed a study (Air Quality Management in the United States) that includes a limited assessment of the performance of the air toxics program. The IG and GAO periodically evaluate aspects of the program. A recent IG report analyzed EPA's method for calculating and reporting results.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: The Budget documents do not clearly indicate the full costs of achieving Air Toxics performance goals.

Evidence: EPA's Annual Plan and Budget Request documents.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The program has acted to correct strategic planning deficiencies. The program continues to work to improve its ability to demonstrate progress toward achieving its long term goals.

Evidence: To address planning deficiencies, EPA's strategic plan now focuses on a smaller set of Agency goals with measurable outcomes. To address weaknesses identified through independent studies and through the PART, the program has developed several new performance measures, including toxicity-weighted emissions. The program has also deployed a national air toxics monitoring network covering a select set of air toxics as a means of increasing the information about air toxics in ambient air and provide trends information over the long-term.

Are all regulations issued by the program/agency necessary to meet the stated goals of the program, and do all regulations clearly indicate how the rules contribute to achievement of the goals?

Explanation: In the 1990 CAAA, Congress required EPA to set standards to reduce air toxics from both stationary and mobile sources, providing specific goals and criteria for these programs. In promulgating these standards, EPA has made clear how they fulfill the requirements of the CAA.

Evidence: Section 112 of the CAA and published MACT standards.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: Though EPA has made significant progress in developing its national air toxic trend sites (NATTS), a national network of 22 long-term monitoring sites, this system does not yet produce useful performance information. EPA is also in the final stages of developing a community-based air toxic monitoring program that will enable air toxic assessments on a local level. EPA has worked with partner at state/local/tribal agencies to design and develop this program. When underway, this program may provide information useful to local air toxics management. EPA continues to improve its air toxics inventory (the 1999 NEI had participation from all but 4 states) and the quality and quantity of the information in the inventory continues to improve, but this data is still of limited value for program management.

Evidence: Ambient data from the 22 NATTS sites will support the development of trends and the eventual correlation with reduction efforts. OMB is reviewing a measurement implementation plan for performance measures based on the NATTS data.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: EPA's contract statement of work includes cost schedules and expected outcomes. EPA and state grantees develop a state grant work plan that incorporates required outcomes and deadlines. Performance standards for Federal managers are linked to Strategic plan goals, but there is no clear evidence that managers and program partners are held accountable for results.

Evidence:  

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: EPA prepares an Operating Plan that OMB and Appropriations Committees approve. Program offices track spending against this plan, which is aligned with the strategic plan. EPA works with grantees to ensure that spending is consistent with the negotiated work plan. As part of each office's post-award monitoring of grants, recipients are required to affirm that funds designated to each program area are indeed spent for the intended purpose.

Evidence: Spending reports for the toxics program indicate timely obligation.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: EPA typically selects contractors through a competitive process based on costs, at times it awards noncompetitive contracts due to looming deadlines. Beginning in FY 2003, EPA awarded non-state grants competitively.

Evidence: OAR relies on work assignments placed against negotiated competitive contracts awarded through full and open competition. Each procurement undergoes a cost analysis that feeds directly into the negotiation process. Each potential contractor must also undergo an outside audit to determine the reasonableness of costs, particularly indirect costs.

Does the program collaborate and coordinate effectively with related programs?

Explanation: The Agency collaborates and coordinates with State, local and tribal air toxics programs during the development of standards, data gathering, and community air toxics assessments. EPA gives States the opportunity to place a representative on workgroups developing a standard. It coordinates with States in developing the National Air Toxics Assessment and its periodic updates. EPA aids communities doing assessments to identify risk and solutions such as diesel retrofits. EPA works with state/local governments to assess mobile source-related exposure and risk. EPA also works with State/local governments to identify and implement voluntary programs. However, there is insufficient evidence that these collaborative activities lead to meaningful results.

Evidence:  

Does the program use strong financial management practices?

Explanation: The air toxics program has no financial weaknesses as reported by the IG and has procedures in place to minimize erroneous payments.

Evidence: IG's FY 2003 identification of material weaknesses. Agency officials have a system of controls and accountability, based on GAAP and other principles, to ensure that improper payments are not made. EPA trains individuals to ensure that they understand their roles and responsibilities for invoice review and for carrying out the financial aspects of program objectives.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: The program has implemented recommendations included in reviews by the NAS, IG and GAO.

Evidence: The program is currently addressing recommendations from the recent NAS study: Air Quality Management in the United States (2004). The program has also taken steps to address weaknesses identified in the FY 2004 PART review. Further, the Agency will soon be developing an action plan to address and implement many of the recommendations in the recent OIG report (due to the IG on June 30, 2004). While the OIG audit list includes Air Toxics - Phase 2 (Residual Risk) as an Agency Candidate Weakness for FY 2004, the program is working to develop several innovative approaches to address potential weaknesses. These approaches include a total facility low risk demonstration and comprehensive residual risk rule that will help the Agency's efforts to target facilities with the highest risk.

Did the program seek and take into account the views of all affected parties (e.g., consumers; large and small businesses; State, local and tribal governments; beneficiaries; and the general public) when developing significant regulations?

Explanation: EPA follows administrative procedures of notice and comment as well as other statutory requirements such as the Small Business Regulatory Enforcement Fairness Act (SBREFA).

Evidence: Regulations are proposed in the Federal Register. All interested parties have an opportunity to submit comments. EPA evaluates them and makes necessary revisions before promulgating regulations. In cases where regulations may impact a substantial number of small entities, EPA sets up SBREFA panels.

Did the program prepare adequate regulatory impact analyses if required by Executive Order 12866, regulatory flexibility analyses if required by the Regulatory Flexibility Act and SBREFA, and cost-benefit analyses if required under the Unfunded Mandates Reform Act; and did those analyses comply with OMB guidelines?

Explanation: EPA's RIAs usually but do not always comply with OMB's economic analysis guidelines. EPA generally reports total costs; cost-per-ton; changes in price, production, and revenues in affected industries; impacts on small entities and the energy sector; and total benefits.

Evidence: RIAs are available at www.epa.gov/ttn/ecas. Some recent rules (e.g., Auto and Plywood MACT standards) did not quantify benefits.

Does the program systematically review its current regulations to ensure consistency among all regulations in accomplishing program goals?

Explanation: As required by the CAA, EPA will review each MACT standards as part of the residual risk program. EPA periodically reviews mobile source regulations for impacts on air toxics as well.

Evidence: CAA section 112(f) and 202(e).

Are the regulations designed to achieve program goals, to the extent practicable, by maximizing the net benefits of its regulatory activity?

Explanation: Under section 112 of the CAA, the Agency must prepare standards for HAPs that meet the requirements set forth by the Act regardless of net benefits. However, EPA may increase cost effectiveness by subcategorizing source categories to distinguish among classes, types, and sizes of affected emission units. In a number of its rulemakings, EPA has taken steps to reduce costs and increase flexibility, and some of the program's regulatory impact statements (RIAs) have demonstrated quantified net benefits to society. As the residual risk program develops, the Agency can use its authority to develop rules that target high-risk facilities. EPA also should utilize its delisting authority and health threshold provision.

Evidence: Evidence that EPA strived to minimize regulatory costs on affected industries: MACT standards for Plywood and Composite Wood Products and Industrial/Commercial/Institututional Boilers and Process Heaters. These rules included a risk-based provision to allow low risk facilities to be exempt

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: Toxicity weighting of emissions reductions has allowed the program to link emissions reductions to risk reduction, and thus to show some progress towards its goals. However, the program lacks ambitious targets for its long-term performance goals. Furthermore, EPA measures progress towards its emissions reduction goals against a questionable baseline that was developed largely upon extrapolated data. Performance information is currently drawn from the National Toxics Inventory (NTI), which is conducted every 3 years. This system of data collection makes it difficult for EPA to confirm that the full reductions have actually occurred. In the future, EPA plans to augment the toxicity weighted emissions using a more direct method for predicting risks by utilizing ambient air toxics data that the Agency has begun to collect. This information will provide a national baseline and sufficient information to analyze trends in ambient concentrations. A measurement implementation plan is under review.

Evidence: In 2005, toxicity-weighted tons (cancer) of emissions will be reduced by 22% from the 1993 baseline level of 4.16 tons. Toxicity-weighted tons (non-cancer) will be reduced by 55% from the 1993 baseline level of 4.85 tons. A Measure Implementation Plan (MIP) for ambient data is under review. The recent Inspector General report, 'EPA's Method for Calculating Air Toxics Emissions for Reporting Results Needs Improvement' discusses problems with the 1993 baseline and the NTI.

Does the program (including program partners) achieve its annual performance goals?

Explanation: Annual estimates of toxicity weighted emissions reductions allows the program to link reductions to risk reduction, and thus to show some annual progress towards its long-term goals. However, the program lacks ambitious targets.

Evidence: Progress on these measures can be found at: Air toxics emissions reductions - www.epa.gov/ttn/chief/net/index.html. Annual targets are in the MIP for Toxicity Weighted emissions and will be published in GPRA documents in 2006.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: EPA continues to improve its cost effectiveness by utilizing experience gained by previous efforts in the program. Analysis shows that the rulemaking process has became less expensive over time. Efficiencies such as combined source categories were used to streamline and speed the process. While undertaking residual risk analyses, EPA will be using a two-tiered approach to determine if standards are necessary before entering a full-blown rulemaking process. The program is developing an efficiency measure that will assess the costs of emissions reductions over time.

Evidence: The EPA Inspector General found in 1996 that the Agency was actively seeking and adopting initiatives to speed and improve the MACT standard process. Increased efficiency over the past decade is demonstrated by assessing the average cost of a typical MACT standard. The average total expenditure (contract$) for a standard promulgated in 1993 was $1.1M, compared to $700K for a standard promulgated in 2003.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: NA. There are no comparable programs to reduce air toxics on a national basis. There are a few State programs that aim at reducing air toxics of local interest, however most States have a very limited or no air toxics program.

Evidence: CAA Section 112

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: The NAS Study: Air Quality Management in the United States discussed progress on air toxics reductions within existing statutory authority. However, assessments indicate that the program has had difficulties demonstrating results. Most recently the OIG has issued an evaluation report 'EPA's Method for Calculating Air Toxic Emissions for Reporting Results Needs Improvement.'

Evidence: NAS study and IG reports.

Were programmatic goals (and benefits) achieved at the least incremental societal cost and did the program maximize net benefits?

Explanation: Progress has been made to minimize costs by allowing low-risk facilities emitting less HAPs to be exempt from a standard or have less costly options for control. Compliance options, emissions averaging, and pollution prevention options in MACT rules have reduced the overall costs of the air toxics program. However, the agency does not collect sufficient information to make a strong determination that it maximizes net benefits.

Evidence: In the Plywood and Composite Wood Products rule, EPA removed a low-risk subcategory, providing a process for individual facilities to demonstrate that they belong in the low-risk subcategory by performing a risk assessment. For Industrial Boilers, EPA included two site-specific compliance alternatives based on the 'health threshold provision' of the Act, each of which will reduce the compliance costs for sources whose noncancer risks are deemed to be below the known exposure threshold.