Detailed Information on the
National Ambient Air Quality Standards and Regional Haze Programs Assessment

Implement improvements within current statutory limitations that address deficiencies in design and implementation and identify and evaluate needed improvements that are beyond current statutory authority.

Improve the linkage between program funding and the associated contributions towards progress in achieving program goals.

Develop at least one efficiency measure that adequately reflects program efficiency.

Measure: Percent reduction in population-weighted ambient concentration of fine particulate matter (PM2.5) in all monitored counties from 2003 baseline.

Explanation:Measures improvements (reductions) in the ambient concentration of fine particulate matter (PM2.5) pollution across all monitored counties, weighted by the populations in those areas. To calculate this weighting, pollutant concentrations in monitored counties are multiplied by the associated county populations. Therefore the units for this measure are "million people-micrograms per meter cubed" (million people-ug/m3). The 2003 baseline is 2,581 million people-ug/m3. The goal of a 6% reduction in 2015 is based on the forecasted effect of successful implementation of emission reduction programs that had been adopted as of the time the goal was set, including the Clean Air Interstate Rule.

Measure: Cumulative percent reduction in population-weighted ambient concentration of fine particulate matter (PM2.5) in all monitored counties from 2003 baseline.

Explanation:Measures improvements (reductions) in the ambient concentration of fine particulate matter (PM2.5) pollution across all monitored counties, weighted by the populations in those areas. To calculate this weighting, pollutant concentrations in monitored counties are multiplied by the associated county populations. Therefore the units for this measure are "million people-micrograms per meter cubed" (million people-ug/m3). The 2003 baseline is 2,581 million people-ug/m3. Weighting by population makes this measure more closely track the aggregate public health improvement from PM2.5 reductions, independent of jurisdictional boundaries. The annual goals represent an expected phase-in of controls by the electric power industry to be able to meet the Clean Air Interstate Rule caps for NOx (in 2009) and SO2 (in 2010).

Measure: Percent reduction in population-weighted ambient concentration of ozone in all monitored counties from 2003 baseline.

Explanation:Measures improvement (reductions) in the ambient concentration of ozone pollution across all monitored counties, weighted by the populations in those areas. Weighting by population makes this measure more closely track the aggregate public health improvement from ozone reductions, independent of jurisdictional boundaries. To calculate this weighting, pollutant concentrations in monitored counties are multiplied by the associated county populations. The units for this measure are therefore "million people-parts per billion" (million people-ppb). The 2003 baseline is 15,972 million people-ppb. The goal of a 14% reduction in 2015 is based on the forecasted effect of successful implementation of emission reduction programs that had been adopted as of the time the goal was set, including the Clean Air Interstate Rule.

Measure: Cumulative percent reduction in population-weighted ambient concentration of ozone in all monitored counties from 2003 baseline.

Explanation:Measures improvement (reductions) in ambient ozone concentrations across all monitored counties, weighted by the populations in those areas. To calculate this weighting, pollutant concentrations in monitored counties are multiplied by the associated county populations. The units for this measure are therefore "million people-parts per billion" (million people-ppb). The 2003 baseline is 15,972 million people-ppb.

Measure: Percent improvement in visibility on 20% worst days, on average for all eastern Class I areas.

Explanation: Measures visibility improvement in certain eastern National Parks and other scenic areas, known as Class I areas, as a percentage improvement toward natural background conditions from average impairment on 20% worst days during 2000-2004 baseline period. The goal for 2018 represents the average of the area-specific, presumptively required milestones on the 60-year paths mandated by EPA's Regional Haze rule. Western and eastern Class I areas are aggregated separately from each other because of substantial differences in the rates-of-progress need to meet their respective 60-year goals. The goal is expressed as an average, rather than a requirement to be met in every area individually, because the EPA rule allows states to deviate from the presumptive path for individual areas based on reasonableness grounds.

Measure: Percent improvement in visibility on 20% worst days, on average for all western Class I areas.

Explanation: Measures visibility improvement in certain western National Parks and other scenic areas, known as Class I areas, as a percentage improvement toward natural background conditions from average impairment on 20% worst days during 2000-2004 baseline period. The goal for 2018 represents the average of the area-specific, presumptively required milestones on the 60-year paths mandated by EPA's Regional Haze rule. Western and eastern Class I areas are aggregated separately from each other because of substantial differences in the rates-of-progress need to meet their respective 60-year goals. The goal is expressed as an average, rather than a requirement to be met in every area individually, because the EPA rule allows states to deviate from the presumptive path for individual areas based on reasonableness grounds.

Measure: Cumulative percent reduction in the number of days to process State Implementation Plan revisions, weighted by complexity.

Explanation:Measures changes in the amount of time it takes to process State Implementation Plan revisions. The baseline is 420 days. The measure takes into account the level of complexity of the revisions by applying a weighting factor.

Is the program purpose clear?

Explanation: This PART covers two programs - the National Ambient Air Quality Standards (NAAQS) program and the Regional Haze program - because they address similar pollutants. The purpose of the NAAQS program is to protect human health and the environment from the potential effects of air pollution associated with six "criteria" air pollutants (carbon monoxide, nitrogen dioxide, lead, sulfur dioxide, particulate matter, and ozone). The program sets health-based standards for these pollutants that protect with human health and the environment and assists states in implementing these standards. According to the Clean Air Act, the purpose of NAAQS programs is "to protect and enhance the quality of the Nation's air resources so as to promote the public health and welfare and the productive capacity of its population". The purpose of the Regional Haze program is to improve visibility in certain areas of special national or regional natural, recreational, scenic, or historic value (called Class I areas). The program's objective is to restore these areas to "pristine" conditions over a 60-year period. The Regional Haze program establishes regulations that set air emission control standards and assists with implementation.

Evidence: The Clean Air Act outlines the purpose and objective of both the NAAQS and Regional Haze programs: Title I, Sec. 101 (findings and purpose); Title I, 108, 109 (specifically (b)(1) for "adequate margin of safety" requirement), 110, 302(h) (setting NAAQS); Title I, 171-193 (Plan Requirements for Non-attainment Areas); Title I, 169A, 169B (Visibility Protection for Federal Class I Areas).

Does the program address a specific and existing problem, interest, or need?

Explanation: In enacting the Clean Air Act (CAA) of 1970, Congress found that "the growth in the amount and complexity of air pollution brought about by urbanization, industrial development, and the increasing use of motor vehicles, has resulted in mounting dangers to the public health and welfare, including injury to agricultural crops and livestock, damage to and the deterioration of property, and hazards to air and ground transportation". To help address this problem, the CAA requires EPA to establish primary standards (called National Ambient Air Quality Standards or "NAAQS") for certain pollutants of concern in order to protect public health, including the health of sensitive populations, with an adequate margin of safety. EPA is also required to establish secondary standards to protect public welfare (e.g. materials, ecosystems, visibility) from any known or anticipated adverse effects. The CAA as amended in 1990 further required EPA to reduce the number of days of worst visibility and increase the number of days of best visibility. The NAAQS program establishes and revises standards to ensure continued protection of human health and the environment. Based on air quality trend information, though clear progress has been made in addressing four of the current criteria pollutants (CO, NO2, Pb, and SO2), air pollution due to ozone and fine particles (PM2.5) continues to be a problem in many areas of the country today.

Evidence: EPA issues air quality trends reports that review air quality improvements from NAAQS programs, which are available on the web. Current and multiple prior-year and special reports are available that show continuing air quality concerns. In addition, certain areas of the country have been determined (or "designated") as not meeting the ozone and particulate matter standards. These areas will now develop plans for coming into compliance (commonly referred to as "attainment"). Designation information is also available on EPA's website.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: The Clean Air Act clearly gives EPA the primary authority to: determine the levels of air pollution that are harmful to human health and the environment; set enforceable standards that protect human health with an adequate margin of safety; and evaluate available control strategies to reduce emissions. Similarly, EPA was given the primary authority to establish rules to help improve visibility in Class I areas around the country. The NAAQS and Regional Haze programs as laid out in the CAA have a number of explicit features to prevent duplication by more than one level of government or federal agency. For example, CAA Section 176 specifies that other federal agencies must ensure their projects conform to the clean air implementation plans. Similarly, Sections 231-233 carefully delineate responsibilities for regulating emissions from aircraft engines between EPA, FAA, DOT, and States. These requirements help prevent overlap of programs and duplication of effort. The CAA also minimizes the potential for duplication by assigning responsibilities to various levels of government. At the federal level, nationwide standards are set for criteria pollutants. EPA designates areas within states that are not in attainment and then the primary responsibility for reaching attainment with the standards and reducing regional haze lies with states.

Evidence: CAA clearly delineates responsibilities and specifies actions. Sections 108-109 discuss setting EPA's responsibility to set primary and secondary standards. Section 107 covers the designation of areas in non-attainment. Requirements for states with areas that are attaining/maintaining standards are in CAA Section 110 and Sections 171-193 discuss implementation plan requirements for non-attainment Areas. CAA Section 176 covers transportation plan conformity requirements, Section 202-219 covers Motor Vehicle Emissions Standards, restrictions on States' regulation of fuels can be found in section 211(c)(4)(A), and aircraft emissions are covered in Sections 231-233. Regional Haze is included in Sections 169A&B.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: While it is important to note the success of the existing program and the need to continue current efforts until such time that identified improvements can be implemented, there is evidence of flaws in the design of the NAAQS program and in some cases a different approach could be more effective at addressing the problem and purpose. The National Research Council recently completed a report on the overall air quality management system in U.S. that included a review of the structure and actions of the NAAQS program under the current Clean Air Act requirements. The report states that clear progress has been made on improving air quality and highlights several successful elements of the NAAQS program but it also concludes that changes are needed to ensure continued progress. Relevant findings range from the need for alternative forms of NAAQS standards to protect ecosystems because the current structure and implementation is inadequate, to the need for a broader multi-pollutant approach for because the current structure causes inefficiencies and may limit the identification and consideration of important information on risks. EPA has already undertaken efforts to address recommendations in the report that are implementable within the current statutory limitations.

Evidence: "Air Quality Management in the United States", National Research Council, Committee on Air Quality Management, The National Academy Press, Washington, D.C., 2004

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: The NAAQS program uses its resources to establish, revise, and help implement nationwide health-based standards for criteria pollutants, which directly address the purpose of protecting the nation's human health and ecosystems from air pollution. The program is designed to establish and review air quality standards for a certain set of pollutants, identify which areas of the country fail to meet the standards, and then assists those areas in developing control programs to bring them into attainment. Evidence that the outcomes of the program are reaching the intended beneficiaries includes the progress made in bringing areas into attainment for carbon monoxide and lead and the air quality improvements that are associated with that progress. Another example of effective targeting would be the recently issued Clean Air Interstate Rule, which addresses pollution transported from one State to another. EPA focused on identifying which downwind areas were negatively impacted by transport, determining which upwind States were contributing to these air quality problems, and establishing appropriate caps on emissions of the key pollutants to address the problem. The regional haze program is specifically designed to target a special subset of areas that need particular attention for visibility concerns. The program tracks air quality and visibility impairment in the nation's 156 Class I areas and requires States to design control strategies specifically for those areas. The NAAQS and Regional Haze programs covered in this PART are not responsible for providing funding to the beneficiaries. NAAQS-related grant programs and certain programs in the Department of Transportation are responsible for this.

Evidence: EPA issues a "Green Book" that lists and tracks areas in non-attainment areas and maps the Class 1 areas protected by Regional Haze program (all available on EPA website). On March 10, 2005, EPA released the Clean Air Interstate Rule (CAIR) designed to significantly reduce sulfur dioxide (SO2) and nitrogen oxides (NOx) emissions using a cap and trade program (40 CFR Parts 51, 72, 73, 74, 77, 78, and 96). On June 15, 2005, EPA issued final amendments to the provisions of the 1999 regional haze rule that require emission controls known as Best Available Retrofit Technology, or BART, for industrial facilities emitting air pollutants that reduce visibility. The amendments help to ensure proper targeting because they include final guidelines, known as BART guidelines, for states to use in determining which facilities must install controls and the type of controls the facilities must use.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: Two new long-term measures have been established for the NAAQS Federal program and two long-term measures have been accepted for the Regional Haze program. All accepted measures are listed in the Measures Section of this PART. The NAAQS measures track reductions in concentrations of two pollutants that are of particular concern - ozone and particulate matter that is less than 2.5 microns in size (PM2.5 or "fine particulate matter"). The Regional Haze measures track visibility improvements in Class 1 areas across the country. Because the Regional Haze program has a 60-year implementation period, visibility improvements directly attributable to the Regional Haze program activities and rules are not expected until after 2010. Visibility improvements are expected in Class 1 areas in the interim due to actions taken under other programs and rules.

Evidence: See Measures Section

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The program has establish baselines and long-term targets for the two new NAAQS measures. The targets are ambitious and based on analysis of expected reductions. Because the measures are new, the targets may need to be revised once initial actual results are available. Both of the Regional Haze program's measures also have adequate baselines and targets. Based on stringent new controls on major sources plus additional controls that will come into play under the Regional Haze program, the Regional Haze program has set an ambitious goal of reducing regional haze in the East by 15% by 2018 from a 2000-2005 baseline. Lower pollution levels in the West, combined with fewer available control measures, mean that visibility improvements in that region will be more expensive and more difficult. The program has established a substantial 5% visibility improvement goal for the western U.S. by 2018; this is 2% higher than current models forecast on the basis of existing controls.

Evidence: The program provided implementation plans for these measures which describe how the targets were established and how data will be analysed to determine progress.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: Two new annual measures have been established for the NAAQS Federal program. All accepted measures are listed in the Measures Section of this PART. The NAAQS measures track reductions in concentrations of two pollutants that are of particular concern - ozone and fine particulate matter (PM2.5 - particulate matter that is less than 2.5 microns in size). These two measures tie directly to the two new concentration reduction long-term measures approved for the NAAQS program. Because visibility improvements directly attributable to the Regional Haze program activities are not expected until after 2010, no annual measures are required for the program at this point. Visibility improvements in Class 1 areas are expected in the interim due to actions taken under other programs and rules.

Evidence: See Measures Section

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The program has establish baselines and long-term targets for the two new NAAQS measures. The targets are ambitious and based on analysis of expected reductions. Because the measures are new, the targets may need to be revised once initial actual results are available. It is important to note that the data necessary for assessing the concentration-based performance measures will not be available until July of the summer following the milestone date of the measure.

Evidence: The program provided implementation plans for these measures which describe how the targets were established and how data will be analysed to determine progress.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: Program partners include contractors, states, the NAAQS Research program within EPA's Office of Research and Development (ORD), EPA's Air Quality Grants program, and EPA's Title V and New Source Review permitting programs. ORD is a key partner in the standard-setting process. It is responsible for developing Air Quality Criteria Documents, which cover the state of the science for NAAQS pollutants and are required for the review of the standards. To ensure ORD's research program efforts are in line with the needs and goals of the NAAQS program, a Research Coordination Team (RCT) has been established, which among other things contributes to the development of ORD's Multi-Year Plans, which contain specific goals and measures for key research areas. The air grants and permitting programs as well as the states are important partners in implementing the standards set by the NAAQS program. State Governors are responsible for helping to identify and designate areas that not in attainment of the standard. The grants program helps defray states' air quality program costs and the permitting programs are critical for implementing controls and ensuring compliance. These programs have related performance measures which demonstrate their commitment to the overall goals and purpose. Contractor work assignments are developed which project deliverables and required reporting procedures and milestones that are necessary for the successful accomplishment of program goals.

Evidence: Office of Research and Development Multi-Year Plans are available on the EPA website. Contractor agreements and performance reports were provided as evidence. Sections 107(d)(1)(A) and 110 of the Clean Air Act address the role of the Governors in the implementation of the NAAQS. Copies of letters from Governors on designations were also provided. The air grants, permitting, and research programs have been reviewed in separate PARTs.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: On a relatively infrequent basis, independent bodies have performed quality reviews of certain aspects of these programs. No evaluations or set of evaluations meet the scope, quality, independence, and frequency requirements of the PART. Umbrella studies have been useful in providing guidance for strategic planning purposes but they have either not met the quality criterion or have been too broad in scope to provide useful feedback on how the federal NAAQS program and its components are effective in meeting set goals. There are data available that demonstrate air pollution due to the criteria pollutants, which the NAAQS program is designed to address, has been declining.

Evidence: Evaluations that do not meet the criteria but provide useful information include: "Air Quality Management in the United States", National Research Council, Committee on Air Quality Management, The National Academy Press, Washington, D.C., 2004. "Federal Planning Requirements for Transportation and Air Quality Protection Could Potentially Be More Efficient and Better Linked", GAO Report No. GAO-03-581. "The Effectiveness and Efficiency of EPA's Air Program", 1998, EPA IG Report No. E1KAE4-05-0246-8100057. "To Breathe Clean Air", National Commission on Air Quality, U.S. Government Printing Office, 1981.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: Congressional Justifications and budget submissions to OMB incorporate budget and performance data in a disconnected manner, making it impossible to see how changes funding for the various NAAQS programs and activities impact progress toward established targets. Budget documents used at the program level also do not transparently link performance to funding. The NAAQS program includes activities in multiple divisions and branches at EPA. The divisions and branches use several systems to track budget and performance information, but there is no place where program or activity funding is tied to specific performance targets. Division funding is tracked in terms of budget object class, not programmatic activity. Object class information is linked to overarching categories and the categories are tracked separately (such as in independently developed spreadsheets). One division has created a system that links some financial data with specific activities and goals. Also, it is unclear whether or how direct and indirect costs needed to attain performance results are accounted for, apportioned, and incorporated in budgets.

Evidence:

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: Strategic planning for the NAAQS program is done as part of the planning process for the Office of Air Quality Planning and Standards (OAQPS). Strategic planning is facilitated via regular meetings of OAQPS Division Directors and Assistant Division Directors, as well as through office-wide strategic seminars. A Senior Management Team conducts regular meetings and off-site retreats that include long-term planning. The team consists of the Office Director, the Deputy Director, Division Directors, and key staff. The process considers program progress as well as external information about air quality concerns, such as the recommendations in the 2004 National Research Council (NRC) report entitled "Air Quality Management in the United States". In 2001, OAQPS began a long-term planning process to adapt air quality management to address future challenges. As part of this process, OAQPS redirected resources to adopt problem-based approaches to air quality problems. Results of this process include the expansion of the Air Quality Index to include particulate matter and improvements to the program's IT data warehouse. In addition, OAQPS has created seven internal teams to oversee implementation of key planning initiatives. The teams focus on issues such as information access, tracking and accountability, innovations, multi-pollutant approaches, and air quality management. Feedback from the teams informs senior management about the steps necessary to improve the NAAQS program. Each team is responsible for reporting key accomplishments and next steps on a monthly basis.

Evidence: EPA Strategic Plan 2003-2008 and associated Regional office Strategic Plans show progress and changes in direction associated with these planning efforts. Other evidence provided included briefing materials for Division Directors on future directions of the program and a matrix outlining the status of the program's responses to 38 recommendations from NRC's Air Quality Management Report and the follow-up Clean Air Act Advisory Committee process.

Are all regulations issued by the program/agency necessary to meet the stated goals of the program, and do all regulations clearly indicate how the rules contribute to achievement of the goals?

Explanation: The Clean Air Act (CAA) requires EPA to set NAAQS to control common air pollutants and to periodically revise those standards as appropriate in light of new scientific information, modeling studies, and monitoring data. EPA periodically reviews available data to determine if new standards are warranted and/or if new regulations providing additional control measures are needed to attain the standards. In promulgating air quality standards, EPA clearly outlines the expected health and environmental benefits. All rules outlining control strategies explain the anticipated impact on pollution levels and public health, the program's main concerns. Each rule undergoes extensive public comment and interagency review, as well as the types of analysis, including cost-benefit analysis, required under EO12866. The CAA also requires EPA to protect visibility in Class I areas by developing rules to reduce air pollution sufficiently to bring haze levels back to background conditions and prevent any future impairment. The Regional Haze rule and its BART requirements fulfill EPA's commitment to address regional haze under the CAA Sections 169A & B. The Regional Haze rule requires that each State develop a federally enforceable SIP to address the BART requirements for certain older major stationary sources and to reduce emissions from major sources as necessary to make reasonable progress toward visibility goals. The Regional Haze rule clearly ties control strategy options and approaches to expected visibility improvements.

Evidence: Regulatory Impact Analysis (RIA) for the Final Clean Air Implementation Rule (CAIR), EPA-452/R-05-002, outlines the rule's potential health benefits (pgs 1-1 to 1-5) and emission reductions (pgs 2-3 to 2-4). CAIR Modeling of Control Costs, Emissions, and Control Retrofits for Cost Effectiveness and Feasibility Analyses also support RIA information. Chapter 2 of the Regional Haze RIA (final Regional Haze Regulation for Protection of Visibility in National Parks and Wilderness Areas) describes the need for the regulations. Other necessary rules include those designating non-attainment areas and pollutant standards (the 1997 particulate matter and ozone rules for example).

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: The program, working with state and federal agency program partners, oversees the development and operation of key monitoring efforts around the country which provide essential air quality data. The networks include the National Air Monitoring Stations and State/Local Air Monitoring Stations (NAMS/SLAMS), the Photochemical Assessment Monitoring Stations (PAMS), and the Interagency Monitoring of Protected Visual Environments (IMPROVE) network, among others. The monitoring stations in the SLAMS network are primarily managed by the states and states provide the NAAQS program with annual summaries of monitoring results and more detailed results upon request. Data from the IMPROVE monitors provides necessary data for implementing the regional haze element of the program. The program uses the data collected through these networks to assess progress toward air quality goals, show trends in air quality, accomplish key program tasks (such as designating the attainment levels of areas across the country), and adjustments in program activities to aid in meeting performance goals. For example, the air quality data showed that additional local air pollution control measures were having limited impact on concentrations of pollutants such as ozone and particulate matter because they were being transported into areas from outside area boundaries. In response the program begin to shift its priorities from local to regional approaches. Once the role of pollutant transport was fully recognized, the program designed a series of rules s to address regional pollution.

Evidence: Data from federal and State sources is maintained in a centralized Air Quality System (AQS) database, which improves the usefulness and accessibility of the information. The data from the networks inform management decisions including non-attainment area designations, standard-setting, permitting, and rulemakings such as the Nitrogen Oxide (NOx) State Implementation Plan (SIP) Call and Clean Air Interstate Rule, both of which address pollutant transport issues. The data was also used to develop and now to maintain the interactive AIRnow.gov website, which provides daily air quality information for the nation.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Performance agreements are used for Federal managers at headquarters and the regional offices, which include requirements for achieving key program results and accountability for NAAQS program objectives. Managers responsible for key elements of the NAAQS program have been identified from the branch chief through office director level and are responsible for monitoring progress toward achieving goals and objectives and taking actions to modify operations when corrective action is needed. The program works closely with EPA's Office of Research and Development (ORD) to ensure that research activities supporting the NAAQS are complete and timely. Cross-office planning bodies called Research Coordination Teams (RCTs) composed of representatives from the ORD labs, program offices, and regional offices meet weekly to discuss performance and facilitate agreements about priorities and schedules. In coordination with the RCTs, ORD develops Multi-Year Plans (MYPs) containing specific goals and measures for each key NAAQS research area and ORD personnel brief OAR managers on performance and related issues. OAR also participates in the allocation of the fiscal year ORD budget to help determine which projects should receive highest funding priority. To hold contractors accountable, statements of work are written outlining support needed. Individual project level work assignments are developed with independent cost estimates, project deliverables, and required reporting procedures and milestones. Project level work assignment managers at EPA must certify that acceptable progress is being made within the project scope and allocated resources, that progress reports are being provided, and that the contractor is on schedule before payments are made.

Evidence: ORD Multi-Year Plans available on the EPA website. Examples of NAAQS review planning documents that outline the scheduling of ORD and NAAQS program activities were provided as evidence. Performance Agreement, Appraisal, and Certification for Senior Executives forms for federal managers (headquarters and regional) were also provided. Multiple completed documents were provided to show chain of contractor accountability, such as the Performance Report, Statement of Work, project-level work assignment, and invoice approval checklist. In addition, contractors' performance is tracked through the NIH Contractor Performance System, an annual performance appraisal system used by multiple agencies in which contractors are rated on a scale from "unsatisfactory" to "outstanding," depending on the quality of the product/service provided; cost; timeliness of performance; and the quality of their business practices.

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: Budgeting and expenditures are managed by branches and divisions, not programs, and programs often cut across branches and/or divisions. Divisions and branches use contract and procurement plans which outline what the funds are to be committed for and provide information on balances. This information is tied to the agency-wide Integrated Financial Management System (IFMS) and the Office of Air and Radiation's Management and Accounting Reporting System (MARS), which track expenditure data at the object class level. IFMS is the financial management tool used by the agency to commit, obligate and expend resources. Staff in each division review and analyze resource reports from IFMS and MARS to ensure that commitments and obligations are valid and on track for expenditure. Summary reports are then prepared for senior management on the utilization of funds. Senior-level program reviews occur at mid-year and end-of-year to discuss program and resource progress and/or needed changes. To ensure contractor funds are obligated and expended for their intended purpose, individual project level work assignments are developed, with independent cost estimates, project deliverables and required reporting procedures and milestones. Before an invoice is paid, the project level work assignment manager must certify acceptable progress is being made within the resources allocated and within the scope of the assignment, progress reports are being provided and the contractor is on target to complete the assignment.

Evidence: Reports from the Integrated Financial Management Systems (IFMS)/Management Accounting Reporting System (MARS) were made available that demonstrate obligation and expenditure tracking at the aggregate level. Multiple completed documents were provided to show chain of contractor accountability, such as the Performance Report, Statement of Work, project-level work assignment, and invoice approval checklist.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: No efficiency measures have been approved for the NAAQS program. One method the program uses to regularly seek out and implement efficiency improvements is through an established Innovations Team. This team identifies and promotes improvements in program design, management approaches, and operational efficiency. Recent examples include an improved emissions inventory quality assurance tool, new web-based tools for air quality analysis (AirExplorer), an automated benefits modeling and analysis tool (BenMAP), and Linux clusters for air quality modeling. BenMAP was developed in response to the need for a fast, flexible, defensible, and transparent tool for estimating the health and environmental benefits of changes in air pollution. Prior to its development, typical benefits analyses in support of rulemakings or legislative proposals took at least three weeks to conduct at a cost of $50,000 to $100,000. BenMAP was designed to allow EPA staff (and others) to produce reproducible analyses within hours to days, and to provide information on the parameters and assumptions used in each analysis to allow for transparent communication with users of benefits analyses.

Evidence: The Benefits Mapping and Analysis Tool, BenMAP, and the Air Explorer collection of visualization tools for air quality analysts are both available on the EPA website. Over the past 3 years, BenMAP has been used in analyzing numerous policies, including the Nonroad Diesel Rule, the Clean Air Interstate Rule, the Clean Air Mercury Rule, and the Clean Air Visibility Rule.

Does the program collaborate and coordinate effectively with related programs?

Explanation: Related programs include key partners, such as EPA's Office of Research and Development, state, local, Tribal, and regional air quality organizations, and other federal agencies. One example of an effective collaborative effort is the IMPROVE (Interagency Monitoring of Protected Visual Environments) monitoring network. EPA partners with DOI through an Interagency Agreement that establishes joint responsibilities for the operation of this network of visibility monitors in Class I areas. Other examples include joint collaborative research with the National Oceanic and Atmospheric Administration (NOAA), joint efforts to ensure proper implementation of certain transportation regulations, and the establishment of a joint public education initiative on the air pollution contributions from mobile sources (such as automobiles).

Evidence: Information about the IMPROVE network collaborative effort is available on the internet at EPA's website as well as others. Memorandums of Understanding with the Department of Transportation "to ensure the proper implementation of the transportation conformity rule's provisions through better EPA and DOT consultation" and with NOAA on "Cooperation in Research to Combat the Effects of Airborne Environmental Pollution" for improved forecasting methods to support the AIRNow.gov website. The "It All Adds Up to Cleaner Air" Program (website) is the product of an EPA-DOT partnership (with participation from other agencies), which is designed to help regional, state and community reduce traffic congestion and air pollution.

Does the program use strong financial management practices?

Explanation: The NAAQS program is free of material weaknesses and also meets several other criteria for this question. The program follows EPA's financial management guidelines for committing, obligating, reprogramming, and reconciling appropriated funds. Program officials have a system of controls and accountability based on GAO, Treasury and OMB guidance as well as generally accepted accounting practices (GAAP), to ensure that improper payments are not made. Program staff routinely review reports from the Agency's Management and Accounting Reporting System (MARS) to manage and track the expenditure of resources, and to provide status reports to management. Mid-year reviews with senior managers involve detailed discussions on overall management of the NAAQS program, as well as resource utilization and needs. For contracts, the program develops project level work assignments and project officer track milestones achieved, ensure contract deliverables are received, and resource expenditure is on target with estimated project cost before invoices are paid. EPA trains individuals to ensure that they understand their roles and responsibilities for invoice review and for carrying out the financial aspects of program objectives.

Evidence: Documentation of Funds Control Officer job qualifications and training requirements was provided. Evidence of financial tracking provided through reports from the Management and Accounting Reporting System (MARS). Financial performance and improvements outlined in annual assurance letters from office to agency head as required by Federal Managers Financial Integrity Act (FMFIA) and in agency Inspector General reports including November 14, 2005 "Audit of EPA's Fiscal 2005 and 2004 Consolidated Financial Statements", audit report 2006-1-00015, and November 15, 2004 "Audit of EPA's Fiscal 2004 and 2003 Financial Statements", audit report 2005-1-00021.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: Senior managers responsible for implementing the NAAQS program elements routinely hold management retreats where they discuss a variety of management issues that include future program directions, management and program deficiencies, human capital and succession planning, incentive programs, administrative support, etc. An example of action resulting from these retreats is the development of an office-wide Rotation Program for staff, which provides developmental opportunities and helps prepare staff to move into positions vacated by retirements/resignations. In addition to the Senior Management Retreats, the Group Leaders conduct bi-monthly meetings where they focus each meeting on a variety of current management issues. Accountability for implementing major changes in program management and structure is managed through these regular meetings as well as through FMFIA Annual Assurance Letters and internal evaluations of senior staff. The program routinely utilizes Assurance Letters to identify management deficiencies and design goals and strategies for improving program management. Each Division Director provides input to the Annual Assurance Letter outlining divisional goals, progress, problems, and potential solutions. Another example of addressing a management deficiency was the development of the Emissions Monitoring and Analysis Division's (EMAD) web-based financial tracking system, which includes program/project codes, approved project allocation, contractor support, and date of commitment, as well as responsible group lead. This automated system allows EMAD staff and management access to updated financial and project information on a real time basis.

Evidence: Minutes from Group Leader Meetings and agendas from Senior Management Retreats were made available. An announcement for and paper describing the office Rotation Program as well as a memo on feedback and revisions to the Rotation Program were provided. Evidence also included the office's FY2003 Integrity Act Annual Assurance Letter and reports from the Emissions Monitoring and Analysis Division (EMAD) web-based financial management tracking tool.

Did the program seek and take into account the views of all affected parties (e.g., consumers; large and small businesses; State, local and tribal governments; beneficiaries; and the general public) when developing significant regulations?

Explanation: EPA follows all appropriate procedures and guidelines for notice-and-comment rulemakings as required by the Administrative Procedure Act. In addition, EPA follows other consultative requirements such as those of the Small Business Regulatory Enforcement Fairness Act (SBREFA), which requires public panels to consider impacts of regulations on small businesses, and the Federal Advisory Committee Act (FACA), which established the Clean Air Act Advisory Committee (CAAAC) in 1990. The CAAAC is a 60 member advisory group that meets 4 times each year; it consists of 60 senior managers and experts representing State/local/Tribal governments, environmental and public interest groups, academic institutions, unions, trade associations, utilities, industry, and other experts. Additionally, EPA follows the requirements of EO 13175 to solicit meaningful and timely input from Tribes in the development of regulatory policies that have Tribal implications. State and local governments are consulted via regional offices and representative organizations. Together, these methods ensure that EPA receives significant input from stakeholders in a variety of public settings.

Evidence: Hearing schedules, notices, and notes, overviews of public comments and complete response to comment documents for rules were provided as evidence and are available on the EPA website. Rules include information on compliance with Regulatory Flexibility Act, Unfunded Mandates Reform Act, SBREFA, etc.. Examples provide include the regulatory impact analysis and preamble pp. 721-777 for the Clean Air Interstate Rule (2005), the Particulate Matter 2.5 Designations rule preamble pp. 27-38, the 8-hour Ozone NAAQS Final Rule (1997) pp. 29-35 and the Regional Haze regulatory impact analysis: Final Regional Haze Regulation for Protection of Visibility in National Parks and Wilderness Areas. Executive Order 13175 (Consultation and Coordination with Indian Tribal Governments), 65 FR 67249, November 9, 2000, outlines tribal consultation requirements EPA. 2004.

Did the program prepare adequate regulatory impact analyses if required by Executive Order 12866, regulatory flexibility analyses if required by the Regulatory Flexibility Act and SBREFA, and cost-benefit analyses if required under the Unfunded Mandates Reform Act; and did those analyses comply with OMB guidelines?

Explanation: EPA routinely prepares regulatory impact analyses, regulatory flexibility analyses and cost-benefit analyses that comply with OMB Guidelines, including the newly revised OMB Circular A-4. Extensive Interagency review and approval coordinated by OMB/OIRA is completed for all EPA analyses that are required. EPA works closely with OIRA in developing new and state-of-the-art approaches to the characterization of the costs and benefits of EPA major regulations, especially in the characterization of uncertainty.

Evidence: Analyses for rules issued by the program include: Regulatory Impact Analysis for the Final Clean Air Interstate Rule (CAIR), EPA-452/R-05-002, March 2005; Interagency Review (Appendix G pp. 369-420), and Uncertainty Analysis (Appendix B pp. 246-289) of the CAIR technical documents; 8-hour Ozone NAAQS Final Rule Regulatory and Environmental Impact Analyses, pp. 29-35, 1997; PM2.5 NAAQS Final Rule Regulatory and Environmental Impact Analyses, pp. 51-58, 1997; and Regional Haze Regulatory Impact Analysis.

Does the program systematically review its current regulations to ensure consistency among all regulations in accomplishing program goals?

Explanation: As required by the Clean Air Act, the program periodically reviews each NAAQS through an extensive consultative process. When existing regulations or programs are replaced or rendered unnecessary, they are revoked or eliminated. As new information about human health, air quality, and environmental goals becomes available, regulations and program expenditures are adjusted as appropriate. Furthermore, the evaluative procedures regularly employed to monitor air quality, model expected trends, and design new control approaches systematically account for the impact and usefulness of existing rules. Part of the analysis conducted on each new rulemaking includes a review of the interaction of existing rules with proposed rules and the joint contribution of the entire suite of regulations to clean air outcomes, which is a particularly challenging aspect of the analysis for these programs given the overlap in pollutants and air emission sources.

Evidence: The Clean Air Interstate Rule (CAIR) preamble, Section IX, describes interactions with other Clean Air Act Requirements (pp. 638-721). CAIR revised the 40 CFR Part 51 emission reporting requirements to make them consistent with the data needs of Rule's new control requirements - a requirement for inventory submissions in 2007 was deleted because it was no longer necessary. All CAIR information is available on EPA's website. The program included revocation of the 1-hr ozone NAAQS in phase 1 of the implementation of the new 8-hr standard (Final Rule to Implement the 8-Hour Ozone National Ambient Air Quality Standard - Phase 1," pp. 20, April 2004). The program has also proposed an update to certain Air Quality Reference Models (April 21, 2000; 65 FR 215406), which adds a new dispersion model, AERMOD, to the Guideline on Air Quality Models, EPA's official listing of EPA-approved models for regulatory applications. Information on this is available at EPA's Support Center for Regulatory Atmospheric Modeling (SCRAM) website.

Are the regulations designed to achieve program goals, to the extent practicable, by maximizing the net benefits of its regulatory activity?

Explanation: The program's regulations are designed to balance several considerations. For regulations considered "significant" under Executive Order 12866, the program does a complete benefit/cost analysis. According to these analyses, many of the program's regulations issued over the past decade have benefits exceeding costs. For example, for the Clean Air Interstate Rule signed on March 10, 2005, projected monetized benefits exceeded projected monetized costs by a ratio of 25 to 1 in 2015. In addition, there can be unquantified benefits, and to a lesser extent, unquantified costs as part of some regulations. There are uncertainties associated with any of these cost benefit measurements, which program's analyses are beginning to characterize. All major rulemakings take these factors into account to ensure that, to the extent feasible, social benefits are maximized while social costs are minimized. CAIR, for example, employs a cap-and-trade emissions trading program to promote cost-effectiveness while making air quality improvements. Lastly, the program has pioneered new methods of computing program benefits, developing a rapid-response Benefits Mapping and Analysis program (BenMAP) benefits model to predict program impacts.

Evidence: Regulatory Impact Analysis (RIA) for the Final Clean Air Interstate Rule (EPA-452/R-05-002) Executive Summary, pp. 1-1 to 1-13, provides a summary of cost benefit analysis. For unquantified benefits, see Table 1-4, pp. 1-10 to 1-11. CAIR preamble, section X.3, pp. 735-744, discusses how benefits compare to the rule's costs. Regional Haze RIA, Chapter 10, includes cost-benefit comparisons. The program's BenMAP benefits model is publicly available on EPA's website.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: Two new long-term measures have been approved for the NAAQS Federal program that measure reductions in the concentration of two pollutants of particular concern - ozone and fine particulate matter (PM2.5). Annual measures that directly reflect progress on these goals have also been approved. Because data is only available for one year, it is not possible to determine if adequate progress is being made toward these long-term goals at this point. Two long-term measures have also been approved for the Regional Haze program that measure improvements in visibility in Class I areas across the country. No progress on the visibility measures that is directly attributable to Regional Haze program action will be seen until after 2010.

Evidence:

Does the program (including program partners) achieve its annual performance goals?

Explanation: Two new annual measures have been approved for the NAAQS Federal program, which tie directly to the two long-term measures that track reductions in the concentration ozone and fine particulate matter (PM2.5). The program has established a 2003 baseline and annual targets through 2008 for one of these two measures. Data has been provided data that demonstrates the program has made some progress in 2004 on these measures. Data is not available for 2005 yet.

Evidence: See Measures section for actual performance results.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The program has no approved efficiency measures to demonstrate year-to-year improvements in efficiency. Over time, discrete steps have been taken by the program to improve efficiency, such as the development and implementation of the Benefits Mapping and Analysis Program (BenMAP). The system was developed in response to the need for a fast, flexible, defensible, and transparent tool for estimating the health and environmental benefits of changes in air pollution. Prior to its development, typical benefits analyses in support of rulemakings or legislative proposals took at least three weeks to conduct at a cost of $50,000 to $100,000. With BenMAP, these analyses now can take only a few days and are performed at a lower cost.

Evidence:

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: There have been no formal comparisons of the NAAQS program to other standard-setting programs or other programs designed to reduce mortality and morbidity impacts. But, some reviews have shown that the benefits of some NAAQS program regulations have compared favorably. EPA, in partnership with OMB's Office of Information and Regulatory Affairs, is exploring methods to compare EPA's air program benefits with other government programs for reducing mortality and morbidity impacts. On March 15, 2005, EPA published its first regulatory analysis which included in an appendix an alternative approach for presenting the benefits of reducing air pollution. This method, called the Quality Adjusted Life Year (QALY) approach, may provide the basis for more directly comparing EPA program benefits with the benefits of other programs. While this analysis approach is still experimental, it does provide evidence that some EPA programs designed to reduce criteria air pollutants could compare very favorably with other mortality/morbidity reduction programs. Other programs and agencies currently working with the QALY approach include EPA's Office of Water, EPA's Office of Transportation and Air Quality (OTAQ), the FDA, and the DOT/National Highway Transportation Safety Administration.

Evidence: FY 2003 President's Budget, Analytical Perspectives chapter 24,"Ranking Regulatory Investments in Public Health" discusses the cost-effectiveness of regulatory action and the potential usefulness of the QALY method. EPA's experimental work with the QALY approach can be found in Appendix G of the Regulatory Impact Analysis for the Final Clean Air Interstate Rule, EPA-452/R-05-002, March 2005. Also see the 2004 Progress report on the United States??Canada Air Quality Agreement for comparison information.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: Because only evaluations that meet the criteria outlined in the guidance for question 2.6 can be considered for this question and there are no available evaluations (or set of evaluations) that meet the criteria, there is nothing available as the basis for credit in this question. One recent National Research Council report titled "Air Quality Management in the United States" has provided useful information on the general effectiveness of various aspects of the national ambient air quality standards programs but does not meet the strict quality critiera of the PART.

Evidence: "Air Quality Management in the United States", National Research Council, Committee on Air Quality Management, The National Academy Press, Washington, D.C., 2004

Were programmatic goals (and benefits) achieved at the least incremental societal cost and did the program maximize net benefits?

Explanation: The program maximizes net benefits as far as possible in its rulemakings. The NAAQS are set, according to Congressional mandate, without regard to cost considerations. In implementing the NAAQS, however, EPA must consider impacts on small businesses, vulnerable populations (esp. children), energy supply and demand, and other factors. Air pollution rules systematically account for these impacts, and EPA adjusts rules to provide the desired level of social benefits. New air pollution control programs are designed to achieve desired benefits at least incremental societal cost. The shift toward more market-based mechanisms and innovative approaches to air pollution control, such as are embodied in the CAIR rule, help ensure that societal costs are minimized. The recently issued CAIR rule is anticipated to provide over $100 billion in health and environmental benefits in 2015, over 25 times the cost of compliance. Costs will also grow as caps get tighter. Data from OMB's 2005 draft report to Congress on regulations indicate that the benefits of EPA's NAAQS regulations accounted for more than half of the total monetized benefits from all federal regulations from 1994-2004 and that EPA's regulations' benefits exceeded costs. This data is generally not based on actual, measured costs and benefits that resulted after the regulations were implemented but largely comes from the analyses done to support the issuance of the regulations.

Evidence: The 2005 OMB, Office of Information and Regulatory Affairs draft Report to Congress on the Costs and Benefits of Federal Regulations is available on the OMB website. Table 1-3, p. 9 provides comparison information. The 2004 OMB, Office of Information and Regulatory Affairs report to Congress on the Costs and Benefits of Federal Regulations and Unfunded Mandates on State, Local, and Tribal Entities, p. 9 also contains information. Regulatory Impact Analyses (RIAs) for all NAAQS program regulations provide data on costs and benefits. Most recent example in the RIA the Final Clean Air Interstate Rule, EPA-452/R-05-002, March 2005.