ExpectMore.gov


Detailed Information on the
Mobile Source Air Pollution Standards and Certification Assessment

Program Code 10002284
Program Title Mobile Source Air Pollution Standards and Certification
Department Name Environmental Protection Agy
Agency/Bureau Name Environmental Protection Agency
Program Type(s) Regulatory-based Program
Assessment Year 2004
Assessment Rating Moderately Effective
Assessment Section Scores
Section Score
Program Purpose & Design 100%
Strategic Planning 67%
Program Management 91%
Program Results/Accountability 73%
Program Funding Level
(in millions)
FY2007 $70
FY2008 $70
FY2009 $73

Ongoing Program Improvement Plans

Year Began Improvement Plan Status Comments
2005

Systematically review existing regulations to maintain consistency and ensure that regulations maximize net benefits. Conduct thorough ex ante economic analyses and evaluations of alternatives in support of regulatory development.

Action taken, but not completed The Program's Locomotive and Marine Diesel Engine Emissions Standards, signed March 14 2008 (published May 6 2008), were comprehensive standards to reduce pollution from new and existing locomotives and marine diesel engines, beginning as early as 2008. Analyses supporting this rule include benefit-cost and evaluations of alternatives.
2005

By the end of March 2008, brief OMB on progress developing two new efficiency measures -- one long and one short-term -- to enable the program to measure further efficiency improvements.

Action taken, but not completed The Program developed annual targets for this efficiency measure.

Completed Program Improvement Plans

Year Began Improvement Plan Status Comments
2005

Request $66 million for EPA??s mobile source programs, $1.5 million more than the 2005 President??s Budget request.

Completed The FY 2005 President's Budget Request included $66.6 million for EPA's mobile source programs.

Program Performance Measures

Term Type  
Long-term Outcome

Measure: Millions of tons of volatile organic compounds (VOCs) reduced since 2000 from mobile sources.


Explanation:The target number was generated in 2002 based upon EPA models which use pollutant emission factors and generate air quality inventories for on-highway vehicles, including passenger cars and trucks, and nonroad engines, such as construction/agricultural equipment. Data for the "actual" column is derived by applying the EPA models with revised estimates for variables such as annual vehicle miles traveled (VMT) for different vehicle categories.

Year Target Actual
2000 Baseline 7.7
2012 2.1
Long-term Outcome

Measure: Millions of tons of nitrogen oxides (NOx) reduced since 2000 from mobile sources


Explanation:The target number was generated in 2002 based upon EPA models which use pollutant emission factors and generate air quality inventories for on-highway vehicles, including passenger cars and trucks, and nonroad engines, such as construction/agricultural equipment. Data for the "actual" column is derived by applying the EPA models with revised estimates for variables such as annual vehicle miles traveled (VMT) for different vehicle categories.

Year Target Actual
2000 Baseline 11.8
2012 4.1
Long-term Outcome

Measure: Tons of fine particulate matter (PM2.5) reduced since 2000 from mobile sources


Explanation:The target number was generated in 2002 based upon EPA models which use pollutant emission factors and generate air quality inventories for on-highway vehicles, including passenger cars and trucks, and nonroad engines, such as construction/agricultural equipment. Data for the "actual" column is derived by applying the EPA models with revised estimates for variables such as annual vehicle miles traveled (VMT) for different vehicle categories.

Year Target Actual
2000 Baseline 510,550
2012 146,921
Annual Outcome

Measure: Tons of particulate matter (PM10) reduced since 2000 from mobile sources


Explanation:The target number was generated in 2002 based upon EPA models which use pollutant emission factors and generate air quality inventories for on-highway vehicles, including passenger cars and trucks, and nonroad engines, such as construction/agricultural equipment. Data for the "actual" column is derived by applying the EPA models with revised estimates for variables such as annual vehicle miles traveled (VMT) for different vehicle categories.

Year Target Actual
2000 Baseline 613,497
2003 37,297 37,297
2004 49,729 49,729
2005 62,161 62,161
2006 74,594 74,594
2007 87,026 87,026
2008 99,458 avail. 2009
2009 111,890
2010 124,322
2011 136,755
2012 149,187
Annual Outcome

Measure: Tons of fine particulate matter (PM2.5) reduced since 2000 from mobile sources


Explanation:The target number was generated in 2002 based upon EPA models which use pollutant emission factors and generate air quality inventories for on-highway vehicles, including passenger cars and trucks, and nonroad engines, such as construction/agricultural equipment. Data for the "actual" column is derived by applying the EPA models with revised estimates for variables such as annual vehicle miles traveled (VMT) for different vehicle categories.

Year Target Actual
2000 Baseline 510,000
2003 36,370 36,370
2004 48,974 48,974
2005 61,217 Available Fall 2008
2006 73,460 Available Fall 2008
2007 85.704 Available Fall 2008
Annual Outcome

Measure: Limit the increase of CO emissions (in tons) from mobile sources, compared to a 2000 baseline.


Explanation:The target number was generated in 2002 based upon EPA models which use pollutant emission factors and generate air quality inventories for on-highway vehicles, including passenger cars and trucks, and nonroad engines, such as construction/agricultural equipment. Data for the "actual" column is derived by applying the EPA models with revised estimates for variables such as annual vehicle miles traveled (VMT) for different vehicle categories.

Year Target Actual
2000 Baseline 79.2
2003 0.51 0.51
2004 0.67 0.67
2005 0.84 0.84
2006 1.01 1.01
2007 1.18 1.18
2008 1.35 avail. 2009
2009 1.52
2010 1.69
2011 1.86
2012 2.03
Annual Outcome

Measure: Millions of tons of nitrogen oxides (NOx) reduced since 2000 from mobile sources.


Explanation:The target number was generated in 2002 based upon EPA models which use pollutant emission factors and generate air quality inventories for on-highway vehicles, including passenger cars and trucks, and nonroad engines, such as construction/agricultural equipment. Data for the "actual" column is derived by applying the EPA models with revised estimates for variables such as annual vehicle miles traveled (VMT) for different vehicle categories.

Year Target Actual
2000 Baseline 11.8
2003 1.02 1.02
2004 1.35 1.35
2005 1.69 1.69
2006 2.03 2.03
2007 2.37 2.37
2008 2.71 avail. 2009
2009 3.05
2010 3.39
2011 3.73
2012 4.07
Annual Outcome

Measure: Tons of fine particulate matter (PM2.5) reduced since 2000 from mobile sources


Explanation:The target number was generated in 2002 based upon EPA models which use pollutant emission factors and generate air quality inventories for on-highway vehicles, including passenger cars and trucks, and nonroad engines, such as construction/agricultural equipment. Data for the "actual" column is derived by applying the EPA models with revised estimates for variables such as annual vehicle miles traveled (VMT) for different vehicle categories.

Year Target Actual
2000 Baseline 510,552
2003 36,370 36,370
2004 48,974 48,974
2005 61,217 61,217
2006 73,460 73,460
2007 85,704 85,704
2008 97,947 avail. 2009
2009 110,190
2010 122,434
2011 134,677
2012 146,921
Annual Outcome

Measure: Millions of tons of volatile organic compounds (VOCs) reduced since 2000 from mobile sources.


Explanation:The target number was generated in 2002 based upon EPA models which use pollutant emission factors and generate air quality inventories for on-highway vehicles, including passenger cars and trucks, and nonroad engines, such as construction/agricultural equipment. Data for the "actual" column is derived by applying the EPA models with revised estimates for variables such as annual vehicle miles traveled (VMT) for different vehicle categories.

Year Target Actual
2000 Baseline 7.7
2003 0.51 0.51
2004 0.68 0.68
2005 0.86 0.86
2006 1.03 1.03
2007 1.20 1.20
2008 1.37 avail. 2009
2009 1.54
2010 1.71
2011 1.88
2012 2.05
Long-term Efficiency

Measure: Percent reduction in time (days) per certificate approval for large engines (Nonroad CI, heavy duty gas and diesel engines)


Explanation:Measures average time in days from receipt of certification application to approval for three categories of large engines. Program cost will be monitored by a supplemental measure of program dollars per heavy duty certificate.

Year Target Actual
2005 Baseline 100 days
2012 50%
Annual Efficiency

Measure: Tons of pollutants (VOC, NOx, PM, CO) reduced per total emission reduction dollars spent (both EPA and private industry).


Explanation:For this measure, the reduction in tons for all pollutants (HC, CO, NOx, and PM2.5) are added on an annual basis. These are the numbers developed from our emission factor models and usage data, e.g., VMT, for on-road vehicles and for nonroad engines. The total annual cost of all of the EPA emission regulations is based on information in our regulatory impact analyses (RIAs).

Year Target Actual
2004 Baseline 0.016
2005 0.015 0.015
2006 0.013 0.013
2007 0.010 0.010
2008 0.010 avail. 2009
2009 0.011
2010 0.011
2011 0.012
2012 0.012

Questions/Answers (Detailed Assessment)

Section 1 - Program Purpose & Design
Number Question Answer Score
1.1

Is the program purpose clear?

Explanation: The purpose of the program is to protect public health by regulating harmful emissions from moving (mobile) sources of air pollution. Under title 2 of the Clean Air Act Amendments (CAAA), EPA is required to set technology-forcing requirements for all categories of new engines and vehicles as well as regulate characteristics of the fuel used in such sources. Mobile sources, including cars and light trucks, large trucks and buses, nonroad recreational vehicles (such as dirt bikes and snowmobiles), farm, industrial and construction equipment, lawn and garden equipment, and locomotives, pollute the air through combustion and fuel evaporation. These emissions contribute to air pollution problems nationwide.

Evidence: Title 2 of the Clean Air Act (CAA).

YES 20%
1.2

Does the program address a specific and existing problem, interest or need?

Explanation: A significant part of the country does not meet health standards for ground-level ozone and particulate matter. Mobile sources are major contributors to nitrogen oxide (NOx), particulate matter (PM), hydrocarbon, carbon monoxide (CO), and sulfur oxide (SOx) inventories nationwide. Given that about 50% of ozone precursors (NOx and VOCs) and particulate matter come from mobile sources, additional reductions are needed.

Evidence: EPA recently told 31 governors that areas of their states do not meet new standards for ozone. These areas represent part or all of 474 counties with a total population of 159 million people. Current estimates for PM2.5 are that 65 million people live in areas that violate the PM2.5 NAAQS. State Implementation Plans (SIPs) reflect the need for federal mobile sources programs in order to attain and maintain air quality standards.

YES 20%
1.3

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: Congress has given EPA primary authority to control emissions from new vehicles and engines in the U.S, and there are EPA regulations or other control measures in place for all mobile source categories. The CAAA allow California to adopt its own emission standards for most mobile source categories (California is pre-empted from regulating emissions for new locomotives, marine vessels, and all construction and agricultural equipment under 175 hp), as California has historically had the nation's worst air quality. California must seek and obtain a waiver from EPA prior to adopting its own standards, thereby assuring that they are not redundant or duplicative. Other states are preempted from adopting their own emission requirements and almost exclusively rely on federal control measures to achieve reductions from mobile sources. There are no private efforts in this area.

Evidence: Title 2 of the CAA. Sections 177, 209 and 211 delineate States' limited authorities to adopt mobile source requirements, including requirements applicable to the California waiver process. Current SIPs show that States rely on federal mobile source control measures to reach attainment and maintain the ozone and PM NAAQS.

YES 20%
1.4

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The CAA provides EPA authority to control emissions from a wide range of mobile source categories. Provisions in Title 2 allow EPA to determine where and when more stringent standards are necessary and appropriate and to take action to implement such controls. EPA has the authority to periodically review its standards to ensure they are meeting the Act's mandate and achieving air quality improvements. EPA also has authority to regulate fuel characteristics, which allows for more effective regulations. Emission credit averaging, banking, and trading programs and other market incentives are tools available to ensure efficiency.

Evidence: Section 202 and 213 of the CAA authorize EPA to promulgate 'and from time to time revise' regulations addressing pollution from mobile sources. EPA's rulemaking documents provide evidence of benefits in excess of costs.

YES 20%
1.5

Is the program effectively targeted, so that resources will reach intended beneficiaries and/or otherwise address the program's purpose directly?

Explanation: Program resources effectively address the program purpose. People nationwide benefit from mobile source pollutant reductions. All sources covered by Title 2 of the Clean Air Act have emission standards and other applicable requirements that are cost effective for ozone, CO, and particulate matter control. Program resources also benefit States, who rely on mobile source emissions reductions to support SIPs. EPA also provides modeling resources to States to use in SIP development.

Evidence: Program resources are targeted to provide high priority benefits to public health based on current scientific understanding of health risks and analysis of where the most cost effective improvements could be made. State support programs are designed with stakeholder input, and because States with the greatest need typically receive the most support, these resources are effectively targeted to beneficiaries.

YES 20%
Section 1 - Program Purpose & Design Score 100%
Section 2 - Strategic Planning
Number Question Answer Score
2.1

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The program's long term purpose is to improve air quality to healthy levels by reducing emissions from mobile sources of air pollution. The program quantifies progress toward this objective with 3 long-term perfomance measures that track reduced emission from mobile sources of 3 main pollutants. As these reductions can reasonably be expected to translate into improved air quality and contribute to attainment of health-based standards, they are considered to be end outcome measures. The program is developing a long-term outcome efficiency measure.

Evidence: Long term performance measures and targets are included in EPA's Strategic Plan. A report from the General Accounting Office (GAO), Managing for Results: EPA faces challenges in developing results-Oriented Performance Goals and Measures (April 2000) classifies this type of performance measure as an end outcome.

YES 11%
2.2

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The mobile source program's long-term performance targets are ambitious considering the level of reductions achieved over the past 30 years and the expected increase in vehicle miles traveled (VMT). For example, obtaining these targets will provide reductions of PM and NOx emissions that exceed reductions achieved from 1970-2001, over a period in which VMT is projected to increase by more than 12 percent.

Evidence: Long term performance measures and targets are included in EPA's Strategic Plan. Emission Reductions are included in EPA Trends Reports: U.S. EPA. U. S. EPA. 2003.

YES 11%
2.3

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The program has several specific and quantified annual performance measures under the EPA strategic plan. The program uses annual emissions reductions to incrementally track its progress toward its long-term performance measures. As an increment of the long term performance measures, these measure can clearly demonstrate progress towards the long term goals. The program is has developed an output-based efficiency measure that will be included in the 2006 GPRA documents.

Evidence: U.S. EPA Fiscal Year 2003 Annual Report. Measurement Implementation Plan (MIP) for compliance certificate efficiency measure.

YES 11%
2.4

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The program has specific baselines to measure progress toward its annual targets. The baselines for the annual performance measures are 1995 emission levels. The annual targets are ambitious considering the level of reductions achieved in the last 30 years, expected increases in vehicle miles traveled (VMT), and long (10-20 year) fleet turnover times.

Evidence: U.S. EPA Fiscal Year 2003 Annual Report. Emission Reductions are included in EPA Trends Reports: U.S. EPA. U. S. EPA. 2003.

YES 11%
2.5

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: The program uses contractors, grantees, and interagency agreements to procure a number of program support services, including technical and analytical support, testing services, data collection and analysis, and information technology support. Contractors submit monthly progress reports, which evaluate performance quality, cost control, and timeliness of performance, but these reports are not designed to include performance goals that clearly align with program goals. In the area of grants management, the Office of Transportation and Air Quality (OTAQ) is beginning to administer an EPA plan that will require increased monitoring along with documentation of how proposed agreements contribute to long term goals, but there is not yet sufficent evidence of effectiveness.

Evidence: Sample Contractor Performance report.

NO 0%
2.6

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: A recent report from the National Research Council (NRC) evaluated EPA's effectiveness in implementing emission controls on mobile sources. While this study likely meets the criteria of quality, scope, and independence in its review of the program, there is no evidence that it is part of of a regularly scheduled review process. Other independent evaluations have not met scope or quality criteria.

Evidence: National Research Council: Air Quality Management in the United States (2004). Other independent reviewers include: the Clean Air Act Advisory Committee Mobile Sources Technical Review Subcommittee; GAO,Air Pollution: Limitation's of EPA's Motor Vehicle Emissions Model and Plans to Address Them (Sept 1997), National Academy of Sciences (NAS), Modeling Mobile Source Emissions(2000) ; Coordinating Research Council. Validation of the U.S. EPA MOBILE6 Highway Vehicle Emission Factor Model(2002); U.S. Department of Transportation, Sensitivity Analysis of MOBILE6 Motor Vehicle Emission Factor Model (2003); EPA's Inspector General (IG); and various industry groups.

NO 0%
2.7

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: The Budget documents do not clearly indicate the full costs of achieving the Mobile Source program's performance goals. The impact of funding decisions on expected performance is also unclear.

Evidence: EPA's Annual Plan and Budget Request documents. The annual budget request is tied to the overall "Healthier Outdoor Air" objective, but documentation does not clearly indicate full costs of achieving the program's goals.

NO 0%
2.8

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The program has taken steps to improve strategic planning. As an ongoing effort to identify deficiencies, the program is an active participant in the Agency's Multi-year Planning Meetings that are held annually with the Deputy Administrator and CFO to review performance progress. These discussions focus on performance progress over a multi-year period, and provide an early opportunity to identify potential obstacles or challenges that may necessitate changes in direction to ensure the program's ability to attain its long-term performance goals. Also, at least twice a year, the program's Senior Leadership Team (SLT), which is comprised of the Office Director, the Deputy Office Director, the Associate Office Director, and the five Division Directors, conducts off-site planning and management discussions to review the organization's strategic goals, long-term performance goals, and current program performance objectives.

Evidence: In 2003 the program revised its long-term program goals and strategies in the EPA Strategic Plan by implementing measurable, outcome-oriented performance goals. The program is also implementing two efficiency measures as a result of the PART review process. Specific deficiencies that were identified and addressed in the most recent SLT strategic planning sessions included: the need to shift additional resources to complete a large rulemaking; the need to internally redirect additional resources to develop a new (unanticipated) and congressionally-mandated rulemaking; undertaking a major internal effort aimed at evaluating the effectiveness of retrofit technologies in the field; and the implementation of an integrated strategy for in-use emissions testing.

YES 11%
2.REG1

Are all regulations issued by the program/agency necessary to meet the stated goals of the program, and do all regulations clearly indicate how the rules contribute to achievement of the goals?

Explanation: The CAA gives EPA authority to determine whether emissions requirements for motor vehicles, highway vehicles and engines, and nonroad engines are appropriate, and if so, to develop appropriate regulations. EPA may also regulate fuel characteristics. Air quality data indicate that mobile source emission reductions continue to contribute to improvement in public health. Final rules generally include an analysis showing the air quality need for action, and the preamble of each of these rules typically states how the specific rule contributes to the achievement of program goals by defining specific tons of emission reductions, and where calculated, health benefits.

Evidence: Title 2 of the CAA and EPA rules and associated technical support documents.

YES 11%
Section 2 - Strategic Planning Score 67%
Section 3 - Program Management
Number Question Answer Score
3.1

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: The program continuously collects performance information from manufacturers and other sources to ensure emission reductions are achieved. As part of the certification process, manufacturers provide testing data on vehicles and engines to demonstrate they meet emissions standards. Manufacturers also must report defective emissions equipment. EPA receives quarterly recall reports from manufacturers and periodically conducts recall audits. EPA also regularly collects information from fuel refiners to ensure compliance with fuel standards. To improve model development, EPA regularly collects performance information and supporting data. EPA also obtains performance information through peer and stakeholder review, validation studies, and other channels of communication such as a list-serve (to communicate with key program partners) and interactive workshops.

Evidence: Data collected on model performance is used in one of three ways to improve model performance: changes are made to the software code itself (e.g., algorithms changed), changes are made to block data or input data that the model uses to make its calculations, and/or changes are made to guidance on the use of the model. Data collected from manufacturers and refiners is used to help determine appropriate compliance measures (e.g., which engine families to test in the recall test program)and to redirect, if appropriate, Agency resources.

YES 9%
3.2

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Senior managers develop and identify the major program commitments and results to be achieved during the year, with a given allocation of resources, and these commitments are incorporated into performance appraisals. The plans are reviewed and updated as appropriate during the year and serve as a major management tool for tracking performance and expenditures. Evaluation criteria in these plans include results-oriented performance measures that relate to the program goals (but typically do not directly reflect the goals themselves). Contractors that provide programmatic work products are held accountable for cost, schedule, and performance results through the written terms of contract awards, reports filed with contract managers, and regular performance reviews.

Evidence: SES Performance Management System. Sample Performance Agreement, Appraisal and Certification documents with program-specific perfomance measures. Written terms of contract awards.

YES 9%
3.3

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: Program funds are obligated in a timely manner and are aligned with the program's statutory authorities and GPRA goals. Obligations and expenditures of funds are tracked in the Agency's IFMS against OMB-approved operating plan levels. During the year, OTAQ's expenditures are monitored regularly by staff using a number of available tracking systems, including IFMS, the OCFO's Budget Automated System (BAS), and OTAQ's own internal tracking system (linked to IFMS). Reports are prepared for OTAQ senior management on the utilization of funds. Status of funds reports are also available to all OTAQ program managers and budget staff through the use of OTAQ's internal tracking system.

Evidence: EPA Annual Operating Plan, IFMS Financial Reports, Budget Automation System Reports, and OTAQ Status of Funds Reports. EPA. Annual Financial Statements.

YES 9%
3.4

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The program has processes in place to ensure efficient use of funds and is developing at least one efficiency measure to measure and achieve greater efficiencies. Contract funding is awarded through full and open competition in compliance with EPA rules and procedures. Cost comparisons are an important element of this process. Grant funding provided by OTAQ is also awarded in accordance with the Agency's competitive award policy. IT and facility improvements will continue to produce efficiencies and achieve cost savings. Private costs are minimized through the use of averaging, banking and trading (ABT) programs.

Evidence: A facility modernization project at the National Vehicle and Fuel Emissions Laboratory (NVFEL) will reduce energy consumption at NVFEL by an anticipated 66 percent, lower energy costs by 74 percent, and reduce water consumption 80 percent. An example of IT improvements is the redesign of the Compliance and Fuel Economy Information System (CFEIS) which will achieve cost savings for EPA and industry. This will result in an annual savings of 2 FTE (equivalent to $200k) and a 50% reduction in maintenance costs. General Motors estimated it could save $470,000 annually. EPA's motor fuels ABT programs employ simplified electronic reporting methods in lieu of traditional paper forms submissions and decreases compliance costs.

YES 9%
3.5

Does the program collaborate and coordinate effectively with related programs?

Explanation: EPA routinely collaborates and coordinates with stakeholders and partners through workshops, conference calls, cross organizational teams, and collaborative testing programs. EPA participates with a DOE program to test advanced diesel engines. This testing directly supports EPA's regulatory programs for diesel engine control. EPA also works closely with CARB on regulatory control programs, including performance issues, and model development issues. EPA and California are working to redesign the certification and compliance database to ease the compliance burden on manufacturers and to facilitate data sharing. The program has also entered into a memorandum of understanding (MOU) with CARB to work together to address heavy duty engine emissions and locomotive emissions. Other examples include; a 3-day workshop with stakeholders in 2003 to share the latest information on mobile source model development; and a test program in Kansas City in 2004/2005 involving a wide range of partners including the STAPPA/ALAPCO, the Coordinating Research Council (CRC), DOT, DOE.

Evidence: Interagency agreement with DOE. Interagency agreement with DOT. MOU with CARB. Statement of Work for Kansas City project. Cooperative Research and Development agreement with CRC.

YES 9%
3.6

Does the program use strong financial management practices?

Explanation: OTAQ follows EPA's financial management guidelines for committing, obligating, reprogramming, and reconciling appropriated funds. EPA received an unqualified audit opinion on its FY 2002 financial statements and had no material weaknesses associated with the audit. OTAQ has no material weaknesses as reported by the Office of the Inspector General.

Evidence: Agency officials have a system of controls and accountability, based on GAAP and other principles, to ensure that improper payments are not made. Annually each Division Director in OTAQ is required to conduct an evaluation of their internal management control systems in accordance with guidance provided by OCFO and to certify that their management systems provide reasonable assurance that they are in compliance with the requirements of the Federal Managers' Financial Integrity Act. Evidence also includes audit reports from the EPA Office of Inspector General.

YES 9%
3.7

Has the program taken meaningful steps to address its management deficiencies?

Explanation: To identify and correct potential management deficiencies, OTAQ requires each Division Direction to prepare an Annual Assurance Letter to the Office Director. These letters evaluate and report on the management controls that support the achievement of programmatic goals and reasonably protect programs from potential mismanagement. These letters address any management integrity weaknesses or challenges that have been identified. The Annual Assurance Letter also provides opportunity for each Division Director to identify any new potential weaknesses or challenges that may require follow-up action. The Office Director then prepares and submits an OTAQ Office-level Assurance Letter to the Assistant Administrator for Air and Radiation.

Evidence: The Annual Assurance Letter system has enabled the program to identify and successfully correct a number of potential management deficiencies. In the FY 2000 and FY 2001 Assurance Letter process, management identified a potential deficiency related to the program's ability to accurately measure compliance with new, more stringent emission standards. Early identification enabled the program to work with the OCFO and OMB to fund improvements in the FY 2003 budget. Through the FY 2001 Assurance Letter process, the program identified and addressed a potential vulnerability related to data collection, modeling, and data systems. The OTAQ 2000 Reorganization Proposal describes how the program made significant changes to its organizational structure in order to address more readily the core business functions and mission of the organization.

YES 9%
3.REG1

Did the program seek and take into account the views of all affected parties (e.g., consumers; large and small businesses; State, local and tribal governments; beneficiaries; and the general public) when developing significant regulations?

Explanation: EPA's mobile source rulemaking process includes extensive outreach and coordination with stakeholders throughout the full range of regulatory development. Information obtained from this outreach, which ranges from telephone calls to written comments to facility visits, is reflected in rulemaking documents. EPA conducts Small Business Regulatory Enforcement Fairness Act (SBREFA) panels when appropriate. EPA also coordinates research activities in support of regulatory efforts by working with the Coordinating Research Council (made up of automobile and oil industry representatives). Final rules are typically supported by industry, state and local governments, environmental and public health groups, and other stakeholders.

Evidence: The program's rulemakings document stakeholder views, and reflect comments and other technical information obtained throughout the process. EPA's Office of Transportation and Air Quality has conducted six SBREFA panels. Except in a few instances, recommendations coming out of the panel process are incorporated into EPA's proposed rulemakings. When developing the heavy duty engine rule, EPA asked the Coordinating Research Council to analyze the effects of sulfur and other fuel components on emissions control technologies. The data generated from this research helped support the Agency's fuel sulfur reduction measures.

YES 9%
3.REG2

Did the program prepare adequate regulatory impact analyses if required by Executive Order 12866, regulatory flexibility analyses if required by the Regulatory Flexibility Act and SBREFA, and cost-benefit analyses if required under the Unfunded Mandates Reform Act; and did those analyses comply with OMB guidelines?

Explanation: The Office of Transportation and Air Quality typically prepares thorough engineering analyses in support of its rules, but these analyses have not always complied with OMB guidelines. However, recent rulemakings from the program have shown improvements by incorporating economic analysis. New guidelines (OMB Circular A-4) will require the program to continue to upgrade its analyses of major rules.

Evidence: EPA mobile source rulemaking documents.

YES 9%
3.REG3

Does the program systematically review its current regulations to ensure consistency among all regulations in accomplishing program goals?

Explanation: The program has reviewed regulations and determined that more protective standards were appropriate. However, there is no regular review process in place to analyze rulemaking to determine if the requirements are still necessary, relevant, and consistent.

Evidence: Intermittent reviews have led to changes in existing rules. Standards for passenger cars, for example, have revised three times in the last decade, and similar trends can be seen in fuel sulfur requirements and emission standards for diesel engines. However, there is no evidence of a regular plan for review of existing rules.

NO 0%
3.REG4

Are the regulations designed to achieve program goals, to the extent practicable, by maximizing the net benefits of its regulatory activity?

Explanation: The program's regulatory process involves regulatory impact analyses, cost-effectiveness analyses, cost-benefit analyses, and small business impact analyses (e.g., SBREFA), which help the Agency in selecting appropriate options when setting standards for mobile sources.

Evidence: EPA mobile source rulemaking documents show evidence of a decisionmaking process that leads to benefits in excess of cost.

YES 9%
Section 3 - Program Management Score 91%
Section 4 - Program Results/Accountability
Number Question Answer Score
4.1

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: Emissions per VMT of key air pollutants from motor vehicles have decreased substantially since enactment of the CAA. Reductions in CO emissions have been linked to significantly reduced overall population exposure. Mobile source programs promulgated within the last decade are expected to further reduce emissions and will contribute to meeting the targets established for 2010. However, mobile sources remain a significant contributor to air pollution in the U.S., and EPA data indicates that nearly half of Americans still live in areas with unhealthy air. Gaps remain in the program's ability to directly relate projected emissions reductions (based on emissions factors and model results) to higher level outcomes such as reduced population exposure.

Evidence: The NRC Air Quality Management report (2004) states that reductions in motor vehicle emissions per mile traveled have been significantly reduced. Another recent NRC study (NRC 2003b) links mobile source standards to reductions in population exposure. Emission reductions and inventory trends are included in EPA's air quality trends report: Latest Findings on National Air Quality: 2002 Status and Trends. Progress towards long term goals is detailed in EPA's Fiscal Year 2003 Annual Report. EPA rule documents include emission reductions expected in different mobile source categories.

LARGE EXTENT 11%
4.2

Does the program (including program partners) achieve its annual performance goals?

Explanation: The program met annual mobile source performance measures in FY 2001- FY 2003 and is on track to meet annual performance measures for FY 2004.

Evidence: U.S. EPA. Fiscal Year 2003 Annual Report.; U.S. EPA Fiscal Year 2004 Annual Performance Goals (APG) and Measures, Goal 1 ' Clean Air and Global Climate Change. Data from EPA's National Emissions Inventory and inventory projections indicate that at mid-year 2004, the mobile source program is on target to meet its annual performance measures.

LARGE EXTENT 11%
4.3

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The program has demonstrated increased efficiency. The light-duty vehicle program has been streamlined to reduce burden on both industry and government while maintaining stringency. In addition, EPA has improved the efficiency of its regulatory programs by addressing vehicle/engines and fuels as a system. Additional regulatory efficiencies have been achieved by relying on technology transfer from previously regulated vehicle/engine categories. With continued compliance program streamlining, and by implementing one or more efficiency measures in the near future, the program will be able to make a stronger demonstration of efficiency improvements.

Evidence: In FY 2003 the Certification program issued 2,565 certificates. This is triple the number of certificates issued in 1995 with fewer FTE. For the 2004 model year, approximately 7% more certificates were processed than 2003, with one fewer FTE. The U.S. EPA CAP2000 rule reduced the annual burden on industry through improved procedures to ensure compliance. It also reduced EPA compliance burden by reducing the number of EPA confirmatory tests and reducing staff time. The agency estimates information reporting savings at over 500,000 burden-hours of effort (about a 54% reduction), resulting in $1.5 million in annual savings. EPA's compliance program is currently working with industry on further streamlining initiatives. The program's proposed efficiency measure is included on the measures tab.

LARGE EXTENT 11%
4.4

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: Because of severe air quality problems, the State of California has been given the authority to develop mobile source regulations to address their specific local issues. A few other states also have chosen to adopt and implement California's light-duty vehicle program to address specific local air quality problems. Although these state programs don't cover the scope of a Federal national program, they are comparable in some aspects and share several goals. However, the Federal program is usually more effective and efficient, as it addresses air quality issues more comprehensively at a national level, which helps reduce costs while increasing the opportunity for additional benefits. There are no private programs to compare with EPA's mobile source program.

Evidence: The programs can be compared based on major with similar goals. For example, California LEV II regulations can be compared with EPA Tier 2 regulations for light-duty vehicles. In 2010, the California LEV II regulations will reduce smog-forming emissions statewide by 56.6 thousand tons/year, at a cost of about $3000/ton. The EPA Tier 2 program is projected to reduce smog-forming emissions (NOx+HC) in 2010 by 1.86 million tons, at a cost of $2150/ton. Another example of comparable EPA and California programs relates to NOx and HC controls for small engines (e.g., lawnmowers) . The cost-effectiveness of the California program is estimated to be $2000 - $4000/ton, while the EPA program has a cost-effectiveness of $507-$857/ton.

YES 17%
4.5

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: The NRC Air Quality Management report (2004) credits the mobile source program with standards that have significantly reduced motor vehicle emissions per mile traveled. Other independent program evaluations, though not necessarily meeting the scope or quality criteria, have supported program acheivments. Multiple studies have linked CO standards to reductions in ambient concentrations, and to related health benefits. Independent reviews of the program's models indicate that emissions reductions will result in air quality improvements. However, evaluations also report that, after almost 3 decades of regluations, mobile source emissions continue to be a major source of air pollution in the U.S.

Evidence: The health impacts of CO reductions are shown in a study published in the Journal of American Medical Association. A recent NAS report analyzed the state of CO pollution in the United States and determined that federal carbon monoxide standards on motor vehicles had effectively eliminated almost all violations of CO ambient air quality standards. Independent evaluations of EPA's mobile source models have indicated that these models are effective and achieve intended results. Three independent evaluations have evaluated EPA's mobile source models (National Research Council, Coordinating Research Council, and U.S. Department of Transportation). InAir Quality Management in the United States(2004), The National Research Council finds that progress has been made in controlling mobile source emissions, but that "emissions from mobile sources are still a major source of air pollution in the United States."

LARGE EXTENT 11%
4.REG1

Were programmatic goals (and benefits) achieved at the least incremental societal cost and did the program maximize net benefits?

Explanation: As stated above, the program has not always complied with OMB guidelines requiring a thorough economic analysis and evaluation of alternatives. Without this type of analysis, determination of maximum benefits is not possible. However, in support of recent regulations, the program has analyzed reasonable alternatives and provided an evaluation of costs and benefits. In some of these rulemakings, the program has projected significant net benefits.

Evidence: EPA mobile source rulemaking documents. The light-duty vehicle rule (Tier 2), the heavy-duty diesel rule for highway trucks, and the engine and fuel standards for nonroad diesel engines all showed projected benefits in excess of costs.

LARGE EXTENT 11%
Section 4 - Program Results/Accountability Score 73%


Last updated: 09062008.2004SPR