Detailed Information on the
Stratospheric Ozone Protection Assessment

Continue to support the Multilateral Fund for the Implementation of the Montreal Protocol.

Continue to monitor progress to ensure that the program is on track to meet goals.

Convert long-term health effects measure into a rate of skin cancer prevalence so that an actual baseline can be established once statistics are available.

By the end of July 2008 brief OMB on progress developing a performance measure and targets to track intermediate outcomes by measuring "thickness" of the ozone layer in the atmosphere. Many of the program's outcome performance measures are extremely long-term, so it is important to establish measurable performance objectives for the near term.

By the end of July 2008 brief OMB on progress developing a long-term performance measure and setting ambitious targets for reduced incidence of non-melanoma skin cancers.

Measure: Elimination of US consumption of Class II ozone depleting substances, measured in tons/yr of ozone depleting potential (ODP).

Explanation:The release of class II ODS destroys stratospheric ozone, which causes an increase in UVA and UVB ray penetration to earth's surface. By eliminating Class II ODS consumption, EPA protects public health and the environment by combating skin cancer and ecological damage caused by over-exposure to ultraviolet rays. EPA collects data quarterly or annually on the production, import, and export of Class II ODS. We analyze this data to report our domestic consumption of ODPs. Targets do not account for future critical and essential use exemptions approved by the Montreal Protocol parties.

Measure: Reduced incidence of melanoma skin cancers, measured by new skin cancer cases avoided per 100,000 population.

Explanation:EPA will look at new skin cancer cases avoided per 100,000 population based on data from the US Cancer Statistics Incidence and Mortality reports. Destruction of the stratospheric ozone allows more ultraviolet rays (including UVB) to penetrate the earth and puts individuals at higher risk of all type of skin cancer. Measuring melanoma incidence is a key indicator in determining EPA's success of protecting public health by implementing measures that protect the stratospheric ozone. Skin cancer (melanoma and non-melanoma) is a long-latency disease and thus demonstrating results from this measure is not immediate but rather long-term. A revised baseline will be developed in 2015 to capture the incidence level around the time when the rate of chlorine loading to the atmosphere is expected to stabilize and concentrations begin to decline.

Measure: Level of total equivalent stratospheric chlorine, measured in parts per billion of air by volume.

Explanation:Stratospheric chlorine (chlorine found in the stratosphere) destroys stratospheric ozone, which in turn allows more UVA and UVB to penetrate onto the earth's surface. Over-exposure to such dangerous ultraviolet rays can cause skin cancer and damage the ecosystem. The overall goal is to see levels of EESC decline to pre-1980 levels (about 1.8 ppb).

Measure: Remaining U.S. consumption of Class II ODS, measured in tons of ozone depleting potential (ODP).

Explanation:The Montreal Protocol provided for a gradual decrease in Class II ODS consumption. The protocol required each participating developed country create a consumption baseline and provided guidance on how to calculate that baseline. The US consumption baseline is 15240 ODP (2.8% of U.S. 1989 CFC consumption plus 100% of U.S. 1989 HCFC consumption. Targets do not account for future critical and essential use exemptions approved by the Montreal Protocol parties.

Measure: Total federal dollars spent per school joining the SunWise program.

Explanation:The program's mission is to restore/protect the ozone layer and protect public health, via two mechanisms: 1) implementing regulatory and voluntary measures to phase-out ODS consumption/release in the United States; 2) public education to raise awareness of how to avoid risk of skin cancer. These efforts should decrease incidence of all types of skin cancer incidence in the US. However, measuring this is difficult, because non-melanoma skin cancer is not regularly surveyed, and because the disease has a latency period.

Is the program purpose clear?

Explanation: The program's purpose is part of a global effort to protect human health and the environment through the restoration of the earth's protective ozone layer. The ozone layer shields earth from excessive ultraviolet (UV) radiation that causes skin cancer, cataracts, and immune suppression. Excessive UV radiation has also been shown to contribute to the reduction of crop yields; adversely impact aquatic organisms, particularly zooplankton, and amphibian and fish larvae; and accelerate the decay of polymers and other materials. This program implements the parallel requirements of Title VI of the Clean Air Act, the Montreal Protocol on Substances that Deplete the Ozone Layer, and subsequent amendments to the Protocol, as ratified by the United States.

Evidence: In 1990, the amendments to the Clean Air Act directed EPA to conform the U.S. phasedown schedule to the Montreal Protocol's requirements for developed nations, including interim reductions and specific exemptions. In the Preamble to the Montreal Protocol, the stated objective is "...protect[ing] the ozone layer by taking precautionary measures to control equitably total global emissions of substances that deplete it, with the ultimate objective of their elimination on the basis of developments in scientific knowledge, taking into account technical and economic considerations and bearing in mind the developmental needs of developing countries" and to "protect human health and the environment against adverse effects resulting or likely to result from human activities which modify or are likely to modify the ozone layer."

Does the program address a specific and existing problem, interest or need?

Explanation: Measurements of stratospheric ozone taken since the 1970's over the Antarctic and Arctic have shown significant depletion of the ozone layer. Science has shown that this is due to emissions of chlorofluorocarbons (CFCs), halons, and other halogenated chemicals. Depletion of stratospheric ozone is an urgent environmental and public health problem because the ozone layer provides critical protection from solar UV radiation for life on the earth's surface. UV radiation is associated with a number of human health affects, and in particular was declared a known carcinogen by the U.S. Department of Health and Human Services in 2002. Skin cancer is the most common form of cancer in the U.S., accounting for 50% or more of all cancers. Rates of melanoma, the deadliest form of skin cancer, have increased 3-6% a year over the last 30 years, and the National Cancer Institute reports that incidence of melanoma has doubled since the mid-1970's, when the ozone hole was identified. In 2004, the American Cancer Society estimates over 59,000 new melanomas will be diagnosed, and over 10,000 deaths will occur due to melanoma.

Evidence: The 2002 Science Assessment for the Ozone Secretariat of the United Nations Environment Program concluded that: 1) the ozone layer will remain particularly vulnerable over the next decade; 2) total chlorine abundance in the stratosphere is near its peak; and 3) bromine concentrations are still increasing. Continuous monitoring by satellites, aircraft, and weather balloons operated by the National Oceanic and Atmospheric Administration(NOAA), the National Aeronautics and Space Administration (NASA) Total Ozone Mapping Spectrometer (TOMS), and the International Centre for Antarctic Information and Research have demonstrated ozone layer destruction by ODS, and increased UV penetration to the earth's surface. The Surveillance, Epidemiology, and End Results (SEER) Program: Melanomas of the Skin, Trends in Incidence and Mortality by Race and Age, shows a 2.7 percent annual increase for melanoma incidence between 1981 and 2000 for people of all races, male and female, in the United States.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: EPA's Stratospheric Ozone Program is the only governmental or private program in the U.S. designed and authorized to eliminate ozone-depleting substances (ODS). It is unique in providing for critical use exemptions, establishing import and export quotas for ozone depleting chemicals, allocating production allowances, and setting long-term phaseout schedules. EPA works with the Department of State to develop International ozone policy, but the Agencies have different roles in coordinating with and negotiating with other Parties to the Protocol. EPA's Significant New Alternatives Policy (SNAP) program reviews alternatives to ODS so as to to facilitate a safe and economically stable transition away from ozone depleting chemicals across multiple industrial, consumer, and military sectors. EPA actively seeks to avoid overlap with other existing regulatory authorities by deferring to standards established by relevant governmental agencies (e.g., OSHA) or private organizations, where applicable.

Evidence: Congress, in the Clean Air Act, delegated responsibility for programs that protect stratospheric ozone to EPA, with few exceptions. When working with other International parties to the Montreal Protocol, EPA generally prepares the initial positions for all international negotiations on ozone related issues., whereas the State Department ensures that the proposed positions are consistent with broader US policy goals.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The program appears to have been designed to efficiently reduce the production, import, export, and emissions of ODS, and there is no strong evidence that another approach would be more effective. Since ozone depletion is fundamentally a global problem, the program has domestic and international components. Domestically, EPA has established market-based systems to meet ODS phasedown objectives while assuring that supply and demand are balanced in an efficient, reduced cost manner. Internationally, EPA participates through its contributions to the Mulitlateral Fund. The U.S. has a permanent seat on the Multilateral Fund Executive Committee and is able to oversee implementation of projects designed to enable developing countries to comply with the Montreal Protocol. Though there is no evidence of major flaws in this design, there is also no strong evidence that the U.S. Government could not get the same or better outcome by expending resources through a different mechanism.

Evidence: EPA estimates total costs of the program to be $57 billion (1990$, with a 2% discount rate) compared to health and ecological benefits between 1990 and 2165 estimated to be $4,300 billion. The U.S. has meet all phaseout schedules, and, in some cases, well ahead of deadlines initially established under the Montreal Protocol. It is estimated that so far allowance trading for CFCs saved industry $298 million, as compared to a command and control system (EPA Chlorofluorocarbon Taxes and Allowance Trading).

Is the program effectively targeted, so that resources will reach intended beneficiaries and/or otherwise address the program's purpose directly?

Explanation: The program's resources reach the intended beneficiaries, which include the national and international communities. For the latter, the Multilateral Fund uses a compliance directed model to target resources to developing countries with specific schedules so that phaseout dates under the Protocol can be met. It also uses cost effectiveness criteria in prioritizing projects in developing countries and has several mechanisms in place to ensure that that the funds allocated to specific projects are accounted for and verified. EPA's domestic program reaches US producers and importers wishing to trade chlorofluorocarbon (CFC) allowances by creating a trading program and establishing an effective framework for trading. EPA aso provides dedicated assistance to producers and importers that trade allowances.

Evidence: The availability of stockpiles of ozone depleting chemicals and of alternatives are regularly evaluated in sector-specific UNEP Technical Options Committee reports (www.teap.org/reports.html), as well as by regular EPA reviews of industry data (e.g., Wickham 2002; Caleb 2000, 2001; ICF Solvent Market Report, 2004). These assessments provides up to date information on industry progress and obstacles to transitions so that the program can target its technology and information transfer, and needed regulatory adjustments to priority sectors. In a 1997 assessment of the Multilateral Fund, GAO found that the Fund "has a number of mechanisms in place that are designed to ensure that funds are properly accounted for and that the amounts of funds allocated to specific projects are reviewed and verified."

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The program's purpose is to ensure that U.S. commitments are met to help restore the ozone layer. The program has several long-term outcome performance measures that meaningfully reflect this purpose. These measures include the specific goals and deadlines in the Montreal Protocol and Title VI of the Clean Air Act (CAA). The measures fall into three categories: health effects (reduction in skin cancers), atmospheric chemical loading, and consumption of Class II ODS (HCFCs). The consumption measure is considered an outcome measure because elimination of consumption correlates directly to elimination of emissions.

Evidence: See Measures tab.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: Ambitous targets and timelines have been established. Consumption reduction targets are based on ODS phaseout targets and timeframes specified in the Clean Air Act and Montreal Protocol, with the final target of elimination of production and import in the U.S. of all ozone depleting substances by 2030. Atmospheric loading targets are based on US production and importation reported to EPA annually and concurrent with periodic WMO Scientific Assessments, which are every 5 years. Health effects targets are quantified and ambitious, but extremely long term, and a counterfactual baseline will make verification impossible.

Evidence: See Measures tab.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The program has an annual performance measure that demonstrates progress toward achieving its specific goals and those of the Montreal Protocol and Clean Air Act. The annual measure uses consumption targets to track progress towards meeting long term goals. This measure is considered an outcome measure because elimination of consumption correlates directly to elimination of emissions. The program is currently developing an efficiency measure using cost per chemical tons consumption avoided, by chemical or chemical class (the cost would include industry costs).

Evidence: See Measures tab.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: Baselines have been established and ambitious annual targets have been developed for 2003 through 2019 for Class II ODS, based on the phaseout steps required by the Montreal Protocol.

Evidence: See Measures tab.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: The program uses contractors, state grantees (via a portion of the 105 grant), other federal agencies and grantees in the academic community for data analysis and collection assistance. Contractors and grantees (other than academic) commit to the long-term goals of the program through contract and annual grant agreements. EPA and state grantees have a Core Performance Measures agreement, wherein states are responsible to report progress on meeting program goals. Contractor work is reviewed and is subject to an evaluation process.

Evidence: EPA contract and grant documentation. Stratospheric ozone program grant proposals and published results.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: Independent organizations have performed evaluations of program effectiveness, but the only evaluations of suitable scope focus on the Montreal Protocol and are not specifically related to the effectiveness of EPA's Stratospheric Ozone program.

Evidence: Reports on EPA and US implementation of the Stratospheric Ozone Protection program have been conducted by the EPA Inspector General, the General Accounting Office, World Resources Institute, World Meteorological Organization, the Global Environmental Technology Fund and the Montreal Protocol's Technology and Economic Assessment Panel of the Multilateral Fund..

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: The Agency estimates and budgets for the full annual costs of operating the regulatory and international programs. Compliance and enforcement activities in support of stratospheric ozone goals are not included in program's budget. The Agency's financial information is integrated with performance and other program data to support day-to day decision making of managers and executives. However, the budget presentation does not make clear the impact of proposed funding on expected performance. The program has not shown that it can demonstrate how changes in funding affect performance.

Evidence:  

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The program regularly reviews planning efforts and reviews its annual business plan in a mid-year report. This gives the program an opportunity to continuously review performance and make corrections to ensure annual goals are met. When deficiencies are identified, resource and programmatic allocations are modified accordingly. An example of how the program addresses problems associated with ODS phaseout is the way exemptions have been created for applications where alternatives are not fully developed. For example, HCFC-141b was phased out in January 2003. The vast majority of users found effective alternatives in time for the phaseout. To address the needs of the remaining users, an exemption program was established to allow production for those who could demonstrate significant technical hurdles in using alternatives and a lack of access to stockpiles. In 2004, exemptions were granted for new production of HCFC-141b for critical applications including the Space Shuttle and defense systems. In addition, the program has created an efficiency measure, the results of which will assist the program in its strategic planning.

Evidence: Each year for phased out chemicals, the program evaluates needs and authorizes production for critical and essential uses such as metered dose inhalers to treat asthma. Since the phaseout of CFC, the program has authorized new production of between 3,000 and 5,000 ODP-weighted metric tons per year, as approved by the Parties to the Montreal Protocol, for use in metered dose inhalers.

Are all regulations issued by the program/agency necessary to meet the stated goals of the program, and do all regulations clearly indicate how the rules contribute to achievement of the goals?

Explanation: The program has only implemented regulations that are necessary to protect human health associated with ozone depletion. All measures have been presented to relevant stakeholders to ensure no unnecessary regulations have been put forward. Rule preambles and guidance documents are clearly written and describe how the rule would contribute to the achievement of specific program goals.

Evidence: 1 Sections 604, 606, 608, 609, 610, 611, 612 and 613 of the Clean Air Act2 Article 2 of the Montreal Protocol specifies ODS for phaseout and Section 614 of the Clean Air Act requires EPA to meet the Protocol requirements3 Policies and Guidelines of the Multilateral Fund www.unmfs.org.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: The program tracks the amount of ozone depleting substances being produced and exported, or used for certain small exceptions or technical and public health reasons, through EPA's Ozone Depleting Substances Tracking System. A 2003 quality assurance audit of the program's tracking system found that the system was accurate and comprehensive in meeting the goals of the program to maintain updated ODS production/export/import data. The program also relies on information from other federal agencies. For example, National Aeronautics and Space Administration (NASA) monitors the ozone layer and collects measuresments daily with Total Ozone Mapping Spectrometers (TOMS). In addition, the National Oceanic and Atmospheric Administration (NOAA) also monitors the ozone layer with satellite, air-borne, and ground-based instruments. These instruments directly measure stratospheric ozone in addition to conditions that change the composition of the ozone layer. This information allows allow EPA to judge progress made in stratospheric ozone recovery and to determine if further measures are necessary to reach adequate concentrations. The program conducts ongoing, real-time evaluations of all contractors on each individual work assignment to assure that data and analyses are of high quality.

Evidence: NASA data is posted at toms.gsfc.nasa.gov/ozone/ozone.html. NOAA data are provided for the North and South Pole and for total global ozone. Data can be accessed at www.ozonelayer.noaa.gov/data/data.htm. EPA, the National Science Foundation, the Smithsonian, NOAA's Office of Global Programs' Surface Radiation Budget Network, and the U.S. Department of Agriculture, all maintain ground-based instruments to monitor ultraviolet radiation reaching the earth's surface daily. EPA regularly reviews performance information that is presented for consideration by the Multilateral Fund's Executive Committee. This includes data on project costs, administrative cost, capital cost expenditures, operational cost expenditures, ODS reductions achieved, project timeliness, and administrative costs.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: The program's annual performance goals are included in managers' performance standards reviews. All managers responsible for implementation of Title 6 of the Clean Air Act have specific performance goals in their standards to which they are held on a continuing basis throughout the year. Contractors are evaluated on an ongoing basIs, and upon completion of a project. The Fund requires submission of a project completion report when each project is completed. The report reviews the performance, cost, and on-time delivery of the agency. In addition, every year the Fund evaluates the status of each project in the agencies' portfolio of projects. Each recipient's performance is measured on the basis of 7-10 performance criteria including the cost of project preparation, cost of tonnage reduced, timeliness of first disbursement on projects and project delivery. This evaluation yields a grade for each funds recipient, which is taken into account in the assignment of future work to each recipient.

Evidence: Examples of performance standards for managers of the Stratospheric Ozone Protection Program.Policies and Guidelines of the Multilateral Fund www.unmfs.org

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: EPA program funds are tracked through EPA's IFMS. Individual expenditures are approved by managers to ensure that they are spent for the intended purpose. Managers also review progress of obligations throughout the year to insure timely, prudent obligation of funds to effective recipients The EPA portion of Multilateral Fund funds are paid for with promissory notes, which allow the Treasury to keep the related funding until it is actually needed for project implementation. The Executive Committee review of projects and their subcomponents ensures that all eligibility criteria are met. Audits under the Fund ensure that funds are spent for their intended use.

Evidence: EPA's Financial Data Warehouse allows program managers to see that grants funds are being expended in a timely manner. Grants have periodic (usually quarterly) progress reports that track progress.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The Stratospheric Ozone Protection Program routinely awards contracts through full and open competition in compliance with EPA rules and procedures. All acquisitions over $2,500 but not exceeding $100,000.00 are competitively awarded unless an exception exists under FAR 13.106-1(b). Procurements undergo a cost analysis that facilitates cost comparisons between contractors and between the contractor costs and independent government cost estimates. Grant funding is also awarded in accordance with the Agency's competitive award policy (7/15/2002). IT improvements will continue to produce efficiencies and achieve cost savings. For example, the Stratospheric Ozone Protection Program saved an estimated $ 200,000/year for each of the past three years by transforming its staffed hotline response method to an automated faxback system. Another high priority IT improvement is EPA's ODS Tracking System, which would reduce burden on the regulated community to report and reduced cost to EPA to evaluate the status and trends of the ODS phaseout.

Evidence: EPA grants competition policy Resource and Office Management Team benchmark presentations

Does the program collaborate and coordinate effectively with related programs?

Explanation: The program regularly collaborates and coordinates effectively with related Federal, State, local, international, and private programs. EPA has worked closely or coordinated efforts with NASA, NOAA, the Department of State, the Department of Agriculture, FDA, DoD, the U.S. Coast Guard, FAA, GSA, as well as international bodies such as the International Maritime Organization (IMO) and the International Commercial Aviation Organization (ICAO) to reduce or eliminate use of ODS and replace them with safe and effective alternatives. EPA continues to work closely with the U.S. Customs, DoJ, and IRS to enforce against illegal imports of halons and CFCs. The effectiveness of EPA's collaborative efforts is demonstrated by the successful elimination of ODS production and adoption of non-ODS alternatives by Federal agencies, continued international progress in meeting Protocol targets, and the granting of the full amount of the US's nominations for essential use exemptions every year since 1996.

Evidence: Interagency Agreements with the Departments of the Army, Navy, and Air Force; Joint Letter from EPA and FAA to the Aerospace Industries Association of America; National Fire Protection Association; (NFPA) 2001 Standard Grant with Halon Alternatives Research Corporation; Draft ISO Standard for Fire Suppressants IMO Maritime Directives for Fire Protection under SOLAS Other examples: Working closely with US Customs, the Department of Justice, and the IRS, over 114 convictions have been made and 2 million pounds of ODS have been seized from joint actions on illegal imports. Working with the General Services Administration (GSA) and the Department of Defense, EPA has adopted regulations that require all federal agencies to conform their procurement regulations to the policies of the Act and maximize the substitution of alternatives for Class I and II substances (40 CFR Part 82 Subpart D)

Does the program use strong financial management practices?

Explanation: The program follows EPA's financial management guidelines for committing, obligating, reprogramming, and reconciling appropriated funds. EPA received an audit opinion on its FY 2002 financial statements and had no material weaknesses associated with the audit. The Stratospheric Ozone Protection Program has no material weaknesses as reported by the Office of the Inspector General. The program has procedures in place to ensure that payments are made for the intended purpose to minimize erroneous payments. Use of EPA's Financial Management Warehouse allows timely, accurate information on expenditures for specific purposes, supporting the program's daily operations.

Evidence: IG's FY 2003 Identification of material weaknesses. EPA's grants office recently audited a random sample of the program's grants and found no problems (most recent audit completed in March, 2004; draft report expected later in 2004).

Has the program taken meaningful steps to address its management deficiencies?

Explanation: The program has a systematic approach for conducting program reviews that address issues in compliance with the Federal Managers' Financial Integrity Act. Program managers regularly review progress on individual projects and program goals to ensure that goals are being met. In addition, contracts and grants are both periodically audited to identify grant and contract management deficiencies. Contract officer representatives must review progress monthly to identify any potential problems and to track spending and progress toward goals. The program contract officer meets multiple times yearly with contractors to identify, discuss, and resolve deficiencies identified by contract officer representatives during monthly reviews. Contract officer representatives rate contractors annually on their performance, and program managers must review and agree to these ratings, allowing further opportunity to identifying problems.

Evidence: The program has implemented recommendations included in reviews by the IG and GAO, and in audits of contracts and grants. For examples, the program introduced competition for grants, improved documentation of cost and budget reviews for grants, and began requiring employee performance standards to address grants management. Similarly, there are now procedures in place for competition for contracts, documentation of cost and budget reviews for grants, requiring employee performance standards to address contracts management, and further documenting issues arising during contract performance. In the summer of 2003, the program's contract officer (CO) expressed concerns regarding contractor financial management of a contract. EPA staff performed an audit in March 2004 and the results were reported via Financial Monitoring Report number FARNSCASSNO 4DFM001. The CO has since established a Plan of Action and Milestones (POAM) which included corrective actions the contractor must take to correct deficiencies.

Did the program seek and take into account the views of all affected parties (e.g., consumers; large and small businesses; State, local and tribal governments; beneficiaries; and the general public) when developing significant regulations?

Explanation: EPA initiated development of The Industry Cooperative for Ozone Layer Protection (ICOLP) to help industries eliminate ODS in their manufacturing processes through an information-exchange of individual industry discoveries and procedures. The program has created workgroups on an ad hoc basis to bring together stakeholders to address specific research, technical, or policy issues.Program staff serve on national and international committees and workgroups where stakeholder views are regularly heard. In developing significant regulations, the program has provided regular informational briefings. SNAP brings together input from a range of stakeholders ' chemical and equipment manufacturers, user groups, trade groups, governmental agencies - to assist in the review of substitutes and help promote the development of innovative technologies. SNAP staff participate in industry conferences and meet regularly with affected businesses so that their concerns are heard directly. EPA representatives serving on the Executive Committee consider the views of affected parties in all Executive Committee decisions.

Evidence: Program rulemaking documents: www.epa.gov/ozone/title6 and www.epa.gov/ozone/snap/regs Regular meetings with with industry associations including the Alliance for Responsible Atmospheric Policy , and many sector-specific groups such as the Association of Home Appliance Manufacturers, the Spray Polyurethane Foam Association, the Polyisocyanurate Manufacturing Association, the Halogenated Solvents Industries Association, the Halon Alternatives Research Corporation, the Crop Protection Coalition, and the National Fire Protection Association (NFPA).

Did the program prepare adequate regulatory impact analyses if required by Executive Order 12866, regulatory flexibility analyses if required by the Regulatory Flexibility Act and SBREFA, and cost-benefit analyses if required under the Unfunded Mandates Reform Act; and did those analyses comply with OMB guidelines?

Explanation: RIAs and EIAs include cost and benefit comparisons, but these analyses do not always comply with OMB guidelines.

Evidence: Program rulemaking documents: www.epa.gov/ozone/title6 and www.epa.gov/ozone/snap/regs

Does the program systematically review its current regulations to ensure consistency among all regulations in accomplishing program goals?

Explanation: Regulations are reviewed and updated where needed to ensure compliance with international obligations and to ensure consistency within the program, but there is currently no plan in place for systematic review of regulations.

Evidence: There is evidence that the program reviews its regulations, and that changes have been made as a result, but the program needs to show evidence of a plan or process to conduct this exercise on a regular basis.

Are the regulations designed to achieve program goals, to the extent practicable, by maximizing the net benefits of its regulatory activity?

Explanation: As part of regulation development, the program assesses (formally or informally) how each additional regulation impacts industry and the general public. Though , there is limited data on costs and benefits, there is evidence that the program designs its regulations to minimize the cost burden to the extent possible within its statutory framework.

Evidence: Program rulemaking documents: www.epa.gov/ozone/title6 and www.epa.gov/ozone/snap/regs

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: The program, including program partners, has demonstrated some progress towards long-term performance goals for the United States. Small extent is given because the program's major goals are extremely long-term (25-160 years) and it is not yet possible to determine whether the program is on track to meet these goals. Also, the program's health effects measure currently uses a counterfactual baseline that would make verification impossible. Converting this measure to a population rate of skin cancer prevalence and establishing an actual baseline will allow the program to demonstrate whether or not it is able to meet this goal. Additional phase-outs in individual sectors and in other countries are necessary to meet the program's health-based long term outcome goals, so it will also be important for the program to show that worldwide reduction targets are likely to be achieved.

Evidence: The National Oceanic and Atmospheric Administration (NOAA) reported that in 1995, the total amount of ozone depleting substances (ODS) in the lower atmosphere declined for the first time. The 2002 WMO Scientific Assessment concluded that concentrations of atmospheric chlorine appear to be stabilizing, and the rate of increase in bromine is slowing. The consensus is that the ozone layer will recover sometime in the middle of this century, but only if there is worldwide compliance with the Protocol. Consumption of HCFCs was reduced from a baseline of 15,240 ODP tons in 1989 to approximately 10,000 ODP tons in 2003. Methyl bromide consumption has been gradually reduced from 25,528 metric tons in 1991 to 6,507 metric tons in 2003.

Does the program (including program partners) achieve its annual performance goals?

Explanation: The program, including program partners, has achieved past annual performance goals. In fact, the program will no longer use a measure representing Class I ODS consumption because the phase-out was successfully completed (not including critical and essential use exemptions approved by the Montreal Protocol Parties). The program appears to be on track to meet its Class II ODS (HCFC) annual measures.

Evidence: 1 Annual Report of Production and Import of ODS to UNEP2 GPRA Performance Report3 2002 Scientific Assessment of Ozone Depletion (WMO)4 EPA Tracking System5 Policies and Guidelines of the Multilateral Fund www.unmfs.orgSee Section 4.1 for a description of reduced chlorine in the atmosphere and expected leveling off of ozone concentrations. Consumption of HCFCs was reduced from a baseline of 15,240 ODP tons in 1989 to approximately 10,000 ODP tons in 2003.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The program shows evidence of cost-effectiveness and managerial efficiencies, but the program is not yet able to make a strong demonstration of improved efficiencies in achieving program goals. Once the new goals have been firmly established and the program has gathered several years of data for its new efficiency measure, it is likely that it will be able to demonstrate tangible efficiency gains.

Evidence: EPA estimates total costs of the program to be $57 billion (1990$, with a 2% discount rate) compared to health and ecological benefits between 1990 and 2165 estimated to be $4,300 billion. Another demonstration of cost-effectiveness is by the growing availability of approved alternatives. In general, the cost of ODS substitutes has been stable or has progressively declined each year since the mid-1990s. By ensuring that alternatives have been made available ahead of the phaseout schedule and at prices comparable to the ODS they are replacing, the program has reduced costs. Efforts to eliminate unnecessary emissions and illegal imports of ODS have also demonstrated increased efficiency. For example, penalties on CFC smuggling operations imposed by US Customs and the Justice Department, as part of EPA's interagency task force have increased substantially each year, without corresponding increases in program resources.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: EPA's Stratospheric Ozone Program is the only governmental or private program in the U.S. designed to eliminate ozone-depleting substances (ODS), so a comparison with otherer similar programs is not applicable.

Evidence: While a comparison with other Federal, State, local, is not applicable here, many of the program's regulatory approaches have been used as models by other countries. EPA's policy approach has also allowed the U.S. to avoid the negative economic and environmental impacts created in other countries which have taken different approaches. For example, whereas EPA's policy approach involves recovery, recycling, and reuse of ODS, some other countries have mandated decommissioning of equipment using ODS, with the unintended negative consequence that end users have released ODS stockpiles, resulting in unnecessary damage to the ozone layer.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: Independent evaluations indicate that the elimination of ozone depleting substances in the U.S. and other developed countries is resulting in a reduction of ozone-depleting compounds in the ozone layer. Recent scientific studies indicate that the concentration of ozone-depleting bromine in the atmosphere has begun declining, and that the level of ozone-depleting chlorine is no longer increasing. Other independent evaluations focusing on the Multilateral Fund, including triennial replenishment evaluations by the Protocols Technology and Economic Assessment Panel, annual country by country reports on compliance, and a 1997 GAO report indicate that the program has been effective in achieving some of its goals. All of these evaluations, however, have not been of sufficient scope or regularity to determine whether EPA is on track to meet health-based outcome goals such as reduced skin cancer prevalence.

Evidence: 1 Office of the Inspector General Audit Report, 1998 2 2002 Scientific Assessment of Ozone Depletion. WMO 3 1997 GAO Report on Multilateral Fund

Were programmatic goals (and benefits) achieved at the least incremental societal cost and did the program maximize net benefits?

Explanation: There is evidence suggesting that programmatic goals were achieved cost-effectively by using creative solutions involving stakeholders. By allowing greater flexibility in ozone depleting substances (ODS) substitutes that may be used and by issuing tradable allowances to producers to obtain emission reductions leading up to a complete phase-out, industry is able to minimize costs of implementation. However, demonstration that the net benefits of the current program are higher than reasonable alternatives is necessary for a Yes. The program needs to show consistent analysis of alternative approaches to demonstrate that its regulatory approach mazimizes net benfits

Evidence: 1 Benefits and Costs of the Clean Air Act 1990 to 20102 EPA Chlorofluorocarbon Taxes and Allowance Trading3 RIA: Protection of Stratospheric Ozone, Vol 1: Regulatory Impact Analysis Document, 1988