State and Local Climate and Energy Program
Determining Results
State Activities
Related Links
- Types of Results
- Importance of Measuring Results
- Key Steps for Measuring Results
- Evaluating Policy Options
- Approaches to Program Evaluation
- Challenges
Types of Results
States consider and implement a variety of clean energy policies and programs using energy efficiency, renewable energy, combined heat and power, and clean distributed generation to meet a range of goals. These policies and programs can offer multiple benefits through their ability to:
- Reduce demand for energy
- Decrease stress on the energy system
- Mitigate climate change, environmental degradation, and related human health concerns
- Promote economic development
Detailed information about the multiple benefits of clean energy is available in Assessing the Multiple Benefits of Clean Energy: A Resource for States.
Importance of Measuring Results
By understanding the extent to which policies and programs achieve various benefits, states develop more comprehensive assessments of their potential clean energy investments and are able to:
- Demonstrate how clean energy policies and programs can help achieve multiple state energy, environmental, and economic benefits in a cost-effective way
- Design or select clean energy options that offer greater energy, environmental, and economic benefits
- Identify opportunities where clean energy can be used to support energy system, environmental, and/or economic development planning strategies across the state
- Build support for clean energy policies and programs
Additionally, measuring results across a spectrum of outcomes provides states with timely information to estimate policy and program impacts and improve implementation. The results can help states answer questions such as:
- Is the policy or program achieving its objectives? If so, how and why?
- How well has the policy or program worked? What is the magnitude of savings?
- How reliable is the policy or program? Will it continue to generate benefits into the future?
- What changes are needed to improve the policy or program?
- Should the policy or program be expanded, adjusted, or cancelled?
By answering these questions, states can identify the most effective approaches, determine how to improve future policies or programs, and decide where to focus for greater impacts.
Communicating results and benefits to key audiences can help states achieve:
- Transparency by documenting progress being made towards goals
- Accountability to funding sources, helping to obtain continued support
- Improvement by providing recommendations for designing better programs
Key Steps for Measuring Results
Four steps are involved in tracking, evaluating, and reporting process for assessing policies and programs:
- Plan – Establish goals, define performance indicators, specify evaluation and reporting approaches, and allocate resources. The level of rigor and analysis required will depend on who needs the information and for what purpose.
- Track/Benchmark – Develop a tracking system, establish baseline reporting conditions and reporting period, and collect and organize performance data.
- Evaluate – Conduct impact, process, and/or market evaluations to measure benefits. States can use available evaluation tools to simplify this process.
- Report – Report evaluation findings, assess results, and modify the program as needed. Information on Engaging Stakeholders is available to assist states.
More details are available in EPA's Model Energy Efficiency Program Impact Evaluation Guide (PDF) (152 pp, 1.44M, About PDF).
Evaluating Policy Options
For clean energy policies, a starting point for screening and evaluating options is to understand the potential energy savings or renewable energy generation output that would be expected to result from the policies. These data can then be used to estimate prospective energy system, environmental, and/or economic impacts (costs and benefits) of the options such as those described in the graphic below.
The tools and approaches available for evaluating the impacts of policies range from basic screening methods to sophisticated dynamic simulation models. In selecting the most appropriate tools or method, states can consider many factors, including:
- Purpose of analysis
- Impact(s) of interest
- Time constraints
- Cost
- Data requirements/availability
- Internal staff expertise
States can either conduct the evaluation themselves or hire consultants. Regardless of who conducts the analysis, states should understand the methods and assumptions used in the analyses as they dramatically affect the results of the analyses.
More details are available in EPA's Assessing the Multiple Benefits of Clean Energy: A Resource for States.
Approaches to Program Evaluation
The terms evaluation, measurement, and verification (EM&V) encompass a wide range of analyses used to assess clean energy programs. EM&V efforts are generally applied to energy efficiency programs administered by utilities, states, and third–party energy efficiency providers, as well as state-mandated renewable energy policies and programs funded by ratepayers.
The goals of EM&V typically include:
- Determining whether overall objectives are being achieved
- Identifying any necessary program improvements
- Assessing program cost–effectiveness
- Estimating impacts and their persistence over time
- Capturing energy (kWh) and demand (kW) impacts in energy planning
Three types of EM&V, each with a different purpose, are frequently undertaken to assess program effectiveness.
| Evaluation Types | Purpose | Information Derived |
|---|---|---|
| Impact Evaluations | Quantifies the direct and indirect benefits of a program or project using measured savings or deemed savings methods. | Determines the quantity of energy and/or demand saved, the monetary value of these savings; can include the amount of emissions reductions and other non-energy benefits. |
| Process Evaluations | Indicates how to improve the structure and delivery of a program or project. These evaluations typically survey program stakeholders, analyze their feedback, and use this information to identify opportunities for program improvement. | Determines how well program or project processes are performing and provides recommendations for how these systems can be improved. |
| Market Effects Evaluations | Indicates how a program affects the structure or functioning of a market — or the behavior of participants in a market — that result from one or more program efforts. | Determines changes that have occurred in state operations and/or private markets, and evaluates how the marketplace is different as a result of the program. |
Challenges
Determining the results of climate and energy programs can be difficult because not everything that a state may want to measure is easily documented. Approaches to overcoming these challenges are presented on the linked pages.
- Calculating Energy Savings presents a challenge of estimating what did happen relative to what “would have” happened.
- When Assessing Air Quality, GHG, and Public Health Benefits, air pollutants can often be measured, but GHGs are more difficult to monitor and therefore must be determined based on emission factors.
- Assessing Electric System Benefits is complex because there are many pieces to an energy system and impacts from relatively minor changes in energy use are difficult to pinpoint.
- When Quantifying Economic Benefits, states must account for the many factors at work in economic systems. Assumptions of various levels of rigor are used, and states must decide what types and portions of economic impacts will be calculated (e.g., environmental externalities, jobs, etc.).