Then Energy Information Administration (EIA) produces projections of energy supply and demand each year in the Annual Energy Outlook (AEO).  The projections in the AEO are not statements of what will happen but of what might happen, given the assumptions and methodologies used. The projections are business-as-usual trend projections, given known technology, technological and demographic trends, and current laws and regulations. Thus, they provide a policy-neutral reference case that can be used to analyze policy initiatives. EIA does not propose or advocate future legislative and regulatory changes. All laws are assumed to remain as currently enacted; however, the impacts of emerging regulatory changes, when defined, are reflected.

The analyses in the AEO focus primarily on a reference case, lower and higher economic growth cases, and lower and higher energy price cases.  In addition, more than 30 alternative cases are generally included in an AEO. Readers are encouraged to review the full range of cases, which address many of the uncertainties inherent in long-term projections.

Each year since 1996, EIA’s Office of Integrated Analysis and Forecasting has produced a comparison between realized energy outcomes and the projections included in previous editions of the AEO. Each year, the comparison adds the projections from the most recent AEO and updates the historical data to the most recently available.  The comparison summarizes the relationship of the AEO reference case projections since 1982 to realized outcomes by calculating the average absolute percent differences for several of the major variables for AEO82 through AEO2005.¹ The average absolute percent difference is the simple mean of the absolute values of the percentage difference between the Reference Case projection and the actual value. The historical data are typically taken from the Annual Energy Review (AER).² The last column of Table 1 provides a summary of the most recent average absolute percent differences for 17 of the most important projection components. The detailed calculation of these differences is shown in Tables 2 through 18. These tables also provide the average absolute difference, which is the simple mean of the absolute value of the difference between the reference case projection and the actual value.

Beginning with last year’s evaluation,³ the basis of measure for comparing dollar denominated values, Gross Domestic Product (GDP), and energy prices was changed.  The GDP series is now evaluated on a nominal dollar basis using the contemporaneous deflator series from each AEO rather than a current deflator series. Each AEO presents a “real” GDP measure and a matched price index.  Prior to last year’s  evaluation, the calculation of nominal GDP was done by applying the latest official historical deflator series to the “real” GDP projection to generate a nominal series for the evaluation. This process has been revised to now use the values of the price index from each AEO document to inflate the “real” numbers to a nominal series. These revisions also change the energy intensity table, which is now presented as energy consumption divided by nominal dollar GDP. These changes improve the consistency of comparing across AEOs in ways discussed in detail below.

As indicated in Table 1, the reference case projections of energy consumption, energy production, and carbon dioxide emissions have been relatively close to realized outcomes, the projections of net energy imports have been moderately close to realized outcomes, and the reference case projections of energy prices have been the furthest from realized outcomes.  Both Table 1 and the individual tables show marked improvement in the absolute average differences for energy prices and net energy imports over time. The calculated absolute average differences can change from one year’s evaluation to the next as an additional year of data and projections are added to the evaluation series and also because of data revisions in the AER and the Monthly Energy Review (MER).

The underlying reasons for deviations between the AEO projections and realized history have tended to be the same from one evaluation to the next. The most significant are:

• Over the last two decades, there have been changes in laws, policies, and regulations that were not assumed in the projections prior to their implementation because of EIA’s statutory requirement to be policy neutral. Many of these actions have had significant impacts on energy supply, demand, and prices. For example, the Powerplant and Industrial Fuel Act (FUA) of 1978 restricted the use of natural gas in power plants and industrial boilers. After FUA was repealed in 1987, use of natural gas for electric generation and industrial processing increased sharply. Consequently, those AEOs completed prior to or immediately after repeal of the FUA, e.g., AEO86, AEO87, and AEO89, had considerably larger underestimation of natural gas consumption in 2000 than AEO90 and all subsequent AEOs.

• AEO85 through AEO90 projections of GDP overestimated GDP growth in the 1990s, whereas later AEOs underestimated GDP growth over that period. The economic slowdown that began in 2001 has led to overestimates of GDP. Because GDP is a good indicator of the economic activity, which drives the energy consumption projections, those projections often have similar difference trends as the GDP projections (Table 17).

• The level of future electricity sales was underestimated for nearly all projection years for the AEO 1991 through AEO 1997 (Table 6). Since about 90 percent of the demand for coal results from electricity generation, the underestimation of electricity sales contributes to underestimation of coal consumption in those years (Table 5). The projection differences of electricity sales were particularly large in AEO94 through AEO96, for these AEOs coal demand exhibits similar patterns of underestimates.

• Overestimation of world oil prices, particularly in publications prior to AEO97 (Table 13), resulted in underestimation of petroleum consumption. For AEO97 forward, prices tended to be underestimated and more neutral consumption projections occurred (Table 3).

• Natural gas generally has been the fuel with the largest differences between reference case projections and realized outcomes. As regulatory reforms that increased the role of competitive markets were implemented beginning in the mid-1980s, the behavior of natural gas in competitive markets was especially difficult to predict. In the earlier projections, EIA’s technology improvement expectations proved conservative, as technological advances made petroleum and natural gas less costly to produce. After natural gas curtailments were eased in the mid-1980s, natural gas was an increasingly attractive fuel source, particularly for electricity generation and industrial uses. Historically, natural gas price instability was strongly influenced by natural gas resource estimates, which steadily rose, and by the world oil price, which was subject to its own projection difference. More recently, the AEO reference case has over-projected natural gas consumption (Table 4) due to natural gas wellhead price projections that proved to be significantly lower than what occurred (Table 14).

• Technological improvements in both the production and use of energy have had a significant impact on the price, supply, and consumption of energy. For the most part, earlier AEOs assumed much slower technology development than actually occurred, accounting for some of the deviation between the projections and history. This trend was recognized, in part, by this type of evaluation exercise. Beginning with the Annual Energy Outlook 1994, the projections in the AEO were produced using the National Energy Modeling System (NEMS). Because NEMS was designed with methodologies to represent technology in a more detailed fashion, there has been an improvement in the capability to represent technological change throughout energy markets. As NEMS has evolved, additional studies on technological improvement have led to more optimistic assumptions in the more recent projections, along with modeling innovations, such as learning-by-doing, in which experience gained with new generation technologies and advanced end-use technologies leads to cost reductions in the model.

• External factors such as severe weather, economic cycles, and other supply disruptions have also had an impact on energy markets; however, these events cannot be anticipated in the mid- to long-term period and are not captured in the models underlying the AEO projections.

Summary of Differences between AEO Reference Cases and Realized Outcomes Printer Friendly Version     All AEOs NEMS AEOs Percent Over-Projected Average Absolute Percent Difference Percent Over-Projected Average Absolute Percent Difference Table 2. Total Energy Consumption, Actual vs. Reference Case 42% 2.2% 49% 2.2% Table 3. Total Petroleum Consumption, Actual vs. Reference Case 31% 2.9% 42% 1.9% Table 4. Total Natural Gas Consumption, Actual vs. Reference Case 32% 6.4% 53% 4.8% Table 5. Total Coal Consumption, Actual vs. Reference Case 39% 3.9% 33% 4.8% Table 6. Total Electricity Sales, Actual vs. Reference Case 33% 2.4% 24% 3.3% Table 7. Total Carbon Dioxide Emissions, Actual vs. Reference Case 25% 3.0% 29% 3.0% Table 8. Domestic Crude Oil Production, Actual vs. Reference Case 48% 4.7% 46% 4.8% Table 9. Natural Gas Production, Actual vs. Reference Case 48% 4.9% 67% 4.7% Table 10. Coal Production, Actual vs. Reference Case 72% 4.2% 60% 4.2% Table 11. Petroleum Net Imports, Actual vs. Reference Case 53% 6.5% 58% 3.5% Table 12. Natural Gas Net Imports, Actual vs. Reference Case 31% 14.9% 47% 8.5% Table 13. Wold Oil Prices, Actual vs. Reference Case 72% 54.2% 41% 17.5% Table 14. Natural Gas Wellhead Prices, Actual vs. Reference Case 66% 64.9% 26% 25.1% Table 15. CoalPrices to Electric Generating Plants, Actual vs. Reference Case 91% 47.0% 78% 17.6% Table 16. Average Electricity Prices, Actual vs. Reference Case 87% 19.5% 69% 9.5% Table 17. Gross Domestic Product, Actual vs. Reference Case 33% 5.4% 15% 4.6% Table 18. Energy Intensity, Actual vs. Reference Case 68% 5.7% 94% 4.8%

The table above provides a summary of the percentage of over-projections as well as the absolute projection differences for the entire Annual Energy Outlook series as well as for just the NEMS AEOs.  Table-by-table, the absolute average differences are smaller in magnitude for the NEMS regime in all but 3 of the 17 tracked concepts.  The percentage of over-projections during the NEMS regime has remained nearly the same as for the entire AEO series.  Over-projections improved (i.e., moved closer to 50%) for only 8 of the 17 concepts tracked in Tables 2 through 18.  As discussed in the "Revisions to Gross Domestic Product and Implications for the Comparisons", the Gross Domestic Product (GDP) concept has changed noticeably over time due to revisions and improvements by the Bureau of Economic Analysis.  These changes distort comparisons of realized GDP outcomes and projections.  Further, since the energy intensity measure has GDP in the denominator, comparisons of realized intensity outcomes and projections are also distorted to an unknown degree.  Consequently, the comparisons for GDP and intensity projections do not track the realized outcomes as well as other concepts.

* Contacts: GDP issues, Ronald.Earley@eia.doe.gov; other issues, Steven.Wade@eia.doe.gov .
1 Note the publication gap in the tables that follow, there was no publication of AEO88.
2 GDP and the GDP price deflators come from the Bureau of Economic Analysis, while coal prices to electric generating plants are from the Monthly Energy Review (MER).
3 Annual Energy Outlook Forecast Evaluation 2004.