Annual
Energy Outlook Forecast Evaluation
by
Susan H. Holte and Eugene J. Reiser
| This paper evaluates
the projections in the Annual Energy Outlook (AEO),1
by comparing the projections from the Annual Energy Outlook 1982 through the Annual Energy
Outlook 1998 with actual historical values and providing the rationale for the
differences. A set of 16 major consumption, production, imports, price, and economic
variables were chosen for evaluation, updating a similar analysis published in the
previous edition of Issues in Midterm Analysis and Forecasting.2
This paper expands on the previous one by adding the most recent AEO to the evaluation,
including 1997 as an additional historical year, adding a comparison of high and low
economic growth cases when available, and including a regression analysis of the
historical data. |
Introduction
Overview
Energy Consumption
Energy Production
Energy Imports and Exports
Energy Prices and Economic Growth
High and Low Economic Growth Cases
Regression Analysis on Historical Data
Conclusion
Appendix A: Total Energy Consumption (TE)
Introduction
This paper presents
an analysis of the forecast record of the Annual Energy Outlook (AEO). It
compares the projections for major energy variables from the reference case for each of
the AEOs published from April 1983 through December 1997 with actual data.3 The purpose of the analysis is to provide a
measure of the accuracy of the forecasts; however, prediction of future energy markets is
not the primary reason for developing and maintaining the models that the Energy
Information Administration (EIA) uses to produce the AEO. Because the EIA models
are developed primarily as tools for policy analysis, a key assumption of the forecasts is
that current laws and regulations will remain in effect throughout the forecast horizon.
This assumption, while necessary to provide a baseline against which changes in policy can
be evaluated, also virtually guarantees that the forecasts will be in error, as laws and
regulations pertinent to energy markets change considerably over the years.
The National Energy
Modeling System (NEMS)—the current EIA model used to produce the midterm projections
in the AEO—and the predecessor models were designed to enforce a discipline on
the process of energy market analysis by providing a comprehensive set of assumptions that
are consistent with our understanding of the factors that affect energy markets—for
example, technological innovation, energy service demand growth, and energy resources. The
models are modified each year to ensure their relevance to evolving energy issues and to
update baseline data, parameters, and assumptions with the most recent historical data.
NEMS, first used for the Annual Energy Outlook 1994 (AEO94),4 was specifically designed for a high level of
technological detail and flexibility to address a wide range of policy options.
These models are
frequently used in studies conducted for the U.S. Congress, the Department of Energy, and
other Government agencies to analyze the impacts of changes in energy policies,
regulations, and other major assumptions on future energy supply, demand, and prices,
typically using assumptions specified by the client. The most recent examples of
analytical studies include an analysis of the Electric System Public Benefits Protection
Act of 19975 at the request of Senator James M.
Jeffords (R-Vt), Chairman of the Senate Committee on Labor and Human Resources; a study of
carbon reduction policies6 for the U.S.
Department of Energy, Office of Policy and International Affairs; a study on the costs and
economic impacts of oil imports7 for the U.S.
General Accounting Office; an analysis for Senator Jeffords on open access regulatory
changes and their impacts on the electricity industry;8
and an analysis of carbon mitigation policies9
prepared for the U.S. Environmental Protection Agency.
Just in the period
analyzed in this paper, many legislative actions and policies have been enacted, including
the National Appliance and Energy Conservation Act of 1987, the Natural Gas Wellhead
Decontrol Act of 1989, the Clean Air Act Amendments of 1990 (CAAA90), the Energy Policy
Act of 1992, the repeal of the Power Plant and Industrial Fuel Use Act of 1978 (FUA), the
North American Free Trade Agreement, the Omnibus Budget Reconciliation Act of 1993, the
Outer Continental Shelf Deep Water Royalty Relief Act of 1995, the Tax Payer Relief Act of
1997, the Climate Change Action Plan developed by the Clinton Administration in 1993 to
achieve stabilization of greenhouse gas emissions, and various orders issued by the
Federal Energy Regulatory Commission (FERC). Examples of FERC orders include Order 636,
which restructured interstate natural gas pipeline companies and required the separation
of sales and transportation functions, and Orders 888 and 889, which provided open access
to interstate electricity transmission lines. These actions have had significant impacts
on energy supply, demand, and prices, but because of the assumption on current laws and
regulations, the impacts were not incorporated in the AEO projections until their
enactment or effective dates.
In several cases,
EIA’s models have been used to evaluate some of the potential impacts of these
changes in laws and regulations before they were enacted, thus fulfilling EIA’s
designated role in policy analysis. For example, EIA provided comprehensive analysis to
the House Energy and Commerce Committee concerning the impacts of the CAAA90 on the coal
and electricity industries. In other cases, the models have been used to analyze policies
that were eventually rejected; a prime example is the British thermal unit (Btu) tax
proposed in early 1993. Both of these uses of the models illustrate the importance of
maintaining a modeling capability apart from the forecasting function, using current laws
and regulations as a baseline assumption.
In addition to
changes in laws and regulations, a number of other factors can cause energy markets to
deviate from the longer term trends represented by the forecasts in the AEO. For
example, the forecasts assume normal weather patterns; however, the weather will rarely,
if ever, be normal in any given year. Although the AEO models have not generally
been used for analysis of weather conditions on energy markets, temperatures that are
colder or warmer than normal for sustained periods have a significant impact on energy
consumption. Strikes and political incidents, such as the Iraqi invasion of Kuwait in
1990, are other unanticipated events whose impacts on energy markets are not captured in a
mid- to long-term energy projection. Any of these events can cause price volatility and
fluctuations in energy consumption and supply. EIA’s Short-Term Energy Outlook
(STEO)10 reflects the impacts of these
events and the near-term adjustments to them, and each AEO adjusts its near-term
forecasts to the most recent STEO projections. By presenting quarterly projections
and accounting for stock fluctuations and other short-term adjustments, the STEO is
more applicable to the analysis of such events than is the AEO, which presents
annual average projections.
Although the primary
purpose of the models is policy analysis, many users of the AEO view the
projections as forecasts. Thus, analyzing the models’ performance and the reasons for
differences between the projections and history is important both for users and for those
responsible for the projections. The models and assumptions used in the AEOs
undergo continuous evaluation and change, in part because of changes in energy markets and
in part as a result of internal assessment of the models’ performance. Natural gas
markets are an example of both points. The representation of natural gas markets has been
revised significantly to reflect deregulation. In addition, the fundamental assumptions
about the size and potential growth of natural gas resources have been revised because
evaluations of past forecasts have shown that price projections for gas were too high.
This paper presents
projections for each AEO from 1982 to 1998.11
The forecast horizon has expanded over the period examined in this paper; for example, the
Annual Energy Outlook 1982 (AEO82)12
projections of energy markets extended only through 1990. Also, although year-by-year
forecasts were produced for each AEO, many AEOs published only selected
years. This evaluation includes all projected years, including unpublished projections
where available. A set of 16 key energy variables is used to provide a comprehensive
picture of the projections. The projections in this analysis were produced by the models
in use at the time. Before 1994, the Intermediate Future Forecasting System was the
primary model for midterm projections; however, this evaluation is not meant to assess a
specific model but rather to assess the forecasts and the underlying assumptions that
shape the results. An evaluation of models is inappropriate at this point, because
NEMS—a longer run model—was first used for the 1994 forecasts, and historical
data for comparison are available only for four short-term years. In this case, the best
effort is to compare the NEMS results with forecasts from other organizations, as is done
in each AEO.
Overview
Table 1 provides a summary
of the average absolute forecast errors,13
expressed as percentage differences from actual, for each of the major variables included
in this analysis.14 As the table indicates, the
forecasts of consumption, production, and economic variables have generally been the most
accurate; net import projections have been less accurate; and the price projections15 have been the least accurate when evaluated on
the basis of average absolute percent errors.
Table 1.
Average Absolute Percent Errors for AEO Forecasts, 1982-1998
Variable |
Average
Absolute Percent Error |
Consumption
|
|
Total
Energy Consumption |
1.7 |
Total
Petroleum Consumption |
2.9 |
Total
Natural Gas Consumption |
5.7 |
Total
Coal Consumption |
3.0 |
Total
Electricity Sales |
1.7 |
Production
|
|
Crude
Oil Production |
4.3 |
Natural
Gas Production |
4.8 |
Coal
Production |
3.6 |
Imports
and Exports |
|
Net
Petroleum Imports |
9.5 |
Net
Natural Gas Imports |
16.7 |
Net
Coal Exports |
22.8 |
Prices
and Economic Variables |
|
World
Oil Prices |
51.3 |
Natural
Gas Wellhead Prices |
72.1 |
Coal
Prices to Electric Utilities |
35.3 |
Average
Electricity Prices |
11.0 |
Gross
Domestic Product |
5.0 |
AEO = Annual Energy Outlook.
Source: Tables 2 through 17. |
Each of the
consumption, production, and economic variables has been projected with an average
absolute percent error of 5.7 percent or less. For both total energy consumption and total
electricity sales, the most accurately projected variables during this period, the average
absolute percent error is 1.7 percent. Average absolute percent errors for net imports
range from 9.5 percent for petroleum to 22.8 percent for coal. For prices, forecasting has
proven to be a much greater challenge. Average absolute percent errors for the world oil
price, the price of coal to electric utilities, and the average natural gas wellhead price
range from 35.3 to 72.1 percent over the period, with natural gas wellhead prices proving
to have the highest error of the variables evaluated. Average electricity price
projections, however, fared better, with an 11.0-percent average absolute percent error.
The following
sections discuss the underlying results in some detail; however, it is clear that
quantities are more amenable to the forecasting methods used in the AEO than are
prices; that the errors in forecasting prices have not, in general, affected the accuracy
of projected quantities; and that natural gas has tended to have the highest average
forecast error within most categories—consumption, production, and prices. Some of
the major factors leading to inaccurate forecasts include the assumption in the earlier AEOs
that the Organization of Petroleum Exporting Countries (OPEC) cartel would maintain the
market power and cohesiveness to set world oil prices; the decline of oil production in
the former Soviet Union; underestimates of the impact of technology improvements on the
production and prices of oil, natural gas, and coal; the impacts of changes in laws and
regulations on natural gas prices; the treatment of fuel supply contract provisions for
natural gas and coal as fixed and binding; and other events that have caused the actual
trends to differ from projected long-term trends, as discussed above.
Energy
Consumption
Total Energy
Consumption
Total energy
consumption forecasts have shown a generally good track record for most of the AEO
publications.16 The overall average absolute
percent error for the period examined here is 1.7 percent (Table 2),
with the largest errors occurring in forecasts for the year 1996 (3.0 percent), and the
smallest errors in forecasts for 1991 (0.9 percent).
In terms of the AEO
publications, the Annual Energy Outlook 1986 (AEO86)17
had the largest absolute and average absolute percent errors for total energy consumption,
at 3.0 quadrillion Btu and 3.4 percent, respectively. There was a significant
underestimate of energy consumption for most of the projected years in AEO86, in
part due to the high fossil fuel prices projected for the publication, which was completed
prior to the 1986 collapse in oil prices and published early in 1987. After AEO86,
there was general improvement in the forecast record, as EIA’s experience with lower
priced energy markets expanded. It is worth noting, however, that the overall average
absolute percent errors for oil price forecasts in AEO86 were better than in the
preceding AEOs. Price forecasts for some years in AEO86 were also better
than in some subsequent AEOs; for example, some of the subsequent AEOs
projected world oil prices that were too low for the years 1989 and 1990, and the Annual
Energy Outlook 1991 (AEO91)18
projected much higher prices for 1991 and 1992.
One of the aspects of
modeling energy consumption that is important in the evaluation of the forecasts is the
effect of regulations such as appliance and automobile efficiency standards. When such
standards are incorporated, some decisions that would otherwise be made by the interaction
of supply and demand factors are in fact set by fiat, helping to reduce some of the
uncertainty associated with the forecasts and reducing at least one source of forecast
error.
Total Petroleum
Consumption
Total petroleum
consumption forecasts have an average absolute percent error of 2.9 percent during the
period covered in this evaluation (Table 3). The least accurate
forecast year was 1988, for which the AEOs averaged about 0.75 million barrels per
day lower than the actual consumption of 17.3 million barrels per day. For 1988, the
forecasts of the world oil price were also consistently too high, as noted later, with an
average absolute percent error of 80.9 percent, the highest error for any year other than
1986 and 1995. As described in the section on world oil prices, the early AEO world
oil price projections were influenced by the notion that OPEC could curtail production
sufficiently and hold prices up throughout the forecast horizon. This led to extremely
high forecasts for 1995 in the early AEOs, like AEO83 and AEO84. In
addition, the forecasts of economic growth in 1988 tended to be too low in most of the AEO
publications, which would also lead to an underestimate of demand.
AEO82, the
earliest publication considered in this analysis,19
and AEO86 had the highest average absolute percent errors for petroleum consumption
at 5.3 and 5.7 percent, respectively. Projections of petroleum consumption were
underestimated for all years in AEO86, which was the last AEO completed
before the oil price collapse. The projections for the years 1985 through 1987 in AEO82
were above actual demand; however, the errors for 1988 through 1990 were much smaller and
in the opposite direction.
The AEO82
forecast for the year 1985 had the highest percent error of all the petroleum forecasts
evaluated. Residential and commercial consumption was projected to be more than 0.4
million barrels per day higher in 1985 than it actually was, and consumption of petroleum
for electricity generation was projected to be more than 1.8 million barrels per day
higher in 1985, more than triple the actual value. Both numbers were reduced in the Annual
Energy Outlook 1983 (AEO83)20 and
were considerably more accurate. Although the AEO82 total petroleum consumption
projection for 1990 was equal to the historical value at 16.99 million barrels per day,
the sectoral projections were not accurate. Residential and commercial demand was
projected to be about 0.6 million barrels per day higher, industrial 1.0 million barrels
per day higher, transportation 2.5 million barrels per day lower, and electricity
generation 1.2 million barrels per day higher than actual. Between AEO82 and AEO83,
the role of natural gas had been reevaluated, giving it a larger role in the residential
and commercial sectors and, in particular, in the electricity sector. The projections for
oil demand in these sectors declined between AEO82 and AEO83, and those for
natural gas demand increased.
Following AEO82,
the projections of residential and commercial oil consumption remained rather close to the
actual values, although the slight downturn in 1990 was missed. A general characterization
of the forecasts is a tendency to underestimate energy consumption for several years after
the Annual Energy Outlook 1984 (AEO84).21
At that time, there was an assumption that residential and commercial customers would
purchase the most energy-efficient technologies, an assumption that led to overly
optimistic expectations of efficiency improvements. The Annual Energy Outlook 1985
(AEO85)22 shows this impact in the
residential and commercial sectors.
In the early
forecasts, industrial consumption of oil was overestimated, partially reflecting somewhat
optimistic assumptions about the growth of energy-intensive industries but also due to an
underestimation of the potential growth of natural gas in an era of high gas prices. Later
projections were somewhat underestimated due to assumptions of higher efficiency gains.
Through many of the
forecasts, transportation consumption was significantly underestimated. The projected
world oil prices were too high; and, in reaction to the higher prices, estimated vehicle
efficiency improvements were too high and vehicle miles traveled too low, leading to
transportation demand forecasts that were up to 2.5 million barrels per day too low in AEO82
and frequently up to 1 million barrels per day too low in the next several AEOs.
These forecasts improved significantly in the Annual Energy Outlook 1987 (AEO87),23 which contained the first set of projections
after the oil price collapse in 1986.
Total Natural Gas
Consumption
The average absolute
percent error for natural gas consumption forecasts for this period is 5.7 percent (Table
4). Projections for 1995 had the highest average absolute percent error at 9.2 percent.
For 1995, all the AEOs underestimated consumption by anywhere from 1 to 22 percent,
primarily due to high natural gas price projections. For many of the statistics presented
in this paper, 1995 through 1997 show some of the highest percent errors, because these
years have many of the oldest projections, which were made 10 to 12 years earlier.
Particularly in the natural gas industry, there were significant changes in energy markets
throughout the 1980s. Natural gas price forecasts were very high, as discussed later, and
were important causes for the underestimation of consumption in many years in the analysis
period, as prices were overstated considerably in comparison with the actual prices.
The FUA also
contributed to low estimates of gas consumption by industrial customers. In reaction to a
perceived scarcity of natural gas, the FUA legislation attempted to restrict gas use by
large electric utility and industrial customers. Because of the number of exemptions
granted to electric utilities; however, the FUA had little impact on the forecasts of gas
consumption by utilities, except in AEO82. The legislation did have some
restraining influence on industrial gas consumption forecasts until its repeal in 1987.
With the exceptions
of the projections for 1985 through 1988 made in AEO83 through AEO85,
natural gas consumption was generally underestimated, concurrent with high price
projections. Where consumption was overestimated, the tendency to conservation and the
impact of higher prices on demand were not fully captured, even though prices were
generally overestimated as well. Before 1995, 1986 was the year with the highest average
absolute percent error, at 7.0 percent. Except for AEO82, all the errors for 1986
were overestimates. Although natural gas price projections for 1986 were high, oil price
projections were also high, and fuel switching from oil to gas was projected.
Among the AEOs,
overall average absolute percent errors ranged from 1.1 to 9.5 percent, excepting the Annual
Energy Outlook 1998 (AEO98),24 which
included a single estimate of the most recent historical year, with a 0.7-percent error. AEO86
and AEO87 had the highest average absolute percent errors, mainly because of
underestimates of natural gas use in the industrial sector, although projections for the
residential and commercial sectors were also low in the later years. Projections in the
1980s underestimated natural gas consumption for most years, particularly the later years
in the horizon, with high price forecasts contributing to the errors. Consumption
forecasts improved considerably starting with the Annual Energy Outlook 1990 (AEO90),25 with average absolute percent errors of 4.1
percent or less. Natural gas price forecasts improved starting with AEO91, with
average absolute percent errors no more than 20.1 percent.
Total Coal
Consumption
The forecasts for
coal consumption have been stable and displayed fairly low average errors, in part due to
the good record in forecasting electricity sales, for which coal is a major fuel. The
average absolute percent error for coal consumption is 3.0 percent (Table
5). As has generally been the case, forecasts for the years 1995, 1996, and 1997 tend
to have the highest errors, averaging 4.4, 5.0, and 5.3 percent, respectively. There was a
strong tendency to overestimate in the earlier AEOs, particularly AEO84,
whose forecast for 1995 was 15.4 percent over actual consumption. Factors contributing to
the overestimate included a 5.6-percent overestimate for electricity sales, an estimate of
efficiency that was about 5 percent too low for coal-fired generating units, and a share
for coal in generation that did not account for the eventual greater role of natural gas,
particularly among nonutility electricity producers. The shares of coal and natural gas in
the industrial sector were similarly affected, with high natural gas price forecasts and
an overly optimistic view of the future of metallurgical coal in steelmaking being the
primary factors.
Until the later AEOs,
AEO84 had the highest average absolute percent error for coal consumption at 5.4
percent, because of the high 1995 projection. Following an increase in natural gas prices
in 1996 and 1997, coupled with declining coal prices, there was a drop in gas consumption
by electricity generators and a notable surge in coal consumption by generators in 1996
and 1997, which caused some of the larger errors for those years in most AEOs.
Consequently, the Annual Energy Outlook 1996 (AEO96)26 and Annual Energy Outlook 1997 (AEO97)27 have average absolute percent errors of 5.4 and
5.7, respectively.
Total Electricity
Sales
Electricity sales
have an average absolute percent error of 1.7 over the period studied (Table
6); 1996 is the year with the highest average absolute percent error of 2.5 percent.
Electricity sales for all years were overestimated in AEO82, and, with the
exception of AEO87, AEO85 through AEO90 tended to underestimate the
earlier years and overestimate the later years. In earlier AEOs, overestimates
tended to occur because of strong growth in electricity demand in the industrial sector
resulting from high projections of oil and gas prices and strong growth in consumption in
the sector in general. This growth projection was moderated in later forecasts, which
incorporated energy efficiency gains and structural shifts in the industrial sector to
less energy-intensive industries.
In the forecasts
since AEO91, electricity sales have been underestimated in most years, primarily as
a result of optimistic estimates of efficiency improvements, coupled with continued growth
in new uses for electricity that was not captured in the projections. In addition,
electricity price forecasts have tended to be overstated in most years, largely due to the
influence of overstated natural gas and coal prices to electricity producers, as discussed
later.
In terms of the AEO
publications, the highest average absolute percent error was that of AEO82, at 2.7
percent, as the models used in that AEO continued to anticipate electricity growth
at a pace near that of economic growth, a ratio that has actually been reduced
considerably in this decade. The error in electricity sales was more than halved in AEO83.
Energy
Production
Crude Oil
Production
Crude oil production
forecasts have an overall average absolute percent error of 4.3 percent over the period
evaluated (Table 7). The largest error for any year was 1989, with
an average absolute percent error of 7.8 percent and all AEOs overestimating actual
production for that year. Since domestic oil production is assumed to be determined by
prices rather than demand, an important input to production forecasts is the world oil
price, which has also been overestimated for most years, particularly in the AEO82
through AEO85 projections. For 1989, the first four AEOs had significantly
high world oil price projections, leading to high production forecasts. Following AEO85,
EIA’s price forecasts were either very close to, or significantly under, the actual
1989 price, with a consequent improvement in production projections.
Each of the AEOs
has had average absolute percent errors for crude oil production of 7.2 percent or lower,
with the exception of AEO83, which had an average absolute percent error of 10.2
percent. AEO83 overestimated crude oil production for all years after 1985, with
particularly large errors for 1989, 1990, and 1995, the latter of which was 23.6 percent,
primarily because of high price forecasts.
Following the oil
price collapse of 1986, there were more underestimations than overestimates of crude oil
production. As price projections have been reduced over time, the forecasts have captured
the impacts of technological improvements in the oil industry, preventing the production
forecasts from falling as precipitously as the price projections.
Natural Gas
Production
The overall average
absolute percent error for natural gas production forecasts is 4.8 percent (Table 8), lower than the 5.7-percent average absolute percent error
for consumption forecasts. Unlike crude oil, most demand for natural gas is met by
domestic production; thus, natural gas production tends to follow the projections for
consumption. Forecasts for 1994 display the highest average absolute percent error, at 6.8
percent, followed by 1995 at 6.5 percent. The highest error for 1995, and for all the
production forecasts, occurred in AEO83, the first AEO to project 1995
production. Despite a very high price forecast, the AEO83 production projection was
about 20 percent below the 1995 actual production, reflecting the low demand projection.
AEO82
underestimated gas production in all years and had an 11.7-percent average absolute
percent error, followed by AEO87 at 7.7 percent; for all the other AEOs the
average error rate has been 6.4 percent (for AEO86) or less. The errors in
production forecasts have resulted primarily from the low consumption forecasts, due to
high price forecasts. In general, the AEOs have understated production, with the
exception of the years prior to 1990 in AEO84 and AEO85, and most of the
errors have been similar to those for the forecasts of natural gas consumption.
The difficulty of
predicting technological improvement in the industry—and, consequently, of predicting
the amount of gas that would be available at a given price—led to the high price and
low production forecasts in the earlier AEOs. Following the gas shortages of the
late 1970s and the low resource estimates by most geologists, the conventional wisdom of
the early to mid-1980s was that natural gas was a scarce resource. This perception changed
as the impact of price controls that had curtailed production began to diminish. Also,
beginning in the mid-1980s, a number of technological advances, such as directional
drilling, 3-D seismic imaging, and slim-hole drilling, lowered the cost of gas exploration
and production and expanded the estimates of the resource base. Beginning with AEO90,
the forecasts of both production and price improved.
Coal Production
Similar to coal
consumption, coal production forecasts have an overall average absolute percent error of
3.6 percent (Table 9). Like those for natural gas, the forecasts
for coal production have generally followed the consumption forecasts, with electricity
sales being the dominant factor. However, an additional input is the level of coal
exports, which also affects coal production significantly. Where coal production has been
overestimated, a large part of the reason has been an overstating of the level of coal
exports, especially for the years 1993 through 1995, as discussed below.
The year 1993 shows
the highest average absolute percent error for coal production, at 9.7 percent. In 1993,
there was a strike by coal miners that sharply curtailed production. Consequently, all AEOs
produced before the strike show high forecast errors for 1993. The second highest average
absolute percent error is for 1995, at 5.7 percent. The forecasts for 1995 in AEO83
through AEO86 range from 8.0 to 18.2 percent above the actual 1995 level, although
later forecasts show errors of 5 percent or less. This reflects the overestimation of coal
consumption, particularly in AEO83 and AEO84, and the higher-than-realized
coal export projections in AEO83 through AEO86, discussed below. The
forecasts for other years average much closer to the actual values, with average absolute
percent errors ranging from 1.3 to 3.8 percent. The AEO publications display little
variation in their overall average errors, with AEO84 showing the highest average
absolute percent error of 5 percent, mainly because of its very high projection for 1995.
Energy
Imports and Exports
While the United
States is a major importer of petroleum, it also imports natural gas, although in much
smaller quantities. Coal is the only fuel for which the United States is a net exporter.
Net Petroleum
Imports
Because domestic
production of petroleum is insufficient to meet demand, imports make up the difference
between demand and supply.28 The average
absolute percent error for net petroleum imports over the period studied was 9.5 percent (Table 10). The forecast year with the highest average absolute
percent error proved to be 1985, for which the AEOs averaged a 28.1-percent error;
subsequent years showed considerable improvement. In general, there was a tendency to
underestimate imports for the mid-1980s, because of underestimates of consumption and
overestimates of production. Except for AEO83 and AEO85, this tendency was
generally reversed in projections of the 1990s, with significant overestimates of net
petroleum imports for many years in AEO84 through the Annual Energy Outlook 1995
(AEO95).29 Although in some AEOs
this corresponded to overestimates of consumption and/or underestimates of production, it
was also exacerbated by the contribution of inaccurate forecasts for other sources of
supply, such as natural gas liquids and processing gain, the treatment of stocks, and
assumptions about the pace of acquisition of crude oil for the Strategic Petroleum
Reserve.
By publication, the AEOs
for 1982 through 1985, 1987, 1989, and 1994 proved to have the highest average absolute
percent errors for forecasts of net petroleum imports. AEO82 strongly overestimated
imports for 1985 through 1987; however, its forecasts for the subsequent years were
markedly better. Because high estimates of oil prices led to high production forecasts, AEO83,
AEO84, and AEO85 strongly underestimated imports in many years, as did AEO86
for the late 1980s. Later reports tended to overestimate imports due to underestimates of
production.
Net Natural Gas
Imports
Net natural gas
imports play a small, but important, supplementary role in meeting natural gas demand. The
overall average absolute percent error for the period covered in this study is 16.7
percent, with the largest average absolute percent error for the year 1986 at 49.2 percent
(Table 11). All the forecasts for 1986 were overstated, with
errors as high as 72.7 percent (AEO82). There was a substantial oil price collapse
in 1986, and petroleum imports displaced other energy sources, such as Canadian gas, for
much of the Nation’s consumption needs, especially in the industrial and electricity
generation sectors. Forecasts for 1987 were overstated in the first four AEOs, but AEO86
and AEO87 reversed the pattern with underestimates. AEO85 also showed high
overestimates through 1992 and underestimates for later years. Most AEOs tended to
underestimate imports, with errors as high as 54.2 percent for 1995 in AEO83.
The major determining
factors of natural gas imports have been the economics of natural gas trade with Canada,
the assumptions of pipeline capacity from Canada, the assessment of liquefied natural gas
imports from Algeria, and prospects for trade with Mexico and Japan. The tendency was for
net gas imports to be overstated for the first four AEOs, except for the 1989,
1990, and 1993 through 1995 forecasts. Since the AEO86 forecast, there has been a
greater tendency to underestimate gas imports. Since the Annual Energy Outlook 1993
(AEO93),30 the projections have been much
closer to the actual values, with average absolute percent errors of 5.6 percent or less,
although the AEO98 projection for 1997 reflects an historical update.
Net Coal Exports
The absolute percent
errors in projections for net coal exports have averaged 22.8 percent over the period of
this study (Table 12). The forecast year 1994 had the highest
average absolute percent error at 48.1 percent, followed by 1993 at 39.9 percent. All the AEOs
except AEO95 overstated 1994 coal exports by anywhere from about 30 to 77 percent.
For AEO84 through AEO94, coal exports were generally underestimated through
1992 and overestimated in later years. AEO95 and AEO96 underestimated
exports by a range of 8 to 19 percent.
AEO82
overestimated future coal exports with an average absolute percent error of 37.5 percent,
due largely to the assumption that U.S. coal exports would garner an ever-increasing share
of world coal trade, which was also expected to grow in reaction to high world oil prices.
AEO83, in contrast, had a much more realistic view of future coal exports and, with
the exception of 1995, had much smaller errors. AEO83, AEO96, AEO97,
and AEO98 were the closest of all the AEOs with respect to projected coal
exports. Projections for 1993 through 1997 in AEO91 through AEO94 were far
too high, in part because of the 1993 coal miners’ strike that reduced this
country’s competitive position in world coal markets. In addition, world coal trade
has not grown as much as previously assumed, because European consumers have turned
increasingly to natural gas for industry and power generation, and environmental concerns
have led some countries to reduce coal consumption as a means of reducing carbon
emissions. AEO95 and AEO96 appear to be overcompensating for this trend. AEO98
reflects historical data for 1997.
Energy
Prices
and Economic Growth31
World Oil Prices
World oil prices have
the second highest average absolute percent errors of all the variables evaluated in this
paper, with natural gas prices at the wellhead having the highest. Overall, the average
absolute percent error for world oil price forecasts has been 51.3 percent (Table 13). However, the earlier AEOs had a much higher
average absolute percent error, and the publications after AEO86 show considerable
improvement, with the exception of AEO91, which was affected by the Iraqi invasion
of Kuwait. AEO91, prepared during the short-term escalation of oil prices caused by
the invasion, projected continually rising prices. In fact, oil prices declined over each
of the next 4 years. Similarly, the year with the highest average absolute percent error
was 1995, followed closely by 1986, with very high percentage errors in the earliest AEOs
only partially offset by smaller errors in the more recent forecasts. In nominal terms,
the first forecast for 1995, from AEO83, was nearly $75 per barrel, compared with
the actual 1995 price of $17.14 per barrel.
For many of the
variables examined in this paper, the highest average errors are seen for the year 1995.
As mentioned before, the 1995 projections include those made furthest in the past—up
to 12 years earlier. In addition, projections for 1991 through 1994 are not available from
the earliest publications, so that 1995 appears to be more of an outlier.
Although the
forecasts of world oil prices appearing in the earlier AEOs were almost uniformly
too high, from AEO86 on there were several instances of forecasts that were too
low. These included the 1987 and 1990 forecasts appearing in AEO86 and AEO87,
the forecasts for 1989 through 1991 appearing in the Annual Energy Outlook 1989 (AEO89)32 and AEO90, and the most recent forecasts
for 1996. Clearly, following the oil price collapse of 1986, EIA’s forecasts were
significantly reduced; as a consequence, the projections for 1990 tended to be too low, in
part because of the rise in oil prices beginning in August 1990 associated with
Iraq’s invasion of Kuwait. Even with the lower price forecasts, 1995 had high
percentage errors until AEO94, as most AEOs continued to show rising prices
in response to perceived rising world oil demand.
The early AEO
projections were strongly influenced by the notion that OPEC would continue to hold a
large measure of power in world oil markets. Conventional wisdom in the early projections
assumed that OPEC would be able to curtail production sufficiently to hold prices up, and
that the cartel’s members would continue their cooperation throughout the forecast
horizon. Even as it became clear that OPEC’s cohesiveness was not permanent, EIA
continued to assume that oil prices would rise with increasing demand, although at a much
slower rate of growth than in the 1970s. Increasing investment in areas outside OPEC and
technological advances in oil exploration and production have contributed to the growth in
oil reserves and production capacity of non-OPEC producers. These trends, combined with
competition from natural gas and energy conservation, have kept prices lower than expected
in the earlier forecasts.
Natural Gas
Prices
Natural gas prices at
the wellhead have had the highest average absolute percentage forecast errors in the AEOs,
with an overall average error of 72.1 percent (Table 14).
Occasionally, near-term gas prices have been underestimated, but most of the projections
were overestimates. Similar to the forecasts for world oil prices, those for natural gas
prices were highest in the earlier AEOs, when the projections for all prices were
influenced by the assumption that market forces would tend to increase demand for, and
therefore prices of, natural gas and coal in response to higher world oil prices.
The year 1995 had the
highest average absolute percent error; with the exception of AEO96, which was
essentially estimating the recent historical year for 1995, the smallest error for 1995
was 28.6 percent in AEO95. The year with the lowest average absolute percent error
was 1985, with an average absolute error for four AEOs of 23.3 percent, even
including the 65.2-percent error in the AEO82 projection for 1985. Despite the
large errors, the forecasts in each subsequent AEO have tended to show considerable
improvement, as the downward trend in gas prices has been better captured from one AEO
to another.
Nevertheless, each AEO
has tended to predict rising prices over time, either because of the assumption in the
earlier AEOs that long-term, high-priced contracts would continue or because the
depletion effects associated with rising consumption were expected to overcome
technological improvement in the more recent forecasts. In summary, three factors have had
significant impacts on the projections:
In the earlier AEOs, it was
assumed that natural gas contracts whose provisions were governed by the Natural Gas
Policy Act of 1978 would not be abrogated and that the prices that prevailed under those
contracts would essentially set the market price over time. In fact, when oil prices fell
in 1986, many of those contracts were abrogated, and the price of natural gas fell,
although not as much as the price of oil.
Estimates of the recoverable
resource base rose and estimates of exploration and production costs fell over time, in
contrast to the assumptions in the earlier forecasts. Because the models use this
information as an input, higher assumed levels of recoverable resources and lower assumed
costs would have resulted in forecasts characterized by more gas available for production
at lower prices. More recent AEOs have allowed for increases in the resource base
and decreases in costs due to technology improvements.
Consistent with the assumption of
existing regulations, the earlier AEOs did not assume that there would be
additional competition in the transmission and distribution sectors of the market;
however, from 1985 on, FERC moved to open access to the interstate pipeline transmission
system, lowering end-use prices and stimulating additional price competition at the
wellhead as well.
Thus, although the
forecasts have improved with additional information, they have continued to be affected by
the impacts of wellhead price deregulation and the changing competitive structure of the
industry and by overestimates of the impacts of reserve depletion relative to technology
improvements.
It is worth noting
that approximately one-fourth of the domestic production of natural gas is as a coproduct
of the crude oil extraction process, which means that, as crude oil production rises with
higher oil prices, there may be a depressing effect on the wellhead price of gas. This
effect has added to the complexity of forecasting natural gas prices.
Coal Prices to
Electric Utilities
Although they are
better than those for oil and gas prices, the AEO forecasts of coal prices to
electric utilities still show an average absolute percent error of 35.3 percent over the
period studied (Table 15). All forecasts were overstated. The
forecasts for 1995 had the highest average absolute percent error of 57.5 percent. There
was, however, significant improvement in the 1995 forecast over time, with the error
improving from 137.9 percent in AEO83 to 10.6 percent in AEO95 (excluding AEO96,
which provided an estimate for the historical year 1995 based on partial year data).
Across forecast years, the further out the forecast, the higher the error, with the lowest
average absolute percent error shown for the year 1985 at 13.3 percent.
The early AEOs—AEO82
through AEO86—tended to have the highest average absolute percent errors,
exacerbated by their forecasts for 1995. There was steady improvement in the AEOs
through AEO90, which had an average absolute percent error of 16.8. After AEO90,
overestimates for 1995 through 1997 adversely affected the overall average errors for a
number of the subsequent AEOs.
The major factors in
the high forecasts of coal prices were assumptions about depletion effects, productivity
improvements, capacity utilization, transportation, and the impacts of CAAA90. Depletion
was assumed to overcome productivity improvements in the long run; however, the onset of
such new technology as longwall mines and the growth of surface mining in the West have
led to continuing productivity improvements. Similarly, with high world oil price
forecasts, the impacts of excess capacity and competition among existing mines were not
seen to be as important as they in fact became. In addition, high world oil prices were
assumed to affect both the production process and the costs of transportation. In fact,
the collapse of oil prices in 1986 reduced the impact on both, and the increasing
competitiveness of rail transportation has held transportation costs below expectations.
Finally, it was assumed that high prices would follow the enactment of CAAA90 as the
demand for low-sulfur coal increased. Price increases did not materialize, however, as
productivity increases and transportation cost reductions made increased production from
western mines possible at lower-than-anticipated prices.
Average
Electricity Prices
Average electricity
prices showed the best forecasting record among the prices examined here, with an average
absolute percent error of 11.0 percent (Table 16). As with all
the price forecasts, because of the projections made 12 years earlier, the year with the
highest average absolute percent error was 1995, which had an average error of 15.5
percent. Except for the two near-term forecasts of 1985 for AEO82 and 1989 for AEO90,
price forecasts have been higher than actual. By publication, AEO83 had the highest
average absolute error of 18.2 percent, and AEO97 had the lowest at 3.3 percent
(with the exception of the AEO98 estimate of the most recent historical year of
1997 based on partial year data). Recent AEOs, from the Annual Energy Outlook
1992 (AEO92)33 on, have had average
absolute percent errors of 9.4 percent or less.
The primary reason
for high price forecasts was the impact of fuel costs and capital costs on expected
prices. Fuel costs were consistently overestimated for oil, natural gas, and coal, with a
strong effect on the estimates of electricity prices, especially for AEO82 through AEO84.
In addition, the costs of new capacity were assumed to be higher in earlier projections
than they actually turned out to be, and this assumption also helped to raise the
forecasts. Finally, a 1992 study34 on the
accuracy of AEO electricity forecasts for 1985 and 1990 indicated that part of the
explanation for high price estimates was public utility commission disallowances and
phase-ins of costs of some capital-intensive generating capacity that were not
incorporated in the projections because actual regulatory practices varied from those
assumed in the projections. For example, some nuclear units had significant shares of
their costs disallowed, and the remaining costs were phased in on a longer time schedule
than the utilities had requested, contributing to lower-than-expected prices in some
years.
Gross Domestic
Product
The economic
forecasts in the AEOs are based on projections from DRI/McGraw-Hill, adjusted for
EIA’s world oil price projections. The forecasts for gross domestic product (GDP)
show an average absolute percent error of 5.0 percent (Table 17).
Most of the projections have been less than 10 percent from actual, with the exception of
some of the forecasts in AEO83, AEO84, AEO85, AEO86, and AEO89
for the mid-1990s, which ranged up to 28.9 percent above the actual GDP. In general, from AEO82
through AEO90, the GDP forecasts tended to be underestimated for the earlier years
and overestimated for the later years. In subsequent reports, GDP has been consistently
underestimated.
The major reason for
the pattern of overestimates in the longer term forecasts in the early AEOs is the
recession that began in the latter part of 1990 and continued into 1991. The economic
forecasts produced for the AEO are trend forecasts, which do not attempt to foresee
the timing or magnitude of business cycles. The economic cycle in 1990-91 created a
breakpoint in the series being used for evaluating forecast errors. Therefore, early AEOs
did not forecast the recession and, consequently, overestimated long-term growth beyond
1991. Conversely, the underestimates in later AEOs resulted in part from
overestimates of world oil prices, which tend to dampen economic growth, plus several
other factors such as actual utility bond rates being lower than expected.
High
and Low
Economic Growth Cases
All the preceding
analysis has focused on the reference case projections from the AEOs. In fact, all
the AEOs have presented projections for more than one case. During the period
covered in this paper, the reports have included two to six alternative cases, which have
varied key reference case assumptions and examined the impacts of those assumptions across
all energy markets. Most frequently, the alternative cases have varied the macroeconomic
growth or world oil market assumptions, although other cases have been examined, such as
different oil and gas resource base assumptions. Also, many AEOs have included a
variety of additional cases that have analyzed the impacts of different assumptions on a
portion of the energy market. AEO98, for example, included 28 such cases in
addition to the reference case, high and low macroeconomic growth cases, and high and low
world oil price cases.
To analyze the
uncertainty associated with varying economic conditions, many AEOs included two
cases with alternative economic growth rates. Where available, the domestic GDP
projections for the high and low economic growth cases are presented here, along with the
accompanying total energy consumption, electricity sales, and coal consumption projections
in Tables 18 through Table 25. These variables were chosen
because total consumption and electricity sales tend to be closely linked to economic
growth, with coal consumption determined by electricity sales to a large degree. Note that
AEO85, AEO89, and AEO90 had no high or low economic growth cases, and
AEO91 included no low economic growth case.
Some caution must be
used in interpreting the results from these cases. First, during the mid-1980s, attention
in the AEOs was focused on international and domestic oil markets. In AEO86
and AEO87, the high economic growth cases included low world oil price assumptions
that would tend to increase projected energy consumption beyond the level caused by the
higher economic growth alone. Conversely, in AEO86 and AEO87, the low
economic growth cases included high world oil price assumptions. The cases were designed
in this way to examine the uncertainty in petroleum imports that results from changes in
both prices and economic growth. The high economic growth case in AEO91 also
included the assumption of low world oil prices in order to present a case with the
highest level of energy consumption from the combination of various price and growth
assumptions. For all the other AEOs examined in this paper, the economic growth
cases included moderate world oil price assumptions.
The second cautionary
note concerns the definition of the economic growth cases. Through the years, the low and
high economic growth cases have sometimes been defined by varying only the growth in
economic output. At other times, labor productivity (output per person), labor force
growth rates, and population have also varied at different rates for the high and low
economic growth cases. In addition, some of the AEOs attempted to define a broad
band of uncertainty around the reference case projections of economic growth rate, while
others defined a more narrow range. In short, the definitions of the economic growth cases
have not been consistent. Nevertheless, the presentation of these results should highlight
some of the ranges of the forecasts presented over the years.
Overall, the GDP
projections for both the low and high economic growth assumptions (Tables 21 and 25) have lower error rates—average
absolute percent errors of 4.6 and 3.9, respectively—than the reference case
projections (5.0 percent average). In part, this is because the AEOs with the worst
errors for the reference case GDP had no economic growth cases (AEO82, AEO83,
AEO85, and AEO89). Excluding these reports and AEO91, which had no
low growth case, yields average absolute percent errors of 4.6, 4.4, and 3.7 percent for
the low, reference and high growth cases, respectively. The largest errors are for the
year 1995 in the earlier AEOs; as a result, those AEOs have the highest
average absolute percent errors in all cases. In the later AEOs, GDP was
consistently underestimated in both the high and low economic growth cases. The low and
high growth GDP paths, in real terms, bracket the reference case. In the short term, low
economic growth results from higher prices, which lead to a higher set of deflators and
some apparent anomalies—with nominal GDP in the low growth case higher than in the
reference case, as in the AEO94 projections for 1993 to 1997.
Total energy
consumption in the low economic growth case (Table 18) shows a larger average absolute
percent error (2.3 percent) than in the reference and high growth cases (1.7 and 1.6
percent, respectively). The majority of the errors in the reference case were
underestimations, many of which became even worse with the lower economic growth
assumptions and were further exacerbated by the AEOs with high world oil price
assumptions (AEO86 and AEO87).
Coal consumption
errors appear worse in the low and high economic growth cases (Table
19 and Table 23), with average absolute percent errors of 3.5 percent and 3.4 percent,
respectively, compared with 3.0 percent for the reference case. When the AEOs with
no economic growth cases (AEO82, AEO83, AEO85, and AEO89) are
eliminated, some of the smaller errors in the reference case are eliminated, raising the
average absolute percent error to 3.4 percent for the reference case, similar to those in
the high and low growth cases.
The average absolute
percent error for total electricity sales in the low economic growth case (Table 20) is
higher at 2.4 percent than those in the reference and high economic growth cases (1.7 and
1.6 percent, respectively). In the reference cases, most AEOs tended to
underestimate electricity sales in most years; however, the underestimates were
exacerbated by the lower economic growth assumptions leading to the larger average errors
in the low economic growth cases.
Across comparable AEOs,
the reference case tended to underestimate GDP growth. Therefore, in the low economic
growth cases, error rates for GDP and consumption were exacerbated. Error rates in the
high economic growth cases tended to be equal to or slightly lower than those in the
reference case.
Regression
Analysis
on Historical Data
Methodology
All the preceding
analyses have focused on comparing the projections from previous AEOs with actual
historical values. This section describes simple regression analyses on historic data for
the 16 variables from Table 1, as recommended by reviewers of an earlier version of this
paper. The results of the regressions are compared with actual values to determine whether
a simple trend analysis would have performed better than the AEO models. (There are
other time series or trend analysis models, such as vector autoregression (VAR), Bayesian
vector autoregression (BVAR), or moving averages, that could also be used for comparisons
with the AEO forecasts and may prove better than a simple regression analysis.)
Historical data for the regressions were obtained from the Annual Energy Review 1996,35 and in most cases go back to 1950.
A simple lag
regression was performed for each of the 16 variables, using the following estimation
equation:
where i = 1, .
. ., 16. Two sets of estimations were made—TREND 85 and TREND 90. TREND 85, for a
given energy variable, is the result of a simple trend analysis, or regression, in which
the one independent variable is the energy term lagged one year, and the last historical
year is 1985. TREND 90 has the same definition, except that the last historical year is
1990. Appendix A provides an example of the estimation performed for total energy
consumption.
TREND 85: Sixteen
estimations were performed, one for each variable in Table 1. The total energy consumption
example had 36 observations, 1950 to 1985. After the coefficients, A and B,
were determined, the above equation was used to compute the values for the forecast
period, 1986 to 1997. The values in the total energy (TE) column of Appendix A for
years 1986 to 1997 correspond to the TREND 85 row for total energy consumption in Table
26. The estimations were repeated for the remaining 15 variables, with the results shown
in Table 26 in the TREND 85 rows.
TREND 90: The methodology
for determining the TREND 90 rows in Table 26 was the same as for
TREND 85, except that there were 41 observations for the time period 1950 to 1990. After
the coefficients were determined, the values were computed for the forecast period 1991 to
1997. The results are shown in the TREND 90 rows of Table 26.
Table 26 also
contains, for each energy variable, the average absolute percent errors between AEO86,
TREND 85, AEO92, and TREND 90, compared with the actual values. AEO86
corresponds to TREND 85 because the first forecast year is 1986. Similarly, the first
forecast year for AEO92 and TREND 90 is 1991.36
Results
In general, the
trend regressions had higher average absolute percent errors than the AEO
projections (Table 26). Trend regressions do not pick up major reversals that occur in the
forecast period. For example, for crude oil production, which declined steadily after
1991, both TREND 85 and TREND 90 overestimated by a large amount, whereas the AEOs,
especially AEO92, were better at picking up the turnaround.
Trend analysis did
poorly for price paths, especially when the directions of the price paths changed. For
example, average electricity prices were initially flat, rose from 1989 to 1993, then
flattened again. TREND 85 overestimated future electricity prices by a large margin, but AEO86
did better. Another example is natural gas wellhead prices, which TREND 85 tended to
overestimate and TREND 90 to underestimate; however, the AEOs also did poorly at
catching the turns in the price path, even though AEO86 performed better than TREND
85.
Of the 16 variables
examined in this analysis, AEO86 had lower average absolute percent errors than
TREND 85 for 10 of the variables. Even for the 6 variables for which TREND 85 had a lower
error rate, the differences between the average absolute percent errors for AEO86
and TREND 85 were less than 1 percent for 3 of them. For all the consumption, production,
import, and macroeconomic variables, AEO92 was consistently better than TREND 90,
with the exception of natural gas production and coal exports. In the case of gas
production, the average absolute percent errors for the two analyses differed by less than
1 percent. For the price variables, TREND 90 performed somewhat better than AEO92,
although average absolute percentage errors for the two analyses were the same for
electricity prices and both had relatively high error rates for all other prices.
In conclusion, a
simple trend analysis model of the type used in this report does not pick up major
reversals occurring in the forecast period; does poorly where many turns occur; and does
not pick up the effects of legislative actions or regulations on the forecast.
Conclusion
Although a primary
function of the models used by EIA to produce its AEO forecasts has been and
remains the analysis of alternative policies, many readers of the AEO use the
projected numbers as forecasts for their own purposes. Thus, it is useful for EIA analysts
and users of the AEO to know the size of and reasons for the differences between
the projections and actual values.
Throughout the AEOs,
the variables with the highest errors, expressed as average absolute percent errors, have
been prices and net imports of natural gas and coal. Natural gas, in general, has been the
fuel with the most inaccurate forecasts, showing the highest average error of all the
fuels for consumption, production, and prices. Natural gas was the last fossil fuel to be
deregulated following the heavy regulation of energy markets in the 1970s and early 1980s,
and the early AEOs assumed that natural gas would continue to be regulated until
new rules were actually promulgated. Even after deregulation, the behavior of natural gas
in competitive markets was difficult to predict.
The overestimation of
prices is the most striking feature of this evaluation. In general, more rapid
technological improvements, the erosion of OPEC’s market power, excess productive
capacity, and market competitiveness were the factors that the AEO forecasts failed
to anticipate. While the errors for prices were large, they appeared to have a relatively
minor impact on the overall projections of demand and production, although some forecasts
were clearly affected, possibly confirming the relatively low price elasticities of supply
and demand embedded in the models. For the period covered by this study, productivity and
technology improvements and the effects of gradual deregulation and changes in industry
structure, such as the treatment of contracts, have more than offset the factors that have
tended to raise fossil fuel prices. In addition, energy markets have evolved differently
than projected as a result of changes in the regulatory environment and the enactment of
changes in legislation, regulations, and standards.
Caution should be
used in drawing conclusions from the analysis of economic growth cases. First, these cases
did not have consistent world oil price assumptions (low, mid, and high). Second, the
definition of the economic growth cases varied for different AEOs. In general, for
the GDP and consumption variables compared, the low growth cases had higher error rates
than the reference cases when comparing across the AEOs that had low economic
growth cases. In general, reference case projections underestimated economic growth, and
the high growth cases thus tended to have lower or similar error rates for the variables
compared.
The most striking
result of the regression analysis described here is that a simple trend analysis model of
the type used does not perform well for projections where many turns occur. This is
especially true for major reversals in the forecast period. Trend analysis also does not
pick up technological improvements or regulatory or legislative changes. AEO86 was
better than its comparable trend analysis for the majority of the variables examined. With
the exception of natural gas production and coal exports, AEO92 consistently
outperformed its comparable trend analysis for all nonprice variables. AEO92 and
the trend analysis had similar errors for electricity prices, and although the trend
analysis was better than the AEOs for all other prices, both had relatively high
error rates.
To:
National Energy Modeling System/
Annual Energy Outlook Conference Summary
Back To:
Issues in Midterm Analysis and Forecasting 1998 Homepage
File last modified: July 20, 1998
URL: http://www.eia.doe.gov/oiaf/issues98/forecast.html
Contact the National
Energy Information Center for questions at (202) 586-8800
If you having technical problems with this site, please contact the EIA Webmaster at wmaster@eia.doe.gov