Skip Navigation

The Economic and Budget Outlook: An Update September 1997

Since the Congressional Budget Office issued its first forecast in 1976, CBO has compiled a record of economic predictions that compares favorably with the track records of five Administrations and the consensus forecasts of a sizable sample of private-sector economists. Although the margin is slight, CBO's forecasts have generally been closer than the Administration's to the actual values of several economic indicators that are important for projecting the budget. Moreover, during the 14 years for which comparisons are possible, CBO's forecasts have been about as accurate as the average of the 50 or so forecasts that make up the Blue Chip consensus survey. Comparing CBO's forecasts with that survey suggests that when CBO's economic predictions missed the mark by a margin wide enough to contribute to sizable misestimates of the deficit, those errors probably reflected limitations that confronted all forecasters.

Those conclusions echo the findings of previous studies published by the Congressional Budget Office and other government and academic reviewers. They emerge from an evaluation of the accuracy of short-term forecasts for four economic indicators: growth in real (inflation-adjusted) output, inflation in the consumer price index (CPI), interest rates on three-month Treasury bills in both nominal and real terms, and interest rates on 10-year Treasury notes and Aaa corporate bonds. In carrying out this evaluation, CBO compiled two-year averages of its forecasts for the four indicators and compared them with historical values as well as with the corresponding forecasts of the Administration and the Blue Chip consensus. In addition to these economic indicators, a measure of taxable incomes--wage and salary distributions plus corporate profits--is also examined and compared with the Administration's forecasts.

Both CBO and the Administration have tended to err on the side of optimism in their forecasts over a two-year period. In other words, the average forecast error for real growth was an overestimate, and the average error for inflation was an underestimate. The Administration has been more optimistic than CBO in forecasting nominal interest rates. Overall, the average errors in the Administration's two-year forecasts were slightly greater than in CBO's. Finally, CBO's forecasts appear to be about as accurate as those of the Blue Chip consensus over the period for which comparable Blue Chip forecasts are available (1982-1995).

CBO's and the Administration's longer-term (five- year) projections of average growth in real output were generally optimistic, but CBO's errors were generally much smaller than the Administration's. For the longer-term projections of real output, CBO's errors averaged only slightly larger than those in its short-term forecasts of real output. Again, CBO's projections were about as accurate as those of the Blue Chip consensus over the comparable period (1979-1992).

The differences among the three forecasts, however, are not large enough to be statistically significant. The small number of forecasts available for analysis makes it difficult to distinguish meaningful differences in their performance from those that might arise randomly. Thus, the statistics presented here are not reliable indicators of the future performance of any of the forecasters.
 

Sources of Data for the Evaluation

Evaluating CBO's forecasting record requires compiling the basic historical and forecast data for growth in real output, CPI inflation, interest rates, and taxable incomes. Although each of these series has an important influence on budget projections, an accurate forecast of the two-year average growth in real output is the most critical economic factor in accurately estimating the deficit for the upcoming budget year. Two-year average forecasts published in early 1996 and 1997 could not be included in this evaluation because historical values for 1997 and 1998 are, of course, not yet available. The data were therefore compiled using forecasts published early in the years 1976 through 1995.

Selection of Historical Data

Which historical data to use for the evaluation was dictated by the availability of actual data and the nature of the individual forecasts examined. Although CBO, the Administration, and Blue Chip all published the same measure for real output growth, selecting a historical series was difficult because of periodic benchmark revisions in the actual data.⁽¹⁾ By comparison, not all of the forecasters published the same measures for CPI inflation and interest rates, but the selection of historical data for those series was clear-cut.

Real Output Growth. Historical two-year averages of growth in real output were developed from calendar year averages of the quarterly chain-type, annual-weighted indexes of real gross national product (GNP) and real gross domestic product (GDP) published by the Bureau of Economic Analysis (BEA). The fact that several real GNP and GDP series were discontinued because of periodic benchmark revisions meant that they were unsuitable historical series. For example, during the 1976-1985 period, the three forecasters published estimates for a measure of growth in real GNP that was based on 1972 prices, which was the measure published by BEA at the time. In late 1985, however, BEA discontinued this 1972-dollar series and began to publish GNP on a 1982-dollar basis. As a result, an official series of values for GNP growth in 1972 dollars is not available for the years after 1984; thus, actual two-year average growth rates are not available to compare with the forecasts made in early 1984 and 1985.

From 1986 to 1991, forecasters published estimates of growth in real GNP based on 1982 prices. BEA revised the benchmark again in the second half of 1991; it discontinued the 1982-dollar GNP and began to publish GNP on a 1987-dollar basis.⁽²⁾ Consequently, the historical annual series for 1982-dollar GNP is available only through 1990, and actual two-year average growth rates are not available for the forecasts made in early 1990 and 1991. The forecasters then published estimates of growth in real GDP on a 1987-dollar basis until 1995, when BEA made another switch, late in the year, to a chain-weighted measure of GDP. Therefore, the historical annual series for 1987-dollar GDP ends with the 1994 annual value, and actual two-year average growth rates are not available for the forecasts made in early 1994 and 1995.

By periodically updating the series to reflect more recent prices, BEA's benchmark revisions yield a measure of real output that is more relevant for analyzing contemporary movements in real growth. But the process makes it difficult to evaluate forecasts of real growth produced over a period of years for series that are subsequently discontinued. The difficulties presented by periodic revisions of the data are avoided here by using one of BEA's alternative measures of real GNP and GDP, the chain-type annual-weighted index.⁽³⁾

CPI Inflation. Two-year averages of inflation in the consumer price index were calculated from calendar year averages of monthly data published by the Bureau of Labor Statistics. Before 1978, the bureau published only one consumer price index series, now known as the CPI-W (the price index for urban wage earners and clerical workers). In January 1978, however, it began to publish a second, broader consumer price index series, the CPI-U (the price index for all urban consumers). CBO's comparison of forecasts used both series.

Until 1992, the Administration published its forecasts for the CPI-W, the measure used to index most of the federal government's expenditures for entitlement programs. By contrast, for all but four of its forecasts since 1979 (1986 through 1989), CBO based its inflation forecast on the CPI-U, a more widely cited measure of inflation and the one now used to index federal income tax brackets. The Blue Chip consensus has always published its forecast of the CPI-U. Although both the CPI-U and CPI-W may be forecast with the same relative ease, and annual fluctuations in the two series are virtually indistinguishable, they differ in some years; for that reason, CBO used historical data for both series to evaluate the alternative forecast records.

Interest Rates. Two-year averages of nominal short- and long-term interest rates were developed from calendar year averages of monthly data published by the Board of Governors of the Federal Reserve System.

The forecasts of short-term interest rates were compared using historical values for two measures of the interest rate on three-month Treasury bills: the new-issue rate and the secondary-market rate. The Administration forecasts the new-issue rate, which corresponds to the price of three-month bills auctioned by the Treasury Department--that is, it reflects the interest actually paid on that debt. CBO forecasts the secondary-market rate, which corresponds to the price of the three-month bills traded outside the Treasury auctions. Because such transactions occur continually in markets that involve many more traders than do Treasury auctions, the secondary-market rate provides an updated evaluation of the short-term federal debt by the wider financial community. Blue Chip has alternated between these two rates; it published the new-issue rate from 1982 to 1985, switched to the secondary-market rate during the 1986-1991 period, and then returned to the new-issue rate in 1992. Clearly, there is no reason to expect the two rates to differ persistently; indeed, the differences between their calendar year averages are miniscule.

The Congressional Budget Office likewise compared the various forecasts of long-term interest rates using historical values for two measures of long-term rates: the 10-year Treasury note rate and Moody's Aaa corporate bond rate. A comparison of forecasts is only possible beginning in 1984 because not all of the forecasters published projections of long-term interest rates before that year. For forecasts made in early 1984 and 1985, CBO projected the Aaa corporate bond rate. Beginning with its early 1986 forecast, however, CBO switched to the 10-year Treasury note rate. The Administration has always published its projection for the 10-year Treasury note rate, but Blue Chip has published the Aaa corporate bond rate.

CBO calculated separate historical values for real short-term interest rates using the nominal short-term interest rate and inflation rate appropriate for each forecaster. In each case, the two-year average nominal interest rate was discounted by the two-year average rate of inflation. The resulting real short-term interest rates were very similar. Because there is no agreed-upon method for calculating real long-term interest rates, they were not included in the evaluation.

Taxable Incomes. Through its influence on the projection for federal government revenues, the forecast for taxable incomes plays a critical role in determining the accuracy of the deficit projection. The income measure examined here--wage and salary distributions plus the book value of corporate profits--combines the two sources of income to which tax receipts are most sensitive. Because the effective rates of tax on wages (including payroll and income taxes) and corporate profits are nearly the same and because these tax rates exceed the rate of taxation of other income sources (such as interest income), it is appropriate to consider wages and profits together.

Although the level of taxable incomes is the factor that most directly affects federal revenue, historical estimates of the levels of incomes are subject to substantial statistical revision. As a result, using the levels of taxable income would distort the forecast comparison. Instead, the forecasts are presented here as changes in taxable incomes as a share of total income; the historical revisions, carried forward consistently to projections, should not affect projections of revenues. Moreover, the shares formulation is closer to the concept that macroeconomists consider when they construct their forecasts.

Sources of Forecast Data

With the exception of the measures of taxable incomes, the evaluation used calendar year forecasts and projections, which CBO has published early each year since 1976, timed to coincide with the publication of the Administration's budget proposals. The Administration's forecasts were taken from its budget in all but one case; the forecast made in early 1981 came from the Reagan Administration's revisions of President Carter's last budget. The corresponding CBO forecast was taken from CBO's published analysis of President Reagan's budget proposals. That forecast did not include the economic effects of the new Administration's fiscal policy proposals.⁽⁴⁾

The average two-year forecasts of the Blue Chip consensus survey were taken from those published in the same month as CBO's forecasts. Because the Blue Chip consensus did not begin publishing its two-year forecasts until the middle of 1981, the first consensus forecast available for use in this comparison was published in early 1982. Average five-year projections, however, are published by Blue Chip only two or three times a year. All but one of its five-year projections used in this evaluation were published in March; the 1980-1984 projection was published in May.

Since 1985, the Congressional Budget Office has regularly included projections of economic profits and wage and salary disbursements in The Economic and Budget Outlook. Because book profits more closely reflect the corporate profits tax base than do economic profits, forecasts of book profits were extracted from CBO's unpublished forecast files. Unpublished CBO forecasts are used for both profits and wages for the period from 1980 through 1984.
 

Measuring Forecast Performance

Following earlier studies of economic forecasts, the evaluation of CBO's forecasts focused on two aspects of their performance: statistical bias and accuracy.

Bias

The statistical bias of a forecast is the extent to which the forecast can be expected to differ from what actually occurs. CBO's evaluation used the mean error to measure statistical bias. That statistic--the arithmetic average of all the forecast errors--is the simplest and most widely used measure of forecast bias. Because the mean error is a simple average, however, underestimates and overestimates offset each other in calculating it. As a result, the mean error imperfectly measures the quality of a forecast--a small mean error would result if all the errors were small or if all the errors were large but the overestimates and underestimates happened to balance out.

Accuracy

The accuracy of a forecast is the degree to which forecast values are narrowly dispersed around actual outcomes. Measures of accuracy more clearly reflect the usual meaning of forecast performance than does the mean error. The evaluation used two measures of accuracy. The mean absolute error--the average of the forecast errors without regard to arithmetic sign--indi-cates the average distance between forecasts and actual values without regard to whether individual forecasts are overestimates or underestimates. The root mean square error--calculated by first squaring all the errors, then taking the square root of the arithmetic average of the squared errors--also shows the size of the error without regard to sign, but it gives greater weight to larger errors.

Measurement Issues

In addition to those three statistical indicators, there are many other measures of forecast performance. To test for statistical bias in CBO's forecasts, studies by analysts outside CBO have used measures that are slightly more elaborate than the mean error. Those studies have generally concluded, as does this evaluation, that CBO's short-term economic forecasts do not contain a statistically significant bias.⁽⁵⁾

In addition, a number of methods have been developed to evaluate a forecast's efficiency. Efficiency indicates the extent to which a particular forecast could have been improved by using additional information that was at the forecaster's disposal when the forecast was made.⁽⁶⁾ The Blue Chip consensus forecasts represent a wide variety of economic forecasters and thus reflect a broader blend of sources and methods than can be expected from any single forecaster. The use of the Blue Chip predictions in this evaluation can therefore be interpreted as a proxy for an efficient forecast. The fact that CBO's forecasts are about as accurate as Blue Chip's is a rough indication of their efficiency.

Such elaborate measures and methods, however, are not necessarily reliable indicators when the sample of observations is small, such as the 20 observations that make up the sample of CBO's two-year forecasts. Small samples present three main types of problems for evaluating forecasts, including forecasts based on the simple measures presented here. First, small samples reduce the reliability of statistical tests that are based on the assumption that the underlying population of forecast errors follows a normal distribution. The more elaborate tests of forecast performance all make such an assumption about the hypothetical ideal forecast with which the actual forecasts are compared. Second, in small samples, individual forecast errors have a relatively large weight in the calculation of summary measures. The mean error, for example, can fluctuate in arithmetic sign when a single observation is added to a small sample. Third, the small sample means that CBO's track record cannot be used in a statistically reliable way to indicate either the direction or the size of future forecasting errors.

Apart from the general caution that should attend statistical conclusions based on small samples, there are several other reasons to view this evaluation of CBO's forecasts with particular caution. First, the procedures and purposes of CBO's and the Administration's forecasts have changed over the past 20 years and may change again in the future. For example, in the late 1970s, CBO characterized its long-term projections as a goal for the economy, whereas it now considers its projections to be what will prevail on average if the economy continues to reflect historical trends. Second, an institution's forecasting track record may not foretell its future abilities because of changes in personnel or methods. Finally, forecast errors increase when the economy is more volatile. All three forecasters made exceptionally large errors when forecasting for periods that included turning points in the business cycle.
 

CBO's Forecasting Record

This analysis evaluated the Congressional Budget Office's forecasts over two-year and five-year periods. The period of most interest for forecasters of the budget is two years. Because the Administration's and CBO's winter budget publications focus on the budget projection for the fiscal year beginning in the following October, an economic forecast that is accurate not only for the months leading up to the budget year but also for the budget year itself will provide the basis for a more accurate forecast of the deficit. A five-year period is used to examine the accuracy of longer-term projections of growth in real output.

Short-Term Forecasts

Historically, the Congressional Budget Office's two- year forecasts are slightly more accurate than the Administration's and suffer from slightly less statistical bias. In most cases, however, the differences are small. Furthermore, CBO's forecasts are about as accurate as Blue Chip's average forecasts.

An accurate prediction of two-year growth in real output is the most important factor in minimizing errors when forecasting the deficit for the budget year. Accurate predictions of nominal output, inflation, and nominal interest rates are less important for forecasting deficits now than they were in the late 1970s and early 1980s. The reason is that given current law and the level of the national debt, inflation increases both revenues and outlays by similar amounts. Revenues increase with inflation because taxes are levied on nominal incomes. Outlays increase because various entitlement programs are indexed to inflation and because nominal interest rates tend to increase with inflation, which in turn raises the cost of servicing the federal debt.⁽⁷⁾

Real Output Growth. For the two-year forecasts made between 1976 and 1995, CBO had a slightly better record than the Administration in predicting growth in real output (see Table A-1). On average, both CBO's and the Administration's forecasts tended to be overestimates. CBO was closer to the actual value in 12 of the 20 forecasts made between 1976 and 1995, the Administration was closer in five periods, and both had identical errors in three periods. CBO's predictions of real growth made between 1982 and 1995 were, on average, as accurate as those of the Blue Chip consensus.
 

Since 1992, all three forecasters--CBO, the Administration, and the Blue Chip--have predicted real GDP growth with striking accuracy. The relatively steady growth trend for real GDP in the 1990s is likely to be a primary influence on this improved record. Consequently, the improvement cannot be relied upon to continue.

CPI Inflation. The records for forecasting the average annual growth in the consumer price index over a two-year period were very similar (see Table A-2). Both CBO and the Administration underestimated future inflation in their forecasts for 1977 through 1980, and both tended to overestimate it in their forecasts for 1981 through 1986. The average measures of bias and accuracy were virtually the same for CBO and the Administration. CBO was closer to the actual value in seven of the 20 periods, the Administration was closer in nine periods, and the two forecasts had identical errors in four periods. For the 1982-1995 period, CBO's forecasts of inflation were as accurate as those of both the Administration and Blue Chip. Moreover, the track records of CBO and the Administration in predicting inflation both seem to have improved in the 1990s.
 

Real Short-Term Interest Rates. For the forecasts made in 1976 through 1995, CBO had a slight edge over the Administration in estimating real short-term interest rates (see Table A-5). Again, the Administration was more likely than CBO to underestimate interest rates and its mean error was greater. CBO and the Administration recorded similar mean absolute and root mean square errors. CBO's forecasts were closer to the actual value in 11 of the 20 periods, the Administration's were closer in eight, and the two registered identical errors in one period. For forecasts made between 1982 and 1995, CBO's errors were generally similar in both direction and magnitude to those of the Blue Chip consensus.
 

In recent years however, both CBO and the Administration have significantly underestimated the change in the wage and profit share. The rapid growth in corporate profits and dividends in both 1995 and 1996 reported in the July 1997 revisions of the national income and product accounts surprised most analysts.

Longer-Term Projections

In projecting real GNP growth for the more distant future, measured here as five years ahead, the Administration's errors were larger than CBO's (see Table A-7). Although that comparative advantage for CBO does not directly affect the estimates of the deficit for the budget year, accuracy in the longer term is obviously important for budgetary planning over several years. Neither the Administration nor CBO, however, considers its projections to be its best guess about the year-to-year course of the economy. The Administration's projections each year are based on the adoption of the President's budget as submitted, and for most years CBO has considered its projections an indication of the average future performance of the economy if major historical trends continue. Neither institution attempts to anticipate cyclical fluctuations in the projection period.
 

The size of the root mean square errors for the entire period for CBO and, to a lesser extent, for the Administration also resulted largely from errors in projections made during the first four years. CBO was more accurate in its winter projections made in the 1980- 1982 period but had a lesser edge in later years. Again, CBO's projections were about as accurate as those of the Blue Chip consensus over the comparable period.

2. As of the 1992 benchmark revision, GDP replaced GNP as the central measure of national output.

3. For a discussion of this index, see Congressional Budget Office, The Economic and Budget Outlook: An Update (August 1995), pp. 71-73.

4. Another exceptional case occurred in early 1993, when the Clinton Administration adopted CBO's economic assumptions as the basis for its budget. Because of that, the errors for the early 1993 forecast are virtually the same for CBO and the Administration.

5. Another approach to testing a forecast for bias is based on linear regression analysis of actual and forecast values. For details of that method, see J. Mincer and V. Zarnowitz, "The Evaluation of Economic Forecasts," in J. Mincer, ed., Economic Forecasts and Expectations (New York: National Bureau of Economic Research, 1969). That approach is not used here because of the small sample size. However, previous studies that have used it to evaluate the short-term forecasts of CBO and the Administration have not been able to reject the hypothesis that those forecasts are unbiased. See, for example, M.T. Belongia, "Are Economic Forecasts by Government Agencies Biased? Accurate?" Review, Federal Reserve Bank of St. Louis, vol. 70, no. 6 (November/December 1988), pp. 15-23.

6. For studies that have examined the relative efficiency of CBO's forecasts, see Belongia, "Are Economic Forecasts by Government Agencies Biased?"; and S.M. Miller, "Forecasting Federal Budget Deficits: How Reliable Are U.S. Congressional Budget Office Projections?" Applied Economics, vol. 23 (December 1991), pp. 1789-1799. Although both of the studies identify series that might have been used to make CBO's forecasts more accurate, they rely on statistics that assume a larger sample than is available. Moreover, although statistical tests can identify sources of inefficiency in a forecast after the fact, they generally do not indicate how such information can be used to improve forecasts when they are made.

7. Rules of thumb for estimating the effect on the deficit of changes in various macroeconomic variables are given in Congressional Budget Office, The Economic and Budget Outlook: Fiscal Years 1998-2007 (January 1997), pp. 91-95.

---