From the December 1998 SURVEY OF CURRENT
BUSINESSKurt Kunze and Teresa L. Weadock contributed to the development of this article. Colleen Ryan provided secretarial support.
The goal of the national income and product accounts (NIPA's) is to provide a comprehensive and reliable picture of the condition of the domestic economy. In particular, the current quarterly estimates of gross domestic product (GDP) are of crucial importance for the analysis of economic conditions that supports economic policy-making and business planning./1/
The reliability of these current quarterly estimates can be evaluated by answering the following four questions.
The first three questions are addressed, with respect to the current quarterly estimates of real GDP for 1983–97, in table 1. The standards of comparison are the latest available estimates, which are assumed to provide the most accurate available measure of economic activity. The first column of the table shows that the three sets of current quarterly estimates each correctly indicated the direction of change 98 percent of the time (that is, in 59 of 60 quarters). The second column shows that the current quarterly estimates correctly indicated the acceleration or deceleration about three-fourths of the time. The final column examines the ability of the current quarterly estimates to indicate whether the economy was increasing at a rate near the trend rate of about 3 percent for 1983–97. It shows that the current quarterly estimates provide correct indications of near-trend rates of increase (1.5 percent to 4.5 percent) about three-fourths of the time.
The fourth question is addressed in table 2. One or more of the current quarterly estimates successfully identified four of the five cyclical peaks since the beginning of 1969. The current quarterly estimates also successfully identified three of the five cyclical troughs. In all but one of the cases for which the peak or trough was not successfully identified, the "miss" was by a lag of one quarter; in the other case, the "miss" was by a lead of one quarter.
This article, which is based on a report recently submitted to the Office of Management and Budget (OMB), updates BEA's evaluation of the GDP estimates./2/ It expands on previous articles by including an evaluation of the components of gross domestic income (GDI), an evaluation of annual estimates of GDP and GDI and their major components, and a review of the estimates of quarterly GDP at cyclical turning points.
The report to OMB follows the requirements of Statistical Policy Directive No. 3, which mandates that for each principal Federal economic indicator, the agency that compiles that indicator must evaluate its performance every 3 years./3/ It requires that the performance evaluation address the "accuracy and reliability of the series..." as well as some other standards for documentation, avoidance of premature disclosure, and promptness in releasing estimates.
As with previous evaluations, this article evaluates GDP performance using measures of revisions. It does not directly address the "accuracy" of GDP, because such an evaluation would require data on the total measurement error, which cannot be observed. This total error arises from errors in the source data and in the estimating procedures that use the source data. Assuming that later estimates are more accurate than earlier ones, the revisions reflect improvements in accuracy relative to earlier estimates, although the later estimates may contain unknown errors.
The measures of revisions presented in this article form the basis of the evaluation of the reliability of the estimates. Revisions are due to the following: (1) Replacement of preliminary source data with revised or more comprehensive data, (2) replacement of judgmental projections with source data, (3) changes in definitions or estimating procedures, and (4) for real estimates, changes in the base year and changes in the index-number formulas used to calculate them. This last item covers a change from fixed-weight methodology to chain-weight methodology for calculating real economic measures. This change was introduced in the most recent comprehensive, or benchmark, revision of the NIPA's; shifts in the base year were incorporated in previous comprehensive NIPA revisions.
This article discusses the reliability of the estimates for 1983–97./4/ Three groups of estimates were used to calculate the measures of reliability used in this article. The first group is the current quarterly estimates, which consist of the "advance," "preliminary," and "final" estimates. The second group is the annual revision estimates, which consist of the first, second, and third annual revision estimates that are released in late July except in the years of a comprehensive NIPA revision. The third group is the "latest" estimates, which include the results of the comprehensive NIPA revision released in January 1996 as well as the most recent annual revisions of the NIPA's released in July 1997 and July 1998./5/
The remainder of this article provides information that amplifies the initial findings presented in tables 1 and 2. First, it describes the three measures used in this performance evaluation of reliability. Second, it provides additional evaluations of the current quarterly estimates of GDP and GDI and their major components. Third, it presents additional information about performance at cyclical turning points. Fourth, it evaluates the annual estimates of GDP and of GDI and their major components. Finally, it describes revisions to the estimates of the level of current-dollar GDP.
GDP estimates are made using a large number of disparate data sources, including government and private surveys, censuses, and administrative records. As a result, it is not possible to calculate some traditional measures of accuracy, such as the coefficients of variation for monthly retail sales that are published by the Census Bureau. The measures used by BEA are bias, dispersion, and relative dispersion, which are calculated as follows.
Bias is the average of the revisions:
where P is the percentage change in the current quarterly (or annual) estimate, L is the percentage change in the latest estimate, and n is the number of quarterly changes.
Dispersion is the average of the absolute values of the revisions:
Relative dispersion expresses the dispersion as a percentage of the average of the absolute values of the latest available estimates:
The two dispersion measures are emphasized in this article because the bias estimates are generally small, and small changes in the time period examined often result in substantial changes in the measures of bias./6/
Table 3 shows dispersions and relative dispersions for current-dollar and real GDP and the major components for 1983–97. There is little or no tendency for improvements across the successive quarterly estimates: The revision measures for current-dollar GDP all have about the same values, a dispersion of about 1 percentage point and a relative dispersion of 16 percent. The dispersions for real GDP are slightly larger, and the relative dispersions for real GDP are 31 to 33 percent. These larger relative dispersions reflect the larger average absolute rates of change for current-dollar GDP than for real GDP: For this period, the average absolute change for current-dollar GDP is 6.3 percent, and the average absolute rate of change for real GDP is 3.2 percent./7/
The patterns for the GDP components are similar, but the disparities in the current-dollar and the real relative dispersions are not as dramatic. With the exception of personal consumption expenditures (PCE), the components' dispersions are considerably larger than the corresponding ones for GDP. The three components of PCE—durable goods, nondurable goods, and services—have dispersions larger than those for total PCE. Likewise, the components of fixed investment have dispersions larger than those for total fixed investment. In contrast, the dispersions for State and local government expenditures are much smaller than those for total government expenditures.
The dispersions for real private investment in producers' durable equipment and real Federal defense expenditures are about 1 percentage point higher, and the dispersions for real imports are more than 2 percentage points higher, than those for the corresponding current-dollar measures. The dispersions for real Federal nondefense expenditures are several percentage points lower.
Because change in business inventories is frequently negative, it is not possible to calculate percent changes or dispersion measures for this GDP component. However, the effects of revisions to change in business inventories can be approximated by comparing the dispersion measures for the three current quarterly estimates of gross private domestic investment (GPDI), which includes change in business inventories, with those for fixed investment, which does not. The dispersions for GPDI are more than double those for fixed investment, indicating that revisions to estimates of inventories contribute significantly to revisions to estimates of GPDI.
Another way to evaluate the effect of revisions to change in inventories is to compare the averages of the dispersion measures for the three current quarterly estimates of GDP with those for final sales of domestic product, which excludes inventories. The current-dollar dispersions and relative dispersions of final sales of domestic product are only slightly larger than those for GDP. Real dispersions average less than 0.2 percentage point higher, and real relative dispersions average less than 7 percent higher. Thus, on average, revisions to inventories tend to be offset by revisions to the other components of GDP.
Table 4 shows dispersions for current-dollar and real GDP and the major components for two shorter periods, 1983–89 and 1990–97. The presentation of two time periods allows a separate evaluation of the estimates for the later period, which reflect a change in the treatment of purchases and sales of agricultural goods by the Commodity Credit Corporation (CCC) and an improvement in the procedures for the processing of information about trade in goods; these two changes substantially affected quarterly changes in business inventories, government expenditures, and trade in goods./8/
The effects of the first of these changes, the revised classification of CCC transactions, may be seen by comparing the dispersions for the estimates of GPDI and of fixed investment. The dispersions for GPDI are substantially smaller in 1990–97, but those for fixed investment are only slightly smaller. Likewise, the dispersions for Federal Government expenditures—and nondefense expenditures—are substantially smaller in 1990–97. (Because the revised treatment of CCC transactions produces offsetting revisions, current-dollar and real GDP are not affected.)
Improvements in the processing of data on trade in goods by the Census Bureau also have had a dramatic effect. In particular, the dispersions for current-dollar and real imports in 1990–97 are less than one-half of those in 1983–89. The improvements had smaller, but still noticeable, effects on exports.
Overall, the dispersions for current-dollar GDP and its components in 1990–97 are generally smaller than the corresponding dispersions in 1983–89. However, the smaller dispersions may not indicate improvements in reliability, because the latest estimates for 1990–97 have been subject to fewer successive revisions.
The dispersions for real GDP are about the same size in the two periods, mainly because the revisions to the real GDP components did not offset each other as much as the revisions to the current-dollar GDP components. The dispersions for most of the real GDP components are smaller in 1990–97.
Table 5 shows the biases for current-dollar and real GDP and the major components. The biases for current-dollar and real GDP are small and negative. (A negative bias means that GDP tended to be revised up.) The biases for current-dollar and real PCE and for most of its major components are also negative, and those for PCE are larger than those for GDP. The biases for current-dollar and real GPDI and for current-dollar and real nonresidential fixed investment are positive and larger than those for PCE. The biases for current-dollar and real exports are also large, but negative. The biases for imports are mixed. The biases for Federal Government nondefense expenditures are also large and positive, but the biases for all Federal Government expenditures are small and positive. The pattern for Federal nondefense expenditures partly reflects the changed classification of CCC transactions. The biases for current-dollar Federal defense expenditures are small, but those for real expenditures are substantial and negative.
Table 6 shows dispersions and relative dispersions for GDI and for national income and its major components for 1983–97. The dispersions and relative dispersions for GDI are somewhat larger than those for GDP, and the dispersions and relative dispersions for national income are even larger. The larger dispersions for national income reflect the substantial dispersions for the components that are added and subtracted from GDI to obtain national income. It should be noted that the measures for these components were greatly affected by the incorporation of a new depreciation pattern into consumption of fixed capital and the new treatment of government investment introduced in the most recent comprehensive NIPA revision. The new depreciation patterns result in revisions both to consumption of fixed capital and to the capital consumption adjustment for the three types of business income; the new treatment of government investment results in the addition to GDI of consumption of fixed capital for government.
Among the components of GDI, only compensation of employees has dispersions and relative dispersions that are similar to those for the components of GDP. The other components of GDI have much larger dispersions and relative dispersions. The relative dispersions for proprietors' income with inventory valuation adjustment and capital consumption adjustment exceed 100 percent, reflecting the typically large revisions to farm proprietors' income; the relative dispersions for nonfarm proprietors' income are about 65 percent. The larger dispersions and relative dispersions reflect the limited availability of source data for all components of GDI except compensation of employees and corporate profits. For the quarterly estimates of the other components, very little current quarterly source data are available. For the annual estimates of these components, most of the final data are incorporated in the second annual revision. As with the product-side components of GDP, there is little or no tendency for improvements when progressing from advance to preliminary to final current quarterly estimates.
One of the more important aspects of the reliability of GDP estimates is how well they identify cyclical peaks and troughs. The advance estimates correctly identified the peaks for the first four cycles beginning with 1969 (top panel of table 2). With the exception of one of the second annual revision estimates, all the later estimates also correctly identified these peaks. For the most recent cycle, the peak in the second quarter of 1990 was not correctly identified until the second annual revision estimate.
The troughs were not identified as reliably as the peaks. The advance estimates correctly identified two of the five troughs, and the preliminary estimates identified an additional trough (bottom panel of table 2). The second annual revision identified the fourth trough, and the fifth trough was not identified until the most recent comprehensive NIPA revision. All of the estimates correctly identified the most recent recession's trough in the first quarter of 1991.
Revisions in the 1990–91 recession.—The beginning of the 1990–91 recession in the second quarter of 1990 was not correctly identified until the second annual revision; previously, the third quarter of 1990 had been identified as the peak. The latest estimate shows that real GDP declined 1.9 percent in the third quarter; the current quarterly estimates showed increases in real GDP ranging from 1.6 percent in the advance estimate to 0.7 percent in the final estimate. In addition to missing the peak, the current quarterly estimates underestimated the rate of decline in real GDP in the fourth quarter of 1990 by about 2 percentage points and overestimated the rate of decline in the first quarter of 1991 by about 1 percentage point (chart 1). The more rapid decline in GDP in the fourth quarter of 1990 did not appear until the second annual revision (in July 1992), and the smaller decline in real GDP in the first quarter of 1991 did not appear until the third annual revision (in July 1994). The three current quarterly estimates, and all successive ones, correctly identified the beginning of the recovery in the second quarter of 1991 and roughly indicated the pace of the recovery.
Revisions to real change in business inventories (CBI) played a major role in the mis-estimation of changes in real GDP in all three recession quarters. Roughly half of the revision from an increase to a decline in real GDP in the third quarter of 1990 reflected a revision to CBI. Most of the downward revision to the change in real GDP in the fourth quarter of 1990 was due to the effects of very large upward revisions to the price indexes for fixed nonresidential investment and exports; in addition, real CBI was revised down much more sharply than current-dollar CBI. The modest upward revision (that is, reduction in the decline) to real GDP in the first quarter of 1991 was due to upward revisions to real fixed investment and to real CBI that more than offset a downward revision to real PCE.
Table 7 shows dispersions and relative dispersions for estimates of annual percent changes in current-dollar and real GDP and some of the major components. The measures marked "final" are the annual estimates published at the time that the final current quarterly estimates for each year's fourth quarter were published. The three successive annual revision estimates shown in the table are those revisions that are typically published in late July for the 3 preceding years; these are labeled first annual, second annual, and third annual. These annual estimates incorporate the increasing amounts of source data that become available following the end of each year./9/ Data that would allow a complete evaluation of the 1996 and 1997 estimates are not yet available.
The estimates of annual GDP and its components have much smaller dispersions and relative dispersions than those for the estimates of quarterly GDP and its components. In contrast to the successive current quarterly estimates, the sizes of dispersion and relative dispersion tend to decrease as the successive annual revision estimates are made. An exception is for Federal Government expenditures, which tends to register small increases in dispersion with the successive annual revisions, possibly reflecting the change in the classification of CCC purchases. Like the quarterly estimates, the relative dispersions for real GDP and its major components are usually considerably larger than those for current-dollar GDP and its major components. Also like the quarterly estimates, the dispersions and relative dispersions for current-dollar and real GDP are generally smaller than those for most of their major components. Among the components, PCE has the smallest dispersions and relative dispersions, and Federal Government expenditures has the largest.
Table 8 shows dispersions and relative dispersions for the annual estimates of national income and its major components. As with GDP and its major components, the annual estimates have much smaller dispersions than the quarterly estimates for national income and for compensation of employees. The smallest values of dispersion and relative dispersion are for the second annual revision estimates; however, the higher dispersions for the third annual revision may be an accidental result of the small number of observations. The dispersions and relative dispersions for national income are larger than those for GDP, and with the exception of compensation of employees, those for the components of national income are larger than those for most components of GDP. The dispersions and relative dispersions for total proprietors' income and for nonfarm proprietors' income are about the same size, and they are much larger than those for compensation of employees but smaller than those for corporate profits.
Like BEA's previous studies of revisions to GDP, this analysis of the reliability of the GDP estimates is primarily based on revisions in change, and the latest estimates are assumed to be the best. Analyses of revisions in the levels of GDP (and its components) are not featured, because the largest revisions in levels almost always occur in comprehensive NIPA revisions, which incorporate both definitional and statistical revisions./10/ Definitional revisions are made to adapt the NIPA's to a changing economy and have little to do with the reliability of earlier vintage estimates. Revisions to the current-dollar annual estimates are reviewed, but revisions to the real levels are not. Changes in reference periods that are incorporated as part of comprehensive NIPA revisions result in substantially different real levels: In an inflationary environment, the adoption of a later reference period produces lower levels of price indexes and higher levels of real GDP.
This section looks at revisions to the level of GDP in annual NIPA revisions, revisions to the level of GDP in the latest comprehensive NIPA revision, and revisions to the average shares of GDP that are accounted for by its major components.
Annual revisions to the levels of GDP are similar in size to the dispersions of the three annual estimates (as measured by the latest estimates). For 1983–94, the annual revisions (without regard to sign) averaged 0.5 percent for the first annual, 0.4 percent for the second annual, and 0.2 percent for the third annual./11/ The dispersions for these years were 0.4 percent for the first and second annual, and 0.3 percent for the third annual.
In the comprehensive NIPA revision released in January 1996, the level of GDP was revised up throughout 1982–94, by amounts ranging from $93 billion in 1982 to $224 billion in 1992 (chart 2). On average, nearly two-thirds of the upward revisions were the result of definitional changes./12/ The upward revisions due to definitional changes increase smoothly from $74 billion in 1982 to $125 billion in 1994 (chart 3). The upward revisions due to statistical changes, which are the net results of both positive and negative revisions of various GDP components, fluctuate considerably around a generally upward trend./13/
It is possible to separate the effects of the definitional and statistical revisions on reliability by comparing the 1982–94 dispersions in annual percent changes for the estimates of GDP before and after the 1996 comprehensive NIPA revision.
| Standard | Dispersion (percentage point) |
| Revised GDP | 0.26 |
| Revised GDP less statistical revisions | 0.06 |
| Revised GDP less definitional revisions | 0.25 |
The dispersions for the comprehensive revision GDP estimates and the revised GDP estimates less statistical revisions—that is, with only definitional revisions—differ by only 0.06 percentage point. In contrast, the dispersion for the revised GDP estimates without definitional revisions—that is, with only statistical revisions—is about the same size as the dispersion for the revised GDP estimates. Thus, definitional revisions have little effect on the revisions to the changes in GDP though they account for most of the revisions to the levels of GDP.
The comprehensive NIPA revision resulted in revisions to the shares of GDP that are accounted for by the major components of GDP. The revisions were small—all were less than 2 percent—with little trend for 1983–94. The revisions to the shares accounted for by PCE averaged 1.1 percent and ranged from 0.9 percent to 1.4 percent. The revisions to the shares accounted for by gross private domestic investment averaged 0.6 percent and ranged from 0.4 percent to 0.7 percent. The revisions to the shares accounted for by net exports averaged -0.1 percent and ranged from -0.1 percent to 0.0 percent. The revisions to the shares accounted for by government consumption expenditures and gross investment averaged -1.7 percent and ranged from -1.5 percent to -1.9 percent.
The comprehensive NIPA revision also resulted in only small revisions to the shares of national income that are accounted for by labor and capital. The revisions to the shares of national income accounted for by employee compensation, which approximates the share of labor, averaged -0.5 percent in 1983–94 and ranged from -0.1 percent to -0.8 percent. The share of the rest of national income, which approximates the share of capital, was revised up correspondingly.
Footnotes:
1. The current quarterly estimates consist of the "advance," "preliminary," and "final" estimates that are released near the end of the first, second, and third months, respectively, after the end of each quarter.
2. See Allan H. Young, "Reliability and Accuracy of the Quarterly Estimates of GDP," SURVEY OF CURRENT BUSINESS 73 (October 1993): 29–43.
3. "Statistical Policy Directive No. 3," Federal Register, Vol. 50, No. 186 (September 25, 1985): 38, 932–34.
4. Estimates for years before 1983 were benchmarked to the 1982 input-output table, which was released in July 1991, and were reviewed in the previous evaluation article. In the most recent comprehensive NIPA revision, the revisions to those current-dollar GDP estimates were small.
5. For additional information about the various vintages of estimates and how they are prepared, see "Updated Summary NIPA Methodologies," SURVEY 78 (September 1998): 14–35.
6. The previous article found that most bias estimates were generally not statistically significant; see Young, 37–38.
7. A 1-percentage-point revision to the change in current-dollar GDP, with no revision to prices or weights, will result in a 1-percentage-point revision to the change in real GDP. Thus, the effect on relative dispersion will be larger for real GDP simply because its denominator—the absolute average rate of change—will be smaller than the denominator for current-dollar GDP. In addition, the revisions to prices generally have only relatively small effects on the revisions to real measures in comparison with the effects of revisions to current-dollar estimates.
8. The reclassification of purchases and sales of the CCC from the government sector to the business sector is described in "The Comprehensive Revision of the U.S. National Income and Product Accounts: A Review of Revisions and Major Statistical Changes," SURVEY 71 (December 1991): 30–31. The improvements in the trade in goods estimates are described in "An Advance Overview of the Comprehensive Revision of the National Income and Product Accounts," SURVEY 65 (October 1985): 24–25.
9. Annual revisions were not made in the years of comprehensive NIPA revisions—1985, 1991, and 1995.
10. Statistical revisions generally reflect incorporation of better data, but it is sometimes difficult to separate revisions that are due to better data from those that are due to methodological improvements. For example, the comprehensive NIPA revision released in January 1996 incorporated a better methodology for calculating depreciation that also incorporated new and revised source data on investment.
11. Revisions to annual levels of GDP were not made in the years of a comprehensive NIPA revision. Thus, the following years are excluded from the calculations: 1984, 1990, and 1994 for the first annual revisions, 1983, 1989, and 1993 for the second annual, and 1988 and 1992 for the third annual.
12. The definitional changes that affected GDP were in "government consumption expenditures and gross investment." One change recognized government investment and resulted in a change in name from "government purchases." The other changed the treatment of Federal Government contributions to military and civilian retirement programs. For a more complete description of the definitional and statistical revisions, see Robert P. Parker and Eugene P. Seskin, "Improved Estimates of the National Income and Product Accounts for 1959–95: Results of the Comprehensive Revision," SURVEY 76 (January/February 1996): 1–27.
13. In the comprehensive NIPA revision that was released in December 1991, the effects of definitional revisions were small in comparison with the statistical revisions. The two largest definitional revisions resulted in largely offsetting revisions to components of GDP and so had almost no effect on GDP. One of these, a redefinition of government sales and nontaxes, lowered government purchases and raised PCE. The other, a reclassification of CCC loans, resulted in corresponding revisions of opposite sign to change in business inventories and to government purchases.
