From the October 1993 SURVEY OF
CURRENT BUSINESS
NOTE.—Albert A. Hirsch, Clinton P. McCully, Robert P. Parker, Teresa L. Weadock, and Richard C. Ziemer contributed to the development of this article.
This article examines the record of revisions in the quarterly estimates of gross domestic product (GDP) for 1978–91 in order to gain insights into the reliability and accuracy of the estimates in the national income and product accounts (NIPA's). This examination is part of a periodic evaluation of the GDP estimates required by the Office of Management and Budget; the first such evaluation was carried out in 1987./1/ Much of that study remains relevant, including the discussion of other approaches to assessing reliability and accuracy.
In this article, the term "reliability" refers to the revisions in the estimates. Revisions come about for four reasons: (1) Replacement of preliminary source data with revised or more comprehensive data, (2) replacement of judgmental projections with source data, (3) changes in definitions or estimating procedures, and (4) in the constant-dollar estimates, updating of the base year.
The term "accuracy" refers to the total measurement error, which is unobserved. The total error arises primarily from error in the source data and secondarily from BEA's estimating procedures that utilize the source data. On the assumption that later estimates are more accurate than earlier ones, revisions can be viewed as measuring part of the total error in earlier estimates. The rest of the error in the earlier estimates, which is unknown, becomes the total error in the later estimates.
As an introduction, chart 1 shows the quarterly changes in real GDP for the period 1978–91./2/ (The GDP estimates in the current study do not reflect the revisions for 1990–91 released in August 1993.) For each quarter, the chart shows the first and each successive "current" estimate and the "latest available" estimate. (For a description of the terms used in this article to designate the various quarterly estimates, see the box on the next page.) It is useful to examine the chart in light of the following questions.
Tallies of the charted data, summarized in table 1, show that the current estimates correctly indicate direction of change almost 90 percent of the time. They correctly indicate acceleration and deceleration between 75 and 80 percent of the time. (If changes between -1 percent and +1 percent are disregarded, these early estimates correctly indicate direction over 90 percent of the time and acceleration about 85 percent of the time). They also correctly separate changes that are close to the long-term trend in GDP growth (between +1 percent and +4 percent) from those that are significantly different from trend (more than +4 percent or less than +1 percent) between 65 and 75 percent of the time.
The remainder of this article examines the revision record for 1978–91 in more detail and compares it with results from previous BEA studies, which cover the period 1947–77. The following are the major findings from this examination.
The last finding suggests that one or both of the later current estimates of GDP might be discontinued without much loss of information. On this point, the most important question to examine further is whether the advance estimate is a suitable substitute for the later current estimates in terms of the composition of GDP. In addition, any change in the GDP estimating schedule would have possible implications for BEA's international and regional economic accounts. If a decision that three current estimates are unnecessary were to be made on the basis of revision results alone, the best course might be to discontinue the preliminary estimate and to delay the final estimate 1 month so that its release would coincide with the advance estimate for the following quarter.
This article generally assumes that the same source data that exist now will be available in the future. It does not consider whether the reliability of the current estimates, specifically the later current estimates, might be increased. Any substantial increase in reliability would depend upon the development of additional and more timely source data./3/
This article is organized in four parts. The first part briefly describes the schedule for preparing the quarterly estimates, the source data used, and the types of errors that affect the estimates. The second part describes the measures of revisions that were calculated in the current and previous studies of revisions and examines the historical record of revisions. The third part presents the results of the current study in more depth, including a comparison of the reliability of the successive current estimates and an examination of whether revisions in the most recent years show signs of either increasing or decreasing in size. The final part considers why the advance estimates perform about as well as the two later current estimates; it also looks ahead at the need to consider the reliability of the composition of GDP estimates and some possible avenues for improving reliability.
GDP estimates for each quarter are prepared on a schedule that calls for three successive "current" estimates—"advance," "preliminary," and " final"—and for subsequent estimates prepared as part of annual and comprehensive NIPA revisions. The advance estimate is prepared about 1 month after the end of the quarter. For most components, the estimate is based on source data for either 2 or 3 months of the quarter. In most cases, however, the source data for the second and third months of the quarter are not final and are subject to revision by the issuing agencies. Where source data are not available, the estimate is based primarily on the estimator's judgment.
One month later, the "advance" estimate is replaced by the "preliminary" estimate, which is typically based on source data for all 3 months of the quarter. However, in some instances, the source data used for the "preliminary" estimates, particularly the data for the third month of the quarter, are subject to further revision.
One month later, the "preliminary" estimate is replaced by the "final" estimate, which incorporates revisions in source data for the third month of the quarter and quarterly source data for some components.
Each quarterly estimate is subject to three successive annual revisions (customarily released in July, this year released in August). The first annual revision incorporates further revisions in the monthly or quarterly source data and introduces some annual source data. The second and third annual revisions incorporate a broad range of annual source data. Each quarterly estimate is also subject to one or more comprehensive revisions, in which information from the economic and demographic censuses is incorporated.
More complete and more accurate information is generally available on an annual basis than on a quarterly or monthly basis. In many cases, annual data are based on larger samples or represent a complete universe count. In addition, annual data often correspond more closely to the desired definitions and therefore require less adjusting, or they may contain more information for making the necessary adjustments. As a result of these factors, quarterly estimates are obtained either by interpolating between annual estimates or by extrapolating from the most recent annual estimate.
Similarly, the annual estimates in many instances represent interpolations or extrapolations of the more complete and accurate information available in economic and demographic censuses, which are conducted every 5 years and 10 years, respectively.
The quarterly and monthly indicators that are used as interpolators and extrapolators are based largely on monthly or quarterly sample surveys conducted by various Federal statistical agencies. Exceptions include budgetary data from the Treasury Department, which are used to estimate Federal Government purchases, and tabulations of export and import documents filed with the Customs Service, which are used to estimate merchandise trade. Another type of exception occurs if no monthly or quarterly data are available—for example, data for some types of consumer purchases of services and for State and local government purchases. In such cases, the quarterly estimates are obtained by interpolation and extrapolation using annual data or related information.
An updated summary of the source data used for the NIPA's is included each year in the SURVEY OF CURRENT BUSINESS article that presents the annual NIPA revision (see the August 1993 SURVEY). For a list of methodological papers and for additional information about the NIPA's, see "A Look at How BEA Presents the NIPA's" in the February 1993 SURVEY.
The GDP estimates contain several kinds of error. The most obvious kind arises in the current estimates either from preliminary or incomplete tabulations of monthly or quarterly source data or, where source data are not yet available, from BEA's judgments. Error also arises in both the current and the latest available estimates because source data do not meet NIPA requirements in terms of timing, valuation, coverage, and definitions. For example, business firms report some types of data on a fiscal year, rather than a calendar year, basis; even though adjustments can reduce the effect of fiscal year reporting, the results differ from those obtained with calendar year reporting.
Error also arises from the sampling errors and biases in the monthly, quarterly, and annual surveys and from biases and other errors in the annual and periodic universe counts. Probably the most troublesome of errors are those due to the delayed introduction of births and deaths of business firms in sample surveys.
Seasonal adjustment is another source of error./4/ Even if the unadjusted source data were free of error, seasonal adjustment would introduce errors. Although some reduction in seasonal adjustment error appears to have been achieved over time in the current estimates through the use of concurrent seasonal adjustment and by combining ARIMA methods with the ratio-to-moving-average method of seasonal adjustment, such errors are still of considerable magnitude.
This article constitutes the fifth study of reliability BEA has conducted. The results of these five studies, which cover almost all years in the period 1947–91, are presented in table 2. Six summary measures are used to describe the revisions: Dispersion, bias, relative dispersion, relative bias, upward revisions, and directional misses. This article primarily considers dispersion and bias; these two measures effectively capture the picture provided by the other measures.
The measures are calculated as follows. Let P represent the percentage change in the current estimates, L the percentage change in the latest available estimates, and n the number of quarterly changes.
Dispersion is the average of the absolute values of the revisions:
Bias is the average of the revisions:
Relative dispersion expresses the dispersion as a percentage of the average of the absolute values of the latest available estimates:
Relative bias expresses the bias as a percentage of the average of the latest available estimates:
Upward revisions expresses the number of times that the current estimate of the quarterly change was revised up by the latest available estimate, as a percentage of the number of quarterly changes.
Directional misses expresses the number of times that the sign of the current estimate of the quarterly change differed from that of the latest available estimate, as a percentage of the number of quarterly changes.
Because of the shift from GNP to GDP as the featured aggregate in the 1991 comprehensive revision, the current study examined revisions for both aggregates, so as to provide a basis for consistent comparisons as well as for an assessment of revisions in the currently featured aggregate. In general, revisions in the two aggregates are very similar.
Both the current study and the one immediately preceding it did not incorporate two types of adjustments that were made in the earlier studies. These adjustments removed the effect of the changes in definitions of the GNP components, and of the change in base year in the constant-dollar estimates, that were made in comprehensive revisions. Consequently, the contribution to the revision that comes from source data is not as well isolated in the two more recent studies.
The measures in table 2 must be viewed in light of two aspects of the estimation process. First, within a given study, reduction in revision size over time is not evidence that reliability is increasing. This type of reduction, which can be observed within each study, reflects different vintages of the latest available estimates.
Second, a change in source data or estimating procedures, which one may assume affects the accuracy of the estimates, is not necessarily reflected in the revision size of estimates of a given vintage. For example, an improvement in the current estimates results in a permanent decrease in revision size. An improvement in the latest available estimates results in a permanent increase in revision size. Improvement in both the current and latest available estimates results in little change. Improvement that is introduced retrospectively into the latest available estimates, as is often the case, results in an increase in revision size for a period of years until the improvement is also reflected in the current estimates.
This section reviews the findings of the BEA revision studies to determine the extent to which dispersion in the revisions of GNP has been affected by improvements in source data and estimating procedures. In so doing, it considers what the record of revisions may suggest about changes in the total error. The review covers 1947–82 for the current-dollar estimates and 1968–82 for the constant-dollar estimates. The year 1982 is the last year for which the latest available estimates fully reflect a comprehensive revision. The review is restricted by the amount of detail retained in the historical record; the consideration of the total error necessarily involves speculation because, as noted previously, total error is unobserved.
In the current-dollar GNP revisions, dispersion averaged 3.0 percent or more in the first decade of the period (study 1 in table 2) and then declined substantially to 1.6 percent in 1958–63 (study 2) and 1.1 percent in 1968–72 (study 3). No doubt, the reduction reflects both a learning curve faced by the estimators in the first years and improvements in source data. In the comprehensive revision in 1958, which incorporated source data from the 1954 Economic Censuses, many techniques that had used pre-World War II information were updated or replaced./5/ In addition, the 1958 comprehensive revision incorporated a thorough reworking of the seasonal adjustments of the GNP components. These improvements were incorporated in revised estimates for the period 1947–57 as well as in estimates for subsequent periods. In the subsequent periods, some of these improvements were incorporated in both the current and latest available estimates, and others in only the latest available estimates.
It is reasonable to conclude that this decline in dispersion corresponds with increases in the accuracy of both the initial and final estimates subsequent to 1957. The improvements in estimating procedures and source data probably more than offset any changes in the structure of the economy in the 1950's and 1960's that were not captured in measured output.
Combining the information in studies 3 and 4, it is apparent that if dispersion had been calculated separately in the fourth study for the period 1973–77, it would have been about 3.0 percent. This increase from the relatively low levels of dispersion in the 1960's reflects the volatile economic conditions in the mid-1970's, which required estimators to use more judgmental techniques in preparing the current estimates. Because these techniques were probably not fully successful in coping with the changed conditions, one can conclude that part of the increase reflects a decrease in the accuracy of the current estimates.
Another consequence of the volatility in the mid-1970's was that the shortcomings it exposed led to the development of new source data and various improved techniques, particularly in the 1980 and 1985 comprehensive revisions. These improvements were introduced retrospectively into the estimates for the 1970's and from then on were incorporated into the current and latest available estimates. Some of these improvements are best viewed as working to restore the lost accuracy of all the estimates, and others as working to increase accuracy.
The decline in dispersion in 1978–82 (study 5) may be viewed as a return to a more normal level. It reflects a smaller impact from the development and introduction of statistical improvements in the 1991 comprehensive revision than in the two previous comprehensive revisions.
In the constant-dollar revisions, dispersion shows a roughly similar pattern to that in the current-dollar revisions from 1968 to 1982. The increase that may be inferred for 1973–77 and the subsequent decrease are, however, more pronounced. The difference reflects the development of improvements in deflation procedures in the 1980 and 1985 comprehensive revisions that were introduced retrospectively./6/
An upward drift from 1964–71 to 1978–82 in both current- and constant-dollar dispersion is probably not evidence that the accuracy of the initial estimates declined over this period. Rather, it likely reflects a lower rate of development and retrospective introduction of improvements in the 1960's than has since been the case. In light of the improved estimation procedures and source data that have been introduced, it is possible that the accuracy of the initial and final estimates of quarterly change has continued to increase since the 1960's. In considering this question, it is important to distinguish changes from levels, which are affected more by long-term developments, such as the shift to services, that are perhaps not adequately measured.
Table 3 shows dispersion for GDP and its major components for 1978–82 and 1983–91. These measures show that the incorporation of additional or more accurate source data in the preliminary and final current estimates of GDP does not substantially improve their reliability in comparison with the advance estimates. Dispersion declines only slightly over the successive current-dollar estimates of GDP. For 1978–82, it declines from 1.93 percentage points in the advance estimates to 1.82 percentage points in the preliminary and final estimates. For 1983–91, it declines from 1.17 percentage points in the advance estimates to 1.14 percentage points in the preliminary estimates and 1.15 percentage points in the final estimates.
Dispersion actually increases slightly over the successive constant-dollar estimates of GDP. For 1978–82, it increases from 1.64 percentage points in the advance estimates to 1.72 percentage points in the preliminary estimate and to 1.75 percentage points in the final estimate. The corresponding figures for 1983–91 are 1.25, 1.27, and 1.33 percentage points.
A similar picture emerges for the major components of GDP. In many cases, the advance estimates provided a smaller measure of dispersion than did the preliminary or final estimates. In 1978–82, the advance estimates provided the smallest dispersion in 4 of the 11 current-dollar components—PCE nondurables and services, residential investment, and Federal Government purchases—that accounted for almost 60 percent of GDP. In 1983–91, the advance estimates provided the smallest dispersion in 3 components—PCE services, residential investment, and State and local government purchases—that accounted for over 40 percent of GDP. The record for the advance constant-dollar estimates is about the same as that for the current-dollar estimates, though the share of GDP for which the estimates perform the best is smaller for 1983–91 (table 4). These results raise the question of whether one or both of the two later current estimates might be discontinued./7/
Table 3 also permits one to compare the size of the dispersion measure in GDP with that in its major components. In general, dispersion in the components was larger than that in GDP. The components with the smallest dispersion—about the same as that for GDP—were total PCE and PCE services. The components with the largest dispersion—roughly 6 to 8 times as large as that for total GDP—were gross private domestic investment and Federal Government purchases. The unusually large dispersion in these components reflected a change in the treatment of the Commodity Credit Corporation (CCC) that was introduced in the 1991 comprehensive revision, whereby the CCC was shifted from government enterprises to general government. This shift affected the timing and valuation of transactions and resulted in large, essentially offsetting revisions in Federal Government purchases and the change in business inventories. Dispersion was also quite large in current-dollar nonresidential structures in 1978–82 and in constant-dollar imports in 1983–91, reflecting statistical improvements introduced in the 1991 comprehensive revision.
In the current-dollar GDP estimates, relative dispersion is about 20 percent in 1978–82 and 17 percent in 1983–91. It is more than twice as large in the constant-dollar estimates—45–48 percent in 1978–82 and 37–40 percent in 1983–91. The primary reason for the larger relative dispersion in the constant-dollar estimates than in the current-dollar estimates is that the denominator in the constant-dollar ratio is smaller than that in the current-dollar ratio. Because of this "denominator effect," the constant-dollar measure would approach infinity in a prolonged period of zero growth.
In none of the current- or constant-dollar GDP estimates is the bias large enough to be statistically significant under assumptions of normality at the 5-percent confidence level (table 5). Bias in the current-dollar GDP estimates is negative in both 1978–82 and 1983–91. In the first period, the considerably larger bias in the advance estimates— -0.93 percentage point, compared with -0.55 percentage point in the preliminary estimates and -0.34 percentage point in the final estimates—is centered mainly in private fixed investment and can be attributed to BEA's judgmental projections. In the second period, the bias is less than -0.3 percentage point in all three estimates, and the successive estimates do not show a pattern of consistent improvement.
In constant-dollar GDP, bias is less negative than in current-dollar GDP for 1978–82 and is slightly positive for 1983–91. This damping of the bias reflects the rebasing of the constant-dollar estimates./8/ The current estimates for 1978–85 were stated in 1972 dollars, and those for 1986–91 in 1982 dollars; the latest available estimates are stated in 1987 dollars.
Table 5 also shows the bias measures for the major GDP components. There is significant bias—sometimes even at the 1-percent confidence level—in several of the current estimates of producers' durable equipment and nonresidential construction (and, thus, in nonresidential fixed investment and private fixed investment). These biases are negative in 1978–82 and positive in 1983–91. In addition, the three estimates of current-dollar PCE nondurable goods and the advance current-dollar estimates of exports show significant negative bias at the 5-percent level in 1983–91.
Table 6 examines revisions year by year to see if reliability of the GDP estimates appears to have changed in recent years. The table shows annual averages of dispersion and bias in the quarterly revisions between the successive current estimates and between the current estimates and the third annual revision estimates. For the revisions between the current estimates, the measures are shown for 1978–92; for the revisions between the current estimates and the third annual estimates, the measures are shown for 1978–89.
Estimates from third annual revisions were used in place of the latest available estimates in order to provide a more nearly comparable standard for the entire period against which to compare the current estimates. Use of third annual estimates abstracts from much of the effect of the economic census and other information that is used in the comprehensive revisions to revise previously prepared third annual estimates. However, it does not remove the effects of definitional changes in the comprehensive revisions, because for most quarters a comprehensive revision intervenes between the current estimates and the third annual estimates.
A visual examination of the annual averages reveals a tendency for the largest entries for both dispersion and bias to be concentrated in the first several years of the period and in 1981 particularly. The overall impression, however, is one of little change. Table 7 shows the results of fitting linear trends to the quarterly revisions that underlie the annual averages from table 6 for two periods—one beginning with 1978 and the other with 1983. A reduction in dispersion is indicated by a negative trend; a reduction in bias is indicated by either a positive or negative trend in which the last trend value is closer to zero than the first trend value.
Although the trends are consistent in most cases with reductions in dispersion and bias, only the slope coefficients for dispersion and bias in the revisions between the preliminary and final current estimates for the period beginning with 1978 are significant at the 5-percent confidence level. Thus, there is not strong evidence of an increase in reliability. Further, several large revisions like those in the first years of the period could very well be encountered again, erasing any suggestion of recent improvement.
The absence of much improvement in the successive current estimates has puzzled both users and estimators for some time. Two seldom recognized factors contribute to the observed result: (1) The small role played by the data for the second and third months of a quarter in determining the change from the previous quarter, and (2) certain sources of error in the preliminary and final estimates to which the advance estimates are immune. In addition, advance estimates of GDP and its major components may benefit more from offsetting errors in the detailed components than the later current estimates; that is, the revisions of the advance estimates may be more negatively (or less positively) correlated than those of the preliminary and final estimates.
This section first discusses the two factors and then addresses the problem of quantifying the total error introduced by the second factor, which embodies seasonal adjustment errors and errors related to the estimation process for certain components. The section concludes with a discussion of the implications for the future.
The role played by the data for second and third months of the quarter in determining the change from the previous quarter is small. The change from the second to the third month receives a weight of only one-ninth in the determination of quarterly change. The weight of the second and third months together is only one-third. The weight of the first month is another one-third, and the second and third months of the previous quarter receive the remaining one-third./9/ Consequently, errors in neither the preliminary source data for the second and third months of a quarter nor in the judgmental projections used in lieu of source data affect the quarterly change as much as one might intuitively expect.
The seasonal adjustment of source data for the final current quarterly estimate introduces errors not present in the judgmental projections, which are developed on a seasonally adjusted basis. The seasonal adjustment factors for the current year are derived from the seasonal patterns of recent preceding years. (The concurrent seasonal adjustment method also includes the seasonal pattern of the current year.) The factors are revised as additional data become available, and they eventually reflect the average seasonal pattern of a period of years that extends symmetrically on either side of the given year. The difference between the initial estimate of the seasonal factor and the final estimate prepared some years later is an error that becomes part of the revision in the final current estimate. To the extent that they are based on judgmental projections, the advance and preliminary quarterly estimates do not contain this error.
PCE services.—PCE services is one of the major components in which the dispersion in the revisions of the advance estimates is smaller than that in the later estimates. It is difficult to isolate the causes for this outcome at the detailed component level because, as described later, the necessary information is not readily available. Informed judgment suggests that two types of situations may have contributed: (1) Where BEA's methodologies that piece together disparate data for the preliminary and final estimates of a detailed component provided erroneous results, and (2) where a survey designed to estimate a specific type of consumer expenditure provided erroneous results.
An example of the first type of situation is the estimates of expenditures on airline fares. Prior to the 1986 annual revision, the preliminary and final estimates were derived from the product of passenger miles and the consumer price index (CPI) for airline fares. In the 1986 annual revision, this information was replaced with data on airline revenues. The previous methodology had performed adequately until discount airfares became widespread. These discounted fares were not reflected in the CPI, and it took BEA some time to recognize the seriousness of the problem and to substitute more appropriate information for the CPI. During this period, the judgmental projection used for the advance estimate performed better than the methodology used for the later current estimates.
In another example, a similar situation has affected the estimates of consumer purchases of gas and electricity. In this case, the advance estimate incorporated a projection, based on degree days, that was replaced in the later current estimates by data from trade associations on units of energy consumed. Investigation showed that this approach introduced error because the variation in utilities' billing cycles was not properly taken into account. Consequently, the later current estimates did not agree closely with the first annual revision estimates, which were prepared from quarterly information on the revenue of utilities.
The second type of situation affected the later current estimates of components of PCE services that were based on sales reported in the Census Bureau's monthly survey of selected services, which was discontinued in the early 1980's. This survey had a small sample, so the monthly estimates were subject to large error. The sum of the monthly survey data did not agree closely with annual data from sources such as tabulations of business receipts from the Internal Revenue Service. Consequently, the projections used for the advance estimates, which reflected past trends in the annual data, outperformed the monthly survey data.
One would like to say that such situations no longer exist, because in each of the three cited instances it was possible to improve the procedures used for the later current estimates. However, the advance estimates of PCE services performed better in terms of dispersion than the later current estimates for 1988–90, which suggests (1) that some old problems persist or some new ones are emerging or (2) that the advance estimates benefit from offsetting error to a greater degree than the later estimates.
Federal Government purchases.—Federal Government purchases is another major component for which the advance estimate performs well in comparison with the later current estimates. The advance estimate depends largely on summary financial data, trends in projected expenditures from the administration's budget, and scheduled deliveries of military equipment. For the final estimate, such information is replaced with detailed financial data and actual deliveries; however, this information is sometimes rough and incomplete. Consequently, the advance estimate, which may be described as reflecting expected trends, may agree more closely with the latest available estimate than the final estimate.
The difference between the revisions to the advance estimate of a detailed component and those to the latest available estimate reflects three types of error: (1) The error in the preliminary monthly source data used for the advance estimates that is corrected in the revised monthly source data used for the final current estimate; (2) the error in the judgmental projections used in lieu of source data for the advance estimate; and (3) the error in the source data used for the final current estimate (including seasonal adjustment error) that replace the judgmental projections used for the advance estimate. The total revision in the advance estimate reflects the first two types of error; the total revision in the final current estimate reflects the third type. One should note that the second type of error, like the third, is determined with respect to the data as they stand several years later.
It would be desirable to determine the size of each of the three types of error at the detailed component level. It would also be desirable to determine the extent of correlation among the component revisions. However, such analyses are not possible, because each vintage of each estimate and the associated source data are not available in a readily usable form. Nevertheless, it may be worthwhile to attempt to confirm and quantify the error in a few detailed components. In addition, the database used in the calculation of the alternative measures of real GDP might be extended so as to retain, not only the latest available estimates, but all the vintages of estimates at the detailed level at which deflation is carried out./10/ Over time, this database would be useful in exploring the outcomes of the estimation process and in developing improved procedures.
An analysis at a fairly high level of aggregation suggests that revisions in seasonal factors may be large enough to contribute significantly to the observed results. Specifically, in some of the series examined, seasonal-factor revisions are as large as the variation in the irregular component./11/ Given that the error introduced by a judgmental projection is likely to be smaller than the irregular variation, this result suggests that for some detailed components, seasonal-factor revisions may play a significant role in causing the revision in the final estimate to be as large as that in the advance estimate.
The reliability of the composition of the quarterly GDP change shown by the advance estimate requires further examination. It is possible that this composition improves with successive current estimates and that the advance estimate is not a suitable substitute for either of the later estimates. This is an especially important issue to forecasters because the composition of the latest quarter's change can affect the trajectory of GDP.
In order to assess the compositional reliability of the advance estimate, it is necessary to select one or more statistical measures. One such measure would be the absolute values of the revisions in percentage shares of the major GDP components in the constant-dollar GDP change (in dollars); the revisions would be averaged over all components and over time. This measure is not necessarily ideal. For example, gross under- or over-estimates of the change in GDP achieved with "correct" proportions in the components would yield a perfect score in terms of composition, despite varying distortions in the component percent changes. It might be appropriate to use a weighted variant of this measure and to supplement it with a measure of the degree of offsetting error, such as the average ratio of the absolute revision in GDP to the sum of the absolute revisions in the components.
In addition to compositional reliability, it would be prudent to consider the relative merits of the successive estimates from the standpoint of the business cycle. One may also want to consider the size of the range from the largest positive revision to the largest negative revision. The advance estimate does not perform quite as well as the later estimates with respect to this property.
With respect to BEA's estimating procedures, the combining of disparate source data in the absence of more suitable data for the current estimates of a detailed component requires careful evaluation. Such procedures are difficult to justify if they do not lead to smaller revisions than do judgmental projections. As noted previously, development of a detailed database would facilitate evaluation. It would also facilitate the development of econometric projection techniques and their comparison with judgmental projections. In this respect, it should be noted that a recently completed study found that judgmental projections compared favorably with econometric techniques for certain import and export components./12/
The question of whether the reliability of the seasonal adjustments on which the current quarterly estimates are based can be improved merits attention. The seasonal adjustments used for source data should be designed from the standpoint of accurately measuring quarterly change. Little attention has been paid to whether the currently used seasonal adjustment procedures are suitable from this standpoint. In addition, BEA should consider whether more use of concurrent seasonal adjustment, with or without ARIMA, would improve reliability.
Finally, because the third month of a quarter receives little weight in the estimate of change for that quarter, there may be instances in which efforts to reduce revisions in the quarterly GDP estimates should focus on improving the final monthly source data rather than the preliminary monthly source data. As shown earlier, for a survey with three successive monthly estimates, two-thirds of the advance quarterly change is based on three monthly final estimates, while only one-ninth is based on the initial monthly estimate for the last month of the quarter.
1. See Allan H. Young, "Evaluation of the GNP Estimates," SURVEY OF CURRENT BUSINESS 67 (August 1987): 18–42. (Prior to 1991, GNP was used in place of GDP as the principal economic indicator of U.S. production.) The Office of Management and Budget requires an evaluation of all principal economic indicators, of which GDP is one, every 3 years. An evaluation was not carried out in 1990, because a new standard against which to determine the reliability of the initial estimates was about to be provided by an upcoming comprehensive revision of the NIPA's.
2. In this article, quarterly percentage changes are at seasonally adjusted annual rates, and revisions are calculated as the difference between these changes.
3. For a recent study that calls for improving the source data, see Martin Fleming, John Jordan, and Kathleen M. Lang, "Measurement Error in the U.S. National Income and Product Accounts: Its Nature and Impact on Forecasts." Paper presented at the 35th Annual Meeting of the National Association of Business Economists, Chicago, Illinois, September 19–22, 1993.
4. and monthly NIPA estimates are seasonally adjusted if necessary. Seasonal adjustment removes from the time series the average impact of variations that normally occur at about the same time and in about the same magnitude each year—for example, weather, holidays, and tax payment dates.
5. U.S. Department of Commerce, Bureau of Economic Analysis (formerly Office of Business Economics), U.S. Income and Output, (Washington, DC: U.S. Government Printing Office, 1958).
6. "Evaluation."
7. Previous studies have also shown that the advance estimates perform well in comparison with the later estimates, but as shown in table 2, the results are not as clear-cut in some periods as in others.
An early study concluded that the advance estimate might be sufficient; see Rosanne Cole, Errors in Provisional Estimates of Gross National Product, National Bureau of Economic Research Studies in Business Cycles, No. 21 (New York: Columbia University Press, 1969). See also Stephen K. McNees, "Estimating GNP, The Trade-off Between Timeliness and Accuracy," New England Economic Review (January/February 1986): 3–10; and Joseph W. Duncan and Andrew C. Gross, Statistics for the 21st Century (The Dun and Bradstreet Corporation, 1993).
8. One should note that the effect of the damping of the current-dollar bias that results from rebasing will not come into play in the recently introduced alternative measures of GDP. As a result, the constant-dollar bias in these measures will likely be about the same size as the current-dollar bias. For information about the alternative measures, see Allan H. Young, "Alternative Measures of Change in Real Output and Prices, Quarterly Estimates for 1959–92," SURVEY 73 (March 1993): 31–41.
9. This may be demonstrated as follows: Let Q1=X1+X2+X3 and Q2=X4+X5+X6, where X1,X2,...X6 are successive months of source data. Then, if d4=X4-X3, d5=X5-X4, and d6=X6-X5, the months in Q2 may be stated as X4=X3+d4, X5=X3+d4+d5, X6=X3+d4+d5+d6, and Q2=3X3+3d4+2d5+d6. Therefore, the quarterly change is
Q2-Q1 = (3X3+3d4+2d5+d6)-(X1+X2+X3)
[(X3-X2)+(X3-X1)]+[3d4+2d5+d6].
Introducing the notation for monthly differences, the first bracketed term becomes d3 + (d2 + d3), and Q2 - Q1 = d2+2d3+3d4+2d5+d6. Normalizing the coefficients on the d's provides weights of 1/9, 2/9, 3/9, 2/9, 1/9 for the five monthly changes that determine the quarterly change.
10. Young, "Alternative Measures of Change."
11. The irregular component is the residual after the systematic components—the seasonal and trend-cycle—are determined by the seasonal adjustment method.
12. Albert A. Hirsch and Michael A. Mann, An Analysis of the Use of Time-Series Models to Improve Estimates of International Transactions, Bureau of Economic Analysis Working Paper 7 (Washington, DC: Bureau of Economic Analysis, April 1993).
