Discussion of the Sample Changes for the Weekly Natural Gas Storage Report |
|
|---|---|
|
August 21, 2008 The Energy Information Administration (EIA) recently modified its weekly underground natural gas storage sample and sample selection procedure to reflect changes in the industry and improve data quality. The transition from the old to the new sample began with the release of the Weekly Natural Gas Storage Report on May 22, 2008, and was completed with the release of June 12, 2008.1 Prior to implementation of the new sample in the published estimates, both samples were used in parallel to estimate weekly natural gas storage stocks from May 4, 2007, through May 9, 2008. This report describes the change in sampling procedure for the current sample and examines the differences between results from the old and new samples. The results are compared with each other as well as with the monthly data from the EIA-191 survey, "Monthly Underground Gas Storage Report." Major findings of this examination follow.
Sample Rotation EIA survey samples are changed periodically as a standard practice.3 The validity of the results from a sample depends on the completeness and accuracy of the frame from which the sample is drawn. The frame for the EIA-912 survey, the "Weekly Underground Natural Gas Storage Report," is drawn from the data from Form EIA-191, "Monthly Underground Natural Gas Storage Report." The Form EIA-191 is completed by all operators of underground natural gas storage fields in the Lower 48 States. Data are provided for storage volumes in each storage field and reservoir. The frame for the EIA-912 survey is prepared by aggregating volumes of working gas in storage by storage operator and region. Over time, the size and number of the storage fields operated by a given company changes and operators enter or exit the industry. As these changes occur, eventually the initial frame is not representative of the current industry. This situation is addressed most often by selecting a new sample from a more current frame, which better reflects the current industry. Selection of a new sample does not necessarily require or reflect a change in estimation methodology. In addition to the changes that occur when a new sample is selected, the number of sample companies changes incrementally over time as companies exit or enter the industry. Storage fields occasionally cease operations or are sold to another company. If a sample company ends operations at all fields, the company is removed from the sample. If a sample company sells its storage fields to another firm that already is in the sample, the number of sample companies goes down by one. On the other hand, the birth of a new storage operator can be handled either by including the new company as a nonsampled company that will be estimated within the system, or by including the new company in the sample. The Previous (2005) Sample The 2005 sample was selected in two phases. Phase 1. A set of 55 companies was selected using a stratified sample design with probability proportionate to size sample selection, with a measure of size equal to the average between selected October and March monthly volumes. The operators in the sample were classified by three regions (the East, West, and Producing regions). Reported volumes of working gas in storage as reported on the Form EIA-191 were aggregated by storage operator and region to prepare the sampling frame for the EIA-912 for each region. For each region, the data were divided into two groups. The first, the certainty group, consisted of the larger operators in the region as well as all operators with storage fields in more than one region. All operators in the certainty group were selected for the sample. The second, the noncertainty group, consisted of all other operators, who were selected for the sample with a probability proportional to their size. The sample of companies was selected from the list of operators to achieve a target standard error of the estimate that was no greater than 5 percent for total working gas in storage in each region.4 Phase 2. The group of companies from the first phase of sampling was augmented with seven additional companies in order to represent salt-field operators in the Producing Region explicitly in the estimation. Salt companies were selected on an analytic basis in which potential companies for the survey were evaluated with respect to the resulting historical estimates for working gas in storage in the Producing Region. The selection was based on the average differences and the largest differences for any single month over a 3-year period (2001-2003). Both sets of differences were examined in order to minimize potential sampling error in general, as well as during a period of unusual circumstances such as when stock levels are exceptionally low. The Current (2008) Sample The selection procedure of the current (2008) sample was similar to that used to draw the companies in the first phase of the previous (2005) sample. The difference was that this selection was drawn with respect to the four strata: East, West, Producing-Nonsalt, and Producing-Salt, rather than the three regions only. Companies were selected with certainty, as described for the previous (2005) sample, and with noncertainty from the list of operators to achieve a target standard error of the estimate that was no greater than 3 percent for total working gas in storage in each region. Since the salt companies in the Producing Region were considered as a separate stratum, there was no need for a second sample selection phase to augment the sample, as had been done for the previous (2005) sample. This process ultimately resulted in a set of 70 respondents for the EIA-912 survey, or a net gain of 4 respondents from the 66 that were in the survey at the time of the transition.5 Comparison of Weekly and Monthly Natural Gas Storage Estimates The EIA-912 system was operated with both samples for the period from May 4, 2007, through May 9, 2008, to allow for a comparison and assessment of the results before implementation of the current (2008) sample.6 The published data, based on the 2005 sample, and the test results from the 2008 sample were compared with the monthly data from the EIA-191 survey. As a census survey, rather than a sample survey, the monthly data generally are considered more accurate because they do not include sampling error (although non-sampling error, such as errors in measurement or reporting errors, still could affect the census data). A comparison of national results by month shows smaller differences between the weekly estimates with the current (2008) sample and monthly data in each month (Table 1). The average difference from the current (2008) sample was -4 billion cubic feet (Bcf), which is less than one-quarter of the -17 Bcf difference based on the previous (2005) sample. These differences correspond to average percent differences of -0.7 percent and -0.2 percent for the results from the previous (2005) and current (2008) samples, respectively. The maximum difference for the current (2008) sample was 35 Bcf, which is less than 80 percent of the absolute value of the maximum difference of -44 Bcf based on the previous (2005) sample. Thus, the estimates from the current (2008) sample were less prone to larger differences than those produced by the previous (2005) estimates.
An examination of summary statistics for both regional and national estimates indicated that the current (2008) sample resulted in generally smaller differences from the monthly data at the regional and national level (Table 2). The average differences from the monthly data declined with the current (2008) sample, indicating a closer fit with the monthly data on average. Additionally, the maximum absolute differences from the current (2008) sample are lower, indicating improvement in all regions.
A Comparison of the Results from the Two Samples Working Gas Stocks. The EIA-912 system produced weekly estimates based on the two samples for the period from May 4, 2007, through May 9, 2008.7 The average difference between the two samples is 5 Bcf, with a maximum of 16 Bcf (Table 3).
The differences between the two series are not statistically significant. This can be seen in the results for two examples: June 29, 2007 (the week of the largest difference: 16 Bcf), and May 9, 2008 (the week preceding the beginning of the transition). The national and regional results for June 29, 2007, differ by less than the corresponding standard errors for either series (Table 4). The standard errors of the estimates also are well below the respective target levels for both the previous (2005) sample (5 percent) and the current (2008) sample (3 percent).
The Transition Period - May 16, 2008, through June 6, 2008. The transition from the previous (2005) sample to the current (2008) sample was accomplished over a 4-week period. The published stock values were calculated as a weighted average of the estimates based on both samples, with a declining weight on the estimates from the old sample. The weights varied by 25 percent each week, starting at 75 percent for stocks based on the 2005 sample. The transition began with the May 16, 2008, data (released May 22) and was complete with the June 6, 2008, data (released on June 12, 2008) (Table 5).
The Impact on the Implied Net Change. EIA estimates the stock of working gas in storage. Nonetheless the implied net change between stock estimates in successive weeks is of considerable interest to the public. The average difference in net change volumes between the two series over the test period of May 4, 2007, through May 9, 2008, was 0 Bcf, with a maximum of 6 Bcf. During the period of transition, the difference between the published net change value and the values from the previous (2005) sample was no more than 3 Bcf (Table 6). The published weekly net change values in the first 3 weeks of the transition were slightly higher than the net change would have been under the old sample because the published net change included an additional increment to shift from the previous level of stocks to a slightly higher level based on the new sample. The net change in the final week was less than that based on the previous (2005) sample because the difference between the two series had diminished in the third and fourth weeks.
1 The affected working gas estimates were for weeks ending May 16, 2008, through June 6, 2008. |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||