Errors of Nonobservation

Finally, several potential sources of nonsampling error and bias result from errors of nonobservation.  The 1994 MECS represents, in terms of sampling coverage, the mail frame of the 1994 ASM or 98 percent of the manufacturing universe, which is consistent with the 1991 MECS.

Unit Nonresponse

Even though the MECS is a legislatively mandated survey and sampled establishments are given sufficient opportunity and time to respond, nonresponse occurs in the MECS and is accounted for in a nonresponse adjustment of sampling weights. Clearly, had these adjustments not been performed, the estimates produced from only the responding establishments would not have been representative of the target universe for the MECS. Such estimates would have been biased. Adjusting the sampling weights to reflect the target universe is an attempt to mitigate the potential effects of such a bias.

Adjustment factors are calculated for each of the 72 published strata to account for the variation of nonresponse between strata. Each stratum represents a relatively homogeneous subgrouping of establishments with respect to primary product output and level of fuel consumption.

Implicit in that procedure is the assumption that primary product output and level of fuel consumption are highly correlated with energy expenditure patterns, so that the establishments within a stratum would also be homogeneous with respect to the quantities, types, and shares of energy consumed as fuels and for nonfuel purposes. Also, the weight adjustment method assumes that the relationship between survey variables of interest and the control variable used for constructing the adjusted sample weight is the same for the population covered by MECS respondents within an adjustment stratum as it is for the rest of the population within that stratum.

To the extent that the nonresponding establishments within the adjustment stratum share the energy expenditure patterns of the responding establishments within the strata, the resulting adjustments to the MECS estimates will tend to be minimally biased. If, on the other hand, the energy expenditure patterns of the responding and nonresponding establishments differ substantially, the resulting adjustments are potentially biased, and the overall estimates may not accurately represent the originally targeted MECS universe.

Item Nonresponse

Item nonresponse is the type of nonresponse that occurs when an item (or several items) is missing in an otherwise completed questionnaire. In 1994, MECS expanded its collection of establishments' characteristics. MECS now collects economic information, such as floorspace, motor purchase evaluations, reasons for fuel switching, energy management activities, and energy efficient technologies. Although a Response Analysis Survey of 1991 respondents indicated that a record-keeping system tracked these items, some establishments did not have a record-keeping system which would have enabled them to respond to these types of questions. To the extent that information systems excluded economic items, the MECS incurred item nonresponse.

Some surveys impute values for item nonresponse. The MECS did not impute for most of these missing items. The complexities and inherent heterogeneity of manufacturing establishments prevent the use of imputation techniques for most of these missing items. However, budget restrictions and timeliness issues halted respondent recontacts before item nonresponse could be eliminated.

MECS has a full reporting of energy consumption from all responding establishments. However, economic variables (such as value of shipments and value added) have historically been obtained from the Annual Survey of Manufactures (ASM). The change in sampling frames from the ASM to the CM had an indirect result on item nonresponse; that is, some establishments in the MECS sample did not have economic data (value of shipments and value added) because they were not included in the 1994 ASM sample. One approach considered by EIA was to query these establishments on their shipments. Unfortunately, duplication issues with the ASM precluded the MECS from querying establishments on their shipments and value added. Hence, these establishments did not have the ability to report their economic data.

Unlike the establishment characteristics, like floorspace, value of shipments and value-added estimates were imputed for all MECS establishments that were not in the sample. Of the 19,292 MECS sample respondents, 13,011 cases have ASM-reported value of shipments data and ASM-reported value-added data. Of the remaining cases, we only have the total number of employees and payroll from the Standard Statistical Establishment List (SSEL). Using aggregate payroll data, by industry (at the four-digit SIC level), value of shipments and value added were imputed by forward indexing. Using value added as an example, this imputation technique is expressed as

where SW₉₄ is payroll from the 1994 SSEL and SW₉₂ and VA₉₂ are payroll and value added from the 1992 Census of Manufactures (CM). Value of shipments was imputed analogously.

The imputation model was evaluated by testing how well it predicted known 1994 ASM value-added data. Matching was done for SIC industries 2000 through 3999. If an establishment was indicated as out-of-business, the observation was deleted. If the ratio SW₉₄/SW₉₂ was less than 1/8 or more than 8, the data were considered either as outliers or as recording errors. Such observations (696 out of 52,227) were removed prior to analysis.

A paired comparison t-test was done for the percent difference between VA_imp and actual VA₉₄ to determine if the mean percent difference was statistically significant from zero. At a 95-percent confidence level, the null hypotheses (the mean percent difference between VA_imp and actual VA₉₄ is zero) could not be rejected. Hence, there was no difference statistically between Va_imp and VA₉₄. A t-test using value of shipments yielded the same conclusion.

Table 1 provides the counts, percents, and weighted percents for the imputed estimates of value of shipments and value added by industry.

More detailed information on sources of nonsampling error in the MECS can be found in the methodological report.

See footnotes at end of table.

Table 1. Number of Establishments and Weighted and Unweighted Percentages of the Imputed Value of Shipments and Value Added (Continued)

W = Withheld to avoid disclosing data for individual establishments.

Source: Energy Information Administration, Office of Energy Markets and End Use, Energy End Use and Integrated Statistics Division, Form EIA-846, "1994 Manufacturing Energy Consumption Survey."

Top of Page
Energy Intensity Page

http://www.eia.doe.gov/emeu/mecs/mecs94/ei/nonobs.html