Chapter 8.
National Compensation Measures

Reliability of Estimates
Errors in the estimates include both sampling errors and nonsampling errors.

Sampling errors occur because the sample makes up only a part of the population. The sample used for this survey is one of a number of possible samples that could have been selected under the sample design, each with a corresponding estimate. A measure of the variation among these sample estimates is the standard error.

The ECI, ECEC, and the NCS wage publications all use some variation of balanced repeated replication (BRR) to estimate the standard error. Standard errors presently are not estimated for the other benefits products (such as benefit access and participation rates). The procedure for BRR is first to partition the sample into variance strata, composed of single sampling strata or clusters of sampling strata, and then to split the sample units in each variance stratum evenly into two variance primary sampling units (PSUs). Next, half-samples are chosen so that each half-sample contains exactly one variance PSU from each variance stratum. Choices are not random, but designed to yield a “balanced” collection of half-samples.

For each half-sample, a “replicate” estimate is computed with the same formula for the regular or “full-sample” estimate, except that the final weights are adjusted. If a unit is in the half-sample, its weight is multiplied by (2-k); if not, its weight is multiplied by k. For all NCS publications, k = 0.5, so the multipliers are 1.5 and 0.5. (Some of the weighting adjustments done as part of the calculation of final weights also are recalculated for each replicate.) The BRR estimate of standard error with R half samples is

where the summation is over all half-samples r = 1,...,R,

is the r-th replicate estimate, and

is the full-sample estimate.

ECEC and NCS wage publications display the standard error as a percentage of the full-sample estimate. This is called the percent relative standard error and is given by .

Nonsampling errors result from not collecting data within a specified sample. The primary sources for nonsampling errors are survey nonresponse, and data collection and processing errors. Nonsampling errors are not measured, but procedures have been implemented for reducing them.

Survey nonresponse includes unit nonresponse and item nonresponse. Unit nonresponse is treated with weight adjustments that redistribute the weights of nonrespondents to similar respondents based on characterics such as industry, establishment size-class, and occupational group. Some adjustments are applied to nonrespondent establishments, and some are applied to nonrespondent jobs within partial respondent establishments.

Item nonresponse occurs when some respondent units do not provide data for all items being collected. Item nonresponse is treated by item imputation. In item imputation, missing values for an item are replaced by values derived from respondents with similar characteristics who completed the item.

Data collection and processing errors are mitigated primarily through quality assurance programs. These programs include the use of data collection reinterviews, observed interviews, computer edits of the data, and systematic professional review of the reports in which the data are recorded. The programs also serve as a training device to provide feedback to the field economists, or data collectors, on errors. They provide information on the sources of errors that can be remedied by improved collection instructions or computer-processing edits. Extensive training of field economists also is conducted to maintain high standards in data collection.

Next:Uses and Limitations