Services Research Outcomes Study (SROS)

1. WEIGHTING OF DATA
   • Weights
   • Weights
   • and Post-Stratification
   • to the Weighting Plan

WEIGHTING OF DATA

Overview

The purpose of weighting survey data is to adjust for differences between the composition of the sample and the composition of the population of interest. These differences arise from two sources. The first is sources related to the sampling design (e.g., differential sampling rates for various types of facilities and, possibly, differential sampling rates by various respondent characteristics). Second, these differences also arise because of differences in cooperation rates (e.g., not every facility or respondent will agree to participate in the survey, and members of some groups may be more likely to cooperate than members of other groups.) Differences between the composition of the sample and the universe may also arise because of various forms of undercoverage (undersampling, disproportionate non-response rates, etc.). Weights are used to compensate for all of these differences between the sample and the population. The sampling plan for SROS was developed to achieve these goals; this plan is described below.

Facility Weights

Facility-level weights are needed to estimate characteristics of the population of facilities. These weights are essentially the products of two factors: (1) the reciprocal of the facility selection probability, and (2) an adjustment factor for nonparticipation.

The probability of selecting a facility is:

{ pi_{j}~=~n{X_j over X_+}}

where:

¶_j is the probability of selecting facility j;

n is the number of facilities;

X_j is the size of facility j; and

X₊ is the sum of the sizes across facilities.

The facility’s base weight is the inverse of its probability of selection:

After the facility base weights have been computed, they can be adjusted for facility-level nonparticipation. The nonresponse-adjusted weight, {W_j}^*, is the base weight, W_ij , multiplied by the inverse of the weighted response rate for a given adjustment cell a:

{{W_j}^* ~= ~left ({{N hat_a}^ES}over {{N hat_a}^PS~} right)~ W_j}where

{N hat _a}^ES is the sum of the base weights of the eligible selected facilities in adjustment cell a, and

{N hat _a}^PS is the sum of base weights of participating facilities within adjustment cell a.

Adjustment cells for facilities are defined according to the original sampling strata.

Respondent Weights

Respondent weights are computed using the same basic procedure used for calculating facility-level weights. First, a base weight is developed. The base weight for Respondent k within Facility j is derived from the final facility weight, {W_j}^*, and reflects the within-facility selection probability

W_jk~=~{W_j}^*``left({N_ij} over {~b`O_{g(k)}}right)where:

_{N_ij} is the total number of eligible respondents in Facility j .

b is equal to the desired number of respondent selections per facility (It is a constant to be determined as a function of the desired total sample size, the degree of oversampling that is required, and the expected overall participation rates of facilities and respondents).

_{O_{g(k)}} is a selection rate adjustment factor associated with the oversampling rate for the group, g(k), to which the respondent belongs. If no oversampling is required, then _{O_{g(k)}}= 1.

Next, the initial respondent weights are adjusted for nonresponse. Again, the initial respondent weights are multiplied by the inverse of the weighted response rate within an adjustment cell.

{W_jk}^*~=~ left({{M hat_a}^ESt} over {{M hat _a}^PSt}``right)`W_jkwhere:

{M hat_a}^EStis the sum of the weights of eligible selected respondents within adjustment cell a, and

{M hat_a}^PStis the sum of weights of participating respondents within adjustment cell a.

Undercoverage and Post-Stratification

Bias in an estimated proportion or mean due to undercoverage of facilities on the frame depends on two factors. The first is the proportion of the population that was excluded from the frame, p. The second is the difference between the parameter for the facilities on the frame and those who were omitted on the frame. That is :

Bias ~ = ~ p `~ ( ` I _ {nc}`- ~` I _ c) ~, where:

_{I _nc} is the parameter (mean or proportion) characterizing the omitted group of facilities, and

I _c is the corresponding parameter characterizing the included group of facilities.

Using this information, the nonresponse adjusted weights can be post-stratified to the control totals by stratum.

Modifications to the Weighting Plan

The estimates in this report were calculated using weights developed by NORC based on specifications approved by the Office of Applied Studies on April 3, 1996, and revised on May 17, 1996. The purpose of these weights was to allow the sample responses to be used to draw inferences about the population of drug users from which they were drawn. The May 17, 1996, revisions to the weighting plan were necessitated by three exceptions which needed to be made to the original specifications.

The first of these exceptions was simply that due to the nature of the data set (including both questionnaire responses and abstracts of archival records) and the different analyses to be performed, three weights were calculated. These differed based on the way the deceased were classified. More specific information on these weights and how they were used in this report is presented below.

The second exception was made necessary by the fact that it was not possible to determine the number of eligible clients in sample facilities that did not cooperate in DSRS Phase II. Because these data were not available, the assumption was made that, within each of the four strata, the weighted cooperation rate of sample facilities in DSRS Phase II would be equal to the unweighted cooperation rate. That is, it was assumed that within each stratum, the percentage of sample facilities that cooperated in DSRS Phase II was equal to the percentage of sample clients who were cooperating in DSRS Phase II facilities.

The third exception involved the post-stratification of the weights within strata using the original stratum totals of eligible clients on the DSRS sampling frame. That is, within a stratum, the number of eligible clients was computed as the sum of the reciprocals of the overall client selection probabilities for sample facilities in that stratum. The decision to post-stratify in this way was made due to the inability to adjust the weights precisely for the effect of facility noncooperation, as described above.

The three weights referred to in the discussion of the first exception above were as follows. "WT1" treats the deceased as respondents. This weight was used for analyses of the abstracted records because in this case, data are available for both living and dead respondents, and comparisons involving both groups were to be computed. "WT2" treats the dead as nonrespondents, and "WT3" treats them as ineligible. The latter was chosen for the analysis of the questionnaire data in this report. This seemed intuitively the more logical disposition for the sample members coded as deceased because they had died before the administration of the questionnaire. However, it should be noted that because this weight was used for the analyses, the population to which the results in this report refer is the population of drug users in treatment in the SROS timeframe who survived at least until the administration of the questionnaire.

This page was last updated on June 03, 2008.

SAMHSA, an agency in the Department of Health and Human Services, is the Federal Government's lead agency for improving the quality and availability of substance abuse prevention, addiction treatment, and mental health services in the United States.