|
 |
|||||||||||
 |
|
|||||||||||
|
|
||||||||||||
A coordinated 5-year sample design was developed for 1999 through 2003. The sample design for the 2003 main study, as a subsample of the 5-year study, consisted of a deeply stratified, multistage area probability design. Exhibit 2.1 presents details of the sample design.
The coordinated 1999–2003 design calls for a 50-percent overlap in first stage units (area segments) between each successive year of the 5-year study following completion of the 1999 survey.
The first stage of the sample selection procedures began by geographically partitioning each state into roughly equal-sized field interviewer (FI) regions. These regions were formed as a means of stratification so that each area would yield roughly the same expected number of interviews during each data collection period. This partitioning divided the United States into 900 FI regions made up of counties or groups/parts of counties.
These FI regions were subdivided into smaller geographic areas—called segments—that served as the primary sampling units. In general, segments consisted of adjacent census blocks and were equivalent to area segments selected at the second stage of selection in National Surveys on Drug Use and Health (NSDUHs) conducted prior to 1999. A total of 96 segments per FI region were selected (with probabilities proportional to size): 24 to field the 5-year study and 72 to serve as backups in case of sample depletion or to field any supplemental studies the Substance Abuse and Mental Health Services Administration (SAMHSA) may request. For the 2003 survey, a total of 7,200 segments within the 900 FI regions were selected. Of the total, 3,600 segments were overlap segments used during the 2002 survey, 3,572 segments were new, and 28 segments were duplicates of segments used in previous years. For this last category, the same area had been listed previously under a different segment identification number, so the original listing was used instead of relisting the same area.
After selecting these new areas, the process of counting and listing (C/L) the dwelling units (DUs) within each new segment ensued. Segments to be used in 2003 were listed between April and November of 2002. Once all DUs for a particular quarter were listed, the second-stage selection process identified screener dwelling units (SDUs) for inclusion in the study.
At the final stages of selection, five age-group strata were sampled at different rates. These five strata were defined by the following age-group classifications: 12–17, 18–25, 26–34, 35–49, and 50 years old and over. No race/ethnicity groups were purposely oversampled for the 2003 main study. However, consistent with previous NSDUHs, the 2003 NSDUH was designed to oversample younger age groups.
Preparations for C/L field activities began with the decision to use the existing NSDUH data collection management structure to supervise counting and listing. All current field supervisors (FSs) were asked to handle the administrative tasks for the listers hired for their area. These tasks included completion of the initial hiring process, segment assignment, managing the timely completion of segments, and weekly approval of time and expense reports. For technical supervision such as how to handle a specific segment, all listers contacted the C/L manager for answers and advice.
Beginning in March 2002, FSs recruited listing staff from their existing staff of FIs. Experienced listers not currently working as NSDUH interviewers were also available for hire. A total of 363 listers were hired, certified, and worked from April through November 2002, to complete C/L operations for the 2003 NSDUH.
All hired listers received a home study training package containing a memorandum and materials including a project C/L manual; C/L videotape; hire letter; Data Collection Agreement; 2003 NSDUH C/L Project Specification Sheet; and a certification packet which included questions about procedures as well as path-of-travel exercises. Staff had two weeks upon receipt of this package to complete the certification test and return it to RTI for evaluation. Of the 388 training packages distributed, 12 hired listers did not pass the certification test. They received feedback about their efforts including copies of the questions missed but were not allowed to work as listers. An additional 13 certified listers did not actually complete any listing work.
A group of RTI survey specialists attended classroom training in June to learn C/L procedures. Training included detailed instruction in proper C/L protocol and the completion of actual segments selected for the State of North Carolina.
All certified listers received their bulk listing supplies. Newly certified listers were then authorized to begin their C/L assignments. All listers sent their completed assignments directly to the Sampling Department at RTI, where the assignments were carefully edited. To improve the quality of the listing process, positive feedback as well as suggestions for improvement were provided to all listers. Segments with significant errors were either refielded (for correction of major errors) or were corrected by sampling staff through discussions with the lister. In some cases, the lister returned to the segment to review the items in question.
Prior to the start of actual C/L field work, segment packets were assembled at RTI. Each packet contained maps of the selected area, listing forms, and blank segment information sheets. A copy of the maps remained at RTI for reference when assisting with problems encountered in the field.
Beginning in April, segment kits were assigned and sent to those listers who had completed the certification process and were ready to begin listing. Once the remaining staff became certified, they received assignments as well. Listers recorded the address or description of up to 400 DUs in each segment.
To reduce the time required to count and list segments, several procedures were implemented to maximize efficiency. In many cases the "count" step was eliminated: the lister could immediately list the segment unless during the initial trip around the boundaries of the segment it was apparent the segment had experienced additional construction or the lister determined that the segment was large (i.e., 400+ DUs). As had been done on prior rounds of the NSDUH, a rough count procedure was allowed for segments containing large geographic land areas, large DU counts (400+ DUs), or significant growth in residential DUs (typically, 1,000+ DUs). This procedure permitted listers to obtain an approximate count of residential DUs in these segments from secondary sources—such as the post office, fire department, or county or city planning office—without having to conduct an exact count.
If a lister came across a segment that needed subsegmenting, the lister called in the initial DU counts to RTI's Sampling Department, who could sometimes subsegment it over the telephone (any segment with more than 400 DUs generally required subsegmenting). In cases involving traveling listers, the telephone subsegmenting process allowed the lister to—in one trip—count and list a segment with 400 or more DUs, rather than experiencing a delay of one or two weeks and necessitating a second trip to the segment. For difficult subsegmenting tasks, the segment materials were sent to RTI to be handled directly by sampling personnel. Of the 3,572 new segments listed for the 2003 survey, 461 required subsegmenting. When obvious and possible, sampling staff completed any needed subsegmenting prior to the assignment of the segment to the lister, although the majority of subsegmenting occurred during the listing process.
The counting and listing of almost all of the segments was completed by the end of November 2002 (the exceptions involved a few access problems or late segments that had to be returned to the field for relisting). Once the segments were listed and the completed segment kits were received at RTI, an editing process of the completed materials checked for and deleted any DUs located outside segment boundaries, ensured that listing sheets matched segment sketches/maps, and verified that proper listing order and related listing rules were observed. During this editing process, the sampling staff also checked all subsegmenting that occurred in the field to ensure it was done correctly.
Listed DUs were keyed into a computer control system. A selection algorithm selected the specific SDUs to be contacted for the study. Prior to the beginning of the appropriate quarter, FSs assigned segments (or partial segments) to their interviewing staff. Interviewers received all assigned SDUs on their Newton handheld computer. Each selected unit and the next listed unit (for use as a sample check to capture missed dwelling units during screening and interviewing) were also printed on Selected DU Lists. These lists, along with copies of the handwritten listing forms and maps, were distributed to the assigned field staff before the start of each quarter.
During the screening process, FIs were trained to identify any unlisted DUs that existed within the SDU or within the interval between the SDU and the next listed DU. If the missed DUs were housing units, they were automatically entered into the Newton (up to established limits) and selected for participation. At most, the FI could independently add 5 missed DUs per SDU and a maximum of 10 missed DUs per segment. If the FI discovered more than these amounts or if the missed DUs were group quarters units, the FI called the FS. The FS then either called RTI's Sampling Department for further instructions or instructed the FI to call the Sampling Department directly, depending on the situation.
While no upper limit was placed on the total number of DUs that could be added to a segment by RTI's Sampling Department, the FIs were instructed to notify RTI of any significant listing problems. In a small number of segments, portions of these segments had to be relisted during the screening and interviewing phase. Table 2.2 indicates the number of segments that experienced added DUs, as well as the total number of added DUs for the 2003 NSDUH.
In many of the major urban areas, field staff had some difficulties gaining access to locked buildings, and listers in particular had some trouble listing very large public housing complexes. Access in some suburban areas proved problematic as well; more and more planned communities have intercoms, guarded gatehouses, or entryways outfitted with cameras and scrambled buzzer systems. Access to military bases, college dormitories, and large retirement communities also proved problematic at times. Based on experience, these types of access problems were expected. Special mechanisms or protocols were in place to handle them promptly and, in some cases, avoid them entirely.
Access problems were typically resolved through effective follow-up efforts of supervisory staff, including situation-specific letters of request and in-person visits by the Field and/or Regional Supervisors. In particularly difficult situations, SAMHSA offered additional support via special refusal conversion letters or telephone follow-ups by the Project Officer.
In the rare case where access to the segment for listing was denied, statisticians used census dwelling unit estimates as the basis for selecting a list of dummy lines, which were then treated as nonrespondents during weighting and analysis.
As in past years, the often problematic access to military bases was handled with a formal and standardized approach for 2003. Through joint RTI/SAMHSA efforts, a contact person within the Pentagon for each branch of the service was identified. These individuals were advised in advance of base selections for the year. They then notified the base commanders regarding RTI's need to access these bases for both listing and screening/interviewing work. Additionally, standard letters and informational packages were sent by RTI staff to help obtain access to all selected bases. These efforts were effective: access to all but one of the selected bases was secured.
Access to colleges and universities is sometimes problematic. RTI used several standard approaches to accommodate the concerns of school administrators. Having standardized letters available that addressed recurring issues with a variety of attachment options was very effective.
Most schools requested or required only a letter stating the sponsor and the purpose of the study, and identifying the lister or data collection staff. However, some schools wanted more complete information and the right to approve the field data collection procedures and personnel working in and around their campuses. Most of these situations resulted in packages being sent that contained:
Included with all letters and packets was an endorsement letter signed by the presidents of Duke University and the University of North Carolina at Chapel Hill. In the end, all of the private educational institutions expressing concerns cooperated in the C/L phase of the 2003 NSDUH.
Thirty segments were identified during the C/L phase as difficult to access during months with unusual weather. Including 9 overlap segments from the 2002 study, there were a total of 39 segments in 2003 with access issues. Most involved roads made impassable by snow during the winter months. Others involved roads inaccessible due to rain, and one or two isolated locations involved water-only access that often froze during the winter months. If segments with weather or geographic access problems were selected for a quarter in which the access would be a problem (generally Quarters 1 or 4), the segment was switched with a segment in the same region for an appropriately paired time period. For example, inaccessible first quarter segments were switched with second quarter segments in the same region that would be more accessible during the first quarter; fourth quarter segments were switched with more easily accessed third quarter segments. Generally the "switched" segment was selected because it had more accessible road surfaces, was more urban, or had fewer inaccessible roads.
In a few locations, such as some areas in Alaska, there were no segments that were better for reassignment during the problematic time period. When that happened, staff made prompt assignments, emphasized early completion of the work, and tried to plan around good weather forecasts to accomplish the field work as early in the period as possible.
| First Stage of Selection for the Main Study: Segments
The 2003 design provided for estimates by State in all 50 States and the District of Columbia. States should therefore be viewed as the "first level" of stratification as well as a reporting variable. Eight States, labeled the "big" States in Table 2.1, had samples designed to yield 3,600 respondents per State. The remaining 43 "small" States1 had samples designed to yield 900 respondents per State. The larger sample sizes obtained at the State level, along with small area estimation techniques refined under previous NSDUH contracts, enabled the development of estimates for all States, for several demographic subgroups within each State (i.e., age group and race/ethnicity group), and for some Metropolitan Statistical Areas (MSAs) and a few small areas in the "big" States. The "second level" of stratification defined contiguous geographic areas within each state and also corresponded in size to the annual assignment for a single FI. These FI regions were of approximately equal population size in terms of allocated samples. Additional implicit stratification was achieved by sorting the first-stage sampling units by an MSA/SES (Metropolitan Statistical Area/socioeconomic status) indicator2 and by percentage of non-Hispanic white. The first-stage sample units for the 2003 NSDUH were selected from this well-ordered sample frame. For the first stage of sampling for the 2003 NSDUH, each of the FI regions was partitioned into noncompact clusters of dwelling units by aggregating adjacent census blocks. Consistent with the terminology used in previous NSDUH studies, these geographic clusters of blocks were referred to as segments. On average, segments were formed so that they contained at least 175 dwelling units and were constructed using 1990 Decennial Census data supplemented with revised population counts obtained from outside sources. A sample dwelling unit in the NSDUH refers to either a housing unit or a group quarters listing unit (such as a dormitory room or a shelter bed). A sample of segments was selected within each FI region, with probabilities proportionate to a composite size measure and with minimum replacement. Segments were formed so that they contained sufficient numbers of dwelling units to support three annual NSDUH samples. This allowed half of the segments used in any given year's main sample to be used again in the following year as a means of improving the precision of measures of annual change. This also allowed for any special supplemental sample or field test that SAMHSA wished to conduct in any given NSDUH year within the same segments. In order to coordinate the sample selection for 1999 through 2003, 96 segments were selected within each FI region. An equal probability subsample of eight segments was used for the 2003 NSDUH. These eight segments were randomly assigned to quarters and to two waves within each quarter. The waves used in the 2003 NSDUH were designated as Waves 5 and 6. Wave 5 segments were used for the 2002 and 2003 surveys. New dwelling units (i.e. those not previously selected for the 2002 study) were selected from the Wave 5 segments for 2003. Wave 6 segments were new for 2003 and will be used again for the 2004 survey. Data from roughly one fourth of the final sample of respondents was collected during each calendar quarter. This important design feature helped control any seasonal bias that might otherwise exist in drug use prevalence estimates and other important NSDUH outcome measures of interest. Second Stage of Selection for the Main Study: Listed Lines Before any sample selection within selected segments began, specially rained staff listed all dwelling units and potential dwelling units within each newly selected area segment. A dwelling unit is either a housing unit for a single household or one of the eligible noninstitutional group quarters that are part of the defined target population. The listings were based primarily on observation of the area segment and could include vacant dwelling units and units that appeared to be dwelling units but were actually used for nonresidential purposes. The objective of the listing was to attain as complete a listing as possible of eligible residential addresses; any false positives for residences were eliminated during the household screening process after the sample was selected. The sampling frame for the second stage of sample selection was the lines of listed dwelling units and potential dwelling units. After accounting for eligibility, nonresponse, and the third-stage sample selection procedures (including a response rate adjustment for the effect of the $30 incentive), it was determined that 182,250 lines were needed to obtain a sample of 67,500 responding persons distributed by state and age group. During the study's implementation, however, a total of 170,762 lines were selected and yielded a final respondent sample of 67,784 (as shown in Table 2.1). These lines were selected among lines not used in the 2002 survey (overlap segments) and the complete list of dwelling units (new segments). As in previous years, if an interviewer encountered any new dwelling unit in a segment or found a dwelling unit missed during the counting and listing activities, the new/missed dwellings were selected into the NSDUH using a half-open interval selection technique.3 That selection technique eliminated any frame bias that might have been introduced because of errors and/or omissions in counting and listing activities and also eliminated any bias that might have been associated with using "old" segment listings. Third Stage of Selection for the Main Study: Persons After dwelling units were selected within each segment, an interviewer visited each selected dwelling unit to obtain a roster of all persons aged 12 and over residing in the dwelling unit. This roster information was then used to select zero, one, or two persons for the survey. Sampling rates were preset by age group and State. Roster information was entered directly into the electronic screening instrument (the Newton) which automatically implemented this third stage of selection based on the State and age group sampling parameters. Using an electronic screening instrument also provided the ability to impose a more complicated person-level selection algorithm at the third stage of selection. As a result of this unique design feature, any two survey-eligible people within a dwelling unit had some chance of being selected—i.e., all survey-eligible pairs of people had some non-zero chance of being selected. This design feature is of interest to NSDUH researchers because it allows analysts to examine how the drug use propensity of one individual in a family relates to that of other family members residing in the same dwelling unit (e.g., the relationship of drug use between a parent and child). Originally added in 2002 with use continuing in 2003, an additional parameter in the person selection process increased the number of selected pairs within dwelling units without unduly diminishing response rates. As illustrated in Table 2.1, at the third stage of selection, 81,631 people were selected from 130,605 screened and eligible dwelling units. A total of 67,784 completed interviews were obtained from these 81,631 selected persons. Expected Precision of NSDUH Estimates The multistage, stratified NSDUH design has been optimally constructed to achieve specified precision for various person subpopulations of interest. These SAMHSA-specified, precision requirements call for the expected relative standard error on a prevalence of 10 percent not to exceed the amounts listed below. For the main study:
To achieve these precision requirements and meet State sample-size requirements, the optimal person-level sample distribution by strata was determined that minimized data collection costs while simultaneously meeting the above-specified precision requirements for several critical NSDUH outcome measures. The precision constraints in the design optimization models were set up using local area predictions of drug use from a project involving small area estimation techniques to generate local area estimates from 1991–1993 NSDUH data. Drug use estimates across strata were appropriately scaled to reflect the generic 10 percent prevalence. |
1 For reporting and stratification purposes, the District of Columbia is treated the same as a State and no distinction is made in the discussion.
2 The four categories are defined as: (1) MSA/low SES, (2) MSA/high SES, (3) NonMSA/low SES, and (4) NonMSA/high SES.
3 In summary, this technique states that if a dwelling unit is selected for the NSDUH and an interviewer observes any new or missed dwelling units between the selected dwelling unit and the dwelling unit appearing immediately after the selection on the counting and listing map page, then all new/missed dwellings between the selection and the next one listed will be selected. If a large number of new/missed dwelling units are encountered (generally greater than ten) then a sample of the missing dwelling units will be selected.
| Statistic | Small States | Big States | Total |
|---|---|---|---|
| Total Sample | |||
| FI Regions | 516 | 384 | 900 |
| Segments | 4,128 | 3,072 | 7,200 |
| Selected Lines | 97,847 | 72,915 | 170,762 |
| Eligible Dwelling Units | 81,616 | 61,869 | 143,485 |
| Completed Screening interviews | 75,491 | 55,114 | 130,605 |
| Selected Persons | 46,374 | 35,257 | 81,631 |
| Completed Interviews | 38,934 | 28,850 | 67,784 |
| Average per State | |||
| FI Regions | 12 | 48 | |
| Segments | 96 | 384 | |
| Selected Lines | 2,276 | 9,114 | |
| Completed Interviews | 905 | 3,606 | |
| Interviews Per Segment | 9.43 | 9.39 | |
| Average per State And Quarter | |||
| Segments per FI Region | 2 | 2 | |
| Interviews per FI Region | 18.85 | 18.78 | |
| Interviews per Segment | 9.43 | 9.39 | |
| Total States | 43 | 8 | 51 |
| Total Interviewers (approximate number that varied by quarter) |
470 | 348 | 818 |
| Note: "Small" States refers to States where the design yielded 905 respondents on average. "Big" States refers to States where the design yielded 3,606 respondents on average. | |||
| Number of Added DUs per Segment (X) | Number of Segments with X Added DUs | Cumulative Number of Added DUs* |
|---|---|---|
| 1 | 501 | 501 |
| 2 | 210 | 921 |
| 3 | 71 | 1134 |
| 4 | 25 | 1234 |
| 5 | 27 | 1369 |
| 6 | 6 | 1405 |
| 7 | 10 | 1475 |
| 8 | 4 | 1507 |
| 9 | 6 | 1561 |
| 10 | 5 | 1611 |
| 11 | 2 | 1633 |
| 12 | 4 | 1681 |
| 14 | 1 | 1695 |
| *Total number of added DUs = 1,695 | ||
|
|
|
 |
Click to Return to OAS Home Page |
|
This page was last updated on May 20, 2008 .
|
|
