2003 NSDUH Data Collection Final Report

8. Quality Control

While every step was designed to help collect the highest quality data possible, the 2003 National Survey on Drug Use and Health (NSDUH) included specific quality control processes which are described in this chapter.

8.1 Field Supervisor/Interviewer Evaluation

8.1.1 Regular Conferences

Each field interviewer (FI) had at least one regularly scheduled weekly telephone conference with his/her field supervisor (FS). During this call, the FI reported progress made toward completing the work; reviewed production, time, and expense information for the week; discussed field problems; and asked any questions that had emerged during the week. The FS then provided feedback on the progress and quality of work and offered solutions to problems or questions encountered. The FS also shared any information from project managers, such as "Data Quality Item of the Week" notices or approaching project deadlines.

Regular weekly telephone conferences were also held between the regional supervisor (RS) and each of the FSs in his/her territory. FI production and performance were discussed during these conferences, as were budget considerations and any problems that were occurring.

8.1.2 Observations at New-to-Project Training/Training Evaluations

Beginning at training, FI performance was monitored closely and consistently throughout the field period. Training classrooms were small enough to observe and evaluate each FI's individual performance and comprehension. The classroom trainers worked together to evaluate FIs on a daily basis, rating each trainee on a 4–point scale:

Additional letter ratings were assigned documenting improved trainee performance or significant problems such as attention difficulties or physical limitations like poor eyesight. Explanations were required for a rating of 1 or 2 or any problematic letter ratings.

In all cases this trainee evaluation system was used strictly as a management tool—ratings were not shared with the trainees. Reports of struggling FIs were given to the site leader daily to help identify problems and develop resolution plans. The information was also forwarded to the trainee's supervisor to keep the FS informed of progress. These evaluations ensured that those FIs who were struggling with training program content but willing and capable of doing the work would receive the necessary help both during and after training to interview successfully on the NSDUH.

Through the certification process (see Section 5.2.1), formal one-on-one evaluation of each trainee occurred. As explained earlier, all trainees were required to complete the certification in order to successfully complete training.

In addition, all new-to-project graduates were mentored (see Section 5.2.5) to observe their behavior in the field and reinforce the important study protocols learned during training.

8.1.3 Observations at Veteran Training/Ongoing FI Knowledge Evaluations

Veteran FIs continuing work on the study in 2003 were tested and trained to be sure they met the standards necessary to serve as NSDUH interviewers. Beginning with the electronic home study (see Section 4.5.1), interviewers could only continue working if they demonstrated knowledge of basic protocols. During veteran training, FIs were monitored through classroom performance.

Periodic evaluations (eVals) of interviewer knowledge occurred during the year (see Section 5.5). This tool not only tested knowledge but reinforced that following protocol helps collect data of the highest possible quality. All interviewers also received a laminated copy of the form "Steps to Maximize Data Quality" (see Exhibit 8.1) which listed the most crucial NSDUH protocol steps.

8.1.4 Field Interviewer Observations

In-person observations of FIs at work provided insights about the survey and its procedures as well as assessments of interviewer performance and attention to project protocol. Field Observations were implemented nationally for the second, third, and fourth quarters of 2003.

Around the country, 319 FIs were observed completing 638 screenings and 414 interviews. Observers, who were regional directors (RDs), RSs, FSs, members of the Instrumentation Team, project survey specialists, or Substance Abuse and Mental Health Services Administration (SAMHSA) staff, had specific forms to complete, noting interviewer behaviors on a number of project protocols. Data from completed forms were used to assess current levels of interviewer knowledge and develop training plans to improve FI skills in identified problem areas. To maintain the integrity of the operation, observers did not give direct feedback to the FIs. Information regarding FI performance was made available to the appropriate FS to share with observed FIs. Results from these observations were formally documented in the 2003 NSDUH Full-Year Field Observation Report.

8.1.5 FS Quarterly Evaluations of FIs

At the end of every quarter of data collection, each FS evaluated the FIs in his/her region to decide how to allocate bonus funds and whether to recommend any merit-based pay raises. FSs considered all the facets of being a "good FI," including production, response rates, adherence to procedures, costs, timeliness, attitude, commitment, attention to details, lack of data quality errors, and willingness to take on additional work (particularly to work on hard refusals). To decide how to divide bonus funds, the FS ranked each FI. Additionally, pay raises were not necessarily related to bonus money; an FI might not receive a bonus but could still be eligible for a raise. For both bonuses and pay raises, RSs and RDs reviewed the FS's decisions.

8.1.6 FS Final Evaluations of FIs

At the end of the calendar year, each FS used a standard RTI multiple-choice form to generate an annual evaluation of FIs who were active on the NSDUH. FIs were rated on a 5–point scale (unsatisfactory, poor, satisfactory, above average, and exceptional) on such standard interviewing skills as quality of work, data collection skills, adherence to deadlines, and productivity. The FS also commented on the FI's strengths and any areas needing improvement. The FS used this same form to provide a final evaluation of FIs who "attrited." Completed evaluations were added to the interviewer's personal data file at RTI. The FS generally completed this form without RS or RD input.

8.1.7 FI Exit Interviews

Every month, NSDUH management personnel received a listing of those field interviewers who had voluntarily chosen to leave the project (those terminated did not appear on this list). The listed FIs were contacted and a short questionnaire was administered (see Exhibit 8.2) to determine the reasons they left the project. These data were then keyed and used to produce a quarterly report for project management summarizing the reasons. Of the 185 FIs who were terminated from the NSDUH in 2003, 109 voluntarily chose to leave the project. The exit interview was completed with 61 of these FIs. Exhibit 8.3 contains the total results for all FI exit interviews conducted during 2003. Table 8.1 summarizes the most important reasons reported by FIs for their resignation. Nine FIs completing the exit interview (15 percent) indicated the most important reason for leaving was some difficulty working with their supervisor, while seven (12 percent) said they did not like working at night and six others (10 percent) found another job.

8.2 Web-based Case Management System (CMS)

Each FS was equipped with a laptop computer and given access to the NSDUH Web-based Case Management System (CMS). FIs transmitted screening data daily from the Newton, including record of calls data, verification information for non-interview cases, added DUs, and address updates. Newton screening data transmitted to RTI were checked by the control system's defined consistency checks, and then posted to the CMS for monitoring purposes. The completed interview data were transmitted to RTI by FIs from their laptop computers and checked against screening data to ensure each completed case was received and that the correct respondent was interviewed.

The FS System on the CMS included the following data quality functions:

• Daily and Weekly Reports with access to archived reports (for comparison data).
  
  
• An interactive data information page for monitoring production.
  
  
• An interactive record of calls page for monitoring FI work patterns.
  
  
• Verification data.

8.2.1 Data Quality Report

The Data Quality Report displayed various data quality issues and allowed the FS to provide specific feedback to FIs who were experiencing problems. The report included missing data items on Quality Control Forms and procedural errors such as Case ID or Verification ID problems. The report also included a list of cases that could not be used due to the FI interviewing the wrong household member.

8.2.2 Missing Screening Data Report

The Missing Screening Data Report displayed by FI the screening data that were missing for specific Case IDs. FSs used this report to monitor the quality of the screening data that each FI collected. The data on this report represented information that the respondent refused to provide or indicated areas where the FI either made errors or may have been taking short-cuts. FSs monitored specific problems and trends and were able to provide immediate feedback and re-train FIs as necessary.

8.2.3 Overdue Cases Report

FSs used the Overdue Case Report to account for completed interviews that should have already arrived at RTI. Interviews were considered overdue if not transmitted within three days of the date of interview (as reported by the Newton Record of Calls data).

Cases displayed on this report were investigated to ensure the completed interview was transmitted or that the correct Case ID was used and reported as a completed interview. FSs and programming staff worked to resolve any pending issues with overdue cases.

8.2.4 Length of Interview Report

The Length of Interview Report listed the completed interviews that were either finished in a relatively short or extremely long amount of time. The times were derived from the computer-assisted interviewing (CAI) interview file (total time and timing of specific sections) so that FSs could monitor possible problem situations (such as short-cutting or problems with the laptop that might cause the time-frame to be strange).

8.2.5 Case Data Information

The Case Data Information portion of the CMS provided all FI production data and allowed the FS to interact with the data and view it in special ways. The type of cases the FS viewed was determined by the drop-down items selected. Each of the following items was available to select (single or multiple items), after which a data table containing all of these items (for the subset of cases) displayed:

• Case ID
  
  
• Type of case (Screening, Interview A, or Interview B)
  
  
• Status and Result Code (record of calls event codes)
  
  
• Result Code Date (date of the record of calls code)
  
  
• # Calls (total number of contacts at the household)
  
  
• FS Note (any notation the FS attaches to the case)
  
  
• Questionnaire Rec'd (date the case was transmitted)
  
  
• Verification Status
  
  
• FI ID (FI assigned to the case)
  
  
• Address of the SDU.

There were special features within this function that displayed additional data:

• Overdue cases (highlighted in yellow)
  
  
• Added DUs (highlighted in green)
  
  
• Cases where a call record had not been entered in more than 14 days (highlighted in ink)
  
  
• Click on CaseID to view entire record of calls
  
  
• Click on Refusal Code to view entire refusal report
  
  
• Click on Verification Status to view verification history of case
  
  
• Click on FI ID for production, time and expense data
  
  
• Click on address to view map of the area.

The data provided in this table allowed the FS to evaluate many aspects of the FI's work.

8.2.6 Filter Record of Calls

The Filter Record of Calls allowed the FS to view the FI's record of calls events by filtering on the following items:

• Case ID
  
  
• Data Type (Screening, Interview A, or Interview B)
  
  
• Result Code
  
  
• Day of week (All days, Mon–Sun)
  
  
• Time periods of day (6am–Noon, Noon–4pm, 4pm–12am, 12am–6am)
  
  
• Date (before a date, after a date, a specific date, or between two dates)
  
  
• FI.

The FS could analyze the FI's work pattern and spot instances where an FI might have entered "false" results.

8.3 Data Quality Team

The Data Quality Team was responsible for the identification, resolution, and distribution of information to field staff concerning data quality and verification issues. The data quality manager supervised a team of data quality coordinators (DQCs) as they monitored the data quality of specific regional areas. The Manager also interacted with supervisors in RTI's Telephone and Internet Operations (TIO) unit (for verification issues), and data receipt and data preparation units to oversee data quality issues. The Data Quality Team also prepared weekly "Data Quality Item of the Week" notices which reviewed or clarified procedures for a particular issue. These notices were given to the RDs each week for use during the RD-RS conference calls. The RSs then passed the information along to the FSs who shared the news with the interviewers.

Each DQC reported the results of the in-house data quality tasks, consistency checks, verification task completion, and interpretation of the results to their RD. They also planned and conducted field verifications as necessary.

8.4 Verification of Completed Cases

In order to verify the quality and accuracy of each FI's work, a complex verification procedure was implemented. This involved the selection and verification of at least 15 percent of final interview cases for each interviewer, as well as at least 5 percent of final non-interview screening cases. Verification contacts for selected cases were made primarily by telephone. For selected interviews where no telephone number was provided, verification was attempted by mail. Whenever possible, all verification contacts were made with the actual respondent. Detailed flowcharts illustrate the process for screening verification (Exhibit 8.4) and interviewing verification (Exhibit 8.5).

The system allowed for the verification of additional work beyond the standard 15– and 5–percent selection rates. Field management staff could elect to increase verification selection up to 100 percent of the FI's completed work. Managers could also select an individual case or a group of specific cases to be verified beyond what was randomly selected. Another available option allowed managers to select all cases completed on a specific day. Managers used higher verification rates for interviewers with significantly large amounts of work within a given state.

8.4.1 In-house Verification

Verification information for completed interviews was obtained from the Quality Control Form completed by each interview respondent (see Exhibit 8.6). For the final non-interview screening codes of 10 (vacant), 13 (not primary residence), 18 (not a dwelling unit), 22 (dwelling unit contains only military personnel), 26 (not eligible for the quarter), and 30 (no one selected for interview), the contact information was recorded in the Newton at the time the case was finalized. For codes 10, 13 and 18, the contact was made with a knowledgeable person, such as a real estate agent, property manager, or neighbor. For codes 22, 26, and 30, the verification was completed most often with the screening respondent.

The telephone verification was conducted by project trained telephone interviewers in RTI's TIO unit. Spanish translations of all materials were available for verifications with Spanish-speaking respondents. Again, most of the selected code 70s and all of the selected codes 10, 13, 18, 22, 26, and 30 were verified by TIO. The NSDUH telephone verification script used depended on the final status code of the case (see Appendix E).

For those selected code 70s that did not have a telephone number on the Quality Control Form but did have an address, verification by mail was attempted. The mail verification letter (see Exhibit 8.7) was sent to the respondent to complete and return by mail to RTI. The completed verification letters were keyed, and the results were displayed in the CMS and on the Verification Reports. Of 307 cases for which mail verification letters were sent, 69 were returned by respondents. Most cases verified by this method verified with no problem discovered.

Telephone verification had two stages. During the first stage as described above, telephone interviewers followed a script when speaking with the respondent to confirm that the FI was professional and followed project protocols. The majority of cases were finalized as having no problems. During the second stage of verification, a follow-up call was made to investigate any serious problems found during the initial call. That follow-up call was made by the Call Back Team, an elite group of telephone interviewers who were trained on all project procedures and protocols.

The Call Back Team was responsible for conducting a thorough investigation of each problem case identified. During the follow-up call, they determined whether or not the FI was adhering to project protocols. If not, the Call Back Team caller determined the types and severity of the FI's deviations from protocol. The Call Back Team documented the results and provided a summary to DQCs. This information was used as a basis for retraining the FI, or, in the case of falsification, as evidence to substantiate terminating the FI.

Unlike the initial telephone interviewer who followed a script for verification, the Call Back Team was given example introductions, the problem or problems identified during the first call, and a list of items to cover for each type of case based on the final result code. The Call Back Team conversed with the respondent asking probing questions that allowed the respondent to talk about what happened during the screening or interview process in an attempt to confirm or resolve the identified problem(s).

The result of the call was either a confirmation that the problem (or additional procedural problems) occurred during the screening or interview or a resolution of the problem by clarifying the issues with the respondent. The Call Back Team documented the results on a formal problem sheet detailing the findings of the call. Problem sheets were then sent to the DQCs who reviewed the information for each case and then assigned a final resolution code:

• No Problem—the case verified and resolved without problems
  
  
• Error—resolved but verification contact indicated breeches in project protocol
  
  
• Unable to Contact—unable to contact the respondent
  
  
• Unresolvable—an unresolvable situation (incorrect phone number, respondent refused, initial error could not be confirmed)
  
  
• Invalid—interview or screening data cannot be used for analysis due to serious protocol violations or falsification.

Tables 8.2 and 8.3 provide summaries of the results of phone verifications for non-interview screening codes 10, 13, 18, 22, 26, and 30 and for completed interviews. We have not included the mail verification results in Table 8.3 because these cases make up a very small percentage of cases verified.

8.4.2 Field Verification

In addition to the verification procedures conducted on completed work received in-house, additional steps were taken in the field to ensure complete and accurate collection of data. This field verification was generally initiated after one of four circumstances occurred:

1. an FI had an unusually large number of in-house verifications "fail";
   
   
2. an FI had a higher than average percentage of cases with no phone numbers (for screening cases) and/or no Quality Control Forms (for interviews);
   
   
3. the FI exhibited unusual or suspicious patterns of work behavior; or
   
   
4. an FI reported numerous cases as being completed but failed to transmit to RTI within three days of completion.

The Data Quality Team worked with the FS and RS to select the cases to be field verified. These finalized cases were transmitted to the Field Verifier's Newton (either the FS or another FI conducting the field verification) so that the screening data could be verified. The Field Verifier returned to the SDUs that were assigned and queried the respondents in an effort to determine whether or not proper contact had been made by the FI in question. The Field Verifier also verified the screening information. If an interview had been completed, the Field Verifier confirmed some of the demographic data from the interview with the respondent. The Field Verifier also reviewed some protocol issues with the respondent to ensure the FI had followed protocol and acted in a professional manner. Results of the field verification were reported to the Data Quality Team and the FS, RS, and RD. If the Field Verifier found the work to be invalid, he or she reworked the case.

In general, the need for such in-field verification was limited, but it did occur. In the 2003 NSDUH, a total of 451 cases were selected for field verification. This process led to the identification and termination of FIs who were determined to have submitted fraudulent work. All their work completed during the current quarter was verified and reworked as necessary. A total of 24 invalid interviews and 34 invalid screenings involving 6 FIs were identified via in-person field verification. The 4 FIs with falsification were terminated. The other 2 FIs had made enough errors to cause a total of 6 screenings to be invalid, but no clear evidence of falsification was found. These 2 FIs were placed on probation, retrained and placed on increased verification.

8.4.3 Verification Monitoring Tools

8.4.3.1 Case Data Information Link

The Verification Status on the Case Data Information link on the CMS allowed project staff to view the verification status of each case and monitor trends across status codes or areas. The following Verification Status codes were used to monitor the verification at the case level:

NF:     No Form (Code 70s)

NP:     No Phone

RE:     Refusal—not selected

NS:     Eligible, but not randomly selected for verification

ST:     Selected for Telephone Verification

SF:     Selected for Field Verification

SM:     Selected for Mail Verification (Code 70s without phone numbers)

OK:     Completed Okay

UC:     Finalized—Unable to Contact

UN:     Finalized—Unresolveable

SS:     Completed—Some shortcuts

IR:     Completed—Invalid, then reworked

IW:     Completed—Invalid, not reworked

Since verification selection was random, it helped to see which cases had been selected. If project staff wanted additional cases to be selected for verification, they worked with their region's DQC to select additional cases to be flagged for verification.

8.4.3.2 Short FI Level Verification Report (Pages 1 and 2)

The Short FI Level Verification Report provided a snapshot of the problems identified during Telephone, Mail, and Field Verification. Page one (see Exhibit 8.8) provided a summary of verification data. Displayed were the number of cases that had no form (code 70 only), no phone, refused, percent of cases with no form/phone (once greater than or equal to 30 percent), percent of cases refused (once greater than or equal to 30 percent), count of other ineligibles, count of eligibles, count of cases selected for telephone, count of cases selected for mail, and count of cases selected for field verification. If applicable, the results of any selected field verification cases were also displayed. From this data, supervisors could see if an FI had a high percentage of cases with no phones, no forms, refused, and how many have been sent to Mail Verification (which is not as successful as Telephone Verification in obtaining a response).

More specific details of the problems displayed on page one were contained on page two of the report (Exhibit 8.9). The second page displayed each problem identified during Telephone and Mail Verification. A case could have multiple problems, so all problems for all cases were displayed here to track trends related to possible shortcutting. There were 50 Problem Codes divided into four groups by Screening and Interview Result Code (Exhibit 8.10).

8.5 Industry and Occupation Coding

During the later part of the interview, the FI asked a series of questions to obtain detailed information about a respondent's job. Periodically through 2003, RTI sent this information to The National Processing Center of the Bureau of the Census so that their team of industry and occupation coders could classify each respondent's job. Details on the end results from the Census coding operation are provided in Appendix F.

To provide feedback to interviewers, RTI developed a report listing interviewers having 3 or more "unable to code" cases in Quarter 1. For interviewers on this list, retraining on the proper administration of the Industry and Occupation questions occurred during Quarter 2. All interviewers received a listing of tips and helpful hints to use when collecting Industry and Occupation data. Based on prior experience, common problem situations were included to provide examples of the level of detail required to assign codes.

Exhibit 8.1 Steps to Maximize Data Quality

This summary is not a replacement for information contained in your FI Manual, but is a listing of some of our most crucial protocols that must be followed.
Be sure that you follow each of these at all times.

Note the FI Manual pages referenced with each key point. Keep in mind that the below protocols are not the only steps that are necessary to follow. Use your FI Manual, Field Supervisor, and project e-mails for information on additional steps to maximize data quality.

• Use your segment maps, and not just the address, to locate your selected DUs. [FI Manual p. 3–16]
  
  
• Display your ID badge when knocking on every door in your segment. [FI Manual pgs. 4–19 and 5–1]
  
  
• Complete screenings in-person with a resident who is 18 or older. The only exception is in the case of emancipated minors. [FI Manual p. 4–20]
  
  
• Give a Study Description to each SR. [FI Manual p. 4–21 and 4–22]
  
  
• Obtain complete and accurate screening information, reading the screening questions verbatim to the SR and immediately entering responses into the Newton. The only missing screening data should be a result of the respondent's refusal to provide information. [FI Manual p. 6–20]

• Read the CAI Introduction and Informed Consent from the Showcard Booklet to the R (choosing the appropriate version based on the respondent's age) before beginning the interview. Before speaking with a selected minor, you must obtain verbal parental permission. If the R was not the SR, give him/her a Study Description. [FI Manual pgs. 7–22 and 7–23]
  
  
• Make it apparent that you are completing the interview in a completely confidential and unbiased manner. [FI Manual pgs. 2–7 and 8–1]
  
  
• To the extent possible, choose an interview location that gives the respondent privacy. [FI Manual pgs. 7–26 and 7–27]
  
  
• Do not rush the respondent. Do not tell the respondent how to make the interview go faster. [FI Manual p. 8–3]
  
  
• Use the Reference Date Calendar and read verbatim the explanation provided on the CAI screen to the R. As appropriate, remind the respondent to use the calendar as a visual aid throughout the interview. [FI Manual p. 8–14]
  
  
• Familiarize the R with the laptop and function keys by reading the provided script in the CAI Interview and allow the R to successfully complete the Computer Practice on his or her own. You must always explain, offer, AND plug in the headphones with each R. [FI Manual pgs. 8–16 and 8–17]
  
  
• Read the interview questions exactly as they appear on the screen. It is never acceptable to use your own words or ‘wing it'. Do not assume you know answers from a previous conversation, question, or interview. [FI Manual p. 8–2 and 8–3]
  
  
• Hand the appropriate Showcard to the respondent when instructed to do so on the CAI screen. [FI Manual p. 8–13]
  
  
• Allow your respondents to complete the ACASI portion of the interview on their own. Never read the questions in the ACASI portion of the interview out loud to the respondent. In cases of extreme physical impairment, it may be necessary to enter the answers into the computer for the ACASI questions, but always allow the ACASI recording to ‘read' the questions and answer categories via the headphones. [FI Manual pgs. 8–21 and 8–22]
  
  
• Have the respondent fill out the top portion of the Quality Control Form and allow the respondent to insert the form into the envelope and seal it. Mail the form promptly. [FI Manual pgs. 8–23 through 8–25]
  
  
• Always protect the confidentiality of your respondents. Never reveal a respondent's answers to anyone, including the respondent's family members. Resist the temptation to reveal even positive information gleaned from an interview to parents or other household members. [FI Manual pgs. 2–7 and 2–8]

Exhibit 8.2 2003 NSDUH Field Interviewer Exit Interview

C. Introduction

Hello. My name is ________ and I work for the Research Triangle Institute in North Carolina. According to our records, you have worked for us as a field interviewer on the National Survey on Drug Use and Health (formerly known as the National Household Survey on Drug Abuse). First, I just need to verify: did you recently resign? (If "no," record comments in the space under question # 10.)

This large national study depends on high quality field staff to gather the information. Any time one of our interviewers elects to leave the project, we are always interested in knowing why. We would like to ask you a few questions about your experience on the NSDUH and to learn why you chose to leave the project. Is now a convenient time for you? This will only take a few minutes.

[1] First, why did you resign?

[2] What could we have done to keep you as an interviewer?

[3] Did the interviewer training sessions you attended adequately prepare you for your job as an NSDUH interviewer?

[4] What areas of the training sessions could have been better?

[5] Before you began interviewing, how accurately did your Field Supervisor describe the Field Interviewing job?

     Extremely accurately
     Very accurately
     Somewhat accurately
     Not very accurately
     Not at all accurately

[6] How would you describe your working relationship with your Field Supervisor?

     Excellent
     Very good
     Good
     Fair
     Poor

[7] What can you tell me about your working relationship with your FS?

[8] Now I am going to read to you a list of reasons that an interviewer might decide to leave the NSDUH project. As you hear each one, please tell me how important it was in your decision to resign. Please rate whether it was: Extremely important in your decision to resign, very important, somewhat important, not very important, or not at all important in your decision to resign.

REASON		Extremely Important	Very Important	Somewhat Important	Not Very Important	Not at all Important
A	I found a new job
B	I didn't like the subject matter of the study
C	I didn't like contacting strangers
D	The equipment and materials we had to carry were too heavy or bulky
E	I didn't feel comfortable using the computers
F	I had difficulty working with my supervisor
G	I was disappointed by the lack of benefits, such as health insurance
H	I was disappointed by the rate of pay
I	There wasn't enough room for advancement
J	I didn't like working at night
K	I didn't like working on the weekend
L	I wasn't available to work the number of hours required each week
M	I was available but there weren't enough lines for me to work
N	I didn't like the continuous pressure to meet weekly production levels
O	I didn't feel safe in the neighborhoods I was assigned
P	I didn't like the distances that I had to drive to get to the sample neighborhoods

[9] Of all the reasons I just named, which one reason was most important in your decision to leave the NSDUH project? (Read each of the reasons in Question 8, if necessary.)

     Item #:

[10] Is there anything else you'd like to let us know?

I want to thank you for your time. The NSDUH management staff certainly appreciate your willingness to provide answers to these questions. Have a nice day/evening.

Exhibit 8.3. Field Interviewer Exit Interview Results

Exhibit 8.4 Overview of NSDUH Screening Verification Process

Exhibit 8.5 Overview of NSDUH Interview Verification Process

Exhibit 8.6 Quality Control Form

Exhibit 8.7 CAI Mail Verification Letters

Exhibit 8.8 Short FI Level Verification Report—Page One

Exhibit 8.10 Short FI Level Verification Report Problem Codes

Code 70 Problems

1     Incorrect phone number for address
2     Correct address/phone but R unknown
3     Roster Incorrect
4     Correct address/phone but no adult to give permission to speak with teen R
5     Not contacted by FI
6     Contacted by FI but did not complete interview
7     Interview completed some other way (not in person or by phone)
8     Interview completed by phone
9     Option not offered to enter answers in computer
10    Tutorial not completed
11    No headphone option
12    FI unable to assist when R had difficulties with computer
17    FI Not Professional
18    R does not recall the reference calendar
21    R did not receive incentive payment
22    R did not receive the correct amount of incentive payment

Code 30 Problems

30    R unknown and not correct phone number for the SDU OR incorrect phone number for the SDU
31    Correct Roster and Address, but SR Unknown
32    Does not remember FI – Correct Address but Roster Incorrect
33    Does not remember FI – Wrong Address but Correct Roster
34    Does not remember FI – Wrong Address and Incorrect Roster
35    Does not remember FI – Refused to verify Address and Roster
36    Remembers FI – Correct Address but Roster Incorrect
37    Remembers FI – Wrong Address but Correct Roster
38    Remembers FI – Wrong Address and Incorrect Roster
39    Remembers FI – Refused to verify Address and Roster
40    Telephone Screening
41    Screening completed some other way (not in person, by intercom, or by telephone)
42    FI wrote screening data on paper (not entered in Newton) at time of screening
43    FI Not Professional
44    R not contacted by FI but address and roster are correct

Code 22 Problems

50    No known contact with FI
51    Speaking to SR, not familiar with address
52    Refuses to verify address or screening data (or doesn't know)
53    All HH members 17–65 not on active military duty
54    Telephone screening
55    Contact some other way (not in person, by intercom, or telephone)
57    FI Not Professional
58    No one familiar with address or FI
59    Non-military household members age 12–16 not included on roster

Code 10, 13, 18, 26 Problems

	Click to Return to OAS Home Page   Click to Email OAS Data Questions    Click For Non-frames / text version of site
This page was last updated on May 20, 2008 . SAMHSA, an agency in the Department of Health and Human Services, is the Federal Government's lead agency for improving the quality and availability of substance abuse prevention, addiction treatment, and mental health services in the United States.     Privacy Statement  |  Site Disclaimer  |   Accessibility What's New Highlights Topics Data Drugs Pubs Short Reports Treatment Help Mail OAS