|
||||||||
National Center for Chronic Disease Prevention and Health Promotion Behavioral Risk Factor Surveillance System BRFSS Home | Contact Us |
|
|
|
|
|
|
This report provides selected statistical indicators of data quality in the Behavioral Risk Factor Surveillance System (BRFSS). The report presents data on three general types of measures by state1:
(1) Outcome measures, including response rates, which are based on disposition codes.
(2) Selection biases with respect to sex, age, and race/ethnicity.
(3) Missing values of income.
The measures in this report are designed to document the quality of BRFSS data.2 “Data quality” in this report refers to the accuracy of BRFSS data.
Outcome Measures
The factors affecting the distribution of disposition codes by state may be grouped into differences in telephone systems, sample designs, surveyed populations, and data collection processes. Different outcome measures are variously affected by differences in these factors.
Tables 1 and 2 present the frequency of the thirty-two individual final disposition codes by state. Table 1 shows the distribution of all telephone numbers of determined eligibility status in the sample. Table 2 presents this distribution for numbers of unknown eligibility.
Table 3 presents brief descriptions of each final call disposition code. For more information on the BRFSS protocol and process for assigning disposition codes, as well as a list of interim disposition codes, see the BRFSS 2004 Year-to-Date Data Quality Report Handbook.
Table 4 shows the frequency distribution and Table 5 shows the percent (of all numbers in the sample) distribution of disposition codes for each state grouped into several descriptive categories. The categories shown in Tables 4 and 5 and used in the calculations of the outcome rates in Table 6 are defined below. P in the table below is the proportion of records with a final disposition code of 210 for which more than half of the core questionnaire prior to the demographics section was completed. An interview is considered to be more than fifty percent complete if any question in the Excess Sun Exposure section or a later section has a value other than 7 or 9. These interviews are included in the response rate numerator but are not used in calculating estimates of risk factors and prevalence estimates.
Categories of Call Outcomes | ||
---|---|---|
Category | Disposition Code Definition | Format in Tables/Formulas |
Completed Interview | 110+120+(210* P) | COIN |
Terminations and Refusals | (210*(1- P))+220 | TERE |
Known Household, Possibly Eligible, Non-interview | 230+240+250+260+270+280+ 305+310+315+335 |
KNHH |
Likely Households | 320+325+330+332+340+355+ 370 |
LIHH |
Answering Machine Unknown | 345+350 | AMUR |
Ineligible Households | 410 | INHH |
Non-Contact | 360+365 | NCUS |
Business Non-Residential | 420 | BUNR |
Non-working Out-of-Scope | 405+430+435+440+450 | NOSN |
Eligible, Non-Interview | 210+220+230+240+250+260+ 270+280 |
Elig HH |
Known or Probable Household, Unknown Eligibility | 305+310+315+320+325+330+ 332+335+340+345+350+355+ 370 |
EUHH |
Unknown Eligibility | EUHH+NCUS | Total Unknown |
Ineligible | 410+420+NOSN | Total Ineligible |
All Known or Probable Households | COIN+TERE+KNHH+LIHH+ INHH |
HH |
Total Records | All numbers in sample | TOTAL |
Table 6 provides seven outcome rates for each state that are used to measure respondent cooperation, data quality, and data collection efficiency. The Resolution Rate is the proportion of all telephone numbers in the sample for which the status of the numbers as households with working numbers has been resolved. Records for which household status remains unknown are excluded from the numerator. The formula for the Resolution Rate is
The Screening Completion Rate is the proportion of all known households in which the presence or absence of an eligible respondent has been determined and in which, for eligible households, an interviewer actually spoke to the selected respondent. Households in which the presence or absence of an adult is unknown are excluded from the numerator. Its formula is
The Interview Completion Rate is the proportion of contacted selected respondents who successfully complete an interview. This rate is a type of cooperation rate. An alternate response rate definition is the product of these three rates. The formula for the Interview Completion Rate is
The Cooperation Rate is the proportion of all respondents interviewed of all eligible units in which a respondent was selected and actually contacted. Non-contacts are excluded from the denominator. This rate is based on contacts with households containing an eligible respondent. The denominator of the rate includes completed interviews plus the number of non-interviews that involve the identification of and contact with a selected respondent. A Cooperation Rate below 65 percent may indicate some problem with interviewing techniques. The denominator of the Cooperation Rate consists of records with disposition codes of 110, 120, 210, 220, 250, and 260. Thus, the formula for the BRFSS Cooperation Rate is
A Response Rate is an outcome rate with the number of complete and partial interviews in the numerator and an estimate of the number of eligible units in the sample in the denominator. A proportion of the terminations (210) are included as partial interviews in the BRFSS CASRO Response Rate calculation because more than fifty percent of the core questionnaire was completed for these telephone numbers. The BRFSS CASRO Response Rate calculation assumes that the unresolved numbers contain the same percentage of eligible households as the records whose eligibility or ineligibility are determined. This estimated level of eligibility provides a conservative response rate due to the fact that the proportion of these unknown eligible telephone numbers that are eligible is probably quite low, given the fifteen or more call attempts required by BRFSS protocol. The formula for the BRFSS CASRO Rate is
The response rate is an indicator of the potential for bias in the results of a survey. It does not indicate the actual amount of bias. The actual amount of bias can be conceptualized as a function of two factors—the amount of non-response, which is measured by a response rate, and the differences between the respondents and the non-respondents. A response rate does not address the latter factor. If the non-respondents are highly similar to the respondents for the characteristics of interest, then even a low response rate will result in little non-response bias.
Table 6 also presents the BRFSS Overall Response Rate. The Overall Response Rate is a more conservative response rate that assumes that more unknown records are eligible and thus includes a higher proportion of all numbers in the denominator. The rate assumes that all likely households are households and that 98 percent of known or probable households contain an adult who uses the telephone number.
The Overall Response Rate formula is
The BRFSS Refusal Rate is the proportion of all eligible respondents that refused to complete an interview or terminated an interview prior to the threshold required to be considered a partial interview. Refusals and terminations (TERE) are in the numerator, and the denominator is the same as that of the Response Rate. This formula is
Selection Biases
Tables 7 through 14 present data on the differences between BRFSS and population data with respect to sex, age, and race/ethnicity by state. In these tables, BRFSS data are weighted for the characteristics of the sample design—disproportionate sampling by geographic and density strata (where they exist), number of phones, and number of adults in the household.3 Since these factors are built into the sample design, they should be adjusted for before comparing survey distributions to population distributions. No definitive standards exist with respect to what constitutes a substantial difference between survey and population percentages. One approach would be to examine the distribution of discrepant values with the purpose of identifying extreme values, or outliers, which may indicate biased data.
Income Missing Values
Table 15 presents the percent missing (Don’t know/Not sure, Refused, or either) income by state. Income is the variable in the survey data with the largest percentage of missing values. A larger percentage of missing values for income implies lower quality data for income and, by extension, for other variables.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
* Links to non-Federal organizations are provided solely as a service to our users. Links do not constitute an endorsement of any organization by CDC or the Federal Government, and none should be inferred. The CDC is not responsible for the content of the individual organization Web pages found at this link.
Policies and Regulations | BRFSS Home | Contact Us Page last reviewed: 04/14/2008
United States Department of Health and Human Services |
|