Appendix E: BRFSS Measures, Data and Benchmarks
In 2003, asthma data were collected under the Behavioral Risk Factor Surveillance System (BRFSS) for 5 process measures, 7 outcome measures, and 3 prevalence measures. Those
measures for adults with asthma include asthma history, routine check ups,
doctor visits for asthma, limited activity due to asthma, medications for asthma,
asthma symptoms, asthma episodes, emergency department visits, urgent care
visits, and sleep difficulty due to asthma. The number of entities reporting
varied from 15 to 54 depending on the measure. All 50 States, DC, and 3 U.S.
Territories collected data on receipt of influenza vaccination in the past
year. In our analysis, adult smokers with asthma were studied to determine
the prevalence of smoking and asthma.
The BRFSS data are based on telephone surveys developed by the Centers for Disease Control and Prevention (CDC) but administered
by each State independently. The survey consists of a core set of questions
developed by CDC, additional questions developed by the States, and separate,
optional modules for States to use. The asthma module, which contains
the quality-of-care questions, is optional for State use. More information
about the BRFSS data and methods as well as interactive databases with some
State and local level asthma data are available at: http://www.cdc.gov/brfss.
Limitations of BRFSS Data
Every data source has limitations that can relate to the population represented,
methods used to collect the data, definitions, and analytic approaches. These
factors affect the estimates generated from a data set. When similar
measures from two data sets differ, the cause can usually be traced to the
limitations of the data sets. By understanding the limitation of a data set,
the strengths and weakness of estimates from the data set can be assessed and
the estimates can be used more responsibly.
Limitations of BRFSS data include the following:
- BRFSS samples are kept small to minimize survey costs for States. The
State BRFSS samples for the year 2001 range from 1,888 to 8,628 respondents
(see: http://www.cdc.gov/brfss/technical_infodata/surveydata/2003.htm). Small
samples increase the variance of estimates and decrease the size of the difference
between two subpopulations that can be detected through the survey responses.
In fact, among the asthma measures, the small sample sizes impeded statistical
tests of differences, as discussed below.
- The BRFSS survey excludes people without a residential phone and people
who are institutionalized. This means that the total population
of interest—all people with asthma—will not be represented in
the estimates that come from the survey.a This
weakness can be dealt with by carefully discussing BRFSS results in relation
to the population it represents.
- BRFSS data are self-reported and reflect the perceptions of respondents. An
advantage of self-reports is that they can reveal information that cannot
be obtained from other sources; for example, the receipt of flu vaccinations
for people who do not see a doctor during the year. A disadvantage
of self-report data is that respondents may have difficulty recalling events,
understanding or interpreting questions, or responding truthfully to questions
such as about compliance with advice. Furthermore, cultural and language
barriers and limited health knowledge can affect the quality of self-reported
data.a These
problems may occur with different propensity for different subgroups.
BRFSS data, like most surveys, are limited by budget constraints. Because
BRFSS is funded by States which vary considerably in resources allocated to
health surveys, these fiscal disparities may affect the quality of the data
across States. Such data quality shortcomings can include bias from differential
response rates, varying followup periods, and variations in interviewer protocols
or skills (for example, extent of probing for answers).
Small Sample Size in BRFSS
Table E.1 shows that small sample sizes in the BRFSS supplemental asthma survey
result in tests that are unreliable. For example for smoking cessation
counseling, 15 of 15 reporting States could not be distinguished from
the average of the top 2 States (or top10 percent of States). This is partly
because smoking cessation counseling is commonly provided across all States
(the distribution of percent counseled is narrow), in combination with the
small numbers of individuals interviewed in BRFSS. The smaller the difference
to be detected, the greater the sample needed. The same issues are apparent
for the measure "average number of symptom-free days in the past 2 weeks."
Fourteen of 19 estimates are indistinguishable from the top decile, again a problem
of small sample size.
By contrast, "flu shots in the past 12 months" is a measure collected
from the core BRFSS survey and thus more reliable estimates result. Eleven
of 54 entities represent States comparable to the best-in-class average of
5 States in the top 10 percent.
The issue of sample size is the main reason that the National Healthcare Quality Report (NHQR), which produces
annual estimates, did not include State-level BRFSS data. For State estimates,
multiple years of BRFSS should be used.
Estimates for individual BRFSS measures by State (including the District of
Columbia and U.S. Territories) are presented in Tables E.2-E.16.
Tables, Appendix E:
Table E.1: Selected quality measures for asthma by State, District of Columbia,
and U.S. Territory, 2003.
Table E.2: Lifetime asthma
prevalence: Percent of people who were ever told by a health professional that
they have asthma by State, District of Columbia, and U. S. Territory, 2003.
Table E.3: Current asthma prevalence:
Percent of people who were ever told they have asthma who still have asthma
by State, District of Columbia, and U.S. Territory, 2003.
Table E.4: Age at asthma diagnosis:
Percent of adults with asthma who were diagnosed before age 10 by State, District
of Columbia, and U.S. Territory, 2003.
Table E.5: Urgent care visits:
Percent of adults currently with asthma who had at least one urgent care visit
for asthma with their provider in the past 12 months by State, District of
Columbia and U.S. Territory, 2003.
Table E.6: Emergency room visits:
Percent of adults with asthma who have had at least one visit to the emergency
room for asthma in the past 12 months by states, District of Columbia and U.S.
Territory, 2003.
Table E.7: Asthma attacks/episodes:
Percent of adults with asthma who had an asthma episode in the past 12 months
by State, District of Columbia and U.S. Territory, 2003.
Table E.8: Limited activity
due to asthma: Average number of days adults with asthma were unable to work
or carry out usual activities in the past 12 months by State, District of Columbia
and U.S. Territory, 2003.
Table E.9: No sleep difficulty
due to asthma: Percent of adults with asthma who had no difficulty sleeping
due to asthma during the past month by State, District of Columbia and U.S.
Territory, 2003.
Table E.10: Routine care for
asthma: Percent of adults with asthma who had 2 or more planned care visits
for asthma during the past 12 months by State, District of Columbia and U.S.
Territory, 2003.
Table E.11: Doctors visits
for asthma: Percent of adults with asthma who had a physician visit for asthma
in the past 12 months by State, District of Columbia and U.S. Territory, 2003.
Table E.12: Medications for
asthma: Percent of adults with asthma who took asthma medication in the past
month by State, District of Columbia and U.S. Territory, 2003.
Table E.13: Asthma symptom-free
days: Average number of days adults with asthma were free of asthma symptoms
in past 2 weeks by State, District of Columbia and U.S. Territory, 2003.
Table E.14: Asthma symptoms:
Percent of adults with asthma who experienced asthma symptoms every day in
past 2 weeks by State, District of Columbia, and U.S. Territory, 2003.
Table E.15: Smoking cessation
counseling: Percent of adults with asthma who were advised to quit smoking
by a health professional by State, District of Columbia, and U.S. Territory,
2003.
Table E.16: Percent of all
adults who received flu shots and percent of adults with asthma who received
flu shots by State, District of Columbia, and U.S. Territory, 2003.
a. Nelson D, Holtzman D,
Bolen J, Stanwyck C, Mack K. Reliability and validity of measures from the
behavioral risk factor surveillance system (BRFSS). Sozial un Praventivmedizin
2001;46(Supp 1):S3-42.
Return to Contents
Proceed to Next Section