![HHS Logo: bird/faces](white%20hhs%20on%20green.JPG) |
ASPE Research
Notes
INFORMATION FOR DECISION
MAKERS |
Acknowledgments: ASPE would like to thank those reviewers at
the Agency for Health Care Policy and Research (AHCPR), the National Center for
Health Statistics (NCHS), and the Census Bureau whose helpful comments and
expertise contributed greatly to this document.
Introduction
The number of uninsured children in the United States has been an
important policy concern for several years, as reflected in initiatives such as
the State Children's Health Insurance Program. The four Federal surveys that
are major sources of data on uninsured children have played an important role
in informing this policy debate. These surveys--the National Health Interview
Survey (NHIS), the March supplement to the Current Population Survey (CPS), the
Survey of Income and Program Participation (SIPP), and the Medical Expenditure
Panel Survey (MEPS)--can each provide useful estimates of the number of
uninsured children during a particular period of time, and in some cases, at a
point in time.1 Both
MEPS and SIPP also provide information on changes in the insurance status of
individuals over time. This note will explain some of the major reasons why
estimates of uninsured children from these surveys differ and explore the
strengths and weaknesses of each survey.
Some Reasons for Differences Among Estimates
-
Survey Design Differences
- The Length of Time Uninsurance is Measured. Uninsurance
estimates from different surveys may vary because the surveys measure a lack of
insurance over different lengths of time.
- The CPS identifies individuals as uninsured if they have
laced coverage for the entire previous calendar year (although, as discussed
below, many analysts believe that respondents provide information about their
current insurance status). Based on this definition, 10.7 million or 15.0% of
all children under age 18 were uninsured in 1997 and 10.6 million or 14.8% of
all children were uninsured in 1996.
- The SIPP can identify individuals who are uninsured for each
month of a 36-month panel, a calendar year, or the entire 36-month panel.
Generally, the SIPP cross-sectional average monthly uninsured estimates are
consistent with the CPS annual uninsured estimates. In contrast, the SIPP
longitudinal data show annual uninsured estimates which are about half as large
as the CPS uninsured estimates.
- The NHIS identifies individuals as uninsured if they lacked
coverage in the month prior to the survey. Because the month individuals are
interviewed varies, the survey produces an average monthly uninsurance
estimate. Thus, an NHIS estimate for a given year is for an "average" month
during that year. Although both short-term and long-term uninsured would be
included in this definition, the 1996 NHIS uninsured count (9.5 million
children or 13.4% of all children under 18) is slightly lower than that of the
1996 CPS (10.6 million or 14.8%).
- The current 1996 MEPS data are from the first round of a
two-year panel survey and therefore will count as uninsured those without
coverage for the entire interview round (an average of 3-5 months). Because of
this time frame difference, MEPS estimates for each round can be expected to be
somewhat higher than the 12-month estimate from the CPS (but many analysts
believe that CPS does not provide an accurate 12-month uninsurance estimate).
For example, the MEPS 1996 round one estimate (just under 11 million children
under age 18) is somewhat higher than the 1996 CPS estimate (10.6 million
children under 18).2
Once complete 1996 data are available, MEPS will also have the capacity to
produce point-in-time, monthly, and annual estimates of health insurance
coverage.
- Point in time estimates. As discussed above, different
surveys provide uninsurance estimates which cover different lengths of time. At
times it may also be useful to know the number who are uninsured at a given
point in time (e.g., the date of the survey interview). One would expect
point-in-time estimates to be larger than estimates which count only those
uninsured over an entire period of time (e.g., an entire year). 1996 MEPS data
can provide point-in-time estimates (as of the Round 1 and Round 2 interview
dates and as of December 31, 1996). The NHIS estimates can be considered close
to being point-in-time as they indicate lack of coverage in an average month in
that year. In addition, many analysts also believe that a number of respondents
to the CPS provide point-in-time information, i.e., information about their
status at the time they are participating in the survey (March) or about their
status at the end of the previous calendar year, despite the fact that
they survey questions ask about the entire previous calendar year. The
CPS annual estimates are similar to point-in-time estimates generated from the
SIPP and the NHIS has been cited as evidence of this occurrence. How many
respondents misinterpret the relevant questions in this manner is unknown, but
this uncertainty may make it more difficult to interpret and compare
uninsurance data.
- Recall periods. Each survey uses different respondent
recall periods. The respondent is required to remember the previous 14 months
for the CPS since the health supplement is conducted in March following the
calendar year to which the questions pertain. Length of recall is two months
for the NHIS, three to five months for MEPS, and four months for the SIPP.
Recall about insurance status wanes over time, affecting both accuracy and
comparability.
- How "Insurance" is Defined. MEPS defines private insurance
as coverage for hospital and physician services, thereby eliminating
single service, serious and dread disease, workers' compensation, accident, and
disability policies from counting as "coverage." NHIS excludes single service
plans, except for those that cover hospital care, from the definition of
private insurance.3 CPS
and SIPP instruct interviewers to not count single service plans (such as
dental plans) as private insurance, but some single service coverage may get
misreported as comprehensive coverage. All four surveys count military and
veterans health plans as health insurance. However, while CPS, SIPP, and NHIS
include health services received directly from the Department of Veterans
Affairs as coverage, MEPS does not.
- How respondents are asked about insurance. In the NHIS,
respondents are directly asked whether they lack insurance coverage. In the
MEPS, SIPP, and CPS, however, respondents are not explicitly asked this
question; those who deny any type of insurance coverage are assumed to be
uninsured.
-
Differences in Data Handling.
- Data adjustments. Analysts often adjust the raw survey
data to reflect assumed under- or over-reporting or to account for
non-response. Since these adjustments can differ depending on the data analyst,
estimates of the same population derived from the same survey can differ from
one another. Analysts from different organizations tend to employ different
decision algorithms for determining how to count various responses, each of
which may be equally valid. For example, the uninsured estimates from CPS
published by the Employee Benefit Research Institute have at times differed
from those published by the Administration because of differences in the way
the data are compiled.
- Medicaid adjustments. The Census Bureau adjusts Medicaid
coverage data on the CPS raw data file by assigning Medicaid coverage to
individuals whose families receive Aid to Families with Dependent Children
(AFDC) and some individuals who report receipt of Supplemental Security Income
(SSI), even if they do not report Medicaid coverage. This imputation may affect
the counts and characteristics of uninsured children obtained from the CPS.
Nevertheless, CPS estimates of Medicaid enrollment have historically been lower
than administrative data on Medicaid enrollment. NCHS uses a similar adjustment
for estimates based on the NHIS.
- The age-range used to define "children." By convention,
the term "children" is often defined to include only those individuals under
age 18. However, analysts sometimes find it appropriate to include all
individuals under age 19 in the definition of "children." The latter definition
is more appropriate for Medicaid estimates because it is consistent with the
program's eligibility rules; many states have opted to expand Medicaid
eligibility to children up to age 19 meeting certain income and age criteria.
If individuals under 19 are included as "children," the CPS estimate increases
from 10.7 million to 11.6 million uninsured in 1997.
-
Differences in Timeliness of Data. Because of different
lag-times between data gathering and data availability, it is often the case
that results cited at the same time actually reflect data gathered in different
years. For example, the most recent data available from the NHIS were collected
in 1996 and reflect 1996 coverage rates. The most recent data available from
the CPS were collected in March of 1998 and reflect 1997 coverage rates. The
most recent MEPS data on insurance coverage were collected in 1996 and reflect
coverage rates for the first half of 1996.
Strengths and Weaknesses of Each Survey
- CPS Data. The CPS data are widely used because the CPS is
based on a very large sample, is designed to produce credible state-level
estimates (less populous states may require 2 or 3 years of CPS data to produce
such estimates), is available on a timely basis, and provides information on
coverage rates for socio-demographic subgroups of the population. However, as
with most surveys, the CPS has been subject to questions regarding over- and
under-reporting. Specifically, it is thought that the CPS over-counts the
number of individuals who have been uninsured for an entire year, possibly
because respondents answer based on current, rather than previous, coverage
status. In addition, Medicaid coverage status is likely under-reported.
- MEPS Data. MEPS data on an individual's health insurance
status is collected several times a year during a two-year panel survey and
will be collected continuously. These data have the capacity to produce a
variety of estimates of health insurance coverage, including point-of-time,
monthly, and annual estimates. In addition, since these data will be collected
over a two-year period, MEPS data will enable analysts to examine health
insurance dynamics, including changes in coverage and spells without coverage.
MEPS data will provide highly reliable estimates of the population's health
insurance status which can be linked to a variety of individual and household
characteristics, including use of and expenditures for health care services.
Once complete 1996 MEPS data are available in 1999, analysts can examine
insurance status in conjunction with data on sources of payment for health care
to add greater accuracy and precision to the insurance status estimates. The
MEPS sample is smaller than the CPS and NHIS samples, does not contain a
representative sample from each state, and thus cannot be used to make
state-level estimates.
- NHIS Data. NHIS data are gathered continuously, are highly
reliable, and provide detailed information on insurance status, including type
of coverage. The survey also provides information on several measures of health
status, health care utilization, and socio-demographic characteristics of
survey respondents. Relative to the CPS, there is a longer lag-time between
data gathering and data availability, which may cause estimates from the two
surveys to differ even when they are released simultaneously. NCHS is taking
steps to address this situation and anticipates shorter turn-around times in
the future with the implementation of CAPI (Computer Assisted Personal
Interviewing). Like MEPS and SIPP, the NHIS does not contain a representative
sample from each state and is not designed to make state-level estimates.
- SIPP Data. As a longitudinal survey, the SIPP data provide the
capacity to examine the dynamics of health insurance. It is possible to
estimate the duration of spells without health insurance. These data are also
capable of producing health insurance estimates for various time periods, such
as point-in-time, monthly, annual, or over the full panel. As part of the core
data collected in the SIPP, health insurance data can also be linked to other
sections of the survey, such as utilization of health care services, child
well-being, and disability. The SIPP sample is smaller than the CPS and NHIS
samples, does not contain a representative sample for each state, and thus can
not be used to make state-level estimates.
Conclusion
The bottom line is that the estimated rate of uninsurance among children
may vary depending upon the data source and data adjustments. The decision of
which survey to use for uninsurance estimates may depend on the purpose of the
analysis. The CPS is the only source of state-level uninsurance estimates. MEPS
and SIPP are the best sources for examining changes in individuals' insurance
status over time. NHIS, MEPS, and SIPP can provide point-in-time estimates of
uninsurance rates.
Despite the differences and the strengths and weaknesses that
distinguish these surveys, the estimates derived from each paint a relatively
consistent picture of health coverage rates in the United States. Critical
policy concerns such as the disparity in coverage rates across income groups
and the number of children that lack coverage are clearly apparent in the
empirical data from all four surveys.
Notes
-
The NHIS is administered by the National Center
for Health Statistics, which is part of the Department of Health and Human
Services (HHS). The CPS and SIPP are administered by the Census Bureau. The
MEPS is administered by the Agency for Health Care Policy and Research, also
part of HHS.
-
According to MEPS 1996 round one data, 15.4% of
all children under 18 are uninsured. The 95% confidence interval around this
estimate (14.6%-16.2%) overlaps the CPS estimate of uninsured children in 1996
(14.8%), which means that there is no statistically significant difference
between the two estimates.
-
In 1997, the National Center for Health
Statistics (NCHS) made some minor changes in how the uninsured are defined
based on the NHIS. These changes have tended to decrease the percent of
uninsured slightly. Those with public assistance coverage or AFDC (but without
a report of Medicaid) are now counted as insured. In addition, fewer persons
are deleted from calculations due to missing data. In 1998, NCHS revised
estimates to count those with only Indian Health Service coverage as uninsured.
This change has had little effect on national estimates. NCHS has produced a
revised series of estimates that appear in Health, United States,
1998.
CONTACT PERSONS: Ariel Winter and M. Eugene Moyer, Office of
Health Policy.
ASPE Research Notes is circulated periodically to the
Department of Health and Human Services by the Office of the Assistant
Secretary for Planning and Evaluation. This paper reflects only the views of
its author and does not necessarily represent the position of the U.S.
Department of Health and Human Services. For further information on health
insurance, call the Office of Health Policy at 202-690-6870.
Selected Differences Between Surveys' Uninsurance
Estimates |
Survey |
Length
of Time Uninsurance Measured |
Time Period of Estimates |
Respondent Recall Period |
Most Recent Data From |
Most Recent Estimate
of Uninsured Children under 18 |
Source of Data
on Health Insurance Dynamics? |
Source of State Estimates? |
CPS |
Previous calendar
year |
Over time (but perhaps
closer to point in time) |
Prior 14
months |
1997 |
10.7 million
(15.0%) |
No |
Yes |
SIPP |
Each month of 36-month
panel, entire calendar year, or entire 36-month panel |
Point in time and over
time |
Prior 4
months |
1993 panel
(2/93-1/96) |
5.4 million (7.1%) (for
calendar year 1994) |
Yes |
No |
NHIS |
Month prior to
survey |
Close to point in
time |
Prior 2
months |
1996 |
9.5 million
(13.3%) |
No |
No |
MEPS |
Entire interview round
(3-5 months) |
Point in time and over
time |
Prior 3-5
months |
First half of
1996 |
11 million
(15.4%) |
Yes |
No |
ASPE RESEARCH NOTES ARTICLES AVAILABLE
- Cost of Teenage Childbearing: Current Trends (August
1992)
- Full HTML Version http://aspe.hhs.gov/daltcp/reports/1992/rn03.htm
- Full PDF Version http://aspe.hhs.gov/daltcp/reports/1992/rn03.pdf
- Counting Persons in Poverty on the Current Population Survey
(August 1998)
- Full HTML Version http://aspe.hhs.gov/daltcp/reports/1998/rn20.htm
- Full PDF Version http://aspe.hhs.gov/daltcp/reports/1998/rn20.pdf
- Disability Among Children (January 1995)
- Full HTML Version http://aspe.hhs.gov/daltcp/reports/1995/rn10.htm
- Full PDF Version http://aspe.hhs.gov/daltcp/reports/1995/rn10.pdf
- Eldercare: The Impact of Family Caregivers' Employment on Formal
and Informal Helper Hours (August 1995)
- Full HTML Version http://aspe.hhs.gov/daltcp/reports/1995/rn14.htm
- Full PDF Version http://aspe.hhs.gov/daltcp/reports/1995/rn14.pdf
- Estimating Eligibility for Publicly-Financed Home Care: Not a
Simple Task
(April 1992)
- Full HTML Version http://aspe.hhs.gov/daltcp/reports/1992/rn01.htm
- Full PDF Version http://aspe.hhs.gov/daltcp/reports/1992/rn01.pdf
- Health Insurance in 1994 from the Current Population Survey:
Measurement Difficulties (November 1996)
- Full HTML Version http://aspe.hhs.gov/daltcp/reports/1996/rn15.htm
- Full PDF Version http://aspe.hhs.gov/daltcp/reports/1996/rn15.pdf
- Informal Caregiver "Burnout": Predictors and Prevention (April
1993)
- Full HTML Version http://aspe.hhs.gov/daltcp/reports/1993/rn05.htm
- Full PDF Version http://aspe.hhs.gov/daltcp/reports/1993/rn05.pdf
- Licensed Board and Care Homes: Preliminary Findings from the 1991
National Health Provider Inventory (May 1993)
- Full HTML Version http://aspe.hhs.gov/daltcp/reports/1993/rn06.htm
- Full PDF Version http://aspe.hhs.gov/daltcp/reports/1993/rn06.pdf
- March 1992 Current Population Survey Shows Health Insurance
Coverage Up in 1991: Number of Medicaid Recipients Also Rises (February
1993)
- Full HTML Version http://aspe.hhs.gov/daltcp/reports/1993/rn04.htm
- Full PDF Version http://aspe.hhs.gov/daltcp/reports/1993/rn04.pdf
- March 1993 Current Population Survey Re-Benchmarked on 1990
Census (March 1995)
- Full HTML Version http://aspe.hhs.gov/daltcp/reports/1995/rn12.htm
- Full PDF Version http://aspe.hhs.gov/daltcp/reports/1995/rn12.pdf
- Number of Medicaid Recipients Up: CPS Shows the Number of
Uninsured Also Rises (April 1992)
- Full HTML Version http://aspe.hhs.gov/daltcp/reports/1992/rn02.htm
- Full PDF Version http://aspe.hhs.gov/daltcp/reports/1992/rn02.pdf
- Population Estimates of Disability and Long-Term Care
(February 1995)
- Full HTML Version http://aspe.hhs.gov/daltcp/reports/1995/rn11.htm
- Full PDF Version http://aspe.hhs.gov/daltcp/reports/1995/rn11.pdf
- Research and Other Developments of Interest in Employer Group
Long-Term Care Insurance ( )
- Full HTML Version http://aspe.hhs.gov/daltcp/reports/rn19.htm
- Full PDF Version http://aspe.hhs.gov/daltcp/reports/rn19.pdf
- The Elderly with Disabilities: At Risk for High Health Care
Costs (February 1994)
- Full HTML Version http://aspe.hhs.gov/daltcp/reports/1994/rn08.htm
- Full PDF Version http://aspe.hhs.gov/daltcp/reports/1994/rn08.pdf
- The Medicaid Personal Care Services Option Part I: Cross-State
Variations and Trends Over Time (November 1993)
- Full HTML Version http://aspe.hhs.gov/daltcp/reports/1993/rn07.htm
- Full PDF Version http://aspe.hhs.gov/daltcp/reports/1993/rn07.pdf
- The Medicaid Personal Care Services Option Part II:
Consumer-Directed Models of Care (December 1994)
- Full HTML Version http://aspe.hhs.gov/daltcp/reports/1994/rn09.htm
- Full PDF Version http://aspe.hhs.gov/daltcp/reports/1994/rn09.pdf
- Trends in AFDC and Food Stamp Benfits: 1972-1994 (May
1995)
- Full HTML Version http://aspe.hhs.gov/daltcp/reports/1995/rn13.htm
- Full PDF Version http://aspe.hhs.gov/daltcp/reports/1995/rn13.pdf
- Understanding Estimates of Uninsured Children: Putting the
Differences in Context (January 1999)
- Full HTML Version http://aspe.hhs.gov/daltcp/reports/1999/rn21.htm
- Full PDF Version http://aspe.hhs.gov/daltcp/reports/1999/rn21.pdf
To obtain a printed copy of this report, send the full report title and
your mailing information to:
U.S. Department of Health and Human Services
Office of Disability,
Aging and Long-Term Care Policy
Room 424E, H.H. Humphrey Building
200
Independence Avenue, S.W.
Washington, D.C.
20201
FAX: 202-401-7733
Email: webmaster.DALTCP@hhs.gov
RETURN TO:
Office of Disability, Aging and
Long-Term Care Policy (DALTCP) Home [http://aspe.hhs.gov/_/office_specific/daltcp.cfm]
Assistant
Secretary for Planning and Evaluation (ASPE) Home [http://aspe.hhs.gov]
U.S. Department of
Health and Human Services Home [http://www.hhs.gov]