Understanding Different Estimates of Uninsured Children: Putting the Differences in Context

Acknowledgments: ASPE would like to thank those reviewers at the Agency for Health Care Policy and Research (AHCPR), the National Center for Health Statistics (NCHS), and the Census Bureau whose helpful comments and expertise contributed greatly to this document.

Introduction

The National Health Interview Survey (NHIS), the March supplement to the Current Population Survey (CPS), and the Medical Expenditure Panel Survey (MEPS) can each provide useful estimates of the number of uninsured children in the United States during a particular period of time, and in some cases, at a point in time. Both MEPS and a fourth survey instrument, the Survey of Income and Program Participation (SIPP), provide information on changes in the insurance status of individuals over time, since both collect data from individuals several times during their respective survey periods.

Some Reasons for Differences Between Estimates

1. Differences in the length of time an individual must have been uninsured to be counted as such when the data are collected.

Uninsurance estimates from different surveys will vary because the surveys measure a lack of insurance over different lengths of time.

• The CPS identifies individuals as uninsured if they have lacked coverage for the entire previous calendar year. Based on this definition, 9.8 million or 13.8% of all children under age 18 were uninsured in 1995, and 10.6 million or 14.8% of all children were uninsured in 1996.
• The SIPP identifies individuals who are uninsured for each month of a 4-month reference period. Generally, the SIPP cross-sectional average monthly uninsured estimates are consistent with the CPS uninsured estimates. In contrast, the SIPP longitudinal data show annual uninsured estimates which are about half as large as the CPS uninsured estimates.
• The NHIS identifies individuals as uninsured if they lacked coverage in the month prior to the survey. Because the month individuals are interviewed varies, the survey gathers uninsurance data for different months. These several monthly estimates are consolidated into an average monthly uninsurance estimate. Thus, an NHIS estimate for a given year is for an "average" month during that year. Although both short-term and long-term uninsured would be included in this definition, the NHIS uninsured count (9.5 million children (standard error of 0.25 million) or 13.3% (standard error of 0.3%) of all children under 18 in 1995) is similar to that of the CPS.
• The current 1996 MEPS data are from the first round of a two-year panel survey and therefore will count as uninsured those without coverage for the entire interview round (an average of 3-5 months). Because of this time frame difference, MEPS estimates for each round can be expected to be somewhat higher than the 12-month estimate from the CPS (but see section 4 on why many analysts believe that CPS does not provide an accurate 12-month uninsurance estimate). For example, the MEPS 1996 round one estimate (just under 11 million children under age 18) is somewhat higher than the 1996 CPS estimate (10.6 million children under 18). According to MEPS 1996 round one data, 15.4% of all children under 18 (standard error of 0.77%) are uninsured. Once complete 1996 data are available, MEPS will also have the capacity to produce point-in-time (see below), monthly, and annual estimates of health insurance coverage.

2. Children's coverage estimates may vary due to differences in the age-range used to define "children."

• Factors independent of the data source have also contributed to the confusion surrounding the number of uninsured children. By convention, the term "children" is often defined to include only those individuals under age 18. However, analysts sometimes find it appropriate to include all individuals under age 19 in the definition of "children." The latter definition is more appropriate for Medicaid estimates because it is consistent with the program's eligibility rules; many States have opted to expand Medicaid eligibility to children up to age 19 meeting certain income and age criteria. By the year 2002, when a Federally-mandated expansion is fully phased-in, Medicaid will cover all children in poverty and under the age of 19. If eighteen-year-olds are included as "children," the CPS estimate increases from 10.6 million to 11.3 million uninsured in 1996.

3. Survey estimates of uninsured populations may differ because of the way "insurance" is defined.

• For example, MEPS defines private insurance as coverage for hospital and physician services thereby eliminating single service and dread disease policies from counting as "coverage." NHIS(1) excludes single service plans, except for those that cover hospital care, from the definition of private insurance. CPS, on the other hand, does not allow for this distinction. In addition, MEPS does not include health care provided by the Department of Veterans Affairs or the Indian Health Service as insurance coverage, while the CPS and NHIS do.

4. Differences in Survey Design

• Point-in-Time versus Period of Time. As discussed above, different surveys provide uninsured estimates which cover different lengths of time. At times it may also be useful to know the number who are uninsured at a given point-in-time (e.g., the date of the survey interview). One would expect point-in-time estimates to be larger than estimates which count only those uninsured over an entire period of time (e.g., an entire year). 1996 MEPS data can provide point-in-time estimates (as of the Round 1 and Round 2 interview dates and as of December 31, 1996). The NHIS estimates can be considered close to being point-in-time as they indicate lack of coverage in an average month in that year. In addition, many analysts also believe that a number of respondents to the CPS provide point-in-time information, i.e., information about their status at the time they are participating in the survey (March) or about their status only at the end of the previous calendar year, despite the fact that the survey questions ask about the entire previous calendar year. That CPS estimates are similar to point-in-time estimates generated from the SIPP and the NHIS has been cited as evidence of this occurrence. How many respondents misinterpret the relevant questions in this manner is unknown, but this uncertainty can create another point of contention when interpreting and comparing data on this issue.
• Recall Periods. Each survey uses different respondent recall periods. The respondent is required to remember the previous 14 months for the CPS since the health supplement is conducted in March following the calendar year to which the questions pertain. Length of recall is two months for the NHIS, and four months for the SIPP. Recall about insurance status wanes over time, affecting both accuracy and comparability.
• Family Respondent(s). The surveys also differ in terms of which and how many family members respond to questions. This too can affect accuracy and comparability.
• Question Presentation. The wording and order of questions differ across surveys. This also can affect accuracy and comparability.

5.Differences in Data Handling

• Data Adjustments. Analysts often adjust the raw survey data to reflect assumed under- or over-reporting or to account for non-response. Since these adjustments can differ depending on the data analyst, estimates of the same population derived from the same survey can differ from one another. Analysts from different organizations tend to employ different decision algorithms for determining how to count various responses, each of which may be equally valid. For example, the uninsured estimates from CPS published by the Employee Benefit Research Institute (EBRI) have at times differed from those published by the Administration because of differences in the way the data are compiled.
• Medicaid Adjustments. Census also adjusts Medicaid coverage data in the CPS by assigning Medicaid coverage to individuals whose families receive Aid to Families with Dependent Children (AFDC) and some who receive Supplemental Security Income (SSI), even if they do not report Medicaid coverage. This imputation may affect the counts and characteristics of uninsured children obtained from the CPS. Nevertheless, CPS estimates of Medicaid enrollment have historically been lower than HCFA(now known as CMS) administrative data on Medicaid enrollment. The National Center for Health Statistics (NCHS) uses a similar adjustment for estimates based on the NHIS.

6. Differences in Timeliness of Data

• Time-lag between data gathering and data availability. Because of different lag-times between data gathering and data availability, it is often the case that results cited at the same time actually reflect data gathered in different years. For example, the most recent data available from the NHIS were collected in 1995 and reflect 1995 coverage rates. The most recent data available from the CPS were collected in March of 1997 and reflect 1996 coverage rates. Recently released MEPS data were collected in 1996 and reflect coverage rates for the first half of 1996.

Strengths and Weaknesses of Each Survey

• CPS Data. The CPS data are widely used because the CPS is based on a very large sample, is designed to produce credible state-level estimates (less populous states may require 2 or 3 years of CPS data to produce such estimates), is available on a timely basis, and provides information on coverage rates for socio-demographic subgroups of the population. However, as with most surveys, the CPS has been subject to questions regarding over- and under-reporting. Specifically, it is thought that the CPS over-counts the number of individuals who have been uninsured for an entire year, possibly because respondents answer based on current, rather than previous, coverage status (see number 4 above, on differences in survey design). In addition, Medicaid coverage status is likely underreported.
• MEPS Data. MEPS data on an individual's health insurance status is collected several times a year during a two-year panel survey and will be collected continuously. These data have the capacity to produce a variety of estimates of health insurance coverage, including point-in-time, monthly, and annual estimates. In addition, since these data will be collected over a two-year period, MEPS data will enable analysts to examine health insurance dynamics, including changes in coverage and spells without coverage. MEPS data will provide highly reliable estimates of the population's health insurance status which can be linked to a variety of individual and household characteristics, including use of and expenditures for health care services. Once complete 1996 MEPS data are available, analysts can examine insurance status in conjunction with data on sources of payment for health care to add greater accuracy and precision to the insurance status estimates. The MEPS sample is smaller than the CPS and NHIS samples, does not contain a representative sample from each state, and is not used to make state-level estimates.
• NHIS Data. NHIS data are gathered continuously, are highly reliable, and provide detailed information on insurance status, including type of coverage. The survey also provides information on several measures of health status, health care utilization, and sociodemographic characteristics of survey respondents. Relative to the CPS, there is a longer lag-time between data gathering and data availability which may cause analyses from the two surveys to differ although they are being released simultaneously. NCHS is taking steps to address this situation and anticipates shorter turn-around times in the future with the implementation of CAPI (Computer Assisted Personal Interviewing). Like MEPS and SIPP, the NHIS does not contain a representative sample from each state and is not designed to make state-level estimates.
• SIPP Data. As a longitudinal survey, the SIPP data provide the capability to examine the dynamics of health insurance. It is possible to estimate the duration of spells without health insurance, for example, how long children go without health insurance. These data also provide the capability of producing a variety of health insurance estimates for various time periods, such as point-in-time, monthly, annual, or over the full panel. As part of the core data collected in the SIPP, health insurance data can also be linked to various children's topical modules, such as utilization of health care services, child well- being, and disability. The SIPP sample is smaller than the CPS and NHIS samples, does not contain a representative sample from each state, and is not used to make state-level estimates.

Conclusion

Despite the various differences and the pronounced strengths and weaknesses that distinguish these surveys, the estimates derived from each actually paint a relatively consistent picture of health coverage rates in the United States. The erosion of employer-based coverage, the disparity in coverage rates across income groups, the fact that a significant number of children lack coverage -- these critical policy concerns are clearly apparent in the empirical data compiled from all four surveys.

The bottom line is that the estimated rate of uninsurance among children may vary depending upon the definition of uninsured, the data source, and the data adjustments.

(1) In the summer of 1997 the National Center for Health Statistics (NCHS) made some minor changes in how the uninsured are defined based on the NHIS. These changes have tended to decrease the percent uninsured slightly. Those with Indian Health Service coverage, public assistance coverage, or AFDC (but without a report of Medicaid) are now counted as insured. In addition, fewer persons are deleted from calculations due to missing data. NCHS has produced a revised series of estimates that will appear in Health, United States, 1998.