|
|
| Income |
|
Guidance on Survey Differences in Income and Poverty Estimates
March 19, 2002
The Census Bureau collects income data on several major national surveys:
Each of these surveys differs from the others in the length and detail of its questionnaire, the number of households interviewed, the methodology used to collect and process the data, and, consequently, in the income and poverty estimates produced. In addition, the Census Bureau has a Small Area Income and Poverty Estimates program (SAIPE) that uses survey data with data from other sources in statistical models to produce income and poverty estimates. As a result of this multiplicity of sources, it is important to understand that different surveys and methods produce different results, and consequently when it is appropriate to use each survey or method.
Highlights
Background
The Annual Demographic Supplement to the March Current Population Survey (CPS) is designed to give official annual (one-year) national estimates of income and of poverty numbers and rates. It is however used for many other purposes. For example, the Balanced Budget Act of 1997 directed the Health Care Financing Administration (now the Centers for Medicare and Medicaid Services) to use a 3-year average of the CPS estimate of the number of children under 19 without health insurance in families with incomes below 200 percent of their poverty threshold, and the Congress later provided funds to increase the sample size in order to reduce the sampling error of that estimate. The House Ways and Means Subcommittee on Human Resources has used the national child poverty rate to demonstrate the effectiveness of the Personal Responsibility and Work Opportunity Reconciliation Act of 1996. In response to requests for sub-national detail, several years ago, the Census Bureau began publishing estimates of state poverty, recommending that a comparison of 2-year moving averages be used to understand changes in poverty and median household income within a state, and that 3-year averages be used for comparisons across states.
Data on income are also released periodically from the Survey of Income and Program Participation (SIPP), a longitudinal survey designed to track changes in income over time for households. Whereas the CPS is a labor force survey with supplementary questions on income, the SIPP focuses on income and typically reports more income (and therefore lower poverty) than the CPS. The SIPP is useful mainly for examining the dynamics of income (and poverty) change over a three-year period, since it collects monthly income by source using a much more detailed questionnaire than the CPS. The National Academies of Science has recommended that the SIPP become the source of official income and poverty statistics.
Sub-national detail is the raison d'etre of the Census 2000 long form. Estimates for 1999 median household and family income, and poverty, among other subjects, will be released on a state-by-state basis during the Summer 2002. The great advantage of Census 2000 is its huge sample size when compared to the CPS - roughly 19 million households were mailed a long form versus roughly 50,000 households per month for the CPS in 2000 - resulting in much lower sampling errors. But there are many other differences between the estimates from the two sources. The income questions asked on Census 2000 were much less detailed than those asked by the CPS, and analysis has shown that asking less detailed questions generally leads the respondent to omit less common income sources, resulting therefore in lower income and higher poverty estimates. The data collection methodology was substantially different, with the CPS being collected by interviewers via personal visits using Computer Assisted Personal Interviewing (CAPI), or by Computer Assisted Telephone Interviewing (CATI). Census 2000, in contrast, used a self-response mail-out/mail-back method, followed by personal visits to non-respondents (but by much less experienced interviewers). Census 2000 used a more up-to-date sampling frame (the Master Address File used by Census 2000) than the CPS, which used the 1990 Census results, updated with new construction. All of these differences lead us to expect differences between Census 2000 income and poverty estimates and CPS income and poverty estimates, just as were seen when the 1990 Census data were released.
The Census 2000 Supplementary Survey (C2SS), as well as the 2001 and 2002 Supplementary Surveys, are part of a development program for the forthcoming American Community Survey (ACS). They also collect income data, for a much larger sample than the CPS (700,000 households per year), but rely heavily on questionnaire responses mailed in by respondents. C2SS estimates were collected on a rolling basis every month throughout calendar year 2000 and the questions were asked about income "in the previous 12 months." Thus, for example, those interviewed in January 2000 were asked about the January to December 1999 period, and those interviewed in December 2000 were asked about the December 1999 to November 2000 period. Thus, the calendar time period for C2SS income estimates does not correspond to the CPS (the previous calendar year), or to Census 2000. The C2SS used the Census 2000 self-response mail-out/mail-back methodology, followed by CATI, followed by CAPI. The C2SS used a more up-to-date sampling frame (the Master Address File used by Census 2000) than the CPS, which used the 1990 Census results, updated with new construction. To develop appropriate weights for the interviews, the C2SS used results from Census 2000, likely to be more accurate than the weights used by the CPS, which were also based on the 1990 Census, as adjusted for the undercount in 1990 and for population change since then. Finally, while the CPS reaches more of hard-to-interview populations such as young Black men than do most other Census Bureau surveys, the C2SS did even better than the CPS, which also might have affected the results. Finally, the C2SS excludes group quarters from its sampling frame, slightly affecting the estimates of poverty, as some people in the poverty universe are in noninstitutional group quarters. The C2SS methodology is being used for the final years of the ACS development period, to field the 2001 Supplementary Survey (SS01) and the 2002 Supplementary Survey (SS02).
Adding yet more sample to the CPS to reduce the sampling error of sub-national estimates can only go so far because of its great cost. The ACS methodology holds the most promise of providing timely sub-national data on income and poverty. But such estimates will never match the official national estimates from the CPS, because of differences in questionnaires, data collection methodology, reference period, edit procedures, etc. The Census Bureau's Small Area Income and Poverty Estimates program (SAIPE) holds the most promise of reconciling the different results from these alternate surveys. SAIPE uses statistical methods to improve sub-national estimates of income and poverty by using information from the CPS, the decennial census, population estimates, and administrative records such as aggregate food stamps and adjusted gross income from tax returns to produce CPS-consistent estimates for population subgroups (such as children) for states and counties for income and poverty. We are now undertaking the research necessary to incorporate the Census 2000 and the C2SS data into the SAIPE program, and expect to release estimates for median household income and poverty by age group, for states and counties for 1999, in Fall 2002.
