|
|
| Income |
|
Guidance on Differences in Income and Poverty Estimates
from Different Sources
August 19, 2004
The Census Bureau reports income and poverty estimates from several major national household surveys and programs:
● Annual Social and Economic Supplement to the Current Population Survey
● Survey of Income and Program Participation
● Census 2000 long form
● Small Area Income and Poverty Estimates program
Each of these surveys differs from the others in some ways, such as the length and detail of its questionnaire, the number of households included (sample size), and the methodology used to collect and process the data. The Small Area Income and Poverty Estimates program, a part of the American Community Survey program, creates statistical models to produce income and poverty estimates by combining survey results with administrative records. As a result of this multiplicity of sources, it is important to understand that different surveys and methods, which are designed to meet different needs, also produce different results. We have provided this guidance to assist data users in selecting the data source most appropriate for their application.
HIGHLIGHTS
Annual Social and Economic Supplement (ASEC) to the Current Population Survey (CPS)
Because of its detailed questionnaire and its experienced interviewing staff trained to explain concepts and answer questions, the CPS ASEC is the source of timely official national estimates of poverty levels and rates and of widely used estimates of household income and individual earnings, as well as the distribution of that income.
The CPS ASEC provides a consistent historical time series of many decades in length at the national level, and can also be used to look at state-level trends and differences (through multi-year averages). The relatively large sampling errors of state-level estimates for smaller states somewhat limit their usefulness.
American Community Survey (ACS)
Starting with 2000, the ACS provides subnational estimates of income and poverty for essentially all places, counties, and metropolitan areas with a population of at least 250,000. Estimates have also been produced for the nation and the states. These estimates have a different reference period than the CPS and a different population universe. 1/ The sample size of this survey since 2000 (about 800,000 addresses per year) makes the ACS exceptionally useful for subnational analyses.
We expect the fully implemented ACS to have an annual sample size in 2005 of about 3 million housing unit addresses across the country. That implementation will lead to release of annual estimates from the ACS in 2006 (and every year thereafter) for all geographic areas with a population of 65,000 or more. Three-year averages would then be available starting in 2008 for areas and subpopulations as small as 20,000. Five-year averages would then be available for census tracts/block groups and for small subgroups of the population starting in 2010. Both will be updated every year after they are first available. Because of its large sample size, estimates from the fully implemented ACS will provide the best survey-based subnational income and poverty estimates. Under full implementation, time series trends for all geographic areas, and for small population subgroups, will be available.
Survey of Income and Program Participation (SIPP)
The SIPP is useful mainly for understanding changes for the same households in income and poverty, that is the dynamics of income and poverty, over time (up to 3 or 4 years), and for examining the nature and frequency of poverty spells and periods of income receipt of less than a year.
Census 2000 long form
The best measure of change over the decade of the 1990s for subnational areas, even small places, and for subpopulations, are the comparisons of Census 2000 results with those from the 1990 Census.
Small Area Income and Poverty Estimates (SAIPE) program
For the geographic areas it covers, the SAIPE program provides the most accurate subnational estimates of median household income and poverty for different age groups, but with a time lag. Its estimates are controlled to match the CPS ASEC annual estimates at the national level, and will soon use the ACS estimates as an additional input to improve its estimates yet further.
BACKGROUND
The Annual Social and Economic Supplement to the Current Population Survey (CPS ASEC) is designed to give annual, calendar-year, national estimates of income and official poverty numbers and rates. It is, nonetheless, used for many other purposes, including funds allocation.
The CPS is basically a labor force survey, not an income survey, and is conducted every month by the Census Bureau for the Bureau of Labor Statistics using Computer-Assisted Telephone Interviewing (CATI) and Computer-Assisted Personal Interviewing (CAPI). The Basic CPS is used to calculate the monthly unemployment rate estimates. Supplements are added in most months; the ASEC is conducted in February, March, and April with a sample of about 100,000 addresses per year. The questionnaire asks about income from more than 50 sources and records up to 27 different income amounts, including receipt of numerous noncash benefits, such as food stamps (coupons used as cash for qualified food purchases), and housing assistance.
The American Community Survey (ACS), an integral part of the plan to redesign the decennial census, will replace the census long form. During the 2000-2004 testing program, the ACS has been collecting income data for a much larger sample than the CPS ASEC (about 800,000 addresses per year). When fully implemented, the ACS sample size will be about 3 million addresses. As with the decennial census long form, the ACS relies heavily on questionnaire responses mailed in by respondents. These estimates are collected on a rolling basis every month throughout the year, and the questionnaire asks about eight types of income received in the previous 12 months. For example, those interviewed in January 2003 were asked about income received in the January to December 2002 period, and those interviewed in December 2003 were asked about the December 2002 to November 2003 period. Thus, the calendar time period differs for ACS income estimates from those from the CPS ASEC and Census 2000, as the latter two use the previous calendar year. The ACS uses the Census 2000 self-response mail-out/mail-back methodology, followed by CATI, followed by CAPI.
The ACS uses a more up-to-date sampling frame (the Census Bureau’s Master Address File updated by using the U.S. Postal Service’s Delivery Sequence File and targeted address canvassing) than does the CPS ASEC. The CPS ASEC will use the Census 2000 sampling frame for the first time in 2005 (as updated with new construction since April 2000). Also, while the CPS ASEC reaches a greater proportion of hard-to-interview populations, such as young Black men, than do most other Census Bureau surveys, the ACS does even better at interviewing such populations than the CPS ASEC. This improved coverage might affect the estimates. Finally, the ACS thus far has excluded group quarters from its sampling frame, slightly affecting the estimates of poverty, as some people in the poverty universe are in noninstitutional group quarters, such as those in group homes and shelters. The ACS is scheduled to include both institutional and noninstitutional group quarters in its sampling frame starting in January 2006.
Data on income are also released periodically from the Survey of Income and Program Participation (SIPP), a longitudinal survey designed to track changes in income over time for the same households. Whereas the CPS ASEC is a labor force survey with supplementary questions on income, the SIPP focuses on income and typically reports more income (and therefore lower poverty) than the CPS ASEC. The SIPP consists of 9 or 12 interviews spaced 4 months apart over a 3- or 4-year period and asks a set of “core” questions about the previous 4 months by telephone and personal visit. Interviewers return to the same household (not housing unit) and attempt to follow each individual interviewed in the first series of interviews, even if they move. About 62,000 addresses were in the initial sample for the 2004 SIPP.
The SIPP is useful mainly for examining the changes in income (and poverty) for particular households and individuals over a 3- or 4-year period or for time periods shorter than a year, since it collects monthly income by source using a much more detailed questionnaire than the CPS ASEC – up to 81 sources of income and up to 73 individual income values. The National Academy of Sciences has recommended that the SIPP become the future source of official poverty statistics.
The SIPP also contains information on many other subject areas that are critical for understanding social and economic well-being. These areas include wealth, disability status, health insurance coverage, child support, pension coverage, and measures of material well-being. The richness of this survey, coupled with its collection of high-quality income data, make the SIPP a unique and extremely valuable federal survey. Its estimates have been used to understand the relationship between job loss and health insurance coverage, to understand the employment of former welfare recipients, to estimate housing affordability, to understand the economic well-being of the disabled, and in many other policy-relevant analyses.
Subnational detail is the raison d'etre of the Census 2000 long form. A sample of about 1-in-6 U.S. households received the long form in 2000. Estimates for 1999 median household and family income, poverty, and other subjects were released on a state-by-state basis during the summer of 2002. The great advantage of Census 2000 was its huge sample size. The Census 2000 sample data were compiled from a sample of approximately 19 million housing units that received the census long form questionnaire compared with roughly 100,000 housing units in the sample for the CPS ASEC and 800,000 for ACS – resulting in much lower sampling errors. The mail-out/mail-back questionnaire used by the Census 2000 long form was very similar to that used by the ACS (asking about eight income types); however, Census 2000 used only personal-visit follow-up using paper questionnaires while the ACS uses CATI, CAPI, and other methods to improve data quality. The estimates from Census 2000 are becoming more and more outdated as time passes, but that is all that is available for census tracts until ACS results are available in 2010.
Key methodological differences between the CPS ASEC and Census 2000 have an effect on the estimates. The income questions asked on CPS ASEC were much more detailed than those asked on Census 2000. The data collection methodology was substantially different, with the CPS ASEC being collected by interviewers via CATI or CAPI. Census 2000, in contrast, used a self-response mail-out/mail-back questionnaire, followed by personal visits to nonrespondents by much less experienced interviewers than those who conduct CPS ASEC or the ACS.
The Small Area Income and Poverty Estimates (SAIPE) program “borrows strength” from multiple data sources, including administrative records and multiple household surveys, to produce estimates with lower variance than estimates from any one source, but they are available a year later than the annual estimates from the CPS ASEC. The SAIPE program uses statistical methods to improve subnational estimates of income and poverty by using information from the CPS ASEC, the decennial census (and soon the ACS), population estimates, and administrative records such as aggregate food stamps and aggregate adjusted gross income from tax returns. Although used for critical purposes, such as in the funding formula that is used to distribute about $14 billion dollars a year to school districts under the Department of Education’s Title I Program, such information is provided only as a characteristics of a specific geographic area. A significant advantage of household surveys is their ability to allow analysis of how income varies along with other household and individual characteristics, such as nativity and work experience.
SUMMARY
The CPS ASEC provides the most timely and most accurate cross-section data for the nation on income and poverty. Because of its large sample size and relatively rapid processing, the ACS methodology holds the most promise of providing timely subnational data on income and poverty. The SIPP is focused on collecting accurate longitudinal income and program participation data to help understand the dynamics of a household’s economic situation. Its timeliness is not comparable since one must wait until after a 3- or 4-year panel has concluded to analyze the longitudinal data.
Yet estimates from any one survey will almost never match the estimates from any other (unless explicitly controlled), because of differences such as in questionnaires, data collection methodology, reference period, and edit procedures. The SAIPE program can reconcile the different results from these alternative surveys and reduce the standard errors of those estimates, although such estimates will always be available with some delay because of the need to acquire relevant administrative records.
1/ The CPS includes people in noninstitutional group quarters such as group homes and collegedormitories; the ACS will include both institutional (such as jails and nursing homes) and noninstitutional group quarters when fully implemented.
The chart below summarizes the recommendations at various geographic levels:
Data Source Recommendation |
Cross-Section Estimates |
Longitudinal Estimates |
|
Geographic Level |
Level |
Year-to-year change |
|
United States |
CPS ASEC |
CPS ASEC |
SIPP |
State |
ACS or CPS ASEC 3-year averages; then SAIPE (when available) |
CPS ASEC 2-year averages for now, ACS when it is fully implemented; then SAIPE (when available) |
SIPP (selected states) |
Substate |
ACS; then SAIPE (when available) for counties and school districts |
ACS; then SAIPE (when available) for counties and school districts |
none |
State-to-Nation comparison |
CPS ASEC |
CPS ASEC |
SIPP |
All other comparisons |
ACS |
ACS |
none |
CPS ASEC=Current Population Survey Annual Social and Economic Supplement
ACS=American Community Survey
SAIPE=Small Area Income and Poverty Estimates program
