Small Area Income & Poverty Estimates

General Cautions About Comparisons of Estimates

Cautions about comparisons of direct survey estimates obtained from different surveys
Differences between direct survey estimates obtained from different sources (Census 2000, CPS ASEC, ACS) reflect both changes in the true levels of income and poverty (when comparing estimates for different places or different years), as well as differences in the methods by which the survey data were collected and the estimates were made. The following papers describe these differences between the data sources.

• Census 2000 and CPS
  Comparing Employment, Income, and Poverty: Census 2000 and the Current Population Survey [PDF 4.4M]
• Census and ACS
  Report 5: Comparing Economic Characteristics With Census 2000. Meeting 21st Century Demographic Data Needs-Implementing the American Community Survey [PDF 4.4M]
  Income in the American Community Survey: Comparisons to Census 2000 [PDF 374k]
• CPS and ACS
  Evaluation of Poverty Estimates: A Comparison of the American Community Survey and the Current Population Survey [PDF 74k]
  What Do We Know About Differences Between CPS and ACS Income and Poverty Estimates? [PDF 713k]
  Evaluation of Median Income and Earnings Estimates: A Comparison of the American Community Survey and the Current Population Survey [Word 175k]

Cautions about comparisons between SAIPE intercensal model-based county estimates and direct survey estimates
SAIPE model-based estimates of poverty and median household income use Census 2000 and ACS direct survey estimates in their construction. Consequently, there is an almost certain positive correlation between the Census 2000 estimates, ACS direct survey estimates, and their intercensal model-based estimates, which should be represented in confidence intervals for their difference. Failure to do so will result in too many differences being considered "not significant". We currently do not have estimates of the individual correlations or advice on a general magnitude that can be assumed.

This caution does not apply to state-level comparisons between Census 2000 because the Census 2000 estimates have close to negligible sampling error.

Cautions about comparisons between SAIPE model-based estimates for different years
Statistical comparisons of SAIPE state and county estimates across years are possible, however certain cross-year correlations must be taken into account. For all years of estimates, there is correlation of the model error, and for the years prior to 2005 there is correlation of the sampling error as well. Methodology for Testing for a Rise in Child Poverty Rate [PDF 141k] describes a methodology for taking into account these cross-year correlations in estimating if any states have a five percent or greater significant change in child poverty rate between two years. A similar methodology for counties is undergoing current research, as well as the modifications required for comparing year-to-year changes starting in 2005.

Of note, there is a break in the SAIPE time-series between 2004 and 2005 due to the switch from CPS ASEC to ACS data in SAIPE modeling. Comparisons across these particular years are not advised because of this break in series. See Estimation Procedure Changes for the 2005 Estimates for more details. Also, with the introduction of group quarters populations to the ACS starting with the 2006 ACS, comparability for certain age groups across 2005 and 2006 is limited. Generally residents of group quarters have higher poverty rates than residents of households, and this affects the comparison. See http://www.census.gov/acs/www/UseData/Comparison_Guidance2006.htm - povpeople for more details.

Cautions about comparisons between SAIPE model-based estimates for different states or different counties in the same year
All SAIPE model-based estimates are correlated because they depend on the same regression coefficients. Therefore, to make comparisons between states or between counties, it is not sufficient to take the variances (implied by the confidence intervals) for the two different places and apply the usual means difference hypothesis test. For counties, there is the additional consideration that the sampling and model errors are probably correlated across counties within a state.  Methods for making such state-to-state or county-to-county statistical comparisons are currently being researched.