Small Area Income & Poverty Estimates

Estimation Procedure Changes for the 2005 Estimates

The following items represent changes in the estimation procedures used for state, county, and school district income and poverty estimates for 2005 from the estimation procedures used for 2004. These changes reduce the comparability of the estimates between years and should be considered when making such comparisons. See General Cautions about Comparisons of Estimates.

A major overall change from 2004 and previous years is the switch to using data from the American Community Survey (ACS) as the basis for the SAIPE state and county models, replacing the data from the Current Population Survey (CPS) Annual Social and Economic Supplement (ASEC) that was used previously. This change was made for essentially two reasons. In 2006 the Census Bureau changed the basis of its official direct state poverty estimates from CPS ASEC data to ACS data. Since SAIPE focuses on estimates at state and lower levels of geography, changing to ACS as the basis for SAIPE is consistent with this change made for the official direct survey estimates. Additionally, the much larger sample size in the ACS (about 3,000,000 addresses nationally) than in the CPS ASEC (about 100,000 addresses nationally) conveys some significant advantages for small area estimation. In general, the larger ACS sample sizes lead to substantially lower variances of the direct survey estimates and to mostly lower variances for the resulting model-based estimates.

In addition to the reduced sampling error in the ACS based estimates, definitional and data collection differences in the two surveys produce differences in direct income and poverty estimates. The 2006 reports Evaluation of Poverty Estimates: A Comparison of the American Community Survey and the Current Population Survey [PDF 74k] and What Do We Know About Differences Between CPS and ACS Income and Poverty Estimates? [PDF 713k] discuss some of the methodological differences and make some comparisons between ACS and CPS ASEC income and poverty estimates.

We can identify certain differences between the surveys that may produce differences in the income and poverty estimates. The following considerations are worth noting.

• The CPS ASEC collects survey responses mostly in March of each year, asking questions about income in the prior calendar year. In contrast, the ACS collects survey responses every month asking questions about income in the prior twelve months. The annual ACS estimates combine results from the corresponding twelve monthly surveys (starting in January and ending in December), and thus use income reports that cover a total of twenty-three months of reference, from January of the prior year through November of the current year. Implications of this timing difference for the timing of the regression variables in the state and county models are discussed in the sections below.
• The CPS ASEC collects data primarily through telephone interviews, while the ACS collects data successively in three stages of mailed paper questionnaires, telephone follow-up to mail nonrespondents, and personal visit follow-up with a subsample of the remaining nonrespondents. In addition, the CPS ASEC asks over fifty questions about income, asking about many particular income sources, while the ACS combines the sources of income into fewer (eight) income questions.
• The CPS ASEC covers the non-institutionalized population of the United States, including residents of non-institutional group quarters (GQ), while the 2005 ACS covered only the household population (people living in housing units). Also, the CPS ASEC counts all people in a housing unit who consider the unit as their usual residence or who have no residence elsewhere, while the ACS counts all people in a housing unit living or staying in the unit for more than two months.
• The CPS ASEC gathers information about the relationships of people in sample housing units that allows determination of "unrelated subfamilies" -- people related to each other but not to the householder. The ACS generally considers such people as separate individuals, which affects both the poverty threshold that applies and whether or not the person is in the poverty universe.

As an example of the implications of the last two points, consider the college dormitory population. The CPS ASEC includes college dormitory residents in the poverty universe and generally counts them at their parents' home addresses, but the 2005 ACS did not include college dormitory residents in the survey. Thus, for example, a family of four with one child living in a college dormitory would have its poverty threshold for ACS 2005 computed for a family with three members, but for CPS ASEC the computation would be based on a family with four members.

Some particular implications of the differences between the ACS and the CPS ASEC as sources of data for the SAIPE state and county models are discussed in the sections below. Further details are given in a 2007 SAIPE report Use of ACS Data to Produce SAIPE Model-Based Estimates of Poverty for Counties [PDF 3.4M].

State Model Changes
The forms of the state poverty ratio and median income models for 2005 remained the same as for the 2004 estimates. The switch to use of ACS data, however, had some implications for how the state models were implemented.

• In the previous state models using the CPS ASEC data, the timing of the regression predictor variables was chosen to correspond, in most cases, to the income year to which the CPS ASEC estimates referred. (The lone exception was the food stamp participation rate, which was based on a 12-month accumulation of monthly data, and for which some prior experimentation had revealed that shifting it some months later than the CPS ASEC data (6 months was used) tended to produce slightly better model fits.) Since the 2005 ACS collected income responses covering overlapping 12-month periods from January of 2004 through November of 2005, the ACS 2005 income and poverty estimates do not refer clearly to a single income year. In fact, the regression predictor variables defined for either 2004 or 2005 have about equal correspondence to the ACS 2005 estimates. Both were tried in the models and the resulting model-based estimates were very similar. The predictor variables for 2004 were chosen for this year's model. This means that they are the same as the predictor variables used last year when modeling the CPS ASEC estimates for income year 2004.
• An important part of the modeling process in previous years involved fitting a "sampling error model" to smooth out direct survey estimates of CPS ASEC sampling error variances using a generalized variance function. This was important with the CPS ASEC data because, with the limited state sample sizes of the CPS, the direct variance estimates were relatively unstable, especially for the smaller states and smaller age groups (e.g., 0-4 years). With the larger state sample sizes of ACS, however, we felt that such smoothing of the direct ACS variance estimates was not necessary, and possibly not desirable. For one thing, such smoothing might have substantially altered the direct variance estimates for some of the large states (e.g., CA, TX, NY, FL, etc.) With the very large ACS samples used in the large states, this seemed undesirable.
• With the large ACS state sample sizes yielding relatively precise direct income and poverty estimates for large states, the Bayesian estimation techniques tend to give great weight to the direct ACS estimates, making the Bayesian estimates (and their variances) close to the direct survey estimates and their variances. In other words, with the ACS data the modeling does not change the estimates and variances for large states very much. The benefits of the state modeling are concentrated in the smallest states with the smallest ACS samples and highest sampling error variances. For these states the model-based estimates can differ substantially from the direct ACS estimates (though this is not always the case), and worthwhile variance reductions are achieved (subject, as always, to the assumption that the model form used is at least approximately valid.)

County Model Changes
The timing of the regressor variables corresponds with the discussion of the regressor variables in the state model: Since the 2005 ACS collected income responses covering overlapping 12-month periods from January of 2004 through November of 2005, the ACS 2005 income and poverty estimates do not refer clearly to a single income year. In fact, the regression predictor variables defined for either 2004 or 2005 have about equal correspondence to the ACS 2005 estimates. Both were tried in the models and the resulting model-based estimates were very similar. The predictor variables for 2004 were chosen for this year's model. This means that they are the same as the predictor variables used last year when modeling the CPS ASEC estimates for income year 2004.

Using the CPS ASEC, sampling and model error variance had to be calculated using indirect methods. Sampling error variance was derived by assuming the variance is proportional to the square root of the sample size. Model error variance was estimated from a census auxiliary equation. Using ACS, data changed the way these variances are calculated. The sampling error variance was directly estimated by using replicate weights. The model error variance was estimated using the ACS data, not an auxiliary equation.

School District Estimation Procedure Changes
The official SAIPE method for producing updated school district child poverty estimates relied on a synthetic, or shares, approach.

Prior to the 2005 estimates, the most recent census results were used to estimate the proportions (shares) of the numbers of school-age children in poverty in each county for each school district that is wholly or partially contained in that county. These shares remained constant through the decade. The shares were used to produce school district piece poverty estimates for a given year by multiplying the SAIPE model-based county estimates of the numbers of school-age children in poverty for that year. Other than districts that experienced a boundary change, this approach provided updated information on poverty only at the county level. For intercensal years, there was no updated information on the distribution of poverty within counties, since the school district to county poverty shares remained constant.
For the current 2005 estimates, recent tabulations of IRS income tax data for school districts provide updated information on the distribution of poverty across school districts within counties. Tabulations of poor exemptions (exemptions on returns with adjusted gross income below the poverty threshold) provide a variable related to poverty, while tabulations of total exemptions (all exemptions on tax returns assigned to a given geographic area) provide a variable related to population,. From these tabulations, we produce a tax-based share of school district to county poverty, then employ an algorithm that minimizes the difference between the tax-based shares and the corresponding census-based shares. For more details see Overview of School District Estimates 2005.