American Community Survey (ACS)

Income in the ACS: Comparisons to the 1990 Census

Kirby G. Posey and Edward Welniak

U.S. Census Bureau
Washington, DC 20233

Prepared for the American Community Survey (ACS) Symposium held at the Bureau of the Census, Suitland, Maryland, March 1998.
Last revised: June 17, 1999.

This paper reports the results of research and analysis undertaken by Census Bureau staff. It has undergone a more limited review than official Census Bureau publications. This report is released to inform interested parties of research and to encourage discussion.

Adjusting 1996 ACS Income Data and Comparing Estimates to the 1990 Census

The long-term goal of the American Community Survey (ACS) is to produce more timely local area estimates. Currently, the only sources of Census Bureau local area income estimates are the decennial census (data only once every ten years) and the Small Area Income and Poverty Estimates (SAIPE) program (biennial county and state estimates of median household income). We designed the battery of income questions as well as the edit and allocation specifications for the ACS to be as similar as possible to that of the Census in order to evaluate the ACS's performance. There are, of course, some differences inherent with the ACS that forced us to make some changes. The most notable change between the Census and the ACS is the reference period, "past 12 months" versus "last calendar year." Other differences include a slight change in the questionnaire and some changes in the edit and allocation process. The purpose of this paper is to explain these differences and provide a preliminary overview of the ACS results.

In the 1990 Census, a battery of eight detailed income questions and a total income question were posed to all persons 15 years old and over. The eight detailed income questions included (1) wages and salary, (2) non-farm self-employment income, (3) farm self-employment income, (4) interest, dividend, and net royalty income, (5) social security, (6) Supplemental Security Income (SSI), Aid to Families with Dependent Children (AFDC), or other public assistance income, (7) retirement, survivor, or disability income, and (8) any other sources of income received on a regular basis. These eight income questions were placed at the end of each set of person questions on the long form (sample) questionnaire so that response levels to the other less sensitive sample questions would not be adversely impacted. A "total income" question was first asked in the 1980 Census to aid in the resolution of income entry problems in the detailed questions. Test results indicated that a sizable number of all income recipients furnished responses to the total income question, but failed to provide answers to some or all parts of the detailed type of income questions, proving the total income item very important.

We used the same battery of income questions including the total income item in the ACS with a few exceptions (see Table 1.) First of all, we combined the non-farm self- employment income item with the farm self-employment income item. We did this mainly because the number of respondents reporting farm self-employment income was small. This should not hinder analysts from identifying farm income recipients because we can still determine this number from other labor force related questions such as "occupation of longest job." We used this combined self-employment format in the 1998 Dress Rehearsal and we are planning to use it in the 2000 census as well.

Again, the biggest difference between collection methods in the ACS and the Census is the income reference period. The ACS collects data throughout the year on an on-going, monthly basis. The ACS asks for a respondent's income over the "past 12 months." The census, however, collected the income data for a fixed period of time -- "during 1989" (the last calendar year.) To evaluate the impact of this reference period change, we conducted a "split panel" test of about 19,000 households (with a 49 percent mail response rate) from October through December of 1997. We collected income statistics using both terminologies, "past 12 months" and "income in calendar year 1996" for the two different randomly assigned treatment groups. The only statistical differences in median income estimates between the two reference periods occurred in the earnings categories, wages / salary and self-employment (see Table 2).

We expected the 12-month reference period to yield slightly higher estimates because of the more recent reference period. As shown in Table 2, that is not always the case. For example, the median wage / salary income reported on forms asking about the PAST 12 MONTHS was $25,000 for individuals. The median wage / salary income reported on forms asking about the PAST CALENDAR YEAR is $26,000. For self-employment income, however, the "past 12 months" reference period yielded the higher figure. All other income items did not differ significantly between the two reference periods.

Since the American Community Survey interviews respondents monthly and asks for income received during the "past twelve months," we end up with twelve different reference periods for our income data (see Figure 1). That is, if we asked a respondent in February 1996 how much income they received during the past twelve months, their reference period was February 1995 through January 1996. If, on the other hand, we interviewed a respondent in October 1996 and asked about income during the past twelve months, their reference period would then be October 1995 through September 1996. When we released the data in the ACS profiles and the Summary Files on the Internet, we used the terminology, "1996 Adjusted Household Income" or "1996 Adjusted Family Income," etc. That is, we presented the income data for a fixed reference period. To accomplish this, we applied adjustment factors to the data depending on the month of interview to adjust everyone's income to constant 1996 dollars. We used the Consumer Price Index (CPI) to do this. (See Figure 2).

Respondents who were interviewed in February 1996 and asked about his or her income from February 1995 through January 1996 had their income adjusted by a factor of 1.027009. On the other hand, respondents who were interviewed in October and whose income reference period was from October 1995 through September 1996 had their incomes adjusted by a factor of 1.007871. These factors were computed as the sum of the 1996 CPI monthly adjustment factors divided by the sum of the CPI monthly adjustment factors for the reference period. The result is that all ACS respondents in a given year end up with incomes in terms of consistent 1996 dollars. Note that on the Public Use files, for disclosure avoidance purposes, there is only one adjustment factor for all respondents. This is to keep from disclosing the month of interview to help protect the individual confidentiality of our respondents. That factor will be the average of the 12 monthly factors.

Also note here that the ACS is not designed to produce income statistics for use on a monthly basis. We collect these data over a 12-month period then aggregate it to an annual amount.

Edits and Allocations

For the most part, we used the same editing scheme for the 1996 ACS as we did for the 1990 Census. For example, in the consistency edits, we checked for things like obvious monthly amounts reported for income sources like social security, public assistance or retirement income. We also checked for common mistakes respondents sometimes make with decimal points. For example, we checked for someone who might have reported $100,000 in public assistance income where they should have reported $1,000. We checked for respondents who might have intended to report a "loss" in self-employment income or in interest, dividends, or net rental income but forgot to mark the "loss" box based on their total reported income. We checked identically reported amounts in the wage/salary question and the total income question. Many times, respondents will confuse the first income item for the total income field and report their total income twice. We used the total income field to resolve differences in the reporting of the components when we could.

We also designed the allocation and imputation specifications for missing data for the 1996 ACS basically the same as those used in the 1990 Census. We sorted the file first by sex. That is, we stored usable data (hot deck values) separately for males and females so that missing economic characteristics are allocated to a male only from another male; never from a female and vice versa. Then, for each sex, we further stored usable data according to race and ethnicity, educational attainment, living arrangements (that is, husband or wife, other family reference person, other family members and unrelated individuals), and finally by residence (or whether a person did or did not live in a metropolitan area.) For example, if a respondent failed to report earnings data but did provide answers to other items such as occupation, class of worker, weeks worked last week, and age, we matched their reported data to that of another respondent who fully reported all items. When we located a match, we simply substituted the fully reported earnings data for the missing information.

Because the ACS data are collected on a monthly basis, at the time of imputation we will have access to a much smaller sample than we did during the census (about 3 million housing units for a full year of data for the ACS compared to around 17 million during the Census). Therefore, we found it necessary to borrow some matching schemes from the Current Population Survey processing system that does not rely on the nearest geographic neighbor as a donor. That is, we used different "keys" when attempting to match to an information donor.

We started the process by looking for donors in fairly detailed matrices hoping to match as closely as possible to another person's characteristics. If we fail to locate a match, we dropped back to a more general and less detailed matrix or a second "key" in order to find a match.

Sometimes respondents will indicate they received a particular type of income but will not report how much they received. Another difference between the ACS and the Census is that the ACS treats respondents who report income recipiency with no amount differently than those respondents who left an income question completely blank. That is, we allocate missing amounts from different matrices depending on whether the respondent marked the "yes" box for that item. In other words, we first try to establish a pattern of recipiency and then go from there.

Test Site Results: Median and mean household income estimates for all four 1996 ACS sites were lower than the 1990 Census figures after they were adjusted for inflation. In all four areas the differences were statistically significant. (See Table 3 for the median household income figures).

We looked into several possible explanations for this. For example, the counties could have failed to keep up with inflation. The lower median household incomes could be because of the changing demographic makeup of the counties during the past seven years. The difference could also be the result of other census / ACS differences including survey design and mode of interviewing.

Another possible explanation for the lower figures for the "PAST 12 MONTHS" reference period is that a respondent read the instructions and "keyed in" on the word "Month" and reported "monthly" amounts in the income fields. The income edits are designed to catch monthly amounts and would have caught many of these types of mistakes but some of the higher monthly amounts indicated by respondents could have remained unchanged thus lowering the median and mean income. As a precautionary measure and to help prevent this situation for future respondents, we are changing the instructions slightly on the questionnaire to emphasize annual amounts. The instructions will be changed to read "Mark (X) the "Yes" box for each type of income this person received, and give your best estimate of the TOTAL AMOUNT during the PAST 12 MONTHS. (NOTE: The "past 12 months" is the period from today's date one year ago up through today.)"

Still another possible explanation for the lower medians could be the fact that we are applying national (and sometimes regional) CPI factors to adjust local area estimates from the 1990 Census and trying to compare the result to our ACS estimates. These factors may not entirely reflect what is actually happening in the local areas. Also, a recession occurred between 1989 (Census) and 1996 (ACS); not all localities fully recovered.

It should also be noted that we published "true medians" for household and family incomes from the ACS. The medians from the 1990 Census were calculated from grouped data using linear interpolation. We compared the "true" 1996 ACS median household income estimates to (1) adjusted 1990 Census figures, (2) county estimates of household income from the Census Bureau's Small Area Income and Poverty Estimates (SAIPE) program adjusted to 1996 dollars, and (3) ACS medians calculated from grouped data using linear interpolation. As shown in Table 4, in most cases, the 1996 ACS "true" median household incomes matched up much more closely with the more recent SAIPE estimates than the older adjusted census figures.

When looking at response rates (the percentage of respondents who answered "yes," "no," or "loss") by the different sources of income, the questionnaire with the "past 12 months" reference period produced slightly higher response rates for every income source although only one income item, public assistance, was statistically significant over the two reference periods. (See Table 5.)

Finally, because response rates were slightly better and to assure that the income questions would be as consistent as possible with the other, more current information collected on the ACS (disability and labor force status, for example), we decided to keep the 12-month reference period. But, as mentioned earlier, we are changing the wording of the instructions slightly to emphasize the fact that we are looking for "Total amounts" for the previous 12 months and not simply a monthly amount.

It is also worth mentioning that when making comparisons between the 1990 Census income figures and those of the ACS, it is also important to keep in mind the different modes of data collection. That is, when gathering information from households who failed to respond via mail, the ACS used computer-assisted telephone interviewing (CATI) and computer-assisted personal interviewing (CAPI) where the 1990 Census did not.