Research on Survey Methodology FY 2002 Awards List

Awards from this competition were jointly reviewed and supported by NSF's Methodology, Measurement, and Statistics Program and a consortium of federal statistical agencies represented by the Federal Committee on Statistical Methodology (FCSM). The following agencies provided direct financial support for these awards:

Bureau of Justice Statistics, DoJ
Bureau of Labor Statistics, DoL
Bureau of Transportation Statistics, DoT
Economic Research Service, USDA
Energy Information Administration, DoE
Internal Revenue Service
National Agricultural Statistics Service, DoA
National Center for Education Statistics, DoE
National Center for Health Statistics, DHHS
Science Resources Statistics, NSF
Social Security Administration
U.S. Census Bureau, DoC

Identifying Causal Mechanisms Underlying Nonignorable Unit Nonresponse Through Refusals to Surveys
0207435
Robert M. Groves
Mick P. Couper
Eleanor Singer
Stanley Presser
University of Michigan

Total Award Duration: 36 months
Amount: $340,313

This project is a set of randomized experiments aimed at systematically both producing and eliminating unit nonresponse error in survey estimates. Specifically, three attributes of the survey request (topic interest, survey sponsor, and monetary incentives) will be experimentally manipulated in concert with choice of target population. Sampling frames containing persons with known characteristics (e.g., occupational groups, interest groups, groups of consumers of specific products or services) will be used. Randomly identified subsamples of these groups will be asked to participate in self-administered surveys on topics of relevance to the frame and topics irrelevant to the frame. Crossed with this factor, the sponsorship of the survey will be experimentally varied, with one sponsor relevant to the frame and one irrelevant to the frame. Finally, the use of a monetary incentive will be crossed with both of the other factors to measure the effects of extrinsic benefits of participation. The key hypothesis is that topic interest and sponsorship act to produce nonignorable nonresponse when the surveys contain items relevant to the frame, and that monetary incentives act to reduce the magnitude of nonresponse error by bringing into the respondent pool sample persons with low topic interest and minimal affect toward the sponsor. The effects of sponsor affect and topic interest are expected to be additive; monetary incentives are expected to counteract the nonignorability influences of both factors.

The practical importance of this work to statistics based on surveys is: a) to help agencies conducting surveys anticipate when different sponsors may obtain different results; b) to provide evidence about potentially harmful effects on nonresponse error of interviewers' emphasizing single purposes of a survey; c) to produce evidence regarding the ameliorating effects on nonresponse error of monetary incentives; and d) to test a conceptual structure that will help survey sponsors anticipate when nonresponse rates will affect error and when they will not. This research is supported by the Methodology, Measurement, and Statistics Program and a consortium of federal statistical agencies under the Research on Survey and Statistical Methodology Funding Opportunity.

Abstract and Additional Information

Collaborative Research: Testing for Marginal Independence Between Two or More Multiple-Response Categorical Variables
0233321
Thomas M. Loughin
Kansas State University
Total Award Duration: 12 months
Amount: $28,065

0207212
Christopher R. Bilder
Oklahoma State University
Total Award Duration: 12 months
Amount: $51,812

Questions that ask respondents to "choose all that apply" from a set of items occur frequently in surveys. Categorical variables that summarize this type of survey data are called multiple response (or pick any/c) categorical variables. It is often of interest to test for independence between two categorical variables. When categorical variables can have multiple responses, traditional Pearson chi-square tests for independence should not be used because of the within-subject dependence among responses. This research will provide methods to test for independence between two or more multiple-response categorical variables. A modified version of the Pearson statistic will perform the test, and bootstrap procedures will provide approximate sampling distributions. First and second-order corrections will allow for chi-square distribution approximations to the sampling distribution. Generalized log linear models and multivariate binomial logit-normal models will provide a model-based approach for the test of independence.

Many survey questions are asked in a multiple-response manner. Examples include: "What types of cars do you own?" and "For what criminal offenses have you been arrested?" Other questions naturally fall into a multiple-response format, but some researchers avoid asking them in this format due to statistical analysis problems. For example, most survey questions dealing with ethnicity allow respondents to make only one choice, which is entirely inappropriate in today's highly multicultural population. Other researchers may analyze multiple-response questions as if they came from single-response categorical variables, which can lead to very conservative tests of independence. This research will allow researchers to incorporate these types of questions into surveys and use statistically correct methods of analysis. The research is supported by the Methodology, Measurement, and Statistics Program and a consortium of federal statistical agencies under the Research on Survey and Statistical Methodology Funding Opportunity.

Abstract and Additional Information

Collaborative Research: Theory and Methods for Nonparametric Survey Regression Estimation
0204642
Jean D. Opsomer
Iowa State University
Total Award Duration: 24 months
Amount: $65,038

0204531
F. Jay Breidt
Colorado State University
Total Award Duration: 24 months
Amount: $71,166

This research project develops new methods for the efficient use of auxiliary information in complex surveys, based on nonparametric regression techniques. Current practice relies on parametric regression techniques, which have good efficiency if the regression model is well specified, and which have a number of appealing operational features. The nonparametric techniques share these operational features, lose little efficiency when the parametric specification is correct, and gain efficiency when the parametric specification is incorrect. The project increases the scope of applicability of the nonparametric regression estimation approach, by considering complex survey designs, varying types of auxiliary information, and alternative smoothing techniques. Specifically, the project investigates multi-stage surveys with cluster or element-level auxiliary information; multivariate auxiliary information; and alternative smoothing techniques. Parametric and nonparametric techniques are blended using semiparametric additive models to provide a flexible tool for use in complex surveys.

Large-scale surveys are used to collect data in a wide range of fields, from studies of human populations to inventories of natural resources. Information external to the survey, such as administrative records or remote sensing, is often available. This research project makes it possible to incorporate auxiliary information easily and effectively into survey estimates, by using nonparametric regression methods. Nonparametric regression, sometimes referred to as smoothing, is widely used in other areas of statistics, but its use in survey estimation has been limited so far. The investigators show that incorporating auxiliary information into survey estimation through nonparametric regression can improve the precision of the surveys, often at reduced costs. This research is supported by the Statistics and Probability Program, the Methodology, Measurement, and Statistics Program, and a consortium of federal statistical agencies under the Research on Survey and Statistical Methodology Funding Opportunity.

Abstract and Additional Information

A Comparison of RDD and Cellular Telephone Surveys
0207843
Charlotte Steeh
Georgia State University

Total Award Duration: 12 months
Amount: $176,296

Since very little research has examined the impact on surveys of the exploding growth of wireless communication devices, this study will assess the extent to which these devices are likely to change telephone surveys. This issue will be addressed by comparing the results of two national surveys, one using the usual list-assisted RDD sample and the other employing a sample of mobile telephone numbers. The questionnaire, which will be identical in both surveys, will include substantive items on important policy issues as well as inquiries about mobile telephone ownership and use. The analyses will search for significant differences between surveys along four dimensions-coverage, nonresponse, data quality, and relationships among variables. Methodological factors, such as the number of attempts, the percentage of sample numbers whose status as working or nonworking is indeterminate, and the effects of caller-id, will also be compared. The basic hypothesis underlying all analyses is that there are major differences between the two modes along each dimension.

Given that wireless communication devices will only become more widely used and more sophisticated in the very near future, it is necessary to determine how they might enrich and supplement the survey process. This research will provide initial evidence. Contact through mobile telephones promises to make hard-to-reach respondents more accessible and to give voice to groups either poorly represented or not represented at all in current surveys. By gauging the reactions of respondents to survey contacts via a cellular telephone, the project also will provide practical guidance on incorporating wireless devices into the survey process. Since methodologists now predict that future surveys will be multi-modal, mixing present and future wireless communication devices with fixed line telephones and the web, this study will describe both the opportunities and the pitfalls involved. The end result will be an expanded definition of surveys and better and more valid data upon which to base public policy. This research is supported by the Methodology, Measurement, and Statistics Program and a consortium of federal statistical agencies under the Research on Survey and Statistical Methodology Funding Opportunity.

Abstract and Additional Information