Challenging Research Issues in Statistics and Survey Methodology at the BLSTopic Statement: Modeling, Estimation and Inference Issues that Arise from the Use of Response-Rate Goals in Survey OperationsKey Words: Incomplete data; Interviewer effect; Missing data; Nonresponse; Pattern-mixture model; Quasi-randomization model; Quota sampling; Survey costs; Total survey error model; Variance estimation. Contact for further discussion: Background, Definitions and Notation: In practical work with survey data, we often encounter nonresponse, in which a selected sample unit does not provide responses to one or more items on the survey data collection instrument. There is a large literature on this topic; see, e.g., Little and Rubin (2002), Groves et al. (2002), Groves and Couper (1998), Madow, Nisselson and Olkin (1983), Madow, Olkin and Rubin (1983), Madow and Olkin (1983), and references cited therein. Much of this literature is based (implicitly or explicitly) on simple quasirandomization models, in which one defines a response indicator However, it appears that due to contractual or regulatory factors, some large survey organizations have goals or incentives tied to achievement of specified response-rate goals, and have relatively little additional incentive to achieve response rates above the specified goals. The effects of such goals can be especially important in panel surveys or other surveys in which there is a relatively brief time available for data collection. In some cases, the abovementioned goals or incentives are operational only at an overall organizational level, while in other cases similar goals or incentives apply to individual field supervisory staff (e.g., a regional or area office manager), or to individual interviewers. At an extreme, these goals or incentives can lead to forms of quota sampling, with its attendant problems with bias and lack of information for appropriate adjustment of point estimators and variance estimators. For surveys subject to response-rate goals or incentives, implementation of those goals or incentives may lead to models for nonresponse that are distinct from standard quasirandomization models (Q.1). For example, in some cases it may no longer be plausible to treat response probabilities as dependent only on the fixed predictors ; and it may no longer be plausible to treat the response indicators as independent across sample units. To illustrate with an over-simplified example, suppose that for a given target population the following conditions hold.
Concentrating nonresponse follow-up efforts in the form suggested in (ii) would lead to several issues in point estimation and inference from the resulting survey data, including the following.
In practice, nonresponse follow-up efforts are more complex than suggested by (i)-(ii). For example the true initial-response probabilities are not known. In addition, incentives can be structured to encourage efforts to collect information from units with relatively low initial response probabilities , e.g., by establishing response-rate goals separately within groups that have, respectively, high and low initial response probabilities . Also, there often will be more than one follow-up attempt for a given nonresponding sample unit. Issue: What are appropriate ways in which to account for the impact of response-rate goals or incentives in development of nonresponse-adjusted point estimators and inference methods? Questions on Nonresponse Adjustment Methods and Related Methodological Work in the Presence of Response-Rate Goals or Incentives:
To what extent can this "callback-based" estimation literature be applied or extended to the conditions described in Questions 1 or 2?
Pattern-mixture models often are considered to be of special interest for certain types of nonignorable nonresponse. To what extent, if any, do the nonresponse models from Questions 1 and 2 lead to pattern-mixture models, and related estimators, that differ substantially from those developed previously in the pattern-mixture literature? Acknowledgements: The author thanks Clyde Tucker and Polly Phipps for comments that led to development of this topic statement; and thanks John Bosley, Steve Cohen, John Dixon, Jennifer Edgar, Larry Ernst, Bill Mockovak, Stuart Scott and Michael Sverchkov for helpful comments on an earlier draft. The views expressed here are those of the author and do not necessarily represent the policies of the Bureau of Labor Statistics. References:Binder, D.A. (1983). On the variances of asymptotically normal estimators from complex surveys. International Statistical Review 51, 279-292. Drew, J.H. and Fuller, W.A. (1980), Modeling nonresponse in surveys with callbacks, Proceedings of the Section on Survey Research Methods, American Statistical Association, 639-642 Drew, J.H. and Fuller, W.A. (1981), Nonresponse in complex multiphase surveys, Proceedings of the Section on Survey Research Methods, American Statistical Association, 623-628 Groves, R.M. and Couper, M.P. (1998). Nonresponse in Household Interview Surveys. New York: Wiley. Groves, R.M., D. Dillman, J.L. Eltinge and R.J.A. Little, eds. (2002). Survey Nonresponse. New York: Wiley. Kennickell, A.B. (2000). Asymmetric information, interviewer behavior and unit nonresponse. Paper presented at the Joint Statistical Meetings, August, 2000. Little, R.J.A. (1993). Pattern-mixture models for multivariate incomplete data, Journal of the American Statistical Association, 88, 125-134 Little, R.J.A. (1994). A class of pattern-mixture models for normal incomplete data', Biometrika, 81 , 471-483. Little, R.J.A. and D.B. Rubin (2002). Statistical analysis with missing data, New York: Wiley. Madow, W. G., Nisselson, J. and Olkin, I., eds. (1983). Incomplete data in sample surveys, volume 1: Report and case studies. New York: Academic Press. Madow, W. G.;Olkin, I., and Rubin, D. B., eds. (1983). Incomplete data in sample surveys, volume 2: Theory and bibliographies. New York: Academic Press. Madow, W. G. and Olkin, I., eds. (1983). Incomplete data in sample surveys, volume 3: Proceedings of the symposium. New York: Academic Press. Merkle, D.M., S.L. Bauman and P.J. Lavrakas (1993). The impact of callbacks on survey estimates in an annual RDD survey, Proceedings of the Section on Survey Research Methods, American Statistical Association, 1070-1075 Oh, H.L. and Scheuren, F.J. (1983). Weighting adjustment for unit nonresponse. Pp. 143-184 in W.G. Madow, I. Olkin and D.B. Rubin, eds., Incomplete Data in Sample Surveys, Volume 2: Theory and Bibliography. New York: Academic Press Potthoff, R.F., K.G. Manton and M.A. Woodbury (1993), Correcting for nonavailability bias in surveys by weighting based on number of callbacks, Journal of the American Statistical Association, 88, 1197-1207 Rao, J.N.K. and Shao, J. (1992), Jackknife variance estimation with survey data under hot deck imputation, Biometrika, 79, 811-822 Rosen, B. (1972a), Asymptotic theory for successive sampling with varying probabilities without replacement, I, The Annals of Mathematical Statistics, 43, 373-397 Last Modified Date: January 06, 2006
Last Modified Date: July 19, 2008 |
Tools |
Calculators |
Help |
Info |