Statistical Standards Program
Table of Contents Introduction 1. Development of Concepts and Methods 2. Planning and Design of Surveys 2-1 Design of Surveys 2-2 Survey Response Rate Parameters 2-3 Developing RFPs for Surveys 2-4 Pretesting Survey Systems 2-5 Maintaining Data Series Over Time 2-6 Educational Testing 3. Collection of Data 4. Processing and Editing of Data 5. Analysis of Data / Production of Estimates or Projections 6. Establishment of Review Procedures 7. Dissemination of Data Glossary Appendix A Appendix B Appendix C Appendix D Publication information For help viewing PDF files, please click here |
PLANNING AND DESIGN OF SURVEYS |
SUBJECT: PRETESTING SURVEY SYSTEMS NCES STANDARD: 2-4 PURPOSE: To ensure that all components of a survey system will function as intended when implemented in the full-scale survey. KEY TERMS: edit, estimation, field test, frame, imputation, instrument, pretest, response rates, stage of data collection, survey system, and variance.
GUIDELINE 2-4-2A: Elements to be tested and measured may include alternative approaches to accomplishing a particular task. Elements to be tested may include: frame development; sample selection; questionnaire design; data collection; response rates; data processing (e.g., entry, editing, imputation); estimation (e.g., weighting, variance computation); file creation; and tabulations. GUIDELINE 2-4-2B: For an ongoing survey, new elements or content should be field tested, along with elements being changed as a result of the evaluation of the survey (see Standard 4-3). GUIDELINE 2-4-2C: The evaluation criteria for a successful field test should be developed before the field test begins. Key evaluation criteria are established during the design stage. If the criteria are not met, that survey component should not be implemented without field testing a redesigned component. GUIDELINE 2-4-2D: The results of a field test should be available and analyzed for internal use prior to making a decision to implement the full-scale survey. GUIDELINE 2-4-2E: Survey design and instrumentation should be revised to reflect modifications suggested by the results of the field test. A revised budget should be developed, if necessary, to reflect both changes in design and knowledge gained during the field test about resource requirements. |