Statistical Standards Program
Table of Contents Introduction 1. Development of Concepts and Methods 2. Planning and Design of Surveys 3. Collection of Data 3-1 Coverage for Frames and Samples 3-2 Achieving Acceptable Response Rates 3-3 Monitoring and Documenting Survey Contracts 3-4 Documenting a Survey System 4. Processing and Editing of Data 5. Analysis of Data / Production of Estimates or Projections 6. Establishment of Review Procedures 7. Dissemination of Data Glossary Appendix A Appendix B Appendix C Appendix D Publication information For help viewing PDF files, please click here |
COLLECTION OF DATA |
SUBJECT: COVERAGE FOR NCES FRAMES AND SAMPLES NCES STANDARD: 3-1 PURPOSE: To ensure that necessary steps are taken to develop and maintain data collections that are used as sampling frames, and that coverage of sampling frames is evaluated and documented. KEY TERMS: capture/recapture, confidentiality, coverage, coverage error, dual-frame estimation, estimation, frame, frame population, freshening, half-open interval, multiplicity estimation, noncoverage, overcoverage, supplemental area frame, survey system, target population, undercoverage, and variance.
GUIDELINE 3-1-1A: Frames can be retrospectively compared against alternative frames found inside and outside of the Department of Education; considering total list count comparisons, matching operations, and dual-frame estimation procedures using capture/recapture procedures to estimate noncoverage; and providing an estimation of missing units. GUIDELINE 3-1-1B: Staff responsible for NCES data collections that are used as sampling frames should maintain two-way communications with survey staff who use their collection as a frame. Procedures such as sharing preliminary data files with survey staff in order to develop frames may be instituted. (For example, staff that use an administrative list of public schools for their frames should be alerted when new data are available and each time there is a major change in the list.)
GUIDELINE 3-1-3A: The first time a survey is conducted, background design and coverage work should be done before choosing the frame. Alternative frames, if applicable, should be considered and compared. GUIDELINE 3-1-3B: Coverage errors
such as over- and under-coverage, bad contact information, classification,
temporal errors, and other listing errors should be minimized before
the use of a frame. Techniques such
as list supplements, multiciplicity estimation, half-open
intervals, and un-duplication
can be used to reduce these errors and improve coverage
of the frame. GUIDELINE 3-1-3D: To reduce coverage error, whenever a frame has important deficiencies with respect to the measurement unit, dual-frame estimation should be considered to correct these deficiencies. Since dual-frame estimation can be expensive, the effect dual-frame estimation has on increasing the variance estimates should also be considered when deciding to use dual-frame estimation.
GUIDELINE 3-1-4A: Sample survey documentation should include a discussion of coverage issues such as alternative frames that were considered, what was done to improve the coverage of the frame, and how data quality and item non-response on the frame may have affected the coverage of the frame. GUIDELINE 3-1-4B: Survey documentation should include any estimation techniques used to improve the coverage of estimates. This would include post-stratification procedures. (For example, a telephone survey could post-stratify estimates of all individuals to account for the exclusion of those without telephones.) GUIDELINE 3-1-4C: NCES survey staff should archive their survey's sampling frames as part of the documentation of the survey system found in Standard 3-4, taking security precautions consistent with confidentiality laws into account. This archiving may be particularly important if a preliminary file was used to develop the frame, or if there is a chance that the frame may be used in the future to further develop research questions.
|