Statistical Standards Program
Table of Contents Introduction 1. Development of Concepts and Methods 2. Planning and Design of Surveys 3. Collection of Data 3-1 Coverage for Frames and Samples 3-2 Achieving Acceptable Response Rates 3-3 Monitoring and Documenting Survey Contracts 3-4 Documenting a Survey System 4. Processing and Editing of Data 5. Analysis of Data / Production of Estimates or Projections 6. Establishment of Review Procedures 7. Dissemination of Data Glossary Appendix A Appendix B Appendix C Appendix D Publication information For help viewing PDF files, please click here |
COLLECTION OF DATA |
SUBJECT: MONITORING AND DOCUMENTING SURVEY CONTRACTS NCES STANDARD: 3-3 PURPOSE: To assist NCES staff in monitoring and documenting survey contract activities. KEY TERMS: edit, estimation, imputation, response rate, stage of data collection, survey system, and variance.
In some instances, the contractor may request technical redirection for unanticipated problems. For simple matters that are clearly within the scope of the contract, such requests may be made verbally. For problems that may require a change in scope, all requests must be in writing and outline the issue(s) and potential options. The COR must use this information in discussions with other NCES senior management in determining the appropriate course of action. All changes in any contract scope of work require action by the Contracting Officer. Whatever course of action is taken, it must be documented and placed in the project files.
GUIDELINE 3-3-1A: The COR should maintain close communication with the contractor. Depending on the nature of the survey, the COR should maintain communication through the use of meetings, phone calls, e-mails, visits, and/or the electronic management information system (MIS) for the purpose of tracking and monitoring the progress of the survey. GUIDELINE 3-3-1B: The COR should review and verify progress reports, vouchers, technical products and documentation, written correspondence, and other documents for the following purposes: GUIDELINE 3-3-1C: The status of each unit of observation should be kept current and available to the COR at each stage of the data collection process. Critical status events may include, but are not limited to, dates of questionnaire mail out, returns, deletions (out-of-scopes), scan editing, data entry, machine editing, callback(s), and addition to the final data files. The COR should request direct and rapid access to the information. GUIDELINE 3-3-1D: To help decide whether any adjustments or corrective actions are needed, soon after initial startup of field operations, and less frequently thereafter, the COR should evaluate the quality of survey operations by comparing a sample of the original returned questionnaires with the information on the data file for the following purposes: GUIDELINE 3-3-1E: On an as-needed basis, CORs may request a copy of "completed" records from the current master file (sometimes referred to as a "pull") and analyze the information for conformance to contract requirements. The extent of the statistical analysis of a pulled database should vary with survey objectives. Simple cross-tabulations and frequencies of discrete variables should normally point out internal coding inconsistencies and also provide interim item response rates. Simple descriptive statistics for continuous variables should provide interim item response rates, measures of dispersion, and outliers. GUIDELINE 3-3-1F: The COR should ensure that software used for weighting, imputations, and variance estimation is accurate. This may be done through a series of "checkpoints" imbedded within the program(s). Another alternative is to have the contractor provide printouts from a series of discrete steps with review by the COR along the way. GUIDELINE 3-3-1G: The COR should keep the CO and NCES management informed of the result of reviews. As an integral part of this work, the COR should offer recommendations for solving any problems, acceptance of deliverables, performance awards, and approval or disapproval of any proposed changes.
GUIDELINE 3-3-2A: It is advisable to include in the contract file all correspondence, such as logs of phone conversations, e-mail and written correspondence, and documentation, describing the approval of or decisions made regarding changes. GUIDELINE 3-3-2B: The COR should keep accurate and complete records of contractor performance, such as lateness, unacceptable deliverables, and cost overrun. Actions or decisions taken by the COR or CO to remedy the problems should also be clearly documented.
GUIDELINE 3-3-3A: The programmer should insert "comments" within the program(s) to describe each discrete section of code. Relationships between programs and data files should be flowcharted or described in a separate document. This includes record layouts and file structures.
|