Statistical Analysis Report:

An Analysis of Total Nonresponse in the 1993-94 Schools and Staffing Survey (SASS)

September 1997

(NCES 98-243) Ordering information

Introduction

The 1993-94 Schools and Staffing Survey (SASS) is the third study of public and private elementary and secondary schools in a series of surveys begun in 1987-88 by the (NCES)./1 Survey data from schools, local education agencies (LEAs), administrators, and teachers in the United States were collected by mail with telephone follow-up of nonrespondents first during the 1987-88 school year and again during the 1990-91 and the 1993-94 school years. The series provides data on school and teacher characteristics, school operations, programs and policies, teacher demand and supply, and the opinions and attitudes of teachers and school administrators about policies and working conditions. The analytic power of the data is enhanced by the ability to link survey data for individual LEAs, schools, administrators, and teachers. In 1993-94 new library, librarian, and student SASS components were initiated that could also be linked. In addition, computer assisted telephone interviewing (CATI) facilities were introduced for the first time during the 1993-94 SASS and were used extensively for nonresponse follow-up.

This study is part of a systematic effort by NCES to evaluate the quality of SASS and as such this report is designed to enable users to understand the limitations of the 1993-94 SASS data and to provide managers information for planning future rounds of SASS. A comprehensive analysis of response rates for the 1990-91 SASS is found in a companion report by Scheuren, Monaco, Zhang, Ikosi, Chang, and Gruber (1996). A number of other reports have also been issued as part of the SASS quality review, notably Jabine (1994).

Results from SASS are affected by two sources of error: sampling error and nonsampling error. Sampling errors are the result of basing survey estimates on a sample rather than all units in the population of interest and are published for selected estimates in all reports based on SASS data./2 In addition, generalized variance functions which provide approximations of sampling errors for all SASS estimates are provided./3 The other source of error is called nonsampling error, and includes all errors that are not due to sampling.

This report is concerned with the most pervasive and challenging source of nonsampling error in estimates from sample surveys which is the error associated with incomplete data. Incomplete data resulting from three sources are of particular importance in sample surveys: item nonresponse, unit nonresponse, and undercoverage./4 Item nonresponse in SASS can arise when a response is missing for an item (e.g., the number of students enrolled in grade 1 at a school around October 1, 1993) in an otherwise completed interview. Unit nonresponse can arise in SASS when a response is not obtained for a sampled unit (e.g., school, local education agency -- LEA, teacher, administrator, library, librarian, student). The concern for nonresponse, whether item or unit, is twofold. Nonresponse reduces the sample size and thus increases the sampling variance. Respondents may also differ significantly from nonrespondents, thus, the estimate obtained from respondents can be biased and the magnitude of this bias may be unknown. Concerns about bias are generally greater as the rate of nonresponse increases. Undercoverage in SASS can arise when units that should be in the frames (e.g., lists of public and private schools in the U.S.) from which a sample is selected are not in those frames, or units in the sample are mistakenly classified as ineligible or are omitted from the sample or from the units interviewed.

The particular focus of this report is to quantify the extent of unit nonresponse in the 1993-94 SASS and to assess the impact of differences in the known characteristics of respondents and nonrespondents for different subgroups of the survey populations in order to provide some indication of the potential effects of nonresponse bias and to suggest priorities for future SASS research. While the scope of the report is chiefly descriptive, inferential modeling of the response rates for one component is also provided as an example for future SASS research.

---

FOOTNOTES:

[1] The U.S. Bureau of the Census carries out the main survey operations for SASS -- including sample selection, data collection, and data processing -- under an interagency agreement, according to specifications provided by NCES.

[2] Each SASS publication includes separate tables with sampling errors for selected estimates included in the publication.

[3] For the 1987-88 SASS generalized variance functions (GVFs) see Salvucci and Holt (1992), Generalized Variance Estimates for SASS. Also for 1987-88 SASS GVFs see Salvucci, Galfond and Kaufman (1993), “Generalized Variance Functions for the Schools and Staffing Surveys,” Proceedings of the Section on Survey Research Methods, American Statistical Association. For the 1990-91 SASS generalized variance functions see Salvucci and Weng (1995), Design Effects and Generalized Variance Functions for the 1990-91 Schools and Staffing Survey (SASS), NCES 95-340-I and Salvucci, Holt, and Moonesinghe (1995), Design Effects and Generalized Variance Functions for the 1990-91 Schools and Staffing Survey (SASS), NCES 95-340-II.

[4] Madow, Nisselson, and Olkin. (1983). Incomplete Data in Sample Surveys, Vol. 1, Report and Case Studies.

Download/view the full report in a PDF file.(1,261K)

Help with PDF files

For more information about the content of this report, contact Kerry Gruber at Kerry.Gruber@ed.gov.

---