Technical Notes

The data in this report come from many sources, including surveys conducted by the National Science Foundation (NSF), other federal agencies, and nonfederal organizations. Many methods of data collection are represented. Therefore, users should take great care when comparing data from different sources. These data often will not be strictly comparable because of differences in definitions, survey procedures, and phrasing of questions, among other things.

Survey accuracy is determined by the joint effects of sampling and nonsampling errors.[1] Sampling errors arise when estimates based on a sample differ from figures that would have been obtained if a complete population had been surveyed. Nonsampling errors can arise from design, reporting, and processing errors as well as from errors due to faulty responses or nonresponses. Nonsampling errors include respondent-based events, such as some respondents interpreting questions differently from other respondents; respondents making estimates rather than giving actual data; and respondents being unable or unwilling to provide complete, correct information. Errors can also arise during the processing of responses, such as during recording and keying.

Reporting Categories

This report draws on data collected from many sources, many of which use differing terms to describe the various statistics presented here. Efforts have been made to maintain consistency throughout these tables, but in some tables, it has been necessary to use distinct terminology that does not match that used in other tables.

Racial/Ethnic Information

The collection and reporting of race/ethnicity data pose several additional problems. First, both the naming of population subgroups and their definitions have changed over time. Second, many of the groups of particular interest are quite small, so it is difficult to measure them accurately without surveys of the entire population of interest. In some instances, sample surveys may not have had sufficient sample size to permit the calculation of reliable racial/ethnic population estimates for all groups; consequently, data are not shown for some groups. The U.S. Bureau of the Census's Technical Notes Current Population Survey, for example, cannot provide data on American Indians. Data on this population are available only from the decennial census. Third, data on race/ethnicity are often based on self-identification. These data are less reliable for certain racial/ethnic groups than for others. Data collected at two points in time indicate that self-identification of American Indians is much less reliable than self-identification of other racial/ethnic groups.[2] Fourth, it is easy to overlook or minimize heterogeneity within subgroups when only a single statistic is reported for a total racial/ ethnic group.

Information About People with Disabilities

Data on people with disabilities who study or work in science and engineering (S&E) are seriously limited for several reasons. First, the operational definitions of disability vary, can include a wide range of physical and mental conditions, and may not be comparable. The Americans With Disabilities Act of 1990 (ADA) encouraged progress toward standard definitions. Under ADA, an individual is considered to have a disability if he or she has a physical or mental impairment that substantially limits one or more of his or her major life activities, has a record of such impairment, or is regarded as having such an impairment. ADA also contains definitions of specific disabilities.

Second, data on disabilities frequently are not included in comprehensive institutional records (e.g., in registrars' records in institutions of higher education). If included at all, such information may be kept only in confidential files at an office responsible for providing special services to students. Institutions of higher education are unlikely to have information regarding students with disabilities who have not requested special services related to their disability from the institution. In elementary/secondary school programs receiving funds to provide special education, however, statistics on all students identified as having special needs are centrally available.

Third, information about people with disabilities that is gathered from surveys is often obtained from self-reported responses. Typically, respondents are asked whether they have a disability and to specify what kind of disability it is. Resulting data therefore reflect individual perceptions rather than objective measures.

The attempt to provide estimates of the proportion of the undergraduate student population with disabilities is an example of how these factors coalesce. Self-reported data on the undergraduate student population, collected through a survey to ascertain patterns of student financial aid, suggest that about 10 percent of this population have some disability. Estimates from population surveys of higher education institutions, in contrast, place the estimate much lower, between 1 and 5 percent. Whether this discrepancy is the result of self-perception, incomplete reporting, nonevident disabilities, or differing definitions is difficult to ascertain.

In the final analysis, although considerable information is available about individuals with disabilities in the education system and in the S&E workforce, it is often impossible to compare statistics from different sources.

Several sources of data on people with disabilities are cited here. They include the National Postsecondary Student Aid Study (NPSAS) conducted by the Department of Education's National Center for Education Statistics (NCES), NSF's Survey of Earned Doctorates (SED), and the integrated Scientists and Engineers Data System (SESTAT). These sources are described in more detail later in this appendix; the following is a brief description of how each treats the issue of disability.

• NPSAS asked students in 2000 whether they have any disabilities "such as hearing, speech, or mobility impairment, or vision problems that can't be corrected with glasses". Response categories are: legally blind or have a vision impairment that cannot be corrected with glasses; hearing impairment; speech disability or limitation; orthopedic or mobility limitation; specific learning disability; and other health-related disability or limitation.
• SED asks if the respondent has a disability, then asks the respondent to mark which categories describe the disability(ies). Response categories include: blind/visually impaired; deaf/hard of hearing; physical/ orthopedic disability; learning/cognitive disability; vocal/speech disability; other disability.
• The SESTAT surveys ask the degree of difficulty—none, slight, moderate, severe, or unable to do—an individual has in seeing (with glasses), hearing (with hearing aid), walking without assistance, or lifting. Those respondents who answered "moderate," "severe," or "unable to do" for any activity were classified as disabled.

Primary Data Sources

Data from several sources are presented here. This section provides summary descriptions of major sources and information about the location of more detailed survey descriptions.

Primary NSF Sources

The following sources from NSF's Division of Science Resources Statistics (SRS) were used for data tables in this publication. Published data tables from these surveys can be accessed on the SRS website at http://www.nsf.gov/statistics. In addition, researchers may access data directly from the Scientists and Engineers Statistics Data System (SESTAT) or the WebCASPAR database system, which also can be accessed from the SRS website.

Survey of Earned Doctorates

SED has been conducted annually since 1957 for NSF, the U.S. Department of Education, the National Endowment for the Humanities, the National Institutes of Health, the National Aeronautics and Space Administration, and the U.S. Department of Agriculture. This is a survey of all recipients of research doctoral degrees from U.S. institutions. SED data are restricted to research doctorates such as the doctor of philosophy (Ph.D.) or doctor of science (D.Sc.); it excludes the recipients of first professional degrees such as the juris doctor (J.D.) or doctor of medicine (M.D.).

Data for SED are collected directly from individual doctorate recipients contacted through graduate deans at all U.S. universities awarding research doctorates. The recipients are asked to provide information about the field and specialty of their degree as well as their personal educational history, selected demographic data, and information about their postgraduate work and study plans. Over time, more than 90 percent of the annual cohort of doctorate recipients respond to the questionnaire.

Partial data from public sources, such as field of study, are added to the file for nonrespondents. No imputations are made, however, for nonresponse for data not available elsewhere, such as race/ethnicity information. The data for a given year include all doctorates awarded in the 12-month period ending on June 30 of that year. Information about SED can be found at http://www.nsf.gov/statistics/srvydoctorates/.

Survey of Graduate Students and Postdoctorates in Science and Engineering

The data collected in the Survey of Graduate Students and Postdoctorates in Science and Engineering (GSS) represent national estimates of graduate enrollment and postdoctoral employment at the beginning of the academic year in all academic institutions in the United States that offer doctoral or master's degree programs in any S&E or health field. Included are data for all branch campuses; affiliated research centers; and separately organized components such as medical or dental schools, schools of nursing, and schools of public health. The survey population consists of approximately 600 graduate institutions. Data are collected at the academic department level.

Approximately 99 percent of institutions and departments respond to the survey. Missing data for partially nonrespondent departments are imputed using the departments' previous year's data, where available, or data from peer institutions in cases where data had not been reported the previous year. Data for completely nonrespondent departments are imputed using data from the previous year, where available.

Available information includes counts of full-time graduate students by source and mechanism of support, including data on women and first-year students enrolled full time, part-time graduate students by sex, and citizenship and racial/ ethnic background of all graduate students. In addition, data on postdoctorates are available by source of support, sex, and citizenship, including separate data on those holding first professional doctorates in the health fields; summary information about other doctoral nonfaculty research personnel is also included.

NSF has collected data on graduate S&E enrollment and postdoctoral appointees since 1966. From fall 1966 through fall 1971, data from a limited number of doctorate-granting institutions were collected through the NSF Graduate Traineeship Program, which requested data only on those S&E fields supported by NSF. During the period 1972–75, this data collection effort was gradually expanded to include additional S&E fields as well as all institutions known to have programs leading to the master's or doctoral degree. Because of this expansion, data for 1974 and earlier years are not strictly comparable with 1975 and later data.

In 1999, the survey presented respondents with new race/ethnicity categories:

• The "Asian/Pacific Islander" category used in previous years' surveys became two categories: "Asian" and "Native Hawaiian/Pacific Islander."
• The categories "more than one race Hispanic/Latino" and "more than one race non-Hispanic/ Latino" were added.
• The "other" category included in previous years' surveys was removed.

These changes are not reflected in the tables in this report. Currently, only about 8 percent of the departments report data in any of the new categories. For the tables in this report, as in other publications using these data, the data reported in the new categories are combined into previous survey categories. The data are combined for the tables as follows: the "Asian" category and the "Native Hawaiian/Pacific Islander" category form the "Asian/Pacific Islander" category; the "one race only Hispanic/Latino" category and the "more than one race Hispanic/Latino" category form the "Hispanic" category; and the "more than one race non-Hispanic" category and the "unknown" category form the "other or unknown" category.

Information about GSS can be found at http://www.nsf.gov/statistics/srvygradpostdoc/.

SESTAT Data System

The SESTAT data system integrates data from three SRS surveys—the Survey of Doctorate Recipients, the National Survey of College Graduates, and the National Survey of Recent College Graduates. The integration of the SESTAT surveys requires complementary sample populations and reference periods, matching survey questions, procedures, and field definitions, as well as weighting adjustments for any overlapping populations.

The surveys provide data on educational background, occupation, employment, and demographic characteristics. These surveys are of individuals and currently have a combined sample size of about 102,300, representing a population of about 22 million scientists and engineers, including people trained in S&E or S&E-related fields or working in S&E or S&E-related occupations. Each of the three surveys that makes up the SESTAT data system collects new data every 2 years. The data reported in this publication were collected in 2003.

SESTAT defines scientists and engineers as residents of the United States with a baccalaureate degree or higher who, as of the study's reference period, were not institutionalized, were age 75 or less, and were either educated as or working as a scientist or engineer. A baccalaureate or higher degree is a bachelor's, master's, doctorate, or professional degree. To meet the scientist or engineer requirement, the U.S. resident had to (1) have at least one baccalaureate or higher degree in an S&E or S&E-related field or (2) have a baccalaureate or higher degree in a non-S&E field but worked in an S&E or S&E-related occupation as of October 1, 2003. Some elements of SESTAT's desired target population were not included within the target populations of any of the three SESTAT component surveys. Bachelor's and master's level S&E or S&E-related personnel missing from the survey frames are predominately:

• U.S. residents as of the survey reference date whose bachelor's and/or master's degrees in S&E or S&E-related fields were received prior to April 2000 or from a foreign institution, who resided outside the United States on April 1, 2000, but not with the U.S. armed forces stationed abroad; or
• U.S. residents as of the survey reference date with no baccalaureate or higher degree in any field as of April 1, 2000, who were awarded a degree in S&E or an S&E-related field after June 2002 by a U.S. institution or after April 2000 by a foreign institution.

Doctorate level S&E or S&E-related personnel missing from the survey frames are predominately:

• U.S. residents as of the survey reference date with doctorates in S&E or health fields received after June 2002 or from a foreign institution, with no baccalaureate or higher degree in any field as of April 1, 2000, and no bachelor's or master's degree in S&E or S&E-related fields received from a U.S. institution between April 1, 2000, and June 2002; or
• U.S. residents as of the survey reference date with doctorates in S&E or health fields received after June 2002 or from a foreign institution but with no bachelor's or master's S&E or S&E-related degree received from a U.S. institution between April 1, 2000, and June 2002, who resided outside the United States on April 1, 2000, but not with the U.S. armed forces stationed abroad.

SESTAT classifies the following broad categories as S&E occupations: computer and mathematical scientists, life and related scientists, physical and related scientists, social and related scientists, and engineers. Postsecondary teachers are included within each of these groups. The following are considered S&E-related occupations: health and related occupations, S&E managers, S&E precollege teachers, S&E technicians and technologists, including computer programmers, and other S&E-related occupations, such as architects and actuaries. Non-S&E occupations include non-S&E managers; non-S&E teachers; non-S&E technicians/technologists; social services and related occupations; sales and marketing occupations; art, humanities, and related occupations; and other non-S&E occupations—for example, clerical and administrative support personnel, farmers, foresters, fishermen, lawyers, judges, librarians, archivists, curators, food service personnel, construction tradespeople, mechanics and repairers, and those involved in precision/production occupations, operators (for example, machine set-up, machine operators and tenders, fabricators, assemblers) and related occupations, transportation/material moving occupations and protective and other service occupations. Information on SESTAT can be found on the Web at http://sestat.nsf.gov/.

Primary Non-NSF Sources

The following non-NSF sources were used for data tables in this report.

The Integrated Postsecondary Education Data System Survey: Fall Enrollment, Completions, and Institutional Characteristics

National Center for Education Statistics
U.S. Department of Education
1990 K Street, NW
Washington, DC 20006
(202) 502-7300
http://nces.ed.gov/ipeds

The Integrated Postsecondary Education Data System (IPEDS) Survey began in 1986 as a supplement to and replacement for the Higher Education General Information Survey (HEGIS), which began in 1966. HEGIS annually surveyed institutions listed in the current NCES Education Directory of Colleges and Universities; IPEDS surveys all postsecondary institutions, including universities and colleges and the institutions that offer technical and vocational education. IPEDS consists of several integrated component surveys that obtain information about types of institutions where postsecondary education is available, student participants, fall enrollments, programs offered and completed, graduation rates, and the human and financial resources involved in the delivery of postsecondary education. Descriptions of these surveys follow.

The IPEDS Institutional Characteristics Survey provides the basis for the list of institutions reported in the Education Directory of Colleges and Universities. The list includes institutions that meet specific accreditation criteria and offer at least a 1-year program of college-level studies leading to a degree. Each fall, institutions listed in the previous year's directory are asked to update information about their school's characteristics.

The IPEDS Completions Survey replaces and extends the HEGIS Degrees and Other Formal Awards Conferred Survey. It is administered to all institutions offering degrees at the bachelor's degree level and above, 2-year institutions, and less-than-2-year institutions.

The IPEDS Fall Enrollment Survey replaces and extends the previous HEGIS surveys of enrollment in institutions of higher education.

The National Postsecondary Student Aid Study

National Center for Education Statistics
U.S. Department of Education
1990 K Street, NW
Washington, DC 20006
(202) 502-7300
http://nces.ed.gov/npsas

NPSAS was established by NCES to collect information about financial aid allocated to students enrolled in U.S. postsecondary institutions. NPSAS was first administered in the fall of the 1986-87 academic year. NCES conducted subsequent cycles of NPSAS during the 1989–90, 1992–93, 1995–96, and 1999–2000 academic years.

The 1999–2000 survey gathered information from about 62,000 undergraduate and graduate students selected from registrars' lists of enrollees at more than 900 postsecondary institutions. The sample included students who did and did not receive financial aid, as well as students' parents. Student information, such as field of study, educational level, and attendance status (part time or full time), was obtained from registrars' records. Types and amounts of financial aid and family financial characteristics were abstracted from school financial aid records. Data pertaining to family circumstances, background demographic data, educational and work experiences, and expectations were collected from students using a computer-assisted telephone interview. The response rate for obtaining institutional record data for all students was 97.0 percent, and the weighted overall student interview response rate was 65.6 percent.

Survey of Engineering and Technology Enrollments and Survey of Engineering and Technology Degrees

Dan Bateson
Engineering Workforce Commission
American Association of Engineering Societies
1111 19th Street, NW Suite 403
Washington, DC 20036
(202) 546-2237
http://www.ewc-online.org/

The Engineering Workforce Commission (EWC) annually conducts surveys of engineering enrollments and degrees conferred in almost 600 institutions, including all of those with curricula approved by the Accreditation Board for Engineering and Technology (ABET). EWC counts the number of students studying for engineering degrees at all ABET-accredited engineering schools throughout the United States and also some schools that are not ABET- accredited for a variety of reasons unique to each school. Some schools are in the process of obtaining ABET accreditation; others have simply asked to be included in the survey. The enrollments and degrees surveys obtain responses from about 91 to 93 percent of the schools. Each year, EWC obtains data from all schools included in the previous year's survey to ensure accurate time-series comparisons.

Sampling Errors

Sampling errors occur when estimates are derived from a sample rather than the entire population. The sample used for any particular survey is only one of a large number of possible samples of the same size and design that could have been selected. Even if all other aspects of the survey remained fixed, such as the questionnaire and instructions, the estimates from each sample would differ from other samples. This difference, termed sampling error, occurs by chance, and its variability is measured by the standard error associated with a particular estimate.

The standard error of a sample survey estimate measures the precision with which an estimate from one sample approximates the true population value, and it can be used to construct a confidence interval for a survey parameter to assess the accuracy of the estimate. See http://sestat.nsf.gov/docs/method.html for information about calculation of standard errors for data from SESTAT, http://www.bls.gov/cps/home.htm for Current Population Survey design and methodology, and http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2006180 for information on standard errors for NPSAS data.