Overview 
Survey Design
Survey Quality Measures
Trend Data
Availability of Data
1. Overview (2003 survey cycle)
a. Purpose
The National Survey of College Graduates (NSCG) is a longitudinal survey, designed to provide data on the number and characteristics of experienced individuals with education and/or employment in science and engineering (S&E), or S&E-related fields in the United States. The 2003 NSCG provides a once in a decade opportunity to study the education and career paths of the nation's college-educated individuals and various characteristics of college-educated individuals in the workforce such as salaries, whether the college-educated population was working in their highest degree field of study, specific occupations, and a gender breakdown of the workforce.
The results of this survey are vital for educational planners within the federal government and in academia. Employers in all sectors (education, industry, and government) use the survey results to understand trends in employment opportunities and salaries in various degree fields and to evaluate the effectiveness of equal opportunity efforts. This survey is designed to complement the other surveys of scientists and engineers conducted by SRS in order to provide a comprehensive picture of the number and characteristics of individuals with education and/or employment in S&E or S&E-related fields in the United States. This combined system is known as SESTAT (Scientists and Engineers
Statistical Data System).
Data from the 2003 NSCG are used to update the 1993 NSCG findings.
b. Respondents
2003 survey respondents were individuals living in the U.S. during the reference week of October 1st, 2003, holding a bachelor's or higher degree in any field, and under age 76.
c. Key variables
- Academic employment (positions, rank and tenure)
- Age
- Citizenship status
- Country of birth
- Country of citizenship
- Disability status
- Educational history (for each degree held: field, level, when received)
- Employment status (unemployed, employed full time, or employed part time)
- Geographic place of employment
- Immigrant module (year of entry, type of entry visa, reason(s) for coming to U.S., etc.)
- Labor force status
- Marital status
- Number of children
- Occupation (current or past job)
- Primary work activity (e.g., teaching, basic research, etc.)
- Publication and patent activities
- Race/ethnicity
- Salary
- Satisfaction and importance of various aspects of job
- School enrollment status
- Sector of employment (academia, industry, government)
- Sex
- Work-related training
2. Survey Design
a. Target population and sample frame
The target population of the 2003 survey consisted of all individuals:
- Under age 76 as of the survey reference date (i.e. born after September 30, 1927)
- Who received a bachelor's degree or higher prior to April 1, 2000
- Who were living in a housing unit or noninstitutionalized group quarters as of April 1, 2000, and
- Who resided in the 50 states, the District of Columbia, Puerto Rico, or the other outlying U.S. territories as of April 2000 and the survey reference week of October 1, 2003.
The 2003 NSCG serves as the baseline survey for future survey cycles in the current decade, much as the 1993 NSCG did. The 2003 survey included a sample of respondents to the 2000 Decennial Census long form who indicated they had a baccalaureate degree or higher in any field of study. However, those holding a Ph.D. earned in the U.S. in an S&E field will not be followed in the future NSCG survey cycles as these individuals are covered in a companion SESTAT survey—the Survey of Doctorate Recipients (SDR).
2003 NSCG also included two different samples of "old cohort" panel cases from earlier surveys. Cases in the "old cohort" sampling frame were either drawn from the 1999 NSCG respondents who originated from the 1993 NSCG (based on the 1990 Decennial Census long form) or panel respondents from the 1993-2001 National Survey of Recent College Graduates (NSRCG). A sample of respondents to the 1999 NSCG (1993 NSCG cases and 1993-1999 NSRCG cases) were included in the 2003 NSCG to allow methodological research to compare the quality and reliability of estimates based on continuation of the cases from the 1990s with those based on a new frame from the 2000
census. A sample of respondents from the 2001 NSRCG were included in the 2003 NSCG to cover new S&E degree recipients between April 15, 2000 and June 30, 2000 to provide complete coverage of the S&E degree holders in the 2003 SESTAT integrated database. The 2003 NSCG file does not include any of the "old cohort" cases and these cases are omitted from the discussions below [1].
b. Sample design
The sample frame for the 2003 survey was drawn from the 2000 Decennial Census long form responses to determine initial eligibility status and stratification cells for the sampling frame. Cases were eligible if they were living in a housing unit or non-institutionalized group quarters; had received a bachelor's degree or higher; resided in the 50 states, the District of Columbia, Puerto Rico, or the other outlying U.S. territories; all as of April 2000; and were age 75 or less as of the 2003 survey reference date of October 1st (i.e., born after September 30, 1927).
The goals of the 2003 NSCG sample design were:
- Develop a design that was comparable to the sample design used in the other two SESTAT surveys;
- Address the sampling inefficiencies that existed in the 1990s NSCG design based on the 1990 Census long form responses to the extent possible;
- Use caution when developing sampling rates in an effort to reduce the sampling rate range across all strata; and
- Allocate the sample to provide complete coverage of science and engineering (S&E) and non-S&E degree holders.
The NSCG sample was stratified by four sampling variables:
- Demographic Group (8 levels) - a composite variable that captured disability status, ethnicity, race, citizenship status at birth, and likelihood of a U.S.-earned degree.
- Highest Degree Type (3 levels) - Bachelor's, Master's, Professional degree, or Doctoral degree. A small number of professional degree cases were grouped with the bachelor's degree cases for stratification purposes.
- Occupation (30 levels) - the other two SESTAT surveys (NSRCG and SDR) categorize the sample degree into one of the five specific S&E fields (Life Sciences, Mathematics/Computer Sciences, Physical Sciences, Social Sciences, and Engineering). However, because the 2000 Decennial Census long form did not collect degree field information, Decennial Census reported occupation was used as a proxy variable for degree field.
- Sex (2 levels) - male and female.
The initial sample size of the 2003 NSCG was set at 201,220. After the cases with blank names and addresses were removed, 197,834 cases remained in the sample. The final sample size was reduced to 177,320 for interviewing due to cost reasons. After data collection, analysis showed that a disproportionate number of cases requiring personal follow-up who had been imputed as college degree holders on the 2000 Census data actually did not have at least a bachelor's degree. To address concerns about coverage bias related to the imputed degree cases being identified as ineligible (i.e., no bachelor's degree), all cases with imputed educational attainment data on the Census were
removed from the sample to increase the probability that all the necessary sampling criteria were satisfied. There were 6,523 of these "imputed degree" cases in the sample resulting in a final 2003 survey sample size of 170,797.
For the old cohort, the initial sample size was set at 40,073.
c. Data collection techniques
The Bureau of the Census conducted the NSCG for NSF. Initial data collection was done through the use of a self-administered mail survey using a prenotification letter, a first mailing, a reminder letter, and a second mailing. If the sample person did not respond to any of the various mailings, automated reminder phone calls were made using a software package called Phone Tree.
Nonrespondents to the mail questionnaire were followed up using computer-assisted telephone interviewing. If the paper questionnaire was not returned and telephone follow-up attempts failed, for certain selected cases, a personal visit (CAPI) was conducted.
Information in the 2003 survey was collected for the week of October 1, 2003. Data collection took place between October 2003 and August 2004.
d. Estimation techniques
The base weight for each case was the inverse of the probability of sample selection. The imputed degree adjustment distributed the weights of the 6,523 imputed degree cases that were removed from the sample across their respective sampling strata. The CAPI adjustment redistributes the weight of final nonrespondents that were not included in the personal interviewing data collection phase across the cases that were included.
The unit nonresponse adjustment inflated the weights of interviewed persons to account for sample persons for whom no interview was obtained. This noninterview adjustment assumed that the noninterview cases contained the same percentage of ineligible cases as the respondent cases. This adjustment was performed at the noninterview cell level. The noninterview cell was formed by collapsing small sampling strata and breaking out large sampling strata by age. Through the adjustment, the weight of the nonrespondents was applied to all complete interview and ineligible cases. After the noninterview adjustment, a raking procedure was performed to adjust the sample weights so they would agree with sampling frame estimates for certain key sampling variables.
3. Survey Quality Measures
a. Sampling variability
The sample size is sufficiently large that estimates based on the total sample are subject to minimal sampling error. However, sampling error is larger with estimates of small subgroups of the population. NSCG variance estimates were calculated using the successive difference replication method. Due to the large amount of data collected in the NSCG, it is not practical to make direct calculations of a variance estimate for every survey estimate.
Generalized variance functions were developed to model the variance estimates for certain characteristics. Generalized variance parameters were then calculated through the use of a generalized variance function. Generalized variance parameters can be used to calculate standard errors for various types of characteristics. Information needed to estimate the standard errors for the NSCG and the SESTAT system as a whole are accessible through the SESTAT web site,
http://sestat.nsf.gov.
b. Coverage
Coverage errors exist because some individuals were not included in the Decennial Census. Additional under-coverage errors may exist because of errors in the Decennial data file that led to incorrectly classifying individuals as not having bachelor's degrees or above when in fact they held such a degree. Any over-coverage was minimized from the NSCG through weighting.
c. Nonresponse
(1) Unit nonresponse - The unweighted response rate for the 2003 survey was 63 percent; the weighted response rate was 73 percent. Weighted response rate for the 1993 survey was 80 percent, and for the subsequent rounds of the NSCG ranged from 91 to 95 percent. Due to the lower than expected response rate in 2003, several measures were taken to evaluate and address potential nonresponse bias in the 2003 data. A detailed nonresponse analysis on various sampling and respondent characteristics identified significant differential response rates by age of sample members, where younger age groups were much more likely to be nonrespondents to the survey than older groups.
Results from the nonresponse research and analysis were used extensively in the nonresponse weighting adjustments. Careful selection of factors for constructing the weighting classes should have reduced possible nonresponse bias in the 2003 data. Weights were also adjusted to control distributions for some variables to known totals from the sample.
(2) Item nonresponse - In 2003, the item nonresponse rate for key items (employment status, type of employment, occupation, and primary work activity) ranged from 0.0 percent to 0.6 percent. Other variables, especially those involving sensitive information, had higher nonresponse rates. For example, salary and earned income had item nonresponse rates of approximately 10 to 12 percent. A Hot Deck imputation method was used to compensate for the item nonresponse.
d. Measurement
The NSCG is a survey of individuals and thus subject to reporting errors from differences in interpretation of questions. It is also true for any multimodal survey (mail, CATI, personal visit) that some measurement errors will differ by modality. To reduce measurement errors the NSCG questionnaires were pretested and focus groups held, and a set of survey concurrent evaluation research was done throughout the 1990s and for the 2003 survey. Some of the results from this research can be found on the SESTAT web site in the Research Compendium section, and in the technical notes section for the individual variables.
4. Trend Data
In 2003 the NSCG covered all those holding a college degree as of April 2000. In 1993 the NSCG covered all those holding a college degree as of April 1990. A decade comparisons can be made between the 1993 and 2003 NSCG data as many of the core questions remained the same. However, there are small but notable differences such as the occupation and education data were collected based on the 2000 revised Standard Occupational Classification (SOC) and Curriculum of Instructional Program (CIP). Also due to the reference date change in 2003 to October instead of April used in 1993-2001 surveys, some seasonal differences may be reflected when the 2003 data are compared to earlier
years.
The 1995-1999 NSCG data sets are not available separately and analyzing trend data for the 1993 and subsequent year NSCG surveys cannot be done. A trend data can be obtained from the SESTAT integrated file.
5. Availability of Data
a. Publications
The data from this survey are available in an InfoBrief and Special Reports. NSCG data are also combined with the NSRCG and SDR surveys in the SESTAT integrated database.
Information from this survey is also included in Science and Engineering Indicators and Women, Minorities, and Persons With Disabilities in Science and Engineering. Information from this survey is also published in a set of SESTAT detailed statistical tables.
b. Electronic access
Data from this survey are available on the SRS Web site and on the SESTAT Web site. Public use data files are available upon request.
c. Contact for more information
Additional information about this survey can be obtained by contacting:
Kelly Kang
Senior Analyst
Human Resources Statistics Program
Division of Science Resources Statistics
National Science Foundation
4201 Wilson Boulevard, Suite 965
Arlington, VA 22230
Phone: (703) 292-7796
E-mail: kkang@nsf.gov
Footnotes
[1] For more information on the "old cohort" cases, see 2003 National Survey of College Graduates Old Cohort Survey Methodology Report (2005), U.S. Department of Commerce, Bureau of the Census.
