 |
 |
 |
 |
 |
<< Go Back |
| Note 6: NAEP, NELS, and HS&B Transcript Studies |
|
At least two methods exist to classify the academic challenge or difficulty of the coursework that high school graduates complete. One method is to measure the highest level of coursework completed in different subjects (e.g., whether a graduate's most academically challenging mathematics course was algebra I, trigonometry, or calculus). The other method is to measure the number of courses completed in different subjects (e.g., whether a graduate completed two, three, or four courses in mathematics). Based on these two methods, analysts have created different taxonomies to categorize the academic challenge or difficulty of the completed coursework in graduates' high school transcripts. This supplemental note describes two of these taxonomies, which are used in the analyses of individual indicators in The Condition of Education. Indicators 24 and 25 use an "academic pipeline" to classify course-taking data according to the highest level of coursework completed. These data come from transcripts of graduates of public high schools, which were collected as part of the U.S. Department of Education's National Assessment of Educational Progress (NAEP), National Education Longitudinal Study of 1988 (NELS), and the High School & Beyond (HS&B) study. (It is important to note that although steps were taken to replicate the data collection and coding methodology in each study, some minor differences did occur. These differences may affect the comparability of data from different data sets.) Indicator 23 uses a taxonomy of "academic rigor" to classify course-taking data, partly according to the number of courses completed. The same data sources are used for these indicators along with information about students' participation in Advanced Placement (AP) courses and tests. ACADEMIC PIPELINES Academic "pipelines" organize transcript data in English, science, mathematics, and foreign language into levels based on the normal progression and difficulty of courses within these subject areas. Each level includes courses either of similar academic challenge and difficulty or at the same stage in the progression of learning in that subject area. In the mathematics pipeline, for example, algebra I is placed at a level lower in the pipeline hierarchy than is algebra II because algebra I is traditionally completed before (and is generally less academically difficult or complex) than algebra II. Classifying transcript data into these levels allows one to infer that high school graduates who have completed courses at the higher levels of a pipeline have completed more advanced coursework than graduates whose courses fall at the lower levels of the pipeline. Tallying the percentage of graduates who completed courses at each level permits comparisons of the percentage of high school graduates in a given year who reach each of the levels, as well as among different graduating classes. The high school courses taken by students are coded according to the academic levels of the pipeline by matching course titles on the student's transcripts with course catalogs from the student's high school describing the contents of those courses. The courses are then coded according to the Classification of Secondary School Courses (CSSC) and the coded courses are assigned to broader course groupings, forming the academic levels of the pipeline in each subject area, using the Secondary School Taxonomy (SST). Steps are taken to replicate the data collection and coding methodology across the transcript studies to assure comparability. Some minor differences may affect the comparability of data from different transcript collections to some extent. Transcript studies are a reliable source of information but they do have limitations. One limitation is that transcript studies can describe the intended—but not the actual—curriculum. The content and instructional methods of one course taught in one school by a certain teacher may be different from the content and instructional methods of another course that is classified as having the same CSSC code but is taught by a different teacher. Nevertheless, validation studies and academic research have shown significant differences between the highest level of academic courses completed by students and their scores on tests of academic achievement (Chaney, Burgdorf, and Atash 1997; Berends, Lucas, and Briggs, forthcoming). In classifying students' courses from their transcripts according to a pipeline, only the courses completed in a subject area are included and not courses attempted. The pipeline also does not provide information on how many courses graduates completed in a particular subject area. Graduates are placed at a particular level in the pipeline based on the level of their highest completed course, regardless of whether they completed courses that would fall lower in the pipeline. Thus, graduates who completed year 3 of (or 11th-grade) French did not necessarily complete the first 2 years. English Pipeline English language and literature courses do not fit neatly into an ordered hierarchical framework. Instead of building on previously studied content, the English curriculum is stratified by the level of academic challenge and intensity of work required within a specific content area rather than among different courses. For example, within the general English curriculum, most schools have three tracks that vary by level of academic challenge: below-grade level or low academic level courses, at-grade or regular courses, and above-grade or honors courses. Thus, unlike the mathematics and science pipelines that are based on progress within a content continuum (e.g., algebra I, geometry, algebra II, trigonometry, and calculus), the English pipeline is constructed to reflect the proportion of coursework completed by graduates in each track. It reflects the quality of a graduate's English coursetaking rather than the progression from low-level to more challenging coursework. The English pipeline has seven categories: no English coursework; 50 percent or more low academic level courses; some, but less than 50 percent low academic level courses; regular, no low or honors courses; some, but less than 50 percent honors courses; 50 percent or more, but less than 75 percent honors courses; and 75 percent or more honors courses. No English No courses classified as English ever completed by graduate. It is possible for a graduate to have taken one or more unclassified English courses and be placed in the "no English" level. For the most part, these unclassified courses were English coursework for blind and deaf students or English as a Second Language courses. Low Academic Level The low academic level is divided into two sublevels, the second of which is considered to be more academically challenging than the first.
Regular All completed English courses classified at grade level; no low academic level or honors courses. Advanced Academic Level The advanced academic level is divided into three sublevels.
Foreign Language Pipeline Coursework in a foreign language follows an ordered, sequential path. Most high school students who study a foreign language progress along such a path, which is typically a sequence of four year-long courses in the language. Not all students do this, however. Some students begin their studies in the middle of a sequence because they have prior knowledge of the language. Some repeat the same year of study, and a few (about 7 percent of 1988 graduates) study more than one language (NCES 2003–343). The highest level of completed coursework in the foreign language pipeline thus may not indicate the total number of years a graduate has studied a foreign language or languages. The foreign language pipeline also does not classify all foreign language study: only courses in French, German, Latin, and Spanish are counted because these are the most commonly offered foreign languages. The next four most commonly offered foreign languages (Italian, Japanese, Hebrew, and Russian) each accounted for less than 1 percent of 1988 graduates who studied foreign languages in the unweighted NELS:88 sample that was used to create the pipeline (NCES 2003–343). Adding these four languages to the four most common languages in the pipeline made less than 0.1 percent difference in the percentage of graduates who studied a single language, though it made more difference (yet less than 1 percent difference) in the percentage of graduates who never studied a language and who studied more than one language. In 1998, the total percentage of students who studied one of these next four most commonly offered languages was 4.5 percent. The foreign language pipeline has six categories: none; year 1 (1 year of 9th-grade instruction) or less; year 2 (1 year of 10th-grade instruction); year 3 (1 year of 11th-grade instruction); year 4 (1 year of 12th-grade instruction); AP instruction. None No courses classified as foreign language study ever completed by graduate. Only courses in the four most common languages (French, German, Latin, and Spanish) are counted as foreign language study, so it is possible for a graduate to have taken one or more courses of some other foreign language and be placed in this category. Low Academic Level Year 1 (1 year of 9th-grade instruction) or less Graduate completed no more than either a full Carnegie unit (1 academic year of coursework) of 9th-grade (year 1) foreign language instruction or half a Carnegie unit of 10th-grade (year 2) foreign language instruction. Year 2 (1 year of 10th-grade instruction) Graduate completed either a full Carnegie unit (1 academic year of coursework) of 10th-grade (year 2) foreign language instruction or completed half a Carnegie unit of 11th-grade (year 3) foreign language instruction. Advanced Academic Level Year 3 (1 year of 11th-grade instruction) Graduate completed either a full Carnegie unit (1 academic year of coursework) of 11th-grade (year 3) foreign language instruction or completed half a Carnegie unit of 12th-grade (year 4) foreign language instruction. Year 4 (1 year of 12th-grade instruction) Graduate completed either a full Carnegie unit (1 academic year of coursework) of 12th-grade (year 1) foreign language instruction or completed half a Carnegie unit of 13th-grade (year 5) foreign language instruction. AP instruction Graduate completed an AP foreign language course. ACADEMIC RIGOR To measure the "academic rigor" of coursework, a taxonomy of four levels of academic rigor has been constructed, using the following criteria:
When information on honors/AP coursetaking is missing, AP test-taking is used as supplementary data. It is assumed that, if AP records indicated that students had taken an AP test, students had taken a honors/AP course. Classifying transcript data into these four levels allows one to conclude that high school graduates who meet the criteria of more "rigorous" levels have completed more academically challenging and difficult coursework than graduates who meet only the criteria of less "rigorous" levels. The primary differences between a taxonomy based on academic pipelines and one based on academic rigor is that the latter classifies students who have completed a set number of "rigorous" courses, whereas the former indicates only the highest level of coursework completed, not the number of courses completed. For indicator 23, the following three levels are used.
|
 |
|
|
