Key Findings: Canada, France, Germany, Italy, Japan, Russian Federation, United States
About one-quarter of 15-year-old students in the United States scored at or below the lowest proficiency level on the PISA 2003 combined mathematics literacy scale, a higher proportion of students than in Germany, France, Japan, and Canada.
The Program for International Student Assessment (PISA) is a system of international assessments that measures 15-year-old students’ capabilities in reading literacy, mathematics literacy, and science literacy every 3 years. In 2003, PISA was conducted in 41 countries, including 30 Organization for Economic Cooperation and Development (OECD) countries and 11 non-OECD countries. PISA 2003 included an in-depth assessment of mathematics literacy, with less detailed assessments in reading and science literacy. In PISA 2003, each student was awarded a score on the combined mathematics literacy scale based on the difficulty of the tasks that he or she could reliably perform. These student performance scores were also used to create six proficiency levels, with level 6 the highest. Students who failed to complete the tasks associated with level 1 were categorized as having proficiency below level 1.
In Japan, Canada, France, and Germany, students performed, on average, at proficiency level 3 on the PISA 2003 combined mathematics literacy scale; in the United States, the average score of 483 (see Indicator 7: Mathematics Performance of 15-Year-Olds Across Content Areas) was above the bottom cut point for level 3 by about 1 score point. Students in the Russian Federation and Italy scored, on average, at level 2 on the combined mathematics literacy scale.
Looking at the distribution of students across the mathematics proficiency levels, 26 percent of U.S. students scored at level 1 or below; these students failed to demonstrate consistently that they have baseline mathematical skills (figure 1). The U.S. percentage was higher than the percentages in four of the other G-8 countries reporting data1 (Germany, France, Japan, and Canada), but lower than the percentages in the Russian Federation and Italy.
The United States had a lower percentage of students at each of the higher proficiency levels of 4, 5, and 6 than did Germany, France, Japan, and Canada. None of the other G-8 countries had a lower percentage of students scoring at level 6 (the highest proficiency level) than the United States. The PISA 2003 results are somewhat different from those for PISA 2000, when reading literacy was the major domain and the United States had a higher percentage of students at the lowest proficiency level, but also a higher percentage of students at the highest proficiency level (Lemke et al. 2001).
Figure 1. Percentage distribution of 15-year-old students on the PISA 2003 proficiency levels for combined mathematics literacy scale, by country: 2003
Definitions and Methodology
PISA defines mathematics literacy as "an individual’s capacity to identify and understand the role that mathematics plays in the world, to make well-founded judgments and to use and engage with mathematics in ways that meet the needs of that individual’s life as a constructive, concerned and reflective citizen" (OECD 2003).
To facilitate the cross country comparison of achievement scores on the PISA 2003 combined mathematics literacy scale, an OECD average was calculated whereby all the participating OECD countries contributed equally. The data were then standardized to set the OECD average at 500, with a range from 0 to 1000 and a standard deviation of 100. Since the individual country means were weighted averages of the student scores, this standardization implied that about two-thirds of the students across all the participating OECD countries scored between 400 and 600.
Mathematics proficiency was defined in terms of six levels (levels 1 through 6) based on student performance scores on the combined mathematics literacy scale. Exact cut point scores are as follows: below level 1 (a score less than or equal to 357.77); level 1 (a score greater than 357.77 and less than or equal to 420.07); level 2 (a score greater than 420.07 and less than or equal to 482.38); level 3 (a score greater than 482.38 and less than or equal to 544.68); level 4 (a score greater than 544.68 and less than or equal to 606.99); level 5 (a score greater than 606.99 and less than or equal to 669.30); and level 6 (a score greater than 669.30). In order to reach a particular proficiency level, a student must have been able to correctly answer a majority of items at that level. Students at each succeeding level are capable of solving mathematical problems of increasing complexity.
Students proficient at level 1 are able to identify information and carry out routine procedures according to direct instructions in explicit situations, such as locating and reading a specified value in a simple table or performing simple calculations involving relationships between two familiar variables. Level 2 can be considered the baseline at which students begin to demonstrate mathematical skills allowing them to use mathematics actively; they can extract relevant information from a single source and make literal interpretations of the results, such as recognizing simple geometric patterns and identifying relevant information in a simple and familiar graph. At level 3, students can use simple problem-solving strategies and skills, such as reasoning in familiar contexts, interpreting tables to locate information, and basic reasoning with simple probability concepts; they can link and connect multiple related representations (e.g., a formula and a graph) and carry out clearly described procedures requiring sequential processes. At level 4, students can reason flexibly and with some insight; they can solve problems that involve reasoning and argumentation in unfamiliar contexts, interpret complex text and graphs, and use multiple representations and multi-step calculations to solve practical problems. Students at level 5 can use well-developed reasoning skills, insight, and interpretation with different representations; interpret complex information about real-world situations; work strategically; use complex and multistep problem-solving skills; and make assumptions or work with assumptions to solve problems. Students proficient at level 6 can identify and combine multiple pieces of information to solve complex problems in the context of unfamiliar real-world situations; they can carry out a complex sequence of calculations and communicate complex arguments and explanations through reflection, insight, and generalization of the results. For more information about how proficiency levels were set for PISA 2003, see the technical appendix in Lemke et al. (2004).
View the PDF Version 
(123 KB) of this indicator which includes text, figure(s) and data table(s).
