|
International Mathematics and Science Achievement
from a U.S. Perspective: Results from TIMSS-R
 |
Good afternoon! My name is Gary
Phillips and I am the Acting Commissioner of the National Center for
Education Statistics. With me today is Dr. Judith Sunley, the
Interim Assistant Director of the Directorate for Education and
Human Resources at the National Science Foundation. We are here to
chat about the results that were released this morning from the
Third International Mathematics and Science Study-Repeat (TIMSS-R).
We are sure there are a lot of questions out there, so let's get
started… |
| Leigh from Whitehall, PA asked: |
| I noticed the panel
that constructed the study only has one public school educator. Why
weren't more private and public school educators included on the
panel? Also, are homeschooled students taken into consideration in
this study? Were any private schools included in the
study? |
| Dr. Gary Phillips & Dr. Judith Sunley: |
There were many people involved in the
development process including many private schools. The devlopement of
TIMSS involved many more people than the panel. We had a representative
sample of U.S. private schools in the study....Gary Phillips |
| Mary Ellen from Phoenix, Arizona asked: |
| I was privileged to
attend the sessions when the TIMSS report was released in Kansas
City. At that time it was evident that one of the reasons for our
poor showing in math was that the focus of instruction in the U.S.
schools was "drill the skills" instead of application of concepts. I
am very interested to learn if instruction in our schools has
changed sufficiently to cause an upsurge in the standing of U.S.
youth. Are the countries showing a high level of achievement still
using a concept/application model for
instruction? |
| Dr. Gary Phillips & Dr. Judith Sunley: |
Mary Ellen..According to our most
recent data, American teachers still use a lot of drill and
practice....Gary Phillips |
| Joseph from San Francisco, California
asked: |
| Will the data files
being released be of the same format as the 1995 data? If not, what
differences will there be? Is there any plan to compare the TIMSS-R
results with the recent NAEP science test
administration? |
| Dr. Gary Phillips & Dr. Judith Sunley: |
Joseph...The data files will be
released in about 6 months and they will be in the same format as the 1995 data. We
will also have a report next year on a link between NAEP and
TIMSS-R...Gary Phillips |
| Bridget from San Antonio, TX asked: |
| What do you feel is the
most significant finding in the study? |
| Dr. Gary Phillips & Dr. Judith Sunley: |
Bridget...The most significant finding
was that after 4 years there was no improvement in U.S. performance. We
may need to wait for more than 4 years to be able to detect
change...Gary Phillips |
| JULIE RIORDAN from WESTON, MA asked: |
| When will results for
participating states be available? |
| Dr. Gary Phillips & Dr. Judith Sunley: |
Julie...The results from the 13 States
and 14 school districts will be available April 4, 2001...Gary
Phillips |
| Paul Trafton from Cedar falls, IA asked: |
| Could you highlight
some of the key findings from the report? |
| Dr. Gary Phillips & Dr. Judith Sunley: |
Paul.. The U.S. was above the
international average in both math and science in the comparison of
38 nations. There was no improvement across the 4 years from 1995
and 1999 in the U.S., and for most nations among the 23 nations that participated in
both years....Gary Phillips |
| Billy from Montclair New Jersey asked: |
| There has been little
change in 8th grade performance from 1995. Are reforms being
implemented that would lead to improvement and when would you expect
to see that improvement? |
| Dr. Gary Phillips & Dr. Judith Sunley: |
Billy...Yes reforms have been
implemented. We hope to see changes when TIMSS is repeated in
2003....Gary Phillips |
| Steve from Hartford, CT asked: |
| Gary and Judy - One
simplified summary statement for TIMSS-R is that nothing has changed in
4 years. Is that an accurate summary and if so, what is a good
answer to the question "Why haven't we
improved?" |
| Dr. Gary Phillips & Dr. Judith Sunley: |
Steve-That is a good summary. We do not
have definitive answers to your question. We will know more when we
analyze the data from the Video Study....Gary
Phillips |
| Carole from Arlington, VA asked: |
| What were the sampling
procedures used in the United States? |
| Dr. Gary Phillips & Dr. Judith Sunley: |
Carole...In the U.S. we used a 3 stage
sampling procedure in which we first sampled geographic regions,
school districts, then schools. The U.S. sample was a nationally
representative sample as it was in all of the nations....Gary
Phillips |
| Peter from Hackensack, NJ asked: |
| Hi. I am a 6th-grade
teacher who believes in and implements standards-based instruction.
Accountability across the mathematics curriculum from K-12, however,
is, in my opinion sorely lacking. Many teachers are either
completely unfamiliar or just find the activities "too difficult."
In order to achieve, students need to be immersed in activities
which are part of a cohesive curriculum guided by skilled "coaches."
My question deals with systemic accountability. How can we best
create a system that embraces the standards in a cohesive,
"spiralling" curriculum from K-12? |
| Dr. Gary Phillips & Dr. Judith Sunley: |
We agree that a systemic approach is
very important. As states and districts review their curriculum,
they need to think in terms of a comprehensive program of
mathematics and science that strategically builds on learning across
the grades. As the earlier TIMSS study pointed out, the curriculum
in the U.S. may cover too many topics without enough depth.
Curriculum, assessment, teacher professional development as well as
state and district policies need to be
aligned...NSF |
| David from Texarkana, Texas asked: |
| What accounts for the
drop in performance from 4th to 8th grade by these
students? |
| Dr. Gary Phillips & Dr. Judith Sunley: |
David....There wasn't a drop in
performance. In the U.S., students learned a lot from the 4th to the
8th grade. However, in other countries they learned at a faster
rate....Gary Phillips |
| Mr. K. from Oradell asked: |
| I'm a sixth-grade
teacher who believes students need to come to class prepared with an
arsenal of "tools" in order to achieve problem-solving success (as
opposed to success at mere "exercises"). Because of the lack of
accountability in our system, students come to grade six without
many of these tools (frac/dec/perc, ratio & prop, integer usage,
et. al.) Mathematics, as the language of science, thrives on
problem-solving. The standards are obviously wonderful and, if
implemented systemwide in each and every district, will yield
outstanding results. Can we hold teachers accountable for
standards-based teaching? |
| Dr. Gary Phillips & Dr. Judith Sunley: |
We agree that the standards, developed
by the mathematical and science communities, have gone a long way in
identifying important mathematical skills and problem solving
abilities that all students need to know and be able to do. To
achieve these standards, teachers must have an understanding of
content, pedagogical skills, and appropriate curriculum. They need
time to carefully plan their lessons. Assessments need to be aligned
with content and focus on both basic skills and problem solving
abilities. States, districts, and teachers can all be held
accountable for ensuring that all students achieve in mathematics
and science. |
| Bridget from San Antonio, TX asked: |
| Following up on the
last question, did other countries improve their scores or were they
also static? |
| Dr. Gary Phillips & Dr. Judith Sunley: |
Bridget...Thank you for the follow up
on an earlier question. There was no improvement in almost all of
the other countries (18-19 countries)....Gary
Phillips |
| Marion from Providence, Ri asked: |
| How was it determined
what was a statistically significant change? I noticed, for example,
that in the 1995-1999 change in math achievement, Canada, with a
10-point difference, and Cyprus, with a 9-point difference, were
considered to have "significantly higher" scores, but not Hong Kong
(with 13 pts) or the U.S. Also, why is there a warning in the study
against comparing the relative change in different
countries? |
| Dr. Gary Phillips & Dr. Judith Sunley: |
Marion...A statistically significant
change is one in which we have a 95% confidence level. The reason
why a 10 point difference might be significant in some countries and
not others is because the margin of error is different in each
country.....Gary Phillips |
| Penny from Weston, MA asked: |
| Do you have any
thoughts about why the performance of black students in mathematics
has increased so substantially? Do you attribute any of the change
to the USI/SSI programs? Also, it looks as if the performance of
students whose parents have a college education has improved. Is
this so? |
| Dr. Gary Phillips & Dr. Judith Sunley: |
Penny...The data do not tell us "why".
However, we have been tracking a reduction in the black/white gap
for some time in our domestic NAEP program. Yes, the students of
parents with college degrees have improved over the 4 years....Gary
Phillips |
| Darnella from Bethesda, Maryland asked: |
| What can you say about
the range of scores? Is the achievement gap
narrowing? |
| Dr. Gary Phillips & Dr. Judith Sunley: |
Darnella...Yes, the achievement gap is
narrowing for blacks and whites, but not for hispanics.....Gary
Phillips |
| Melissa from Atlanta, GA asked: |
| Has there been any work
done to cross-examine the findings of the TIMSS-R and the Glenn
Commission report? |
| Dr. Gary Phillips & Dr. Judith Sunley: |
Melissa-Yes. The Glenn Commission used
the TIMSS data extensively....Gary
Phillips |
| Jill from Maryland asked: |
| In what sub-areas did
African-African students improve the most dramatically in
mathematics? |
| Dr. Gary Phillips & Dr. Judith Sunley: |
Jill...African American students
improved across the board in math.....Gary
Phillips |
| kathryn from Ft. Collins, Colorado
asked: |
| 1. How can we ensure
conceptual as well as skills knowledge? 2. What are perceived as the
primary reasons for the differences in the TIMSS results? 3. Is
there a math program you could recommend for American junior high
school? |
| Dr. Gary Phillips & Dr. Judith Sunley: |
A rigorous curriculum taught by well
qualified teachers in a system that supports them are the best
strategies. There are a number of mathematics programs developed
with support from NSF that emphasize conceptual and skills
knowledge. Among these are Connected Mathematics (Prentice Hall);
Mathematics in Context (Encylopedia Britannica) and MATHematics
(McDouglal Littell). -NSF |
| Gerald from Washington, DC asked: |
| How much variation do
you anticipate among the state and local school achievement reports?
|
| Dr. Gary Phillips & Dr. Judith Sunley: |
Gerald....We expect as much variation
among our States and districts as we found across the countries in
TIMSS-R.....Gary Phillips |
| Fred from Potomac, MD asked: |
| Will you gather the
data on the curriculum/training changes made by each participating
school system? |
| Dr. Gary Phillips & Dr. Judith Sunley: |
Fred...We will have some information on
this among States and districts, but not from the
countries....Gary Phillips |
| Karla from Arlington, VA asked: |
| When will the NAEP
correlation be available? |
| Dr. Gary Phillips & Dr. Judith Sunley: |
Karla...The results from the
NAEP/TIMSS-R linking study will be available next summer.....Gary
Phillips |
| Lane from Columbia, SC asked: |
| I'm sure the TIMSS-R
results will spur many reform activities. However, until the next
assessment, (FIMSS maybe?), how will we gauge the effectiveness of
what we are doing? When is the next international assessment
planned? |
| Dr. Gary Phillips & Dr. Judith Sunley: |
The most helpful strategy is to
constantly evaluate your programs in order to gauge effectiveness of
science and mathematics education. This is what NSF does. Further,
many districts are using TIMSS findings to plan programs to assess
their progress. The IEA is planning to conduct another assessment in
2003. We think trend data are very important, but the decision for
the U.S. to participate has not yet been made.
-NSF |
| Miriam from Pennsylvania asked: |
| I know my school
participated in TIMSS-R as part of the US cohort. Will we receive
our scores? |
| Dr. Gary Phillips & Dr. Judith Sunley: |
Miriam...Unfortunately we are not
giving back results to schools this time. We do not have
representative data for each school. Instead the study is nationally
representative....Gary Phillips |
| Ellen R. Delisio from Educationworld, Wallingford, CT
asked: |
| How will these results
affect curriculum? |
| Dr. Gary Phillips & Dr. Judith Sunley: |
TIMSS findings as well as those of
TIMSS-R reinforce the need for curricula solidly based in important
concepts that are taught by inquiry, problem solving and other
standards-based techniques. -NSF |
| Bridget from San Antonio, TX asked: |
| Just one last
question...If there was no change in performance in the majority of
the countries studied, is that why you said 4 years may not have
been long enough to see a change? And, if that is the case, then, in
your opinion, what is the value or benefit of the
study? |
| Dr. Gary Phillips & Dr. Judith Sunley: |
Bridget. Thanks for your questions.
They were good ones. We do not really know how long to wait to be
able to detect a change. We only know by actually doing the study.
This is the first international study that explicitly tracked
progress over time....Gary Phillips |
| Mary Ellen from Phoenix, Arizona asked: |
| It is absolutely
incredible that no change has been shown in improvement. Is there
anyway that this information can be disseminated to states and
school districts? I may be mistaken, but the last report did not
seem to have a wide circulation. |
| Dr. Gary Phillips & Dr. Judith Sunley: |
Mary Ellen...We are working to make the
information more available to States and districts. You can do your
part by alerting people to the TIMSS website at http://nces.ed.gov/timss/....Gary
Phillips |
| Billy from Montclair New Jersey asked: |
| What factors do you
think contribute to the superiority of several Asian countries on
TIMSS and TIMSS-R? |
| Dr. Gary Phillips & Dr. Judith Sunley: |
Billy...Our data show that U.S. teachers
teach how to do math and science, but Asian teachers teach students
to understand math and science....Gary
Phillips |
| Patti from Ft. Sumner, New Mexico asked: |
| The study points to the
fact that students in other countries have teachers with a degree in
either math or science while US teachers usually have degrees in
education. What provisions are being made to raise teacher salaries
in order to allow people with subject degrees to even contemplate
education as a viable job? |
| Dr. Gary Phillips & Dr. Judith Sunley: |
As you know, teacher assignments as
well as their salaries are determined locally. However, NSF can
affect the quality of teacher preparation and practice in
mathematics and science. Specifically, programs at NSF encourage
undergraduate degrees or substantive coursework in a content major
and content based professional development.
-NSF |
| Thomas from Delaware asked: |
| Nationally, only 25% of
American 8th graders take Algebra. Have you examined the math
achievement of this cohort of students for
TIMSS? |
| Dr. Gary Phillips & Dr. Judith Sunley: |
Thomas...No we do not have data broken
down in this way....Gary Phillips |
| Frank Cole from Oakhurst New Jersey
asked: |
| If no improvement has
occurred in the participating countries, could it be that it is not
due to a "lack of data" but rather that a particular method (ie.
drill & practice or concept based) is reaching its maximum
potential. |
| Dr. Gary Phillips & Dr. Judith Sunley: |
Frank...We will know more about this
issue when we analyze the Video data next year.....Gary
Phillips |
| Paul D Boyer PhD from Kenosha, WI asked: |
| Although I'm rather
shocked at the general lack of improvement, is there any sign that
the science education standards are 1) being incorporated into
learning outcomes and 2) making any difference in terms of
conceptual learning vs rote memorization?
Thanks. |
| Dr. Gary Phillips & Dr. Judith Sunley: |
NSF is supporting the development of
new curricula on learning outcomes that is grounded in science
standards. Studies in classes of teachers who are implementing
science standards indicate that their students have increased their
conceptual understanding. With the dissemination of new materials,
it is hoped that this effect will increase. Where standards based
curricula are implemented appropriately by well educated teachers
with assessments that are in line with the materials, students learn
at high levels. -NSF |
| Marion from Providence, RI asked: |
| A very technical little
question: is the country listed as "Chinese Taipei" what we
generally call Taiwan, or is it a more limited
sample? |
| Dr. Gary Phillips & Dr. Judith Sunley: |
Marion. Yes, that is Taiwan. The sample
covered the full Island.....Gary Phillips |
| Cindy from Pittsburgh, Pennsylvania
asked: |
| As a followup to
Billy's question - You summarized the differences between how our
teachers and their Asian counterparts teach. I assume that came from
a review of the TIMSS videos, but isn't it mostly the case that our
unofficial "national" curriculum (as represented in MOST textbooks
we use) presents a curriculum that is too fragmented and unfocused
to support meaningful instruction, hence focused
achievement? |
| Dr. Gary Phillips & Dr. Judith Sunley: |
Cindy...The answer is yes. In general
the Asian curriculum is more focused and textbooks are
shorter....Gary Phillips |
| Billy from Montclair, New Jersey asked: |
| Our data show that US
teachers teach how to do math and science, but Asian teachers teach
students to understand math and science....Gary Phillips SO,
SHOULDN'T TEACHER EDUCATION AND PROFESSIONAL DEVELOPMENT BE
CHANGING--SIGNIFICANTLY AND RAPIDLY--TO REFLECT
THAT? |
| Dr. Gary Phillips & Dr. Judith Sunley: |
Billy...YES....Gary
Phillips |
| Billy from Montclair New Jersey asked: |
| What steps will NSF
take that can lead to concrete reforms in the nation's
schools? |
| Dr. Gary Phillips & Dr. Judith Sunley: |
NSF has developed many programs that
lead to changes in teaching practices and student learning. For
example, our systemic initiatives as well as our local systemic
change program specifically address reform in schools. Our new
Centers for Learning and Teaching will rebuild connections between
schools and higher education to collaboratively address reform.
-NSF |
| Dale from Philadelphia asked: |
| What is the most
troubling trend in this report? |
| Dr. Gary Phillips & Dr. Judith Sunley: |
The most troubling aspect of the report
is that the U.S. continues to lag well behind the top performing
countries. However, when comparing the results of 1995 to those of
1999, we must keep in mind that improvement in a large diverse
educational system will be slow. We have time for one more question
-NSF |
| Cindy from Pittsburgh, Pennsylvania
asked: |
| What efforts are
underway to better understand the context of curriculum and
instruction in our middle schools that might make a difference on
how our 8th graders perform? Are there any chances of being able to
look at subsets of data from schools that have adopted/used
exemplary curricula? |
| Dr. Gary Phillips & Dr. Judith Sunley: |
NSF has funded several projects to
study the impact of innovative instructional materials on student
achievement. Results from these studies will not be available for
another year or two. However, TIMSS-R results for a number of states
and consortia of schools will be released in April 2001.
-NSF |
| Thank you for taking
the time to ask these questions. They were all very good. If you
have additional questions, please feel free to contact NCES or NSF.
At NCES, you can contact Patrick Gonzales (patrick.gonzales@ed.gov),
and at NSF you can contact Elizabeth VanderPutten
(evanderp@nsf.gov). Today's release of Pursuing Excellence:
Comparisons of International Eighth-Grade Mathematics and Science
Achievement from a U.S. Perspective, 1995 and 1999 is the first in a
series of several reports focusing on mathematics and science
achievement, teaching, and learning in an international context that
will be released over the next several years. We look forward to
continuing this dialogue with you. |
|
What's New?