MS. NEUMAN: The first presentation is by Russ Gersten. I have read so much of his work over the years. He's at the University of Oregon. He's done a lot of work on reading comprehension, teacher knowledge, and today what he's going to be talking about is the scientific based evidence and what that means for math education and achievement.

     MR. RUSSELL GERSTEN: This is actually an easy topic to be brief on because there isn't a lot of scientific research in math. There's some. There's some promising directions, but it is a somewhat depressing topic.

     There are two things going on. One, in elementary education there is no question that most teachers, even most parents,—the reading is the big emphasis there compared to math. But it's not that simple. For other reasons, the math community of math educators at least for forty-plus years has looked at their role as reform, as change, as re-conceptualizing.

     Therefore, there hasn't been this steady tradition. There are a few exceptions of really systematically using the methods that Valerie and others talked about earlier to build a knowledge base, but rather to study using the more qualitative methods: teachers understandings, kids understandings.

     So, this is something that can change. There have always been little glimmerings of change. There's a slight increase in the amount, but overall the math education community has been quite resistant to that, where let's say in the reading field there have always been at least two schools of thought, one in the experimental group.

     But rather than just dealing with how little we know and getting us all depressed, I am going to give some highlights of some work we recently did actually for the state of Texas who was beginning a big initiative in the area of math, getting kids ready for algebra. So, it was basically, these kind of low achieving kids who got to middle school and just were weak in all areas of math. We tried to put together the scientific research, using the procedures we've heard about in terms of meta-analysis and all, in the area of math for low achieving kids. I did this with my colleagues Scott Baker and Dae Sik Lei.

     I'm going to quickly go through the criteria, and they resonate with what we've been hearing about during the first session. We looked for studies that used random assignment. We did include the quasi-experiments, the ones that are kind of close, but they only were included if they had measures showed that the groups were comparable at the beginning. So, if they just used the school down the road, they were thrown out. They had to have at least one math performance measure, which sounds weird. But there were articles published in journals that either had teachers grades or students attitudes or certain interviews that we had no idea were they valid or reliable.

     We found four categories. Notice the small number of studies we found on this. Now, we limited ourselves to low achieving students. These were students whose documentation was well below grade level, at least below the 35th percentile on some standardized measure.

     But some of the things that worked, and again we don't have a lot of replications, but they were pretty decent studies, is that when kids and/or their teachers get ongoing information, every two weeks, every four weeks, of where they are in math in terms of either the state standards or some framework, it invariably enhances performance.

     This sounds kind of a little boring, it's not as romantic, there's so much of romantic work done in math. But the idea of having a system to know where kids are and what they really know, rather than saying this kid is struggling, this kid is struggling with fractions, manipulating fractions, more than one, with dividing fractions, with a sense of place value once you get into the hundreds. That information can be critical for low achieving kids, can be a life or death issue.

     The second group we found, there was only six studies, is peer assisted learning. It's usually tutoring. This is something that could revolutionize practice. Invariably, when kids are partnered up, and it seems to be better if they're heterogeneous pairs, there's one stronger student and one weaker student and they switch off, achievement in math is always improved.

     So, peers can be excellent tutors. I'm not talking here about cooperative groups of four, five, six kids. It's two. And if you see the difference in classrooms when there are two, it's very easy for the teacher to quickly monitor and get a sense of what's going one. Because kids are either working on stuff together, giving each other feedback, taking turns, or they're not. When it's a group of four or five, you're never quite sure what's this group discussing, these two kids look zoned out, but maybe they're finished.

     So, the advantage of this, again we're not dealing with these profound things but with these kind of building blocks of improving practice and especially if this is based on the kind of data we were talking about can lead to reliable, replicated improvements in performance.

     The one thing about the studies, and then we'll go on with the finding, is that 60 percent of them used random assignment so they met the gold standard. Another third were this quasi-experimental group, so overall the small set we had were of good quality. And seven percent were partial—they randomly assigned teachers and gave us some evidence that the groups were equal at the beginning which in the scheme of things is very, very good.

     This is something that wasn't discussed so much earlier and is critical is did somebody come in and see were people doing what they're supposed to be doing? Because one of the key findings from the 1960s is sometimes these evaluations were done of people who were supposed to be doing science this way, or math this way, reading this way, but there was no evidence that they were really doing it. And, in fact, when people did drop-in site visits, they found they were not doing it.

     So, two out of three studies did have an observer come in once or twice a week and make sure the thing was happening which sounds mundane and all but was a critical thing. So the quality indicators of the studies were good.

     I'll go back to just kind of a quick summary, trying to speed this up. With the peer-assisted learning, the six studies consistently showed moderate effects—and I'm not giving the exact numbers, but there's statistical ways to cut across called meta-analyis—and that is an important finding.

     When kids saw the data, and it was almost always on the computer, how they were doing, which skills they needed work on, whether they were making progress, these were moderately large, these were pretty large. This was especially true not so much for special education students but for that other that kind of at-risk group who are sometimes in Title I programs who sometimes need tutoring, that giving kids this kind of feedback seems invariably to help.

     A very small number of studies on instruction. We broke them two ways: explicit instruction, that includes both the very, very heavily tightly sequenced work that Carnan and some of his colleagues did in math which has everything sequenced exactly for kids and a beautiful array of examples, and some of these other approaches to teach kids problem solving strategies.

     In both cases, and we only have a small set because we're looking kindergarten through eighth grade, but there is some evidence that providing this degree of explicitness to kids, showing them strategies, letting them take over and showing what they know is helpful.

     This is hardly a revolutionary finding but it is important because there are many in the schools who do not advocate for such practice. This is invariably useful and when that's removed from children, especially the children below average, it tends to lower or decrease their achievement.

     Contextualized instruction was our way to fit together very, very, very exciting ideas about the discussion teaching fractions and getting kids immersed in real world problems that involve measuring and fractions and equivalents. And the results? I put a question mark there. When we averaged them together—and again we're only dealing with four studies—it came out about zero.

     So, basically, there is something there but how to get it into an effective package requires a lot of work.

     This is an interesting thing. There were only two studies here that were done in inner city Philadelphia schools in terms of giving concrete feedback to parents on how kids are doing. These are low achieving kids and we're getting into the middle school years.

     What the researchers found and they did two things. They set up the tutoring, was one thing they did, and then using this randomized idea for about half those kids and about half of the control group kids they also gave the parents feedback when the kid was doing well.

     This was their reasoning—and this isn't the only approach in terms of communicating with parents—that often by middle school when kids are D students and basic math, whatever it may be called, the lower track courses, in tending to get feedback it tends to be very negative. So, the teachers, if the kid was having problems, they gave that information for the peer tutoring session. But when the kid did well, they sent notes home, they called, now they could e-mail—these studies were done a while ago—and said you're kid is doing well you folks should celebrate this. Go walk up the mountain, a pizza party, whatever it is. So that the parents started to know the weeks, their daughter or son was doing well in math.

     Now, that isn't a lot. I wish to say we had a hundred other findings. We don't. I just have a couple thoughts towards the future. Susan, if I could have a couple of minutes?

     There are other lines of research that are not controlled intervention studies taking place in classrooms. I think we need hundreds more of those studies. Because as you see from this very small group of approximately 15 studies, we found some things that could be immediately useful for helping the below average, the at-risk kid in math.

     But in terms of really conceptualizing and thinking about math, a couple of just my thoughts on what I envision is. As in the area of early reading about twenty years or so ago there was this insight and some beginning work on the phonological or phoneme awareness idea and how critical that was. Initially, it was very vague and no one quite knew what to do with it. There were some programs that seemed to have parts of it. It took a long time for that to solidify.

     There's some very, very interesting work especially done by the late Robbie Case and Bob Siegler and others, in the beginnings of math. And, at least in math, unlike years ago, we do have some measures that can predict. In kindergarten, we're doing some work in Eugene Research Institute in both Oregon and Texas at looking at predicting things by the end of kindergarten that will tell you which kids are likely to be at-risk. So you can start to screen and get a sense of stuff.

     So, we do have at least a couple of measures that seem to validly predict and I know David Gehry at NIH is doing some work along this lines. So, we're maybe twenty years behind reading in this early intervention mode in terms of starting in kindergarten, starting in preschool, but we can move a lot faster now. We have the model of what succeeded in reading.

     The other thing is we have this concept which is still elusive called "number sense." You'll see it around a lot. Nobody knows exactly what it is. It's sort of a sense of numbers, the way some kids just sort of take to it. You ask them, well, you know, here are six things, we want nine, how many more do you need? They'll just go "three." And, others will just go, "Well, you need some more."

     But, it's just basically, the idea of both performing and understanding and doing and strategizing. We have his general notion. It seems a fascinating one. It seems a wonderful spur for a generation of new researches to do the kind of array of scientific methods. So, that's one huge area.

     And I'm only going to do one other one. But this is something we've thought a lot about. One reason there's so little intervention research in education is people who've done it you leave totally exhausted. You're developing a new curriculum, you're training teachers, you're going in to see are they implementing it the right way. You're problem solving. You're going, oh, my god, why did we sequence the fourth week this way. You know, these things happen.

     Then you're trying to develop valid and reliable measures. You know, you do one or two of those. Then you say, well, maybe I'll do more, you know, literature reviews or correlational studies or descriptive case studies, because it is absolutely exhausting.

     (Laughter.)

     And you look at any discipline, and it's amazingly few people who have the endurance to do this.

     But one system that the late Ann Brown developed is a very good one. What it calls for it says let's be honest. You can't just run in there and say this is a good way to teach math problem solving, where kids learn the stuff and then they practice in context. You need a while to do what she called "design experiments." To really go in and see what happens and collect data and not do the control groups and the randomization. You need one or two of those to get the thing working.

     And they are not really just pilot studies. They are serious investigations of taking these phenomenal insights from cognitive psychology, from developmental psychology, but trying to put them into useable packages that there is some data to support.

     Math is a long way from this. But this combination of doing the design experiments, but then not stopping there, to then test with the kind of controlled studies we were talking about before.

     Those to me are the two at a national scope for future research. In terms of the last one, towards the future, I think because we're seeing such consistence sense that when the teachers or kids get ongoing data where kids are and what they need to learn once a month as opposed to once a year. It's a great way in October to say, you know, this kid doesn't know how to multiply fractions. So, he's in the 7th grade, but let's get that under her belt, his belt, so we can move forward and this kid isn't going to get lost in pre-algebra. So, we need strategies and measures to get this into practice.

     The last thing is, as we look at what's going on in the field, we could do as twenty years ago Thomas Goode and Douglas Grouse did, which is look at what's happening in schools and try to link them to outcomes. Because we've got a huge array of measures in math, but we don't have a sense of which ones lead to better achievement or not.

     So, those are my four thoughts towards the future and my sense of some pockets of knowledge we know for this average population.

     (Applause.)