A r c h i v e d  I n f o r m a t i o n

Chapter 8: Findings from Two Evaluations

In this chapter, we bring together specific outcome data from the first and second Even Start evaluations in order to elaborate upon what we have learned about the effects of Even Start on children's cognitive development and adult literacy. This chapter incorporates data from both evaluations and presents a detailed summary of findings to date on the effects of Even Start on children's cognitive development and parents' basic skills development.

On the basis of previous research on early childhood educational programs that target at-risk children, we might expect to find evidence of longer-term effects on children's cognitive development and academic/social performance (Barnett, 1992; Campbell and Ramey, 1994; Scheinwart and Weikart, 1997). We might also expect to see evidence of effects or impact in the short term, but some research suggests that short-term effects of such programs are not maintained over time (Casto and Mastropieri, 1986; St.Pierre, 1994). We might expect that the evaluation of Even Start would reveal some impact, at least in the short term, on children's developmental and/or educational measures. Both the first and second national evaluations have indeed found evidence of some short-term effects of participation in the Even Start program.

First, we consider evidence based on children's scores on the PSI, using differences between children's scores at various ages. The line labeled "developmental growth" in Exhibit 8.1 shows that children's scores on the PSI increase as they age, by about .40 items per month. This estimate, derived from an analysis of PSI pretest scores collected during the first evaluation, illustrates how we can expect Even Start children to perform on the PSI in the absence of any intervention. The two lines labeled NEIS and ESIS show the observed PSI gains for children in the first and second evaluations, respectively. Note that these two lines both indicate positive changes in observed test scores for children as they develop and grow older. The average changes in scores are greater than one might expect given the .40 items per month boost one would attribute to another month of age. This analysis would lead us to conclude that Even Start has an important impact on children's PSI scores.

But other data also are available. The two lines on Exhibit 8.1 labeled IDS Even Start and IDS Control show the gain scores for children in the five sites that participated in an experimental In-Depth Study during the first evaluation. The changes in scores for children in these groups fall between expected developmental growth and the NEIS "universe" group. This is not unexpected; the five sites selected for the In-Depth Study cannot be expected to mirror the universe of Even Start projects exactly.

On the other hand, the difference in gain scores between the randomly assigned Even Start and control group children is not statistically significant, indicating that we see no special effect of being in Even Start. Additional analyses, presented in the first evaluation report, show that the straight line differentials presented in Exhibit 8.1 are a bit misleading. In particular, Even Start children in the IDS sites outscored control group children during their first year in the program, when control group children typically were not in any organized preschool program. However, control group children?s PSI scores caught up once they entered preschool or the public school system.

The true growth rate data described in Chapter 7 both corroborate and challenge findings obtained during the first evaluation, that is, that children do indeed grow while in Even Start, and that their growth goes beyond what one might expect given maturation alone. But there are some important points that emerge from the analyses reported earlier. One is that examining a cross-sectional sample of Even Start children to hypothesize about growth may provide incomplete information. On the basis of the longitudinal analyses of the multiple-wave children, it is clear that the pretest scores for children with only one wave are systematically different from those with two or more waves. Further, it is also clear that the growth rates vary across critical predictors, and this has potential implications for thinking about program design and service delivery. Children who enroll in Even Start at earlier ages grow at a different rate?and at a more accelerated rate?than one might predict on the basis of age alone. The evidence from both the PSI and the PLS suggests that children who remain in Even Start for longer periods of time may experience growth in outcome measures that begin to tap into the domain of cognitive achievement.

What remains to be seen is whether the growth observed would be sustained, and that question can only be addressed through a study that follows children well into their elementary school years. When we conducted a follow-up study of children from the In-Depth Study of the first national evaluation, we found no real differences between Even Start and comparison children (Gamse, Conger, Elson, and McCarthy, 1997), although the study examined children two and three years later. The evidence from a few long-term follow-up studies consistently finds meaningful differences after a greater amount of time between program participation and subsequent post-program data collection. In fact, other impact studies of early childhood education programs typically do not find evidence of meaningful differences until several years after program participation (Schweinhart and Weikart, 1997).

The same pattern is seen in an analysis of data from the Peabody Picture Vocabulary Test (PPVT), a measure used only in the first evaluation. Exhibit 8.2 shows that the universe of children in the first evaluation gained at a faster rate than children in the IDS study. Children in the IDS Even Start group gained more than IDS control group children from pretest to the first posttest. However, control group children caught up by the time the second posttest was administered.

Analysis of data collected on the PLS-3 in the second national Even Start evaluation shows growth from pretest to posttest (over a single program year as well as when examining rates of growth over a longer time period, as described earlier). This growth suggests that the gap between Even Start children and children in the PLS norms group begins to narrow over time (Exhibit 8.3, see also Exhibit D.2a, Appendix D). Again, because the norming population?s scores are scaled such that an average child would score 100 at any age of administration, one would not anticipate observing growth (within the norming population). The fact that the distance between Even Start children?s scores and the flat scaled score appears to decrease over time suggests that participation in Even Start is having an effect on children?s educational outcomes.

It appears from all of these measures that children get a "boost" in cognitive development when they first are exposed to an organized school setting (preschool or the public schools). Enrollment in Even Start ensures that such an exposure occurs at an earlier age, so Even Start children get an earlier boost than control group children. The question to be answered by future research is whether that early boost translates into other types of benefits for Even Start children.

-###-

[What Were the Outcomes on Other Progress Indicators?]

[Adult Education Development]