Summary Report, Good Start, Grow Smart, State Early Learning Guidelines Roundtable
From Blueprint to Reality: Early Learning Guidelines Implementation
Index: Preface, About The GSGS Initiative, Overview of the GSGS Roundtables, Summary of Key Points: Effective Collaboration; Infrastructure for Support; Dissemination of Information; Integration of ELGs into the Professional Development System; Assessment and Evaluation, Conclusion/Acknowledgements, Appendix: Agenda, Participant List, Work Group List | (This document is also available in 
PDF format.)
Summary of Key Points: Assessment and Evaluation
A final theme that emerged from the Roundtable was the importance of assessment and evaluation, and the approaches that States are taking to (1) assess children’s development and evaluate programs in relation to the ELGs, and (2) link ELGs to academic outcomes in elementary school (usually third-grade outcomes). While this type of assessment is not required under GSGS, many States are pursuing assessment as an important part of their school readiness infrastructure. Evaluation data is necessary to assess the effectiveness of continued program funding, and to sustain involvement and interest of the legislature and the public. However, obtaining accurate data that can be used in this way is challenging. Key recommendations that emerged from State discussions are identified below:
• Reconcile different definitions and vocabulary used within the early childhood community to describe accountability and assessment. Currently there is a lack of consensus on such key terms as “school readiness.” Some use this term to refer specifically to early reading and cognitive development, while others use it to mean a broader approach, focusing on development across multiple domains. This lack of consensus hinders agreement on the focus of early childhood assessments in the accountability process. Should these assessments focus only on early language and literacy development, on cognitive development more broadly, or also include a focus on social and emotional development, approaches to learning, and physical and motor development? Differences also exist in the early childhood community in views of appropriate practices for early childhood assessment (i.e., which kinds of assessments are appropriate to use in the accountability process, and which assessment methods to employ). Participants felt that States need to be clear on the goals for assessment and address issues of sample size, measurement tools, and data analysis and reporting, as well as the training necessary to effectively conduct assessments of young children. States suggested working towards a set of principles to guide their practices on accountability and assessment.
• Weigh alternatives when selecting approach to assessment and evaluation. There is a general recognition that different approaches are taken by different States and by different sectors of the early childhood community with regard to assessment and evaluation. Some States, such as Florida, are conducting universal screening at kindergarten entry as mandated by the legislature. In other States, school districts are under local control, and/or participation in assessment may be voluntary. Each of these situations may make it difficult to train teachers to administer assessments uniformly and to establish universal assessment across a State. Several States noted that training providers and teachers in the reliable use of assessment tools was essential to obtaining useful assessment data. Teacher turnover creates a challenge in providing adequate training.
Some States are using “home-grown” approaches to measuring quality, while others apply well-established measures of the quality of early care and education programs. Maryland has used the Work Sampling system developed by Sam Meisels to gather data on all kindergarten children in the State. This has been very helpful in providing 30 indicators that can be tracked for multiple cohorts of children. However, Maryland and other States stressed the considerable time required for adequate training of teachers and providers in the use of a system in order to support valid and reliable data collection. This experience suggests that States would be prudent to use at least the first 2 years of data collection to establish reasonable validity and reliability of data, before using the data for assessment and/or program evaluation purposes.
Several States, including Arkansas and Ohio, are using pre/posttest designs with standardized measures. California is using the early childhood environment rating scales (the ECERS-R, ITERS, and FDCRS), in addition to parent surveys, and teacher observation profiles as part of an accountability system. Ohio is piloting the use of an assessment of the literacy environment in the classroom (Early Language and Literacy Classroom Observation - ELLCO) in programs in which a sample of prekindergarten children participates.
States voiced concern over feeling pressure to gather evaluation data using tools that are not appropriate for that particular use. There was general agreement among the States that standardized measures, rather than “home-grown” measures, should be used whenever possible to measure domains addressed in the State’s ELGs. There was also agreement that screening tools used to identify developmental needs of individual children, and assessments intended for informing teacher practice should not be used for program evaluation purposes.
• Align assessments with content of ELGs. Another issue regarding assessment and ELGs is how to match what is assessed with the content of ELGs. For example, although ELGs stress multiple aspects of young children’s development, such as social-emotional development and health, many of the assessments of young children used in States focus primarily on literacy and numeracy. Partly, this problem is due to a lack of adequate measures of young children’s social-emotional development. However, a related issue is the strong focus on tracking children’s academic progress, even at a young age, with not as much regard to other important aspects of development.
• Communicate to policymakers and funders realistic expectations of evaluation results from a 1-year intervention. Arkansas is required by legislation to carry out longitudinal research, starting in kindergarten and following the children into the fourth grade. There is some concern among the States about the use of results from longitudinal studies in drawing conclusions about a 1-year preschool program. Specifically, there is a concern that comparisons might not be useful, or that conclusions about a preschool program’s effectiveness will be based on whether noticeable differences in performance occur years later. It was pointed out that longitudinal data do not always take into account changes in teachers, curriculum, or instructional practice across the years, all of which could affect outcomes over time.
In addition, States are using opportunities to educate legislators and policymakers about the research on effective assessment and evaluation to help them establish clear purposes and reasonable expectations for early care and education programs. Several States felt that politicians do not understand the difference between an intensive intervention with many resources, such as the Abecedarian or Perry Preschool project, and the associated results achieved, and the much more modest 1-year interventions being implemented currently in their States.
At the same time, there was acknowledgment that data needed to be gathered and shared on program performance in order to track progress and sustain funding for early childhood programs. The key was being clear on what a program is capable of accomplishing. Several success stories were shared. Maryland is conducting a universal assessment with Work Sampling, where data has indicated that children in child care did as well or better than children in Head Start and public prekindergarten. The self-image of child care providers was raised by these results. Another outcome is that funding for 4-year-old education of high-risk children was included in the State budget based on these assessment results. The view of Maryland’s education system now encompasses children from birth to Pre-K. These developments have strengthened the collaboration among child care providers and other early childhood educators.
In Georgia, a study began in 2001 utilizing probability samples from three groups of children attending preschool (Georgia Pre-K, Head Start, private preschool). The study followed the children through their first-grade year. Results from the first year and a half of the evaluation showed that children who attended preschool made gains of at least four points against the national norms on standardized assessments from preschool to kindergarten. An additional finding of the study was that summer learning loss was especially substantial for African American children.
In Georgia, during the summer of 2004, skills of children from economically disadvantaged households were studied through an enrichment program that served 173 children. Children enrolled in the program gained on standardized scores in expressive language, number skills, and story comprehension. These are skills where previous research indicates that Georgia’s children are behind upon kindergarten entry. Results will be used to support requests for more funding to expand summertime enrichment programs. |
Remaining challenges identified in the area of assessment and evaluation include:
• What is appropriate content to assess when children are in preschool?
• What approaches to assessment are appropriate with young children?
• How can States best respond to pressures to gather and use data in ways inconsistent with best practices in assessment?
• How can States effectively assess the mediating effects of curriculum and teacher practice on the relationship between ELGs implementation and outcomes for children?