ACF OPRE: Head Start Impact Study and Follow-up

NATIONAL HEAD START IMPACT RESEARCH

This report may contain external links. ACF cannot attest to the accuracy of information provided by external links. Providing links to a non-ACF Website does not constitute an endorsement by ACF or any of its employees of the sponsors of the site or the information or products presented on the site. Also, be aware that the privacy protection provided on the ACF domain (see ACF's Privacy Policy) may not be available at the external link.

Background

Highlights of the Advisory Committee Report

Summary of Accomplishments

Purpose of the Research

Overview of Study Design

Sampling Plan

Field Test

Data Collection

Local Study Staff

Notification and Informed Consent

Section 649(g) of the Head Start Act, as amended by the Coats Human Services Reauthorization Act of 1998 (P.L. 105-285) required the Department of Health and Human Services (HHS) to conduct a national analysis of the impact of Head Start. The legislation also charged the Secretary to appoint an independent panel of experts to review and make recommendations on the design of a plan for research on the impact of Head Start within one year after the date of enactment of P.L. 105-285, to advise the Secretary regarding the progress of the research, and to comment on the interim and final research reports. There were three Reports to the Congress mandated in the legislation. The first of two required interim reports, summarizing the deliberations and recommendations of the Advisory Committee, was completed and transmitted to Congress in October 1999. The current Report to the Congress is being submitted to fulfill the requirements of a second mandated interim report to “describe the status of the research and preliminary findings of the research, as appropriate.” Thus, this current report will describe the background and purposes of the study, the progress made to date in implementing the study, the current activities being undertaken to conduct a field test and prepare for the full study, and the projected timeline for completing all phases of the study.

Background

Since 1965, the Head Start program has served nearly 18 million low-income children and their families. The program has provided support to families seeking enriching experiences for their children, and it has provided direction and leadership to the fields of early child development and education. With increased attention to outcomes and accountability of Federal resources and programs, the Head Start program has been challenged to document its effectiveness in new ways. A series of reports from the U.S. General Accounting Office and concern among Congressional leaders about the need for rigorous experimental research testing the effectiveness of the program prompted Congress to call for the formation of an independent panel of experts to review and make recommendations to the Secretary of Health and Human Services on the design of a study or set of studies that provide a “national analysis of the impact of Head Start”.

The HHS Secretary chartered an Advisory Committee on Head Start Research and Evaluation on March 23, 1999, in order to meet the legislative requirement of section 649(g) of the Head Start Act. Membership of the Committee included a selection of distinguished experts in both early childhood development and the use of evaluation methodologies in early education and social programs, as well as a senior researcher from the General Accounting Office, Head Start expert practitioners with experience linking research and service delivery, and several HHS officials. The Committee met three times in the Spring and Summer of 1999. The Committee completed its report, Evaluating Head Start: A Recommended Framework for Studying the Impact of the Head Start Program, in October of 1999, including recommendations to the HHS Secretary. The Committee set forth a framework for impact research in Head Start that it believed to be both scientifically credible and feasible. The Committee meetings were open to the public and summaries of the meetings, as well as the report and other materials on the proceedings, were made available to the public via the Head Start website (http://www.acf.hhs.gov/programs/hsb/research/hsreac/index.htm).

Highlights of the Advisory Committee Report

In its recommendations to the HHS Secretary, the Advisory Committee built upon the basic legislative requirements and concluded that a study or set of studies must address two main research questions:

First, the evaluation research must answer the question of impact: “What difference does Head Start make to key outcomes of development and learning (and, in particular the multiple domains of school readiness) for low-income children? What difference does Head Start make to parental practices that contribute to children’s school readiness?”
Second, the evaluation research must address the question of how impact varies in certain key situations: “Under what circumstances does Head Start achieve the greatest impact? What works for which children? What Head Start services are most related to impact?”

The Advisory Committee saw its charge as developing recommendations for a research design that is capable of answering these questions and that meets two key criteria. First, an acceptable design must be credible to a wide audience, including researchers, policy-makers, and practitioners. Second, an acceptable design must be feasible, that is, capable of being implemented effectively.

Within the context of the legislative requirements, the Advisory Committee also proposed a framework with several key elements:

Children and families should be randomly assigned to Head Start and non-Head Start groups at a diverse group of sites located across the country.
Sites where Head Start saturates the community should be excluded from the random assignment portion of the study or studies. Similarly, sites that are out of compliance with Head Start standards or extremely new to the program should be excluded.
Every effort should be made to ensure that the sites selected are representative of Head Start sites nationally and reflect the range of Head Start quality across the country.
The study or studies should collect comparable information on Head Start children and control group children across all areas of measurement, including the quality of the care provided.
Outcome measurement in the study or studies should focus on the multiple domains important for school readiness of children (as defined by the National Education Goals 2000, Goal 1) and on parental practices that contribute to school readiness.
Quasi-experimental and other embedded studies can be used to supplement the information from the randomized impact study or studies.
Incentives should be offered to parents and programs to secure participation in the study or studies.
The research and findings from the impact study or studies should be used to improve the effectiveness of Head Start programs for the benefit of children and families.
The impact research should draw on information from existing Head Start research efforts, and thus, be part of the overall research agenda, not a substitute for it.

Summary of Accomplishments

In May 2000, HHS published a competitive procurement announcement, seeking proposals for a qualified contractor to design and conduct a comprehensive study to provide a national analysis of the impact of Head Start. In October 2000, HHS awarded a contract to Westat in collaboration with the Urban Institute, American Institutes for Research, and Decision Information Resources to conduct the research.

In conjunction with the initial study planning activities following the contract award, a fourth meeting of the Advisory Committee was convened in January 2001 to provide the Committee with an overview of the proposed study design and seek additional consultation. The other major accomplishments completed to date include the following: (1) selection of the eight sites for the field test; (2) on-site recruitment and implementation activities in the eight field test sites; (3) training of the field test data collectors; (4) selection of the 75 programs for the main study; (5) initial information gathering from the 75 programs in the main study to assist in the sampling of centers from within each of the 75 grantee/delegate agencies; (6) review and approval of a data collection package by the Office of Management and Budget; and (7) the establishment of several workgroups of external consultants to provide input on possible new measures or measurement approaches. In addition, the Advisory Committee currently is being reconstituted and it is anticipated that the next meeting will be held later this year to review the progress of the study. The following section provides a more specific description of the overall study design and accomplishments to date.

Purpose of the Research

The research design of the Head Start Impact Study is derived primarily from the legislative language of the Head Start Act, as well as the set of recommendations from the report of the Advisory Committee on Head Start Research and Evaluation. As noted above, the study has two primary goals:

To determine, on a national basis, how Head Start affects the school readiness of children participating in the program as compared to children not enrolled in Head Start. In other words, does Head Start improve children’s cognitive development, general knowledge, approaches to learning, social and emotional development, communication skills, fine and gross motor skills, and physical well-being? In addition, how does Head Start affect the parental practices that contribute to children's school readiness?
To determine the conditions under which Head Start works best and for which children. To meet this goal, the study will examine various factors that could affect the results of the Head Start program including differences among the types of children attending Head Start, children’s home environments, types of Head Start programs (home or center-based, quality indicators such as staff-child ratio, curriculum, part- vs. full-day programs, one year versus two years of exposure), and the availability and quality of other child care and preschool programs in the community.

Overview of Study Design

The Head Start Impact Study is a longitudinal study involving approximately 5,000-6,000 first time enrolled three and four year old children across an estimated 75 nationally representative grantee/delegate agencies (in communities where there are more eligible children and families than can be served by the program). The children participating will be randomly assigned to either a Head Start group or a control group. Every effort will be made to minimize the burden on individual programs and to avoid significantly changing typical enrollment and recruitment procedures.

As noted above, the primary purpose of this study is to determine whether Head Start has an impact on participating children, and if so, whether impacts vary as a function of the characteristics of children, their families, Head Start grantees/delegate agencies, and their communities. “Impact” for this study means the difference between outcomes observed for Head Start participants and what would have been observed for these same individuals had they not participated in Head Start.

The key question, then, is how to determine what outcomes would have been observed if the children had not participated in Head Start? In many studies, researchers have used a variety of non-randomized methods to construct a “participant- like” comparison group, but even the best such attempts have significant drawbacks primarily related to what evaluators call “selection bias,” i.e., the extent to which program participants are determined, or selected, by a process that makes them different from non-participants on factors that are often difficult to measure but that lead to different outcomes independently of the Head Start “treatment.”

To avoid such difficulties, Congress and the Advisory Committee on Head Start Research and Evaluation (1999) recommended the use of a randomized research design for the Head Start Impact Study. As such, the study will involve the selection of a sample of Head Start applicants who will then be randomly assigned to either a program group (these children and their families will receive Head Start services) or to a control group (these children will not be enrolled in Head Start, but may receive other available, non-Head Start services selected by their parents). Under this randomized design, a simple comparison of outcomes for the two groups yields an unbiased estimate of the impact of the treatment condition—for example, the effect of Head Start on children’s language and literacy readiness for school.

The advantage of this research design is that if random assignment is well-implemented and not severely compromised by either the individuals in the study or by grantee/delegate agency staff, program participants should not differ in any systematic or unmeasured way from non-participants. More precisely, although there may be differences between the treatment and control groups, the expected or average value of these differences is zero (i.e., selection bias is removed by random assignment).

The Advisory Committee also recommended a field test to gain additional knowledge before full implementation of the study. The current study includes such a field test, which began in the Spring of 2001. It will run somewhat parallel with the initial recruitment of sites for the full study (in which full implementation including recruitment and random assignment of the main sample of families will not occur until the summer of 2002). The timing of these early study activities will allow multiple opportunities to take advantage of knowledge gained from the field test (or from early site selection efforts for the main study) to modify or further refine the design of the full-scale study, as necessary. Further discussion of the field test plans, including some potential issues that might be encountered during implementation of the field test activities, and a summary of the overall study design are provided below. A detailed description of the design is available at http://www.acf.hhs.gov/programs/opre/hs/impact_study/index.html.

Sampling Plan

As outlined in the legislative mandate, the Head Start Impact Study is to provide “…a national analysis of the impact of Head Start” based on the selection of Head Start grantees/delegate agencies that “…operate in the 50 states, the Commonwealth of Puerto Rico, or the District of Columbia and that do not specifically target special populations. ”Furthermore, the Advisory Committee recommended that the sample of Head Start grantees/delegate agencies should reflect variation in a variety of characteristics including, “…region of the country, race/ethnicity/language status, urban/rural, and depth of poverty in communities,” and “…design of program as a one-year or two- year experience for children; program options (e.g., center-based, home-based, part-day, full-day); auspice (e.g., Community Action Agency, public school, non-profit organization); community-level resources; alternative child care options for low-income children; and, the nature of the child care market and the labor market in the community studied.” The sampling strategy begins with the legislative-mandated requirement, under the first study goal, to have a national impact estimate that captures the wide variety of grantees/delegate agencies operating in all 50 states, the District of Columbia, and the Commonwealth of Puerto Rico, and that do not specifically target special populations. In consideration of the final requirement, several initial exclusions have been incorporated into the design that can be defined at the outset of the sampling process1.

Based on the legislative language, children enrolled in Early Head Start, Migrant Head Start programs and those operated by Tribal organizations will not be included in the study. Additionally, the Advisory Committee recommended the exclusion of those programs considered extremely new (i.e., in operation approximately less than 2 years), and those considered severely out of compliance with the Head Start Program Performance Standards. Due to ethical concerns about the potential denial of services to families in locations with relatively few unserved, eligible families, the Advisory Committee also recommended the exclusion of programs in communities considered to be “saturated.” “Saturation” was used to refer to locations where the program already is serving most or all of the children in the community who are eligible for, and wish to attend Head Start, such that there are no additional unserved children within that community. The ethical concerns arise in saturated communities where there are not enough eligible children to randomly assign enough children to the treatment and control groups, thereby leaving unused, funded slots in the Head Start program.

Several sampling steps have been completed to date, and others will occur over the next year. Steps for sampling include:

The initial listing of the universe of Head Start grantees/delegate agencies2 was drawn from the 1999-2000 Program Information Report (PIR) database maintained by the Head Start Bureau within HHS. Based on the list of Head Start grantees/delegate agencies, all of the grantees/delegate agencies were then clustered into groups based on their geographic proximity to each other. Each county in the United States that contained one or more grantees (and, as a consequence, each Head Start program and participating child) was included in one of the created grantee clusters. This was done to ensure that each Head Start grantee/delegate agency and participant child has a known probability of selection into the study sample, and—at later points in the sampling process—that the probabilities will be approximately the same for each child.
The grantee clusters were then divided into a total of 25 strata, each stratum having approximately the same number of 3- and 4-year old Head Start children across the cluster. Four stratifiers were used (1) region, (2) urbanicity, (3) state policy regarding comprehensive pre-kindergarten programs, and (4) the racial/ethnic mix of the Head Start children served. One grantee cluster from each of the approximately 25 strata was selected with probability proportional to the total Head Start enrollment of 3- and 4-year olds in the grantee cluster.3
Further information was then obtained on all the grantee/delegate agencies within each of the 25 selected clusters to determine their eligibility. This information was used to select an average of three grantees/delegate agencies from within each of the 25-grantee clusters for a total of 75 grantees/delegate agencies for recruitment into the study. The grantees/delegate agencies were stratified prior to selection. The probability of being selected was proportional to the overall size of the grantee/delegate agency. To ensure that very small grantees were selected, they were paired with another grantee. Grantees/delegate agencies that are unable or unwilling to participate will be replaced only to maintain the sample sizes needed to support important sub-group analyses.
Considerable efforts have been undertaken to ensure maximum communication and coordination among Department staff, Regional office staff, the Impact Study research team and grantee staff as study implementation proceeds. Agency expectations for full cooperation and participation on the part of grantees also have been emphasized.
Once the selected grantees have been recruited and detailed information collected on all of their centers (on average, grantees operate about nine centers each), centers will be selected.4 The plan is to select an average sample of approximately three Head Start centers per grantee/delegate agency for a total of about 225 centers nationally. Centers will be selected with probabilities proportional to size (i.e., larger centers would have a greater chance of being selected for the study), and the planned sample of approximately 80 children per grantee/delegate agency (48 treatment and 32 control) would be distributed evenly across the selected centers. The 60:40 ratio of Head Start to control children in the sample was selected to maximize the efficiency of sampling, while at the same time minimizing the resulting loss of sampling precision due to unequal Head Start and control group sizes. Centers that are “saturated” (i.e., it would not be possible to find unserved children for a control group) would not be considered for inclusion in the study sample.
In the selected 225 Head Start centers, spread across the 75 study grantees/delegate agencies, an initial sample of approximately 3,137 newly entering 3- year-old participants and 2,541 newly entering 4-year-old participants will be selected for random assignment. The number of 3-year-olds is slightly higher than the number of 4-year-olds to account for attrition in the additional year between initial recruitment and the final data collection point in the Spring of first grade.

Field Test

As noted above, a field test will be conducted to test various measures and procedures, as well as to learn more about the best strategies to deal with the complexities that exist across different communities with respect to both Head Start programs and other available care options for low-income children. This field test will involve approximately 600 first time enrolled three and four year old preschool children across eight grantee/delegate agencies, and 24 Head Start centers. Critical lessons to be learned from the field test include:

The degree to which saturation may exist across different communities and the degree to which more recent changes in Head Start partnerships with other providers and other "blending" options have increased the number of saturated communities.
The factors that may affect grantee/delegate agency decisions to participate in the study, and how the contractor can work to maximize program cooperation and participation, without disruptions to program operations or undue burdens being placed on staff.
How the proposed random assignment procedures will work in a range of Head Start programs and communities, as well as how to minimize or eliminate any difficulties with implementing random assignment that may potentially compromise sample integrity and/or diminish data quality.
To what degree normal program recruitment strategies and program operations may be altered if the program has to recruit a small to modest number of additional families. In other words, does the additional recruitment effort result in a substantially different composition of children and families than those normally served?
The feasibility of proposed data collection procedures and instruments for gaining complete information about family demographics, staff characteristics, program quality, and child outcomes, particularly the most efficient ways of assessing children and interviewing families within the home environment.
What happens to children and families who are assigned to the control group and their utilization of a wide range of available care options in their communities. Little, if anything, is known about what families whose children currently do not get into Head Start do in terms of finding alternative arrangements for their child, and thus, what are the likely settings that control group children will experience. This information would better prepare the study staff for likely consequences (as well as points of difficulty in implementing random assignment at the program level).

The field test sample plan follows.

Three geographic clusters that are NOT part of the full-scale study sample of 25 clusters were selected for the field test. The clusters were purposively selected to have varied community contexts – one where most other program or child care options (beyond the available Head Start programs) are fairly comprehensive Head Start-like programs, one where there are program options which have some comprehensive program components, and one where there are few available options.
Within each cluster the number of Head Start grantees/delegate agencies and/or centers included in the field test provides variation across auspices, number of unserved children, local service richness, grantee size, ethnicity of population served and urbanicity. These characteristics are combined across eight grantees where no two are alike on all characteristics. Intentionally selecting grantees with a diverse set of characteristics allows the field test to explore the potential impediments to random assignment as well as to test study data collection procedures across a variety of settings.

In order to inform the plans for the full-scale study, data collection for the field test is scheduled for Fall 2001 and Spring 2002. Consequently, all field test sites have been recruited and application procedures and parental notification put in place. The actual Fall data collection began towards the end of September 2001.

Main Study Site Recruitment

The core of the approach in working with grantees/delegate agencies selected for the main study is to establish strong partnerships with the grantees/delegate agencies, actively address potential concerns, especially possible concerns of the Policy Councils (the local, shared governance body comprised of parents and community members), and work to mitigate Head Start staff and study participants’ concerns regarding issues such as random assignment, to the extent possible. Plans for the approach have been drawn from a proven set of tools and strategies and will build upon the valuable information that will be derived from the work in the field test sites. These include the following:

Provide early and ongoing communication and support from the Head Start Bureau and Regional Office staff.
Coordinate program recruitment efforts in close collaboration with regional office staff in order to minimize any potential confusion with the requirements of study participation and normal program compliance or monitoring requirements.
Obtain the support and endorsement of the National Head Start Association (NHSA).
Conduct sample selection and establish contact with selected grantees early so there is time for problem solving, mutual learning, and the essential give-and-take needed to arrive at a design acceptable to all.
Provide ample advance on-site time to explain the details of the study and reach a shared understanding with local program leaders about the study’s importance and the essential nature of random assignment.
Provide adequate resources to support the programs’ additional study-related activities and thereby minimize the burden placed on any program for participating in the study, as well as reduce the likelihood of any disruptions to the quality of program service delivery.
Respond to concerns about the denial of services resulting from the use of random assignment by: allowing staff to provide information to control group families on alternative services in the community; providing very limited exemptions from control group assignment for severely "at-risk," disadvantaged children meeting criteria specified in advance; and emphasizing the basic fairness of allocating Head Start’s limited capacity among many deserving families by using a lottery-like method of selection.
Emphasize the advantages of randomization from a program standpoint—its ability to show the value added of program services and point toward more effective service strategies for the future.
Hold down the number of control group cases sought from any given grantee, delegate agency, or Head Start center, so as to minimize the potential burden or disruption to normal program operations.
Make appropriate resources available to grantees having problems meeting the requirements of random assignment—to support presentations to community groups, scripts for dealing with control group families, and training and related support if expanded outreach is necessary.
Enlist peers as allies in the recruitment process—program administrators and policy makers, preferably from the Head Start community, with previous, positive experience participating in random assignment evaluations and the field test.

Selecting sites, establishing contact, starting program recruitment and developing a working partnership with all grantees/delegate agencies will be conducted early, at least six months before the earliest target date for the beginning of random assignment, and 12 months in advance wherever possible. It is anticipated that the recruitment process will entail ongoing contact with the sites via personal visits and telephone. Each recruitment effort must work through any potential concerns about participating in the study, develop individualized study plans with the grantees, and obtain information on the community context.

Local recruitment also includes gaining participation of organizations and individuals that provide child care and related services to children in the control group. Once the sample of children is selected, it will be incumbent upon the recruitment team to develop community plans for recruiting the various types of control group providers. Information obtained from the field test is expected to help with the development of a range of strategies and incentives used to recruit and retain the various types of non-Head Start providers. Similarly, the field test will help refine or develop strategies to meet the Advisory Committee's recommendation to "collect the same or comparable information on children in Head Start and control group or comparison children (e.g., services received; quality and intensity of the intervention; and cost, descriptive, and contextual information.)"

A further key to successful site recruitment is to ensure program administrators that the use of random assignment will not impose too many burdens on potential participant families, nor generate dissatisfaction in this vital client population. The largest step toward this goal is the decision to conduct random assignment only in Head Start grantees/delegate agencies operating at or near full capacity and where there is an additional pool of unserved families in the community known to be interested in services. This ensures that the evaluation will randomly assign children to the control group only in communities where Head Start grantees/delegate agencies currently cannot serve all eligible applicants due to limitations in the number of funded slots.

Data Collection

Data collection for the full-scale study will begin in Fall 2002 and extend through Spring 2006, as the children in the main sample will be followed through first grade. Data collection procedures for the Impact Study will build upon the set of instruments already developed for the Head Start Family and Child Experiences Survey (FACES) and will include the following components: child assessment, parent and center director interviews, surveys of teachers, family service workers, education coordinators and comparable staff in other child care settings, teacher/child care provider ratings of children, observations of the quality of different care settings, and record review.

The child assessment will be conducted individually in the Fall and Spring of the Head Start years and the Spring of kindergarten and first grade. The assessment battery is composed of a short series of tasks across a broad range of school readiness skills (e.g., early literacy, math, writing, small motor skills, counting, naming colors, book knowledge and vocabulary) that are feasible and interesting to preschoolers.
The various child, parent and teacher measures selected for inclusion in the study will be the most valid and reliable measures available. Also, wherever possible, measures will be used that allow for comparisons not only between children, families and care settings in the treatment and control groups, but also with data from other comparable studies and national norms, when available.
The parent/primary caregiver interview will be conducted one to two times during each year the child is in the study. Topics to be addressed will include: parental beliefs and attitudes towards their child’s learning, parental participation in and satisfaction with the program; family household and demographic information; and parent ratings of their child’s behavior problems, social skills, and competencies.
A center director interview will be conducted in the Spring of each Head Start year (2003 & 2004) focusing on the operation and overall quality of the program.
In the Spring of each Head Start year (2003 & 2004), teachers, family service workers, education coordinators and comparable staff in other child care settings will be asked to complete surveys and rating scales on a variety of topics including the program’s components, children’s experiences, behavior and development, family issues, and attitudes on working with and teaching children. In addition, once the children move into kindergarten and first grade, teachers and relevant child care providers also will be asked to complete similar surveys and rating scales on each of the sampled children concerning their continued experiences, development and behavior.
Observations of the quality of programs or care settings will be conducted each Spring during the preschool years. Observers will spend enough time in each class or child care setting to ensure observation of a major portion of the daily schedule and a variety of classroom, center or child care activities. Secondary sources will be used to assess quality at the kindergarten and first grade levels.
Children’s records will be reviewed each Spring to gain information on child attendance as well as other tracking information.

Local Study Staff

Both a site coordinator and an on-site liaison will be identified to assist in the study’s activities. The site coordinator, a contractor field supervisor, will serve a pivotal role as the primary local contact. Their responsibilities will include facilitating random assignment; enlisting cooperation and maintaining the participation of children, parents, and staff; coordinating all data collection activities; tracking study participants; and ensuring quality control. Grantee/delegate agencies will be asked to identify a staff person who will serve as the main point of contact for the study staff. This on-site liaison will be invaluable in helping the site coordinator obtain the cooperation of other staff; securing the trust and cooperation of parents and the community; making logistical arrangements for random assignment and data collection; and assisting with tracking study participants in both the Head Start and control groups.

Notification and Informed Consent

Parents will be informed of the study as part of the existing grantee/delegate agency’s recruitment and enrollment procedures. Along with their application, parents will receive a letter explaining that the program will not have space to admit all eligible applicants, and informing them that random selection will be used to select some eligible children. The letter also will notify them that the program will take part in the Head Start Impact Study and will include a brief overview of the sponsorship, goals and scope of the study.

A roster of eligible applicants will be compiled with the help of grantee/delegate agency staff and random assignment will be completed by late summer. After a list of children assigned to the Head Start and control groups is identified, the agency will notify both Head Start and control group families of their selection for potential participation in the study. A fact sheet will be provided to inform parents of the purpose of the research and describe the various data collection activities, including incentives. Parents will be told that unless they object by a specified date, someone from the study team will contact them to explain the study.

Subsequently, written informed consent will be obtained from the parents of all participating children prior to collecting any data. The informed consent letters will tell parents that their and their children’s participation is voluntary and will not affect the way the parent or child is treated by Head Start. The letters will state that parents may withdraw at any time and that all information collected during the study will be kept confidential using a variety of systems that have been approved by the contractor’s Institutional Review Board and the Office of Management and Budget.

Timeline

The following table summarizes the major milestones of the Head Start Impact Study:

First Interim Report to Congress	October, 1999
Competitive procurement published	May, 2000
Evaluation contract awarded	October, 2000
Field Test - sites selected	May, 2001
Main Study - sites selected	May, 2001
Field Test - Fall Head Start data collection	September, 2001
Second Report to Congress	December, 2001
Main Study - sites recruited	Fall, 2001
Main Study - first wave of Head Start data collection	Fall, 2002
Main Study - second wave of Head Start data collection	Spring, 2003
Third Report to Congress	September, 2003
Main Study - final wave of first grade data collection	Spring, 2006
Final Report to Congress	September, 2006

Summary

In summary, HHS is pleased with the progress of the Head Start Impact Study activities to date, including the support received from the Head Start community and the general cooperation of the programs that have been contacted. While, as noted above, it is anticipated that there will be significant challenges encountered during the implementation of this important study, every effort is being made to anticipate and proactively address such challenges. HHS remains committed to ensuring that this scientifically rigorous study remains on schedule and adequately addresses the full set of legislative requirements. The Department looks forward to reporting to the Congress on the continued progress of the study, as the work continues, and on results, as they become available.

1Several other exclusions are discussed below which cannot be easily defined in advance for the universe of Head Start grantees/delegate agencies and children.(back)

2Grantees that provide direct services to children are considered to be a separate “program” from any delegate agencies they may operate. Grantees that do not provide direct services are not included for sampling purposes.(back)

3Selecting grantee clusters proportional to size is done to give clusters within each strata with more Head Start children a higher chance of being selected. This will result in approximately equal sample size of Head Start children from each selected grantee cluster, as well as approximately equal probabilities of inclusion for all Head Start children in the nation. This can be best achieved if greater weight is given to larger clusters because they should represent a greater proportion of the overall average program impact than very small areas. To make up for this greater weight given to larger concentrations of Head Start children, the sampling rate for children will be set lower in larger clusters at a later step in the sampling. If larger areas were not given greater weight at the point of selecting clusters, the number of sampled children within each selected larger cluster would have to be very high.(back)

4The average number of centers per grantee/delegate agency is 9.4 and the median is 6.0. The range is from 1 to 166 centers.(back)

Return to Previous page