Table of Contents | Previous | Next |
3.0 Methodology
"Great program-the primary concern is the children
and this is evident in the decisions they make."
—Head Start Staff
3.1 Overview
Chapter 3 describes the data collection methodology, with primary focus on the sampling plan, weighting of the study sample to provide national estimates for Head Start, the development of the data collection instruments, and the data collection procedures employed with parents and staff at the study sites.
3.2 Description of Head Start Universe
Data on Head Start programs were obtained from the 1992-93 PIR, a self-completed report submitted by each program at the conclusion of the program year. Because this study was conducted during the 1993-94 academic year, the 1992-93 PIR was the most recent source of information on all Head Start programs available for use in constructing the sampling frame.(4) However, the 1993-94 PIR data best describe the universe of Head Start programs at the time of data collection for this study (April-June, 1994). Exhibit 3-1 compares information from the 1992-93 PIR and the 1993-94 PIR, and contains data specific to the 40 study programs. This table shows that the earlier data from 1992-93 were sufficiently comparable to the data from the 1993-94 study period to produce a valid sample for this study. Also, the selected sample of programs matched characteristics of the overall Head Start universe very well, with some minor exceptions. These differences were corrected through sample reduction, statistical weighting, and the sampling strategies.
|
Sample Programs** |
||||
|---|---|---|---|---|
|
1992-93 |
1993-94 |
1992-93 |
1993-94 |
|
|
Number of Programs*** |
1796 |
1834 |
40 |
40 |
|
Total Number of Children |
666,492 |
732,218 |
39,118 |
42,999 |
|
Proportion of Children |
Percent |
Percent |
Percent |
Percent |
|
4-Year-Old Children |
65.9 |
64.5 |
68.5 |
65.9 |
|
Children Enrolled in Second Year of Head Start |
19.5 |
19.2 |
17.8 |
18.4 |
|
Home-Based Children |
7.4 |
6.8 |
9.2 |
9.3 |
|
Families Enrolled in Medicaid/EPSDT**** |
67.3 |
69.4 |
66.0 |
70.4 |
| *PIR - Head Start Program Information Report. **Data for sample programs are based on all enrolled children, not just the respondents to this study. ***Excluding migrant programs and programs without 4-year-old children. ****EPSDT - Early and Periodic Screening, Diagnostic, and Treatment Program. |
||||
3.3 The Sampling Plan
The primary objective of providing a national probability sample of children enrolled in Head Start was accomplished through a multi-stage sampling strategy. The first-stage sample consisted of 40 Head Start programs selected from the universe of programs identified in the 1992-93 PIR database. The second stage of sampling yielded 80 Head Start centers (two centers per program). The final stage selected 15 children per center, to yield 1,200 interviews with parents, for a nationally representative sample of Head Start families. The details of each stage of sampling are described below.
The first-stage sample generated 40 randomly selected Head Start programs. The available programs were stratified on the basis of three variables-Census Region (Northeast, Midwest, South, and West), Urbanicity (whether or not the Zip Code associated with the address of the Head Start program was located inside an Urbanized Area),(5) and the percentage of minority Head Start children (greater than or equal to 50% minority enrollment versus less than 50% minority enrollment).
In the second stage of sampling, two Head Start centers were selected from each of the 40 programs. For several programs where centers selected had fewer than twenty 4-year-old children enrolled, an additional backup center was identified.
The third stage of sampling involved the selection of individual Head Start children. The target population consisted of 4-year-old children enrolled in Head Start at the beginning of the academic year. Using enrollment lists provided by the centers, the research staff identified a random sample of 15 primary children and 8 alternates per center. The data collection effort was directed at completing 15 interviews per center from 80 centers for a total of 1,200 parent interviews. In cases where the family of one of the 15 pre-selected children could not be located or if a parent declined to be interviewed, alternates were used. A total of 1,189 parents were actually interviewed.
3.4 Description of Study Sample
3.4.1 Head Start Children and Parents
|
[D] |
The national distribution of States covered by this sample of 40 Head Start programs is shown in Exhibit 3-2. All programs selected on the first draw agreed to participate, making it unnecessary to draw any replacement programs. A total of 81 centers were visited. One of the programs was entirely home-based, and consequently had only one center. In two other cases, a center was drawn that did not have the requisite number of 4-year-old children (15), and an additional center was drawn to complete the sample.
Descriptive statistics for the sample of children represented by the 1,189 parent interviews are found in Exhibit 3-3. Of these, 51% of the children whose parents were interviewed were male. The broad racial background of the children was assessed in the parent interview using the standard U.S. Bureau of the Census classification, included 6% of the sample identified as "Other," a category which was used by parents to record a multiracial or multiethnic background for their child.
|
Racial Background (Reported by parents) |
Unweighted |
Weighted |
|---|---|---|
|
Total |
1,189 |
100 |
|
American Indian |
36 |
4.2 |
|
Asian or Pacific Islander |
19 |
1.6 |
|
Black (not of Hispanic origin) |
399 |
32.8 |
|
Hispanic |
222 |
18.2 |
|
White (not of Hispanic origin) |
438 |
37.5 |
|
Mixed Race |
74 |
5.7 |
|
Missing |
1 |
|
|
Gender (Recorded from child health files) |
Unweighted |
Weighted |
|
Female |
602 |
50.9 |
|
Male |
582 |
49.1 |
|
Missing |
5 |
|
|
Primary Language Spoken at Home* |
Unweighted |
Weighted |
|
English |
1,029 |
86.9 |
|
Spanish |
224 |
18.1 |
|
Other |
47 |
4.7 |
|
*Some families indicated more than one language. |
||
3.4.2 Head Start Staff
The final totals for completed staff interviews are shown in Exhibit 3-4. All totals do not equal 40, usually because staff were not available (e.g., on leave) or a staff position was not filled at that time. Some programs had more than one individual filling the Health Coordinator and Parent Involvement Coordinator roles, resulting in totals greater than 40.
While 81 centers were visited, only 59 Center Director reports were available. Occasionally a Center Director was not available, but more often this individual was responsible for more than one center within a program, not an uncommon situation. No staff member refused to be interviewed for this study.
|
Staff Position |
Number of Interviews |
|---|---|
|
Center Director |
59 |
|
Health Coordinator |
42 |
|
Mental Health Coordinator |
37 |
|
Nutrition Coordinator |
39 |
|
Parent Involvement Coordinator |
42 |
| Note: Individuals could be counted more than once due to multiple roles. | |
3.5 Data Collection Instruments
3.5.1 Data Sources
The research staff developed data collection measures for nine data at both the program level or the center level, depending on the type of information needed and the structure of each specific program. In addition, each program office provided documentation concerning the operation of the Health Component. The primary data sources were as follows:
- Center Director/Lead Teacher (interview)
- Health Coordinator (interview);
- Nutrition Coordinator (interview);
- Mental Health Coordinator (interview);
- Parent Involvement Coordinator (interview);
- Budget Manager (questionnaire);
- Parents of 4-year-old children (15 per center) (interview);
- Child health files for the children of the interviewed parents (record abstraction); and
- Meal observations at each center (observation form).
Interview forms and other data collection instruments for these primary data sources are found in Appendix B of Volume IV, including Spanish versions of all the interview forms.
3.5.2 Instrument Development
Where appropriate, specific interview questions were consistent with current Head Start protocols. The instrument developed for abstracting data from the child health records was based on the information detailed in the 1992 version of the Head Start Child Health Record. A Technical Advisory Panel, consisting of five consultants across the four health domains, four representatives from relevant Federal agencies, a local Head Start Health Coordinator, and a Head Start parent, also contributed to the development of the research instruments as well as the data collection procedures.
3.6 Staffing
CDM and Abt organized a three member site visit team for each program. Each site visit team was led by a research associate from CDM or Abt and including a trained, experienced data collector and an on-site staffer from the Head Start program being visited. The research associate spent one week on-site, and the data collector was scheduled for a two week visit, although in most cases all data were collected in the first week. The on-site staffer helped to recruit parents and schedule interviews. More detailed descriptions are provided below.
- The research associate, the senior member of the site visit team, had primary responsibility for the data collection effort at the site.
- The data collector, often recruited from the same community as the Head Start program, interviewed parents, reviewed the child health files, and assisted the research associate, as necessary.
- The on-site staffer was typically recommended by the Program Director to assist the other members of the site visit team. The on-site staffer recruited parents into the study, coordinated the distribution and collection of consent forms, scheduled interviews, and explored potential parent recruitment and scheduling strategies.
3.7 Data Collection Procedures
The procedures used by CDM and Abt to complete the collection of data are described below.
Following the selection of the final sample of programs, the research staff notified the appropriate Administration for Children and Families (ACF) Regional Offices about the study and identified the programs that were selected to participate within their respective Regions. A member of the research staff then telephoned the Program Director at each study site and also sent a follow-up letter. The names and sizes of the centers operating within that program were identified, as were the names and titles of the primary staff working in the Health Component. Centers were selected using a PPS strategy, and the Center Directors/Lead Teachers at the selected centers then faxed rosters of the eligible 4-year-old children to the research office.
Prior to the site visit, the on-site staffer received the list of the selected children and alternates, and implemented procedures for recruiting the sample. The on-site staffer distributed and collected the Parent Consent forms and arranged the parent interview schedule. The researchers obtained signed, informed consent forms from all parents prior to the latter's participation in the interviews and prior to the record reviews. Copies of the consent forms were provided to the local Head Start centers.
For both the program staff and parents, all information that could link individuals with their responses was kept under lock and key by the CDM Project Director at the research office. All information that might link respondents to the data instruments was removed from the forms.
The principal reason for the high response rate (78% of parents identified in the primary sample lists) was the cooperation of the Head Start parents and staff members. To facilitate the interviews, the research associates and data collectors were available to conduct interviews after normal working hours or on weekends for respondents who were unavailable during the work day. Whenever interviews could not be conducted in English, they were completed in the native language of the respondent. To maximize comfort with the interview setting, the research staff gave each parent the opportunity to determine the site for the interview, although parents were encouraged to come to the Head Start center.
3.8 Record Keeping Issues
Because the site visit teams anticipated that the Head Start Child Health Record would be employed for the great majority of children sampled, the record review instrument was based on that form. However, only 58.2% of the records reviewed were actually maintained on the Head Start Child Health Record and these were not always the most recent version. The remainder of the health records were kept on locally developed forms that did not always provide the information sought on the record abstraction form. This created instances of missing data for certain health conditions or health status reports.
Often, child health records were not helpful in determining the assessment, prevalence, and treatment of conditions that fall under the mental health and dental domains. For example, mental health reports were often included in the child's education file, and the site visit teams did not always have access to these. Also, mental health evaluations and treatment reports were not always maintained in a child's file because of confidentiality issues. Similarly, approximately 45.0% of the children did not have a record of having previously seen a dentist. This does not mean that dental visits were not completed, but rather that this information was not located in the health records. The data collectors made every effort to complete the record reviews, but some results must be interpreted with some caution.
3.9 Data Analysis
This study uses data at two levels. The basic database was constructed at the child level for analyzing data from parent interviews and reviews of the Head Start child health records. These analyses apply to data on immunizations, health conditions, health screenings, exams, referrals, and services provided to the target children. Data at this level are weighted to produce national Head Start estimates. The second level of data is the Head Start program, where staff interviews yielded detailed information about health service delivery, about the barriers and difficulties faced by centers in the provision of required health services, and about staffing of the Health Component.
Wherever possible, staff attempted to collect comments from respondents (staff and parents). These were transcribed and coded for use in quantitative analyses. The transcription of the comments are included in Volume III. As part of the routine of data analysis, categorical and ordinal data were compared across the three stratification variables (urbanicity, geographic region, and above or below 50% minority enrollment). These findings are only reported in the text in cases where meaningful differences are noted across the sub-groups.
4. Since only 4-year-old children were targeted in the study, the universe of Head Start programs included programs operating during the 1992-93 school year that also enrolled 4-year-old children during that year. Migrant programs were excluded because they were participating in a separate study in which the Health Component was one of the primary interests. (back)
5. During the process of selecting centers for the study, it became apparent that Head Start programs often manage centers located at substantial distances from the main program address. In such cases, the program address is often in an Urbanized Area while most or all of the program centers are located in rural areas surrounding the main program location. Occasionally, the reverse was true as well. Therefore, any comparisons of urban and rural data for this study should be viewed with considerable caution before any conclusions are drawn. (back)
| Table of Contents | Previous | Next |