Table of Contents | Previous | Next |
Chapter A5 : Results of the Meta-Analysis
In this chapter, we present the results of the meta-analysis in terms of the two major study questions: what are the effects of family support programs and services? and what characteristics of programs or the families they serve account for differences in effects?
What are the Short-Term Effects of Family Support Programs and Services on Children?
The broadest of the study's research questions asked whether family support programs and services produce better outcomes for children and families. Improved outcomes for children are, in a sense, the ultimate goal of all family support programs. The population they serve consists of families with children; their services, whether focused on adults or on children or on the family unit, are intended to promote children's wellbeing. As we saw in Chapter A-3, most of the programs included in the meta-analysis focused on families with children who face either environmental risks (most commonly poverty), biological risks, or a combination of both. Family support is widely viewed as a strategy for helping children overcome these risks and achieve their full potential.
At the same time, rather than intervening directly with children, most of these family support programs worked primarily with parents, perceiving them as the agents of positive outcomes for children. While less than 20 percent of the programs provided early childhood education services directly to children, all but one or two provided parenting education, more than half offered social support services and more than 40 percent provided case management services, counseling and/or referral to services. This emphasis on parents reflects an important tenet of family support, articulated by Bernice Weissbourd in 1994, namely that "the capacity of parents to raise their children effectively is influenced by their own development" (Kagan & Weissbourd, 1994, p.32). In the same volume, Sharon Lynn Kagan sets forth guidelines for judging the quality of family support programs that include the following:
- Programs recognize the importance of parental nurturing and seek to enhance parents' capacity for growth and development: and
- Programs understand that support can strengthen family coping capacities and strive to foster independence and empowerment. (Ibid, p.379)
In recognition of the dual emphasis of family support programs, our analyses examined an array of outcomes for both children and parents. Child outcomes were grouped in four categories: cognitive development and school performance, social and emotional development, health, and safety (injury, abuse, neglect). Parent outcomes were grouped in five categories: parent attitudes and knowledge, parenting behavior, family functioning; parental mental health and health risk behaviors; and economic well-being. Although no study reported outcomes in all nine domains, most reported outcomes in at least one child and one parent outcome domain.
The results of the meta-analysis are presented below in terms of these nine outcome domains. Because a majority of studies reported outcomes at the end of the program only, this section focuses on the short-term effects of family support programs. A brief section at the end of the chapter discusses findings from the minority of studies that reported effects after follow-up periods of varying length.
Summary of Findings
Family support programs included in the meta-analysis, which represent a broad cross-section of programs that provide family support services, have small but statistically-significant average effects in all nine outcome domains.
Small positive effects are found in the following areas:
- Children's cognitive development
- Children's social and emotional development
- Parenting attitudes and knowledge, parenting behavior, and family functioning
Statistically significant effects in favor of family support programs are found in the following areas, but the practical meaning of these effects is less certain:
- Children's physical heath and development
- Children's safety
- Parents' mental health or risk behaviors
- Producing change in families' economic self-sufficiency
It is important to note that, in every outcome domain, a small group of programs accounted for the average effect; that is, in each outcome domain, more than half of the studies reported an effect size that was smaller than .20. Analysis of variation in program effects showed that:
- Programs that focus on children with special needs have larger effects on children's cognitive outcomes.
- Programs that provide early childhood education directly to children have larger effects on children's cognitive outcomes.
- Programs that provide parents with opportunities for peer support have larger effects on children's cognitive outcomes; programs that use home visiting as a primary intervention have weaker effects on children's cognitive outcomes.
- Programs for families with children with developmental delays or behavioral problems that use professional staff to work with parents in group settings rather than through home visits have greater effects on children's social-emotional development.
- Programs for children at biological risk are least effective in producing positive change in children's health and physical development.
- Programs that target teenage parents with young children and combine case management with parent-child activities are more effective in protecting children from accidental injury, abuse or neglect.
- Programs that use professional staff to help parents to be effective adults, and that provide opportunities for parents to meet in support groups, are more effective in producing positive outcomes for parents.
- Programs that work with parents of children with special needs, and provide opportunities for peer support, have greater effects on parents' attitudes towards and knowledge of childrearing and child development.
Overall Effects
Family support programs have small but statistically-significant effects in all outcome domains. These effects are evident when only randomized studies were included in the analysis and when both quasi-experimental and randomized studies are included (Exhibit A5.1). However, although each of the average effect sizes is statistically greater than zero (as shown by the significance of the T-tests in Exhibit A5.2), it is difficult to conclude that all of the effects should be interpreted as being educationally or psychologically meaningful. The convention in the social sciences is that effect sizes below .20 are not "educationally meaningful" (Cohen, 1986). Effect sizes between .2 and .5 are considered small and potentially meaningful; effect sizes between .5 and .8 represent moderate effects; and only effect sizes larger than .8 are considered "large." In three of the domains-child cognitive achievement, child social and emotional functioning, and parenting behavior-the effects are consistently meaningful (.2 or above in the two samples of randomized studies alone and combined with quasi-experimental studies) albeit small. It should be pointed out, however, that the "meaningfulness" of effect sizes is not always clear, and must be considered in the context of evidence about (a) their potential importance for a given population and (b) the cost of obtaining the effect.
Child Outcomes
Cognitive Development and School Performance
Family support programs, generally and in this sample, provide services to families with children ranging in age from newborns to children in high school. Not surprisingly, in the studies of these programs, a wide variety of measures are used to assess children's cognitive outcomes including tests of cognitive and language development, aptitude tests, tests of developmental progress, school achievement and performance measures (grades, use of special education services, retention in grade, promotion, graduation). In our analysis, the effect sizes for all these different measures are grouped under the rubric of "cognitive outcomes." However, since the majority of cognitive outcomes reported in the studies in the meta-analysis are for children under five years of age, most of the measures grouped under "cognitive outcomes" are assessments of children's cognitive and language functioning rather than school-related performance measures.
Family support programs have a small but positive effect on children's cognitive outcomes. The effect size for the experimental and quasi-experimental studies combined is .29; when only experimental studies are included in the analysis, the effect is somewhat smaller-only .25. An effect size of .25 translates into a difference of four points on a typical standardized test of cognitive functioning with a mean of 100 and a standard deviation of 15; that is, children whose families received the family support services would score, on average, four points higher than children not in the program.
![[Exhibit A5.1: Average Adjusted Effect Sizesa, b in Nine Outcome Domain]](images/exhib5_1.gif)
| Child Cognitive Development | Child Social-Emotional Development | Child Physical Health & Development | Child Injury, Abuse, Neglect | Parenting Attitudes & Knowledge | Parenting Behavior | Family Functioning/ Family Resources | Parent Mental Health/ Health Risks | Family Economic Self-Sufficiency | |
|---|---|---|---|---|---|---|---|---|---|
| Results | |||||||||
|
Randomized Studies Only
|
|||||||||
|
n of studies
|
110 | 96 | 68 | 22 | 71 | 118 | 18 | 98 | 71 |
|
Average effect sizea, b
|
0.253 | 0.258 | 0.091 | .134d | 0.182 | 0.246 | .284d | 0.093 | 0.095 |
|
Standard error
|
0.031 | 0.021 | 0.024 | 0.064 | 0.027 | 0.031 | 0.092 | 0.024 | 0.02 |
|
95% confidence interval
|
(.192, .314)*** | (.203, .313)*** | (.046, .136)*** | (.010, .260)* | (.125, .239)*** | (.186, .306)*** | (.104, .464)** | (.042, .144)** | (.046, .144)*** |
|
Homogeneity of variance test c
|
P2 = 207.2*** | P2 = 101.8*** | P2 = 102.0** | P2 = 131.9*** | P2 = 89.2* | P2 = 254.3*** | P2 = 81.7*** | P2 = 162.2** | P2 = 64.6 |
|
Randomized & Quasi- Experimental Studies
|
|||||||||
|
n of studies
|
187 | 139 | 99 | 38 | 108 | 166 | 32 | 144 | 91 |
|
Average effect sizea, b
|
0.293 | 0.223 | 0.123 | .213d | 0.23 | 0.257 | .169d | 0.137 | 0.099 |
|
Standard error
|
0.025 | 0.025 | 0.024 | 0.068 | 0.026 | 0.026 | 0.052 | 0.023 | 0.022 |
|
95% confidence interval
|
(.244, .342)*** | (.175, .271)*** | (.075, .170)*** | (.080,.346)** | (.179,.281)*** | (.206, .308)*** | (.067, .271)** | (.092,.182)*** | (.056,.142)*** |
|
Homogeneity of variance test c
|
P2 = 384.5*** | P2 = 198.8*** | P2 = 199.9*** | P2 =213.2*** | P2 = 141.9*** | P2 = 344.4*** | P2 = 72.0*** | P2 = 270.6*** | P2 = 95.3 |
|
w p < .05 ii p < .01 iii p < .001 a Means adjusted for within-study variation associated with (1)
number of effect sizes in average, (2) sample size for each effect
size, (3) types of measures in effect sizes, (4) accuracy of methods
used to compute effect sizes, (5) whether study was published, and
(6) whether the study was a randomized or quasi-experimental design. |
|||||||||
This effect is an average for children of all ages. The average effect of family support programs is greater when only outcomes for young children (under 5 years of age) are considered. For preschool children, the average effect is .39 across all studies and .35 for randomized studies only. This is similar to the size of the effect on cognitive development reported for the children in the Perry Preschool study at the end of the preschool year. (Weikart, Bond, & McNeil, 1978).
Child Social and Emotional Outcomes
Children's social-emotional development is an important component of their readiness for school and also plays an ongoing role in their ability to function successfully throughout their school years. In addition, many programs focused on families with children whose development in this area might be compromised, e.g., families at risk for abuse or neglect or families with a child with special needs or with behavior problems. Measures of social-emotional development reported in the studies include: social skills; behavior problems; emotional stability; school behavior; and delinquency.
Family support programs have a small average effect on social-emotional outcomes. The average effect size is .22 across all studies combined and .26 for the randomized studies analyzed separately. On a well-known rating of children's social and emotional problems, the Achenbach Child Behavior Checklist, on the scale for externalizing (or acting-out problems), an effect size of .26 translates into a difference of three points on a 100- point scale, which could be large enough to make a meaningful difference in whether a child is judged to have a clinical level of problem behavior.
Children's Health and Physical Development
Children's healthy physical development is frequently a goal of family support programs. Some programs have it as a primary goal, as in the case of programs for pregnant women aimed at reducing the incidence of low-birth-weight babies and other birth complications. Other programs are concerned with children's health as part of a more global concern with promoting children's well-being. In either case, improvement in children's health and physical development is addressed primarily through (a) parent education about the importance of preventive health care for their children, good nutritional practices, home safety, etc.; or (b) referrals to medical care. A few programs offer medical care directly in the form of checkups, developmental screening and other preventive care. A wide variety of health outcomes are reported in these studies, including growth indicators (height, weight), health (illnesses, hospitalization); and diet and nutrition.
Family support programs have no meaningful effects on children's physical health and development. The average effect size is .09 for the randomized studies and .12 when the randomized and quasi-experimental studies are combined.
Child Safety
Although none of the programs in this sample are treatment programs for parents identified as having abused or neglected their children, a number of the programs are funded as child abuse and neglect prevention programs because they work with families defined as at-risk for abuse and neglect for a variety of reasons. These reasons include the presence of relatively specific risks such as evidence of inadequate or dysfunctional parenting practices, and more general risk factors such as poverty, teen parenthood, low level of parent education, isolation, and lack of social support. Despite the fact that many programs have as a goal reducing the likelihood that parents will abuse or neglect their children, most do not in fact measure the incidence of abuse or neglect.1 Instead, the programs rely on measures of parenting behavior. Therefore, we have only a small number of studies with direct measures of child safety. These measures include reports of abuse or neglect, child removal for abuse or neglect, as well as reports of accidents, injuries and ingestions and hospitalizations due to these.
On average, the programs have no consistently meaningful effects on children's safety-.13 for the randomized studies and .21 for the randomized and quasi-experimental studies combined.
Parent Outcomes
Parenting Attitudes and Knowledge
The average effect of family support programs on parent knowledge and attitudes is small, about a fifth of a standard deviation. The average effect size is .23 when the randomized and quasi-experimental studies were combined, and smaller-.18-when only experimental studies were included in the analysis. Both of these effect sizes are on the edge of being considered educationally meaningful, according to convention. It is not clear whether a difference of this size represents a change that is large enough to have the effect on children's well-being that it is ultimately intended to bring about.
Parenting Behavior
The average effect of family support programs on parenting behavior is also small-about a quarter of a standard deviation. The average effect size is .25 for the randomized studies alone and .26 for the randomized and quasi-experimental studies combined. It is difficult to assess whether an effect of this size would be meaningful on most measures of parenting. To try to understand its implications, we looked at observational data on parenting behavior taken from a large evaluation study of low-income families. The NCAST Teaching Scale is based on an observation of mothers teaching their children a simple task. In this study, the mean score for parenting behavior was 40.2, with a standard deviation of 5.6.2 An effect of .25 would translate into a difference of about one and a half points on the scale.
Family Functioning/Family Resources
A relatively small number of studies report measures of family functioning, limiting our power to detect relationships between effects and program characteristics. The average effect of family support programs on family functioning is more than a quarter of a standard deviation among the randomized studies. This effect size of .28 is considered educationally meaningful. However, the average effect size is much smaller for the randomized and quasi-experimental studies combined-.17.
Parent Mental Health and Health Risk Behaviors
The efforts made by many family support programs to provide or strengthen parents' social support networks are intended to reduce feelings of loneliness and isolation and the depression that can accompany them and help parents to cope better with their lives. Other programs use a home visitor to accomplish some of the same ends. Improved coping skills and heightened self-esteem may result in reduced health risk behaviors such as smoking or substance abuse. Measures of all these types of outcomes are grouped together here.
Family support programs have no meaningful effect on parents' mental health. The average effect is .14 standard deviation for randomized and quasi-experimental studies combined, and even smaller for the randomized studies alone-.09 standard deviation. These effect sizes are so small as to call into question whether the programs are likely to make a meaningful difference on a measure of mental health outcomes.
Family Economic Self-sufficiency
Only a few of the programs included in the study are job-training programs but, as family support programs increasingly targeted low-income families, the goal of helping families improve their life circumstances through continued education, job training or employment became more important and studies began to measure and report on educational and economic outcomes. Grouped together in this category are measures of educational progress and achievement, job training, employment and income. Family support programs have very little effect on parents' economic well-being. The average effect size is 10 percent of a standard deviation. The average effect size is similarly small for randomized studies alone and when combined with quasi-experimental studies.
Variation in Effects
While the average effect size gives us a summary statistic about the effects of family support programs, it does not tell us about the amount of variation in effects across studies. The range in effect sizes in the nine domains is shown in Exhibit A5.3 (child outcomes) and Exhibit A5.4 (parent outcomes). In each of the domains, effects are unevenly distributed. Between 50 and 60 percent of the studies report effect sizes of between -.19 and .19. The question we are interested in answering is whether there are systematic differences between the programs with positive effects and the programs that have essentially no meaningful effects. That is, are there patterns in the array of findings that point to features of programs that predict stronger effects? The analyses that were designed to answer these questions are discussed in the section that follows.
It is important to note that these analyses of the effects of different program features were constrained by an important characteristic of the data. Despite the fact that there is substantial variation in effects across the studies in most of the domains, most of the variation in effects is not related to variation in program characteristics. On the contrary, most of the variation in effects is related to sampling error, or variation within studies. That is, while the expected value of the study's effect size is some common "true" effect, the estimates of that effect vary from study to study because of sampling error. In any study, the estimated treatment effect is a combination of the effectiveness of the intervention as well as any chance differences between the two groups. We tried to account for some of this within-study variation by including the "control variables" in each of our analyses. Once these sources of variation are accounted for, there is not much "true" variation left to be attributed to aspects of the intervention. This means that we cannot expect to identify program characteristics that explain a large portion of the variation in effect sizes. In fact, in one of the outcome domains--family economic self-sufficiency-we could not conduct the relational analyses because there was not sufficient variation left to predict after accounting for the control variables (see the results of homogeneity of variance tests in Exhibit A5.2).
![[Exhibit A5.3: Distribution of Average Effect Sizes on Children's Outcomes: Cognitive Development/School Performance]](images/exhib5_3a.gif)
|
![[Exhibit A5.3: Distribution of Average Effect Sizes on Children's Outcomes: Social-Emotional Functioning]](images/exhib5_3c.gif) |
![[Exhibit A5.3: Distribution of Average Effect Sizes on Children's Outcomes: Physical Health and Development]](images/exhib5_3b.gif)
|
![[Exhibit A5.3: Distribution of Average Effect Sizes on Children's Outcomes: Injury, Abuse, Neglect]](images/exhib5_3d.gif)
|
![[randomized studies only | quasi-experimental studies]](images/exhib5_key.gif){kind=small} |
|
![[Exhibit A5.4: Distribution of Average Effect Sizes on Parenting Outcomes: Parenting Behavior]](images/exhib5_4a.gif)
|
![[Exhibit A5.4: Distribution of Average Effect Sizes on Parenting Outcomes: Parent Attitudes/Knowledge]](images/exhib5_4b.gif)
|
![[Exhibit A5.4: Distribution of Average Effect Sizes on Parenting Outcomes: Family Functioning]](images/exhib5_4c.gif)
|
![[Exhibit A5.4: Distribution of Average Effect Sizes on Parenting Outcomes: Parent Mental Health/Health Risks]](images/exhib5_4d.gif) |
|
|
![[Exhibit A5.4: Distribution of Average Effect Sizes on Parenting Outcomes: Economic Self-Sufficiency]](images/exhib5_4e.gif)
Which Program Characteristics are Related to Differential Effects of Family Support Programs and Services?
Analyses were conducted to assess the strength of the relationship between individual programmatic, treatment and population factors and the magnitude of effects on outcomes. Exhibits A5.5 (randomized studies alone) and Exhibit A5.6 (for randomized and quasi-experimental studies combined) summarize the results of the analyses that individually tested each variable, i.e., each program, treatment or population characteristic, as a predictor of the size of a program's effect in each outcome. Once the univariate analyses were completed, multivariate analyses were used to determine the combined effects of different combinations of program characteristics.
In this discussion, it is important to remember two things about the database. First, program findings can be represented in more than one outcome domain. Although we describe findings in nine different outcome domains, we are really talking about nine overlapping, not independent, samples of programs. Therefore, findings from the separate analyses across the domains must not be interpreted as independent. Second, program characteristics are not themselves independent. That is, some characteristics occur together, at least in the set of programs in this meta-analysis, which makes it harder to disentangle the unique relationship of individual characteristics to effects.
Because sample sizes within domains are sometimes relatively small, because we try to account for as many control variables as possible before estimating the effects of program predictors, and because we know that the total amount of variation we have to try to predict is small, the models we create are not complicated. Usually we can test the effects of only one or two program predictors simultaneously.
Child Outcomes
There are a number of significant relationships between program features and effects on children that are consistent across the two samples of studies (randomized studies alone and combined with quasi-experimental studies). Two program features stand out as being associated with stronger outcomes for children:
- Targeting special needs children: Programs that target children with special biological or developmental needs have stronger effects on children's cognitive and social-emotional outcomes.
- Home visiting: Programs that use home visiting as a primary method of working with parents have smaller effects on child outcomes.
The findings on types of program activities linked to effects are mixed, with different findings in different outcome domains. Early childhood education services are strongly linked to cognitive outcomes for children but not to other outcomes. The amount and intensity of services are not related to any of the child outcomes.
Cognitive Development and School Performance
Four program characteristics are associated with the size of the effect on children's cognitive outcomes: whether the program targets children at risk because of biological, developmental or behavioral problems; whether the program provides some direct early childhood education to children; whether the program relies on parent groups or home visits for delivering parent education; and whether the program provides opportunities for peer social support among the parents (Exhibit A5.5).
As Exhibit A5.7 shows, the differences in average effect sizes associated with these four program characteristics are quite large.
Programs that target children with physical or developmental disabilities, or children who are at risk because of low birth weight, have larger effects on cognitive development than other programs. Although most of the other programs also target at-risk children, they usually define risk in terms of environmental conditions associated with poverty and/or inadequate parenting rather than in terms of demonstrated biological, health or developmental problems. Programs that serve children at biological risk have higher average effect sizes than other programs, and the difference was large-2.5 standard deviations difference (Exhibit A5.7).
The finding that programs that provide early childhood education have, on average, larger effects on children's cognitive development is not, by itself, a surprising finding. It simply replicates three decades of research on the effects of early childhood education. Among the randomized studies, the average effect size for programs with early childhood education services is substantially larger than the effect size for other programs.
| Program Characteristic |
Child |
Child |
Child n=68 |
Child n=22 |
Parent |
Parent n=118 |
Family n=18 |
Parent n=98 |
Econ n=71 |
|---|---|---|---|---|---|---|---|---|---|
|
Primary Program Goals
|
|||||||||
|
Social support
|
insufficient
variation in effect sizes to test predictors |
insufficient variation in effect sizes to test predictors |
|||||||
|
Parent self-help/self development
|
+ * | ||||||||
|
Prevention of child abuse/neglect
|
+ * | ||||||||
|
Economic self sufficiency/literary
|
+ * | + * | |||||||
|
Child mental health/behavior
|
|||||||||
|
Community participation
|
|||||||||
|
Targeting
|
|||||||||
|
Universal
|
|||||||||
|
Biological/developmental child risk
|
+ *** | - * | |||||||
|
Biological risks |
+ ** | - * | +* | ||||||
|
Developmental risks
|
+ µ | - ** | |||||||
|
Teenage parents
|
|||||||||
|
Population served
|
|||||||||
|
Majority families low-income
|
|||||||||
|
% Minority families
|
|||||||||
|
Majority teenage parents
|
+ * | ||||||||
|
Child age: infant toddler vs. older
|
- * | ||||||||
|
Child age at end of services (months)
|
- * | - *** | |||||||
|
Parenting education
|
|||||||||
|
Staff qualifications: all professionals
|
- * | + µ | |||||||
|
Home visits (vs. parent groups)
|
- ** | ||||||||
|
Services provided: Types of services
|
|||||||||
|
Intended length of services
|
|||||||||
|
Any early childhood education
|
+ ** | ||||||||
|
Any parent/child activities
|
|||||||||
|
Peer support activities
|
+ * | - * | + * | - µ | |||||
|
Any adult education activities
|
- | ||||||||
|
Any case management
|
+ * | - * | - * | ||||||
|
Collaboration with other agencies
|
- µ | ||||||||
|
Any health services
|
- * | ||||||||
|
Any community advocacy activities
|
- * | ||||||||
|
Services provided: Amount of services
|
|||||||||
|
Months services provided |
|||||||||
|
Hours of Early childhood education
|
|||||||||
|
Intensity of early childhood education
|
|||||||||
|
Amount (hrs) of parent education
|
|||||||||
|
Intensity of parent education
|
|||||||||
|
Amount (hrs) of case management
|
|||||||||
|
Intensity of case management
|
|||||||||
| µ p<.10 * p<.05 ** p<.01 *** p<.001 a Effect of each characteristic based on regression analysis (hierarchical linear modeling approach) with control variables accounted for. | |||||||||
| Program Characteristic |
Child |
Child |
Child n=99 |
Child n=38 |
Parent n=108 |
Parent n=166 |
Family n=32 |
Parent n=144 |
Econ n=91 |
|---|---|---|---|---|---|---|---|---|---|
|
Primary Program Goals
|
|||||||||
|
Social support
|
insufficient variation in effect sizes to test predictors |
||||||||
|
Parent self-help/self development
|
+ ** | + * | + ** | ||||||
|
Prevention of child abuse/neglect
|
+ * | + [] | |||||||
|
Economic self sufficiency/literary
|
- [] | - ** | |||||||
|
Child mental health/behavior
|
+ * | ||||||||
|
Community participation
|
|||||||||
|
Targeting
|
|||||||||
|
Universal
|
+ * | ||||||||
|
Biological/developmental child risk
|
- * | ||||||||
|
Biological risks |
- ** | - [] | |||||||
|
Developmental risks
|
+ * | ||||||||
|
Population served
|
|||||||||
|
Majority families low-income
|
- * | ||||||||
|
% Minority families
|
|||||||||
|
Majority teenage parents
|
+ ** | ||||||||
|
Child age: infant toddler vs. older
|
|||||||||
|
Child age at end of services (months)
|
+ ** | ||||||||
|
Parenting education
|
|||||||||
|
Staff qualifications: all professionals
|
+ ** | + * | |||||||
|
Home visits (vs. parent groups)
|
- ** | - [] | |||||||
|
Services provided: Types of services
|
|||||||||
|
Intended length of services
|
|||||||||
|
Any early childhood education
|
|||||||||
|
Any parent/child activities
|
+ * | ||||||||
|
Peer support activities
|
+ * | + * | |||||||
|
Any adult education activities
|
- [] | - ** | |||||||
|
Any case management
|
- ** | + * | - * | ||||||
|
Collaboration with other agencies
|
|||||||||
|
Any health services
|
|||||||||
|
Any community advocacy activities
|
+ [] | - * | |||||||
|
Services provided: Amount of services
|
|||||||||
|
Months services provided |
|||||||||
|
Hours of Early childhood education
|
|||||||||
|
Intensity of early childhood education
|
|||||||||
|
Amount (hrs) of parent education
|
|||||||||
|
Intensity of parent education
|
|||||||||
|
Amount (hrs) of case management
|
- ** | ||||||||
|
Intensity of case management
|
|||||||||
| µ p<.10 * p<.05 ** p<.01 *** p<.001 a Effect of each characteristic based on regression analysis (hierarchical linear modeling approach) with control variables accounted for. | |||||||||
| Program Characteristic | Present | Absent | Effect Size of Differencea |
|---|---|---|---|
|
Early childhood education
|
0.48 | 0.25 | 2.1 s.d. |
|
Targeted to special needs children
|
0.54 | 0.26 | 2.5 s.d. |
|
Peer support opportunities for parents
|
0.4 | 0.25 | 0.9 s.d. |
|
Home visiting (vs parent groups)
|
0.26 | 0.49 | 1.4 s.d. |
| a The effect size is the number of standard deviations represented by the difference between the two means. | |||
| Targeted to Children at Biological Risk | Not Targeted | |
|---|---|---|
|
Early childhood education
|
0.67 | 0.45 |
|
No early childhood education
|
0.5 | 0.26 |
| Note: A difference of .05 represents an effect size of one standard deviation. |
| Primary method of delivering parent education |
Targeted to Children at Biological Risk |
Not Targeted |
|---|---|---|
|
Home visiting
|
0.36 | 0.09 |
|
Parent groups
|
0.54 | 0.27 |
| Note: A difference of .11 represents an effect size of one standard deviation. | ||
| Randomized Studies | Randomized and Quasi-Experimental Studies | |||||
|---|---|---|---|---|---|---|
| Program Characteristic | Present | Absent | Effect Size of Differencea | Present | Absent | Effect Size of Differencea |
|
Program goal: parent self-help-development
|
0.56 | 0.25 | 5.2 s.d. | 0.41 | 0.22 | 1.9 s.d. |
|
Home visiting (vs. parent groups
|
0.15 | 0.26 | 2.4 s.d. | 0.1 | 0.23 | 1.4 s.d. |
|
Professional parent education staff (vs. paraprofessional)
|
0.43 | 0.27 | 4.0 s.d. | 0.39 | 0.23 | 1.6 s.d. |
|
Case management provided
|
0.08 | 0.27 | 6.0 s.d. | 0.08 | 0.23 | 1.5 s.d. |
|
Targeted to children developmentally at-risk
|
- | - | ns | 0.39 | 0.22 | 1.5 s.d. |
|
Serves majority low-income families
|
- | - | Ns | 0.12 | 0.22 | 1.0 s.d. |
|
Program collaborates with other community agencies
|
0.08 | 0.26 | 5.5 s.d. | - | - | Ns |
|
Opportunities for peer social support for parents
|
0.16 | 0.26 | 1.5 s.d. | - | - | Ns |
| Parent Education Staff Qualifications |
Goal of Parent Self-Development | Not a Goal |
|---|---|---|
|
Home visiting
|
0.57 | 0.41 |
|
Parent groups
|
0.39 | 0.23 |
| Note: A difference of .08 represents an effect size of one standard deviation. | ||
|
Parent Education Staff Qualifications
|
Goal of Parent Self-Development | Not a Goal | Goal of Parent Self-Development | Not a Goal |
|---|---|---|---|---|
|
All professionals
|
0.43 | 0.25 | 0.55 | 0.41 |
|
Some paraprofessionals
|
0.3 | 0.12 | 0.37 | 0.24 |
| Note: A difference of .07 represents an effect size of one standard deviation. | ||||
| Sample | Targeted to Children with Biological Risks | Not Targeted to Children with Biological Risks |
|---|---|---|
|
Randomized studies
|
0.57 | 0.41 |
|
Randomized & quasi-experimental studies
|
0.39 | 0.23 |
| Note: For randomized studies, a difference of .06 represents an effect size of one standard deviation. | ||
| For the combined sample, a difference of .13 represents an effect size of one standard deviation. | ||
|
Randomized Studies
|
Randomized and Quasi-Experimental Studies | |||||
|---|---|---|---|---|---|---|
| Program Characteristic | Present | Absent | Effect Size of Differencea |
Present | Absent | Effect Size of Differencea |
|
Target children 3 years and younger
|
0.56 | 0.13 | 2.0 s.d. | - | - | Ns |
|
Case management provided
|
0.55 | 0.13 | 1.9 s.d. | 0.68 | 0.2 | 1.4 s.d. |
|
Parent-child activities provided
|
- | - | Ns | 0.86 | 0.21 | 1.9 s.d. |
|
Targeted to teenage parents
|
- | - | Ns | 0.61 | 0.2 | 1.2 s.d. |
| a The effect size is the number of standard deviations represented by the difference between the two means. | ||||||
| Age of Children | Case Management Provided | No Case Management |
|---|---|---|
|
Targeted to children 3 years and younger
|
0.86 | 0.49 |
|
Targeted to older children
|
0.5 | 0.13 |
| Note: In this sample, a difference of .19 represents an effect size of one standard deviation. | ||
| Case Management Provided |
No Case Management |
|
|---|---|---|
|
Teenage parents a
|
||
|
Primary teenage parents
|
1.04 | .58 |
|
Not primary teenage parents
|
.66 | .20 |
|
Parent-child activities b
|
||
|
Parent-child activities
|
1.21 | .80 |
|
No parent-child activities
|
.62 | .21 |
| a A difference
of .32 represents an effect size of one standard deviation. b A difference of .31 represents an effect size of one standard deviation. |
||
| Randomized Studies | Randomized * Quasi-Experimental Studies | |||
|---|---|---|---|---|
| Age of Parents | Parent-Child Activities | No Parent-Child Activities | Parent-Child Activities | No Parent-Child Activities |
|
Serves primarily teenage parents
|
1.40
|
.90
|
1.00
|
.50
|
|
Not Primarily teenage parents
|
1.11
|
.61
|
.71
|
.21
|
| Note: A difference of .29 represents an effect size of one standard deviation. | ||||
Parent Outcomes
For parent outcomes, the significant relationships among program features and effects are more scattered than they were for child outcomes (Exhibit A5.5 and A5.6). In general, the features most consistently related to the size of effects were the program goals and activities. The programs with the largest effects focus on developing parents' skills as effective adults-their self-confidence, self-empowerment, family management and parenting. These programs also tend to provide opportunities for parents to meet in groups to provide peer support to each other. Programs that use professional staff to work with parents have stronger effects on parent outcomes than programs that rely more heavily on paraprofessional staff.
Parent Attitudes and Knowledge
One program characteristic is significantly related to effects on parent attitudes and knowledge, for both randomized studies alone and when combined with quasi-experimental studies: Programs that provide opportunities for peer support for parents have larger effects on parent attitudes and knowledge (Exhibits A5.5 and A5.6). The average effect size for programs that provide peer support opportunities is around .30, while the effect sizes for programs that do not provide these opportunities were around .20 (Exhibit A5.19). Among the randomized studies, the other significant predictor is whether a program targets children with special needs. Those programs have an average effect size three times larger than the programs that do not target services to this group of children (Exhibit A5.19).
The multivariate analyses show that programs with both characteristics-targeting special needs children and providing peer support opportunities to parents-have an average effect size of .65, a large effect, while programs with neither characteristic have an average effect size (.17) that did not reach the level of being meaningful (Exhibit A5.20). When the randomized and quasi-experimental studies are combined, the second significant predictor is whether a program has a goal of parent self-development. These programs have a significantly larger average effect size-.35 versus .23 for programs that do not have this goal (Exhibit A5.18). In this sample of studies, programs that provide peer support opportunities also tend to have parent self-development as a goal; therefore we could not model their combined effects.
| Randomized Studies | Randomized & Quasi-Experimental Studies | |||||
|---|---|---|---|---|---|---|
| Program Characteristic | Present | Absent | Effect Size of Difference |
Present | Absent |
Effect Size |
|
Peer support opportunities for parents
|
.33
|
.17
|
2.3 s.d.
|
.36
|
.22
|
1.9 s.d
|
|
Target children with biological risks
|
.57
|
.18
|
.61
|
-
|
-
|
|
|
Parent self-devlopment as program goal
|
-
|
-
|
|
.35
|
.23
|
1.4 s.d.
|
| Note: A difference of .29 represents an effect size of one standard deviation. | ||||||
|
Targeted to Special |
Not Targeted to Special Needs Children |
|
|---|---|---|
|
Peer support opportunities provided
|
.86
|
.49
|
|
Peer support opportunities not provided
|
.50
|
.13
|
| Note: A difference of .06 represents an effect size of one standard deviation. | ||
Parenting Behavior
A cluster of program and treatment characteristics are related to the size of effects on parenting behavior (Exhibit A5.5 and A5.6). Three program features are related to effects for randomized studies alone and combined with quasi-experimental studies: whether the program has parent self-development as a goal, whether case management is provided, and whether the program is concerned with community change. As Exhibit A5.20 shows, their effects are as follows:
- Programs for which parent development or self-help is a primary goal have an average effect size nearly twice as large as programs that do not.
- Programs in which community change is a focus have a significantly lower average effect size than programs that do not; whereas the programs with an emphasis on community change have essentially no effects, other programs have an average effect that is small but reaches a meaningful level.
- Programs that provide case management services have smaller effects than the programs that do not provide case management. Programs that provide case management have essentially no effect on parenting behavior, while the average effect size for other programs is a quarter of a standard deviation-a meaningful effect.
In the sample of randomized studies, an additional significant predictor of effects is whether or not the program has a goal of improving family economic self-sufficiency. In these studies, programs that focus on economic outcomes have significantly lower effects than other programs.
For the randomized and quasi-experimental studies combined, one other program feature is related to the size of effects on parenting behavior. Programs that focus on families with children with behavior problems have much larger effects on parenting behavior than other programs.
These predictors are themselves highly inter-correlated. The programs for which economic self-sufficiency is a primary goal are also the ones that tend to use case management and that tend to engage in community advocacy/change activities. These program features are all negatively related to effects on parenting. At the same time, these programs tend not to have parent self-development as a primary goal. They also tend not to be programs that focus on child behavior change.
In the multivariate analyses, we examined the effects of combinations of program features. As shown in Exhibit A5.21, among the randomized studies, programs that provide case management (which also tend to focus on improving family economic self-sufficiency and to include activities focused on community change) have a very small average effect size (.11), while programs that focus on developing parent skills and capacities have an average effect size of .39 if no case management services are provided and .25 if these services are provided.
Among the randomized and quasi-experimental studies combined, programs with the goal of developing parent skills and capacities, which also focus on families with children with behavior problems, have a very large average effect size-1.08 if no case management is provided and .97 if case management services are also provided (Exhibit A5.22).
|
|
Randomized Studies
|
Randomized & Quasi-Experimental Studies
|
||||
|---|---|---|---|---|---|---|
| Program Characteristic | Present | Absent | Effect Size of Difference(a) |
Present | Absent | Effect Size of Difference(a) |
|
Parent self-development as program goal
|
0.42 | 0.24 | .90 s.d. | 0.49 | 0.26 | 1.21 s.d. |
|
Case management services
|
0.08 | 0.25 | .87 s.d. | 0.12 | 0.26 | .70 s.d. |
|
Community advocacy activities
|
0.04 | 0.25 | 1.08 s.d. | 0.1 | 0.26 | .80 s.d. |
|
Child behavior change as program goal
|
- | - | Ns | 0.83 | 0.26 | 2.76 s.d. |
|
Economic self-sufficiency as program goal
|
0.06 | 0.25 | .90 s.d. | - | - | Ns |
| a The effect size is the number of standard deviations represented by the difference between the two means. | ||||||
| Program Services | Targeted to Special Needs Children |
Not Targeted to Special Needs Children |
|---|---|---|
|
Case management services provided
|
0.25 | 0.11 |
|
Case management services not provided
|
0.39 | 0.25 |
| Note: A difference of .19 represents an effect size of one standard deviation. | ||
|
|
Case Management Provided | No Case Management | ||
|---|---|---|---|---|
| Program Focus | Goal of Parent Self-Development | Not a Goal | Goal of Parent Self-Development | Not a Goal |
|
Focus on child behavior change
|
0.97
|
0.75
|
1.08
|
0.86
|
|
Not focused on child behavior change
|
0.37
|
0.15
|
0.48
|
0.26
|
Family Functioning/Family Resources
A relatively small number of studies reported measures of family functioning, limiting our power to detect relationships between effects and program characteristics. One program feature is related to effects for both randomized studies alone and randomized and quasi-experimental studies combined. Programs that have child abuse prevention as a primary goal have larger effects on family functioning than other programs (Exhibit A5.5 and A5.6). The average effect size for programs that focus specifically on child abuse and neglect is large-.78 among the randomized studies and .52 when the quasi-experimental programs are included. The average effect size for other programs is .26 for randomized studies and .17 for randomized and quasi-experimental studies combined (Exhibit A5.23). Once again, programs that provide peer support opportunities have significantly larger effect sizes than other programs (Exhibit A5.23).
Programs that focus on prevention of abuse and neglect also tend to be the programs that provide activities for peer support. Because of the inter-relationship among the predictors, we cannot say which of these characteristics is responsible for the stronger effects.
| Randomized Studies | Randomized & Quasi-Experimental Studies | |||||
|---|---|---|---|---|---|---|
| Program Characteristic | Present | Absent | Effect Size of Differencea | Present | Absent | Effect Size of Differencea |
|
Child abuse and neglect prevention as program goal
|
0.78 | 0.26 | 2.4 s.d. | 0.52 | 0.17 | 1.96 s.d. |
|
Peer support opportunities provided
|
- | - | Ns | 0.42 | 0.16 | 1.4 s.d. |
| a The effect size is the number of standard deviations represented by the difference between the two means. | ||||||
Parent Mental Health and Health Risk Behaviors
A number of program characteristics are related to the size of the effect on parent health outcomes, most in a negative direction (Exhibits A5.5 and A5.6). That is, a set of program features tended to be associated with very small program effects, including (a) a focus on improving family economic self-sufficiency; (b) provision of adult education services, and (c) provision of case management (Exhibit A5.24). In fact, these characteristics were correlated-a cluster of programs has all three characteristics, and this group of programs has essentially no effect on parent health and risk behaviors. The only program characteristic that is associated with larger and more meaningful effects on parents' mental health, is use of professional staff. Programs that use professional staff to work with parents have an average effect size of .29, compared with an average effect size of .14 among programs that use some or all paraprofessional staff.
The multivariate analyses examined the simultaneous effects of three of the program features in the sample of randomized and quasi-experimental studies: program focus on economic self-sufficiency, use of professional staff, and targeting of children with special needs. The only combination of program features that produces meaningful average effect on parent mental health is use of professional staff to work with parents, for programs that do not focus on economic self-sufficiency and do not target children with special needs (Exhibit A5.28).
| Randomized Studies | Randomized and Quasi-Experimental Studies | |||||
|---|---|---|---|---|---|---|
| Program Characteristic | Present | Absent | Effect Size of Differencea | Present | Absent | Effect Size of Differencea |
|
Improved economic self-sufficiency as program goal
|
-0.07 | 0.09 | 1.23 s.d. | -0.02 | 0.14 | 1.0 s.d. |
|
Provides adult education services
|
-0.06 | 0.09 | 1.15 s.d. | 0 | 0.14 | .88 s.d. |
|
Provides parent-child activities
|
- | - | -0.06 | 0.14 | 1.25 s.d. | |
|
Professional parent education staff
|
- | - | 0.29 | 0.14 | .88 s.d. | |
|
Targeted to special needs children
|
-0.11 | 0.09 | 1.43 s.d. | - | - | |
|
Age of child at post test (months)
|
-0.005 for each month of age | -0.003 for each month of age | ||||
| a The effect size is the number of standard deviations represented by the difference between the two means. | ||||||
| Focus on Economic Self-sufficiency | Not a Program Focus | |||
|---|---|---|---|---|
| Professional Staff | Some Para-Professionals | Professional Staff | Some Para-professionals | |
|
Peer support opportunities provided
|
-0.08 | -0.25 | 0.11 | -0.07 |
|
Peer support opportunities not provided
|
0.13 | -0.04 | 0.31 | 0.14 |
| Note: A difference of .16 represents an effect size of one standard deviation. | ||||
Family Economic Self-sufficiency
Most (80%) studies show no effect on family economic self-sufficiency, and only a small number of studies show a moderate or large effect. The amount of inter-study variation in effects was not significant. This means that not only was the overall effect on economic self-sufficiency small, there was too little variation in effects to try to predict them.
What are the Long-Term Effects of Family Support Programs and Services on Children and Families?
Follow-up data on outcomes after the end of services are reported in 158 of the 351 randomized or quasi-experimental studies in the end-of-treatment database. Across these studies, the follow-up effects are measured as early as a few months after the end of services, up to as long as nine years after the end of services. In this database, the average length of follow-up is 31 months. Most of the follow-up data are obtained less than two years after the end of services, as shown below:
| 6 months or less | 19% |
| 7 - 12 months | 22% |
| 13 - 24 months | 27% |
| 25 - 36 months | 6% |
| 37 - 60 months | 10% |
| 60+ months | 17% |
In the analyses that are described below, all of the follow-up data are combined. Length of follow-up period is tested as a predictor of size of effects and was found not to be significant.
Overall Average Effects
Exhibit A5.26 shows the average effects at follow-up in the nine outcome domains. There are only small samples of studies in most of the domains. For the family functioning outcomes, the sample size is too small to support analysis. For the randomized studies alone, the average effect sizes range from .04 for child health outcomes to .39 for economic self-sufficiency outcomes. Statistical tests indicate that the average effects are statistically significant in five of the domains (Exhibit A5.27). For the randomized and quasi-experimental studies combined, the average effect sizes are higher than for the randomized studies alone, ranging from zero for child health outcomes to .46 for economic self-sufficiency outcomes. The effect sizes are statistically significant in all of the outcome domains.
Program Characteristics Related to Differential Long-term Effects of Family Support Programs and Services
Exhibits A5.28 and A5.29 show the significant relationships between the individual programmatic predictors and the long-term effects in the nine outcome domains, for randomized studies alone and in combination with the quasi-experimental studies. For the randomized studies, there are very few significant relationships-so few, in fact, that we have to conclude that we do not know how to characterize the programs with larger long-term effects. When the quasi-experimental studies are included, there are more significant relationships, especially for the child outcomes. For this larger set of studies, case management is associated with smaller long-term effects on children. Another finding is that programs that provide peer support activities for parents tend to have smaller effects on children at follow-up. Parent/child activities, on the other hand, are positively related to long-term effects on children. In addition, programs that targeted children with biological risks tend to have larger long-term effects on children.
Because there were so few statistically significant relationships between the programmatic predictors and follow-up effects, no multivariate modeling was done on the follow-up outcome data.
![[Exhibit A5.26 : Average Adjusted Effect Sizesa,b in Nine Outcome Domains: Follow-up Outcomes]](images/exhib5_26.gif)
| Results | Child Cognitive Development | Child Social- Emotional Development |
Child Physical Health & Development | Child Injury, Abuse, Neglect | Parenting Attitudes & Knowledge | Parenting Behavior | Family Function in g/Family Resources | Parent Mental Health/Health Risks | Family Economic Self-sufficiency |
|---|---|---|---|---|---|---|---|---|---|
|
Randomized Studies Only
|
|||||||||
|
n of studies
|
48 | 51 | 20 | 14 | 12 | 38 | 4 | 18 | 12 |
|
Average effect sizea, b
|
0.304 | .094d | .049d | .115e | .152d | 0.178 | .190f | .165e | .386f |
|
Standard error
|
0.045 | 0.028 | 0.031 | 0.074 | 0.075 | 0.053 | 0.198 | 0.055 | 0.161 |
|
95% confidence interval
|
(.216, .392)*** | (.039, .149)*** | (-.012, .104) | (-.030, .260) | (.005, .299) | (.074, .282)** | (-.198, .578) | (.038, .273)** | (.070, .702)* |
|
Homogeneity of variance test c
|
P2 = 63.8* | P2 = 39.4 | P2 = 16.1 | P2 = 43.2*** | P2 = 25.0** | P2 = 41.5 | P2 = 20.0*** | P2 = 27.2* | P2 = 151.8*** |
| Randomized & Quasi-Experimental Studies | |||||||||
|
n of studies
|
101 | 82 | 28 | 21 | 22 | 55 | 6 | 25 | 21 |
|
Average effect size a,b
|
0.345 | .150d | .112d | .152d | .273d | 0.204 | .002f | .226e | .464e |
|
Standard error
|
0.04 | 0.024 | 0.036 | 0.068 | 0.054 | 0.041 | 0.045 | 0.051 | 0.089 |
|
95% confidence interval
|
(.267, .423)*** | (.103, .197)*** |
(.041, .183)** |
(.019, .285)* | (.167, .379)*** | (.123, .285)*** | (-.086, .010) | (.116, .236)*** | (.290, .639)*** |
|
Homogeneity of variance testc
|
P2 = 184.5*** | P2 = 101.4* | P2 = 33.3 | P2 = 68.6*** | P2 = 27.1 | P2 = 65.7* | P2 = 9.30 | P2 = 47.8** | P2 = 58.0*** |
|
* p < .05 ** p < .01 *** p < .001 |
|||||||||
|
Outcome Domain
|
|||||||||
|---|---|---|---|---|---|---|---|---|---|
| Program Characteristic |
Child n=48 |
Child n=51 |
Child n=20 |
Child n=14 |
Parent n=12 |
Parent n=38 |
Family n=4 |
Parent n=18 |
Econ n=12 |
|
Primary Program Goals
|
insufficient variation in effect sizes to test predictors |
insufficient variation in effect sizes to test predictors |
insufficient variation in effect sizes to test predictors |
insufficient variation in effect sizes to test predictors |
|||||
|
Social Support
|
|||||||||
|
Parent Self-help/self-development
|
|||||||||
|
Prevention of child abuse/neglect
|
+ [] | ||||||||
|
Economic self-sufficiency/literacy
|
|||||||||
|
Child mental health/behavior
|
|||||||||
|
Community participation
|
|||||||||
|
Targeting
|
|||||||||
|
Universal
|
|||||||||
|
Biological/developmental child risk
|
+ * | + * | |||||||
|
Biological risks
|
|||||||||
|
Developmental Risks
|
|||||||||
|
Teenage parents
|
|||||||||
|
Population served
|
|||||||||
|
Majority families low-income
|
|||||||||
|
% minority families
|
|||||||||
|
Majority teenage parents
|
|||||||||
|
Child age: infant/toddler vs. older
|
|||||||||
|
Child age at end of services (months)
|
- * | - ** | |||||||
|
Parenting education
|
|||||||||
|
Staff qualifications: all professionals
|
- * | ||||||||
|
Home visits vs. parent groups
|
- ** | - * | |||||||
|
Services provided : Types of services
|
|||||||||
|
Intended length of services
|
|||||||||
|
Any early childhood education
|
|||||||||
|
Any parent/child activities
|
- [] | ||||||||
|
Peer support activities
|
+ * | + * | |||||||
|
Any case management
|
- [] | - * | |||||||
|
Collaboration with other agencies
|
|||||||||
|
Any health services
|
|||||||||
|
Any community advocacy activities
|
|||||||||
| Services provided: Amount of services | |||||||||
|
Months services provided
|
|||||||||
|
Hours of early childhood education
|
|||||||||
|
Amount (hrs) of parent education
|
|||||||||
|
Intensity of parent education
|
|||||||||
|
Amount (hrs) of case management
|
|||||||||
|
Intensity of case management
|
|||||||||
| T p<.10 * p<.05 ** p<.01
*** p<.001
a Effect of each characteristic based on regression analysis (hierarchical linear modeling approach) with control variables accounted for. |
|||||||||
|
Outcome Domain
|
|||||||||
|---|---|---|---|---|---|---|---|---|---|
|
Program Characteristic
|
Child n=101 |
Child n=82 |
Child n=28 |
Child n=21 |
Parent n=22 |
Parent n=55 |
Family n=6 |
Parent n=25 |
Econ n=21 |
|
Primary Program Goals
|
insufficient variation in effect sizes to test predictors |
insufficient variation in effect sizes to test predictors |
insufficient variation in effect sizes to test predictors |
||||||
|
Social Support
|
|||||||||
|
Parent Self-help/self-development
|
+ * | ||||||||
|
Prevention of child abuse/neglect
|
+ * | ||||||||
|
Economic self-sufficiency/literacy
|
|||||||||
|
Child mental health/behavior
|
|||||||||
|
Community participation
|
|||||||||
|
Targeting
|
|||||||||
|
Universal
|
|||||||||
|
Biological/developmental child risk
|
|||||||||
|
Biological risks
|
|||||||||
|
Developmental Risks
|
|||||||||
|
Teenage parents
|
|||||||||
|
Population served
|
|||||||||
|
Majority families low-income
|
- * | ||||||||
|
% minority families
|
+ * | ||||||||
|
Majority teenage parents
|
|||||||||
|
Child age: infant/toddler vs. older
|
|||||||||
|
Child age at end of services (months)
|
- ** | ||||||||
|
Parenting education
|
|||||||||
|
Staff qualifications: all professionals
|
|||||||||
|
Home visits vs. parent groups
|
- + | ||||||||
|
Services provided : Types of services
|
|||||||||
|
Intended length of services
|
|||||||||
|
Any early childhood education
|
|||||||||
|
Any parent/child activities
|
|||||||||
|
Peer support activities
|
|||||||||
|
Any case management
|
|||||||||
|
Collaboration with other agencies
|
|||||||||
|
Any health services
|
+ * | ||||||||
|
Any community advocacy activities
|
|||||||||
|
Services provided: Amount of services
|
|||||||||
|
Months services provided
|
|||||||||
|
Hours of early childhood education
|
|||||||||
|
Amount (hrs) of parent education
|
|||||||||
|
Intensity of parent education
|
|||||||||
|
Amount (hrs) of case management
|
|||||||||
|
Intensity of case management
|
|||||||||
| [] p<.10 * p<.05 ** p<.01 ***
p<.001
a Effect of each characteristic based on regression analysis (hierarchical linear modeling approach) with control variables accounted for. |
|||||||||
Conclusions
The goal of the evaluation in general, and of the meta-analysis in particular, was to investigate the effectiveness of family support services in improving outcomes for parents and, ultimately for children. The results of the meta-analysis offer some encouraging messages as well as some warnings. In addition, the findings offer some clues about how family support services might be strengthened.
Family support services produce small but significant effects across a range of outcomes for parents and children. Family support programs and services are generally small-scale efforts with modest budgets. The levels of effects reported here seem, on the whole, consonant with the level of investment made in these programs. On the other hand, given our limited understanding of the practical meaning of these effects, one should use caution in making strong claims for family support as an intervention strategy likely to make a meaningful difference in families' lives.
There is no single effective program model. Family support has been promoted as an effective strategy to address a host of social problems, from child abuse and neglect to school failure and delinquency. Although we have identified some strategies that have proven effective with specific populations, there is no single program approach, curriculum or service strategy that has demonstrated effectiveness across a range of populations. With one or two exceptions, the models that have shown larger effects have been tested in single-site research and demonstration initiatives and have not been widely replicated.
The effects of family support are not evenly distributed across different program models and service strategies. There are hundreds of family support programs across the country, as well as thousands of schools, hospitals and other institutions that include family support services in the programs they offer. For most of them, the core service provided is some form of parenting education. It is sobering to realize that much of this effort may not produce even the modest effects that programs hope for. Almost two-thirds of the programs we studied had very small or no effects on parents' understanding of child development, attitudes about childrearing or behavior with their children. More than half of the programs had small or no effects on family functioning.
Why are these services relatively ineffective? We can begin to understand some of the reasons, if we look at the strategies that produced stronger effects. Programs that use professional staff and deliver parent education and support through group meetings had stronger positive effects on parenting behavior and, in addition, on outcomes for children. Programs that focused services on specific types of families rather than on, for example, all low-income families in a neighborhood tended to be more effective.
However, family support service strategies have moved toward delivering such services through home visits, usually by paraprofessional staff. In addition, the family support philosophy emphasizes the desirability of non-targeted services. These strategies show the weakest effects on both parent and child outcomes.
Family support services are effective in promoting children's cognitive development and school readiness only if they provide services directly to children. The assumption that many parents lack the necessary skills to be effective teachers of their children has led to the widespread use of parenting education in family support programs. There is no evidence of its effectiveness in promoting children's cognitive development. Nor is it clear that adding parent education to direct services to children confers an additional benefit. Other major reviews of the relevant research found that adding parenting education to preschool programs did not increase their effectiveness.
Family support services are effective with some important and vulnerable populations. Given the concerns that generated the federal legislation, it is encouraging that family support services that focused on teenage mothers with very young children, families that contain a child with special needs or families that have a child with behavior problems, all had strong positive effects on parents, on children or on both. It is important to recognize that, in addition to focusing services rather narrowly, these programs also tended not to use paraprofessionals to deliver services or home visiting as a service delivery strategy. Parent groups led by professional staff were important for parents of children with special needs and for parents of children with behavior problems. For teenage parents, organized parent-child activities were important. This was the one group of parents who appeared to benefit from having a case manager.
The hearings on the legislation that provided additional funding for family support services produced testimony that promoted family support as an effective primary prevention strategy for child welfare. The findings from the meta-analysis make it clear that much more work needs to be done to identify effective ways to work with and support parents in their efforts to raise their children. The family support movement has made a significant difference in the way that hospitals, schools and social service agencies regard and treat families. It may need to re-examine some of its assumptions about the kinds of services that are truly useful to families for enhancing the development of their children.
1 The low incidence of maltreatment, even in high-risk populations, makes it difficult to detect the effect of an intervention, especially in small studies.(back) 2 Based on the sample from the evaluation of the Comprehensive Child Development Program (St. Pierre, Layzer, Goodson, & Bernstein, 1997). (back)
| Table of Contents | Previous | Next |