Table of Contents | Previous | Next |
IV. The Supporting Healthy Marriage Study Design
The study is designed to achieve several goals. First, it will use a random assignment research design to test the effectiveness of the program model that we have described and to learn for which low-income families it works best and least well. Second, the study will describe how the programs operated in each site, both to shed light on patterns in program impacts and to provide lessons for future program operators. Third, because of the experimental research design, the project can provide unique scientific information about the causal links between family risk factors and couple and child outcomes.
Implementation Study
SHM represents a new type of public program that the federal government has only recently begun to provide at any scale. This early stage of program design and development across the country means that the implementation research to be conducted will play a critical role in informing future efforts to provide marriage-related services. As the research team documents how the programs are operated in each site, we will be explaining how a variety of community-based organizations begin to weave this new set of services into their program menus.
The main objective of the implementation research is to document how each site operated the program “on the ground.” Knowing how the program was experienced by participants, as opposed to how it was designed on paper, provides critical context for interpreting the impacts achieved. It also provides practical information that future program operators will need to replicate the program, should it prove effective. How did each site define the mission of its program and how did their program operations reflect that perspective? What challenges did these early programs face and how did the resolve them? What methods did they use to recruit large numbers of couples into this voluntary program and keep them engaged in services over time? How did couples respond to this offer and to their experiences in the program? Did sites deliver the marriage education curricula “as intended” or did they make changes to or supplement these and other aspects of their programs in response to the needs of their local populations? Did programs develop innovative strategies for encouraging participation among fathers, parents with irregular work schedules, and other individuals who are typically more difficult to engage in group activities? Did these marriage education programs have any effect on the services offered by the rest of the organization or did they operate in isolation from the broader organization? Findings from the process study will begin to provide some basic information to this nascent field on challenges in, and promising practices for, providing marriage education to low-income couples. In so doing, we hope to explore broader lessons for providing voluntary services to disadvantaged families and for engaging fathers in social service settings that have historically focused primarily on serving mothers and children.
The results of the implementation research will be reported for each individual site, providing the information needed to interpret variations in the impacts seen in different sites. The study is not designed to rigorously test how variations in site characteristics or program operating structures affect the impacts of the program. However, by documenting variations in program philosophies and operations, the implementation study does allow us to examine whether the patterns of impacts are consistent with particular hypotheses about how programs produce impacts on couples and families. Do the sites with the highest rates of participation and duration in the program consistently produce the greatest impacts on couple relationships? How are organizational and staff issues at the various sites related to the outcomes observed? Are there other variations in program operations that seem consistently to be associated with program impacts?
The implementation study will draw on both quantitative and qualitative data to document program operations. Data will include automated program tracking data to measure recruitment, participation patterns in marriage education groups, and individual or couple contacts with staff for the program group; 12-month survey data to measure participation in a range of community services for both program and control groups; observations of marriage education classes in each site; observations of other staff — participant meetings and interactions; debriefings with curriculum developers, technical assistance providers, and managers in each site to understand the specific programmatic challenges faced by each site and any innovative solutions to those challenges; reviews of program documents; and interviews with program staff and participants.
Impact Study
As discussed earlier, the primary goal of SHM is to learn whether marriage education improves low-income married couples’ relationships, helps them stay married, and benefits their children. In that light, the project will concentrate on two broad questions about the program’s effects: (1) Does the SHM intervention improve outcomes for low-income couples and their children? and (2) What family outcomes are affected?
The answer to these questions will flow from SHM’s basic evaluation method: random assignment. The goal is for each participating program to randomly assign up to 800 couples to either a program group or a control group. Random assignment assures that systematic differences that later appear between the program and control groups can reliably be attributed to the marriage education services being studied rather than to preexisting differences. If sample sizes permit, the impact analysis will estimate the effects for each site, as well as the pooled impacts across sites, since they are all operating within a common program framework.
The impact study will reflect as comprehensively as possible the multiple domains of family functioning that could be affected by the SHM program model, as hypothesized in the conceptual model in Figure 10.1. These include the functioning of couples’ relationships, or marital quality; marital duration and stability; the mental health and employment status of each individual spouse; family income; co-parenting and parenting behavior; and child well-being. These potential program impacts will be measured at 12 and 36 months after random assignment — and possibly at 60 months if earlier findings reveal impacts.
The 12-month survey will estimate the outcomes considered most proximate to the intervention, particularly couple relationship quality and stability. As discussed earlier, measures of relationship quality will include those with negative as well as positive valence. As part of the 12-month data collection effort, the research team also plans to collect videotaped direct observations of couple, co-parenting, and parent-child interactions for a subset of the sample in each site. This aspect of the study draws on the techniques used in basic research on marriage to understand the patterns of interaction that predict couples’ relationship satisfaction and stability. Much of this previous research has been conducted in university laboratories. Analogous to the growing use of in-home observations of parent-child relationships in large-scale surveys and experiments, to our knowledge SHM is the first study to videotape structured interactions of couples in their homes as part of a large-scale multisite experiment.
Whereas self-reports provide an important portrait of couples’ appraisals of their relationships, videotaped observations of couples’ interactions allow for direct assessment of specific patterns of communication and interaction. Independent observations are also capable of capturing aspects of couple and family interactions that are not readily captured by self-reports, in part because couples may not be aware of their nuanced interaction patterns. As with self-reports, we will record interactions that include both topics of disagreement as well as topics that elicit emotional support, reflecting current research on the importance of both positive and negative aspects of marital quality. In addition, the observational study will include parent-child interactions, providing an opportunity to measure effects of the intervention on co-parenting and parenting, which have been found in some previous work to be an important mediator between marriage interventions and improvements in child well-being. Thus, the evaluation will provide a full picture of program effects covering outcomes in multiple domains measured using multiple methods and perspectives.
The 36-month survey will provide longer-term follow-up of the same family outcomes, particularly marital outcomes, as measured on the 12-month survey. In addition, it will include direct assessments of children’s cognitive and behavioral development. Because improvements in parental relationships could theoretically produce different effects on children of different ages, if sample sizes allow, child well-being will be assessed for children whose ages at random assignment range from infancy to early adolescence.
Although the primary research question in SHM is to understand whether marriage education programs have any effect on marital relationships, the data collected will also provide some basis for understanding how the SHM program has any effects that are observed. As illustrated in the project’s conceptual framework, impacts might occur through a number of path-ways. By assessing impacts at different points in the model, we hope to shed light on how these interventions affect couples and families. Do we see impacts on individual spouses’ attitudes, insights, or expectations about their marriage; in couples’ communication and problem-solving skills; or in aspects of the relationships that are not as explicitly targeted by the curricula, such as couples’ levels of commitment to their marriages? Do the programs help couples reduce neg-ative interactions, increase positive ones, or both? Do the programs reduce levels of exposure to chronic and acute stressors, either directly, by providing supplemental services, or indirectly, by helping couples improve their coping skills? When programs improve couples’ ability to cope with external sources of stress, do we also see heightened improvements in marital satisfaction and stability? Answering some of these questions about the kinds of impacts achieved will help us understand how these programs might operate most effectively in the future.
A second critical set of mediators concern how any impacts on marital relationships are ultimately associated with any impacts on child well-being. As shown in the conceptual model, we hypothesize that marriage education could affect children by improving their parents’ mental health, marital quality, marital stability, co-parenting behavior, or parenting behavior. Parents’ improvements in some or all of these areas could lead children to have access to more positive relationship role models, to witness harmful conflict less frequently, to have better relationships with one or both of their parents, or to benefit from greater family income. Each of these areas of parental behavior has been associated with improved child well-being, primarily in well-designed, nonexperimental studies. SHM gives us the opportunity to observe whether an intervention that improves one or more of these outcomes for parents, in turn, improves child well-being.
We will also examine which types of families benefit the most or
the least from marriage education. It is possible that impacts of
marriage education programs could vary depending on couples’
characteristics at the time they entered the study. A survey administered
at the time couples enter the study will provide information on a
range of individual and family characteristics that have been found
to be associated with marital outcomes and could, in turn, affect
couples’ responses to a marriage education intervention. These
include, for example, the length of time couples have been married;
couples’ initial levels of marital satisfaction or distress;
each spouse’s level of commitment to the marriage; the mental
health of each spouse; the number and characteristics of children;
family income or education levels; families’ ethnic or racial
characteristics; and the existence of significant external stressors,
such as unemployment or a family member with chronic health problems.
The impact analysis can rigorously assess whether program impacts
vary depending upon these family or individual characteristics by
directly comparing the impacts achieved for different subgroups of
families. Policymakers and practitioners could use this type of information
to target future SHM-type services to couples and families for whom
these services were most effective, while developing new strategies
for serving subgroups of couples or families for whom these services
were less effective.
| Table of Contents | Previous | Next |