Children's Bureau Website - New Hampshire- Services for Caregivers with Substance Use Disorders

Illinois

Demonstration Type:	Services for Caregivers with Substance Use Disorders
Approved:	September 24, 1998
Implemented:	November 15, 1999
Completed:	November 30, 2005¹
Interim Evaluation Report Date:	September 12, 2003
Final Evaluation Report Date:	Completed, issuance pending

Target Population

New Hampshire’s waiver demonstration targeted families with an allegation of child abuse and/or neglect in which the caretaker’s substance abuse was cited as a major factor in the maltreatment referral. All families that met these criteria could participate in the demonstration regardless of their children’s age or title IV-E eligibility status.

Jurisdiction

New Hampshire implemented the demonstration in two Child Protection Service (CPS) District Offices in the State, one in the City of Nashua and one in the City of Manchester. The demonstration was implemented in the Nashua District Office in November 1999 and in the Manchester District Office in November 2000. These two district offices serve the majority of Hillsborough County, the most populous county in New Hampshire.

Intervention

Through New Hampshire’s waiver demonstration, known as Project First Step, Licensed Alcohol and Drug Abuse Counselors (LADCs) worked with child protection workers in an advisory and supportive capacity by providing training, assessment, treatment, and case management services. LADCs conducted an initial drug and alcohol assessment concurrently with the CPS maltreatment investigation and were involved from the outset in the risk and safety assessment to facilitate better decisions regarding child safety and out-of-home placement. Depending on a parent’s level of cooperation, LADCs could provide direct outpatient treatment or facilitate treatment access by removing resource barriers and engaging in outreach on the parent’s behalf. LADCs could treat caregivers directly without regard to payment eligibility, thereby improving the timeliness of access to substance abuse treatment services and increasing the likelihood of positive treatment outcomes. In addition, LADCs had the option to continue working directly with caretakers for an additional two months following completion of the maltreatment assessment or CPS case opening.

Enrollment into Project First Step occurred immediately at the time of an initial CPS maltreatment report. Following receipt of this report, the State’s evaluation contractor at the University of New Hampshire randomly assigned families to experimental (i.e., “Enhanced”) group or control (i.e., “Standard”) group. The caregiver’s formal substance abuse assessment occurred after assignment to the demonstration and was conducted by the LADC using the Substance Abuse Subtle Screening Inventory (SASSI). Only caregivers assigned to the experimental group underwent a formal substance abuse assessment.

New Hampshire had originally planned to pursue a five-year extension of its waiver demonstration. However, after the State determined that the demonstration could not maintain cost neutrality with respect to the use of title IV-E funds, the State withdrew its application for a long-term waiver extension in February 2005 but continues to operate Project First Step using title IV-B funds.

Evaluation Design

The evaluation of Project First Step consisted of process and outcome components, as well as a cost analysis that examined the utilization of title IV-E funds. Using an experimental research design, an independent evaluator randomly assigned families to either the experimental or control groups. Families assigned to the experimental group received enhanced prevention and intervention services through a LADC, whereas families assigned to the control group received standard child protection and substance abuse services.

Sample Size

New Hampshire originally planned to enroll 240 families into the demonstration at a 1:1 ratio (120 in the experimental group and 120 in the control group). To increase the likelihood of detecting significant outcomes, the State received approval in April 2001 to increase the evaluation’s sample size. By July 2003, a total 437 families had enrolled in the demonstration, with 222 families in the experimental group and 213 in the control group. The State discontinued further enrollment into the demonstration to allow LADCs to manage their existing caseloads in an optimal manner. Small sample sizes relative to the demonstration’s potentially eligible target population had limited statistical power and were less likely to reveal statistically significant effects of the demonstration on child welfare outcomes of interest.

Data Collection

The evaluation team conducted a total of 212 baseline interviews (49 percent of the total sample of eligible families) and 156 follow-up interviews (74 percent of baseline participants). Interview data were collected to provide in-depth information on each case at study intake and follow-up, regardless of the case’s maltreatment determination status. Data were also collected from New Hampshire’s SACWIS, client record files, and LADC service records.

Process Study

New Hampshire’s process evaluation focused on the following variables: substance abuse assessment rates; prevalence of drug and alcohol problems among participating families; service utilization; LADC and CPS worker contacts with families; substance abuse treatment access and participation rates; and organizational factors, such as staffing issues, that affected project implementation.

Outcome Study

The State’s outcome evaluation focused on the following child welfare outcomes: rates of entry into out-of-home placement, length of stay in foster care, reunification rates, rates of maltreatment recurrence, and child and caregiver well-being and functioning.

Evaluation Findings

Process Findings

Referral and Enrollment of Cases

The State described several challenges with respect to client enrollment and engagement in substance abuse treatment services. First, families were enrolled in Project First Step at the start of a maltreatment investigation but prior to a substantiation of abuse or neglect. The decision to target families during one of the most adversarial points in the case management process, combined with the voluntary nature of the program, increased the challenge of enlisting the active and willing participation of caregivers in the demonstration. Participation in Project First Step could only be mandated by the court if a maltreatment investigation led to a substantiation of abuse or neglect. Moreover, the substance abuse assessment and offer of enhanced services occurred at a time when some caretakers had difficulty recognizing or acknowledging their substance abuse issues, a factor that may have had an additional negative impact on assessment completion and treatment participation rates. Nevertheless, LADCs remained available to CPS workers for ongoing consultation and support regardless of a caretaker’s level of participation in the demonstration.

Substance Abuse Assessment Rates

By the end of the demonstration, 132 experimental group caregivers (61 percent) had completed a substance abuse assessment. Reasons for lower-than-anticipated assessment rates included caregivers’ refusal to give informed consent to participate in research, clients’ unwillingness to acknowledge a substance abuse problem, parents’ concerns about losing custody of their children if they shared information about their substance abuse, and staff turnover that led to a six-month gap without a LADC at one demonstration site.

The voluntary nature of the substance abuse assessment, combined with very low maltreatment substantiation rates and subsequent CPS case openings, meant that experimental group caregivers could decline to participate in the assessment or participate in substance abuse treatment. According to New Hampshire’s March 2004 progress report, 86 percent of maltreatment investigations in cases assigned to the experimental group were unsubstantiated, leading the State to close these cases without the ability to require further assessment or services. Although all families were identified at CPS intake as having substance abuse as a potential risk factor, experimental group families were significantly more likely than control group families to have substance abuse documented as a risk factor by the end of the maltreatment investigation.

Treatment Access and Participation

The experimental and control group caregivers participated in substance abuse treatment at similar levels. According to client case record data, 26 percent of experimental group caregivers (n = 101) received substance abuse treatment services compared with 24 percent of caregivers in the control group (n = 106); this difference was not statistically significant. However, among cases with a substantiated allegation of maltreatment, the data suggest that experimental group caregivers were more likely to access intensive, long-term treatment services. By the project’s end, ten experimental group caregivers (20 percent) with a substantiated maltreatment allegation had received long-term inpatient substance abuse treatment services, compared with only three control group caregivers (6 percent).

Differences in Program Fidelity

The State’s evaluators observed that the Manchester CPS District Office implemented the demonstration in a manner that was more consistent with Project First Step’s intended service model. In particular, staff turnover in the Nashua District Office undercut efforts by that site to maintain fidelity to the demonstration’s original model, which required consistent and intensive connections with experimental group families. Over a six month period, Nashua had only part-time assistance from a LADC in another CPS office while it searched for a qualified applicant to fill the vacant full-time position. In addition, CPS closed some cases without informing the substance abuse counselor, thus leaving little or no time to engage these experimental group families. Differences in implementation fidelity may explain the significant differences in some child welfare outcomes observed between these two sites.

Outcome Findings

Placement Rates

No statistically significant differences in placement rates emerged between families receiving enhanced substance abuse services and those receiving traditional services. Overall, 63 percent of experimental group families that ever had an open CPS case during the demonstration had at least one child enter placement compared with 62 percent of control group families. However, experimental group families were more likely to have children placed with kin (22 percent) than control group families (16 percent), a difference that approached statistical significance (p<.10).

Placement Duration

Average length of placement per child did not differ significantly for experimental and control group families. On average, experimental group children who had been removed from the home spent slightly more time in out-of-home placement (287 days) than control group children (260 days).

Reunification Rates

Among children who entered or began the demonstration in out-of-home placement, 44 percent of those in the experimental group had returned home compared with 39 percent in the control group; this difference was not statistically significant. When the probability of reunification was examined in individual CPS offices, however, experimental group caregivers from the Manchester site were twice as likely as control group caregivers to achieve reunification (p <.05). In contrast, no significant differences in reunification rates were found between the experimental and control groups served through the Nashua site.

Maltreatment Recurrence

The availability of enhanced substance abuse services resulted in lower, albeit statistically insignificant rates of repeat maltreatment substantiations in the experimental group (8.8 percent) than in the control group (11.4 percent). However, when maltreatment recurrence was examined in individual CPS offices, the State found that experimental group families served through the Manchester site were significantly less likely than those in the control group to have a subsequent substantiation (p <.05). In contrast, no significant differences were found between experimental and control group families served through the Nashua site.

Child and Family Well-Being

Interviews conducted using the Child Behavior Checklist (CBCL) indicated greater declines in problem behaviors in seven out of eight categories for experimental group children, including incidents of anxiety, depression, sleep problems, attention deficits, and aggressive behavior. Although these findings pointed in a positive direction, none reached statistical significance. When school status was examined, however, experimental group children were significantly less likely to repeat a grade than control group children (p<.05). With respect to physical health, experimental group children had equivalent or slightly better status on four out of six health outcomes, although none of these differences was statistically significant.

Among enrolled caregivers, those in the experimental group were significantly more likely to be employed full-time than control group caregivers (38 percent versus 24 percent) and were somewhat more likely to be enrolled in a vocational or educational program (28 percent versus 17 percent).

Although many of these well-being findings lacked statistical significance, the pattern of somewhat improved outcomes for children and adults across several domains suggests a positive trend for families that received enhanced substance abuse services.

¹This is one of three Illinois Child Welfare Demonstration Projects. The evaluation findings reported in this profile are limited to the five years of the original title IV-E waiver and are based on information submitted by the State as of January 2006. Back

Back to Table of Contents