|
|---|
|
|
||||
|---|---|---|---|---|
|
|
|
|||
| ACF Home | Services | Working with ACF | Policy/Planning | About ACF | ACF News | HHS Home | ||||
Questions?
|
Privacy
|
Site Index
|
Contact Us
|
Download Reader™
|
Print
 �
|
|---|
| Demonstration Type: | Mississippi - Intensive Service Options1 |
| Approved: | September 17, 1998 |
| Implemented: | April 1, 2001 |
| Completed: | September 30, 2004 |
| Interim Evaluation Report Date: | NA |
| Final Evaluation Report Date: | June 30, 2005 |
Mississippi’s demonstration targeted title IV-E-eligible and non-IV-E eligible children ages 0-18 involved in the child welfare system who met one of the following criteria: (1) in State custody (and, in most cases, in out-of-home placement), or (2) not in State custody but who had been removed from the physical custody of their original caretaker and whose permanency plan was reunification, or (3) not in State custody but determined to be at risk of future maltreatment or out-of-home placement.
In addition, waiver services were targeted at the parents, foster parents or potential foster parents, custodial relatives, siblings, and adoptive or potential adoptive parents of these eligible children.
The State’s waiver demonstration was implemented in eight counties located within two child welfare districts in the State: Holmes, Madison, Rankin, Yazoo, Covington, Jones, Lamar, and Pearl River. The State selected these counties as representative of the State as a whole with respect to key demographic and socioeconomic variables.
The waiver project in Mississippi was designed in response to specific findings of the 1995 ACF/DHHS assessment of the State’s child protection system. Through its demonstration, the State sought to test the effectiveness of a family-centered practice model that gave participating counties broad latitude in using title IV-E funds to respond to the needs of families involved in the child protection system. Greater emphasis was placed on home-based services, prevention services, and enhanced supports for foster parents, especially relative caregivers. The State served families in the experimental group using an array of existing and newly created services to prevent out-of-home placement, expedite permanency, reduce maltreatment risk, and improve the overall well-being of children and their adult caregivers. Services and supports provided to families included, but were not limited to, transportation, clothing, payments to foster care and independent living facilities, school supplies, medical care, rental assistance, and utility payments.
In addition to a broader array of intensive services, Mississippi planned to implement Family Team Meetings–facilitated by the waiver’s regional coordinators–as a major demonstration component. The goal of Family Team Meetings was to involve family members more directly in case planning and create a strong and permanent circle of support for them.
Mississippi’s evaluation included process and outcome components, as well as a cost analysis. The State’s evaluation plan stipulated an experimental research design with random assignment to experimental and control groups at a 1:1 ratio. Cases that met screening criteria were randomly selected for inclusion into one of the two study groups. A computer-based software program was developed by the evaluators for the random selection process, which was then downloaded onto laptop computers. Each waiver county received one of these laptop computers and workers received training in the use of the random assignment software.
The State’s evaluation plan estimated that approximately 1,174 families would be assigned to each study group, for a total study population of about 2,348 families. However, a combination of factors, including slow project startup, inadequate staff to screen and process new enrollments, and the early termination of the State’s waiver, substantially curtailed the number of families that actually enrolled in the demonstration. During the 42 months of the project’s operation, only 667 families met the project’s screening criteria and underwent random assignment, with 346 families assigned to the experimental group and 321 families entering the control group. These families included 1,549 children, 777 of whom were in the experimental group and 772 in the control group.
The evaluation’s main research database included child welfare and cost data obtained directly from participating counties and from the Mississippi Automated Child Welfare Information System (MACWIS). The first extraction of MACWIS data was received by the evaluation team in October 2002. Although the demonstration was suspended in September 2004, collection of MACWIS data continued until January 2005.
The evaluation’s process component involved regular site visits to state and county child welfare offices and interviews with state and regional child welfare administrators, local child welfare supervisors, and social workers. The final site visits and interviews were completed in February 2005.
Through the evaluation’s outcome component, the State sought to determine the effects of the intensive services demonstration on several child welfare outcomes, including maltreatment recurrence, placement avoidance, length of time in out-of-home placement, reunification with families of origin, and overall child well-being.
Process Evaluation
The waiver demonstration did not begin simultaneously in all eight counties as originally planned but was phased in over an eighteen-month period. Several factors led the State to phase in the waiver incrementally. These included the introduction of the State’s new MACWIS information management system, and delays in obtaining approval for modifications to its cost allocation plan. Implementation began in April 2001 in Rankin and Jones Counties, was extended to Holmes and Lamar Counties in April 2002, and was completed by September 2002 in the final four counties of Madison, Yazoo, Pearl River, and Covington.
In addition to a delayed startup, Mississippi’s demonstration faced several other barriers during the course of its implementation, including the following:
In response to these challenges, the demonstration’s original service model changed substantially over the course of the demonstration. The de facto loss of one waiver coordinator led to the suspension of Family Team Conferences, as well as greatly reduced technical assistance and support for child welfare staff.
Mississippi suspended its intensive demonstration on September 30, 2004, 42 months after it began in the first two counties. The most significant reason for the waiver’s early termination was an ongoing inability to remain cost neutral, specifically with respect to administrative cost overruns. Mississippi’s low title IV-E eligibility rate for children made it difficult to recoup the cost of intensive services provided to non-IV-E-eligible enrolled children and families. This situation played a major role in the State’s failure to meet the Federal cost neutrality requirement.
Despite chronic implementation problems and its early termination, Mississippi’s intensive services demonstration succeeded in providing more and a greater variety of services to experimental group families than to control group families:
Outcome Evaluation
In addition to providing more and a greater diversity of intensive services to experimental group families, Mississippi’s waiver demonstration produced statistically significant positive results in two key child welfare outcomes.
Maltreatment Recurrence
Experimental group children were significantly less likely to have a new maltreatment report following assignment to the demonstration. At the end of the demonstration, 14.5 percent of experimental group children had a new maltreatment report compared to 19.7 percent of control group children, a statistically significant difference at p = .004. A reduction in reports of physical abuse accounted for most of this difference, with 3.7 percent of experimental group children having a new report of physical abuse compared with 6.0 percent of control group children. A survival analysis confirmed this finding by demonstrating that control group children experienced new reports sooner and, therefore, more reports during the follow-up period.
Placement Avoidance
Experimental group children who had not been removed from their homes prior to the start of the demonstration were less likely to be removed and placed in an out-of-home care setting than control group children. Overall, 9.1 percent of experimental group children without a prior placement were removed from their homes compared to 14.1 percent of control children, a statistically significant difference at p = .005. A subsequent survival analysis confirmed that control children group experienced out-of-home placement sooner and more often during the follow-up period, with the difference between their survival rates (i.e., time until first placement) statistically significant at p = .025.
Although not statistically significant, the evaluation revealed positive trends in favor of the experimental group in several other key child outcomes:
Recurrence of Substantiated Reports
Overall, 5.7 percent of experimental group children had a new substantiated report compared with 6.2 percent of control group children. This finding was consistent over time and was observed among both preexisting cases and new cases.
Reunification
Among all children who were in or entered out-of-home placement during the demonstration, 22.4 percent of experimental group children and 19.6 percent of control group children were reunified with their families of origin before the end of data collection in January 2005. This difference was in the hypothesized direction and represented a trend that may have reached statistical significance if the demonstration had continued.
Time in Out-Of-Home Placement
When examining all children enrolled in the demonstration, the mean number of days spent in non-emergency out-of-home placement was nearly identical for experimental group children (147 days) and control group children (145 days). When this analysis was restricted only to children who entered foster care after assignment to the demonstration, however, the mean number of days in placement was less for experimental group children (41 days) than for control group children (56 days).
No differences emerged between the experimental and control groups in other outcomes of interest, including the likelihood of placement with relatives, placement of siblings together, placement in geographic proximity to the child’s family of origin, and the frequency of moves between foster care providers. Due to the waiver’s early termination, no reliable data were available regarding the effects of the demonstration on several measures of family and child well-being, including householder wages, public assistance participation, school performance, and children’s emotional well-being.
Cost Analysis
Total dollars spent from all funding sources on experimental group families for non-placement services exceeded the total spent on non-placement services for the control group; however, the difference in service expenditures between the two groups was considerably less than what was accounted for by the outlay of waiver funds. The disparity resulted from greater average expenditures from other public, non-waiver sources to pay for services for control group families. This finding corroborated anecdotal evidence that the availability of the waiver allowed counties to spend more money from other sources on services for control group families, an unintended “contamination” effect that may have diminished observable differences in outcomes between the experimental and control groups.
By comparing initial program investment costs with the long-term costs incurred to serve families, the State’s evaluation team observed that average per child expenditures – including costs for both placement and non-placement services – were greater for experimental group children ($3,737) than for control group children ($3,200). However, when this analysis was restricted to children not in placement at the time of waiver assignment, average non-placement expenditures were greater for control group children ($1,162) than for experimental group children ($1,003). This analysis was heavily skewed by the truncated period available for cost and outcome data collection; however, the State’s evaluators hypothesized that given the demonstration’s success in reducing subsequent maltreatment reports, long-term costs for all experimental group children may have been lower if adequate follow-up had been conducted.
Cost-Effectiveness Analysis
To assess the cost-effectiveness of its waiver demonstration, the State’s evaluation team examined direct per child service costs in relation to child welfare outcomes. Through this analysis, the State’s evaluators determined that it cost an average of $270 more per experimental group child than per control group child to produce a 5.2 percent overall reduction in subsequent maltreatment reports. However, the analysis also found that it cost an average of $37 less per experimental group child than per control group child to realize a 5.0 overall percent reduction in out-of-home placements. Although this latter finding suggests that intensive services may have prevented more placements at lower cost, it remains uncertain whether these savings would have been sufficient in the long run to offset the administrative cost overruns incurred by the waiver demonstration. In light of the early termination of Mississippi’s demonstration and the subsequent truncation of data available for a more comprehensive cost analysis, these cost effectiveness findings should be regarded as preliminary. The State’s evaluators recommend caution in interpreting the data.
1Based on information from Mississippi’s June 2005 final evaluation report. Back
Back to Table of Contents
