Search Type Title Author/Investigator Search Advanced Search

Return to Previous page

PDF Version, Color

National Survey of Child and Adolescent Well-Being

No. 6: How Do Caseworker Judgments Predict Substantiation of Child Maltreatment?

Research Brief

Findings from the NSCAW Study

What is Substantiation?

Substantiation denotes child welfare system (CWS) services’ official decision about the validity of maltreatment allegations. This decision is an important one, with implications for how much a child and family are involved with the CWS, what services they receive, and how child maltreatment is counted in state statistics.

Substantiation is based on the answers to two questions:

• Is the harm to the child severe enough to constitute child maltreatment? This part of substantiation can include both current harm and the risk of harm in the future.

• Is there sufficient evidence to support the designation of the case as one of child maltreatment?1

What constitutes evidence of child maltreatment varies from state to state: each state has its own definition of child maltreatment and its own evidentiary standard for determining child maltreatment.2 Some states count infants’ prenatal exposure to drugs as child maltreatment, for example, while others do not.3 Some states require a preponderance of the evidence for a case to be substantiated as child maltreatment, as in civil court cases, while other states have a standard of probable cause or other standards.2

Most states have a statutory requirement for CWS services to decide whether or not to substantiate, once an investigation of maltreatment has been completed. State statutes direct the CWS to conduct an investigation with standardized components (e.g., child interview when the youngster’s developmental level permits, interview with caregivers, home visit, interview with alleged perpetrator) and to complete it within a specified period, which ranges from 10 days to 4 weeks or longer, depending on the state.2 The investigating caseworker, typically in consultation with a supervisor, then decides whether or not the allegations of maltreatment are substantiated. A minority of states have a third category, usually called indicated, which means that some evidence of maltreatment exists, but not enough for substantiation.

Despite the importance of the substantiation decision, there are few data on how caseworker judgments influence the substantiation process. In this research brief, we examine the relationship of caseworker judgments to the substantiation decision, using data from the National Survey of Child and Adolescent Well-Being (NSCAW). Specifically, we address the following questions:

• How does the degree of perceived harm to the child relate to the decision?

• How important is the caseworker’s perception of the severity of future risk to the child?

• How does the caseworker’s judgment about the strength of the evidence relate to the decision?

National Sample of Cases Involving Allegations of Maltreatment

NSCAW is a national longitudinal study of 5,501 children and youths whose allegations of maltreatment were investigated or assessed by CWS services in 1999 and 2000. Though this analysis focuses on caseworker data, interviews were also conducted with children and caregivers, and teachers completed questionnaires.4

This analysis is based on the 4,514 children who were involved in investigations in which a decision was made about whether or not to substantiate. (In a minority of cases, either there was no substantiation determination, or data on substantiation were missing from the caseworker data set.)

Substantiation data were gathered at the NSCAW baseline interview with the caseworker, which was conducted 4 to 5 months after the completion of the investigation. Caseworkers responded according to their experience with the case, as well as their review of case records. Caseworkers were first asked the outcome of the case with regard to substantiation. They were then asked the following three questions, each of them prefaced by the phrase “regardless of the outcome of the investigation”:

• How would you describe the level of harm to [child’s name]? Would you say none, mild, moderate, or severe?

• How would you describe the level of severity of risk? Would you say none, mild, moderate, or severe?

• How sufficient was the evidence to substantiate the case? Would you say . . . no evidence of maltreatment, evidence clearly not sufficient, probably not sufficient, probably sufficient, or evidence was clearly sufficient?

Questions were clearly framed as historical information about the investigation. Nevertheless, the time interval between the substantiation decision and the caseworker interview may have introduced some measurement error. It is possible that the substantiation decision affected caseworkers’ retrospective judgments, despite the instruction to respond “regardless of the outcome of the investigation.” Also, new information, learned after the substantiation decision, may have influenced caseworkers’ memories. However, given that the different sources of possible measurement error might lead to either higher or lower scores on harm, risk, and evidence, we see no reason to expect a particular bias.

What Percentage of Maltreatment Cases Are Substantiated?

As seen in Figure 1, almost one third (29.7%) of investigated cases were substantiated. This rate is close to that produced by administrative data in the National Child Abuse and Neglect Data System.5 A small number of cases (8.2%) were neither substantiated nor unsubstantiated, but instead had a disposition of indicated.

[D]

There were surprisingly few differences in substantiation rates by characteristics of the child. Substantiation rates did not differ by race/ethnicity. There were also no differences by the type of alleged maltreatment (e.g., physical abuse, sexual abuse, neglect) or by urban as opposed to rural residence.

Maltreatment allegations against girls were more likely to be substantiated than those against boys (34.2%, as opposed to 25.4%). Substantiation rates also varied significantly by the age of the child, although this variation did not follow a straightforward pattern. For children aged 2 or younger, 31.8% of reports were substantiated, compared with 20.8% for children aged 3 to 5, 35.1% for those aged 6 to 10, and 27.2% for youths aged 11 to 14.

How Do Caseworkers Assess Harm, Risk, and Sufficiency of Evidence Overall?

Figure 2 presents the frequency distributions of caseworker assessment of harm and future risk to the child. These assessments are for all cases, both substantiated and unsubstantiated. The distributions are remarkably similar. Caseworkers judged that 27.6% of children had experienced moderate to severe harm and that 32.1% were at moderate to severe risk for future harm.

[D]

Figure 3 displays the frequency distribution of caseworkers’ assessments of the strength of evidence that maltreatment had occurred. One third of cases (34.3%) were assessed to have no evidence whatsoever. Almost as many cases (28.7%) were deemed to have clearly sufficient evidence of maltreatment.

[D]

What Is the Relationship of Caseworker Judgments to Substantiation?

How do caseworker assessments of perceived harm, future risk, and strength of evidence relate to the substantiation decision? Figure 4 examines the relationship of substantiation rates to how caseworkers rated the severity of harm to the child and rated the strength of the evidence. The vertical axis shows the substantiation rate, and the horizontal axis shows the four categories of harm: none, mild, moderate, and severe. The three lines plot the relationship between these two as a function of the strength of the evidence: the dark blue line indicates the evidence was clearly sufficient, the light blue indicates the evidence was probably sufficient, and the green indicates the range between probably insufficient and no evidence.

[D]

Substantiation rates clearly rose as the rating of the level of harm increased, particularly from none or mild harm to moderate or severe harm. Within each level of harm, substantiation rates increased with the strength of the evidence.

Consider cases in which the caseworker deemed harm to the child to be mild. With insufficient or no evidence, essentially no cases were substantiated. When the evidence was deemed probably sufficient to indicate mild harm, the likelihood of substantiation was 46.3%. For cases with clearly sufficient evidence, the substantiation rate jumped to 70.2%. However, even in the small number of cases in which the caseworker judged that there was severe harm but insufficient evidence, reports were unlikely to be substantiated.

A similar graph constructed with categories of risk, rather than harm, showed similar results. A statistical model including all of these variables showed that ratings of harm, risk, and strength of evidence were all significant predictors of substantiation, but evidence was the single most important predictor.

There are two caveats. First, one explanation for evidence’s being the strongest predictor might be the specific wording of the question, “How sufficient was the evidence to substantiate the case?” Second, caseworkers were asked about investigations 4 to 5 months after they had closed; the delay might explain some apparent inconsistencies between caseworker judgments and substantiation decisions. These are minor considerations, given the strong results showing the importance of evidence and given the finding that maltreatment reports are substantiated for most but not all children at high levels of harm or risk.

How Often Do Harm, Risk, and Evidence Match Substantiation?

The data clearly show consistency of harm, risk, evidence, and substantiation in the majority of cases and the disjunction among them in a small but important minority of cases. In nearly three quarters of cases, caseworker judgments matched substantiation reasonably well:

• 51.7% of cases had low levels of harm/risk, had little evidence of maltreatment, and were not substantiated;

• 21.6% had high levels of harm/risk, had probably sufficient to clearly sufficient evidence, and were substantiated.

Nonetheless, in a meaningful proportion of cases, caseworkers thought that children had moderate to high levels of harm or risk, but the case was not substantiated. This outcome was more common when evidence was lacking. There were also some cases in which caseworkers judged there to be little or no risk or harm but the availability of evidence of maltreatment led to substantiation. Specifically, findings were as follows:

• 5.4% of cases had moderate to high levels of caseworker-assessed harm/risk and evidence but were not substantiated;

• 8.0% had moderate to high levels of harm/risk but had probably insufficient to clearly insufficient evidence and were not substantiated; and

• 5.0% of cases had low levels of harm/risk but probably sufficient to certainly sufficient evidence and were substantiated.

What Can We Conclude About Caseworker Judgments, Substantiation, and Need for Services?

The findings are strong and clear, even when taking into account our limitations in measuring caseworker judgments. Consistent with what we know from analyses of administrative data, the majority of cases investigated by child protective services are unsubstantiated. Caseworkers take into account their judgments of harm to the child, future risk to the child, and evidence of maltreatment when they make substantiation decisions. Evidence plays an important role. Even when caseworkers believe children have been harmed or are at risk, substantiation is unlikely unless evidence of maltreatment is sufficient.

Harm, risk, and evidence do not perfectly predict substantiation, however. Some cases are not substantiated even when all these judgments are consistent with child maltreatment. This result is perhaps not surprising, because system variables such as caseworker workload and whether a state has the option of an indicated decision are related to substantiation rates.6 Both the unpredictability of substantiation decisions and their dependence on the availability of evidence support previous researchers’ conclusions that substantiation is an imperfect measure of child maltreatment.7

The finding of a general lack of differences in substantiation by racial and ethnic group is meaningful. Other research has found that African American children are overrepresented in the CWS.8 This overrepresentation does not appear to be because of differences in substantiation.

One worrisome finding is the presence of a relatively small but still meaningful proportion of children judged to be at significant risk of harm but with too little evidence for substantiation. Indeed, children who have been harmed or remain at risk without substantiation should be a group of high concern for the CWS. At least some of these cases should receive an evaluation of need for child development and well-being services, as well as family services. They also call for regular monitoring of child development and well-being. Because of such groups, most states make CWS services available in at least some cases that are not substantiated.9 Another brief in this series analyzes service delivery for children in substantiated and unsubstantiated cases.10

---

 

National Survey of Child and Adolescent Well-Being Research Brief Available at: National Data Archive on Child Abuse and Neglect (NDACAN), Cornell University, ndacan@cornell.edu Administration for Children and Families (ACF, OPRE) http://www.acf.hhs.gov/programs/opre/abuse_neglect/nscaw/ This is the sixth in a series of NSCAW research briefs focused on children who have come in contact with the child welfare system. Additional research briefs focus on the characteristics of children in foster care, the provision of services to children and their families, the prevalence of special health care needs, and the maltreatment investigation substantiation process.

 

Return to Previous page