Children's Bureau Safety, Permanency, Well-being  Advanced
 Search

Home > Statistics & Research > Child Welfare Outcomes 2002-2005: Report to Congress > Introduction

Introduction

Child Welfare Outcomes 2002–2005: Report to Congress (Child Welfare Outcomes Report) is the seventh in a series of annual reports prepared by the U.S. Department of Health and Human Services (the Department) in accordance with section 479A of the Social Security Act.17 Appendix A provides the specifications of section 479A.) The reports reflect the Department's responsibility for monitoring State performance in operating child protection and child welfare programs under titles IV-B and IV-E of the Social Security Act. The Children's Bureau is the agency within the Department's Administration for Children, Youth and Families that carries out this responsibility and prepares the Child Welfare Outcomes Reports.

In preparation for the first Child Welfare Outcomes Report, the Department consulted with State and local child welfare agency administrators, child advocacy organizations, child welfare researchers, State legislators, and other experts in the child welfare field to identify outcomes and measures reflecting widely accepted performance objectives for child welfare practice. This consultation process resulted in the following national child welfare outcomes:


This report includes data on the 12 original Child Welfare Outcomes Report measures and 15 new outcome measures developed for the second round of the Department's Child and Family Services Reviews

The first six Child Welfare Outcomes Reports provided information on 12 measures developed by the Department to assess State performance on these national outcomes. In the current report, information is presented on these 12 measures and also on 15 additional measures. The 15 additional measures were developed in 2006 to be used as part of the assessment of State performance during the second round of the Department's Child and Family Services Reviews (CFSRs), which began in March 2007.18 Although the new measures were originally developed for use during the CFSRs, the Department determined that information regarding performance on these measures should be included in the Child Welfare Outcomes Reports. The new measures assess critical performance areas relevant to the seven national child welfare outcomes, and therefore are consistent with the mission of the Child Welfare Outcomes Reports.

The Department was in the process of developing the new measures during 2005 and 2006. Because it was considered important to include these measures in all future Child Welfare Outcomes Reports, a decision was made to delay publication of the 2004 data until the measures were finalized and could be included in the report. Consequently, the 2004 data for the Child Welfare Outcomes Reports were not published in a separate report but instead are included in this report, Child Welfare Outcomes 2002–2005. This report incorporates data from 2004 and 2005 on the 15 new measures as well as data from 2002 through 2005 on the 12 original measures.

Outcome Measures

Descriptions of the 27 outcome measures included in this Child Welfare Outcomes Report are provided in appendix B. Data for both the original and new outcome measures come from the Department's two national child welfare-related data systemsthe National Child Abuse and Neglect Data System (NCANDS) and the Adoption and Foster Care Analysis and Reporting System (AFCARS). The specific NCANDS and AFCARS data elements used to calculate each outcome measure are presented in appendix C

Through NCANDS, the Department collects and analyzes State data pertaining to children who come into contact with public child protective services agencies as alleged victims of child abuse or neglect. Data from this system are published annually in a report entitled Child Maltreatment. The highlights of Child Maltreatment 2005 are presented in appendix D. The Child File is the component of NCANDS that is the primary data source for the safety-related outcome measures included in the Child Welfare Outcomes Reports. The NCANDS Child File is a data file that States submit annually on a voluntary basis. The file contains detailed case information about children who are the subjects of an investigation or assessment in response to a maltreatment allegation. A Child File is established for all children associated with reports that have received a disposition during the reporting period. Although the disposition usually refers to a "finding" regarding the allegation, it also can include those reports that were "closed without a finding." AFCARS, which is a mandatory data system for States, is the Department's national system for collecting, analyzing, and reporting data on children who are served by State foster care systems. Appendix E presents the annual AFCARS report for 2005.

The availability and quality of data for the Child Welfare Outcomes Reports have improved substantially over the past 6 years.

The quality and availability of data for the original outcome measures have improved extensively since the publication of the first Child Welfare Outcomes Report—Child Welfare Outcomes 1998. For example, 49 States submitted an NCANDS Child File for 2005 compared to 22 States submitting this file for 1998. Similarly, 52 States reported sufficient 2005 data to AFCARS to calculate the 12 original AFCARS-related outcome measures, compared with 33 States providing sufficient 1998 data for all measures. This progress may be attributed to substantial State efforts over the past several years to improve their information systems, as well as technical assistance provided by the Department in the area of information systems and data quality. In addition, as States improved their data information systems, they resubmitted corrected data for prior years, thus increasing the accuracy of the data for all of the years included in the report. Although some data quality issues remain in a few States, the data presented in Child Welfare Outcomes 2002–2005 can be interpreted with greater confidence than ever before.

The 15 new measures included in the current Child Welfare Outcomes Reports were developed in response to recommendations regarding potential improvements to the CFSRs. In the first round of the CFSRs (conducted from 2001 to 2004), six national data measures were used as part of the assessment of State performance. National standards were established for each of the six measures and States were expected to meet the national standards (as well as case review requirements) in order to be in substantial conformity with the particular outcome being assessed.

After completion of the first round of the CFSRs in 2004, the Department contracted with a consultant to convene a workgroup to consider potential improvements to the CFSR process. One of the improvements suggested by the workgroup was to replace the six single data measures used in the first round of the CFSRs with data composites incorporating multiple measures addressing a particular program area. The Department agreed that a data composite approach would provide a more comprehensive picture of State performance and began a process for developing data composites to be used for the CFSRs.19 The end products of this process were four data composites, incorporating a subset of 15 individual measures pertaining to permanency and placement stability for children in foster care. Composite scores were generated for the four data composites and national standards were established for the composite scores, but not for the individual measures. The measures subsumed under each composite are detailed in appendix B.

No data composites were developed pertaining to children's safety. After careful consideration and analyses of several measures, the Department determined that data composites in the area of safety were not possible given the available data elements and, in some situations, difficulty interpreting performance. However, for the CFSRs, two minor changes were made in the two existing safety measures. Maltreatment in foster care is now evaluated based on 12 months of data rather than 9, and both safety measures are stated and evaluated in the positive direction (Absence of Maltreatment in Foster Care and Absence of Recurrence). The Child Welfare Outcomes Report incorporates the 12-month framework for the maltreatment in foster care measure, but continues to use the original wording for the measures.20

Context Data in the Child Welfare Outcomes Report

In addition to data pertaining to State performance on the outcome measures, the Child Welfare Outcomes Reports present data on child welfare-related contextual factors relevant to understanding and interpreting State performance on the outcome measures.21 The context factors include the following:

Data Analyses in the Child Welfare Outcomes Report

The Child Welfare Outcomes Report presents key findings of analyses conducted across States. These findings pertain to variations across States in performance on the outcome measures, change in performance on the measures over time, and the relationships between contextual factors and State performance.

Change in State performance over time was assessed by calculating a percent change in performance on the measures.26 Using a percent change calculation permits an assessment of the extent of change occurring over time by taking into account the size of the percentages being compared. For example, a change from 6 percent in 2002 to 7 percent in 2005 represents a change of only 1 percentage point; however, it reflects a +16.7 percent change. In contrast, a change from 75 percent to 80 percent represents a change of 5 percentage points, but only a +6.7 percent change. Consistent with the Department's approach in prior Child Welfare Outcomes Reports, a percent change of 5.0 or greater in either direction was used as a general indicator that meaningful change occurred. That is, for purposes of the analyses presented in the Child Welfare Outcomes Report, if the percent change in performance from 2002 to 2005 was less than 5.0 in either direction (i.e., positive or negative), a determination was made that there was no change in performance.

Relationships between measures and context variables were assessed using correlation coefficients, specifically Pearson's r. This coefficient can range from -1 to +1. In the Child Welfare Outcomes Report, these coefficients are interpreted in accordance with J.P. Guilford's suggested interpretations for correlation coefficient values.27 These are as follows:

Organization of the Child Welfare Outcomes Report

Child Welfare Outcomes 2002–2005: Report to Congress is organized as follows:

Chapter I: Keeping Children Safe. This chapter focuses on the findings of the data analyses pertaining to State performance on the two outcome measures assessing children's safety and the relationships between performance on these measures and various context variables.

Chapter II: Finding Permanent Homes for Foster Children. In this chapter, findings are presented for the nine measures that address the general issue of State performance in achieving permanency for children. Two of these measures address the issue of the length of time children spend in foster care when they are emancipated without having found a permanent home.

Chapter III: Timeliness of Reunifications and Adoptions for Foster Children. In this chapter, data are presented and discussed concerning State performance on the timeliness of adoptions and reunifications.

Chapter IV: Achieving Stable and Appropriate Placement Settings in Foster Care. This chapter presents and discusses key findings pertaining to the placement stability of children in foster care for varying periods of time and the placement settings for children who enter foster care when they are age 12 or younger.

Chapter V: State Performance Relevant to the Seven National Child Welfare Outcomes. This chapter includes data for each State on each of the measures and context variables. It also includes a comment from the State, if one was provided, and a brief overview of performance.

Appendices

17 All years indicated in the Child Welfare Outcomes Report reflect Federal fiscal years (October 1-September 30), unless otherwise noted. Back

18 Complete information on the CFSR process is provided on the Children's Bureau's website at http://www.acf.hhs.gov/programs/cb/cwmonitoring/index.htm#cfsr. Back

19 Because the details about this process and about the composites, including the statistical methodology used, and feedback received from the field are provided in the Federal Register Announcement posted on the Children's Bureau's website, they will not be reviewed in this report. See the following website for the Federal Register Announcement regarding the data indicators for the second round of the CFSRs. http://www.acf.hhs.gov/programs/cb/cwmonitoring/legislation/fed_reg.htm. Back

20 Detailed information about the safety-related measures to be used in the second round of the CFSRs is provided in the Federal Register Announcement posted on the Children's Bureau's website, http://www.acf.hhs.gov/programs/cb/cwmonitoring/legislation/fed_reg.htm. Back

21 For purposes of this report, the term "State" refers to the 50 States, the District of Columbia, and Puerto Rico. Therefore, the report provides information on 52 States. Back

22 These data come from the Census Bureau and reflect estimates rather than actual numbers. Back

23 Child Maltreatment 2005 provides the following definitions: (1) Substantiation refers to a type of investigation disposition that concludes that the allegation of maltreatment or risk of maltreatment was supported or founded by State law or State policy; (2) Indicated refers to an investigation disposition that concludes that maltreatment cannot be substantiated under State law or policy, but there was reason to suspect that the child may have been maltreated or was at risk of maltreatment. This is applicable only to States that distinguish between substantiated and indicated dispositions. Back

24 The term "foster care" as it is used in the Child Welfare Outcomes Report refers to a variety of out-of-home placement settings, including foster family homes, group homes, shelters, residential treatment facilities, and similar placements for children who are under the care and supervision of a State child welfare agency for at least 24 hours. Back

25 There is no Federal definition for a child waiting to be adopted. The definition used in the Child Welfare Outcomes Reports includes children who have a goal of adoption and/or whose parental rights have been terminated. It excludes children 16 years of age or older whose parental rights have been terminated and who have a goal of emancipation. This definition may differ from definitions used by individual States. Back

26 Percent change is calculated by subtracting old data from new data, dividing by old data and multiplying by 100. For example, if maltreatment recurrence was 9.2 in 2002 and 7.6 in 2005, the percent change would be [(7.6-9.2)/9.2] x 100 = -17.4 percent change. Back

27 Guilford, J. P. (1956). Fundamental statistics in psychology and education (3rd ed.). New York: McGraw-Hill. Back

 



Return to Table of Contents

 << Executive Summary | Chapter I >>