Summary
In 2001, the Department of Health and Human Services' (HHS) Administration for Children and Families (ACF) implemented the Child and Family Services Reviews (CFSR) to increase states' accountability. The CFSR uses states' data profiles and statewide assessments, as well as interviews and an on-site case review, to measure state performance on 14 outcomes and systemic factors, including child well-being and the provision of caseworker training. The CFSR also requires progress on a program improvement plan (PIP); otherwise ACF may apply financial penalties. This report examines (1) ACF's and the states' experiences preparing for and conducting the statewide assessments and on-site reviews; (2) ACF's and the states' experiences developing, funding, and implementing items in PIPs; and (3) any additional efforts that ACF has taken beyond the CFSR to help ensure that all states meet federal goals related to children's safety, permanency, and well-being.
ACF and many state officials perceive the CFSR as a valuable process and a substantial undertaking, but some data enhancements could improve its reliability. ACF staff in 8 of the 10 regions considered the CFSR a helpful tool to improve outcomes for children. Further, 26 of 36 states responding to a relevant question in our survey commented that they generally or completely agreed with the results of the final CFSR report, even though none of the 41 states with final CFSR reports released through 2003 has achieved substantial conformity on all 14 outcomes and systemic factors. Additionally, both ACF and the states have dedicated substantial financial and staff resources to the process. Nevertheless, several state officials and child welfare experts we interviewed questioned the accuracy of the data used in the review process. While ACF officials contend that stakeholder interviews and case reviews complement the data profiles, many state officials and experts reported that additional data from the statewide assessment could bolster the evaluation of state performance. Program improvement planning is under way, but uncertainties have affected the development, funding, and implementation of state PIPs. Officials from 3 of the 5 states we visited said ACF's PIP-related instructions were unclear, and at least 9 of the 25 states reporting on PIP implementation in our survey said that insufficient funding and staff were among the greatest challenges. While ACF has provided some guidance, ACF and state officials remain uncertain about PIP monitoring efforts and how ACF will apply financial penalties if states fail to achieve their stated PIP objectives. Since 2001, ACF's focus has been almost exclusively on the CFSRs and regional staff report limitations in providing assistance to states in helping them to meet key federal goals. While staff from half of ACF's regions told us they would like to provide more targeted assistance to states, and state officials in all 5 of the states we visited said that ACF's existing technical assistance efforts could be improved, ACF officials acknowledged that regional staff might still be adjusting to the new way ACF oversees child welfare programs.
Recommendations
Our recommendations from this work are listed below with a Contact for more information. Status will change
from "In process" to "Implemented" or "Not implemented" based on our follow up work.
Director:
Team:
Phone:
Kay E. Brown
Government Accountability Office: Education, Workforce, and Income Security
(202) 512-8403
Recommendations for Executive Action
Recommendation: To ensure that ACF uses the best available data in measuring state performance, the Secretary of HHS should expand the use of additional data states may provide in their statewide assessments and consider alternative data sources when available, such as longitudinal data that track children's placements over time, before making final CFSR determinations.
Agency Affected: Department of Health and Human Services
Status: Implemented
Comments: HHS has several steps underway to address necessary data improvements. With the agency's assistance, states have begun to submit more accurate information in their Adoption and Foster Care Analysis and Reporting System (AFCARS) and National Child Abuse and Neglect data profiles. In May 2005, HHS reported that revisions to AFCARS were being considered as part of an overall effort to evaluate that system. In 2006 and 2007, HHS reported that a notice of proposed rulemaking to update AFCARS was on the agency's unified agenda. On January 11, 2008, HHS posted a notice of proposed rulemaking (NPRM) proposing numerous improvements to AFCARS data, including changes to the reporting populations, data structure, elements, compliance determination and penalty provisions. In 2006, ACF announced new data measures, data composites, and national standards that will replace the six national data standards used during the first round of Child and Family Services Reviews (CFSRs) to assess the performance of State child welfare agencies (71 FR 32969, 6/7/2006). According to the notice, the goals of the data composites are to provide a more holistic view of State performance in a particular domain, account for both the strengths and weaknesses of a State, allow for the development of national standards that account for variations in State practices and policies, and provide a more effective assessment of State performance.
Recommendation: In addition, to ensure that ACF regional offices and states fully understand the PIP development, approval, and monitoring processes, and that regional offices fully understand ACF's prioritization of the CFSR as the primary mechanism for child welfare oversight, the Secretary of HHS should take the following two actions: (1) issue clarifying guidance on the PIP process and evaluate states' and regional offices' adherence to this instruction and (2) provide guidance to regional offices explaining how to better integrate the many training and technical assistance activities for which they are responsible, such as participation in state planning meetings and the provision of counsel to states on various topics, with their new CFSR responsibilities.
Agency Affected: Department of Health and Human Services
Status: Implemented
Comments: According to HHS officials, the agency has continued to provide technical assistance and training to states and regional offices when appropriate. It is committed to continually assessing and addressing training and technical assistance needs. In July 2004, the Administration for Children and Families (ACF) reported that the Children's Bureau planned to conduct training specifically related to the development, implementation, and monitoring of program improvement plans in each ACF regional office that will also include state staff. In addition, ACF reported in July 2005 that ten CFSR-related trainings have occurred over the course of this fiscal year, which benefit both regional and central office staff. In 2007, HHS reported that training with regional staff is ongoing and that ACF provided training in every region in FY 2006 related to implementing Round 2 of the FY 2007 CFSRs. ACF published a CFSR procedures manual in November 2006, which includes a chapter on the Program Improvement Plan (PIP) development and monitoring process and a PIP content checklist. In addition, the Children's Bureau posted PIP instructions and a matrix, last updated in October 2007, which are intended to provide guidance for States in developing a PIP following receipt of the Final Report on the child and family services review. This guidance document presents a suggested format for preparing a PIP and outlines the content or elements that must be included in the PIP regardless of the format used by the State, as described in the regulations.
