ACYF-CB-IM-01-07

ACF Administration for Children and Families	U.S. DEPARTMENT OF HEALTH AND HUMAN SERVICES Administration on Children, Youth and Families
	1. Log No.: ACYF-CB-IM-01-07	2. Issuance Date: August 16, 2001
	3. Originating Office: Children's Bureau
	4. Key Words: National Standards, Program Improvement Plans, Child and Family Services (CFS) Reviews

INFORMATION MEMORANDUM

TO:	State and Territorial Agencies Administering or Supervising the Administration of Titles IV-B and IV-E of the Social Security Act, ACF Regional Administrators and Hub Directors
SUBJECT:	Updated National Standards for the Child and Family Service Reviews and Guidance on Program Improvement Plans
LEGAL AND RELATED REFERENCES:	Titles IV-B and IV-E of the Social Security Act, the Social Security Act Amendments of 1994 (Public Law 103-432), 45 CFR 1355.34(a) and (b), 45 CFR 1355.35(a) and (e), 45 CFR 1355.36(d), ACYF-CB-IM-00-11, and ACYF-CB-IM-01-01
PURPOSE:	The purposes of this information memorandum (IM) are: (1) to provide updated information on the national standards that are used to determine, in part, whether or not States are operating in substantial conformity with State plan requirements, and; (2) to provide guidance for use by States and Regional Offices in negotiating the amount of improvement necessary toward meeting the national standards through an approved program improvement plan (PIP).

INFORMATION:

Background

In accordance with 45 CFR 1355.34(a) and (b), a State must, in part, meet national standards for certain statewide data indicators and achieve certain outcomes for children and families to be determined in substantial conformity in a child and family services review. In ACYF-CB-IM-00-11¹ we transmitted national standards for the six statewide data indicators used, in part, to determine if States are operating in substantial conformity with titles IV-B and IV-E of the Social Security Act (the Act). We have updated the national standards because of corrected State data that form the basis for the standards and a number of improved procedural measures that we describe below.

In addition, if a State is determined not to be in substantial conformity because the State's data did not meet the national standards, the State is required to develop a PIP (45 CFR 1355.35(a)) that sets forth the amount of progress the State will make toward meeting the national standards for the statewide data indicator(s). In addition to updating the national standards, we are providing guidance for use by States and Regional Offices in determining the amount of improvement that States must make toward meeting the standards through the implementation of a PIP. We note that a PIP is required of States determined not to be in substantial conformity on any of the outcomes and systemic factors examined as a part of the child and family services reviews. A State may be determined not to be in substantial conformity for reasons other than failure to meet the national standards, such as the failure of a sufficient number of cases reviewed onsite to be rated as substantially achieved. This Information Memorandum addresses the extent of improvement that will be required of States through a PIP due to determinations that States are not operating in substantial conformity based in whole or in part on the failure of the State to meet the national standards, and does not address States' failure to achieve substantial conformity due to other reasons.

Updated National Standards

The updated national standards reflect changes in four areas:

The use of corrected State data submitted to AFCARS and NCANDS for the time periods used to calculate the standards;
The application of new rules that disregard skewed data points in calculating the national standards;
The inclusion of a sampling error associated with each national standard; and
The rounding of the national standards to one decimal point.

We discuss each of these changes below.

Corrected State data. In preparing for the CFS reviews, a number of States examined their data submissions to AFCARS and NCANDS in previous years and re-submitted their data for purposes of the CFS reviews. We encouraged States to analyze their data critically so that the reviews would reflect the most accurate picture available of outcomes for children and families. Therefore, we have incorporated the corrected data into the national standards.

New rules for disregarding extreme data points. When recalculating the national standards, we applied new rules that disregarded extreme data points in individual States' data that were used to calculate the national standards. We excluded State data indicators that were 0% and 100% from the data points used to determine the standards, believing that such extreme values were unlikely to be accurate and would skew the standards in one direction or another.

Inclusion of sampling error in the standards. In calculating the new national standards, we included the sampling error associated with each data indicator as part of the actual standard. Even with the use of more accurate data to calculate the national standards, we realize that there is a certain range within which a State's data can be considered to meet the standard statistically. We chose to incorporate the sampling error into the standards because we do not believe it is appropriate to subject a State to a potential penalty if the State is within a statistically acceptable range of the national standard.

Rounding of the national standards. When we updated the national standards to incorporate sampling errors, we believed that the full benefits to the States of including the sampling error in the standard would be best applied by rounding the standards using decimal points. We chose to round the standards to only one decimal point because we do not believe it is appropriate to attempt to measure State performance to the degree of precision implied by two decimal places. The exception to this rounding procedure is the incidence of child abuse and/or neglect in foster care and has been rounded to two decimal points due to the small number of such incidents reported by States.

Determining a State's Achievement of the National Standard

In determining whether or not a State meets the national standard for a particular statewide data indicator, we will compare the State's data used in its CFS review to the updated national standards below. We will round the State's data for individual indicators using the same rationale described above for rounding the national standards, i.e., rounding to one decimal point for each of the six indicators except the incidence of child abuse and/or neglect in foster care, which will be rounded to two decimal points. As shown in the chart below, each of the updated national standards works to the advantage of the States in meeting the standards. For States that have already undergone CFS reviews in FY 2001, we will apply the updated national standards retroactively to decisions regarding substantial conformity.

Our methodology for calculating the updated national standards and for determining whether a State's data meet the national standards are indicated in the chart below:

Data Indicator	75^th Percentile	Sampling Error	National Standard^*	Old National Standard (For comparison only)	Method of Determining if State Meets National Standard
Recurrence of maltreatment	5.15%	0.90%	6.1%	5%	A State will be determined to meet the national standard if its data on this indicator are equal to, or less than, the updated national standard.
Incidence of child abuse and/or neglect in foster care	0.4247%	0.1439%	0.57%	0.4%	A State will be determined to meet the national standard if its data on this indicator are equal to, or less than, the updated national standard.
Foster care re-entries	7.23%	1.35%	8.6%	6%	A State will be determined to meet the national standard if its data on this indicator are equal to, or less than, the updated national standard.
Stability of foster care placements	88.58%	1.90%	86.7%	89%	A State will be determined to meet the national standard if its data on this indicator are equal to, or greater than, the updated national standard.
Length of time to achieve reunification	78.62%	2.42%	76.2%	78%	A State will be determined to meet the national standard if its data on this indicator are equal to, or greater than, the updated national standard.
Length of time to achieve adoption	34.89%	2.90%	32.0%	36%	A State will be determined to meet the national standard if its data on this indicator are equal to, or greater than, the updated national standard.

* Rounded as noted above Back

Determining a State's Improvement with Regard to the National Standards

In situations where a State does not meet a national standard, the regulations require that the PIP, which is jointly developed by the Regional Office and the State, set forth the amount of progress the statewide data will make toward meeting the national standards. (45 CFR 1355.35(a)(1)(i),(iv)). The PIP must also specify a period in which ACF can measure a State's progress in achieving the negotiated improvement (45 CFR 1355.5(e)(1)). We will consider the sampling error for each indicator as a minimum percentage of improvement for a State to make over the course of a PIP. For example, if a State's recurrence of child maltreatment is 10% at the time of the review, at a minimum we would expect the State to improve by at least .90% on this indicator as part of an approved PIP, thus reducing the incidence of recurrent maltreatment to 9.1%, in order to consider that improvement has been made. We believe that using the sampling error as a framework is appropriate because where a State's improvement exceeds the sampling error, we can conclude that a statistically significant improvement has been made.²

In addition to the sampling error, the Regional Office may also consider other programmatic factors in negotiating the amount of improvement States are required to make in a PIP. Such programmatic factors may include the extent to which a State's data fall below or exceed the national standard, the factors affecting the State's lack of substantial conformity, the difficulty and time involved in program improvement efforts in achieving overall improvement, whether the area needing improvement is an egregious area affecting child safety, and the amount of performance improvement the State may have already achieved subsequent to generating the original data profiles used in the reviews.

Evaluating a State's Improvement in Meeting the National Standards

To determine the amount of progress a State has made toward meeting the national standard, we will compute the difference between the State's data as indicated in the original data profile and the subsequent period specified in the PIP. For example, if a State fails to meet a national standard based on data from Federal fiscal year 1999, and the PIP includes the negotiated provision that we use data from Federal fiscal year 2003 to gauge the State's progress toward meeting the national standard, we will compute the difference between the data from those two time periods.

If the difference indicates a level of improvement that meets or exceeds the approved amount of improvement specified in the PIP, we will determine that the State has successfully completed that portion of the PIP and the associated penalties will be rescinded (section 1355.36(d)). If the State does not meet or exceed the approved amount of improvement specified in the PIP, the associated penalties will be withheld as specified in 45 CFR 1355.36. States that do not achieve the updated national standard in subsequent reviews will again be determined out of substantial conformity and be required to establish new benchmarks of improvement to be made toward attainment of the national standard, using the same procedures described in this Information Memorandum³ and subject to the penalty provisions in 45 CFR 1355.36.

INQUIRIES TO: ACF Regional Offices

____________________________
James A. Harrell
Acting Commissioner
Administration on Children, Youth and Families

¹ ACYF-CB-IM-00-11 was later amended by ACYF-CB-IM-01-01 due to a typographical error in describing our methodology for calculating the national standard for the statewide data indicator, "foster care re-entries." We are rescinding both ACYF-CB-IM-00-11 and ACYF-CB-IM-01-01 due to the revisions in the national standards and are placing a background paper on the Children's Bureau website that includes relevant information from the two Information Memoranda noted above. Back
² In situations where the amount of a State's improvement on an indicator exceeds the sampling error, we can determine that improvement has occurred in a statistical sense, since exceeding the sampling error will indicate a statistically significant difference (improvement) between the State's performance on the period under review and the subsequent evaluation period specified in the PIP. Back
³ More information on developing program improvement plans can be found in chapter 7 of the CFS review procedures manual. Back