|
|---|
|
|
||||
|---|---|---|---|---|
|
|
|
|||
| ACF Home | Services | Working with ACF | Policy/Planning | About ACF | ACF News | HHS Home | ||||
Questions?
|
Privacy
|
Site Index
|
Contact Us
|
Download Reader™
|
Print
 �
|
|---|
Section I of this Technical Bulletin contains information on ACF's approach to the second round of Child and Family Services Reviews (CFSRs) and to evaluating States' achievement of the goals outlined in the Program Improvement Plans (PIPs) developed after the first round of CFSRs. Section II contains definitions and technical information.
A. Procedure for Scheduling the Second Round of CFSRs
ACF is currently planning a process for scheduling the second round of CFSRs, now that we have completed the initial CFSR in all 50 States, Puerto Rico, and the District of Columbia (referred to hereafter as "the States"). Federal regulations at 45 CFR 1355.32 (b)(2) (ii) require that States that were not in substantial conformity in the initial review begin a full review two years after approval of their Program Improvement Plan (PIP). This requirement applies to all States that experienced an initial CFSR.
We will consider two interrelated factors in determining how States can best comply with this requirement for a second review. The first factor is the availability of data from the two Federal data sources used in the CFSR, including the States' submissions to the Adoption and Foster Care Analysis and Reporting System (AFCARS) and the National Child Abuse and Neglect Data System (NCANDS). The second factor is our desire to assure that States have had the opportunity to implement fully all provisions of their approved PIPs during the two years allowed by Federal regulation.
As a result, we will not use data for the second CFSR that overlap with the prior PIP implementation period in an effort to assure that States have the full two-year PIP period to implement programmatic changes. At the same time, we recognize the need to avoid any unnecessary delays in proceeding with the second round of CFSRs. Therefore, we will use AFCARS data for a State's second CFSR that encompass a rolling annual file based on two consecutive AFCARS submissions, beginning with the next six-month AFCARS period that immediately follows the end of a State's PIP implementation period. Thus, we will be able to construct an annual AFCARS file every six months for two consecutive 6-month periods ending on March 31st or September 30th. These data will be used to generate all permanency related aspects of a State's "Data Profile (DP)" including the denominator for the maltreatment in foster care measure. Although NCANDS data are officially submitted for the 12-month period ending September 30th of each year, those data will not be used for these processes because using the March 31st due date will delay the start of the subsequent CFSR unduly. Instead, as applicable, for 12-month periods ending September 30th, States will be asked to make an additional NCANDS data submission voluntarily for these 12-month periods by January 1st. As applicable, for the 12-month period ending March 31st, States will be asked to make an additional NCANDS data submission voluntarily for these 12-month periods by July 1st. Therefore, the time periods for the rolling annual AFCARS files and the 12-month NCANDS files will coincide. Technical information on this process is included in the attachment. (Note: Twelve months, as used in this text, will always refer to 4 consecutive quarters of the FFY.)
Following these procedures, ACF will schedule the beginning of each State's second CFSR statewide assessment at the earliest possible date following the completion of its PIP and the preparation of data profiles based on 12 months of data as specified above. We will schedule the reviews over the FFY in a manner that allows us to manage the review process effectively.
B. Evaluating States' Attainment of Goals with Regard to Data Indicators in the PIP
This section applies to PIPs resulting from the initial round of CFSRs. In particular, it only pertains to evaluating States' progress toward meeting the PIP goals related to the national data standards included in the CFSR, as listed in the attachment. There are two situations where determinations of progress toward attaining these PIP goals may be addressed. The first situation occurs when States meet their PIP targets prior to PIP closeout, or have target dates specified in the PIP that occur prior to the end of the PIP. The second occurs at the time that the State's 2-year PIP implementation period ends.
States should use the most accurate and current AFCARS and NCANDS data as a basis for establishing baselines for implementing their PIP. In approving States' PIPs, we require that the agreed-upon amount of improvement (AAI) with regard to the national standards be achieved by the end of the PIP implementation period or, if stated specifically in a State's PIP, at an earlier date. In the latter case, as well as when the State meets a target goal before the end of the PIP implementation period, if ACF verifies that States did achieve the AAI specified in their PIPs with regard to any of the six data indicators, we will consider the State to have satisfied that requirement. States and ACF will determine what data will be submitted to ACF for verification on a case-by-case basis.
At the conclusion of States' PIPs (PIP closeout), if States have not attained their AAI with regard to any of the applicable data indicators, ACF will determine AAI attainment by using the data covered by the same twelve-month period of time described in Section I above that ACF will use to begin the subsequent CFSR. This provision will provide States with the full two-year PIP period to implement all required program improvements, followed by a one-year evaluative period with non-overlapping data following PIP closure. However, States may also submit two consecutive 6-month periods of AFCARS data or four consecutive quarters of NCANDS data, or other data approved by the Children's Bureau, as evidence of the State having met the requirement for a particular data indicator, earlier than the end of this non-overlapping 12-month period, including during the 2-year PIP implementation period and extending through the 12-month non-overlapping period following the end of the PIP. ACF will accept such data submissions as evidence of the State having met the requirement for attaining the AAI with regard to a particular data indicator provided the data are verified or reproduced by ACF. In such a situation, where the State can demonstrate that it has met its AAI with regard to a particular data indicator, there will be no need to wait for the conclusion of a 12-month period of non-overlapping data to determine that the State has met its goal.
ACF expects that any concerns that a State or ACF have regarding the quality and accuracy of States' data will be identified and resolved prior to the end of the non-overlapping twelve-month period, as States will have up to a full 12 months of non-overlapping data following the completion of the PIP to determine if they have met their goals. This will necessitate States' resubmitting their data no later than the end of this period since ACF will use the data submitted at the end of this period to make the final determination of States' compliance with the provisions of their PIPs, rather than delay the determination further.
If a State's PIP requires that it attain the AAI prior to the end of the two-year PIP implementation period, rather than at the end of the PIP period, ACF will use the same principles outlined in this Technical Bulletin to provide the opportunity, as needed, for a year of non-overlapping data following the required completion date in the PIP to determine if the State achieved its goal for that particular indicator.
Thus, ACF will use an entire 12-month period of data covering four quarters to determine if States did attain the AAI for their remaining data indicators. These 12-month periods of data will be used both for monitoring improvement over the course of the PIP and, at the conclusion of the PIP implementation period, for closeout purposes, unless a previously approved PIP includes the use of a different timeframe for measurement. The rationale for this is as follows:
In evaluating improvement for the four permanency data indicators (PDI) in the CFSR, both during the PIP implementation period and at the conclusion of the PIP period, if applicable, ACF will use AFCARS data that represent 12 consecutive months covering four quarters, or will verify State-generated data replicating AFCARS submissions representing 12 consecutive months covering four quarters. The 12-month data period will correspond to 2 consecutive AFCARS submissions, unless a State's PIP specifies evaluating improvement on a quarterly basis, in which case ACF will use data from the quarter in which the evaluation is made and the preceding 3 quarters.
There are two applicable processes for evaluating improvement in the two safety data indicators (SDIs). The first process concerns AAI verification of the SDIs that can be computed from States' case-specific NCANDS Child Files (CFs). Depending on the quarter during which AAI achievement was attained, States will voluntarily submit these files for the 12-month periods ending on December 31st, March 31st, June 30th, or September 30th on April 1st, July 1st, October 1st, or January 1st, respectively, allowing approximately 90 days after the end of a quarter to transpire for data completeness. The second process concerns PIP closure. To be in concert with the procedures for PIP closure related to the four PDI, States whose SDIs were computed from their case-specific NCANDS CFs and did not achieve their AAI during the PIP implementation period will be evaluated for the 12 month period of non-overlapping data following completion of the PIP, ending on March 31st or September 30th. As a result, States will voluntarily submit their NCANDS CFs on July 1st or January 1st, respectively, allowing approximately 90 days to transpire for data completeness. Again, if a State's PIP specifies evaluating improvement on a quarterly basis, ACF will use data from the quarter in which the evaluation is made and the preceding 3 quarters. For monitoring quarterly improvement in their SDIs during and after the PIP implementation period, States are encouraged to generate their SDIs on a quarterly basis. Technical information on generating these data is attached.
In calculating the SDI, maltreatment in foster care, ACF will be able to generate corresponding AFCARS data for the denominator based on States' AFCARS submissions.
NOTE: The information in this Technical Bulletin regarding States generating their own safety data, as well as the expedited time frames for submitting the data, pertains only to those States and those situations where ACF will need to use the data either to determine improvement toward meeting the goals of the PIP or in preparing data profiles for the next CFSR. In all other situations, the established time frames and procedures for submitting NCANDS data will continue to apply. In addition, in these situations where States make early submissions for the next CFSR, or calculate the data on an interim basis for evaluating improvement in the PIP, ACF will continue to expect States to make their regularly scheduled NCANDS submissions. We will accept re-submissions of corrected or more complete data for use in the CFSR, as we have done in the past, if a State believes that the data it submits according to the expedited time frames in this Technical Bulletin are inaccurate although, as noted in Section I-B of this Technical Bulletin, we expect States to resolve data quality concerns prior to the end of the 12-month non-overlapping period following PIP completion in order not to delay the start of the second CFSR or final closeout of the PIP.
Note also that other goals and activities included in PIPs must also be completed and evaluated before PIPs are closed out. While this Technical Bulletin primarily addresses the procedures for determining if a State has met its PIP goals toward meeting the national standards in the CFSR, the same principle will apply to States' submission of the results of qualitative case reviews, where such measures are included in an approved PIP to determine progress toward reaching goals other than those associated with the data indicators with national standards. That is, States may submit twelve consecutive months of case review results through the end of the 12-month overlapping period following the end of the PIP, or as provided for in the approved PIP.
A. Definitions:
Federal fiscal year (FFY) - October 1, YYYY through September 30, YYYY
Federal Quarter 1 (FQ1) - October 1, YYYY through December 31, YYYY
Federal Quarter 2 (FQ2) - January 1, YYYY through March 31, YYYY
Federal Quarter 3 (FQ3) - April 1, YYYY through June 30, YYYY
Federal Quarter 4 (FQ4) - July 1, YYYY through September 30, YYYY
Intervening Federal Quarter (IFQ) - Quarter ending December 31st (FQ1) or Quarter ending June 30th (FQ3)
"A" Period (AFCARS "A" submission) - October 1, YYYY through March 31, YYYY (First six-months of a FFY) submitted by May 15, YYYY
"B" Period (AFCARS "B" submission) - April 1, YYYY through September 30, YYYY (Second six-months of a FFY) submitted by November 14, YYYY
Annual files using rolling periods of data - Annual files constructed on the basis of rolling periods of data (e.g., using AFCARS six-month data submissions) are principally structured like a simple moving average (SMA). Like an SMA, the annual file moves, because as the most recent period of data is added, the oldest period of data is dropped. For example, two consecutive six-month AFCARS data submissions are merged to construct an annual file, when the subsequent six-month submission becomes available, it is added to this annual file and the oldest six-month data submission is dropped.
Data Indicators with national standards referenced in this IM:
B. Technical Information on Scheduling Subsequent CFSR and Preparing Data Profiles
States submit AFCARS data twice during the Federal Fiscal Year (FFY). These data represent the "A" and "B" period. Currently, States submit NCANDS data only once during the FFY and include data for the entire FFY.
Depending upon when a State's PIP implementation period ends, the twelve-month period of time covered by a State's data for the subsequent CFSR could consist of either a combination of "A" and "B" AFCARS submissions from the same FFY, or a combination of a "B" AFCARS submission from one FFY and the "A" AFCARS submission from the next FFY. This approach allows for the construction of an AFCARS annual file every six months.
The CFSR methodologically requires parallel timeframes in the use of AFCARS and NCANDS data. Because States annually submit NCANDS data we need to have a process in place that permits us to use NCANDS data from time periods corresponding to those covered by the AFCARS data that we will use in the States' second reviews. For States that will use a combination of "A" and "B" AFCARS submissions in their second review, in addition to their regular NCANDS data submission representing the FFY, we will allow 90 days for late data entry following the end of the FFY for States to collect and enter NCANDS data from the prior FFY and then voluntarily submit on or about January 1st their NCANDS data, explicitly their case-specific Child File (CF). Those States submitting aggregate Summary Data Component (SDC) files and those CF States whose data, for whatever reason, preclude the calculation of at least one of the safety indicators, will submit these safety indicators from alternate data sources. Appropriate documentation including the ACF-generated templates, description of the methodology employed, the data dictionary, and the actual numerators and denominator for recurrence of maltreatment must be provided with this voluntary submission, as applicable.
For States that will use a combination of "B" and "A" AFCARS submissions in their second review, in addition to their regular NCANDS data submission representing the FFY, we will allow 90 days for late data entry following March 31st, for States to collect and enter NCANDS data from the 12 month period ending March 31st, and voluntarily submit on or about July 1st, their NCANDS CF data. Those SDC States and those CF States whose data, for whatever reason, preclude the calculation of at least one of the safety indicators, will submit these safety indicators from alternate data sources. Appropriate documentation including the ACF-generated templates, description of the methodology employed, the data dictionary, and the actual numerators and denominator for recurrence of maltreatment must be provided with this voluntary submission, as applicable.
As needed, ACF will provide technical assistance to States in this effort through the National Resource Center for Child Welfare Data and Technology (NRC-CWDT). We will work individually with States that choose not to use this process in order to obtain the safety-related data needed for the second CFSR within these time frames, using an alternative data source, provided for at 45 CFR 1355.33 (b)(2).
The chart below provides an example of the process we will use to schedule the subsequent CFSR for a State whose PIP ended in December 2003:
| December 31, 2003 | April 1, 2004 | March 31, 2005 | July 1, 2005 | Post July 1, 2005 |
|---|---|---|---|---|
| Last date for State to complete PIP activities | Begin date for 12-month data cycle for use in next CFSR (AFCARS & NCANDS) | End date for 12-month data cycle for use in next CFSR (AFCARS & NCANDS) | End of 90-day period for State to voluntarily submit NCANDS Child File or alternative data source SDIs with proper documentation. | ACF schedules begin date for next Statewide Assessment. |
C. Technical Information on Evaluating States' Attainment of Goals with Regard to Data Indicators in the PIP
Evaluating improvement for the four PDI
In evaluating improvement for the four PDI with national standards, ACF will use States AFCARS data. ACF will verify the indicators before concurring that a State did satisfy its AAI if any State asserts that it has achieved the agreed upon amount of improvement (AAI) based on its generated PDI using the federally approved syntax.
ACF may determine that States have met the AAI relative to a national standard during the implementation period of the PIP. In doing so, ACF will construct an annual file based on rolling periods, corresponding to six-month AFCARS submissions or FFY quarters. This approach will result in using either two consecutive AFCARS submissions or a combination of quarters contained in three AFCARS submissions if the amount of improvement to be determined has occurred at the end of an IFQ (quarter ending December 31st or June 30th).
If States' PIPs require evaluating improvement on a quarterly basis, unless otherwise specified in the PIP, we will ask States to calculate their applicable PDI using an annual file based on rolling quarters (12 months) following the same general time frames and procedures outlined in the first section of this Information Memorandum (IM) pertaining to producing profiles for the second round of the CFSR. However, ACF data staff will not be able to generate a State's PDI in-house for comparison to the States for concurrence until the State makes its regular AFCARS submissions.
If, at the conclusion of the States' PIPs, States have not demonstrated AAI-attainment with regard to their applicable PDI included in their PIPs, ACF will make a final determination of the status of their PDI as compared to their PIP goals (close-out PIP) by using the data covered by the same non-overlapping twelve-month period of time described in Section I of this IM that ACF uses to begin the subsequent CFSR. This means, for example, for States that have PIPs closing during the months of October 2004 through March 2005, ACF will use the 2005B (April 2005 - September 2005) and 2006A (October 2005 - March 2006) AFCARS data. For States that have PIPs closing during the months of April 2005 through September 2005, ACF will use 2006A (October 2005 - March 2006) and 2006B (April 2006 - September 2006) AFCARS data.
States will also be allowed to demonstrate AAI attainment with regard to PDIs for 12-month periods that end quarterly between the conclusion of the PIP and the end of the non-overlapping twelve-month period. In those situations, as noted in Section II of the body of this IM, States will need to generate their data on a quarterly basis.
Evaluating improvement for the two safety data indicators (SDI)
In monitoring improvement for the two SDIs with national standards, States will produce the data needed to monitor their performance on the two safety indicators used in the CFSR (not an entire safety profile). States should use the same definitions and methodology for computing the indicators as ACF uses to compile the initial data profiles for the CFSR, i.e., definitions included in the NCANDS Child File and AFCARS for the relevant data elements and the federally-approved syntax. This is necessary because ACF will use these at PIP closeout or at the time the AAI is met prior to PIP closeout.
If States PIPs require evaluating improvement on a quarterly basis, unless otherwise specified in the PIP, we will ask States to calculate the safety data indicators using a rolling four-quarter period (12 months) of NCANDS data, following the same general time frames (90 days after the end of the rolling four-quarter period) and procedures outlined in the first section of this IM pertaining to producing profiles for the second round of the CFSR. However, States should submit the results according to the procedures associated with the second round of reviews if they determine that they have satisfied their AAI during an intervening quarter.
ACF will use NCANDS data that correspond to the AFCARS time periods described above if, at the conclusion of their PIPs, States have not already demonstrated that they have attained the AAI with regard to the two SDI included in their PIPs. This means for example, that States with PIPs ending during the months of October 2004 through March 2005 will use a successive four-quarter period of CF data ending on March 31, 2006. States that have PIPs ending during the months of April 2005 through September 2005, must produce the two SDI using the same general time frames and procedures for submitting and validating their data described in the first section of this IM pertaining to producing profiles for the second round of the CFSR. (Note: We prefer this approach because we afford States additional time to make their regular NCANDS submission in order to accommodate the collection of services data.)
States will also be allowed to demonstrate AAI attainment with regard to SDIs for 12-month periods that end quarterly between the conclusion of the PIP and the end of the non-overlapping twelve-month period. In those situations, as noted in Section II of the body of this IM, States will need to generate their data on a quarterly basis.
In all situations, States will use the alternate data source methodology that was approved by ACF in the PIP if they do not submit their NCANDS CFs or their CF data preclude their calculating any SDI. States will assure that the quality of the data and accuracy of the computations are consistent with the definitions of corresponding NCANDS data elements and of the national standard indicators. In addition, States will provide a plan for ACF's approval that addresses, at a minimum, sample sizes, if applicable, databases and methodology used to compute the data indicators, using the alternative source data for evaluating the PIP. This plan also should indicate the quality of the databases used and completed ACF-generated templates designed for this purpose. This will allow ACF to recalculate States' measures upon plan approval and States can then submit the actual data from their approved alternate sources to ACF.
The chart below provides an example of how we will evaluate improvement toward achieving the goals associated with data indicators and national standards for States whose PIPs ended in December 2003:
| December 31, 2003 | April 1, 2004 | March 31, 2005 | July 1, 2005 | Post July 1, 2005 |
|---|---|---|---|---|
| Last date for State to complete PIP activities. Goals for two data indicators in PIP have not yet been met, including one permanency indicator and one safety indicator | Begin date for 12-month data cycle for use in determining if State met its goals. (AFCARS & NCANDS) | End date for 12-month data cycle for determining if State met its goals. (AFCARS & NCANDS) | End of 90-day period for State to voluntarily submit NCANDS Child File or alternative data source SDIs with proper documentation. | ACF determines if State met its PIP goals based on the 12-month non-overlapping data. |
