|
|---|
|
|
||||
|---|---|---|---|---|
|
|
|
|||
| ACF Home | Services | Working with ACF | Policy/Planning | About ACF | ACF News | HHS Home | ||||
Questions?
|
Privacy
|
Site Index
|
Contact Us
|
Download Reader™
|
Print
 �
|
|---|
The AFCARS Assessment Reviews (AAR) are conducted by the Children's Bureau, the ACF Office of Information Services (OIS), and the ACF Regional Offices. They incorporate an understanding of a State's child welfare policies and practices with an analysis of the information system designed to collect information and the technical documentation used to extract and report the AFCARS data to ACF.
The AFCARS Assessment Review process goes beyond the edit checks that must be met by a State in order to pass the AFCARS compliance error standards. The AAR also ascertains the extent to which a State meets all of the AFCARS requirements and the quality of its data. Additionally, while the AAR is an assessment of the State agency's collection and reporting of AFCARS data, it is also an opportunity for Federal staff to provide substantive technical assistance to State agency staff. During the AAR, the Federal team identifies improvements to be made to the system and recommends changes to the program code used to extract the AFCARS data.
Back to Table of Contents
The AFCARS Assessment Review process consists of five components. They are:
Exhibit 1 outlines the major activities that occur in each phase. The remainder of this section provides a high level overview of each phase.
1. AAR Initiation Phase
The purpose of this phase is to develop the AAR schedule. The initial step in the process is to identify States that are eligible for an AAR. Although this is a Federal task, and not covered in detail in this Guide, States may request an AAR. Guidance is provided in section E of this chapter on how to request an AAR. Once States have been identified for an AAR, they are contacted in order to finalize scheduling of the AAR.
2. Pre-site Visit Phase
This phase includes activities that are necessary to prepare for the on-site review. It involves the exchange of documentation between the Federal and State team, and a preliminary analysis of the State's system documentation, by Federal staff, in order to develop initial findings and identify those areas that require emphasis during the on-site review. During this phase the State enters a set of test cases and extracts the test data for transmission to ACF. The pre-review phase begins no later then ten weeks prior to the scheduled site visit. Chapter IV further addresses the procedures for this phase.
3. On-site Visit Phase
The on-site review is conducted over five business days. During this time the Federal team reviews its preliminary findings with the State team, views the demonstration of the State's system using a sample case scenario, and reviews paper case files. At the end of the AFCARS Assessment Review, an exit briefing is held with the State review team and management officials. During the exit briefing a summary of the major compliant and non-compliant findings is presented. In addition to the briefing, draft findings documents are left with the State team in order to allow the State to begin making necessary changes, if any, to the system. Chapter V further addresses the procedures for this phase.
4. Post-site Visit Phase
The post-review phase involves a final assessment of the information gathered during the on-site review, including any corrections the State makes to its extraction code within 45 days after the site visit. This phase should last no more then 90 business days from the end of the on-site visit. A report is written noting the major findings from the assessment review, with copies attached of the detailed findings. This report is signed by the Associate Commissioner of the Children's Bureau and mailed to the State agency. Chapter VI further addresses the procedures for this phase.
5. AFCARS Improvement Phase
In the event there are areas the State must correct in order to meet the requirements established in the AFCARS Federal regulation, the State will start the AFCARS Improvement Phase. This phase begins once the State receives the report and improvement plan and ends when the State has made all system changes and the quality of the data has improved and remains consistent. The elements/general requirements that receive a rating factor of "3" or lower will make up the action items in the plan. Chapter VII further addresses the procedures for this phase.
Exhibit 1: AFCARS Assessment Review Process
| PHASE | AAR Initiation | Pre-site Visit | On-site Visit | Post-Site Visit | AFCARS Improvement |
|---|---|---|---|---|---|
Activity |
Federal Identification of States for AAR State Requests an AAR Notify State of AAR Finalize AAR Schedule for FFY |
Conduct Conference Call(s) Schedule Site Visit Exchange Documentation Enter Test Cases Select Sample for Case File Review Identify Case Files Finalize Arrangements for On-site Review |
Conduct Entrance Conference System Demonstration Using a Sample Case Scenario Review Case Files Review AFCARS Population and Data Element Requirements Conduct Technical Review Document Review Findings Conduct Exit Conference |
Final Review and Findings Prepare and Send Letter and Final Report to State Agency State drafts Timeline for Corrections |
State Makes Corrections to Information System/ Screens and Program Code Implement Data Quality Improvement Measures |
Timeframe |
|
10 Weeks Prior to the On-site Review Phase |
5 Business Days |
90 Business Days following the On-site Review |
2 years from final report date |
Back to Table of Contents
C. Requirements Subject To Review
States' AFCARS are reviewed against the requirements in 45 CFR 1355.40, the appendices to 45 CFR 1355, AFCARS policy, and Technical Bulletins. There are two major areas that are assessed during an AFCARS Assessment Review. They are: 1) the AFCARS general requirements, and 2) the data elements. The general requirements include the population to be reported to AFCARS and the technical requirements for constructing a data file. During an AAR, each of the 103 data elements and each of the 28 AFCARS general requirements will be assessed on the basis of the requirements in the regulation and other policy and technical issuances. Information for each item that is collected from each of the components of the AAR is combined and a rating factor is assigned. The general information requirements will be evaluated and rated separately from the data elements. Chapter V provides more detail on the evaluation process. Additional information is also available in the following appendices:
Appendix A lists the AFCARS policy issuances and technical bulletins.
Appendix B lists the AFCARS general requirements.
Appendix C lists the AFCARS data elements.
Back to Table of Contents
A scale of 1 (does not meet AFCARS standards) to 4 (fully meets AFCARS standards) is used to assign a rating factor for each of the AFCARS general requirements and data elements. Exhibit 2 lists the rating factors and definitions. The State is not given an overall numerical rating based on the AFCARS Assessment Review.
A new category has been added to the rating factors. This category applies to States that have received SACWIS-level FFP. If during an AAR, the Federal team finds that the State's AFCARS data are not collected or extracted from a SACWIS, the appropriate area will be given a rating factor of "0." The State's AAR will be considered incomplete in regard to the area not collected in the SACWIS. The items that are collected in the system will be evaluated based on the AFCARS requirements. For instance, if the foster care elements are collected in the SACWIS but the adoption elements are not, the foster care elements will be given the appropriate rating factor between one and four, and the each adoption element will be given a zero. Once the State completes the tasks associated with enhancing the SACWIS to collect and report the AFCARS data, the Federal team will complete the AFCARS Assessment Review. Depending on the degree of changes needed, this may require a second on-site review.
Since an AAR is conducted on those States that are collecting, extracting, and submitting both the foster care and the adoption files from the SACWIS, the Children's Bureau expects that a rating factor of "0" will be rare. If it is found during a SACWIS Assessment Review that the State is not collecting and reporting its AFCARS data from the system in compliance with 45 CFR 1355.53(b)(1), it is the Children's Bureau's expectation that this requirement be implemented immediately and given the State's highest priority. As noted in section C.2 of Chapter I, this may result in a disallowance of some of the Federal incentive funds used by the State to develop its SACWIS.
Exhibit 2: Rating Factors
| Rating Factor | Definition |
|---|---|
| 4 | All of the AFCARS requirements have been met. The information system is functioning as required, and the information is being accurately collected and extracted. |
| 3 | There are data quality issues. For example:
|
| 2 | The technical requirements for AFCARS reporting are not fully met. For example:
|
| 1 | An AFCARS requirement(s) has not been implemented in the information system. For example:
|
| 0 | States operating an automated information system for which they received SACWIS-level FFP were found to be using an external automated information system, or a database (such as Excel or Access), and are not collecting and reporting the AFCARS data from the SACWIS system. In addition, there is no program code for the extraction of data from the SACWIS. |
Some data elements are directly related to each other. When this occurs, all related elements are given the same rating factor. This is because incorrect programming logic could affect the values to all of the related data elements.
It is possible that the problem with the data element and data are due to both system issues and case worker data entry issues. In this case, the element is given a "2" to denote the need for the system program source code or logic to first be modified. Once the corrections are made to the system, then the data needs to be re-analyzed by ACF. If problems related to case worker training or data entry still exist, then a "3" is assigned to the requirement. A finding of fully meeting the AFCARS standard (a factor of "4") is not given to the element until all system issues and/or data quality issues have been addressed.
A State that has been determined to not fully satisfy the AFCARS standards (a rating factor of 3 or lower) must make the corrections identified by the review team to improve data quality. Action steps are identified in the AIP and included in the final report. The State is required to make the changes to the information system, extraction program, and/or data entry in order to fully satisfy the applicable requirements and standards. Since the AFCARS data are used for several significant activities at the Federal and State level, the State should implement the action items as soon as possible as a way to improve the quality of its data.
To the extent that the State takes action to correct its collection and extraction of AFCARS data, it is possible that a resulting improvement will bring to light weaknesses masked by prior problems. For example, a State's program logic may be written to ensure there are no missing data by mapping missing information to a known AFCARS value (i.e., setting defaults). Once this is corrected the State needs to ensure that workers are actually entering the required information.
States found not to be in compliance during the AFCARS Assessment Review do not incur penalties. We expect the State to use the AIP as a way to formalize technical assistance needs, action steps, and improve the quality of its data.
Back to Table of Contents
1. Federal Selection Process
As noted previously, this phase is primarily a Federal task. It is included in this Guide in order to assist States to know when they may be selected for an AAR. In order to determine a State's readiness for an AAR, the Children's Bureau determines whether a State has developed and is operating a SACWIS. States that have not developed a SACWIS, and are not planning to do so, are considered ready for an AFCARS Assessment Review.
To be considered ready for an AFCARS Assessment Review, States that have developed, but not completed the implementation of, a SACWIS, must be:
The priority for conducting AFCARS Assessment Reviews is in those States that have either completed the on-site process of a SACWIS Assessment Review or will not develop a SACWIS. When determining the order in which States are reviewed, the following additional factors are taken into consideration:
A State may contact its ACF Regional Office to inquire when it may be scheduled for an AAR.
2. State Request for an AFCARS Assessment Review
States may request an AFCARS Assessment Review. This section outlines the steps for a State to request an AFCARS Assessment Review.
Back to Table of Contents
Continue to Chapter III: The Review Teams
Attachments:
State Guide to an AFCARS Assessment Review HTML or PDF(332 KB)
