The Government Performance and Results Act (GPRA) requires that each Federal agency prepare an annual performance plan, including specific performance indicators that can be verified and validated. The Workforce Investment Act (WIA) and implementing regulations require the states to submit annual progress reports addressing adult, youth, and dislocated worker programs' performance measures that can be validated and verified as accurate, and gives the states monitoring and oversight responsibility for this performance information.

Because of insufficient local, state, and Federal oversight, the Employment and Training Administration (ETA) has little assurance that the state-reported WIA performance outcomes data are either accurate or verifiable. Without validating 100 percent the data to assure that the measures (and underlying data used to compute these measures) are accurate, states should use a statistically valid sampling method for validating reported data. A scientific statistical approach to data validation would provide consistent definitions of key program terms and documentation guidelines.

The Office of Inspector General (OIG) conducted a performance audit to determine what oversight and monitoring procedures were in place to ensure that state-reported WIA performance data were accurate and supportable. Our audit covered procedures at the ETA national and regional offices, 4 states (Colorado, Louisiana, Kentucky, and California), and 12 Local Workforce Investment Boards (LWIB)/One Stop Centers⁽¹⁾ to determine the extent of performance data monitoring at each program level.

Our audit of WIA performance data oversight at the local, state, and Federal levels disclosed the following:

• Local level

Although 4 of the 12 One Stops monitored their contractors, none of the 12 One Stops had adequately documented procedures for validation of participant performance data. Sufficient procedures would include policies addressing data validation, monitoring guidelines, and a statistically valid approach to correcting management information system (MIS) data for the reporting system as a whole.

• State Level

Two of the four states had not monitored performance data at the One Stop case file level. Additionally, none of the four states had sufficient procedures to assure the accuracy of their reported performance data. Using scientific sampling and statistical estimation would provide the states a basis for assuring the accuracy of the reported performance data.

• Federal Level

ETA has limited its monitoring procedures over WIA performance outcomes data to desk reviews and computer edit checks to determine the reasonableness of the data reported. ETA did not have plans to implement a formal monitoring process at the Federal level. ETA has issued field memoranda and guidance letters to the states as instruction on monitoring and validating the performance data, indicating the states' responsibilities for ensuring the integrity of their reported performance data. However, as of the end of our onsite fieldwork, the regional ETA staff had not reviewed any of the states' monitoring policies and procedures that might have been in place to ensure the integrity of the reported performance data. However, ETA recognizes that additional steps are needed to assure the reliability of WIA performance outcomes data. Development of a comprehensive data validation and verification system has been funded through a contract with Mathematica Policy Research, Inc.

We recommend the Assistant Secretary for Employment and Training:

• Continue ETA's current efforts to establish a standardized, statistical method for WIA performance data validation and require the states to either use this method or have an equivalent, documented method in place to ensure data integrity.

• Provide the states with consistent definitions and documentation guidelines to ensure that all states are reporting consistent data.

• Have the regional ETA representatives:
  • require the states to include data validation in their monitoring policies/programs;
  • require the states to establish minimum documentation standards to support the activities and outcomes reported; and
  • require the states to independently calculate the reported performance measures as part of their performance monitoring.

ETA's Response to the Draft Report

ETA agreed with our conclusions and responded that they are well on their way to developing an effective strategy for validating and verifying performance outcomes data. ETA has been developing a systematic approach to data validation and verification that includes the framework for a standardized and statistical method for WIA performance data validation and anticipates announcing their data validation policy by December of this year. ETA did not respond directly to the recommendations in our report because they believe the substance of the recommendations in our report falls within the framework of ETA's data validation project and their current plan for deployment and execution.

OIG's Conclusion

As long as ETA's statistical approach to data validation and implementing policy address our specific recommendations, we agree with ETA's approach to rectify this issue. However, ETA's response does not indicate whether this data validation process will be mandatory for all states or voluntary. If some states voluntarily validate performance data and others do not, ETA still will not have assurance that national performance data are accurate.

Based on ETA's planned actions, all recommendations in this report are resolved. To close these recommendations, ETA needs to provide us with evidence that the planned validation system and policy guidance have been implemented.

Get Complete Report

-- DISCLAIMER

Send technical comments to: Webmaster@oig.dol.gov
Comments relating to policy, content or style should be directed to:
rpts-coordinator@oig.dol.gov .