December 2010 — GAO’s Comments on Fifth Round of Recipient Reporting
What GAO Found
Analysis of Fifth Round Recipient Reporting Data Shows Data Quality Is Relatively Stable
According to Recovery.gov as of October 30, 2010, recipients reported on over 200,000 awards indicating that the Recovery Act funded approximately 671,607 jobs during the quarter beginning July 1, 2010, and ending September 30, 2010[1]. This included over 2,400 prime reports associated with Head Start grantees. As reported by the Recovery Accountability and Transparency Board, job calculations are based on the number of hours worked in a quarter and funded under the Recovery Act—expressed in FTEs.
Using the fifth reporting period data, GAO continued its monitoring of errors or potential problems by repeating many of the analyses and edit checks reported in its earlier reports. GAO reviewed 77,711 prime recipient report records from Recovery.gov for this fifth round, an increase of 3,462 or about 4.6 percent from round four. For its analyses, in addition to the fifth round of recipient report data, GAO also used the previous four rounds of data as posted on Recovery.gov as of October 30, 2010.
In examining recipient reports, GAO looked for progress in addressing several key limitations found in its prior reports. In prior rounds GAO has reviewed data logic and consistency and reviewed unusual or atypical data. Data logic and consistency provide information on whether the data are believable, given program guidelines and objectives; unusual or atypical data values indicate potential inaccuracies. As with previous quarterly report rounds, these reviews included (1) the ability to link reports for the same project across quarters and (2) concerns in the data logic and consistency, such as reports marked final that show a significant portion of the award amount not spent. GAO continued to see results similar to those of past reviews. For example, GAO continued to find a small number of reports for which there were potential linkage issues across quarters. This may impact the ability to track project funding and FTEs over quarters. GAO also continued to see a small number of reports marked final for which there appeared to be some discrepancies, such as reports marked final but for which project status was marked as less than 50 percent completed.
As part of its review of data logic and consistency, GAO found some inconsistencies in the Head Start data's agency review flag field that were similar to inconsistencies GAO found in its previous reviews. GAO's analysis suggests that this field may not correctly reflect the extent of the agency's review process. Prime recipient report records include a review flag on whether or not the federal agency reviewed the record during the data quality review time frames. As GAO has noted in past reports, these analyses of this agency review field, in conjunction with other fields and its discussions with agency officials, have indicated potential problems and inconsistencies. It appears that this continues to be the case with Head Start reports, only 17 percent of which were marked as having been reviewed by the agency. However, agency officials stated that all reports were reviewed. Another data field on the recipient report shows whether or not a correction was initiated; analysis of that field indicates it is likely that additional agency reviews took place. A correction could be initiated by either the prime recipient or the reviewing agency. Thirty percent of the reports had the correction flag set to 'yes' even though the review flags suggested that neither the agency nor prime recipient had reviewed those reports. Officials noted that they continued to experience difficulties in marking reports as reviewed. Agency officials indicated that they would continue to update these flags during the continuous review process.
As part of its additional analysis, GAO found that a previously identified concern with FTE calculations for Head Start reports is being addressed. Agency officials stated that following the December 2009 Office of Management and Budget (OMB) memorandum clarifying the FTE definition, all grants associated with cost-of-living adjustment/quality improvement COLA/QI were reviewed with respect to the reporting of FTEs. Agency officials reviewed these grants and communicated to recipients that the cost-of-living increases were not to count toward the FTE totals. GAO reviewed the FTEs reported to determine if Head Start recipients were no longer reporting cost-of-living increases as FTEs. In examining the impact on the FTE figures before and after this clarification guidance was issued, GAO observed a significant drop in the number of COLA/QI FTEs reported in all reporting periods following the updated guidance. GAO further observed that for the fifth round of reporting, over 60 percent of these COLA/QI recipient reports showed no FTE value. For the remaining reports showing FTEs, GAO looked to see if, in the 20 reports with the greatest number of FTEs, the fields describing the award, project, and job creation explained their FTE counts. In almost all cases, one of these narrative fields accounted for activities that would be expected to produce FTEs—hiring staff, expanding existing staff hours, or hiring other types of workers to improve facilities, in accordance with Head Start quality improvement guidance. Agency officials noted that they continue to monitor these grants to ensure that cost-of-living increases are not being reported among the FTEs.
What GAO DidBack to top
This section of this review responds to the Recovery Act's mandate that GAO comment on the estimates of jobs created or retained by direct recipients of Recovery Act funds. For its review of the fifth submission of recipient reports, covering the period from July 1, 2010, through September 30, 2010, GAO built on findings from its four prior reviews of the reports, covering the period from February 2009 through June 30, 2010. GAO performed edit checks and basic analyses on the fifth submission of recipient report data that became publicly available at Recovery.gov on October 30, 2010.
To understand how the quality of jobs data reported by Recovery Act recipients, particularly Head Start grantees has changed over time, GAO compared the five quarters of recipient reporting data that were publicly available at Recovery.gov on October 30, 2010. GAO performed edit checks and other analyses on the Head Start subset of recipient reports, which included matching OHS-provided data (Financial Assistance Awards from the Grants Administration Tracking and Evaluation System database) and HHS-provided data (grantee expenditures from PMS) on grantees with recipient reports. GAO also reviewed documentation and interviewed federal agency officials from HHS, who have responsibility for ensuring a reasonable degree of quality across their programs' recipient reports. Due to the limited number of recipients reviewed and the judgmental nature of the selection, GAO's full-time equivalent findings are limited to those Head Start programs and time periods examined and are not generalizable to any other program's FTE reporting.
- [1] Under the continuous corrections period, recipients were allowed to modify submissions from November 2, 2010, to December 6, 2010. The final update of this round of recipient reported data should occur on December 8, 2010.