June 2011 — GAO's Comments on the Latest Round of Recipient Reporting

• Transportation Infrastructure:
• What GAO Found
• What GAO Did


• Water Infrastructure:
•               What GAO Found
• What GAO Did

For the latest round of recipient reporting, GAO looked at two areas: transportation infrastructure funding at the Department of Transportation, and the Clean Water State Revolving Fund and Drinking Water State Revolving Fund at the Environmental Protection Agency.

Transportation Infrastructure

What GAO Found

Analysis of Recipient Reporting Data Shows Data Quality Remains Relatively Stable

GAO focused its review on the quality of data reported by transportation grant recipients and efforts made by FHWA to validate that data. Using transportation recipient data from the seventh reporting period, ending March 31, 2011, GAO continued to check for errors or potential problems by repeating analyses and edit checks reported in previous reports. GAO reviewed data associated with 12,443 transportation recipient reports posted on Recovery.gov for the seventh reporting quarter.

GAO found few inconsistencies and was generally satisfied with the stability of the data quality. Additionally, GAO's analysis of the data showed a decrease of 759 recipient reports, or about a 5.7 percent drop from the previous quarter. Likewise, the total number of FTEs reported also decreased over the past two reporting quarters. In the most recent quarter which ended March 31, 2011, for example, the percentage of prime recipients of highway funds reporting any FTEs dropped from approximately 51 percent to approximately 39 percent. DOT officials said that the decreases in the number of recipients reporting any FTEs is likely due to several factors, including projects being completed or functionally complete and awaiting financial closeout. DOT officials noted that decreases in FTEs could also be due to such factors as a winter shutdown of projects in colder climates. GAO also observed a variety of patterns in the quarterly reporting of FTEs, including consecutive quarters of no FTE reporting. For example, for the 2010 calendar year, approximately 13.5 percent of the highway recipients and approximately 16.7 percent of the transit recipients that filed reports each quarter did not report any FTEs during the year. According to DOT officials, several additional factors that could extend reporting during periods of low job activity include projects awaiting final invoice from contractors, projects delayed in litigation, or recipients' withholding of final payments to cover periods of maintenance guarantees. They also noted that projects need to be considered on an individual basis and that recipients may use Recovery Act funds to purchase materials and use other funding sources to pay for labor.

DOT Continues to Perform Automated Checks to Help Improve Data but Is Not Planning to Use Recipient Reported Data Internally

Each quarter, FHWA performs quality assurance steps on the data that recipients provide to FederalReporting.gov and officials reported that the data quality continues to improve. Based on these reviews and their interactions with recipients, FHWA officials reported that recipients now understand the reporting process and each reporting period has gone better than the previous one. One measure of recipients’ understanding of the reporting process is in the number of noncompliant recipients. According to information available on Recovery.gov, the number of DOT-related noncompliant recipients decreased from 37 in the quarter ending September 30, 2010, to 13 in the quarter ending December 31, 2010, but it increased in the most recent quarter to 19 noncompliant recipients. FHWA officials told GAO that they routinely check for noncompliance, notify non-compliant recipients of the projects that have not been reported, and follow up with non-compliant recipients to obtain corrective action plans and ensure that errors are corrected and subsequent reports are filed on time.

As in previous quarters, FHWA performed a number of automated checks to help ensure data quality of highway and rail recipients’ Recovery Act data. To support recipients’ data quality, FHWA asks recipients of highway and rail Recovery Act funds to report each month into FHWA’s RADS system. FHWA officials conduct two data verification steps in RADS to assess the quality of data submitted by recipients, including automated data verification and validation reports.

• The automated data verification tests occur when state departments of transportation upload monthly data into RADS. If a record does not satisfy one of FHWA's data verification rules, the state department of transportation is provided with a brief message listing the record and what data check failed. Data cannot be uploaded into RADS until the state department of transportation corrects the error. Examples of data verification rules include rules such as the federal project numbers must be entered without dashes or parentheses, and total cost estimates cannot be less than total Recovery Act estimates for the particular project.
• The data validation report highlights projects or awards that fail certain verification rules, such as whether the federal project number is in FHWA's Financial Management Information System, but not in RADS. FHWA also applies data checks based on assumptions about expenditures reported and FTEs reported.
• The data validation report highlights projects or awards that fail certain verification rules, such as whether the federal project number is in FHWA's Financial Management Information System, but not in RADS. FHWA also applies data checks based on assumptions about expenditures reported and FTEs reported.

FHWA officials reported they also check data quality for nearly 70 data field each quarter by comparing the data in each recipient report against the corresponding RADS data. According to FHWA guidance, data that does not correspond to the recipient report is flagged for comment and review. Specifically, RADS runs automated quality checks to ensure that data provided by states into RADS match what the states are providing to FederalReporting.gov. If inconsistencies are found, FHWA representatives work with state transportation officials to resolve discrepancies by requiring states to amend or justify state-reported data. Some state transportation officials told GAO that the number of errors detected in their reports decreased as the reporting system was refined and guidance issued.

Finally, according to DOT officials, recipient reported FTE data provide increased transparency on the use of transportation program funds, but DOT does not plan to use recipient reported data internally for a variety of reasons. For example, recipient reported data are only valid on a quarterly basis and cannot be used for monthly or cumulative analysis. In addition, agency officials told GAO that they prefer to use RADS data for most internal analysis because the RADS data are reported monthly and are more detailed than the recipient reported data.

What GAO DidBack to top

The recipient reporting section of this report responds to the Recovery Act's mandate that GAO comment on the estimates of jobs created or retained by direct recipients of Recovery Act funds. For GAO's review of the seventh submission of recipient reports, covering the period January 1 – March 31, 2011, GAO continued its monitoring of errors or potential problems by repeating many of the analyses and edit checks reported in GAO's 6 prior reviews covering the period from February 2009 through December 31, 2010. To examine how the quality of jobs data reported by recipients of Recovery Act transportation funds has changed over time, GAO compared the 7 quarters of recipient reporting data that were publicly available at Recovery.gov on April 30, 2011. GAO performed edit checks and other analyses on the transportation recipient report data which included matching DOT-provided funding data from the Financial Management Information System with recipient reported funding data and reviewing FTE reporting patterns. GAO's match showed a high degree of agreement between DOT recipient funding information and the information reported by recipients directly to FederalReporting.gov.

GAO also examined the reliability of recipient reported data and reviewed FHWA's efforts to ensure reliability of the recipient reported data by comparing it with data contained in DOT's Recovery Act Data System. GAO's assessment activities included reviewing documentation of system processes, conducting logic tests for key variables, and assessing data for out-of-range values. GAO reviewed agency documentation for the Recovery Act Data System and FHWA's guidance for validating recipient reported data in that system. GAO also reviewed a February 2010 OIG report assessing the Recovery Act recipient data oversight at DOT and other agencies. In general, GAO considered the data used to be sufficiently reliable for purposes of this report. The results of GAO's FTE analyses are limited to the transportation programs and time periods reviewed and are not generalizable to any other program's FTE reporting.

Water Infrastructure

What GAO foundBack to top

Federal and State Agencies Continue to Oversee the Quality of Recipient Reporting Data, Including Jobs

Officials at EPA and in the nine states have regularly checked the quality of data on Recovery.gov and stated that the quality has remained relatively stable, although GAO identified minor inconsistencies in the FTE data that states reported. Overall, the 50 states reported that the Recovery Act SRF programs funded an increasing number of FTE positions for the quarter ending December 2009 through the quarter ending June 2010, from about 6,000 FTEs to 15,000 FTEs. As projects were completed and funds spent, these FTEs had declined to about 6,000 FTEs for the quarter ending March 2011.

EPA Continued Performing Data Quality Checks and Said Data Quality Is Relatively Stable

According to EPA officials, the overall quality of the states' SRF data on Recovery.gov, which EPA officials have checked each quarter, is stable. The officials said that the states' initial unfamiliarity with a newly developed reporting system has been resolved, the Federalreporting.gov help desk has improved, and guidance issued by OMB has clarified reporting issues over time. During the seventh round of reporting, which ended on March 31, 2011, EPA officials continued to perform data quality checks as they had in previous quarters. Specifically, EPA used data from the agency's grants database, contracts database, and financial management system to compare with recipient-reported data. These systems contain authoritative data for every award made to the states, including the award identification number, award date, award amount, outlays, Treasury Account Symbol codes, and recipient names. According to EPA officials, they use the agency data to ensure that recipient-reported information for a given award corresponds with the information on EPA's official award documents. EPA staff can raise questions about any inconsistent data through the Federalreporting.gov system. State recipients may make appropriate changes to the data through the end of the reporting period, and after public release, during the continuous correction cycle. According to EPA officials, this process has resolved any questions and comments from EPA's reviews.

To facilitate its oversight of state-reported data, EPA required states to use its Clean Water Benefits Reporting (CBR) system and Program Benefits Reporting (PBR) system to report on certain Recovery Act project information, such as the project name, contract date, construction start, Recovery Act funding, jobs created or retained, and project purpose and anticipated benefits. EPA officials said that they do not routinely collect state expenditure data in these systems and that they rely on regional officials to review expenditures reported by the states on Recovery.gov. GAO compared EPA's data on awards and funds drawn down by states with data reported by states on Recovery.gov and found only a few minor inconsistencies in the data.

Similarly, in September 2010, EPA's OIG reported that the Recovery.gov data for EPA's SRF programs contained a low rate of errors. The OIG audited EPA's controls for reviewing recipient-reported data after the second round of reporting, which ended December 31, 2009, comparing EPA data on award type, award number, funding agency code, award agency code, and award amount to state-reported data on Recovery.gov. The OIG report found that EPA's controls helped lower the rate of errors for these key data and recommended some improvements to EPA's process. EPA's Clean and Drinking Water SRF program officials said that they have had few errors in the SRF data in the last three rounds of reporting.

Nine States Checked Quality of Recovery Act Data Quarterly, but Minor FTE Discrepancies Occurred

Officials in the nine states GAO reviewed indicated that the quality of recipient data has remained relatively stable, although GAO found that the states differed in how they reported state agencies' FTE data and did not report some subrecipients' FTE data. Water program officials in these states said that they check the quality of data that are reported on Federalreporting.gov and then Recovery.gov. In addition, officials in Alabama, Connecticut, Maryland, Missouri, and New Mexico said that they examined payroll data submitted by contractors to verify FTE data. In some cases, state officials said that they contact subrecipients for clarification about data that are missing or inconsistent.

In addition to department-level checks, in most of the nine states GAO reviewed, state-level Recovery Office staff checked the data before submitting the information to Federalreporting.gov. In four of the nine states—Alabama, Maryland, Missouri, and New Mexico—Recovery Office staff monitored Recovery Act implementation and performed independent data quality checks of the data reported by state agencies. According to several state officials, this reporting structure provided an additional level of review of state agency data. In Maryland, for example, officials said that their state-level reporting system relieves subrecipients of certain reporting duties. Subrecipients submitted the FTE and payroll information to Maryland's StateStat office, and staff in that office reviewed and validated the data, completed the required federal reports, and submitted them to Federalreporting.gov. Furthermore, for control purposes, only two staff members handled the information. In addition, staff in Nevada's Recovery Office conducted quality checks; however, each state agency then submitted its data directly to the appropriate federal agency. The remaining four states—Connecticut, Michigan, Washington state, and Wyoming—did not have a Recovery Office staff check data quality.

GAO found minor problems with the FTE data that some of the nine states reported. Specifically, (1) states differed in how they reported the FTEs associated with their own program staff—that is, those who conduct document reviews, site inspections, and other key program duties; and (2) three states identified missing or incorrectly reported FTE data on Recovery.gov, and these data have not been corrected. In particular:

• Six of the nine states reported the FTEs for their state employees who were paid with Recovery Act funds, while two states did not. Officials in Maryland and Michigan noted that they did not report all the time their state employees spent on program activities in Federalreporting.gov, although these FTEs were paid for with Recovery Act funds. EPA officials said that they provided states with OMB guidance and that OMB guidance requires states to report FTEs paid for with Recovery Act funds.
• Washington state officials who administer the Clean Water SRF program changed the time frame for reporting FTE data in the third round of reporting, and as a result, missed reporting one quarter of data. During the first two reporting rounds, because some subrecipients were finding it difficult to submit complete FTE data to the state by the state's deadline, staff reported data from two months of the current quarter and one month of the previous quarter. During the third reporting quarter, the state began reporting three months of current data. However, the state received data from a subrecipient after the deadline for reporting and did not correct the data during the correction period. As a result, officials said about 18 FTEs remain unreported. EPA officials told them to keep a record of these FTEs in case there is an opportunity to correct the data.
• Officials in New Mexico did not report a few FTEs for the state's Drinking Water SRF program in the first two rounds of reporting. The officials explained that the revisions were submitted to the state after the reporting period ended and therefore the FTEs were not captured in Recovery.gov.
• Officials in Wyoming identified incorrectly reported FTEs for two quarters. The FTEs were incorrect because the state entered the data for one clean water project for one quarter in the next quarter. As a result, one quarter's data were overstated by a few FTEs, and the other quarter's data were understated by a few FTEs. The state official explained that the data changed after they were initially reported in Recovery.gov and were not updated during the correction period.

As the bulk of Recovery Act funding is spent, EPA officials said that the states were beginning to complete their projects. Officials said that before the next reporting round begins in July 2011, they plan to issue a memorandum to states on how to complete their Recovery Act grants and when to stop reporting to Recovery.gov. During the seventh round of reporting, one state in each program indicated in Recovery.gov that the grant—including all projects that received money from the grant—was complete. EPA officials told GAO that as of early May 2011, 629 clean water and 383 drinking water projects have been completed across all states.

Some state officials charged with coordinating state-level Recovery Act funds also said that they are winding down their activities. In Michigan, for example, the Recovery Office was originally a separate office under the Governor, but has since been moved under the Department of Management and Budget. In Nevada, the Recovery Act Director said that his office will be eliminated at the end of June 2011. At that point, the Department of Administration's centralized grant office will take responsibility for Nevada's remaining Recovery Act efforts. Similarly, officials at the New Mexico Office of Recovery and Reinvestment said that their office is currently funded through the Recovery Act State Fiscal Stabilization Fund through the end of June 2011.

Because of the high-level nature of SRF recipient reporting for Recovery.gov and the availability of information in its own data systems, EPA officials do not anticipate using data from Recovery.gov. The officials said that whereas Recovery.gov summarizes information on many projects at the state level, the data from CBR and PBR are more useful for understanding states' projects than data on Recovery.gov because the internal data are provided by project and include more detail. EPA officials said that by the end of 2011 they plan to use information in these two internal systems to assess anticipated benefits of the Recovery Act SRF program funds. EPA Clean Water officials said that they would perform case studies of completed projects and assess anticipated benefits. Drinking Water officials said that they are considering three major studies, some joint with the Clean Water SRF program. These studies may include assessments of project distributions, green projects' benefits, and subsidy beneficiaries.

What GAO DidBack to top

To examine approaches federal agencies and selected states have taken to ensure data quality for jobs reported by Recovery Act recipients, GAO conducted work at both the national and state level. The recipient reporting section of this report responds to the Recovery Act's mandate that GAO comments on the estimates of jobs created or retained by direct recipients of Recovery Act funds. For GAO's national review of the seventh submission of recipient reports, covering the period from January 1, 2011, through March 31, 2011, GAO continued its monitoring of errors or potential problems by repeating many of the analyses and edit checks reported in the six prior reviews covering the period from February 2009 through December 31, 2010. To examine how the quality of jobs data reported by recipients of Clean and Drinking Water SRF grants has changed over time, GAO compared the seven quarters of recipient reporting data that were publicly available at Recovery.gov on April 30, 2011. GAO performed edit checks and other analyses on the Clean and Drinking Water SRF prime recipient reports and compared funding data from EPA with funding amounts reported on the recipient reports. GAO also reviewed documentation and interviewed federal agency officials from EPA who are responsible for ensuring a reasonable degree of quality across their programs' recipient reports.

At the state level, GAO interviewed state officials in the nine states it reviewed about the policies and procedures they had in place to ensure that FTE information for Recovery Act projects was reported accurately. For selected Recovery Act data fields, GAO asked state program officials in the nine states to review and verify EPA's Recovery Act data from CBR and PBR and provide corrected data where applicable. For the nine states, GAO compared state-reported Clean and Drinking Water SRF FTE data from the sixth submission of recipient reports, the period covering October 1, 2010, through December 31, 2010, with corresponding data reported on Recovery.gov. GAO addressed any discrepancies between these two sets of data by contacting state program officials. GAO's national and state work in selected states showed agreement between EPA recipient information and the information reported by recipients directly to Federalreporting.gov. In general, GAO considers the data used to be sufficiently reliable for purposes of this report. The results of GAO's FTE analyses are limited to the two SRF water programs and time periods reviewed and are not generalizable to any other program's FTE reporting.