Youthbuild Program: Analysis of Outcome Data Needed to Determine Long-Term Benefits

GAO-07-82 February 28, 2007
Highlights Page (PDF)   Full Report (PDF, 42 pages)   Accessible Text   Recommendations (HTML)

Summary

Since 1993, the Department of Housing and Urban Development (HUD) has provided funding for Youthbuild, a competitive grant program that trains and educates disadvantaged youth and helps build low-income housing. In 2006, Youthbuild was transferred to the Department of Labor (Labor) to better align the Youthbuild program with existing youth workforce and training programs. In response to concerns about the overall quality of Youthbuild, a Senate report directed GAO to assess the program. GAO's objectives included (1) evaluating how HUD assessed and oversaw the program, (2) determining what results the program achieved, and (3) assessing how successful grantees were in obtaining outside funding. GAO analyzed Youthbuild performance data, visited Youthbuild sites, and interviewed agency officials.

While HUD requires grantees to report basic performance data, such as the number of program participants and graduates and job placements, HUD has not aggregated or analyzed the data and conducted limited oversight of grantees. According to HUD officials, they did not have staff available to analyze the closeout reports that grantees must submit, and a lack of resources also limited oversight of grantees. The monitoring HUD did primarily focused on compliance with program requirements such as documentation rather than on performance. As a result, HUD largely was unable to tell how the individual Youthbuild grantee programs performed. Limited outcome data preclude any overall assessment of the performance of the Youthbuild program; further, the few other analyses available such as the one GAO did in this study to augment limited existing data cannot be generalized programwide. GAO analyzed 245 closeout reports, representing 46 percent of the grantees or 12,863 participants. While GAO could determine percentages of participants who received high school diplomas or were placed in jobs, GAO could not determine outcomes over time, partly because the reports lacked baseline information and grantees were not required to and generally did not follow participants after graduation. Further, while closeout reports include information about impediments to program success and "best practices," HUD did not systematically review this information or share it with its primary technical assistance contractor. Consequently, the lack of programwide evaluations, follow-up data, and dissemination of best practices make it very difficult to assess the performance of Youthbuild over time and determine which programs and strategies have worked best. Reporting on post-program performance outcomes, such as the number of participants placed and retained in construction-related employment, could increase the value of the closeout reports and better measure program results. Labor officials indicated that they would consider including such measures for program reporting. Grantees had varying success in obtaining funds from outside sources, but YouthBuild USA data suggest that continued (multiyear) HUD funding was critical to sustaining grantee operations and attracting leveraged funds. Grantees' success in obtaining additional funds varied widely, from 21 grants reporting no additional funding sources to 40 reporting more than $1 million. While most grantees have generated outside funding, YouthBuild USA reported that most grantees have had difficulty continuing operations without continued HUD funding. Their data show that 90 percent of grantees ceased operations if not funded for 3 consecutive years by HUD. Further, YouthBuild USA also noted that grantees with follow-on HUD funding achieved better performance outcomes, such as higher rates of job placements, than grantees that did not receive subsequent HUD funding.



Recommendations

Our recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Implemented" or "Not implemented" based on our follow up work.

Director:
Team:
Phone:
William B. Shear
Government Accountability Office: Financial Markets and Community Investment
(202) 512-4325


Recommendations for Executive Action


Recommendation: To improve the reporting and assessment of performance for Youthbuild grantee programs and develop the bases for an overall assessment of the program, the Secretary of HUD should analyze closeout reports by grant and share information on identified problems and "best practices" with its technical services contractor and Labor.

Agency Affected: Department of Housing and Urban Development

Status: In process

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: To improve the reporting and assessment of performance for Youthbuild grantee programs and develop the bases for an overall assessment of the program, the Secretary of Labor should develop and monitor post-program performance outcome measures for the YouthBuild program, such as the types of employment graduates attained and retained, wage rates, and degrees or certifications received, and share the data with the grantees.

Agency Affected: Department of Labor

Status: In process

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

Recommendation: To improve the reporting and assessment of performance for Youthbuild grantee programs and develop the bases for an overall assessment of the program, the Secretary of Labor should consider whether multiyear funding could be useful in helping YouthBuild grantees attract additional outside funding.

Agency Affected: Department of Labor

Status: In process

Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.