This is the accessible text file for GAO report number GAO-07-82 
entitled 'Youthbuild Program: Analysis of Outcome Data Needed to 
Determine Long-Term Benefits' which was released on February 28, 2007. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as part 
of a longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

Report to Congressional Committees: 

United States Government Accountability Office: 

GAO: 

February 2007: 

Youthbuild Program: 

Analysis of Outcome Data Needed to Determine Long-Term Benefits: 

GAO-07-82: 

GAO Highlights: 

Highlights of GAO-07-82, a report to congressional committees 

Why GAO Did This Study: 

Since 1993, the Department of Housing and Urban Development (HUD) has 
provided funding for Youthbuild, a competitive grant program that 
trains and educates disadvantaged youth and helps build low-income 
housing. In 2006, Youthbuild was transferred to the Department of Labor 
(Labor) to better align the Youthbuild program with existing youth 
workforce and training programs. In response to concerns about the 
overall quality of Youthbuild, a Senate report directed GAO to assess 
the program. GAO’s objectives included (1) evaluating how HUD assessed 
and oversaw the program, (2) determining what results the program 
achieved, and (3) assessing how successful grantees were in obtaining 
outside funding. GAO analyzed Youthbuild performance data, visited 
Youthbuild sites, and interviewed agency officials. 

What GAO Found: 

While HUD requires grantees to report basic performance data, such as 
the number of program participants and graduates and job placements, 
HUD has not aggregated or analyzed the data and conducted limited 
oversight of grantees. According to HUD officials, they did not have 
staff available to analyze the closeout reports that grantees must 
submit, and a lack of resources also limited oversight of grantees. The 
monitoring HUD did primarily focused on compliance with program 
requirements such as documentation rather than on performance. As a 
result, HUD largely was unable to tell how the individual Youthbuild 
grantee programs performed. 

Limited outcome data preclude any overall assessment of the performance 
of the Youthbuild program; further, the few other analyses available 
such as the one GAO did in this study to augment limited existing data 
cannot be generalized programwide. GAO analyzed 245 closeout reports, 
representing 46 percent of the grantees or 12,863 participants. While 
GAO could determine percentages of participants who received high 
school diplomas or were placed in jobs, GAO could not determine 
outcomes over time, partly because the reports lacked baseline 
information and grantees were not required to and generally did not 
follow participants after graduation. Further, while closeout reports 
include information about impediments to program success and “best 
practices,” HUD did not systematically review this information or share 
it with its primary technical assistance contractor. Consequently, the 
lack of programwide evaluations, follow-up data, and dissemination of 
best practices make it very difficult to assess the performance of 
Youthbuild over time and determine which programs and strategies have 
worked best. Reporting on post-program performance outcomes, such as 
the number of participants placed and retained in construction-related 
employment, could increase the value of the closeout reports and better 
measure program results. Labor officials indicated that they would 
consider including such measures for program reporting. 

Grantees had varying success in obtaining funds from outside sources, 
but YouthBuild USA data suggest that continued (multiyear) HUD funding 
was critical to sustaining grantee operations and attracting leveraged 
funds. Grantees’ success in obtaining additional funds varied widely, 
from 21 grants reporting no additional funding sources to 40 reporting 
more than $1 million. While most grantees have generated outside 
funding, YouthBuild USA reported that most grantees have had difficulty 
continuing operations without continued HUD funding. Their data show 
that 90 percent of grantees ceased operations if not funded for 3 
consecutive years by HUD. Further, YouthBuild USA also noted that 
grantees with follow-on HUD funding achieved better performance 
outcomes, such as higher rates of job placements, than grantees that 
did not receive subsequent HUD funding. 

What GAO Recommends: 

GAO recommends that (1) HUD analyze closeout reports by grant and share 
information on identified problems and best practices with its 
technical assistance contractor and Labor, (2) Labor develop post-
program performance outcomes, and (3) Labor consider multiyear funding 
of grants for the program. HUD and Labor provided comments on a draft 
of this report. Labor generally agreed with the recommendations. HUD 
agreed but cited resource constraints in implementing its 
recommendation. 

[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-82]. 

To view the full product, including the scope and methodology, click on 
the link above. For more information, contact William B. Shear at (202) 
512-4325 or shearw@gao.gov. 

[End of Section] 

Contents: 

Letter: 

Results in Brief: 

Background: 

HUD's Grant-Making Procedures Incorporated Legislative Criteria: 

HUD Oversight of Grantees' Performance Has Been Limited: 

Incomplete Data on Participants' Outcomes Precludes an Overall 
Assessment of Program's Success: 

Youthbuild Grantees Have Experienced Varied Success in Obtaining 
Outside Funding Sources, and Many Cited Continued HUD Funding as 
Crucial: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments and Our Evaluation: 

Appendix I: Comments from the U.S. Department of Labor: 

Appendix II: GAO Contact and Staff Acknowledgments: 

Figures: 

Figure 1: Youthbuild Appropriations, Fiscal Years 1993-2006: 

Figure 2: Characteristics of Participants Entering YouthBuild USA- 
Affiliated Programs, 1997-2005: 

Figure 3: NOFA Point-Scoring System for Fiscal Year 2005 Applications: 

Figure 4: HUD Awards by Funding Category for Fiscal Years 2001-2006: 

Figure 5: Youthbuild Grant Applications and the Number Funded by Fiscal 
Year (1996-2006): 

Figure 6: Based on Closeout Reports, Youthbuild Performance Measures 
Indicate Varied Outputs: 

Figure 7: Outside Funding Amounts Varied Widely for Grants We Reviewed: 

Abbreviations: 

CPD: Community Planning and Development: 

GED: General Equivalency Diploma: 

GPRA: Government Performance and Results Act: 

HUD: Department of Housing and Urban Development: 

NOFA: Notice of Funding Availability: 

OMB: Office of Management and Budget: 

WIA: Workforce Investment Act: 

United States Government Accountability Office: 
Washington, DC 20548: 

February 28, 2007: 

The Honorable Patty Murray: 
Chairman: 
The Honorable Christopher Bond: 
Ranking Minority Member: 
Subcommittee on Transportation, Housing and Urban Development, and 
Related Agencies: 
Committee on Appropriations: 
United States Senate: 

The Honorable John W. Olver: 
Chairman: 
The Honorable Joe Knollenberg: 
Ranking Minority Member: 
Subcommittee on Transportation, Housing and Urban Development, and 
Related Agencies: 
Committee on Appropriation: 
House of Representatives: 

Since 1993, the Department of Housing and Urban Development (HUD) has 
provided funding for Youthbuild, a "second chance" program that trains 
and educates disadvantaged youth and helps build housing for low-and 
very low-income families across the United States. As stated in the 
authorizing legislation, the purposes of the Youthbuild program are (1) 
expanding the supply of affordable housing by utilizing economically 
disadvantaged young adults, (2) providing economically disadvantaged 
young adults with opportunities for meaningful work and service to 
their communities, (3) enabling economically disadvantaged young adults 
to obtain the education and employment skills necessary to achieve 
economic self-sufficiency, and (4) fostering the development of 
leadership skills and commitment to community development among young 
adults in low-income communities.[Footnote 1] Although a few studies 
have shown instances where Youthbuild graduates credit the program for 
helping them overcome difficult circumstances, gain critical job 
skills, and continue their education, concerns have been expressed 
about whether HUD's program oversight has been sufficient to adequately 
measure program outcomes. 

The Housing and Community Development Act of 1992 authorized the 
Youthbuild program. The program was based on the Youth Action Program 
developed in East Harlem, New York, in the late 1970s in which high 
school dropouts received academic training and learned job skills by 
renovating abandoned buildings. The YouthBuild Transfer Act shifted 
program oversight of grants issued beginning in fiscal year 2007 to the 
Department of Labor (Labor)--in part, to better align the program with 
existing federal workforce development and youth training 
programs.[Footnote 2] The Transfer Act also expanded the program's 
scope to allow construction and rehabilitation of community facilities 
in addition to housing. HUD will continue to oversee Youthbuild grants 
funded through fiscal year 2006. That is, HUD's oversight 
responsibility could continue for several more years since grants can 
remain active for 3 years or more. 

Noting concerns regarding the overall quality of the Youthbuild 
program, the Senate report accompanying the fiscal year 2006 
Transportation, Treasury, Housing and Urban Development, the Judiciary, 
the District of Columbia, and Independent Agencies Appropriations Act 
directed us to comprehensively assess the Youthbuild program's overall 
success, including its ability to attract local support and funding. 
Specifically, the objectives of this report are to (1) describe how HUD 
makes Youthbuild program grants; (2) evaluate how HUD assesses and 
oversees the Youthbuild program; (3) determine how successful the 
Youthbuild program has been in achieving desired results, such as 
providing the education and job training needed to achieve economic 
self-sufficiency and supplying affordable housing; and (4) assess how 
successful local Youthbuild organizations have been in obtaining 
private funding and support. 

We focused our analysis on reviewing HUD's grant making and oversight 
processes and evaluating available data for measuring program success. 
To determine how HUD makes annual program grants, we reviewed the 
application and review process HUD uses to identify and select 
organizations for grant awards.[Footnote 3] To evaluate how HUD 
assesses and oversees the program, we interviewed HUD officials at the 
national and field offices. We also obtained information on the extent 
to which grantees are monitored and the results of the monitoring. 
Further, we obtained information on the type of performance data 
grantees submit to HUD and HUD's role in reviewing the information. We 
also obtained funding and grant award information from HUD. We assessed 
the reliability of these data by comparing the information with other 
data sources to identify obvious problems with completeness or 
accuracy, and interviewing knowledgeable agency officials about the 
data. We determined that the data were sufficiently reliable for the 
purposes of this report. To develop information on how successful the 
Youthbuild program has been, we consulted relevant literature, 
including three studies of the Youthbuild program's effect on 
participants' lives.[Footnote 4] However, the studies were limited in 
scope and methodology and constrained by the absence of a database 
containing performance information from all the grantees in the 
Youthbuild program. We also interviewed officials and received 
information from HUD's primary technical assistance contractor, 
YouthBuild USA. Further, because HUD lacked a centralized database on 
grantees' program performance results, we analyzed 245 grant closeout 
reports submitted by grantees to HUD from April 2000 to August 
2006.[Footnote 5] We also visited nine Youthbuild programs in 
California, Illinois, Massachusetts, Pennsylvania, Rhode Island, and 
Washington, D.C.[Footnote 6] The sites were chosen in part because the 
programs varied in size, leadership style, and teaching approach, 
including the charter school format. The closeout reports we reviewed 
represented about 46 percent of the closed grants awarded from 1997 
through 2003. We were unable to review the remainder of the reports 
either because HUD was not able to provide a copy of the closeout 
reports, the grantee had not submitted a closeout report, or the report 
did not have sufficient information for us to use. In addition, we 
reviewed monitoring reports covering 16 grantees prepared by HUD field 
office staff as part of their program oversight duties. To assess how 
successful local Youthbuild organizations have been in obtaining 
private funding and support, we analyzed the closeout reports to 
determine the extent to which the individual grantees obtained 
additional funding. From a list of 100 grantees from YouthBuild USA 
that closed their operations, we obtained information on a 
nonprobability selection of 23 grantees by directly contacting them or 
from other knowledgeable sources to determine how HUD funding decisions 
contributed to their closure. We selected and contacted grantees 
primarily based on those that had received more than one grant and 
which we were able to locate because they were still providing other 
social services. We also reviewed literature and reports on Labor's 
youth employment and training programs and met with officials from 
Labor, where oversight of the Youthbuild program will be transferred. 
We conducted our work in Boston and Somerville, Massachusetts; 
Washington, D.C; and various Youthbuild locations in the states noted 
above from March 2006 through February 2007 in accordance with 
generally accepted government auditing standards. 

Results in Brief: 

Each year, HUD staff scored grant applications based on factors 
outlined in the enacting legislation for serving economically 
distressed areas and detailed in an annual Notice of Funding 
Availability (NOFA). The Housing and Community Development Act of 1992 
required HUD to establish selection criteria such as the qualifications 
or potential capabilities of applicants, the feasibility of the 
proposed Youthbuild program, the potential for developing a successful 
Youthbuild program, and the applicants' commitment to obtain outside 
resources. HUD incorporated these criteria in an annual NOFA, which 
directed applicants and detailed the scoring process for grant awards. 
The award process has been competitive, with HUD having sufficient 
resources to fund less than one-third of the applicants since 1996. 

While HUD requires grantees to submit closeout reports with basic 
performance data, HUD has not verified, aggregated, or analyzed the 
data and conducts limited oversight of grantees. The data in the 
closeout reports were intended for performance assessments; 
specifically, in its instructions to grantees, HUD states that closeout 
reports will be used to help evaluate the grantees' programs. Further, 
in accordance with federal performance management standards, accurate 
and complete performance data are needed to inform Congress and the 
public about the effectiveness of programs and help agencies 
effectively manage and oversee programs.[Footnote 7] HUD officials said 
that they did not have staff available to verify, aggregate, or analyze 
the closeout reports that they required Youthbuild grantees to submit; 
instead, they contracted with YouthBuild USA to collect a set of 
performance outputs on a quarterly basis, which provides a more limited 
perspective on grantee performance because the data have not been 
aggregated for the term of the grants. While HUD has recognized the 
importance of performance information by planning for a computerized 
data collection system to capture performance information on Youthbuild 
and other competitive grant programs, the implementation has been 
delayed since 2003 due to funding and technical problems. In addition, 
according to HUD officials, a lack of resources also limited assessment 
and oversight of Youthbuild grantees. For example, HUD field offices, 
which are responsible for overseeing the Youthbuild program, typically 
were limited to visiting and reviewing one or two grantees annually. 
HUD's monitoring primarily focused on compliance with program 
requirements such as documentation rather than on performance. As a 
result of a lack of verified and aggregated data and analysis and 
limited oversight, HUD largely was unable to tell how well the 
individual Youthbuild grantee programs performed. 

In addition, lack of programwide performance data and analysis limited 
any overall assessment of the success of the Youthbuild program, and 
limited available analyses cannot be used for programwide assessments. 
Since 2002, a few studies have provided some evidence of the success of 
individual Youthbuild programs. While limited in scope, the studies 
have shown that, at a small number of grantees, Youthbuild has been 
successful in assisting youth to enter higher education programs and 
obtain job skills. The value of the data that YouthBuild USA collects 
in its Web-based system is limited as well; the data are self-reported 
and contain little information on participant outcomes, partly because 
grantees are not required to follow participants after graduation. To 
augment limited existing data, we analyzed the performance information 
available in 245 closeout reports that HUD supplied to us, which 
covered 12,863 program participants. The results cannot be generalized 
to the entire program. For each of the grantees that reported 
information, we calculated percentages of participants who completed or 
placed in an education program or in employment. The median percentage 
of participants who obtained a general equivalency or high school 
diploma across all the programs that reported information was 28, 
compared with 36 percent who were placed in jobs, and 7 percent who 
went on to higher education. However, because the reports lack some 
baseline information and grantees generally did not follow participants 
after program completion, we were unable to determine long-term 
outcomes such as percentages or number of participants receiving 
degrees or retaining employment. Moreover, while closeout reports do 
include information about impediments to program success and "best 
practices," HUD did not systematically review this information or share 
it with YouthBuild USA, which is the primary technical assistance 
contractor for the program with responsibilities that include 
identifying new strategies for operating successful programs. As a 
result, the lack of follow-up data and programwide evaluations make it 
very difficult to determine which programs and strategies have worked 
best and assess the performance of Youthbuild over time. As Labor 
assumes program oversight responsibilities, officials said they would 
consider that the closeout reports include program participant data, 
such as degrees obtained prior to program entry and details on 
participant outcomes such as the types of construction jobs obtained 
and retained upon program completion. The officials also told us that 
they believe impediments and best practices should be evaluated and 
disseminated to grantees, but they had not yet determined who would 
have primary responsibility for doing so. 

Grantees had varying success in obtaining funds from outside sources, 
but grant officials told us that HUD funding was critical to attracting 
the leveraged funds. In assessing applications, HUD gave consideration 
to program applicants that were able to secure outside funding. 
Applicants receive points based on the ratio of committed leveraged 
funds compared with the amount of Youthbuild funds requested in the 
application. Data from the closeout reports showed the median amount 
spent by HUD Youthbuild grantees was $400,000 in HUD funds and an 
additional $430,000 from outside sources. However, success in obtaining 
additional funds varied widely, from 21 grants reporting no additional 
funding sources to 40 reporting more than $1 million. While most 
grantees have been successful in generating outside funding, several 
grant officials told us that many existing and potential contributors 
would not be willing to continue funding if they perceived that the 
program would not be able to continue without the HUD grant. YouthBuild 
USA reported that most grantees have had difficulty continuing 
operations without continued HUD funding. Further, YouthBuild USA also 
noted that grantees with follow-on HUD funding achieved better 
performance outcomes than grantees that did not receive subsequent HUD 
funding. 

This report contains three recommendations for the Secretaries of HUD 
and Labor to analyze closeout reports and share the best practices 
identified, develop post-program performance outcomes and share the 
data with grantees, and consider whether multiyear funding could be 
useful in helping Youthbuild grantees attract additional outside 
funding. We provided a draft of this report to HUD and Labor for their 
review and comments. Officials from HUD's Office of Community Planning 
and Development (CPD) provided comments. They stated that they did not 
disagree with the report's overall findings, but added that HUD's 
oversight activity was limited by resource constraints. Labor's 
Assistant Secretary for Employment and Training wrote that she agreed 
with our recommendation to develop and monitor post program performance 
outcome measures and share data with grantees. She also wrote that 
Labor agreed that there could be value in multiyear funding and that 
the department was considering such an approach for existing grantees. 

Background: 

CPD manages the Youthbuild program. CPD programs generally fall into 
three categories: (1) formula grants, of which the largest are the 
Community Development Block Grant and the HOME Investment Partnership 
Program, which account for the majority of CPD funding; (2) competitive 
homeless programs; and (3) competitive nonhomeless programs, including 
Youthbuild. While no staff are dedicated solely to Youthbuild 
oversight, CPD monitors the program as part of its oversight 
responsibilities for 22 programs and initiatives, which 829 staff (233 
in headquarters and 596 in field offices) administered as of the 
beginning of fiscal year 2007. In fiscal year 2006, Congress 
appropriated $19.1 billion for the CPD programs and initiatives, of 
which Youthbuild represents a small portion.[Footnote 8] The first 
program appropriation for Youthbuild was $40 million for fiscal year 
1993. Appropriations reached a high of $65 million in fiscal year 2002, 
but dropped to $49.5 million in fiscal year 2006, the last year in 
which HUD was authorized to award grants (see fig. 1).[Footnote 9] 

Figure 1: Youthbuild Appropriations, Fiscal Years 1993-2006: 

[See PDF for image] 

Source: HUD. 

[End of figure] 

HUD has provided program funding for Youthbuild through an annual 
competitive grant application process. Any applicant can receive up to 
a $700,000 grant; however, new applicants and rural/underserved 
applicants primarily receive grants up to $400,000. Grant awards can 
last up to 30 months (or longer if the grantee is given an extension), 
but some grantees use the funds at a much quicker rate and usually 
reapply every year, while others may take the full 30 months. 

Prospective grantees are state, private, nonprofit, and local entities 
that implement the Youthbuild program by serving low-income youth, ages 
16 to 24, who demonstrate educational need--either by virtue of being 
high-school dropouts or by justifying their need for inclusion in the 
program. According to program regulations, at least 75 percent of the 
participants must come from very low-income families and have dropped 
out of high school.[Footnote 10] The program allows up to 25 percent of 
the youth to have a high-school diploma or have a slightly higher 
income; however, data developed by YouthBuild USA show that most 
organizations report that less than 25 percent of participants have 
high-school diplomas or General Equivalency Diplomas (GED) upon 
entering their programs. Also, according to data from YouthBuild USA, a 
number of the youths have been adjudicated (judicially determined to be 
delinquent), were on welfare upon entering the program, or lived in 
public housing (see fig. 2). Other studies we reviewed identified 
similar participant characteristics. Program regulations require 
participants to spend 50 percent of their program time receiving job 
training on construction sites, building affordable housing, and 50 
percent receiving academic training. Recently, some of the Youthbuild 
grantees have added nonconstruction programs to their offerings, such 
as in the areas of computer repair and health care. Funding for the 
additional programs has come from AmeriCorps.[Footnote 11] 

Figure 2: Characteristics of Participants Entering YouthBuild USA- 
Affiliated Programs, 1997-2005: 

[See PDF for image] 

Source: YouthBuild USA. 

Note: Participants could have one or more of depicted characteristics. 

[End of figure] 

In 1994, YouthBuild USA, located in Somerville, Massachusetts, became 
the primary technical assistance contractor for the Youthbuild program. 
YouthBuild USA is a national support center and intermediary for the 
program. As such, it provides training, develops materials and 
handbooks to help grantees replicate the program, conducts peer-to-peer 
seminars, and disseminates materials on best practices. YouthBuild USA 
also has established an affiliated network that grantees can choose to 
join. As of September 2006, 107 of 173 Youthbuild grantees with active 
grants were affiliates. Since 2002, YouthBuild USA has been using a Web-
based system, WebSTA, to collect participant characteristic and 
performance data from its affiliates. 

Among the purposes of transfering the YouthBuild program to Labor was 
better alignment of existing federal workforce and youth training 
programs. Labor's Employment and Training Administration offers 
programs that assist disadvantaged youths, similar to the programs that 
Youthbuild grantees offer. The Workforce Investment Act (WIA) of 1998 
established a variety of programs to serve low-income youths who face 
barriers to employment. Labor allocates funds for the youth service 
programs to state and local areas based on a distribution formula. The 
grants are made to states, and the states in turn allocate funds to 
approximately 600 local workforce investment boards. The amount of 
funding available to states is determined by the distribution of 
unemployed individuals and disadvantaged youths by state. From 2000 
through 2005, Labor administered the Youth Opportunity Grant program, 
which also was authorized by WIA. This grant program focused on 
improving education and employment opportunities for at-risk youths, 
ages 14 to 21, in high-poverty areas. Funding for the program was 
eliminated beginning in fiscal year 2004. In addition, Labor 
administers the Job Corps program, which began in 1964, and provides 
education, training and support services to economically challenged 
youths, ages 16 through 24, who face multiple barriers to employment. 
Private companies--chosen through competitive contracting processes-- 
and other federal agencies operating under interagency agreements with 
Labor, operate the Job Corps centers (located throughout the United 
States and Puerto Rico) on behalf of Labor. 

HUD's Grant-Making Procedures Incorporated Legislative Criteria: 

Each year, HUD staff rated and ranked grant applications for serving 
economically distressed areas based on factors outlined in the enacting 
legislation and detailed in an annual NOFA. The Housing and Community 
Development Act of 1992 required HUD to establish selection criteria 
such as the qualifications or potential capabilities of applicants, the 
feasibility of the proposed Youthbuild program, the potential for 
developing a successful Youthbuild program, and the applicants' 
commitment to obtain outside resources.[Footnote 12] 

Accordingly, HUD annually issued a NOFA that described the grant 
application process, outlined criteria for grant awards, and included 
scoring factors (see fig. 3). The major factors considered were the 
capacity and relevant experience of the organization (performance), 
need and extent of the problem, applicant's soundness of approach, 
leveraging of nonhousing funds, and ability to show how it would 
achieve and measure results. Under these factors, grant applicants had 
to consider area poverty and high-school dropout rates, describe their 
management and training capabilities, and offer proof of access to 
property and the existence of partners to build low-income housing. HUD 
issued its last NOFA for the Youthbuild program on March 8, 2006, and 
announced the winning grantees in November 2006. 

Figure 3: NOFA Point-Scoring System for Fiscal Year 2005 Applications: 

[See PDF for image] 

Source: HUD. 

[End of figure] 

Since fiscal year 2001, HUD has grouped applicants into three separate 
scoring categories--new, general (that are typically awarded to 
established Youthbuild organizations), and rural/underserved area 
grantees. Figure 4 shows the awards by category. The total numbers of 
grants as well as the grants by the individual categories have varied 
by year. 

Figure 4: HUD Awards by Funding Category for Fiscal Years 2001-2006: 

[See PDF for image] 

Source: HUD. 

[End of figure] 

The awards process also has been competitive. HUD historically funded 
less than one-third of all the applicants (see fig. 5). Since 1996, the 
number of applicants funded ranged from about 7 to 44 percent. 

Figure 5: Youthbuild Grant Applications and the Number Funded by Fiscal 
Year (1996-2006): 

[See PDF for image] 

Source: HUD. 

[End of figure] 

HUD Oversight of Grantees' Performance Has Been Limited: 

Although HUD requires grantees to submit closeout reports, which are 
intended to help evaluate the programs, it has not verified, 
aggregated, or analyzed data from those reports. Further, the closeout 
reports do not include other data that could help determine participant 
characteristics upon entry or the types of jobs they received after 
completing the program. YouthBuild USA, the technical services provider 
for HUD, collects quarterly information, but it has not aggregated the 
information to obtain performance information over the span of a grant. 
HUD also has encountered significant delays in developing a database 
intended to gather additional performance information from grantees. 
Finally, citing resource constraints, HUD has conducted limited on-site 
oversight of grantees. As a result of a lack of verified and aggregated 
data or analysis and limited oversight, HUD largely was unable to tell 
how well the individual Youthbuild grantees performed. 

HUD Has Not Collected or Analyzed Data That Could Help It Assess 
Grantees: 

HUD requires grantees to submit performance reports, including closeout 
reports, but HUD has not developed or fully utilized information-- 
particularly from the closeout reports--that could help it assess 
Youthbuild grantee programs and has lacked resources to conduct 
comprehensive oversight of grantees. Such information serves as the 
basis for management and oversight of programs. For instance, HUD 
states in the instructions for filling out the closeout reports that 
the reports will be used to monitor and evaluate progress of grantees 
and programs. Further, the Government Performance and Results Act of 
1993 (GPRA) shifted the focus of federal managers from the number of 
tasks completed or services provided to a more direct consideration of 
the outcomes or results of the program. GPRA provides a performance- 
based management framework for agencies to set goals; measure progress 
toward those goals; deploy strategies and resources to achieve them; 
and ultimately, use performance information to make the programmatic 
decisions necessary to improve performance. Accurate and complete 
performance data that document outcomes are needed to inform Congress 
and the public about the effectiveness of programs and help agencies 
conduct effective oversight. The combination of baseline information 
and intermediate outputs or outcomes can be used to show progress or 
contribution toward final outcomes, that is, the intended results of 
carrying out the program.[Footnote 13] For example, in Youthbuild, 
baseline information such as the number of participants entering a 
program with and without GEDs or high-school diplomas could be combined 
with intermediate outputs and outcomes such as graduation rates and 
jobs attained, which could then be linked to desired program outcomes 
such as the number of Youthbuild participants attaining economic self- 
sufficiency. 

However, HUD has not set up a system to verify, aggregate, or analyze 
the data from closeout reports--information and analyses that would 
form the basis for assessments of grantee performance. Citing resource 
constraints, HUD filed the reports in the individual grant files in 
headquarters and field offices and did not further review them. In 
addition, some basic baseline data are missing from the closeout 
reports, including the number of participants who enter the programs 
with a GED or high-school diploma. Such information is necessary to get 
an accurate count of the number of participants receiving a GED degree 
or diploma through Youthbuild programs and also provides a basis for 
comparison with which performance can be measured. Furthermore, while 
closeout reports require that some performance outputs be collected, 
including the number of participants receiving GEDs or high-school 
diplomas, placed in jobs, or entering higher education programs, the 
reports do not collect other output information such as how many 
participants were placed in construction-related employment. 

Citing resource constraints and a need to comply with the Office of 
Management and Budget (OMB) performance reporting requirements, HUD 
contracted with YouthBuild USA in 2004 to collect quarterly grantee 
performance information for all Youthbuild grantees. Programs 
administered by six federal agencies, including HUD and Labor, are 
subject to "common measures" reporting which OMB established to provide 
agencies with the ability to describe, in a similar manner, their core 
performance outputs across different grantee programs.[Footnote 14] The 
common measures include employment or education placements, degrees or 
certificates attained, and literacy and numeracy skills increased 
(increases in educational functioning). Consequently, YouthBuild USA 
provides HUD with program information including participant gender and 
ethnicity data, GED attainment statistics, grade level improvements, 
job or school placements, and the number of new and rehabilitated 
housing units that were completed. HUD uses the reports to provide 
performance information to Congress and meet OMB reporting 
requirements. However, because the information is captured on a 
quarterly basis, it only represents the performance of a program at a 
particular point in time and YouthBuild USA has not aggregated the 
information to obtain data for the term of individual grants. As a 
result, HUD has been largely unable to track program outcomes, but its 
data will be needed for ongoing program assessments. 

Moreover, HUD has encountered significant delays in developing a 
database intended to provide additional performance outcomes on grantee 
programs. Recognizing this deficiency, HUD officials said that they 
have been planning for about 3 years to implement an electronic data 
collection system to help HUD measure the success of Youthbuild 
programs, along with its other nonhomeless competitive grant programs. 
However, the system's implementation has been delayed because of 
funding and technical problems. As a result of not systematically 
collecting or analyzing needed information from closeout reports, 
relying predominately on limited point-in-time program information, and 
not being able to utilize a planned database to aggregate performance 
outcomes, HUD is limiting its ability to assess grantees. Further, 
although the YouthBuild Transfer Act shifted oversight of grants issued 
in fiscal year 2007 to Labor, HUD will be required to continue 
monitoring existing Youthbuild grants for approximately the next 3 
years. Therefore, the data HUD has at its disposal or could develop 
will continue to be valuable for ongoing assessments of the program. 

Citing Resource Constraints, HUD Performed Limited Oversight of 
Grantees: 

In addition to not having or utilizing available performance data, HUD 
has exercised limited on-site review and oversight of grantee 
performance. According to HUD's Office of Inspector General, on-site 
monitoring is an essential tool for HUD (through CPD) to assess program 
performance and identify and address potential program problems. Such 
monitoring is also useful in motivating grantees to exercise sound 
judgment in carrying out their grant activities. GAO's Standards for 
Internal Control in the Federal Government note that controls generally 
should be in place to ensure that ongoing monitoring occurs during 
normal operations. Monitoring should be performed continually, be 
ingrained in an entity's operations, be used to assess the quality of 
performance over time, and support the prompt resolution of any 
identified problems.[Footnote 15] 

CPD goals require on-site monitoring of only a small percentage of 
grants. For fiscal year 2003, the plan called for the field offices to 
monitor 35 percent of formula grants; 20 percent of competitive 
homeless grants; and 7 percent of competitive nonhomeless grants, 
including Youthbuild. HUD has since changed its baseline monitoring 
goals (requiring monitoring of 20 percent of grantees with active, open 
grants established since 2004) to give its field offices more 
flexibility to meet monitoring priorities based on local grantee 
knowledge. As a result, goals are no longer set by program. Field 
offices decide on the number of formula versus competitive grantees to 
be monitored based on risk analyses that evaluate each grantee on 
factors such as the size of the grant, perceived project management 
capacity, grantee responsiveness, and the extent of citizen complaints. 
HUD officials cited a lack of resources as the primary reason for not 
requiring the monitoring of a larger percentage of grantees. In 2006, 
according to HUD officials, field offices typically had one person 
working part time responsible for overseeing the individual Youthbuild 
grantees in the state, visiting no more than one or two grantees per 
year. As a result, field staff have not been able to comprehensively 
monitor grantees and verify the accuracy of the output data submitted 
by the grantees. 

In general, we found that the monitoring reports that we reviewed 
(which HUD field offices produced) on grantees primarily addressed 
various compliance issues, rather than reporting on the overall 
performance of a particular grant. For example, several reports 
identified grantees' failure to receive prior approval from HUD for 
changing proposed job training construction sites. Other reports 
identified inadequate record keeping, insufficient documentation, and 
lack of sufficient financial internal controls on the part of grantees. 
However, although a few reports did comment on some individual 
performance outputs, such as job placements or GEDs attained, most did 
not. 

The lack of analysis of performance information of grantees also has 
implications for assessing the Youthbuild program overall, particularly 
because HUD has devoted few staff to overall program management and 
oversight. Since 2000, in headquarters, Youthbuild had a program 
director with one or two staff managing the program each year. During 
fiscal year 2006, with the expected program transfer to Labor, the HUD 
program manager was assigned to other duties, and a part-time program 
manager was assigned to oversee the 2006 grant process, in which HUD 
received 308 applications and awarded 74 grants. Although they are 
aware of the limited oversight of Youthbuild, CPD division officials 
said it has provided appropriate staff resources for program oversight 
of Youthbuild grantees, commensurate with the program's size--the 
annual appropriation for which represents less than 1 percent of CPD's 
usual annual budget of about $8 billion. However, without sufficient 
oversight HUD was unable to tell how the individual Youthbuild grantee 
programs performed. 

Incomplete Data on Participants' Outcomes Precludes an Overall 
Assessment of Program's Success: 

HUD has not performed the programwide analysis needed to determine 
overall program success, and weaknesses in other assessments of the 
programs and data preclude an overall assessment of Youthbuild success. 
While limited in scope, three studies suggest Youthbuild helped youths 
achieve economic self-sufficiency. And, although YouthBuild USA tracks 
performance data, grantees do not have the follow-up information needed 
to measure performance outcomes. To augment limited existing data, we 
analyzed information in 245 closeout reports, but the results cannot be 
generalized to the entire program. We found a wide range of results in 
outputs such as job placements; however, because grantees generally did 
not follow up on participants, we were unable to determine long-term 
outcomes such as percentages of participants retaining employment. 
Moreover, while closeout reports include impediments to success and 
"best practices," HUD did not systematically review this information or 
share it with YouthBuild USA. As a result, the lack of follow-up data 
and programwide evaluations make it very difficult to determine which 
programs and strategies have worked best and assess the performance of 
Youthbuild over time. Labor officials said they would consider that the 
closeout reports include outcomes such as jobs obtained and retained 
and said that they believe impediments and best practices should be 
evaluated and disseminated to grantees. 

A Few Studies Document Individual Youthbuild Program Successes but Are 
Limited in Scope: 

A few studies have provided some evidence of the success of individual 
Youthbuild programs. Although subject to many limitations, the studies 
suggest that identified programs have achieved certain Youthbuild 
objectives. However, without comparable information for the entire 
population of Youthbuild grantees, it is not possible to determine the 
extent to which these outcomes and reported successes are reflective of 
what was achieved by the grantees not included in the studies. More 
specifically: 

* In June 2004, researchers from Brandeis and Temple Universities 
published a study entitled, "Life after YouthBuild - 900 YouthBuild 
Graduates Reflect on Their Lives, Dreams, and Experiences," in 
collaboration with YouthBuild USA. The study was based on a survey of 
graduates from 11 Youthbuild grant sites and included in-depth 
interviews with 57 graduates from eight of these sites. The researchers 
determined that, for those 882 participants (for a 23 percent response 
rate) who responded to the survey, many are doing well and were helped 
considerably by the Youthbuild program. Many participants found jobs or 
furthered their education and have volunteered in community activities. 
In addition, a number of participants surveyed indicated they also 
obtained intangible benefits from the program, such as increased self- 
esteem and had developed positive attitudes about themselves and their 
community. A limited nonresponse analysis was conducted by YouthBuild 
USA to compare the 882 survey responses and the 57 interviews with one 
site that had a 98 percent response rate. However, using one site that 
had a high response rate is not representative and cannot be 
generalized to the whole sample. As such, it is not known if the 77 
percent who did not respond were doing more or less well than those who 
responded to the survey. 

* In August 2003, HUD issued a study entitled, "Evaluation of the 
Youthbuild Program," in which a HUD contractor compared program results 
and costs per participant from the Youthbuild Program and four other 
federal employment and training programs--Job Corps, JOBSTART, both 
designed to serve disadvantage youth, and the youth components of the 
Job Training Partnership Act and Supported Work Demonstration 
programs[Footnote 16]. The researchers conducted their work at 20 
Youthbuild sites, selected in part because they had received funding 
for at least two grants, obtained a variety of organizational 
sponsorships, built new or rehabilitated housing, and covered a variety 
of funding category sizes. The study found that 36 percent of the 
Youthbuild participants at these sites obtained jobs after enrolling 
and 29 percent obtained either a GED or high-school diploma. In 
addition, 12 percent of the Youthbuild graduates in the 20 sites 
reportedly went onto higher education. However, results at the five 
federal programs were not directly comparable because researchers 
obtained data from Youthbuild program participants at the time they 
exited the program while data from the other programs' were obtained 
from tracking participants up to 3 years after leaving the program. In 
addition, the limited scope of this study precludes an overall 
assessment of the program's success. 

* In February 2003, the state of Minnesota issued a study entitled, 
"Minnesota Youthbuild Program: A Measurement of Costs and Benefits to 
the State of Minnesota." This study analyzed the costs and benefits of 
the state-funded Youthbuild program in Minnesota. The researchers 
measured post-program participant performance outcomes by tracking 
program year participants beyond graduation. They found that Youthbuild 
graduates earned more than double the minimum wage, whereas youth with 
similar characteristics who had not been through the program typically 
earned only the minimum wage. In addition, the graduates who had been 
adjudicated had a considerably lower rate of recidivism (rearrest, 
reconviction, or return to prison) than other youth involved with the 
correctional system.[Footnote 17] Using state administrative data to 
estimate the amount of revenues and savings generated through increased 
tax revenues and reduced prison costs, the study estimated first-year 
benefits to the state of about $1.5 million dollars versus costs of 
about $877,000. Over a 4-year period, the study estimated the total net 
benefit to the state at about $7.3 million (by 2006). The study was 
limited to costs and benefits directly related to state programs and 
did not include potential benefits such as reductions to welfare rolls, 
state-sponsored medical care, or other reductions in human services 
costs. 

YouthBuild USA WebSTA Data Have Limitations, Particularly for Assessing 
Outcomes: 

Since 2002, YouthBuild USA has been using a Web-based system--WebSTA-- 
to collect participant characteristics and performance data from its 
affiliates. According to YouthBuild USA officials, as of December 2006, 
about 90 percent of its affiliates were entering data into WebSTA to 
some extent. However, the value of YouthBuild USA's WebSTA data is 
limited, largely because grantees self-report the data, not all 
grantees are meeting the requirement to send in information, and the 
information is not reported at the grant level. That is, if affiliates 
have more than one Youthbuild grant, the data for each grant are not 
reported separately. As a result, the data are not verified, 
comprehensive, or comparable. Also, because the grantees submit the 
information quarterly, the data provide only a point-in-time snapshot 
of the program's performance and cannot be summed to provide annual 
data. Recognizing these limitations, according to YouthBuild USA's 
President, upgrades are planned to be implemented in early 2007 that 
will allow grantees to enter data in WebSTA by individual grant, which 
in turn will allow YouthBuild USA to analyze grant-level performance. 

We also found that the limited depth of the information collected 
restricts the value of WebSTA information for performance assessments. 
For example, WebSTA collects information on participant attendance, 
retention and graduation rates, and initial job and college placements 
and participant characteristics (such as welfare, public housing, and 
adjudication status) when participants enter Youthbuild programs. 
However, many grantees do not follow up on participants once they leave 
the programs. As a result, they often do not know if students placed in 
jobs were still working at those jobs weeks or months later. Also, 
several program directors we visited said that most of their program 
graduates who go on to higher education sign up for free remedial-level 
courses at nearby colleges. Although they have not had the funding to 
systematically follow the students' progress, some said they generally 
have found that the students have had varying success in advancing to 
college-level credit courses. 

Without grantee-level follow-up systems, WebSTA cannot track program 
outcomes that would reflect whether the graduates are achieving 
economic self-sufficiency--such as the percentage of program graduates 
who remove themselves from the welfare rolls, move out of public 
housing, or stay out of the court system. The system also does not 
collect follow-up information on the type of jobs attained (including 
construction), pay raises, benefits received, and college or trade 
degrees or certifications received. Analyzing such performance outcomes 
could provide, HUD, Labor, and other interested parties with the 
measures needed to evaluate overall program performance and success. 

Closeout Reports We Analyzed Provided Little Detail on Long-Term 
Outcomes Because Grantees Are Not Required to Follow up on Graduates: 

Because HUD had not aggregated or analyzed data for Youthbuild, we 
obtained and analyzed 245 closeout reports that grantees submitted to 
HUD. Figure 6 provides selected performance information that we 
summarized from the closeout reports. However, because HUD has not 
established objective performance goals for the program, which would 
serve as baseline criteria against which to judge performance, the 
information related to the experience of the 12,863 participants who 
entered the program under the 245 grants gave us limited perspectives 
about how individual programs were working. In addition, because we had 
information on only 245 (or 46 percent) of the grants, the results 
cannot be generalized to the entire program. 

As shown in figure 6 the percentages for participants graduating, 
receiving a GED, or being placed in employment varied widely. As 
previously discussed, HUD has not set benchmarks, and critical data 
elements (baseline data) are missing in the closeout reports. For 
instance, it is not possible to obtain an accurate number of diplomas 
and GEDs attained from the closeout reports because the reports do not 
contain data on the number of participants that enter the program with 
a high-school diploma or GED. As result, we could not determine how 
well the programs (except for those with the very highest percentages) 
performed individually or in comparison with each other. 

Figure 6: Based on Closeout Reports, Youthbuild Performance Measures 
Indicate Varied Outputs: 

[See PDF for image] 

Source: GAO analysis of grantee closeout reports. 

[A] Not all grantees provided information for each category. 

[B] Includes 1,239 participants that were reported as still active 
because they were carried over to a subsequent grant or allowed to stay 
on the books to receive continuing education and other support 
services. In addition, the drop outs, graduates, and still active 
categories do not sum to the number of participants category because 
all grantees did not report information for each category. 

[C] Regulations allow up to 25 percent of the participants who enroll 
to already have a GED or high-school diploma. As a result, this number 
may be underrepresentative since the data do not specify the number of 
participants that entered the program with a GED or high-school 
diploma. 

[D] The same participants can be included in the employment and higher 
education categories. 

[End of figure] 

The closeout reports also do not provide consistent and reliable data 
on the number of units rehabilitated or built by Youthbuild 
participants. A basic purpose for which Youthbuild was authorized was 
to expand the supply of affordable housing. According to the closeout 
reports, 158 grantees reported that their participants built a total of 
1,483 new units, while 139 grantees reported that their participants 
rehabilitated a total of 2,238 units.[Footnote 18] Some of the 
Youthbuild program directors with whom we met said that during a 
typical reporting period they would complete from one to four new or 
gut-rehabilitated housing units. Other directors said that when a 
program reports about 200 units rehabilitated, it generally means the 
students are performing minor repetitive tasks such as changing locks 
or painting a limited number of interior walls. As a result, the number 
of units rehabilitated likely is overstated in the closeout reports. 
Although construction training is a major part of the Youthbuild 
program, HUD's closeout reports also do not track the number of 
graduates obtaining construction-related jobs or specify the types of 
jobs obtained (construction or otherwise). 

In addition, HUD officials told us that grantees are not required to 
follow up on participants after they leave the program. While some 
grantee officials with whom we met recently had attempted to establish 
follow-up systems, most grantees cited funding constraints as the 
reason for not following up on graduates. Several officials further 
noted that it was difficult to locate many participants after they had 
left the program. However, as a result, it is not possible to obtain 
comprehensive post-program results. 

Finally, some program directors with whom we spoke cautioned that any 
follow-up system should not be strictly focused on students obtaining 
higher education degrees or obtaining high-paying jobs. They emphasized 
that such goals were beyond the abilities of some of their 
participants, but that moving from a fifth-grade to a ninth-grade 
reading level or obtaining a service sector job sometimes was a real 
victory and should be counted as a success. 

Limited Information We Developed from Site Visits Pointed to Varied 
Results for Programs, and Also Provided Little Detail on Long-Term 
Outcomes: 

The information we obtained from our visits also demonstrates varied 
results for continued employment and academic success, and little 
detail on long-term success. As with our closeout report analysis, the 
information cannot be generalized to the universe of Youthbuild 
grantees--although YouthBuild USA officials told us that they believe 
we visited some of the more innovative grantees. For example, although 
not required, some of the nine grantees we visited used other funding 
sources, such as United Way or AmeriCorps funds, to establish a 
participant follow-up system. As a result, some of the grantees were 
able to provide us with varying amounts of information on the status of 
program graduates. For example: 

* One grantee, which was able to track individuals who completed the 
program since 2001, provided us with data that showed 70 of the 82 
program participants were either employed or pursuing higher education. 
Seven of the 70 participants were working and going to school. The 
hourly wages of those employed ranged from $7 to $13.75. However, the 
grantee did not have information on the types of employment. 

* Another grantee that recently established a follow-up system showed 
that 11 of the 26 program graduates who started in 2004 had obtained 
employment. The positions ranged from pizza delivery person to union 
carpenter. However, only the union carpenter and a graduate hired by 
the program as a construction trainer were working in the construction 
field. The hourly wages of the graduates ranged from $5.50 to more than 
$16 per hour for a graduate hired by the grantee. 

* Another grantee used WEBSTA to track continuing education and work 
history, including each job held. The program director told us that 
such a follow-up system allows the program to spot trends, which in 
turn can be used to fine-tune the program. Of 60 graduates from 2002 to 
2005, the program continually has tracked 53 and reported that 8 are 
receiving higher education, 24 are working in nonconstruction jobs, and 
16 are working in construction jobs. The pay for the graduates ranged 
from $6.50 to $15 per hour. The grantee continues to keep in touch with 
the graduates by helping them with housing, legal aid, school advice, 
tutoring, and help toward achieving a GED. 

Grantees are not required to collect such post-program follow-up 
information. Yet, both grantees and program participants emphasized 
that the Youthbuild program created a very supportive atmosphere that 
many participants had never experienced prior to joining a program. 
During our visits, we met with students who credited the program with 
improving their lives by either helping get them off drugs, get out of 
the gang lifestyle, or disengage from other activities that likely 
could have resulted in incarceration or worse. The studies we mentioned 
previously had similar findings. However, it is difficult to gain 
insight into how many program participants remain drug free, crime 
free, in advanced degree programs, or working in construction or other 
trades without post-program performance data. It is also difficult to 
gain insight into which program strategies are most effective for 
serving these youth. 

HUD Does Not Review or Disseminate Impediments and Best Practices 
Identified in Closeout Reports That Could Help Improve Program 
Performance: 

Citing resource constraints, HUD did not systematically review the 
impediments and best practices identified in closeout reports or share 
the information in them with YouthBuild USA. In particular, identifying 
best practices (or the converse, impediments) can provide a model for 
improvement for other organizations with similar missions. The closeout 
reports contain a section addressing what the grantees considered to be 
best practices. For example: 

* Several grantees set-up educational opportunities, such as GED 
preparation, at local colleges for participants. Two grantees noted 
that being on a college campus provided a positive experience for the 
participants. Two grantees noted that they made an arrangement with a 
local technical college to offer free enrollment to all participants 
that successfully completed the Youthbuild program. Another grantee 
noted that working through community college GED programs allowed them 
to lower their per-student cost and serve more students. 

* One grantee established an alumni council consisting of elected 
representatives from each of its seven graduating classes. All 
graduates receive quarterly mailings that include an alumni newsletter, 
new job and educational opportunities, and a quarterly calendar of 
events. 

* Another grantee has maintained a program serving graduates for up to 
12 months to provide on-going employment support, including assistance 
in securing a second job in the event that the initial placement did 
not work out. 

* One grantee started a "drug free" club and found that students really 
wanted to participate and were anxious to become drug free and would 
publicly announce their success. 

* Another grantee worked extensively recruiting area college students 
to volunteer to help with tutoring and program community service 
activities. 

Closeout reports also contain information on impediments that grantees 
encountered in implementing and operating the programs. Impediments to 
program success varied in scope and severity. Some of the more 
significant impediments mentioned by numerous grantees included the 
following. 

* Turnover of key staff, especially on-site construction managers; 

* Negative and apathetic attitudes among participants; 

* Alcohol and drug abuse issues among participants; 

* Difficulties in dealing with municipal organizations to obtain 
building permits and contractual difficulties, which delayed 
construction training activities; and: 

* Acceptance of students with very low educational levels, which made 
obtaining high GED levels very difficult. 

HUD's contract with YouthBuild USA specifically calls for the 
contractor to assist HUD in the management, supervision, and 
coordination of Youthbuild programs by strengthening Youthbuild program 
design and disseminating information on best program practices. 
Although analyzing the impediment and best practices falls within the 
scope of HUD's contract with YouthBuild USA, two CPD program officials 
said that HUD never considered sharing the reports with its technical 
services provider. YouthBuild USA's chief operating officer said that 
the organization would welcome access to the closeout reports. The 
officer said his staff already spent many hours each month trying to 
identify best practices from Youthbuild grantees and would be anxious 
to analyze the closeout reports for both the problems and successes 
identified and forward the information to interested grantees. 
YouthBuild USA's President told us that she not only agreed that 
reviewing the closeout reports for problems and "best practices" would 
be beneficial, but that she believes the closeout reports needed to "be 
analyzed to get to the bottom line because the total impact (outcomes) 
of a grant could not be determined from data provided on a rolling 
quarterly basis."[Footnote 19] As a result of not sharing the closeout 
reports with YouthBuild USA, HUD may have missed opportunities to 
improve the program as a whole and help grantees improve performance. 

Labor Officials Indicated They Will Consider Using Information from 
Closeout Reports to Assess and Improve YouthBuild Performance: 

According to Labor officials, they will consider analyzing information 
from the closeout reports to assess and improve the performance of the 
YouthBuild program. Until 2006, Labor assessed past performance of its 
youth employment and training grantees through measures required under 
WIA. For example, under the WIA Youth and Job Corps programs, Labor 
tracked and analyzed 6-month retention and earnings wage change 
information. Labor officials told us that they would use OMB's common 
measures to track and analyze the performance of YouthBuild grantees 
but said that they have not finalized the information they would 
require YouthBuild grantees to collect. The officials indicated to us 
that they would consider developing post-program performance outcomes 
such as the types of employment graduates attained and retained 
(including construction), wage rates, and college or trade degrees or 
certifications received. Although Labor may incur some additional costs 
in developing such information, the ability to measure program success 
would improve. Labor officials also noted that the information 
currently contained in the closeout reports would be considered with 
other factors they deem as pertinent to assessing grantees' 
performance. However officials said they would also likely consider 
modifying the closeout reports to include other program participant 
data, such as degrees obtained prior to program entry and details on 
the types of construction jobs obtained upon program completion. 

Labor officials also acknowledged that information on program 
impediments and best practices should be evaluated and passed to 
grantees and that performance data needed to be aggregated, verified, 
and analyzed to determine which programs and strategies worked best. As 
of October 2006, Labor had not determined whether its Employment and 
Training Administration or an outside contractor would have primary 
responsibility for collecting, aggregating, and disseminating the 
information. 

Youthbuild Grantees Have Experienced Varied Success in Obtaining 
Outside Funding Sources, and Many Cited Continued HUD Funding as 
Crucial: 

Our analysis of available closeout reports showed that the grantees 
have had varying success in obtaining additional funding sources. In 
assessing grant applications, HUD gave consideration to program 
applicants that were able to secure outside funding. Applicants 
received points for leveraging nonhousing resources compared with the 
amount of Youthbuild funds requested in the application. Overall, the 
median amount spent by a HUD Youthbuild grantee was $400,000 in HUD 
funds; the HUD grant represented about 48 percent of the average total 
spending per grantee, which included additional funding sources of 
about $430,000. However, success in obtaining additional funds varied 
widely, from 21 grants reporting no additional funding sources to 40 
reporting more than $1 million (see fig. 7). 

Figure 7: Outside Funding Amounts Varied Widely for Grants We Reviewed: 

[See PDF for image] 

Source: GAO analysis of HUD Closeout Reports. 

[End of figure] 

According to information in the closeout reports and the officials from 
the grantees that we visited, the additional funding they obtained 
largely came from federal, state, and local agencies, businesses, and 
charitable organizations. According to a YouthBuild USA official, 
states have provided funding to local Youthbuild grantees over the past 
several years. For instance, the Massachusetts State Department of 
Education has provided funding in its budget for the state's Youthbuild 
program. The Massachusetts funds are provided to existing Youthbuild 
programs, based on a formula the state developed in conjunction with a 
state coalition of Youthbuild grantees. In Minnesota, 10 Youthbuild 
programs in 2005 shared $754,000 in state Youthbuild program funding, 
which the state supplemented with more than $3 million in matching 
funds from local partners. In Minnesota as elsewhere, urban grantees 
have had greater opportunities to obtain funding or in-kind support 
from local businesses and charitable organizations than rural grantees 
that typically have had access to fewer businesses and might not be a 
focus of charitable activity. 

Regardless of their success in obtaining outside funds, several 
grantees we visited stated that their Youthbuild program could not be 
sustained without the HUD grant because the HUD funding allowed them to 
attract the leveraged funds. Also, several grant officials said that 
many existing and potential contributors would not be willing to 
continue funding if they perceived that the program would not be able 
to continue without the HUD grant. One exception (which we did not 
visit) was in Minnesota where the state Youthbuild program director 
said that only 4 of the state's 10 programs have had HUD funds in the 
past, including 2 programs which received HUD funding in 2004 through a 
HUD grant administered by the state. The program director added that 
the structure of the Minnesota state Youthbuild program is similar to 
that of HUD's, although the programs are typically smaller, having from 
10 to 25 students. 

Information from YouthBuild USA generally agrees with grantees' 
contention about the need for continuous HUD funding. Overall, 
YouthBuild USA found that 60 percent of programs that HUD did not fund 
for 2 consecutive years ceased operations, and 90 percent of programs 
not funded for 3 consecutive years ceased operations. According to 
YouthBuild USA, 173 of 462 grantees funded since 1994 currently are 
operating with active HUD grants. YouthBuild USA, also found that 
grantees with continuous HUD funding achieved higher outputs, such as 
GED attainments and job placements, than the ones that did not receive 
one or more follow-on grants. According to some grantee officials, that 
success is partly due to continuous funding that allows grantees to 
develop infrastructure and maintain experienced staff to establish a 
better program. In its fiscal year 2005 Performance and Accountability 
Report, HUD noted the benefits of making grants to previous awardees: 
"Having established programs and experience running a Youthbuild 
program, these grantees are more efficient in enrolling students, 
resulting in a greater number of youth trained." Furthermore, 
Youthbuild program directors with whom we met emphasized the intangible 
benefit that such established programs provide such as a family 
atmosphere that allows graduates to return for advice and support as 
needed--a benefit that goes away when programs close. 

HUD has not kept statistics to find out why some grantees did not 
reapply for additional grants or why grantees did not qualify for 
follow-up grants. Through discussions with a nonprobability selection 
of 23 of the 100 grantees that closed programs, we found that some did 
not reapply because they were no longer interested in dealing with at- 
risk youth and others closed due to poor management. But most of the 
closed grantees with whom we spoke said they closed because they lost 
HUD funding for one or more years and were unable to obtain sufficient 
private funds to make up for the lost funding. Of the sites we 
contacted, two were still in operation after ceasing to receive HUD 
funds--but they appeared to be the exception. Both were already 
operating youth technical training programs prior to their first HUD 
Youthbuild grant. 

In addition, officials from several grantees that also received funding 
from AmeriCorps stated that they preferred the AmeriCorps concept of 
providing dedicated funds for 3 years. These officials stated that 
knowing that funding levels were longer-term provided more continuity 
to their program, enabled better planning, and allowed them to build a 
cushion through leveraged funding in the event that they did not 
receive a HUD grant for a year or two. Finally, when the question of 
multiyear funding was raised at a Web-based listening session in 
December 2006, designed to help Labor identify Youthbuild program 
administration issues and concerns, most participating grantees spoke 
out in favor of multiyear funding. 

Conclusions: 

HUD has not taken advantage of several opportunities to use existing 
information for purposes of program assessment and oversight. Such 
opportunities should not be ignored, particularly because HUD has 
limited resources with which to assess and oversee current grantees. 
Specifically, HUD has not verified, aggregated or analyzed closeout 
reports, which grantees are required to submit. HUD also has conducted 
limited oversight of grantees, citing resource constraints; the 
impending transfer of oversight to Labor bodes no change to this 
specific situation. However, given these conditions, analysis of 
existing closeout reports would help provide crucial information on the 
performance of the Youthbuild programs, particularly in the absence of 
a centralized HUD database that could be used to track program 
performance. At a minimum, the closeout reports form a logical basis 
for providing useful information that HUD, Labor, and other interested 
parties can use to judge the performance of Youthbuild. HUD still will 
maintain an active involvement for several more years, so coordination 
between Labor and HUD will be critical to producing reliable 
assessments of how successful the program has been and will be. HUD 
also does not share reported constraints and best practices with the 
technical services contractor it pays for performing such work and 
disseminating best practices. By not utilizing performance data and 
sharing information, HUD has missed opportunities to determine whether 
the overall Youthbuild program has achieved its intended results, among 
them enabling disadvantaged young adults to gain economic self- 
sufficiency. Therefore, Congress and the public also lack the 
information needed to make such determinations. 

Labor has experience in managing youth training and education programs 
and collecting performance information, and Congress transferred 
Youthbuild to Labor in part to better align the program with existing 
federal workforce development and youth training programs. In offering 
perspectives for Labor to consider, we stress the value that closeout 
reports have in the absence of more comprehensive and programwide data, 
although the existing reports lack certain elements that could help 
program managers determine and report on outcomes. The reports do 
contain valuable information on impediments and best practices, which 
could provide both program managers and grantees with much broader 
perspectives on how successful the program has been and suggestions for 
improvements. Labor has indicated that it would consider developing 
post-program performance outcomes such as the types of jobs graduates 
attained and retained (including construction), wage rates, and college 
or trade degrees or certifications received. As a result, while there 
may be some additional costs involved, the ability to measure program 
success would improve. And, because alternatives to the HUD data are 
limited, the grant-level assessment discussed above could act as the 
building blocks of an overall program assessment. Labor also has agreed 
that information on impediments and best practices would be valuable 
and ought to be passed on to grantees, but has not yet determined how 
this might be done. 

Finally, data on grantee success in attracting other funding, although 
limited, offer more perspectives for Labor to consider because outside 
funding helps leverage limited federal dollars. While many grantees 
have obtained extensive outside funding and support, YouthBuild USA has 
found that very few operated for more than a few years without 
continued HUD funding. Tellingly, their data show that 90 percent of 
programs not funded by HUD for 3 consecutive years ceased operations. 
According to YouthBuild USA officials, their data indicate that 
grantees with repeat awards also produced better outcomes such as a 
higher level of job placements. Further, when HUD funding stopped, 
potential contributors became concerned that grantees would no longer 
stay in business and were reluctant to jeopardize their own funds. 
These data, while not definitive, do suggest that multiyear funding has 
the potential to produce dividends for the program. Many grantees 
expressed similar belief in the benefits of longer term funding at a 
Department of Labor YouthBuild program listening session. 

Recommendations for Executive Action: 

To improve the reporting and assessment of performance for Youthbuild 
grantee programs and develop the bases for an overall assessment of the 
program, we are making the following three recommendations: 

* that the Secretary of HUD analyze closeout reports by grant and share 
information on identified problems and "best practices" with its 
technical services contractor and Labor; 

* that the Secretary of Labor develop and monitor post-program 
performance outcome measures for the YouthBuild program, such as the 
types of employment graduates attained and retained, wage rates, and 
degrees or certifications received, and share the data with the 
grantees; and: 

* that the Secretary of Labor consider whether multiyear funding could 
be useful in helping YouthBuild grantees attract additional outside 
funding. 

Agency Comments and Our Evaluation: 

We provided HUD and Labor with a draft of this report for review and 
comment. HUD's comments are summarized below. Labor's Assistant 
Secretary for Employment and Training provided written comments that 
are presented in appendix I. 

Officials from HUD's Office of Community and Planning and Development 
provided comments. They stated that they did not disagree with GAO's 
overall findings, but added that HUD's oversight responsibility was 
limited by resource constraints. They cited the fact that the 
Youthbuild program represents 0.63 percent of the Community Planning 
and Development Division's entire portfolio. Nevertheless, because HUD 
will have monitoring responsibilities for existing grants for 3 years 
or more, we believe it is important that HUD analyze the closeout 
reports in order to provide Labor with information about how current 
grantees are performing. 

Labor's Assistant Secretary for Employment and Training wrote that she 
believes that the report will be very useful to Labor as it assumes 
responsibility for administering the YouthBuild program. She noted that 
Labor agreed with our recommendation to develop and monitor post 
program performance outcome measures for the YouthBuild program and 
share these data with grantees. She added that Labor was building a Web-
based Management Information System to collect and report on 
participant information. Finally, she wrote that Labor agreed that 
there could be value on multiyear funding and that the Department was 
considering such an approach for existing grantees. 

We are sending copies of this report to other interested congressional 
committees and the Secretaries of Housing and Urban Development and 
Labor. We will also make copies available to others upon request. The 
report will also be available at no charge on GAO's Web site at 
http://www.gao.gov. 

If you or your staff have questions regarding this report, please 
contact me at (202) 512-4325 or shearw@gao.gov. Contact points for our 
Offices of Public and Congressional Affairs may be found on the last 
page of this report. GAO staff who made major contributions to this 
report are listed in appendix II. 

Signed by: 

William B. Shear: 
Director, Financial Markets and Community Investment: 

[End of section] 

Appendix I: Comments from the U.S. Department of Labor: 

U.S. Department of Labor: 
Assistant Secretary for Employment and Training: 
Washington, D,C. 20210: 

Feb 20 2007: 

Mr. William B. Shear: 
Director, Financial Markets and Community Investment: 
Government Accountability Office: 
441 G Street NW: 
Washington, D.C. 20548: 

Dear Mr. Shear: 

Thank you for the opportunity to comment on your Government 
Accountability Office (GAO) draft report, "YouthBuild Program: Analysis 
of Outcome Data Needed to Determine Long Term Benefits" GAO-07-82. This 
is a valuable report that will be very useful to the Department of 
Labor (DOL) as we assume responsibility for administering the 
YouthBuild program. I will focus my comments on the two recommendations 
addressed to the Department of Labor. 

The Department of Labor agrees with the GAO recommendation to develop 
and monitor post-program performance outcome measures for the 
YouthBuild program and share these data with grantees. The Department 
of Labor is building a web-based Management Information System (MIS) 
that will collect individual record data for all YouthBuild 
participants enrolled in the DOL-funded YouthBuild programs. At this 
time, we intend to require that YouthBuild grantees collect and report 
participant information on demographics, assessment information (such 
as educational attainment upon program enrollment), services, and 
outcomes. The grantees will also be required to provide one year of 
follow-up services and collect one year of post-program follow-up data. 

DOL intends to collect quarterly performance reports, generated by the 
web-based MIS, which will include the three youth common performance 
measures (placement in employment/ education, attainment of a degree/ 
certificate, and literacy/ numeracy gains) as well as a six month 
retention rate, a recidivism rate, and additional data, such as wage 
rates at placement and types of employment placements. DOL has the 
ability to aggregate individual participant data to create grantee and 
national level reports along with additional management reports for 
grantees and DOL to support performance and improvement. 

As for the second recommendation to consider multi-year funding of 
grants for the program, the Department of Labor is currently evaluating 
the most effective way to manage the grant funding cycle for 
YouthBuild. We agree that there may be value in developing a multi-year 
grant cycle for a variety of reasons. Therefore, we are giving 
consideration to a two tiered approach, which would combine both multi- 
year cycles to support continuity of funding and service delivery and a 
single-year funding cycle to promote opportunities for new grantees. 

If you would like additional information, please don't hesitate to call 
me at (202) 693-2700: 

Sincerely, 

Signed by: 

Emily Stover DeRocco: 

[End of section] 

Appendix II: GAO Contact and Staff Acknowledgments: 

GAO Contact: 

William B. Shear, (202) 512-4325, shearw@gao.gov: 

Staff Acknowledgments: 

In addition to the individual named above, Andy Finkel, Assistant 
Director; Rich LaMore, John McGrail, Marc Molino, Luann Moy, Barbara 
Roesmann, and Tom Taydus made key contributions to this report. 

FOOTNOTES 

[1] Youthbuild was originally authorized by provisions contained in the 
Housing and Community Development Act of 1992. The YouthBuild Transfer 
Act repealed the original authorizing legislation, effective September 
2006, when the Youthbuild program was transferred to the Department of 
Labor (Labor). The original authorizing legislation was in effect 
during the time period covered by our report. 

[2] HUD spells Youthbuild with a lower case "b," as written in the 
Housing and Community Development Act of 1992. As such, when referring 
to the HUD program, we spell Youthbuild with a "b." The YouthBuild 
Transfer Act used the upper case "B" spelling. To be consistent with 
the legislation, Labor plans to use the capitalized spelling when it 
takes over program administration. 

[3] Grantees include public or private nonprofit organizations, states 
or units of local government, or any entity eligible to provide 
education and employment training under other federal employment 
training programs. 

[4] Andrew Hahn, Thomas Leavitt, Erin McNamara Horvat, and James Earl 
Davis, "Life after YouthBuild, 900 YouthBuild Graduates Reflect on 
Their Lives, Dreams, and Experiences" (YouthBuild USA: June 2004); 
Maxine V. Mitchell, CRE et al., "Evaluation of the Youthbuild Program 
(U.S. Department of Housing and Urban Development, Office of Policy 
Development and Research: August 2003); and Minnesota Department of 
Economic Security, "Minnesota Youthbuild Program: A Measurement of 
Costs and Benefits to the State of Minnesota" (Commissioned by the 
Minnesota Youthbuild Coalition, Prepared by MDES staff: February 2003). 

[5] HUD identifies the reports as Final Performance Evaluation Reports. 
However, in the report we use the term "closeout reports." 

[6] Youthbuild locations we visited included San Francisco, California; 
Bloomington and Rockford, Illinois; Brockton, Lowell, and Springfield, 
Massachusetts; Philadelphia, Pennsylvania; Providence, Rhode Island; 
and Washington, D.C. 

[7] Government Performance and Results Act of 1993 and S. Rep. No. 58, 
103rd Cong., 1st Sess. (1993); Office of Management and Budget Circular 
(OMB) A-11, Part 6, "Preparation and Submission of Strategic Plans, 
Annual Performance Plans, and Annual Program Performance Reports" (June 
2006). 

[8] Appropriations for CPD programs and initiatives in fiscal year 2006 
were significantly higher than the $8 billion in prior fiscal years 
because of the approximately $11.5 billion Congress added for disaster 
assistance to five Gulf Coast states affected by hurricanes. 

[9] HUD completed the grant awards process in November 2006. 

[10] Very low-income families are those with incomes 50 percent or less 
of area median income. 

[11] The Corporation for National and Community Service, a public- 
private partnership, administers AmeriCorps. The program allows 
participants to earn education awards to help pay for postsecondary 
education in exchange for performing community service. 

[12] HUD program officials also complied with the fairness, 
transparency, and nondisclosure requirements of sections 102 and 103 of 
the HUD Reform Act of 1989. 

[13] GPRA; S. Rep. No. 58, 103rd Cong., 1st Sess. (1993); and OMB 
Circular A-11. 

[14] In addition to HUD and Labor, the Departments of Education, 
Health, and Human Services, Veterans Affairs, and Interior are subject 
to the common measures. 

[15] GAO issued these standards as required by 31 U.S.C. §3512(c). Also 
see GAO, Standards for Internal Control in the Federal Government, GAO/ 
AIMD-00-21.3.1 (Washington, D.C.: November 1999) and Internal Control 
Management and Evaluation Tool, GAO-01-1008G (Washington, D.C.: August 
2001). 

[16] The Job Training Partnership Act program was replaced by the 
Workforce Investment Act, which was implemented in 2000. This 
evaluation included data from 1996 through 1999 when the Job Training 
Partnership Act programs were still in effect. 

[17] Minnesota Youthbuild program participants entered unsubsidized 
employment with an average starting wage of $11.60 an hour. The minimum 
wage of $5.15 an hour was used as a baseline for the comparison group. 
Fourteen (5 percent) of the participants with one or more offenses 
prior to enrollment were rearrested, reconvicted, or returned to a 
state correctional facility within 2 years after enrollment in the 
Minnesota Youthbuild program, compared with a 20 percent rate of return 
for juveniles within 2 years of release from Minnesota correctional 
facilities. 

[18] Some grantees reported that they used their program to both 
construct new units as well as rehabilitated others under the same 
grant. 

[19] As we previously discussed, the quarterly data cannot be summed to 
provide an annualized report. 

GAO's Mission: 

The Government Accountability Office, the audit, evaluation and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the performance 
and accountability of the federal government for the American people. 
GAO examines the use of public funds; evaluates federal programs and 
policies; and provides analyses, recommendations, and other assistance 
to help Congress make informed oversight, policy, and funding 
decisions. GAO's commitment to good government is reflected in its core 
values of accountability, integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's Web site (www.gao.gov). Each weekday, GAO posts 
newly released reports, testimony, and correspondence on its Web site. 
To have GAO e-mail you a list of newly posted products every afternoon, 
go to www.gao.gov and select "Subscribe to Updates." 

Order by Mail or Phone: 

The first copy of each printed report is free. Additional copies are $2 
each. A check or money order should be made out to the Superintendent 
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. 
Orders should be sent to: 

U.S. Government Accountability Office 441 G Street NW, Room LM 
Washington, D.C. 20548: 

To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202) 
512-6061: 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov 
Automated answering system: (800) 424-5454 or (202) 512-7470: 

Congressional Relations: 

Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400 U.S. 
Government Accountability Office, 441 G Street NW, Room 7125 
Washington, D.C. 20548: 

Public Affairs: 

Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800 
U.S. Government Accountability Office, 441 G Street NW, Room 7149 
Washington, D.C. 20548: