Detailed Information on the
Youthbuild Assessment

Setting long-term and annual targets for the youth job training common performance measures in program year 2008, using historic performance data.

Improving coordination with other training and employment programs, including the One-Stop system and registered apprenticeship programs.

Implementing efficiency measures that are linked to performance outcomes, account for all costs, and facilitate comparisons across Department of Labor training and employment programs.

Assisting in the successful transfer of the YouthBuild program to the Department of Labor. The legislation that authorizes moving the program to the Department of Labor was signed into law in September 2006.

Adopting efficiency measures that are linked to performance outcomes, account for all costs, and facilitate comparisons across Department of Labor training and employment programs.

Measure: Placement in Employment or Education

Explanation:Percent of participants who are in employment or enrolled in post-secondary education or training This is a federal job training program common measure, which enables the YouthBuild program to describe in a similar manner the core purposes and results of the program compared to other education, employment and job training programs. For example, while the target population for the YouthBuild program is youth who are between the ages of 16-24, have been high school dropouts, and are members of at least one of the eligibility groups (youth offender, foster youth, low-income youth, youth who is an individual with a disability, child of an incarcerated parent or a migrant youth), the ultimate outcomes for this program are the same as for all other employment and training programs. Common measures remove a barrier to service integration among programs by ensuring that programs no longer have different definitions and methodologies for measuring performance. For the YouthBuild program, the performance indicator measure s how many participants received a placement according to the following formula: The number of youth participants who are in employment (including the military) or enrolled in post-secondary education and/or advanced training/occupational skills training in the quarter divided by the number of youth participants who exited the program in the previous quarter. This definition for placement in employment or education was adopted for YouthBuild in PY 2007 and baseline results will be available at the end of PY 2008.

Measure: Attainment of a Degree or Certificate

Explanation:Percent of participants who attain a diploma, GED or certificate This is a federal job training program common measure, which enables the YouthBuild program to describe in a similar manner the core purposes and results of the program compared to other education, employment and job training programs. For example, while the target population for the YouthBuild program is youth who are between the ages of 16-24, have been high school dropouts, and are members of at least one of the eligibility groups (youth offender, foster youth, low-income youth, youth who is an individual with a disability, child of an incarcerated parent or a migrant youth), the ultimate outcomes for this program are the same as for all other employment and training programs. Common measures remove a barrier to service integration among programs by ensuring that programs no longer have different definitions and methodologies for measuring performance. In this case, the performance indicator measures how many participants attain a degree or certificate, according to the following formula: Of those who enrolled in education (at the date of participation or at any point during the program) - The number of youth participants who attain a diploma, GED, or certificate by the end of the third quarter after the exit quarter divided by the number of youth participants who exited during the quarter. A certificate includes any employer-recognized occupational skills certificates that are developed or endorsed by employers and signify the attainment of technical or occupational skills. This definition for attainment of a degree or certificate was adopted for YouthBuild in PY 2007 and baseline results will be available at the end of PY 2008.

Measure: Literacy or Numeracy Gains.

Explanation:Percent of youth who increase their basic educational functioning level in literacy or numeracy This is a federal job training program common measure, which enables the YouthBuild program to describe in a similar manner the core purposes and results of the program compared to other education, employment and job training programs. For example, while the target population for the YouthBuild program is youth who are between the ages of 16-24, have been high school dropouts, and are members of at least one of the eligibility groups (youth offender, foster youth, low-income youth, youth who is an individual with a disability, child of an incarcerated parent or a migrant youth), the ultimate outcomes for this program are the same as for all other employment and training programs. Common measures remove a barrier to service integration among programs by ensuring that programs no longer have different definitions and methodologies for measuring performance. In this case, the performance indicator measures the increase in literacy and numeracy skills of participants through a common assessment tool administered at program registration and regular intervals thereafter. The result is measured according to the following formula: Of those YouthBuild participants who are basic skills deficient at enrollment - The number of youth participants who increase one or more educational functioning levels (roughly equivalent to two grade levels) divided by the sum of the number of youth participants who have completed a year in the program (i.e. one year from the date of enrollment) and the number of youth participants who exited before completing a year in the YouthBuild program. The common measure definition for literacy and numeracy gains was adopted for YouthBuild in PY 2007, but requires two years to obtain valid results. Baseline results will be available at the end of PY 2009.

Measure: Cost per Participant

Explanation:The program calculates the average cost for each participant by dividing the total annual appropriation for the program by the number of participants.

Is the program purpose clear?

Explanation: The program purpose is clearly described in the statute. The purposes of the Youthbuild program are: 1) to expand the supply of permanent affordable housing for homeless individuals and members of low- and very low-income families by utilizing the energies of economically disadvantaged young adults; 2) to provide economically disadvantaged young adults with opportunities for meaningful work and service to their communities in helping to meet the housing needs of homeless individuals and members of low- and very low-income families; 3) to enable economically disadvantaged young adults to obtain the education and employment skills necessary to achieve self-sufficiency; and, 4) to foster the development of leadership skills and commitment to community development among young adults in low-income communities.

Evidence: The program purpose for the Youthbuild program is detailed in: Subtitle D of title IV of the National Affordable Housing Act (42 U.S.C. 8011), as amended by section 164 of the Housing and Community Development Act of 1992 (Public Law 102-550).

Does the program address a specific and existing problem, interest, or need?

Explanation: Youthbuild is a 13 year-old program that focuses on at-risk youth and attempts to develop their work and life skills through education and training in housing construction activity. Youth in distressed communities are at risk of entering adulthood without the necessary life and work skills. In the annual competition for Youthbuild grants, HUD's Notice of Funding Availability (NOFA) requires applicants to respond to that need by demonstrating levels of distress in the target area for their prospective program, as defined by levels of poverty, unemployment, and high school dropouts.

Evidence: A significant number of youth continue to be at-risk and need general academic and trade skills. Based on data from Youthbuild USA (2000-2004), 32% of youth drop out of high school without a diploma, and in inner city neighborhoods, the figure rises to 50%. The program focuses on the poorest urban and rural communities that cover most states. Of the youth targeted, 13% have felony convictions, 28% use public assistance, 17% live in public housing, 28% are parents, 88% do not have GEDs, and the average reading level is 7.2 grade. Youthbuild is a program that expands economic opportunities for at-risk youth to avoid poverty through education, training, leadership development, and job placement and/or entrance to higher education.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: The Youthbuild program design is similar to other employment programs. There are other Federal, State, local and private efforts whose objectives are to employ at-risk youth through job training and education program. Although many do not focus only on low-income housing construction and rehabilitation, as does Youthbuild, they provide youth the opportunity to choose the construction industry based on a more flexible and individually tailored program design.

Evidence: In regards to Federal efforts, the U.S. Department of Labor administers at-risk youth employment programs. Job Corps provides job training, GED, and other supportive services. Many of its participants are in Job Corps' residential component, but there is also a non-residential component. In addition to Job Corps, DOL also operates other Workforce Investment Act programs that target at-risk youth and provide them with the opportunity to receive training in a wide range of demand-driven industries, including housing construction and rehabilitation. State employment programs complement these programs with similar services. There are also local private non-profits that also have similar program designs. Youthbuild was based on the example of a privately-funded project originating in New York in which youth who had dropped out of school participated in a program of personal and community development that taught leadership and public-speaking skills, helped the youth to earn their General Educational Development (GED) certificate, and provided both formal and informal counseling. While in this program, participants also developed job skills through projects to renovate abandoned buildings in their neighborhoods. A demonstration program in the early 1990s encouraged the implementation of a national program. Since then, a national network of Youthbuild programs has evolved, incorporating the same elements as the New York model. HUD administers this program and provides funding, through an annual competitive grant application process, to local organizations that implement Youthbuild programs.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The program is designed to assist at-risk youth obtain education and employment skills in the housing construction trades. In FY 2005, the program exceeded its goals for the number of youths trained and for the numbers of housing units constructed and rehabilitated. An evaluation of Youthbuild's efficiency in cost reflects a high cost to participant ratio compared to other non-residential youth employment programs, but the costs were on par with the Supported Work Demonstration, another program reviewed that offers subsidized work for participants (AREA, 2003). Currently, the GAO is conducting an assessment of the Youthbuild program to determine the efficacy of the program.

Evidence: The Youthbuild program is a competitive grant program, so this competitive process inherently injects some market efficiencies among applicants. Also, the program leverages non-housing resources from other Federal, state, local, and private resources. These include social services, such as counseling and training, volunteer labor, and other teaching materials and personnel. The Notice of Funding Application process in the annual competition is used to assess the projects proposed for funding to determine the actual award amounts to Youthbuild grantees (2006 Youthbuild NOFA). Also, program performance data are available in HUD's 2005 Performance and Accountability Report on the program website: http://www.hud.gov/offices/cpd/economicdevelopment/programs/youthbuild/index.cfm.

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: The program design ensures that resources are used directly and effectively to reach the target population. To receive assistance, program participants must be between the ages of 16 and 24, very low-income individuals or members of very low-income households, and who can demonstrate educational need, such as high school dropouts.

Evidence: The Youthbuild regulations (see CFR Part 24, Section 585.305) and NOFA target resources to the Youthbuild program's purpose and the beneficiaries.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: Youthbuild has not developed specific long-term performance measures.

Evidence: The program does not have and currently, does not plan to develop any long-term measures, in part due to the Administration's proposal to transfer program from HUD to DOL. See President's Budget Request FY 2006 and FY 2007; proposed legislation to transfer Youthbuild transmitted in 2005.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: Please see response to 2.1

Evidence: Please see response to 2.1

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: HUD's annual performance measures gauge progress in providing training and jobs for Youthbuild program participants. These measures, developed with OMB as common performance measures for youth programs in 2003, allow individual grantees to evaluate their progress on each of the measures through semi-annual progress reports and annual applications for funding.

Evidence: Beginning with the 2003 competition, HUD implemented the four OMB common performance measures for youth programs, and these are shown in Rating Factor 5 of the 2006 Youthbuild NOFA, and will be published in the HUD 2007 Annual Performance. Please see Performance Measures section. Although these measures are available and useful, HUD has not developed long-term outcome measures related to these annual goals (see questions 2.2 and 2.3).

Does the program have baselines and ambitious targets for its annual measures?

Explanation: After adopting the OMB common measures for youth programs in 2003, HUD began to collect baseline data. Youthbuild already had baseline data on its annual output measures. However, targets related to these annual measures were either not developed and/or the targets were not ambitious, as some targets were below actual performance.

Evidence: See performance measurement section; 2007 APP Plan (soon to be published, with an appendix that amends the 2006 plan); 2006 Youthbuild NOFA.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: Goals are available to grantees and they contribute to the annual goals of the program. When applicants compete in the annual NOFA process, applicants must describe how they will measure and report on their performance in their programs. As a part of the grant agreement, grantees agree to commit to minimum annual goals. In the 2006 NOFA, the logic model was introduced so that grantees can include the minimum number of goals required by the program, and adopt additional outcome measures.

Evidence: Grantees report on progress toward achieving program goals through semi-annual reports over the 30-month grant period. HUD monitors these grants through review of grantees' reporting, site visits, and compliance with grant agreements.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: An independent evaluation of Youthbuild was completed in 2003. The evaluation was contracted with the Applied Real Estate Analysis, Inc. (AREA), an independent and respected social policy research firm. The research methodology focused on a review of the literature on the Youthbuild program, comparison with other youth programs, examination of HUD records, site visits to 20 Youthbuild grantees, cost-effectiveness, and program efficiencies. However, regular program evaluations have not been scheduled due to the Administration's proposal to transfer the program to Department of Labor.

Evidence: "Evaluation of the Youthbuild Program" by the Applied Real Estate Analysis, Inc. (AREA), August 2003. After a full and open competition, and in accordance with the HUD Reform Act, AREA was contracted to be the independent contractor to evaluate Youthbuild, through HUD's Office of Policy Development and Research.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: HUD did not include a Youthbuild program budget request in FY 2006 or FY 2007. The Administration has transmitted draft legislation to transfer the Youthbuild program from HUD to DOL, based on the recommendation of the White House Task Force on Disadvantaged Youth.

Evidence: See President's Budget Request FY 2006 and FY 2007; proposed draft legislation to transfer Youthbuild transmitted in July 2005.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: HUD has taken steps to identify some deficiencies in its programs, including developing performance measures that focus more on outcomes directly related to achieving the program's purpose, rather than only on outputs and processes. However, long-term outcome measures and targets have not been developed, in part due to the Administration's proposal to transfer the program from HUD to DOL. More importantly, it is unclear how the program offices systematically identify and address strategic planning deficiencies and a time frame for when they may be corrected.

Evidence: HUD's Strategic Plan includes the development of annual outcome-based performance measures and beginning in 2003, HUD changed its performance measures to focus more directly on achieving four key outcomes (see 2.3 and Performance Measures).

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: While the program requires grantees to collect data on a regular basis, and HUD field offices work closely with Youthbuild grantees on program delivery, HUD has inconsistent data and does not use the information and performance measures to adjust program priorities, allocate resources, or take other management action.

Evidence: Each grantee is required to submit to HUD and Youthbuild USA, HUD's technical assistance contractor for Youthbuild, a semi-annual progress report and a final closeout performance report. Grantees report on performance indicators, but as seen in the Performance Measures section, data is inconsistent (e.g., efficiency measure adopted, but data unavailable), and the program has not used previous year data to adjust targets for subsequent years. The latest annual summary is available in the Youthbuild USA Data Report for 2004-2005 (September 30, 2006); Grantee semi-annual progress reports and final closeout performance reports are also available.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Grantees are held accountable for cost, schedule, and performance through the semi-annual reporting process and obligation timeliness standards established in the statute, and subsequent departmental office notices. Beginning in 2002, the statute required that all Youthbuild program funds be obligated three years following appropriation. Each Youthbuild grantee develops a work plan as part of its competitive application that HUD field staff use to monitor performance. Field staff use the Line of Credit Control System (LOCCS), the monthly drawdown reports (A-67), and the semi-annual progress reports to develop the field Risk Analyses that are used to identify grantees that may have cost, schedule, or performance issues. While grantees are held accountable, it is unclear how HUD program managers are held to comparable factors.

Evidence: Grant agreements detail provisions for award execution, including project timelines and performance objectives, and for carrying out work plans outlined in the grantee's competitive application. In order to ensure accountability, field offices perform project risk analysis based on common risk factors, and provide ongoing monitoring and compliance reviews of program costs and performance (Chapter 18 narrative and exhibits of HUD Handbook 6509.2, Community Planning and Development Monitoring Handbook). HUD also uses prior performance in these areas by rating grantees when they apply for additional funding (see Rating Factor 5, Achieving Results and Program Evaluation, in the Youthbuild NOFA). However, HUD program manager annual performance plans have not incorporated standards that hold staff accountable for Youthbuild cost, schedule, and performance results.

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: HUD statute requires that funds be obligated within three years or funds will be returned to the Treasury. This statute creates a maximum timeframe, which ensures a level of accountability. Since the majority of the grant award funding is for items such as, teaching materials, technical assistance, stipends, they are normally expended at a relatively fast rate. In regards to eligible activities, HUD assesses each project budget to ensure all expenses are eligible. Expenditures are then monitored by field office staff through the LOCCS tracking system to ensure that the funds are being spent timely and on intended purposes.

Evidence: With an exception of a technical anomaly in 1994, the program has efficiently obligated the funds. The technical anomaly was due to appropriations language that changed the Youthbuild account and delayed the grant competition cycle. HUD's CPD budget office provides monthly reports that show the financial status of all active Youthbuild projects. The report shows the obligation date, cumulative reserved and obligated amounts, unobligated balance, cumulative disbursed amount and the unexpended amount for each grant.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: While some mechanisms are in place for operating a cost effective program, the program does not have data to help it improve program performance and efficiency. Youthbuild is a competitive grant, which encourages a certain level of efficiency amongst Youthbuild providers. HUD has also adopted OMB's common performance measures for youth programs, including the efficiency measure of annual cost per participant. However, Youthbuild cannot provide the necessary efficiency measure data and currently does not have targets for efficiency.

Evidence: Please see Performance Measurement section. The measure is adopted, but corresponding targets and actuals are unavailable.

Does the program collaborate and coordinate effectively with related programs?

Explanation: Through the NOFA process and grant agreement, grantees are required to participate in Local Workforce Investment Act One-Stop Centers authorized by the Workforce Investment Act of 1998. Grantees also coordinate their program activities with those of other agencies in their service areas. Applicants are also encouraged to enter into partnerships with other Federal, state, local, and private entities and to use other government-sponsored, private, or nonprofit housing programs. On the Federal level, HUD was an active participant in a 2003 White House Task Force for Disadvantaged Youth that looked at improving coordination among federal agencies that administer youth work programs.

Evidence: While the grantees are required to collaborate with other partners, HUD does not have examples of how collaboration has lead to meaningful actions in management and improved resource allocation.

Does the program use strong financial management practices?

Explanation: Once funds have been awarded, grantees must adhere to timeliness standards outlined in annual NOFAs. Youthbuild grantee funds are managed through HUD's Line of Credit Control System/Voice Response System (LOCCS/VRS), the same system used for the financial management of other HUD formula and competitive programs. Each grantee uses the Line of Credit Control System (LOCCS) voice response financial management system to drawdown funds and track expenditures. The LOCCS system automatically notifies field office staff of expenses requiring further clarification. The use of this system ensures that payments are made properly and erroneous payments minimized. LOCCS meets all statutory requirements for such systems and provides financial information that is accurate and up-to-date. LOCCS has no identified material internal control weaknesses. The grantee is required to maintain receipts or other specified documentation to support compliance with the statutory and regulatory requirements for eligible activities.

Evidence: Field staff annually conducts both remote and on-site monitoring evaluations of competitive Youthbuild grant recipients. On-going closeout procedures are used to recapture unspent funds and these funds are then re-used in the current competition (See Chapter 18 narrative and exhibits of HUD Handbook 6509.2, Community Planning and Development Monitoring).

Has the program taken meaningful steps to address its management deficiencies?

Explanation: Youthbuild addresses management deficiencies when they are identified. For example, HUD's Office of Inspector General (OIG) conducted an audit of the Youthbuild program in 1998 and found weaknesses in the ranking and review of Youthbuild applications and in the monitoring of Youthbuild recipients. HUD addressed the application deficiencies by incorporating previous performance evaluation reports into the review process and through a quality review of the scoring. To improve the oversight of Youthbuild grantees, HUD transferred the monitoring responsibilities to Field staff in 1999, and developed a monitoring guidebook for staff. Monitoring procedures were recently updated in the HUD Handbook 6509.2, Chapter 18, Youthbuild.

Evidence: HUD OIG Audit Report, 99-PH-156-0001, dated November 10, 1998; and memorandum to program office closing out audit findings. See also Youthbuild Program Field Monitoring Guidance, and HUD Handbook 6509.2, Chapter 18, Youthbuild.

Are grants awarded based on a clear competitive process that includes a qualified assessment of merit?

Explanation: All of Youthbuild funds are awarded based on merit, administered with transparency, and is conducted consistent with stringent HUD Reform Act requirements. The annual NOFA specifies the requirements for competitive submissions and describes the rating factors for assessing the quality of a Youthbuild program. Projects are reviewed for merit based on need, quality, capacity, past performance, and financial management. Satellite web broadcasts give applicants the opportunity to understand how applications are scored. These web casts are also posted on HUD's website for viewing by the public. While HUD encourages new Youthbuild applicants, the program office has not increased outreach due to the lack of budget request and the proposal to transfer the program from HUD to DOL.

Evidence: See the HUD web site for archived SuperNOFA web casts for 2001-2006. See also the 2006 Youthbuild NOFA. Information forthcoming about the results of the competition, how many applicants applies, grants awarded, and any relevant trends.

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: There are multiple oversight processes to ensure compliance. Eligible activities are identified during the Youthbuild review process. Both field office and Headquarters staff review each grantee's proposed budget. Field staff review a more detailed budget in each grantee's technical assistance submission and then track expenses through the Line of Credit Control System (LOCCS). LOCCS allows Field office staff to track grantee spending within each activity category and the program's semi-annual progress reports Youthbuild grantees program progress and performance. Field office staff conduct annual risk assessments of all grantees, but prioritize and dedicate resources for on-site monitoring of higher-risk grantees. As a result, resources for remote and on-site monitoring are directed at the highest risk projects each year. Federal staff and/or designated contractors provide technical assistance to grantees when deficiencies are identified in any aspect of fund administration, reporting, performance, or financial operations.

Evidence: The practices are described in a number of sources, including Youthbuild NOFA; LOCCS data; and HUD Handbook 6509.2, Chapter 18, Youthbuild.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: The program collects performance data semi-annually, but it is not available in a timely or usable fashion for the public. Decisions to develop a system to publicize Youthbuild performance data are on hold pending the proposal to transfer the program to DOL.

Evidence: Although there is no performance data that is friendly and widely available to the public, HUD does develop annual performance targets for the Youthbuild program in its Annual Performance Plan (2007), and reports the results in its annual Performance and Accountability Report (last published 2005).

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: Youthbuild has not developed specific long-term performance measures.

Evidence: See Questions 2.1 and 2.2

Does the program (including program partners) achieve its annual performance goals?

Explanation: Youthbuild has not developed ambitious annual performance goals and has not achieved many of its targets. Although some of its actual outputs (e.g., participants) exceed its targets, most of the targets are not very ambitious (e.g., GED completion, housing construction/rehabilitation) and the methodology for tracking inconsistent. For many outcome measures and the program's efficiency measure, there are no available targets to measure against.

Evidence: Please see Performance Measures section. Examples of inconsistent tracking methodology: 1) Literacy and numeracy skills gains were double counted in 2005; 2) Program did not establish housing units constructed or rehabilitated targets, the projections were provided by the grantees and had subsequent changes. (2005 HUD Performance and Accountability Report)

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: Although Youthbuild has an adopted efficiency measure, program efficiencies and cost effectiveness of program cannot be determined at this time.

Evidence: The data is unavailable to answer this question.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: The 2003 evaluation of Youthbuild showed mixed results, at best. The research study found that Youthbuild cost slightly more per program participant than four other youth job programs: Job Corps, JOBSTART, Supported Work Demonstration, and the Job Training Partnership Act. When the benefits derived from Youthbuild's added dimension of rehabilitating and producing housing is factored in, it may be less expensive than the other programs. The study found that the HUD Youthbuild program achieves its goal of producing affordable housing that benefits the low- and very low-income households for which it is intended. However, the number of units produced is small??usually one to two units each year per program grantee. While some Youthbuild programs report a larger number of completed housing units, their participants have usually contributed a small portion of the labor required to produce the dwellings. Youthbuild also has a leadership component that may be beneficial for its participants, but it is not quantifiable. The graduation/entrance in higher education rate was reported as comparable to the control group. Therefore, while there are some benefits for the participants, the program shows mixed and somewhat minimal positive results.

Evidence: See the independent study, "Evaluation of the Youthbuild Program" (AREA, 2003).

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: As indicated in response to question 2.6 and 4.4, the independent evaluation encompassed a review of five youth work programs, including Youthbuild. Based on a review of the outcomes and costs of the four federally-funded employment training programs with which the HUD Youthbuild program was compared, the study found that it is a very difficult task to exert positive impacts on the employment, earnings, and educational achievement of disadvantaged youth. The results are mixed and may indicate minimal positive results.

Evidence: For a nonresidential program, Youthbuild is relatively expensive (in present-value dollar terms) when compared with the other four programs. And based on a very rough comparison, the GED results and employment rates of Youthbuild participants seem to be, on average, closer to those of the control group (those not served) in the Job Corps study than those who went through the Job Corps program. Unlike the comparison programs, Youthbuild has not been rigorously evaluated, so the researchers could not make definitive claims about its impact and effectiveness. Even if HUD were to invest in a more rigorous assessment of Youthbuild's cost and effectiveness, such an evaluation would be complicated by the fact that, relative to the comparison programs, Youthbuild providers have different program delivery mechanisms and it would a very costly study relative to the size of the program.