Detailed Information on the
Workforce Investment Act - Youth Activities Assessment

Proposing giving the Secretary of Labor and States increased authority to reallocate resources to areas of need.

Conducting an evaluation to determine WIA services' impact on employment and earnings outcomes for participants.

Consolidating the program with three other State grant programs, to increase the number of workers trained, improve services, and eliminate unnecessary overhead.

Strengthening the quality of youth performance data, specifically supplemental data and administrative records, through data validation.

Implementing efficiency measures that are linked to performance outcomes, account for all costs, and facilitate comparisons across Department of Labor training and employment programs.

Strengthening accountability for employment outcomes and skill attainment by adopting common performance measures and targets to allow for comparisons with other federal job training programs.

Adopting efficiency measures that are linked to performance outcomes, account for all costs, and facilitate comparisons across Department of Labor training and employment programs.

Measure: Placement

Explanation:Percent of youth who enter employment or enroll in post-secondary education and/or advanced training/occupational skills training or the military in the first quarter after exit. 2002 data are for previous, similar placement measure for youth aged 14-18.

Measure: Credential Attainment

Explanation:Percent of students who attain a GED, high school diploma, or certificate by the end of the third quarter after exit. New measure implemented in 2005. Data prior to 2005 are for a previous, similar measure.

Measure: Literacy or Numeracy Gains

Explanation:Percent of basic-skills-deficient students who achieve literacy or numeracy gains of one or more Adult Basic Education level. New in PY 2006.

Measure: Cost per Participant

Explanation:Total annual appropriation for the program divided by the number of participants.

Is the program purpose clear?

Explanation: The Workforce Investment Act (WIA) Youth program supports formula-funded grants to States to provide education and training to low-income youth aged 14-21, and who also face barriers to employment. Eligible youth are deficient in basic skills or are homeless, are a runaway, are pregnant or parenting, or are offenders, school dropouts, or foster children. The program provides a variety of services for improving educational and employment outcomes for youth, including occupational skills training, educational and alternative school services, leadership development and community service opportunities, and supportive services. The program serves both in- and out-of-school youth, which includes both high school dropouts and high school graduates who are neither employed nor in post-secondary education. Through its strategic youth vision, DOL has issued specific guidance to States, stating that the neediest youth population, particularly out-of-school youth and high school dropouts, should be served by the WIA Youth program.

Evidence: Section 129 of the Workforce Investment Act (29 U.S.C. 2854 - Public Law 105-220); United States (U.S.) Department of Labor Strategic Plan, Fiscal Years 2006-2011, available at http://www.dol.gov/_sec/stratplan/main.htm. Training and Employment Guidance Letter (TEGL) 3-04. The Employment and Training Administration's (ETA's) New Strategic Vision for the Delivery of Youth Services Under the Workforce Investment Act (WIA). Available at http://wdr.doleta.gov/directives/corr_doc.cfm?DOCN=1589.

Does the program address a specific and existing problem, interest, or need?

Explanation: According to the U.S. Census Bureau, there were 37 million youth aged 16-24 in the United States in 2006. Of those, 3.3 million were not enrolled in school and did not have a high school diploma, and 6.3 million lived below the poverty line. Between October 2005 and October 2006, 444,000 young people dropped out of high school. The labor force participation rate for these youth (51.4 percent) was much lower than the labor force participation rate of recent high school graduates who had not enrolled in college (76.4 percent) (CPS 2007). The WIA Youth program is aimed at improving the educational and employment outcomes of these youth.

Evidence: White House Task Force Report on Disadvantaged Youth, 2003. Available at http://www.acf.hhs.gov/programs/fysb/content/docs/white_house_task_force.pdf. University of Chicago, Chapin Hall Center for Children. 2002. Employment Outcomes for Youth Aging Out of Foster Care. Available at http://aspe.hhs.gov/hsp/fostercare-agingout02/. Bureau of Labor Statistics, Current Population Survey, Table 2, Labor force status of persons 16 to 24 years old by school enrollment, educational attainment, sex, race, and Hispanic or Latino ethnicity, October 2007 - http://www.bls.gov/news.release/hsgec.t02.htm. U.S. Bureau of the Census, Current Population Survey, 2007. Data available at http://www.census.gov/cps/. Harrington, Paul. The Cost of Dropping Out: The Effects of the High School Drop Out Rate on America's Competitiveness. Center for Labor Market Studies, Northeastern University. Presented at Education Day on October 17, 2007. Available at www.workforce3one.org.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: The WIA Youth program provides education and training to at-risk youth, ages 14-21, and places special emphasis on serving out-of-school youth, and those youth that have become disconnected from education and employment. There are a number of other Federal programs that provide similar education and training services to disadvantaged youth including Job Corps, YouthBuild, the National Guard's YouthChalleNGe, Americorps*NCCC (National Civilian Conservation Corps), and ED's Adult Education State Grants. The DOL administered Job Corps program provides job training, GED, and other supportive services to disconnected youth. Many of its participants are in Job Corps' residential component, but there is also a non-residential component. DOL's YouthBuild program also targets out-of-school youth, and provides them with the opportunity to receive skills training in construction and housing rehabilitation. Training in the construction field is also available to WIA Youth participants. YouthChalleNGe, and NCCC, while both residential programs also include a community service focus, and provide supportive services to at-risk youth. The Department of Education's Adult Basic Education and State Literacy Grants provide workplace literacy and education programs to a population that overlaps with the WIA Youth program The 2003 White House Task Force for Disadvantaged Youth cited a lack of communication, coordination, and collaboration among federal agencies providing services to the neediest youth. DOL has taken several steps to increase collaboration between the WIA Youth program and related programs. In response to the 2003 White House Task Force Report on Disadvantaged Youth, DOL jointly established the Shared Youth Vision Federal Partnership. The Federal Partnership, addressed in more detail in Section 3.5, formed an interagency workgroup to improve the collaboration and communication among Federal, state and local agencies who serve the neediest youth population.

Evidence: U.S. Department of Labor, Bureau of Labor Statistics, Employment and Earnings February 2008, Table A-16, Employment Status of the Civilian Non-institutional Population 16 to 24 Years of Age by School Enrollment, Educational Attainment, Sex, Race, and Hispanic or Latino Ethnicity - http://www.bls.gov/opub/ee/emp/empearn200802.pdf. U.S. Census Bureau, Census 2000 Reports, Areas with Concentrated Poverty: 1999; July 2005, Table 3, Distribution of People, Families, Households by Poverty Level of Census Tracts and Other Selected Characteristics, 1999 - http://www.census.gov/prod/2005pubs/censr-16.pdf. Table provides overall number of persons in poor census tracts and the percentage of children 18 and under in poor census tracts. Estimate is based on there being 14.64 million children under 18 in poor census tracts, and that youth ages 14 to 17 represent roughly 22 percent of these. U.S. Census Bureau, Current Population Survey, 2007 Annual Social and Economic Supplement - http://pubdb3.census.gov/macro/032007/pov/new01_100_01.htm. U.S. Department of Labor, PY 2006 WIASRD Data Book (Social Policy Research Associates) - http://www.doleta.gov/Performance/results/PY2006_WIASRD_Data_Book.pdf. U.S. Department of Labor, Office of the Job Corps, PY 2005 Annual Report - http://www.dol.gov/_sec/media/reports/annual2005/main.htm. YouthBuild USA Web site - www.youthbuild.org/. Jobs for America's Graduates Web site - www.jag.org/. The Corps Network Web site - www.corpsnetwork.org/. AmeriCorps Web site - www.americorps.org/.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The existence of separate funding sources for the Employment Service and the WIA Adult, Dislocated Worker, and Youth programs impedes States' and localities' efficiency, effectiveness, and flexibility to build and design a system or programs with innovative approaches to prepare participants for employment. In addition, current regulations for the program include an inadequate and inaccurate definition of "administrative costs" that results in local offices' diverting significant amounts of resources from providing services to paying administrative overhead. The Administration's WIA reauthorization and reform proposal is designed to address these flaws by streamlining and consolidating multiple funding sources to provide Governors with the flexibility needed to better address workers' and employers' needs, while tightening the definition of "administrative costs" to ensure fewer funds are being spent on overhead. In its reauthorization proposal, the Administration proposes Career Advancement Accounts that will, in part, target funds to out-of-school youth. To further this effort in lieu of reauthorization, DOL has pursued strategies to encourage the more effective targeting of resources by states. In PY 2004, DOL issued guidance to WIA youth service providers, stating that WIA-funded youth programs should serve the neediest youth by providing them with quality educational opportunities and high-growth employment opportunities. Additionally, the program continues to collaborate and coordinate with other youth-serving Federal programs through the Shared Youth Vision Federal Partnership (see Section 3.5).

Evidence: Training and Employment Guidance Letter (TEGL) 3-04. The Employment and Training Administration's (ETA's) New Strategic Vision for the Delivery of Youth Services Under the Workforce Investment Act (WIA) - Available at http://wdr.doleta.gov/directives/corr_doc.cfm?DOCN=1589.

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: WIA is hampered by funding allocation issues, including statutory funding formulas that are ineffective. Formula factors used to allocate funds are not aligned with the target population and do not adequately target the youth most in need of program services. As a result, states' funding levels may not always be consistent with the actual demand for services. Moreover, the population served by this program is a very small portion of those eligible to receive services. There are 37 million youth age 16-24, of whom 3.3 million are high school dropouts and 5.9 million have completed high school but have no college education. The WIA Youth program serves fewer than 200,000 youth. A recent GAO study found that States and WIBs could benefit from guidance to develop and implement contracts that allow local programs to serve the neediest youth??while still achieving performance goals. DOL is developing the guidance which will include specific examples of implementing contracts with local service providers that will still allow them to successfully serve youth at varying skill levels. To prepare for issuing this guidance, the Employment and Training Administration (ETA) will convene WIA state board staff, local board staff, and youth service providers in the summer and fall of 2008. The convened meetings will help ETA gain an understanding of contracting issues and provide examples of local contracts with the components and flexibility to successfully serve the youth most in need.

Evidence: Bureau of Labor Statistics, Current Population Survey, Table 2, Labor force status of persons 16 to 24 years old by school enrollment, educational attainment, sex, race, and Hispanic or Latino ethnicity, October 2007 - http://www.bls.gov/news.release/hsgec.t02.htm. Government Accountability Office, "Disconnected Youth: Federal Action Could Address Some of the Challenges Faced by Local Programs that Reconnect Youth to Education and Employment," GAO-08-313, February 28, 2008 - http://www.gao.gov/cgi-bin/getrpt?GAO-08-313. Government Accountability Office, "Workforce Investment Act: Issues Related to Allocation Formulas for Youth, Adults, and Dislocated Workers," GAO-03-636, April 2003 - http://www.gao.gov/new.items/d03636.pdf.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The WIA Youth program uses the Job Training Common Measures for youth: (1) percentage of youth placed in employment, post-secondary education, advanced training/occupational skills training, or the military in the first quarter after their exit quarter; (2) percentage of youth, who attain a degree or certificate by the end of the third quarter after their exit quarter; and (3) the percentage of basic skills deficient youth who increase one or more education functioning levels within one year after their date of first program service. These measures support the program's purpose because they gauge the program's success in assisting youth achieve core education and employment outcomes.

Evidence: Section 136 of the Workforce Investment Act (29 U.S.C. 2871) (P.L. 105-220) - http://www.doleta.gov/seniors/other_docs/105-220.pdf. Training and Employment Guidance Letter (TEGL) 17-05 (February 2006). Common Measures Policy for the Employment and Training Administration's (ETA) Performance Accountability System and Related Performance Issues. Available at http://wdr.doleta.gov/directives/corr_doc.cfm?DOCN=2195. DOL's Strategic Plan, Fiscal Years 2006-2011. Strategic plans are available at http://www.dol.gov/sec/stratplan/main.htm.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: .The WIA Youth program has established ambitious targets and timeframes for three of its four long-term measures. Results for the literacy/numeracy measure are not reported until a youth participant has completed a full year in the program. DOL will have baseline data on the literacy/numeracy measure in November 2008. Once the baseline data is available, the program will set ambitious annual targets for literacy/numeracy gains. Program Year (PY) 2012 targets are: 66 percent for placement in employment or education, an increase of six percent when compared to the PY 2006 outcome of 60 percent; and 50 percent for attainment of a degree or certificate, an increase of six percent when compared to the PY 2006 outcome of 44 percent.

Evidence: WIA Youth Program PART Improvement Plan - http://www.whitehouse.gov/omb/expectmore/detail/10000342.2003.html. PY 2006 WIASRD Data Book (Social Policy Research Associates) - http://www.doleta.gov/Performance/results/PY2006_WIASRD_Data_Book.pdf. Training and Employment Guidance Letter (TEGL) 3-04. The Employment and Training Administration's (ETA's) New Strategic Vision for the Delivery of Youth Services Under the Workforce Investment Act (WIA). Available at http://wdr.doleta.gov/directives/corr_doc.cfm?DOCN=1589. DOL's Strategic Plan, Fiscal Years 2006-2011. Strategic plans are available at http://www.dol.gov/_sec/stratplan/main.htm. DOL's FY 2007 Performance and Accountability Report - http://www.dol.gov/_sec/media/reports/annual2007/2007annualreport.pdf.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The WIA Youth program's annual performance measures are identical to the long-term measures described in Section 2.1: (1) percentage of youth placed in employment, post-secondary education, advanced training/occupational skills training, or the military in the first quarter after their exit quarter; (2) percentage of youth, who attain a degree or certificate by the end of the third quarter after their exit quarter; and (3) the percentage of basic skills deficient youth who increase one or more education functioning levels within one year after their date of first program service. WIA requires states to negotiate with DOL to establish expected performance levels for each measure. Results for the three performance outcome measures are reviewed quarterly, and national targets are updated annually to ensure that the program is making progress toward its long-term goals. The efficiency measure is addressed in more detail in Questions 3.4 and 4.3.

Evidence: Training and Employment Guidance Letter (TEGL) 17-05 (February 2006). Common Measures Policy for the Employment and Training Administration's (ETA) Performance Accountability System and Related Performance Issues. Available at http://wdr.doleta.gov/directives/corr_doc.cfm?DOCN=2195. DOL's FY 2007 Performance and Accountability Report - http://www.dol.gov/_sec/media/reports/annual2007/2007annualreport.pdf.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: DOL has established baselines and ambitious annual targets for three of its four annual measures. DOL will have baseline data on the literacy/numeracy measure in November 2008. Results for the literacy/numeracy measure are not reported until a youth participant has completed a full year in the program. Once the baseline data is available, the program will set ambitious annual targets for literacy/numeracy gains. The PY 2007 target for placement in employment or education is 61 percent (compared to the PY 2005 - 53 percent baseline and the PY 2006 - 60 percent result). The PY 2007 target for the attainment of a degree or certificate measure is 45 percent (compared to the PY 2005 - 35 percent baseline and the PY 2006 - 44 percent result). These targets are ambitious, but also achievable, and attaining these annual targets will put the program on track to meet its long-term goals.

Evidence: WIA Youth Program PART Improvement Plan - http://www.whitehouse.gov/omb/expectmore/detail/10000342.2003.html. DOL's FY 2007 Performance and Accountability Report - http://www.dol.gov/_sec/media/reports/annual2007/2007annualreport.pdf.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: States are mandated to adopt specific goals established in statute, negotiate with DOL to establish individual state performance levels, and report on their progress on achieving these levels. States also negotiate performance levels with local workforce investment areas that then contract with local service providers. The contracts between local workforce investment areas and local service providers are almost exclusively performance-based contracts. As prescribed by WIA, states are held accountable by DOL for their performance and may receive incentive funds or suffer financial sanctions based on whether they meet performance levels. Beginning in PY 2007, states must provide complete performance reports that meet DOL's data validation standards to be eligible for an incentive reward. If a state's report validation error rate is greater than two percent for any element, it will be automatically disqualified from incentive consideration.

Evidence: Sections 136 and 503 of the Workforce Investment Act (29 U.S.C. 2871 and 29 U.S.C 9273) (P.L. 105-220) - http://www.doleta.gov/seniors/other_docs/105-220.pdf. Training and Employment Guidance Letter (TEGL) 8-99. Negotiating Performance Goals; and Incentives and Sanctions Process Under Title I of the Workforce Investment Act (WIA). Available at http://wdr.doleta.gov/directives/corr_doc.cfm?DOCN=1200. Training and Employment Guidance Letter (TEGL) 14-00 Change 1. Guidance on the Workforce Investment Act (WIA) Management Information and Reporting System. Available at http://wdr.doleta.gov/directives/corr_doc.cfm?DOCN=1343. Training and Employment Guidance Letter (TEGL) 9-07. Revised Incentive and Sanction Policy for Workforce Investment Act Title IB Programs. Available at http://wdr.doleta.gov/directives/corr_doc.cfm?docn=2544. WIA State Negotiated Performance Levels for PY 2000 to PY 2008 - http://www.doleta.gov/Performance/goals/st_neg_perf_level.cfm.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: Process evaluations have been conducted of the WIA implementation. However, ETA has not conducted evaluations on a sufficiently regular basis to support program improvements. An impact evaluation of the program's predecessor was completed in 1992. ETA is in the process of selecting an independent evaluation contractor to conduct a seven-year random assignment evaluation of the WIA programs, including Youth Activities. The evaluation will examine the program's impacts on participants' post-program employment and earnings and their cost effectiveness in approximately 25 local workforce investment system operations across the country. The evaluation will compare outcomes of randomly-selected WIA participants to the outcomes of similar individuals who do not receive WIA services. Until the results from this impact evaluation are realized, little is known about the effectiveness of the WIA Youth program, as there have not been any evaluations of sufficient scope and quality to evaluate the program's effectiveness. WIA's performance data alone do not provide a complete picture of WIA services.

Evidence: The request for proposals for the WIA Gold Standard Evaluation can be found online at http://www.doleta.gov/sga/rfp/RFP_WIA_GOLD_STD_EVAL.pdf. Social Policy Research Associates, "The Workforce Investment Act after Five Years: Results from the National Evaluation of the Implementation of WIA" - http://www.doleta.gov/reports/searcheta/occ/papers/SPR-WIA_Final_Report.pdf.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: The WIA Youth program's budget requests and the annual Performance Accountability Report (PAR) associate program performance goals with funding and spending. However, the current cost model does not support quantified estimates of funding impact on services or outcomes. Workload summaries included in the budget requests provide rough estimates of the expected changes in participant numbers as determined by resource levels. Under the current law, the FY 2008 budget of $924 million will serve 249,411 disadvantaged youth in PY 2008 and the FY 2009 budget request of $840.5 million will serve 229,019 disadvantaged youth in PY 2009. The actual costs, including overhead, are defined using the Department's cost accounting model and presented in relation to the long-term and annual performance indicators in the PAR. The PAR also describes the resource, legislative or policy factors that influence the costs over the past several years, and explains if these factors had any influence on performance results.

Evidence: DOL's FY 2008 and FY 2009 Performance Budgets are available at http://www.dol.gov/dol/budget/. DOL's FY 2007 Annual Performance and Accountability Report - http://www.dol.gov/_sec/media/reports/annual2007/2007annualreport.pdf

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The first PART review of this program (in 2003) identified weaknesses in collaboration with related programs, in regular independent evaluation of program effectiveness, and in linking funding requests to performance. It also found that the program could strengthen accountability by implementing the common measures. DOL is taking meaningful steps to correct this program's strategic planning deficiencies. In response to the 2003 White House Task Force Report on Disadvantaged Youth, DOL jointly established the Shared Youth Vision Federal Partnership. The Federal Partnership, addressed in more detail in Section 3.5, formed an interagency workgroup to improve the collaboration and communication among Federal, state and local agencies who serve the neediest youth population. In June 2007, DOL and the Federal Partnership awarded grants to 16 Shared Youth Vision pilot states to implement their collaborative strategy to serve the neediest youth and to support integrated systems development. DOL will use the lessons learned from the pilot state projects to inform its strategic planning and to provide guidance to non-pilot states, who are working to implement DOL's strategic vision to serve the neediest youth through the WIA Youth program. A final Shared Youth Vision implementation study, conducted by ABT Associates, will be received in the fall of 2008. To facilitate comparison with similar programs and strengthen accountability, DOL implemented the Job Training Common Measures. Long term targets were established in the DOL's FY 2006-2011 Strategic Plan and then revised in PY 2007 in light of the program's performance in PY 2006. DOL is also in the process of selecting an independent evaluation contractor to conduct a 7 year random assignment evaluation of the WIA Youth program that will examine the program's impacts on participants' post-program employment and earnings and its cost effectiveness. Funding for the WIA Youth program is determined by a statutory formula that hinders DOL's ability to link resource distribution more closely to annual and long-term goals. The funding formula can only be changed through WIA reauthorization, which has not yet occurred. Currently, DOL works to link funding distributions and performance outcomes through the incentive and sanction process. States that do not meet their negotiated performance goals on a WIA youth measures for two years are subject to monetary sanction. Conversely, States that exceed their negotiated performance levels can receive incentive funds.

Evidence: DOL's Strategic Plans, Fiscal Years 2006-2011. Strategic plans are available at http://www.dol.gov/_sec/stratplan/main.htm. Training and Employment Guidance Letter (TEGL) 13-06. Instructions for Workforce Investment Act and Wagner-Peyser Act State Planning and Waiver Requests for Years Three and Four of the Strategic Five-Year State Plan (Program Years 2007 and 2008). Available at http://wdr.doleta.gov/directives/corr_doc.cfm?DOCN=2298. Training and Employment Guidance Letter (TEGL) 9-07. Revised Incentive and Sanction Policy for Workforce Investment Act Title IB Programs. Available at http://wdr.doleta.gov/directives/corr_doc.cfm?docn=2544. Training and Employment Guidance Letter (TEGL) 3-04. The Employment and Training Administration's (ETA's) New Strategic Vision for the Delivery of Youth Services Under the Workforce Investment Act (WIA). Available at http://wdr.doleta.gov/directives/corr_doc.cfm?DOCN=1589. Shared Youth Vision Pilot State Team Application - http://www.doleta.gov/ryf/Resources/ATA%20Pilots%20final.pdf

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: States report performance data quarterly and annually to DOL. DOL uses States' annual performance reports to award incentive grant and sanction decisions. States that fail to achieve one or more negotiated levels of performance are provided technical assistance. ETA also provides a Performance Enhancement Project (PEP) that offers technical assistance to states that are having difficulty meeting the negotiated performance levels. Recent GAO reports have raised concerns regarding the quality of WIA performance data. According to GAO's findings, there is little assurance that the States' performance data for all WIA programs, including the Youth program, are either accurate or complete because of inadequate oversight of data collection and management at all program levels. Previous DOL OIG reports have echoed GAO's findings by also citing inconsistent documentation requirements at the state level, as well as inadequate oversight procedures for monitoring state data reporting at the federal regional office level. DOL has implemented a data validation procedure designed to reduce errors in performance data reported by grantees. States are required to compare data elements reported by the local area against source documentation to verify that the data are accurate. States use DOL's validation software to select a sample of WIA participant files for validation. To address inconsistencies in calculating performance measures, States are also required to use DOL's validation software to verify the accuracy of reported outcomes. This software is based on specifications used to calculate the common performance measures for WIA Adults, Dislocated Workers and Youth programs. As part of their state monitoring requirements, ETA staff conducts periodic data validation reviews that include checking a sample of participant records. DOL developed a standard monitoring tool to help standardize data monitoring procedures. ETA has also recently made state data validation results one of the criteria for incentive awards. While DOL has taken steps to address the data quality issues raised by GAO and OIG, there has been no external assessment of DOL's validation process, and still little is known about whether the process has improved performance data quality. OIG is also in the process of auditing the data validation system.

Evidence: U.S. Department of Labor, Office of Inspector General, "Workforce Investment Act: Evaluation of Youth Program Enrollments, Services, and Recorded Outcomes," Report 06-03-006-03-390, September 30, 2003. Available at http://www.oig.dol.gov/public/reports/oa/2003/06-03-006-03-390.pdf. Government Accountability Office, "Workforce Investment Act: Labor Actions Can Help States Improve Quality of Performance Outcome Data and Delivery of Youth Services," GAO-04-308, February 2004. Available at http://www.gao.gov/new.items/d04308.pdf. Government Accountability Office, "Workforce Investment Act: Labor and States Have Taken Actions to Improve Data Quality, but Additional Steps Are Needed," GAO-06-82, November 2005. Available at http://www.gao.gov/new.items/d0682.pdf. Government Accountability Office, "Workforce Investment Act: Additional Actions Would Further Improve the Workforce System," GAO-07-1051T, June 28, 2007. Available at http://www.gao.gov/new.items/d071051t.pdf. Training and Employment Guidance Letter (TEGL) 9-07. Revised Incentive and Sanction Policy for Workforce Investment Act Title IB Programs. Available at http://wdr.doleta.gov/directives/corr_doc.cfm?docn=2544. WIA Quarterly and Annual Reports - http://www.doleta.gov/Performance/guidance/wia.cfm. WIA Data Validation Handbook - http://www.doleta.gov/Performance/reporting/docs/WIA7_0/DRVS_WIA7_UsersGuide.pdf. PY 2006 WIASRD Data Book - Youth Activities - http://www.doleta.gov/Performance/results/PY2006_WIASRD_Data_Book.pdf.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Federal managers are held accountable through their employee performance rating system. Improved in recent years, the system now aligns performance ratings for managers and supervisors to the achievement of DOL's strategic priorities, and program-specific goals and outcomes. States are held accountable for cost and performance results through monitoring, financial audits, incentives and sanctions. As stipulated in WIA, DOL awards incentive grants to States that meet or exceed negotiated levels of performance. In addition, the statute authorizes the Secretary to apply sanctions to States that fail to meet agreed-upon performance levels. DOL provides technical assistance to States the first time annual performance targets are missed; if the targets are missed a subsequent year, States are subject to monetary sanctions. For PY 2000 and PY 2001, financial sanctions were applied to four states for failing one or more WIA Youth measures. These four States were given the opportunity to develop and implement a corrective action plan, and once completed, request replacement funds up to the amount of the sanction. All four States received the requested replacement funds. The Regional Administrators recommended no States receive a financial sanction based on PY 2001 and PY 2002 performance. However, five States were required to submit corrective action plans outlining strategies for improving program performance. No States were subject to a sanction under WIA for the performance periods of PY 2002, PY 2003, PY 2004, and PY 2005. For PY 2006, two States are eligible for sanctions. Both States are required, at a minimum, to submit performance improvement plan that will require modification to the State's strategic plan. Additionally, the States may be subject to a 1% reduction to PY 2007's WIA Youth Allotment for failure to meet these youth goals for two consecutive years. A recent GAO study found that States and WIBs would benefit from guidance to develop and implement contracts that allow local programs to serve the neediest youth??while still achieving performance goals. DOL is developing the guidance which will provide specific examples of ways to develop contracts with local service providers that allow them to successfully serve youth at varying skill levels.

Evidence: Sections 503 and 136(g) of the Workforce Investment Act (29 U.S.C. 9273 and 2871, P.L. 105-220) - http://www.doleta.gov/seniors/other_docs/105-220.pdf. DOL's Revised Performance Management Plans for Senior Executives (Form DL 1-2059, Rev. 10/2001) and for Supervisors and Managers (Form DL 1-382, Rev. 10/2001). States' Annual Reports - http://www.doleta.gov/Performance/results/Reports.cfm?#wiastann. Government Accountability Office, "Disconnected Youth: Federal Action Could Address Some of the Challenges Faced by Local Programs that Reconnect Youth to Education and Employment," GAO-08-313, February 28, 2008 - http://www.gao.gov/cgi-bin/getrpt?GAO-08-313.

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: DOL promptly and fully obligates the program's funding to the States, consistent with WIA requirements. States are required to obligate at least 80 percent of their funds each year. Labor uses states' financial reports to determine whether there are any unspent funds that may need to be redistributed among states. States have continued to exceed the threshold requirement to obligate 80 percent of funds, thus it has been unnecessary to recapture funds that are unobligated. Although the Federal program fully obligates almost $1 billion in Youth formula funds manner annually in accordance with WIA, States and localities carry over approximately $42 million in unexpended funds, annually.

Evidence: 20 CFR Part 667??Administrative Provisions under Title I of WIA - http://www.access.gpo.gov/nara/cfr/waisidx_00/20cfr667_00.html. Training and Employment Guidance Letter (TEGL) 22-06. Workforce Investment Act (WIA) Adult, Dislocated Worker and Youth Activities Program Allotments for Program Year (PY) 2007. Available at http://wdr.doleta.gov/directives/attach/TEGL/TEGL22-06_Acc.pdf. 9130 Financial Report - http://www.doleta.gov/grants/docs/ETA-9130-straightSF269grants122007.pdf.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: Currently, DOL has an efficiency measure as one of the interagency Job Training Common Measures and established annual and long-term targets for the measure through PY 2009. The annual efficiency measure for the WIA Youth program is the cost-per-participant, calculated by dividing the total annual appropriation for the program year divided by the number of participants for the program year. DOL is working to improve how it measures efficiency and cost-effectiveness. The efficiency measure for the WIA Youth program could be enhanced to more accurately state the cost of services and to reflect the return on its investments in youth. DOL will conduct an analysis to define appropriate outcome-based efficiency measures for the job training programs by September 2008. It will develop, adopt and implement the new efficiency measure by June 2009. The program has developed guidance and instructions for grant recipients to use cost-benefit analysis and participant cost-effectiveness analysis in WIA Annual Reports. This guidance is currently under review and will include revised information on the use of unit cost for WIA programs' various services. The guidance on reporting unit costs of services will help ETA refine the measure(s) for capturing program efficiencies connected to employment-based outcomes. Finally, in conjunction with the DOL's Office of Chief Financial Officer, ETA is reviewing the feasibility of accelerating targeted reduction of improper payments for WIA (including Youth program) in 2009 and 2010.

Evidence: Section 123 of the Workforce Investment Act (29 U.S.C. 2854 - Public Law 105-220). DOL's FY 2007 Performance and Accountability Report - http://www.dol.gov/_sec/media/reports/annual2007/2007annualreport.pdf. DOL's FY 2008 and FY 2009 Performance Budgets - http://www.dol.gov/dol/budget/. Training and Employment Guidance Letter (TEGL) 14-00 Change 1. Guidance on the Workforce Investment Act (WIA) Management Information and Reporting System. Available at http://wdr.doleta.gov/directives/corr_doc.cfm?DOCN=1343.

Does the program collaborate and coordinate effectively with related programs?

Explanation: The WIA Youth program collaborates and coordinates effectively with related youth programs at the Federal, state and local levels through the WIA strategic planning process and the Shared Youth Vision Federal Partnership. In response to the 2003 White House Task Force Report on Disadvantaged Youth, DOL implemented a strategic vision for the delivery of services to America's neediest youth. DOL jointly established the Shared Youth Vision Federal Partnership, a workgroup involving eight other Federal agencies that works to increase the collaboration and communication among youth-serving agencies at the Federal, state and local levels. In its efforts to increase program collaboration and improve outcomes for neediest youth, the group has engaged in numerous activities, including hosting forums, announcing competitions for and funding pilot demonstration grants, and providing technical assistance to States and local workforce systems to support their implementation of the Shared Youth Vision. In June 2007, DOL and the Federal Partnership jointly awarded financial assistance to the 16 Advanced Level state teams to implement their collaborative strategy and to support integrated systems development. These pilot States have made progress in using WIA Youth funds strategically to provide services to the neediest youth and in leveraging funds to improve service delivery. For example, Ohio has implemented a project in four counties piloting the integration of services for youth offenders between Temporary Assistance for Needy Families (TANF), WIA and the State's Department of Youth Services. By integrating funding streams, youth offenders in the participating counties can now access wrap-around services funded by TANF and WIA through their local One-Stop. In November 2007, Florida issued guidance on targeting WIA services to youth in the juvenile justice and foster care systems to the statewide workforce development system.

Evidence: Shared Youth Vision (SYV) Federal Partnership Facts & Results. Training and Employment Guidance Letter (TEGL) 3-04. The Employment and Training Administration's (ETA's) New Strategic Vision for the Delivery of Youth Services Under the Workforce Investment Act (WIA). Available at http://wdr.doleta.gov/directives/corr_doc.cfm?DOCN=1589. Training and Employment Notice (TEN) 28-07. The Shared Youth Vision: A Collaborative Approach to Prepare Youth for Success in a Global, Demand-Driven Economy. Available at http://wdr.doleta.gov/directives/corr_doc.cfm?docn=2584. Shared Youth Vision Pilot State Team Application - http://www.doleta.gov/ryf/Resources/ATA%20Pilots%20final.pdf.

Does the program use strong financial management practices?

Explanation: The program is free of material internal control weaknesses as identified by auditors and the DOLAR$ financial system meets the statutory requirements as indicated by audit findings from FY 2007. No material weaknesses were noted in the FY 2007 audit of DOL's financial statements. In 2006, an analysis of the single audit clearinghouse data system determined that the extent of questionable costs reported annually in the WIA system, including the WIA Youth program, was small. Federal Project Officers (FPOs) monitor all active grantees on a quarterly basis using the Grants Electronic Management System (GEMS) to ensure funds are spent on allowable goods and services and to track performance. The annual A-123 Internal control assessment of key business processes (i.e. grants management) and related Information Technology controls concluded that the Department's internal controls over financial reporting were properly designed and operating effectively as of June 30, 2007. When financial reporting issues are identified, the regional offices provide on-site technical assistance and follow-up with the grantee to ensure that the issue has been resolved. Depending on the nature of the issues, closure may require grantees to submit documentation, copies of revised policies, accounting system adjustments, or descriptions of actions taken to close issues with sub-recipients. If unallowable costs are involved, and appropriate resolution cannot be reached, recommendations may be made to the grant officer to use the Findings and Determination process.

Evidence: DOL's FY 2006 and 2007 Performance and Accountability Reports - http://www.dol.gov/dol/aboutdol/budget.htm and http://www.dol.gov/_sec/media/reports/annual2007/, respectively. Financial Reporting 9130 - http://www.doleta.gov/grants/docs/ETA-9130-straightSF269grants122007.pdf.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: DOL has worked to improve performance accountability of Federal managers, management of grantees and sub-recipients, and erroneous payments. Federal employees' performance standards and reviews are aligned on a fiscal year cycle to help cascade standards to non-supervisory personnel. DOL has issued guidance, advising grantees that performance ratings for managers and supervisors should align to the achievement of Departmental strategic and program-specific goals and outcomes. DOL has developed a strategy for program reporting to address inconsistencies in the reporting of program information. All States are required to use DOL's data validation software to verify the annual report outcomes submitted to the Department. This software is based on specifications used to calculate the common performance measures for WIA Adults, Dislocated Workers and Youth programs. In addition to certifying the reliability of the reports, the software draws a stratified sample of participant records for review. Each participant file that is selected as part of the sample is reviewed to ensure the data collected is complete, accurate, and verifiable. Beginning in PY 2007, data validation results will be one of the criteria for determining eligibility for WIA incentive awards. If a state's report validation error rate is greater than two percent for any element, it will be automatically disqualified from incentive consideration.

Evidence: Memorandum from Patrick Pizzella, Assistant Secretary for Administration and Management to Agency Heads regarding FY 2008 Senior Executive Service (SES) Performance Plan Guidance, October 29, 2007. Memorandum from Dina Rambert, Director, Office of Human Resources, to ETA Executive Staff and Administrative Liaison Offices regarding Performance Plans, October 2007. DOL's Revised Performance Management Plans for Senior Executives (Form DL 1-2059, Rev. 10/2001) and for Supervisors and Managers (Form DL 1-382, Rev. 10/2001).

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: ETA and this program do have in place an electronic grants administration reporting system to document grantees' use of funds for eligible activities and to track expenditures. As part of their formal grant agreement, all ETA grantees are notified of their requirement to procure and complete an internal audit on an annual basis in accordance with OMB Circular A-133. DOL's electronic grants (E GRANTS) administration system contains the Grants Electronic Management System (GEMS) and is used to track grantee funding and performance. This system provides automated tools for conducting grant monitoring activities, including performing risk assessments and generating reports. DOL's regional federal project officers (FPO) use GEMS to track and document grant management activities, including on-site monitoring and technical assistance reviews. FPOs are responsible for conducting fiscal and program performance desk reviews of the quarterly performance and financial reports, and tracking the extent to which projects have met/exceeded all applicable goals (either programmatic or as outlined in the grant's statement of work). DOL requires regional offices to conduct a review of each State's WIA Youth program every two years. The review includes visits to several local areas. If necessary, regional Federal staff and grantees jointly develop corrective action plans to address program deficiencies. Despite these tools and procedures, ETA and this program face limitations in their knowledge of grantees' financial activities, but the Administration has proposed changes to WIA to improve this situation. Due to limited WIA financial tracking and reporting standards, ETA does not have extensive knowledge on the State and local commitments, or obligations, to spend program funds that they receive from DOL are firm. Therefore, it is difficult for DOL to say confidently the amount of program funds that remain available within a grant award period for States to provide local services or whether funds have been spent as intended.

Evidence: ETA's Core Monitoring Guide and Supplemental Financial Monitoring Guide. Employment and Training Order (ETO) No. 1-03, Improving the Administration of Grants Within the Employment and Training Administration. Significantly, the Order shifted substantial post-award grants management responsibilities to the Regional Offices; provided new, limited Grant Officer functions for Regional Administrators; instituted quarterly grant desk review/risk assessment procedures for Federal Project Officers (FPOs) and defined new parameters for on-site visits to grantees. The U.S. DOL ETA Web-based Financial Report (ETA-9130), required for use by all ETA grantees beginning with the quarter ending September 30, 2007, includes a new Federal Cash Section and a Federal Administrative Expenditures line item. Provision to collect program-specific data is also now accommodated in a special section of the report - http://www.doleta.gov/regions/reg05/Pages/libraryforms.cfm. Training and Employment Guidance Letter (TEGL) No. 16-99 and 16-99, Change- 1. Workforce Investment Act Financial Reporting. Available at http://wdr.doleta.gov/directives/corr_doc.cfm?DOCN=1229 and http://wdr.doleta.gov/directives/corr_doc.cfm?DOCN=1433, respectively. Government Accountability Office, "Workforce Invesment Act: States' Spending Is on Track, but Better Guidance Would Improve Financial Reporting," GAO-03-239, November 2002 - http://www.gao.gov/cgi-bin/getrpt?GAO-03-239. Government Accountability Office, "Workforce Investment Act: Labor and States Have Taken Actions to Improve Data Quality, but Additional Steps Are Needed," GAO-06-82, November 2005. Available at http://www.gao.gov/new.items/d0682.pdf. DOL's FY 2007 Performance and Accountability Report - http://www.dol.gov/_sec/media/reports/annual2007/2007annualreport.pdf. OIG Findings and Recommendations Identified in an Audit of the Consolidated Financial Statements for the Year Ended September 30, 2006 - http://www.oig.dol.gov/public/reports/oa/2007/22-07-001-13-001.pdf.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: States submit WIA performance information to DOL quarterly and annually. On a quarterly basis, DOL works with state grantees to review and analyze States' performance information and helps States translate their performance to support policy decisions. On its Website, DOL provides an easy link to quarterly and annual performance results for the nation and individual states. This information includes summaries of WIA participant characteristics, services received and comparisons of states' performance. DOL works continually to enhance its performance Website to make it easier for the public to find performance information, to understand what DOL and its partners seek to achieve, and to determine whether those goals have been attained.

Evidence: PY 2006 WISARD Data Book - http://www.doleta.gov/Performance/results/Reports.cfm. DOL's FY 2001 through FY 2006 Performance and Accountability Reports - http://www.dol.gov/dol/aboutdol/budget.htm. DOL's FY 2007 Performance and Accountability Report - http://www.dol.gov/_sec/media/reports/annual2007/2007annualreport.pdf.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: The outcomes, for both the common measures for PY 2005-06 and the statutory measures in recent years, demonstrate significant progress in achieving long-term goals for the program. In PY 2005, the WIA Youth program established baselines for placement in employment or education (53 percent) and attainment of a degree or certificate (35 percent). In PY 2006, the program met its target of 60 percent in the placement in education or employment measure and exceeded its target of 40 percent in the attainment of a degree or certificate measure with a result of 44 percent. Before common measure implementation, the WIA Youth program used two other measures??entry into employment and diploma attainment. While very similar in outcome types, these measures differed in methodology from the common measures. Results under the statutory measures were much higher because the calculations for these measures included a more narrow set of participants. In PY 2005, the WIA Youth program continued to collect data on these older measures, while collecting baseline data on the common measures. In PY 2005, the program achieved 76 percent in the entry into employment measure and 65 percent in the diploma attainment measure. In PY 2004 and PY 2003, the program exceeded its targets on both measures. In PY 2004, it achieved a result of 72 percent in the entry into employment measure, four percent above the target of 68 percent, and a result of 65 percent on the diploma attainment measure, 12 percent above the target of 53 percent. In PY 2003, it achieved a result of 71 percent in the entry into employment measure, six percent above the target of 65 percent, and a result of 63 percent in the diploma attainment measure, 11 percent above the target of 52 percent. The results from PY 2003 to PY 2005 demonstrate continual improvement in the entry in employment measure. Performance in the diploma attainment measure improved from PY 2003 to PY 2004, and remained constant from PY 2004 to PY 2005. Long-term goals have not yet been set for literacy/numeracy gains.

Evidence: DOL's FY 2007 Performance and Accountability Report -http://www.dol.gov/_sec/media/reports/annual2007/2007annualreport.pdf. DOL's FY 2008 and FY 2009 Performance Budgets - http://www.dol.gov/dol/budget/

Does the program (including program partners) achieve its annual performance goals?

Explanation: The program's annual performance measures and goals are the same as the long-term measures and goals in Question 4.1. The outcomes, for both the common measures for PY 2005-06 and the statutory measures in recent years, demonstrate significant progress in achieving long-term goals for the program.. In PY 2005, the WIA Youth program established baselines for placement in employment or education (53 percent) and attainment of a degree or certificate (35 percent). In PY 2006, the program met its target of 60 percent on the placement in education or employment measure and exceeded its target of 40 percent in the attainment of a degree or certificate measure with a result of 44 percent. Before common measure implementation, the WIA Youth program used two other measures??entry into employment and diploma attainment. While very similar in outcome types, these measures differed in methodology from the common measures. Results under the statutory measures were much higher because the calculations of the statutory measures included a more narrow set of participants. In PY 2005, the WIA Youth program continued to collect data on these older measures, while collecting baseline data on the common measures. In PY 2005, the program achieved 76 percent on the entry into employment measure and 65 percent on the diploma attainment measure. In PY 2004 and PY 2003, the program exceeded its targets on both measures. In PY 2004, it achieved a result of 72 percent in the entry into employment measure, four percent above the target of 68 percent, and a result of 65 percent in the diploma attainment measure, 12 percent above the target of 53 percent. In PY 2003, it achieved a result of 71 percent in the entry into employment measure, six percent above the target of 65 percent, and a result of 63 percent in the diploma attainment measure, 11 percent above the target of 52 percent. The results from PY 2003 to PY 2005 demonstrate continual improvement in the entry in employment measure. Performance on the diploma attainment measure improved from PY 2003 to PY 2004, and remained constant from PY 2004 to PY 2005.

Evidence: DOL's FY 2007 Performance and Accountability Report -http://www.dol.gov/_sec/media/reports/annual2007/2007annualreport.pdf. DOL's FY 2008 and FY 2009 Performance Budgets - http://www.dol.gov/dol/budget/.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: ETA and this program have in place some procedures related to efficiencies and cost effectiveness, but they are insufficient to fully measure and achieve efficiencies and cost effectiveness in program execution. Currently, DOL has an efficiency measure as one of the interagency Job Training Common Measures and established annual and long-term targets for the measure through PY 2009. However, the cost-per-participant for the WIA Youth program has increased in recent years. In PY 2006, the cost-per-participant was $3,778, an increase over the PY 2005 cost-per-participant of $3,615. ETA is pursuing strategies to support increased efficiency and effectiveness by encouraging the leveraging of resources among service delivery systems through the collaborative efforts of the Shared Youth Vision.

Evidence: U.S. Department of Labor, PY 2006 WIASRD Data Book (Social Policy Research Associates) - http://www.doleta.gov/Performance/results/PY2006_WIASRD_Data_Book.pdf. DOL's FY 2007 Performance and Accountability Report - http://www.dol.gov/_sec/media/reports/annual2007/2007annualreport.pdf. Training and Employment Guidance Letter (TEGL) 3-04. The Employment and Training Administration's (ETA's) New Strategic Vision for the Delivery of Youth Services Under the Workforce Investment Act (WIA). Available at http://wdr.doleta.gov/directives/corr_doc.cfm?DOCN=1589.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: The WIA Youth program compares favorably to similar programs. Department of Education's Adult Basic Education and State Literacy Grants provide workplace literacy and education programs to a population that overlaps with the WIA Youth program: all adults and out-of-school youth over the age of 16. Results for similar performance measures are comparable between the Adult Basic Education and the WIA Youth programs. The Adult Basic Education program has two measures for placement in post-secondary education and employment. For 2006, the Adult Basic Education program achieved a 35 percent rate on its placement in post-secondary education measure and a 48 percent rate on its placement in employment rate. In PY 2006, the WIA Youth program achieved 60 percent on its placement in employment or education measure, exceeding Adult Basic Education's performance on both of its placement measures by impressive margins. In 2006, 49 percent of Adult Basic Education program participants with a high school completion goal earned a diploma or equivalent. For the same period, 44 percent of WIA Youth participants attained a degree or certificate. While the Adult Basic Education program appears to compare favorably, it should be noted that its measures include qualifiers, such as the "percentage of adults with a goal to enter postsecondary education or training" and "the percentage of adults with a high school completion goal." The WIA Youth performance measures include all participants, without regard to personal employment, placement, or educational goals. The National Guard YouthChalleNGe, a residential program that costs approximately $14,000 per participant, reports a high school graduation rate of 67 percent for PY 2005, the most recent year for which data is available. The WIA Youth program attained a comparable result of 65 percent for PY 2005 on its similar statutory diploma attainment rate. The WIA Youth program achieved this performance at a cost of $3,615 per participant in PY 2005. DOL's Jobs Corps program reports higher performance results in both the placement and attainment of a degree or certificate measures. However, as Job Corps is a residential program, its costs per participant are considerably higher than the WIA Youth program.

Evidence: DOL's FY 2007 Performance and Accountability Report -http://www.dol.gov/_sec/media/reports/annual2007/2007annualreport.pdf. AEFLA: Adult Basic and Literacy State Grants FY 2007 Program Performance Report - http://www.ed.gov/about/reports/annual/2007report/g5aeflaadultbasic.doc. National Guard Youth ChalleNGe 2005 Annual Report - http://www.ngycp.org/aboutus_dependant_T9_R66.php.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: Although there have been a number of independent evaluations of WIA programs, there have not been any of sufficient scope and quality to evaluate the program's effectiveness, especially its impact on participants' employment and earnings. Evaluations to date have focused on the administration and processes of WIA programs. These evaluations have provided some valuable insight about delivering employment services and job training, and including implementation strategies that certain states have adopted. ETA is in the process of selecting an independent evaluation contractor to conduct a seven-year random assignment evaluation of the WIA Youth program.

Evidence: Social Policy Research Associates, "The Workforce Investment Act after Five Years: Results from the National Evaluation of the Implementation of WIA" - http://www.doleta.gov/reports/searcheta/occ/papers/SPR-WIA_Final_Report.pdf. Nelson A. Rockefeller Institute of Government, "The Workforce Investment Act in Eight States" - http://www.doleta.gov/reports/searcheta/occ/papers/Rockefeller_Institute_Final_Report2-10-05.pdf. The request for proposals for the WIA Gold Standard Evaluation can be found online at http://www.doleta.gov/sga/rfp/RFP_WIA_GOLD_STD_EVAL.pdf. Government Accountability Office, "Workforce Investment Act: Labor Actions Can Help States Improve Quality of Performance Outcome Data and Delivery of Youth Services," GAO-04-308, February 23, 2004 - http://www.gao.gov/new.items/d04308.pdf. Government Accountability Office, "Workforce Investment Act: Youth Provisions Promote New Service Strategies, but Additional Guidance Would Enhance Program Development," GAO-02-413, April 5, 2002 - http://www.gao.gov/new.items/d02413.pdf.