United States Department of Labor

	All DOL  OSEC    Advanced Search
A to Z |  Site Map |  FAQs |  Forms |  About DOL |  Contact Us |  Español

DOL Annual Report, Fiscal Year 2009
Performance and Accountability Report

Strategic Goal 1: A Prepared Workforce

Develop a prepared workforce by providing effective training and support services to new and incumbent workers and supplying high-quality information on the economy and labor market.

America's greatest resource is its skilled workforce. A steady stream of workers who possess skills required by today's employers is essential to fuel the Nation's economy. To expand the size and capabilities of the labor pool, DOL provides comprehensive training programs that focus on specific occupational skills while taking into account job seekers' circumstances. The Department also produces labor statistics that individuals and businesses can use to better understand the job market and the economy. DOL agencies and offices supporting this goal are:

• Bureau of Labor Statistics (BLS),
• Office of Job Corps (OJC),
• Employment and Training Administration (ETA),
• Veterans' Employment and Training Service (VETS), and
• Women's Bureau

A prepared workforce has the skills and education that employers demand. Education — from literacy to vocational training — plays a fundamental role in preparing workers for life-long employment. DOL programs tailor training to the unique needs of individuals who face exceptional barriers to successful employment, such as low-income youth and homeless veterans. Here are a few highlights of program outcomes for FY 2009:

For Youth

• Job Corps students' credentials attainment rate and literacy/numeracy gains improved by two percent and five percent, respectively, over the past year.
• Workforce Investment Act (WIA) Youth program participants' placement in employment, post-secondary education or advanced occupational training improved by five percentage points, and literacy/numeracy gains improved by almost nine percentage points, compared to the previous year.

For Workers in the Trades

• Eighty-seven percent of workers completing apprenticeship programs remained employed for at least six months, and their average wages continued to rise.

For Veterans

• Entered employment and employment retention rates for all veterans served by One-Stop Career Centers fell by 3.8 and 1.2 percentage points, respectively.

For more specific information, see the Performance Goal narratives.

---

When Mercedes enrolled at the Wolf Creek Job Corps Center a few years ago, she was working two entry-level, low-paying jobs and had no immediate plans for the future. She graduated from Wolf Creek after completing her career technical training in construction craft labor and earning her GED. Her goal of employment as a Construction Craft Laborer for the Laborers’ International Union of North America was clearly within reach. Shortly after graduating, Mercedes started work as a laborer and pre-apprentice on a bridge rehabilitation project in Winchester, Oregon earning $15.50 an hour. She assists with preparing and stabilizing the bridge using a thirty-five pound concrete drill. Her daily commute to the job site includes climbing eight flights of scaffolding and walking across metal beams suspended by cables attached to the main bridge. She loves her job and the crew she works with. Without the Job Corps program, Mercedes does not believe she would have had the skills needed to succeed. [Photo credit: Tracy Placido]

---

Job Corps and the WIA Youth program both utilize the Federal job training program common measures for youth.¹⁴ These measures reflect shared outcome goals and facilitate comparison for management purposes. The adjacent chart provides placement and credentials attainment results to date for both programs. Although both programs target out-of-school and at-risk youth, results may continue to differ because one is primarily a longer-term residential program, while the other provides services to youth in their communities. Also, they have followed different common measures implementation trajectories. For more detailed discussion, see each program's performance goal narrative in the pages that follow.

The table below provides net costs for all performance goals and indicators associated with this strategic goal.¹⁵ Those with labels that begin with "08" operate on a Program Year (PY) basis, and are reporting on the period from July 1, 2008 to June 30, 2009 due to the forward funding authorized in the Workforce Investment Act of 1998.

Goal or Indicator	Net Costs ($Millions)16
	FY 2007 PY 2006	FY 2008 PY 2007	FY 2009 PY 2008
Strategic Goal 1: A Prepared Workforce	$3,267	$3,464	$3,687
Performance Goal 09-1A (BLS) Improve information available to decision-makers on labor market conditions, and price and productivity changes.	574	574	589
Percent of output, timeliness, accuracy, and long-term improvement targets achieved for labor force statistics	268	276	278
Percent of output, timeliness, accuracy, and long-term improvement targets achieved for prices and living conditions	198	192	198
Percent of output, timeliness, accuracy, and long-term improvement targets achieved for compensation and working conditions	95	92	96
Percent of output, timeliness, accuracy, and long-term improvement targets achieved for productivity and technology	12	13	13
Customer satisfaction with BLS products and services (e.g., the American Customer Satisfaction Index)	0	0	3
Cost per transaction of the Internet Data Collection Facility	1	1	1
Performance Goal 08-1B (Job Corps) Improve educational achievements of Job Corps students and increase participation of Job Corps graduates in employment and education.	1,485	1,589	1,640
Percent of participants entering employment or enrolling in post-secondary education or advanced training/occupational skills training in the first quarter after exit	1,485	1,589	1,640
Percent of students who attain a GED, high school diploma or certificate by the end of the third quarter after exit
Percent of students who achieve literacy or numeracy gains of one Adult Basic Education (ABE) level
Performance Goal 08-1C (WIA Youth) Increase placements and educational attainments for youth served through the WIA youth program.	866	966	1,125
Percent of youth who enter employment or the military or enroll in post secondary education and/or advanced training/occupational skills training in the first quarter after exit	866	966	1,125
Percent of students who attain a GED, high school diploma, or certificate by the end of the third quarter after exit
Percent of students who achieve literacy or numeracy gains of one Adult Basic Education (ABE) level
Performance Goal 09-1D (Apprenticeship) Improve the registered apprenticeship system to meet the training needs of business and workers in the 21st Century.	24	25	25
Percent of those employed nine months after registration as an apprentice	24	25	25
Average hourly wage gain for tracked entrants employed in the first quarter after registration and still employed nine months later
Percent of participants employed in the first quarter after exit	—	—
Percent of participants employed in the first quarter after exit still employed in the second and third quarters after exit	—	—
Average earnings in the second and third quarters after exit (six months' earnings)	—	—
Performance Goal 08-1E (VETS Employment Services) Improve the employment outcomes for veterans who receive One-Stop Career Center services	181	190	219
Percent of Veteran participants employed in the first quarter after exit	90	93	109
Percent of Veteran participants employed in the first quarter after program exit still employed in the second and third quarters after exit
Percent of Disabled Veteran participants employed in the first quarter after exit	90	95	109
Percent of Disabled Veteran participants employed in the first quarter after exit still employed in the second and third quarters after exit
Dollars not associated with indicators	2	2	2
Other (Youth Offender Reintegration, Indian and Native American Youth Programs, Homeless Veterans' Reintegration Program, etc.)	136	120	88

Performance Goal 09-1A (BLS)

Improve information available to decision-makers on labor market conditions, and price and productivity changes.

Indicators, Targets and Results

*Indicator target reached (Y), improved (I), or not reached (N)		FY 2006 Goal Not Achieved	FY 2007 Goal Achieved	FY 2008 Goal Not Achieved	FY 2009 Goal Achieved
Percent of output, timeliness, accuracy, and long-term improvement targets achieved for labor force statistics	Target	85%	80%	92%	88%
	Result	79%	92%	86%	92%
	*	N	Y	N	Y
	Cost	—	$268	$276	$278
Percent of output, timeliness, accuracy, and long-term improvement targets achieved for prices and living conditions	Target	85%	90%	90%	92%
	Result	94%	90%	92%	92%
	*	Y	Y	Y	Y
	Cost	—	$198	$192	$198
Percent of output, timeliness, accuracy, and long-term improvement targets achieved for compensation and working conditions	Target	85%	86%	96%	96%
	Result	77%	96%	96%	100%
	*	N	Y	Y	Y
	Cost	—	$95	$92	$96
Percent of output, timeliness, accuracy, and long-term improvement targets achieved for productivity and technology	Target	85%	86%	100%	86%
	Result	100%	100%	71%	100%
	*	Y	Y	N	Y
	Cost	—	$12	$13	$13
Customer satisfaction with BLS products and services per the American Customer Satisfaction Index	Target	75%	79%	79%	82%
	Result	79%	79%	82%	82%
	*	Y	Y	Y	Y
	Cost	—	$0	$0	$3
Cost per transaction of the Internet Data Collection Facility	Target	$2.58	$1.79	$1.11	$0.74
	Result	$1.82	$1.12	$0.76	$0.56
	*	Y	Y	Y	Y
	Cost	—	$1	$1	$1
Goal Net Cost (millions)		$573	$574	$574	$589

Source(s): Budget submissions and internal BLS management meeting structure documents and American Customer Satisfaction Index Annual Report on Federal government scores.
Note: Net Cost reflects the full cost of each program as assigned by DOL entities to the Department's outcome goals less any exchange revenue earned. Full cost consists of (a) both direct and indirect costs, and (b) the costs of identifiable supporting services provided by other segments within DOL and by other Federal agencies.

Program Perspective and Logic

The Bureau of Labor Statistics (BLS) is the principal fact-finding agency in the Federal government in the broad field of labor economics. As an independent national statistical agency within the Department of Labor, BLS collects, processes, analyzes, and releases essential statistical data to the American public, the U.S. Congress, other Federal agencies, State and local governments, business, and labor. BLS provides information that supports the formulation of economic and social policy and decisions that affect virtually all Americans.

BLS reports performance for this goal by aggregating underlying measures of output, timeliness, accuracy, and long-term improvement for its statistical programs into four comprehensive performance indicators associated with Labor Force Statistics, Prices and Living Conditions, Compensation and Working Conditions, and Productivity and Technology. Over 100 performance and workload measurements make up these four performance indicators. Information on the underlying performance and workload measures is available on the BLS website at http://www.bls.gov/bls/dwpt2010.pdf.

---

Duncan is a director of an $8 billion chemical company with operations in more than 40 countries. His company supplies products for generating power, manufacturing commercial and industrial products, and conducting scientific research. Duncan incorporates automatic price adjustment clauses in his contracts that use inflation data from the BLS Consumer Price Index (CPI) and Producer Price Index (PPI). This provides a fair and objective means of setting prices; gives suppliers an incentive to become more efficient, since the price they receive for their goods or services is preset; allows for better cost planning by reducing the uncertainty associated with day-to-day market fluctuations; and reduces the administrative costs associated with having to frequently renegotiate contracts. [Photo credit: DOL/BLS]

---

The underlying performance measures encompass enhancements to BLS products. Examples include publication, for the first time, of employment information on people with disabilities (Labor Force Statistics indicator); publication of a new group of indexes that track price changes for specific services, regardless of the services' industry of origin (Prices and Living Conditions indicator); production of national estimates of workplace injuries and illness incurred by State and local government workers (Compensation and Working Conditions indicator); and expansion of industries covered by labor productivity measures (Productivity and Technology indicator).

BLS measures customer satisfaction with its Occupational Outlook Handbook (OOH) Web page (www.bls.gov/OCO/), a nationally recognized source of career information. BLS also reports on the efficiency of its Internet Data Collection Facility (IDCF), a single, secure architecture that BLS surveys use to collect information from respondents online. IDCF offers a wider range of reporting options to respondents and a more economical means of data collection.

Analysis and Future Plans

BLS reached the targets for all six of its performance indicators, achieving its performance goal and equaling or exceeding the 2008 results for all of its indicators. BLS reached 92 percent of the underlying targets for its labor force statistics indicator, compared to 86 percent in FY 2008. For its prices and living conditions indicator, BLS reached 92 percent of its underlying targets — unchanged from FY 2008. BLS reached 100 percent of its underlying targets for both the compensation and working conditions and productivity and technology indicators, compared to the performance of 96 percent and 71 percent respectively in FY 2008. BLS received a customer satisfaction score of 82 percent, which is the same score received in FY 2008.

Indicator	*	What worked	What didn't work
Labor force statistics	Y	In late 2008, identified 2009 projects that were critical to the BLS mission. Prepared detailed documentation and workplans to help keep projects on schedule, which helped ensure that targets were met. Held regular meetings between senior management and statistical program offices to monitor progress on mission-critical projects and other goals. Ensured potential concerns were brought to the attention of senior management before they developed into significant problems.	Labor force statistics: Missed an output target in the Current Employment Statistics program due to a decline in several industry sectors. BLS eliminated series that no longer could be released due to confidentiality issues. Labor force statistics: Missed a workload target in the Quarterly Census of Employment and Wages due to slower than projected business growth. Prices and living conditions: Missed a workload target in the Consumer Expenditure surveys due to below average response rates. Although the target was not reached, in later months, BLS increased the response rates by improving the sampling process. Prices and living conditions: Missed a workload target in the International Price Program (IPP). Also missed an accuracy target in IPP due to the volatility of petroleum data.
Prices and living conditions	Y
Compensation and working conditions	Y
Productivity and technology	Y
Customer satisfaction	Y	Completed a comprehensive website redesign in late 2008.
IDCF efficiency	Y	Expanded usage of IDCF, which decreased cost per transaction.
Program Performance Improvement Plan
BLS will continue to improve its statistical products by analyzing and evaluating new economic survey methods, new technologies, and new survey design and collection approaches. BLS will better inform the public by increasing customer awareness of data products, and using feedback from data users to determine how the data or data dissemination methods could be changed to better serve the American public. Further expand use of the IDCF to further improve the data collection process and enhance the quality of data produced.

*Target reached (Y), improved (I), or not reached (N)

Q: Who does BLS count as unemployed? A: Persons are classified as unemployed if they do not have a job, have actively looked for work in the prior four weeks, and are currently available for work. Workers expecting to be recalled from layoff are counted as unemployed, whether or not they have engaged in a specific jobseeking activity. In all other cases, the individual must have been engaged in at least one active job search activity in the four weeks preceding the interview and be available for work (except for temporary illness).

Net costs for BLS activities rose three percent from FY 2008-2009 due to budgeted increases in personnel compensation and benefits costs.

In 2010, BLS' activities will provide statistical data needed by policymakers in support of the following outcome goals in the Department's Strategic Vision of Good Jobs for Everyone:

• Increasing workers' incomes and narrowing wage and income inequality;
• Securing safe and healthy workplaces, particularly in high-risk industries;
• Securing wages and overtime;
• Assuring skills and knowledge that prepare workers to succeed in a knowledge-based economy, including in high-growth and emerging industry sectors like "green" jobs;
• Improving health benefits and retirement security for all workers;
• Providing workplace flexibility for family and personal care-giving; and
• Voice in the workplace.

Program Assessments, Evaluations and Audits

Q: Is the count of unemployed persons limited to just those people receiving unemployment insurance benefits? A: No; the estimate of unemployment is based on a monthly sample survey of households. All persons without jobs who are actively seeking and available to work are included among the unemployed. There is no requirement or question relating to unemployment insurance benefits in the monthly survey.

Findings and recommendations from an assessment of the Bureau of Labor Statistics prompted specific actions to improve performance. Here is a summary of progress in 2009:

• Conducting an independent evaluation on how the agency is improving its effectiveness in meeting the needs of its data users. The scope of the evaluation will be the National Longitudinal Surveys program. BLS will be working with the Committee on National Statistics, an independent agency with a reputation for credibility and professionalism, to prepare for the evaluation.
• Developing additional efficiency and cost-effectiveness measures to demonstrate ongoing program improvement. BLS continues to report on its cost per transaction of the IDCF. In 2010, BLS will update the efficiency measure to reduce the cost per housing unit initiated in the Consumer Price Index (CPI) as part of an initiative to implement a more representative and current sample of geographic areas, as well as a continuously updated housing sample, in the CPI. Also, BLS is reporting on an efficiency measure to produce more American Time Use Survey estimates without commensurate increases in cost.
• Establishing more ambitious targets for its long term and annual performance measures to drive continued improvement. In 2008 and again in 2009, BLS raised the targets for several of its performance indicators.

More information is available at http://www.whitehouse.gov/omb/expectmore/summary/10000326.2003.html.

Independent evaluations and audits completed in FY 2009 are summarized below.

"Strengthening Formal Written Procedures May Decrease the Possibility of Inconsistencies Occurring in Survey of Occupational Injuries and Illnesses Data Releases," March 2009 (Office of Inspector General)
Relevance: The Office of Inspector General (OIG) conducted this performance audit to determine if the BLS Survey of Occupational Injuries and Illnesses (SOII) complied with Federal standards for development and data collection.
Findings and Recommendations: While BLS implemented OMB's 2006 Standards and Guidelines for Statistical Surveys related to its SOII development and data collection, its formal written policies and procedures did not fully incorporate all of the OMB requirements.		Next Steps: The SOII program will enhance formal documentation and develop a centralized source of information that identifies how SOII design and/or procedures meet each of the OMB statistical standards.
Additional Information: The report is available at http://www.oig.dol.gov/public/reports/oa/2009/02-09-201-11-001.pdf.
"Customer Satisfaction with the BLS Occupational Outlook Handbook (OOH) Website," September 2009 (Federal Consulting Group)
Relevance: As BLS continues to provide more information to customers on its Web site, it is important to know how satisfied customers are with the delivery of BLS products and services. Improvements to the OOH portion of the BLS Web site in areas such as search or navigation can increase the usefulness of the Web site to BLS customers.
Findings and Recommendations: BLS received a customer satisfaction score of 82 percent for 2009.	Next Steps: Information from the OOH customer satisfaction survey, as well as other customer feedback is being used to develop a new and improved online OOH by enhancing its content and presentation. The goal is to release the new online product in 2012.
Additional Information: Please contact BLS Quality Management Staff at 202-691-7755.

Q: Does the official unemployment rate exclude people who have stopped looking for work? A: Yes; however, there are separate estimates of persons outside the labor force who want a job, including those who have stopped looking because they believe no jobs are available (discouraged workers). In addition, alternative measures of labor underutilization (discouraged workers and other groups not officially counted as unemployed) are published each month in the Employment Situation news release.

Data Quality and Top Management Challenges

Data quality for this performance goal is rated Excellent.¹⁷ BLS has instituted rigorous, systematic, and comprehensive controls to ensure that the data quality retains its Excellent rating. The BLS executive team meets with program management throughout the year to discuss progress on mission-critical projects, long-term planning, and other significant issues. BLS also conducts its own program reviews and contracts for external reviews, as necessary. These assessments ensure that survey data are accurate, reliable, and released in a timely fashion; systems and procedures are documented adequately; program performance meets or exceeds standards; and pre-release data are kept confidential.

Performance Goal 08-1B (OJC)

Improve educational achievements of Job Corps students and increase participation of Job Corps graduates in employment and education.

Indicators, Targets and Results

*Indicator target reached (Y), improved (I), or not reached (N)		PY 2004 Goal Not Achieved	PY 2005 Goal Not Achieved	PY 2006 Goal Not Achieved	PY 2007 Goal Not Achieved	PY 2008 Goal Not Achieved
Percent of participants entering employment or enrolling in post-secondary education or advanced training/occupational skills training in the first quarter after exit	Target	85%	85%	87%	82%	74%
	Result	84%	80%	74%	73%	66%
	*	N	N	N	N	N
Percent of students who attain a GED, high school diploma or certificate by the end of the third quarter after exit	Target	64%	64%	65%	64%	54%
	Result	64%	60%	57%	53%	55%
	*	Y	N	N	N	Y
Percent of students who will achieve literacy or numeracy gains of one Adult Basic Education (ABE) level (approximately equivalent to two grade levels)	Target	45%	45%	58%	58%	54%
	Result	47%	58%	58%	53%	58%
	*	Y	Y	Y	N	Y
Goal Net Cost (millions)		$1,309	$1,402	$1,485	$1,589	$1,640

Source(s): Job Corps Management Information System.
Legacy Data: Some indicators not shown for PY 2004. Complete indicators, targets and results for PY 2004 are available in the FY 2006 report at http://www.dol.gov/_sec/media/reports/annual2006/PGD.htm. See Performance Goal 05-1.1B.
Note: Net Cost reflects the full cost of each program as assigned by DOL entities to the Department's outcome goals less any exchange revenue earned. Full cost consists of (a) both direct and indirect costs, and (b) the costs of identifiable supporting services provided by other segments within DOL and by other Federal agencies. Costs are not allocated to the indicator level for placement, credential and literacy/numeracy measures because program activities are not separable into categories associated with one or another. "Participants" in the first measure above includes graduates and former enrollees, but not "uncommitted" — which are those students in the program for less than 60 days or who exit because of the zero tolerance policy.

Program Perspective and Logic

Job Corps is an intensive educational and vocational training program (primarily residential) for economically disadvantaged youth ages 16 through 24 who often face multiple barriers to gainful employment. This program provides career counseling, technical skills and academic training, social education, and other support services, such as housing, transportation and family support resources to more than 60,000 individuals at 122 centers nationwide. Job Corps centers, ranging in size from 200 to 2,000 students, are located in both urban and rural communities. Job Corps centers provide individually-tailored services to help students achieve the skills and credentials required to be successful, productive citizens and to obtain work opportunities that lead to long-term employment.

In recent years, an increasingly knowledge-based labor market has challenged Job Corps to revise its training strategies. In response, Job Corps developed and implemented a training strategy that focuses on applied knowledgeby increasing the rigor and relevance of academic and career technical training; incorporating industry-based standards and certifications; reinforcing a standards-based curriculum approach; and providing a more comprehensive system of on-center and post-center support services. Industry-based standards, as used in this report, refer to worker performance specifications that have been developed or are being developed by business and industry-based organizations in partnership with Job Corps. These standards define a facet of student performance that is measurable and built on the skills learned as students' progress through the educational system and into the workplace. Job Corps then works to develop curricula based on these standards. This strategy is providing students with valuable credentials and competitive skills they need to pursue promising careers — especially in a period of rising national youth unemployment rates. Performance of the Job Corps program is assessed using the Federal job training program common measures for youth — placement in employment or education, attainment of a degree or certificate, and literacy or numeracy gains — as indicators of student achievement in improving their long-term employability.

Recovery Act

The Recovery Act provided Job Corps with $250 million; close to 85 percent or $211.6 million of this funding will be invested in construction, rehabilitation and acquisition of Job Corps centers. To date, Job Corps has awarded over 50 percent (totaling approximately $118 million) of its construction contracts; as a result of these awards construction work will be performed in 29 states and Puerto Rico. Approximately $10 million of the funds will be used to incorporate green technologies and training into Job Corps' career technical training programs. For more information, see http://www.recovery.gov/?q=content/program-plan&program_id=7714.

Analysis and Future Plans

Job Corps did not achieve its performance goal for PY 2008. Placement of Job Corps graduates and former enrollees in employment, the military or post-secondary education continued to mirror the national employment picture; the placement result was eight points below the 74 percent target. Certificate attainment outcomes, however, reversed a three-year decline and literacy/numeracy gains rose five points. It appears that the strategic focus on academics had a positive impact.

Indicator	*	What worked	What didn't work
Placement in employment or education	N	Placement of Job Corps students into post-secondary education or advanced skill training rose from 11 percent in PY 2007 to 14 percent in PY 2008.	In many areas of the country, the local economic situation made placement in the workforce more difficult. Although the overall placement rate fell from 73 percent to 66 percent in PY 2007-2008, initial wages increased slightly.
Attainment of a degree or certificate	Y	Utilization of quality on-line High School Diploma programs and well- established local partnerships contributed to the increase in credential attainment rates.	Increases in career technical training completions were limited due to the introduction of comprehensive curriculum revisions over the past two years.
Literacy or numeracy gains	Y	The results show that special programs such as STARS (Speakers, Tutors, Achievement, Retention and Success) and after-hours tutoring programs had a positive impact on student literacy and numeracy.	Initial impact from online administration of the Test of Adult Basic Education (TABE) seems to have leveled out.
Program Performance Improvement Plan
To improve employment and earning outcomes, Job Corps is aligning career technical training with industry-based standards and standards-based instruction (organized around industry clusters). Each Region will measure the effectiveness of these program enhancements; final evaluations are expected by September 2010. To better serve the Hispanic community, all Job Corps centers have implemented the English Language Learner's (ELL) Program for students with Limited English Proficiency to ensure meaningful access to and participation in the program. Job Corps is evaluating various English Language Assessment tools and related curricula to ensure ELL students are identified correctly and placed into appropriate ELL courses. Job Corps' current policies regarding ELL students will be revised by June 2010. In PY 2009, Job Corps is working closely with the Departments of Education, Health and Human Services, and Agriculture, to provide training and career pathways in high-growth and emerging industry sectors. Linkages will also be expanded with community colleges, registered apprenticeship programs and workforce investment boards.

*Target reached (Y), improved (I), or not reached (N)

Job Corps net cost increased by 3 percent from PY 2007-2008 due to higher administrative expenses, such as salaries and benefits, and due to a general inflation increase to center, outreach, admission, and career tranisition services operating contracts.

In 2010, Job Corps' activities will contribute to the following outcome goals in support of the Department's Strategic Vision of Good Jobs for Everyone:

• Increasing workers' incomes and narrowing wage and income inequality;
• Assuring skills and knowledge that prepare workers to succeed in a knowledge-based economy, including in high-growth and emerging industry sectors like "green" jobs; and
• Helping workers who are in low-wage jobs or out of the labor market find a path into middle-class jobs.

Program Assessments, Evaluations and Audits

Findings and recommendations from a Job Corps program assessment completed in 2007 prompted specific actions to improve performance. Here is a summary of progress in FY 2009:

• Exploring opportunities to improve cost effectiveness and performance outcomes at the 28 non-DOL owned properties which are operated for Job Corps by the Departments of Agriculture and Interior. In June 2008, Job Corps contracted with an outside accounting firm to audit internal agency procedures for the management of Job Corps funds; the final report was received in March 2009 and Job Corps is addressing the report recommendations. Job Corps continues to reconcile discrepancies in agency material accrual methods; as a result, the Department of Interior (DOI) is fully compliant with reporting and Job Corps is compliant with oversight requirements. Job Corps has requested additional information from the Department of Agriculture (USDA) to facilitate final reconciliation.
• Adopting efficiency measures that are linked to performance outcomes, account for all costs, and facilitate comparisons across Department of Labor training and employment programs. Job Corps will continue to examine additional alternatives for reporting efficiency measures that are linked to performance outcomes and account for all costs.

More information is available at http://www.whitehouse.gov/omb/expectmore/summary/10002372.2007.html.

Independent evaluations and audits completed in FY 2009 are summarized below.

"Job Corps' Reported Performance Measures Did Not Comply With All Legislative Reporting Requirements," March 2009 (Office of Inspector General)
Relevance: The Office of Inspector General (OIG) conducted this performance audit to determine if Job Corps' reported performance measures comply with all legislative reporting requirements.
Findings and Recommendations: Job Corps did not fully comply with Workforce Investment Act reporting requirements. Although Job Corps collected the necessary data, OIG found that the agency did not submit to Congress the WIA-required annual reports that include performance results for each Job Corps center. OIG found that Job Corps did not fully comply with the Office of Management and Budget's reporting requirements of job placement outcomes for all students who left the program. Job Corps only reports placement outcomes for graduates and former enrollees.	Actions: Job Corps will comply with the WIA requirement to submit an annual report to Congress. Job Corps will work with OMB to ensure that participant placement reporting complies with OMB common measure requirements.
Additional Information: The report is available at http://www.oig.dol.gov/public/reports/oa/2009/04-09-003-01-370.pdf.
"Better Targeted Career Training and Improved Pre-Enrollment Information Could Enhance Female Residential Student Recruitment and Retention," June 2009 (GAO)
Relevance: The Government Accountability Office (GAO) conducted this audit to review the extent to which Job Corps centers are operating at or near capacity for residential students, particularly female residential students.
Findings and Recommendations: Because much of the program's costs are fixed, GAO concluded that program efficiency is compromised when Job Corps centers operate under capacity — and represents a missed opportunity to train students who might benefit from the program. While the program nearly achieves its planned enrollment for males, it is struggling to meet female enrollment targets. Officials at all levels of Job Corps affirmed the need for students to have, prior to enrolling in the program, a clear understanding of what it would be like to live and train at a center. GAO believes that absent additional steps, Job Corps will likely continue to face difficulty in recruiting and retaining students, particularly female students.	Actions: Job Corps acknowledged the need to offer career training programs to increase female enrollment. However, Job Corps noted that in selecting new offerings, it routinely looks beyond occupations traditionally dominated by females as they seek to maximize opportunities that may result in long-term self-sufficiency. GAO concurs but continues to recommend a more systematic assessment of career training curricula, particularly at centers with low female enrollment. To increase recruitment and retention of students, Job Corps is producing a number of resources that provide an overview of the program and highlight information about individual centers. Also, Job Corps has established new Outreach and Admission performance-based service contracting procedures, created new recruitment materials, and launched a new recruitment Web site.
Additional Information: The report is available at http://www.gao.gov/new.items/d09470.pdf.

Data Quality and Top Management Challenges

Data quality for this performance goal is rated Excellent.¹⁸ When the OIG cited insufficient management controls over performance data in 2004, DOL designed and implemented new data integrity procedures that require regional offices to utilize targeted samples (highlighting where error or manipulation may have occurred) for audit reviews conducted in conjunction with on-site assessments. When Regional Offices find that Center operators have not complied with contractual obligations, liquidated damages are identified and assessed. Since 2004, $731,904 in liquidated damages has been recovered. The Office of Job Corps is confident that the new data integrity strategy is producing more reliable student outcome data from Job Corps centers and career transition service providers.

To address Ensuring the Effectiveness of the Job Corps Program (see the Top Management Challenges in the Other Accompanying Information section), Job Corps' Regional Offices have improved the way they monitor the performance of outreach and admissions, center and career transition providers. Regional Offices provide oversight, perform desk reviews and conduct formal assessments, including rigorous data integrity audits, of each contract at least once every 24 months. To improve student health and safety, Job Corps uses a comprehensive communication network of safety advisories and conducts multiple annual safety inspections and reviews. This past fiscal year, the program achieved 100 percent timely filing of safety injury claims. Also, to improve student safety, centers continue to separate students who violated the program's strict Zero Tolerance policy (for drugs and violence).

Performance Goal 08-1C (ETA)

Increase placements and educational attainments for youth served through the WIA Youth program.

Indicators, Targets and Results

*Indicator target reached (Y), improved (I), or not reached (N) **Estimated		PY 2005 Goal Achieved	PY 2006 Goal Achieved	PY 2007 Goal Achieved	PY 2008 Goal Achieved
Percent of participants entering employment or enrolling in post-secondary education or advanced training/occupational skills training in the first quarter after exit	Target	baseline	60%	61%	59.4%
	Result	57.8%	60%	62%	66.7%**
	*	Y	Y	Y	Y
Percent of students who attain a GED, high school diploma or certificate by the end of the third quarter after exit	Target	baseline	40%	45%	50.9%
	Result	36%	44%	57%	58.2%**
	*	Y	Y	Y	Y
Percent of students who achieve literacy or numeracy gains of one Adult Basic Education (ABE) level (approximately equivalent to two grade levels)	Target	—	baseline	baseline	23.2%
	Result	—	—	30%	38.6%**
	*	—	—	—	Y
Goal Net Cost (millions)		$1,017	$866	$966	$1,125

Source(s): Annual State WIA performance reports (ETA-9091) and quarterly reports (ETA-9090).
Note: Net Cost reflects the full cost of each program as assigned by DOL entities to the Department's outcome goals less any exchange revenue earned. Full cost consists of (a) both direct and indirect costs, and (b) the costs of identifiable supporting services provided by other segments within DOL and by other Federal agencies. Costs are not allocated to the indicator level for placement, credential and literacy/numeracy measures because program activities are not separable into categories associated with one or another. This goal was reported Substantially Achieved in the FY 2007 PAR based on estimated data.

Program Perspective and Logic

The Workforce Investment Act authorizes services to low-income youth (age 14-21) with barriers to employment. The program serves both in- and out-of-school youth, including youth with disabilities and other youth who may require specialized assistance to complete an educational program or to secure and hold employment. Youth participants are prepared for employment and post-secondary education by stressing linkages between academic and occupational learning. Services available to youth include tutoring, alternative secondary school offerings, summer employment, occupational skill training, paid and unpaid work experience, leadership development opportunities, mentoring, comprehensive guidance and counseling, supportive services such as assistance with child care and housing, and follow-up services such as on-going career counseling after a youth exits the program.

DOL collects data for three performance indicators, the Federal job training program common performance measures for youth, that enable the program to describe and compare the outcomes of its core purposes to other education, employment and job training programs focusing on youth. The first measure, percent of youth participants who are in employment or enrolled in post-secondary education or training, indicates whether DOL is transitioning youth into the workforce or post-secondary education — a key to successful careers. The second, percent of participants who attain a diploma, GED, or certificate, is a proxy for the effectiveness of the program in preparing youth participants for high-demand occupations. The third indicator measures literacy/numeracy gains by out-of-school youth participants who are deficient in basic skills.

Recovery Act

The Recovery Act made an additional $1.2 billion available for WIA Youth activities, using the same formula as the regular appropriation. Formula funds were distributed to States and local workforce investment boards based on the three factors required for WIA — the number of unemployment for Areas of Substantial Unemployment (ASUs), the higher number of either excess unemployment individuals or ASU excess, and the number of economically disadvantaged youth. The grantees, using these funds, provided additional authorized WIA Youth activities with a specific focus on creating summer employment opportunities for youth. The Recovery Act also extended the youth eligibility age from 21 to 24 and encourages local areas to expose participating youth to opportunities in green jobs in the construction, energy efficiency, renewable energy, and other related industries.

Three measures will be used to assess the effectiveness of WIA Youth Recovery Act funds at providing summer employment opportunities for youth: the work readiness indicator, the number of participants placed in summer employment, and the summer youth completion rate. DOL will utilize the three youth common performance measures to assess long-term outcomes for youth who receive other services funded by the Recovery Act. ETA is also conducting a process evaluation of youth activities funded by the Recovery Act, with a focus on summer employment. The evaluation will incorporate both qualitative and quantitative analysis of the infusion of Recovery Act funds into programs for youth. The study is based on a selected sample of local workforce investment areas. For more information, see http://www.recovery.gov/?q=content/program-plan&program_id=7666.

---

Andrea is proud of her progress and grateful for Virginia’s Middle College program, which allows individuals lacking a high school degree to pursue a high school credential, community college education and workforce certification in a college environment. At first, she was nervous. She had dropped out of high school just two credits shy of graduating to take care of her child. Andrea soon realized that her opportunities were limited without an education, so she explored the options at a local community college. After two months of preparation classes, Andrea earned her high school credential. She quickly found full-time employment and continues her studies at the community college. “I couldn’t have got a job without a high school credential. Earning great scores made me feel proud and confident. When I went in to sign up for my college classes, I wasn’t afraid or intimidated. I belong in college.” [Photo credit: Randy Holmes]

---

Analysis and Future Plans

The WIA Youth program achieved its performance goal for PY 2008. The program exceeded its placement in employment or education target by 7.3 percentage points, surpassed the degree or certificate attainment target by 7.3 percentage points, and exceeded the literacy or numeracy gains target by 15.4 percentage points. In PY 2008, DOL continued its Shared Youth Vision (SYV) collaborative efforts to assist States in coordinating resources and program delivery strategies to achieve positive outcomes for the youth most in need. The SYV Federal Partnership includes the Departments of Health and Human Services, Education, Justice, Housing and Urban Development, Transporation, Agriculture, plus the Social Security Administration and the Corporation for National and Community Service. DOL and the SYV Federal Partnership have continued to support 32 Shared Youth Vision cross-agency state teams in their collaborative efforts to better serve the youth most in need through monthly community of practice calls, a Federal Partners' Solutions Desk, and peer to peer forums.

Indicator	*	What worked	What didn't work
Placement in employment or education	Y	Collaborative efforts with partners that included providing States a self-assessment tool to facilitate state-level coordination. Adopted a statistical model to establish targets that account for external factors that impact program performance.	As stated in the GAO report 08-013 (discussed in more detail below), many local workforce areas are utilizing performance-based contracts that result in disincentives to serve the most at-risk youth and still achieve performance targets.
Attainment of a degree or certificate	Y
Literacy or numeracy gains	Y
Program Performance Improvement Plan
DOL will assist States and local areas, through the issuance of guidance, on how to structure service provider contracts to allow WIA Youth programs to successfully serve youth most at-risk.

*Target reached (Y), improved (I), or not reached (N)

Costs associated with this performance goal rose by seventeen percent from PY 2007 to PY 2008, reflecting fluctuations in the timing of expenditures and the number of participants served. In addition, increased costs in PY 2008 reflect expenditures in the last quarter that can be attributed to the expansion of summer employment opportunities under the Recovery Act.

In 2010, WIA Youth Program's activities will contribute to the following outcome goals in support of the Department's Strategic Vision of Good Jobs for Everyone:

• Increasing workers' incomes and narrowing wage and income inequality;
• Assuring skills and knowledge that prepare workers to succeed in a knowledge-based economy, including in high-growth and emerging industry sectors like "green" jobs; and
• Helping workers who are in low-wage jobs or out of the labor market find a path into middle-class jobs.

Program Assessments, Program Evaluations and Audits

Findings and recommendations from a WIA Youth Program assessment completed in 2008 prompted specific actions to improve performance. Here is a summary of progress in FY 2009:

• Conducting an evaluation to determine WIA services' impact on employment and earnings outcomes for participants. ETA convened a Peer Review Board of independent researchers and stakeholders to review and provide comments on the design of the Workforce Investment Act Gold Standard Evaluation (WGSE). The review provided useful input on the contractor's evaluation design. ETA is continuing to refine the design and hopes to begin implementation of the evaluation in FY 2010. Also, as discussed above, ETA is conducting a process evaluation of youth activities funded by the Recovery Act, with a focus on summer employment.
• Implementing efficiency measures that are linked to performance outcomes, account for all costs, and facilitate comparisons across Department of Labor training and employment program. An independent study completed in PY 2008 recommended outcome-based efficiency measures for which the Department is collecting baseline data. These data will inform the selection of appropriate measures for each program. Additional research on setting performance standards and targets will be completed by December 2009.
• Strengthening the quality of youth performance data, specifically supplemental data and administrative records, through data validation. In May 2009, DOL issued revised guidance and an update to the Data Reporting and Validation Software. These updates will allow all States to submit timely and accurate youth performance data. Regional Office staff will continue to monitor the States' validation of youth performance data during periodic state monitoring reviews.

More information is available at http://www.whitehouse.gov/omb/expectmore/summary/10000342.2008.html.

Independent evaluations and audits completed in FY 2009 are summarized below.

"Disconnected Youth: Federal Action Could Address Some of the Challenges Faced by Local Programs that Reconnect Youth to Education and Employment," February 2008 (GAO)
Relevance: GAO examined 39 local youth programs (15 of which are WIA-funded) to learn how Federal agencies are helping them address the challenges they face in serving youth who have been disconnected from education and employment.
Findings and Recommendations: GAO found that Workforce Investment Board (WIB) contracts with local service providers often require that providers achieve performance goals for participants in one-year time frames. This is very challenging for programs serving the most at-risk youth, who may need more time to obtain skills. Funding in subsequent years is often contingent on meeting performance goals, making the contract structure a disincentive to serving these youth. GAO recommended that DOL work with States and WIBs to better ensure they have the information and guidance needed to develop and implement more effective contracts to allow local programs to serve youth most in need of assistance while still achieving performance goals.	Actions: In 2008, DOL convened WIA State, local, and youth service providers to get a better understanding of contracting issues and gather local examples of contracts that have the components and the flexibility to successfully serve the youth most in need. Using input from that discussion, DOL will issue guidance to the workforce system in the fall of 2009 including specific examples of ways to develop contracts with local service providers that allow them to successfully serve youth at varying skill levels.
Additional Information: The report is available at http://www.gao.gov/new.items/d08313.pdf.
"GAO Recovery Act: Funds Continue to Provide Fiscal Relief to States and Localities, while Accountability and Reporting Challenges Need to Be Fully Addressed," September 2009 (GAO)
Relevance: Reviews were conducted in ten states to understand the effect of the implementation of the Recovery Act funds specifically targeted for the Workforce Investment Act Youth program and summer youth employment activities.
Findings and Recommendations: States and localities reported difficulties in responding to the Recovery Act's focus on green industries without a clear standard of what constitutes a green job. DOL must provide additional guidance about the nature of these jobs and strategies that could be used to prepare youth for careers in green industries, to better support state and local efforts with employment and training in green jobs. Flexibilities given to States and local areas on measuring work readiness — the sole indicator of summer activities — lacks comparability and provides little understanding about actual individual outcomes and program achievement. DOL must provide additional guidance on how to measure the work readiness of youth, with a goal of improving the comparability and rigor of the measure.	Next Steps: The Department plans to leverage the results of the Recovery-Act funded competitive grants for green job training to provide insights on delivering services to youth, and others, along green career pathways. The Bureau of Labor Statistics is developing a definition for green sectors and green jobs that will inform state and local workforce development efforts to identify and target green jobs and their training needs. Technical assistance forums will be held in late 2009 to focus on strategies to prepare out-of-school youth for careers in green sectors and industries; and for implementing services with remaining Recovery Act funds. An evaluation of the WIA Youth Program was initiated to better understand issues and challenges; and to gather lessons learned related to measuring work readiness. Methodologies, used this summer, to measure work readiness are being assessed and further refinement of the work readiness indicator is planned. Should significant summer activities be continued in the future, then further guidance that provides for more consistent and meaningful data and reporting will be issued.
Additional Information: The report is available at http://www.gao.gov/new.items/d091016.pdf

Data Quality and Top Management Challenges

The data quality rating for this performance goal also applies to Performance Goals 2A and 2B. These goals rely on the same data collection system to determine employment outcomes for WIA program participants. Data quality for these performance goals are rated Very Good.¹⁹ There are challenges with data validation of the WIA outcomes, including maintenance and updating of the data validation software, providing State staff with needed technical assistance — especially given high turnover of staff at the Federal and State levels — and monitoring reviews by ETA regional office staff of State implementation of the data validation initiative. However, ETA made progress in expanding use of its data validation system for monitoring data quality at both the national and regional levels (see also Improving Performance Accountability of Grants, which is one of the Top Management Challenges in the Other Accompanying Information section). All States comply with ETA's requirement to validate the outcome data they submit annually in the WIA Annual Reports, and the majority use ETA's data validation software to verify the outcome data they submit annually. States are required to provide report validation results by October 1 and data element validation results by February 1 following each program year. DOL also includes data report validation results as one of the criteria for determining eligibility for WIA incentive awards and those States subject to sanctions for poor performance.

Performance Goal 09-1D (ETA)

Improve the registered apprenticeship system to meet the training needs of business and workers in the 21st Century.

Indicators, Targets and Results

*Indicator target reached (Y), improved (I), or not reached (N) **Estimated		FY 2005 Goal Achieved	FY 2006 Goal Achieved	FY 2007 Goal Achieved	FY 2008 Goal Not Achieved	FY 2009 Goal Not Achieved
Percent of those employed nine months after registration as an apprentice	Target	baseline	78%	79%	84%	81.6%
	Result	78%	82%	83%	84%	76.8%**
	*	Y	Y	Y	Y	N
Average hourly wage gain for tracked entrants employed in the first quarter after registration and still employed nine months later	Target	baseline	$1.26	$1.33	$1.51	$0.61
	Result	$1.26	$1.32	$1.50	$0.61	$0.17**
	*	Y	Y	Y	N	N
Percent of participants employed in the first quarter after exit	Target	—	—	—	—	baseline
	Result	—	—	—	—	73.9%**
	*	—	—	—	—	—
Percent of participants employed in the first quarter after exit still employed in the second and third quarters after exit	Target	—	—	—	—	baseline
	Result	—	—	—	—	87.2%**
	*	—	—	—	—	—
Average earnings in the second and third quarters after exit (six months' earnings)	Target	—	—	—	—	baseline
	Result	—	—	—	—	$18,967**
	*	—	—	—	—	—
Goal Net Cost (millions)		$23	$25	$24	$25	$25

Source: Registered Apprenticeship Partners Information Data System (RAPIDS).
Legacy Data: One indicator for FY 2005 was dropped; it is not included in this table. Complete indicators, targets, and results for FY 2005 are available in the FY 2006 report at http://www.dol.gov/_sec/media/reports/annual2006/PGD.htm. See Performance Goal 06-1.1A.
Note: Net Cost reflects the full cost of each program as assigned by DOL entities to the Department's outcome goals less any exchange revenue earned. Full cost consists of (a) both direct and indirect costs, and (b) the costs of identifiable supporting services provided by other segments within DOL and by other Federal agencies. "Tracked entrants" is defined as the cohort of apprentices registered and entered into RAPIDS during a given reporting period. The 25 States that have Federally-registered apprenticeship programs enter data on individuals into the system. Costs are not allocated to the indicator level for employment, retention and earnings measures because program activities are not separable into categories associated with one or the other.

Program Perspective and Logic

Established in 1937, the National Registered Apprenticeship System is a partnership of the Department of Labor, State agencies, sponsors, industry leaders, employers, employer associations, labor and management organizations, and educational institutions. It provides opportunities for jobseekers to secure jobs with career paths, earn competitive wages, and obtain nationally-recognized industry credentials. The apprenticeship training system promotes and registers programs and apprentices, certifies standards, safeguards the welfare of the apprentices, and provides a nationally recognized system for skilled and technical occupational training programs throughout the United States. Apprenticeship programs use a combination of classroom training and on-the-job learning, under close supervision of a skilled worker, to teach apprentices the practical and theoretical requirements of a highly skilled occupation. Most of the training costs are borne by apprenticeship program sponsors.

The Department promotes the Registered Apprenticeship System to potential sponsors and participants and registers and monitors these partners' apprenticeship programs, in some cases via State Apprenticeship Agencies (SAAs), which are recognized by the Secretary of Labor. SAAs certify that standards are met for quality, fairness, and equal opportunity, and that apprenticeship programs incorporate appropriately supervised on-the-job learning and occupation-related technical instruction.

As a system based on voluntary industry and employer participation, apprenticeship program performance is directly impacted by external factors such as the wage rates determined by local apprenticeship sponsors and by the demand for skilled and technical labor in local markets. Two performance indicators, apprentices employed nine months after registration and average hourly wage gain for tracked entrants, are used to indicate progress towards program completion and the value of skills obtained. In FY 2009, the Office of Apprenticeship (OA) is collecting baseline data for transition to the Federal job training program common measures. Common measures (entered employment rate, employment retention rate, and average earnings) enable comparisons to be made to education, employment, and job training programs that share similar purposes.

---

Rob, an exceptional employee of the City of San Diego’s Communications Division, is participating in a four-year Apprentice Program that produces highly qualified journey-level Communications Technicians. The Communications Division is responsible for design, installation and maintenance of all wireless communications systems and services for the city. Their primary customers include the Police and Fire-Rescue Departments. Rob has maintained a perfect grade point average and mentored several other apprentices using design software called Circuit Wizard. He has also designed, built, and integrated with the existing system real time sensors for remote mountain top sites. The apprenticeship has provided a good opportunity for Rob to use his existing skills and to gain knowledge of wireless communication concepts and systems. He has also enhanced his value to the city’s workforce. [Photo credit: DOL/ETA]

---

Analysis and Future Plans

The National Registered Apprenticeship System did not achieve its performance goal for FY 2009. The retention result of 76.8 percent was almost five points below target and the 17-cent average hourly wage gain was 44 cents below target. Apprenticeships are tied to employment and therefore have declined in number with the nation's widespread job losses over the past year — especially since a large majority of apprentices are in the construction and manufacturing industries, both of which have been severely affected by the recession. However, based on preliminary Common Results Information System (CRIS) data, wages for apprentice exiters are more than 26 percent above other selected ETA programs' exiters (WIA Adults and Dislocated Workers, and Trade Adjustment Assistance).

For FY 2010, the program will establish performance targets for the new common measures. Over the next year, OA staff will work with SAAs to ensure compliance with the revised Federal regulations published in October 2008. The revised regulations will advance the National Registered Apprenticeship System by providing new options and increased flexibility, promoting Registered Apprenticeship as a workforce development strategy for today's regional economies, strengthening consistency and continuity across the system, and establishing a consistent framework to promote improved performance outcomes and quality.

Indicator	*	What worked	What didn't work
Retention in apprenticeship	N	Linkages with the workforce investment system to provide apprentices with supportive services and other services to help reduce non-completion. Activities to integrate apprenticeship on the state level through three Regional Action Clinics held in Regions 4, 5, and 6 proved successful.	Lack of work projects, employment cutbacks, and wage freezes negatively impacted results.
Average hourly wage gain	N	Employers hiring new apprentices offered higher than expected starting wages.	See Retention indicator above.
Program Performance Improvement Plan
Continue the technical assistance strategy to better align and integrate Registered Apprenticeship with the workforce investment and education systems. Support collaboration and partnerships as well as prepare and refer One-Stop job seekers to Registered Apprenticeship. Play a key role in bringing about collaboration and change among the systems creating opportunities to support apprentices and outreach to new businesses. Hold additional Regional Action Clinics on the East Coast in early FY 1010. Action Clinics promote further collaboration and partnerships between the education, workforce and Registered Apprenticeship systems; and leverage Registered Apprenticeship as a key workforce development strategy in the public workforce system.

*Target reached (Y), improved (I), or not reached (N)

Net costs for Apprenticeship activities were virtually unchanged from FY 2008-2009.

In 2010, the Office of Apprenticeship's activities will contribute to the following outcome goals in support of the Department's Strategic Vision of Good Jobs for Everyone:

• Increasing workers' incomes and narrowing wage and income inequality; and
• Helping workers who are in low-wage jobs or out of the labor market find a path into middle-class jobs.

Program Assessments, Evaluations and Audits

Findings and recommendations from a Job Training Apprenticeship program assessment completed in 2005 prompted specific actions to improve performance. Here is a summary of progress in FY 2009:

• Implementing the common measures for earnings and retention and establishing an Internet-based apprenticeship registration system to efficiently obtain comprehensive performance data. The Registered Apprenticeship Partners Information Data System (RAPIDS) debuted in November 2007. Phase 2 of RAPIDS is scheduled for completion in FY 2009 and will allow SAAs and sponsors to upload data electronically.
• Evaluating and reporting participants' employment and earnings after they leave the program to compare apprenticeship program outcomes with those of other training models. During the FY 2009 transition period, OA reported on the common measure results for apprenticeship completers extracted from RAPIDS and tracked through the Common Results Information System.
• Addressing underrepresentation of women in apprenticeship programs through a reinvigorated Equal Employment Opportunity review process and tracking and reporting performance. The economic downturn and its impact on the construction industry have led to less than favorable results from the six Women in Apprenticeship and Non-Traditional Occupations (WANTO) grants awarded in FY 2006 and 2007. The lack of hiring in the construction industry affected the grantees' ability to place participants in jobs. However, the six grantees have established the capacity to prepare women participants to enter construction careers, and they are ideally positioned to leverage additional resources as the Recovery Act investments begin to generate more employment opportunities for women in nontraditional occupations. Also, in FY 2010, the Department will be funding an evaluation to gain insight on the impediments impacting and strategies to increase the recruitment, enrollment, and retention of women in Registered Apprenticeship programs.
• Implementing efficiency measures that are linked to performance outcomes, account for all costs, and facilitate comparisons across Department of Labor training and employment programs. An independent study completed in PY 2008 recommended outcome-based efficiency measures for which the Department is collecting baseline data. These data will inform the selection of appropriate measures for each program. Additional research on setting performance standards and targets will be completed by December 2009.

More information is available at http://www.whitehouse.gov/omb/expectmore/summary/10003901.2005.html.

Independent evaluations and audits completed in FY 2009 are summarized below.

"The Benefits and Challenges of Registered Apprenticeship: The Sponsors Perspective," March 2009 (Urban Institute and American University)
Relevance: Sponsors of registered apprenticeships were surveyed to determine the effectiveness of apprenticeships and identify program operation improvements for the Registered Apprenticeship System.
Findings and Recommendations: Strong support from current Registered Apprenticeship sponsors exists; and sponsors are likely to be helpful in efforts to promote Registered Apprenticeship. Sponsors' interactions with the workforce investment system were limited. One-Stop Career Centers and unions were the least frequently identified sources for recruitment. Sponsors want help recruiting and screening applicants, finding related instruction, and designing competency-based programs. SAAs are "good" or "excellent" in promoting and publicizing registered apprenticeship.		Actions: OA will continue to expand efforts to engage the workforce investment system to provide registered apprentices with support services available through the One-Stop Career Centers. OA will encourage program sponsors to partner with the workforce investment system to access resources available to assist in screening applicants and identifying sources of related instruction. OA will continue outreach and technical assistance efforts to promote advantages of competency-based programs.
Additional Information: The final report can be found at http://wdr.doleta.gov/research/.
"Registered Apprenticeship: Findings from Site Visits to Five States," November 2008 (Planmatics, Inc.)
Relevance: Site visits were conducted in five states with registered apprenticeship sponsors, apprentices, SAA staff, One-Stop Career Center managers, and related instruction providers to better understand current issues in Registered Apprenticeship.
Findings and Recommendations: Registered apprenticeship enjoyed strong support from the sponsors and apprentices interviewed and most sponsors stated that they would strongly recommend registered apprenticeship to other employers. Sponsors noted some difficulty in finding high-quality related instruction. Coordination and linkage between registered apprenticeship and the One-Stop Career Center System appeared to be limited. Recommendations suggested by program sponsors included increased promotion of Registered Apprenticeship, reduced paperwork, more support from other programs in the workforce investment system, and increased monitoring of program quality.	Actions: In 2007 — 2009, ETA issued policy guidance and hosted three regional Action Clinics to promote further collaboration and partnerships between the education, workforce and Registered Apprenticeship systems. OA will continue these efforts to link program sponsors with resources available through One-Stop Career Centers. Completion of RAPIDS design and upgrades will streamline paperwork required of program sponsors. Implementation of revised regulatory framework, which includes a new section on performance standards, will support increased program quality and enhanced accountability.
Additional Information: The final report can be found at http://wdr.doleta.gov/research/. See ETAOP 2009-02.

Data Quality and Top Management Challenges

Data quality for the Apprenticeship program is rated Fair.²⁰ All Federally-administered programs (in 25 States) are using RAPIDS, which automated data entry and established greater quality controls for data entered into the system by program sponsors. System modifications to implement common measures for earnings and retention data have been completed, and an internet-based format has been successfully implemented to capture performance data. Unemployment insurance wage records are matched with the information collected in RAPIDS. This process allows OA to routinely monitor the data and conduct automatic data error checks. Quality assessments and Equal Employment Opportunity compliance reviews have been completed on 90 percent of programs with five or more apprentices within a five-year cycle established in FY 2005. However, apprentices' employment and wage data provided by employers cannot be verified.

RAPIDS implementation also addresses an ongoing Departmental commitment to improve data quality by developing a cost-effective strategy for collection of data from programs in 25 States and three territories that are not administered by the Department. RAPIDS has undergone system application verification testing that found RAPIDS provides real-time data through ad-hoc and quarterly reports, and that the system is effective in identifying data errors. Results are used to provide evidence that the system logic is functioning correctly and supports rationale for current or future system changes. Further enhancements are scheduled in FY 2009 that will enable sponsors and the 25 SAAs to participate directly in RAPIDS or provide the option to electronically upload apprenticeship data into RAPIDS.

Performance Goal 08-1E (VETS)

Improve employment outcomes for veterans who receive One-Stop Career Center services.

Indicators, Targets and Results

*Indicator target reached (Y), improved (I), or not reached (N)		PY 2004 Goal Achieved	PY 2005 Goal Achieved	PY 2006 Goal Not Achieved	PY 2007 Goal Achieved	PY 2008 Goal Not Achieved
Percent of Veteran participants employed in the first quarter after exit	Target	58%	59%	60%	61%	62.5%
	Result	60%	62%	60%	62.5%	58.7%
	*	Y	Y	Y	Y	N
Percent of Veteran participants employed in the first quarter after exit still employed in the second and third quarters after exit	Target	80%	81%	81%	80%	82.0%
	Result	81%	81%	79%	81.8%	80.6%
	*	Y	Y	N	Y	N
	Cost	—	$89	$91	$93	$109
Percent of Disabled Veteran participants employed in the first quarter after exit	Target	54%	55%	55%	56%	58.5%
	Result	56%	57%	55%	58.4%	55.8%
	*	Y	Y	Y	Y	N
Percent of Disabled Veteran participants employed in the first quarter after exit still employed in the second and third quarters after exit	Target	78%	79%	79%	79%	81.0%
	Result	79%	80%	78%	81.1%	79.7%
	*	Y	Y	N	Y	N
	Cost	—	$89	$91	$95	$109
Goal Net Cost (millions)		—	$182	$181	$190	$219

Source(s): Quarterly Labor Exchange Reporting System reports included in the Enterprise Business Support System and Unemployment Insurance records.
Legacy Data: Some indicators not shown for PY 2004-07. Complete indicators, targets and results for PY 2004-07 are available in the FY 2008 report at http://www.dol.gov/_sec/media/reports/annual2008/SG1.htm. See Performance Goal 07-1D.
Note: Net Cost reflects the full cost of each program as assigned by DOL entities to the Department's outcome goals less any exchange revenue earned. Full cost consists of (a) both direct and indirect costs, and (b) the costs of identifiable supporting services provided by other segments within DOL and by other Federal agencies. Costs are not allocated to the indicator level for employment and retention measures because program activities are not separable into categories associated with one or another of them. However, this goal includes two distinct target populations. Costs for each group (all veterans and disabled veterans) are provided in the cost cell opposite the retention indicators, where available.

Program Perspective and Logic

Jobs for Veterans State grants support the delivery of employment services needed by veterans and transitioning service members to promote their success in the civilian workforce. These grants support approximately 2,000 disabled veterans' outreach specialists and local veterans' employment representatives stationed at the nationwide network of nearly 3,000 comprehensive and affiliate One-Stop Career Centers. These staff serve as experts on workforce resources available for veterans. The local representatives emphasize the provision of services for recently separated veterans and handle outreach to employers, while the outreach specialists focus their efforts on intensive services for disabled veterans and other veterans with significant barriers to employment.

One-Stop Career Centers serve younger, recently separated veterans who have limited civilian work experience and older veterans with civilian experience who have become unemployed. DOL applies the Federal job training program common measure definitions of entry to employment and retention in employment as the critical indicators of successful outcomes for all veterans and all disabled veterans who receive One-Stop services. During PY 2008, VETS is collecting baseline data on average earnings; in PY 2009, those two indicators (one for all veterans and another for disabled veterans) are being added to the four employment and retention performance indicators.

Analysis and Future Plans

VETS did not achieve this performance goal for PY 2008. That shortfall and the decline in measured outcomes between PY 2007 and PY 2008 are attributable to increases in the national unemployment rates, which were already rising at the outset of PY 2008 and accelerated in the second quarter. For PY 2009, VETS has revised its performance targets to reflect the Administration's economic assumption of continued high unemployment during PY 2009.

In response to the impact of deteriorating economic conditions on veteran participants' outcomes, VETS implemented two initiatives during PY 2008 (see the Improvement Plan in the table below).

Net costs for activities under VETS' State Grants program increased by 15 percent from PY 2007-2008. The appropriations for this program annually include a provision that permits State grantees to expend during the first quarter of a fiscal year any residual funding obligated by the Department during the prior fiscal year. As the current economic recession began to take hold, some States' cost containment efforts led to delays in filling funded positions that had become vacant through normal staff turnover — pushing expenditure of funds obligated by DOL in the prior year into this reporting period.

Indicator	*	What worked	What didn't work
Entered employment rate for all veterans	N	Ongoing delivery of services to veterans prevented even steeper declines in measured performance.	Employment services referring veterans to available job openings were less effective due to the shortage of openings.
Employment retention rate for all veterans	N	The relatively modest decline in the retention rate suggests that, once employed, veterans had significant success maintaining that status.	Involuntary terminations dictated by economic conditions inevitably resulted in the failure of some veterans to retain employment.
Entered employment rate for disabled veterans	N	Disabled veterans experienced a less drastic decline in entry to employment than all veterans.	Economic conditions added a further barrier to disabled veterans seeking employment.
Employment retention rate for disabled veterans	N	Disabled veterans experienced a decline in retention that was only slightly higher than for all veterans.	Economic conditions likely contributed to the consistent decline in retention.
Program Performance Improvement Plan
Consistent with new regulations regarding priority of service for veterans (effective January 2009), DOL is encouraging program operators in the public workforce system to use a significant share of the additional Recovery Act funds to more aggressively deliver the full range of workforce services to veterans. The Recovery Act provided new authority to provide incentives for employers to hire veterans under the Work Opportunity Tax Credit (WOTC). ETA intends to delegate to the state workforce professionals funded by VETS the authority to certify job applicants' veteran status — thus qualifying employers who hire them to receive the tax credit.

*Target reached (Y), improved (I), or not reached (N)
In 2010, VETS' activities will contribute to the following outcome goals in support of the Department's Strategic Vision of Good Jobs for Everyone:

• Increasing workers' incomes and narrowing wage and income inequality;
• Assuring skills and knowledge that prepare workers to succeed in a knowledge-based economy, including in high-growth and emerging industry sectors like "green" jobs;
• Breaking down barriers to fair and diverse workplaces so that every worker's contribution is respected;
• Providing workplace flexibility for family and personal care-giving;
• Facilitating return to work for workers experiencing workplace injuries or illnesses who are able to work and sufficient income and medical care for those who are unable to work;
• Helping workers who are in low-wage jobs or out of the labor market find a path into middle-class jobs; and
• Helping middle-class families remain in the middle-class.

Program Assessments, Program Evaluations and Audits
Findings and recommendations from a Veterans' Employment and Training State Grants assessment completed in 2005 prompted specific actions to improve performance. Here is a summary of progress in FY 2009:

• Implementing recommendations from follow-up evaluation to assess veteran outcomes. The study is now scheduled for completion in December 2009. VETS will review the study's findings and begin implementation of recommendations in the second and third quarters of FY 2010.
• Improving the integration of services between the Jobs for Veterans State Grants and Veterans' Administration's (VA) Vocational Rehabilitation and Employment (VR&E) program. Interagency work groups are coordinating service delivery and reporting procedures. The comprehensive reporting envisioned is dependent on implementation of a "next generation" individual record based system.
• Streamlining and upgrading grants management and reporting for the Jobs for Veterans State Grants. The forms required to implement the streamlined procedures have been submitted for review and approval by the Office of Management and Budget. The current schedule calls for implementation to occur during the second half of PY 2009.

More information is available at http://www.whitehouse.gov/omb/expectmore/summary/10003907.2005.html.

Data Quality and Top Management Challenges

Data for this performance goal, rated Very Good by the Department's criteria,²¹ are tracked using the reporting system for One-Stop Career Centers (Performance Goal 08-2C). Therefore, the data quality assessment for that goal also applies to VETS' indicators. VETS has no DOL top management challenges.

Footnotes

¹⁴The three measures, also referred to as placement, credential attainment and literacy/numeracy, are: Percent of participants entering employment or enrolling in post-secondary education or advanced training/occupational skills training in the first quarter after exit; Percent of students who attain a GED, high school diploma or certificate by the end of the third quarter after exit; and Percent of students who will achieve literacy or numeracy gains of one Adult Basic Education (ABE) level (approximately equivalent to two grade levels).
¹⁵Rows labeled "Dollars not associated with indicators" indicate costs that cannot be associated with the current set of performance indicators. For some goals, indicator costs are intentionally combined by merging cells because program activities are not separable into categories associated with one or another of them (e.g., job training program common measures — entered employment, employment retention and average earnings).
¹⁶Net Cost reflects the full cost of each program as assigned by DOL entities to the Department's outcome goals less any exchange revenue earned. Full cost consists of (a) both direct and indirect costs, and (b) the costs of identifiable supporting services provided by other segments within DOL and by other Federal agencies. Sums may not equal higher level totals due to rounding.
¹⁷Information on DOL's Data Quality Assessments, conducted annually for each performance goal, can be found in the Introduction to the Performance Section.
¹⁸Information on DOL's Data Quality Assessments, conducted annually for each performance goal, can be found in the Introduction to the Performance Section.
¹⁹Information on DOL's Data Quality Assessments, conducted annually for each performance goal, can be found in the Introduction to the Performance Section.
²⁰Information on DOL's Data Quality Assessments, conducted annually for each performance goal, can be found in the Introduction to the Performance Section.
²¹Information on DOL's Data Quality Assessments, conducted annually for each performance goal, can be found in the Introduction to the Performance Section.