Detailed Information on the
Job Corps Assessment

Improving cost effectiveness by assigning students to centers based on their career objectives rather than geographic location.

Adopting efficiency measures that are linked to performance outcomes, account for all costs, and facilitate comparisons across Department of Labor training and employment programs.

Reducing the proportion of program participants who fail to complete the program.

Improve the employment and earnings outcomes of Hispanic and 18 and 19-year-old enrollees and reduce the share of enrollees (currently half) who fail to complete the program.

Improve program efficiency by, for example, using non-residential centers for a larger portion of the population and increasing employer and provider cost sharing.

Implement the following new initiatives to improve the program's results: Career Success Standards; Speakers, Tutors, Achievement, Retention, and Success (STARS); and Pre-enrollment Drug Screening.

Exploring opportunities to improve cost effectiveness and performance outcomes at the 28 non-DOL owned properties which are operated for Job Corps by the Departments of Agriculture and Interior as Civilian Conservation Centers.

Improving the tracking and reporting of Job Corps facility conditions and using the data to inform resource allocation decisions regarding improvements and maintenance.

Providing capital asset information during budget formulation and involve the CFO and Procurement Executive in its capital planning process.

Revising its budget presentation to show clearly the components of the program's $1.5 billion budget.

Proceeding with common measures implementation, which has been delayed.

Measure: Placement in Employment or Education

Explanation:Percentage of Job Corps participants who enter employment, enlist in the military, or enroll in post-secondary education or advanced/occupational skills training in the first quarter after exit from the program This measure tracks Job Corps' success in achieving one of its primary purposes, to help eligible youth obtain jobs, pursue further education, or enter the military. Job Corps' academic curriculum, career success skills and career technical training are all designed to assist students in obtaining meaningful employment and providing them with the tools necessary to advance in education and their chosen occupations. Job Corps participants include not only graduates, but those students who are terminated for zero tolerance infractions or leave the program voluntarily. Targets and results for 2004 and 2005 are for a similar, previous measure that included only program graduates.

Measure: Attainment of a Degree or Certificate

Explanation:Percentage of students who attain a high school diploma, GED, or certificate while enrolled in the program. Performance for this measure is impacted by the Job Corps' National Certification Initiative. This initiative requires more stringent student attainments and graduations requirements and aligns Training Achievement Records (TARs) with industry-standards and certifications. A TAR is a list of competencies or skills for each major vocational program offered at a Job Corps center. While it is anticipated that these elevated training requirements (once fully realized) will impact student achievement in a positive manner, the transition to the more rigorous TARs will continue to cause the performance results for this measure to decline for the short-term.

Measure: Literacy or Numeracy Gains

Explanation:Percentage of students who achieve literacy or numeracy gains of one Adult Basic Education (ABE) level, equivalent to two grade levels. The Literacy/Numeracy Gains measure acknowledges that basic literacy and numeracy skills are an important factor in employability and lifelong learning. Many students arrive at Job Corps with very low basic academic skills; Job Corps, therefore, focuses programming to increase their basic literacy and numeracy which will positively impact their ability to obtain and retain employment.

Measure: Cost per participant

Explanation:Job Corps currently reports efficiency as the cost per participant (total program appropriation level divided by the number of program participants served). Job Corps, however, recognizes that this measure of efficiency is inadequate as it is not linked to performance outcomes and does not facilitate comparisons across Department of Labor training and employment programs. The program, therefore, is currently reviewing various accounting methodologies for linking costs to specific performance outcomes and is currently exploring ways to improve cost efficiencies.

Is the program purpose clear?

Explanation: The program's purpose is to assist eligible disadvantaged youth (ages 16-24) who need and can benefit from intensive education and training services to become more employable, responsible, and productive citizens. Services are provided in group settings in residential and, to a lesser extent, non-residential centers.

Evidence: Authorizing statute - Workforce Investment Act of 1998 (P.L. 105-220, Title 1, Subtitle C, Sec. 141) at http://www.doleta.gov/usworkforce/wia/wialaw.txt; Job Corps regulations (20 CFR section 670.110): http://www.dol.gov/dol/allcfr/ETA/Title_20/Part_670/20CFR670.110.htm

Does the program address a specific and existing problem, interest, or need?

Explanation: Each year, Job Corps serves approximately 60,000 economically disadvantaged youth, aged 16-24, who also have one or more of the following characteristics: basic skills deficient; a school dropout; homeless, a runaway, or a foster child; a parent; or an individual who requires additional education, vocational training, or intensive counseling and related assistance in order to participate successfully in regular schoolwork or to secure and hold employment. All of these factors serve as barriers to employment. For example, high school dropouts are more likely than high school completers to be unemployed, have lower average incomes, and receive public assistance. In PY 2005, three-quarters of students entering Job Corps were high school dropouts; approximately one-third came from families on public assistance; and 55% of entering Job Corps students were reading below the eighth grade level.

Evidence: According to the U.S. Census Bureau, there were 37 million youth aged 16-24 in the U.S. in 2005. Of these: 15.9 million (43%) were not enrolled in school; 6.4 million (17.4%) lived below the poverty line; and 14.8 million (39.9%) did not have a high school diploma or its equivalent (Bureau of the Census, Current Population Survey, 2006). The most recent unemployment rate for 16-24 year old dropouts, according the Bureau of Labor Statistics, is 17.3%, versus 10.3% for this age group overall. In 2005, 15.8% of 20-25 year olds with personal income less than $25,000 per year had no high school diploma or its equivalent. In contrast, of those 20-25 year olds with personal income of $25,000 or more per year, only 6.1% had no high school diploma or its equivalent (Bureau of the Census, Current Population Survey, 2006). Job Corps participant data is available through Job Corps' Executive Information System (EIS).

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: Although there are other Federal, State, and local programs that provide employment, training, and educational services to disadvantaged youth, Job Corps is unique in terms of its intensive residential model and exclusive focus on disadvantaged youth. At the Federal level, the AmeriCorps*NCCC (National Civilian Community Corps) and National Guard's Youth ChalleNGe program are residential programs, although there are substantial differences between these and Job Corps. Specifically, NCCC does not: focus services on disadvantaged youth (all youth aged 18-24 are eligible); offer education during participation in the program (completers of the 10-month service period receive a $4,725 award to use towards approved education-related expenses); or provide career technical training (training is predominantly in CPR, first aid, and public safety). Although the Youth ChalleNGe program serves a population closer to that of Job Corps ("at-risk" youth aged 16-18), offers intensive academic classes and provides some career preparation services, it does not offer career technical training or placement assistance (although graduates are partnered with mentors who assist with post-program transition). There are other programs that have some parallels to Job Corps, but they are also more limited in terms of structure, breadth of services offered, or eligibility criteria. For example, while YouthBuild is similar to Job Corps in that it provides academics and training to disadvantaged youth aged 16-24, it is a non-residential program that only offers training in one trade-area (construction) as opposed to the more than 100 occupations offered by Job Corps. At the state and local levels, state youth corps programs offer similar services as Job Corps, but, again, few are residential.

Evidence: While no recent studies have been found that compare Job Corp with other similar programs, two past research studies have highlighted the uniqueness of the Job Corps program. GAO compared Job Corps to state and local youth training programs in terms of four Job Corps characteristics: 1) serving a disadvantaged population, 2) providing basic educational instruction, 3) focusing on vocational training services, and 4) providing services in a residential setting. GAO found that youth corps programs were most similar to Job Corps, but only two (California Conservation Corps and Seaborne Conservation) included all four characteristics. However, GAO found that these programs differed from Job Corps in significant ways (e.g., the California Conservation Corps lacked a specific focus on disadvantaged youth and provided limited vocational training). (General Accounting Office), "Job Corps: Comparison of Federal Program with State Youth Training Initiatives" (GAO/HEHS-96-92, March 1996). A 1995 study commissioned by DOL compared Job Corps to nine other public and private programs offering education to youth in residential settings. The study found that while Job Corps shared many characteristics with other public and private residential education programs, Job Corps was unique in a number of ways, including scope, exclusive focus on low-income individuals; provision of child care; primary focus on job training/readiness; and self-paced nature. Heidi Goldsmith, "Overview of U.S. Residential Education Programs for Youth" (International Center for Residential Education, 1995). Youthbuild USA Annual Report 2005: http://www.youthbuild.org/atf/cf/{22B5F680-2AF9-4ED2-B948-40C4B32E6198}/AnnualReport2005.pdf U.S. DHHS, Family and Youth Services Bureau, Fact Sheet: Transitional Living Program for Older Homeless Youth: http://www.acf.hhs.gov/programs/fysb/content/youthdivision/programs/tlpfactsheet.htm National Guard Youth ChalleNGe 2005 Annual Report: http://www.ngycp.org/ngycp/2005_NGYCP_Annual_Report.pdf Child Welfare League of America-Foster Care Independence Act of 1999 http://www.cwla.org/advocacy/indlivhr3443.htm Child Welfare League of America-Programs and Resources for Youth Aging Out of Foster Care: http://www.cwla.org/programs/fostercare/agingoutresources.htm Americorps National Civilian Conservation Corps: http://www.americorps.gov/about/programs/nccc.asp Job Corps Program Information Notice 05-06 and 05-06a, National Framework for Youth Challenge Partnerships, 8/25/2005 Memorandum of Understanding Demonstration Project for Income Eligibility Waiver between The Louisiana Youth ChalleNGe Program & The Carville Job Corps Center, 6/1/2005; Memorandum of Understanding between Oklahoma Military Department and Tulsa Job Corps Center, Tulsa, OK, 2/8/2006; Memorandum of Agreement between Turner Job Corps Center and Fort Stewart Youth Challenge Academy, 10/2/2004 Job Corps Program Instruction Notice 06-07, Serving Students in the Foster Care System, 9/8/2006

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: There are two major design flaws that limit the program's effectiveness and efficiency. First, Job Corps is currently administered from within the Office of the Secretary (OSEC). The location of Job Corps within OSEC denies Job Corps the true integration with the broader workforce investment system that is administered by the Employment and Training Administration (ETA). ETA possesses the Department's expertise in workforce development issues and administers all of the other job training programs under the Workforce Investment Act. The recent relocation of Job Corps to the Office of the Secretary, mandated in the 2006 Appropriations Act, has created significant administrative and programmatic inefficiencies. Second, Job Corps' authorizing legislation states that a selected applicant to Job Corps must be assigned to the center nearest to their home, except where: a) the enrollee chooses a vocational training program, or requires an English literacy program, that is not available at such center; b) the enrollee would be unduly delayed in participating in the Job Corps program because the closest center is operating at full capacity; or c) the parent or guardian of the enrollee requests assignment of the enrollee to another Job Corps center due to circumstances in the community of the enrollee that would impair prospects for successful participation in the Job Corps program. This requirement impedes Job Corps' ability to match participants with the Job Corps Centers that specialize in the fields of most interest or most appropriate for program participants. Job Corps is focusing on improving career technical training programs by aligning them with certification requirements and standards within high-growth, high-demand industry sectors. Job Corps is planning a so-called "career cluster" approach, whereby training programs for related occupations within an industry sector are offered at specific centers in each region, based on local labor market demand. It is anticipated that this change will improve training and employment opportunities for students and also provide long-term efficiencies by allowing centers to focus their efforts on specific occupations and trades, reducing duplication in training programs, and increasing retention of program participants by directing them to centers with programs that fit their skills and vocational goals.

Evidence: Mathematica Policy Research, Inc. "National Job Corps Study and Longer Term Follow-up Study" (August 2006) WIA, Title 1, Subtitle C, sec 147b: http://www.doleta.gov/usworkforce/wia/wialaw.txt; San Jose Center for Employment and Training: http://www.cet2000.org/index.php Job Corps Curriculum Change Notice 06-02, Job Corps Industry Certification Implementation??Revised Training Achievement Records and Implementation Guides for the New Retail Sales Program, December 21, 2006; Job Corps Curriculum Change Notice 06-03, Job Corps' Industry Certification Implementation Revised Training Achievement Records and Implementation Guides: Computer Networking/Cisco Program, March 19, 2007; Job Corps Curriculum Change Notice 06-04, Job Corps' Industry Certification Implementation Revised Training Achievement Records and Implementation Guides: Collision Repair and Refinish Technician Program, March 20, 2007; Job Corps Curriculum Change Notice 06-01, Job Corps Industry Certification Implementation??Revised Training Achievement Records and Implementation Guides for the New Manufacturing Technology Program, December 20, 2006; Sample Program Instruction and Curriculum Change Notice released to Job Corps Staff to announce a new training program: Job Corps Program Instruction Notice 05-16, Implementation of Job Corps' Industry-Aligned Automobile Technician Training Program, March 9, 2006.

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: Job Corps is required by law to serve all persons that meet basic eligibility requirements (economically disadvantaged youth, age 16-24, who have one or more specified barriers to employment). To address concerns about the high rate of non-completion, however, the program has taken steps to enroll, within the eligible population, those who are "ready" for the program, either deferring enrollment for those who are not or referring them to a more suitable program. Most notably, in PY 2005, an interdisciplinary committee of Job Corps field staff, federal staff, and mental health and wellness professionals developed and released Policy and Requirements handbook appendix that provides methods and tools for admissions counselors to observe and document applicant behavior, as well as court and institutional records, to effectively evaluate whether the student is ready to meet the rigor of the program (while not negatively affecting the training and education of other students in the program). Additionally, Job Corps is beginning to develop profiles of its centers to determine which ones are effective at meeting the needs of specific segments of our population. For example, some centers excel at serving younger youth and therefore should focus on this portion of the eligible population. Job Corps has also launched a Pre-enrollment Drug Screening Pilot to determine if drug testing prior to entry (versus after enrollment) will decrease the number of student separations due to the zero tolerance policy for drugs. Job Corps utilizes National, Regional, and local marketing efforts to reach out to the target population and their parents/legal guardians. Job Corps' National Media Recruitment Campaign, a focus-group tested outreach effort comprised of television and radio advertisements and in-school publications, directs prospective students to the Job Corps National Call Center. The call center receives over 400,000 calls annually, over 150,000 of which request a referral to an admissions counselor in their area to continue the admissions process and enroll in the program. At a local level, admissions counselors partner with local school districts, community and faith-based organizations, government and foster care agencies, and local businesses to recruit eligible youth. To ensure that participants meet legal eligibility requirements, Job Corps uses a standardized eligibility determination protocol with documentation/verification requirements for each of the eligibility factors. Applicants are screened by outreach and admissions contractors, and must provide documentation (e.g., proof of legal U.S. residency, valid social security card, proof of age, proof of income eligibility, and, where applicable, school transcript, driver's license, and court records verifying that the student is eligible for participation) to verify their eligibility. The eligibility determination and enrollment process is supported by Job Corps' automated system, Outreach/Admission Student Input System (OASIS), which captures and records all eligibility data elements for each applicant. Information regarding the validation process for these data can be found in Section 3.1.

Evidence: The Policy and Requirements Handbook (PRH) specifies eligibility and screening factors and requirements for documentation and verification of eligibility. (See OJC PRH, 'Chapter 1 Outreach and Admissions'. OJC, PRH, Chapter 3, Section 3.2, and Appendices 306 and 307 - The Career Technical Training Change Process). Behavioral guide described above is PRH Appendix 104, entitled "Admissions Counselor Guide for Evaluating Applicant Behavior and Court History.". WIA, Title 1, Subtitle C, sec 144-145: http://www.doleta.gov/usworkforce/wia/wialaw.txt; Standard Operating Procedure (SOP) No. 4.11.7 Job Corps PRH Change Notice 05-17, PRH Chapter 3: Section 3.2 and New Appendices 306 and 307 - The Career Technical Training Change Process, June 7, 2006. Job Corps-Relevant LMI by State for Philadelphia Region; Job Corps-Relevant LMI by State for Atlanta Region; Job Corps-Relevant LMI by State for Dallas Region; Job Corps-Relevant LMI by State for Chicago Region; Job Corps-Relevant LMI by State for San Francisco Region.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: Job Corps is part of the Job Training Common Measures initiative. Accordingly, in 2004 it adopted long-term performance measures that are meaningful, outcome-oriented, and tied to the program's purpose of improving the employment opportunities of disadvantaged youth. The program's goal is to improve the educational achievements of Job Corps students and to increase the participation of graduates in employment and education. The program has established long-term measures in line with the Job Training Common Measures for youth and lifelong learning: (1) percentage of youth entering employment, post-secondary education, occupational skills training, or the military one quarter after program exit; (2) percentage of youth earning a high school diploma, GED, or certificate while in the program; (3) participants' attainment of literacy and numeracy gains; and (4) efficiency in achieving outcomes. The efficiency measure is addressed in more detail in questions 3.4 and 4.3.

Evidence: Authorizing statute - Workforce Investment Act of 1998 (P.L. 105-220; Title 1, Subtitle C, Sec. 141): http://www.doleta.gov/usworkforce/wia/wialaw.txt; Job Corps regulations (20 CFR section 670.110) http://www.dol.gov/dol/allcfr/ETA/Title_20/Part_670/20CFR670.110.htm; DOL Strategic Plan for FY 2006 - FY 2011 (http://www.dol.gov/_sec/stratplan/); and DOL Employment and Training Administration (ETA), Training and Employment Guidance Letter (TEGL) Number 17-05: Common Measures Policy (Feb 2006) which replaced earlier guidance with a single unified Department of Labor guidance document on the common measures and WIA Section 136 performance accountability system: http://wdr.doleta.gov/directives/attach/TEGL17-05.pdf

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: DOL's FY 2006-FY 2011 Strategic Plan includes long-term measures (and ambitious targets) for Job Corps' performance measures. By 2011 the goals and targets are: 63% Literacy/Numeracy gains (versus a baseline of 45%); 68% attainment of a GED high school diploma, or certificate by the end of the program (versus a baseline of 64%); and 87% placement of all students in employment or education in the first quarter after exit. Literacy/Numeracy Gains: The PY 2011 target of 63% is ambitious given the baseline data. Job Corps began reporting on literacy/numeracy data in PY 2004; by the end of that program year, Job Corps had exceeded its 45% goal. While results further increased in PY 2005 to 58%; PY 2006 YTD results indicate that performance has leveled off at 59%. Despite the expectation that it will be difficult to continue this upward trend, Job Corps has raised the PY 2011 target to be sufficiently challenging. Certificate/Degree Attainment: The following table presents annual targets for Certificate/Degree Attainment: The PY 2011 68% target is ambitious, given the impact of a number of factors including: 1. Job Corps measures degree/certificate attainment during program enrollment (versus the end of the third quarter after exit, as under the common measures methodology); 2. new data integrity auditing practices resulted in a recalibration of performance data; 3. Challenges presented by many states (currently 24) that now require students to pass high school exit exams in order to attain their diplomas; and 4. Programmatic changes that Job Corps is making to align Job Corps certifications with industry-recognized certification, apprenticeship prep, and state licensure requirements will require even higher levels of attainment for students to earn certificates. Placement in Employment and Education: The 89% placement target is extremely ambitious given that the pool for placement has expanded beyond graduates to all students. The 87% placement goal was established based on Job Corps' traditionally high placement rate, which averaged 90% for PY 2000 - PY 2004, (for program graduates only). The pool for CM reporting of placement covers all students, including those who are expelled for drugs and violence or who leave for other reasons. Job Corps has not traditionally tracked these uncommitted students once they leave the program, but given the substantial Federal investment in these individuals believes it is worthwhile to do so. The efficiency measure (cost per participant) is addressed in Questions 3.4 and 4.3.

Evidence: US DOL, Budget Justification of Appropriation Estimates for Committee on Appropriations, FY 2008, Volume 1. http://www.dol.gov/dol/budget/2008/PDF/CBJ-2008-V1-04.pdf and http://www.dol.gov/dol/budget/2008/PDF/CBJ-2008-V1-11.pdf. ETA Performance Chapter http://www.dol.gov/dol/budget/2008/PDF/CBJ-2008-V1-11.pdf ; DOL Strategic Plan for FY 2006 - FY 2011 (Sept 2006) (http://www.dol.gov/_sec/stratplan/); Office of Job Corps (OJC) Policy and Requirements Handbook (PRH) Change Notice No. 06-01 (July 2006) "Update/Revision to PRH Chapter 5, Appendix 501: PY 2006 Performance Management System-Appendices 501a to d." Performance targets are published annually as Appendix 501 to Job Corps' PRH. Appendix 501 specifies targets for the following components: Centers (Appendix 501a), Outreach/Admissions (Appendix 501b), Career Transition Services (Appendix 501c) and Vocational Programs (Appendix 501d); Workforce Investment Act of 1998, P.L. 105-220; Title 1, Subtitle C: Job Corps, Section 159 (c) (1) (D): http://www.doleta.gov/usworkforce/wia/wialaw.txt; and Job Corps' Common Measures Long-Term Performance Targets - Discussion Paper.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: Job Corps' annual performance measures are identical to the four long-term measures described in Section 2.1: specifically, (1) percentage of youth entering employment, post-secondary education, occupational skills training, or the military one quarter after program exit; (2) percentage of youth earning a high school diploma, GED, or certificate while in the program; (3) participants' attainment of literacy and numeracy gains; and (4) efficiency in achieving outcomes. Annually, the program updates measures, goals and weights in the program's Outcome Measurement System (OMS) that reflect the program's priorities and long-term goals. In 2004, Job Corps began including indicators that are more aligned to common measure goals in the OMS. The measures, goals and weights continue to be refined annually to strengthen areas that require additional focus. For PY 2006, the center and Career Transition Services providers' OMS measures include the following: GED/HSD attainment, vocational completion, literacy gains, numeracy gains, job training match, post-enrollment placement, graduate placement and graduate placement wage, 6-month placement and earnings, and 12-month placement. The efficiency measure is addressed in more detail in Questions 3.4 and 4.3.

Evidence: US DOL, Budget Justification of Appropriation Estimates for Committee on Appropriations, FY 2008, Volume 1. http://www.dol.gov/dol/budget/2008/PDF/CBJ-2008-V1-04.pdf and http://www.dol.gov/dol/budget/2008/PDF/CBJ-2008-V1-11.pdf. Performance Budget Overview Chapter http://www.dol.gov/dol/budget/2008/PDF/CBJ-2008-V1-01.pdf; DOL ETA, TEGL Number 17-05: Common Measures Policy (Feb 2006); and OJC, PRH Change Notice No. 06-01 (July 2006), "Update/Revision to PRH Chapter 5, Appendix 501: PY 2006 Performance Management System-Appendices 501a to d."

Does the program have baselines and ambitious targets for its annual measures?

Explanation: Job Corps has established baselines for all four annual measures and, given PY 2005 performance results, the targets are ambitious. The Outcome Measurement System (OMS), which is used to track Job Corps center operators' performance and described in more detail in 2.5, is the vehicle used to drive performance to meet longer-term targets. Each year, Job Corps also considers economic factors that affect the national economy when establishing the national goals. OMS goals established for Job Corps' contractors correlate directly to the national goals that Job Corps sets for itself. Annual revisions are made to OMS goals to require improvement over recent performance trends. Each national performance goal has multiple corresponding OMS measures that contribute to the achievement of the goal. Some OMS targets are national goals, while others take local conditions into account to ensure that goals are ambitious for each center taking into consideration where external factors may significantly impact outcomes.

Evidence: US DOL, Budget Justification of Appropriation Estimates for Committee on Appropriations, FY 2008, Volume 1. http://www.dol.gov/dol/budget/2008/PDF/CBJ-2008-V1-04.pdf and http://www.dol.gov/dol/budget/2008/PDF/CBJ-2008-V1-11.pdf. ETA Performance Chapter http://www.dol.gov/dol/budget/2008/PDF/CBJ-2008-V1-11.pdf ; Job Corps Center Report Card, OMS-10: PY 2004 (7/1/04- 6/30/05); PY 2005, 2nd Qtr, (7/1/05-121/31/05); PY 2005 (7/1/05- 6/30/06); and PY 2006, 2nd Qtr, (7/1/06-12/31/06); and OJC, PRH Change Notice No. 06-01 (July 2006), "Update/Revision to PRH Chapter 5, Appendix 501: PY 2006 Performance Management System-Appendices 501a to d."

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: All Job Corps partners play an important role in the program's attainment of both annual and long-term measures of the program. The Outcome Measurement System (OMS) provides the report card framework by which contractors are scored and ranked according to their performance. The OMS includes: (1) the Outreach and Admissions (OA) Report Card, which assesses the performance of OA contractors in terms of arrival and commitment rates; (2) the Center Report Card, which gauges centers' performance in terms of students' attainment of academic and career technical outcomes; (3) the Career Transition Services (CTS) Report Card, which measures CTS contractors' performance in the placement and earnings outcomes of students; and (4) the Vocational Training Report Card, which assesses career technical training programs in terms of placement rate and earnings. OMS targets are also tied to the national measures (see 2.1 and 2.3) and are established based on: (1) managers' and contractors' recommendations; (2) past results; and (3) national targets. In addition to monthly reviews of OMS performance, center and CTS contractors are held accountable for past performance as measured by annual on-site reviews, quarterly student satisfaction surveys, monthly financial reviews, and student achievement data. Past performance is used in making contract awards and renewal decisions, and accounts for one-fifth of total points awarded in rating proposals. Job Corps has implemented performance-based service contracting (PBSC) for all center operations and CTS contracts, whereby the contractor receives a base fee, plus incentive fees, tied to their performance (assuming it is above a certain level).

Evidence: OJC, PRH Change Notice No. 06-01 (July 2006), "Update/Revision to PRH Chapter 5, Appendix 501: PY 2006 Performance Management System-Appendices 501a to d."; Job Corps Information Notice No. 06-04 (Aug 2006) prescribes performance ranges for payment of incentive fees under center operations contracts; Job Corps Information Notice No. 06-05 (Aug 2006) prescribes performance ranges for incentive fees under career transition services contracts. Such incentive-fees currently apply only to performance-based service contracts for center operators and CTS providers; and Job Corps Center Report Card, OMS-10: PY 2004 (7/1/04- 6/30/05); PY 2005, 2nd Qtr, (7/1/05-121/31/05); PY 2005 (7/1/05- 6/30/06); and PY 2006, 2nd Qtr, (7/1/06-12/31/06).

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: Due to its rich historical performance data and the large Federal investment in this program, independent evaluations are frequently conducted to both determine the effectiveness of Job Corps and support program improvements. In 2001, Mathematica Policy Research, Inc. released findings from its comprehensive, four year impact study on the program's effectiveness in delivering education/training services to youth. In 2003, Mathematica released an update to this study that reviewed 10 years of earnings-related administrative data to further evaluate the cost-benefit of the program and in 2006, Mathematica released the final report that interpreted the main evaluation results from the past twelve years. Findings from this large-scale random assignment suggest that the Job Corps model as currently structured, while not cost effective and having little to no long-term impact on earnings and employment, does have promise; program participation increased educational attainment and literacy and reduced criminal activity and welfare receipt. HMA Associates' Study of Hispanics in Job Corps (2004-2005), which explored factors (i.e., language and community support networks) that can affect the post-program economic impact for Hispanics in Job Corps, has been used extensively by Job Corps management in the development of programs targeted to assist recruitment and retention of Hispanic students. DOL's OIG regularly reviews the program to assess Job Corps' efficiency in operations and performance reporting. In 2003, Job Corps commissioned an Economic Impact Study which sampled a cross-section of center sizes, geographic, urban and rural locations, and ages to determine the economic impact of centers on their local communities. Results were positive, showing that for every dollar that a Job Corps center spends in its local area, $1.91 in economic activity results within the area; impact is even stronger when a new center is built. Commissioned evaluations of pilot projects have also been used to make decisions to discontinue ineffective pilots when analyses determined that pilot data did not support nationwide expansion.

Evidence: Mathematica Policy Research, Inc., "National Job Corps Study: Findings Using Administrative Earnings Records Data", (Oct 2003); Summary Report: Mathematica Policy Research, "Does Job Corps Work? Summary of the National Job Corps Study" (June 2001): http://wdr.doleta.gov/opr/fulltext/01-jcsummary.pdf; Mathematica Policy Research, Inc., "National Job Corps Study and Longer-Term Follow Up Study: Impact and Benefit-Cost Findings Using Survey and Summary Earnings Records Data" (Aug 2006); HMA Associates, Inc., "Study of Hispanics in Job Corps" (2004-2005); OIG: Job Corps Performance Measurement Outcomes, Report Number 09-04-004-03-370 (Sept 2004); OIG: Performance Audit of Job Corps Center Operating Costs, Report Number 03-05-004-03-370 (Mar 2005); OIG: Kittrell Job Corps Center: Manipulation of Student Attendance and Training Records, Report Number 09-05-001-03-370 (Mar 2005); OIG: San Diego Job Corps Center: Student Attendance and Training Overstated, Report Number 09-05-004-03-370 (Sept 2005); OIG: Strengthening Efforts to Assess and Account for Students with Cognitive Disabilities would help Job Corps Achieve Its Mission, Report Number 09-06-001-03-370 (Nov 2005); OIG: Audit of Job Corps Student Pay, Allotment and Management Information System for the 2 Years Ended September 30, 2005, Report Number 03-06-003-01-370 (Sept 2006); OIG: Performance Audit of Job Corps Center Operating Costs for the Period October 1, 2004 - March 31, 2005, Report Number 03-06-005-01-370 (Sept 2006); OIG: ABC Georgia Overstated Job Corps Placement Outcomes, Report Number 09-06-004-01-370 (September 2006); OIG: Los Angeles Job Corps Center: Allegations of Misuse of Department of Labor Funds and Property, Report Number 09-06-005-01-370 (Sept 2006); OIG: Boston Regional Office Procurement of Job Corps Center Operator and Service Providers in Puerto Rico, Report Number 26-06-002-01-370 (Sept 2006); OIG: Audit of Complaint Involving the Cincinnati Job Corps Center, Report Number 03-06-004-01-370 (OIG Reports can be found at: http://www.oig.dol.gov/cgi-bin/oa_rpts.cgi?s=job+corps&y=all&a=all); Job Corps Program Information Notice No. 04-14, Change to the Program Assessment Guide (PAG) Reinforcing the Validation of Performance Data (Dec 2004); Sample Regional Office Center Assessment (ROCA), Philadelphia Job Corps Center Program Assessment (Jan 2006); Center Response, and Region II Philadelphia Monitoring Trip Report (June 2006); and Decision Information Resources, Inc., Analytic Briefs: "The Economic Impact of Job Corps Centers on Their Local Communities", (July 2004); "Placement Retention Outcomes of 16- to 17-Year-Old Placed Graduates" (Aug 2004); "Age Effects on Post-center Outcomes: First Year Continued Employment" (June 2005); "Impact of On-Center and Post-Center Experiences on First-Year Placement Credit at 6- and 12-months" (July 2005); "Ethnicity Differences in On-Center Outcomes: Initial, Final, and Changes in Reading and Math TABE Scores" (July 2005); "Gender Differences in On-Center Outcomes: Initial, Final, and Changes in Reading and Math TABE Scores" (July 2005); "Demographic Analysis of Zero Tolerance Terminations: Who are the ZTs?" (July 2005); "Ethnicity Differences in Post-Center Outcomes: First-Year Placement Credit at 6 and 12 Months?" (July 2005); "Gender Differences in On-Center Outcomes: Vocational Completion and GED Attainment" (Feb 2006); "Ethnicity Differences in On-Center Outcomes: Vocational Completion and GED Attainment" (Feb 2006); "Gender Differences in Post-Center Outcomes: First-Year Placement Credit at 6 and 12 Months" (Feb 2006); "Graduation Rates by Gender, Ethnicity and Age Subgroups" (Dec 2006).

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: While Job Corps' budgets provide information regarding program performance and link performance goals to the budget, they do not make clear the impact of funding, policy, or legislative decisions on program performance. In addition, while the transparency of Job Corps' budget has improved in recent years (notably with the addition of specific information on the components of the operating budget for the budget year), it needs further improvement to be truly transparent to the reader. For example, the budget should include a table with the various operating and construiction budget components that clearly delineates and explains the year-to-year changes in the various components.

Evidence: US DOL, Budget Justification of Appropriation Estimates for Committee on Appropriations, FY 2008, Volume 1. http://www.dol.gov/dol/budget/2008/PDF/CBJ-2008-V1-04.pdf and http://www.dol.gov/dol/budget/2008/PDF/CBJ-2008-V1-11.pdf

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: Job Corps has aligned the daily decision making process with its strategic objectives by implementing many improvements related to capital planning. The program has developed a formal Capital Asset Plan, a 3-Year Timeline, and updated the Asset Management Plan (to address weaknesses highlighted in the previous PART review). Job Corps has submitted Exhibit 300s, in accordance with the OMB A-11, for the OMB Budget submissions which incorporate risk analysis, cost-benefit analysis, and earned value management for all major projects. Additionally, Job Corps has integrated elements of the capital plan into the Facility Survey process, Budget Volume 1, and the Job Corps Inventory of Needs (ION) project selection system. Job Corps formed the Capital Planning workgroup to address the issues involving asset performance, prototype designs, and the strategic plan; the group met several times and will continue to make progress in Capital Planning. While other youth training programs may take credit for certificates attained up to nine months after program participation, Job Corps made the decision to limit credit to certificate attainment made during program participation. In the career technical training area, Job Corps is moving to industry certification standards to produce better equipped students, although completion rates may temporarily dip as a result of the more intensive training required. In order to track employment by all students for common measures (and to continue to focus on post-center placement services for graduates and former enrollees), Job Corps has entered into an agreement with the Kansas Department of Commerce to receive aggregate data through the Wage Record Interchange System (WRIS). Despite current limitations of using this data (as of March 1, 2007, only 27 states have signed the data sharing agreement to participate), Job Corps is attempting to gather post-center placement data for as many former students as possible, in the most cost-efficient manner. The program has also been responsive to potential programmatic shortcomings identified by independent studies. For example, the STARS (Speakers, Tutors, Achievement, Retention and Success) program, currently being piloted at 13 centers nationwide, is a mentoring program designed to increase student retention, academic achievement, and Job Corps completion. In response to data and evaluation findings regarding Hispanic students and younger youth, new programs targeting these subgroups of participants are being developed.. Job Corps has long tried to focus on helping students secure high-paying jobs that will sustain them and allow them to be contributing members to society. This is still apparent as Job Corps moves to industry certification standards, high-demand occupations and those with career ladders to improve long-term wages and continued employment. When the OIG identified concerns regarding the quality of Job Corps' data as reported by contractors, the system moved to develop and implement a rigorous data integrity system including targeted sampling, more intensive regional oversight, and securing liquidated damages where abuse was found. The system has also clarified definitions and business rules where needed to ensure consistent data reporting.

Evidence: US DOL, Budget Justification of Appropriation Estimates for Committee on Appropriations, FY 2008, Volume 1. http://www.dol.gov/dol/budget/2008/PDF/CBJ-2008-V1-04.pdf and http://www.dol.gov/dol/budget/2008/PDF/CBJ-2008-V1-11.pdf; Hawaii/Maui Job Corps Center, Facility Planning Report Volume 1, Planning Strategy (May 2006); Job Corps Capital Asset Plan; Sample Exhibit 300: Pinellas Job Corps Center; Job Corps Information Notice No. 06-21 http://www.dol.gov/dol/budget/2008/PDF/E300-2008-041.pdf; "Construction, Rehabilitation, and Acquisition Funding Allocation - PY 2007 (February 2007) with Attachment A. ION Web Site Instructions and Attachment B. Federal Real Property Assessment Management; USDOL Real Property Asset Management Plan (Dec 2006); USDOL Real Property Asset Management Plan, Three-Year Timeline (Dec 2006); Getting to Green in Real Property Asset Management, (Feb 2007); Memorandum of Understanding Between The Kansas Department of Commerce, and the United States Department of Labor, Office of Job Corps (June 2006); Job Corps Information Notice No. 06-09, "STARS Update/Status to Regions, Centers, and Contractors" (Sept 2006); and Job Corps Information Notice No. 04-14, "Change to the Program Assessment Guide (PAG) Reinforcing the Validation of Performance Data (Dec 2004).

Has the agency/program conducted a recent, meaningful, credible analysis of alternatives that includes trade-offs between cost, schedule, risk, and performance goals, and used the results to guide the resulting activity?

Explanation: The Department of Labor has established an Asset Management Plan and the Job Corps Capital Asset Plan, which are intended to govern the allocation of resources and selection of projects are made. Recent decisions, such as the decision to open a new center in Iowa, for example, have been made without cost-benefit analyses, consideration of alternatives or the opportunity costs of opening a new center (in light of ongoing maintenance needs). Further, the recent closure of the Department of Interior-operated Oconaluftee Job Corps Center in North Carolina??a highly unusual step??in response to health and safety concerns that had not been addressed, points to the need to ensure the accuracy of facility condition assessments (the center had been rated in almost-perfect condition) and fix safety and health issues before they escalate to a point requiring closure and disruption of students. Utilizing information from the facilities surveys done at each center every three years by the A&E contractor, identified projects are rated according to importance to mission, life safety concerns, regulatory (environmental, accessibility). There is also an opportunity for the Center operators and Regional Offices to provide input, utilizing an on line data base application to rate the greatest needs from their perspective. Following the prioritization of the projects based upon these criteria, a cost analysis is conducted to determine the funded regional projects for the program year. The budget is developed by the Architectural and Engineering (A&E) Contractor, taking into account project costs and benefits (e.g., reduced maintenance, energy, or lease costs), performance measures, needs identified by the Job Corps managers, program goals as established by the Job Corps National Director and the Department, and other issues (e.g., regulatory, environmental, security, safety). The Office of Job Corps has provided Exhibit 300s, consistent with the A-11 (that incorporates risk analysis, cost benefit analysis, and earned value management for major capital projects), in support of its capital budget request and has a formal Capital Asset Plan. The CRA budget, after approved by the National Director of Job Corps, is reviewed by the Departmental Budget Center and Department's Budget Committee (which includes the Real Property Officer, Procurement Executive, and the Chief Financial Officer).

Evidence: PY 2006 Job Corps CRA Budget Summary (PY 2006 - PY 2011 proposed); US DOL, Job Corps Design & Construction Program, "The Pre-Construction Conference Text" is a business practices manual of quality procedures which includes the processes used to develop the budget and manage projects; Sample Exhibit 300: Pinellas Job Corps Center, http://www.dol.gov/dol/budget/2008/PDF/E300-2008-041.pdf; and Job Corps Capital Asset Plan.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: All of Job Corps' contractors must report on their activities and performance into the national database via the Job Corps Outcome Measurement System (OMS). Data integrity is managed according to a process outlined in the Data Validation System paper. The information collected through this system is used to both determine the baselines for performance goals and hold partners accountable for performance outcomes. Monthly and annual reports detailing all partners' results under Job Corps' performance measurement system are computed, disseminated and used to monitor performance and develop corrective action plans. The Job Corps National Office ensures that data integrity, reporting quality and timeliness are sustained at all levels. The Data Validation System paper also outlines the multiple components that are used to manage data element and data reporting validation for the entire Job Corps system. The National Data Center in Austin, Texas manages the administration and maintenance of the student database, payroll system, customer support (helpdesk), as well as provides information technology support. Performance data reports are available on-line through the Job Corps Resource Library (JCRL), which is also used to manage the program strategically. Another more recent feature of the Job Corps community Web site is the Executive Information System (EIS), a Web application (part of the CDSS Suite of Applications) designed specifically for National Office and Regional Office staff. EIS contains a variety of tools (e.g., Dashboard, Program Monitor, Trend Analyzer and Data Investigator) which allow staff to query data at the national, regional, and center level (depending upon access limitations) for data-based decision making. For example, upon noting a growing number of "quick" placements (i.e., placements in jobs that require less intensive vocational training), measures were taken to emphasize more rigorous career technical training using industry-recognized standards and to revise (and align) Training Achievement Records (TARs) with industry certifications in high growth industry sectors (which yield higher wages and better career opportunities). Another example of how performance data has been used to adjust program priorities occurred when Job Corps identified that student retention rates had declined to 56% in PY 2005 from a previous peak of 62%. To improve performance in this area, Job Corps implemented the STARS program (see Section 2.8), a student support program that provides literacy and numeracy tutoring and mentoring services designed to increase student retention. Job Corps is also piloting the evidence-based READ 180 Program in eight Job Corps centers to increase reading comprehension, after recent analyses showed that 45% of Job Corps graduates exit the program reading below the 8th grade level. In recent years, the OIG has performed a series of audits to assess Job Corps' processes for ensuring the reliability of contractor-reported performance outcomes. These reports identified [brief description of problem.] In response to the OIG's findings and recommendations, Job Corps took steps to ensure reliability of performance data by requiring mandatory audits of student records concurrent with annual center quality assessments. The Facilities Management Unit of Job Corps utilizes an earned value management (EVM) system for capital asset projects and reports the information in the capital asset plan, exhibit 300b. Project data is inputted into the EVM formula to ascertain the performance information.

Evidence: Job Corps Data Center, "The Data Validation System of the Office of Job Corps" (Mar 2007); Job Corps PRH, Chapter 1.2 Eligibility Determination and Screening Factors, §R7. Documentation (Mar 2006) outlines requirements for Outreach and Admissions to use Job Corps' CIS database system; Job Corps PRH, Chapter 4.5 Documentation, Reporting and Verification, §R1a. Record Keeping (Nov 2005) outlines requirements for CTS operators to use Job Corps' CIS database system; Job Corps Program Instruction Notice No. 06-06, "Updating Center Information in the Center Information System (Sept 2006); and Job Corps Data Center, Executive Information System (EIS) Technical Guide (Jan 2006).

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: All Job Corps Federal Managers, contractors and agency partners are held responsible for the achievement of program results, financial management, and timeliness of performance. Additionally, Job Corps senior managers are held accountable for the fiscal, data and program integrity of all contracts awarded as well as those elements that directly relate to the program's internal and external goals. Contractors are held accountable for past performance, as measured by annual on-site reviews, quarterly student satisfaction surveys, monthly financial reviews, student achievement data, and monthly performance on OMS reports. The DOL FY 2006 Performance and Accountability Report identified four major challenges facing the Job Corps program that all program partners are involved in effectively addressing: 1) ensuring student safety and health, 2) ensuring student success, 3) monitoring and verifying performance and 4) ensuring proper procurement, updated agreements, and program effectiveness. Health and Safety: Regional Offices are held accountable for monitoring ZT compliance, as well as effective oversight of facilities for safety issues. Students with Cognitive Disabilities: A part-time disability coordinator position has been established in each region to assist in the development of effective center disability programs that meet the PRH requirements. Monitoring Performance: Job Corps has developed and implemented policies and procedures for its data collection and reporting systems in response to OIG findings. Improvements have been made through revisions to the PAG, utilization of targeted sampling for audit reviews, and procedures for identification and collection of liquidated damages. Procurement and Interagency Agreements: Job Corps has improved monitoring of procurement at the national level by instituting a more rigorous review process for option year awards, and is in the process of updating interagency agreements with the Departments of Interior and Agriculture in an effort to more effectively serve students at Civilian Conservation Centers. For center operations and CTS contracts Job Corps utilizes PBSC, consisting of a base fee, plus incentive fees, tied to performance (assuming it is above a certain level). Past performance is also used in making contract award and renewal decisions and accounts for one-fifth of total points awarded in rating proposals. Previously, under ETA, Job Corps contracts for center, OA and CTS agencies were awarded and option year packages reviewed by Job Corps' six regional offices. Upon recommendations stemming from an OIG audit in 2006, (Final Report No. 26-06-002-01-370), Job Corps revised its Compendium for Job Corps Procurements and modified the procurement process in conjunction with Office of the Assistant Secretary for Administration and Management guidelines. The Compendium includes guidelines for denial of option years and notification of unsuccessful bidders. In addition, Job Corps is in the process of implementing a more rigorous option year review process than previously conducted. For example, the Contractor Past Effectiveness Report is more extensive and the Option Year package now requires additional documentation (to support performance narratives) before a submission is accepted by the national office (where detailed compliance, data and fiscal reviews is conducted). A recent determination where the third option year of an OA contract was recommended as "not in the best interest of the Government to exercise the option year" (submitted as evidence) clearly demonstrates the extent to which option year packages are now being reviewed. The new option year review process is being implemented in conjunction with a move toward use of an Award Term Contracting process, whereby the granting of an award term is intended to provide an incentive to the contractor for high quality performance in achieving the negotiated goals and objectives.

Evidence: Performance Management Plan for Supervisors and Managers (Sept 2006); The OMS system is published annually in Job Corps Policy and Requirements Handbook (PRH), Appendix 501 a-d; Performance-based service contracting (PBSC) provisions are detailed in center operations and career transition services contracts, Section G; Job Corps Center Operations Model RFP, Section G: Contract Administration; Job Corps CTS Model RFP, Section G: Contract Administration; Job Corps Center RFP, Sections J-10: Past Experience/Effectiveness Confirmation Questionnaire; Job Corps Center RFP, Sections M3: Evaluation Factors for Award; Job Corps PRH, Appendix 502: Center Financial Management; Job Corps PRH, Appendix 503: Outreach/Admissions and Career Transition Services Financial Reporting; Sample Contractor Past Effectiveness Rating Report (CPER); US Department of Labor, FY 2006 Performance and Accountability Report: http://www.dol.gov/_sec/media/reports/annual2006/PDF/2006annualreport.pdf; Job Corps Program Information Notice No. 04-14, Change to the Program Assessment Guide (PAG) Reinforcing the Validation of Performance Data (Dec 2004); and Sample Regional Office Center Assessment (ROCA), Philadelphia Job Corps Center Program Assessment (Jan 2006); Center Response, and Region II Philadelphia Monitoring Trip Report (June 2006).

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: Since FY 1999, Job Corps has obligated 97.7% of all program dollars by the end of the period of availability. To ensure that all the financial information for both national programs and partners is aligned with the overall program plan, as well as with current program and policy initiatives, a comprehensive set of policies and procedures has been established with oversight administered from the National Office. The Job Corps AAPP/FOP (Advance Annual Procurement Plan - Financial Operating Plan), is a computer-based system used by the National Office of Job Corps to control the allocation of Job Corps program funds. This system maintains comprehensive information about the Job Corps contracts, inter-agency funding agreements, and other Job Corps budget entities, along with a financial operating plan that stores and reports data about specific contract or project funding approvals. Contractors use the AAPP amounts to build their budgets, which are called 2181s. In addition, contractors must submit monthly spending reports (called 2110s) via the web-based Financial Management System (FMS). Contractors must report their expenditures in each of 29 categories via FMS on a monthly basis, and must be able to explain any significant variances between their negotiated budget and actual expenditures (for each element, plus or minus 3% or $2,500, whichever is greater). Regional managers review the monthly reports, follow-up with the center when necessary, and assist the center in developing action plans to address and correct the variances.

Evidence: Sample ETA 2181: Contract Center Operations Budget from the FMS System; Sample 2110 Report, Job Corps Contract Center Financial Report; Job Corps PRH, Appendix 502: Center Financial Management; and Job Corps PRH, Appendix 503: Outreach/Admissions and Career Transition Services Financial Reporting.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: Job Corps developed a baseline for reporting efficiency beginning with the FY 2004 Budget, and has projected the efficiency targets through PY 2011 (See the performance measures section of the PART.) Specifically, efficiency is currently measured as the total operating expenses (which assumes the strict use of expenses from the program appropriation and disregards overhead and other add-on expenses) incurred during a program year, divided by the number of new enrollees in that program year. In addition, the program is exploring alternative cost-analysis methodologies that will more accurately calculate efficiency (i.e., cost per program outcome). Regarding Capital Asset and Service Acquisition efficiencies, Job Corps regularly uses competitive procedures. Competitive Sourcing/Cost Comparisons: In accordance with WIA, almost all Job Corps contracts are awarded on a competitive basis (exceptions are contracts awarded using a sole-source process, either to 8(a) small businesses, Indian or Native American entities authorized by statute, and Job Corps' nine National Training Providers). To ensure adequate competition, Job Corps procurement actions are publicly advertised using Fedbizopps. A Procurement Panel Guide process rates proposals based on technical merit and cost. Additionally, an estimated cost profile from the Advanced Annual Procurement Plan (AAPP) is prepared for each procurement action (see Section 3.3). To ensure competitive pricing, the proposal evaluation process involves preparation of a Government Cost Model, detailed analysis of the elements of each proposal, comparison of costs to the cost model, cost history and peer operations, and a determination of cost realism. Contracts are awarded to the contractor offering the best value to the government (past performance is also considered, as described in 3.2). IT Improvements: Job Corps continues to advance its technology and system capabilities, nationwide, at the regional and national office levels and at individual centers. Thin Clients - A thin client is a solid state computer device used as a "terminal" for end users to access a centralized computing environment (servers). Compared to using traditional PCs, use of thin clients is cheaper, they have longer lifespan and lower power consumption, are easier to manage and more secure. The adoption of thin clients across the Job Corps system has resulted in significant improvements in both performance and effectiveness by dramatically reducing down-time, requiring less staff maintenance, offering better control of licenses, and lowering costs by increasing standardization across the system. Virtual Servers -Virtual servers, software that allows a single server to behave and function as if it were three different servers, has allowed the program to increase its service delivery to the Job Corps community while significantly reducing the program's need to purchase additional individual servers. VOIP - Voice-Over Internet Protocol has been implemented across the Job Corps system and is present at most centers. Moving the entire Job Corps communications system off the older PBX platforms to the more cost-efficient VOIP has resulted in recurring savings of more than one million dollars per year in telephone costs. Appropriate Incentives: Contracts for center and CTS operations feature incentive-fee provisions as part of Job Corps' performance-based service contracting (PBSC) initiative. Incentive fees are tied directly to contractors' performance against targets established in Job Corps OMS.

Evidence: USDOL, OASAM Compendium for Job Corps Procurements, Sections 1 through 5 (Mar 2007).

Does the program collaborate and coordinate effectively with related programs?

Explanation: To expand opportunities for its students, Job Corps maintains both formal and informal partnerships and collaborative relationships with other employment/training and youth programs, Federal agencies, industry partnerships, and community service programs. Job Corps collaborates regularly with ETA's other youth/adult training programs to share training materials and discuss approaches for similar populations. As required under WIA, Job Corps is represented on both state and local workforce investment boards (WIBs) and youth councils, and operates Industry Advisory Councils and Community Relations Councils. Outreach and Admissions counselors conduct local outreach in conjunction with One-Stop centers and other community organizations. During the CTS phase, students visit local One-Stop centers for job referrals, access to other support services (further training opportunities, housing and transportation) and to learn about One-Stops as a potential resource when they return home after Job Corps. Additionally, Job Corps continues to work with the Office of Apprenticeship to market additional opportunities to Job Corps students. Other Youth Programs. Per the White House Youth Taskforce recommendations, Job Corps has developed formal MOUs with the Department of Defense's Youth ChalleNGe program. These partnerships allow eligible youth to either consecutively or concurrently enroll in both programs are are structured to avoid duplication of services. Over 500 students have participated in the combined Job Corps/Youth ChalleNGe program. Centers acknowledge the positive effect that Youth ChalleNGe participants have on center culture as role models of good behavior and discipline. Job Corps is also partnering with agencies that serve foster care youth. These partnerships occur primarily at the local level, since guidelines differ from state to state. Through collaboration with foster care agencies, Job Corps offers a valuable education and training alternative for youth aging out of the foster care system. Department of Education. In June 2001, the Secretaries of Labor and Education established a MOU to increase the numbers of Job Corps graduates with high school diplomas, under which Education staff provided valuable information/technical assistance on school accreditation, teacher professional development and online high schools. The percentage of centers offering high school programs through partnerships with local school districts, online programs, or by becoming accredited schools, has increased from 39% to 98%. Job Corps has developed a host of partnerships with national employers and industry associations to support training, work-based learning, and employment opportunities for students. Industry partners include AAMCO Transmissions, American Culinary Federation, CISCO, the National Automobile Training and Education Foundation (NATEF), National Healthcareer Association, National Retail Federation, Roadway Trucking, and Toyota. Industry partners have also assisted with the alignment of Job Corps training programs to industry-recognized standards and certifications which help students succeed in numerous high-growth, high-demand careers.

Evidence: WIA specifies Job Corps as a partner in the One-Stop system (section 121(b)(1)(B)) and Youth Councils (section 117(h)(2)): http://www.doleta.gov/usworkforce/wia/wialaw.txt; Sample Youth ChalleNGe MOA: Memorandum of Agreement Between Turner Job Corps Center and Fort Stewart Youth Challenge Academy; Sample Youth ChalleNGe MOU: Memorandum of Understanding Between Oklahoma Military Department and Tulsa Job Corps Center; and Memorandum of Understanding between the US Department of Education and the US Department of Labor (June 2001).

Does the program use strong financial management practices?

Explanation: In FY 2006, Job Corps Property was listed as a Reportable Condition by the independent auditor of DOL's financial statements for the third consecutive year. The Office of Job Corps has established proper controls to better ensure that real property is accurately reported to safeguard against loss or misuse and capital assets are accurately reported in DOL's financial statements. After reconciling differences between the Job Corps real property inventory and CATARS, the responsibility for maintaining the system was moved from ETA to OASAM due to the Congressionally directed transfer of Job Corps to the Office of the Secretary. The Office of Job Corps Support in OASAM now maintains the CATARS data and makes changes to the system based on the information received from the Architectural and Engineering contractor. For the past two years, Job Corps has reconciled each center's CATARS listing of real property through visual inspection and notes any changes to the report; these changes are verified by OASAM and entered into CATARS. All buildings are now assigned the same identifiers in CATARS and the Job Corps inventory. The Office of Financial Management Operations in OASAM reconciles the data to DOLAR$ and the financial statements. Job Corps' financial management and cost reporting system tracks expenditures against contract budgets to ensure funds are used for their designated purposes. To ensure effective administration of funds, a detailed financial management section containing instructions and requirements for budgeting and reporting costs has been added to the Job Corps PRH. These requirements provide Job Corps program managers with important information for efficiently managing resources. Public vouchers that detail reimbursable expense information are reported in the monthly Center Financial Report (2110). The DOL OIG audits a sample of Job Corps centers each year and audit findings are forwarded to the appropriate Regional Office for follow-up with the contractor, who is required to correct audit deficiencies, and in the case of a non-reimbursable cost determination, make financial restitution. Please see the FMS explanation provided under Section 3.3 for further details. Contractors report on their expenditures and explain any variance between their negotiated and actual budgets on a monthly basis via Job Corps' automated Financial Management System (FMS). The system includes an Advance Annual Procurement Plan/Financial Operating Plan, which records information about contracts and inter-agency agreements and project funding approvals.

Evidence: Annual Review Findings from 2006 Audits are presented in the KPMG, Independent Auditors' Report; Job Corps PRH, Appendix 502: Center Financial Management; Job Corps PRH, Appendix 503: Outreach/Admissions and Career Transition Services Financial Reporting; and Center Financial Report (2110) copy can be found in Question 3.3 Evidence.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: Job Corps is pro-active in identifying and addressing management deficiencies and areas for program improvement. Job Corps periodically examines its policies and procedures to ensure that they promote and support program goals, initiatives and objectives. Job Corps' PRH was thoroughly reviewed and revised in 2001 to align with the Career Development Services System (CDSS) framework of service delivery, to streamline policies, and to more clearly define expected program outcomes. Certain appendices (such as Appendix 501 for the performance management system) are updated annually and re-issued. PRH Changes Notices convey policy changes and are done on an as-needed basis as policy becomes outdated or needs additional clarification; PRH Changes are issued to the field electronically and are updated online as revisions occur. OASAM conducts Regional Office Procurement reviews to assess Job Corps' compliance with Federal and departmental acquisition regulations. As a result of these reviews, Job Corps incorporated additional procurement guidance into the new 2002 Job Corps Procurement Compendium (with a revised version released in June 2003) to improve the quality and consistency in procurement activity and has initiated Contractor Purchasing System Reviews to ensure that contractors spend Government funds effectively, efficiently, and in accordance with applicable policies. Annually, Job Corps analyzes its Outcome Measurement System (OMS) and in partnership with contractors, identifies new strategies, refines performance measures and goals, and identifies approaches to correct deficiencies. Job Corps also works diligently to address issues identified through external evaluations by OIG and GAO. For example, Job Corps established the following systems all designed to correct deficiencies identified by OIG and GAO: a sampling process to ensure applicant eligibility, procedures for initial placement verification, verification of placements by a third party, and new student transportation procedures. Reports by such external agencies are constructive in providing Job Corps with internal strategies to improve program efficiency and effectiveness. To help identify any issues related to the OMS and data integrity, Job Corps commissioned a Data Integrity Group (DIG). This group consists of data analysts, who track and scrutinize performance (at a detail level) to ensure that the high standards of the performance management system continue to be met. The team looks at data validity, integrity, and collection to achieve its mission of continuously improving program effectiveness as it relates to performance and data. Additionally, Job Corps encourages staff to attend regional- and center-level training sessions that focus on operational and developmental needs of staff (such as training on new policies and procedures) to ensure the most efficient delivery of services and most up-to-date knowledge of the tools and resources available to the field.

Evidence: USDOL, OASAM Compendium for Job Corps Procurements, Sections 1 through 5 (Mar 2007); Job Corps Program Information Notice No. 04-14, Change to the Program Assessment Guide (PAG) Reinforcing the Validation of Performance Data (Dec 2004); Sample Regional Office Center Assessment (ROCA), Philadelphia Job Corps Center Program Assessment (Jan 2006); Center Response, Region II Philadelphia Monitoring Trip Report (June 2006); and Examples of Training Provided to Job Corps Staff: Career Success Standards Expectation Tool (CS SET) presentation for Job Corps Spotlight Training 2007; Administering the Job Corps Student Network (JCSN) - hand-out materials for Trainees; CD/Academic/Career & Technical Skills Conference Agenda; and Presentation on New PRH Requirements and Guidelines regarding Eligibility and Suitability.

Is the program managed by maintaining clearly defined deliverables, capability/performance characteristics, and appropriate, credible cost and schedule goals?

Explanation: All design and construction contracts are firm fixed-cost and competitively bid, with the exception of small business contracts. Job Corps uses a professional Architectural and Engineering (A&E) Support firm to provide expertise on acquisition, design services, and construction and meets monthly with the A&E contractor to review the project status. The A&E contractor uses Microsoft Project to track each project for performance versus cost and schedule and notifies the Job Corps project manager of any difficulties that could potentially result in delays or cost overruns. The Facilities Management unit holds a monthly status meeting to ensure that projects remain on schedule. Additionally, on-site inspections and meetings with contractors that provide a detailed breakout of each project (showing the current status and cost) are held on a regular basis. Monthly status reports (that show the percentage of projects on schedule and within budget) are also provided to the Job Corps project manager. The A&E contractor does periodic on-site inspections of projects and conducts periodic facility surveys. As part of this process the contractor looks at regionally funded projects (smaller or emergency projects) to ensure that the work was finished and complies with applicable codes and quality standards. Job Corps staff also inspects projects, as needed. Operations costs are tracked by center as part of the performance measures of the Department's Asset Management Plan and the Real Property Scorecard. The costs are collected from the financial data submitted in the Financial Management System (FMS) and analyzed against benchmarks established by the International Facilities Management Association (IFMA). Centers significantly above the benchmark are targeted for analysis and an improvement plan. During the facility surveys, conducted every three years, the program's A&E firm reviews projects that have been funded through the Regional Office to assure that they were done in accordance with sound and cost-efficient practices. The projects include small repairs, such as lighting fixtures or window hardware that do not require the use of complex acquisition procedures. They also review repairs done in response to an emergency request, such as damage from a fire, wind or flooding, that were completed through small-scale local procurement. In addition to ensuring that the work was actually finished, the contractor also verifies that the work was accomplished in accordance with the applicable codes and met quality standards. The Department's OIG does annual reviews of the Job Corps program, including reconciliation of the capital asset inventory with the accounting data, verification of accountable property inventory and records, and review of invoice payments. In addition, the OIG staff audits approximately 10 to 12 centers per year with visual inspection and review of the most recent facility survey for that center.

Evidence: Monthly Design and Construction Status Report for March 2004; Hawaii/Maui Job Corps Center, Facility Planning Report Volume 1, Planning Strategy (May 2006); Annual Review Findings from 2006 Audits are presented in the KPMG, Independent Auditors' Report; Sample Project Milestone chart from Microsoft Project, Milwaukee, PY 02 New Center; and OIG FY 2007 Office of Audit Workplan (Oct 2006).

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: Due to a series of recent changes in measure definitions (Job Corps adopted the Common Measure indicators in PY 2004), it is difficult to accurately determine long term performance trends in the placement, credential attainment, and learning gains indicators. The program, however, has reported placement and educational achievement for a number of years, although results have been mixed. Placement in Employment or Education: Job Corps reported initial graduate placements, under GPRA, through PY 2002, and continued placement at 6-months (after initial placement) through PY 2003. The program was successful in meeting its initial placement targets for 2 out of the 3 reported years. While graduate initial placements dipped from roughly 90% in PYs 2000-2001 to 87% in PY 2002, recent data indicate this decline in graduate placements was a temporary fluctuation that occurs periodically (PYs 2003- 2004 initial graduate placement results again increased to the 90% range, while in PY 2005 performance dropped to 88%). Job Corps' performance on the long-term placement measure (continued placement at 6-months) has remained fairly consistent at 63-64% from PY 2001 through PY 2003, although this has fallen short of meeting the target. Since reporting under the modified definition of the Common Measures' placement indicator (see discussion in 2.2 and 2.4), Job Corps has not achieved its placement targets for PYs 2004 or 2005, achieving results of 84% and 80% respectively. Education/ Attainment of a Degree or Certificate: Overall, educational targets were reached three years (of four). Job Corps reported high school diploma attainment under GPRA during PYs 2002 and 2003. Performance for this measure was positive and Job Corps exceeded its target (of raising the number of HSDs attained by 20% over the previous program year) during both years. Job Corps met its certificate attainment target under Common Measures in PY 2004, although the program fell short in PY 2005, achieving a 60% result against the 64% goal. Literacy/Numeracy Gains: Job Corps has reported performance on literacy/numeracy gains since PY 2004. Historical data for this exact measure are not available, making it difficult to extrapolate two years performance into a long-term performance trend, although initial results have been positive with performance exceeding the annual targets (performance was 47% in PY 2004 to 58% in PY 2005).

Evidence: DOL FY 2006 PAR, DOL Office of Job Corps Annual Report." (PY 2005): http://www.dol.gov/_sec/media/reports/annual2006/PDF/2006annualreport.pdf. DOL Strategic Plan for FY 2006 - FY 2011 (http://www.dol.gov/_sec/stratplan/). DOL Employment and Training Administration Training and Employment Guidance Letter Number 17-05: Common Measures Policy (dated February 17, 2006): http://wdr.doleta.gov/directives/attach/TEGL17-05.pdf

Does the program (including program partners) achieve its annual performance goals?

Explanation: In the most recent year (PY05), Job Corps met one of its three annual goals. The program surpassed its goal (45%) of literacy or numeracy gains with a result of 58%. The program did not, however, meet its goals for placement in employment, education, or the military (79% versus the target of 85%); and attainment of a high school diploma, GED, or certificate while in the program (60% versus the target of 64%). Centers' performance largely mirrored these trends. In PY05, centers achieved the following under the Center Outcome Measurement System: On-center performance: 87.4% of the GED/HSD attainment goal, 89.7% of its vocational completion goal, 121.1% of its literacy gain goal and 127.1% of its numeracy gain goal. Short-Term Placement Performance: 103.1% of its JTM placement goal, 94.4% of its combined former enrollee and graduate initial placement goal, 92.7% of its graduate initial placement goal, and 101.0% of its graduate weekly earnings at initial placement goal. Long-Term Placement Performance: 97.0% of its 6-month placement goal, 95.5% of its 12-month placement goal, and 101.6% of its 6-month graduate earnings goal. Prior to reporting on Common Measures, Job Corps reported on annual goals that centered on the performance of the program's graduates. During PY 2005, graduate outcomes continued to remain positive: 76.1% attained a literacy gain; 78.4% attained a numeracy gain; 74.1% attained a high school diploma or GED; 93.4% completed their career technical trade (CTT); 64.5% attained a "combo" (a CTT completion and a GED or an HSD); 87.6% were placed in a job, the military or education program; and those who were employed earned an average hourly wage of $8.41.

Evidence: Office of Job Corps Reconciled Center Performance Reports: OMS10 and OMS10-R (PY 2005). Office of Job Corps Outcome and Admissions Performance Reports: OAOMS10 and OAOMS10-R (PY 2005). Office of Job Corps Career Transition Services Performance Reports: POMS10 and POMS10-R (PY 2005). DOL Strategic Plan for FY 2006 - FY 2011 (http://www.dol.gov/_sec/stratplan).

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: Regarding the program efficiency measure (cost-per-participant), Job Corps has not been able to demonstrate improved efficiencies or cost effectiveness in recent years. The program, however, has achieved savings in some operations in the most recent program year. Job Corps contractors and federal operators are pressured by ever-increasing prices for market goods and services, such as utilities, fuel, medical care, insurance, and other consumable goods and services. Efforts have been made by Job Corps to explore ways to maintain cost efficiencies in many of these areas, and implement projects that meet this goal. For example, Job Corps has reduced Federal Telecommunications System (FTS) costs by over $1 million through the use of IP voice over and improvements in technology, while expanding the use of video conferencing and e-learning on Centers. While energy costs have steadily risen, Job Corps has reduced consumption of energy through investment in energy saving projects, training of staff and students to control demand, an on-line system to review and analyze billing, and procurement of energy contracts in deregulated markets. Job Corps has also implemented many new cost savings strategies with respect to Facility Management. The Asset Management workgroup met to establish updated prototype standard designs using a questionnaire to determine current needs. The savings will be noticed in the areas of constructions costs and utilities usage. The Energy Report validates the progress Job Corps has made toward its commitments to EPACT and EO 13423 performance targets, and the progress in decreased energy use has resulted in cost savings. Through the purchase of utilities in deregulated markets, Job Corps has avoided approximately $1,000,000 in utility costs. Additionally, www.energywatchdog.com is a utility management tool that allowed Job Corps to receive a rebate of approximately $520,000 in FY06 for overcharged utility costs. The Job Corps renewable energy implementation plan for 2007 will invest in Renewable Energy, and solar energy panels will be installed at Job Corps centers and will generate energy for many years to come, thus helping to increase efficiency in energy use and reduce utilities expense. Energy Audits are conducted regularly to both identify the problem areas and propose solutions to remedy the problems at centers where energy usage is above the benchmark. Benchmarking operating costs also helps Job Corps stay in control of centers spending.

Evidence: FY 2006 Federal Agency Annual Report on Energy Management; Energy Management Implementation Plan for FY 2007; FY 2006 Federal Agency Energy Score Card; DOL Energy Score Card, January 2007; FY 2006 Energy Management Data Report; FY 2003 Energy Data Baseline Worksheet; FTS report (FY 2008 Working Capital Fund Budget Estimates Request as of December 15, 2006; FY 2006 WCF Totals for CJ's)

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: To a large extent, Job Corps compares favorably to other similar programs and demonstrations. While a recent Job Corps evaluation conducted by Mathematica found that "the benefits to society of Job Corps are smaller than substantial costs," a recent Youthbuild evaluation conducted by AREA, Inc. comparing Youthbuild to several similar programs concluded that Job Corps (although not cost-effective from a cost-benefit standpoint) was the only program to have positive impacts on educational attainment and earnings. A follow-up study by Mathematica determined that the post-program improvements to employment and earnings for the full sample of those participants studied did not persist. In summary, among evaluations of programs that were found to have little or no evidence of cost-effectiveness, Job Corps performs better than Youthbuild, the Job Start demonstration, JTPA, and another small-scale demonstration project. DOD's Youth ChalleNGE program is in the middle of a rigorous evaluation and study findings have yet to be published. When compared using performance measures, rather than impact analysis, Job Corps continues to, to a large extent, compare favorably. Measured by placement rate (80% of graduates and former enrollees in PY05), Job Corps is more successful than the WIA Youth Activities program (which had placement results of 58% in PY05), YouthBuild (which had a placement rate of 75% in 2005), and the Youth ChalleNGe program (which reported a 65% placement rate one month into the post-residential phase in its 2005 Performance and Accountability Highlights report). The ChalleNGe program had a higher HSD/GED attainment rate at 61% (during program participation), but when compared to the other youth programs Job Corps had more positive outcomes. In PY 2005, 42% of Job Corps students attained a HSD or GED prior to separating from the program, whereas both WIA Youth Activities and YouthBuild had a 36% attainment rate. While Job Corps is significantly more expensive on a per-participant and per-placement basis than most other youth programs, a 2003 AREA evaluation of YouthBuild (which is a nonresidential program) found that Job Corps costs slightly less per participant than YouthBuild ("Evaluation of the YouthBuild Program") .Other state and local models (for example, San Jose's Center for Employment and Training [CET] and its replication sites) have shown positive results although they serve a population that is significantly older (CET serves low-income, disadvantaged youth 17 ?? years old through adults in their 60s) and who typically have more positive outcomes.

Evidence: Mathematica Policy Research, Inc. "National Job Corps Study and Longer Term Follow-up Study" (August 2006); Mathematica Policy Research, Inc. "Summary of the National Job Corps Study" (June 2001): ): http://wdr.doleta.gov/opr/fulltext/01-jcsummary.pdf; San Jose Center for Employment and Training: http://www.cet2000.org/index.php; DOL Strategic Plan for FY 2006 - FY 2011 (http://www.dol.gov/_sec/stratplan).

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: A series of studies have been conducted by independent evaluators to assess the effectiveness and overall impact of Job Corps. The most recent 2006 update of an ongoing longitudinal evaluation of Job Corps participants (conducted by Mathematica) stated the program, as a whole, does not appear to be cost-effective, and found no statistically significant difference in the 16-20 age group. The study does note, though, that Job Corps is the "only federal training program that has been shown to increase earnings for this population for several years after program exit" for participants in the 20-24 age group. The 2001 report found that, compared to the control group, Job Corps program group members had more positive outcomes in terms of: functional literacy; GED attainment (42% vs. 27%) and vocational certificates (38% vs. 15%); average weekly earnings ($16 higher) by year 4; and received fewer public benefits (but had slight reductions in HSD attainment). No employment and earnings gains were found for Hispanics and 18-19 year olds. In 2003, an update of the study with 10 years of earnings-related administrative data was released that indicated that Job Corps has no long-term impact for the full program group on employment or earnings, as was originally reported in the 2001 study. In 2006, the final report was issued that interpreted the main evaluation results from the past 12 years. The 2006 study found that the effect of the higher earning gains experienced in the first 2 years after program exit gradually decreases, so that by year 10 there is no statistical significance between control and program groups, except for older students (20-24 year olds) where the effect persisted through years 5-10, producing a net benefit of $16,853. A study by HMA Associates researched factors (language, culture and community support networks) that can affect the post-program economic impact of Hispanics in Job Corps. The study found no apparent language barriers, and that Hispanics were more likely to take the first job they found (regardless of salary and whether it was training-related) due to family-related obligations to obtain work as quickly as possible. Family and cultural ties were identified as very important and close relatives could impact successful program completion. Job Corps has also commissioned analytic reports to review program and placement outcomes. Studies of placement retention rates by age groups found, as compared to 20-24 year olds, 16-17 year olds are one-third less likely, while 18-19 year olds are 10-13% less likely, to remain in a placement at 6- and 12-months after initial placement. Male participants are 24% more likely to remain in a placement at 6-months and 34% more likely at 12-months than female participants. Analyses of credential attainment indicate that females, older students, and those with more years in school prior to Job Corps generally have higher credential attainments then the rest of the population. Females and older students are more likely to graduate from Job Corps.

Evidence: Mathematica Policy Research, Inc., "National Job Corps Study: Findings Using Administrative Earnings Records Data", (Oct 2003); Summary Report: Mathematica Policy Research, "Does Job Corps Work? Summary of the National Job Corps Study" (June 2001): http://wdr.doleta.gov/opr/fulltext/01-jcsummary.pdf; Mathematica Policy Research, Inc., "National Job Corps Study and Longer-Term Follow Up Study: Impact and Benefit-Cost Findings Using Survey and Summary Earnings Records Data" (Aug 2006). Memorandum to the Assistant Secretary for Employment and Training, dated 9/29/2005 with Foreword from the Assistant Secretary of Labor for Employment and Training; DIR Analytic Reports: "Placement Retention Outcomes of 16- to 17-Year-Old Placed Graduates" (Aug 2004); "Age Effects on Post-center Outcomes: First Year Continued Employment" (June 2005); "Gender Differences in On-Center Outcomes: Vocational Completion and GED Attainment" (Feb 2006); "Ethnicity Differences in On-Center Outcomes: Vocational Completion and GED Attainment" (Feb 2006); "Gender Differences in Post-Center Outcomes: First-Year Placement Credit at 6 and 12 Months" (Feb 2006); "Graduation Rates by Gender, Ethnicity and Age Subgroups" (Dec 2006); HMA Associates, "Study of Hispanics in Job Corps, 2004-2005", (April 2006).

Were program goals achieved within budgeted costs and established schedules?

Explanation: The program achieved just one of the four annual performance goals discussed in Section 2. Additionally, the program has not yet provided data to determine the extent to which projects were completed within baseline budgeted costs and established schedules. OJC expects that 92% of projects started in PY05 and PY06 will be completed within the baseline budget, and that 77% of projects will be completed within 10% of the scheduled completion date, but did not provide data on actual performance. The program's change order rate is 6.2%, which is well within the industry standard of 5 - 10%.

Evidence: Monthly Design and Construction Status Report for March 2, 2007; PY 2006 Job Corps CRA Budget Summary (PY 2006 - PY 2011 proposed); Sample Project Milestone chart from Microsoft Project, Milwaukee, PY 02 New Center; Job Corps Exhibit 300s for FY 2008: http://www.dol.gov/dol/budget/2008/PDF/E300-2008-041.pdf; Job Corps Capital Asset Plan; DOL Energy Score Card, January 2007; FY 2006 Chart of Progress to EO 13123 Goal.