Detailed Information on the
Employment Service Assessment

Working with the Congress to consolidate this and other workforce programs, so that States and local communities receive one funding stream and have greater ability to be innovative and effective.

Tracking the levels of self service participants versus staff assisted participants to improve workforce system integration.

Improving reporting efficiency, program management and accountability through the collection of new information with WISPR, a common reporting system for WIA, TAA, and Wagner Peyser Programs.

Adopting efficiency measures that are linked to performance outcomes, account for all costs, and facilitate comparisons across Department of Labor training and employment programs.

Collecting new performance data, including earnings, outcomes by type of service, and new categories of information on veterans receiving employment services.

Negotiating performance outcomes with State grantees, using the common employment outcome performance measures used in the Workforce Investment Act programs.

Measure: Entered Employment Rate.

Explanation:Percent of participants employed in the first quarter after exit. This is a Federal job training program common measure, which enables the Wagner-Peyser Act funded Employment Service (ES) to describe in a similar manner the core purposes and results of the program compared to other education, employment and job training programs. For example, while ES provides universal access through one stop career centers and electronic tools, the ultimate outcomes for this program are the same as other more targeted employment and training programs. Common measures remove a barrier to service integration among programs by ensuring that programs no longer have different definitions and methodologies for measuring performance. In this case, the performance indicator measures how many participants got a job according to the following formula: Of those who are not employed at the date of participation - The number of participants who are employed in the first quarter after the exit quarter divided by the number of participants who exited during the quarter. This definition for entered employment was adopted in PY 2005, while prior years used a similar methodology for calculating the result. Experience shows actual performance results for this measure tend to fluctuate from year to year. A one percent target increase for PY 2007 would equate to an estimated 46,864 more people who enter employment in the first quarter after exit in PY 2007 over PY 2006 results, if the number of people exiting in PY 2007 remained the same as PY 2006.

Measure: Employment Retention Rate.

Explanation:Percent of participants employed in the first quarter after program exit still employed in the second and third quarters after exit This is a Federal job training program common measure, which enables the Wagner-Peyser Act funded Employment Service (ES) to describe in a similar manner the core purposes and results of the program compared to other education, employment and job training programs. For example, while ES provides universal access through one stop career centers and electronic tools, the ultimate outcomes for this program are the same as other more targeted employment and training programs. Common measures remove a barrier to service integration among programs by ensuring that programs no longer have different definitions and methodologies for measuring performance. In this case, the performance indicator measures how many participants retained their employment once placed in a job, according to the following formula: Of those who are employed in the first quarter after the exit quarter - The number of participants who are employed in both the second and third quarters after the exit quarter divided by the number of participants who exited during the quarter. This definition for employment retention was adopted in PY 2005, and made the measure more challenging by including an additional quarter in which a participant needs to be employed. Experience shows actual performance results for this measure tend to fluctuate from year to year. A one percent target increase from PY 2006 to PY 2007 would equate to an estimated 69,048 more people who retained employment in the second and third quarters after exit in PY 2007 over PY 2006 results, if the number of people exiting in PY 2007 remained the same as PY 2006.

Measure: Average earnings.

Explanation:The average six-month earnings This is a Federal job training program common measure, which enables the Wagner-Peyser Act funded Employment Service (ES) to describe in a similar manner the core purposes and results of the program compared to other education, employment and job training programs. For example, while ES provides universal access through one stop career centers and electronic tools, the ultimate outcomes for this program are the same as other more targeted employment and training programs. Common measures remove a barrier to service integration among programs by ensuring that programs no longer have different definitions and methodologies for measuring performance. In this case, the performance indicator measures participants' average six-month earnings once placed in a job (note: the average earnings for a year can be obtained by doubling the performance measure result), according to the following formula: Of those participants who are employed in the first, second, and third quarters after the exit quarter - Total earnings in the second quarter plus total earnings in the third quarter after the exit quarter divided by the number of participants who exited during the quarter. The common measure definition for average earnings was adopted in PY 2006 - just one year after establishing a baseline for another measure: earnings change. In PY 2008, participants evaluated for the earnings measure are expected to earn 2.5% more per year than participants represented in the PY 2007 result. This means that an individual would earn $24,334 annually, $2,028 monthly, or $507 on a weekly basis.

Measure: Cost per participant

Explanation:The program calculates the average cost for each participant by dividing the total annual appropriation for the program by the number of participants.

Is the program purpose clear?

Explanation: The Wagner-Peyser Act of 1933, as amended by the Workforce Investment Act of 1998 (WIA), created the Employment Service (ES) to bring together individuals who are seeking employment and employers who are seeking workers. Through ES grants to States, labor exchange services are provided at no charge through a nationwide network of public One-Stop Career Centers, with specialized services for migrant and seasonal farm workers, Unemployment Insurance (UI) claimants, and veterans. In addition, the ES administers the Work Opportunity and Welfare-to-Work tax credit programs, provides foreign labor certification services to employers, administers the UI work test (the process of ensuring that UI claimants register with the ES and search for work), and provides reemployment services to UI claimants who are identified as having high probabilities of exhausting their benefits.

Evidence: (1) Authorizing law: Wagner-Peyser Act of 1933, accessible at www.doleta.gov/programs/w-pact_amended98.cfm;(2) The Workforce Investment Act of 1998, accessible at www.doleta.gov/usworkforce/wia/wialaw.pdf, amended the Wagner-Peyser Act;(3) 20 CFR 652.2-3 (Scope and purpose of the employment service system and basic labor exchange system); and(4) 20 CFR 652.210 (Wagner-Peyser Act requirements for UI work test administration and services to UI claimants).

Does the program address a specific and existing problem, interest or need?

Explanation: While the ES was created in the Depression, there is a continuing demand for its services. For program year (PY) 2002, over 15 million job seekers registered with the ES, and employers listed over 10 million job openings. What has changed over time is the way the labor exchange services and labor market information are delivered, with increasing reliance on computers and the Internet. ES services are provided in three ways: self-service, facilitated self-help, and staff-assisted. In each local workforce investment area, all ES services are available on-site in at least one comprehensive One-Stop Career Centers. Labor exchange services can also be accessed electronically, through Internet-based self-service tools, known as the CareerOneStop, available 24-hours a day, 7 days a week. The CareerOneStop tools include America's Job Bank, America's Career InfoNet (information on labor markets and job skills requirements), and America's Service Locator (locations of local offices).

Evidence: (5) Employment Service Program Year 2002 Data, ETA 9002 A Quarterly Report dated 12/31/03, shows that over 15 million jobseekers registered with the ES;(6) CareerOneStop (COS) e-tools web sites: CareerOneStop (COS) - www.careeronestop.org/ America's Job Bank (AJB) - www.ajb.org/ America's Career InfoNet (ACINet) - www.acinet.org/ America's Service Locator (ASL) - www.servicelocator.org/Occupational Information Network (O*NET) - online.onetcenter.org/

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: The WIA Adult and Dislocated Worker programs provide similar core services as those funded under the Wagner-Peyser Act to the unemployed, the under-employed, and the economically disadvantaged, as well as to businesses seeking qualified workers. The Administration's proposal for WIA reauthorization would eliminate duplication in the delivery of core employment and workforce information services among One-Stop program partners and provide flexibility to States in determining how delivery of such services will be administered. In addition, private entities provide some of the services that the ES has been delivering but with a fee charged either to the job seeker, to the employer, or both.

Evidence: (7) Workforce Investment Act of 1998 is accessible at www.doleta.gov/usworkforce/wia/wialaw.pdf, amended the Wagner-Peyser Act;(8) 20 CFR 652.2-3 (Scope and purpose of the employment service system and basic labor exchange system);(9) Private site job posting price lists are available at get.monster.com/ecomproduct.asp?bundleTypeID=1&ecomad=emplogin,jobs,www.careerbuilder.com/JobPoster/?sc_cmp2=JS_HP1_Nav_PostJobNow, www.hotjobs.com/htdocs/employer/index-us.html.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: In addition to overlap and duplication with certain WIA core services in some communities, the requirement that ES be a One-Stop partner for delivery of services imposes the rigidities of the ES grants on the One-Stop operators as do separate WIA funding streams. One-Stop operators must track and manage separate funding streams and manage employees in separate personnel systems. Additionally, the Secretary of Labor has long required ES services to be delivered by State employees ('merit staff') to ensure impartiality and statewide service delivery to all job seekers and employers. The Administration's proposal to consolidate the ES with the WIA Adult and Dislocated Worker programs is designed to expand State flexibility and assist in the further elimination of duplication of effort that exists. The expected results will be an expansion of seamless local service delivery and improved efficiency.

Evidence: (10) Wagner-Peyser Act Annual Funding Agreement Assurance 1.5 "Grantee will comply with the Intergovernmental Personnel Act of 1970 (42 U.S.C. 4728) relating to prescribed standards for merit systems for programs funded under one of the nineteen statutes or regulations specified in Appendix A of OPM's Standards for a Merit System of Personnel Administration (5 CFR 900, Supart F)."

Is the program effectively targeted, so that resources will reach intended beneficiaries and/or otherwise address the program's purpose directly?

Explanation: Wagner-Peyser funded services are provided universally to the general population and are not 'targeted' to any special population. Funding is distributed to States via formula grants based on each State's share of the civilian labor force and unemployment for the current calendar year. Of the 15 million registered job seekers the ES served in PY 2002, 88% or 13 million were unemployed at the time of registration and 42% or 6.2 million were eligible UI claimants. In addition, businesses listed 10.2 million job openings with the State workforce agencies and America's Job Bank.

Evidence: (11) 20 CFR 652.207 (How does a State meet the requirement for universal access to services provided under the Wagner-Peyser Act?);(12) TEGL 23-02 titled 'FY 2003 Congressional Rescissions for WIA Adults and Dislocated Workers, Workforce Investment Act (WIA) Allotments for Program Year (PY) 2003, Wagner-Peyser Act Final Allotments for PY 2003, Reemployment Services Allotments for PY 2003, and Work Opportunity and Welfare-to-Work Tax Credit Allotments for FY 2003', is accessible at www.ows.doleta.gov/dmstree/tegl/tegl2k2/tegl_23-02.htm, includes an explanation of the ES funding formula.(13) Employment Service Program Year 2002 Data, ETA 9002 A Quarterly Report dated 12/31/03 shows the UI status of the job seekers that registered for ES services and the number of job openings.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The program is part of the Job Training Common Measures initiative. Accordingly, it has adopted four long-term goals that will better measure the impacts of the program than current measures and allow comparisons across similar programs. DOL is implementing the common measures and is to establish numerical targets in PY 2004. The Administration's WIA reauthorization proposal identifies the common measures and applies them to a new comprehensive adult program to be created by consolidating this program with the current WIA Adult and Dislocated Worker State grant programs. Current ES labor exchange measures include: Entered Employment Rate, Employment Retention Rate, Job Seeker Customer Satisfaction, and Employer Customer Satisfaction. These are similar to measures currently used in the WIA programs but do not include earnings gains. In addition, ES has measures designed to assess the performance of the electronic, Internet-based services, which do not require registration and consequently do not have information on labor market outcomes of those users.

Evidence: (14) Department of Labor's adoption of the Job Training Common Measures is evidenced in its FY 2000-2004 Strategic Plan, accessible at www.dol.gov/dol.aboutdol/main.htm; (15) Labor Exchange Performance Measures, 66 FR 29848-53, May 31, 2001, and Federal Register, Vol. 66, No. 109, June 6, 2001;(16) Guidance and information about the labor exchange performance measures, and performance measures reporting and data validation can be accessed at www.doleta.gov/usworkforce/documents/

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: This program is part of the Job Training Common Measures Initiative. Accordingly, it has adopted four long-term annual goals that will enhance the measurement of the program's impact (e.g., wage gain) and allow for comparisons across similar programs. DOL is implementing the OMB common measures for employment and training programs in calendar year 2004. Targets and timeframes for the program's long-term measures will be negotiated during calendar year 2004.

Evidence: (17) Department of Labor's FY 2000-2004 Strategic Plan at www.dol.gov/dol.aboutdol/main.htm reflects that the agency has adopted OMB's common performance measures for employment and training programs;(18) Department of Labor's FY 2004 and FY 2005 Congressional Justifications are accessible at www.dol.gov/_sec/Budget2004/2004app-toc.htm and www.dol.gov/_sec/Budget2005/overviewpb-toc.htm, respectively. Both justifications reflect the Department's commitment to the implementation of OMB's common measures.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: This program is part of the Job Training Common Measures initiative. Accordingly, the program has adopted new annual measures that: (1) will better indicate participants' employment and earnings outcomes, as well as program efficiency, and (2) facilitate comparisons with similar programs. DOL is implementing the common measures and will establish numerical targets in 2004. The Administration's WIA reauthorization proposal identifies the common measures and applies them to a new comprehensive adult program to be created by consolidating this program with the current WIA Adult and Dislocated Workers programs. The program is considering developing a measure to more directly capture business service outcomes.Currently, the ES labor exchange annual performance measures are: Entered Employment Rate (EER), Employment Retention Rate (ERR), Job Seeker Customer Satisfaction, and Employer Customer Satisfaction. The Internet-based CareerOneStop tools have performance measures that reflect their goals and objectives.

Evidence: (19) Department of Labor's FY 2000-2004 Strategic Plan at www.dol.gov/dol.aboutdol/main.htm reflects the agency's adoption of OMB's Job Training Common Measures;(20) Department of Labor's FY 2004 and FY 2005 Congressional Justifications are accessible at www.dol.gov/_sec/Budget2004/2004app-toc.htm and www.dol.gov/_sec/Budget2005/overviewpb-toc.htm, respectively. Both justifications reflect the Department's commitment to the implementation of OMB's Job Training Common Measures.(21)Labor Exchange Performance Measures, 66 FR 29848-53, May 31, 2001;(22) Guidance and information about the labor exchange performance measures, and performance measures reporting and data validation can be accessed at www.doleta.gov/usworkforce/documents/;(23) FY 2004 Revised and Final Annual Performance Plan is accessible at www.dol.gov/dol/aboutdol/main.htm; (24) ETA's Quarterly Report of Workforce System Results is accessible at www.dol.gov/_sec/media/reports/annual2003/.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: DOL is implementing the Job Training Common Measures and is to establish baseline data in PY 2004 and then ambitious numerical targets for the program. ES' annual measures reflect its long-term measures. The current annual measures are: Entered Employment Rate (EER), Employment Retention Rate (ERR), Job seeker Customer Satisfaction, and Employer Customer Satisfaction. In PY 2003, ES established numerical targets for its annual measures based on its experience, the results of a joint pilot with the Veterans' Employment and Training Service (VETS) using UI wage records, and state-developed baselines during PY 2002. In PY 2004, States will be setting levels of performance for the program's annual measures based on GPRA goals and actual performance data.

Evidence: (25) The results of the ES/VETS Pilot Test of Labor Exchange Performance Measures showed that Entered Employment Rate (EER) ranged between 60%-65% and that the Employment Retention Rate (ERR) ranged between 75%-80%, which are in alignment with ES GPRA goals. The states used these results to establish ES numerical targets for the annual measures.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: For the ES formula grants, prior to PY 2004 and the implementation of the labor exchange performance measures, States did not negotiate outcome levels of performance for ES. But as a result of a joint Federal-State workgroup, in PY 2004, States will negotiate numerical targets for labor exchange for the first time. This process will be continued annually.

Evidence: (26) Training and Employment Guidance Letter (TEGL) 22-03, titled "Negotiating PY 2004 Expected Levels of Performance for the Labor Exchange Entered Employment Rate and Employment Retention Rate Performance Measures" (06/03/04), is accessible at wdr.doleta.gov/directives/corr_list.cfm.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: Numerous independent evaluations have been conducted to support ES program improvements and evaluate the effectiveness and efficiency of its services. As a synthesis of these studies, Chapter 5 in Labor Exchange Policy in the United States examined nine studies of the ES conducted over the last 20 years. All studies involved a comparison group design which permits an estimate of the incremental effect of an intervention. The methodology, called net impact estimation, compares post-program labor market outcomes of participants with a group of similar characteristics. Evaluations of ES activities have focused on three topics: job interview referrals, job search assistance, and targeted job search assistance. The review of the ES studies was prepared by a staff member of the W.E. Upjohn Institute for Employment Research, an independent research organization, and the studies reviewed had been conducted by independent researchers.

Evidence: (27) Labor Exchange Policy in the United States, W.E. Upjohn Institute for Employment Research (2004) examined the results of nine independent evaluations of ES activities.-Additional independent studies include:(28) UI Work Search Rules and Their Effect on Employment, W. E. Upjohn Institute for Employment Research (February 2004), accessible at www.workforceatm.org/sections/pdf/2004/UI_Work_Search.pdf;(29) Measuring the Effect of Public Labor Exchange (PLX) Referrals and Placements in Washington and Oregon, Westat (October 2003), accessible at workforcesecurity.doleta.gov/dmstree/op/op2k/op_06-00.pdf;(30) Evaluation of Public Labor Exchange (PLX) Services in a One-Stop Environment: New Evidence from North Carolina, Westat (June 2002);(31) The Effectiveness of the U.S. Employment Service, prepared for the Advisory Commission on Unemployment Compensation, Westat (March 1994);(32) America's Job Bank: Outcome Study, TATC (March 2002).*Additional studies evaluating the effectiveness of the ES in the past 20 years have reported low costs per customer served. These evaluations are available in hard copy by contacting the ES at 202-693-3046.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: There is little evidence that program funding and achievement of performance goals are related. The current extent of program performance and cost integration does not yet permit an assessment of the impact of budget levels, or policy or legislative changes on performance outcomes. Nonetheless, DOL's FY 2005 Performance Budget and previous budget requests have followed OMB Performance Budgeting guidelines, providing an overview of the Secretary's priorities, strategic goals, and outcome goals. This information is presented in a transparent manner and is posted on the Internet for public scrutiny and use at www.doleta.gov/budget/. DOL has also made progress in providing full program costing.

Evidence: (33) Department of Labor's FY 2004 and FY 2005 Budget Requests are accessible at www.dol.gov/_sec/Budget2004/2004app-toc.htm and www.dol.gov/_sec/Budget2005/overviewpb-toc.htm, respectively.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The major strategic planning deficiency in ES has been the lack of an approach that rewards good performance outcomes balanced by an approach that would sanction poor performers, which is embodied in WIA. The Administration's proposal for a single consolidated funding stream for the ES and WIA programs would apply the incentives and sanctions of WIA. As noted above, DOL implemented a program to measure ES outcome performance with revised reporting requirements in PY 2002. In PY 2004, for the first time, ES and States will negotiate expected levels of performance for entered employment rate (EER) and employment retention rate (ERR). These measures are similar to the WIA entered employment and retention performance indicators, but have a shorter time frame in order to measure how quickly the job seeker finds work. The measurement period for EER and ERR is strictly limited to the two quarters following the job seeker registration quarter. Job seekers entering employment after that period cannot be included in the measurement.

Evidence: (34) Department of Labor's FY 2003-2008 Strategic Plan and Department of Labor's FY 2005 Annual Performance are accessible at www.dol.gov/dol/aboutdol/main.htm, and propose a consolidated funding stream for the ES and WIA Adult/Dislocated Worker programs;(35) Department of Labor's FY 2004 and FY 2005 Budget Requests are accessible at www.dol.gov/_sec/Budget2004/2004app-toc.htm and www.dol.gov/_sec/Budget2005/overviewpb-toc.htm, respectively, and propose a consolidated funding stream for the ES and WIA Adult/Dislocated Worker programs.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: ETA staff monitor grantee performance and financial practices through analyses of the quarterly data and regular on-site reviews. Every fiscal year Federal staff are required to review 50% of formula grantees as well as all grant activities identified as high risk. If necessary, Federal staff and grantees jointly develop corrective action plans to address program deficiencies. To assist them in grants management, Federal staff developed the Grants E-Management System (GEMS), an online internal tool that helps identify immediate and potential risks in grant activities and provides access to all Enterprise Information Management System (EIMS) components.

Evidence: (36) Performance Management Plans for Managers and Supervisors (07/03) prescribes the parameters for grant management activities--which includes the frequency of desk and on-site grant reviews; (37) Employment and Training Order No. 1-03 titled 'Improving Administration of Grants within the Employment and Training Administration' (04/03) provides Federal staff with guidance on effective grants management; (38) Labor Exchange Reporting System (LERS) and federal data systems such as the EIMS, Dashboard, and Grants E-Management System (GEMS) of the Employment and Training Administration are tools that assist Federal staff in conducting grants management activities.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Accountability for performance results by program partners (i.e., grantees) is insufficient. The ES block grants to States have little accountability for cost or performance. Unlike the WIA programs, there are no incentive grants or sanctions for States regarding negotiated levels of performance. Federal managers are held accountable. Performance ratings for national and regional office managers and supervisors are tied to the achievement of DOL and program-specific goals and outcomes. Managers' performance standards are reflected in the performance plans for non-managerial staff. Each year, DOL programs identify broad strategies and anticipated levels of performance for its programs, which are published in its Annual Performance Plan. Federal staff conduct mid-year and annual reviews of programs to gauge success in achievement of annual GPRA goals and progress on implementation of program-specific strategies.

Evidence: (39) Performance Management Plans for Managers and Supervisors (07/03); (40) Employment and Training Order No. 1-03 titled 'Improving Administration of Grants within the Employment and Training Administration' (04/03); (41) Sample Wagner-Peyser Act Annual Grant Agreement includes the performance and financial report requirements for ES grantees; sSamples are available from ETA at (202) 693-3046; (42) Sample AJB Cost Report Spreadsheet is submitted by the AJB grantee to demonstrate how resources have been obligated and expended; samples are available from ETA at (202) 693-3046.

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: All grantees obligate and spend funds for the intended purpose in a timely manner. DOL provides base grant planning targets to grantees before the beginning of each program year. Spending authority is provided immediately following OMB's apportionment of appropriations. Grantees submit quarterly financial status reports for obligations and expenditures. Expenditures are monitored by Federal staff.

Evidence: (43) TEGL 23-02 titled 'FY 2003 Congressional Rescissions for WIA Adults and Dislocated Workers, Workforce Investment Act (WIA) Allotments for Program Year (PY) 2003, Wagner-Peyser Act Final Allotments for PY 2003, Reemployment Services Allotments for PY 2003, and Work Opportunity and Welfare-to-Work Tax Credit Allotments for FY 2003', accessible at www.ows.doleta.gov/dmstree/tegl/tegl2k2/tegl_23-02.htm;(44) 20 CFR 652.8 (Administrative provisions for Wagner-Peyser Act grants) requires grantees to have financial and program management systems in place;(45) Sample Wagner-Peyser Act Annual Grant Agreement; samples are available from ETA at (202) 693-3046;(46) One-Stop Comprehensive Financial Management Technical Assistance Guide is accessible at www.doleta.gov/sga/pdf/FinalTAG_August_02.pdf.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: There are no procedures in place to improve the efficiency or cost effectiveness of the ES formula grants, which account for the bulk of ES funding. Such procedures are left to the discretion of the grantees. In program areas where ETA does not use formula grants, it is able to take steps to improve efficiency. Per ETA's guidance, the grantee for America's Job Bank (AJB) regularly contracts with consultants to perform independent reviews and evaluations of its processes and to recommend improvements to achieve efficiencies. ETA requires the AJB grantee to renegotiate its vendor contracts on a periodic basis to ensure ensure vendor services are available at the most cost-effective rate. The most recent contract renegotiation resulted in a 20% reduction in the cost of labor rates. ETA is also working with the Chief Financial Officer on developing a cost accounting system, Cost Accounting Manager (CAM), that can provide data on resource use for cost efficiency analysis.

Evidence: (47) Phase 1: Data Center Sourcing Strategy, Executive Summary Prepared for America's Job Bank Service Center, Gartner Consulting (November 2001) is an independent review commissioned by America's Job Bank grantee; (48) Cost Accounting Manager (CAM) will enable ETA to achieve its performance goals in a more cost-effective manner through strategic deployment of staff and other resources.

Does the program collaborate and coordinate effectively with related programs?

Explanation: WIA made ES a required partner in each One-Stop Career Center. Through memoranda of understanding, local workforce investment boards and States negotiate the deployment of ES staff and resources at the local level. States provide unified plans to DOL addressing their employment and training services under the ES as well as the WIA Adult and Dislocated Worker programs.Wagner-Peyser-funded services are an integral component of the 1,900 comprehensive One-Stops. For many job seekers, ES is the point of entry to the intensive services (e.g., specialized assessment of skills levels, individual counseling, case management) and training provided by WIA and other One-Stop partners. In some areas, ES augments the services provided in the comprehensive One-Stops through satellite offices.

Evidence: (49) Federal Register, Vol. 64, No. 37, 2/24/99, Planning Guidance and Instructions for Submission of the Strategic Five-Year Plan for Title I of the Workforce Investment Act of 1998 and the Wagner-Peyser Act available at www.access.gpo.gov/su_docs/fedreg/a990225c.html;(50) 20 CFR 652.8 (Administrative provisions for Wagner-Peyser Act grants) requires grantees to have financial and program management systems in place;(51) Workforce Investment Act, Section 121(c) Memorandum of Understanding at www.doleta.gov/usworkforce/wia/wialaw.pdf provides guidance to One-Stop partners on allocation of resources, how to structure service delivery, etc.;(52) One-Stop Comprehensive Financial Management Technical Assistance Guide at www.doleta.gov/sga/pdf/FinalTAG_August_02.pdf; www.servicelocator.org/partners.

Does the program use strong financial management practices?

Explanation: The ES program is free of material internal control weaknesses. However, DOL's Office of the Inspector General (OIG) as recently as January 2004 identified grant accountability, performance, and effectiveness among the most serious management challenges facing DOL. Direct Federal oversight of grants for employment and training activities is difficult because a large share of the funding is passed down through the States to sub grantees and contractors. The OIG recognized that the Employment and Training Administration (ETA) took a significant step in April 2003 when it issued an order implementing its Grant/Contract Administration Plan. Employment and Training Order No. 1-03 is designed to improve grantee accountability, compliance, and performance. In carrying out the order, ETA has issued a Regional Accounting Policies and Procedures Manual and is making improvements to post-award grant monitoring, among other things. The OIG is encouraged by ETA's early efforts, but in the meantime, OIG audits continue to highlight this area of concern.

Evidence: (53) OIG's Top Management Challenges Facing the U.S. Department of Labor report (January 2004) is accessible at www.oig.dol.gov/doltopchall.pdf; (54) Employment and Training Order No. 1-03 titled "Improving Administration of Grants within the Employment and Training Administration" (04/03).

Has the program taken meaningful steps to address its management deficiencies?

Explanation: ES has taken steps to ensure that data collection and reporting procedures are consistent with ETA's performance accountability initiatives, including: revision of reporting requirements, implementation of new data collection and reporting systems (LERS, LEARS), creation of an Internet-based grants management tool (GEMS), development of report validation software, and participation in ETA's data element validation initiative. In addition, Federal staff provide technical assistance to grantees when deficiencies are identified in any aspect of ES funds administration, reporting, performance, or financial operations. The Administration's WIA reauthorization proposal to consolidate the ES with WIA Title I adult programs would streamline administrative procedures such as data collection and reporting systems.

Evidence: (55) Guidance and information relative to the new labor exchange reporting system can be accessed at www.doleta.gov/usworkforce/documents/;(56) Guidance for the ES Report Validation System developed in partnership with Mathematica Policy Research is accessible at 166.97.195.30/rptvalidation.cfm

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: ES grantees submit financial data quarterly. Grantees report performance data through the data collection and reporting tools contained in ETA's Enterprise Information Management System (EIMS). Federal staff utilizes the Grants E-Management System (GEMS) as a comprehensive grant management tool. GEMS tracks and documents grant management activities, including on-site monitoring and technical assistance reviews. It also includes a risk assessment module and a case notes tool. Grantee performance and financial practices are monitored by Federal staff through data analyses and regular on-site reviews. Additionally, Federal staff participates in bi-weekly conference calls on performance management. Every fiscal year Federal staff are required to review 50% of formula grantees as well as all grant activities identified as high risk. If necessary, Federal staff and grantees jointly develop corrective action plans to address program deficiencies. Grantees, in turn, monitor sub-grantees in the accomplishment of Federal and state performance measures and fiscal responsibility.

Evidence: (57) See Evidence for 3.2

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: Standard performance and financial reports are collected on a quarterly and annual basis. Grantees submit performance information on a quarterly basis using a 'rolling 4 quarter' format. Grantee performance is summarized quarterly and the data are made available to the public via the Internet and through the Quarterly Report Workforce System Results, although the ability of the public to locate the report could be improved. In addition, DOL produces the ES Annual Report, which analyzes performance across employment services, including the CareerOneStop tools and labor market and workforce information. DOL Annual Reports are published and posted on the DOL website for public accessibility.

Evidence: (58) Guidance and information relative to the new labor exchange reporting system can be accessed at www.doleta.gov/usworkforce/documents/; (59) National and state data for PY 2000 and PY 2001 in the Department of Labor's Annual Performance Reports can be viewed at www.uses.doleta.gov/arp01/; (60) State-by-State ES Data are accessible at www.uses.doleta.gov/arp01; (61) ETA's Quarterly Report of Workforce System Results is accessible at www.dol.gov/_sec/media/reports/annual2003/.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: The program has adopted four new long-term goals as part of the Job Training Common Measures initiative. DOL is to establish numerical annual and long-term targets in PY 2004. Partial credit is based on the fact that DOL achieved its previous long-term goals.In 2001, the ES established four labor exchange performance measures: Entered Employment Rate; Employment Retention Rate, and Job Seeker and Employer Customer Satisfaction. UI wage records and State electronic databases are the only allowable sources of outcome data. States began reporting under the new performance measures reporting system in PY 2002 (July 2002). Entered Employment and Employment Retention results reported to ETA for the second and third quarters of PY 2003 indicate that ES is meeting its goals. The Entered Employment Rate reported in the second and third quarters of PY 2003 is 61% and 62%, compared with the target of 58%. Employment Retention Rate results were first available in May 2004 for performance reported in the third quarter of PY 2003--and the result of 79% exceeds the target of 72%.

Evidence: (62) National Summary - Placement and Entered Employment Rate provides a seven year study (1995-2001) of placement and entered employment rates for the ES;(63) ETA's Quarterly Report of Workforce System Results at www.dol.gov/_sec/media/reports/annual2003/.

Does the program (including program partners) achieve its annual performance goals?

Explanation: This program has adopted four new long-term goals as part of the Job Training Common Measures initiative. DOL is to establish numerical annual targets in PY 2004. Partial credit is based on the fact that DOL achieved many of its key annual goals in PY 2002 and prior years. The Entered Employment Rate for jobseekers registered during the first two quarters of PY 2002 (July 1, 2002 ' December 31, 2002) was 61% (vs. a target of 55%). The Employment Retention Rates for jobseekers who registered during the first quarter of PY 2002 was 79% (vs. a goal of 72%). ES also achieved its GPRA goals in 2000 and 2001 for job seekers who entered employment, although the definitions and data sources used then are not comparable to current measures. Since PY 1998, ES has used a job openings indicator for both the State workforce agencies and America's Job Bank. As of the third quarter of PY 2003, 93% (9,727,383/10,426,835) of the annual target number of job listings was achieved, indicating that the PY 2003 goals will likely be achieved.

Evidence: (64) PY 2001 Annual Report for Wagner-Peyser Act Funded Activities is accessible at www.uses.doleta.gov/arp01/;(65) National Summary of Job Openings Received and Openings Filled (PY 1994 through PY 2002);(66) National Summary ' ETA 9002 Quarterly Reports (07/01/95 through 06/30/03).

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: While the program received a 'NO' in Question 3.4 because DOL does not have procedures in place to increase the efficiencies and cost-effectiveness of State program operations, DOL is able to measure State efficiency and cost effectiveness. An "NA" is appropriate because evaluations have shown that ES services are very cost-effective with low unit costs. For PYs 2001 and 2002, ES costs have been under $55 per participant, based to a large extent on a service delivery strategy that is computer-based with minimal staff assistance. This allows the ES to provide basic labor exchange services and labor market information to some 15 million job seekers.

Evidence:  

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: While the ES is duplicative of some other Federal and private efforts to provide job matching and other basic reemployment services (see Question 1.3), there is no performance information available for these other programs. WIA grantees typically do not register those job seekers who participate only in the WIA core services that are most comparable to those provided by the ES, so performance outcomes for those participants are not known. Private sector operators of job banks and placement agencies treat their performance information as proprietary.

Evidence:  

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: While several independent evaluations of the effectiveness of certain ES services have been done, the results depend on practices of local offices, which can vary substantially across States. Consequently, while labor exchange and job-finding assistance can be effective services, these conclusions cannot be extrapolated to the ES as a whole. DOL encourages States to adopt effective service strategies and cites research findings when providing policy guidance. Most of the research reports are available on ETA's web site. Among the ES studies, a preliminary 2000 report of a five-year evaluation measured the effect of job referrals and placements in Washington and Oregon; it found these services reduced the number of weeks of unemployment benefits paid and earnings gains more than coverd the cost of services. The 2003 study, Evaluating the Effectiveness of Labor Exchange Services, looked at nine previous studies of the effectiveness of job interview referrals, job search assistance, and targeted job search assistance and found that such services are cost effective.

Evidence: (67) See Evidence for Question 2.6. All independent evaluations cited concluded that the ES is cost efficient and effective. Additional studies evaluating the effectiveness of the ES in the past 20 years have reported low costs per customer served. These evaluations are available in hard copy by contacting ETA at (202) 693-3046;(68) Evaluating the Effectiveness of Labor Exchange Services, W.E. Upjohn Institute for Employment Research (October 2003);(69) ETA's Quarterly Report of Workforce System Results at www.dol.gov/_sec/media/reports/annual2003/ reflects how effective the program is and achieving results;(70) Employment Services Cost Efficiency PY 1997-PY 2002 (05/07/04) provides a history of the low cost of providing services to ES customers.