5. Performance Measurement

The Department of Labor recognizes that GPRA requires a combination of valid indicators of program accomplishments and continuous assessment of performance against those indicators. Obtaining and using valid data is the crucial first step. The Department has positioned itself to make comprehensive improvements in the quality, timeliness, and accuracy of the data it uses to monitor program performance.

Several approaches to improve performance measurement were initiated in FY 2000 and will continue into FY 2001. At the agency and program level, specific steps will be taken to address known challenges in data quality. The Department will actively assist individual program areas to identify better data sources, improve reporting procedures, and increase the validity of indicators that are used to define program success.

Working together, DOL executives and program managers will continuously improve the quality of the indicators, data sources and baselines enumerated in Appendix B. The Deputy Secretary, Assistant Secretaries and agency heads play key roles in the Department-wide effort to assess both data quality and program performance throughout DOL. The goal is to build confidence that the detailed measures being used by DOL components support the Department's three strategic goals and ultimately lead to a prepared and secure American workforce, and quality workplaces in our nation and throughout the world.

5.1 Addressing Specific Performance Measurement Challenges

Within the larger Departmental framework, individual DOL agencies will address data challenges that are unique to the agency's program environment and develop solutions that are consistent with the Department's reporting requirements. While some DOL programs currently have adequate systems in place, others must overcome barriers to the production of timely, accurate, and relevant performance data. In FY 2001, DOL and its agencies will continue efforts initiated in FY 2000 to address three issues: lack of data, insufficient validation of data, and untimely reporting. The following examples describe several initiatives in progress to improve the measurement of our program results.

1. The Employment Standards Administration will continue its efforts to improve data measuring the results of three major programs in FY 2001. First, the Wage and Hour program has introduced a new computer system, the Wage and Hour Investigative and Reporting Database (WHISARD), permitting more timely access to extensive information. Because there is no unbiased database on labor standards violations or compliance, Wage and Hour faces a major challenge in determining industry-wide levels of compliance and measuring changes in that compliance. To determine the impact of Wage and Hour efforts, a statistically sound method has been developed for establishing baselines and measuring compliance using investigation-based compliance surveys of targeted industries and areas. Data on the outcomes of repeat investigations will also be used to evaluate the relative effectiveness, or return on investment, of the various types of interventions.

The new WHISARD system provides many advantages when compared to its predecessor. For example, data is entered into the system directly at the source (by Wage and Hour investigators) rather than manually batched and mailed to a central source for data entry later. As a result, data is much more current, which facilitates tracking the progress of investigative activity. Information on a complainant's case is readily available which enhances customer service and satisfaction. All users of the WHISARD system have direct access which was not previously possible and enhances efficiency and enforcement effectiveness. In addition, WHISARD can produce data on a much broader range of activities, such as local enforcement initiatives, which was not possible with the prior system, and assists in the agency's strategic planning activities.

The Office of Federal Contract Compliance Programs' (OFCCP) Case Management System (CMS) is used to measure program performance. Through software and hardware enhancements, field office managers are now able to access CMS data to track the accomplishments of individual organizational units.

OFCCP will complete cognitive testing and field testing of its Equal Opportunity (EO) Survey during the first two quarters of FY 2000. Once the EO Survey is implemented, federal contractor response data will be input into a database and analyzed, with the results of data analysis to be used in making scheduling determinations for compliance evaluations. Development of the database structure, including the analytical model, will be completed before the end of FY 2000, with full implementation planned for FY 2001.

Increasingly sophisticated databases detailing the case histories of injured federal workers have been developed in the course of formulating new strategies for the Federal Employees' Compensation Act (FECA) program. These systems allow tracking of the goal of reducing lost production days, but also permit precise evaluation of various program initiatives such as the impact of alternative return to work techniques on various groupings of employees or injury categories.

The Federal Employees' Compensation automated system is undergoing a complete redesign covering every major staff function. The redesign will replace a patchwork of loosely-linked programs each with its own database and rules, and provide a single automated system that will make data accessible to all users. This redesign will allow workers' compensation claims staff to work more efficiently while providing improved customer service to injured workers, medical providers, and employing Federal agencies.

2. The Department's Employment and Training Administration will take specific actions to improve its data collection methods. The Department is seeking OMB approval to implement quarterly performance reporting by States and other employment and training program grantees and, based on OMB's recommendations, will publish a proposal in the Federal Register and obtain public comments. Under current legislation, the Department receives information on employment and training programs on a program year basis, and the data is not available for 15 months after the fiscal year. The approval and enactment of this proposal would significantly improve the timeliness of information on the results of major programs, and assist the Department to respond more effectively to program needs.

The Department continues to address the need to assure the accuracy and reliability of performance data submitted by our employment and training system partners which serves as the foundation for key program decisions. The development of a comprehensive data validity system for the core indicators of the WIA program, Wagner-Peyser, and other key employment and training programs is in the preliminary stages, and this project will extend into FY 2001.

3. GAO and OIG reports have raised particular concerns about the accuracy of data on the employment placement and retention of Job Corps participants. Job Corps program indicators will undergo continuous improvement during FY 2001. Revisions to Job Corps' procedures have already addressed some of the issues cited by GAO and OIG, and additional changes are in process. For example, data collection for both the initial job placement and retention in employment measures are now conducted by a neutral third party to provide increased assurance of validity. The Department is continuing to work with OIG to introduce additional improvements to the reliability of Job Corps measurements and procedures.

4. In FY 2001, progress will continue to be made for indicators where data are currently unavailable or incomplete. For example, for employment outcomes for TAA/NAFTA program participants, the expansion of pension coverage, or indicators affected by implementation of WIA, baselines will be established and refined. This will be done at the Agency and program level in conjunction with the Department-wide effort to improve the quality of indicators, which is described below.

5.2 Improving Performance Indicators

DOL strategic goals are carried out by agencies that vary considerably in structure and mission. Some agencies have a function that is largely regulatory (OSHA and MSHA), some (e.g., BLS) gather data, others (ILAB, the Women's Bureau) have a significant advocacy function. The Department's focus across these diverse agencies and programs is to ensure that the indicators selected provide an effective, and to the maximum extent possible, quantifiable measures of the accomplishment of performance and outcome goals. The Department also recognizes the need for an information technology infrastructure that meets daily operational requirements and satisfies the data collection, validation, analysis, and information dissemination that is central to GPRA. DOL will continue on-going initiatives to strengthen two types of information systems -- the performance measurement systems that provide the foundation for program evaluations, both tactical and strategic, and the financial and cost accounting systems used to monitor the cost of performance.

5.3 Linking Costs to Performance

The Department has a solid financial systems infrastructure from which a cost accounting capability will be developed using the resources of a reliable, established accounting system -- the Department of Labor Accounting and Related Systems (DOLAR$). DOLAR$, serving as the system of record for financial results throughout the Department, has been modified to capture, aggregate, allocate and report costs. A new cost accounting module has been developed to allow aggregation of costs across agency lines and to allocate direct and indirect costs to the Strategic Outcome and Performance Goal levels established in the Department's Strategic Plan. The Department has maintained cost accounting information, beginning in FY 1999, for the outcome goals in the Department's Strategic Plan. In addition, DOL will continue to develop the capability to consolidate data from a variety of program and financial system sources and link that data as needed to meet the performance reporting requirements of GPRA.