The Department of Labor has implemented GPRA using a systematic, staged
approach based on sound business practices. Our first priority has been the
establishment of enduring outcome goals that target continual improvement in
the achievement of the core results the Department is committed to delivering
on behalf of working men and women. After several years of piloting and
refining goals, a majority of the Department's programs have stabilized their
goals, and DOL's focus has progressively shifted toward ensuring the
reliability of our performance measures and effectively using performance data
to enhance our program results.
The Department recognizes that transitioning to a fully
performance-based organization depends upon the availability of reliable and
timely information concerning the results of DOL's programs. The challenges to
performance measurement vary significantly among DOL's programs, with the data
sources and the agencies' level of control over the reporting systems being the
primary factors influencing the reliability and usefulness of the Department's
performance information. Many DOL agencies collect critical program data from
third parties, including State and local government agencies, community-based
organizations and private sector employers. The Department's authorities to
increase the frequency of reporting, establish data standards or verify the
accuracy of the information reported by third parties are limited in some
cases. In other programs, data important to fully assess the effectiveness of
strategies adopted to achieve our GPRA goals are not readily available, or
DOL's information technology systems require enhancements to support routine
access by program managers to key performance data.
Despite these measurement challenges, DOL is continuing its shift toward
performance-based management through a number of initiatives in FY 2003. First,
the Department plans to continue sponsoring workshops for program staff on
performance management and measurement issues. Through these workshops, program
managers will learn about measurement approaches from experts in the field and
will also exchange ideas and best practices with each other. Second, the Office
of Inspector General will continue its reviews of the reliability and quality
of selected performance data used by DOL programs in FYs 2002 and 2003, and the
Department will work closely with program staff in addressing problems
identified in these reviews. Finally, the Department plans to further increase
the use of program evaluations, particularly in areas identified by the PART
and other sources as lacking critical performance data.
Working together, DOL executives and program managers are continuously
improving the quality of the Department's performance information and the use
of that information in management decisions. However, the success of these
efforts largely depends on DOL's authorities to collect and verify data and the
availability of resources needed to do so.
5.1 Addressing Specific Performance Measurement
Challenges
Within the larger Departmental framework, individual DOL agencies will
address data challenges that are unique to their program environment and
develop solutions that are consistent with the Department's reporting
requirements. While some DOL programs currently have adequate systems in place,
others must overcome barriers to the production of timely, accurate, and
relevant performance data. In FY 2004, DOL and its agencies will continue
efforts to address four issues: the availability of data, validation of data,
timeliness of reporting, and the use of data in managing for results. Below are
examples of several initiatives that agencies are undertaking to further
improve the measurement of our program results.
Employment Standards Administration
FECA's automated system is undergoing a complete redesign covering every
major program operation. The redesign will replace a patchwork of loosely
linked programs each with its own database and rules, and provide a single
automated system that will make data accessible to all users. This redesign
will allow workers' compensation claims staff to work more efficiently while
providing improved customer service to injured workers, medical providers, and
employing Federal agencies.
Employment and Training Administration
The Department continues to address the need to ensure the accuracy and
reliability of performance data submitted by our employment and training system
partners, which serves as the foundation for key program decisions. The
development of a comprehensive data validity system for the core indicators of
the WIA program, Wagner-Peyser, and other key employment and training programs
is expected to be substantially completed in FY 2002.
In FY 2003, the Department will establish a policy on data validity and
verification that applies to its job training and employment programs. It will
also focus attention on wide-scale training and technical assistance to deploy
the data validity system among states and other grantees. Office of Inspector
General reports have raised issues about the accuracy and reliability of data
used to measure the outcome performance of the Welfare-to-Work competitive
grant program and the Trade Adjustment Assistance and North American Free Trade
Agreement Transitional Adjustment Assistance programs. Both
programs are expected to be the subject of special attention with regard to the
validity and verification system.
The Department has taken a leadership role in implementing common
performance measures for several of its job training and employment programs
under the President's Management Agenda goal of Linking Budget and Performance.
The goals set forth in this plan are framed as the common measures. The
Department will be undertaking steps to implement the measures during FY 2003
so that FY 2004 performance will establish the baseline using the new measures.
The data validation and verification activities described above may be updated
to reflect the common measures.
Veterans' Employment and Training Service
Accurate outcome oriented performance data has been an issue for VETS
since performance planning under the GPRA commenced in 1999 and placed emphasis
on outcomes. Reporting data prior to GPRA was developed to respond to the
reporting requirements of Title 38, Section 4107, which is essentially activity
based. In addition, reporting of placements on the ETA 9002 has been subject to
error and under-counting because it requires that a veteran specialist (DVOP or
LVER) make contact with a registrant to confirm that the person got a job
before their employment may be reported on the ETA 9002.
VETS initiated a special study in Maryland that utilized Unemployment
Insurance wage records to gain a better understanding of the outcome of
services provided by veterans specialists to veterans. This study has shown
that a much larger number of veterans get jobs than are likely to have been
captured by the information collection and reporting systems now in place. This
result has been corroborated by other State studies conducted by ETA.
VETS is an active partner with ETA in designing new reporting systems
that will use the results of these studies to better capture performance
outcomes. Both VETS and ETA will be implementing outcome oriented performance
measures with the public labor exchange system, which has been fully integrated
into the One-Stop environment. Unemployment Insurance wage records will be the
key data source supporting the new performance measures.
VETS will continue to implement the recommendations made in the July,
2000 data capacity report on VETS programs prepared by a contractor. This will
help VETS ensure that its measurement of performance is accurate and verifiable
to the best degree possible, considering VETS biggest programs (DVOP and LVER)
must rely on data from the States.
In FY 2003, VETS intends to have a new program evaluation system in
place that will capture performance data on services provided to veterans by
the public labor exchange system. A key consideration that the new system will
address is how to effectively verify services to veterans when the old paradigm
has changed. In past years, all veterans entering a local office were
registered and a formal file established. This made statistical sampling of
files an effective way to verify services to veterans that were reported on the
VETS 200 and the ETA 9002. However, in the One-Stop environment there has been
an emphasis on self-service, where there is no registration. In fact, in some
States, there is no registration even if a mediated service is provided. This
may require replacing the traditional method of statistical sampling with other
verification techniques, such as surveys.
Mine Safety and Health Administration
MSHA has a significant database and collection system that captures most
of the information necessary to track performance under the strategic plan.
Data has been collected for many years and the database is well established for
performance measures under the performance goal of reducing the number of mine
fatalities and the nonfatal-days-lost injury incidence rate. However, MSHA
relies on mine operators and contractors to comply with legal requirements to
report accurately and timely employment, injuries and accidents. MSHA conducts
periodic audits to ensure compliance. The number of audits conducted may
influence the degree of compliance. Quarterly reports are generated for
managers showing active mines that do not report quarterly employment and
production information. Routine follow-up visits are conducted at these mines
and audits increased as appropriate depending upon findings. If analysis of
these visits/audits determine any pattern or problem with the quality or
reliability of the data, MSHA takes corrective action.
MSHA's system for determining compliance with the coal respirable dust
standard has been in place since the 1970s and procedures are well established
to ascertain the accuracy and reliability of the data. Automated devices are
used to weigh the inspector dust samples and automatically enter the results
into a custom designed program that updates the dust data files daily. A
quality control program developed jointly by MSHA and the National Bureau of
Standards assures that the weighing process continues to produce reliable
results over time, and computer edit checks assure the accuracy of the
database.
Metal and Nonmetal inspectors have conducted industry-wide sampling
since the 1970s. Health policies and the management information system are well
established and reliable. Automated devices are used to weigh inspector dust
samples at MSHA's analytical lab that was accredited by the American Industrial
Hygiene Association in FY 1998. Computer edits assure the accuracy of
Management Information System data input. Between 1994 and 1998 the Agency
workforce that inspects metal and nonmetal mines declined and began to slightly
increase during 1999 to 2002; however, the number of metal and nonmetal mines
has continually increased. Metal and nonmetal samples are generally collected
at the discretion of the inspector based upon the conditions observed at the
mine. Designations of high-risk occupations and new sampling procedures have
been established for targeting sampling at Metal and Nonmetal Mines. Using the
new procedures, samples collected in FY 2001 are used to establish a baseline.
A new goal was established in FY 2002 and continues in FY 2004.
MSHA's current database and collection system will be used to capture
data and samples relating to noise exposures and diesel particulates.
Occupational Safety and Health Administration
In FY 2003, OSHA will continue to use program data to manage its
programs and ensure the quality of safety and health data. The agency will rely
on the results of program evaluations to identify the best strategies to
achieve its strategic and performance goals.
OSHA will maintain its efforts to improve data validation procedures.
Validation of data generated by the agency for current performance measures
will be addressed through a variety of means such as annual on-site audits of
the injury and illnesses records of employers to determine the accuracy and
reliability of the OSHA 200 Logs, the source of data for the OSHA Data
Initiative and the BLS Annual Survey; information and outreach programs and
enforcement of the injury and illness recordkeeping regulations; revision of
injury and illness record keeping system (regulations, forms, and guidelines)
to improve the quality of records by simplifying forms and regulations,
providing clearer guidance for employers, and incorporating incentives for
employers to maintain high quality records; and continuing the various methods
OSHA developed for validating and verifying data in the OSHA Integrated
Management Data System (IMIS).
Program evaluations will assess how well OSHA's programs, policies, and
procedures are working, including the effectiveness of specific standards,
customer satisfaction, and specific approaches towards reducing occupational
injuries and illnesses.
5.2 Linking Costs to Performance
The Department has a solid financial systems infrastructure from which a
cost accounting capability is being developed using the resources of a
reliable, established accounting systemthe Department of Labor Accounting
and Related Systems (DOLAR$). DOLAR$, serving as the system of record for
financial results throughout the Department, has been modified to capture,
aggregate, allocate and report costs. The Department's cost systems allow
aggregation of costs across agency lines and the allocation of direct and
indirect costs to the strategic and outcome goal levels established in the
Department's Strategic Plan. In the FY 2001 Annual Report on Performance and
Accountability, the Department provided cost data for FY 1999 through FY
2001 to accompany and provide perspective on each outcome goal presentation in
the Performance Report section.
The Department has maintained cost accounting information, beginning in
FY 1999, for the outcome goals in the Department's Strategic Plan. In this FY
2004 plan, the Department continued its practice of linking budget authority
and outlays to both strategic and outcome goals, and for the first time this
year, has added information on the cost to achieve each of the core performance
goals shown in Chapter 4. In addition, Appendix C provides an overview of the
linkage between budget activities and outcome goals. DOL will continue to
develop the capability to consolidate data from a variety of program and
financial system sources and link that data as needed to meet the performance
reporting requirements of GPRA.
A significant accomplishment for the Department has been the development
and implementation of its first integrated agency-level performance budget
format for the FY 2004 budget cycle. In addition, the Department will develop
for FY 2005 prototype goals to improve the cost effectiveness of a job training
and one regulatory program, with the expectation that these prototypes could be
adapted to other DOL job training and regulatory programs.
|