[Federal Register: January 16, 2004 (Volume 69, Number 11)]
[Notices]
[Page 2630-2634]
From the Federal Register Online via GPO Access [wais.access.gpo.gov]
[DOCID:fr16ja04-107]
-----------------------------------------------------------------------
DEPARTMENT OF LABOR
Employment and Training Administration
Proposed Information Collection Submitted for Public Comment and
Recommendations: Data Validation Requirement for Employment and
Training Programs
AGENCY: Employment and Training Administration (ETA), Labor.
ACTION: Notice.
-----------------------------------------------------------------------
SUMMARY: The Department of Labor, as part of its continuing effort to
reduce paperwork and respondent burden,
[[Page 2631]]
conducts a pre-clearance consultation program to provide the general
public and Federal agencies with an opportunity to comment on proposed
and/or continuing collections of information in accordance with the
Paperwork Reduction Act of 1995 (PRA95) (44 U.S.C. 3506(c)(2)(A)). This
program helps to ensure that requested data can be provided in the
desired format, the reporting burden (time and financial resources) is
minimized, the collection instruments are clearly understood, and the
impact of collection requirements on respondents can be properly
assessed. The Employment and Training Administration (ETA) is
soliciting comments on the establishment of a data validation
requirement for the following employment and training programs:
Workforce Investment Act (WIA) Title IB, Labor Exchange, Trade
Adjustment Assistance (TAA), Migrant and Seasonal Farmworkers (MSFW),
Native American Employment and Training, and Senior Community Service
Employment Program (SCSEP).
DATES: Submit comments on or before March 16, 2004.
ADDRESSES: Send comments to: Gail Eulenstein, Performance and Results
Office, Employment and Training Administration, U.S. Department of
Labor, 200 Constitution Avenue, NW., Room N-5309, Washington, DC 20210;
telephone: (202) 693-3013 (this is not a toll-free number); fax: (202)
693-3991; e-mail: Eulenstein.Gail@dol.gov.
FOR FURTHER INFORMATION CONTACT: Gail Eulenstein, Performance and
Results Office, Employment and Training Administration, U.S. Department
of Labor, 200 Constitution Avenue, NW., Room N-5309, Washington, DC
20210; telephone: (202) 693-3013 (this is not a toll-free number); fax:
(202) 693-3991; e-mail: Eulenstein.Gail@dol.gov.
SUPPLEMENTARY INFORMATION:
I. Background
The accuracy and reliability of program reports submitted by States
and grantees using Federal funds are fundamental elements of good
public administration, and are necessary tools for maintaining and
demonstrating system integrity. The President's Management Agenda to
improve the management and performance of the Federal government has
emphasized the importance of complete information for program
monitoring and improving program results.
States and grantees receiving funding under WIA Title I, Wagner-
Peyser Act, TAA, and SCSEP are required to maintain and report accurate
program and financial information (WIA section 185 (29 U.S.C. 2935) and
WIA Regulations 20 CFR 667.300(e)(2); Wagner-Peyser Act section 10 (29
U.S.C. 49i), Older Americans Act section 503(f)(3) and (4) (42 U.S.C.
3056a(f)(3) and (4)), and TAA Regulations 20 CFR 617.57). Further, all
States and grantees receiving funding from ETA and the Veterans'
Employment and Training Service are required to submit reports or
participant records and attest to the accuracy of these reports and
records.
Recent performance audits conducted by the Department of Labor's
Office of the Inspector General, however, found that the accuracy of
reported performance outcomes cannot be assured due to insufficient
local, State, and Federal oversight. To address this concern and meet
the Agency's goal for accurate and reliable data, ETA committed to the
development and implementation of a data validation process in order to
ensure the accuracy of data collected and reported on program
activities and outcomes.
Data Validation. The data validation requirement for employment and
training programs will strengthen the workforce system by ensuring that
accurate and reliable information on program activities and outcomes is
available. Data validation is intended to accomplish the following
goals:
[sbull] Ensure that critical performance data are accurate.
[sbull] Detect and identify specific problems with a State's or
grantee's reporting process, including software and data issues, to
enable the State or grantee to correct the problems.
[sbull] Help States and grantees analyze the causes of performance
successes and failures by displaying participant data organized by
performance outcomes. In addition, the process will allow States and
grantees to select appropriate validation samples necessary to compute
statistically significant error rates.
Data validation consists of two parts:
(1) Report validation evaluates the validity of aggregate reports
submitted to ETA by checking the accuracy of the reporting software
used to calculate the reports. Report validation is accomplished by
processing an entire file of participant records into validation counts
and comparing the validation counts to those reported by the State or
grantee.
(2) Data element validation assesses the accuracy of participant
data records. Data element validation is performed by reviewing samples
of participant records against source documentation to ensure
compliance with Federal definitions.
Data Validation Pilot Test. Two States participated in a pilot test
of the validation process in the summer and fall of 2002. Grantees in
the MSFW program, Native American Employment and Training program, and
SCSEP will begin pilot tests by the end of CY 2003. The pilot States
conducted validation for the WIA Title IB, Labor Exchange, and TAA
programs. The States received preparatory training prior to beginning
validation and technical assistance throughout the pilot from ETA's
validation contractor. The pilot test indicated the following:
[sbull] States and grantees will generally be able to conduct data
validation with a reasonable but sustained level of effort.
[sbull] The validation process allows States and grantees to
identify and address reporting errors.
[sbull] States and grantees do make reporting errors which need
detecting and fixing.
[sbull] The average staff requirements for a State to complete
validation for the WIA Title IB, Labor Exchange, and TAA programs will
be about 882 hours per year. The average annual time required by
grantees operating MSFW programs, Native American Employment and
Training programs, and SCSEP to complete validation is approximately
102 hours. Start-up activities in the initial year of validation will
require an additional 264 hours on average per State and 74 hours on
average per grantee.
II. Desired Focus of Comments
Currently, ETA is soliciting comments about the proposed new
collection of information on the validity of data that States and
grantees report to the Agency. ETA is seeking Office of Management and
Budget (OMB) approval under PRA95 to establish a data validation
requirement for the following employment and training programs: WIA
Title IB, Labor Exchange, TAA, MSFW, Native American Employment and
Training, and SCSEP. Data validation will increase the reporting burden
by requiring States and grantees to submit reports on data validity to
ETA.
A copy of the proposed information collection request can be
obtained by contacting the office listed above in the ADDRESSES section
of this notice. The Department of Labor is particularly interested in
comments which:
[sbull] Evaluate whether the proposed collection of information is
necessary for the proper performance of the functions of the Agency,
especially whether the information will have practical utility;
[[Page 2632]]
[sbull] Evaluate the accuracy of the Agency's estimate of the
burden of the proposed collection of information, including the
validity of the methodology and assumptions used;
[sbull] Discuss how to enhance the quality, utility, and clarity of
the information to be collected; and
[sbull] Suggest how to minimize the burden of the collection of
information on those who are to respond, including through the use of
appropriate automated, electronic, mechanical, or other technological
collection techniques or other forms of information technology (e.g.,
permitting electronic submissions of responses).
III. Current Actions
The Department proposes the following plan for implementing and
operating data validation:
[sbull] In order to ensure the accuracy of reported information
throughout the workforce investment system, States and grantees will be
required to conduct data validation and submit validation output
reports to ETA. States will initiate data validation for WIA Title IB,
Labor Exchange, and TAA by the end of CY 2003, and grantees operating
MSFW programs, Native American Employment and Training programs, and
SCSEP will initiate validation during CY 2004.
[sbull] Data validation will be required annually. States and
grantees will be required to send validation output reports to ETA
within 120 days after the submission of required annual reports or
participant records to ETA. Report validation will be performed prior
to the submission of reports. Data element validation will be completed
within 120 days after required annual reports or participant records
are due to ETA.
[sbull] ETA has developed a set of validation tools discussed
below--instructional handbooks, software, and user guides--that States
and grantees can use to validate data. States and grantees may use an
alternative methodology and tools as long as the methodology meets
standards for sampling methods and confidence intervals. States or
grantees that do not use the validation tools provided by ETA will be
required to document that the alternative methodology is statistically
valid.
[sbull] In addition to performing validation, the ETA software can
be used to generate the aggregate information required in reports
submitted to ETA. States or grantees that use the software provided by
ETA to generate this aggregate information will not be required to
perform report validation.
[sbull] ETA will establish acceptable levels for the accuracy of
reports and data elements. These accuracy standards will be established
in phases. The initial validation year will focus on detecting and
resolving any issues with State and grantee data and reporting systems.
Error rates collected in the second year will be analyzed, and, based
on this information, standards for accuracy will be established prior
to the third year of validation.
[sbull] Once accuracy standards are established, States and
grantees will be held accountable for meeting those standards and will
be required to address any issues concerning data accuracy. States and
grantees that fail to meet accuracy standards will receive technical
assistance from ETA and will develop and implement a corrective action
plan. Data that does not meet accuracy standards will not be acceptable
for measuring performance, and may keep the State or grantee from being
eligible for incentives that are awarded based on performance data.
Significant or unresolved deviation from accuracy standards may be
deemed a failure to report.
Resources. The requirement to perform validation derives from
States' and grantees' responsibility to provide accurate information on
program activities and outcomes to ETA. States and grantees are
expected to provide resources for conducting validation from their
administrative funds. Validation of program performance is a basic
responsibility of grantees, who are required to report on program
performance, under Department of Labor regulations (29 CFR 95.51 and
97.40). ETA has taken a number of steps to minimize the resources
needed for data validation, including developing tools that States and
grantees can use to perform validation. The estimates provided below,
which are based on state pilot experiences, indicate that annual staff
requirements for continuing operations of data validation will be on
average 882 hours (or less than \1/2\ of a staff year) for a State and
102 hours (or about \1/20\ of a staff year) for a grantee.
Data Validation Tools. To reduce startup costs related to
implementing data validation, ETA has developed standardized software,
instructional handbooks, and user guides that States and grantees can
use to perform data validation:
[sbull] Software developed by ETA generates samples, worksheets,
and reports on data accuracy. For report validation, the software will
validate the accuracy of aggregate reports that are generated by the
State's or grantee's reporting software and will produce an error rate
for each reported count. For data element validation, the software
generates a sample of the participant records and data elements for the
state or grantee to validate. The software produces worksheets on which
the validator records information after checking the source
documentation in the sampled case files. The software calculates error
rates for each data element, with confidence intervals of 3.5 percent
for large States/grantees and 4 percent for small States/grantees.
[sbull] Handbooks provide detailed information on the validation
methodology, including sampling specifications and data element
validation instructions for each data element to be validated.
[sbull] User guides developed for each ETA validation software
application guide States and grantees through the process of installing
the application, building and loading a validation file, and completing
report and data element validation.
Data Recording and Reports. States and grantees will record the
results of their validation on spreadsheet software prepared as an
accompaniment to their handbooks. Initially, the spreadsheets can be
transmitted by e-mail to ETA. Eventually, the results will be submitted
in the same manner as other reports. The results will be stored in a
dataset in the National Office in Washington, DC, and compiled in an
annual validation accuracy report.
Training and Technical Assistance. ETA provided validation training
to States in regional sessions during the summer of 2003. Training for
grantees of the MSFW and Native American Employment and Training
programs will be held during winter 2003/04, and training will be
provided for SCSEP grantees during spring 2004. States and grantees may
obtain technical assistance on validation procedures and the use of the
validation tools by contacting ETA's data validation contractor.
Type of Review: New.
Agency: Employment and Training Administration.
Title: Data Validation Requirement for Employment and Training
Programs.
OMB Number: 1205-0NEW.
Recordkeeping: States and grantees must maintain complete records
of all validation activities for three years. The retention requirement
will apply to records of all validation activities, including files,
worksheets, reports, and source documentation.
Affected Public: State, local, and tribal government entities and
private non-profit organizations.
Total Respondents: 317 (53 states will perform validation for the
WIA Title IB,
[[Page 2633]]
Labor Exchange, and TAA programs annually. 264 grantees operating MSFW
programs, Native American Employment and Training programs, and SCSEP
will perform validation annually).
Frequency: Complete data validation annually.
Total Responses: 317 (53 responses from states annually and 264
responses from grantees annually).
Estimated Time per Response: 882 hours per year on average for a
state to complete validation of the WIA Title IB, Labor Exchange, and
TAA programs. 102 hours per year on average for a grantee operating a
MSFW program, Native American Employment and Training program, or SCSEP
to perform validation.
Total Burden Hours: An estimated 46,732 hours per year will be
required for all states to complete validation for the WIA Title IB,
Labor Exchange, and TAA programs. An estimated 13,992 hours will be
necessary by all states for startup activities in the initial year of
validation. An estimated 26,830 hours per year will be required for all
grantees operating MSFW programs, Native American Employment and
Training programs, and SCSEP to perform validation. An estimated 19,552
hours will be necessary by all grantees for startup activities in the
initial year of validation.
Total Burden Cost (startup): The start-up cost is estimated to be
$454,740 for all states in the initial year of validation for the WIA
Title IB, Labor Exchange, and TAA programs ($8,580 on average per
state). The start-up cost is estimated to be $312,322 for all grantees
in the initial year of validation for MSFW, Native American Employment
and Training, and SCSEP ($1,183 on average per grantee).
Total Burden Cost (operating): The cost is estimated to be
$1,518,791 per year for all states to complete validation for the WIA
Title IB, Labor Exchange, and TAA programs ($28,656 on average per
state). The cost is estimated to be $495,767 per year for all grantees
operating MSFW programs, Native American Employment and Training
programs, and SCSEP to perform validation ($1,878 on average per
grantee).
Summary of Burden
Calculation of Combined Annual Burden for WIA Title IB, Labor Exchange, and TAA
----------------------------------------------------------------------------------------------------------------
No. of states Annual hours Rate in $/hr\1\ Cost Cost
------------------------------------------------------------------------------------------------------------ ------
Large State......................... 18 1,332 $32.50 $43,297
Medium State........................ 18 836 32.50 27,180
Small State......................... 17 453 32.50 14,718
All States.......................... 53 46,732 32.50 1,518,791
Average per State................... ................ 882 32.50 28,656
----------------------------------------------------------------------------------------------------------------
\1\ Hourly rate is the estimated average hourly earnings for employees in State Unemployment Insurance (UI)
agencies in FY 2003 (as used for FY 2003 UI budget formulation purposes).
Calculation of Combined Startup Burden for WIA Title IB, Labor Exchange, and TAA
----------------------------------------------------------------------------------------------------------------
No. of states Hours Rate in $/hr \1\ Cost
----------------------------------------------------------------------------------------------------------------
State................................... 53 264 $32.50 $8,580
All States.............................. 53 13,992 32.50 454,740
----------------------------------------------------------------------------------------------------------------
\1\ Hourly rate is the estimated average hourly earnings for employees in State Unemployment Insurance (UI)
agencies in FY 2003 (as used for FY 2003 UI budget formulation purposes).
Calculation of Annual Burden for MSFW, Native American Employment and Training, SCSEP
----------------------------------------------------------------------------------------------------------------
No. of grantees Annual hours Rate in $/hr\1\ Cost
----------------------------------------------------------------------------------------------------------------
MSFW Grantee............................ 52 158 $10.75/32.50 $1,896
Native American Employment & Training 144 53 10.75 569
Grantee................................
SCSEP Grantee........................... 68 162 10.75/32.50 4,637
All Grantees............................ 264 26,830 10.75/32.50 495,767
Average per Grantee..................... ................ 102 10.75/32.50 1,878
----------------------------------------------------------------------------------------------------------------
\1\ Hourly rates used to calcuate cost depends upon the type of organization receiving the grant. For State
government grantees, the hourly rate is the estimated average hourly earnings for employees in State
Unemployment Insurance (UI) agencies in FY 2003 (as used for FY 2003 UI budget formulation purposes). For
private non-private grantees, the hourly rate is the average hourly earnings in the social assistance industry
(May 2003, Current Employment Statistics Survey, U.S. Census Bureau).
Calculation of Startup Burden for MSFW, Native American Employment and Training, SCSEP
----------------------------------------------------------------------------------------------------------------
No. of grantees Hours Rate in $/hr\1\ Cost
----------------------------------------------------------------------------------------------------------------
MSFW Grantee............................ 52 72 $10.75/32.50 $864
Native American & Training Grantee 144 72 10.75 774
Employment.............................
SCSEP Grantee........................... 68 80 10.75/32.50 2,293
All Grantees............................ 264 19,552 10.75/32.50 312,322
Average per Grantee..................... ................ 74 10.75/32.50 1,183
----------------------------------------------------------------------------------------------------------------
\1\ Hourly rates used to calculate cost depends upon the type of organization receiving the grant. For State
government grantees, the hourly rate is the estimated average hourly earnings for employees in State
Unemployment Insurance (UI) agencies in FY 2003 (as used for FY 2003 UI budget formulation purposes). For
private non-profit grantees, the hourly rate is the average hourly earnings in the social assistance industry
(May 2003, Current Employment Statistics Survey, U.S. Census Bureau).
[[Page 2634]]
Data validation is estimated to require an annual burden of 73,562
hours and $2,015,000 for all six programs subject to the validation
requirement. An additional 33,544 hours and $767,000 will be required
for startup activities for all six programs in the initial year of
validation.
Comments submitted in response to this comment request will be
summarized and/or included in the request for OMB approval of the
information collection request; they will also become a matter of
public record.
Signed in Washington, DC, on January 9, 2004.
Emily Stover DeRocco,
Assistant Secretary for Employment and Training.
[FR Doc. 04-990 Filed 1-15-04; 8:45 am]
BILLING CODE 4510-30-P