MEMORANDUM FOR:
RAYMOND J. UHALDE
Acting Assistant Secretary for
Employment and Training
/ s /
FROM:
JOHN J. GETEK
Assistant Inspector General
for Audit
SUBJECT:
Audit of Central Texas Council of Governments
JTPA Title III Demonstration Grant No. 5502-5-00-87-60
Final Audit Report 06-98-003-03-340
The attached subject final report is submitted for your resolution action. We request a response to this report within 60 days.
Your office is responsible for submitting the report to the grantee. However, we have provided a courtesy copy of the report to the grantee this date.
If you have any questions regarding this report, please contact John Riggs, Regional Inspector General for Audit, Dallas, at 214-767-6980.
Attachment
BACKGROUND, OBJECTIVES, SCOPE, AND METHODOLOGY 2
FINDINGS AND RECOMMENDATIONS 3
ATCPIC Austin/Travis County Private Industry Council
CMA Career Management Account
CTCOG Central Texas Council of Government
DOL U.S. Department of Labor
ETA Employment and Training Administration
JTPA Job Training Partnership Act
OIG Office of Inspector General
RFP Request for Proposal
SDA Service Delivery Area
SPIR Standardized Program Information Record
SSG
Substate Grantee
We conducted a limited scope performance audit of the Job Training Partnership Act (JTPA), Title III, Career Management Account (CMA) Demonstration Grant No. 5502-5-00-87-60 between the U.S. Department of Labor (DOL), Employment and Training Administration (ETA), and the Central Texas Council of Government (CTCOG), Service Delivery Area (SDA). Our objectives were to determine how the CMAs operated under this grant and to determine if there was a significant difference between the training strategies and outcomes for the CMA and control group participants.
We found that:
The grantee's response attempted to provide justifications and explanations for the report findings but did not refute any facts provided in the report. The grantee's response is provided after each finding in the report and in its entirety at Appendix I. Our findings remain unchanged.
Recommendations
We recommend the Assistant Secretary for Employment and Training:
Our objectives were to determine how the CMAs operated under this grant and to determine if there was a significant difference between the training strategies and outcomes for the CMA and control group participants.
The audit period was July 1, 1995 through February 24, 1997, during which time 134 CMA and 67 control group participants were served. Since this demonstration grant was still in progress at the completion of audit fieldwork, we do not have the reported final performance statistics or dollar amounts expended.
We reviewed case files for a random sample of 76 of the 134 CMA participants and 36 of the 67 control group participants. The 112 participants in our sample consisted of 51 active and 61 terminated participants. Our purpose for the participant case file reviews was to determine what happened to the CMA and control group participants during their JTPA enrollment and compare the program outcomes for the two groups.
A separate Management Letter Report No. 06-98-004-03-340 was issued to the Assistant Secretary for Employment and Training related to ETA's management of this grant.
The audit was performed in accordance with the generally accepted auditing
standards and Government Auditing Standards (1994 Revision) promulgated
by the Comptroller General.
Neither the JTPA nor applicable regulations provide for the payment of participants' stipends without a determination of the participants' bona fide needs. Yet, by May 31, 1997, approximately $117,785 of the $578,612 grant funds (20 percent) disbursed had been spent on participants' stipends.
The grantee mandated that these funds be used only for cash stipends.
The grantee's policy was that each CMA limit for stipends was $1,000 but
in practice the amount was usually greater if the length of training resulted
in the participant exhausting this amount.
At least 15 of the 76 CMA participants (20 percent) whose files we reviewed
attended training classes part-time, as few as 4 hours per week, yet still
collected full stipends. Participants were required to have class attendance
forms signed by instructors and returned to the SDA to receive a stipend.
It apparently did not matter how many hours per week the participant was
scheduled for training.
Office of Management and Budget Circular No. A-87 (1995 revision), Attachment A, paragraph C1, provides:
Grantee's Response
"The ETA Request for Proposals states that the ultimate purpose of the
Career Management Account Demonstration 'is to determine if career management
accounts are an administratively feasible approach for achieving goals.'
The RFP further states that 'demonstration grants are not subject to
the cost limitations for formula-funded Title III grants at section 315
of the JTPA.... [however,] supportive services that exceed 25 percent of
the funds requested in the application shall provide a narrative justification.'
Additionally, while attending the initial grantees meeting in September
1995, [an ETA representative] directed grantees that funds are not 'through
block grants; they are not bound by JTPA cost limitations, record keeping,
or reporting, and they don't have to follow formula rules.' He further
suggested that projects 'should document adjustments made to make a
program work.' During the same grantees [sic] meeting, [another ETA
representative] explained that while budget and extensive program modifications
required written modification through grant officers, process modifications
required discussing the modification with the grant program manager."
"When developing individual career management accounts, one of the barriers faced by CTCOG and Austin/Travis staff was the need to tell individual customers from the beginning of training how much funding is available for support services. The intent was to provide customers with the ability to make informed decisions based on a specific amount of funding to be used for supportive services and, to the extent possible, replicate a true vouchering system. The payment system, which required turning in attendance documentation, was devised as a means of ensuring attendance and participation while encouraging case management contact to document activities. The use of stipends and fixed amounts for supportive services, while not generally allowable under JTPA, Title III, is allowable as a JTPA, Title IIA, expenditure. Specifically, Section 627.310(2): (f) An SDA or SSG may set fixed levels of benefit for any supportive service. (g) (1) For purposes of title II, financial assistance is defined as a general supportive service payment for the purpose of retraining (sic) participants in training. (2) Financial assistance payments may be considered to be necessary for participation in training for title II participants, i.e., a separate, individual determination of need is not necessary."
Auditor's Conclusion
The grantee's response regarding cost limitations is not relevant to the reason the costs were questioned. We questioned these costs for the grantee's failure to assess each participant's needs before making stipend payments to all participants enrolled in training. We did not question the costs for a cost limitations exception.
The grantee's reference to Title IIA regulations is also not relevant
to the finding. The JTPA Title IIA program embraces a different type of
clientele (economically disadvantaged) than the
Finally, CMA funds earmarked for stipends were mandated by the
grantee for that purpose and could not be used for such things as additional
training. In our opinion, this mandate does not lend itself to a true voucher
system.
Recommendation
We recommend the Assistant Secretary for Employment and Training disallow
the $117,785 in questioned stipend payments during our audit period and
any such additional payments after the audit period.
2. The grantee failed to terminate CMA participants
90 days after the final service was provided.
Of the 76 CMA participants whose case files we reviewed, 16 (21 percent) were not terminated after the participants failed to receive any training or services for more than 90 days while enrolled in the JTPA program. These 16 participants should have been terminated negatively from the program.
Of these 16 participants, 4 were eventually terminated prior to completion of our audit fieldwork but were not terminated until after the participants found employment. Twelve participants (75 percent) still had not been terminated at the completion of audit fieldwork, having been inactively enrolled in the program as much as 1 year beyond the 90-day limitation. These facts appear to indicate that the grantee was not terminating participants as required to avoid negative terminations.
The failure to terminate these participants violates the JTPA Title III Standardized Program Information Record (SPIR) reporting requirements (Appendix C) -- an extension of the JTPA regulations -- which reads in part:
DOL/ETA does not grant waivers for the required termination of JTPA participants beyond the time periods specified above.
o An individual meets the goals identified in his/her Individual
Service Strategy (i.e., attains GED), or
o Participant gains employment.
Grantee's Response
"The Career Management Account Request for Proposals proposed demonstration accounts that promoted: 'increased customer choice and allowed maximum flexibility in customizing services, and services delivery.' The RFP further states that 'the career management account is intended to provide maximum flexibility to the individual worker in selecting the types, timing and sources of career and employment assistance. The account is tied to the individualized worker.' Therefore, based on the criteria of the RFP, participation and/or inactivity had initially been identified by CTCOG and Austin Travis County partners as a factor of customer choice. However, Regional Department of Labor in Dallas conducted a site visit/audit in December 1996 and indicated that inactivity must result in termination and should not be considered a matter of customer choice."
"Prior to DOL review:
Because the control group was to be treated exactly as is expected under
Title III, they were terminated for inactivity. However, because the experimental
design was to reflect unlimited customer choice, experimental customers
were not required to report regularly to case managers and were even given
the option to select training/service timing. Stipends were based on attendance
and at no time were stipends paid to those customers who were inactive."
An attempt was made to contact all inactive CMA customers. They were
required to either begin participating or be terminated. No participants
were inactive for more than 90 days after the December review and recommendation
by DOL."
Auditor's Conclusion
We understand that the CMA should provide maximum flexibility. However,
as reflected in the report, most of the participants in the "failure to
terminate" category had either dropped out of the program and/or had moved
and could not be contacted. This situation should not be considered a factor
of a customer's choice but rather concrete grounds for terminating the
participants.
The grantee's own proposal/statement of work calls for termination when
"an individual significantly fails to participate in their chosen service
delivery by failing to attend, participate, or achieve in a manner that
demonstrates effort on the part of the individual."
We do not dispute the grantee's statement that an attempt was made to contact all inactive participants. In fact, many attempts were made to contact participants. However, we do dispute the grantee's statement that no cases were open more than 90 days after ETA's December 1996 review. The OIG's participant case file review took place during March/April 1997. While we cannot effectively comment on the cases reviewed in March (within 90 days of December 1996), the April case file reviews included some cases that had been inactive and still not terminated. For example, two participants relocated to other states while a third participant moved to another city within the state. These participants had not been successfully contacted since early to mid 1996 with their case files inactive for at least as long.
Recommendation
We recommend that the Assistant Secretary for Employment and Training
ensure that all grantees operating demonstration grants currently, and
in the future, follow guidelines regarding participants' termination from
the programs after inactivity for 90 days to allow for non-biased comparisons
between demonstration programs and regular JTPA programs.
The purpose of the CMA grant was to provide some participants greater
flexibility in choosing their training curriculum and training provider.
While there were exceptions, for the most part CMA participants were enrolled
in the same training institutions and programs that were endorsed for the
control group. Additionally, several of the CMA participants were already
engaged in the training prior to JTPA enrollment, all of them attending
college. It appeared that their primary motive to enroll in JTPA was simply
to obtain the financial aid to continue their education. All of these participants
were still enrolled in JTPA as of March 1997, with approximately 3 months
remaining in the grant.
The major difference between the CMA and control group participants
was the CMA group's ability to visually observe the funds credited and
debited to their CMA; i.e., they knew how much JTPA funds had been spent
on their behalf and how much was still available.
Although the CMA group was provided more flexibility, both the placement
rate and starting hourly wage for the control group were significantly
higher than for CMA participants.
Although we did not have access to the final performance data, we determined
the outcome comparison for the two groups based upon an audited
sample of 112 case files -- 76 CMAs, 36 control group. At the time audit
fieldwork was performed, 78 percent (28 of 36) of the control group had
been terminated from JTPA versus only 43 percent (33 of 76) of the CMA
group. However, due to inactivity for more than 90 days, an additional
12 of the 33 active CMA participants should have been terminated. (See
finding 2.) We found no evidence that any of the control group active participants
had been inactive for more than 90 days.
Our determination that the control group out performed the CMA group is based on the following data.
Additional evidence that the control group outperformed the CMA group is the difference in the average starting hourly wage for all participants claimed as placed:
Grantee's Response
"As with any experimental design, outcomes vary based on experimental application -- Each experimental/control group was provided services in accordance with their groups service design. One could simply state that outcomes varied based on the effectiveness of approaches to service delivery. However, because of the time-limited nature of CMA funding several intervening factors have been identified as possible explanations for the diminished performance of the experimental group."
"Specifically, grant award was received on July 1, 1995 without prior
notification and the initial grantee meeting followed in September. The
program 'roll-out' was delayed accordingly resulting in participant enrollment
beginning in October with two-year training institutions beginning Spring
enrollment the following January. Therefore, CMA participants who selected
two-years of training were not provided with enough time to expend funds
and complete a two year training program. Because CMA criteria stresses
customer choice, customers were given the option of selecting a shorter
training program, possibly with less potential for success, or selecting
a two-year training program and completing the program with alternate funding.
While most customers selected two-year training institutions and programs
very similar to those used by the control group, these selections could
potentially diminish the outcomes of the experimental group in comparison
to the control group -- Shorter training, for example, could result in
(1) a lesser wage affecting wage at placement, or (2) a less desired employment
opportunity contributing to increased customer drop out. Long-term training
could result in (1) training completed after CMA termination which would
reflect as a negative termination to CMA but a positive outcome for the
customer, or (2) inability to continue training because of lack of available
funding."
"Finally, experimental design, by definition, includes projected outcomes which may or may not be obtained. Variance in the performance of each of the groups has clearly been observed, but as stated above, due to the nature of the funding, there are multiple factor(s) which could be attributed to the variance."
Auditor's Conclusion
By applying JTPA Title III regulations and guidelines, we determined that the CMA group did not perform as well as the control group.
The customer groups may have been randomly selected, but the fact remains that all of the participants who were enrolled in training prior to enrollment in JTPA were in the CMA groups; none were in the control group. While we do not question the eligibility of these participants, some participant case files indicate that the only reason for enrolling in JTPA was to obtain financial aid to continue with their education. Also, three of these participants had obtained JTPA Title III eligibility status under the "long term unemployed" criteria, defined as unemployed 15 weeks of the 26 weeks immediately prior to JTPA enrollment. One might assume that if an individual was pursuing an education and not looking for work, that individual might very well be unemployed for this period of time.
The JTPA dislocated worker program, in our opinion, was never intended
to serve as a means for individuals to obtain financial aid for the purpose
of attending college in pursuit of various degrees, but rather to provide
truly dislocated workers with appropriate services to help them obtain
employment. To allow enrollment into JTPA of those participants already
enrolled in college curricula, either for undergraduate or advanced degrees,
diminishes available funds to those truly dislocated workers who
need the services to quickly return to the labor market.
Finally, we agree that there may be multiple factors which could attribute to the fact that the CMA participants performed worse than the control group. However, the fact remains that they did perform worse.
Recommendation
We recommend that the Assistant Secretary for Employment and Training
encourage grantees operating Title III demonstration grants, and other
Title III programs, to enroll only those participants who are truly dislocated
workers.