Detailed Information on the
Vocational Rehabilitation State Grants Assessment

In the upcoming reauthorization, work with the Congress to align federal funding with performance, and hold all states accountable for their performance. In this reauthorization, the Administration will revisit whether funding for this program should be reclassified as discretionary.

Collect data to allow comparison with other job training programs, including the necessary data to support new common measures.

Establish specific performance targets in the outyears. Also, consider whether any additional measures are appropriate for this program.

Continue efforts initiated in FY 2007 to improve program management by using performance data to monitor State agency performance and provide technical assistance to State agencies in areas of poor performance.

Take significant steps to improve program management using existing outcome data and make these data available to the public in a more timely manner.

Measure: Percent of State VR agencies (excluding VR agencies for the Blind) that assist at least 55.8 percent of individuals receiving services to achieve employment.

Explanation:

Measure: Percent of State VR agencies (excluding VR agencies for the Blind) that assist at least 85 percent of individuals with employment outcomes to achieve competitive employment (employment in an integrated setting at/or above the minimum wage).

Explanation:

Measure: Percent of State VR agencies (excluding agencies for the Blind) for which at least 80 percent of the individuals achieving competitive employment are individuals with significant disabilities. The criterion in 2005 was increased to reflect more ambitious targets.)

Explanation:

Measure: Percentage of general and combined State VR agencies that demonstrate an average cost per employment outcome between $6,000 and $16,500.

Explanation:The cost per employment outocmes will be calculated by dividing total Federal grant funds by the number of individuals achieving an employment outcome.

Measure: Percentage of general and combined State VR agencies that demonstrate an average annual consumer expenditure rate of at least 83 percent.

Explanation:The consumer service expenditure rate will be calculated by dividing the the State VR agency's total program expendirures by consumer expenditures.

Measure: Job Training Common Measure: Entered Employment - Percentage employed in the first quarter after program exit.

Explanation:Numerator: Of those who are not employed at registration, the number of adults who have entered employment by the end of the first quarter after exit. Denominator: Of those who are not employed at registration, the number of adults who exit during the quarter. The Department conducted a field test of State reported Job Training Common Measures data and is using the information in designing next data collection. The Department has faced numerous challenges in implementing the common measures, including States' ability to collect and report the data and changes in the Common Measures.

Measure: Job Training Common Measure: Retention in Employment - Percentage of those employed in the first quarter after exit that were still employed in the second and third quarter after program exit.

Explanation:Numerator: Of those who are employed in the first quarter after exit, the number of adults who are employed in the second and third quarter after exit. Denominator: Those who are employed in the first quarter after exit. The Department conducted a field test of State reported Job Training Common Measures data and is using the information in designing next data collection. The Department has faced numerous challenges in implementing the common measures, including States' ability to collect and report the data and changes in the Common Measures.

Measure: Job Training Common Measure: Increase in Earnings - Percentage change in earnings: (i) pre-registration to post program; and (ii) first quarter after exit to third quarter.

Explanation:Numerator 1:Participant's earnings first quarter after program exit minus participant's earnings two quarters prior to registration. Numerator 2:Participants earnings third quarter after program exit minus participant's earnings first quarter after program exit. Denominator 1: Participant's earnings two quarters prior to registration. Denominator 2: Participant's earnings first quarter after program exit. The Department conducted a field test of State reported Job Training Common Measures data and is using the information in designing next data collection. The Department has faced numerous challenges in implementing the common measures, including States' ability to collect and report the data and changes in the Common Measures.

Measure: Job Training Common Measure: Annual cost per participant

Explanation:Annual cost per participant is calculated by dividing the total program appropriaton by the number of program participants.

Measure: The percentage of general and combined state VR agencies that demonstrate an average annual cost per participant between $1,200 and $3,300. (New measure, added February 2007)

Explanation:A State agencys average cost per participant is calculated by dividing the total annual Federal grant funds awarded to the State under the VR State Grants program by the total number of individuals served by the State agency during the fiscal year.

Is the program purpose clear?

Explanation: The program's purpose is clearly defined in the authorizing statute and regulations: To provide services to individuals with disabilities so they can prepare for and engage in gainful employment.

Evidence: Section 100 (a) (2) of the Rehabilitation Act of 1973 and 34CFR part 361.1 and 361.5 (16).

Does the program address a specific interest, problem or need?

Explanation: Program addresses the specific need to help individuals with disabilities obtain employment. Individuals with disabilities are employed at lower rates than their nondisabled peers.

Evidence: According to the 1994-95 National Health Interview Survey -- 79 percent of adults without disabilities were working at time of interview and only 37 percent of those with disabilities were employed.

Is the program designed to have a significant impact in addressing the interest, problem or need?

Explanation: The Vocational Rehabilitation (VR) Grants to States program is the primary Federal vehicle for helping individuals with disabilities prepare for and obtain employment. Each year, about 1.2 million individuals with disabilities are in various stages of the VR process. Federal funding pays for over 80% of the program's costs, and performance data shows that this program helps many of these individuals with disabilities obtain employment (see IV.2). Given the large Federal share of total spending, and the unique role this program plays (see I.4), eliminating Federal funding for VR would significantly affect program outcomes.

Evidence: The I.4 response discusses the program's unique role, and IV.2 provides some annual outcome data. Per the authorizing statute, the State matching requirement for this program is 21.3 percent; VR program data shows that most States do not provide much more than this amount.

Is the program designed to make a unique contribution in addressing the interest, problem or need (i.e., not needlessly redundant of any other Federal, state, local or private efforts)?

Explanation: While the federal government supports many other job training programs, most of them are not tailored to meet the specific needs of individuals with disabilities. Individuals with disabilities often have special needs when it comes to job training and employment. For instance, individuals with disabilities receive the following services under the VR program: supported employment services (e.g., job coaches); personal assistance on the job; modified workplaces; assistive technologies (e.g., a screen reader for a blind individual); family support (so family members can help an individual obtain employment); and other services that individuals without disabilities normally would not benefit from or do not require. These services are generally not provided through other Federally-supported job training programs. Also, the Federal government's other disability-specific job training programs -- such as the Social Security Administration's Ticket to Work program and Projects with Industry -- often work through the VR program. VR agencies, for instance, are statutorily authorized to act as Employment Networks in the Ticket to Work program.

Evidence: Section 103 of the Rehabilitation Act lists the services provided under the VR program, many of which are specific to individuals with disabilities. Section 101(a)(5) of the Act requires VR agencies to give priority to serving individuals with the most significant disabilities, many of whom benefit from the specialized services provided under VR (about 86% of the individuals served are individuals with significant disabilities). Section 1148(c) of the Ticket to Work and Workforce Incentives Improvement Act explains the role of VR agencies in the Ticket to Work program. According to Department of Education evaluations and data, many individuals served by Projects with Industry grantees also receive services under the VR State Grants program.

Is the program optimally designed to address the interest, problem or need?

Explanation: While other approaches are potentially more effective than the current VR program (such as direct vouchers for employment services), there is no conclusive evidence that this is true.

Evidence:  

Does the program have a limited number of specific, ambitious long-term performance goals that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The VR program has annual performance goals that focus on outcomes and meaningfully reflect the purpose of the program. However, to date the Department of Education has not established measurable long-term performance goals for this program.

Evidence: Government Performance and Results Act performance indicators, and the performance indicators mandated under section 106 of the Rehabilitation Act.

Does the program have a limited number of annual performance goals that demonstrate progress toward achieving the long-term goals?

Explanation: The VR program has a limited number of annual performance goals that are discrete, quantifiable, and measurable. Some of these annual performance goals are similar to the common measures for job training programs. Performance targets are in place for FY 2002, FY 2003, and 2004.

Evidence: See annual goals in Section IV.2.

Do all partners (grantees, sub-grantees, contractors, etc.) support program planning efforts by committing to the annual and/or long-term goals of the program?

Explanation: State grantees are required to annually report data to the Department of Education. The Department uses these data to measure program performance on a national and State-by-State basis. However, from State to State, VR agencies have different philosophies as to the clientele that should be served, appropriate job placements for VR clients, and the focus VR agencies should place on job placement as opposed to independent living skills.

Evidence: Program regulations at 34CFR 361. Subpart D

Does the program collaborate and coordinate effectively with related programs that share similar goals and objectives?

Explanation: As a required One-Stop partner under the Workforce Investment Act (WIA), VR agencies are required to coordinate with specific Department of Labor programs, and the programs administered Office of Vocational and Adult Education (also within ED). The VR program also coordinates with SSA's Ticket to Work program. Finally, the VR program coordinates with ED's Special Education programs on the school to work transition of individuals with disabilities. However, at the federal level the VR program has historically not coordinated well with these other programs, even though many of them are within the same agency. Fortunately, there are promising signs that this collaboration may be improving. For instance, last year the Rehabilitation Services Administration (which administers VR) worked with other agenceis to develop common measures for job training programs.

Evidence: The Individuals with Disabilities Education Act, the Ticket to Work and Workforce Incentives Improvement Act, and the Workforce Investment Act all require VR agencies to collaborate with other federal programs.

Are independent and quality evaluations of sufficient scope conducted on a regular basis or as needed to fill gaps in performance information to support program improvements and evaluate effectiveness?

Explanation: The Department of Education initiated a longitudinal study of the VR program in 1992, and completed this study in 2001. While the Department has no plans at this time to conduct another large scale comprehensive evaluation, it is in the process of developing a 5-year evaluation plan.

Evidence: The Department's longitudinal study tracked 8,500 VR consumers at 37 locations for three years. The study examined attributes of those served, services provided, costs, resources available, local environments, and both short and long-term outcomes.

Is the program budget aligned with the program goals in such a way that the impact of funding, policy, and legislative changes on performance is readily known?

Explanation: The VR program does not have a good sense of how increases in federal appropriations translate into improved performance on program goals. The disconnect between program funding and goals is exacerbated by the fact that the VR program's funding is classified as mandatory, and automatically receives an inflationary increase each year.

Evidence: Rehabilitation Act of 1973, Section 100(b)(1).

Has the program taken meaningful steps to address its strategic planning deficiencies?

Explanation: There is evidence that the Rehabilitation Services Administration (RSA), which administers the VR program, has taken steps to address its strategic planning deficencies. For instance, RSA worked with other agencies to develop common measures for job training programs, and is continuing to develop the Standards and Indicators for VR performance required by Section 106 of the Rehabilitation Act.

Evidence: The FY 2004 President's Budget includes the common measures for job training programs developed by the Department of Education and other agencies. In the coming year, these agencies will develop guidelines for comparing the performance of similar programs using these measures.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: The Department of Education regularly collects credible performance information from the VR State agencies, and uses this information to monitor State activities and provide technical assistance. Still, overall the Department's use of this information to manage the VR program is weak. This information is also not timely, which makes it harder for the Department to use it to manage the program.

Evidence:  

Are Federal managers and program partners (grantees, subgrantees, contractors, etc.) held accountable for cost, schedule and performance results?

Explanation: This program has not instituted an appraisal system that holds Federal managers accountable for grantee performance. However, as part of the President's Management Agenda, the Department is planning to implement an agency-wide system -- EDPAS -- that links employee performance to progress on strategic planning goals. Grantee performance is monitored on an annual basis through review and approval of annual budget plans, compliance reviews, audits, and site visits. Further, the VR program uses statutorily-required Standards and Indicators to increase State accountability for performance. Grantees that do not meet Federal requirements are required to submit improvement plans and can have grants reduced or discontinued for serious or persistent failures to comply.

Evidence: Rehabilitation Act of 1973, Section 106.

Are all funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: The Department of Education obligates all VR program funding in the current year. States may carry over funding to the next fiscal, but must match funds in the current year (see matching requirement in I.3). Before the end of the fiscal year, States return funds they cannot obligate to ED for reallotment to other States. Grantees are also required to report expenditures to the Department (on an SF-269), and conduct an independent annual audit.

Evidence: The VR program has not lapsed Federal funds. When the Department of Education (ED) determines that a State has spent VR funds for an unallowable activity (typically arising from an audit), ED issues a Program Determination Letter and requests repayment. Although ED issues Program Determination Letters whenever necessary, they are rare.

Does the program have incentives and procedures (e.g., competitive sourcing/cost comparisons, IT improvements) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: This program has not yet instituted procedures to measure and improve cost efficiency in program execution. However, as part of the President's Management Agenda, the Department is implementing an agency-wide initiative to re-evaluate the efficiency of every significant business function, including the development of unit measures and the consideration of competitive sourcing and IT improvements.

Evidence:  

Does the agency estimate and budget for the full annual costs of operating the program (including all administrative costs and allocated overhead) so that program performance changes are identified with changes in funding levels?

Explanation: The Department of Education's FY 2004 Budget materials satisfy the first part of the question by presenting the anticipated S&E expenditures (including retirement costs) for this program, which constitute one percent of the program's full costs. While this is a small percentage of the total, the administrative costs associated with this program are high compared to the Deparment's other formula grant programs. ED has not satisfied the second part of the question because program performance changes are not identified with changes in funding levels. The VR program does not have sufficiently valid and reliable performance information to assess the impact of the Federal investment.

Evidence:  

Does the program use strong financial management practices?

Explanation: No internal control weaknesses have been reported by auditors.

Evidence:  

Has the program taken meaningful steps to address its management deficiencies?

Explanation: There is evidence that the Rehabilitation Services Administration (RSA), which administers the VR program, has taken steps to address its strategic planning deficencies. For instance, RSA worked with other agencies to develop common measures for job training programs, and is continuing to develop the Standards and Indicators for VR performance required by Section 106 of the Rehabilitation Act.

Evidence:  

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: Section 107 of the Rehabilitation Act requires RSA to conduct annual reviews and periodic on-site monitoring to determine whether State VR agencies are complying substantially with the provisions of its State plan (established under section 101 of the Act) and with the VR Evaluation Standards and Performance Indicators (established under section 106). RSA uses a uniform instrument to monitor State VR agency performance and maintain accountability in States' expenditure of federal funds.

Evidence:  

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: RSA collects performance data from VR agencies on an annual basis, but these data have not been timely, and are not readily available to the public. To illustrate this problem, performance data on the Standards and Indicators were not available online until recently. Also, RSA has failed to publish an annual report to Congress on the VR program (and other Rehabilitation Act programs), as required in the statute, since 1997.

Evidence: Limited data are available on the Department of Education's website.

Has the program demonstrated adequate progress in achieving its long-term outcome goal(s)?

Explanation: The Department has not established long-term outcome goals for its programs.

Evidence:  

Does the program (including program partners) achieve its annual performance goals?

Explanation: In general. the program has been successful in meeting its annual performance goals.

Evidence: Government Performance and Results Act annual reports. VR Standards and Indicators data.

Does the program demonstrate improved efficiencies and cost effectiveness in achieving program goals each year?

Explanation: The Department of Education collects data that may be able to illustrate whether State VR agencies are becoming more efficient in achieving their program goals. In addition, an efficiency measure is included as part of the Common Measures exercise (see IV.1 and IV.2). However, to date, the Department has been unable to provide data that informs on the common measures, including efficiency. The answer to this question could change if the Department provides the necessary data. Still, one factor that may make it harder to show increasing efficiencies is that, by law, VR agencies must give priority to individuals with the most significant disabilities. Since this requirement has been in place, VR agencies have served an increasing number of individuals with significant disabilities (i.e., harder cases).

Evidence: Rehabilitation Act of 1973, Section 101(a)(5)

Does the performance of this program compare favorably to other programs with similar purpose and goals?

Explanation: Past analyses has shown that VR agencies' job retention performance compared favorably to the former Job Training Partnership Act (JTPA) program, and a cursory comparison to similar federal programs shows that VR grantees still performs relatively well. However, to date, the Department has been unable to provide data on the Common Measures, which will allow for a better comparison. The answer to this question could change to "Large Extent" or "Yes" when the Department provides the necessary Common Measures data.

Evidence: In 1998, JTPA-funded programs had a 62% employment retention rate after 13 weeks. VR, in comparison, had a 84% retention rate after one year (source: the longitudinal study discussed in IV.5). In another comparison, VR and Projects with Industry have similar employment placement rates (both about 62%), even though the VR program services individuals with more significant disabilities (i.e., individuals who are typically harder to place).

Do independent and quality evaluations of this program indicate that the program is effective and achieving results?

Explanation: The Department of Education recently completed a multi-year Longitudinal Study of the VR State Grants program. First initiated in 1992, this study tracked 8,500 VR consumers at 37 locations for three years, and provided comprehensive information on the VR program including: characteristics of the persons served; the services VR agencies provide; program costs; resources available; local environments; and both short and long-term outcomes. Data from this study show that VR participants benefit form the program, particularly in terms of improvements in employment and earning status. The study also found that VR consumers had excellent job retention rates.

Evidence: Among the VR participants who obtained competitive employment, 84% were working one year after their case service records were closed (closure) and 78% were still working 3 years later. During this period, VR consumer's wages increased significantly, from $7.56/hour at time of closure to $13.48/hour three years later. Individuals with disabilities who obtained competitive employment worked, on average, about 35 hours a week.