Detailed Information on the
Independent Living for People with Disabilities Assessment

Devise and implement an improved audit and site visit system to ensure that the agency is meeting its statutory oversight requirements.

Conduct periodic and high quality evaluations of each of the IL programs.

Develop efficiency measures and report data at the program and grantee levels. Work with grantees to improve their efficiency using these data.

Develop long-term performance goals and measures that reflect the four core areas of services and the standards and assurances for the IL State Grants and CIL programs and use these data to evaluate the effectiveness of the program.

Reduce the time needed to collect and analyze grantee performance reports and make the aggregate data available to the public on the Department's website in an accessible format.

Revise the Section 704 data reporting requirements to reflect any statutory changes due to reauthorization of the program authority.

Measure: As a result of direct services provided by a Center for Independent Living (including referral to another service provider), the percentage of CIL consumers who report having access to previously unavailable transportation, appropriate health care services, and/or assistive technology resulting in increased independence.

Explanation:Baseline data for this new outcome measure will be collected for the fiscal year 2006 project period and will be available in March 2007.

Measure: The percentage of Centers for Independent Living staff, board members, and/or consumers participating in advocacy initiatives, such as community committees, public information campaigns, and other community events designed to increase the accessibility of transportation, health care, assistive technology, and affordable housing within their communities.

Explanation:Baseline data for this new outcome measure will be collected for the first time for the fiscal year 2006 project period and will be available in March 2007.

Measure: Number of months from the end of each annual performance rating period until data are made publicly available.

Explanation:

Measure: Cost of providing effective independent living services.

Explanation:This measure will be calculated by dividing the number of consumers served by the net resources of the grantee (total budget minus pass-through funds). This information is currently available, but ED is determining how costs can be weighted for effectiveness using data from the new outcome measures that will be available in March 2007.

Is the program purpose clear?

Explanation: Independent Living (IL) programs promote: (1) leadership, empowerment, independence, and productivity of individuals with disabilities; and, (2) the integration and full inclusion of individuals with disabilities into mainstream American society.

Evidence: Rehabilitation Act of 1973, as amended, Title VII, Chapter 1, Parts B and C

Does the program address a specific and existing problem, interest, or need?

Explanation: Unlike vocational rehabilitation and other disability programs within ED, IL programs provide services (information and referral, independent living skills training, peer counseling, and systems and individual advocacy) to persons whose disabilities might make competitive employment difficult to obtain, but for whom independent living goals are feasible. The IL programs are uniquely designed among Federal disability programs to help states and localities assist individuals find the support they need to live independently in a community-integrated setting, as mandated by the Supreme Court decision Olmstead v. LC In addition, demographic trends in aging and disability mean the need for these services is likely to increase.

Evidence: By 2020, the number of people with disabilities is projected to increase to 53.7 million or 1 in every 6 Americans (SOURCE: Census Bureau Projection). The likelihood of having a disability increases with age. Moreover, within the disabled population, the number of Americans 65 and over aging with disabilities was estimated to be 34.9 million on November 1, 2000, which is 12 percent higher than it was in the 1990 Census. According to the most recent disability supplement to the National Health Interview Survey conducted by the Census Bureau, 4.9 million people receive help in at least one basic activity of daily living (ADL) and an additional 8.3 million people receive help with an instrumental activity of daily living (IADL) but not an ADL. (SOURCE: NHIS-D, 1994-95). The NHIS-D data also show that 45 percent of people with disabilities live alone, and 26 percent of those who live with others, need more help than they are getting.

Is the program designed so that it is not redundant or duplicative of any Federal, state, local or private effort?

Explanation: In most States, centers for independent living (CILs) are supported through a combination of Federal, state, and local funding. Redundancy and duplication are limited through a two-tiered funding structure. Program funds are first allocated among States according to their relative share of population and then distributed through intra-state competitive grants. But States must justify the need for new CILs or increased funding to existing CILs through a Statewide Plan for Independent Living (SPIL) which describes efforts to coordinate Federal and State funding for CILs and services. However, the CIL program overlaps with RSA's Independent Living State Grants program. Since funding for CILs and the IL State Grants program requires that States demonstrate their ability to serve the entire state through the SPIL, the programs could be consolidated into a formula grant to the States, with set-asides for State administrative expenses and Statewide Independent Living Council administrative expenses.

Evidence: SPIL requirements are stated in Title VII, Chapter 1, Section 704(k) of the Rehabilitation Act of 1973, as amended.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: There are no major flaws in the design of this program but RSA could improve program efficiency while maintaining its goal of fostering independence for persons with disabilities. One alternative would be to administer the CIL program as a formula grant and allow RSA staff to devote more time to Federal activities'monitoring, promoting best practices, addressing policy and legal issues, and fostering improved financial management practices among grantees.

Evidence:  

Is the program effectively targeted, so program resources reach intended beneficiaries and/or otherwise address the program's purpose directly?

Explanation: There is no evidence that services are being provided to beneficiaries who do not need or merit them. States must demonstrate in their SPILs that new funding is being used to address unserved or underserved communities. Statutory changes would be required to authorize targeting of services within a CIL based on financial need, membership in an underserved disability group, or other characteristics, such as age or gender.

Evidence: State Plans for Independent Living, Section 704 reports submitted by CILs.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The program has not established measures that focus on outcomes in the long-term. States currently collect data for GPRA on outputs - such as the number of individuals who leave nursing homes and other institutions for community-based housing and the number of individuals at risk of entering nursing homes who are receiving IL services and can remain at home. However, variation in data reported by CILs casts doubt on their validity. As part of its revision of the Section 704 reporting requirements, RSA is currently re-examining the four core areas of service and translating them into long-term, performance-oriented goals and measures. The Department is also working with OMB on developing an appropriate efficiency measure for this program.

Evidence: Forthcoming: Corrective action plan or other document explaining the proposed changes to the Section 704 reporting requirements and providing justification for these changes.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: Pending development of new long-term outcome measures.

Evidence:  

Does the program have a limited number of specific annual performance measures that demonstrate progress toward achieving the program's long-term measures?

Explanation: RSA currently collects annual data for GPRA on numbers of individuals served under the CIL program and the IL State Grants program and the percentage of consumers served under these programs who achieve their goals. RSA is revising its Section 704 reporting requirements and developing annual measures that will show progress toward long-term outcomes. The Department is also working with OMB on developing an appropriate efficiency measure for this program.

Evidence: Forthcoming: Corrective action plan or other document explaining the proposed changes to the Section 704 reporting requirements and providing justification for these changes.

Does the program have baselines and ambitious targets and timeframes for its annual measures?

Explanation: Pending development of new annual performance measures.

Evidence:  

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, etc.) commit to and work toward the annual and/or long-term goals of the program?

Explanation: OSERS takes specific steps to ensure that all partners commit to and work toward the existing annual goals. Program solicitations (priority packages) explicitly include all program goals, and grant applications and progress reports assess performance and continuing relevance against these goals. Although existing program measures do not meaningfully measure the program's responsiveness to its stated goals, all partners do commit to and work toward these goals. Program staff are also currently working to develop both annual and long-term goals that are more appropriate for this program. Once the revised annual and long-term goals are implemented, OSERS can continue to use its current process to ensure that all program partners actually commit to and work toward the new measures.

Evidence: Forthcoming: Corrective action plan or other document explaining the proposed changes to the Section 704 reporting requirements and providing justification for these changes.

Are independent and quality evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: RSA has conducted regular evaluations of the Independent Living programs and used the findings and recommendations to improve program management. The final report for the most recent evaluation is currently under ED review. Unlike other principal offices within the Department however, OSERS administers its own program evaluations, potentially limiting the independence and objectivity of the findings.

Evidence: Previous evaluations: Berkeley Planning Associates, 1986; Research Triangle Institute, 1996; Research Triangle Institute, 1998; CESSI, Inc., 2002.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: The budget request has typically focused on program outputs, such as the number of new centers that could be supported, rather than program outcomes, such as reducing unmet need or increasing the number of individuals meeting independent living goals or the number of individuals leaving nursing homes.

Evidence: Congressional budget justifications.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The IL program is in the process of revising the Section 704 reporting requirements. RSA is re-examining the four core areas of service and will translate the priorities associated with them into long-term, performance-oriented goals and measures. The criteria being developed will enable the IL program to collect on a yearly basis specific information tied to outcome measures.

Evidence: Forthcoming: Corrective action plan or other document explaining the proposed changes to the Section 704 reporting requirements and providing justification for these changes.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: Each CIL grantee is required to have an annual independent audit. RSA uses these audit findings, site visit reports, and annual performance data submitted by grantees for section 704 reporting requirements to identify and correct program weaknesses. Grantees currently collect data based upon the minimal requirements set forth in section 725 of the authorizing legislation. RSA's ability to draw meaningful conclusions about center outcomes based on these data has been limited. RSA is currently revising the section 704 reporting requirements and will propose indicators to collect higher quality annual outcome data to inform management decisions.

Evidence: Forthcoming: Corrective action plan or other document explaining the proposed changes to the Section 704 reporting requirements and providing justification for these changes.

Are Federal managers and program partners (grantees, subgrantees, contractors, cost-sharing partners, etc.) held accountable for cost, schedule and performance results?

Explanation: EDPAS which links employee performance to relevant Strategic Plan goals and action steps, and is designed to measure the degree to which a manager contributes to improving program performance. However, ED cannot demonstrate specific ways by which RSA managers are held accountable for linking their performance standards to the program's long term and annual measures. Program partners are subject to project reviews and grant monitoring but these oversight activities are not designed to link partners to specific performance goals.

Evidence:

Are all funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: IL programs successfully obligate appropriated funds by the end of each fiscal year. Funds are spent in accordance with the program authority and Department regulations. For the past two years, however, applicants have had fewer than 45 days to submit applications for grants under this program. RSA should take steps to ensure that grant competitions are announced on a regular schedule that provides sufficient time for the preparation and review of applications.

Evidence: Audit reports, ED grant award database, and solicitations for grant competitions in the Federal Register.

Does the program have procedures (e.g., competitive sourcing/cost comparisons, IT improvements, approporaite incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: This program has not yet instituted procedures to measure and improve cost efficiency in program execution. However, as part of the President's Management Agenda, the Department is implementing "One-ED" -- an agency-wide initiative to re-evaluate the efficiency of every significant business function, including the development of unit measures and the consideration of competitive sourcing and IT improvements.

Evidence:  

Does the program collaborate and coordinate effectively with related programs?

Explanation: At the federal level, the independent living programs collaborate with: each other, other RSA and Federal programs, such as Social Security. To receive funding under the State grants program, each Designated State Unit must have a State Plan for Independent Living (SPIL) that demonstrates that it has appropriate planning, financial support and coordination, and other assistance to appropriately address, on a statewide and comprehensive basis, needs in the State for the provision of independent living services. The plan must document the working relationship between programs providing independent living services and independent living centers, the vocational rehabilitation program, and other programs providing services for individuals with disabilities. At the local level, CILs must demonstrate in their section 704 reports that they collaborate with other disability, health, and employment service providers to coordinate services that will enable individuals with significant disabilities to live independently in their own communities.

Evidence: State Plans for Independent Living, Section 704 reports submitted by CILs.

Does the program use strong financial management practices?

Explanation: No internal control weaknesses have been identified for the Independent Living programs.

Evidence: Inspector General Department audits

Has the program taken meaningful steps to address its management deficiencies?

Explanation: RSA has not demonstrated that it has a system in place to identify and address management deficiencies within these programs.

Evidence:  

Are grants awarded based on a clear competitive process that includes a qualified assessment of merit?

Explanation: RSA administers all available program funds through a clear competitive process that uses external panels of peer reviewers. However, grantees that have successfully competed for an award are not required to compete again as long as they meet the performance standards for the program. As a result, the majority of appropriated funds are distributed each year non-competitively.

Evidence: Rehabilitation Act of 1973, as amended, Title VII, Chapter 1, Part C, Section 722.

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: Independent annual audits are required for each CIL to ensure that grantees are meeting the terms of their awards and spending funds appropriately. RSA is also required by statute to perform site visits of 15 percent of grantees and 1/3 of the designated state units each year. However, the regional and headquarters staff have had difficulty meeting the site visit requirement.

Evidence: Audit and site visit reports.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: RSA collects annual performance data through the Section 704 reporting requirements. For reasons discussed above, the current reporting requirements do not generate data with which RSA can draw meaningful conclusions about grantee performance. In addition, the data collected through the Section 704 reporting requirements have not been made available to the public. Through a cooperative agreement with RSA, the Independent Living Resource Utilization (ILRU) Center has agreed to compile program performance data through grantee responses to the section 704 reporting requirements and publish this performance data on its website. The most recent data available online are from 1998 and the most recent data for SILCs are from 1997.

Evidence: ILRU website: www.ilru.org/704/index.html

Has the program demonstrated adequate progress in achieving its long-term outcome performance goals?

Explanation: These programs have not established measures that focus on outcomes in the long-term.

Evidence: N/A

Does the program (including program partners) achieve its annual performance goals?

Explanation: These programs are currently working to develop and implement more appropriate annual performance goals.

Evidence: N/A

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program performance goals each year?

Explanation: The Department is working with OMB on developing an appropriate efficiency measure for this program.

Evidence: N/A

Does the performance of this program compare favorably to other programs, including government, private, etc., that have similar purpose and goals?

Explanation: Although programs with similar goals and purposes may exist, no current studies, analyses, or evaluations have attempted to compare the Independent Living program to these programs.

Evidence: N/A

Do independent and quality evaluations of this program indicate that the program is effective and achieving results?

Explanation: Previous evaluations of these programs did not attempt to measure its effect on outcomes. However, the evaluations do indicate that the programs were meeting the legislative requirements.

Evidence: Previous evaluations: Berkeley Planning Associates, 1986; Research Triangle Institute, 1996; Research Triangle Institute, 1998; CESSI, Inc., 2002.