Detailed Information on the
Traumatic Brain Injury Assessment

Develop annual performance measures.

Collecting data for newly approved long-term measure to determine baseline and targets.

Administration proposes to phase-out the TBI program. States can continue key program activities with funds from the Maternal Child Health block grant and state, local, and private resources.

Program to conduct independent evaluation of the program, via contract, including assessement of feasibility of health outcome measures.

Develop and propose a long-term health outcome measure for the program to more effectively gauge program performance.

Measure: Decrease the applicaton and reporting time burden of grantees by 5% per year for 4 years, thereby collecting more accurate and timely data.

Explanation:Some targets may be shown as NA (not applicable) because the program is not proposed for funding.

Is the program purpose clear?

Explanation: By statute, the purpose of the Health Resources and Services Administration's (HRSA) Traumatic Brain Injury (TBI) program is to 'improve access to health and other services regarding traumatic brain injury' (Section 1252 of the Public Health Service Act). To accomplish this purpose, the program competitively awards states planning, implementation, and post demonstration grants. These grants require a cash or 'in-kind' match of $1 for each $2 of Federal funds. States can use the grants for 1) expanding and improving access to comprehensive and coordinated community services and support; 2) implementing best practices; and 3) increasing their capacity to serve individuals with TBI. States are expected to generate support from local and private sources to sustain TBI projects after the Federal grant period expires. Grants cannot be used to support primary injury prevention initiatives, research initiatives, or the provision of direct services. The program also awards formula Protection and Advocacy Traumatic Brain Injury (PATBI) grants to states to support their Protection and Advocacy (P&A) Systems. By statute, the first $3 million appropriated to TBI funds PATBI grants. State P&A Systems are federally mandated programs that pursue legal, administrative and other appropriate remedies to protect the rights of individuals with developmental disabilities.

Evidence: Planning grants require that states develop the following: 1) a designated government agency; 2) advisory board; 3) a needs and resource assessment of TBI services in the state, and 4) a plan of action to improve TBI services which was based on the needs/resource assessment. Planning grants are designed to assure that States create a solid base for the provision of TBI services and develop a strategic plan for improvement of TBI services. Implementation grants are awarded to states who have developed the four componpents of the planning grant. These grants support the states in implementing parts of their strategic plan and are used to develop best practices. Post Demonstration grants are awarded to states who have completed an Implementation grant. This grants supports the state in developing, changing, or enhancing community-based service services for individuals with TBI. (FY 2004 HRSA's Traumatic Brain Injury State Grants Programs Application Guidance) Protection and Advocacy Traumatic Brain Injury (PATBI) grants are awarded to states on a population basis. States use the PATBI grant to evaluate their TBI P&A capacity and develop plans to ensure P&A service (e.g. individual and family advocacy, self-advocacy training, information and referral services, and legal representation). Each state is required to have a P&As System. In addition to PATBI, state P&A Systems are supported by seven other Federal programs: Protection & Advocacy for Persons with Developmental Disabilities (PADD), Protection & Advocacy for Individuals with Mental Illness (PAIMI), Protection & Advocacy for Individual Rights (PAIR), the Client Assistance Program (CAP), Protection & Advocacy for Assistive Technology (PAAT), Protection & Advocacy for Beneficiaries of Social Security (PABSS), and Protection & Advocacy for Voting Accessibility (PAVA). The governor in each state must designate an agency to be the P&A system and insure that the P&A system is independent of service providers (2003 National Association of Protection and Advocacy Systems, Inc.. Annual Report).

Does the program address a specific and existing problem, interest or need?

Explanation: Traumatic brain injury is the sudden physical damage to the brain, often caused by motor vehicle crashes, falls, sports injuries, violent crimes, and child abuse. Although TBI can cause chronic physical impairments, often the individual has more disability due to problems with cognition, emotional functioning, and behavior in connection with interpersonal relationships, school, or work. An estimated 5.3 million Americans are living with the effects of TBI. Many individuals with TBI require a variety of long-term support services to remain in the community. The Federal TBI program provides an opportunity for States to aggregate existing resources to create a coherent service delivery mechanism. The grant requires that states identify service gaps and develop an action plan to target resources to the areas of greatest need. Seven states have not surveyed their resources or developed a strategic plan for addressing the health needs of individuals with TBI.

Evidence: 1. US General Accounting Office. Traumatic Brain Injury: Program Supporting Long-Term Services in Selected States. (1998)2. Corrigan, John D. Ph. D., Conducting Statewide Needs Assessments for Persons with Traumatic Brain Injury, Journal of Head Trauma Rehabilitation 2001; 16(1):1-19.3. CDC Traumatic Brain Injury Facts (www.cdc.gov)4. HRSA's Guide to State Government Brain Injury Policies, Funding and Services (www.tbitac.org)

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: The TBI program is the only source for funding that focuses solely on coordinated systems development for TBI services. TBI grants cannot be used to support primary injury prevention initiatives, research initiatives, or the provision of direct services.

Evidence: In 2003, 23 states provided long-term community-based services to adults with TBI through Medicaid waivers. The Department of Education's National Institute on Disability and Rehabilitation Research (NIDRR) administers a TBI Model Systems research program. NIDRR carries out a comprehensive and coordinated program of rehabilitation research and related activities. The National Institute of Neurological Disorders and Stroke (NINDS) supports and conducts research, both basic and clinical, to reduce the burden of neurological disease, including traumatic brain injury. NINDS fosters the training of investigators in the basic and clinical neurosciences, and seeks better understanding, diagnosis, treatment, and prevention of neurological disorders. The National Highway and Transportation Safety Administration's (NHTSA) administer prevention programs to reduce the incidence of TBI and other accident-related injuries.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: TBI Program is designed to build infrastructure for traumatic brain injury services by providing state agencies with resources to initiate development of and enhancements to medical and social systems of services and supports for individuals with TBI. The three competitive grant types (Planning, Implementation, and Post Demonstration) provide each state with the opportunity for assistance that is appropriate for its level of planning and a process for continued help as the state progresses. Funds can only be awarded to a state agency or an entity designated by the state. Awarding funding to the states rather than localities is efficient. Because states license many of the TBI service providers, they are in a position to bring together the needed partners and coordinate TBI services. States must provide a cash or 'in-kind' match of $1 for each $2 of Federal funds. This increases the likelihood that states will continue support for TBI services after the Federal grant expires.

Evidence: 1. FY 2004 HRSA's Traumatic Brain Injury State Grants Programs Application Guidance 2. Section 1252 of the Public Health Service Act

Is the program effectively targeted, so that resources will reach intended beneficiaries and/or otherwise address the program's purpose directly?

Explanation: The intended beneficiaries of the program are individuals with TBI. The TBI program provides resources to states to target the development of infrastructure to improve access to appropriate services for individuals with TBI. Forty-nine of the fifty-six eligible states and territories have been awarded Planning grants and have increased their ability to coordinate services for individuals with TBI. States must use Implementation or Post Demonstration grants to accomplish a component of their state strategic plan.

Evidence: 1. Section 1252 of the Public Health Service Act. 2. FY 2004 HRSA's Traumatic Brain Injury State Grants Programs Application Guidance 3. HRSA FY 2005 GPRA Report

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: No long-term health outcomes measures exist for the program or were developed for the FY 2006 PART. The program currently measures the number of states who have received a planning grant. The program has proposed measuring the number of states who have implemented 50% of their TBI action plan and the number of services provided, but has not developed a measure to track whether or not the program is improving health outcomes for individuals with TBI.

Evidence: HRSA FY 2005 GPRA Report

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The program has not established long-term health outcomes measures. Therefore, associated ambitious targets with clear time frames have not been developed.

Evidence:  

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The program lacks a long-term health outcome goal. Therefore, the program does not have annual performance measures that directly support a long-term outcome goal. The program has a developmental efficiency measure. HRSA's Maternal Child Health Bureau anticipates implementing a new web-based grant application system by the end of FY 2004 to streamline the grant application process.

Evidence:  

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The program has not established long-term health outcomes measures. Therefore, associated ambitious targets with clear time frames have not been developed. The program's developmental efficiency measure does not have a baseline. HRSA's Maternal Child Health (MCH) Bureau plans to implement a new web-based discretionary grant application system by the end of FY 2004 to streamline the grant application process. HRSA MCH Bureau expects that the system will reduce the time needed to complete an application by 5% per year for the next four years. Once the system is in place, the program will be able to establish baselines and targets.

Evidence:  

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: Because long-term health outcome measures have not been developed for the FY 2006 PART, partners and grantees do not commit to and work toward the annual and/or long-term goals of the program. The program does not monitor the State's performance in improving access to comprehensive services for individuals with TBI. All State programs must commit to develop the four core components of the TBI program (Advisory Board, Lead Agency, Needs and Resources Assessment and State Action Plan). However, States determine how they define and implement the four components. The program does not include a performance requirement in either the competitive or protection & advocacy grants.

Evidence: 1. FY 2004 Grant Application Guidance 2. Grantee annual reports.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: The program does not have regularly scheduled objective, high quality, independent evaluations that examine how well the program is accomplishing its mission. The national association of state head injury programs, the National Association of State Head Injury Administrators, conduced a survey of 23 states in 2002. However, this was not a comprehensive survey. It focused on the impressions of state health injury administers and did not evaluate comprehensively whether or not the program has improved TBI services. In addition, NASHIA has a potential conflict of interest and most respondent states were located in the northeast or southwest, introducing a potential bias.

Evidence: 1. National Association of State Head Injury Administrators, sdResults of 2002 Public Policy Survey

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: The program does not provide a presentation that makes clear the impact of funding, policy or legislative decisions on expected performance nor does it explain why a particular funding level/performance result is the most appropriate.

Evidence: HRSA FY 2005 Justification of Estimates for Appropriations Committees

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The majority of deficiencies highlighted in questions 2.1 through 2.7 have not been addressed. The program does not have plans to develop health outcome measures or conduct an independent evaluation. The program is, however, developing an efficiency measure that would apply to all HRSA Maternal and Child Health Bureau programs in the reporting of financial and program performance data. It is anticipated that a new web-based system will be implemented by the end of FY 2004. The program also anticipates that this system will greatly reduce the application and reporting burden for grantees. Baseline data are not yet available, but are expected prior to the release of the FY 2006 Budget. The program is aiming to reduce the amount of time it takes to complete applications by at least 5 percent per year for the next 4 years. To date, HHS/HRSA has not tied their budget requests to the accomplishments of the annual and long-term performance goals. HHS does plan to submit a performance-based budget beginning in FY 2006, but is it unclear whether this budget will show the marginal impact of funding

Evidence:  

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: HRSA requires funding recipients to submit annual progress reports and requires a final report from each grantee at the end of the project period. While there is not enough evidence to demonstrate full use of performance data to improve program performance, these reporting mechanisms are designed to achieve that end.

Evidence: Financial status reports are dues 90 days after the end of the fiscal year. Grantees are required to submit annual reports and a report at the end of the project period. The program is developing a reporting mechanism for P&A grantees.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Federal managers for the TBI program are officers in the Public Health Service Commission Corps. Commission Corps members receive a standard annual performance evaluation. While the performance of the TBI Program can be considered in the the evaluation of the Program Director and supervising Division Director, evaluations do not explicitly consider the management oversight of the program's performance, costs, and schedule. The program's GPRA goals are not required to be considered as part of the TBI federal managers' formal performance assessment. All grantees are held to fulfilling any conditions placed on their grants by the review panel. Progress toward meeting grant conditions is monitored by both program and grants management staff. Changes in the objectives of the grant project must be submitted for approval by the Program Officer. Contractors are required to provide quarterly reports of activities, accomplishments and challenges as well as a final summary report.

Evidence: 1. Generic Commission Corps annual evaluation 2. Grantee Annual Report 3. Draft P&A Reporting form 4. FY 2004 HRSA's Traumatic Brain Injury State Grants Programs Application Guidance

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: Federal funds from this program have been obligated in a timely manner. Grant funds are obligated via the automated Grants Management system. The Program Director monitors the expenditure of funds through monitoring of progress reports, grant actions and annual calls with grantees. There have been no unobligated Federal Program funds at the end of the fiscal year. All grantees have obligated funds by 90 days after the end of the fiscal year.

Evidence: Federal funds were appropriated in February 2004 and a portion of the state grant awards were made on April 1st. The remainder of the funds is scheduled for obligation on September 30th. All funding requests are reviewed for consistence with grant guidance. Any inconsistent applications are not awarded. All post-award project changes must be approved by HRSA.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: TBI grant applications are currently paper-based. HRSA's Maternal Child Health (MCH) Bureau is in the process of implementing a web-based grant application system. The TBI Program outsources technical assistance and resource development through a competitive contract.

Evidence: Beginning in September 2004, all MCH Bureau applications will be web-based.

Does the program collaborate and coordinate effectively with related programs?

Explanation: The Federal TBI Program has engaged in meaningful collaboration with the Department of Education's TBI Model Systems research program, Centers for Disease Control and Prevention's TBI Surveillance program, and the Department of Labor.

Evidence: The program worked with the Department of Education's TBI Model Systems research program to educate State TBI systems grantees about the Model Systems' research findings. This process has assisted the TBI systems grantees in identifying which the Model Systems programs that are doing research in their area of concern. The program has worked with the Centers for Disease Control (CDC) TBI Surveillance program to arrange for grantees to collaborate in creating a services system. States that have been awarded a CDC surveilance grant are required place the CDC grantee on the TBI State Advisory Board. CDC and HRSA jointly published an outcomes document on children with TBI. The HRSA TBI program collaborated with the Department of Labor to sponsor training on how to respond to the Supreme Court's Olmstead decision.

Does the program use strong financial management practices?

Explanation: In FY 2003, HHS OIG conducted an HHS financial statement audit. The audit reported that the Department had serious internal control weaknesses in its financial systems and processes for producing financial statements. OIG considered this weakness to be material. The audit recommended that HHS improve their reconciliations, financial analysis, and other key controls. The September 30, 2002 HRSA independent auditor's report found that the preparation and analysis of financial statements was manually intensive and consumed resources that could be spent on analysis and research of unusual accounting. The audit also found that HRSA's interagency grant funding agreement transactions were recorded manually and were inconsistent with other agencies' procedures. Finally, the audit found that HRSA had not developed a disaster recovery and security plan for its data centers.

Evidence: 1. HHS FY 2003 Performance and Accountability Report 2. HRSA's 2002 audit report

Has the program taken meaningful steps to address its management deficiencies?

Explanation: HHS' long-term strategic plan is to resolve the internal control weaknesses is to replace existing accounting systems and other financial systems within HHS with the Unified Financial Management System (UFMS). HHS plans to fully implement the UFMS Department-wide by 2007. HRSA developed a corrective action plan to address the reportable conditions identified in the September 30, 2002 independent auditor's report. For each aspect of the five reportable conditions, HRSA assigned an office responsibility. The plan also outlines milestones and target completion dates.

Evidence: 1. HHS FY 2003 Performance and Accountability Report 2. HRSA Corrective Action Plan for FY2002 Financial Statement Audits as of 4/30/2003 - waiting for this.

Are grants awarded based on a clear competitive process that includes a qualified assessment of merit?

Explanation: The TBI Program awards grants on a competitive process using an objective review panel. Applications are read by multiple reviewers, presented to the Panel, discussed and then assessed against the review criteria. Reviewers score each application against the review criteria. The applications are then ranked by their individual scores for funding decisions. To insure fairness, the review panel process is conducted by HRSA's Division of Independent Review and not by the program. P&A grants are awarded via formula to the 59 eligible states and territories.

Evidence: Approximately $6 million of the program's annual appropriation is allocated by the HRSA's Division of Independent Review. The program uses the statutory formula to allocate $3 million annual in P&A grants.

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: The Federal Program collects information on contacts, grant objectives, accomplishments and products produced. The Program uses HRSA's Grants Electronic Management System to track grantee financial status and to maintain follow up on grant conditions and recommendations. Project activities are outlined and reviewed as part of the continuation application process. Either the federal project officer or the TBI technical assistance center staff is in contact with the grantee on a monthly basis. The grantee's Final Report is required to summarize the project, its accomplishments and remaining challenges.

Evidence: Grantees are required to submit annual reports and a report at the end of the project period.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: The Accomplishments are collected on an annual basis and summarized in Grantee Profiles. The Grantee Profiles are distributed as a hard copy at the Annual Federal TBI Program Grantee meeting and are available on the program's website.

Evidence: The program's TBI website (www.tbitac.org) has a TBI and P&A profile for each state that has received a grant. Each state's fact sheet lists the objectives of the Federal TBI grant and the state's progress towards achieving the objectives.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: The program has not adopted long-term health outcome goals. The outcome of the program should be to improve the health and/or well-being of individuals with TBI.

Evidence: Questions 2.1 and 2.2

Does the program (including program partners) achieve its annual performance goals?

Explanation: The program has not developed a long-term health outcome measure associated annual goals.

Evidence: Questions 2.3 and 2.4

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: During the PART process, the program developed an efficiency measure. HRSA's Maternal Child Health Bureau anticipates implementing a new web-based grant application system by the end of FY 2004 to streamline the grant application process. Once the system is in place, the program will be able to track progress towards the new efficiency measure.

Evidence: Question 2.8

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: The TBI program is the only source for funding that focuses solely on coordinated systems development for TBI services. There are no programs with similar purpose.

Evidence: Question 1.3

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: No independent evaluations of sufficient scope and quality have been conducted to date.

Evidence: Question 2.6