Skip Navigation HRSA - U.S Department of Health and Human Services, Health Resources and Service Administration U.S. Department of Health & Human Services
Home
Questions
Order Publications
 
Grants Find Help Service Delivery Data Health Care Concerns About HRSA
FY 2008 Annual Performance Review
 
PDF FY 2008 Annual Perfomance Review
(PDF – 908 KB)

Evaluations Completed in FY 2008

Health Centers Program

Many Underserved Areas Lack a Health Center Site, and the Health Center Program Needs More Oversight (GAO-08-723)

The Government Accountability Office (GAO) examined (1) to what extent medically underserved areas (MUA) lacked health center sites in 2006 and 2007 and (2) HRSA’s oversight of training and TA cooperative agreement recipients’ assistance to grant applicants and its provision of written feedback provided to unsuccessful applicants. To do this, GAO analyzed HRSA data, grant applications, and the written feedback provided to unsuccessful grant applicants and interviewed HRSA officials. GAO found that grant awards for new health center sites in 2007 reduced the overall percentage of MUAs lacking a health center site from 47% in 2006 to 43% in 2007.  GAO also found that HRSA’s oversight of training and TA cooperative agreements is limited in key respects and does not always provide clear feedback to unsuccessful grant applicants.  The limitations include lack of standardized performance measures and lack of comprehensive on-site reviews.  Further more than a third of the written feedback HRSA sent to unsuccessful Health Center Program grant applicants in fiscal years 2005 and 2007 contained unclear statements. The GAO recommended that HRSA undertake the  following:  collect data on the types of services provided at each health center site, develop and implement standardized performance measures for training and TA cooperative recipients, reevaluate its policy of requiring comprehensive on-site reviews of training and TA cooperative agreement recipients every 3-5 years, ensure that the discussion of an applicant’s strengths and weaknesses in all summary statements is clear.

Health Professions Training Program

Impact of Title VII Training Programs on Community Health Center Staffing and National Health Service Corps Participation. Annals of Family Medicine, Vol. 6, No.5, September/October 2008.

The study examined the association between physician’s attendance in training programs funded by Health Resources and Services Administration (HRSA) Title VII Section 747 Primary Care Training Grants and two outcome variables:  work in a Community Health Center (CHC) and participation in the National health Service Corps Loan Repayment Program (NHSC LRP).  The study linked the 2004 American Medical Association Physician Masterfile to HRSA Title VII grants files, Medicare claims data, and data from the NHSC.  The study conducted retrospective analyses to compare the proportions of physicians working in CHCs among physicians who either had or had not attended Title VII-supported medical schools or residency programs and to determine the association between having attended Title VII-funded residency programs and subsequent NYSC LRP participation.  The study found that 3 percent of physicians who had attended Title VII-funded medical schools worked in CHCs in 2001-2003, compared with 1.9% of physicians who attended medical schools without Title VII funding (P<.001).  The study found a similar association between Title VII funding during residency and subsequent work in CHCs.  These associations remained significant (P<.001) in logistic regression models controlling for NHSC participation, public vs. private medical school, residency completion date, and physician sex. A strong association was also found between attending Title VII-funded residency programs and participation in the NHSC LRP, controlling for year completed training, physician sex, and private vs. public medical school.  The study concluded that continued federal support of Title VII training grant programs is consistent with federal efforts to increase participation in NHSC and improve access to quality health care for underserved population through expanded CHC capacity.

Maternal and Child Health Block Grant
Meeting State MCH Needs:  A Summary of State Priorities and Performance Measures, January 31, 2008, Prepared by the Cecil G. Sheps Center                             

As part of their collaborative relationship with HRSA’s Maternal and Child Health Bureau, State and jurisdictional Maternal and Child Health grantees (also referred to as States or State block grantees) participate in extensive planning and evaluation processes.  Beginning with a comprehensive needs assessment conducted every five years, States evaluate the needs of their MCH population, assess State resources, identify priority needs, and specify how they will measure success in meeting these needs.  The Child Health Program of the Cecil G. Sheps Center for Health Services Research at the University of North Carolina at Chapel Hill, under contract with HRSA’s Maternal and Child Bureau, reviewed changes in State and jurisdictional priority needs and how performance relative to these needs is measured.  A final report was prepared for MCHB to provide a broad overview of the areas identified by States as important for their constituents, to anticipate to some degree what priorities might arise in the next 5-year State planning cycle, and to determine what data States are using and what data they may need.  The report is divided into three sections that correspond to the three aspects of the 2005 Needs Assessment and priority setting process that were examined:  1) comparison of priority needs identified in 2005 with those identified in 2000, 2) other needs identified by States in the 2005 Needs Assessment process, and 3) review of State performance measures and how they link to State priority needs.       

Based on the findings of the 5-year Needs Assessment, each State is required to specify at least seven but no more than ten priority needs for their population.  Under the comparison of priority needs section, a key finding is the significant increase in State priorities to address specific health and health care issues, such as obesity and overweight.  In the section addressing the other needs identified by States in the needs assessment process but not included in the listed priority needs, the report identified as a key finding two main categories of needs-  those that are relatively new (ex. health insurance) and those that are longstanding (ex. oral health).  Each State MCH agency is required to develop seven to ten State Performance Measures to address their identified priority needs to the extent that they have not been addressed by the 18 National Performance Measures.  For the third area of review, the report concludes that the State performance measures, in the most general sense, reflect the priority needs identified by States.                    

State Maternal and Child Health Early Childhood Comprehensive Systems Grant Program (ECCS) - Two Year Evaluation Final Report, February 2008, Prepared by The Lewin Group.   
This report is the second in a series of two reports as part of a two-year national evaluation of MCHB’s ECCS grant program.  The purpose of the evaluation is two-fold:  1) to assess the progress and effectiveness of the ECCS initiative in meeting its goal of fostering early childhood systems development at the state-level and 2) to assess the quality, effectiveness, and timeliness of the technical assistance provided to the grantees.  Three primary data sources were used for this assessment:  1) document review of grantee applications and progress reports, 2) a minimum data set survey, and 3) telephone monitoring calls. 

Findings of the report include the usefulness of identifying an ECCS champion to support the project.  States have found that a high-level champion, often a politically connected person like a governor’s wife, can create a good deal of visibility and political support. Other findings include the crucial role of the ECCS project as a facilitator among systems building partners to provide support in such activities as convening meetings and focusing the efforts of partners.   The report also states that ECCS projects need to be flexible in their approach to be able to react to changing conditions and mentions the importance of setting realistic goals that can be accomplished. The importance of frequent communication among partners and of parental participation was also referenced.     

The report contains one recommendation regarding the ECCS project as a whole-   that the individual strategies used by the grantees be studied more in-depth to discern the reasons why some strategies were more successful than others.  The report notes several recommendations provided by the grantees regarding the provision of technical assistance (TA) including the need for a list to be provided to grantees of currently available TA services, the process of requesting TA should be clearly explained to grantees, and the TA resources available to grantees should include an orientation composed of trainings and/or other materials for new ECCS directors.  

The Assessment of the Title V Block Grant Program’s and the Discretionary Grants Programs’ Performance Measures,  May 2008, Prepared by Mathematica Policy Research

Mathematica Policy Research, Inc was awarded a contract to evaluate the utility and comprehensiveness of the National Performance Measures (NPM) used by the Maternal and Child Health Bureau (MCHB) in monitoring its supported programs.  NPM have become a vital method of monitoring program accomplishments in MCHB, and this evidence is used in the budgeting and resource allocation process.  Therefore, the evaluation of these measures will enable MCHB to continue to address current and emerging issues related to Maternal and Child Health (MCH) and to have objectives that are responsive to the leadership, technical assistance and training needs of MCHB grantees.  In addition, this information will better inform not only resource allocation but policy and management decisions relative to MCHB programs.

The Assessment of the Title V Block Grant Program’s and the Discretionary Grants Programs’ Performance Measures consisted of two phases.  In Phase 1, specific goals and objectives were developed in addition to the initial evaluation design, methodology, construction of instruments, and techniques of data analysis.  This phase also included a detailed review of the relevant literature concerning performance measures and their use.  In Phase 2, the design of the evaluation was refined based on the results of the literature review carried out in Phase 1. 
The final design included a plan for data collection, analysis and reporting.  The plan entailed the examination of all MCHB-sponsored programs to identify data sources reported for each measure, data compatibility and recommendations for future reporting.

Upon completion of the evaluation, Mathematica produced the following four reports:

  • Analysis of the review notes about Data Sources for Title V Block Grant Performance Measures in the Title V Information System (TVIS);
  • Analysis of the review notes on data sources, definitions, sub-elements and instructions for selected Discretionary Grant Performance Measures in the Discretionary Grant Information System (DGIS);
  • Analysis of Form #13 (family participation) in the TVIS and the Discretionary Grant Performance Measure #6 and #7 (family participation) in DGIS, and comparison of reported data and notes from both the TVIS and DGIS forms; and
  • Analysis of the review notes on data sources for NPM #14 in the TVIS, concerning overweight and obesity (Percentage of children, ages 2 to 5 years, receiving Women, Infants, and Children (WIC) services with a Body Mass Index (BMI) at or above the 85th percentile).  

The findings and recommendations from these reports provided the Bureau with valuable insight and methods to improve the effectiveness of the MCH Block Grant and Discretionary Programs in areas such as data reporting, suggested data sources and systematic data collection.

Ryan White HIV AIDS Program

Review of the Puerto Rico Health Department's Compliance with the Ryan White CARE Act Payer-of-Last-Resort Requirement (OIG Audit A-02-02000)

For grant years 2002-2004, OIG (Audit A-02-06-02000) estimated that Puerto Rico claimed $24.3 million in unallowable Federal funds under Title II of the Ryan White Comprehensive AIDS Resources Emergency (CARE) Act of 1990. These claims did not comply with the payer-of-last-resort requirement that Title II funds not be used to pay for HIV/AIDS drugs or services that are eligible for coverage by other Federal, State, or private health insurance.  Of the 100 prescriptions that were sampled in the Audit, 57 prescriptions were incorrectly claimed for patients who had other Federal, State, or private health insurance that would have covered the HIV/AIDS drugs.  The 57 prescriptions included 50 prescriptions for patients covered by Medicaid and seven prescriptions for patients covered by four private health insurance plans.  Based on the sample results, OIG estimated that the Health Department claimed unallowable Federal funding totaling $24,340,789.  This overpayment occurred because the Health Department had not developed procedures to bill HIV/AIDS drugs to the insurance plans with primary payment responsibility.  The OIG recommended that Puerto Rico refund $24.3 million and develop procedures to bill HIV/AIDS drugs to the Federal, State, or private health insurance plans with primary payment responsibility.