ExpectMore.gov


Detailed Information on the
Healthy Community Access Program Assessment

Program Code 10003515
Program Title Healthy Community Access Program
Department Name Dept of Health & Human Service
Agency/Bureau Name Health Resources and Services Administration
Program Type(s) Competitive Grant Program
Assessment Year 2005
Assessment Rating Ineffective
Assessment Section Scores
Section Score
Program Purpose & Design 20%
Strategic Planning 62%
Program Management 70%
Program Results/Accountability 20%
Program Funding Level
(in millions)
FY2007 $0
FY2008 $0
FY2009 $0

Ongoing Program Improvement Plans

Year Began Improvement Plan Status Comments
 

 

Completed Program Improvement Plans

Year Began Improvement Plan Status Comments
2006

The program sunset in FY 2006

Completed

Program Performance Measures

Term Type  
Annual Efficiency

Measure: Increase the amount saved per average number of Shared Electronic Health Records created per HCAP grantee among those grantees operational in this activity by 3% each year.


Explanation:The program sunset in fiscal year 2006. The program is no longer reporting on targets and actuals.

Annual Output

Measure: Increase the average additional number of patients completing specialty care referrals in HCAP communities operational in the activity by 300 per year. (Developmental Measure)


Explanation:The program sunset in fiscal year 2006. The program is no longer reporting on targets and actuals.

Annual Output

Measure: Increase the average number of patients receiving free and/or reduced cost pharmaceuticals in HCAP grantee communities operational in these activities by 500 per year.


Explanation:The program sunset in fiscal year 2006. The program is no longer reporting on targets and actuals.

Annual Output

Measure: Increase the average number of additional individuals assigned to a Medical Home in HCAP grantee communities operational in the activity by 400 per year.


Explanation:The program sunset in fiscal year 2006. The program is no longer reporting on targets and actuals.

Long-term Outcome

Measure: Increase the difference between the rate of preventable/avoidable emergency department (ED) visits (per 1,000) for individuals in HCAP community hospitals, as compared to all other communities in States with HCAP communities.


Explanation:The program sunset in fiscal year 2006. The program is no longer reporting on targets and actuals.

Long-term Outcome

Measure: Increase the difference between the rate of Hospitalizations for Ambulatory Care Sensitive Conditions (ACSC) (per 1,000) for individuals in HCAP community hospitals, as compared to all other hospitals in States with HCAP communities.


Explanation:The program sunset in fiscal year 2006. The program is no longer reporting on targets and actuals.

Annual Output

Measure: Increase the proportion of health and human service HCAP consortium members using Electronic Integrated Eligibility Program Screening & Enrollment Systems among grantees operational in implementing such systems. (Developmental Measure)


Explanation:The program sunset in fiscal year 2006. The program is no longer reporting on targets and actuals.

Questions/Answers (Detailed Assessment)

Section 1 - Program Purpose & Design
Number Question Answer Score
1.1

Is the program purpose clear?

Explanation: The Healthy Community Access Program (HCAP) seeks to improve access to health services for low-income individuals. The program's authorization and grant guidance broadly state that grantees should expand access to care for individuals without insurance or chronic conditions through better integration of health services within communities. Communities can spend programs funds on a wide range of activities, including buying computers, developing disease management protocol, and developing referral arrangements. Thus, the mechanism that the program uses to achieve greater access to health care varies across grantees. Grantees stated goals and objectives in grant applications also vary.

Evidence: 1. The program provides grants to local consortia of public health departments, health centers, hospitals with a low-income utilization rate of 25% or more, and health care providers that have traditionally served the medically underserved. The average new HCAP initial grant award is $900,000, with decreasing Federal support in Years 2 and 3. By statute, only 15% of program funds can be spent on direct patient care. 2. Health Care Safety Act Amendments of 2002 (Public Law 107-251) 3. PIN 2004-12 (HRSA 04-038) Funding Opportunities for New and Fourth Year Competing Continuation Healthy Communities Access Program (HCAP) Grant Applicants

NO 0%
1.2

Does the program address a specific and existing problem, interest, or need?

Explanation: In 2003, there were an estimated 36 to 45 million Americans without health insurance. Responsibility for paying for and providing care to uninsured persons is fragmented and ill-defined. The safety net is a patchwork of public and private institutions, clinics, physician's offices, and hospitals. The uninsured are three times more likely to report problems getting medical care, even for serious conditions and more likely to be hospitalized or use the ER for health problems that have worsened due to delayed care. The number of uninsured affects communities as a whole and can reduce increase the cost of health care. HCAP grants seek to address these problems by providing resources to communities to increase their capacity to care for the uninsured.

Evidence: 1. DeNavas-Walt, C., Proctor B., and Mills R., (August 2004) Income, Poverty, and Health Insurance Coverage in the United States: 2003, Current Population Reports U.S. Census Bureau: P60-226 U.S. Government Printing Office, Washington, DC 2. Two recent studies conclude that the CPS overestimated the number of uninsured because it undercounted the number of residents enrolled in Medicaid programs. The Actuarial Research Corporation estimated that nine million, or 20%, of the CPS 45 million uninsured had health insurance coverage in 2003. The Urban Institute, based on data from 2001, estimates that four million of the 45 million uninsured identified by CPS had coverage in 2003. 3. Kaiser Commission on Medicaid and the Uninsured. November 2004. The Uninsured: A Primer. Key Facts About Americans Without Health Insurance. 3. Lewin ME, Altman S., Editors.(2000) America's Health Care Safety Net: Intact but Endangered. Institute of Medicine National Academy Press, Washington D.C. 4. Institute of Medicine Board on Health Care Services (2003) A Shared Destiny: Community Effects of Uninsurance National Academy Press, Washington D.C.

YES 20%
1.3

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: There are a multitude of federal programs that seek to address the problems that uninsured individuals face through providing access to health insurance or affordable health care. Health Centers deliver high-quality, affordable, primary and preventative care to nearly 14 million patients, regardless of ability to pay at 3,740 sites annually. In 2006, Health Centers will serve an estimated 16 percent of the Nation's population at or below the 200 percent of the Federal Poverty Line. Medicare will provide an estimated $396 billion in FY 2006 and Medicaid will provide an estimated $X billion in FY 2006 to enhance access to health insurance. Health Savings Accounts and other tax incentives have also expanded access insurance. When the program was administered as the Community Access Program (CAP), program materials indicated that the program shared some of the same goals of the W.K. Kellogg Foundation's Community Voices Program and the Robert Wood Johnson Foundation's Communities in Charge Program. The W.K. Kellogg Foundation's Community Voices Program has provided 41 grants since 1998 to improve safety net provider networks. The Communities in Charge Program has provided 34 communities to develop integrated health care delivery systems for the uninsured.

Evidence: 1. HRSA Community Access Program, Notice of availability of funds. (bphc.hrsa.gov/cap/CAPfrn.htm) 2. WK Kellogg Foundation (www.wkkf.org/Programming/Grants.aspx?CID=1&CatID=7&glse=ProjectStartDate+DESC&glpi=1) 3. Robert Wood Johnson Foundation's Communities in Charge (www.communitiesincharge.org/) 4. FY 2006 HHS President's Budget

NO 0%
1.4

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The program is not designed to achieve a lasting impact in the targeted communities. The program provides a three to four year Federal investment to fund up-front capital costs of infrastructure integration and coordination. Grantees are expected to maintain the activities funded by the Federal grant after the 3-year grant period expires, but there is no mechanism to ensure that the activities are continued. Other programs fund or subsidize access to care for low-income individuals over a sustained period of time, including Health Centers, Medicaid, SCHIP, and tax incentives to encourage individuals to purchase insurance.

Evidence: 1. Health Care Safety Act Amendments of 2002 2. PIN 2004-12 (HRSA 04-038) Funding Opportunities for New and Fourth Year Competing Continuation Healthy Communities Access Program (HCAP) Grant Applicants

NO 0%
1.5

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: It is unclear what amount of the federal investment is reaching uninsured and underserved individuals and ultimately expanding access to health care. Grantees are given wide discretion to allocate funds in their communities and funds can be used for wide variety of activities. The program cannot ensure that the most useful activities are being funded. The program's grant application guidance provides 19 different examples of activities that HCAP funds can be used for, but states that it is not an exhaustive list. The outcomes of many of the activities has not been established and it is likely the program is subsidizing some activities are ultimately of limited use to the community.

Evidence: 1. Health Care Safety Act Amendments of 2002 2. PIN 2004-12 (HRSA 04-038) Funding Opportunities for New and Fourth Year Competing Continuation Healthy Communities Access Program (HCAP) Grant Applicants

NO 0%
Section 1 - Program Purpose & Design Score 20%
Section 2 - Strategic Planning
Number Question Answer Score
2.1

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The program's first Long-Term Measure is: Increase the difference between the rate of Hospitalizations for Ambulatory Care Sensitive Conditions (ACSC) (per 1,000) for individuals in HCAP community hospitals, as compared to all other hospitals in States with HCAP communities. The program will report on the rates for 0-17 year olds and 40-64 year olds. This measure captures all HCAP/CAP grantees communities since the inception of the program. The program's second Long-Term Measure is: Increase the difference between the rate of preventable/avoidable emergency department (ED) visits (per 1,000) for individuals in HCAP community hospitals, as compared to all other communities in States with HCAP communities. This measure captures all HCAP/CAP grantees communities since the inception of the program.

Evidence: Long-term Measure #1: Ambulatory Care Sensitive Conditions identify hospital admissions that evidence suggests could have been avoided, at least in part, through high-quality primary care. Many researchers have used hospital data to assess access problems and to identify areas within a community where there may be more substantial barriers to timely and effective ambulatory care (AHRQ Publication No. 03-0027, September 2003). The analysis includes all hospitals in HCAP service area zip codes regardless of membership status in consortia. The rate will be adjusted to take into account the income of the area. The rates will be adjusted using regression analysis in which the ratio of observed to predicted income is multiplied by the mean rate for the state. Long-Term Measure #2: Reliance on emergency departments (EDs) as a regular source of care is often used as an indicator of poor access and compromised quality of care and EDs have also been documented as being used disproportionately by patients without insurance, patients with Medicaid, patients without primary care physicians, members of racial and ethnic minorities, and other "vulnerable populations". Preventable/Avoidable conditions are those classified as: 1) Non-emergent: Cases where immediate care is not required within 12 hours (e.g., sore throat). 2) Emergent-primary care treatable: Care is needed within 12 hours, but care could be provided in a typical primary care setting (e.g., infant with a 102?? fever). 3) Emergent-ED care needed: preventable/avoidable: Immediate care in an ED setting is needed, but condition could potentially have been prevented or avoided with timely and effective ambulatory care (e.g., asthma, diabetic ketoacidosis).

YES 12%
2.2

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The program does not yet have baselines for its two long-term measures. The program has information from 2001 and 2003 on several grantees and states. For Long-Term Measure #1, the program has information on 22 grantees in 5 states. For Long-Term Measure #2, the program has information on 12 grantees in 5 states. These grantees and states were selected by the National Opionion Research Center (NORC) and were choosen to give mix of urban and rural areas and old and new grantees. NORC is conducting an evaluation of the program as part of an HCAP report to Congress. The program has had 228 grantees over the life of the program in 45 states, the District of Columbia and the US Virgin Islands. The sample size is not sufficiently large and the sampling methodology is not sufficiently random to ensure a reliable baseline. The program will need to collect information on more, if not all, HCAP grantee communities and states before reliable baselines can be set. The program has established targets and timeframes for its two long-term measures. However, targets may need to be adjusted once baselines are established. The benchmark group for both measures is the non-HCAP communities in States that have an HCAP community. The targets for Long-Term Measure #1 are broken out by age group. For children ages 0-17, the target is to have an ACSC admission rate that is 9% lower than the benchmark group by 2012. In 2003, for the 22 grantees the program had information on, the HCAP communities had an ACSC admission rate that was 7% lower than the benchmark group. The long-term target is 25% above the 2003 information on 22 grantees. For adults ages 40-64, the target is to have an ACSC admission rate that is 22% lower than the benchmark group by 2012. In 2003, for the 22 grantees the program had information on, the HCAP communities had an ACSC admission rate that was 18% lower than the benchmark group. The long-term target is 25% above the 2003 information on 22 grantees. The target for Long-Term Measure #2 is to have a preventable/avoidable ED visit rate that is 21% lower than the benchmark group by 2012. In 2003, for the 9 grantees the program had information on, the HCAP communities had a preventable/avoidable ED visit rate that was 17% lower than the benchmark group. The long-term target is 25% above the 2003 information on 9 grantees.

Evidence: In the future, the program will be much less limited in the number of states and communities they obtain data from. The program plans to develop an interagency agreement with AHRQ to have access directly to their HCUP data set to obtain ACSC and ED data from a much larger number of states and grantees Currently, HCUP has data on ACSC hospital discharge data for 36 states and hospital ED data for 13 states. Long-Term Measure #1 is broken out by age group: children 0-17 and adults 40-64. Long-Term Measure #1, Children 0-17: In 2001, the 22 grantees had an ACSC admission rate of 8.4 per 1,000. That same year, in the benchmark 5 states, the non-HCAP communities had an ACSC admission rate of 8.9 per 1,000. The HCAP communities had, on average a rate that was 6.5% lower than the benchmark communities. In 2003, the 22 grantees had an ACSC admission rate of 17.6 per 1,000. That same year, in the benchmark 5 states, the non-HCAP communities had an ACSC admission rate of 21.3 per 1,000. The HCAP communities had, on average a rate that was 17.6% lower than the benchmark communities. Long-Term Measure #1, Adults ages 40-64: In 2001, the 22 grantees had an ACSC admission rate of 16.2 per 1,000. That same year, in the benchmark 5 states, the non-HCAP communities had an ACSC admission rate of 19.7 per 1,000. The HCAP communities had, on average a rate that was 17.4% lower than the benchmark communities. In 2003, the 22 grantees had an ACSC admission rate of 8.2 per 1,000. That same year, in the benchmark 5 states, the non-HCAP communities had an ACSC admission rate of 8.8 per 1,000. The HCAP communities had, on average a rate that was 7.1% better than the benchmark communities. Long-Term Measure #2: In 2001, the 9 grantees had a preventable/avoidable ED visit rate of 604 per 1,000. That same year, in the benchmark 4 states, the non-HCAP communities had a preventable/avoidable ED visit rate of 643 per 1,000. The HCAP communities had, on average a rate that was 6% lower than the benchmark communities. In 2003, the 9 grantees had a preventable/avoidable ED visit rate of 658 per 1,000. That same year, in the benchmark 4 states, the non-HCAP communities had a preventable/avoidable ED visit rate of 788 per 1,000. The HCAP communities had, on average a rate that was 17% lower than the benchmark communities.

NO 0%
2.3

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: Annual Measure #1: Increase the average number of additional individuals assigned to a Medical Home in HCAP grantee communities operational in the activity by 400 per year. Annual Measure #2: Increase the average number of patients receiving free and/or reduced cost pharmaceuticals in HCAP grantee communities operational in these activities by 500 per year. Annual Measure #3: Increase the average additional number of patients completing specialty care referrals in HCAP communities operational in the activity by 300 per year. (Developmental Measure) Annual Measure #4: Increase the proportion of health and human service HCAP consortium members using Electronic Integrated Eligibility Program Screening & Enrollment Systems among grantees operational in implementing such systems. (Developmental Measure)

Evidence: The annual measures track only those current grantees that are engaging in the activity and report on the activity in the 6 Month Monitoring Report. The program explicitly choose note to look at the total number of medical homes, reduced cost/free pharmaceuticals, or referrals. Instead, an average per grantee was used because it is anticipated that the number of grantees in the future will change and/or grantee interest in engaging in the activity may change. The program has only limited control over what activities grantees choose to engage in. In 2002, the program had 158 current grantees and in 2003 the program had 193 grantees. Annual Measure #1 looks at the number of NEW patients per grantee each year since assigning to medical homes is not an activity usually repeated. Annual Measures #2 looks at new and existing patients because some existing patients may stay and some may drop off, depending on the individual's eligibility for insurance coverage. There is double counting in this measure because the same patient may be counted twice because they received some free and some reduced cost Rx. Annual Measure #3 looks new and existing patients because some existing patients may stay and some may drop off, depending on the individual's eligibility for insurance coverage. Annual Measure #3 is a developmental measure. These data have only recently been included in the collected in 6 Month Monitoring Reports. The program has collected information on the number of individuals receiving specialty care referrals. This developmental measure is a stronger measure because it looks at individuals actually receiving care from a specialist. A patient that receives a referral could encounter difficulty receiving the visit for a variety of reasons, including the specialist declining to see them, lack of transportation, and lack of funds. Annual measure #4 is a developmental measure as these data are only now being collected in the revised version of the 6 Month Monitoring Reports

YES 12%
2.4

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The program has targets for three of its annual measures and baselines for two of its annual measures. The program has baselines and targets for Annual Measures #1 and #2. The program has not yet developed a baseline for Annual Measure #3. Current data for this measure are only available for 35 to 70 grantees that are in the early stages of implementing the activity. The program has historical information on a similar measure which allowed it the program to set annual targets. The program has collected information on the average number of individuals receiving specialty care referrals and Annual Measure #3 looks at the average number of individuals completing specialty care referrals. Annual Measure #4 does not yet have a baseline or targets. Current data for this measure are only available for 35 to 70 grantees that are in the early stages of implementing the activity.

Evidence: Annual Measure #1: In 2002, the program had 87 grantees engaging in assigning individuals to Medical Homes. These grantees, on average, placed 3,964 individuals in Medical Homes. In 2003, the baseline year, the program had 90 grantees engaging in assigning individuals to Medical Homes. These grantees, on average, placed 4,271 individuals in Medical Homes. From 2002 to 2003, the average per grantee increased by 307, or 7.7%. The program's target is +400 individuals into Medical Homes each year, which is above their historical performance. Annual Measure #2: In 2002, the program had 45 grantees engaging increasing access to free and/ore reduced cost pharmaceuticals. These grantees, on average, provided 1,748 individuals access to free pharmaceuticals and 1,726 individuals access to reduced cost pharmaceuticals. (Note: The total number of individuals receiving free/reduced cost drugs, 3,476, is a duplicated count as some individuals receive both.) In 2003, the program had 55 grantees engaging increasing access to free and/ore reduced cost pharmaceuticals. These grantees, on average, provided 2,452 individuals access to free pharmaceuticals and 2,748 individuals access to reduced cost pharmaceuticals. (Note: The total number of individuals receiving free/reduced cost drugs, 5,200, is a duplicated count as some individuals receive both.) From 2002 to 2003, the average number individuals receiving free/reduced cost pharmaceuticals increased by 1,724, or 50%. The program's annual target is +500 average number individuals receiving free/reduced cost pharmaceuticals. The program believes that the 2002 to 2003 improvement was not indicative of a level that could be sustained in the future. This is because the grantees in reporting in 2002 were relatively new and the grantees in 2003 were, on average older. While only 15% of grantees overall were in their 3rd year of funding in 2002, nearly 59% of grantees overall were in their 3rd year of funding in 2003 due to the large cohort of 113 grantees initially funded in FY 2001. An increase of +500 per year is ambitious and will achieve approximately a 10% improvement each year. Annual Measure #3: The program does not have performance information for this developmental measure. However, the program has information on a similar measure, the average number of patients receiving specialty care referrals. This measure was the basis for the Annual Measure #3 targets. Historical information on the similar measure is as follows: In 2002, the program has 66 grantees engaging in providing specialty care referrals. These grantees, on average, gave referrals to 2,420 individuals. In 2003, the program had 77 grantees engaging in providing specialty care referrals. These grantees, on average, provided 3,053 referrals. From 2002 to 2003, the average number of individual receiving a specialty care referrals increased by 633, or 42%. The program's annual target is +300 average number of individuals completing speiclaity care referrals. The program believes that the 2002 to 2003 improvement was not indicative of a level that could be sustained in the future. This is because the grantees in reporting in 2002 were relatively new and the grantees in 2003 were, on average older. In addition, the program does not have actual historical information on the measure itself, only a similar measure. It is possible that the target will be well above the program's historical annual improvement. An increase of +300 per year is ambitious and will achieve approximately a 10% improvement each year. Annual Measure #4: The program does not yet have performance information to base targets off of. The program expects to have data available in late September and a baseline and target developed by the end of October.

YES 12%
2.5

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: The programs measures were developed for PART and have not yet been incorporated into program application guidance. In addition, only those grantees that engage in the activities included in the annual measures will report on those activities. The program does not have tools to require grantees to engage in these activities and thus commit towards the annual goals. In 2003, the program funded 158 grantees. Sixty percent of grantees reported on Annual Measure #1, 35 percent reported on Annual Measure #2, and 49 percent report on a measure that is similar to Annual Measure #3. Annual Measure #4 is developmental and actual performance information is not available. The program does have existing mechanism to evaluate and track grantees commitment towards the long-term and annual goals. Every six months, grantees must report their progress towards achieving their individual goals. Grantees must submit an annual non-competing continuation application, which reports on the progress to date, and plans for the next budget period.

Evidence: PIN 2004-12 "Funding Opportunities for New and Fourth Year Competing Continuation Healthy Communities Access Program (HCAP) Grant Applicants" The program plans to include the measures in any future program guidance. PIN 2004-16 "Application Guidance for Non-Competing Continuation Grant Applications for the Healthy Communities Access Program"

NO 0%
2.6

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: Since 2000, the program has allocated over $2.2 million to program evaluation activities. In FY 2004, HRSA contracted with the National Opinion Research Center (NORC) for a program evaluation. This evaluation is underway and is not yet complete. The evaluation will describe the extent to which HCAP projects have been successful in reaching program goals. The evaluation also will assess achievement of key program outputs and outcomes. To date, the program has been assessed twice by external evaluators, but these evaluations do meet each of the standards of quality, independence, scope, and regularity. The evaluations do not look at overall program impact and effectiveness and are focused on program implementation and describe grantee activities. In addition, the evaluations are not performed on a regular or semi-regular basis, but instead appear to be one-time assessments. In FY 2001, HRSA contracted with the Center for Health and Public Service Research (CHPSR) at New York University and Rutgers Center for State Health Policy (CSHP). This report focused on program implementation only. The report did not provide information on the effectiveness of the program or examine the impact of the program. In FY 2002, HRSA contracted with Navigant Consulting, Inc.. This report provided a series of descriptive case studies of the Community Access Program (CAP). The study described the activities of grantees in the following areas: primary care expansion, disease management, and sustainability.

Evidence: 1) Center for Health and Public Service Research (CHPSR) at New York University and Rutgers Center for State Health Policy (CSHP): HRSA Community Access Program: Local Achievements and Lessons Learned 2) Navigant Consulting, Inc.: Practices to Expand Primary Care Services, Implement Disease Management Protocols, and Achieve Sustainability Among Community Access Program Grantees

YES 12%
2.7

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: The program did not have a funding request in the President's FY 2006 Budget. The program had a funding request in the FY 2005 Budget, but did not provide a presentation that makes clear the impact of funding, policy or legislative decisions on expected performance nor did it explain why a particular funding level/performance result is the most appropriate.

Evidence: HRSA FY 2006 Justification of Estimates for Appropriations Committees

NO 0%
2.8

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The program plans to develop reliable baselines for Long-Term Measure #1 and #2. In the future, the program will be much less limited in the number of states and communities they obtain data from. The program plans to develop an interagency agreement with AHRQ to have access directly to their HCUP data set to obtain ACSC and ED data from a much larger number of states and grantees. The program plans to develop a baseline for Annual Measure #3 by Fall 2005. The program plans to develop a baseline and targets for Annual Measure #4 by Fall 2005. The program plans to get commitment to work towards the program goals from the grantees. The program plans to include the new long-term and annual goals in any future grant guidance.

Evidence:  

YES 12%
Section 2 - Strategic Planning Score 62%
Section 3 - Program Management
Number Question Answer Score
3.1

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: The program regularly collects credible performance information from grantees. All grantees complete a Six-Month Monitoring Report (6MMR). This report collects information on the consortium members, their vision statement, progress towards goals, consortium development and collaboration activities The program uses the 6MMR to monitor the grantees and improve program performance. The program uses the report to inform program planning, such as determining the need for technical assistance. Project Officers use the 6MMR to monitor individual grantee progress toward goals. This report will also be used as a data source for the new annual measures. An outside contractor, John Snow International (JSI), validates the 6MMR data. The program has also worked with evaluation consultants to develop a standard set of definitions to accompany the 6 Month Reporting Tool to ensure standardized terminology of data elements reported upon. The program solicits feedback from grantees in a variety of settings. After each bi-weekly TA call, grantees evaluate the benefit and usefulness of the call and to suggest future topic areas. Grantees that receive TA or attend workshops complete an evaluation form that assesses the value of the assistance. The HCAP statute also allows for up to 3 percent of appropriated funds to be utilized for technical assistance and other program support activities. These contracts provide logistical and administrative support. The program collects evaluation forms from the participants of bi-weekly TA conference calls to evaluate the performance of the contractors. The data for the long-term measures come form the AHRQ HCOP database. The program plans to develop an interagency agreement with the agency in order to facilitate data collection for the long-term measures.

Evidence: 1. 6MMR definitions 2. PIN 2004-16 "Application Guidance for Non-Competing Continuation Grant Applications for the Healthy Communities Access Program

YES 10%
3.2

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: The long-term and annual goals are new and were developed for PART. The long-term and annual measures are not included on the program managers' annual performance appraisals. The program tracks a grantee only on the activities that the grantee includes in their stated objectives. The program does not control what activities are in the grantees work plan and cannot require them to engage in the actitives included in the annual measures. Thus, the program does not track many of their grantees on the annual measures. If a grantee does not fulfill the grant requirements, the program can recommend termination of funds.

Evidence: PIN 2004-12 (HRSA 04-038) Funding Opportunities for New and Fourth Year Competing Continuation Healthy Communities Access Program (HCAP) Grant Applicants

NO 0%
3.3

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: The program obligates funds in a timely manner on grant awards and supporting contracts. All competing and non-competing continuation grant funds are made available before the end of the fiscal year. Grantees undergo annual audits. A-133 audits are done outside audit firm. Grantees also report on planned and actual expenditures and provide a cash transaction report indicating the drawdown of funds and balances on a quarterly basis through the Financial Status Report (FSR). Project scopes are monitored for compliance with program regulations by project officers throughout the year. In addition, project officers monitor significant rebudgeting requests from grantees through a formal approval process with the Division of Grants Management and review all annual non-competing budget requests for appropriateness with approved grantee project scope. If it is determined that a grantee is inappropriately expending Federal dollars during the course of a grant project, there are several options available to the agency as provided by both grant regulation and in grant administrative policies. The agency can either request the grantee to return the funds in question or make administrative offsets against other requests for reimbursement, and designate the grantee "high-risk". Designating a grantee as high-risk permits the agency to impose special award conditions which permit the agency to more closely monitor grantee performance and expenditures under an award. HHS grants policies provide HRSA and other HHS components with a series of enforcement actions which are at the agency's disposal to help ensure compliance. These range in significance and severity. If the determination that a grantee was expending grant dollars for unallowable costs was the result of an A-133 audit conducted after the grant closeout, the agency can seek recovery of the funds in question by requesting repayment of the funds or through other means.

Evidence: 1. Copy of HCAP Notice of Grant Award 2. SF 424 Standard Budget Form 3. SF 269 Financial Status Report 4. Title 45 CFR Parts 74.14 and 92.12 5. HHS grants policies, Grants Policy Directive 3.07, Termination and Enforcement 6. Title 45 CFR Part 30, Claims Collection 7. 45 CFR Part 74, Subpart D- After-the-Award Requirements 8. 45 CFR Part 92, Subpart D- After-the-Grant Requirements.

YES 10%
3.4

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The program's efficiency measure is: Increase the amount saved per average number of Shared Electronic Health Records created per HCAP grantee among those grantees operational in this activity by 3% each year. In 2002 and 2003, 24 grantees were operational in Shared Electronic Health Records and an additional number of grantees were in the planning but not yet operational stage (and thus not reporting data) in this activity. Shared Electronic Health Records is a key element in developing a Health Information Technology (HIT) system. The benefits of HIT include fewer mistakes and less cost. The HHS Office of the National Cooridanter for Health Information Technology (ONCHIT) estimates that HIT can reduce health care costs by 20% per year. Based on the literature, the program has estimated that the average cost savings per medical record is $4. In 2003, the program estimates that the average grantee engaging in the activity created 22,599 Shared Electronic Health Records. The average savings per grantee is estimated at $90,397 in 2003. The program has a goal increase the savings from Shared Electronic Health Records per grantee by 3% per year by increasing the average number of Shared Electronic Health Records by 3% per year. At the program level, certain logistical and technological support functions have been outsourced to the private sector through a competitive process to increase efficiency and cost effectiveness of program execution. Through these contracts, the grantee's 6 month monitoring report was shifted to a web-based format, allowing grantees to complete their 6 month reports online and the data to be captured in a relational database that much more efficiently and effectively collects, stores and queries the program-wide data.

Evidence: The literature and HIT industry reports several types of savings from moving to electronic medical records. Providers reduce their costs because they doe have to buy chart materials, pay for copying costs, storage. Within the medical records industry it is estimated that the cost associated with maintaining a paper record average $3 per medical chart. (Source: www.misysemr.com/home.html) Provider also save money because they do not have to manually pull paper chart. The average cost of chart pull is approximately $5, accounting for the time and cost of medical records personnel to retrieve and then re-file a paper chart. After conversion to the electronic medical record system, chart pulls can be reduced by 600 charts per year, based on the experience at one Partners Health Care clinic. Transcription costs were reduced by 28% from partial elimination of dictation. (Source: Wang S.J., et al. A Cost-Benefit Analysis of Electronic Medical Records in Primary Care. American Journal of Medicine 2003;114:397- 403.) The program estimated the cost savings per record created to be $4.00 per record, the median of the per unit savings cited above.

YES 10%
3.5

Does the program collaborate and coordinate effectively with related programs?

Explanation: The authorizing legislation requires that grantees represent a consortium of health care providers, including a Health Center, a public health department, and a disproportionate share hospital. Thus, the grantees must coordinate with the relevant providers in their communities. The program is located in HRSA's Bureau of Primary Health Care (BPHC), in the Division of State and Community Assistance. BPHC also includes the Health Centers program and HCAP collaborated with other BPHC programs to hold an all-grantee meeting in 2005. The program is co-located in the Division of State and Community Assistance with the Health Center Controlled Network office. The program collaborates with HCAP and the Health Center Controlled Networks hold joint technical assistance calls, and share technical expertise through peer-to-peer technical assistance.

Evidence: 1. Health Care Safety Act Amendments of 2002 (Public Law 107-251) 2. Bureau of Primary Health Care organization chart

YES 10%
3.6

Does the program use strong financial management practices?

Explanation: In FY 2004, HHS OIG conducted an HHS financial statement audit. The audit reported that the Department had serious internal control weaknesses in its financial systems and processes for producing financial statements. OIG considered this weakness to be material. The audit recommended that HHS improve their reconciliations, financial analysis, and other key controls. The September 30, 2002 HRSA independent auditor's report found that the preparation and analysis of financial statements was manually intensive and consumed resources that could be spent on analysis and research of unusual accounting. The audit also found that HRSA's interagency grant funding agreement transactions were recorded manually and were inconsistent with other agencies' procedures. Finally, the audit found that HRSA had not developed a disaster recovery and security plan for its data centers.

Evidence: 1. HHS FY 2004 Performance and Accountability Report 2. HRSA's 2002 audit report

NO 0%
3.7

Has the program taken meaningful steps to address its management deficiencies?

Explanation: HHS' long-term strategic plan is to resolve the internal control weaknesses is to replace existing accounting systems and other financial systems within HHS with the Unified Financial Management System (UFMS). HHS plans to fully implement the UFMS Department-wide by 2007. HRSA developed a corrective action plan to address the reportable conditions identified in the September 30, 2002 independent auditor's report. For each aspect of the five reportable conditions, HRSA assigned an office responsibility. The plan also outlines milestones and target completion dates. When the new performance measures are in use, results from these measures will be available to public via the HRSA website.

Evidence: 1. HHS FY 2004 Performance and Accountability Report www.hhs.gov/of/reports/account/acct04/pdf/section4.pdf 2. HRSA Corrective Action Plan for FY 2002 Financial Statement Audits

YES 10%
3.CO1

Are grants awarded based on a clear competitive process that includes a qualified assessment of merit?

Explanation: All initial HCAP grants are distributed according to a competitive process, which includes an independent merit review and ranking of applications. Program applications for nationally announced competitive grant cycles are reviewed by an objective review committee. The committee reviews the project plan and budget based on criteria (out of 100 points) announced publicly in the application guidance. All applications meeting the application deadline are reviewed initially for eligibility and conformance. Those applications that are determined to be ineligible, incomplete, or non-responsive will be returned to the applicant without further consideration. Applicants competing for federal HCAP funds receive an objective and independent review performed by a committee of experts that have been screened to avoid conflicts of interest. The review committee is given clear criteria to evaluate applicants and make a funding recommendation. HRSA program officials are responsible for all final award selections. The Division of Independent Review (DIR) within the Office of the Administrator is responsible for the overall management of HRSA's objective review function. For HRSA, DIR plans, directs, and carries out the objective review (including peer review) of applications for discretionary grants and cooperative agreements. DIR ensures that the review process is fair, open, and competently performed. DIR is responsible for developing, implementing, and maintaining policies and procedures necessary to carry out this responsibility. As such, DIR, in consultation with program offices, is responsible for all aspects of objective review. Grantees are initially awarded funding for one year. Grantees can receive funding for an additional two years, contingent upon reasonable and demonstrated progress towards meeting the goals set forth in their approved application. Grantees that have experiences extraordinary circumstances that prevented them from completion the goals set forth in their application may receive an optional fourth-year extension.

Evidence: PIN 2004-12 (HRSA 04-038) Funding Opportunities for New and Fourth Year Competing Continuation Healthy Communities Access Program (HCAP) Grant Applicants

YES 10%
3.CO2

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: The program reviews data collected through the Six-Month Monitoring Report (6MMR) and annual continuation applications to evaluate progress on grantees activities. The program uses the 6MMR to monitor the grantees and improve program performance. An outside contractor, John Snow International (JSI), validates the 6MMR data. The program has also worked with evaluation consultants to develop a standard set of definitions to accompany the 6 Month Reporting Tool to ensure standardized terminology of data elements reported upon. The program also reviews Technical Assistance (TA) reports from consultants or grantees that have provided TA to other grantees. All HCAP grantees are assigned to a "Project Officer." The primary duty of HCAP Project Officers is to provide ongoing and daily monitoring and oversight of their grantees. HRSA's Office of Performance Review conducts site visits once every five years. The HCAP technical assistance contractors provide consulting services, most often on-site. Grantees usually receive multiple TA visits during their 3-year grant period. In addition, if necessary, the program allocates technical assistance resources via a contractor for diagnostic site visits to those grantees demonstrating limited or slow progress. The diagnostic site visits address critical management and progress issues. Based on these visits, contractors may recommend actions to project officers such as drawdown restrictions on grant funds, further technical assistance or action plans to remedy concerns. Through the Division of Grants Management, the program has access to financial oversight information through grantee fund draw-down reports available through HRSA's Payment Management System, annual A-133 audit findings, and annual Financial Status Reports (FSRs).

Evidence: 1. Office of Performance Review Report Protocol (www.hrsa.gov/performancereview/)

YES 10%
3.CO3

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: The program collects performance data on an annual basis through its Six-Month Monitoring Report (6MMR) that captures descriptive and output data from all grantees on their HCAP activities. However, the report is not made available to the public. No grantee information is readily available to the public. The program does not currently publish any performance information. The list of grantee is now on the webpage - you can search for a grantee on the BHPC website. Select data elements from the 6MMR are made available to HCAP grantees through the program's Data Management System (DMS). The DMS allows grantees to query the data and find out information on such areas as grantee activities, consortium members, and in-kind/other funding received by grantees. Grantees can use it to query each others data.

Evidence: HCAP website - http://bphc.hrsa.gov/cap/

NO 0%
Section 3 - Program Management Score 70%
Section 4 - Program Results/Accountability
Number Question Answer Score
4.1

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: The program does not yet have baselines for its two long-term measures. The program has information from 2001 and 2003 on several grantees and states. For long-term measure #1, the program has information on 22 grantees in 5 states. For long-term measure #2, the program has information on 12 grantees in 5 states. The program has had 228 grantee in 45 states, the District of Columbia and the US Virgin Islands. The sample size is not sufficiently large and the sampling methodology is not sufficiently random to ensure a reliable baseline and historical performance. The program will need to collect information on more, if not all, HCAP grantee communities and states before reliable baselines can be set.

Evidence: 1. Questions 2.1 and 2.2 2. Measures Tab

NO 0%
4.2

Does the program (including program partners) achieve its annual performance goals?

Explanation: The program has two years of data for Annual Measures #1 and #2. For Annual Measure #1, the average number of new Medical Homes per grantee increased by 307, or 7.7% from 2002 to 2003. This is below, but close to, their annual target established for the PART of +400 per year. For Annual Measure #2, the average number individuals receiving free/reduced cost pharmaceuticals per grantee increased by 1,724, or 50% from 2002 to 2003. This is well above their annual target established for the PART of +500 per year. As noted in Questions 2.1 and 2.2, the program does not yet have actual data on Annual Measure #3. The program does have data on a similar measure, the number average number of individuals receiving specialty care referrals. For this measure, from 2002 to 2003, the average number of individual receiving a specialty care referrals increased by 633, or 42%. This is a good proxy for the increase in the number of individuals completing referrals because the program expects that as the number of referrals given increases, the number of individual completing referrals will also increase. The 2002 to 2003 performance on the similar measure was above their annual target established for PART of +300 average number of individuals completing specialty care referrals The program does not yet have performance information or targets for Annual Measure #4.

Evidence: 1. Questions 2.3 and 2.4 2. Measures Tab

LARGE EXTENT 13%
4.3

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The program has one year of historical data for the HCAP efficiency measure. In 2003, the program estimates that the average grantee engaging in the activity created 22,599 Shared Electronic Health Records. The average savings per grantee is estimated at $90,397 in 2003. The program has a goal increase the savings from Shared Electronic Health Records per grantee by 3% per year by increasing the average number of Shared Electronic Health Records by 3% per year. It is unknown if the program achieved a 3% increase from 2002 to 2003 or in prior years.

Evidence: 1. Question 3.4 2. Measures Tab

SMALL EXTENT 7%
4.4

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: As of 2005, the government has invested over $525 million in the program without knowing what return has been received. There are other federal programs that provide access to health care that have demonstrated results. Health Centers deliver high-quality, affordable, primary and preventative care to nearly 14 million patients, regardless of ability to pay at 3,740 sites annually. In 2006, Health Centers will serve an estimated 16 percent of the Nation's population at or below the 200 percent of the Federal Poverty Line. Medicare serves an estimated X million elderly and disabled individuals and Medicaid serves an estimated X million low-income individuals. In 2006, Medicare and Medicaid will provide an estimated X billion in Disproportionate Share Hospital (DSH) payments to hospitals that serve a high percentage of Medicaid patients. In addition, the government offers a number of tax breaks to incentivize the purchase of health insurance. For example, low-income individuals' under age 65 who are not enrolled in public or employer-sponsored health plans are eligible for a refundable tax credit that can be paid in advance directly to the health plan. The credit can be used for traditional health insurance plan or a Health Savings Account (HSA) and it is estimated that it will provide access to health insurance for 15 million individuals. Small employers receive a tax credit of up to $500 per employee with family coverage and $200 per employee with individual coverage to encourage them to contribute to their employees HSAs.

Evidence:  

NO 0%
4.5

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: No independent evaluations of sufficient scope and quality have been conducted to date.

Evidence: 1) Center for Health and Public Service Research (CHPSR) at New York University and Rutgers Center for State Health Policy (CSHP): HRSA Community Access Program: Local Achievements and Lessons Learned 2) Navigant Consulting, Inc.: Practices to Expand Primary Care Services, Implement Disease Management Protocols, and Achieve Sustainability Among Community Access Program Grantees

NO 0%
Section 4 - Program Results/Accountability Score 20%


Last updated: 09062008.2005SPR