GPRA Goal 3: Foster translation and dissemination of new knowledge into practice by developing and providing information, products, and tools on outcomes, quality, access, cost, and use of care. (HCQO)
Strategy: Cycle of Research Phase 3: Translation and Dissemination
AHRQ is committed to ensuring that the knowledge gained through health care research is translated into measurable improvements in the American health system. AHRQ is focusing on closing the gap between what we know and what we do. Under the "Translating Research Into Practice" initiative, the Agency invests in demonstration projects, public (Federal, state, and local government) and private-sector partnerships, and targeted dissemination activities to develop and test implementation strategies in different settings in the health care system and demonstrate their applicability to widespread dissemination in other areas of the system.
Types of Indicators
Output and process.
Use of Results by AHRQ
The indicators regarding number of partnerships, attendees at User Liaison Program meetings, or hits on the AHRQ web site help the Agency determine whether what it produces is of use to major audience segments. The Agency evaluates the results of the GPRA plan indicators in combination with other information such as details about what products were released, feedback from attendees at programs, where the hits are on the web site, and feedback from customers to manage and improve its dissemination efforts.
Frequently, the results of research are not readily implemented in the health care system without an interim step such as the creation of a tool that facilitates use. A major focus for the Goal 3 indicators, therefore, is to look at the creation and use of tools. The indicators for the Agency's investment in training helps the Agency track its success in furthering the field of health services research by fostering new talent. The number of trainees funded are a reflection of Agency commitment and the success of the training programs in attracting successful candidates. This data can be used in combination with other information about individual trainees, their research projects, professional credentials of professors and mentors, etc. to assess the overall success of the program.
Data Issues
The AHRQ has implemented several computer-based reporting tools to monitor usage of Agency information systems and Web sites. Accurate statistics are recorded on the usage of the National Guideline Clearinghouse™, Publications Clearinghouse, and various other Agency Web sites and systems using commercially available reliable and accurate tools, e.g., WebTrends. These tools are used by many corporations and government agencies nationwide to monitor usage and have been certified by various information technology testing and review groups. Information on all grants, which can be word searched, is included in the Agency Management Information System (AMIS).
AHRQ tracks print media (newspapers, health care-related trade journals and newsletters, and consumer magazines) and on-line news services for stories about or involving the Agency through the contractor, Burrelle's, which is one of the Nation's largest and oldest news clipping services. The contractor, Video Monitoring Service, monitors TV and radio news reports on selected studies in major markets around the United States. The other statistics are maintained by Agency program staff during the normal monitoring of contracts and grants. Certain items, such as the release of a CONQUEST product, are documented on the AHRQ web site when ready so that consumers are aware of the availability. Other items such as the statistics on the User Liaison Program (ULP) are monitored through the management of support contracts, travel arrangements, and other records kept in administering the program. Anecdotal information is verified with the primary source before being used by the Agency in this report and for any other uses.
Return to Contents
GPRA Goal 3 - Fiscal Year 2000 Results
Objective 3.1: Promote distribution of AHRQ publications, products, and tools through intermediary organizations.
Indicator 1
Formation of a minimum of 5 partnerships to support dissemination of AHRQ products through intermediary organizations, such as health plans and professional organizations.
Results
In Fiscal Year 2000, AHRQ partnered with over 30 diverse public and private organizations, including Web-based groups, to disseminate evidence-based information. Below is a sample of these partnerships.
AHRQ Public/Private Partnerships
PPIP
The University of New England, Area Health Education Center in Biddeford, ME, will reprint and disseminate PPIP materials.
In addition, the following 2 organizations have partnered with AHRQ in marketing and/or disseminating PPIP materials: HCFA and AARP
Your Guide to Choosing Quality Health Care
The University of California - Human Resources/Benefits is reprinting Your Guide to Choosing Quality Health Care.
Now You Have a Diagnosis: What's Next?
The National Association for Home Care is reprinting and disseminating Now You Have a Diagnosis: What's Next?
AHRQ Smoking Cessation
The Pharmacy Council on Tobacco Dependence - FL is collaborating with AHRQ to disseminate tobacco cessation materials.
Web Partner Activities
Web MD has worked on collaborative projects with AHRQ to disseminate and market AHRQ materials.
-
NASHP
AHRQ in partnership with RWJ, Commonwealth Foundation and CHCF partnerships for NASHP to evaluate state error reporting systems. A final report on this project will be due in Spring 2001. A preliminary report is available now.
CAHPS®
Partnership with CHCF on the translation of CAHPS® instruments into other languages.
Objective 3.2: Maximize dissemination of information, tools, and products developed from research results for use in practice settings.
Indicator 1
Number of hits on the Web site (Baseline: Fiscal Year 1999 - 15.5 M hits 2.9 million per year in 1997, nearly triple the hits in 1996.)
Results
AHRQ promotes widespread distribution and implementation of its information and research products through a variety of dissemination methods: publication in professional journals; provider and consumer materials, media events and outreach; interviews and story placement with medical/trade press and organizations' newsletters; and articles in the popular press. AHRQ also employs public-private partnerships, direct mail, and the World Wide Web to distribute its information.
The AHRQ Web site also provides access to the summaries of reports issued from the Evidence-based Practice Centers. The full text of these reports can be obtained at the National Library of Medicine, accessible through the AHRQ Web site. The Agency continued to work with the National Library of Medicine to upload evidence reports, technology assessments, and preventive services materials for clinicians on the full-text retrieval system HSTAT.
The "healthfinder®" gateway site, developed and maintained by the Department of Health and Human Services, was prominently featured on the AHRQ site, and in turn provided 50,000 referrals to the Agency's online consumer health and patient information materials. Nearly 10,000 external Web sites link to the AHRQ Web site home page or content within the site, almost double that of the previous year.
The site was redesigned this year to address feedback from an online customer satisfaction survey. Usability testing has also been conducted with various constituencies to ensure the quality has been upgraded, the content is accessible, and the navigational approaches facilitate information retrieval for users. The site will further be refined based on the results of this effort.
A follow-up online evaluation occurred and the redesigned Web site was highly rated on content, presentation, and ease of use, and for the quality, quantity, and timeliness of information.
The Web site mailbox is used by constituency groups to both communicate with the Agency and ask the Agency for help on a variety of issues. In Fiscal Year 2000, there were 18.8 million Hits on the AHRQ Web site. This is a 21% increase over the 1999 baseline.
Indicator 2
Number of inquiries handled on Web site. (Baseline: Fiscal Year 1999 - 2950; Fiscal Year 1998 - 2500; Fiscal Year 1997 - 1300)
Results
AHRQ handled 3,500 electronic inquiries during Fiscal Year 2000 through it Web site mailbox, a 19% increase over the baseline of 3,000 established in Fiscal Year 1999. These inquiries included requests for Agency information products, funded research, consumer health issues and concerns, technical assistance, referrals to other resources, and requests to use AHRQ electronic content on other Web sites or in electronic or print products.
Indicator 3
Number of Uploaded documents. Baseline: Fiscal Year 1999 - 4000; Fiscal Year 1998 - 1450; Fiscal Year 1997 - 950.
Results
4,400 Documents were uploaded in Fiscal Year 2000 (10% increase over the baseline of 4000 established in Fiscal Year 1999). In addition to the information provided for the English-speaking public, the AHRQ Web site offers a "Spanish" button, Información en Español. A popular feature on the Web site which consistently is within the Top100 features accessed each month, it provides translations of our consumer health and patient information materials, and averages about 1,200 visits each month. Many users are the Spanish-speaking public, but we have also received feedback from clinicians with large Hispanic patient populations on the utility of these materials for their patient education efforts.
Indicator 4
Reports from user surveys on how the information requested was used.
Results
The AHRQ Web site was highly rated on content, presentation, and ease of use, and for the quality, quantity, and timeliness of information. Based on feedback from an online evaluation of the Web site, various audience groups came to the site for:
- New funding opportunities and subsequent award announcements.
- Press releases with contact information on key staff involved.
- Informative electronic newsletters on research activities.
- Clinical research resources to improve practice and health outcomes.
- Strategic planning to establish priorities and directions for State health programs.
- Facilitating health services research at the university level.
- Summaries of evidence-based information for medical practice.
- Research information related to treatment of specific health conditions, such as diabetes,
arthritis.
- Recommendations on quality of care issues and suggestions for improving health care.
- Learning more about AHRQ and related resources.
- Keeping abreast of new developments and up-to-date information on the changing health care
system.
- Best practices and information on reducing the cost of providing health insurance.
- Data and statistics on health care costs and use.
The following characterizes requests from outside organizations for use of electronic content from the AHRQ Web site:
- Consumer materials on specific conditions and also to better understand the health care system
have been incorporated on numerous consumer health Web sites as well as in corporate intranets for
employees, e.g. low back pain, quit smoking, and health insurance choices.
- Clinical materials have been adapted for medical Web sites oriented for clinicians as well as in
hospital and health plan systems.
- Clinical and research materials have been included in course packs for both medical education programs
and graduate training in public health issues.
- Information on reducing medical errors has been distributed by managed care organizations through
their member Web sites and newsletters.
- Preventive services information for both adults and children have been used for health education
initiatives of health plans and employers.
Indicator 5
Number of State and local governments trained in the understanding and use of health services research findings through User Liaison Program (ULP) Workshops.
- Number of ULP meetings held.
Baseline: 10 meetings held in Fiscal Year 1997; 9 held in Fiscal Year 1998; 13+ Fiscal Year 1999
- Number of attendees.
Baseline: 834 attendees in CY 1999; 538 attendees in CY 1997
- States represented.
Baseline: 48 States plus D.C.
Results
In Fiscal Year 2000, 17 ULP meetings were held at the national, state and local level (a 70% increase over the baseline of 10 meetings established in Fiscal Year 1999). These included:
- 13 national workshops
- 1 workshop for Tribal leaders interested in health care issues
- 1 State-specific seminar
- 2 3-hour audio-conferences with telephone call-in capability
The total number of attendees at ULP workshops, seminars and audio-conference calls (1196) increased 143% over the baseline of 834 attendees established in Fiscal Year 1999, and included:
- 635 State participants in all workshops and seminars
- 513 participants for audio-conference calls
- 48 Tribal workshops
State representation increased 4% to include participants from all the 50 States plus the District of
Columbia. Below is the breakdown of participation by State.
State Participant Breakdown
| State |
Number of
Participants |
|
Alaska |
4 |
|
Arizona |
17 |
|
Arkansas |
6 |
|
Bahamas |
0 |
|
Bermuda |
0 |
|
California |
35 |
|
Canada |
0 |
|
Colorado |
8 |
|
Connecticut |
12 |
|
Delaware |
3 |
|
District of Columbia |
5 |
|
Florida |
29 |
|
Georgia |
23 |
|
Hawaii |
9 |
| Idaho |
11 |
| Illinois |
13 |
| Indiana |
5 |
| Iowa |
8 |
| Kansas |
7 |
| Kentucky |
11 |
| Louisiana |
9 |
| Maine |
8 |
| Maryland |
13 |
| Massachusetts |
15 |
| Michigan |
15 |
| Minnesota |
15 |
| Mississippi |
5 |
| Missouri |
17 |
| Montana |
4 |
| Nebraska |
8 |
| Nevada |
6 |
| New Hampshire |
9 |
| New Jersey |
8 |
| New Mexico |
19 |
| New York |
23 |
| North Carolina |
12 |
| North Dakota |
1 |
| Ohio |
11 |
| Oklahoma |
23 |
| Oregon |
15 |
| Pennsylvania |
33 |
| Puerto Rico |
0 |
| Rhode Island |
9 |
| South Carolina |
6 |
| South Dakota |
2 |
| Tennessee |
6 |
| Texas |
15 |
| Utah |
15 |
| Vermont |
3 |
| Virgin Islands |
0 |
| Virginia |
9 |
| Washington |
18 |
| West Virginia |
3 |
| Wisconsin |
18 |
| Wyoming |
1 |
Indicator 6
Reports from annual participants on how the information was used in decisionmaking.
Results
Two examples of how participants used information learned at ULP meetings include the following:
- Legislation was introduced in Massachusetts during calendar year 2000 intended to reduce the
number of medical errors based on information a State senator learned while attending a User Liaison
Program (ULP) workshop in March 2000.
- Wyoming decided to implement evidence-based disease management and have been interviewing consultants
to help them as a result of a member of the Department of Health attending a ULP meeting in May 2000 on
"Using Evidence: Technology Assessment, Disease Management, and Coverage Decisions."
Indicator 7
Statistics on usage of National Guideline Clearinghouse™ including number of hits, requests, organizations, and total users. (Baseline: See Fiscal Year 1999 results for details.)
Results
The statistics on usage of the National Guideline Clearinghouse™ (NGC) illustrate the success AHRQ has achieved in Fiscal Year 2000 in disseminating information to users. In hits alone, there has been a 1037% increase over the baseline established in Fiscal Year 1999.
NGC Statistics
Number of requests: 21,663,521
Number of visits: 2,003,786
Average number of requests per visit: 10.81
Average visit duration: 00:05:03
Number of organizations: 68,190
Number of U.S. organizations: 15,999
Number of Canadian organizations: 248
Number of International organizations: 6,626
Number of unknown organizations: 45,307
Indicator 8
Survey of a sample of NGC users to understand the impact of use on decisions and patient care.
Results
The National Guideline Clearinghouse™ First Annual Customer Satisfaction Survey was completed in June 2000. To the survey question, "How has NGC changed your healthcare-related activities or practices?", 902 respondents indicated the impact of NGC use include informing practice patterns, use in educating healthcare staff, and as an aid to cutting health care costs and increasing quality of care. The following excerpts demonstrate the range of impact:
- Assists in my research and informs my clinical care.
- We are using it to make sure staff are competent in guideline use.
- Improved analysis of prescribing practice.
- Changed practice pattern based on guideline.
- Made it easier for me to use cutting edge information in a very rural environment.
- Changed management based on practice guidelines.
- More evidence-based practice, help cut health care costs but increase quality of care.
- We have used the guidelines to set up our pain management practices at the hospital where I work.
- Have updated several clinical protocols with information obtained.
- Many of our policies are based on clinical.
The average satisfaction score assigned by the 902 respondents was 1.7 on a scale of 1 to 5, with 1 being very satisfied and 5 being very dissatisfied. The great majority of survey respondents rated the NGC as either good or very good in terms of the comprehensiveness of the guideline collection; whether the guideline summaries present key attributes; the usefulness of the search, browse and index functions; the usefulness of the guideline summaries, synthesis, and tabular comparisons; and the reasonableness of the time required to locate specific information.
An evaluation of the NGC is underway to assess the impact of its use on decisions and patient care and expected to be completed in mid-year 2001.
Indicator 9
At least 10 purchasers/businesses use AHRQ findings to make decisions.
Results
The widespread use of AHRQ findings is providing purchasers with valuable information for making healthcare decisions. The following are examples of this use:
- More than 90 million Americans use AHRQ's Consumer Assessment of Health Plans (CAHPS) to help them
decide which health plan best meets their health care needs. CAHPS is now used by more than 20 States;
corporations such as Daimler Chrysler, Ford and General Motors; health plans; and employer groups across
the country.
- The Health Care Financing Administration (HCFA) has begun using CAHPS to survey Medicare managed care
plan enrollees, and the U.S. Office of Personnel Management used CAHPs to report consumer assessments of
health plans available to Federal workers and retirees for its Fiscal Year 2000 open season.
- The Health Care Financing Administration decided to cover cryosurgery as a primary treatment for
prostate cancer for Medicare patients, but to leave unchanged it's decision to not cover the operation as
a salvage therapy because of the findings of AHRQ's technology assessment, Cryosurgery for Recurrent
Prostate Cancer Following Radiation Therapy.
- Purchasing decisions of employers are often influenced by how well health plans score on NCQA's Health
Employer Data Set (HEDIS). Plans' scores will now reflect how well they screen for chlamydia, as rated by
a measure developed through AHRQ's Q-Span research initiative.
- The Leapfrog Group - created by Fortune 500 companies, including General Motors and General Electric,
that are committed to using a common set of health insurance purchasing principles—adapted
information from AHRQ's 20 Tips to Help Prevent Medical Errors for educating employees about
medical errors and patient safety.
- Federal agencies are also using 20 Tips to Help Prevent Medical Errors. The U.S. Office of
Personnel Management, which purchases health care for over nine million Federal workers and retirees, has
adapted materials from the publication for inclusion in Fiscal Year 2001 open season informational materials, and
the U.S. Department of Defense is making materials adapted from AHRQ's publication available to members of
the armed forces and their dependents.
- PEPCO of Washington, D.C. purchased 4,000 copies of the English-language edition of AHRQ's Put
Prevention into Practice guide, Staying Healthy at 50+, for distribution to the company's employees
and retired workers.
- AHRQ partnered with the American Association of Retired Persons, an advocacy group representing health
insurance and other interests of over 33 million Americans 50 and older, to partner with it and HRSA to
produce and disseminate Staying Healthy at 50+ in English and Spanish.
- To help their employees in the United States make more informed health care decisions, Bell and
Howell, General Motors, YMCA, Inc., and Bank One are giving them information adapted from AHRQ's Your
Guide to Choosing Quality Health Care.
- Kaiser and the VA are using a health education program for self-management by patients with the four
chronic diseases of chronic obstructive pulmonary disease, coronary artery disease with angina,
neurological disease with normal mentation and chronic arthritis. The health education program resulted
from the AHRQ-funded research project, Improving Chronic Disease by Self-Management Education.
Return to Contents
Internet Citation:
Performance Plans for FY 2000 and 2001 and Performance Report for FY 1999. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/about/gpra2001/gpra01a.htm
540 Gaither Road Rockville, MD 20850