This section provides an overview of each of the 18 COER OPT projects, along with a description of the project's impact by type, or level, of impact.
In Level I (impact on further research) findings are presented within the following categories, as appropriate:
These summaries are based on review of study-related publications, as well as interviews with some principal investigators and users of the research results.
Principal Investigator: Joseph Lau, M.D.
Grant Number: HS07782
Project Period: March 1993-February 1997
Overview
This project advanced the analytical tools available to translate the results of randomized controlled trials (RCT) into clinically useful information. In addition, the project applied these tools to a broad range of key clinical conditions (e.g., renal disease, AIDS, hypertension, serious infection). Research efforts related to this project resulted in:
Methodologically, the project also advanced the science of meta-analysis, in particular, enhancing the value of relatively small, randomized, controlled trials.
The project developed a computer-based automatic process called Real-Time Meta-Analysis System (RTMAS). This process organizes, routinely updates, and displays randomized controlled trials (RCTs) and meta-analyses of pharmaceuticals in a simple and easy-to-understand matrix format. This significantly reduces the time required to conduct a systematic review and/or meta-analysis of the effectiveness or comparative effectiveness of therapeutic agents. A major contribution of the project is in the methodological improvement of meta-analysis with the development of control rate meta-regression.
Research related to this project resulted in 38 articles to date, including seven articles in the Annals of Internal Medicine, five in the Journal of Clinical Epidemiology, two in The British Medical Journal, and one in the Journal of the American Medical Association.
Impact on Clinical Practice (Level III)
Hospitalization and Infection: A meta-analysis conducted with the RTMAS confirmed that a single dose of aminoglycosides is as effective as and is less toxic than the standard multiple doses. A topic ripe for investigation, this study was one of the earliest of a number of studies in the field that confirmed this finding. According to the Project PI, anecdotal evidence suggests a definite shift in treatment patterns towards an increase in single-dose treatment as a result of this finding.
The meta-analysis compared a once-daily regimen versus the standard practice of multiple daily doses of aminoglycoside for serious infections. No significant difference was found in efficacy between the strategies in patients with serious infections, and there was a reduced risk of nephrotoxicity with the once-daily regimen. "Once-daily dosing can reduce the cost of administration and the toxicity of aminoglycoside in the inpatient setting. In addition, the once-daily dosing regimen could result in increased "ambulatory" use of these potent antibiotics instead of current practice of primarily inpatient use. The adoption of this approach has the potential for lowering costs for patients by obviating or shortening their lengths of hospitalization."3
Impact on Policies (Level II)
As further described below, work conducted under this project was influential in the development of guidelines for chronic kidney disease, recommendations for the management of acute sinusitis, a framework used by researchers in reporting RCT results, and syntheses of research on congestive heart failure and hypertension.
Impact on Further Research (Level I)
Both the RTMAS model itself, as well as applications of the model, have resulted in significant findings furthering research in a wide range of areas.
Research Tools and Translating Research for Clinical Care
Meta-analysis: As mentioned above, the RTMAS offers a methodological improvement of meta-analysis through the development of control rate meta-regression. This latter methodological innovation was supported in a well-recognized study on the discrepancies between meta-analysis and a large RCT. Another methodological area explored was the comparison of meta-analysis of large studies versus meta-analysis of small studies; the results of smaller studies are usually compatible with the results of large studies, but discrepancies do occur even when diversity across studies is considered. Examining sources of heterogeneity between the results of large and smaller trials provides information that may not be conveyed by estimates of average treatment effect. Further, the investigators identified the need to "to re-examine the idea that we should not start a randomized control trial unless sufficient patients are available to avoid reasonable Type I and II errors. Meta-analyses will come to the rescue, provided trials continue to be published at the present rate."4
The project addresses Impact Level I in that RTMAS provides a unique and powerful way of visualizing the availability of evidence. The ability to dynamically generate evidence tables by answering a series of questions makes it more efficient to conduct systematic reviews. From a collection of studies of various designs, RTMAS could readily determine the quantity of various levels of evidence available for synthesis and decide whether certain questions can be answered.4
Bringing Practice Up to Speed with Research: Comparison of the results of RCTs with clinical expert recommendations reveal that these recommendations are not often synchronized with accumulating evidence. The rational use of therapeutic drugs can be achieved only through the routine use of meta-analysis on high-quality clinical data. This implies that clinical trials related to the same topic should be viewed as a continuum of experience; meta-analyses should be revised regularly; and a method of regularly updating and reporting findings to individual physicians and investigators should be developed.
Proposed steps to pursue include ensure all clinical trials follow an agreed-upon scientific format; set up a mechanism for registering all clinical trials undertaken; ensure that all registered clinical trials with some outcome data are included in ongoing cumulative meta-analyses; ensure that experts trained both in the specialties of interest and in clinical trials monitor databases of earlier noted recommendations, and agree on information to go to health practitioners; and institute regular physical and knowledge examinations for physicians.6
Treatment Effectiveness
Public Health and Prevention, and Chronic and Persistent Disease
Gastrointestinal and Hepatic Diseases: This analysis finds that:
"RCTs are so necessary to evaluate the small incremental changes in the treatment of gastrointestinal and hepatic diseases evolving out of basic research that their future potential is enormous. As the suggested improvements in treatment become smaller, and the use of a no-active-treatment group less ethically acceptable, individual trials will seldom be large enough. Cumulative meta-analyses will be increasingly important as one contributor to the decision needing to be made in a timely fashion as to whether a new treatment is better, worse or the same as the standards. Acute viral hepatitis is so variable in its course that RCTs will be crucial to the proper evaluation of the many new antiviral agents coming out of the laboratory in the next few years."7
The study provides suggestions for future research directions, including selective RCTs that focus on specific treatment issues e.g., "the presently popular proximal or selected vagotomy for peptic ulcer is not entirely satisfactory and many attempts at improvement can be expected.... Also, the five methods of treating bleeding peptic ulcers through the endoscope need to be compared in well-controlled studies."7
Principal Investigator: William M. Tierney, M.D.
Grant Number: HS07763
Project Period: July 1993-January 1997
Overview
In this project, the investigators incorporated treatment guidelines into a computer-based system in a large institutional setting where all test and prescription ordering must be carried out on computer stations. The randomized controlled trial was conducted to demonstrate the impact of this system on treatment patterns, prescriptions, and patient-centered outcomes, but concluded without positive result. However, the research resulted in significant lessons about:
The project helped to fuel research in other areas such as aging and palliative care. In addition, the project resulted in the adoption of new policies and procedures for the large integrated delivery system in which the study was conducted. This health services delivery system continues to attract attention for its innovations in system-wide and centralized computerization for clinical care.
Impact on Further Research (Level I)
Cost and Economics of Health Care
Tools for Patient Management
Computerizing Practice Guidelines: The final report for this project, as well as other project-related publications, highlight some critical lessons about when and how guidelines can most effectively be incorporated into electronic systems from the provider's perspective. The final report describes the necessary conditions for success in computerizing practice guidelines: "The guidelines must be accepted as the local standard of practice by clinicians...computer prompts must be timely and use data that are already available electronically...and computerized guidelines should have measurable effects." The study also concludes that "changing from a paper-based outpatient prescription system to a microcomputer-based prescription-writing system had profound effects on the types of activities carried out by pharmacists in a hospital-based outpatient pharmacy." The report notes "that delivering real-time drug utilization review (DUR)-based treatment suggestions to outpatient pharmacists had dramatic and salutatory effects on their [pharmacists'] work patterns, increasing opportunities for them to advise patients, consult with providers, and solve problems." With respect to physicians, the report also concludes "that the favorableness of respondents' opinions of the inpatient workstations declined as the level of training increased, a trend that was independent of computer literacy. Increasing computer use by physicians would probably require modification of the educational and socialization process rather than mere reliance on increasing computer literacy."16
A related article finds that the AHCPR Heart Failure Guidelines as written cannot be automated in existing electronic medical record systems. The author proposes the following recommendations to improve future versions of these and other clinical practice guidelines:
Write all guidelines in a simple "if-then-else" format; make algorithm logic hinge on explicitly defined values of accepted clinical parameters; expect that local translation of guidelines will be necessary and help guide that process; include rules about errors of commission and omission, and don't ignore common co-morbid conditions; balance the costs of diagnosis and treatment (not only in terms of dollars) and consider the likelihood that individual patients will benefit from the guidelines' recommendations; and evaluate the resulting guidelines using real patients and representatives of all those who will use them.17
Electronic Guideline-based Treatment Suggestions for Pharmacists: The component of the project that looked at pharmacist work patterns found favorable results: "The results of this study indicate that a dramatic, favorable change in work patterns occurred when pharmacists were provided with an electronic display of guideline-based, patient-specific treatment suggestions. Specifically, pharmacists who had access to treatment suggestions spent less of their time preparing and filling prescriptions and more of their time functioning in an advisory role with patients, physicians, and nurses. Moreover, more of the pharmacists' time was spent solving problems.... The delivery of patient-specific information to pharmacists at the time of dispensing significantly and favorably influenced pharmacist work patterns by increasing opportunities for pharmacists to advise, consult and solve problems."18
Researchers also learned about the barriers pharmacists face in talking with patients, which is important for prospective utilization review. Patients often do not like to interact with pharmacists in the drug store. Further, pharmacists may feel intimidated dealing with physicians, and prefer the less threatening E-mail response system.19
Translating Research to Influence Practice
Special Needs of Target Populations
Principal Investigator: Wayne A. Ray, Ph.D., M.S.
Grant Number: HS07768
Project Period: July 1993-July 1998
Overview
This project addressed the use of non-steroidal anti-inflammatories (NSAIDs) for osteoarthritis in the elderly population. The project:
Impact on Policies (Level II)
Guidelines for medical management of osteoarthritis of the hip and of the knee were published in Arthritis and Rheumatism24, and in Archives of Family Medicine.25
Impact on Further Research (Level I)
Studies associated with this project identified the significant cost savings that can result from prior authorization features of drug utilization review. It also illustrated the advantages and challenges of using administrative data (e.g., Medicaid data) for research purposes.
Cost and Economics
Translating Research to Influence Practice
Principal Investigator: Stephen Berman, M.D.
Grant Number: HS07816
Project Period: July 1993-June 1997
Overview
This study focused on the comparative cost-effectiveness of various antibiotic treatments of otitis media (OM). The investigators used the Colorado State Medicaid database to identify exposure and outcomes. Information related to this work has been used by a variety of organizations in development of policies and educational programs. In addition, the study identified important issues for further research in the area of diagnosis and treatment of the condition.
Impact on Policies (Level II)
Dr. Berman has worked with the Colorado State Department of Health to develop guidelines for Colorado Medicaid. He has also worked with the American Medical Association on a program to evaluate outcomes of otitis media that will be pilot tested in Utah. He has developed an Internet-based interactive Web site to teach current guidelines on the treatment of OM. The site and a companion workshop are sponsored by the American Academy of Pediatrics. Dr. Berman reports working with a number of managed care companies to develop guidelines but impact has not been quantified. Finally, Health Plan Employer Data and Information Set (HEDIS) measures include the use of amoxicillin or trimethoprim/sulfamethoxazole (TMP/SMX) as initial therapy for otitis media.29
Concern about the development of resistant strains of bacteria have led a number of key public health policy and delivery organizations—such as the Centers for Disease Control and Prevention, State health departments, and HMOs—to become interested in this research, because OM is one of the primary conditions for which antibiotics are prescribed. Researchers involved with the project noted that it is important that public agencies continue to support medical outcomes studies that provide a balanced perspective on such things as the use of limited spectrum versus broad spectrum antibiotics when appropriate, including for otitis media.
Impact on Further Research (Level I)
The findings of this project not only suggest that physicians over-prescribed antibiotics, and often failed to obtain appropriate audiology testing, but also show that the more expensive antibiotics were not associated with better outcomes.
Treatment Effectiveness
Cost and Economics
The Implications of the Cost of Otitis Media Treatment for Future Vaccination: This analysis developed information about OM-related medical expenditures to help evaluate the cost-effectiveness of vaccines being developed for respiratory pathogens that cause OM. It used data on medical visits, pharmaceuticals and surgical procedures for 87,057 children 13 years of age or younger who were continuously enrolled in Colorado's fee-for-service Medicaid program during 1992.
Findings showed that expenditures for visits are the largest expenditure component, therefore case management guidelines should emphasize reducing unnecessary visits by improving physician training in pneumatic otoscopy and by scheduling followup visits for asymptomatic children 3 to 4 weeks after diagnosis rather than 10 to 14 days. Further, because 40 percent of expenditures are incurred between 1 and 3 years of age, vaccines designed to reduce the incidence of otitis media are most likely to be cost-effective if they can be administered before the child's first birthday.34
Tools for Patient Management
The Effect of Continuous Medicaid Enrollment and Having an Assigned PCP: This analysis looked at the likelihood of a Medicaid child enrollee having a primary care physician (PCP) if he or she were continuously enrolled in Medicaid as compared to discontinuously enrolled. It also looked at the relationship between having a PCP and emergency room use for otitis media.
Children and continuously enrolled in Medicaid throughout the entire year were more than four times as likely to always or sometimes have a PCP compared with children who were discontinuously enrolled. Further, the likelihood of ever using the emergency department for an otitis media-related visit was increased by 26 percent when a child sometimes had a PCP, and increased by 50 percent when a child never had a PCP. Finally, the likelihood of ever filling an antibiotic for otitis media was reduced by 23 percent and 34 percent respectively when a child sometimes or never had a PCP compared with always having a PCP.35