Christopher Forrest, M.D., M.P.H.
Assistant Professor, Johns Hopkins School of Hygiene and Public Health
Lawrence C. Kleinman, M.D., M.H.A.
Chief of Health Studies, Lehigh Valley Hospital
Andrea K. Biddle, M.P.H., Ph.D.
Associate Professor, University of North Carolina at Chapel Hill
Uma Kotagal, M.B., B.S., M.S.C.
Director, Health Policy and Clinical Effectiveness, Cincinnati Children's Hospital
Euni Lee, Pharm.D., Ph.D.
Center on Drugs and Public Policy
Maryam Navaie-Waliser, Dr.P.H.
Research Associate, Center for Home Care Policy and Research, Visiting Nurse Service of New York
Ilene Zuckerman, Pharm.D., BCPS
Associate Professor, Center on Drugs and Public Policy, University of Maryland, Baltimore
Introduction
Assessment of Pharmacologic Management of Asthma in Children Using Medicaid Databases
Methodological Issues in Studying the Impact and Cost-Effectiveness
of Care Coordination Services
Assessing Guideline Use Linking Outcomes to Performance
Conceptual Framework for HSR and the Uncertainty Index
Dr. Lawrence C. Kleinman introduced the objectives of the session:
In achieving these objectives, it is important to begin by clarifying definitions of assessment, program evaluation, and key factors such as quality, structural/system components, process of care, and outcomes:
Dr. Kleinman's orientation was followed by three presentations on CHSR research that emphasized the process and steps of research as well as the importance of different research contexts. The various research settings included:
Child Health Services Research can take advantage of public databases such as medical claims data, and systems such as Medicaid drug utilization review. A study presented by Euni Lee, Ilene Zuckerman and Diane McNally described how they used these resources to look at adherence to guidelines of the National Asthma Education and Prevention Program (NAEPP, 1997) with respect to the use of a quick relief medication.
The purpose of the study was to evaluate the therapeutic appropriateness of asthma drug use among Pennsylvania fee-for-service enrollees. The researchers outlined the following research steps:
The study's topic focus was asthma management with short-acting beta 2-agonist in Medicaid children. Asthma is the most common childhood chronic disease; 4.8 million children were affected in 1996 and there have been increasing rates of mortality and hospitalization.
Criteria for evaluating prescribing patterns were developed from the NAEPP guidelines: the drug of choice for quick relief medication is SA inhaled beta-2 agonist and long-acting (LA) inhaled beta 2-agonists should not be used to treat acute exacerbations. Over-reliance of SA beta 2-agonists and poor asthma management was defined as using greater than 1 MDI canister per month and more than or equal to 2 MDI canisters within 25 days at least twice.
The database included all pharmacy claims of any asthma drug from prescription records for children in Medicaid with asthma medications between July 1, 1998 and January 31, 1999. Data collection required substantial editing and validation of key elements of the Medicaid claims data. For example, algorithms needed to be developed to eliminate duplicate claims; data entry errors or missing information such as dual eligibility (Medicare and Medicaid) status needed to be determined based on other information.
The study identified 14,292 children as asthma drug users during the 7-month period. Of these, 392 children were identified as SA beta 2-agonist inhaler overusers, 42 children received non-recommended SA beta 2-agonist inhalers, and 37 children used salmeterol as a LA beta2 inhaler without SA beta2-agonist inhalers. This information was used to assist the Pennsylvania Department of Public Welfare as groundwork for educational interventions with physicians and pharmacies.
The advantage of Medicaid data use includes:
At the same time, some potential limitations were recognized, including information about patient utilization and patient history which could demonstrate contraindications to specific medications for individual patients.
Estimating the impact and cost-effectiveness of North Carolina's care coordination program relative to infant survival through the first year of life was the objective of a study presented by Drs. Maryam Navaie-Waliser, Andrea Biddle, and Sandra Martin. Their presentation focused on methodology and the 12 key steps involved in the study, listed below:
The presenters concluded by emphasizing that the benefits of economic analyses of intervention programs include the potential for improved targeting of programs to populations likely to receive the greatest benefit.
Research often takes place in the context of delivery systems that offer special opportunities for research but also present particular challenges. Dr. Uma Kotagal presented an assessment of compliance with evidence-based guidelines among eight hospitals participating in the Child Health Accountability Initiative (CHAI).
CHAI was developed to design, evaluate and implement collaborative projects and research to promote clinical practices that lead to improved health outcomes for children. This hospital network provided a basis for research collaboration and opportunities to achieve an adequate sample size. At the same time, it presented challenges related to satisfying institutional (as well as research) objectives, and also coordinating research across sites.
The study looked at process and outcome measures associated with implementation, during the winter of 1999, of a guideline for Bronchiolitis at the eight CHAI hospitals. Process measures that were looked at included:
Data related to these measures was gathered via:
Outcome measures included:
The latter involved looking at such things as lingering symptoms, economic impact and child care use. A telephone survey of parents was employed to gather outcomes-related data.
The study found that the NAEPP guideline was followed in 79 percent of cases, although compliance was not associated with illness duration, or post-discharge burden or resource use. Satisfaction with care was associated with resolution of symptoms. Finally, there was a significant economic impact of the acute illness on families involved. A third of the families took significant time off, in part because lingering symptoms were a significant factor in the study sample.
With respect to the use of the hospital network as the basis for research, the study found that demonstrating an effect, or no effect, may require multiple sites even for common illnesses. Also, measuring process and outcomes requires multiple data sources, and studying practice in situ provides opportunities for linking process indicators to outcomes.
In concluding the session, Dr. Kleinman reiterated the importance of establishing a conceptual framework for health services research, and also for translating practice into research (or TRIP as referred to by AHRQ). He also identified the need to think of ways to evaluate where research is most needed, and where research dollars should be devoted.
In this regard he discussed the Uncertainty Index, or UNI, which indicates the proportion of non-refuted current practice that is also not corroborated by the research base. UNI will be higher for less-researched areas, and lower for well-researched areas. This allows in some respects flexibility for standards of evidence, with lower standards for evidence of areas with high UNI, and higher standards for evidence of areas with low UNI. UNI can be used to address the fact that there is a disproportionate amount of research devoted to adults as compared with children in the United States.
Current as of June 2000
Internet Citation:
Advanced Models and Methods, Part 2. Role of Partnerships: Second Annual Meeting of Child Health Services Researchers. June 27, 2000. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/research/chsr2mm2.htm