Evaluation of Process and Effects of Change

Differences Between Quality Improvement and Clinical Research Data

An understanding of the differences between quality improvement and clinical research data can help users to structure systems changes and their evaluation so as to effectively close the gap between knowledge and practice.

Collection of Data – Clinical Research vs. Quality Improvement

To develop new knowledge, clinical research uses rigorous methods to measure variables of interest and prevent the effects of confounding variables.

By comparison, clinical improvement studies aim to improve outcomes through the application of known information.

• The interventions are observable to the investigator (e.g., self-management education to enable patients to meet their management goals) 
• Just enough data is collected to test whether the change results in an improvement (Did patient education improve self-care practices?).
• Multiple steps are tested in a sequential fashion (Cycle 1: invitation to participate in self-management education. Cycle 2: level of patient participation. Cycle 3: Use of self-care practices, etc).

Enumerative vs. Analytic Statistics

Enumerative statistics are used in clinical research to evaluate the outcome of testing a hypothesis. The analysis assumes a stable system - one in which all variables are held constant except the one under study. The goal is to estimate whether the outcomes between the control and study group are different. The statistics ascribe a degree of confidence to the accuracy of the estimate.

Analytic statistics are used to evaluate clinical improvement. Using Plan-Do-Study-Act (PDSA) cycles is one way to evaluate clinical improvement. The goal of the analysis is to determine the stability of the process producing the data.
For example, will the patient recall system that increased the rate of eye exams from 36 percent to 70 percent consistently result in the higher percentage of patients having annual exams?

In this example the accuracy of the measure is not the issue (was the improvement in the rate of eye exams 70 percent or 68 percent or 72 percent?) Rather, if the process is statistically stable, one can assess its current performance and take action either to predict future performance or to measure the effects of an improvement intervention. For example, now that eye exam rates have improved to 70 percent, how can we further improve the system to increase the rate to 90 percent?

Translation of Clinical Trials

While much clinical knowledge of diseases and their treatment is generated through clinical trials, the results of those trials will best be applied to patient populations through the application of clinical improvement methods. Unlike the trials that generated such knowledge, patients live in a world with many sources of variation that cannot be controlled. Practice environments vary from primary care providers in office settings to sub-specialists in tertiary care centers. Payment schedules vary from public health clinics to managed care organizations to fee-for-service practices.

Individual physicians will interpret the results of studies and determine whether or how to incorporate the findings in their clinical practice. Unlike clinical studies, the patients receiving treatment are not a selected population so results of clinical trials are applied in an environment that differs from a research setting. To make better predictions and decisions, analytic statistics can assess deviations from expected results and identify sources of variation.

-Return to the Top-
-Return Home-

Continue to Evaluation: Resources