Improving the Fit Between Evidence-Based Treatment and Real World Practice
Outreach Partnership Program 2005 Annual Meeting
Saturday April 2, 2005
David Chambers, Ph.D., Program Official
NIMH Division of Services and Intervention Research, Services Research and Clinical Epidemiology Branch
Dr. Chambers began his talk by describing the continuing gap between research and practice. He gave the following examples: many interventions are tested in controlled settings (i.e., in randomized clinical trials), interventions are often developed without consideration for how providers and complex systems will implement them; and outcomes relevant to consumers may be different from the population targeted by the intervention. While an intervention given as part of a study may result in a decrease in a particular symptom among the study population as a whole, the consumer is probably more interested in improvements in an individual’s functioning or improved quality of life.
We are attempting to challenge the prevailing assumptions that treatments are best developed in a linear fashion — moving from basic research, through efficacy and effectiveness studies — to application in the real world. In fact, evidence from treatments is just one part of the picture. We need to learn how evidence interacts with other factors including the following: the organizational structure and climate; the external environment (stigma and financing); provider knowledge and behavior; the needs of consumers; and family access and engagement.
There have been some steps in the right direction. For example, an NIMH Council report, “Bridging Science and Service,” emphasized the need for partnerships and communication between intervention researchers, treatment implementers, and the people who receive services. At NIMH and in several other institutes at NIH, this trend is reflected in a robust and growing series of research studies spanning science and service. Intervention researchers are increasingly interested in implementation issues. Of course, implementation research requires both a different perspective and a different skill set. A number of collaborations on both the federal and local levels are examining how science and service fit together.
We need to learn effective ways of disseminating knowledge to stakeholders, and this is an area where the experience of Outreach Partners can be very helpful. We want to know how evidence-based practices can be effectively implemented in a variety of clinical settings. In some cases, adaptation is required. We need to know how to determine whether an intervention used in practice is optimally effective. We also need to develop new interventions and address implementation issues from the start. This means we have to think about cost and feasibility at the beginning of a study.
In writing program announcements, NIMH defines the difference between dissemination research and implementation research as follows: dissemination is targeted distribution of a well-defined set of information through, for example, publication in a journal. It is unidirectional. Implementation is the process of introducing or changing practice to fit into a specific context, like a local setting. It assumes that adaptation will be needed, and that interventions will change when they are implemented in different settings.
David Sackett, who is considered the father of evidence-based practice (EBP), defined EBP as “the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients.” The key here is the combination of evidence and decision making to try to reach the best possible outcome for the care of individual patients. The weight is on both evidence and experience.
NIMH is interested in the following dissemination research topics: the factors that influence every stage of the dissemination process (from the way that the evidence is created to the way it is packaged and then transmitted); the way that it is received by a variety of stakeholders; the effectiveness of both individual and systemic dissemination strategies; and the differences in outcomes in specific settings.
Currently, NIMH supports several dissemination studies. One study is testing whether a Web-based information system can facilitate communications between consumers and providers and thus improve patient outcomes. Another is looking at how (and if) providers are implementing evidence-based clinical practice guidelines for treating aggressive behavior in children and adolescents. Other studies are looking at the influence of treatment guidelines on the Web on consumer outcomes, and the influence of direct-to-consumer advertising on provider behavior.
As in dissemination research, there are important questions that need to be asked at each stage of implementation. Examples include the following:
- How is the evidence created, packaged, and interpreted?
- Is the program ready to be implemented?
- Is there a fit between the program and the setting?
- Is a monitor in place to assure effectiveness?
Real world implementation is influenced by content, context, and process. Factors that must be considered include the following: pressures on the system, the organizational culture, the structure of a particular clinic or setting, and the attitudes and behaviors of local stakeholders. NIMH encourages researchers to look at clinical team structure within particular organizations and the dynamics of clinical team delivery of treatment to patients who may have complicated and complex needs.
In addition to studies of change in individuals and clinical teams, NIMH is looking at change in systems at various levels. NIMH is borrowing from the expertise of organizational behavior experts in an effort to link them with mental health experts for the following purpose: to produce meaningful research that can answer questions about how to implement change in mental health service systems. To support the development of research and service delivery agendas by state agencies, NIMH is co-funding two Requests for Applications (RFAs) with the Center for Mental Health Services (CMHS) at the Substance Abuse and Mental Health Services Administration. Last October, NIMH held a workshop designed to advance the science of implementation. Mental health practitioners, services researchers, interventions researchers, and experts in organizational behavior and social network analysis attended. In the future, NIMH and CMHS plan on holding regional meetings with the state mental health commissioners to examine and address the specific science to service needs of different regions.
The vision for the future looks like this: interventions based on service needs; constituents playing a more active role in setting the research agenda; intervention developers who include service settings in the research process; the testing of interventions with implementation in mind; and the development of service settings that have the capacity to monitor how well interventions are working.
Dr. Chambers listed several opportunities for collaboration between Outreach Partners and his branch at NIMH. First and foremost, he invited the Outreach Partners to contact him directly to share ideas for research projects. He called attention to the Program Announcement: Dissemination and Implementation in Mental Health, and provided the Web site address. He commented that Outreach Partners can offer a perspective, based on their vast experience that is different from researchers. He encouraged them to propose their own dissemination research studies or to form partnerships with researchers to evaluate dissemination efforts and/or the usefulness of interventions. He also emphasized the need to balance science with systems strengths and limitations. Finally, he said the Outreach Partners could play a valuable role in facilitating connections between researchers and consumers.
Questions and Answers
One participant asked about the barriers to evidence-based practice presented by cost controls imposed by managed care and the new requirements for prior authorization for prescriptions. Dr. Chambers said that NIMH has just funded a grant looking at the impact of prior authorization.
Disclaimer
* This document is intended to summarize a speaker presentation at the NIMH Outreach Partnership Program’s Annual Meeting and is not an official statement or opinion of the NIMH. This information is in the public domain and may be used or reproduced for educational purposes without additional permission from the NIMH.
