Practice Gap
Advances in Patient Safety: From Research to Implementation Volume 3. Implementation IssuesBeyond the Dusty Shelf: Shifting Paradigms and Effecting Change
Dwight McNeill, Howard Holland, Kerm Henriksen
AbstractThis paper addresses how to make happen the improvements in the quality of health care that have been identified from significant investments in patient safety research by the Agency for Healthcare Research and Quality (AHRQ). We make the case that the usual supply-side research model is inefficient to produce the health care changes expected from AHRQ. We propose a shift to a demand-side paradigm that engages users throughout the research process, and two models to guide the management of "action production." The first model is based on Rogers' model of diffusion of innovations, which indicates that users must absorb a great deal of information in a variety of staged and specific ways in order to make a successful passage from knowledge to action through tactics including awareness, persuasion, adoption, implementation, and confirmation. The second is a decision model, termed distillation, which provides a framework for determining the potential utility and priority of an innovation based on the strength of the science, potential impact, adoptability, and readiness. We address lessons learned from the application of these models to the early implementation experiences of five early outputs from the AHRQ patient safety portfolio. We find that the implementation of the early findings places a strong reliance on information dissemination mostly at the awareness and persuasion stages
efforts directed at the later stages of decision, implementation, and confirmation have been modest. Ongoing evaluation of the impact of these approaches on patient safety practices and quality of care will indicate if the models provide useful guidance in making change happen.
In 2001, the Agency for Healthcare Research and Quality (AHRQ) launched a series of grants and contracts to stimulate research and demonstrations to improve patient safety and reduce medical errors. Together, these projects make up an integrated set of activities to explore different approaches for reducing patient harm, increasing the Nation's knowledge base, and making change happen by converting what is known into what is practiced.
Six program areas make up the core of these activities: (1) centers of excellence for patient safety research and practice; (2) developmental centers for evaluation and research in patient safety; (3) health system reporting, analysis, and safety improvement research demonstrations; (4) clinical informatics to promote patient safety; (5) research on the effects of working conditions on quality of care and patient safety; and (6) patient safety research dissemination and education. A year earlier, AHRQ funded six grants to demonstrate and evaluate systems-related best practices. An in-depth discussion of the features of these program areas has been provided by Keyes and colleagues. 1
The findings from the patient safety research initiative have started to add to an accumulating knowledge base called for in two widely cited Institute of Medicine (IOM) reports. 2, 3 The challenge for AHRQ is to get the findings and ensuing publications off the dusty shelf that is so often the destination of quality and safety initiatives and into use so that they may foster behavioral change that improves the quality of health care. AHRQ and other Federal agencies are increasingly accountable for the impact of such research, and not solely for the number of studies funded or completed. This accountability is underscored in AHRQ's new mission statement, "to improve the quality, safety, efficiency, and effectiveness of health care for all Americans." 4 However, to achieve this goal requires changes in the "business as usual" research paradigm of knowledge production and information dissemination.
The purpose of this paper is to make the case for a demand-side approach to the creation and utilization of research; to propose two models to guide the management of "action production," including diffusion and distillation; and to address lessons learned from the application of these models to the early implementation experiences of the AHRQ patient safety initiative.
The usual supply-side research paradigm is illustrated in Figure 1. Researchers reside largely in their own world and decide what the interesting questions are, develop hypotheses and study designs to test them, and then publish results in journals and present their findings at conferences. The purpose of the research is to contribute to the knowledge base in a particular field and, indirectly, to change the behavior of an end user.
Users live in a different world. They are looking for new discoveries from research to help them solve current challenges. For such a match to occur, the research product must find a user "receptor site." The pathway to find the receptor site is not well charted, but rather long and winding. Perhaps the user in need of the findings will learn about them in a journal, digest, or conference. However, even when the research product approaches a match with the unique receptor site, it may not fit precisely because there is some incompatibility with the particular population, setting, political and institutional context, or existing ideologies and beliefs. Hence, the efficiency of getting relevant research to an intended user is low, as the results pass through a number of users who might not have pressing need for or interest in the findings, a closer fit is required between the research and user need, and the setting and context must be receptive.
A different approach comes from a demand-side perspective. Economists describe the demand and the supply curves intersecting to achieve equilibrium, which in the case of prices represents the optimal price point to satisfy both suppliers and demanders. Similarly, the influence of the demand side on the supply side in research may produce an analogous balance to achieve more efficient production and use of research outcomes. Here, as seen in Figure 2, the need (receptor site) of the user is identified a priori and brought into the line of sight of researchers, who then target their questions and methods to produce results for the user. This can result in a series of targeted research results, some of which are published, but all of which are delivered back to the community of users with similar receptor sites in ways that meet the users' information needs. Notice that the winding road in search of receptor sites is eliminated.
Diffusion
The challenge in driving safety and quality improvements in health care is to provide the right informationin the right way, at the right time, and to the right userin order to maximize uptake and the conversion of knowledge to action. One model for understanding this process is the Rogerian model of diffusion of innovations.
5
He defines five stages of the diffusion of innovation and stage-specific tactics, as illustrated in Figure 3.
The challenges at each stage from the perspective of user needs are described below.
Awareness. The first stage in the process of producing safety improvement is awareness. At this stage, users are interested in being informed about new knowledge that may provide potential benefits to them. The best vehicle for introducing knowledge to the largest possible audience is mass media, which may include specific tactics such as general, trade, and professional media, as well as marketing through multimedia formats.
Persuasion. At the second stage, potential users grapple with what the meaning of the data is to them in relation to their own particular needs. Data can become more relevant when transformed into information through an analysis of gaps that exist between the user's actual data and benchmarks that "make the case" that there is a strong potential to save expenditures and lives through action. Specific tactics may include fact sheets and issue briefs focused on the interests of user groups, customized analysis of data, and presentations at professional meetings.
Decision. At the third stage, users need to know about programs that have actually been implemented that are related to their area of interest. They want to hear about success stories involving others like them, and learn about how the program works. They need to evaluate if the approach would work in their own setting by experiencing the program in a vicarious way. They want to do some mental modeling by putting the pieces of the program together according to the needs and requirements of their own situation. Specific tactics may include case studies, resource guides and workbooks, as well as targeted professional conferences.
Implementation. Implementation follows the mental decision to go ahead. In order to complete this stage and "turn on the switch" of a safety improvement program, users seek specific how-to information, e.g., where to get the program software, how to operate it, and what to do and who to turn to if the program goes astray from the expected course. Hence, they seek a deeper understanding of program specifics, available resources, and ways to sustain improvement. Specific tactics may include individualized technical assistance, where experts are brought to users to help with the installation, simulation exercises, and very targeted and participatory conferences with program "champions."
Confirmation. Lastly, as the adoption of a new practice or policy takes hold, users need to understand performance in order to make adjustments in the design and operations so that it becomes institutionalized or altered to fit the changing needs of the organization. Specific tactics may include evaluation of the practice or policy to understand how well it works and what factors contribute to its outcomes. Ongoing evaluation can also provide a learning loop to allow for the continuous adaptation of the program to the setting.
In general, the job of action productioni.e., making change happen through implementationis not done until it is "locked and loaded" into an operating system. Users who reach this point will have passed through each of the diffusion stages and experienced many of the tactics. Of course, some users may leapfrog over some stages. Of most concern, however, is when there is a blockage and the diffusion process terminates and does not reach fruition, which probably occurs far more frequently than completion of the process. Much effort can be expended on mass media, conferences, and customized data analysis, for example, but if this effort is not converted into a decision to implement and sustain safety improvement measures, then the investment is essentially wasted.
This raises the issue of defining the role and contribution of AHRQ in making change. Heretofore, as basically a research institution, its job has been knowledge production. Now, as it takes responsibility in its mission statement for results, it wrestles with what its core capabilities are in the diffusion cycle, how it allocates resources across the various strategies and tactics, how it partners with other organizations that are in the same change and accountability domain, and how it demonstrates that its efforts contributed to change.
There are many findings, products, and innovations coming out of AHRQ's patient safety portfolio of research. It is important to identify and prioritize those that represent "knowledge ready for action" and are likely to achieve the greatest impact. Our distillation model, which was developed by the AHRQ patient safety team, differs from our diffusion model. This analysis framework provides a guide on the potential utility and priority of an innovation, finding, or product based on the appropriateness of the evidence, potential impact, adoptability, and readiness.
Appropriateness of evidence. The first order of business for a science-driven agency is to ensure that the knowledge produced and ready for adoption is based on appropriate science. Are findings based on an appropriate methodology for the research hypotheses investigated? Are the findings supported by similar findings in the literature? Have the findings been generally accepted by the field? Will attempts to spread implementation be dashed, due to criticism about the appropriateness of the evidence base?
Potential impact. What is the potential magnitude of the innovation in terms of improved health care and health, e.g., reductions in mortality, lessened disability, improved functioning, and decreased adverse events? What is the impact on expenditures? What is the cost-benefit equation?
Adoptability. What is the perceived relative advantage of the innovation as compared to the prevailing practice? Is it compatible with existing beliefs, values and practices, prior successful innovations, or other desirable ideas? Is it simple or complex? Is it relatively easy to try and evaluate, given the organization's infrastructure and culture? Is it a complex "contingent" decision that requires layers of decisions?
Readiness. Are there any gaps in the above categories that need to be filled? Is it packaged for success, e.g., documentation, user manual, helpline, technical assistance? Are communication channels lined up correctly? Are network connections built and primed to act on the innovation? Is there a change agent available to influence opinion leaders?
The distillation and diffusion models are complementary. The distillation model helps identify priorities for diffusion and identifies issues to be addressed during the stages of diffusion.
We concentrate on five early outputs from the AHRQ patient safety portfolio to gauge the usefulness of the models: infection control with chlorhexidine, patient information on safer health care, Patient Safety Improvement Corps, improving the culture of safety in hospitals, and patient safety reporting.
ChlorhexidineResearchers supported in part by AHRQ have demonstrated that chlorhexidine gluconate is a superior, cost-effective alternative to betadyne to prevent catheter-related bloodstream infections. This reinforces previous research on the relative advantage of chlorhexidine and extends the case for its use by showing better effectiveness and savings, despite a higher unit price. In terms of its priority for adoption, the evidence base is strong, the impact clear, and the results compatible with the literature. However, it is equally clear that challenges exist in fostering further adoption of chlorhexidine and identifying the steps that AHRQ could take to support that process.
First, the business case for chlorhexidine's use may need to be stronger for potential users to realize its relative advantage and to overcome natural barriers presented by the purchasing practices in most hospitals. Hospitals buy many supplies packaged together in bulk, achieving discounts and economies of scale as a result. Therefore, achieving greater use of chlorhexidine is dependent, in part, on demonstrating why doing so is worth any short-term financial impact because the long-term cost savings will ultimately be greater.
Second, while components of the infection control community support the use of chlorhexidine, 6 not everyone involved in clinical areas where infection control is a priority is necessarily committed to chlorhexidine's use. One concern expressed by potential users is that chlorhexidine is colorless, whereas betadyne is not, and practitioners may not know if the agent has been applied.
Considering issues like these, it is critical to know the end user and the application context much better when attempting to implement complex clinical findings. This emphasizes the need for a demand-side strategy, where receptor sites are known and ready to receive new knowledge that would move the process along toward implementation. As a result, when devising any implementation strategy, there must be a great deal of detective work, involving lengthy discussions with professionals in the field to understand their needs, concerns, and possible solutions. This is a very time-consuming enterprise and tests the extent to which AHRQ needs to invest in these types of intensive assessments and conversions. It also requires consideration of the need for partnerships and intermediary relationships with those closer to the action and adept at change-agent facilitation.
In partnership with the American Hospital Association (AHA) and the American Medical Association (AMA), AHRQ has put in place an education campaign, via distribution of posters, based on the fact sheet, "Five Steps to Safer Health Care." This fact sheet was first developed by the Federal Quality Interagency Coordination (QuIC) Task Force and further refined as a result of focus-group testing by the Centers for Medicare and Medicaid Services. The campaign was launched by then-Health and Human Services (HHS) Secretary Tommy G. Thomspon at the AHA's annual leadership meeting in August 2002. The poster is being distributed to the mass media and to hospital waiting rooms and doctors' offices through the partners' networks.
This strategy is largely one of awareness with mass media tactics. It is a relatively easy output to produce at low cost. However, it does not progress beyond the awareness stage in the diffusion model, and one may question if such an awareness program is sufficient to change patient behavior. It is not clear how effective such a strategy of patient education is in inducing changes in provider behaviors related to safe care. The diffusion model would suggest that users (patients, in this case) need to be persuaded after they are informed. Do other groups pick up on AHRQ's information and drive the point home to patients through conferences and clinical encounters? Patients then need to decide to act by asking questions and making choices. The question is whether they are sufficiently empowered to do so.
The goal of the PSIC, a partnership between AHRQ and the Department of Veterans Affairs, is to improve patient safety by providing knowledge and skills to teams of State field staff and hospital partners selected by the States. The field staff include hospital patient safety officers and those responsible for patient safety reporting, analysis, and initiatives to improve safety and quality of care. In particular, the goal is to equip these teams to conduct effective investigations of reports of medical errors (for example, "close calls" or errors that happen without injury to patients), how to prepare meaningful reports on the findings, and then how to develop and implement change at a system level that can be sustained and evaluated over time.
The first annual program enrolled 51 participants from 15 States. The second program, which is currently underway, has approximately 60 participants from 21 States. As these State teams return to their institutions and share the knowledge that they have gained with their colleagues at their own hospitals and other facilities in their States, they act as extension agents that reach beyond their own organizations. Thus, the PSIC facilitates every stage of the diffusion model by building the capacity for persuasion, decision, implementation, and confirmation at the local level, as described in Rogers' diffusion model. However, sustaining change in an institution is difficult for the many reasons discussed previously. It raises the issue of the need for ongoing support to help these "extension agents" with specific implementation challenges and with evaluation of the installed programs so that they can be fine tuned.
AHRQ has developed a Hospital Survey on Patient Safety Toolkit to assess the culture of patient safety in hospital settings. This survey was pilot tested in 21 hospitals, and the resulting data helped refine the toolkit and provide guidelines for its administration. An implementation plan has been developed that would involve private-sector hospital groups, such as Premier, and public-sector groups, such as the Department of Defense, to help distribute the tool. However, getting the tool distributed may not necessarily lead to its use and, subsequently, to actual behavioral change. Again, this project tests the boundaries of implementation at AHRQ and begs the question of how far beyond product packaging and dissemination does AHRQ's involvement continue? There has been discussion of the possibility of providing technical assistance to implementation sites to facilitate the process of culture improvement.
AHRQ supports the development of safety reporting systems that can ensure public accountability and promote mutual trust, openness, and learning. It has made the case for such systems to a number of States. In Massachusetts, legislation has been passed to set up a system to publicly report information on egregious errors. Supporting States and other entities that pass patient safety legislation is an important public policy goal for AHRQ. In terms of the diffusion model, strong patient safety legislation includes awareness, persuasion, and decision. However, the adoption decision of a legislature does not necessarily translate into the actual installation of an effective reporting system. It certainly lays the foundation for implementation and, if adequately funded and with sufficient regulation to ensure action, it can lead to substantial change. The question is whether AHRQ's job is done at the point of enactment of legislation. Should AHRQ concentrate its efforts on achieving similar results from multiple States, or should it proceed in a more focused way to ensure that the legislation is translated into effective implementation?
AHRQ's patient safety research portfolio represents one of the most significant bodies of work that the Agency has carried out. It is also one of the most sizeable single investments that the Congress has made in the area of patient safety. As a result, AHRQ has the potential to have a very positive effect on the quality and safety of the Nation's health care system. Indeed, Congress and patients themselves will be counting on it.
Making health care changes that results in distal impacts of improved health and safety for populations is hard to do, hard to evaluate, and hard to prove. The National Healthcare Quality Report produced by AHRQ
7
demonstrates that it is far more common to see no annual change in national measures of the quality of health care than to see significant changewhether improvement or deterioration. For the patient safety measures with trend data, spanning the period 1998 to 2002, 10 measures were unchanged (e.g., the percent of the elderly who reported that their usual source of care did not ask them about medications prescribed by other doctors), 6 improved (e.g., ventilator-associated pneumonia among adults and children), and 6 deteriorated (e.g., iatrogenic pneumothorax). Hence, moving the national needle on patient safety measures is a big challenge.
Rogers' model of diffusion is a hard task master. It strongly suggests that users must absorb a great deal of information in a variety of specific ways in order to make a successful passage from knowledge to action. Awareness is not enough, and neither is persuasion, implementation, or adoption. Change must be sustainable and programs need to evolve and adapt to changing local concerns and new knowledge. What is clear is that the old ways of creating knowledge and making it known to potential users in the supply-side mode is inefficient for the new generation of performance and outcome-driven Federal health and health care agencies. Agencies must demonstrate their programs' impact to the Office of Management and Budget on an annual basis. Rogers' model, conceived and tested in the seemingly incongruent context of agriculture, provides very useful guidance on how to make this happen.
The preliminary lessons learned from the implementation of the first cohort of patient safety findings and products indicate a strong reliance on information dissemination, mostly at the awareness and persuasion stages. The realignment of funds and efforts focused on decision, implementation, and confirmation has been modest, but is growing.
The expectations to improve patient safety are high and diffuse. AHRQ needs to make management decisions on how to spend scarce resources. This involves trade-offs between breadth and depth, between content areas, and between dissemination and implementation strategies and tactics. The distillation model provides helpful guidance on the prioritization of innovations that are ready and important to implement. The distillation model can also be helpful before the research is commissioned so that these strategic imperatives of appropriate evidence, impact, adoptability and readiness are built into the design of the conduct of the research, including specific attention to its implementation. In this way, significant synergies with a demand-side approach can be achieved.
These models are guides, not blueprints. The models help in organizing the work, but the work in many senses has a life of its own, with the twists and challenges that come when dealing with the real world where health care providers and policymakers operate. The diffusion model provides some navigational aids, but the course is largely uncharted. As one goes out into the wilderness of implementation, the process is a lot like detective workit begins with what can be gleaned from professional articles, the researchers who did the studies, and a hands-on examination of the findings. The significance of the findings or the ability to foster implementation activities becomes clear only after much more lengthy discussions with those professionals in the field who are involved in the area under consideration.
These models are not the only ones. Others include Hasenfeld and Brock's model on policy implementation,
8
Pressman and Wildavsky's model on public program implementation,
9
Gladwell's tipping point model,
10
complexity theory,
11
and social networking.
12
All of these have a stronger emphasis on developing partnerships, cultivating relationships with intermediaries, ongoing learning collaboration, and understanding organizational barriers and levers for a change agent that can provide insight for implementation. These models will be discussed, debated, and incorporated into patient safety implementation strategies as more knowledge is put to action and the models are evaluated to understand their ability to explain and guide impact on health care.
All authors are affiliated with the Agency for Healthcare Research and Quality, Rockville, MD.
Address correspondence to: Dwight McNeill, Agency for Healthcare Research and Quality, 540 Gaither Road, Rockville, MD 20850; phone: 301-427-1734; e-mail: DMcNeill@ahrq.gov.
1. Keyes MA, Ortiz E, Queenan D, et al. A strategic approach for funding research: an overview of the Agency for Healthcare Research and Quality's 2000-04 Patient Safety Initiative. Rockville, MD: Agency for Healthcare Research and Quality; in press.
2. Kohn LT, Corrigan JM, Donaldson MS, editors. To err is human: building a safer health system. A report of the Committee on Quality of Health Care in America, Institute of Medicine. Washington, DC: National Academy Press; 2000.
3. Institute of Medicine. Crossing the quality chasm: a new health system for the 21st century. Washington, DC: National Academy Press; 2001.
4. Agency for Healthcare Research and Quality. Mission statement. Available at: http://www.ahrq.gov/about/ budgtix.htm . (Accessed 2004, June 9).
5. Rogers E. Diffusion of innovations. New York: The Free Press; 1995.
6. Centers for Disease Control and Prevention. Guidelines for the prevention of intravascular catheter-related infections. MMWR recommendations and reports. Volume 51, Number RR-10; Aug 16, 2002. Available at: http://www.cdc.gov/ncidod/ hip/IV/Iv.htm .
7. Agency for Healthcare Research and Quality. National healthcare quality report. Rockville, MD: U.S. Department of Health and Human Services; 2004.
8. Hasenfeld Y, Brock T. Implementation of social policy revisited: a political economy perspective. Administration and Society 1991. 22:451-79.
9. Pressman J, Wildavsky A. Implementation. Berkeley, CA: University of California Press; 1984.
10. Gladwell M. The tipping point. New York: Random House; 1999.
11. Stacey, RD. Complex responsive processes in organizations: learning and knowledge creation. New York: Routledge; 2001.
12. Barabasi AL. Linked: the new science of networks. New York: Plume; 2003.