Chapter 23. Pre-Anesthesia Checklists To Improve Patient Safety

Andrew D. Auerbach, M.D., M.P.H.
University of California, San Francisco School of Medicine

Harvey J. Murff, M.D.
Harvard Medical School

Salim D. Islam, M.D.
University of California, San Francisco School of Medicine

Background

No matter how rote the task or how vigilant the anesthesiologist, "slips" and other errors represent expected aspects of human performance.¹ Evaluation and subsequent improvement of standard checkout procedures promises to increase patient safety in the perioperative period by removing more of the "human factors" so often implicated in anesthesia adverse events.² The use of pre-flight checklists has been considered a key method in improving airline safety, largely due to the regular systematizing of complex procedures, and improvement of team dynamics through authority-neutral tasks. A checklist system has been proposed as part of routine pre-anesthesia care, with the American Society of Anesthesiologists and the US Food and Drug Administration (FDA) issuing general guidelines supporting checklists in 1986.³ Subsequently, anesthesia professional societies in Great Britain and Europe adopted similar standards.⁴

Practice Description

In 1987, the FDA published the "Anesthesia Apparatus Checkout Recommendations" in the Federal Register (February, 1987). This "checkout list" provides practitioners with a standardized approach to checking anesthetic equipment prior to its use in order to ensure that the delivery system is correctly connected, adjusted, and functional. The original checklist included 24 specific processes to be performed as an initial checkout at the beginning of each day; 11 are performed between cases and after initial equipment evaluation.⁵ Many clinicians regarded these protocols as too long and complex for routine use. Parties involved in revising the recommendations agreed that the average clinician should be able to check an anesthesia machine in 5 minutes or less (not possible with the 1987 recommendations). The revised checklist included only 14 major processes, 9 of which can be omitted or substantially abbreviated when the anesthesia provider uses the same equipment in successive cases.⁶ The revised recommendations are available online.⁷

Prevalence and Severity of the Target Safety Problem

The earliest observational studies of mishaps within anesthesia found that equipment failure was the cause of 14% of anesthesia critical incidents.⁸ Subsequent studies reduced this estimate considerably. Although equipment failure is now implicated in only 4% of anesthesia adverse events, 22% are related to failing to check equipment adequately.² Equipment failures can result in delivery of hypoxic gas mixtures or excessive doses of inhalational agent, or hypoventilation due to ventilator failure. These situations can be catastrophic if unrecognized, and even if recognized may result in significant morbidity (e.g., delayed extubation, stroke, or myocardial infarction).^2,9

Opportunities for Impact

The FDA checkout list is considered a template which local users are "encouraged to modify to accommodate differences in equipment and variations in local clinical practice." Estimating the frequency with which it or other checklists are used (in modified or unmodified forms) in anesthesia practice would be speculative,⁵ but a survey of 4 states suggests that usage is minimal.³ There are 41.5 million inpatient procedures each year requiring anesthesia in the US. Probably about half of these involve general anesthesia. Therefore, although the frequency of equipment failure is low, any improvement in safety from anesthesia preoperative checklists could have substantial impact.

Study Designs

Using a structured MEDLINE search, we identified 15 articles that discussed anesthesia checklists. Of these, only 2 studies came close to meeting our inclusion criteria (Chapter 3).^3,10 Because no other studies could be found, we abstracted these studies and review them here (Table 23.1).

The first investigation evaluated the ability of participating providers to detect standardized equipment faults using their own checking methods compared with the FDA checkout list.³ Although the study involved a prospective design, it does not fit neatly into our classification system because the participants (anesthesiology residents and practitioners) served as their own controls. The second study¹⁰ involved the revised FDA checkout list, and its design was modeled on the previous study.

Study Outcomes

The first study's outcome of interest was the detection of 4 standardized equipment faults created by the investigators (Level 2).³ The second study used similarly designed outcomes (simulated equipment faults), but defined detection of at least 50% of these faults as the primary outcome (Level 2).¹⁰ Neither study directly connected the use of pre-anesthesia checklists to patient outcomes, although inadequate preanesthesia equipment checks have been implicated in adverse events related to equipment failures, as discussed above.²

Evidence for Effectiveness of the Practice

March et al³ found that the FDA checklist was more likely to detect faults with the nitrous oxide system (65% vs. 43%, p<0.001), but that the FDA checklist was no better than individual practitioners' checklists in detecting the 7 other pre-set faults. No method detected 100% of faults, but those physicians with more clinical practice and those with better knowledge of the FDA checklist were more effective at detecting equipment faults. In the study of the revised FDA checklist,⁹ approximately half of the participants failed to detect at least half of the faults using both their own methods and the FDA checkout list (no statistically significant difference).

Both of these studies contain important methodologic flaws. The participants knew they were involved in a study assessing their ability to detect machine faults, and so undoubtedly approached this task with increased sensitivity. Moreover, the participants' own methods may have been quite similar to the use of the FDA checklist. Both of these problems bias these studies against finding a difference between checklist and controls. Importantly though, all methods performed poorly in both studies. Thus, even if the FDA checkout list is in fact superior to anesthesia providers' own methods, the observed utility of this practice still is likely to be quite low.

Potential for Harm

We were unable to find literature that implicated checklists as causes of adverse events. There exists a theoretical possibility of delays due to complex checklists, but these have not been borne out in any published studies. There has also been concern raised about compulsory checklist procedures as an unnecessary duplication of work already performed by operating room technical personnel.

Costs and Implementation

The FDA checklist, or local variants thereof, is widely implemented and inexpensive. Although several authors have mentioned that checkout processes are not used in 100% of cases, this does not seem to reflect problems with checklists themselves.^4,12,13 The work of March and Crowley³ suggests that checklists training may be critically important. However, the most important barrier to implementation is the heterogeneity of anesthesia delivery devices themselves, which makes creation of a single, effective, broadly generalizable checklist difficult if not impossible.

Comment

Given its face validity and the theoretical connection between anesthesia checklists and those used so effectively in aviation, preanesthesia checklists represent plausible safety tools. However, the reliability of modern anesthesia machines has reduced the frequency of mechanical failure to such a degree that adverse outcomes due to anesthesia machine failure are exceedingly rare. The checklists examined in the cited studies only examine the integrity of the anesthesia machine and ventilatory monitors, not cardiovascular monitors, airway equipment, intravenous apparatus, infusion pumps, or medications. Standardized checklists for these other critical components of anesthesia care do not exist in the literature. This may explain the paucity of literature exploring the use of checklists and their real effects on patient outcomes. Furthermore, the inability of anesthesiologists to detect preset faults in the cited studies may simply reflect the infrequency with which such faults are encountered in modern anesthesia practice.

The face validity of checklists and the difficulty of "probing" their value in clinical studies make additional "proof of concept" studies unlikely. Although future investigations could not ethically study the anesthesia machine checklist per se, they could seek to determine more effective methods for its implementation, or could develop additional checklists with a broader scope. The little evidence we have been able to uncover suggests that, like the use of voluntary guidelines elsewhere in medicine (Chapter 51), checklists are not used uniformly. This may result from a sense that these checks are low-yield, redundant, onerous, or all of the above.

The need for effective checkout procedures is likely to grow as the complexity of anesthesia equipment increases. This will increase the need to make checklists more sensitive and specific in detecting faults, while improving usability. These worthy goals may then serve as templates for other technologically dependent medical specialties, such as cardiac electrophysiology, and interventional radiology.

Table 23.1. Evaluations of the FDA "checkout list" for preoperative anesthesia equipment assessment

Study	Study Design, Outcomes	Results
March, 19913: Exposure of a total of 188 anesthesiology residents and practitioners from multiple sites to a "mobile anesthesia study center" with 2 anesthesia machines pre-set to one of two different "fault sets." Each of these sets included 4 independent machine faults. Practitioners were instructed to use their own checkout methods to assess a machine with one of the "fault sets." After completion of this assessment, the machine was adjusted to display the other fault set, and the participant invited to use the FDA checkout list in the second assessment.	Mixture of Levels 2 & 3, Level 2	Participants detected an average of only 1 in 4 machine faults. A statistically significant improvement in the fault detection rate with use of the FDA checkout compared with the participants' individual methods was observed for only 1 of the 4 fault types (involving the oxygen/nitrous oxide ratio).
Manley, 19969: Similar study to above, but involving only 22 participants (a mixture of anesthesiologist, nurse anesthetists, and senior nurse anesthetist students) from only one site.	As above	For both of the fault sets, approximately half of the participants detected fewer than 50% of the 4 target faults using their own individual methods. The detection rates using the revised FDA checkout list did not differ significantly (p=0.48) from these results.

References

1. Reason JT. Human Error New York: Cambridge Univ Press; 1990.

2. Cooper JB, Newbower RS, Kitz RJ. An analysis of major errors and equipment failures in anesthesia management: considerations for prevention and detection. Anesthesiology 1984;60:34-42.

3. March MG, Crowley JJ. An evaluation of anesthesiologists' present checkout methods and the validity of the FDA checklist. Anesthesiology 1991;75:724-729.

4. Charlton JE. Checklists and patient safety. Anaesthesia 1990;45:425-426.

5. Walker JS, Wilson M. Clinical risk management in anaesthesia. Qual Health Care 1995;4:115-121.

6. Anesthesia Apparatus Checkout Recommendations, 1993. Fed Regist 1994;59:35373-74.

7. U.S. Food and Drug Adminstration. Anesthesia apparatus checkout recommendations. Available at: http://www.fda.gov/cdrh/humfac/anesckot.pdf. Accessed June 10, 2001.

8. Cooper JB, Newbower RS, Long CD, McPeek B. Preventable anesthesia mishaps: a study of human factors. Anesthesiology 1978;49:399-406.

9. Spooner RB, Kirby RR. Equipment-related anesthetic incidents. Int Anesthesiol Clin 1984;22:133-147.

10. Manley R, Cuddeford JD. An assessment of the effectiveness of the revised FDA checklist. AANA J 1996;64:277-282.

11. Degani A, Wiener EL. Cockpit checklists - concepts, design, and use. Human Factors 1993;35:345-359.

12. Cooper JB, Cullen DJ, Eichhorn JH, Philip JH, Holzman RS. Administrative guidelines for response to an adverse anesthesia event. The Risk Management Committee of the Harvard Medical School's Department of Anaesthesia. J Clin Anesth 1993;5:79-84.

13. Currie M. A prospective survey of anaesthetic critical events in a teaching hospital. Anaesth Intensive Care 1989;17:403-411.

Return to Contents
Proceed to Next Chapter