DESIGNING A MEDICAL DEVICE SURVEILLANCE NETWORK

Section 213 of the FDA Modernization Act of 1997 legislated a change to universal user reporting of device related adverse events. The new law mandates that the current system be replaced by one that is limited to a "subset of user facilities that constitutes a representative profile of user reports" for device related deaths and serious illnesses or injuries. This provides FDA with the opportunity to design and implement a national surveillance network to provide critically important data on medical devices.

In planning for the new system, FDA did research in three areas. Exploratory research in the form of focus groups and site visits was carried out with staff from clinical facilities; a year long, Phase I pilot was conducted; and interviews were held with experts in the areas of surveillance and patient safety. The results of the research effort identified barriers to reporting of adverse events and methods to overcome these barriers in the proposed system.

Now that Phase I is completed, FDA wants to move rapidly to implement a large-scale Phase II study. This study will pilot the initiatives planned for implementation in the national Medical Device Surveillance Network (MeDSuN). The President’s FY 2000 budget includes the request for a $3.2 million increase in Medical Devices to ensure this work can begin quickly, and to enable the agency to make progress toward the new national surveillance program as mandated by Section 213 of FDAMA.

The proposed design of the national program, MeDSuN, which will be implemented by regulation following the large-scale Phase II study, is one aimed at improving the protection of the health and safety of patients, users, and others by: reducing the occurrence of medical device related events; serving as an advanced warning system from the clinical community; and creating a two way communication channel between FDA and the user- facility community. This system will allow the dissemination of data regarding newly emerging device problems to health care professionals, both in the Network and outside it, and to the public. It will allow FDA to apply the knowledge gained from the reported data to the device approval process and to prevention and control programs. The proposed Medical Device Surveillance Network will provide FDA with a setting within which research will be conducted on current device issues.

Design features important to the success of the system are: assuring confidentiality to reporters through the use of a neutral third party; providing meaningful incentives to encourage participation; minimizing the burden of participation; and providing timely feedback to the participants that demonstrates the value of their reporting.

The Safe Medical Devices Act (SMDA) of 1990 legislated mandatory reporting of device-related adverse events by all user facilities in the country. Section 213 of the FDA Modernization Act of 1997 (FDAMA) legislated the replacement of universal user facility reporting by a system that is limited to a "…subset of user facilities that constitutes a representative profile of user reports" for device related deaths and serious illnesses or injuries. This legislation provides the Food and Drug Administration (FDA) with the opportunity to design and implement a national surveillance network, composed of well-trained clinical facilities, to provide high quality data on medical devices in clinical use.

At FDA, the Center for Devices and Radiological Health (CDRH) sees first hand the technological advances in health care as new medical devices are reviewed and cleared for marketing. The increasing complexity of medical technology, perhaps coupled with economic pressures and organizational change within health care institutions, increases the potential for unanticipated and unintended consequences.¹ These changes demand that postmarket surveillance move from its defensive stance to an offensive strategy that includes an understanding of how organizations encounter devices, how problems are perceived and reported, and what characteristics of the system contribute to any event. To the extent that product failures can be identified before patients are injured, FDA can join with manufacturers and health care professionals in creating a safer health care environment.

The Center for Devices and Radiological Health is responsible for ensuring that medical devices on the market are reasonably safe and effective. CDRH pursues its mission by evaluating new devices for marketing, developing and monitoring product quality and performance standards, taking action against firms that violate the law, educating professionals and consumers on the safe use of devices, performing research on device problems, as well as developing guidance documents and standards that augment all of these activities. This mission is also accomplished by ensuring the enforcement of the 1976 Medical Device Amendments to the Federal Food, Drug and Cosmetic Act, the 1990 SMDA, the 1992 Medical Device Amendments, 1992 Mammography Quality Standards Act, and the 1997 FDAMA.

Many important questions about medical devices, such as those concerning user experience, durability, and rare effects may not be answered until after the device has been marketed. Postmarket surveillance, authorized by Sections 519 and 522 of the Federal Food and Drug Cosmetic Act, enhances consumer protection from risks associated with medical device usage that are not foreseen or apparent during the premarket notification and review process. Currently, the FDA monitors postmarket adverse events via both mandatory and voluntary reporting systems.

The history of this program’s activities started in 1973 with a voluntary program entitled the Medical Device Laboratory Product Problem Reporting Program (PRP). The PRP originally relied upon voluntary participation of health care professional organizations as co-sponsors of the PRP. The PRP was FDA’s primary source of information about device problems from 1973 to 1984. In 1984, FDA implemented the Medical Device Reporting regulation (MDR). This regulation required manufacturers and importers of devices to report deaths, serious injuries and malfunctions to the FDA.

The FDA continued to operate both voluntary and mandatory programs after 1984, with the voluntary program seen as the mechanism for the health care community to report their concerns to the Agency. In 1993, the FDA decided to consolidate its various reporting programs by developing a format to be used for both voluntary and mandatory information for all FDA regulated medical products. This program is known as MedWatch and has replaced the former PRP program and its reporting form.

In addition to the changes that the MedWatch Program had on the voluntary CDRH postmarket program, enactment of SMDA in 1990 and the Medical Device Amendments of 1992 made a significant impact upon the mandatory aspects of CDRH’s postmarket surveillance program. SMDA initiated mandatory, universal reporting of adverse events by user facilities. The 1997 FDA Modernization Act made a substantial change to FDA’s postmarket surveillance program, also directly affecting user facilities, by requiring that FDA establish a program that could limit reporting by user facilities to a subset of facilities.

SMDA, effective in November of 1991 for user facilities, requires that user facilities report incidents that reasonably suggest that a medical device has caused or contributed to the death of a patient or to a serious injury or serious illness of a patient. User facilities include hospitals, nursing homes, ambulatory surgical facilities and outpatient diagnostic and treatment facilities. The number of facilities affected was estimated to be over 40,000.²

Specifically, SMDA requires that user facilities report device-related deaths directly to FDA and to manufacturers within ten work days of having knowledge of the incident, and serious injuries or illnesses to the manufacturer within the same timeframe. Reports are submitted on the FDA form 3500A. An additional requirement is for the user facility to provide FDA with quarterly summaries of any reports made to the manufacturers. It was assumed that these summaries from the user facilities would act as an audit tool for assessing reporting by manufacturers under Section 519(a). In accordance with the FDAMA, this regulation has now been changed to require only annual summaries.

Although SMDA made the reporting of adverse events involving serious injury, serious illness, or death mandatory for user facilities, it is a passive system, relying on the health care professional to recognize that an event may be device-related and then to initiate a report through their organization to the manufacturer or FDA. FDA made initial attempts to train the user community in reporting by establishing a network of trainers at the FDA district offices. This program showed some initial success but proved difficult to sustain over time given the size of the reporting universe and lack of resources.

In 1998, manufacturers submitted a total of 980 device-related deaths to FDA compared to 277 deaths submitted by user facilities. Because the law requires that both manufacturers and users submit device-related death reports directly to FDA, comparison of the number of deaths submitted by manufacturers to the number submitted by user facilities is one measure of underreporting.

In addition to concern about underreporting from health care facilities, reports received from users often have not been timely or informative. The average length of time between the occurrence of an event and the receipt of a death report from a user facility at FDA during 1998 was 50 days. Analysts, who are responsible for reviewing the reports, have complained that often there is too little information to be useful without additional follow-up, a complaint that has been echoed by the manufacturers.

To better understand the problems encountered with user reporting, FDA developed a three-phase approach. The first phase was exploratory research directed at identifying the barriers to user facility reporting. The second phase was to use this information to design and conduct a feasibility study; and finally, the third phase was to explore the characteristics of successful surveillance systems and interview experts in the areas of medical error reporting.

The FDAMA required FDA to replace the universal user reporting system by one that limits user reporting to a subset of facilities. Because CDRH had recognized problems with user reporting prior to the legislation, exploration into barriers to user reporting and a pilot study (hereafter referred to as Phase I pilot) were underway at the time FDAMA was passed.

In September of 1996, CODA Inc., a small business, was awarded a contract to conduct a study to evaluate the feasibility and effectiveness of a sentinel reporting system for adverse event reporting of medical device use in user facilities. The study was intended to evaluate the barriers to user reporting and the feasibility of creating an active reporting system in postmarket surveillance to improve the number of reports received and the quality of the clinical data received by FDA.

The exploratory research effort was designed to provide CODA and CDRH with essential information about knowledge, attitudes and practices of user facilities with respect to the SMDA and MedWatch. This information would then be used in the design of the Phase I project. The exploration was carried out through a series of focus groups and site visits in an attempt to learn more about user facilities’ perceptions regarding SMDA reporting requirements and about the existing barriers to reporting medical device adverse events.

Four focus groups with persons responsible for Medical Device Reporting were held initially. Staff who attended the first session were from eight hospitals in the greater Washington/Baltimore area, primarily risk managers and biomedical engineers. A similar session was held in Boston and was attended by staff from about six hospitals in that area. A session was held in North Carolina for staff from eight outpatient diagnostic and treatment facilities in the Raleigh/Durham area, and finally a small session was conducted with three representatives from nursing homes in the Northern Virginia area.

Following the focus group sessions, CODA project staff made site visits to a number of hospitals and nursing homes in Washington and the Raleigh/Durham area to gain a closer view of their internal decision-making processes for reporting of adverse events.

This exploratory research revealed highly variable levels of awareness among user facilities concerning the requirements of the SMDA. Even among those who had a reasonably high level of awareness, interpretations of what was reportable varied greatly. There was also a great deal of variation in the inclination of facilities to submit voluntary reports concerning events in which there was potential for serious illness or injury from a medical device.

The most prominent concerns among the participants in this exploratory effort included the following:

As a result of the initial exploratory research, the Phase I pilot surveillance system was designed to address many of the problems that had been identified with the current system. An important constraint in developing the design was finding a way to accomplish the objectives with as little burden to participating facility staff as possible. Also, Phase I had to be designed so that the procedures, if successful, could be implemented in a much larger Phase II pilot, and finally, at the national level (to be implemented after Phase II, and the publication of a regulation).

The Phase I pilot study, called DeviceNet, was composed of a purposeful sample of hospitals located in the Washington/Baltimore and Raleigh/Durham areas, with one hospital in Boston. Recruitment took place between late May of 1997 and September of 1997. By July of 1997, 38 hospitals and 18 nursing homes had been contacted and 12 hospitals and 5 nursing homes had agreed to participate. Although this number of facilities met the initial recruitment goal, 6 additional hospitals (from the 38) and 1 additional nursing home came into the project by September, for a total of 18 hospitals and 6 nursing homes.

Each participating facility was asked to designate a Study Coordinator; these were generally risk managers for hospitals and directors of nursing for nursing homes. Study Coordinators were asked to either attend an orientation session in Washington D.C. or have DeviceNet staff conduct a 2 to 3 hour orientation session at their facilities.

To address the concerns recognized during the exploratory research, the following features were part of DeviceNet:

Details of the results of the Phase I pilot can be found in the final report for the study that is Appendix A. This discussion will focus on the most significant results, particularly those that clearly affect the design considerations for the national system. The conclusions reached are based on data received in the Phase I pilot study, a formal debriefing held for the Study Coordinators and informal conversations held with study participants over the year of data collection.

Number of Reports: The DeviceNet Phase I pilot project received a total of 315 adverse event reports during the period from October 1, 1997 to September 30, 1998. All of these reports were from hospitals. Table 1 shows a comparison of the number of reports from hospitals during the same time period for SMDA. As evident from the table, the average number of reports per hospital was significantly higher in DeviceNet compared to reporting to FDA under SMDA during the same time period. While it is probable that the hospitals that agreed to participate in Phase I were already biased towards reporting, such an enormous increase suggests that Phase I was successful in creating a more favorable environment for reporting.

Type of Reports: When enrolling into the DeviceNet Phase I pilot, facilities were encouraged to submit "voluntary" reports for incidents that the staff perceived as having the potential for patient harm although none actually occurred. Two CODA staff, who were very familiar with the regulatory guidelines after working on the project for over a year, attempted to classify the reports received as voluntary or mandatory according to the SMDA regulation. Looking again at Table 1, 56% of the total number of reports were classified as voluntary (or malfunctions) compared to 20% of the reports received at FDA during the same time for SMDA.

Significance of Reports: Of greater importance than the number of the reports is the significance of the reports in terms of informing FDA of a potential or actual device hazard. Two experienced nurse analysts categorized the 315 reports received during Phase I on an "urgency scale." Table 2 shows that more than 1/3 of the reports, 113 out of 315, were considered at least "somewhat urgent" by the nurse analysts; however, only 19 of these 113 reports were clearly mandatory reports under SMDA. Conversely, many of the mandatory reports submitted during the Phase I pilot were classified as requiring routine monitoring. Another important finding displayed in Table 2 is that the distinction between mandatory and voluntary could not clearly be made for one third of the reports.

Nursing Homes: Despite additional training sessions at two of the nursing homes, none of the nursing homes submitted adverse event reports. Although there is considerable speculation on factors that contribute to this finding, additional research is needed to fully explore the problem and identify solutions.

To learn more about the reactions to different aspects of participation in the DeviceNet project, a debriefing was held with the Study Coordinators after the close of the data collection period. Since nursing homes did not submit reports, it was decided to invite only participating staff from the hospitals.

In general, the Study Coordinators were very positive about their DeviceNet experience. Most found the assistance offered by the contractor to be of value. Representatives from two facilities explained that participation had affected internal relationships within their facilities, such as an increased rapport and interaction between the risk management and biomedical engineering offices.

Most of the Study Coordinators reported using the video provided to train staff in their facilities on recognizing and reporting device events. One Coordinator expressed some frustration because she felt the video prompted too much reporting from staff, and several felt that the 12-minute video was still too long.

The DeviceNet Newsletter received a very favorable review. Study Coordinators used the newsletter in different ways. Several indicated that the summary of event reports received during the Phase I pilot was the item they turned to first to compare the reports they had submitted to what was submitted by other facilities. The Coordinators reported that the articles on medical devices were helpful and interesting and they shared these articles with the appropriate staff.

During the debriefing, at least three of the Study Coordinators expressed their desire to have the study run for at least another year. This seemed to be either because they liked the assistance in completing the forms or because the process of educating their staff was gradual and still ongoing at the end of the project year.

A related theme was the need for educating staff on device event recognition and reporting was constant, principally because of staff turnover. Reminders are helpful to keep health care workers constantly mindful of the need for reporting adverse medical device events and the mechanisms for reporting.

Initially, discussions were held with experts involved in the design, implementation and monitoring of surveillance systems. These included: the Center for Disease Control’s (CDC) National Nosocomial Infections Surveillance System (NISS); the FoodNet System, jointly administered by FDA, CDC and the United States Department of Agriculture (USDA); the National Cancer Institute’s (NCI) Surveillance, Epidemiology and End Results (SEER) system; The Consumer Product Safety Commission’s (CPSC) National Electronic Injury Surveillance System (NEISS); the Vaccination Adverse Event Reporting System (VAERS), jointly administered by CDC and FDA; the Federal Aviation Administration’s (FAA) Aviation Safety Reporting System (ASRA), administered by the National Aeronautics and Space Administration (NASA); and The United States Pharmacopoeia’s (USP) MedMarx system for reporting medication errors. Interviews with the experts focused on: who is the population under surveillance; who are the reporters for the system; how does the system protect the data that flows into the system; what is the burden to the reporter and how is the reporting burden minimized; is the system mandatory or voluntary; and what type of feedback or incentives are given to the reporters.

A second group of interviews was held with representatives from various stakeholders, such as the American Hospital Association, the American Society of Risk Managers, and the American Society of Healthcare Engineers. These groups were queried as to what features of an adverse reporting surveillance system could be incorporated to make reporting less burdensome and reduce liability concerns, and what incentives could be used to encourage participation.

Several statistical experts were interviewed to give advice on how the sample of facilities should be selected and what would be the relative merits of different strategies. Because of issues related to concern about liability, several people were consulted with expertise in these legal areas.

Finally, contact was made with individuals considered to be experts in understanding issues related to safety and medical error. The ever increasing technological complexity and organizational changes in health care have caused some to speculate that "health care in 1998 stands where nuclear power stood at the end of 1979," facing growing demands for enhanced patient safety.³ Much of what has been learned about understanding accidents, or misadventures, in organizational systems is relevant and important in considering the design of the device surveillance network. For example, it is important to be informed of "close calls" and the steps taken by a health care professional to prevent a patient injury by "recovering" from an near incident or doing a "workaround." Only by having this information, will FDA be able to identify the potential for error and be in a position to work with the health care community to prevent patient injury.

The goal of the Medical Device Surveillance Network (MeDSuN) is to improve the protection of the health and safety of patients, users and others by reducing the likelihood of the occurrence of medical device related adverse events and, if they do occur, reducing the likelihood that they will be repeated.

To accomplish this overall goal, the surveillance system needs to achieve the following more specific objectives:

Ideally, the system would provide a laboratory for studies of device use and how errors occur, and would also facilitate epidemiological studies. In this way, it could lead to a better understanding of the causes of adverse medical device events and point to ways of minimizing their occurrence and lessening their impact.

The Medical Device Surveillance Network, MeDSuN, would also serve as an advance warning system from the clinical community that would allow FDA to become aware of developing device problems and to prevent resulting injuries, or at least lessen the chances of such injuries recurring. It should be a vehicle for efficiently and effectively collecting and disseminating data about adverse medical device events. Its success depends on finding a way to encourage a high level flow of information regarding adverse medical device events from facilities and health care professionals.

To shed light on the underlying problems of medical device errors and their impact, MeDSuN should also be a setting within which research studies can be conducted, either by FDA or by others. These studies could be epidemiological in nature, such as attempts to estimate incidence or prevalence of certain types of adverse medical device events or vehicles to address some of the most elusive issues (i.e. biomaterial sensitivity) concerning the use of certain medical devices.

Research efforts could also include surveys designed to complement the surveillance system in a variety of ways. For example, one could use surveys to study such phenomena as underreporting of adverse medical device events – the extent and causes of underreporting, circumstances associated with underreporting, and characteristics of events that are most likely to go unreported. Other issues, important to the FDA mission, include the investigation of how errors are perceived within the organization, what information is viewed as important in investigating these events, and how organizations do or do not learn from errors.

Finally, an important by-product of MeDSuN should be the creation of a two-way channel of communication between FDA and the user facility community, via the contractor. This system will provide a means for gathering or providing fast, effective feedback from and to that community (FDA retains it’s direct channel of contact to all user-facilities in the United States via Health Advisories and Safety Alerts). By having a network in place, FDA could use this system to quickly gain input from a representative sample of user facilities, and facilities could use the network to seek information from other facilities or from FDA (either indirectly via the contractor, or directly, if the facility chooses to do so).

Information from the MeDSuN will provide an important piece of the postmarket picture. Mandatory reporting of adverse events from manufacturers, postmarket studies under Section 522 and field inspections are all important components of postmarket surveillance. However, none of these mechanisms provide FDA with a clear vision of how devices are used or misused in the clinical setting and what steps can be taken to systematically decrease patient injury related to device use.

The challenge in designing the national Medical Device Surveillance Network (MeDSuN) is to combine the results of:

Based on the results of Phase I and the advice of the experts, FDA has developed a plan for the National MeDSuN program that will be implemented via regulation. This plan is described in detail in the next few pages.

However, prior to issuing a Proposed Rule which would incorporate the current proposed national plan (outlined in detail in the following pages) FDA plans first to implement a large-scale Phase II pilot study. Even with the considerable research that has been done to design the best possible system, a number of features of the proposed system will still benefit from additional testing. If refinements need to be made, it is both easier and less expensive to make the changes at the initial stages, before the system is up and running across the nation.

Phase II will follow the current proposed plan for the final, national implementation (description beginning on page 15) with the following exceptions.

Given the understanding that Phase II will take place before publishing a Proposed Rule which describes the following plan for the national MeDSuN program, and given that this national plan will be impacted by the lessons learned in Phase II, the features of the current proposed national system include the following:

The main purpose of the device surveillance network will be the identification of emerging hazards rather than making precise statistical estimates of occurrence of adverse medical device events. However, using statistical sampling procedures to develop a probability-based sample design provides a number of potential advantages. First, the more rigorous the sample design, the more confidence FDA and other researchers can have that reports from MeDSuN facilities are representative of reports that would be received if all user facilities could be included. Second, a Network based on sound statistical design would increase one’s ability to monitor trends over time and to conduct related research studies. Third, such a design will lend itself to conducting correlated research studies using Network facilities; and finally, to the extent that participation is a burden, it avoids any appearance of unfairness in the selection process.

The simplest approach to achieving a sample based on strict probability procedures would be to draw a random sample from the universe of all user facilities. This would produce a sample that would distribute itself across the country more or less the way that the health care facilities serving the population are distributed. The resulting sample points would be clustered in areas of high population density but would have a fairly wide scatter across the remainder of the country. Although this sample design allows for some clustering, the distribution of facilities across the country can still make running the network more expensive because of scatter.

A more clustered sample could be achieved through the use of a multi-stage sample design. For example, one could cluster the sample by Census region or state or county. A certain number of first stage units (regions, states or counties) would then be drawn, and then a sample of health care facilities within these units would be selected as secondary sampling units.

In the multi-stage design, the second stage units would be stratified to assure representativeness. For example, separate strata within each region or county might be based on bed size, ownership characteristics, teaching vs. nonteaching and so forth. Facilities would then be drawn within each strata to assure that the sample contained facilities that represented each of these dimensions. The multi-stage-clustered sample is a less costly system to run because of the additional clustering within first stage units.

To implement and run MeDSuN, FDA will select, through the competitive procurement process, a contractor with experience in conducting national health care studies. The contractor will need to demonstrate experience in working with health care facilities and have access to staff with appropriate technical expertise. The contractor will be asked to provide a sample design plan, develop study materials, train facility staff, and receive, edit and do follow-up on adverse event reports. The contractor will also be responsible, with FDA input, for providing timely feedback to the participating facilities. Quality control is an essential element in any data collection effort and this will also be carried out by the contractor, along with organizing regional meetings of the participating facilities.

As shown earlier in this report, under SMDA, about 70% of the adverse event reports received from user facilities come from hospitals. In the DeviceNet Phase I pilot, despite additional training sessions held on site at two facilities, none of the six nursing homes submitted reports. Although the Phase I researchers are able to speculate on reasons for this, it was felt that this issue deserves separate exploration and the resources for this have not yet been available.

An important decision to be made is whether to initially include only hospitals in the Network, all types of facilities included under SMDA, or just certain types. Evidence to date suggests that reporting of adverse events related to medical devices from facilities other than hospitals is clearly more problematic and to date, there is little information on what additional barriers are perceived by these organizations. Without better information, it may not be worthwhile in terms of costs and benefits to include other types of facilities in a surveillance network, at least in the initial stages.

There are a number of options in reaching out to other types of facilities to obtain information, such as promoting a strong voluntary program that reaches the staff of these facilities, collaboration with other agencies to gather and analyze data, such as HCFA, and finally, separate research efforts that target these types of facilities. For example, some have speculated that reporting in nursing homes is difficult to achieve because the staff who do the day-to-day care are often not licensed staff, English may not be their first language, and there is high staff turnover. There are a number of ways these and other theories could be explored, including conducting focus groups with the staff who do the patient care.

Because of the current managed care environment, it is also likely that a number of large facilities selected into the Network will have within their organization some other types of clinical units such as free standing outpatient clinics, short term nursing home units, or separate dialysis units. These units would be part of the adverse event reporting structure of the participating facility. The reporting from different types of clinical units within the Network will provide additional information on how best to learn about device problems within these specialty units and allow FDA to evaluate the need for more data.

Exploratory research, both with surveillance experts and the focus groups composed of clinical staff, in addition to the experience obtained in the Phase I pilot program, suggest that the extent to which reporters believe that data submitted are secure, the number as well as the quality of reports increases. Reporters must believe that neither their organizations nor their jobs will be threatened by reporting an adverse event. Many of the expert consultants believe strongly that full disclosure of adverse events, without blame, leads to a reduction in medical errors and that reducing medical error reduces cost and risk to the facility. This has been demonstrated in the Federal Aviation Administration’s successful Aviation Safety Reporting System and is the principle underlying the development of the Veterans Administration’s new "close call" reporting system.

While there seems to be a trend within the provider community toward creating "blameless" organizational cultures in which full disclosure of errors is encouraged in the long-range interest of patient safety and cost effectiveness, facilities that are moving in this direction still constitute the exception. In spite of assurances of data protection, it appears that most medical facilities are very reluctant to allow information on medical error to leave the institution, particularly to a federal regulatory agency. And, while there seems to be a general understanding in the medical community that FDA is also a public health agency and has little regulatory power in the daily function of the health care arena, within these institutions there is little differentiation between FDA and other regulatory agencies that may carry a much heavier hammer.

Health care professionals and researchers in the field of patient safety report that the disincentive to admit to errors created by liability concerns looms quite large. This is particularly true of device-related problems where an error in use by the health care professional is usually in question. One method of offsetting these concerns is to design a reporting system in which the reports are transmitted with no identification of the individual reporter or the facility. However, it is likely that such a system would substantially reduce the value of the information received, because being able to obtain additional details about incidents through follow-up has proven to be key in understanding the event. Under SMDA, both the FDA analysts and the manufacturers report that the narrative description of the event is the most valuable part of the report, but both report that they frequently need to contact the reporter in order to obtain important additional information before they can make a judgment as to what really happened.

It seems that the greatest concern that facilities have regarding their possible risks in reporting to FDA is about leakage of information during a discovery process related to a lawsuit. A secondary concern is the possibility of governmental disciplinary action against user facilities. This concern regarding reporting to a regulatory agency that is perceived as threatening can be diminished by involving one or more third-party organizations that have credibility within the health care community and pose no perceived threat. This third party "cushion" between the reporter and the regulatory agency mimics both the DeviceNet Phase I pilot and the successful Aviation Safety Reporting System where NASA acts as the third party intermediary.

The use of a third party provides an opportunity for reporters to submit reports with identification, but not to the regulatory organization. Positioning a third party between the health care reporting facility and FDA is one way of reducing disincentives related to concern about regulatory actions. In the design scenario proposed, the third party will be a contractor. It is proposed that the data be submitted to the third party , the contractor, with identifying information. The report will be evaluated for completeness and during the time in which the reporter’s identity is known, the third party will gather additional information as needed. Once the report is considered complete, the identification data will be stripped before the report is entered into the study database and sent to FDA for a more complete analysis. The identifying information will remain available to the third party until FDA has seen the report. If the user facility requests, the de-identified report will also be sent to the manufacturer at this point. Again, if the manufacturer requests additional information, the third party will contact the reporter. If neither FDA nor the manufacturer has additional information requests, the identifying information will be deleted at the third party and data forwarded to the FDA.

Because the primary significance for FDA and the manufacturers is the ability to obtain additional details about adverse events, FDA believes the use of a third party provides the ability to obtain this important information while reducing the user facilities’ concerns about confidentiality.

FDA can design a surveillance system that gives additional security to reporters, but FDA cannot directly change the cultural environment within health care organizations. However, eventually, the Surveillance Network will demonstrate, as the Aviation Surveillance Reporting System has, that a blameless environment is a major step in reducing error and injury. In the short term, FDA will vigorously promote voluntary and/or anonymous reporting through the Network and the FDA MedWatch system.

Many believe that to the extent that the user community perceives the network as another burden imposed by the Federal government, participation is likely to become perfunctory and unproductive. One only has to compare the performance of the institutions voluntarily participating in the DeviceNet Phase I pilot to institutions mandatorily reporting under SMDA to see the impact of what appears to be the voluntary nature of participation, coupled with education and feedback which may have been equally, if not more important. Other successful voluntary reporting systems, such as the CDC’s National Nosocomial Infection Surveillance System and the FAA’s Aviation Surveillance Reporting System, add credence to this theory.

This issue is somewhat complicated by the Phase I finding that a significant number of reports classified under the regulation as voluntary, were in fact, seen as very important to the agency. In general, these reports were ones with information concerning problems that had not resulted in a death or serious injury, but had the potential for injury. The value of receiving this information when the Agency may be able to prevent injury is apparent. An added advantage of receiving reports of incidents that have not resulted in a patient injury is that usually there are no liability concerns; the reporters, therefore, are more likely to share the details of the incident with FDA (via the contractor) and with the manufacturer.

Evidence to date suggests that universal, mandatory reporting for user facilities, both in terms of facilities understanding what should be reported and in terms of compliance with reporting requirements, has not been very successful. The current legislation continues the mandatory aspect of SMDA reporting, but provides for the development of reporting by only a subset of facilities.

FDA is proposing the use of substantial encouragement and well-directed incentives to encourage facilities to participate actively in the voluntary aspect of the system, the reporting of incidents that are considered "close calls" or "near misses." These are incidents that have the potential to harm the patient and probably could be prevented in the future. The voluntary reports would not be subject to the time limit of 10 days that exits under the SMDA mandatory reporting. This would allow health care facilities more time to follow-up and investigate incidents. Voluntary reporting with the health care facilities would be coupled with "grass roots" efforts at the local level to increase voluntary reporting by health care professionals to the FDA MedWatch program.

Evidence from the DeviceNet Phase I pilot and investigations of other successful surveillance systems, suggests that timely feedback to the participants is an extremely important motivating factor in surveillance reporting. Feedback demonstrates the value of reporting and participating in the Network. Each report will be acknowledged promptly so the reporting facility knows it was received and is being reviewed.

A regular newsletter is a good mechanism for reporting on FDA activities and concerns along with providing health care facilities with information on the status and successes of the Network. It is important that FDA, health care professionals, and device manufacturers all contribute to the newsletter.

An equally important type of feedback will be data and software useful for quality improvement efforts. In the Center for Disease Control and Prevention’s National Nosocomial Infection Surveillance System, the software and data provided to the participants is a significant motivator for joining the surveillance system. A database accessible on the Internet that presents tabular data that the Network facilities could use for profiling their facility’s performance or searching to see problems with specific devices would both offer an incentive to participate and possibly result in real improvements in quality care for patients.

A third type of feedback will be regional meetings of Network participants. These meetings should take place on at least a semi-annual basis. The meetings would update participants on study procedures, give results of the reporting to date, provide information on practices or procedures that may increase safety, and provide other information that is considered important to the participants.

Education and training support will also be provided to the participating facilities. Risk managers and others concerned with quality improvement and minimizing risk are often faced with the task of developing in-house staff education and training programs concerning reporting on adverse events. Such programs are typically designed for use during staff orientation sessions and in in-service training session. The materials developed for the DeviceNet Phase I pilot were well received. Providing well-developed training materials not only helps to reduce the burden for hospital staff; it also allows the opportunity to standardize the information received by the health care professional.

An important objective of the network design will be to make reporting as simple and easy as possible. Facilities will be offered both a flexible approach to reporting and technical assistance for all reports as needed. Flexibility includes allowing reporters to submit data electronically, by FAX, by phone (i.e. "hot line") or by mail. Toll free FAX and phone numbers provided freely to all reporters make reporting and asking questions easy. An Internet Web site for reporting, with information about the variety of ways to report and with instructions for completing the computerized form, will be useful to many reporters.

With the use of telephone, email, and reporting assistance, the burden to participating facilities will be minimized. Technical assistance such as help in determining what events or problems are reportable and providing specific information that can be difficult and time-consuming for the facilities to obtain (such as manufacturer’s addresses) is also helpful.

Health care facilities participating in the Network will also be relieved of the responsibility to code the data submitted to the contractor. The coding of data will be centralized at the third party sites to relieve burden and to better standardize the data reduction.

Several methods will be used to validate the monitoring system. For example, conducting prospective epidemiological studies, monitoring the number of adverse events over time (the number of events should decrease in specific areas as users become aware of medical use errors and as device design is improved), and comparing the reports received under MeDSuN to the Complaint Report file kept by the device manufacturer are planned methods of validating the system.

It is critical that device manufacturers continue to receive feedback about problems with their devices. As mentioned above, mandatory reporting to manufacturers by user facilities under SMDA will remain part of the MeDSuN system; health care facilities in the Network will still be required by law to submit reports of deaths and serious injuries to the device manufacturers. Given the low level of reporting by user facilities under the current universal system, and recalling the results of the Phase I study, even with fewer facilities participating, it is likely that manufacturers will be receiving more reports under MeDSuN and reports of better quality.

The overall goal of the Medical Device Surveillance Network, to decrease patient injuries from medical products and improve patient outcomes, can best be accomplished by designing and operating the best system possible. This is best done by including the insight, recommendations, and collaboration of many different people from a number of different organizations. For this reason, FDA recommends that an outside Advisory Board be established to assist in providing recommendations and guidance. The contractor will establish the Advisory Board with recommendations from FDA. Representatives to the Advisory Board might include industry or trade groups, academia, the American Hospital Association, the National Patient Safety Foundation, and other Federal Agencies such as the Centers for Disease Control and Agency for Health Care Policy Research (AHCRP). The Advisory Board will meet several times a year to review the status of the project, and make recommendations to FDA.

Phase II funding will initially be needed for selecting the sample of facilities for the Network and recruiting the facilities, training facilities on the study protocol and recognizing and reporting adverse events, developing materials for education and continual feedback, organizing and conducting regional meetings and meetings of the Advisory board, quality control activities, and implementing hardware/software. The contractor responsibilities will also include providing technical expertise for reviewing adverse event reports and conducting follow-up and for coding of reports. The projected budget estimates a cost of 3.2 million dollars for FY 2000. The President’s FY 2000 budget requests this amount. These resources will provide funding for the contractor to implement Phase II in three areas of the country, assuming about 50 clinical facilities in each. The projected costs include two FTE positions needed within FDA to manage Phase II and the ongoing planning for the national Network activities.

The continuing changes in health care in the United States make it difficult to predict with certainty how receptive health care facilities will be to Network participation. However, as the "culture of blame" gives way to a culture of safety, the value of identifying problems and errors in medical care will continue to become apparent as it has already in other systems such as the Aviation Surveillance Reporting System. The benefits of reporting problems that can be identified for additional research and resolution will far outweigh the effort and perceived risks of reporting. In this scenario, participation in the Network will be desirable and may well result in user facilities requesting participation, as is the case in CDC’s National Nosocomial Infection Surveillance System.

# Hospitals Participating	5,000-6,000	18
Total User Reports Received	4,400	315
% of Reports from Hospitals	75%	100%
% of Reports Voluntary	20%	56%

¹ Cook, Richard I., Woods, David D., Miller, Charlotte. A Tale of Two Stories: Contrasting Views of Patient Safety. Report from a workshop; Assembling the Scientific Basis for Progress on Patient Safety, National Health Care Safety Council of the National Patient Safety Foundation at the American Medical Association, 1998.

² Website for the National Information Center for Health Services Administration: http://www.nichsa.org.

³ Cook, Richard I., Woods, David D., Miller, Charlotte. A Tale of Two Stories: Contrasting Views of Patient Safety. Report from a workshop; Assembling the Scientific Basis for Progress on Patient Safety, National Health Care Safety Council of the National Patient Safety Foundation at the American Medical Association, 1998.

REGULATORY STATUS	AT LEAST SOMEWHAT URGENT	ROUTINE MONITORING	NOT SIGNIFICANT OR INSUFFICIENT INFO.	TOTAL
Mandatory Reports	19	23	3	45
Voluntary Reports	52	72	51	175
Status Not Clear	42	46	7	95
Total	113	141	61	315
	(36%)	(45%)	(19%)	(100%)

DESIGNING A MEDICAL DEVICE SURVEILLANCE NETWORK Report to Congress

DESIGNING A MEDICAL DEVICE SURVEILLANCE NETWORK
Report to Congress