Advances in Patient Safety: From Research to Implementation Volume 3. Implementation IssuesLessons Learned from the Evolution of Mandatory Adverse Event Reporting Systems
Ellen Flink, C. Lynn Chevalier, Angelo Ruperto, Peg Dameron, Frederick J. Heigel, Ruth Leslie, Janet Mannion, Robert J. Panzer
AbstractNew York State has had a mandatory incident reporting system in place since 1985. The current system, the New York Patient Occurrence Reporting and Tracking System (NYPORTS), was implemented in 1998 pursuant to New York State Public Health Law Section 2805-l, Incident Reporting. NYPORTS is a secure Web-based system that simplifies reporting, coordinates with other reporting systems, and allows hospitals to obtain feedback on their own reporting patterns. The authors review the evolution and implementation of NYPORTS and its predecessors, the Hospital Incident Reporting System and the Patient Event Tracking System. Discussion and data comparisons are made between the Joint Commission on Accreditation of Healthcare Organizations' voluntary sentinel event reporting system and NYPORTS. Critical elements for success of a mandatory incident reporting system include collaborative system design; basing the system on statute, with clear definitions and objective reporting criteria; providing meaningful data that can be analyzed and disseminated for improving patient safety; and adequate resources to maintain the system. Innovative program features may be of interest to other States implementing reporting systems.
The evolution of an adverse event reporting system is complex, creates many challenges, and provides several lessons learned. This paper focuses on the New York State Patient Occurrence Reporting and Tracking System (NYPORTS), describing the history, evolution, and implementation of the system over a 19-year time span and three distinct system iterations. Critical characteristics of NYPORTS and its use in quality improvement will be described. Lessons learned from the system's evolution and implementation will be shared.
BackgroundIn 1985, The New York State Department of Health (NYSDOH) developed its first mandatory adverse event reporting system. The program required reporting of preventable adverse events and a description of steps taken to address underlying deficiencies in hospital systems and/or practitioner training and capabilities. Incident reporting in New York State emerged statutorily in 1986 as part of the Malpractice Prevention Program. 1 This program was created by the legislature to capture preventable events caused by human or mechanical error resulting in patient harm. The statute requires hospitals to collect and report information on negative health outcomes and incidents, placing the already-existing reporting system into the context of malpractice prevention. Incident reporting expanded to include patient responses to illness and treatment, clouding the issue of "preventability." This led to the initial development and standardization of definitions to clarify which events were reportable and to minimize variation in the events reported.
There were several redesigns of the system, culminating in the current NYPORTS system. The evolution of incident reporting in New York State led to the inclusion of several critical characteristics and lessons learned that can be shared with other States developing or revising their own incident reporting systems.
The pervasive focus on medical errors in the U.S. health care system gained momentum in 1991, when a landmark study documented the type and extent of medical errors in 30,000 hospital discharges in New York State. 2 This momentum continued to build through the 1990s with several published reports on medical error fatalities. The Institute of Medicine (IOM) report, To Err Is Human: Building a Safer Health System, 3 brought widespread public attention to medical errors and was an impetus in making patient safety a national priority. The report also highlighted the importance of creating mandatory adverse event reporting systems as a mechanism to learn from these events and prevent similar events in the future. NYPORTS was implemented prior to the IOM report, illustrating the seriousness with which NYSDOH approaches improving patient safety.
As a result of the IOM report, several actions occurred to bring adverse event/medical error reporting systems into the forefront of public policy. President Clinton ordered the development of the Quality Interagency Coordination Task Force (QuIC) to recommend strategies for improving patient safety and health care quality. The QuIC report in 2000 4 detailed many strategies, including the establishment of mandatory reporting systems in all 50 States. The Agency for Healthcare Research and Quality (AHRQ) was designated as the lead Federal agency for improving health care quality and funded demonstration projects to study adverse event/medical error reporting in 2001. The National Academy for State Health Policy (NASHP) analyzed legal and policy issues of State mandatory reporting systems, concluding that mandatory and voluntary systems can work together to help reduce death and serious injury in the health care system. 5
The Healthcare Financing Administration (now the Centers for Medicare and Medicaid Services [CMS]) began using its peer review organizations to reduce errors of omission among its beneficiaries. Information on adverse events related to treatment, such as nosocomial infections and unintended effects of drugs and medical devices, is collected by the Centers for Disease Control (CDC) and the Food and Drug Administration (FDA), respectively. The Department of Veterans Affairs (VA) has both mandatory and voluntary reporting systems, capturing both adverse events and near misses. The Joint Commission on Accreditation of Healthcare Organizations (JCAHO) implemented its sentinel event policy in 1996 to evaluate sentinel events in JCAHO-accredited hospitals. This policy emphasizes the gathering and analysis of error-related information. 6 The IOM released a report in November 2003 calling for development of a standardized report format to support the full range of existing reporting systems in all settings. 7 The report notes that the electronic medical record will make these systems useful, ensuring that information is transferred back to the point of care and analyzed for lessons learned. The 2003 IOM report recommends that near misses also be included in reporting, as they represent most of the events that occur.
Mandatory versus voluntary reporting of adverse events and medical errors is a current issue being addressed in Congress and in the medical community. Both the House and the Senate have bills (S. 720; H.R. 663) 8, 9 to establish a national voluntary reporting system. Many believe that a voluntary system is the most effective way of encouraging reporting in a nonpunitive culture (e.g., JCAHO; the aviation safety reporting system; the Medication Errors Reporting Program [MER]; MEDMARxsm, a national database for medication errors). The Anesthesia Patient Safety Foundation (APSF), a pioneer in reporting adverse events/medical errors, strongly supports voluntary efforts and feels that the call for a national mandatory reporting system is premature. 10 Others believe that the system should be mandatory, while opponents fear this would actually discourage reporting and create liability issues for providers. 11
IOM's 2003 report discusses Scherkenbach's "cycle of fear" as a model of how using performance data can instill fear and provoke defensive behavior on the part of providers.
7
The NYPORTS system is an example of a mandatory system that successfully uses data at both the State and facility levels, perhaps due to the fact that the focus is on systems improvement. Also, information gathered from the root-cause analysis (RCA) process and the RCA itself is protected from discovery.
New York State has a long history of implementing efforts to improve the quality of hospital care. New York State Public Health Law 2805-l 12 requires hospitals to report and investigate incidents of deaths and other serious injuries in circumstances other than those related to the natural course of illness, disease, or proper treatment. Public Health Law 2805-m 13 protects the confidentiality of the reports and prevents disclosure of incident reports under the Freedom of Information Law. 14 Regulations requiring adverse event reporting became effective in October 1985. The objective of the public health law and the regulations is to ensure that incidents are identified and reported promptly and that a thorough investigation is completed, assessing the causes of the incident and developing corrective actions to mitigate reoccurrence. The first mandatory adverse event reporting system, the Hospital Incident Reporting System (HIRS), was a paper-based system that relied on telephone and mail communication. Reports were entered into the HIRS database by NYSDOH staff. Data collection became burdensome and there was no routine feedback mechanism for hospitals.
In 1993, the mandatory incident reporting system was redesigned and renamed the Patient Event Tracking System (PETS). PETS was an e-mail system based on a decision algorithm of patient harm and therapeutic treatment. It involved subjective judgment regarding incidents through a peer review process, which led to inconsistent data.
In 1995, as part of Governor Pataki's regulatory reform effort, a statewide workgroup of industry experts and a consumer representative was convened to develop, test, implement, and oversee a new mandatory reporting system. The group brainstormed to determine the purpose of the system (Table 1). Building on a collaborative model with the workgroup, the NYSDOH and hospital association representatives aimed to produce a system that was simple, clear, and outcomes-driven, providing useful information for hospitals to improve their own care. The New York Patient Occurrence Reporting and Tracking System 15 was created to focus on quality improvement in conjunction with simplifying reporting, streamlining occurrence coding, and coordinating with other existing systems. The intent of NYPORTS is to allow hospitals to obtain feedback on their own reporting patterns and compare their experience with similar hospitals regionally and statewide (Table 2). Hospital-generated reports allow comparison of experiences with regional, statewide, or peer group aggregate data to enhance quality improvement activities. The most serious occurrences require facilities to conduct a root-cause analysis and develop risk-reduction strategies with corresponding measures of effectiveness to prevent similar future occurrences.
Table 1. NYPORTS brainstorming exercise| What is our purpose? |
| Improve the health of New Yorkers |
|
Valuable system
Increase confidence of public in hospitals Reduce liability for M.D.s and hospitals Improve the accountability of the State |
|
Identify and release relevant information to the public
Produce meaningful outcomes Increase safety of patient care Address the bad 2% Shared vision of hospital and State |
|
Demonstrate utility of collaboration between regulator and regulated
System where labor does not become overwhelming Look for trends that lead to improvement Stimulate quality assurance (QA) efforts Promote collaboration among organizations Understand limits of system Define goals of the reporting process Determine if incident reporting works with QA Improve the models of incidents How do we weave education into the process User-friendly system Shared success Do not violate legislative mandate If it doesn't work, stop Consistent definitions System where there is no fear of reporting |
|
Evaluate cost of system vs. poor quality
Improve accuracy of data reported Reduce input, increase output Data for benchmarking Coordination of data collection efforts Increase utility of data collection Database accessible to all |
| Name | Hospital Incident Reporting System (HIRS) (1985-1993) | Patient Safety Event Tracking System (PETS) (1993-1998) | New York State Patient Occurrence Reporting Tracking System (NYPORTS) (1998-present) |
| System type | Reporting system | Reporting and tracking system | Reporting and tracking system |
| System focus | "Serious event"-driven | "Harm"-driven | "Outcomes"-driven |
| Foundation of system | No focused definitions, open-ended indicators, general categories of reportable events | Decision algorithms based on patient treatment, harm, and hospital peer review process | Includes/excludes lists of clearly defined reportable occurrences |
| Method of data collection | Manual system of data collection and submission | Electronic and manual system of data collection and submission | Electronic system of data collection, submission, and feedback |
| System design | Quality Assurance Model | Quality Management Model | Continuous Quality Improvement Best Practices Model |
| Data utility | Nonspecific data collection | Too much variation in reporting due to widespread interpretation of definitions and algorithms | Uniform, consistent data with limited variability due to clear and comprehensive includes/excludes lists |
| Types of events captured |
|
|
Short form (trackable) occurrences:
|
NYPORTS, a secure Web-based system, is based on an "includes/excludes" list
16
of clearly defined reportable occurrences, which was developed and extensively field tested during 1996 and 1997 (Appendix A).
3
After comprehensive regional training, statewide implementation was completed in April 1998 for 250 hospitals. Major revisions to the includes/excludes list and implementation of system enhancements took place in June 2000. Training included a standardized format for root-cause analyses consistent with the JCAHO model for sentinel event analysis.
17
Annual statewide, regional, and ad hoc education and training sessions are ongoing. NYPORTS remains a dynamic system that will continually evolve over time.
Appendix A is available electronically at http://www.health.state.ny.us/nysdoh/hospital/nyports/annual_report/2000-2001/appendixb.htm.
The success of NYPORTS as a mandatory incident reporting system is dependent upon many factors that are integrated to enhance and support the system. The following elements are considered to be integral to the success of NYPORTS.
Secure, Internet-based, user-friendly systemFacilities access NYPORTS through a secure Internet site. The data is protected by dual firewalls within the NYSDOH. Facility staff members who request access to NYPORTS must sign a confidentiality attestation to further ensure data protection.
Public Health Law ![[section]](https://webarchive.library.unt.edu/eot2008/20090830210508im_/http://www.ncbi.nlm.nih.gov/books/bookres.fcgi/system_default/sect.gif)
2805-l, Incident Reporting,
12
mandates incident reporting by "hospitals," defined as "a facility or institution engaged principally in providing services by or under the supervision of a physician...for the prevention, diagnosis, or treatment of human disease, pain, injury, deformity, or physical condition..." This definition applies to general hospitals and diagnostic and treatment centers.
Certain incidents (e.g., patient deaths or impairments, fires, equipment malfunctions, poisonings, strikes, disasters, and termination of vital hospital services) are specified in the statute, comprising the foundation of NYPORTS reporting. The statute further requires an investigation of a subset of these incidents to discern their root causes. The NYSDOH has expanded the required list of events to include a total of 54 distinct reporting codes.
Public Health Law 2805-m, Confidentiality,
13
guarantees the protection of adverse event data submitted to NYPORTS from disclosure. Further protection exists in the State Education Law, section 6527.
18
Facilities within New York State have designated coordinators who report occurrences. Facilitywide support for NYPORTS is a vital part of quality/process improvement. Leadership support is critical in creating a culture of patient safety, where there is a nonpunitive approach to occurrence/event investigations, reporting, and the root-cause analysis process. NYPORTS case identification is accomplished either on a concurrent or retrospective basis. Case managers, nurse managers, and/or other professional staff are able to identify a high volume of specific NYPORTS occurrences, as they are "hands on" in the medical records on a daily basis. Using administrative ICD-9CM codes and clinical databases such as radiology as a safety net to identify potential NYPORTS reportable cases that were not captured by the concurrent methodology are examples of a retrospective case finding method.
19
The development and ongoing support of a mandatory reporting system requires allocation of finances and dedicated NYSDOH professional and support staff. Professional staff, such as registered nurses, pharmacists, and physicians, are essential to the success of all phases of operation, including data analysis and dissemination.
The NYPORTS Statewide Council, an advisory group to the NYSDOH, meets on a quarterly basis to discuss issues of significance to NYPORTS. The group consists of approximately 50 facility representatives, provider association representatives, and NYSDOH staff. Six of the original 10 workgroup members continue to be active members of the Council. NYPORTS subcommittees were formed to focus on specific aspects of the system, such as the codes and definitions, data analysis, root-cause analysis, education, and dissemination. Progress reports of subcommittee work and recommendations regarding system improvements are presented at the quarterly Council meetings.
The NYPORTS includes/excludes list
16
is designed to specifically define reportable occurrences. This list is part of the NYPORTS User's Manual,
16
which also outlines policies, procedures, definitions, examples, and operational guidance. The manual is revised periodically to reflect changes in definitions and policy, provide clearer examples, and explain system enhancements.
A review of statistics of JCAHO's sentinel events from its national voluntary reporting system indicate that 2,405 events were reported from January 1995 through December 2003.
20
JCAHO received these reports of sentinel events from multiple health care settings. The majority (1,556) of these sentinel events were reported from general hospitals. Of interest is the low number of self-reported sentinel events (106) from 1,326 New York State JCAHO-accredited facilities and an additional 70 unreported sentinel events from other sources (such as media reports, complaints, CMS or State reports), bringing the total to 176 events.
21,
22
In comparison, NYPORTS collected 11,028 reports of similar types of occurrences (NYPORTS codes 911
963), from 1998 to December 2003, from 250 New York State hospitals. In addition, a total of 149,697 occurrences have been reported to the NYPORTS system since 1998. The number of reports increased annually from 11,266 in 1998 to 28,972 in 2003.
15
An analysis of the sentinel event types reported to JCAHO
20
indicates that two of the events, wrong-site surgery and infant abduction, correlate closely with two NYPORTS codes
911 (wrong-patient/wrong-site surgical procedure) and 961 (infant abduction). A review of these NYPORTS codes indicates that from June 1, 2000, to December 31, 2003, there were 104 cases of wrong-patient/wrong-site surgery (code 911) for New York alone,
15
as compared to 300 cases reported to JCAHO
20
from 1995 to 2003 for the entire country. Code 961 (infant abduction) shows 7 cases reported to NYPORTS
15
from June 1, 2000, to December 31, 2003, as compared to 18 cases reported to JCAHO
20
from 1995 to 2003. The comparison of the number of reports in a mandatory system (NYPORTS) versus a voluntary system (JCAHO) shows the potential utility of mandatory reporting (Tables 3 and 4).
| Event* | Code | # | % |
| Wrong patient/sitesurgical procedure |
911 | 163 | 2.4 |
| Incorrect procedure/treatment invasive | 912 | 538 | 7.9 |
| Retained foreign body | 913 | 527 | 7.8 |
| Unexpected death | 915 | 2,866 | 42.3 |
| Unexpected cardiac arrest (BLS/ACLS intervention) | 916 | 643 | 9.5 |
| Unexpected removal of organ | 917 | 191 | 2.8 |
| Unexpected loss/impairment of limb | 918 | 458 | 6.8 |
| Unexpected loss/impairment of bodily function | 919 | 634 | 9.4 |
| Error of omission/delay leading to death/serious injury | 920 | 357 | 5.3 |
| Crime leading to death/serious injury | 921 | 20 | 0.3 |
| Suicide/attempted suicide | 922 | 278 | 4.1 |
| Elopement from hospital leading to death/serious injury | 923 | 44 | 0.6 |
| Equipment malfunction leading to death/serious injury | 938 | 22 | 0.3 |
| Infant abduction | 961 | 8 | 0.1 |
| Infant discharged to wrong family | 962 | 2 | 0.2 |
| Rape by another patient/staff | 963 | 25 | 0.4 |
|
* N = 6,776 | |||
| Type of sentinel event* | # | % |
| Patient suicide | 374 | 15.2 |
| Op/post-op complication | 315 | 12.8 |
| Wrong-site surgery | 300 | 12.2 |
| Medication error | 282 | 11.5 |
| Delay in treatment | 161 | 6.6 |
| Patient death/injury in restraints | 112 | 4.6 |
| Patient fall | 110 | 4.5 |
| Assault/rape/homicide | 84 | 3.4 |
| Transfusion error | 69 | 2.8 |
| Perinatal death/loss of function | 67 | 2.7 |
| Patient elopement | 48 | 2.0 |
| Fire | 45 | 1.8 |
| Ventilator death/injury | 38 | 1.5 |
| Anesthesia-related event | 35 | 1.4 |
| Infection-related event | 34 | 1.4 |
| Medical equipment-related | 32 | 1.3 |
| Maternal death | 28 | 1.1 |
| Infant abduction | 18 | 0.7 |
| Utility systems-related event | 18 | 0.7 |
| Other less frequent types | 285 | 11.6 |
|
* N = 2,455 since 1995 | ||
The analysis of NYPORTS data can have a significant positive impact on patient safety. The challenge for any data collection system is to provide system users with an easy and efficient way to extract data for meaningful analysis. NYPORTS allows analysis to occur at both the facility and NYSDOH levels for use in quality assurance activities or statewide analysis of individual reporting codes.
NYPORTS reporting and the resultant access to comparative data have prompted individual facilities to conduct internal studies, targeting areas of concern through analysis of patterns and trends. The results of these studies have been significant in improving patient care and safety, as well as reducing hospital costs. For example, one facility reduced the risk of patients developing deep vein thrombosis and pulmonary embolus. This area of focus was identified through the system's comparative reporting function. A comprehensive risk-factor assessment and prophylaxis protocol was established. This identifies patients at risk upon admission and ensures that they receive appropriate prophylaxis, decreasing the number of hospital-acquired thromboembolic events.
Quality improvement opportunities
NYPORTS provides data for statewide performance improvement efforts and individual hospitals to identify trends or patterns of occurrences over time. New systems or processes can be implemented and measured to determine their effectiveness for reducing specific occurrence codes. Hospitals can export their own NYPORTS data into a Microsoft![[reg]](https://webarchive.library.unt.edu/eot2008/20090830210508im_/http://www.ncbi.nlm.nih.gov/books/bookres.fcgi/system_default/reg.gif)
Access database to create customized reports. The system allows creation of reports comparing individual hospital data to regional, statewide, and peer groups. NYPORTS is well integrated into many hospital and systemwide quality management systems. Hospitals can share data on several levels, including individually by practitioner or service. Hospital networks use NYPORTS data to identify trends within their systems, promoting performance improvement at the individual practitioner, facility, or system level.
In a presentation at an AHRQ conference in August 2003, Robert Panzer, M.D., chief quality officer at the University of Rochester Medical Center, discussed his overall approach to patient safety using NYPORTS data at Strong Memorial Hospital.
23
Reporting and tracking of less serious but common adverse events may encourage clinical leaders to implement preventive measures, while reporting more serious events leads to root-cause analysis and systems improvement driven by senior leadership. This supports the use of NYPORTS as an integral part of promoting a culture of patient safety.
NYPORTS captures data on 54 specifically defined reportable occurrences. Analysis of NYPORTS data allows the NYSDOH and facilities to identify those areas where errors can occur and to implement interventions to reduce the likelihood of these errors occurring in the future. An analysis of wrong-patient/wrong-site surgical errors led to the development of the New York preoperative protocols final report in January 2001, which focused on strategies to reduce or prevent wrong-patient/wrong-site surgery, wrong procedures, and procedures conducted on the wrong patient.
24
Hospitals and other health care facilities were expected to develop and implement procedures to ensure that at least three independent verifications of surgical site location and correct patient identification occur. The report stresses the importance of the surgeon seeing and talking to the patient in the perioperative area and the importance of communication among members of the surgical team and the patient. It strongly recommended delaying any procedure where discrepancies of information exist. As a result of the adoption of the protocols and NYPORTS analysis, the number of wrong-patient/wrong-site events decreased in 2002 (25 events) and 2003 (17 events).
15
Findings and recommendations remain applicable to hospitals, ambulatory surgery centers, and office-based surgery settings and will complement the JCAHO Universal Protocols effective July 1, 2004.
25
Continuous Quality ImprovementNYPORTS is a dynamic system that continues to evolve and improve over time. Since its inception, several multidisciplinary subcommittees have played critical roles in creating and maintaining the current quality system. The refinement subcommittee reviews the includes/excludes list to determine whether occurrence codes need to be added, deleted, or modified and to provide clarification to the definitions manual. The training and education subcommittee coordinates and schedules regional and statewide training sessions and disseminates information. The medication error subcommittee develops medication error reporting categories and the medication error supplemental form. 16 This committee has analyzed 108 medication errors and associated root-cause analysis submissions for a 2-year period and has presented findings at a statewide NYPORTS Council meeting and multiple professional conferences. A data analysis panel has reviewed approximately 300 NYPORTS occurrences and associated root-cause analyses of unexpected deaths over an 18-month period. Experts reviewed and trended the root causes and risk-reduction strategies identified by facilities and reviewed evidence-based practices to support additional facility system fixes. The Root Cause Analysis Subcommittee is responsible for modifying the RCA form and providing training on the RCA process. An RCA evaluation tool, designed by the NYSDOH, is used by DOH regional coordinators to ensure information contained in RCAs is thorough and credible.
The NYSDOH works with a computer firm to enhance NYPORTS and implement changes/upgrades annually. The University at Albany School of Public Health contracts with the NYSDOH to analyze NYPORTS occurrence codes and report findings at quarterly statewide Council meetings. Links between NYPORTS and New York State's administrative database, the Statewide Planning and Research Cooperative System (SPARCS), have been established to identify and carry out projects to improve NYPORTS reporting. The School of Public Health is studying AHRQ's Patient Safety Indicators and comparing them to the NYPORTS occurrence codes that are similar to determine potential ways to streamline the reportable occurrences in NYPORTS.
The NYSDOH disseminates data analysis results using various methods in order to achieve system and patient safety improvements that impact quality of care. Key dissemination activities include sharing lessons learned, educational programs, presentations, the NYPORTS News and Alert newsletter, 26 the NYPORTS bulletin board, 26 publicly publishing comprehensive reports, 27, 28 and the New York State Patient Safety Award program (Table 5).
Table 5. NYPORTS information dissemination activities| Methods | Description | Example |
| Statewide Council meetings | The NYPORTS Statewide Council is an advisory group to the NYSDOH in matters concerning NYPORTS. The Council consists of facility representatives, provider association representatives, original workgroup members, and NYSDOH staff. | Code 915Unexpected Death Analysis presented at January and May 2003 meetings. Deaths in the following specialties were analyzed:
|
| Regional forums | Regional hospital associations disseminate information from Statewide Council meetings and address NYPORTS-related issues at periodic meetings. | Regional forums have presented system enhancements, analysis, and general NYPORTS information to hospital personnel. |
| Professional organizations | Department of Health staff presents results of NYPORTS analysis and operational topics at conferences hosted by professional organizations. | NYSDOH staff presented Medication Analysis results at the NYS Council of Hospital Pharmacists and St. Elizabeth's Hospital. |
| NYPORTS News & Alert | The NYPORTS News & Alert is a newsletter periodically issued to educate facilities about analysis, interpretations, and system use. | Issue #13 provided guidance to facilities regarding prevention of surgical fires and burns. |
| NYPORTS bulletin board | The NYPORTS bulletin board is accessed on the secure Web-based system, providing an opportunity for the NYSDOH to post relevant information for system users. | Postings include, but are not limited to, the User's Manual, News & Alert issues, and system enhancement information. |
| Dear Chief Executive Officer letters | Letters are issued to the CEO/administrators of facilities to share advisories, policy clarifications, and other information. | Topics include NYPORTS, as well as a variety of other topics relative to hospital administration and regulatory compliance. |
| Patient Safety Conference | The NYSDOH will hold a statewide Patient Safety Conference to disseminate analysis and promote patient safety awareness in New York State. | Proposed presentations include Patient Safety Award recipients and results of DOH-sponsored analysis. |
| Hospital Association/ Department of Health training | The NYSDOH, in conjunction with hospital associations, gives periodic education sessions to hospital personnel. The sessions are often videoconferenced to multiple sites, and CD-ROMs are made and distributed for future use. | The Hospital Association of New York State (HANYS) and the DOH have conducted multiple NYPORTS educational sessions which are available statewide. The latest session took place 11/3/04. |
| NYPORTS annual reports | The NYSDOH publishes a comprehensive NYPORTS annual report that includes statewide reporting and analysis data and provides a resource for patient safety information in New York State. | The DOH has issued two NYPORTS annual reports, one for 1999 and one for 2000/2001. Both reports may be accessed on the public Web site. |
| New York Patient Safety Awards | Hospitals, nursing homes, and Federally Qualified Health Care Centers (FQHC) are eligible to receive an award and a grant of up to $200,000 to promote their patient safety strategy that has shown measurable decreases in adverse patient outcomes. | Annual award recognizes facilities for successful patient safety initiatives. The DOH assists the award recipients to disseminate their patient safety strategies statewide. |
Current proposals for voluntary adverse event/medical error reporting systems build on elements from existing systems such as the Aviation Safety Reporting System (ASRS) and the VA's National Center for Patient Safety's Patient Safety Reporting System. These systems are public and nonpunitive. Events are reported to a national nonregulatory agency. Near misses are a vital part of voluntary reporting systems. 29, 30 The VA also operates the Patient Safety Information System, 30 a voluntary confidential reporting system for adverse events. Recommendations for developing a national mandatory system include focusing on errors related to licensing and other regulatory issues in a confidential system. There is nearly universal consensus that information technology needs to play a role for any system to be effective in detecting adverse events. Systems that rely on spontaneous reporting are ineffective and contribute to current underreporting issues. 31
The APSF
10
believes that creating a mandatory reporting system is very complex and there is no evidence to show that it results in meaningful improvement in practice. The American Medical Association (AMA) and the American Hospital Association (AHA) oppose mandatory reporting and believe that any reporting that is tied to punitive action or public disclosure will encourage making the reporting system a "numbers game" and drive reporting underground by perpetuating a culture of blame.
32
However, the volume of reports in the NYPORTS system and its use of the root-cause analysis framework demonstrate that mandatory reporting can result in important systems improvement. The NYPORTS system incorporates elements of a mandatory reporting system as recommended by the IOM and NASHP, including
- operation by a state regulatory agency;
- collection of standardized information;
- inclusion of serious adverse events and medical errors;
- ongoing statewide analysis of patterns and systemic issues;
- external data validation; and
- required followup.
NYPORTS is in compliance with JCAHO's sentinel event reporting system.
There is a central conflict underlying the development of mandatory reporting systems that creates a significant barrier. The public desires accountability from physicians and other providers, while physicians and hospitals fear malpractice liability and damage to their reputations. Physicians and hospitals support voluntary reporting and sharing of the information to improve patient safety, which is desirable when the primary goal is to learn from prior mistakes and experience. However, the public feels that mandatory reporting improves accountability. 33 The NYPORTS system integrates these elements providing accountability within a learning environment.
Lessons learnedThe experience of developing adverse event incident reporting systems in New York State resulted in several lessons learned that might help shape the design of future systems. One of the most critical lessons learned is that information gathered into the system must be meaningful and useful to those who are reporting events. This is more likely to occur if key stakeholders help develop the system, as they will help build the consensus needed regarding the importance and utility of the system. Designing a Web-based system where facilities can access their own data and create comparative reports is the foundation upon which NYPORTS was built. Without this ability, facilities would be dependent upon the NYSDOH to create reports. In our earlier reporting systems, data were not timely, delaying knowledge learned from the reports.
The initial NYPORTS system was designed using a "short form/long form" concept. The short form, containing demographic data and a narrative description of the event, must be submitted for every occurrence. The long form was a narrative-free text investigative report submitted on the most serious events. In June 2000, the RCA framework replaced the long form. This added structure to the reports and allowed more meaningful analysis to occur. RCA analysis revealed that the quality of the information submitted was variable, despite ongoing educational efforts. A tool was designed and implemented to evaluate submitted RCAs against specific criteria expected in a thorough and credible RCA and to assure more consistency.
The turnover of hospital staff affects reporting rates and the quality of the reports submitted. A standard NYPORTS tutorial ensures that new staff members are consistently trained. Educational forums include NYPORTS statewide and regional Councils. Memberships of these advisory groups were initially structured with 3-year cycles, which never happened. Instead, the meetings have steadily grown in attendance. It is difficult to limit the involvement of people who see the value of the system and want to be active participants.
An incident reporting system must remain open for continual improvement. The NYPORTS system's three phases of field testing in 1996 and 1997 led to initial meaningful changes and improvements. NYPORTS and other incident reporting systems need a process to ensure that regular reviews, updating, and system enhancements occur that can be balanced with the desire to track longitudinal data. Definitions are improved and clarified on an ongoing basis, often as a result of scenarios presented by facilities. Although NYPORTS has been in place for 6 years, gray areas still exist.
Based on NYSDOH's experiences with the development and evolution of our mandatory reporting system, we have found that certain critical elements are necessary for its success. These critical elements include making the system legally required, with protection from discovery; developing the system collaboratively, including all stakeholders in the system's design and implementation; clear and objective definitions of reporting criteria as a basis for collecting accurate and consistent data; ongoing training and educational support for system users; and having a stakeholder advisory group for ongoing assessment and recommendations, ensuring the system's relevance and viability. Other elements vital to the success of NYPORTS include having a secure Web-based system and ensuring that adequate resources and supports are dedicated to operating and maintaining the system. Ultimately, the success of the system also requires that users receive feedback regarding their own performance. It must be possible to analyze data at both the facility and statewide levels, incorporating dissemination of lessons learned. Collectively, these elements are the basis of performance improvement efforts that will positively impact patient safety and move quality and patient safety to the next level.
The authors wish to acknowledge the workgroup members for their dedication, participation and expertise: Benjamin Lankheet, M.D., F.F., Thompson Hospital; Brad Truax, M.D., Independent Health; Michael Jakubowski, M.D., Ellis Hospital; Arthur Levin, M.P.H., Center for Medical Consumers; Edwina Thompson, R.N., New York University Medical Center; Joseph Conte, M.P.A., Staten Island University Hospital.
We wish to thank Lorraine Vollmer for assistance in manuscript preparation.
New York State Department of Health (EF, CLC, AR, PD, FJH, RL, JM). University of Rochester Medical Center (RJP).
Address correspondence to: Ellen Flink, M.B.A., NYS Department of Health, 161 Delaware Avenue, Delmar, NY 12054; phone: 518-408-1234; e-mail: emf02@health.state.ny.us.
1. Public Health Law
2805-j, Medical, dental and podiatric malpractice prevention program (Added L. 1986, c.266). 2. Brennan TA, Leape LL, Laird NM, et al. Incidence of adverse events and negligence in hospitalized patients. Results of the Harvard Medical Practice Study I. NEJM 1991. 324(6):370-6. Feb7. (PubMed)
3. Kohn LT, Corrigan JM, Donaldson, MS, editors. To err is human: building a safer health system. A report of the Committee on Quality of Health Care in America, Institute of Medicine. Washington, DC: National Academy Press; 2000.
4. Quality Interagency Coordination Task Force to the President. Federal Actions to reduce medical errors and their impact; 2000 Feb.
5. Flowers L, Riley T. State-based mandatory reporting of medical errors. National Academy for State Health Policy; 2001 Mar.
6. Joint Commission on Accreditation of Healthcare Organizations. A journey through the history of the Joint Commission; 2004. Available at: http://www.jcaho.org/about+us/history/index.htm .
7. Institute of Medicine. Patient safety: achieving a new standard of care. Washington, DC: National Academy Press; 2003.
8. Amendment to Title IX of the Public Health Service Act, S.720, 108th Congress. Library of Congress. http://frwebgate.access.gpo.gov/cgi-bin/getdoc.cgi? dbname=108_cong_bills&docid=f:s720rs.txt.pdf .
9. Amendment to Title IX of the Public Health Service Act, H.R.663, 108th Congress. Library of Congress. http://frwebgate.access.gpo.gov/cgi-bin/getdoc.cgi? dbname=108_cong_bills&docid=f:h663rh.txt.pdf .
10. Stoelting R. APSF response to the IOM report. Comments from the Anesthesia Patient Safety Foundation; 2000. http://www.gasnet.org/societies/ apsf/safety/iom_response3.php .
11. Brennan, T. The Institute of Medicine report on medical errors: could it do harm? NEJM 2000. 342(15):1123-5. Apr 13. (PubMed)
12. Public Health Law
2805-1, Incident Reporting (Added L. 1986, c.266). 13. Public Health Law
2805-m, Confidentiality (Added L. 1986, c.266). 14. Public Officer's Law, Article 6,
87 (Added L. 1977, c.933). 15. New York Patient Occurrence Reporting and Tracking System Database. Secure database for authorized users. Data retrieved on 2004 Apr 14.
16. New York State Department of Health. NYPORTS user's manual. Version 2.2. New York State Department of Health; 2002.
17. Joint Commission on Accreditation of Healthcare Organizations. Sentinel events: evaluating cause and planning improvement. Oakbrook Terrace, IL: JCAHO; 1998.
18. New York Education Law, sec. 6527(3); 2000.
19. Tuttle D, Panzer R, Baird T. Using administrative data to improve compliance with mandatory State event reporting. Jt Comm J Qual Improv 2002. 28(6):349-58. June. (PubMed)
20. Joint Commission on Accreditation of Healthcare Organizations. Sentinel event statistics. Available at: http://www.jcaho.org/accredited+organizations/office+based+surgery/sentinel+events/se_stats_9_04.pdf . (Accessed 2004, Apr 14.)
21. Joint Commission on Accreditation of Healthcare Organizations. Self-reported events by State, January 1995 through December 2003. Available at: http://www.jcaho.org/accredited+organizations/ambulatory+care/sentinel+events/set+self-reported.htm . (Accessed 2004, Jul 8.)
22. Joint Commission on Accreditation of Healthcare Organizations. Total reviewed events by State, January 1995 through December 2003. http://www.jcaho.org/accredited+organizations/ambulatory+care/sentinel+events/set+total+reviewed.htm . (Accessed 2004, Jul 8.)
23. Panzer RJ. Facility Patient Safety Practices; Strong Memorial Hospital. AHRQ Conference: Making New Jersey a Model for Patient Safety; 2003 Aug 20.
24. New York State Department of Health. Homepage on the Internet. http://www.health.state.ny.us/press/ releases/2001/preopletter.htm (Accessed 2004, Apr 14.)
25. Joint Commission on Accreditation of Healthcare Organizations. Universal Protocol for Preventing Wrong Site, Wrong Procedure, Wrong Person Surgery. TM December 2003. http://www.jcaho.org/ accredited+organizations/patient+safety/universal+ protocol/universal+protocol.pdf .
26. NYPORTS. On-line New York Patient Occurrence Reporting and Tracking System. Albany, NY: New York State Department of Health. c1998-2004 (Updated Apr 9, 2004.) https://commerce.health.state.ny.us/hpn/nyports/intro.html . (Log in required; accessed 2004, Apr 14.)
27. New York State Department of Health. NYPORTS
The New York Patient Occurrence Reporting and Tracking System Annual Report. New York State Department of Health; 1999. http://www.health.state.ny.us/nysdoh/hospital/nyports/index.htm
28. New York State Department of Health. NYPORTS
The New York Patient Occurrence Reporting and Tracking System Annual Report. New York State Department of Health; 2001-2002. http://www.health.state.ny.us/nysdoh/hospital/nyports/index.htm
29. Aviation Safety Reporting System. http://www.asrs.arc.nasa.gov .
30. Department of Veterans Affairs, National Center for Patient Safety. Patient Safety Reporting System. Program Overview. http://psrs.arc.nasa.gov/ program_briefing.htm .
31. Bates D, Evans S, Murff H, et al. Detecting adverse events using information technology. J Am Med Inform Assoc 2003. 10(2):115-28. Mar/Apr. (PubMed) (Full Text in PMC)
32. Kaufman M. Clinton seeks medical error reports: proposal to reduce mistakes includes mandatory disclosure, lawsuit, shield. The Washington Post 2000. 22: p. A-2. Feb.
33. Leape L. Reporting on adverse events. NEJM 2002. 347(20):1633-8. Nov20. (PubMed)