IMPROVING PATIENT SAFETY:  HEALTH SYSTEMS REPORTING, ANALYSIS, AND SAFETY 
IMPROVEMENT RESEARCH DEMONSTRATIONS

Release Date:  February 2, 2001

RFA:  RFA-HS-01-003

Agency for Healthcare Research and Quality
 
Letter of Intent Receipt Date:  April 2, 2001
Application Receipt Date:       April 27, 2001
 
PURPOSE

The Agency for Healthcare Research and Quality (AHRQ) announces the 
availability of  approximately 13-14 Cooperative Agreements (U18) for up to 
three years to support large demonstrations in states, health care systems, 
and/or networks of providers to test reporting strategies and patient safety 
interventions.  With these projects, AHRQ is interested in sponsoring 
research that will improve the safety of patients being cared for in a wide 
variety of medical settings.  This goal can be accomplished through changes 
in health care delivery procedures to reduce the risk of harm to patients, 
changes in public policy or public oversight of health care delivery systems 
intended to foster improved safety, and the provision of information to the 
public or to purchasers to encourage choices of health care providers that 
will reduce risk to patients.  These projects should use technology, staff 
training, and other methods to reduce such errors; develop replicable models 
that minimize the frequency and severity of medical errors; develop 
mechanisms that encourage reporting, prompt review, and corrective action 
with respect to medical errors; and develop methods to minimize any 
additional paperwork burden on health care professionals.  AHRQ is seeking 
applications for demonstration projects that will assess the effectiveness of 
various methods of collecting and using information to reduce medical errors 
and their impact.  It is also seeking applications that will test various 
methods of informing patients (or their family members) when they have been 
injured by a medical error. 

By testing a variety of reporting, analysis, and implementation strategies, 
AHRQ wishes to evaluate what data should be collected, how it should be 
aggregated and analyzed, how it should be reported to provide useful 
information to those trying to reduce patient injuries from medical errors, 
and how it can be protected from unintentional disclosure.  AHRQ is seeking 
applications that will demonstrate and evaluate:

o   A variety of methods of identifying errors or opportunities for improving 
safety and reporting them to a database that promotes analysis, 
understanding, and action to reduce the risk of harm to patients; 
o   Different concepts of what information is needed and useful in reducing 
risk to patients; 
o   Effective methods of disseminating information to those who need the 
information to improve safety through choice, oversight of, or changing the 
delivery system; and
o   New methods of changing the delivery system in ways that can reduce 
hazards, including innovations in informatics and educational changes.  

AHRQ is particularly interested in research projects that will test systems 
that publicly disclose information on errors or risks and comparing those 
systems to ones that are principally designed to enhance providers’ ability 
to learn to prevent errors.  It is also interested in the feasibility of 
different methods of electronic data collection and analysis that will 
facilitate the processing and use of large amounts of data.  AHRQ expects 
that, to the extent possible, with the data collected, each funded grant team 
will use the data to examine segments of the population that may be 
especially at risk for injuries from medical errors.  For example, people 
with chronic conditions requiring frequent interaction with the health care 
system, people with limited English proficiency children, or people living in 
rural areas with limited access to certain technologies or health 
professionals may be more vulnerable to some kinds of error (e.g., missed 
diagnoses, misdiagnoses, incorrect therapies being applied, therapies being 
applied incorrectly, failure of communication to other caregivers, failure of 
communication with the patient or family, etc.).  In addition to the broader 
information dissemination, AHRQ is interested in testing various approaches 
to informing the patients or their family members when they have been injured 
by an error. 
   
AHRQ is interested in providing resources to evaluate the effectiveness of 
existing systems or systems that will be functional within a few months of 
receipt of funds, rather than in supporting their development.  AHRQ 
anticipates that the principal use of funds from this solicitation will be to 
enable a comprehensive evaluation of the existing systems and proposed 
applications of or changes to the systems to enable them to more effectively 
collect, analyze, disseminate, and act on the information collected.  
Infrastructure and data collection to facilitate the evaluation, to the 
extent necessary, can be funded under this RFA.   AHRQ also expects the 
funded organizations to have or develop a plan for sustaining the reporting 
system and all its component parts once the grant expires.   

This Request for Applications (RFA) is one in a series of research 
solicitations to be issued by AHRQ over the next several months.  They form 
an integrated set of activities to design and test best practices for 
reducing errors in multiple settings of care, develop the science base to 
inform these efforts, improve provider education to reduce errors, capitalize 
on the advances in information technology, translate proven effective 
strategies into widespread practice, and build the capacity to further reduce 
errors.  At the heart of the competitive demonstration program described in 
this request for applications is a portfolio of projects to test the 
effectiveness, cost, and cost-effectiveness of diverse reporting strategies 
and information technology application and innovation on the identification, 
management, and reduction of medical errors.  These activities are supported 
and amplified by: 1)  the establishment of multidisciplinary centers of 
excellence in patient safety; 2)  targeted efforts to understand the impact 
of provider education, skills, staffing, and organization on error rates; 3) 
partnerships with health systems, professional organizations, states, and 
other groups to build capacity for error reduction activities, disseminate 
effective strategies, and coordinate public and private efforts; and 4) 
cross-cutting activities that capitalize on data already collected by the 
Federal government which can be enhanced to support research and action in 
patient safety.

Other RFAs to be released as part of this program include:

1)  Clinical Informatics to Promote Patient Safety (CLIPS) -- to develop and 
test the use of appropriate technologies to reduce medical errors, such as 
hand-held electronic medication and specimen management systems and 
prescription pads, training simulators for medical education, bar-coding of 
prescription drugs, patient bracelets, and automated dispensing of medication 
in a hospital setting. 

2)  Centers of Excellence for Patient Safety Research and Practice -- to 
support established cross  -- cutting teams of researchers and health care 
facilities and organizations in geographically diverse locations (including 
rural and urban areas) which will determine the causes of medical errors, 
develop new knowledge to support the work of the demonstrations.

3)  Developmental Centers for Evaluation and Research in Patient Safety 
(DCERPS) -- to develop new multidisciplinary research teams to improve the 
nations capacity in patient safety research, to expand the patient safety 
knowledge base, and to establish mechanisms to assure that new knowledge is 
incorporated into actual practice and that its impact is assessed. 

4)  Effect of Working Conditions on Patient Safety -- to develop an 
understanding of how the environment of care impacts the ability of providers 
to improve safety (e.g. the effect of fatigue, stress, sleep deprivation, and 
shift work on cognitive ability and the relationship to patient safety) and 
how interactions with the built environment impact the provision of safe 
care. 

5)  Patient Safety Research Dissemination and Education -- to fund 
researchers and organizations (e.g., professional associations, hospital 
groups, national organizations) to develop, demonstrate, and evaluate new 
approaches to improving provider education in order to reduce errors, such as 
teaching new knowledge on patient safety and developing curricula, continuing 
education, simulation models, and other provider training strategies.

Additional components of the Agency’s patient safety initiative will build on 
existing Agency programs.  AHRQ will also support a coordinating center to 
provide guidance and facilitate interaction among the entities funded under 
the Agency’s patient safety programs.  The coordinating center will not be 
one of the funded demonstration projects, but rather a separate entity under 
contract with AHRQ.  The coordinating center will encourage the transition of 
new knowledge from the research centers and informatics projects to the 
demonstration sites. 

Research themes under this RFA and others to be released by the Agency will 
be generated by a variety of activities including public and private sector 
national summits and research agenda- setting activities on medical errors 
and patient safety, as well as key stakeholder meetings. Research themes will 
also be the outgrowth of ongoing activities of the Quality Interagency 
Coordination Task Force (QuIC) and its report to the President -- Doing What 
Counts for Patient Safety - Federal Actions to Reduce Medical Errors and 
Their Impact (http://www.quic.gov).  In carrying out its responsibilities, 
the National Forum for Health Care Quality Measurement and Reporting may also 
influence themes for this research initiative.  AHRQ sponsored a National 
Summit on Medical Errors and Patient Safety Research on September 11, 2000 
and a follow up discussion with provider associations on November 30, 2000.  
A summary of the summit and the research agenda that was developed based on 
this input are available at http://www.quic.gov/summit/resagenda.htm.

HEALTHY PEOPLE 2010 

The Public Health Service (PHS) is committed to achieving the health 
promotion and disease prevention objectives of “Healthy People 2010,” a PHS-
led national activity for setting health improvement priorities for the 
United States. AHRQ encourages applicants to submit grant applications with 
relevance to the specific objectives of this initiative. Potential applicants 
may obtain a copy of “Healthy People 2010" at 
http://www.health.gov/healthypeople/.

ELIGIBILITY REQUIREMENTS

Applications may be submitted by domestic, public and private non-profit 
organizations, including universities, clinics, health care systems, health 
plans, professional societies, clinician networks, units of State and local 
governments, and eligible agencies of the Federal government.   Applications 
may include foreign components, but foreign organizations are not eligible to 
apply. Under recently enacted reauthorization legislation, AHRQ is authorized 
to enter into cooperative agreements with for-profit organizations as well as 
with public and not-for-profit entities.  Thus, for-profits organizations are 
invited to respond to this notice with research applications for cooperative 
agreements.  Applications will be administered in accordance with 45 CFR Part 
74 and 42 CFR Part 67 Subpart A.  The latter regulation was amended on 
December 6, 1999, and reflects changes in Agency name and authority.  (See 
AHRQ reauthorization at http://www.ahrq.gov/hrqa99a.htm.)  Organizations 
described in section 501(c)4 of the Internal Revenue Code that engage in 
lobbying are not eligible.

AHRQ encourages women, members of racial and ethnic minority groups and 
persons with disabilities to apply as Principal Investigators.  Special 
preference will be accorded to applications from investigators who have not 
been funded by AHRQ as a Principal Investigator within the last two years.  
AHRQ also encourages minority institutions to apply for funding under this 
RFA, and encourages collaboration on projects between minority institutions 
and non-minority institutions.  Minority institutions have had a significant 
role in delivering health care to underserved communities and represent a 
valuable resource to facilitate collaboration with those communities.  A 
partnership between a minority institution and an institution with 
substantial research experience is encouraged for this RFA.  

AHRQ’s recent reauthorization, the Health Care Research and Quality Act of 
1999, directed AHRQ to focus more extensively on a number of priority areas 
and populations:  inner-city areas; rural areas, including frontier areas; 
low-income groups; minority groups; women; children; the elderly; and 
individuals with special health care needs, including individuals with 
disabilities and individuals who need chronic care or end-of-life health 
care.  AHRQ strongly encourages applicants for these demonstration projects 
to indicate if their data collection, analysis, or safety improvement 
strategies will permit them to focus on one or more of these areas and 
populations.

AHRQ also encourages practice-based research networks (PBRNs) to submit 
applications focusing on medical errors and patient safety efforts in 
community-based ambulatory practice sites.  Funds will be set aside 
specifically to support ambulatory care network applications, and proposal 
abstracts should indicate the network’s intention to apply for these funds 
(see FUNDS AVAILABLE).  For the purpose of this RFA, an ambulatory care 
network is defined as a group of 10 or more ambulatory practices devoted 
principally to the care of patients, affiliated with each other (and often 
with an academic or professional organization) in order to investigate 
questions related to community-based practice.  This definition includes a 
sense of ongoing commitment to the research endeavor and an organizational 
structure that transcends a single study.  Applications may be submitted by 
ambulatory networks composed of primary care and/or specialty clinicians.  
See further details under Special Requirements.

MECHANISM OF SUPPORT

The administrative and funding instrument to be used for this program will be 
a cooperative agreement (U18), an "assistance" mechanism (rather than an 
"acquisition" mechanism), in which substantial AHRQ scientific and/or 
programmatic involvement with the awardee is anticipated during performance 
of the activity.  Under the cooperative agreement, the AHRQ purpose is to 
support and/or stimulate the recipient's activity by involvement in and 
otherwise working jointly with the award recipient in a partner role, but it 
is not to assume direction, prime responsibility, or a dominant role in the 
activity.  Details of the responsibilities, relationships and governance of 
the study to be funded under cooperative agreement(s) are discussed later in 
this document under the section “Terms and Conditions of Award.” 
 
The total project period for applications submitted in response to the 
present RFA may not exceed three years.  The anticipated award date is 
September 28, 2001.  

This RFA is a one-time solicitation.  AHRQ has not determined whether or how 
this solicitation will be continued beyond the present RFA.

Note that the scope of this initiative and the capacity-building aspects of 
the U18 allow new investigators and institutions to be drawn together to work 
collaboratively with a team of experienced researchers.  AHRQ particularly 
encourages collaboration with researchers from Historically Black Colleges 
and Universities, Hispanic Serving Institutions, and other minority and 
minority serving institutions.
 
FUNDS AVAILABLE

AHRQ expects to award up to $25 million in total costs (direct costs plus 
facilities and administrative costs) in fiscal year 2001 to support the first 
year of approximately 13 to 14     projects under this RFA.  It expects to 
award similar amounts for the duration of the projects.  The actual number of 
applications funded is dependent on the number of high quality applications 
and their individual budget requirements; it is not the intent of AHRQ that 
the awards be equal in size. Applicants seeking to submit budgets in which 
direct costs will exceed $2 million in any year of the project must call 
Nancy Foster (See Inquiries) for prior approval.  Applications seeking 
budgets in which direct cost exceed $ 2 million that do not have prior 
approval will be returned without review.  Representatives of recipient 
demonstration projects will meet with Center for Quality Improvement and 
Patient Safety (CQuIPS) staff and the coordinating center staff up to 3 times 
each year.  Budget requests should therefore include travel expenses for that 
purpose.   

Although this program is provided for in the financial plans of AHRQ, awards 
pursuant to this RFA are contingent upon the availability of funds for this 
purpose.  Funding beyond the initial budget period will depend upon annual 
progress reviews by AHRQ and the availability of funds.

It is anticipated that at least two awards will be made for projects that 
look explicitly at medical errors and patient safety in office-based 
settings.  These awards will total approximately $2 million (including direct 
and indirect costs) per year out of the $25 million in support of this RFA.  
Should additional funds become available, however, and if AHRQ receives a 
sufficient number of meritorious applications, AHRQ reserves the right to 
fund additional applications under this RFA.

Terms and Definitions

The following terms, definitions, and explanations are provided for purposes 
of this RFA, and may not be wholly consistent with other organizations’ use 
of the terms:

o   Medical error - The failure of a planned action to be completed as 
intended or the use of a wrong action to achieve an aim.  Errors can include 
problems in practice, products, procedures, and systems.  (See Ref #1)

o   Adverse events - Undesirable and unintended incidents in care that may 
result in adverse outcomes or may require additional care efforts to thwart 
an adverse outcome.

o   Adverse outcomes - Undesirable and unintended outcomes of care such as 
death, disability, or temporary disability. 

o   Clinical responses to errors - Data systems may collect information on 
clinical responses to errors, such as the number of times an antidote was 
given or a measure intended to counter an unintended consequence was used.

o   Dangerous situations - Where both human and latent failure exist that 
create a hazard increasing the risk of harm, this is a dangerous situation. 
Information may be collected from individuals familiar with the process of 
care in organizations about conditions that are highly likely to cause an 
injury to a patient or patients. 

o   Errors - The failure of a planned action to be completed as intended or 
the use of a wrong plan to achieve an aim.  Errors can include problems in 
practice, products, procedures, and systems.  (See Ref # 2)

o   Hazard - Anything that can cause harm.  (See Ref # 3)

o   Near Misses -  As defined by Van der Schaaf, (See Ref # 4) these are 
events in which the unwanted consequences were prevented because there was a 
recovery by identification and correction of the failure.  Such a recovery 
could be by a planned or unplanned barrier.

o   No harm events - Events that have occurred but resulted in no actual harm 
although the potential for harm may have been present.  Lack of harm may be 
due to the robust nature of human physiology or pure luck.  An example of 
such a benign event would be the issuing of an ABO incompatible unit of blood 
for a patient, but the unit was not transfused and was returned to the blood 
bank. 

o   Preventable adverse events - A subset of adverse outcomes that are judged 
to have been avoidable if appropriate and reasonable steps had been taken.  
An anaphylactic reaction to penicillin is an adverse event.  It is a 
preventable adverse event if the patient’s intolerance of penicillin is noted 
in his or her chart and that chart is available to the clinician, or if the 
patient knows of his or her reaction to penicillin and is able to communicate 
it to the clinician.  Reporting systems that are intended to collect 
preventable adverse events must have a definition of what is considered 
preventable.
  
o   Positive sentinel events - Information may be collected about situations 
in which a patient’s care is unexpectedly successful, and investigations may 
lead to discoveries about why other patients’ care does not go as smoothly.  

o   Risk -  The likelihood that somebody or something will be harmed by a 
hazard, multiplied by the severity of the potential harm.  (See Ref # 3)

o   Sentinel events - Events in which death or serious harm to a patient has 
occurred. 

RESEARCH OBJECTIVES

Background

In late 1999, AHRQ’s reauthorization language specifically directed the 
Agency to conduct and support research and build public-private partnerships 
to (1) identify the causes of preventable health care errors and patient 
injury in health care delivery; (2) develop, demonstrate, and evaluate 
strategies for reducing errors and improving patient safety; and (3) 
disseminate such effective strategies throughout the health care industry.  
The Agency’s FY01 appropriation directs it to conduct health system 
demonstration projects that study the effectiveness of mandatory and 
voluntary reporting systems, the effectiveness of efforts to ensure providers 
tell patients or their family members when the patient had been injured by a 
medical error, and the use of information technologies, professional 
education, and other mechanisms to enhance the safety of medical care. (See 
Ref # 5)

When the President’s Advisory Commission on Consumer Protection and Quality 
in the Health Care Industry presented its Final Report to the President of 
the United States Quality First: Better Health Care for All Americans (1998), 
reduction of medical errors was listed as one of its top priorities.  Much 
attention to the issue of medical errors has arisen since the publication of 
the Institute of Medicine’s (IOM) report:  To Err Is Human in November 1999 
(See Ref #6), and the subsequent articles in the medical literature and the 
popular press. The Institute of Medicine highlighted one critical aspect of 
health care quality -- the ability of the system to render care to the 
patient without causing injury in the process.  The IOM’s report synthesized 
the available evidence on patient safety and noted that medical errors are a 
leading cause of death and injury. (See Ref # 6)  It called for rapid and 
sustained action to reduce the incidence of patient injury and death due to 
mistakes in the care that was given.  The IOM recommended that actions to 
improve safety be based on information about the underlying causes of the 
errors.  The IOM’s report acknowledged there is a need for reporting systems 
that aim to collect information on medical mistakes, such as the Joint 
Commission for the Accreditation of Healthcare Organization Sentinel Event 
reporting system, state-based reporting systems, the Food and Drug 
Administration’s Adverse Drug Event Reporting System, and the Centers for 
Disease Control and Prevention’s National Nosocomial Infection Surveillance 
System.  One researcher cited by the IOM notes that “error in medicine is 
real and common”. (See Ref # 7) 

The IOM concluded that, in the main, errors are the result of problems in the 
system of health care delivery, not the fault of incompetent or malicious 
individuals.  The IOM outlined four critical steps in reducing the risk to 
patients from errors in health care delivery.  They are:  1)  create a 
national focus on medical error and institute leadership for improving care; 
2) develop systems to collect information on errors that are occurring and 
learn from those mistakes to reduce errors for other patients;  3)  raise 
expectations about the performance of the health care system; and 4)  
institute safer practices in the delivery of health care.  

Within days of the IOM report’s issuance, President Clinton called on the 
Quality Interagency Coordination Task Force (See Ref # 8)  to review the 
report and to tell him what the Federal agencies could do to reduce health 
care errors. The QuIC sent its report back to the President, Doing What 
Counts to Improve Patient Safety:  Federal Efforts to Reduce Medical Errors 
and Their Impact, in February 2000, (a QuIC report citation) , 
http://www.quic.gov and it articulated more than 100 actions the Federal 
agencies could take to reduce health care mistakes.  The QuIC’s report was 
framed in terms of the IOM’s recommended four steps to improving patient 
safety.  Of particular note for the purposes of this Request for 
Applications, the QuIC promised that research would be conducted to identify 
the most effective methods for collecting and using information on the errors 
currently occurring in the health care system to identify safety hazards.  
From the example of other industries and some preliminary work in health 
care, it is clear that having a good base of information that leads to the 
identification of safety hazards is necessary, but not sufficient.  The QuIC 
also promised that research would be conducted to study potentially effective 
methods of reducing errors and preventing harm to patients.  In addition, the 
QuIC’s report recognized that the patient who is harmed by an error or his or 
her family members have the right to be told when an error has occurred that 
has resulted in harm to the patient.  This RFA also seeks to test effective 
methods of communicating that information.  

Thus, the focus of this solicitation is the evaluation of error reporting 
systems that can collect and analyze data to identify risks to patient safety 
and develop effective methods of providing that information to providers, 
purchasers, policymakers and the public who can use it to improve safety 
through interventions like education, use of incentives, reorganization of 
work, introduction of new tools, the redesign of processes, changes in 
policies or purchasing strategies, or through their own choice of providers.   
AHRQ is specifically interested in the following questions:

o   What information is most useful in preventing injuries to patients? 
o   What types of reporting systems are best for collecting the needed 
information? 
o   What types of analyses can turn the reported data into useful information 
for improving patient safety (e.g., root cause analyses of single incidents, 
data sorting for common causes, human factors analyses)?
o   What reports can be produced, how reports should be formatted and 
transmitted, and to whom should they be sent to prompt action that will 
reduce the risk of avoidable harm to patients?  
o   What type of information is most useful to providers in informing their 
actions to alter health care delivery practices, processes, procedures, and 
systems to reduce risk to patients?
o   What type of information is most useful to those providing public 
oversight so that they can act to reduce risk to patients?
o   What type of information is most useful to individuals who wish to act to 
reduce their own risk and to those who may act on behalf of a large group of 
individuals, such as purchasers?
o   What protections are effective in protecting the privacy and 
confidentiality of the data from unintended disclosure?
o   What information is most useful to those who have been injured during 
their medical care and how can that information best be delivered?  
Particularly, what errors do patients want to know about? 

The IOM and many who have reviewed To Err is Human have suggested that to 
improve patient safety, the United States would need a combination of 
mandatory and voluntary medical error reporting systems.  The IOM cited as a 
model for safety improvement the Aviation Safety Reporting System (ASRS) and 
its companion mandatory reporting of crashes to the FAA.  The ASRS collects 
information on perceptible missteps in the process that seemed to signal a 
threat to safety.  Some may be events that result in no harm, while others 
might be near misses or result in only minor harm.  Reporting systems have 
proven to be extremely useful in error management in a number of industries 
including aviation (See Ref # 9), nuclear power (See Ref # 10), and 
petrochemicals (See Ref # 4).  The data collected by these and other safety 
reporting systems are, in fact, not always reports of events or errors. The 
data collected by the FAA are adverse events -
-- crashes.  Experts analyze the physical evidence and the reports to 
determine if an error occurred and to determine the nature of the error.  

To improve health care safety, reports of errors may not be the most useful 
information to collect, analyze and report to the public to enhance their 
decision-making or to providers to prompt changes in their systems and 
process.  Safety hazards can be identified from the collections of adverse 
events or sentinel events, adverse outcomes, indicators that there has been a 
clinical response to an adverse event or from surveys of those who work in 
the system of care and who can identify dangerous situations.  Demonstration 
projects supported under this RFA are intended to explore what information 
can be collected, how it can be used to identify the problems that are 
leading to preventable adverse outcomes, and to develop and test changes that 
may reduce the number of patients harmed by errors in the care they receive.  
These demonstrations may include the collection of information on: medical 
error, adverse events, adverse outcomes, clinical responses to errors, 
dangerous situations, hazard, near misses, no harm events, preventable 
adverse events, positive sentinel events, sentinel events, and other data 
that will lead to the discovery of underlying systems problems that heighten 
the risk to patients.

It is likely that the analysis of any of the aforementioned types of reports 
can be useful in improving the safety of the health care system if they all 
lead to the discovery of underlying system problems. There is some evidence 
to suggest that they do.  Nagel (See Ref #12) has indicated that the 
underlying causes of near miss events in commercial aviation are very similar 
to those associated with full-blown disasters.  Kaplan, Battles, and Mercer 
(See Ref # 13) confirm this finding indicating that there was no difference 
in the underlying causes for near miss and adverse events in transfusion 
medicine. 

To date, very little research has focused on errors or adverse events 
occurring outside hospitals, for example, in office practices or ambulatory 
care clinics.  Methodological challenges to this type of research include the 
lack of reliable ambulatory documentation of medical decisions and inadequate 
knowledge about the types of errors that are clinically important and most 
common in ambulatory settings.  Optimally, this research needs to be 
conducted in multiple ambulatory sites, since findings from such studies may 
be significantly influenced by particular practice settings.  

Collaborative networks of office-based practices or clinics with efficient 
and reliable data systems have the potential to overcome many barriers to the 
study of medical errors and patient safety in ambulatory settings.  Network 
study populations are likely to be representative of the general public and 
the clinicians involved in the network can serve as investigators or 
consultants to help define common and clinically important errors or adverse 
events that occur in their practices.  AHRQ has recently supported the 
planning and development efforts of nineteen primary care practice-based 
research networks (see RFA HS-00-004, PBRNs), a few of which have developed 
electronic data systems dedicated to collecting, aggregating and reporting 
primary research data.  Other networks across the country, including networks 
composed of the ambulatory practices of specialty clinicians, also may have 
the capacity to conduct research on reporting systems for medical errors and 
patient safety and to test models of data analysis.

Other useful analyses of medical event reporting have been done.  Barach and 
Small (See Ref # 14) reviewed the literature on non-medical event reporting 
systems for clues to their success.  Battles et. al. (See Ref # 15) 
identified what they believe to be ideal attributes of medical event 
reporting systems.  Kaplan et. al. (See Ref # 13) have described a 
methodology to determine underlying causes of errors or events that can be 
used with medical event reporting systems.  Riley and Flowers (See Ref # 16) 
have reviewed the attributes of the current state of reporting systems and 
provided a general description and set of data on the number of reports and 
types of information that is collected.  A more in depth study of the most 
robust of these state reporting systems is underway and likely to be released 
at the end of February 2001.   

In addition to the collection of information, various models of analysis of 
the data are to be tested by the successful applicants to this RFA.  In some 
of the current systems, reporters are asked to identify the underlying causes 
or potential causes of errors.  In others, including the aviation reporting 
systems, those filing reports are not asked to identify what the error was or 
its root cause, because they may not know or be able to know from their 
vantage point.  For example, a pilot may report that while taxiing at an 
airport, his plane was shaken by a jet blast from another nearby airplane.  
The reporting pilot is not expected to know if the incident was the result of 
a misstep on his part, that of the other pilot, or the traffic controller, or 
if it was the result of some sort of malfunction.  He simply reports the 
incident that is a symptom of an error of some sort.  Those investigating the 
incident are expected to piece together all the data and to identify 
precisely what the error was and to understand if it is likely to be an 
isolated event that is unlikely to reoccur or something that requires broader 
attention.  If it is the latter, then information on what the error was, the 
underlying causes of the error and recommended steps to be taken to reduce 
the potential for reoccurrence are disseminated to airlines, airport 
personnel, and others who are able to act on it.  The public is most likely 
to be told what the underlying cause is and what steps have been taken to 
reduce the likelihood that they will be imperiled.  

A particular area for investigation within the context of this RFA is to 
understand what information should be given to the patients who are injured 
or their family members.  A recent article in the Times - Picayune (See Ref # 
17) about the results of the mandatory communication of information to people 
who had been injured by their care at the Veterans’ Administration Hospital 
in Lexington, KY suggests that patients and the organization both benefit 
when providers informed their patients of the errors.  A recent survey 
conducted by the Kaiser Family Foundation and AHRQ indicates that 73% of  
Americans favor some public disclosure of patient safety events. (See Ref # 
18) 

What patients want to know once an injury has been sustained may be different 
than what they want to know in general about the safety of the providers, 
hospitals, and other health care delivery organizations they use.  Although 
not directly related to errors, Levinson and Miller (See Ref # 19) have 
suggested that patients may not want to know some things about their 
provider, even when those things may affect the care they receive.  
Discovering more about what people want to know about patient safety before 
an encounter with a clinician or health care organization, and what they want 
to know if they (or a loved one) are injured is part of the purpose of this 
RFA.

SPECIAL REQUIREMENTS

Applicants must be able to demonstrate that within their own organization, or 
the team formed for purposes of this demonstration, they can: 1) collect, 
analyze, and communicate information that will help to identify risks from 
medical care in health care systems, 2) test applications of potentially 
safer approaches to the delivery of care, and 3) document the effectiveness 
of the data collection system in collecting and using information, and 
testing interventions.  The applicant must be able to document that the data 
collected will be from more than 10 hospitals, health delivery organizations, 
clinics, nursing homes or other health care delivery sites.  Partnerships 
with other entities are encouraged under this RFA to ensure that the 
applicant has the capability to accomplish all of the research aims.  This 
may include partners with established research capacity in data collection or 
patient safety research, such as the existing state data collection agencies, 
Peer Review Organizations funded through the Health Care Financing 
Administration, the Veterans Affairs Centers of Safety, and AHRQ’s seven 
Centers for Education on Research and Therapeutics (CERTs). 

To achieve the goals that have prompted AHRQ to issue this solicitation, all 
applicants must be able to show that:

1.  They can collect data on medical errors, preventable adverse events, or 
other aspects of patient safety in hospitals, office-based practices, or 
other health care settings - Applicant organizations need to demonstrate that 
they have an existing mechanism for collecting data to improve patient safety 
or could have such a mechanism fully functional by July 1, 2001.  This 
mechanism must, at a minimum, be able to collect information that will help 
to improve patient safety either through public disclosure that helps people 
choose safer health care providers or through feedback about hazards to 
providers or oversight organizations.  

o   Specify the type of errors they will be collecting, the mechanisms they 
will use to collect the information, the potential deterrents to reporting 
that they anticipate, and how they will seek to overcome the deterrents.  The 
Institute of Medicine suggested that mandatory and voluntary reporting 
systems for medical errors needed to co-exist, serving somewhat different 
purposes in the effort to improve health care safety.  The Department of 
Veterans Affairs has instituted both a mandatory and a voluntary error 
reporting system for its facilities, which collect somewhat different 
information and are used in different ways to improve safety.  AHRQ is 
interested in studying the strengths of mandatory and voluntary reporting 
systems.  It will seek a balance among mandatory and voluntary reporting 
systems, and aims to fund at least three projects that include mandatory 
reporting and at least three that include voluntary reporting.  The applicant 
will need to state whether its system is mandatory, voluntary, or has 
elements of both (see Definitions). 

o   In States that have mandatory error reporting systems, must describe how 
their system will interact with the mandatory system.  Such applicants are 
encouraged to integrate representatives of the state reporting system into 
their team.

o   To the extent that the applicant institutions anticipate collecting data 
on errors related to drugs, devices, or blood safety, or to nosocomial 
infections, the applicant institution must specify how it will ensure that 
appropriate data are reported to the Food and Drug Administration (drugs, 
devices, and blood product safety) and the Center for Disease Control’s 
National Nosocomial Infection Surveillance System.  

o   Data collection strategies may include incidence reporting, near miss 
reporting, and/or surveillance and reporting systems for common types of 
errors.

2.  They can analyze the collected data to identify risks or vulnerabilities 
in the delivery of care that increase the likelihood of harm to patients.  
Reporting systems from other industries include analyses that use both 
individual incident analyses (a.k.a., root cause analysis), data coding and 
sorting strategies to look for commonalities, and other approaches.  These 
analyses in other error reporting systems often include individuals with 
expertise in a variety of disciplines, such as human factors analysts, 
epidemiologists, and experienced practitioners of the relevant professions.  
Applicants must specify what techniques they will use to analyze the data, 
how they will ensure that the data are protected from unintended disclosure, 
and what types of information will be available from their analyses.  They 
must also indicate what types of expertise will be involved in their analyses 
and what they expect that expertise to bring to the understanding of the 
data.  

3.  They can protect data from unintended disclosure.  These reporting 
systems contain very sensitive information.  Some of the information might 
lead to the identification of personal health information about a particular 
patient.  Some will include identifying information for health care personnel 
and institutions.  In all cases, applicants will need to describe what steps 
are being taken to prevent the disclosure of personal health information 
about individual patients.  In addition, applicants will need to describe 
what information, if any, will be publicly accessible, how it will be made 
accessible, and how they will work to ensure it is useful and presented in a 
usable format to the public.  In addition, the applicants need to describe 
what steps they are taking to prevent unintended disclosure of information 
about medical personnel or facilities.  These must be in compliance with the 
privacy regulations created in accordance with the Health Insurance 
Portability and Accountability Act.  The final Privacy Rule was published in 
the Federal Register on December 28, 2000 (65 FR 82462) and is available at 
http://www.hhs.gov/ocr/hipaa.html.  Information about compliance with the 
regulations will be made available at the technical assistance meeting on 
March 16, 2001. (See APPLICATION PROCEDURES of this RFA, under section 
“Workshop for Prospective Applicants,” for information on this meeting.) 
 
4.  They can report the information arising from the data to those who need 
the information.   The applicant must show that it can feed back information 
from its analysis to providers, leaders of provider organizations, 
appropriate policy makers, the public, or others who have the capacity to act 
on the information.  AHRQ is interested in organizations that will 
demonstrate a variety of different feedback mechanisms, and will look for 
innovative methods that are likely to capture the attention of the decision-
makers.  It will give preference to applicants that have commitments from 
healthcare providers, delivery organizations, and others to use the 
information.  Such potential users should describe what process they will use 
to assess the information, determine appropriate action, and then act on the 
information received. 

5.  The information can be used to change current practices to foster safer 
health care - No reporting system is useful unless that information emerging 
from the system is actually used to reduce the risk to patients.  This might 
be done through changes in the delivery system, selection of safer health 
care providers for care of individuals, or the use of appropriate 
technologies to reduce medical errors (such as hand-held electronic 
medication and specimen management systems and prescription pads, training 
simulators for medical education, bar-coding of prescription drugs, patient 
bracelets, and automated dispensing of medication in a hospital setting), or 
other means. 

o   Applicants intending to change the delivery of care through their data 
collection and feedback must describe how they will use the information to 
change practice through the training of health professionals, implementation 
of new technologies, and other methods to reduce the probability of future 
medical errors.  They must show how they will track what provider 
organizations are doing with the information emerging from the reporting 
system to alter practice and affect patient outcomes.

o   Applicants intending to improve safety by providing better information to 
the public, purchasers, or oversight organizations must describe how they 
will use the information to educate the public or other audiences, what types 
of behavior changes they are expecting from the provision of the information 
and how they will track what types of changes actually occur and the effect 
on patient safety.

o   All applicants must describe how they will test the results of the users 
attempts to improve patient safety, and how they use feedback from their 
users to improve the information that they make available.

6.  Ambulatory care research network applicants must, in addition to 
fulfilling the other requirements of this RFA, document in their applications 
that they meet, at minimum, the following qualifications:

o   The network includes a core of at least 10 ambulatory practices or 
clinics located in the U.S. and devoted principally to the primary or 
specialty care of patients.

o   The network has an accepted statement of its purpose and research mission 
that includes an ongoing commitment to the research endeavor.

o   A director, with appropriate support staff, is responsible for most 
administrative and financial functions.

o  An organizational structure exists that transcends a single study, 
including multiple systems of communication with and among participating 
practices in the form of regularly produced newsletters, e-mail or list-
serves, conference calls, and/or face-to-face meetings of various 
combinations of network members 

Policy Relevance and Dissemination

Studies under this RFA are expected to contribute to our basic understanding 
of effective methods of collecting and analyzing information that will lead 
to increases in patient safety.  Additionally, it is expected that they will 
help to build capacity to answer associated questions of applied and policy 
relevance and to produce information in formats useful to participants in the 
formulation of public policy and changing operational behavior.  Applicants 
should be concrete in describing (1) the organizations most likely to use the 
results of their analyses of the collected information to improve patient 
safety and (2) how applicants anticipate their results being used for public 
policy purposes.   Proposed information dissemination strategies should not 
be limited to publication in peer-reviewed journals, but may encompass a 
variety of approaches, such as distribution of information through 
associations of private and public officials; educating health organizations 
executives, providers, payors and others in seminars; and outreach to local 
media.

Data Privacy

All information about identifiable individuals or organizations obtained for 
the research purpose of carrying out a demonstration project supported 
pursuant to this RFA under AHRQ’s statutory authority is protected by a 
federal confidentiality statute, section 924 of the Public Health Service Act 
, 42  USC 299c-3(c).  Under this law, the only permissible uses or 
disclosures of the information are those agreed to by the subject individuals 
and organizations or by those who supply the information to the researchers.  
There are substantial statutory penalties (up to $10,000) for improper use or 
disclosure of this information.  Thus, 42 USC 299c-3(c) provides a Federal 
statutory basis for resisting any Federal or state court order or subpoena to 
the extent that carrying out the order or subpoena would violate the 
protective restrictions of  the statute.  Upon request, AHRQ will provide 
legal assistance in defending adherence to these statutory confidentiality 
protections with respect to identifiable data obtained in carrying out Agency 
research activities.  

Note that identifiable data obtained by AHRQ-supported researchers is 
protected by this law when it is obtained to carry out AHRQ-supported 
research, but this law would not protect against access to this same 
information in medical records  if it were gathered and  recorded there for 
other purposes.  If providers collect certain information solely for purposes 
of this research, it should be so marked and if necessary to protect its 
confidentiality, kept separately.  Note also that this law does not protect 
or restrict aggregate and nonidentifiable statistical or data analysis and 
conclusions developed by the researchers that would not disclose information 
about any identifiable individuals or establishments

Pursuant to section 924(c) of the Public Health Service Act (42 USC 299c-
3(c)), information obtained in the course of any AHRQ-study that identifies 
an individual or entity must be treated as confidential in accordance with 
any promises made or implied regarding the possible uses and purposes of the 
data collection.  In the Human Subjects section of the application, 
applicants must describe procedures for ensuring the confidentiality of such 
identifying information.  The description of the procedures should include a 
discussion of who will be permitted access to the information, both raw data 
and machine readable files, and how personal identifiers and other 
identifying or identifiable data will be restricted and safeguarded.

The grantee should ensure that computer systems containing confidential data 
have a level and scope of security that equals or exceeds those established 
by the Office of Management and Budget (OMB) in OMB Circular No. A-130, 
Appendix III - Security of Federal Automated Information Systems.  The 
National Institute of Standards and Technology (NIST) has published several 
implementation guides for this circular.  They are: An Introduction to 
Computer Security: The NIST Handbook; Generally Accepted Principals and 
Practices for Securing Information Technology Systems; and Guide for 
Developing Security Plans for Information Technology Systems.  The circular 
and guides are available on the web at 
http://csrc.nist.gov/publications/nistpubs/800-12/handbook.pdf.

The application of these confidentiality and security standards to 
subcontractors and vendors should be addressed as necessary.
Rights in Data

AHRQ grantees may copyright or seek patents, as appropriate, for final and 
interim products and materials including, but not limited to, methodological 
tools, measures, software with documentation, literature searches, and 
analyses, which are developed in whole or in part with AHRQ funds.  Such 
copyrights and patents are subject to a worldwide irrevocable Federal 
government license to use and permit others to use these products and 
materials for government purposes.  In accordance with its legislative 
dissemination mandate, AHRQ purposes may include, subject to statutory 
confidentiality protections, making research materials, data bases, results, 
and algorithms available for verification or replication by other 
researchers; and subject to AHRQ budget constraints, final products may be 
made available to the health care community and the public by AHRQ or its 
agents, if such distribution would significantly increase access to a product 
and thereby produce public health benefits.  Ordinarily, to accomplish 
distribution, AHRQ publicizes research findings through the media and other 
appropriate channels in coordination with AHRQ’s Office of Health Care 
Information but relies on grantees to publish research results in peer-
reviewed journals and to market grant-supported products.

Important legal rights and requirements applicable to AHRQ grantees are set 
out or referenced in the AHRQ’s grants regulation at 42 CFR Part 67, Subpart 
A (Available in libraries and from the GPO’s website 
http://www.access.gpo.gov/nara/cfr/index.html).

INCLUSION OF WOMEN, MINORITIES, AND CHILDREN IN RESEARCH STUDY POPULATIONS

It is the policy of AHRQ that women and members of minority groups be 
included in all AHRQ-
supported research projects involving human subjects, unless a clear and 
compelling rationale and justification are provided that inclusion is 
inappropriate with respect to the health of the subjects or the purpose of 
the research.  

All investigators proposing research involving human subjects should read the 
UPDATED "NIH Guidelines on the Inclusion of Women and Minorities as Subjects 
in Clinical Research," published in the NIH Guide for Grants and Contracts on 
August 2, 2000 
(http://grants.nih.gov/grants/guide/notice-files/NOT-OD-00-048.html); a 
complete copy of the updated Guidelines are available at 
http://grants.nih.gov/grants/funding/women_min/guidelines_update.htm: To the 
extent possible, AHRQ requires adherence to these NIH Guidelines.

Investigators may obtain copies from the above sources or from the AHRQ 
contractor, listed under INQUIRIES, or from the NIH Guide Website
http://grants.nih.gov/grants/guide/notice-files/NOT-OD-00-048.html.

AHRQ also encourages investigators to consider including children in study 
populations, as appropriate.  

AHRQ program staff may also provide additional information concerning these 
policies (see INQUIRIES). 

TERMS AND CONDITIONS OF AWARD

These special Terms of Award are in addition to and not in lieu of otherwise 
applicable OMB administrative guidelines, HHS grant administration 
regulations at 45 CFR Parts 74 and 92, 42 CFR Part 67 Subpart A, and other 
HHS, PHS grants administration policy statements.  Applicants should be 
familiar with the Agency’s grant regulations, 42 CFR Part 67 Subpart A, and 
particularly sections 67.18-67.22.  
 
The administrative and funding instrument to be used for this program will be 
a cooperative agreement (U18), an "assistance" mechanism (rather than an 
"acquisition" mechanism), in which substantial AHRQ scientific and/or 
programmatic involvement with the awardee is anticipated during performance 
of the activity.  Under the cooperative agreement, the AHRQ purpose is to 
support and/or stimulate the recipient's activity by involvement in and 
otherwise working jointly with the award recipient in a partner role, but it 
is not to assume direction, prime responsibility, or a dominant role in the 
activity. Consistent with this concept, the dominant role and prime 
responsibility for the activity resides with the awardee(s) for the project 
as a whole, although specific tasks and activities in carrying out the 
studies will be shared among the awardees and the AHRQ Program Official(s).
 
1.   Awardee Rights and Responsibilities:   

Awardees will have primary and lead responsibilities for the project as a 
whole, including research design and protocol development, participant 
recruitment and follow-up, data collection, quality control, preparation of 
publications exclusively about the data collection, analysis, information 
dissemination and testing of interventions to improve safety within their 
demonstration project.  They will be responsible for cooperating and 
collaborating with other AHRQ awardees working on patient safety improvement 
with assistance from the AHRQ Program Official(s) and the coordinating center 
(described below).   In addition, awardees are required to cooperate with 
AHRQ’s evaluation contractor that will be assessing the relative 
effectiveness of the different demonstrations funded under this project.  
 
Awardees will retain custody of and have primary rights to the data developed 
under these awards, subject to Government rights of access consistent with 
current Department of Health and Human Services, Public Health Services, and 
AHRQ policies.
 
Awardees must work with AHRQ and the coordinating center to develop a common 
vocabulary for use across all the funded demonstration projects under this 
RFA.  In addition, awardees must work with AHRQ and the coordinating center 
to develop a common method that will be used across all awardees to calculate 
risk for patient injury.  This method will be used to assess each system’s 
effectiveness in reducing risk for patient injury, and so comparisons can be 
made among various interventions that seek to reduce risk of injury. 

2.   AHRQ Staff Responsibilities 
 
The AHRQ Program Officials for this program include the Director of the 
Center for Quality Improvement and Patient Safety (CQuIPS) and the project 
officers for the funded cooperative agreements.  They will have substantial 
scientific-programmatic involvement in all matters related to developing 
common vocabulary, common evaluation methods, and data collection that will 
enable collaboration and comparisons of the relative strengths and weaknesses 
of each funded reporting system.  The dominant role and prime responsibility 
for the activity resides with the awardee(s) for the project as a whole, 
although specific tasks and activities in carrying out the studies will be 
shared among the awardees and the AHRQ Program Officials.   AHRQ program 
officials may participate in the analysis of data and author or co-author 
articles that compare two or more of the awardees reporting systems or the 
interventions to improve patient safety.  

AHRQ expects award recipients to work with each other to test the 
effectiveness of various methods of collecting, analyzing, and reporting 
information on patient safety.  To enable these comparisons, AHRQ and the 
coordinating center (described below) will work with the awardees to develop 
a common method for calculating risk for patient injury so that each system’s 
effectiveness in reducing risk for patient injury can be determined, and so 
that comparisons can be made among various interventions that seek to reduce 
risk of injury.  AHRQ also expects awardees to work to enhance the ability of 
an independent evaluator to review the projects and synthesize information 
from the demonstrations about which methods for data collection, analysis, 
and communication are most effective in improving patient safety or reducing 
risks to patients.  

Specifically, the (CQuIPS) within AHRQ will promote collaboration and 
comparison of information gathered by the demonstrations, the sharing of 
analytic methods and results, and information on effective uses of the 
collected data to reduce harm to patients.  AHRQ will contract with a 
coordinating center to facilitate the sharing of information and to 
facilitate collaborations among recipient demonstration projects.   In 
addition, the AHRQ-supported coordinating center will facilitate 
communication and sharing of ideas between the demonstration projects and the 
Agency’s funded Centers of Excellence in Patient Safety Research and the AHRQ 
Developing Centers of Excellence in Research on Patient Safety.  These 
Centers will be charged with testing promising ideas for improvements in 
patient safety.  A complete description of the purpose of these Centers can 
be found in the requests for applications for each, which are available at 
http://www.ahrq.gov/fund.

In addition, AHRQ intends to contract with an organization to conduct an 
evaluation of the demonstration projects.  This evaluation organization will 
not be affiliated with any of the demonstration projects or Centers.  It will 
be responsible for assessing the strengths and weaknesses of each approach to 
reporting.  AHRQ will work with the demonstration projects to ensure they are 
collecting information that is necessary for the evaluation.  AHRQ expects 
that all funded demonstration projects will cooperate with the evaluation 
contractor to provide needed information.

3.   Steering Committee  

A Steering Committee, composed of the principal investigator of each grant, 
the director of the Coordinating Center, and AHRQ Program Officials will be 
the main governing board of the study and will have primary responsibility 
for developing common vocabulary, research designs, facilitating the 
monitoring of studies, and reporting results of comparisons between and among 
the demonstration projects (including the interventions that are tested).  
The principal investigator from each grant and the Coordinating Center, and 
the AHRQ Program Officials will each have one vote. 
 
Awardees will be required to accept and implement the common protocols and 
procedures approved by the Steering Committee.

Principal Investigators will participate in Steering Committee meetings three 
times per year and general collaborative conference calls on a schedule as 
determined by the Steering Committee. 
   
4.  Arbitration
 
Any disagreement that may arise on scientific/programmatic matters (within 
the scope of the award), between award recipients and AHRQ may be brought to 
arbitration.  An arbitration panel will be composed of three members -- one 
selected by the Steering Committee (with the AHRQ members not voting) or by 
the individual awardee in the event of an individual disagreement, a second 
member selected by AHRQ, and the third member selected by the two prior 
selected members.  This special arbitration procedure in no way affects the 
awardee's right to appeal an adverse action that is otherwise appealable in 
accordance with the PHS regulations at 42 CFR Part 50, Subpart D and HHS 
regulation at 45 CFR Part 16.

LETTER OF INTENT 

Prospective applicants are asked to submit a letter of intent by April 2, 
2001.  It should include a descriptive title of the proposed research, the 
name, address, and telephone number of the Principal Investigator, the 
identities of other key personnel and participating institutions, and the 
number and title of the RFA in response to which the application may be 
submitted.  

Although a letter of intent is not required, is not binding, and does not 
enter into consideration of any subsequent application, the information that 
it contains allows AHRQ staff to estimate the potential review workload and 
avoid conflict of interest in the review.  AHRQ will not provide responses to 
letters of intent.

The Letter of Intent is to be sent to:

Lisa Krever
Center for Quality Improvement and Patient Safety 
Agency for Healthcare Research and Quality
2101 East Jefferson Street, Suite 502
Rockville, MD 20852-4908
Telephone (301) 594-6625
FAX:  (301) 594-2155
E-mail Address: lkrever@ahrq.gov

APPLICATION PROCEDURES

The research grant application form PHS 398 (4/98) is to be used in applying 
for these demonstration grants.  State and local government applicants may 
use PHS 5161-1, Application for Federal Assistance (rev. 5/96), and follow 
those requirements for copy submission.  Applicants are encouraged to read 
all PHS 398 instructions prior to preparing an application in response to 
this RFA.  The PHS 398 type size requirements (p. 6) will be enforced 
rigorously, and non-compliant applications returned without review.   In 
part, the PHS 398 states that the application must be clear, readily legible, 
and conform to all of the following requirements: 1) the height of the 
letters must not be smaller than 10 point; 2) type density must be no more 
than 15 characters per inch (cpi); and 3) no more than 6 lines of type must 
be within a vertical inch.  The PHS 398 application is available on the web 
at http://grants.nih.gov/grants/forms.htm.

AHRQ is not using the Modular Grant Application and Award process.  
Applicants for funding from AHRQ should ignore application instructions 
concerning the Modular Grant Application and Award process and prepare 
applications according to instructions provided in form PHS 398.

Beginning with applications for AHRQ submitted for the February 1, 2001 
receipt date, Institutional Review Board (IRB) approval of human subjects is 
not required prior to peer review of an application unless otherwise 
indicated by the Agency 
(http://grants.nih.gov/grants/guide/notice-files/NOT-HS-00-003.html.  
Given the nature of this project and the speed with which 
successful applicants will need to begin work, the Agency has determined that 
IRB approval is required prior to peer review of all applications submitted 
in response to this RFA.  All investigators/applicants proposing research 
involving human subjects should pay particular attention to the instructions 
in the form PHS 398 regarding human subject involvement.
  
Application kits are available at most institutional offices of sponsored 
research.  They may also be obtained from the Division of Extramural Outreach 
and Information Resources, National Institutes of Health, 6701 Rockledge 
Drive, MSC 7910, Bethesda, MD 20892-7910, telephone (301) 435-0714, E-mail:  
grantsInfo@nih.gov.

AHRQ applicants may also obtain application materials from the AHRQ 
Publications Clearinghouse (see INQUIRIES).

The RFA label and line 2 of the application form PHS 398 (4/98) should both 
indicate the RFA number.  The RFA label must be affixed to the bottom of the 
face page.  Failure to use this label could result in delayed processing of 
the application such that it may not reach the review committee in time for 
review.  In addition, the RFA title must be typed on line 2 of the face page 
and the YES box must be marked.  The sample RFA label available at 
http://grants.nih.gov/grants/funding/phs398/label-bk.pdf has been modified to 
allow for this change.  Please note this is in pdf format.

The completed and signed, typewritten original of the application, including 
the Checklist, and three signed, photocopies, must be submitted in one 
package to:
 
Center for Scientific Review
National Institutes of Health
6701 Rockledge Drive, Room 1040 - MSC 7710
Bethesda, MD  20892-7710
 (20817 for express service)
 
At the time of submission, two additional copies of the application, each 
labeled “Advance Copy” must also be sent to:

Lisa Krever
Center for Quality Improvement and Patient Safety 
Agency for Healthcare Research and Quality
2101 East Jefferson Street, Suite 502
Rockville, MD 20852-4908
Telephone (301) 594-6625
FAX:  (301) 594-2155
E-mail Address: lkrever@ahrq.gov

Applications must be received by April 27, 2001.  An application received 
after the deadline may be acceptable if it carries a legible proof-of-mailing 
date, assigned by the carrier, and the proof-of-mailing is not later than 1 
week prior to the deadline date.  If an application is received after that 
date, it will be returned to the applicant without review.  

Application Preparation

For applications that propose to use Medicare or Medicaid data that are 
individually identifiable, applicants should state explicitly in the 
"Research Design and Methods" section of the Research Plan (form PHS 398) the 
specific files, time periods, and cohorts proposed for the research.  In 
consultation with HCFA, AHRQ will use this information to develop a cost 
estimate for obtaining the data.  This estimate will be included in the 
estimated total cost of the grant at the time funding decisions are made.  

Applicants should be aware that for individually identifiable Medicare and  
Medicaid data, Principal Investigators and their grantee institutions will be 
required to enter into a Data Use Agreement (DUA) with HCFA to protect the 
confidentiality of data in accordance with OMB Circular A-130, Appendix 
III--Security of Federal Automated Information Systems.  The use of the data 
is restricted to the purposes and time period specified in the DUA.  At the 
end of this time period, the grantee is required to return the data to HCFA 
or certify that the data have been destroyed.  Grantees must also comply with 
the confidentiality requirements of Section 903(c) of the PHS Act.

In carrying out its stewardship of human resource related programs, the AHRQ, 
at some point in the future, may begin requesting information essential to an 
assessment of the effectiveness of Agency research programs.  Accordingly, 
grant recipients are hereby notified that they may be contacted after the 
completion of awards for periodic updates on publications resulting from AHRQ 
grant awards, and other information helpful in evaluating the impact of 
sponsored research.  AHRQ expects grant recipients to keep the Agency 
informed of publications or the impact from Agency sponsored research.

Complete information about the proposed strategy for data collection, 
analysis, dissemination of information, and testing of interventions to 
improve safety must be submitted with the application. The narrative portions 
of the Demonstrations application, described below under “Research Plan,” 
should be limited to twenty-five pages of text. The application should be a 
complete document that includes all essential information necessary for its 
evaluation. While additional explanatory material may be submitted as 
appendices, such appendices should not be used to bypass page limitations in 
the application because only selected reviewers will receive copies of the 
appendices. Since the form PHS 398 was developed for research grant 
applications, the following supplemental instructions should be used as a 
guide in the preparation of the application. 

1.   Detailed Budget.  In general, budget items for these research 
demonstration projects can include personnel costs, consultant costs, 
supplies, travel expenses for the planning director and other key personnel 
to attend up to 3 meetings each year with the AHRQ-supported coordinating 
center, and other justifiable operating expenses for the project.  The 
purchase of equipment is discouraged, but with strong justification funds may 
be approved for this purpose.  AHRQ plans to separately fund the coordinating 
center and an evaluation contractor.  Applicants may request funds for 
expenses that they anticipate incurring for collecting needed data, 
communicating, and other related expenses for interaction with the 
coordinating center and evaluation contractor. 

2.   Resources available to the research demonstration project should be 
described once.  Applicants should follow instructions provided in the PHS 
Form 398, under Resources.

3.   Research Plan. This narrative part of the application should contain the 
following elements:

a)   Description of the purpose of the reporting system.  Specify who the 
intended audience(s) is (are) for the information that is developed by the 
reporting system, what types of information the audience will receive, and 
what types of actions to improve patient safety the information is intended 
to induce.  If more than one audience will receive information (e.g., 
providers and the public), indicate what information each will receive and 
what types of actions might be taken by each.

b)   Description of the data collection.  Describe the elements of the data 
collection that will form the basis of this demonstration.  Specify the type 
of information collected; the kinds of health delivery organizations on which 
data are collected; who can report data; what, if any, mandates or incentives 
to report exist; and, any proposed expansions or enhancements of the data 
collection to further the goal of improving patient safety.  Describe the 
infrastructure that supports the data collection and any proposed 
enhancements that will be needed to facilitate the improvements in patient 
safety sought by the applicant as part of this project.  This discussion 
should include details of any computer-based or other information systems 
currently in use (or planned) to collect and aggregate research data.

c)   Description of the data analysis.  The proposal should indicate how the 
data will be sorted and how risks to patient safety will be identified.  The 
discussion should also include references to the existing, or proposed, 
mechanisms for obtaining advice and feedback from those using the information 
so that their analyses can be improved over time. 

d)   Description of information dissemination strategies. Indicate how 
information is to be transmitted back to those audiences that are to act on 
it.  Indicate how frequently the information is to be transmitted, what form 
it will take, whether there are opportunities for audience members to gain 
greater clarity or insight into the information.  If more than one audience 
is to receive information, describe what information each receives and how it 
will receive it.

e)   Evaluation of reporting system success.  Describe what data collection 
exists or will be created as part of this application to know whether or not 
the types of intended actions have been taken and what their impact has been. 

f)   Testing of improvement strategies.  If the reporting system is intended 
to induce change in health care practice, processes, tools, or systems of 
care, indicate how the applicant team will test improvement strategies, 
including professional education. 

g)   Description of key personnel and infrastructure for the project. The 
project director and other key staff should be identified, including a 
description of their qualifications and sources of salary support.  

Note that letters of intent to collaborate with the applicant organization 
signed by the appropriate institutional official from each participating 
organization must be included in the application. 

In addition, the Agency is interested in examining effective methods for 
health care delivery systems and the providers of care who work within them 
to communicate with the patient or his or her family member when the patient 
has been harmed by a medical error.  Applicant organizations are encouraged 
to specify how they will provide this communication and assess the reaction 
of the patients, family members, health care professionals, and health care 
organizations to the sharing of this information.  Funding priority will be 
given to organizations proposing potentially effective methods for telling 
patients and their family members and assessing the impact.  In addition, the 
Agency will seek to fund at least three proposals that include provision for 
testing methods of telling the patients or their appropriate family members 
when an error has caused harm to the patient.

If partnerships or affiliations exist or have been created for the purpose of 
fulfilling the aims of this research proposal, name all of the partners; 
specify the expertise, consultants, and other resources available to the 
network as a result of the affiliations; and, include letters of agreement to 
participate signed by individuals who have the authority to commit the 
organizations. These and other letters of support should be included as an 
appendix and referenced in this section of the application. 

h)   Progress to date in collecting and analyzing data.  Include a 
description of the database that exists (or is being assembled).  Include 
information on the kind of data included and the numbers of events currently 
in the database.  Describe any planned changes in this data collection.  
Describe how the data are analyzed, what types of reports are currently 
generated, who receives the reports. 

i)   Progress to date in inducing changes in behavior.  Describe any formal 
or informal systems or mechanisms, current or planned, to disseminate the 
results of the analyses to clinicians, policy makers, and the public.  Give 
evidence, if any is available, that the information dissemination has had an 
effect on altering the provision of care, the policies enacted, or people’s 
choices of providers or other aspects of care.   

Workshop for Prospective Applicants 

AHRQ plans to convene a special technical assistance workshop to assist 
potential applicants. The purpose of this conference is to give background 
information and respond to questions of potential applicants to this RFA.  
The workshop will be held at AHRQ, 6010 Executive Boulevard, 4th Floor 
Conference Center, Room A, Rockville, MD, 20852 on March 16, 2001, from 9 
a.m. to 11 a.m.  Attendance is not a prerequisite to applying.  Attendees 
must pay for their own travel and accommodation costs.  The workshop will be 
open to any individual or organization intending to apply.  Attendees are 
encouraged to submit questions prior to the session.  A synthesis of 
pertinent Questions and Answers discussed at the prospective applicants' 
meeting will be available from the AHRQ Publications Clearinghouse, listed 
under "Inquiries" and the agency's web page.  For further information on the 
conference, contact Lisa Krever at the address listed above. 

REVIEW CONSIDERATIONS
 
Upon receipt, applications will be reviewed for completeness and 
responsiveness.  Incomplete and non-responsive applications will be returned 
to the applicant without further consideration.  The PHS 398 type size 
requirements (p. 6) will be enforced rigorously, and non-compliant 
applications returned without review. Applications that are complete and 
responsive to the RFA will be evaluated for scientific and technical merit by 
an appropriate peer review group convened by AHRQ in accordance with the 
review criteria stated below.  

As part of the merit review, all applications will receive a written 
critique, and also may undergo a process in which only those applications 
deemed to have the highest scientific merit will be discussed and assigned a 
priority score.  

General Review Criteria

Applications will be assessed in two general areas: 1) technical merit of the 
proposed demonstration of patient safety data collection, analysis, 
information dissemination, and improvement strategies; and 2) The potential 
of the demonstration to enhance its capacity to conduct patient safety 
research and translate research into safety improvements.  Peer reviewers 
will be asked to specifically comment in these two areas.  Applications that 
do not address both general areas will be returned without review.  The final 
priority scores will reflect the peer reviewers’ overall assessment based on 
their judgements of the following  two review criteria. 

1.   The technical merit of the proposed demonstration of patient safety data 
collection, analysis, and improvement strategies. 

a)   Clarity and appropriateness of objectives of the reporting system; 
b)   Likelihood the data collection system will receive a sufficient number 
of reports to enable it to fulfill its stated mission;
c)   Capacity of the reporting system to conduct appropriate analysis of the 
data and to produce useful information for the intended audiences;
d)   Appropriateness of the strategies for disseminating that information to 
its intended users and to provide necessary assistance in interpreting the 
information to those who wish to use it; 
e)   Capacity of the applicant to test various strategies for enhancing the 
use of the information by its intended audiences;
f)   In those cases where providers are a principal audience, ability of the 
applicant to test appropriate care delivery interventions to reduce risk, 
such as the introduction of computerized support, educational sessions for 
clinicians, and process redesign; 
g)   Qualifications and appropriateness of the proposed project team, 
especially the principal investigator, to conduct the proposed work;   
h)   Extent to which there is strong organizational commitment from all 
partners in the applicants’ proposed research.  This may be manifested not 
only through the letters of agreement to participate, but also through future 
financial support for the planning and research activities; to allow key 
personnel to devote time to the project; and to facilitate communication 
between the various entities;
i)   Extent to which the proposed budget for the research demonstration 
project supports the evaluation of the existing system so that its strengths 
and weaknesses can be identified, and extent to which it offers the 
opportunity to take appropriate actions to bolster the system’s 
effectiveness;
j)   Extent to which the proposed project is likely to be replicable or 
expandable to cover other organizations and other geographic areas; and
k)   Likelihood that the proposed project will help others who have or who 
are considering creating data collection systems to understand how to develop 
a system that will meet their intended objectives.

2.   The potential of the demonstration to enhance its capacity to conduct 
patient safety research and translate research into safety improvements. 

a)   Adequacy of the populations included in the demonstration project to 
support patient safety research, and the potential for research that includes 
racial and ethnic minority and/or other AHRQ priority populations;
b)   Ability of the demonstration to test education strategies to improve 
patient safety and examine and test appropriate care delivery interventions 
to reduce risk, such as the introduction of computerized support;
c)   Adequacy and stability of the demonstration’s administrative, 
organizational and management capabilities; and 
d)   Extent to which the award of grant funds will enhance the ability of the 
demonstration to conduct future activities for enhancing patient safety 
research and translating research findings into practice. 

The peer review group will also examine the appropriateness of the proposed 
project budget to perform the stated tasks. 

Additional Review Criteria for Set-aside Funds:

Applicants for the medical errors and patient safety in office-based settings 
set-aside funds should explicitly note their intent to have the application 
considered for those funds.  In addition to the review criteria above, these 
applications will be evaluated on the degree to which the project 
demonstrates a meaningful collection and analysis of ambulatory patient 
safety issues. 

AWARD CRITERIA

Applications will compete for available funds with all other U18 applications 
under this RFA. Applications deemed eligible for set-aside funds will compete 
with other eligible applications for set-aside funds, and if not funded, will 
compete with all other applications under this RFA.  The following will be 
considered in making funding decisions: 1) quality of the proposed U18 as 
determined by peer review;  2) availability of funds;  3) responsiveness to 
the goals and objectives of the RFA;  4) relevance to the formulation of 
public policy;  5) appropriate targeting of proposed projects to the  
priorities identified at the QuIC Patient Safety Research Summit; 
and 6) portfolio balance, including an emphasis on Demonstration projects in 
areas throughout the United States and attention to AHRQ’s priority areas and 
populations.

Special preference will be accorded to applications from investigators not 
recently (i.e., not within the past two years) or currently funded as 
Principal Investigator of an AHRQ grant for research on quality improvement 
strategies. In addition, preference will be accorded to those demonstrations 
which have the potential to inform efforts to ensure injured patients or 
their family members are appropriately and effectively told when they (or 
their family member) has been injured during medical care. 

To receive an award, applicants must agree to submit an original and 2 copies 
of an abstract, executive summary, and full report of the research results in 
the format prescribed by AHRQ no later than 90 days after the end of the 
project period.  The executive summary should be sent at the same time on a 
computer disk which specifies on the label the format used (WP5.1 or WP6.0 is 
preferable).

Applicants must also agree to notify Nancy Foster in CQuIPS immediately when 
a manuscript based on research supported by the grant is accepted for 
publication, and to provide the expected date of publication as soon as it is 
known, regardless of whether or not the grant award is active or has ended.

INQUIRIES
  
Copies of the RFA and copies of the grant application form PHS 398 (rev. 
4/98) are available from:

AHRQ Publications Clearinghouse
P.O. Box 8547
Silver Spring, MD 20907-8547
Telephone: 1-800-358-9295
TDD Service: 888-586-6340

This RFA is also available on AHRQ’s Web site, http://www.AHRQ.gov, and 
through AHRQ InstantFAX at (301) 594-2800.  To use InstantFAX, you must call 
from a facsimile (FAX) machine with a telephone handset.  Follow the voice 
prompt to obtain a copy of the table of contents, which has the document 
order number (not the same as the RFA number).  The RFA will be sent at the 
end of the ordering process.  AHRQ InstantFAX operates 24 hours a day, 7 days 
a week.  For comments or problems concerning AHRQ InstantFax, please call 
(301) 594-6344.

AHRQ welcomes the opportunity to clarify any issues or questions from 
potential applicants who have obtained and read the RFA.  Written and 
telephone inquiries concerning this RFA are encouraged.  Direct inquiries 
regarding programmatic issues, including issues related to the inclusion of 
women, minorities, and children in study populations to:

Nancy Foster
Center for Quality Improvement and Patient Safety
Agency for Healthcare Research and Quality
2101 East Jefferson Street, Suite 502
Rockville, MD 20852-4908
Telephone (301) 594-1609
FAX:  (301) 594-2155
E-mail Address: nfoster@AHRQ.gov

Direct inquiries regarding fiscal and eligibility matters to:

Michelle Burr
Grants Management Specialist
Agency for Healthcare Research and Quality
2101 East Jefferson Street, Suite 601
Rockville, MD  20852
Telephone:  (301) 594-1840
FAX:  (301) 594-3210
Email: mburr@ahrq.gov

AUTHORITY AND REGULATIONS

This program is described in the Catalog of Federal Domestic Assistance No. 
93.226.  Awards are made under authorization of Title IX of the Public Health 
Service Act (42 USC 299-299c-7) as amended by P.L. 106-129 (1999).  Awards 
are administered under the PHS Grants Policy Statement and Federal 
Regulations 42 CFR 67, Subpart A, and 45 CFR Parts 74 and 92.  This program 
is not subject to the intergovernmental review requirements of Executive 
Order 12372 or Health Systems Agency review.

The PHS strongly encourages all grant and contract recipients to provide a 
smoke-free workplace and promote the non-use of all tobacco products.  In 
addition, Public Law 103-227, the Pro-Children Act of 1994, prohibits smoking 
in certain facilities (or in some cases, any portion of a facility) in which 
regular or routine education, library, day care, health care or early childhood 
development services are provided to children.  This is consistent with the PHS 
mission to protect and advance the physical and mental health of the 
American people.

REFERENCES

1.  QuIC report to the President.  Doing What Counts for Patient Safety: 
Federal Actions to Reduce Medical Errors and Their Impact.  February 21, 
1999.

2.  Quality Interagency Task Force.  Report to the President: Doing what 
counts for patient safety-Federal actions to reduce medical errors and their 
impact.  February, 2000.

3.  An Organisation with a Memory: Report of an expert group on learning for 
adverse events in the NHS.  Department of Health, London, England.  The 
Stationery Office, 2000.  

4.  Van der Schaaf TW, Lucas DA, Hale AR, eds.  Near miss reporting as a 
safety tool.  Oxford: Butterworth and Heinemann. 1991.

5.  U.S.  Public Law 106-554, Section 1(a)1.

6.  Kohn LT, Corrigan JM, Donaldson MS, eds.  To Err is Human; Building a 
Safer Health System. Washington, D.C.: National Academy Press.  November 29, 
1999.

7.  Leape LL, Woods DD, Hatlie MJ, et al.  Promoting Patient Safety by 
Preventing Medical Error.  “Journal of the American Medical Association.” 
280:  1444-1447, 1998.

8.  QuIC report to the President.  The QuIC is a body consisting of the 
Federal Agencies that have health care responsibilities.  It includes 
organizations that provide, purchase, regulate, oversee, and conduct research 
on health care.  For a fuller description, see http://www.quic.gov.

9.  Reynard, W.D., Billings, C.E., Cheaney, E.S., Hardy, R. The Development 
of the NASA Aviation Safety Reporting System.  Rockville, MD: National 
Aeronautics and Space Administration Science and Technical Information Branch 
(NASA Reference Publication vol 1114), 1986.

10.  Ives G. Near miss reporting pitfalls for nuclear plan.  In: Vander 
Schaaf TW, Lucas DA, Hale AR, eds.  Near miss reporting as a safety tool.  
Oxford: Butterworth and Heinemann, 1991:51-6.

11.  Freitag, M., Hale A.  Structure of event analysis.  In: Hale A., 
Wilpert B., Freitag M., eds.  After the event: from accident to 
organizational learning.  New York, Elsevier Science Ltd. 1997; 11-22.

12.  Nagel ID.  Human error in aviation operations. In:Weiner EL, ed.  Human 
factors in aviation.  San Diego: Academic Press, Inc., 1988.

13.  Kaplan HS, Battles JB, van der Schaaf TW, Shea CE, Mercer SQ.  
Identification and classification of the causes of events in transfusion 
medicine.  Transfusion 1998 38: 1071-1081.  1998.

14.  Barach P., Small S.D.  Reporting and Preventing Medical Mishaps: 
Lessons from Non-
medical Near Miss Reporting Systems.  BMJ 2000; 320:759-763.

15.  Battles JB, Kaplan HS, van der Schaaf TW, Shea CE.  The attributes of 
medical event reporting systems.  Archives of Pathology Laboratory Medicine; 
122:3:132-138.  1998.

16.  National Academy of State Health Policymakers report (original). 

17.  Times-Picayune.  September 26, 1999.  Honesty a Policy at VA Hospital: 
Aggressive Disclosure Reduces Anger, Claims.   Andrea Gerlin.  

18.  Full Results are available at http://www.kff.org

19.  Miller TE; Horowitz CR.  Disclosing doctors' incentives: will consumers 
understand and value the information? Health Affairs;  2000 Jul-
Aug;19(4):149-55

Return to Volume Index

Return to NIH Guide Main Index

	Office of Extramural Research (OER)			National Institutes of Health (NIH) 9000 Rockville Pike Bethesda, Maryland 20892			Department of Health and Human Services (HHS)
Note: For help accessing PDF, RTF, MS Word, Excel, PowerPoint, RealPlayer, Video or Flash files, see Help Downloading Files.