CLINICAL INFORMATICS TO PROMOTE PATIENT SAFETY
 
Release Date:  February 22, 2001

RFA:  RFA-HS-01-006

Agency for Healthcare Research and Quality (formerly AHCPR)
 
Letter of Intent Receipt Date:  April 6, 2001
Application Receipt Date:       April 23, 2001

PURPOSE
 
The Agency for Healthcare Research and Quality (AHRQ) announces the 
availability of Research Demonstration and Dissemination Projects (R18) on the 
use of clinical informatics and information technology (IT) to reduce medical 
errors and improve patient safety.  Specifically, AHRQ seeks projects to 
develop and test the use of innovative technologies, such as hand-held 
electronic medication and specimen management systems, training simulators for 
medical education, computerized bar-coding, patient bracelets, smart cards, 
and automated medication dispensing systems in clinical settings.  The main 
objective of this RFA, Clinical Informatics to Promote Patient Safety (CLIPS), 
is to assess the extent to which such innovations, when applied in various 
health care settings, contribute to measurable and sustainable improvements in 
patient safety and quality of care.  Research resulting from this RFA will 
help us to better understand the opportunities and barriers to using IT to 
improve the process and outcomes of care for patients and providers.

Researchers are encouraged to propose projects that emphasize non-inpatient 
health care settings (including ambulatory and long-term care settings) and 
priority populations.  AHRQ is strongly committed to supporting research in 
priority populations, which include women, children, elderly, minority, rural, 
urban, and low-income populations, as well as patients with special health 
care needs and disabilities. Applicants will be required to develop projects 
that use rigorous scientific methods to assess the effect of the IT innovation 
on medical error rates and other measures of patient safety.  It is expected 
that a successful project will evaluate IT tools that alert providers to 
information that may be critical to the provision of high quality care, 
develop strategies to address barriers to successful adoption of innovative IT 
applications, document the costs and resources associated with the IT 
applications, or evaluate transferability to other settings.  The objective of 
this RFA is to assess the extent to which innovative IT applications, when 
applied in various health care settings, contribute to measurable and 
sustainable improvements in patient safety.  

This RFA is one in a series of research solicitations to be issued by AHRQ 
over the next several months.  The solicitations form an integrated set of 
activities to design and test best practices for reducing errors in multiple 
health care settings, develop the science base to inform these efforts, 
improve provider education to reduce errors, capitalize on the IT advances to 
translate proven effective strategies into widespread practice, and build  
capacity to further reduce errors.  In addition to CLIPS, RFAs to be released 
as part of this program include:

1)   Health System Error Reporting, Analysis, and Safety Improvement 
Demonstrations - to support large demonstrations in states, health care 
systems (including systems that integrate acute and long-term care), and 
networks of providers (e.g., integrated delivery systems and primary care 
networks) to test reporting strategies and patient safety interventions.

2)   Centers of Excellence for Patient Safety Research and Practice -  to 
support established cross-cutting teams of researchers and health care 
facilities and organizations in geographically diverse locations (including 
rural and urban areas), which will determine the causes of medical errors and 
develop new knowledge to support the work of the demonstrations.

3)   Developing Centers for Patient Safety Research and Practice - to develop 
new multidisciplinary research teams to improve the nation’s capacity in 
patient safety research, to expand the patient safety knowledge base, and to 
establish mechanisms to assure that new knowledge is incorporated into actual 
practice and that its impact is assessed. 

4)   Effect of Working Conditions on Patient Safety - to develop an 
understanding of how the environment of care impacts the ability of providers 
to improve safety (e.g. the effect of fatigue, stress, sleep deprivation, and 
shift work on cognitive ability and the relationship to patient safety) and 
how the interactions between the built environment impacts the provision of 
safe care.  

5)   Patient Safety Research Dissemination and Education - to fund researchers 
and organizations (e.g., professional associations, hospital groups, national 
organizations) to develop, demonstrate, and evaluate new approaches to 
improving provider education in order to reduce errors, such as applying new 
knowledge on patient safety to curricula development, continuing education, 
simulation models, and other provider training strategies.

At the heart of this competitive demonstration program is a portfolio of 
projects that test the effectiveness and costs of diverse reporting strategies 
and IT innovation on the identification, management, and reduction of medical 
errors.  These activities are supported and amplified by: 1)  the 
establishment of multidisciplinary centers of excellence in patient safety, 2)  
targeted efforts to understand the impact of provider education, skills, 
staffing, and organization on error rates, 3)  partnerships with health 
systems networks, professional organizations, states, and other groups to 
build capacity for error reduction activities, disseminate effective 
strategies, and coordinate public and private efforts, and 4)  cross cutting 
activities that capitalize on data already collected by  Federal or State 
governments which can be enhanced to support research and action in patient 
safety.

Additional components of the Agency’s patient safety initiative will build on 
existing Agency programs.  AHRQ will also support a coordinating center to 
provide guidance and facilitate interaction between the entities funded under 
the Agency’s patient safety programs.  The coordinating center will assure 
explicit translation of the new knowledge from the safety centers of 
excellence and informatics research projects to the demonstration sites.  The 
coordinating center will also facilitate data transfers from the demonstration 
sites to the centers of excellence and process reporting data for transmittal 
to AHRQ.

Research themes under this RFA have been generated through a variety of 
activities including key stakeholder meetings as well as public and private 
sector national summits and research agenda setting activities on medical 
errors, patient safety, and informatics.  For example, the Agency sponsored a 
National Summit on Medical Errors and Patient Safety Research on September 11, 
2000, and findings from the summit also contributed to the research themes.  A 
summary of the summit is available on the Agency’s web page 
(http://www.ahrq.gov/).  Research themes also developed from ongoing 
activities of the Quality Interagency Coordination Task Force (QuIC) and its 
report to the President -- Doing What Counts for Patient Safety - Federal 
Actions to Reduce Medical Errors and Their Impact (http://www.quic.gov/). In 
carrying out its responsibilities, the National Forum for Health Care Quality 
Measurement and Reporting may also influence themes for this research 
initiative.

In responding to this RFA, applicants should address the following central 
research topics:  (1)  the role of informatics in improving clinical decision-
making, reducing errors, and advancing patient safety, (2)  barriers to 
acceptance and adoption of health information technology for improved patient 
safety and quality, (3)  utilization of effective strategies to improve 
patient safety while maintaining patient confidentiality.
 
Definitions

For the purpose of this document:  

Clinical informatics is broadly defined as the use of computers and 
information technology in the provision of patient care.  It includes, but is 
not limited to, electronic records, electronic communication, computerized 
decision support systems, physician order entry, web-based applications, and 
hand-held wireless devices.

Medical error is defined as the failure of a planned action to be completed as 
intended or the use of a wrong action to achieve an aim.  Errors can include 
problems in practice, products, procedures, and systems (Quality Interagency 
Coordination Task Force, 2000)

Patient safety applies to initiatives designed to prevent adverse outcomes 
from medical errors.  The enhancement of patient safety encompasses three 
complementary activities: preventing errors, making errors visible, and 
mitigating the effects of errors (Quality Interagency Coordination Task Force, 
2000).

Adverse events are defined as undesirable and unintended incidents in care 
that may result in adverse outcomes or may require additional care efforts to 
thwart an adverse outcome.

Preventable adverse events are a subset of adverse events that are judged to 
have been avoidable if appropriate and reasonable steps had been taken.  An 
anaphylactic reaction to penicillin is an adverse event.  It is a preventable 
adverse event if the patient’s intolerance of penicillin is noted in his or 
her chart and that chart is available to the clinician, or if the patient 
knows of his or her reaction to penicillin and is able to communicate it to 
the clinician. 

Adverse outcomes are defined as undesirable and unintended outcomes of care 
such as death, disability, or temporary disability.

Near misses are events in which the unwanted consequences were prevented 
because there was a recovery by identification and correction of the failure 
(Van der Schaaf).  Such a recovery could be by a planned or unplanned barrier.  
The term “close calls” is often used synonymously with “near misses.”

HEALTHY PEOPLE 2010

The Public Health Service (PHS) is committed to achieving the health promotion 
and disease prevention objectives of "Healthy People 2010," a PHS-led national 
activity for setting health improvement priorities for the United States.  
AHRQ encourages applicants to submit grant applications with relevance to the 
specific objectives of this initiative.  Potential applicants may obtain a 
copy of "Healthy People 2010" at http://www.health.gov/healthypeople.
 
ELIGIBILITY REQUIREMENTS

Applications may be submitted by domestic, public and private non-profit 
organizations, including universities, clinics, units of State and local 
government, firms and foundations.  For the purpose of this RFA, AHRQ will 
make grants only to non-profit organizations, however, for-profit 
organizations, such as IT vendors may only participate in grant projects as 
members of consortia or as subcontractors.  Foreign institutions are not 
eligible to apply.  Organizations described in section 501(c)4 of the Internal 
Revenue Code that engage in lobbying are not eligible.

AHRQ encourages women, members of racial and ethnic minority groups, and 
persons with disabilities to apply as Principal Investigators. 

AHRQ strongly encourages collaboration with researchers from Historically 
Black Colleges and Universities, Hispanic Serving Institutions, and other 
minority and minority serving institutions.  Information on minority 
institutions is available at http://www.hr.doe.gov/ed/index.html.

MECHANISM OF SUPPORT

Projects supported under this initiative will use the Research Demonstration 
and Dissemination Project (R18) grant mechanism.  Responsibility for the 
planning, direction, and execution of the proposed project will be solely that 
of the applicant.  The focus of the applications should be on the evaluation 
of clinical informatics tools.  It is not the intent of this RFA to support 
extensive development of new IT tools, but rather to demonstrate and evaluate 
the application of IT strategies that patients and providers may use to 
improve patient safety.  The total project period for an application submitted 
in response to this RFA may not exceed 5 years.  The earliest possible award 
date is September 28, 2001.

This RFA is a one-time solicitation.  There are no plans to reissue this RFA 
at any time. 

FUNDS AVAILABLE

AHRQ expects to award up to $3.5 million in total costs (i.e., including 
facilities and administrative costs) in fiscal year 2001 to support 10-14 
projects with an average of $250,000-$300,000 per application for the first 
year R18s selected under this initiative.  In addition, one or two projects 
may be funded as a collaboration with the HRSA/HIV/AIDS bureau.  The focus 
would be to work with non-hospital based (community based) Ryan White medical 
care grantees who target underserved populations such as homeless, communities 
of color, women, youth, uninsured and multiple diagnosed clients.  The actual 
number of applications funded is dependent on the number of high quality 
applications and their individual budget requirements.  It is not the intent 
of AHRQ that the awards be equal in size.  Although the financial plans of 
AHRQ provide for this program, awards pursuant to this RFA are contingent upon 
the availability of funds for this purpose.  Should additional funds become 
available, however, and if AHRQ receives a sufficient number of meritorious 
applications, the Agency reserves the right to fund additional applications 
under this RFA. The anticipated level of continuation funding across the 
awarded R18s for subsequent approved project years is at an equivalent level 
(i.e., about $3.5 million per year for total costs of the initiative).  
However, funding beyond the initial budget period will depend upon annual 
progress reviews by AHRQ and the availability of funds.

RESEARCH OBJECTIVES

Background

For the past thirty years, AHRQ, and its predecessors (the National Center for 
Health Services Research, “NCHSR”, and the Agency for Health Care Policy 
Research, “AHCPR”) have investigated and funded much of the seminal research 
in patient safety.  In the late 1990s, as medical errors and patient safety 
issues moved higher on the national public policy agenda, AHRQ began to take 
on a leadership role in this area.  For example, when the President’s Advisory 
Commission on Consumer Protection and Quality in the Health Care Industry 
presented its Final Report to the President of the United States on Quality 
First: Better Health Care for All Americans (1998), reduction of medical 
errors was listed as one of its top priorities.  A year later, The Institute 
of Medicine released its report on medical errors and patient safety, To Err 
is Human, and called for a strong, comprehensive response to this urgent issue 
(Kohn, et al, 1999).

In early 2000, the Quality Interagency Coordination Task Force (QuIC), with 
AHRQ as lead agency, responded with its report to the President, Doing What 
Counts for Patient Safety, Federal Action to Reduce Medical Errors and Their 
Impact.  This report, which outlined a road map for action including more than 
100 activities, addressed issues such as national focus and leadership, as 
well as identifying and learning from errors, setting performance standards 
and expectations for safety, building public and purchaser awareness, working 
with providers, using decision-support systems and information technologies, 
using standardized procedures, addressing and strengthening standards, and 
integrating data for reporting and analysis (QuIC, 2000). 

When Congress reauthorized (and renamed) the Agency in late 1999, it 
specifically directed the Agency to conduct and support research and build 
public-private partnerships to (1) identify the causes of preventable health 
care errors and patient injury in health care delivery, (2) develop, 
demonstrate, and evaluate strategies for reducing errors and improving patient 
safety, and (3) disseminate such effective strategies throughout the health 
care industry.  AHRQ"s reauthorization also directed the Agency to evaluate 
informatics applications, decision support systems, and computerized patient 
records to reduce medical errors, improve patient safety, and promote quality 
improvement in diverse patient settings.  

Role of Informatics in Patient Safety

As the delivery of health care becomes increasingly complex, the reliance on 
effective systems to prevent medical error will be increasingly critical.  
However, health care continues to take place in systems with little 
standardization and few safeguards, especially when compared to other 
industries (Bates, 2000).  In this complicated environment, it is not 
surprising that medical errors would occur.
 
The vast amount of medical information, coupled with the rapid growth in new 
pharmacotherapies and technologies, increasing time constraints placed on 
health care providers, mounting pressures to reduce costs, and suboptimal 
systems for delivering care, make it virtually impossible for individual 
clinicians to provide high-quality, error free care on a consistent basis.  As 
stated by a leader in health care informatics, “humans are inherently fallible 
information processors (McDonald, 1976).”  This reliance on imperfect memory 
can lead to compromised patient safety and increased rates of medical errors 
(Leape, 1994).  As health care becomes increasingly more complex, information 
systems are needed to assist providers in making the best possible clinical 
decisions.

As attention has turned to the role of systems in preventing medical errors 
and injuries, evidence has become available that computerized decision support 
systems can help to improve patient safety.  The data processing capacity of 
computers has been harnessed by a number of electronic systems designed to 
assist in clinical decision-making.  These systems range from computerized 
reminders about preventive services to alerts about drug-drug interactions to 
specific guidelines and therapeutic recommendations. 

There are numerous examples of clinical informatics applications that have 
been demonstrated to improve patient safety at a few premier medical 
institutions. At LDS Hospital in Salt Lake City, a computerized decision 
support system with physician order entry has reduced the incidence of adverse 
drug events related to antibiotic administration by 75% (Evans, 1995). This 
system also significantly reduced orders for drugs for which patients reported 
allergies and adverse effects caused by antibiotics (Evans, 1998).   At the 
Regenstrief Institute for Health Care in Indianapolis, researchers 
demonstrated that automated computerized reminders increased orders for 
recommended interventions from 22% to 46% (Overhage, 1997).  The physician 
order entry system at Brigham and Women’s Hospital has clearly demonstrated 
its role in reducing medical errors and improving patient safety.  
Investigators demonstrated that use of the physician order entry system with 
decision support led to increased use of appropriate medications for high-risk 
clinical situations, such as an increase in the use of subcutaneous heparin to 
prevent venous thromboembolism from 24 percent to 47 percent.   Errors related 
to medication dosing were reduced by 23 percent, errors in medication choices 
by 44 percent, errors in delays by 77 percent, errors in ordering by 19 
percent, transcription errors by 84 percent, known allergy errors by 56 
percent, and overall serious medication errors by 55 percent (Bates, 1998).  
Perhaps most importantly, these changes persisted at the 1- and 2-year follow-
up periods. (Teich, 2000)

A recent systematic review that assessed the effects of 68 computer-based 
clinical decision support systems demonstrated a beneficial, though variable 
impact on physician performance in 43/65 studies (66%) and a beneficial effect 
on patient outcomes in 6/14 studies (43%)  (Hunt, 1998). However, some 
applications, such as diagnostic expert systems, have met with disappointing 
results (Berner 1994, Kassirer 1994). 

While these examples illustrate that physician order entry systems and 
decision support can help reduce medication errors, improve prescribing 
patterns and guide treatment decisions, only a small number of health care 
systems in the United States have such systems in place.  Though these systems 
require significant up-front capital investment, these costs may be superseded 
by annual savings of $5-10 million per year (Bates, 1998).  

While recent innovations in information technology have been touted over the 
past several years in the lay and industrial press, many of these applications 
have not been adequately evaluated in the health care arena.  For example, 
wireless hand-held computers, also referred to as personal digital assistants 
(PDA) offer great potential in health care.   In preliminary studies, these 
devices have demonstrated an improvement in the detection and prevention of 
adverse events (Shabot, 2000).  PDAs, pagers, and other small portable 
electronic devices, such as digital cellular phones with wireless access, are 
promising new applications with the potential for improving the quality of 
healthcare. 

There are many other cutting edge technologies that could have a significant 
impact on medical errors and patient safety.  Medication bar codes and 
automated dispensers are being utilized in some healthcare settings to reduce 
errors.  The use of scannable patient bracelets containing drug, allergy and 
other medical information is also generating interest.  Such devices could 
help ensure that medications, blood products, and other therapeutics that are 
administered to patients are appropriate.  They could also be used to generate 
alerts about allergies, drug-drug interactions, and other potential problems 
before erroneous administration occurs.  The Veterans Administration has been 
a national leader in the use of some of these technologies.

“Smart cards,”electronic devices the size of credit cards that store and 
process medical information on a microprocessor chip, may also lead to safety 
improvements.  This technology would give patients a portable means of 
carrying their own medical information, which could then be accessed 
electronically by their provider or healthcare facility.  Interactive “smart 
cards” could also generate alerts about potential adverse events resulting 
from medication and other types of errors.  This technology remains in the 
early stages of development and has not been utilized or evaluated to a 
significant degree in the clinical setting. 

While clinical informatics has demonstrated promise in patient safety and 
quality improvement, greater evaluation and research of these technologies are 
needed.  Through support of diffusion of health information technologies to 
other healthcare settings, as well as support for research on the 
effectiveness of innovative information technology tools to improve patient 
safety, this RFA hopes to provide opportunities to harness the power of 
technology for improving healthcare delivery and outcomes. 

Objectives and Scope 

This RFA provides research opportunities to demonstrate the effectiveness of 
clinical informatics tools in improving quality of care.  Research 
demonstration projects supported under this RFA should seek to generate new 
knowledge that can be used by providers, patients, payers, and policymakers to 
promote the use of information technology to reduce medical errors and improve 
patient safety.  Applicants are encouraged to develop partnerships with 
developers and commercial vendors of IT tools and systems.

AHRQ is especially interested in research on the development and use of 
technologies that reduce medical errors, such as hand-held electronic 
medication and specimen management systems, bar coding of medications, 
automated medication dispensers, training simulators for medical education, 
and patient-specific identifiers (e.g., bracelets, smart cards).  

AHRQ is also interested in further research on the opportunities and barriers 
to using information technology for improving the process and outcomes of care 
for patients and providers.  While evidence is accumulating that real-time 
decision support tools, such as physician order entry, may result in improved 
patient safety, further research is needed to develop tools that optimize 
decision support and that assess their impact on other aspects of care such as 
satisfaction, productivity, outcomes, and overall quality.  Researchers are 
encouraged to propose projects that evaluate the impact of IT on clinically 
important outcomes and to emphasize different health care settings and 
priority populations.  

Research in the following specific areas will be considered:

1.  Evaluate the use of appropriate technology to improve patient safety:

a)   The effect of clinical IT tools in reducing medical errors and improving 
the quality of patient care.

b)   Real-time IT decision support systems that alert providers to information 
that may be critical to the provision of high quality care.

c)   The use of evidence-based, real-time decision support systems on 
patient/provider decision-making, patient safety, and outcomes. 

d)   Standardized data elements required in information systems that allow the 
identification and classification of medical errors across health care 
settings.

e)   The impact of a minimum patient data set across information systems and 
health care settings on patient safety.

f)   The use of interactive technology, such as electronic mail, web-based 
medical records, hand-held wireless devices, computer kiosks, and electronic 
patient assessment tools that facilitate informed shared decision-making and 
patient safety. 

g)   Data exploration technologies (e.g., data mining) to assess patterns of 
medical errors within health care systems.

h)   The strengths and limitations of existing systems in providing 
information on patient safety and medical errors in different health care 
settings, including hospital, ambulatory care, home care, and long-term care 
facilities.

2.   Assess barriers to acceptance and adoption of health information 
technology for improved patient safety and quality by providers and patients: 

a)   The impact of IT on efficiency, productivity, time management, workload, 
satisfaction (patient and provider), training, and return on investment.

b)   The influence of human factors on the acceptance and utilization of 
decision support systems.

c)   The sociology/culture of health care professionals that limit greater 
acceptance of new technology.

d)   The costs of informatics solutions, cost-effectiveness, value and return-
on-investment of health information technology for improving the quality and 
safety of health care.

e)   Development of effective strategies to overcome identified barriers. 

3.   Utilize effective strategies to improve patient safety while maintaining 
patient confidentiality: 

a)   Patient and provider preferences regarding privacy and use of medical 
information and explore methods of ensuring data security. 

b)   The trade-offs between maintaining the confidentiality of personal health 
information and using data to improve patient safety and quality of care.

Methods:

Individual R18 projects may use a combination of qualitative and quantitative 
methods.  Qualitative and quantitative methods should be rigorous and use 
state-of-the-art methodologies.  Projects using such methods should be 
grounded in appropriate theoretical frameworks.  Hypothesis-testing projects 
should clearly present competing hypotheses.  Qualitative methods can be used 
on their own to heighten understanding of health care delivery systems or to 
complement quantitative methods and thereby strengthen the research design.
Policy and Practice Relevance and Dissemination

Studies under this RFA are expected to contribute to our basic understanding 
of medical errors and patient safety, build capacity to answer associated 
questions of policy relevance, and produce information that is useful to 
participants in delivering safer health care and to decision makers in 
formulating public policy.  Applicants should be concrete in describing how 
their research results would translate into safer and higher quality health 
care.   Dissemination strategies for effective informatics applications should 
not be limited to publication in peer reviewed journals but may encompass a 
variety of approaches, such as translating results into nontechnical 
monographs (e.g., healthcare guides to key features of physician order entry 
systems) and distributing them through associations of private and public 
officials, educating legislators, public administrators, health plan 
executives, employers, professional leaders, patients and patient advocacy 
groups, and others in seminars, partnerships, and collaboration with health 
care delivery organizations to implement results, and outreach to the media.  

In addition, applicants should budget for and plan to attend an annual 
conference with other centers of excellence and demonstration centers 
supported under the broader AHRQ patient safety program.  These meetings will 
be organized by a central coordinating center. 

SPECIAL REQUIREMENTS

Data Privacy

All information about identifiable individuals or organizations obtained for 
the research purpose of carrying out a demonstration project supported 
pursuant to this RFA under AHRQ’s statutory authority is protected by a 
federal confidentiality statute, section 924 of the Public Health Service Act 
, 42  USC 299c-3(c).  Under this law, the only permissible uses or disclosures 
of the information are those agreed to by the subject individuals and 
organizations or by those who supply the information to the researchers.  
There are substantial statutory penalties (up to $10,000) for improper use or 
disclosure of this information.  Thus, 42 USC 299c-3(c) provides a Federal 
statutory basis for resisting any Federal or state court order or subpoena to 
the extent that carrying out the order or subpoena would violate the 
protective restrictions of the statute.  Upon request, AHRQ will provide legal 
assistance in defending adherence to these statutory confidentiality 
protections with respect to identifiable data obtained in carrying out Agency 
research activities.  

Note that identifiable data obtained by AHRQ-supported researchers is 
protected by this law when it is obtained to carry out AHRQ-supported 
research, but this law would not protect against access to this same 
information in medical records if it were gathered and recorded there for 
other purposes.  If providers collect certain information solely for purposes 
of this research, it should be so marked and if necessary to protect its 
confidentiality, kept separately.  Note also that this law does not protect or 
restrict aggregate and nonidentifiable statistical or data analysis and 
conclusions developed by the researchers that would not disclose information 
about any identifiable individuals or establishments

Pursuant to section 924(c) of the Public Health Service Act (42 USC 299c-
3(c)), information obtained in the course of any AHRQ-study that identifies an 
individual or entity must be treated as confidential in accordance with any 
promises made or implied regarding the possible uses and purposes of the data 
collection.  In the Human Subjects section of the application, applicants must 
describe procedures for ensuring the confidentiality of such identifying 
information.  The description of the procedures should include a discussion of 
who will be permitted access to the information, both raw data and machine 
readable files, and how personal identifiers and other identifying or 
identifiable data will be restricted and safeguarded.

The grantee should ensure that computer systems containing confidential data 
have a level and scope of security that equals or exceeds those established by 
the Office of Management and Budget (OMB) in OMB Circular No. A-130, Appendix 
III - Security of Federal Automated Information Systems.  The National 
Institute of Standards and Technology (NIST) has published several 
implementation guides for this circular.  They are: An Introduction to 
Computer Security: The NIST Handbook, Generally Accepted Principals and 
Practices for Securing Information Technology Systems, and Guide for 
Developing Security Plans for Information Technology Systems.  The circular 
and guides are available on the web at 
http://csrc.nist.gov/publications/nistpubs/800-12/handbook.pdf.

The application of these confidentiality and security standards to 
subcontractors and vendors should be addressed as necessary.

Rights in Data

AHRQ grantees may copyright or seek patents, as appropriate, for final and 
interim products and materials including, but not limited to, methodological 
tools, measures, software with documentation, literature searches, and 
analyses, which are developed in whole or in part with AHRQ funds.  Such 
copyrights and patents are subject to a worldwide irrevocable Federal 
government license to use and permit others to use these products and 
materials for government purposes.  In accordance with its legislative 
dissemination mandate, AHRQ purposes may include, subject to statutory 
confidentiality protections, making research materials, data bases, results, 
and algorithms available for verification or replication by other researchers, 
and subject to AHRQ budget constraints, final products may be made available 
to the health care community and the public by AHRQ or its agents, if such 
distribution would significantly increase access to a product and thereby 
produce public health benefits.  Ordinarily, to accomplish distribution, AHRQ 
publicizes research findings through the media and other appropriate channels 
in coordination with AHRQ’s Office of Health Care Information but relies on 
grantees to publish research results in peer-reviewed journals and to market 
grant-supported products.

Important legal rights and requirements applicable to AHRQ grantees are set 
out or referenced in the AHRQ’s grants regulation at 42 CFR Part 67, Subpart A 
(Available in libraries and from the GPO’s website 
http://www.access.gpo.gov/nara/cfr/index.html).

INCLUSION OF WOMEN, MINORITIES, AND CHILDREN IN RESEARCH STUDY POPULATIONS

It is the policy of AHRQ that women and members of minority groups be included 
in all AHRQ-supported research projects involving human subjects, unless a 
clear and compelling rationale and justification are provided that inclusion 
is inappropriate with respect to the health of the subjects or the purpose of 
the research.  

All investigators proposing research involving human subjects should read the 
UPDATED "NIH Guidelines on the Inclusion of Women and Minorities as Subjects 
in Clinical Research," published in the NIH Guide for Grants and Contracts on 
August 2, 2000 
(http://grants.nih.gov/grants/guide/notice-files/NOT-OD-00-048.html).  
A complete copy of the updated Guidelines is available at 
http://grants.nih.gov/grants/funding/women_min/guidelines_update.htm.  To the 
extent possible, AHRQ requires adherence to these NIH Guidelines.

Investigators may obtain copies from the above sources or from the AHRQ 
Publications Clearinghouse, listed under INQUIRIES, or from the NIH Guide 
Website http://grants.nih.gov/grants/guide/notice-files/NOT-OD-00-048.html.

AHRQ also encourages investigators to consider including children in study 
populations, as appropriate.

AHRQ program staff may also provide additional information concerning these 
policies (see also INQUIRIES).

LETTER OF INTENT

Prospective applicants are asked to submit a letter of intent by April 6, 
2001.  It should include a descriptive title of the proposed research, the 
name, address, and telephone number of the Principal Investigator, the 
identities of other key personnel and participating institutions, and the 
number and title of the RFA in response to which the application may be 
submitted.  Although a letter of intent is not required, is not binding, and 
does not enter into consideration of any subsequent application, the 
information that it contains allows AHRQ staff to estimate the potential 
review workload and to plan the review.  AHRQ will not provide responses to 
letters of intent.

The Letter of Intent is to be sent to:

Eduardo Ortiz, M.D., M.P.H.
Senior Service Fellow
Agency for Healthcare Research and Quality
Center for Primary Care Research
6010 Executive Blvd., Suite 201
Rockville, MD 20852
Telephone 301-594-6236
Fax 301-594-3721
E-mail eortiz@ahrq.gov
APPLICATION PROCEDURES
 
The research grant application form PHS 398 (4/98) is to be used in applying 
for these grants.  State and local government applicants may use PHS 5161-1, 
Application for Federal Assistance (rev. 5/96), and follow those requirements 
for copy submission.  The PHS 398 type size requirements (p. 6) will be 
enforced rigorously, and non-compliant applications returned.  In part, the 
PHS 398 states that the application must be clear, readily legible, and 
conform to all of the following requirements: 1) the height of the letters 
must not be smaller than 10 point, 2) type density must be no more than 15 
characters per inch (cpi), and 3) no more than 6 lines of type must be within 
a vertical inch. Applicants are encouraged to read all form PHS 398 
instructions prior to preparing an application in response to this RFA.

AHRQ is not using the Modular Grant Application and Award process.  Applicants 
for funding from AHRQ should ignore application instructions concerning the 
Modular Grant Application and Award process, and prepare applications 
according to instructions provided in form PHS 398 (rev. 4/98).

The “AHRQ Revised Policy for IRB of Human Subjects Protocols in Grant 
Applications” was published in the NIH Guide on September 27, 2000.  All 
investigators proposing research involving human subjects should read this 
revised policy. 
(http://grants.nih.gov/grants/guide/notice-files/NOT-HS-00-003.html) 

Beginning with applications for AHRQ submitted for the February 1, 2001 
receipt date, Institutional Review Board (IRB) approval of human subjects is 
not required prior to peer review of an application unless otherwise indicated 
by the Agency 
(http://grants.nih.gov/grants/guide/notice-files/NOT-HS-00-003.html).  
Given the nature of this project and the speed with which 
successful applicants will need to begin work, the Agency has determined that 
IRB approval is required prior to peer review of all applications submitted in 
response to this RFA.  All investigators/applicants proposing research 
involving human subjects should pay particular attention to the instructions 
in the form PHS 398 regarding human subject involvement.

AHRQ applicants are encouraged to obtain application materials from the AHRQ 
Publications Clearinghouse (see INQUIRIES).  The Grant Application Form (PHS 
398) is also available online at the following URL: 
http://grants.nih.gov/grants/funding/phs398/phs398.html.

The RFA label and line 2 of the PHS 398 (4/98) form should both indicate the 
RFA number.  The RFA label must be affixed to the bottom of the face page of 
the application.  Failure to use this label could result in delayed processing 
of the application such that it may not reach the review committee in time for 
review.  In addition, the RFA title must also be typed on line 2 of the face 
page and the YES box must be marked.  The sample RFA label available at 
http://grants.nih.gov/grants/funding/phs398/label-bk.pdf has been modified to 
allow for this change.  Please note this is in pdf format.

A complete, signed, typewritten original of the application, including the 
Checklist, and three signed, legible photocopies, must be submitted in one 
package to:
 
Center for Scientific Review
National Institutes of Health
6701 Rockledge Drive, Room 1040 - MSC 7710
Bethesda, MD  20892-7710
(20817 for express or courier service)
 
At the time of submission, two additional copies of the application, labeled 
“Advance Copy” must also be sent to:

Eduardo Ortiz, M.D., M.P.H.
Agency for Healthcare Research and Quality
Center for Primary Care Research
6010 Executive Blvd., Suite 201
Rockville, MD 20852
Telephone 301-594-6236
Fax 301-594-3721
Email eortiz@ahrq.gov

Applications must be received by April 23, 2001.  An application received 
after the deadline may be acceptable if it carries a legible proof-of-mailing 
date, assigned by the carrier, and the proof-of-mailing is not later than 1 
week prior to the deadline date.  If an application is received after that 
date, it will be returned to the applicant without review.  

Conference for Prospective Applicants 

AHRQ plans to convene a special technical assistance workshop to assist 
potential applicants. The purpose of this conference is to give background 
information and respond to questions about the preparation of an application 
in response to this RFA.  The workshop will be held on March 12, 2001, at 
AHRQ, 6010 Executive Blvd., Conference C, Rockville, MD, 20852 from 8:30 a.m. 
to 12:00 p.m.  Attendance is not a prerequisite to applying. Attendees must 
pay for their own travel and accommodation costs.  The workshop will be open 
to any individual or organization intending to apply. Attendees are encouraged 
to submit questions prior to the session.  A synthesis of pertinent Questions 
and Answers discussed at the prospective applicants’ meeting will be available 
from the AHRQ Publications Clearinghouse, listed under “Inquiries,” and the 
Agency’s web page. Further information on the conference will be published 
through the Agency’s website at http://www.ahrq.gov.

Application Preparation (including use of HCFA Data) 

For applications that propose to use Medicare or Medicaid data that are 
individually identifiable, applicants should state explicitly in the "Research 
Design and Methods" section of the Research Plan (form PHS 398) the specific 
files, time periods, and cohorts proposed for the research.  In consultation 
with HCFA, AHRQ will use this information to develop a cost estimate for 
obtaining the data.  This estimate will be included in the estimated total 
cost of the grant at the time funding decisions are made.
  
Applicants should be aware that for individually identifiable Medicare and 
Medicaid data, Principal Investigators and their grantee institutions will be 
required to enter into a Data Use Agreement (DUA) with HCFA to protect the 
confidentiality of data in accordance with OMB Circular A-130, Appendix 
III--Security of Federal Automated Information Systems.  The use of the data 
is restricted to the purposes and time period specified in the DUA.  At the 
end of this time period, the grantee is required to return the data to HCFA or 
certify that the data have been destroyed.  Grantees must also comply with the 
confidentiality requirements of Section 903(c) of the PHS Act.

In carrying out its stewardship of assistance programs, AHRQ, at some point in 
the future, may begin requesting information essential to an assessment of the 
effectiveness of Agency research programs.  Accordingly, grant recipients are 
hereby notified that they may be contacted after the completion of awards for 
periodic updates on publications resulting from AHRQ grant awards and other 
information helpful in evaluating the impact of sponsored research.  AHRQ 
expects grant recipients to keep the Agency informed of publications or the 
impact from Agency sponsored research.

REVIEW CONSIDERATIONS

Upon receipt, applications will be reviewed for completeness and 
responsiveness.  Incomplete applications will be returned to the applicant 
without further consideration. Applications that are complete but are not 
responsive to the RFA will be returned to the applicant without further 
consideration, applicants may submit this project as an investigator initiated 
R01 grant application to AHRQ to compete with other unsolicited applications 
at the next review cycle.   Applications that are complete and responsive to 
the RFA will be evaluated for scientific and technical merit by an appropriate 
peer review group convened by AHRQ in accordance with the review criteria 
stated below.  

All applications will be judged on the basis of the scientific merit of the 
proposed project and the documented ability of the investigators to meet the 
RESEARCH OBJECTIVES of the RFA. Although the technical merit of the proposed 
project is important, it will not be the sole criterion for evaluation of a 
study.  Other considerations, such as importance and timeliness of the 
proposed study and access to patients/data will be part of the evaluation 
criteria.  

As part of the merit review, all applications will receive a written critique. 
Those applications deemed to have the highest scientific merit will be 
discussed and assigned a priority score.  
 
General Review Criteria
 
The reviewers will be asked to discuss the following aspects of the 
application in their written critiques in order to judge the likelihood that 
the proposed research will have a substantial impact on the pursuit of these 
goals.  Each of these criteria will be addressed and considered by the 
reviewers in assigning the overall score and will be weighted as appropriate 
for each application.  Note that the application does not need to be strong in 
all categories to be judged likely to have a major scientific impact and thus 
attain a high priority score.  For example, an investigator may propose to 
carry out important work that by its nature is not innovative but is essential 
to move a field forward.
 
1.  Significance.  Does this study address an important problem?  If the aims 
of the application are achieved, how will scientific knowledge be advanced?  
What will the effect of these studies be on the concepts or methods driving 
this field?

2.  Approach.  Are the conceptual framework, design, methods, and analyses 
adequately developed, well integrated, and appropriate to the aims of the 
project?  Are the proposed data sources appropriate and adequate?  Does the 
applicant acknowledge potential problem areas and consider alternative 
tactics?

3.  Innovation.  Does the project employ innovative IT applications, concepts, 
approaches or methods?  Are the aims original and innovative?  Does the 
project challenge existing paradigms or develop new methodologies or 
technologies?
 
4.  Investigator.  Is the investigator appropriately trained and well suited 
to carry out this work?  Is the work proposed appropriate to the experience 
level of the principal investigator and other researchers?  Is the project (or 
work plan) well organized?  Does the proposed study team reflect the multi-
disciplinary approach required to address patient safety issues?
 
5.  Environment.  Does the scientific environment in which the work will be 
done contribute to the probability of success?  Do the proposed experiments 
take advantage of unique features of the scientific environment or employ 
useful collaborative arrangements?  Is there evidence of institutional 
support?

6.  Policy Relevance.  Will the project provide Federal and State 
policymakers, and others participating in the formulation of such policy, with 
the evidence-based information they need to improve patient safety?  Does the 
application provide a sound plan for achieving this purpose?

7.  Responsiveness to the RFA.  Does the application respond closely to the 
challenges and requirements identified in this RFA?  Do the proposed studies 
appropriately target priorities identified at the QuIC Patient Safety Research 
Summit?

The initial review group will also examine: proposed dissemination activities, 
the appropriateness of proposed project budget and duration, the adequacy of 
plans to include both genders, children, and minorities and their subgroups as 
appropriate for the scientific goals of the research and plans for the 
recruitment and retention of subjects,  the provisions for the protection of 
human and animal subjects,  and the safety of the research environment.

In addition, the following criteria will be applied:

o   The extent to which the study results will be applicable and generalizable 
to situations beyond that of the study and contribute to the reduction or 
elimination of medical errors across a range of settings, levels of care, and 
populations.

o   The extent to which meaningful and sustainable linkages between 
researchers and change agents (e.g., organizations, institutions, 
practitioners, policy makers, patients and patient advocacy groups) are in 
place and used to accelerate the adoption of findings into practice.

AWARD CRITERIA

Applications will compete for available funds with all other applications.  
The following will be considered in making funding decisions:  1) scientific 
merit as determined by peer review, 2) availability of funds, 3) balance among 
target populations, with priority given to understudied populations, and among 
program areas, 4) balance among geographic areas, and 5) balance among all 
program areas.

INQUIRIES

Copies of the RFA and copies of the grant application form PHS 398 (rev. 4/98) 
are available from:

AHRQ Publications Clearinghouse
P.O. Box 8547
Silver Spring, MD 20907-8547
Telephone: 1-800-358-9295
TDD Service: 888-586-6340

This RFA is also available on AHRQ’s Web site, http://www.AHRQ.gov, and 
through AHRQ InstantFAX at (301) 594-2800.  To use InstantFAX, you must call 
from a facsimile (FAX) machine with a telephone handset.  Follow the voice 
prompt to obtain a copy of the table of contents, which has the document order 
number (not the same as the RFA number).  The RFA will be sent at the end of 
the ordering process.  AHRQ InstantFAX operates 24 hours a day, 7 days a week.  
For comments or problems concerning AHRQ InstantFAX, please call (301) 594-6344.

AHRQ welcomes the opportunity to clarify any issues or questions from 
potential applicants who have obtained and read the RFA.  Written and 
telephone inquiries concerning this RFA are encouraged.  Direct inquiries 
regarding programmatic issues, including issues related to the inclusion of 
women, minorities, and children in study populations, should be addressed to:

Eduardo Ortiz, MD, MPH
Senior Service Fellow
Center for Primary Care Research 
Agency for Healthcare Research and Quality
6010 Executive Boulevard, Suite 201
Rockville, MD 20852
Telephone (301) 594-6236FAX:  (301) 594-3721
E-mail Address:  eortiz@ahrq.gov

Direct inquiries regarding fiscal and eligibility matters to:
 
George (Skip) Moyer
Grants Management Specialist
Agency for Healthcare Research and Quality
2101 East Jefferson Street, Suite 601
Rockville, MD  20852
Telephone:  (301) 594-1842
FAX:  (301) 594-3210
Email: smoyer@ahrq.gov
 
AUTHORITY AND REGULATIONS

This program is described in the Catalog of Federal Domestic Assistance, 
Number 93.226.  Awards are made under authorization of Title IX of the Public 
Health Service Act (42 USC 299-299c-7) as amended by P.L. 106-129 (1999).  
Awards are administered under the PHS Grants Policy Statement and Federal 
Regulations 42 CFR 67, Subpart A, and 45 CFR Parts 74 or 92.  This program is 
not subject to the intergovernmental review requirements of Executive Order 
12372 or Health Systems Agency review.

The PHS strongly encourages all grant and contract recipients to provide a 
smoke-free workplace and promote the non-use of all tobacco products.  In 
addition, Public Law 103-227, the Pro-Children Act of 1994, prohibits smoking 
in certain facilities (or in some cases, any portion of a facility) in which 
regular or routine education, library, day care, health care or early 
childhood development services are provided to children.  This is consistent 
with the PHS mission to protect and advance the physical and mental health of 
the American people.

REFERENCES

1.   Bates DW, Leape LL, Cullen DJ, Laird N, Peterson LA, Teich JM, et al. 
Effect of computerized physician order entry and a team intervention on 
prevention of serious medication errors. JAMA. 1998,280:1311-16.

2.   Bates DW.  Using information technology to reduce rates of medication 
errors in hospitals.  Br Med J.  2000, 320: 788-790.

3.   Berner ES, Webster GD, Shugerman AA, Jackson JR, Algina J, Baker AL, et 
al. Performance of four computer-based diagnostic systems. N Engl J Med. 
1994,330:1792-96.

4.   Evans RS, Classen DC, Pestotnik SL, Clemmer TP, Weaver LK, Burke JP. A 
decision support tool for antibiotic therapy. In: Gardner RM, ed. Proceedings 
from the Nineteenth Annual Symposium on Computer Applications in Medical Care. 
Philadelphia, PA: Hanley and Belfus, 1995:651-55

5.   Evans RS, Pestotnik SL, Classen DC, Clemmer TP, Waever LK, Orme JF, et 
al. A computer-assisted management program for antibiotics and other 
antiinfective agents. N Engl J Med. 1998,338:232-38.

6.   Hunt DL, Haynes RB, Hanna SE, Smith K. Effects of computer-based clinical 
decision support systems on physician performance and patient outcomes. JAMA. 
1998,280:1339-45.

7.   Kassirer JP. A report card on computer-assisted diagnosis – the grade: C. 
N Engl J Med. 1994,330:1824-25.

8.   Kohn LT., Corrigan JM., Donaldson MS., eds. To Err is Human, Building a 
Safer Health System. Washington, D.C.: National Academy Press, 1999.

9.   Leape LL. Error in medicine.  JAMA. 1994, 272(23): 1851-1857.

10.   McDonald CL. Protocol-based computer reminders and the non-
perfectability of man. N Engl J Med. 1976,295:1351-55.

11.   Overhage JM, Tierney WM, Zhou XH, McDonald CJ. A randomized trial of 
“corollary orders” to prevent errors of omission. JAMIA. 1997,4:364-75.

12.   Quality Interagency Task Force.  “Report to the President: Doing what 
counts for patient safety-Federal actions to reduce medical errors and their 
impact.” February, 2000.  (http://www.quic.gov/report/)

13.  Shabot MM, LoBue M, Chen J.  Wireless clinical alerts for physiologic, 
laboratory, and medication data.  Proceedings of the 2000 Annual Symposium of 
the American Medical Informatics Association, 789-93.

14.   Teich JM, Merchia PR, Schmiz JL, Kuperman GJ, Spurr CD, Bates DW. 
Effects of computerized physician order entry on prescribing practices. Arch 
Intern Med. 2000,160:2741-47.

15.  Van der Schaaf TW, Lucas DA, Hale AR, eds.  Near miss reporting as a 
safety tool.  Oxford: Butterworth and Heinemann. 1991.

Weekly TOC for this Announcement
NIH Funding Opportunities and Notices

	Office of Extramural Research (OER)			National Institutes of Health (NIH) 9000 Rockville Pike Bethesda, Maryland 20892			Department of Health and Human Services (HHS)
Note: For help accessing PDF, RTF, MS Word, Excel, PowerPoint, RealPlayer, Video or Flash files, see Help Downloading Files.