Technical Assistance Workshop
This document summarizes the technical assistance workshop for the Developmental Centers for Evaluation and Research in Patient Safety (DCERPS) request for applications (RFA). Select to access the RFA.
The technical assistance workshop was held at the Agency for Healthcare Research and Quality (AHRQ) conference center in Rockville, MD, on December 15, 2000. If after reading this document you have any questions or comments, please feel free to contact: Shana Christrup at SChristrup@ahrq.gov (programmatic questions); Al Deal ADeal@ahrq.gov (budgetary questions).
Select to access slide presentation (PowerPoint® Slides [80 KB]; Word® file [42 KB]; Text Version).
Contents
Other RFAs
Basic Information
P20 Mechanism
Funding Years
Total Costs
Patient Safety Research Agenda
Review Criteria
Internal Funding Process
Program Balance
Minority Set-aside
IRB Approval
Data Privacy
Letter of Intent
Application Information
Awardee Requirements
Applicants from Other Federal Sectors
Applicants from For-profit Organizations
Applicants from a Hospital Rather than a System-wide Organization
Program Director (PI)
Junior Members
Educational Materials
Supplemental Funding
Other RFAs
The DCERPS RFA is one of six RFAs to be released by AHRQ during Fiscal Year 2001. We realized that we had been funding patient safety research sporadically, and 2 years ago we decided to change the approach to target specific patient safety issues. In doing so, we have decided on the six following RFAs (listed in the preamble of the DCERPS RFA):
Thus far, the DCERPS RFA and the Centers of Excellence RFA have been released. The others will be released over the next few months. We do expect to release the Demonstrations RFA soon. We will make every attempt to stagger the receipt dates so individuals may respond to more than one RFA if they choose. However, the National Institutes of Health (NIH) determines publication dates. The total amount of money for these six RFAs is approximately $50 million. Each RFA in the series will have its own technical assistance workshop.
Centers of Excellence vs. DCERPS RFA
In determining whether an applicant would be better applying to the Centers of Excellence or DCERPS RFA, there are two major considerations—research team and purpose:
- The Centers of Excellence RFA concentrates its efforts on recruiting an experienced researcher supported by a team that has already produced research in patient safety.
- The DCERPS RFA is targeted toward "new" researchers and research teams—individuals or teams who have not conducted a lot of patient safety research, have not conducted specific portions of the patient safety research agenda and would like to expand in that area, or have not previously worked together as a full team.
For example, an eligible DCERPS team would be a research team that may have produced research findings in the area of translating research into practice but would like to expand its capabilities into studying the epidemiology of errors. Because the Centers of Excellence RFA is targeted toward "fueling" the current patient safety engine to produce products, it is a $5 million grant for up to 5 years with each center required to complete four to seven major research projects and no less than three concurrently running at any given time. However, the DCERPS RFA is targeted toward "building" the engine by producing capacity with a $2 million grant for up to 3 years with only one pilot study required at the end of the planning phase. After completion of the DCERPS RFA, it is expected that all awardees will be eligible to compete for funding similar to the Centers of Excellence RFA and that funding from either AHRQ or other sources will be available.
Applying to Both the Centers of Excellence and DCERPS RFA
A research team may collaborate with a Centers of Excellence and still apply for the DCERPS RFA as long as the proposed projects are mutually exclusive. Applicants must be clear that no duplicative work is being funded (content of work, staff time). However, we discourage applicants from being a Principal Investigator (PI) for both the Centers of Excellence and DCERPS RFA. (Select for more information.)
Number of Individuals Applying
We are not sure of the number of applications for the DCERPS RFA. A similar P20 mechanism in Fiscal Year 2000 (Primary Care Based Practice Networks [PBRNs]) received about 60 applications. We generally tend to fund 20 percent of the RFA applicants and 11 percent of the investigator-initiated applicants.
Return to Contents
Basic Information
The overall goal of the DCERPS RFA is to build capacity related to patient safety research. Specifically, we are requesting that each applicant provide information about their planning activities related to three major areas:
- Building a multidisciplinary team.
- Forming ties to the delivery system.
- Producing education materials about the importance of patient safety and the current evidence about improvement strategies.
Applicants are also asked to focus on topics related to:
- Learning from errors and communicating that information.
- Studying the epidemiology of errors.
- Addressing issues related to systems and cultures.
- Implementing informatics strategies.
Building a multidisciplinary team
The multidisciplinary team must be a functional team that may incorporate members from other industries. We have learned from the aviation industry that appropriate safety teams must include the pilots, crew, engineers, individuals with informatics expertise, etc. You are not required to have individuals from each discipline, but you must have a cohesive team capable of performing your proposed pilot project. In forming that cohesive team, you must have either identified a team or have developed a cogent plan to develop that cohesive team.
If you have already assembled your team, then you should include the most relevant information about the key team members within the application, not just information about the program director (PI). We encourage you to tap into resources outside of medicine when forming your research team, but that is not required for this RFA. We also realize that there may be some overlap in the individuals available to participate in the various patient safety initiatives.
Delivery system and formal data sharing agreements
An appropriate delivery system would be any system that provided direct patient care though systems. Systems producing important, generalizable results would be preferred. (Individuals may decide to concentrate on specific delivery systems, such as an emergency room, but they would need to address generalizability and importance.)
A specific example of a tie to a delivery system includes a formal data-sharing agreement. We mentioned this specific example due to a reluctance for sharing information within delivery systems. The formal data-sharing agreements are not requirements within this RFA, but some specific tie to a delivery system is required. A specific formal data-sharing agreements should be mentioned in the letters of support. When forming agreements with individual physicians who are not part of a network, it is not reasonable for us to request letters of support from each of them. However, your application should address how you intend to achieve that support, and you may find assistance with the Primary Care Based Research Networks (PBRNs).
Return to Contents
P20 Mechanism
The DCERPS RFA is funded through a P20 mechanism.1 There are no specific review or application requirements for the P20 mechanism. Please review the RFA itself as a guide for using the PHS 398 form in responding to this RFA. Where the fit between the application materials and the requested information is awkward, we have tried to provide guidance for you within the RFA itself.
1.
If you would like more information about the P20 mechanism, please E-mail Al Deal to request the information from the internal IMPACT guidelines or preview the Fiscal Year 2000 Primary Care-Based Research Network (PBRN) RFA.
Return to Contents
Funding Years
The funding for the RFA is for up to 3 years. The amount of time that you request should be dependent on your plan for development. (Select for information on Phases.) If you require the maximum amount of time for Phase I (12 months), then it is more likely that you should request the full 3-year grant. However, if you require less time in Phase I, then you might not require the full 3 years of funding. The total funding time is completely up to the applicant.
Return to Contents
Total Costs
The total costs for the grant is approximately $200,000 per year, and this amount includes both direct and indirect costs, given that we are setting aside $2 million for approximately 10 centers. We can only guarantee you funding for the first year at the time of the grant. All future funding years are contingent upon receiving appropriate funds from Congress. When providing the budget for Phase II, it is an approximate budget. We realize that the nature of the application limits the specifics for that budget.
Even if you complete Phase I before the close of the first year, you cannot receive additional funds for Phase II. You are only allocated approximately $200,000 per fiscal year, and those funds may be dedicated to either Phase I or Phase II depending on the proposed plan for development. The earliest possible award date is August 30, 2001. RFAs released after the DCERPS RFA will more than likely have a later award date. Budget issues are generally considered secondary to the proposed project description.
Return to Contents
Patient Safety Research Agenda
The research agenda concentrates on nine different areas:
- Epidemiology of errors.
- Infrastructure to improve patient safety.
- Information systems.
- Knowing which interventions should be adopted.
- Adoption Issues.
- Using the Information.
- Transition Issues.
- Ongoing Issues.
- Mechanisms.
We expect all awardees to have proposed pilot projects tied to that research agenda. Questions in the RFA are examples of the questions in that agenda, but you are not limited to the RFA questions as long as the final focus is within the aforementioned research agenda. As a followup to the National Summit on Medical Errors and Patient Safety Research, we had a meeting in Chicago with clinicians and their managers. You can use the Chicago summary as a guide for your applications, but you are not required to be responsive to the Chicago meeting because it was released after this RFA.
Return to Contents
Review Criteria
Please examine the review criteria rather closely. The study section uses the criteria when examining each of the applications. The review criteria are not given a formal weight by AHRQ staff. Rather, individual members of the study section will apply their own weight to each of the criteria.
In-kind Contribution
One of the review criteria includes the amount of institutional support designated by in-kind contribution. We recognize that most of this support will be targeted toward the PI or other senior staff.
Study section
We realize that we must reorganize the study section if we are to promote the user-driven agenda developed from the National Summit. Therefore, we will be working this year to develop a study section that is not solely comprised of methodologists but also includes individuals from the groups represented at the National Summit. Thus, the study section for the DCERPS RFA will not be comprised of the same individuals from last year's study section. Rather, it will be a special emphasis panel that may have some overlap with the reviewers for the Centers of Excellence RFA.
Return to Contents
Internal Funding Process
Each grant application is reviewed for responsiveness (e.g., has the basic components, appropriate font size, appropriate page limit, etc.), and all eligible applications are assigned to the DCERPS study section. At the study section meeting, the program staff are available to answer questions and to direct the study section members to concentrate on the review criteria. At the close of the study section, each application receives a funding score determined by all of the study section members. The projects are then placed in rank order and internally reviewed for funding. The default mechanism is to fund applications in rank order, but AHRQ staff may request different funding order when examining other important items, such as program balance.
Return to Contents
Program Balance
After study section review, each application will be reviewed internally to ensure that we have achieved a "spread" of capacity-building efforts. In developing an appropriate "spread," we will be considering whether we have funded DCERPS grants:
- Across geography (e.g., regions of the United States).
- Across research focus (e.g., informatics, epidemiology of errors).
- Across populations (e.g., priority populations).
- Across settings of care (e.g., hospital and ambulatory care settings).
When examining spread in geography, we realize that the research team for the DCERPS RFA need not be located under one roof but may be organized as "virtual centers." As virtual centers, they would be considered geographically diverse but would need to demonstrate a plan to develop functional linkages between components of the virtual centers. Simply providing an E-mail list would not be a sufficient functional link.
Return to Contents
Minority Set-aside
There is a minority set-aside for three centers that collaborate directly with a Historically Black College or University (HBCU) or a Hispanic Serving Institution (HSI). In the collaboration, the minority or non-minority institution is eligible to apply. However, both a minority and non-minority institution must be involved in the collaborative effort. If you are eligible to compete for the set-aside and you are not funded, then you compete again in the "general pool" of applicants. Generally, it is in your favor to apply to the minority set-aside. If you are primarily serving a minority population but you are not directly linking with the qualified HBCUs or HSIs, you are not eligible for the set aside funds. However, you should note the minority population in your application because it will be a factor in determining our Program Balance.
Return to Contents
Two Phases
The DCERPS RFA awardees would be expected to complete two different phases. In Phase I, the primary activities are targeted toward various planning activities, while Phase II concentrates on performing the pilot project.
Phase I
Because the DCERPS RFA is a P20 mechanism designed to build capacity, most of the information with the application should concentrate on the actual planning process, rather than focusing on potential pilot projects. Specific activities within Phase I could include:
- Training workshops.
- Developing plans for the pilot project.
- Recruiting other individuals as part of the research team.
Phase II
When determining what to supply for the description of Phase II, please realize that methodologists will not be critically reviewing your proposed pilot projects and that these proposals are nonbinding. Rather, these proposals are a means by which to assess your capacity to perform future research and your ability to highlight the strengths of your research team. You should ensure that your Phase I activities prepare you specifically for what you will be performing in Phase II.
Given these goals, you may not want to use the three-page maximum for each of the proposed pilot projects. If you are considering developing one of your proposed pilot projects into a grant application to another RFA, you should probably propose more than one pilot project. When determining the hypothesis of your proposed pilot project, applicants may choose to concentrate on errors of omission as well as commission. However, errors of omission may require more documentation to the link to patient safety, as it is described within the RFA.
All pilot projects should concentrate on patient safety, rather than broader quality issues. Applicants should simply make a strong case that the central hypothesis within the pilot project is tied to patient safety. For some hypotheses, the link to patient safety is self-evident, but others may require more justification.
Return to Contents
IRB Approval
Because this grant has an application receipt date before February 1, 2001, all applicants are held under the old IRB approval process that requires IRB approval at the time of the application submission or within 60 days after the application receipt date. For applications with a receipt date on or after February 1, 2001, the IRB approval will not be required for peer review. (Select for more information about the IRB approval process.) Because of the nature of this RFA, most applicants will not require IRB approval for Phase I but will require approval for Phase II. Therefore, an appropriate response would be to state that the applicant is IRB-exempt and then include IRB approval at the appropriate time within the planning process. If a proposed pilot project introduces some serious IRB considerations or if new or existing data is used in Phase I or II, then prior IRB approval may be necessary. If individuals are applying and the primary institution does not have an IRB approval process but the secondary institution does, then the secondary institution must provide IRB approval.
The IRB approval process is very dynamic. Please contact the Office of Human Research Protections or AHRQ's Office of Research Review, Education, and Policy (ORREP) for updates concerning the IRB approval process.
Return to Contents
Data Privacy
Because of the nature of this RFA, applicants will not initially need to submit proof of compliance for data privacy standards. However, as with the IRB approval process, applicants who are collecting data should plan to submit proof of compliance before Phase II begins.
Return to Contents
Letter of Intent
The letter of intent for the DCERPS RFA is due Wednesday, January 3, 2001. The letter of intent is strictly used as an internal planning document to help us formulate review panels specific with expertise to respond to the potential grant applicants. It is not a binding agreement. You may send a letter of intent and then not apply. Similarly, you can still apply even if you did not send a letter of intent. We are not able to publicly release any information contained within the letter of intent.
When formulating your one-or-two page letter of intent, we only ask that you include:
- The name of the proposed program director (PI).
- Team member names.
- General focus area.
- Whether you intend to apply for the set-aside funds.
- The RFA name and number.
- (Potentially) a few sentences about your proposed pilot project(s).
Return to Contents
Application Information
The application receipt date is January 24, 2001. To obtain a copy of the grant application form (PHS 398), you can download it or request an application kit from the Division of Extramural Outreach and Information Resources, NIH, 6701 Rockledge Drive, MSC 7910, Bethesda, MD 20892-7910.
Please read the application materials carefully. We will be very strict on the font size limitations. When completing the application, each of the Phase II project proposals are included within the page limit, but letters of support may be included in the appendices.
Return to Contents
Awardee Requirements
A DCERPS awardee would be expected to work with the coordinating center and evaluation center, complete one site visit at the close of Phase I, and attend up to two meetings established by the coordinating center. Three to four months before the close of Phase I, each awardee will be required to submit an updated budget and project proposal for Phase II to be reviewed by AHRQ staff.
Coordinating Center
We have yet to circulate the contract for the coordinating center, though we expect to have the contract let by the time the DCERPS grants are awarded. The specific contract language will not be available before the January 24, 2001, receipt date, and that language would be available on our Web site or as part of the Commerce Business Daily publication (not on the NIH Web site).
The coordinating center is not one of the Centers of Excellence or DCERPS awardees. The role of the coordinating center is to help share information among all of the patient safety RFA grantees, including the systems-related practices grantees from Fiscal Year 2000 (6 total grants). For the DCERPS awardees, the primary purpose of the coordinating center will be to provide some technical assistance in Phase I (e.g., aid grantees in locating individuals with specific expertise) and assist communication across patient safety grantees. With the P20 mechanism, the coordinating center will simply be an advisory body for the DCERPS grantees. We view the coordinating center as an added benefit for the awardees, not as a competition driver.
Evaluation Center
A separate contract will be let to evaluate the success of the patient safety grants.
Site Visit
Each awardee will be expected to complete a site visit at their institution. AHRQ and coordinating staff members will be scheduling those site visits at the close of the Phase I process. We intend for these visits to add value, even though they may raise some logistical concerns.
Return to Contents
Applicants from Other Federal Sectors
We will review applications from other federal sectors. However, the Federal institution must ensure that its own legislation allows it to receive Public Health Service (PHS) grants. Additionally, no career status employees may receive salary support except under specific circumstances, i.e., only people hired specifically for the grant may be support. An interested party may want to apply through a Veterans Administration (VA) or Department of Defense (DoD) research foundation that is designed to administer grants in conjunction with a VA or DoD hospital. In this case, personnel would be employees of the foundation and the limitations on allowable costs are no different from any other nonprofit organization.
Return to Contents
Applicants from For-profit Organizations
Applicants from for-profit organizations are not eligible to apply. However, they may assist a nonprofit group as either a member of a consortium or as a subcontractor. The interaction between the nonprofit and for-profit organization will be carefully examined to determine that there is no "pass through" of funds from the nonprofit to the for-profit. Because of the limit of the amount of interaction for a for-profit organization, it is generally acknowledged that the PI could not be from a for-profit organization.
Return to Contents
Applicants from a Hospital Rather than a System-wide Organization
Applicants from a single hospital should address the special review criteria of generalizability. A single hospital would be less likely to produce generalizable results when compared with a system-wide organization. Individuals from an academic health center with its multiple clinics and locales would probably produce generalizable results.
Generalizability
When trying to determine the amount of emphasis on generalizability, you should look at the capacity of the proposed research to assist the Nation. When addressing the issue within your application, you could provide just one or two sentences unless your project has a relatively narrow focus. Generalizability is generally not a dominant focus of the review, but those relative weights may change.
Return to Contents
Program Director (PI)
Because the DCERPS RFA is designed to build capacity, the research team for the DCERPS RFA should be the same research team for future funding requests. As such, the level of seniority should be appropriate for leading future research projects, such as applications to RFAs similar to the Centers of Excellence RFA, as well as the pilot project in Phase II. The exact level of expertise required is your decision. We are also advising applicants not to apply as the PI for both the DCERPS and Centers of Excellence RFA, even though there is no formal restriction for doing so. We just realize the amount of time required to complete each of the RFAs; thus, we are asking applicants to critically think about which RFA would be best for them.
Technically, the PI could shift between Phase I and Phase II, but that shifting is strongly discouraged for two major reasons. First, reviewers tend not to see a "changing of the guard" in the middle of any grant because of potential loss of forward momentum. Further, there is a fear that is the same leader is not involved with both the planning and the execution of a pilot project, then there will be a disconnect between the "front end" planning phase in preparation for successful execution of the project and the "back end" execution phase of the pilot project. We require that the PI receive some institutional support, and requirement is not a request for matching awards but a statement that this project is not his/her full time job.
Return to Contents
Junior Members
You may request support for junior members of your research team under "other justifiable operating expenses of the planning effort." As always, you would need to justify the amount of time/money dedicated to the junior members.
Return to Contents
Educational Materials
During Phase II of the project, each awardee is required to send copies of their "educational materials" to the coordinating center. The final deliverable is an outcomes report. These materials can include anything that helped you inform individuals about the importance of the issue and the evidence-based mechanisms to improve patient safety (e.g., primer in the importance of research, notes from a series of human factors lectures), which includes information distributed for "buy in" or information collected in determining the evidence-based method for moving forward. These educational materials can be targeted toward the internal research team or external members (e.g., hospital staff, administrators, patients, etc.).
Return to Contents
Supplemental Funding
Any applicant can apply for supplemental funding from sources other than AHRQ. However, the award by AHRQ cannot depend on the potential funding from another source. If funding from AHRQ and another source are required to complete the program, the other supplemental funding must have provided positive confirmation of funding as part of the application to AHRQ. At any point in the funding cycle, DCERPS awardees may apply for supplemental funding. DCERPS awardees could potentially withdraw from the grant award.
If you have any further questions or comments, please contact Shana Christrup (SChristrup@ahrq.gov).
Return to Contents
Current as of December 2000
Internet Citation:
Developmental Centers for Evaluation and Research in
Patient Safety. Technical Assistance Workshop Summary. December 2000. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/fund/DCERPSta.htm.
540 Gaither Road Rockville, MD 20850