Technical Assistance Conference Call: Summary
November 9, 2006
CERTs is a peer-reviewed program that conducts state-of-the-art
research to increase awareness of new uses of drugs, biological products,
and devices; ways to improve the effective use of drugs, biological products,
and devices; risks of new uses; and risks of combinations of drugs and
biological products.
On November 9, 2006, the Agency for Healthcare Research and
Quality (AHRQ) held a conference call to provide potential applicants with
background information and to respond to questions about the preparation
of applications in response to CERTs Funding Opportunity Announcements
(FOAs) . The conference call is summarized below.
Contents
Background
Introductions
Slide Presentation
Responses to Presubmitted Questions
General
Advice
Questions
About Research Centers and Themes
Infrastructure
Partnerships
Health
Information Technology Research Center
Coordinating
Center
Eligibility
and Cross-funding Opportunity Announcement Questions
Currently
Funded CERTs
Budget
Review
Issues
Electronic
Submissions
Open Question Period for Conference Call
Participants
Closing
Background
The CERTs program began with the release of RFA HS-99-004
in 1999 and the subsequent award of grants pursuant to the Food and Drug
Administration (FDA) Modernization Act of 1997. The legislation authorized
the Agency that is now AHRQ to carry out a demonstration program with the
FDA to conduct research and provide objective information on drugs, biologics,
and devices. AHRQ continues to manage this program in coordination with
the FDA under the Agency's reauthorizing legislation of 1999, Public Law
106-129.
The CERTs program was created in response to a legislative
mandate to conduct state-of-the-art research to increase awareness of:
- New uses of drugs, biological products, and devices.
- Ways to improve the effective use of drugs, biological products, and devices.
- Risks of new uses and risks of combinations of drugs and biological products.
The CERTs provide objective clinical information to:
- Health care practitioners and other providers of health care goods or services.
- Pharmacists, pharmacy benefit managers, and purchasers.
- Health maintenance organizations and other managed health care organizations.
- Health care insurers and government agencies.
- Patients and consumers.
The CERTs investigate ways to improve
the quality of health care while reducing its cost through an increase
in the appropriate use of drugs, biological products, and devices and the
prevention of adverse effects of drugs, biological products, and devices.
The CERTs also conduct research on the comparative effectiveness, cost-effectiveness,
and safety of drugs, biological products, and devices.
The purpose of the CERTs FOA's Research Centers, Coordinating Center,
Health information technology addendum, and
budget submission instructions are to:
- Solicit
new CERTs Research Centers, including one on health information technology
(IT).
- Invite
applications from nonprofit and for-profit organizations to act as the Coordinating
Center for all CERTs Research Centers and the CERTs Steering Committee.
Return to Contents
Introductions
Welcome everyone to the technical assistance
call on the Centers for Education and Research on Therapeutics funding
opportunity announcements. We're very pleased to have you joining us today. I
am Anne Trontell. The people around the table at AHRQ are:
- Michael
Handrigan, CERTs Project Officer.
- Michelle Burr, Grants Management Specialist.
-
Carmen Kelly, Pharmacist and AHRQ project officer of the current CERTs Coordinating Center.
- Scott
Smith, Director of AHRQ's Pharmaceutical Outcomes Portfolio.
- Carl
Ohata, Scientific Review Administrator.
- Gurvaneet
Randhawa, CERTs Project Officer.
- Art
Sedrakyan, CERTs Project Officer.
- Skip
Moyer, Special Assistant to AHRQ's Deputy Director.
- Jon
White, Manager, AHRQ Health IT Portfolio.
Return to Contents
Slide Presentation
Those of you who registered for the call
received a copy of the slide set (PowerPoint®
file, 690 KB; Text Version).
Slides 1 and 2: AHRQ's mission is to improve the quality, safety, efficiency, and effectiveness
of health care for all Americans.
Slide 3:
The CERTs program is a research demonstration program funded as a U18 cooperative
agreement by AHRQ in partnership with the FDA. It focuses on research and
education on therapeutic products, by which we mean drugs, medical devices,
and biologics already marketed in the United States. The CERTs also work with other agencies in the Department of Health and
Human Services that are involved with therapeutics. This may include not
only FDA, but also the Centers for Medicare & Medicaid Services (CMS).
Slide 4: By
way of CERTs legislative history, the Food and Drug Modernization Act authorized
the CERTs program, which has since been incorporated into AHRQ's reauthorization. The
CERTs program is a permanent program at the Agency. The CERTs legislation
calls for a cooperative agreement mechanism, which is a grant mechanism
that allows Government involvement but not the level of extensive control
that occurs with contracts. In addition to their support from AHRQ for
infrastructure and pilot projects, the CERTs leverage their individual
and combined expertise through public or private partnerships under the
oversight of the CERTs Steering Committee.
Slide 5: This
slide shows the many rich interrelationships between the CERTs Research
Centers, the Coordinating Center, the Steering Committee (which is comprised
of the principal investigators of the Research Centers and other members),
a wide variety of public agencies, and private industry and foundations. The
Coordinating
Center is a vital communication link among all these various organizations.
Slide 6:
The CERTs vision is to be a trusted national resource for people seeking
to improve their health through the best use of medical therapy. Its mission
is to conduct research and provide education that advances the optimal
use of drugs, medical devices, and biological products.
Slides 7 and 8: The links to the funding opportunity announcements, including the addendum
for the
Health
Information
Technology
Research
Center, are displayed here along with the announcements for the technical
assistance call and budget submission instructions.
The last item is a recent posting that will
be important to consider in your applications.
Slide 9: By
way of orientation to the Health IT Research Center, it is open to all
who are eligible to apply the CERTs Research Centers FOA or the Coordinating
Center FOA, as well as to CERTs Research Centers initially funded in 2006.
The Health IT
Research
Center is described in the addendum. Its objectives are to develop, demonstrate,
and disseminate evidence and educational/translation materials on the role
and use of health IT tools in the safe and effective use of therapeutics.
The FOA lists eight bulleted items that are potential areas of study.
Slide 10: This
slide shows the plans and current configuration of the CERTs. There are
currently 11 funded CERTs Research Centers, a Coordinating
Center, and a Steering Committee. Seven of these centers are in their final
year of funding and are open for competition. With the addition of dedicated
funding for a CERTs research Center on health IT, a total of
$8 million per year is planned, with up to $1 million per year for each
Research Center, for a total of 4 years.
The CERTs network's Coordinating Center, which is up for competition, has anticipated funding of $400,000
annually for 4 years. Four CERTs Research Centers were funded this year
and are anticipated to continue for another 4 years. Their names and associated
themes are:
- Iowa
(with the theme of aging).
- Rutgers
(with the theme of mental health).
- Cornell
(with the theme of devices).
- MD
Anderson/Baylor (on patient adherence and consumer education).
Slide 11: This
slide describes a center. There is certainly flexibility in what a center
will look like. They are similar in that AHRQ funding supports infrastructure
and the conduct of three to five proposed projects. The Centers, as I
stated earlier, support the mission of multiple Health and Human Services
agencies of the Department of Health and Human Services (DHHS). They may
modify projects to support programmatic needs that are informed by AHRQ
and FDA. The Centers are to have identifiable and quantifiable program
goals based on patient outcomes and are to develop outreach to improve
local and national health outcomes through partnerships. Themes are to
be determined by the applicants in consideration of a number of factors
that are described in depth in the FOA and that we'll discuss further later
in this call in the context of the questions we've received.
Slide 12: Let
me conclude the slide presentation by reminding you of key dates.
- The
next key date is the date for the receipt of letters of intent; that is
November 10. These are requested but not mandatory.
- The
date of December 14 for receipt of applications is mandatory. All applications
are by electronic submission. I urge those of you who have not used this
system to initiate registration as quickly as possible.
- We
anticipate the peer review date to be sometime in March 2007.
- The
earliest anticipated start date would be July 2007.
Return to Contents
Responses to Presubmitted Questions
During this segment, we will read the questions
and answers that we've received so far on the FOAs. We have anonymized
those that reveal individual plans that might be more appropriate to discuss
offline. After we have gone through these questions, we'll open the floor
to your questions and we'll take them in the following order:
- Research
Centers including the Health IT Center.
- Coordinating
Center questions.
- Questions
about budgeting and grants management.
- Questions
about grants review.
- Budgeting
and Grants Management.
- Grants
Review.
General advice
We start by offering some general advice.
The first is really a disclaimer to alert you to our intent in this call
to assist you in finding and understanding the information in the FOA. It's
important to bear in mind that the information in the FOA is considered
to be final and definitive. It's really where all the answers lie, and
if what we say today in any way appears to contradict the FOA, the FOA
is the authority.
A useful hint for those of you who view
these documents electronically is to use the Ctrl-F function to search
for key words that might be relevant to your question. Should the FOA
be silent on your questions, look to the electronic 424 Grant Application
Form, which draws upon PHS Form 398. There are many links available to
help assist you if this is your first electronic submission. The registration
process may take as long as 4 weeks, so we urge you to initiate registration
if you have not already done so.
In instances where the FOA or the various
application forms are silent, we will give you our best knowledge or opinion. However,
it's important to recognize that if our opinion is in conflict with the
documents you have, then the documents are right and we are wrong.
We alert you to the language in the FOA. Where
the word "must" is used, that in fact is a factor that will be used in
determining whether or not an application is responsive. When we use the
word "should," that means it's a matter for the review panel to consider. We
direct you to look closely at Section 5, application review information.
In particular, Sections V.2 and V.2.A offer insights on how applications
will be reviewed and scored, as well as considered for funding. Recall
the link to instructions for budget submissions which were recently published
in the NIH Guide.
Questions about Research Centers and Themes
Question: One question asked what themes were in the
scope of the FOA.
Answer: The
theme must be relevant to therapeutics, which are defined to include drugs,
devices, and biological agents.
Question: Several questions concern how to determine
whether or not a chosen theme will be considered adequate. Others have
asked whether some of the themes suggested in the FOA might be used in
part or in whole.
Answer: The
language in the FOA states that the theme should be broadly based in its
focus, such as a particular setting or population, rather than any single
disease or condition. We expect that the Centers will select and document
themes that have a high likelihood of potential impact on practice, and
that applicants will have developed priorities and a strategic plan for
comprehensive education and educational research in therapeutics within
a chosen theme. The chosen theme and associated projects should include
a strong justification as to how the research and education will be translated
into public health impact.
In FOA Section I, following the areas of
potential themes, there's a paragraph that lists eight considerations that
define AHRQ priority topic areas or conditions. In addition, in the Section
V, Application Review Information, there's a description of review criteria
that may be applicable to an applicant's proposed theme. Applicable criteria
are described under the headings of Significance, Responsiveness, and Relevance
to Legislative Mandate Funding decisions will consider multiple factors,
including relevance to program priorities and program balance.
Question: Is there continued interest in having more
work in the thematic areas of the seven CERTs that are now in their final
year of funding?
Answer: The
FOA states that applicants are required to develop proposals that address
gaps in the current CERTs portfolio. The four
CERT
Research
Centers that were funded in 2006, as well as their thematic areas, were already
described in the slide presentation. Applicants may choose themes from those
Research Centers that are in their final year of funding. They may address
other themes that are listed in the FOA or they may develop themes with the
guidance of the criteria that I've just described.
Question: Are multiple sub-themes allowed?
Answer: A
sub-theme is described in the FOA as being of value to the entire CERTs
program. The FOA describes the option of an applicant having a sub-theme. If
additional sub-themes are proposed, they should be well justified and supportable
within the proposed application.
Question: How important is it to capture multiple priority
areas in the proposal as opposed to focusing on just one?
Answer: AHRQ
included these priority areas to help in choosing a theme for the Research
center that will have a beneficial impact on public health. They will
be part of what the review panel will consider in evaluating the proposals
for significance.
Question: Although we thought we had stated it clearly
in the FOA, a number of questions concerned how many projects should
be included in the application for the Research Centers. We've also
been asked how much detail should be included on these projects.
Answer: The
FOA states that funds should be budgeted to provide for staff support to
undertake the design, development, and implementation of between three
and five research and education projects. Each proposed project within
the theme should have a limited number (no more than two) of quantifiable,
desired patient outcomes. The applicant should include quantifiable methods
and data sources for measuring progress toward these goals, with the goals
linked to a plan for implementation.
Question: What should we do for the future, or out
years, of the application? Is it possible to solicit pilot projects
from institutions outside the institution making the application and
to fund them with proposal funding?
Answer: We
remind you that AHRQ funding decisions are based upon the review panel's
evaluation of the proposed projects in the application. Grants management
strongly discourages the use of award funds for projects that do not undergo
a peer review process.
Question: Given that this is a 4-year grant cycle,
the first year may involve gathering and synthesizing new information
that will shape the research and education projects for Years 2-4. How
much detail do you want about proposed studies and educational interventions
planned for Years 2-4? How much detail do you want for the educational
interventions?
Answer: Proposed
studies should provide sufficient information that the review panel can
evaluate them for the review criteria listed in the FOA and previously
described.
Infrastructure
Question: Please describe in more depth the key elements you wish applicants
to address in their discussion of the infrastructure for the grant.
Answer: The
FOA indicates that the Research Center funding provides infrastructure
support, which means a research team with adequate facilities, institutional
support, and access to appropriate expertise needed to perform the research,
education, dissemination, and translation of research into practice. AHRQ
funds are intended to provide infrastructure support for each
Research Center to allow it to function effectively. Funds should be budgeted
to provide for staff support to undertake the design, development, and implementation
of between three and five research and education projects.
The additional review criteria indicate
that evidence should be presented of an infrastructure capable of maintaining
the
CERTs
Research Center.
Partnerships
Question: Is there an opportunity for collaborative
funding with partners, i.e., how much should be focused on using CERTs
funds vs. other sources of funding)?
Answer: Partnerships
are a very important part of the current CERTs program. A variety of
Federal and private partners exist. The CERTs Web site describes a number
of these and also describes the principles by which these partnerships
operate, as well as other relevant information.
Partnerships are also addressed in the FOA
in several areas. In particular, the applicant should identify and describe
relationships with several intermediary and end-user groups. Intermediaries
may include appropriate clinical specialty organizations. End-user groups
might be consumers or patients. The applicant may consult with these intermediary
and end-user groups on research products and implementation strategies
as they are designed. The partners must be willing to help foster dissemination
and translation of findings and products into practice. Letters of support
from these organizations provide evidence of these partnerships.
Bullet 5 of Additional Guidance, under Part
II, Section 1, Funding Opportunity Description, Research Objectives, describes
considerations for partnerships.
We note that, for those projects where a
nongovernmental partner has been identified, the applicant should include
a description of the steps that have been taken to assure adherence to
the principles set forth in the CERTs principles for public or private
partnerships. These principles can be viewed on the CERTs Web site.
Also, the review and selection criteria
under the section on budget discuss the review considerations that are
relevant to partnerships.
Question: Under the additional guidance bullet, AHRQ
was asked for clarification about the statement: "Each Research Center
should incorporate measures and goals of cost-effectiveness and efficiency
into research and education activities."
Answer: This
additional guidance indicates that cost-effectiveness and efficiency considerations
can inform the Research Center's choice of proposed research and education
activities, and that the review panel may consider these as well. Recall
our previous general advice on what is described as "must" and what is
described as
"should."
Question: What should be the length of the research
plan?
Answer: The
length is the default of the SF424 Application Guidelines, which is 25
pages.
Health Information Technology Research Center
Question: Will the Health IT CERT be a standalone CERT
with some partnerships? Will its main function be to support the other
CERTs?
Answer: The
Health IT CERT will be a standalone Research Center like the other CERTs
and will be encouraged to collaborate and form partnerships. Support of
the CERTs Research Centers is a function of the Coordinating Center and is described in the FOA for the Coordinating Center.
Question: Does the IT CERT consider information technology
as a therapeutic device or as an adjunct to other therapies (e.g., translating
research into practice or safety assessment)?
Answer: In
the majority of cases, health IT would be considered to be an adjunct to
other therapies. However, we can imagine there might be cases where Health
IT itself is the therapeutic device.
Question: Can U18 grant resources be used for software
writing?
Answer: As
long as it's not prohibited in the FOA, software writing costs can be included.
Like all allowable costs, however, they must also be reasonable, allocable,
and necessary. A good budget justification is helpful. Reviewers and
funding decision-makers will look at requested costs in the context of
the overall proposal.
Question: What was the impetus for adding the Health
IT CERT?
Answer: AHRQ
has been doing a lot of therapeutic work in the health IT grant portfolio,
and health IT is an increasingly important mechanism for delivering and
gathering information. There's also a desire to understand the Health
IT systems themselves and how they can improve therapeutic delivery and
coordination of care. Funding for the Health IT CERT comes from AHRQ's
health IT portfolio, which is an extension of AHRQ's patient safety initiative.
Question:
Do current CERTs have any advantages compared to non-CERTs in competing
for becoming the Health IT CERT?
Answer: No.
All applications will be reviewed by the same panel without preference
or stratification by experience. The health IT CERTs applications will
be compared against each other for the final funding decision. The CERTs
FOAs indicate that program balance (avoidance of duplication and overlap
in the funding of Research centers) and geographic distribution,
along with other factors, will be weighed in making funding decisions.
Coordinating Center
Question: Section I: In the description of the
Coordinating Center (CC) and the Principal Investigator (PI), AHRQ uses
the terminology "a global program perspective." Should that be taken
literally to imply that the CERTs effort in the future will turn more
international (i.e., global), or simply that the PI is expected to have
a comprehensive view of the CERTs program?
Answer: The
words "a global program perspective" mean a comprehensive view of the CERTs
program, looking at CERTs goals and objectives being fulfilled. The FOA
explicitly states that the focus of these programs is to increase the implementation
of safe practice interventions in the
United States. For that reason, foreign institutions are not eligible to apply. No awards
will be made to them. However, foreign organizations may participate in
projects as members of consortia or as subcontractors.
Question: Section III, Other-Special Eligibility
Criteria/Special Requirements:In the bullet about educating users, does the ". need
to develop a plan to . implement widespread dissemination to audiences
such as State policymakers and clinicians..." imply that this is a new
or expanded education and dissemination requirement? Would this entail
working directly with the User Liaison program of the Office of
Communications and Knowledge Transfer?
Answer: This
bullet indicates a responsibility of the CC and emphasizes the need to
further existing education and dissemination efforts of the CERTs program. This
includes supporting relevant subcommittees of the Steering Committee, which
includes a cross-Center consortium of educational leads from each CERTs
Research Center. This is referred to in the FOA as the ESC, or Educational Subcommittee. It
will also involve working with AHRQ programs and AHRQ's Office of Communications
and Knowledge Transfer to continue to develop plans for dissemination.
Question: Can AHRQ explain what "economies of scale"
means in the context of the eighth bullet under Section III, Other-Special
Eligibility Criteria/Special Requirements?
Answer: "Economies
of scale" refer to the effective use of resources to look across all of
the Center activities to find ways resources can be used most efficiently. One
example would be the use of combined meetings, such as "Think Tanks" or
Steering Committee meetings, where two meetings are convened at the same
location and adjacent time slots to minimize the need for traveling to
separate meetings and to maximize participation.
Question: In the last bullet under Section III,
Other-Special Eligibility Criteria/Special Requirements, on publications
and "complete bibliography," can AHRQ clarify the meaning of this requirement?
Whose publications should be included and please define "complete bibliography." How
does this requirement relate to the publications that are included
in the appendix?
Answer: The
publications of the PI plus key personnel within the CC should be included
with a complete listing in the form of a bibliography. The bibliography
refers to a complete listing of citations for articles that have been published. The
FOA indicates the appendix may include up to 10 publications, manuscripts
accepted for publication, abstracts, patents, or other printed materials
directly relevant to the proposed project.
Question: Section IV, Part 3.C, Use of CMS Data: The
CC would not itself (presumably) be doing projects that would require
acquisition of CMS public-use data. Therefore, can AHRQ clarify what
should be discussed in the CC application about this issue? The same
general issues may arise about use of HCUP (Healthcare Cost and Utilization
Project) and MEPS (Medical Expenditure Panel Survey) data. These don't
seem relevant for the CC responsibilities.
Answer: If
an application for the CC does not contain plans for use of CMS data or
other data resources, then this section does not apply and it does
not need to be addressed.
Question: Section V, Part 2.A, Additional
Review Criteria: The RFA refers to AHRQ policies on protections
for human subjects, inclusion of both males and females and members
of racial or ethnic minorities, or persons with AHRQ priority conditions. In Part
2.D,Sharing Research Data (and in Section IV, the RFA calls for
a plan for sharing research data.
Are these relevant considerations for
Coordinating Center applications, given that the CC does not appear to have research,
evaluation, or demonstration projects or subjects of its own, and presumably
will not be doing projects that generate research data? If it is, can the
Government explain more fully what information is being requested in the
CC applications with respect to these requirements?
Answer: Applicants
for the CC should consider closely whether any aspects of your application
might pertain to the protection of human subjects, even if such protection
is limited to issues of confidentiality or privacy. Please state in your
proposal if you believe DHHS policies for human subjects protection do
not apply in your case. Similarly, if your application does not propose
any sharing of research data, please state this.
These sections were included to assist Coordinating Center applicants that may choose to submit proposals where these requirements would in fact apply.
All Research Center applications should have a section on human subjects protections,
which is a DHHS regulation. They should state whether human subjects are
involved and the protections involved.
If no human subjects are involved, then they should so state. If data
are public, applicants should state that there is an exemption (i.e., E4). Even
Coordinating Centers, which would manage human subjects indirectly, should
address human subjects protections.
Similarly, all applications
should address the inclusion of both genders, minors, and AHRQ priority
populations. The committee will decide whether the applicant
has responded acceptably or unacceptably. Relevant links in the FOA are:
Question: Section VI, Part 2: The support mechanism
for the Coordinating Center is a U18 (cooperative agreement) and CERTs
are funded via a grants mechanism, according to the RFA. Can AHRQ clarify
why Office of Management and Budget (OMB) clearance is needed for surveys
that might entail asking identical questions of more than nine individuals?
This is usually a requirement only of contracts.
Answer: In
the context of OMB clearance, cooperative agreements are currently treated
the same as contracts.
Eligibility and Cross-Funding Opportunity Announcement Questions
Question:
There are several statements in the CERTs FOA about the "institution"
rather than the "individual" being the applicant.." What
does this mean with regard to how I/my institution applies? What sort of
(and level of) institutional letter/commitment does the application require─e.g.,
Dean's Letter or Chancellor's Letter?
Answer: All applicants should work with their institution's
sponsored programs or research administration office however it may be
named. This will be the office that deals with obtaining outside funding
for the institution. Each institution should have an authorized institutional
official who must endorse the application before it is submitted. The
applicant institution is legally responsible and accountable to the Federal
Government for the use of funds provided and for the performance of the
grant-supported project or activities. The Principal Investigator, however,
is responsible and accountable to the recipient organization for the proper
conduct of the program.
As you probably noticed, the FOA mentions no requirement
for a specific type of institutional support letter, but does mention institutional
support in the reviewer's evaluation criteria, particularly in those sections
with the headings Environment and Budget.
Question: Questions about eligibility to apply have
been raised with regard to the following instructions in the FOA: "Applications are encouraged from existing
integrated programs, other than those already funded as CERTs (e.g.,
Centers funded through the National Center for Research Resources, General
Clinical Research Center (GCRC) Program, the Centers for Disease Control
and Prevention's Prevention Centers, AHRQ-funded Evidence-based Practice
Centers [EPCs]). Specifically, there was uncertainty
whether the Centers listed in the parentheses could apply.
Answer: The
existing integrated research programs listed in the parentheses are encouraged
to apply, but not the four CERTs Research Centers that were awarded
in April 2006 and which have continuing funding past 2007. Those
four Research Centers may apply for Health IT Research Center funding.
Question: Is there any prohibition against two CERTs
applications being submitted from a single institution?
Answer: There
is no prohibition against two CERTs applications being submitted from a
single institution. Note that the FOA stipulates that an investigator may be a PI for only one CERT. The
CERTs FOAs indicate that funding decisions will consider program balance
(avoidance of duplication and overlap in funding Research Centers, geographic
distribution) along with other factors.
Question: Can a CERT Center (either a Research Center
or Coordinating Center) have two or more PIs?
Answer: AHRQ
does not recognize more than one PI per Center. There can be co-investigators,
but not co-PIs. The applicant must designate only one person
to be the responsible PI.
Question: Would AHRQ consider it positive, negative,
or neutral to have a single institution awarded two or more CERTs, such
as a regular CERT and an IT CERT, a regular CERT and a Coordinating
Center, or an IT CERT and a Coordinating Center?
Answer: AHRQ
will consider application reviews, program balance, geographic distribution, and other
factors listed in the FOA in making funding decisions.
Question: How might it be possible for two or more
institutions independently applying for CERTs funding to propose a joint
project that leverages the resources of each institution?
Answer: If
two or more institutions that are submitting separate applications wish
to propose collaboration on a joint project, each proposal should have
a description, plan, and budget for that project that would be sufficient
unto itself, so that the application could be reviewed on its own merits
and not in comparison to any other application.
Question: How much support (financial and intellectual)
from the Coordinating Center should be anticipated by applicants for
Research Centers?
Answer: The
FOA indicates that the CERTs Coordinating Center will handle monthly calls;
Steering Committee agendas and logistics; project tracking; the production
of CERTs program materials, including the annual report; cross-CERTs communication;
and the dissemination of research findings. The CC administers the
public-private partnership committee described previously. The extent
of the "intellectual support" of the Research Centers by the Coordinating Center will be determined in part on the award of that cooperative agreement. Each Research Center has its own AHRQ project officer to assist in overall direction.
The Additional Guidance section of the FOA describes in bullet 15 what each
Research Center must include in its budget funds for the Steering Committee,
CC, and AHRQ activities. That section begins "Each research center must
include in its budget funds that allow for active participation in Steering
Committee, CC, and AHRQ activities.."
Question: Are there already existing guidelines for
specific details of how our center would interact with the Coordinating
Center, or should we propose the details of this relationship in our
application?
Answer: Please
refer to the Coordinating Center FOA, which lists 13 bulleted items listing
the rights and responsibilities of the Coordinating Center under Section
VI. Award Administration Information, Subsection 2.A.1.
Currently Funded CERTs
Initially
funded in 2006:
Question: Can a PI at one of the four CERT Research
Centers (RCs) funded beyond fiscal year (FY) 2006 be an investigator
or subcontractor of a new RC?
Answer: In
general, the FOA states this is allowed, but obviously specific circumstances
of funding and project commitments need to be considered.
With
final year of funding beginning FY 2006:
Question: Please clarify the following language: CERTs
projects whose final year of funding begins in FY 2006 may submit new
applications but may not submit competing continuations.
Answer: CERTs
Centers whose final year of funding begins in FY 2006 can apply to this
FOA with new applications. If an institution wishes to include in its
application a modified or expanded CERTs project where the final year of
funding begins in FY 2006, any such project should be fully described in
the application.
Budget
Question:
How can we submit separate budgets within the 424 application?
Answer: A
recent announcement in the NIH Guide describes how this should be handled: http://grants.nih.gov/grants/guide/notice-files/NOT-HS-07-030.html
Question: Is the overhead for a subcontract with another group considered
part of the applicant's direct cost in the budget?
Answer: The
indirect costs of a third party count as direct costs to the applicant
organization.
Review issues
Question: How much familiarity will the special panel
have with the CERTs program to assist in determining how much of the
research plan should be taken up with background on the program?
Answer: The
special emphasis panel will have information on the CERTs program, but
not on the specifics of historically funded CERTs Research Centers or projects. To
avoid confusion or ambiguity, applicants should include all information
that they consider necessary for a full evaluation of the scientific merits
of their application.
Question:
Will institutions that were previously CERTs be given any different consideration
in review? Did they have advance notice of this solicitation?
Answer: All applications will be reviewed in the same fashion,
assigned a priority score, and then funding considerations described in
the FOA will be applied. The existing CERTs had no advance knowledge of
the FOA being released.
Electronic Submissions
Question: What organizational structure
and format should I use for the research plan?
Answer: Please refer to the online guides for full information. The
research plan will need to be submitted with the relevant sections described
in the FOA, which may be selected from among the following: Introduction
to Application, Specific Aims, Background and Significance, Preliminary
Studies/Progress Report, Research Design and Methods, Inclusion Enrollment
Report, Progress Report Publication List, Human Subjects Sections (including
four subsections), Vertebrate Animals, Select Agent Research, Consortium/Contractual
Arrangements, Resource Sharing Plan, and Letters of Support.
Return to Contents
Open Question Period for Conference Call Participants
In addition to the responses shown above
to questions received in advance of the call, additional questions or points
of clarification about the FOAs were asked on the call.
Question: I'm still a little confused about how the
number of projects spans over 4 years and your comment that basically
the research plan should describe all research to be done over the entire
4 years. My impression is that many of the CERTs do more than three
to four or three to five projects over a 4-year period when you look
at what's on the Web site. So how do we handle that?
Answer: The
specific mention of three to five projects in this FOA is actually a new
specification in this FOA. AHRQ has not historically placed a limit on
the number of projects. Some of the projects that are listed for the CERT
may in fact be leveraged projects that have some component of CERTs funding,
as well as outside partner organizations. Some projects outside the three
to five that are proposed in this FOA may have been to be developed in
collaboration with the AHRQ project officer via the U18 cooperative agreement
mechanism. Does that answer your question?
Question: What it implies then is that if we expect
to do additional projects, then we need to allocate some part of our
budget to having people and resources available to participate in additional
projects. Given the budget side, it seems three to five projects per
side is a small number.
Answer: The
issue of the specific projects, their size, and how you fit them in with
the infrastructure and other budgeting is actually going to be a matter
for the individual institutions applying. We cannot give you any more
directive advice than that.
Question: I have a variation of the last question;
it's still not clear to me. It seems to me that if the projects are
pilot projects, in all likelihood, they would not last the duration of
the 4 years of funding.
I had assumed, both by reading carefully the FOA and
also by looking on the Web site of the existing CERTs, that these projects
might last a year or two and that we would then continue to do other
similar-sized projects. That raises in my mind two problems that I'd
like an explanation for:
1. Within a 25-page application, it's hard to imagine
giving enough information about both the infrastructure and, say, 10
or 12 projects that might be done over 4 years.
2. The earlier comment that any project would have to
be reviewed implies to me that perhaps if we develop new projects, say
in Year 2, having set aside a certain component of the budget to cover
new projects in subsequent years or in out years, they would then have
to be submitted for review to AHRQ.
Could you clarify the issue of the kind, number, and
length of projects? Or is it expected that these projects that we've
described would last the whole time? And also the review of any new
projects that might be developed? If I could clarify that last issue,
you said in your comments earlier and the FOA says that the infrastructure
should be in place to sort of identify and develop new projects,; so
that does imply that we would be developing projects over the course
of the 4 years.
Answer: In
terms of describing the project adequately in 25 pages, we recognize the
challenge with that page limitation. We can only advise you to do your
best in terms of allowing enough information for those projects to be reviewed.
In terms of future projects, one mechanism
that we believe may be acceptable to the review panel would be to describe
the process or criteria for an adequate and full review of the scientific
merits of those projects. Saying in your proposal, "We have allocated
X dollars for future projects in this area,"
without stating how that funding decision would be handled, would not be
well regarded, in contrast to saying, "We have a peer review process" or
some process of consultation with AHRQ in making decisions about additional
projects that might be included over the life of the funding.
Question: My question is about the themes and what
are acceptable as themes. Much of the research work in health information
technology is cross-cutting, applicable to many different populations
and different types of projects. So how does that relate to the selection
of the theme when examples of the theme have tended to focus on particular
populations?
Answer: Are
you asking whether or not the Health IT Research Center should propose
a theme other than health IT?
No, I'm asking about a proposed theme
for a Health IT Research Center. Can you propose a health IT theme that
could be cross-cutting across many different populations, or do you have
to propose a Health IT theme as you will apply it to a particular population,
such as the example themes given?
Answer: You
can propose a Health IT theme that's cross-cutting. You have to be careful
not to duplicate the four existing themes that have currently been funded.
Question: The current document indicates that the start
date could be July 15. What should the current Centers do whose expiration
date is much later, such as September 30? Is there any special direction
that they should use in looking at a potential start date? Is there
a preference that AHRQ would encourage those centers to consider?
Answer: Any
applicant, including the existing CERTs, can propose a desired start date
in their application. The expected start date would have to be no later
than September 30, 2007. The decision on the start date will be negotiated by
AHRQ with the grantees in the light of funding decision dates, administrative
issues, and other matters. So the start date is something you can propose
that will be handled in final negotiations, if applicable.
Question: Can we return to project number? I thought
that I understood what we should propose in this discussion but now I
am quite confused. The RFA mentioned three to five projects. It did
not mention three to five projects per year, so I interpreted that to
mean having to propose three to five projects for the entire 4-year period. Subsequent
discussion on this call seems to suggest the possibility of proposing
additional projects, or more than five. Could you clarify, please?
Answer: In
the section on additional guidance, the language says that funds should
be budgeted to provide for staff support to undertake the design, development,
and implementation of between three and five research and educational projects. This
will be a matter of review by the special emphasis panel.
One other way of looking at this is that
we are looking at two to five projects overall. So you have flexibility
in how broadly you plan and define those projects. It could be one project
for the first year, then moving on to the second project in the second
year, or you have the flexibility of proposing two to five projects that
would each require the entire 4 years of research. Of course, there will
be the judgment of the peer reviewers, but this gives flexibility in how
you can propose a project. We don't want to constrain it to making it
only 1 year for each project or 4 years for one project. That actually
is up to you.
Question: We interpreted the RFA to suggest that the
project could be relatively broad in scope with different components
and different parts as long as we could make a clear justification for
the interrelatedness of the component parts to the special emphasis panel. Would
that be a fair statement on our part?
Answer: Yes. We
think that's actually a good interpretation in terms of having a broadly
based project that might in fact result in a variety of research products,
such as several publications or other matters. This perhaps gives you
a little better sense of the breadth to which we may be applying the term "project," so
that you might think of them more as major project areas.
Question: To follow on with one additional question,
we had also been presuming that it would be effective to weave an educational
component into each of the projects, as opposed to setting one of the
projects aside to be mainly educational. Is that also a fair interpretation?
Answer: That
is going to be a matter for the review panel to decide.
Question: When you write these three to five proposals,
are you held to doing all three of those five proposals or can they can
be modified or substituted once you get into the system?
Answer: We
don't encourage individuals to propose projects that they wouldn't plan
to carry through. When the programs are up for continuation, modification,
or substitution on an annual basis, this can be a matter of discussion
and renegotiation with the AHRQ project officer. There is flexibility
over the life of the project for modification based upon programmatic input.
Question: Can the letter of intent be sent by e-mail?
Answer: Yes. It goes to Amy Lindinha.
Question: This question has to do with the outcome
specified for each of the research or education projects--the measurable
outcome. We were wondering whether that has to do with outcomes in the
study population or measurable outcomes in improving therapeutics in
the United States.
Answer: It
is difficult to answer that question in the specific. Overarching goals
of the CERTs program are that the research and educational projects undertake
and have some measurable benefit, ideally in influencing health outcomes. We
will leave it to the individual applicant and the review panel to comment
on the specific merits of what's proposed.
Question: Are we supposed to have separate human subject
details for each of the pilot projects in terms of numbers of children,
elderly, minorities, etc.?
Answer: For
purposes of your application, it's probably most important to make sure
that the reviewers are clear that human subject protection, if it is relevant,
has been addressed. It really is the applicant's prerogative in terms
of how you handle the material. You could have one major human subject
section for the entire application or, at the end of each project; you
can have human subjects information It's really your prerogative.
Question: Would it be okay to have a blanket statement
that covers all the projects saying something like, "Human subject protection
will be addressed by the appropriate institutional review board,"
without going through all the details that we would for an RO1 or some other
type of grant in terms of listing targeted and planned enrollment, talking
about inclusion of women and minorities, and similar levels of detail.
Answer: You
have to provide some sort of detail because the reviewers will look at
your application to see whether you really do have a plan to ensure the
protection of subjects in your specific projects. They want to see that
protection for human subjects is ensured. A broad statement that says, "We'll
take care of this," will not necessarily provide sufficient reassurance
to the reviewers. You'll have to balance the level of detail you provide
with what you think would assure the review panel that human subject concerns
are satisfactorily addressed.
Question: Regarding the Coordinating Center, under
special eligibility criteria, special requirements, the second bullet,
it notes that the CC convenes the Steering Committee which meets three
to four times per year. Travel will be coordinated through the CC. Does
that mean that the CC would have a role in assisting the sites and other
participants or is it simply coordinating travel for its own staff?
Additionally, it notes that the meeting will be held
at the Eisenberg Center, at the AHRQ Conference Center. Is there a charge for use of that facility that the Coordinating Center needs to budget?
Answer: There
is no charge for the Eisenberg Center's facilities.
Question: Then going down to bullet number nine, it
said the CC application must have a business plan. Could you define
that a little more in terms of whether this should be something that
entails the sub-budgets or describes specific functions?
Under bullet 10, it says the CC will develop programmatic
products for the CERT. I wonder if you could get us some guidance about
the number and complexity of products?
Also, who should be responsible for budgeting any sort of reproduction and
distribution cost? Related to that, under qualification, bullet number five,
it said the CC must be able to provide writing and editorial assistance on
an ongoing basis for meeting minutes, which is clear, but then it also refers
to "written manuscripts and research summaries" and says that the CC may
be asked to develop original manuscripts. This area could be very limited
or very open ended in terms of the level of support you would be expecting
the
Coordinating Center to provide to the individual sites, looking at the number
of manuscripts they could be expected to produce. Could you say a little
more about the level of detail we should use in budgeting that?
Answer: We
can answer some of your questions. We will defer the one about the business
plan to subsequent e-mail communications. On the issue of travel coordination,
the work of the Coordinating Center is largely in organizing. Other than Coordinating Center staff who might need to travel, the actual expense of travel is borne
by the Research Center.
In general, the Coordinating Center would not bear the printing cost of any materials that are produced. The Agency bears those costs.
(Note: Follow-up of the unanswered questions
from this caller was still pending at the time this Summary was prepared.
We are posting the Summary without this information to avoid undue delays
for those who were unable to participate in the call.)
Question: To be clear on the plan for sharing research
data in this FOA, is a data-sharing plan required?
Answer: The
FOA refers to directions should you plan to be sharing data. That is certainly
the prerogative of the applicant to propose. If it is proposed, the necessary
information should be provided.
Question: There was discussion earlier about providing
a bibliography for the PI. Is that an additional requirement to the
biosketch, which would have those publications listed?
Answer: The
question that we answered previously pertained to the PI of the Coordinating
Center. Some may have questioned whether that PI would have to cite any
publications or bibliography.
Question: Our team conceives of this project as four
specific projects plus one that's in underlying infrastructure support. Do
I understand that we would have 25 pages describe each of the four projects
as well as describe our infrastructure needs? Or do we have to combine
all of that into one and relate that to the infrastructure? Should we
conceptualize the infrastructure as something separate or something that
has to be built into our projects?
Answer: For
your four specific projects and the underlying infrastructure project,
whether you should develop them separately or build the infrastructure
into the projects is really a matter of your choice in your application.
The page limitation for your entire research plan is 25 pages.
Question: How would we treat intellectual property
regarding the software that we develop as part of this project?
Answer: For
software that is developed under a grant mechanism, the copyright or the
intellectual property right remains with the grantee. This is the way
that it's been treated in the past in the Federal Government. However,
as with many other products that are created under a grant mechanism, the
grantee agency─--the Federal Government─--retains the right
to distribute and use that product free of charge as we choose. So the
property, --the intellectual property rights, --would remain with the grantee
for software domain, but the Government has a license to use and distribute
the product free of charge.
Question: Regarding the budget justification, I noted
there's only one attachment allowed. How did you want to structure the
budget justification? Did you want to structure it for all your projects
separately or combined for the entire proposal?
Answer: Applicants
are allowed to structure their budget justification as they think fit for
their proposal. The budget justification will have to clearly identify
which project and organizations the budget represents. A project-by-project
format usually works well when presenting the budget justification for
a multiple-project application. It must be clear to the reviewers and
to our staff what costs are associated with each project. The budget justification
also needs to address costs for future years and pay particular attention
to explaining any fluctuations in costs in future years.
Question: Regarding infrastructure and the predefined
projects, it's difficult for me to see how you budget for infrastructure
when you have a budget and budgets for three projects that would have
to add up to the total. I'm a little confused about project-specific
budgets and how actually one would include infrastructure and allocate
infrastructure to a specific project. Can you clarify this for me?
Answer: Traditionally,
we have seen separate infrastructure budgets and then individual project
budgets. If for some reason you feel that for your proposal you can allocate
the infrastructure under the projects, which would be fine. But a centralized
infrastructure project is acceptable.
Question: Was that mentioned in the RFA?
Answer: I don't think that the level is detailed. I think it just said that you
need to have an infrastructure component.
Question: So a separate budget for infrastructure is
appropriate then?
Answer: Yes, that's fine.
Question: I'm calling in regard to budget questions. Since
this is an electronic submission, and I must use the application budget
form to capture multiple-budget projects, am I just to fill out the composite
budget on the main form and then attach the various spreadsheets that
break down that composite budget by the projects? If so, where is
that attachment going to go?
Answer: That's
an excellent question and, hopefully it is answered in the Notice that
was just issued on November 7. If you refer to and read that and have
additional questions, you can contact Michelle Burr directly. But we do
give detail on how the budget will have to be done separately, using separate
spreadsheets that you will devise yourself. They will have to be converted
to PDF format and uploaded as part of the budget justification.
Question: I did get the addendum for the budget submission
instructions. However, it's not exactly specific as to where the attachment
should go on the actual electronic application. It does tell me how
to name that stuff, which is fine.
Answer: It
does state that it should be submitted as part of the budget justification
section of the application.
Question: It was stated that the applicant should be
structured with the flexibility to develop new projects on an annual
basis. Do you have guidance as to how much funding should be budgeted
for this?
Answer: We
can't give you any specific guidance as to the amount of money that you
might want or the mechanism you would set up to fill that flexibility. But
as we described previously, on an annual basis and in consultation with
AHRQ as to its programmatic needs, you might specify a process that would
provide some assurance to the review panel that whatever projects were
undertaken would have high scientific merit.
Question: Since the applications will not be reviewed
until March 2007, is there any possibility of extending the deadline
for submission?
Answer: We
cannot extend the deadline for submission. There are other processes built
in between receipt of the application and the meeting of the review panel.
Question: I have a question about the budget and the
Coordinating Center. Does that amount include direct as well as indirect
costs?
Answer: Yes,
that amount includes direct and indirect costs.
Question: Regarding the 25-page limitation, I wanted
to make sure that budget justification and appendices are separate from
those 25 pages.
Answer: That's
correct.
Question: I have
another question in regard to the budget, and how it is related to the
sub-award budget. I'm not sure exactly how much detail you need. I notice,
for example, that the
University of
Washington requires different, more detailed information than the system
will let me see uploaded. So can you give some direction on what we should
ask on some of our sub-awards in terms of the documents that they will need
to submit?
Answer: For
any projects that involve sub-awards, the sub-award budget needs to be
the same detailed budget that you will submit for the main project. It's
the same level of detail.
Question: I have one attachment also that's allowed
in this, the budget justification. Do you require details in terms of
materials like resources and environment?
Answer: Are
you talking about the resources and environment for the overall project
or for each?
Question: My question is about the sub-award.
Answer: For
the sub-award, I believe that should actually be included when you describe
the resources available to the overall project. I think you can address
each organization separately. You can e-mail Michelle Burr if you have
further questions and we'll look into it.
Return to Contents
Closing
Since there are no further questions, we
can conclude our call. We will be sending information to those of you
who participated about when and where the transcript of this call will
be available as well as the slides. Thank you all for your participation
and for your interest in this funding opportunity.
Return to Contents
Current as of November 2006
Internet Citation:
Centers for Education and Research
on Therapeutics (CERTs) Technical Assistance Conference Call. Summary. November 2006. Agency for Healthcare Research and Quality, Rockville, MD http://www.ahrq.gov/fund/certstacall06.htm
540 Gaither Road Rockville, MD 20850