Ambulatory Safety and Quality
Technical Assistance Call
This document summarizes
the technical assistance call for the Ambulatory Safety and Quality: Improving
Quality Through Clinician Use of Health IT funding opportunity announcement
(FOA). Select to access the FOA.
The technical assistance
teleconference was held at the Agency for Healthcare Research and Quality
(AHRQ) conference center in Rockville, MD, on January 4, 2007. If after reading
this document you have any questions or comments, contact ImprovingQuality@AHRQ.hhs.gov.
Introductions
Good afternoon, everyone, and welcome to our Third
Technical Assistance Call. Today, we are talking about the Ambulatory Safety
and Quality program and Improving Quality through Clinician Use of Health
IT Funding Opportunity Announcements (FOAs).
My name is David Meyers. I'm a medical officer
in the Center for Primary Care, Prevention, and Clinical Partnerships here
at the Agency for Healthcare Research and Quality. Helping me with this call
today are several of our staff members:
- Joan Metcalfe, Grants Management.
- Rebecca Roper, Scientific Review Administrator.
- Yvette Davis, Scientific Review Administrator.
Between the four of us, we bring the content expertise,
the budgeting and financial expertise, and the review process expertise that
we hope will be able to help you through the rest of this process.
Slide Presentation
We'll be following a set of slides today that was
provided electronically before the call began: (PowerPoint® file, 185 KB;
Text Version).
If you didn't receive a set, don't worry. We will
also be posting the slides as well as the answers to any questions we'll review
today on our Web site after the call.
At the beginning of the call, it's important for
me to remind everyone that while this technical assistance call is here to
assist you, really what we're doing today is reviewing and highlighting the
FOA and answering any questions that you may have had after reading the FOA.
If at any time something I say conflicts with what's
in the FOA, the FOA wins. It is the true arbiter and everything we say should
be a reflection of what's in it.
With that, we can turn to the second
slide on the conference call agenda. I'm going to talk about the overall
program that this is part of, and some specifics about this FOA on improving
quality through clinician use of Health Information Technology (Health IT).
We've
prepared some frequently asked questions in the slide set that we'll go over
with you. And at the end of the call, we should have plenty of time to answer
questions from the dial-in audience. When we get to that point, we'll ask
everyone to queue up and ask your questions, then the staff as a whole will
answer them as a whole. We won't go (question-answer-question-answer).
Moving
on to the next slide, eligible institutions. This slide is clearly out of
order. But it's here so that if you're not eligible we can let you off the
call now. Eligible institutions under this FOA include organizations that
are public or not-for-profit private institutions. These include healthcare
organizations, universities, faith-based and community organizations. Additionally,
eligible units of local and State government may apply, as well as Indian
and Native American tribal governments or tribally-designated organizations.
For-profit organizations are not eligible to lead the application. They are,
however, encouraged to participate in consortia or as subcontractors with
eligible lead organizations. Additionally, for-profit organizations that
produce Health IT systems are encouraged to apply in partnership with a nonprofit
lead organization.
Because
the focus of this FOA is on improving America's healthcare system, foreign
institutions are not eligible to apply as the lead organization. They too,
however, may come in as members of consortia or subcontractors.
Moving
on. This initiative
that we're speaking about at this hour is part of a larger ambulatory safety
and quality program here at the Agency for Healthcare Research and Quality.
The overall purpose of the initiative is to improve the safety and quality
of ambulatory healthcare in the United States.
We recognize
that much of our earlier work in improving patient safety and quality in the
United States was focused on the inpatient setting. And at this time, we're
not—we are continuing that work while, at the same time, through this program,
expanding it to the increasingly complex ambulatory care setting.
As you'll
note, a large part of the ambulatory safety and quality program has an interest
in harnessing the power of health information technology. This is bringing
together two of our large portfolios of work in quality and patient safety,
and Health IT.
If you'll
click to the next slide, you'll see that this initiative has four components.
And, this afternoon, we're talking about Improving Quality through Clinician
Use of Health IT. For those of you who have been with us earlier in the day,
I do apologize if some of this is repetitive, but we do have some new folks
on the call.
In these
FOAs, we define ambulatory care services as the types of health services
provided by healthcare professionals on an outpatient basis. For these FOAs,
we note that this setting may include the healthcare clinician offices, both
large and small practices, outpatient clinics, community health centers, emergency
departments, urgent care centers, and ambulatory surgery centers. While home
care is often not typically included as part of ambulatory care, for the purposes
of this FOA, home care entities are considered ambulatory and are welcome
to apply.
RFA-HS-07-006
has quite a—you'll see quite a large number of aims and goals. And that's
because, as we'll say later, we recognize that this spans the spectrum from
discovery to translation to measurement and then to widespread dissemination
of strategies. We're interested in projects that investigate novel methods
and projects that evaluate existing strategy for clinician use of Health IT
in the ambulatory care setting.
Earlier
today, we had a call about patient centered care. That FOA has the patient
as the focus. This initiative puts the clinician as the focus of its work.
The whole health care system also is part of this initiative.
We're
looking to improve outcomes through more effective clinical decision support.
So clinical decision support, medication management and care delivery are
all focuses of this FOA. Additionally, we are interested in projects that
examine how Health IT fits into workflow and how the effective use of clinical alerts
and reminders can be integrated into health care system delivery.
Moving
on to the next slide, more specifically on the themes where AHRQ in this FOA
is focused. These include the impact of Health IT on outcomes in the ambulatory
setting, and across high-risk transitions of care, looking at novel approaches
to providing high-quality, appropriate care through the use of Health IT, and the
essential strategies for safe, successful and productive Health IT adoption
in the ambulatory setting.
On the
next slide, entitled "Areas of Interest," we note that the core theme for
this FOA is the clinician's use of Health IT to improve quality. And we have
some illustrative questions including providing information and decision support
to clinicians, clinical workflow and cross-disciplinary teams (that comes
up several times in the FOA and is clearly an interest of ours) as well as
care coordination.
The next
slide brings up the idea that this application is done on the SF 424 (R&R)
form. We'll talk more about the specifics towards the end of the call.
But like in most grants—research grant applications—we expect that:
- You'll describe the specific Health IT application and intervention that you will study.
- You'll develop the project design
and the methodology plan, which includes research design, specific outcome
measures and an evaluation plan.
- You're required also to have your
milestones and there is a requirement to include a timeline.
- You'll assess the primary and secondary
outcomes through a robust analysis plan.
- And then we expect that there'll
be a plan for dissemination included in your project.
The next
several slides just highlight—again, all of these come directly out of the
FOA—some of the requirements of the research design and methods section.
And I'm just going to point out a few specific things on each of these slides.
So these are the highlights of the highlight.
On the
first slide, the applicants should describe the development of the intervention
and the methodology for implementation. Specifically, when possible, the
Health IT intervention should conform to interoperability standards and use
certified products. And if folks have questions about that, we can talk more
at the end.
On the
next slide, you'll see that in addition to specifying the primary and secondary
outcome measures that you would like to use for your project, in the areas
that your project will move forward, AHRQ is requiring, when appropriate,
some additional measurements. Please note, these are not supposed to be the
focus or necessarily will be the focus of your work. This is what AHRQ can
use to look across all of the projects it runs in this FOA as part of the
larger program, as part of our Health IT and patient safety projects as an agency,
and as part of the Department of Health and Human Services.
So, we
are requiring, when appropriate, that you give us some basic measurements
of the adoption of Health IT. They're specified here. There are five of
them. They include:
- The percent of adoption and use
of Health IT in the settings with which you work.
- The percent of patients who receive
appropriate care for prevention, treatment and medication therapy.
- The percent of eligible patients
within practices who have access to their personal health information.
- Patients' and providers' access
to, and utilization of, quality measurement reports and especially if those
are aggregated through an Health IT health information exchange.
- The percent of ambulatory clinicians
within the practices with which you are working that routinely use measurement
tools to evaluate their patients' experience with care.
We give you one way of doing that with
our CAHPS® clinician and group survey tool. Again, we are asking you, when
appropriate, to tell us how you can report to us data on these measurements.
On the
next slide, there are two things I'd like to point out. One is that we are
interested in you and your application talking about the future sustainability
of your intervention after the grant program or the grant funding has ended.
And finally, all projects are required to include a dissemination plan. The
dissemination plan needs to be both in the narrative text of your document
and incorporated into your proposed budget.
We're
very happy to announce that we believe there'll be many successful projects
that we would like to have additional dissemination, above and beyond what
has already been proposed, and that we would like to have a limited competition
for additional dissemination grants for folks under this application. However,
your application to get into the program has to be done without using those
funds for your budget. So your dissemination plan needs to stand alone and
make sense without this additional funding. But just to think in the future,
we're going to help disseminate—we're going to ask you to help us disseminate
further the results of your work.
In the
next slide, I'd like to highlight our concern and the importance that we place
on privacy and security. All applicants are required to describe how, both
in the development and the implementation of their project, privacy and security
issues related to the exchange of sensitive health information will be identified
and addressed. The FOA gives some very nice descriptions of what we mean by
this, some sample ideas of how this can be done, as well as links to resources
to further think about the issues in privacy and security. And as you will
see later, how you've addressed privacy and security issues is one of the
criteria on which all applications will be reviewed by the peer review committee.
On the
next slide, we highlight that this FOA has three special funding preferences.
These are not priority areas that the review committee, the committee of your
peers, will be asked to speak to. These are going to be criteria that AHRQ
uses in determining how we use the pools of money that we have set aside to
fund this FOA.
There
are three of them, and I'll start by noting that we expect many projects will
come in under more than one, if not all three, of these funding preference
areas. To do that, you'll follow the instructions in the FOA—these are
detailed in Section 3.3—and that you will include this information not
in the text of your application but in Appendix 1. Even if you choose not
to come in for one, two or three of them—if you decide you're not interested
in being considered for any of the funding preferences, we ask that you also
have an Appendix 1 and that you include the sentence, "No funding preference
requested." If folks have more specific questions about this, we'd be happy
to talk about that at the end.
The next
slide points out to you that AHRQ is going to use two large considerations
to make its decisions about the funding of applications under this FOA. The
first is the scientific merit of the proposed project as determined by peer
review. This is the area that you have control over. This is what you say
in your application and it's material that will be reviewed by a committee
of your peers. And we'll talk on the next slide about what specifically they
have been tasked to look for.
But the
rest of the slide points out to you that there are many other areas that AHRQ
will consider when making its final decisions about funding considerations,
including how your projects fit into AHRQ's larger goals, other projects being
funded within the Agency and the healthcare needs of the country as a whole.
On the
next page are the specific review criteria that are outlined in the FOA.
These are the headings. I strongly encourage you to read the specific questions
that this review committee will be given and charged with when reviewing your
application. They're categorized into six main areas: significance and impact;
usefulness and generalizability; approach, which includes methods; the investigators
and the partnerships brought to this application; the settings in which the
study is proposed to be conducted; and your budget. I point out that many
people have not seen the terms "usefulness" and "generalizability" used as
review criteria in the past. So, take a look at those.
Additionally,
there are other considerations which the review committee will be asked to
discuss. Those include your degree of responsiveness to the FOA as a whole
in three specific areas worth noting—privacy and security (which we already
mentioned), protection of human subjects from research risk, and inclusion
of priority population. It is to your advantage to make it easy for your
reviewers to understand how you are addressing these three specific areas
in your application.
Additionally,
when you complete your SF 424 (R&R), there are other requirements that
all applications to the Department of Health and Human Services are required
to fill out—additional language on human subjects protection as well as the
inclusion of women and minorities. There are specific tables that you're
asked to complete. Your review committee, while not using it in your score,
are required to state whether or not you've successfully addressed those.
If you do not, you cannot get funding. So please, pay attention to all of
these things.
The review
experts in the room wanted me to correct something I said. Those additional
things in the SF 424 can in fact be used by the reviewers when they consider
your application because it's under your "additional considerations,"
and can be used to downgrade your score if you don't address them satisfactorily,
as well as grounds for not funding the application outright.
On the
next slide, you'll see our mechanisms, some details about the FOA. These
are just the basic things. This is an R18 mechanism. It's a research demonstration
and dissemination project mechanism, the goal of which is to provide support
designed to develop, test and evaluate health service activities. This is
a one-time solicitation. At this time, AHRQ has no plans to reissue this.
You may want to take that into your planning.
We projected
$9 million in total costs in FY 2007 under this FOA which would fund between
20 and 40 grants. All applications must not exceed 36 months in duration.
The total budget cost over the period must not exceed $1.2 million, and the
budget in any one year may not exceed $500,000. Failure to meet any one of
those three criteria are cause for the application not to be reviewed (and
to be returned).
A few
more details.
On the
next slide, we say that AHRQ does not accept modular budgets. We only use
the detailed research and related budget, which is what you'll find in the
SF 424. We do not use the PHS 398 modular budget, and any application that
attempts to use that or submit that will be returned without review.
Specifically,
the SF 424 has replaced the PHS 398, and it's an electronic application.
All applications under this FOA must be submitted electronically using the
forms available and for this specific FOA. There are details in the FOA itself
about how to get the electronic forms as well as how to register your institution
and the principal investigator (PI). And that's worth a little extra time for those who have not done
this before.
Registering
is required before you submit your application, and the process of registering
an institution as well as the PI can take up to six weeks to complete. The
institution must register in two places and the PI also must be registered
in one. Details about that can be found on the AHRQ Web site under the section
under Funding Opportunities and Information on Electronic Application. Additionally,
there are links in the FOA itself.
Going
on, a few last slides. We've
already talked about who's eligible. On the next slide, we talk about the
principal investigator. The details of the skills and qualifications of the
PI are detailed in the FOA and highlighted here. I'd like to point out that
the FOA says that the PI should devote a considerable portion of his or her
time to the project. If they will devote less than 20 percent time, the application
must then include an explicit justification.
Whenever
the FOA says "must" that is a review criteria which are our grants team will
look for. They will go through all applications and make sure that all of
them must have been done. Applications that do not comply with this may not
advance to the peer review process. So it's worth taking a look at what the
musts are. And most of those have been highlighted in today's presentation.
On the
next page, you'll see the key dates for this FOA. Letters of Intent (LOIs) are due
by January 19, 2007. Details about how to submit a letter of intent and what
we are expecting are in the FOA. I'd like to point out that this is a really
important service that you can do both for us and for yourselves. Submitting
a letter of intent is not binding, but it helps AHRQ understand how many applications
we may receive, what types of work we should be ready to evaluate, and those
two things help us plan who should be on the special emphasis panel that will
be reviewing your grant.
So, if
you let us know what you are up to, we can make sure there's somebody who
knows how to speak your language. If you don't let us know, we may be surprised
and that might decrease the quality of the review you will receive. So please,
consider submitting an LOI. You can fax it to us, you can E-mail it to us,
you can send it by mail, but send us a letter of intent.
The application
submission date, the date these are due, is February 14, 2007. You will note
in the FOA itself it also has something called the closing date. The closing
date is just when the computer system shuts down. It is irrelevant for your
purposes. What you need to know is that your application is due on February
14, 2007.
We anticipate
peer reviews will take place in late spring of 2007. There are many factors
that may change that. Using that as our guide, we'd like to tell you that
your earliest anticipated start date when you put together your budget and
timeline should be July 2007.
With that,
we're going through a few questions that folks have asked us in advance before
we turn it over to the group.
So flipping
on to the next slide, "Are hospitals allowed to apply?" I'm sure most of
you know the answer. Yes, public and nonprofit healthcare institution, including
hospitals, are eligible to apply. For-profit organizations, including for-profit
healthcare organizations and Health IT vendors can be included in consortia
or as subcontractors but cannot lead the application.
On the
next slide, "Can an application be submitted for all four ASQ funding opportunities?"
And the answer is: an institution can submit applications to more than one
Requests for Application (RFA), and each of these RFAs will be reviewed by a different study section.
That said, you may not turn in the same application to more than one FOA.
Suppose you're working on clinician-patient communications. So it's both
patient-centered and clinician data. You've got to make the call about what
outcomes are you most interested in and applying only for that FOA. But from
an institutional perspective, if different teams are working on different
projects, they may apply for more than one application as an institution.
On the
next slide, "Does AHRQ accept modular budgets?" I hope the answer is clear,
but it's worth repeating. No. Applications received with a modular budget
will be returned without review.
On the
next slide, "Are emergency rooms or emergency departments ambulatory care
sites?" For the purpose of this FOA, absolutely, please join us. We list
again, as we've described earlier, the different settings that are involved,
and note home care agencies—and care delivered in the home are included under
this FOA.
On the
next slide, a very often asked question is, "Does AHRQ accept co-PIs?" The
answer is no. AHRQ requires that the lead institution designate one and only
one individual as the project's principal investigator. If you'd like to
give other titles to other members of the team such as co-investigators, that's
acceptable. But there needs to be one primary principal investigator that's
listed in the application.
And the
next question, "How much detail should the letter of intent include?" Again,
we use this to estimate how many peer reviewers and what expertise we need.
We do not give you any feedback on the letter itself. It is nonbinding.
The content that we request from you is described here as well as in the FOA.
It is not required if you don't get it together in time. That's okay. And
I'd also say, if it's the 20th of January and you realized you haven't sent
it, send it anyway—it still helps us. But we're on a time deadline here,
so we request that they come in here as early as possible.
Question and Answer Period
All of today's discussions and the answers as well
as the slides will be posted on our Web site, and we will try to send an electronic
notification to everyone who's registered when those are available for review.
If your question doesn't get answered today, you can submit your questions
to us electronically at ImprovingQuality@AHRQ.hhs.gov. That E-mail
is on the last slide as well as in the FOA. Any questions received by the
end of today, we will try to include in the transcripts as well as have answers
for them.
Finally, we've discovered, this being our third
call, some questions and comments from one FOA are relevant to another. So
it may be beneficial for you, even if you are only interested in this FOA,
to review the technical assistant questions of the other FOAs to make sure
you've got the whole picture.
Responses to Presubmitted Questions
Role of the Principal Investigator
Question: What types of education or experience are expected
for the PI?
Answer: We refer you back to the FOA, where there is a section on eligibility of
PIs where we described what is expected. There are many different people
who can fill this role. It is expected that the PI will not have expertise
in all aspects of any particular project but that the PI will coordinate the
work of the team. And over the course of your application, you'll explain
how the team works together to meet and be able to do all of the things that
you're setting out to do.
One requirement is that the PI must or is encouraged
to devote a significant portion of his/her time, defined in this FOA as at
least 20 percent. If you believe your application is strong even without
your PIs devoting 20 percent of their time, you must include a detailed justification
for why you believe that this project can still go forward in that type of
structure.
Question: Can we have the same PI on two projects and can that
go forward?
Answer: Well, you've heard the requirement that the PI be able to devote 20 percent
of his or her time to the project. If the PI has the time to give 20 percent
to Project A and 25 percent to Project B, that's acceptable. If that would
push them over to the Federal limits and they're now working 122 percent,
that's not acceptable. But the same PI can be listed on more than one application.
You should note whether they have the time, if they were so fortunate as to
be funded under both applications.
Question: How do you select the PI?
Answer: There is a combination of researchers, clinicians, and vendors and who's
the right person? There's no specific answer. It's your job in the application
to justify who you've picked and how they will coordinate the team and how
the efforts of all the members of the team come together to give us the complete
package. So it can be an M.D., a researcher, or none of the above; it can
be the person from the ambulatory setting, the research setting, or even the
vendor setting, if all the other requirements of the grant are met.
Question: The application notes that the PI must or should be affiliated
with the applicant organization. Does that mean that the PI must be
employed by the lead applicant organization?
Answer: From our perspective, that is the easiest application to process and is
most easily acceptable.
If the PI is not employed by the lead organization,
the application still may be considered. It is critical, however, that the
application explains how the lead organization, who is entering into a fiscal
responsibility relationship with AHRQ, can guarantee accountability of the
actions of this person. So, showing a defined, contractual relationship is
one way that that might be achieved. And good documentation and description
of how decisions are made and people are accountable and that the projects
will move forward is going to be required and looked at very closely in an
application where the PI is not employed by the lead institution.
Vendor-Based Systems
Question: Is there any issue to consider in regard to implementing a technology
on a vendor-based system?
Answer: While clearly, there are multiple issues, I think the main question here
was, is it okay to move forward or to have something that is a commercial
product that's already in existence? And the answer to that is yes and there's
even language in the FOA that supports that and encourages people to work
with things that have already been done and explore furthering their use.
The use of a vendor-based or proprietary software, hardware, or system is
acceptable. However, the vendor in that case, if they are a for-profit institution,
cannot be the lead. But under this proposed scenario, it would probably be
a subcontractor and therefore okay.
Methodology
Question: Is a particular methodology such as the randomized control trial preferred
for the application?
Answer: No. The FOA is an R18 and a research application.
But specifically, we ask that your methods are the best methods to achieve
the outcome that you've set out for yourself. These aren't simply implementation
grants, it's not just "can-you-make-a-system;" you do need specific
outcomes that will be measured in an analysis plan and there must be a research
structure. That said, however, the FOA gives you clear guidance to pick whatever
methodology makes more sense for the project that you're working on, and then
give that justification to your peer review committee.
Budget Concerns
Question: Are there any potential restrictions placed on the
budget? More specifically, Can the money be used for infrastructure or is
it to be used for evaluation only?
Answer: In this grant, funding from the AHRQ portion of the budget may be used
for hardware, software, research, consultation services, or even for things
involved in the implementation of Health IT. However, there is no set figure or
percentage outlined in the FOA. With that said, I recommend that you look
at the review criteria and note that your application will be reviewed on
multiple criteria including the budget. And, specifically, your peers will
be asked to review whether the budget you've proposed is reasonable and appropriate
for the work that you've set out to do. And they're also going to be saying,
are the outcomes that you are proposing and the methods you have for evaluating
appropriate?
So the grants that are very high in implementation
with no evaluation, while they would meet the criteria, may not be favorably
reviewed based on their outcome. A project that spent a lot of money on technology
may not be considered in the budget section as being responsive to the FOA.
However, that budget is for you to decide and to make the best case you can
for how you spend your money. The review people also have asked me to say
that any money spent on technology that is proposed in the budget needs to
be justified in the budget section as well as in the description of why it's
necessary in your research plan.
Question: Can funds be used retroactively to pay for an Health IT
system that was previously purchased?
Answer: That is not acceptable. Funding is only prospective and begins on the
date of your start date based on if you get an award. It'll be specified in
your award letter. However, that question went on to say, what if we've got
a contract that's ongoing? Could we pay our next month's technology bill
that's our contractual obligation using this fund? If you are funded and
you've given the justification and your peers say yes, that seems like an
appropriate use of these funds, that potentially would be acceptable to the
extent you prorate it, that is, to the present time, and to the extent used
by this project.
Question: Can a State primary care association (PCA) that represents
federally qualified health centers apply under the vulnerable populations
funding preference on behalf of its members?
Answer: I would separate that out into two parts and say absolutely, any State
PCA, if it is a nonprofit institution that meets all the other Federal requirements
to receive funding, most certainly can be a lead applicant. They would be
able to apply any special preferences based on their work; they would apply
based on the settings in which the work is going to be done, and then could
list their federally qualified health centers. The qualifications are specifically
listed in Section 3.3, including "are they federally qualified health
centers?" And they could list them and that they are qualified and therefore
request funding preference.
Projects Yet To Be Implemented
Question: What if we have our electronic medical records (EMR) system chosen but have not implemented it yet? We don't have an EMR system in place today, but we will in the future,
so can we write an application?
Answer: Yes, absolutely. However, there's a big contingency with that. If you
can't give the specifics about how things are going to be done, your reviewers
may not be convinced that you have the ability to get it done and measure
the outcomes that you've stated you will do. So the more definitive you can
be in the application, the more definitive you can be about your methods and
your outcome. However, if you're in process or beginning to implement, you
can include that as part of this project and in your description. So that's
a yes—but.
Open Question Period for Conference Call Participants
Question: Please make the distinction between the Improving
Quality FOA and the Patient-Centered Care FOA, because clearly, these two
have the potential to overlap.
Answer: The Patient-Centered Care FOA puts as its central focus the patient's
experience of care. It's harnessing Health IT to improve quality and safety, but
really to improve the patient's experience of care. Therefore, the primary
outcomes of such a study are going to be the patient's experience of care,
the patient's self-management skills, or definitions of patient-centered care.
This FOA, Improving Quality, puts as its central focus the clinician
and the clinician's ability to improve quality by using health information
technology. Its outcomes may be patient health outcomes, efficiency outcomes,
quality outcomes, or it may be others or the outcomes described in the FOA.
The difference is not so much in what the intervention is, the same intervention
could be done in two different ways, it's from whose perspective you are measuring.
Clearly, somebody could come up with an idea that would wrap around both,
and if you do that, we're asking that you pick which one you are going to
put as primary. Look at your primary outcomes. If it's the patient-centered
measures of care, then you would go under RFA-HS-07-007; if you're looking
at improving quality health outcomes, quality outcomes, or efficiency outcomes
from the system or the clinician's perspective, that would be under RFA-HS-07-006,
the one we're discussing at the moment. If you have additional questions,
feel free to contact us about that.
Question: My question relates to the mechanics of putting together the application.
Can you clarify where letters of support should be provided in the electronic
application? Is it under Item 16 of the research plan, or as one of the attachments
in the Appendix (where I have instructions) in both places?
Answer: The attachment section is a good place for most kinds of letters of support.
If there is a letter of support that rises to such an important level that
you'll need it to be part of your research plan (it's not intuitive to me
what it would be but if it was), you would be able to include that as part
of your research plan, but the FOA describes how those are the attachments,
and what type of attachments, and letters of support in general belong in
that section.
Question: Is the inclusion enrollment report under the research
plan relevant to this particular funding opportunity?
Answer: A report that's required of all grant applications under the SF 424 in
Federal health agencies. It appears in your SF 424 in its correct place.
You should complete it; it may be used in the review process. However, it
does not add to your page limit for your research section plan. That said,
how you handle human subject protection, privacy and security is a larger
issue that may in fact need to be part of your research plan and specifically
part of the evaluation criteria.
Question: Also, regarding the resource sharing plan under the
research plan, is that essentially the same thing as the discussion of a plan
for dissemination?
Answer: Page 190 of the SF 424(R&R) Guidelines
describes what a resource sharing plan is. The guidelines describe a specific
aspect of data sharing for large grants over $500,000 in direct costs, which
does not apply to this FOA (where the maximum of $500,000 applies to total
costs.) The resource sharing plan is different from a dissemination plan,
which addresses the activities of the project in sharing its findings, lessons
learned, and/or tools produced.
Question: Are patient-centered outcomes or patient measures necessary for this
application?
Answer: You can look specifically in the review criteria to see they are not required
unless appropriate. They may be appropriate; they may be the best measures
for some projects. Other people's projects may like to look at other outcomes.
For example, efficiency outcomes in general could also be considered acceptable,
and examples are given in the review criteria that you may look at more.
Question 1: Under the slide, "Mechanism of support and Funds
Available," the bullet point that says "The budget also may not exceed $500,000
in total cost in any single project year," that $500,000, is that AHRQ funds
or is that our budget?
Question 2: If our total budget exceeds what is available in
the grant, how are we supposed to specify what portions of the budget are
going to be coming from our organization versus what we're going to use from
the grant money?
Answer: Two questions were asked about the total budget and how to apply. They
were wonderful questions that get to the idea that AHRQ encourages in-kind
support from vendors, from institutions involved in the application. That
is looked upon very favorably. In your 424 budget, however, include only the
funds that you are requesting from AHRQ. You will then have a budget justification
section. The budget justification section is where you can put those kinds
of in-kind contributions and note them. That answers our next question of
how do we show the $500,000 in 1 year. The budget per year that you're going
to submit is only for the AHRQ-funded portion. But again, in-kind support
is encouraged and should be included in your budget justification section,
and even in your research plan or description of the plan.
Question: There were five measurements, outcome measurements,
that were indicated on the slide. On one of them, what if there is no measurement
tool in use to evaluate a patient's experience? Can one start with the grant
to put that into effect?
Answer: From that general question, the FOA says, "when appropriate."
If it is not applicable to the work you are doing and impossible for you to
do, state that; that is an acceptable answer.
However, the question specifically asked about Measure 5, which
is about the patient's experience with care. AHRQ strongly encourages that
you use AHRQ's CAHPS® measurement set for evaluating patient's experience
with care. If your clinicians are currently using another instrument and
you want to tell about that, because the question says, "our patients," you
could answer, Part A, how many of our providers are measuring it at all, and
then could ask, Part B, how many are using the AHRQ CAHPS® measurement.
That would be a lovely way to handle it.
Question: Will there be other patient measures coming out of
this FOA, and Are we providing other ones?
Answer: No. We strongly recommend your using AHRQ CAHPS®.
Question: Could you give us a little more clarification on
the dissemination plan as it relates to Health IT? Are we are talking about the
findings? Are we talking about that which has been developed? If you can
just give us a little more clarification on that, I'd appreciate it.
Answer: Specifically, are we asking you to tell the world about your findings?
Yes, we are. That is generally what is considered to be part of the determination
plan. It can also be about their broader interpretation; you are welcome to
tell people about the methods you used, your experience and lessons learned
in the implementation process, and disseminate the other learning(s) that
took place. You'll find those would've also been on this part of your findings.
There is not much description of the dissemination plan in the
earlier part of the FOA, but if you'd like to understand what AHRQ is interested
in, look on Page 23 of the FOA, where we describe and give you the questions
that the review panel would be asked in reviewing your application. Under
"usefulness and generalizability," they give you the ideas—the
specific questions there describe the kinds of things AHRQ wants you to be
talking about in your dissemination plan.
Question: Our local Practice-based Research Network (PBRN) is participating as one of several
ambulatory sites. Would that be listed in the funding preferences section
or do they have to be the lead organization?
Answer: Does the PBRN need to be the lead organization to qualify for the funding
preference? No, it does not. The lead organization must be a non-profit et
cetera. If a PBRN is participating as a partner they should be mentioned in
the funding preference section, but if they are one of a 100 partners, we
would need to know that. If they are 60 percent of the sites, we would want
to know that, to understand how you meet this funding preference.
Question: I just wanted a little clarification about what budget
funds can be spent on, whether they primarily have to be evaluation or whether
they could also include supporting the IT intervention?
[See other budget-related responses.] I think we've
answered that.
Question: I have two questions. One is regarding home care—what
kind of medicine is covered? Second, how much of the budget can be used for
development of the IT portion itself?
Answer: Absolutely. For the purposes of this FOA, care delivered in the home is
considered ambulatory care; however, there needs to be an organization that
is ambulatory care focused involved in the application.
[See budget-related responses for second part of question.]
Question: Slide 11 suggests that the applicant has to be an ambulatory health
provider or be implementing the Health IT system in a provider organization.
What would the status be if a nonprofit is implementing but not in a provider
organization, but in free-standing offices?
Answer: There needs to be a nonprofit organization
as the lead and overall in the project. You must be working in some way with
ambulatory clinicians or ambulatory providers. So, if you have a lead that's
a non-ambulatory, but they're working with ambulatory, you're okay; but the
lead still has to be the non-profit.
Question: If we're fortunate enough to receive funding from
you from two different parts of the Ambulatory Safety and Quality Program,
would we be able to use the same PI for both projects?
Answer: [Go to: Presubmitted Responses.]
Question: Is a project applicable that would focus much of
the cost and the activity on developing the IT application, and the evaluation
would be through a subcontract? And, if that is acceptable, is the PI the
project director who is developing and implementing the IT application, or
is it the subcontractor who's actually researching and evaluating the impact
of that application?
Answer: [Go to: Presubmitted Responses.]
Question: Are pharmacists and ambulatory clinician providers
using Health IT to improve the quality of care an acceptable group under this FOA?
Answer: If your project applies to the pharmacist and will show their ambulatory
clinicians are using Health IT to improve the quality of care, that's an acceptable
group under this FOA.
Question: We work in a collaborative relationship with tribes in a three-state
area under the direction of a tribal health directors' organization and an
overarching board composed of tribal leaders. And we do work collaboratively
with university systems in the area of research. We also do quite a bit in
the area of epidemiology and in financial planning. Is it good enough to have
a collaborative relationship even though we may not be a provider per se?
Answer: If the lead is a non-profit and you're working with ambulatory clinicians
or in an ambulatory setting, you are able to apply, so that grantee should
feel free to apply.
Question: Yes. I had two questions. First, could you clarify what is meant by
machine-actionable? And second, could you speak to your preference for an
randomized controlled trial (RCT) model versus a demonstration project with an outcome?
[Go to: Presubmitted Responses for second part of question.]
Answer: The FOA uses the phrase "machine actionable information to improve quality."
It is a term of art. As you look through this whole field, we believe you
will see that some of those definitions talk about the idea that a computer
could have the capability to use a piece of information, translate it, and
add to its value.
Question: Can grant funding be used retroactively to purchase
any equipment or software or, for that matter, any ongoing payments to already
contracted Health IT services if these prior purchases or continuing payments relate
to the grant?
Answer: [Go to: Presubmitted Responses.]
Question: Health IT adoption by patients or physicians was heavily influenced by behavioral
aspects for both. Is there room in these grants to evaluate behavioral aspects?
How will reviewers consider that as part of the evaluation plan?
Answer: That would be acceptable. Just talking about how people think and their
interactions and their behaviors around computers, without relating it to
the ability to improve quality, would not be acceptable. We refer you to the
significance and impact review criteria section and the questions asked there
as well as the usefulness and generalizability section.
Question: Please speak to the distinction between the primary
and the secondary outcome measures you're looking for. And also, how these
should be related in the narrative to the five outcomes that are required?
Answer: This is an important distinction to make. The primary outcomes are what
you're really interested in, as well as your secondary outcomes. You're going
to write a research plan about those outcomes and how you're going to measure
them. You'll talk about how your intervention is going to make a difference
in the quality and how you therefore are going to measure the differences
and show an outcome change. That whole set has to be done and it's a complete
package.
We are asking you to report to us how you can report data on
the five measures. These are not what your intervention needs to be focused
on. Nonetheless, there are things that, as you go along in your business,
we are asking you to collect data about and feed them back to us. They do
not have to change. You may report zero even though zero is the worst answer
to a question or you may say 97 percent. We're not going to grade you or evaluate
you on what the number is; but we do need to see in this application that
you're willing to help AHRQ collect the standard measures that are being used
to compare across multiple initiatives. If you have additional questions about
that, please let us know.
Question: When we register our PI, will we be told for certain
how long it will take to process? Because right now, we're about 6 weeks
out from the deadline, and if it takes 6 weeks to process, we probably shouldn't
go ahead with our application.
Answer: I said that registering may take 6 weeks and that's why this call is occurring
now so you actually can start. It rarely takes 6 weeks. It will take 6 weeks
if you make lots of mistakes and have lots of complications. It took 6 weeks
closer to the beginning of the registration when, many institutions around
the country were suddenly trying to register. Because this process is now
well under way, the turnaround time can be as quick as 1 week. So, I would
encourage somebody who hasn't yet started the process to start it now but
if they're interested in applying, not to give up, this can be done.
Question: This is our first opportunity to submit a National Institutes of Health (NIH) research
grant and we were wondering if there is any possibility of reviewing past
applications—some idea of what it looks like when it's finished?
Answer: Unfortunately, we don't have that. There are some videos available on
the AHRQ and NIH Web sites about the application process that are essentially
tutorials on applying and preparing a grant application. They are very useful
for explaining the different sections, writing a budget, preparing the study
section, the type of feedback you can expect.
Question: Our clinics currently don't have an EMR in place,
but our IT team is actually in the final stages of selecting our EMR system.
Am I correct in saying that we already need to have this EMR system chosen
in order to complete the application? The application can't be filled out
with just a general broad base as it needs to have details specific to the
exact system and its components? Is that correct?
Answer: [Go to: Presubmitted Responses.]
Question: Please provide more information about the security
and privacy language.
Answer: AHRQ is certainly not interested in proprietary secrets. Additionally,
if you work with some group, and there was something you determined was proprietary,
if there was a "freedom of information act" request, you're given
an opportunity to address that as your personal trade secret. So, don't worry
about that.
If you look at the FOA, and again, I recommend everybody
do so, the section immediately preceding Section 2 and above the paragraph
on outcome metrics, there are two paragraphs that describe that we're talking
about policy. We want to know that systems are put in place to ensure that
privacy and security matters are continually addressed and reassessed. Now
clinicians are broadening the partners in this process, and even if nothing
ever goes wrong, the right policies should be there to ensure that as things
change in the future, these issues are being incorporated into your planning
and redevelopment. We want to understand how you began to think about incorporating
privacy and security when you developed your plan.
This is a policy-level question that we encourage
everyone to more actively think through. There are some wonderful resources
within the FOA to help you further your thinking, as well as specific examples
to explain the kinds of things we expect to see in this section.
Closing
We will pull all of these questions together and provide
a transcript as well as copies of the slides to you electronically.
Letters of intent are due on January 19 and the FOA is due in February.
We look forward to hearing from many of you. Thank you so much.
Good night.
Current as of January 2007
Internet Citation:
Improving Quality Through Clinician Use of Health IT. Technical
Assistance Conference Call. January 2007. Agency for Healthcare
Research and Quality, Rockville, MD. http://www.ahrq.gov/fund/taiqhit.htm.
540 Gaither Road Rockville, MD 20850