Chapter 3: Getting Involved in the Research Process (continued)

Technical Briefs

Technical Briefs are rapid reviews of what is known about a specific medical intervention rather than comprehensive evaluations and therefore do not undergo Topic Refinement but proceed directly from the Kick-off call to the Research Review. Because one of the objectives of Technical Briefs is to identify future research needs, the Technical Brief process does not include the production of a separate Future Research Needs document.

Research Review

Since Technical Briefs are assessments of the current state and implications of new medical technologies, Key Informant interviews are an important resource for identifying how the technology in question is currently used, the major issues and controversies surrounding the technology, and strategies for acquiring information on the technology. Therefore, an integral part of the research process for Technical Briefs is interviews with subject experts and end-users of the technology, such as patients and caregivers, practicing clinicians, relevant professional and consumer organizations, purchasers of health care, and others with experience in making health care decisions relevant to the topic.

This input may be gathered through key informant calls, which are scheduled and coordinated by the EPC assigned to do the research, or the EPC may carry out individual interviews, depending on the topic.

Get Involved as a Key Informant Key informants are stakeholders with direct experience with the topic being researched, as patients or caregivers, clinicians, policymakers, insurers, or other health care decisionmakers. Key informants offer unique perspectives that help to refine key questions before the research begins. They also provide context, as well as help direct questions for specific considerations such as side-effects, benefits, harms, and quality-of-life issues. The EPC assigned to a research topic will invite approximately six to eight individuals to participate in the topic refinement process as key informants. The nominator of the topic will often be invited to participate to address the original intent of the nomination and to increase the likelihood that the end products will meet the originating need. All key informants must complete Disclosure of Interest forms, and may be asked to submit a short description of their experience with the topic. The requirement of disclosure bolsters transparency, assists in mitigating bias, and helps create a balance of perspectives among the key informant group. Once key informants are approved by AHRQ to participate, the EPC is responsible for scheduling and coordinating conference calls and/or other opportunities for input. The number of calls or methods used to collect input will vary depending on the complexity of the topic. It is recommended that calls include as many of the identified key informants as possible to foster more robust discussions. In the event that this is not possible, it may be necessary to schedule individual calls, calls with subsets of the identified key informants, or use other methods for soliciting input from key informants. The research phase of a Technical Brief is expected to take 4 months. Key informants should expect to: Submit a completed Disclosure of Interest forms. Submit a brief description of their experience with the topic. Participate in at least one phone call with the EPC, either individually or with other key informants. Typically, calls last 1 hour, and use a toll-free number. Often it is necessary to schedule multiple calls. Be available to answer questions if the research team requires additional information. Have their names and contact information shared with the Scientific Resource Center for potential consultation on future work, unless otherwise requested. Key informants who are also nominators should anticipate that the original nomination may be altered or changed during the topic refinement process, in order to ensure the greatest possible application and relevancy. The Effective Health Care Program assists EPCs in identifying and supporting key informants in the topic refinement process. To indicate interest in participating as a key informant, contact the Effective Health Care Program at (301) 427-1502 or EffectiveHealthCare@ahrq.hhs.gov.

Peer and Public Review

Once a draft of the research review has been completed by the EPC, a peer review panel is assembled to provide additional and technical review of the report. The peer review process is coordinated through the Scientific Resource Center.

Get Involved in the Peer Review Process Research products undergo a peer review process to ensure scientific integrity and quality of research reports. AHRQ, the Scientific Resource Center, and the assigned EPC identify peer reviewers for specific topics. Decisionmaker organizations (such as professional societies) are encouraged to suggest experts to participate in peer review. Suggestions for peer reviewers can be made during the topic nomination, topic development, and topic refinement processes by contacting EffectiveHealthCare@ahrq.hhs.gov. The Scientific Resource Center coordinates the peer review process, which typically lasts 3 months. The EPC considers all peer review comments and modifies the final report as appropriate. Peer reviewers should expect to: Complete their review of the draft research review within 28 days. Have their names and contact information shared with the Effective Health Care Program for potential consultation on future work unless otherwise requested.

Simultaneous with the draft report undergoing peer review, it is posted on the EHC Program Web site for public comment. An announcement is sent through the EHC Program LISTSERV® that the draft is available for comment. Reports are typically available online for public comment for 4 weeks. To sign up for EHC Program listserv notification, go to the EHC Program Web site (http://www.effectivehealthcare.ahrq.gov) and select "Join the E-mail List" in the lower left corner.

Get Involved by Providing Public Comments on the Draft Report Anyone may comment on a draft report, and the assigned EPC will consider incorporating feedback in the final key questions. It is critical that the questions posed by these documents reflect the concerns and dilemmas of consumers, clinicians, policymakers, and other health care decisionmakers. Public posting is another opportunity for involvement of the whole range of stakeholders and a way to ensure the broadest possible relevancy of the research report. Anyone who wishes to comment on the Key Questions should expect to: Post their comments through the Web site during the 4-week posting period at http://effectivehealthcare.ahrq.gov/index.cfm/research-available-for-comment/. Have their names and contact information shared with the Effective Health Care Program for potential consultation on future work unless otherwise requested. You can sign up at http://effectivehealthcare.ahrq.gov/index.cfm/join-the-email-list1/ to receive alerts when draft reports are posted for clinical areas that interest you.

After all public and peer review comments have been received, the final report is prepared. The process of responding to and addressing public comments can take up to 3 months. The final report is then posted on the EHC Program Web site at http://www.effectivehealthcare.ahrq.gov. A notice of availability is also sent to individuals and organizations who have signed up through the AHRQ listserv to receive announcements.

AHRQ supports and is committed to the transparency of its review processes. Therefore, the following are posted on the EHC Program Web site within 3 months after a final report is posted: (1) all comments received from the public, and (2) all the responses made by the authors of a draft report to the public comments (i.e., the "disposition of comments"). Each comment will be listed with the name and affiliation of the commentator, if such information is provided. Public commentators are not required to provide their names or affiliations to submit suggestions or comments. Contact information will be used to communicate with commentators if there are questions about submitted comments.

C. Evidence Generation

Research Reviews, developed by Evidence-based Practice Centers, represent synthesis of existing literature, most of which is already published, while the Research Reports, developed by the DEcIDE (Developing Evidence to Inform Decisions about Effectiveness), are original research. Since the Research Reports are usually submitted for publication to medical journals, AHRQ honors standard journal embargo policy for original research, which generally does not allow pre-release of study results before they are published. This policy allows for independent review of a study by journal peer review as well as the opportunity to disseminate study results through the journal publication. Hence, the main opportunity for comment on Research Reports is similar to other original research funded at AHRQ and the National Institute of Health (NIH); i.e., as letters to the editor, invited editorials, etc. In the future, AHRQ may allow for comment on the study questions for Research Reports but currently we are working directly with stakeholders to identify and refine research questions.

Peer Review

Once a draft of the research report has been completed by the DEcIDE Center, a peer review panel is assembled to provide additional and technical review of the report. The peer review process is coordinated through the Scientific Resource Center.

Get Involved in the Peer Review Process Research products undergo a peer review process to ensure the scientific integrity and quality of research reports. AHRQ, the Scientific Resource Center, and the assigned DEcIDE identify peer reviewers for specific topics. Suggestions for peer reviewers can be made by contacting EffectiveHealthCare@ahrq.hhs.gov. The Scientific Resource Center coordinates the peer review process, which typically lasts 3 months. The DEcIDE Center considers all peer review comments and modifies the final report as appropriate. Peer reviewers should expect to: Complete their review of the draft research review within 4 weeks. Have their names and contact information shared with the Effective Health Care Program for potential consultation on future work unless otherwise requested.

After all public and peer review comments have been received, the final report is prepared. The process of responding to and addressing public comments can take up to 3 months. The final report is then posted on the EHC Program Web site at http://www.effectivehealthcare.ahrq.gov. A notice of availability is also sent to individuals and organizations who have signed up through the AHRQ LISTSERV® to receive announcements.

D. Product Translation and Dissemination

The John M. Eisenberg Center for Clinical Decisions and Communications Science produces and disseminates user-friendly, actionable summaries of research reviews and reports for health care consumers, clinicians, and policymakers. These guides are designed to facilitate effective communication and decisionmaking about test or treatment choices between clinicians and patients, and to provide evidence-based decision tools for policymakers.

Translation

To ensure that translations of EHC Program research accurately reflect the needs of audience members as well as the science itself, the Eisenberg Center is involved throughout the systematic review process, listening carefully to key informants and technical expert panelists as they interact with EPCs, and interacting with investigators as they review public comments and refine reports. The Eisenberg Center also engages their own medical content experts to better understand the factors that must be considered by both patients and clinicians when making medical decisions on specific topics and to assist the Center in producing scientifically accurate translations of research findings.

Translation of research into practical decisionmaking tools for consumers, clinicians, and policymakers is a pivotal aspect of AHRQ's overarching goal to support the practice of evidence-based medicine. Effective translation of research is critical to ensuring that decisionmakers have access to high quality evidence and tools for making informed decisions.

Input from patients, their caregivers, clinicians, and policymakers guides the creation of summaries and decision tools by providing a context for decisionmaking as well as feedback on the tools themselves. For each topic, a consumer panel, clinician panel, and policymaker panel are created early in the process. Panel members may participate in several group and individual conversations conducted by the Eisenberg Center or its associates.

Topic Decision—Context Groups

Consumer, clinician, and policymaker panelists can first participate in focus groups or an individual interview that explores the context of a specific topic currently under systematic review. During these interviews, panelists may be asked to share their experiences with the condition or certain treatment choices, their values and preferences in information-seeking and decisionmaking, and their challenges in deciding the best choice for themselves or others. This information is used to guide the Eisenberg Center in developing contextually relevant materials that speak directly to decisionmaking needs and situations among a broad audience base. The Center is careful to ensure that all panelist information is kept confidential, and that information gathered is never identified as from a specific individual. Patient/caregivers panelists who have personal experiences with the condition being studied, particularly those who represent one of AHRQ's priority populations are sought for these conversations. Clinician and policymaker panelists who have experience treating or setting policies on the test or treatment being studied are also sought for these conversations. Chosen panelists are paid for their time.

Get Involved in Topic Decision—Context Groups The role of a panelist is to participate in interviews or group discussions to provide context and experience related to a specific health condition, and to test product messages. Panelists should expect to: Participate in a 30- to 45-minute conversation with an Eisenberg Center associate, either in person or over the telephone and either alone or with other consumers/clinicians/policymakers. This conversation is recorded and transcribed, although individual panelist names are not included in the transcript or reported. Discuss their own health and medical experiences, their values and preferences, and their habits or information collection, whether it be from brochures, magazines, television, radio, internet, or other sources. Discuss the challenges faced and strategies used as a patient of, providing clinical treatment for, or deciding policy about the specific condition being studied. Provide informed consent of their participation following a full disclosure of all possible risks and benefits of participating in the interview. Receive compensation for their time spent in conversation. Have their names and contact information shared with the EHC for potential consultation on future work unless otherwise requested.

Summary Guide Review

The Eisenberg Center consults with AHRQ and the SRC to identify and invite individual representatives of decisionmaker organizations to review and provide feedback on draft information products and decision tools. The purpose of these reviews is to ensure the scientific accuracy of the products, and to confirm the contextual relevance of the content. Reviewers include individuals that have been involved throughout the research process, as well as those who have not been involved and can provide a "fresh eye" on decisions made throughout development. Feedback received from reviewers of these products is used to revise and improve the content or graphics of the guides. Product reviews generally take less than an hour for reviewers to complete.

Get Involved in Summary Guide Review The role of a product reviewer is to provide review and comment on specific draft products. Product reviewers should expect to: Receive information products and decision tools by mail or electronic delivery for review. Receive a set of instructions and a formal review form to assist in the product review process. Receive a clear timeline and return path for the submission of comments. Receive compensation for their time spent reviewing products. Have their names and contact information shared with the Effective Health Care Program for potential consultation on future work unless otherwise requested.

Product User Feedback

Once draft summary guides are developed and reviewed, the Eisenberg Center invites consumers, clinicians, policymakers, and other decisionmakers who are potential users of the products to provide feedback on their ease of comprehension, usefulness, and actionability. Feedback received from potential users of these products is used to revise and improve the content or graphics of the guides. Often, panelists may encounter several "rounds" of products to confirm if changes are leading to a more understandable and useful information product or decision tool. Length of time involved in these interviews is between 60 and 90 minutes.

Get Involved by Providing User Feedback The role of a User panelist is to participate in interviews or group discussions to test products. Panelists should expect to: Receive samples of draft information products/decision tools in either print or electronic version to read, or Attend a session where Eisenberg Center associates can observe the panelists' interaction with the information product/decision tool to understand their initial interaction process with these products. Answer specific questions about their interaction with the guides as they experience them. Provide honest feedback and suggestions on making the guides more understandable or useful. Provide informed consent of their participation following a full disclosure of all possible risks and benefits of participating in the interview. Receive compensation for their time spent in conversation. Have their names and contact information shared with the Effective Health Care Program for potential consultation on future work unless otherwise requested.

Dissemination

Having stakeholders distribute, talk about, model usage, and report outcomes from the use of EHC Program research products and summary guides is critical to maximizing the understanding of how the work of the EHC Program improves the quality of health care decisionmaking. AHRQ and the Eisenberg Center employ a variety of strategies to disseminate products, including distribution of resources through consumer and professional organizations, societies and associations, and databases such as the National Library of Medicine and electronic clinical decision support services.

Get Involved in Product Dissemination Stakeholders can get involved in dissemination by: Distributing products to their organization's members or clients. Sharing information about the EHC Program and its products in their organization's newsletters or other communication. Participating in efforts to measure the use and impact of the products, programs or policies derived from EHC reports. Linking to the EHC Program Web site and/or the EHC Program products. Making presentations to their organization or other audiences regarding the Effective Health Care Program, or any of its products, including successes they have had from using them.

To obtain EHC Program products or tools, visit the Web site at http://www.effectivehealthcare.ahrq.gov, or request copies by calling 1-800-358-9295.

Return to Contents
Proceed to Next Section