Skip Navigation
< Back

What Is the Evidence Rating?

Every profile includes an Evidence Rating, presented in the "Snapshot" and "Did It Work?" sections, which is an assessment of the quality and strength of the evidence that the results described in the profile are due to the innovation and not to other factors. The Evidence Rating is one type of information that a potential adopter of the innovation may consider when making a judgment about the likely benefits and risks of implementing the innovation in their own organization.

How to Use the Evidence Rating

The adoption decision is complex and must take into consideration other factors besides the innovator’s results, even when those are compelling. This is because local delivery systems, policies, resources, and culture vary, and may have a profound impact on the results of any change in service delivery practices1. Innovation adoption is not simply a matter of selecting an effective innovation. Any innovation must also be implemented well in order to achieve its maximal impact, taking into account the particular needs, history, and context in the adopting organization. The AHRQ Innovations Exchange includes a variety of resources to guide the adoption decision and implementation process including:

  • Information from the innovator in each innovation profile such as:
    • Context of the Innovation – Describes the innovating organization.
    • Adoption Considerations – Information on starting and sustaining the innovation.
    • Planning and Development Process – Discusses the pre-innovation phase.
    • Results and Evidence Rating – Describes the innovator’s evaluation methodology and findings.
    • Contact the Innovator – All innovators have agreed to be contacted.
    • References – Information on relevant research studies in other locations, and other material.
  • In-depth information on Taking Innovations to Scale on this topical page featuring innovation spread stories, strategies, tools, and voices from the field (http://www.innovations.ahrq.gov/takingInnovationsToScale.aspx).
  • A Decisionmaker’s Guide to Adopting Innovations, a tool for assessing the “fit” of an innovation and understanding whether it will work for you (http://www.innovations.ahrq.gov/guide/guideTOC.aspx).
  • A variety of articles on innovation basics and expert perspectives (http://www.innovations.ahrq.gov/articles.aspx).

These resources can help place the Evidence Rating in context when assessing the ability of an innovation to address local problems.

Components of the Evidence Rating

The Evidence Rating is based on three types of information:

  • Information about the design of the evaluation of the innovation
  • The results of any quantitative evaluation of the innovation
  • Non-experimental, qualitative, or anecdotal information about the effects of the innovation

The Evidence Rating draws on aspects of several established systems that implicitly or explicitly assess the strength of study design and comprehensiveness of reporting including the GRADE guidelines for rating the quality of evidence2, the Consolidated Standards of Reporting Trials (CONSORT) statements which are an evidence-based, minimum set of recommendations for reporting on randomized controlled trials3, and the STROBE initiative for STrengthening the Reporting of OBservational studies in Epidemiology (STROBE)4. These and other criteria for the conduct of health services research studies are integrated in the Patient-Centered Outcomes Research Institute Preliminary Draft Methodology Report5, an evolving document meant to be a catalyst for scientifically rigorous, patient-centered outcomes research that can inform decision-making. Evidence for health care policy innovations must support a connection between the innovation and an impact on either social structures and institutions, the delivery of health care services, or health outcomes6.

Many innovations have not been around long enough to have been rigorously evaluated. In fact, by the time many ideas are rigorously evaluated, they are no longer new or innovative. As a result, the Evidence Rating was designed to cover not only well-established, evidence-based innovations, but also cutting-edge ideas that are not as well studied.

Three Categories for Rating the Strength of Evidence

Drawing on elements of these established systems, the Innovations Exchange uses three categories to provide meaningful distinctions in assessing the strength of the link between the innovation and the observed results:

Strong: The evidence is based on one or more evaluations using experimental designs based on random allocation of individuals or groups of individuals (e.g. medical practices or hospital units) to comparison groups. The results of the evaluation(s) show consistent direct evidence of the effectiveness of the innovation in improving the targeted health care outcomes and/or processes, or structures in the case of health care policy innovations.

Moderate: While there are no randomized, controlled experiments, the evidence includes at least one systematic evaluation of the impact of the innovation using a quasi-experimental design, which could include the non-random assignment of individuals to comparison groups, before-and-after comparisons in one group, and/or comparisons with a historical baseline or control. The results of the evaluation(s) show consistent direct or indirect evidence of the effectiveness of the innovation in improving targeted health care outcomes and/or processes, or structures in the case of health care policy innovations. However, the strength of the evidence is limited by the size, quality, or generalizability of the evaluations, and thus alternative explanations cannot be ruled out.

Suggestive: While there are no systematic experimental or quasi-experimental evaluations, the evidence includes non-experimental or qualitative support for an association between the innovation and targeted health care outcomes or processes, or structures in the case of health care policy innovations. This evidence may include non-comparative case studies, correlation analysis, or anecdotal reports. As with the category above, alternative explanations for the results achieved cannot be ruled out.

If the available qualitative and quantitative information is insufficient to place the innovation in one of the three categories above, the activity fails to meet the minimum inclusion criterion for evidence, and therefore is not eligible for inclusion as an Innovation Profile in the AHRQ Health Care Innovations Exchange. It may, however, qualify for inclusion as an Innovation Attempt.

If you have questions about the Evidence Rating in general, or about the rating on a specific innovation, please contact us at info@innovations.ahrq.gov.


1. Shekelle, S et.al. 2011. Advancing the Science of Patient Safety. Annals of internal Medicine. 154(10):693-6.

2. Guyatt GH, et.al., 2011, “GRADE guidelines: A new series of articles in the Journal of Clinical Epidemiology”, J of Clinical Epidemiology, 64:380-2.

3. CONSORT Web site http://www.consort-statement.org/ External Web Site Policy, accessed July 27, 2012.

4. STROBE Web site http://www.strobe-statement.org/index.php?id=strobe-home External Web Site Policy, accessed July 27, 2012.

5. Pre-public comment draft report of the Patient-Centered Outcomes Research Institute (PCORI) Methodology Committee. Presented to the PCORI Board of Governors May 10, 2012 and published online June 4, 2012.

6. Burris, S., Wagenaar, A.C., Swanson, J., Ibrahim, J.K., Wood, J., and Mello, M.M. (2010). Making the case for laws that improve health: A framework for public health law research. The Milbank Quarterly, 88(2), 169-210.

Last updated: January 23, 2013.