Home > About NREPP > FAQs
Frequently Asked Questions (FAQs)
General Information
- What is NREPP?
- What does Quality of Research mean?
- What does Readiness for Dissemination mean?
- What is comparative effectiveness research?
- Are environmental interventions/community-level interventions included in NREPP?
- What is the recommended format for referencing an NREPP intervention summary?
- Who rates the interventions reviewed?
- How do I become a reviewer for NREPP?
- How are interventions reviewed?
- How do I contact NREPP with a question or comment?
Using the NREPP Site
- How do I decide which intervention to use?
- How do I find an intervention?
- How do I get more information on an intervention I found on the site?
- Why are only select intervention summaries translated into Spanish?
- Why do some intervention summaries contain two review dates?
Information for Developers Interested in Submitting an Intervention for Review
- How do I find out if my intervention qualifies for submission?
- Is there any cost to submit an intervention to NREPP?
- Where can I find information on preparing a submission to NREPP?
- Will every intervention that is submitted be reviewed?
- Will there be another application cycle?
- Can I submit multiple interventions for review?
- Can interventions developed outside the United States be reviewed by NREPP?
- Why aren't prevention and treatment interventions considered under separate criteria?
Information for Developers Whose Intervention Has Been Accepted for Review and/or Posted
- My intervention has been accepted for review. How can I refer to its status, as well as NREPP, in my materials?
- Once my intervention has been reviewed by NREPP, am I required to post the results on the NREPP Web site?
- After my intervention has been reviewed, will I be able to approve the intervention summary before it is posted on the Web site?
- My intervention has been reviewed and posted on the site. How can I refer to NREPP in my materials?
- Does my intervention qualify for a second review?
General Information
What is NREPP?
The National Registry of Evidence-based Programs and Practices (NREPP) is a searchable online registry of interventions that support mental health promotion, substance abuse prevention, and mental health and substance abuse treatment. Through NREPP, users can identify and learn more about interventions that may meet their needs. All interventions that are included in NREPP have been voluntarily submitted, have met a set of minimum requirements, and have been assessed by independent reviewers.
What does Quality of Research mean?
Quality of Research (QOR) is one of two main categories of NREPP ratings. The QOR category contains six criteria (reliability of measures; validity of measures; intervention fidelity; missing data and attrition; potential confounding variables; and appropriateness of analysis), which are used in rating the strength of evidence supporting the results or outcomes of the intervention. Because interventions may target multiple outcomes, and the evidence supporting the different outcomes may vary, each outcome is rated separately. Each intervention summary contains QOR ratings as well as a description of the study strengths and weaknesses that may have contributed to those ratings. For more information on the QOR ratings, see the Quality of Research page.
What does Readiness for Dissemination mean?
Readiness for Dissemination (RFD) is one of two main categories of NREPP ratings. The RFD category contains three criteria (availability of implementation materials; availability of training and support resources; and availability of quality assurance procedures), which are used in rating the quality and availability of an intervention's training and implementation materials, that is, how easily the intervention can be implemented with fidelity using the materials and services that are currently available to the public. Each intervention summary contains RFD ratings as well as a description of the dissemination strengths and weaknesses that may have contributed to those ratings. For more information on the RFD ratings, see the Readiness for Dissemination page.
What is comparative effectiveness research?
The Federal Coordinating Council for Comparative Effectiveness Research defines comparative effectiveness research, or CER, as follows:
Comparative effectiveness research is the conduct and synthesis of research comparing the benefits and harms of different interventions and strategies to prevent, diagnose, treat, and monitor health conditions in "real world" settings. The purpose of this research is to improve health outcomes by developing and disseminating evidence-based information to patients, clinicians, and other decision-makers, responding to their expressed needs, about which interventions are most effective for which patients under specific circumstances.
To provide this information, comparative effectiveness research must assess a comprehensive array of health-related outcomes for diverse patient populations and subgroups. Defined interventions compared may include medications, procedures, medical and assistive devices and technologies, diagnostic testing, behavioral change, and delivery system strategies. This research necessitates the development, expansion, and use of a variety of data sources and methods to assess comparative effectiveness and actively disseminate the results.
This definition was published in the Federal Coordinating Council's June 30, 2009, Report to the President and the Congress (PDF, 3.6MB).
Each intervention summary included in NREPP indicates whether or not the intervention was evaluated in comparative effectiveness research studies (i.e., whether or not at least one of the studies included in the NREPP review met the definition of comparative effectiveness research given above).
Are environmental interventions/community-level interventions included in NREPP?
Yes.
What is the recommended format for referencing an NREPP intervention summary?
NREPP recommends the following format for references:
Substance Abuse and Mental Health Services Administration. ([year, month of review]). [Program name]. Retrieved from the National Registry of Evidence-based Programs and Practices Web site, [URL of summary]
For example:
Substance Abuse and Mental Health Services Administration. (2007, December). Motivational Interviewing. Retrieved from the National Registry of Evidence-based Programs and Practices Web site, http://www.nrepp.samhsa.gov/ViewIntervention.aspx?id=130
Who rates the interventions reviewed?
NREPP contract staff, not SAMHSA, assign two sets of two reviewers to independently review each intervention: one set for the Quality of Research review and one set for the Readiness for Dissemination review. The ratings provided in the intervention summary are the average reviewer ratings. For more information, see the Selection and Training of Reviewers page.
How do I become a reviewer for NREPP?
Potential reviewers are recruited, selected, and approved by SAMHSA on the basis of their experience and areas of expertise. All potential reviewers must meet minimum requirements, which differ for Quality of Research and Readiness for Dissemination; each category requires different kinds of expertise. Potential reviewers also must complete training on the procedures and criteria they will use to rate interventions. For more information, see the Selection and Training of Reviewers page.
How are interventions reviewed?
Each intervention is reviewed by two sets of reviewers who have been selected to participate on the basis of their qualifications. One set reviews Quality of Research, and one set reviews Readiness for Dissemination. NREPP staff send a review packet to each reviewer, who independently conducts a review using either the Quality of Research or Readiness for Dissemination review criteria. The reviewers submit their ratings and comments to NREPP staff, who include this information in the intervention summary that is posted on the NREPP Web site.
How do I contact NREPP with a question or comment?
NREPP staff are available to answer any questions by email, phone, or online contact form. See the Contact Us page for details or to send a comment or question through the contact form.
Using the NREPP Site
How do I decide which intervention to use?
First, identify your community's needs so you can determine which type of intervention will be the most appropriate. Next, assess your organization's capacity (e.g., financial resources, organizational commitment, community buy-in) to implement an intervention with fidelity. Once you have this information, search NREPP or another registry to find an intervention that matches your community's needs and that can be implemented given your organization's capacity. Information on selecting and implementing an intervention can be found in NREPP's Learning Center — A Road Map for Implementing Evidence-Based Programs.
How do I find an intervention?
If you know the name of the intervention, you can either type it in the keyword Search box or select it from the View All Interventions list. From these pages on the Web site, you can access summaries describing interventions that have been reviewed. You can also view a list of interventions that have been Accepted for Review. However, the interventions listed in Accepted for Review have not been summarized on the Web site yet because their review process is not complete. If you sign up for e-updates, you will receive announcements with links to interventions recently added to the site.
If you do not know the name of the intervention, you can find relevant interventions by typing keywords in the Search box or by selecting criteria through the Advanced Search to narrow the list of relevant interventions by area of interest, outcome category, geographic location, age, race/ethnicity, setting, gender, and/or keyword.
If you have further questions, please call 1-866-436-7377, email us at nrepp@samhsa.hhs.gov, or use the online contact form. See the Contact Us page for details or to send a comment or question through the contact form.
How do I get more information on an intervention I found on the site?
To obtain more information about an intervention listed on this site, please contact the developer directly, using the contact information provided in the intervention summary. To assist you in gathering additional information, NREPP has compiled a list of Questions To Ask as You Explore the Possible Use of an Intervention (PDF, 54KB).
Why are only select intervention summaries translated into Spanish?
Spanish translations are provided for summaries of interventions for which the materials needed to deliver the intervention (e.g., treatment manuals, curricula, training guides, information for administrators) are available for dissemination in Spanish. To see a list of the summaries that have been translated into Spanish, go to the View All Interventions page, click Language Translations in the list of search criteria on the left, select Spanish, and click Search. You also can use the Language Translations criterion to refine a search after using the Basic Search or Advanced Search.
Why do some intervention summaries contain two review dates?
In 2011, NREPP began the process of reviewing new research and/or dissemination materials for select interventions that had already been reviewed by NREPP at least 5 years ago and summarized on the Web site. Summaries for these interventions contain information from the original Quality of Research (QOR) and Readiness for Dissemination (RFD) review as well as information from a second QOR and/or RFD review, depending on the research conducted or materials developed since the original review. The summary uses review dates to present the information stemming from each review.
Information for Developers Interested in Submitting an Intervention for Review
How do I find out if my intervention qualifies for submission?
Minimum requirements and other details regarding submission are provided in a Federal Register notice that is posted the summer before each open submission period. The open submission period is generally held annually from November to February, and submission requirements sometimes differ for each open submission period. To receive announcements about future open submission periods, please sign up for our Submissions mailing list. The current Federal Register notice, as well as those for past submission periods, is available on the NREPP News page.
Is there any cost to submit an intervention to NREPP?
There is no cost to submit an intervention to NREPP. However, applicants must be prepared to send all relevant intervention materials and three copies of all dissemination materials to be included in the review at their own expense.
Where can I find information on preparing a submission to NREPP?
This information can be found in NREPP's Learning Center — Preparing an Intervention for NREPP Submission and Potential Review. This course was created for developers of mental health or substance abuse programs, including principal investigators of relevant research studies, who are working toward submission to NREPP.
Will every intervention that is submitted be reviewed?
No. Each submission undergoes a preliminary assessment to determine whether it has met the minimum requirements outlined in the Federal Register notice published the summer before each submission period. Submissions that do meet the minimum requirements may receive an invitation for review from SAMHSA, depending on the availability of current funding. In addition, SAMHSA may choose to give special consideration to interventions that meet one or more conditions, as outlined in the Federal Register notice.
Will there be another application cycle?
Yes. Open submission periods are generally held annually from November to February. To receive announcements about future open submission periods, please sign up for our Submissions mailing list.
Can I submit multiple interventions for review?
Yes. However, each intervention must be submitted separately.
Can interventions developed outside the United States be reviewed by NREPP?
Yes, if they meet the minimum requirements and have implementation materials that can be disseminated in the United States.
Why aren't prevention and treatment interventions considered under separate criteria?
The developers and Federal sponsors of NREPP believe that the scientific standards for the evidence-based practice assessments contained within NREPP apply equally to the prevention and treatment components of behavioral health care.
Information for Developers Whose Intervention Has Been Accepted for Review and/or Posted
My intervention has been accepted for review. How can I refer to its status, as well as NREPP, in my materials?
Intervention materials, including Web sites, can state that the intervention has been accepted for review by NREPP. NREPP maintains a list of interventions that have been Accepted for Review, and an intervention's inclusion on this list can be noted. However, acceptance for review does not constitute an endorsement, promotion, or approval of the intervention by NREPP or SAMHSA.
Once my intervention has been reviewed by NREPP, am I required to post the results on the NREPP Web site?
Although NREPP is a voluntary registry, each NREPP review represents a considerable investment of time and public funds. Any individual or organization that applies for an NREPP review is expected to authorize publication of the results in the form of an intervention summary. In rare cases, a program developer may choose to decline the posting of a summary. To provide full disclosure to all users of this site, NREPP will publish the names of these reviewed, but unlisted, interventions on the NREPP Web site, and note that the developer declined to post the intervention summary.
After my intervention has been reviewed, will I be able to approve the intervention summary before it is posted on the Web site?
NREPP works closely with the program developer throughout the review and posting process. After the intervention summary has been completed, the developer is given the opportunity to review the summary and provide comments. The intervention summary is posted on the NREPP Web site after it has been approved by both the developer and SAMHSA.
My intervention has been reviewed and posted on the site. How can I refer to NREPP in my materials?
Intervention materials, including Web sites, can state that the intervention has been reviewed and posted on the NREPP Web site. Shortly after the intervention summary has been posted on the NREPP Web site, the program developer will receive an email with image files of the NREPP logo. The logo also can be used in the intervention materials. However, the posting of an intervention summary on the NREPP Web site does not constitute an endorsement, promotion, or approval of the intervention by NREPP or SAMHSA.
Does my intervention qualify for a second review?
A second review may be conducted for an intervention summarized on the NREPP Web site if the original review occurred at least 5 years ago. The second review may be limited to Quality of Research (QOR) or Readiness for Dissemination (RFD), or it may include both, depending on the following requirements:
- To qualify for a second QOR review, developers need to have (1) a new study (or studies) of the intervention that evaluated the same intervention with the same target population or a slightly modified intervention with the same or different target population or (2) updates to the study (or studies) already reviewed by NREPP, with results from secondary analysis of the original dataset and/or analysis of longitudinal data. In either case, the studies must meet NREPP's minimum requirements for review: the studies have produced statistically significant findings related to substance abuse or mental health behavioral outcome(s) at the p ≤ .05 level, have used a quasi-experimental or experimental design, and have been described in a published form (journal article and/or comprehensive evaluation report).
- To qualify for a second RFD review, the intervention's dissemination materials must have changed markedly--or new materials must have been developed--since the original review was conducted.
NREPP has already extended invitations for a second review to several cohorts of interventions, and new cohorts will be invited depending on available funding.