The What Works Clearinghouse (WWC) seeks to provide users with the information they need to be effective consumers of education research. Please review the glossary of terms to better understand the information provided in the WWC's resources.
Click on a Letter to Browse the Glossary:
A | B | C | D | E | F | G | H | I | J | K | L | M | N | O | P | Q | R | S | T | U | V | W | X | Y | Z
A
Attrition – The proportion of original sample members that are not included in the analysis. High levels of attrition can call into question the validity of results. See also Differential Attrition and Overall Attrition.
B
C
D
Differential Attrition – The difference between the rate of attrition in the treatment group and the rate of attrition in the control group (for a randomized controlled trial) or the comparison group (for a quasi-experimental design). High levels of differential attrition suggest that the treatment group (as analyzed) may differ from the control/comparison group (as analyzed) in ways other than the treatment. See also Attrition and Overall Attrition.
Does Not Meet Evidence Standards – This is the rating given to studies with a research design that is not rigorous enough to conclude with confidence that the intervention is causing observed changes in outcomes.
Doing What Works (DWW) – DWW is a website dedicated to helping educators identify and make use of effective teaching practices. DWW relies primarily on IES to evaluate and recommend practices that are supported by rigorous research. Then, DWW provides examples of possible ways educators might apply those research findings. Whenever companion information is available on the DWW website for a particular WWC report or topic, users will see the 
icon. Clicking on the icon will bring users to the associated information on DWW.
Domain – See “improvement outcome.”
E
Effect Size – The effect size is a standardized measure of the effect of an intervention (treatment) on student outcomes. The effect size represents the change (measured in standard deviations) in an average student's outcome that can be expected if that student is given the treatment. Because effect sizes are standardized, they can be compared across studies. For more information, visit the Technical Details of WWC-Conducted Computations page and the Effect Size Substantive Interpretation Guidelines: Issues in the Interpretation of Effect Sizes page. See also Improvement Index.
Evidence Rating – For each intervention, the evidence rating summarizes the results of the studies that met WWC evidence standards. The evidence rating shows whether the studies indicate that the intervention has a positive effect, a negative effect, or no effect. The rating also reflects the degree of confidence in those estimated effects. The evidence rating takes into account four factors: the quality of the research design, the statistical significance of the findings, the size of the difference between participants in the intervention and the comparison conditions, and the consistency in findings across studies. For more information, visit the Intervention Rating Scheme page.
Evidence Standards – Evidence Standards are used to evaluate the strength of the evidence of the effectiveness of educational interventions. The WWC has established rigorous Evidence Standards for the review of causal research. Studies are classified as “Meets Evidence Standards,” “Meets Evidence Standards with Reservations,” and “Does Not Meet Evidence Standards.” For more information, visit the Evidence Standards page.
Extent of Evidence – The Extent of Evidence categorization was developed to inform readers how much evidence was used to determine the intervention evidence rating, focusing on the number and sizes of studies. This scheme has two categories: small and medium to large. Domains with a small extent of evidence include only one study, or one school, or findings based on a total sample size of less than 350 students and, assuming 25 students in a class, a total of less than 14 classrooms across studies. Domains with a medium to large extent of evidence include more than one study, more than one school, and findings based on a total sample size of 350 students, or assuming 25 students in a class, a total of at least 14 classrooms across studies. For more information, visit the Extent of Evidence page.
External Validity – A study of an intervention is considered to be externally valid if the impacts estimated in the study would be expected if the intervention were administered on other subjects and in other settings. Moreover, a study is externally valid if the estimated impacts are not sensitive to the operational definitions of other variables used in estimating the impacts.
F
G
H
I
Improvement Index – The improvement index is an alternative presentation of the effect size of an intervention (treatment). Where the effect size represents the change in an average student's outcome that can be expected if the student is given the treatment, the improvement index reflects the change in an average student's percentile rank that can be expected if the student is given the treatment. Unlike the evidence rating, the improvement index is based entirely on the size of the effect, regardless of the statistical significance of the effect, the study design, or the analyses. The improvement index can take on values between –50 and +50, with positive numbers denoting results favorable to the intervention group. For more information, visit the Technical Details of WWC-Conducted Computations page.
Improvement Outcome – Commonly referred to as “domain,” an improvement outcome is the result, or a set of closely related results, through which intervention studies claim effectiveness (for example, Fluency in the Beginning Reading topic area). The WWC assesses the rigor of the research evidence on the effectiveness of interventions on outcomes through intervention reports.
Institute of Education Sciences (IES) – Established by the Education Sciences Reform Act of 2002, IES is the research arm of the U. S. Department of Education. Its mission is to expand knowledge and provide information on the condition of education, practices that improve academic achievement, and the effectiveness of Federal and other education programs.
Internal Validity - A study is considered to be internally valid if there is confidence that the differences in outcomes are explained by the factors examined in the study, and are the result of the independent variable, not of some uncontrolled extraneous variable or variables.
Intervention – An educational program (such as whole school reform), product (such as a textbook or curriculum), practice (such as mixed-age grouping), or policy (such as class size reduction) aimed at improving student outcomes. See also Treatment.
Intervention Report – For each intervention in a topic area, the WWC assesses the rigor of research evidence on the effectiveness of the intervention and releases an intervention report. Intervention reports assess all studies of a specific intervention within a topic area, rating each of them based on the WWC evidence standards. Each report offers an overview of the intervention, summarizes all relevant research, and provides an overall rating of effectiveness and improvement index for outcomes associated with the topic. Intervention reports go through a rigorous peer review process.
J
K
L
M
Mathematica Policy Research, Inc. – Contracted by IES to administer the WWC, Mathematica is a nationally recognized leader in education research and rigorous reviews of scientific evidence.
Meets Evidence Standards – Studies assigned this rating are generally well-conducted randomized controlled trials (RCTs) that do not have problems with randomization or attrition, or regression discontinuity designs that do not have problems with attrition. “Meets Evidence Standards” is the highest rating for a study reviewed by the WWC.
Meets Evidence Standards with Reservations – Generally, studies assigned this rating are strong quasi-experimental studies that have comparison groups with demonstrated equivalence and meet other WWC Evidence Standards. Studies assigned this rating may also include randomized trials with problems in randomization or attrition, as well as regression discontinuity designs with attrition problems.
Mismatch – Occurs when the unit of assignment in a study does not match the unit of analysis. For instance, a study may have assigned entire classrooms (the unit of assignment) to the intervention and control conditions, but then analyzed data at the individual student level. If not accounted for when estimating impacts, this type of mismatch can lead to inflated estimates of statistical significance.
N
O
Outcome – See Improvement Outcome.
Overall Attrition – The proportion of original sample members that are not included in the analysis. Overall attrition refers to the total attrition observed in the treatment and control groups combined. High levels of overall attrition can call into question the validity of results. See also Attrition and Differential Attrition.
P
Practice Guide – Practice Guides contain practical recommendations for educators to address challenges in their classrooms and schools. Written by a panel of nationally recognized experts, practice guides take into consideration research in a particular area, as well as the expert opinions and experiences of the panel members. Along with the recommendations, practice guides provide educators with information on how to carry out the recommendations, clearly state the level of research supporting each recommendation, and address possible implementation roadblocks.
Principal Investigator (PI) – Experts in their field, principal investigators are responsible for leading the development of intervention and topic reports for a particular topic area. Click here for a list of current PIs.
Q
Quasi-Experimental Design (QED) – A research design in which subjects are assigned to “treatment” (that is, they receive the intervention being studied) and “comparison” groups through a process that is not random. For a QED to be rigorous, the treatment and comparison groups must be similar before the treatment is applied. Strong QED studies that meet other WWC Evidence Standards are classified as “Meets Evidence Standards with Reservations.”
Quick Review – WWC quick reviews examine recently released research papers and reports. They are designed to provide education practitioners and policymakers with timely and objective assessments of the quality of the research evidence. Quick reviews focus on studies of the effectiveness of education or school-based programs serving students in the pre-kindergarten through post-secondary grades. To be eligible for the quick review process, a study must: (1) examine the effectiveness of an education or school-based program in improving student outcomes, and (2) be recently released to the public and reported in a major national news source. The ratings and research summaries included in WWC quick reviews are based only on publicly available information.
R
Randomized Control Trial (RCT) – An experiment in which investigators randomly assign eligible subjects into groups to receive or not receive one or more interventions that are being compared. Strong RCTs (that is, randomization is not compromised and attrition is not substantial) are classified as “Meets Evidence Standards.”
Rating of Effectiveness – The WWC rates the effects of an intervention in a given improvement outcome domain as: positive, potentially positive, mixed, no discernible effects, potentially negative, or negative. The rating of effectiveness takes into account four factors: the quality of the research design, the statistical significance of the findings, the size of the difference between participants in the intervention and the comparison conditions, and the consistency in findings across studies. For more information, visit the Intervention Rating Scheme page.
Registry of Evaluation Researchers – An online database of researchers who conduct research on the effects of educational interventions. This resource is designed to help schools, school districts, and educational program developers identify potential researchers (individuals and organization) to conduct studies on educational outcomes.
Review Protocol – The review protocol is a formal document developed to determine the scope of a topic review area. Protocols include key definitions, express the types of research studies and interventions reviewed, set topic parameters (including the types of interventions and outcomes to be examined), and explain methodology. For each topic, the review protocol is authored by the principal investigator prior to assessing whether studies in the topic area meet WWC standards.
S
Statistical Significance – Statistical significance refers to the probability that a result occurred by chance alone. A result is considered “statistically significant” if the probability it occurred by chance alone is 5 percent or less.
Study – A research-based evaluation, focused on the effectiveness of an intervention.
T
Topic – An area of study by the WWC. Topics are defined by intended outcome (such as improving reading skills), intended population (such as elementary school students), and types of interventions that may improve outcomes for students.
Topic Report – Topic reports compile the information from intervention reports in a topic area and provide an overview of ratings of effectiveness, improvement indexes, and effect size by outcome areas. Topic reports enable WWC users to easily compare the ratings and indexes of numerous programs in one area. These reports also provide a list of programs and products reviewed with no studies meeting the WWC evidence standards.
Treatment –The program, product, practice, or policy aimed at improving student outcomes. RCTs and QEDs compare students subject to the intervention – referred to as the treatment group – with students not subject to the intervention – referred to as the control group (in RCTs) or the comparison group (in QEDs). See also Intervention.
U
V
W
What Works Clearinghouse (WWC) – Established in 2002 by the U.S. Department of Education's Institute of Education Sciences to provide educators, policymakers, researchers, and the public with a central and trusted source of scientific evidence of what works in education.
