Control group refers to the population of subjects (e.g., students) who do not receive or participate in the treatment being studied (e.g., an online class) but whose performance or other outcomes are being compared to those of students who will receive or participate in the treatment.

Formative evaluations generate information aimed at helping program stakeholders better understand a program or its participants, often by examining the delivery or implementation of a program. Findings from these evaluations are generally used to make program improvements or influence future decisions.

Hierarchical linear modeling, also called multi-level modeling, is used for the same purpose as regression analysis—to understand what factors are the best predictors of an outcome, such as a test score. But researchers use hierarchical linear modeling to take into account factors at different levels of an education system, such as the characteristics of the class or school in which students are situated. Hierarchical linear modeling helps statisticians address the fact that students are generally not grouped randomly within classes or schools and that classroom- and school-level factors are often related to student outcomes.

Quasi-experiments are experimental studies in which subjects are not assigned at random to treatment and control groups, as with RCTs (see below). Quasi-experimental studies may be used, for example, when controlled trials are infeasible (e.g., when evaluators cannot assign students randomly to participate in a treatment) or are considered too expensive. According to the U.S. Department of Education's What Works Clearinghouse, strong evidence of a program's effectiveness can be obtained from a quasi-experimental study based on one of three designs: one that "equates" treatment and control groups, either by matching groups based on key characteristics of participants or by using statistical methods to account for differences between groups; one that employs a discontinuity design in which participants are assigned to the treatment and control groups based on a cutoff score on a pretreatment measure that typically assesses need or merit; or one that uses a "single-case design" involving repeated measurement of a single subject (e.g., a student or a classroom) in different conditions or phases over time.¹⁹

Randomized controlled trials (RCTs) are experimental studies that randomly assign some study participants to receive a treatment (e.g., participation in a class or program) and others to not receive the treatment. This latter is known as the control group. In an RCT, evaluators compare the outcomes (e.g., test scores) of the treatment group with those of the control group; these results are used to determine the effectiveness of the treatment. RCTs can provide strong evidence of a program's effectiveness.

Regression analysis is a statistical technique used in research to determine the factors or characteristics (e.g., gender, family income level, whether a student participated in a particular program) that are the best predictors of an outcome. Regression analyses help statisticians isolate the relationships between individual factors and an outcome and, thus, are useful when trying to understand the relationship of a program to student achievement.

Summative evaluations examine the effects or outcomes of a program. Findings from these evaluations are generally used to assess how well a program is meeting its stated goals.

Treatment group refers to the population of subjects (in this case, students) who receive or participate in the treatment being studied (e.g., an online class).