This guide is designed as a resource for leaders and evaluators of K-12 online learning programs. In this guide, the term "online learning" is used to refer to a range of education programs and resources in the K-12 arena, including distance learning courses offered by universities, private providers, or teachers at other schools; stand-alone "virtual schools" that provide students with a full array of online courses and services; and educational Web sites that offer teachers, parents, and students a range of resources.¹ The guide features seven evaluations that represent variety in both the type of program or resource being evaluated, and in the type of evaluation. These evaluations were selected because they offer useful lessons to others who are planning to evaluate an online learning program or resource.

Of course, evaluating online learning is not altogether different from assessing any other type of education program, and, to some degree, evaluators may face the same kind of design and analysis issues in both instances. Still, online program evaluators may encounter some unanticipated challenges in the virtual arena owing, for example, to the distance between program sites and students, participants' unfamiliarity with the technology being used, and a lack of relevant evaluation tools. This guide examines a range of challenges that online program evaluators are likely to meet, some that are unique to online settings and others that are more general. It also describes how online environments can sometimes offer advantages to evaluators by presenting opportunities for streamlined data collection and analysis, for example.

The guide specifically focuses on challenges and response strategies. All of the evaluations described here illustrate strong assessment practices and robust findings, and they are models for demonstrating how program leaders and evaluators can handle the challenges of evaluating online learning.