Several recurring themes can be found in the seven featured evaluations, as well as in the research and comments from experts in the field of online learning. These sources point to a handful of overarching recommendations:

• Begin with a clear vision for the evaluation. Determine what you want the evaluation to accomplish and what questions you hope to answer. Program leaders and evaluators may want to consider the questions listed in the box on page 51 as they get started.

• Determine the most appropriate evaluation methods for meeting your goals. Consider the different types of evaluations discussed in the guide (see Part I for examples of formative and summative; internal and external; experimental, quasi-experimental, and other types of evaluations) and the costs and benefits of each. What type of evaluation is appropriate given your stated purpose? What research methods will best capture program effects?

• Budget to meet evaluation needs. Limited budgets are a common barrier to evaluators. When designing evaluations, consider whether there are available funds to cover all planned data collection and analysis activities, plus the costs of any needed background research, internal communications and reporting, and incentives for study participants. If available funds are not sufficient, scale back the evaluation and focus on the highest-priority activities.

• Develop a program culture that supports evaluation. Discuss evaluation with staff members, clearly explaining its value and their roles in collecting and analyzing data. Incorporate data collection and analysis activities into staff members' everyday responsibilities instead of treating these tasks as one-time efforts or burdens. Make external evaluators less threatening to program staff members by creating opportunities for dialogue between the evaluator and program leaders and staff. Incorporate evaluation data into the process of development of annual program goals and strategic planning activities.

• Communicate early and often with anyone who will be affected by the evaluation.

• Dedicate adequate time and money to communicating with internal and external stakeholders at all phases of the evaluation. However, be sensitive to program concerns about keeping evaluation results confidential and avoiding media scrutiny. Always keep those managing the program "in the loop." Communicating through the program director is often the best course of action.