Build in adequate time to fully communicate the purpose and design of the evaluation to everyone involved. Inform program staff who will play a role in the evaluation, as well as anyone who will help you gather evaluation evidence. Explain how study participants will benefit from the evaluation.

Be prepared to repurpose the methods you use to conduct an evaluation, without losing sight of the evaluation's original purpose. Collecting multiple sources of evidence related to the same evaluation question can help to ensure that the evaluator can answer the question if a data source becomes unavailable or a research method proves infeasible.

Seek out valid and reliable instruments for gathering data. Existing data-gathering instruments that have been tested and refined can offer higher-quality data than locally developed instruments.

Consider administering surveys online, to boost response rates and easily compile results.

Consider if there are innovative ways to collect data electronically about how online resources are used (e.g., tracking different pathways users take as they navigate through a particular online tool or Web site, or how much time is spent on different portions of a Web site or online course).

If response rates are too low, consider redesigning or refocusing the evaluation. Find other indicators that get at the same phenomenon. Find other data sources that put more control into the hands of evaluators (e.g., observations, focus groups).

If collecting and aggregating data across multiple sources, define indicators clearly and make sure that data are collected in compatible formats. Define exactly what is to be measured, and how, and distribute these instructions to all parties who are collecting data.

Research relevant data privacy regulations well in advance.

Determine the process and build in adequate time for requesting student data from states, districts, or schools. Determine early on whether data permissions will be needed and from whom, and how likely it is that sensitive data will actually be available. Have a "Plan B" if it is not.

If the program does not have good access to the complete school record of enrolled students, encourage it to support school improvement and program evaluator needs by collecting NCLB subgroup and other key student record data as part of its regular program registration process.

Consider creative and flexible arrangements for protecting student privacy (e.g., sign confidentiality agreements, physically travel to a specific site to analyze data, employ a disinterested third party to receive data sets and strip out any identifying information).

Incorporate precautions into the study protocol to protect students' privacy. When possible, avoid contact with students' names and refer to students through student identification numbers.