To evaluate the overall success of its education research program, the Institute
annually assesses the quality and relevance of newly funded research projects,
as well as the quality of research publications that result from its funded research
projects. Two indicators address the quality of new projects. First, external
panels of qualified scientists review the quality of new research applications,
and the percentage of newly funded projects that receive an average panel score
of excellent or higher is determined. Second, because much of the Institute's
work focuses on questions of effectiveness, newly funded applications are evaluated
to identify those that address causal questions and then to determine what percentage
of those projects use randomized field trials to answer the causal questions.
To evaluate the relevance of newly funded research projects, a panel of experienced
education practitioners and administrators reviews descriptions of a randomly
selected sample of newly funded projects and rates the degree to which the projects
are relevant to educational practice.
Two indicators address the quality of
new research publications, both print
and web-based, which are the products
of funded research projects. First, an
external panel of eminent scientists
reviews the quality of a randomly
selected sample of new publications,
and the percentage of new publications
that are deemed to be of high quality is
determined. Second, publications that
address causal questions are identified,
and are then reviewed to determine the
percentage that employ randomized
experimental designs. As funded
research projects are completed, the
Institute will subject the final reports to
similar reviews.
To evaluate impact, the Institute surveys K-16 policymakers and administrators
once every three years to determine the percentage who report routinely considering
evidence of effectiveness before adopting educational products and approaches.