To evaluate the overall success of its education
research program, the Institute annually
assesses the quality and relevance of
newly funded research projects, as well
as the quality of research publications
that result from its funded research
projects. Two indicators address the
quality of new projects. First, external
panels of qualified scientists review the
quality of new research applications,
and the percentage of newly funded
projects that receive an average panel
score of excellent or higher is
determined. Second, because much of
the Institutes work focuses on questions
of effectiveness, newly funded
applications are evaluated to identify
those that address causal questions and
then to determine what percentage of
those projects use randomized field
trials to answer the causal questions. To
evaluate the relevance of newly funded
research projects, a panel of experienced
education practitioners and
administrators reviews descriptions of a
randomly selected sample of newly
funded projects and rates the degree to
which the projects are relevant to
educational practice.
Two indicators address the quality of
new research publications, both print
and web-based, which are the products
of funded research projects. First, an
external panel of eminent scientists
reviews the quality of a randomly
selected sample of new publications,
and the percentage of new publications
that are deemed to be of high quality is
determined. Second, publications that
address causal questions are identified,
and are then reviewed to determine the
percentage that employ randomized
experimental designs. As funded
research projects are completed, the
Institute will subject the final reports to
similar reviews.
To evaluate impact, the Institute
surveys K16 policymakers and
administrators once every three years to
determine the percentage who report
routinely considering evidence of
effectiveness before adopting
educational products and approaches.