A Guarantee That All Published REL Reports Meet
High Quality Standards
Assuring Research Quality
Peer review is used to evaluate reports in virtually all fields of science-from medical studies that examine the effectiveness of drugs to articles about the discovery of new planets or ancient species. Scientists use the peer review process to ensure that the information they report to the public is accurate, relevant, original, and supported by appropriate evidence. Simply put, peer review is a method by which scientists who are experts in a particular field examine another scientist's work to verify that it makes a valid contribution to the evidence base. With that assurance, a scientist can report his or her work to the public, and the public can trust the work.
Unfortunately, much education research is routinely published without undergoing rigorous peer review that applies sound scientific standards. Too often, these education studies may not be vetted for the appropriateness of the data, sampling procedures, the process used for data collection, analytic methods, and/or how the interpretations are drawn. This means that these reports may reach conclusions that are not supported by reliable evidence, oftentimes suggesting policy prescriptions that go far beyond the actual evidence. As a result, policymakers and practitioners in their role as major decisionmakers are faced with confusing claims made in publications that are not peer reviewed. Another complication is that for the non-researcher, it is often difficult to sort valid factual claims supported by evidence from unwarranted conclusions based on opinion or an unscientific approach. The bottom line is this: Peer review matters—it takes the guesswork out of interpreting research findings.
All Regional Educational Laboratory (REL) scientifically based reports are required to undergo rigorous external peer review. This ensures that all reports meet the Institute of Education Sciences (IES) standards for scientifically valid research before being published as online reports on the REL website at http://ies.ed.gov/ncee/edlabs. In this way, policymakers and practitioners, the primary users of REL reports, can be assured that these reports have met high standards for scientific quality, and that the information in the reports is valid and reliable, and therefore can be trusted.
The peer review process used to examine REL reports employs rigorous scientific standards to ensure that the information in the reports is valid and reliable. The review process begins when a REL completes a research study and prepares a report for online publication. Each report is reviewed thoroughly by external content and methodology experts, guided by the following questions that are indicative of good science and are consistent with IES standards for research and evaluation:
Relevance and significance:
- Does the study address a well-defined question that can be investigated empirically?
- Will the findings be relevant and accessible to the target audience?
- Does the study use and build on the existing research base and offer new knowledge?
Data and methods:
- Are the data sources clearly identified and appropriate for the research questions?
- Are the methods of analysis clearly identified, appropriate for the research questions, and implemented correctly?
- Are the strengths and limitations of the data and methods clearly stated?
Presentation of findings:
- Are the findings clearly aligned with the research questions, and are all findings clearly supported by data?
- Are the findings presented accurately and objectively without introducing a point of view and without telling a reader what to think or do about the findings?
- Are limitations to the ability to draw conclusions in the report clearly noted?
If the answers to the above questions are “yes,” then the report has sufficiently met the IES standards for soundness and accuracy and this report will have passed peer review and can be published online and distributed.
