Under a competitive preference priority, the U.S. Department of Education gives competitive preference to an application by either (1) awarding additional points, depending on how well or the extent to which the application meets the competitive preference priority or (2) selecting an application that meets the competitive priority over an application of comparable merit that does not meet the priority. The competitive priority supports applications based on rigorous scientifically based research methods to assess the effectiveness of a particular intervention. This priority will allow program participants and the Department to determine whether the project produces meaningful effects on student achievement or teacher performance.

The Department awards up to an additional 25 points to an application, depending on the extent to which the application meets this priority. In awarding additional points to applications that address this competitive preference priority, the Department considers only those applications that receive top-ranked scores on the basis of the Selection Criteria in Section V, "Applicant Review Information."

Section V (h), which focuses on the selection criteria for this competition, addresses the "Quality of the project evaluation." All applications will be reviewed on the basis of the criteria set forth in this section. The maximum point assignment for addressing the selection criterion evaluation in this section is 20 points.

Applicants may choose to address the competitive priority in their evaluation plan to receive up to 25 additional points beyond the 20 points available in the selection criterion for evaluation.

All applicants will provide one evaluation plan. Applications that address the competitive preference priority will be reviewed using a two-stage process. In the first stage review, the evaluation plan will be rated against the criteria set forth in Section V (h) without taking into account the competitive preference priority. The highest rated applicants will be moved to the second stage review. In the second stage review, the evaluation plan will be rated in its entirety for those applicants that address the competitive priority.

Applicants will be rated on the extent to which the methods of the proposed evaluation include the use of objective performance measures that are clearly related to the intended outcomes of the project and produce quantitative and qualitative data to the extent possible. A strong evaluation plan should be included in the application narrative along with benchmarks to monitor progress toward specific project objectives and outcome measures to assess the impact on teaching and learning or other important outcomes for project participants. The plan should identify the individual and/or organization that to serve as evaluator for the project and describe the qualifications of the evaluator(s).

The evaluation design should include the following components: (1) what types of data will be collected; (2) when various types of data will be collected; (3) what methods will be used; (4) what instruments will be developed and when; (5) how the data will be analyzed; (6) when reports of results and outcomes will be available; and (7) how the applicant will use the information collected through the evaluation to monitor progress of the funded project and to provide accountability information both about success at the initial site and effective strategies for replication in other settings.

In determining the quality of the evaluation method, we will consider the extent to which the applicant presents a feasible, credible plan that includes the following: (1) the type of design to be used (that is, random assignment or matched comparison). If matched comparison, include in the plan a discussion of why random assignment is not feasible; (2) outcomes to be measured; (3) a discussion of how the applicant plans to assign students, teachers, classrooms, or schools to the project and control group or match them for comparison with other students, teachers, classrooms, or schools; and (4) a proposed evaluator, preferably independent, with the necessary background and technical expertise to carry out the proposed evaluation. An independent evaluator does not have any authority over the project and is not involved in its implementation.

In general, depending on the implemented program or project under a competitive preference priority, random assignment evaluation methods will receive more points than matched comparison evaluation methods. An applicant that addresses this competitive priority, and finishes high enough in the review of the selection criteria in Section V. to warrant review under the competitive priority, could receive up to 25 points.

The notice of final priority for "Scientifically Based Evaluation Methods" was published in the Federal Register on January 25, 2005 (70 FR 3586-3589) and provides the definitions of relevant evaluation strategies. A copy of the notice is included in the application package and on the Department's website at http://www.ed.gov/legislation/FedRegister/finrule/2005-1/012505a.html.