[Federal Register: April 11, 2000 (Volume 65, Number 70)]
[Notices]
[Page 19572-19577]
From the Federal Register Online via GPO Access [wais.access.gpo.gov]
[DOCID:fr11ap00-150]
-----------------------------------------------------------------------
DEPARTMENT OF EDUCATION
National Awards Program for Effective Teacher Preparation
AGENCY: Office of Educational Research and Improvement (OERI),
Department of Education.
ACTION: Notice of Eligibility And Selection Criteria.
-----------------------------------------------------------------------
SUMMARY: The Assistant Secretary for OERI announces eligibility and
selection criteria to govern competitions under the National Awards
Program for Effective Teacher Preparation for fiscal year (FY) 2000 and
future years. Using these criteria, the awards program will recognize
programs that effectively prepare elementary school teachers or
secondary school mathematics teachers and that lead to improved student
learning.
EFFECTIVE DATE: These eligibility and selection criteria are effective
May 11, 2000.
FOR FURTHER INFORMATION CONTACT: Sharon Horn, Office of Educational
Research and Improvement, U.S. Department of Education, 555 New Jersey
Avenue, NW., room 506E, Washington, DC 20208-5644. Telephone: (202)
219-2203 or FAX to (202) 219-2198. Inquiries also may be sent by e-mail
to: sharon_horn@ed.gov
If you use a telecommunications device for the deaf (TDD), you may call
the Federal Information Relay Service (FIRS) at 1-800-877-8339.
Individuals with disabilities may obtain this document in an
alternate format (e.g., Braille, large print, audiotape, or computer
diskette) on request to the contact person listed in the preceding
paragraph.
SUPPLEMENTARY INFORMATION: This notice announces eligibility and
selection criteria to govern applications for recognition that are
submitted under the National Awards Program for Effective Teacher
Preparation. The criteria established in this notice would be used to
select award recipients in the program's initial year, FY 2000, and in
subsequent fiscal years.
This new program, which is part of a continuing effort to honor
excellence in education, is the result of an increased emphasis across
the country on teacher quality and the well-established principle that
high-quality K-12 teachers are critical to the ability of children in
our nation's schools to achieve to high standards. The program
represents the first systematic approach for identifying entities that
have successfully linked their programs for preparing teachers to
improved student
[[Page 19573]]
achievement at the K-12 level. We believe that the current emphasis on
heightened academic standards for elementary and secondary students and
the need for teachers to gain the knowledge and skills necessary to
teach to those standards makes this program, which focuses attention on
those teacher preparation programs that are particularly effective in
preparing teachers who, in turn, are effective in helping students
improve their learning, all the more timely.
The Assistant Secretary for OERI published a Notice of Proposed
Eligibility and Selection Criteria for this program in the Federal
Register on January 21, 2000 (65 FR 3427). As stated in that notice, we
recognize that demonstrating the link between teacher preparation
programs and the ability of program graduates to improve student
learning is not an easy task. Nevertheless, the difficulty involved
makes that link no less critical. We intend to select for awards no
more than five pre-service teacher preparation programs that are on the
leading edge in this effort. Our chief goal in recognizing these
programs is to foster an understanding of how these noteworthy programs
design their teacher preparation activities to increase K-12 student
achievement and how their approaches can be replicated or built upon by
other institutions that prepare teachers. For that reason, the criteria
for selecting award recipients, as described in this notice, focus
significantly on the ability of applicants to provide compelling
evidence of effectiveness in preparing teachers who positively impact
student learning.
The timeliness of this new awards program is also supported by the
fact that institutions producing teachers, and the states that certify
them, are increasingly coming under scrutiny as the public seeks higher
standards and greater accountability for public schools and school
teachers. The Department, as well as many States, is currently
implementing new accountability measures and reporting requirements for
States and for colleges and universities receiving Federal grants to
support teacher training programs. Some institutions have already
implemented accountability measures, while others have started to take
steps to improve and to become accountable for the teachers they train.
We hope that bringing attention to those teacher preparation programs
that are effective in this area will serve to assist other programs in
their efforts to improve their level of accountability.
In order to align the program with nation-wide efforts to improve
achievement levels in math and reading, this awards program will focus,
in its initial year, on programs that prepare elementary teachers
(since elementary school teachers often teach both math and reading)
and programs that prepare middle or high school mathematics teachers or
both. Thus, to be selected for an award, applicants must be able to
show that their graduates are effective in helping all students improve
their learning in reading and mathematics at the elementary level or
mathematics at the middle and high school level or both. By ``all
students,'' we mean the diverse population of students that graduates
of teacher education programs may encounter in the classroom or other
educational setting, including regular and special education students,
students from diverse backgrounds, and students with limited English
proficiency. The selection process will also depend on the ability of
applicants to demonstrate that their graduates have a depth of content
knowledge in mathematics and reading or both, acquire general and
content-specific pedagogical knowledge and skills, and develop skills
to examine attitudes and beliefs about learners and the teaching
profession.
Note: This notice does not solicit applications. A notice
inviting applications under this competition is published elsewhere
in this edition of the Federal Register.
Analysis of Comments and Changes
In response to the Assistant Secretary's invitation in the notice
of proposed eligibility and selection criteria, two parties submitted
comments. An analysis of the comments and of the changes in the
eligibility and selection criteria since publication of the notice of
proposed criteria follows.
Generally, we do not discuss technical and other minor changes; nor
do we discuss comments that are unrelated to the content of the
eligibility or selection criteria. Substantive issues are addressed
below under the appropriate section to which they pertain.
Eligible Applicants
Comments: One commenter questioned whether the proposed eligibility
(and selection) criteria placed greater emphasis on achievement in
reading, as opposed to mathematics, at the elementary school level.
Discussion: As noted in the preamble discussion above, and in the
notice of proposed eligibility and selection criteria, the National
Awards Program for Effective Teacher Preparation is focused, in this
first year, on the preparation of both reading and mathematics teachers
at the K-12 level. It is anticipated that an entity that prepares
elementary school teachers will focus its application on increased
student learning in reading and mathematics since program graduates
teaching in elementary schools typically teach both subjects. Each
discipline--reading and math--is given equal emphasis in this awards
program. On the other hand, entities that prepare middle school
teachers or high school teachers (or both) must focus their
applications on increased student learning in mathematics, a discipline
routinely taught in middle and high schools.
Changes: None.
Background and Program Description
Comment: One commenter suggested that applicants be required to
consider addressing, as part of the background description of their
program, any applicable State or district policies affecting their
efforts in preparing teachers.
Discussion: In addition to requiring applicants to provide the
mission statement, goals and objectives, and components of their
teacher preparation program, the Background section of the proposed
selection criteria instructed applicants to consider including certain
types of information (e.g., recruitment policies, program structure,
resources, etc.) as part of a full description of their program. We
agree that teacher preparation programs also may be affected by State
or local policies regarding, for example, academic course requirements
for teachers, or other factors that relate to the training of teachers
in a certain geographic region. Thus, we have amended the proposed
criteria to include applicable State or district policies among the
list of items applicants can consider addressing in their applications.
We also note, however, that the list of items to be considered, other
than the mission, goals and objectives, and program components, are
provided only as examples. Applicants are advised to address any one or
more of the identified factors, or other factors, that are most
pertinent to their teacher preparation program.
Changes: This section of the proposed selection criteria has been
amended to refer to State or district policies as an area that
applicants may address as part of the description of their program.
Program's Criteria for Effectiveness
Comment: One commenter suggested that the proposed criteria under
this section be modified to require an explanation of the specific
standards on
[[Page 19574]]
which the applicant's program is based. The commenter indicated that
requiring applicants to explain the standards they follow--whether they
be State licensure, higher education, K-12, or other applicable
standards--will draw attention to the criteria used by award recipients
in their efforts to prepare effective teachers.
Discussion: In this section of the proposed selection criteria, the
question is posed to applicants, ``What are the criteria the program
uses to evaluate [the effectiveness of its teacher preparation
program]?'' This question is designed to ensure that each applicant
describes the relevant standards that it uses to evaluate its program
and guide improvements and modifications. Nevertheless, we agree that
referring to specific examples of standards that might be used in this
regard (e.g., the standards issued by the National Council for
Accreditation of Teacher Education (NCATE) as identified by the
commenter, state teacher licensure standards, or other criteria) will
further guide applicants in addressing this question.
Changes: This section of the proposed selection criteria has been
amended to identify some examples of the types of standards that
entities use for purposes of evaluating the effectiveness of their
teacher preparation program.
Evidence of Effectiveness
Comment: One commenter asked that applicants be required to
demonstrate the impact that their teacher preparation program has on
learning for all students and not just on certain populations of
students.
This commenter also pointed out that applicants may face certain
obstacles in collecting data on teachers, or on K-12 students, that is
needed to demonstrate the effectiveness of their program. For instance,
the commenter noted that it may be difficult for entities preparing
teachers to track graduates who teach in other geographic regions,
while data on reading or math achievement by K-12 students, if used by
an applicant, will vary by State depending upon how often, and the
extent to which, students in the State are tested. For these reasons,
the commenter suggested that applicants be asked to discuss in the
application any intervening factors that impact the evaluation of their
teacher preparation program.
Discussion: We fully agree with the concern expressed by the
commenter that applicants focus on improved learning for all students
and believe that the proposed criteria made clear that selection for an
award will be based significantly on the extent to which an applicant
can demonstrate that their program for preparing teachers leads to
improved student achievement for all students taught by program
graduates. As noted above, and in the preamble guidance to the notice
of proposed criteria, ``all students'' refers to the diverse population
of students that teachers may work with in the classroom (or other
appropriate educational setting). Thus, applicants should provide
evidence of their program's effectiveness on learning for regular
education students, students receiving special education, students from
diverse ethnic backgrounds, students with limited English proficiency,
students in urban and rural areas, and any other identified population
of students, to the extent that program graduates teach such
populations and to the extent that such evidence is available.
In addition, we agree with the commenter that applicants are likely
to encounter different challenges in collecting data and compiling
their evidence of effectiveness. Consequently, this section of the
final selection criteria will invite applicants to discuss those
challenges and how they have overcome any such obstacles in order to
evaluate their program.
Changes: This section of the proposed selection criteria has been
amended to include a note inviting applicants to discuss factors
affecting their data collection efforts and their success in dealing
with these factors in the course of evaluating the effectiveness of
their graduates.
Eligibility, Application, and Selection Criteria
Eligible Applicants
Eligible applicants are institutions in the States (including the
District of Columbia, Puerto Rico, and the outlying areas) that prepare
elementary teachers, or middle or high school mathematics teachers, for
initial certification. Institutions of higher education as well as
institutions that are not part of a college or university are eligible
to apply. Since this program focuses on initial preparation of
teachers, alternative certification programs are eligible, while in-
service programs are not.
For purposes of this notice, a ``teacher preparation program''
refers to a defined set of experiences that, taken as a whole, prepares
participants for initial (or alternative) certification to teach.
Detailed instructions for applying for this award, including formatting
instructions, are provided within the application package and must be
followed to receive an award.
Application Content Requirements
Applicants are free to develop their application in any way they
choose as long as they comply with the requirements set out in the
application package. In evaluating applications for the National Awards
Program for Effective Teacher Preparation, reviewers will look to see
whether the application, taken as a whole, demonstrates that the
applicant's teacher preparation program leads to improved teacher
effectiveness and increased student achievement at the K-12 level. In
doing so, reviewers will be guided by the extent to which and how well
applicants address the following components of the application, the
most important of which concern objective evidence of effectiveness
under Section C of the application.
Sections A, B, and D of the application provide reviewers with
information describing the teacher preparation program and its
potential as an example for others. Reviewers will use the information
in these three sections to determine the extent to which there is a
logical connection between the various aspects of the program and the
results achieved. In other words, they will check for consistency
between the information provided in these sections and the applicant's
claims of effectiveness under section C.
In section C, applicants provide formative, summative, and
confirming evidence that their program is effective in preparing
graduates who are able to help all K-12 students improve their learning
in reading and mathematics at the elementary level or mathematics at
the middle or high school level.
Where appropriate, the following sections of the application
include one or more questions that are designed to help applicants
formulate their responses.
A. Background and Program Description
In this section, applicants must provide the mission statement,
goals and objectives, and the components of their teacher preparation
program and explain how these items relate to the effective preparation
of elementary teachers or middle and/or high school mathematics
teachers.
In responding to this section, applicants are encouraged to provide
information about:
1. Recruitment policies for faculty and candidates.
2. Selection procedures for faculty and candidates.
[[Page 19575]]
3. Program structure (e.g., course and field experiences, support
for preservice and novice teachers, mechanisms for monitoring
participants' progress).
4. State or district policies or mandates that affect the
components of the teacher preparation program.
5. Resources that support the program.
6. Methods for collaboration between the program and K-12 schools.
7. Graduation or completion criteria and rates.
8. Job placement and retention rates of graduates.
B. Program's Criteria for Effectiveness
In this section, applicants must describe the principles,
standards, or other criteria that the applicant uses to judge the
effectiveness of its teacher preparation program.
Note: Applications are not being evaluated against a given set
of principles for all programs, but are expected to include relevant
criteria for guiding program improvement and modifications).
In responding to this section, applicants should consider the
following questions:
1. What are the criteria or standards (e.g., NCATE, INTASC, NBPTS,
NCTM, state teacher licensure requirements and other appropriate
standards) the program uses to evaluate its effectiveness?
2. How does the program ensure that program components such as
courses and instructional practices are consistent with the evaluation
criteria or standards under Question 1?
C. Evidence of Effectiveness
In this section, applicants must provide three separate types of
evidence that demonstrates the effectiveness of their teacher
preparation program: formative, summative, and confirming evidence.
``Formative evidence'' refers to the use of data to make
adjustments to the program throughout its various stages. These data
are collected as participants (i.e., preservice teachers) move through
the program.
``Summative evidence'' demonstrates that the program is effective
in helping graduates acquire the necessary knowledge and skills to
improve student learning. Summative evidence is collected as preservice
teachers complete the program.
``Confirming evidence'' links teacher preparation and K-12 student
learning by demonstrating that program graduates are effective in
helping all K-12 students improve their learning. Confirming evidence
is collected on graduates who are employed by schools or districts.
Applicants would supply a brief description for each evidence item
submitted. This description must include information about the nature
of the data, the methods used to collect the data, and a summary of the
data analysis.
In responding to this section, applicants must consider the
following questions:
1. What evidence is there that the program, described in section A,
gathers data about the effectiveness of the various stages of the
program and uses that data to make improvements to the program?
(Formative evidence)
2. What evidence is there that the program is effective in helping
graduates acquire the knowledge and skills needed to improve student
learning in reading and mathematics for all elementary school students
or in mathematics for all middle or high school students? (Summative
evidence)
Note: Summative evidence in this section should address
graduates' content knowledge, pedagogical knowledge and skills, and
skills to examine beliefs about learners and teaching as a
profession.
3. What evidence is there that the program's graduates are
effective in helping all K-12 students improve their learning in
reading and mathematics at the elementary level or mathematics at the
middle or high school level? (Confirming evidence)
Note: If there are obstacles that affect data collection (e.g.,
local or State regulations prohibit the release of student data),
applicants may describe these factors and explain how they have
overcome any obstacles to collecting data for purposes of evaluating
the effectiveness of their program.
D. Implications for the Field
A primary goal of this awards program is to share with the public
effective examples that might be adopted or otherwise used by others to
improve teacher preparation programs throughout the country. In this
section, applicants must discuss the challenges they have faced and
overcome in administering their teacher preparation program, as well as
the resulting lessons they have learned.
In responding to this section, applicants should consider the
following:
1. What is at least one significant challenge that the program
encountered within the last five years and how was it overcome? (Note:
Since demonstrating the link between teacher preparation and K-12
student learning is a primary focus of the awards program, applicants
should consider describing challenges related to this issue.)
2. What lessons that would benefit others have been learned about
designing, implementing, or evaluating a program that prepares
graduates who are effective in helping improve student learning for all
K-12 students?
3. What program materials (e.g., videos, Web sites, course
outlines, manuals, strategies, processes) are available that could
benefit others?
4. How have or could you help others adapt the aspects of your
program that contribute most to graduates' effectiveness with K-12
students?
Selection Criteria
Reviewers will evaluate the information provided in each
application based on three criteria: rigor, sufficiency, and
consistency. These criteria, and the performance levels applicable to
each, are identified in the rubric shown in Figure 1. Reviewers will
use this rubric as the review instrument to judge the quality of each
application.
The Evidence of Effectiveness provided by an applicant under
section C, the most critical portion of the application, will be
evaluated on the basis of its rigor and sufficiency. The level of
``rigor'' applied to the evidence submitted will be determined by the
extent to which the qualitative or quantitative data presented is found
to be valid and reliable. The level of ``sufficiency'' applied to the
evidence submitted will be determined by the adequacy and the extent of
the data provided.
The application as a whole will be evaluated on the basis of its
consistency. The level of ``consistency'' of the application will be
based on the extent to which there is a logical link between various
aspects of the program as described in Sections A, B, and D of the
application and the evidence of effectiveness provided under Section C.
For example, if an applicant indicates in sections A, B, or D of its
application that field experiences are important to the preparation of
teachers, then the application should describe the variety of field
experiences that are spread over the duration of the program and also
include, for purposes of ``consistency,'' documentation of the
effectiveness of these experiences.
The rubric in Figure 1 identifies a range of performance levels,
from 1 to 4, that reviewers will use to judge the quality of an
application with regard to the three criteria--rigor, sufficiency, and
consistency. Reviewers will assign a level of the rubric, 1 to 4, for
each criterion based on their judgment of how well the information
provided in the application matches the descriptions in the rubric of
the relevant performance levels. Prior to reviewing applications,
[[Page 19576]]
reviewers will receive extensive training in using the rubric to ensure
inter-rater reliability.
Figure 1.--Rubric for Evaluating Evidence of Effectiveness
----------------------------------------------------------------------------------------------------------------
Selection criteria
Performance levels --------------------------------------------------------------------------
Rigor Sufficiency Consistency
----------------------------------------------------------------------------------------------------------------
4.................................... The evidence is highly There are extensive Components of the
credible. The data are data that support program are consistent
valid and indicators claims of with the vision of the
are free of bias. effectiveness. The program. Program
Reliability is evidence includes data components are
supported by multi- from multiple sources monitored to determine
year data from several with multiple if they are being
sources. indicators. instituted as
designed. Evidence
supports an intended,
logical link between
the program components
and the outcomes. The
evidence supports the
link between program
components and program
success. The
consistencies support
the credibility of the
evidence.
3.................................... The evidence is There are adequate data There are minor
credible. Validity has to support the claims inconsistencies
been addressed for of effectiveness. between the vision of
most of the data. There are multiple the program and
There may be some sources of evidence program components.
questions of bias. and multiple Some components of
Reliability is indicators for at program may not be
supported by two or least one source. monitored or there may
more years of data be some
from at least one data inconsistencies
source. between the evidence
provided and the
identified successful
components of the
program. The
inconsistencies do not
weaken the credibility
of the evidence.
2.................................... The evidence has There are limited data There are several
limited credibility. to support the claims inconsistencies
The rigor is of effectiveness. The between the vision of
compromised by issues data are collected the program and
of bias or validity/ from only one or two program components.
reliability. There are sources. There are no There are significant
no multiyear data from multiple indicators inconsistencies
any source. for the data between the evidence
source(s). provided and the
identified successful
components of the
program. The
inconsistencies raise
questions about the
credibility of the
evidence.
1.................................... The evidence has little There are not enough There are numerous
or no credibility. The data to support claims inconsistencies
rigor is significantly of effectiveness. between the vision of
compromised by issues There is only a single the program and its
of bias, or there is source of data. components. The
not enough information evidence provided is
to determine rigor. not linked to the
The data lack validity/ components of the
Reliability. There is program that have been
no multi-year data. identified as
contributing to the
program's success. The
inconsistencies raise
significant questions
about the credibility
of the evidence.
----------------------------------------------------------------------------------------------------------------
Selection Procedures
Award recipients will be selected through a five-stage process.
Stage 1. During the first stage, applications will be initially
screened by Department staff to determine whether the submitting party
meets the eligibility requirements and whether the application contains
all necessary information (including the three types of evidence
required under section C) and meets the formatting requirements.
Stage 2. The second stage of review, to determine up to 10 semi-
finalists, will be conducted by non-Departmental teams representing a
broad range of teacher educators, practitioners (e.g., mathematicians,
mathematics educators, K-12 teachers, reading specialists), and
policymakers (e.g., superintendents, school board members, principals)
who will evaluate the quality of the applications against the selection
criteria and applicable performance levels.
Stage 3. In the third stage, non-Department expert teams (team
members would differ from the reviewers involved in Stages 2) will
conduct site visits to verify information presented in the semi-
finalists' applications and, to the extent available, to collect
additional information. These teams will draft site-visit reports of
their findings.
Stage 4. During the fourth stage, a non-Departmental national
awards panel (panel members will differ from the reviewers involved
Stages 2 and 3) will review the semi-finalist applications and site
visit reports. Panel members will then present final recommendations to
the Department on which teacher preparation programs merit national
recognition.
Stage 5. In the fifth and final stage, the Department will review
data collected throughout the review process and select for national
recognition no more than 5 applications of the highest quality. The
Secretary intends to publicly honor and recognize these awardees at a
national ceremony in Washington, DC.
Goals 2000: Educate America Act
The Goals 2000: Educate America Act (Goals 2000) focuses the
Nation's education reform efforts on the eight National Education Goals
and provides a framework for meeting them. Goals 2000 promotes new
partnerships to strengthen schools and expands the Department's
capacities for helping communities to exchange ideas and
[[Page 19577]]
obtain information needed to achieve the goals.
These eligibility and selection criteria address the National
Education Goal that the Nation's teaching force will have the content
knowledge and teaching skills needed to instruct all American students
for the next century.
Intergovernmental Review
This program is subject to Executive Order 12372 and the
regulations in 34 CFR part 79. One of the objectives of the Executive
order is to foster an intergovernmental partnership and a strengthened
federalism. The Executive order relies on processes developed by State
and local governments for coordination and review of proposed Federal
financial assistance.
This document is intended to provide early notification of our
specific plans and actions for this program.
Program Authority: 20 U.S.C. 8001.
Electronic Access to This Document
You may review this document, as well as all other Department of
Education documents published in the Federal Register, in text or Adobe
Portable Document Format (PDF) on the Internet at either of the
following sites:
http://ocfo.ed.gov/fedreg.htm
http://www.ed.gov/news.html
To use the PDF you must have the Adobe Acrobat Reader Program with
Search, which is available free at either of the previous sites. If you
have questions about using the PDF, call the U.S. Government Printing
Office (GPO), toll free, at 1-888-293-6498; or in the Washington, DC
area, at (202) 512-1530.
Note: The official version of this document is the document
published in the Federal Register. Free Internet access to the
official edition of the Federal Register and the Code of Federal
Regulations is available on GPO Access at: http://
www.access.gpo.gov/nara/index.html.
Dated: April 6, 2000.
C. Kent McGuire,
Assistant Secretary for Educational Research and Improvement.
[FR Doc. 00-8933 Filed 4-6-00; 1:51 pm]
BILLING CODE 4000-01-U