[Federal Register: February 24, 1998 (Volume 63, Number 36)]
[Proposed Rules]
[Page 9391-9397]
From the Federal Register Online via GPO Access [wais.access.gpo.gov]
[DOCID:fr24fe98-24]
[[Page 9391]]
_______________________________________________________________________
Part V
Department of Education
_______________________________________________________________________
34 CFR Part 702
Standards for Conduct and Evaluation of Activities Carried Out by the
Office of Educational Research and Improvement (OERI); Evaluation of
the Performance of Recipients of Grants, Cooperative Agreements, and
Contracts; Proposed Rule
[[Page 9392]]
DEPARTMENT OF EDUCATION
34 CFR Part 702
RIN 1850-AA54
Standards for Conduct and Evaluation of Activities Carried Out by
the Office of Educational Research and Improvement (OERI); Evaluation
of the Performance of Recipients of Grants, Cooperative Agreements, and
Contracts
AGENCY: Department of Education.
ACTION: Notice of proposed rulemaking.
-----------------------------------------------------------------------
SUMMARY: The Assistant Secretary proposes to establish regulations
pursuant to OERI's authorizing legislation, the Educational Research,
Development, Dissemination, and Improvement Act of 1994. The major
purpose of these standards is to ensure that the research, development,
and dissemination activities carried out by the recipients of grants
from and contracts and cooperative agreements with OERI meet the
highest standards of professional excellence.
DATES: Comments must be received by the Department on or before April
27, 1998.
ADDRESSES: All comments concerning these proposed regulations should be
addressed to Sharon Bobbitt, U.S. Department of Education, 555 New
Jersey Avenue, NW., room 508c, Washington, DC 20202-5651. Comments may
also be sent through the Internet to: comments@ed.gov
You must include the term Phase III in the subject line of your
electronic message.
Comments that concern information collection requirements must be
sent to the Office of Management and Budget at the address listed in
the Paperwork Reduction Act section of this preamble. A copy of those
comments may also be sent to the Department representative named in
this section.
FOR FURTHER INFORMATION CONTACT: Sharon Bobbitt. Telephone: (202) 219-
2126. Internet: (Sharon__Bobbitt@ed.gov). Individuals who use a
telecommunications device for the deaf (TDD) may call the Federal
Information Relay Service (FIRS) at 1-800-877-8339 between 8 a.m. and 8
p.m., Eastern time, Monday through Friday.
Individuals with disabilities may obtain this document in an
alternate format (e.g., Braille, large print, audiotape, or computer
diskette) on request to the contact person listed in the preceding
paragraph.
SUPPLEMENTARY INFORMATION:
Invitation to Comment
Interested persons are invited to submit comments and
recommendations regarding these proposed regulations. To ensure that
public comments have maximum effect in developing the final
regulations, the Department urges commenters to identify clearly the
specific section or sections of the proposed regulations that each
comment addresses and to arrange comments in the same order as the
proposed regulations.
The Secretary particularly requests comments on the role of
Department of Education staff in the implementation of the Standards.
For example, should Department staff serve as reviewers on peer review
panels under these regulations? See proposed Sec. 702.10(d) of these
regulations in this regard. Should there be a maximum number or maximum
percentage of Department staff on peer review panels? Should the
participation of Department staff vary by size of the grant, contract,
or cooperative agreement? What other issues about the role of
Department staff in the peer review process should the Secretary
consider?
All comments submitted in response to these proposed regulations
will be available for public inspection, during and after the comment
period, in Room 600, 555 New Jersey Avenue, N.W., Washington, D.C.,
between the hours of 8:30 a.m. and 4:00 p.m., Eastern time, Monday
through Friday of each week except Federal holidays.
On request the Department supplies an appropriate aid, such as a
reader or print magnifier, to an individual with a disability who needs
assistance to review the comments or other documents in the public
rulemaking docket for these proposed regulations. An individual with a
disability who wants to schedule an appointment for this type of aid
may call (202) 205-8113 or (202) 260-9895. An individual who uses a TDD
may call the Federal Information Relay Service at 1-800-877-8339,
between 8 a.m. and 8 p.m., Eastern time, Monday through Friday.
To assist the Department in complying with the specific
requirements of Executive Order 12866 and its overall requirement of
reducing regulatory burden, the Secretary invites comments on whether
there may be further opportunities to reduce any regulatory burdens
found in these proposed regulations.
Background
On March 31, 1994, President Clinton signed Pub. L. 103-227, which
includes Title IX, the Educational Research, Development,
Dissemination, and Improvement Act of 1994 (the Act). The Act
restructured OERI and provided it with a broad mandate to conduct an
array of research, development, dissemination, and improvement
activities aimed at strengthening the education of all students.
Statutory Requirements
The Act directed the Assistant Secretary to develop, in
consultation with the National Educational Research Policy and
Priorities Board (the Board), such standards as may be necessary to
govern the conduct and evaluation of all research, development, and
dissemination activities carried out by OERI to ensure that these
activities meet the highest standards of professional excellence. The
Board is responsible for reviewing and approving the standards. The
legislation requires that the standards be developed in three phases.
In the first phase, standards were created and promulgated to
establish the peer review process and evaluation criteria to be used
for the review of applications for grants and cooperative agreements
and proposals for contracts. The final regulations setting out these
standards were published on September 14, 1995 (60 FR 47808). In the
second phase, standards were created and promulgated to establish the
criteria to be used in reviewing potentially exemplary and promising
educational programs. The final regulations setting out these standards
were published on November 17, 1997 (62 FR 61427).
In the third phase, which is the subject of this notice of proposed
rulemaking (NPRM), the Act requires that OERI develop standards for
evaluating and assessing the performance of all recipients of grants
from and cooperative agreements and contracts with OERI. This
evaluation must take place both during and at the conclusion of the
performance of the grant, cooperative agreement, or contract, and must
include the use of a system of peer review for the final assessment.
In developing the standards, the Assistant Secretary was required
to review the procedures utilized by the National Institutes of Health
(NIH), the National Science Foundation (NSF), and other Federal
departments or agencies engaged in research and development and to
solicit recommendations from research organizations and members of the
general public. OERI has reviewed the procedures used to evaluate the
performance of recipients of grants, contracts, or cooperative
agreements by several offices within NIH and NSF, the Office of Energy
Research in the
[[Page 9393]]
Department of Energy, the Food and Drug Administration, the National
Institute of Standards and Technology, the National Aeronautics and
Space Administration, and the University Research Initiative of the
Department of Defense. Recommendations concerning these standards have
been obtained from the American Educational Research Association, the
Council for Educational Development and Research, and the Organization
of Research Centers. Public comment is invited in response to this
NPRM.
Standards
The standards have been developed by the Assistant Secretary in
consultation with the Board. The standards in this NPRM would:
* Require interim and final assessments of the performance
of recipients of grants, cooperative agreements, and contracts.
* Establish procedures for selecting peer review panels to
conduct these assessments.
* Establish procedures and criteria that the peer review
panels use in conducting these assessments.
* Establish specific additional criteria that peer review
panels use in conducting these assessments for National Research and
Development Centers, Regional Educational Laboratories, Field-Initiated
Studies, and ERIC Clearinghouses.
In an effort to fulfill the law's intention of ensuring high-
quality research, development, and evaluation, OERI has developed
standards in which interim and final assessments may be supplemented by
a self-assessment by the recipient of a grant, cooperative, agreement,
or contract. The Board and the Assistant Secretary believe that the
collection and review of evidence on one's own performance is itself a
useful tool for improvement.
These standards cover all grants, cooperative agreements, and
contracts administered by OERI, ranging from the smallest purchase
orders and commissioned papers to the largest research projects and
research centers. The Department will require a single interim
assessment by a peer review panel for total awards of $5,000,000 or
less. At least one interim review by peer review panel will be required
for larger awards. A final assessment by a peer review panel will be
required for all awards.
The Government Performance and Results Act requires the
establishment of performance indicators for Department activities.
Information collected pursuant to those indicators will be considered,
as appropriate, in the evaluation of individual recipients.
Executive Order 12866
1. Potential Costs and Benefits
These proposed regulations have been reviewed in accordance with
Executive Order 12866. Under the terms of the order the Secretary has
assessed the potential costs and benefits of this regulatory action.
The potential costs associated with the proposed regulations are
those resulting from statutory requirements and those determined by the
Secretary as necessary for administering this program effectively and
efficiently. Burdens specifically associated with information
collection requirements are identified and explained elsewhere in this
preamble under the heading Paperwork Reduction Act of 1995.
In assessing the potential costs and benefits--both quantitative
and qualitative--of these proposed regulations, the Secretary has
determined that the benefits of the proposed regulations justify the
costs.
The Secretary has also determined that this regulatory action does
not unduly interfere with State, local, and tribal governments in the
exercise of their governmental functions.
To assist the Department in complying with the specific
requirements of Executive Order 12866, the Secretary invites comment on
whether there may be further opportunities to reduce any potential
costs or increase potential benefits resulting from these proposed
regulations without impeding the effective and efficient administration
of the program.
Summary of Potential Costs and Benefits
The potential costs of the proposed regulations are discussed in
this preamble under the Paperwork Reduction Act of 1995. The benefit of
these standards is to ensure that the research, development, and
dissemination activities carried out by the recipients of grants from
and contracts and cooperative agreements with OERI meet the highest
standards of professional excellence.
2. Clarity of the Regulations
Executive Order 12866 requires each agency to write regulations
that are easy to understand.
The Secretary invites comments on how to make these proposed
regulations easier to understand, including answers to questions such
as the following: (1) Are the requirements in the proposed regulations
clearly stated? (2) Do the proposed regulations contain technical terms
or other wording that interferes with their clarity? (3) Does the
format of the proposed regulations (grouping and order of sections, use
of headings, paragraphing, etc.) aid or reduce their clarity? Would the
proposed regulations be easier to understand if they were divided into
more (but shorter) sections? (A ``section'' is preceded by the symbol
``Sec. '' and a numbered heading; for example, Sec. 702.2 What
activities must be evaluated by these standards?) (4) Is the
description of the proposed regulations in the ``Supplementary
Information'' section of this preamble helpful in understanding the
proposed regulations? How could this description be more helpful in
making the proposed regulations easier to understand? (5) What else
could the Department do to make the proposed regulations easier to
understand?
A copy of any comments that concern how the Department could make
these proposed regulations easier to understand should be sent to
Stanley M. Cohen, Regulations Quality Officer, U.S. Department of
Education, 600 Independence Avenue, SW. (Room 5121, FB-10), Washington,
D.C. 20202-2241.
Regulatory Flexibility Act Certification
The Secretary certifies that these proposed regulations would not
have a significant economic impact on a substantial number of small
entities.
The small entities that would be affected by these proposed
regulations are small local educational agencies (LEAs) and private
schools receiving Federal funds under this program. However, the
regulations would not have a significant economic impact on the small
LEAs and private schools affected because the proposed regulations
would not impose excessive regulatory burdens or require unnecessary
Federal supervision. The proposed regulations would impose minimal
requirements to ensure the proper expenditure of program funds.
Paperwork Reduction Act of 1995
Sections 702.22 and 702.23 contain information collection
requirements. As required by the Paperwork Reduction Act of 1995 (44
U.S.C. 3504(h)), the Department of Education has submitted a copy of
these sections to the Office of Management and Budget (OMB) for its
review.
Collection of Information: Standards for Evaluation of the
Performance of Recipients of OERI Grants, Cooperative Agreements, and
Contracts.
These regulations affect the following types of entities eligible
to enter into
[[Page 9394]]
grants, cooperative agreements, or contracts: any public or private
agency, organization or institution, or individual.
The public reporting burden is estimated to range from 8 to 120
hours for each interim or final assessment. The actual burden will be
determined by how much descriptive information each recipient wishes to
provide.
Organizations and individuals desiring to submit comments on the
information collection requirements should direct them to the Office of
Information and Regulatory Affairs, OMB, room 10235, New Executive
Office Building, Washington, DC 20503; Attention: Desk Officer for U.S.
Department of Education.
The Department considers comments by the public on this proposed
collection of information in--
* Evaluating whether the proposed collection of information
is necessary for the proper performance of the functions of the
Department, including whether the information will have practical use;
* Evaluating the accuracy of the Department's estimate of
the burden of the proposed collection of information, including the
validity of the methodology and assumptions used;
* Enhancing the quality, usefulness, and clarity of the
information to be collected; and
* Minimizing the burden of the collection of information on
those who are to respond, including through the use of appropriate
automated, electronic, mechanical, or other technological collection
techniques or other forms of information technology; e.g., permitting
electronic submission of responses.
OMB is required to make a decision concerning the collection of
information contained in these proposed regulations between 30 and 60
days after publication of this document in the Federal Register.
Therefore, a comment to OMB is best assured of having its full effect
if OMB receives it within 30 days of publication. This does not affect
the deadline for the public to comment to the Department on the
proposed regulations.
Assessment of Educational Impact
The Secretary particularly requests comments on whether the
proposed regulations in this document would require transmission of
information that is being gathered by or is available from any other
agency or authority of the United States.
Electronic Access to This Document
Anyone may view this document, as well as all other Department of
Education documents published in the Federal Register, in text or
portable document format (pdf) on the World Wide Web at either of the
following sites:
http://gcs.ed.gov/fedreg.htm
http://www.ed.gov/news.html
To use the pdf you must have the Adobe Acrobat Reader Program with
Search, which is available free at either of the previous sites. If you
have questions about using the pdf, call the U.S. Government Printing
Office toll free at 1-888-293-6498.
Anyone may also view these documents in text copy only on an
electronic bulletin board of the Department. Telephone: (202) 219-1511
or, toll free, 1-800-222-4922. The documents are located under Option
G-- Files/Announcements, Bulletins and Press Releases.
Note: The official version of this document is the document
published in the Federal Register.
List of Subjects in 34 CFR Part 702
Education, Educational research, Reporting and recordkeeping
requirements.
Dated: December 23, 1997.
Ricky Takai,
Acting Assistant Secretary for Educational Research and Improvement.
(Catalog of Federal Domestic Assistance Number does not apply)
The Secretary proposes to amend Chapter VII of Title 34 of the
Code of Federal Regulations by adding a new part 702 to read as
follows:
PART 702--STANDARDS FOR CONDUCT AND EVALUATION OF ACTIVITIES CARRIED
OUT BY THE OFFICE OF EDUCATIONAL RESEARCH AND IMPROVEMENT (OERI)--
EVALUATION OF THE PERFORMANCE OF RECIPIENTS OF GRANTS, COOPERATIVE
AGREEMENTS, AND CONTRACTS
Subpart A--General
Sec.
702.1 What is the purpose of these standards?
702.2 What activities must be evaluated by these standards?
702.3 What additional activities may be evaluated by these
standards?
702.4 When is performance assessed under these standards?
702.5 What definitions apply?
Subpart B--Selection of Peer Review Panels
702.10 What are the characteristics of peer reviewers?
702.11 What constitutes a conflict of interest for grants and
cooperative agreements?
702.12 What constitutes a conflict of interest for contracts?
702.13 How are peer reviewers selected for panels?
Subpart C--The Evaluation Process
702.21 How does a peer review panel evaluate the performance of a
recipient?
702.22 What information does a peer review panel consider for an
interim assessment?
702.23 What information does a peer review panel consider for a
final assessment?
702.24 What evaluation criteria are used for performance
assessments?
Authority: 20 U.S.C. 6011(i), unless otherwise noted.
Subpart A--General
Sec. 702.1 What is the purpose of these standards?
(a) The standards in this part implement section 912(i) of the
Educational Research, Development, Dissemination, and Improvement Act
of 1994 (the Act).
(b) These standards are intended to ensure that the research,
development, and dissemination activities carried out by the recipients
of grants from and contracts and cooperative agreements with the Office
of Educational Research and Improvement (OERI) meet the highest
standards of professional excellence.
(Authority: 20 U.S.C. 6011(i)(2)(F))
Sec. 702.2 What activities must be evaluated by these standards?
These standards apply to activities carried out by OERI using funds
appropriated under section 912(m) of the Act including activities
carried out by the following entities or programs:
(a) The National Education Research Institutes.
(b) The Office of Reform Assistance and Dissemination.
(c) The Educational Resources Information Center.
(d) The Regional Educational Laboratories.
(e) The Teacher Research Dissemination Demonstration Program.
(f) The Goals 2000 Community Partnerships Program.
(g) The National Educational Research Policy and Priorities Board.
(Authority: 20 U.S.C. 6011(i)(1))
[[Page 9395]]
Sec. 702.3 What additional activities may be evaluated by these
standards?
(a) The Secretary may apply these standards to other activities
funded by the Department.
(Authority: 20 U.S.C. 6011(I)(1))
Sec. 702.4 When is performance assessed under these standards?
(a) The Secretary will assess the performance of recipients of OERI
grants, contracts, and cooperative agreements subject to these
standards during and at the conclusion of their period of performance.
(b) The Department requires a single interim assessment by a peer
review panel for total awards of $5,000,000 or less. At least one
interim review by peer review panel is required for larger awards.
(c) A final assessment by a peer review panel is required for all
awards.
(d) As used in this part--
(1) Interim assessment is one conducted during a recipient's period
of performance.
(2) Final assessment is one conducted at the conclusion of a
recipient's period of performance.
(Authority: 20 U.S.C. 6011(I)(2)(F))
Sec. 702.5 What definitions apply?
(a) Definitions in the Educational Research, Development,
Dissemination, and Improvement Act of 1994.
The following terms used in this part are defined in 20 U.S.C.
6011(1):
Development
Dissemination
Educational research
(b) Definitions in the Education Department General Administrative
Regulations. The following terms used in this part are defined in 34
CFR 77.1:
Application
Award
Department
Grant
Project
Secretary
(c) Definitions in the Federal Acquisition Regulation. The
following term used in this part is defined in 48 CFR Chapter 1:
Contract Proposal
(Authority: 20 U.S.C. 6011(I)(2)(F))
Subpart B--Selection of Peer Review Panels
Sec. 702.10 What are the characteristics of peer reviewers?
(a) The Assistant Secretary selects each peer reviewer. Each peer
reviewer must have the necessary knowledge and expertise in the area of
the project being reviewed to evaluate the performance of a recipient.
This experience may include--
(1) Expert knowledge of subject matter in the area of the
activities to be reviewed;
(2) Expert knowledge of theory or methods or both in the area of
the activities to be reviewed;
(3) Practical experience in the area of the activities or type of
institution or both to be reviewed;
(4) Knowledge of a broad range of education policies and practices;
(5) Experience in managing complex organizations; or
(6) Expertise and experience in evaluation theory and practice.
(b) Each peer reviewer must be free of conflict of interest, as
determined in accordance with Sec. 702.11 or 702.12.
(c) The Assistant Secretary may solicit nominations for peer
reviewers from professional associations, nationally recognized
experts, and other sources.
(d) OERI and other Department staff who possess the qualifications
in paragraphs (a) and (b) of this section may serve as peer reviewers.
(Authority: 20 U.S.C. 6011(I)(2)(B))
Sec. 702.11 What constitutes a conflict of interest for grants and
cooperative agreements?
A peer reviewer assessing the performance of the recipient of a
grant from or cooperative agreement with OERI is considered an employee
of the Department for the purposes of conflict of interest analysis. As
an employee of the Department, the peer reviewer is subject to the
provisions of 18 U.S.C. 208, 5 CFR 2635.502, and the Department's
policies used to implement those provisions.
(Authority: 20 U.S.C. 6011(I)(2)(B))
Sec. 702.12 What constitutes a conflict of interest for contracts?
A peer reviewer assessing the performance of the recipient of a
contract with OERI is considered an employee of the Department in
accordance with the Federal Acquisition Regulation (FAR), 48 CFR 3.104-
4(h)(2). As an employee of the Department, the peer reviewer is subject
to the provisions of the FAR, 48 CFR Part 3, Improper Business
Practices and Personal Conflict of Interest.
(Authority: 41 U.S.C. 423)
Sec. 702.13 How are peer reviewers selected for panels?
(a) The Assistant Secretary assigns peer reviewers to panels that
conduct the performance assessments.
(b) The Assistant Secretary may establish panels by category of
recipient, such as a panel to review the performance of all Regional
Educational Laboratories. Each recipient is evaluated individually by
reviewers who have been assigned to this type of panel.
(Authority: 20 U.S.C. 6011(I)(2)(B))
Subpart C--The Evaluation Process
Sec. 702.21 How does a peer review panel evaluate the performance of a
recipient?
(a) In each evaluation, a peer review panel--
(1) Considers relevant information about the recipient's
performance, as described in Secs. 702.22 and 702.23; and
(2) Makes judgments about the recipient's performance, using the
criteria in Sec. 702.24.
(b) Each peer reviewer prepares a report based on the reviewer's
assessment of the quality of the project according to the evaluation
criteria.
(c) After each peer reviewer has evaluated each project
independently, the panel may be convened to discuss the strengths and
weaknesses of the project. Each reviewer may then independently re-
evaluate each project with appropriate changes made to the written
report.
(d) The report of the interim assessment must include any
recommendations the peer reviewer may have for improving the
recipient's performance.
(e) The report of the final assessment must contain each peer
reviewer's evaluative summary of the recipient's performance, from the
beginning of the contract, grant, or cooperative agreement to its
conclusion.
(Authority: 20 U.S.C. 6011(I)(2)(F))
Sec. 702.22 What information does a peer review panel consider for an
interim assessment?
(a) Sources of information for the interim assessment must
include--
(1) The original request for proposals or grant announcement and
the contract proposal or grant application;
(2) Documentation of any changes in the work described in the
contract, grant, or cooperative agreement, including reasons for the
changes;
(3) Any progress reports delivered to the Department or made
available to the public by the recipient;
(4) Examples of products delivered to the Department or made
available to the public by the recipient;
(5) Any relevant reports written by OERI staff, including reports
of site visits by OERI staff;
(6) Any performance evaluations conducted under the FAR or the
Education Department General Administrative Regulations (34 CFR part
75).
[[Page 9396]]
(7) Any relevant information provided by the recipient in response
to Government Performance and Results Act (GPRA) (Pub. L. 103-62)
requirements; and
(8) Any reports from program evaluations commissioned by the
Department.
(b) Sources of information for the interim assessment may also
include--
(1) A self-assessment, prepared by the recipient, addressing the
criteria in Sec. 702.24;
(2) One or more site visits by the peer review panel;
(3) One or more oral or written presentations to the panel by the
recipient describing its performance; or
(4) Other information about the recipient's performance.
(Authority: 20 U.S.C. 6011(I)(2)(F))
Sec. 702.23 What information does a peer review panel consider for a
final assessment?
(a) Sources of information for the final assessment must include--
(1) The original request for proposals or application notice and
the contract proposal or grant application, together with documentation
of any changes in the work described in the proposal or application,
including reasons for the changes;
(2) If consistent with the recipient's contract, grant, or
cooperative agreement with OERI, a written report or oral presentation
or both by the recipient summarizing its activities and
accomplishments;
(3) Any relevant information provided by the recipient in response
to Government Performance and Results Act (GPRA) (Pub. L. 103-62)
requirements; and
(4) Any reports from program evaluations commissioned by the
Department.
(b) The final assessment may also include other sources of
information, such as one or more of those listed in Sec. 702.22.
(Authority: 20 U.S.C. 6011(I)(2)(F))
Sec. 702.24 What evaluation criteria must be used for performance
assessments?
(a) Peer reviewers (and those recipients who conduct self-
evaluations) shall use the criteria in paragraph (b) of this section to
assess performance and, in case of interim assessments, to identify
areas in which the performance of recipients may need improvement.
(b) The following evaluation criteria are to guide the assessment
process undertaken by peer reviewers. The peer reviewers determine the
extent to which recipients meet these criteria:
(1) Implementation and management. (i) Peer reviewers shall
consider the degree to which the recipient has fully executed its
program of work. In doing so, peer reviewers shall consider evidence on
the extent to which the recipient completes the work described in the
approved application or contract, including any approved modifications,
in the time period proposed and in an efficient manner.
(ii) In examining the degree of implementation, peer reviewers may
also consider evidence on the extent to which--
(A) The recipient implements and utilizes a quality assurance
system for its products or services or both; and
(B) The recipient conducts self-assessment or self-evaluation
activities, including periodically seeking out independent critiques
and evaluations of its work, and uses the results to improve
performance.
(2) Quality. (i) Peer reviewers shall consider the degree to which
the recipient's work approaches or attains professional excellence. In
determining quality, peer reviewers shall consider evidence on the
extent to which--
(A) The recipient utilizes processes, methods, and techniques
appropriate to achieve the goals and objectives for the program of work
in the approved application; and
(B) The recipient applies appropriate processes, methods, and
techniques in a manner consistent with the highest standards of the
profession.
(ii) In determining quality, peer reviewers may also consider the
extent to which the recipient conducts a coherent, sustained program of
work informed by relevant research.
(3) Utility. (i) In determining the utility of the recipient's
products or services or both, peer reviewers shall consider evidence on
the extent to which the recipient's work (including information,
materials, processes, techniques, or activities) is effectively used by
and is useful to its customers in appropriate settings.
(ii) In determining utility, peer reviewers may also consider the
extent to which the recipient has received national recognition; e.g.,
articles in refereed journals and presentations at professional
conferences.
(4) Outcomes and impact. (i) Peer reviewers shall consider the
results of the recipient's work. In examining outcomes and impact, peer
reviewers shall consider evidence on the extent to which--
(A) The recipient meets the needs of its customers; and
(B) The recipient's work contributes to the increased knowledge or
understanding of educational problems, issues, or effective strategies.
(ii) In examining outcomes and impact, peer reviewers may also
consider the extent to which recipients address issues of national
significance through its products or services or both.
(c) For National Research and Development Centers, peer reviewers
also shall consider evidence on the extent to which recipients meet the
following criteria:
(1) Quality. (i) The recipient uses a well-conceptualized framework
and sound theoretical and methodological tools in conducting
professionally rigorous studies; and
(ii) The recipient conducts work of sufficient size, scope, and
duration to produce sound guidance for improvement efforts and future
research.
(2) Utility. The recipient documents, reports, and disseminates its
work in ways to facilitate the effective use of its work in
appropriately targeted settings.
(3) Outcomes and impact. (i) The recipient's work contributes to
the development and advancement of theory in the field of study,
including its priority area; and
(ii) The recipient addresses issues of national significance
through its products or services or both.
(d) For the Regional Educational Laboratories, peer reviewers also
shall consider evidence on the extent to which recipients meet the
following criteria:
(1) Quality. (i) The recipient utilizes a well-conceptualized
framework and sound theoretical and methodological tools in conducting
professionally rigorous studies;
(ii) The recipient conducts work of sufficient size, scope, and
duration to produce sound guidance for improvement efforts; and
(iii) The recipient's products are well-tested and based on sound
research.
(2) Utility. The recipient documents, reports, and disseminates its
work in ways to facilitate its effective use in appropriately targeted
settings, particularly in school improvement efforts of States and
localities.
(3) Outcomes and impact. (i) The recipient assists States and
localities to implement comprehensive school improvement strategies
through the provision of research-based information (including well-
tested models and strategies), materials and assistance; and
(ii) The recipient's work results in widespread access to
information regarding research and best practices, particularly within
its region.
(e) For Field-Initiated Studies, peer reviewers also shall consider
evidence on the extent to which recipients meet the following criteria:
[[Page 9397]]
(1) Implementation and management. The recipient's work responds to
the goals, objectives and mission of the National Institute from which
it is funded.
(2) Quality. The recipient utilizes a well-conceptualized framework
and sound theoretical and methodological tools in conducting
professionally rigorous studies.
(3) Utility. The recipient documents, reports, and disseminates its
work in ways to facilitate its effective use in appropriately targeted
settings.
(4) Outcomes and impact. (i) The recipient's work contributes to
the development and advancement of theory and knowledge in the field of
study; and
(ii) The recipient addresses issues of national significance
through its products or services or both.
(f) For the ERIC Clearinghouses, peer reviewers also shall consider
evidence on the extent to which recipients meet the following criteria:
(1) Quality. The recipient applies an integrated approach to
acquiring and disseminating significant and high-quality educational
literature and materials to maintain and enhance the ERIC database.
(2) Utility. The recipient contributes to the development of the
ERIC database as a source of literature and materials that reflects
trends and issues within its scope.
(3) Outcomes and impact. (i) The recipient meets the informational
and educational needs of its customers through dissemination and
outreach approaches and the development of an array of print and non-
print materials; and
(ii) The recipient provides national leadership on the use of
current computer, networking, and information technology.
(Authority: 20 U.S.C. 6011(I)(2)(F))
[FR Doc. 98-4690 Filed 2-23-98; 8:45 am]
BILLING CODE 4000-01-P