[Federal Register: June 3, 1996 (Volume 61, Number 107)] [Proposed Rules] [Page 27989-27993] From the Federal Register Online via GPO Access [wais.access.gpo.gov] [[Page 27989]] _______________________________________________________________________ Part V Department of Education _______________________________________________________________________ 34 CFR Part 701 Standards for Conduct and Evaluation of Activities Carried Out by the Office of Educational Research and Improvement (OERI)--Designation of Exemplary and Promising Programs; Proposed Rule [[Page 27990]] DEPARTMENT OF EDUCATION 34 CFR Part 701 RIN 1850-AA52 Standards for Conduct and Evaluation of Activities Carried Out by the Office of Educational Research and Improvement (OERI)--Designation of Exemplary and Promising Programs AGENCY: Department of Education. ACTION: Notice of proposed rulemaking (NPRM). ----------------------------------------------------------------------- SUMMARY: The Assistant Secretary is developing these standards pursuant to the Office of Educational Research and Improvement's authorizing legislation, the ``Educational Research, Development, Dissemination, and Improvement Act of 1994.'' The major purpose of these standards is to provide quality assurance that programs designated by the Department of Education as either exemplary or promising have met criteria that will allow educators, professional organizations, and others to use these programs with confidence. DATES: Comments must be received on or before August 2, 1996. ADDRESSES: All comments concerning these proposed regulations should be addressed to Eve M. Bither, U.S. Department of Education, 555 New Jersey Avenue, N.W., Room 500, Washington, D.C. 20208-5530. Comments may also be sent through the Internet to: (Eve__Bither@ed.gov). Comments that concern information collection requirements should be sent to the Office of Management and Budget at the address listed in the Paperwork Reduction Act section of this preamble. FOR FURTHER INFORMATION CONTACT: Sharon Bobbitt, U.S. Department of Education, 555 New Jersey Avenue, N.W., Room 508, Washington, D.C. 20208-5643. Telephone: (202) 219-2126. Internet: (Sharon__Bobbitt@ed.gov). Individuals who use a telecommunications device for the deaf (TDD) may call the Federal Information Relay Service (FIRS) at 1-800-877-8339 between 8 a.m. and 8 p.m., Eastern time, Monday through Friday. SUPPLEMENTARY INFORMATION: Background On March 31, 1994, President Clinton signed Pub. L. 103-227, which includes Title IX, the ``Educational Research, Development, Dissemination, and Improvement Act of 1994'' (the ``Act''). The Act restructured the Office of Educational Research and Improvement (OERI) and provided it with a broad mandate to conduct an array of research, development, dissemination, and improvement activities aimed at strengthening the education of all students. Statutory Requirements The legislation directed the Assistant Secretary to develop, in consultation with the National Educational Research Policy and Priorities Board, such standards as may be necessary to govern the conduct and evaluation of all research, development, and dissemination activities carried out by the Office to ensure that those activities meet the highest standards of professional excellence. The legislation requires that the standards be developed in three phases. In the first phase, standards were promulgated to establish the peer review process and evaluation criteria to be used for the review of applications for grants and cooperative agreements and proposals for contracts. The final regulations setting out these standards were published in the Federal Register on September 14, 1995 (60 FR 47808). These proposed regulations address the second phase of development by establishing the criteria to be used in reviewing potentially exemplary and promising educational programs. The Assistant Secretary will publish at a later date additional proposed regulations for phase three of the standards, which are to govern evaluation of the performance of recipients of grants and contracts and cooperative agreements with OERI. The OERI legislation requires that expert panels be established to review educational programs and recommend to the Secretary those programs that should be designated as exemplary or promising and disseminated through the Department's National Education Dissemination System. The legislation further requires the Assistant Secretary to develop standards that describe the procedures the panels will use in reviewing the educational programs. Section 941(a)(3) of the legislation broadly defines ``educational program'' to include ``educational polices, research findings, practices and products.'' Educational programs may range in size and complexity from an individual instructional program--such as an elementary school science program--to a comprehensive reform initiative involving multiple goals and participants. Programs at all levels of education--preschool, elementary, secondary, and postsecondary--are eligible for consideration. The Act also requires that the Assistant Secretary review the procedures utilized by the National Institutes of Health (NIH), the National Science Foundation (NSF), and other Federal departments or agencies engaged in research and development and actively solicit recommendations from research organizations and members of the general public. In developing the review and evaluation procedures for the proposed standards, OERI has reviewed and considered dissemination practices and procedures used for identifying promising and exemplary programs by various foundations, research organizations, associations, and Federal agencies including NIH, NSF, the Federal Emergency Management Agency, the Department of Health and Human Services, and the National Endowment for the Arts. OERI adapted these review and evaluation procedures as appropriate. Proposed Standards The proposed standards have been developed by the Assistant Secretary in consultation with the Board. The standards proposed in this NPRM-- * Require that expert panels be established to review educational programs and recommend to the Secretary those programs that should be designated as exemplary or promising and disseminated through the Department's National Education Dissemination System; and * Establish a process that panels will use to review and evaluate educational programs and determine which programs to recommend to the Secretary for designation as exemplary or promising. Educational programs may be submitted at any time for consideration for designation as exemplary or promising. In addition, the Assistant Secretary will periodically establish and announce in the Federal Register specific topic areas of high priority for which programs will be invited or sought out. The legislation also provides that the Secretary may identify educational programs for the panels to review. Educational program submissions may include, as evidence of the effectiveness of the program, a range of assessments, evaluative information from users, and other objective performance indicators that are appropriate to the program. The legislation ensures that a panel may not eliminate any program from consideration based on the lack of one type of supporting data such as test scores. [[Page 27991]] A standing group of experts, which will include teachers and others, will be appointed by the Assistant Secretary as appropriate. From that group, the Assistant Secretary will select members who have relevant knowledge and experience in specific topic areas to form expert panels to review programs in accordance with the criteria in these proposed regulations. In determining whether an educational program should be recommended as exemplary or promising, the panel is required by the legislation to consider (a) whether, based on empirical data, the program is effective and should be designated as exemplary, or (b) whether there is sufficient evidence to demonstrate that the program shows promise for improving student achievement and should be designated as promising. These proposed regulations require a panel to evaluate whether a program has met all of the criteria of educational effectiveness set forth in Subpart C of these proposed regulations. A panel may determine that a program shows promise for improving student achievement and recommend that the program be designated as promising if the program has met all of the criteria with respect to one context, or with one population. A panel may determine that a program is effective and recommend that the program be designated as exemplary if the program has met all of the criteria with respect to multiple contexts, or with multiple populations. Use of these criteria for evaluating programs will ensure that programs disseminated by the Department are high-quality, research- based programs that have provided evidence indicating they have improved teaching or learning or both. The Department's dissemination system is designed to make programs available to the public as quickly as possible. The system will enable the Department to respond to all forms of requests for information and assistance, and to support the applications of research and best practice. The system will use electronic networking and the capabilities of: National Research Institutes; Educational Resources Information Center (ERIC); Regional Educational Laboratories; Department-supported technical assistance providers; National Library of Education; and Other public and private nonprofit entities, including education associations and networks. Prior to the adoption of these standards, exemplary programs were validated by the Department's Program Effectiveness Panel (PEP) and disseminated through the National Diffusion Network (NDN). With the adoption of these standards, the Department will recognize and disseminate promising educational programs in addition to exemplary programs. Regulatory Flexibility Act Certification The Secretary certifies that these proposed regulations would not have a significant economic impact on a substantial number of small entities. The small entities that would be affected by these proposed regulations are small local educational agencies (LEAs) and private schools receiving Federal funds under this program. However, the regulations would not have a significant economic impact on the small LEAs and private schools affected, because the regulations would not impose excessive regulatory burdens or require unnecessary Federal supervision. The regulations would impose minimal requirements to ensure the proper expenditure of program funds. Paperwork Reduction Act of 1995 Section 701.4 contains information collection requirements. As required by the Paperwork Reduction Act of 1995, the Department of Education will submit a copy of this section to the Office of Management and Budget (OMB) for its review. (44 U.S.C. 3504(h)) These regulations affect the following types of entities eligible to submit a program for review: Any public or private agency, organization or institution, or individual. The public reporting burden is estimated to range from 2 to 6 hours for each program submitted for review. The actual burden will be determined by how much descriptive information about their program each entity wishes to provide. Organizations and individuals desiring to submit comments on the information collection requirements should direct them to the Office of Information and Regulatory Affairs, OMB, Room 10235, New Executive Office Building, Washington, D.C. 20503; Attention: Wendy Taylor. The Department considers comments by the public on this proposed collection of information in-- * Evaluating whether the proposed collection of information is necessary for the proper performance of the functions of the Department, including whether the information will have practical utility; * Evaluating the accuracy of the Department's estimate of the burden of the proposed collection of information, including the validity of the methodology and assumptions used; * Enhancing the quality, usefulness, and clarity of the information to be collected; and * Minimizing the burden of the collection of information on those who are to respond, including through the use of appropriate automated, electronic, mechanical, or other technological collection techniques or other forms of information technology (e.g., permitting electronic submission of responses). OMB is required to make a decision concerning the collection of information contained in these proposed regulations between 30 and 60 days after publication of this document in the Federal Register. Therefore, a comment to OMB is best assured of having its full effect if OMB receives it within 30 days of publication. This does not affect the deadline for the public to comment to the Department on the proposed regulations. Invitation To Comment Interested persons are invited to submit comments and recommendations regarding these proposed regulations. All comments submitted in response to these proposed regulations will be available for public inspection, during and after the comment period, in Room 600, 555 New Jersey Avenue, NW., Washington, DC, between the hours of 8:30 a.m. and 4 p.m., Monday through Friday of each week except Federal holidays. Assessment of Educational Impact The Secretary particularly requests comments on whether the proposed regulations in this document would require transmission of information that is being gathered by or is available from any other agency or authority of the United States. List of Subjects in 34 CFR Part 701 Education, Educational research, Reporting and recordkeeping requirements. Dated: May 22, 1996. (Catalog of Federal Domestic Assistance Number does not apply) Sharon P. Robinson, Assistant Secretary for Educational Research and Improvement. The Secretary proposes to amend Chapter VII of Title 34 of the Code of Federal Regulations by adding a new Part 701 to read as follows: [[Page 27992]] PART 701--STANDARDS FOR CONDUCT AND EVALUATION OF ACTIVITIES CARRIED OUT BY THE OFFICE OF EDUCATIONAL RESEARCH AND IMPROVEMENT (OERI)--DESIGNATION OF EXEMPLARY AND PROMISING PROGRAMS Subpart A--General Sec. 701.1 What is the purpose of these standards? 701.2 What definitions apply? 701.3 What entity is eligible to submit a program for review? 701.4 What must an entity submit for review? Subpart B--Selection of Panel Members 701.10 How are panels established? 701.11 Who may serve as a member of the standing group? 701.12 How is the membership of expert panels determined? Subpart C--The Expert Panel Review Process 701.20 How does an expert panel evaluate programs? 701.21 What is the difference between an exemplary and a promising program? 701.22 What criteria are used to evaluate programs for exemplary or promising designation? Authority: 20 U.S.C. 6011(i) Subpart A--General Sec. 701.1 What is the purpose of these standards? (a) The standards in this part implement section 941(d) of the Educational Research, Development, Dissemination, and Improvement Act of 1994. (b) These standards are intended to provide quality assurance that programs designated by the Department of Education as either exemplary or promising have met criteria that will allow educators, professional organizations, and others to use these programs with confidence. (Authority: 20 U.S.C. 6011(i)(2)(E)) Sec. 701.2 What definitions apply? Definition in the Educational Research, Development, Dissemination, and Improvement Act of 1994. The following term used in this part is defined in 20 U.S.C. 6041(a)(3): Educational program (Authority: 20 U.S.C. 6041(a)(3)) Sec. 701.3 What entity is eligible to submit a program for review? Any public or private agency, organization, or institution, or an individual, may submit an educational program for review. (Authority: 20 U.S.C. 6011(i)(2)(E)) Sec. 701.4 What must an entity submit for review? (a) To have its educational program considered for designation as exemplary or promising, the eligible entity must submit to the Secretary a description of the program and a discussion of the program's educational effectiveness, responsive to the criteria in Subpart C, Sec. 701.22. (b) Information submitted must include, to the extent relevant to the particular program-- (1) A program abstract of 250 words or less; (2) A description of the salient features of the program; (3) A description of the program's philosophy and history; (4) Site information, including demographics; (5) A description of evaluation results; (6) Funding and staffing information; and (7) Organization name, address, telephone and fax numbers, e-mail address (if available), and contact person. (Authority: 20 U.S.C. 6011(i)(2)(E)) Subpart B--Selection of Panel Members Sec. 701.10 How are panels established? (a) The Assistant Secretary, in consultation with the National Educational Research Policy and Priorities Board established under 20 U.S.C. 6021, establishes a standing group of educational experts. The Assistant Secretary may expand the membership of the standing group as necessary. (b) The Assistant Secretary selects members from the standing group, based on their areas of expertise, to serve on expert panels in specific topic areas for the purpose of reviewing and evaluating educational programs and recommending, to the Secretary, those programs that should be designated as exemplary or promising. (Authority: 20 U.S.C. 6011(i)(2)(E), 6041(d)) Sec. 701.11 Who may serve as a member of the standing group? An individual may serve as a member of the standing group for the purpose of reviewing and evaluating educational programs for exemplary or promising status if that individual possesses two or more of the following qualifications: (a) Demonstrated expertise and experience in one or more specific educational areas. (b) Demonstrated expertise and experience across a broad range of educational policies and practices. (c) Experience in evaluating educational programs. (d) Experience or expertise in developing educational products. (e) Current employment as a teacher, principal or other school- based or community-based professional (such as a guidance counselor, school media specialist, or health professional). (Authority: 20 U.S.C. 6011(i)(2)(E), 6041(d)) Sec. 701.12 How is the membership of expert panels determined? (a) For the review of each program, or group of programs, the Assistant Secretary establishes an expert panel comprised of individuals who are members of the standing group. (b) In establishing the membership of each expert panel, the Assistant Secretary-- (1) Selects individuals who have in-depth knowledge of the subject area or content of the program or group of programs to be evaluated; (2) Selects at least one current teacher, principal, or other school-based or community-based professional; (3) Ensures that no more than one-third of the panel members are employees of the Federal Government; and (4) Ensures that each panel member does not have a conflict of interest, as determined in accordance with paragraph (c) of this section, with respect to any educational program the panel member is asked to review. (c) Panel members are considered employees of the Department for the purposes of conflicts of interest analysis and are subject to the provisions of 18 U.S.C. 208, 5 CFR 2635.502, and the Department's policies used to implement those provisions. (Authority: 20 U.S.C. 6011(i)(2)(E), 6041(d)) Subpart C--The Expert Panel Review Process Sec. 701.20 How does an expert panel evaluate programs? (a) Each panel member shall-- (1) Independently review each program based on the criteria in Sec. 701.22; (2) Provide written comments based on an analysis of the strengths and weaknesses of the program according to the criteria; (3) Participate in site visits if appropriate; and (4) Participate in a meeting of the expert panel, if appropriate, to discuss the reviews. (b) A panel may not eliminate an educational program from consideration based solely on the fact that the program does not have one specific type of supporting data, such as test scores. [[Page 27993]] (c) Each expert panel shall make a recommendation to the Secretary as to whether the program is exemplary, promising, or neither. (Authority: 20 U.S.C.6011(i)(2)(E), 6041(d)) Sec. 701.21 What is the difference between an exemplary and a promising program? (a) A panel may recommend to the Secretary that a program be designated as promising if the panel determines that the program has met each of the criteria of educational effectiveness in Sec. 701.22 with respect to one context or one population. (b) A panel may recommend to the Secretary that a program be designated as exemplary if the panel determines that the program has met each of the criteria of educational effectiveness in Sec. 701.22 with respect to multiple contexts or multiple populations. (Authority: 20 U.S.C.6011(i)(2)(E), 6041(d)(2)) Sec. 701.22 What criteria are used to evaluate programs for exemplary or promising designation? In determining whether an educational program (``program'' includes educational polices, research findings, practices and products) should be recommended as exemplary, promising, or neither, each expert panel shall consider the following criteria of educational effectiveness: (a) Evidence of success. The expert panel considers-- (1) Whether, based on a range of assessments, information from users, or other indicators as appropriate, the program contributes to solving substantial or important problems in teaching or learning; and (2) The extent to which-- (i) Program effects are beneficial to the populations for whom the program was designed; or (ii) The product performs as expected for the educational consumers it was said to benefit. (b) Quality of the program. The expert panel considers-- (1) Whether the program has clear goals, is based on sound research and practice, and incorporates accurate and up-to-date content; (2) Whether the program represents a substantially improved alternative to existing options; (3) The extent to which the program promotes equity and is free of bias based on race, gender, age, culture, ethnic origin, disability, or limited English proficiency status; (4) Whether the program is based on high expectations for the success of all participants; (5) Whether the program is appropriate to the target audiences; and (6) The extent to which any materials associated with the program conform to accepted standards of technical quality. (c) Educational significance. The expert panel considers-- (1) The extent to which the program has the potential to increase knowledge or understanding of educational problems, and issues, or effective strategies for teaching or learning; and (2) Whether the program is described clearly enough so that it can be adapted or adopted in new sites. (d) Usefulness to others. The expert panel considers-- (1) Whether the cost of the program (including money, staff time, and other required resources) is reasonable in light of expected benefits and compared to other alternatives; and (2) Whether the program is available for use by others. (Authority: 20 U.S.C. 6011(i)(2)(E), 6041(d)(2)) [FR Doc. 96-13801 Filed 5-31-96; 8:45 am] BILLING CODE 4000-01-P