[Federal Register: September 16, 1998 (Volume 63, Number 179)] [Notices] [Page 49557-49560] From the Federal Register Online via GPO Access [wais.access.gpo.gov] [DOCID:fr16se98-47] ----------------------------------------------------------------------- DEPARTMENT OF EDUCATION National Assessment Governing Board AGENCY: National Assessment Governing Board; Department of Education. ACTION: Notice of information collection activity; request for comment. ----------------------------------------------------------------------- SUMMARY: In compliance with the Paperwork Reduction Act (44 U.S.C. 3501 et seq.), this notice announces a proposed information collection request (ICR) of the National Assessment Governing Board. The information collection is to conduct validity studies in conjunction with the pilot study of the proposed national tests in 4th grade [[Page 49558]] reading and 8th grade mathematics, in March 1999. Before submitting the ICR to the Office of Management and Budget (OMB), the Governing Board is soliciting comments on the information collection as described below. DATES: Comments must be submitted on or before November 16, 1998. ADDRESSES: Submit written comments identified by ``ICR: Voluntary National Test-Pilot Validity Studies'' by mail or in person addressed to Ray Fields, Assistant Director for Policy and Research, National Assessment Governing Board, Suite 825, 800 North Capitol Street, NW., Washington, DC 20002. Comments may be submitted electronically by sending electronic mail (e-mail) to Ray__Fields@ED.GOV. Comments sent by e-mail must be submitted as an ASCII file avoiding the use of special characters and any form of encryption. All written comments will be available for public inspection at the address given above from 8 a.m. to 4:30 p.m., Monday through Friday, excluding legal holidays. FOR FURTHER INFORMATION CONTACT: Ray Fields, Assistant Director for Policy and Research, National Assessment Governing Board, Suite 825, 800 North Capitol Street, NW., Washington, DC 20002, Telephone: (202) 357-0395, e-mail: Ray__Fields@ED.GOV. SUPPLEMENTARY INFORMATION: I. Information Collection Request The National Assessment Governing Board is seeking comments on the following Information Collection Request (ICR). Type of Review: New. Title: Validity Studies of the Voluntary National Tests in 4th Grade Reading and 8th Grade Mathematics. Affected Entities: Parties affected by this information collection are state, local, Tribal Government or non-public education agencies. Abstract: Pub. L. 105-78 vests exclusive authority to develop the voluntary national tests in the Governing Board and also prohibits the use of Fiscal Year 1998 funds for pilot testing, field testing, implementation, administration, or distribution of voluntary national tests. If Congress does not prohibit further development of the voluntary national tests after September 30, 1998, the Governing Board intends to begin pilot testing of items, (i.e. test questions) and conduct validity studies of test procedures in March 1999. Pub. L. 105-78 also requires the Governing Board to make four determinations about the voluntary national tests: (1) The extent to which test items selected for use on the tests are free from racial, cultural, or gender bias, (2) whether the test development process and test items adequately assess student reading and mathematics comprehension in the form most likely to yield accurate student achievement in reading and mathematics, (3) whether the test development process and test items take into account the needs of disadvantaged, limited English proficient, and disabled students, and (4) whether the test development process takes into account how parents, guardians, and students will appropriately be informed about testing content, purposes and uses. The purpose of the validity studies is to assess procedures for administering the proposed voluntary national tests in reading and mathematics. Since test administration can affect student performance, the validity studies determine if characteristics of test performance, such as non-standard conditions for students with special needs, affect student performance. Three studies are included in this information collection. Since the VNT is designed to be administered in two, 45- minute sessions, the Effect of Break Length between Testing Sessions validity study will examine this impact on examinee test scores and make recommendations for future administrations of the test. The Effects of Calculator Type validity study will investigate how student familiarity with a particular calculator affects test performance. Design specifications for the 8th grade mathematics test call for the use of a calculator in one testing session. The National Assessment Governing Board undertakes this study to inform policy, for proposed future field and operational tests, about whether to issue standard calculators or permit students to take the test with their own calculator. The Effects of Extended Time and Small Group Administration Accommodations validity study investigates non-standard test administration procedures for the inclusion of students with disabilities and students with limited English proficiency in the voluntary national tests. This study considers the testing accommodations of extended time (up to two times the standard administration test length) and small group administration for students with disabilities, students with limited English proficiency (LEP) and comparison groups of non-disabled, non-LEP students. This study will measure the magnitude of the effect of test accommodations on student performance. Effects of Break Length between Testing Sessions. The specifications for the VNT call for the test to be administered in two- 45 minute sessions given on the same day but do not dictate the specific scheduling of these two sessions. As school personnel will administer the operational VNT, it is likely that there will be some variation in test administration procedures at participating schools. We anticipate that most schools will opt for two morning sessions with a short break between, but some schools may have logistical reasons for administering the test in one morning and one afternoon session. Data will be collected from test administrators on the test administration schedule and activities that occurred during testing sessions. This validity study would describe break lengths and activities, investigate their effects on test performance, and would allow AIR to advise NAGB on scheduling options for the VNT field-test and operational administration. The Effects of Calculator Type validity study will use released NAEP questions for which national statistics are available, rather than test questions for the VNT pilot test, Eighteen 8th grade students would be recruited to participate in a ``think-aloud'' procedure. Students would take one group of mathematics questions using either their own or a standard issue calculator, followed by another group of questions using the opposite calculator. While answering these questions, examinees would respond to interviewer prompts about their thinking processes. Students would also complete a brief questionnaire about their calculator use in schools. Data would be analyzed to determine the ways in which calculator use and familiarity with calculator features affects student performance. The Effects of Extended Time and Small Group Administration Accommodations validity study will investigate the impact of accommodations on the test performance of special populations; it will be conducted in two parts. In the first part, 900 students at each grade level will be added as an augmentation to the original pilot sample (300 students with disabilities who would take the VNT either unaccommodated or with accommodations that do not require altered test formats (e.g., large print, oral presentation), 300 students with limited English proficiency, and 300 non-disabled, non-LEP students) would either take the VNT under the standard time condition (two 45- minute [[Page 49559]] sessions) or under an extended time accommodation (two sessions, each up to 90 minutes). Large schools and schools with large numbers of LEP students would be recruited to participate and would be assigned to one of the two conditions. Because of the small number of eligible students with disabilities in a given school, 110 schools (60 for 4th grade reading, 50 for 8th grade mathematics) will be recruited to participate in this study. For the second part, the accommodations of extended-time and small- group administration would both be considered for a sample of 750 students with disabilities eligible for small-group or extended time testing accommodations at each grade from schools participating in the pilot sample, both from students in classrooms that will be sampled from those schools for pilot study participation and from the students in the remaining classrooms in those schools. Thus, all students will be selected from schools selected to participate in the main pilot test and some of the students selected for participation in this study will also have been selected for inclusion in the main pilot test. Small group accommodations are often offered to students with disabilities but the manner of the accommodation varies. The small group accommodation could be a ``pull-out'' session in another classroom, or, due to space or staffing restrictions, the small group may be ``embedded'' in a larger setting (such as a library or cafeteria) where other activity is present. This study distinguishes between these two methods of providing small group accommodations for students with disabilities. A subsample of schools that are already participating in the main pilot VNT would be assigned to one of five conditions: (1) Standard time, small group ``pull-out'' administration, (2) extended time, small group ``pull-out'' administration (3) standard time, ``embedded'' small group administration, (4) extended time, ``embedded'' small group administration, (5) standard time, standard (large) group administration. All eligible students with disabilities in the school would then take the VNT under the prescribed condition. A total of 750 students with disabilities would be recruited at each grade level, resulting in 150 students in each of the five conditions mentioned above. Due to the small number of eligible students with disabilities in a given school, it is estimated that 290 schools (4th grade) and 267 schools (8th grade) would be needed to complete the sample size. Parental consent will be sought for all students selected to participate under conditions different from those identified in their IEPs. All students in both parts of this study who take a test under the extended time would, at the end of the first 45 minutes of a testing session, be asked to switch to a different color pencil. Students in this condition could then answer remaining items or return to skipped items, using the different color pencil, until they have completed the test or until the end of the extended time period. All students (all conditions) would be asked to complete a short questionnaire about the length of the test. Data from this study would be analyzed to determine which groups benefit from the accommodation of extended time, which method of small group administration maximizes performance for students with disabilities, and how much accommodations affect test performance. Analyses would also be done to determine if the current test administration procedures provide sufficient time for students to take the VNT. Recommendations will be made for providing appropriate accommodations for students with disabilities and students with limited English proficiency on future field tests and operational tests of the VNT. In order to ensure adequate control and proper identification of the booklets of test items, and conduct necessary analyses of the data that results from the information collection, the following background information will be collected on the cover of the booklets of test questions: student name, date of birth, race/ethnicity, and sex (all to be supplied by the student), and special education status, limited English proficiency status, disadvantaged status, test administration accommodations, and primary language (collected by the test administrator under contract). Although students will write their name on each booklet for identification purposes during the administration of the pilot test, the students' names will be removed from the booklet shortly after the pilot test. Student names will not be included in the database for analysis and will not leave the school building where pilot testing is taking place. Instead, a unique numeric or alphanumeric identifier will be assigned to each booklet for tracking and analysis purposes. No third party notification or public disclosure burden is associated with this collection. Burden Statement Effects of Break Length between Sessions: This study will not require any increase in burden for students above that required for the pilot study. School staff burden for this study is approximately 10 minutes per school (672 for reading and 372 for math) or 174 hours total. Effects of calculator use study: The respondent burden for this study is 40.5 hours, or 2.25 hours for each of the 18 students participating in the Cognitive Laboratory study. This estimate is based on 90 minutes of cognitive think aloud and 45 minutes of test administration procedures by research staff. Testing accommodations validity study: The annual burden respondent estimate is based on 90 minutes of testing and 30 minutes of test administration activities (e.g., delivering instructions, handing out and collecting booklets, and providing background information as described above) per student, or two hours per student, in the standard time condition. Students who take the test in the extended time condition have up to three and a half hours each: 30 minutes of test administration activities, and up to 180 minutes of testing time, although this may be less if students finish early. Five hundred seventy-three of the students participating in this study will not require any increase in testing burden above that required for the pilot study, 382 will require an increase of 1.5 hours of testing burden, 327 will be new students with a total testing burden of 2 hours, and 218 will be new students with a total testing burden of 3 hours. In addition, there is an additional 6 minutes of burden per each of the 3,300 students to answer questions about the length of the test. Total student burden for the 3,300 students participating in the 4th grade reading test and 8th grade mathematics test is 7,270 hours. School staff burden for this study, for both reading and mathematics tests, is 225 hours. This includes questions about classroom practices to be asked of school staff by test administrators, 5 minutes each for the 2,700 students with disabilities and students with limited English proficiency included in the study. There is no school staff burden for the 600 students without disabilities or limited English proficiency. Participation in the pilot test and these validity studies is voluntary. State, local, and non-public education agencies are not mandated or required to participate. [[Page 49560]] Summary The total number of students involved in the validity studies, above that which has already been requested in the pilot VNT collection, is 3,318 with a total burden of 7310.5 hours. Total school staff burden for these validity studies is 399 hours. II. Request for Comments The National Assessment Governing Board solicits comments to: (a) Evaluate whether the proposed collection of information is necessary for the proper performance of the functions of the Governing Board, including whether the information will have practical utility; (b) Evaluate the accuracy of the Governing Board's estimates of the burden of the proposed collection of information; (c) Enhance the quality, utility and clarity of the information to be collected; (d) Minimize the burden of the collection of the information on those who are to respond, including through the use of appropriate automated, mechanical or other technological collection techniques or other forms of information technology, e.g., permitting electronic submission of responses. III. Public Record A record has been established for this action. A public version of this record, including printed, paper versions of electronic comments, is available for inspection from 8 a.m. to 4:30 p.m., Monday through Friday, excluding legal holidays. The public record is located in Suite 825, 800 North Capitol Street, NW., Washington, DC 20002. Comments may be submitted electronically by sending electronic mail (e-mail) to Ray__Fields@ED.GOV. Comments sent by email must be submitted as an ASCII file avoiding the use of special characters and any form of encryption. The official record for this action, as well as public version, as described above will be kept in paper form. Accordingly, the National Assessment Governing Board will transfer all comments received electronically into printer, paper form as they are received and will place the paper copies in the official record which will also include all comments submitted directly in writing. The official record is the paper record maintained at the National Assessment Governing Board, Suite 825, 800 North Capitol Street, NW, Washington DC 20002. List of Subjects Pilot tests for the voluntary national tests in 4th grade reading and 8th grade mathematics, validity studies, and Information Collection Request. Dated: September 11, 1998. Roy Truby, Executive Director, National Assessment Governing Board. [FR Doc. 98-24849 Filed 9-15-98; 8:45 am] BILLING CODE 4000-01-M