[Federal Register: September 16, 1998 (Volume 63, Number 179)]
[Notices]
[Page 49557-49560]
From the Federal Register Online via GPO Access [wais.access.gpo.gov]
[DOCID:fr16se98-47]
-----------------------------------------------------------------------
DEPARTMENT OF EDUCATION
National Assessment Governing Board
AGENCY: National Assessment Governing Board; Department of Education.
ACTION: Notice of information collection activity; request for comment.
-----------------------------------------------------------------------
SUMMARY: In compliance with the Paperwork Reduction Act (44 U.S.C. 3501
et seq.), this notice announces a proposed information collection
request (ICR) of the National Assessment Governing Board. The
information collection is to conduct validity studies in conjunction
with the pilot study of the proposed national tests in 4th grade
[[Page 49558]]
reading and 8th grade mathematics, in March 1999. Before submitting the
ICR to the Office of Management and Budget (OMB), the Governing Board
is soliciting comments on the information collection as described
below.
DATES: Comments must be submitted on or before November 16, 1998.
ADDRESSES: Submit written comments identified by ``ICR: Voluntary
National Test-Pilot Validity Studies'' by mail or in person addressed
to Ray Fields, Assistant Director for Policy and Research, National
Assessment Governing Board, Suite 825, 800 North Capitol Street, NW.,
Washington, DC 20002. Comments may be submitted electronically by
sending electronic mail (e-mail) to Ray__Fields@ED.GOV. Comments sent
by e-mail must be submitted as an ASCII file avoiding the use of
special characters and any form of encryption.
All written comments will be available for public inspection at the
address given above from 8 a.m. to 4:30 p.m., Monday through Friday,
excluding legal holidays.
FOR FURTHER INFORMATION CONTACT:
Ray Fields, Assistant Director for Policy and Research, National
Assessment Governing Board, Suite 825, 800 North Capitol Street, NW.,
Washington, DC 20002, Telephone: (202) 357-0395, e-mail:
Ray__Fields@ED.GOV.
SUPPLEMENTARY INFORMATION:
I. Information Collection Request
The National Assessment Governing Board is seeking comments on the
following Information Collection Request (ICR).
Type of Review: New.
Title: Validity Studies of the Voluntary National Tests in 4th
Grade Reading and 8th Grade Mathematics.
Affected Entities: Parties affected by this information collection
are state, local, Tribal Government or non-public education agencies.
Abstract: Pub. L. 105-78 vests exclusive authority to develop the
voluntary national tests in the Governing Board and also prohibits the
use of Fiscal Year 1998 funds for pilot testing, field testing,
implementation, administration, or distribution of voluntary national
tests. If Congress does not prohibit further development of the
voluntary national tests after September 30, 1998, the Governing Board
intends to begin pilot testing of items, (i.e. test questions) and
conduct validity studies of test procedures in March 1999.
Pub. L. 105-78 also requires the Governing Board to make four
determinations about the voluntary national tests: (1) The extent to
which test items selected for use on the tests are free from racial,
cultural, or gender bias, (2) whether the test development process and
test items adequately assess student reading and mathematics
comprehension in the form most likely to yield accurate student
achievement in reading and mathematics, (3) whether the test
development process and test items take into account the needs of
disadvantaged, limited English proficient, and disabled students, and
(4) whether the test development process takes into account how
parents, guardians, and students will appropriately be informed about
testing content, purposes and uses.
The purpose of the validity studies is to assess procedures for
administering the proposed voluntary national tests in reading and
mathematics. Since test administration can affect student performance,
the validity studies determine if characteristics of test performance,
such as non-standard conditions for students with special needs, affect
student performance. Three studies are included in this information
collection. Since the VNT is designed to be administered in two, 45-
minute sessions, the Effect of Break Length between Testing Sessions
validity study will examine this impact on examinee test scores and
make recommendations for future administrations of the test. The
Effects of Calculator Type validity study will investigate how student
familiarity with a particular calculator affects test performance.
Design specifications for the 8th grade mathematics test call for the
use of a calculator in one testing session. The National Assessment
Governing Board undertakes this study to inform policy, for proposed
future field and operational tests, about whether to issue standard
calculators or permit students to take the test with their own
calculator. The Effects of Extended Time and Small Group Administration
Accommodations validity study investigates non-standard test
administration procedures for the inclusion of students with
disabilities and students with limited English proficiency in the
voluntary national tests. This study considers the testing
accommodations of extended time (up to two times the standard
administration test length) and small group administration for students
with disabilities, students with limited English proficiency (LEP) and
comparison groups of non-disabled, non-LEP students. This study will
measure the magnitude of the effect of test accommodations on student
performance.
Effects of Break Length between Testing Sessions. The
specifications for the VNT call for the test to be administered in two-
45 minute sessions given on the same day but do not dictate the
specific scheduling of these two sessions. As school personnel will
administer the operational VNT, it is likely that there will be some
variation in test administration procedures at participating schools.
We anticipate that most schools will opt for two morning sessions with
a short break between, but some schools may have logistical reasons for
administering the test in one morning and one afternoon session. Data
will be collected from test administrators on the test administration
schedule and activities that occurred during testing sessions. This
validity study would describe break lengths and activities, investigate
their effects on test performance, and would allow AIR to advise NAGB
on scheduling options for the VNT field-test and operational
administration.
The Effects of Calculator Type validity study will use released
NAEP questions for which national statistics are available, rather than
test questions for the VNT pilot test, Eighteen 8th grade students
would be recruited to participate in a ``think-aloud'' procedure.
Students would take one group of mathematics questions using either
their own or a standard issue calculator, followed by another group of
questions using the opposite calculator. While answering these
questions, examinees would respond to interviewer prompts about their
thinking processes. Students would also complete a brief questionnaire
about their calculator use in schools. Data would be analyzed to
determine the ways in which calculator use and familiarity with
calculator features affects student performance.
The Effects of Extended Time and Small Group Administration
Accommodations validity study will investigate the impact of
accommodations on the test performance of special populations; it will
be conducted in two parts. In the first part, 900 students at each
grade level will be added as an augmentation to the original pilot
sample (300 students with disabilities who would take the VNT either
unaccommodated or with accommodations that do not require altered test
formats (e.g., large print, oral presentation), 300 students with
limited English proficiency, and 300 non-disabled, non-LEP students)
would either take the VNT under the standard time condition (two 45-
minute
[[Page 49559]]
sessions) or under an extended time accommodation (two sessions, each
up to 90 minutes). Large schools and schools with large numbers of LEP
students would be recruited to participate and would be assigned to one
of the two conditions. Because of the small number of eligible students
with disabilities in a given school, 110 schools (60 for 4th grade
reading, 50 for 8th grade mathematics) will be recruited to participate
in this study.
For the second part, the accommodations of extended-time and small-
group administration would both be considered for a sample of 750
students with disabilities eligible for small-group or extended time
testing accommodations at each grade from schools participating in the
pilot sample, both from students in classrooms that will be sampled
from those schools for pilot study participation and from the students
in the remaining classrooms in those schools. Thus, all students will
be selected from schools selected to participate in the main pilot test
and some of the students selected for participation in this study will
also have been selected for inclusion in the main pilot test.
Small group accommodations are often offered to students with
disabilities but the manner of the accommodation varies. The small
group accommodation could be a ``pull-out'' session in another
classroom, or, due to space or staffing restrictions, the small group
may be ``embedded'' in a larger setting (such as a library or
cafeteria) where other activity is present. This study distinguishes
between these two methods of providing small group accommodations for
students with disabilities. A subsample of schools that are already
participating in the main pilot VNT would be assigned to one of five
conditions: (1) Standard time, small group ``pull-out'' administration,
(2) extended time, small group ``pull-out'' administration (3) standard
time, ``embedded'' small group administration, (4) extended time,
``embedded'' small group administration, (5) standard time, standard
(large) group administration. All eligible students with disabilities
in the school would then take the VNT under the prescribed condition. A
total of 750 students with disabilities would be recruited at each
grade level, resulting in 150 students in each of the five conditions
mentioned above. Due to the small number of eligible students with
disabilities in a given school, it is estimated that 290 schools (4th
grade) and 267 schools (8th grade) would be needed to complete the
sample size. Parental consent will be sought for all students selected
to participate under conditions different from those identified in
their IEPs.
All students in both parts of this study who take a test under the
extended time would, at the end of the first 45 minutes of a testing
session, be asked to switch to a different color pencil. Students in
this condition could then answer remaining items or return to skipped
items, using the different color pencil, until they have completed the
test or until the end of the extended time period. All students (all
conditions) would be asked to complete a short questionnaire about the
length of the test.
Data from this study would be analyzed to determine which groups
benefit from the accommodation of extended time, which method of small
group administration maximizes performance for students with
disabilities, and how much accommodations affect test performance.
Analyses would also be done to determine if the current test
administration procedures provide sufficient time for students to take
the VNT. Recommendations will be made for providing appropriate
accommodations for students with disabilities and students with limited
English proficiency on future field tests and operational tests of the
VNT.
In order to ensure adequate control and proper identification of
the booklets of test items, and conduct necessary analyses of the data
that results from the information collection, the following background
information will be collected on the cover of the booklets of test
questions: student name, date of birth, race/ethnicity, and sex (all to
be supplied by the student), and special education status, limited
English proficiency status, disadvantaged status, test administration
accommodations, and primary language (collected by the test
administrator under contract). Although students will write their name
on each booklet for identification purposes during the administration
of the pilot test, the students' names will be removed from the booklet
shortly after the pilot test. Student names will not be included in the
database for analysis and will not leave the school building where
pilot testing is taking place. Instead, a unique numeric or
alphanumeric identifier will be assigned to each booklet for tracking
and analysis purposes. No third party notification or public disclosure
burden is associated with this collection.
Burden Statement
Effects of Break Length between Sessions: This study will not
require any increase in burden for students above that required for the
pilot study. School staff burden for this study is approximately 10
minutes per school (672 for reading and 372 for math) or 174 hours
total.
Effects of calculator use study: The respondent burden for this
study is 40.5 hours, or 2.25 hours for each of the 18 students
participating in the Cognitive Laboratory study. This estimate is based
on 90 minutes of cognitive think aloud and 45 minutes of test
administration procedures by research staff.
Testing accommodations validity study: The annual burden respondent
estimate is based on 90 minutes of testing and 30 minutes of test
administration activities (e.g., delivering instructions, handing out
and collecting booklets, and providing background information as
described above) per student, or two hours per student, in the standard
time condition. Students who take the test in the extended time
condition have up to three and a half hours each: 30 minutes of test
administration activities, and up to 180 minutes of testing time,
although this may be less if students finish early. Five hundred
seventy-three of the students participating in this study will not
require any increase in testing burden above that required for the
pilot study, 382 will require an increase of 1.5 hours of testing
burden, 327 will be new students with a total testing burden of 2
hours, and 218 will be new students with a total testing burden of 3
hours. In addition, there is an additional 6 minutes of burden per each
of the 3,300 students to answer questions about the length of the test.
Total student burden for the 3,300 students participating in the 4th
grade reading test and 8th grade mathematics test is 7,270 hours.
School staff burden for this study, for both reading and mathematics
tests, is 225 hours. This includes questions about classroom practices
to be asked of school staff by test administrators, 5 minutes each for
the 2,700 students with disabilities and students with limited English
proficiency included in the study. There is no school staff burden for
the 600 students without disabilities or limited English proficiency.
Participation in the pilot test and these validity studies is
voluntary. State, local, and non-public education agencies are not
mandated or required to participate.
[[Page 49560]]
Summary
The total number of students involved in the validity studies,
above that which has already been requested in the pilot VNT
collection, is 3,318 with a total burden of 7310.5 hours. Total school
staff burden for these validity studies is 399 hours.
II. Request for Comments
The National Assessment Governing Board solicits comments to:
(a) Evaluate whether the proposed collection of information is
necessary for the proper performance of the functions of the Governing
Board, including whether the information will have practical utility;
(b) Evaluate the accuracy of the Governing Board's estimates of the
burden of the proposed collection of information;
(c) Enhance the quality, utility and clarity of the information to
be collected;
(d) Minimize the burden of the collection of the information on
those who are to respond, including through the use of appropriate
automated, mechanical or other technological collection techniques or
other forms of information technology, e.g., permitting electronic
submission of responses.
III. Public Record
A record has been established for this action. A public version of
this record, including printed, paper versions of electronic comments,
is available for inspection from 8 a.m. to 4:30 p.m., Monday through
Friday, excluding legal holidays. The public record is located in Suite
825, 800 North Capitol Street, NW., Washington, DC 20002. Comments may
be submitted electronically by sending electronic mail (e-mail) to
Ray__Fields@ED.GOV. Comments sent by email must be submitted as an
ASCII file avoiding the use of special characters and any form of
encryption.
The official record for this action, as well as public version, as
described above will be kept in paper form. Accordingly, the National
Assessment Governing Board will transfer all comments received
electronically into printer, paper form as they are received and will
place the paper copies in the official record which will also include
all comments submitted directly in writing. The official record is the
paper record maintained at the National Assessment Governing Board,
Suite 825, 800 North Capitol Street, NW, Washington DC 20002.
List of Subjects
Pilot tests for the voluntary national tests in 4th grade reading
and 8th grade mathematics, validity studies, and Information Collection
Request.
Dated: September 11, 1998.
Roy Truby,
Executive Director, National Assessment Governing Board.
[FR Doc. 98-24849 Filed 9-15-98; 8:45 am]
BILLING CODE 4000-01-M