Archived: Data From National Assessments

A r c h i v e d I n f o r m a t i o n

To Assure the Free Appropriate Public Education of all Children with Disabilities - 1995

Data From National Assessments

Prior to the 1990s, people with disabilities participated in national assessments at extremely low rates, and data were seldom reported for those who did participate. Now, however, people with disabilities are being included in assessments at an increasing rate. The 1992 National Adult Literacy Survey (NALS) included people with disabilities, and even though methodological inadequacies may have affected the reliability of the results for those people, it is notable that the attempt was made. In addition, during 1993-1994, the National Center for Education Statistics (NCES) began making significant planning efforts to provide accommodations for and include students with disabilities in such assessments as the National Assessment of Educational Progress (NAEP). This section describes the NALS study and the plans for administering the Early Childhood Longitudinal Study (ECLS).

National Adult Literacy Survey (NALS)

In 1988, Congress asked the Department of Education to conduct a national literacy study of American adults. As a result, NALS was conducted in 1992. The instrument was a national household survey, and 13,600 individuals age 16 and older participated. Approximately 12 percent of the respondents indicated, through self-reporting, that they had a physical, mental, or other health condition that kept them from participating fully in work or other activities. No accommodations were provided to people who identified themselves as having a disability.

The NALS study defined literacy as "using printed and written information to function in society, to achieve one's goals, and to develop one's knowledge and potential" (Kirsch, Jungeblut, Jenkins, & Kolstad, 1993, p. 2). Three scales were identified:

Prose literacy -- the knowledge and skills needed to understand and use information from texts that include editorials, news stories, poems, and fiction. For example, finding a piece of information in a newspaper article, interpreting instructions from a warranty, inferring a theme from a poem, or contrasting views expressed in an editorial.

Document literacy -- the knowledge and skills required to locate and use information contained in materials that include job applications, payroll forms, transportation schedules, maps, tables, and graphs; for example, locating a particular intersection on a street map, using a schedule to choose the appropriate bus, or entering information on an application form.

Quantitative literacy -- the knowledge and skills required to apply arithmetic operations, either alone or sequentially, using numbers embedded in printed materials; for example, balancing a checkbook, figuring out a tip, completing an order form, or determining the amount of interest from a loan advertisement.(pp. 3-4)

These scales were the basis for the 165 tasks in the total survey. Each respondent was asked during a personal interview to complete one booklet containing tasks estimated to require a total of 45 minutes to complete. Each participant was asked to complete a number of tasks related to each literacy area, thus completing a subset of the total set of literacy tasks. Sampling procedures were used to ensure that all tasks were administered to a nationally representative sample. During the part of the interview in which background and personal information were obtained, respondents were also asked to describe any illnesses and disabilities. Four questions were used to identify individuals with "physical, mental, or other health conditions":

One question asked respondents whether they had a physical, mental, or other health condition that kept them from participating fully in work, school, housework, or other activities. Two other questions asked whether they had visual or hearing difficulties. Finally, respondents were asked whether they had a learning disability, any mental or emotional condition, mental retardation, a speech disability, a physical disability, a long-term illness (for six months or more), or any other health impairment. Respondents were permitted to report each type of disability or condition they had. (Kirsch, Jungeblut, Jenkins, Kolstad, 1993, p. 42.)

The NALS report included the results of the literacy assessment of individuals in ten self-reported disability condition categories. The following percentages of respondents reported that they had various conditions: physical disability (9 percent), long-term illness (8 percent), visual difficulty (7 percent), hearing difficulty (7 percent), other health impairment (6 percent), learning disability (3 percent), mental or emotional condition (2 percent), speech disability (1 percent), and mental retardation (< 1 percent). Overall, these individuals with disabilities were more likely than people without disabilities who participated in the survey to perform at the lowest literacy levels. The first NALS report described this lower performance of individuals with various disabilities and conditions in the excerpts below.

Adults with mental retardation, for example, were about four times more likely than their peers in the total population to perform in Level 1¹ on the prose, document, and quantitative scales. On the prose scale, 87 percent of the respondents with mental retardation were in this level, compared with 21 percent of adults in the population as a whole. (Kirsch, Jungeblut, Jenkins, Kolstad, 1993, p. 43)
On each scale, more than half of the individuals with vision difficulties performed in Level 1 (53 to 55 percent), for example, and another 24 to 26 percent performed in Level 2. (Kirsch, Jungeblut, Jenkins, Kolstad, 1993, p. 45)
Finally, it is interesting to note the average performance differences between individuals who reported having certain disabilities and adults in the population as a whole. The smallest gap was between those who said they had difficulty hearing and adults in the population overall; the difference was 24 to 29 points on each literacy scale. Across the other groups, the performance gap between those who reported having a particular disability or illness and those in the total population ranged from 32 to 72 points. The only exception was among adults who reported having some form of mental retardation; here the gap ranged from 120 to 154 points across the scales. (Kirsch, Jungeblut, Jenkins, Kolstad, 1993, p. 45).

However, Kirsch et al., also reported that within nearly every disability group, in each literacy category (prose, document, quantitative) there are some individuals with disabilities who perform at levels 4 and 5, the top two levels of literacy reached by about 20 percent of the 13,600 individuals in the total sample. The percentages of individuals in these two levels within each condition or disability are shown in table 4.6.

TABLE 4.6 Percentage of Adults with Disabilities and Other Conditions Performing in Levels 4 and 5

                                             Literacy Scale Disability/Condition                 Prose   Document  Quantitative
------------------------------------------------------------------------ Physical, mental, or other health       6        6           7  Visual difficulty                       5        6           6  Hearing difficulty                     10        9          13  Learning disability                     5        5           5  Mental/emotional condition             10       10          10  Mental retardation                      4        3           1  Speech disability                       7        6           7  Physical disability                     7        6           8  Long-term illness                       8        7          10  Other health impairment                 8        7          11

Source: Data are from Kirsch, Jungeblut, Jenkins, & Kolstad (1993, p. 44).

The NALS report data are among the first available from a national education survey that includes data about people with disabilities. Unfortunately, the data related to people with disabilities have some major limitations. First, relying on self-reports can result in underestimation of the incidence of certain disabilities in the sample. For example, learning disabilities would probably be underreported by adults who left the school system before schools became widely aware of such disabilities and how to diagnose them. Also, the stigma sometimes still associated with disabilities such as mental retardation or emotional disturbance may discourage respondents from reporting the condition. Second, self-reports can also result in overestimation of some disabilities. Third, the lack of accommodations that might have enabled some people to better accomplish some assessment tasks may also have affected the results. For example, people with visual difficulties might have been at a disadvantage because print may not have been large enough for them to see accurately.

Plans for Future Assessments (ECLS)

NCES has also begun work on a new data collection effort, the Early Childhood Longitudinal Study (ECLS). A kindergarten cohort is to be tracked for five years. A birth cohort might also be tracked if other agencies wish to participate. Various ways for including students with disabilities in this study are being explored as the study is designed and the instruments are developed. This approach to achieving inclusion is consistent with that recommended by participants in a meeting of NCES personnel, NCEO, and other experts in assessment and national data collection programs (Ysseldyke, Thurlow, McGrew, & Vanderwood, 1994).

¹ Performance was based on 5 levels. Level 1 tasks involve matching or entering information onto a document while Level 5 tasks require the reader to extract information from a complex display or to infer information.
-###-

[Data from State Assessment]

[Summary and Implications]