Global Measures of
Disability
Statistics Canada's
experience so far...
Submitted to the First Meeting of the
Washington Group on Disability Measurement,
February 18-20, 2002
Renée Langlois
Housing, Family and Social Statistics
Division
Statistics Canada
Global Measures of
Disability:
Statistics Canada’s
experience so far…
Statistics
Canada has been involved in surveying persons with disabilities since the early
1980’s when a special parliamentary committee was formed to study issues
surrounding this population. The
Committee published its findings in a report entitled Obstacles in early 1981.
Among its 130 recommendations to the Parliament of Canada was a directive
that Statistics Canada “give a high priority to the development and
implementation of a long-term strategy which will generate comprehensive data
on disabled persons in Canada.”[1] In response, Statistics Canada initiated a
research program to build a national database on disability, which would
include all types of disabilities and provide data for all geographic areas in
the country.
Following
early trials at finding a definition of disability, Statistics Canada chose to
adopt the World Health Organization’s 1980 model. This definition was operationalized through the use of
twenty-five questions on Activities of Daily Living (ADL’s) developed by the OECD, used to screen respondents with
disabilities. The difficulties involved
in including all 25 screening questions on most social survey questionnaires
led to efforts to develop a disability filter
question, otherwise known as a global disability indicator. The purpose of the filter questions was to
identify the target population (the population reporting an activity limitation
to any of the 25 screening questions of the survey) through a set of reduced
questions. Accordingly, a two-question
set was developed and included in the 1986, and eventually 1991, census of
population questionnaires.
In
the intervening years between 1991, when the last post-censal disability survey
was conducted (1991 Health and Activity
Limitation Survey) and 2001 when the current post-censal survey (2001 Participation and Activity Limitation
Survey) was fielded, Statistics Canada redesigned its disability statistics
strategy along two components:
-
the
development and use of new filter questions to improve the identification of
persons with disabilities during the census with a view to select a sample for
a post-censal survey;
-
the
use of common survey items to identify persons with disabilities in a range of
social surveys, in order to broaden the scope of disability statistics.
Thus
began the development of a set of questions to serve as a “global measure of
disability”, or “global disability indicator”.
A global
disability indicator is a short set of questions used to identify respondents
in a general population survey or census who perceive themselves to have some
level of activity limitation/participation restriction related to long-term
health conditions. Many of the global
indicators currently in use have from one to four questions. The number of questions in use depends on
the questionnaire type (whether a census or survey), the amount of space or
interview time available, the amount of detail required and the data collection
method used.
Generally,
global disability indicators have one of the following objectives:
-
they
may serve a descriptive purpose: a useful tool to provide broad information
on the functional status of a given population. This is of most interest to health service provides, disability
and social policy analysts, demographers, the disability community,
researchers, etc. The brevity of the
question set can ensure that disability is included as a topic in social
surveys that may otherwise ignore it, and the availability of a common
indicator across all surveys expands the information available on the impact of
disability in many different domains.
In these cases, the indicator should be very clear about the definition
of disability it is using, and must be as precise as possible in identifying
the population of interest.
-
they
may serve as a screening
mechanism: some surveys use a short set
of questions to identify a "pool" of respondents who are highly
likely to have the characteristics of interest for follow-up surveys. In these cases, it is important for the
indicator to be as comprehensive as possible not to exclude persons in the
target population. In fact, it can be
overly broad, since the second phase of data collection should take care of
eliminating those respondents who don't present the characteristics of
interest.
Statistics
Canada’s primary objective was to develop a global disability indicator that
would act as a screening mechanism for a post-censal disability survey. The
primary focus of the post-censal survey is persons with “significant”
disabilities, i.e., persons reporting that their daily lives are affected by
activity limitations or social participation restrictions associated with a
health-related condition. Although
persons with milder levels of disability are also part of the target population
(they have their own policy implications), the target population is not seen as
including persons with very mild disability levels. Since the principal source of disability statistics is the
post-censal survey, it was important to develop a screening mechanism that
would identify the target population well, without excluding a significant
portion of it.
The
use of the global indicator as a descriptive item was a secondary, although not
negligible, objective. An important
component of the new disability statistics strategy was to expand the
availability of information on disabled persons by using common concepts and
survey items to define disability in other social surveys in Statistics Canada,
such as surveys on employment and income, health, children and youth and
Aboriginal peoples. Accordingly, the
new indicator being developed would serve two purposes: as a screening mechanism for the post-censal
survey, and as a global measure of disability in other surveys.
Given that the global disability indicator was to be used in many population-based surveys, a certain set of criteria was desired:
·
the
questions had to apply to the whole population - children, adults and the
elderly – residing in households.
·
the
set of questions had to be succinct enough to be inserted in a number of survey
instruments, whether social or general health surveys, and in the census long
form.
·
the
language used in the questions had to be clear enough to be used in
self-administered surveys, without intervention from an interviewer, but should
also be applicable in telephone and face-to-face interviews.
·
the
questions had to "make sense" to respondents; the objective of the
questions had to be easily grasped.
·
the
questions had to be broad enough to allow persons with all types and levels of
disability to be included (remembering that a primary objective was to develop
a survey methodology that would eliminate the use of a sample of respondents
reporting no limitations to the filter questions).
The development of a global disability indicator began in the fall of 1997 with a two-phase research project: qualitative research on the attributes of the questions themselves, and a quantitative phase to measure their performance.
Briefly, the chosen methodology consisted in the following main steps.
·
firstly,
gathering data on the comprehension and interpretation of the terms in the
filter questions used in 1986 and 1991 (for ease of comparison, these are
labeled “old filter questions”); this was achieved through one on one
interviews with persons with disabilities
·
secondly,
developing alternative sets and gathering data on comprehension and
interpretation of these
·
finally,
testing the old set against the new in terms of correlation of their answers
with the screening questions used in the 1991 disability survey.
The
qualitative phase of the research project revealed many interesting findings
about the perception of respondents regarding their activity limitations and
the terms of the filter questions they were asked to review. The linguistic analysis of the questions revealed
that a major cause of non-reporting of disability to the old filter questions
was the use of negative sounding terminology.
Terms such as “long-term”, “disabilities”, “handicaps” were viewed as
extremely negative and suggesting a high level of severity. In particular, the terms “disabilities” and
“handicaps” were seen as not applicable to many respondents. The term “disabilities” was viewed as
difficult to define, and consequently, difficult to apply to oneself and the
term “handicap” was viewed as antiquated and not politically correct. This led many respondents with milder
activity limitations to exclude themselves from the answers.
Another
highly problematic phrase was “is this person limited in…”. Many respondents disagreed that a health
problem or condition was limiting them;
rather, their activities were limited.
“Being limited” was viewed as pejorative and indicative of quite severe
disabilities. In other words, having an
activity limitation is very different from being limited. This was true for all types of respondents,
but especially so for parents of children with disabilities who resented the
implicit labeling of the term “limited”.
The
two-item answer categories to the questions were another source of
non-reporting. The choice between a
“yes” and a “no” was not always easy or clear for many respondents. Some reasons included:
-
situations
of mild or cyclical disabilities, which vary in frequency and intensity
-
variable
levels of restriction associated with some conditions
-
the
variety of environments (some facilitating, some neutral and some presenting
barriers) in which persons find themselves
Drawing
on these findings, a number of alternatives were tested during this phase of
the project. A final question set was
selected and quantitatively tested in a simulated post-censal survey
setting. This phase began with the
National Census Test is October 1998 and culminated with the 2001 post-censal
survey pilot test. Information on the
test results can be found in the documents listed as references.
The
questions to be used as filter questions in the 2001 census and as global
measures of disability in other surveys, read as follows:
1. Does this person have any difficulty hearing, seeing, communicating, walking, climbing stairs, bending, learning or doing any similar activities?
□ Yes,
sometimes
□ Yes,
often
□ No
2.
Does
a physical condition or mental condition or health problem reduce the amount of
the kind of activity this person can do:
(a)
At
home?
□ Yes, sometimes
□ Yes, often
□ No
(b)
At
work or at school?
□ Yes, sometimes
□ Yes, often
□ No
□ Not applicable
(c)
In
other activities, for example, transportation or leisure?
□ Yes, sometimes
□ Yes, often
□ No
Compared
to the previously used filter questions, this version has eliminated negative
or severe sounding terms, and has shifted the limitation to the activity rather
than to the person. In fact, the term
“limit” has been replaced by “reduce”.
The answer categories have been broadened and use a multiple-item
response scale, to allow the possibility of reporting limitations that are not
constant but variable depending on any number of factors. Finally, the questions on disabilities and
handicaps have been replaced by a question on difficulties associated with
certain activities of daily living (in fact, a shortened version of the ADL’s
in the screening questions), something that was much more easily understood by
respondents.
Looking
at the language of these questions, it is clear that their focus is on activity
limitations and participation restrictions related to health conditions. The aim of the questions is not to produce a
prevalence rate of impairments or conditions, but rather to provide an estimate
of persons experiencing limitations in their daily activities or restrictions
in their participation in various settings.
The domains listed cover most of the domains of the ICF. The domains of personal care and
interpersonal relationships were not mentioned specifically in the questions as
they were viewed as sensitive issues ant not appropriate to include in a census
context. They can be handled quite
well, however, in a disability or health survey. This situation illustrates some of the constraints of a global
measure: it is difficult to cover all
domains because of the necessary brevity of the question and some domains are
simply not appropriate to include in some surveying contexts.
The
term “difficulty” in the first question is broad in its applicability; although
the term isn’t defined for respondents, cognitive testing has shown that it is
interpreted as covering the components of quality, quantity, time required and
assistance required to perform the tasks or actions mentioned. In our experience, the notion of difficulty
is confusing in two cases especially.
Firstly, persons with very mild limitations are sometimes unsure as to
where to draw the line between a “real difficulty” and normal change associated
with aging. However, as the severity of
the difficulty increases, the uncertainty diminishes.
Secondly,
qualifying the abilities (or limitations) of children in various domains is a
difficult task for parents since children are in a constantly changing state of
learning and acquiring skills. The huge
amount of variation in the acquisition of skills during childhood renders it
difficult to ascertain whether a difficulty is a “normal” or not situation. Generally, parents will wait to have a
professional diagnosis of a health-related condition before feeling confident
enough to report an activity limitation; this can take many months or even
years to occur. This will result in
under-reporting of “borderline” of developing disabilities. On the other hand, some parents do report
difficulties in the specified domains because their child is only beginning to
acquire the skills involved. This
results in reporting situations that are not part of the target of the
questions. In conclusion, then, it is
difficult to capture disability in children with the same global measure used
for adults. In the Canadian context,
the census constraints mean that only one set of questions can be included on
the census questionnaire; therefore the same global questions must be used for
adults and children. Further analysis
of results of the post-censal child disability survey will enable us to
determine whether the approach is valid enough to pursue for this segment of
the population.
The
issue of assistance is somewhat problematic.
Our cognitive testing has shown that most respondents phrase their
answers to the global questions by taking into account the level of assistance
(whether personal or technical) they require to perform certain tasks. However, the impact of the assistance goes
both ways. For instance, persons
wearing glasses generally report no difficulty in seeing because they assess
their capacity in their usual state, that is with glasses (unless their
impairment is not corrected by glasses and limitations still occur). The same holds for persons using hearing
aids. On the other hand, persons
requiring assistance, especially personal assistance, of a less common nature
tend to report having difficulties or activity limitations in spite of having the assistance.
This results mostly from two factors.
Firstly, it seems that assistance rarely eliminates completely the
activity limitation. Secondly,
respondents realize that assistance varies in different environments; it can be
lost due to circumstances beyond the recipient’s control (changes in
governmental social policies, for example).
Again, the brevity of global indicators makes it difficult to tease out
these differences. This is an area that
would benefit from further research.
Finally,
the issue of the environment is not included specifically in the Statistics
Canada global disability questions (as evidenced by the paragraph above on the
impact of technical aids and assistance).
However, the multiple item response categories are useful to respondents
whose activity limitations or participation restrictions vary according to the
level of accommodation in various environments. However, whereas this simplifies the task of respondents to
indicate some level of limitation, it does not provide data users with any
indication on the impact of environmental facilitators or barriers. This is another area where global measures
would benefit from further research.
The
current survey on disability, the Participation
and Activity Limitation Survey (PALS), was fielded in the fall of
2001. The population was defined by
using the new filter questions on the census form and selecting only a sample
of respondents indicating at least one YES to the filter questions. The PALS interview began with the same
filter questions as appeared on the census form followed by a series of
screening questions. These screening
questions contain a series of ADL’s, as well as questions on activity
limitations and participation restrictions due to health-related conditions.
The 2001 strategy to define the population with
disabilities will also involve using a consistent conceptual approach. In order to broaden the scope of the
disability database at Statistics Canada, the new filter questions will be used
in other Statistics Canada social surveys that require a global disability
indicator, such as labour, health, education and Aboriginal peoples
surveys. This strategy of using
consistent definitions (i.e., common questions to identify the presence of a
disability) will ensure the development of a comprehensive database on persons
with disabilities. Although it is
recognized that resulting disability rates will vary depending on collection
methods, methodology and survey context, the consistent use of common
disability filter questions, or of a global disability indicator, will enhance
the analysis of the resulting data.
Preliminary results from some surveys indeed show remarkable consistency
in the rates being reported. For
instance, where the disability rate for Phase 1 of the PALS pilot test was
19.4%, the same filter questions used by Cycles 13 and 14 of the General Social
Survey show results of 20% and 21%. The
results of the 2001 PALS will provide guidelines for the interpretation of the
data coming out of the various surveys by developing algorithms to estimate
severity, for example. Various data
collection activities are planned over the next few years that will include the
global disability indicator, and a body of data will soon be available for
analysis.
APPENDIX A
1986 and 1991
Disability Filter Questions
1. Is this person limited in the kind or amount of activity that he/she can do because of a long-term physical condition, mental condition or health problem:
(a)
At
home?
Response
categories: No, not limited
Yes,
limited
(b)
At
school or at work?
Response
categories: No, not limited
Yes,
limited
Not
applicable
(c)
In
other activities, e.g., transportation to or from work, leisure time
activities?
Response
categories: No, not limited
Yes
limited
2.
Does
this person have any long-term disabilities or handicaps?
Response
categories: No
Yes
REFERENCES
Gravel, Ronald. Évaluation des questions filtres de l’Enquête sur la santé et les limitations d’activités (ESLA). Internal Report, July 1998.
Houle,
Anne-Marie. Analyses of the PALS Pilot
Test: Disability Rates and Comparison of False Negative and True Positive
Respondents. Internal Report, March
2001.
Langlet,
Éric. Comparison of Screening
Strategies for the Health and Activity Limitation Survey. Proceedings of the Section of Survey Research
Methods, Vol. II, American Statistical Association, 560-565.
Langlois,
Renée. The Development of a Global
Disability Indicator: Recent
Developments at Statistics Canada.
Proceedings of the International Seminar on Measurement of Disability,
June 2001
Note: copies of Statistics Canada internal reports are available by contacting Renée Langlois at Statistics Canada (telephone: 613-951-0878; fax: 613-951-4378; e-mail: renee.langlois@statcan.ca).
[1] Obstacles, Report of the Special
Parliamentary Committee on the Disabled and the Handicapped, Ottawa, February
1981, page 131