Washington Group on Disability Statistics

Seventh Meeting -- Executive Summary

PDF Version 75KB

Report of the Washington Group (WG) on Disability Statistics:
Executive Summary of the 7th Annual Meeting

Purpose:

The main purpose of the WG is the promotion and co-ordination of international co-operation in the area of health statistics by focusing on disability measures suitable for censuses and national surveys. The aim is to provide basic necessary information on disability which is comparable throughout the world. More specifically, the WG aims to guide the development of a short set(s) of disability measures, suitable for use in censuses, sample-based national surveys, or other statistical formats, for the primary purpose of informing policy on equalization of opportunities. The second priority of the Washington Group is to recommend one or more extended sets of survey items to measure disability, or guidelines for their design, to be used as components of population surveys or as supplements to specialty surveys. These extended sets of survey items are intended to be related to the short set(s) of disability measures. The WHO International Classification of Functioning, Disability and Health (ICF) has been accepted as the basic framework for the development of the sets. All disability measures recommended by the group, short or extended, will be accompanied by descriptions of their technical properties and methodological guidance will be given on their implementation and their applicability to all sections of the population. The WG will disseminate work products globally through the world-wide web and scientific presentations.

Year Organized: 2001

Participants:

Representatives of national statistical offices, international organizations, organizations representing persons with disabilities, and other non-government organizations have participated in the last 6 meetings.

Current country representatives include (from national statistical offices):
 

 Albania, Argentina, Australia, Austria, Armenia, Barbados, Belgium, Bermuda, Bolivia, Brazil, Cambodia, Canada, Chile, China (Hong Kong Special Administrative Region, Macao Special Administrative Region, and Mainland), Columbia, Cuba, Czech Republic, Democratic Republic of Congo, Denmark, Egypt, Finland, France, Gambia, Ghana, Greece, Guatemala, Hungary, India, Iran, Iraq, Ireland, Israel, Italy, Ivory Coast, Japan, Jordan, Kenya, Latvia, Lebanese Republic, Lesotho, Lithuania, Malawi, Mauritius, Mexico, Micronesia, Mongolia, New Zealand, Norway, Occupied Palestinian Territory, Panama, Paraguay, Peru, Philippines, Poland, Romania, Serbia and Montenegro, Sierra Leone, Slovenia, South Africa, Spain, Saint Lucia, Sweden, Syria, Tanzania, Thailand, The Netherlands, Turkey, Tonga, Trinidad, Uganda, United Kingdom of Great Britain and Northern Ireland, United States of America, Venezuela, Viet Nam, Zambia, and Zimbabwe.

Current non-government organizations include:

European Disability Forum, Rehabilitation International, Inter-American Institute on Disability, EUROSTAT, International Labor Organization, Organization for Economic Cooperation and Development, National Disability Authority-Ireland, Inter-American Development Bank, International Development Project, World Bank, World Health Organization, United Nations Economic and Social Commission for Asia and the Pacific, United Nations Economic and Social Commission for Western Asia, United Nations Economic Commission of Europe, and United Nations Statistics Division.

Governmental Organizations of Persons with Disabilities:

 Coordenadoria Nacional para Integracao da Pessoa Portadora de Deficiencia (CORDE) in Brazil, Secretaria Nacional para la Integracion de las personas con Discapacidad (SENADIS) in Panama, and Disabled Organization for Legal Affairs and Social Economic Development (DOLASED) in Tanzania.

Meeting summaries / major outcomes:

First meeting: February 18-20, 2002 in Washington, DC, USA

It was agreed that: 1) it is important and possible to craft a short set/s of internationally comparable disability measures; 2) short and long set(s) of measures that are inter-related are needed; 3) the ICF model will be used as a framework in developing disability measures; and 4) census questions are the first priority.

Second meeting: January 9-10, 2003 in Ottawa, Canada

A link was established between the purpose/s of a short measure on disability and aspects of measurement. A conceptual matrix was developed linking the purpose of a short disability measure with conceptual definitions and question characteristics. An empirical matrix was developed evaluating the characteristics of short set(s) of disability measures currently in use according to the dimensions of the conceptual matrix. Both matrices helped the WG to identify gaps in disability measurement.

Third meeting: February 19-20, 2004 in Brussels, Belgium

Since disability is multidimensional, it is not possible to ascertain the single "true" disabled population. Different purposes are related to different dimensions of disability or different conceptual components of disability models. Equalization of opportunities was selected as the purpose for which an internationally comparable short disability measure would be developed. A workgroup was designated to generate a draft set of questions related to this purpose. In addition, two other workgroups were formed to propose methods for implementing the short set and to propose an approach for developing extended measurement sets related to the short set. Finally, a plan for WG governance was adopted.

Fourth meeting: September 29-October 1, 2004 in Bangkok, Thailand

Major outcomes of the 4th WG meeting were: 1) conceptual agreement on a draft set of questions for the general disability measure, but wording revisions were required prior to pre-testing; 2) formation of a new workgroup operating in conjunction with a consultant to develop six implementation protocols for pre-testing the short set of disability measures; 3) begin development of the first extended measurement set; and 4) formation of a new workgroup on methodological issues.

Regional workshops: 1) June 20-22, 2005 in Nairobi, Kenya; 2) September 19-20, 2005 in Rio de Janeiro, Brazil

The Washington Group held two regional workshops in 2005, in Africa and Latin America, primarily directed toward countries in the region who were interested in including disability questions in their national censuses. The workshops familiarized countries in the region with the short set of WG questions on disability, the accompanying rationale, and the procedures for pre-testing the questions.

Fifth meeting: September 21-23, 2005 in Rio de Janeiro, Brazil

Revisions were suggested for the short measurement set, the accompanying rationale, and the implementation protocols. A new workgroup was formed to plan and implement analyses of the WG pre-tests. All results pertaining to the six WG questions will be considered by the new workgroup including the WG sponsored pre-tests, the WHO/ESCAP test, and other testing activities.

Sixth meeting: October 10-13, 2006 in Kampala, Uganda

Based on the outcomes of the pre-tests, the WG endorsed the six question set for use in censuses. The set comprises questions on four core functional domains (seeing, hearing, walking, and cognition) as well as two additional domains desired by member countries (self care and communication).

Detailed analyses of the pre-test data were presented at the meeting, however as there was much more analytical work that can be done that would be informative, the methodological workgroup merged with the data analysis workgroup to address three specific issues:

1. Portability of questions across administration modes;
2. How the questions work for specific subpopulations such as those with severe disability, children, or the institutionalized population; and
3. The use of proxy informants.

The workgroup on extended measures was charged with self-organizing in order to accomplish their work, and drafting a position paper specific to developing the first extended set with a purpose of equalization of opportunities. The paper was to include a plan and approach (blueprint) for carrying out development of the extended set including the purpose, rationale, and justification for the extended set as well as the issue of international comparability. The group was charged with adding questions on the existing domains and adding domains as appropriate to assess equalization of opportunities. The group was to review and select existing questions and pre-test the question set if time permits.

Seventh meeting: September 19-21, 2007 in Dublin, Ireland

The seventh meeting was hosted by the Central Statistics Office Ireland (CSO) with assistance from the National Disability Authority (NDA). The meeting was attended by 58 persons;
- 25 representing national statistical authorities from 22 countries (Austria, Bermuda, Brazil, Canada, Czech Republic, Cambodia, Estonia, France, Greece, Hungary, India, Ireland-3, Italy, Latvia, Lithuania, Mexico, Norway, Poland, Slovak Republic, Slovenia, Sweden and Uganda-2);
- 4 representatives from the National Center for Health Statistics;
- 22 representatives from national institutes of public health or other national research bodies or ministries (Belgium-2, China, Czech Republic, Denmark, France, Finland, Ireland-5, Italy, Japan, Kenya, Slovenia, South Africa, Spain, Tanzania, The Netherlands-2, United Kingdom);
- 7 representatives from international organizations (UNSD, UNESCAP, World Bank, WHO, UNECE, Eurostat, European Disability Forum)

Objectives for the 7th WG meeting were to:
1) Present additional work on the short set:

•  Present results of additional pre-testing
•  Present results of additional analyses of pre-test data
•  Present any revisions to original six questions
•  Present work on use of short set as a screener
•  Present option for measuring upper body function

2) Present a proposal for the extended set and test results if available.
3) Discuss strategic issues.

Objectives for the seventh meeting emanated from work presented at the sixth meeting. Three workgroups were to address these major topics.

Workgroup 1

Considered minor revisions to the short question set. In addition, the group addressed the development of an alternative (optional) question on upper body function.

1) At the 6th WG meeting the representative from Viet Nam raised a concern about false negatives, i.e. people who were unable to do some task but responded as having "no difficulty" on the short set, only to be discovered as having a large difficulty on the extended sets.

This issue was addressed by analyzing data on vision in the 2002 National Health Interview Survey (NHIS) in United States which contained questions similar to the WG, as well as several follow-up questions. The analysis found that eye disease does not appear to be related to degree of activity limitations. Some people with one or more eye diseases indicated no problem seeing. Furthermore, it was found that the proportion of false positives is higher than false negatives - i.e. more people respond yes to the main question and no to extended questions than visa versa. Explanations for the results were that some of the extended questions reflect a possible overlap of physical and visual limitations (i.e. questions related to going down steps and driving); and heterogeneity among respondents in terms of their tolerance for (and willingness to indicate) difficulty.

Future analysis was called for to:

•  Control for other functional limitations
•  Control for eye diseases captured in survey
•  Examine results within age context
•  Further elaboration of answer patterns (short vs. extended) and the characteristics of respondents who report the different patterns

Conclusion: The concern about false negatives on vision was found not to be a significant problem based on results from the USA data.

2) At the 6th meeting the workgroup was charged with the responsibility to assess improvements and additions to Short Set:

• Find a solution to the vision clause problem
• Consider the length of the communication question
• Provide a question on upper body functioning for countries that might prefer to have that measure.

Conclusions:
Vision clause: Instruct countries to translate the phrase in a way that is culturally appropriate to capture the idea in the question , it is not necessary to translate the question word for word, but to convey the idea that difficulty seeing must be present even if the respondent is using corrective lenses of any type.

Length of Communication question: The introductory clause is to be used to introduce the whole set of questions rather than just the last one.
Reformat the communication question as follows:
Using your usual (customary) language, do you have difficulty communicating; for example understanding or being understood?

Additional Upper Body Question: Upper body measurement focuses on 4 actions: pushing or pulling heavy objects, lifting 10 pounds or more, lifting arms overhead and grasping or some form of fine motor skill. The challenge in an Upper Body Question then is to limit the domain to one question that includes as many of the 4 actions as feasible while maintaining structural simplicity to compliment the other short set questions and using the same response categories. It would also be desirable to include actions that may be related to employment. Most of the questions currently in use reflect a single action; these were reviewed. ILO has repeatedly requested a question addressing upper body function. It was recommended that the issue of upper body function be dealt with in extended sets and to analyze data upon testing.

Workgroup 2

provided the results of additional analyses of the characteristics of the short set.

1) Assessment of how well the WG questions work to identify disabled people for prevalence estimates

Two years ago, cognitive testing began in 15 countries to determine how well respondents understood the questions. Combined analyses were performed on a sample of 1290 respondents. Inconsistencies were found between the WG questions and the follow-up questions. Reasons for inconsistencies included:

• True response error (in WG question or follow-up questions)
• Characteristics of respondents disability that were not captured in follow up questions
•  Data entry/Interviewer error

The question remained: How do inconsistencies affect prevalence and identification of type of difficulty? Analyses revealed that using the threshold of  'Some difficulty' to define disability yielded more inconsistent responses than 'a lot of difficulty'.
Higher rate of inconsistencies were found in the in cognition and self care domains.
Conclusion: WG short set is useful in identifying people for prevalence of disability--very few false positives.

2) Assessment of how each question captures functional limitations within its specific domain; i.e. to what extent does each question falsely identifies people as having a disability and what are the reasons for misidentifications and which population(s) are most likely to be misidentified.

Misidentification can occur for a number of reasons. It is important to determine whether false positives or negatives are occurring systematically or randomly. For cases identified as 'true errors', determine if there is any association with gender, country, age, disability or health status. The true errors identified for the vision question were more than likely related to the glasses clause and misunderstanding of the question. There is a potential for false positives in the extended questions on cognition as a high rate of inconsistencies was found; however, unlike the vision question, the inconsistencies are more likely to be a result of interpretation issues and not obvious misunderstanding.

Conclusions:

• WG questions taken as a group are good at generating general prevalence estimate
• Confirm that glasses clause is significant issue, but needs to be addressed at country level-- language, custom
• Country differences in response error are significant -- suggests need for country specific cognitive testing in question development
• Preliminary results suggest no real sign of demographic bias

3) Assessment of the potential of WG Short Set as screener

Data from Canada was used to evaluate how the WG questions compared to more detailed questions. In Canada a census was used to develop a frame for a disability survey. The Participation and Activity Limitation Survey (PALS) was carried out as a post-censal survey with additional follow-up studies. That allowed for the testing of false positives, false negatives and the performance of the WG questions. Face to face interviews were then conducted with 50 false positives, 100 false negatives and 50 "soft" disabilities (e.g. learning, etc. who are normally difficult to identify). The WG questions were included in all interviews.

Findings: The WG questions seem to miss emotional/psychological disabilities, learning disabilities, agility disabilities and mild to moderate pain disabilities.

As a screener the WG questions are easily understood, concise and appropriate to a Census, and appropriate to proxy response. However, they are less adaptable in terms of being inclusive of all types of disabilities and covering all age groups.

Workgroup 3

was to draft a position paper specific to developing an extended question set for the purpose of assessing equalization of opportunities. The position paper was to include the plan, purpose, and approach for developing the extended set. It was agreed that the primary issue for this extended set is expansion of the existing domains covered in the short set and adding to the existing domains. The WG will be collaborating with UNESCAP on development of the extended set/s.

In its proposal for Extended Question Sets, the following principles were agreed upon for the development of extended set(s) of questions:

1) Feasibility and cross-country comparability
2) ICF framework (terminology) as a basis
3) Review of existing sets as a basis for further development
4) Congruency and coherence between short and extended sets

The purpose of extended sets will be to address issues of equalization of opportunities and the determination of disability prevalence. In addition, data derived from the extended sets will be used for the individual needs of the country collecting the information (policy development, advocacy, monitoring and evaluation of interventions, international reporting, and providing summary measures on disability in general or individual impairments).

The compilation of extended set(s) will require the resolution of several issues:
1) a closer look at "cross-country comparability" and what some of the limitations may be in meeting this requirement
2) a discussion of how the extended set(s) will serve the stated purpose of "equalization of opportunities"
3) the specific wording of questions and response options (4 vs. 5 answer categories)
4) how to best measure environmental factors
5) Choice and desire--the issue of needing or wanting to do the activity

Future work should focus on:

• Deciding on the structure of the proposed extended set(s) (includes purpose, nature and number)
• Compiling the sets
• Building an evidence base (cognitive testing, statistical analysis of existing data)
• Analysing and reporting (summary measures, individual question responses, prevalence estimate, deciding cut off point)
• Developing guidelines

There was general agreement that there should be (at least) two extended sets.

• The first set would build upon the Short Set, re-visit domains excluded from Short Set and further develop Short Set domains. Additional domains would include: upper body functioning, psychological functioning, expansion of the cognition domain, fatigue*, and pain*. (*The consensus was that pain and fatigue should be included. However, the method of inclusion, as separate domains or as characteristics of other domains, will be addressed by the Extended Sets workgroup.)
• The second set would be used to get more detailed information. There was consensus on a need for questions on cause, age of onset, duration and environmental factors (including assistive technology and assistance and environmental barriers/ facilitators).
  With respect to basic and complex activities: The objective would be to extend the information collected on the basic activities (Short Set) in order to explain more complex activities (participation). This is complimentary to equalization of opportunity. In order to assess how a person improves their participation in society (e.g. through education, employment etc.) questions on assistive devices/environmental factors need to be included in each domain. These participation questions are important with respect to policy.
  In summary, there was agreement to work with UNESCAP in the development of extended set(s).

Also, the Extended Sets workgroup will:

• Re-visit the Short Set of basic activity (functioning) domains (adding possibly multiple questions to certain domains)
• Decide on the use of an upper body domain
• Decide on the inclusion of supplementary questions within domains (cause, onset, duration, etc...)
• Decide on how best to capture environmental factors (micro, meso, and macro levels)
• Explore different ways to measure participation

Furthermore, the workgroup will:

• Coordinate work with the work of other groups (BI, Eurostat, UNESCAP)
• Compile list of questions being used in other workgroups
• Determine timeline for extended set
• Determine who wants to be involved in the Extended Sets Workgroup. (Margie Schneider, chair)

Other agenda items

In addition to the sessions reporting on workgroup activities, updates were presented from the United Nations Statistical Division, UN affiliates, and the World Bank about their activities related to disability statistics, as well as country specific presentations. Finally, the Steering Committee chair led a discussion on strategic issues including planning the next steps.

Harmonizing work on Extended Sets

 with the Budapest Initiative, health-related Eurostat projects and UNESCAP - UN Development Account Project on Improvement of Disability Statistics: The importance of coordinating and linking with the activities of other groups in the field was stressed. WG will prepare a proposed strategy to maximize synergy in the work of the Washington Group, the Budapest Initiative and the UNESCAP Disability Project. This strategy will be distributed to the Steering Committee. In addition a copy of the Eurostat questionnaire will be requested.

Technical Assistance to NSOs:

The Washington Group has completed its work on the short set of disability questions to be included on Censuses. A survey by UNSD suggests that some technical assistance is needed in the area of implementation, interpretation and analysis of disability questions on Censuses. To fill this need, the Washington Group proposes to develop two documents that address issues related to the adoption of the new question set on Censuses. In particular, issues related to how to interpret the data produced by the questions will be addressed. One document will be designed for use by the NSO and one for use by policy and advocacy groups. Other possible activities include:

• development of a presentation and a packet of materials to be used at regional workshops or technical assistance meetings on the 2010 Census Round;
• participation of WG members at regional workshops or technical assistance meetings on the 2010 Census Round;
• provision of technical assistance by WG members as requested by NSO's
  If there is agreement on these activities, the WG steering committee chair will contact the Director, UNSD to discuss next steps.

The World Bank Project:

United Nations (UN) approached the World Bank (WB) with an idea for developing a set of indicators to monitor the implementation of the UN Convention on the Rights of Persons with Disabilities. It was proposed that the WG develop a tool for monitoring based on the short set of questions and further development through the extended set. Funds for this project would be raised through the World Bank Donor Trust Fund and would be allocated to pay for consultant services to assist the group in developing the tool (extended set), cognitive testing in several countries, and to fund a special meeting of countries for final agreement. If there is agreement, a proposal is required
The meeting agreed to take on the World Bank Project.

Country presentations:

1) Report on the Irish Census
In 2004 the Government decided to conduct a post-censal National Disability Survey to establish prevalence, severity and impact of disability in Ireland and to identify improvements needed in policy and service provision. A sample was based on responses to the 2006 Census questions on disability. Questionnaires were developed in consultation with NDA, Government departments, representative groups, and disability research experts.

Conclusions: Matching NDS records to Census at the person level increases the statistical value of NDS output. A high proportion of the Census false positive responses are due to responses in the 'Other'  category. Less than 3/4 of the sample reported a disability in both surveys. The Census questions/methodology resulted in a much higher level of single disability reporting then in the NDS. Learning related difficulties were prevalent among children while mobility and pain were most prevalent disabilities for older people, and people of working age had a more mixed range of reported difficulties.

2) Report on Three Country Pilot Study - Brazil (IBGE), Argentina (INDEC) and Paraguay (DGEE)
The pilot study was carried out in November 2006: interviews were conducted in Brazil (4039), Argentina (1903) and Paraguay (2009). Information was collected on disability (core, extended, and IBGE questions), housing and demographic characteristics.
Findings: In all countries WG questions identified more people than the extended sets especially at the D1 (minimum response to WG questions is some difficulty) and D2 (minimum response is a lot of difficulty) levels, and at D3 (minimum response is can't do it at all) the opposite was found, where higher rates were observed in the extended sets.
In Brazil a single vision question captured less people than 2 questions.

3) Report on Uganda Pilot Study
A pretest of the 6 WG questions was part of the 2006 Uganda Demographic and Health Survey (UDHS). Data were collected between May and October 2006 and included 8,870 households selected from 368 EAs (clusters) covering the entire country. The disability questions were administered to all household members over 5 years of age.
Findings: Vision, hearing, walking, and cognition difficulties increase with age while self-care difficulties present a U-shaped curve and communication difficulties increase sharply at age 50-59. The prevalence of recorded difficulty in at least one functional area increased drastically at age 30, with overall most people falling into the category "some Difficulty". A clear challenge will be to consider levels of severity in the computation of disability prevalence.

4) Introduction to Surveys and Research on Disability in China
National Sample Surveys on Disability in China are planned every 20 years. To-date surveys have been carried out in 1987 and 2006 with annual monitoring using smaller samples. The Institute of Population Research at Peking University is responsible for these activities. In addition, the China Disability and Development Research Center has recently been established at Peking University as the first national institute on Disability Studies.
An International Forum on Disability and Development was planned for December 10-15, 2007.

5) Functioning and disability in Europe--Measuring Health and Disability in Europe (MHADIE) project results: ICF Functional profiles in 12 selected health conditions in Europe. MHADIE is a three-year Coordination Action financed by the EU Commission, involving 16 European Centres and 10 different countries. It aims to demonstrate the utility and feasibility of ICF model in measuring different types and prevalence of impairments and limitations.

Data were collected on health conditions:

Bipolar Disorder, Depression, Ischemic Heart Disease, Migraine, Multiple Sclerosis, Musculoskeletal Conditions, Parkinson Disease, Stroke, Traumatic Brain Injury;

Demographic information:

Nationality, gender, age, marital status, educational level, current job, risk factors (smoke and alcohol consumption);

Functional outcomes and profiles:

The ICF Checklist, The WHO Disability Assessment Schedule (WHODAS II), The Health System Responsiveness and Satisfaction with Health Care (HSR&S) scale, Short Form 36 (SF-36), The WHO Quality of Life Questionnaire (WHOQoL).
It was found that ICF-based datasets demonstrate the utility of the ICF and its related tools in describing functioning, health and disability across a variety of settings, clinical conditions and countries. ICF can be graphically represented to compare clinical samples for different variables: health condition, clinical setting, age group or country.

The correlations between ICF data and selected clinical variables demonstrates that ICF provides a common base of data for distinguishing functional patterns among different conditions, but also gives the possibility of drawing functional profiles besides the health condition.

Governance issues

The 8th WG meeting will be held October 29-31, 2008 in Philippines.

In keeping with UN guidelines, issues of gender bias and other potential sources of bias will be a consideration of all WG work.

Products

Proceedings from the meetings (presentations and papers), reports to the UN Statistical Commission, final meeting reports, and information on upcoming meetings can be accessed through the Washington Group website, currently hosted by the National Center for Health Statistics, U.S.A. website.

WG Points of contact:

Washington Group Secretariat (NCHS, U.S.A.)

Cordell Golden
Statistician
National Center for Health Statistics
3311 Toledo Road, Room 6428
Hyattsville, MD 20782 (USA)
(Phone) 301-458-4237
(Fax) 301-458-4038
(Email) CGolden@cdc.gov

Mitch Loeb
Associate Service Fellow
National Center for Health Statistics
3311 Toledo Road, Room 6435
Hyattsville, MD 20782 (USA)
(Phone) 301-458-4248
(Fax) 301-458-4037
(Email) MLoeb@cdc.gov

Jennifer Madans
Associate Director for Science
National Center for Health Statistics
3311 Toledo Road, Room 7202
Hyattsville, MD 20782 (USA)
(Phone) 301-458-4500
(Fax) 301-458-4020
(Email) JMadans@cdc.gov

Washington Group Content

Washington Group Meetings

National Center for Health Statistics
3311 Toledo Road
Hyattsville, MD 20782
Phone:
1-800-232-4636
nchsquery@cdc.gov

Centers for Disease Control and Prevention, 1600 Clifton Rd, Atlanta, GA 30333, U.S.A
Tel: (404) 639-3311 / Public Inquiries: (404) 639-3534 / (800) 311-3435