|
|
 |
|
|
Section
III: Overview of the Measurement
Framework and Identification of
Specific Measures for Cultural
Competence
I. Overview
of the Measurement Framework and Identification of Specific
Measures for Cultural Competence
Previous
sections of this report addressed the concept of cultural
competence and the spheres or domains in which it should
be manifested in health care settings and the topic areas
within each domain.
In looking towards the development of a measurement
profile for cultural competence, we now turn to a presentation
of a measurement framework, a structured way of thinking
about the types of measures relevant to health care settings.
We then use this framework to report findings from
the literature on specific indicators and measures that
might be applicable to measuring cultural competence.
A more detailed display of these findings is presented
in Attachment 3.
This section of the report:
-
Provides
a description of the methodology used to review the
measurement literature;
-
Presents
an overview of the measurement framework for cultural
competence;
-
Provides
a description of the current state of the field in identifying
measures and indicators of cultural competence by domain
and;
-
Discusses
the different levels at which indicators and measures
can be applied.
It
is important to note that the measurement framework and
specific indicators and measures reported in this
section do not represent the final stage in developing
a measurement profile of cultural competence.
Rather, this serves as a starting point that will
be further refined and informed by the input of key stakeholders
(e.g., Technical Expert Panel, conversations with other
experts, and site visits to health care delivery settings
that employ innovative approaches to delivering culturally
competent care). The description of measures used in the
field of cultural competence presented here is continually
being updated to capture a “universe” of performance measures
that will be further refined and developed.
In selecting from this “universe” of measures for
the evolving measurement profile, consideration will be
given to identifying measures that are meaningful, quantifiable,
practical, and are useful to the variety of stakeholders
interested in measuring cultural competence in health care
settings.
A. Measurement
literature review methodology
The
review of the measurement literature included a review of
assessment tools, standards and guidelines developed by
and for specific stakeholders, and articles focused on measuring
cultural competence. The literature identified either actual
performance measures, provided suggestions of potential
measurement areas, or described assessment tools. For the
purposes of this report, these disparate types of information
will all be classified as “measures.”
Sources
of measures or indicators of cultural competence were found
across a variety of literature, both federal and non-federal.
Federal agencies, including the Centers for Medicare and
Medicaid Programs (CMMP, formerly the Health Care Finance
Administration), the Department of Health and Human Services’
Office of Minority Health (OMH), and the Substance Abuse
and Mental Health Services Administration (SAMHSA), have
sponsored studies to identify measures of cultural competence
for various health settings. For example, CMMP sponsored
a report to develop a set of recommendations for measures
of cultural competence for managed care organizations that
provide care to Medicare and Medicaid beneficiaries under
contracts with CMMP or with State Medicaid agencies.
SAMHSA has also sponsored studies to develop measures by
which managed care organizations could be assessed on cultural
competence.
DHHS’ OMH sponsored a study that developed consensus and
standards regarding what constitutes cultural and linguistic
competence in health care service delivery (i.e., CLAS).
Other federal sources
include works by DHHS’ Health Resources and Services Administration
(HRSA). Among these are a set of measures applied by HRSA’s
Maternal and Child Health Bureau to review potential and
existing grantees for cultural competence.
The project team also reviewed criteria used in HRSA’s Cultural
Competence Works competition to recognize exemplary HRSA-funded
programs for their culturally competent services
and programs highlighted by HRSA’s Bureau of Primary Health
Care as innovative in delivering services that bridge the
cultural gap between providers and patients.
The innovative activities performed by these programs were
reviewed to identify indicators of cultural competence that
could serve as the basis for developing actual measures
of cultural competence. The project team also reviewed tools
developed by various authors and programs to assess the
cultural competence of personnel and to conduct organizational
assessments of cultural competence. For example, the National
Center for Cultural Competence (NCCC) has developed a series
of self-assessment tools for use by providers and organizations.
Authors such as Flores, G., Mason, J.L., Lavizzo-Mourey,
R., Mackenzie, E.R., and Tirado, M. have also developed
either tools or models for measuring cultural competence.
Refer to the Annotated Bibliography for a description of
these and other sources used to identify measures and Attachment
3 for a comprehensive listing of measures.
B. Overview
of the measurement framework
Considering
issues related to the structure, process, and outcome of
care is a well-established and useful way of thinking about
measurement in the health field, especially as related to
assessing quality of care.
Building on this formulation, we add an additional
area of assessment, “organizational viewpoint,” and use
four categories, described below, as a measurement framework
by which to describe the types of measures applicable to
cultural competence found in the literature. This framework
provides one way of addressing the questions. “How do you
know cultural competence when you see it?” You should know
it, in part, by an organization’s structures, processes,
outcomes and viewpoint.
Type of measure
-
Capacity/structure
measures: assess the organization’s
capability to support cultural competence through adequate
and appropriate settings, instrumentalities and infrastructure,
including staffing, facilities and equipment, financial
resources, information systems, governance and administrative
structures, and, other features related to organizational
context in which services are provided.
-
Process measures:
assess the content and quality of activities, procedures,
methods and interventions in the practice of culturally
competent care and in support of such care.
-
Impact/outcome
measures: assess the contribution of cultural competence
to the achievement of various levels of objectives (e.g.,
intermediate, ultimate), with respect to the provision of
care, the response to care, and the results of care.
-
Organizational
viewpoint measures: assess the values, principles, perspectives,
outlook, and organizational attitudes espoused and displayed
by an organization as these relate to cultural competence.
Vantage
point and usage
In addition,
we include in the measurement framework categories for examining
the different vantage points from which different measures
can be examined. Measures
can be categorized in terms of the stakeholders to whom
this measure would be of interest
(e.g., payers, providers, etc.), the purposes for
which these measures might be used (e.g., oversight, quality
improvement, formative evaluation), and the level of analysis
(e.g., individual, organization, health system, or societal)
for which the measure might be useful.
Exhibit III provides a graphical depiction
of the measurement framework which will be applied to identifying
indicators and measures of cultural competence by domain
and topic area. Attachment
3 includes the comprehensive, categorized list of measures.
Exhibit
III
Measurement
Framework Summary Table
|
Domain
|
Topic Area
|
Measures/ Indicators
|
Type
of measure
|
Vantage
point
|
Usage
|
Citation
|
|
Capacity/
Structure
|
Process
|
Outcome/ Impact
|
Org. Viewpoint
|
|
|
|
|
|
|
|
|
|
|
|
C. Measures
by Type
A
review of the literature found wide variation in the type
of measures currently available to assess the domains of
cultural competence, with a greater representation of process,
capacity/structure, and organizational viewpoint, than outcome/impact
measures. The
following summarizes this variation and the types of measures
are discussed according to their frequency. Refer to the
Annotated Bibliography for a description of documents used
to identify measures and Attachment
3 for a comprehensive listing of measures.
-
Process
measures: Process measures were the most prevalent
types of measures across all the domains. The communication
domain included process measures focused on the use
of interpreters and translated materials and the training
and staff development domain assessed the number of
staff trained in cultural competence. In the polices
and procedures domain, there were process measures focused
on ensuring cultural competence through ensuring proportionality
of racial/ethnicity providers to consumers and hiring
of adequate numbers of multicultural/multilingual staff.
The intervention and treatment model domain and family
and community participation domain included measures
that demonstrated the inclusion of consumer input in
the treatment development, planning and decision-making
processes. Exhibit
IV provides a sample list of measures found in the literature categorized
by domain.
Exhibit
IV
|
Sample
of Process Measures by Domain
|
|
Domain
|
Topic
Areas
|
Measures/Indicators
|
|
Communication
|
Interpreter
|
Yearly
updated directory of trained interpreters is available
within 24 hours for routine situations and within
one hour or less for urgent situations.
|
|
Communication
|
Interpreter
|
Percentage
of clients with limited English proficiency (LEP)
who have access to bilingual staff or interpretation
services.
|
|
Communication
|
Translated
materials
|
Percent
of clients who receive oral and written notices,
including translated signage at key points of
contact, in their primary language informing them
of their right to receive no-cost interpreter
services.
|
|
Communication
|
Linguistically
competent organization
|
|
|
Communication
|
Language
ability, written and oral of the consumer
|
Consumer
reading and writing levels of primary languages
and dialects is recorded.
|
|
Policies
and procedures
|
|
|
|
Policies
and procedures
|
Staff
hiring, recruitment
|
|
|
Training
and staff development
|
Training
and professional development
|
-
%
of staff with cultural competence training.
-
%
of staff attending ongoing cultural competence
training.
-
%
of ongoing cultural competence training completed.
|
|
Training
and staff development
|
Training
and professional development
|
Cultural
competence training is part of the credentialing
process for case managers.
|
|
Intervention
and treatment model features
|
Diagnosis,
care planning, referral
and treatment
|
Client
assessments are conducted in client’s primary
language.
|
|
Intervention
and treatment model features
|
|
Indicators
of culturally competent treatment plan in health
plan:
-
The
Treatment Plan reflects both consumer and
family involvement in its development and
agreement. The degree of family involvement
depends on the wishes of the consumer.
-
The
organization has a written policy and a demonstrated
practice linking families to advocacy and
education groups.
|
|
Intervention
and treatment model features
|
Input
into treatment decision and service quality
|
Indicators
of culturally competent treatment plan in health
plan:
-
The
Treatment Plan reflects both consumer and
family involvement in its development and
agreement.
-
There
is evidence in the Treatment Plan of the use
of racial/ethnic community services and resources.
-
The
Treatment Plan was developed with a culturally
competent clinician or consultation from such
a clinician
-
Consumer
and family involvement and investment in the
development of, and agreement with, the Care
Plan.
-
Culturally
defined needs addressed in the care plans
of consumers from various racial/ethnic
groups.
-
Leadership
by racial/ethnic Mental Health Specialists
in the care planning process for consumers
from various racial/ethnic groups.
|
|
Family
and community participation
|
Community
and consumer participation
|
Degree
to which families participate in key decision-making
activities.
-
Family
participation on advisory committees or task
forces
-
Hiring
of family members to serve as consultants
to providers/programs
-
Inclusion
of family members in planning, implementation
and evaluation of activities
|
-
Capacity/structure
measures: These types of measures were widely dispersed
across the various domain areas. Capacity/structure
measures found in the communication domain focused on
the amount of financial resources dedicated to interpretation
and translation services, the conduct of audits of provider
networks to measure the linguistic capacity of the provider,
and availability to mechanisms to disseminate culturally
competent information to consumers. In the policies
and procedures domain, capacity/structure measures focused
on whether organizations had governing boards or advisory
committees composed of diverse ethnic/racial /cultural
groups, used creative financing mechanisms to ensure
access to traditional healers in health plan benefit
packages, or had mechanisms in place to track consumer
grievances and complaints. Capacity/structure measures
were also found in the facility characteristics, capacity
and infrastructure domain with measures that assessed
whether organizations had adequate mechanisms in place
to maintain and track data on the ethnic/racial/cultural
composition of its service population. Capacity/structure
measures in the monitoring, evaluation and research
domain focused on whether organizations had the infrastructure
to: conduct self-assessments on cultural competence
through internal audits; conduct evaluations of health
plan decision-making based on enrollee ethnicity; or
conduct culturally competent community need assessments.
Exhibit V provides a sample
list of measures found in the literature.
Exhibit
V
|
Sample
of Capacity/Structure Measures by Domain
|
|
Domain
|
Topic
Areas
|
Measures/Indicators
|
|
Communication
|
Translated
materials
|
Allocated
resources for interpretation and translation services
for medical encounters and health education/promotion
material.
|
|
Communication
|
Linguistic
capacity of the provider
|
Ability
to conduct audit of the provider network which
includes the following components:
-
Languages
and dialects of community available at point
of first contact
-
#
trained translators and interpreters available
-
#
of clinicians and staff proficient in languages
of the community
|
|
Communication
|
Provide
information, education
|
-
Organization
has the capacity to disseminate information
on health care plan benefits in languages
of community.
-
Organization
has the capacity to disseminate information
and explanation of rights to enrollees.
|
|
Policies
and procedures
|
Grievance
and conflict resolution
|
Organization
has structures in place to address cross cultural
ethical and legal conflicts in health care delivery
and complaints or grievances by patients and staff
about unfair, culturally insensitive or discriminatory
treatment, or difficulty in accessing services
or denial of services.
|
|
Policies
and procedures
|
Grievance
and conflict resolution
|
Organization
has feedback mechanisms in place to track # of
grievances and complaints and # incidents.
|
|
Policies
and procedures
|
Planning
and governance
|
Composition
of the governing board, advisory committee, other
policy-making and influencing groups, and consumers
served reflects service area demographics.
|
|
Facility
characteristics, capacity, and infrastructure
|
Available
and accessible services
|
-
Transportation
is available from residential areas
to cultural competent provider
-
Organization
has the flexibility to conduct home visits
and community outreach
-
Cultural
competent services are available evenings
and weekends
|
|
Facility
characteristics, capacity, and infrastructure
|
Information
systems
|
Capacity
for tracking of access and utilization rates for
population of different racial/ethnic groups in
comparison to the overall service population.
|
|
Monitoring,
evaluation and research
|
Organizational
assessment
|
Ability
to conduct ongoing organizational self-assessments
of cultural and linguistic competence and integration
of measures of access, satisfaction, quality and
outcomes into other organizational internal audits
and performance improvement programs.
|
-
Organizational
viewpoint measures: Organizational viewpoint measures
were distinguished from process measures because they
reflect an organization’s attempt at changing the attitude,
values, belief, communication and culture of an organization
or understanding patient-specific values and beliefs.
These measures were found in the cultural sensitivity,
values and attitudes, and communication domains where
measures and assessment tools examined an organization’s
or provider’s ability to appreciate, value and respect,
the culture of others, how patients view their own health,
as well as how to communicate effectively with different
cultures. Organizations could demonstrate this ability
through the development of culturally competent mission
statements, the conduct of patient health beliefs inventory,
and the appreciation and use of various communication
styles. In the policies and procedures and training
and staff development domains, there were examples of
measures that assessed an organization’s commitment
to cultural competence through the incorporation of
cultural competence targets in staff incentive systems
and required demonstration by providers of respecting
and valuing diverse cultures. Exhibit
VI provides a sample list of measures found in the
literature.
Exhibit
VI
|
Sample of Organizational Viewpoint Measures by Domain
|
|
Domain
|
Topic
Areas
|
Measures/Indicators
|
|
Cultural
sensitivity
|
Culturally
sensitive encounters
|
Conduct
health beliefs inventory of patient to understand
the patient’s explanatory model for illness.
|
|
Values
and attitudes
|
Appreciate,
respect
|
Checklist
from the National Center for Cultural Competence
(NCCC) on Values and Attitudes. Indicate A= things
I do frequently, B= things I do occasionally,
C= things I do rarely or never.
-
I
recognize and accept that folk and religious
beliefs may influence a family's reaction
and approach to a child born with a disability
or later diagnosed with a disability or special
health care needs.
-
I
understand that traditional approaches to
disciplining children are influenced by culture.
-
I
understand that families from different cultures
will have different expectations of their
children for acquiring toileting, dressing,
feeding, and other self help skills.
-
I
accept and respect that customs and beliefs
about food, its value, preparation, and use
are different from culture to culture.
|
|
Values
and attitudes
|
Mission,
vision
|
Mission/vision
statement commit to the delivery of culturally
and linguistically competent services.
|
|
Values
and attitudes
|
Mission,
vision
|
MCO
self-certification that its mission statement/strategic
vision support diversity and cultural competence.
|
|
Communication
|
Communication
styles
|
Checklist
from the NCCC on Communication style. Indicate
A= things I do frequently B= things I do occasionally
C= Things I do rarely or never.
-
For
children who speak languages or dialects other
than English, I attempt to learn and use key
words in their language so that I am better
able to communicate with them during assessment,
treatment or other interventions.
-
I
attempt to determine any familial colloquialisms
used by children and families that may impact
on assessment, treatment or other interventions.
-
I
use visual aids, gestures, and physical prompts
in my interactions with children who have
limited English proficiency.
-
I
use bilingual staff or trained volunteers
to serve as interpreters during assessment,
meetings, or other events for parents who
would require this level of assistance.
|
|
Communication
|
Linguistically
competent organization
|
Interpreters
and bilingual staff demonstrate bilingual proficiency
and receive training that includes the skills
and ethics of interpreting, and knowledge in both
languages of the terms and concepts relevant to
clinical or non-clinical encounters.
|
|
Communication
|
Linguistically
competent organization
|
Consumer
education information respects cultures, reflects
literacy levels and is in different formats
|
|
Communication
|
Administration
and staff should be able to translate, cultural
brokering
|
Promote
and support the attitudes, behaviors, knowledge,
and skills necessary for staff to work respectfully
and effectively with patients and each other in
a culturally diverse work environment.
|
|
Policy
and procedures
|
|
Demonstration
of staff knowledge and skills regarding group
values, traditions, expression of illness, cultural
competence principles (e.g., credentialing and
performance based testing).
|
|
Training
and staff development
|
Assessment
of the knowledge and skills/attitudes of the provider
|
-
Demonstrate
attitudes that indicate a respect for the
consumer’s immigration, migration, colonization,
and acculturation experiences.
-
Demonstrate
attitudes that indicate a respect for the
diverse heritages, cultures, and experiences
of consumers from the four groups.
-
Demonstrate
attitudes that indicate a willingness to work
with culturally, ethnically, and racially
diverse populations.
|
-
Outcome/impact
measures: Outcome/impact measures represented the
least prevalent type of measure found in the literature.
There were measures in the communication domain that
examined misdiagnosis and inadequate treatment planning
resulting from inappropriate communication styles. Other
outcome/impact measures focused on the impact of cultural
competence on increased use of preventive services (e.g.,
increased mammography rates) among minority populations.
In the intervention and treatment model features domain,
measures were found that provided a checklist to assess
the impact of specific interventions, such as outreach.
However, these checklists usually measured impact of
the intervention via process or capacity/structure types
of indicators rather than health status improvement
or other traditional outcome/impact measures. The monitoring,
evaluation and research domain did include measures
on consumer, member and family satisfaction with services.
Exhibit VII provides a sample list of measures found in the literature.
Exhibit
VII
|
Sample
of Outcome/Impact Measures by Domain
|
|
Domain
|
Topic
Areas
|
Measures/Indicators
|
|
Communication
|
Diagnosis,
care planning, referral and treatment
|
Decrease in misdiagnosis
and inadequate treatment plans resulting from
failure to communicate effectively with consumers
from various racial/ethnic groups.
|
|
Intervention
and treatment model features
|
Diagnosis,
care planning, referral and treatment
|
Deaths
of infants and children aged 0 through 24 years
enumerated by age, subgroup, race and ethnicity.
|
|
Intervention
and treatment model features
|
Diagnosis,
care planning, referral and treatment
|
The
ratio of the black infant mortality rate to the
white infant mortality rate
|
|
Intervention
and treatment model features
|
|
Indicators
of a culturally competent treatment services provided
to minority members in health plan:
-
Consumer
and family satisfaction with treatment services.
-
Inclusion
of culturally specific activities and domains
of daily living (e.g., housing, access to
primary health care and maintenance, family
role, behavioral/developmental, vocational/
educational/employment, and community tenure)
in treatment services.
-
Rates
of symptom relapse and recidivism into restrictive
level of care or other restrictive placements.
Benchmark: Comparable to overall population
served and significant reductions over time.
-
Rates
of medication side effects, adverse incidents,
and utilization of latest pharmacological
interventions.
-
|
|
Monitoring,
evaluation and research
|
Consumer/
member satisfaction and feedback
|
Satisfaction rates due to
communication styles and linguistically competent
services to racial/ethnic consumers.
|
|
Monitoring, evaluation and research
|
Consumer/
member satisfaction and feedback
|
|
D. Type of Measure
by Level of Analysis
Another
aspect of potential measures of cultural competence is provided
by examining the different levels of analysis to
which the measures could apply or the perspectives from
which they can be viewed. This is related to looking at measures for vantage point and
usage.
In
health services research, the traditional levels of analysis
are the individual, organizational, and societal level.
For the purposes of this study, an additional level of analysis
is critical - the health care delivery system. Following
is a review of selected process measures and capacity/structure
measures by these levels of analysis to illustrate how the
same measure can be viewed from various perspectives.
Process
measures are typically viewed as measuring internal practices
and activities that are believed to be related to specific
outcomes. As one would expect and is demonstrated in Exhibit
VIII, many of the process measures could easily be analyzed
at the organizational level. However, while process measures
may actually be measured at the organizational level, their
impact can also be analyzed at the individual and health
care delivery system level. For example, the process measure
of training clinicians in cultural competence practices
can be analyzed at the individual level by: examining the
satisfaction level of the individual patient who is receiving
the services from the culturally competent trained clinician;
or assessing the change in the individual behavior and values
of the clinician due to his/her training.
In
addition, process measures that focus on the production
and availability of translated materials can be analyzed
at the health care delivery system level to assess the potential
change in the delivery system via improvements in the quality
of access points (i.e., culturally competent communication
via translated materials may promote improved access). Similarly,
process measures that focus on policies regarding contractual
arrangements with health plans can be analyzed at the delivery
system level to highlight potential changes in non-clinical
aspects of the delivery system, such as insurers adopting
culturally competence practices.
In
reviewing capacity/structure measures, there is also the
potential for multiple levels of analysis. Because many
of the capacity/structure measures are focused on infrastructure,
these measures would be analyzed at an organizational level
of analysis. However, these measures can also be analyzed
from other perspectives. For example, measures that assess
an organization’s capacity to develop and maintain information
systems to capture and track relevant data pertaining to
cultural competence could be analyzed to gauge the impact
of these efforts on the health care delivery system via
increased information sharing and linkage of various components
of the health care delivery system. Similarly, measures
that assess an organization’s capacity to provide transportation
services or provide outpatient services at convenient hours
of operation can also be analyzed at the health care delivery
system level to reflect the potential for creating flexible
venues of care. In
addition, measures that assess an organization’s capacity
to conduct community needs assessment and create governing
boards that reflect the influence of various consumer groups
could be analyzed at the individual level to reflect the
individual needs, values and perspectives of the local community.
Refer to Exhibit VIII for a demonstration of how the discussed process and
capacity/structure measures can be analyzed at multiple
levels of analysis.
|
Exhibit
VIII
Sample
of Process Measures by Level of Analysis
|
|
Domains
|
Topic
Areas
|
Measures/Indicators
|
Level
of Analysis
|
|
Individual
|
Organization
|
Health
Care Delivery System
|
Societal
|
|
Training
and staff development
|
Training
and professional development
|
Percentage of staff receiving
at least five hours of training annually in cultural
competence awareness.
|
X
|
X
|
|
|
|
Training
and staff development
|
Training
and professional development
|
|
X
|
X
|
|
|
|
Communication
|
Translated
materials
|
Percent of clients who receive
oral and written notices, including translated
signage at key points of contact, in their primary
language informing them of their right to receive
no-cost interpreter services.
|
|
X
|
X
|
|
|
Policies
and procedures
|
|
|
|
X
|
X
|
|
|
Exhibit
VIII
Sample
of Capacity/Structure Measures by Level of Analysis
|
|
Domains
|
Topic
Areas
|
Measures/Indicators
|
Level
of Analysis
|
|
Individual
|
Organization
|
Health
Care Delivery System
|
Societal
|
|
Facility
characteristics, capacity and infrastructure
|
|
|
X
|
X
|
|
|
|
Facility
characteristics, capacity and infrastructure
|
|
|
|
X
|
X
|
|
|
Policies
and procedures
|
|
|
X
|
X
|
|
|
|
Monitoring,
evaluation and research
|
|
Organization
has the capacity to conduct community profiles
containing information on the percentage of the
following that characterize target population:
|
|
X
|
X
|
|
Back to Top
|
|
|
