I. ASSESSING CULTURAL COMPETENCE

A. About the Project

"How do we know cultural competence when we see it?" is the central question that prompted the Health Resources and Services Administration (HRSA) to sponsor a project to develop indicators of cultural competence in health care delivery organizations. Throughout the nation, a growing consensus is emerging about the nature and importance of cultural competence as an essential component of accessible, responsive, and high quality health care. However, the pursuit of cultural competence in health care delivery organizations is constrained, in part, by the health field's lack of systematic approaches and tools for assessing cultural competence--that is, for gauging its presence, level, quality, and contribution to good health and health care.

This project aimed to contribute to the methodology and state-of-the-art of cultural competence assessment. The product - An Organizational Cultural Competence Assessment Profile - builds upon previous work in the field, such as the National Standards for Culturally and Linguistically Appropriate Services (CLAS) , and serves as a future building block that advances the conceptualization and practical understanding of how to assess cultural competence at the organizational level.

The specific objectives of this project were to: 1) develop an analytic framework for assessing cultural competence in health care delivery organizations; 2) identify specific indicators that can be used in connection with this framework; and 3) assess the utility, feasibility and practical application of the framework and its indicators. The project was implemented through a contract with The Lewin Group, Inc. HRSA's Office of Minority Health and Office of Planning and Evaluation provided both oversight and substantive input to the project.

The project team employed several methods to reach these objectives. The first was a synthesis of over 120 published and unpublished literature sources to provide a resource document for the field and to inform the project team's initial decisions in developing an Assessment Profile. The results of this review are presented in an interim project report entitled, Measuring Cultural Competence in Health Care Delivery Settings: A Review of the Literature. This report, available at http://www.hrsa.gov/omh, provides documentation that supports the approach taken in this project. This documentation is not repeated in this companion final report.

Another important aspect of this project was the input of an organized Technical Expert Panel (TEP) comprised of individuals with widely recognized expertise on issues related to cultural competence (Appendix A). The TEP was not a consensus panel, but rather a group of advisors that shared information, insights, and opinions on an ongoing basis through meetings and written commentary.

The project team also held discussions with (or received input from) a range of private- and public-sector persons knowledgeable about cultural competence and measurement who served as key informants on the content of the Assessment Profile (Appendix B). Further, the project included input from a Workgroup of HRSA's Cultural Competence Committee (Appendix C).

Finally, the project team made site visits to best practice settings, i.e., health care delivery sites that have been recognized for their innovations in cultural competence. Visits were made to both HRSA-funded and non-HRSA-funded sites, including: Betances Health Unit, Inc; Community Health of South Dade, Inc.; International Community Health Services; Kaiser Permanente, San Francisco; Multnomah County Health Department; South Cove Community Health Center; and Sunset Park Family Health Center Network (Appendix D). These sites varied in size, auspices, populations served, and history and breadth of cultural competence activities. The visits were not evaluations of the sites, but rather opportunities to get practical, experience-based perspectives about assessing cultural competence and the utility and feasibility of the Assessment Profile.

The project used an iterative process in developing the Profile. In the first stage, a preliminary assessment framework and initial set of indicators were developed based on the literature review. In the second stage, the preliminary framework, set of indicators, and related assumptions were refined following feedback from the Technical Expert Panel, the HRSA Workgroup, and key informants. The framework and indicators resulting from this second stage were further revised based on advice from the Technical Expert Panel members, as well as input from a range of persons during the site visits.

For the purposes of this project, cultural competence is defined as "a set of congruent behaviors, attitudes, and policies that come together in a system, agency, or among professionals and enable that system, agency, or those professionals to work effectively in cross-cultural situations." In developing a tool to assess cultural competence in the context of health care, the project team concentrated on the organizational level rather than the individual level. The project was undertaken with the following perspectives. First, organizational cultural competence is an integral component of systematic patient-centered care and has the potential to improve access to care, quality of care, and, ultimately, health outcomes. Second, organizations can serve as the "engine" driving the development and maintenance of individual provider cultural competence by providing the managers, policies, and systems to support the realities of culturally competent encounters. Further, organizational cultural competence not only affects service delivery, but can be a mechanism for maintaining and increasing an organization's market share among diverse cultural groups. Thus, "cultural competence service delivery is both a quality and business imperative" that should be incorporated at every level of an organization. Finally, the assessment or measurement of cultural competence is an important aspect of organizational behavior and should be a regular management function. The result of such assessment is organizational learning that can lead to continuous service and management improvements by providing information for decision-making. The Assessment Profile presented in this report offers an approach to obtaining such information.

B. About the Organizational Cultural Competence Assessment Profile

The Assessment Profile is an analytic or organizing framework and set of specific indicators to be used as a tool for examining, demonstrating, and documenting cultural competence in organizations involved in the direct delivery of health care and services. The Profile is most pertinent for organizations that are community-oriented. For the purposes of this project, "community" is defined as the population residing in the geographic areas served by or potentially served by a health care delivery organization. While this project was funded by HRSA, the Profile has relevance beyond HRSA-funded programs to other community-oriented health care delivery organizations.

In answering the question "How do we know cultural competence when we see it?," the Profile addresses whether an organization has or exhibits the particular features that should be evident or manifest in a culturally competent organization across the spectrum of critical areas or domains of organizational functioning. Use of the Profile is most appropriate for a health care delivery organization's internal assessment of cultural competence. At a general level, the Profile can help organizations frame and organize their perspectives and activities related to the assessment of cultural competence. More specifically, the Profile can be used in routine performance monitoring, regular quality review and improvement activities, assessment of voluntary compliance with cultural competence standards or guidelines, and periodic evaluative studies.

The Profile is not intended to be prescriptive; rather, it is designed to be adapted, modified, or applied in ways that best fit within an organization's context. However, while the Profile can be used in whole or in part, the full application enables an organization to comprehensively assess its level of cultural competence.

The Profile may also be of interest to entities such as health plans, accrediting bodies, oversight agencies, community groups, and others interested in promoting quality of care through cultural competence at the direct care level because it provides a potential way to define expectations and standards and assess the extent to which these are met. However, at the Profile's current stage of development, it is not recommended for use by external stakeholders to formally evaluate health care organizations.

The Profile is presented in a tabular/matrix form that classifies indicators by critical domains of organizational functioning and by whether the indicators relate to the structures, processes, outputs, or outcomes of the organization. The indicators reflect the view that the assessment of cultural competence should encompass both qualitative and quantitative data and assess progress or movement toward achieving results, not just the end results. The outcome indicators focus on intermediate outcomes for which a plausible or credible connection/attribution to cultural competence can logically be made. Broader and more ultimate outcomes, such as the elimination of health disparities, are not included in this Profile because of the multiplicity and complexity of factors that can influence such longer-term outcomes.

To develop the Profile, the project team employed an additive process typically used in developing assessment tools that first involved the formulation of performance areas to be assessed and then the development of performance indicators for each area. This work is a first step along a continuum that includes further refinement of the indicators, identification of particular qualitative or quantitative measures for each indicator, identification or development of data sources and data collection instruments, and formal field testing. The scope of this project did not allow for these additional steps. Thus, the Assessment Profile should be considered a work-in-progress.

II. KNOWING CULTURAL COMPETENCE WHEN WE SEE IT: COMPONENTS OF THE PROFILE

The Assessment Profile has three major components: 1) domains of cultural competence; 2) focus areas within domains; and 3) indicators relating to focus areas, by type of indicator.

A. Domains and Focus Areas: Where to Look for Evidence of Cultural Competence

The project team identified seven domains (or performance areas) for assessing cultural competence. These are the critical arenas or spheres in which cultural competence should be evident or manifest in an organization. These seven domains reflect to a great extent, although not exclusively, the underlying construct of cultural competence in health care delivery organizations and are areas to examine for evidence of cultural competence. Within each of the domains, the project team developed several focus areas. Focus areas are the substantive topic areas that characterize the domain. They are more specific arenas to examine for evidence of cultural competence and form the particular focus for identifying indicators. The Profile's domains are described below.

Organizational Values: An organization's perspective and attitudes with respect to the worth and importance of cultural competence and its commitment to provide culturally competent care.

Governance: The goal-setting, policy-making, and other oversight vehicles an organization uses to help ensure the delivery of culturally competent care.

Planning and Monitoring/Evaluation: The mechanisms and processes used for: a) long- and short-term policy, programmatic, and operational cultural competence planning that is informed by external and internal consumers; and b) the systems and activities needed to proactively track and assess an organization's level of cultural competence.

Communication: The exchange of information between the organization/providers and the clients/population, and internally among staff, in ways that promote cultural competence.

Staff Development: An organization's efforts to ensure staff and other service providers have the requisite attitudes, knowledge and skills for delivering culturally competent services.

Organizational Infrastructure: The organizational resources required to deliver or facilitate delivery of culturally competent services.

Services/Interventions: An organization's delivery or facilitation of clinical, public-health, and health related services in a culturally competent manner.

Exhibit 1 lists the focus areas for each domain of the Profile.

Exhibit 1: PROFILE DOMAINS AND FOCUS AREAS

B. Indicators by Type: Specific Evidence to be Used in Assessing Cultural Competence

Within each of the domains and focus areas, the project team identified specific indicators of cultural competence in health care delivery organizations. Indicators are the particular observable or measurable characteristics of an organization that signify cultural competence. The indicators directly answer the question: "How can cultural competence be monitored and assessed?" by identifying the specific items on which information is to be gathered. The project team identified only those indicators deemed as critical and reasonable exemplars of organizational cultural competence to minimize complexity and facilitate the use of the Profile. While the indicators included in no way represent the universe of indicators possible for each domain and focus area, they still reflect a comprehensive view of cultural competence. Indicators were also selected because of their particular relevance to cultural competence. For the most part, more generic indicators of performance, quality, or access are not included. Further, the Profile contains indicators that are either qualitative or quantitative in nature. (Again, it is important to note that the Profile does not present performance measures, which typically specify baselines and target values for those indicators that are quantifiable.)

Indicators in the Profile were classified into four types: 1) structure indicators, 2) process indicators, 3) output indicators, and 4) intermediate outcome indicators.

Structure indicators are used to assess an organization's capability to support cultural competence through adequate and appropriate settings, instrumentalities, and infrastructure, including staffing, facilities and equipment, financial resources, information systems, governance and administrative structures, and other features related to the organizational context in which services are provided.

Process indicators are used to assess the content and quality of activities, procedures, methods, and interventions in the practice of culturally competent care and in support of such care.

Output indicators are used to assess immediate results of culturally competent policies, procedures, and services that can lead to achieving positive outcomes.

Intermediate outcome indicators are used to assess the contribution of cultural competence to the achievement of intermediate objectives relating to the provision of care, the response to care, and the results of care.

The Profile presents structure, process, and output indicators for each domain. Intermediate outcome indicators are assumed to cut across domains and, therefore, are not categorized by domain. Instead, intermediate outcome indicators are categorized by perspective, i.e., by whether they are organizational-level, client-level, or community-level outcomes. Exhibit 2 depicts the components of the Assessment Profile.

Exhibit 2: ASSESSMENT PROFILE COMPONENTS

III. KNOWING CULTURAL COMPETENCE WHEN WE SEE IT: THE ASSESSMENT PROFILE

The complete Assessment Profile is provided in Exhibit 3, Parts I and II. Part I presents the structure, process and output indicators by domain. Part II presents the intermediate outcome indicators by perspective. In reviewing the Profile, several important factors should be kept in mind. First, given the multi-faceted and interconnected nature of cultural competence, the domains tend to overlap with one another and do not suggest mutually exclusive categories. Therefore, specific indicators might fit well within more than one domain. However, despite the interconnected nature of the domains, the indicators are positioned in the domain for which there is the most relevance and applicability. The presence of "shaded cells" in the Profile matrix does not imply missing information, but rather that the project team did not identify any particularly salient indicators for that cell. Whether to develop indicators for these "shaded cells" in the future should be determined based on issues of salience, appropriateness, and feasibility. In some cases, additional indicators may not be warranted.

Part I: Structure, Process and Output Indicators

DOMAIN: Organizational Values

An organization's perspective and attitudes regarding the worth and importance of cultural competence, and its commitment to providing culturally competent care.

INDICATORS

Information/Data Relevant to Cultural Competence" may include the following: ethnic/racial demographics, client language preference, epidemiological data related to various cultural groups served, community needs assessment, etc.

Part I: Structure, Process and Output Indicators (Cont'd)

DOMAIN: Governance

The goal-setting, policy-making, and other oversight vehicles an organization uses to help ensure the delivery of culturally competent care.

INDICATORS

Part I: Structure, Process and Output Indicators (Cont'd)

DOMAIN: Planning and Monitoring/Evaluation

The mechanisms and processes used for: a) long- and short-term policy, programmatic, and operational cultural competence planning that is informed by external and internal consumers; and b) the system and activities needed to proactively track and assess an organization's level of cultural competence.

INDICATORS

*"Information/Data Relevant to Cultural Competence" may include the following: ethnic/racial demographics, client language preference, epidemiological data related to various cultural groups served, community needs assessment, etc.

Part I: Structure, Process and Output Indicators (Cont'd)

DOMAIN: Communication

The exchange of information between the organization/providers and the clients/population, and internally among staff, in ways that promote cultural competence.

INDICATORS

Part I: Structure, Process and Output Indicators (Cont'd)

DOMAIN: Staff Development
An organization's efforts to ensure staff and other service providers have the requisite attitudes, knowledge and skills for delivering culturally competent services.

INDICATORS

Part I: Structure, Process and Output Indicators (Cont'd)

DOMAIN: Organizational Infrastructure
The organizational resources required to deliver or facilitate delivery of culturally competent services.

INDICATORS

Part I: Structure, Process and Output Indicators (Cont'd)

DOMAIN: Services/Interventions
An organization's delivery or facilitation of clinical, public-health, and health related services in a culturally competent manner.

INDICATORS

Part II: Intermediate Outcome Indicators

INTERMEDIATE OUTCOME INDICATORS

IV. OBSERVATIONS AND CONCLUSION

The Assessment Profile builds on previous work and evidence in the field, as documented in the project's Review of the Literature, and is informed by input from many individuals with varying perspectives and expertise, including service providers and managers, researchers and analysts, policy makers, and others interested in developing, monitoring, and assessing cultural competence in health care delivery organizations. Listed below are several observations related to cultural competence assessment and the Profile that emerged from the site visits and the discussions with the Technical Expert Panel, HRSA Workgroup, and key informants.

A. Key Observations

Assessment is Not an Isolated Event

• Commentators generally agreed that the assessment of cultural competence should not be considered an isolated event, but rather a continuous process that is emphasized and integrated in an organization's overall assessment activities. Cultural competence assessment, like other significant management activities, should be clearly identifiable and targeted to garner the leadership and resources required, while being an integral part of an organization's regular performance and quality assessment activities.
  
  Importance of Assessing Institutionalization
  
  
• Many commentators indicated that it is important to assess the "institutionalization" of cultural competence in an organization, i.e., the extent to which cultural competence is an integral part of the organization's service, management and business functions. They noted that the Profile begins to address this phenomenon given its scope of indicators and the several indicators relating to integration.
  
  Validation of the Components of the Profile
  
  
• The perspectives and activities of the health care sites visited for this project give credence to the Profile's seven evidence-based domains as appropriate performance areas for assessing cultural competence. The sites emphasized the importance of assessing the domain of Organizational Values as the necessary precursor to culturally competent performance. In particular, dedicated leadership for championing and implementing cultural competence and cultural competence-related data collection and analysis were noted as two critical indicators of an organization's commitment to cultural competence.
  
  
• The site visits also supported the credibility of the Profile's focus areas and specific indicators. Either through the range of activities conducted or planned by the sites or through site recommendations, every indicator was confirmed as important evidence of cultural competence. Sites often suggested additional indicators for the Profile that reflected what they viewed as important to assess and monitor.
  
  
• The inclusion of structure, process, and output indicators in the Profile was considered a strength by the TEP, HRSA Workgroup, key informants, and persons at the sites. Individuals consistently noted that it is just as important to measure progress on the journey towards cultural competence as well as the results of cultural competence. There was similar agreement that emphasis on intermediate outcomes vs. ultimate outcomes was the more appropriate focus for the Profile because of the plausibility of attributing these more proximate outcomes to cultural competence-related activities.
  
  Potential Uses of the Profile
  
  
• A contribution of the Profile is the organizing framework it provides and the ability of organizations to use it to systematically assess their cultural competence. The Profile can assist organizations in identifying the critical elements for measuring cultural competence. Its application can help organizations gauge the level of their cultural competence and provide guidance on steps to be taken to achieve greater cultural competence. The fact that the Profile is comprehensive, while including substantial depth, was often noted as valuable in providing a holistic view of the complex construct of cultural competence at the organizational level.
  
  
• In addition to its use in structured quality assurance and other performance measurement activities, the value of the Profile as a "readiness" tool in helping organizations respond to mandates and standards was highlighted during the site visits. For example: one site reported that the Profile helped it prepare for an accreditation visit by the Joint Commission on Accreditation of Healthcare Organizations (JCAHO); another noted its use of the Profile to understand how well the site was doing relative to the National Standards for Culturally and Linguistically Appropriate Services (CLAS).
  
  
• The Profile is viewed as useful to organizations serving multiple cultural/ethnic groups as well as those serving a single cultural/ethnic group. The Profile captures many generic aspects of cultural competence that are pertinent whatever the specific population served. Even in serving a single cultural group, intra-group diversity and change should be taken into account and assessed in ways suggested by the Profile. Additionally, sites noted that the Profile could be beneficial in assessing their progress in adapting to the ever changing demographics of their communities.
  
  
• The Profile is potentially useful for organizations at different levels of cultural competence development. The site visits suggest that more "mature" organizations, in which cultural competence activities and assessment are largely institutionalized and integrated, could use the Profile as a mental checklist to assure the critical elements suggested by the Profile are captured in their quality assessment activities. Organizations that are at earlier stages in their cultural competence development might use the Profile in more explicit cultural competence assessment activities as they work toward institutionalization and integration of cultural competence activities.
  
  
• While most applicable to organizations involved in health care service delivery, with further development, the Profile can be used by organizations such as accrediting bodies, to define standards of cultural competence for use in evaluating health care delivery organizations. When cultural competence assessments become part of the public domain through such external assessments, their findings can create opportunities for broader systems change.
  
  Feasibility of Applying the Profile
  
  
• The flexibility offered by the Profile contributes to its practicality/feasibility. Organizations can pick-and-choose one or more aspects of the Profile for assessment depending on where the organization is in its stage of development or based on other organizational needs and resources. However, to fully assess cultural competence, an organization should address or revisit all the domains to understand the extent to which they are culturally competent.
  
  
• Where to start?" was a question raised given the comprehensiveness of the Profile. The TEP members and the literature suggest a number of considerations for selecting a starting point in the assessment of cultural competence. These include taking into account: the organization's assessment of the importance of particular indicators, the feasibility of implementing the indicators, and the temporal order in which cultural competence activities will or have taken place.
  
  
• All sites were engaged in some aspect of tracking, monitoring or assessment that mirrored aspects of the Profile. In addition, the Profile's focus on structures, processes, outputs, and outcomes is a framework familiar to and used by health care delivery organizations. These factors and the ability to apply the Profile in whole or in part make application of the Profile less daunting than it might be otherwise.
  
  
• Some sites, however, noted that data-related factors would affect their ability to fully use the Assessment Profile. While inadequacies in management information systems, especially integration across business and clinical functions, were noted, the lack of "analytic capacity" or persons to collect and analyze the data was particularly highlighted as a limiting factor. Such limitations are, of course, not unique to an organization's cultural competence assessment, but apply more generally to performance and quality measurement efforts.
  " The relative lack of reliable and widely accepted data collection instruments, such as survey tools to assess client perspectives, is also recognized as a limitation. In addition, ensuring that existing or future instruments are developed and applied in a culturally competent manner was noted as a challenge.
  

B. Conclusion

The goal of this project was to advance the ability of health care organizations to recognize and assess cultural competence in order to inform their decisions about maintaining and improving the management and delivery of health care services to their communities. Included in the project's interim report, Measuring Cultural Competence in Health Care Delivery Settings: A Review of the Literature, is a listing of assessment tools and evaluative models that document previous and current attempts at assessing cultural competence. The Cultural Competence Assessment Profile presented in this final report contributes to the assessment field by offering a tangible and targeted approach for conducting organizational assessments and serves as another step in the development of viable assessment tools. Additional work is needed to take the Profile to the next level. This would include further refinement of the performance areas/domains and indicators, definition and validation of performance measures, identification or development of data sources and data collection instruments, and field testing the Profile. Feedback from the Technical Expert Panel, key informants, and the health care delivery sites indicates that the Profile provides a solid foundation for further development. The feedback also suggests that the Assessment Profile can be useful even in its current form as an organizing framework, a "readiness" tool, and a guide to an organization's own development of indicators and measures of cultural competence.

C. Appendix

Acknowledgements

The Lewin Group extends a special acknowledgement and thanks to the co-authors of this report, Willine Carr, DrPH, Senior Staff Fellow, Office of Planning and Evaluation and Jeanean Willis, DPM, Senior Public Health Analyst, Office of Minority Health. As the HRSA Project Officers for this project, Drs. Carr and Willis provided the leadership, guidance, and substantive input to make this a truly collaborative effort. Both served as strong advocates of HRSA's mission to increase the visibility of cultural competence as a mechanism for ensuring the delivery of high quality health care services. Their vision, insight, and dedication guided the development of a product that undoubtedly will serve to further advance the field of cultural competence measurement.

We also want to acknowledge the invaluable contributions of the project's Technical Expert Panel (Appendix A), experts who served as key informants (Appendix B), the workgroup of HRSA's Cultural Competence Committee (Appendix C), and the staff from the seven sites that graciously participated in our site visits (Appendix D). Their work on behalf of cultural competence is exemplary, and they are truly leaders in the field.
We also acknowledge two former members of The Lewin Group project team, Susanna Ginsburg, MSW and Yolanda Partida, Ph.D., who were instrumental in the initial development of this project.

.Prepared for:

The Health Resources and Services Administration
U.S. Department of Health and Human Services
Parklawn Building
5600 Fishers Lane
Rockville, Maryland 20857