Chapter IV. Data to Support Work on Disparities

To reduce disparities, firms need to know what disparities exist and make changes in response.  Of course, concepts become more complex in execution.  Available firm data on the race and ethnicity of members are limited, making it difficult to measure member disparities in care processes and outcomes by race or ethnicity.  In addition, several firms reported that important gaps exist in understanding disparities.  For example, there is a limited evidence base to determine how best to improve care to reduce disparities in outcomes associated with members' racial or ethnic characteristics.  There is little agreement on how best to measure effective reductions in disparities because absolute change in outcomes, and relative change in outcomes for one group versus another, may yield different conclusions.  Faced with these constraints, there is a tension between taking the time to develop better measures and understanding of disparities and moving more immediately to implement interventions believed to have some promise in reducing disparities even if the evidence or ability to measure their effects is limited.

Measuring disparities was one of the four main areas the Collaborative sought to address (Figure IV.1). A major focus of Phase I involved RAND working with firms to better estimate race and ethnicity for their members in order to assess disparities in diabetes care (using HEDIS indicators) and potentially other care.  The results of this analysis helped inform firm leadership and in some cases formed the basis for intervention. Several firms saw weaknesses in what geocoding and surname analysis provided them; such limitations actually encouraged them to begin collecting their own data on the race and ethnic composition of their members.  Few firms shared their HEDIS data on diabetes by race and ethnic subgroup with others.

This chapter provides an overview of findings.  We review why capturing racial and ethnic data to measure disparities poses a challenge for firms.  We discuss why geocoding and surname analysis were an initial focus of the Collaborative, and what these approaches did and did not accomplish.  We then summarize, using the available information, firms' current status in collecting patient-level racial and ethnic data.  We conclude with a discussion of the Collaborative's generally unsuccessful effort to motivate firms to report HEDIS measures on diabetic members to each other.

Readers should note that Chapter IV focuses primarily on measuring disparities among firms' commercial members, the focus of the Collaborative.

A. Summary of Findings

Gathering the necessary data to analyze disparities consumed much of Phase I of the Collaborative.  Geocoding and surname analysis took much longer than anticipated and were controversial with sponsors for at least that reason, yet many firms found the results beneficial.  Few firms had good data on the racial and ethnic characteristics of their members, but most assumed, because of national research, that disparities existed.  The majority of firms involved in the effort at geocoding and surname analysis shared their results with firm leadership and said that the findings elevated the disparities issue within the firm.  A few firms were disappointed in the results of geocoding and surname analysis because the technique was not sufficiently robust to provide insight relevant to patterns in their market.  (A few also expressed disappointment that the geocoding/surname analysis yielded only proxy data that could not be used to target specific members for specific interventions.)  Often, however, firms were able to use the results to some end.  Although they were disappointed that the work took as long as it did, firms blamed themselves as much as RAND for delays, and perceived that on balance the process had a favorable benefit/cost ratio.

The Collaborative supported presentations of what leading firms were doing to collect race and ethnicity data directly from their members, but did not do more to directly support some firms' desire for assistance in modifying national policy to make it easier for them to obtain data on the race and ethnicity of their members.  This omission was a point of contention among some participants in the Collaborative.  Phase II will place more emphasis on primary data collection related to disparities, including efforts to define aspects of the way firms approach this to promote consistency.

The Collaborative did not succeed in getting all or most firms to share their data for common HEDIS measures.  Such sharing was very important to sponsors and some support organizations, but firm buy-in appears to have been lacking from the beginning.  The experience in the area of common measures highlights the challenges of communication and conflicting goals among participants in the Collaborative.

Return to Contents

B. The Challenges in Capturing Racial and Ethnic Data

National policy on whether, how, and what to collect about the racial and ethnic characteristics of the population served by the health care system was still evolving over the period in which the Collaborative proceeded, a fact that shaped the opportunities and challenges faced by firms seeking such data (Appendix B).  Firms found it easier to capture racial and ethnic data for the Medicare and Medicaid populations than for their commercial members, because the Centers for Medicare and Medicaid Services (CMS) collected some of these data for Medicare beneficiaries and required states to provide them for Medicaid beneficiaries (Bierman, Lurie, Collins, and Eisenberg 2002; AHIP 2004).  Firms sometimes maintained race and ethnicity data for particular subgroups of their commercial members.  For example, many firms in the Collaborative structured protocols for disease management programs so that such data were collected as part of a health risk appraisal.  However, these data were not necessarily stored in ways that made them accessible across the firm.

Despite isolated efforts to secure direct data on the racial and ethnic composition of their membership, few, if any, national health plans had (or currently have) complete data on the racial and ethnic composition of all or even most of their members.  Collecting racial and ethnic data requires both a process for obtaining information and a mechanism for maintaining and sharing the information across the organization.

Most commercial members enroll through employer groups.  Some employer groups have racial and ethnic data on their employees and may be willing and legally able to share the information.  The data tend to be specific to subscribers, not to others covered by the policy, such as a spouse or children.  Further, unless employers require subscribers to re-enroll affirmatively each year, new requests for information will generate data only for those filing that year—those new to coverage, those changing family status, or those switching plans.  Given the difficulties in reaching agreements with a broad range of purchasers, some participating firms with an interest in disparities have started by obtaining data on their own employees.

Firms can obtain racial and ethnic data by asking members directly, although they must comply with state-level legal restrictions or approval requirements.  After member enrollment, the collection of racial/ethnic data is subject to fewer legal constraints, especially if the response is voluntary.  Another alternative, especially for firms with strong linkages with providers, is to collect such data at points of service and possibly incorporate it as part of an electronic medical record.  Most firms sponsoring health plans however do not have such strong linkages to providers.  Regardless of their strategy for obtaining data, all firms must meet federal and other requirements that provide appropriate safeguards related to privacy and other concerns.  Collaborative firms have found that even when they decide to collect data, there are no perfect strategies for doing so; despite the best intentions, progress is slow.

Firms also face challenges in maintaining and manipulating racial and ethnic data, especially if their systems were not initially designed to support such work.  Unless the firm's IT platform has one or more fields for entering data on race and ethnicity, appropriate fields must be added, a process that is typically costly and time-consuming; in fact, such an addition may not be possible if the vendor of an old system no longer maintains it, as one firm found.  In addition, there may be more than one IT platform in place across a firm and its affiliates, thereby limiting the pooling of data and access to it.  Provider networks are complex; consequently, only a small share of affiliated providers may have racial/ethnic data or be willing to share the information.  Willing providers may have IT platforms that are incompatible with those of the firm.  Such inconsistencies occur even if the firm has providers integrated with the health plan. Many firms sponsoring health plans were themselves formed from mergers spanning several companies over several years.  Each legacy firm may bring its own IT platform. In many cases where integration is a goal, the process is ongoing. 

At the Collaborative's inception, only a few participating firms had begun to collect data on the race and ethnicity of all of their members, with a few others planning to do so.  Aetna had already started to collect members' race and ethnicity, which helped motivate other firms' interest in the Collaborative.  Another regional plan was beginning to collect data, and two firms had policies in place that supported such data collection but found implementation challenging, in part because of competing demands.  Of participating firms, only the sole Medicaid dominant plan in the Collaborative had such data for its entire membership—and that was because it could obtain this information from state agencies.

Recognizing that capturing racial/ethnic data would take time, RAND offered to work with interested firms in the early days of the Collaborative to apply geocoding and surname analysis to give participants a preliminary understanding of any disparities in their firm. RAND staff hoped that doing so would reinforce firms' perception that disparities were a problem warranting their attention, and motivate efforts to reduce disparities. Geocoding/surname analysis was also a technique in which RAND's staff were personally interested and experienced (Fremont and Lurie 2004).

Return to Contents

C. Experience with Geocoding and Surname Analysis

1. Geocoding and Surname Analysis, and RAND's Approach

The goal of geocoding and surname analysis is to allow firms to generate estimates specific to the race and ethnicity of their members.  The estimation technique assumes that firms already have the outcome data of interest for the population—such as membership-based HEDIS measures for diabetes—and lack mainly descriptor information on the racial and ethnic characteristics of members for whom outcomes are reported.  In short, geocoding and surname analysis use proxy information that is known for members to estimate racial and ethnic characteristics.  These data are then linked to outcome measures, such as HEDIS.  HEDIS measures are more likely to be captured for HMOs than for other products because quality improvement goals, measures, and requirements are more developed there than elsewhere.  Disparities are thus easier to measure in HMOs and other products that employ such measures.

RAND staff explained that most of the agreements with firms were structured such that firms provided individual surnames and physical addresses for relevant members—specifically, those with diabetes (the Collaborative's target population) and others of interest to the firm.  Firms, rather than RAND, defined whom to include in the population of interest.  RAND staff then analyzed surnames to identify Latinos and Asians, and converted member addresses to census block groups (of around 1,000 people).  RAND next examined data on the census block of residence for members not classified as Latinos or Asians through surname analysis.  While geocoding lends itself to several approaches, RAND's technique for the Collaborative coded as African American individuals who reside in census block groups with a population that is more than two-thirds African American and others as white or other.⁸  Based on its geocoding and surname analyses, RAND classified members into one of four mutually exclusive categories:  African American, Asian, Hispanic, or white/other.

RAND returned the identifying information to the firm with its racial/ethnic code. In most cases, RAND did not have access to firm HEDIS data, as firms were sensitive about releasing such information.  With the information from RAND, firms were to construct HEDIS diabetes indicators for the relevant population. HEDIS includes four process measures for diabetes (HbA1C monitoring, lipid profile, diabetic eye examination, and urine protein) and two outcome measures (HbA1c level controlled and lipid level controlled).  Some firms calculated the subset of HEDIS measures that could be computed with administrative data without chart audits, since measures requiring chart audit can be expensive.  Some firms provided information on a broader set of members that went beyond just those in the commercial market with diabetes and used the information to develop a broader set of measures about disparities.

For firms that were willing to share HEDIS data, RAND could do more to help them with analysis.  For a few firms that expressed interest, RAND incorporated the data into a mapping tool to help firms visually analyze variations in HEDIS outcomes across geographic areas with diverse racial and ethnic characteristics.  Based on firm experience in the first round of estimation, some firms contracted with RAND to provide specialized support whereas others either had or built such capacity internally or rejected the geocoding/surname analysis approach entirely. To the best of our knowledge, RAND has not developed a report documenting the work of the Collaborative on geocoding and surname analysis—perhaps because of firm agreements and sensitivities about public reports on their internal processes and data, or other reasons.  As a result, information about this process comes from firm presentations to the Collaborative or interviews conducted for the evaluation.

RAND staff members indicated that geocoding works best in highly homogeneous areas—with high concentrations of members in particular racial and ethnic groups—although they believe that it also can be used effectively elsewhere, particularly with recent refinements.  Given that geocoding is based on geography rather than on the individual, the technique is best suited for comparing HEDIS or other outcome measures across geographic areas that are known to vary in racial/ethnic composition.  Firms can map areas to visually display the diversity therein and identify priorities for interventions.  Mapping by geographic coordinates also allows firms to merge many other kinds of data available geographically.  The geocoded/surname analyzed data are typically less useful as longitudinal measures of outcomes for person-specific interventions because of the assumptions used in constructing racial and ethnic identifiers using geocoding and surname techniques.

2. Firm Experience with Geocoding and Surname Analysis

At the July 8, 2004 meeting, RAND proposed to work with firms to support the analysis of racial and ethnic disparities by using geocoded and surname analyzed data; RAND then formed a workgroup of interested firms.  All firms in the Collaborative participated in the geocoding and surname analysis process except one that already had race/ethnic data for all its members and a second that was actively engaged in capturing such data nationwide.⁹  Originally intended to provide analysis that could be used in the first Collaborative meeting in September 2004, the work took much longer to complete (as discussed later).  The delay reflects an often considerable underestimate of the time required to establish the necessary legal agreements with firms to share data and to have the firms' information systems generate the member data upon which racial/ethnic proxies are based.

All seven of the firms originally participating in geocoding/surname analysis ultimately received data with geocodes and surname identifiers for at least one time period and had an opportunity to use the data to develop measures of disparities. (An eighth firm recently began talking with RAND about developing such analysis.)  In our round two interviews, we discussed the experience with geocoding and surname analysis with staff from each firm involved in the effort.  The interviews varied in specificity and did not allow us to describe firms' geocoding experiences in detail with any rigor or consistency.¹⁰  They do, however, provide a good indication of the range of firm experiences with the process (Table IV.1).  Since that time, some of the firms have continued their geocoding and surname analysis work and several have become more involved in the use of mapping techniques for visual display and analysis of data by neighborhoods and other areas.

Focus of Work. Firms varied markedly in the content and scope of the data they provided to RAND for geocoding and surname analysis.  While some firms restricted their scope to diabetes, others went beyond this and included events such as Acute Myocardial Infarction (AMI).  One regional firm included all of its adult commercial members in the Consumer Assessment of Healthcare Providers and Systems (CAHPS®) sample frame for its dominant state, along with a subgroup of Medicare enrollees and a targeted group of Medicaid patients.  This firm and a few others solicited support for several years of measures; others appear to have limited their focus to a single year.  Firms structured their requests to match their needs.  For example, one firm excluded members for whom it already had racial/ethnic data, and another used the rules it applies in defining all those categorically eligible for disease management.  Many firms included in their request only a subset of their plans or geographic regions so that they could limit burden, address divergent interests among their affiliates, or handle any inconsistencies in IT platforms.  Regardless of the variation, the total number of lives that appear to have been included in the exercise is impressive for the potential—provided the technique works—to understand the disparities by race and ethnicity in firms.

Analytic Sophistication. The geocoding and surname analysis process was structured in such a way that its value depended at least partly on what firms did with the data they received.  Firms varied in the analytic skills and resources available to support the analysis and in their preferences for support.  At least half the firms had some experience with geocoding, typically for African Americans.  A few of these firms preferred their own geocoding techniques for designating race to those used by RAND.  Analysts in one firm, for example, relied on RAND only for surname analysis and used the firm's own probabilistic techniques to assign racial codes.¹¹  Another firm favored the same approach.  Some firms did extensive analysis with the data.  At least two firms examined the relative role of race and socioeconomic status in contributing to disparities and their differential effects on diabetes process measures versus outcomes measures.  The firms used the results to develop a better understanding of disparities and the approaches most likely to be effective in designing interventions.  Firms that could not access sufficient analytic support did far less analysis.  For example, one large firm was limited in the programming resources available for geocoding-related analysis and found its progress substantially delayed.  It had to purchase additional help from outside vendors for tasks other firms could easily handle in-house.

Perceptions of RAND Support. Those involved with the geocoding and surname analysis project generally expressed satisfaction with RAND's support.  They felt that RAND staff met their expectations and that the help was valuable.  They also reported that the exercise was not very burdensome.  The main substantive disappointment we heard from a few particularly sophisticated firms focused on the fact that RAND staff did not provide more specific technical guidance, such as how to judge the substantive rather than statistical significance of a disparity. One firm perceived the support to focus more on the rigor required for research than the firm's needs.  Otherwise, the main limitation, as noted, related to the delays associated with establishing the necessary administrative agreements with RAND to support the geocoding and surname analysis work. Firms typically attributed delays equally to RAND and their own administration. Delays were likely inevitable as firms sought to satisfy the Health Insurance Portability and Accountability Act's (HIPAA) privacy and other concerns. However, some reports suggest to us that management and administrative staff at the participating firms and at RAND could have been more nimble in moving the process forward.

3. Ultimate Value and Use of Geocoding and Surname Analysis

While firms varied in how valid they considered the results of geocoding and surname analysis for their markets, they generally said that they benefited from their involvement in the process.  They perceived a positive benefit/cost ratio or provided examples suggesting as much.

Perceived Value. Most firms involved in geocoding and surname analysis stated that, despite the limitations of the resulting data, the technique was sufficiently robust to support the intended uses of the data.  The firms shared their results with firm leaders.  In some cases, the results provided new and valuable insights that helped firms better conceptualize the issues behind disparities.  In others, the findings confirmed what firms already knew, reinforcing the importance of work in the disparities area, particularly among non-clinical staff who might need more convincing.  Most firms reported that the analyses revealed some disparities.  A few were pleased that disparities were less extensive than they thought or than in the general population.  Firms also found value in analyses showing specific geographic areas that were more or less problematic on different measures.  Firms using mapping found it valuable in graphically illustrating disparities for internal discussion.

Two firms and some staff in a third firm found the geocoding results disappointing.  In one firm, the estimated proportion of African Americans based on geocoding was substantially below what the firm derived from patients with self-reported data; as a result, firm staff did not use the geocoded data.  Another firm, perhaps unrealistically, had not realized that the analysis would be less useful in supporting member-specific rather than geographically targeted interventions.  In this firm and another with a geographically diverse service area, staff in certain regions felt that the geocoding technique was not well suited to their market.  They explained that the disappointing analyses stemmed from markets with very heterogeneous residence patterns by race/ethnicity.  Most commonly, geocoded results were at issue.  Some had only limited diversity in their membership; therefore, if the strategy for a particular subgroup did not work, the exercise had no other value.  Firms with particularly diverse enrollments were also disappointed if the technique did not yield the sensitivity to isolate desired subgroups. (As mentioned before, RAND perceives that recent refinements to the methods address some of these concerns.)

Applications of the Analysis.  For most firms—whether or not they found the results compelling—involvement in geocoding and surname analysis proved valuable.  By our round two interviews, two firms had already used the data to formulate pilot projects, and several more were in the process of doing so. Others said that they planned to use the information to help them further identify needs and areas to target. One of the firms that found the results invalid used its failure as a vehicle for reinforcing its decision to capture primary data on member race and ethnicity; respondents from two other firms similarly commented that limitations in geocoding and surname analysis solidified firm commitment to primary race and ethnicity data collection.  Another firm had not yet found the data useful, but it reported that the process enhanced communication among midlevel staff responsible for such analyses, leading to an ad hoc group that is encouraging further firm investment in analyzing disparities and designing pilot interventions.  This firm said that improved communication and the willingness to consider allocating more resources to disparities work were a direct result of participation in the Collaborative. 

Future Plans for Geocoding/Surname Analysis. The Collaborative will not support firms in their individual efforts at geocoding and surname analysis during Phase II.  However, of the firms that used these techniques in Phase I, over half have plans to continue the analysis, in some form.  RAND staff indicated that at least half of the firms decided to use the mapping tool that RAND developed, one firm based on its own earlier experience and the others after another firm that used the tool during Phase I gave a presentation of their results at the June 2006 meeting.  The lead contact from another firm indicated that they already had a similar mapping tool, but would be interested in continuing to do geocoding/surname analysis if the financial burden of doing so were minimal.  One other firm generally lagged behind the others in this work during Phase I, due to internal reorganization, but has plans to continue geocoding and surname analysis with RAND under a separate contract, unassociated with their commitment to the Collaborative.  This firm has hired an analyst to help it gain internal capacity to study disparities and hopes to use the RAND contract for training and other help getting started.  Although, as discussed later, all but one of the firms have begun or have plans to begin primary race and ethnicity data collection, putting such systems in place takes time; current and continued work around geocoding and surname analysis holds appeal in that it allows firms to begin to address disparities in their minority populations, while developing longer-term systems to collect and maintain race and ethnicity directly from members.  However, one of the firms that used geocoding and surname analysis extensively in the past has not expressed interest in continuing it in the future.

A potential issue for firms involves how to transition from building their geocoding and surname analysis using the support provided through the Collaborative to using their own resources.  RAND's tools are not publicly available though we understand RAND has agreed to make its algorithms for assigning surnames available to firms in the Collaborative and is providing advice on vendors and low cost ways to purchase geocoding software.¹²  Because of the way our firm interviews were timed, we did not learn about firm reactions to these options.  At least two firms have contracted with RAND independently to support the geocoding/surname analysis efforts. While internalizing the function can help firms institutionalize the process, some do not have the expertise or staff to do so. In addition, converting to other software may result in inconsistencies with prior analysis, thus detracting from firms' ability to leverage past work and trend experience.

Return to Contents
Proceed to Next Section