Chapter VII. Perspectives on the Collaborative's Infrastructure and Process

This chapter examines the infrastructure and process of the Collaborative in terms of its ability to promote the goals of the Collaborative.  The underlying premise is that collective action will be more successful than each firm acting on its own. This analysis draws from our interviews with firms and other organizations, our observations during a variety of Collaborative meetings and calls, and a network analysis of relations between Collaborative members.  It examines the communication across participating firms, the tangible and intangible resources brought to or generated by the Collaborative, and the work required of participating firms. The chapter concludes with an assessment of how well the infrastructure and process of the Collaborative matched its goals, and a delineation of issues that warrant consideration in structuring future endeavors.

A. Summary of Findings

Firm responses to the network analysis and interviews clearly paint a positive picture of the Collaborative overall; firms viewed the Collaborative as contributing to their organizations' goals.  However, firms did not appear to benefit as much as they might have had they shared information more openly with each other, or had the Collaborative been better structured to facilitate substantive learning, particularly about evidence on how to reduce disparities.  To some extent, limited sharing is a function of firms' culture and the markets in which they operate. However, firms anticipated that the more specific focus of work in Phase II would facilitate better communication, as would the experience they had working with one another. 

Almost universally, firms said their biggest disappointment was the Collaborative's inability to address their interest in knowing about "what works"—particularly interventions that might reduce disparities. While some of this may be a reaction to the lack of a solid evidence-based knowledge in this area, it also appears that more could have been done to connect firms with sources and people who could provide insight on this issue and to structure agendas so that firms could learn more from one another.  The learning sessions, for example, could have included faculty presentations on what is known.  The effort required of CHCS to coordinate the Collaborative and also the large number of support organizations probably came at a cost in resources that could be devoted to more substantive support in this area.  In hindsight, it appeared that firms wanted information on what works, even if they did not want to use the Collaborative to talk about what they might do with the information.  The most contentious issue for firms involved the structure the Collaborative sought to impose and the requirements it placed on firms. Reporting requirements were a particular concern, and some firms at least viewed the cumulative number of requests from sponsor-affiliated groups to be burdensome.

The experience of the Collaborative yielded a number of valuable lessons for organizations seeking to engage in future efforts with large firms that sponsor health plans, as discussed at the end of the chapter.

Return to Contents

B. Strength of Collaboration: Insights from the Network Analysis

We used network analysis to help us understand how the Collaborative functioned.²¹  The network analysis involved asking each participant in the Collaborative—firms, sponsors, and support organizations—to complete a brief structured feedback form that included two primary sets of questions: one on the Collaborative overall and the other on the relationships among (and assessments of) other participants. The questions were asked in round two, or the end of 2005, about six to nine months before the end of the Collaborative.  (Some perspectives may have changed by the end of Phase I.) The analysis was designed to provide tools to assess the way participants interacted with one another and the support provided by diverse organizations to the Collaborative's goals. We summarize here the main findings from this analysis whose methods, findings and conclusions are documented more completely in Appendix C.

1.  Overall Perceptions of the Collaborative

Participants' responses clearly paint a positive picture of the Collaborative overall (Table VII.1).  All but one participating organization felt that the Collaborative was at least somewhat important to attaining organizational goals (Question 1).  In fact, 10 of the 15 organizations in the Collaborative (6 of the 9 firms and 4 of the 6 nonfirm organizations) reported that the Collaborative was very important or crucial for achieving organizational goals related to reducing health disparities.  On average, organizations—at least as of January 2006—felt that the Collaborative had carried out its responsibilities and commitments "to a considerable extent," with firms slightly more positive than nonfirm organizations (Question 2).

Participants were positive about their own participation in the Collaborative, although some firms acknowledged that they themselves had done less than they might have been able to. When asked to rate their own organization on carrying out responsibilities and commitments to the Collaborative, almost all organizations were very positive (Question 3).  Two firm representatives (both from national firms), however, indicated that their organizations carried out their responsibilities and commitments only to "a little extent" (one) or "some extent" (one).  Almost all respondents reported that the relationship between their organization and the Collaborative was productive and worthwhile (Questions 4 and 5); support and sponsor organizations were somewhat more positive than firms on these dimensions.  Likewise, all organizations reported that they were satisfied with the relationship between their organizations and the Collaborative to at least some extent (Question 6). 

Support and sponsor organizations reported changing or influencing Collaborative activities more than firms did (Question 7), a response that is not surprising, given the role of these organizations in shaping the Collaborative and working one-on-one with firms. In comparison, firms were more likely than other organizations to say that the Collaborative had changed or influenced their organization's activities, with six of the nine firms saying this was the case to a "considerable extent" or "great extent" (Question 8).  All organizations said that the payoffs of the Collaborative were reasonable relative to contributions, with firms somewhat more positive on this dimension than sponsor and support organizations (Question 9).

2.  Dynamics within the Collaborative

In addition to asking each participant about the Collaborative overall, we asked them to rate each of the other organizations in various ways, often involving the same questions as used for the overall Collaborative assessment as part of the structured feedback form. Our network analysis highlighted a number of key findings about how communications occurred in the Collaborative and the roles played by diverse participants (Appendix C for more detail). 

The analysis highlighted the central role that key support and sponsor organizations played in the Collaborative.  Not only were they viewed as visible and active participants in the Collaborative process, but they also appeared to act as the "glue" that held the Collaborative together.  They had the most contact with participating firms and formed the primary pathways linking participants.  The support and sponsor organizations also engaged in a substantial amount of contact with one another. Key support and sponsor organizations also played an important role in contributing to action and change among other organizations.

Though named a "Collaborative" the network analysis revealed that there was much less communication between firms than between firms and the individual sponsors or the support organizations.  On process measures such as communication, firms reported limited one-on-one interaction from firm to firm (although a few firms were seen as providing many good ideas to the Collaborative process).  Only a few firms characterized other firms as having a "considerable" influence on them or their actions.  However, firms still viewed each other as important, and most respondents reported that other organizations were carrying out their responsibilities and commitments to the Collaborative, at least to a small extent.  

A few of the firms—namely, one national and one regional firm—stood out as more important and influential members of the Collaborative than others.  Conversely, several firms consistently ranked toward the bottom on these measures.  Organizations' ratings of whether the Collaborative was productive and worthwhile and whether it yielded a reasonable payoff compared with the level of organizations' contributions were all fairly favorable and did not appear to vary greatly with by organizational standing (though one national firm with low standing tended to rate the Collaborative lower than other firms).

Return to Contents

C. Firm Perspectives On Communication Within The Collaborative

Consistency between Network Analysis and Round Two Interviews. The findings from the network analysis were consistent with the profile of the Collaborative we obtained concurrently in our second round of interviews. Given the competitive nature of the health plan industry, it is not surprising that firms generally did not communicate with one another outside of formal Collaborative meetings.  Unless firms operated in different markets (as is the case with regional firms) or had a business imperative for additional collaboration (as at least one regional organization had with a national firm), the firms were likely competitors. Firms also participated in the Collaborative for different reasons (Chapter III). Although firms did not necessarily say that they were reluctant to share information, the way they described their internal clearance processes made it clear that release of firm-specific information was an important threshold decision that needed to be approved by top leadership. 

Firm Reaction to Interim Report Findings. Because the interim evaluation report was shared with firms, they had the opportunity to see what the network analysis and other findings revealed about the way the Collaborative was functioning. We did not explicitly ask firms to react to the analysis, but we were able to gain insight into their perspectives on these issues both through discussions in subsequent meetings of the Collaborative and also through their responses to our questions on their overall views of the Collaborative in the final round of interviews.

At the final leadership meeting of the Collaborative in September 2006 (where the topic was gaining consensus on the content of Phase II), AHRQ staff raised the issue of openness of communication, noting that there wasn't as much sharing in Phase I as there could have been, both within and outside the Collaborative. This norm limited group learning and, AHRQ staffers noted, the ability to leverage firm's work in the Collaborative to generate more broad based knowledge and action on issues related to disparities.  Firms seemed to accept the communication shortfalls noted in the interim evaluation report as valid. For example, one participant (from one of the firms that was more open in their communications) remarked that company executives reviewing the interim evaluation asked "how this was a Collaborative when there is very little collaboration." In the meeting or in our interviews, a few participants noted that on occasion they heard information about other projects being pursued by participating firms and wondered why these examples had not been brought to and shared with the Collaborative.  In some cases, the communication failure could be attributed to the way agendas were set or the fact that the staff participating in the Collaborative may not have been the same as those involved in the activities of interest. But in other cases, participants were perplexed about why a firm would be more open in discussing relevant accomplishments in some venues than others, wondering if the interest was more in generating publicity than sharing and collaborating. 

Firms acknowledged the downsides of less open communication, but were frank about the dynamics that might lead to this occur. As one large participating firm noted, "It [communication] is a double edged sword.  To learn you have to tell." Firms observed that disparities had been a "hot" issue this year, attracting attention in a very brutal and competitive marketplace in which a net decline in the number of covered lives enhanced competition among firms and necessitated "casting a wide net" in attracting a wide spectrum of racial and ethnic groups. For competing national firms in particular, these dynamics and the involvement of marketing staff increased the reluctance to share information that might, in the firms' views, reduce their competitive edge. Some regional firms expressed disappointment with these dynamics, which they viewed as slowing down progress of the group.

Attitudes toward this issue were not necessarily consistent across firms. For example, one regional firm thought there was more collaboration than it expected, even though it acknowledged that competition limited firms' willingness to share information. A participant with extensive experience in other kinds of collaboratives suggested that it was better to view the Collaborative as a "learning community" than a true collaboration, since the latter is by nature more focused on synergy and team building, elements that were not a big part of the Collaborative. Perhaps with a similar perspective, a participating support organization suggested the "Learning Laboratory" model; on the other hand, at least one firm expressed concern over a focus on learning or research per se, rather than work that was supportive of the firm's overall business objectives.

The Collaborative members were at different stages in their work on disparities, which many firms viewed as limiting progress.  While firms agreed that the group was heterogeneous, firms' own perceptions of their status did not necessarily correspond to the views of others. That is, firms had a tendency to perceive their efforts to address disparities more positively than others saw it, despite consensus on a few industry leaders.  In a few cases, the differences between how firms perceived themselves and how others perceived them was striking, as can be seen in the network analysis.

Looking Toward the Future.  Despite the limited communication, a review of what firms did through their work with the Collaborative indicates that they were adapting what they learned about others' efforts to suit their own needs. Aetna's Web portal, for example, was a mechanism several firms adopted.  Many firms also expressed a sense that communications were becoming a little more open towards the end of the Collaborative than they were initially (given the trust that had developed among organizations over time), and that the Collaborative had a stronger shared identity. Both firms and the sponsor/support organizations were optimistic, within bounds, that there would be more sharing across firms in the second phase of the Collaborative than in the first. Communication, they thought, would be enhanced by the focus on a specific set of activities. 

At the same time, expecting extensive sharing may be unrealistic for this type of endeavor, which involves large businesses, the largest of whom are in direct competition with one another. While Collaborative participants may like each other as individuals, they represent organizations that at times bring a history of fierce rivalry, resulting in antagonism.  If a major strength of the Collaborative is that the participants have strong links with top executives, participants probably exist in a culture in which "corporate sign off" is critical to any external communication.  Researchers and "content experts" would likely feel more empowered to share because exhibiting their expertise either adds to their value within the firm or is a cost that firms are willing to pay to retain such experts. Senior managers, in contrast, are members of or are closely affiliated with line management, and may be expected to be more cautious in sharing information externally or making commitments that could be viewed as binding for the company.  Although these are our speculations, they might explain why there was so little sharing of what appears, at least to an external observer, to be fairly general content. 

Another factor behind the limited sharing relates to the spillover effects of closed communication on curiosity. One of the support organizations told us that in their experience, sharing data is the first step to seriously interesting participants in sharing and understanding each firm's experience. With firms limiting how much was revealed, the "meat" that generated true collaboration may have been missing from the Collaborative. To the extent these dynamics apply, it will be valuable to recognize and deal with them in Phase II of the Collaborative. 

Return to Contents
Proceed to Next Section