Chapter VII. Perspectives on the Collaborative's Infrastructure and Process
This chapter examines the infrastructure and process of the
Collaborative in terms of its ability to promote the goals of the Collaborative.
The underlying premise is that collective action will be more successful than
each firm acting on its own. This analysis draws from our interviews with
firms and other organizations, our observations during a variety of
Collaborative meetings and calls, and a network analysis of relations between
Collaborative members. It examines the communication across participating
firms, the tangible and intangible resources brought to or generated by the
Collaborative, and the work required of participating firms. The chapter
concludes with an assessment of how well the infrastructure and process of the
Collaborative matched its goals, and a delineation of issues that warrant
consideration in structuring future endeavors.
A. Summary of Findings
Firm responses to the network analysis and interviews clearly
paint a positive picture of the Collaborative overall; firms viewed the
Collaborative as contributing to their organizations' goals. However, firms
did not appear to benefit as much as they might have had they shared
information more openly with each other, or had the Collaborative been better
structured to facilitate substantive learning, particularly about evidence on
how to reduce disparities. To some extent, limited sharing is a function of
firms' culture and the markets in which they operate. However, firms
anticipated that the more specific focus of work in Phase II would facilitate
better communication, as would the experience they had working with one
another.
Almost universally, firms said their biggest disappointment
was the Collaborative's inability to address their interest in knowing about
"what works"—particularly interventions that might reduce disparities. While
some of this may be a reaction to the lack of a solid evidence-based knowledge
in this area, it also appears that more could have been done to connect firms
with sources and people who could provide insight on this issue and to
structure agendas so that firms could learn more from one another. The
learning sessions, for example, could have included faculty presentations on
what is known. The effort required of CHCS to coordinate the Collaborative and
also the large number of support organizations probably came at a cost in
resources that could be devoted to more substantive support in this area. In
hindsight, it appeared that firms wanted information on what works, even if
they did not want to use the Collaborative to talk about what they might do
with the information. The most contentious issue for firms involved the
structure the Collaborative sought to impose and the requirements it placed on
firms. Reporting requirements were a particular concern, and some firms at
least viewed the cumulative number of requests from sponsor-affiliated groups
to be burdensome.
The experience of the Collaborative yielded a number of
valuable lessons for organizations seeking to engage in future efforts with
large firms that sponsor health plans, as discussed at the end of the chapter.
Return to Contents
B. Strength of Collaboration: Insights from the Network Analysis
We used network analysis to help us understand how the
Collaborative functioned.21
The network analysis involved asking each participant in the
Collaborative—firms, sponsors, and support organizations—to complete a brief
structured feedback form that included two primary sets of questions: one on
the Collaborative overall and the other on the relationships among (and
assessments of) other participants. The questions were asked in round two, or
the end of 2005, about six to nine months before the end of the Collaborative.
(Some perspectives may have changed by the end of Phase I.) The analysis was
designed to provide tools to assess the way participants interacted with one
another and the support provided by diverse organizations to the
Collaborative's goals. We summarize here the main findings from this analysis
whose methods, findings and conclusions are documented more completely in
Appendix C.
1. Overall Perceptions of the Collaborative
Participants' responses clearly paint a positive picture of
the Collaborative overall (Table VII.1). All but one participating
organization felt that the Collaborative was at least somewhat important to
attaining organizational goals (Question 1). In fact, 10 of the 15
organizations in the Collaborative (6 of the 9 firms and 4 of the 6 nonfirm
organizations) reported that the Collaborative was very important or crucial
for achieving organizational goals related to reducing health disparities. On
average, organizations—at least as of January 2006—felt that the Collaborative
had carried out its responsibilities and commitments "to a considerable
extent," with firms slightly more positive than nonfirm organizations (Question
2).
Participants were positive about their own participation in
the Collaborative, although some firms acknowledged that they themselves had
done less than they might have been able to. When asked to rate their own
organization on carrying out responsibilities and commitments to the
Collaborative, almost all organizations were very positive (Question 3). Two
firm representatives (both from national firms), however, indicated that their
organizations carried out their responsibilities and commitments only to "a
little extent" (one) or "some extent" (one). Almost all respondents reported
that the relationship between their organization and the Collaborative was
productive and worthwhile (Questions 4 and 5); support and sponsor
organizations were somewhat more positive than firms on these dimensions.
Likewise, all organizations reported that they were satisfied with the
relationship between their organizations and the Collaborative to at least some
extent (Question 6).
Support and sponsor organizations reported changing or
influencing Collaborative activities more than firms did (Question 7), a
response that is not surprising, given the role of these organizations in
shaping the Collaborative and working one-on-one with firms. In comparison,
firms were more likely than other organizations to say that the Collaborative
had changed or influenced their organization's activities, with six of the nine
firms saying this was the case to a "considerable extent" or "great extent"
(Question 8). All organizations said that the payoffs of the Collaborative
were reasonable relative to contributions, with firms somewhat more positive on
this dimension than sponsor and support organizations (Question 9).
2. Dynamics within the Collaborative
In addition to asking each participant about the Collaborative
overall, we asked them to rate each of the other organizations in various ways,
often involving the same questions as used for the overall Collaborative
assessment as part of the structured feedback form. Our network analysis
highlighted a number of key findings about how communications occurred in the
Collaborative and the roles played by diverse participants (Appendix C for
more detail).
The analysis highlighted the central role that key support and
sponsor organizations played in the Collaborative. Not only were they viewed
as visible and active participants in the Collaborative process, but they also
appeared to act as the "glue" that held the Collaborative together. They had
the most contact with participating firms and formed the primary pathways
linking participants. The support and sponsor organizations also engaged in a
substantial amount of contact with one another. Key support and sponsor
organizations also played an important role in contributing to action and
change among other organizations.
Though named a "Collaborative" the network analysis revealed
that there was much less communication between firms than between firms and the
individual sponsors or the support organizations. On process measures such as
communication, firms reported limited one-on-one interaction from firm to firm
(although a few firms were seen as providing many good ideas to the
Collaborative process). Only a few firms characterized other firms as having a
"considerable" influence on them or their actions. However, firms still viewed
each other as important, and most respondents reported that other organizations
were carrying out their responsibilities and commitments to the Collaborative,
at least to a small extent.
A few of the firms—namely, one national and one regional
firm—stood out as more important and influential members of the Collaborative
than others. Conversely, several firms consistently ranked toward the bottom
on these measures. Organizations' ratings of whether the Collaborative was
productive and worthwhile and whether it yielded a reasonable payoff compared
with the level of organizations' contributions were all fairly favorable and
did not appear to vary greatly with by organizational standing (though one
national firm with low standing tended to rate the Collaborative lower than
other firms).
Return to Contents
C. Firm Perspectives On Communication Within The Collaborative
Consistency between Network Analysis and Round Two
Interviews. The findings from the network analysis were consistent with
the profile of the Collaborative we obtained concurrently in our second round
of interviews. Given the competitive nature of the health plan industry, it is
not surprising that firms generally did not communicate with one another
outside of formal Collaborative meetings. Unless firms operated in different
markets (as is the case with regional firms) or had a business imperative for
additional collaboration (as at least one regional organization had with a
national firm), the firms were likely competitors. Firms also participated in
the Collaborative for different reasons (Chapter III). Although firms did not
necessarily say that they were reluctant to share information, the way they
described their internal clearance processes made it clear that release of
firm-specific information was an important threshold decision that needed to be
approved by top leadership.
Firm Reaction to Interim Report Findings. Because the
interim evaluation report was shared with firms, they had the opportunity to
see what the network analysis and other findings revealed about the way the
Collaborative was functioning. We did not explicitly ask firms to react to the
analysis, but we were able to gain insight into their perspectives on these
issues both through discussions in subsequent meetings of the Collaborative and
also through their responses to our questions on their overall views of the
Collaborative in the final round of interviews.
At the final leadership meeting of the Collaborative in
September 2006 (where the topic was gaining consensus on the content of Phase
II), AHRQ staff raised the issue of openness of communication, noting that
there wasn't as much sharing in Phase I as there could have been, both within
and outside the Collaborative. This norm limited group learning and, AHRQ
staffers noted, the ability to leverage firm's work in the Collaborative to
generate more broad based knowledge and action on issues related to
disparities. Firms seemed to accept the communication shortfalls noted in the
interim evaluation report as valid. For example, one participant (from one of
the firms that was more open in their communications) remarked that company
executives reviewing the interim evaluation asked "how this was a Collaborative
when there is very little collaboration." In the meeting or in our interviews,
a few participants noted that on occasion they heard information about other
projects being pursued by participating firms and wondered why these examples
had not been brought to and shared with the Collaborative. In some cases, the
communication failure could be attributed to the way agendas were set or the
fact that the staff participating in the Collaborative may not have been the
same as those involved in the activities of interest. But in other cases,
participants were perplexed about why a firm would be more open in discussing
relevant accomplishments in some venues than others, wondering if the interest
was more in generating publicity than sharing and collaborating.
Firms acknowledged the downsides of less open communication,
but were frank about the dynamics that might lead to this occur. As one large
participating firm noted, "It [communication] is a double edged sword. To
learn you have to tell." Firms observed that disparities had been a "hot"
issue this year, attracting attention in a very brutal and competitive
marketplace in which a net decline in the number of covered lives enhanced
competition among firms and necessitated "casting a wide net" in attracting a
wide spectrum of racial and ethnic groups. For competing national firms in
particular, these dynamics and the involvement of marketing staff increased the
reluctance to share information that might, in the firms' views, reduce their
competitive edge. Some regional firms expressed disappointment with these
dynamics, which they viewed as slowing down progress of the group.
Attitudes toward this issue were not necessarily consistent
across firms. For example, one regional firm thought there was more
collaboration than it expected, even though it acknowledged that competition
limited firms' willingness to share information. A participant with extensive experience
in other kinds of collaboratives suggested that it was better to view the
Collaborative as a "learning community" than a true collaboration, since the
latter is by nature more focused on synergy and team building, elements that
were not a big part of the Collaborative. Perhaps with a similar perspective, a
participating support organization suggested the "Learning Laboratory" model;
on the other hand, at least one firm expressed concern over a focus on learning
or research per se, rather than work that was supportive of the firm's overall
business objectives.
The Collaborative members were at different stages in their
work on disparities, which many firms viewed as limiting progress. While firms
agreed that the group was heterogeneous, firms' own perceptions of their status
did not necessarily correspond to the views of others. That is, firms had a
tendency to perceive their efforts to address disparities more positively than
others saw it, despite consensus on a few industry leaders. In a few cases,
the differences between how firms perceived themselves and how others perceived
them was striking, as can be seen in the network analysis.
Looking Toward the Future. Despite the limited
communication, a review of what firms did through their work with the
Collaborative indicates that they were adapting what they learned about others'
efforts to suit their own needs. Aetna's Web portal, for example, was a
mechanism several firms adopted. Many firms also expressed a sense that
communications were becoming a little more open towards the end of the
Collaborative than they were initially (given the trust that had developed
among organizations over time), and that the Collaborative had a stronger
shared identity. Both firms and the sponsor/support organizations were
optimistic, within bounds, that there would be more sharing across firms in the
second phase of the Collaborative than in the first. Communication, they
thought, would be enhanced by the focus on a specific set of activities.
At the same time, expecting extensive sharing may be
unrealistic for this type of endeavor, which involves large businesses, the
largest of whom are in direct competition with one another. While Collaborative
participants may like each other as individuals, they represent organizations
that at times bring a history of fierce rivalry, resulting in antagonism. If a
major strength of the Collaborative is that the participants have strong links
with top executives, participants probably exist in a culture in which
"corporate sign off" is critical to any external communication. Researchers
and "content experts" would likely feel more empowered to share because
exhibiting their expertise either adds to their value within the firm or is a
cost that firms are willing to pay to retain such experts. Senior managers, in
contrast, are members of or are closely affiliated with line management, and
may be expected to be more cautious in sharing information externally or making
commitments that could be viewed as binding for the company. Although these
are our speculations, they might explain why there was so little sharing of
what appears, at least to an external observer, to be fairly general content.
Another factor behind the limited sharing relates to the
spillover effects of closed communication on curiosity. One of the support
organizations told us that in their experience, sharing data is the first step
to seriously interesting participants in sharing and understanding each firm's
experience. With firms limiting how much was revealed, the "meat" that
generated true collaboration may have been missing from the Collaborative. To
the extent these dynamics apply, it will be valuable to recognize and deal with
them in Phase II of the Collaborative.
Return to Contents
Proceed to Next Section