The Sherman Kent Center for Intelligence Analysis

Occasional Papers: Volume 3, Number 2, Oct. ‘04

Rethinking “Alternative Analysis” to Address Transnational Threats
Warren Fishbein and Gregory Treverton
Sherman Kent Center

The Kent Center is pleased to publish as part of its Occasional Papers series this “think piece” on how to understand and address the unique analytical challenges posed by complex and fast-moving transnational threats such as terrorism, WMD proliferation, and organized crime. Distilled from a longer piece also co-authored by Warren Fishbein of the Kent Center’s Global Futures Partnership and Gregory Treverton of the RAND Corporation, this paper proposes some practical ideas for adapting the organizational culture and processes in which analysis of these issues is done to improve understanding and warning.

The authors use as a springboard for their discussion the ideas generated by a series of unclassified, multidisciplinary workshops with outside experts convened by GFP and RAND during 2003 to explore “Developing Alternative Analysis for Transnational Issues.” (Reports of these workshops are published separately by RAND Corporation in report CF-200.) In this paper, workshop insights are coupled with findings from further research on concepts such as intuitive thinking, sense-making, and mindfulness to suggest an approach for applying what the authors call “alternative sense-making” to complex transnational issues.

The ideas suggested here, however, are less a prescription for analytical practice than an invitation to dialogue, debate, and further research that will help advance the doctrine of analysis for transnational threats. The Kent Center welcomes this contribution to the literature on intelligence analysis and looks forward to continued exploration of the arguments presented here and in the longer version, “Making Sense of Transnational Threats,” published in Kent Center Occasional Papers, Volume 3, Number 1.

The Director
Sherman Kent Center

Summary

Understanding complex transnational issues, such as terrorism and weapons proliferation, requires an alternative analysis approach that is more an ongoing organizational process aimed at promoting “mindfulness”—continuous wariness of analytic failure—than a set of tools that analysts are encouraged to employ when needed. This means that Intelligence Community analytic organizations need to institutionalize sustained, collaborative efforts by analysts to question their judgments and underlying assumptions, employing both critical and creative modes of thought. For this approach to be effective, significant changes in the cultures and business processes of analytic organizations will be required.

These are the key conclusions arising from a project undertaken by the CIA’s Global Futures Partnership in the Sherman Kent School for Intelligence Analysis and the RAND Corporation to rethink “alternative analysis”—tools designed to help analysts and decision-makers employ rigorous self-review, question judgments, and explore alternative outcomes—to better address threats in the increasingly important realm of transnational issues. In a series of unclassified workshops, Intelligence Community analysts and analytic managers came together on a non-attribution basis with outside thinkers in a broad range of fields relevant to the analytic process, including cognitive psychology, psychiatry, organizational behavior, artificial intelligence, knowledge management, intelligence studies, and the foreign policy process. Through presentations and discussions among participants, the workshops sought to generate broad concepts about adapting alternative analysis to enhance warning of out-of-the-ordinary actions undertaken by non-state actors, epitomized in the September 11 attacks. What follows in this report are some of the more intriguing practical ideas that surfaced at the workshops, augmented with insights from related studies that elaborate these ideas. [1]

The Nature and Role of Alternative Analysis

Traditional intelligence analysis generates forecasts or explanations based on logical processing of available evidence, whereas alternative analysis seeks to help analysts and policy-makers stretch their thinking through structured techniques that challenge underlying assumptions and broaden the range of possible outcomes considered. Properly applied, it serves as a hedge against the natural tendencies of analysts—like all human beings—to perceive information selectively through the lens of preconceptions, to search too narrowly for facts that would confirm rather than discredit existing hypotheses, and to be unduly influenced by premature consensus within analytic groups close at hand. Alternative analysis involves a fairly intensive, though time limited, effort to challenge assumptions or to identify alternative outcomes, depending on the technique employed, with the results captured, implicitly or explicitly, in a written product delivered to relevant policy-makers. [2]

In theory, use of alternative analysis techniques can help to reduce the likelihood of “intelligence failures,” which historically have stemmed in part from such mental errors (e.g. the ingrained belief that the Japanese could not mount a successful attack against Pearl Harbor). In reality, however, alternative analysis has not been particularly effective within the Intelligence Community as it has been employed only sporadically, at best, and more often than not as a “nice to have” supplement tacked on to traditional analysis rather than integrated at the outset as an essential component of the analytic enterprise in a world of uncertainty and deception.

Moreover, alternative analysis may be less effective at enhancing the warning process for threats in the transnational realm. Alternative analysis involves the application of structured argumentation to address discrete questions—what if Ruritania acquires “the Bomb?” But for amorphous, continuous threats, such as terrorism and weapons proliferation, the specificity and product-driven focus of alternative analysis can be more problematic.

Analysis of any kind involves breaking down problems into constituent parts, such as causes and effects, and using logical operations to identify and test hypotheses for the purposes of explanation and prediction. Traditional analysis is relevant for bounded problems—such as state-to-state diplomacy or internal politics—in which there is a relatively restricted range of outcomes or hypotheses to be evaluated. Alternative analysis tools can support the overall analytic process by ensuring that more than one explanatory hypothesis or projected outcome is seriously considered.

Intelligence problems in the transnational realm are, however, generally less bounded than are those in the state-to-state realm. Transnational groups, like terrorist cells or proliferation networks, are likely to be smaller and more numerous than states, less constrained by rules and historic precedent, and more affected by tactical and situational circumstances as opposed to deeper internal drivers. They are thus more difficult to understand than state actors and more capable of engaging in a wide array of unpredictable behaviors. The transnational arena also arguably is more affected by “information overload” than many facets of the state-to-state arena, with analysts having to contend with enormous volumes of questionable data about their shadowy targets and little historical context for evaluating such data. Transnational problems are, thus, often not very susceptible to “analysis,’ whether of the traditional or alternative variety, because there are too many hypotheses to consider and too little solid information to support logical judgments. [3]

Analyzing Complex Issues: The Concept of Sense-Making

How then can intelligence address such problems and what are the appropriate “alternative” processes for tackling mental biases? One potentially relevant concept, developed by management scientists to describe how business and public sector organizations cope with uncertainty, is the process of “sense-making.” This is a continuous, iterative, largely informal effort by members of organizations to understand, or “make sense” of what is going on in the external environment that is relevant to their goals and needs.

Sense-making involves the collective application of individual “intuition”—experience-based, sub-consciously processed judgment and imagination—to identify changes in existing patterns or the emergence of new patterns. [4] Several leading management and cognitive scientists argue, based on empirical research, that intuitive judgment, rather than formal analysis, actually underpins most organizational decisions and that it provides at least equally reliable results in predicting outcomes in most cases. Moreover, they argue that it is superior to analysis for problems marked by high ambiguity or uncertainty, because efforts to “reduce” such problems (to identify a handful of key variables) to fit into structured analytic frameworks will produce misleading results. [5] Sense-making aggregates and refines intuitive judgment through conversations within the organization in which members construct interpretations of reality and develop explanations or stories to account for perceived anomalies.

Intelligence analysts already engage in something approaching sense-making: for instance, in the more thoughtful aspects of the current intelligence process. These involve ongoing efforts to update the story line on an issue, usually without recourse to formal analytic techniques. Even many longer-term assessments are more accurately described as intuitive rather than analytical, if the latter concept is strictly defined as involving rigorous comparison of competing hypotheses. [6] But intelligence “analysis” differs from sense-making in some key respects: it is generally less collective (primary responsibility for sense-making often resides with an individual analyst) and more formal (with ideas shared through the often-adversarial process of coordination rather than through conversation).

A more effective system of intuition-based intelligence sense-making offers a way to comprehend those complex transnational threats whose speed, uncertainty, and interactivity defy traditional analytic approaches. One presenter at the GFP-RAND workshops, a noted expert on intelligence, made this point in observing that, in contrast to traditional Soviet-threat type issues, the transnational arena (terrorism in particular) benefits far more from lots of “pairs of eyes” looking at data for emerging signs of threat than from detailed analyses of narrowly drawn issues. While lots of eyes would, of course, be beneficial in addressing traditional issues, they arguably are crucial to success in following transnational issues, where having multiple intuitions wrestling with and sharing perspectives on enormous information flows should significantly improve the odds in favor of identifying threat patterns.

Addressing Mental Biases in Sense-Making

Like traditional analysis, intelligence sense-making suffers from the effects of mental biases. The process of challenging assumptions in this domain will be different, however, from that used in the more familiar analytic domain. Whereas alternative analysis enhances a formal, periodic process for exploring a limited range of outcomes, “alternative sense-making” must address a process that is informal, continuous, and focused on issues marked by very high uncertainty.

A useful starting point for identifying an alternative analysis approach suitable for sense-making is the work of noted organizational theorists Karl Weick and Kathleen Sutcliffe. [7] The authors examined a number of organizations, such as nuclear power plants and aircraft carriers, which face challenges in addressing uncertainty akin to those faced by intelligence organizations. Those organizations on a daily basis confront continually shifting conditions that can give rise to unexpected outcomes, with the potential for catastrophic consequences. [8] Nonetheless, carriers experience exceptionally low rates of error and accident in comparison to other fields that also continually confront uncertainty, such as medicine. The same can be said of nuclear power plants where, according to another writer, “there is no ‘regular’ functioning of the plant—the status of the plant is continually changing and therefore…anomalies are very difficult to track.” [9]

In studying these types of “high reliability organizations” (HROs), Weick and Sutcliffe have identified lessons that can be applied generally to addressing uncertainty. The unifying trait of HROs is that they exhibit the quality of “mindfulness,” defined as:

“…the combination of ongoing scrutiny of existing expectations, continuous refinement and differentiation of expectations based on new experiences, willingness and capability to invent new expectations that make sense of unprecedented events, …and identification of new dimensions of context that improve foresight and current functioning.” [10]

Mindfulness is the result of a never-ending effort to challenge expectations and to consider alternative possibilities. Such eternal vigilance helps eliminate “blind spots” that result in organizations missing early warning signs of unexpected and unwanted change. A mindful orientation “redirects attention from the expected to the irrelevant, from the confirming to the disconfirming, …from the more certain to the less certain, …and from the consensual to the contested.”

Weick and Sutcliffe identify several attributes of organizational culture that contribute to mindfulness. Two that appear to be critical to anticipating uncertainty, as opposed to reacting to it, are most relevant to intelligence organizations:

• A preoccupation with failure, both past and potential.
  
• A “refusal to simplify.”

High reliability organizations “tend to view any failure, no matter how small, as a window on the system as a whole.” This preoccupation also extends to thinking about future downsides: HRO’s tend to be “skeptical, wary, suspicious of quiet periods,” because they know that “not all failure modes have…been experienced or exhaustively deduced.” Underpinning these attitudes is a “learning culture” in which it is safe and even valued for members of the organization to admit error and raise doubts.

The refusal of HROs to simplify involves a refusal to take things for granted or to rely on standard interpretations. Personnel in HROs are relentless in their efforts to try to understand the complexities of the situations they face. As an example, mechanics in nuclear power plants almost never rely on the simplifications of blueprints when they have to intervene in the system, but rather they personally “walk the system” to assure that no subsequent changes have been made that could affect the outcome. Particularly relevant to the intelligence environment, different departments constantly interact when confronted with a problem, generating hypotheses about “what is going on, what can be done, and what the long-term, system-wide consequences of the proposed action might be.” Such interdepartmental interaction brings different perspectives to the table, building a progressively more complex vision of the problem at hand, and reducing the chances that key aspects of a problem will be overlooked.

Applying Alternative Sense-making to Transnational Issues

The concept of mindfulness, as practiced by HROs, provides a construct for designing processes in intelligence practice to improve understanding and warning for complex, transnational issues. For threats that can suddenly emerge at any time, anywhere, and in a variety of forms, analysts need to think more in terms of a broad mental readiness to perceive early warning signs of threat than in terms of challenging specific assumptions or identifying specific alternative outcomes.

The following principles can guide the development of an alternative process for intelligence sense-making. Such a process must be:

• Continual
  
• Creative
  
• Collaborative
  
• Counter-intuitive
  
• Consumer-friendly

Continual: The conventional model for employing alternative analysis—identify an issue too important to “afford getting it wrong” and then challenge assumptions and identify alternative outcomes—is not really suitable for ongoing complexities. There are too many outcomes to be considered, too much potential for sudden change, and too many contingent interactions for any “one-off” effort to be particularly useful. Moreover, cognitive research suggests that such efforts may not make a sufficient imprint on thinking to affect ongoing analysis. This is because information that is inconsistent with expectations is less likely to be remembered than information that is consistent. Since alternative thinking goes against the grain of established thought, its ability to have asustained impact on understanding, individual or collective, is always open to question, all the more so if, as is often the case, analytic cadres turn over fairly rapidly. [11]

A continuous, sustained program of small to medium-sized efforts, however, would regularly explore different possible outcomes and debate assumptions, all linked to incoming information about the issue under consideration. This probably is best thought of as an ongoing conversation (both face-to-face and electronic) among interested parties, structured to encourage divergent thinking. Larger efforts, such as multiple scenario workshops or multi-player games, would aim at feeding results into the ongoing dialogue, not simply publishing and moving on. Information technology to capture and automatically recall both previous judgments and alternatives, perhaps cued by keywords, would be essential to supplement human memory, further stimulate debate, and provide continuity that survives personnel turnover.

Creative: Traditional alternative analysis is a fairly formal process with some elements of creativity but with a strong emphasis on logical argument to come to clear conclusions. Alternative “sense-making” for complex issues would, by contrast, be more freewheeling and creative. In part, firm conclusions are not desirable given the higher levels of uncertainty inherent in these issues. In addition, the objective is to stimulate pattern recognition—to connect the dots—a creative process in itself. Cognitive research suggests that judgments about the likelihood of events often reflect the “availability” and vividness of memories about similar types of events. [12] To the extent that alternative sense-making can help to suspend premature judgment and make an array of possibilities come alive, it may head off “failures of imagination” by stimulating exploration of alternative dot arrangements.

Increasing creativity within the intelligence field to enhance intuitive judgment can be accomplished in a number of ways. Press reports indicate that the CIA has worked, for example, with the film industry to create audio-visual games to help analysts “think like terrorists.” Such major efforts would have a more lasting impact if they were supplemented by smaller, more frequent efforts geared toward cultivating creativity at both the individual and organizational levels. Systematic use of the wide array of divergent thinking techniques as a precursor to the preparation of intelligence products would help to put a wide range of possibilities on the table before provisional intellectual closure was reached. Subject matter specialists would be paired with analytic methodologists with expertise in the application of such divergent thinking techniques. [13]

A conscious mixing of mental biases would also help to assure that different types of “eyes” are searching for threatening patterns. The specific suggestion raised in workshop discussions involved the practice—known as “barbelling”—used by some Wall Street firms, which involves pairing young financial professionals with those over 50 to marry adventurousness with caution born of experience. Firms with such an age distribution, according to a noted presenter from the financial community, tend to perform better than those whose professionals cluster in the thirty-to-fifty age range. Recruiting and assigning of analysts also could be designed to produce teams that consciously mix biases, such as perceptual and judgmental thinking styles (as identified in Meyers-Briggs type tests).

Providing analysts with much greater time and freedom to think about problems than is normally allowed by hectic intelligence production schedule could also foster creativity. Ideas most often “pop out” of what cognitive psychologist Guy Claxton has described as the “contemplative” mode of thought, which is slow moving and largely unconscious, rather from the “deliberative” mode, which is fast moving and conscious. A state of mental relaxation is conducive to the playful workings of this contemplative mode, something hard to achieve in the midst of a grueling production schedule. Other options include enabling analysts to pursue independent projects, contacts, and conversations seemingly peripheral to the main tasks at hand, as well as occasional working retreats—for physical setting is related to creativity as well. [14]

Collaborative: Although many forms of traditional alternative analysis are collaborative, an individual analyst can do others, like devil’s advocacy or “what-if” analysis. Such individual efforts are possible because the aim is to develop or to challenge logical arguments, something that the individual mind does well. However, examining alternatives on a sustained basis for transnational issues (and for comparably complex and dynamic traditional issues) involves a team effort embracing a variety of disciplines and regional specialties—far beyond the capabilities of a single analyst. Moreover, introspection is extremely difficult for most individuals, as a leading cognitive psychologist noted at one workshop and is amply documented in psychological literature. To promote sustained awareness of possible flaws in one’s thinking requires a continuous dialogue among individuals with different perspectives.

One collaborative technique discussed at length at the workshops was “out loud sense-making”—a structured process for orally reviewing assumptions and alternatives. Such a process is used by surgical teams at a New England hospital in response to recurrent errors by anesthesiologists in diagnosing respiratory reactions—often fatal—to anesthesia. Now, when confronted with breathing problems, the team—including non-physicians—systematically reviews possible causes out loud before arriving at a conclusion, dramatically reducing the error rate. Such processes, enhanced by skilled custom design and facilitation, could be regularly employed in the intelligence arena whenever anomalous data emerged.

Another collaborative technique identified at the workshops involves the greater use of web-logs (or “blogs”) as a platform for intelligence production. US embassies already use this art form—which allows for continuous posting of information—as their primary means for conveying information. Unlike published papers, intelligence web-logs would be a form of “unfinished” production in which both intuitions and more formal arguments could be posted, and then challenged by those with alternative opinions. Indeed, web-logs could be the forum for a facilitated virtual dialogue—the electronic equivalent of out loud sense-making.

Counter-intuitive: Intelligence analysts need to focus on patterns that are different, even contradictory to those they expect, to lessen the degree to which their mental models inhibit their ability to perceive new information. And they must do so fairly regularly to promote continuous awareness of possibilities. This is not easy given the natural human tendency—especially under time pressure—to revert to established ways of viewing the world.

One possible approach is to institutionalize brief, informal exercises in which analysts regularly focus on how they could be wrong, along the lines of what Klein calls a “pre-mortem.” [15] In this group exercise, individuals imagine fiascoes relating to their areas of responsibility—in the case of intelligence analysis, a warning failure—and brainstorm about how they might come about and how they could be headed off. The very pointed emphasis on the negative, Klein argues, helps to shake up complacency that arises from common overconfidence in judgments. Such an exercise would be a functional equivalent of the “low probability/high impact” exercise of traditional alternative analysis, with the difference that its informal nature lends it to more frequent use.

Workshop participants also discussed changing the culture within intelligence organizations to one that is more conducive to self-questioning. Assessment of analytic performance often is done by outside bodies with negative findings exploited by media and external critics, thus prompting a defensive stance among intelligence professionals. There has been only modest emphasis internally on looking at failures—and even less on examining successes—with an eye to drawing lessons for self-improvement, notwithstanding Sherman Kent’s exhortation for such introspection over 40 years ago.

The systematic and non-punitive approach of the nuclear power industry to collecting information on, and learning from, errors—associated with its low failure rate—provides one approach worth considering. Another example is provided by the US Army’s Center for Army Lessons Learned (CALL), which continuously looks at operational problems in order to draw implications for improving performance. A dedicated effort to continually take a friendly look at getting it both right and wrong through periodic “After Analysis Reviews”—modeled on the Army’s After Action Review—might help to create a more introspective environment. [16] Allowing analysts the time for such exercises—or mandating them—would powerfully demonstrate organizational commitment to learning.

Consumer-friendly: One of the greatest challenges of any “alternative” effort is to effectively communicate the message to those who occupy decision-making roles. Decision-makers are buried by both information and tasks. Motivating them to spend time reading sophisticated analysis in general, let alone analysis that queries existing analytic lines, is a considerable challenge. Several presenters at the workshops who had been senior officials in the terrorism area stressed the extent to which information overload had grown in the post-September 11 environment. Moreover, in the transnational domain, many potential key consumers are in middle and lower operational levels, or outside the government, and thus have even fewer contexts for understanding intelligence information. And, of course, in order to carry out “alternative sense-making,” alternative points of view would need to be put before this wide array of harried individuals on a fairly regular basis.

One way to accomplish this is to rethink the concept of the intelligence “product.” Intelligence organizations continue to insist upon written prose and formal briefing as the “gold standard” for disseminating information even though adults rarely retain more than ten percent of what they are “told” either orally or in written form. Instead, more experiential, interactive formats, as discussed at the workshops, might better capture the attention and imagination of intended audiences and strengthen retention of insights.

• Use of web-logs would give consumers—particularly non-senior consumers without formal feedback processes—the opportunity to tap in from time to time on debates within the analytic community and to pose questions themselves.
  
• RapiSims—rapid simulations enabled by increasingly sophisticated spreadsheet-based programs—would allow consumers to manipulate variables to generate alternative outcomes. Decision-makers could quickly and easily explore a range of possibilities in a way that is more likely to be retained than if presented in a long and dry formal tome.
  
• Half day “gaming” sessions—intentionally kept brief to allow even the most harried to participate on occasion—could help decision-makers experience, at a minimum, the uncertainties surrounding an issue.

Another avenue would be to try to strengthen personal relationships, such as through increased face-to-face contacts, between individual analysts and consumers in order to facilitate informal exchanges on alternative outcomes. One presenter, a specialist in corporate innovation, argued that the real product of the analytic organization is the “analyst rather than the analysis,” just as professionals in other knowledge industries—management consulting, law, etc.—rather than specific products, are the key selling points for their organizations. [17] A professional known by and enjoying the trust of consumers is in a far better position to influence their thinking than is any specific report or briefing. Such a professional, if inculcated in alternative sense-making values and processes, also would be in a position to periodically draw consumers’ attention to the results of internal exercises on assumptions and outcomes.

Some of the practical ideas stemming from workshop discussions are summarized in the following table.

Key Practical Ideas

Conclusion

Understanding complex transnational issues requires an alternative analysis approach that is more an ongoing organizational process aimed at promoting continuous “mindfulness” than a set of tools that analysts are encouraged to employ when needed. The latter approach lays the responsibility for alternative analysis too heavily on the shoulders of individual analysts, who may not have the incentives or time to question their own or their colleagues’ lines of thinking. And in the case of highly uncertain, rapidly changing, boundary-crossing threats such as those in the transnational arena, such questioning can really only take place as part of an ongoing, organizationally-supported collaborative effort, given the wide range of disciplines and institutions that contribute to shaping the ultimate analytic product.

With the global landscape likely to feature a growing array of hard-to-track threats in the years ahead—as a result of the increasing availability of technical know-how and soaring interconnectivity, among other factors—the need for a process to produce sustained mindfulness will only grow. Even in the case of more traditional intelligence issues, developing and enhancing organizational processes designed to promote alternative analysis might encourage more systematic use of these tools.

Thinking of alternative analysis as an organizational process has important implications for how the intelligence analytic organization is managed. It means that senior intelligence officials must make challenging assumptions and considering alternative possibilities a high, indeed critical, priority. It calls for reward systems that encourage analysts to think about how they could be or have been wrong. Production schedules and staffing requirements need to be adjusted to allow time for analytic reflection. Throughout the organization there must be a “culture”—one inculcated by exhortation, training, and example—that values continuous, collective introspection.

Rethinking the concept of alternative analysis may even mean rethinking how the intelligence field defines itself. For instance, the CIA’s Directorate of Intelligence has self-consciously modeled itself after the academic realm and to a lesser extent the press, where questions of accuracy—getting the data and facts right—are of paramount importance. But should it not think of itself as more equivalent to an aircraft carrier or nuclear power plant, where data and facts are only the beginning, and questions of failure drive every aspect of operations?

[1] This paper is an abbreviated version of a project report entitled Making Sense of Transnational Threats, Kent Center Occasional Paper, Vol. 3, No. 1., available on the CIA public website at www.cia.gov. Summaries of the discussions at each of the project’s four workshops can be found in the RAND Corporation report (CF-200) Enhancing Warning for Transnational Threats: Workshop Reports.

[2] Alternative analysis includes techniques to challenge analytic assumptions (e.g. “devil’s advocacy”), and those to expand the range of possible outcomes considered (e.g. “what-if analysis,” and “alternative scenarios.”

[3] The same can be said, however, of many aspects of the state-to-state arena such as state failure, battlefield dynamics, and crisis diplomacy, which are also marked by multiple potential outcomes, interactivity, and extreme information overload.

[4] See Karl Weick, Sensemaking in Organizations, (London: Sage Publications, 1995).

[5] Gary Klein, Intuition at Work: Why Developing Your Gut Instinct Will Make You Better at What You Do, (New York: Doubleday, 2002); Thomas Stewart, “How to Think with Your Gut,” in Business 2.0, November 2002, http://www.business2.0.com Guy Claxton, Hare Brain, Tortoise Mind: How Intelligence Increases When You Think Less, (New York: Harper Collins, 2000), pp. 3-11.

[6] Richards S. Heuer, Jr., The Psychology of Intelligence Analysis (Washington: Center for the Study of Intelligence, 1999). See also Rob Johnston, The Culture of Analytic Tradecraft: An Ethnography of the Intelligence Community (Washington: Center for the Study of Intelligence, Central Intelligence Agency), in press.

[7] See Managing the Unexpected: Assuring High Performance in and Age of Uncertainty, (San Francisco: Jossey-Bass, 2001).

[8] Weick and Sutcliffe note that an aircraft carrier deck, for instance, combines a volatile mixture of jet fuel, weaponry, “controlled crash” landings, shifting sea and weather conditions, noise, and young and often inexperienced personnel.

[9] Klein, pp. 132-134.

[10] This and the following quotes from Weick and Sutcliffe are found on pages 25-62.

[11] See see Ziva Kunda, Social Cognition: Making Sense of People, (Cambridge: MIT Press, 2001), pp. 161-210.

[12] Kunda, pp. 89-101.

[13] For a discussion of creativity in an organizational context see John Kao, Jamming: Art and Discipline ofCorporate Creativity, (New York: Harper Business, 1997).

[14] Claxton, pp. 201-226. Also see Judith H. Heerwagen, “Creativity” at http://www.science.doe.gov (last visited 13 July 2004).

[15] Cited above, pp. 88-91.

[16] For a discussion of the Army’s after action process, see Nancy M. Dixon, Common Knowledge: How Companies Thrive By Sharing What They Know, (Boston, Harvard Business School Press, 2000), pp. 37-38.

[17] Former CIA Deputy Director for Intelligence Winston Wiley often said that the Directorate of Intelligence produces two products: intelligence analysis and intelligence analysts.