The Sherman Kent Center for Intelligence Analysis

Occasional Papers: Volume 1, Number 2, Sept. ‘02

Improving CIA Analytic Performance:
Analysts and the Policymaking Process
Jack Davis,
Sherman Kent Center

A host of reports have been written over the 50 years of CIA history evaluat­ing analytic performance and recommending changes in priorities and trade­craft. These “post-mortem reports” have been issued by Agency leaders and components as well as by Congressional committees and commissions and non-governmental organizations concerned about intelligence performance. Starting with the 1990s, post-mortem reports increased in number, generated both by charges of specific intelligence failures and by general recognition that the post-Cold War period presented new challenges to intelligence.

The recent post-mortem reports have helped Directorate of Intelligence leaders to examine current doctrine and practice critically, and to address identified challenges in training programs. This Occasional Paper is one of a series of assessments of what recent critiques have said about the key challenges facing the DI in the new century.

The present paper addresses the challenge of establishing effective analyst-policymaker relations. It reviews five post-mortem critiques: (1) Twentieth Century Fund Task Force on the Future of Intelligence, In from the Cold (1996); (2) Independent Task Force of the Council on Foreign Relations, Making Intelligence Smarter: The Future of U.S. Intelligence (1996); (3) Commis­sion on the Roles and Capabilities of the United States Intelligence Commu­nity, Preparing for the 21^st Century: An Appraisal of U.S. In­telligence (1996); (4) Report of the Commission to Assess the Ballistic Missile Threat to the United States (1998); (5) Working Group on Intelligence Re­form of the National Strategy Information Center, The Future of US Intelligence (1996).

Analysts and Policymakers

Sherman Kent observed some 50 years ago that the most important relationship for analysts, that with the policy officials they seek to inform, does not fall naturally in place, but requires careful thought to set right and constant efforts to keep effective (Strategic Intelligence for National World Policy, 1949). Kent articulated the basic challenge to effective ties when he observed that if analysts get too close to their policymaking and action-taking clients, they would be in danger of losing the independence of mind and the substantive depth and analytic expertise that enabled them to make a distinctive professional contribution to national security. Yet if they stay too far apart from those they are charged to serve, they would be cut off from the feedback and other guidance essential for making that contribution.

The challenge of ensuring effective relationships has indeed endured. From its start, protecting the independence of the analytic process from policymaker influences became a central professional value of Agency analysts. But often at a high price. Analysts who kept their distance from the policymaking process learned to their sorrow that there is no way to get policy officials to pay serious attention to analytic assessments not judged to be helpful in getting their policymaking jobs done.

In recent decades, to redress the balance, the DI has relied on tough-minded analytic tradecraft to protect professionalism while moving closer to the policymaking action. In particular, the DI regularly tables assessments customized to take account of the specific questions and policymaking agendas of individual or small groups of policy officials. The DI has made a greater overall effort to provide analytic value-added to the policymaking process, especially via the more direct and informal venues of briefings, inter-agency meetings, teleconferences, telephone calls.

Nonetheless, post-mortem critiques of analytic performance issued during the 1990s that address the issue convey a general dissatisfaction with the effectiveness of analyst-policymaker relations. While the sharpness of the criticism and the recommendations for improved performance are not uniform, the main signals are—intelligence must improve both the quality and policy utility of analysis in order to be joined effectively to its designated consumers and thus effectively serve the national interest.

Case for Closer Ties to Policymaking

The report of the Twentieth Century Fund Task Force on the Future of Intelligence (1996), In from the Cold, presents in some detail the general indictment against analytic practice in the early 1990s as perceived by the critics. The Task Force included prominent representatives of the academic, policy, business, and intelligence communities, including former DDI Richard Kerr and former D/INR Morton Abramowitz.

1. Intelligence-Policy interactions “need to be vastly more productive and effective” in order to ensure that senior civilian and military officials receive the information and analysis they need when they need it. In short, better integration of intelligence into the policymaking process is needed.

2. Military clients make good use of analytic production, but outside of the Department of Defense, policymakers too often consider intelligence analysis to be unreliable, unhelpful, or irrelevant. Intelligence, by the “test of the marketplace,” is losing out to the growing competition for policymaker reliance for information and insight from the media, think tanks, advocacy groups, and the policy officials’ own expert staffs.

The recommendations for improvement are equally direct.

1. First, intelligence must improve the quality of analysis in order to motivate policymakers to share their issue agendas, their decision and action schedules, and their own analytic assumptions on what drives a priority issue and determines which US security interests are at stake—that is, essential guidance for providing analytic value-added to the policymaking process.

2. Second, intelligence must be prepared to lower the wall between it and the policymaking process. The report commends as a model the British system under which policy officials are integrated into the intelligence efforts to support effective decisionmaking “at every step, from writing to approving intelligence analysis.” Any concomitant danger of politicization of analysis is to be combated through reliance on high standards of analytic quality.

Also in 1996, an Independent Task Force of the Council on Foreign Relations (CFR) provided similar views in a report entitled Making Intelligence Smarter: The Future of U.S. Intelligence. Once again, former DDI Kerr and D/INR Abramowitz were among the broad-based prominent membership. If with somewhat less sharpness, once again the call went out that first the quality and then the actual policymaking value of analysis have to be improved in order for intelligence to fully carry its weight on behalf of national security.

Many current and former policymakers are critical of the analysis they receive, and both intelligence consumers and producers often share a frustration over its perceived lack of utility and hence impact.

Regarding improved analytic quality, the CFR report recommended more rigorous training, greater exposure of careerists via rotational assignments to other governmental and private sector organizations, and reduced intelligence isolation through greater recourse to the talents of outside experts.

The best way to ensure high-quality analysis is to bring high-quality analysts into the process. Analysis would be improved by increasing the flow of talented people into the intelligence community from outside the government. Greater provision should be made for lateral and mid-career entry of such analysts as well as for their short-term involvement in specific projects. Closer ties between universities and the intelligence community is desirable in this regard. Careerists would benefit from greater opportunities to spend time in other departments and nongovernmental organizations, including those involved in commerce and finance.

Regarding effective relations per se, the report insists the analytic and policymaking communities must become better informed on the other’s needs. Towards this end, a rotational assignment to a policymaking unit should be made a requirement for promotion to senior intelligence levels.

Closer ties could increase the danger of politicization—“the potential for the intelligence community to distort information or judgment in order to please political authorities.” Among the safeguards recommended are forceful defense by intelligence leaders of the practice of analytic independence on the part of their troops and competitive (deliberately non-consensual) analysis, especially on controversial and can’t-get-it-wrong issues.

In any case, the CFR report finds “irrelevance…[an]…arguably bigger problem for analysts than politicization.”

Intelligence analysis rarely impresses itself upon policymakers, who are inevitably busy and inundated with more demands on their time and attention than they can possibly meet. Intelligence officials must draw attention to their product and market their ideas. This is especially true in the case of any early-warning or intelligence-related development that has potentially significant consequences for important interests. A phone call, a personalized memorandum, a meeting—any and all are required if the situation is sufficiently serious.

Ambassador Richard Haas, Project Director for the CFR study, later in the decade put the issue directly in a presentation to senior DI analysts.

You in the intelligence community must not produce literature. You must produce results. And, in order to produce results, you have to get much closer to the policymakers.

The “Brown Commission” (named after former Secretary of Defense Harold Brown) joined the 1996 post-mortem chorus calling for enhanced analytic quality and more effective policy ties as prerequisites for fulfilling the most vital of intelligence missions: relevant and timely assessments in support of national security decision making. Chartered by Congress to conduct a thorough study of intelligence requirements for the post-Cold War era, the report of the Commission on the Roles and Capabilities of the United States Intelligence Community is entitled Preparing for the 21^st Century: An Appraisal of U.S. Intelligence.

The Commission notes that high level policy officials who rate a personal briefer, a veteran DI analyst on rotation to their staff, or an intelligence liaison officer are generally satisfied with their analytic support. Some lower ranking officials establish ad hoc relationships with analysts that prove satisfactory. Many others, however, feel under-served or ill served. The report calls for an extension of closer ties, pointing out that consumers as well as producers must put greater effort into making relationships work.

The report addresses directly Sherman Kent’s dilemma of relations being either too distant or too close.

Many observers have cautioned that in promoting closer relationships with consumers, the chances are increased that intelligence analysis will lose its objectivity. As analysts become aware of their customers’ assumptions and policy preferences, they would, consciously or not, produce analysis that conformed to those preferences.

The Commission believes this problem is real, but manageable. The need to present the “unvarnished truth” to policymakers is at the core of every analyst’s training and ethos. It is, in a real sense, why intelligence exists. Further, there are checks and balances within the system. Virtually all analysis is reviewed by multiple experts, not all of whom are familiar with the biases of a particular consumer. Some analyses, such as national estimates, are reviewed by interagency boards.

The role of intelligence analysts is to inform the policy process. One witness before the commission [N.B., the author of the present memorandum] expressed the view that “if an intelligence analyst is not in some danger of being politicized, he is probably not doing his job.” The Commission agrees. The greater danger lies not in becoming “politicized” but in becoming irrelevant to the process of government.

The Rumsfeld Commission, also authorized by Congress, and chaired by now Secretary of Defense Donald Rumsfeld, echoes the point that policymakers share with intelligence leaders the responsibility for ensuring effective analytic support of their efforts. The Report of the Commission to Assess the Ballistic Missile Threat to the United States (1998) avers that senior policymakers have an obligation to ask probing questions of analysts to make sure they have adequately considered alternatives to their preferred bottom-line judgments. In other words, policymakers are more likely to get quality analysis if they make direct demands for it.

The Rumsfeld Commission’s firm recommendation for analysts is that on complex issues involving serious potential threats to US security interests, intelligence assessments must go beyond a preferred single bottom-line, and include serious attention to plausible, more menacing threats. Commission member Paul Wolfowitz (now Deputy Secretary of Defense) reflected the general views of his colleagues when he opined in 1995 that the proper role of intelligence is to serve as a tool for effective debate among competing policymakers (via multiple outcome analysis)—and not as a weapon that one group of policymakers can wield against the others (single-outcome analysis).

Proper Role of the Analyst

The four post-mortem reports cited above recommend specific actions required to reposition intelligence analysts closer to the policymaking process but do not provide a general characterization for the impact of the changes on the relationship. Still another 1996 critique of intelligence performance, issued by the Working Group on Intelligence Reform of the National Strategy Information Center (NSIC), and entitled The Future of US Intelligence, is more forthcoming in redefining the proper role of the analyst. The membership of the Working Group was broadly based (including the author of the present memorandum); the direct experience with intelligence of the principal authors, Gary Schmitt and Abram Shulsky, was service with executive branch policy agencies and Congressional and Presidential intelligence oversight entities. Their conclusion:

Ultimately, the purpose of analysis is to help the policymaker shape the future, not to predict it.

The analysts are charged, in effect, with leveraging their expertise to enable the USG to gauge the seriousness of national security threats and anticipate policy opportunities. The general label for the sought-after intelligence deliverable is “opportunity analysis” (also called “action analysis”), elaborated as follows in an earlier NSIC publication.

Opportunity-oriented assessments would help policymakers identify:

1. Opportunities to advance US interests through diplomatic, military, and economic means, public diplomacy, and covert action.
2. Strategic and tactical vulnerabilities of foreign leaders, parties, and…movements.
3. Factors subject to US influence.
4. The likely results of given US courses of action on foreign societies.

Former DDI Doug MacEachin (1993-1995) made a similar point in comparing the role of the analyst in the policymaking process to that of the scout in relation to the football coach. The job of the scout is not to predict in advance the final score of the game, but to assess the strengths and weaknesses of the opponent so that the coach can devise a winning game plan. Then the scout sits in a booth with powerful binoculars, to report on specific vulnerabilities the coach can exploit.

Summary Recommendations

A strong case can be made that most of the recommendations of the post-mortem reports to the intelligence side for improving relations with policymakers and increasing the general utility of analysis for the policymaking process were already being implemented in the early 1990s and have been employed more extensively since. DI analysts serving as personal briefing officers, sent on rotational assignments to policymaking units, and posted as intelligence liaison officers for executive branch departments are ever increasing in number. Though harder to quantify, DI analysts have increased their direct contact with the policy community generally—including one-on-one on the telephone and interacting with small groups at Interagency Working Groups and such. Opportunity or action analysis is regularly underscored as a central DI deliverable.

Yet the criticisms of the post-mortem reports were sincerely given by informed parties. And even as DI performance improves, the bar for effective policy support invariably rises, the competition from other providers of information and insight grows keener, and the sheer need of policymakers and action-takers for analytic support to deal with difficult national security issues ever increases. Moreover, frequent policymaker turnover and periodic broadening of the policy customer base (as with Homeland Defense after the terrorist attack of 11 September 2001) generate an ongoing need to establish access, credibility, and effective relations generally with new policy players. DI performance in support of the policymaking cannot rest on its laurels.

What follows are some summary recommendations to clarify the choices of DI analysts as they face what Sherman Kent so aptly depicted as a never-ending effort to keep intelligence-policy relations effective.

1. Define the analysts’ mission realistically. The standard of “Delivering Truth to Power” is an essential but often symbolic standard. On complex issues where analysts are not fully informed on US intentions, much less those of adversaries and other players, the more realistic standard for analysts is to “Tell It like They Think It Is.” When estimative prediction is called for, remember the veteran policy official is also an analyst: make sure the argument made is transparent and sound, the supporting evidence is authentic and ample, and plausible alternative outcomes are given their due.

2. Become well informed on policymaking in Washington. To get the DI’s job done, analysts must become expert on how the US government works as well as the foreign governments and groups they track abroad, especially regarding where key policy clients are on their decision cycles and learning curves. How? Debrief senior colleagues who attend interagency meetings and have other direct contacts. Read what the senior officials say and what is said about them in the media. Cultivate relations with their policy staff officers. As former DDI Bob Gates averred:

Unless intelligence officers are down in the trenches with the policymakers, understand the issues, and know what US objectives are, how the process works, and who the people are, they cannot possibly provide either relevant or timely intelligence that will contribute to better-informed decisions.

3. Place your trust in tradecraft. If the mission of intelligence analysis is to inform policymaking—to help the US government anticipate threats and seize opportunities—then customization of analysis is the essence of professional practice, not a defilement of it (i.e., politicization). In effect there is no such thing as an unprofessional policymaker question for intelligence to address so long as the answer reflects professional analytic tradecraft (e.g., tough-minded weighing of evidence and open-minded consideration of alternatives).

In the absence of explicit policymaker tasking, analysts should couch the assignment the way policy professionals would to get their jobs done, and then answer it like an intelligence professional—for example, a critical review of key foreign players and their stakes for and against a US policy initiative.

4. Learn from the DI’s own “best practices.” The DI regularly engages in two kinds of transactions with policy officials that greatly increase the utility of the analysts’ specialized substantive and tradecraft expertise for the policymaking process. To the extent applicable, the approaches used to inform policymaking in crisis management and with orally delivered analysis should be adopted in all intelligence deliverables.

Crisis Management analysis. Like anticipation of hanging, a policy crisis focuses the mind. For policymakers as well as analysts, crisis shortens the line of communication regarding what analytic support is needed and whether the deliverable did the job. Drawing on personal experience, or that of a colleague or a case study, assess what kinds of questions policy officials ask and what kinds of answers experienced analysts find appropriate. If a US warfighter needs and appreciates the analysts best judgment on how the United States can put pressure on the enemy’s leadership during a military engagement, why cannot analysts address in cost-benefit terms US tactical alternatives in a broad number of non-crisis assessments.

Orally delivered analysis. Delivering analytic support to policy officials directly—telephone exchanges, teleconferences, briefings, interagency meetings—also adds the efficiencies of instant guidance, feedback, and follow-up. Not only can the analyst gain a better measure of what his client needs, but the policy official also gains a better measure of what the analyst knows that can help get the policy job done. If these exchanges are deemed both professional and effective when delivered orally, why not adopt when appropriate the underlying techniques for written assessments. For example, simulate an oral exchange by organizing the written assessment by means of thoughtful policymaker questions and professionally couched analyst answers.

5. Balance estimative analysis with action analysis. DI analysts have an unqualified professional responsibility for comprehensive assessments, including—when justified by tradecraft standards—strategic warnings, delineation of perceived foreign obstacles to US objectives, and other forms of “bad news.” But a special effort should be made to ascertain whether the policy community has heard the message. If so, analysts should then supplement bottom-line estimating with action analysis; say, the opportunities in cost-benefit terms for incremental advancement of US interests. Policymakers are much more likely to pay serious heed to analysts’ assessments of what the United States “can’t do,” if these are balanced with assessments that address professionally what the United States “can do.”

Disclaimer:
All statements of fact, opinion, or analysis expressed in Occasional Papers are those of the authors. They do not necessarily reflect official positions or views of the Kent School, the Central Intelligence Agency, or any other US Government entity, past or present.

Nothing in the contents should be construed as asserting or implying US Government endorsement of an article's factual statements and interpretations.

These papers have been prepared with the support of Central Intelligence Agency funds and are published with the consent of the authors.