Q-Sort Methodology

Observations always involve theory. --Edwin Hubble[1] 

As described earlier, this work reflects triangulation of the data derived from the literature Q-sort, interview responses, and observations.[2] The data include 489 interviews, direct and participant observation of 325 analysts performing their jobs, participation in a variety of analytic tasks, and focus groups conducted to generate the taxonomy of variables that guided this study.

The first Q-sort of the data was aggregated according to the function of each intelligence organization, as listed in Table 1. The data were then analyzed to determine response context according to job type and to develop variable categories.

The organizational Q-sort generated the broad variable groupings used to create the second Q-sort parameters. The variable categories that emerged during the interpretive analysis of the first Q-sort of the data were compiled again, and a second Q-sort was performed based on those categories. The data was then aggregated according to categorical or variable groupings of the second Q-sort, Table 2.

The use of two separate Q-sort strategies generated the variables and then de-contextualized the data in order to find consistent trends throughout the Intelligence Community. That is, this strategy resulted in broad categories of findings that apply across many agencies. In those cases where interview and observational data could have been sorted into several categories, I based the placement of the data on the question that generated the interview response.

Table 1. Q-Sort 1. Data Grouping According to Organizational Function.

In several instances throughout the text, the quotes that were used may well fit in a number of other categories. Once the data were sorted by variable, the coding and context identifier notes were removed from all data in order to assure participant anonymity, in keeping with the American Anthropological Association Code of Ethics, section III, A.[3]

Table 2. Q-Sort 2. Data Grouping According to Variable Categories.

The quotes that appear throughout the text are exemplars from each variable category and indicate trends found in the data-set. Although the exemplar quotes are not universal, nor are they necessarily subject to generalization, they do represent consistent findings from the interview and observation data. Utilizing this approach to develop theory is similar to the method in which grounded theory is employed in sociology, specifically, using grounded data to generate theory rather than using some a priori technique. The significant advantage to this approach is that the theory is directly tied to data, providing it additional validity. Another advantage is that the individuals who allowed me to interview and observe them are given some voice in the final product by way of direct quotes, which also provides some qualitative context.

Footnotes:

[1]Edwin Hubble discovered the first evidence to support the Big Bang theory that the universe is expanding and that the Milky Way is not the only galaxy in the universe. He also developed the Hubble Galaxy Classification System and Hubble’s Law (the farther away a galaxy is from Earth, the faster its motion away from Earth). Edwin Hubble, The Realm of the Nebulae.

[2]William Stephenson, The Study of Behavior: Q-Technique and its Methodology.

[3]American Anthropological Association, Code of Ethics of the American Anthropological Association.