|
Brainstorming
ACSRB's planning process began by inviting leading applied cancer screening experts and academics from the public and private sectors to participate in an anonymous web-based brainstorming process. Experts were identified through authorship on relevant peer-reviewed publications, known involvement or interest in cancer screening related activities, and/or because they had received National Cancer Institute (NCI) support to conduct applied cancer screening research. Additionally, an ad hoc core planning committee of experts was convened to assist with interpreting data and finalizing the plan (Table 1). Because the web-based portion of the process was anonymous, it is not known how many experts actually participated (other than those on the core planning committee).
All participants and committee members were asked to brainstorm statements that would complete the following:
"In order to address its mission, one specific research area and/or topic that I think the ACSRB should focus on over the next 3 years is…."
Brainstorming elicited more than 180 responses, which were then edited for redundancy. The list was pared down to a final group of 80 statements (Table 2).
Sorting and Rating Statements
In the next phase, statements generated during the brainstorming session were sorted and rated. Initially, individual statements were sorted into gross categories of similar thematic statements and labels were applied to the new thematic categories (Table 3). The only restrictions were that sorters could not make 80 groups containing only one statement, could not make one pile with all 80 statements, and could not make a group of "miscellaneous" statements with dissimilar items.
Next, individual statements were rated on two dimensions - "importance" (compared with other statements) and "feasibility" (relative to the next three to five years) - on a five-point Likert-type response scale (1=relatively unimportant compared with the rest of the statements" and 5=extremely important compared to the rest of the thematic statements) for relative "importance" to ACSRB. For each statement, the mean of the ratings was obtained.
Interpreting Results
Data (i.e., the 80 statements) underwent a two-part analysis that involved multidimensional scaling and hierarchical cluster analysis (Kruskal and Wish, 1978; Davidson, 1983). These results, displayed as a series of maps based on the analytic structure derived from the planning process (e.g., sorting and ranking), were presented to the ad hoc planning committee to assist with interpretation and refinement of the final product.
The first graphic displays a cluster-point or statement map which locates each of the 80 statements as a point (Figure 1). Next to each point is the number of the statement so each point is identified. Statements, within close proximity to each other, were more likely to have been sorted together more frequently and represent thematic similarity in comparison with more distant statements, with peripheral correspondence. More distant statements were sorted together less frequently.
Once committee members familiarized themselves with the point map, it became apparent that the numbers formed a general pattern with 10 distinct groups. These distinctions were discussed and partitions were drawn on the map to indicate different regions, which were subsequently identified as the 10 priority cancer screening areas or priority themes.
Among the 10-priority cancer screening areas highlighted below (the number of individual statements comprising each thematic priority are indicated in parenthesis), it was apparent that any one of these pathways alone could advance knowledge and understanding of cancer screening, but the overarching premise identified in this process was the wide and diverse areas of research considered "important". A description of statements grouped by priority area can be found in Table 3.
A second cluster map (Figure 2) shows the four clusters that represent higher order conceptual groups of the original 80 statements and 10 priority areas, each enclosed by polygon-shaped boundaries. It should be noted, that larger clusters encompass broader concepts and smaller clusters contain concepts that are narrower in scope. Contained within any cluster are statements that are closer to an adjacent cluster than to other statements in its own priority area group. A third cluster map shows the cluster areas clearly defined (Figure 3). Categorization of the four thematic areas and complementary priority areas are described as follows:
- Health Services Research includes the primary area of cost effectiveness as well as part of the decision making, Health Services, and Policy areas.
- Theory and Methods contains two major areas, Methodology and Theory.
- Risk Communication includes the primary area of Genetic Risk, as well as part of the Communication and decision making areas.
- Unscreened Populations comprises two primary areas, Determinants and Disparities, together with part of the Communication area.
Two subsequent maps show how the 10 priority areas were rated in terms of "importance" and "feasibility". The two-dimensional Importance Rating Map (Figure 4) illustrates the relative average importance of each priority area. The number of layers indicates the influence of the "importance" rating. The higher the area, the more important it is. The average represented by the layers is actually a double averaging - across all participants and all of the factors in each priority area. The map clearly shows that statements associated with Theory ranked highest in importance followed by Disparities, Determinants, cost effectiveness, and decision making.
In a two-dimensional Feasibility Rating Map, perceived "feasibility" was averaged across all individuals and across each item in a priority area (Figure 5). As with the Importance Rating Map, the depth of the layering suggests which of the 10 priority areas should be given more emphasis because of their significance, viability, and potential to address the Branch's mission. The map of perceived feasibility suggests that cost effectiveness, Communication, and decision making were considered most feasible and Policy the least feasible to achieve change over the next 3 to 5 years.
Viewed from a different perspective is a ladder graph which shows correlated average scores for those priority areas that ranked above average in both "importance" and "feasibility" (Figure 6). As suggested by this graph, all thematic priority areas except Policy ranked as highly "feasible". Also of note here is the compilation of priority area statements that ranked above average in "importance" but not above average in "feasibility". These lower ranked priority area statements indicate potential barriers to implementing the Branch's mission. Similarly, priority area statements that ranked above average in "importance" and "feasibility" may not automatically become priorities for the Branch because they may already be the responsibility of another National Cancer Institute division, or another non-Federal institution, or they may already be well funded or explored.
To examine the relationship between "importance" and "feasibility" for individual priority areas, a series of figures shows the two variables plotted against one another in a bivariate plot. Statement identification numbers in the figures can be linked to specific priority area statements resulting from the initial brainstorming (Figures 7-16).
Concept Map Synthesis
Each of the previous maps emphasizes a different part of the conceptual information, and conveys useful facts essential to understand the entire picture. For instance, when sorting statements into priority areas, it was apparent that individual statements contained several different ideas and to reach consensus on how to describe priority areas succinctly and accurately was challenging. Also challenging was the overlapping nature of the priority areas, which created problems when attempting to navigate between the larger strategic view and translating the perspective into specific actions. The process became clearer when subsequent cluster maps displayed individual statements into conceptual regions that began to resemble distinct geographic sections. The visual structure of the point and cluster maps, further strengthened by the "importance" and "feasibility" ratings, provided additional analytic insight and after discussion, eventually formed the basis for a strategic plan.
As a combined effort the concept map represented the insight of prevailing thinking about macro level areas and served as the road map for ACSRB's strategic plan (Figure 17). The plan identifies thematic priority areas, in addition to core areas that are consistent with the ACSRB mission. The strategic plan developed through concept mapping methodology is a vehicle to guide creative development of research proposals that address the Branch's mission and advance applied cancer screening knowledge. Investigators are encouraged to give consideration to these thematic areas as they develop new research proposals.
Priority Areas for Applied Cancer Screening Research
References
Davidson M.L. (1983). Multidimensional Scaling. New York, NY: John Wiley and Sons.
Kruskal J.H. & Wish M. (1978). Multidimensional Scaling. Beverly Hills, CA: Sage Publications.
Trochim W. (1989). An introduction to concept mapping for planning and evaluation. In
W. Trochim (Ed.) A Special Issue of Evaluation and Program Planning, 12, 1-16.
Concept Systems Incorporated: http://www.conceptsystems.com/
|
