III International Confer. On Forest Fire Research
14th Confer. On Fire and Forest Meteorology
Vol. II, pp. 2209-2225, Luso, 16/20 November 1998

IMPROVING DECISION MAKING PROCESS FOR THE FOREST SERVICE WILDLAND FIRE SITUATION ANALYSIS

Armando González-Cabán (1) and Donald G. MacGregor(2)
(1) Pacific Southwest Research Station, USDA Forest Service, Forest Fire Laboratory, 4955 Canyon Crest Drive, Riverside, California 92507 Tel: (909) 680-1525; FAX: (909) 680-1501
Email: agonzalezcaban@fs.fed.us

(2) MacGregor-Bates, Inc., PO Box 10105, Eugene, Oregon 97440 Tel: (541) 942-5727;
FAX: (541) 942-8041 Email: donaldm@epud.net

SUMMARY
USDA Forest Service policy mandates that a wildland Fire Situation Analysis (WFSA) must be completed for escaped fires that extend into the next burning period (12 hours). The current WFSA process is manual, cumbersome, and a lengthy process to perform under the pressure conditions of an ongoing large wildfire. Seventy-one senior fire managers, resource managers, fire planners, and line officers from all five Federal agencies (the Forest Service, and USDI Bureau of Land Management, National Park Service, Bureau of Indian Affairs, and Fish and Wildlife Service) with fire protection responsibilities, participated in this study. Informants indicated that there was nothing wrong with the WFSA process but expressed concerns to pulling back from fire suppression activities to do a WFSA. Most informants believed that the WFSA process does a moderately good job in determining the final outcome of fire suppression strategies. Slightly over one third of informants thought that the process could “substantially control” large-fire suppression costs, while about one- fourth indicated that only a “very small amount of control: was possible. Informants noted that much of the information needed to execute the WFSA was not available at the time of the analysis or was not available in a readily usable form. The results of the study strongly suggest that a great number, if not the majority, of the problems associated with WFSA implementation are traceable (directly and indirectly) to potential inadequacies in training and education about alternative fire suppression strategies.

INTRODUCTION

The United States Department of Agriculture Forest Service (FS) is one of five Federal Government agencies with fire protection responsibilities (the others are the Bureau of Land Management, National Park Service, Bureau of Indian Affairs, and Fish and Wildlife Service). The Forest Service is responsible for protecting about 80 million hectares of forest lands. Protection of these lands is very expensive. During the last decade alone over $5 billion has been spent in fire suppression.
Forest Service policy on fire management requires that a fire situation analysis (FSA) be prepared by the responsible line officer, evaluating initial suppression action on each uncontrolled wildfire after the first burning period (Forest Service Manual (FSM) 5131). The Policy requires the FSA to be reviewed and validated daily before each burning period (12 hours). The FSM establishes that, at a minimum, the FSA should consider the expected suppression costs, damage, and probability of success or failure of containing the fire within some established boundaries. If the fire does not meet established fire management direction, including minimization of fire suppression costs and damage from fire, the fire should be declared an escaped fire.

After a fire has been declared escaped, the line officer is mandated to organize and implement an appropriate suppression response based on a Wildland Fire Situation Analysis (WFSA). A WFSA must be completed for all escaped fires that extend into the next burning period.
The WFSA decision making process requires the following steps: a) identification of evaluating criteria that reflect land and resource management objectives, potential suppression costs, potential damage to natural rescues, safety, and other concerns; b) development of suppression strategies in sufficient numbers to represent a reasonable range of alternatives for the situation; and c) analysis of suppression alternatives addressing the evaluation criteria and other significant economic, environmental, political and social concerns.
The current WFSA process is manual, cumbersome, and a lengthy process to perform under the pressure conditions of an ongoing large wildfire. These problems have affected its use in the intended manner mandated in the Manual. To develop a new WFSA process, or to improve the existing one, a clearer understanding is needed of how the current process is used, why it is used the way that is, what elements are missing from the process, and why it may not be used as intended.

The purpose of this study was to evaluate the WFSA process based on the experiences of senior fire management and natural resource management personnel by examining how WFSA is currently being used as a support for decision making in fire management; how well the information needs of WFSA are being met; the causes of poor quality WFSA’s; and potential improvements to the WFSA process (including information, training, and expertise) based on the judgments of experienced WFSA users.

METHODOLOGY

The study was conducted using both interview and survey research methodologies. Structure for the interviews was provided by a set of key questioning points based on elements of the WFSA process. The questions were structured in the form of an interview guide. Interviews were completed in about 1 1/2 hours to complete. The interviews were conducted in a discussive atmosphere and informants were encouraged to think broadly about the topics presented to them. Interviews were taped recorded to provide a basis for analysis of informant responses to key questioning in the interview guide. Two to three weeks following the interview, each participant received a follow-up survey. The follow-up survey probed key variables based on the interview.

Informants for the study were senior fire management personnel, resource specialists, and agency administrators from all fire management agencies: USDA Forest Service (FS), USDI Bureau of Land Management (BLM), National Parks Service (NPS), Bureau of Indian Affairs (BIA), and Fish and Wildlife Service (FWS). A total of 71 informants were included in the study.

Informants Selection Process

A Regional contact person was identified and briefed on the nature of the study for each of the six FS Western Regions (see figure 1), and asked to provide the names of 12 to 15 individuals having the relevant expertise and background in WFSA. Potential informants were contacted by telephone, briefed on the nature and purpose of the study, and a meeting time was scheduled when an interview could be conducted. This general strategy meant that some individuals identified as potential informants could not be included in the study because of time, availability, and distance constraints. Agencies and number of informants from each one included:

Agency Number
Forest Service (FS) 52
Forest Service/BLM (jointly) 4
Bureau of Land Management (BLM) 5
Bureau of Indian Affairs (BIA) 4
National Park Service (NPS) 3
Fish and Wildlife Service (FWS) 3

Figure1. Western states of the study area

Fire and WFSA Frequency

To determine the prevalent use of the WFSA process, we asked the informants to estimate the yearly average frequency of fires in their land management units, as well as the yearly average frequency of escaped fires requiring a WFSA. Both estimates were made as “best guesses” based on each informant’s experience in their respective land management unit. Because of the large area over which the study was done, it was expected that there would be considerable variability in fire frequency. These estimates ranged from a low of 25 fires per forest in some FS Rocky Mountain Region (Region 2) forests, to more than 500 for some forests in the FS Southwestern Region (Region 3). The estimates were considerably lower for the other federal agencies. All informants noted the tremendous variability in the number of fires from year to year.

Estimation of the yearly average number of fires requiring a WFSA proved to be much more difficult, largely because WFSA fires occur very infrequently and year-to-year variability is quite high. Estimates of the actual number of WFSA fires on a yearly average basis at the forest level ranged from a low of 0.1 (one WFSA every 10 years) to a high of seven. The typical range was from two to three fires per year. Again, however, informants noted the high level of variability in WFSA fires. At the Ranger District Level, the frequency of WFSA fires is even less. For many land management units, line officers may conduct a WFSA on a less-than yearly basis, and frequently only once or twice over a five- (or more) year period consequently, line officers may get little practice or feedback and therefore, are deficient in the WFSA process.

Perceived Value and Utility of the WFSA Process

A major determinant of the quality with which a decision making process is implemented is the perceived value of the process by those who use it. In both the interview and the follow-up survey we asked the informants to indicate their perception of the value or usefulness of the WFSA process. These included the usefulness of WFSA as a decision making tool; the ability of WFSA to identify the best fire suppression strategy; the ability of WFSA to consider land management objectives and final outcomes of a wildfire action; and the ability of the WFSA to control large fire costs.

RESULTS

Generalized Utility of WFSA

In general, the WFSA process is viewed as having utility for fire management decision making. Over 90% of the informants indicated that they found the WFSA process to be “somewhat” or “very” useful as a decision making tool in fire management, although only approximately one-third found it very useful (Table 1). None of the informants indicated that they found it not at all useful, and only a very small number (9.3%) found it slightly useful.

The WFSA process is intended to aid in the selection of the best fire management strategy for dealing with an escaped or wildfire incident. However, when asked whether the WFSA generally leads to selection of the best fire management strategy, only slightly over 52% indicated that it does so “always or most of the time.” Although none of the informants indicated that it rarely leads to the best strategy, a somewhat large percentage (46.3%) indicated that it “sometimes” lead to the best strategy. We can say that in general, informants were positive about the usefulness of WFSA as decision making tool, but slightly less so about its ability to aid strategy selection. The relationship of the two items in Table 1 shows that the general perception of the usefulness of WFSA was unrelated to its perceived ability to lead to the best fire management strategy (x2 = 3.78 ; df = 4; p > .20).

Table 1. Perceived value of WFSA by informants as a decision making process

In your experience, how useful have you found WFSA
as a decision making tool in fire management? Percentages
Not useful 0.0%
Slightly useful 9.3
somewhat useful 55.6
Very useful 35.2

In your experience, does WFSA generally lead to the
best fire suppression strategy?
Rarely 0.0%
Sometimes 46.3
Most of the time 51.9
Always 1.9

a Percentages for a given item may not total to 100% due to rounding.

Value of WFSA in Considering Outcomes and Controlling Wildfire Costs
The WFSA process includes a careful consideration of land management objectives and the impact of fire on those objectives. Thus, successful WFSA is, in part, one that selects a fire management strategy that is consistent with land management objectives and that achieves an outcome of the suppression effort that is consistent with the land management plan. In addition, the WFSA policy identifies the cost of suppression efforts as criteria against which alternative strategies should be compared. Given the escalating costs of suppressing wildfires, a reasonable question to ask of WFSA is the degree to which it can, if appropriately applied, help control such costs.

When considering land management objectives and final outcomes of fire suppression action, 65.4% indicated that the WFSA does this “moderately” well (Table 2). Very few informants indicated that WFSA accomplished this objective “very well” or “not well.” A strong relationship was found between WFSA’s ability to select the best fire management strategy and its consideration of outcomes; those who tended to judge WFSA as better at considering outcomes were also more likely to see WFSA as leading to the best fire management strategy (X2 = 15.23; df = 6; p <.01). It appears from these results that the perceived quality of WFSA as a decision making process is related, at least in part, to judgments about its ability consider outcomes of fire suppression actions in light of land management objectives.

Table 2. Perceived value of WFSA in meeting land management and controlling suppression costs

How well does WFSA consider land management
objectives and final outcomes of the wildfire action? Percentages
Not well 7.7%
Slightly 21.2
Moderately 65.4
Very well 5.8

With regard to the cost of large-fire suppression,
how much do you thing the WFSA process, if done
properly, can help control such costs?
Not at all 0.0%
Very small amount 22.2
Some control 46.3
Substantial control 31.5

a Percentages for a given item may not total to 100% due to rounding.
Control of suppression costs is a matter of critical concern in fire management. Informants were asked to indicate how much control the WFSA process can have over large-fire suppression costs, assuming the WFSA process was done properly (Table 2). A wide range of responses was exhibited in this question. Thirty-one and a half percent indicated that “substantial” control was possible, while another 46.3% indicated only “some control” was possible. At the very extreme, 22.2% indicated only a “very small amount” of control was possible.

Judgments about the reliability of WFSA to control wildfire suppression costs were not related to either general perception about the usefulness of WFSA as a decision making tool (X2 = 4.18; df = 4; p >.20), or to judgments about WFSA’s ability to lead to the best suppression strategy (X2 = 2.56; df = 4; p >.20).

Availability of Information for the WFSA

The WFSA process requires a number of types of information, including natural resource management objectives and natural resource values, much of which comes from the land management planning process. The availability and usability of this information for the WFSA process was investigated (Table 3). In the interviews informants frequently noted that much of the information needed to execute the WFSA was not available at the time of the analysis, or was not available in a form readily usable for the analysis. The information most frequently found lacking were critical resource objectives from the forest or land management plans and quantitative natural resource values use in quantitatively assessing the impact of alternative fire management strategies in terms of damage to the natural resource base.

Availability of Natural Resource Values for WFSA

Only 13% of the informants indicated that the natural resource values are “usually available” in a form that can be used directly in a WFSA (Table 3). More than 60% of the respondents indicated that resource values are “seldom available.” There was a weak relationship between the availability of natural resource values information and the general usefulness of the WFSA as a decision making tool. Availability of natural resource information was unrelated to judgment about the ability of the WFSA to select the best fire management strategy. These finding suggest that resource values are perceived to be of marginal importance in aiding actual fire management decision making. However, this does not mean that improvements in the availability of natural resource values are not or would not be viewed positively.

Table 3. Responses to two items assessing quality of information for WFSA

Are natural resource values typically available in a
form that can be used directly in a WFSA? Percentagesa
Not available 1.9%
Seldom available 60.4
Sometimes available 24.5
Usually available 13.2

Typically, how usable are land management plans
for the needs of WFSA?
Not usable 5.7%
Slightly usable 32.1
Somewhat usable 45.3
Very usable 17.0

a Percentages for a given item may not total to 100% due to rounding.

Usability of Land Management Plans by WFSA

Slightly more than 32% of respondents indicated that land management plans are “slightly usable” for the needs of WFSA, while 45.3% indicated that they are “somewhat usable.” A strong relationship was found between the usability of Land Management Planning (LMP’s) and the judged ability of the WFSA to lead to the best fire management strategy. Those who saw land management plans as more usable by the WFSA process also tended to judge the WFSA as leading to the best suppression strategy (X2 = 16.71; df = 6; p <.01). Thus, it appears that better quality land management planning is a necessary precursor to improving fire management decision making and to improve the quality of implementation of the WFSA process.

Assessment of Probability of Success/Failure

Assignment of probability of success/failure to each alternative suppression strategy included in the analysis is a key assessment in the WFSA process. From interviews results, it appears that probability assignment is the most difficult of all the judgments required in the WFSA. Informants tended to indicate that lack of fire experience leads to inadequate consideration of the national or regional availability of suppression resources and, therefore, to overly optimistic assessments of the likelihood that a given suppression strategy will accomplish its objectives (e.g., containment size, containment time, suppression costs).

When asked to judge the level of precision with which probability assessments of success/failure can be made very few informants (3.8%) thought that probabilities could be assessed with high precision (i.e., ± 5%). Only a slightly larger percentage (9.6%) thought that probabilities could only be assessed with a precision of ±40% or more. The vast majority thought that probabilities could be assessed with moderate to limited precision (i.e., ±15% to ±30%).

Causes of Poor Quality of WFSA’s

Informants were asked to judge how often each of a number of factors were attributable as causes of poor quality WFSA’s. The highest rated causes of poor WFSA’s were related to inadequacies in WFSA training and experience. For example, more than 74% indicated that poor WFSA’s are “often” or “almost always” caused be lack of adequate WFSA training on the part of the line officer responsible for the WFSA. Almost the same percentage (75.4%) indicated that poor WFSA’s are “often” or “almost always” caused by lack of line officer WFSA experience. It is not surprising then, that in a factor analysis the items of training and experience were the principal factor, explaining more than 33% of the variance. The overall factor analysis revealed that judgments of the causes of poor WFSA’s can be conceptualized as relating to lack of WFSA training and experience, lack of experience (fire management experience), and lack of natural resource and fire expertise (such as fire behavior analysis) needed to accomplish the WFSA.

Perceived Improvement of the WFSA Process

As part of the study, we asked informants to offer suggestions or ideas that would improve implementation of the WFSA process. A set of the most commonly mentioned were included in the follow-up questionnaire. Based on their experience, informants rated how much improvement would be gained from each of the suggestions. Responses were classified in a four-category response scale: no, slight, moderate or large improvement. Later the categories were collapse into only two: no/slight improvement, and moderate/large improvement. Only a subset of those judged to be contributing the most improvement to the WFSA process are shown in Table 4.

Table 4. Responses to follow-up survey question: “Listed below are a number of possible
changes to WFSA. In your opinion, and based on your experience, how much
improvement in WFSA would result from each one?” (Partial List)

IMPROVEMENT
Potential WFSA Improvement No/Slight Moderate/Large
Better Natural resource values 13.2% 86.8%

Better training on how to analyze alternative fire
suppression strategies 25.9 74.1

Better definition of the role of fire in land management plans 25.9 74.1

More fire management experience for line officers 20.4 79.6

a Percentages for a given item may not total to 100% due to rounding.

More than 86% of the respondents to the follow-up survey indicated that “better natural resource values would lead to a “moderate or large” improvement in the WFSA. This appears somewhat inconsistent with informants’ views regarding the need for economic analysis in the initial WFSA. This inconsistency can be explained by the informants perceived views that the WFSA process is taking on greater importance than in the past, and that fire management decisions today require stronger economic justification. Although the WFSA has always had an economic analysis, this aspect of the WFSA has been dealt with historically by using subjective ratings.

The results of a factor analysis (13 categories considered) revealed that 70% of the variance in informants’ responses to items relating to improvements to the WFSA process can be explained by four factors. The four general categories of factors include better training in analysis of wildland fire, better natural resources values and planning, more fire experience incorporated into the WFSA process, and better planning concerning the role of fire.

Computerized Version of the WFSA

A computerized version of the WFSA (PC-WFSA) was being introduced in some far-western forests, particularly in the FS Pacific Northwest Region (Region 6, Oregon and Washington), and the Pacific Southwest Region (Region 5, northern California) while conducting this study. We did not address any issues related to implementation of the computerized PC-WFSA. However, it was clear to us that many informants had either been using the PC-WFSA, or had beliefs about how it might benefit (or not benefit) the WFSA process once it became available to them.
When asked about whether the PC-WFSA would yield much improvement in the WFSA process more than 69% thought that it would; while almost one-third (30.7%) thought that it would yield little or no improvement. By and large, informants thought that the PC-WFSA would help people “get through the process.” However, very few individuals interviewed indicated that the PC-WFSA would lead to a better quality analysis of fire suppression alternatives.

CONCLUSIONS AND RECOMMENDATIONS

The results of the study provide no evidence to indicate that the policy regarding WFSA, as identified and defined in Forest Service Manual (FSM) 5100, is in some way flawed, broken or otherwise inapplicable to decision making in the domain of fire management. Nor is there any evidence to indicate that the policy as stated in FSM 5100 cannot, in principle, be implemented. When asked of interview informants what type of process should be applied in wildland fire situations, assuming no process were presently in place, the answer typically received was “a process like we have now.” None of the informants offered any indication that an analysis of alternatives based on decision analysis makes no sense in the context of wildland fire management. Nor was there any indication that fire managers should be managing fire solely on the basis of their intuitions rather than conduct a formal analysis. However, informants frequently note that it is difficult to suspend fire suppression and do the analysis.
What the results of the study do indicate is that the WFSA process is generally viewed as a useful form of analysis for decision making in fire management, but that its utility is often compromised by a variety of problems associated with how the process has been implemented and with the base of fire experience among those who are charged with conducting the WFSA. A key issue appears to be the quality and extent of training provided in WFSA -- essentially, the WFSA process is rooted in decision analysis. For the most part, WFSA training appears to have focused on its procedural aspects, rather than on a broader education in the principles of analysis. A deeper review of current WFSA training methods and materials is required to identify how training could be improved.

A general observation based on a large number of interviews with senior fir management personnel is that the WFSA is a relatively simple and robust process that entered into fire management at a time when the vast majority of the line officers had a great deal of fire suppression experience. Though the WFSA itself is not a tactical document, it does call upon those responsible for its execution to make a number of judgments that require tactical knowledge and experience. Historically, the fire suppression knowledge and expertise was inherent in the line officer position. However, fire suppression experience is less a hallmark of the line officer position today than it was historically the case. Consequently, much of the expertise required to accomplish the WFSA must come from others on the line officer’s staff, particularly the fire management officer’s and, or the resource area specialists. In some situations, these individuals are not available at the time the WFSA must be done, particularly in forests or districts that are minimally staffed. Thus, the WFSA process faces a number of implementation challenges that need to be addressed if the quality of WFSA’s done in the field is to be maintained and, or improved. It is in this spirit that the following recommendations are made.

Leave Current WFSA Policy Unchanged

There is no evidence from the present study that the current WFSA policy is either broken or flawed, nor is there evidence that the policy is fundamentally unimplementable. Policy changes without undertaking steps to improve WFSA implementation would be premature. The current WFSA policy is rooted in principles from the decision sciences. These same principles have been applied successfully in many other decision making contexts. There is no reason to believe that they cannot be applied successfully in fire management as well.

Review and Evaluate Current WFSA Training Practices

A great number, if not the majority, of the problems associated with WFSA implementation are traceable (directly and indirectly) to potential inadequacies in training and education surrounding the analysis of alternative fire suppression strategies. The current training program, including materials, for WFSA should be thoroughly reviewed to determine the degree to which it meets the requirements necessary to effectively implement the WFSA policy. Training and education in decision analysis, including generation of alternatives and assessment of uncertainties appear to be either deficient or (in some cases) absent. The relatively low frequency of WFSA fires (i.e., 1% or less of total fires) suggests that the skills necessary to maintain WFSA proficiency may not be exercised with sufficient frequency to insure good quality WFSA’s. The potential for WFSA proficiency standards should be explores, with periodic review and retraining if necessary.

Strengthen the Link Between Land Management Planning and the WFSA

A fundamental requirement for a high quality WFSA is a good set of natural resource management objectives. Currently, the land management planning process is not closely enough linked with the needs of WFSA to meet this requirement. The potential to link the WFSA to land management planning and documentation should be explored. For example, management areas within forest plans could be made available to the WFSA directly. In addition, a greater consideration of the role of fire in land management planning needs to be incorporated directly into land management plans and made available to the WFSA process.

Improve the Availability and Usability of Natural Resource Values for the WFSA

Better quality natural resource values need to be made available for the WFSA process. Currently, the WFSA’s appear to vary widely in the degree to which quantitative natural resource values are used as a basis for comparing alternative strategies in terms of impacts to the natural resource base. The potential for disseminating a database of natural resource values through a computerized WFSA should be explored. Additional efforts should be devoted to developing reliable and meaningful values for noncommodity natural resources.

Evaluate the Potential of the PC-WFSA to Support Judgment and Decision
Processes Relevant to the WFSA

The potential for additional WFSA decision support through the PC-WFSA implementation should be explored. For example, case or rule-based systems to assist with difficult judgment situations, such as cost estimation, should be examined as potential enhancements to the PC-WFSA. Likewise, the potential for front-loading local WFSA information based on land management planning areas should be studied. Some forests have already taken steps along these lines, and their experiences should be documented and disseminated to other forest who have not yet implemented the PC-WFSA process to the same degree.

Identify and Evaluate Approaches for Increasing the Level of Fire Suppression
Experience Available to Conduct the WFSA

A theme that consistently arose in the interviews and was reflected in the follow-up survey is the need for line officers to have fire experience in order to carry out a good WFSA. There was also a clear perception on the part of study informants that a significant portion of the line officer corps lacks such experience. At this time, is difficult to make a specific recommendation concerning how to address this problem. On the one hand, line officers routinely have available staff personnel with fire management experience. On the other hand, these individuals are sometimes not available at the time the WFSA is done.

We want to emphasize that analyzing fire management decisions is quite different from making fire management decisions. The former (analysis) is prospective and involves structuring a decision problem, establishing goals and objectives, assessing values and obtaining information. The latter (decision making) involves integration, justification and responsibility. In this latter area, the line officer has available the results of the WFSA plus whatever other experience and knowledge they have to make the best decision possible at the time. As some interview informants commented, “the WFSA can only take one so far; after that, it is a matter of experience.” Clearly, there is a need to increase the base of fire experience among the line office corps if the results of WFSA’s are to bear a closer match to both expected and actual outcomes of fire management activities. Approaches for establishing a clear delineation and documented delegation of responsibilities of WFSA components to land management and incident team members should be explored. In addition, approaches for increasing the fire experience base of line officers, relevant to conduct of the WFSA, should be examined. These options could include applying relevant WFSA and fire experience in nearby districts, making more use of computerized decision support technology, use of standardized WFSA templates, and greater use of preplanned WFSA’s.

Acknowledgments

We want to thank the dedicated fire management professionals who participated in this study. It was their willingness, cooperation and sincere dedication to improving fire and natural resource management practices that made this effort possible. They are outstanding professionals, all, and it was a pleasure working with them. This study was funded by the USDA Forest Service under Research Joint Venture Agreement #PSW-97-004-RJVA between the Pacific Southwest Research Station and MacGregor-Bates, Inc.