SR/SSD 97-16 4-15-97

Technical Attachment

SUMMARY OF THE OSF WARNING DECISION-MAKING WORKSHOP
"TO WARN OR NOT TO WARN"

Kevin Pence, NWSFO Birmingham
Henry Steigerwaldt, NWSO Nashville

1. Introduction

The Operational Support Facility (OSF) Training Branch conducted the first Warning Decision-Making Workshop during the period of March 18-20, 1997. The instructors were Liz Quoetone and John Ferree (OSF/OTB), David Andra (SOO, NWSFO Norman), and Bill Bunting (WCM, NWSO Pleasant Hill). There were 35 NWS forecasters from across the country who participated in the three-day workshop, including 14 from the Southern Region. The goal of the workshop was to discuss the decision-making process as it relates to the NWS warning environment.

2. Situation Awareness

A guest speaker, Dr. Debra Jones, discussed the first topic of the workshop--situation awareness (SA). A high percentage of poor decision-making comes down to errors in situation awareness. SA represents a person's understanding of a situation. SA takes time to acquire, but it can be lost very quickly. Situation awareness is not the same thing as decision quality. A forecaster may get lucky and make a good warning decision even though that person has poor situation awareness. On the other hand, the forecaster could also make what was perceived as a good warning decision at the time, but it turned out to be a bad decision because the situation was misinterpreted. Factors that may affect SA include attention (including the avoidance of distractions), working memory, utilization of mental models, and workload.

The presence of biases during the decision process can adversely affect decision quality. For example, a conservative bias takes place when a forecaster approaches a situation with a certain idea about what is happening and sticks with this idea longer than he or she should, even when presented with contrary evidence. A forecaster actively looking for information to confirm his or her idea about what is happening, while ignoring disconfirming evidence, could be described as having a confirmation bias. There also are three levels of SA errors:

3. Survey Results

Participants were given surveys to fill out prior to attending the OSF workshop. We were asked to list various factors that were significant for assuring a successful warning event and to rank them in order of importance. The factor identified as number one by most attendees was understand the pre-storm environment, followed by (in order):

4. Warning Methodology

Why do we issue warnings? We issue warnings because life and/or property are threatened, the thunderstorm is severe; it is too close to not warn, false reports, and data misinterpretation. Why are events missed? Some severe weather events are missed because the warning criteria were not met, relevant data were not observed, lack of reports, and warning criteria met but not recognized.

The warning process involves anticipation, product selection; feature recognition; spotter reports; generating and disseminating the warning; and non-meteorological factors such as staffing, experience of staff, equipment problems, human stress and fatigue. Warning events are most effective if the event is anticipated, relevant data are available, and the warning forecaster has time and ability to correctly assimilate and quickly react. To reach a high level of warning success requires training, experience, preparation, and support.

Factors affecting performance in a warning environment can be grouped into three general categories:

Task-related variables include knowledge of the pre-storm/storm environment to anticipate different modes of convection, radar presentation, storm history, and information flow and dissemination. Regarding the last item, there should be a steady stream of sensitized weather information that is specific and frequently updated. It is critical that towns and times of arrival are mentioned in warnings and updates. It is also important to include quantitative threat information, such as "dime size" or "golf ball size hail" instead of "large hail." Avoid hedging terms such as "possible." Include storm reports in real-time.

Emergency management officials love the severe weather outlook product. The ideal scenario would begin with a severe weather outlook, followed by severe thunderstorms in the forecast, then a watch, then a nowcast describing where and when severe thunderstorms can be expected, followed by warnings, and finally warning updates. The warning updates (SVSs) should be short and issued frequently. Every warning needs to be updated at least once!

Physical environment-related variables include the operations area layout (proximity to dissemination systems, etc.), information sources, and the warning operations plan (which clearly establishes responsibilities for each person involved in the office warning process).

Decision-maker variables include experience, education and training, the ability to perform under stress in a team structure, alertness, and strategies for optimizing performance.

5. Mini Scenarios

One of the most enjoyable parts of the workshop was the time spent on warning mini-scenarios. We were separated into groups and had to decide what kind of warnings, if any, to issue for a given set of parameters. The parameters usually included a very general description of the synoptic and mesoscale situation, section of the country, time of the day and year, and basic radar information. Each group weighed all of the parameters individually, as either positive or negative as far as severe weather was concerned, as part of the warning decision process. There were 36 unique mini-scenarios that were discussed by the class.

Each participant was given a CD-ROM containing slides (that are largely self-explanatory) which were used by instructors during the presentation on "Warning Methodology." The CD-ROM also contains a warning scenario that allows users to analyze radar data and make warning-decisions for three different convective systems. Participants were encouraged to take this material back to their offices and share it with their own staffs as part of local training activities. (Copies of the same materials have also been provided to the other Southern Region WFOs.) This CD-ROM is a beta version which will be enhanced in the near future, largely to incorporate suggestions from the workshop attendees. Although OSF/OTB will plan to conduct additional Warning Decision-Making workshops in the future, it may be some time before significant numbers of forecasters can attend.

6. Weaknesses

The next part of the workshop involved addressing some weaknesses that each office felt they had to deal with during severe weather events. We were once again divided into groups, and each group had a separate list of weaknesses. Each group did some brainstorming and came up with potential ways to improve each weakness. The list of weaknesses included the following:

7. WDSS

J. T. Johnson (NSSL) made a presentation on the last day of the workshop detailing the features and capabilities of the Warning Decision Support System (WDSS). WDSS has been implemented at a number of NWS offices (Norman, Fort Worth, Jackson, and Melbourne in the Southern Region) as part of its operational evaluation. The system is an alternative to the RPG, in the sense that it ingests WSR-88D base data from the RDA and runs its own set of processing algorithms. Many of the new WSR-88D algorithms developed by NSSL are first tested as part of the WDSS software before they are made available to the OSF. Output from WDSS, for example, provides numerous trends of radar data, all relevant products (no RPS list), and the system uses enhanced algorithms. NSSL developed the latest WSR-88D storm cell identification and tracking algorithm, as well as the hail detection algorithm. They are presently fine-tuning mesocyclone detection, tornado detection, and downburst prediction algorithms. A WDSS type of system is expected to eventually end up being part of AWIPS.

8. Conclusion

Issuing timely and accurate warnings is the most important function of the NWS. Anything that can assist in more timely and accurate warnings should be utilized. The warning decision-making process involves more than just an understanding of meteorology. The materials presented in the workshop, the discussions that ensued between course instructors and students and among the students themselves, the information taken back to each office, and the interactive training module can all be used to improve the warning decision-making process.

We believe this workshop conducted by the OSF Training Branch has contributed to this effort by providing additional education to field forecasters on the decision-making process as it relates to the NWS warning environment. To extend the benefits of the workshop to as many as possible, it is up to the participants to pass this valuable information on to others at their respective offices.

For additional information about the Operational Support Facility and the Operations Training Branch, including PC-GRIDDS data and more materials related to the CD-ROM warning scenarios, check out the following Internet home pages:

Operational Support Facility--http://www.osf.noaa.gov
Operations Training Branch -- http://www.osf.noaa.gov/otb/otb.htm