Summary

Using advanced systems and trained specialists located in 122 weather forecast offices throughout the country, the National Weather Service (NWS) provides storm and flood warnings and weather forecasts to protect life and property and to enhance the national economy. To improve the efficiency of its operations, in November 2006, NWS approved an effort to develop a prototype of an alternative way of operating. Under this prototype, weather forecasting offices would share selected responsibilities. GAO (1) determined the status of and plans for the prototype, (2) evaluated whether the prototype's justification was sufficient, (3) determined whether NWS's plans to evaluate the prototype are adequate, (4) evaluated whether NWS is sufficiently involving stakeholders in its prototype plans, and (5) determined how NWS plans to ensure that there will be no degradation of service during and after the prototype. To do so, GAO analyzed agency documentation and interviewed program officials and stakeholders.

NWS's prototype is currently on hold pending a reevaluation of the agency's approach. The agency had just begun the first phase of its three-phased prototype to demonstrate a new concept of operations over a 2-year period when, in late March 2007, the Department of Commerce's Under Secretary suspended the prototype because of concerns about the agency's approach. In the first phase, NWS established a program manager and began planning for the next two phases. During the remaining phases, NWS planned to have 20 weather forecasting offices share responsibilities in 2-office pairs and then in 4-office clusters. NWS then planned to decide whether to implement the new concept of operations on a national basis. The justification for the prototype was not sufficient. Before the prototype was suspended, the agency had approved moving forward with its prototype without conducting a cost-benefit analysis. NWS estimated that the prototype would cost approximately $9.3 million and would offer qualitative benefits, such as increased efficiency and an improved ability to focus on severe weather events, but did not quantify benefits or the expected return on its investment. If NWS were to proceed with the prototype without a cost-benefit analysis, it would lack assurance that its approach would be a cost-effective investment for the agency. NWS identified goals and selected measures to evaluate during its prototype activities, but it did not establish a rigorous evaluation plan. Specifically, NWS did not define a full set of needed measures, how it planned to compare prototype results with baseline performance, or how its selected measures supported the prototype goals. If NWS were to proceed without a rigorous evaluation plan, the agency would run an increased risk of not sufficiently measuring the impact of changes on its performance and could make decisions affecting the nation's weather on the basis of incomplete or flawed data. Although NWS involved internal stakeholders in planning its prototype, it did not (1) involve external stakeholders or (2) establish a plan that identified key stakeholders, the stakeholders' responsibilities, and a time line for involving stakeholders and addressing their comments. If NWS were to proceed with its prototype without such a plan, it could not ensure that stakeholder interests would be identified and addressed. NWS planned to mitigate the risk of degradation during and after the prototype by conducting laboratory exercises to understand the impact of the prototype, using a dedicated workstation in each office so that the office could switch to original systems if warranted, and monitoring its systems and products during the prototype. NWS officials stated that should the agency decide to implement the prototype on a national basis, it has standard procedures for testing and validating systems and software to avoid any degradation of service.

Recommendations

Our recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Implemented" or "Not implemented" based on our follow up work.

Recommendations for Executive Action