The Techniques Development Laboratory (TDL) currently is developing a new objective forecast guidance package based on the Aviation run (AVN) of the NCEP Global Spectral Model (GSM; Kanamitsu 1989). Two components of this package are the probability of precipitation (PoP) and the quantitative precipitation forecast (QPF). TDL is employing the Model Output Statistics (MOS) technique (Glahn and Lowry 1972) to generate PoP/QPF guidance at projections up to 72 hours from the 0000 and 1200 UTC model cycle times. The MOS technique gives an idea of the surface weather to expect in light of historical observations under similar model conditions. This new AVN PoP/QPF system represents the first time that TDL's primary short-range statistical guidance package includes precipitation amount forecasts through day three. Since a goal of the National Weather Service (NWS) is to extend operational precipitation forecasts into the medium range (NWS 1999), this new system should be an important tool for NWS field forecasters.

In this paper, we discuss the development and configuration of the new AVN MOS PoP/QPF guidance and explain any differences from previous MOS systems. We also present preliminary verification results from tests conducted on independent samples out to the day two and day three projections. We highlight any improvement over previous MOS systems, specifically the current short-range MOS system based on the Nested Grid Model (NGM; Hoke et al. 1989). Finally, we discuss challenges facing this new AVN MOS development effort.

2. THE MOS PoP/QPF SYSTEM

In 1989, TDL implemented the first NGM- based operational MOS PoP guidance (Jacks et al. 1990) for 204 stations in the continental United States (CONUS). This guidance provided forecasts of the probability that 0.01 inches or more of liquid-equivalent precipitation would occur in 12-h periods ending at 24, 36, 48, and 60 hours after 0000 and 1200 UTC. Since the NGM only generates forecasts through the 48-h projection, all NGM-based MOS guidance at projections beyond 48 hours is based on an extrapolation in which 48-h model fields are used as predictors of weather elements at the later projections.

In 1992, TDL expanded this NGM MOS guidance to include PoP forecasts for 717 CONUS stations and to provide PoP forecasts for both 6-h and 12-h periods (Su 1993). Categorical forecasts of quantitative precipitation for these same time periods were added to the NGM MOS package in 1993 (Antolik 1995, 1998a). QPF guidance is derived from several equations, each equation giving the probability that liquid-equivalent precipitation equal to or exceeding a specific cutoff value will occur. The set of probability forecasts is converted to one categorical forecast. Thus, the cutoff values from this set of equations form the bounds of the MOS QPF categories. There are five categories for the 6-h periods and six categories for the 12-h periods (Table 1). In 1995, TDL completed the NGM MOS precipitation package by adding PoP and QPF data for 60 Alaska locations. These QPF and PoP forecasts constitute the primary statistical precipitation guidance currently available to NWS and other forecasters for projections out to 60 hours.

Table 1. MOS QPF Categories.

Category	Forecast Precipitation Amount (inches)
0	No measurable precipitation
1	0.01 - 0.09
2	0.10 - 0.24
3	0.25 - 0.49
4	0.50 - 0.99
5	1.00 or more (6-h periods)
	1.00 - 1.99 (12-h periods)
6	2.00 or more (12-h periods only)

In 1992, TDL implemented a medium- range guidance package that included objective PoP guidance for 12- and 24-h periods at projections out to 192 hours for over 200 stations in Alaska and the CONUS. The PoP guidance was produced by applying a calibrated perfect prog technique (Jensenius et al. 1992) to output from the Medium-Range Forecast (MRF) run of the GSM. In 1994, analogous PoP guidance was produced for projections out to 72 hours from both the 0000 and 1200 UTC cycles of the AVN run of the GSM. A MOS-based PoP product utilizing output from the GSM replaced the perfect prog guidance beginning in 1995. The AVN-based MOS guidance is discussed in Jensenius et al. (1995).

TDL currently is developing an enhanced AVN-based MOS package that will replace the NGM-based MOS as the primary statistical guidance package of the NWS. Like the current AVN MOS system, the new AVN MOS will include PoP guidance for projections out to 72 hours. How ever, there are several significant changes planned for the new package. For the first time, QPF guidance will be available out to 72 hours. Also, 24-h PoP/QPF will be provided to supplement the 6-h and 12-h information. In addition, the number of stations for which guidance will be available eventually will exceed 1000. These will include new sites in the CONUS and Alaska and, for the first time, stations in Hawaii and Puerto Rico. Finally, a more subtle change in the guidance will be the elimination of a separate PoP development effort. For the new AVN MOS guidance package, PoP and QPF will be developed simultaneously. This will not only streamline the development process, but should also result in greater consistency between forecasts of these two elements.

3. EARLY TEST RESULTS

Development efforts for the new AVN MOS PoP/QPF are still underway at TDL. Final test results are not available at this time. In this paper, however, we will discuss early results and whether the new package might perform at a level consistent with the current operational systems.

Experimental equations for the new AVN MOS system were developed by using a dependent sample from the warm season (April-September) of 1997 and the first half of each 1998 warm season month. To expand the dependent sample somewhat and to provide for a smoother transition between seasons, additional dependent data were included from the 15 days before and after both warm seasons. Probabilistic forecast equations for 12-h periods at each of the six MOS QPF thresholds (Table 1) were derived for the 0000 UTC run of the AVN model out to 72 hours. This development included 534 CONUS stations grouped into 10 regions based on a precipitation climatology. The purpose of grouping stations during development is to increase the sample size used to develop equations. This in turn improves the stability of the equations, especially for rare events.

These experimental equations were tested on independent data from the last half of each 1998 warm season month. Results from this set of equations were compared to results from the corresponding set of equations used in the current NGM and AVN systems. The Brier score (Brier 1950), which for a two-category system measures the mean squared error of the forecast probability of a precipitation event, was used as the basis for comparison. Lower Brier scores correspond to greater forecast accuracy.

Figure 1 compares the performance of the NGM MOS, AVN MOS, and new AVN MOS at 197 stations common to each system. Scores were calculated for four thresholds corresponding to the lower bounds of MOS categories 1 - 4 (Table 1) out to 72 hours. For the 0.01 inch threshold, forecasts from the new AVN MOS were compared to the existing NGM and AVN MOS PoP equations.

Fig. 1. Brier score comparisons for the NGM MOS (squares), current AVN MOS (plus signs), and new AVN MOS (triangles) at 197 stations common to each system. The forecasts verified were valid for 12-h periods ending at the time shown. Sets of lines show scores for forecasts of the cumulative probability of occurrence of each of the precipitation amounts indicated.

For such an early development result, the performance of the new AVN MOS appears to be remarkably similar to that of the current MOS systems. Each system shows an expected increase in forecast error with increasing projection. The characteristic decrease in Brier score as the overall likelihood of an event decreases (from 0.01 inches to 0.50 inches) also is evident for the NGM MOS and new AVN MOS.

4. DISCUSSION

Much work remains to be done before the new AVN MOS PoP/QPF system is ready for implementation. Even so, the new system, at this early stage, does show promise of performing as well as or better than the current NGM MOS. This may be due in part to performance improvements in the underlying numerical model, as well as to the manner in which we have handled other issues discussed in this section. These issues include the ever-changing AVN model, the small sample size, and the nature of precipitation data obtained from the Automated Surface Observation system (ASOS).

The MOS approach tends to give the most reliable results when a sufficient sample of developmental data is available to derive equations, and when the underlying numerical model is not undergoing changes (Glahn et al. 1991). However, for the current development effort, there is neither a large sample of developmental data nor a stable numerical model (e.g., NWS 1998). Experience has demonstrated that careful use of the MOS approach is required in this situation. Since the MOS approach takes into account the specific characteristics of the model from which the equations were derived, any changes to that model can affect the MOS forecasts. For instance, if changes to the model eliminate random errors, it is quite possible that the MOS forecast may actually improve. If, however, a particular bias is removed from a model variable, and if that variable is used as a predictor in the MOS equations, it is possible that the skill of the MOS forecasts will deteriorate. Several changes to the AVN were made during our developmental sample, but we have not yet seen any adverse impact.

Results from an experimental Eta-based MOS system at TDL further suggest that skillful MOS precipitation forecasts are possible even when the underlying numerical model had not been frozen to design changes (Antolik 1998b). Many of the issues relevant to that study also apply here. For example, even though the AVN resolution changed during this development, we maintained a consistent archive at a somewhat coarser resolution than the effective resolution of the revised model. Therefore, the effects of changes in the statistical properties of the model output used as predictor input to the MOS system may have been mitigated.

Other strategies are possible during development of MOS equations to help adapt the technique to situations where only small and/or potentially unstable samples of model data are available. One strategy is to choose predictors which would be somewhat resistant to changes in model design. For example, care may be taken to avoid predictors in the lowest layers of the model, where possible changes to boundary layer physics might have an acute impact on the statistics of output variables. The effects of small sample size also may be mitigated by the pooling of data from larger regions. Large regions dilute the unique relationship that may exist between model predictors and weather elements at individual stations. The use of historical relative precipitation frequencies at each station may help alleviate this problem.

In addition to changes in the model output, changes in the nature of predictand data over the period of development may also affect the performance of MOS and other statistical systems. Specifically, over the last several years, data obtained by the ASOS network gradually have been replacing manual observations. A benefit of this is an increase in the number of stations that routinely report precipitation and that can be used when developing new MOS systems. However, one potential drawback is that our sample of predictand data may represent a mixture of the statistical properties of both observing systems. In addition, the NGM MOS PoP/QPF system was developed with a dependent sample consisting exclusively of manual observations. When the performance of this earlier system is verified using ASOS data, some apparent degradation in its skill may result. For these reasons, slight performance differences should be expected between the older and newer MOS systems.

A more serious problem with data obtained from the ASOS network concerns the possibility of incorrect precipitation reports due to the nature of ASOS equipment. For example, heavy dew accumulations may register as precipitation in the ASOS gauges. Conversely, snow and ice might not be measured as precipitation until they begin to melt. The result may be a report of precipitation uncorrelated to the atmospheric conditions which produced it. To avoid these problems, TDL performs extensive automated quality control before archiving precipitation reports for use in MOS development. An other problem not unique to ASOS is that stations report precipitation amounts only when precipita tion has occurred. When a station does not report an amount, it may be that no measurement was taken for one reason or another even though precipitation actually may have fallen. Extensive manual quality control also has been undertaken to help minimize the effect of this problem.

5. SUMMARY

A goal of the NWS is to extend operational precipitation forecasts into the medium range. The new AVN system discussed here represents the first time that TDL's primary statistical guidance package includes precipitation amount forecasts through day three. This enhanced AVN MOS package will replace the NGM- based MOS as the primary statistical guidance package of the NWS. Early verification results from an experimental AVN-based MOS PoP/QPF system suggest that this new guidance will perform at least as well as the current operational systems.

Developmental work on the new AVN MOS system will continue at TDL through 1999, with implementation expected in late 1999. The latest information on the development of these and the other AVN MOS elements is available at http://www.nws.noaa.gov/tdl on the World Wide Web.

6. REFERENCES

Antolik, M. S., 1995: NGM-based quantitative precipitation forecast guidance: Performance tests and practical applications. Preprints, 14th Conf. Wea. Analysis and Forecasting, Dallas, Amer. Meteor. Soc., 182-187.

_____, 1998a: NGM-based statistical quantitative precipitation forecast guidance for the contiguous United States and Alaska. NWS Technical Procedures Bulletin No. 451, National Oceanic and Atmospheric Administration, U.S. Department of Commerce, 29 pp.

_____, 1998b: Eta-based MOS quantitative precipitation forecasts in an era of rapidly changing numerical models: How well can we do? Preprints, 16th Conf. Wea. Analysis and Forecasting, Phoenix, Amer. Meteor. Soc.

Brier, G. W., 1950: Verification of forecasts expressed in terms of probability. Mon. Wea. Rev., 78, 1-3.

Glahn, H. R., and D. A. Lowry, 1972: The use of model output statistics (MOS) in objective weather forecasting. J. Appl. Meteor., 11, 1203-1211.

_____, A. H. Murphy, L. J. Wilson, and J. S. Jensenius, Jr., 1991: Lectures and papers presented at the WMO training workshop on the interpretation of NWP products in terms of local weather phenomena and their verification, Wageningen, The Netherlands, World Meteorological Organiza tion, Geneva, 340 pp.

Hoke, J. E., N.A. Phillips, G. J. DiMego, J. J. Tucillo, and J. G. Sela, 1989: The regional analysis and forecast system of the National Meteorological Center. Wea. Forecasting, 4, 323-334.

Jacks, E., J. B. Bower, M.C. Erickson, and J. C. Su, 1990: NGM-based MOS guidance for maximum/minimum temperature, probability of precipitation, cloud amount, and surface wind. NWS Technical Procedures Bulletin No. 387, National Oceanic and Atmospheric Administration, U.S. Department of Commerce, 15 pp.

Jensenius, J. S. Jr., K. K. Hughes, and J. B. Settlemaier, 1992: Calibrated perfect prog temperature and probability of precipitation forecasts for medium-range projections. 12th Conf. Prob. Statistics in the Atmospheric Sciences, Toronto, Amer. Meteor. Soc., 213-218.

_____, _____, and _____, 1995: The National Weather Service's AVN- and MRF-based statistical weather forecast systems. 14th Conf. Wea. Analysis and Forecasting, Dallas, Amer. Meteor. Soc., 163-169.

Kanamitsu, M., 1989: Description of the NMC global data assimilation and forecast system. Wea. Forecasting, 4, 335-342. National Weather Service, 1998: Further changes to the 1998 NCEP operational MRF model analysis/forecast system. NWS Technical Procedures Bulletin No. 450, National Oceanic and Atmospheric Administration, U.S. Department of Commerce.

_____, 1999: The modernized end-to-end forecast process for quantitative precipitation information: Hydrometeorological requirements, scientific issues, and service concepts. NOAA, NWS, Office of Meteorology, Silver Spring, MD 20910.

Su, J. C., 1993: NGM-based MOS guidance for the probability of precipitation (PoP). NWS Technical Procedures Bulletin No. 409, National Oceanic and Atmospheric Administration, U.S. Department of Commerce, 14 pp.