US National Oceanic and Atmospheric Administration
Climate Test Bed Joint Seminar Series
ESSIC/UMD, College Park, Maryland, 17 November 2008                                                                                        [Print Version]

Regional and Seasonal Improvements

in the Skill and Value of CPC 3-Month Outlooks

Edward A. O’Lenic, David A. Unger, Kenneth S. Pelman, and Mike Halpert

Climate Prediction Center, NOAA/NWS/NCEP

O’Lenic et al (2008) reported on the use of an objective technique which, on dependent data from 1995 through 2004, improved the average Heidke skill score of CPC’s operational ½-month-lead 3-Month temperature and precipitation outlooks from 22 to 24, for temperature, and from 8.8 to 12.1, for precipitation.

This paper extends that work by calculating the increase in the percentage of the time non-EC (equal chances) probabilities are predicted, over the 1995-2004 dependent data period. The fraction of the map covered by non-EC probabilities is one metric of usefulness which users are quite sensitive to. Reducing the area covered by EC (33.33…% for each of the three tercile categories) is highly desirable.

Each map in the top row of Fig. 1 shows the average Heidke skill score (HSS, lines, see Appendix) over 1995-2004 for 30 official (OFF) forecasts of ½-month lead precipitation, made in real-time.

The second row from the top shows the same information as the top row, except for forecasts made using an objective technique (Unger et al, 2009) to numerically combine the identical set of forecast tools which were available to the forecasters in real-time during 1995-2004. This is referred to as the consolidation (CON). These tools include the 2-tier NCEP coupled model described in Ji and Leetmaa (1995), the optimum climate normals (OCN, or trend) (Court, 1967-68), the canonical correlation analysis (CCA) (Barnston, 1997), and the screening multiple linear regression (SMLR). The latter two tools are multi-variate statistical techniques.

The lines and colors in the set of maps on the third row show the arithmetic difference between the CON and OFF maps in rows 1 and 2. A number near the bottom right side of each map shows the map average of this difference. The CON technique averages a higher percentage of non-EC forecasts than did the OFF by 8%, in spring, 18%, in summer, 16%, in fall, and 20%, in winter. Since the trend is relatively small for 3-month precipitation, these improvements are smaller than those for temperature, but still large enough to be noticeable to users.

The results for temperature are shown in Fig. 2.

The consolidation technique produces large increases in non-EC forecast percentages for temperature (Fig. 2): 11%, for spring, 31%, for summer, 40%, for fall, and 55%, for winter. This large increase is due, in part, to the fact that the trend is more strongly reflected in temperature than it is in precipitation (Fig. 1).

Since these results are on dependent data, we show, in Fig. 3, the result of using the CON technique in operational ½-month lead 3-Month temperature outlooks since 2006. There is a clear break in the time series of 48-month running mean HSS which commences when the CON was implemented into CPC operations in early 2006. This performance is evidence that the CON technique is a reasonable way to present forecasters with an accurate first-guess from which to begin forecasting.

Figure 1

Figure 2

Figure 3

Appendix

The Heidke skill score (HSS) is a categorical score which compares the accuracy of a forecast of interest (e.g., 3-month outlooks) with that of a reference forecast, such as climatology (random) forecasts:

HSS= (c-e)/(t-e)*100%

where c = # gridpoints forecast correctly
            e = # gridpoints expected correct randomly
             t = # gridpoints in total

In a 3-class, tercile system, -50 ≤ HSS ≤100

References

Barnston, A. G., 1994:  Linear statistical short-term climate predictive skill in the Northern Hemisphere, J. Climate, 7, 1513-1564.

Court, A., (1967-68):  Climate normals as predictors:  Parts I-IV.  Science Reports, Air Force Cambridge Research Laboratory, Bedford MA, Contract AF19(628)-5176.

Ji, M. A., A. Kumar, and A. Leetmaa, 1994:  A multi-season climate forecast system at the National Meteorological Center.  Bull. Amer. Meteor. Soc., 75, 569-577.

O’Lenic, E. A., D. A. Unger, M. S. Halpert, and K. S. Pelman, 2008:  Developments in operational long-range prediction at CPC.  J. Weather and Forecasting, 23, 496-515.

O’Lenic, E.A., D.A. Unger, M.S. Halpert, and K. S. Pelman, 2008:  Corrigendum.  J. Weather and Forecasting, 23, 1044.

Unger, D., H. van den Dool, E. OL'enic, and D. Collins, 2009:  Ensemble regression.  Mon. Wea. Rev., in press.

Contact  Edward A. O’Lenic