Impact of May 15, 2001 changes to the AVN/MRF model on the MOS guidance:

MDL worked with the EMC modelers in March and April to assess the impacts of the model changes on the MOS guidance. We (MDL) presented these results last week at a CAFTI meeting. If you'd like to see the presentation, check out the following web site:

Essentially, we found the following:

1. In March, we thought the temperature biases in the new MRF model (we'll call this the "parallel" from here on to distinguish it from the "operational" model {operational at that time}) were suspiciously bad. In fact, the modelers then found a mistake in the radiation package calculations over snowcover, and corrected the problem. This change invalidated our results comparing the operational and parallel versions of the model.

2. We then did some MOS comparisons on retrospective model runs made for the month of February. We found that the temperature biases for the "parallel" MOS were colder than for the operational MOS. Since the operational MOS tended to be a little warm, this cooling in the MOS guidance actually had the net effect of improving the guidance! So for the February tests, the parallel MOS temperature guidance was actually slightly more accurate than the operational guidance. The parallel dewpoint guidance was slightly more accurate than the operational guidance, but the parallel seemed to have a distinct dry bias. The differences in the PoP and cloud guidance were slight. BOTTOM LINE: For the February test, we found no systematic degradation in the MOS guidance with the parallel MRF, except in Alaska, where a cold bias in the guidance was worsened. In the CONUS, the skill of the MOS guidance tended to stay the same, or, in some cases, actually increase.

3. After the "bug" in the MRF model was fixed, we ran more tests for the period of April 14 to the present. We verified the first 14 days of that period. We continued to find that the parallel MOS guidance was cooler. For the min temperatures, the difference in skill between the parallel and operational MOS was minimal. However, the increase in the cold bias seemed to cause the parallel MOS max temperature guidance to deteriorate by as much as an average 0.5 degree F mean absolute error compared to the operational MOS max guidance. The skill of the parallel MOS PoPs also seemed to deteriorate slightly, but this was a very small, dry sample.

4. Bottom Line: our samples were small, and we're reluctant to say that the MOS guidance will be less (or more) accurate after the AVN/MRF model is changed. However, we suspect that the temperature guidance, particularly during the warmer part of the day, will tend to be slightly cooler than last warm season. The dew point guidance will be slightly drier. If our test results hold up on a larger sample, the PoPs in the warm season will tend to be slightly less accurate, but may be better in the cool season.

5. We're continuing to produce MOS packages from the parallel and operational versions of the MRF although our ability to compare the two MOS guidance packages will end Tuesday, May 15th. We're also looking at the model thermal fields to determine if we should redevelop the temperature and dewpoint guidance after eliminating certain model variables. We'll do additional experimentation as we can.