|
Search by city or zip code. Press enter or select the go button to submit request
|
|
|
|
|
Help Guide to Understanding Verification Graphics
Interval Type
(This is not
applicable to the categorical scoring method.) |
- Observed & Forecast
- This means that each gridpoint value is used twice in a score
computation, once in the interval where the observed value would put
it and once in the interval where the forecast value would put it.
- Observed
- This means that the observed value is the value used to assign the
observed-forecast pair to an interval. Because the observation
sample is identical for every QPF product, this approach also
insures that every QPF product (NGM, Eta... RFC) has the same
sub-sample withing each interval.
- Forecast
- This means that the forecast value is the value used to assign the
observed-forecast pair to an interval. Because the forecast sample
varies from one QPF product to the other (NGM, Eta... RFC), a
consequence of this approach is that the sub-sample for an interval
will vary a month the QPF products (this is especially true in the
higher intervals where sub-samples will be small).
|
| Statistic Score Type |
- Continuous
- This means that the statistics are calculated using the numeric
value of the forecast and observation at each point. Scores are
computed for each of six mutually-exclusive precipitation intervals.
Due to the graphing program's limitations you can interpret each
interval as:
| 0.00 | 0.00 - 0.009 |
| 0.01 | 0.01 - 0.099 |
| 0.10 | 0.10 - 0.249 |
| 0.25 | 0.25 - 0.499 |
| 0.50 | 0.50 - 0.999 |
| 1.00 | Greater than or equal to 1.00 |
- Categorical
- This means that the statistics are calculated from a contingency
table, where each forecast-observation pair is tabulated in the
appropriate cell. Based on the six intervals used by the continuous
method, this would result in a 6x6 contingency table. Because most
of the categorical scores are actually computed for "threshold"
intervals (wherein an event occurrence means observed or forecast
precipitation was equal to or greater than the threshold value),
entries in the table are appropriately combined to form a 2x2 table
for each threshold. Due to the graphing program's limitations you
can interpret each threshold as:
| 0.01 | Greater than or equal to 0.01 |
| 0.10 | Greater than or equal to 0.10 |
| 0.25 | Greater than or equal to 0.25 |
| 0.50 | Greater than or equal to 0.50 |
| 1.00 | Greater than or equal to 1.00 |
|
| Forecast Period |
- F006 through F024
- This means that the Fxxx forecast for each day of the month has
been used to calculate this statistic.
- Day1 through Day3
- This means that the four 6 hour forecast included in the day have
been used to calculate this statistic.
| day1 | F006, F012, F018, F024 |
| day2 | F030, F036, F042, F048 |
| day3 | F054, F060, F066, F072 |
|
| Continuous Score |
- Mean Absolute Error (MAE)
-
| This score is the mean of the absolute differences between
the observations and forecasts in the interval. The score
provides a good measure of the accuracy of a QPF product (NGM,
Eta... RFC). The closer the MAE is to zero the better the
accuracy. (See figure on right) |
|
- Root Mean Square Error (RMSE)
-
| This score is the square root of the mean of the squared
differences between the observations and forecast in the
interval. The score provides a good measure of the accuracy
of a QPF product (NGM, Eta... RFC) while giving a greater
weight to the larger differences than the MAE does. The
closer the RMSE is to zero the better the accuracy. (See
figure on right) |
|
- Mean Error (ME) (bias)
-
| This score is the mean of the arithmetic differences between
the observations and forecasts in the interval. The score is
a measure of forecast bias, where positive values denote
overforecasting, negative values denote underforecasting, and
zero indicates no bias. (See figure on right) |
|
- Volumetric Bias (CVBIAS)
-
| This score is the ratio of the sum of the forecast values to
the sum of the observed values in the interval. Thus, it is
a measure of forecast bias, where a value of one denotes no
bias, grater values denote overforecasting, and lesser values
denote underforecasting, of accumulative amounts of
precipitation. This bias is a measure of
cumulative-volumetric bias meaning that the forecast and
observed values, which individually represent water volume
over some sub-area, are accumulated independently of one
another over the scoring area. From this standpoint this
bias measure is different than the Mean Error Bias. Also,
because of the inherent independent summing of the forecast
and observed values the volumetric bias will have identical
values for the observed, forecast and observed & forecast
interval types. (See figure on right) |
|
- Interval Distribution (DIST)
Observed & Forecast Interval Type: This
shows how many cases are used to calculate the scores in each
interval for each product. (NGM, Eta... RFC)
Observed Interval Type: This shows how many observations
there were in each interval and how many forecast there were fore
each product. The "OBS" (green) line shows you how many cases are
in each interval for all products.
Forecast Interval Type: This shows how many observations
there were in each interval and how many forecast there were fore
each interval for each product. The product (NGM, Eta... RFC) lines show you how
many cases are in each interval for each respective product.
|
Categorical Scores
(Interval type does not apply to these scores) |
- Threat Score (TS)
-
| This score is a measure of the degree of coincidence (or
overlap) of the forecast and observed areas of a threshold
precipitation amount. With no overlap the TS is zero, and
with perfect coincidence the TS is one. (See figure on right)
(NOTE: An improved score results from slight overforecasting.)
|
|
- Equitable Threat Score (ETS)
-
| This score is the same as the TS but with forecast and
observed overlap area due to chance (or luck) removed. This
adjustment (in the ETS) allows for appropriate comparison of
the score between locations with different precipitation
climatology. (See figure on right) |
|
- Probability of Detection (POD)
-
| This score is the fraction of the observed area of a
threshold precipitation amount that was correctly forecast.
QPFs with a perfect POD have a value of one, and forecast with
the worst possible POD have a value of zero. (See figure on
right) |
|
- False Alarm Rate (FAR)
-
| This score is the fraction of the forecast of a threshold
precipitation amount that were incorrect. The worst is one
the best is zero. (See figure on right) |
|
- Bias
-
| This score is the ratio of the number of forecasts to the
number of observations given the threshold amount. Forecast
with perfect bias have a value of one, overforecasting results
in bias greater than one, and underforecasting results in
bias less than one. (See figure on right). |
|
- Threshold Distribution (DIST)
This shows how many observations there were in
each threshold and how many forecast there where for each threshold
for each product (NGM, Eta,..RFC). The "OBS" (green) line shows you
how many observations are in each threshold. The other lines will
show you how many forecast are in each threshold for each
respective product.
|
|
www.nws.noaa.gov 
