MMAB page | Main wave model page
NWW3, AKW or WNA
validation page
|
This page presents error statistics of the global NWW3 wave model and the wind fields driving this model. These data are presented to help understand peculiarities of the models. To present the most complete picture of the model behavior, the data presented here also covers the experimental or parallel version of this model (formerly know as the NOAA Experimental Wave model NEW). Orriginally, an attempt was made to remove systematic biases in the wind fields by means of statistical correction. Presently such a correction is no longer deemed necessary. The models are compared to buoy data and to ERS2 fast-delivery (FD) altimeter and scatterometer data. For the period of February through November 1997 only hindcast results are available. Starting January 1998, forecast errors are also analyzed. Note that in March 1999, hardware problems caused a loss of continuity in the model. No buoy data are available for the second half of this month, and no altimeter collocations are available for the entire month. Similarly, no validation data are available for September 27, 1999 through October 26, 1999.
The buoy data are considered to be ground truth. For all buoys graphical model output is presented on the data interface pages linked above. The buoys used here are generally identified by their WMO ID. For buoy instrumentation reference is made to the agencies responsible for their maintenance. For hindcast validation, continuous hourly time series are used. For the forecast validation, only model results at the valid times are used.
Satellite data, in particular FD products, are known to incorporate systematic errors. Such errors are assessed and corrected statistically as is described on the ERS2 page. Scatterometer data are used to validate the driving wind fields where possible (analyzed winds only). The altimeter data are used to validate the both the forecast and hindcast of the wave model. To assure the use of all satellite data and hence good global coverage, the forecast validation using altimeter data considers 12h windows around valid times, collocating satellite and model data with tri-linear interpolation from hourly wave fields. For the hindcast, this implies that the continuous hindcast time series is used. For the forecasts, forecast intervals of 18-30h, 42-54h and 66-72h are used to assess the error of the 24, 48 and 72h forecasts, respectively. Satellite data are collocated with the model results by means of tri-linear interpolation using hourly fields of model results. Global error assessments are obtained by subsequent collocation with the wave model grid. In the global plots presented below some latitudinal smoothing is applied to remove signatures of individual tracks.
Additional validation results can be found in:
- Tolman (1998d, see references) for the
period December 1994 through February 1995. This study includes a comparison
with the WAM model with identical wind fields and resolutions
- The NEW-WAM comparison page, on which the
previous parallel vesrion of NWW3 is compared to the operational WAM model at
NCEP (January - June 1998)
|
Top of page | Main wave model page
|
Wind Validation And Correction |
In this section of the NWW3 validation page the driving wind fields of the
wave model are analyzed in several ways. First the winds are compared to the
conventional deep-ocean buoy observations. Second,
the original and corrected hindcast (analyzed) wind fields are compared to the
ERS2 scatterometer data. Third, the corrected
hindcast winds are compared to the altimeter wind
data, and forecast winds are compared to the analyzed winds to establish
trends. Finally, corrections of the wind
fields are discussed.
|
Buoy Data |
Buoy data in principle are available every hour. Only a fraction of these data are used in the GDAS, and in the GDAS, these data presents only a small fraction of all data used. The buoy data can therefore be considered as independent data for GDAS. The abundance of buoy data allows for an analysis per region. Considered are Japan (21004 and 22001), Hawaii (51000 series), Gulf of Mexico (42000 series), NE Pacific (46000 series), NW Atlantic (41000 and 44000 series) and NE Atlantic (62000 through 64000 series). For the GDAS winds, it makes sense to address wind speed error as a function of wind speed. This is done by performing an error-corrected bin-averaged (BA) analysis (Tolman 1998a) as in a previous wind analysis for the experimental version of NWW3 (Tolman 1998b). For forecast winds such an analysis is irrelevant, as the unavoidable random errors in a forecast will result in an apparent decrease of the slope of the regression lines of the modeled on the observed wind speeds. This would erroneously suggest systematic biases (see, e.g., Tolman 1998a). Alternatively, distributions, which are less sensitive to random errors, can be compared. Furthermore, as for forecasts only observations at the valid times are considered, the amount of data is more than an order of magnitude less than for the hindcast, making statistically stable BA results virtually impossible to obtain.
Considering the above, the following graphs are presented for the validation of NWW3 wind speeds with buoy data:
- Bias and random error estimates as a function of
the wind speeds as well as calculated and observed wind
speed distributions from the GDAS for
1997/03 - 1997/05
, 1997/06 - 1997/08
, 1997/09 - 1997/11
, 1997/12 - 1998/02
, 1998/03 - 1998/05
, 1998/06 - 1998/08
, 1998/09 - 1998/11
, 1998/12 - 1999/02
, 1999/03 - 1999/05
, 1999/06 - 1999/08
, 1999/09 - 1999/11
, 1999/12 - 2000/02
, 2000/03 - 2000/05
, 2000/06 - 2000/08
, 2000/09 - 2000/11
, 2000/12 - 2001/02
, 2001/03 - 2001/05
, 2001/06 - 2001/08
, 2001/09 - 2001/11
, 2001/12 - 2002/02
, 2002/03 - 2002/05
, 2002/06 - 2002/08
or 2002/09 - 2002/11.
- Monthly bias, rms and scatter index (rms error normalized with mean observed wind speed) for the wind speeds used in the NWW3 hindcast, 24h forecast, 48h forecast ,72h forecast , 96h forecast and 120h forecast
Remarks on the above Figures:
- The observed wind speed distributions for in particular the NE Atlantic include noise due to the fact the data is archived at a resolution close to that at which the pdf is estimated
- The winds driving the wave model at the Japanese buoy locations are of rather poor quality both in terms of rms error and in terms of scatter indices, particularly in the northern hemisphere summer. This appears to be related at least in part to the representation of Typhoons. In the northern hemisphere winter, the corresponding wind field errors more closely follow the composite data set
- For the buoys around Hawaii, the GDAS wind fields show a systematic low bias of approximately 0.5 m/s, and small rms errors and scatter indices. The negative bias appears to disappear in the 48h forecast, and the bias becomes slightly positive in the 72h forecast
- In the Gulf of Mexico , biases and rms wind speed errors are generally small. Due to the low mean wind speeds, in particular in the summer, scatter indices nevertheless may become rather large
- In the NE Atlantic and Pacific Oceans, small biases, rms error and scatter indices are generally found. Whereas the rms errors show a moderate seasonal cycle, scatter indices stay fairly constant throughout the year
- In contrast, rms error for the NW Atlantic Ocean are somewhat larger and fairly constant throughout the season, resulting in a clear seasonal cycle in the scatter indices, with the largest values in the northern hemisphere summer. In this area, wind speeds appear biased high by up to 0.5 m/s
- In particular in the NW Atlantic Ocean in the summer, extremely stable atmospheric conditions occur in particular around buoys 44004, 44008, and 44011. In such conditions the surface winds become separated from the higher atmosphere, and the (already low) winds are systematically overestimated by the model. This is particularly obvious in corresponding wind distribution functions above
- For the period of the middle of June 1998 through the beginning of October, 1998 the winds were generated by the T70 resolution GDAS and AVN. This model had serious problems with generating tropical systems, which translates into larger rms error and scatter indices for this period. With the subsequent reduction in resolution errors have been reduced again
|
Satellite Data |
| The satellite data available to validate the wind fields used in NWW3 are the ERS2 FD altimeter and scatterometer data, with their appropriate error correction. The scatterometer winds are potentially the most valuable for the validation, as they are least contaminated by the background wave field (Tolman 1998b and ERS2 page). These data have been used in the GDAS since November 5 1998, and therefore cannot be used after this date for an independent validation. Due to the large amount of data considered, it was not feasible to compare the scatterometer winds to the forecasts wind fields. The table below gives access to global average wind speeds from the scatterometer, biases, rms error and scatter indices (rms error normalized with mean observed value) from the GDAS and corrected GDAS wind fields. Note that it does not make sense to estimate the error as a function of the wind speed, due to the saturation behavior of the scatterometer- (and altimeter)-derived wind speeds at high wind speeds. |
Comments on Figures in the above Table regarding peculiarities of the altimeter winds and comparison to scatterometer winds (1997/02-1997/11):
- In the areas where the buoys are located (North Atlantic and Northeast Pacific) biases of the analyzed wind fields relative to the altimeter data are similarly small as the biases compared to the scatterometer
- In the tropics, the analyzed winds appear to show a strong negative bias when compared to the altimeter data. This bias is systematically larger than the corresponding bias as estimated from the scatterometer data. In this area the wave field is severely swell dominated. This tentatively results in a positive bias for the altimeter winds (swell energy is mistaken for small scale roughness, i.e., wind), and hence a spurious negative bias of the analyzed wind fields
- A similar but smaller systematic difference in bias patterns as obtained from both instruments is found in the southern oceans. This can tentatively be explained by the fact that the vastness of the southern oceans is more conducive to the presence of swell, and hence to a spurious positive bias of altimeter winds that have been tuned using northern ocean data
Comments on Figures in the above Table regarding content:
- Starting with the 1997/09-1997/11 period, the negative biases in the tropics increase significantly. This might well be related to the ensuing extreme El Nino event. In the 1997/12-1998/02 period, an anomalous bias pattern also appears to occur in the Northern Pacific. Note particularly the positive biases north of Hawaii, where during the previous nine month the
altimeter suggest a persistent negative bias. This suggests that in 1997/12-1998/02 a serious positive wind bias exists for the high wind speed area north of Hawaii. Note also that there are virtually no buoy observations in this area. For 1998/03-1998/05 the bias pattern north of Hawaii has disappeared again, and the biases in the tropical Pacific are diminishing
- During the above mentioned El Nino event, the forecast wind fields show a strong positive bias compared to the analysis, particularly in the tropics. In the following three months (1998/03-1998/05), forecast wind fields are much less biased when compared to the analysis, and large positive biases mainly occur in the eastern tropical Pacific
- Starting with the 1998/06-1998/08 period a higher resolution wind model was used. This leads to better defined bias patterns in 1998/06-1998/08 when compared to previous periods, e.g., 1998/03-1998/05, particularly in the tropics. Unfortunately, the new wind model had serious problems with tropical systems (hurricanes and typhoons), as is also obvious from the anomalously large biases in the tropical Pacific off Mexico and the Philippines. Such biases are not found in the Atlantic as the first hurricane actively here did not occur until late August
|
Bias Corrections |
| For the initial test of the new wave model for the period of 1994/12-1995/02 it was found that the wind fields could be improved by applying a statistical bias correction, consisting of a blended coastal and deep ocean bias correction. the former was based on buoy data, the latter on ERS-1 scatterometer data (Tolman 1998b). Since then, the GDAS and AVN models have undergone significant changes, particularly in late 1995 (Caplan et al. 1997). For the wind fields used here since early 1997 the bias corrections therefore have been re-derived, using the updated models and appropriate data sources. Starting in early 1997, the following coastal and deep-ocean corrections are used, blended as described in Tolman (1998b). |
Ucor,coastal = -0.3 m/s + U
Ucor,deep ocean = -1. m/s + 1.05U
|
In November 1998, the amount of data ingested by the GDAS again has increased dramatically, now including amongst others the ERS-2 scatterometer winds. The impact of this on the GDAS winds seemed to be small in an average sense, for which reason the above bias correction have been left unchanged.
With the increased resolution of the GDAS and AVN winds to T170, and corresponding models changes, the bias correction has (initially) been removed completely as of June 15, 1998. With the subsequent reduction of the resolution, the bias corection was not re-introduced.
|
Top of page | Main wave model page
|
Wave Model Validation |
Buoy Data |
The wave data from the buoys is separated into geographical regions and analyzed in the same way as the wind data. For the hindcast, a BA analysis is performed to estimate biases and random errors as a function of the wave height, and the distribution of observed and modeled wave heights are produced. For forecasts, only bulk statistics are presented. The following links give access to plots of validation results.
- Bias and random error estimates as a function of
the wave height as well as calculated and observed wave
height distributions for
1997/03 - 1997/05
, 1997/06 - 1997/08
, 1997/09 - 1997/11
, 1997/12 - 1998/02
, 1998/03 - 1998/05
, 1998/06 - 1998/08
, 1998/09 - 1998/11
, 1998/12 - 1999/02
, 1999/03 - 1999/05
, 1999/06 - 1999/08
, 1999/09 - 1999/11
, 1999/12 - 2000/02
, 2000/03 - 2000/05
, 2000/06 - 2000/08
, 2000/09 - 2000/11
, 2000/12 - 2001/02
, 2001/03 - 2001/05
, 2001/06 - 2001/08
, 2001/09 - 2001/11
, 2001/12 - 2002/02
, 2002/03 - 2002/05
, 2002/06 - 2002/08
, 2002/09 - 2002/11
, 2002/12 - 2003/02
, 2003/03 - 2003/05
, 2003/06 - 2003/08
, 2003/09 - 2003/11
, 2003/12 - 2004/02
or 2004/03 - 2004/05.
Note that before 2000/03 this model is identified
as NEW (NOAA Experimental Wavemodel) in these plot.
- Monthly bias, rms and scatter index (rms error normalized with mean
observed wave height) for the hindcast, 24h forecast, 48h forecast, 72h forecast, 96h forecast and 120h forecast
Comments on above figures :
- Observed wave height distributions for in particular the NW
Atlantic Ocean show a large impact of the fact that the data is
binned with bin sizes close to the resolution of the archived wave heights
(see Tolman 1998b,d)
- In general the wave model performs poorly when compared to the Japanese
buoys , in particular in terms of scatter indices. There are several
potential explanations for this behavior: (i) The corresponding wind fields
include large errors (see discussion of winds above). (ii) Low mean wave
height in this area making the scatter index sensitive. (iii) A significant
part of the error appears to be due to systematic positive biases, which might
be related to the unresolved Ryukyu Islands (see discussion of the validation
with satellite data below). (iv).The diffiulty of wind models in dealing with Typhoons
- The wave model shows excellent behavior when compared to the buoys around Hawaii ,
with small rms errors and scatter indices around 15% for the hindcast. In the
northern hemisphere winter, however, a large positive bias occurs, and the
scatter index grows to about 25%. This is at least partially due to the fact
that all buoys except for 51001 then are to some extend sheltered by the
islands from swell propagating form the north. Because the islands are not
resolved by the wave model, swell dissipation at the shores of the islands is
not modelled, and a positive bias will therefore occur south of the islands.
This can be observed in the validation with altimeter data below. In contrast,
the wave model shows much better result for buoy 51001, which is not sheltered
behind the islands. For instance, scatter indices for buoy 51001 for the
hindcast from January and February 1998 are 16%, and 20%, respectively. Wave
conditions in this area are mostly dominated by swell fields, which are less
sensitive to forecast errors than wind seas. Error growth with forecast time
therefore is smaller than forthecompositebuoydataset
- For the Gulf of Mexico , the wave model
shows small biases and rms errors. Due to the generally low wave heights,
scatter indices are high in particular in the northern hemisphere summer. Note
that the scatter index of almost 40% for the hindcast for July 1997
corresponds to an rms error that is as small as 0.2m. Because the Gulf of
Mexico is an enclosed basin, wave conditions are dominated by wind seas.
Because wind seas are more prone to forecast errors, error growth with
forecast time in this area is much larger than in the deep ocean
- In the NE Pacific Ocean (Alaska) the
wave model generally behaves well with moderate biases and rms errors and with
moderate hindcast scatter indices of around 18% throughout the season. Error
characteristics (in particular the scatter index), are slightly better than
average throughout the forecast period
- In the NW Atlantic Ocean the wave
model shows systematic negative biases, both in terms of bulk statistics and
as a function of the wave height. This might be related to the fact that the
local wave fields often are dominated by wind seas generated by offshore
winds. In the GDAS and AVN, winds close to the coast might be expected to be
underestimated due to the mixing of 'land' and 'sea' winds in this area. This
might explain the systematic low bias. If this explanation holds true, the
bias should be reduced with the introduction of higher resolution wind models.
Another possible explanation might be found in the presence of the Gulf
Stream. The rms errors in this region follow the composite dataset closely.
Scatter indices are higher in the northern hemisphere summer due to the low
average wave heights associated with this season
- In the NE Atlantic Ocean the wave
model tend to show positive biases for unknown reasons. A possible explanation
is that at least one of the buoys (63111) is sheltered by the non-resolved
Shetland Islands, but considering that this is only one of the eight buoys in
this group this explanation appears unlikely
- For the 1997/12-1998/02 period the
composite buoy data set shows a positive bias both in terms of bulk statistics
and as a function of the wave height. This behavior might be related to
anomalous model behavior in the NE Pacific Ocean as is discussed with the
satellite data below
- For the period of mid June 1998 through early October 1998 serious errors
occured in the prediction and analysis of tropical systems as described above.
This translates in elevated error levels for the wave model, in particular at
the buoys exposed to tropical wave systems (Japan, Gulf of Mexico, NW Atlantic)
|
Satellite Data |
| The wave model is validated with averaged and quality controlled ERS2 FD altimeter wave height observations. These data are collocated with the model results by means of multi-dimensional interpolation. The resulting model and altimeter data are then collocated with the wave model grid to obtain a global impression of model behavior. To avoid noise related to track signatures in the figures in the tables below, the results have been averaged in longitudinal direction with a filter width of approximately 5o. The tables below access figures with mean observed wave heights
(Hs), biases of the model relative to the
altimeter, rms errors of the model against the altimeter (all in meters), and
the scatter index (S.I.), defined as the rms error normalized with the mean
observed wave height. The hindcast represents continuous time series of data,
obtained by tri-linear interpolation from hourly fields. For the forecast,
time windows of 12 hours are used to assure that all altimeter data can be
used in the comparison. |
