European Centre for Medium Range Weather Forecasts Diagnostic Analyses (ECMWFDIAG)

• ecmwfflx - ECMWF: radiative fluxes at altitude, 1-hr avg, entire coverage
  
  netCDF File Header Description (Data Object Design File)
• ecmwfsfc - ECMWF: surface variables and fluxes, entire coverage, 1-hr avg
  
  netCDF File Header Description (Data Object Design File)
• ecmwfsfc1l - ECMWF: surface variables and fluxes, single layer, 1-hr avg
  
  netCDF File Header Description (Data Object Design File)
• ecmwfsfce - ECMWF: near-surface and surface variables (extra), entire coverage, 1-hr avg
  
  netCDF File Header Description (Data Object Design File)
• ecmwfsfcml - ECMWF: model multilevel surface fields at 4 levels, entire coverage
  
  netCDF File Header Description (Data Object Design File)
• ecmwften - ECMWF: total and physical tendencies for met and cloud vars, entire coverage, 1-hr avg
  
  netCDF File Header Description (Data Object Design File)
• ecmwfvar - ECMWF: model met. and cloud variables at altitude, entire coverage, 1-hr avg
  
  netCDF File Header Description (Data Object Design File)

ecmwfflx

These files contain the radiative fluxes throughout the atmosphere. The fluxes are averages over the last hour.
They are archived on the so called flux levels of the model (often referred to as half-levels). The full levels on which the variables are stored are placed between two flux levels. Hence, the largest level number represents the surface and level 1 the top of the atmosphere. Full level k is between half level k and k+1. The information of level can be found in ecmwfvar data.

• net shortwave flux (W/m^2)
• net longwave flux (W/m^2)

In addition, new measurements were added on 20101207:
• Sensible heat flux ( (W/m^2)
• Turbulent momentum flux - u component (Pa)
• Turbulent momentum flux - v component (Pa)

ecmwfsfc (through 2000-09-12)

These files contain surface variables and fluxes. The level number is set to 1.
The measurements are:

• soil temperature in layer 1 (7 cm deep) (K) **
• shortwave radiation (W/m^2)
• longwave radiation (W/m^2)
• latent heat flux over water (W/m^2) **
• latent heat flux over ice/snow (W/m^2) **
• sensible heat flux (W/m^2)
• ground heat flux between layer 1 and layers 2/3/4 (W/m^2) **
• soil moisture in layer 1 (m^3/m^3) **
• large-scale rain (mm/s)
• convective rain (mm/s)
• large-scale snow (mm/s)
• convective snow (mm/s)
• surface pressure (Pa)
• turbulent moisture flux kg/kg * kg/(m^2*s)) ++

** measurements in nsa and sgp files; no longer contained in new data stream immediately following.
++ measurement in twp files; no longer contained in new data stream immediately following.

Note the following two data streams replace ecmwfsfc beginning 2000-09-13)

The change in datastreams and measurements are a direct result of changes in the land-surface parametrization which became operational for the forecast starting 2000-09-12. The surface, "sfc", data was split into two types of files - sfc1l and sfcml. "1l" stands for 1 layer and is available for all sites. "ml" stands for multi-level and are only available for sites which are considered land in the model (SGP and NSA and Darwin (DAR) for TWP also the mobile facilities PYE, NIM, FKB, HFE).

ecmwfsfc1l (beginning 2000-09-13)

This data stream contains the same measurements as ecmwfsfc which it replaces with the exceptions noted above. The new data stream also contains consistent measurement names across all the sites. The surface level for the TWP sites is now 1; in the older version of these files, it was set to 61. The measurements that are carried forward are:

• shortwave radiation (W/m^2)
• longwave radiation (W/m^2)
• sensible heat flux (W/m^2)
• large-scale rain (mm/s)
• convective rain (mm/s)
• large-scale snow (mm/s)
• convective snow (mm/s)
• surface pressure (Pa)

In addition, a new measurement has been added:

• surface latent heat flux (W/m^2)

ecmwfsfcml (beginning 2000-09-13)

These measurements are all multilevel at 4 levels as represented in the ECMWF land-surface parametrization. The soil model layers have the following depths starting from the top layer (layer 1 in the files):

1. 7 cm
2. 21 cm
3. 72 cm
4. 189 cm

The measurements are:

• soil temperature (K)
• heat flux at layer bottom (W/m^2)
• moisture flux at layer bottom (kg/(m^2*s))
• soil wetness (kg/m^2)

ecmwften

These files contain the total and physical tendencies for u,v,T and q on the model levels. The units are as in the ecmwfvar files but per second.
The tendencies are:

• u total (m/s/s)
• u physics (m/s/s)
• v total (m/s/s)
• v physics (m/s/s)
• T total (K/s)
• T physics (K/s)
• q total (kg/kg/s)
• q physics (kg/kg/s)

ecmwfvar

These files contain the model variables on each model level. Level 1 is the top full level of the model (10 hPa). The level with the highest number is the lowest altitude full level.
The measurements include:

• p - pressure (Pa)
• u - zonal wind component (m/s)
• v - meridional wind component (m/s)
• T - temperature (K)
• q - specific humidity (kg/kg)
• l - specific cloud liquid water content (kg/kg)
• i - specific cloud ice content (kg/kg)
• a - cloud fraction (percent/100.)
• R - relative humidity (percent/100.)
• w - omega=vertical velocity in pressure coordinates (Pa/s)

ecmwfsfce (beginning January 1998)

These files contain extra near surface and surface variables including:

• 2m Temperature (K)
• 2m specific humidity (kg/kg)
• 10m wind u-component (m/s)
• 10m wind v-component (m/s)
• skin temperature (K)
• albedo
• downward surface solar radiation (W/m^2)
• downward surface longwave radiation (W/m^2)

In addition, a new measurement was added on 20101207:
• Surface evaporation (Kg/m^2)

The number of distinct locations that ECMWF can configure in their diagnostic analysis product is limited. Therefore, and since the ECMWF Physical Parametrization group is producing these products as a courtesy to ARM, a prior location often needs to be terminated before a new one can be brought online. The data are therefore not necessarily continuous for all ARM locations. Here's a summary as of 2009-07-29:

Site	Begin Date/End Date
NSA	1997-09
SGP IOPs	1995-04-17/05-07, 1995-07-17/08-14, 1995-09-22/11-01, 1996-04-01/04-30, 1996-05-01/05-10, 1996-07-15/08-05
SGP	1996-08
TWP Darwin	2002-03-20
TWP Manus	1997-06
TWP Nauru	1999-01
NIM	2005-05-05/2007-03-27
PYE	2005-04-20/2005-10-21
FKB	2007-04-01/2008-02-29
HFE	2008-05-01/2009-04-30
GRW	2009-05-01- present
PGH	2011-10-01-present
GAN	2011-10-01-present
SMT	2011-11-15-present
PVC	2012-07-01-present

Note: sgpecmwfsfcX1.c1 (netcdf files) begin 1996-08-01; prior to that only ascii files are available for the surface data.

These diagnostic files are available at the ARM Archive with an approximately one month delay.

The single column data in the diagnostic files are either individual ECMWF model grid points or averages over several grid points in a rectangular domain defined by their upper left (NW) and lower right (SE) corners. The coordinates of the single grid points in the file are the closest model grid point coordinates to the desired location. When the resolution of the model changes (as it did on 2000-11-22) that can result in a change in those coordinates, see Data User Notes below.

SGP

There are three SGP domains over which all the quantities are averaged.
They are numbered 27,28,29. The areas for those domains are:

point number	Longitude upper left	Latitude upper left	Longitude lower right	Latitude lower right	comment
27:	260.3	38.8	264.6	34.8
28:	261.7	34.8	263.	34.2	(eliminated 2002-03-20)
29:	261.7	37.	263.	36.

NSA

The NSA diagnostic data are available for two single points (numbered 13 and 19) located at:

point number	Longitude	Latitude	Comment
13:	202.00	70.41	(beginning 2002-03-20)
19:	203.20	71.05	(before 2000-11-22)
19:	204.00	71.12	(beginning 2000-11-22)

TWP DAR

There are three single points representing TWP Darwin. These data begin on 2002-03-20. Other domains in SGP, TWP Manus and TWP Nauru were eliminated to make room for these new points and for the new NSA point.
The Darwin points are number 6, 28 and 50. The points are:

point number	Longitude	Latitude
6:	130.43	-12.82
28:	129.73	-12.47
50:	131.13	-13.52

TWP MAN

There are three TWP Manus domains over which all the quantities are averaged.
They are numbered 48,49,50. The areas for those domains are:

point number	Longitude	Latitude	Comment
48:	147.30	-1.93	(before 2000-11-22)
48:	147.37	-1.97	(beginning 2000-11-22)

point number	Longitude upper left	Latitude upper left	Longitude lower right	Latitude lower right	comment
49:	146.60	-1.10	148.10	-2.80
50:	146.00	-0.60	148.80	-3.30	(eliminated 2002-03-20)

TWP NAU

There are three domains for the TWP Nauru diagnostic data. They are numbered 4, 5, 6.
The areas for those domains are:

point number	Longitude	Latitude	Comment
4:	167.06	-0.28	(before 2000-11-22)
4:	167.99	-0.53	(beginning 2000-11-22)

point number	Longitude upper left	Latitude upper left	Longitude lower right	Latitude lower right	comment
5:	166.10	0.30	167.70	-1.30
6:	165.50	0.90	168.30	-1.90	(eliminated 2002-03-20)

The Manus and Nauru areas in the ECMWF model are all model sea points.

AMF PYE 2005

The Point Reyes diagnostic domain consists of two single points:

point number	Longitude	Latitude
18	237.08	38.11
23	236.67	38.11

AMF NIM 2006

The Niamey diagnostic data domain consists of a single point

point number	Longitude	Latitude
76	2.46	13.52

AMF FKB 2007

The Black Forest diagnostic data domain consists of 3 COPS super-site locations:

point number	Longitude	Latitude	Nearest Supersite
115	8.32	48.46	Supersite M, ARM AMF Murgtal, Heselbach: 8.405°E, 48.545°N, ca. 500 m
116	8.67	48.68	Supersite S, Gliders Airport Deckenpfronn, 8.813°E, 48.635°N, ca. 600 m
117	7.67	48.68	Location POLDIRAD, Waltenheim sur Zorn 7.610°E, 48.739°N, ca. 120 m

AMF HFE 2008

The China diagnostic data domain consists of 4 COPS super-site locations:

point number	Longitude	Latitude	Nearest Supersite
14	120.25	31.37	ARM AMF Taihu: 120.22°E, 31.42°N, ca. 7 m
15	117.00	39.69	ARM AMF Xianghe: 116.96°E, 39.75°N, ca. 36 m
115	116.75	32.49	ARM AMF Shouxian: 116.78°E, 32.56°N, ca. 23 m
116	100.69	39.01	ARM AMF Zhanghe, 100.58°E, 38.93°N, ca. 1483 m

AMF GRW 2009

The Graciosa Island, Azores diagnostic data domain consists of a single point:

point number	Longitude	Latitude	Nearest Supersite
14	331.88	39.01	Graciosa Island, Azores: 39° 5' 28" N, 28° 1' 45" W, ca. 15.24 m

AMF PGH 2011

The Ganges Valley, India diagnostic data domain consists of two points:

point number	Longitude	Latitude	Nearest Supersite
119	79.53	29.31	Ganges Valley, India: 29° 21' 33.84" N, 79° 27' 29.27" E, ca 1943m
120	80.85	26.78	Lucknow, India: 26° 50′ 24″ N, 80° 57′ 36″ E

AMF GAN 2011

The Gan Island, Maldives diagnostic data domain consists of one point:

point number	Longitude	Latitude	Nearest Supersite
121	73.12	-0.63	Gan Island, Maldives: 73.14990° East, 0.69060° South, with ca. 1m.

AMF SMT 2011

Summit Station, Greenland diagnostic data domain consists of one point:

point number	Longitude	Latitude	Nearest Supersite
116	321.75	-72.60	Summit Station, Greenland:72° 34′ 46.5″ N, 38° 27′ 33.07″ W with ca. 3216m

AMF PVC 2012

Highlands, Cape Cod, MA diagnostic data domain consists of eleven points:

point number	Longitude	Latitude	Nearest Supersite
124	290.00	41.96	Highlands, Cape Cod, MA:42.03048° N, 70.04932° W with ca. 47.9m
125	289.81	41.68	Highlands, Cape Cod, MA:42.03048° N, 70.04932° W with ca. 47.9m
126	292.31	43.51	Highlands, Cape Cod, MA:42.03048° N, 70.04932° W with ca. 47.9m
127	292.50	43.08	Highlands, Cape Cod, MA:42.03048° N, 70.04932° W with ca. 47.9m
128	292.88	42.66	Highlands, Cape Cod, MA:42.03048° N, 70.04932° W with ca. 47.9m
129	292.59	41.96	Highlands, Cape Cod, MA:42.03048° N, 70.04932° W with ca. 47.9m
130	292.68	41.26	Highlands, Cape Cod, MA:42.03048° N, 70.04932° W with ca. 47.9m
131	292.14	40.55	Highlands, Cape Cod, MA:42.03048° N, 70.04932° W with ca. 47.9m
132	291.78	40.13	Highlands, Cape Cod, MA:42.03048° N, 70.04932° W with ca. 47.9m
133	290.16	39.85	Highlands, Cape Cod, MA:42.03048° N, 70.04932° W with ca. 47.9m
134	287.28	40.70	Highlands, Cape Cod, MA:42.03048° N, 70.04932° W with ca. 47.9m

The diagnostic analysis files are received in ASCII format (1 file per point per type [var,ten,sfc1l,sfcml,sfce]) from the ECMWF and archived in daily tar bundles in the following .00 level data-streams:
pvcecmwfX1.00
pghecmwfX1.00
ganecmwfX1.00
smtecmwfX1.00
fkbecmwfX1.00
grwecmwfX1.00
hfeecmwfX1.00
nimecmwfX1.00
nsaecmwfX1.00
pyeecmwfX1.00
sgpecmwfX1.00
twpecmwfX10.00
twpecmwfX20.00
twpecmwfX30.00

Daily File Notification Form
Monthly File Notification Form

ECMWF Web Site
ECMWF Model Revision History
ECMWF Operational upgrades

• ARM Mentor
Shaocheng Xie
• ARM Software Developer
Laurie Gregory

• Institution
Physical Parametrization Group at ECMWF

• Reprocessing Date

Manus(X10): 199706-201206
Nauru(X20): 199901-201206
Darwin(X30): 200203-201206


• Reason Reprocessing

All twpecmwf data were reprocessed with the new facility ID (X10 -man, X20- 
nau and X30-dar). Below is the comparative chart of the change.
 

           OLD                                  NEW 

twpecmwfX1.00 -----> twpecmwfX10.00
twpecmwfmanflxX1.c1 ----> twpecmwfflxX10.c1
twpecmwfmansfcX1.c1 ----> twpecmwfsfcX10.c1
twpecmwfmansfc1lX1.c1 ----> twpecmwfsfc1lX10.c1
twpecmwfmansfceX1.c1 ----> twpecmwfsfceX10.c1
twpecmwfmantenX1.c1 ----> twpecmwftenX10.c1
twpecmwfmanvarX1.c1 ----> twpecmwfvarX10.c1


twpecmwfnauX1.00 -----> twpecmwfX20.00
twpecmwfnauflxX1.c1 ----> twpecmwfflxX20.c1
twpecmwfnausfcX1.c1 ----> twpecmwfsfcX20.c1
twpecmwfnausfc1lX1.c1 ----> twpecmwfsfc1lX20.c1
twpecmwfnausfceX1.c1 ----> twpecmwfsfceX20.c1
twpecmwfnautenX1.c1 ----> twpecmwftenX20.c1
twpecmwfnauvarX1.c1 ----> twpecmwfvarX20.c1


twpecmwfdarX1.00 -----> twpecmwfX30.00
twpecmwfdarflxX1.c1 ----> twpecmwfflxX30.c1
twpecmwfdarsfc1lX1.c1 ----> twpecmwfsfc1lX30.c1
twpecmwfdarsfcmlX1.c1 ----> twpecmwfsfcmlX30.c1
twpecmwfdarsfceX1.c1 ----> twpecmwfsfceX30.c1
twpecmwfdartenX1.c1 ----> twpecmwftenX30.c1
twpecmwfdarvarX1.c1 ----> twpecmwfvarX30.c1

Q: The old twpecmwf[xxx]sfcX1.c1 (pre 2000-09-13) files had "turbulent moisture flux". Does this match to anything in the new files (or to anything in SGP/NSA)?

A: (from Christian Jacob, ECMWF): "Yes and no. Due to the backing out procedure we had to apply for the TWP sites, we did not have the surface latent heat flux at those sites and replaced it by the moisture flux (which can be converted into the former). Now that we cleaned up the mess, we do have surface latent heat flux instead, which is cleaner. Again, it comes in different units and is therefore not a direct replacement, although it contains the same information."

Q: Often soil wetness in a generic sense is expressed in terms of (volumetric) soil moisture content, which is unitless or has units of m^3/m^3. For example, a corresponding old term was "soil moisture in layer 1 (m^3/m^3) (soil_moisture in sgp and nsa)." The new "soil wetness" seems to represent the amount of water integrated with depth for each layer. Can someone confirm that this interpretation is correct?

A: The interpretation of soil wetness in the question is correct.

Q: Is the model elevation at Barrow 8 m?

A: No, it is 5 m.

Q: Is there any information on model ozone available?

A: No, there is no information on ozone in any of these products.

Q: Are the variables in the diagnostic files to be interpreted as hourly averages for the hour prior to the time-stamp?

A: From Martin Köhler at ECMWF: Yes and No! The answer depends on the type of variable:
for fluxes and tendencies: average over previous hour before time stamp (e.g. latent heat flux)
other variables: instantaneous value at time stamp (e.g. T2m or cloud cover)

Acknowledgment:

Publications should refer to: "The ECMWF operational analysis and forecasting system: The full evolution http://www.ecmwf.int/products/data/operational_system/index.html"

Why single data points moved in diagnostic ECMWF files

To generate point data in the diagnostic files, ECMWF specifies to the model the lat-lon of the observation point, and get as an output the nearest model point. When the model grid changes (such as on 2000-11-22, where the model resolution increased from ~60 km to ~40 km), the location of the nearest model point changed.

Discussion from Christian Jacob on changes to diagnostic files on 2000-09-13.

In June 2000 we introduced a new land surface parametrization scheme into our model. We immediately realised that the diagnostic tool that we are using (amongst many other applications) to get out the data for the ARM sites was totally inadequate for the new scheme. This, together with some things we did not like even for the old parametrization scheme, led us to invest some time to completely rewrite the surface part of this diagnostic tool. The changes that you see are repercussions of this rewrite. The rewritten diagnostics became operational on 2000-09-12, hence affecting ARM data from 2000-09-13 onwards.

To give you an example of the sort of changes: The old diagnostic tool did not provide any direct information at the surface over sea points in the model. It so happens that Manus and Nauru are way too small islands to be land points even in our highest of model resolutions. Therefore, we had to play all sorts of tricks to back out things like surface fluxes from other parts of the archive. That is the reason why you had different files for the TWP sites in the past. Obviously, since we rewrote the software anyway, we wanted to get rid of that problem, and we did. Hence, now all the files are exactly the same. The only difference is that you have extra files at the locations that are really land in the model (SGP and NSA), that deal with real land parameters, such as soil temperature and soil moisture, which obviously don't exist at sea points.

The difference in measurement names between old and new are also due to the cleanup.

ECMWF	European Centre for Medium Range Forecasts
GRIB	GRIdded Binary

"The ECMWF operational analysis and forecasting system: The full evolution http://www.ecmwf.int/products/data/operational_system/index.html"