ORIGIN OF THE DATA The 3-hourly archive data come from NCEP's GDAS. The GDAS is run 4 times a day, ie, at 00, 06, 12, and 18 UTC. Model output is for the analysis time and 3, 6, and 9-hour forecasts. NCEP post-processing of the GDAS converts the data from spectral coefficient form to 1 degree latitude-longitude (360 by 181) grids and from sigma levels to mandatory pressure levels. Model output is in GRIB format. See the web link above for current information on the GDAS. ARL saves the successive analyses and 3-hour forecast, four times each day to produce a continuous data archive. Some fields such as precipitation and surface fluxes are not available at the analysis time, therefore these are taken from the 6-hour forecast files.
ARL PROCESSING The ARL archiving program produces a 3 hourly, global, 1 degree latitude longitude dataset on pressure surfaces. The data are put into weekly files and made available online at the ARL server for access via ftp. 7-day archive file size is about 600 MB.
DATA DESCRIPTION The archive data file contains the data in synoptic time sequence, without any missing records (missing data is represented by nulls and the forecast hour is set to negative 1). Therefore it is possible to position randomly to any point within a data file. Each file contains data for one week except for files containing data past the 28th of the month. At each time period, an index record is always the first record, followed by surface data, and then all data in each pressure level from the ground up. GDAS1 data are available in the files called gdas1.mmmyy.w#, where mmm is the month (e.g. jul) and yy is the year (05) and # refer to:
#=1 - days 1-7
#=2 - days 8-14
#=3 - days 15-21
#=4 - days 22-28
#=5 - days 29 - rest of the month
Data Grid The data are on a 360 by 181 latitude-longitude (grid). The lower-left corner (1,1) is (0W,90S). The upper-right corner (360,181) is (1W, 90N). In Table 1, the data grid is identified by the model that produced the data, a grid identification number, the number of X and Y grid points, the Pole position (latitude and longitude) of the grid projection, a reference latitude and longitude, the grid spacing (km) which is true at the reference point, the orientation with respect to the reference longitude, the angle between the axis and the cone, and a point on the grid in grid units and latitude and longitude . The given pole position results in the lowest left grid point to have a value of (1,1).
Table 1. Data Grid Specifications
Model Type | ID # | X Max | Y Max | Pole Lat. | Pole Lon. | Ref. Lat. | Ref. Lon. | Ref. Grid | Orientation | Cone Ang. | Sync X | Sync Y | Sync Lat. | Sync Lon. |
GDAS | 99 | 360 | 181 | 90N | 359E | 1N | 1E | 111.1984 | 0 | 0 | 1.0 | 1.0 | 90.0S | 0.0 |
Table 2. Meteorological Fields contained in the GDAS Archive.
Field | Units | Label | Data Order |
Pressure at surface | hPa | PRSS | S1 |
Pressure reduced to mean sea level | hPa | MSLP | S2 |
Accumulated precipitation (6 h accumulation) | m | TPP6 | S3 |
u-component of momentum flux (3- or 6-h average) | N/m2 | UMOF | S4 |
v-component of momentum flux (3- or 6-h average) | N/m2 | VMOF | S5 |
Sensible heat net flux at surface (3- or 6-h average) | W/m2 | SHTF | S6 |
Downward short wave radiation flux (3- or 6-h average) | W/m2 | DSWF | S7 |
Relative Humidity at 2m AGL | % | RH2M | S8 |
U-component of wind at 10 m AGL | m/s | U10M | S9 |
V-component of wind at 10 m AGL | m/s | V10M | S10 |
Temperature at 2m AGL | K | TO2M | S11 |
Total cloud cover (3- or 6-h average) | % | TCLD | S12 |
Geopotential height | gpm* | SHGT | S13 |
Convective available potential energy | J/Kg | CAPE | S14 |
Convective inhibition | J/kg | CINH | S15 |
Standard lifted index | K | LISD | S16 |
Best 4-layer lifted index | K | LIB4 | S17 |
Planetary boundary layer height | m | PBLH | S18 |
Temperature at surface | K | TMPS | S19 |
Accumulated convective precipitation (6 h accumulation) | m | CPP6 | S20 |
Volumetric soil moisture content | frac. | SOLM | S21 |
Categorial snow (yes=1, no=0) (3- or 6-h average) | CSNO | S22 | |
Categorial ice (yes=1, no=0) (3- or 6-h average) | CICE | S23 | |
Categorial freezing rain (yes=1, no=0) (3- or 6-h average) | CFZR | S24 | |
Categorial rain (yes=1, no=0) (3- or 6-h average) | CRAI | S25 | |
Latent heat net flux at surface (3- or 6-h average) | W/m2 | LHTF | S26 |
Low cloud cover (3- or 6-h average) | % | LCLD | S27 |
Middle cloud cover (3- or 6-h average) | % | MCLD | S28 |
High cloud cover (3- or 6-h average) | % | HCLD | S29 |
Geopotential height | gpm* | HGTS | U1 |
Temperature | K | TEMP | U2 |
U-component of wind with respect to grid | m/s | UWND | U3 |
V-component of wind with respect to grid | m/s | VWND | U4 |
Pressure vertical velocity | hPa/s | WWND | U5 |
Relative humidity | % | RELH | U6 |
Meteorological Fields and Vertical Structure The archived data files contain only some of the fields produced by the model at NCEP. These fields were selected according to what is most relevant for transport and dispersion studies and disk space limitations. In Table 2, the fields are identified by a description, the units, and a unique four character identification label that is written to the header label (see Data Grid Unpacking Procedure in a later section) of each record. Data order in the file is given by a two digit code. The first digit indicates if it is a surface (or single) level variable (S) or an upper level variable (U). The second digit indicates the order in which that variable appears in the file. The upper level GDAS data are output on the following 23 pressure surfaces. Table 3 gives the level number corresponding to each data level, which is also written to each header label.
Table 3. Description of Vertical Levels
Level | Height (hPa) |
23 | 20 |
22 | 50 |
21 | 100 |
20 | 150 |
19 | 200 |
18 | 250 |
17 | 300 |
16 | 350 |
15 | 400 |
14 | 450 |
13 | 500 |
12 | 550 |
11 | 600 |
10 | 650 |
9 | 700 |
8 | 750 |
7 | 800 |
6 | 850 |
5 | 900 |
4 | 925 |
3 | 950 |
2 | 975 |
1 | 1000 |
Missing Data Missing data are written as an array of nulls with a forecast hour of -1 in the header label. The associated field label may be either "NULL" or the label given in Table 2.
Definition File The definition file given in Appendix A summarizes the grid specifications and data fields. The format is such that the first 20 characters are the dummy ID field followed by the data. Much of the information is written into the index record of each time period.
Record 1 consists of a four character string that identifies the source of
the meteorological data.
Record 2 is the integer identification of the
meteorological data grid (Table 1).
Record 3 is an integer number that
identifies the vertical coordinate system. Only four coordinate types are
recognized: 1-pressure sigma; 2-pressure absolute; 3-terrain sigma; 4-hybrid
sigma.
Records 4 & 5 identifies the pole position of the grid
projection. Most projections will either be defined at +90 or -90 depending upon
the hemisphere. The longitude would be the point 180 degrees from which the
projection is cut.
Records 6 & 7 is the reference position at which
the grid spacing is defined.
Record 8 is the grid spacing in km at the
reference position.
Record 9 is the grid orientation or the longitude of
the meridian which is parallel to the up-down direction of the grid.
Record 10 is the angle between the axis and the surface of the cone. For
regular projections it is equal to the latitude at which the grid is tangent to
the earth's surface. A polar stereographic grid would be tangent at either 90 or
-90, while a Mercator projection is tangent at 0 latitude. A Lambert Conformal
projection would be in between the two limits. An oblique stereographic
projection would have a cone angle of 90.
Records 11 & 12 are used
to equate a position on the grid with a position on the earth as given in
Records 13 & 14. Any position is acceptable. It need not even be on the
grid.
Record 15 is not currently used.
Records 16 & 17
identify the number of grid points in each direction.
Record 18 is the
number of levels in the vertical, including the surface level.
Record
19, through the number of levels, identifies the height of each level in
appropriate units according the definition of the vertical coordinate, the
number of variables at that level, and the four character identification string
for each variable. The height coordinate is as follows for each type of vertical
coordinate: 1-sigma (fraction); 2-pressure (mb); 3-terrain (fraction); 4-hybrid
(mb-offset.fraction)
Index (INDX) record - first record of each
time period
The key to reading the meteorological files is decoding the ASCII index record, the first record of each time period. The first 50 characters of the index record contain the same "header" information as do the other records in the given time period. The four-character label is "INDX". The format for this record is given below. Complete descriptions are similar to the variables in the discussion above of the Definition File.
Format of the Index Record
Format | Information |
A4 | Data Source |
I3 | Forecast hour |
I2 | Minutes associated with data time |
12F7. | 1) Pole Lat, 2) Pole Long, 3) Tangent Lat, 4) Tangent Long, 5) Grid Size, 6) Orientation, 7) Cone Angle, 8) X-Synch pnt, 9) Y-Synch pnt, 10) Synch pnt lat, 11) Synch pnt long, 12) Reserved |
3I3 | 1) Numb x pnts, 2) Numb y pnts, 3) Numb levels |
I2 | Vertical coordinate system flag |
I4 | Length in bytes of the index record, excluding the first 50 bytes |
F6. | height of the first level |
I2 | number of variables at that level |
A4 | variable identification |
I3 | rotating checksum of the packed data |
1X | Reserved space for future use |
Data Grid Unpacking NCEP typically saves their model output in GRIB format. However, at ARL the data are stored in a more compact form and can be directly used on a variety of computing platforms with direct access I/O.
The data array is packed and stored into one-byte characters. To preserve as much data precision as possible, the difference between adjacent grid point=s values is saved and packed rather than the actual values. The grid is then reconstructed by adding the differences between grid values starting with the first value, which is stored in unpacked ASCII form in the header record at grid point (1,1). To illustrate the process, assume that a grid of real data, R, of dimensions i,j is given by the below example.
1,j 2,j .... i-1,j i,j 1,j-1 2,j-1 .... i-1,j-1 i,j-1 .... .... .... .... .... 1,2 2,2 .... i-1,2 i,2 1,1 2,1 .... i-1,1 i,1
The packed value, P, is then given by
Pi,j = (Ri,j - Ri-1,j)* (2**(7-N)),
where the scaling exponent
N = ln dRmax / ln 2 .
The value of dRmax is the maximum difference between any two adjacent grid points for the entire array. It is computed from the differences along each i index holding j constant. The difference at index (1,j) is computed from index (1,j-1), and at 1,1 the difference is always zero. The packed values are one byte unsigned integers, where values from 0 to 126 represent -127 to -1, 127 represents zero, and values of 128 to 254 represent 1 to 127. Each record length is then equal in bytes to the number of array elements plus 50 bytes for the header label information. The 50 byte label field precedes each packed data field and contains the following ASCII data:
Field | Format | Description |
Year | I2 | Greenwich date for which data valid |
Month | I2 | " |
Day | I2 | " |
Hour | I2 | " |
Forecast* | I2 | Hours forecast, zero for analysis |
Level | I2 | Level from the surface up (see Table 3) |
Grid | I2 | Grid identification (see Table 1) |
Variable | A4 | Variable label (see Table 2) |
Exponent | I4 | Scaling exponent needed for unpacking |
Precision | E14.7 | Precision of unpacked data |
Value 1,1 | E14.7 | Unpacked data value at grid point 1,1 |
Sample Program A sample FORTRAN90 program is available from the ARL ftp server ( ftp://gus.arlhq.noaa.gov/pub/archives/utility/chk_data.f ) that can be used to unpack and read the first few elements of the data array for each record of an ARL packed meteorological file.
Appendix A. Definition File - GDAS.CFG
MODEL TYPE: | GDAS |
GRID NUMB: | 99 |
VERT COORD: | 2 |
POLE LAT: | 90. |
POLE LON: | 359. |
REF LAT: | 1. |
REF LON: | 1. |
REF GRID: | 0. |
ORIENTATION: | 0. |
CONE ANGLE: | 0. |
SYNC X: | 1.0 |
SYNC Y: | 1.0 |
SYNC LAT: | -90.0 |
SYNC LON: | 0. |
SPECIAL: | 0. |
NUMB X: | 360 |
NUMB Y: | 181 |
NUMB LEVELS: | 24 |
LEVEL 1: | 0. 29 PRSS MSLP TPP6 UMOF VMOF SHTF DSWF RH2M U10M V10M T02M TCLD SHGT CAPE CINH LISD LIB4 PBLH TMPS CPP6 SOLM CSNO CICE CFZR CRAI LHTF LCLD MCLD HCLD |
LEVEL 2: | 1000.0 6 HGTS TEMP UWND VWND WWND RELH |
LEVEL 3: | 975.00 6 HGTS TEMP UWND VWND WWND RELH |
LEVEL 4: | 950.00 6 HGTS TEMP UWND VWND WWND RELH |
LEVEL 5: | 925.00 6 HGTS TEMP UWND VWND WWND RELH |
LEVEL 6: | 900.00 6 HGTS TEMP UWND VWND WWND RELH |
LEVEL 7: | 850.00 6 HGTS TEMP UWND VWND WWND RELH |
LEVEL 8: | 800.00 6 HGTS TEMP UWND VWND WWND RELH |
LEVEL 9: | 750.00 6 HGTS TEMP UWND VWND WWND RELH |
LEVEL 10: | 700.00 6 HGTS TEMP UWND VWND WWND RELH |
LEVEL 11: | 650.00 6 HGTS TEMP UWND VWND WWND RELH |
LEVEL 12: | 600.00 6 HGTS TEMP UWND VWND WWND RELH |
LEVEL 13: | 550.00 6 HGTS TEMP UWND VWND WWND RELH |
LEVEL 14: | 500.00 6 HGTS TEMP UWND VWND WWND RELH |
LEVEL 15: | 450.00 6 HGTS TEMP UWND VWND WWND RELH |
LEVEL 16: | 400.00 6 HGTS TEMP UWND VWND WWND RELH |
LEVEL 17: | 350.00 6 HGTS TEMP UWND VWND WWND RELH |
LEVEL 18: | 300.00 6 HGTS TEMP UWND VWND WWND RELH |
LEVEL 19: | 250.00 6 HGTS TEMP UWND VWND WWND RELH |
LEVEL 20: | 200.00 6 HGTS TEMP UWND VWND WWND RELH |
LEVEL 21: | 150.00 6 HGTS TEMP UWND VWND WWND RELH |
LEVEL 22: | 100.00 6 HGTS TEMP UWND VWND WWND RELH |
LEVEL 23: | 50.000 4 HGTS TEMP UWND VWND |
LEVEL 24: | 20.000 4 HGTS TEMP UWND VWND |