EPIC - Documentation - EPIC netCDF General Conventions

The purpose of these netCDF Conventions is to promote sharing of oceanographic observed or in-situ data and software based on the netCDF data format.

Documentations located at ftp://ftp.unidata.ucar.edu/pub/netcdf/Conventions/PMEL-EPIC describe the Conventions common to all PMEL-EPIC style netCDF data files. Please see the accompanying documents describing Conventions specific to specific data types:

This document describes the netCDF implementation utilized by NOAA's Pacific Marine Environmental Laboratory (PMEL) for use with the EPIC software package for oceanographic data. PMEL has developed the EPIC I/O system library (EPS library), which is layered on top of the netCDF input/output library, to write netCDF files with the conventions described here. This library is not intended to provide the complete functionality which is available with the netCDF library, but rather to simplify production of a standardized implementation of netCDF for oceanographic data. It transparently provides support for some commonly used variations on the recommended standard implementation, and could be modified to provide support for others. With the use of this library, all application programs are independent of data file format, and formats other than the Unidata netCDF format can be supported by the addition of a set of "format dependent" routines to the "format dependent" layer of the library. At present, tow other formats (Classic EPIC and formatted ASCII file) are supported in addition to the netCDF format.

Data files with the PMEL-EPIC conventions are compatible with the netCDF calculator function which PMEL uses in conjunction with the PPLUS graphics package. Both EPIC and the EPS library including example programs and other applications and tools are available via anonymous ftp.

General Conventions

The EPIC system library is intended for reading and writing geophysical data, and assumes this data is represented by three spatial axes and one time axis. Each data file has dimensions, or axes, of longitude, latitude, depth and time. One or more of these axes may be collapsed to a single point, but four axes are always present. For example, data files containing oceanographic observational data, such as time series data or CTD profile data, may have latitude and longitude axes each consisting of a single point. Although we provide the ability to read netCDF files with fewer than four axes, we do not recommend this practice. Our underlying philosophy is that all geophysical data is located by longitude, latitude, depth and time, and if this information is not included as an axis, it must be included elsewhere within the data file, if the file is to be self-describing.

Axes

EPIC axis conventions are described here. These conventions are compatible with the netCDF calculator function which PMEL uses in conjunction with the PPLUS scientific graphics package.

Geographic axes in a PMEL-EPIC netCDF file are, at present, described by a numeric variable code included as an attribute in the data file. There is a disk file named "epic.key" which contains all EPIC variable codes with other related information. The numeric variable code is a unique identifier for the variable or axis, and is described more fully in the section below on "VARIABLES". The use of the numeric variable code to define the axis is necessary for EPS libary V2.1 and earlier, and for PPLUS V1.2c and earlier. In future releases of both the EPS libarary and PPLUS, the use of an axis variable code may be replaced by the use of units from UDUNITS, and may no longer be required.

1) Longitude axis

Many longitude representations are possible, however, for compatability with the netCDF calculator function of PPLUS V1.2c (and earlier versions), use of the the West longitude convention is required. The West longitude convention, described in detail below, defines the numeric representation of the longitude axis in the netCDF file, with positive values for west longitudes. In future releases of the netCDF calculator function of PPLUS and also of the EPS library, both East and West longitude conventions will be supported. The recommended West longitude convention is described in the following paragraphs.

We recommend that users represent the longitude axis with the West longitude convention. This means that western longitudes are positive numbers, for example, 170W is +170.0. Eastern longitudes can have either of two representations, both of which are supported by the EPS library, EPIC system application programs, and the PPLUS graphics system:

If the data being represented would best be described as continuous across the dateline, then east longitudes are written as (360.-long).
EXAMPLE: 170E is written as 190.0.
If the data being represented would best be described as continuous across the Greenwich meridian, then east longitudes are written as (-long).
EXAMPLE: 170E is written as -170.

The units of the longitude axis in the EPIC system library routine call should be selected either from the Unidata netCDF's udunits.dat file, or the EPIC system key file (see the EPS manual). The default spatial axis data type is real number. The recommended units are degree_west (epic variable code 501).

2) Latitude axis

The recommended latitude convention is for north latitudes to be represented by positive numbers (e.g., 10N is +10.0), and south latitudes by negative numbers (e.g., 10S is -10.0). The units of the axis should be compatible with UDUNITS. The default spatial axis data type is real number. The recommended units are degree_north (epic variable code 500).

3) Depth axis

The depth axis should be given with the oceanographic convention of depth as a positive number, increasing downwards from the surface of the water towards the bottom of the ocean. The units of the axis should be compatible with UDUNITS. The default spatial axis data type is real number. Recommended units include dbar (pressure axis with epic variable code 1) or meters (depth axis with epic variable code 3).

Although the EPS library and PPLUS will support depth as a negative number, decreasing from the surface towards the bottom of the ocean, EPIC system application programs do not support this convention.

At present, only surface atmospheric data is being written to EPIC style netCDF data files. Conventions could be expanded to include this axis type, if desired, e.g., if upper air data is written in future, this axis could have the name "Height", with a corresponding epic variable code created to identify it uniquely.

4) Time axis

The EPIC system library routines return the time axis from a data file to the calling routine in the form of a two-integer array, in which the first integer is the "True Julian Day Number" with units of days, and the second integer is the number of milliseconds since 0000 GMT of the True Julian Day. The True Julian Day (eg, May 23, 1968 is 2,400,000), used by astronomers, should not be confused with the "year-day" (eg, Feb 2 is year-day 33). The "year-day" is frequently called julian day (incorrectly) by oceanographers and meteorologists. Our double-dimensioned integer time word (word1=True Julian Day, word2=milliseconds since 0000 GMT of the True Julian Day) allows millisecond accuracy for time periods extending over centuries. There is a complete set of EPS routines for manipulation, calculation, and character string representation of this standard representation of time. The representation of the time axis within the data file can be of several types, for both read and write, including the UDUNITS standard, the two-integer array which is used internally by the EPS routines, and some other time representations which are supported for compatibility with other in-house software packages. Time axes can be written or read in either real or integer format. Note that, regardless of the format of the time axis in the netCDF data file itself, the values used internally by the EPIC system library will be the two-integer array.

Variables

Each variable in an EPIC data file is described by a numeric variable code in the data file. There is a disk file named "epic.key" which contains the variable code and other related information about the variable. The numeric variable code is a unique identifier for the variable. Although the idea of a numeric code identifying the variable may seem at first unnecessary for netCDF files, there are several advantages to having a dependable variable identifier associated with the variables in the file. One reason is that it is unlikely that all the information about the variable will actually be included in the netCDF file (for example, exactly what algorithm was used to calculate salinity). Another reason is that it is difficult to build tools that will do complex tasks based on generic netCDF files. For example, salinity calculated by two different methods could be identified with two different numeric variable codes, but application software can be written to recognize both of these codes as a "salinity" value for use in the calculation of density. Therefore, we recommend that the variable codes be used to identify variables in the data file. Although the EPS library will write netCDF files which omit the numeric code entirely, this omission will result in a lack of information available to EPIC or other application programs.

Attributes

There are standard attributes created by the EPS library for EPS files, as described in the following table, and additional attributes may be defined by the user as desired.

      =======================================================

      Standard Attributes Automatically Written for EPS Files

      =======================================================





Category   Attribute Name   Type  Comments 

------------------------------------------------------------------

global     CREATION_DATE    c     File creation date and time 

           Conventions      c     ** Name of Conventions used by  

                                     the file,(eg. PMEL-EPIC/CTD)

------------------------------------------------------------------

variable   name             c     variable name 

           long_name        c     variable long name 

           generic_name     c     variable generic name 

           FORTRAN_format   c     data Fortran format 

           units            c     variable unit 

           epic_code        i     variable code; write only when 

                                   it is defined in epic.key file 

------------------------------------------------------------------

axis       FORTRAN_format   c     axis data Fortran format 

           units            c     axis unit 

           type             c     axis data type 

           epic_code        i     axis variable code; write only  

                              when it is defined in epic.key file 

------------------------------------------------------------------

 NOTES:	"i" is integer

	"c" is character

	 ** The global attribute "Conventions" is not automatically

	    written by EPS libary V2.1, or earlier, but will be  

	    written automatically by later releases of EPS.

The following table lists attributes which should always be included in the file to assure
compatibility with EPIC programs.

      ==============================================

      Attributes Required for Use with EPIC Programs

      ==============================================



Attribute Name   Parameter Name Type Length  Comment

-----------------------------------------------------------------

INST_TYPE        EPINSTTYPE      c    26   Instrument type  

DATA_TYPE*       EPDATATYPE      c    any  must be one of: 

                                           "CTD", "TIME", 

                                           "TRACK" or "GRID"

DATA_SUB_TYPE*   EPSUBTYPE       c    any  Description of 

                                           DATA TYPE (see below)

DATA_ORIGIN      EPDATAORIGIN    c    26  Data Origin Information 

COORD_SYSTEM     EPCOORDSYS      c    12   ``GEOGRAPHICAL'' for

                                           classic EPIC file



 *    Valid DATA_TYPEs and DATA_SUB_TYPEs

      ===================================



DATA_TYPE        DATA_SUB_TYPE     Description

-----------------------------------------------------

"CTD"            "CTD"             CTD casts

"CTD"            "XBT"             XBT casts

"CTD"            "BOT"             Bottle data

-----------------------------------------------------

"TIME"            any              no subtype required 

                                   for time series

-----------------------------------------------------

"TRACK"          "ADCP"            Shipboard ADCP

"TRACK"          "TCTD"            Towed CTD

-----------------------------------------------------

"GRID"           "XY"              X-Y gridded data

In addition, EPIC programs maintain a list of "Program Comments" where a brief description of the data modification/processing is being kept. These program comments are devised as a stack, i.e. a new comment is written to PROG_CMNT1, the others are pushed down by one position, the oldest comment is lost.

      ==========================

      Program Comment Attributes

      ==========================



Attribute Name Parameter Name Type Length  Comment

-------------------------------------------------------------



PROG_CMNT1       EPPROGCMNT1    c  40   Latest program comment 

PROG_CMNT2       EPPROGCMNT2    c  40   Next program comment 

PROG_CMNT3       EPPROGCMNT3    c  40   Next program comment 

PROG_CMNT4       EPPROGCMNT4    c  40   Oldest program comment

Similarly, standard global attributes are created for EPS data files containing specific types of oceanographic data (e.g., PMEL-EPIC/CTD and PMEL-EPIC/Time_Series). Additional global and variable attributes may be defined by the user as desired. The standard global attributes for each specific data type are described in the Conventions documents in those subdirectories of pub/netcdf/Conventions/PMEL-EPIC (e.g., pub/netcdf/Conventions/PMEL-EPIC/CTD ).