Go to the previous, section, or Table of Contents.


EPS Implementation of ASCII Data File Format

Read only support is provided for following ASCII file formats: (1) generic formatted ASCII data files, and (2) format used internally at PMEL for processing PROTEUS mooring data.

EPS Generic ASCII file format

To read a generic ASCII data files using EPS interface, you must provide a data file table (or called it pointer file) which contains information about the data files and your data files must follow certain roles.

I. Data File

The EPS interface can read any ASCII data file by following these constraints:

  1. A data file table must be provided which contains some information about the data files. (We will discuss data file table in the next section).

  2. The data file can contain any number of header lines.

  3. Assume all variables are one dimension variables. The rest of the axes will be treated as either constants or as variables.

  4. Except time axis, all data (variables or axis) read into MATLAB must NOT be character string.

  5. Time axis must be in the formats descripted below. Any combined of these formats are acceptable.

Time String Format Description
FormatDescription
Dday formatted as 1-31
DDday formatted as 01-31
DDDday of week written as 3 character name
DDDDday of week, full format
Mmonth formatted as 1-12
MMmonth formatted as 01-12
MMMmonth formatted as 3 character name
MMMMmonth formatted as full name
YYyear written as last two digits. (Not available for strtoeptime().)
YYYYyear written as all four digits
hhour written as 1-24
hhhour written as 01-24
mminute written as 1-59
mmminute written as 01-59
ssecond written as 1-59
sssecond written as 01-59
ffraction of second written as 0-9
fffraction of second written as 00-99
ffffraction of second written as 000-999
Nyear-day number as 1-365
NNyear-day number 01-365
NNNyear-day number 001-365

 

Example of an ASCII formatted data file:

R/V Hakuhomaru TOGA-COARE Meteorological Data(10 Nov - 30 Nov 1992)
*** Corrected for Tair, Tsea, SR, LR ***
 
YYMMDD hhmm LAT(N+) LON(E)  SSP  SCO   WD  WS  Tair  q   Tsea   SR    LR 
                            knot degN deg  m/s degC g/kg degC  W/m2  W/m2     
---------1---------2---------3---------4---------5---------6---------7----
921110 0000 -4.206 152.171  0.0 138.8 130  3.5 28.8 17.1 29.0  985.2 394.2
921110 0015 -4.206 152.171  0.0 118.5 143  3.6 29.0 17.5 29.0  999.7 393.4
921110 0030 -4.206 152.171  0.0 133.1 140  3.9 29.3 17.5 29.0 1028.8 391.5
921110 0045 -4.206 152.171  0.0 150.0 143  4.3 29.2 17.5 29.0 1044.4 390.6
921110 0100 -4.206 152.171  0.0 128.9 121  4.6 28.9 17.1 29.0 1038.8 391.6
921110 0115 -4.206 152.171  0.0 118.5 136  4.2 29.3 17.3 29.0 1033.2 395.7
921110 0130 -4.206 152.171  0.0 151.4 138  5.1 29.2 17.5 29.0 1048.9 396.6
921110 0145 -4.206 152.171  0.0 136.4 136  4.8 29.4 17.5 29.0 1037.7 401.0
921110 0200 -4.206 152.172  0.2 153.0 143  6.0 29.1 17.6 29.0 1043.3 404.7
II. Data File Table

A data file table must be provided to descript the data files: number of header to skip, variable names, and formats, etc. You must use this data file table as input EPS file name when ep_open (or epopen for Fortran) is called. The data file table must be formatted in this way:

  1. The first line must contains a string "ASCII DATA FILE".

  2. Any line begins with "#" will be considered as comment line and will be ingored.

  3. Use "File name prefix = " to describe the data file path name. This is optional. The default will be the current directory.

  4. Use "EPIC key table =" to describe the table file which contains lines of variable name vs. variable EPIC code. This is optional. The default will be not using the EPIC key code. The format of this key file is:
          variable_name(string) variable_code(integer)  missing_value(float)
    
  5. Specify one data file per line with this format:
          filename   hdr_skip   x  y  z  t   varlist  formatlist
    
    where filename is data file name; hdr_skip is the number of headers in data file will be skipped; x, y, z, and t are the x-, y-, z-, and t-axis values respectively (See following notes regarding axis values); varlist is the variable name list you want to read. Each variable name in the list is separated by ",", no space is allowed. (see following notes regarding the variable name); and formatlist is the Fortarn format list of these variables you want to read.

Notes:

  1. Axis value specification:

    Since we assume variables are one dimensional, the rest of the axes will be treated as either constants or variables. When giving the x, y, z, and t axis values, you must specify one and only one axis with "-" indicating variables in the file are the variable of this axis and the value of this axis will be read from the data file. You must specify the rest of the axes with constant values. These values can be either true values or dummy values. When dummy value is given, the value of that axis will be read from the data file and it will be treated as a variable. The unit of x-axis is Longitude degree (East is positive), the unit of y-axis is Latitude degress (North is positive), the unit of z-axis is meter (doneward is positive), and time must in format of YYMMDDhhmmss. (For example, the x, y, z, t specification for a CTD data file could be: "165.5 0 - 920110053000", indicating the data file is recorded at location 165.5E, on Equator, at Jan 10, 1992 5:30 and variables are varying with depth.)

  2. Variable names:

    You can use any name for any variable except time axis. The time axis name must be "timeFORMAT" where FORMATshould be the time string format decribed in above time string format table, (e.g. timeYYMMDD).

Example of Data File Table

Following is the example data file table for above example data file. In the data file, time-axis, longitude-axis and latitude-axis are all recorded with data variables. We will assum variables vary with time-axis and treated longitude and latitude axis as variables. Therefore, dummy x-axis and y-axis values will be specified (0 0), z-axis is 0 M (sea surface) and t-axis is "-". There are 6 lines of header in data file will be skipped. Each data file entry must remain in one line. The names of time axis are "timeYYMMDD" and "timehhmm".

ASCII_DATA_FILE
File name prefix =  /home/tao/epswork/examples/programs/c
#here begins data file entry.  Each line for each file.
#filename hdr_skip x y z t varlist formatlist
coare.dat 6 0 0 0 -  timeYYMMDD,timehhmm,lat,lon,ssp,tair, i6,x,i4,f7.3,f8.3,f5.1,16x,f5.1

Internal PMEL ASCII PROTENUS mooring Data File

PROTEUS mooring data files are ASCII formatted files containing time series data from PROTEUS moorings (contact Paul Freitag, NOAA/PMEL for more information). Each data file contains 1 line of header (which is simply skipped by EPSLIB) followed by formatted data records. The data records in the data file itself includes a time-date stamp and one or more variables.

Here is an example data file:

TOGA/FR FU6 AP09 9  60.0  -4.0 ERRFLG   
 15 0 15 992    3.54    1.22    3.75  70.9 26.50 29.17 83.06 14.82 29.13 34.01   0.0     1
 16 0 15 992    3.29    0.39    3.31  83.3 26.64 29.19 82.27 14.83 29.15 34.02   0.0     2
 17 0 15 992    2.65    0.90    2.80  71.3 26.77 29.21 80.99 14.82 29.19 34.02   0.0     3
 18 0 15 992    3.16    0.03    3.16  89.5 26.98 29.20 80.99 14.84 29.16 34.01   0.0     4
 19 0 15 992    3.03   -1.00    3.19 108.3 27.10 29.25 79.82 14.77 29.26 34.04   0.2     5
The variable names and formats are described in the data file table (pointer file). Here is an exmaple of data file table:

Proteus Mooring 
TC2   0  0.4N 156  1.8E    0700 19 Dec 93   0000 29 Apr 94
Time variable prefix = hr,mn,da,mo,yr,
Time format prefix =  1x,2i2,1x,3i2,
File name prefix = /home/pagoo3/data/tc1/
EPIC key table = /home/pagoo/freitag/hourly/proteus_epic.key
#filename dep tbase    inst  variables         format
tc2b.wnd   -4 cccce    a2    uw,vw,sw,dw,sr    3f8.2,f6.1,36x,f6.1
tc2b.wnd   -3 ee       a2    at,rh             30x,f6.2,6x,f6.2

You must use this data file table (or called it pointer file) as input EPS file name when ep_open (or epopen for Fortran) is called. The "EPIC Key table" indicates the the table file which contains lines of variable name vs. variable EPIC code.


Go to the previous, section, or Table of Contents.