Internal Data Format for the Hydrophone Data Files
NOAA uses its
own internal format to store the T-Phase data collected from the NOAA's Autonomous
Hydrophone Array. The T-Phase
data stored in this internal format will be referred to as
NHP (NOAA HydroPhone) data files. The
structure of the NHP format is shown below. The NHP data files are
stored as binary data.
The first 4 bytes of the NHP data file forms
a signed long integer value. Shown below is the header size in bytes.
The second 4 bytes forms an unsigned long integer value for
the data size in bytes. Both integers are stored in Little Endian
format, therefore, byte swapping may be required (including the decoding
data) if
a Big Endian computer is used. After the first 8 bytes, the header
begins. After the header, the actual data starts. Each NHP
data file can be viewed as follows:
NHP Data File Structure
|<-------- 1 Data File --------|
|HSZE|DSZE|<-- Header -->|<-- Data -->|
Byte position -> 0123 4567 89 --------- ^ ^ ^
begins with 0 H-1 H ------ N-1 <- Byte Position
where HSZE = Header Size in bytes, DSZE = Data Size in bytes.
The Header is from byte position 8 to H - 1 where H = HSZE + 8
Data startw from H to N - 1 where N = DSZE + H = Total file size in bytes.
NHP Header Contents
To see the contents of the header, the header bytes must be converted to characters.
The following shows an example of the header which can be found in all the NOAA
internal data format files.
Start Time: 1999 360-00:00: 0.000
End Time: 1999 360-23:59:59.991
Sample Rate (Hz): 110.7761690
Sample Size : 2 Bytes (Little Endian)
HPhone Lat (Deg): 31.865383
HPhone LNG (Deg): -34.438017
HPhone Depth (m): 927
Data Source: Model 2v0
N Channels: 1
X, Y, Z (meters): 0, 0, 0
A/D Voltage Range (from 0 to): 5
Mean Voltage approximately: 2.5
Number of Bits of the Digitizer: 8
Hydrophone Sensitivity (dB): -192
Filter Cutoff (Hz): 45
Points from the Pre-Amp Response (for reading the next 2 lines): 8
Hz: 1 2 5 10 20 30 40 50
dB: 35.0 49.0 57.0 60.0 62.0 63.0 64.0 60.0
Start Time and End Time show
the date range of the file. The time ranges are in Year, Julian Date, Hour:Minute:Second.
Julian Date is day of the year which can be from 001 to 366.
Sample Rate is
Hz or data points per second. For the NOAA's Autonomous
Hydrophone data file, the sample rate will be around 100, 110,
250, and 1000 Hz; but they will not be exact! The sample rate is computed
by how many data points were collected within a given time, e.g. (Total data
points of an hour)/(3600 seconds) = Sample Rate.
Sample Size indicates how
each data point is stored in 2 bytes short integers in Little Endian format. For
the Autonomous Hydrophone data file, the Sample Size will primarily be 2.
The HPhone Lat and LNG are
the actual hydrophone locations of the hydrophone in latitude ( > 0 is North
and < 0 is South ) and longitude ( > 0 is West
and < 0 is East ) in degrees.
The HPhone Depth (m) shows
the depth in meters of the hydrophone after it was deployed into the sea.
Data Source indicates the hydrophone version.
N Channels shows the number of
channels that were used. When N Channels
is > 1, e.g. 5, the header records from X,Y,Z (meters): to dB: will be repeated
5 times. For the NOAA's Autonomous
Hydrophone , the N Channels will be always equal to 1.
X, Y, Z (meters) useful for the
relative position in terms of X, Y, and Z coordinates
when multiple channels are available. Position
(0,0,0), the origin or point of love, will be refereed to the HPhone Lat, Lng,
and Depth respectively. Other (X,Y,Z) channel position will be a distance
away from the point of
love.
A/D Voltage Range shows the maximum
range. The minimum range
will be 0. It is the range used to measure the electric potential as
the data before converting them to digital units.
Mean Voltage is
approximately equal to half way of A/D Voltage Range.
Number of Bits
of the Digitizer can
be 8 or 12, 16 etc., it shows the signal resolution.
Filter Cutoff indicates frequency
in Hz cutoff for a low-pass
filter.
Hydrophone Sensitivity is in dB
units
Points from the Pre-Amp Response show
Number of Points for the Pre-Amp Response curve. From the example
above, the next 2 records
will each contains 8 numbers: 8 Frequency in Hz (or the X coordinate) and 8
the Response in dB (or the Y coordinate). The Pre-Amp Response curve
together with the Hydrophone Sensitivity will provide the system sensitivity
or response.
From Data Source to Pre-Amp
Response are the system settings
for the hydrophone during a deployment (see Hydrophone
System Settings that shows l the settings used in the past deployments).
NHP Data
The data after the header are stored in either short or
long integers as indicated by the Sample Size in the header. Total data
size in bytes is stored in the second 4 bytes (byte position 4 to 7) at the beginning
of the NHP file. The
Total Data Points will be: (Total Data Size in Byte)/(Sample Size).
The data are stored in binary in a Little
Endian format. If a Big Endian or High-Order Byte workstation is used, Byte
Swapping is needed to decode the data.
Decoding NHP Data Block using MatLab Software
The following MatLab commands will decode a NHP data file. Note that this
section is assuming readers know MatLab and how to refer to the MatLab manual
for details.
Enter MatLab ( in UNIX system, type MatLab then the MatLab working window
will show up )
>>> FID = fopen( 'H00N095W98198Z.nhp', 'r', 'l' );
where H00N095W98198Z.nhp is a NHP data file name,
'r' indicates read only, and
'l' is a lower case L, indicates the data are from
a Little Endian machine, i.e. 'l' is required if the
local machine is using Big Endian format.
>>> hdrsze = fread( FID, 1, 'int32' ) # Heade size in bytes, e.g 590.
>>> bytsze = fread( FID, 1, 'uint32' ) # Total data in bytes, e.g 17109806.
>>> header = fread( FID, hdrsze, 'char' ); # Reading the Header in bytes.
>>> char( header )' # Display the header in character strings as shown
below between the 2 desh lines.
----------------------------------------------------------------
Start Time: 1998 198-00:00: 0.000
End Time: 1998 198-23:59:59.990
Sample Rate (Hz): 99.0150926
Sample Size : 2 Bytes (Little Endian)
HPhone Lat (Deg): 0.12650000
HPhone LNG (Deg): -94.926833
HPhone Depth (m): 741
Data Source: Model 1v0
N Channels: 1
X, Y, Z (meters): 0, 0, 0
A/D Voltage Range (from 0 to): 5
Mean Voltage approximately : 2.5
Number of Bits of the Digitizer: 8
Hydrophone Sensitivity (dB): -192
Filter Cutoff (Hz): 40
Points from the Pre-Amp Response (for reading the next 2 lines): 7
Hz: 1 2 5 10 20 30 40
dB: 41.9 53.3 61.0 63.2 65.0 66.4 60.4
----------------------------------------------------------------
Define the Sample Rate and the Sample Size as shown in the header above.
>>> sample_rate = 99.0150926; # in Hz.
>>> sample_size = 2 ; # in bytes.
>>> data_type = 'int16'; # or "short"
>>> n_data = bytsze/sample_size # Total data points.
>>> data = fread( FID, n_data, data_type ); # Read in all data.
Note that if memory is an issue, data can be read in segments.
Also if sample_size is equal to 4 or 8, then data_type will be
equal to 'int32'/'long' or 'int64'/'double' respectively.
>>> fclose( FID ); # Close the NHP data file.
>>> t_index = [1:n_data]/sample_rate/3600; # Time indexes in hours.
>>> plot( t_index, data ) # Display all data.
::: At this point data can be processed and saved.
>>> exit; # Out of MatLab when ready.
Decoding NHP Data File using IDL Software
Package
The following IDL commands will decode a NHP data file's
Header and
1 data block at a time. Note that this section is assuming readers know IDL and
how to refer to the IDL manual for details.
Enter IDL ( in UNIX system, type IDL then IDL> should
show up )
IDL>OPENR, 10, 'H16N034W00101Z.nhp'
Define the BLK to read in 1st 8 bytes as 2 unsigned long integer values.
They are the header and data sizes in bytes.
IDL>BLK = ASSOC( 10, ULONARR( 2 ) )
IDL>H = BLK[0] ; H = A long integer array with 2 elements.
The 2 unsigned long integers are stored in Little Endian format.
Swap the byte order if a Big Endian workstation is used.
IDL>BIG_ENDIAN = (BYTE(1,0,2))[0] EQ 0B
IDL>IF BIG_ENDIAN THEN BYTEORDER, H, /LSWAP ; Bytes swapping is needed.
IDL>HDR_SIZE = H[0] ; Header Size in bytes.
IDL>TOTAL_DATA_BYTES = H[1] ; Total number of data bytes after the header.
Redefine the BLK to read in the whole header.
Note that the offset of the 1st 8 bytes is needed.
IDL>BLK = ASSOC( 10, BYTARR( HDR_SIZE ), 8 )
IDL>HEADER = BLK[0] ; Read in the Header.
IDL>PRINT, STRING( HEADER ) ; Show the contents of the Header.
:::
:::
Assuming the Sample Size is 2 and reading 1st 1024 data points.
IDL>OFFSET = HDR_SIZE + 8 ; For skipping all the header information.
IDL>BLK = ASSOC( UNIT, INTARR( 1024 ), OFFSET )
IDL>DATA = BLK[0] ; Read in the 1st 1024 data points.
IDL>IF BIG_ENDIAN THEN BYTEORDER, DATA, /SWAP ; If it's needed.
IDL>PLOT, DATA ; Display the data.
IDL>DATA = BLK[1] ; Read in the 2nd 1024 data points.
:::
Note that the total data points will most likely not be evenly divided
by the block size such as 1024 or other any other integers; therefore,
for the last uneven block, the correct OFFSET must be calculated and reset
the ASSOC( ) function again before reading the remaining data points.
IDL>CLOSE, 10 ; Close the NHP data file.
IDL>EXIT ; Out of IDL
Converting NHP Data to Other Data Formats
If NHP data are needed to be in data formats other
than CSS and WAV, NHP data must be retrieved in CSS format. Consult the ORFEUS
Seismological Software Library as it provides a wide range of programs
that are able to convert CSS to other formats such as SAC and SEED.
Hydrophone Data in other formats: CSS | wav.
Back to retrieving data. |
