This document describes a set of conventions that were developed by the EPIC group at NOAA/PMEL for remotely accessing in-situ geophysical data through the OPeNDAP network DAP 2.0 protocol (pdf). These converntions are implemented by the Dapper server
Geophysical in-situ data consists of a set of measurements of physical quantities that are sampled at irregular space and time coordinates. Examples include time series of weather data collected by weather stations, CTD measurements of ocean salinity and temperature as a function of depth, and radiosonde measurements of atmospheric temperature and pressure as a function of height.
It is difficult to model in-situ data with the the conventions that already exist for gridded data (e.g. COARDS and CF). We have developed conventions that, we believe, are more natural for in-situ data and we hope that these conventions will facilitate greater sharing of in-situ data in the scientific community.
Each Dapper in-situ dataset is represented as a DAP 2.0 two level nested sequence. This sequence consists of one outer and one inner sequence, where the inner sequence is contained by the outer sequence. The outer sequence contains a latitude, longitude, unique identification number, and either a depth or height variable (z) or a time (t) variable. The inner sequence contains all of the measurement variables (temperature, salinity, etc.) and a t variable (if z is defined in the outer sequence), or a z variable (if t is defined in the outer sequence).
For example, time series data from weather stations can be represented as the following OPeNDAP DDS:
Dataset {
Sequence {
Float32 lat;
Float32 lon;
Float32 elev;
Int32 _id;
Sequence {
Float32 visibility;
Float32 max_wind_gust;
Float32 dewp;
Float64 time;
Float32 slp;
Float32 temp;
Float32 wind_speed;
Float32 max_temp;
Float32 max_sustained_wind_speed;
Float32 min_temp;
Float32 precip;
} time_series;
} location; ...
} gsod_time_series;
and CTD measurements can be represented as:
Dataset {
Sequence {
Float32 lat;
Float64 time;
Float32 lon;
Int32 _id;
Sequence {
Float32 depth;
Float32 temp;
Float32 salinity;
Float32 pressure;
} profile;
} location; ...
} northAtlantic;
Dapper uses the COARDS conventions for coordinate and data variable attributes.
Structure {All members of the structure must have a String type.
String station_id; String station_name;
} attributes;
Structure { Structure { Float64 valid_range[2]; } depth;could be used to store the min/max values of each depth variable for a dataset of CTD measurements.
} variable_attributes;
Structure {The bounds of the x,y,z,and t axes are stored in the lon_range, lat_range, depth_range, and time_range variables respectively, and each of these variable has the same type as the corresponding axis variable.
Float32 lon_range[2];
Float32 lat_range[2];
Float32 depth_range[2];
Float64 time_range[2];
} constrained_ranges;
Dapper global attributes are stored in a NC_GLOBAL attribute structure to maintain compatibility with OPeNDAP to netCDF translators. The following attributes are mandatory:
Attribute | Type | Value | Description |
max_profiles_per_request | Int32 | Maximum number of time series or profiles that the server will return in one request | |
total_profiles_in_dataset | Int32 | The total count of all the time series or profiles contained in the dataset | |
Conventions | String | "epic-insitu-1.0" | Specifies that the server conforms to the EPIC/Dapper conventions |
lon_range | Float64[2] | Longitude range that contains all of the profiles or time series in the dataset | |
lat_range | Float64[2] | Latitude range that contains all of the profiles or time series in the dataset | |
depth_range | Float64[2] | Depth range that contains all of the profiles or time series in the dataset | |
time_range | Float64[2] | Time range that contains all of the profiles or time series in the dataset |
The Dapper server supports the COARDS conventions for variable attributes (e.g. units, missing_value, long_name, etc.). In addition, each coordinate variable must have a string attribute named axis that has a value of "X","Y","Z",or"T". There should only be one of each value of axis specified in the DAS.
The OPeNDAP specification requires that an OPeNDAP server support all constraints on all variables. We have found this to be difficult to implement in practice, especially with large databases. We instead have more relaxed requirements:
Following is a full DDS for a large in-situ collection of ocean profiles:
Dataset {
Sequence {
Float32 lat;
Float64 time;
Float32 lon;
Int32 _id;
Sequence {
Float32 temp_qc;
Float32 salinity_qc;
Float32 oxygen;
Float32 depth;
Float32 temp;
Float32 chlorophyll;
Float32 salinity;
Float32 oxygen_qc;
Float32 pressure_qc;
Float32 pressure;
Float32 chlorophyll_qc;
} profile;
Structure {
String title;
String data_origin;
String source;
String Conventions;
String country_code;
String cruise;
String cruise_number;
String cast_number;
} attributes;
Structure {
Structure {
Float32 valid_range[2];
} depth;
} variable_attributes;
} location;
Structure {
Float32 lon_range[2];
Float32 lat_range[2];
Float32 depth_range[2];
Float64 time_range[2];
} constrained_ranges;
} northPacific;
Following is the DAS for a large in-situ collection of ocean profiles:
Attributes {
NC_GLOBAL {
Int32 max_profiles_per_request 15000;
Int32 total_profiles_in_dataset 1707616;
String version "1.1.0";
String owner "";
String contact "";
String Conventions "epic-insitu-1.0";
Float64 lon_range 100.0, 289.180000305176;
Float64 lat_range 1.20000004244503E-5, 90.0;
Float64 depth_range 0.0, 10860.900390625;
Float64 time_range -1.1432412E12, 1.10778048E12;
}
location._id {
String long_name "sequence id";
Int32 missing_value 2147483647;
String units "";
}
location.lat {
String units "degree_north";
String long_name "latitude";
Float32 missing_value NaN;
String axis "Y";
}
location.profile.temp_qc {
String units "";
String long_name "temperature quality flag";
Float32 missing_value NaN;
}
location.profile.salinity_qc {
String units "";
String long_name "salinity quality flag";
Float32 missing_value NaN;
}
location.time {
String units "msec since 1970-01-01 00:00:00 GMT";
String long_name "time";
Float64 missing_value NaN;
String axis "T";
}
location.profile.oxygen {
String units "ml/l";
String long_name "oxygen";
Float32 missing_value NaN;
}
location.profile.depth {
String units "m";
String long_name "depth";
Float32 missing_value NaN;
String axis "Z";
}
location.lon {
String units "degree_east";
String long_name "longitude";
Float32 missing_value NaN;
String axis "X";
}
location.profile.temp {
String units "degc";
String long_name "temperature";
Float32 missing_value NaN;
}
location.profile.chlorophyll {
String units "ug/l";
String long_name "total chlorophyll";
Float32 missing_value NaN;
}
location.profile.salinity {
String units "psu";
String long_name "salinity";
Float32 missing_value NaN;
}
location.profile.oxygen_qc {
String units "";
String long_name "oxygen quality flag";
Float32 missing_value NaN;
}
location.profile.pressure_qc {
String units "";
String long_name "pressure quality flag";
Float32 missing_value NaN;
}
location.profile.pressure {
String units "dbar";
String long_name "pressure";
Float32 missing_value NaN;
}
location.profile.chlorophyll_qc {
String units "";
String long_name "total chlorophyll quality flag";
Float32 missing_value NaN;
}
}