ERDDAP Easier access to scientific data

Brought to you by NOAA NMFS SWFSC ERD

ERDDAP > tabledap > Documentation

Using tabledap to Request Data and Graphs from Tabular Datasets

The URL specifies what you want: the dataset, a description of the graph or the subset of the data, and the file
type for the response.

• (easy) You can get data by using the dataset's Data Access Form or Subset form.
  They make the URL for you.
• (easy) You can make a graph or map by using the dataset's Make A Graph form. It makes the URL for you.
• (not hard) You can bypass the forms and get the data or make a graph or map by generating the URL by
  hand or with a computer program or script.

Tabledap request URLs must be in the form
   http://coastwatch.pfeg.noaa.gov/erddap/tabledap/datasetID.fileType{?query}
For example,
   http://coastwatch.pfeg.noaa.gov/erddap/tabledap/erdGlobecBottle.htmlTable?longitude,latitude,time,bottle_posn,temperature1&time>=2002-08-17T00:00:00Z&time<=2002-08-19T20:18:00Z
Thus, the query is often a comma-separated list of desired variable names, followed by a collection of
constraints (e.g., variable<value), each preceded by '&' (which is interpreted as "AND").

Details:

• Requests must not have any internal spaces.
• Requests are case sensitive.
• {} is notation to denote an optional part of the request.
   
• datasetID identifies the name that ERDDAP assigned to the source web site and dataset
  (for example, erdGlobecBottle). You can see a list of datasetID options available via tabledap.
   
• fileType specifies the type of table data file that you want to download (for example, .htmlTable).
  The actual extension of the resulting file may be slightly different than the fileType (for example,
  .smallPdf returns a small .pdf file).
  The fileType options for downloading tabular data are:
   

  Data fileTypes	Description	Info	Example
  .asc	View OPeNDAP-style comma-separated ASCII text.	info	example
  .csv	Download a comma-separated ASCII text table (line 1: names; line 2: units; ISO 8601 times).	info	example
  .csvp	Download a .csv file with line 1: name (units). Times are ISO 8601 strings.	info	example
  .das	View the data's metadata via an OPeNDAP Dataset Attribute Structure (DAS).	info	example
  .dds	View the data's structure via an OPeNDAP Dataset Descriptor Structure (DDS).	info	example
  .dods	OPeNDAP clients use this to download the data in the DODS binary format.	info	example
  .esriCsv	Download a .csv file for ESRI's ArcGIS 9.x and below (separate date and time columns).	info	example
  .fgdc	View the data's FGDC .xml metadata.	info	example
  .geoJson	Download longitude,latitude,otherColumns data as a GeoJSON .json file.	info	example
  .graph	View a Make A Graph web page.	info	example
  .help	View a web page with a description of tabledap.	info	example
  .html	View an OPeNDAP-style HTML Data Access Form.	info	example
  .htmlTable	View a .html web page with the data in a table. Times are ISO 8601 strings.	info	example
  .iso19115	View the data's ISO 19115-2/19139 .xml metadata.	info	example
  .json	View a table-like JSON file (missing value = 'null'; times are ISO 8601 strings).	info	example
  .mat	Download a MATLAB binary file.	info	example
  .nc	Download a flat, table-like, NetCDF-3 binary file with COARDS/CF/THREDDS metadata.	info	example
  .ncHeader	View the header (the metadata) for the NetCDF-3 file.	info	example
  .ncCF	Download a NetCDF-3 CF Discrete Sampling Geometries file (Contiguous Ragged Array).	info	example
  .ncCFMA	Download a NetCDF-3 CF Discrete Sampling Geometries file (Multidimensional Array).	info	example
  .odvTxt	Download longitude,latitude,time,otherColumns as an ODV Generic Spreadsheet File (.txt).	info	example
  .subset	View an HTML form which uses faceted search to simplify picking subsets of the data.	info	example
  .tsv	Download a tab-separated ASCII text table (line 1: names; line 2: units; ISO 8601 times).	info	example
  .tsvp	Download a .tsv file with line 1: name (units). Times are ISO 8601 strings.	info	example
  .xhtml	View an XHTML (XML) file with the data in a table. Times are ISO 8601 strings.	info	example

  
  For example, here is a complete request URL to download data formatted as an HTML table:
    http://coastwatch.pfeg.noaa.gov/erddap/tabledap/erdGlobecBottle.htmlTable?longitude,latitude,time,bottle_posn,temperature1&time>=2002-08-17T00:00:00Z&time<=2002-08-19T20:18:00Z

  ArcGIS .esriCsv - ArcGIS is a family of Geographical Information Systems (GIS) products from ESRI:
  ArcView, ArcEditor, and ArcInfo. To get data from ERDDAP into your ArcGIS program (version 9.x and below):

  1. In ERDDAP, save some data (which must include longitude and latitude) in an .esriCsv file
     (which will have the extension .csv) in the directory where you usually store ArcGIS data files.
     In the file:
     • Column names have been changed to be 10 characters or less
       (sometimes with A, B, C, ... at the end to make them unique).
     • longitude is renamed X. latitude is renamed Y to make them the default
       coordinate fields.
     • Missing numeric values are written as -9999.
     • Double quotes in strings are double quoted.
     • Timestamp columns are separated into date (ISO 8601) and time (am/pm) columns.
  2. In ArcMap, navigate to Tools : Add XY Data and select the file.
  3. Click on Edit to open the Spatial Reference Properties dialog box.
  4. Click on Select to select a coordinate system.
  5. Open the World folder and select WGS_1984.prj.
  6. Click a Add, Apply and OK on the Spatial Reference Properties dialog box.
  7. Click OK on the Add XY Data dialog box.
  8. Optional: save the data as a shapefile or into a geodatabase by right clicking
     on the data set and choosing Data : Export Data ...
  
  The points will be drawn as an Event theme in ArcMap.
  (These instructions are a modified version of ESRI's instructions.)

  Ferret is a free program for visualizing and analyzing large and complex
  gridded datasets. Because tabledap's tabular datasets are very different
  from gridded datasets, it is necessary to use Ferret in a very specific way to
  avoid serious problems and misunderstandings:

  1. With the ERDDAP web pages, download a subset of a dataset in a plain .nc file.
  2. Open that file in Ferret. The data isn't in an ideal format for Ferret, but
     you should be able to get it to work.
  
  WARNING: Ferret won't like profile data (and perhaps other data), because the time
  variable will have the same time value for all of the rows of data for a given
  profile. In fact, Ferret will add some number of seconds to the time values
  to make them all unique! So treat these time values accordingly.

  IDL - IDL is a commercial scientific data visualization program. To get data from
  ERDDAP into IDL, first use ERDDAP to select a subset of data and download a .nc file.
  Then, use these instructions to import the data from the .nc file into IDL.

  JSON .json files are widely used to transfer data to JavaScript scripts running on web pages.
  Tabledap will format the data in a table-like structure in the .json file.

  JSONP .json and .geoJson - Requests for .json and .geoJson files may include an
  optional jsonp request by adding &.jsonp=functionName to the end of the query.
  Basically, this just tells ERDDAP to add functionName( to the beginning of the response
  and ")" to the end of the response.
  If originally there was no query, leave off the "&" in your query.

  MATLAB .mat - Matlab users can use griddap's .mat file type to download data from within
  MATLAB. Here is a one line example:

  load(urlwrite('http://coastwatch.pfeg.noaa.gov/erddap/tabledap/erdGlobecBottle.mat?bottle_posn,temperature1&time=2002-08-19T10:06:00Z&.draw=lines', 'test.mat'));

  The data will be in a MATLAB structure. The structure's name will be the datasetID
  (for example, erdGlobecBottle).
  The structure's internal variables will be column vectors with the same names
  as in ERDDAP (for example, use fieldnames(erdGlobecBottle)).
  You can then make a scatterplot of any two columns. For example:

  plot(erdGlobecBottle.bottle_posn, erdGlobecBottle.temperature1)

  ERDDAP stores datetime values in .mat files as "seconds since 1970-01-01T00:00:00Z".
  To display one of these values as a String in Matlab, you can use, e.g.,
  datastr(cwwcNDBCMet.time(1)/86400 + 719529)
  86400 converts ERDDAP's "seconds since" to Matlab's "days since". 719529 converts
  ERDDAP's base time of "1970-01-01T00:00:00Z" to Matlab's "0000-01-00T00:00:00Z".

  NetCDF .nc - Requests for .nc files will always return the data in a table-like, version 3,
  32-bit, .nc file:

  • All variables will use the file's "row" dimension.
  • All String variables will also have a dimension indicating the maximum number of
    characters for that variable.

  Don't use NetCDF-Java, NetCDF-C, NetCDF-Fortran, NetCDF-Perl, or Ferret to try to
  access a remote ERDDAP .nc file. It won't work. Instead, use this approach.

  .ncHeader - Requests for .ncHeader files will return the header information (text) that
  would be generated if you used ncdump -h fileName on the corresponding .nc file.

  .ncCF - Requests for a .ncCF file will return a version 3, 32-bit, NetCDF .nc file with the
  Contiguous Ragged Array Representation associated with the dataset's cdm_data_type,
  as defined in the newly ratified CF Discrete Geometries conventions
  (which were previously named "CF Point Observation Conventions").

  • Point - Appendix H.1 Point Data
  • TimeSeries - Appendix H.2.4 Contiguous ragged array representation of timeSeries
  • Profile - Appendix H.3.4 Contiguous ragged array representation of profiles
  • Trajectory - Appendix H.4.3 Contiguous ragged array representation of trajectories
  • TimeSeriesProfile - Appendix H.5.3 Ragged array representation of timeSeriesProfiles
  • TrajectoryProfile - Appendix H.6.3 Ragged array representation of trajectoryProfiles
  
  A request will succeed only if the dataset has a cdm_data_type other than "Other"
  and if the request includes at least one data variable (not just the outer, descriptive variables).
  The file will include longitude, latitude, time, and other required descriptive variables, even if
  you don't request them.
  Since these are the newly ratified file formats, one can hope that they won't change again.

  .ncCFMA - Requests for a .ncCFMA file will return a verion 3, 32-bit, NetCDF .nc file
  with the Complete or Incomplete, depending on the data, Multidimensional Array Representation
  associated with the dataset's cdm_data_type, as defined in the CF Discrete Sampling Geometries
  conventions, which were previously named "CF Point Observation Conventions".
  This is the file type used by the NODC Templates.
  A request will succeed only if the dataset has a cdm_data_type other than "Other"
  and if the request includes at least one data variable (not just the outer, descriptive variables).
  The file will include longitude, latitude, time, and other required descriptive variables, even if
  you don't request them.

  NetCDF-Java, NetCDF-C, NetCDF-Fortran, and NetCDF-Perl -
  Don't try to access an ERDDAP tabledap dataset directly with a NetCDF library.
  It won't work, because these libraries just work with gridded data, not sequence data.
  Instead, by hand in a browser or with a program like curl, download a .nc file with a
  subset of the dataset. Then, open that local file with one of these libraries.
  With NetCDF-Java for example, use:
  NetcdfFile nc = NetcdfFile.open("c:\downloads\theDownloadedFile.nc");
  or
  NetcdfDataset nc = NetcdfDataset.openDataset("c:\downloads\theDownloadedFile.nc");
  (NetcdfFiles are a lower level approach than NetcdfDatasets. It is your choice.)
  In both cases, you can then do what you want with the nc object, for example,
  request metadata or request a subset of a variable's data as you would with any other
  .nc file.

  Ocean Data View .odvTxt - ODV users can download data in a ODV Generic Spreadsheet Format .txt file
  by requesting tabledap's .odvTxt fileType.
  The selected data MUST include longitude, latitude, and time variables.
  Any longitude values (0 to 360, or -180 to 180) are fine.
  After saving the resulting file (with the extension .txt) in your computer:

  1. Open ODV.
  2. Use File : Open.
     • Change Files of type to Data Files (*.txt *.csv *.jos *.o4x).
     • Browse to select the .txt file you created in ERDDAP and click on Open.
       The data should now be visible on a map in ODV.
  3. See ODV's Help menu for more help using ODV.

  OPeNDAP Libraries - Although ERDDAP is an OPeNDAP-compatible data server, you can't use
  most OPeNDAP client libraries, including NetCDF-Java, NetCDF-C, NetCDF-Fortran, NetCDF-Perl,
  or Ferret, to get data directly from an ERDDAP tabledap dataset because those libraries don't
  support the OPeNDAP Selection constraints that tabledap datasets use for requesting
  subsets of the dataset. (But see this other approach.)
  But you can use the Pydap Client or Java-DAP2, because they both support Selection
  constraints. With both the Pydap Client and Java-DAP2, when creating the initial
  connection to an ERDDAP table dataset, use the tabledap dataset's base URL, e.g.,
  http://coastwatch.pfeg.noaa.gov/erddap/tabledap/erdGlobecBottle

  • Don't include a file extension (e.g., .nc) at the end of the dataset's name.
  • Don't specify a variable or a Selection-constraint.
  
  Once you have made the connection to the dataset, you can request metadata or a
  subset of a variable's data via a Selection-constraint.

  Pydap Client users can access tabledap datasets via ERDDAP's standard OPeNDAP services.
  See the Pydap Client instructions for accessing sequential data.
  Note that the name of a dataset in tabledap will always be a single word, e.g., erdGlobecBottle
  in the OPeNDAP dataset URL
  http://coastwatch.pfeg.noaa.gov/erddap/tabledap/erdGlobecBottle
  and won't ever have a file extension (unlike, for example, .cdp for the sample dataset in the
  Pydap instructions). Also, the name of the sequence in tabledap datasets will always be "s"
  (unlike "location" for the sample dataset in the Pydap instructions).

  R Statistical Package - R is an open source statistical package for many operating systems.
  In R, you can download a .csv file from ERDDAP and then import data from that .csv file
  into an R structure (e.g., test). For example:

    download.file(url="http://coastwatch.pfeg.noaa.gov/erddap/tabledap/erdGlobecBottle.csv?longitude,latitude,time,bottle_posn,temperature1&time>=2002-08-17T00:00:00Z&time<=2002-08-19T20:18:00Z", destfile="/home/bsimons/test.csv")
    test<-read.csv(file="/home/bsimons/test.csv")

  Making an Image File with a Graph or Map of Tabular Data
  If a tabledap request URL specifies a subset of data which is suitable for making a graph or
  a map, and the fileType is an image fileType, tabledap will return an image with a graph or map.
  tabledap request URLs can include optional graphics commands which let you
  customize the graph or map.
  As with other tabledap request URLs, you can create these URLs by hand or have a computer
  program do it.
  Or, you can use the Make A Graph web pages, which simplify creating these URLs (see the
  "graph" links in the table of tabledap datasets).

  The fileType options for downloading images of graphs and maps of table data are:

  Image fileTypes	Description	Info	Example
  .kml	View a .kml file, suitable for Google Earth.	info	example
  .smallPdf	View a small .pdf image file with a graph or map.	info	example
  .pdf	View a standard, medium-sized .pdf image file with a graph or map.	info	example
  .largePdf	View a large .pdf image file with a graph or map.	info	example
  .smallPng	View a small .png image file with a graph or map.	info	example
  .png	View a standard, medium-sized .png image file with a graph or map.	info	example
  .largePng	View a large .png image file with a graph or map.	info	example
  .transparentPng	View a .png image file (just the data, without axes, landmask, or legend).	info	example

  Image Size - ".small" and ".large" were ERDDAP's original system for making
  different-sized images. Now, for .png and .transparentPng images (not other
  image file types), you can also use the &.size=width|height parameter to request
  an image of any size.

  .transparentPng - The .transparentPng file type will make a graph or map without
  the graph axes, landmask, or legend, and with a transparent (not opaque white)
  background. This file type can be used for any type of graph or map.
  For graphs and maps, the default size is 360x360 pixels.
  Or, you can use the &.size=width|height parameter to request an image of any size.

  Incompatibilities
  Some results file types have restrictions. For example, Google Earth .kml is only
  appropriate for results with longitude and latitude values. If a given request is
  incompatible with the requested file type, tabledap throws an error.

  Command Line Downloads with curl
  If you want to download a series of files from ERDDAP, you don't have to request each file's
  ERDDAP URL in your browser, sitting and waiting for each file to download.
  If you are comfortable writing computer programs (e.g., with C, Java, Python, Matlab, r)
  you can write a program with a loop that imports all of the desired data files.
  Or, if are comfortable with command line programs (just running a program, or using bash or tcsh
  scripts in Linux or Mac OS X, or batch files in Windows), you can use curl to save results files
  from ERDDAP into files on your hard drive, without using a browser or writing a computer program.
  ERDDAP+curl is amazingly powerful and allows you to use ERDDAP in many new ways.
  On Linux or Mac OS X, curl is probably already installed as /usr/bin/curl.
  On Windows, or if your computer doesn't have curl already, you need to download curl
  and install it. To get to a command line in Windows, use "Start : Run" and type in "cmd".
  ("Win32 - Generic, Win32, binary (without SSL)" worked for me on Windows XP and Windows 7.)
  Instructions for using curl are on the curl man page and in this curl tutorial.
  But here is a quick tutorial related to using curl with ERDDAP:

  • To download and save one file, use
    curl -g "erddapUrl" -o fileDir/fileName.ext
    where -g disables curl's globbing feature,
      erddapUrl is any ERDDAP URL that requests a data or image file, and
      -o fileDir/fileName.ext specifies the name for the file that will be created.
    For example,

    curl -g "http://coastwatch.pfeg.noaa.gov/erddap/tabledap/cwwcNDBCMet.png?time,atmp&time%3E=2010-09-03T00:00:00Z&time%3C=2010-09-06T00:00:00Z&station=%22TAML1%22&.draw=linesAndMarkers&.marker=5|5&.color=0x000000&.colorBar=|||||" -o NDBCatmpTAML1.png

    The erddapUrl must be percent encoded.
    If you get the URL from your browser's address textfield, this may be already done.
    If not, in practice, this can be very minimal percent encoding: all you usually
    have to do is convert % into %25, & into %26, " into %22, + into %2B,
    space into %20 (or +), < into %3C, > into %3E, ~ into %7E, and convert
    all characters above #126 to their %HH form (where HH is the 2-digit hex value).
    Unicode characters above #255 must be UTF-8 encoded and then each byte
    must be converted to %HH form (ask a programmer for help).
     
  • To download and save many files in one step, use curl with the globbing feature enabled:
    curl "erddapUrl" -o fileDir/fileName#1.ext
    Since the globbing feature treats the characters [, ], {, and } as special, you must also
    percent encode them in the erddapURL as %5B, %5D, %7B, %7D, respectively.
    Fortunately, these are rare in tabledap URLs.
    Then, in the erddapUrl, replace a zero-padded number (for example 01) with a range
    of values (for example, [01-05] ),
    or replace a substring (for example TAML1) with a list of values (for example,
    {TAML1,41009,46088} ).
    The #1 within the output fileName causes the current value of the range or list
    to be put into the output fileName.
    For example,

    curl "http://coastwatch.pfeg.noaa.gov/erddap/tabledap/cwwcNDBCMet.png?time,atmp&time%3E=2010-09-03T00:00:00Z&time%3C=2010-09-06T00:00:00Z&station=%22{TAML1,41009,46088}%22&.draw=linesAndMarkers&.marker=5|5&.color=0x000000&.colorBar=|||||" -o NDBCatmp#1.png

  
   
• query is the part of the request after the "?". It specifies the subset of data that you want to receive.
  In tabledap, it is an OPeNDAP DAP selection constraint query in the form: {resultsVariables}{constraints} .
  For example,
  longitude,latitude,time,bottle_posn,temperature1&time>=2002-08-17T00:00:00Z&time<=2002-08-19T20:18:00Z
  • resultsVariables is an optional comma-separated list of variables (for example,
    longitude,latitude,time,bottle_posn,temperature1).
    For each variable in resultsVariables, there will be a column in the
    results table, in the same order.
    If you don't specify any results variables, the results table will include
    columns for all of the variables in the dataset.
  • constraints is an optional list of constraints, each preceded by & (for example,
    &time>=2002-08-17T00:00:00Z&time<=2002-08-19T20:18:00Z).
    • The constraints determine which rows of data from the original table
      table are included in the results table.
      The constraints are applied to each row of the original table.
      If all the constraints evaluate to true for a given row,
      that row is included in the results table.
      Thus, "&" can be roughly interpreted as "and".
    • If you don't specify any constraints, all rows from the original table
      will be included in the results table.
      For the fileTypes that return information about the dataset (notably,
      .das, .dds, and .html), but don't return actual data, it is fine not
      to specify constraints. For example,
      http://coastwatch.pfeg.noaa.gov/erddap/tabledap/erdGlobecBottle.dds
      For the fileTypes that do return data, not specifying any constraints
      may result in a very large results table.
    • tabledap constraints are consistent with OPeNDAP DAP selection constraints,
      but with a few additional features.
    • Each constraint is in the form variableoperatorvalue
      (for example, latitude>45).
    • The valid operators are =, != (not equals), =~ (a regular expression test), <, <=, >, and >= .
    • tabledap extends the OPeNDAP standard to allow any operator to be used with any data type.
      (Standard OPeNDAP selections don't allow <, <=, >, or >= to be used with string variables
      and don't allow =~ to be used with numeric variables.)
      • For String variables, all operators act in a case-sensitive manner.
      • For String variables, queries for ="" and !="" should work correctly for almost all
        datasets.
      • For all constraints of String variables and for regex constraints of numeric variables,
        the right-hand-side value MUST be enclosed in double quotes (e.g., id="NDBC41201")
        and any internal special characters must be backslash encoded: \ into \\, " into \",
        newline into \n, and tab into \t.
      • For all constraints, the variableoperatorvalue MUST be percent encoded.
        If you submit the request via a browser, the browser will do the percent encoding
        for you. If you submit the request via a computer program, then the program
        needs to do the percent encoding.
        In practice, this can be very minimal percent encoding: all you usually
        have to do is convert % into %25, & into %26, " into %22, + into %2B,
        space into %20 (or +), < into %3C, > into %3E, ~ into %7E, and convert
        all characters above #126 to their %HH form (where HH is the 2-digit hex value).
        Unicode characters above #255 must be UTF-8 encoded and then each byte
        must be converted to %HH form (ask a programmer for help).
      • WARNING: Numeric queries involving =, !=, <=, or >= may not work as desired with
        floating point numbers. For example, a search for longitude=220.2 may
        fail if the value is stored as 220.20000000000001. This problem arises because
        floating point numbers are not represented exactly within computers.
        When ERDDAP performs these tests, it allows for minor variations and tries
        to avoid the problem. But it is possible that some datasets will still
        have problems with these queries and return unexpected and incorrect results.
      • NaN (Not-a-Number) - Many numeric variables have an attribute which identifies a
        number (e.g., -99) as a missing_value or a _FillValue. When ERDDAP tests
        constraints, it always treats these values as NaN's. So:
        Don't create constraints like temperature!=-99 .
        Do     create constraints like temperature!=NaN .
        • For numeric variables, tests of variable=NaN (Not-a-Number) and variable!=NaN
          will usually work as expected.
          WARNING: numeric queries with =NaN and !=NaN may not work as desired
          since many data sources don't offer native support for these queries and
          ERDDAP can't always work around this problem.
          For some datasets, queries with =NaN or !=NaN may fail with an error message
          (insufficient memory or timeout) or erroneously report
          Your query produced no matching results.
        • For numeric variables, tests of variable<NaN (Not-a-Number) (or <=, >, >=) will
          return false for any value of the variable, even NaN. NaN isn't a number so these
          tests are nonsensical.
      • variable=~"regularExpression" tests if the value from the variable on the left matches the
        regular expression on the right.
        • tabledap uses the same regular expression syntax (tutorial) as is used by Java.
        • WARNING - For numeric variables, the =~ test is performed on the String representation
          of the variable's source values. So if the variable's source uses scale_factor
          and/or add_offset metadata (which ERDDAP uses to unpack the data) or if the variable
          is an altitude variable that ERDDAP has converted from a depth value, the =~ test
          will be applied to the raw source values. This is not a good situation, but there
          is no easy way to deal with it. It is fine to try =~ tests of numeric variables, but
          but be very skeptical of the results.
        • WARNING: queries with =~ may not work as desired since many data sources
          don't offer native support for these queries and ERDDAP can't always work
          around this problem.
          ERDDAP sometimes has to get the entire dataset to do the test itself.
          For some datasets, queries with =~ may fail with an error message
          (insufficient memory or timeout) or erroneously report
          Your query produced no matching results.
          .
    • tabledap extends the OPeNDAP standard to allow constraints (selections) to be applied to any variable
      in the dataset. The constraint variables don't have to be included in the resultsVariables.
      (Standard OPeNDAP implementations usually only allow constraints to be applied
      to certain variables. The limitations vary with different implementations.)
    • Although tabledap extends the OPeNDAP selection standard (as noted above),
      these extensions (notably "=~") sometimes aren't practical because tabledap
      may need to download lots of extra data from the source (which takes time)
      in order to test the constraint.
    • tabledap always stores date/time values as numbers (in seconds since 1970-01-01T00:00:00Z).
      Here is an example of a query which includes date/time numbers:
      http://coastwatch.pfeg.noaa.gov/erddap/tabledap/erdGlobecBottle.htmlTable?longitude,latitude,time,bottle_posn,temperature1&time>=1029542400&time<=1029788280
      Some fileTypes (notably, .csv, .tsv, .htmlTable, .odvTxt, and .xhtml) display
      date/time values as ISO 8601:2004 "extended" date/time strings
      (e.g., 2002-08-03T12:30:00Z).
      ERDDAP has a utility to Convert a Numeric Time to/from a String Time.
      See also: How ERDDAP Deals with Time.
    • tabledap extends the OPeNDAP standard to allow you to specify time values in the ISO 8601
      date/time format (YYYY-MM-DDThh:mm:ssZ, where Z is 'Z' or a ±hh:mm offset from UTC).
      If you omit Z (or the ±hh:mm offset), :ssZ, :mm:ssZ, or Thh:mm:ssZ from the ISO date/time
      that you specify, the missing fields are assumed to be 0.
      Here is an example of a query which includes ISO date/time values:
      http://coastwatch.pfeg.noaa.gov/erddap/tabledap/erdGlobecBottle.htmlTable?longitude,latitude,time,bottle_posn,temperature1&time>=2002-08-17T00:00:00Z&time<=2002-08-19T20:18:00Z .
    • tabledap extends the OPeNDAP standard to allow you to specify constraints for
      time and timestamp variables relative to now. The constraint can be simply,
      for example, time<now, but usually the constraint is in the form
        now(+| |-)positiveInteger(second|seconds|minute|minutes|
        hour|hours|day|days|month|months|year|years)
      for example, now-7days.
      Months and years are interpreted as calendar months and years (not
      UDUNITS-like constant values).
      This feature is especially useful when creating a URL for an <img> (image)
      tag on a web page, since it allows you to specify that the image will always
      show, for example, the last 7 days worth of data (time>now-7days).
      Note that a '+' in the URL is percent-decoded as ' ', so you should
      percent-encode '+' as %2B. However, ERDDAP interprets "now " as "now+",
      so in practice you don't have to percent-encode it.
  • Server-side Functions - The OPeNDAP standard supports the idea of
    server-side functions, but doesn't define any.
    TableDAP supports some server-side functions that modify the results table before
    the results table is sent to you.
    Server-side functions are OPTIONAL. Other than 'distinct', they are rarely needed.
    Currently, each of the functions takes the results table as input and
    returns a table with the same columns in the same order (but the rows may be
    be altered). If you use more than one function, they will be applied in the order
    that they appear in the request. Most of these options appear near the bottom
    of the Data Access Form and Make A Graph web page for each dataset.
    • &distinct()
      If you add &distinct() to the end of a query, ERDDAP will sort all of the
      rows in the results table (starting with the first requested variable, then using the
      second requested variable if the first variable has a tie, ...), then remove all
      non-unique rows of data.
      For example, the query stationType,stationID&distinct() will return a
      sorted list of stationIDs associated with each stationType.
      In many situations, ERDDAP can return distinct() values quickly and efficiently.
      But in some cases, ERDDAP must look through all rows of the source dataset.
      If the data set isn't local, this may be VERY slow, may return only some of the
      results (without an error), or may throw an error.
    • &orderBy("comma-separated list of variable names")
      If you add this to the end of a query, ERDDAP will sort all of the rows in the results
      table (starting with the first variable, then using the second variable if the first
      variable has a tie, ...).
      Normally, the rows of data in the response table are in the order they arrived from
      the data source.
      orderBy allows you to request that the results table be sorted in a specific way.
      For example, use the query
      stationID,time,temperature&time>2009-04-01&orderBy("stationID,time")
      to get the results sorted by stationID, then time.
      Or use the query
      stationID,time,temperature&time>2009-04-01&orderBy("time,stationID")
      to get the results sorted by time first, then stationID.
      The orderBy variables MUST be included in the list of requested variables in the
      query as well as in the orderBy list of variables.
    • &orderByMax("comma-separated list of variable names")
      If you add this to the end of a query, ERDDAP will sort all of the rows in the results
      table (starting with the first variable, then using the second variable if the first
      variable has a tie, ...) and then just keeps the rows where the value of the last
      sort variable is highest (for each combination of other values).
      For example, use the query
      stationID,time,temperature&time>2009-05-21&orderBy("stationID,time")
      to get just the rows of data with each station's maximum time value (for stations
      with data from after 2009-05-21).
      (Without the time constraint, ERDDAP would have to look through all rows of the
      the dataset, which might be VERY slow.)
      The orderByMax variables MUST be included in the list of requested variables in the
      query as well as in the orderByMax list of variables.
      This is the closest thing in tabledap to griddap's allowing requests for the [last]
      axis value.
    • &units("value")
      If you add &units("UDUNITS") to the end of a query, the units will be described
      via the UDUNITS standard (for example, degrees_C).
      If you add &units("UCUM") to the end of a query, the units will be described
      via the UCUM standard (for example, Cel).
      On this ERDDAP, the default for most/all datasets is UDUNITS.
      See also ERDDAP's units converter.
  • Graphics Commands - tabledap extends the OPeNDAP standard by allowing graphics commands
    in the query.
    The Make A Graph web pages simplify the creation of URLs with these graphics commands
    (see the "graph" links in the table of tabledap datasets).
    So we recommend using the Make A Graph web pages to generate URLs, and then, when
    needed, using the information here to modify the URLs for special purposes.
    These commands are optional.
    If present, they must occur after the data request part of the query.
    These commands are used by tabledap if you request an image fileType (.png or .pdf)
    and are ignored if you request a data file (e.g., .asc).
    If relevant commands are not included in your request, tabledap uses the defaults and
    tries its best to generate a reasonable graph or map.
    All of the commands are in the form &.commandName=value .
    If the value has sub-values, they are separated by the '|' character.
    The commands are:
    • &.colorBar=palette|continuous|scale|min|max|nSections
      This specifies the settings for a color bar. The sub-values are:
      • palette - All ERDDAP installations support a standard set of palettes:
        BlackBlueWhite, BlackRedWhite, BlackWhite, BlueWhiteRed, LightRainbow,
        Ocean, OceanDepth, Rainbow, RedWhiteBlue, ReverseRainbow, Topography,
        WhiteBlack, WhiteBlueBlack, WhiteRedBlack.
        Some ERDDAP installations support additional options.
        See a Make A Graph web page for a complete list.
        The default (no value) varies based on min and max: if -1*min ~= max,
        the default is BlueWhiteRed; otherwise, the default is Rainbow.
      • continuous - must be either no value (the default), 'C' (for Continuous),
        or 'D' (for Discrete). The default is different for different datasets.
      • scale - must be either no value (the default), Linear, or Log.
        The default is different for different datasets.
      • min - The minimum value for the color bar.
        The default is different for different datasets.
      • max - The maximum value for the color bar.
        The default is different for different datasets.
      • nSections - The preferred number of sections (for Log color bars,
        this is a minimum value). The default is different for different datasets.
      If you don't specify one of the sub-values, the default for the sub-value will be used.
    • &.color=value
      This specifies the color for data lines, markers, vectors, etc. The value must
      be specified as an 0xRRGGBB value (e.g., 0xFF0000 is red, 0x00FF00 is green).
      The default is 0x000000 (black).
    • &.draw=value
      This specifies how the data will be drawn, as lines,
      linesAndMarkers, markers (default), sticks, or vectors.
    • &.font=scaleFactor
      This specifies a scale factor for the font
      (e.g., 1.5 would make the font 1.5 times as big as normal).
    • &.land=value
      This specifies whether the landmask should be drawn under or over the
      topography data.
      The default is different for different datasets (under the ERDDAP administrator's
      control via a drawLandMask in datasets.xml, or via the fallback drawLandMask
      setting in setup.xml).
      This is only relevant if the first two results variables are longitude and latitude.
      Terrestrial researchers usually prefer under (showing topography).
      Oceanographers often prefer over (showing just bathymetry).
    • &.legend=value
      This specifies whether the legend on .png images (not .pdf's) should be at the
      Bottom (default), Off, or Only (which returns only the legend).
    • &.marker=markerType|markerSize
      markerType is an integer: 0=None, 1=Plus, 2=X, 3=Dot, 4=Square,
      5=Filled Square (default), 6=Circle, 7=Filled Circle, 8=Up Triangle,
      9=Filled Up Triangle.
      markerSize is an integer from 3 to 50 (default=5)
    • &.size=width|height
      For .png images (not .pdf's), this specifies the desired size of the image, in pixels.
      This allows you to specify sizes other than the predefined sizes of .smallPng, .png, and
      .largePng.
    • &.trim=trimPixels
      For .png images (not .pdf's), this tells ERDDAP to make the image shorter by removing
      all whitespace at the bottom, except for trimPixels.
    • There is no &.vars= command. Instead, the results variables from the main part
      of the query are used.
      The meaning associated with each position varies with the &.draw value:
      • for lines: xAxis,yAxis
      • for linesAndMarkers: xAxis,yAxis,Color
      • for markers: xAxis,yAxis,Color
      • for sticks: xAxis,uComponent,vComponent
      • for vectors: xAxis,yAxis,uComponent,vComponent
      If xAxis=longitude and yAxis=latitude, you get a map; otherwise, you get a graph.
    • &.vec=value
      This specifies the data vector length (in data units) to be scaled to the size
      of the sample vector in the legend. The default varies based on the data.
    • &.xRange=min|max
      This specifies the min|max for the X axis. The default varies based on the data.
    • &.yRange=min|max
      This specifies the min|max for the Y axis. The default varies based on the data.
    
    A sample graph URL is
    http://coastwatch.pfeg.noaa.gov/erddap/tabledap/erdGlobecBottle.png?bottle_posn,temperature1&time=2002-08-19T10:06:00Z&.draw=lines

    Or, if you change the fileType in the URL from .png to .graph,
    you can see a Make A Graph web page with that request loaded:
    http://coastwatch.pfeg.noaa.gov/erddap/tabledap/erdGlobecBottle.graph?bottle_posn,temperature1&time=2002-08-19T10:06:00Z&.draw=lines
    That makes it easy for humans to modify an image request to make a
    similar graph or map.

    Or, if you change the fileType in the URL from .png to a data fileType
    (e.g., .htmlTable), you can download the data that was graphed:
    http://coastwatch.pfeg.noaa.gov/erddap/tabledap/erdGlobecBottle.htmlTable?bottle_posn,temperature1&time=2002-08-19T10:06:00Z&.draw=lines

    A sample map URL is
    http://coastwatch.pfeg.noaa.gov/erddap/tabledap/erdGlobecBottle.png?longitude,latitude,temperature1,time,bottle_posn&time>=2002-08-13T00:00:00Z&time<=2002-08-20T00:00:00Z&bottle_posn=1&.draw=markers&.marker=5|5

    Or, if you change the fileType in the URL from .png to .graph,
    you can see a Make A Graph web page with that request loaded:
    http://coastwatch.pfeg.noaa.gov/erddap/tabledap/erdGlobecBottle.graph?longitude,latitude,temperature1,time,bottle_posn&time>=2002-08-13T00:00:00Z&time<=2002-08-20T00:00:00Z&bottle_posn=1&.draw=markers&.marker=5|5

    Or, if you change the fileType in the URL from .png to a data fileType
    (e.g., .htmlTable), you can download the data that was mapped:
    http://coastwatch.pfeg.noaa.gov/erddap/tabledap/erdGlobecBottle.htmlTable?longitude,latitude,temperature1,time,bottle_posn&time>=2002-08-13T00:00:00Z&time<=2002-08-20T00:00:00Z&bottle_posn=1&.draw=markers&.marker=5|5

• Data Model - Each tabledap dataset can be represented as a table with one
  or more rows and one or more columns of data.
  • Each column is also known as a "data variable" (or just a "variable").
    Each data variable has data of one specific type.
    The supported types are (int8, uint16, int16, int32, int64, float32, float64, and
    String of any length).
    Any cell in the table may have a no value (i.e., a missing value).
    Each data variable has a name composed of a letter (A-Z, a-z) and then 0 or more
    characters (A-Z, a-z, 0-9, _).
    Each data variable has metadata which is a set of Key=Value pairs.
  • Each dataset has global metadata which is a set of Key=Value pairs.
  • Note about metadata: each variable's metadata and the global metadata is a set of 0
    or more Key=Value pairs.
    Each Key is a String consisting of a letter (A-Z, a-z) and then 0 or more other
    characters (A-Z, a-z, 0-9, '_').
    Each Value is either one or more numbers (of one Java type), or one or more Strings
    of any length (using \n as the separator).
     
• Special Variables
  • In tabledap, a longitude variable (if present) always has the name "longitude"
    and the units "degrees_east".
  • In tabledap, a latitude variable (if present) always has the name "latitude"
    and the units "degrees_north".
  • In tabledap, an altitude variable (if present) always has the name "altitude"
    and the units "m" above sea level.
    Locations below sea level have negative altitude values.
  • In tabledap, a time variable (if present) always has the name "time" and
    the units "seconds since 1970-01-01T00:00:00Z".
    If you request data and specify a time constraint, you can specify the time as
    a number (seconds since 1970-01-01T00:00:00Z) or as a
    String value (e.g., "2002-12-25T07:00:00Z" in the UTC/GMT/Zulu time zone).
  • In tabledap, other variables can be timeStamp variables, which act like the time
    variable but have a different name.
  • Because the longitude, latitude, altitude, and time variables are specifically
    recognized, ERDDAP is aware of the spatiotemporal features of each dataset.
    This is useful when making images with maps or time-series, and when saving
    data in geo-referenced file types (e.g., .geoJson and .kml).
     
• Incompatibilities
  • Constraints - Different types of data sources support different types of constraints.
    Most data sources don't support all of the types of constraints that tabledap
    advertises. For example, most data sources can't test variable=~"RegularExpression".
    When a data source doesn't support a given type of constraint, tabledap
    gets extra data from the source, then does that constraint test itself.
    Sometimes, getting the extra data takes a lot of time or causes the remote
    server to throw an error.
  • File Types - Some results file types have restrictions.
    For example, .kml is only appropriate for results with longitude and latitude
    values. If a given request is incompatible with the requested file type,
    tabledap throws an error.
     
• Requesting Compressed Files
  ERDDAP doesn't offer results stored in compressed (e.g., .zip or .gzip) files.
  Instead, ERDDAP looks for accept-encoding in the HTTP GET request header sent
  by the client. If a supported compression type ("gzip", "x-gzip", or "deflate") is found
  in the accept-encoding list, ERDDAP includes "content-encoding" in the HTTP response
  header and compresses the data as it transmits it.
  It is up to the client program to look for "content-encoding" and decompress the data.
  Browsers and OPeNDAP clients do this by default. They request compressed data and
  decompress the returned data automatically.
  Other clients (e.g., Java programs) have to do this explicitly.