2.7.1 What is DODS?

DODS is now called OPeNDAP; we continue to refer to it as DODS in this manual for now. DODS, the Distributed Oceanographic Data System, allows users to access data anywhere from the Internet using a variety of client/server methods, including Ferret. Employing technology similar to that used by the World Wide Web, DODS and Ferret create a powerful tool for the retrieval, sampling, analyzing and displaying of datasets; regardless of size or data format (though there are data format limitations).

For more information, please see the OPeNDAPhome page at

http://www.opendap.org/

Similar to the WWW, DODS is an emerging technology and is under development. As a result, it is likely that the details with which things are accomplished will be changing.

2.7.2 Accessing Remote Data Sets

Datasets are accessed through Ferret using their raw Universal Resource Locator (URL) address. Make sure to enclose the address in quotes, as for any dataset name that includes a path. For example, to access the COADS climatology, hosted at PMEL:

yes? use "http://ferret.pmel.noaa.gov/thredds/dodsC/data/PMEL/coads_climatology.nc"

Once the dataset has been initialized, it is used just like any other local dataset.

yes? list/x=140w/y=2n/t="16-Feb" sst
             VARIABLE : SEA SURFACE TEMPERATURE (Deg C)
             FILENAME : coads_climatology.nc
             FILEPATH : http://ferret.pmel.noaa.gov/thredds/dodsC/data/PMEL/
             LONGITUDE: 141W
             LATITUDE : 1N
             TIME     : 15-FEB 16:29
          26.39

To locate DODS data, you can search the NVODS /DODS List of DODS datasets at http://www.opendap.org/data/datasets.cgi?&exfunction=none&xmlfilename=datasets.xml or the Global Change Master Directory at http://gcmd.gsfc.nasa.gov/

For the time being, netCDF and HDF files can be read via DODS by Ferret. As DODS (OPeNDAP) netCDF libraries become available, other data types will be made available.

Note that HDF files can be read by Ferret only via DODS, that is the HDF file must first be put on a DODS server and then Ferret can access it by giving its DODS URL. Even by this means, Ferret will be successful in reading the file only if the HDF file is similar in its structure to a COARDS or CF netCDF file. Often, you will need to apply the USE/ORDER= qualifier to change the ordering of the coordinate axes.

If a file is on a DODS server, you can look at the DAS in your browser (the URL that ends in .das). When you look at the attribute data check to see if there are dimension variables with attributes that look like a Latitude or Longitude as in the COARDS conventions.

2.7.3 Debugging Access to Remote DODS Data Sets

To find out more information about a particular dataset, or to debug problems, there are three elements of the dataset which may be accessed via a web browser. To access this information, merely append a dds, das, or info to the dataset name. For example:

http://ferret.pmel.noaa.gov/thredds/dodsC/data/PMEL/coads_climatology.nc.dds

DDS stands for Data Description Structure and this will return a text description of the data sets structure.

http://ferret.pmel.noaa.gov/thredds/dodsC/data/PMEL/coads_climatology.nc.das

DAS stands for Dataset Attribute Structure and this will return a text description of attributes assigned to the variables in the data set.

http://ferret.pmel.noaa.gov/thredds/dodsC/data/PMEL/coads_climatology.nc.info

This will return a text description of the variables in the dataset.

2.7.4 Security

Some DODS data providers will choose to control access to some or all of their data. When you request data from one of these servers, the DODS client will prompt you for a username and password. If you want to avoid the prompt, you can embed a username and password in it, like this:

http://user:password@www.dods.org/nph-dods/etc...

2.7.5 Sharing Data Sets via DODS

One of the most powerful aspect of DODS is the ease with which it allows for the sharing of data. With just a few simple steps, anyone running a web server can also be a DODS data server, thereby allowing data set access to anyone with an Internet connection.

Simply copying a few precompiled binaries into the cgi-bin directory of an already configure httpd server is all it takes to become a DODS server. Once the server is configured, adding or removing data sets is as simple as copying them to the server data directory or deleting them from that directory.

This ability has such immense potential that it bears extra emphasis. Imagine that within seconds of finishing a model run, a remote colleague is able to look at your dataset with whatever DODS client he/she desires, be it Ferret, or Matlab, etc. No need for you to package up the data or for your colleague to download and/or reformat it, it is ready to be analyzed right away.

2.7.6 DODS caching

This feature allows caching of frequently accessed DODS served datasets to produce a quicker response when requesting remote data. The first time you access a DODS data set, a file in the users home directory will be created called .dodsrc. This file is the DODS client initialization file. Please see the DODS Users Guide; http://www.opendap.org/user/guide-html/guide.html for details of the paramaters that this file contains. Initially, DODS caching will be turned off. In order to turn caching on, change the line in the newly created ~/.dodsrc file from

USE_CACHE=0
to
USE_CACHE=1

Note that if you edit the .dodsrc file, make sure that the lines within it all start in the first column.

The next time Ferret is run, and a DODS-served dataset is accessed, a file called .dods_cache will be created, typically in the users home directory. The location of the DODS cache directory can be controlled by the line

CACHE_ROOT=/home/twaits/.dods_cache/

in the user's .dodsrc file. This directory is where all the cached information is stored. To clear the DODS cache, simply delete the .dods_cache directory and all of it's contents (for example, rm -rf ~/.dods_cache). This directory will be recreated and repopulated with caching information the next time data is accessed via DODS, if caching is turned on. All of the paramater values in the .dodsrc file can be modified to better suit individual needs, and will be incorporated the next time Ferret is run and DODS served data is accessed. Again, see the DODS User guide at see the section "The OPeNDAP Client Initialization File (.dodsrc)" in the DODS Users Guide (http://www.opendap.org/user/guide-html/guide.html) for more detailed information

It is often a useful diagnostic exercise to turn caching off and/or clear out the cache directory when attempts to access datasets in Ferret appear inconsistent. For example, if Ferret attempted to access a DODS-served dataset that was unavailable because the DODS server was down, that information may get cached and adversely effect the next attempt at retrieving the data.

For more detailed information on using OPeNDAP, and on setting up an OPeNDAPserver, see the OPeNDAP pages http://www.opendap.org

2.7.7 Proxy servers

A DODS client can negotiate proxy servers, with help from directions in its configuration file. The parameters that control proxy behavior are fully documented in the DODS Users Guide, see the link above.