Welcome to the DataPool WebAccess user interface help document. The intent of this document is to give you some quick feedback on how to search for data in the ECS DataPool and to answer questions you may have on how this site works. The DataPool is an on-line cache of frequently requested earth science data from the EOSDIS Core System (ECS) archives. For more information regarding background and purpose of DataPool in general, please see the ‘About DataPool' documentation. The intent of the WebAccess site is to allow users to quickly drill-down through the DataPool database to find data of interest. This site does not require plugins of any kind, but we do use some java and some javascript so it is useful to have those things turned on if your browser supports them. Other than that, there are no major technical requirements to use this site. To begin, select either Data Group, Theme or Data Set from the drop-down menu on the home page and click ‘Start Search’. Note that some sites may configure the home page to have static links to begin the search instead)

Drilling-down (or Searching)

The WebAccess user interface employs a 'drill-down' approach to allow for quick and easy, one-click searching. In general, the values of each drill-down parameter are presented as web links. To make a choice, simply click on the link. With each choice, you are reducing the number of granules that will be returned, effectively burrowing down into the database. Hence the term 'drill-down'. The more complex parameters (those that can have ranges for example) are usually supported by a non-click method of selection as well (date selection form fields and spatial applet are good examples).

Once you make a selection, the next drill-down parameter is displayed on a new page. You can make selections until you run out of parameters or decide to initiate a search for granules. A granule is the most basic unit of ECS data. At its core, a granule is the science data for which you're looking. However, in ECS parlance, the term granule is inclusive of other pieces of data that can be associated with the science data. These include browse data (sub-sampled science data) and metadata. These are all part of a granule (for more on this, see the related section in the glossary). DataPool allows access to the science data (in HDF file format), the browse data (jpg only) and the full granule metadata (xml format).

In general, your options on each drill-down page are to select a value for the current drill-down parameter, skip the current parameter (i.e., don't make a selection for this parameter and move on to one of the remaining parameters), get the granules associated with the currently selected search criteria, or start a new search. Exceptions are the Data Group (or Theme) and Data Set drill-down parameters for which you must make a selection in order to initiate a search.

Page (or screen) layout for the drill-down is generally split up into four parts: a table of the current search criteria, summary statistics, options for what to do next, and a user interface supporting the selection of the next drill-down parameter value. Each section is explained below:

  1. The current search criteria table shows the parameters which have been selected thus far and their values. Note: you can click on the linked name of the parameter to return to the drill-down page for that parameter to make a new selection, thereby following a new drill-down path to different data.
  2. Summary statistics describe the set of granules that would result if you issued the current search without selecting any additional search criteria. The summary includes a count of the granules and the total volume of selected granules measured in megabytes.
  3. The options section describes your options for continuing with the drill-down from the current point. Up to four continuation options will be listed depending on their relevance to the current point in the drill-down. The four options are: select a value or values for the current drill-down parameter and continue the drill-down with the next parameter, skip selection of a value for the current drill-down parameter (thus effectively selecting no values for that parameter) and continue the drill-down with the next parameter (which is selectable), end the drill-down at the current point and get the granules that match the current selection of drill-down parameters and values, or start a new search (this will remove all references to your current search).
  4. The user interface for selecting values for the drill-down parameters varies by whether the parameter is 'simple' or 'complex'. Simple parameters (i.e., those with a finite list of valid values) are presented in a table where each row represents a parameter value and has three columns: a link to a value of the parameter, a description of that parameter value and a count of the granules that would match both that parameter value and the existing search criteria. User interfaces for complex parameters are described individually in the following section. There are five complex parameters in this release of DataPool: Skip drill parameter link group date, time, spatial area-of-interest, cloud cover and science quality. end of drill parameter link group

The following parameters are available for searching:

The order of drill-down parameters above is presented alphabetically and does not mean to convey a default ordering of drill-down parameters. The as-delivered, default order of presentation is either Data Group, Theme or Data Set (each of these is initially displayed on the home page) followed by Date, Time, Spatial, Cloud Cover, Day/Night Flag and Science Quality Flag. This default order (including the removal of parameters) may be changed by the hosting site as a matter of operational configuration. You may also change this order via the ‘Set Search Order’ page. For more information on selecting your own search order, please see the section on ‘Search Order Selection’. Also note that it is very easy to skip parameters in which you are not interested. Please see the section on ‘Skipping Selections’ for more information on this topic. Below are basic descriptions of each parameter:

Spatial coverage is NOT a mandatory parameter. You may select it, skip it, get the granules given the existing criteria or start a new search.

Please note that the drill-down parameter that is configured to be last will not offer a skip option (since it is the last parameter listed in the drill-down sequence). For more on this, see the ‘Skip Options’ section below.

 

Search Order Selection

The nature of the drill-down user interface requires that parameters be presented one at a time. However, the order of presentation is configurable. To configure the order of presentation, click on the ‘set the order’ link displayed on the Webaccess home page. Clicking this link will display the ‘Set Search Order’ page. This page contains a set of option menus that allow you to define the order of presentation with the following restrictions:

  1. You must choose Data Group, Theme or Data Set to be the first parameter in your customized sequence. No other parameters are allowed to be the initial selection.
  2. If you choose 'Data Group' as the first parameter, Data Set must be the second.
  3. Data Set must be present in the drill-down sequence in order for 'Science Quality Flag' to be included.

For example, you may configure the following order: Data Group -> Data Set -> Spatial -> Cloud Cover (and 'None' for parameters 5 - 8). In this instance, Cloud Cover would be the last selectable parameter and would be followed by the results page once a selection had been made. Note that Date, Time and Science Quality Flag have all been left out (Theme was excluded as it is only valid as a starting parameter). Once you have created the search order you want, select 'Set'. 'Reset' will re-instate the default order (i.e., Data Group, Data Set, Date, Time of Day, Spatial, Cloud Cover, Day/Night Flag, Science QA). Navigate back to the home page (‘Go Back to Home Page’) to start the search.

Please note that selecting a parameter for display in the drill-down sequence will not always guarantee its appearance as the display of parameters is context-sensitive depending on data set. For example, if a data set has a global spatial distribution (i.e., its granules have spatial extents that are global in nature), the spatial page will not be displayed for that data set (because a spatial search would not reduce the number of granules returned). Cloud Cover and Day/Night Flag are other examples of parameters that are data set dependent (i.e., if site operations has configured the data sets to be defined as requiring Day/Night Flag or having valid cloud cover, these pages will be available in a drill-down sequence assuming that you have either kept the default sequence in place or configured these parameters for display).

 

Skip Options (also known as 'Flexible Drill-down')

Each drill-down page (with the exception of Data Group, Theme and Data Set and the page for the 'last' drill-down parameter) supports the option to skip to another available page without selecting a value from the current page. A 'next' page from the default drill-down order will be displayed automatically on the skip option menu. However, if you wish to jump to a different page, simply make a selection from the skip option menu and click 'Go'. Note that only parameters which have not yet been selected are offered as skip choices. Additionally, note that if you choose a skip option other than the default 'next' option, the skip option menus on subsequent drill-down pages will only display parameter options 'below' the choosen skip option in the currently set drill-down order (i.e., you cannot use the skip option to change the drill-down order).

 

Using the Spatial Applet

This is a quick reference for the functions of the spatial coverage applet. There are seven functional groups of controls and/or displays within the applet: Coordinates, Layers, Density Map, Grid Size, Zoom, and Map Interaction and the selection buttons.. Each functional group is discussed in the sections below:

Please note that the applet make take several seconds to start up even if the java virtual machine (JVM) has been previously run in your browser. The applet should run correctly on any browser that has a Java 1.1.8 or better JVM installed.

 

Viewing Results

A summary of your results is displayed above the results table. The summary captures notable data set level information such as average granule size, source parameter for cloud cover, product quality summary link or whether the data for a data set is typically compressed. The summary contents are as follows:

The granules that match your search criteria are displayed in the results granule table. The results table is organized such that there is one granule displayed per row. The column organization of the results table is as follows:

The results table header and footer are primarily concerned with results table navigation and contain the following:

 

Downloading Science Files

The first four icons(download granule icon, download granule using zip compression (*.zip) icon, download granule using gnu zip compression (*.gz) icon and  download granule using Unix compression (*.Z) icon) in the 'Granule' column of the results table are the hdf or science file icons. The last three of these four icons are for downloading the science files in one of the available compression formats. To ftp the granule's science files to your machine, simply click on any of these icons. Again, note that these icons, like the others, are optional: they will only be displayed if the science files are present for the corresponding granule in your results set.

If there is only a single science file for your granule, clicking on the granule link or any of the four downloading icons should result in a 'Save As...' dialog being popped up. If the granule has multiple science files associated with it, when you click on any of the downloading (ftp) links or icons, a dialog containing links for the multiple science files will pop up. You may either click on the individual links in the dialog to get the science files separately or click on the single tar file link to get them all in a single tar file. For granules having science files stored in a compressed format, the option for additional compression upon download will not be offered. Hence, for granules having single science file, only one downloading icon is displayed in the result table for granules with compressed science files. Also, in the pop up for multiple science files, only one icon is displayed to download the science files separately for granules with compressed science files. Please see the Compressed Data section for more details on granules with compressed science files.

Note that each of the file links displayed for downloading files is an ftp link. It is possible that the browser will not understand the file type referenced in this link, and will therefore, try to render it as text. In order to stop this from occurring, you have the choices listed below:

 

Viewing Granule Metadata

The next icon ( metadata viewer icon ) shown in the 'Granule' column of the results table is the icon for displaying metadata. To view the full metadata for a granule simply click on this icon. This will pop up a new browser window showing the full metadata hierarchy for the granule. To see the complete metadata hierarchy, scroll down or expand the popup window (or both). Navigation links in the popup window let you navigate through the metadata hierarchies for all the granules in the results set if you like (just click on the 'Next' or 'Previous' links depending on whether you'd like to see the next granule or the previous one; the same applies to 'First' and 'Last' links for the first or last granules). To save the xml file to your local disk, click on the 'Save Metadata File' icon. A 'Save As...' dialog should popup and prompt you for a file name and location. Note: the same problem described for downloading science files described above can occur when trying to save the xml file. Please see the section above for problem description and solutions.

You may also access the metadata by clicking the combined browse/metadata icon (metadata and browse viewer icon). The combined viewer shows both metadata and browse information on the same page.

 

Viewing Browse Imagery

Note: ECS Browse files which do not contain jpg images will not be available in DataPool. If a granule has an associated jpg browse available in DataPool, a browse icon ( browse viewer icon ) will be displayed in the 'Granule' cell in the results table. Simply click on this icon to pop up the browse viewer. Note that for granules with multiple browse images, the images are currently shown in a scrollable area with the browse viewer (you've got to scroll down to see all of the browse images in this case). If you want to save any of the images, just right-click on the image. A popup menu will appear and one of the options will be 'Save As...' or 'Save Image As...', etc. When you select this option, you'll get a popup dialog asking you where you'd like to save the file and under what name.

You may also access browse imagery by clicking the combined browse/metadata icon (metadata and browse viewer icon). The combined viewer shows both browse and metadata information on the same page.

 

Converting Data (Re-Formatting, Re-Projection and Subsetting)

DataPool WebAccess provides the option to reformat, reproject or subset granules using an HDF-EOS to GeoTIFF (HEG) conversion tool. This tool, the HDF-EOS-to-GeoTIFF (HEG) Converter converts data adhering to the HDF-EOS standard. In order to make conversion requests, convertible granules must first be added to the shopping cart. Granules that are convertible will have an accompanying converter icon () displayed in the 'Granules' column of the result table. Conversion parameters are as follows:

Conversion of granules is achieved by adding granules to the shopping cart ( add granule to shopping cart icon ). Conversion parameters may be selected from the shopping cart page. Note that because conversion requests typically take a significant amount of time (minutes to hours), all conversion requests in DataPool are worked off-line, or as orders (though there is no charge for converted data). As such, you will receive e-mail from us when the conversions are complete.

Conversion parameters are as follows:

In the shopping cart page, you can apply the format selection to all the applicable granules in your cart by clicking the icon (apply all icon) next to the format drop-down menu list.

 

Ordering Data

The simplest way to acquire data using DataPool Webaccess is to download it directly using the anonymous ftp capability (accessed by simply clicking on one of the file download icons for a granule contained within the results table). However, Webaccess also supports the conversion of data and the ordering of data on different forms of media. Conversion of data can take significant amounts of time (minutes or hours) as can placing your data onto various forms of media. For this reason, DataPool WebAccess will take your order for converted data and/or data on media and send an e-mail notification when the data is ready. Note that converted data may be ordered on physical media (it is not restricted to FTP download). The DataPool WebAccess order process begins in the shopping cart and entails the following:

  1. Once you have all the granules you want in the shopping cart, select a media type using the ‘Select Media’ button. This assumes that you have not already selected a media type. When you first add data to the shopping cart, Webaccess will prompt you to select a media type. You may select a type at that time or wait until you start the order process, in which case, ‘Select Media’ will be the next available option (otherwise, ‘Checkout’ will be displayed).
  2. Once you've selected a media type, click on the 'Checkout' button and then enter your profile information on the checkout page. If you've selected 'download' as your media type, your profile consists primarily of your e-mail address (mandatory). If you've opted to get your data on a physical media type, your profile consists of a shipping address (mandatory) and contact address (optional). Enter the information as required and click ‘Order the data’.
  3. DataPool sends you an e-mail acknowledgment which contains the specification and status of your order. This acknowledgment or order status page may be bookmarked and reloaded to get the updated status of the order.
  4. Once the data is actually ready (could be anywhere from minutes to days depending on what you have asked the system to do), DataPool sends you an Order Notification. If you opted to download your data via ftp, the order notification will contain links to your data. At that point, it's your responsibility to download the converted data via the links. It will be removed after the amount of time indicated in the e-mail. If you selected physical media, the order notification lets you know the data is being mailed to you at the shipping address you specified. If you opted to have your data uploaded to you, the notification will indicate when and where this operation was performed.

Note: it is possible that, due to maintenance on a data set, orders for certain granules may be temporarily prohibited. Please see the section on Data Set Maintenance for more details on data set availability.

 

Using the Shopping Cart

The shopping cart is used to select multiple granules for simple downloading or conversion and ordering. You may even add granules from different results sets. Granules are added from the results page to the shopping cart. From the results page there are several ways to move granules into the shopping cart:

Moving granules into the shopping cart may result in media warnings if the granule size, the number of granules, or the total data size of the granules intended for the shopping cart violates the maximum physical capacity, granule count, or data size of any media type, respectively. Please see the Media section for more details on distribution media.

There is a 1000 granule limit on the size of a shopping cart. After the 1000 granule limit is reached, granules must be removed from the shopping cart or an order must be placed, thereby emptying the shopping cart, before additional granules can be added to the shopping cart. Removing granules from the shopping cart may result in media warnings if minimum data size limits are violated. Additionally, there is a limit on the number of granules having associated conversion requests. This limit is configurable by the DAAC. Please see the Media section for more details on distribution media.

Note: The shopping cart is persistent for only as long as the session is active and the shopping cart cannot be bookmarked.

The shopping cart is organized in two sections: data set summary and granule table.

The data set summary captures the notable data set level information matching the granules in your shopping cart and is displayed above the granule table.. The summary contents are as follows:

The granule table section itself is comprised of a header and footer containing controls allowing you to navigate the table data and a listing of the granules currently in the cart. These navigation controls are the same as those listed for the results table on the results page.

The granule table header and footer contain the following controls that support navigation:

Granule table columns are the following:

Distribution Media

The specifications of DataPool data distribution options are displayed in a media options table. The media options table is organized such that there is one media type displayed per row. The column organization of the media options table is as follows:

Currently, there are 6 data distribution options available for DataPool data - CDROM, DLT, DVD, 8MM, and the option to download or upload data (via FTP). Note that 'File Format' is not really a valid consideration for FTP download as you simply download the files directly as you like. Compression options are available however, and will be presented to you when the order is ready. Typical compression or packaging options are tar, tar.Z, tar.gz and zip.

The media page may be presented the first time you add data to the shopping cart depending on the quantity of data added. The media page may also be the first page presented in the order process once you've finished adding data to the shopping cart. If the media page is displayed prior to ordering data, you may choose to select media then or wait until you have finished adding data to the shopping cart. If you do opt to select a media type the first time you add data to the shopping cart, the 'Checkout' button will be displayed at the bottom of the shopping cart page. Click 'Checkout' to begin the order process. If you haven't made a media type selection when you display the shopping cart, a 'Select Media' button will appear at the bottom of the shopping cart page. Click 'Select Media' to begin the order process. Once a media selection is made, click the 'OK' button to go to the next step, which is checkout. Please see the Checkout section for more details on the order checkout process. Or, if you are dissatisfied with your media selection you can cancel it by clicking the 'Cancel' button. In addition, clicking the ‘Go Back to Cart’ or ‘Go Back to Result Page’ buttons will take you to the shopping cart or result pages, respectively. Your media selection will be valid until the session times out or until you change it.

The media page will display media warnings and error messages as necessary to indicate whether a media type is no longer valid (and therefore made unavailable) due to the size of the current shopping cart contents. Errors may be displayed upon order submission to address any conflicts due to updates in media type specifications (media limits are configurable by the DAAC operators and are dynamically acquired by DataPool at the point of order submission) or the automatic removal of unavailable granules from an order (granules may be unavailable due to Data Set Maintenance). Warnings may be displayed during the addition or removal of granules to the shopping cart to address distribution media limitations.

To turn off the media warnings displayed during shopping cart content modifications check the "Don't show this media warning page again" box in the media page.

 

Order Profile/Checkout

Profile information is presented during the final step in the order submission process and is contained within the 'Checkout' page. The profile is configured depending on whether you are downloading or uploading your data via ftp or ordering your data on physical media. Additionally, the profile presented for a download is configured depending on whether you are ordering data with conversion requests or doing a simple download (i.e., no conversion processing) of your data directly from DataPool (this type of order bypasses the Order Management Subsystem). The profile configurations and their fields follow.

Download without conversion processing:

Download with conversion processing:

Upload with conversion processing:

Upload has the same fields as Download with conversion processing and adds a set of fields called 'Upload Parameters'.

Physical media (e.g., CD-ROM, DVD, DLT, etc.):

Physical media does not show or require any Upload information, but it adds full contact and shipping information as described below. All fields are mandatory except where noted.

Once you've filled in the profile information, click ‘Confirm Order’ to proceed to the confirmation page.

 

Order Confirmation

The confirmation page is displayed to give you a last chance to review and revise your request. The granule table lists the granule UR and size, in megabytes, for every granule in your order. Next, the distribution media type name and format for your order is listed. Also listed are the packaging options for your order, whether to include metadata files in your order and the comments to include with your order. Lastly, your shipping address (and contact address, if provided) is displayed. If all aspects of your request are correct, click the ‘Please Submit Your Order’ button to submit your request. Otherwise, click the ‘Go Back to Cart ’ or ‘Go Back to Checkout ’ buttons to go back to the shopping cart or checkout pages, respectively, and modify elements of your order as necessary.

After submitting the request for your order, if the availability of all the data in your order can be verified, an order status page will be presented. Please see the Order Status section for more details on the status of a submitted order. Otherwise, an order submission warning or error may be displayed if the availability of some or all of the data in your order cannot be verified. Please see the Data Set Maintenance section for more information on Data Set availability.

 

Order Status

The order status page displays the order ID and the current status of order. Revisiting or bookmarking the order status page will allow you to retrieve the status of an order after a request is made to the DataPool. Status codes are:

 

Bookmarking a Page

All and any drill-down (search) pages and results pages may be bookmarked. Please note that when you bookmark a page, it is the search criteria and the application identifier that are saved. In other words, when you bookmark a results page, you aren't really bookmarking the granules for that results set, you're bookmarking the criteria that produced that result. Therefore, when you access a bookmark, a new search is performed. This means the results may be different as granules may have been added to or deleted from the DataPool since you last issued this search. Note also that the shopping cart may not be bookmarked.

If you need additional information about bookmarks, check the bookmark section in your browser documentation.

 

Convertible Data Sets

Convertible data is identified by the converter icon (converter icon) in the results screen and by the presence of conversion options in the shopping cart (i.e., the converion columns, format, projection and subsetting, will be popluated with form controls and not just blank, '-'). For more information on which data sets are convertible, please contact User Services at the DAAC.

 

Compressed Data

Science files for granules in the DataPool can be stored in a compressed format in order to reduce the amount of disk space necessary for their storage. Metadata and browse files are not stored in a compressed format at this time. Compression is accomplished by using DAAC-provided compression algorithms. Compression is controlled at the data set level and can be turned on or off by the DAAC operators. Therefore, it is possible that only a portion of the granules in a data set may have compressed science files. The text '(compressed)' appears with the average granule size statement for a data set in the results page and the shopping cart summaries to indicate the data set may have granules with compressed science files. At the granule level in the results table and in the shopping cart, the existence of compressed science files is indicated with bolded text in the size column of the result table. Additionally, the compression downloading options that are normally made available with data in the results set are not made available for compressed granules. Appropriate decompression algorithms can be requested from DAAC user services if necessary.

 

Data Set Maintenance

Access to a data set may be temporarily prohibited to allow for data maintenance by site (DAAC) operations. If a data set is unavailable, users may be prevented from drilling-down on the data set, browsing or downloading files belonging to the data set, or ordering granules for the data set. At order submittal time, orders containing only granules associated with unavailable data sets will be prevented until maintenance is complete on those data sets. For an order with granules associated with a mixture of available and unavailable data sets, a warning message will be displayed and users will be given the option to ‘Submit Anyway ’, which will submit the order with only the granules that are part of available data sets. Contact user services for more specific information on the maintenance cycle for a particular data set.