Customized Data Extraction at the ACRF Archive
Sean Moore | Mission Research |
Raymond McCord | Oak Ridge National Laboratory |
Giri Palanisamy | Oak Ridge National Laboratory |
Category: Infrastructure & Outreach
Customized data extraction using NCVweb. This screenshot shows how a user can select just the items of interest in order to generate and download a custom NetCDF subset file. Here the user is only interested in cloud base heights and has no need for the backscatter data that comprises the bulk of the data stream.
The ACRF Data Archive is developing new software to improve data discovery, access, and delivery for its user community. One new module, known as the ARM NetCDF Data Extraction Request System (ANDERS) will allow users to order and download specific measurements, without requiring them to download an entire data stream. This module has been incorporated into the NCVweb data viewing application and will be available at the Archive and at the Data Management Facility for interactively extracting data from previously ordered files. In order to efficiently extract measurements from many years of data, we anticipate adding the ANDERS module as part of the normal data ordering process. When the user selects measurements to be ordered from the Archive Data Browser, those measurements will optionally be automatically extracted from the source NetCDF files before the user is notified by e-mail that the data request is ready. For many years, NCVweb has allowed users to select a subset of measurements from a NetCDF file and convert them to ASCII for use with spreadsheets or other tools. Now, with the ANDERS module, the user is able to select any combination of measurements from a data stream and generate a customized NetCDF file. For example, the user can ask for and receive just the downwelling radiation without downloading all the extra diagnostic fields. Eventually this capability will be extended to allow parametric selection of data. For example, the user might only want radiation data when the solar zenith angle is small. This new feature allows many months or years of specific data to be combined into a single file for ease of use and more efficient downloading. Feedback on additional requirements for data extraction will be solicited from users to further improve the effectiveness of the Archive.
This poster will be displayed at ARM Science Team Meeting.