OPeNDAP software

NQuery
A Network-enabled data-based data selection tool

What is NQuery?

NQuery is a Java-based, network-enabled data-based query tool that assists the scientist who has selected in-situ datasets within a desired time-space bounding box, and wishes to refine that data selection based on characteristics of the data itself. Examples of such a query might be a request for average temperatures exceeding a specific value, or values along a bathymetry contour. Such a capability is particularly important when the data collections or datasets are very large and located on remote, network accessible data servers (such as OPeNDAP data servers).

What problem does NQuery address?

The ability of a scientist to work productively with very large datasets has been limited because of the difficulties related to the network transfer of significant subsets of the data, and the subsequent search through the data for specific observations or data characteristics of interest. Many different sources of data are available on the network through OPeNDAP (in-situ data collections and gridded datasets), yet in order for scientists to determine whether each data file will suit their needs, they must sort through each file, wasting time and energy. This difficulty is compounded when the scientist needs to inter-relate these results with the extensive climatological data sets that have been created and are now available directly from the network. .

What is the solution?

We have developed NQuery, a network-enabled data-based query tool that allows a scientist who has selected datasets that meet the desired time-space criteria, to then determine whether each data file will suit their needs, based on the characteristics of the data within each of the data files. The scientist specifies the specific desired data characteristics, and the Nquery tool returns the specific datasets that meet those criteria.

The mechanism used within the NQuery tool is to load pertinent subsets of user-selected multi-disciplinary datasets into a temporary, on-the-fly relational database, perform local calculations, and then uses the scientists specifications to construct sophisticated SQL queries to locate subsets of interest. A critical design consideration, with implications for the implementation methodology, was the speed of this application. Selecting calculations only as necessary makes the query process as efficient as possible.

Results

NQuery enables scientists to quickly and easily navigate through pre-selected datasets, filter the data to their needs, and then use the discovered data in their research.

A simple two-step process takes the user from pre-selected data to a set of selected data files. The user is unaware of the complexity of the underlying process. First, the user determines what observed and computed variables will be included in the database, and therefore available for subsequent queries. This selection is accomplished via a graphical interface that lists all available variables, and computed variables, such as mixed layer depth, apparent oxygen utilization, and sigma levels. The space and time ranges can be set to restrict the domain of interest. A single click then begins working through the pre-selected dataset to extract and compute the requested variables. An on-the-fly MySQL database is created and populated as each data file is processed team.

Where is NQuery used?

Currently, the only desktop application that can ingest the XML pointer files from NQuery is Java OceanAtlas 4.0 (http://odf.ucsd.edu/joa).

More Information