The ACIS System

NWS NOWDATA Webpage for Aberdeen, SD

Diagram showing the processes of data capture, archive, quality control, and production of a value-added climate data product.

Metadata

Metadata is simply defined as data describing other data. ACIS metadata is managed using a relational database system that is queried extensively during the process of data discovery, data entry and extraction, and product generation. The metadata determines the availability of stations and station data elements required by a selected product, determines links between stations to expand temporal extent of data records, guides conversion of data collected in incompatible units, and evalues the quality of data records based on data value flags.

Climate Data

Climate data available in ACIS consists of in-situ observations collected from a variety of federal, regional, state, and local weather and climate networks. Both archive-quality historical data and near-real-time data ingested from operational data transmission streams are available. ACIS data are primarily used for the generation of operational climate products but also provide quality archive datasets. New data sources are continually added contingent upont demand, need, time and funding (RCC resources).

Ingest

The Data ingest process collects real-time data from internet and satellite sources, as well as historical data obtained using semi-automated file transfer methods. These sources disseminate data in different transmission formats. The Ingest process using separate modules for each transfer method and format to route the incomming data to the appropriate quality control routines and route the incoming data to local data and metadata archives within the ACIS framework. Data entry processes exist that allow for inclusion of unofficial, non-published station data. This allows for manual data entry/edit followed by 'injection' of these data into standard ACIS data ingest processes.

Quality Control

ACIS data are subject to a hierarchy of quality control that includes comparison to physical and climatological limits and inter-comparison of data elements. A more comprehensive system has been developed for ACIS that includes enhanced checks based on per-station statistical limits and an advanced geospatial regression model. This system is being tested by the RCCs and NCDC and shows great promise in identifying errors while minimizing mis-identification of data extremes that are valid observations. This system is scheduled for operational implementation in 2007. RCC Regional Climatologists will review observations flagged by the automated QC system as suspected errors. This two-tiered system of QC will ensure that data entered into ACIS will be of the highest possible quality.

Archive

ACIS data are currently stored using the Network Common Data Form (netCDF). This common format lends itself to data access speed, efficient use of storage space, and acceptance by a wide community of climate data users. The implementation of redundant ACIS data stores at multiple RCCs ensures timely availablility of data and products in the event of network or data center failure. Synchronization of datasets among the RCC data centers ensures that identical information is provided regardless of point of access. A remote switching system automatically distributes ACIS requests among the available servers to provide distributed load and timely system response.