DOC | NOAA | NOAA Research | ESRL | GSD | MADIS HOME

Meteorological Assimilation Data Ingest System (MADIS)

The demands for finer scale meteorological services have increasingly required higher resolution observations to initialize and evaluate weather and climate models, applications, and products. In response to these demands, the National Oceanic and Atmospheric Administration (NOAA) Research (Oceanic and Atmospheric Research (OAR)) Earth System Research Laboratory (ESRL) Global Systems Division (GSD) developed the Meteorological Assimilation Data Ingest System (MADIS) to collect, integrate, quality control (QC), and distribute observations from NOAA and non-NOAA organizations. MADIS leverages partnerships with international agencies; federal, state, and local agencies (e.g. state Departments of Transportation); universities; volunteer networks; and the private sector (e.g. airlines, railroads) to integrate observations from their stations with those of NOAA to provide a finer density, higher frequency observational database for use by the greater meteorological community. MADIS observational products and services were first provided to the public in July of 2001.

MADIS runs operationally in real-time in the National Weather Service (NWS) with a distributed architecture consisting of ingest and distribution services at the Telecommunications Operations Center (TOC) with processing performed at the National Centers for Environmental Prediction (NCEP) Central Operations (NCO). MADIS also runs quasi-operationally in a research test environment at ESRL/GSD, where new advances are developed and tested prior to being put into operations. The ESRL/GSD system also has an archive of saved real-time data, and serves as the backup to the operational system.

Note there will be some differences in the data available from the GSD test environment and from the NWS operational systems.

MADIS subscribers have access to an integrated, reliable, and easy-to-use database containing the real-time and saved real-time observational datasets described below. Also available are real-time gridded surface analyses that assimilate all of the MADIS surface datasets (including the highly-dense integrated mesonet data). The grids are produced by the Rapid Update Cycle (RUC) Surface Assimilation System (RSAS), which incorporates a 15-km grid stretching from Alaska in the north to Central America in the south, and also covers significant oceanic areas. The RSAS grids are valid at the top of each hour, and are updated every 15 minutes.

MADIS ingests data files from NOAA data sources and non-NOAA data providers, decodes the data and then encodes all of the observational data into a common format with uniform observation units and time stamps. Quality control checks are conducted and the integrated data sets are stored in the MADIS database with a series of flags indicating the quality of the observation from a variety of perspectives (e.g. temporal consistency and spatial consistency), or more precisely, a series of flags indicating the results of various QC checks. MADIS users and their applications can then inspect the flags and decide whether or not to use the observation. MADIS data is made available to the enterprise using multiple data transfer protocols via the Internet, including ftp, Unidata's Local Data Manager (LDM) software, or for the surface datasets through the Text/XML Viewer web service found below. Users can subscribe to the entire database, or ask for only particular datasets of interest.

MADIS also includes an Application Program Interface (API) that provides users with easy access to the observations and quality control information. The API allows each user to specify station and observation types, as well as QC choices, and domain and time boundaries. Many of the implementation details that arise in data ingest programs are automatically performed. Users of the MADIS API, for example, can choose to have their wind data automatically rotated to a specified grid projection, and/or choose to have mandatory and significant levels from radiosonde data interleaved, sorted by descending pressure, and corrected for hydrostatic consistency. The API is designed so that the underlying format of the database is completely invisible to the user.

MADIS data files are compatible with AWIPS-I and AWIPS-I-like display systems and the analysis software provided by the Local Analysis and Prediction System (LAPS), the Weather Research and Forecasting (WRF) Model Variational Data Assimilation System, and the Atmospheric Model Evaluation Tool (AMET) provided to the air quality modeling community by the Community Modeling & Analysis System at the University of North Carolina at Chapel Hill. They have also been used to initialize the Advanced Regional Prediction System (ARPS) , MM5 , and Coupled Ocean/Atmosphere Mesoscale Prediction System (COAMPStm) forecast models.

The MADIS database and API are freely available to interested parties in the meteorological community.

Information

Data Access

MADIS subscribers can receive real-time data or obtain access to an on-line archive of saved real-time data by requesting an ftp or Text/XML Viewer account. For real-time data, LDM access is also available. Text/XML Viewer accounts are recommended for those who have a need for data on-demand, as contrasted with continuous real-time access. In general, users who require a continuous datafeed will get better performance by accessing the data via ftp or LDM. Also note that at this time, the Text/XML Viewer can only be used for the surface observation datasets.

Text/XML Viewer

Public MADIS Meteorological and Hydrological Surface datasets can be viewed in either text or XML formats. The unrestricted data are available from the links below. All users should read the Data Use Policy and the Disclaimer in the MADIS Data Application. Users eligible for the restricted mesonets should request a Text/XML Viewer account.

Compatible Data Assimilation Software

The links below provide information about, and downloads of, the data assimilation software packages that are directly compatible with the MADIS datasets.

Compatible Display Software

The link below provides information about, and downloads of, the display software packages that are directly compatible with the MADIS datasets.

Observation Displays

Contact Information

All questions should be sent to madis-support@noaa.gov.

Staff

Last updated 27 November 2013

Contact Us | Privacy Policy | Accessibility | Disclaimer