Document Archive

Data Integration Framework (DIF) Project

In December, 2006, the NOAA Executive Council and NOAA Executive Panel approved the formation of NOAA’s Integrated Ocean Observing System (IOOS) Program within the National Ocean Service. This included approval for the IOOS Program to develop a Data Integration Framework (DIF) project with a nominal duration of three years, from February 1, 2007 to February 1, 2010. The DIF was designed as a limited-scope, risk reduction project to measure the value of integrated data on four (4) ocean decision-support tools that span multiple NOAA mission goals. Following the nominal three year project duration, experience gained through the DIF was used to inform long-term IOOS data management planning and implementation.

Data Integration Framework (DIF) Final Assessment Report

In November, 2010 Data Integration Framework (DIF) Final Assessment Report was published, describined the DIF project, it’s outcomes, and recommendations for IOOS data management planning and implementation.

DIF-related documents

• IOOS Regional Data Integration Framework Implementation Workshop (pdf)
  

• The NOAA IOOS Data Integration Framework: Initial Implementation Report: :Slides (pdf, 315kb) and associated paper (pdf, 125kb) presented at MTS/IEEE Oceans 2008 conference in Quebec, Canada 2008-09-17.

• Project Plan: (pdf, 168kb)The Project Plan records planning ideas and agreement details, approved scope, cost, and schedule baselines.

• Functional Requirements: (pdf, 333kb)This document defines the functional requirements of the DIF and is a guide to further system documentation and design.
  

• Concept of Operations: (pdf, 410kb)The Concept of Operations (ConOps) document is the translation of the DIF functional requirements into operational scenarios. The ConOps provides important input into the high-level system design.

• DIF Customer Implementation Project Summary and Performance Assessment Plan: (pdf, 457kb) This document describes the 4 customer implementation projects used to assess the value of DIF's integrated data. Provides overview of benchmarks, metrics, and performance assessment methodology.

• DIF Design Document: (pdf, 1,039kb)

• DIF Customer Implementation Project Summary and Performance Assessment Plan: (pdf, 457kb)

• IOOS DIF Draft Architecture & Recommendations
  

• Data Integration Framework (DIF) Project Charter
  

• DIF As-Is Baseline Systems Document

 

Technical Information

Schemas and Specifications

• XML Encoding: IOOS DIF XML Schema beta v0.6.1 - This Geography Markup Language (GML) application schema defines the XML encoding used by the NDBC and CO-OPS SOS to serve in situ data. This encoding is designed to be complete and machine-readable, and may be transformed to simplified representations using code such as the Extensible Styleheet Language Transformation (XSLT) example below.

• Text encoding: IOOS conventions for comma-separated value (CSV) (PDF, 84kB) and tab-separated value (TSV) (PDF, 95kB) encoding. CSV and TSV are offered by the NDBC SOS and eventually by other IOOS SOS. The CSV is directly readable by common spreadsheet programs. The TSV includes a few specialized column headers to make it more easily readable by the Ocean Data View (ODV) software package.

Software

The following software is provided on an as-is basis with no support or warranty. Read the full disclaimer before downloading.

• NDBC SOS software - version 1: This software was developed by the National Data Buoy Center (NDBC) to provide a SOS interface to their MySQL database back-end. Offers data in GML, CSV, TSV and KML encodings.

• GCOOS SOS software beta version 0.6.1(zip, 88kb)This PHP script was modified from the NDBC version to support SQLite database engine. This beta software is still under active development and test.

• Format converter:IOOS SOS to CSV v0.6.1This XSLT file converts the IOOS SOS v0.6.1 observation values from GML to comma-separated value (CSV) format suitable for use by Excel and other simple applications.

• Format converter:netCDF to BUFR converter beta v0.1 This sample application was developed at AOML. It converts data from a netCDF format accepted by DIF to the Binary Universal Form for the Representation of meteorological data (BUFR) format maintained by the World Meteorological Organization (WMO). The archive contains C source code, descriptor file, and brief help on compilation and execution.

Tcl/Tk Example These Tcl/Tk routines are used by the Coastal Inundation customer for retrieving water level observations from the CO-OPS SOS.

(top)

 

 

Workshops

IOOS Regional Data Management Workshop, Sept 11-13, 2012

IOOS Regional Associations DMAC Coordination Briefings

• AOOS (pdf), Rob Bochenek
• CariCOOS (pdf), Jorge Capella
• CeNCOOS (pdf), Steven Le and Jennifer Patterson
• GCOOS (pdf), Matt Howard
• GLOS (pdf), Tad Slawecki
• MARACOOS (pdf), Kyle Wilcox
• NANOOS (pdf), Emilio Mayorga
• NERACOOS (pdf), Eric Bridger
• PacIOOS (pdf), Jim Potemra
• SCCOOS (pdf), Lisa Hazard and Darren Wright
• SECOORA (pdf), Vembu Subramanian and Jeremy Cothran

IOOS DMAC Workshop Session Presentations

• Collaborative SW Development (PDF), Kyle Wilcox, MARACOOS
• Data Access Services Brief (PDF), Derrick Snowden, IOOS Program Office
• DMAC Workshop 2012 Cloud Hosting (PDF), Steven Le, CeNCOOS
• Improving Aquatic Biological and Ecological Observing Data Integration and Dissemination (PDF), Hassan Moustahfid, IOOS Program Office
• IOOC DMAC ST Brief (PDF), Charly Alexander, IOOS Program Office
• IOOS Catalog Brief, Dave Neufeld (PDF), NGDC
• IOOS Modeling Testbed Brief (PDF), Kyle Wilcox, MARACOOS
• IOOS Summit Brief (PDF), Carl Gouldman, IOOS Program Office
• IOOS Vocabulary Management (PDF), Sare Haines, SECOORA, Emilio Mayorga, NANOOS, Rob Ragsdale and Hassan Moustahfid, IOOS Program Office
• NODC Session Brief (PDF), Steve Rutz, NODC
• Ocean Workspace Demonstration (PDF), Rob Bochenek, AOOS
• QARTOD Brief (PDF), Charly Alexander, IOOS Program office
• Status of IOOS 52N SOS (PDF), Shane StClair, AOOS
• System Architecture Design and Content Standards for AAT Data (PDF), Hassan Moustahfid, IOOS Program Office

(top)

Interagency Ocean Observation Committee (IOOC)
Data Management and Communications (DMAC) Steering Team Meeting, May 11-12, 2011

On May 11-12, the U.S. Data Management and Communications Steering Team of the Integrated Ocean Observing Committee met to discuss ocean observation data management standards for the nation. The team is comprised of federal members across seven core agencies and regional participants from non-federal organizations and industry. The U.S. Integrated Ocean Observing System (IOOS®), which resides within NOAA, will chair these important deliberations.  

Meeting Agenda, the Federal Register Announcement and notes are availabe at:
http://www.iooc.us/committee-news/dmac/dmac-st-meetings/

 

IOOS Regional Data Management Workshop, April 27 - 28, 2011

Workshop Details

Workshop Theme: Advancing IOOS RA DMAC Interoperability
Workshop Goal: Resolve challenges and issues to advancing interoperability among the eleven IOOS Regional Associations.
Workshop Objectives:

• Review state of the IOOS Regional Association’s Web Services.

• Identify and discuss challenges (technical and/or governance) to implementing interoperable Web Services.

• Coordinate with other IOOS activities

IOOS Regional Data Management Workshop, March 23-24, 2010

Workshop Report (pdf, 11,380kb)

IOOS Regional Data Integration Framwork Implementation Workshop

IOOS Regional Data Integration Framwork Implementation Workshop:(pdf, 13mb)