WOCE Upper Ocean Thermal Data Assembly Centres Coordination Group Report of the Sixth Meeting (UOT/DAC-6)


NODC, Silver Spring, USA
18-19 April, 1996

WOCE Report No. 145/96


CONTENTS
1. Introduction
2. Review of Action Items from Workshop on Quality Control of UOT Data
3. WOCE Scientific Steering Group Review of the UOT DACs
4. Present Status of the WOCE UOT DAC System
5. Flag Intercomparison
6. Transfer of Expertise to National Data Centres
7. Standardization of Quality Control
8. Data Delivery
9. Further Meetings
Appendix A: Agenda
Appendix B: Participants

1. Introduction

The sixth meeting of the WOCE Upper Ocean Thermal (UOT) Data Assembly Centres (DACs) was chaired by P. Holliday and held at the new site of the US National Oceanographic Data Centre (NODC) in Silver Spring, MD, USA. P. Holliday opened the meeting at 8.30 a.m. on Thursday, 18 April 1996 and welcomed the participants, thanking NODC (in particular D. Hamilton and M. Hamilton) for hosting the meeting. The aim of the meeting was to discuss the ways in which the UOT DACs were to meet the requirement of WOCE and future programmes. In particular the issues to be discussed included providing data and products for the WOCE Scientific Workshops, the role of the DACs in long term climate study programme such as CLIVAR, and producing the globally consistent data set. The meeting was held in conjunction with the fourth meeting of the IOC-WMO Steering Group of the Global Temperature and Salinity Pilot Project (GTSPP). This report contains some references to the report of the GTSPP-IV meeting (IOC/INF-1033) where issues were discussed jointly.


2. Review of Action Items from Workshop on Quality Control of UOT Data

The list of action items from the Workshop on Quality Control of Upper Ocean Thermal Data, May 1995 (WOCE Report No. 133/95) was reviewed and the outstanding action items revisited. Several actions were discussed later in the meeting and so are not referred to here.

Items 1-3 all relate to preserving information on data quality from data originators. The Workshop recommended that quality flags and information from the source of the data be preserved, so utilizing the expertise of those closest to the collection of data. This recommendation was passed to the IGOSS Ship-of-Opportunity Programme meeting (SOOP-V) in October 1995, and to the IGOSS and IODE meetings in late 1995 and early 1996. It is too early to say if the recommendation has been acted on, as few delayed mode data have been received from originators since then. The representatives from NODC indicated that at this stage NODC can only store this information if the data are submitted in the MEDS ASCII format.

Item 4 concerned the optimum number of inflexion points in a BATHY message; WOCE has not specified the tolerance required from real time data so this issue could not be resolved. The issue should be revisited at future meetings.

Item 9 concerned the automatic flagging of data outside climatalogy limits. At this time NODC are the only centre still giving data Flag 2 when they lie outside of climatology.

Action: NODC was requested to reconsider automatic flagging of data outside climatology and recommended that instead the QC expert be alerted of a test failure and to view the profile individually.

Item 13 discussed the overall profile quality flag ("Q" record). It was agreed at the GTSPP meeting that this flag should reflect the "worst" quality data in the profile; i.e.. to retain the meaning as described in the GTSPP QC manual (IOC Manuals and Guides 22).

The GTSPP-IV meeting had previously discussed the role of the data centres in the GTSPP Web pages; Item 15 of the Workshop recommended that all centres have their own Web pages, and at present only CSIRO do not have such facilities. Item 17 requested the GTSPP QC Manual be placed on the Web, and R. Wilson was requested to coordinate this action by GTSPP-IV .

E. Lindstrom and R. Bailey reported that the planned "UOT-DAC QC Handbook" discussed in Item 21 is currently being compiled.

Items 26-28 were concerned with providing helpful information to ship operators and the officers deploying profiles from ships of opportunity. Jim Hannon of Sippican Inc. has compiled some information points to be distributed. The issue was discussed at the IGOSS Ship-of-Opportunity Programme meeting in October 1995.

3. WOCE Scientific Steering Group Review of the UOT DACs

The WOCE SSG is carrying out a review of the entire WOCE data management system in order to document what added value each component brings, both for WOCE and in terms of future programmes. As part of this review, the SSG tasked a small ad hoc group to review and document the UOT DAC system which has several participating data and science centres. The review group aims to document the added value that scientific quality control brings to the data set, how the delivery schedule for UOT data and products affects results from WOCE, and how the flow in particular of delayed mode data can be improved.

Eric Lindstrom, chairman of the WOCE Data Products Committee, opened this agenda item with a summary of the requirements of the SSG review and reiterated that the overall aim of the UOT DAC is to produce a consistent high quality global data set with flags that users can understand and utilize. The field phase of WOCE will end after 1997 and a series of basin-wide scientific workshops is planned, starting with the Pacific Workshop in August 1996. The purpose of the Workshops is to initiate cooperative research activities to address the major scientific issues. It is envisaged that they will encourage the synthesis of all available data sets from each basin to study various aspects of the general circulation. It is apparent from the schedule for delivery of UOT data (see WOCE Report 133/95, the Workshop for Quality Control of UOT Data) that the scientifically quality controlled UOT data will have a limited impact on these Workshops simply because they will not be available. This is mainly a result of the slow submission of delayed mode data. It was noted that large UOT data sets which have received a certain level of quality control are available already; all real time data have been subjected to GTSPP quality control checks by MEDS, and some delayed mode data, such as the CSIRO and ORSTOM sourced XBT data, have been subjected to scientific quality control prior to entering the UOT/DAC system.

The UOT DAC quality control will have an important and significant impact in the detection of long term climatic signals; the scientific quality control eliminates apparent climate signals which are in fact caused by the instruments. The SIO DAC has shown that apparent anomalies in Pacific heat storage maps can disappear or have their signs reversed after the scientific quality control has been applied.

Within the GTSPP/WOCE UOT DAC cooperation, new techniques for quality control have been developed by both data and science centres. The cooperative arrangement has helped data centres to improve/calibrate their QC techniques with those employed by science centres in the project. It is envisaged that eventually as much as possible of the techniques and skills developed at the Science Centres will be transferred to national data centres (Section 6). Thus the WOCE UOT DACs provide a service essential for WOCE and which will be invaluable to long term programmes such as CLIVAR, GOOS etc. It was recommended that the added value the Science DACs are providing should be documented and widely distributed within the scientific community (for example through the WOCE Newsletter).

Action: WOCE IPO to coordinate the documentation of the added value of the UOT DAC quality control.

Action: R. Bailey to coordinate information from the Science Centres for the SSG Ad Hoc   review group.

The Group concluded that the UOT DACs need to address the issue of submission of delayed mode data with some urgency. Immediately after NODC and Brest have completed their exchange of data, the outstanding delayed mode data must be identified and the ship operators contacted to obtain the data. Information held by the IGOSS Operations Coordinator will be used to identify the programme managers responsible.

Action: NODC, Brest, IGOSS Operations Coordinator and WOCE IPO to carry out urgent delayed mode data tracking exercise:

NODC/Brest provide inventory by call sign of real time, delayed mode data and "matches" received for years 1990­1994.

IGOSS Operations Coordinator and WOCE IPO to identify and contact the ship operators or programme managers responsible for the known outstanding data.

NODC to contact the national oceanographic data centres of programme managers with outstanding delayed mode data.

Regarding the WOCE Scientific Workshops, it was noted by the Science Centres that is was not clear to them what level of data and which products are required for the Workshops.

Action: The WOCE Synthesis and Modeling Working Group was requested to provide guidance on exactly what was required of the DACs for the Scientific Workshops.


4. Present Status of the WOCE UOT DAC System

Each component of the UOT DAC system was asked to provide a short summary of their achievements in 1995­1996 and the status of their data sets and products. R. Bailey presented the status of the Indian Ocean DAC at CSIRO, Hobart. He reported that the 1990 data set has been submitted to NODC, and the 1991 data set was almost complete, and provided figures on the percent of profiles with the different levels of QC flags (Table 1). He noted that the 1991 data set contained considerably more real time data than 1990. He stressed the Indian UOT DAC did not receive separate funding but was totally supported by existing resources of the contributing organizations. The Indian Ocean DAC is developing a regional climatology (already in existence along CSIRO XBT lines) and is starting to produce a standard set of analyses (although the latter is made difficult by the data distribution which tends to be along shipping tracks).

TABLE 1. QUALITY FLAG DISTRIBUTION FOR YEARS 1990 AND 1991




AOML CSIRO SIO


1990 1991 1990 1991 1990 1991
A. No. in GTSPP file 19028
5975
5544
4728
47564


B. Unique Profiles QC'ed 16866 100% 5957 100% 5313 100% 4428 100% 40892 100% 34173 100%
C. All Class 1 or 2 15732 93% 5186 87% 4710 89% 4214 95% 35866 88% 30458 89%
D. Some or All Class 3 or 4 1134 7% 771 13% 603 11% 214 5% 5024 12% 3615 11%
E. All Class 3 or 4



207 4% 169 4% 59 0% 132 0%
F. Flagged Bad by MEDS/NODC 446 3% 182 3%



3082 8% 2086 6%
G. Re-flagged as Good 230 1% 14 0%



336 1% 86 0%
H. Flagged Bad differently 196 1% 166 3%



-
-
I. Additionally Flagged as Bad 918 5% 603 10%



1942 5% 1547 5%
J. Flagged differently to MEDS/NODC




1%
2%




Explanatory Notes:
Class 1. Good data
Class 2. Probably Good data
Class 3. Probably Bad data
Class 4. Bad data
(See WOCE Report No 133/95 for full description of flag classes)

A. Number of profiles in the data set provided by NODC.
B. The number of profiles actually quality controlled by the Science Centres. The difference between A and B is due to some duplicate profiles remaining in the NODC set, and those profiles from marginal seas which are beyond the expertise of the DACs.
C. The number of profiles containing only "Good" or "Probably Good" data.
D. The number of profiles with any data points flagged as "Probably Bad" or "Bad".
E. The number of profiles with no good data at all; a subset of category D.
F. The number of profiles classed as "Bad" by NODC/MEDS
G. The number of profiles the Science Centre re-set flags to "Good" (Subset of F).
H. The number of profiles the Science Centre set flags to "Bad" but differently to MEDS/NODC (Subset of F).
I. The number of profiles that MEDS/NODC declared as "Good" but the Science Centre classed as "Bad".

S. Diggs reported on the achievements of the Pacific DAC at SIO over the last year. The DAC has a series of products currently available on the Web, including data distribution maps, climatology, real-time analyses, animations, historical analyses, and temperature anomaly maps. The quality controlled 1990 and 1991 data sets have been delivered to NODC. They have found that with their growing experience and expertise they have more confidence in the quality control of the 1991 set and plan to re-process the 1990 data set by May 1996 as a result. NODC will not load the Pacific 1990 set until they have received the updates version from the Pacific DAC. During the discussion it was noted that the DAC receives a quantity of marginal seas profiles which they do not quality control because they do not have the appropriate knowledge to do so; in these cases the profiles are not returned to NODC with the quality controlled set.

Y. Daneshzadeh summarized the status of the Atlantic DAC at AOML; both the 1990 and 1991 data sets have been quality controlled and delivered to NODC. She noted that the 1991 data set has a higher number of bad profiles than 1990 (Table 1). R. Molinari informed the Group that the DAC is currently developing data products and their Web pages. They are also compiling a delayed mode data report which will include heat content and temperature at selected depths and anomaly maps. The report will show how representative the WOCE field phase years are (Goal 2 of WOCE) and the difference the scientific quality control makes to the data set.

Action: Science Centres to coordinate their products (e.g. maps of heat content, sea surface temperature, temperature at selected depths, etc.) to achieve global consistency for users.

M. Hamilton summarized the achievements of NODC over the past 12 months. A key new ability is being able to manage the quality control information provided by the Science Centres. Recently they have been carrying out an extensive comparison of their data holdings with Brest after it was discovered that there is a substantial difference between the two centres. The comparison has revealed that NODC requires around 13,000 profiles retained at Brest, and Brest requires around 71,000 profiles from NODC (many of these are extra-tropical profiles; Brest became a global centre in 1993). In particular a large number of delayed mode profiles from 1992 are archived at Brest and not at NODC, the exchange of data will go some way to improving the level of delayed mode data available for the Science Centres to quality control. The two centres are developing mechanisms for ensuring that there is continual exchange of data between them to ensure such a discrepancy does not arise again.

M-C. Fabri outlined the highlights of the past year at Global Subsurface Data Centre in Brest. The database now resides on a new machine and the increased capability means it is no longer necessary to perform any data reduction in their working database. Recorder and probe type are recorded when information is provided. A new input format has been developed, and the centre can routinely provide a new data distribution map for the CLIVAR-WOCE XBT/XCTD Programme Planning Committee. Information and statistical products are available on the Web.

MEDS continues to acquire and perform quality control and duplicates resolution of the low resolution data transmitted over the GTS. There have been a few problems with receiving data from FNOC but generally the process is working well. MEDS produces monthly reports to WOCE on ship performance and is working to include data from other sources that do not get inserted onto the GTS.

5. Flag Intercomparison

The primary product of the UOT DAC is a globally consistent data set and it is valuable to carry out flag comparison exercises both to quantify the added value of scientific quality control and to ensure consistency between the Science Centres. MEDS, NODC and CSIRO are presently performing an intercomparison between a common data set. R. Bailey presented the Group with some examples of how the different centres flag the same profile in a slightly different way. CSIRO distributed a small data set to MEDS and NODC which was already QC'ed by CSIRO (both scientifically, and by an automated objective mapping system). The test highlighted how automatic flagging and objective analysis schemes often missed erroneous profiles where the instrument had clearly malfunctioned. Procedures where each individual profile is examined by the operator, such as developed by CSIRO and implemented by MEDS, were found to be much more accurate.

Action: R. Keeley to document the results of the MEDS, NODC and CSIRO flag comparison exercise, and how it helps to improve the quality control systems. This should be completed by May 1996 and the results published widely, for example in the WOCE International Newsletter.

It was suggested that each Science Centre participate in such a comparison exercise.

Action: R. Keeley to coordinate an intercomparison exercise of high resolution data between all Science Centres, MEDS and NODC. This should be completed by September 1996 (prior to the WOCE SSG meeting) and the results documented fully.

6. Transfer of Expertise to National Data Centres

An important legacy of the WOCE data management infrastructure is the system itself and the skills, experience and expertise that have been developed over the WOCE period. From the start, it has been envisaged that the WOCE DACs would not be permanent centres, but that eventually their role would be fulfilled by the more permanent national oceanographic data centres. The UOT DAC has been particularly active in working towards achieving that long term goal. There have been many meetings of the various centres involved under the umbrella of the GTSPP and the WOCE UOT DAC, during which experiences and techniques have been demonstrated and exchanged. This cooperation has led to changes and improvement to the procedures employed by the national data centres (MEDS, AODC, NODC, GSDC) and has contributed to the success of the GTSPP.

An example of a partnership that has progressed further than most is that between the Indian Ocean Science Centre at CSIRO and the Australian Oceanographic Data Centre (AODC). Close cooperation between the two centres has enabled AODC to use the interactive QC system developed at CSIRO. The process, planned from the start, has been a long one, with much time and resources being committed by both centres to ensure its success. It has involved several exchanges of personnel between the centres for substantial periods of training, and the purchasing of new equipment by AODC.

The WOCE field phase is coming to an end in 1997. However, requirements for new and improved quality control procedures, data products and data dissemination techniques will continue through CLIVAR. As indicated above, the WOCE Science Centres have made many contributions to the national data centers, and it is envisaged that they can play a similar role for CLIVAR (particularly since new data sets will come on-line, e.g., profiling floats, continuous observation of surface salinity, etc.). Many issues must be resolved to evaluate the utility of such a role for the science centres. For example, using the Science Centres as a site for development of techniques used by the national data centres will require substantial commitment from both sides in terms of time and money. To demonstrate the resources required, AODC and CSIRO were requested to document the extent of their exchange of expertise, in order to demonstrate the resources required. It was also considered important that the two centres demonstrate the tangible benefits for both centres.

Action: AODC and CSIRO were requested to document the extent of their exchange of expertise, and the tangible benefits this has provided.

It was suggested that the UOT DACs request a commitment from particular national data centres to facilitate the exchange. However, this was not felt to be appropriate at this stage. Instead, the UOT DAC Group reported the following recommendation to the GTSPP Steering Committee during their joint session:

Establishing improved quality control procedures is an important goal of UOT DACs. The transition of these procedures to the National Data Centres is another goal that has been performed (with the exception of CSIRO and AODC) on an ad hoc  basis. The Group recommended that formal mechanisms for transferring new techniques from the DACs to the Data Centres should be established, e.g. personnel transfers and exchange/purchases as required, training over a prolonged periods, commitment from both sides. The ultimate objective is to establish the National Data Centres as the sites for the highest level quality control needed to meet as yet unspecified accuracy criteria.

The GTSPP was asked to consider how such mechanisms could be put in place, and to obtain commitment from appropriate national data centres to acquire the techniques from the WOCE Science Centres. Such commitments could not be made at the GTSPP-IV meeting, but possibilities will be explored during the intersessional period and reported on at GTSPP-V.


7. Standardization of Quality Control

The report of the Workshop for Quality Control of WOCE UOT data (WOCE Report No. 133/95) discussed the issue of "start-up transient" data. Agreement was reached at the Workshop but AOML and SIO requested further discussion since the Science Centres were still employing different procedures. The IGOSS Task Team for Quality Control of Automated Systems has identified that the data from 0­3.7 m of each XBT profile is unreliable since the instrument takes some time to equilibrate. At present only CSIRO remove data from 0­3.7 m and flag them as changed values (retaining the original values in the history table). It was argued that not every profile contained bad data at the top. AOML noted that in the 1991 dataset they identified only about 2% of profiles with start-up transient problems. However, it is not clear if the other profiles started at 0 m or had already had the top 3.7 m of data removed. This requires further clarification before another recommendation can be made.

8. Data Delivery

The UOT DACs aim to provide WOCE with the best available data set in as timely way as possible. The delivery schedule agreed at the Workshop for Quality Control of UOT data (WOCE Report 133/95) still remains, and the DACs will aim to provide the best quality data available to the WOCE Scientific Workshops.

Action: The Pacific DAC will quality control as much available Pacific data as possible (through 1994) prior to the Workshop in August 1996.

9. Further Meetings

The Group agreed that the transfer of expertise to the national data centre was an important task which required further guidance and monitoring. It was agreed that this should be done within the framework of the GTSPP, and so the UOT DACs should meet in conjunction with the GTSPP Steering Group. A tentative date for the next GTSPP meeting is October 1997.


Appendix A: AGENDA
Tuesday­Thursday, 16­18 April GTSPP only
Thursday, 18 April UOT DAC only
Friday, 19 April UOT DAC with GTSPP

18 April - UOT DAC
1. Opening and welcome; local arrangements
  Review of May 1995 Workshop Action Items

2.  Present Status
  1995 Activities/Achievements (short reports from SIO, AOML, CSIRO, MEDS, Brest, NODC)
  Funding situation
  Status of 1990­92 datasets
  Flags and codes at NODC

3. Intercomparison
  Results of data/flag intercomparison exercises:
  NODC/MEDS cf Science Centres
  Science Centres cf Science Centres

4. Standardization
  Start-up transient data
  Flagging spikes vs deleting spikes

5. SSG Review of UOT DACs

6. Priorities
  Schedule of data delivery
  WOCE Workshops

19 April - UOT DAC and GTSPP

7. Data Submission (GTSPP Agenda 3.3)
  NODC/Brest comparison of data holdings
  Delayed Mode data tracking and submission

8.  DAC Output and Products (GTSPP Agenda 3.5)
  Data and Product availability
  Web sites
  Operator Feedback
  UOT DAC QC Handbook

9. Looking Ahead (GTSPP Agenda 4)
  Future of the DACS within WOCE
  Links to CLIVAR and other programmes

Appendix B: PARTICIPANTS

Dr. Rick Bailey
CSIRO Division of Oceanography
G.P.O. Box 1538
Hobart, Tasmania 7001
Australia
Tel: 61-02-32-5295/5222
Fax: 61-02-32-5123
e-mail: bailey@ml.csiro.au

Dr. James Churgin
College of Marine Studies
University of Delaware
Lewes, DE 19958-1298
USA
Tel: 1-302-645-4273
Fax: 1-302-645-4007
e-mail:

Mr. James Crease
WOCE Data Information Unit
College of Marine Studies
University of Delaware
Lewes, DE 19958-1298
USA
Tel: 1-302-645-4240
Fax: 1-302-645-4007
e-mail: jimc@diu.cms.udel.edu

Dr. Yeun-Ho Daneshzadeh
NOAA/AOML/PHOD
4301 Rickenbacker Causeway
Miami, FL 33149-1097
USA
Tel: 1-305-361-4332
Fax: 1-305-361-4582
e-mail: chong@adrift.aoml.er1.gov

Mr. Steve Diggs
Scripps Institution of Oceanography
University of California at San Diego
Mail Code 0230
9500 Gilman Drive
La Jolla, CA 92093-0230
USA
Tel: 1-619-534-1108
Fax: 1-619-534-7452
e-mail: sdiggs@ucsd.edu

Dr. Marie-Claire Fabri
Global Subsurface Data Center
IFREMER Brest/SISMER
B.P. 70
29280 Plouzané
France
Tel: 33-98-22-42-00
Fax: 33-98-22-46-44
e-mail: marie.claire.cfabri@ifremer.fr

Ms. Melanie Hamilton
SSMC, 4th. Floor
NOAA/NESDIS/NODC
1315 East-West Highway
Silver Spring, MD 20910
USA
Tel: 1-301-713-3283 x123
Fax: 1-301-713-3302
e-mail: melanie@birdie.nodc.noaa.gov

Mr. Bruce Hillard
IGOSS Operations Co-ordinator
IOC/UNESCO
SC/IOC B 5.09
1 rue Miollis
75732 Paris Cedex 15
France
Tel: 33-1-45-68-39-75
Fax: 33-1-40-56-93-16
e-mail: b.hillard@unesco.org

Ms. N. Penny Holliday
WOCE IPO
Southampton Oceanography Centre
Empress Dock
Southampton SO14 3ZH
UK
Tel: 44-1703-596643
Fax: 44-1703-596204
e-mail: penny.holliday@soc.soton.ac.uk
woceipo@soc.soton.ac.uk

Mr. J. Robert Keeley
Chief - Ocean Information & Systems Development
Marine Environmental Data Service
Fisheries and Oceans - Canada
1202-200 Kent Street
Ottawa, Ont
Canada K1A 0E6
Tel: 1-613-990-0246
Fax: 1-613-990-4658
e-mail: keeley@meds-sdmm.dfo-mpo.gc.ca

Dr. Eric J. Lindstrom
Director
US GOOS Project Office
NOAA/NOS
SSMC4 N/US GOOS
1305 East-West Highway
Silver Spring, MD 20910-3281
USA
Tel: 1-301-713-3063 x139
Fax: 1-301-713-4263
e-mail: elindstrom@ocean.nos.noaa.gov

Dr. Robert Molinari
Physical Oceanography Division
NOAA/AOML
4301 Rickenbacker Causeway
Miami, FL 33149-1097
USA
Tel: 1-305-361-4344
Fax: 1-305-361-4392
e-mail: molinari@aoml.noaa.gov

Dr. Christopher Noe
SSMC4 Room 6308
NOS/OES
NOAA
1305 East-West Highway
Silver Spring, MD 20910
USA
Tel: 1-301-713-2790
Fax: 1-301-713-4499
e-mail: noe@nos.noaa.gov

Mrs. Yvette Raguenes
IFREMER/SISMER
BP70
29280 Plouzané
France
Tel: 33-98-22-42-00
Fax: 33-98-22-46-44
e-mail: yvette.raguenes@ifremer.fr

Mr. Mike Simmons
NOAA/NESDIS/NODC
SSMC3 4th Floor
1315 East-West Highway
Silver Spring, MD 20910
USA
Tel: 1-301-713-3288 x143
Fax: 1-301-713-3302
e-mail: msimmons@nodc.noaa.gov

Ms. Edwina Tanner
Data Manager
Australian Oceanographic Data Center
Maritime Headquarters
Wylde St
Potts Point, NSW 2011
Australia
Tel: 61-2-563-4806
Fax: 61-2-563-4820
e-mail: edwina@aodc.gov.au

Mr. Bertrand J. Thompson
WOCE Data Information Unit
College of Marine Studies
University of Delaware
Lewes, DE 19958-1298
USA
Tel: 1-302-645-4240
Fax: 1-302-645-4007
e-mail: thompson@diu.cms.udel.edu