What is the NOAA Big Data Program?

The NOAA Big Data Program (formerly known as the Big Data Project during CRADA phase) is a combination of three incredibly powerful resources: the expansive collection of high-quality environmental data and expertise of NOAA, the vast infrastructure and scalable computing capabilities of our industry collaborators, and innovation and energy of the American economy. Under the BDP, NOAA works with Infrastructure-as-a-Service (IaaS) providers to broaden access to NOAA’s data resources. The partnerships, are designed in order to facilitate full and open data access at no net cost to the taxpayer, helps bring together these resources to make NOAA's data more publicly accessible in order to foster innovation and collaboration.

In what way is the Big Data Program different from what NOAA is already doing with its data?

The Big Data Program (BDP) differentiates itself from NOAA's current offerings by positioning NOAA's data within modern platforms with high-performance computer, analytic, and storage services to create a new economic space and expand public access to the data. NOAA has a long and well-established history of sharing its environmental data with private industry, the academic research community, and the general public. There are no current plans to change these existing offerings. Instead, the BDP is designed to augment these by creating another means of access to NOAA's open data resources. 

What kind of access to NOAA data will the general public have?

The purpose of the BDP is to expand upon the existing full and open access to NOAA’s data to all parties, including the general public. The general public can access data made available through the Cloud Service Providers (CSPs) for free, and the data is available to the public without restriction on use. These data will include the original and unaltered NOAA data content, and may additionally be provided using any broadly accessible and open format or formats.

Will this change how the public receives its environmental data for its day-to-day purposes?

No. The only change this will have on how the public currently receives its environmental data is by providing an additional means of access to the data. While data remains accessible through NOAA’s existing data access portals, they will now be co-located with powerful cloud computing platforms to make its analysis more readily available to everyone. Based on the initial results, this means that new applications can be developed faster with a potentially greater speed of service while also reducing the load on NOAA’s servers and networks. 

In addition to the program being conducted at no net cost to the taxpayer, it will not alter the day-to-day operations within NOAA. This means that all of NOAA’s offerings, from the National Weather Service 122 weather forecast offices providing daily weather predictions to the National Ocean Service’s work monitoring changes in the coastline, will remain available to the public through the same channels on a day-to-day basis.

What is the role of the Data Broker?

North Carolina Institute for Climate Studies (previously CICS-NC) which is now known as the Cooperative Institute for Satellite Earth System Studies (CISESS) is a partner of the BDP, and acts as a “data broker” between NOAA and the Cloud Service Providers (CSPs). Their data and information technology experts play key roles in the transfer and certification of data sets, and their high-performance computing cluster serves as a critical gateway for delivering data to the Cloud. NOAA delivers copies of data to CSPs for scalable public access - a “one-to-several-to-many” data access model.  Marginal NOAA activities and labor costs associated with data delivery from NOAA to the cloud have been collapsed into a “data broker” role which has been identified as a key function for the project.

Will the public have to pay for data that is currently accessible to them free of charge?

No. NOAA’s data will remain free and available to the public without restriction on use, including through existing NOAA portals. The BDP Cloud Service Providers (CSPs) are tasked with distributing the original data content only. Any use of value-added products on the CSP's platform that use NOAA data, can incur a charge for those products in the same way that they do today. However, a CSP must ensure everyone wishing to access the data is given the same opportunity to do.