Continuous Evolution

ESDS science products are evolved through a combination of competitive awards and sustained and strategic investments.

Earth Science Open Data and Open Source Policy
Cloud Evolution
Interagency Implementation and Advanced Concepts Team (IMPACT)
Competitive Programs
Science Tools and Portals
Data Access Systems
Earth Science Data System Working Groups (ESDSWG)
Program Review

Earth Science Open Data and Open Source Policy

NASA’s Earth observation data are collected continuously. For over half a century, these invaluable records of Earth processes have provided a critical resource for scientists and researchers. Since 1994, all data produced by NASA, including the code and algorithms used to produce these data, have been free and open to all users for any purpose as quickly as practical after instrument checkout and calibration.

Starting in 2015, all data systems software developed through research and technology awards, including the competitive programs mentioned below, or through in-house development has been made available to the public as Open Source Software.

Cloud Evolution

Over the past 25 years, NASA’s Earth Observing System Data and Information System (EOSDIS) has continuously evolved to take advantage of new and innovative technology to meet the increasing demands of data providers and users. By 2022, the ingest rate of data into the EOSDIS archive is projected to grow from the current 3.9 petabytes (PB) per year to as much as 47.7 PB per year. As this ingest rate increases, the total volume of data stored in the EOSDIS archive is also expected to grow—from its current size of 27 PB to more than 37 PB by 2020; by 2025, the volume of data in the EOSDIS archive is projected to be 250 PB.

This anticipated growth in both the data ingest rate as well as the overall archive volume poses challenges for archiving, distribution and analysis of data. To address these challenges, the ESDS program has adopted a strategic vision to develop and operate multiple components of EOSDIS in a commercial cloud environment that will meet the needs of future missions with high data volumes-such as Surface Water Ocean Topography (SWOT) and NASA-ISRO Synthetic Aperture Radar (NISAR)-and provide improved data management and user access for many ongoing Earth science missions.

ESDS has completed an evaluation of the relative cost, technical performance, and security implications of utilizing the Amazon Web Services commercial cloud environment for ingest, archive, management, and distribution of data. Based on the findings of that evaluation, the program has implemented "Cumulus," an open source, cloud-based framework.

Benefits of adopting a cloud-based architecture:

Improve the efficiency of NASA’s data systems operations – continues free and open access to data
Prepare for planned high-data-rate missions
Increase opportunity for researchers and commercial users to access/process PBs of data quickly without the need for data management
Transparent/extendable open source processing framework

The platform has been deployed for testing with the SWOT Science Data System and Distributed Active Archive Center (DAAC) at NASA's Jet Propulsion Laboratory and is integrated with the NASA Compliant General Application Platform (NGAP).

Interagency Implementation and Advanced Concepts Team (IMPACT)

NASA's IMPACT develops portals and new toolsets, including a deep learning-based hurricane intensity estimation portal prototype to objectively and accurately estimate hurricane wind speeds in real time and display the estimates in a situational awareness portal.

The Analysis and Review of CMR (ARC) team’s metadata review tool was developed to review NASA Earth Observing System Data and Information System (EOSDIS) collection and granule level metadata in the Common Metadata Repository (CMR) for correctness, completeness, and consistency in order to improve data discovery and accessibility.

IMPACT supports flight missions through the Airborne Data Management Group (ADMG), which ensures that airborne scientific data gathered in flight missions are discoverable and usable to the airborne and other scientific communities.

Competitive Programs

Competitive awards provide a constant source of innovation to improve data products, advance systems, and nurture interdisciplinary tools. ESDS Program Competitive Award Programs include:

Advancing Collaborative Connections for Earth System Science (ACCESS)
NASA's ACCESS Program develops and implements technologies to effectively manage, discover, and utilize NASA’s archive of Earth observations for scientific research and applications. Program objectives are to enhance and improve existing components of the distributed and heterogeneous data and information systems infrastructure (EOSDIS) that support NASA's Earth science research goals.

Visit the ACCESS Program page.

Citizen Science for Earth Systems Program (CSESP)

NASA's CSESP is focused on developing and implementing projects that harness contributions from members of the general public to advance our understanding of the Earth as a system. Recent selected projects focus on air quality, snow observations, animal biodiversity, measuring aerosol optical depth, tracking water storage, and global kelp cover.

Visit the CSESP page.

Making Earth System Data Records for Use in Research Environments (MEaSUREs) Program
Through the MEaSUREs Program, NASA is continuing its commitment to expand understanding the Earth system using consistent records. Program objectives: are to develop long-term, consistent, and calibrated data and products that are valid across multiple missions and satellite sensors. Awarded projects provide Earth science data products and services driven by NASA’s Earth science goals and contributing to advancing NASA Earth system “missions to measurements” concept.

Visit the MEaSUREs Program page.

Science Tools and Portals

Science portals comprehensively address and communicate information about a specific scientific topic in collaboration with research and analysis programs. The first of these portals, the Sea Level Change Portal (SLCP) provides a gateway to information about regional and global sea level change. The site includes access to data, analysis tools, and simulations of glaciers, ice sheets, sea level and the solid Earth.

Data Access Systems

ESDS continues to develop and enhance tools and systems to improve access to data. These systems allow users to find, download, and visualize data appropriate for their needs.

Data Access Systems include:

Earth Science Data System Working Groups (ESDSWG)

ESDSWG provides a forum for Principal Investigators from the competitive programs and system managers and engineers from EOSDIS to colIaborate and infuse new ideas and technology cooperatively. Membership is comprised of working groups organized around key technology and information system issues. Membership is limited to NASA-funded researchers and EOSDIS personnel.

Program Review

Significant evolutionary activities are initiated as a result of program reviews. In 2015, NASA proactively tasked the Earth Science Division (ESD) / Earth Science Data and Information Systems (ESDIS) Project to sponsor an independent, holistic review to study potential efficiencies and enhanced capabilities from a variety of perspectives based on:

Science discipline
Optimizing common data operation tasks across NASA DAACs