Data Analytics
Overview
NERSC engages in research and development to ensure highly-scalable, productive data, AI and analytics tools are available and deployed at NERSC; as well as to promote the use of cutting-edge data and analytics approaches and technologies in science. Recent activity is outlined below.
Python and Jupyter
Python continues to be popular with users- we've measured more than 2k unique users in 2020. We have also deployed new NERSC monitoring that captures all Python library imports for each user. We are working to build infrastructure to analyze this large amount of data and also gain insights about how people are using Python at NERSC. An additional R&D thrust is Python GPU preparedness. We are working to understand and test the various Python GPU frameworks to help our users transition their Python code to Perlmutter. Through this work we lead the Python HPC community - e.g. through co-chairing the HPC track at SciPy (https://www.scipy2020.scipy.org/talk-poster-presentations)
Jupyter
The Jupyter interactive environment is enabling a new mode of computing for scientists at NERSC. Scientists love Jupyter because it combines documentation, visualization, data analytics, and code into a document they can share, modify, and even publish.
DAS and the Usable Software Systems group in CRD are partnering to enhance the Jupyter framework for high performance scientific computing environments, and developing new capabilities in Jupyter for the next generation of scientific analysis. NERSC and CRD have taken a collaborative approach to our development of Jupyter-centric tools, where we collaborate with scientific partners to develop tools that are useful to the broader community, and deploy these for all users at NERSC.
Use cases from scientific collaborations such as NCEM, LCLS and others provide us with insights into how Jupyter Notebooks can help make HPC more accessible to scientists. These collaborations also identify general patterns for enhancements and tools the the entirety of the NERSC user base can capitalize on. Other topics of the research collaboration include finding ways to leverage Jupyter notebooks, containers, and best practices for software development to address reproducibility in science and HPC.
AI and Machine Learning
MLPerf HPC
We developed first set of HPC AI benchmarks in MLPerf, the industry standard AI benchmarking effort with broad support from industry and academia. The benchmark suite was presented at the SC19 BOF, “Benchmarking Machine Learning Ecosystem on HPC Environment”, the 2020 ECP annual meeting and the 2020 ISPASS MLBench workshop, as well as being featured on InsideHPC. For more information, see https://github.com/mlperf-hpc/mlperf-hpc.
Deep Learning Applications
NERSC is involved in several projects to push the state-of-the-art in deep learning for science. Recent publications include:
- Jiang et al., 2020, "MeshfreeFlowNet: A Physics-Constrained Deep Continuous Space-Time Super-Resolution Framework" - https://arxiv.org/abs/2005.01463, accepted for publication at SC20 and Best Sudent Paper finalist
- Chattopadhyay et al., 2020, "Deep spatial transformers for autoregressive data-driven forecasting of geophysical turbulence" - https://eartharxiv.org/cqmb2/, accepted for publication at Climate Informatics 2020
- Mayur Mudigonda et al., 2020, "Climatenet: Bringing The Power Of Deep Learning To Weather And Climate Sciences Via Open Datasets And Architectures", ICLR 2020
- Karthik Kashinath, Mayur Mudigonda, Kevin Yang, Jiayi Chen, Annette Greiner and Prabhat Prabhat, 2019, "ClimateNet: Bringing the power of Deep Learning to weather and climate sciences via open datasets and architectures", Proceedings of the 9th International Workshop on Climate Informatics: CI 2019 (No. NCAR/TN-561+PROC). doi:10.5065/y82j-f154
- Wang et al., 2020, "Towards Physics-informed Deep Learning for Turbulent Flow Prediction", KDD, https://doi.org/10.1145/3394486.3403198
- Jiang et al., 2020, "Enforcing Physical Constraints in CNNs through Differentiable PDE Layer", ICLR 2020 Workshop on Integration of Deep Neural Models and Differential Equations
- Muszynski et al., 2020, "Atmospheric Blocking Pattern Recognition in Global Climate Model Simulation Data", ICPR200
- Wu et al., 2020, "Enforcing statistical constraints in generative adversarial networks for modeling chaotic dynamical systems", JCP, Volume 406, 109209 https://doi.org/10.1016/j.jcp.2019.109209
- Toms et al., 2020, Testing the Reliability of Interpretable Neural Networks in Geoscience Using the Madden-Julian Oscillation, Geosci. Model Dev
- "ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning" - https://arxiv.org/abs/2006.00719 , submitted to triple AAAI
- Prabhat et al., "ClimateNet: an expert-labelled open dataset and Deep Learning architecture for enabling high-precision analyses of extreme weather", Geoscientific Model Development.
- Kashinath et al., 2020, "Physics-informed knowledge-guided Machine Learning for weather and climate modeling: progress and challenges", Submitted to the Proceedings of the Royal Society
- Pathak et al., 2020, Using Machine Learning to Augment Coarse-Grid Computational Fluid Dynamics Simulations, https://arxiv.org/abs/2010.00072
- Pathak et al., 2020, ML-PDE: A Framework for a Machine Learning Enhanced PDE Solver, NeurIPS ML4PS
- Hayat et al. Estimating Galactic Distances From Images UsingSelf-supervised Representation Learning, NEeurIPS ML4PS
Climate Analytics
NERSC has been involved in several Climate Analytics applications including "Exascale Deep Learning for Climate Analytics" that was the first deep learniing application to achieve 1 Exaflop (FP16) with Tensorflow and Horovod and won the Gordon Bell Prize (Kurth et al. SC18)
Recently ClimateNet was launched to bring the power of Deep Learning to the climate community by creating community-sourced open-access expert-labeled datasets and architectures for improved accuracy and performance, details of which were presented at ILCR and other venues.
ClimateNet
I/O and Data Management
ExaHDF5
The ExaHDF5 project will productize features and techniques prototyped in earlier projects, explore optimization strategies on upcoming architectures, maintain and optimize existing HDF5 features for ECP applications, and release these new features in HDF5 for broad deployment on HPC systems. Focusing on the challenges of exascale I/O, we will develop technologies based on the massively parallel storage hierarchies that are being built into pre-exascale systems. We will enhance HDF5 software to achieve efficient parallel I/O on exascale systems in ways that will impact a large number of DOE science applications. Read More
Proactive Data Containers (PDC)
Moving toward new paradigms for SSIO in the extreme-scale era, the Proactive Data Container (PDC) project proposes to investigate novel object-based data abstractions and storage mechanisms that take advantage of the deep storage hierarchy and enable proactive automated performance tuning. In order to achieve these overarching goals, we propose a fundamental new data abstraction, called Proactive Data Containers (PDC). A PDC is a container within a locus of storage (memory, NVRAM, disk, etc.) that stores science data in an object-oriented manner. Managing data as objects enables powerful optimization opportunities for data movement and transformations. In this project, we will research: 1) formulation of object-oriented PDCs and their mapping in different levels of the exascale storage hierarchy; 2) efficient strategies for moving data in deep storage hierarchies using PDCs; 3) techniques for transforming and reorganizing data based on application requirements; and 4) novel analysis paradigms for enabling data transformations and user-defined analysis on data in PDCs. The intent of our research is to move the field of HPC SSIO in a direction where it may ultimately be possible to develop scientific applications without the need to perform cumbersome and inefficient tuning to optimize data movement on every system the application runs on. Read More
