Oak Ridge Leadership Computing Facility

Researchers can multitask and access files simultaneously with version 1.4

Researchers using supercomputers to glean insight into scientific problems can spend much of their time just trying to get massive amounts of data in and out of a machine. Oak Ridge National Laboratory’s (ORNL’s) Scientific Data Group works to minimize this process, and this week its members took a big step forward by releasing its newest update to the Adaptable Input/Output System (ADIOS), version 1.4, which allows users more time to focus on achieving scientific insight and less on managing data.

“Our goal is to take our research and put it in production,” says Scott Klasky, group leader of the Scientific Data Group at ORNL. Klasky’s group researches methods to streamline this process, called input/output (I/O), on supercomputers.

The group’s project started in 2008, when Klasky and his team created ADIOS to help increase by ten-fold I/O on the Oak Ridge Leadership Computing Facility’s (OLCF’s) Jaguar supercomputer. Four updates and 65 journal publications later, Klasky’s team is still looking for ways to make I/O even more efficient.

Version 1.4 represents a fundamental shift in the middleware that goes in tandem with shifts in supercomputing architectures. The Cray XT5 Jaguar, capable of 3.3 thousand trillion calculations per second, is being overhauled and transformed into a Cray XE6 dubbed Titan. The machine will be capable of 20 thousand trillion calculations per second by using a combination of traditional central processing units (CPUs) and fast and efficient graphics processing units (GPUs).

Supercomputers could only efficiently perform one task at a time during I/O until recently. The 1.4 release lays a foundation for future generations of exascale supercomputers to be able to run multiple tasks simultaneously. Researchers can now theoretically begin doing visualizations with data as their supercomputers continue running codes for mathematical models. This staging, as it is called, was spearheaded by ORNL’s Hasan Abbasi with Georgia Tech researchers Karsten Schwan, Matthew Wolf, Fang Zheng, and Greg Eisenhauer and Rutgers University researcher Manish Parashar and his students Ciprian Docan, Tong Jin, and Fan Zhang.

Another major upgrade, developed by ORNL’s Qing Liu and Norbert Podhorszki, increases ADIOS’s speed for reading data on a supercomputer. This staged reading uses subfiles in application codes so researchers can quickly access their data. Often times, researchers only want to access a small slice of their data, and postdoctoral research Yuan Tian, with her adviser, Auburn University’s Weikuan Yu, developed Smart I/O, a tool within the greater ADIOS framework, to help researchers access slices of data while simultaneously performing other calculations. She received first place in the Association for Computing Machinery’s Student Research Competition for this work.

Roselyne Tchoua has worked to make ADIOS easy for every researcher to use. Her focus for the most recent release is universalizing the middleware so researchers can easily insert their information no matter what terms, or schema, they use to describe them.

In addition to the the Scientific Data Group, ADIOS collaborators include Jeremy Logan, Sean Ahern, Luis Chacon, Dave Pugmire, and Ricky Kendall of ORNL; Joel Saltz and Tahsin Kurc of Emory University; Jong Choi of the University of Tennessee–Knoxville; William Tang, C.S. Chang, Seong-Hoe Ku, Weixing Wang, and Stephane Ethier of the Princeton Plasma Physics Laboratory; Zhihong Lin of the University of California–Irvine; Ray Grout of National Renewable Energy Laboratory; Jackie Chen, Jay Lofstead, Ron Oldfield, Ken Moreland, and Todd Kordenbrock of Sandia National Laboratories; Xiaosong Ma, Nagiza Samatova, and Sriram Lakshminarasimhan of North Carolina State University; and Wei Xue of Tshinghua University.—by Eric Gedenk