Oak Ridge Leadership Computing Facility

The High Performance Storage System (HPSS) at the OLCF provides longer-term storage for the large amounts of data created on the OLCF compute systems. The mass storage facility consists of tape and disk storage components, servers, and the HPSS software.

Each OLCF user receives an HPSS account on automatically. Users can transfer data to HPSS from any OLCF system using the HSI or HTAR utilities. Initially, data transferred to HPSS is written to disk; the system then migrates the data to tape for longer-term archival.

HPSS has (3) SL8500 tape libraries, each holding up to 10,000 cartridges. The libraries house a total of (24) T10K-A tape drives (500 GB cartridges, uncompressed) and (36) T10K-B tape drives (1 TB cartridges, uncompressed). Each drive has a bandwidth of 120 MB/s.

ORNL’s work in mass storage began in the early 1990s to support the Atmospheric Radiation Measurement project and to provide storage for simulation results generated on the NCCS’s Paragon supercomputers. To support those projects, ORNL acquired and ran the NSL UniTree storage management product.

In 1993 it became clear that NSL UniTree would eventually become inadequate for the projected needs of the NCCS. At that time a follow-on to NSL UniTree, known as HPSS, was being designed by IBM and a collaboration of Department of Energy (DOE) national laboratories (Sandia, Livermore, and Los Alamos). ORNL joined that collaboration and took on responsibility for the storage system management (SSM) portion of the product, for which the ORNL HPSS development team continues to be responsible.

ORNL continued with NSL UniTree production use until 1997, at which time the conversion to HPSS was completed. Also in 1997, HPSS won an R&D 100 Award based on an entry initiated and prepared at ORNL.

In 1998 a proposal to establish a storage testbed was developed and sent to the Mathematics, Information and Computer Sciences (MICS) office of DOE; that testbed was named Probe. It was funded in fiscal year 1999 as a collaboration between ORNL and the National Energy Research Scientific Computing Center (known as NERSC).

In late 2002 the ORNL storage group changed the tape robotics to StorageTek Powderhorn libraries using 9840 and 9940 tape drives and cartridges. This vastly increased the NCCS’ storage capacity in response to the relentless accumulation of user data.

In 2006, total data stored in HPSS at ORNL surpassed 1 petabyte for the first time. After taking 8.5 years to store 1 petabyte, the second petabyte took less than 2 years and the third took only 6 months; it was clear that users were storing data at an increasing rate.

As storage, network, and computing technologies continue to change the OLCF’s storage system evolves to take advantage of new equipment that is both more capable and more cost-effective. 2010 saw the addition of another SL8500 which holds (10) petabytes of data with the current generation of tape drives. The tape library provides room to store and use up to 40,000 tape cartridges.