HPSS Mass Storage

The High Performance Storage System (HPSS) is a modern, flexible, performance-oriented mass storage system. It has been used at NERSC for archival storage since 1998.

At NERSC, the data in storage doubles almost every year. As of January 27 2009, we have over 3.9 petabytes of data stored in over 66 million files in our HPSS User system (archive.nersc.gov) and 2.8 petabytes of data stored in over 12 million files in our HPSS Backup system (hpss.nersc.gov). HPSS sustains an average transfer rate of more than 100 MB/s, 24 hours per day, with peaks to 450 MB/s.

NERSC has two HPSS systems:

• archive
• Used for storing user files
• Default system used when no argument is given on the HSI or PFTP command on NERSC production machines.
• Better disk cache performance
• Generally delivers higher transfer rates and lower response times.
• hpss ("regent")
• Used for computer system backups.
• Initial releases of new storage software are deployed here first.
• Original NERSC HPSS production system.

Some characteristics of the NERSC HPSS systems:

• Theoretical capacity: 37 Petabytes.
• Buffer (disk) cache: 140 Terabytes.
• Theoretical maximum throughput: 3.2 GB/sec.

Users can access NERSC's HPSS machines through a variety of clients such as hsi, htar, ftp, pftp, and grid clients.