About

HPSS Mass Storage

The High Performance Storage System (HPSS) is a modern, flexible, performance-oriented mass storage system. It has been used at NERSC for archival storage since 1998.

HPSS is Herarchical Storage Management (HSM) software developed by a collaboration of DOE labs, of which NERSC is a participant, and IBM.  The HSM software enables all user data to be ingested onto high performance disk arrays and automatically migrated to a very large enterprise tape subsystem for long-term retention.  The disk cache in HPSS is designed to retain 5 days worth of new data and the tape subsystem is designed to provide the most cost-effective long-term scalable data storage available.

NERSC has two HPSS systems:

• Archive is used for storing user files. This sytem is the default archival storage system and has excellent disk cache performance and high transfer rates. 
• "hpss" is used for computer system backups. It is used to test new system software and holds some user files from the time when it was NERSC's only HPSS system.

Some characteristics of the NERSC HPSS systems (January 2013)

• Data stored in archive system: 24 PB, >128 million files
• Data stored in backup system: 14 PB, >15 million files
• Growth Rate: 1 PB/month
• Current Maximum capacity: 240 Petabytes.
• Buffer (disk) cache: 288 Terabytes.
• Average transfer rate: 100 MB/sec
• Peak measured transfer rate: 1 GB/sec

Users can access NERSC's HPSS machines through a variety of clients such as hsi, htar, ftp, pftp, and grid clients.  For more information about the specific features of each HPSS client see Accessing HPSS.