Accessing HPSS - HSI

HSI is a flexible and powerful interface utility to HPSS. The HSI commands are similar to those in ftp and pftp (e.g., put and mput) and UNIX (e.g., mv, mkdir, rm, cp, cd). HSI also has commands similar to those in CFS. HSI can be used both interactively or in batch scripts.

A related utility, HTAR, is useful for archiving multiple files to HPSS without using the intermediate local file storage that would be needed if one first used the tar utility followed by HSI.

For complete documentation on HSI for the archive.nersc.gov system (version 2.1+), see the HSI documentation. There are man pages for HSI on production NERSC computers.

Authentication

After September 9, 2008 NERSC HPSS will be in a transitional period where authentication for the two HPSS systems, hpss.nersc.gov (regent) and archive.nersc.gov, is handled in different ways. At some point in the future archive.nersc.gov will be upgraded to behave the same as hpss.nersc.gov.

Authentication instructions for archive.nersc.gov

The archive.nersc.gov system requires DCE authentication. Your DCE Principal (login name) is the same as your NERSC user id. Your DCE password, however, is handled differently. See the HPSS Passwords section of this document.

Your first login to archive.nersc.gov with HSI will require the use of both your DCE principal (login name) and your DCE password. After this HSI will have established a local credential on the computer from which you connect. Subsequent connections from that system will be automatically authenticated. To force generation of new credentials, use the -L option, as follows:

% hsi -L

Authentication instructions for hpss.nersc.gov

Authentication on hpss.nersc.gov (regent) is accomplished by generating a .netrc file that contains an encrypted string. See the HPSS Passwords page for details.

Connecting with HSI

From NERSC machines

To connect to the main user system (archive) from a NERSC machine:

% hsi -s archive

From NERSC machines do not specify archive.nersc.gov as this will force HSI to use the slower external network interface.

NERSC's other HPSS system - the original HPSS system at NERSC - is named "hpss". (It seemed like a good idea at the time, but now causes confusion. It is known internally as "regent"). This system is now used for backups, and does not offer the same capacity and performance as "archive." However, it does contain some older user data. To connect to it from within NERSC use the command:

% hsi -s hpss

From outside NERSC

NERSC's HPSS system can be accessed via HSI from outside the NERSC domain. The discussion above applies except the DNS (internet) names of the machines should be specified as one of:

• archive.nersc.gov
• hpss.nersc.gov

Remember that hpss.nersc.gov is intended for backup data and is not the current NERSC production machine for new users.

HSI binaries for a number of different platforms can be downloaded by NERSC users.

Starting and Using HSI

HSI can accept input several different ways; some examples:

HSI can also read from standard input and write to standard output using pipes.

For "get" and "put" operations, HSI uses a special syntax to identify and separate the local and HPSS file names:

1. The local file name is always on the left, and the HPSS file name is always on the right.
2. A ":" (colon character) always separates the local pathname from the HPSS pathname, and the colon character must be surrounded by whitespace.

Examples:

% put local_file : hpss_file  
% get local_file : hpss_file  

Recursive operations are allowed for the following commands:

    cget, chgrp, chmod, chown, cput
    delete, get, ls, mdelete, mget, mput
    put, rm, stage, touch

Wildcards are supported.

Frequently Used Commands

HSI's command set is rich, and will look familiar to users of UNIX, FTP, and other storage utilities. A small set of commands will satisfy most user storage needs.

Short List of HSI Commands by Function

HPSS File and Directory Commands

Local File and Directory Commands

File Administrative Information

Miscellaneous HSI commands