Accessing HPSS - HTAR

While HSI is a very fast and flexible tool for dealing with large data transfers, it can be slow for transferring a large number of small files or data from a stream buffer. For these cases the HTAR utility performs well. HTAR

• creates tar files directly in HPSS, thereby avoiding the extra copying and creation of a tar file on production machine filesystems
• significantly outperforms piping data, e.g. through hsi, by computing the appropriate class of service for the aggregate size of the files to be transfered
• produces easily readable indices for finding and retrieving data archived in tar files

Connecting with HTAR

Connections with HTAR operate in much the same way as in HSI. See the HSI page for information about connecting to NERSC's HPSS systems. HTAR is available on NERSC computers and is available for downloading for NERSC users to use from their local machines.

The target HPSS system is specified with the -Hserver option, e.g.:

% htar -Hserver=archive.nersc.gov -tvf blah.tar

Using HTAR

For details, see the HTAR man page.

HTAR operates much the same as the unix tar command but with the tarfile archive residing in HPSS storage. Archive creation "-c" puts data into HPSS and archive extraction brings data to your local machine.

Basic Syntax:

The core syntax for HTAR is analogous to unix tar:

	htar -{c|K|t|x|X} -f tarfile [directories],[files] 

As in the unix tar command the "-c" "-x" and "-t" options respectively function to create, extract, and list tar files. The "-K" option verifies an existing tarfile in HPSS.

One useful feature of HTAR is the creation of indexes ".idx" files. These files provide a means to find the location of files prior to retrieval of a tarfile. Using the "-X" option will cause htar to build an index file for a specifed standard tar file, so it can subsequently be used with htar.

Frequently Used HTAR Invocations

A small set of cases will satisfy most user storage needs.

The HTAR man page has more detailed information on this utility.