Accessing HPSS - ftp/pftp

Files can be transferred to and from HPSS via the standard internet protocol ftp and HPSS pftp utility. There is no sftp (secure ftp) or scp access.

As standard ftp clients only support authentication via the transmission of unencrypted passwords, which NERSC does not permit, special procedures must be used with ftp and pftp. The procedures are described below. The NERSC HPSS ftp daemons also support kerberos ftp clients.

pftp/ftp Authentication

After September 9, 2008 NERSC HPSS will be in a transitional period where authentication on the two HPSS systems, hpss.nersc.gov (regent) and archive.nersc.gov, is handled in different ways. See below for details on the two systems:

1. pftp/ftp Authentication on archive.nersc.gov
2. pftp/ftp Authentication on hpss.nersc.gov

At some point in the future archive.nersc.gov will be upgraded to behave the same as hpss.nersc.gov.

pftp/ftp Authentication on archive.nersc.gov

NERSC has developed an ftp access method that does not send your username/password pair over the network in plain text. Your plain text username and password will not work when you use ftp to connect to HPSS.

To be able to use ftp you must generate two text strings which contain information about your account in encrypted form. These strings are then used as your ftp "username" and "password." Each encrypted pair also contains information about the specific subnet from which they were generated. Additional encrypted pairs must be generated for each subnet from which you want to use pftp/ftp to connect to HPSS.

Encrypting your password

In the following steps, all text the user must type is shown in red.

Step 1

You need to log on to the authentication server, "auth.nersc.gov. to encrypt your username/password. If you don't know the special login/password pair to log on to this server, the information can be obtained by logging into any NERSC system and typing the command:

      module help WWW
      

Note that this special login/password pair is only for initial access to the authentication server and is not to be confused with your DCE/HPSS login and password that you will be encrypting.

Step 2

In a window (xterm) on your workstation, connect via ssh to the NERSC authentication server, "auth.nersc.gov".

  highline 10: ssh auth.nersc.gov -l {special login}
  auth@mover2.nersc.gov's password: {special password}
  <Login notice info removed>
  You are in an authentication shell
  Type help to list the commands you can run
  [auth]:

Now you are in a restricted shell that will accept only a few commands. Among them is "ftppass", which will be used in step 3. You can see the allowed commands via the "help" command:

  [auth]: help
  The following commands are the only ones recognized:
 
  ftppass    ftpproxy   chpass     help       h
  quit       q          exit
 
  For abbreviated help on commands type 'help commandname'
  The commands: q, quit and exit will all exit auth
 
  [auth]:
   

Step 3

Use the "ftppass" command to generate an encrypted_string combo of your HPSS username and password; these will be used to access pftp/ftp instead of your usual HPSS login id and password.

  [auth]: ftppass
  DCE Principal: your_HPSS_username
  DCE Password: your_HPSS_password
 
  login [encrypted_string]
  password [encrypted_string]
  
  [auth]: exit
 
  Bye
  Connection to auth.nersc.gov closed.
        

The encrypted_strings are those returned in the lines beginning with "login" and "password". These are to be used as your "login" and "password" when connecting to HPSS via ftp.

Proxy Servers

If you are behind a firewall and make pftp/ftp connections through a proxy server you can use the ftpproxy command to connect to auth.nersc.gov from one network and generate keys for another network.

The syntax for a proxy server with address 123.45.56.78 is

        [auth]: ftpproxy 123.45.56.78
        

Replace the IP address above with that of your IP proxy server.

Automatic authentication using a .netrc file

On UNIX hosts your may place your encrypted strings in a .netrc that resides in your HOME directory. This is a text file with sets of three-line entries, one for each system you wish to access, of the following form:

1. The first line specifies the name of the storage system;
2. The next two lines are the "login" and "password" lines returned by auth.nersc.gov.

For example

  machine archive.nersc.gov
  login [encrypted_string]
  password [encrypted_string]

Multiple pftp/ftp hosts can be put in the .netrc file, separated by blank lines.

Make sure the UNIX permissions for the ".netrc" file is "600" or "Owner Read-Write"; if they are anything else, the file will not be used by pftp/ftp and the process will not work.

When you have stored your encrypted_strings in your .netrc file, you will not need to type in your username/password combination to gain pftp/ftp access to HPSS.

ftp/pftp Authentication on hpss.nersc.gov

The hpss.nersc.gov system has a special encrypted password for ftp and pftp, but HSI and HTAR currently use the same password and .netrc file. See special process for encrypting username/password pairs. for screenshots and instructions on how to setup a password or .netrc file.