HPSS Mass StorageRelated InformationHPSS Software DownloadsHPSS Collaboration |
Accessing HPSS - ftp/pftpFiles can be transferred to and from HPSS via the standard internet protocol ftp and HPSS pftp utility. There is no sftp (secure ftp) or scp access. As standard ftp clients only support authentication via the transmission of unencrypted passwords, which NERSC does not permit, special procedures must be used with ftp and pftp. The procedures are described below. The NERSC HPSS ftp daemons also support kerberos ftp clients. PFTPPFTP is a variant of ftp which is available on NERSC systems. It is better than ftp for large file transfers (> 100 MB) because it is multi-threaded and has some tuning parameters available for transfers. PFTP has the advantage of being compatible with NERSC "sleepers," which will gracefully suspend connections when HPSS is down or unavailable. pftp/ftp AuthenticationAfter September 9, 2008 NERSC HPSS will be in a transitional period where authentication on the two HPSS systems, hpss.nersc.gov (regent) and archive.nersc.gov, is handled in different ways. See below for details on the two systems: At some point in the future archive.nersc.gov will be upgraded to behave the same as hpss.nersc.gov.
pftp/ftp Authentication on archive.nersc.govNERSC has developed an ftp access method that does not send your username/password pair over the network in plain text. Your plain text username and password will not work when you use ftp to connect to HPSS. To be able to use ftp you must generate two text strings which contain information about your account in encrypted form. These strings are then used as your ftp "username" and "password." Each encrypted pair also contains information about the specific subnet from which they were generated. Additional encrypted pairs must be generated for each subnet from which you want to use pftp/ftp to connect to HPSS. Encrypting your passwordIn the example to follow, this machine is named "highline".In the following steps, all text the user must type is shown in red. Step 1You need to log on to the authentication server, "auth.nersc.gov. to encrypt your username/password. If you don't know the special login/password pair to log on to this server, the information can be obtained by logging into any NERSC system and typing the command: module help WWW Note that this special login/password pair is only for initial access to the authentication server and is not to be confused with your DCE/HPSS login and password that you will be encrypting. Step 2In a window (xterm) on your workstation, connect via ssh to the NERSC authentication server, "auth.nersc.gov". highline 10: ssh auth.nersc.gov -l {special login} auth@mover2.nersc.gov's password: {special password} <Login notice info removed> You are in an authentication shell Type help to list the commands you can run [auth]: Now you are in a restricted shell that will accept only a few commands. Among them is "ftppass", which will be used in step 3. You can see the allowed commands via the "help" command: [auth]: help The following commands are the only ones recognized: ftppass ftpproxy chpass help h quit q exit For abbreviated help on commands type 'help commandname' The commands: q, quit and exit will all exit auth [auth]: Step 3Use the "ftppass" command to generate an encrypted_string combo of your HPSS username and password; these will be used to access pftp/ftp instead of your usual HPSS login id and password. [auth]: ftppass DCE Principal: your_HPSS_username DCE Password: your_HPSS_password login [encrypted_string] password [encrypted_string] [auth]: exit Bye Connection to auth.nersc.gov closed. The encrypted_strings are those returned in the lines beginning with "login" and "password". These are to be used as your "login" and "password" when connecting to HPSS via ftp. Proxy ServersIf you are behind a firewall and make pftp/ftp connections through a proxy server you can use the ftpproxy command to connect to auth.nersc.gov from one network and generate keys for another network. The syntax for a proxy server with address 123.45.56.78 is [auth]: ftpproxy 123.45.56.78 Replace the IP address above with that of your IP proxy server. Automatic authentication using a .netrc fileOn UNIX hosts your may place your encrypted strings in a .netrc that resides in your HOME directory. This is a text file with sets of three-line entries, one for each system you wish to access, of the following form:
For example
machine archive.nersc.gov login [encrypted_string] password [encrypted_string] Multiple pftp/ftp hosts can be put in the .netrc file, separated by blank lines. Make sure the UNIX permissions for the ".netrc" file is "600" or "Owner Read-Write"; if they are anything else, the file will not be used by pftp/ftp and the process will not work. When you have stored your encrypted_strings in your .netrc file, you will not need to type in your username/password combination to gain pftp/ftp access to HPSS. ftp/pftp Authentication on hpss.nersc.govThe hpss.nersc.gov system has a special encrypted password for ftp and pftp, but HSI and HTAR currently use the same password and .netrc file. See special process for encrypting username/password pairs. for screenshots and instructions on how to setup a password or .netrc file. |
Page last modified: Wed, 15 Oct 2008 16:19:47 GMT Page URL: http://www.nersc.gov/nusers/systems/hpss/usage_ftp.php Web contact: webmaster@nersc.gov Computing questions: consult@nersc.gov Privacy and Security Notice |