Bassi File Systems

• 1. Your $HOME Directory
• 2. Your $SCRATCH Directory
• 3. The project Directory
• 4. Quota Summary
  • 4.1 Bassi /scratch Disk Space Usage

All user-accessible file storage is provided by the General Parallel File System (GPFS), which provides a large amount of fast data storage and is available from all nodes. User home directories and a large amount of scratch space are both provided by GPFS.

1. Your $HOME Directory

When you log in, you are put into your home directory by default. Home directories are available from all nodes. Your home directory is owned by you (your NERSC username) and the group ownership is your personal file group, which has the same name as your login name. You are the only member of that file group.

Your home directory can (and should) always be referred to by the environment variable $HOME. The absolute path to your home directory (e.g., /u4/joe/) may change, but the value of $HOME will always be correct.

Home directories are not backed up (except for file system recovery purposes). Please save all your important files to HPSS on a regular basis.

For security reasons, you should never allow "world write" access to your $HOME directory or your $HOME/.ssh directory. NERSC scans for such security weakness, and, if detected, will change the permissions on your directories.

In your home directory are various login control files (e.g. ".login", ".cshrc", ".profile"). These are symbolic links to common files that contain definitions shared by all users. Please do not remove the links in your home directory. If you wish to customize your login behavior, please place all your personal customizations in extension files with names such as .login.ext, .cshrc.ext, and .profile.ext.

The home directories are part of the General Parallel File System (GPFS). Each user has a quota of allowed disk space usage in $HOME and a separate quota for the number of inodes. Each file or directory that you own counts as one inode against your quota. The myquota command (with no options) will give you information on the limits in your $HOME directory. For example:

% myquota
            ---------- Block (GB) ----------  ------------ Inode -------------
FileSystem   Usage    Quota   InDoubt  Grace   Usage    Quota   InDoubt  Grace
----------  -------  -------  -------  -----  -------  -------  -------  -----
scratch          33      250       11      -    33485    50000      699      -
u2                0        5        0      -     1094     7500       99      -

The output shows the limit on file space (Block Quota) and inodes (Inode Quota), as well as the current usage. If you reach the "Quota" value, you will not be able to save anything else to disk in that file system. In this example the home directory is in /u2. The "InDoubt" columns tells you that the reported "Usage" may be off by that amount. This occurs because GPFS does not update this information immediately upon change.

2. Your $SCRATCH Directory

$SCRATCH

The environment variable $SCRATCH refers to a user's directory space in the /scratch GPFS file system. $SCRATCH is available from all nodes and is tuned to be higher performing than $HOME. The contents of $SCRATCH may be deleted at any time after the job finishes if the system's disks are near capacity. In general, files in $SCRATCH will persist for at least 7 days, but users are "taking chances" by using $SCRATCH to store after the job finishes and should not rely on it to be "semi-permanent" file storage space.

There is a (large) quota for users on the /scratch file system. Use the myquota command to check your usage and quota.

/tmp

The /tmp file system is local to each node and is small. DO NOT USE /tmp; it may crash your node!

Some software, such as the compilers and editors, will try to use /tmp unless the user has specified a value for the environment variable TMPDIR. NERSC has set TMPDIR to be SCRATCH. If TMPDIR is not defined, user Fortran codes which open files with status="scratch" will write those files into /tmp.

3. The project Directory

4. Quota Summary

NERSC sometimes grants temporary quota increases for legitimate purposes. To apply for such an increase, please see Disk Quota Change Request Form.

4.1 Bassi /scratch Disk Space Usage