Franklin File Systems

• $HOME directory
• $SCRATCH directory
• The project directory
• Quota Summary
• Scratch Space Usage History

All user-accessible file storage is provided by the Lustre File System, which provides a scalable, high performance data storage and is available from all nodes. User home directories and a large amount of scratch space are both provided by Lustre.

$HOME Directory

At log in, users are in the $HOME directory by default. Home directories are available from all nodes. Home directories are owned by the user (NERSC username) and the group ownership is the personal file group (also the NERSC username). Personal filegroups contain only a single user.

Home directories can (and should) always be referred to by the environment variable $HOME. The absolute path to a home directory (e.g., /u4/joe/) may change, but the value of $HOME will always be correct.

Home directories are not currently backed up, although we intend to perform back ups for the purposes of catastrophic file system recovery when the tools required become available. Please save all important files to HPSS on a regular basis.

For security reasons, never allow "world write" access to a $HOME directory or $HOME/.ssh directory. NERSC scans for such security weakness, and, if detected, will change the permissions on the directories to disable "world write" access.

The home directory contains various login control files (e.g. ".login", ".cshrc", ".profile"). These are symbolic links to common files that contain definitions shared by all users. Users should not remove or modify these links. Customizations of login behavior should be specified in extension files with names such as .login.ext, .cshrc.ext, and .profile.ext.

The home directories are part of the Lustre Parallel File System. Each user has a quota of allowed disk space usage in $HOME and a separate quota for the number of inodes. Each file or directory that owned by a user counts as one inode against the quota. The myquota command (with no options) will display information on the limits in the $HOME directory. For example:

% myquota  
Displaying quota usage for user aaa:
            -- BLOCK (GB) ---   --- INODE ------
FileSystem   Usage    Quota      Usage    Quota  
----------  -------  -------    -------  -------
u0                4       15      1094    25000 
scratch         133      250     33485    50000

The output shows the limit on file space (Block Quota) and inodes (Inode Quota), as well as the current usage. If a "Quota" value is reached, further storage on that file system will be blocked. In this example the home directory is in /u0.

$SCRATCH Directory

$SCRATCH

The environment variable $SCRATCH refers to a user's directory space in the /scratch Lustre file system. $SCRATCH is available from all nodes and is tuned to be higher performing than $HOME. The contents of $SCRATCH may be deleted at any time after the job finishes if the system's disks are near capacity. In general, files in $SCRATCH will persist for at least seven days, but users are at risk of files being purged when using $SCRATCH to store files after the job finishes. Users should not rely on it to be "semi-permanent" file storage space.

There is a (large) quota for users on the /scratch file system. Use the myquota command to check usage and quota.

/tmp

The /tmp file system is local to each node and is small. DO NOT USE /tmp; it may crash the node if /tmp is filled with user files and the systems cannot obtain enough space.

Some software, such as the compilers and editors, will try to use /tmp unless the user has specified a value for the environment variable TMPDIR. NERSC has set TMPDIR to be SCRATCH. If TMPDIR is not defined, user Fortran codes which open files with status="scratch" will write those files into /tmp.

The $PROJECT Directory

Quota Summary

NERSC sometimes grants temporary quota increases for legitimate purposes. To apply for such an increase, please see Disk Quota Change Request Form.

Franklin /scratch Usage

Franklin /u0 ($HOME) Usage