Franklin User Environment

Operating System

Franklin's compute nodes run a light weight Linux-based OS, Compute Node Linux (CNL), which is optimized for application performance. CNL interacts with with an application process in a limited way, including managing virtual memory addressing, providing memory protection and performing basic scheduling. CNL reduces system overhead, ensures reproducible run-times for MPP jobs, and high performance, low latency MPI and SHMEM communications, which are critical for the system to scale to thousands of processors.

User Environment

Shell Initialization Files

Environment initialization files ("dot-files") in home directories have been configured for maximum flexibility and ease of maintenance, both for the user and for NERSC staff. A long listing of a newly-created home directory (for user aaa) should look similar to this:

% ls -al
total 156
drwx------  4 aaa  aaa  4096 Feb  2 13:39 .
drwxr-xr-x  3 bin  bin  4096 Feb  2 13:39 ..
lrwxrwxrwx  1 aaa  aaa    38 Feb  2 13:39 .bash_profile -> /usr/common/usg/etc/skel/.bash_profile
-rw-------  1 aaa  aaa   157 Feb  2 13:39 .bash_profile.ext
lrwxrwxrwx  1 aaa  aaa    32 Feb  2 13:39 .bashrc -> /usr/common/usg/etc/skel/.bashrc
-rw-------  1 aaa  aaa   141 Feb  2 13:39 .bashrc.ext
lrwxrwxrwx  1 aaa  aaa    31 Feb  2 13:39 .cshrc -> /usr/common/usg/etc/skel/.cshrc
-rw-------  1 aaa  aaa   160 Feb  2 13:39 .cshrc.ext
lrwxrwxrwx  1 aaa  aaa    31 Feb  2 13:39 .kshrc -> /usr/common/usg/etc/skel/.kshrc
-rw-------  1 aaa  aaa   138 Feb  2 13:39 .kshrc.ext
lrwxrwxrwx  1 aaa  aaa    31 Feb  2 13:39 .login -> /usr/common/usg/etc/skel/.login
-rw-------  1 aaa  aaa   159 Feb  2 13:39 .login.ext
lrwxrwxrwx  1 aaa  aaa    33 Feb  2 13:39 .profile -> /usr/common/usg/etc/skel/.profile
-rw-------  1 aaa  aaa   144 Feb  2 13:39 .profile.ext
lrwxrwxrwx  1 aaa  aaa    32 Feb  2 13:39 .tcshrc -> /usr/common/usg/etc/skel/.tcshrc
-rw-------  1 aaa  aaa   162 Feb  2 13:39 .tcshrc.ext

The files .bash_profile, .bashrc, .cshrc, .kshenv, .login, .profile, and .tcshrc are links to read-only files, and should not be deleted. All individual customizations (aliases, environment variables, etc.) should be made in the files named .bashrc.ext, .cshrc.ext, .kshenv.ext, .login.ext, .profile.ext, and .tcshrc.ext. These .ext files are sourced by the corresponding dot-files.

If these links and .ext files are missing, please run the fixdots, as shown below, to re-create them; then log out and log in again. All current dot files will be saved in a directory named $HOME/KeepDots.$Current_Time.

% /usr/common/usg/bin/fixdots

After running the fixdots tool, modify the appropriate .ext files to customize the login shell, using the settings saved in the KeepDots directory.

Shells

The default login shell for all new accounts on Franklin is C-shell (csh). Other available login shells are bash, csh, ksh, and sh. Users may not change their login shell directly on Franklin; NIM is used to accomplish this function. See Change Default Login Shell.

Default Programming Environment

Upon login, the example list of the default modules being loaded are:
(Note: Please do not unload these modules unless being swapped to suitable equivalents!)

% module list 
Currently Loaded Modulefiles:
  1) modules/3.1.6          7) pgi/7.0.7             13) xt-libc/2.0.24b
  2) moab/5.1.0             8) xt-libsci/10.2.0      14) xt-os/2.0.24b
  3) torque/2.2.0           9) xt-mpt/2.0.24d        15) xt-catamount/2.0.24b
  4) MySQL/4.0.27          10) xt-pe/2.0.24b         16) xt-boot/2.0.24b
  5) xtpe-target-cnl       11) PrgEnv-pgi/2.0.24d    17) xt-lustre-ss/2.0.24b
  6) acml/3.6.1            12) xt-service/2.0.24b    18) Base-opts/2.0.24b

The default programming environment is PGI (via module PrgEnv-pgi). The default PGI compiler version (in this example, pgi/7.0.7) is also loaded.

Other loaded modules include Torque/Moab for batch system management (via modules torque and moab), Message Passing Tool Kit (via module xt-mpt) for MPI, Shmem, and OpenMP libraries, ACML and Cray LibSci math and scientific libraries (via modules acml and libsci), CNL glibc functions (via module xt-libc), and Lustre support (via module xt-lustre-ss).

Process Limits

For processes running on Franklin service nodes (launched from login nodes or from batch scripts) there is a CPU limit of 60 min being enforced and the memory limit being set to 2,097,152 kbytes (2 GB soft limit and hard limit).