darshan

Description

Darshan is a light weight IO profiling tool capable of profiling POSIX IO, MPIIO and HDF5 IO. We encourage all users to turn on Darshan for their application running on Hopper. Darshan will not only help the users to identify IO bottleneck and improve performance, but also help NERSC to better understand the IO usage of its users and shape its future plans. 

How to Use Darshan

Compile

To use Darshan, you just need to load the darshan module (shown below). This will automatically wrap the 'cc' and 'ftn' commands

module load darshan

Note: if you are using a non-default programming environment (PrgEnv-xxx), you need to change the programming environment BEFORE load the darshan module:

module swap PrgEnv-pgi PrgEnv-gnu #(or PrgEnv-intel, etc)
module load darshan 

You can also add the above changes to your dotfiles so that you get Darshan enabled by default. 

Runtime (Only needed if you use dynamic libraries)

If you use statically linked program, you don't need to load the Darshan module when running it. 

Add to your PBS script the following line (or to your dotfiles):

module load darshan

See the result

All Darshan results are located in /usr/common/usg/spool/darshanlogs/year/month/day/user-executablename_idPBS_JOBID_xxx.gz. You can use darshan-analyze or darshan-parser to see the details of the results. 

Example

The following script will setup a test directory in $SCRATCH, automatically compile and run it, and point you to the profiling output. 

#if you use bash or sh
/usr/common/usg/darshan/example/rundarshanExample.sh

#if you use tcsh or csh
/usr/common/usg/darshan/example/rundarshanExample.csh

#if you don't know which shell you have
env|grep SHELL 

Availability

Contact

If you have any question about Darshan at NERSC, please email consult@nersc.gov