VASP
Description
VASP is a package for performing ab initio quantum-mechanical molecular dynamics (MD) using pseudopotentials and a plane wave basis set. The approach implemented in VASP is based on a finite-temperature local-density approximation (with the free energy as variational quantity) and an exact evaluation of the instantaneous electronic ground state at each MD step using efficient matrix diagonalization schemes and an efficient Pulay mixing.
Access to VASP
VASP is available only to NERSC users who already have an existing VASP license. If you have a VASP license, send your license information to the VASP support at Vienna at vasp.materialphysik@univie.ac.at (and copy vasp_licensing@nersc.gov) requesting that they confirm your VASP license to NERSC staff at vasp_licensing@nersc.gov in order to gain access to the VASP binaries at NERSC. Please note, VASP 5 requires a separate license. The VASP 4.6 users will NOT be automatically upgraded to VASP 5. Therefore, you need to confirm your VASP 5 license separately if you are a VASP 4 user and just upgraded your license to VASP 5.
Modules
NERSC uses modules to manage access to software. To access VASP at NERSC you must first verify your VASP license via email. When your request has been processed you will be able to access the vasp executables.
For instructions on how to obtain access to VASP at NERSC, please login to a NERSC machine (e.g., Franklin) and issue the command:
% module show vasp
To see the available vasp modules, type:
% module avail vasp
To access the default version of VASP, type:
% module load vasp
To access VASP 5, type:
% module load vasp/5.2
Using VASP on Franklin and Hopper
Currently we have VASP 4.6.35 (an optimized version by Paul Kent, module name vasp/4.6.35.pkent) and 4.6.31 (module name vasp/4.6.31) installed on Franklin. The default version on Franklin is 4.6.35.pkent, which should be significantly faster for large concurrency jobs. The Climbing-Image NEB codes from G. Henkelman's group are included in this version. If you type
% module show vasp
or
% module show vasp/4.6.35.pkent
You can see where the VASP executables reside (the bin directory). There are several VASP executables available in the bin directory,
% ls -l /usr/common/usg/vasp/4.6.35.pkent/bin
total 33696
lrwxrwxrwx 1 zz217 usg 17 Sep 23 22:26 gvasp -> vasp.xt_cnl_gamma
lrwxrwxrwx 1 zz217 usg 11 Sep 23 22:26 vasp -> vasp.xt_cnl
-rwxr-x---+ 1 zz217 usg 11826601 Dec 17 2008 vasp.xt_cnl
-rwxr-x---+ 1 zz217 usg 10892132 Dec 17 2008 vasp.xt_cnl_gamma
-rwxr-x---+ 1 zz217 usg 11761766 Dec 17 2008 vasp.xt_cnl_so
lrwxrwxrwx 1 zz217 usg 14 Sep 23 22:26 vasp_ncl -> vasp.xt_cnl_so
drwxrws--- 4 zz217 usg 4096 Apr 18 2009 vtstscripts
These are "production" builds of the gamma-point only (executable name:gvasp), the non-collinear (executable name: vasp_ncl), and the collinear (executable name: vasp) versions of the code, respectively. The directory vtstscripts contains the shell and perl scripts for analysing results of VASP calculations (from G. Henkelman's group). You need to choose an appropriate executable to run your job.
There are two ways of running VASP on Franklin and Hopper: submitting a batch job, or running interactively in an interactive batch session.
Sample batch script to run VASP on Franklin and Hopper
#PBS -N test_vasp
#PBS -q debug
#PBS -l mppwidth=32
#PBS -l walltime=00:30:00
#PBS -j oe
#PBS -V
cd $PBS_O_WORKDIR
module load vasp
aprun -n 32 vasp
Then submit the job script using the qsub command, e.g., assuming the job script name is test_vasp.pbs:
% qsub test_vasp.pbs
To run VASP interactively on Franklin and Hopper
To request an interactive batch session, issue the command (e.g., requesting 32 PEs):
% qsub -I -V -q interactive -l mppwidth=32
When the batch session is returned to you, issue the following commands:
% cd $PBS_O_WORKDIR
% module load vasp
% aprun -n 32 vasp
Sample job script to run on unpacked nodes on Hopper
For large memory vasp jobs, users often need to run then on unpacked nodes, i.e., using only part of the cores available and leave some of them idle to allow more memory for the cores in use. Here is a same job script to run on unpacked nodes, eg., using only 8 cores out of 24, and please note, the -S1 option is important to get the optimal MPI task placement on a unpacked node. More information about how Hopper places MPI tasks onto its cores on the node can be found here.
#PBS -l mppwidth=768,walltime=00:30:00
#PBS -q debug
#PBS -j oe
#PBS -V
cd $PBS_O_WORKDIR
module load vasp
aprun -n384 -N12 -S3 vasp
Sample job script to run multi-threaded VASP on Hopper
VASP doesn't have OpenMP directives implemented in the source code, but we linked the code to the Cray multi-threaded scientific libraries to make use of the memory savings from using threads, and also the potential performance gains, although it is limitted, for users who have to run their jobs on the unpacked nodes to allow more memory for the cores in use. We suggest users to use low threads counts (2 or 3) to avoid the peformance penalty from NUMA effects. More performance results of the multi-threaded VASP can be found here. The sample job script is as follows:
#!/bin/bash -l
#PBS -l mppwidth=768,walltime=00:30:00
#PBS -q debug
#PBS -j oe
#PBS -V
cd $PBS_O_WORKDIR
module load vasp
export OMP_NUM_THREADS=2
aprun -n384 -N12 -d2 vasp
Using VASP on Carver
You must first load the VASP module; this will set your LD_LIBRARY_PATH environment variable to include a library (Intel MKL) needed to run VASP. If you attempt to run VASP without loading its module you will get an error because the library was not loaded.
#PBS -N test_vasp
#PBS -q debug
#PBS -l nodes=4:ppn=8
#PBS -l walltime=00:30:00
#PBS -j oe
#PBS -V
cd $PBS_O_WORKDIR
module load vasp
mpirun -n 32 vasp
Using VASP Via the Web with NOVA
NERSC's Online Vasp Application (NOVA) is an experimental tool that enables users to submit VASP jobs to NERSC resources (currently, only Carver). NERSC users who are registered VASP users (i.e., in the vasp user group at NERSC) can use it to set up VASP jobs and monitor their progress from a web browser. NOVA jobs use the same directory and file structure as regular command-line VASP jobs, so it is possible to import existing VASP jobs into NOVA. You can also quickly modify an existing job and save it as a new NOVA job. NOVA makes it easy to select a pseudopotential, use automatic k-point generation, and set run parameters for VASP as well as the NERSC system running the job (currently Carver). It also allows the user to visualize convergence and monitor the status of multiple jobs. You can prepare a job partially, save it, and then come back another time to finish.
The best way to get acquanted with NOVA is to log on and import a sample job. Log on with your NIM user name and password, then click the Import Job button. In the dialog, give your job a name, choose a sample file set from the drop-down menu under Location of Job to be Imported, then select a place for NOVA to store your working copy of the sample files. We recommend selecting your scratch directory. Click Setup, and NOVA will create the directory and copy the necessary sample files into it. The sample files will all have POSCAR, POTCAR, KPOINTS, and INCAR files already. You'll just need to tweak what you like, then choose and save your computational settings.
When all the files are ready, click Run This Job to submit it to a queue. Once the job is submitted, NOVA will show you basic information about it, and once the OSZICAR file has some data, you should see a plot of the energy at each step. To stop the job or work on another one, navigate to the Job Control page. Here you can copy the job, delete it, move its files elsewhere, kill it, or get information about any job. You can also import another job or create a new one from scratch.
NOVA is still a work in progress, so feedback from users is much appreciated. If you give it a try, please let us know how it went and what we could do better. Comments can be sent to amgreiner@lbl.gov or deskinner@lbl.gov.
Trouble Shooting and Error Messages
In this section, we list some common errors users may run into when running VASP jobs on our machines, and our recommendations about how to resolve them.
If you use our VASP modules, and loaded a vasp module successfully, but still get an error on Hopper or Franklin like this:
"aprun: file vasp not found"
or on Carver like this:
"mpirun was unable to launch the specified application as it could not find an executable:
Executable: vasp"
it usually means that you don't have the permission to access the precompiled VASP binaries that NERSC provides. You need to confirm your license by following the instructions posted in the Description session of this page.
Building VASP
Some users have reported a build error on Hopper.
Build error: /usr/bin/ld: cannot find -lsci_mc12_mp
Cray has recently renamed the multi-threaded libsci libraries to help the performance of dynamic libraries applications on Hopper. So the compiler wrapper now links to the -lsci_pgi_mp (for pgi compiler) instead of -lsci_mc12_mp. You can simply remove the "-lsci_mc12_mp" in the makefile, or replace the -lsci_mc12_mp with the -lsci_pgi_mp, then the code should link without a problem (it will link to the -lsci_pgi_mp by default).
Documentation
Availability
| Package | Platform | Category | Version | Module | Install Date | Date Made Default |
|---|---|---|---|---|---|---|
| VASP | carver | applications/ material sciences | 4.6.35 | vasp/4.6.35 | 2010-02-10 | 2010-02-10 |
| VASP | carver | applications/ material sciences | 5.2 | vasp/5.2 | 2010-02-10 | |
| VASP | carver | applications/ material sciences | 5.2.11 | vasp/5.2.11 | 2010-12-29 | |
| VASP | carver | applications/ material sciences | 5.2.11.IKPTD | vasp/5.2.11.IKPTD | 2012-01-13 | |
| VASP | carver | applications/ material sciences | 5.2.11.tbdyn | vasp/5.2.11.tbdyn | 2012-01-13 | |
| VASP | carver | applications/ material sciences | 5.2.12 | vasp/5.2.12 | 2012-01-13 | |
| VASP | carver | applications/ material sciences | 5.2.12_vtst | vasp/5.2.12_vtst | 2012-04-19 | |
| VASP | carver | applications/ material sciences | 5.2.8 | vasp/5.2.8 | 2012-01-13 | |
| VASP | carver | applications/ material sciences | 5.2.orig | vasp/5.2.orig | 2012-01-13 | |
| VASP | carver | applications/ material sciences | 5.3.2 | vasp/5.3.2 | 2012-11-05 | |
| VASP | carver | applications/ material sciences | 5.3.2_vtst | vasp/5.3.2_vtst | 2012-11-05 | |
| VASP | edison | applications/ material sciences | 5.3.2 | vasp/5.3.2 | 2013-01-04 | 2013-02-05 |
| VASP | edison | applications/ material sciences | 5.3.2_vtst | vasp/5.3.2_vtst | 2013-01-04 | |
| VASP | edison | applications/ material sciences | 5.3.3 | vasp/5.3.3 | 2013-01-04 | |
| VASP | hopper | applications/ material sciences | 4.6.35 | vasp/4.6.35 | 2010-02-10 | |
| VASP | hopper | applications/ material sciences | 4.6.35.pkent | vasp/4.6.35.pkent | 2010-11-13 | 2010-11-13 |
| VASP | hopper | applications/ material sciences | 4.6.35.pkent | vasp/4.6.35.pkent | 2010-11-13 | |
| VASP | hopper | applications/ material sciences | 5.2 | vasp/5.2 | 2009-10-27 | 2009-10-27 |
| VASP | hopper | applications/ material sciences | 5.2 | vasp/5.2 | 2010-11-13 | 2010-11-13 |
| VASP | hopper | applications/ material sciences | 5.2 | vasp/5.2 | 2010-11-13 | |
| VASP | hopper | applications/ material sciences | 5.2.11 | vasp/5.2.11 | 2011-03-10 | 2011-03-10 |
| VASP | hopper | applications/ material sciences | 5.2.11.tbdyn | vasp/5.2.11.tbdyn | 2010-11-13 | |
| VASP | hopper | applications/ material sciences | 5.2.12 | vasp/5.2.12 | 2012-01-13 | |
| VASP | hopper | applications/ material sciences | 5.2.12 | vasp_ccm/5.2.12 | 2012-04-04 | 2012-04-04 |
| VASP | hopper | applications/ material sciences | 5.2.12-cce | vasp/5.2.12-cce | 2012-09-11 | |
| VASP | hopper | applications/ material sciences | 5.2.12_vtst | vasp/5.2.12_vtst | 2012-04-19 | |
| VASP | hopper | applications/ material sciences | 5.2.12_wannier90 | vasp/5.2.12_wannier90 | 2012-05-15 | |
| VASP | hopper | applications/ material sciences | 5.3.2 | vasp/5.3.2 | 2012-09-18 | |
| VASP | hopper | applications/ material sciences | 5.3.2-pgi | vasp/5.3.2-pgi | 2012-11-09 | |
| VASP | hopper | applications/ material sciences | 5.3.2-pgi_vtst | vasp/5.3.2-pgi_vtst | 2012-11-09 | |
| VASP | hopper | applications/ material sciences | 5.3.2_vtst | vasp/5.3.2_vtst | 2012-09-21 |
