How usage is charged
MPP Charging (Computational Systems)
When a job runs on a NERSC MPP system, such as Hopper, charges accrue against one of the user's repository allocations. The unit of accounting for these charges is the "MPP Hour".
A parallel job is charged for exclusive use of each multi-core node allocated to the job. The MPP charge for such a job is calculated as the product of:
- the job's elapsed wall-clock time in hours,
- the number of nodes allocated to the job (regardless of the number actually used),
- the number of cores available on each allocated nodes,
- a machine charge factor (MCF) based on typical performance of the machine relative to Hopper (MCF=1.0), and
- a queue charge factor (QCF). Queue priority scheduling gives users some control over how quickly their jobs are scheduled for execution in the batch system. A premium job is scheduled for execution before an otherwise equivalent regular job. A low job has a lower priority for scheduling.
A serial job is charged for the time used by its single core. The MPP charge for such a job is calculated as the product of:
- the job's elapsed wall-clock time in hours,
- a machine charge factor (MCF) based on typical performance of the machine relative to the quad core Cray XT4 Franklin architecture (MCF=1.0), and
- a queue charge factor (QCF=1.0 for the serial and mag_serial queues)
Account information is available via the NIM web interface. Charges are updated in NIM once a day (at about 3:00 AM Pacific time).
Batch Charging
The charge against your repo for a parallel batch job is
(wallclock hours used)
* (machine charge factor or MCF)
* (number of nodes allocated)
* (cores per node)
* (Queue Charge Factor or QCF)
The charge against your repo for a serial job is
(wallclock hours used)
* (machine charge factor or MCF)
* (Queue Charge Factor or QCF)
The machine charge factor is:
- 1.0 for Hopper
- 1.5 for Carver
- 1.0 for Euclid
The queue charge factor is:
- 1.0 for regular, debug, interactive, and serial jobs
- 0.5 for low priority jobs
- 2.0 for premium priority jobs
A 32-node regular priority job on Hopper that begins at 12:00:00 and ends 8 hours later at 20:00:00, will have a charge in MPP hours of:
8 hours * 32 nodes * 24 Cores/node * 1.0 MCF * 1.0 QCF = 6,144 MPP Hours
Note that on Hopper a discount is applied to jobs that run on a large number of processors. See Batch Queues and Policies on Hopper.
Charging your batch job to a specific project
If you do not specify which project account (called repository or repo) to charge, your charges will normally accrue to your default repository, unless you are out of time in that repository. To charge a batch job to a specific repository use the PBS keyword:
#PBS -A repo_name
Interactive Charging
Only Euclid collects interactive charges, which are for the actual CPU time used by your interactive jobs. These charges accrue to your default repository (you cannot specify a specific repository for intractive charges). Euclid has a machine charge factor of one, and interactive jobs have a queue charge factor of one as well.
HPSS Charging (Storage)
HPSS charging is based on allocations of Storage Resource Units (SRUs) which are awarded into accounts called HPSS repos. If a login name belongs to only one HPSS repo all of its usage is charged to that repo. If a login name belongs to multiple HPSS repos its daily charge is apportioned among the repos using the project percents for that login name. Default project percents are assigned by NIM based on the size of each repo's storage allocation.
Users can see their project percents on the Account Usage Summary page in NIM. This is the page that is displayed when you first login to NIM (or select My Account Usage from the NIM main menu). Look at the column Proj % in the HPSS repo display area.
To change your project percents, select Change SRU Proj Pct from the Actions pull-down list in the NIM main menu.
For more detailed information about HPSS charging see HPSS charging.
Running out of Allocation
Accounting information is sent from the computational machines and HPSS to NIM once daily (in the early morning, Pacific Time). At this time actions are taken if a repo or user balance is negative.
If a repo runs out of time (or Storage Resource Units) all login names which are not associated with another active repository are restricted:
- On computational machines restricted users are able to log in, but cannot submit batch jobs, or run parallel jobs, or run lengthly interactive programs.
- For HPSS restricted users are able to read data from HPSS and delete files but cannot write any data to HPSS.
Login names that are associated with more than one repo (for a given resource -- MPP or HPSS) are checked to see if the user has a positive balance in any of her or his repos (for that resource). If he or she does have a positive balance in some repo (for that resource) s/he will not be restricted and the following will happen:
- On computational machines the user will not be able to charge to the restricted repo. If the restricted repo had been the user's default repo, a new (temporary) default repo is assigned to the user (it can be any one of the user's remaining repos).
- For HPSS, repos that are negative continue to incur SRU charges every day for each member that has HPSS files. This is because there is a daily charge for files stored within HPSS. Also, project percents are not adjusted when a repo goes negative. See HPSS Charging.
Likewise, when a user goes over her/his individual user quota in a given repo, that user is restricted if s/he has no other repo to charge to. A PI or Project Manager can change the user's quota.
Usage Reports
Usage for MPP and HPSS resources are collected and displayed by the NERSC Information Management system, NIM. The two most commonly used usage reports in NIM are:
- My Account Usage: This is the default display for most users when they first log into NIM. It shows your personal usage by repository.
- Repository Usage: To see repository usage, type the repo name into the Repository box in the NIM main menu, then click Go.
There is also an inquiry-only command called getnim available on the computational systems. Users can use getnim interactively and in scripts to get their account balances.
