HPSS Charging

NERSC uses Storage Resource Units (SRUs) to help manage HPSS storage.  The goal is to provide a balanced computing environment with appropriate amounts of storage and adequate bandwidth to keep the compute engines fed with data.  Performance and usage tracking allows NERSC to anticipate demand and maintain a responsive storage environment.  Storage management also recognizes storage as a distinct resource in support of an increasing amount of data intensive computing.  Storage management and the quota system are intended to encourage efficient usage by the user community.

SRU Management

SRUs are reported and managed through the NERSC Information Management (NIM) system.  If a user is out of SRUs in all of their HPSS repositories that user will be restricted so that they can no longer write data to HPSS (although they will continue to be able to read data).

Users can check their HPSS SRU balances by logging in to the NIM and clicking on their "Account Usage" tab.

Calculating a User's SRUs

Three measures of use are used in computing SRUs:

1. Number of files stored (files)
2. GB of space used in the archive (space)
3. GB of I/O transferred (I/O).

The formula used to compute the number of SRUs incurred by a user each day is

   daily user SRUs  =  0.0000393 x files  
		    +  0.0131147 x space (GB)  
		    +  4.0 x I/O (GB)

where 1GB is 1024³ bytes.  Yearly usage is the sum of daily usage and the formula for yearly charges is

   yearly user SRUs  =  0.01436 x Avg files  
                     +  4.787 x Avg space (GB)  
                     +  4.0 x I/O (GB).

For an explanation on how the formula was derived see Storage Resource Unit (SRU) Formula Coefficients.

To estimate the number you SRUs you will use see the SRU Calculator.

Apportioning User SRUs to Repositories: Project Percents

DOE's Office of Science awards SRUs to each NERSC project every year.  The SRUs are deposited into the project's HPSS group account which is called the HPSS repository (or repo).  Users charge their HPSS SRU usage to the HPSS repos of which they are members. 

If a user belongs to only one HPSS repo all usage is charged to that repo.  If a user belongs to multiple repos daily charges are apportioned among the repos using the project percents for that login name.  Default project percents are assigned based on the size of each repo's storage allocation.  Users (only the user, not the project managers) can change their project percents by selecting Change SRU Proj Pct from the Actions pull-down list in the NIM main menu.  Users should try to set project percents to reflect their actual use of HPSS for each of the projects of which they are a member.  Note that this is quite different from the way that computational resources are charged.

• On each computational system each job is charged to a specific repository.  This is possible because the batch system has accounting hooks that handle charging to project accounts.  The HPSS system has no notion of project accounting but only of user accounting.  Users must say "after the fact" how to distribute their HPSS usage charges to the HPSS repos to which they belong.  For a given project the MPP repository and the HPSS repository usually have the same name.

• On PDSF users have access to the resources based on the share their experiment contributes to PDSF.  To see the HPSS repo your PDSF experiment uses you can enter the experiment name in the Repository box in NIM's Main Menu and hit Go.   In the resulting Project/Repo display area click on the User Status by Repo tab to see the HPSS repo name.  In some cases multiple PDSF experiments (called PDSF repos in NIM) share the same HPSS repo.

If users change their project percents this change will apply to all remaining days in the month but not to days prior to when the change was made.

If a user is added to a new repo or removed from an existing repo

If a user is added to a new repo or removed from an existing repo the project percents for that user are adjusted based on the size of the SRU allocations of the repos to which the user currently belongs.  However, if the user has previously changed the default project percents the relative ratios of the previously set project percents are respected.

As an example user u1 belongs to repos r1 and r2 and has changed the project percents from the default of 50% for each repo to 40% for r1 and 60% for r2:

If u1 then becomes a new member of repo r3 which has a storage allocation of 100,000 SRUs the project percents will be adjusted as follows (to preserve the old ratio of 40:60 between r1 and r2 while adding r3 which has the same SRU allocation as r1+r2):

If SRUs are added to or taken from an HPSS repo

If SRUs are added to or taken from an HPSS repo the project percents for the users in that repo are adjusted as needed to reflect the new sizes of each repo's storage allocation unless the user has changed the project percents from their default values (in this case the project percents are not changed).

For example if user u2 belongs to repos r1 and r2 and has not changed the default project percents and then r2 gets a new infusion of SRUS the project percents will be as follows:

User Quotas or Allowed Percents

The default User Quota or Allowed Percent for each user is 100% but Principal Investigators, PI Proxies and Project Managers can change them for the users in their repo(s).

A user's HPSS allowed and used percentages as well as SRU balances are shown in NIM's Account Usage display:
% Used:
The percentage of the repo's HPSS SRU allocation that the login name has used
% Allowed:
The percentage of the repo's HPSS allocation that the login name is authorized to use (also known as the "user quota")
Balance:
The user's SRU balance for this repo.  A user's balance is computed by subtracting the usage within that repo from its "Allowed Percentage" of that repo.  If the balance of the repository as a whole is less than the user's computed balance, than the lesser number (the repo's balance) is used instead.  This user balance is shared with the other repo members.

User Statuses for HPSS

Within NIM the term User Status is used to display two sorts of statuses:  

Repository user statuses:  In the Account Usage Summary area NIM displays the User Status for each (login, repo) pair.  The Repository User Status (for the login name in that repo) is one of:

• Active:  The user is a member of the repo and has a positive user balance in that repo.
• Restricted:  The user is a member of the repo but has a negative user balance in that repo.
• Limited:  The user is no longer a member of the repo but still has limited access to its resources.
• Deleted:  The user has been remobed from this repo.
• Admin Member:  The user is an administrative member (PI, PI Proxy or Project Manager) of the project who doesn't use this resource.

HPSS Repo User Statuses are also displayed in NIM's "project/repo" display area under the "HPSS Usage, User %'s" tab.

Machine user statuses (for computational resources or HPSS):   In the Login Info area under the Logins by Host tab NIM displays the status for each user on each machine the user has access to.  These Machine User Statuses have the following meaning for HPSS:

• Active:  The user is in a normal active state with no restrictions.  The user can read and write data.
• Restricted:  The user is restricted because it has no repo to charge to.  The user can read data from HPSS but cannot write to HPSS.
• Disabled:  The user has been temporarily disabled.
• Limited:  The user is restricted because the user is no longer a member of any active repository.  The user can read data from HPSS but cannot write to HPSS.  On HPSS a user remains limited for about one year prior to being deactivated.
• Deactivated:  The user has been disabled and the user's HPSS files can be archived to the "crypt".  The user no longer has access to HPSS.
• Crypt:  The user's files have been moved to the crypt and may be deleted at any time.
• Deleted:  The user has been removed from HPSS.

What happens if a repo or user SRU balance is negative?

Accounting information is sent from HPSS to NIM once daily (in the early morning, Pacific Time).  At this time actions are taken if a repo or user SRU balance is negative.

If a repo runs out of SRUs all users in the repo are marked as restricted for that repo (see repository user statuses).

A user is "HPSS restricted" if all of the repos to which the user belongs are restricted (see machine user statuses).  HPSS restricted login names are able to read data from HPSS but cannot write any data to HPSS.

Likewise, when a user goes exceeds their individual "allowed percent" in a given repo the user is marked as restricted in that repo.  The user is HPSS restricted only if the user status is restricted for each repo to which the user belongs.

 HPSS repos that are negative continue to incur SRU charges every day for each member that has HPSS files or I/O activity.  This is because there is a daily charge for files stored within HPSS and for I/O activity.  Note that restricted users can still incur I/O charges by reading files.   Also, project percents are not adjusted when a repo goes negative.

Likewise, a user who has exceeded their allowed percent in a given repo will continue to incur charges in that repo.  Project percents are not automatically adjusted when a user exceeds their allowed percent.

SRU Usage Reports

The following SRU Usage Reports are available in NIM: 

Search Daily HPSS User Usage:

Use this query to see the actual number of files and gigabytes a user (login name) has stored in HPSS on a daily basis,  I/O transactions to and from HPSS, as well as the amounts these three usage factors (files, space and I/O) contribute to the user's SRU charge.

From the Reports pull-down menu in NIM's main menu frame select Daily User HPSS. You will see the following query:

>> View Search Daily HPSS Usage 

Note that HPSS usage data is stored by "Begin Date" and "Last Date".  In every usage record the number of files and Gbytes stored remains constant from Begin Date to End Date.  The End Date can be thought of as an approximation of the Job Date used for MPP usage queries.
By default the user's organization will not be displayed in the report (it will be if you deselect the Hide? box).

Search Monthly HPSS User Usage:

Use this query to see the average number of files and gigabytes a user has stored in HPSS each month, the total I/O transactions to and from HPSS in the month, and the amounts these three usage factors (files, space and I/O) contribute to the user's SRU charge

Search Yearly User/Repo Usage:

Use this query to search yearly or year-to-date (for the current fiscal year) usage information.  For FY 2003 and later HPSS and MPP usage is available, for FY 2002 HPSS and MPP usage is available, and for FY 2001 MPP and PVP usage is available.  Project Managers can use this query to find users who are within a certain percentage of their user quota (or allowed percent).

From the Reports pull-down menu in NIM's main menu frame select Yearly Usr/Repo.  You will see the following query (for an HPSS report set the Resource Type to HPSS):

>> View Search Yearly HPSS Usage