NERSC HPSS Charging

NERSC uses Storage Resource Units (SRUs) to help manage HPSS storage. The goal is to provide a balanced computing environment with appropriate amounts of storage and adequate bandwidth to keep the compute engines fed with data. Performance and usage tracking allows NERSC to anticipate demand and maintain a responsive storage environment. Storage management also recognizes storage as a distinct resource, in support of an increasing amount of data intensive computing. Finally, storage management and the quota system are intended to encourage efficient usage by the user community.

SRUs are reported and managed via the NERSC Information Management (NIM) system. If a user is out of Storage Resource Units in all their HPSS repositories that user will be restricted so that they can no longer write data to HPSS (although they will continue to be able to read data). See: What happens if a repo or user SRU balance is negative?.

Users can check their HPSS SRU balances by logging into the NERSC Information Management System and looking at the resource "HPSS" in your account usage summary.

A SRU Calculator is available for estimating SRU usage.

• Calculating a User's Storage Resource Units
• Apportioning User SRUs to Repositories: Project Percents
• User Quotas or Allowed Percents
• User Statuses for HPSS
• What happens if a repo or user SRU balance is negative?
• SRU usage reports

Calculating a User's Storage Resource Units

Three measures of use are included in computing SRUs:

1. Number of files stored (files)
2. GB of space used in the archive (space)
3. GB of I/O transferred (I/O).

The formula used to compute the number of SRUs incurred by a user each day is:

   daily user SRUs  =  0.0000393 x files  
		    +  0.0131147 x space (GB)  
		    +  4.0 x I/O (GB)

Where 1GB is 1024³ bytes. Yearly usage is the sum of daily usage; the yearly formula is:

   yearly user SRUs  =  0.01436 x Avg files  
                     +  4.787 x Avg space (GB)  
                     +  4.0 x I/O (GB)

For an explanation on how the formula was derived see SRU Formula Coefficients.

Apportioning User SRUs to Repositories: Project Percents

DOE's Office of Science awards Storage Resource Units to each NERSC project every year. The SRUs are deposited into the project's HPSS group account; this group account is called the HPSS repository (or repo). Users charge their HPSS SRU usage to the HPSS repos of which they are members.

If a login name belongs to only one HPSS repo all of its usage is charged to that repo. If a login name belongs to multiple repos its daily charge is apportioned among the repos using the project percents for that login name. Default project percents are assigned based on the size of each repo's storage allocation. The user (only the user, not the project managers) can change her or his project percents by selecting Change SRU Proj Pct from the Actions pull-down list in NIM's Main Menu. Users should try to set project percents to reflect their actual use of HPSS for each of the projects of which they are a member.

Note that this is quite different from the way that computational resources are charged.

• On each computational system each job is charged to a specific repository. This is possible because the batch system has accounting hooks that handle charging to project accounts. The HPSS system has no notion of project accounting, but only of user accounting. Rather, the user must say "after the fact" how to distribute her or his HPSS usage charges to the HPSS repos he or she is a member of. For a given project the MPP repository and the HPSS repository have the same name.
• On the PDSF users have access to the resources based on the share their experiment contributes to the PDSF. To see the HPSS repo your PDSF experiment uses you can enter the experiment name in the Repository box in NIM's Main Menu and hit Go. In the resulting project / repo display area click on the User Status by Repo tab to see the HPSS repo name. In some cases multiple PDSF experiments (called PDSF repos in NIM) share the same HPSS repo.

If a user changes her or his project percents this change will apply to all days in the month the change is made, but not to days prior to the month in which the change is made.

If a login name is added to a new repo or removed from an existing repo

If a login name is added to a new repo or removed from an existing repo the project percents for that user are adjusted based on the size of the SRU allocations of the repos the login name currently belongs to. However, if the user has previously changed the default project percents the relative ratios of these previously set project percents are respected.

For example: say that user u1 belongs to repos r1 and r2 and has changed the default project percents from 50% for each repo to 40% for r1 and to 60% for r2:

   Login   Repo  Repo Allocation   Proj%
    u1      r1     50,000 SRUs      40
    u1      r2     50,000 SRUs      60

Now assume that u1 becomes a new member of repo r3 which has a storage allocation of 100,000 SRUs. The project percents will be adjusted as follows (to preserve the old ratio of 40:60 between r1 and r2 while adding r3 which has the same SRU allocation as r1+r2):

   Login   Repo  Repo Allocation   Proj%
    u1      r1     50,000 SRUs      20
    u1      r2     50,000 SRUs      30
    u1      r3    100,000 SRUs      50

If SRUs are added to or taken from an HPSS repo

If SRUs are added to or taken from an HPSS repo the project percents for the users in that repo are adjusted as needed to reflect the new sizes of each repo's storage allocation unless the user has changed the project percents from their default values (in this case the project percents are not changed).

For example: say that user u2 belongs to repos r1 and r2 and has not changed the default project percents. Repo r2 gets a new infusion of SRUS:

Login   Repo  Old Repo Alloc   Old Proj%   New Repo Alloc   New Proj%
 u2      r1    50,000 SRUs       50           50,000 SRUs     25
 u2      r2    50,000 SRUs       50          150,000 SRUs     75

User Quotas or Allowed Percents

Principal Investigators, PI Proxies and Project Managers can assign Allowed Percents (or user quotas) to each user in their repo. These allowed percents have been operational for MPP and PVP repos for a long time, but have only recently been available for HPSS (with the integration of SRUS in NIM).

The default Allowed Percent is 100% for each user; Project managers can change these as appropriate.

A user's HPSS allowed and used percentages as well as SRU balances are shown in NIM's Account Usage display:

% Used:

the percentage of the repo's HPSS SRU allocation that the login name has used

% Allowed:

the percentage of the repo's HPSS allocation that the login name is authorized to use (also known as the "user quota")

Balance:

the user's SRU balance for this repo. The login name's balance is computed by subtracting the login's usage within that repo from its "Allowed Percentage" of that repo. If the balance of the repository as a whole is less than the login's computed balance, than the lesser number (the repo's balance) is used instead. This user balance is shared with the other repo members.

User Statuses for HPSS

Within NIM the term User Status is used to display two sorts of statuses:

1. Repository user statuses: In the Account Usage Summary area NIM displays the User Status for each (login, repo) pair. The Repository User Status (for the login name in that repo) is one of:

   Active

   The user is a member of the repo and has a positive user balance in that repo.

   Restricted

   The user is a member of the repo but has a negative user balance in that repo.

   Limited

   The user is no longer a member of the repo but still has limited access to its resources.

   Deleted

   The user has been removed from this repo.

   Admin Member

   The user is an administrative member (PI, PI Proxy or Project Manager) of the project who doesn't use this resource.

   HPSS Repo User Statuses are also displayed in NIM's project / repo display area under the HPSS Usage, User %s tab.

2. Machine user statuses (for computational resources or HPSS): In the login info area under the Logins by Host tab NIM displays the status for each login name on each machine the user has access two. These Machine User Statuses have the following meaning for HPSS:

   Active

   The login name is in a normal active state with no restrictions. The login name can read and write data.

   Restricted

   The login name is restricted because it has no repo to charge to. The login name can read data from HPSS but cannot write to HPSS.

   Disabled

   The login name has been temporarily disabled.

   Limited

   The login name is restricted because the user is no longer a member of any active repository. The login name can read data from HPSS but cannot write to HPSS. On HPSS a login name remains limited for about one year prior to being deactivated.

   Deactivated

   The login name has been disabled and its HPSS files can be archived to the "crypt". The login name no longer has access to HPSS.

   Crypt

   The login name's files have been moved to the crypt and may be deleted at any time. This has not yet been implemented.

   Deleted

   The login name has been removed from HPSS.

What happens if a repo or user SRU balance is negative?

Accounting information is sent from HPSS to NIM once daily (in the early morning, Pacific Time). At this time actions are taken if a repo or user SRU balance is negative.

If a repo runs out of SRUs all login names associated with it are marked as restricted for that repository (see repository user statuses).

Login names are "HPSS restricted" if all of the repos associated with this login name are restricted (see machine user statuses). HPSS restricted login names are able to read data from HPSS but cannot write any data to HPSS.

Likewise, when a login name goes over its individual "allowed percent" in a given repo, that (login, repo) pair is marked as restricted. The login name is HPSS restricted only if the (login, repo) repository user status is restricted for each repo associated with this login name.

HPSS repos that are negative continue to incur SRU charges every day for each member that has HPSS files or I/O activity. This is because there is a daily charge for files stored within HPSS and for I/O activity. Note that restricted users can still incur I/O charges by reading files. Also, project percents are not adjusted when a repo goes negative. See Calculating a User's SRUs and Apportioning User SRUs to Repos.

Likewise, a user who has gone over her or his allowed percent in a given repo will continue to incur charges in that repo. Project percents are not automatically adjusted when a login name exceeds its allowed percent, although a Project Manger can ask the user to adjust them.

SRU Usage Reports

The following SRU Usage Reports are available in NIM:

Search Daily HPSS User Usage:

Use this query to see the actual number of files and gigabytes a user (login name) has stored in HPSS on a daily basis, I/O transactions to and from HPSS, aw well as the amounts these three usage factors (files, space and I/O) contribute to the user's SRU charge.

From the Search & Reports pull-down menu in NIM's Main Menu frame select Use: Daily User HPSS. You will see the following query:

View "Search Daily HPSS Usage"

Note that:

• HPSS usage data is stored by "Begin Date" and "Last Date"; in every usage record the number of files and Gbytes stored remains constant from Begin Date to End Date. You can think of End Date as an approximation of the Job Date used for MPP usage queries.
• By Default the user's organization will not be displayed in the report (it will if you deselect the Hide? box).

View a sample HPSS Daily and Monthly Usage report.

Search Monthly HPSS User Usage:

Use this query to see the average number of files and gigabytes a user (login name) has stored in HPSS each month, total I/O transactions to and from HPSS by month, aw well as the amounts these three usage factors (files, space and I/O) contribute to the user's SRU charge.

View a HPSS Daily and Monthly Usage sample report.

Search Yearly User/Repo Usage:

Use this query to search yearly or year-to-date (for the current fiscal year) usage information. For FY 2003 and later HPSS and MPP usage is available; for FY 2002 HPSS, MPP, and PVP usage is available; for FY 2001 MPP and PVP usage is available. Project Managers can use this query to find users who are within a certain percentage of their user quota (or allowed percent).

From the Search & Reports pull-down menu in NIM's Main Menu frame select Use: Yearly Usr/Repo. You will see the following query (for an HPSS report set the Resource Type to HPSS):

View "Search Yearly HPSS Usage"

View a Year-to-Date User/Repository Usage sample report.

Search Year to Date Repository Usage:

Use this query to get a summary usage report of all NERSC repositories. From the Search & Reports pull-down menu in NIM's Main Menu frame select Use: Yearly Repo. For an HPSS report set the Resource Type to HPSS. For more information see Year to Date Repository Usage Query and Year to Date Repository Usage Report.