Running Jobs

When you log into Jaguar, you are placed on one of the four login nodes. Login nodes should be used for basic tasks such as file editing, code compilation, data backup, and job submission.

The login nodes should not be used to run production jobs. Production work should be performed on the system’s compute resources. On Jaguar, access to compute resources is managed by the Portable Batch System (PBS). Job scheduling is handled by Moab, which interacts with PBS and the XT system software. The following pages provide information for getting started with the batch facilities of PBS with Moab as well as basic job execution.

Notice:
Compute nodes can see only the Lustre work space.

The NFS-mounted home, project, and software directories are not accessible to the compute nodes.

  • Executables must be executed from within the Lustre work space.
  • Batch jobs can be submitted from the home or work space. If submitted from a user’s home area, the user should cd into the Lustre work space directory prior to running the executable through aprun. An error similar to the following may be returned if this is not done: 
            aprun: [NID 94]Exec /tmp/work/userid/a.out failed: chdir /autofs/na1_home/userid
        No such file or directory
  • Input must reside in the Lustre work space.
  • Output must also be sent to the Lustre file system.

Batch Scripts

Batch scripts can be used to run a set of commands on a system’s compute partition. The batch scripts page gives example batch scripts, how to submit the batch scripts, and common usage tips.

Interactive Batch Jobs

Interactive batch jobs give users interactive access to compute resources. A common use for interactive batch jobs is debugging. The interactive batch jobs page shows a user how to run interactive jobs through the batch system and provides common usage tips.

Common PBS Options

The common PBS options page gives a quick overview of common PBS options.

PBS Environment Variables

The PBS environment variables page gives a quick overview of useful environment variable sets within PBS jobs.

Altering Batch Jobs

The altering batch jobs page lists ways to remove or alter batch jobs.

Monitoring Job Status

The monitoring job status page lists ways to monitor jobs in the batch queue.

Queues

Queues are used by the batch scheduler to aid in the organization of jobs. The queues page lists the available queues and their benefits.

Job Execution

Once access to compute resources has been allocated through the batch system, users have the ability to execute jobs on the allocated resources. The job execution page gives examples of job execution and provides common tips.

Job Accounting

Projects are charged based on usage of compute resources. The job accounting page gives details on how we calculate each job’s usage.

Scheduling Policy

The scheduling policy page gives details of the scheduling policies.