MPT

Category: Libraries-Communication

Description

MPT is Cray's MPI library. This release of MPI derives from MPICH-2 and implements the MPI-2 standard, except for spawn support. It also implements the MPI 1.2 standard.

The Message-Passing Interface (MPI) supports parallel programming across a network of computer systems through a technique known as message passing. The goal of the MPI Forum, simply stated, is to develop a widely used standard for writing message-passing programs. MPI establishes a practical, portable, efficient, and flexible standard for message passing that makes use of the most attractive features of a number of existing message passing systems, rather than selecting one of them and adopting it as the standard.

This version of MPICH can support multiple interconnect devices for a single MPI job. It allows each process (rank) of an MPI job to create the most optimal messaging path to every other process in the job, based on the topology of the given ranks. The two device drivers that are supported are the shared memory (SMP) driver and the portals device driver. The SMP device driver is based on shared memory and is used for communication between ranks that share a node. The portals device is used for communication between ranks that span nodes.

See the mpi(3) and aprun(1) man pages for more information.

Use

The default MPT is always loaded when a programming environment is loaded (PrgEnv). The PrgEnv-pgi programming environment is loaded by default. Users can switch their programming environment to pathscale or gnu with

  module swap PrgEnv-pgi PrgEnv-pathscale    # switch to pathscale PrgEnv

It is advised to always use the default MPT that comes with the PrgEnv module or use available newer MPT versions. It is not advised that older MPT versions be used.

To build application with MPI, use either the cc, CC, ftn, or f77 command. If you invoke a compiler directly by using a command such as mpicc, the resulting executable will not run on a Cray XT system.

Launching an MPI application on Cray XT systems is done via the Application Level Placement Scheduler (ALPS).

In the followig example, an MPI application is first compiled using the cc Cray wrapper. The resulting executable is then launched using 128 processes:

         cc -o my_prog my_prog.c
         aprun -n 128 ./my_prog    # must be done in a batch job

See the mpi(3) and aprun(1) man pages for more information.

Further Information

MPT 3.1.02

This version is necessary for XT5 for large core-count and has several new beneficial features. The new features are: Each of these are described in more detail below.

Move from MPICH2 1.0.4 to MPICH2 1.0.6 MPICH2

The move to MPICH2 1.0.6 fixed a number of problems, some of which have already applied to the MPT 3.0 release on an as needed basis. Many of the new features added in MPICH2 1.0.6 don't affect XT users but it is important to stay close to the MPICH2 latest releases to take advantage of improvements they are making. That being said, we have already found and fixed several regressions introduced by MPICH2 1.0.5 or MPICH2 1.0.6 and need more exposure to ensure there are not more. Here are just some of the features in MPICH2 1.0.5 and MPICH2 1.0.6 listed from their changes document that may affect XT users:

Support for over 64,000 MPI ranks

This feature modifies some hard limits for how high MPI jobs could scale on XT systems. The new limit is 256,000 MPI ranks.

Support for over 32,000 SHMEM PEs

This feature modifies some hard limits for how high SHMEM jobs could scale on XT systems. The new limit is 256,000 SHMEM PEs.

In order to support higher scaling, changes were made to the SHMEM header files that require a recompile when using this new version. The new library will detect this incompatibility and issue a FATAL error message telling you to recompile with the new headers.

Automatically-Tuned Default Values for MPICH environment Variables

Several of the MPICH environment variable default values are now dependent on the total number of processes in the job. Previously, these defaults were set to static values. This feature is designed to allow higher scaling of MPT jobs with fewer tweaks to environment variables. For more information on how the new defaults are calculated, please see the mpi manpage. As before, the user is able to override any of these defaults by setting the corresponding environment variable. The new default values are displayed via the MPICH_ENV_DISPLAY setting.

MPT 3.0.x: Differences and incompatibilities from MPT 2.0

MPT 2.0/3.0 conversion issues:

To verify you have linked with the MPI 3.0 libraries, set the MPICH_VERSION_DISPLAY environment variable.

Below is an example of setting MPICH_ENV_DISPLAY to 1 for a test case that spans nodes:

% export MPICH_ENV_DISPLAY=1

% aprun -n 4 ./hello
PE 0: MPICH environment settings:
PE 0: MPICH_ENV_DISPLAY = 1
PE 0: MPICH_VERSION_DISPLAY = 0
PE 0: MPICH_ABORT_ON_ERROR = 0
PE 0: MPICH_CPU_YIELD = 0
PE 0: MPICH_RANK_REORDER_METHOD = 1
PE 0: MPICH_RANK_REORDER_DISPLAY = 0
PE 0: MPICH/SMP environment settings:
PE 0: MPICH_SMP_OFF = 0
PE 0: MPICH_MSGS_PER_PROC = 16384
PE 0: MPICH_SMPDEV_BUFS_PER_PROC = 32
PE 0: MPICH_SMP_SINGLE_COPY_SIZE = 131072
PE 0: MPICH_SMP_SINGLE_COPY_OFF = 0
PE 0: MPICH/PORTALS environment settings:
PE 0: MPICH_MAX_SHORT_MSG_SIZE = 128000
PE 0: MPICH_UNEX_BUFFER_SIZE = 62914560
PE 0: MPICH_PTL_UNEX_EVENTS = 20480
PE 0: MPICH_PTL_OTHER_EVENTS = 2048
PE 0: MPICH_VSHORT_OFF = 0
PE 0: MPICH_MAX_VSHORT_MSG_SIZE = 1024
PE 0: MPICH_VSHORT_BUFFERS = 32
PE 0: MPICH_PTL_EAGER_LONG = 0
PE 0: MPICH_PTL_MATCH_OFF = 0
PE 0: MPICH_PTL_SEND_CREDITS = 0
PE 0: MPICH/COLLECTIVE environment settings:
PE 0: MPICH_FAST_MEMCPY = 0
PE 0: MPICH_COLL_OPT_OFF = 0
PE 0: MPICH_BCAST_ONLY_TREE = 1
PE 0: MPICH_ALLTOALL_SHORT_MSG = 1024
PE 0: MPICH_REDUCE_SHORT_MSG = 65536
PE 0: MPICH_ALLREDUCE_LARGE_MSG = 262144
PE 0: MPICH_ALLTOALLVW_FCSIZE = 32
PE 0: MPICH_ALLTOALLVW_SENDWIN = 20
PE 0: MPICH_ALLTOALLVW_RECVWIN = 20
PE 0: MPICH/MPIIO environment settings:
PE 0: MPICH_MPIIO_HINTS = (null)

-------------------------

Support

This package has the following support level : Supported

Available Versions

All versions of this software are provided by the system vendor and are not installed by NICS staff.