Bassi — IBM POWER 5

• 1. About Bassi
  • 1.1 Bassi Configuration
  • 1.2 Bassi specifications
• 2. Detailed Information
  • 2.1 IBM Publications
  • 2.2 Bassi System Specificiations
  • 2.3 Bassi p575 Power 5 Processors

Born on October 31, 1711 in Bologna, Bassi was the daughter of a lawyer. Educated privately, Bassi studied logic, metaphysics, philosophy, chemistry, hydraulics, mathematics, mechanics, algebra, geometry, and ancient and modern languages (Greek, Latin, French, and Italian).

Bassi was appointed Professor of Anatomy at the University of Bologna in 1731, was elected to the Academy of the Institute for Sciences in 1732, and was given the Chair of Philosophy at the University of Bologna in 1733. In 1738, she married her colleague Dr. Giuseppe Veratti. They had 12 children.

While raising her family, she successfully petitioned for wider responsibilities and a higher salary to cover the cost of equipment for physical and electrical experiments. She continued her lifelong interest in physics, lecturing from her home while her children were small then returning to the university at age 65 as a Professor of Experimental Physics in 1776. Living to the ripe old age of 66, she died on February 20, 1778.

More info: Laura Bassi

1. About Bassi

The NERSC IBM p575 POWER 5 system, named Bassi, is a distributed memory computer with 888 processors available to run scientific computing applications. The machine is named in honor of Laura Bassi, a noted Newtonian physicist of the eighteenth century.

Each Bassi processor has a theoretical peak performance of 7.6 GFlops. The processors are distributed among 111 compute nodes with 8 processors per node. Processors on each node have a shared memory pool of 32 GBytes. A Bassi node is an example of a Shared Memory Processor or SMP.

The compute nodes are connected to each other with a high-bandwidth, low-latency switching network. Each node runs its own full instance of the standard AIX operating system. The disk storage system is a distributed, parallel I/O system called GPFS. Additional nodes serve exclusively as GPFS servers.

Bassi's network switch is the IBM "Federation" HPS switch which is connected to a two-link network adapter on each node.

1.1 Bassi Configuration

• 122 8-processor nodes (with 32GB memory each)
• 1.9 GHz POWER 5 processors
• 111 compute nodes (888 processors)
• 3.5 TB aggregate memory on compute nodes
• 7.6 GFlops/sec peak processor speed
• 6.7 TFlops theoretical peak system performance
• 100 TB of usable disk space in GPFS
• 2 login nodes
• 6 nodes supporting the General Parallel Filesystem (GPFS)
• Nodes configured to use 20 GB "Large Page" memory

1.2 Bassi specifications

2. Detailed Information

2.1 IBM Publications

• IBM System p5, @eserver p5, pSeries, OpenPower and IBM RS/6000 Performance Report, January 11, 2006
• IBM e-server p5 575 cluster node
• POWER5 system microarchitecture, IBM J. RES. & DEV. VOL. 49 NO. 4/5 JULY/SEPTEMBER 2005

2.2 Bassi System Specificiations

• IBM Cluster 1600
• p5-575 nodes = POWER 5 nodes of a specific (575) configuration
  • 8-way single core SMP nodes
  • 1.9 GHz single-core POWER 5 64-bit processors (DCM: Dual Chip Module with one active core)
  • 7.6 GFlops/sec theoretical peak per processor
  • 2MB on-board L2 & 36MB L3 cache
  • 200 GB/s max cumulative peak theoretical memory bandwidth
  • 32 GB memory per node via 64 512 MB pluggable DIMMs
  • 48GB/s theoretical peak I/O Bandwidth via 6 GX+ buses
  • One 2-link High Performance Switch adapter
  • Four integrated 10/100/1000Mb Ethernet ports
• Production System
  • 111 Compute Nodes (888 Compute Processors)
  • 2 Interactive Nodes
  • 3 Spare Nodes
  • 6 Storage Nodes
  • 1 p550 Cluster System Management (CSM) node
  • 12 Hardware Management Control (HMC) 3.06 GHz Xenon
  • ~100 TB formatted usable RAID GPFS global file storage
  • 111*8*7.6 GFlops/sec = 6.7 TFlops/sec theoretical system peak
• I/O Subsystem Configuration
  • 6 Virtual Shared Disk servers to support GPFS, each with 2 High Performance Switch (HPS) links. (See next section)
  • Each Virtual Shared Disk server has 16 2 Gb Fibre Channel links.
  • Disk subsystem consists of 24 IBM DS4300 storage systems, each with 42 146 GB drives configured as 8 RAID 5 4+P arrays, with 2 hot spares per DS4300. Each DS4300 has dual controller and each controller has dual Fibre Channel ports.
  • Servers and controllers are configured for failover so each server acts primary for 8 controller ports and backup for another 8 controller ports.
  • Each array is a separate LUN. The scratch filesystem has 118 LUNs, of which 12 are used for 2 metadata replicas. Data and metadata on scratch are separated for performance.
• High Performance Switch (HPS) (Also known as "Federation")
  • Each node: One 2-link High Performance Switch adapter
  • Each node: attachs to the interconnect with 2 links, one to each of 2 separate planes
  • LAPI and MPI communication
  • GPFS control uses IP over HPS; data uses LAPI over HPS
  • Peak HPS bandwidth - 2 GB/s per link each direction
  • MPI latency less than 5 µs
  • The network uses 123 of 128 available connections on each plane.
• Racks and Frames
  • p5-575 nodes are housed in a 2U by 24-inch wide by 48-inch deep node package
  • Up to 12 p5-575 modules and 2 High Performance Switches can be contained in a single 24-inch IBM system frame
  • There are 12 system frames housing production system p575 nodes, node switches and I/O drawers.
• Software (original version in parentheses)
  • IBM AIX Version 5.3 ML 4 (5.2 ML 5)
  • IBM Parallel Environnent (PE) V. 4.2
  • IBM C/C++ Enterprise Edition V 7.0
  • IBM Fortran Enterprise Edition V 9.1
  • IBM LoadLeveler 3.3
  • IBM Cluster System Manager (CSM) V. 1.4
  • IBM General Parallel Filesystem (GPFS) V. 2.3
  • IBM Parallel Engineering Scientific Subroutines Library (PESSL) V 3.2
  • IBM Engineering Scientific Subroutines Library (ESSL) V 4.2
  • HPC Toolkit for POWER5 (no later than 4Q 2005)
  • AIX Toolbox for Linux Applications

2.3 Bassi p575 Power 5 Processors

The following image depicts dual-core chips; Bassi has a single core.

• 1.9 GHz single core Power 5 chips
• 1.92 MB Level 2 Cache per CPU (10-way associative, 3x640 KB)
• 36 MB Level 3 Cache per CPU
• Processor-to-memory bandwidth (peak): ~12 GB/s (per chip?)
• L3 to L2 peak bandwidth: 243.2 GBps (per node?)
• 120 rename registers for integer and floating point
• maximum 5 instructions per cycle
• maximum 4 flops per cycle
• 64 KB 2-way associative Instruction cache
• 32 KB 4-way associative Data cache