Franklin
Quad Core Upgrade
Quick Start Guide
for IBM Users
Quick Start Guide
for Jacquard Users
Access to Franklin
User Environment
Accounts/Charges
File Storage
Programming
Running Jobs
Software
Performance Tools
Performance Monitoring
IO Strategies
Franklin Timeline
Vendor Manuals
Status & Stats
NERSC MOTD
Announcements
Known Problems
Current Queue Look
Completed Jobs List
Job Stats
Franklin - Cray XT4
Benjamin Franklin, today known primarily for his central role in founding the United States, was in his own time internationally recognized as America’s leading scientist, primarily for his work with electricity and energy. Franklin's theory of electricity laid the foundation for the science of electrostatics. He coined many of the electrical terms we use today, such as battery, conductor, condenser, positive and negative charge, electric shock, and electrician. He is also remembered as the inventor of the Franklin stove, lightning rod, bifocals, medical catheter, swim fins, glass armonica, and odometer.
Franklin’s scientific interests were wide-ranging and pioneering. He was the first to realize that the Earth's climate could change, first to plot the course of a hurricane, and first to chart the Gulf Stream. He studied cooling by evaporation, experimented with oil films on water, warned of the dangers of lead poisoning, and hypothesized about continental drift and the wave theory of light. Franklin founded the American Philosophical Society to help scientists share their discoveries.
Franklin won every major scientific honor of his time, including the Royal Society's Copley Medal (the equivalent of today’s Nobel Prize); election to the Royal Societies of England, France, and Germany; and honorary degrees from Oxford and other universities. His fame as a scientist opened doors and paved the way for his success as a diplomat during the American Revolution and the negotiation of the Treaty of Paris.
(Portrait: 1777, by Jean-Baptiste Greuze)
About Franklin
The NERSC Cray XT4 system, named Franklin, is a massively parallel processing (MPP) system with 9,660 compute nodes. Each node has dual processor cores, and the entire system has a total of 19,320 processor cores available for scientific applications. The system is named in honor of Benjamin Franklin.
Each of Franklin's compute nodes consists of a 2.6 GHz dual-core AMD Opteron processor with a theoretical peak performance of 5.2 GFlop/sec. Each compute node has 4 GBytes of memory, and each service node (e.g. login node) has 8 GBytes of memory. The full system consists of 102 cabinets with 39 TBytes of aggregate memory. The theoretical peak performance of Franklin is about 101.5 TFlop/sec. Each compute node is connected to a dedicated SeaStar2 router through Hypertransport with a 3D torus topology which ensures high performance, low-latency communication for MPI and SHMEM jobs.
Franklin uses two different operating systems. Full-featured SuSE Linux is run on service nodes (16 login nodes and other I/O, network and system nodes). A light weight OS based on Linux, Compute Node Linux (CNL), is run on each compute node. CNL reduces system overhead, and is critical for the system to scale to very large concurrency. The Parallel file system on Franklin is provided by Lustre with approximately 350 TBytes of user disk space.
Quad Core Upgrade
NERSC is upgrading Franklin to a quad-core XT4 system from July to October 2008. Please refer to Franklin Quad Core Upgrade Plan for the details on the upgrade, including a detailed time line.
Franklin Configuration
Franklin Specifications | |
|---|---|
| Number of compute nodes | 9,660 |
| Processor cores per node | 2 |
| Number of compute processor cores | 19,320 |
| Number of spare compute nodes | 20 |
| Processor Core type | Opteron 2.6 GHz |
| Processor core theoretical peak | 5.2 GFlop/sec |
| System theoretical peak (compute nodes only) | 101.5 TFlop/sec |
| Physical memory per compute node | 4 GBytes |
| Usable memory per node | 3.75 GBytes |
| Number of login nodes | 16 |
| Switch Interconnect | SeaStar2 |
| File System | Lustre |
| Usable disk space | 350 TBytes |
| Batch system | Torque/Moab |
Software Configuration
- SuSE SLES 9.2 Linux on service nodes
- Compute Node Linux (CNL) for all compute nodes
- Portals communication layer
- Lustre Parallel File System
- ALPS utility (aprun) to launch compute node applications
- Torque resource management system with the Moab scheduler
- Programming Environment
- PGI compilers: assembler, Fortran, C, and C++
- Gnu compilers: C, C++, and Fortran F77
- Pathscale compilers: Fortran, C, and C++
- Parallel Programming Models: Cray MPICH2 MPI and Cray SHMEM
- AMD Core Math Library (ACML): BLAS, LAPACK, FFT, Math transcendental libraries, Random Number generators, GNU Fortran libraries
- LibSci scientific library: ScaLAPACK, BLACS, SuperLU
- A special port of the glibc GNU C library routines for compute node applications
- CrayPat and Cray Apprentice2
- Performance API (PAPI)
- Modules
