Cray’s XT Jaguar has raced ahead of its competitors and promises to accelerate the pace open scientific research.

Supercomputing has never been any more super than it is now at the Oak Ridge National Laboratory (ORNL) in Oak Ridge, Tennessee. In November 2008, ORNL researchers announced that the recently upgraded Cray XT Jaguar supercomputer offered a peak performance of 1.64 quadrillion calculations per second—or 1.64 petaflops—making it the most powerful computer ever seen for open scientific research.

Jaguar is the second supercomputer to break the “petaflop barrier,” the first being the IBM Roadrunner system installed at Los Alamos National Laboratory (LANL) in Los Alamos, New Mexico (see “Roadrunner Keeps an Eye on the Nation’s Nuclear Arsenal,” page 82). The new Jaguar uses 45,376 of the latest quad-core Opteron processors from AMD and features 362 trillion bytes (362 terabytes) of memory and a 10-petabyte file system. The machine has 578 terabytes per second of memory bandwidth and an unprecedented input/output bandwidth of 284 gigabytes per second to greatly facilitate the movement of data into and out of processors.

The system has already demonstrated its prowess, boasting the only two real (i.e., nonbenchmark) scientific applications to date that have exceeded a petaflop. One, which investigates high-temperature superconductors, achieved an unprecedented 1.35 petaflops to win the prestigious Gordon Bell Prize, awarded to recognize the world’s fastest scientific computing application. The second, which simulates the electronic structure of systems such as magnetic hard drives, achieved 1.05 petaflops. Both achievements came from teams led by ORNL’s Thomas Schulthess.

Jaguar also took top honors in the annual High-Performance Computing (HPC) Challenge awards, which recognize excellence in handling computing workloads. Jaguar took first place in two of the four categories (speed in solving a dense matrix of linear algebra equations and sustainable memory bandwidth) and third place in another (speed in executing the Global-Fast Fourier Transformation, a common algorithm used in many scientific applications).

Jaguar also recently has been aiding in the search for new drugs against Alzheimer’s disease. In 2007, ORNL researchers Edward Uberbacher and Phil LoCascio used 100,000 processor-hours on Jaguar to investigate the mechanisms by which a new class of drugs acts. The drugs, called caprospinols, may stop the growth of Alzheimer’s fibrils and even disassemble the threadlike fibers. According to Uberbacher, it’s the first time molecular dynamics has been used to simulate the mechanisms these drugs use to interact with Alzheimer’s fibrils and reduce their growth. The findings, he adds, will provide researchers with new ideas about how to improve the drugs that treat the disease.

ORNL originally contracted with Cray in 2004 to build a supercomputer for the researchers and scientists who use the laboratory’s national supercomputing user facility. Since then, Cray and ORNL researchers have rapidly upgraded Jaguar, and it is now the undisputed leader of the pack for open scientific research. Jaguar has been put to work on a variety of petascale research, including designing more efficient car engines, probing the mysterious dark matter pervading the universe, reproducing propagating shockwaves in a supernova and simulating new superconducting materials.

The supercomputer may play a key role in handling the simulations required for the next report from the Intergovernmental Panel on Climate Change, the vanguard organization that predicts the climate’s future. “Climate change is one area where you don’t want to do the experiments,” explained Thomas Zacharia, ORNL’s associate laboratory director for computing and computational sciences. “Instead, you want to model and simulate what’s happening. And you want the highest-fidelity calculations to minimize uncertainty. That way, the policy decisions that have to be made are based on the most comprehensive, most sound scientific calculations.”

Petascale Earth climate simulations enable scientists to incorporate increasingly complex global models that take into consideration such parameters as atmospheric wind flow, ocean currents and land and sea ice melt. The models help scientists better understand the ebb and flow of climate processes and feedback, such as the global water cycle, which in turn produces information on the timescales useful to resource managers and planners. Only supercomputers on a par with Jaguar can provide models with the kind of predictive skill needed for making decisions about the future of Earth’s climate.

Supercomputer experts predict that Jaguar and other petascale systems will transform science, acting as a “third pillar” to discovery, one that complements the centuries-old pillars of theory and experimentation. That is, supercomputers and their ability to simulate complexity now offer another broad avenue toward understanding the universe. “Supercomputing, in many ways, touches all aspects of humanity,” Zacharia reflected. “It has great power to accelerate progress. In fact, I think of supercomputers as time machines. For the first time, they allow us to predict the future.”