High-performance computing eases stress on transportation infrastructure

As Argonne researchers consider the future of transportation, they are turning increasingly to computer simulations that can break down complex systems into millions of constituent parts. These models create an accurate and detailed picture that addresses the many challenges that face our intricate transportation infrastructure and saves time and money for the people who use it.

In the control tower at the DuPage County Airport in West Chicago, air traffic controllers direct planes taking off and landing on the main runway. In the building next door, Argonne scientists and engineers at the Transportation Research and Analysis Computing Center (TRACC) have begun to take a more grounded approach to transportation. They focus on easing the strain on overtaxed road and rail infrastructure and on saving lives in accidents and disasters, either natural or man-made.

Under a multiyear grant from the U.S. Department of Transportation (DOT), TRACC will staff and operate a state-of-the-art, high-performance computing center that will provide the necessary computational tools and resources to address these important problems.

In a small office building in the middle of the DuPage National Technology Park sits TRACC's heart and soul: a brand new supercomputer that contains 512 processors with a combined speed of nearly two teraflops. This system will add significantly to the computational resources of the DOT's user community by providing a production-level, high-performance computing environment that also uses the Linux operating system, allowing it to run nearly any piece of commercially available software as well as software under development by the transportation research community.

“By enlisting both high-performance computing resources and technical staff with expertise in parallel computing and engineering analysis applications, TRACC represents a valuable resource for the DOT research community,” said TRACC Director Dave Weber. TRACC will become a hotbed of focused computation-based research in areas of critical importance to DOT.

A bridge to tomorrow

Last August 1, during the height of evening rush hour, the I-35W bridge that spanned the Mississippi River just outside of Minneapolis, Minn., unexpectedly collapsed, killing 13 people and injuring more than 100. While bridges fail occasionally, such tragedies usually result from the impact of unusually heavy stresses: flooding, high winds or collision. The I-35W bridge, however, had experienced none of those, leaving engineers struggling to provide an explanation.

Recent research at TRACC aims to help bridge architects avoid similar catastrophes in the future. “It turns out that a lot of bridges that fail prematurely fail because of hydraulic reasons,” said Tanju Sofu, an Argonne nuclear engineer who has turned his attention to constructing computational fluid dynamics (CFD) models of river beds and bridge supports.

According to Sofu, water that flows under a bridge often transports sediment that can “scour,” or erode, the bridge supports. This underwater process, though gradual and invisible to normal observers, can so thoroughly wear away a bridge support that the entire structure collapses.

To prevent scour, bridge architects will have to either design their supports or redirect the current in a way that minimizes the quantity and the velocity of sediment that washes up against the pillars. These designs necessitate scientific methodologies that can take into account large sections of the bridge structure at once.

For that reason, Sofu explained, experimental testing cannot provide all the varieties of data that engineers need. “Bridge design experiments are not conducted on the bridges themselves but in controlled laboratories on a much smaller scale,” he said. “Researchers are looking for a way to cut down on these experiments because they're very expensive to begin with and cannot provide information that will pertain to every single bridge.”

Computer modeling, on the other hand, has that potential, Sofu said. “With access to supercomputers, we can simulate how individual bridges interact with sediment transport, local topography and changing climate conditions. We can create as many bridges as we want on the computer without actually having to build and test them.”

While civil engineers consider scour one of the most insidious bridge stresses, they also study the effects of more recognizable strains. In the aftermath of the devastation wrought by Hurricane Katrina on the infra structure of New Orleans, they have redoubled their efforts to ensure that bridges can withstand heavy floods and high winds.

While CFD models give research ers a good picture of how fluid pressures will be applied to a bridge, finite- element analysis predicts how the bridge structure will react to these pressures. Argonne engineer Ron Kulak and researchers from Turner-Fairbank Highway Research Center use models of cable-stayed bridges with hundreds of thousands of elements. That way, Kulak says, they can model the motion of every single part of the bridge, down to the individual fibers of the supporting cables, as traffic, heavy winds or other meteorological phenomena stress the bridge. Computer-based research at this highly detailed level promises to prevent future bridge disasters and save lives.

Finding the key to gridlock

At 8:24 a.m. on a bright September morning, a man pulls into the exit lane and turns off the Kennedy Expressway on his way to work on Roosevelt Road downtown. One minute and 17 seconds later, his wife pulls into a parking lot in Des Plaines en route to a dentist appointment. Twelve minutes and 32 seconds later, their son's school bus drops him off for the first day of fifth grade in Evanston.

It's just another Tuesday in the Transportation Analysis Simulation System (TRANSIMS), the software used by Argonne researcher Hubert Ley and his team at TRACC to simulate the “second-by-second movement of absolutely everybody during an entire day in the entire Chicago metropolitan area,” as Ley puts it.

TRANSIMS tracks each of the 25.5 million automobile trips and millions of bus and train rides each day on Chicago's transportation grid, a vast network of roads that stretches from Kankakee north to Milwaukee and Lake Michigan west to Rockford.

Obviously, the program isn't psychic. TRANSIMS won't enable you to find out where Mayor Daley eats lunch or to make sure your spouse stops at the grocery store for eggs on the way home in the evening. Instead, the millions of “people” whose movements TRANSIMS follows consist of composites created by Ley from detailed census data that represent the inhabitants of metropolitan Chicago. This project provides an example of how basic research can work in the long run to improve the daily lives of millions.

This type of modeling, known in the transportation industry as “microsimulation,” offers a number of advantages over older re-creations of the transportation grid that looked only at road capacities and typical loads. “It's not silly to wonder, ‘why are we doing it this way? Why do we have to follow every single person at every single second if there are surely simpler methods?' ” Ley said. “But the key is, although we can say ‘there are 15,000 cars per hour on this road or 12,000 on that one,' only the new model can tell us if a car will turn left at the next inter section or keep going straight.”

Although the results generated from microsimulation do not have a great deal of reliability at the level of a single street, the composite behavior of large numbers of synthetic “people” yields a thoroughly realistic representation of a typical day's traffic pattern in Chicago.

Ley's models work because they incorporate information compiled from surveys of Chicago-area residents that describe their movements during the course of an entire day, he said. Ley classifies the destinations of every trip gleaned from the survey as a particular “activity location,” such as “work,” “shopping,” “school” or “home.”

While TRANSIMS calculations are based on behavior patterns during a typical day, the detailed models of the traffic grid will enable researchers to better understand and anticipate likely bottlenecks during an emergency.

“It's a real challenge to try and use surveys of normal days to simulate behavior in emergency situations,” said Argonne transportation researcher Young-Soo Park. “That's something that has not been done before and that we're only able to do through our micro-level simulation of individual cars.”

Pileups from the bottom up

Substituting accurate computer models for most real-world crash tests can save huge amounts of both money and time. “Maybe something goes wrong during one crash test and you don't get data, but in the end you've still wrecked a car or perhaps even several cars,” said engineer Kulak, who leads TRACC's effort in computational structural mechanics.

The algorithms that TRACC uses to simulate car accidents work by breaking down a car's components into hundreds of thousands of small, but mathematically digestible, “finite elements.” Each finite element, whether a little section of fender or air bag, is separately calculated, microsecond by microsecond, using LS-DYNA, an advanced simulation software package specifically designed to analyze complex physical problems and created by the Livermore Software Technology Corporation.

The crash then unfolds onscreen as the computer's 512 processors recombine the data from the separate finite elements. Each simulation typically involves between 500,000 and 2.4 million separate elements, Kulak said. The visualizations of the crash often show the cars as a collage of different colors—a navy front wheel well, a pink taillight, a maroon windshield—to indicate which processors contributed which calculations.

Kulak and his Argonne and DOT colleagues are able to perform these intricate simulations only because they have access to an enormous number of processors. A single run of a crash simulation using just the two processors of a powerful personal computer would take more than 17 hours, according to Kulak. TRACC's new computer, by contrast, can perform the same computations in roughly 20 minutes.

While TRACC is not the first organization to under take crashworthiness finite-element analysis, Kulak and his colleagues at DOT hope to use the same methodology to study trauma suffered by passengers in motor vehicle collisions. “In the earlier days of crashworthiness testing, engineers only considered the damage to the vehicle,” he said. “But eventually, they realized that what was important wasn't the vehicle, it was the occupants, like you and me.”

Scientists who had previously tried to assess the impacts of crashes on drivers and passengers relied on simplistic representations of the human body. These early models treated each part of the body as a single entity, ignoring the complex interactions of bones, organs and soft tissues that occur during an accident.

While these re-creations provided scientists and engineers with important data and helped save lives, Kulak believes that finite-element analysis offers a more comprehensive and realistic picture of an accident victim because of its ability to incorporate biomechanical information.

With almost surgical precision, a finite-element model of a person partitions each part of the body—lung, wrist, heart, thigh, skull and others—and assigns tiny sections of them to individual processors. These processors then analyze the body's behavior during an accident and compile the data. To achieve this degree of fidelity, the simulations need to incorporate between five and ten million discrete elements, heavily taxing the capabilities of even the most advanced supercomputers, Kulak said.

For more information, please contact Dave Baurac (630/252-5584 or media@anl.gov) at Argonne.