Millions of protons, approaching the speed of light, will collide per second, when the Large Hadron Collider (LHC) comes online this fall. The experiment will generate more data than the international scientific community has ever tried to manage. Scientists suspect the outcome of these “subatomic smashups” will provide valuable insights into the origins of matter and dark energy in the Universe. As thousands of researchers across the globe anxiously await the results of this experiment, getting the massive amounts of data to them is no insignificant task. Fortunately, network engineers at the U.S. Department of Energy’s (DOE) Energy Sciences Network (ESnet) foresaw this data challenge years ago and developed ESnet 4, a new large-scale science data transport network with enough bandwidth to transport multiple streams of 10 gigabits of information per second – the equivalent of transmitting 500 hours of digital music per second for each 10 gigabit line

DOE Press Release... To Learn More: The US LHC Web Site... The CERN Web Site... ESnet Web Site...

The LHC will be the first experiment to fully utilize the advanced capabilities of this network, which connects DOE national laboratories to researchers across the country.

“ESnet 4 is one of the most robust scientific data networks in existence,” says Steve Cotter, Department Head for ESnet. “The science environment of today is very different from that of a few years ago. ESnet 4 provides the high-speed, extremely reliable connectivity between labs and U.S. and international research institutions required to support the inherently collaborative, global nature of modern large-scale science.”

ESnet is funded through the DOE, and based at the Lawrence Berkeley National Laboratory, Berkeley, Calif.

Flowing Information to America

The European Center for Nuclear Research (CERN), which manages the LHC, will initially collect experiment’s data. The information will then migrate across the Atlantic Ocean via fiber optics, on a network called USLHCnet, which is managed by the researchers at the California Institute of Technology in Pasadena, Calif.

Like a virtual Ellis Island, an ESnet hub on 8^th street in Manhattan will be the US entry point for LHC data. From there, ESnet will deliver data from the LHC’s ATLAS detector to Brookhaven National Laboratory, in Upton, N.Y. where it will be processed and stored. Meanwhile, data from the LHC’s CMS detector will go to the Fermi National Accelerator Laboratory in Batavia, Ill., for processing and storage.

Researchers at universities and DOE Laboratories across the country will then be able to connect to these databases through the ESnet4, the DOE’s next-generation scientific network. Internet2, the country’s leading education and research network, and ESnet officially launched a partnership in 2006 to develop and deploy ESnet4 just in time for the LHC experiment.

To maximize efficiency, ESnet4 utilizes three main elements:

• A circuit-oriented Science Data Network for moving terabytes of data. Like a direct line connecting two endpoints, this dedicated network allows information to flow directly at high data rates from one remote host to another.
• An Internet Protocol (IP) network for typical data transfers. Unlike the Science Data Network, the IP network is connected to many computers, and can have multiple endpoints. To reach a destination, information traveling on an IP network will constantly encounter “gateways” that quickly direct and redirect it. Like air-traffic controllers, the gateways virtually determine which routes are “preferred” and find the most efficient routes for travel. Because information will move through numerous gateways before reaching its destination, IP networks are not the most efficient tool for moving massive datasets. Like hundreds of cars trying to pass on a toll-road, the gateways can cause virtual “traffic jams” if too much information is trying to pass through. Thus, the Science Data Network is ideal for moving large datasets.

• Metropolitan Area Networks (MANs) are the last component of ESnet4. This system carries both Science Data and IP networks to effectively connect research centers in the same geographic region. Currently, 11 ESnet sites are served by MANs. The Long Island MAN and the Chicago Area MAN were specifically built to facilitate the movement of data from the LHC experiments.

“LHC is just the beginning. ESnet4’s innovative and reliable infrastructure allows scientists from all over the world, and across disciplines, to exchange large datasets and analyses in an efficient way. It is these collaborations, this sharing of information, that allows us to better understand the world around us,” says Joe Burrescia, General Manager for ESnet.

About ESnet and Berkeley Lab

ESnet is funded by the DOE Office of Science to provide network and collaboration services in support of the agency's research missions. A pioneer in providing high-bandwidth, reliable connections, ESnet enables researchers at national laboratories, universities and other institutions to communicate with each other using the collaborative capabilities needed to address some of the world's most important scientific challenges. The ESnet Department is part of the Computational Research Division at Berkeley Lab. For more information about ESnet visit: http://www.es.net/

Berkeley Lab is a U.S. Department of Energy national laboratory located in Berkeley, California.  It conducts unclassified scientific research and is managed by the University of California. Visit our Website at www.lbl.gov/