|
|
|
|
|
|
NETWORKING: |
Big science requires big computers that are not just scaled-up desktop personal computers. Big computers are fundamentally different from PCs in their ability to model enormous systems, generate immense volumes of data, and, as a payoff, solve uniquely difficult scientific problems. To put this difference in perspective, next-generation science datasets will approach or exceed a petabyte in size. If one of today's desktop PCs had a disk able to hold a petabyte-sized file, the PC would require over three years to read the file. To make their discoveries, scientists must interact with supercomputers to generate, examine, and archive huge datasets. To turn data into insight, this interaction must occur on human time scales—not over days or weeks, but over minutes. Large-scale science projects increasingly require close collaboration among domain experts located across the nation or around the globe. However, because even high-performance networks cannot deliver next-generation science datasets to users on human time scales, supercomputers require physical proximity of the scientists who want to interact with the data. Entire teams must travel, sometimes for days, to a supercomputer they will use for as little as an hour. Thus, the development of super-networks scaled to these supercomputers holds enormous promise for improving the efficiency and productivity of big science projects. The super-networks will require unprecedented capabilities, significantly beyond today's Internet technologies. First, there is the issue of sheer bandwidth. The fastest commercial networks available today have a theoretical bandwidth of 10 gigabits per second (Gbs). If such a network could deliver all its capacity to a supercomputer, it would still take a month to move a petabyte file to the researcher. Bandwidth must be greatly increased. Second, network technologies do not exist that can stably support interactive visualizations and steer computations of this scale. With current Internet protocols visualizations drop out of sync and computations wander off into unwanted regions, wasting precious computing power. This situation is also unacceptable. Clearly, the next generation of supercomputers will require comparable network capabilities to make them productive. Several ORNL networking projects, funded by the Department of Energy, National Science Foundation, and Defense Advanced Research Projects Agency, are under way to address these important networking problems.
DOE UltraScience Net The Center for Computational Sciences at ORNL has been tasked by DOE to develop the next generation of scientific networks to address the challenges of large science applications. The techniques developed in Oak Ridge will eventually filter out into the high end of the business world. Just as yesterday's scientific supercomputers have become today's central business and engineering computers, the same transfer will result in this network, called the DOE UltraScience Net, becoming the core of tomorrow's commercial networks. Today's commercial networks are optimized for carrying huge numbers of small data streams. What big science needs are networks optimized for small numbers of large data streams and high precision control. High-energy physics, climate modeling, nanotechnology, fusion energy, astrophysics, and genomics are among the research areas that will benefit from the UltraScience Net. Computational steering and instrument control required to run experiments remotely place different types of demands on a network, making this task far more challenging than designing a network system solely for transferring data. ORNL researchers will take advantage of current optical networking technologies to build the prototype network infrastructure. This infrastructure will enable development and testing of the scheduling and signaling technologies that will be needed to process requests from users and optimize the system. The UltraScience Net, which will operate at 10 Gbs to 40 Gbs, will develop (and demonstrate) the techniques that will allow networks to deliver as much as 200 Gbs. Eventually, the UltraScience Net could become a high-impact, special-purpose network that will connect DOE laboratories, collaborating universities, and institutions around the country. Such a 200 Gbs network will provide scientists with dedicated on-demand access to UltraScale data. While these new networking techniques of tomorrow are being developed, they will need a yardstick against which they can be compared. The yardstick will be provided by the TeraGrid. Cyber-Science via the TeraGrid
A visiting
neutron scattering scientist at ORNL sends data from her experiment
to a San Diego supercomputer for analysis. The calculation
results are sent to Argonne National Laboratory, where they are turned
into "pictures." These visualizations are sent to a
collaborating scientist's workstation at North Carolina State University
(NCSU), one of the core universities of UT-Battelle, which manages
ORNL for DOE.
The $1.4 billion Spallation Neutron Source is on time for completion in 2006 in Oak Ridge. The High Flux Isotope Reactor is the world's most powerful source of thermal neutrons used to unlock the molecular secrets of materials and to provide radioisotopes for a number of medical, industrial, and academic uses. Data from the two neutron sources will be made available through CCS. As ORNL becomes the world's foremost neutron science center, researchers from around the nation will be able to analyze its data up close or at a distance. Because of a $3.9 million grant from NSF to the Center for Computational Sciences at ORNL, a network hub and high-performance network connections are being established to support access to ORNL's neutron science instruments across the TeraGrid. The TeraGrid is part of a high-speed network that will provide scientists with extraordinary amounts of data from ORNL's High Flux Isotope Reactor and the accelerator-based Spallation Neutron Source. When complete, the TeraGrid's network backbone will operate at 40 Gbps, making it the fastest research network in the world today. The ORNL-led addition to the TeraGrid, called the Southeastern TeraGrid Extension for Neutron Science (SETENS), will allow scientists working at these facilities to use the massive computing and data storage resources on the TeraGrid to rapidly make detailed analyses and visualizations of the data from neutron scattering experiments. The system will provide near real-time feedback. SETENS represents a major commitment to the region and to economic growth in the Southeast. From an economic development perspective, these resources show a continued commitment to build the intellectual capital of the Southeast—an investment that will reap benefits in terms of new business and research opportunities for decades to come. eScience Science projects that are enabled by computational and other resources distributed across networks are often referred to as eScience projects. Both UltraScience Net and TeraGrid projects provide the networking infrastructures in support of such projects—the former provides dedicated channels and the latter provides shared links like the Internet. ORNL is involved in jointly developing the associated technologies needed to bring the network capabilities to eScientists in a transparent manner. Two joint NSF projects are under way in providing the transport, middleware, and visualization capabilities, all optimized to UltraScience Net and Internet infrastructures. Funded at $3.5 million, the CHEETAH project is a joint venture involving ORNL and the University of Virginia, City University of New York, and NCSU. The NetReact project involves ORNL and Georgia Tech and is funded at $1.2 million. While both projects specifically target the SciDAC Terascale Supernova Initiative, led by ORNL, they also provide valuable enabling networking technologies for other DOE large science projects. Using these technologies over UltraScience Net, the goal is to control supernova computations at ORNL by researchers at NCSU and UVA. Precision Control Controlling remote devices and instruments over the Internet is extremely challenging. The randomness caused by Internet traffic makes control very complicated, and few technologies are available to help. Under joint $1 million funding from DOE and DARPA, however, technologies are being developed at ORNL for performing such control operations across the networks. In short, ORNL networking research capabilities, and the projects described here, will advance the state of the networking art, helping position CCS to host leadership-class scientific computing.
|
 |
 |
 |
 |
 |
Web site provided by Oak Ridge National Laboratory's Communications and External Relations |
