Connectivity
As a national facility whose users routinely transfer massive quantities of data, NERSC needs fast networks as much as fast computers. DOE’s Energy Sciences Network (ESnet) provides high performance connections between DOE sites as well as links to other networks on the Internet. This year NERSC upgraded its border router connection to ESnet to OC-48 (2.4 Gb/s). Two one-gigabit Ethernets channeled together link the router with NERSC’s internal network. The router itself was upgraded to Jumbo Frames capability, enabling NERSC to send 9,000-byte data packets across the Internet instead of the previous 1,500-byte packets.
Integration of NERSC’s computing and storage systems into the DOE Science Grid is almost complete. All production systems have GridFTP data transfer functionality, including Seaborg, HPSS, PDSF, and the math and visualization servers. Globus libraries have been installed on Seaborg’s interactive nodes and will be running on all nodes in early 2004, when the Globus Gatekeeper interface will also be installed, providing basic data and job submission services. A Grid interface to the NERSC Information Management (NIM) system will make it easier for users to get Grid authentication certificates. The active intrusion-detection program Bro is being modified to monitor Grid traffic. And a Grid-enabled Web portal interface called VisPortal is being developed to deliver visualization services to Grid collaborators (see Advanced Devlopment).
In the past year, viruses and worms with names like SoBig, Blaster, Nimda, Klez, Slammer, and Bugbear were wreaking havoc across the Internet, causing estimated losses of $35 billion. In the case of SoBig, over one million computers were infected within the first 24 hours and over 200 million computers were infected within a week. Nevertheless, NERSC had no major security incidents, thanks to its alert security staff and intrusion detection by Bro, which is updated regularly to defend against emerging threats. In addition, NERSC’s server team upgraded the email server and added a spam filter.
When DOE officials realized that they were unable to categorize and measure different types of network traffic on ESnet, NERSC’s networking and security group, led by Howard Walter, stepped in to meet the challenge. In less than a month, using network monitoring data from Bro, routers, and databases, Brent Draney, Eli Dart, and Scott Campbell produced data that clearly categorized more than 95% of all network traffic (Figure 1). The categories they developed—bulk data, database, Grid, interactive, mail, system services, and Web—are now in use at all DOE laboratories and facilities to monitor and measure network traffic. The most important outcome of this effort was the demonstration that ESnet is very different from commercial Internet service providers, and that DOE network traffic is clearly driven by science.
 |
Figure 1 |
Although the Berkeley Lab/NERSC Bandwidth Challenge team (Figure 2) retired from the competition after winning three years in a row at the annual SC conference on high performance computing and networking, NERSC’s networking and security staff continued to play a major role at SC2003 in Phoenix, helping to create an infrastructure providing the 7,500 attendees with high-speed wired and wireless connections. Bro was used to monitor the conference network for security breaches, and passwords sent unsecurely over the network were displayed on a large plasma screen in the exhibits hall. The display continually attracted viewers, looking to see if their passwords had been grabbed. Another screen showed the number of complete and incomplete network connections emanating from the conference. The strong interest in Bro was demonstrated by a standing-room-only turnout for a presentation on Bro in the Berkeley Lab booth shortly before the end of the conference.
 |
Figure 2 |