Oak Ridge Leadership Computing Facility

User group and developer sign development contract

The future looks bright for the Lustre parallel file system. In August, OpenSFS and software developer Whamcloud agreed on key improvements to the open-source software system, resulting in a $2.1 million, multi-year contract.

Lustre is the file system used by over 50 percent of the world’s top 500 fastest computers. OpenSFS is a non-profit, technical organization which facilitates collaboration among users and developers of high-end, open-source file systems. Whamcloud is a venture-backed company formed by veteran Lustre developers with the goals of maintaining and enhancing the software for the user community as a hardware-neutral file system.

Over the next two years, the contract work will address three areas of development: Single Server Metadata Performance, Distributed Name Space, and the Online Lustre File System Checker. “The contract is specifically aimed at improving the metadata performance of the system, improving robustness and reliability of the system, and scaling the system through a distributed namespace,” said David Dillow, HPC Systems Engineer at the Oak Ridge Leadership Computing Facility (OLCF) and co-chair of the OpenSFS Technical Working Group.

With the expected March 2012 release of Lustre 2.2, users will see improved metadata performance for parallel operations in a single directory—a scenario commonly found when a parallel application writes its data. “While we expect to see early results in 2.2, the bulk of the work is more involved and should land in 2.3 or 2.4 as it gets completed,” said Dillow.

When the Oracle Corporation bought Sun Microsystems, and the rights to Lustre, in 2010, the user community anxiously awaited information about the file system’s future. Oracle only supports the 1.8 version of Lustre, so OpenSFS was formed to help drive community development. OpenSFS’s goals are to determine and bring about desired user requirements in future Lustre enhancements, ensure access to the open source file system, and foster and enable a healthy user community. Members include Oak Ridge National Laboratory, Cray Inc., Lawrence Livermore National Laboratory, Data Direct Networks, Indiana University, Sandia National Laboratories, Terrascala, Raid Inc., SGI, and Whamcloud. —by Sandra Allen McLean