Participating Institutions: Oak Ridge National Laboratory The University of British Columbia North Carolina State University |
FreeLoader: Distributed Storage Scavenging |
                    People                     Publications                     Testbed                     Positions |
Increasingly scientific discoveries are driven by analyses of massively distributed bulk data. This has led to the proliferation of high-end mass storage systems, storage area clusters and data centers as storage fabric elements for supercomputing, offering excellent price/performance ratio and good storage speeds, but increasing maintenance and administrative costs. A promising alternative then, is to harness the collective storage potential of individual workstations much as we harness the idle CPU cycles due to the affordable economics in aggregating commodity storage and low usage to available space ratio. However, such aggregated commodity storage is prone to volatility, machine failures, performance concerns and trust issues.
The FreeLoader project is an effort to aggregate space, and I/O bandwidth contributions from commodity desktop storage within a domain to provide a shared cache/scratch space for large, immutable data sets. The FreeLoader architecture comprises of contributing benefactor nodes that are aggregated into pools, steered by a management layer. Collectively, these entities provide services such as reliability, high performance, availability and load balancing.
The following Supercomputing 2005 paper, ORNL-TR-P05-123435, reflects our current thinking.
FreeLoader Primary Usecase Picture
Other FreeLoader Usecases Picture
Check out Dilbert's view on user desktop sharing... :) (comic strip 11/28/2004).
NEW!: ORNL: Internships available all through the year