NERSC logo National Energy Research Scientific Computing Center
  A DOE Office of Science User Facility
  at Lawrence Berkeley National Laboratory
 

Scientific Data Management Technology

The following tables list and briefly describe SDM applications installed on NERSC systems, or under development. Links to NERSC systems are available from the High Performance Computing Resources page. Each NERSC system page has a link to the software installed on that system.

Note that different versions of an application may be installed on the same system or on different systems. For complete platform and version information, see the NERSC resource pages for Sofware Applications (General) and I/O Libraries. These pages also indicate whether you will need to load the application using the module command. If you have questions about the applications listed below, please send email to consult@nersc.gov.




NERSC Tools for File Transfers and Accessing HPSS

     bbcp   NERSC information on file transfers using bbcp (on davinci and jacquard)
     GridFTP   NERSC information on file transfers using GridFTP
     ftp/pftp   NERSC information on accessing HPSS using ftp/pftp
  hsi   NERSC information on accessing HPSS using hsi
  htar   NERSC information on accessing HPSS using htar

 



NERSC Tools for Database Systems and Data Formats

ApplicationPlatformsDescriptionDocs
hdf5  bassi, davinci, franklin, jacquard, seaborg  HDF5 I/O (new Hierarchical Data Format) library. HDF5 is a successor to HDF. Its a complete redesign of the format and library with improved support for parallel I/O. It currently supports Fortran 90 and C interfaces.   NERSCVendor 
MySQL  bassi, davinci, franklin, jacquard, seaborg  MySQL is an open-source database management application.   NERSCVendor 
netcdf  bassi, davinci, franklin, jacquard, seaborg  netCDF(network Common Data Form) - Software for storing and retrieving scientific data.    NERSCVendor 
Parallel NetCDF  bassi, franklin, jacquard  Parallel NetCDF Library   NERSCVendor 
root  pdsf  The ROOT system provides a set of OO frameworks with all the functionality needed to handle and analyse large amounts of data in a very efficient way.    Vendor 

More information on these applications

     HDF5   HDF5 Papers
  MySQL   White Papers
  NetCDF   Writing NetCDF Files: Best Practices

 


Tools Under Development

ApplicationDescriptionDocs
H5Part Simple data storage schema and API that simplify the reading/writing of particle data to the HDF5 file format. H5Part is built on top HDF5. HowToMore Information
HDF5-FastQuery  High-level API to HDF5 that accelerates complex queries on HDF5 and H5Part files using multi-dimensional semantic indexing (Fast Bitmap Indices). The current implementation supports indices with and without bins. Case StudyMore Information 
SRM Unified API for location transparent file access and transfer. SRM manages transfers and access of large files across different storage systems both on disk and tape, provides dynamic space allocations, and is fault-tolerant. HowToCase StudyMore InformationNERSC Staff Only

Top of Page 


LBNL Home
Page last modified: Wed, 12 Sep 2007 00:36:58 GMT
Page URL: http://www.nersc.gov/nusers/analytics/sdm/sw/
Web contact: webmaster@nersc.gov
Computing questions: consult@nersc.gov

Privacy and Security Notice
DOE Office of Science