Scientific Computing

From Data to Science Click image for a larger version.

The NSSD Scientific Computing Group has wide ranging duties, which include data management, data reduction, analysis, simulation, and visualization.  Central to this is managing the lifecycle of data produced by a data acquisition system, which is then cataloged, stored, and archived.  The data are stored in a hierarchical storage system with more recent data stored on fast access disk storage and older data will be stored on a tape archive system.  Experiment data have metadata associated with it and a subset of this metadata such as facility name, instrument name, principal investigator name, experiment title, creation date, or sample name (and more) are used to catalog the data.  Once data are cataloged, users can then use these fields to search for data of interest to them.

A web portal interface has been adopted to facilitate easy access to data and computing resources.  A user logs in to the portal and can browse and search their data.  Data which users produce via analysis or simulation are stored in their own data areas which they can then access.  Collaboration is facilitated as groups are formed according to accepted proposals and all members of a group can access the same experiment data, but still have a home area with data private to themselves. 

The data management system is suitable to support multiple facilities.  Initially data from SNS and HFIR are stored and cataloged while support for additional facilities such as Lujan and IPNS can be added at a later date.  Data will be automatically cataloged once the centralized data management system receives a message that new data are available for catalog and storage.

The portal also supports running analysis and simulation computing jobs such as data reduction or McStas simulation runs.  These jobs may require significant computing resources and may work with large data sets.  As the number of users requesting computing resources grows, the ability to manage compute jobs is crucial and is being incorporated with the portal.  In this way, jobs can be run in a systematic way without overwhelming resources.  To increase computing capacity, access to the NSF TeraGrid is being provided to users via a community account.  The TeraGrid is an extensive national computing resource available to users and our initial use has been to run McStas simulations on the TeraGrid.

In addition to the data reduction software being produced at SNS, community produced software is also being made available such as the DAVE software developed at NCNR, ISAW developed by IPNS, DANSE software as it is developed via the NSF DANSE project.  The portal provides an open environment for hosting a variety of software, easy, personalized access to data, and access to extensive computing resources.