The Simple, Scalable, Script-based Science Processor for Measurements (S4PM) is a
system for highly automated processing of science data. It is the main
processing engine at the Goddard Earth Sciences Data and Information Services
Center (GES DISC). In addition to being scalable up to large processing
systems such as the GES DISC, it is also scalable down to small,
special-purpose processing strings.
It consists of two main parts: the kernel is the Simple, Scalable,
Script-based Science Processor (S4P), an engine, toolkit and graphical monitor
for automating script-based, data-driven processing. The S4PM system is built
on top of S4P and implements a fully functioning processing system that
supports a variety of science processing algorithms and scenarios.
S4PM requires Perl (ideally 5.6 or higher) along with the Perl Tk module and has been run successfully on
Irix, Linux (Red Hat), Solaris, Macintosh OS X, and Windows.
S4PM was released to the open source community under the NASA Open Source Agreement in April 2005 with version 5.6.2. The software is available at SourceForge at this URL:
http://sourceforge.net/projects/s4pm/.
Goals of S4PM
The main goal of S4PM is to automate science processing to the extent that a
single operator can monitor all of the processing in an "industrial-size" data
processing center. A second goal is to be flexible enough to easily add new
processing strings or new algorithms to an existing string with a minimum of
effort.
High usability is another key goal of S4PM, deriving from the need for more
automation at less operational cost. Specific goals are:
- Allow a single operator to manage and monitor hundreds of jobs
simultaneously.
- Drill down to troubleshoot a problem in two mouse clicks.
- Set up a new processing string in less than 30 minutes.
Where To Get S4PM
S4PM is available under the NASA Open Source Agreement at http://sourceforge.net/projects/s4pm/. The current stable version is 5.27.0. Version 5.28.0 is expected out in late 2008.
Download S4PM now!
Future Directions
The architecture of S4PM and S4P was specifically designed to be highly
modular so that it could evolve quickly and flexibly. It has already evolved
from data-driven processing of MODIS
instrument data to AIRS processing to
on-demand subsetting based on user requests.
For the future, S4PM will evolve to:
- Support an ever-increasing variety of processing algorithms, scenarios
and data interfaces.
- Increase the automation of failure monitoring and recovery.
- Reduce the time and expertise needed to setup and adapt S4PM to new
processing algorithms.
We hope that some or all of these goals will be reached by collaborating with
the open source community.
S4PM Resources
The following resources are available to help you use and understand S4PM.
You may also e-mail us at s4pm-disc@listserv.gsfc.nasa.gov.
|