S4PM

The Simple, Scalable, Script-based Science Processor for Measurements (S4PM) is a system for highly automated processing of science data. It is the main processing engine at the Goddard Earth Sciences Data and Information Services Center (GES DISC). In addition to being scalable up to large processing systems such as the GES DISC, it is also scalable down to small, special-purpose processing strings.

It consists of two main parts: the kernel is the Simple, Scalable, Script-based Science Processor (S4P), an engine, toolkit and graphical monitor for automating script-based, data-driven processing. The S4PM system is built on top of S4P and implements a fully functioning processing system that supports a variety of science processing algorithms and scenarios.

S4PM requires Perl (ideally 5.6 or higher) along with the Perl Tk module and has been run successfully on Irix, Linux (Red Hat), Solaris, Macintosh OS X, and Windows.

S4PM was released to the open source community under the NASA Open Source Agreement in April 2005 with version 5.6.2. The software is available at SourceForge at this URL:

http://sourceforge.net/projects/s4pm/.

Goals of S4PM

The main goal of S4PM is to automate science processing to the extent that a single operator can monitor all of the processing in an "industrial-size" data processing center. A second goal is to be flexible enough to easily add new processing strings or new algorithms to an existing string with a minimum of effort.

High usability is another key goal of S4PM, deriving from the need for more automation at less operational cost. Specific goals are:

S4PM is available under the NASA Open Source Agreement at http://sourceforge.net/projects/s4pm/. The current stable version is 5.27.0. Version 5.28.0 is expected out in late 2008.

Download S4PM now!

Future Directions

The architecture of S4PM and S4P was specifically designed to be highly modular so that it could evolve quickly and flexibly. It has already evolved from data-driven processing of MODIS instrument data to AIRS processing to on-demand subsetting based on user requests.

For the future, S4PM will evolve to:

• Support an ever-increasing variety of processing algorithms, scenarios and data interfaces.
• Increase the automation of failure monitoring and recovery.
• Reduce the time and expertise needed to setup and adapt S4PM to new processing algorithms.

We hope that some or all of these goals will be reached by collaborating with the open source community.

S4PM Resources

The following resources are available to help you use and understand S4PM.

• All Release Notes
• S4PM 5.27.0 Release Notes
• S4PM Installation and Configuration Guide April 2008 (PDF)
• S4PM Operations Guide (PDF)
• S4PM Design Document February 2008 (PDF)
• S4P Users Guide September 2008 (PDF)
• Simple, Scalable, Script-Based Science Processor (S4P) - Paper presented at the 2001 IGARSS Meeting (Microsoft Word, 77 kB)
• S4PM Project on SourceForge
  

You may also e-mail us at s4pm-disc@listserv.gsfc.nasa.gov.