S4PM 5.25.0 RELEASE NOTES

Bug 4937 Fixed: Made Dependency on S4P::EchoSearch Optional - A change was made so that the SOAP::Lite module isn't required unless S4P::EchoSearch is actually invoked (it isn't in typical S4PM installations). This was accomplished by making the function that requires the SOAP module, S4P::EchoSearch, itself optional.

Bug 5036 Fixed: Support For String Type Data Versions - A number of fixes and modifications were made so that string type data versions can be fully supported in S4PM.

Bug 5153 Fixed: Proper Handling of Regex/Non-Regex Work Order Types - Fixed stationmaster.pl to properly handle mixed regex and non-regex commands hashes.

Bug 5189 Fixed: Additional Fixes for Data Version Handling - By default, data versions are treated as strings. The data versions as read in from the metadata are unmodified by S4PM be they numeric or alphanumeric. To have S4PM automatically pad out numeric data versions with zeros (e.g. 1 -> 001), you need to set the $use_legacy_dataversion parameter in the Stringmaker string-specific configuration file to non-zero.

Bug 4776 Fixed: View Disk Allocate and Usage Button Grayed Out in Multi-User Mode - Fixed tkstat.pl to account for different number of standard buttons in multi-user mode (i.e., no Stop Fast button)

Bug 5075 Fixed: Acquire Data Via S4PA Machine Request Interface (MRI) - Enhancements were made so that Acquire Data can retrieve data via the S4PA machine request interface (MRI). To handle this, enhancements were made to the setting of the %s4pa_data_map hash, nominally in the Stringmaker Data Types configuration file.

Bug 5169 Fixed: Multi-User Mode Does Not Support Multiple Users Running Stringmaker - A bug prevented all but one user from being able to run Stringmaker. Only the user who created the multi-user mode string could rerun Stringmaker. This should be fixed now.

Bug 5542 Fixed: DPREP Bootstrap Fails in Multi-User Mode - A number of changes were made to s4pm_bootstrap_dprep.pl to get it to work in multi-user mode.

Bug 5648 Fixed: S4PM::DataRequest.pm Requires SOAP::Lite - A bug that caused the S4PM::DataRequest.pm module to require that SOAP::Lite be installed even in cases where it wasn't used has been fixed. The SOAP::Lite is only needed when SOAP services are invoked. This happens in S4PA, but not S4PM.

Bug 5835 Fixed: Data Version of 000 Not Supported - A bug caused failures when the data version was set to '000'. This has been fixed and such data versions are now supported.

Bug 5906 Fixed: Automatically Restart Defunct Jobs - Stationmaster has been enhanced to optionally restart defunct jobs (i.e., RUNNING.* directories whose process_id is no longer active) automatically on restart. This is triggered through a configuration variable $cfg_restart_defunct_jobs.

Bug 5952 Fixed: Production Date/Time Error in S4PM::QC::production_summary() - An error caused the month part of the production date/time to be off by one resulting in January being month 00. This has been fixed.

Enhancement to S4PM_TEST Package - The fake algorithms that run in this S4PM test suite were enhanced to improve testing. Rather than the first algorithm outputting only one output, it now outputs two, both of which are needed by the second algorithm.

New Stringmaker Extensions Configuration File - Some configurations to an S4PM string may be too cumbersome to include the the Stringmaker string configuration file (e.g. entire new stations). Thus, Stringmaker has been modified to read in s4pm_stringmaker_extensions.cfg file as the very last file, if it exists. It comes after the s4pm_stringmaker_derived.cfg file. Anything in this file benefits from variables defined in the earlier files read in. Small station tweaks, however, are better left in the Stringmaker string configuration file.

File in S4PM_TEST. Changed the VersionID in this fake metadata file from 5 to "005".

Modified check_job to allow for interrogating whether the process_id in a running directory is still active

Added is_defunct_job to detect defunct jobs

Added restart_defunct_jobs to restart all defunct jobs

Moved fail_job() and rename_job_dir() from stationmaster.pl

For Bug 5075, added http_get() utility routine

Moved the block of code that calculates the granule size outside ECHO search iterator.

Removed the statements that appended a black spatial element when a bounding box is not specified as a search parameter.

For Bug 5075, added http_get() method.

Added some "use strict" directives, along with some "my" statements to be compliant.

Fix for Bug 5835 with changes in in data_version() and ssh_get().

Fixes for Bug 5189 in handling string type data versions. The S4P::FileGroup::data_version() function was recoded to handle both string data versions as well as legacy integer data versions (padded with zeros) via the environment variable S4P_LEGACY_DATAVERSION. If set, data versions are handled as integers padded with zeros (%03d) if all numeric, as strings otherwise. If S4P_LEGACY_DATAVERSION is not set, data versions are assumed to be strings by default or, if a format specifier is passed in, according to that format.

For Bug 5075, added http_get() method.

To help with Bug 5169, added the -m option that indicates multi-user mode.

For Bug 5036, modified function new() so that when setting the data version, it passes in a string format specifier (%s) unless the version is purely numeric.

For Bug 5036, modified function data_version() so that if a format specifier is passed to it, it will use it rather than the default %03d.

Fix for Bug 5648. Modified so that SOAP::Lite is only loaded when actually required.

Added functions to query S4PA via Machine Request Interface

search_s4pa_mri() - the main function
s4pa_mri_query() - executes MRI query
parse_mri_granule() - parses XML response from MRI
results_key() - generates hash key for results returned from all search functions
parse_xml_tag() - utility routine to extract value from XML tag
is_extra_packet_file() - determines if file is a packet file other than the main one

Refactored find_urls_by_datatime to be more modular (and accept mri routines):

Moved most of previous code into search_metfiles()
Added function identify_datafiles()

For Bug 5036, modified function granule2file_group() so that when setting the data version, it passes in a string format specifier (%s) unless the version is purely numeric.

Switched to using "require" instead of "use" for including S4P::EchoSearch. Also, fixed the bug that was using wrong keys in hash being returned from S4PM::DataRequest::seach_echo().

For Bug 5036, modified function test_data_suite() so that when setting the data version, it passes in a string format specifier (%s) unless the version is purely numeric.

File in S4PM_TEST. Modified the file size of the metadata file in this template DN file.

Fix for Bug 5952 in function production_summary().

Added ability to acquire data via PDR->http_get().

Fixes for Bug 5542 and Bug 6500. A relatively large number of changes had to be made since much was hardwired. The code was using the userid (s4pmops, s4pmts1, s4pmts2) to decide what mode it was in and then using this mode to determine directory locations. I removed the checking for userid and made the paths relative so that the mode should need to be known (or exist).

In working on Bug 5036, I found that make_patterns_hash() was still assuming a fixed file name pattern. I modified it to use S4PM::make_patterned_glob() instead.

Bug fix for 5036 so that numeric data versions are padded with zeros when file names are constructed. Modifications were made to data_info() and set_ur().

For Bug 5036, modified function fix_pdr() so that when setting the data version, it passes in a string format specifier (%s) unless the version is purely numeric.

For Bug 5036, modified function mk_file_group() so that when setting the data version, it passes in a string format specifier (%s) unless the version is purely numeric.

For Bug 5036, modified functions write_moredata_filegroups() and write_request_file_group() so that when setting the data version, it passes in a string format specifier (%s) unless the version is purely numeric.

Fixes for Bug 5169:

When in multi-user mode, configuration files will be read/write for both user and group. In regular mode, these same files will be read/write for user, but read only for group.
Fixed bug in subroutine change_mode() that prevented the function from having any affect when in multi-user mode.
Modified so that it only deals with the ACQParmfile if ECS is an input archive for the string.
Added the proper setting of permissions on the $global_root directory if in multi-user mode.
In make_creation_script, rather than always deleting and reconstituting symbolic links, it first checks to see if it is necessary (i.e. if the target has changed from current setting or not.
In function version(), changed the permission such that the VERSION file is readable/writable rather than just readable.
When in multi-user mode, modified to call s4p_pooldb.pl (Allocate Disk station) with the -m option.

Added support for an optional s4pm_stringmaker_extensions.cfg configuration file. If it exists, it is the last file read in (coming after s4pm_stringmaker_derived.cfg). Being last, it has at its disposal all variables defined earlier and may be most useful for adding entire stations to a string; it may be less cumbersome than putting it in the string-specific configuration file.

Added subroutine get_stations() from s4pm_stringmaker_derived.cfg and did essentially a rewrite of it so that it would properly order the stations according to the new display_order attribute (see s4pm_stringmaker_derived.cfg).

For Bug 5075, updated description of the %s4pa_data_map parameter to include information on how to use it for the machine request interface (MRI).

File in S4PM_TEST. Updated for the changes to the fake algorithms in which the first algorithm outputs two data types (instead of one) and the second algorithm uses both data types as input.

For Bug 5906, added code to set $cfg_restart_defunct_jobs in all stations.

For Bug 5075, added code to handle the new attributes in the %s4pa_data_map hash that support the MRI.

Modified so that the ACQParm file is only built if ECS is an input archive.

Added new stations parameter: display_order to indicate numerically the position on the display of the station. Default values were set for all standard stations so that they are displayed in the order to which we have become accustomed, starting at 10 and incrementing by 10 for each subsequent station. This allows stations to be inserted in between current stations easily. For example: $stations{'allocate_disk'}{'display_order'} = 150;

Also, moved the subroutine get_stations() out and into s4pm_stringmaker.pl.

For Bug 5189, if $use_legacy_dataversion is set, the environment variable S4P_LEGACY_DATAVERSION gets set in a number of the stations that handle data versions. In this way, the environment variable only gets set for those Stationmaster children that need it (as opposed to requiring that it be set in the users startup scripts like .bashrc).

Brand new type of Stringmaker configuration file with documentation and examples.

Modified documentation on the use of the $user parameter so that it is clear that if in multi-user mode, this parameter should not be set.

For Bug 5906, added description and instructions for the new $auto_defunctjob_restart parameter.

Minor fix to move the setting of $has_qc to under where it is explained.

This file, part of the S4PM_TEST package, was synced with the latest version of s4pm_stringmaker_string.cfg.

Updated for the changes to the fake algorithms in which the first algorithm outputs two data types (instead of one) and the second algorithm uses both data types as input.

Added call to restart_defunct_jobs() if $cfg_restart_defunct_jobs is set to non-zero.

Bug fix for 5153: modified to check for exact matches in looking up commands before going to the regex checking.

Moved code out of main() and into subroutines as follows:

lock_station()
run_reserved_job()
require_command_packages()
add_station_to_perllib()

File in S4PM_TEST. Updated for the changes to the fake algorithms in which the first algorithm outputs two data types (instead of one) and the second algorithm uses both data types as input.

Modified code to account for different number of buttons in multi-user mode.

NASA Official: Steve Kempler
Website Curator: Anthony Drake

Last updated: July 20, 2009 20:32:46 GMT

GES DISC

Additional Features