Printer-friendly version Add This

ETC Minutes 05/29/2012

Attendance

To be updated

ECHO System Status & PM Planning

  • Operations and PT Reverb - Performance issues are under investigation by development team
  • Blocked port ordering issues* have been investigated and are mostly fixed. ESI (port 22500) in LPDAAC is still in work
  • *Time synch issues among machines in PT were causing orders to be stuck in the SUBMITTING state
  • ECS providers to decide on common port for ordering/service interfaces
  • 5/16 - Operations PM window was extended to allow for full syncing of Elastic Search nodes
  • 5/16 - Operations EIAT resumption was delayed due to documentation issues.
  • 5/24 - Unplanned outage in Operations, related to timeouts being detected by our high availability proxy system
  • 5/24 - Partner Test PM window was extended due to database virtual machine issues
  • OPs Ingest Several Pauses in Operations: GSFCS4PA, LPDAAC_ECS, USGS_EROS, GHRC, LAADS
  • Partner Test EIAT is not up at the moment. Will be investigating this issue.
  • Moving to a consolidated support tool over the next month or so, used by other Earthdata teams
  • Reverb moving to reverb.earthdata.nasa.gov in the upcoming weeks

 

Additional Information (5/30/2012)

We have been busy on our side investigating errors and slowness in both the operations and partner test environments. A flurry of NCRs have been opened and are scheduled for work during the upcoming 10.49 sprint. 

In addition, a possible source of some performance issues has been discovered in the operations environment. We have discovered as part of our normal scanning and monitoring processes we can negatively impact search performance. We are currently able to manually correct this problem and are investigating a long term solution.

Recent Reverb Metrics

 

Upcoming Release 10.47

  • NCR 11010708 Ordering multiple products from the same input granule, in test now, check your provider policy in PUMP, working with Abdul Khan on ECS side
  • Orbital searching algorithm fixes
  • NCR 110111946 Select Granules page needs adjustability by users (long filenames truncated in search results)
  • NCR 11012095 Large Granule Orders Slow
  • NCR 11012411 Text needed on the log in screen to let users know they are logging in with, or creating, a EOSDIS URS ID*
  • Backported special characters ingest work

 

In Work 10.48

  • Finishing up this week
  • Backported special characters work to 10.47 (pushed to PT last week)
  • Investigation of Reverb slowness and mysterious error reports (NCR 11012417, 11012434, 11012476, 11012477)
  • Granule "Show Metadata" fix
  • Housekeeping for increased stability

 

New ETC Topic: User Changes

  • Proposed change to user schema, OpsCon to be posted by COB today, link will be included as part of ETC minutes
  • Moving the Country field out of the Address element to fix compatibility issues with URS

 

 

ECHO User Address Changes Ops Concept

The OpsConcept can be found at the following link:
 
 

 

 

Upcoming Events

  • PM window tomorrow from 8am - 12pm: all environments could be affected
  • Operations 10.47 Update - 5/30/2012 TBD
  • Testbed 10.48 Update - 5/30/2012 6/6/2012
  • Other work migrating power systems, no expected outages, but we are playing it safe for Partner Test, Reverb-beta, Testbed

 

Open Floor