Hot News Archive

1/12/2009 10:07:59 PM - NIH Enterprise Directory (NED) Access Issues
Description:   Event Details: Currently, the NIH Enterprise Directory (NED) is unavailable to some customers. There is no current estimated time to resolution. Further information will be provided as it is made available. Impacted Service/Application: NIH Enterprise Directory (NED) Impacted Users: Customers attempting to use NED Expected Resolution Time: Unknown at this time. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Access to the NIH Enterprise Directory (NED) has been restored.
1/12/2009 12:13:45 PM - Power Outage at 5635 Fishers Lane
Description:   There is currently a Power Failure at 5635 Fishers Lane, in Suite 900. Approximately 40 telephones lines have been affected. The known IC affected at the moment is NHLBI. Pepco has been contacted, and a Verizon technician has been dispatched to fully assess the situation. Further updates will be provided once they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The power outage at 5635 Fishers Lane has been resolved. All affected telephone lines have been restored to service.
1/12/2009 10:34:32 AM - Building 12 Data Access Switch Issues
Description:   At 9:49 AM we experienced a momentary loss of connectivity to a Building 12 Datacenter Access switch of which connects to several EMIB servers. Connection has been restored, and NIHnet engineers are investigating the root cause. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Initial/final Hot News has been sent.
1/7/2009 1:27:07 PM - Mailbox Server Reboot
Description:   Mailbox server NIHCESMLBX10 will be rebooted this evening at 10 PM to resolve a memory depletion issue. Service should be restored within 15 minutes of the reboot. Impacted Service/Application: NIHCESMLBX10 Impacted Users: NCI Customers Expected Resolution Time: 15 minutes Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  NIHCESMLBX10 reboot was completed successfully. All services have now been restored.
1/5/2009 7:35:38 AM - ITAS Connectivity Issues
Description:   Currently the ITAS system is experiencing connectivity issues and may have been unavailable to users. Engineers are aware of this issue and are working on a resolution. Further updates will be provided once they are made available. If you have any questions, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  ITAS is now available. Engineers may need to bring down the system for further maintenance if they find anything additional during the investigation
1/5/2009 7:02:43 AM - ITAS Connectivity Issues
Description:   At 4:08 AM ITAS was experiencing connectivity issues and may have been unavailable to users. Engineers are aware of this issue and are working on a resolution. As of 6:50 AM ITAS has been brought to service and is available to users. If you have any questions, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Initial and finale notice has been sent. Issue has been resolved.
1/2/2009 10:55:05 AM - Flooding Reported in CRC D-Wing
Description:   At this time, there has been flooding reported to CIT in the CRC D-Wing, 6 and 7th floors. People in the areas affected have been relocated. Technicians have been dispatched to assess the closet and phone status of the areas. Further update will be provided once they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The initial repairs and walk-thru have been completed and wireless service does not seem have been affected. Structural repairs will continue within the building until the spaces affected have been reoccupied. If you have any questions, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
12/30/2008 7:29:41 AM - Upgrading Formulas for NAppMan
Description:   The NAppMan team is upgrading the third-party package that provides much of NAppMan's functionality. The upgrade is to the Formula product. The company that developed Formula is changing its name to its name-sake, Managed Objects. The new version is Managed Objects 4.0. NAppMann will be moving through 4.0 to an even newer release, 4.5, but the engineers must perform a database upgrade in 4.0 to allow the 4.5 upgrade. The upgrade to 4.5 will occur on 1/13/09. Impacted Service/Application: NAppMan (NIH Application Manager) Impacted Users: During this time users will be unable to access NAppMan. Expected Resolution Time: 12:00 PM (noon) Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  NAppMan is back up and running. The upgrade was successful.
12/29/2008 1:57:03 PM - EMIB BOSB3 SAN Hardware Replacement Maintenance
Description:   On Monday, 29 December from 6:00 PM to 6:30 PM, EMIB engineers will be performing a hardware replacement maintenance to address a bad drive on the BOSB3 SAN. No interruption in service or connectivity is expected to affect the servers located on the SAN. The servers and ICs that use the affected SAN are: -- NIHCESMLBX3 - NIDCR, OD, CIT, NINR, NCRR, CSR and NCMHD -- NIHCESMLBX5 - NIAID -- NIHCESMLBX6 - NHLBI and NIEHS -- NIHCESMLBX16 - NIAID, NCI, OD, NHLBI and NIEHS -- NIHCESPUB5 - Public Folder Connectivity could be affected -- NIHCESSMTP3 - None -- NIHHRSAMLBX - HRSA -- NIHHT03 - None Engineers will provide an update at the completion of the maintenance window. If you have any questions, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The work to the EMIB SAN BOSB3 has been completed.
12/24/2008 1:21:29 PM - NIH Help Desk Holiday Hours on Friday, 26 December
Description:   Due to the Federal holiday on Friday, December 26, the NIH Help Desk will be open from 8:30 AM to 5:00 PM per normal holiday hours. The NIH Help Desk will resume normal business hours on Monday, December 29th. If you have any questions, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  This is an initial and final notification.
12/20/2008 8:23:09 PM - Intermittent Power Outages in Fernwood Building
Description:   Fernwood Building is currently experiencing intermittent power outages. Property Management has been notified and they are working to resolve this issue. Further updates will be provided as they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  No further power outages have been experienced. Power is fully restored.
12/20/2008 11:12:40 AM - Operator Services Phone Outage
Description:   Currently the Operator Services number of 301-496-4000 is experiencing technical difficulties. Technicians are onsite working towards a resolution. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  As of 11:15 AM the Operator Services number of 301-496-4000 has been restored to service. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
12/20/2008 9:47:42 AM - Extended Outage of SharePoint Services
Description:   As of December 20, 2008 at 9:00 AM, the SharePoint maintenance has been completed. However, there have been some technical problems on the SharePoint Backend configuration which will impact Search, Excel Calculation Service and Infopath features. Engineers are currently working with Microsoft on a resolution. Updates will be provided once they are made available. Affected Services: SharePoint Search, Excel Calculation Service and Infopath Service. All other SharePoint features are intact. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  All services of the MOSS/SharePoint service are functioning correctly. The SharePoint service is re-indexing the SharePoint site and should be fully searchable at 9:30 pm.
12/20/2008 6:51:25 AM - NBS is Unavailable
Description:   As of 6:40 AM, the NBS system is unavailable. NIH users who attempt to access the system are receiving a "page cannot be displayed" error. CIT engineers have been notified of this issue, but there is currently no estimated time of resolution. Further updates will be provided as they become available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The NBS system has been restored to service.
12/19/2008 4:08:32 PM - Mail Delivery Problems for ORS.OD.NIH.GOV Domain
Description:   Several individuals have reported that email messages to NIH addresses ending in @ors.nih.gov are getting rejected at the sender's end. Senders will likely receive an NDR (Non-Delivery Receipts), generally stating that their server was not able to find a mail handler for the ors.od.nih.gov domain. Engineers are aware of this issue and have already taken steps to correct the issue. However, the problem may persist as the changes are propagated to other domains. Further updates will be provided as they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  This was an initial and final resolution.
12/18/2008 2:56:26 PM - Emergency Server Patching
Description:   EMIB will be patching the following servers at 10 p.m. tonight, December 18th for emergency security patch. MS08-078 (Critical): Security Update for Internet Explorer (960714) NIHADM3, NIHADM4, NIHADMRPT, NIHADMWEB, NIHADMWEB2 NIHPSS2 NIHEAS, NIHEAS2, NIHEASNDX Effected services include, Email archive, iforgotmypassword.nih.gov, password.nih.gov, Active Roles Server ADM is a redundant service so outage will be minimal. All other services will be unavailable for a maximum of 15 minutes. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Patching of all the servers have been completed successfully.
12/16/2008 2:11:35 PM - Emergency NED Restart - 2:15 p
Description:   It will be necessary to restart NED (https://ned.nih.gov/ned) at 2:15 p.m. Please complete or save any tasks you are currently working on and log off the system. Expected downtime is 20 minutes and we will notify you when the system is back online. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The emergency restart has been performed and NED (https://ned.nih.gov/ned) is back online.
12/15/2008 11:56:48 AM - NIH Exchange Server 4 Latency Issues
Description:   Currently, users with e-mail accounts on MLBX4 are experiencing issues accessing their e-mail through desktop clients and through E-mail Web Access due to a large e-mail queue. The following ICs may experience slow or inaccessibility of their mail accounts: FIC NCI NIAAA NIAID NIDA OD Engineers are aware of the issue and are monitoring the situation closely. Impacted Service/Application: NIH Exchange Mailbox 4 Impacted Users: FIC, NCI, NIAAA, NIAID, NIDA, and OD users with e-mail accounts on Mailbox 4 Expected Resolution Time: Unknown at this time. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  As of December 15, 2008 at 1:14 AM the service for Exchange Server 4 has been restored. Engineers continue to monitor the situation. If you have any questions, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
12/15/2008 10:13:26 AM - NED is Unavailable
Description:   At this time the NED (https://ned.nih.gov/ned) may be inaccessible to some users. Engineers are aware of this issue and are working towards a resolution. Additional information will be provided as it becomes available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  As of December 15, 2008 at 10:48 AM the NED system was brought back to service. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
12/12/2008 10:28:41 AM - NED is Unavailable
Description:   At this time the NED (https://ned.nih.gov/ned) is unavailable. Engineers are aware of this issue and are working towards a resolution. Additional information will be provided as it becomes available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  As of December 12, 2008 at 11:00 AM the NED service (https://ned.nih.gov/ned) was brought back online. If you have any questions, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov.
12/11/2008 6:35:45 PM - BBSM Server Down
Description:   Currently, the BBSM Server is down. All wireless guests users will not be able to login through the NIHnet wireless network. Further updates will be provided as they are made available, there is no estimated time for a resolution. Impacted Users: All NIH wireless guests users Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.nit.nih.gov
Resolution:  As of 7:22 PM the BBSM server has returned to normal service. NIH guests users can now login through the NIHnet wireless network.
12/11/2008 1:50:11 PM - Remedy Services are Currently Unavailable
Description:   Currently, the Remedy service is unavailable. Web-submit tickets created through http://ithelpdesk.nih.gov/support are not able to process properly at this time. Engineers will be rolling back changes made to the Remedy services during last night’s weekly scheduled maintenance window to correct the issues. Further updates will be provided as they are made available. Impacted Service/Application: Remedy services Ticket Web-submit (http://ithelpdesk.nih.gov/support) Impacted Users: Customers who use the Remedy ticketing system. Customers attempting to create tickets via web-submit. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.nit.nih.gov.
Resolution:  The Remedy service is now available. The web-submit ticket form located at http://ithelpdesk.nih.gov/support is now processing tickets properly. If you continue to have problems with the Remedy service, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
12/9/2008 1:09:38 PM - NIH Integrated Services Center F5 Upgrade
Description:   On Wednesday December 10th, from 8 p.m. to 12 a.m. the NIH Integrated Services Center will perform a system upgrade in its Sterling location. The ISC will upgrade their F5 devices to ensure compliance with FIPS 140 mandates. On Sunday December 14th from 8 a.m. to 10 a.m., this upgrade will also be implemented in Building 12. In addition, the ISC will take the opportunity to begin phasing in the processes for the HSPD-12 user authentication and identity system. Impact: There is no expected impact to any ISC services, as a result of our existing redundant servers. ISC CALENDAR OF EVENTS To view this and other maintenance activities planned by the ISC, we encourage users to visit the ISC Calendar on the Portal, which can be located using the following links: https://my.nih.gov/portal/server.pt?open=512&objID=1055&PageID=1655925&cached=true&mode=2&userID=1931593 (NIH Login required.). Should you have any questions or concerns regarding this maintenance, please send your inquiries to NIH ISC Support.
Resolution:  The NIH Integrated Services Center F5 Upgrade maintenance has been successfully completed.
12/9/2008 7:52:36 AM - NBS Oracle Intermittent Access Issues
Description:   The NIH Help Desk has determined that currently there are intermittent access issues with NBS Oracle. NBS Management has been notified. Further information will be provided as we hear. Impacted Service/Application: NBS Oracle Impacted Users: NBS Oracle Users Expected Resolution Time: No estimated resolution at this time. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The intermittent access regarding NBS Oracle has been resolved.
12/8/2008 3:42:40 PM - Emergency NED Restart
Description:   NED will be restarted in approximately 3:40 PM. Users should save any tasks they're currently working on and log off the system. Expected downtime is 20 minutes. Update will be provided once the NED is brought back to service. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  As of December 8, 2008 at 4:08 PM the NED service was brought back online. If you have any questions, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
12/8/2008 2:16:06 PM - NIHnet Building 45 Opdiv DMZ Router Hardware Refresh Maintenance
Description:   On Saturday, December 13, 2008 at 10:00 AM - 6:00 PM EST, NIHnet engineers will perform a hardware refresh of the Building 45 OpDiv DMZ Router. Impact: Please note that the NIHnet Building 45 OpDiv DMZ Router Hardware Technology Refresh Maintenance has been assessed with a Possible Service Affecting-Low (PSA-Low) impact. Specifically, the OpDiv traffic that is currently traversing the Building 12 OpDiv path will be shifted to the Building 45 OpDiv path during this maintenance to verify operation of the newly upgraded Building 45 Opdiv router. Upon completion of this maintenance activity, traffic will remain on the Building 45 OpDiv path for continued evaluation and monitoring of new hardware. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The change management request "NIHnet Building 45 Opdiv DMZ Router Hardware Technology Refresh Maintenance" has successfully been completed.
12/5/2008 2:31:51 PM - Reboot of NIHCESMLBX2
Description:   On Saturday December 6th, 2008 at 9:00 PM, there will be a reboot of the NIHCESMLBX2 server. An authentication method issue with some mobile devices authenticating to the server has been found. This seems to have occurred during the last patching window and requires a reset to complete the fix. Affected Users: This will affect NINDS & NIMH users with mailboxes on NIHCESMLBX2. Mailboxes will be unavailable during the downtime. Expected time of Resolution: Downtime is expected to last approximately 10 minutes. Updates will be provided once they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  NIHCESMLBX2 server reboot has been completed successfully.
12/5/2008 1:59:48 PM - NIH Web Meeting Maintenance
Description:   NIH Web Meeting (aka "Connect", formerly "Breeze") services will be restarted in succession starting Saturday, December 6th at 6:30am. There is no expected downtime and all connections should fail over to redundant servers. Work is expected to complete by 7:00am. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The NIH Web Meeting maintenance has been completed and all services are running.
12/5/2008 10:55:10 AM - NED System Inaccessibility
Description:   The NED system is currently unavailable. Engineers are aware of this issue and are working towards a resolution. Updates will be provided once they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  As of December 5, 2008 at 11:26 AM the NED System Inaccessibility issues were successfully resolved. If you have any questions, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
12/4/2008 12:07:55 PM - Issues Communicating with AOL Contacts using OCS
Description:   Currently, users of the Office Communicator System that are enabled for public IM connectivity are having issues communicating with AOL contacts using Office Communicator. Engineers are aware of the situation and believe the problem to originate on AOL's gateway servers. They continue to monitor the situation and will provide updates as they are made available Impacted Service/Application: OCS to AIM communication Impacted Users: OCS users attempt to communicate with AOL contacts. Expected Resolution Time: Unknown at this time. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  At 12:18 PM, service between OCS and AOL Instant Messenger was restored. OCS users should now be able to communicate normally with their AOL contacts.
12/4/2008 11:33:34 AM - NED is currently unavailable
Description:   Currently there is an availabilty issue with NED and engineers are currently investigating. More information will be provided as it becomes available. We apologize for the inconvenience. Impacted Service/Application: NIH Enterprise Directory Impacted Users: Any user attempting to use NED Expected Resolution Time: No estimated end time. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  NED issue has been resolved, all services have been restored.
12/3/2008 12:33:48 PM - Building 10 Network Connectivity Issues
Description:   Currently, Building 10 is experiencing network connectivity issues. NIHnet Engineers are aware of this issue and are working towards a resolution. Further updates will be provided as they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Building 10 connectivity issues has been resolved. All network services have been restored
12/3/2008 10:56:03 AM - ITAS PR 17.0 Release
Description:   ITAS is scheduled to release a new version, PR 17.0, to production on Tuesday, December 9, 2008, from 6:00PM - 8:00PM. During the deployment of this release to production, ITAS will not be available. We apologize for any inconvenience this may cause you. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The maintenance was successfully completed within requested time frame.
12/2/2008 10:54:57 AM - NCI Servere Latency
Description:   NCI is receiving severe latency issues. Engineers are working to resolve this issue. Impacted Service/Application: MLBX10 and MLBX11 Impacted Users: NCI Expected Resolution Time: No estimated resolution at this time. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Latency problems experienced by NCI customers when accessing email have been resolved.
12/2/2008 10:17:03 AM - VPN Connectivity Issue
Description:   There was a network interruption that may have caused VPN users to lose connection to NIHnet.
Resolution:  Users will need to reboot to regain connection to VPN.
11/23/2008 1:19:27 PM - eRA Common Webservice
Description:   Event Details/Description: Currently users are having connectivity issues with eRA Commons endpoints which are Service Affecting. Engineers are currently working towards a resolution. There is no estimated resolution time. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  As of November 24, 2008 the eRA Common Webservice issue was successfully resolved. If you have any questions, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
11/23/2008 1:16:49 AM - SILK and Web Sponsor Unavailable
Description:   As of 12:55 AM, the SILK and Web Sponsor websites, (silk.nih.gov and websponsor.cit.nih.gov) are unavailable to NIH users. Users are receiving "Server Error 500" warnings when trying to access the sites. CIT engineers have been notified of the issue, but there is currently no estimated time of resolution. Further updates will be provided as they become available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The Silk and Web Sponsor websites have returned to normal service and are available to NIH users.
11/20/2008 6:54:08 AM - Sterling Data Center Network Event
Description:   Beginning at 6:00 AM, the Qwest Sterling Data Center has been experiencing a network event that may have impacted Hosting Services. Some NIH customers may experience latency while trying to access applications hosted in the Sterling facility. Further updates will be sent as they become available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  At 9:50 AM on 20 November, Qwest Hosting Services replaced the affected card in the Qwest owned and maintained Center Router 2. During the emergency maintenance, customers may have experienced latency to assets hosted in the Sterling Data Center. Latency issues should be resolved at this time. If you are still experiencing issues with assets hosted in the Sterling Data Center, please contact the NIH Help Desk at 301-496-4357 or visit us online at http://ithelpdesk.nih.gov.
11/19/2008 10:08:56 AM - NBS Oracle Intermittent Login Problems
Description:   It seems that NBS Oracle is having intermittent problems allowing users to log in. Users that are already in the system will not have problems, but users that are trying to log in will. The NBS Management team is currently working on the issue. There currently is no estimated time for resolution. Further updates will be provided once they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The NBS Oracle system was restored to service at 11:15 AM.
11/17/2008 7:46:15 AM - Problem Accessing Data Sets From Wylbur
Description:   Users are currently unable to access data sets from Wylbur. Engineers are aware of this issue, and are working on a resolution. Tentatively engineers are anticipating the issue may take until Tuesday until it's completely resolved. Please use EDSUTIL program as an alternative until the issue has been resolved. Further updates will be provided once they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  As of November 19, 2008 at 10:00 AM the issue accessing data sets from Wylbur has been successfully resolved. If you have any questions, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
11/12/2008 5:11:54 PM - Building 10 Emergency Generator Systems Test
Description:   Description: On Saturday, November 15, 2008 at 7:30 AM - 10:00 AM EST, ORF facility engineers will conduct the monthly Emergency Generator Systems Test in Building 10 and the ACRF facilities. Additionally, ORF facility engineers will conduct the monthly Emergency Generator Systems Test in the Hatfield Clinical Research Center from 10:30 AM to 12:30 PM. Impact: The Building 10 Emergency Generator Systems Test Maintenance activity is not expected to impact the Building 10, ACRF, or Clinical Research Center facilities' NIHnet data network infrastructure equipment or NIHnet connectivity outside of Building 10 because all Building 10 Core, Distribution, and Access data network devices are supported by sufficient UPS and/or generator back-up power. However, a small percentage of individual IC networks and NIHnet Wireless Access Units (APs) will be impacted at various locations within Building 10 and ACRF facilities due to the use of commercial power as a sole power source. In addition, CIT / DNST Personnel will be onsite during and after the completion of the Building 10 Emergency Generator Systems Test Maintenance activity to ensure the complete restoration of voice and NIHnet data network infrastructure services. If you have any questions Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or visit us online at http://ithelpdesk.nih.gov.
Resolution:  As of 11:32 AM the Emergency Generator Systems Test in the Hatfield Clinical Research Center has been successfully completed. If you have any questions, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
11/12/2008 10:35:49 AM - Issue with calling T-Mobile Cellular Devices
Description:   There is currently a problem calling T-Mobile cellular devices from NIH land lines. When the call is initiated from an NIH telephone, the T-Mobile customer cannot hear the NIH caller. Calls initiated from T-Mobile cellular devices are working fine. Engineers are aware of this issue and are working towards a resolution. Updates will be provided once they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Repairs between T-Mobile and Verizon circuits have been completed. T-Mobile cell phone service has been restored and T-mobile subscriber can now receive calls originating from NIH numbers.
11/8/2008 7:49:38 AM - NBS is Unavailable Via NIH Portal
Description:   The NBS applications are currently inaccessible from the NIH Portal. NIH users are receiving error messages when attempting to access NBS Communities from the NIH Portal window. CIT engineers have been notified of the issue and are working to resolve it as soon as possible. There is currently no estimated time of resolution. Further updates will be sent as they become available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Issues with NBS applications being inaccessible from NIH Portal have been resolved.
11/6/2008 3:45:56 PM - TESTING!!! Network Outage in Fernwood
Description:   TESTING!!! There is currently a network outage in the Fernwood building. Engineers are aware of the issue and are working to resolve the problem as soon as possible. Further updates will be provided
Resolution:  TEST ONLY -- Please disregard The Fernwood network has been restored to service.
11/5/2008 11:21:38 PM - Automatic Deprovisioning of AD Accounts
Description:   Wednesday, November 12, EMIB will be implementing the automatic deprovisioning of AD accounts marked as Secondary. The rule for Secondary account creation is the account must map to an active NED ID. We will be syncing with NED and any active AD Secondary accounts found with inactive NED records will be disabled and moved to _Inactive. Any active AD Secondary accounts found without NED IDs will also be disabled and moved to _Inactive. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Automatic deprovisioning of AD accounts was successfully completed.
11/5/2008 2:48:10 PM - NIH Login Encryption Key Rollout
Description:   On Wednesday, Nov 5th at 6:15 PM, NIH Login technicians may need to roll the NIH Login encryption keys to resolve an on-going outage impacting NBS. This plan will be carried out only if the technicians are unable to restore service using less invasive measures. However if the technicians do roll keys, please monitor your applications as in the past this has required some restarts for some applications. Further updates will be provided as they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The encryption key rollout was completed and the issues with the NBS system were resolved.
11/5/2008 2:08:59 PM - NIH ADM Server Maintenance
Description:   Starting at 02:00 AM on Thursday November 6th, 2008, NIHADM3 will be unavailable until 06:30 AM. ITPro's that use the website https://adm.nih.gov will still be able to connect and operate normally, although, they may have to reconnect to the service at 02:00. For those that are using the ADM console and are connecting directly to NIHADM3, please reconnect to NIHADM4. If necessary the steps to reconnect can be found at the following URL: https://isdp.nih.gov/openaccess/admmmc/Installation_for_the_ADM__6.0.4_console.doc Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The scheduled maintenance of NIHADM3 has been successfully completed.
11/5/2008 7:13:27 AM - NBS Accessibility issues
Description:   The NBS system is currently experiencing some technical difficulties. Engineers are aware of this issue and are working towards a resolution. Impacted Service/Application: NBS Oracle, PRISM or GELCO Further updates will be provided as they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The NBS system was restored to service as of 7:15 PM on Wednesday, November 5.
11/4/2008 10:10:26 AM - NIH Portal Maintenance on Friday, 11/7/2008
Description:   On Friday, November 7, 2008 from 7:00 PM to 8:00 PM, NIH Portal administrators will be conducting routine system maintenance. During this maintenance window, there is a possibility that the NIH Portal will become unavailable. Any interruptions in service are expected to be brief. Please plan your access to the NIH Portal and any hosted applications or files accordingly. An update will be provided at the conclusion of the maintenance window. Impacted Service/Application: NIH Portal Impacted Users: Any users attempting to use the NIH Portal during the maintenance window. Expected Resolution Time: 11/7/2008, 8:00 PM. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  As of November 7, 2008 at 7:54 PM, the testing was successfully completed and NIH portal is functioning as normal.
11/3/2008 9:24:52 AM - Titan System Inaccessible
Description:   We have lost all network connectivity to Titan. This will effect connectivity to all applications running on Titan such as the ADB, DB2, and Shadow. Engineers are currently working to determine the cause and get it resolved as soon as possible. Further updates will be provided as they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  As of 9:27 AM the connectivity issues in which Titan was experiencing has been resolved. If you have any questions please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
10/31/2008 1:14:14 PM - NSC Building Water Shut-down
Description:   Water will be shut-off in the NSC building, Sunday night at 9:00pm. This will affect the NIDA Server Room A/C unit and require that the NIDA Network be shutdown due to the absence of proper cooling. So Sunday night at 8:00pm the NIDA Network will be shutdown and then returned to normal operations at 6:00am Monday morning. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The NIDA servers have been powered on and are now fully operational.
10/30/2008 9:42:22 AM - NIHCESMLBX15 Memory Problems
Description:   NIH engineers are currently monitoring a developing situation on the NIHCESMLBX15 Exchange server. The Exchange server is currently experiencing memory issues. Engineers are attempting corrective actions to restore normal operating conditions on the server. Users who are using POP and IMAP clients will be unable to access their e-mail accounts while the actions are being undertaken. There is a high possibility that engineers will need to reboot the server to restore full functionality and service. Further updates will be provided as they are made available. Impacted Service/Application: NIHCESMLBX15 Impacted Users: NLM users and other Exchange accounts that are housed on MLBX15. Expected Resolution Time: Unknown at this time. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  At 10:33 AM, NIH engineers have successfully rebooted the NIHCESMLBX15 Exchange server and have restored all affected services for users. Users will now be able to access their e-mail accounts hosted on the Exchange server. If users are still having troubles accessing their e-mail accounts, please contact the NIH Help Desk for assistance.
10/30/2008 8:15:06 AM - Remedy Server Patching, Saturday November 8, 8:00 PM
Description:   On Saturday, November 8th, the Remedy team will be patching the Remedy production server. At 8:00 PM, the server will be taken offline for approximately 3 hours to apply the patches. A follow-up Hot News will be sent once patching has been completed. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  As of November 8, 2008 at 9:32 PM the Remedy team has successfully completed the patching to the Remedy server. If you have any questions, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
10/29/2008 1:04:15 PM - SecureEmail.nih.gov is Currently Experiencing Issues
Description:   The SecureEmail.nih.gov website is currently experiencing an issue sending notifications about new deliveries. The notices are being queued. CIT engineers are working to get this problem resolved as soon as possible. Packages are still able to be created and opened via the website. This affects all users of that use this service. Further updates will be provided once they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Issues with SecureEmail.nih.gov have been resolved.
10/28/2008 10:54:10 PM - NIH Login Implementation In NAppMan
Description:   On Wednesday, October 29, 2008, NAppMan will be unavailable between the hours of 7AM to 10 AM due to several changes being made to the NAppMan environment. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  As of October 29, 2008 at 11:10 AM the NIH Login Implementation in NAppMan maintenance was successfully completed. If you have any questions please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
10/27/2008 10:43:25 AM - ITAS Scheduled Maintenance
Description:   ITAS is scheduled for web agent upgrade, on Tuesday, Oct 28 from 6:00PM - 7:00PM. The system will not be available during that time. We apologize for any inconvenience this may cause you. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  ITAS Maintenance was completed successfully.
10/25/2008 3:45:24 PM - Sharepoint Service Emergency Maintenance
Description:   Currently, engineers are working to resolve an issue with the NIH SharePoint Services. To resolve the problems there is a high possibility that the SharePoint Services will need to be rebooted, causing users to lose connection to any SharePoint sites or services. Further updates will be provided as they are made available. Impacted Service/Application: SharePoint services Impacted Users: Any users attempting to use an NIH SharePoint site. Expected Resolution Time: Unknown at this time. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Engineers have completed their work on the NIH SharePoint Service. No reboot was necessary and all SharePoint sites are available for access.
10/24/2008 10:38:27 PM - Performance Alerts for NIH Login, NEES, ITAS, and NBS
Description:   Due to tonight's scheduled security patching, users of the following CIT services may experience degraded performance: ITAS, NBS, NIH Portal, NIH Login, nVision, and NIDB. These systems are currently accessible, but users may experience slow response times or unusual behavior. CIT engineers are aware of the issues and are working to resolve them. There is no estimated time of resolution. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The alerts for the affected applications have cleared and all applications appear to be functioning normally.
10/24/2008 10:24:40 PM - Helix System Unavailable
Description:   The Helix System is currently unavailable. NIH users are unable to login to the Helix at the present time. CIT technical staff are working with the vendor to diagnose the problem, but there is currently no estimated time of resolution. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  As of midnight, Helix has been rebooted and is available to users.
10/24/2008 9:55:18 PM - ITAS emergency maintenance WINDOWS - critical security patches
Description:   ITAS is scheduled for critical security patches on Saturday, Oct 25, from 8:00AM - 9:00AM. The system will not be available during that time. We apologize for any inconvenience this may cause you. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  As of October 25, 2008 at 8:42 AM the ITAS emergency maintenance has been successfully completed. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
10/24/2008 9:59:48 AM - Password Server Emergency Security Patching
Description:   To implement an emergency security patch, engineers will be patching the iforgormypassword.nih.gov and password.nih.gov servers today, October 24th between 11:45 AM and 12:00 Noon. These services will NOT be available during this maintenance window for approximately 5-10 minutes. Impacted Service/Application: iforgotmypassword.nih.gov password.nih.gov Impacted Users: Users attempting to access these websites during the maintenance window. Expected Resolution Time: 12:00 Noon Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The iforgotmypassword.nih.gov and password.nih.gov servers have been patched and are now available for all users.
10/23/2008 10:12:06 AM - Nihadm4 Reboot
Description:   Nihadm4 will be rebooted at 10:15 to deploy a patch. This will not affect customers using https://adm.nih.gov. Anyone connecting directly to nihadm4.nih.gov with the ADM console will just need to reconnect. The outage will be no longer than 5 minutes. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The Reboot was successful 5 minutes after it started.
10/22/2008 3:13:23 PM - Biowulf System Issue
Description:   The Biowulf system is currently experiencing a problem that makes the system inaccessible to users. Members of the Helix systems group are investigating the problem and hope to have it resolved as soon as possible. Updates will be provided once they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  As of October 22, 2008 at 3:32 PM the problem which Biowulf was experiencing has been successfully resolved. The Biowulf head node is now accessible to users. If you have any questions, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
10/22/2008 11:31:13 AM - NIHEXTERNALDC Issues
Description:   A domain controller in the nihnext.nih.gov external forest has suffered a catastrophic hardware failure and will need to be rebuilt. There is no current time of resolution. Updates will be provided once they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  As of October 29, 2008 at 8:00 AM, the NIHEXTERNALDC Domain Controller was successfully rebuilt and brought back to production. If you have any questions please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
10/20/2008 4:40:34 PM - Fernwood Power Outage
Description:   Parts of the Fernwood Buliding is currently experiencing a power outage; it appears that only lighting is currently affected. Engineers are currently working towards a resolution. Further updates will be provided once they are available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Power has been completely restored.
10/19/2008 10:55:29 PM - Telephone Outage CRC Building 4th and 9th Floors
Description:   Currently there is a telephone outage affecting the CRC building on floors 4 and 9. Technicians have been dispatched and are working towards a resolution. Currently there is no estimated time of resolution. Updates will be posted as soon as they are available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  As of October 20, 2008 at 9:25 AM the telephone outage on the 4th and 9th floors of the CRC building has been successfully restored to service. If you have any questions, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
10/17/2008 4:31:09 PM - Helix System Emergency Maintenance
Description:   The Helix system will be unavailable to users due to emergency system maintenance starting at between 4:30 and 5:30 this afternoon. It is anticipated that the maintenance will last up to 2 hours. Users who are either logged into the system or running jobs have been notified of the expected service interruption. Impacted Service/Application: Helix Impacted Users: Helix Users Expected Resolution Time: Between 6:30 PM and 7:30 PM Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The Helix System has been restored to service in a degraded mode.
10/15/2008 11:29:53 AM - NIHCESMLBX15 Server Reset
Description:   Exchange server NIHCESMLBX15 is exhibiting symptoms of a degrading condition and will be reset at 11:35 AM. Updates will be provided once they are made available. Impacted Service/Application: Exchange Mailbox 15 Impacted Users: NLM Expected Resolution Time: Unknown at this time. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  At 12:14 PM, NIHCESMLBX15 was restored to service and all mailboxes are now available. Engineers continue to investigate the root cause of the problem and will provide updates as they are available.
10/15/2008 4:21:50 AM - NED Service is Unavailable
Description:   The NIH Enterprise Directory website (ned.nih.gov) is currently unavailable. NIH users are unable to access the website. There is currently no estimated time of resolution. Further updates will be sent as they become available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The NED website has been restored to service.
10/11/2008 1:51:19 PM - Baltimore Network Outage
Description:   NIDA, NHGRI, NIA, and CIT/DCSS users at Bayview Hospital Center, Harbor Hospital Center, and Holibird Hospital Center in Baltimore may have experienced a temporary loss of service due to a network outage that lasted approximate 3 minutes, starting at 12:50pm. Impacted Service/Application: Network Resolution Time: Date: October 11, 2008 Time: 12:53 PM Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  At 12:53 PM the temporary network outage which NIDA, NHGRI, NIA, and CIT/DCSS users at Bayview Hospital Center, Harbor Hospital Center, and Holibird Hospital Center in Baltimore was successfully resolved.
10/10/2008 9:07:35 AM - NED Processing Delays
Description:   The NED team has determined that there is a delay processing registrations and updates. Users may experience delays seeing updates applied and actions executed. NED is currently available. Engineers are working to resolve this issue. Expected Resolution Time: Unknown, Updates will be provided as information becomes available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The issue was resolved. No root cause has been received.
10/9/2008 1:32:48 PM - NED is Unavailable
Description:   NED is currently unavailable to users. Engineers are working on a resolution. Impacted Service/Application: NED Service. Impacted Users: NIH users. Expected Resolution Time: Unknown, Updates will be provided as information becomes available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The issue with NED has been resolved. NED is now available.
10/9/2008 11:36:20 AM - NED - AD Sync Agreement Issues
Description:   The NED - AD sync agreement is blanking out some AD account last names. EMIB is currently researching why and will be fixing this as soon as possible. User account access is unaffected by this issue. Impacted Service/Application: NED Expected Resolution Time: Unknown Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  All recent AD data changes have been corrected.
10/9/2008 9:18:45 AM - Helix System Issue
Description:   The Helix system is currently experiencing a problem. Administrators are investigating the problem and hope to have it resolved as soon as possible. Impacted Service/Applications: Users may be prevented from accessing the system. Impacted Users: Helix users. Expected Resolution Time: Unknown at this time. Update will be provided as it becomes available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The Helix system is back online and fully functional.
10/8/2008 12:05:17 PM - NIH IRT Server Maintenance
Description:   The NIH IRT will be performing server maintenance this weekend, October 11 & 12. The services listed below will be unavailable starting at 8:00 AM on Saturday October 11th. All maintenance should be complete by 5:00 PM Sunday October 12th. Updates will be provided once they are made available. Impacted Service/Application: OCIO web server (ocio.nih.gov) SARA/Nessus self scanners Gideon Appscan Expected Resolution Time: Date: October 12, 2008 Time: 5:00 PM Please send any questions or concerns to the IRT at irt@mail.nih.gov.
Resolution:  Time: 8:34 PM The maintenance has been successfully completed.
10/6/2008 11:50:32 AM - NED Issue
Description:   NED is currently experiencing slowness issues. It is possible that you will be unable to access the system. Engineers are currently working on a resolution and updates will be provided as information becomes available. Impacted services: NED Expected time of resolution: Unkown at this time Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  At 1:14 PM the issue which NED was experiencing has been resolved. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
10/3/2008 5:14:12 PM - SharePoint SQL Upgrade Maintenance
Description:   The Hosting Services Branch will be upgrading our SharePoint Database infrastructure on Friday 10/3/2008 starting at 10:00PM EST. The entire upgrade process should take about three hours, during this time the SharePoint services will intermittently be unavailable from 10:00PM to 1:00AM EST. Please be advised that all users should refrain from adding content to their SharePoint site during this time. The infrastructure upgrade will significantly improve SharePoint content read/write performance and handle the increase capacity of SharePoint users. Impacted Service/Application: SharePoint Database Impacted Users: All Sharepoint users that have sites hosted at CIT/HSB. Expected Resolution Time: 1:00 AM Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The SharePoint SQL Server infrastructure upgrade has been successfully completed. Thank you for your patience.
10/3/2008 11:41:20 AM - 6701 Rockledge Telephone Outage
Description:   An outage has been reported at 6701 Rockledge DR. There are at least 48 telephone lines are without service due to water damage. There is 1 technician on site accessing the damage and working on resolving the issue. Further updates will be provided once they are made available. Impacted Service/Application: Telephone Service Impacted Users: CSR Expected Resolution Time: Unknown at this time Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  As of Friday October 3, 2008 at 3:15 PM all telephone service in the 6701 Rockledge building was restored. If you have any questions, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
10/3/2008 11:41:11 AM - Building 49 Telephone Outage
Description:   An outage has been reported at Bldg 49, floors 1-5. It is unknown how many telephone lines are without service due to water damage. There are 2 technicians on site accessing the damage and working on resolving the issue. Impacted Service/Application: Telephone Service Impacted Users: NICHD, NIMH, NINDS, NEI, NCI, NIDCR, NHGRI, ORS and DNST users in bldg 49. Expected Resolution Time: Unknown at this time Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  All telephone service has been restored to building 49.
10/1/2008 9:48:10 AM - HHS Learning Portal Issue
Description:   NIH users are currently experiencing problems in registering for Online Training like Alternative Dispute Resolution, NIH Environmental Management System Awareness Training and other online training through the HHS Learning Portal (Learning Management System). The issue has been reported to the appropriate staff and they are currently working in resolving it. Additional information will be provided once the issue has been resolved. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  NIH users can now have access to Online Training like Alternative Dispute Resolution, NIH Environmental Management System Awareness Training and other online trainings. The issue has been resolved. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
9/30/2008 9:19:08 PM - Fisher Lane & Twinbrook Pkway Experiencing Power Outages
Description:   Due to the storm 5625 & 5635 Fisher Lane is experiencing a partial power outage. The 12735 Twinbrook Parkway building has also been affected by the storm and has no power at this time. Pepco has been notified on both of these issues and is working towards restoring power to these building. Currently there is no estimated time of resolution. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Power was restored at 11:48 PM. All network devices have returned to normal service.
9/30/2008 10:44:31 AM - Emergency IPL of Titan System 9
Description:   The Titan System 9 was IPL'ed to correct issues with DB2 and SHADOW (web) services. At this time, all services have been restored. Impacted Service/Application: DB2 and SHADOW (web) services Impacted Users: Users of following areas: DWQuery Budget & Finance DWQuery Research Contracts & Grants nVision Balance of Accounts - Drill to Detail Only Budget Tracking DWAnalyze Budget & Finance - Drill to Detail Only Expected Resolution Time: 10:40 AM Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  This was the initial and final notice.
9/30/2008 9:07:37 AM - Power Failure at the Parklawn Building.
Description:   A power failure has occurred at the Parklawn Building. The C-Wing, Headhouse and parts of the North B-Wing are currently without power. All workers of the affected areas are not expected to report to work today. If you have due dates or essential work that must be completed today, contact your supervisor and make arrangements to work at home or be relocated to office space that has power. Impacted Service/Application: Internet connectivity. Impacted Users: HRSA users in Parklawn Expected Resolution Time: Additional information will be provided when it comes available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  As of 9:15 PM, power was restored to the Parklawn Building. The building will be open and fully operational on Wednesday, October 1.
9/28/2008 10:30:07 AM - NIH Fellows Account Issue
Description:   There are a number of NIH Fellows whose NED records were inadvertently deactivated and their Active Directory accounts were disabled. Engineers are working to remedy the situation. For those needing immediate assistance, please contact your Administrative Office to have the NED record re-activated, and then contact the NIH Help Desk to have the Active Directory account re-enabled. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  NIH Fellows Account Issue has been resolved.
9/26/2008 10:26:40 PM - NIHnet CIT Fernwood-6500-3NE Switch IOS/Memory Upgrade
Description:   On Friday, October 17, 2008 at 4:30am - 6:00am ET, NIHnet engineers will complete an IOS and memory upgrade of Fernwood-6500-3NE, followed by the installation of several new 10/100/1000 PoE line-card modules. Impact: Please note this NIHnet CIT Fernwood-6500-3NE Switch IOS/Memory Upgrade and 10/100/1000 PoE Module Installation Maintenance activity has been assessed with a Service Affecting (SA) impact. Specifically, 3rd floor NE Fernwood users and services connected through FERNWOOD-6500-3NE will experience a loss of connectivity during the first 30 minutes of this one hour maintenance activity. This includes Aspect management network connections. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The NIHnet CIT Fernwood-6500-3NE Switch IOS/Memory Upgrade has been successfully completed.
9/26/2008 6:45:27 AM - NEES Service Outage
Description:   The NIH NEES service is currently unavailable due to an unscheduled system outage. NIH users are currently unable to access the NEES website. CIT staff are aware of the issue, but there is currently no estimated time of resolution. Further updates will be sent as they become available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  At 8:03 AM Service to NEES was restored.
9/24/2008 3:55:29 PM - NIHnet Internet DMZ Routing Optimization Maintenance
Description:   On Saturday, September 27, 2008 at 10:30am - 1:30pm ET NIHnet engineers will perform a proactive NIHnet Internet Service routing scheme optimization maintenance activity on the Building 12 and Fernwood outside DMZ routers as a means to reduce the overall size of the Internet routing table as received (announced) by Sprint and Level(3). The NIHnet Internet DMZ Routing Optimization Maintenance activity has been given a Possible Service Affecting - Low (PSA-Low) probability impact assessment for all data network traffic traversing the NIHnet Internet DMZ. This low impact assessment was determined based on the fully redundant design of the NIHnet Internet Service such that traffic between NIH and the Internet will be systematically shifted from one DMZ path to other before router configuration changes are made to each non-active path. Additionally, when the new Internet DMZ outside router configurations are activated during the first 30 minutes of the three-hour maintenance window, there is a very low probability for disruption to NIH - Internet data network connectivity with only a minimal amount of packet transit delay expected as data traffic is shifted from one Internet DMZ path to a redundant DMZ path. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  NIHnet Internet DMZ Routing Optimization Maintenance was completed successfully with out any complications.
9/22/2008 4:35:48 PM - NIHnet OpDiv DMZ Routing Optimization and Path Failover Maint.
Description:   On Thursday, September 25, 2008 at 5:30pm - 8:30pm ET NIHnet engineers will restore the redundant path connectivity for the NIHnet OpDiv DMZ and optimize the routing scheme as a means to provide improved path failover capabilities between the Building 12 and Building 45 NIHnet OpDiv DMZ network paths. The NIHnet OpDiv DMZ Routing Optimization and Path Failover Maintenance activity has been given a Possible Service Affecting - Low (PSA-Low) probability impact assessment for all data network traffic traversing the NIHnet OpDiv DMZ. Specifically, when the data network connectivity between NIHnet and OpDiv network peers is returned to a redundant Building 12 and Building 45 dual-path DMZ architecture, there is a very low probability for disruption to NIHnet - HHSnet OpDiv data network connectivity and only a minimal amount of packet transit delay is expected as data traffic is shifted from the secondary DMZ path back to the primary OpDiv DMZ path. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  NIHnet Internet DMZ Routing Optimization Maintenance was completed successfully without any complications.
9/20/2008 1:21:49 AM - NIH Login and NEES Service Interruption
Description:   As of Saturday, September 20 at 1:00 AM, NIH Login and the NEES application are unavailable. Notifications will be sent when the applications are restored to service, but there is currently no estimated time of repair. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The NEES, NIH Login, and ITAS services are all functioning normally at this time.
9/18/2008 9:17:56 AM - Network Outage on 10th Floor, Democracy 2
Description:   Currently, users on the 10th floor of Democracy 2 are unable to access the NIHnet. Engineers are aware of the issue and are on site and working to correct the problem as soon as possible. Further updates will be provided as they are made available. Impacted Service/Application: Network access and network-dependant resources. Impacted Users: Users located on the 10th floor of Democracy 2 Expected Resolution Time: Unknown at this time. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  All affected devices have been restored and are operating normally at this time.
9/15/2008 8:44:59 AM - 5 Research Court Telephone Outage
Description:   Currently, customers in 5 Research Court are experiencing a telephone outage. Approximately 175 users have been affected. Technicians have been dispatched to correct the problem. Further updates will be provided as they are made available. Impacted Service/Application: Telephones at 5 Research Court Impacted Users: 175 phone lines for the following IC's: NCI, NIDCD, NIAMS, NHLBI, OD/ORF, OD/ORS. Expected Resolution Time: No current estimated time of resolution. If you have any questions, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  As of September 15, 2008 at 11:00 AM telephone service in 5 Research Court was successfully restored. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
9/13/2008 8:12:05 AM - Building 10B Electrical Maintenance
Description:   On Saturday, September 13 from 6:00 AM until 11:00 AM ORF will be conducting an electrical maintenance activity in the Second floor Cafe/Kitchen of Building 10B. This scheduled maintenance is currently impacting NIHnet, FACnet, and wireless connectivity for Building 10 users from the following ICs: NIAAA, OD/ORS, NEI, NIAMS, NINDS, and NIEHS. Further updates will be sent when the maintenance is completed. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  As of Saturday, September 13 2008 at 2:10 PM all devices impacted by the Building 10B Electrical Maintenance have been fully restored. If you have any questions, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
9/12/2008 12:51:35 PM - CIT's Pilot Wiki Service Outage
Description:   CIT's Wiki Service Pilot Project server (kiwi.nih.gov) experienced a hardware failure this morning. Technicians are in the process of restoring the service now. All wikis hosted on this server are unavailable. Individual wiki owners have been notified. Impacted Service/Application: Wiki Service Pilot Project server (http://kiwi.nih.gov) Impacted Users: Users accessing wiki's hosted on http://kiwi.nih.gov Expected Resolution Time: Friday, 12-Sep-2008 3:00 pm Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Time: 4:30PM CIT's Wiki Service Pilot Project server has returned to normal service and all wikis are available once again as of 4:30 pm this afternoon. Individual wiki owners have been notified regarding the status.
9/12/2008 6:28:05 AM - Executive Boulevard Intermittent Wireless Connectivity
Description:   As of 5:50 AM, wireless network devices in the 6100 Executive Boulevard building have been experiencing intermittent connectivity issues. Connectivity is currently impacted for Executive Boulevard users from NICHD and OD. Network engineers are currently investigating the issue, but there is no estimated time of resolution. Further updates will be sent as they become available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  As of September 12, 2008 at 10:04 AM the intermittent wireless connectivity issues in the 6100 Executive Boulevard building have been resolved. If you continue to have intermittent wireless connectivity issues, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
9/9/2008 4:01:41 PM - Building 13 Telecom Closet 213 Maintenance
Description:   Event Details: CIT will be performing work in the Telecom closet 213 on the 2nd floor of Building 13 during normal business hours beginning on Wednesday, 9/10/08 and ending on Tuesday, 9/16/08. Users may experience phone-related issues from anywhere between 5 and 15 minutes. If they experience any issues that extend past this period, they should report them to the ASF by dialing 611; Verizon will have prior knowledge of the event. Impacted Service/Application: Telephones Impacted Users: Users on the 2nd floor of Building 13. Expected Resolution Time: Tuesday, 9/16/2008, 4 PM. If you have any questions, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Time: 3:00PM Engineers have successfully completed work on closet 213 in Building 13 as of 3:00pm this afternoon.
9/9/2008 11:17:06 AM - Building 37 Telecommunications Outage
Description:   Currently, customers in Building 37 are experiencing a telecommunications outage as a result of a planned power outage performed by building maintenance engineers. Technicians are on site to repair the affected telecommunications lines. There is no current estimated time to repair and updates will be provided as they are made available. Impacted Service/Application: At this time it is reported that approximately 500 telephone lines have been affected. Impacted Users: Building 37 users. Expected Resolution Time: No current estimated time to repair. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Telephone service has been restored to the 150 phone lines affected on the 1st and 3rd floors of Building 37. Technicians state that the telephone lines are stable at this time. If you are still having issues with your telephone line, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
9/8/2008 5:04:53 PM - NIHnet Internet Building 12 Firewall Emergency OS Upgrade Maint.
Description:   Date: Monday, September 8th, 2008 Time: 6:00pm - 7:00pm Description: On Monday, September 08, 2008 at 6:00pm - 7:00pm ET NIHnet engineers will perform an emergency operating system upgrade maintenance activity on the NIHnet Internet DMZ Building 12 firewall to eliminate a confirmed operating system bug on the Building 12 firewall. Impact: This maintenance activity has been given a service affecting (SA) impact assessment because it will be necessary to shift NIHnet Internet traffic that is presently on the Fernwood DMZ path to the Building 12 DMZ path as means to test stability, reliability, and performance of the Building 12 DMZ firewall after the operating system upgrade has been completed. The necessary shift in NIHnet Internet traffic from the Fernwood path to the Building 12 path firewall will occur during the last 30 minutes of this one-hour maintenance window with an estimated 15 - 30 second timeframe for potential NIH Internet connectivity disruption during the failover from one path to the other redundant path. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Time: 7:01pm The maintenance was not completed and was backed out successfully. The maintenance may be rescheduled at a later date.
9/6/2008 2:25:46 PM - NIH Fire Department 2Way Radio Outage
Description:   The NIH Fire Department 2 way radio system is currently suffering service outages. The proper repair team has been dispatched to work on a resolution. Further updates will be provided once they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  As of September 8, 2008 at 12:38 PM, the problem with the T1 Circuit has been identified and repaired. At this time the LMR Two-way Radio Network for the NIH Fire Department has returned to normal operation. For further questions, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
9/5/2008 12:00:44 PM - NIHnet Internet DMZ Building 12 Firewall Isolation
Description:   The NIHnet Internet DMZ Building 12 firewall is experiencing unstable operating system performance and stability. Therefore, it was necessary for NIHnet engineers to proactively fail all NIH Internet traffic to the redundant Fernwood NIHnet Internet DMZ firewall path as a means to ensure stable and reliable connectivity and performance between NIH and the Internet. NIHnet engineers are actively performing root-cause analysis on the isolated, out-of-service Building 12 Internet DMZ firewall device. Upon completion of all trouble management activities, NIHnet engineers will plan, schedule, and execute a restoration change management activity to permanently resolve this unwanted chronic problem. If you have any questions, please contact the NIH Help Desk at 301-496-4357 or visit us online at http://ithelpdesk.nih.gov.
Resolution:  This is an initial and final notification.
9/5/2008 7:37:45 AM - NIHnet Internet DMZ Fernwood-7600-Outside GBIC Replacement
Description:   On Saturday, September 6, 2008 at 10:30am - 12:00pm ET NIHnet engineers will perform an emergency maintenance activity on the NIHnet Internet DMZ to repair a defective fiber optic interface connector on the Fernwood path DMZ outside router. This maintenance activity has been given a service affecting (SA) impact assessment because Internet traffic routed via the Fernwood Internet DMZ path will experience a 15 - 30 second connectivity disruption during the first 30 minutes of the maintenance window while the defective fiber optic interface connector is removed from the Fernwood path DMZ outside router. Upon completion of installation and testing of the new replacement fiber optic interface connector in the Fernwood path DMZ outside router, Internet traffic routed via the Fernwood Internet DMZ path will experience another 15 - 30 second connectivity disruption during last 30 minutes of the maintenance window while NIH Internet traffic is moved back onto the Fernwood Internet DMZ path. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  As of September 6, 2008 at 12:06 PM the NIHnet Internet DMZ Fernwood-7600-Outside Port-Channel GBIC Replacement Emergency Maintenance was completed successfully without complications.
9/2/2008 2:58:43 PM - NED Upgrade
Description:   A major upgrade to the NED system including deployment of a new user interface is scheduled to begin Friday, 10/3/08, at 5:00PM and is expected to be completed by Monday, 10/6/08, at 7:00 AM. During this maintenance NED AO/AT interface, NED self-service update, and AD link editor will be unavailable. The NED public search will be unavailable beginning Friday, 10/3/08 at 5:00PM and should be available by Saturday, 10/4/08 at 6:00PM. Applications, systems and NED Oracle account holders will not have access to the NED Oracle database. Constellation AD account provisioning will be unavailable. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The NED System upgrade has been completed successfully.
9/2/2008 8:32:05 AM - Outlook Web Access Log-On Problems
Description:   Outlook Web Access is currently experiencing an issue when users attempt to log-on. Engineers are aware of the issue and are working towards a resolution. Further updates will be provided as they are made available. Impacted Service/Application: Outlook Web Access Impacted Users: Users attempting to access their e-mail through mail.nih.gov. If you have any questions, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  At 8:58 AM, the Outlook Web Access log-on issues have been resolved and the OWA has been restored to service.
9/2/2008 7:54:39 AM - NIH Active Directory Management Tool Issues
Description:   Currently, the NIH Active Directory Management Tool is experiencing issues. Engineers are aware of the issue and expect full restoration within 10 - 15 minutes. Impacted Service/Application: NIH Active Directory Management (NIHADM) Expected Resolution Time: 8:10 AM If you have any questions, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  At 8:01 AM, the NIH Active Directory Management Tool was restored to service. Engineers are currently investigating the root cause of the issue.
8/29/2008 3:14:47 PM - NIH CMS Services Server Reboot
Description:   Time: 3:15pm There are currently latency issues with the NIH CMS Services, and engineers are currently working resolve to it. A server reboot will be required to resolve the issue. Updates will be provided as they are made available. If you have any questions, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The NIH CMS services server reboot has been completed, and the latency issues have been resolved. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
8/28/2008 9:55:20 AM - SSH Server Issues on TITAN
Description:   On Wednesday, August 27 at 10:00 PM, the SSH servers on TITAN experienced issues. Engineers have corrected the issues and restored service to the SSH server at 9:40 AM on Thursday, August 28. We apologize for any inconvenience this may have caused. Impacted Service/Application: SSH server on TITAN Resolution Time: 9:40 AM, August 28, 2008. If you have any questions, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  This is an initial and final notification.
8/28/2008 7:37:59 AM - Building 1 Power Issues
Description:   Currently, Building 1 is experiencing power issues which are affecting wireless network connectivity. There is a possibility that the power issues are also affecting non-CIT monitored network devices. Building maintenance engineers have been dispatched to the building to investigate and repair the power issue. There is no current estimated time of repair. Further updates will be provided as they are made available. Impacted Service/Application: Wireless Network Impacted Users: Building 1 wireless network users Expected Resolution Time: No current known estimated time to resolution If you have any questions, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  At this time, the power issues in Building 1 have been repaired and wireless networks have been restored.
8/27/2008 2:51:16 PM - NIH Email Web Access Upgrade
Description:   Friendly Reminder: As noted in the communications sent on August 12, 2008 to the NIH IT Infrastructure Contacts ALL distribution list, the NIH CES engineers will be upgrading the NIH CES OWA interface. This upgrade was performed as planned on August 22, 2008. The upgrade was rolled back due to problems encountered with a minority of email clients. CES engineers have resolved the problem and will redeploy the upgrade at 7:00 AM on Thursday the 28th. The service disruption will be less that 1 minute and requires no action from the NIH community. Users accessing OWA at 7:00 AM may need to log back in. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  NIH Email Web Access Upgrade was completed successfully.
8/27/2008 10:48:58 AM - NIH Web Collaboration Maintenance
Description:   On August 27, 2008 from 9:00 PM - 12:00 AM there will be maintenance performed on one of the NIH Web Collaboration (Connect/Breeze) servers. Service may be unavailable during this maintenance. Further updates will be provided once they are made available. Impacted Service/Application: NIH Web Collaboration (Connect/Breeze) Expected Resolution Time: Date: August 28, 2008 Time: 12:00 AM Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The scheduled maintenance on the NIH Web Collaboration (Connect/Breeze) servers has completed. Services are now available.
8/27/2008 9:04:27 AM - SiteMinder software upgrade on the Support Site Web Server
Description:   On Friday August 29, 2008 from 9:00 PM - 9:30 PM there will be a scheduled SiteMinder software upgrade maintenance. Activities: The maintenance will include a version upgrade of the siteminder product followed by a server reboot. Impacted Service/Application: NIH Helpdesk site (including Web Ticket Submit), Training Site, Remote Access, Publications, Public KB. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  SiteMinder software upgrade maintenance was completed successfully.
8/27/2008 8:28:23 AM - Internet Connectivity Issues
Description:   NIHnet Engineers have observed connectivity issues to the NIHnet. Users may experience a loss of internet connection. Engineers are aware of this issue and are working on a resolution. Updates will be provided once they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  NIHnet engineers have restored internet connectivity and stability at this time.
8/27/2008 7:16:37 AM - Problems sending messages from BlackBerry
Description:   As of August 27, 2008 at 6:20 AM, some NIH users have reported issues sending e-mail messages from their Blackberry devices. This is a known issue, and engineers are working on a resolution. Updates will be provided once they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  As of August 27, 2008 at 7:35 AM users have been able to successfully send e-mails from their BlackBerry devices. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
8/25/2008 2:35:49 PM - NIH Mailbox Server Maintenance
Description:   The power issues of the past weekend have left a number of systems in a degraded state. Mailbox servers: NIHCESMLBX2 (NIMH, NINDS) NIHCESMLBX4 (FAES, FIC, NIAAA, NIAID/VRC, NIDA, ORS) NIHCESMLBX8 (NCI, NIDA, NIDCD) Will need to be reset to allow Backups to initialize and run. This will affect the ICs noted above next to the server name. This event is scheduled for 10 PM this evening, 08/25/2008 and is not expected to exceed 15 minutes. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  All mailbox servers are back online.
8/25/2008 8:29:45 AM - Web Email Problems
Description:   CIT engineers are aware of an issue related to the toolbar in the new premium web version of Outlook (Outlook Web Access) which has occurred as a result of some maintenance performed over this past weekend. As a result of this, the maintenance has been rolled back and will be rescheduled. The rollback was as of August 25, 2008 at 8:20 AM. Updates will be provided once they are made available. Impacted Service/Application: Outlook Web Access (mail.nih.gov) Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Time: 12:52PM The rollback of the changes to OWA yesterday have resolved all of the issues being seen by users.
8/23/2008 7:07:04 AM - Building 13 Telecommunications Maintenance
Description:   During normal business hours of the week of September 2 - September 5, CIT will be conducting an extended maintenance of telecommunications equipment on the second floor of Building 13. During this window, Building 13 users may experience phone-related issues of 5-15 minute duration. Any issues extending beyond this timeframe should be reported to the ASF by dialing 611. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The telecommunications maintenance activity scheduled for Building 13 has been postponed and will be rescheduled at a later date. If you have any questions, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
8/22/2008 7:00:56 PM - Water Shut Off in Fernwood Building
Description:   Due to an unexpected plumbing problem on the basement level in the Fernwood building, the water has been shut off for the entire building. The water is expected to be turned back on by Saturday, August 23, 2008 at 12:00 noon. If you have any questions, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Water service is currently functional in the Fernwood building. Further updates will be sent if the situation changes. If the problem continues, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
8/22/2008 4:49:14 PM - NIH Email Web Access Upgrade
Description:   Friendly Reminder: As noted in the communications sent on August 12, 2008 to the NIH IT Infrastructure Contacts ALL distribution list, the NIH CES engineers will be upgrading the NIH CES OWA interface tonight, August 22, 2008 at 10:00PM. The upgrade is expected to take less than 15 minutes and requires no action from the NIH community. If you have any questions, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The NIH email web access upgrade was completed successfully by 10:15 PM.
8/22/2008 8:04:34 AM - Petri Server Reboot
Description:   The Petri Server is being rebooted to clear some issues.. The reboot should only take approxiately 5 minutes and will be restored momentarily. An update will be provided when Petri is available. Impacted Users: All CIT Staff
Resolution:  Petri has been restored to service.
8/22/2008 7:54:03 AM - Building 12 Power Interruption
Description:   Several network devices in the Building 12 Data Center are currently experiencing a service interruption. A variety of CIT services, including Remedy, ITAS, NEES, and EOS are currently impacted. CIT engineers are currently working to restore network services as soon as possible. Further updates will be provided as they become available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The faulty UPS in Building 12 was replaced and the Data Center has been returned to commercial power. All affected IT services have been restored.
8/21/2008 10:16:47 AM - ns.nih.gov Access Issues
Description:   Due to an unexpected issue, ns.nih.gov is currently unavailble. Engineers are aware of this issue and are working towards a resolution. Impacted Service/Application: NIH DNS Further updates will be provided once they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  As of August 21, 2008 at 11:05 AM the ns.nih.gov issue has been resolved. NIH DNS is a fully redundant service and was never down throughout this issue. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
8/21/2008 7:00:31 AM - NAppMan Implementation of nVision Changes
Description:   On August 21, 2008 from 10:00 AM - 11:00 AM the NAppMan team will be installing an update to accommodate recent changes in nVision architecture. NAppMan will not be available during the update. Further updates will be provided once they are made available. Impacted Service/Application: NAppMan Expected Resolution Time: Date: August 21, 2008 Time: 11:00 AM Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  NAppMan is now available with the new nVision architecture in place.
8/20/2008 4:32:25 PM - EMIB Maintenance
Description:   As part of the EMIB maintenance, NIHADM, NIHPSS2, and NIHDDNS will be patched tonight starting at 10 PM. Maintenance should be completely done by 11:15 PM. The following services will be affected by this maintenance: 1. Active Directory Management 2. Password Self Service If you have any questions, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Patches on NIHDDNS, NIHADM, and NIHPSS2 have been applied successfully; all servers are up and functional at this time.
8/19/2008 4:23:43 PM - Federal PKI Root CA Desktop Configuration Change Test
Description:   What: We are preparing to test a configuration change to CIT desktop computers. The change will be accomplished by linking the CIT\Computers OU of Active Directory to a new Group Policy object. The intended effect is to add the new Federal PKI root certificate authority to Internet Explorer. This change is needed on all desktop computers in the Federal Government to enable users to obtain and use new digital certificates. The change should have no adverse effect on the desktop computers. If the change proves successful in CIT, then we will recommend to the other ICs that they make the same change in their environments. When: August 21, 10:00 a.m. - ongoing if successful Scope: The test is limited to CIT desktop computers that are registered in the CIT\Computers OU. (This is the bulk of CIT's desktop computers.) Possible Actions: It should just take us a few minutes to examine the test computers and access the impact of the change. If we see any problems, we will unlink the GPo immediately so no other CIT computers will be affected by it. Worst case, a computer adversely affected by the change would need to be restarted (or have its policy settings refreshed by an Administrator.) If you have any questions, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The test was completed successfully. The configuration change remains in place and will soon be applied to other CIT desktop computer OUs.
8/19/2008 8:44:08 AM - NIHnet Internet DMZ Fernwood Path Monitoring Optimization Maint.
Description:   Date: Monday, August 25th, 2008 Time: 6:00pm - 8:00pm Description: On Monday, August 25th, 2008 at 6:00pm - 8:00pm EDT On Monday, August 25, 2008 NIHnet network engineers will performance a NIHnet Internet DMZ traffic utilization monitoring maintenance activity on the redundant Fernwood Internet DMZ --- Internet path. Impact: The NIHnet Internet DMZ Fernwood Path Traffic Monitoring Optimization Maintenance activity has been given a Possible Service Affecting-Low (PSA-Low) impact assessment for data network traffic traversing the NIHnet Internet DMZ infrastructure. Specifically, when the data network traffic monitoring optimization changes are implemented on the Fernwood Internet DMZ infrastructure, the Building 12 Internet DMZ path will provide the primary Internet traffic transit support as the Fernwood infrastructure is optimized. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Time: 7:07PM The maintenance has been completed successfully without complications.
8/18/2008 9:18:26 PM - NIHADM Service Reboot
Description:   At 10:00pm engineers will be rebooting NIHADM. Expected outage time is 10-15 minutes. This will affect users using the ADM console and adm.nih.gov. If you have any questions, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Time: 10:10PM The NIHADM service has been restored and is fully functional.
8/18/2008 2:52:25 PM - NIHnet Bldg 31 MDF UPS Emergency Power Redistribution Maintenance
Description:   Date: Tuesday, August 19, 2008 Time: 5:00am - 8:00am Description: On Tuesday, August 19, 2008 at 5:00am - 8:00am ET, NIHnet engineers will move Building 31 IC and FACnet network device redundant power supplies to UPSs not on the Emergency Power Grid so they will not be disrupted during periodic ORF Emergency Power testing events. Following this maintenance activity ORF engineers will also be conducting an emergency generator power test beginning at 8:15am until 9:15am. Impact: Please note this NIHnet Building 31 MDF UPS Emergency Power Redistribution Demand Maintenance activity has been assessed with a Possible Service Affecting-Low (PSA-Low) impact. Specifically, during the duration of this maintenance activity there should be a momentary impact to the wireless network and not to other network devices in the Building 31 MDF while the redundant power supplies are moved to commercial power due to each UPS having dual power connectivity. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  As of August 19, 2008 at 9:02 AM the NIHnet Building 31 MDF UPS Emergency Power Redistribution Demand Maintenance has successfully been completed.
8/18/2008 12:28:21 PM - NIDA Server Shutdown for Scheduled Maintenance Activity
Description:   Due to the scheduled maintenance on the NSC Building A/C, all NIDA servers will need to be shutdown. Thus, from 10pm on Friday, August 22nd to 6am on Saturday, August 23rd, all NIDA servers will be shutdown affecting the following services: 1. NIDA Share Drives for Branches and Divisions 2. NIDA User Home Drive (F:) 3. NIDA Remote Terminal Servers 4. NIDA Webpages 5. NEPS Please note that this will not affect NIH Services, (Email, VPN, and NIH Websites). Once the maintenance is completed after 6:00am on Saturday, August 23rd, NIDA services will be returned to normal operations. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  On August 23, 2008 at 7:09 AM the building HVAC maintenance has been successfully completed, and all NIDA services are now fully operational. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
8/18/2008 10:41:09 AM - NIH Web Collaboration (Connect/Breeze) Service Issues
Description:   Currently, the NIH Web Collaboration (Connect/Breeze) service is experiencing intermittent connection issues. Engineers are aware of the issue and are investigating the problem. There is no current estimated time to restoration. Further updates will be provided as they are made available. If you have any questions, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The maintenance on the NIH Connect service was successfully completed as scheduled.
8/18/2008 9:39:34 AM - Telecommunications Outage between Building 10 and Fernwood
Description:   Currently, there is a telecommunications outage between Building 10 and the Fernwood Building. Affected services include Aspect, Data, and Video communications. Network connectivity between the two buildings are unaffected by this outage. Technicians have been dispatched, however there is no current estimated time to repair. Further updates will be provided as they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  At 11:30 AM, the telecommunication issues between Building 10 and the Fernwood building were resolved and all affected services have been restored. If you have any questions, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
8/18/2008 8:30:14 AM - NIH Guest Wireless Registration Site Currently Unavailable
Description:   The NIH Guest Wireless Registration website (http://wirelessguest.nih.gov) is currently unavailable, impacting the creation of new guest wireless accounts. CIT engineers are investigating the problem at this time. Further updates will be provided as they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  As of 10:10 AM, the NIH Guest Wireless Registration website has been restored to service. Users are now able to successfully create guest wireless accounts through the website http://wirelessguest.nih.gov. If you have any questions, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
8/16/2008 11:52:31 PM - Power Outage in Rockledge 1 and 2
Description:   There is currently a power outage affecting the Rockledge 1 and 2 buildings. This power outage is affecting network connectivity for users from OD and NHLBI. Pepco engineers have been notified and are working to resolve the outage as soon as possible. There is currently no estimated time of repair. Further updates will be provided as they become available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Power was restored to Rockledge 1 and 2 at approximately 4:20 AM, and network devices are functioning as normal.
8/15/2008 1:51:17 PM - NIHNet CIT Fernwood Fernwood-6500-Conversion Maintenance
Description:   Date: Tuesday, August 26, 2008 Time: 6:00pm - 7:30pm Description: On Tuesday, August 26, 2008 at 6:00pm - 7:30pm ET, NIHnet engineers will be performing re-configuration maintenance on the Fernwood DR routers. Impact: Please note this NIHNet CIT Fernwood Fernwood-6500-DR1 to DR2 Portchannel2 Layer2 Conversion Maintenance activity has been assessed with a Possible Service Affecting-Low (PSA-Low) impact. Specifically, during the duration of this maintenance activity there should be no noticeable affect on Fernwood end-users. The BGP peers that may bounce are utilized by the Remote Access VPN solution, but due to the built-in routing redundancy, service should not be affected. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Maintenance was completed successfully and without complications.
8/14/2008 6:04:09 PM - Vulnerability in Windows Messenger
Description:   A vulnerability (MS08-050 vulnerability in Windows Messenger) is present in Microsoft MSN Messenger and Windows Messenger that could allow for disclosure of sensitive information. Several users at NIH have reported getting strange Instant Messages (IMs) from people in their IM contact list. The message is only a link to myimgzz.com but this link may change. It is highly recommended that you do not use links in IM and do not click on any unexpected links. Users will be directed to a page that appears to be a MSN login page if they click on the link. Successful exploitation will allow the disclosure of the users IM login ID, initiate sessions, and obtain contact information. However, the exploit does NOT expose the user's system login ID (NIH ID) or password. Change your IM login password immediately if you have been exposed to this attack. Recommendations: Download and install the patch available from Microsoft (955702): http://www.microsoft.com/technet/security/Bulletin/MS08-050.mspx Users should change the messenger password immediately if they have been affected by this vulnerability. Do not use Microsoft MSN Messenger or Windows Messenger until the software has been updated to the latest version. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  As of August 29, 2008 a security patch has been used to resolve the vulnerability in windows messenger. Since the patch deployment, no signs of the vulnerability have been detected. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
8/14/2008 2:01:38 PM - Building 12 NIH Login Server Outage
Description:   As of August 14, 2008 at 1:41 PM the NIH Login servers in building 12 are unavailable. The affected applications by this outage include: - ITAS - my.nih.gov - Annual Reports - NIDB Engineers are aware of this issue, and are working towards a resolution. Further updates will be provided once they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  As of August 14, 2008 at 4:30 PM, Building 12 NIH Login Servers have been brought back to service. All services have now been restored.
8/13/2008 4:14:56 PM - CIT Remedy Server Reboot
Description:   The CIT Remedy Server will be rebooted at 4:30 AM on Thursday 08/14/2008. The service will be unavailable for approximately 15 minutes at this time. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The CIT Remedy Server has been rebooted. Remedy is available since 4:41 AM.
8/13/2008 9:50:51 AM - Building 10 Water Line Break
Description:   On August 13, 2008 at 8:35 AM, there was a water line break in Building 10 in the old patient care area. There is substantial flooding from the 11th floor to the B-1 level. The exact impact to the NIH data systems has not been determined at this time. Engineers are on site and assessing the impact to the facility and NIH data systems. Further updates will be provided once they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  On August 14, 2008 facility engineers have repaired the water line break in building 10. There were no reports of affected network resources. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
8/12/2008 3:56:30 PM - Email Requests to the NIH Help Desk Delayed
Description:   At approximately 11AM, due to a failure with the Remedy mail processing engine, requests to the Help Desk via email were not processed immediately. The issue was identified and quickly resolved at approximately 2PM. No requests were lost during this failure and were successfully re-queued. The Help Desk is currently processing the backlog of requests. Thank you for your patience and the Help Desk Management apologizes for the delay in service. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  This issue has been resolved.
8/11/2008 10:01:05 PM - NIHnet ENS-Hatfield Clinical Research Ctr. Bldg. Generator Maint.
Description:   Date: Saturday, August 16, 2008 Time: 10:30am - 12:30pm Description: On Saturday, August 16, 2008 at 10:30am - 12:30pm EDT CRC facility engineers will perform an emergency electrical power systems test for the Hatfield Clinical Research Center (CRC) facility. The test is required monthly, to satisfy the requirements of the Joint Commission Association of Hospital Organizations (JCAHO). http://orf2.od.nih.gov/utilityshutdown/scripts/utility_shutdown.asp?seq=9384 Impact: The impact of the NIHnet ENS - Hatfield Clinical Research Center (CRC) Building Generator Test Maintenance activity will cause the facility to be without the use of the normal electrical power system. Only emergency electrical power will be available (the "RED" electrical outlets and Egress LIGHTING will not be affected). Therefore, the NIHnet ENS - Hatfield Clinical Research Center (CRC) Building Generator Test Maintenance activity is considered PSA-Low due to all CRC building UPS devices that support NIHnet ENS data network equipment RE connected to "red" outlet electrical power receptacles throughout the CRC Building. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  As of August 16, 2008 at 11:45 AM the NIHnet ENS-Hatfield Clinical Research Ctr. Building Generator Maintenance has successfully been completed. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
8/8/2008 1:41:15 PM - SOA Webservices Experiencing Communication Issues
Description:   Currently, the SOA Webservices are experiencing communication issues. Engineers are aware of the issue and are currently investigating the cause. Impacted Service/Application: All applications dependent upon SOA Webservices. Expected Resolution Time: There is no current estimated time to resolution. Further updates will be provided as they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  SOA Webservices has been restored and all applications are fully functional.
8/7/2008 5:04:21 PM - NIH Login Migration
Description:   NIH Login - the authentication method used to grant users access to many applications at NIH - is updating to a more robust environment. NIH Login will begin a series of migrations over the next month. NIH Portal, ITAS, nVision, NBS, and several IC specific applications will be the first in the series to migrate this weekend. Migration efforts will begin after 4pm on Friday, August 8th and continue throughout the weekend. During this window, there is minimal potential that users will experience brief rolling outages of approximately 10 minutes. Additionally, users may be required to login twice in order to have their credentials validated if navigating between systems. Thank you for your patience as we improve NIH Login services for the NIH Community. If you have any questions, please contact the NIH Help Desk at 301-496-4357 or visit us on the web at http://ithelpdesk.nih.gov.
Resolution:  The NIH Login maintenace has been completed successfully.
8/5/2008 3:50:38 PM - NIHADM Reboot
Description:   NIHADM will be restarted tonight at 10 p.m. to clear up memory issues with the box. This outage will be approximately 10 minutes. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The NIHADM reboot has been completed and the server has been restored to service.
8/5/2008 2:44:53 PM - ITAS Emergency Maintenance
Description:   ITAS is scheduled for emergency maintenance on August 5, 2008 evening between 6:30PM - 7:00PM for resolving an e-mail queue problem. During the maintenance, ITAS will not be available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  ITAS emergency maintenance has been completed successfully.
8/4/2008 9:05:13 AM - ORS Intermittent Network Issues
Description:   ORS is currently experiencing intermittent network issues. ITB is aware of the issue and working to resolve the issue. Affected Services, -Access to all network resources Updates will be provided as they become available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  As of 1:34 PM on Monday August 4, 2008 all network services have been restored.
8/2/2008 8:41:40 AM - Building 31B Electrical Maintenance
Description:   An unexpected loss of NIHnet and FACnet data network services has occurred in building 31 due to a scheduled electrical maintenance in the B wing of Building 31. Network devices are being temporarily moved to a new circuit. Connectivity will be restored shortly. Updates will be provided as they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  As of 5:35 PM, Building 31B Electrical Maintenance was completed successfully.
7/31/2008 4:36:39 PM - OD-OIT Firewall Interface Maintenance
Description:   Date: Thursday, July 31, 2008 Time: 10:00pm - 12:00am Description: On Thursday, July 31, 2008 commencing at 10:00pm and ending at 12:00am on Friday, August 1, 2008 NIHnet engineers will perform an Emergency maintenance activity on an OD/OIT managed firewall in Building 10 as a means to replace a faulty GBIC component within the Building 10 OD/OIT Managed Firewall. Impact: Please note the NIHnet OD/OIT Building 10 Firewall GBIC Emergency Replacement maintenance activity has been given a Service Affecting (SA) impact assessment. Specifically, OD data network end-users will experience a brief, one-time network connectivity outage lasting approximately one (1) minute in duration at the beginning of the 10:00pm maintenance window when NIHnet engineers perform a fail-over between firewalls that support OD end-users in NIH Campus Buildings 1, 2, 10, and 13. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The OD-OIT Firewall Interface Maintenance was successfully completed.
7/31/2008 7:56:57 AM - NAppMan Downtime
Description:   On July 31, 2008 from 10:00 AM until 11:00 AM the NAppMan team will be installing new monitoring for the NIH Login service. NAppMan will be unavailable during this window. Impacted Service/Application: NAppMan Further updates will be provided once they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  As of July 31, 2008 at 11:04 AM the NAppMan service is now available. If you continue to have issues with NAppMan, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
7/30/2008 1:11:14 PM - Customer Service Web Server Restart
Description:   STG will be restarting the DCS Web Server at 1:15 PM. Service will be disrupted for the following websites: IT Helpdesk site, Training Site, Publications Site, Remedy Web interface and the Remote Access site. The interruption is expected to last no more than 10 minutes. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Service has been restored to the REMEDY Web interface. Users who have their browser open with the REMEDY web inteface site up are encouraged to close and restart the browser. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
7/30/2008 12:00:14 PM - Westat Oracle Systems Connectivity Issues
Description:   Currently, the following Westat Oracle systems are unavailable: -RSS (Regulatory Support System) -CTSU Member Site -OPEN System Engineers are aware of this issue and are working on a resolution. Currently there is no estimated time to repair. Further updates will be provided once they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  On July 30, 2008 at 12:23 PM connectivity to the Oracle systems have been restored, and the following systems are now available: -RSS (Regulatory Support System) -CTSU Member Site -OPEN System If you have any further issues, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
7/29/2008 8:10:24 AM - NIHADM Server Reboot
Description:   NIHADM server will be rebooted, and there is an expected outage of 10-15mins. Users will be disconnected and the service will be unavailable during that time. Updates will be provided once they are made available. Impacted Service/Application: NIHADM Impacted Users: NIHADM Users Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  As of July 29, 2008 at 9:45 AM the NIHADM service has been restored. If you continue to have issue with the NIHADM, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
7/28/2008 9:40:35 AM - Building 12 2nd Floor CIT Users Network Access Issues
Description:   At this time, some CIT Users in Building 12, 2nd Floor are experiencing issues connecting to the NIHnet due to a failed access switch module. Engineers are aware of the issue and are working to repair network access as soon as possible. Further information will be provided as it is made available. If you have any questions, please contact the NIH Help Desk at 301-496-4357 or visit us on the web at http://ithelpdesk.nih.gov.
Resolution:  As of 10:13 AM, network access has been restored for all affected CIT users in Building 12, 2nd Floor. If you have any questions or are still having difficulty accessing the network, please contact the NIH Help Desk at 301-496-4357 or visit us online at http://ithelpdesk.nih.gov.
7/28/2008 8:48:53 AM - NIH SharePoint Service Access Issues
Description:   At 8:30 AM, the NIH SharePoint Services experienced issues when users attempted to access SharePoint websites. Issues included slow loading pages and inability to access pages. At 8:44 AM, engineers restored access to the SharePoint Services. At this time, all SharePoint websites should be functioning normally. If you have any questions, please contact the NIH Help Desk at 301-496-4357 or visit us on the web at http://ithelpdesk.nih.gov.
Resolution:  At 8:44 AM, access to the SharePoint Services was restored by engineers.
7/25/2008 3:31:18 PM - NIHnet CWDM Ring I Optimization Maintenance
Description:   Date: Saturday, August 9, 2008 Time: 10:00am - 6:00pm Description: On Saturday, August 9, 2008 at 10:00am - 6:00pm EDT NIHnet engineers will perform a proactive maintenance activity on the NIHnet Off-Campus CWDM Ring I data network infrastructure as a means to improve data network connectivity redundancy to existing NIHnet OpDiv DMZ connections from HHS OpDiv and NIH vendor networks located on Executive Blvd and Fishers Place. Impact: The NIHnet CWDM Ring I Optimization Maintenance activity has been given a Possible Service Affecting-Low (PSA-Low) impact assessment for data network traffic traversing the NIHnet Off-Campus CWDM Ring I data network infrastructure. Specifically, when the new redundant network connection is activated across the Off Campus CWDM ring, NIHnet data network connectivity for Executive Blvd and Fishers Place networks is not expected to fail. However, there is a low potential for momentary, limited packet transit delay between the NIHnet Core Backbone and the Executive Blvd and Fishers Place networks as each node on the NIHnet Off-Campus CWDM Ring I data network infrastructure is tested for performance and reliability post new redundant connection activation. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  As of August 9, 2008 at 2:13 PM the NIHnet CWDM Ring I Optimization Maintenance has successfully been completed. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
7/25/2008 2:55:12 PM - NIHnet OD/OIT Bldg 10 Firewall GigE NIC Emergency Replacement
Description:   Date: Friday, July 25, 2008 Time: 10:00pm - 12:00am Description: On Friday, July 25th, 2008 commencing at 10:00pm and ending at 12:00am on Saturday, July 26, 2008 NIHnet engineers will perform an Emergency maintenance activity on an OD/OIT managed firewall in Building 10 as a means to repair a faulty Gigabit network interface card (NIC). Impact: Please note the NIHnet Managed Firewall OD/OIT Building 10 Firewall GigE NIC Replacement Emergency Maintenance activity has been given a Service Affecting (SA) impact assessment. Specifically, OD data network end-users will experience a brief, one-time network connectivity outage lasting approximately one (1) minute in duration at the beginning of the 10:00pm maintenance window when NIHnet engineers perform a fail-over between firewalls that support OD end-users in NIH Campus Buildings 1, 2, 10, and 13. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The Building 10 Firewall NIC replacement maintenance was unsuccessful and will be rescheduled for a later date.
7/25/2008 11:11:08 AM - NIHnet Building 31 MDF Switch Technology Refresh Maintenance
Description:   Date: Saturday, August 16, 2008 Time: 7:30am - 5:30pm ET Description: On Saturday, August 16, 2008 at 7:30am - 5:30pm ET, NIHnet engineers will perform a NIHnet Building 31 Enterprise Network Service (ENS) Technology Upgrade Maintenance activity. Impact: Please note the NIHnet Building 31 Enterprise Network Service (ENS) Technology Upgrade Maintenance activity has been given a Service Affecting (SA) impact assessment. Specifically, throughout the duration of this maintenance window all affected IC data networks located in Building 31 will experience a sequential 20 – 30 minute rolling disruption of network connectivity to NIHnet while each individual Building 31 IC VLAN is migrated to the new redundant state-of-the-art Cisco Aggregation Switch (AGS) data network infrastructure located in the Building 31 MDF network room. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The NIHnet Building 31 Enterprise Network Service (ENS) Technology Upgrade Maintenance activity scheduled for Saturday, August 16, 2008 at 7:30am - 5:30pm has been postponed due to scheduling conflicts. Therefore, this technology upgrade maintenance activity shall be rescheduled and announced for a later date and time in the very near future.
7/24/2008 2:29:15 PM - NIHnet OpDiv DMZ Path Failover Maintenance
Description:   Date: Wednesday, July 30, 2008 Time: 5:00am - 6:00am Description: On Wednesday, July 30, 2008 at 5:00am - 6:00am EDT The purpose of the NIHnet OpDiv DMZ Path Failover Maintenance activity is for NIHnet engineers to proactively redirect data network traffic between NIHnet and HHSnet OpDiv networks from the active NIHnet OpDiv DMZ Building 12 path to the redundant NIHnet OpDiv DMZ Building 45 network path. This NIHnet OpDiv data network path failover maintenance activity is in support of an impending Building 12 NIHnet OpDiv DMZ firewall operating system upgrade maintenance activity that will provide improved firewall security features and processing performance. Impact: The NIHnet OpDiv DMZ Path Failover Maintenance activity has been given a Possible Service Affecting-Low (PSA-Low) impact assessment for data network traffic traversing the NIHnet OpDiv DMZ. Specifically, when data network connectivity between NIHnet and OpDiv network peers is redirected from the Building 12 DMZ path to the redundant Building 45 DMZ path, there is a low probability for disruption to network connectivity and only a minimal amount of packet transit delay is expected as existing traffic flows shift from the primary network path to the secondary network path.
Resolution:  This maintenance has been completed successfully without any complications. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
7/24/2008 8:41:49 AM - NIHADM Intermittent Issues
Description:   Currently the NIHADM is experiencing intermittent issues and the service needs to be restarted. The system will be rebooted at 8:45am, and there will be 10-15 minutes were the NIHADM service will be unavailable. Further updates will be provided once they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The NIHADM service has been successfully restored. If you continue to have issues with the NIHADM, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
7/21/2008 1:00:07 PM - NIHDDNS Hardware Refresh Maintenance
Description:   Engineers will be replacing the NIHDDNS on Thursday, July 24, 2008 beginning at 9:00pm. NIHDDNS2 will remain online and responsive during the maintenance period and NIH DNS name resolution should be unaffected. If you have any questions, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The NIHDDNS replacement has been successfully completed. DNS has been restored to service on the new server and is responding properly to name lookups.
7/21/2008 9:48:50 AM - Telephone Outage for NCI in Building 37/ 1st Floor
Description:   Approximately 40 telephones belonging to NCI customers from Building 37/1st floor have lost service. A technician is on the way to the building to assess & repair. Updates will be provided as soon as further information is available. Please do not reply to this message. If you have any questions, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  As of 1:57 PM service has been restored to all of the telephones that were out of service.
7/21/2008 7:24:04 AM - PDF Reader on BlackBerry Devices
Description:   Due to a recently reported vulnerability with the BlackBerry PDF reader, CIT engineers have temporarily disabled the ability to read PDFs from BlackBerries as recommended by the vendor. This feature will be re-enabled once the patches are fully tested and implemented on the NIH BlackBerry servers. Updates will be provided as they become available. Please do not reply to this message. If you have any questions, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  CIT engineers have completed installation of a software patch to correct the Blackberry PDF vulnerabilities. NIH Blackberry service were restored by 11:00 PM and PDF viewing has been re-enabled.
7/20/2008 3:48:54 AM - Building 29/29A Power Outage
Description:   As of 1:19 AM on Sunday, July 20, 2008, Building 29A is experiencing a power outage. This outage may affect network connectivity for Building 29A users from CIT, FDA, and FACnet. Engineers are working to restore power, but there is no estimated time of resolution. Further updates will be sent as they become available. Please do not reply to this message. If you have any questions, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Power to Building 29/29A has been restored.
7/18/2008 8:07:16 AM - NIHADM Service Disruption
Description:   The NIHADM service will undergo an immediate service disruption. During this time, NIH users will be unable to access the ADM service. The outage is expected to be resolved within 5-10 minutes. If you have any questions, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The service has been restored as of 12:00 PM on Friday July18th, 2008.
7/16/2008 4:43:34 PM - nVision Data Warehouse Maintenance
Description:   The nVision Data Warehouse websites, reporting and databases will be unavailable on Friday, July 18, 2008 from 7:00 a.m. until 3:00 pm due to major security enhancements. If you have any questions, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Time: 2:54PM The nVision Data Warehouse websites have been updated and are now available.
7/15/2008 6:08:25 PM - FDA Staff Unable to Access ITAS Through ITAS Production Link
Description:   Currently there are many FDA staff members who are unable to access ITAS using the ITAS production link. The issue is currently being investigated by engineers. There is no estimated time of resolution. Updates will be made available as they are received. If you have any questions, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Time: 6:35PM The issue has been resolved. FDA staff members can now access ITAS through the production link.
7/13/2008 9:44:18 AM - Remedy Service Unavailable
Description:   The CIT Remedy service is currently unavailable. CIT engineers are currently working to address the issue, but there is currently no estimated time of resolution. Further updates will be sent as they become available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The CIT Remedy service is back up. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
7/11/2008 10:42:00 PM - ENS CIT Fernwood 3NE LAN Switch Upgrade Maintenance
Description:   Description: On Saturday, July 26, 2008 at 8:00am - 6:00pm EDT, NIHnet engineers will perform a technology upgrade maintenance activity on the Cisco LAN switches located in the Fernwood 3NE LAN closet. This technology upgrade maintenance will consist of total LAN switch upgrades of the existing devices to include installation of a new chassis, module cards, operating systems, and patch cable standardization. Impact: The ENS CIT Fernwood - Phase 4 Fernwood 3NE LAN Switch Technology Upgrade Maintenance activity has been assessed with a service affecting (SA) impact. Specifically, all Fernwood 3rd floor NE NIHnet connected end-users will experience a loss of connectivity to NIHnet throughout the duration of this maintenance window. Please note the Aspect system management network connections will be moved to a temporary network facility during this maintenance window and therefore are expected only to momentarily lose connectivity to NIHnet at the beginning and end of the maintenance window as Aspect network connections are moved to / from the temporary network switch. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Time: 8:11PM Maintenance has been completed successfully with minor complications.
7/10/2008 4:55:13 PM - NIHnet OpDiv DMZ Path Failover Maintenance
Description:   Date: Friday, July 18th, 2008 Time: 1:00am - 2:00am Description: The purpose of this maintenance is to move Opdiv traffic off of the Building 12 path and over to the Building 45 path. Opdiv traffic will remain on the Building 45 path until NSS performs a firewall code upgrade maintenance on the NIH-Opdiv-B12-PIE1. Once this is complete the Opdiv traffic will be moved back to the Building 12 path. The firewall code upgrade has not been scheduled as of yet. NSS recently performed a major code revision upgrade on the NIH-Opdiv-B45-PIE1 firewall and will need to monitor the traffic across this firewall for a period of time to verify any problems that may have occurred, due to the code upgrade, have been resolved prior to upgrading the code on the NIH-Opdiv-B12-PIE1 firewall. During this period, if the Building 12 Opdiv path is required, it will need to be manually re-enabled since its BGP peering session will be shut down. Impact: OpDiv traffic is currently traversing the Building 12 OpDiv path and will be shifted to the Building 45 OpDiv path during this maintenance. Due to the fact the firewalls on these two paths do not share state between each other OpDiv users may experience brief latency during the traffic shift while their connections automatically re-established across the OpDiv Building 45 path. Please don't hesitate to contact the CAP office at 301-402-0724 or via email at citdcscap@mail.nih.gov if you have any questions regarding this maintenance.
Resolution:  The maintenance had to be backed out and will be rescheduled for a later date.
7/9/2008 8:12:03 AM - CIT Customer Support Reports Service and Website issues
Description:   The CIT Customer Support Reports service did not fully mirror the previous day's data overnight and is experiencing issues this morning. Any reports you may pull this morning may be incomplete, and customers may be unable to access the various components of the service. We are working to diagnose and correct the problem and will provide further details as they become available. If you have any questions, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The service has been restored, all data has been mirrored and the automatic subscription jobs are completing. Customer who received two copies of their overnight reports should discard the earlier one and refer to the later one for the correct data. Any reports run manually before 8:45 AM this morning. should be rerun to get the correct data. We appreciate your patience and cooperation while we worked to resolve this issue.
7/2/2008 1:20:08 PM - Nihcesmlbx15 Server Reboot
Description:   The Nihcesmlbx15 exchange server is currently being rebooted. The mailbox will be unavailable for 10 minutes. Further updates will be provided once they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  As of July 2, 2008 at 1:38PM the Nihcesmlbx15 server was restored, and is now operational. If you have any further issues, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
7/1/2008 8:06:29 AM - 5635 Fishers Lane Telephone Outage
Description:   At approximately 6:48 AM on Tuesday July 1, 2008 a telephone outage was reported to Verizon at 5635 Fishers Lane lower level. Technicians are on site trying to resolve the problem. Updates will be provided once they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  As of July 1, 2008 at 8:00 AM, Verizon restored service to all the telephone lines affected by this outage. If you have any further issues, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
6/30/2008 8:30:45 PM - NIHCESMLBX12 Server Maintenance
Description:   CES engineers will need to reset the services on NIHCESMLBX12 this evening, June 30, 2008, at 10:00pm due to a resource issue. Access to mailboxes on this server will be down for about 10 minutes while the services are being reset. The following ICs are affected: CC NIAID If you have any questions, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  NIHCESMLBX12 server has been reset. Mailbox access has been restored and is now operational.
6/27/2008 10:20:33 AM - Iforgotmypassword Service Upgrade
Description:   On Monday June 30, 2008 at 6:00am, the iforgotmypassword service will be upgraded. This service outage will last no longer than 45 minutes. Impacted Service/Application: iforgotmypassword Expected Resolution Time: Date: June 30, 2008 Time: 6:45am Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The upgrade was completed successfully. All services are back up and fully functional.
6/26/2008 2:17:56 PM - SharePoint Service Outage
Description:   Currently, the SharePoint services are out of service. Engineers are working to resolve this issue. Further updates will be provided as they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Engineers have identified the issue and have stabilized the environment. All SharePoint sites are now up and running.
6/25/2008 11:20:16 AM - Scheduled Maintenance for VTC Bridging
Description:   VTC High Definition Bridging will be down for maintenance between 5:00pm - 5:30pm on Wednesday June 25, 2008. During this maintenance HD VTC Bridging will not be available. Impacted Service/Application: Video Conferencing High Definition Bridging Expected Resolution Time: Date: June 25, 2008 Time: 5:30 PM If you have any questions regarding this maintenance, please send an e-mail to the VTC administrators at NIHVTCAdmin@mail.nih.gov.
Resolution:  All HD video bridging services have been restored.
6/24/2008 11:22:46 AM - ITAS Scheduled Maintenance (June 25, Wednesday, 8:00PM - 10:30PM)
Description:   ITAS is scheduled for maintenance on June 25, Wednesday evening between 8:00PM - 10:30PM for resolving a domain controller issue. During the maintenance, ITAS will not be available. If you have any questions, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  ITAS scheduled maintenance has successfully completed. ITAS is now available.
6/18/2008 11:20:32 PM - Fernwood Telephone Outage June 19-20
Description:   Title: Fernwood Telephone Ductbank Move Maintenance Event Details: Beginning Thursday, June 19 at 11:00 PM and running to Friday, June 20 at 5:00 AM, technicians will be transferring the Fernwood telephone lines from their current ductbank to a new ductbank to support ongoing construction at the Fernwood building. During the maintenance window, all phone lines at the Fernwood building will not be available. Essential phone lines will be transferred to back-up numbers during the maintenance window to maintain a continuity of critical operations. If you have any questions, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov .
Resolution:  As of 3:30 AM on Friday, June 20, the Fernwood telephone ductbank move maintenance has been successfully completed.
6/18/2008 8:18:01 AM - CIT Customer Support Reports Service and Website issues
Description:   The CIT Customer Support Reports service experienced issues overnight. The service has been mostly restored. Once full service is available and confirmed the overnight reports will be run. Updates will be provided once they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  All data has been mirrored and the overnight reports have completed. If you received two copies of your overnight reports, please refer to the latest. The report server data is now current. Thank you for your patience as we worked to address this issue. If you have any further issues, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
6/16/2008 2:32:25 PM - Helix, Doublehelix, and Biowulf Systems Downtime June 16, 2008
Description:   All Helix Systems (Helix, Doublehelix and the Biowulf cluster) will be unavailable this evening, June 16, through to early morning, June 17, during which time major upgrades will be performed. Helix and Doublehelix will become unavailable to users at 9 pm this evening. The Biowulf batch system will be shut down at 3 pm and the Biowulf login node will become unavailable at 6 pm. During the downtime, all Helix services will be inaccessible including Helix email via POP & IMAP clients, Squirrelmail and pine. Email to Helix will be queued during the downtime for delivery once the system is back up. We anticipate that all services will be available by 9 am on Tuesday morning. If you have any questions please contact staff@helix.nih.gov.
Resolution:  As of June 17, 2008 at 9:30 AM the Helix Systems downtime has been completed. The Helix system was successfully upgraded and the system was made available starting at 9:30 this morning. If you have any further issues, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
6/14/2008 2:44:56 PM - Problems on NIH Login Cluster 2
Description:   Date: June 14, 2008 Time 2:00 PM At the moment, there are intermittent issues with the NIH login. Engineers are aware of this issue and are working on a solution. Updates will be provided once they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The intermittent issues associated with the NIH Login have been resolved. NIH Login is now operational.
6/11/2008 4:06:24 PM - Campus Home Server Emergency Reboot
Description:   There were connectivity problems with the OD campus home directory server, and a reboot was required. The system was rebooted at 3:45 p.m. OIT expects to have the problem resolved shortly thereafter. Further updates will be provided as they are made available. If you have any questions or concerns please contact the CAP team on 301-402-0724 or citdcscap@mail.nih.gov
Resolution:  This issue has been resolved. Everything is now back to normal.
6/11/2008 8:02:26 AM - NAppMan Downtime
Description:   CIT engineers will be bringing NAppMan down at 10:00 AM on Wednesday June 11, 2008 to install NEES on the dashboard. The outage should be for one hour. Impacted Service/Application: NAppMan Expected Resolution Time: Date: June 11, 2008 Time: 11:00 AM Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  As of June 11, 2008 at 10:31 AM NAppMan dashboard is once again available, and NEES is reporting properly. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
6/10/2008 8:10:19 AM - Immediate NBS, DbGap, caBig Integrator, CIMS PTMS Outage
Description:   The following applications and websites are now available: - NBS - caBig Integrator - ITO Trainee Tracking Database for NHGRI - DbGap - CIMS PTMS - http://inside.genome.gov - http://Mynci.cancer.gov - http://Mynciadmin.cancer.gov Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The NBS outage was fully resolved after problems with the redundant SSO login server were resolved.
6/9/2008 2:17:47 PM - Building 9 UPS-2 Replacement Maintenance
Description:   Time: 1:00am - 2:00am Description: On Wednesday, June 18th, 2008 at 1:00am - 2:00am EDT, NIH engineers will replace the defective redundant Building 9, B9-UPS-2 which will restore maximum backup time and electrical protection for the Building 9, B9-6500 & B9-5509 switches. Impact: Please note this B9-UPS-2 Replacement Maintenance has been assessed with a Possible Service Affecting-Low (PSA-Low) impact. Specifically, during the replacement of the B9-UPS-2, the B9-UPS-1 should provide sufficient backup power for the Building 9, B9-6500 and B9-5500 switches and not impact end-user network connectivity. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The Building 9 UPS-2 Replacement Maintenance was completed successfully.
6/9/2008 12:15:59 PM - 2115 East Jefferson Telephone Outage on 4th Floor
Description:   Date: June 9th, 2008 Time: 12:05 PM Description: At this time there is a telephone outage on the 4th floor in 2115 East Jefferson. The TIB engineers are aware of the situation and are working towards a resolution. Update will be provided as they become available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  As of 2:29 PM all telephone service at 2115 East Jefferson Street/4th floor has been restored.
6/8/2008 10:37:24 AM - Remedy Service is Unavailable
Description:   At the moment, the Remedy service is unavailable. CIT engineers are aware of this issue and are working on a solution. There is currently no estimated time of resolution. Updates will be provided once they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  As of June 8th 2008 at 10:30 AM CIT engineers resolved the issue, and Remedy is now accessable. If you have any further issues, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
6/5/2008 11:08:11 AM - Executive Plaza and East Jefferson Power Outage
Description:   Due to a power outage, many buildings in Executive blvd and East Jefferson have lost of network resources. Engineers are aware of this issue, and are working on a solution. Updates will be provided once they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  As of June 6th 2008 at 8:56 AM Executive Blvd and East Jefferson's power and network connectivity was restored. All known outstanding issues have been repaired. If you have any further issues, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
6/5/2008 10:52:48 AM - NIH Home Page Unavailable
Description:   NAppMan Alert - NIH Home Page is experiencing problems and is not available.
Resolution:  The NIH Home Page was unavailable between 10:41 AM and 11:03 AM. The problem has been resolved and will not reoccur. If you have any questions, contact the NIH Help Desk at 301-496-4357 or visit us on the web at http://ithelpdesk.nih.gov.
6/4/2008 6:37:12 PM - Power Outage at Fishers Lane
Description:   Due to power outages in the Fishers Lane cluster, NIHnet equipment is currently running on battery power. PEPCO has not provided a time for restoration due to the large number of outages throughout the region. The NIHnet primary data facility in 5635 Fishers Lane has sufficient battery power to sustain connectivity until late Thursday morning. However, customers within 5625 and 5635 may experience loss of connectivity if individual closet-based batteries (UPS) are depleted prior to power restoration. Locations connected to emergency generator power should not be impacted. Further updates will be provided as they are made available. We apologize for any inconvenience this may have caused. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  As of 8:14 AM on June 5th, 2008 all power and network devices located in Poolesville and Fishers lane were restored to service. If you have any further issues, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
6/4/2008 5:05:53 PM - Telephone Outage Building 12B Room 2N207
Description:   Due to the inclement weather, there is currently a telephone outage in Building 12B Room 2N207. A technician is en-route to restore service. Further updates will be provided as they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  As of June 5th, 2008, 12:00 PM Verizon repair restored telephone service to Building 12B room 2N207. If you have any further issues, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
6/4/2008 4:08:20 PM - Telephone Outage 2nd Floor Building 10ACRF
Description:   Due to the inclement weather, there is currently a telephone outage in Building 10ACRF 2nd floor affecting approximately 40 telephones. A technician is en-route to restore service. Further updates will be provided as they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Repair technicians have restored the service to all 40 telephone lines.
6/4/2008 3:59:53 PM - HRSA Power Outage
Description:   At the moment, HRSA is experiencing a power outage, and have taken all servers’ offline. Engineers are aware of this issue, and working on a solution. Updates will be provided once they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  As of June 5th 2008 at 6:00 AM, servers were brought back up, and everything is back up and running. If you have any further issues, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
6/4/2008 3:41:34 PM - Power Outage in Rockledge 1
Description:   Rockledge 1 is currently without power. The building is also without Aspect and Remedy. Engineers are aware of this issue and are working towards a resolution. Further updates will be provided as they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Rockledge 1 is back up and fully operational.
6/4/2008 10:12:05 AM - NIHADM2 Service Reset
Description:   On June 4th, 2008 at 11:45 AM the NIHADM2 service will be reset . This service reset should take no longer than 10 minutes. ADM users connected to NIHADM2 can redirect to the redundant NIHADM server. If you have any further issues, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  As of June 4th 2008 at 12:00 PM NIHADM 2 was back up and running. If you have any further issues, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
6/3/2008 3:21:30 PM - IForgotMyPassword Service Update
Description:   As of 6:50 AM on June 4th, 2008 EMIB has restored service to the Password Self Service (iforgotmypassword.nih.gov). If you have any further issues, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  As of 7:00 AM on June 4th, 2008 EMIB has restored service to the Password Self Service (iforgotmypassword.nih.gov). If you have any further issues, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
6/3/2008 9:04:14 AM - NHGRI Server Room shutdown
Description:   At the moment, NHGRI has shut down a number of servers in Democracy 2 due to the AC units going offline. Technicians are currently working on putting the AC units back online. Updates will be provided as they become available. Impacted Service/Application: All NHGRI Resources hosted at Democracy 2, including website and databases. Impacted Users: NHGRI Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  At 11:00 AM the AC units in Democracy 2 were repaired, and Democracy 2 data center is back online. If you continue to have problems please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
6/2/2008 8:00:51 PM - Biowulf Login Node Issues
Description:   The "login node" for gaining access to the NIH Biowulf cluster (biowulf.nih.gov) is currently down due to an unknown hardware problem. DCSS/SCB system administrators are attempting to address the issue. Users of the NIH Biowulf Cluster will not be able to login to biowulf.nih.gov until the issue is resolved. Users will not be able to submit new jobs or check on the status of existing jobs. Jobs currently running on the cluster are not affected by the failure of the login node. Currently there is no estimated time of resolution. Updates will be made available as they are received. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Time: 9:44PM The Biowulf login node has been restored to standby hardware. Biowulf users should now be able to login.
5/31/2008 6:55:26 AM - Redundant Circuit to Sterling Data Center Unavailable
Description:   The redundant circuit to the Sterling Data Center is currently unavailable. Because this is a redundant circuit, NIH users should not experience any loss in network connectivity. Qwest engineers are investigating the issue. There is currently no estimated time of resolution. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The redundant circuit has been restored to service. CIT engineers will continue to monitor the situation for changes.
5/30/2008 2:40:17 PM - Fibergate Fernwood Building Fiber Relocation Maintenance
Description:   Date: Saturday, June 7, 2008 Time: 06:00am - 6:00pm Description: On Saturday, June 7, 2008 beginning at 06:00 a.m. and ending on Saturday, June 7, 2008 at 6:00 p.m., Fibergate will perform a scheduled maintenance activity to relocate Fernwood fiber optic connections in support of a new parking garage project next to the Fernwood building. Impact: Please note the Fibergate Fernwood Building Fiber Relocation Maintenance activity has been given a Possible Service Affecting-Low (PSA-Low) impact assessment for NIHnet connectivity for Democracy I, Democracy II, and Fernwood network end-users. Specifically, NIHnet data network connectivity for these locations will automatically failover to redundant network paths as each fiber pair is relocated in a predetermined sequence order. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Fibergate Fernwood Building fiber relocation maintenance completed successfully without any complications.
5/30/2008 1:28:52 PM - Electronic Voicemail Issues
Description:   Date: May 30, 2008 Time: 1:26 PM Event Description: At this time, the EVM system for electronic voicemail for the 594, 496, 480, 435 and 402 exchanges is not working. Verizon is working on the solution and it should be up and working soon. Updates will be provided as they become available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  As of 2:31 PM on May 30, 2008 this issue was resolved. All systems are up and functional.
5/30/2008 11:35:54 AM - Websites access issues
Description:   Date: May 30, 2008 Time: 11:18 AM Event Description: At this time customers may experience intermittent issues accessing some websites. Some reported websites include: http://content.nejm.org/ http://circ.ahajournals.org/contents-by-date.0.shtml http://jama.ama-assn.org/cgi/content/abstract/288/17/2144 http://eurheartj.oxfordjournals.org/cgi/content/abstract/24/4/320 http://Cancer.org http://jbc.org http://ucsc.edu www.sciencemag.org Updates will be provided as soon as they become available. CIT engineers are working to resolve the problem as soon as posible. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  As of 1:40 PM on May 30, 2008 this issue was resolved.
5/28/2008 2:25:04 PM - NIH Building 12 Data Center Equipment Relocation Maintenance
Description:   Date: Wednesday, June 4, 2008 - Thursday, June 5, 2008 Time: Start 5:00pm (6/4) - End 6:00pm (6/5) Description: On Wednesday, June 4, 2008 at 5:00pm EDT NIHnet engineers will proactively migrate NIHnet Internet traffic from the Building 12 Data Center path to the Fernwood Internet path in preparation for the NIH Building 12 Data Center Internet DMZ Device Relocation Maintenance activity on Thursday, June 5, 2008 On Thursday, June 5, 2008 at 8:00am EDT NIHnet engineers will relocate the Building 12 Internet DMZ network devices to a new cabinet in the Building 12 Data Center. The relocation of the Internet DMZ network devices expected to be completed by 5:00pm on Thursday, June 5, 2008 with NIHnet Internet traffic redirected back to the Building 12 Data Center path by 6:00pm. Impact: Please note the NIH Building 12 Data Center Internet DMZ Device Relocation Maintenance activity has been assessed with a Possible Service Affecting-Low (PSA-Low) impact even though all NIHnet Internet traffic will be redirected to the Fernwood Internet path prior to the physical relocation of the Building 12 Data Center Internet DMZ network devices. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  NIH Building 12 Data Center equipment relocation maintenance has been completed successfully without any complications.
5/27/2008 9:02:00 AM - Inbound ISDN Video Conferencing is Down
Description:   At this time, Inbound ISDN Video Conferencing traffic is down. Engineers are aware of the problem and are working with their external partners to resolve the issue. There is no estimated time of resolution. Updates will be made available as they are received. Impacted Service/Application: Inbound ISDN Video Conferencing Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  As of 8:30 AM today May 28, 2008 this issue was resolved. At this time all Inbound ISDN Video Conferencing traffic is up and fully functional.
5/23/2008 12:42:08 PM - Connectivity Issues with FDA and SOA Environments
Description:   At this time the FDA Federation Login & SOA Tiers are experiencing connectivity issues. Engineers are aware of the problem and are working with their external partners to resolve the issue. There is no estimated time of resolution. Updates will be made available as they are received. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Time: 1:28PM The FDA Federation Login & SOA Tiers have regained connectivity. All services are functional at this time.
5/23/2008 10:58:46 AM - PKI System Connectivity Issues
Description:   The HHS PKI system is currently down. At this time, the system is unable to issue or revoke any certificates. Current live certificates are still valid. The engineers are aware of the situation and are working towards a resolution. Update will be provided as they become available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Service has been restored to the HHS PKI digital certificate system. The site will continue to show the banner describing a temporary outage will be removed later today.
5/22/2008 4:57:07 AM - Brief Power Outage in Executive Boulevard 6001 and 6100
Description:   On Thursday, May 22 at 4:04 AM, the 6001 and 6100 Executive Boulevard buildings experienced a brief power loss affecting network connectivity for the following ICs: OD, NIDA, NIMH, NICHD, NSC, and CIT. Power began returning to these buildings starting at 4:17 AM. CIT engineers continue to monitor the situation for changes, but all service appears to have been restored at this time. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Power began returning to service at 4:17 AM.
5/21/2008 10:11:12 AM - Building 6700A Rockledge Outage
Description:   There was a fire at 6:30am on the 3rd floor electrical closet of building 6700A Rockledge. Due to the fire there is limited electricity service. Network and wireless connectivity are nonfunctional. Engineers are aware of the issue, and are working on a solution. Updates will be provided as they are made available. Impacted Users: All Building 6700A Rockledge Users Expected Resolution Time: There is no estimated time for resolution at the moment. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Time: 10:53AM At this time engineers have been able to return all affected services to normal.
5/19/2008 2:29:35 PM - NIHITS II Access Issues
Description:   NIHITS II is experiencing issues that are preventing users from accessing the system. OIT engineers are working to resolve the issues. Updates will be made available as they are received. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Time: 4:00PM The NIHITS II access issue has been resolved at this time and all operations have returned to normal.
5/17/2008 12:06:07 AM - NIHCESMLBX15 Downtime
Description:   As part of CIT's scheduled maintenance of NIH email services, the NIHCESMLBX15 exchange server is currently unavailable. This extended maintenance downtime is affecting NLM users. The maintenance activity is scheduled to be completed no later than 6:00 AM. Further updates will be sent as they become available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The NIHCESMLBX15 server, which services NLM, is back in service as of 8:30 AM on Saturday, May 17. EMIB engineers will be continuing their evaluation of the incident. No further service disruptions are planned.
5/16/2008 12:46:46 PM - Telephone Outage in Building 50 on the 6th Floor
Description:   At this time there are approximately 40 telephones out on the 6th floor in Building 50. Verizon technicians are on site and are waiting for a piece of equipment to arrive in order to resolve the issue. The estimated time of service restoration is at 1:45PM. Updates will be made available as they are received. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Time: 3:24PM The service to the telephones in Building 50 has been restored at this time.
5/16/2008 4:42:21 AM - Reload of Building 29 B29-4000 Switch
Description:   During the early morning of Friday, May 16, NIHnet engineers conducted a reload of the Building 29 B29-4000 switch to rectify performance issues. The reload may have briefly impacted NIHnet connectivity for users from OD, SCADA, and FDA. This maintenance has been successfully completed, and CIT engineers continue to monitor the device for further issues. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Initial/resolved Hot News notification.
5/15/2008 2:33:51 PM - Guest Wireless DMZ B45 Firewall Code Upgrade Maintenance
Description:   Time: 5:00pm - 7:00pm On Monday, May 19, 2008 at 5:00pm - 7:00pm EDT the NIHnet Guest Wireless network security infrastructure located in Building 45 will be upgraded to improve performance and reliability. Impact: There will be brief service interruptions to Guest Wireless Network users only with each service interruption lasting for several minutes while the Building 45 Guest Wireless Firewall operating system is upgraded. The NIHnet Wireless Network Service for all NIH personnel WILL NOT be affected by this maintenance activity because NIHnet Wireless Network access and VPN connectivity will fail-over to the redundant path via the Building 12 Wireless DMZ infrastructure. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Time: 6:11PM The maintenance has completed successfully without complications.
5/14/2008 1:43:25 PM - ITAS Scheduled Maintenance
Description:   ITAS is scheduled for maintenance on May 15, Thursday evening between 6:00PM - 6:30PM for server performance improvement. During the maintenance, ITAS will not be available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Maintenance has been completed successfully without any complications.
5/14/2008 12:08:42 PM - Portal Server Maintenance
Description:   On Thursday evening, May 15th, two portal servers will undergo maintenance. There should be little or no user impact during this time. It may be necessary to simply refresh portal pages that were displayed during this period should any content look incorrect. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Time: 7:45PM The maintenance has been completed successfully.
5/14/2008 12:32:55 AM - Adobe Connect meetings running without SSL
Description:   Adobe Connect meetings are currently running without SSL. While login to Connect is still protected by SSL, the actual meeting traffic is no longer encrypted. Encryption should return to service by 10pm Wednesday, May 14th. Please direct any questions to: webmeeting@mail.nih.gov.
Resolution:  This issue has been resolved and Adobe Connect meetings are now encrypted as normal.
5/13/2008 9:32:03 PM - Mailbox Server Reset for NIHCESMLBX4 and NIHCESMLBX8
Description:   The NIH CES Administrators will need to perform an emergency reset of NIHCESMLBX4 and NIHCESMLBX8 tonight at 10:00 PM due to an issue with the backup software. The IC's effected are: NIHCESMLBX4: OD (2045) NIAID (34) NIAAA (296) FIC (77) VRC (186) NIDA (111) NCI (81) NIHCESMLBX8: NIDCD(292) NIDA (548) NCCAM (142) This outage is not expected to exceed 10 minutes per server. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The reset maintenance for NIHCESMLBX4 and NIHCESMLBX8 has been successfully completed.
5/13/2008 4:06:43 PM - Guest Wireless DMZ B12 Firewall Code Upgrade Maintenance
Description:   Date: Friday, May 16th, 2008 Time: 5:00pm - 7:00pm Description: On Friday, May 16th, 2008 at 5:00pm - 7:00pm EDT The NIHnet Guest Wireless network security infrastructure located in Building 12 will be upgraded to improve reliability. Impact: There will be brief service interruptions lasting for several minutes during the maintenance window with each service interruption ONLY affecting connected Guest Wireless end-users. The NIHnet Wireless Service WILL NOT be affected by this maintenance activity because NIHnet Wireless access and VPN connectivity is supported by a redundant path via the Building 45 Wireless DMZ infrastructure. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Time: 8:21PM The maintenance has been completed successfully without complications.
5/13/2008 10:05:27 AM - Building 5 to Valve Vault 5 Directional Bore Maintenance Project
Description:   ORF will be continuing their fire alarm project on the NIH Campus by drilling a directional bore between Building 5 to Valve Vault 5 on Thursday, May 15, 2008. Cable Management Branch is aware of the maintenance and is prepared for any unforeseen outages as a result of the maintenance work. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  No issues have been reported from this maintenance activity from the support team nor customers.
5/12/2008 1:44:55 AM - Sterling Data Center Redundant Internet Connection Outage
Description:   Currently the Sterling Data Center redundant Internet connection is down. Engineers have been notified and are working to resolving the issue. At this time there is no estimated time of resolution. Further details will be posted as soon as they are available.
Resolution:  The fiber cut between Virginia and DC has been repaired and the redundant connection is operational at this time. Network Operations engineers will continue to monitor the circuit closely for the next 24 hours.
5/9/2008 10:06:30 AM - FIC Building 16 LAN Closet Power Outage
Description:   At this time, FIC users in Building 16 are experiencing network connectivity issues due to a power outage in the FIC Building 16 LAN Closet. Engineers are aware of the issue and are working to resolve it as soon as possible. Updates will be provided as they are made available. Impacted Service/Application: Network connectivity Impacted Users: FIC Building 16 users ONLY Expected Resolution Time: There is no current estimated time to resolution. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The power in Building 16 LAN Closet was restored and at this time all the connectivity issues were resolved. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
5/9/2008 12:52:54 AM - Power Outage in Rockledge 1 and 2
Description:   There is currently a power outage in the Rockledge 1 and 2 buildings. This outage is currently affecting network connectivity for Rockledge 1 users from the following ICs: NLM, NHLBI, NICHD, and OD. It is also affecting Rockledge 2 users from the following ICs: NHLBI, OD, and NICHD. Pepco engineers are on the way to resolve the issue, but there is currently no estimated time of repair. Further updates will be sent as they become available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  All network connectivity in Rockledge 1 and 2 appears to be restored at this time. CIT network engineers will continue to monitor the situation for changes.
5/7/2008 4:30:51 PM - Connect System Maintenance
Description:   Time: 9:00pm - 10:00pm Description: The Connect (formerly Breeze) service will be unavailable on Monday, May 12 starting at 9pm for approximately 1 hour due to scheduled maintenance. Date: Friday, May 9th, 2008 Impact: Users will be unable to use Connect (formerly Breeze) service. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Connect System Maintenance has been completed successfully.
5/7/2008 11:06:30 AM - NIHCESMLBX Server is Currently in Distress
Description:   Date: May 7th, 2008 Time: 11:00 AM Event Details: NIHCESMLBX Server is currently in distress and will be rebooted to correct the situation. Impacted Services: The email service for the following IC's will be affected by this outage:NCRR, NIBIB, NIGMS, NINR, NHGRI and NEI.
Resolution:  The NIHCESMLBX Server was rebooted and at this time all services are back up and fully functional. If you still have difficulties please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
5/6/2008 11:21:31 AM - Building 12 B12-DC-PIE1 Firewall Maintenance
Description:   Date: Thursday, May 8th, 2008 Time: 6:30am - 7:30am Description: On Thursday, May 8th, 2008 at 6:30am - 7:30am EDT, NIH engineers will apply ruleset changes to the b12-dc-pie1 firewall that will only affect vlans 4 & 6. This change will enhance security for all servers on VLAN 4 and VLAN6 including: kafka.net.nih.gov, list.nih.gov, ns.nih.gov and ntp1.nih.gov. Also, this maintenance will not affect normal email, dns, and time services to those servers. Impact: Please note this Building 12 B12-DC-PIE1 Firewall Maintenance activity has been assessed with a Service Affecting-Low (PSA-Low) impact. Specifically, this maintenance activity will limit unauthorized network traffic to some devices supporting the listserv, NTP, and DNS services. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  This maintenance was completed successfully as scheduled.
5/6/2008 8:12:56 AM - Fernwood-7600-Outside Internet
Description:   Date: Tuesday, May 06, 2008 Time: 7:35 PM There was a problem with the Fernwood-7600-outside due to the Power issue in the Fernwood building. The Primary Internet Path has been moved to alternate Building 12 path to resolve the issue. NIH engineers are currently investigating the issue to ensure stability. Further updates will be provided as they are made available. Impacted Service/Application: Internet Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Issue has been resolved and everything is working now.
5/5/2008 2:59:37 PM - Notification of Foreign Travel Website is Down
Description:   Date: May 5th, 2008 Time:8:00 AM Event Details: At this time the Notification of Foreign Travel website is experiencing connectivity issues. The engineers are aware of the issue and are working towards a resolution. At this time there is no Estimated Time of Resolution. Updates will be provided as they become available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Engineers have resolved this issue, and the Notification of Foreign Travel system is back up and running.
5/2/2008 5:24:28 AM - Reload of B29-4000 Router
Description:   During the early morning of Friday, May 2, 2008, the NIH B29-4000 router experienced a hardware failure. CIT network engineers will conduct an emergency reload to correct the hardware failure. The expected time of resolution is currently unknown. Further updates will be sent as needed. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The device reload was completed at 5:50 AM. The device is now functioning normally at this time. Engineers are investigating the root cause of the issue and will provide updates as they are made available.
5/1/2008 11:23:33 AM - Building 50 Telephone Outage
Description:   There has been a telephone outage reported, with 14 - 15 lines down in Bldg. 50 / 6th Floor. Technicians have been dispatched to the location. Further updates will be provided as they become available. Impacted Users: NIAID Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  All Building 50 telephone lines have been restored.
4/30/2008 3:49:08 PM - AT&T Blackberry Data Connectivity Issues
Description:   Date: Wednesday, April 30th, 2008 Time: 3:45 PM It appears that the AT&T is experiencing a data outage and most users with AT&T cannot receive messages intermittently, if at all. Due to the unofficial nature of the issue there is no Estimated Time for Resolution. Updates will be provided as they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  AT&T Blackberry data connectivity issues have been resolved.
4/30/2008 12:56:47 PM - NAppMan Downtime
Description:   NAppMan will be down from 1:00 - 2:00 PM today Wednesday, April 30th, to implement monitoring changes for the NIDB system. Impacted Service/Application: NAppMan Expected Resolution Time: Date: April 30th 2008 Time: 2:00 PM Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Maintenance has been completed and NAppMan is up and running.
4/30/2008 5:12:32 AM - Building 12 6500- 4A Module is currently down
Description:   The Building 12 6500-4A Module on a distribution router is currently down. Network engineers have been dispatched and are currently working to resolve the issue. Further updates will be posted as soon as they become available.
Resolution:  Engineers have resolved the issue. Service was restored as of 5:30 AM.
4/29/2008 9:40:38 PM - NIHnet CIT Fernwood-6500-C Module-2 is Down
Description:   Due to the NIHnet CIT Fernwood-6500-C Module-2 Replacement Emergency Maintenance, Fernwood-6500-C Module-2 is currently down. Engineers are working to resolve this issue. Further updates will be provided as they are made available. Currently there is no time of resolution for this issue. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The Fernwood-6500-C Module-2 is back online service has been restored.
4/28/2008 2:37:06 PM - NAppMan Application is Experiencing Issues
Description:   The NAppMan application is currently experiencing issues. Engineers are aware of the issue and are working towards a resolution. Updates will be provided as they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Time: 2:54PM The engineers have restored service to the NAppMan application. The engineers will continue to investigate the root cause of the issue.
4/28/2008 8:46:58 AM - Building 12 Distribution Router Module Failure
Description:   Date: Monday, April 28, 2008 Time: 8:11 AM Description: One of the modules on a distribution router in Building 12 failed. Due to the redundancy of the NIH network no users are currently affected. NIHnet engineers are currently working to resolve this issue. Updates will be provided as they become available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  This issues was resolved as of 9:50 AM.
4/26/2008 8:32:33 AM - Phone Outage at the Police Department
Description:   Date: Saturday, April 4th, 2008 Time: 6:45 AM Details: A phone outage has occured at the Police Department on NIH Campus. The Police have put temporary procedures in place to cover the down lines and can still receive calls. A Verizon technician has been called in and will assess the situation. Update will be provided as they become available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  At 9:50 AM the telephone outage at the Police Department has been repaired. All lines are back in service.
4/26/2008 1:36:02 AM - Building 12 Switch Reset
Description:   On Saturday, April 26, a network switch in the Building 12 Data Center experienced a brief reset. All connectivity has been restored at this time. CIT network engineers continue to monitor the situation. Further updates will be sent if needed. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The B12-6500-MR-B was restored to service at 1:14 AM.
4/25/2008 3:30:13 PM - 10-15 service disruption in Sharepoint
Description:   There have been reports of intermittent slowness in connecting to the SharePoint websites. We have investigated and identified the possible culprit for the slowness in page load. Necessary actions have been taken to resolve the issue with load balancers and a more through patch will be to be applied during a later scheduled maintenance window to extensively resolve the problem. Currently all HSB hosted SharePoint Sites are up and the intermittent issue should have been rectified.
Resolution:  There have been reports of intermittent slowness in connecting to the SharePoint websites. We have investigated and identified the possible culprit for the slowness in page load. Necessary actions have been taken to resolve the issue with load balancers and a more through patch will be to be applied during a later scheduled maintenance window to extensively resolve the problem. Currently all HSB hosted SharePoint Sites are up and the intermittent issue should have been rectified.
4/24/2008 3:58:40 PM - Mailbox Server NIHCESMLBX5 is Currently Experiencing Issues
Description:   NIHCESMLBX5 experienced a system crash. The mailbox server is currently not responding. CES Engineers are investigating the issue. Updates will be provided as they are made available. Impacted Service/Application: Mailbox Server Impacted Users: NIAID Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The NIHCESMLBX5 mailbox server has been restored to service. CES Engineers will continue to monitor the server closely.
4/24/2008 3:19:08 PM - ITAS Production Hardware Migration
Description:   ITAS is scheduled for production hardware migration on Tuesday, April 29, starting at 6:00PM EST and ending at approximately 9:00PM EST. During this time, ITAS will NOT be available. We apologize for any inconveniences this may cause you. Impacted Service/Application: ITAS Expected Resolution Time: Date: Tuesday, April 29 2008 Time: 9:00 PM Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  ITAS hardware migration has completed successfully. ITAS is back online.
4/23/2008 4:49:45 PM - Remedy Server restart April 24 12:30AM
Description:   Date: Thursday, April 24th, 2008 Time: 12:30 AM Description: The CIT Customer Support Remedy server will be briefly out of service while being rebooted to correct an issue with email notifications. Outage should not last more than ten minutes. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Remedy server was rebooted and returned to service as of 12:37 AM.
4/21/2008 8:16:56 AM - CIT Print Server Issues
Description:   Date: Monday, April 21st, 2008 Time: 6:00 AM Description: At this time the CIT Print Server is experiencing connectivity issues. Impact: No users within CIT will be able to print on any CITPRINT managed printers. Engineers are aware of the issue and are working towards a resolution. Updates will be provided as they become available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  At 1:11 PM, engineers restored and confirmed the CITPRINT server was restored to service. Users are now able to print using networked printers. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
4/14/2008 1:19:34 PM - Building 50 MDF Refresh Maintenance
Description:   Date: Saturday, April 26, 2008 Time: 8:00am - 5:00pm Description: On Saturday, April 26, 2008 at 8:00am - 5:00pm EDT NIHnet engineers will upgrade the aggregation switches and UPS in Building 50. Impact: Please note that this Building 50 MDF refresh maintenance has been rated with a Service Affecting (SA) impact. Specifically, NHGRI, NHLBI, NIDCD, NIAMS, NCI, OD/ORS, CIT-WLAN, NIDDK, FACnet and the CIT Video in Building 50 will lose network connectivity to NIHnet while the aggregation switches in Building 50 are being upgraded. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  All the customers that have tested at the end of the maintenance have reported that all services are up and running. As of 4:35 PM the maintenance is complete.
4/8/2008 9:41:53 PM - Iforgotmypw Upgrade - Email Delivery Concerns
Description:   As part of todays upgrade to the new NIH Password Self Service product, NIH and HRSA staff that are currently registered for Iforgotmypw had their primary mail field in AD updated with the data they had entered during their initial registration for iforgotmypw. This may have been a problem for some users. Some users may have had email to them bounce back to the sender (or NDR). EMIB has updated all potentially affected users to reset this field to the correct value. It is unknown at this time exactly how many PSS registered users were affected. EMIB staff worked to correct the problem as quickly as possible for all potentially affected users. Anyone experiencing a problem would have had it occur between 4 and 9 pm. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Email Delivery Issues have been resolved.
4/4/2008 1:04:29 PM - Password Self-Service Availability
Description:   Date: Monday, April 7th, 2008 till Wednesday, April 9th, 2008 Time: Starting at 9:00 AM Description: Iforgotmypw.nih.gov or Password Self-Service service will be unavailable starting Monday, April 7th after 9AM until Wednesday, April 9th. This is due to a service upgrade. Impact: During this outage, customers will be unable to use this service to change their passwords or unlock their accounts. Current users of the system include NIH, HRSA and External.nih.gov. In addition, Password.nih.gov will also be affected by this service upgrade, but only intermittently. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Service Upgrade for Iforgotmypw.nih.gov & Password Self-Service has been completed successfully.
4/2/2008 1:07:11 PM - Password.nih.gov Software Upgrade
Description:   Password.nih.gov will be unavailable for the next 30 minutes due to a software upgrade. Customers may experience a brief interruption (5 min or less) during this upgrade. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Software Upgrade Maintenance for Password.nih.gov has now been completed. Password.nih.gov is now available.
4/1/2008 1:09:33 PM - CRC-4500-B2D Power Supply Replacement Maintenance
Description:   Date: Friday, April 11, 2008 Time: 1:00am - 3:00am Description: On Friday, April 11, 2008 at 1:00am - 3:00am ET NIHnet engineers will replace a failed power supply on the CRC-4500-B2D. This device was damaged during a power surge caused by construction workers and is now in a HazCon state. Impact: Please note this CRC-4500-B2D Power Supply Replacement Maintenance activity has been assessed with a Possible Service Affecting-Low (PSA-Low) impact. Specifically, during this maintenance activity, NIHnet access should not be affected as long as the redundant power supply replacement procedure goes as planned. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The CRC-4500-B2D Power Supply Replacement Maintenance was partially completed.
3/26/2008 10:23:34 AM - Microsoft Jet Database Engine Vulnerability
Description:   Due to a recently announced security vulnerability in Microsoft Jet Database Engine affecting users of Microsoft Word running on Windows 2000, XP, and Server 2003 SP1 and which is being actively exploited in targeted attacks, all .MDB file attachments will be blocked at the email gateways effective immediately. Only the attachments will be dropped and users in most cases will receive notification that the file has been removed. Impacted Service/Application: E-mails containing a .MDB file attachment will have the file removed prior to delivery. Impacted Users: Users running Microsoft Word on Windows 2000, XP and Server 2003 SP1. This is an informational Hot News. There will be no further updates. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  This is an informational Hot News only. No further updates will be sent.
3/25/2008 2:09:50 PM - Building 12 Wireless and Wireless Guest Issues
Description:   Currently building 12 users are experiencing Wireless and Wireless Guest Issues. NIHnet engineers are aware of this issue, and working towards a resolution. Impacted Service/Application: Wireless and Wireless Guest Impacted Users: Building 12 Users Expected Resolution Time: Unknown at this time Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Building 12 wireless and wireless guest issues have been resolved.
3/24/2008 8:05:30 AM - NAppMan Implementation of Tibco Maintenance
Description:   On March 24, 2008, between 10:00am - 11:00am, NAppMan engineers will be implementing a change that will represent the Tibco service on the dashboard. Tibco will not have a service status, but it will have a component status. During this implementation NAppMan will not be available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Time: 10:55AM The implementation of Tibco to NAppMan was completed successfully.
3/18/2008 3:28:37 PM - Cisco VPN Client Version 5.x Mandatory Upgrade - MS Windows only
Description:   If you use Microsoft Windows and run Cisco VPN client software to access NIH remotely, please be advised of the following: Due to a security vulnerability, at 8:00 AM Wednesday, March 26, 2008, remote Windows machines will need Version 5.0.01.0600 and above of the Cisco VPN client software in order to login. Any Cisco VPN client version below version 5.0.01.0600 must be upgraded. The upgraded software is available via the iSDP software distribution site: http://isdp.cit.nih.gov/downloads/vpn_tools.asp. Please be sure the existing VPN client version is uninstalled prior to upgrading to the new version. Note: both Windows 2000 and XP require administrator privileges in order to upgrade the software. If you do not have administrator privileges, you will need to obtain assistance in upgrading your VPN software. This cut-off date does NOT apply to you if you are running VPN client software on Linux or Mac OS X. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or http://ithelpdesk.nih.gov if you have any questions.
Resolution:  All Windows versions of the Cisco VPN client must now be version 5.0.01.0600. Please contact the NIH Help Desk if you have issues upgrading your client to the proper version
3/18/2008 2:30:04 PM - Constellation Issues
Description:   Constellation experienced technical difficulties and was not creating accounts earlier today. CIT has resolved the issue and the backlog of transactions should begin to be processed by Constellation. Impacted Service/Application: Constellation Resolution Time: Tuesday, March 18th 2008 at 2:00 PM Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  This issue was resolved as of 2:00 PM on March, 18th 2008.
3/18/2008 1:43:02 PM - NEES Production Server Downtime
Description:   At 4:00pm today, March 18, 2008, the NEES production server will be taken out of service to apply a new minor release. The system is expected to return to normal service before 4:30pm this afternoon. Any employees who have filed or need to file a request for "Approval of Outside Activity" as well as the NIH Ethics community will be affected during this service outage. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  NEES Production Server is back in service. Updates were made successfully.
3/17/2008 7:48:19 AM - ORS Network Services Outage
Description:   Buildings 14, 28, and 28A are currently experiencing a network outage. Impacted Service/Application: All network resources are being affected. Impacted Users: OD/ORS users from Buildings 14, 28 and 28A are experiencing loss of connectivity with their network resources. Expected Resolution Time: At this time there is no Expected Resolution Time. Updates will be provided as they become available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Network connectivity has been fully restored. Users may need to reboot their computers to regain network access. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
3/14/2008 2:57:15 PM - NIHnet CWDM Ring I - Core Network 10GigE Upgrade Maintenance
Description:   As a follow-up to the March 8-9, 2008 NIHnet Bandwidth Upgrade in North Bethesda, Rockville, & Gaithersburg maintenance activity, it is necessary for NIHnet engineers to install and activate a 10 Gigabit network connection between the Executive Blvd Bunker-6500-DR router and the B12-7600-Core router. Impacted Service/Application: No Services/Applications are expected to be impacted by this maintenance. Impacted Users: No users are expected to be impacted by this maintenance. Expected Resolution Time: Thursday, March 20, 2008 at 6:00 PM Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Maintenance has been successfully completed as of Tuesday, March 20th 2008 at 5:33 PM. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
3/14/2008 12:44:32 PM - PETRI Server Outage
Description:   On Friday, March 21st beginning at 10:00PM, during the scheduled monthly maintenance window, the PETRI server will be shutdown and replaced with new hardware by CIT DCSS Engineers. All users of the Petri server will be affected and will not be able to connect to the server during the maintenance window. As this server provides home directories containing Outlook PST (email archive) files for most CIT staff members, we are kindly asking that all CIT staff exit their email prior to departing for the weekend. CIT DCSS staff anticipates the duration of this outage to be 4 hours or less. Impacted Service/Application: Petri Impacted Users: All CIT users of the Petri server Expected Resolution Time: March 22, 2008 at 2:00 AM Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The maintenance on the PETRI server has been completed successfully.
3/12/2008 8:42:12 AM - Changes to CIT Ticketing System: 3/12/2008 and 3/15/2008
Description:   CIT Support Technologies is preparing to implement a significant enhancement to the CIT Service Ticket in the Remedy Ticketing system. This change will allow for special customer roles and properties to be associated with each customer and is a significant improvement over the current jumble of radio buttons on the "Roles" tab. In preparation for this enhancement STG will be locking down all role buttons on the Customer Database this evening, Wednesday, March 12th. The balance of the user-impacting work will be performed on Saturday, March 15th. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  This maintenance has been completed successfully without any complications. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
3/11/2008 3:55:16 PM - NIHnet Internet Building 12 DMZ Firewall Hardware Upgrade Maint.
Description:   Date: Tuesday, March 18, 2008 Time: 6:00pm - 10:00pm Description: On Tuesday, March 18, 2008 at 6:00 p.m. - 10:00 p.m. ET NIHnet engineers will perform a Scheduled Maintenance activity on the NIHnet Internet DMZ Building 12 path to upgrade and optimize NIHnet Internet DMZ Firewall infrastructure to the next generation hardware platform. Impact: Please note this NIHnet Internet Building 12 DMZ Firewall Hardware Upgrade Maintenance activity has been assessed with a Service Affecting (SA) impact. Specifically, there will be a maximum of five expected NIHnet Internet traffic flow disruptions during the first three hours of this four hour maintenance window. Each NIHnet Internet traffic disruption is expected to last less than five minutes in duration as NIHnet Internet traffic is moved over to the new firewall infrastructure in the Building 12 Internet DMZ path and tested for performance and reliability. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  NIHnet Internet Building 12 DMZ Firewall Hardware Upgrade Maintenance was completed successfully.
3/10/2008 9:25:40 AM - BlackBerry Send Issues
Description:   Some NIH Blackberry users may report a problem sending emails, while receiving is working normally. NIH CES engineers have identified the problem and it is being corrected. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  NIH CES engineers have identified the problem and it has been corrected. All devices should be functioning normally at this time. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
3/7/2008 2:51:14 PM - Building 12 Remote Access Implementation Phase II
Description:   On Tuesday, March 11, 2008 at 5:00pm - 11:00pm ET NIHnet engineers will perform Phase II of the hardware upgrade of the Building 12 Remote Access Implementation. Impacted Service/Application: Please note this Building 12 Remote Access Implementation Phase II activity has been assessed with a Possible Service Affecting-Low (PSA-Low) impact. Specifically, during this maintenance activity, all NIHnet remote access traffic will automatically and seamlessly balance between the Building 12 and Fernwood remote access devices. Therefore, there is a very low probability of a NIHnet Remote Access service disruption due to the redundant remote access design Expected Resolution Time: Date: Tuesday, March 11, 2008 Time: 11:00pm Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Time: 9:17PM The maintenance was completed successfully without complications.
3/6/2008 1:01:54 PM - CRC B2B LAN Closet Power Issues
Description:   Currently, there is a power issue in the Clinical Research Center B2B LAN closet. The issue was impacting a switch that supports network connectivity for NCI users. NIHnet and facility engineers have been dispatched and are working to eliminate any additional power issues that may exist. Impacted Users: NCI Expected Resolution Time: The impacted NCI switch was restored to service at 12:55 PM. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  This is an initial and final message.
3/4/2008 10:17:18 AM - Issues Receiving Emails From External Hosts
Description:   Currently there are issues with NIH customers receiving emails from external hosts outside of NIH. CIT engineers are investigating the issue. Updates will be provided as they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Issue has been resolved.
3/3/2008 5:52:23 PM - PKI Website Issues
Description:   Currently there is a server outage impacting the HHS PKI website (https://www.pki.hhs.gov/hhspki/home/). Users are receiving error messages when attempting to get new certificates. The HHS PKI team is aware of the issue and is working to resolve the outage. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  As of 12:30 PM, system engineers have restored the PKI service to operational status. Users should now be able to successfully create new PKI certificates. If you have any questions, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
3/3/2008 3:17:32 PM - Wireless Guest Administration Maintenance
Description:   On Tuesday, March 4 from 6:00pm - 6:30pm ET, the NIH Wireless Guest Access Management System (WLGAMS) will be offline for routine database maintenance. During the 30-minute maintenance period, account creators will not be able to activate new wireless guest accounts. Please note that this maintenance will have no impact on users of the Wireless Guest service, and users with existing accounts will be able to authenticate normally. The maintenance only affects the administrative application for activating guest accounts. If you have any questions, please contact the NIH Help Desk at 301-496-4357 or visit us on the web at http://support.cit.nih.gov.
Resolution:  Maintenance was completed successfully.
2/28/2008 5:48:05 PM - ITAS Scheduled Maintenance
Description:   ITAS is scheduled for maintenance on March 4, Tuesday evening between 10:00PM - 10:30PM for production passwords changes. During the maintenance, ITAS will not be available. We apologize for any inconveniences this may cause you. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The ITAS maintenance has completed successfully.
2/28/2008 3:43:44 PM - Constellation Sync Process
Description:   Event Details/Description: March 2nd, 2008 constellation will initiate a sync process with NCI. This will start at 4:00 PM and expected end time should be between 6-8 hours later. Constellation will continue to run, but the synchronization with AD will be slow. Impacted Service/Application: Constellation Impacted Users: NCI Expected Resolution Time: Between 10:00PM - 12:00 PM Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The Constellation Sync Process was completed at 8:45 AM on Monday, March 3rd, 2008. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
2/28/2008 10:46:27 AM - NIHCESMLBX Server Maintenance
Description:   On February 28, 2008, beginning at 10:00pm, engineers will be repairing one of the redundant paths on the NIHCESMLBX server's SAN connection. During that time any ICs that have mailboxes on that server will be unable to reach their emails for approximately 45 minutes. The following ICs will be affected during the maintenance window: NIAAA, NIGMS, NBS, NEI, NHGRI, NIDA, and ORS. NIAAA has less than 50 mailboxes on the server being repaired. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov if you have any questions.
Resolution:  The server maintenance has been successfully completed.
2/27/2008 7:41:28 AM - NBS Production Servers are Down
Description:   At this time all NBS Production Servers are down. Impacted Service/Application: All NBS Applications are down and customers will not be able to access them. The NMC engineers are aware of the situation and are working towards a resolution. There is no Estimated Time of Resolution and updates will be posted as they become available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  At 10:05 AM, the NBS system was verified as operational. Users can now access the NBS system.
2/26/2008 4:59:01 PM - ITAS Emergency Maintenance
Description:   There will be an ITAS Emergency Maintenance this evening, Feb 26 between 6:00PM - 8:00PM due to timecards for next pay period (PP-6) was erroneously generated. During the maintenance, ITAS will not be available. Any ITAS activities (such as "Verify Timecard", "Request Leave", "Approve Leave", "Approved Timecard", and/or "Timecard action" etc..) that were performed by users after 10:00AM this morning (Feb 26) is required to be re-entered in ITAS after the ITAS maintenance is completed this evening. We apologize for any inconveniences this may cause you. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The maintenance has been completed successfully. The ITAS records have been restored as of 10:00 AM this morning.
2/25/2008 3:23:22 PM - Connect Server Database Maintenance
Description:   Tonight, February, 25th, 2008 between 8:00 PM and 10:00 PM the Connect Server will be down for emergency database maintenance. Impacted Service/Application: The Connect Server will be inaccessible to the customers for the duration of the maintenance. Expected Resolution Time: This maintenance should be completed no later than 10:00 PM. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov if you have any questions.
Resolution:  Connect server database maintenance was completed successfully.
2/25/2008 2:41:45 PM - Building 12 Core Router Hardware Upgrade Maintenance
Description:   Date: Saturday, March 1, 2008 Time: 8:00am - 12:00pm On Saturday, March 1, 2008 at 8:00am - 12:00pm ET NIHnet engineers will perform a hardware upgrade of the NIHnet Building 12 Core Router in support of the NIHnet Core Network technology refresh project. Impact: Please note this Building 12 NIHnet Core Router Hardware Upgrade Maintenance activity has been assessed with a Possible Service Affecting-Low (PSA-Low) service impact. Specifically, all NIHnet data network traffic will automatically and seamlessly failover to the fully redundant Building 10 Core Router data network path when the Building 12 Core Router is powered down for hardware replacement. Therefore, there is a very low probability for NIHnet data network service disruption due to the existing fully redundant Core Router network design on NIHnet. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  This maintenance has now been completed, with some complications. There was a loss of connectivity to the NIHnet between 8:00 AM and 8:20 AM. Additionally, there was a loss of internet connectivity between 11:00 AM and 11:40 AM. There is no root cause as of yet, but further updates will be provided as they are made available. If you have any questions, please contact the NIH Help Desk at 301-496-4357 or visit us on the web at http://support.cit.nih.gov.
2/23/2008 10:02:30 PM - HRSA Mail Server NIHHRSAMLBX Connectivity Issues
Description:   The HRSA mail server NIHHRSAMLBX is currently experiencing some technical issues, and engineers are working on a solution. This will affect HRSA users' ability to access e-mail. A resolution is expected shortly, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov. Further updates will be provided as soon as they become available.
Resolution:  The NIHHRSAMLBX server has been restored to service.
2/22/2008 2:33:51 PM - Fibergate Off-Campus Fiber Relocation Maintenance - Phase II
Description:   On Friday, February 29, 2008 beginning at 10:00 p.m. and ending on Saturday, March 1, 2008 at 6:00 a.m., Fibergate will perform a scheduled maintenance to relocate 8 fiber pairs in support of a road widening project at Route 355 and Randolph Road in Rockville, MD. Impact: Please note this Fibergate Off-Campus Fiber Relocation Maintenance activity is Phase II of a three phase road widening project that has been given a Service Affecting (SA) impact assessment for NIHnet connectivity to the NIH off-campus locations at 9800 Medical Center Dr, 5 Research Ct, 6700B Rockledge, and 5635 Fishers Ln (NHLBI LOBOS connection to Building 12). These 4 NIH off-campus sites will experience minimal network connectivity disruption to NIHnet of less than 15 minutes while each fiber pair is re-spliced onto a new path. Additionally, this Fibergate Off-Campus Fiber Relocation Maintenance activity has also been given a Possible Service Affecting Low (PSA-Low) impact assessment for NIHnet connectivity to the NIH off-campus locations at 6500 Fishers Ln (Parklawn), 5625 Fishers Ln, 5635 Fishers Ln, 6000 Executive Blvd (EB Bunker I), and the Fernwood building. Specifically, NIHnet data network connectivity for these NIH off-campus locations will automatically failover to redundant network paths provided the fiber service provider maintains the correct sequence order for fiber pair relocation. Please contact the NIH Help Desk at 301-496-HELP, or visit us at http://support.cit.nih.gov.
Resolution:  The Fibergate Off-Campus Fiber Relocation Maintenance was successfully completed.
2/20/2008 2:03:34 PM - NAppMan Downtime for Patching
Description:   To accommodate necessary patches to the Solaris system, the NAppMan application will be taken down for maintenance on 21st of February, 2008 between 8:15 AM and 11:30 AM. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov if you have any questions.
Resolution:  Time: 12:49PM The patches for NAppMan were installed successfully and the application has returned to normal service.
2/20/2008 11:39:59 AM - Alert Messages
Description:   Due to testing of the Password Self Service system, some alerts are being sent to users from the following email address: dogaimspss@mailtest.nih.gov dogpss@mailtest.nih.gov dogqpmpss@mail.nih.gov The alerting mechanism is being disabled at this time. Please ignore any of these messages. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov if you have any questions.
Resolution:  Time: 11:59AM The issue has been resolved since the notifications were turned off yesterday afternoon, February 20, 2008, and no additional problems have been reported since that time.
2/20/2008 8:28:52 AM - CIT Remedy Server Restart - 8:45 AM
Description:   The CIT Remedy Server will be restarted at 8:45 AM to address a minor issue with the service. This will result in an anticipated downtime of approximately ten minutes. We apologize for the inconvenience and appreciate your patience. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The reboot was successful and the server is back up. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
2/19/2008 3:38:43 PM - PETRI Server Emergency Maintenance
Description:   PETRI server will be rebooted tonight at 6pm for emergency maintenance. Scheduled downtime should be no more than 15 minutes. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  PETRI server emergency maintenance was completed successfully.
2/19/2008 12:36:30 PM - Petri Server Maintenance Reminder
Description:   On Friday, February 22, 2008 starting at 6:00 PM, the Petri server will be undergoing an upgrade maintenance. Petri will NOT be available to end users during the maintenance. Engineers will be upgrading the storage of the server to facilitate increased storage requirements. We request that all users log off their Outlook clients and close all their PST files before leaving their workstations on February 22nd. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  The maintenance on the Petri server has been successfully completed.
2/16/2008 11:50:55 AM - Clinical Center Generator Test
Description:   The Emergency Power Generator Test in the Clinical Center started at 10:30 AM and it was completed as scheduled at 11:30 AM. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov if you have any questions.
Resolution:  Maintenance complete.
2/15/2008 12:24:31 PM - Building 10 Core Router Hardware Upgrade Maintenance - Phase II
Description:   On Sunday, February 24, 2008 at 8:00am - 12:00pm ET NIHnet engineers will perform the Phase II hardware upgrade of the NIHnet Building 10 Core Router technology refresh project. Impact: Please note this Building 10 NIHnet Core Router Hardware Upgrade Maintenance - Phase II activity has been assessed with a Possible Service Affecting-Low (PSA-Low) impact. Specifically, all NIHnet data network traffic will automatically and seamlessly failover to the fully redundant Building 12 Core Router data network path when the Building 10 Core Router is powered down for hardware replacement. Therefore, there is a very low probability for NIHnet data network service disruption due to the existing fully redundant Core Router network design on NIHnet. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Time: 11:15AM The maintenance was completed successfully without complications.
2/14/2008 9:01:43 AM - Building 12 Data Center Circuit Issue
Description:   Currently we are experiencing multiple circuit issues for circuits originating from Building 12 Data Center. Engineers are on-site and currently working to reslove this problem. Further updates will be provided as they are made available. The T1 circuits to the following locations are impacted: Poolesville Rockwall Phoenix Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  At 9:49 AM, service was restored to all affected connections.
2/14/2008 8:13:13 AM - CIT Customer Support Report Services Outage
Description:   The server that provides the CIT Customer Support report services is currently experiencing distress. This is primarily impacting the overnight and morning Customer Support reports (which did not run), along with some internal report websites. In order to resolve the issue, additional troubleshooting and work on the server may be necessary, so other services delivered by the server may be impacted. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or visit us on the web at http://support.cit.nih.gov.
Resolution:  Time: 9:39AM The CIT Customer Support Report Server is back up and reporting services have been restored to normal. Overnight and morning reports that did not run are actively being re-queued and will be delivered as they complete. Thanks for your patience and cooperation while we worked to resolve this issue.
2/13/2008 2:12:35 PM - Building 12 Data Center Circuit Issue
Description:   Currently we are experiencing multiple circuit issues for circuits originating from Building 12 Data Center. Engineers have been dispatched to investigate this issue. Further updates will be provided as they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  All services have now been restored.
2/13/2008 6:56:34 AM - 3206 Tower Oak Blvd Circuits are Down
Description:   3206 Tower Oak Blvd circuits are down possibly affecting network connectivity. Verizon is investigating this issue; updates will be posted as soon as they become available. Currently there is no estimated time of resolution.
Resolution:  As of 2:00 PM, the power to 3206 Tower Oak Blvd was restored.
2/13/2008 5:35:40 AM - 9650 Wisconsin Ave Power Outage
Description:   Currently there is a power outage at 9650 Wisconsin Ave. Pepco engineers are working to resolve this issue. There is no estimated time of resolution.
Resolution:  Power has been restored.
2/11/2008 5:04:24 PM - BlackBerry Outage
Description:   There is currently an outage on part of the BlackBerry Network Infrastructure. BlackBerry subscribers may be unable to send or receive messages. Subscribers may also be unable to register their device, roam in another location, or use other services such as Internet browsing. There is no estimated time of resolution at this time. Further updates will be provided as they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The nationwide Blackberry service interruption resolved Monday evening at or around 7:45 PM EST. NIH message activity returned to normal by 9:00 PM EST. Please report any additional or residual problems to the NIH Help Desk at http://support.cit.nih.gov/.
2/11/2008 10:36:04 AM - OD H Drive and Home Servers are down
Description:   At this time the Home Server in Executive Boulevard is experiencing connectivity issues. The engineers are aware of the situation and are working towards a resolution. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Time: 11:06 AM The OD H drive is back up and fully functional. All users should be able to reconnect to their home drives. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
2/8/2008 9:07:32 AM - NIH Application Manager (NAppMan) down for scheduled maintenance.
Description:   The NAppMan monitoring application will be unavailabe from 10:00 AM this morning until 12:00 noon. The NAppMan team will use this time to prepare for an implementation of NBS components on the dashboard. An outage entry has already been provided in the Remedy Outage Calendar. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  NAppMan scheduled maintenance is complete. The dashboard is available and reliable.
2/7/2008 11:14:49 AM - Issues Receiving Inbound Phone Calls into NIH
Description:   Currently we are experiencing problems receiving inbound calls to NIH. Technicians are aware of the issue and have been dispatched. There is no estimated time of resolution. Further updates will be provided as they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Issue is related to HN07080. Please see ticket ST2996839 for further details.
2/7/2008 7:33:13 AM - Intermittent Power Outages Affecting Several Campus Buildings
Description:   There are electrical issues affecting buildings 7, 12A, 12B, 31, 45, and 50 which may have been caused by power surge from the local power plant. Other campus buildings may have been affected. Engineers are aware of the issue and are working towards a resolution. Updates will be provided as additional information becomes available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Time: 9:50AM All power has been restored to the affected buildings and all network devices on back online and fully operational.
2/6/2008 6:08:31 PM - Fibergate Off-Campus Fiber Relocation Maintenance - Phase I
Description:   On Friday, February 15, 2008 beginning at 10:00 p.m. and ending on Saturday, February 16, 2008 at 6:00 a.m., Fibergate will perform a scheduled maintenance to perform a forced relocation of 144 backbone fibers in support of a road widening project at Route 355 and Randolph Road in Rockville, MD. Please note this Fibergate Off-Campus Fiber Relocation Maintenance activity is Phase I of a three phase road widening project that has been assessed with a Service Affecting (SA) impact for 2115 East Jefferson and 1 Choke Cherry connectivity to NIHnet. These two off campus sites will experience minimal network disruption of less than 15 minutes while each fiber pair is re-spliced onto a new path. Additionally, 6001 and 6101 Executive Blvd and 5600 Fisher Lane have been assessed with a PSA-High (PSA-High) impact. These NIH off campus locations will failover to redundant network paths to NIHnet provided the fiber service provider (Fibergate) maintains the correct sequence order for fiber pair relocation. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Fibergate Off-Campus Fiber Relocation Maintenance was successfully completed as scheduled.
2/5/2008 5:34:19 PM - Cardozo Clinic Network Outage
Description:   The T1 from Building 10 to Cardozo Clinic is currently down. Verizon has stated that there is a hardware failure between their facility in Building 10 and their Central Office in Silver Spring. Verizon engineers are working on replacing the defective part of the facility. Further updates will be provided once they made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  As of February 6th, 2008 at 3:36 PM the T1 circuit that connects Cardozo clinic to the NIHnet is back up and stable. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
2/5/2008 4:52:59 PM - ADM Maintenance
Description:   Tonight at 8:00 PM, EMIB will be performing a brief maintenance on the following service. NIHADM2 will undergo multiple reboots. You may be disconnected from NIHADM2 during this work but will be able to reconnect to the redundant server NIHADM. NIHADM will be up and available the entire time. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  EMIB maintenance was completed successfully. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
2/5/2008 1:40:36 PM - EES/HHS Users Receiving Error Messages When Sending Email
Description:   Currently EES/HHS users are experiencing difficulties with sending emails to NIH users. EES/HHS engineers are aware of the issue and are working towards a resolutions. Updates will be provided as they become available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The mail flow issue with HHS and EES was resolved as of February 6th, 2008 at 4:15 PM.
2/5/2008 12:03:58 PM - Connect System Maintenance
Description:   The Connect system will be down on Monday, February 11, 2008 in order to install a patch. The system will be unavailable from 9pm-10pm. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Connect System Maintenance was completed successfully. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
2/4/2008 12:42:31 PM - NIH-Wide Telephone Service Issues
Description:   Currently users are experiencing intermittent problems calling into NIH, inbound and outbound calls are both reportedly experiencing static conditions. Engineers have been dispatched and are currently working to resolve this matter. Further updates will be provided as they become available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Verizon has completed hardware repairs and service has been fully restored.
2/2/2008 12:06:34 PM - REBOOT OF ODSQLE (Windows and SQL Server ODSQLE)
Description:   The ODSQLE windows box and the ODSQLE Services will be restarted on Saturday February 2nd, 2008 between 8:30 PM and 8:45 PM. This is required for an Emergency Issue were experienced on Friday, February 1st, 2008. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The reboot was successfully completed and all services have returned.
2/1/2008 2:13:39 PM - Connectivity/Power Issues at 6011 Executive Blvd.,
Description:   The OD Network at 6011 Executive Blvd. is experiencing sporadic power outages. Technicians are investigating the problem; however, they believe the problem is isolated to 6011. Some staff are without the use of phones, lights, and computers.
Resolution:  All services were restored as of 11:54 AM on Saturday 02/02/08. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
1/31/2008 9:51:05 AM - Maintenance on NIH-Wide Public Folder Access Server
Description:   EMIB Facilities needs to perform a necessary maintenance on NIHCESPUB3 tonight. This maintenance will cause a service affecting outage for Public Folders on that server for approximately 15 minutes while the server reboots. This maintenance will begin at 10:00 pm and will be completed by 10:15 pm. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The maintenance was successfully completed.
1/30/2008 8:02:34 AM - Possible Phone and Power Outage in Building 1
Description:   Due to constuction outside the perimeter of the Building 1 some utility cables were cut. This may potentialy affect phone connectivity and power. The technicians are on site and are working towards a resolution. At this time there is no Estimated Time of Repair. Updates will be posted as soon as they become available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The technicians have successfully restored all telephone services. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
1/29/2008 1:06:25 PM - NIHCESPUB5 and NIHCESPUB7 Emergency Server Reboot
Description:   EMiB Facilities needs to perform a necessary maintenance on NIHCESPUB5 and NIHCESPUB7. This maintenance will cause a service affecting outage for Public Folders on those servers for approximately 15 minutes while the servers reboot. This maintenance will begin at 1:15 pm and will be completed by 2:15 pm. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The emergency server reboot and maintenance was completed on 1/29/2008.
1/28/2008 10:30:54 PM - NIH Wide Telephone Outage
Description:   Currently users are experiencing intermittent problems calling into NIH . Engineers have been dispatched and are currently working to resolve this matter . Further updates will be provided as they become available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Telephone service has been restored.
1/28/2008 11:54:01 AM - Commons and IMPACII Applications are down.
Description:   IMPPRD is down. Operations is working on this issue.
Resolution:  Between 11:40 AM and 11:50 PM the IMPPRD has experienced some connectivity issues and as a result the Commons and IMPACII Applications were down. At this time all eRA applications are up and running within normal parameters.
1/25/2008 4:04:25 PM - ADM Service Disruption
Description:   Tonight Friday, January 25th, 2008 starting at 10pm EMIB will be performing a brief maintenance on the ADM system. Both nihadm and nihadm2 will undergo multiple reboots. You may disconnect during this work but should be able to reconnect to the redundant server. EMIB will attempt to keep one of the redundant systems up at all times. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The ADM system maintenance was successfully completed.
1/19/2008 8:41:27 AM - Building 10 Generator Test
Description:   Currently, an Emergency Generator Test in Bldg 10 and the ACRF has started. At this time, only systems on Emergency Power should be affected. Normal Power should not be affected. Technicians are on site to monitor any telephone outages and walk thru the areas to check for service. Further updates will be provided once they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The testing is completed. All systems are up and functional.
1/17/2008 10:55:35 AM - NED to Active Directory Connector Issues
Description:   We are currently experiencing problems with the NED to Active Directory (AD) connector. This is preventing NED changes from being propagated to AD. For those ICs using Constellation, this will delay account creations. Further updates will be provided as they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  NED to Active Directory Connector issues have been resolved. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
1/16/2008 3:05:06 PM - New Password Policy
Description:   On January 21st, NIH's password policy will be decreasing from 90 to 80 days for password age. January 28th the password age will drop from 80 to 70 days. February 4th the password age will drop one more time, from 70 to 60 days, and the unique passwords will go from 10 to 24. EMIB will be amending the password notification starting January 21st reminding NIH staff to change their password immediately upon receipt due to the step down process. If you have any problems or need help, please contact the NIH Help Desk at 301.496.4357 or 866.319.4357 toll free, e-mail nihhelpdesk@mail.nih.gov
Resolution:  CIT has completed the transition to the new password policy.
1/16/2008 2:21:24 PM - NIHnet Internet DMZ Firewall Hardware Upgrade Maint. - Fernwood
Description:   Date: Tuesday, January 22, 2008 Time: 6:00pm - 10:00pm ET Description: On Tuesday, January 22, 2008 between 6:00pm - 10:00pm ET NIHnet engineers will perform a Scheduled Maintenance on the NIHnet Internet DMZ Fernwood Path to upgrade the existing firewall hardware to the next generation hardware to address current load issues. Impact: Please note this NIHnet Internet DMZ Firewall Hardware Upgrade Maintenance - Fernwood Path activity has been assessed with a Possible Service Affecting - Low (PSA-Low) impact. Specifically, during this maintenance activity, all service work will be performed on the Fernwood path while traffic transverses through the redundant Building 12 path. Once the maintenance is complete, traffic will be rerouted back through the Fernwood path. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The maintenance was successfully completed at 8:17 PM on Tuesday, January 22, 2008. There were no reported impacted devices.
1/16/2008 11:58:37 AM - Internet Connection Latency
Description:   At 11:40 AM, NIH Users experienced high latency when attempting to browse external websites. Engineers have corrected the issue and internet browsing is now operational. If you have any problems or need help, please contact the NIH Help Desk at 301.496.4357 or 866.319.4357 toll free, e-mail nihhelpdesk@mail.nih.gov
Resolution:  Issue has been resolved.
1/9/2008 5:24:46 PM - SoftMed Downtime 1/9/08 4-7pm
Description:   SoftMed Users: All SoftMed applications will be unavailable today 1/9/08 from 4:00 to 7:00 pm for a software update. The dictation system will NOT be affected. Some of the applications that will be unavailable include: ESA ChartScript ChartLocator ChartScan ChartFact Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The downtime of the SoftMed applications has been completed as of 10:52 PM. All SoftMed applications should now be available.
1/8/2008 5:31:19 PM - Changes to CIT Ticketing System: 1/9/2008 and 1/12/2008
Description:   CIT Support Technologies is preparing to implement a significant enhancement to the CIT Service Ticket in the Remedy Ticketing system. This change will allow for special customer roles and properties to be associated with each customer and is a significant improvement over the current jumble of radio buttons on the "Roles" tab. In preparation for this enhancement STG will be locking down all role buttons on the Customer Database on Wednesday evening, January 9th. The balance of the user-impacting work will be performed on Saturday, January 12th. Further information and documentation on this change will be sent out as an update to this Hot News tomorrow morning (Wednesday January 9th). Thank you for your patience and support as we work to improve the ticketing system. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The maintence period will be rescheduled to occur in March to implement the planned changes. As we get closer to the rescheduled maintenance window, a new Hot News will be released. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
1/8/2008 4:19:56 PM - ISDN Line Down at 6116 Executive Blvd
Description:   There is an ISDN line that is down at 6116 Executive Boulevard suite 504 room 5034. A tech has been dispatched to this location and is working on fixing it. As of now there is no set time for resolution. Updates will be provided as they become available. If you have any problems or need help, please contact the NIH Help Desk at 301.496.4357 or 866.319.4357 toll free, e-mail nihhelpdesk@mail.nih.gov
Resolution:  This issue has been resolved at 2:43 PM. If you have any further problems or need help, please don't hesitate to contact the NIH Help Desk at 301.496.4357 or 866.319.4357 toll free, e-mail nihhelpdesk@mail.nih.gov.
1/7/2008 7:29:22 PM - ITAS Scheduled Maintenance (Jan 8, 2008 after 10:00PM)
Description:   ITAS is scheduled for changing production password tomorrow evening, Tuesday, Jan 8 after 10:00PM (passing the ITAS extended operation hours). The change requires about 10-15 minutes. During that time, ITAS will not be available. If you have any problems or need help, please contact the NIH Help Desk at 301.496.4357 or 866.319.4357 toll free, e-mail nihhelpdesk@mail.nih.gov
Resolution:  The ITAS Scheduled Maintenance is now complete and ITAS is now available.
12/31/2007 4:44:43 PM - Videocast Service is Down
Description:   At this time the NIHVIDEO3 Server is experiencing connectivity issues. The NIH engineers are aware of the situation and are working towards a resolution. There is no Estimated Time of Resolution. Updates will be posted as they become available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  At this time the Videocast service has been restored. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
12/31/2007 7:36:11 AM - Power Outage in Buildings 29 & 29A
Description:   There are currently power outages in campus Buildings 9 and 29A. The network in these buildings are affected. Currently there is no estimated time of resolution. More information will be posted as soon as it becomes available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The power was restored to the mainframe at 12:15 PM and the walkthrough was completed at 2:49 PM. All systems are up and functional.
12/29/2007 10:08:34 AM - Customers Receiving False Password Reset E-mails
Description:   Customers have contacted the NIH Help Desk in regards to an e-mail received stating that their passwords were set to expire though the customer had recently changed their passwords.
Resolution:  Time: 11:22AM The error that caused customers to receive false notifications about their passwords expiring has been corrected.
12/26/2007 9:22:35 AM - Phone Outage in 6707 Democracy Blvd on the 6th & 7th Floors
Description:   A phone outage has been reported in 6707 Democracy Blvd. affecting the 6th & 7th floors. The technician on site has reported that there is currently no power in the phone closets. The technician is contacting building maintenance to look into the issue. There is no estimated time of resolution. Updates will be made as they become available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The power supply was replaced at 1:52 PM and the phone service was restored shortly after. All services are now available and fully functional. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
12/24/2007 4:55:12 AM - Democracy II (6707 Democracy Blvd)- Power Outage
Description:   Democracy II building experienced a power outage this morning. Pepco was notified and the power has since been restored. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Power was restored at 3:54 AM.
12/21/2007 2:03:17 PM - Petri Rebooted
Description:   On December 21, 2007 at 1:46 PM, the Petri server rebooted itself and was restored to service at 1:55 PM. EMIB is investigating the cause of the issue. This is an initial and final notification of the event.
Resolution:  The initial notification of the initial and final Hot News had the incorrect date. The event occured on Friday, December 21. Please see below for the corrected version of the Hot News notification.
12/21/2007 12:39:38 PM - Constellation Maintenance
Description:   The Constellation Developers will be performing Constellation maintenance starting 12/26/07 and ending 12/29/07. They will be doing Constellation full syncing per participating IC OU. Full logging will be deployed to catch for duplicate AD account creation or potential AD account deletion. If you have any problems or need help, please contact the NIH Help Desk at 301.496.4357 or 866.319.4357 toll free, e-mail nihhelpdesk@mail.nih.gov
Resolution:  Maintenance was completed without complications.
12/17/2007 3:36:46 PM - Mailfwd.nih.gov is currently experiencing a delays
Description:   Mailfwd.nih.gov is currently experiencing a delay in delivering e-mail. NIH CES Operations is aware of the issue and working to clear the delay as quickly as possible. There is no Estimated Time of Resolution at this time. Updates will be provided as they become available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  This issue was resolved.
12/17/2007 2:03:43 PM - Building 10 NIHnet Core Router Hardware Upgrade Maintenance
Description:   Time: 9:00am - 12:00pm Description: On Saturday, December 22, 2007 at 9:00am - 12:00pm ET NIHnet engineers will perform the Phase I hardware upgrade of the NIHnet Building 10 Core Router technology refresh project. Impact: Please note this Building 10 NIHnet Core Router Hardware Upgrade Maintenance - Phase I activity has been assessed with a Possible Service Affecting-Low (PSA-Low) impact. Specifically, all NIHnet data network traffic will automatically and seamlessly failover to the fully redundant Building 12 Core Router data network path when the Building 10 Core Router is powered down for hardware replacement. Therefore, there is a very low probability for NIHnet data network service disruption due to the existing fully redundant Core Router network design on NIHnet. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The maintenance was completed at 11:36 AM. There was a complication with the maintenance that was resolved. All devices are now operating normally.
12/16/2007 10:53:31 PM - Building 82 Power Outage
Description:   Currently Building 82 is without power. Per Pepco estimated time of power restoration is 12/17/07 at 1:00 AM If you have any problems or need help, please contact the NIH Help Desk at 301.496.4357 or 866.319.4357 toll free, e-mail nihhelpdesk@mail.nih.gov.
Resolution:  Power has been restored.
12/16/2007 8:34:32 PM - Power Outage On Executive Boulevard
Description:   All the Buildings on Executive Boulevard are currently without power. Central call desk has contacted the Executive Boulevard's building maintenance to report this issue. Currently there is no time of resolution. Further updates will be provided as they become available. If you have any problems or need help, please contact the NIH Help Desk at 301.496.4357 or 866.319.4357 toll free, e-mail nihhelpdesk@mail.nih.gov
Resolution:  Power has been restored.
12/14/2007 8:23:12 PM - NIH Telephone Operator Services Connectivity Issues
Description:   At this time the NIH Telephone Operator Services main telephone number 301-496-4000 is inoperable. Verizon technicians were informed and dispatched to our location in order to resolve the issue. There is no Estimated Time of Resolution. Updates will be provided as they become available.
Resolution:  NIH Telephone Operator Services main telephone number 301-496-4000 is now operational. Technicians were able to restore service and are now researching the root cause of this trouble.
12/14/2007 9:52:33 AM - Merlin Telephone Outage at 5 Research Court
Description:   There is currently a Merlin telephone outage at 5 Research Court. Root cause was due to a power outage. Telephone service is affected on the lower lever, approximately 50 telephone stations. A technician is on-site to restore service. The affected IC's are: NCI NIDCD NHLBI OD/ORF and ORS NIAMS Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Merlin Telephone Service has been restored on the Lower Level of 5 Research Court. The outage was caused by a commercial power interruption. Technicians restored dial tone at approximately 10:30am this morning.
12/14/2007 8:49:29 AM - NIHnet OD Network Connectivity Issues
Description:   At this time OD users on the NIH Campus are currently experiencing network connectivity issues. The NIHnet engineers are aware of the issue and are working towards a resolution at this time. Updates will be provided as they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Network connectivity was verified restored at 10:10 AM. Engineers are still investigating the root cause of the issue and will provide updates as they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
12/11/2007 4:24:09 PM - Partial Merlin Outage In Bldg 10, ACRF, C Wing
Description:   There is a partial Merlin Outage in Bldg 10, ACRF, C Wing, Division of Lab. Medicine (20004 Clinical Center). Users are able to make calls, but not receive calls on their Merlin System. Verizon has been notified and are working to repair their outage. Updates will be reported as they become available. It involves approximately 80 Merlin telephone sets. The affected IC at this time is the Clinical Center. If you have any problems or need help, please contact the NIH Help Desk at 301.496.4357 or 866.319.4357 toll free, e-mail nihhelpdesk@mail.nih.gov
Resolution:  The Merlin System located in Building 10, Library of Medicine, ACRF, C Wing for the Clinical Center was restored to service at approximately 8:00 AM this morning. If you have any problems or need help, please contact the NIH Help Desk at 301.496.4357 or 866.319.4357 toll free, e-mail nihhelpdesk@mail.nih.gov.
12/11/2007 2:14:42 PM - Constellation Will be Out of Service.
Description:   On Saturday, 12/15, the servers that support Constellation will be out of service for a few hours. Constellation NED to AD data synchronization and account provisioning will be down from approximately 10:00 am until approximately 2:00 pm. Once the servers are restored, all pending changes will be processed, no data or actions will be lost. If you have any problems or need help, please contact the NIH Help Desk at 301.496.4357 or 866.319.4357 toll free, e-mail nihhelpdesk@mail.nih.gov
Resolution:  Time: 7:46AM Constellation is operating normally at this time.
12/11/2007 1:14:05 PM - Internet Connection Latency
Description:   Currently, users are experiencing extremely long load times when accessing internet websites outside of the NIHnet. Engineers are aware of the issue and are working to resolve it. Updates will be provided as they are made available. If you have any problems or need help, please contact the NIH Help Desk at 301.496.4357 or 866.319.4357 toll free, e-mail nihhelpdesk@mail.nih.gov
Resolution:  At 1:32 PM, all internet service was restored to normal operation.
12/11/2007 12:24:38 PM - Connect Downtime Saturday, 12/15.
Description:   Connect will be down on Saturday, 12/15 from 10am to 12pm to perform a database upgrade. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Connect system database upgrade was completed successfully.
12/10/2007 11:08:04 AM - NIHCESMLBX8 Exchange Server Issues
Description:   Exchange Server NIHCESMLBX8 is currently experiencing issues and affecting users' access to Outlook and OWA. The affected ICs at this time are: NCCAM NIDA NLM NIDCD NCRR OD NLM/NCBI NCI/OCE NIH NICHD Updates will be provided as they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Time: 2:46PM The MLBX8 issue is resolved. It was caused by an abnormal email load placed on the server by one of the IC's hosted upon it. Regretfully it took some significant time to correct and resolve the problem. EMIB is exploring ways to prevent this type of problem in the future.
12/7/2007 3:08:42 PM - Maintenance on wirelessguest.nih.gov website
Description:   Please be advised that today Friday, 12/07/07 between 5:00 PM and 6:00 PM maintenance will take place on the wirelessguest.nih.gov website to resolve a reported issue. During this maintenance the wirelessguest.nih.gov website will be offline. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The maintenance was completed successfully and all systems are back online.
12/5/2007 1:28:17 PM - Inbound SMTP Mail Delays
Description:   Currently, there is an approximately 10-15 minute delay in receiving inbound mail from the SMTP gateways. Engineers are investigating the issue and will issue updates as they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Engineers have successfully resolved the delays associated with Inbound SMTP mail. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
12/4/2007 9:34:04 AM - Connectivity Outage in Poolesville
Description:   The Primary DS3 to Poolesville is currently down. The local vendor is currently working to resolve the issue. There is no current estimated time to repair the issue. Further updates will be provided as they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The connection to Poolesville is now operating normally. All connections are functioning properly. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
12/3/2007 9:11:30 AM - Building 13 Access Switch Module Failure
Description:   Currently the building 13 access switch module 5 is down. NIHnet engineers have been dispatched to the site to resolve the issue. The following ICs in building 13 are affected: OD/ORS NICHD NCRR NCI NINDS NHLBI Updates will be communicated as they become available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Time: 11:19AM NIHnet engineers have restored all connections to the module and all network resources have been restored.
11/30/2007 1:36:10 PM - Building 37 3rd Floor Telephone Outage.
Description:   Currently, users are experiencing telephone outages on the 3rd floor of Building 37. There are approximately 50 telephone lines affected by this outage. Engineers have been made aware of the issue and a technician is en route to address the problem. There is no current estimated time of repair. Updates will be provided as they are made available. If you have any problems or need help, please contact the NIH Help Desk at 301.496.4357 or 866.319.4357 toll free, e-mail nihhelpdesk@mail.nih.gov
Resolution:  The affected phone lines have been restored to service. Technicians are investigating the root cause of the issue. If you have any problems or need help, please contact the NIH Help Desk at 301.496.4357 or 866.319.4357 toll free, e-mail nihhelpdesk@mail.nih.go
11/30/2007 12:40:51 PM - Websense Intermittently Blocking Sites
Description:   Currently, users are experiencing issues accessing previously accessible websites. They are receiving "Access Denied" messages from NIH when attempting to go to sites. Engineers are aware of the issue and are working to resolve it as soon as possible. Further updates will be provided as soon as they are made available.
Resolution:  Issues with Websense intermittently blocking websites has been resolved. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
11/29/2007 10:20:47 AM - Telephone Outages in Building 7, Floors 2 and 3
Description:   Currently, users are experiencing telephone outages in Building 7 on the 2nd and 3rd floors. Engineers have been made aware of the issue and a technician is en route to address the problem. There is no current estimated time of repair. Updates will be provided as they are made available. If you have any problems or need help, please contact the NIH Help Desk at 301.496.4357 or 866.319.4357 toll free, e-mail nihhelpdesk@mail.nih.gov
Resolution:  The affected phone lines have been restored to service. Technicians are investigating the root cause of the issue.
11/27/2007 9:00:45 AM - The NIH Application Manager (NAppMan) unavailable on 11/28/07.
Description:   The Hosting Services Branch is upgrading NAppMan's MS SQL database to SQL 2005. There is less impact to monitored systems if the work is performed during business hours. The estimated downtime ranges from just a few hours to as many as nine hours, so we are taking NAppMan down for the business day on November 28th from 8:00 AM until 7:00 PM.
Resolution:  NAppMan is up and running again. The conversion to MS SQL 2005 was successful.
11/26/2007 10:34:04 AM - Network Blade Switch Maintenance
Description:   This Thursday at 6 PM maintenance will be performed on the network blade switches that service portions of the NIH Central Email Service and other infrastructure services. These devices are redundant and it is not expected that this will be a service impacting maintenance. Should there be a problem during the maintenance the outage would not exceed 10 minutes. Some of the services on the network under going maintenance have users directly connected. These Mailbox servers service the following IC's: MLBX3: NINR, NCMHD, NCRR, NIBIB, CSR, NIDCR, CIT MLBX4: ORS, VRC, NIDA, NIAAA, NIDA, FIC, FAES MLBX14: OD The NIH ADM service is in this network maintenance. ITPro's performing account management should be aware of the potential for disruption. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Network Blade Switch Maintenance was successfully completed within the scheduled maintenance window.
11/21/2007 1:45:41 PM - PH Directory Service Retirement
Description:   The NIH Directory and Email Forwarding service, also known as "PH," will retire on November 27, 2007. Functions currently provided by PH will be assumed by other existing NIH services. Thus, PH directory features will be assumed by the official NIH Enterprise Directory (NED); the PH email forwarding capability will be transferred to the NIH Central Email Service (CES); and the PH/CSO query protocol will be replaced by the standard LDAP directory query protocol. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The service was successfully retired on 11/27/2007.
11/21/2007 12:42:10 PM - Democracy2 - DECA Server Switch and VLAN Separation Project
Description:   Date: Wednesday, November 28, 2007 Time: 5:00pm to 7:00pm. ET Description: On Wednesday, November 28, 2007 at 5:00pm to 7:00pm ET, NIHnet engineers will perform Phase II of the Democracy2 - DECA Server Switch and VLAN Separation Project which consists of the activation and patching of the new DEM2-6500-3 switch, the creation of two additional VLANs, updating appropriate trunks, IP separation, and interface sorting (into their correct VLANs). Impact: Please note this Democracy2 - DECA Server Switch and VLAN Separation Project -- PHASE II Maintenance activity has been assessed with a Service Affecting (SA) impact. Specifically, during this two hour maintenance activity, connectivity to CIT/DECA servers will be impacted while they are transitioned to the new server switch (DEM2-6500-3). Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The maintenance was completed successfully at 7:00 PM.
11/21/2007 9:35:02 AM - Telephone Outage in 6707 Democracy Boulevard
Description:   6707 Democracy Boulevard is experiencing a telephone outage of at least 146 telephone lines. The issue is due to an outage of the switch feeding the building. Technicians are currently on-site to resolve this problem. Frequent updates will be provided as to the turn around time and trouble status. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The telephone outage in 6707 Democracy Boulevard has been restored. All 146 telephone lines are now operational. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
11/20/2007 11:09:22 AM - NIHnet Bldg. 12 Data Center Telco Device Relocation Maintenance
Description:   Description: On Saturday, November 24, 2007 at 8:00am - 4:00pm ET NIHnet engineers will perform a maintenance activity in the Building 12 NIH Data Center to relocate redundant NIHnet WAN data network telecommunications hardware to a new cabinet. Impact: The NIHnet Building 12 Data Center Telco Device Relocation Maintenance activity has been assessed with a Possible Service Affecting-Low (PSA-Low) impact for a limited number of NIHnet WAN remote-sites. Specifically, during this eight hour maintenance window, the NIHnet WAN remote-sites listed below will automatically fail-over to redundant connections to NIHnet in Building 10 and Building 45. Baltimore Fredrick RTPNC WHC NIHFCU Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The maintenance was completed successfully with no reported incidents.
11/19/2007 4:21:22 PM - Helix System Maintenance
Description:   The Helix system (helix.nih.gov) will be unavailable to users tomorrow Tuesday, November 20, 2007 beginning at 6:00 am in order to perform a file system reconfiguration. The anticipated downtime is expected to be completed by approximately 7:00 am. Updates will be made available once the work has been completed. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Helix system maintenance was completed successfully at 6:45am and the system became available to users again at 6:50am.
11/19/2007 1:42:13 PM - Bldg 6116 Telephone Service in relations to the Fire Evacuation
Description:   A technician is on standby to verify dial tone at 6116 Executive Boulevard. Once employees are allowed entrance to the building, the technician will begin verifying all working telephone lines. Frequent updates will be provided on the status of this telephone issue. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Time: 2:51PM A technician has verified that there are no telephone outages at 6116 Executive Boulevard in relation to the recent Fire Evacuation.
11/19/2007 1:16:21 PM - Building 6116 evacuated due to a fire
Description:   Building 6116 was evacuated due to a fire and all network equipment have been shut down including the main server for NCI Enterprise Activations. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  All personnel is back in the Building 6116 and all computer systems are back up and available.
11/19/2007 10:47:38 AM - Telephone Outage in 6707 Democracy Boulevard
Description:   6707 Democracy Boulevard is experiencing a telephone outage of at least 100 telephone lines. Technicians are currently on-site to resolve this problem. Frequent updates will be provided as to the turn around time and trouble status. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Time: 7:00AM The telephone service in 6707 Democracy Blvd. has been restored and users should be able to make and receive phone calls normally at this time.
11/17/2007 1:45:34 PM - NCI Network Connectivity
Description:   At this time the NCI users are experiencing Network Connectivity Issues. The engineers are aware of the situation and are working towards a resolution. There is no curent Estimated Time of Repair. Updates will be provided as they become available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  At this time the issue with the NCI network connectivity has been resolved by the NCI technical engineers.
11/17/2007 12:00:54 PM - Zantas EAS Email Archiving Issues
Description:   Following the monthly EMIB maintenance window, the Zantaz EAS Email Archiving service did not return to full functionality. Mail already archived is available for retrieval, however new mail is not being indexed or submitted for archival. CES is investigating the issue with support from the vendor. Currently, there are 71 users distributed across NIH who are participating in the EAS Archived Mail Service who are affected by this issue. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Zantaz EAS archiving service has been restored.
11/16/2007 4:36:24 PM - Short NIH Portal Outage
Description:   Tonight Friday 10/16/07 between 10:00 PM and 11:00 PM there will be a short outage on the NIH Portal service. This is done in order to perform a scheduled maintenance on the SQL Server by MS SQL DBAs. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The maintenance was completed as scheduled.
11/16/2007 5:30:33 AM - GDC Warehouse Circuit Unavailable
Description:   The circuit between the GDC Warehouse and the Building 12 Data Center is currently unavailable. Users in the Gaithersburg Distribution Center may experience difficulty accessing the NIHnet until the circuit is restored. CIT engineers are currently working to resolve the issue. Further updates will be sent as they become available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The circuit between the GDC Warehouse and Building 12 Data Center has been restored to service.
11/15/2007 2:45:58 PM - Democracy 2 - DECA Server Switch and VLAN Separation Project
Description:   On Wednesday, December 5, 2007 at 5:00pm to 8:00pm ET, NIHnet engineers will perform Phase III of the Democracy2 - DECA Server Switch and VLAN Separation Project which consists of the activation and patching of the new DEM2-6500-3 switch, the creation of two additional VLANs, updating appropriate trunks, IP separation, and interface sorting (into their correct VLANs). Impact: Please note this Democracy2 - DECA Server Switch and VLAN Separation Project -- PHASE III Maintenance activity has been assessed with a Service Affecting (SA) impact. Specifically, during this three hour maintenance activity, connectivity to CIT/DECA servers will be impacted while they are transitioned to the new server switch (DEM2-6500-3). Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Democracy 2 - DECA Server Switch and VLAN Separation Project was completed successfully without any complications. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
11/15/2007 11:52:13 AM - Building 9 UPS Emergency Replacement Maintenance
Description:   On Friday, November 16, 2007 at 1:00 a.m. - 2:00 a.m. ET, NIHnet engineers will perform an Emergency maintenance activity to replace the defective Building 9 UPS. Impact: Please note this Building 9 UPS Emergency Replacement Maintenance activity has been assessed with a Service Affecting (SA) impact. Specifically, during this one hour maintenance window, the Building 9 Institutes listed below (which have already received individual Advanced Maintenance Notification) will experience a brief (<30 min) network disruption while network devices are migrated over to the new UPS. NHGRI NICHD NIAMS NCI Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Building 9 UPS Emergency Replacement Maintenance was successfully completed.
11/14/2007 6:40:03 PM - ITAS scheduled maintenance
Description:   ITAS is scheduled for maintenance on Friday, November 16 between 6:00 PM and 6:30 PM. During this time the system will NOT be down and the impact on our customers should be minimal. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The ITAS maintenance has been completed successfully.
11/14/2007 5:53:15 PM - Phone Outage at 6707 Democracy Blvd; 4-5 floors out of service
Description:   Due to a power outage at 6707 Democracy Blvd, several floors have telephone lines that are without service. Technicians are on site right now and Verizon reports that personnel from the SLC group are needed to resolve this issue. Further updates will be provided as they become available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  At 8:00 AM, service was restored to the affected phone lines at 6707 Democracy Blvd. The issue was caused by a fault in the central office.
11/14/2007 3:42:01 PM - Internet 2 Connection Emergency Maintenance
Description:   Internet 2 connection is under going an emergency maintenance tonight between 6:00 pm and 7:00 pm. Internet 2 traffic should failover to the alternative pathway via the Internet if the Internet 2 connection experiences any outages during this maintenance activity. Users should not experience any disruptions with their Internet connectivity. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Internet 2 connection emergency maintenance was successfully completed.
11/14/2007 12:18:57 PM - Building 9 NIHnet Access Switch
Description:   The Building 9 NIHnet building access switch restarted. All services have been restored. NIHnet engineers are currently on site performing root cause analysis. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  This was an initial and close HN.
11/13/2007 11:50:23 AM - Network Blade Switch Maintenance
Description:   This Thursday at 6 PM maintenance will be performed on the network blade switches that service portions of the NIH Central Email Service. These devices are redundant and it is not expected that this will be a service impacting maintenance. Should there be a problem during the maintenance the outage would not exceed 10 minutes. None of the services on the network under going maintenance have users directly connected. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Maintenance was successfully completed.
11/13/2007 11:24:04 AM - NIH Blackberry Device Issues
Description:   Some NIH BlackBerry customers are not receiving email (and perhaps other data) on their devices. At this time, the extent of the problem is being investigated by the NIH BlackBerry Service Provider. Some devices, for which we have received customer calls, have been issued a reload command by the NIH BlackBerry Service Provider. The problem seems to have started at approximately 8:00AM EST yesterday, 11/12/2007. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  This issue was resolved.
11/9/2007 10:51:42 AM - Fibergate Off-Campus Fiber Relocation Maintenance
Description:   On Wednesday, November 14, 2007 starting at 10:00 p.m. through Thursday, November 15, 2007 at 6:00 a.m., Fibergate will perform a scheduled maintenance to relocate fiber in support of a road widening project in Rockville, MD. Impact: Please note this Fibergate Off-Campus Fiber Relocation Maintenance activity has been assessed with a Service Affecting (SA) impact with affected off-campus sites experiencing minimal network disruption of less than 15 minutes while each fiber pair is re-spliced onto a new path. During the eight (8) hour maintenance window, the NIH off-campus locations listed below which have already received individual advanced notification will experience a momentary connectivity disruption to NIHnet. 9800 Medical Center Drive: 5 Research Court: 1 Choke Cherry Also, please note the Fernwood NIHnet GigE connection to the Internet will be affected by this Fibergate maintenance activity. Therefore, all traffic between NIHnet <---> Internet will go via the Building 12 Internet path. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Fibergate Off-Campus Fiber Relocation Maintenance was successfully completed within the scheduled window.
11/8/2007 12:35:42 PM - Issues with Outlook Calendaring
Description:   There are reports from some users of problems accessing calendar information when scheduling meetings with other participants. These users are not able to see the free/busy information for some meeting invitees. CIT engineers have determined the root cause and the problem appears to be resolved from the server side. For any remaining Windows users whose information cannot be seen by others, it is recommended that they launch Outlook in the following manner: 1) Click START 2) Select RUN 3) Enter the following from the OPEN: dialog box: outlook.exe /cleanfreebusy 4) Create a calendar appointment and leave the Outlook client software open for at least 30 minutes to allow the client to publish the updated scheduling information Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions or further problems.
Resolution:  CIT CES has resolved the Outlook calendaring issues.
11/7/2007 3:17:31 PM - Exchange free/busy information issues
Description:   We are currently experiencing some issues with reading Exchange free/busy information. This isn't an across the board problem -- as we've only gotten reports from a few isolated folks -- namely ORF individuals who work in NIEHS spaces in Raleigh. The issue could happen whenever you are attempting to view free/busy info from a user who is on a different Exchange mailbox server than you; the free/busy will show up as hash marks where they previously was information. There is no known cause for this incident but CES engineer are investigating. We are asking any user who has their information missing from the server (those who's schedules have hashes) to run Outlook from a command line with the /cleanfreebusy switch (that would look like: "c:\program files\microsoft office\office12\outlook.exe /cleanfreebusy" -- or the like) -- to then create an appointment in their calendar and let the client stay up for at least 30 minutes to give time for the client to publish the scheduling information. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Please refer to HN06977.
11/6/2007 9:05:13 PM - Rockledge Centre I emergency power transfer.
Description:   Description: OIT has been notified that on Wednesday, November 7, 2007 at 5:00am - 6:00am ET there will be an emergency power transfer at Rockledge 1. Impact: The lights might flicker during the transfer and the power may be down for a max of 15 minutes. OIT will power down OD Network servers during this time to prevent any potential problems. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Emergency Transfer has completed. All systems are back online as of 6:13AM.
11/6/2007 5:19:57 PM - Building 10 B10-MSFC-B IOS Upgrade Maintenance
Description:   Subject: Building 10 B10-MSFC-B IOS Upgrade Maintenance Date: Tuesday, November 20, 2007 Time: 7:00pm - 8:00pm. Description: On Tuesday, November 20, 2007 at 7:00pm - 8:00pm ET NIHnet engineers will proactively upgrade the B10-MSFC-B device operation system to the prescribed revision level to address a Cisco identified Vulnerability. Impact: Please note this B10-MSFC-B IOS Upgrade Maintenance activity has been assessed with a Service Affecting (SA) impact for a limited number of ICs located in building 10. Specifically single attached customer networks listed below will experience a network disruption to NIHnet during the first 15 minutes of this one-hour maintenance activity. NEI NIDCD NHLBI NINDS OD/ORS Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Building 10 B10-MSFC-B IOS upgrade maintenance was successfully completed without any complications.
11/6/2007 5:17:21 PM - Building 10 B10-MSFC-A IOS Upgrade Maintenance
Description:   Subject: Building 10 B10-MSFC-A IOS Upgrade Maintenance Date: Thursday, November 15, 2007 Time: 7:00pm - 8:00pm. Description: On Thursday, November 15, 2007 at 7:00pm - 8:00pm ET NIHnet engineers will proactively upgrade the B10-MSFC-A device operation system to the prescribed revision level to address a Cisco identified Vulnerability. Impact: Please note this B10-MSFC-A IOS Upgrade Maintenance activity has been assessed with a Service Affecting (SA) impact for a limited number of ICs located in building 10. Specifically single attached customer networks listed below will experience a network disruption to NIHnet during the first 15 minutes of this one hour maintenance activity. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Building 10 B10-MSFC-A IOS Upgrade Maintenance was successfully completed within the scheduled maintenance window.
11/6/2007 9:06:35 AM - NIH ADM Account Creation Issues
Description:   Currently, the NIH ADM is experiencing issues with account creation. Engineers are aware of the issue and are working towards a resolution. Further updates will be provided as they become available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Time: 10:33AM The NIH ADM issues have been resolved. Customers should now be able to reconnect and create accounts at this time.
11/6/2007 8:20:04 AM - Network Connectivity issues, Rockledge 1
Description:   There was an power outage that occurred this morning at 6AM in the Rockledge 1 building, power has since been restored. Currently users are having issues with network connectivity. Users should try to reboot their computer as network connectivity has partially been restored. Technicians are working to fully restore network connectivity, currently there is no estimated time of resolution. More updates will be posted as soon as they become available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Network connectivity has been restored all systems are available.
11/6/2007 8:16:52 AM - Connection between Sterling Data Center and Bldng 12 is down
Description:   Currently the connection between Sterling Data Center and Building 12 Data Center is down. NIH engineers are working with the provider to restore the connection. Currently there is no estimated time of resolution. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Network connectivity between NIHnet Campus and the NCCS Data Center in Sterling, Va has been completely restored at 8:10 AM yesterday Tuesday 11/06/07. The root-cause analysis is being performed via an Incident Review process.
11/5/2007 2:42:25 PM - NIH Portal Maintenance
Description:   Friday, November 9, 2007 between 8:00 PM and 11:00 PM the NIH Portal will be unavailable due to a scheduled upgrade maintenance on the SQL server. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The NIH Portal maintenance activity has been successfully completed.
11/5/2007 9:07:20 AM - NIH Wireless Guest Registration System Outage
Description:   At this time the NIH Wireless Guest Registration System is experiencing problems. The web site application support group has been notified. At this time new requests for wireless guest users can not be submitted. Previous requests are not impacted by this outage. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Time: 11:41AM The issue with the Wireless Guest Registration system has been resolved.
11/2/2007 2:07:11 PM - 6101 Executive Blvd. NIMH ENS Equipment Refresh Maintenance
Description:   Subject: 6101 Executive Blvd. NIMH ENS Equipment Refresh Maintenance Date: Saturday, November 10, 2007 Time: 9:00am - 3:00pm. ET Description: On Saturday, November 10, 2007 at 9:00am - 3:00pm ET, NIHnet engineers will replace an existing Cisco Catalyst 4000 switch at 6101 Executive Blvd. that services NIMH, with a Cisco Catalyst 4506 series switch. Impact: Please note this 6101 Executive Blvd. NIMH ENS Equipment Refresh Maintenance activity has been assessed with a Service Affecting (SA) impact. Specifically, the customer LAN will lose connectivity while the switch is replaced, brought online, and fiber uplinks are reconnected. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The maintenance was completed successfully at 12:46 PM.
11/2/2007 2:05:36 PM - CN Change in Active Directory Environment
Description:   On Monday, November 19, 2007 EMIB will be changing AD user objects CN values in the production AD environment to equal the samAccountname. This change was done to all user objects - people, resource and group objects in the preproduction AD environment (dogfood). Please take this time to test your applications. If you do not have a presence in the pre-production environment, you can rename a single account CN value in production, test then either leave it or change it back to equal displayName. If you have any questions, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov.
Resolution:  CN change has completed replication via ADUC and ADM. ADM web site can be used to create accounts. Contact NIH AD Operations for any new CN name issues.
11/2/2007 1:13:11 PM - NIHnet CWDM Ring I Network Upgrade and Optimization Maintenance
Description:   On Saturday, December 1, 2007 at 8:00am - 8:00pm and Sunday, December 2, 2007 at 8:00am - 8:00pm NIHnet engineers will perform a multi-phased upgrade maintenance activity to the NIHnet Core and CWDM Ring I networks. This complex network upgrade and optimization maintenance activity will increase network bandwidth and enhance network performance and reliability between the NIHnet Core and Executive Boulevard NIH Institute data networks. Impact: Saturday, December 1, 2007 The Saturday, December 1, 2007 8:00am - 8:00pm ET maintenance window has been assessed with a Possible Service Affecting - Low (PSA-Low) service impact for NIH Institute and Center (IC) data network transmission between the NIHnet Core and CWDM Ring I networks. Impact: Sunday, December 2, 2007 The Sunday, December 2, 2007 8:00am - 8:00pm ET maintenance window has been assessed with a Service Affecting (SA) impact for a specific list of Off-Campus NIH IC data network connections. Each affected IC will experience approximately 30 minutes of total loss of connectivity to NIHnet during the sequential migration of CWDM Ring I connections to the new network infrastructure. Please note the NIH ICs that will be affected by this maintenance activity have already received individual advanced notification announcements regarding this service affecting (SA) maintenance activity. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  NIHnet CWDM Ring I Network Upgrade and Optimization maintenance activity was successfully rolled back. The NIHnet is back to its pre-maintenance state.
10/30/2007 4:01:57 PM - NIHCADHUB Upgrade
Description:   EMIB will be removing NIHDCADHUB from the nih.gov domain tomorrow Wednesday 10/31/07 after 6:00 PM. This upgrade will NOT impact the user community. A replacement server will be brought back into nih.gov Thursday morning at 7:00 AM with the same name. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  NIHCADHUB Upgrade has been postponed until further notice. Further details will be provided as they become available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
10/30/2007 12:26:15 PM - CIT Remedy Email Notification Backlog
Description:   The Remedy Email Notification system stopped functioning at approximately 9:11 AM this morning. At 12:40 PM STG will restart the engine and all queued Remedy notifications will be sent out. This may result in recipients who frequently get notifications receiving several messages. We apologize for the inconvenience and are working to address the issue.
Resolution:  The cause of the outbound email issues has been traced to the virus scanning software. The issue has been resolved and the emails are being sent. We apologise again for the inconvenience and appreciate your support. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
10/29/2007 4:14:56 PM - Connect System Maintenance
Description:   On Thursday, Nov. 1st between 9:00 PM and 10:00 PM the Connect System will be down for maintenance. During this time The Connect System will NOT be available to the customers. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Connect System Maintenance was successfully completed.
10/29/2007 3:48:07 PM - B12-7600-Outside Router IOS Downgrade Maintenance
Description:   Date: Tuesday, November 6, 2007 Time: 6:00pm - 8:00pm Description: On Tuesday, November 6, 2007 at 6:00pm - 8:00pm ET NIHnet engineers will perform an IOS version downgrade on the B12-7600-Outside router to a more stable version of code. Impact: Please note this PSA-Low maintenance activity has a very low probability of affecting NIHnet Internet connectivity because all NIHnet Internet traffic will be traversing the Fernwood path of the NIHnet Internet DMZ during this maintenance window. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  B12-7600-Outside IOS downgrade maintenance was successfully completed at 7:25 pm.
10/29/2007 3:18:31 PM - ITAS Deployment Maintenance
Description:   ITAS is scheduled for maintenance on Tuesday, October 30th at 6:00PM. This maintenance takes approximately an hour. During this time ITAS will not be available. We thank our users for their patience as we strive to improve ITAS service. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The ITAS scheduled maintenance has been completed successfully. The application is available to the NIH community. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
10/29/2007 11:10:32 AM - 6101 Executive Blvd NINDS ENS Switch Upgrade Maintenance
Description:   Date: Saturday, November 3, 2007 Time: 9:00am - 5:00pm ET Description: On Saturday, November 3, 2007 at 9:00am - 5:00pm ET, NIHnet engineers will replace the remaining Cisco Catalyst 4000 switch that services NINDS with a Cisco Catalyst 4506 series switch. Impact: Please note this 6101 Executive Blvd NINDS ENS Switch Upgrade Maintenance activity has been assessed with a Service Affecting (SA) impact. Specifically, the customer LAN will lose connectivity while the switch is replaced, brought online, and fiber uplinks are reconnected. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The maintenance was completed successfully at 4:30 PM.
10/29/2007 6:53:56 AM - Primary fiber connection between Bld 45 and Frederick is down
Description:   The primary fiber connection between Bldg 45 and Frederick is down the traffic is currently traversing the secondary path. The fiber provider is currently working along with NIH engineers to resolve this issue. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The fiber vendor has repaired the damaged fiber. It is currently back up and is working optimally.
10/26/2007 2:45:20 PM - B10-6500-DR SUPII/MSFC2 Module Replacement Maintenance
Description:   Date: Saturday, November 3, 2007 Time: 9:00am - 11:00am Description: On Saturday, November 3, 2007 at 9:00am - 11:00am ET NIHnet engineers will replace the defective supervisor module in the B10-6500-DR NIHnet router to resolve a console port access issue. Impact: Please note this B10-6500-DR SUPII/MSFC2 Module Replacement Maintenance activity has been assessed with a Service Affecting (SA) impact for the single-attached NIH IC network connections listed below. The service disruption to NIHnet will occur during the first 30 minutes of this two hour maintenance activity. OD/ORS OD/OIT OD/DES NINDS NIAMS Additionally, this B10-6500-DR SUPII/MSFC2 Module Replacement Maintenance activity has also been assessed with a Possible Service Affecting-Low probably (PSA-Low) impact for all dual-attached NIH IC network connections with each dual-attached connection expected to automatically failover to a predetermined redundant path when the B10-6500-DR router is reloaded. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The maintenance was completed successfully at 10:04 PM with no reported impacts.
10/25/2007 11:33:49 AM - DW Query Budget & Finance and Research Contracts & Grants.
Description:   The DW Query Budget & Finance and Research Contracts & Grants will be unavailable to the users from 11:30 AM to 12:45 PM to add NBS Financial data to the nVision Data Warehouse.
Resolution:  The DW Query Budget & Finance and Research Contracts & Grants are now available.
10/24/2007 2:52:52 PM - NIH VPN Client Version Update Maintenance
Description:   Date: Tuesday, October 30, 2007 Time: 10:00am - 11:00am ET Description: On Tuesday, October 30, 2007 at 10:00am, NIHnet network engineers will apply an update to the NIH VPN service that will enable the VPN service to alert MAC, Linux and Solaris OS users when their VPN client software is not at the latest VPN client version available to the NIH VPN user community. Impact: There is no expected impact to the NIH VPN service during this maintenance activity. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The maintenance was completed successfully without any applications being adversely affected.
10/24/2007 9:17:35 AM - NIHDCADHUB2 Upgrade
Description:   EMIB will be removing NIHDCADHUB2 from the nih.gov domain today Wednesday 10/24/07 after 6:00 PM. This upgrade will NOT impact the user community. A replacement server will be brought back into nih.gov tomorrow morning at 7:00 AM with the same name.
Resolution:  NIHDCADHUB2 upgrade was successfully completed at 08:13 AM. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
10/23/2007 12:33:45 PM - Internet Latency when attempting to Browse to External Sites
Description:   At 12:00 Noon, there was high latency when attempting to browse to external websites. Engineers have corrected the issue and internet browsing is now operational.
Resolution:  From 12:00 Noon to approximately 12:20 PM there was a noticable internet latency. Engineers resolved the issue at 12:20 PM.
10/23/2007 11:42:09 AM - Telephone Outage in 6610 Rockledge Dr.
Description:   There are reported telephone outages due to a major power interruption at 6610 Rockledge. Technicians were dispatched. The estimated repair time is pending analysis. Please contact the Help Desk at 301-496-HELP, 301-496-8296 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The issue was resolved at 11:30 AM when the vendor reset the equipment. There were 182 phones afffected by the outage that are now fully operational. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
10/22/2007 11:12:34 AM - The web interface to ADM is experiencing technical issues
Description:   The web interface to ADM is experiencing technical issues. Engineers are working to fix the problems but this may require some server reboots and temporary outages. The ADM console is still available with all functionality. Please contact the Help Desk at 301-496-HELP, 301-496-8296 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The NIH ADM web interface service was restored to service at 3:00 PM. Users should ensure that they are connecting to the NIHADM website, not the NIHADM2. When connecting to https://adm.nih.gov, users should see the following message at the top of the page: Active Directory Management Tool NIHADM If users do not see that message, they should log out and keep trying their browser until they connect to NIHADM.
10/22/2007 9:16:03 AM - Telephone outage on 14th floor of Bldg 10
Description:   On Monday, October 22, 2007 at 8:45am a power (phone line) outage was reported at Building 10 on the fourteenth floor. Technicians have been notified and are in the process of making repairs. We will keep you informed on the status of this issue. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The service to all phone lines has been restored as of 9:25 AM.
10/22/2007 8:19:06 AM - NHLBI Novell Logging issues
Description:   NHLBI is experiencing problems with users logging onto Novell. The NHLBI engineers are aware of the issue and are working towards a resolution. There is no ETR at this time. The hot news will be updated as more information will become available.
Resolution:  NHLBI engineers have resolved the Novell logging issue.
10/19/2007 9:35:05 AM - 6101 EB Aggregation Switch Refresh Maintenance
Description:   Date: Saturday, October 20, 2007 Time: 9:00am - 5:00pm. ET Description: On Saturday, October 20, 2007 at 9:00am - 5:00pm ET, NIHnet engineers will perform a maintenance refresh of the 6101 EB aggregation switch. Impact: Please note this 6101 EB Aggregation Switch Refresh Maintenance activity has been assessed with a Service Affecting (SA) impact. Specifically, customer LANs on the 6101 EB Aggregation Switch will lose connectivity while this switch is replaced, brought back online, and fiber uplinks are reconnected. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  NIHnet Engineers have successfully completed the maintenance. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
10/18/2007 9:22:10 AM - Telephone Outage in Building 45.
Description:   Entire Building 45 is currently experiencing a telephone outage. Verizon has been notified, and is currently on its way to Building 45. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Telephone issue in Building 45 has been resolved. Telephone service is fully operational at this time.
10/17/2007 11:13:18 AM - NIH Help Desk Email Requests
Description:   The NIH Help Desk experienced a problem with a development server causing customers to receive auto-reply messages indicating that the NIH Help Desk mailbox was full. Please note that all requests went through properly and are being appropriately processed. This was only an issue with a copy of the email sent and these notes can be disregarded. Please contact the Help Desk at 301-496-HELP, 301-496-8296 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Time: 11:38 AM. This issue has been resolved.
10/16/2007 12:29:46 PM - Emergency reboot of Petri.
Description:   EMIB will be performing an emergency reboot of the CIT file server "Petri" at 9 PM tonight. The expected down time should be about 20 minutes and will only affect CIT users.
Resolution:  Emergency Reboot completed at 9:44 PM
10/15/2007 8:11:37 AM - NBS Oracle and Prism are not functioning.
Description:   NBS PRISM and ORACLE are experiencing connectivity issues at this time. NBS Management Center is aware of this issue and is working towards a resolution. There is no ETR at this time and updates will be posted as they become available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  NBS Management Center has resolved the issue as of 9:44 AM.
10/12/2007 8:50:31 AM - NIH Application Manager (NAppMan) is experiencing problems,
Description:   NAppMan is not receiving alerts from any underlying monitors. The NAppMan team is currently looking into the problem. No ETA for resolution. Monitored systems are not affected.
Resolution:  NAppMan is once again receiving alert messages. It is available and reliable.
10/12/2007 7:30:06 AM - 6006 Executive Blvd Electrical Shutdown
Description:   Date: Sunday, October 14, 2007 Time: 8:00am - 2:00pm ET Description: On Sunday, October 14, 2007 at 8:00am - 2:00pm ET, Facility Electrical Engineers at 6006 Executive Blvd will perform a power maintenance activity in the 6006 Executive Blvd building that will require a complete electrical power shutdown in the entire 6006 Executive Blvd building. The 6006 Executive Blvd building power maintenance activity will commence at 8:00am ET and is expected to last for approximately six (6) hours. Impact: Please note this 6006 Executive Blvd Electrical Shutdown Maintenance activity has been assessed with a Service Affecting (SA) impact. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  This maintenance was completed successfully with no complications within the scheduled maintenance window.
10/11/2007 2:17:47 PM - Telephone Outage in Building 7
Description:   In Building 7, ther telephones on the second and third floors are currently experiencing issues. About 78 phones out of service. A technician is on site. Currently there is no power to the closet on the affected floors. The technician is working with a building engineer to restore service. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Engineers have restored the normal power configuration for the affected phone systems and all systems are now operating normally.
10/9/2007 10:04:39 AM - NIH Application Manager (NAppMan) Downtime for Web Certificate.
Description:   The NIH Application Manager will be unavailable on Wednesday, 10/10/07 from 10:30 AM until 12:00 noon. The downtime is necessary to test the installation process of and the installation of an updated web certificate. Monitored applications are unaffected by this outage.
Resolution:  The NIH Application Manager is once again available. The web certificate is in place.
10/5/2007 11:30:32 AM - Building 50 NHGRI Gig Link Upgrade Maintenance
Description:   Date: Saturday, October 6, 2007 Time: 9:00am to 11:00am Description: On Saturday, October 6, 2007 at 9:00am to 11:00am NIHnet engineers will upgrade the network connectivity for NHGRI in Building 50 from Fast Ethernet to Gigabit Ethernet. Impact: Please note this Building 50 NHGRI Gig Link Upgrade Maintenance activity has been assessed with a Service Affecting (SA) impact. Specifically, during this maintenance activity, there will be an outage to the devices (<15 min) and a spanning-tree re-calculation resulting from the uplink port change. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The maintenance was completed successfully with no complications within the scheduled maintenance window.
10/4/2007 11:04:12 AM - Fernwood-7600-Outside Router IOS Downgrade Maintenance
Description:   On Thursday, October 11, 2007 at 6:00pm to 8:00pm NIHnet engineers will perform a proactive downgrade of the Fernwood-7600-Outside router IOS version to a more stable version of code. Please note this PSA-Low maintenance activity has a very low probability of affecting NIHnet Internet connectivity because all NIHnet Internet traffic will be traversing the Building 12 path of the NIHnet Internet DMZ during this maintenance window. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Time: 8:36PM Maintenance was completed successfully with some TACACS complications. No services were impacted.
10/3/2007 11:25:19 AM - Bldg 12 Data Center B12-6500-DR Module Replacement Maint.
Description:   Date: Thursday, October 11, 2007 Time: 6:00pm to 7:00pm Description: On Thursday, October 11, 2007 at 6:00pm to 7:00pm NIHnet engineers will install a new WS-X6816-GBIC module in slot 13 of the Building 12 Data Center B12-6500-DR followed by proactively reloading this device. Impact: Please note this Building 12 Data Center B12-6500-DR Module Replacement-Reload Maintenance activity has been assessed with a Service Affecting (SA) impact for single attached customer networks and Possible Service Affecting-Low (PSA-Low) for dual attached networks. Specifically single attached customer networks listed below will experience a network disruption to NIHnet during the first 15 minutes of this 1 hour maintenance activity. OD/ORS OD/OIT NIAMS CIT/DNST Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Maintenance has been completed successfully as of 8:20 PM.
10/2/2007 2:43:49 PM - NIHCESMLBX8 Server Issues
Description:   At this time, the NIHCESMLBX8 Exchange Server is having connection issues. Known IC's on this server are NCCAM, NIDA, NIDCD NLM and NCBI. Engineers are aware of the issue and are working on restoring service at this time. Updates will be provided as they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  At 2:58 PM, service was restored to the NIHCESMLBX8 Exchange Server. Engineers are working with the vendor to identify the root cause of the issue.
10/1/2007 9:14:43 AM - HHS Video Conferencing Bridge
Description:   Date: Monday, October 1, 2007 Time: 8:45am Description: Monday, October 1, 2007 The madge video switch in building 10 is currently experiencing hardware problems. CIT's Video application engineers are working to resolve the problem. For now, there is no Estimated Time of Resolution. Impact: Due to hardware problems with one of CIT's video switches the following HHS Video Conferencing services will be disrupted: Calling into a scheduled HHS hosted bridge meeting. Calling from an IP video conferencing unit registered to the NIH Gatekeeper to an ISDN number. Hot News will updated as information becomes available Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The video switch in Bldg 10 has been replaced and is fully operational.
9/27/2007 12:39:39 PM - OD Home Server is Having Issues
Description:   This will effect connection to their H drive, and possibly Outlook. OIT is working on the issue and will update this Hot News shortly.
Resolution:  The OD home drive server is now functioning normally. There was a conflict with the anti-virus scan and rebooting the server corrected the problem.
9/25/2007 5:42:52 PM - Connect System Maintenance Downtime
Description:   On Monday, October 8, 2007 from 9:00 PM to 10:00 PM, the Connect system will be taken offline to apply a service pack. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Connect System Maintenance has completed successfully with no complications. Please Note: The Connect Meeting add-in has been updated and Connect now requires Flash Player 8.
9/22/2007 2:42:30 AM - Remedy Restored to Service
Description:   Tonight's issues with the CIT Remedy server have been resolved. Remedy services are now available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Remedy has been restored to service as of 2:30 AM.
9/19/2007 3:07:45 PM - Production NIH Portal Database Issue
Description:   Due to an unknown SQL error, the NIH Portal production database was recently inaccessible. After the SQL DBAs brought it back, portal was immediately functional but some portlets, notably Collaboration, were showing errors.
Resolution:  All portal servers (nihportal20-27) were consequently restarted. Service was completely restored.
9/17/2007 3:26:44 PM - Silk Application Issues
Description:   Silk applications are inaccessible to some users. The CIT staff is working towards a resolution. Updates will be posted as they become available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Engineers believe they have resolved the issue and that the system is working correctly. If you are still having issues accessing the Silk applications, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
9/17/2007 3:10:00 PM - NIHWEB5 Emergency Outage
Description:   Engineers will be bringing down the NIHWEB5 server for approximately twenty minutes on Monday, September 17, 2007 at 5:00 PM to address a memory issue with the server. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The maintenance has been completed and the memory issues have been corrected.
9/17/2007 9:21:21 AM - NAppMan Outage
Description:   The NIH Application Manager (NAppMan) will experience approximately one hour of downtime at 12:00 noon today. The outage will be used to correct a virtual frame buffer problem.
Resolution:  The virtual frame buffer problem has been resolved. The NAppMan dashboard is once again reliable.
9/16/2007 10:31:56 PM - NIHCESMLBX10 is Unavailable
Description:   The NIHCESMLBX10 exchange server will be reset at 10:30 PM. This emergency reset will cause a brief (10 minute) loss of email service for NCI users. An update will be sent when service is restored. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The reset of NIHCESMLBX10 has been completed. The exchange server is now up and available for users.
9/15/2007 12:31:44 PM - Monthly JCAHO Generator Test: Old Building 10 & ACRF
Description:   On September 15th, starting at 1:00pm testing of the Emergency electrical power systems in Old Bldg.10 and the ACRF Facilities will be performed. The purpose of this test is for the hospital to be in compliance with the requirements of (JCAHO). During the test only emergency power and lighting will be interrupted for approximately (30) minutes in duration. During this time all emergency power will be switch over to generator power. This will affect ALL ROOMS on ALL FLOORS in "Old" Bldg.10 and the ACRF Facilities. However limited emergency electrical power (RED OUTLETS) will be available. The following critical areas will be affected; Surgical Suites, Clinics, Laboratories, Vivariums, Constance Temperature rooms, some Corridor lighting, HVAC (Heating, Ventilation and Air Conditioning) systems, Exhaust Air systems, Bio-Safety/Fume hood cabinets, Refrigerators/Freezers, Auto-claves, Incubators, Centrifuges and all other equipment that are operating on emergency power only. With the exception of the following areas; Bldg.10-A, NMR (the Invi-vo center), B3 Oncology, the First (1) floor of Bldg. 10's Library, the South Entrance into Bldg. 10, and A-wing addition floors (3) three thru (7) seven. Additional Remarks - Normal power and lighting WILL NOT be affected during this test. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The emergency electrical power systems test in Old Building 10 and the ACRF Facilities has been completed.
9/15/2007 8:21:28 AM - Monthly JCAHO Generator Test: CRC
Description:   On Saturday, September 15, starting at 8am, the Hatfield CRC emergency generators will be tested for approximately 30 minutes. All power will be off at the start of the test and only emergency circuits will be energized as the genarators come on line. This monthly test is required to meet new JCAHO regulations. Note: A window of 8am to 11 am has been listed on the Shutdown to allow for unexpected delays to the start of the test. The purpose of this maintenance activity is to test the CRC Emergency Power Generators for JCAHO compliance Additional Remarks - CRC Maintenance will be on site during the test to monitor and restart HVAC equipment.
Resolution:  The CRC Generator test is complete. Technicians are verifying phone status to ensure there are no outages. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
9/15/2007 6:48:11 AM - NIHCESMLBX14 is Unavailable
Description:   The NIHCESMLBX14 exchange server is currently down. CIT engineers are working to resolve the issue as soon as possible. More updates to follow. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The issues affecting the NIHCESMLBX14 server have been successfully resolved.
9/14/2007 12:05:27 PM - Slowness with ITAS
Description:   At this time ITAS is experiencing extreme slowness due to heavy traffic. ITAS Administrator has advised the current degredation is based on usage and the issue is not the application itself. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  ITAS performance is back at normal speed, the degradation is no longer an issue. Thank you for your patience and continuous support while we were working on resolving the issue.
9/13/2007 11:57:15 AM - Immediate Outage of the NBS
Description:   Attention: NBS Budget and Finance and Warehouse Personnel. The NBS will be going down at 11:25 am for approximately 10 minutes. Applicable users will be disconnected and will need to log back into the system. Please notify your staff members who may be affected by the system outage. If you have any questions, please contact the NIH Help Desk at (301) 496-HELP or send email to helpdesk@mail.nih.gov
Resolution:  Resolved
9/12/2007 4:06:24 PM - NIH Help Desk Email Requests
Description:   The NIH Help Desk experienced a problem with a development server causing customers to receive auto-reply messages indicating that the NIH Help Desk mailbox was full. Please note that all requests went through properly and are being appropriately processed. This was only an issue with a copy of the email sent and these notes can be disregarded. Please contact the Help Desk at 301-496-HELP, 301-496-8296 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Issue has been resolved.
9/7/2007 3:05:35 PM - NIH Primary Video Conferencing Gatekeeper Maintenance
Description:   On Friday, September 7, 2007 at 5:00pm - 5:15pm the NIH Primary Video Conferencing Gatekeeper will be down for maintenance. Impact: Due to loss of connectivity with the NIH Primary Gatekeeper the following CIT video conferencing services will be disrupted: Calling from an IP video conferencing unit to an ISDN number. Calling an IP video conferencing unit using an E.164, alias, number and not the actual IP address. Hot News will be updated as information becomes available.
Resolution:  The NIH Primary Video Conferencing Gatekeeper maintenance has been completed and all services are back to normal.
9/6/2007 6:33:52 PM - Building 12 Data Center HSB HP Blade Switch Replacement
Description:   The Building 12 Data Center HSB HP Blade Switch Replacement maintenance was completed at 6:29 PM with no devices adversely affected.
Resolution:  The Building 12 Data Center HSB HP Blade Switch Replacement maintenance was completed at 6:29 PM with no devices adversely affected.
9/4/2007 1:04:40 PM - Building 12 B12-6500-DR IOS Upgrade Maintenance
Description:   On Thursday, September 13, 2007 at 6:00pm - 8:00pm NIHnet engineers will proactively upgrade the Building 12 B12-6500-DR IOS to the prescribed revision level to address a Cisco identified, Crafted IP Vulnerability. Impact: Please note this Building 12 B12-6500-DR IOS Maintenance activity has been assessed with a Service Affecting (SA) impact for single attached customer networks and Possible Service Affecting-Low (PSA-Low) for dual attached networks. Specifically single attached customer networks listed below will experience a network disruption to NIHnet during the first 15 minutes of this 2 hour maintenance activity. OD/ORS OD/OIT CC/DNA NIAMS NINDS CIT/DNST Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The maintenance was completed successfully with complications.
9/4/2007 7:26:22 AM - Building 45 B45-6500-DR IOS Upgrade Maintenance
Description:   On Tuesday, September 11, 2007 at 6:00pm - 8:00pm NIHnet engineers will proactively upgrade the Building 45 B45-6500-DR IOS to the prescribed revision level to address a Cisco identified, Crafted IP Vulnerability. Impact: Please note this Building 45 B45-6500-DR IOS Maintenance activity has been assessed with a Service Affecting (SA) impact for single attached customer networks and Possible Service Affecting-Low (PSA-Low) for dual attached networks. Specifically single attached customer networks listed below will experience a network disruption to NIHnet during the first 15 minutes of this 2 hour maintenance activity. OD/ORS OD/OIT CC/DNA NIAMS NINDS CIT/DNST Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The maintenance was completed successfully at 6:30 PM with no devices adversely affected.
8/31/2007 9:57:36 AM - Power Outage in ACRF and Building 49
Description:   A commercial power outage has been reported, affecting the ACRF Building and Building 49. ORF resources are engaged to address the issue. A CIT Telecommunications technician is en route to address the 23 phones that are known to be affected. More details to follow. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or submit a request via our website at http://support.cit.nih.gov/ to report any related issues.
Resolution:  The Merlin Telephone Issues affecting the 2C385 area of Building 10/ACRF have been resolved. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or submit a request via our website at http://support.cit.nih.gov/ to report any related issues.
8/30/2007 3:18:58 PM - NIHCESMLBX5 & NIHCESMLBX8 Server Slowness
Description:   Currently, we are experiencing http access issues on the NIHCESMLBX5 and NIHCESMLBX8 mailbox servers. CES engineers are investigating the issue at this time. Updates will be provided when they are available. Mailbox server NIHCESMLBX8 will be brought down and back up shortly.
Resolution:  The NIHCESMLBX5 and NIHCESMLBX8 servers were rebooted to clear the problem situation and response times have improved since the reboots were completed. Engineers will continue to perform root cause analysis.
8/29/2007 1:55:38 PM - Building 10 B10-6500-DR IOS Upgrade Maintenance
Description:   On Tuesday, September 4, 2007 at 6:00pm - 8:00pm NIHnet engineers will upgrade the Building 10 B10-6500-DR router operating system to address a known Cisco Crafted IP vulnerability. Impact: Please note this B10-6500-DR operating system upgrade maintenance activity has been assessed with a Service Affecting (SA) impact for the single-attached NIH IC connections listed below. The service disruption to NIHnet will occur during the first 15 minutes of this two hour maintenance activity. OD/ORS OD/OIT OD/DES CIT/DNST NINDS NIAMS Additionally, this B10-6500-DR operating system upgrade maintenance activity has also been assessed with a Possible Service Affecting-Low probably (PSA-Low) impact for all dual-attached NIH IC network connections with each dual-attached connection expected to automatically failover to a predetermined redundant path when the B10-6500-DR router is reloaded. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Building 10 B10-6500-DR IOS Upgrade Maintenance was completed successfully without complications.
8/25/2007 12:03:18 AM - Building 10 Phone System Maintenance Test
Description:   CIT will conduct a maintenance test of the phone system in building 10 on the 7th floor on Tuesday, August 28, 2007 between 5 a.m. and 6 a.m. If service is interrupted during the test, it is anticipated that all service will be fully restored by 6:00 a.m. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or submit a request via our website at http://support.cit.nih.gov/ if you have any questions.
Resolution:  Time: 7:00AM The Building 10 phone system maintenance test was completed successfully.
8/24/2007 3:58:37 PM - CIT Remedy System Outage to Perform Server Move
Description:   Start: Wednesday, September 5, 2007 4:00 AM Duration: 60 Minutes Description: The DCSS Hosting Service Branch will be moving the servers supporting the CIT Remedy System and associated web sites starting at 4:00 AM on Wednesday, September 5, 2007. The servers need to be relocated to support functional and organizational configurations. The servers should be unavailable for no more than 60 minutes. Users will be unable to access CIT Remedy services during this outage which includes submitting service requests and registering for CIT Training courses. The following web sites will also be inaccessible during this outage: NIH Help Desk Support Site (http://ithelpdesk.nih.gov/); CIT Training Site (http://training.cit.nih.gov/); CIT Knowledge Base (http://kb.nih.gov); Statistical Software and Resources Site (http://statsoft.nih.gov/); CIT Publications, Documentation and Software (http://publications.cit.nih.gov/); and CIT Remote Access Site (http://remoteaccess.nih.gov/). This outage will also affect all systems that access the CIT Customer Gateway for creating tickets or requests. Please contact the NIH Help Desk at 301-496-HELP (4357) or 301-496-8294 (TTY) if you have any questions.
Resolution:  Maintenance has completed successfully. All systems are operational as of 4:48 AM.
8/23/2007 10:58:09 AM - Fernwood Building Partial Power Outage on Saturday, 25 August
Description:   On Saturday, 25 August, the power supply to the North side of the Fernwood building will be shut off in order to restore power to the part of the building under construction. The emergency generator will engage to supply power to those areas of the North side of the building that are designated for generator backup. There is no anticipated power impact to the South side of the building. The actual start time and end time of the power outage is unspecified at this time. Please funnel questions and concerns about generator-supported and non-generator supported areas through your functional management chain. During or following the power outage on Saturday, please report any issues or unexpected impact to the CIT DCS CAP Team at 301-402-0724 for prompt coordination with facility management.
Resolution:  The power was restored at approximately 3:40 PM. The card reader systems were restored at approximately 4:50 PM. All systems are functioning normally.
8/22/2007 8:59:24 AM - Remedy System Problems with Inbound/Outbound Messages
Description:   There are issues with the email engine in Remedy, so inbound and outbound messages are NOT being processed. This affects requests for service submitted by email AND the Remedy ticketing system's notification processes. All support teams are engaged, but the estimated return to service is unknown at this time. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or submit a request via our website at http://support.cit.nih.gov/ if you have any questions.
Resolution:  The problem has been corrected and the service is running normally. Engineers will be monitoring Remedy through the evening to ensure that all systems continue to run normally.
8/20/2007 11:22:18 AM - The Password Self Service Server Will Be Rebooted at 11:30am
Description:   The Password Self Service (PSS) server will be rebooted at 11:30am today. The interruption will be brief. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Time: 11:39AM The reboot of the PSS server has been completed successfully and all client services are back online.
8/20/2007 9:29:17 AM - Telephone Outage in CRC-Building 10, Floors 1, 3, and 11
Description:   There is currently a telephone outage in CRC-Building 10 on floors 1, 3, and 11. Technicians are working towards a resolution. There is no estimated time of resolution. Updates will be provided as they are made available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  All telephone repairs have been completed in CRC building 10. Service is fully functional at this time.
8/20/2007 7:59:44 AM - Internet Connectivity to External Websites
Description:   There is currently an issue with connecting to external websites through NIHnet. NIHnet engineers are currently working towards a resolution. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Time: 8:18AM The connectivity issues to external websites through NIHnet have been resolved at this time.
8/18/2007 4:59:26 PM - NIMH hardware update
Description:   Engineers will be updating the NIMH hardware in Building 31, Building 10, and Cedar lane on Saturday, August 18, 2007 from 5:15 PM until 6:00 PM.
Resolution:  The maintenance was completed successfully.
8/18/2007 4:54:42 PM - Cedar Lane NIMH Gigabit Upgrade Maintenance
Description:   NIMH engineers will be upgrading hardware at Cedar Lane on Wednesday, August 22, 2007 from 4:00 PM until 6:00 PM. Users on the Cedar Lane segment may experience short-period, intermittent network outages during the maintenance window.
Resolution:  The maintenance was completed successfully at 5:05 PM and alll systems are functioning normally.
8/17/2007 1:37:09 PM - Advanced Maintenance Notification: NED JoinEngine Upgrade
Description:   The NED back-end JoinEngine will undergo an upgrade on Saturday, September 8, 2007 from 12:00 PM until 4:00 PM EST. Web interfaces to NED (NEDWeb), the public accessible NED data, and NED customer tables in Oracle and DB2 will be functional during the maintenance window. Applications and systems accessing NED data will not see changes that are made using the NEDWeb interfaces. This includes changes made via NEDWeb interfaces (including the NED self-service and the NED Link Editor) and changes made via Constellation used for AD provisioned ICs.
Resolution:  The NED maintenance update was successful and all NED systems are functional. The ORS badging staff has not responded to changes needed on ORS systems and thus there may be some delays on Monday of new badge authorizations being recognized by ORS.
8/16/2007 5:12:54 PM - Telephone Outages in Multiple Rockledge Buildings
Description:   The power outage that affected Rockledge 1 on Thursday, August 16, 2007 also had an affect on telephones for all the Rockledge buildings. The majority of phone lines that were affected have been restored. Technicians will be dispatched and onsite on the morning of Friday, August 17, 2007 to restore any remaining telephone outages. Please refer any telephone-specific issues to Susan Garrett at garretts@mail.nih.gov or 301-435-2020.
Resolution:  Telephone service has been restored.
8/16/2007 3:02:36 PM - Power Outage in Rockledge 1
Description:   Since approximately 2:20 PM, Rockledge 1 has been experiencing a power outage. The outage is affecting the entire building. There is currently no estimated time to resolution of the issue. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Network connectivity has been restored.
8/16/2007 9:39:57 AM - Security Vulnerability in the NIH Cisco VPN Client for Windows
Description:   Cisco System has released a security advisory (Advisory ID: cisco-sa-20070815-vpnclient) detailing a serious vulnerability in their Virtual Private Networking (VPN) Client software. CIT has tested and posted the latest VPN Client for Windows 2000 and Windows XP. This new VPN Client (version 5.x) is the client Cisco has developed to resolve the vulnerability. NIH VPN users are advised to upgrade their Windows VPN Client. The New VPN Client is available from CIT via the iSDP website: http://sdp.cit.nih.gov/downloads/vpn_tools.asp This new VPN Client does not support Windows 9x, Windows ME or Windows NT. Because the older clients that supported these operating systems are vulnerable, CIT will remove the older VPN Client from distribution. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The vulnerability issue with the NIH Cisco VPN Client for Windows has been addressed and customers are able to receive the necessary guidance in regards to this issue.
8/14/2007 4:57:19 PM - Remedy System Outage
Description:   On Friday, August 17 at 9:45 PM to Saturday, August 18 at 6:15 AM, the Remedy system will be unavailable due to the SAN maintenance that is taking place from 10:00 PM on Friday, August 17 to 6:00 AM on Saturday, August 18. This outage means that people will not be able to submit tickets via the web during this time. Tickets submitted via e-mail will be queued and processed once the system is restored. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Users have confirmed operational status.
8/13/2007 2:24:10 PM - Connect/Breeze Scheduled Maintenance
Description:   At 9:00 PM on Wednesday, August 15, 2007 Connect/Breeze will be going down for scheduled maintenance. Service will return by 9:30 PM. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The maintenance was completed successfully with no complications.
8/10/2007 1:02:09 PM - Building 10 telephone connectivity issues
Description:   Currently there is a telephone outage reported in Building 10 in the 4N301 area affecting approximately 80 telephones. Technicians are on site conducting repairs and should have the problem rectified shortly.
Resolution:  Repairs to the telephone service in Building 10 in the 4N301 are complete and service has been restored. If you experience any lingering problems with your telephones please contact either the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), helpdesk@nih.gov or the telephone repair desk at 301-402-9935.
8/8/2007 6:54:25 PM - NIH Data Center Power Maintenance
Description:   As a follow up to the Building 12 Data Center power outage on June 28, 2007, CIT will be coordinating a configuration change on the generator switch to provide a 30 second rollover from battery backup to generator power. This configuration change is scheduled for Tuesday, August the 14th, from 6:00-7:30 PM. The impact of this maintenance and testing window has been assessed with a low probability of service affect. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  As of 9:00 PM, the maintenance was completed successfully and all systems are operational.
8/8/2007 3:02:32 PM - NappMan Outage For New Release.
Description:   NAppMan will be unavailable from 9:00 AM until 12:00 noon on Thursday, August 9th, so that a new release can be implemented.
Resolution:  The new release of NAppMan is now available.
8/8/2007 1:01:39 PM - Fernwood NIHnet Internet DMZ Outside Router Reload Maintenance
Description:   On Tuesday, August 14, 2007 between 8:00pm - 9:00pm ET NIHnet engineers will proactively reload the Fernwood NIHnet Internet DMZ Outside Router as part of their ongoing network optimization project. Impact: Please note this Fernwood NIHnet Internet DMZ Outside Router Reload Maintenance has been assessed with a Possible Service Affecting - Low (PSA-Low) impact. Specifically, end-users may experience a momentary delay while their traffic is redirected to the Building 12 B12-7600-outside router. The momentary delay in data traffic should occur within the first 10 minutes of the one-hour maintenance window and is not expected to last more than five minutes in duration. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The maintenance was completed without complication at 9:33 PM.
8/8/2007 12:01:15 PM - Building 13 Telephone issues
Description:   Bldg 13 is experiencing a telephone outage involving 46 telephones. A technician is on site, restoring service. Service is expected to be restored shortly.
Resolution:  The 46 telephones in Bldg 13 that were out of service due to generator testing, have been restored. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
8/7/2007 2:34:37 PM - Building 45 Emergency Generator Power Test Maintenance
Description:   Date: Wednesday, August 8, 2007 Time: 7:00am - 8:00am ET Description: On Wednesday, August 8, 2007 at 7:00am - 8:00am NIH Facilities engineers will perform a Scheduled Emergency Generator Power Test Maintenance activity in Building 45. Impact: Please note the Building 45 Emergency Generator Power Test Maintenance activity will cause a momentary interruption in the Wireless connectivity to active Wireless Guest Users only. The NIHnet Wireless Service WILL NOT be affected by this maintenance activity because NIHnet Wireless access and VPN connectivity is supported by a redundant path via the Building 12 Wireless DMZ infrastructure. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  This maintenance has been postponed to a future date that has not yet been determined. You will be notified when the maintenance is scheduled for a new date.
7/31/2007 8:00:08 AM - Problem with Remedy Mirror Data
Description:   The Database job that transfers the CIT Service Ticket data to the Remedy Mirror failed last evening. We are currently working on getting the job run again but until the job completes, reports based on the Service Ticket data will not return any information. This also includes any reporting done through the "Remedy Mirror". The job will take approximately 6 hours to run during the day and hopefully the data will be available again early this afternoon. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Service Ticket data transfer has completed and the data is now available.
7/30/2007 2:35:56 PM - Baltimore WAN Redundant Circuit Emergency Repair Maintenance
Description:   Date: Monday, July 30, 2007 Time: 3:00pm - 4:00pm Description: On Monday, July 30, 2007 at 3:00pm - 4:00pm NIHnet engineers will perform an emergency repair maintenance on a NIHnet Baltimore WAN router interface to eliminate a systemic fault that has forced NIHnet --- Baltimore traffic via the redundant Catonsville path. Impact: Please note this NIHnet --- Baltimore WAN Redundant Circuit Emergency Repair Maintenance activity has been assessed with a Service Affecting (SA) impact. Specifically, the NIH IC networks directly connected to the Baltimore-7200-1 router will be down while a router DS-3 port adaptor is replaced. All affected network POCs have been contacted regarding this Service Affecting maintenance prior to execution of this emergency repair. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Time: 3:35PM This maintenance was completed successfully without complications. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
7/30/2007 10:42:30 AM - Telephone Outages Bldg 21, 45 (Conf Cntr) 6705 Rockledge
Description:   Due to the weekend storms, there are currently telephone outages in Building 21, Building 45 Conference Center, and 6705 Rockledge/2nd and 3rd floors. Technicians are on-site to restore service. Any new information will be communicated when available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Telephone service has been restored to Building 5, Building 21, Building 45 Conference Center, and 6705 Rockledge/2nd and 3rd floors. If there are any additional telephone outages reported, please contact the service desk.
7/29/2007 4:15:29 PM - Network Connectivity Issues in Buildings 12, 16, and 45
Description:   There are network connectivity issues in buildings 12, 16, and 45 at this time. Network engineers are currently investigating the issues. Updates will be provided as they become available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The connectivity for the Buildings 12, 14 and 45 was restored. All devices are back up and fully functional.
7/27/2007 5:06:45 PM - NIH CIT Content Management Web Sites Affected by NED Outage
Description:   Due to the scheduled NED maintenance outage on Saturday, August 4, from 4:00 PM to 6:00 PM, some pages on the public web sites running on CIT’s Content Management Service (CMS) will be inaccessible. The web sites will not be affected, only some pages that connect to the NED will be affected. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The NED maintenance was scheduled to be completed on Saturday, August 4 at 6:00 PM. NED is now available.
7/27/2007 3:05:40 PM - ITAS Security Maintenance
Description:   ITAS is scheduled for an security maintenance on Monday, July 30, 2007 from 10:05PM - 10:25PM. During this time ITAS will not be available. We appreciate your patience as we improve the ITAS service. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The maintenance was completed successfully at 10:50 PM.
7/27/2007 9:07:29 AM - NIHnet CRC-6500-6G Module Installation Maintenance
Description:   Subject: NIHnet CRC-6500-6G Module Installation Maintenance Date: Friday, August 10, 2007 Time: 2:00am - 3:00am ET Description: On Friday, August 10, 2007 between 2:00am - 3:00am ET NIHnet engineers will perform a maintenance activity to install a 48 port expansion module in slot 12 of the CRC-6500-6G switch in order to increase port density. Impact: Please note this NIHnet CRC-6500-6G Module Installation Maintenance has been assessed with a Service Affecting (SA) impact. Specifically, this maintenance activity will cause a brief network disruption (<15 min) during the planned reload of the CRC-6500-6G switch after the 48 port expansion module is installed. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The NIHnet CRC-6500-6G Module Installation was successfully completed within the scheduled maintenance window.
7/24/2007 5:06:50 PM - MSN Messenger Phishing Attempt
Description:   MSN Messenger Users are receiving random Instant Messsages from people on their contact list telling them to go to a link to see who has deleted or blocked them from their contact list. The messages reported are similar to this: "Check out this web http//my-msn.servebeer.com it tells you who deleted or blocked you from their MSN" The following links have been reported and are currently being blocked inside the NIHnet: http//messenger-scan.redirectme.net http//www.messenger-tips.com http//my-msn.servebeer.com Do NOT click on these links and do NOT login to these sites. If you have already logged in, please change your Windows Messenger password as soon as possible. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  All reported sites have been blocked. Users should not have any more issues with the MSN Messenger.
7/23/2007 7:42:03 AM - NBS Oracle is currently unavailable
Description:   NBS Oracle is currently unavailable, this is only affecting NBS Acquisitions access. PRISM and Travel are currently available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Service has been restored.
7/20/2007 8:53:18 AM - Remedy Reports Server Hung
Description:   One of the jobs that transfers data to the Remedy reports server appears to be hung. As a result, reports and data queries that depend on the audit trail went out blank this morning and will continue to do so until the issue is resolved. The majority of Remedy reports will not be affected by this issue. CIT engineers are working to repair the problem and expect it to be resolved later today. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  This issue has been successfully resolved. The jobs have finished processing and all Audit Trail data is available.
7/19/2007 1:49:38 PM - NIHnet Server Status webpage - "Watchman" to be decommissioned
Description:   On August 6, 2007, the NIHnet Server Status web page located at URL http://watchman.net.nih.gov/ipchkrefreshed.html will be decommissioned and the host machine watchman.net.nih.gov turned down. Functionality provided by the NIHnet Server Status page has been replaced by the NAppMan application at URL http://nappman.cit.nih.gov:8080/NAppMan/index.jsp . Contact: Saundra Emma emmas@mail.nih.gov; 301-402-4764 with questions or concerns.
Resolution:  The decommissioning of the Watchman website was completed successfully at 10:30am today, August 7, 2007. Contact: Saundra Emma emmas@mail.nih.gov; 301-402-4764 with questions or concerns.
7/18/2007 8:54:57 AM - Qwest Building 12 to Sterling VA SONET Equipment Maintenance
Description:   On Thursday, July 26, 2007 at 11:00pm to Friday, July 27 at 5:00am, Qwest engineers will perform a Scheduled Maintenance Activity in Building 45 on their SONET equipment to resolve suspected internal damage that causes cards to randomly reset and drop traffic. Impact: Please note this Qwest Building 12 to Sterling VA SONET Equipment Maintenance activity has been assessed with a Service Affecting (SA) impact. Specifically, during the first 60 minutes of this six-hour maintenance activity, all NIHnet data network connectivity between the Building 12 ---- Sterling, VA Data Centers will be down while Qwest repairs their SONET equipment located in Building 45. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Qwest Building 12 to Sterling VA SONET Equipment Maintenance has been completed.
7/17/2007 7:42:20 AM - IMS is Currently Unavailable.
Description:   IMS (Information Management System) is currently Unavailable. Users are receiving the following error message "EMS1164E Pass mode logon request failed for application IMS". IMS administrators are aware of the issue and they are currently working on a resolution. There is currently no estimated time of resolution. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  This issue has been resolved and IMS is functioning properly again.
7/16/2007 2:28:37 PM - NIHnet Building 12 Wireless DMZ Device Relocation Maintenance
Description:   Date: Saturday, July 28, 2007 Time: 9:00am - 5:00pm ET Description: On Saturday, July 28, 2007 at 9:00am - 5:00pm ET NIHnet engineers will perform a maintenance activity to relocate the Building 12 NIHnet Wireless DMZ path router and firewall to a new cabinet located in the Building 12 NIH Data Center. Impact: Please note the NIHnet Building 12 Wireless DMZ Device Relocation Maintenance will cause a momentary interruption in Wireless connectivity to active Wireless Guest Users only. The NIHnet Wireless Service WILL NOT be affected by this maintenance activity because NIHnet Wireless access and VPN connectivity is supported by a redundant path via the Building 45 Wireless DMZ infrastructure. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Building 12 Wireless DMZ maintenance was completed successfully and services were verified as operational at 6:07 PM.
7/16/2007 11:27:53 AM - Sterling-6500-PRI Switch Operating System Upgrade Maintenance
Description:   Description: On Thursday, July19, 2007 at 5:00pm - 7:00pm NIHnet engineers will proactively upgrade the Sterling-6500-PRI Switch Operating System to the prescribed revision level. Impact: Please note this Sterling-6500-PRI Switch Operating System Upgrade Maintenance activity has been assessed with a Service Affecting (SA) impact. Specifically, during the first 15 minutes of this 2 hour maintenance activity, only VLANS that are connected to the Sterling-6500-PRI Switch located in the Sterling NCSS Data Center will be impacted. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The maintenance was completed successfully with no affected devices.
7/13/2007 4:36:24 PM - Building 31 NIHnet Access Switch GigE Upgrade Maintenance
Description:   Date: Saturday, July 21, 2007 Time: 8:00am - 9:00am Description: On Saturday, July 21, 2007 at 8:00am - 9:00am NIHnet engineers will install a 16-port GigE module in the Building 31 NIHnet Access switch. Impact: Please note this Building 31 NIHnet access switch GigE port capacity upgrade maintenance has been assessed with a Service Affecting (SA) impact. Specifically, during the first 15 minutes of this one hour maintenance activity, Building 31, Children's Inn and the Bldg 15 complex will lose network connectivity to NIHnet while the B31-6500 Access Switch is reloaded with the new image. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  As of 10:00 AM, the Building 31 NIHnet Access Switch upgrade has been successfully completed.
7/13/2007 8:44:10 AM - Building 10A/10B Electrical Power Systems Test
Description:   On Saturday, July 14, 2007 at 5:30am - 10:00am (10B) and 11:00AM - 3:00PM (10A) the facility department will be conducting Emergency Power Systems testing. http://orf2.od.nih.gov/utilityshutdown/scripts/utility_shutdown.asp?seq=7628 (10A) http://orf2.od.nih.gov/utilityshutdown/scripts/utility_shutdown.asp?seq=7627 (10B) Impact: The Building 10A and 10B Electrical Power Systems Test Maintenance activities are not expected to impact the Building 10 NIHnet data network infrastructure equipment because all Core, Distribution, and Access data network devices are supported by sufficient UPS and / or generator back-up power. However, there may be limited Wireless Network connectivity at various locations within Building 10 due to a limited number of Wireless Access Points (AP) using commercial power as a sole power source. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The test of the Building 10A/10B electrical power systems has been successfully completed.
7/11/2007 8:09:38 AM - CIT Customer Support Web Server is down
Description:   The CIT web server is currently experiencing issues. The following sites are affected: Knowledge Base NIH Help Desk support site CIT Training page Remote Access site CIT Reports site Publications site Stat Soft site EMIB is currently investigating the issue; no estimated time of resolution is available at this time. Updates will be posted as they become available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The CIT Customer Support Web Server issues have been resolved.
7/10/2007 11:44:07 AM - NIHnet Remote Access ISDN Service Turndown
Description:   Description: Effective July 16, 2007, the NIH will discontinue the ISDN remote access service. This is different from the NIH low-speed PARACHUTE dialup service. If you still use the NIH PARACHUTE dialup service, you will not be impacted by this maintenance. Because of the prevalence of high-speed DSL and cable Internet services, there are only a few users who still access the NIH via ISDN. The users who have used ISDN in the past 12 months have been contacted and have obtained alternate methods to access the NIH remotely and the Internet. Impact: No user impact is anticipated . Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The NIH ISDN remote access service has been discontinued as planned. If you have any questions please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov.
7/10/2007 7:36:24 AM - NBS not functioning
Description:   NBS is unavailable as of 10:50 AM. Expected return to service is 11:30 AM. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Time: 1:29PM NBS is now functioning and all systems are back online.
7/5/2007 4:54:28 PM - CIT Remedy Server Reboot
Description:   At approximately 5:30 PM on Saturday July 7th, the CIT Remedy server will be rebooted to apply a security feature. This will result in a brief (less than five minute) outage for the Remedy service. A brief period of unavailability or inconsistent behavior may also occur on services that rely on Remedy for data, including but not limited to the CIT Training site, the IT Help Desk site, the CIT Maintenance and Outage Calendar and other similar sites. We appreciate your patience and support while we work to ensure the security and continued functionality of the Remedy service. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The CIT Remedy server was rebooted with no complications.
7/5/2007 11:21:33 AM - NIH ADM Service Experiencing Difficulties
Description:   NIH ADM services are having problems loading the NIH.GOV domain objects. The service administrators are working on the problem and expect it to be resolved shortly. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The NIHADM and NIHADM2 boxes had to be recycled and were restored to service as of 11:26 AM. These boxes will have to be rebooted again at around 1:00 PM.
7/2/2007 1:48:27 PM - Connect Downtime for Service Pack
Description:   Connect will be down from 10pm until 10:30pm EST to apply Adobe Connect Service Pack 2. This service pack will supply bug fixes and add the option of having time stamps on the chat pod. If you have any questions, please send them to: webmeeting@mail.nih.gov. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The service pack was applied successfully.
7/2/2007 9:20:53 AM - Unable to access the NIH Application Manager (NAppMan).
Description:   An automated process is accessing the NAppMan web page on a regular basis obtaining all available concurrent licenses. Anyone attempting to access NAppMan will receive a "Too many users are currently logged in." message. If you already have access to NAppMan, you are unaffected by this problem. NAppMan will continue to work properly (i.e., send alerts, perform escalation, store uptime statistics, etc.) while this problem exists. The NAppMan team is working with the owners of the application that is causing the problem. ETA for solution: Probably no more than a business day. It will likely be less.
Resolution:  The application was found and the problem has been resolved. NAppMan is once again available to anyone within the NIH network.
6/29/2007 5:22:12 PM - 6120 Executive Blvd. HVAC Maintenance
Description:   OD/ORF will be conducting an HVAC maintenance in the 6120 Executive Plaza building from Friday, June 29 at 6:00 PM until Sunday, July 1 at 6:00 PM. This maintenance has been assessed as Service Affecting (SA). Network devices will be powered down due to loss of AC during the maintenance, resulting in a loss of wireless network connectivity for NIDCD users in the 6120 facility. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The maintenance has been completed successfully without complications.
6/29/2007 8:53:03 AM - Security Awareness Site Issues
Description:   The NIH Computer Security Awareness Training web site is currently not able to obtain student record data. NIH customers will be able to log into the site with their NED ID but the system is not able to display that customer's student record information. There are no error messages displayed. The service provider is working to restore the service to full operation. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Securtiy Awareness Site is now functioning properly.
6/27/2007 4:40:17 PM - NBS Unavailable Due to Hardware Issues
Description:   The NIH Help Desk has been informed that the NBS (all business areas) is unavailable due to a hardware issue. The NMC and CIT staff are working on resolving this issue. This Hot News will be updated as additional information becomes available.
Resolution:  The NBS hardware issues have been resolved. NBS is now up and available
6/27/2007 1:45:44 PM - Network Outage in Building 35
Description:   We are currently experiencing a network outage in Building 35 that is affecting network services. NIHnet engineers are onsite and are diligently working to resolve the problem. More information will be provided as it becomes available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The network outage affecting Building 35 has been resolved and all network connectivity has been restored.
6/26/2007 7:41:11 AM - NIH Remote Access VPN Replacement Maintenance
Description:   Date: Sunday, July 15, 2007 Time: 6:00am - 7:00am ET Description: On Sunday, July 15th at 6:00 AM, CIT will replace the general pool of IP addresses used by two of the NIH Remote Access VPN Servers. This task is being performed in order to enhance and refine the distribution of NIHnet IP addresses. To accomplish this task, some addresses used by the NIHnet Remote Access VPN service must be re-assigned. This change will not impact existing IC-specific VPN group IP address assignments. To complete this task with as little disruption of Remote Access VPN services as possible, CIT will distribute the old and new IP addresses to just the ISSO for each Institute and Center and the Incident Response Team (IRT). Please contact your ISSO if you have firewall or servers that utilize the general IP addresses assigned to remote access VPN users. Impact: This NIH Remote Access VPN Replacement Activity has been given a possible service affecting-low probability impact assessment (PSA-Low) because there is a very low potential for Remote Access VPN users to experience a service disruption. This change will NOT impact existing IC-specific VPN group IP address assignments. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The maintenance was completed successfully at 6:12 AM.
6/24/2007 10:21:16 AM - EOS system unavailable
Description:   DESCRIPTION: The EOS Server and databases are experiencing downtime due to a performance issue. System Administrators are working toward a resolution. Currently there is no estimated time of resolution. Updates will be sent as they become available. IMPACT: Users may experience performance or slowness issues when attempting to use the services listed below. EOS affects the following systems: ---NIH Customers AFPS (Administration and Finance Payroll System) DOCR (Dental, Oral and Craniofacial Research System) ERA CRISP production database only NCI Portal Life Ray Portal for NCI NIDB (NIH Intramural Database) NIGMS NIGMS Oracle Database Application NIH EHRP (NIH Enterprise Human Resources and Payroll Project) SCSONDEMAND (IMPAC Extensions) EGG (IntraMall) NED NIH Employee Directory HHSWEB - Web Content Management (CMS) ---Non-NIH customers ACF Administration for Children and Families HHS Content Management System (HHSCMS) Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  System is up and available for use, and all databases have been restarted.
6/22/2007 1:40:05 PM - NIHnet CRC-6500-8D Module 3 Reseat Maintenance
Description:   Description: On Friday, July 6, 2007 between 2:00am - 3:00am ET NIHnet engineers will perform a maintenance activity to reseat module 3 in the CRC-6500-8D switch in an effort to eliminate recurring error messages on the daily Router Log Report. Impact: Please note this NIHnet CRC-6500-8D Module Reseat Maintenance has been assessed with a Service Affecting (SA) impact. Specifically, this maintenance activity will cause a brief network disruption (<15 min) to users connected to ports on module 3 of the CRC-6500-8D switch during the reseating process. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The NIHnet CRC-6500-8D Module 3 Reseat Maintenance was successfully completed.
6/21/2007 2:41:37 PM - Building 12 NIH Data Center Network Maintenance Activity
Description:   On Saturday, June 30, 2007 between 8:00am - 12:00pm ET NIHnet engineers will perform a scheduled field trial maintenance activity in the Building 12 NIH Data Center on a select list of Blade Server devices to improve network switching performance and reliability. Impact: This Building 12 NIH Data Center Network Maintenance Activity has been given a possible service affecting-high probability impact assessment (PSA-High) because during the maintenance window it will be necessary to reload each target field trial BNT switch twice to bring each switch up to the prescribed operating system revision. Therefore, there is the potential for the server blades that are installed in each of the target field trial BNT switch chassis to experience a momentary loss of connectivity to the NIHnet Building 12 Data Center network while each BNT field trial switch is reloaded. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Building 12 NIH Data Center Network Maintenance was completed successfully.
6/19/2007 5:14:40 PM - Replication of all NIH.GOV domain controllers
Description:   On June 29, 2007, EMIB will be enabling strict replication on all nih.gov domain controllers. No application or services are impacted by this change.
Resolution:  The strict replication on nih.gov domain controllers has been enabled.
6/18/2007 4:57:20 PM - Scheduled Maintenance Window NIH Login Production, June 19, 2007
Description:   NIH Login Production is scheduled to have a small maintenance window tomorrow night, Tuesday June 19th at 9:00PM. The maintenance will last approximately 30 minutes in order to clear some hardware errors that are being seen on one of the NIH Login F5s. The error may be a false positive and the errors are fully expected to go away. There is no expected downtime due to redundant servers for both NIH Login and F5s. Users might experience a small delay if they happen to log on when the service fails over to the secondary F5. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The maintenance on NIH Login Production was successfully completed by 10:30 PM on Tuesday, June 19.
6/18/2007 11:43:46 AM - DNS referral cleanup
Description:   Event Description: On June 20th, CIT EMIB will be coordinating the removal of specific internal DNS referrals (_msdcs, _sites, _tcp, _ucp) between the External BIND DNS and Border DDNS. The operational reasons for them to be listed in BIND no longer exist and they impede the progress of more than one project. This is not expected to cause any problems as this is a referral only. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The scheduled removal of internal DNS referrals has been completed.
6/18/2007 9:20:46 AM - NIHnet Building 12 OpDiv DMZ Device Relocation Maintenance
Description:   Description: On Saturday, June 30, 2007 at 9:00am - 5:00pm ET, NIHnet engineers will perform a scheduled maintenance activity to relocate the Building 12 OpDiv DMZ path router and firewall to a new cabinet located in the Building 12 NIH Data Center. Impact: Please note this NIHnet Building 12 OpDiv DMZ Device Relocation Maintenance has been assessed with a Service Affecting (SA) impact for a limited number of outside vendor and contractor connections to NIHnet. Also, this maintenance activity has been assessed with a PSA-Low impact for HHSnet, NLM, and HRSA connectivity to NIHnet due to redundant connections that will not be affected by this device relocation maintenance activity. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The maintenance was completed successfully with no complications.
6/15/2007 11:54:05 AM - Connect Downtime
Description:   The Connect (formerly Breeze) services will go down tonight at 6:00PM for 15 minutes to correct a known issue with Connect. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or helpdesk@nih.gov if you have any questions.
Resolution:  Time: 6:13PM The Connect server maintenance has been completed successfully.
6/13/2007 12:15:35 PM - Power Maintenance In 6100 Executive BLVD on Saturday, June 16
Description:   On Saturday, June 16, 2007, Pepco will shutdown the power at 6100 Executive Blvd from 7am to 5pm for maintenance. Impact: IT services for that will be impacted during this maintenance include: --All ISDN phones will be unavailable --NIHnet wireless network services will be unavailable --OD/OIT will shutdown local network resources, so OD Staff in Bldg 6100 will not have access to the internet, email, home directory servers and applications from 6am to 6pm. If there are additional services to be impacted, or if you have additional questions or concerns about this utility maintenance activity, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov.
Resolution:  The power maintenance was completed successfully and all devices have been restored.
6/13/2007 10:36:48 AM - NIHnet B12-7600-Inside-DMZ Router Operating System Upgrade
Description:   On Tuesday, June 19, 2007 between 6:00 p.m. - 8:00 p.m. ET NIHnet engineers will proactively upgrade the B12-7600-Inside-DMZ router IOS version to the prescribed revision level to address a Cisco identified crafted IP vulnerability. Impact: This NIHnet B12-7600-Inside-DMZ Router Operating System Upgrade Maintenance Activity has been given a possible service affecting-low probability impact assessment (PSA-Low). Specifically, this maintenance activity is not expected to disrupt NIHnet Internet traffic because traffic will be traversing the NIHnet Internet DMZ via its redundant Fernwood path. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Time: 6:39PM The maintenance was completed successfully without complications.
6/12/2007 3:59:09 PM - Rebooting Server NIHCESPUB7
Description:   Technicians found an issue with server NIHCESPUB7 which will require a reset of the computer services this evening at 9:00pm. The expected downtime is less than 10 minutes. The public folders will be affected during this window for the following IC's: NCCAM NCMHD NCRR NEI NHGRI NIA NIAID NIDCR OD (part) VRC Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The server was rebooted successfully at 10:12PM. All services were fully restored at 10:18PM.
6/11/2007 10:33:06 AM - Phone Outage 6011 Executive Blvd 3rd floor
Description:   The 3rd floor of 6011 Executive Blvd is currently experiencing a telephone outage. Technicians were dispatched to the site to repair the issue, there is currently no estimated time of resolution. Updates will be posted as soon as they become available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Phone service has been restored.
6/10/2007 9:21:47 PM - Building 12 Data Center Access Switch Module Failure
Description:   Building 12 Data Center access switch B12-6500-DCD Module 4 has failed. NIHnet Engineers are currently working to restore service. Those affected are currently being notified. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Time: 9:32pm The module is back up and the connections have been restored.
6/8/2007 10:57:01 AM - Building 12 NIH Data Center Redesign Phase V Part 1 Maintenance
Description:   On Saturday, June 16, 2007 between 8:00am - 12:00pm ET NIHnet engineers will perform the first part of a data network maintenance activity to support phase V of the NIH Building 12 Data Center Network Redesign Project. This maintenance activity will focus on moving systems associated with network management and NIHnet infrastructure in the Building 12 NIH Data Center to a new network firewall infrastructure. Impact: This NIH Building 12 Data Center Network Redesign Project Maintenance Activity has been given a possible service affecting-low probability impact assessment (PSA-Low) because there is a very low potential for the target systems to experience a service disruption as each grouping of systems (i.e. VLAN) is moved to the new network firewall infrastructure. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Building 12 Phase V, Part 1 Maintenance was unsuccessful and has been backed out. All services are online and operational at this time.
6/7/2007 7:23:57 AM - B12-6500-Core Router Reload Maintenance
Description:   On Saturday, June 9, 2007 at 2:00am - 3:00am NIHnet engineers will be performing an emergency reload of the Building 12 B12-6500-Core Router to address a chronic video services issue. Impact: Please note this B12-6500-Core Router Reload Maintenance activity has been assessed with a Service Affecting (SA) impact. Specifically, during the first 15 minutes of this 1 hour maintenance activity network connectivity will be disrupted between the NIHnet <----> Sterling, VA Data Centers while the Building 12 B12-6500-Core Router reloads. Please note this B12-6500-Core Router Reload Maintenance activity will not affect NIHnet connectivity to NLM, Internet, or Internet(2) via the NCCS Data Center in the Qwest Sterling, VA. facility. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The B12-6500-Core Router Reload Maintenance was completed successfully.
6/6/2007 2:51:52 PM - Access to NIH Directory and Email Forwarding Service
Description:   On Friday, June 8th at 10:00 AM outside access from the Internet to the NIH Directory and Email Forwarding Service at http://directory.nih.gov will be restricted. This website and related services will continue to be available from within NIHnet. Please contact the NIH Help Desk at helpdesk@mail.nih.gov or 301-496-4357 if you have any questions or concerns.
Resolution:  This notification served as a means of getting the required information out to the appropriate people. This Hot News is being closed.
6/6/2007 2:41:21 PM - Building 12 to Baltimore T3 Outage
Description:   The T3 connection between Building 12 and Baltimore went down at 2:10 PM due to a power issue at the Baltimore GRC building. The circuit is back up and all network connectivity has been restored. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The circuit is back up and all network connectivity has been restored.
6/6/2007 12:11:03 PM - Widespread issue accessing ITAS
Description:   ITAS is currently experiencing connectivity issues. We apologize for any inconveniences this may cause you and appreciate your patience while we work to resolve the problem. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions
Resolution:  The ITAS system is now functioning normally.
6/6/2007 6:43:09 AM - ADB System is unavailable
Description:   At this time the ADB system is experiencing some issues. The engineering team is aware of the situation and is working towards a resolution. Updates will be posted as they become available.
Resolution:  The ADB System is back up and fully functional.
6/5/2007 8:23:59 AM - NIHnet Fernwood-6500-Inside-DMZ Router OS Upgrade Maintenance
Description:   Subject: NIHnet Fernwood-6500-Inside-DMZ Router OS Upgrade Maintenance Date: Tuesday, June 12, 2007 Time: 6:00pm - 9:00pm ET Description: On Tuesday, June 12, 2007 at 6:00pm - 9:00pm NIHnet engineers will be performing an upgrade of the Fernwood-6500-Inside-DMZ router IOS version to the prescribed revision level. Impact: This NIHnet Fernwood-6500-Inside-DMZ Router OS Upgrade Maintenance activity has been assessed with a possible service affecting low probability (PSA-Low) and is not expected to disrupt NIHnet Internet traffic due to redundant connectivity via the Building 12 NIHnet Internet DMZ path. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The maintenance was completed successfully at 6:58 PM.
6/4/2007 3:17:41 PM - Building 12 NIH Data Center Network Maintenance Activity
Description:   Date:  Saturday, June 9, 2007   Time: 8:00am - 12:00pm ET   Description: On Saturday, June 9, 2007 between 8:00am - 12:00pm ET NIHnet engineers will perform a scheduled field trial maintenance activity in the Building 12 NIH Data Center on a select list of Blade Server devices to improve network switching performance and reliability.   Impact: This Building 12 NIH Data Center Network Maintenance Activity has been given a possible service affecting-high probability impact assessment (PSA-High) because during the maintenance window it will be necessary to reload each target field trial BNT switch twice to bring each switch up to the prescribed operating system revision.  Therefore, there is the potential for the server blades that are installed in each of the target field trial BNT switch chassis to experience a momentary loss of connectivity to the NIHnet Building 12 Data Center network while each BNT field trial switch is reloaded.   Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Saturday, June 9, 2007 Building 12 NIH Data Center Network Maintenance Activity has been postponed and will be rescheduled for a later date.
6/1/2007 2:26:48 PM - Poolesville WAN Connectivity Optimization Maintenance
Description:   On Wednesday, June 6, 2007 at 5:00 p.m.-7:00 p.m. ET, NIHnet Engineers will perform a scheduled maintenance to optimize network routing and increase WAN bandwidth from three (3) T-1 circuits to (4) T-1 circuits between Poolesville and the NIH campus. Impact: Please note this Poolesville WAN Connectivity Optimization Maintenance activity has been assessed with a Service Affecting (SA) impact. Specifically, Poolesville end-users will intermittently experience a complete loss of data network connectivity to NIHnet during the first 30 minutes of the two-hour maintenance window. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  At 6:19 PM, DNST reports that the maintenance was completed successfully without complications.
5/30/2007 4:16:30 PM - Doublehelix System Currently Unavailable
Description:   The Doublehelix system (doublehelix.nih.gov) is currently unavailable due to a hardware malfunction. Technicians are aware and are working to resolve the issue.
Resolution:  As of 4:34 PM, the Doublehelix system is available and stable. Connectivity outages were caused by an arp-table bug in the load-balancing switch software. After the poisoned entries timed out, connectivity was restored.
5/30/2007 3:17:06 PM - Intermittent Delays on some websites
Description:   NIH employees are experiencing intermittent delays browsing some Internet websites. These delays do NOT impact any public access to NIH resources. NIHnet engineers have isolated the source of the delays and are executing a workaround while further analysis is conducted.
Resolution:  As of 4:05 PM, the workaround was replaced with a more stable solution. The solution required a modification to the load balancing algorithm. The new algorithm should distribute load more evenly across the various network devices.
5/25/2007 2:38:19 PM - Executive Boulevard Telephone Outages
Description:   NIH users in Executive Boulevard locations are experiencing random telephone outages due to a cut cable. Verizon staff are working to resolve the problem, but there is currently no estimated time of repair. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  All reported telephone outages at 6011 Executive Blvd. have been restored.
5/24/2007 3:03:13 PM - NIHnet Listserv Emergency Maintenance
Description:   Description: On Friday, May 25, 2007 between 8:00am - 9:00am ET NIHnet engineers will perform an Emergency Maintenance to replaced a failed redundant power supply and one failing DIMM (memory module)on the NIH Listserv server. Impact: Please note that the list.nih.gov website will not be available during the first 30 minutes of this one-hour maintenance activity. In addition, all undelivered emails that were queued during this maintenance activity will be scheduled for delivery once the NIHnet Listserv server is back online. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The NIHnet Listserv Emergency Maintenance was completed successfully.
5/22/2007 3:20:42 PM - Latency when visiting External Websites
Description:   Description: Internet Latency Impact: Users are experiencing internet latency when visiting external websites. NIHNet Engineers are investigating the issue and more details will be posted when they become available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  CIT engineers were able to implement a solution for the latency problems at around 8:00 PM on Tuesday, May 22.
5/18/2007 1:48:17 PM - UFMS Firewall Infrastructure Optimization Maintenance
Description:   Date: Saturday, May 19, 2007 Time: 7:30am - 8:30am ET Description: On Saturday, May 19, 2007 from 7:30am - 8:30am NIHnet data and security network engineers will optimize the Building 12 NIH Data Center firewall security infrastructure that supports the Unified Financial Management System (UFMS) application service. Impact: This UFMS Firewall Infrastructure Optimization Maintenance activity has been assigned a service affecting (SA) impact assessment because there will be intermittent and brief (<5 minutes) network connectivity disruptions to the UFMS application service while data and security network engineers upgrade firewall operating systems; enable new switch ports; and activate new VLANs on the UFMS Firewall Infrastructure. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Due to physical limitations of the switch port and firewall, portions of the UFMS Firewall Infrastructure Optimization maintenance were backed out.
5/16/2007 9:57:23 AM - Building 4 Telephone Outage
Description:   Building 4 is experiencing a telephone outage, technicians have been notified. There is no current estimated time of resolution.
Resolution:  All phone services in Building 4 have been restored.
5/15/2007 1:33:07 PM - NIHnet CRC-6500-8D Module Installation Maintenance
Description:   Date: Wednesday, May 23, 2007 Time: 2:00am - 3:00am ET Description: On Wednesday, May 23, 2007 between 2:00am - 3:00am ET NIHnet engineers will perform a maintenance activity to proactively install a 48 port expansion module in the CRC-6500-8D switch in an effort to support port augmentation. Impact: Please note this NIHnet CRC-6500-8D Module Installation Maintenance has been assessed with a Service Affecting (SA) impact. Specifically, this maintenance activity will cause a brief network disruption (<15 min) during the planned reload of the CRC-6500-8D switch after the 48 port expansion module is installed. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The maintenance was completed successfully at 2:28 AM.
5/14/2007 2:21:40 PM - Parachute Dial-Up Security Enhancement Upgrade
Description:   Date: Wednesday, May 16, 2007 Time: 7:30AM Description: On Wednesday May 16th at 7:30 am, CIT Division of Network Systems and Telecommunications Engineers will apply a security enhancement to the Parachute dialup network routers. This change is not expected to be service affecting. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The maintenance was completed successfully.
5/12/2007 5:02:07 AM - Fernwood 6500-a Module 9 Emergency Maintenance
Description:   On Saturday May 12, 2007 at 3:01 AM a hardware replacement maintenance activity was successfully completed on Fernwood 6500-a Module 9. If any connectivity issues still exist please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov.
Resolution:  Hardware replacement maintenance activity was successfully completed on Fernwood 6500-a Module 9 at 3:01AM on Saturday, May 12th.
5/12/2007 12:05:02 AM - Emergency Scheduled Maintenance on NIH Central Email Service
Description:   Date: Friday, May 11, 2007 Scheduled Time: 10:00 PM - 12:00 PM Description: On Friday, May 11th between 10:00 PM and 11:00 PM there was emergency maintenance performed by NIH CES Administrators on the NIH Central Email Service. The maintenance was successfully completed at 11:00 PM on Friday, May 11th. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The maintenance was successfully completed at 11:00 PM on Friday, May 11th.
5/11/2007 1:53:10 PM - eRA Firewall Scheduled Maintenance
Description:   Date: Saturday, May 12, 2007 Time: 7:00pm - 7:30pm ET Description: On Saturday, May 12, 2007 from 7:00pm - 7:30pm NIH Network Engineers will upgrade the firewalls supporting eRA services. This maintenance will increase network capacity improving eRA services to the NIH community. Impact: During the maintenance window, intermittent connectivity disruptions will occur between eRA servers in Building 12 and other locations. This maintenance activity is scheduled within the regularly scheduled eRA maintenance window. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The eRA firewall maintenance has been backed out.
5/9/2007 2:19:53 PM - Testing Permissions--Disregard
Description:   Test. Disregard.
Resolution:  Testing Complete. Closed.
5/9/2007 1:03:41 PM - NIHnet B10-6500-Core Router Reload Maintenance
Description:   Date: Thursday, May 17, 2007 Time: 2:00am - 3:00am ET Description: On Thursday, May 17, 2007 between 2:00am - 3:00am ET NIHnet engineers will perform a proactive router reload maintenance activity on the NIHnet Building 10 Core Router. Impact: There is low potential for this router reload maintenance activity to affect NIHnet backbone connectivity because all connections via the Building 10 Core Router are also supported by redundant connections via the Building 12 Core Router. Therefore, there is a very low expectation that end-users will see any network service disruption while the Building 10 Core Router is reloaded during the first 30 minutes on this one-hour maintenance window. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The NIHnet B10-6500-Core Router Reload Maintenance was successfully completed at 3:00 AM on Thursday, May 17th, 2007.
5/4/2007 12:39:40 PM - Building 10 Emergency Generator Systems Test
Description:   Date: Saturday, May 12, 2007 Time: 7:00am - 12:00pm Description: On Saturday, May 12, 2007 at 7:00am - 12:00pm there will be an Emergency Generator Systems test in Building 10 and the ACRF facilities. http://orf2.od.nih.gov/utilityshutdown/scripts/utility_shutdown.asp?seq=7398 Impact: The Building 10 Emergency Generator Systems Test Maintenance activity is not expected to impact the Building 10 or ACRF facilities’ NIHnet data network infrastructure equipment because all Core, Distribution, and Access data network devices are supported by sufficient UPS and/or generator back-up power. However, there may be limited Wireless Network connectivity at various locations within Building 10 and Building 10B due to a limited number of Wireless Access Points (AP) using commercial power as a sole power source. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The generator systems test was completed and all systems are now online and operational.
5/2/2007 10:25:24 AM - NIHnet Bldg 12 OpDiv Perimeter Router OS Upgrade Maintenance
Description:   Date: Tuesday, May 8, 2007 Time: 6:00 p.m. - 9:00 p.m. Description: On Tuesday, May 8, 2007 at 6:00 p.m.- 9:00 p.m. ET, NIHnet engineers will perform a scheduled maintenance activity to upgrade the B12-6500-OpDiv router operating system to mitigate a known low threat crafted TCP vulnerability. Impact: Please note this NIHnet Bldg 12 OpDiv Perimeter Router OS Upgrade Maintenance has been assessed with a Service Affecting (SA) impact for a limited number of NIH remote-site vendor and contractor connections. Also, this maintenance activity has been assessed with a PSA-Low impact for HHSnet, NLM, and HRSA connectivity to NIHnet. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  NIHnet Building 12 OpDiv Perimeter Router Operating System Upgrade Maintenance - Phase II, was completed successfully at 8:30 PM.
4/27/2007 4:37:09 PM - Building 10 Distribution Switch Emergency Reload Maintenance
Description:   Date: Friday, April 27, 2007 Time: 8:00pm - 9:00pm Description: On Friday, April 27, 2007 at 8:00pm to 9:00pm, an emergency device reload maintenance will be performed on the Building 10 B10-6500-B Distribution Switch. This emergency device reload maintenance is necessary to proactively eliminate an existing high CPU utilization condition. Impact: The Building 10 B10-6500-B Distribution Switch Emergency Reload Maintenance activity will cause a brief (<30 minutes) connectivity disruption to the IC’s listed below: B7-B10-NEI B10-OD/DES/SCADA B10-NIDCD B10-NHLBI B10-NIDCR B10-NIMH B10-NINDS B10-NIEHS/CIT/SLA B10-NCRR B10-NCI B82-NCI B10-NICHD B10-OD-ORS CIT-TIB (CIT Telephone Infrastructure Branch) CIT NSS (CIT NSS) Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Date: Friday, April 27, 2007 The Building 10 Distribution Switch Emergency Reload maintenance was completed successfully at 8:23PM.
4/27/2007 6:55:39 AM - Network Service Interrruption
Description:   We experienced a brief network service interrruption between the NIH Campus and the Sterling data center. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The connection between the NIH Campus and the Sterling Data Center has been restored and is stable at this time. Services have been up and stable since 6:29 AM. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
4/26/2007 6:41:51 PM - Network Issues with VIDEO, BREEZE, and SERCH Applications
Description:   Date: April 26, 2007 DNST is currently working to resolve network issues affecting VIDEO, BREEZE, and SERCH applications. Engineers expect to have these issues resolved within the 30 minutes. Updates will be provided as they become available.
Resolution:  Date: April 26, 2007 Time: 6:50PM Network connectivity was restored to affected servers. All services have been restored and access to aplications tested.
4/26/2007 7:15:48 AM - Network connectivity Issues-OD Users Executive Blvd 2nd Floor
Description:   OD users are experiencing network connectivity issues the incident as of right now has been isolated to 2nd floor users only. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Network connectivity has been restored. Issue has been resolved.
4/24/2007 10:37:47 PM - eRA Service Downtime
Description:   On Friday, April 27 from 6:00 PM until 9:00 PM EST the eRA system will undergo a planned downtime for system deployment. eRA Commons will be unavailable for the extent of the downtime. Also, all IMPAC II systems will be unavailable from 8:00 PM to 9:00 PM on the same evening. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Date: Friday, April 27, 2007 The eRA system deployment was completed successfully.
4/24/2007 7:49:05 PM - Emergency Maintenance on the CIT Remedy Server
Description:   The Support Technologies Group (STG) needs to conduct emergency maintenance on the Remedy Server. The server will be offline from 5:00 AM to 5:30 AM on April 25th. During this time STG personnel will make some necessary changes to the Remedy System. We apologize for any inconvenience this may cause.
Resolution:  The emergency maintenance of the CIT Remedy server was successfully completed as of 5:16 AM.
4/24/2007 3:45:37 PM - Scheduled maintenance for Nimbus
Description:   The scientific computer Nimbus, which is part of the Helix Systems, is scheduled to undergo maintenance for the replacement of a defective memory board on Wednesday April 25 at 7:00 AM. It is anticipated that the system will be unavailable to users for no more than an hour while the maintenance is being performed. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The maintenance on Nimbus was successfully completed at 7:20 AM on Friday, April 27th. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
4/24/2007 7:22:20 AM - Building 37 Power Outage
Description:   There is currently a major power outage in Building 37 affecting approximately half of the building. It is expected that a significant proportion of Building 37 users will be unable to establish network connectivity due to dropped LAN connections as a result of the power loss. There is currently no estimated time of repair. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The power outage in Building 37 was repaired over the weekend. All systems should now be functioning as normal.
4/23/2007 1:45:50 PM - Building 12 NIH Data Center Access Switch Uplink Maintenance
Description:   Date:  Saturday, May 5, 2007 Time:  6:00am - 11:00am ET   Description: On Saturday, May 5, 2007 between 6:00am - 11:00am ET NIHnet engineers will perform an uplink connectivity activation maintenance on the Building 12 NIH Data Center access switches. This maintenance activity is in support of continuing efforts to implement the Building 12 NIH Data Center Re-Design. Therefore, it is now necessary to establish Layer 2 connectivity between the existing access switches and the new data center routers. Impact: This Building 12 NIH Data Center Access Switch Uplink Maintenance activity has been assigned a possible service affecting - low probability (PSA-Low) impact assessment because there is a very low expectation for this maintenance activity to cause any adverse impact to the Building 12 NIH Data Center data network infrastructure during the five-hour maintenance window. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Maintenance Completed Successfully
4/23/2007 11:06:42 AM - Approximately 10 ISDN Phone Lines Down in Building 12A
Description:   Date: April 23, 2006 Description: There are approximately 10 phone lines down in Building 12A. The technicians are aware of the situation and are working towards a resolution. At this time there is no estimate time of resolution. Updates will be posted as they become available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  As of noon on Tuesday, April 24, Verizon technicians were able to successfully restore service to the affected phone lines in Building 12A.
4/21/2007 10:45:20 AM - NIHCESMLBX6 will be cycled at 11:00 AM
Description:   Date: Saturday, April 21, 2007 Time: 11:00 AM Description: NIHCESMLBX6 will be cycled at 11:00 AM on Saturday April 21, 2007 to facilitate the emergency configuration of critical security software components. The IC's affected by this will be NHLBI and NIEHS. The cycle should take about 15 minutes. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The work on NIHCESMLBX6 is now complete and all services are 100% restored.
4/20/2007 10:40:32 AM - Building 10 Emergency Power Systems Test
Description:   Date: Saturday, April 28, 2007 Time: 7:00am - 12:00pm Description: On Saturday, April 28, 2007 at 7:00am - 12:00pm there will be an Emergency Power Systems test in Building 10 and the ACRF facilities. http://orf2.od.nih.gov/utilityshutdown/scripts/utility_shutdown.asp?seq=7373 Impact: The Building 10 Emergency Power Systems Test Maintenance activity is not expected to impact the Building 10 or ACRF facilities’ NIHnet data network infrastructure equipment because all Core, Distribution, and Access data network devices are supported by sufficient UPS and / or generator back-up power. However, there may be limited Wireless Network connectivity at various locations within Building 10 and Building 10B due to a limited number of Wireless Access Points (AP) using commercial power as a sole power source. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Building 10 Emergency Generator Test that was scheduled for Saturday, April 28, 2007 has been officially canceled by ORF.
4/19/2007 8:10:33 PM - FACnet in Building 10 is Offline
Description:   Date: Thursday, April 19, 2007 Description: FACnet is offline in Building 10 causing FACnet outages. Engineers are currently working to resolve the issue. There is no estimated time of resolution. Updates will be sent as they become available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Date: Thursday, April, 19, 2007 Time: 9:10PM The NIHnet engineers have resolved the Building 10 FACnet issues. All devices are up and the network is stable.
4/19/2007 3:07:15 PM - Parklawn Power Systems Maintenance
Description:   Subject: Parklawn Power Systems Maintenance Start Date: Saturday, April 21, 2007 Start Time: 6:00pm End Date: Sunday, April 22, 2007 End Time: 6:00am Description: On Saturday, April 21, 2007 starting at 6:00pm and continuing until Sunday, April 22, 2007 6:00am ET, PEPCO will conduct a scheduled electrical shutdown of the Parklawn building at 5600 Fishers Lane to perform maintenance on the high voltage electrical switchgear within this building. Impact: Please note this Parklawn Power Systems Maintenance has been assessed with a Service Affecting (SA) impact. Specifically, this maintenance activity will result in a loss of OPDIV and downstream IC data network connectivity throughout the duration of the 12-hour maintenance window. In addition, the IC Point-of-Contacts have been contacted and made aware of this impending maintenance activity. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Parklawn Power Systems Maintenance has been successfully completed.
4/19/2007 10:14:32 AM - ITAS Maintenance, Apr 24, 6:00pm-6:30pm
Description:   ITAS is scheduled for maintenance (partial release) on Tuesday, April 24 from 6:00PM - 6:30PM. During this time ITAS will not be available. We thank our users for their patience, as we strive to improve the ITAS service. Very Best Regards, CIT/DECA ITAS Technical Team.
Resolution:  The maintenance has been successfully completed.
4/18/2007 3:33:30 PM - Internet DMZ Router Optimization Maintenance Activity
Description:   Date: Tuesday, April 24, 2007 Time: 6:00pm - 8:00pm ET Description: On Tuesday, April 24, 2007 at 6:00pm - 8:00pm ET NIHnet engineers will perform the final phase of the Internet DMZ Router Optimization project. Impact: Please note this final maintenance activity for the Internet DMZ Router Optimization project has been assessed with a Possible Service Affecting - High Probability (PSA-High) impact. Specifically, this maintenance activity will cause intermittent and momentary Internet traffic performance degradation during the first 60 minutes of the two-hour maintenance window. The expected Internet traffic performance impact should be limited to minor packet loss while new router configurations are applied and the Internet dynamic routing tables are updated. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The maintenance was completed successfully at 8:09 PM.
4/18/2007 1:16:53 PM - 100 Telephone Lines Down at 5 Research Court
Description:   Date : Wednesday, April 18, 2007 There is a telephone outage of approximately 100 lines at 5 Research Court. TIB technicians are currently working to resolve this issue. At this time, there is no estimated time of resolution. Updates will be posted as they become available. Please refer all telephone trouble calls for this location to Sean Kostic at (301) 451-6649. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The telephone outage at 5 Research Court has been resolved. Dial tone has been restored and verified on 4/19/2007.
4/18/2007 7:12:24 AM - NIH Remote Access VPN Maintenance
Description:   On Wednesday, April 18, 2007 at 10:00 AM, CIT engineers will conduct a maintenance activity to implement mandatory connection timeouts. The result of this change will be that all users of NIH Remote Access connections via the NIH Virtual Private Network will start "timing out" after 30 minutes of inactivity. In addition, Remote Access passwords will no longer be locally stored on individual computers. Consequently, after a "timeout", all NIH Remote Access users will have to reenter their password in order to connect to the network. This security change is for NIH Remote Access only and will not impact NIH internal wireless VPN users. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The NIH Remote Access VPN Maintenance was completed successfully at 10:15AM.
4/18/2007 1:54:27 AM - Blackberry Service Interruption
Description:   Beginning on April 17, 2007 at 8:11 PM, subscribers to BlackBerry All service for the Americas have been experiencing a service interruption. Users are currently unable to send or receive messages. Subscribers also are unable to access the subscriber website, self-provision, create new accounts or integrate new ISP accounts. BlackBerry Enterprise Clients will lose their connection and not be able to reconnect, and new users will not be able to register their devices. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Per a note from Research In Motion (RIM), the affected service has been returned to normal service. The backlog of messages is slowly being delivered. If there are any devices experiencing an outage, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov.
4/16/2007 6:25:11 PM - Power Outage in Rockledge 1
Description:   Date : Monday, April 16, 2007 Description: At this time there is a power outage in Rockledge 1. The power is out in the entire building. The technicians are aware of the situation and they have already contacted Pepco Electric in order to investigate and restore power. Currently there is no estimated time of resolution. Updates will be sent as they become available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The power outage that started at 6:20 pm was restored at 7:25 pm. All systems are back up and fully functional.
4/14/2007 9:57:53 AM - Building 10 Nutrition Department Call Center Issues
Description:   Date: Saturday, April 14, 2007 Description: The Nutrition Department Call Center in Building 10 is currently down. Engineers are currently investigating the issue. Currently there is no estimated time of resolution. Updates will be sent as they become available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Nutrition Department Call Center Issues have been resolved. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
4/12/2007 12:26:37 PM - MSN Messenger Issues
Description:   Date: Thursday, April 12, 2007 Description: Some NIH users are currently experiencing connectivity issues with MSN Messenger. CES Administrators are aware of the problem and are currently investigating the issue. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  MSN's Messenger Service is now functional. NIH users should be able to connect to the Messenger service. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
4/12/2007 7:50:18 AM - Building 12 Data Center NIHnet Internet DMZ Firewall Upgrade
Description:   Date: Thursday, April 19, 2007 Time: 6:00pm - 7:00pm Description: On Thursday, April 19, 2007 between 6:00pm - 7:00pm ET NIHnet engineers will perform a Scheduled Maintenance on the NIHnet Internet DMZ to upgrade the existing B12 Data Center Internet DMZ Firewall Operating System. Impact: Please note this Internet DMZ Firewall maintenance activity will cause a few momentary disruptions to non-TCP traffic between NIHnet <--> Internet during the first 15 minutes of the maintenance window. The expected loss of TCP connectivity and performance degradation between NIHnet <--> Internet should be minimal due to firewall redundancy. In the unlikely event that a rollback is required, an additional disruption to non-TCP traffic between NIHnet <--> Internet will occur. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Maintenance completed successfully at 6:25 PM.
4/11/2007 12:38:46 PM - nVision Production Databases Maintenance
Description:   Date: Wednesday, April 11 - Thursday, April 12, 2007 Time: 6:00PM - 7:00AM ET Description: nVision Production databases will be unavailable starting at 6pm on April 11th for maintenance backup. The database will not be available to the community until April 12th at 7am. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The nVision Production Database Maintenance was completed. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
4/11/2007 11:56:53 AM - Internet DMZ Outside Router Optimization Maintenance Activity
Description:   Date: Tuesday, April 17, 2007 Time: 5:30pm - 8:30pm ET Description: On Tuesday, April 17, 2007 at 5:30pm - 8:30pm ET NIHnet engineers will perform a routing and EtherChannel optimization maintenance on the NIHnet Internet DMZ outside routers. Impact: Please note the Internet DMZ Outside Router Optimization Maintenance Activity has been assessed with a Possible Service Affecting - High Probability (PSA-High) impact. Specifically, this maintenance activity may cause a momentary Internet connectivity disruption during the first 30 minutes of the three-hour maintenance window and again during the last 60 minutes of the maintenance window. Each of the expected Internet connectivity disruptions during this maintenance window should not exceed five minutes in duration while the NIHnet --- Internet dynamic routing tables are updated. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Internet DMZ Outside Router Optimization Maintenance Activity was completed successfully.
4/10/2007 2:04:50 PM - NIHCESMLBX6 Issues
Description:   At approximately 1:45PM the information store on NIHCESMLBX6 experienced some issues. This may have affected NHLBI and NIEHS customers' ability to access and use email. CES engineers have corrected the issue and they are currently investigating the cause of the issue.
Resolution:  NIHCESMLBX6 has been brought back online.
4/9/2007 12:00:12 PM - Building 10 Electrical Power Systems Test
Description:   Date: Saturday, April 14, 2007 Time: 7:00am - 12:00pm Description: On Saturday, April 14, 2007 at 7:00am - 12:00pm there will be an Emergency Power Systems test in Building 10 and the ACRF facilities. http://orf2.od.nih.gov/utilityshutdown/scripts/utility_shutdown.asp?seq=7272 Impact: The Building 10 Electrical Power Systems Test Maintenance activity is not expected to impact the Building 10 or ACRF facilities’ NIHnet data network infrastructure equipment because all Core, Distribution, and Access data network devices are supported by sufficient UPS and / or generator back-up power. However, there may be limited Wireless Network connectivity at various locations within Building 10 and Building 10B due to a limited number of Wireless Access Points (AP) using commercial power as a sole power source. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Building 10 Electrical Power Systems Test has been completed.
4/9/2007 10:48:10 AM - NIHCESMLBX12 is experiencing issues
Description:   Description: The mailbox server NIHCESMLBX12 is experiencing an issue. Impact: This is affecting CC and NIAID customers ability to access and use email. CES is aware of the issue and they are working on a resolution. There is currently no Estimated Time of Resolution.
Resolution:  The issue with the mailbox server NIHCESMLBX12 has been resolved.
4/7/2007 4:16:03 PM - Outlook Web Access (http://mail.nih.gov) Issues
Description:   Date: Saturday, April 7, 2007 Outlook Web Access (http://mail.nih.gov) is currently down. CES Engineers are currently investigating the issue. No estimated time of resolution is available at this time. Updates will be sent as they become available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Outlook Web Access (http://mail.nih.gov) issues have been resolved.
4/5/2007 1:24:23 PM - NIHnet Off Campus CWDM Ring I West Emergency Repair Maintenance
Description:   Date: Thursday, April 5, 2007 Time: 5:00pm - 6:00pm ET Description: On Thursday, April 5, 2007 between 5:00pm - 6:00pm ET NIHnet engineers will perform an Emergency Maintenance activity to repair one of the two redundant Off Campus network paths that failed on Monday, April 2, 2007. This Emergency Maintenance activity is necessary to restore complete NIHnet network access path diversity for the Executive Blvd and Fishers Place data networks. Impact: There is low potential for this emergency maintenance activity to affect normal NIHnet traffic to / from the Executive Blvd and Fishers Place data networks because only the failed redundant path on the CWDM Ring I infrastructure will be reconfigured as a means to restore dual-path redundancy between NIHnet -- Executive Blvd & Fishers Place data networks. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The NIHnet Off Campus CWDM Ring I West Emergency Repair Maintenance was completed successfully.
4/3/2007 12:43:11 PM - New Link for ADB Web
Description:   Due to the transition of the Datawarehouse reporting system to nVision. The link to the ADB web is no longer available through Data Town. Users will now need to use the QWS3270 application to access ADB. If you have any questions or need the QWS3270 application installed please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov.
Resolution:  A new link for ADB Web Access has been provided to users. The new link is for users to access ADB web is http://www.cit.nih.gov/deca/enterprise/adb.html.
4/2/2007 3:20:36 PM - CIT Remedy System Outage to Perform Maintenance
Description:   Start: Wednesday, April 4, 2007 8:00 PM Duration: 30 Minutes Description: The Support Technologies Group will be performing Maintenance on the CIT Remedy System starting at 8:00 PM on Wednesday, April 4, 2007. The system needs to be stopped and started to apply a change to the operating system. The server should be down for no more than 30 minutes. Users will be unable to submit Service Requests and register for CIT courses during this outage. Please contact the NIH Help Desk at 301-496-HELP (4357) or 301-496-8294 (TTY) if you have any questions.
Resolution:  The maintenance was completed successfully. The Remedy system is back up and fully functional.
4/2/2007 3:00:00 PM - CIT Report Server Maintenance
Description:   Date: Wednesday, April 4, 2007 Time: 6:30PM Duration: 30 minutes Description: The CIT Reports server that supports reporting for the NIH Helpdesk needs to be restarted to clear a lingering issue. The Support technologies Group (STG) will restart the system at 6:30 PM to clear the lingering issue. This maintenance is expected to last no more than 30 minutes. We apologize for any inconvenience this may cause.
Resolution:  The CIT Report Server Maintenance was completed successfully.
4/2/2007 1:37:48 PM - Redundant off-campus connectivity to NIHnet
Description:   The redundant off-campus connectivity to NIHnet for the Executive Blvd and Fishers Place sites is down due to a possible fiber issue between Building 45 and 5635 Fisher's Lane. A fiber vendor and NIHnet engineers are working on a resolution. There is currently no estimated time for repair. Impact: Because of the redundancy of the network path this is not a service affecting event. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The redundant connectivity for Executive Blvd. and Fishers Place networks to NIHnet has been restored.
4/2/2007 10:23:40 AM - Long Distance Problems
Description:   NIH Staff are having problems making and receiving long distance phone calls. CIT and Verizon technicians are currently working to resolve this issue.
Resolution:  The problem with the telephone lines was corrected by the vendor. At this time all services are back up and fully functional.
3/31/2007 3:07:40 AM - Building 9 Network Connectivity Issues
Description:   Building 9 users experienced a brief network connectivity issue due to hardware failure. NIHNet engineers were dispatched and network connectivity has been restored.
Resolution:  Network Connectivity has been restored.
3/29/2007 4:12:46 PM - Google Talk Connectivity Issues
Description:   Date: Thursday, March 29, 2007 Users attempting to use Google Talk from within the NIH network are experiencing intermittent connectivity issues with the service. The issue is currently being investigated. An estimated time of resolution is not currently available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  EMIB has resolved this issue and tested Google Talk's IM communication successfully. EMIB advises Google Talk users to please be patient and give their client a minute while signing in.
3/28/2007 5:08:46 PM - NIHFILER Performance Issues
Description:   NIHFILER is currently experiencing performance degradation issues. In order to correct this issue the server will be rebooted today, Wednesday March 28th, at 6PM. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Issue has been resolved. NIHFILER is back online.
3/28/2007 11:41:10 AM - CIT - Issues with Personal Folders and H: Drive Access
Description:   Some CIT users are experiencing intermittent issues accessing Personal Folders in Outlook and accessing their H: Drives. EMIB is currently investigating the issue. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  PETRI maintenance was completed successfully. All services are back on line.
3/27/2007 10:14:53 AM - The NIHMCMSProd server is currently experiencing issues
Description:   The NIHMCMSProd server is experiencing difficulties with the web services. The system administrators are working on resolving the problems at this time. The web sites that are affected are CSR, NIAAA, and the NIHLibrary Content Management Sites.
Resolution:  The issue that the NIHMCMSProd server was experiencing has been resolved and all sites are operational.
3/26/2007 2:36:56 PM - NIHnet Internet DMZ Path Failover Maintenance
Description:   Date: Monday, March 26, 2007 Time: 5:00pm - 5:30pm ET Description: On Monday, March 26, 2007 between 5:00pm - 5:30pm ET NIHnet engineers will perform an Emergency Maintenance activity to failover the NIHnet --- Internet DMZ traffic from the Building 12 path to the Fernwood Internet DMZ path. This Emergency Maintenance activity is necessary as a means to isolate the redundant Building 12 Internet Firewalls which require hardware and software analysis to resolve a suspected performance issue. Impact: There is low potential for this emergency maintenance activity to affect normal NIHnet traffic to/from the Internet while routing tables are updated to support the new primary path via the Fernwood building. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  This maintenance activity was completed successfully at 5:05 PM.
3/26/2007 1:32:35 PM - Redundant off-campus connectivity to NIHnet
Description:   Description: The redundant off-campus connectivity to the NIHnet for the Executive Blvd and Fishers Place sites is down due to a possible fiber issue between Building 45 and 5635 Fisher's Lane. Fiber vendors and NIHnet engineers are working on a resolution. There is currently no estimated time of resolution. Impact: Because of the redundancy of the network path this is not a service affecting event. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The redundant connectivity for the Executive Blvd and Fishers Place off-campus sites has been restored. The NIHnet engineers located the source of the redundant path failure in 5635 Fishers Lane and have since resolved the problem.
3/23/2007 3:24:45 PM - Bldg 12 NCCS Firewall Image Upgrade Maintenance Activity
Description:   Date: Thursday, March 29, 2007 Time: 6:00pm - 8:00pm ET Description: On Thursday, March 29, 2007 at 6:00pm - 8:00pm ET, NIHnet engineers will perform a scheduled maintenance activity to upgrade the firewall image on the Building 12 NIH Data Center firewall devices that provide network security for the NCCS Data Center in Sterling, VA. Impact: Please note this firewall image upgrade maintenance activity will cause a maximum of three (3) momentary disruptions to connectivity to the NCCS Data Center in Sterling, VA during the first 90 minutes of the two-hour maintenance window while the Building 12 NIH Data Center firewall devices are upgraded and tested. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  This maintenance was completed successfully at 8:12PM.
3/23/2007 3:02:15 PM - EOS.NIH.GOV Performance Issue
Description:   EOS.NIH.GOV is experiencing performance issues and may be experiencing a hang condition. System administrators are currently investigating the issue. Affected applications include NED, NIDB, EGG, CRISP, and ACF databases. No estimated time of resolution is available at this time. Updates will be provided when they become available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or helpdesk@nih.gov if you have any questions.
Resolution:  System is now stable issue has been resolved.
3/23/2007 10:34:18 AM - NLM Internal Connectivity Issue
Description:   Due to the recent Transparent Web Proxy Implementation activity performed on Thursday, March 22, 2007 at 7:00pm - 8:00pm, NIHnet connectivity to NLM adversely affected the NLM Intranet performance and reliability. The issue with connectivity between NIHnet and NLM has since been resolved and all NLM network service is now in a restored state.
Resolution:  The issue with connectivity between NIHnet and NLM has since been resolved and all NLM network service is now in a restored state.
3/23/2007 9:43:29 AM - Building 10 Electrical Power Systems Test
Description:   Date: Saturday, March 31, 2007 Time: 7:00am -12:00pm Description: On Saturday, March 31, 2007 at 7:00am - 12:00pm there will be an Emergency Power Systems test in Building 10 and the ACRF facilities. http://orf2.od.nih.gov/utilityshutdown/scripts/utility_shutdown.asp?seq=7272 Impact: The Building 10 Electrical Power Systems Test Maintenance activity is not expected to impact the Building 10 or ACRF facilities’ NIHnet data network infrastructure equipment because all Core, Distribution, and Access data network devices are supported by sufficient UPS and / or generator back-up power. However, there may be limited Wireless Network connectivity at various locations within Building 10 and Building 10B due to a limited number of Wireless Access Points (AP) using commercial power as a sole power source. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Electrical Power Systems Test originally scheduled for Saturday, March 31, 2007 for the old building 10 and the ACRF, has been cancelled. The test will be rescheduled and the date will be announced later.
3/21/2007 4:33:58 PM - Internet DMZ Transparent Web Proxy Implementation
Description:   Date: Thursday, March 22, 2007 Time: 6:00pm - 8:00pm ET Description: On Thursday, March 22, 2007 between 6:00pm - 8:00pm ET NIHnet engineers will implement a transparent web proxy on the Internet DMZ. The original maintenance activity which occurred on March 20, 2007 was rolled back to ensure service availability. This change management implementation activity is in support of efforts to improve web browsing (HTTP) and increase consistency between protocols by modifying the web caching system to transparently cache HTTP requests. This change will cause external web servers to see the IP address of the client initiating the HTTP request as opposed to the IP address of the web caching system. This will make HTTP transactions consistent with HTTPS, FTP, SSH, and all other IP based protocols. NIH will continue to filter inappropriate web requests and cache frequently requested items. No changes will be necessary on the part of NIH clients. Impact: This Internet DMZ Transparent Web Proxy Implementation activity is not expected to affect normal NIHnet Internet connectivity or web browsing (HTTP) activity during the maintenance window. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Because some websites were found to be inaccessible, this implementation was backed out at 11:48 PM on Thursday, March 22, 2007.
3/20/2007 4:34:32 PM - Connect Service Downtime
Description:   The Connect (formerly Breeze) services will go down tonight at 6:00PM for preventative maintenance. The downtime should be no longer than 15 minutes. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY), or helpdesk@nih.gov if you have any questions.
Resolution:  All the services are back up.
3/19/2007 2:20:11 PM - Building 12 NIH Data Center Network Maintenance Activity
Description:   Date: Saturday, March 24, 2007 Time: 7:00am - 11:00am ET Description: On Saturday, March 24, 2007 between 7:00am - 11:00am ET NIHnet engineers will perform a scheduled field trial maintenance activity in the Building 12 NIH Data Center on a select list of Blade Server devices to improve network switching performance and reliability. Impact: This Building 12 NIH Data Center Network Maintenance Activity has been given a possible service affecting - low probability impact assessment (PSA-Low). This impact assessment has been based on the outcome of completed advanced lab testing and the actual maintenance window activity task execution and completion testing procedures. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Building 12 NIH Data Center trial maintenance activity has been postponed and will be rescheduled for a later date.
3/17/2007 7:59:54 AM - T1 connection between Building 12 and Cockeysville is down.
Description:   The T1 connection between Building 12 and Cockeysville is down. Verizon technicians are currently investigating the issue. Currently there is not an estimated time of resolution. Updates will be sent as more information becomes available.
Resolution:  Upon further investigation, it has been learned that the T1 connection between Building 12 and Cockeysville was under going scheduled maintenance by Verizon. The maintenance is scheduled to end at 6:00 GMT.
3/16/2007 2:19:59 PM - MSN Messenger Connectivity Issues
Description:   Connectivity issues with MSN Messenger are currently being experienced. System administrators are working to resolve the issue. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  This issue was resolved.
3/16/2007 10:10:38 AM - Remedy Issues
Description:   Remedy is currently experiencing issues. Systems Admins are working towards a resolution. Updates will be sent as they become available.
Resolution:  Users are no longer encountering the pop-up error message which indicates an incompatibility between the Remedy client version and the revised Daylight Savings Times. If any users are still encountering issues related to the pop-up error message please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov.
3/16/2007 7:03:01 AM - DNS Change Website and Printshare Maintenance
Description:   Date: Saturday, March 17, 2007 Time: 7:00am - 10:00 am ET Description: On Saturday, March 17, 2007 between 7:00am - 10:00am ET NIHnet engineers will perform a Scheduled Maintenance to install patches on the DNS Change Website and Printshare Service server. These services are hosted on the same machine. http://www.net.nih.gov/dns , printshare.nih.gov Impact: During this maintenance activity, the www.net.nih.gov/dns website as well as the Printshare service will not be available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Patches on Kafka have been successfully deployed. All DNS services and PrintShare have been returned to normal.
3/14/2007 10:51:51 AM - Internet DMZ Transparent Web Proxy Implementation
Description:   Date: Tuesday, March 20, 2007 Time: 7:00pm - 10:00pm ET Description: On Tuesday, March 20, 2007 between 7:00pm - 10:00pm ET NIHnet engineers will implement a transparent web proxy on the Internet DMZ. This change management implementation activity is in support of efforts to improve web browsing (HTTP) and increase consistency between protocols by modifying the web caching system to transparently cache HTTP requests. This change will cause external web servers to see the IP address of the client initiating the HTTP request as opposed to the IP address of the web caching system. This will make HTTP transactions consistent with HTTPS, FTP, SSH, and all other IP based protocols. NIH will continue to filter inappropriate web requests and cache frequently requested items. No changes will be necessary on the part of NIH clients. Impact: This Internet DMZ Transparent Web Proxy Implementation activity is not expected to affect normal NIHnet Internet connectivity or web browsing (HTTP) activity during the maintenance window. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Several websites were inadvertently blocked as a result of this activity. After further review, CIT engineers have decided to back out of this implementation.
3/14/2007 10:46:22 AM - Premiere integration with Connect is currently down
Description:   The Premiere teleconferencing service integration with Connect (formerly Breeze) is currently down. Users can still dial into Premiere calls, but cannot dial out from Connect, nor can teleconferences be recorded. VoIP can still be recorded. Premiere engineers are currently working on the problem, but there is no estimate on when service will return right now.
Resolution:  The Premiere teleconferencing service integration with Connect (formerly Breeze) issues have been resolved.
3/12/2007 10:48:33 AM - Wireless BBSM authentication issues
Description:   Wireless BBSM/Radius internet users are currently experiencing connectivity issues. Users are able to connect and authenticate for a few minutes but are disconnected and prompted for log in credential repeatedly. NIHnet engineers are aware of and actively working to resolve the problem. Updates to follow. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  After rebooting the BBSM servers, the wireless connectivity issues appear to have been resolved. Please contact the NIH Help Desk if you experience any further problems.
3/7/2007 12:04:38 PM - NIHnet Internet Router IOS Upgrade Maintenance
Description:   Date: Wednesday, March 14, 2007 Time: 8:00pm - 10:00pm ET Description: On Wednesday, March 14, 2007 between 8:00pm - 10:00pm ET NIHnet engineers will perform a Scheduled Maintenance activity to update the IOS on the Building 12 Internet Outside Router. This NIHnet Internet Router IOS Upgrade Maintenance activity is in support of efforts to provide a permanent solution to two (2) low-threat Cisco vulnerabilities which have already been mitigated by temporary workaround solutions. Impact: This NIHnet Internet Router IOS Upgrade Maintenance activity WILL NOT affect normal NIHnet --- Internet connectivity. However, this maintenance activity will affect EXTRANET Site-to-Site VPN and Dedicated VPN connectivity during the first 30 minutes of the two-hour maintenance window as the B12-7600-Outside router is reloaded to activate the new IOS image. During the device reload sequence, affected customers should expect to experience a complete loss of connectivity for approximately 10 - 15 minutes as the B12-7600-Outside router boots its new IOS image and establishes connectivity and routing tables. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  NIHnet Internet Router IOS Upgrade Maintenance successfully completed with a minor complication. An external customer redundant peer session didn’t come back up; the customer does have network connectivity only one of his backup links is down. The NOC is working with the customer at this time.
3/5/2007 1:25:35 PM - Building 12 Wireless VPN Server Update
Description:   On Thursday, March 8, 2007 at 1 am, the Building 12 Wireless VPN Server will have a patch applied to adjust the Daylight Savings Time (DST) settings. The update requires a reboot of the VPN hardware. The NIH Wireless VPN devices are completely redundant. The individual maintenance activity will have no impact on the availability of the Wireless VPN service. There is no expected impact on any other part of the network. Please contact the NIH Help Desk at 301-496-HELP or helpdesk@nih.gov if you have any questions.
Resolution:  The Building 12 Wireless VPN Server Update was completed as of 1:03 am March 8, 2007.
3/5/2007 11:24:14 AM - NIH Operator services located in the Fernwood Bldg.
Description:   The NIH Operator services located in the Fernwood Building have reported a loss of network connectivity. NIHnet Engineers are aware of the issue and they are currently working on a resolution. There is curently no estimated time of resolution. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  This issue has been resolved. NIHnet Engineers will continue to monitor the NIH Operator Services network throughout the day.
3/5/2007 9:23:02 AM - ITAS Maintenance: DST Patches
Description:   As you may be already aware, the period of daylight savings time (DST) extends beginning in Mar 11, 2007 and ends a week later than usual on November 4, 2007. This DST change requires patches on the operating system level in the ITAS production environment. ITAS production is scheduled for applying DST patches Thursday, March 8, 2007 after 10:00PM. This patching takes less than 2.5 hours to complete. During this time, ITAS will be unavailable. If you have any questions, please contact NIH Helpdesk via email at nihhelpdesk@mail.nih.gov
Resolution:  Solaris DST patching was finished. ITAS service is up and running.
3/2/2007 6:19:57 PM - Connect (formerly Breeze) server downtime for maintenace
Description:   Date: Friday, March 2, 2007 Time: 6:00 PM - 12:00AM Description: On Friday, March 2, 2007 the Connect (formerly Breeze) servers will be down for maintenance starting at 6pm. Engineers will be applying patches and rolling out with SSL. We should be back up by Saturday, March 3 at 12:01am
Resolution:  The maintenance completed successfully.
3/2/2007 3:00:07 PM - NIH Listserv DST Maintenance
Description:   Date: Sunday, March 4, 2007 Time: 7:00am - 7:30am ET Description: On Sunday, March 4, 2007 between 7:00am - 7:30am ET NIHnet engineers will perform a Scheduled Maintenance to install DST patches on the NIH Listserv server. Impact: Please note that the list.nih.gov website will not be available during the first 15 minutes of this maintenance activity. In addition, no emails will be lost during this maintenance activity. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The NIH listserv maintenance has been successfully completed.
3/2/2007 2:57:26 PM - DNS Change Website DST maintenance
Description:   Date: Sunday, March 4, 2007 Time: 7:00am - 9:30am ET Description: On Sunday, March 4, 2007 between 7:00am - 9:30am ET NIHnet engineers will perform a Scheduled Maintenance to install DST and related patches on the DNS Change Website server. http://www.net.nih.gov/dns Impact: During this maintenance activity, the www.net.nih.gov/dns website will not be available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The maintenance that was performed on the DNS Change Website was completed successfully.
3/1/2007 2:29:42 PM - EOS.NIH.GOV Performance Issue
Description:   EOS.NIH.GOV is currently experiencing an extremely high load that is preventing access to the system. Administrators are investigating with no estimated time to resolution. Affected applications include NED, EGG, CRISP, and ACF databases.
Resolution:  This issue has been resolved after administrators rebooted all databases and they came back up. They are still going to investigate the issue; however, everything has been restored.
2/28/2007 3:13:45 PM - Fernwood Power Issue
Description:   The Fernwood building is currently experiencing a power issue. Facility engineers have been dispatched and they are investigating the issue. Updates will be posted when they become available.
Resolution:  The Fernwood power issues have been resolved. It appears that all services have returned to normal. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov immediately if any outstanding issues are known.
2/27/2007 12:03:04 PM - Remote Access Service Impairment
Description:   The NIH Remote Access Service is impaired at this time. NIHnet engineers are working on this issue. There is currently no estimated time of resolution.
Resolution:  This issue has been resolved by fixing the firewall rules.
2/26/2007 3:14:23 PM - Sterling-6500-NCCS Router IOS Upgrade Maintenance
Description:   Event Details: Date: Wednesday, February 28, 2007 Time: 6:00pm - 8:00pm ET Description: On Wednesday, February 28, 2007 at 6:00pm - 8:00pm NIHnet engineers will perform a Scheduled Maintenance Activity on one of the redundant distribution routers located at the Sterling, VA NIH Data Center. The purpose of this maintenance activity is to upgrade the Sterling-6500-NCCS Router IOS software as a means to mitigate a known, low-threat Cisco crafted IP packet vulnerability. Impact: Please note this Sterling-6500-NCCS Router IOS Upgrade Maintenance activity will cause a maximum of three (3) momentary disruptions to connectivity to the NIH Data Center in Sterling, VA during the first 60 minutes of the two-hour maintenance window while the Sterling-6500-NCCS IOS is upgraded and tested. Please contact the NIH CAP Team at 301-402-0724 or citdcscap@mail.nih.gov if you have any questions.
Resolution:  The maintenance was completed successfully.
2/24/2007 10:43:45 AM - Power Issues At 6700 Rockledge Dr
Description:   There is a power outage at 6700 Rockledge Drive. Pepco has been dispatched and is investigating the issue. Currently there is no estimated time of resolution. Updated information will be sent as it becomes available. Please contact the NIH Help Desk at 301-496-HELP or helpdesk@nih.gov if you have any questions.
Resolution:  The power issues at 6700 Rockledge Drive were resolved as of 12:19pm.
2/23/2007 3:02:52 PM - VPN Software Update
Description:   Between February 25, 2007 and March 5, 2007, all NIHnet VPN Servers will have a patch applied to them to correct a software bug and adjust the Daylight Savings Time (DST) settings. Each update requires a reboot of the VPN hardware. Only one device will be rebooted per day. Please note that both Remote Access VPN and Wireless VPN servers will be patched. The NIH Wireless VPN server patching will take place at 1:00 am on February 25 and March 4. The Remote Access VPN Server patching will take place each day at 7:30 am between February 27 and March 2nd. There is no expected impact on any other part of the network. Please contact the NIH Help Desk at 301-496-HELP or helpdesk@nih.gov if you have any questions.
Resolution:  The Remote Access VPN Server patching has been completed. The patching of NIH Wireless VPN servers has been completed with the exception of the device for CRC, Old Building 10 and ACRF. The upgrade for this device has been postponed.
2/22/2007 2:24:32 PM - NIH Login Production Maintenance
Description:   On Wednesday, February 28, 2007 at 8PM NIH SSO Admins plan to do the following: 1. Update the cert chain for nihlogin.nih.gov 2. Update the COMMONS Authentication to pass IMPACT II Authorization data 3. Patch for DST The maintenance will last for approximately 2.5 hours. Each server will need to be rebooted after the DST patch but there is no expected downtime due to redundant systems. Please send e-mails directly to NIH SSO Admins if you experience any problems.
Resolution:  Maintenance completed successfully.
2/16/2007 2:03:04 PM - NIHnet Internet DMZ Path Failover Maintenance
Description:   On Tuesday, February 20, 2007 between 6:00pm - 7:00pm ET NIHnet engineers will perform a Scheduled Maintenance activity to failover the NIHnet --- Internet DMZ traffic from the Building 12 path to the Fernwood Internet DMZ path. This NIHnet --- Internet DMZ path failover maintenance is in support of efforts to prepare the Building 12 Internet DMZ outside router for an impending IOS upgrade maintenance activity to be performed in mid-March 2007. Impact: There is low potential for this NIHnet Internet DMZ Path Failover maintenance activity to cause a momentary data traffic delay across the NIHnet Internet DMZ while routing tables are updated to support the new primary path via Fernwood. The momentary delay in data traffic may occur within the first 30 minutes of the one-hour maintenance window and is not expected to last more than five minutes in duration. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The maintenance was completed successfully.
2/14/2007 9:46:33 AM - Connect (Formerly Breeze)
Description:   The Premiere teleconferencing service integration with Connect (formerly Breeze) is currently down. Users can still dial into Premiere calls, but cannot dial out from Connect, nor can teleconferences be recorded. VoIP can still be recorded. Premiere engineers are currently working on the problem, but there is no estimate on when service will return right now. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Premiere teleconferencing service integration with Connect (formerly Breeze) is back online.
2/13/2007 1:04:36 PM - CIT Remedy Server DST Patching - Saturday February 17th
Description:   On Saturday February 17th at 7:00 AM the CIT Remedy Server and will be unavailable for up to 60 minutes while the Remedy Daylight Savings Time patch is applied. Impact: During this time all interfaces into Remedy, including the API, email and web interface will experience periods of unavailability, as will online ticket submission for the NIH Help Desk. This will also impact other customer support websites that draw information from Remedy, including the IT Help Desk and CIT Training sites. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The CIT Remedy Server DST patching has been completed. The Remedy system is now available.
2/12/2007 12:01:38 PM - Fernwood IRT GigE Upgrade Maintenance
Description:   Date: Friday, February 16, 2007 Time: 7:00am - 7:30am ET Description: On Friday, February 16, 2007 at 7:00am - 7:30am NIHnet Off Campus engineers will perform a scheduled LAN switch module upgrade maintenance activity in support of efforts to meet IRT’s increased bandwidth requirements. Impact: This Fernwood IRT GigE Maintenance activity IS NOT expected to disrupt end-user traffic because the new switch modules will be installed without powering down the target LAN switches. However, there is still a low probability the LAN switches may unexpectedly reload while the new LAN modules are installed. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Fernwood IRT GigE Upgrade Maintenance was successfully completed at 7:50am.
2/9/2007 5:03:43 PM - CIT Video Server Maintenance
Description:   On Saturday, February 10, 2007 at 9:00am - 11:00am there will be maintenance to increase disk space on CIT video servers. Impact: The NIH VideoCast and Video Conference systems are not expected to be impacted. However, there is the potential of limited access for a brief amount of time. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The CIT Video Server Maintenance has been completed successfully.
2/7/2007 3:48:26 PM - Performance Issue With NIHCESMLBX8
Description:   NIHCESMLBX8 is currently experiencing a performance degradation condition. This server provided mailbox access to users for: NCCAM NIDA NLM NIDCD CIT OD NLM/NCBI This issue is currently being investigated by CES Engineers. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  This issue was resolved.
2/7/2007 3:02:37 PM - NIHnet Internet DMZ Firewall Traffic Capture Maintenance
Description:   On Friday, February 9, 2007 at 6:00pm - 9:00pm NIHnet network security engineers will perform a scheduled NIHnet Internet DMZ Firewall Capture maintenance activity in support of efforts to troubleshoot an existing Internet DMZ router login issue. Impact: This NIHnet Internet DMZ Firewall Capture maintenance activity IS NOT expected to affect normal NIHnet --- Internet traffic because the firewalls being analyzed are expected to be performing at low-peak CPU utilization during the packet capture window. In the unlikely and unwanted event of performance degradation due to this packet capture maintenance activity, the NIHnet network security engineers will immediately end the data capture activity to allow normal NIHnet --- Internet traffic flows to resume. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Maintenance was completed with no complications.
2/6/2007 3:13:33 PM - test
Description:   test
Resolution:  test
2/5/2007 10:11:08 PM - TEST
Description:   This is a test of the Hot News system. Please disregard.
Resolution:  Test completed.
2/5/2007 10:57:42 AM - Building 10 Electrical Power Systems Test
Description:   Date: Saturday, February 10, 2007 Time: 7:00am - 11:00am Description: On Saturday, February 10, 2007 at 7:00am - 11:00am there will be an Emergency Power Systems test in Building 10 and the ACRF. http://orf2.od.nih.gov/utilityshutdown/scripts/utility_shutdown.asp?seq=7103 Impact: The Building 10 Electrical Power Systems Test Maintenance Activity is not expected to impact the Building 10 NIHnet data network infrastructure equipment because all Core, Distribution, and Access data network devices are supported by sufficient UPS and / or generator back-up power. However, there may be limited Wireless Network connectivity at various locations within Building 10 due to a limited number of Wireless Access Points (AP) using commercial power as a sole power source. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Building 10 Electrical Power Sytems Test Maintenance that was scheduled for Saturday, February 10, 2007 has been canceled by the NIH Facilities Department.
2/5/2007 9:11:23 AM - nVision Reports Available
Description:   Date: Monday, February 5th, 2007 Event Description: There is an nVision and Data Warehouse Siteminder problem. It currently affects nVision and Data Warehouse users. Technicians are currently working on the issue. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions
Resolution:  The New nVision/DW Portal was released today. This issue was resolved.
2/2/2007 11:23:33 AM - NIHCESPUB5 will be restarted at noon today (02/02/2007)
Description:   Date: Friday, February 2nd, 2007 Time: 12:00 PM (noon) Description: Due to an issue with mail delivery into a public folder store, Central Email Service technicians will be restarting NIHCESPUB5 at noon on Friday, February 2nd. Expected downtime is less than 10 minutes. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  NIHCESPUB5 was restarted successfully.
2/2/2007 9:17:13 AM - NIHnet Wireless DMZ Firewall Upgrade Maintenance
Description:   Date: Sunday, February 4, 2007 Time: 1:00am - 2:00am ET Description: On Sunday, February 4, 2007 at 1:00am - 2:00am NIHnet network security engineers will perform an operating system upgrade maintenance on the firewalls located on the Building 45 Wireless DMZ path. Impact: The NIHnet Wireless Service WILL NOT be affected by this Building 45 Wireless DMZ firewall operating system upgrade maintenance activity because all wireless access and VPN connectivity is supported by a redundant path via the Building 12 Wireless DMZ infrastructure. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The upgrade has been completed.
2/1/2007 11:25:22 AM - NIHDC removed from BIND
Description:   NIHDC is being removed from BIND
Resolution:  This maintenance was completed today.
1/31/2007 11:39:09 AM - Restart of CIT Customer Support Web Server at 11:45AM
Description:   Description: Due to latency and Remedy web interface issues, the CIT Customer Support Web Server will be restarted at 11:45. This will cause a brief outage (less than five minutes) of several sites including the following: IT Help Desk CIT Training Program CIT Knowledge Base Remote Access Website eRA Online Ticket Submission Thank you for your patience as we work to resolve this issue. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  This issue has been resolved. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
1/31/2007 10:12:56 AM - NIHnet OpDiv and Wireless Network Path Migration Maintenance
Description:   Date: Friday, February 2, 2007 Time: 6:00pm - 7:00pm ET Description: On Friday, February 2, 2007 between 6:00pm to 7:00pm NIHnet engineers will migrate redundant OpDiv and Wireless DMZ network paths to the new Fernwood Internet DMZ router. Impact: Please note this OpDiv and Wireless DMZ path migration Maintenance activity is NOT expected to cause a service disruption to the NIHnet Internet, OpDiv, or Wireless traffic because all Internet, OpDiv, and Wireless traffic is supported by redundant network paths via Building 12 and Building 45. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Maintenance was completed successfully without any complication.
1/30/2007 5:03:05 PM - Reboot of Exchange Server NIHCESMLBX3
Description:   NIH Central Email Service Engineers will reboot the Exchange Server NIHCESMLBX3 at 9:00 PM this evening, January 30. This reboot is due to an issue with Outlook Web Access performing slowly. The reboot will affect mailboxes for the following IC's: CIT, NIMH, NEI, NIDCR, NHGRI, CSR. The outage will be less than 15 minutes.
Resolution:  NIH Central Email Service Engineers have successfully rebooted the Exchange Server NIHCESMLBX3. This issue is resolved.
1/29/2007 4:15:52 PM - Connect (Formerly Breeze) Maintenence Window tonight 1/29/06
Description:   The Connect (formerly Breeze) servers will be down for maintenance tonight Monday, January 29th 2006 at 6:00 PM for 15 minutes. The technicians will be fixing an issue regarding failover connections.
Resolution:  The maintenance is complete.
1/29/2007 9:24:22 AM - Problems accessing NIHCESMLBX8 from OWA
Description:   Due to problems accessing NIHCESMLBX8 mailboxes from OWA, engineers are going to have to reset the services on this server. Engineers are planning on performing this on Monday, January 29th at 9:30AM. Expected downtime is less than 10 minutes. This server reset will affect NCCAM, NIDA, NLM, & NIDCD users.
Resolution:  The reset has been completed successfully.
1/26/2007 1:55:54 PM - NIHnet Internet Connectivity Bandwidth Upgrade Maintenance
Description:   Date: Wednesday, January 31, 2007 Time: 6:00pm - 7:00pm ET Description: On Wednesday, January 31, 2007 from 6:00pm - 7:00pm NIHnet engineers will be performing a bandwidth upgrade maintenance activity to the NIHnet Internet connection via the Building 12 Internet DMZ. Impact: This NIHnet Internet Connectivity Bandwidth Upgrade Maintenance activity has been assessed with a possible service affecting low probability (PSA-Low) impact due to redundant NIHnet Internet connectivity via the Fernwood Building AND expected rapid path / route convergence as a new GigE connection to the Internet is brought up in the Building 12 NIH Data Center. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  This maintenance was completed successfully.
1/26/2007 7:13:24 AM - Breeze Service Downtime
Description:   On Friday, January 26th at 6pm, the Breeze service will be taken down for a system upgrade. Service will return no later than Sunday, January 28th. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Breeze system upgrade has been completed.
1/25/2007 2:26:22 PM - NIHnet Wireless DMZ Firewall Upgrade Maintenance
Description:   Date: Sunday, January 28, 2007 Time: 1:00am - 2:00am ET Description: On Sunday, January 28, 2007 from 1:00am - 2:00am NIHnet network security engineers will perform an operating system upgrade maintenance on the firewalls located on the Building 12 Wireless DMZ path. Impact: The NIHnet Wireless Service WILL NOT be affected by this Building 12 Wireless DMZ firewall operating system upgrade maintenance activity because all wireless access and VPN connectivity is supported by a redundant path via the Building 45 Wireless DMZ infrastructure. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The maintenance has been successfully completed.
1/23/2007 11:23:35 AM - Video Conferencing Connectivity Problems
Description:   Please note that the NIH MXM Gatekeeper is experiencing connectivity problems. The Video application engineers are working to resolve this problem. For now there is no estimated time of resolution, but we expect to have this problem resolved as soon as possible. Impact: The following services will be disrupted: Calling from an IP video conferencing unit to an ISDN number Calling an IP video conferencing unit using an E.164 alias number and not the actual IP address. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Video conferencing services are now back online.
1/23/2007 10:23:35 AM - Building 10 Electrical Power Systems Test
Description:   Date: Saturday, January 27, 2007 Time: 7:00am - 11:30am Description: On Saturday, January 27, 2007 at 7:00am - 11:30am there will be an Emergency Power Systems test in Building 10 and the ACRF. http://orf2.od.nih.gov/utilityshutdown/scripts/utility_shutdown.asp?seq=6988 Impact: The Building 10 Electrical Power Systems Test Maintenance Activity is not expected to impact the Building 10 NIHnet data network infrastructure equipment because all Core, Distribution, and Access data network devices are supported by sufficient UPS and / or generator back-up power. However, there may be limited Wireless Network connectivity at various locations within Building 10 due to a limited number of Wireless Access Points (AP) using commercial power as a sole power source. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Building 10 Electrical Power Systems Test Maintenance Activity is complete. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
1/19/2007 2:11:49 PM - Baltimore Ethernet Backbone Upgrade Maintenance
Description:   On Wednesday, January 24, 2007 from 6:00pm - 7:00pm NIHnet engineers will be installing a FastEthernet module in the Baltimore-7500 router to increase the bandwidth capacity of the Baltimore remote-site network infrastructure. Impact: Please note this Baltimore Ethernet Backbone Upgrade Maintenance activity has been assessed with a Service Affecting (SA) impact. Specifically, during the first thirty minutes of the maintenance window, local Baltimore LAN connectivity to NIHnet will be disrupted while a new FastEthernet module is installed in the Baltimore-7500 router. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Baltimore Ethernet Backbone Upgrade Maintenance has been successfully completed.
1/19/2007 1:50:43 PM - CIT Remedy Server Extended Outage During Maintenance Window
Description:   Description: The CIT Remedy Server will experience a more extended outage during this month’s maintenance window (occurring tonight, Friday January 19th). The outage is expected to begin at approximately midnight and last for up to 90 minutes. During this time, all Remedy services will be unavailable, including the client and web interfaces, email and web ticket submissions, CIT Training online registrations, and portions of the IT Help Desk page, among other services. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The CIT Remedy Server extended outage is complete. Remedy Services are available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
1/19/2007 12:03:34 PM - 5625 Fishers Lane Power Test
Description:   On Saturday, January 20, from 7:00 AM until 11:30 AM, Cynergy Electric will perform a test at the 5625 Fishers Lane building to confirm the transfer capability of the new generator and transfer switches. This test will require that the building be removed from commercial power. The Emergency generator will be running during this period. The Life Safety generator will be operable for the SAFETY SYSTEMS only. The outage will occur at 7:30 AM and will last approximately four (4) hours. An incremental backup of the scientific data servers will occur prior to shutting them down at 6:00 a.m. This means for everyone except users on the 5th floor, please follow the guidelines below: (1) Copy your scientific data and administrative files to data servers by 6:00 p.m., Friday, January 19. The scientific data servers must be shut down prior to the power outage. Any file copied to a scientific data server past 6:00 p.m. on Friday will not be copied to backup tape before the power down. (2) Shut down your computers, printers, scanners, and faxes. You are advised to unplug all of your office equipment from the wall outlet for the duration of this test. Keep your electronic devices unplugged until you have been informed that the building power grid is functioning normally. Power will be restored to the servers when the 5625 Fishers power grid is functioning normally. Please contact the NIH Help Desk at 301-496-4357 if you have any questions.
Resolution:  The scheduled electrical test at 5625 Fishers Lane Building has been completed. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
1/18/2007 5:40:44 PM - CES will be restarting NIHCESMLBX3
Description:   CES will be restarting NIHCESMLBX3 at 5:50 PM on Thursday, January 18th. The outage will be 10 minutes and service will be restored by 6:00 PM. This outage only affects CIT and NHGRI.
Resolution:  The NIHCESMLBX3 server has been rebooted and is now back in service.
1/18/2007 1:38:25 PM - Cingular Data Service is experiencing reoccurring network outages
Description:   CIT has been receiving multiple outage reports from Cingular that appear to be both intermittent and ongoing over the entire southeast US due to storm damage and associated maintenance activities. Due to these outages and maintenance activities, some Blackberry users may not be able to send or receive e-mail or data. Users may receive a "data connection refused error" or may not receive data if their devices do not display all uppercase "GPRS" or "EDGE" next to the radio signal strength indicators. To enable re-registration to the network, users may turn the radio off and then on again. Or remove the battery from the Blackberry, waiting approximately 30 seconds and re-inserting the battery. There is no known time to resolution on this issue and we will provide updates as we receive additional information from the carrier. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Cingular Data Service connectivity issue has been resolved. Blackberry users should be able to normally send and receive data.
1/18/2007 7:45:11 AM - NIH DDNS Maintenance
Description:   nihdc, nihdc2, nihdc3 will be removed from BIND and exist solely in DDNS. Change will occurr at 11am
Resolution:  DNST made the change at 11AM
1/17/2007 1:21:15 PM - NIHnet Internet Router Replacement Maintenance
Description:   Date: Friday, January 19, 2007 Time: 6:00pm - 7:00pm ET Description: On Friday, January 19, 2007 between 6:00pm - 7:00pm NIHnet engineers will replace an end-of-life router on the Fernwood path of the NIHnet Internet DMZ. Impact: Please note this Fernwood NIHnet Internet Router Replacement Maintenance activity IS NOT expected to cause a service disruption to the NIHnet --- Internet traffic because all Internet traffic is currently traversing the Building 12 Internet DMZ path. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The NIHnet Internet Router Replacement Maintenance was completed successfully.
1/16/2007 2:34:10 PM - NIHnet Internet DMZ Failover Firewall Maintenance
Description:   Date: Wednesday, January 17, 2007 Time: 6:00pm - 7:00pm ET Description: On Wednesday, January 17, 2007 between 6:00pm - 7:00pm NIHnet engineers will migrate traffic from the active Building 12 Internet firewall to the standby Building 12 Internet firewall. Impact: Please note this NIHnet Internet DMZ Firewall traffic migration maintenance activity may cause a momentary disruption to all traffic between the NIHnet --- Internet during the first 30 minutes of the one-hour maintenance window while Internet traffic is migrated from the active firewall to the standby firewall on the Building 12 Internet DMZ path. The expected loss of connectivity and performance degradation between NIHnet --- Internet should be minimal while the standby firewall is activated. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The NIHnet Internet DMZ Failover Firewall Maintenance was completed successfully.
1/11/2007 1:16:36 PM - Baltimore Link Outage
Description:   The Primary circuit between Baltimore and NIH is down. Baltimore traffic is currently up on the backup T-1. The circuit provider has been informed and is currently working to restore service. If you have any questions regarding this outage, please contact the NIH Help Desk at 301-496-4357.
Resolution:  The issue was resolved.
1/9/2007 11:04:32 AM - ITAS new release and Web/App server upgrade on 1/10/07, 6PM - 8PM
Description:   The CIT/DECA ITAS Technical Team will be deploying a new version of ITAS (V.PR15) and upgrading the ITAS Web/App server tomorrow Jan. 10, 2007 from 6:00PM - 8:00PM. During this period of time ITAS will not be available. We thank our users for their patience, as we strive to improve the ITAS service. Very Best Regards, CIT/DECA ITAS Technical Team. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The deployment of the new version of ITAS and the upgrade of the ITAS Web/App server has now been completed. ITAS services are now available.
1/9/2007 7:59:45 AM - nVision will NOT be available today
Description:   Event Details: Date: January 09, 2007 Description: Due to a system problems overnight, the nVision will NOT be available today. The NIH Data Warehouse is fully functional. Work will continue today to resolve this problem. Impact: This problem prevented updating Balance of Accounts; therefore the DW Balance of Accounts is current as of COB 1/5/2007. All other Financial data is current as of COB 1/8/2007. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  nVision is now available. The nVision team working with the CIT Hosting group has resolved the hardware issue from last night sooner than originally expected. The data is current as of COB 1/5/2007. Monday’s data will be included in the normal updates taking place tonight. Thank you for your patience.
1/8/2007 1:18:22 PM - Building 10/10B Electrical Power Systems Test
Description:   Date: Saturday, January 13, 2007 Time: 7:00am - 11:00am Description: On Saturday, January 13, 2007 at 7:00am - 11:00am there will be an Emergency Power Systems test in Building 10 and Building 10B. http://orf2.od.nih.gov/utilityshutdown/scripts/utility_shutdown.asp?seq=6926 Impact: The Building 10/10B Electrical Power Systems Test Maintenance activity is not expected to impact the Building 10 or Building 10B NIHnet data network infrastructure equipment because all Core, Distribution, and Access data network devices are supported by sufficient UPS and / or generator back-up power. However, there may be limited Wireless Network connectivity at various locations within Building 10 and Building 10B due to a limited number of Wireless Access Points (AP) using commercial power as a sole power source. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The electrical power systems test has been completed.
1/8/2007 7:50:38 AM - Power Outage 2115 East Jefferson
Description:   The power is out in 2115 East Jefferson. Techncians are in the process of resolving the issue, but the estimated time of return to service is unknown at this time.
Resolution:  The power issue at the 2115 East Jefferson Building has been resolved.
1/5/2007 12:18:30 PM - Bldg 12 -- Sterling, Data Center Connectivity Maintenance
Description:   Event Details: Date: Friday, January 12, 2007 Time: 5:00pm - 6:00pm ET Description: On Friday, January 12, 2007 at 5:00pm - 6:00pm NIHnet engineers will perform a Scheduled Maintenance Activity to upgrade the physical memory on a Distributed Forwarding Card (DFC) in the B12-6500-NCSS router. This memory upgrade is necessary to improve the overall routing and packet forwarding performance for data traffic traversing the connections between the Building 12 -- Sterling, VA Data Centers. Impact: This maintenance activity will affect all NIHnet data network connectivity between the Building 12 ---- Sterling, VA Data Centers while the B12-6500-NCCS router is powered down to receive the DFC memory upgrade. The loss of network connectivity between the Building 12 - Sterling, VA data centers should not exceed 30 minutes while the B12-6500-NCCS router receives its DFC memory upgrade. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The maintenance was completed successfully.
1/4/2007 10:52:19 AM - ITAS Emergency Maintenance
Description:   Because of unprecedented usage in ITAS generated by the unanticipated holiday this pay period, and the very short work week ITAS is experiencing serious performance degradation. Users are having difficulty fulfilling their work requirements before the end of the pay period. In response to this, the CIT/DECA ITAS Technical Team has decided to implement emergency performance improvements immediately. These improvements were planned and scheduled for a later release, but because of the degradation we feel the need is pressing enough to implement them today in order to ensure users can complete their work before the end of the pay period. In order to implement these changes it will be necessary to bring the ITAS system down for approximately one hour. We have decided to do this during today's lunch hour from 12:00pm-1:00pm. During this time ITAS will not be available. We apologize for the inconvenience, and appreciate your patience as we take steps to correct the ITAS service. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The ITAS Emergency Maintenance has been completed and ITAS is now available.
1/3/2007 5:15:41 PM - Restart of Production SQL Server for ITAS
Description:   Date: Wednesday, January 3rd, 2007 The ITAS Technical Team will be restarting the production SQL Server for ITAS at 10PM. The ITAS service will experience an outage of about 10 minutes at that time. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The restart of the production SQL Server for ITAS has been completed. The ITAS service is now available.
12/15/2006 1:15:58 PM - Building 12 B12-4000-ADS SUP Module Emergency Maintenance
Description:   Date: Friday, December 15, 2006 Time: 5:00pm - 6:00pm ET Description: On Friday, December 15, 2006 between 5:00pm - 6:00pm NIHnet engineers will perform an Emergency Maintenance to replace a faulty Supervisor module in the Building 12 B12-4000-ADS distribution switch. Impact: Please note this Building 12 B12-4000-ADS Supervisor Module Emergency Replacement Maintenance activity has been assessed with a Possible Service Affecting-Low (PSA-Low) service impact due to the redundant nature of NIHnet distribution switch topology that supports campus, off campus, and WAN connectivity. Therefore, any disruption to NIHnet data connected services during this maintenance activity should not exceed 30 seconds in duration as redundant network devices and paths continue to provide network connectivity while the B12-4000-ADS supervisor module is replaced. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Building 12 B12-4000-ADS Switch Supervisor Module Emergency Replacement Maintenance was completed successfully.
12/15/2006 7:45:31 AM - Customer Support Report Server Data issue
Description:   The CIT Customer Support Report server data did not mirror properly last night. The issue has been corrected and all data except Remedy data has been transferred and is now current. Remedy data is currently being mirrored and should be available by approximately 9:00 AM.
Resolution:  The Remedy ticket data has completed it's transfer successfully.
12/14/2006 1:25:18 PM - Remedy Ticket Assignment Notification Issues
Description:   Date: December 14, 2006 Description: Remedy ticket assignment notifications are not being generated properly. Support Technologies Group Staff is currently investigating the issue and working to resolve the problem. No ETA but we do not anticipate that the issue will last for long. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Remedy email problem was due to an issue with last night’s (Wednesday, December 13th, 2006) rollout and has been resolved. Some ticket assignment notifications were not generated properly between the approximate hours of 7:00 PM yesterday and 1:30 PM today so you may wish to review your ticket queues to ensure that no tickets are overlooked.
12/14/2006 12:57:46 AM - SSO Staging Tier Unavailable
Description:   The SSO staging tier is currently down. EMIB Facilities is investigating the issue with the server. It may be a hardware issue. There is currently no estimated time of resolution.
Resolution:  The server was restored and is the service is available as of 7:25 AM.
12/13/2006 7:21:03 AM - Building 10 and Building 45 Fiber Plant Optimization Maintenance
Description:   Subject: Building 10 and Building 45 Fiber Plant Optimization Maintenance Date: Wednesday, December 20, 2006 Time: 6:00am - 8:00am ET Description: On Wednesday, December 20, 2006 at 6:00am - 8:00am NIHnet fiber engineers will perform a Fiber Plant Optimization maintenance activity on the fiber paths between Building 10 and Building 45. Impact: Please note this Building 10 --- Building 45 Fiber Plant Optimization Maintenance activity has been assessed with a Possible Service Affecting-Low (PSA-Low) service impact due to the redundant nature of the existing fiber plant infrastructure between NIHnet Core and Distribution routers that support campus and off-campus connectivity to NIHnet. Therefore, there is a low probability chance for connectivity and performance impact while each fiber pair is move to a new, optimal path between Building 10 --- Building 45 because all interconnected routers will automatically fail-over to redundant connections via separate, diverse fiber paths. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Fiber Plant Optimization maintenance has been completed.
12/12/2006 5:45:28 PM - Breeze Downtime for Maintenance
Description:   Date: December 13, 2006 Time: 6:00 PM - 6:15 PM On December 13, 2006 between 6:00 and 6:15 PM the Breeze service will be unavailable due to maintenance. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Breeze services are successfully restored.
12/11/2006 3:21:26 PM - Possible mail delay affecting MAILFWD and NON-CES Customers
Description:   Possible mail delay for message being sent via MAILFWD and from external sites to non-CES mailboxes. The delay is about 15 minutes for messages being sent to non-CES (non-Exchange mailboxes) and for messages being sent via mailfwd.nih.gov. CES engineers are currently working on addressing the problem.
Resolution:  CES Engineers isolated the cause of the delay and have corrected this issue.
12/11/2006 1:37:05 PM - Building 10 Electrical Power Systems Test
Description:   DESCRIPTION: On Saturday, December 16, 2006 at 7:00am - 11:00am there will be an Emergency Power Systems test in Building 10 and the ACRF. Please copy and paste in the address bar of your browser the following in order to see a more descriptive explanation of the power test: http://orf2.od.nih.gov/utilityshutdown/scripts/utility_shutdown.asp?seq=6863 IMPACT: The Building 10 Electrical Power Systems Test Maintenance Activity is not expected to impact the Building 10 NIHnet data network infrastructure equipment because all Core, Distribution, and Access data network devices are supported by sufficient UPS and / or generator back-up power. However, there may be limited Wireless Network connectivity at various locations within Building 10 due to a limited number of Wireless Access Points (AP) using commercial power as a sole power source. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The scheduled electrical maintenance in ACRF has been completed.
12/8/2006 9:51:54 AM - nVision INDUS O/S Upgrade
Description:   Date: Saturday, December 9, 2006 Time: 7:00a.m. Description: nVision production servers will be unavailable starting 7 AM on Saturday December 9th due to scheduled maintenance. nVision will be unavailable approximately 24 hours. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Maintenance is complete per email sent by Stephanie Clark on 12/11/06 at 12:45pm
12/8/2006 9:44:07 AM - nVision/DW Financial Status
Description:   The financial data from Thursday has been received by the nVision/DW team and will be processed this evening along with today’s data. We will be current on Monday as of COB 12/8/2006.
Resolution:  All data is now up to date. Per Stephanie Clark email recieved on 12/11/06. Closing out hotnews.
12/8/2006 7:58:57 AM - Fernwood Distribution Router Optimization Maintenance
Description:   On Thursday, December 14, 2006 between 6:00pm – 7:00pm ET NIHnet engineers will perform a Scheduled Maintenance to optimize fail-over routing capabilities between the two NIHnet Fernwood distribution routers. Please note this Fernwood Distribution Router Optimization maintenance activity may cause a momentary data traffic delay across NIHnet while routing paths are reestablished and routing tables are normalized. The momentary delay in data traffic will occur within the first 30 minutes of the one-hour maintenance window and is not expected to last more than five minutes in duration. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Fernwood Distribution Router Optimization Maintenance was completed successfully.
12/6/2006 2:44:00 PM - Bldg 12 <--> Sterling, Data Center Connectivity Maintenance
Description:   Date: December 15, 2006 Time: 6:00pm - 8:00pm ET Description: On Friday, December 15, 2006 at 6:00pm – 8:00pm NIHnet engineers will perform a Scheduled Maintenance Activity to replace a defective and problematic Cisco 6509 chassis located in the Building 12 NIH Data Center. Impact: This maintenance activity will affect all NIHnet data network connectivity between the Building 12 <----> Sterling, VA NIH Data Centers because the problematic device being replaced provides all connectivity to the NCCS Data Center from the Building 12 NIH Data Center. The loss of network path and connectivity between the Building 12 <----> Sterling, VA NIH Data Centers should not exceed one-hour and will occur during the first 60 minutes of the maintenance window. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Bldg 12 <--> Sterling, Data Center Connectivity Maintenance was completed successfully.
12/4/2006 10:14:23 AM - Phone Outages in Building 10
Description:   Customers located in the 4D corridor in Building 10 are currently experiencing intermittent phone outages. Technicians have been dispatched and there will be updates to follow.
Resolution:  This issue has been resolved.
12/4/2006 9:12:47 AM - Test Test Test
Description:   Event Description Remedy Application server will be rebooted at 10:00 am.
Resolution:  Remedy is now up and running as of 11:15 am.
11/30/2006 1:00:56 PM - Fernwood Remote Access Firewall Optimization Maintenance
Description:   Date: Monday, December 4, 2006 Time: 6:00pm - 10:00pm ET Description: On Monday, December 4, 2006 at 6:00 - 10:00 p.m. CIT network engineers will perform a scheduled maintenance on the NIHnet Remote Access Service firewalls located in the Fernwood Building. During this maintenance window, all properly configured NIHnet Remote Access VPN clients will connect to NIHnet via the Building 12 Remote Access VPN connection point. Impact: Due to the redundant design of the NIHnet Remote Access Service there is no expected impact to end-user connectivity during this maintenance window. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The maintenance was completed.
11/30/2006 12:58:01 PM - Fibergate Off-Campus Fiber Relocation Maintenance
Description:   Event Details: Date: Saturday, December 9, 2006 Time: 6:00a.m. - 12:00p.m. ET Description: On Saturday, December 9, 2006 at 6:00 a.m. - 12:00 p.m., Fibergate will perform a scheduled maintenance to relocate a 72-count fiber cable in support of footpath bridge construction in Rockville, MD. Impact: Please note this Fibergate Off-Campus Fiber Relocation Maintenance activity has been assessed with a Service Affecting (SA) impact with affected off-campus sites experiencing minimal network disruption of less than 15 minutes while each fiber pair is re-spliced onto a new path. During the six (6) hour maintenance window the NIH off-campus locations listed below will experience a momentary connectivity disruption to NIHnet. 9800 Medical Center Drive: NHGRI, NHLBI, and NICHD 5 Research Court: NHLBI and NIDCD 9610 Medical Center Drive: NCI Also, please note the Fernwood NIHnet GigE connection to the Internet will be affected by this Fibergate maintenance activity. Therefore, all traffic between NIHnet <---> Internet will go via the Building 12 OC-3 path. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Maintenance was completed successfully.
11/29/2006 11:41:48 AM - Delay in Email for MAILFWD and NON-CES messages
Description:   NIH E-Mail gateway's are seeign a possible delay of 10-15 minutes for messages being delivered to NON-CES Messaging Systems (Non-Exchange systems), as well as mail being sent through the MAILFWD mail relay service. CES Engineers are currently working on this issue and have engaged the gateway appliance vendor support.
Resolution:  delays have cleared and mail is flowing normally
11/21/2006 2:11:09 PM - Building 49 Upgrade and GigE Module Installation Maintenance
Description:   Event Details: Date: Wednesday, November 29, 2006 Time: 7:30pm - 8:30pm ET Description: On Wednesday, November 29, 2006 at 7:30pm – 8:30pm NIHnet engineers will perform a maintenance activity to upgrade the Building 49, B49-6500 CatOS and install a GigE Module. Impact: Please note this Building 49, B49-6500 CatOS Upgrade and GigE Module Installation Maintenance has been assessed with a Service Affecting (SA) impact. Specifically, network access within Building 49 will be disrupted during the first 30 minutes of the one-hour maintenance window while the B49-6500 switch is rebooted and the IOS is upgraded. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Maintenance completed successfully.
11/21/2006 9:30:10 AM - Fernwood Internet DMZ Optimization Maintenance - Phase I
Description:   Date: Wednesday, November 29, 2006 Time: 6:00pm - 7:00pm ET Description: On Wednesday, November 29, 2006 at 6:00pm - 7:00pm NIHnet engineers will be implementing Phase I of the Fernwood Internet DMZ Optimization Maintenance. This maintenance will provide standardization of the NIHnet Internet Outside Routers by replacing the Fernwood-6500-Router with the Fernwood-7600-Outside router. Impact: Please note that the Phase I Fernwood Internet DMZ Optimization Maintenance has been assessed with a Service Affecting (SA) impact. Specifically, this maintenance activity will cause a momentary network access disruption during the one-hour maintenance window at 6:00pm - 7:00pm. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The maintenance has been completed.
11/20/2006 10:15:23 AM - Commons and IMPAC II applications are experiencing issues
Description:   Commons, QVR, and IMPAC II applications are experiencing issues. Engineers are aware of the issue and they are working diligently to resolve it.
Resolution:  This issue has been resolved and all applications are functioning properly.
11/20/2006 8:02:27 AM - Network Outage in Building 31/Children’s Inn
Description:   We are currently experiencing a network outage in Building 31 that is affecting network and internet connectivity at Building 31 and the Children's Inn. NIHnet engineers are aware of the issue and are onsite and working diligently to resolve this issue. Updates to come...
Resolution:  At 8:26 a.m the primary B31 switch was restored. As of 9:10 a.m this morning all network connectivity to B31 and Children's Inn was re-established.
11/20/2006 7:33:20 AM - NCI users are experiencing login and network issues
Description:   NCI users located in the EPN Building and EPS building are experiencing issues logging into their computers and accessing their home drive.
Resolution:  The server was rebooted and users should be able to aceess their home drives and login to the network.
11/16/2006 3:08:18 PM - NIHnet: VPN Users Experiencing Issues
Description:   Description: Some customers connecting to NIH VPN are experiencing intermittent connectivity issues via Building 12 equipment. NIHnet Engineers are currently failing over to Fernwood to resolve the connectivity issues. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The VPN issues have been resolved.
11/15/2006 1:36:16 PM - Connectivity and Data Transfer Issues with the MSN IM client
Description:   Event Details: Currently, users may experience connectivity and data transfer issues when attempting IM communication via the MSN IM client only over the NIH IM gateway service. The EMIB IM Administrators are aware of the issue and are working to resolve the situation as quickly as possible.
Resolution:  Connectivity to MSN has been resolved.
11/13/2006 10:38:09 AM - NIHnet Remote Access Service Firewall Scheduled Maintenance
Description:   Date: Saturday, November 18, 2006 Time: 7:00am - 10:00am ET Description: On Saturday, November 18, 2006 at 7:00 - 10:00 a.m. CIT network engineers will perform a scheduled maintenance on the NIHnet Remote Access Service firewalls located in the Fernwood Building. During this maintenance window, all properly configured NIHnet Remote Access VPN clients will connect to NIHnet via the Building 12 Remote Access VPN connection point. Impact: Due to the redundant design of the NIHnet Remote Access Service there is no expected impact to end-user connectivity during this maintenance window. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The NIHnet Remote Access Service Firewall maintenance has been completed.
11/13/2006 9:14:51 AM - The NIHCESGAL.NIH.GOV LDAP instance needs to be rebuilt.
Description:   Due to problems over the weekend, the NIHCESGAL.NIH.GOV LDAP instance needs to be rebuilt. It is being worked on currently and updates will be posted during the day.
Resolution:  The nihcesgal directory is currently back online and available for service.
11/12/2006 3:48:00 AM - HN Test - disregard
Description:   hot news test
Resolution:  test is resolved
11/8/2006 6:03:29 PM - CRC is not receiving Overhead Pages
Description:   Currently the overhead paging system is non- operable in CRC. Technicians are aware of the issue and are investigating the cause. Updates will be provided as they become available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The issue with the CRC overhead paging system has been resolved.
11/7/2006 2:58:42 PM - Building 12 <--> Sterling, VA NIH Data Center Maintenance
Description:   Event Details: Date: Thursday, November 9, 2006 Time: 5:00pm - 6:00pm ET Description: On Thursday, November 9, 2006 at 5:00pm - 6:00pm NIHnet engineers will perform a Demand Maintenance Activity to replace a defective GigE module in the B12-6500-NCCS router located in the Building 12 NIH Data Center. Impact: This maintenance activity is not expected to impact connectivity between the Building 12 <--> Sterling, VA NIH Data Centers because all traffic flowing between locations will be moved to an alternate path prior to removing the defective GigE module from the B12-6500-NCCS router and installation of a new GigE module. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Building 12 <--> Sterling, VA NIH Data Center Maintenance was unsuccessful. The maintenance was backed out due to faulty chassis slots.
11/6/2006 7:57:34 AM - Upcoming 48 Hour NTLM Test
Description:   NTLMv1 will be shutdown starting on November 7, 2006 at 8:00 a.m. for 48 hours to determine what systems and applications will be impacted when NTLMv1 is turned off. This is taking place prior to the permanent shutdown of NTLMv1.
Resolution:  The 48 hour NTLMv1 Shutdown test is over. We have returned to the previous level (level 4) which allows NTLMv2 and NTLMv1 and refuses LM.
11/5/2006 2:23:28 PM - Batch Processing Not Available on Titan System
Description:   Date: Sunday, November 5, 2006 At this time, no batch jobs are running on the Titan System. The technicians are aware of the problem and are currently working to resolve it. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  All mainframe systems are back up and online as of 20:00.
11/3/2006 3:34:58 PM - Building 45 Generator Power Test Maintenance Activity
Description:   Date: Monday, November 6, 2006 Time: 7:00am - 8:00am ET Description: On Monday, November 6, 2006 at 7:00am - 8:00am NIH Facilities engineers will perform a Scheduled Generator Power Test Maintenance activity in Building 45. http://orf2.od.nih.gov/utilityshutdown/scripts/utility_shutdown.asp?seq=6640 Impact: The Building 45 Generator Power Test Maintenance Activity is not expected to affect the NIHnet data equipment in Building 45 because electrical power to the data network equipment will be protected by dedicated UPS power while normal power is transferred to emergency power systems during the generator test maintenance window. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Building 45 Generator Power Test Maintenance Activity was completed successfully.
11/3/2006 10:12:32 AM - OD Home Server connection problems for Executive Blvd
Description:   Home server memory problem is causing intermittent connection issues and slow response times. The Network Operations team will reboot the server at 5:30p.m. this evening. Until that time they will monitor and manage the memory allocation and make adjustments as needed.
Resolution:  The OD Home Server connection problems for Executive Blvd have been resolved. The server was rebooted and is now functional.
10/24/2006 5:45:27 PM - NIH Portal Servers Maintenance
Description:   Date: Tuesday 10/24/06 Time: 8:20 PM - 9:00 PM Due to issues from Saturday's network outage, the NIH Portal servers will need to be rebooted this evening. This will mean a temporary service outage between approximately 8:20 and 9 PM. During that time, the temporary maintenance page should be displaying, permitting access to many applications currently in the portal. Thank you for your understanding, and please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The NIH Portal Servers Maintenance was completed successfully.
10/23/2006 5:17:40 PM - Helix Systems Downtime for Maintenance
Description:   Helix Systems Downtime: Helix, Nimbus, Quasar, NIH Biowulf cluster (biowulf.nih.gov) and Web of Science / Porpoise (publishorperish.nih.gov) will be unavailable on Thursday October 26 starting at 6 PM in order to perform system maintenance. It is anticipated that the systems will be available by no later than midnight. During this period, Helix mail will not be accessible, but no messages will be lost. This message is being sent to all users of the Helix Systems.
Resolution:  Helix Systems Maintenance was completed successfully.
10/23/2006 4:13:38 PM - NIH Campus elevator and blue light phones outage
Description:   We are currently experiencing loss of dial tone for approximately 65 elevator and blue light emergency telephone lines on campus. Equipment in the 5ESS telephone switch failed causing the outages. The estimated repair time is at 7pm on Monday, October 23, 2006.
Resolution:  The elevator phones and blue light phone lines are restored.
10/23/2006 11:44:56 AM - Connectivity Issues in Data Center
Description:   We recently experienced a loss of connectivity in the Building 12 Data Center due to internal network spanning tree issues. All Building 12 NIH Data Center services have been resotred.
Resolution:  This issue has been resolved.
10/20/2006 3:22:39 PM - Rockledge 1 is experiencing power issues.
Description:   Date: 10/20/2006 3:20PM Description: Rockledge 1 is experiencing power issues. The issue is currently being investigated. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Rockledge 1 power issue has been resolved.
10/18/2006 6:00:34 PM - Central E-mail Service issues
Description:   Date:10/18/06 Time:6:00 PM The Central E-mail service is currently experiencing issues sending outbound mail to the internet. CES engineers are investigating. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Central E-mail Service issues were resolved.
10/17/2006 7:51:17 PM - Construction around Campus Buildings 1 and 2
Description:   The installation of a telecommunications duct bank has been scheduled from 7:00 am until 4:00 PM on the following dates: -- Saturday and Sunday, 21-22 October -- Saturday and Sunday, 28-29 October -- Saturday, 4 November This work will affect parking and sidewalk access to areas near campus Buildings 1 and 2. Network and telecommunication services are NOT expected to be affected, but please do promptly report any issues to the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov.
Resolution:  The installation of the telecommunications duct bank was successful but additional work will be scheduled.
10/17/2006 6:49:11 PM - NIDA Network Maintenance
Description:   Date: Friday, October 20, 2006 Time: 7:00pm - 10:00pm Description: Friday, (Oct. 20, 2006) between 7:00pm - 10:00pm, CIT will be working with IRMB support installing new network equipment for the NIDA network in the NSC building. Impact: During the (3 - Hour) maintenance window Friday night, all the NIDA servers will be shutdown while the new network equipment is being installed. There will be no access to the NIDA Share drives, Archive Email, NEPS or Terminal Servers. NIH Remote Access services, (VPN, Parachute and OWA Email) will not be affected. After the maintenance is completed the NIDA Network will be returned to full operation. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Maintenance has been completed.
10/16/2006 1:49:30 PM - NTLMv1 Shutdown Test
Description:   Date: October 17, 2006 8:00 a.m - October 18, 2006 8:00 a.m. Time: 24 hour span Description: NTLMv1 will be shutdown for a 24 hour time period to determine what systems and applications will be impacted when NTLMv1 is turned off. This is taking place prior to the permanent shutdown of NTLMv1 on October 31, 2006. Impact: It is known that many Macintosh computers, some scientific equipment and some software applications will be adversely affected. There are other unknown systems/devices that use NTLMv1 for authentication which may experience problems related to access. Known Issues: 1. Macintosh users running Outlook 2001 a. Solution: Use OWA for duration of test 2. Macintosh users running OS versions prior to 10.4 may not have access to their files and applications on the NIH network 3. Other applications (PC and/or Mac) have been identified by local IC staff
Resolution:  The NTLMv1 test was completed at 8:00 AM today.
10/12/2006 7:18:46 PM - Building 13 Power Maintenance
Description:   Date: Saturday, October 14, 2006 Time: 6:00am - 6:00pm ET Description: On Saturday, October 14, 2006 at 6:00am - 6:00pm NIH Facilities engineers will perform a Scheduled Power Maintenance activity in Building 13. http://orf2.od.nih.gov/utilityshutdown/scripts/utility_shutdown.asp?seq=6597 Impact: This Building 13 Power Maintenance Activity will affect all NIHnet data networking device throughout the duration of the 12-hour power maintenance activity. Therefore, NIHnet engineers will proactively power down all NIHnet data network devices one-hour(5:00am) prior to the scheduled power shutdown at 6:00am. After completion of the Building 13 Power Maintenance, NIHnet engineers will power up all NIHnet data network devices and verify connectivity and service restoration. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Building 13 Power Maintenance has been completed.
10/12/2006 6:06:07 PM - Internet Firewall Changes
Description:   Date: Tuesday, October 24, 2006 Time: 8:00am To better support the existing policy of filtering ports that pose a risk to the Internet community, NIH will implement egress filtering on ports 135, 137, 138, 139, and 445 for TCP and UDP traffic at 8 am on October 24, 2006. This change will prevent NIH machines from initiating communication to the Internet on these ports. This change will not affect web browsing (HTTP), file transfers (FTP), secure shell (SSH), or other common Internet activities. An exception procedure will be available to permit traffic as necessary. To obtain an exception, contact the IRT (irt@mail.nih.gov). The ports will still be available internally to NIH, including remote access, and between NIH and HHS. This change has been approved by the NIH Chief Information Security Officer, the IC ISSO community, and the Information Technology Management Committee (ITMC).
Resolution:  The maintenance has been successfully completed.
10/12/2006 4:03:04 PM - Sterling NCCS Phase IV Maintenance Activity
Description:   Date: Thursday, October 19, 2006 Time: 5:00pm - 9:00pm Description: On Thursday, October 19, 2006 at 5:00pm - 9:00pm NIHnet engineers will be implementing Phase IV of the Sterling NCCS Implementation design. Impact: Please note this Sterling NCCS Phase IV Maintenance activity has been assessed with a Service Affecting (SA) impact. Specifically, between the hours of 5:00pm - 9:00pm, this maintenance activity will cause approximately 20 minute outages for each of the VLANs located in the Sterling NCCS Data Center. Please note this Sterling NCCS Phase IV Maintenance activity will not affect NIHnet connectivity to NLM, Internet, or Internet(2) via the NCCS Data Center in the Qwest Sterling, VA. facility. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Sterling NCCS IV Maintenance Activity has been completed.
10/11/2006 5:52:34 PM - Dogfood pre-production environment patching
Description:   EMIB is patching systems in the "Dogfood" pre-production environment for the NIH Active Directory, with an expected rolling outage of a few minutes per host. This will not affect any production services; it will only affect development systems that may interact with that environment. The maintenance has an estimated completion time of one hour. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Maintenance was completed successfully.
10/11/2006 10:03:49 AM - Remedy Mail Engine Upgrade - October 14, 2006
Description:   On Saturday October 14th the CIT Support Technologies Group will be upgrading the Remedy email engine. The work is scheduled to run from 9:00 AM to 4:00 PM. During this time, no outbound email notifications will be generated and inbound email will have to be processed manually. There will also be periods of general Remedy slowness and a reboot may be necessary at some point during the maintenance. When the maintenance is complete, Remedy users will see some differences in the behaviors of Remedy notifications. A CIT Knowledge Base article has been created to document these differences. The record can be found at http://kb.cit.nih.gov/ww2_record_cit.asp?id=4601. Should any other differences become apparent as we proceed with the upgrade we will update this article appropriately. Problems with the upgrade impacting anticipated return to service will be documented in this Hot News. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Remedy Email Engine Upgrade is functionally complete. While a few specific cleanup issues do remain, all broadly-service affecting issues have been addressed. Thank you for your patience while we worked through this critical upgrade.
10/11/2006 8:18:51 AM - Building 12 NIH Data Center Network Scheduled Maintenance
Description:   On Saturday, October 21, 2006 at 8:00am - 4:00pm ET NIHnet engineers will perform a Scheduled Maintenance activity on the Building 12 NIH Data Center network routers in support of efforts to optimize network redundancy on the Building 12 NIH Data Center Network. Please note this Building 12 NIH Data Center network maintenance has been assessed with a Service Affecting (SA) service impact. Specifically, this maintenance activity will cause a momentary data network connectivity disruption for certain servers located in the Building 12 NIH Data Center while each network uplink is sequentially optimized for connectivity redundancy. Expected outages for any particular system should last no longer than 15 minutes. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Building 12 NIH Data Center network maintenance was unsuccessful and backed out. All network connectivity, applications, and data services have been restored. Please contact CIT DCS CAP (citdcscap@mail.nih.gov) regarding any problems related to this maintenance.
10/10/2006 9:40:10 PM - Breeze Server Downtime
Description:   The Breeze servers will be down for maintenance on Wednesday, 11 October from 7:30-8:00 AM. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions
Resolution:  Breeze maintenance has been successfully completed.
10/10/2006 11:30:04 AM - NIHnet Internet DMZ Failover Firewall Implementation
Description:   On Wednesday, October 11, 2006 between 6:00pm - 7:00pm NIHnet engineers will activate a redundant firewall on the Fernwood Internet DMZ path. Please note this NIHnet Internet DMZ Firewall Implementation activity will cause a momentary disruption to all traffic between NIHnet <--> Internet during the first 30 minutes of the one-hour maintenance window while NIHnet <--> Internet traffic is migrated from the Building 12 path to the Fernwood Internet DMZ path. Everyone at NIHnet that uses NIHnet to connect to the Internet may see a very quick disruption when sending / receiving data traffic to / from the Internet via NIHnet. The expected loss of connectivity and performance degradation between NIHnet <--> Internet should be minimal while the new redundant Fernwood Internet DMZ firewall is activated. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Internet DMZ Failover Firewall Implementation maintenance has been completed successfully.
10/10/2006 10:39:07 AM - NCI 6116 users cannot access shared drives.
Description:   NCI users are unable to access to their shared drives.
Resolution:  The server is now back up and users should be able to access shared drives.
10/4/2006 3:23:57 PM - NIHnet <--> Internet DMZ Demand Maintenance Activity
Description:   On Thursday, October 5, 2006 between 6:00pm - 7:00pm ET NIHnet engineers will perform a Demand Maintenance on the Fernwood path of the NIHnet Internet DMZ in support of a Gigabit bandwidth upgrade to the Level(3) service provider. Impact: This Fernwood Internet DMZ maintenance activity has a high probability of causing a single momentary disruption to NIHnet <--> Internet traffic during the first 30 minutes of this one-hour maintenance window while NIHnet <--> Internet traffic migrates onto the new GigE connection to Level(3). Please note the total number of expected momentary disruptions to traffic flow between NIHnet <--> Internet during the maintenance window should not exceed more than one (1) disruption while the NIHnet Internet path is migrated and tested. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The maintenance has been completed successfully.
10/3/2006 3:31:05 PM - CIT Remedy System Outage to Perform Maintenance
Description:   The Support Technologies Group will be performing Maintenance on the CIT Remedy System starting at 8:00 PM on Sunday October 8th. The system will be taken offline at 8:00 PM and is expected to return to service at 1:00 AM on Monday, October 9th. Users will be unable to submit Service Requests and register for CIT courses during this outage. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Support Technologies Group has completed the maintenace. The affected systems have been returned to operation. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
10/3/2006 2:09:12 PM - Building 13 Power Shutdown Maintenance
Description:   Date: Saturday, October 7, 2006 Time: 6:00am-9:00pm Description: On Saturday, October 7, 2006 at 6:00am-9:00pm the NIH facilities department will be performing electrical maintenance in Building 13. During this maintenance, the basement LAN closet equipment will be affected due to both commercial and emergency power being turned off while a new emergency generator transfer switch is installed. Impact: Please note this Building 13 Power Shutdown Maintenance has been assessed with a Service Affecting (SA) impact. Specifically, this maintenance activity will result in a loss of data network connectivity to devices that are connected to LAN equipment within the building 13 IBC basement LAN closet. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Maintenance activity has been cancelled.
10/3/2006 1:10:19 PM - Breeze Maintenance
Description:   Breeze will be down today, Tuesday, October 3rd at 6pm for maintenance. The downtime should be no more than 15 minutes.
Resolution:  The Breeze maintenance has been completed.
10/3/2006 11:15:17 AM - ITAS currently unavailable
Description:   Users are currently unable to access ITAS. Users are receiving "login error" notifications when trying to enter the website. ETR is unknown at this time.
Resolution:  As of 11:32 AM, ITAS issues have been resolved.
9/29/2006 7:43:21 AM - Fishers Lane Power Outage
Description:   There is a power outage in 5625 Fishers Lane that has affected all network devices. CIT network engineer is en route. There is no ETR at this time. Updates will be posted as they become available.
Resolution:  The power has been restored and the issue has been resolved.
9/27/2006 4:33:32 PM - Sterling NCCS Phase II & III Implementation Maintenance
Description:   Date: Friday, October 6, 2006 Time: 6:00pm - 10:00pm Description: On Friday, October 6, 2006 at 6:00pm - 10:00pm NIHnet engineers will be implementing Phase II & III of the Sterling NCCS Implementation design. Impact: Please note this Sterling NCCS Phase II & III Maintenance has been assessed with a Service Affecting (SA) impact. Specifically, between the hours of 6:00pm - 8:00pm, this maintenance activity will cause a momentary network connectivity disruption between the Building 12 <----> Sterling, VA Data Centers. Additionally, connectivity to the Internet and Internet(2) in the Sterling Data Center will also be momentarily disrupted during the first two hours of the maintenance window (6:00pm - 8:00pm). Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Sterling NCCS Phase II & III Implementation Maintenance has been completed.
9/25/2006 12:49:08 PM - NIH Portal Maintenance Monday 9-25-06 at 8:45 PM
Description:   There will be a short portal outage Monday evening 9-25-06 between 8:45PM and 9:00 PM that should not last longer than 10-15 minutes. During that period the portal temporary page will provide notice and direct users to alternate access for the many applications integrated with the Portal. Thank you for your understanding as we work to improve this service.
Resolution:  The maintenance has been completed successfully.
9/25/2006 7:01:28 AM - ORS users unable to access their H drives
Description:   ORS users are currently unable to access their H drives. The services and applications that are impacted are personal folders in Outlook and files in the H drive
Resolution:  This issue has been resolved.
9/20/2006 3:21:33 PM - NIHnet Internet DMZ Failover Firewall Implementation (Cancelled)
Description:   Date: Wednesday, September 27, 2006 Time: 6:00pm - 7:00pm Description: On Wednesday, September 27, 2006 between 6:00pm - 7:00pm EST NIHnet engineers will activate a redundant firewall on the Fernwood Internet DMZ path. Impact: Please note this NIHnet Internet DMZ Firewall Implementation activity will cause a momentary disruption to all traffic between NIHnet <--> Internet during the first 30 minutes of the one-hour maintenance window while NIHnet <--> Internet traffic is migrated from the Building 12 path to the Fernwood Internet DMZ path. The expected loss of connectivity and performance degradation between NIHnet <--> Internet should be minimal while the new redundant Fernwood Internet DMZ firewall is activated. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Please note the impending NIHnet Internet DMZ Failover Firewall Implementation activity has been cancelled and will be rescheduled during the mid-October 2006 timeframe. Therefore, please disregard the initial maintenance announcement for Wednesday, September 27, 2006 between 6:00pm - 7:00pm EST.
9/18/2006 8:30:06 AM - Remedy Outbound Messages
Description:   The Remedy outbound messages are experiencing some issues. The STG team is aware of the issue and they are working on a resolution. There is no ETR at the moment.
Resolution:  This issue has been resolved.
9/15/2006 9:15:01 AM - 2006 Building 10 Electrical Power Systems Test
Description:   Date: Saturday, September 23, 2006 Time: 7:00am - 3:00pm Description: On Saturday, September 23, 2006 at 7:00am – 3:00pm there will be an Emergency Power Systems test in Building 10 and the ACRF. http://orf2.od.nih.gov/utilityshutdown/scripts/utility_shutdown.asp?seq=6465 Impact: The Building 10 Electrical Power Systems Test Maintenance Activity is not expected to impact the Building 10 NIHnet data network infrastructure equipment because all Core, Distribution, and Access data network devices are supported by sufficient UPS and / or generator back-up power. However, there may be limited Wireless Network connectivity at various locations within Building 10 due to a limited number of Wireless Access Points (AP) using commercial power as a sole power source. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Building 10 Electrical Power Systems Test has been completed.
9/13/2006 2:19:39 PM - Sterling Data Center Phase 1 Redundancy Implementation
Description:   Date: Wednesday, September 20, 2006 Time: 6:00pm - 8:00pm Description: On Wednesday, September 20, 2006 at 6:00pm - 8:00pm NIHnet engineers will be implementing Phase 1 of the NCCS Redundancy Design. Phase 1 will provide a redundant location to attach external peers, as well as provide redundant connection points for the local firewalls. Impact: Please note this Sterling Data Center Phase 1 Redundancy Implementation has been assessed with a Service Affecting (SA) impact. Specifically, this maintenance activity will cause a momentary network access disruption during the two-hour maintenance window at 6:00pm - 8:00pm. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Sterling Data Center Phase 1 Redundancy Implementation maintenance was completed successfully.
9/12/2006 6:25:36 PM - NCI network is down at Executive Boulevard
Description:   There is a power outage in the vicinity of Executive Blvd affecting NCI. Pepco and NCI personnel are diligently working towards a swift resolution. There is no ETR at this time.
Resolution:  All power at 6116 Executive Blvd have been restored and all services to NCI that were disrupted as a result of the power incident have been resolved. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
9/12/2006 11:06:41 AM - Building 13 ORS phones lines have been cut.
Description:   The phone lines in Building 13 have been cut. Engineers are aware of the issue and they are working towards a resolution. There is no ETR at the moment.
Resolution:  Phone service and cable TV service have been restored in building 13. If customers experience any problems please contact telecommunications repair at (301) 402-9935.
9/6/2006 11:41:28 AM - NAppMan monitoring of NIDB available once again.
Description:   Updates were made to the NIDB system that allow for effective monitoring by NAppMan.
Resolution:  The component column of the NAppMan dashboard has been turned on for NIDB.
9/1/2006 9:18:05 AM - Water Leak affecting Phone Lines in Building 49
Description:   Leaking water issues are affecting 150 analog telephone lines in Building 49--the lines are not operational at this time. There is currently no estimate for time of repair.
Resolution:  All phone service has been restored to Building 49.
8/30/2006 10:16:55 PM - NIH Login Services
Description:   Date: Wednesday, August 30, 2006 Time: 9:00PM - No ETA Description: NIH Login Services are experiencing intermittent connectivity. Emergency maintenance is under way. There is no ETA.
Resolution:  NIH Login Services have been restored.
8/30/2006 3:21:20 PM - AntiVirus Updates on NIH Portlet Servers
Description:   Date:Thursday, August 31, 2006 Time: 6:00 AM to 6:30 AM, Description: On Thursday, August 31, from approximately 6 to 6:30 AM, AntiVirus updates will be installed on the NIH Portlet Servers. Impact: Some portlet content may be affected and not appear momentarily. Refreshing the portal page after this window should restore content. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  AntiVirus Updates on NIH Portlet Servers have been successfully completed. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
8/30/2006 2:32:04 PM - Building 12 B12-6500-NCCS Router Emergency Reload Maintenance
Description:   Date: Wednesday, August 30, 2006 Time: 7:00pm - 8:00pm Description: On Wednesday, August 30, 2006 at 7:00pm 8:00pm an emergency device reload maintenance will be performed on the Building 12 B12-6500-NCCS router. This emergency device reload maintenance is necessary to proactively eliminate suspected memory allocation problems on the Building 12 B12-6500-NCCS router. Impact: The Building 12 B12-6500-NCCS Router Emergency Reload Maintenance activity will cause a momentary (<15 minutes) connectivity disruption between Building 12 NIHnet <----> Sterling, VA NIHnet data network facilities. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Maintenance performed by NIHnet Engineers was successfully completed.
8/28/2006 4:19:39 PM - Democracy II Diverse Fiber Path Optimization Maintenance
Description:   Date: Tuesday, September 5, 2006 Time: 7:00pm - 9:00pm Description: On Tuesday, September 5, 2006 at 7:00pm - 9:00pm NIHnet engineers will be implementing a new Democracy II uplink fiber path to NIHnet. This new fiber path will eliminate an existing single point of failure at Building 10. Impact: Please note this Democracy II Fiber Path Optimization Maintenance has been assessed with a Service Affecting (SA) impact. Specifically, this maintenance activity will cause a momentary network access disruption during the two-hour maintenance window at 7:00pm - 9:00pm. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The maintenance event was completed without complication.
8/26/2006 7:02:18 AM - Blackberry Use Not Available
Description:   Blackberry usage is unavailable. There is currently no ETR.
Resolution:  Blackberry issue resolved. This outage is not due to the Data Center Maintenance.
8/25/2006 1:18:18 PM - No access to ADB (IMS is not responding)
Description:   The IMS Access to ADB is not responding. Support teams are working the issue; no ETA at this time.
Resolution:  The ADB issues have been resolved.
8/23/2006 1:11:51 PM - Aspect Call Center Maintenance: 23 August 2006
Description:   The Aspect Call Center switch is undergoing maintenance from 9:00-10:00 pm on Wednesday, 23 August to accommodate additional licenses and trunk lines. A system reboot will be necessary to complete the process. During this one hour timeframe, phone numbers supported by this switch will be re-routed to alternate phone numbers where service and/or informational messages will be provided. NIH Help Desk customers are asked to use the web (http://ithelpdesk.nih.gov/) or email (ithelpdesk@nih.gov) to submit service requests during this period. The 24x7 NOC and CAP teams will be directly available via the re-routed phone numbers. If you have any questions or concerns about the impact of this maintenance, please contact the Continuity Assurance Program at citdcscap@mail.nih.gov.
Resolution:  STG confirmed that the maintenance was completed sucessfully at 10:25 pm
8/22/2006 8:56:49 PM - "My Bookmarks" and "Community Bookmarks" Portlets Maintenance
Description:   Date: Wednesday, August 23, 2006 Time: 8PM-11PM Description: The portlets "My Bookmarks" and "Community Bookmarks" will undergo maintenance Wednesday evening from approximately 8 pm till 11 pm. During this period content in these portlets may be unavailable. This also includes any other portlets created from these corresponding web services.
Resolution:  NIH Portal Administrator advised that the maintenance was completed successfully.
8/22/2006 3:30:53 PM - NIH CES Maintenance - OWA/POP/IMAP
Description:   CES Engineers need to perform a brief maintenance on the NIH CES load balancers tonight at 22:00 PM. The expected service impact is 5 minutes. Please accept our apologies for the short notice. OWA/POP/IMAP will drop during this maintenance. Very best regards, Kevin
Resolution:  this maintenance was completed successfully
8/22/2006 3:18:23 PM - Third external DNS server for NIH now available
Description:   The following applies to servers currently using the BIND external DNS servers ns.nih.gov and ns2.nih.gov. If you own a machine that is configured to use the BIND servers for DNS services please read on. In order to increase redundancy in the NIH external (Internet) DNS system, CIT has added a third external DNS server in Sterling Virginia. This server is equivalent to ns.nih.gov and ns2.nih.gov. In order to take advantage of the redundancy provided by the additional DNS server, if you are currently using ns.nih.gov (128.231.128.251) and ns2.nih.gov (128.231.64.1) as DNS resolvers we recommend adding ns3.nih.gov (165.112.4.230) to your DNS configuration as an additional resolver. If your server is located in Sterling Virginia we recommend placing the ns3.nih.gov as your first DNS server in your configuration file. NIH external DNS administrators
Resolution:  Event was entered in the KB as a permanent record.
8/22/2006 10:35:39 AM - Email Server Emergency
Description:   NCI users may have experienced a brief email outage. EMIB engineers are aware of the problem and are currently fixing same.
Resolution:  EMIB Exchange engineers have resolved the issue with the NCI mailbox server. All Exchange services have been restored and all NCI users should now have full access to their mailboxes.
8/21/2006 8:12:58 AM - Network Service Disruption Affecting Multiple ICs
Description:   Description and Impact : The NIH Network Team has reported a network disruption that is affecting the following customers ; OD, NHLBI, NCI in Buildings 10, 13, 14, 28 and possibly others. NIHnet engineers are on site working to resolve this issue. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Network connectivity has been restored to full capacity.
8/18/2006 12:29:32 PM - Reminder - NIH CES extended maintenance
Description:   This is a reminder of a previously annouanced maintenenace. The NIH Central Email Service will undergo an extended maintenance window tonight, Friday beginning at 10:00 PM. The maintenance involves significant Storage Area Network (SAN) upgrades that will require extended downtime. We expect to complete the work by 3 AM. Please note that SMTP mail will queue for delivery and will not be delivered until the SAN has been returned to service. We regret any inconvenience this required maintenance may cause for your customers.
Resolution:  This work was completed.
8/18/2006 11:42:51 AM - No access to ADB (IMS is not responding)
Description:   As of approximately 11:15 this morning, the IMS Access to ADB is not responding. Support teams are working the issue; no ETA at this time. IMPACT: Customers will not be able to access ADB (they'll see Error: EMS1164E Pass mode logon request failed for application IMS)
Resolution:  The NIH Help Desk was notified by the IMS/ADB support team at 11:50 that the issue has been resolved. Access to ADB has been restored.
8/17/2006 3:46:24 PM - Building 12 NIH Data Center Network Scheduled Maintenance
Description:   Date: Saturday, August 26, 2006 Time: 6:00am - 8:00pm Description: On Saturday, August 26, 2006 at 6:00am - 8:00pm ET NIHnet engineers will perform a Scheduled Maintenance activity on the new Building 12 NIH Data Center network routers in support of efforts to implement the new Building 12 NIH Data Center Network Design. This maintenance will complete activities originally scheduled during the failed July 29th maintenance. Impact: Please note this Building 12 NIH Data Center network maintenance has been assessed with a Service Affecting (SA) service impact. Specifically, this maintenance activity will cause a momentary data network connectivity disruption for certain servers located in the Building 12 NIH Data Center as the remaining 4 Data Center switches are migrated to the new infrastructure. Expected outages for any particular system should last no longer than 15 minutes. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  As of now there will be no more modification done to the Network in the Data Center. The network is up and stable and we will continue to monitor for any service impact. Please call NIH Help Desk to report any issues.
8/16/2006 9:53:16 PM - Building 10 Electrical Power Systems Test
Description:   Date: Saturday, August 19, 2006 Time: 7:00am - 10:00am Description: On Saturday, August 19, 2006 at 7:00am - 10:00am there will be an Emergency Power Systems test in Building 10 and the ACRF. Impact: The Building 10 Electrical Power Systems Test Maintenance Activity is not expected to impact the Building 10 NIHnet data network infrastructure equipment because all Core, Distribution, and Access data network devices are supported by sufficient UPS and / or generator back-up power. However, there may be limited Wireless Network connectivity at various locations within Building 10 due to a limited number of Wireless Access Points (AP) using commercial power as a sole power source. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Building 10, 10B Utility Electric shutdown Maintenance was completed successfully at 9:30am on Saturday, 19 August.
8/16/2006 3:24:09 PM - HOT NEWS TEST - PLEASE DISREGARD.
Description:   Testing Hot News Application.
Resolution:  Resolved
8/16/2006 3:22:53 PM - Building 10 Electrical Power Systems Test
Description:   Date: Saturday, August 19, 2006 Time: 7:00am - 10:00am Description: On Saturday, August 19, 2006 at 7:00am - 10:00am there will be an Emergency Power Systems test in Building 10 and the ACRF. Impact: The Building 10 Electrical Power Systems Test Maintenance Activity is not expected to impact the Building 10 NIHnet data network infrastructure equipment because all Core, Distribution, and Access data network devices are supported by sufficient UPS and / or generator back-up power. However, there may be limited Wireless Network connectivity at various locations within Building 10 due to a limited number of Wireless Access Points (AP) using commercial power as a sole power source. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  This Hot News was closed due the fact that the initial notifications did not go to the customers. Remedy Notifications were undergoing repairs.
8/16/2006 3:18:59 PM - NIDB monitoring unavailable until further notice.
Description:   NAppMan monitoring of NIDB components will be unavailable until further notice. Updates are necessary to accommodate recent changes to NIDB. NIDB components will appear on the NAppMan dashboard with a status of, "System is experiencing scheduled down time." NIDB itself is available and unaffected by the need for monitoring changes.
Resolution:  NAppMan monitoring of the NIDB system will be unavailable until further notice.
8/15/2006 4:41:34 PM - CES Mailbox servers - Emergency security patching
Description:   CES engineers will be applying security patches to the Central Email Service mailbox servers this evening at 9 PM. Service disruption will be 15 minutes.
Resolution:  The Emergency Security patching that was performed on the CES Mailbox servers was successfully completed.
8/15/2006 3:06:52 PM - Remedy Mail Upgrade Issues
Description:   Though all broadly service-affecting issues with Remedy Mail have were resolved yesterday, over the past few hours there have been several reports of minor issues trickling in, including some erroneous test notifications. We continue to work on these as we become aware of them. Therefore, please be sure to report any such issues you know of to your CIT Relationship Manager. Thanks for your patience and support.
Resolution:  The Remedy Mail Upgrade has been rolled back and Remedy notifications have been restored to service. Most notifications should be back at their pre-upgrade status. The modified Hot News notification interface that was implemented with the upgrade has been left in place and is working as intended. Thanks again for your patience while we worked through a challenging week.
8/15/2006 2:58:41 PM - Breeze Servers Going Down for Emergency Maintenance
Description:   The Breeze Servers have been shut down for an Emergency Maintenance. The predicted downtime for this maintenance is 15 minutes.
Resolution:  The Emergency Maintenance on NIH Breeze Servers has been successfully completed.
8/15/2006 1:54:32 PM - Reboot of CIT Remedy Server
Description:   The CIT Remedy Server requires an emergency reboot to allow for the application of emergency security patches. The server will be restarted at 2:15. Anticipated downtime will be 15 minutes. Thanks for your patience.
Resolution:  The server has been restarted and restored to service. Thanks for your patience!
8/15/2006 1:39:39 PM - Emergency Patching of Portal Servers
Description:   Date: Tuesday, August 14, 2006 Time: 1:15 PM - 1:45 PM Description: There will be an emergency maintenance on all Portal Servers. This is done in order to do some necessary Security Patching against an increase WORM activity. Impact: There will be a short interruption in portal service during this time.
Resolution:  The maintenance was completed successfully.
8/15/2006 1:05:15 PM - CIT File Server PETRI - Emergency reboot
Description:   EMIB staff will be rebooting PETRI to apply security hotfixes.
Resolution:  work completed
8/15/2006 12:39:41 PM - 6101 Executive Blvd Building Power Maintenance Activity
Description:   Date: Saturday, August 19, 2006 Time: 2:00pm - 6:00pm Description: On Saturday, August 19, 2006 at 2:00pm - 6:00pm there will be power repair maintenance at 6101 Executive Blvd that will affect all commercial power to the entire 6101 Executive Blvd building. Impact: The 6101 Executive Blvd Building Power Maintenance Activity should not impact the NIHnet data network infrastructure at the 6101 Exec. Blvd site because the NIHnet data network equipment is supported by UPS and generator power. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  6101 Exec. Blvd Building Power Maintenance was completed successfully.
8/15/2006 11:49:02 AM - NIHnet Remote Access VPN Firewall Hardware Upgrade Maintenance
Description:   Date: Thursday, August 16, 2006 Time: 6:00am - 8:00am Description: On Thursday, August 16, 2006 at 6:00am - 8:00am NIHnet engineers perform a firewall hardware upgrade activity on the NIHnet Remote Access VPN data network infrastructure. Impact: There is a high probability some Remote Access VPN connections will be disconnected and will require a reconnection request when a new Remote Access firewall is brought on-line. The probability of dropping a VPN connection will be dependant on the type of application that is in-use across the established Remote Access VPN session. Typical web-based traffic is not expected to be affected by this maintenance activity. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The NIHnet Remote Access VPN Firewall Hardware Upgrade was completed successfully.
8/14/2006 3:28:02 PM - NIH CES Blackberry Service - Emergency Maintenance
Description:   In order to apply required security patches we will be performing emergency maintenance on the NIH CES BES servers between 21:00 and 22:00 this evening - 8.14.06. This is a rolling outage - each server will be unavailable for approx 15 minutes. User impact is a service disruption of 15 minutes.
Resolution:  This maintenenace was completed at 21:20.
8/12/2006 1:59:16 PM - Scheduled Maintenance - CES mailbox servers
Description:   This maintenance had been previously announced. IC's have been informed. CES administrators and facilities engineers will perform firmware upgrades on multiple mailbox servers. The systems will be up and down for multiple reboots. The service disruption will be between 30 and 45 minutes. At 09:00 the following IC's are affected: NHLBI, NIEHS MLBX6 At 10:00 the following IC's are affected: NLM, NIDCD, NIDA, MLBX8 At 11:00 the following OPDivs's are affected: HRSA HRSAMLBX
Resolution:  Maintenance on the CES Mailbox Servers has been completed.
8/12/2006 7:58:52 AM - Scheduled Maintenance - CES mailbox servers
Description:   This maintenance had been previously announced. IC's have been informed. CES administrators and facilities engineers will perform firmware upgrades on multiple mailbox servers. The systems will be up and down for multiple reboots. The service disruption will be between 30 and 45 minutes. At 09:00 the following IC's are affected: NIDDK, NIAMS MLBX9 At 10:00 the following IC's are affected: NIA, NICHD, MLBX7
Resolution:  This maintenance was completed
8/11/2006 8:04:14 PM - Scheduled Maintenance - CES mailbox servers
Description:   This maintenance had been previously announced. IC's have been informed. CES administrators and facilities engineers will perform firmware upgrades on multiple mailbox servers. The systems will be up and down for multiple reboots. The service disruption will be between 30 and 45 minutes. At 22:00 the following IC's are affected: NIMH, NEI, NHGRI, NIDCR, CIT, CSR, ORS, NIAAA, FIC, FAES, VRC, NCI MLBX3 MLBX4 MLBX10 MLBX11 At 23:00 the following IC is affected, CC MLBX12
Resolution:  This maintenenace was completed
8/11/2006 1:19:00 PM - Building 12B Power Issues
Description:   Building 12B has lost power. NIH Engineers are on site investigating the issue. Updates will be sent as more information becomes available.
Resolution:  Power has been restored to Building 12B. All devices are functional.
8/11/2006 12:23:48 PM - NIHPRINT is currently experiencing problems
Description:   NIHPRINT is currently experiencing problems. Users are not able to connect to the print server. EMIB is currently looking into the root cause.
Resolution:  The NIHPRINT server issues have been resolved.
8/11/2006 11:58:55 AM - CWDM Ring I Diverse Fiber Path Optimization Maintenance
Description:   Date: Saturday, August 19, 2006 Time: 7:00am - 10:00am Description: On Saturday, August 19, 2006 at 7:00am - 10:00am NIHnet engineers will temporarily migrate Off-Campus CWDM traffic from the East-bound fiber path to the REDUNDANT West-bound fiber path in support of efforts to build a new, diverse East fiber path between 6000 Executive Blvd Bunker <----> Building12 NIH Data Center. Impact: There will be a momentary disruption to Off-Campus CWDM connected NIH Institutes and HHS OpDivs while traffic is migrated from the East-bound to the West-bound path. The connectivity disruption is expected to be minimal while traffic is moved to West-bound path between 7:00am-8:00am and then back onto the East-bound path between 9:00am-10:00am. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The CWDM Ring I East Diverse Fiber Path Maintenance was completed successfully at 8:36 AM on Saturday, 19 August.
8/9/2006 2:30:50 PM - CIT Remedy Server Maintenance - Sunday August 13, 9:00 AM
Description:   Maintenance on the CIT Remedy Server will be performed on Sunday, August 13th between the hours of 9:00 AM and 4:00 PM. During this time Remedy's inbound email processing and outbound email notifications will be disabled. There may also be short periods of system unavailability during this time, though it is anticipated that the other server functionality will remain largely unaffected for most of this period.
Resolution:  The Support Technologies Group has successfully completed the maintenance on the CIT Remedy Server.
8/5/2006 12:42:48 PM - Flooding in building 10, 5th floor, NINDS server room.
Description:   The NINDS server room located in building 10 on the 5th floor is experiencing flooding. All servers and services are unavailable until the situation has been assessed.
Resolution:  The issue has been resolved. All systems are operational.
8/1/2006 12:55:46 PM - ITAS database maintenance tonight from 9-10pm.
Description:   The ITAS Technical Team will be conducting routine monthly database maintenance tonight from 9-10pm. During this time users may experience slow response times and brief interruptions in service. Thanks for your patience! NIH/CIT/DECA ITAS Technical Support Team.
Resolution:  Maintenance was finished without incident.
7/31/2006 3:04:01 PM - Emergency Reboot of NIHCESMLBX10
Description:   NIHCESMLBX10 will be rebooted at 10:00PM on 7/31/2006
Resolution:  Reboot successful. Problem cleared. Server back online at 10:15 PM
7/27/2006 2:13:08 PM - NIH Portal possibly affected by NIH Data Center Maintenance
Description:   Due to the scheduled maintenance in the NIH Data Center on Saturday July 29th, it is possible the NIH Portal will experience some interruptions in service. If this occurs at all it is likely to be between the hours of 7 and 9 AM, but could also occur at any point between 6 AM and 4 PM which is the scheduled window for the Data Center maintenance.
Resolution:  Maintenance completed. See HN06209 for more details.
7/26/2006 2:48:48 PM - Maintenance work on NIHCESMLBX
Description:   Update driver on nihcesmlbx server; this issue was uncovered during the maintenance window last friday (7/21). Downtime is expected to be about 10 minutes.
Resolution:  Maintenance on this box was successful and is now complete.
7/26/2006 11:48:30 AM - Breeze Server Interface Upgrade
Description:   The Breeze servers will be down from 6 - 6:30 tonight to accomodate the interface upgrade.
Resolution:  Interface upgrade was successful. The Breeze server is back online.
7/26/2006 10:57:34 AM - CIT Remedy Services to be Affected by NIH Data Center Maintenance
Description:   Maintenance on the Network supporting the CIT Remedy Server is scheduled for Saturday July 29, 2006 from 6:00 Am to 4:00 PM. During this time Remedy's inbound email processing and outbound email notifications will be disabled. There may also be short periods of system unavailability during this time, though it is anticipated that the other server functionality will remain largely unaffected for most of this period.
Resolution:  Mail notifications have been re-enabled. Remedy is fully functional.
7/26/2006 10:45:32 AM - CIT Remedy System Outage to Perform Maintenance
Description:   The Support Technologies Group will be performing Maintenance on the CIT Remedy System starting at 8:00 PM on Sunday July 30, 2006. The system will be taken offline at 8:00 PM and is expected to return to service at 1:00 AM on Monday, July 31. Users will be unable to submit Service Requests and register for CIT courses during this outage. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Due to the network issues in the data center, this maintenance window has been postponed to a later date to be announced. Thanks for your patience.
7/25/2006 6:32:40 AM - NIDB (NIH Intramural Database) Server Move
Description:   Date: Tuesday, July 25th 2006 Time: 8:00am to 3:00 pm Description: Due to a mandated move of the NIDB's(NIH Intramural Database) servers from CIT's Data Center to CIT's Customer Service Area, NIDB services will be unavailable on Tuesday, July 25th from 8 am - until 3:00 pm. The work is expected to be completed by 3:00 pm. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The planned move for the NIDB production servers has been completed and all servers are back online ahead of schedule.
7/24/2006 8:47:26 AM - Video conferencing problems.
Description:   Event Details: Please note that the NIH MXM Gatekeeper is experiencing connectivity problems and will have to be rebooted The Video application engineers are working to resolve this problem. For now, there is no ETR established but we expect to have this problem resolved as soon as possible. Impact: Due to loss of connectivity with the NIH MXM Gatekeeper the following services will be disrupted: Calling from a IP video conferencing unit to an ISDN number. Calling a IP video conferencing unit using an E.164, alias, number and not the actual IP address. Hot News will updated as information becomes available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Connectivity to the NIH MXM Gatekeeper has been restored.
7/21/2006 11:08:02 AM - ADB login error.
Description:   ADB is currently not accessible. You may receive the following error when attempting to login to ADB: DFS065 00:00:00 TRAN/LTERM STOPPED. ADB Tier III is currently working to resolve the issue. No ETR at this time.
Resolution:  The ADB login issue has been resolved.
7/20/2006 4:34:59 PM - CIT Remedy Server Maintenance - Saturday July 22, 10:00 AM
Description:   Maintenance on the CIT Remedy Server will be performed on Saturday July 22nd between the hours of 10:00 AM and 5:00 PM. During this time Remedy's inbound email processing and outbound email notifications will be disabled. There may also be short periods of system unavailability during this time, though it is anticipated that the other server functionality will remain largely unaffected for most of this period.
Resolution:  The Remedy Maintenance was not successful. The system has been returned to service and notifications are now being sent and received by the system. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or email helpdesk@nih.gov if you have any problems.
7/20/2006 3:48:59 PM - ITAS Mail Issue
Description:   ITAS is experiencing problems with e-mail notification due to the recent migration to the new mail server. We are aware of the issue and hope to have it fixed by this evening without affecting users during the normal business hours. Please complete timecards prior to Pay Period 15 close by Friday, July 21, 2006. Timely salary payments cannot be guaranteed for any timecard not approved during the current pay period. Thank you for your patience in this matter.
Resolution:  Problems with ITAS e-mail notification have been been resolved. No service interruption was necessary. ITAS e-mail has been requeued and will be sent out starting tonight. We anticipate delivery of all the mail will be complete by tomorrow morning. Please complete timecards prior to Pay Period 15 close by Friday, July 21, 2006. Timely salary payments cannot be guaranteed for any timecard not approved during the current pay period. Thank you for your patience in this matter. CIT/DECA ITAS Technical Support Team. Very Best Regards, CIT/DECA ITAS Technical Support Team
7/19/2006 4:34:43 PM - Helix Systems Downtime
Description:   Helix (helix.nih.gov) will be unavailable to users between 7 am and 8.30 am on Thursday, July 20, 2006. This will also affect access to email on Helix via clients, Squirrelmail or pine. Nimbus, Doublehelix and the Biowulf cluster will be unaffected.
Resolution:  Downtime is complete and Helix is now available to users.
7/19/2006 11:50:39 AM - NIHnet: HRSA Regional Office Network Issue
Description:   Description: Users in HRSA regional offices are reporting problems getting to NIH Resources. CIT Engineers are investigating the issue. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Users in the HRSA regional offices are now able to access NIH Resources.
7/19/2006 11:16:48 AM - Building 12 NIH Data Center Network Scheduled Maintenance
Description:   Date: Saturday, July 29, 2006 Time: 6:00am – 4:00pm Description: On Saturday, July 29, 2006 at 6:00am – 4:00pm ET NIHnet engineers will perform a Scheduled Maintenance activity on the new Building 12 NIH Data Center network routers in support of efforts to implement the new Building 12 NIH Data Center Network Design. Impact: Please note this Building 12 NIH Data Center network maintenance has been assessed with a Service Affecting (SA) service impact. Specifically, this maintenance activity will cause a momentary data network connectivity disruption to each of the servers located in the Building 12 NIH Data Center as the 11 data network access switches are sequentially migrated to new ingress / egress routers. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  8:30 pm. All NIH Data Center services have been restored and verified.
7/18/2006 4:48:15 PM - Emergency maintenance to NIH CES services
Description:   Date: 07-19-06 Time: 9:00 pm to 10:00 pm Description: There will be a brief outage at 9:00 PM Wednesday evening to allow for a system reset of the NIH Central Email Services. This is to allow for the application of an MS Exchange security patch. Outage will be ~10 minutes per server. The work is expected to be completed by 10:00 pm. Impact: All customers will lose email connectivity and service for this period of time. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Work was completed at 9:45pm. All systems were returned to operational status.
7/18/2006 3:36:36 PM - CIT Remedy System Outage to Perform Maintenance
Description:   The Support Technologies Group will be performing Maintenance on the CIT Remedy System starting at 9:00 PM on Wednesday July 19, 2006. The system will be taken offline at 9:00 PM and is expected to return to service at 10:00 PM. Users will be unable to submit Service Requests and register for CIT courses during this outage. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The maintenance on the CIT Remedy System has been completed by the STG staff. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
7/14/2006 4:12:35 PM - The PH System is currently being repopulated.
Description:   The PH system at http://directory.nih.gov is currently being repopulated. Some users may have trouble updating their records and others whose records are not currently displaying will display when this process is completed.
Resolution:  The PH system update has been completed and users should be able to update their records at this time.
7/14/2006 2:53:49 PM - The NIH Application Manager (NAppMan) notification.
Description:   Major changes are occurring to the monitoring of the NIH Portal application on 7/17/06. NAppMan will show NIH Portal with the following status: "System is experiencing scheduled down time" between 8:00 AM and 12:00 PM. This allows monitoring changes to occur without NAppMan complaining to technical teams. During this time, NAppMan will not be showing the actual status of the NIH Portal application.
Resolution:  The maintenance window is complete. NAppMan is, once again, reflecting the proper status for NIH Portal.
7/14/2006 2:04:48 PM - NHLBI Building 10 Network Connectivity Issues
Description:   NHLBI Building 10 is experiencing network connectivity issues. NHLBI network support and DNST are currently investigating the issue
Resolution:  The network connectivity issues in Building 10 have been resolved.
7/14/2006 11:19:24 AM - Fernwood Andover door access issue.
Description:   There is an issue with the Andover door access card readers in the Fernwood building that may be network related. DNST-NEB is working on this issue. There is no ETR.
Resolution:  The issue with the Andover door access card readers in the Fernwood building has been resolved.
7/12/2006 11:35:59 PM - NIH Hompage search engine inaccessable
Description:   NIH Search Engine network issue - Unable to perform searches and subsearches on the NIH Hompage. Technicians are currently working on this issue and have no ETR at this time. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  NIH Search Engine and sub-searches are now reachable on the network and able to perform searches.
7/10/2006 10:44:16 AM - CIT Remedy System Outage to Perform Emergency Maintenance
Description:   The Support Technologies Group will be performing Emergency Maintenance on the CIT Remedy System starting at 11:00 AM on Monday July 10, 2006. The system will be taken offline at 11:00 AM and will be back online as soon as possible. Users will be unable to submit Service Requests and register for CIT courses during this outage. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Remedy system has been returned to service.
7/7/2006 4:05:19 PM - NIH AD, mail enabled account creation delays
Description:   IT Pros are reporting delays in the mail enabling portion of the account creation process. EMIB admins are currently researching this problem.
Resolution:  The new user accounts in question were updated (stamped by the Recipient Update Service) as of early Saturday morning. The associated mailboxes are ready for access.
7/7/2006 12:14:00 PM - IM Service Delay
Description:   Some users may experience a delay in Instant Messge communications. This issue may effect any of the IM clients. IM Admins and SQL DBA's are aware of the issue and are working on a fix.
Resolution:  IM communication flow has been restored to normal rates.
7/6/2006 3:03:30 PM - Building 12 NIH Data Center Network Scheduled Maintenance
Description:   Date: Thursday, July 13, 2006 Time: 7:00pm - 12:00am Description: On Thursday, July 13, 2006 at 7:00pm - 12::00am ET NIHnet engineers will perform a Scheduled Maintenance activity on the new Building 12 NIH Data Center network routers in support of efforts to implement the new Building 12 NIH Data Center Network Design. Impact: Please note this Building 12 NIH Data Center network maintenance has been assessed with a Service Affecting (SA) service impact. Specifically, this maintenance activity will cause a momentary data network connectivity disruption to each of the servers located in the Building 12 NIH Data Center as the 11 data network access switches are sequentially migrated to new ingress / egress routers. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions
Resolution:  The Thursday, July 13, 2006 Building 12 NIH Data Center Network Scheduled Maintenance Activity has been cancelled. Please note a new maintenance activity date and time will be set in the near future and an advanced HotNews advanced notification will be posted prior to the new maintenance window date. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
7/6/2006 10:10:19 AM - OWA Issues
Description:   There is a 60 to 90 second delay issue when users attempt to access OWA. EMIB is currently working on the issue. There is no ETR.
Resolution:  EMIB has resolved the delay issue.
7/5/2006 1:55:04 PM - NIH Portal Portlets may be unavailable
Description:   Between 6:15 PM and 6:30 PM this evening, 07/05/2006, the portal banner and any other Publisher-driven portlets may be unavailable due to a memory upgrade on some portal servers.
Resolution:  The memory upgrade was completed successfully.
7/5/2006 11:47:05 AM - ITAS Currently Unavailable
Description:   ITAS is currently unavailable. ITAS engineers are working towards a resolution. There is no ETR at this time.
Resolution:  The issues concerning NIH Login and ITAS have been resolved.
7/4/2006 4:17:32 PM - ITAS monthly database maintenance tonight 9-10PM
Description:   The ITAS Technical Team will be conducting routine monthly maintenance tonight on the ITAS database from 9-10pm. Users can expect slow performance, and potential short outages during this period of time. Thank you for your patience. CIT/DECA ITAS Technical Support Team.
Resolution:  Maintenance complted normally.
6/29/2006 3:01:24 PM - Qwest Data Network Upgrade Between Bldg12 - Sterling, VA Data Ctr
Description:   Date: Wednesday, July 5 Thursday July 6, 2006 Time: 11:00pm to 6:00am Description: On Wednesday, July 5, 2006 to Thursday, July 6, 2006 between 11:00am – 6:00am ET, Qwest engineers will perform another Scheduled Maintenance on the data network link between the Building 12 <---> Sterling, VA NIH Data Centers in support of efforts to upgrade the existing telecommunications facility between these two hosting locations. Impact: Please note this upgrade to the Qwest data network link will cause multiple momentary disruptions to traffic between Campus NIHnet <--> Sterling, VA, each of which is expected to last less than 50 milliseconds in duration. The expected loss of connectivity and performance degradation between Campus NIHnet <--> Sterling, VA NIHnet should be minimal while Qwest engineers activate and migrate to new telecommunication facilities. This maintenance will have no impact to Internet and Abilene (Internet 2) connections to the Sterling, VA facility. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Qwest Data Network Connection Upgrade between Building 12 <--> Sterling, VA. Data Centers was completed successfully.
6/29/2006 8:02:26 AM - Ticket error on CIT Remedy system
Description:   The CIT Remedy system is currently experiencing an issue which is causing an error message when users attempt to open certain tickets. STG Staff is currently working to address the issue. The server is up and functional, this is more of an annoyance. No ETA at this time
Resolution:  The CIT Remedy server and the Remedy database server have both been restarted and the problem has been corrected. The recent Hot News notification containing digits rather than intelligible information was related to this issue - therefore that problem has been corrected as well. We apologize for the inconvenience and confusion and appreciate your patience while we worked to correct this issue.
6/28/2006 12:00:21 PM - Building 12 NIH Data Center Network Scheduled Maintenance
Description:   On Monday, July 3, 2006 at 7:00pm – 9:00pm EST NIHnet engineers will perform a Scheduled Maintenance activity on the Building 12 NIH Data Center network ingress routers in support of efforts to implement the new Building 12 NIH Data Center Network Design. Impact: Please note this Building 12 NIH Data Center network maintenance has been assessed with a Possible Service Affecting – High Probability (PSA-High) service impact. Specifically, in the event the primary and secondary Building 12 NIH Data Center Network ingress / egress routers experience a Spanning-Tree recalculation during the maintenance window, there is a high probability connectivity to / from the Building 12 NIH Data Center Network will be disrupted for approximately 60 seconds while the layer 2 network infrastructure re-establishes connectivity for all connected switches within the Building 12 NIH Data Center Network. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Maintenance completed successfully with minor modifications to the original plan.
6/27/2006 4:35:53 PM - NIH ADM Server Services to be restarted.
Description:   NIHADM and NIHADM2 ActiveRoles Server Services will be restarted at 5pm. This will cause all current connections to be terminated. Users should be able to reconnect after a 10 min period.
Resolution:  Services were reconfigured
6/27/2006 8:58:13 AM - The NIH Application Manager (NAppMan) is experiencing a problem.
Description:   The connectivity status information on the NAppMan dashboard is currently unavailable. Technicians are working with a firewall to resolve the problem. No ETA was provided.
Resolution:  The firewall issues have been resolved. The NAppMan dashboard is once again reliable.
6/26/2006 3:51:48 PM - CIT Remedy System Outage to Perform Emergency Maintenance
Description:   The Support Technologies Group will be performing Emergency Maintenance on the CIT Remedy System starting at 8:00 PM on Monday June 26, 2006. The system will be taken offline at 8:00 PM and is expected to return to service at 8:30 PM. Users will be unable to submit Service Requests and register for CIT courses during this outage. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Remedy system has been returned to service. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or email helpdesk@nih.gov if you have any problems.
6/26/2006 3:02:38 PM - ITAS Maintenance
Description:   Dear ITAS User, ITAS is scheduled for maintenance this Wednesday evening, June 28, 2006 from 6:00PM - 6:30PM. During this time ITAS will not be available. We apologize for any inconvenience this may cause you. If you have any questions, please contact Helpdesk at 301-594-6248 or via email at nihhelpdesk@mail.nih.gov <<<<>>>> As always, we appreciate your patience. Very Best Regards, ITAS Technical Team, DECA/CIT
Resolution:  Dear ITAS User, ITAS maintenance scheduled for Wednesday evening, June 28, 2006 has been completed. If you have any questions, please contact the TASC Helpdesk at 301-594-6248 or via email at nihhelpdesk@mail.nih.gov <<<<>>>> As always, we appreciate your patience. Very Best Regards, ITAS Technical Team, DECA/CIT
6/26/2006 2:28:19 PM - NIHnet<-->Internet 2 Fiber Relocation Demand Maintenance
Description:   Date: Tuesday, June 27, 2006 Time: 7:00am - 7:30am Description: On Tuesday, June 27, 2006 at 7:00am - 7:30am EST NIHnet engineers will perform a Demand Maintenance activity on the Building 12 fiber path between NIHnet <--> Internet(2) to relocate a fiber optic cable to a new location. Impact: Please note this Internet(2) maintenance activity is not expected to disrupt service between NIHnet <--> Internet(2) because traffic destine to MAX via Building 12 will automatically re-route to MREN via the Qwest collocation facility in Sterling, VA. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  NIHnet<-->Internet 2 Fiber Relocation Demand Maintenance was successfully completed with no complications.
6/22/2006 6:24:10 PM - Some features in the NIH Portal are temporarily unavailable
Description:   Some features in the NIH Portal are temporarily unavailable. The Portal administrators are currently working to correct this. If you need access to enterprise applications, please use these direct links: ITAS NBS Travel (Production) NED Admin HRIBS nVision
Resolution:  All of the features in the NIH Portal are now available.
6/22/2006 12:27:55 PM - Network Connection Issue for OD in the E. Jefferson Ln Building
Description:   There is a network connectitvity problem affecting the East Jefferson Lane Building (OD) on the third and fourth floors. The DNST Network Operations Team is aware of the issue and is onsite working to resolve the issue. There is currently no ETR. Updates to follow.
Resolution:  Per the OD OIT Support Team the connection issue is not related to any network disruptions. The OIT Support Team is currently working to resolve the issue.
6/22/2006 8:22:57 AM - NIHnet <--> Internet DMZ Optimization Maintenance Activity
Description:   Date: Friday, June 30, 2006 Time: 7:00pm - 11:00pm Description: On Friday, June 30, 2006 between 7:00pm – 11:00pm EST NIHnet engineers will perform a Scheduled Maintenance on the NIHnet Internet DMZ in support of final efforts to optimize the network paths between NIHnet <--> Internet. Impact: This Internet DMZ maintenance activity will cause multiple momentary disruptions to traffic between NIHnet <--> Internet during the first two hours of the four-hour maintenance window while NIHnet <--> Internet traffic is migrated between the Building 12 and Fernwood Internet DMZ paths. Please note the total number of expected momentary disruptions to traffic flow between NIHnet <--> Internet during the maintenance window should not exceed four (4) disruptions while the NIHnet Internet perimeter is optimized and tested. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  NIHnet <--> Internet DMZ Optimization Maintenance Activity completed successfully without complications within the maintenance window.
6/20/2006 12:54:35 PM - CES experiencing inbound SMTP delay
Description:   CES is experiencing a very high volume of inbound SMTP traffic from the Internet causing 10-15 minute delays in e-mail delivery. CES Engineers are investigating.
Resolution:  The issue has been resolved
6/19/2006 12:11:53 PM - Qwest Data Network Link Upgrade Between Building 12 and Sterling
Description:   Qwest Data Network Link Upgrade Between Building 12 <--> Sterling, VA CIT Data Centers Date: Saturday, June 24, 2006 Time: 12:00am – 4:00am Description: On Saturday, June 24, 2006 between 12:00am – 4:00am EST Qwest engineers will perform a Scheduled Maintenance on the data network link between the Building 12 <---> Sterling, VA CIT Data Centers in support of efforts to upgrade the existing telecommunications facility between these two CIT hosting locations. Impact: Please note this Qwest data network link upgrade will cause a momentary disruption to traffic between Campus NIHnet <--> Sterling, VA NIHnet which is expected to last less than 50 milliseconds in duration. The expected loss of connectivity and performance degradation between Campus NIHnet <--> Sterling, VA NIHnet should be minimal while Qwest engineers activate and migrate to new telecommunication facilities.
Resolution:  The maintenance was successful with no detectable impact to the network.
6/19/2006 10:04:11 AM - Problems with this week's Porpoise search results
Description:   There was a problem with this week's Porpoise literature search results that were emailed to customers overnight. Symptoms include: truncated or empty reports and the link to "Report Summary" fails, We are currently trying to identify the cause of the problem; when the problem is resolved we will send corrected results to all customers.
Resolution:  The problem has been resolved. Corected profiles will be sent to customers by this evening.
6/16/2006 12:20:29 PM - The NIH Application Manager (NAppMan) is experiencing a problem.
Description:   The NAppMan system must be restarted immediately. When the dashboard is once again visable it will be reliable.
Resolution:  The vendor for Formula has provided the necessary tuning to accommodate the increased activity on NAppMan. NAppMan is up and stable. Closing ticket.
6/16/2006 12:12:52 PM - Network Outage in Children's Inn
Description:   There is a network outage in Children’s Inn (Building 62) that is affecting network and internet connectivity at the Children’s Inn. The DNST Network Operations Team is aware of the issue and onsite working to resolve the issue. Updates to come.
Resolution:  All network connectivity and telephone service has been restored to the Children's Inn and the entire Building 15 complex that was impacted by the cable cut.
6/14/2006 3:45:32 PM - nih.gov Dogfood domain patching
Description:   Dogfood will have intermitent outages while Microsoft security patches are being applied
Resolution:  upgraded successfully
6/14/2006 3:20:25 PM - NIHnet Internet DMZ Firewall Operating System Upgrade Maintenance
Description:   Date: Wednesday, June 21, 2006 Time: 7:00pm - 10:00pm Description: On Wednesday, June 21, 2006 between 7:00pm – 10:00pm EST NIHnet engineers will perform a Scheduled Maintenance on the NIHnet Internet DMZ in support of efforts to upgrade the existing Fernwood Internet DMZ Firewall Operating System. Impact: Please note this Internet DMZ Firewall maintenance activity will cause a momentary disruption to traffic between NIHnet <--> Internet during the first 60 minutes of the three-hour maintenance window while the Fernwood Firewall OS is upgraded and NIHnet <--> Internet traffic is migrated to the Fernwood path. The expected loss of connectivity and performance degradation between NIHnet <--> Internet should be minimal while the new OS is installed and Internet traffic is migrated to the Fernwood path. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The NIHnet Internet DMZ Firewall Operating System Upgrade Maintenance was successful.
6/13/2006 8:14:45 PM - Remedy Server Issues, Customer Support Web Issues
Description:   The CIT Remedy server is currently unresponsive, as are some of the CIT Customer Support websites, including the IT Help Desk site. We are still working to determine the nature and the cause of the issues. There is no ETA at this time. Thanks for your patience while we work to resolve these issues.
Resolution:  The CIT Remedy Server has been returned to service and the CIT Customer Support web presence is back online. We appreciate your patience while we worked to resolve this issue.
6/13/2006 12:44:06 PM - Resolved-nVision System Maintenance
Description:   nVision will be down for necessary system maintenance today from 5 - 6:30 PM.
Resolution:  Maintenance completed.
6/12/2006 1:41:10 PM - Office of Research Services webpage has broken links
Description:   The access to some links on http://www.ors.od.nih.gov/ are not available at this time. ITB is working diligently to resolving the situation. No ETR provided.
Resolution:  The issue with the broken links from the Office of Research Services webpage has been resolved.
6/9/2006 6:53:25 PM - Power Outage Bldg 82 and Cedarlane
Description:   Please be advised that NIH is currently experiencing a partial power outage in Buildings 82 and Cedar lane, causing a loss of network connectivity for users in that building. There is no estimated time of repair at this time. Please contact the NIH Help Desk at (301) 496-HELP or helpdesk@nih.gov if you have any questions.
Resolution:  Power restored by Pepco technicians
6/9/2006 12:50:22 PM - The NIH Application Manager (NAppMan) is experiencing a problem.
Description:   Due to increased usage of NAppMan, adjustments to Formula are necessary. Formula will be stopped and restarted to take advantage of the adjustments.
Resolution:  The vendor for Formula made some additional memory adjustments. Formula is up and running. The dashboard is once again reliable.
6/9/2006 8:35:55 AM - CIT Remedy System Outage to Perform Emergency Maintenance
Description:   The Support Technologies Group will be performing Emergency Maintenance on the CIT Remedy System starting at 8:00 PM on Monday June 12, 2006. The system will be taken offline at 8:00 PM and is expected to return to service at 10:00 PM. Users will be unable to submit Service Requests and register for CIT courses during this outage. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Remedy Services have been patched and the system returned to service. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or email helpdesk@nih.gov if you have any problems.
6/8/2006 8:01:12 AM - NIMBUS Emergency Maintenance
Description:   The Helix group will be performing emergency maintenance on NIMBUS this morning and it will not be available. ETR is 10:30AM. Updates will be posted as they become available.
Resolution:  The NIMBUS emergency maintenance has been completed successfully.
6/6/2006 11:30:25 AM - Time correction: ITAS Maintenance tonight 9-10PM
Description:   The ITAS Technical Team will be conducting routine database maintenance tonight from 9-10PM. During this time users may not be able to access ITAS. Thanks for your patience! CIT/DECA ITAS Technical Team.
Resolution:  Maintenance was complted normally.
6/6/2006 10:14:56 AM - Latest Nuisance SPAM
Description:   Many ITPro's and staff are reporting receipt of a spoofed email message with nothing more than a single numeric string on the Subject line as well as in the message body. NIH CES Anti-SPAM engineers are aware of this latest SPAM and are working to determine if there is a way to filter it. But it should be understood now, that this SPAM is from a spoofed address, has a random subject and message body of numbers only and is from random internet hosts. This makes blocking it is difficult if not impossible without introducing serious potential for blocking or tagging legitimate email. This should be considered "nuisance SPAM" for now and can just be deleted. Here is an example: _______________________________ From: Hobson, Kevin (NIH/CIT) [E] Sent: Tuesday, June 06, 2006 7:59 AM To: Hobson, Kevin (NIH/CIT) [E] Subject: 586876 969
Resolution:  Resolution will be ongoing
6/5/2006 11:38:13 AM - AOL IM not accessible.
Description:   AOL is experiencing service issues with AOL IM. Users cannot log in and cannot send messages. This behavior is not system wide. AOL is aware of this issue and is working to resolve the issue.
Resolution:  The AOL IM network problems experienced earlier has been resolved.
6/5/2006 9:19:32 AM - NIHCESMLBX5 mailbox service down
Description:   The Central Email Service Exchange Server NIHCESMLBX5 serving NIAID is currentlly having a problem serving mailbox requests. CES Engineers are currently working on the problem.
Resolution:  Server was rrestarted to allow clean shutdown and startup of the mailbox Store services.
6/3/2006 11:55:28 PM - NIHnet Outage: CRC Network Issues
Description:   We are currently experiencing problems with a switch servicing the Clinical Research Center. NIHnet engineers have been dispatched and are working diligently to resolve this issue. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  NIHnet engineers have resolved the switch problems and network connectivity has been restored.
5/31/2006 4:14:40 PM - NIHnet Internet DMZ Firewall Operating System Upgrade Maintenance
Description:   On Wednesday, June 7, 2006 between 7:00pm – 10:00pm EST NIHnet engineers will perform a Scheduled Maintenance on the NIHnet Internet DMZ in support of efforts to upgrade the existing Building 12 Internet DMZ Firewall Operating System. Please note this Internet DMZ Firewall maintenance activity will cause a momentary disruption to traffic between NIHnet <--> Internet during the first 60 minutes of the three-hour maintenance window while the Building 12 Firewall OS is upgraded and NIHnet <--> Internet traffic is migrated to the Building 12 path. The expected loss of connectivity and performance degradation between NIHnet <--> Internet should be minimal while the new OS is installed and Internet traffic is migrated to the Building 12 path. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The NIHnet Internet DMZ Firewall Operating System Upgrade Maintenance was successful.
5/30/2006 11:07:11 AM - Nimbus emergency maintenance
Description:   nimbus.nih.gov will undergo emergency maintenance starting at approximately 1 pm, 30 May, for up to 4 hours.
Resolution:  Nimbus was returned to service at 5.30 pm
5/30/2006 10:45:57 AM - NIH Portal Upgrade
Description:   NIH Portal will be upgraded to version 6.0 on Friday evening, June 2nd at 5:00p.m. During the upgrade work, my pages, communities and collaboration functionality of the portal will be unavailable. Users that log in to the Portal during this time will be directed to the Portal temporary page which will respond with links to the most popular applications within the portal. At this time there is no ETA for the upgrade.
Resolution:  The NIH Portal Upgrade was successful.
5/30/2006 9:09:55 AM - NAppMan not reporting correctly
Description:   Description: NAppMan services will be restarted to correct the reporting issue. Currently there is no ETR. Hot News will updated as information becomes available. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Service has been returned to the NIH Application Manager (NAppMan). The dashboard is once again reliable.
5/25/2006 5:41:36 PM - ODHSRVO1 Server Reboot
Description:   ODHSRVO1 server will be rebooted shortly to resolve connectivity issues. Some OD customers may experience inaccessibility to their home drives. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Server reboot was successful.
5/23/2006 5:30:12 PM - Telephone Cable Maintenance: Phase I
Description:   Dates: May 30, 2006 - July 15, 2006 Description: Over the next year, CIT will be working in phases with our telephone and cable providers to upgrade all the cables that provide telephone service to the Bethesda Campus of NIH. Impact: During this maintenance window, telephone service in the affected buildings may be unavailable for a maximum of 5 minutes as the cable is replaced. Once the cable is upgraded, telephone service will be immediately restored. The first phase will briefly impact telephone service for users in the following buildings: Building 15K Building 6 Building 31 Additional Information: Although not expected, should you experience a telephone outage greater than 5 minutes, please call 611 or contact the NIH Help Desk. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or email helpdesk@nih.gov if you have any additional questions.
Resolution:  Phase I of the telephone cable maintenance has been completed.
5/23/2006 9:52:05 AM - CIT NCCS Sterling Firewall Impairment
Description:   Please note the firewalls in the CIT Sterling, VA data center are experiencing excessive performance degradation due to unforeseen problems encountered as a result of an operating system upgrade performed yesterday evening (5/22/2006). NIHnet security engineers are working diligently to find a solution to eliminate this problem. For now, there is no ETR established but we expect to have this problem resolved as soon as possible.
Resolution:  NIHnet security engineers restored service to the firewalls in the CIT Sterling, VA data center at 3:30pm. We apologize for any inconvenience this may have caused and thank you for your patience. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
5/20/2006 3:44:17 PM - Phone Outage In Building 10
Description:   During routine phone maintenance a power surge occurred and has disabled up to 200 phone lines in building 10. Verizon has been notified and is working to resolve the issue. At this time there is no estimated time of resolution.
Resolution:  All phone services to building 10 have been restored. We apologize for any inconvenience this may have caused and thank you for your patience. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
5/19/2006 1:24:14 PM - Network Hardware Disruption on 3rd floor of Fernwood
Description:   Description: Description: Some users on the 3rd floor of Fernwood may experience network connectivity issues due to a network hardware disruption.. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Network connectivity has been restored to full capacity on the 3rd floor in the Fernwood building.
5/19/2006 12:45:58 PM - NIH Central Ticketing System (Remedy) experiencing issues
Description:   Remedy is experiencing issues when trying to save tickets. The Remedy support group is aware of the issue and all functionality should be returned by 12:55PM 5/19.
Resolution:  The service provider has identified & resolved the problem. All Remedy customers should be able to save changes to tickets now. Please contact the NIH Help Desk at 301-496-4357 if further service problems are discovered.
5/17/2006 8:57:21 AM - CIT Remedy System Outage to Perform Major Upgrade
Description:   Description: The Support Technologies Group will be performing a Major Upgrade on the CIT Remedy System starting at 10:00 PM on Saturday May 20, 2006. Personnel from the Support Technologies Group will be performing backups starting at 9:00 PM which may cause some degradation in the Remedy service. The system will be taken offline at 10:00 PM and is expected to return to service by Sunday May 21, 2006 at 6:00 AM. Users will be unable to submit Service Requests and register for CIT courses during this outage. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  STG has completed the upgrade of the Remedy System. Please contact the NIH Helpdesk if you have any problems.
5/16/2006 10:24:30 AM - Unscheduled maintenance - Central Email Service
Description:   There will be a brief outage at 9:00 PM this evening to allow for a system reset of the NIH CES services. This is to allow for the application of an MS Exchange security patch. Outage will be 10 minutes.
Resolution:  This work was completed.
5/15/2006 7:22:20 AM - Democracy II, 3rd Floor Network Connectivity Issue
Description:   Description: Users in Democracy II, 3rd Floor are reporting a lack of network connectivity. NIHnet Engineers are working diligently to restore service. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  NIHnet Engineers have resolved the network connectivity issues, they will continue to monitor the network to ensure network stability. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
5/12/2006 4:10:13 PM - Public Folder Content Replication
Description:   There may be problems with some Public Folders due to replication conflicts. CES Engineers are currently working on the problem.
Resolution:  Public Folder Synchronization has updated Public Folder Content. Not all Public Folders were affected with this synchronization problem. Customers are encouraged to verify the contents of Public Folders and any applications that access them.
5/11/2006 2:41:48 PM - Internal CIT support web service issues
Description:   Currently there are problems accessing some CIT support web services. This includes the knowledge base, online ticket submissions, and the remote access website. CIT engineers are working to resolve the issue.
Resolution:  The CIT internal sites are back up and running now.
5/10/2006 11:05:30 AM - IXOS Email Vaulting Experiencing Issues
Description:   Description: IXOS Email Vaulting system is experiencing issues. Impact: Customers using IXOS as an email archiving tool will be unable to access their archives. CES administrators are working diligently to restore services Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  CES administrators have resolved the issue involving IXOS Email Vaulting. We apologize for any inconvenience this may have caused and thank you for your patience. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
5/9/2006 8:19:44 AM - ADM experiencing problems
Description:   ADM is currently experiencing problems after cutover last night. EMIB is aware of the problem and currently looking at it. IT community should use nihadm.nih.gov until the issue is resolved. MMC is ok to use
Resolution:  ADM problem has been resolved. IT community should resume to use the link https://ADM.NIH.GOV
5/8/2006 10:15:26 AM - Connectivity column not working in NAppMan
Description:   Maintenance was performed on the hardware that collects information from strategically located probes throughout our network. Although the database is up and running properly, NAppMan is unable to get to the information. We are actively working on the problem. No ETA.
Resolution:  Communication issues between the database and NAppMan have been resolved.
5/7/2006 9:18:43 AM - NIH Help Desk Unable to Receive Calls via Phone Channel
Description:   The NIH Help Desk is currently experiencing difficulties with its' phone channel due to a problem with a bad switch. Technicians are currently working to resolve this issue and will provide updates as additional information becomes available. Customers are unable to contact the NIH Help Desk via the phone channel. Customers are advised to use other channels such as, the support page at http://ithelpdesk.nih.gov or send email to ithelpdesk@mail.nih.gov. for support related issues.
Resolution:  Phone service has been restored to the NIH Help Desk. Thanks for your patience while we worked to resolve this issue.
5/4/2006 4:14:05 PM - nimbus.nih.gov down
Description:   nimbus.nih.gov is currently unavailable due to hardware issues. We are working to resolve the problem.
Resolution:  UPDATE: The nimbus.nih.gov web site issue has been resolved and customers now may again access the site.
5/4/2006 8:08:01 AM - EHRP Emergency Outage
Description:   The EHRP system will be unavailable due to emergency system maintenance from 7:30 AM through 1:00 PM on Thursday, May 4th, 2006. The downtime is required to replace and restore data on one of the system drives in the EHRP production environment. We apologize for the short notice and inconvenience to processing this may cause during this emergency outage.
Resolution:  EHRP System has been restored to full capacity.
5/3/2006 2:56:20 PM - Building 12 Wireless Network DMZ Firewall Upgrade Maintenance
Description:   Date: Sunday, May 14, 2006 Time: 1:00am - 3:00am Description: On Sunday, May 14, 2006 between 1:00am - 3:00am EST NIHnet engineers will perform a scheduled Network DMZ Firewall Upgrade Maintenance. This Maintenance activity is necessary in support of final efforts to optimize the NIHnet Wireless Network DMZ. Impact: Please note VPN Wireless Network users that are connected via the Building 12 WVPN path will lose connectivity during the first 30 minutes of the maintenance window. It will be necessary to reconnect to the Wireless Network via the Building 45 path until the scheduled maintenance is completed at 3:00am. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Maintenance has been completed.
5/3/2006 9:27:55 AM - NIHnet <--> Emergency Internet DMZ Optimization Maintenance
Description:   Date: Thursday, May 4, 2006 Time: 7:00pm - 8:00pm On Thursday, May 4, 2006 between 7:00pm – 8:00pm EST NIHnet engineers will perform a Scheduled Emergency Maintenance on the NIHnet Internet DMZ to optimize the network paths between NIHnet <--> Internet. Impact: Please note this Internet DMZ maintenance activity may cause a momentary disruption to NIHnet <--> Internet traffic. The expected loss of connectivity between NIHnet <--> Internet should be transparent to NIHnet users. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Maintenance completed.
5/2/2006 4:06:15 PM - ITAS Database maintenance tonight
Description:   The ITAS Technical Team will be conducting database maintenance from 9-10pm tonight. During this time, the ITAS applicaiton may become sluggish or inaccessible. We appreciate your patience. Thanks, The CIT/DECA ITAS Technical Team.
Resolution:  The ITAS maintenance was completed on schedule.
5/2/2006 8:42:19 AM - NIHCESPUB3 under stress
Description:   nihcespub3 is having issues; it's being addressed by CES engineers
Resolution:  service was brought back online
5/1/2006 10:56:58 AM - Instant Messenger Service disruption
Description:   NIH IM users may experience a temporary disruption in IM service. EMIB IM engineers are aware of the issue and are currently resolving same.
Resolution:  Communications have been re-established on IM servers.
4/27/2006 10:54:56 AM - Emergency Maintenance on CIT Support Website
Description:   Date: 4/27/2006 Time: 11:30am Description: STG (Support Technologies Group) will be restarting the web services that support ITHELPDESK.NIH.GOV Impact: Customers may experience a brief interruption as services restart. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  STG (Support Technologies Group) has completed the Emergency Maintenance. We thank you for your patience and apologize for any inconvenience caused by this maintenance.
4/26/2006 11:58:16 AM - Possible Exchange connectivity problem
Description:   Possible DDNS issues are affecting access to Email and internal web pages. DDNS engineers are aware of the issue and are looking into a resolution. No ETA at this time
Resolution:  The DDNS issue has been resolved. All connectivity to email, internal sites and services have been restored. Customers unable to connect to email or internal sites may require a reboot. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
4/25/2006 3:30:12 PM - NIH Login Issues
Description:   Description: NIH Login is having technical difficulties. Impact: Some users may not be able access websites protected by NIH Login. We apologize for any inconvenience and appreciate your patience. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The NIH Login issue has been resolved. We apologize for any inconvenience this may have caused and thank you for your patience. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
4/25/2006 2:58:11 PM - ASR system is unavilable
Description:   The ASR System is unavailable at this time. Systems team is looking into the problem. This problem also affects HRIBS. At this time there isn't an estimate on when the system will be available.
Resolution:  The ASR System is now available
4/24/2006 12:46:07 PM - NIHnet Wireless Network DMZ Optimization Maintenance
Description:   Description: On Wednesday, April 26, 2006 at 5:30pm - 6:30pm EST NIHnet engineers will perform a scheduled maintenance on the NIHnet Wireless Network DMZ as part of the final efforts to optimize the redundant paths within the Wireless perimeter network. Impact: Please note this Wireless Network maintenance activity will cause a momentary interruption in Wireless connectivity to Wireless Guest Users only. This expected loss of connectivity will be limited only to Wireless Guest users while Wireless DMZ path failover testing is conducted during the second half of the 5:30pm – 6:30pm maintenance window. Normal Wireless Network connectivity for all non-guest users will not be impacted during this maintenance activity. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Scheduled Maintenance to Optimize the Redundant Paths within the Wireless Perimeter Network was successfully completed without complications on Wednesday, April 26, 2006 between 5:30pm – 6:30 pm.
4/22/2006 8:13:01 AM - CES service
Description:   There will be a reboot of each CES mailbox server beginning over the next hour. This is to remediate an issue that occurred during last night's maintenance window.
Resolution:  The issue was corrected.
4/20/2006 2:55:10 PM - NIHCESMLBX2 E-mail Server Issue
Description:   Email service is not currently available to customers on the NIHCESMLBX2 server. CIT\CES engineers are actively troubleshooting the problem. This server provides e-mail service to OD, NBS, and DEAS customers. At this time there is no estimated time of repair. We apologize for any inconvenience.
Resolution:  CIT\CES engineers have resolved the issue with the NIHCESMLBX2 E-mail Server. Thank you for your patience.
4/18/2006 12:25:11 PM - AOL Users Cannot Log In or Send Messages
Description:   AOL is experiencing service issues with AOL IM. Users cannot log in and cannot send messages. This behavior is not system wide.
Resolution:  AOL has resolved the outages that occurred previously today. The AOL IM networking is in a nominal state currently.
4/13/2006 2:10:14 PM - Email Exchange Issue
Description:   The IC that may have been affected is OD. EMIB Exchange Engineers recognized and resolved email issue that may minimally interrupted service to users on Nihcesmlbx2.
Resolution:  This issue was fixed immediately by EMIB Exchange Engineers and full email service is restored on Nihcesmlbx2.
4/13/2006 9:24:58 AM - CIT Public Folders
Description:   We're currently experiencing a BIG replication that's taking a long time in CES Public folders under the CIT tree. Folder data is there, but you may not see current data.
Resolution:  As a precautionary measure, CIT's preferred PF server has been changed until the replication is complete.
4/11/2006 12:16:32 PM - New Hot News------- NIHnet Wireless Network DMZ Maintenance
Description:   Date: Thursday, April 13, 2006 Time: 5:30pm - 6:30pm Description: On Thursday, April 13, 2006 at 5:30pm - 6:30pm EST NIHnet engineers will perform a scheduled maintenance on the NIHnet Wireless Network DMZ as part of on-going efforts to optimize the redundant paths within the Wireless perimeter network. Impact: Please note this Wireless Network maintenance activity will cause a momentary interruption in Wireless connectivity to Wireless Guest Users only. This expected loss of connectivity will be limited only to Wireless Guest users while Wireless DMZ path failover testing is conducted during the second hour of the 5:30pm – 7:30pm maintenance window. Normal Wireless connected users will not be impacted during this maintenance activity. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Maintenance Completed on Date
4/8/2006 3:45:55 PM - Slowness of NIHCESMLBX12
Description:   CES engineers will reset the information store on this server. Expected downtime is 5-10 minutes.
Resolution:  database was reset; server is fully functional again.
4/7/2006 7:04:02 PM - HHS CES - HHSCES4 - server in distress
Description:   CES engineers are actively troubleshooting a problem with this server. This server provides service to HHS/OS. Email service is not available to a portion of HHS/OS staff.
Resolution:  This hardware problem was resolved.
4/7/2006 2:36:34 PM - New Hot News------- NIHnet DNS Firewall Relocation Maintenance
Description:   Date: Tuesday, April 11, 2006 Time: 6:30am – 7:00am Description: On Tuesday, April 11, 2006 between 6:30am – 7:00am EST NIHnet engineers will perform a Scheduled Maintenance to relocate the NIHnet DNS Firewall to a new row and rack location in the Building 12 CIT Data Center. Impact: Please note this NIHnet DNS Firewall relocation is not expected to cause any disruption in data network connectivity due to the redundant aspects of the NIHnet DNS architecture. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Please note the Scheduled Maintenance to relocate the NIHnet DNS Firewall to a new row and rack location in the Building 12 CIT Data Center was successfully completed without complications on Tuesday, April 11, 2006 between 6:30am – 7:00am.
4/6/2006 4:12:02 PM - Emergency Maintenance to CIT Remedy System
Description:   Date: 4/6/2006 Time: 10:00 – 11:00 PM The Support Technologies Group (STG) needs to perform emergency maintenance to the CIT Remedy Server. This maintenance will require us to have all users logged out of the system while we restore data that was corrupted by the updates implemented last night. We apologize for any inconvenience this may cause and appreciate your patience while we work to correct this issue.
Resolution:  The Support Technologies Group (STG) has successfully completed the Emergency Maintenance.
4/4/2006 1:33:46 PM - AOL Users Cannot Log In or Send Messages
Description:   AOL is experiencing service issues with AOL IM. Users cannot log in and cannot send messages. This behavior is not system wide. AOL is aware of this issue and is working to resolve the issue
Resolution:  AOL has resolved the outages that occurred previously today. The AOL IM networking is in a nominal state currently.
4/4/2006 12:48:25 PM - ITAS monthly database maintenance tonight 9-10PM
Description:   The first Tuesday of each month the ITAS Technical Team conducts routine database maintenance from 9-10PM. During this time ITAS may experience slowdowns and some loss of connectivity. Thanks for your patience. The ITAS Technical Team.
Resolution:  Maintenance was completed without problem.
4/3/2006 11:23:16 AM - NIHCESMLBX2 Latency
Description:   There is an issue with an attachment in a calendar entry and EMIB staff is working to resolve the issue. The IC's that are currently affected are OD, NBS, OD/OER, OD/OM, OD_Fellows, OD_Internship, OD.
Resolution:  Problem attachment removed from mailbox store.
4/3/2006 9:54:46 AM - NIHnet: Building 6B Power Outage
Description:   We are currently experiencing a Power Outage in Building 6B; devices in Building 6A are also affected. NIHnet engineers are onsite, and building maintenance has been notified. There is no ETR at this time. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions .
Resolution:  Power is back up in the building 6B and all network connectivity has been restored
3/31/2006 11:05:00 AM - Planned Maintenance - MAILFWD SMTP forwarding service
Description:   Monday, April 3rd. 21:00. EMIB will be performing maintenance on this service. Expected service unavailability is 5-10 minutes.
Resolution:  Work was completed
3/30/2006 3:45:36 PM - Web of Science Literature Search Emergency Outage
Description:   The Web of Science hardware requires an emergency reset - service will be unavailable for approximately 10 minutes.
Resolution:  Emergency maintenance completed
3/29/2006 7:09:59 AM - Remedy Maintenance
Description:   Date: March 30, 2006 Time: 3:30 AM EST Duration: 30 Minutes Description: On March 30, 2006 the Support Technologies Group will be rebooting the Remedy Server. The reboot is necessary to fix an error that STG is experiencing in conducting some routine maintenance to the system. The outage is expected to take no longer than 30 minutes. Remedy users will not be able to access the system during this time and the Remedy web interface will also be unavailable. We apologize for any inconvenience this may cause and appreciate your patience.
Resolution:  The Reboot of the Remedy System has been successfully completed.
3/27/2006 4:17:59 PM - NIHnet Remote Access - VPN Maintenance
Description:   Date: Sunday, April 2, 2006 Time: 1:00am - 3:00am Description: On Sunday, April 2, 2006 EST NIHnet engineers will perform a scheduled maintenance on the NIHnet Remote Access VPN Service as part of on-going efforts to optimize the routing within the NIHnet perimeter networks. Impact: Please note this NIHnet Remote Access VPN Service maintenance activity will cause a momentary disruption to all Remote Access VPN connections within the first 15 minutes of the maintenance window while the inbound connection path is moved from an old router to a new router in the NIHnet Internet DMZ infrastructure. Expected loss of connectivity and performance degradation between NIHnet <---> Remote Access VPN connected end-users should last no longer than 15 minutes while the new path through the NIHnet perimeter network is established during this maintenance window. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  NIHnet Remote Access - VPN Maintenance Activity was completed successfully without complications.
3/27/2006 12:34:33 PM - HHS Resource Email/Public Folder Outage
Description:   HHS and HRSA users may experience resource email and public folder outages. CIT EMIB staff are currently working on the situation. Status update to follow.
Resolution:  Exchange Email and Public Folder services have been restored to HHSSWIFT.
3/24/2006 2:58:08 AM - NIHnet Outage: Executive Blvd Power Outage
Description:   Description: The NIHnet NOC has confirmed that EB6100 and EB6110 are currently without power, due to a maintenance being performed by the utility company. This maintenance is scheduled to end by 6am. All power should be restored by that time. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Maintenance has been completed. All connectivity has been restored.
3/23/2006 4:30:32 PM - ITAS Maintenance tomorrow evening 6:30-7PM
Description:   The ITAS Technical Team will be releasing a new version of the ITAS Web application tomorrow evening from 6:30-7PM. During this time users may experience slowdowns or loss of connectivity with ITAS. Thanks for your patience as we continue to improve the ITAS service.
Resolution:  ITAS Maintenance is now complete.
3/23/2006 1:29:52 PM - Inbound Mail Delay
Description:   There is currently a delay of approximately 5 to 20 Minutes for Inbound Mail to NIH, due to a high-volume of Inbound mail. CES Engineers are currently monitoring this issue.
Resolution:  Currently all queues are clear. modifications have been made to the problem componenet.
3/22/2006 3:23:39 PM - Inbound Mail Delay
Description:   There is currently a delay of approximately 5 to 20 Minutes for Inbound Mail to NIH, due to a high-volume of Inbound mail. CES Engineers are currently monitoring this issue.
Resolution:  CES Engineers have identified the problem component and made necessary adjustments. All Queues have cleared
3/20/2006 2:56:46 PM - NIHCESMLBX6 email issues
Description:   NHLBI users may have experienced a temporary email outage. Email is availible now and emib staff are looking at the cause of this issue. The current IC's affected are NHLBI, NICHD, NIEHS
Resolution:  Service was restored by EMIB engineers. Root cause analysis is ongoing
3/20/2006 11:27:05 AM - NIHnet <--> Internet(2) Routing Optimization Maintenance
Description:   Date: Tuesday, March 21, 2006 Time: 7:00pm - 9:00pm Description: On Tuesday, March 21, 2006 EST NIHnet engineers will perform a demand maintenance on the connection between NIHnet <---> Internet(2) as part of on-going efforts to optimize the routing within the NIHnet perimeter networks. Impact: Please note this Internet (2) maintenance activity will cause a momentary disruption to Internet(2) connectivity within the first 15 minutes of the maintenance window while NIHnet <---> Internet(2) traffic is moved to a new path across the NIHnet perimeter networks. Expected loss of connectivity and performance degradation between NIHnet <---> Internet(2) should be minimal while the new path through the NIHnet perimeter network is established during this maintenance window. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Please note the Internet(2) Routing Optimization Maintenance Activity (ST199468) was completed successfully without complications.
3/16/2006 10:53:20 AM - NIHnet Maintenance: Certificate Web Server Move
Description:   Event Description: On Friday, March 17, 2006 at 10:00 am, the NIH certificate website, http://tls.nih.gov will need to be moved. The server will be unavailable up to 1 hour, during this process. Please contact the NIH Help Desk at (301) 496-4357 if you have any questions.
Resolution:  The certificate web server move was completed successfully.
3/15/2006 2:53:06 PM - RIM network outage
Description:   RIM has confirmed a "Temporary minor network interruption." In which some BES's may lose connectivity to RIM. I have confirmed all but 1 or our SRP keys is connected to RIM and we are routing email. The 1 key in question has 25 users. And I cannot confirm if they are having issues or not at this time. RIM has no ETA on this at the moment.
Resolution:  BSRPTEST replying on all BES's. BESBLADE4A's instance SRP that was not connected is now.
3/14/2006 3:36:48 PM - IMPAC II is Currently Down
Description:   IMPAC II is currently down; eRA is aware of the issue and is investigating into this matter.
Resolution:  As of 4:31pm IMPAC II services are back on-line. If you are still experiencing issues connecting to IMPAC please contact the eRA Help Desk.
3/14/2006 12:12:10 PM - ITAS Maintenance
Description:   ITAS will be unavailable tonight from approximately 8:00pm to 9:00pm for routine maintenance. Thank you for you patience and cooperation.
Resolution:  Maintenance is complete. ITAS is available again.
3/14/2006 11:43:40 AM - OD Email access problems for 170 OD users
Description:   There are duplicate user objects in the directory for 170 OD users. EMIB is working with OD IT staff to get this resolved. OD and DCS Help Desk have the list of affected users. Users will expereince problems accessing thier mailboxes.
Resolution:  Duplicate accounts were deleted.
3/10/2006 2:35:09 PM - NIHnet Maintenance: Listserv Upgrade
Description:   Date: March 13, 2006 Time: 6:30am - 7:00am Description: CIT/DNST will be performing a software upgrade on the server that hosts Listserv. During this maintenance window, the Listserv website, http://list.nih.gov, will not be available. Please note that all listserv e-mail will be queued and will be processed on completion of the maintenance. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Maintenance Completed Sucessfully
3/10/2006 2:06:39 PM - NIHnet Internet DMZ Firewall OS Upgrade Maintenance
Description:   Date: Wednesday, March 15, 2006 Time: 9:00pm - 10:00pm EST Description: On Wednesday, March 15, 2006 between 9:00pm - 10:00pm EST CIT / DNST network security engineers will perform a scheduled NIHnet Internet DMZ firewall OS upgrade. This firewall upgrade is necessary to improve overall firewall performance, stability, and resolve a known problem with H.323 video conferencing. Impact: Please note this Internet DMZ firewall OS upgrade activity will cause two (2) momentary interruptions in service and connectivity between the Internet <--> NIHnet with each interruption lasting approximately less than five (5) continuous minutes in duration. The first interruption will occur as traffic is moved onto and alternate path before the firewall reload and the second interruption will occur as traffic is moved back to the original path after the OS upgrade is complete and the firewall deemed stable. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Maintenance Completed Sucessfully
3/9/2006 4:44:59 PM - Email Submission problem NIH Help Desk
Description:   The NIH Help Desk Inbox is currently not accepting incoming or outgoing email messages. Action is being taken to restore the email capabilities. At this time, please use an alternate method to contact the NIH Help Desk: Web Submit: http://support.cit.nih.gov/ (Select the “Submit Service Request” option from the upper left hand corner) Phone: 866-319-4357 (Toll free) or 301-496-8294 (TTY)
Resolution:  The NIH Help Desk Inbox has now been restored and is accepting email submissions; if you are still experiencing issues with your submission please contact the NIH Help Desk at Phone: 866-319-4357 (Toll free) or 301-496-8294 (TTY)
3/8/2006 2:27:10 PM - Building 10B - ACRF Electrical Maintenance
Description:   On Thursday, March 9, 2006 at 6:00am - 7:00am OD/ORF electricians will be performing a power maintenance on emergency power in the Building 10B - ACRF area. http://orf2.od.nih.gov/utilityshutdown/scripts/utility_shutdown.asp?seq=5806 Please note this electrical maintenance activity is expected to have a VERY LOW probability of service impact to the NIHnet data network infrastructure in Building 10B - ACRF because all CIT operated data network equipment is supported by UPS devices that are expected to provide sufficient battery backup in the event power to data network equipment is lost during this one-hour maintenance window. However, if you believe you have Institute operated LAN equipment that is powered by emergency (Red) electrical outlets then please consider checking UPS battery support as a means of obtaining an estimation of battery backup duration. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Maintenance Completed.
3/7/2006 9:49:19 AM - SERCH Latency
Description:   The SERCH team has reported around 9:09am this morning that they are experiencing latency this while using SERCH. The SERCH team is aware of the problem and is working to resolve the issue.
Resolution:  SERCH latency issues have diminished, and network connectivity should be running as expected. Please contact the NIH Help Desk with any additional questions or concerns.
3/1/2006 5:26:33 PM - NIH CES SMTP inbound email - possible delay in delivery
Description:   CES admins are seeing delivery delays for INbound SMTP mail. This is a negative result of activities surrounding the move of MAILFWD service to CES at 3 PM today. We are actively working the issue and expect resolution soon. Customers may see a delay of 10 to 20 minutes.
Resolution:  Systems required some leveling after the service additions. We believe performance will be returning to normal shortly.
3/1/2006 11:55:08 AM - ATV System - Login Issues
Description:   Customers are reporting that they are unable to access ATV, for now please relate all tickets to ATV Clinical Center Technical until more information is provided.
Resolution:  Time Resolved: 1:20 pm EST CC Technical Operations has reported to the NIH Helpdesk that the ATV Log In Issue has been resolved.
3/1/2006 10:49:15 AM - NIHnet Off-Campus Network Optimization Maintenance
Description:   On Saturday, March 11, 2006 between 8:30am - 4:00pm CIT/ DNST network implementation and operations personnel will perform a scheduled Off-Campus network optimization maintenance activity in select Executive Boulevard and Fishers Place buildings. This network optimization maintenance activity is in support of continuing efforts by CIT / DNST to improve and expand Off-Campus network performance, reliability, and capacity. Please note this activity is expected to have a very low probability of service impact to any of the NIHnet Off-Campus locations AND all potentially affected NIH Institute and HHS OpDiv technical POCs have already been notified prior to posting of this HotNews notification. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Maintenance completed on Saturday, March 11, 2006
3/1/2006 7:52:07 AM - Unscheduled Network Outage Affecting ORS
Description:   Time Reported: 7:30am EST System Affected : ORS is currently experiencing network connectivity issues. Action being taken : ITB is currently working with CIT to resolve this issue. More information to follow.
Resolution:  Time Resolved: 8:48 am EST ORS has reported to the NIH Helpdesk that the network connectivity issue has been resolved.
2/28/2006 1:36:17 PM - Remedy Escalation Process Stuck
Description:   The Remedy Escalation Process is currently stuck. In order to resolve this issue we will quickly restart the Remedy service. This should result in a very limited loss of connectivity. Once the Escalation process has resumed escalations which have been waiting to run will process. This may result in some old or invalid notifications being sent. We apologize for any inconvenience this may cause.
Resolution:  The Remedy Service has been restarted and the Escalation process has resumed normal functions. We again apologize for any inconvenience this may have caused.
2/28/2006 11:53:47 AM - Public Folder routing issues
Description:   A problem with a newly deployed CES Public Folder server caused problems for a limited number of customers that use PF's for email management or email enabled applications. CES engineers resolved that the only way to safely release the PF stored email and ensure none was lost was to allow one of two things to happen - either NDR it or delay delivery. This is what was done. So users may have received an NDR in the last 12 hours of the posted message or its delivery may have been delayed. CES admins are working with MS to determine why this the anomalous behavior occurred in the new server.
Resolution:  CES Admins continue to work with vendor
2/27/2006 3:52:43 PM - CES SMTP delivery problems
Description:   Customers are reporting delays in SMTP email delivery. CES engineers are investigating.
Resolution:  Problem identified and resolved. SMTP queues are clearing or cleared.
2/27/2006 11:49:03 AM - Changes to Mail Forward Service
Description:   CIT will move the MAILFWD SMTP service to a new platform on March 1, 2006. The new service platform will provide SMTP service as well as improved anti-virus and anti-SPAM support. This service leverages the impressive mail processing power of the IronPort technology that has been running in production in the NIH Central Email service for 2 years. This solution will provide NIH with a number of feature improvements and long-term vendor support. We intend to make the migration in a manner seamless to end users, so you should not need to make local configuration changes. The email list for support, CIT MailfwdAdmin, and the MAILFWD domain names will remain the same. Unless your system uses the IP address for the mail forward systems, there is no action required. If you have hardcoded the IP address into your systems or firewalls, you will need to transition to the new IP address prior to the transition date. The new IP addresses are: 128.231.90.106 128.231.90.107 128.231.90.108
Resolution:  MAILFWD SMTP has been moved successfully.
2/26/2006 12:57:41 PM - NIHFILER
Description:   Nihfiler is currently inaccessible and EMIB staff is currently working on resolving the issue.
Resolution:  The was rebooted much earlier and is now accessible.
2/23/2006 12:36:01 PM - CES MLBX4 - user reports of slowness to access
Description:   NIAID VRC customers on this mail server are complaining of slow access. This server services ORS, NIAAA, FIC, FAES, VRC. CES engineers are trying to determine the cause of the complaint.
Resolution:  Closed
2/23/2006 7:00:34 AM - NIHCESMLBX10 in distress
Description:   Server is being rebooted. NCI users are on this mailbox server. Service will be restored after reboot
Resolution:  server back up - email service restored
2/22/2006 10:47:15 AM - NIHCESMLBX2 Slowness
Description:   CES Admins are investigating reports of slowness on NIHCESMLBX2. Only OD users are affected at this time. EMIB will update as more info becomes available.
Resolution:  NIHCESMLBX2 and NIHCESMLBX4 response times are back to normal. The issue has been resolved.
2/21/2006 3:24:57 PM - Exchange Storage Group Down
Description:   NIHCESMLBX10 has run out of space for Storage Group Four databases. The five storage groups may shut down due to lack of space. CIT is working to add additional storage space to the server.
Resolution:  Exchange server is available for NCI users.
2/21/2006 2:34:53 PM - NIHnet Internet DMZ Scheduled Maintenance
Description:   On Friday, February 24, 2006 between 7:00pm - 9:00pm CIT / DNST network operations personnel will perform a scheduled Internet DMZ maintenance activity. In support of continuing efforts to optimize the NIHnet Internet DMZ it is necessary for CIT/DNST network operations personnel to perform a non-business hours network optimization maintenance on the Fernwood path of the NIHnet Internet DMZ. During this maintenance window, there is a high probability of intermittent service disruption of Internet traffic between NIHnet and the Internet as new network connectivity and routing paths stabilized as a result of this planned change to the NIHnet Internet DMZ infrastructure. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Maintenance Activity Completed
2/17/2006 3:53:54 PM - ITAS Web service is experiencing problems
Description:   The ITAS Web service is experiencing a problem. ITAS Technical Team members are reviewing the problem and expect to have a resolution quickly. Sorry for the inconvenience.
Resolution:  The problem with the ITAS Web server has now been resolved, and all ITAS systems are fully operational.
2/16/2006 11:24:19 AM - NIHnet: Bldg 35, 3rd floor Network Issues
Description:   Some users in bldg 35, 3rd floor are reporting network connectivity issues. CIT DNST engineers are onsite and investigating the problem. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  CIT DNST Engineers replaced a faulty module and all network connectivity has been restored
2/16/2006 10:10:36 AM - CIT Remedy Server notification issues
Description:   CIT Remedy Server notifications are not currently functioning. STG Staff are working on the problem, which is requiring the running of an extended process on the Remedy database. ETA of return to service is currently 11:00 AM.
Resolution:  STG believes that they have rectified the issue but Remail right now is running a huge backlog and could be experiencing some latency. Remmail back operational and working correctly
2/15/2006 12:03:23 PM - Sitescope Monitor issue
Description:   One of the underlining monitors, Sitescope, for NappMan is currently unavailable. As a result, many applications monitored by NappMan may recieve false alerts. Affect systems have reported that they are currently operating normally. Engineers currently working on the problem - no ETA
Resolution:  rebuild to previous version
2/14/2006 6:31:46 PM - Problems with PETRI server
Description:   CIT staff may experience problems with PETRI while EMIB staff researches the cause.
Resolution:  Windows users should now be able to access their personal files on petri, if they are still having problems have users restart their computers in order to get shared drives. Macintosh users should also be able to access their personal files on petri via SMB (Server message Block) instead of AFP (Apple Filing Protocol). SMB: smb://petri/doej$ AFP: afp://petri.cit.nih.gov/ The #Mac team can assist customers who are not sure how to connect or have problems connecting to Petri via SMB. Please forward these customers too the NIH Help Desk.
2/14/2006 2:04:19 PM - NIHnet Emergency Maintenance: Perimeter Network
Description:   Date: 2/14/2006 Time: 8:00 pm - 9:00 pm Description: CIT/DNST will perform an emergency maintenance on the secondary path of our perimeter network. Impact: There may be a very brief loss of connectivity as the NIHnet routing converges. This loss of connectivity is expected to be very brief with minimum packet loss and data retransmission. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Maintenance Completed
2/13/2006 3:27:44 PM - Emergency Maintenance on the CIT Remedy Server
Description:   STG needs to conduct emergency maintenance on the Remedy Server. The server will be offline from 8 to 8:30 PM this evening. During this time STG personnel will make some necessary changes to the Remedy System. We apologize for any inconvenience this may cause.
Resolution:  Maintenance has been completed and services restored. Actual AR System downtime was less than 5 minutes. Thank you for your understanding and patience
2/13/2006 3:26:49 PM - Emergency Maintenance on the CIT Remedy Server
Description:   STG needs to conduct emergency maintenance on the Remedy Server. The server will be offline from 8 to 8:30 PM this evening. During this time STG personnel will make some necessary changes to the Remedy System. We apologize for any inconvenience this may cause.
Resolution:  Duplicate event
2/13/2006 11:58:23 AM - NIHCESMLBX8 users experiencing slowness
Description:   Users on NIHCESMLBX8 are experiencing slowness. EMIB is currently troubleshooting the issue. Affected users are NLM, NLM NCBI, NLM LHC, NCCAM, NIDA_HQ and NIDA_IRP, NIDCD
Resolution:  server rebooted and issues resolved
2/10/2006 2:52:02 PM - NIHnet Maintenance: Mailfwd Migration
Description:   CIT is preparing to move the MAILFWD SMTP service to a new platform. The new service platform will provide SMTP service as well as improved anti-virus and anti-SPAM support. This service leverages the impressive mail processing power of the IronPort technology that has been running in production in the NIH Central Email service for 2 years. This solution will provide NIH with a number of feature improvements and long-term vendor support. We intend to make the migration in a manner seamless to end users, so you should not need to make local configuration changes. The email list for support and the MAILFWD DNS names will remain the same. Next week, we will send another email with the specific date for transition. Unless your system uses the IP address for the mail forward systems, there is no action required. If you have hardcoded the IP address into your systems or firewalls, you will need to transition to the new IP address prior to the transition date. The new IP addresses are: 128.231.90.106 128.231.90.107 128.231.90.108 Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Closing this Hot News at this time. More updates will follow at a later date.
2/7/2006 3:21:29 PM - Helix Systems down for scheduled maintenance
Description:   The Helix Systems machines will be down for scheduled maintenance starting 4 pm on Tuesday, 7 Feb 2006. It is anticipated that the systems will be available no later than midnight. Systems affected include Helix, Nimbus, Quasar, the Biowulf cluster, and the NIH Mascot server.
Resolution:  All systems up and available.
2/4/2006 9:51:00 AM - Remedy Server Restart
Description:   The CIT Customer Support Remedy server will be briefly out of service while being rebooted to correct an issue with email notifications. Outage should not last more than ten minutes.
Resolution:  The server restart has been completed, and the email notification issue is resolved. Thanks for your patience while we worked to correct this problem.
2/4/2006 7:22:56 AM - NIH CES Email Maintenance - SAN MOVE
Description:   The maintenance did go past the expected 06:00 end time. The SAN was successfully moved and we began bringing services online at 6:45. EMIB will be testing and checking services the remainder of the morning. Queues are starting to clear.
Resolution:  closed
2/4/2006 6:52:01 AM - NIH CES Email Maintenance
Description:   This is a Hotnews reminder. NIH CES email services will be unavailable from 10 PM Friday to 6 AM Saturday, during a maintenance window that begins Friday February 3rd. Tonights maintenance was originally scheduled for 12/16. CIT is working to improve the overall infrastructure of the NIH Datacenter. There is a requirement to physically move the row of systems where most of the NIH Central Email Service is housed. This move involves the relocation of the CES Storage Area Network (SAN). This is a significant undertaking. We have taken all necessary and available steps to minimize downtime. Tonight EMIB will be moving the CES SAN and we expect to use the entire scheduled 8 hour window. While there may be intermittent service, plan that CES email will be unavailable for the entire maintenance window. Storage for SMTP mail is separately maintained and so INbound SMTP will queue during the SAN outage
Resolution:  Work complete
2/3/2006 7:25:10 PM - REMINDER – NIH CES Email unavailable Friday 2/3/06 10 PM to 6 AM
Description:   This is a Hotnews reminder. NIH CES email services will be unavailable from 10 PM Friday to 6 AM Saturday, during a maintenance window that begins Friday February 3rd. Tonights maintenance was originally scheduled for 12/16. CIT is working to improve the overall infrastructure of the NIH Datacenter. There is a requirement to physically move the row of systems where most of the NIH Central Email Service is housed. This move involves the relocation of the CES Storage Area Network (SAN). This is a significant undertaking. We have taken all necessary and available steps to minimize downtime. Tonight EMIB will be moving the CES SAN and we expect to use the entire scheduled 8 hour window. While there may be intermittent service, plan that CES email will be unavailable for the entire maintenance window. Storage for SMTP mail is separately maintained and so INbound SMTP will queue during the SAN outage
Resolution:  Work Complete
2/3/2006 4:14:07 PM - 102 Pager Web Service Relocated
Description:   As of 2/3/2006, the web version of the 102 Paging Service -- http://102pager.nih.gov/ -- has been relocated to a different network location & thus has resulted in a change to the server address used to get to the service. NOTE: This move does not affect the Emergin WirelessOffice client/server product. However, customers are urged to migrate to the web verson. For customers who have been using the web version on the computer they are using will very likely be informed by their web browser that the site couldn't be found. There will be a login window displayed at this point & in this login window, there will be fields for the customer to put in their name & a password as well as a field for the Server. A password is NOT required. However, a Username is required -- use NIH domain/login username. In the Server field, the old address will be displayed. Customers simply need to change the Server field to: 102pager.nih.gov Customers who have NEVER before used the web version on the computer they are using will have no problems though customers are advised to change the server to 102pager.nih.gov -- this can be done by selecting Change Server from the Pager's File menu. Make sure to enter a Username -- NIH domain/login username -- in the Username field, checkmark the Remember Password option. If you have further questions, please contact the NIH Help Desk at 301-496-4357 or visit us at http://ithelpdesk.nih.gov/ or email us at ithelpdesk@mail.nih.gov
Resolution:  No resolution was necessary. This Hot News was sent out at the request of DNST Video Services & the NIH Help Desk manager as an informational alert to the NIH IT community.
2/3/2006 10:50:05 AM - Power Outage in ATC Building
Description:   NCI has reported to the NIH Help Desk that they are experiencing a power outage in the ATC bldg location Currently Back Up Generators are up and running but not all customers in that building have power. More Information forthcoming when available...
Resolution:  NCI has reported to the NIH Help Desk that the power outage has been resolved
2/2/2006 4:57:31 PM - ITAS is scheduled for maintenance Monday, Feb 6 from 6pm - 7pm.
Description:   ITAS is scheduled for maintenance Monday, Feb 6 from 6pm - 7pm. This maintenance includes production upgrade and new ITAS release. During this time, ITAS will not be available. Sorry for any inconvenience this may cause you. If you have any questions, please contact Helpdesk at 301-594-6248 or via email at nihhelpdesk@mail.nih.gov <<<<>>>> Very Best Regards, ITAS Technical Team, DECA/CIT
Resolution:  ITAS maintenance is now complete.
2/2/2006 2:55:40 PM - NIHnet Maintenance: Fernwood Power Maintenance
Description:   Date: Saturday, 2/4/2006 Time: 3:00 am - 11:00 pm Description: The Fernwood Building will have an extended scheduled power outage on Saturday, February 4, 2006. As a result of this power outage, the Local Area Networks in the Fernwood Building will be unavailable from Saturday, February 4th at 3:00 a.m. and will be back up by 11:00 p.m. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Fernwood Building power maintenance has been completed and all network connectivity has been restored.
2/1/2006 2:46:30 PM - MSN IM OUTAGE
Description:   The MSN Messenger network is currently experiencing difficulties. Users are unable to log on and some features may not be available for logged on users.
Resolution:  Today, Microsoft reported that it was having difficulties with the MSN Messenger network. They have informed us that end users are no longer seeing the problem.
1/30/2006 2:02:12 PM - CSR Computers missing from the Domain
Description:   The CSR Computers OU was deleted. We are currently restoring all objects. Computers may need to be rejoined back to the domain.
Resolution:  The Computers OU has been restored with all computer objects
1/27/2006 11:19:32 AM - ADM is currently unavailable
Description:   EMIB staff are working to correct.
Resolution:  ADM client console is now available
1/26/2006 11:54:22 AM - Multiple CIT Customer Support Web Sites Unavailable
Description:   Currently the Remote Access, Support Staff, Parachute, and cable Modem web sites are not available. The Support Technologies Group is aware of the problem and is working to correct it. Presently there is no estimated time when the sites will be available. We apologize for any inconvenience this may cause.
Resolution:  The Domain Name Service issue has been resolved. The Remote Access, Support Staff, Parachute, and Cable Modem web sites are currently available. We apologize for any inconvenience this may have caused.
1/20/2006 9:37:33 AM - NIH Portal Maintenance
Description:   The NIH Portal will undergo maintenance this evening from 7 to approximately 9 pm. The portal will be up and available during this period. However, we would recommend not using it if at all possible. It may be slow to respond, and searching may be adversely affected.
Resolution: 
1/19/2006 9:32:00 AM - CIT Report Server (CITDCSRPT) Database Issues
Description:   The CIT Reports server that supports reporting for the NIH Helpdesk is currently experiencing database issues. We are currently researching the cause of the problem and do not have an estimated time to correct the database. We apologize for any inconvenence this may cause. If you have an urgent requirement to generate a report please contact the NIH Helpdesk at 301-496-4357.
Resolution:  The database issues with the CIT Report server have been corrected.
1/17/2006 3:31:57 PM - NIDB currently down
Description:   NIDB is experiencing Oracle database connectivity issue. Technicans are working on the problem. Estimated time of completion approximately 6:30 PM
Resolution:  Update: NIDB back online
1/13/2006 11:06:29 AM - NIHnet Maintenance: Fernwood Power Maintenance
Description:   The Fernwood Building will have an extended scheduled power outage on Saturday, January 14, 2006 until Sunday, January 15, 2006. As a result of this power outage, the Local Area Networks in the Fernwood Building will be unavailable from Saturday, January 14th at 1:00 a.m. and will be back up by Sunday, January 15th at 7:00 a.m. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Power has returned to the Fernwood Building and network connectivity has been restored,
1/6/2006 2:17:05 PM - NIHnet Maintenance: Perimeter Network
Description:   Date: RESCHEDULED for Thursday, 1/12/2006 Time: RESCHEDULED 7:00 p.m. - 10:00 p.m. Description: CIT/DNST will be completing the firewall technology upgrade on the perimeter network on Thursday, 1/12/2006. Impact: There may be a brief interruption in connectivity as we switch our internet path to accommodate the upgraded equipment. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  CIT/DNST engineers successfully upgraded the firewall technology on our Perimeter Network.
1/3/2006 12:32:05 PM - Errors on Remedy Web Interface
Description:   Users of the Remedy web interface have reported problems accessing the CIT Service Ticket form via the Remedy web interface. The Support Technologies Group (STG) is aware of the situation and is working to isolate and correct the problem. There is no estimated time to service restoration currently. We apologize for any inconvenience this may cause.
Resolution:  STG has corrected the problem with the CIT Service Ticket form on the Remedy web interface. We apologize for any inconvenience this may have caused.
1/3/2006 - NT 4.0 Trusts Removed
Description:   Per previously announced directive from the ITMC ARB, EMIB has removed trusts between nih.gov domain and legacy NT 4.0 domains
Resolution:  Per previously announced directive from the ITMC ARB, EMIB has removed trusts between nih.gov domain and legacy NT 4.0 domains
12/23/2005 9:59:10 AM - ITAS Emergency Maintenance
Description:   ITAS will be unavailable for an emergency maintenance to prevent hardware failure today from 11:30AM - 12:30PM. Sorry for the inconvenience this may cause you.
Resolution:  Maintenance work on ITAS has been completed. ITAS is now available.
12/22/2005 4:09:51 PM - ITAS Hardware Maintenance tomorrow 5:30AM-6:30AM.
Description:   A component on the ITAS Web application server is showing signs of failure and needs to be replaced in order to maintain smooth operations. The ITAS Technical Team is working with CIT/SOSB to replace the problem component tomorrow between 5:30AM and 6:30AM. Users can anticipate the system to be unavailable at this time.
Resolution:  The ITAS maintenance was unsuccessful, and the new component has been backed out. The procedure will have to be rescheduled once technicians have identified the underlying problem with the procedure. ITAS has been restored to normal operations with the old hardware for the time being.
12/22/2005 11:21:37 AM - Titan Tape Backup Facility Unavailable
Description:   The NIH help Desk has been informed of the following: The offsite datacenter backup facility became unavailable last night. Offsite backups could not be performed. Technicians are currently working to resolve the problem. Please note that local backups were made as usual last night. Only the offsite copies were not made. As a result, there should not be any user impact. Thank you
Resolution:  This problem was resolved and successfully tested last evening. Thank you
12/21/2005 6:33:03 PM - TEST FROM HOT NEWS
Description:   TEST FROM HOT NEWS
Resolution:  TTest Re Test Resolved
12/21/2005 12:30:16 PM - Novell Password sync service affecting AD password changes
Description:   We are currrently experiencing a problem with the Novell component of the infrastructure that sync's passwords with the Novell networks in certain IC's. This component is negatively impacting all AD password mods however and is being temporarily shutdown.
Resolution:  Service was reenabled
12/20/2005 3:50:00 PM - NIH CES - NCBI users SMTP INBound mail bouncing
Description:   CES admins are looking into reports of NCBI users not receiving SMTP mail. This is likely a result of scheudled CES SMTP routing changes today. NCBI is the only name space reporting a problem with the change that took place at 11 AM this morning. CES admins are working with NCBI to resolve.
Resolution:  CIT has reversed the upgrades to the NIH CES in order to rectify NCBI SMTP mail issues.
12/20/2005 2:24:04 PM - MSN Messenger Networ Outage
Description:   The MSN Messenger network is currently experiencing difficulties. Users are unable to log on and some features may not be available for logged on users. We will issue another advisory when we have an estimated time of resolution from Microsoft.
Resolution:  According to MSN Messenger website, all systems are up and running at this time.
12/20/2005 10:50:07 AM - NBS Production System Currently Unavailable
Description:   The NBS production system is currently unavailable and cannot be accessed through the NIH Portal or by using the direct link. The service provider is currently working on a resolution. This notification will be updated as additional information becomes available. Thank you for your patience as a resolution is being sought.
Resolution:  The NBS production system is now available and accessible via the NIH Portal. Thanks for your patience
12/20/2005 10:27:20 AM - NIH Portal Maintenance
Description:   The NIH Portal will be unavailable on Wednesday, December 21 from 7 to 8 PM so that the SQL Server running the portal database can be updated and patched to the latest Microsoft specifications. Please plan accordingly. During this period, a temporary page will be displayed at my.nih.gov with links to frequently used enterprise applications. We apologize for any invonvenience this may cause. Thank you for your understanding. NIH Portal Administrators
Resolution:  Maintenance completed, database server upgrade and portal returned to full service.
12/19/2005 12:35:47 PM - NIHnet: Guest Wireless Network
Description:   Description: Customers are reporting a problem with guest wireless access. CIT/DNST engineers are working on the issue. Impact: Please note that this affects guest wireless only and that VPN wireless users are not impacted by this issue. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  All problems on the guest wireless network have been resolved.
12/16/2005 4:41:22 PM - NIHCESMLXB2 SG2 reset
Description:   NIHCESMLBX2 experienced a brief outage. Users effected are part of OD. Services have been restored
Resolution:  NIHCESMLB2 back online
12/14/2005 9:14:46 PM - CES scheduled maintenance - CES SAN move - CANCELLED
Description:   NIH Datacenter construction and NIH Central Email Service hardware moves. Tonights move of the CES SAN must be postponed. It will not occur during this maintenance window. Technical problems prohibit this move at this time. The work will have to be rescheduled. We apologize for the inconvenience.
Resolution:  This maintenance is postponed
12/13/2005 10:02:40 PM - CES scheduled maintenance #3 - datacenter server moves
Description:   NIH Datacenter construction and NIH Central Email Service hardware moves. Tonights moves involve a number of IC's. Mailbox servers NIHCESMLBX7 - NIHCESMLBX8 will move beginning at 11 PM. Downtime is not to exceed 1 hour per server. All server moves are to complete by 12:00. All affected IC's were previously notified and concurred.
Resolution:  This maintenance is complete.
12/13/2005 10:02:11 PM - CES scheduled maintenance #2 - datacenter server moves
Description:   NIH Datacenter construction and NIH Central Email Service hardware moves. Tonights moves involve a number of IC's. Mailbox servers NIHCESMLBX5 - NIHCESMLBX6 will move beginning at 10 PM. Downtime is not to exceed 1 hour per server. All server moves are to complete by 11:00. All affected IC's were previously notified and concurred.
Resolution:  This maintenance is complete.
12/13/2005 8:17:57 PM - CES scheduled maintenance - datacenter server moves
Description:   NIH Datacenter construction and NIH Central Email Service hardware moves. Tonights moves involve a number of IC's. Mailbox servers NIHCESMLBX - NIHCESMLBX4. Will move beginning at 9 PM. Downtime is not to exceed 1 hour per server. All server moves are to complete by 10:00. All IC's were previously notified and concurred.
Resolution:  This maintenance was completed on time.
12/13/2005 6:58:02 AM - NIHCESMLBX11 storage group one offline
Description:   the first storage group of NIHCESMLBX11 had to be taken off-line for 3 minutes this morning to ensure transaction logs were committed.
Resolution:  logs checked for committed status and copied off. All databases accessible.
12/12/2005 8:40:45 AM - Helix Systems Scheduled Downtime, Monday December 12, 6 PM
Description:   Helix, Nimbus and Quasar will be unavailable Monday December 12 starting at 6 PM for scheduled maintenance. The Biowulf cluster will NOT be affected by this downtime. We anticipate having all systems available to users by midnight.
Resolution:  System Maintenance successfully completed on Helix, Nimbus and Quasar by 11PM, and all three systems returned to users at that time.
12/8/2005 8:10:57 PM - CES scheduled maintenance - NCI server moves
Description:   NIH Datacenter construction and NIH Central Email Service hardware moves. Tonights moves involve NCI. All NCI mailbox servers will move beginning at 9 PM. Downtime is not to exceed 1 hour per server. All server moves are to complete by 12:00. IC was previously notified and concurred.
Resolution:  This worked was completed at 11 PM
12/7/2005 7:45:24 PM - CES scheduled maintenance - Clinical Center server moves
Description:   NIH Datacenter construction and NIH Central Email Service hardware moves. Tonights moves involve CC. CES25 and CES26 will be moved at 10:00 PM. Downtime is not to exceed 1 hour. IC was previously notified and concurred.
Resolution:  Work completed at 10:45
12/7/2005 6:30:54 AM - Exchange reboot
Description:   Exchange Queues stuck. Expected outage less than 5min.
Resolution:  After Service reset the queues are clearing.
12/6/2005 2:02:26 PM - AOL IM intermittent issues
Description:   AOL is having intermittent issues with the AIM network. Most users are unaffected. Affected users are being disconnected, and the ability to log back in limited. AOL working to resolve this problem. They have not yet provided an estimated time to resolution.
Resolution:  AOL has resolved their AIM networks.
12/5/2005 10:40:57 AM - Usage issues on the CIT Aspect Switch
Description:   The CIT Aspect switch that supports the NIH Helpdesk, the Network Operations Center, the CRIS Helpdesk, the eRA Helpdesk, and NLM is currently experience a high volume of calls. Due to the high volume Customers may get a busy signal when trying to contact one of these Helpdesks. STG is aware of the problem and is currently working to alleviate the current load. There is no estimate when the problem will get corrected at this time. We apologize for any inconvenience and appreciate your patience as we work to correct this issue.
Resolution:  The Maintenance to the switch is complete and additional capacity has been added. This maintenance was done to correct the usage problems customer's have been experiencing over the past few days.
12/2/2005 7:38:09 AM - Usenet News Server (news.nih.gov)
Description:   The Usenet News server is currently unavailable. DNST engineers are currently working to resolve this issue. There is no estimated time of repair.
Resolution:  The Usenet News server has been restored.
11/28/2005 3:49:05 PM - CES Server Maintenance
Description:   The EMIB admins are rebooting NIHEXCHANGE2 due to an issue with the MTA. This will incur a brief service outage to users on that server while the server reboots, but will restore service that is currently impaired.
Resolution:  The server has been rebooted and the queue cleared. The server is now processing email normally.
11/28/2005 12:36:07 PM - Scheduled maintenance of NIH CES SMTP mailers.
Description:   EMIB will be performing scheduled maintenance of the CES SMTP mailers. Downtime is expected to be 5 minutes or less. No impact to NIH is expected. The work will be done Wednesday evening at 9pm.
Resolution:  work completed on schedule; forgot to close hotnews.
11/21/2005 4:13:08 PM - NIH LISTSERV -- Email Latency
Description:   The NIH LISTSERV service is experiencing some email latency due high email volume. Engineers are working on the problem. There is no estimated time to service restoration at this time. If you have further questions, please contact the NIH Help Desk at http://ithelpdesk.nih.gov or 301-496-4357 or ithelpdesk@nih.gov
Resolution:  At approximately 11:00PM last night, service returned to normal operating levels.
11/21/2005 8:00:12 AM - Helix Access Problem
Description:   Helix Support team are working to resolve a problem w/ the Helix File Server (disk storage) which is interfering w/ users from connecting to Helix. Further update will be provided as they become available.
Resolution:  Helix system has been restored. Problem has been identified and resolved. Users are now able to log in to Helix as normal.
11/19/2005 8:31:59 AM - NIHnet: Partial Power Outage in Building 10
Description:   Description: There is currently a partial power outage in Building 10 affecting some local networking equipment. Building Maintenance is aware and they are working on the issue. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Power was restored to area affected by the partial outage.
11/19/2005 7:59:09 AM - NIH CES Blackberry service - possible service interruption
Description:   Some users may be experiencing a problem with their service. Not all users are affected. EMIB engineers are actively working on a resolution. We believe the problem is identified and expect normal operations in 30 minutes.
Resolution:  Service is returned to normal
11/17/2005 9:02:10 AM - NIH Antivirus Web Site is temporarily unavailable
Description:   Http://antivirus.nih.gov is temporarily unavailable
Resolution:  The web site is back on line.
11/17/2005 9:00:26 AM - ITAS is Currently Unavailable
Description:   ITAS is currently unavailable. The service provider is currently working on a resolution. This notification will be updated as additional information becomes available. Thanks for your patience and understanding as we work to resolve this issue.
Resolution:  ITAS is now available. Thanks for you patience
11/17/2005 7:59:54 AM - OD Power outage &Network connectivity problems in Executive Blvd.
Description:   Currently there is a network connectivity issue occurring at Executive Blvd. Application servers are not available for users in Executive Blvd. In addition, there is a power outage in Bldg 6011 which will affect all OD users in Rockledge and Democracy. (Development Domain will is also not be available). OD Technicians are working to resolve these problems as quickly as possible. Further updates will be provided as they become available.
Resolution:  Following the restoration of electrical power at Executive Blvd, the network infrastructure and all servers have been restored to operation.
11/15/2005 3:36:06 PM - NIH CES Blackberry event - service disruption
Description:   There is a communications issue between RIM and 1 of the multiple BES servers. We expect to have the problem resolved very quickly. The problem is not on the server side and requires RIM to resolve. This will affect any user on that system and their ability to send and receive from their Blackberry device. We are unable to provide a list of affected users.
Resolution:  RIM has re-enabled the SRP connection for this system
11/15/2005 2:30:31 PM - NIH CES SMTP INbound Queues
Description:   Please note - INbound SMTP queues are still in the clearing process at this time. No email was lost. We expect the service to be back to normal by COB. Thank you
Resolution:  Service returned to normal operation
11/15/2005 10:29:42 AM - inbound SMTP mail
Description:   there appears to be an issue with inbound SMTP Mail for CES. CES staff is aware and working on the problem
Resolution:  Ironport appliances were upgraded and rebooted. Mail is clearing now
11/10/2005 5:07:50 PM - Emergency Maintenance: Remedy Server Reboot
Description:   We will be rebooting the Remedy server at 6:00 PM on Friday November 11, 2005. It is expected that Remedy service will return within 15 minutes after the reboot.
Resolution:  The Server has been rebooted and the services restored.
11/3/2005 5:03:38 PM - Problem accessing HHS Web sites
Description:   Users on the NIH network may be experiencing problems accessing www.dhhs.gov or www.os.dhhs.gov. Network Operation Center is currently investigating to resolve the issue.
Resolution:  Problem accessing www.dhhs.gov and www.os.dhs.gov has been resolved.
11/2/2005 4:09:22 PM - ITAS maintenance on tonight from 6-7:30PM
Description:   Dear ITAS User, We will be performing an ITAS technical upgrade this evening, Nov 3, 2005 from 6:00PM - 7:30PM. During this time ITAS will not be available. We apologize for any inconvenience this may cause you. If you have any questions, please contact the NIH Help Desk: Phone: 301-496-4357(6-HELP) (local) 866-319-4357 (toll free) 301-496-8294 (TTY or Relay Users Only) Email: helpdesk@mail.nih.gov Web: http://ithelpdesk.nih.gov As always, we appreciate your patience. Very Best Regards, ITAS Technical Team, DECA/CIT
Resolution:  The ITAS maintenance is now complete.
11/2/2005 9:16:30 AM - NIH Portal Maintenance
Description:   On Saturday, November 5, the NIH Portal will be undergoing required maintenance. We are performing the process of removing '-AD' from user names. This will require database scripts and a search index rebuild. Users may see the maintenance page during this time. We expect the work to be completed and portal fully available by early afternoon.
Resolution:  Portal is now back up and available. However, the search index is being rebuilt. Users now have access to the portal but searching will not work properly until the index has completed rebuilding.
11/2/2005 5:02:54 AM - NIHCESPUB3 exessive queues
Description:   Exessive queues, will need to reboot server to clear
Resolution:  server back up queues cleared
10/28/2005 5:24:47 PM - NIHnet: Building 10 Partial Power Outage
Description:   Description: CIT is getting reports of a partial power outage in Building 10. The CIT/DNST network infrastructure is not impacted by this power outage at this time; however there is some loss of power for local ICs in that building. OD/ORF is currently aware of the power issue. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions regarding the network.
Resolution:  Power was restored in building 10. A transformer that blew on the 14th floor appears to have caused the outage.
10/26/2005 12:39:38 PM - Remedy-NED Integration Down 10/26/2005
Description:   Remedy-NED Integration is currently down. While this will not affect day to day operations of the helpdesk, neither new records nor record changes in the NED will be reflected within the CIT Customer Database. At this time we have no estimate when services will be restored.
Resolution:  The NED-Remedy integration has been returned to service and updates are in the process of being pushed across from NED to the CIT Customer Database in Remedy. The databases should be fully synched shortly. Thanks for your patience while we worked to resolve this issue.
10/25/2005 12:49:33 PM - Exchange 2003 Service Problems
Description:   Multiple Exchange 2003 servers are having issues. CES Operations staff are investigating the problem.
Resolution:  Service restored - EMIB staff are researching the cause/source of the outage.
10/24/2005 10:35:12 AM - CIT Customer Support Remedy Server - Service Restart at 10:50 AM
Description:   To address outbound mail problems and slowness issues, we will be restarting the Remedy service on the CIT Remedy Server at 10:50 AM. This will cause a momentary delay on the server. If you are in the process of updating a ticket, your save action may time out. Should this happen, simply wait a moment and attempt to save again. Your subsequent update should go through without a problem. Thanks for your patience and support while we work to resolve this issue.
Resolution:  The Remedy service was restarted as indicated and further monitoring has shown that the problem is resolved.
10/19/2005 6:02:46 PM - Maintenance: NIHFILER
Description:   NIHFILER will be rebooted at 7:00pm this evening. Downtime is expected to be no more than 10 minutes
Resolution:  Maintenance is complete and server is restored full function
10/19/2005 4:58:48 PM - Remedy-NED Integration down
Description:   Remedy-NED Integration is currently down. While this will not affect day to day operations of the helpdesk, neither new records nor record changes in the NED will be reflected within the CIT Customer Database. At this time we have no estimate when services will be restored.
Resolution:  The NED-Remedy integration has been returned to service and updates are in the process of being pushed across from NED to the CIT Customer Database in Remedy. The databases should be fully synched shortly. Thanks for your patience while we worked to resolve this issue.
10/17/2005 7:30:08 AM - NIH Help Desk has limited support channels
Description:   Due to the power outage in the Fernwood building, the NIH Help Desk currently is experiencing major service delays. (note: the Help Desk is located in the Fernwood building). Currently, our telephone numbers are unavailable to our customers. To receive support at this time, it is recommended that customers contact us by using the "Online Service Request" form at ithelpdesk.nih.gov or via email (ithelpdesk@nih.gov). Our customers may have increased wait times via these other channels during this time and we apologize for the inconvenience. There is no estimated time of resolution for the outage according to the lastest word from the power company. Thank you.
Resolution:  The NIH Help Desk is now operating at normal capacity. We thank you for your patience as we worked to resolve these issues.
10/17/2005 6:13:24 AM - NIHnet: Partial Power Outage in Fernwood
Description:   Description: There is currently a power outage in the Fernwood Building affecting some telephones, workstations and network connectivity. Building engineers are working on the issue. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Power has been restored to the Fernwood Building and all telephone and network connectivity has returned.
10/16/2005 5:36:12 PM - NIHnet: Power Outage in Rockville Area
Description:   Description: There is a power outage in the Rockville area, currently affecting NIH networks connected via the Parklawn Building and 5635 Fishers Lane. PEPCO is aware of the problem and working on the issue, although there is no estimated time of repair.
Resolution:  Power has returned to the Rockville area and network connectivity has been restored.
10/14/2005 1:28:03 PM - T-Mobile Blackberry Outage
Description:   Per T-Mobile: Service(s): GPRS - All data services impacted Fault Description: Subscribers may experience failures utilizing GPRS Data services. Market(s): All Status Update: Subscribers may experience failures utilizing GPRS Data services. Geographic Location(s): All. Please be advised that during this outage customers with T-Mobile Blackberry service may unable to send/receive and sync wirelessly. Updates to follow when available
Resolution:  Per T-Mobile: Status Update: GPRS services have been restored.
10/13/2005 10:13:05 PM - CIT Customer Support Remedy Server Restart
Description:   To address an issue with the CIT Customer Support Remedy server, we are going to restart the server at 10:20 this evening. There will be a brief outage while the server restarts.
Resolution:  The restart resolved the mailbox issues. The server is now back up and fully functional. Thanks for your patience and support.
10/13/2005 5:23:20 PM - Security patching of the Central Email Service
Description:   The Central Email Services will require immediate patching to safeguard against recently announced vulnerabilities in certain components of the OS. There is potential for a worm or other malware being constructed to take advantage of that vulnerability in the near term. EMIB has tested these patches and will be applying to CES services at 9 PM this evening. The CES is expected to be unavailable for 15 minutes.
Resolution:  Work is complete
10/12/2005 10:04:12 AM - NIHnet: Wireless Issues
Description:   Description: Guest wireless users are currently experiencing authentication issues. CIT/DNST engineers are investigating the problem. There is no estimated time of repair at this time. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  All NIHnet wireless issues have been resolved at this time. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
10/10/2005 5:07:32 PM - Reboot of NIHFiler
Description:   NIHFiler will be unavailable for approx. 15 minutes for a reboot
Resolution:  Mailtenance completed
10/6/2005 1:58:08 AM - CES Mailbox Server in Distress
Description:   NIHCESMLBX8 is in distress. EMIB Engineers are investigating. Users serviced by that server may not be able to connect to their mailboxes.
Resolution:  The server has been returned to service. Users on that server should now be able to access server resources as before.
9/29/2005 12:50:09 PM - Customer Support Web Server - Emergency Maintenance
Description:   Currently the web sites supported by the CITDCSWEB2 server are not responding and the server is in distress. These sites include the CIT Training, Knowledgebase, Statsoft, and NIH Helpdesk sites. STG and EMIB will be rebooting the server and hope to have the problem resolved shortly.
Resolution:  The server has been rebooted. The affected web sites are now operational and responding.
9/28/2005 9:17:20 AM - NIH Data Warehouse Budget and Finance One Day Behind
Description:   The NIH Data Warehouse Budget and Finance business area is currently one day behind. Data is current as of close of business 9/26/05. We expect it to be updated by this afternoon.
Resolution:  Resolved--------Data is now up to date as of 9/27/05.
9/26/2005 9:31:19 AM - OD Home Directory connectivity issue.
Description:   OD Home Directory server is experiencing connectivity issue. This is effecting Executive BLVD and Rockledge. OD Network group is aware and currently working on the issue. As soon as we have more information, we will send an update.
Resolution:  OIT restored the Home server to repair the hardware problem that has caused the connectivity issues.
9/25/2005 12:32:06 PM - CIT File Server - PETRI -
Description:   CIT EMIB is will be rebooting PETRI momentarily to replace faulty hardware.
Resolution:  Work complete - system is returned to service
9/23/2005 9:20:02 AM - NIHnet: NHGRI/Baltimore Network Issues
Description:   Description: NHGRI users located in Baltimore are experiencing a loss of network connectivity at this time. CIT/DNST engineers are en route to investigate. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Network connectivity has been restored for NHGRI users in Baltimore. The issue was related to a power outage earlier today.
9/20/2005 4:03:14 PM - LISTSERV -- Unable to receive or deliver email messages
Description:   The NIH Help Desk has been informed by the NIH LISTSERV administrators that LISTSERV is currently unable to receive or deliver email messages. At this time there is no estimate on return to service. If you have further questions, please contact the NIH Help Desk at: Phone: 301-496-4357 (local) or 866-319-4357 (toll free) or 301-496-8294 (TDD) Email: ithelpdesk@nih.gov Web: http://ithelpdesk.nih.gov
Resolution:  The issue has been resolved. The LISTSERV is now able to recieve or deliver email messages.
9/19/2005 4:43:07 PM - Maintenance of Production Customer Support SQL Servers
Description:   On Wednesday September 21, CIT’s Support Technologies Group will apply SQL maintenance service packs to the three production Customer Support SQL servers. The process should take no more than an hour for each server and will be performed sequentially (one server at a time). Impacted systems may show some sluggishness during the upgrade and will be down for a short period (5 - 7 minutes) as the server is rebooted once the upgrade process has been completed. The patching will start at 10:00 PM and should take no longer than three hours to complete. The servers will be patched in this order: - SQL Database Server (non-Remedy, non-Reporting): This may impact the CIT Knowledge Base (KB), some portions of the support page (ITHelpdesk.nih.gov or support.nih.gov), some portions of Statsoft.nih.gov and some portions of Remedy pertaining to the KB - Report Server (CITDCSRPT): This may impact the reports web page under ithelpdesk.nih.gov and will impact customers who are running their own reports against the server. - Remedy Database Server: This may impact Remedy, some portions of the support page (ITHelpdesk.nih.gov or support.nih.gov), Mid-Tier (support.nih.gov/remedy and support.nih.gov/remedy3), the KB and training.cit.nih.gov We appreciate your support as we work to maintain the currency of the software levels on our production servers.
Resolution:  All three servers successfully patched. Services tested and everything appears back to normal. Thank you for your patience and understanding.
9/17/2005 11:23:37 AM - NIH Help Desk Email Notifications showing incorrect FROM address.
Description:   Email notifications generated by the NIH Help Desk Remedy ticketing system are showing "CRIS Help" in the "From" field. We are investigating the problem. No ETA at this time. We appreciate your patience while we work to resolve this issue.
Resolution:  The mail profile issues have been resolved by the system restart. Email notifications appear to be coming from the proper addresses now. We will continue to monitor the server throughout the weekend to ensure that it continues to function properly. We appreciate your patience while we worked to resolve this issue.
9/17/2005 10:53:28 AM - CES - Emergency reset of NIHCESMLBX
Description:   This server will be reset at 11:45 am today. The system will be unavailable for 15 minutes. This will affect the following IC's: CIT, CSR, NIGMS, NCRR, NIBIB, NINDS. This is to allow for correction of post-maintenance window issues with the server's anti-virus service.
Resolution:  This work was completed successfully
9/17/2005 8:02:38 AM - NIHCESMLBX4 reboot
Description:   NIHCESMLBX4 is being rebooted. Downtime is expected to last 5 minutes. Affected IC's: OD/ORS, NIAAA, FIC, VRC.
Resolution:  work was completed
9/17/2005 6:47:56 AM - NIHCESMLBX reboot
Description:   NIHCESMLBX is being rebooted. Downtime is expected to last 5 minutes. Affected IC's: CIT, CSR, NIGMS, NCRR, NIBIB, NINDS.
Resolution:  Server rebooted and returned to service.
9/15/2005 12:47:24 PM - IT Help Desk Ticket Submission not functioning for Titan issues.
Description:   The ticket submission form on the IT Help Desk page is currently returning an error if the user answers "Yes" to the "Is this request related to TITAN (Mainframe Services)?" We are currently investigating this issue. We do not have an ETA at this time, but we appreciate your patience while we work to resolve the problem.
Resolution:  We have located and corrected the offending code. Titan tickets are now being successfully submitted from the IT Help Desk website. Thanks for your patience and cooperation while we worked to resolve this issue.
9/14/2005 2:20:04 PM - Web of Science literature search service
Description:   The Web of Science literature search service is down for emergency hardware maintenance. We expect the service to be restored by 3:00 pm. We apologize for the short notice.
Resolution:  Problem has been resolved and the system is back in service as of 3:30 pm.
9/12/2005 8:22:08 AM - NIHnet: Network Connectivity Issues at 5625 Fishers Lane
Description:   Users have reported network connectivity issues at 5625 Fishers Lane. CIT DNST Engineers are currently investigating. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The problem was isolated to the third floor switch, and has been resolved.
9/9/2005 4:48:16 PM - ITAS maintenance Saturday Sept 10 3PM - 3:30PM.
Description:   ITAS will be down for maintenance tomorrow Saturday Sept 10 3PM - 3:30PM.
Resolution:  ITAS maintenance is now complete.
9/9/2005 1:34:43 PM - AD Account Deletion
Description:   NIDCR DIR users have been deleted. EMIB is in the process of restoring all users and linking back to the mailbox.
Resolution:  CIT restored the computers, groups and users that NIDCR staff removed. All was completed with an hour.
9/9/2005 12:12:09 PM - NIHEXCHANGE13 in distress
Description:   NCI Users reporting connectivity problems, EMIB staff investigating
Resolution:  service has been restored. NCI users connecting
9/8/2005 4:28:50 AM - NIHCESMLBX issue
Description:   Mail queues for NIHCESMLBX were stuck. Effects CIT, NINDS, NIGMS, NCRR, NIBIB, CSR.
Resolution:  services restarted and is back in operation.
9/7/2005 6:28:22 PM - NIHnet Maintenance: Fiber Relocation
Description:   Date: Saturday, 9/10/2005 Time: 6:00 am - 1:00 pm Description: In our continuing preparation for the upcoming construction in Building 10, CIT/DNST will be relocating select fiber connections that traverse our old router room in Building 10. Impact: The wireless network in Building 40 will be unavailable for 30 minutes during the maintenance window. Due to the redundant nature of the NIHnet, there is little impact expected to the rest of the network as a result of this maintenance. The administrators of those networks that will be impacted have been notified. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The fiber relocation maintenance has been successfully completed. As always, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
9/2/2005 12:04:45 PM - Outbound Notification Problems with CIT Remedy System
Description:   The CIT Remedy System that supports the CIT Service Ticket and other applications is currently experiencing intermittent problems sending notifications. The CIT Support Technologies Group is currently looking at the problem and does not have an estimate on when the problem will be corrected. We apologize for any inconvenience this may cause.
Resolution:  The Support Technologies Group has identified and corrected the problem with the outbound notifications. The problem affected notifications being sent dating back to Monday, August 29. The affected notifcations will be resent. We apologize for any inconvenience this may have caused. If you have any questions, please contact the NIH Helpdesk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov.
9/1/2005 12:40:31 PM - Slow performance on NIHCESMLBX.
Description:   NIHCESMLBX is under heavy load right now, and as a result mail client performance may not be optimal. CIT/DCSS/EMIB engineers are investigating the problem.
Resolution:  Event was associated with someone sending email to multple people in the GAL which impacted mail delivery performance. The mail storm has been corrected, and the server has leveled out.
8/31/2005 12:53:56 PM - Remedy-to-NED Connection Disabled
Description:   Due to a problem stemming from this past weekend's upgrade of the Remedy server, the Remedy-to-NED connection has been temporarily disabled. We do not at this time have an ETA for return to service. For organizations that have already been integrated, the customer data currently in the customer database is accurate as of Saturday evening, 8/27/2005. Until the connector is returned to service, updates to the NED will not be reflected in the Remedy Customer DB. To minimize the impact on customers, we will be disabling the email notifications that go to customers informing them when their ticket data does not match their NED data. We apologize for the inconvenience and appreciate your patience as we work to correct the problem. Look for updates to this Hot News as events progress.
Resolution:  The Remedy-to-NED connector has been re-enabled. The service outage resulted in a few duplicate customer records in the database which we will have cleaned up. Otherwise, the data in the customer database should now be current and accurate. We will be turning the customer notifications back on later today. We appreciate your patience while we worked to resolve this issue.
8/31/2005 9:08:31 AM - NIHnet: Wireless Network Issues
Description:   Description: NIHnet users are reporting problems with the wireless network. CIT/DNST engineers are currently investigating this issue. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The NIHnet wireless connectivity issues have been resolved. CIT/DNST engineers will continue to investigate the root cause.
8/30/2005 2:24:04 PM - Portal Maintenance
Description:   On Saturday morning, September 3rd, between the hours of 10 and noon, the NIH Portal may intermittently have a temporary page displayed while maintenance is being performed.
Resolution:  Portal Maintenance was completed as planned.
8/26/2005 10:43:48 AM - ITAS maintenance Aug 30, 2005 from 6:00PM - 6:30PM.
Description:   Dear ITAS User, We will be releasing an update to ITAS this Tuesday evening, Aug 30, 2005 from 6:00PM - 6:30PM. During this time ITAS will not be available. We apologize for any inconvenience this may cause you. If you have any questions, please contact the NIH Help Desk: Phone: 301-496-4357(6-HELP) (local) 866-319-4357 (toll free) 301-496-8294 (TTY or Relay Users Only) Email: helpdesk@mail.nih.gov Web: http://ithelpdesk.nih.gov As always, we appreciate your patience. Very Best Regards, ITAS Technical Team, DECA/CIT
Resolution:  ITAS maintenance is now complete.
8/25/2005 4:16:26 PM - NIHFILER in distress
Description:   The NIHFILER server is in distress. EMIB Staff are investigating right now. This may cause disruption to services that depend on this server.
Resolution:  NIHFILER has been restored to service. EMIB staff are investigating the reason for outage.
8/25/2005 8:52:54 AM - Remedy Defined Searches Returning Incorrect Results
Description:   Due to a problem with last night's weekly update to the Remedy server, the pre-defined searches are returning incorrect results. We are currently working roll back last night's changes. ETA for resolution is approximately 9:30 AM.
Resolution:  Roll back has been completed. In order to load the changes to the Defined Searches, users must restart their Remedy User Tool application. We appreciate your patience and apologize for any inconvenience this may have caused.
8/24/2005 - NIHEXCHANGE28 reset itself this morning.
Description:   One NLM/LHC user is causing thousands of logon failures a minute on the server, which is suspect of causing the server to reset it self.
Resolution:  NLM Staff have been contacted and have been asked to aide in the reconfiguration of the one user's machine.
8/24/2005 9:31:32 AM - Outlook Web Access (OWA) degraded on NIHCESMLBX8
Description:   CIT EMIB Exchange engineers are working with Microsoft PSS to troubleshoot and correct an issue with Exchange 2003 OWA. We do not have an estimated time to repair. This issue affects NLM customers recently migrated to NIHCESMLBX8.
Resolution:  OWA 2003 issue has been resolved. NIHCESMLBX8 returned to service.
8/23/2005 8:36:36 AM - Remedy Notification Issues
Description:   The CIT Remedy system is experiencing problems with the sending of notifications and the creation of tickets from inbound emails. Currently, not all notifications are being sent properly, and inbound emails are not creating tickets successfully (though no inbound email is being lost). CIT Support Technologies Group is currently working to resolve the problem and currently do not have an estimated time to restore the services.
Resolution:  The CIT Support Technologies Group has isolated and corrected the problem. It may take as long as 60 minutes for services to return to normal operations as the system processes the inbound e-mails and sends the outbound notifications that have queued during the outage. We appreciate your patience and apologize for any inconvenience this may have caused.
8/22/2005 4:40:59 PM - NIH Portal Undergoing Maintenance
Description:   The NIH Portal will be undergoing a short maintenance on Tuesday, 8/23/05 from 9:00 PM to approximately 9:15 PM. A temporary page will be up alerting users to this fact.
Resolution:  This maintenance was completed as scheduled.
8/22/2005 1:42:39 PM - NIH Help Desk After Hours weekend service announcement
Description:   Beginning on Saturday, August 27th, 2005 at 9:30PM to Sunday, August 28th, 2005 at 6:00AM, the NIH Help Desk will experience a temporary reduction in its number of contact channels. Because the CIT Service Ticket System (ref: Hot News #: HN05795) will be unavailable for scheduled hardware upgrades, we ask that all of our customers use our phone number(s) for emergency After Hours service requests. Our web channel (Online Service Requests at http://ithelpdesk.nih.gov/) will be unavailable and our email requests (ithelpdesk@nih.gov) will be processed manually during this time. Please realize that service ticket numbers can not be issued until these requests are entered into the Ticketing System after the hardware maintenance is finished on Sunday morning. Thank you.
Resolution:  On Sunday, August 28, 2005 10:09 AM the CIT Support Technologies Group completed the Hardware Upgrade for the CIT Remedy System. Please contact the NIH Helpdesk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any problems accessing the Remedy System.
8/20/2005 7:03:30 AM - Petri having network connection problems
Description:   After maintenance work and patching Petri will not stay connected to the network
Resolution:  Issue resolvedd and Petri is accepting connections
8/19/2005 4:44:12 PM - Exchange
Description:   Users may not be able to access theit email for a brief period while we reboot the server. The IC that may have problems are as follows: NIAID
Resolution:  The server required a reboot to bring it back into a stable condition.
8/19/2005 11:13:55 AM - Power Outage - Building 10
Description:   Please be advised that building 10 is currently experiencing a partial power outage in Building 10 causing a loss of network connectivity for users in that building. There is no estimated time of repair at this time. Please contact the NIH Help Desk at (301) 496-HELP or helpdesk@nih.gov if you have any questions.
Resolution:  Please be advised that power has been restored.
8/18/2005 3:46:57 PM - Exchange
Description:   Users may not be able to access theit email for a brief period while we reboot the server. The IC that may have problems are as follows: VRC, FIC, ORS, NIAAA
Resolution:  NIHCESMLBX is backup and responding and user will be able to get back into their mail. EMIB is going to investigate into the problem.
8/16/2005 5:50:11 PM - CES Outage
Description:   NIHCESMLBX5 is in distress -- we are investigating and will reboot the server if neccessary.
Resolution:  The server has been rebooted and service restored. EMIB engineers are investigating the cause.
8/16/2005 2:27:12 PM - CIT/EMIB hosted Windows systems - emergency security work
Description:   DCSS/EMIB will be performing emergency security patching of all servers it hosts. We are moving our regular patching schedule up 3 days to better avoid another worm outbreak and to ensure NIH systems integrity. This work will begin at 8 PM. This will be a rolling outage with systems down for 15 minutes or less. [This work does not involve CES or AD]
Resolution:  This work has been completed.
8/15/2005 6:24:58 PM - NIH CES Infrastructure services emergency work
Description:   EMIB will be performing emergency security patching of all CES servers. We are reacting to concerns expressed in the Infosec community regarding vulnerabilities we were not able to guard against during last evenings and this mornings efforts. We are moving our regular patching schedule up 4 days to better avoid another outbreak and to ensure NIH systems integrity. This work will begin at 8 PM. This will be a rolling outage with systems down for 15 minutes or less.
Resolution:  Work was completed at 9:30
8/14/2005 10:09:43 PM - MS05-039 Worm
Description:   CIT has detected infections of the MS05-039 worm (aka W32/Zotob-A) across the NIH. We haven’t seen any destructive activity related to this worm, but it is replicating. Information about cleaning and patching can be found at:http://antivirus.nih.gov.
Resolution:  All known infected systems have been patched but we may see new systems that have been compromised by this worm as staff return from vacation.
8/14/2005 8:15:50 PM - CIT hosted Windows systems - emergency work
Description:   EMIB will be performing immediate emergency security patching of all servers it hosts. We are moving to protect against W32/Zotob-A specifically - but other recently announced vulnerabilities in general. This would have been done at the next window but we are concerned about a possible outbreak and want to ensure NIH systems integrity. This work begins now and will continue throughout the evening. This will be a rolling outage with systems down for 15 minutes at most.
Resolution:  Security patch work completed
8/14/2005 8:00:04 PM - NIH AD and CES Infrastructure services emergency work
Description:   EMIB will be performing immediate emergency security patching of all AD and CES servers. We are moving to protect against W32/Zotob-A specifically - but other recently announced vulnerabilities in general. This would have been done at the next window but we are concerned about a possible outbreak and want to ensure NIH systems integrity. This work begins now and will continue throughout the evening. This will be a rolling outage with systems down for 15 minutes at most.
Resolution:  Security patch work completed
8/14/2005 5:33:51 PM - NIHnet: OD/OIT Network Issues
Description:   Description: OD/OIT in buildings 1,2,10,13 is currently experiencing network problems. Engineers are working to resolve the problem as soon as possible. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  CIT/DNST engineers in cooperation with OD/OIT have resolved the network connectivity problems with OD/OIT in Buildings 1,2,10,13.
8/14/2005 5:11:49 PM - CIT EMIB Servers/Network
Description:   EMIB is experiencing issues with some servers in the machine room in Bldg 12B. Please be aware that the admins are looking into the issue and as soon as we have more information, we will send an update. Thank you for your patience and cooperation. Impact: Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Problems caused by Zotob worm. Reference Hot News HN05805.
8/13/2005 2:01:18 PM - Remedy Server Reboot for NED Integration
Description:   We will be rebooting the Remedy server within 5 minutes in an attempt to resolve some issues with NED integration. It is expected that Remedy service will return within 5 minutes after the reboot.
Resolution:  Remedy service has been returned after server reboot
8/13/2005 1:46:26 PM - BlackBerry outage for some users this afternoon
Description:   We had an SRP connection failure to RIM on 1 BES. Some users may have experienced a 5-15 minute outage at approximately 1-1:15 PM today.
Resolution:  SRP connection re-stablished approximately 1:15 PM.
8/12/2005 2:15:43 PM - NIHnet: Mailfwd Issues
Description:   Description: CIT is getting reports of user problems with mailfwd. CIT/DNST engineers are investigating the issue. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The problem with mailfwd has been resolved at this time, however, it may take up to one hour for the DNS changes to propagate. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
8/12/2005 1:03:02 PM - Emergency Maintenance on NIH CES BES servers
Description:   On 8/12 at 9 PM CES Admins will perform required maintenance on the NIH CES Blackberry servers. The expected downtime is 15 minutes.
Resolution:  Work was completed.
8/12/2005 8:34:31 AM - CES Blackberry - Intermittent connectivity
Description:   Some NIH BB users may be experiencing connectivity problems. NIH CES Admins are activity working this. We are unable to be more specific on which users are affected.
Resolution:  This issue is resolved
8/11/2005 3:39:50 PM - Hardware Upgrade for the CIT Remedy System
Description:   Starting at 10:00 PM on August 27th, 2005, the Support Technologies Group will be taking down the Remedy system to conduct an upgrade of the Hardware. This outage is expected to last until 6:00 AM on August 28th, 2005. During this time access to the Remedy system will not be available. This will affect all client, API and web access to the system to include the CIT Training Registration, CIT Service Ticket, CIT Hot News, and other IC specific applications. We appreciate your patience as we work to improve these services. If you have any questions about this activity please contact the NIH Helpdesk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov.
Resolution:  CIT Support Technologies Group has completed the Hardware Upgrade for the CIT Remedy System. Please contact the NIH Helpdesk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any problems accessing the Remedy System.
8/11/2005 8:50:36 AM - NIHnet: Building 37
Description:   Description: We are currently experiencing a loss of connectivity to the NIHnet Devices in Building 37 due to a loss of power in the LAN closet. NIH Building Maintenance has been notified. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  CIT/DNST engineers have completed their work and network connectivity has been restored for devices in B37.
8/10/2005 9:50:13 AM - NIHnet Bldg 35 Network Connectivity Issues
Description:   Users are experiencing network connectivity issues in Bldg 35. CIT/DNST Engineers are currently investigating this issue.
Resolution:  The bldg 35 network connectivity issues have been resolved.
8/9/2005 11:35:02 AM - CES BlackBerry Server maintenance - Wedn 8-10-05 10 PM
Description:   We will be installing Windows 2003 Service Pack 1 on 2 production BlackBerry servers (NIHBESBLADE, NIHBESBLADE2). Downtime should be minimal (10 minutes or so) while the service pack installs and the server reboots.
Resolution:  Patches successfully applied to 2 BES's.
8/8/2005 8:22:11 AM - HHS Mail Connectiviey
Description:   HHS mail connectivity is affected due to HHS to NIH trust problems. NIH Services are not affected. HHS is blocking ports needed by Exchange and AD to establish connectivity to the mailboxes. HHS Help Desk has been contacted.
Resolution:  Resolved by HHS
8/5/2005 4:29:24 PM - ITAS Wireless service
Description:   Blackberry access to the ITAS Wireless service was inadvertently changed by a recent CES Blackberry server upgrade. Users may now be prompted to accept an "untrusted certificate" when logging onto the service. If the message, "The server certificate is not trusted by your MDS..." appears, select "Continue" to proceed to the NIH Login page. Users may have to select "continue" more than once. Enter your credentials as you would normally, you will be able to login to ITAS. NIH CES Wireless Admins are working on the resolution. If you have problem accessing Wireless ITAS, please contact Helpdesk at 301-594-6248 or via email at nihhelpdesk@mail.nih.gov.
Resolution:  CIT staff continues to work on this issue.
8/5/2005 2:12:55 PM - OD Exchange Server Maintenance
Description:   Apply MS Exchange STORE.EXE hot fix to the email server for OD, Nihcesmlbx2, to help alleviate the calendaring issues. This activity will require a reboot of the server. There will be a 10 minute interruption in the email service on this server starting at 12:10 AM on the morning of August 7th. Thank you for your patience as we work to restore this server to 100% stability.
Resolution:  STORE.EXE Hotfix sucessfully applied to OD email server. Server was rebooted and email has been returned to service. Please contact the Help Desk if you experience any issues with your NIH email.
8/4/2005 3:05:08 PM - OD Exchange Server Maintenance
Description:   Apply MS Exchange STORE.EXE hot fix to the email server for OD, Nihcesmlbx2, to help alleviate the calendaring issues. This activity will require a reboot of the server. There will be a 10 minute interruption in the email service on this server starting at 10PM this evening. Thank you for your patience as we work to restore this server to 100% stability.
Resolution:  Please be aware that the scheduled STORE.EXE hot fix install for the OD mailbox server, Nihcesmlbx2, will NOT be occurring. This action is due to problems that CIT found with the hot fix, so it will no be applied.
8/3/2005 8:18:01 PM - BlackBerry Server maintenance 8-3-2005 10 PM
Description:   Blackberry server maintenance on the SQL database tonight. All BES's must be taken offline in order to upgrade the database. 15-20 minute outage is expected on all devices.
Resolution:  BES database successfully upgraded to BES 4.0 Service Pack 1. Everything looks fine... Downtime was 10-15 minutes or so...
8/3/2005 4:00:45 PM - Exchange Emergency Maintenance
Description:   Rebooting Nihcesmlbx to restore email services to OD community
Resolution:  Server rebooted and Exchange Email Service returned in full to customers
8/3/2005 1:53:40 PM - ITAS Wireless and J2EE unavailable from 8-8:30pm 08/03/05
Description:   Wireless and J2EE ITAS will be unavailable this evening, Aug 3, from 8:00 PM - 8:30 PM for a security subsystem upgrade. Regular ITAS will not be affected. If you have any questions, please contact the NIH Help Desk at 301-594-6248 or via email at ITHelpDesk@nih.gov We apologize for any inconvenience this may cause you. Very Best Regards, ITAS Technical Team, DECA/CIT
Resolution:  The wireless and J2EE ITAS upgrade has been completed. Per the ITAS Technical Team, DECA/CIT, . We do apologize for the belated notification. Thank you
8/3/2005 10:50:22 AM - BlackBerry service degradation
Description:   A number of customers serviced by the NIH CES Blackberry service experienced an outage this morning that may have begun at 10:30 PM last night. This outage affected approx 300 users as best we can tell. This problem was compounded by a nationwide RIM service outage.
Resolution:  The affected service has been reset and stablized.
8/2/2005 10:53:02 AM - BlackBerry Server Maintenance Tuesday 8-2-05 10 PM
Description:   We will uprgaded the BES SQL server tonight. In oder to upgrade this server, all existing BES's must be stopped to disconnect from the database. Downtime expected to be approximately 15 minutes.
Resolution:  Sequel patch run without incident. Servers were back up within 20 minutes.
8/2/2005 8:54:55 AM - NIHEXCHANGE28 reboot
Description:   NIHEXCHANGE28 is being rebooted. CIT EMIB Exchange engineers are investigating. The reboot will impact NLM users.
Resolution:  NIHEXCHANGE28 ha been returned to service. CIT EMIB Exchange engineers continue to investigate.
8/1/2005 2:25:27 PM - BlackBerry Server Maintenance 10PM 8-1-05
Description:   We will be performing BlackBerry server maintenance at 10 PM this evening (Monday 8-1-05). Some users may experience intermittent messaging capability for approximately 15 minutes.
Resolution:  Maintenance done, all BES's returned to normal operation.
7/29/2005 5:11:53 PM - NIHnet: Remote Access Issues
Description:   Description: There is currently a problem with remote access to NIHnet via Dialup and VPN. CIT/DNST engineers are working to resolve the issue. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Remote access to NIHnet via Parachute and VPN has been fully restored at this time. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
7/29/2005 2:24:23 PM - EHRP Scheduled Maintenance
Description:   The EHRP system will be unavailable for maintenance from 10:00 PM on Friday, July 29th until 9:00 AM on Saturday morning, July 30th. If you have any questions, please contact the Help Desk by any of the following methods: Phone: 301-496-EHRP (301-496-3477) Email: helpdesk@nih.gov Website:
Resolution:  EHRP maintenance was completed successfully.
7/27/2005 5:44:37 PM - ITAS server in distress.
Description:   The ITAS Techncial Team is reviewing the problem.
Resolution:  ITAS Services have been restored.
7/26/2005 4:31:44 PM - NIHCESMLBX undergoing emergency reboot 11/27/05 @ 10PM.
Description:   The CES staff need to reconfigure that antivirus product on this server, which requires a reboot. An outage of approximately 10 minutes is expected.
Resolution:  Work performed succesfully and the server was fully operational at 10:10PM.
7/26/2005 4:25:11 PM - Help Desk Ticketing System (Remedy) email service interruption
Description:   At 10:00 PM on Thursday, July 28, the NIH Help Desk will be shutting down all Remedy email services in order to perform mailbox migration to an Exchange 2003 server. The duration of the outage is not expected to exceed 30 minutes. Both Inbound and Outbound processes will be affected. Inbound Emails: Emails sent to NIH Help Desk, IT Help Desk and TASC should queue up for processing when the migration is completed, but there is a slight possibility of bounced mail. Senders whose email is bounced will receive an email notification from the mail server and should wait 30 minutes before re-sending. Alternatively they may submit tickets at http://ithelpdesk.nih.gov at any time during the service interruption. Outbound Notifications: Outbound notifications will queue up on the Remedy server. Once the system is restarted, they will be delivered. The following notification addresses will be affected: NIH Help Desk Notifications (this includes Hot News) CIT Training CIT Knowledgebase DCS Webscans All queued email notifications will be processed by Remedy once the system is restarted. All other Remedy processing will be unaffected by this outage.
Resolution:  The Help Desk Ticketing System email services have been restored. We appreciate your patience during this period and apologize for any inconvenience this may have caused.
7/25/2005 9:41:56 AM - Remedy Latency
Description:   The Remedy system that supports the Service Tickets, Hot News, and Customer Database is currently experiencing latency issues. The System Administrators have been alerted to the latency and are working to correct the problem. The it is estimated that the correction will take five minutes or less to conduct. We appreciate your patience during this period and apologize for any inconvenience this may cause. Support Technologies Group Division of Customer Support, CIT
Resolution:  The Remedy server has been restarted and service have been restored. We appreciate your patience during this period and apologize for any inconvenience this may have caused.
7/23/2005 7:51:01 AM - NIHCESMLBX4
Description:   NIHCESMLBX4 is having issue and EMIB staffs are currently working to resolve the problem.
Resolution:  The problem has been resolved.
7/22/2005 6:07:58 PM - IMS is currently down.
Description:   Ray Danner indicated that the IMS is currently down and that the shift supervisor (Jeannie) is aware of the problem and working to resolve the issue. There is no ETA at this time.
Resolution:  This update is to inform that, connectivity to IMS had been restored as of 7:30 PM on Friday, 7/22/05.
7/22/2005 3:27:21 PM - NIHnet Maintenance: Fiber Relocation
Description:   Date: Saturday, 8/6/2005 Time: 3:00 am - 9:00 am Description: In preparation for upcoming construction in Building 10, CIT/DNST will be working with our fiber provider to move select fiber connections that connect some remote sites to the Bethesda Campus. Impact: Due to the redundant nature of our off-campus network connectivity, there is very little impact expected as a result of this maintenance. The administrators of those networks that will be impacted have been notified. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Maintenance complete
7/22/2005 7:47:43 AM - NIHCESMLBX3 Emergency Maintenance
Description:   Experiencing difficulties with the Exchance Server EMIB rebooting server
Resolution:  Server is now back online Problem resolved
7/21/2005 11:53:14 AM - NIHnet: Network Connectivity Issues in Rockville
Description:   Currently NIHnet users at the following locations are experiencing a loss of network connectivity: Building 5 Research Court Medical Center Drive Key West CIT/DNST engineers are working with the fiber provider to investigate the issue. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The fiber provider repaired the fiber cut and all network connectivity was restored at 8:20 p.m.
7/21/2005 7:25:12 AM - Problems with NIHCESMLBX4
Description:   NIHCESMLBX4 is experiencing problems. EMIB staff are investigating.
Resolution:  Server was restarted. Services are operating as normal.
7/18/2005 4:01:01 PM - CIT Customer Support reports server maintenance
Description:   The CIT Customer Support reports server (CITDCSRPT) will be rebooted this evening at 10 PM Eastern time. This will affect any program accessing data from the server. It will also affect dynamic reports from the CIT Customer Support Reports website (http://ITHelpdesk.nih.gov/reports or http://support.nih.gov/reports). The outage is expected to last less than 30 minutes.
Resolution:  Maintenance complete.
7/18/2005 3:15:44 PM - ITAS Email server migration.
Description:   Dear ITAS users, CIT will be migrating the email account ITAS uses for email notifications tonight at 10PM to a new Microsoft Exchange server. A brief outage in email notifications from ITAS will likely be experienced. We will monitor the progress of the migration and correct any email notification issues promptly if any should arise. Thanks for you patience! Very Best Regards, ITAS Technical Team, DECA/CIT
Resolution:  Email migration complete.
7/15/2005 3:20:33 PM - NIHnet Maintenance:
Description:   Date: July 21, 2005 Time: 10:00 - 10:30 am Description: CIT/DNST will be replacing one of the three time servers (ntp1.nih.gov) that provide NTP service to the NIHnet community. Impact: Due to the redundant nature of the time service design, there should be no impact to anyone on NIHnet as ntp2.nih.gov and ntp3.nih.gov will continue to provide a time reference. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  This maintenance was completed successfully.
7/14/2005 2:42:47 PM - CIT Customer Support reports server maintenance
Description:   The CIT Customer Support reports server (CITDCSRPT) will be rebooted this evening at 10 PM Eastern time. This will affect any program accessing data from the server. It will also affect dynamic reports from the CIT Customer Support Reports website (http://ITHelpdesk.nih.gov/reports or http://support.nih.gov/reports). The outage is expected to last less than 30 minutes.
Resolution:  maintenance was rescheduled.
7/5/2005 11:25:47 AM - Remedy Latency
Description:   The Remedy system that supports the Service Tickets, Hot News, and Customer Database is currently experiencing latency issues. The System Administrators have been alerted to the latency and are working to correct the problem. The it is estimated that the correction will take five minutes or less to conduct. We appreciate your patience during this period and apologize for any inconvenience this may cause. Support Technologies Group Division of Customer Support, CIT
Resolution:  The problem has been corrected and service has been restored.
7/2/2005 12:26:21 AM - NIHnet: Power Outage at 5635 Fishers Lane
Description:   Description: There is currently a power outage at 5635 Fishers Lane, causing a loss of network connectivity for networks connected via that site. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Power and network connectivity were restored at 9:30 a.m.
7/2/2005 12:18:50 AM - NIHnet: Power Outage at 6011 EB
Description:   Description: Due to the thunderstorms that went through the area tonight, 6011 Executive Boulevard has lost power and therefore lost network connectivity. OD staff will restore service Saturday morning, once power is restored. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Power and network connectivity have been restored to 6011 Executive Boulevard. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
7/1/2005 9:09:06 PM - Emergency service reset of NIHEXCHANGE12
Description:   Emergency service reset of NIHEXCHANGE12 At 10 pm there will be a 10 minute service outage for NIHEXCHNAGE12. The following IC's are affected. NHGRI, FIC, NIAAA, NIA IRP, NIDCD, NIAMS, NINR.
Resolution:  Reset completed
6/30/2005 10:36:19 AM - NIHnet Maintenance: Building 10 Backbone Device Relocation
Description:   Date: Saturday, 7/9/2005 Time: 7:00 a.m. - 7:00 p.m. Description: In preparation for upcoming construction in Building 10, CIT/DNST engineers will be moving several NIHnet backbone devices to a new location in Building 10. Impact: Due to the redundant nature of the backbone network, very minimal impact is expected. There may be very brief interruptions in network connectivity during this maintenance window as the network fails back and forth to the redundant path and the NIHnet routing converges. This loss of connectivity is expected to be very brief with minimum packet loss and data retransmission. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The Building 10 backbone device relocation maintenance has been successfully completed. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
6/30/2005 8:55:09 AM - Remedy Latency
Description:   The Remedy system that supports the Service Tickets, Hot News, and Customer Database is currently experiencing latency issues. The System Administrators have been alerted to the latency and are working to correct the problem. The it is estimated that the correction will take five minutes or less to conduct. We appreciate your patience during this period and apologize for any inconvenience this may cause. Support Technologies Group Division of Customer Support, CIT
Resolution:  The problem has been corrected and service has been restored. If you are still experiencing problems accessing the Remedy system, please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov.
6/29/2005 12:32:47 PM - NIHEXCHANGE13: Not-Responding
Description:   NIHEXCHANGE13 is currently experiencing issues. CIT is aware of the problem and is working toward a resolution.
Resolution:  Exchange13 is fully operational IC-NCI
6/28/2005 9:46:57 AM - The ITAS J2EE Pilot version is still experiencing problems.
Description:   The ITAS J2EE Pilot version is still experiencing problems. Pilot users have been temporarily moved back to the older version of ITAS web until the problem has been corrected, therefore no users are without primary ITAS service at this time. The problem is also effects ITAS BlackBerry Wireless, which is currently unavailable. The ITAS Technical Team is continuing to work on the problem. Thanks for your continued patience.
Resolution:  All ITAS Services have been restored.
6/28/2005 8:20:39 AM - ITAS J2EE Pilot Unavailable
Description:   Good morning, J2EEE ITAS Pilot is experiencing a problem. We apologize for any inconvenience this may cause and appreciate your patience while we work to resolve the problem. Another e-mail will be sent when J2EE ITAS is available.
Resolution:  Closing this ticket and opening another as I cannot update it.
6/28/2005 7:07:31 AM - NIHnet: Executive Boulevard Network Issues
Description:   Description: One of the modules in the router that services networks that go through Executive Boulevard is currently failing. This is causing a loss of network connectivity for some networks along Executive Boulevard. CIT/DNST engineers are en route to investigate and replace the failing module. Estimated Time of Repair (ETR): There is no ETR at this time. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  The faulty router module has been replaced and network connectivity has been restored..
6/27/2005 9:51:03 AM - NIHnet: Building 10 Network Issues
Description:   Description: We are currently experiencing problems with the routers that service the local networks in Building 10. CIT/DNST engineers are on-site investigating the issue. Estimated Time of Repair: There is no estimated time of repair at this time.
Resolution:  Network connectivity has been restored to all local networks in and around Building 10. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any further issues.
6/26/2005 8:27:45 PM - NIHnet: Building 10 Connectivity Problems
Description:   Description: We are currently experiencing problems with one of the routers that services some of the local networks in Building 10. CIT/DNST engineers are en route to investigate the issue. Estimated Time of Repair (ETR): There is no ETR at this time. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
Resolution:  Network connectivity to the local networks in Building 10 has been restored at this time.
6/22/2005 1:31:54 PM - Some users may experience email delays with BlackBerry
Description:   Half of NIH's BlackBerry servers have lost remote connection to RIM. This will result in no email being transferred to devices. RIM is aware of the issue, and working to resolve it, but has no estimated time of repair. RIM ticket for NIH is RQST00001996707
Resolution:  SRP connections returned 2:03 PM SRP's returned approximately 3:15
6/20/2005 9:31:48 AM - Intermittent Public IM connectivity.
Description:   The service that provides Public IM protection to the NIH is experiencing intermittent connectivity with internal clients. NIH/CIT/DCSS/EMIB engineers are working with the vendor to resolve this situation ASAP. No ETA on a resolution at this time.
Resolution:  NIH/CIT/DCSS/EMIB engineers located a system incompatibility and resolved it at 6PM yesterday. Public IM usage has been normal since that time.
6/18/2005 5:58:46 PM - OD/OIT Network Maintenance Extended
Description:   Please note the OD\OIT maintenance in Bldg 1, 2, 10 & 13 is being extended due to technical difficulties. No ETR at this time.
Resolution:  Maintenance Completed
6/17/2005 4:47:44 PM - ITAS BlackBerry Wireless service unavailable
Description:   The ITAS BlackBerry Wireless service is currently unavailable. Technicians are reviewing the problem and hope to restore service soon. All other ITAS services are available.
Resolution:  The problem with ITAS Wireless has now been corrected.
6/17/2005 2:00:58 PM - ITAS maintenance Tuesday evening, June 21, 2005 6-6:30PM.
Description:   Dear ITAS User, ITAS is scheduled for maintenance this Tuesday evening, June 21, 2005 from 6:00PM - 6:30PM. During this time ITAS will not be available. We apologize for any inconvenience this may cause you. If you have any questions, please contact Helpdesk at 301-594-6248 or via email at nihhelpdesk@mail.nih.gov <<<<>>>> As always, we appreciate your patience. Very Best Regards, ITAS Technical Team, DECA/CIT
Resolution:  ITAS maintenance is now complete.
6/17/2005 10:11:43 AM - Degradation of RIM BlackBerry service
Description:   Trouble Ticket Number: BB000029645 Severity: Medium Level Description of Issue: Degradation of RIM BlackBerry service. Location of Issue: National Start Time: 06/17/05 (The outage began this morning. We are awaiting confirmation of the start time.) Symptoms you MAY experience: Customers may be unable to send or receive messages. Estimated time of repair: Unknown Technicians are working towards resolution as quickly as possible. We are sorry for any inconvenience that this may cause. Thank you for your patience as we work towards resolving this issue. If you have any questions, please contact the Enterprise Data Help Desk for more information.
Resolution:  Traffic is flowing. Delays as queued messages are being delivered are expected...
6/16/2005 3:55:53 PM - NIHnet: Internet Connectivity Issues
Description:   Description: Some users are reporting that they are experiencing problems accessing the Internet. CIT/DNST engineers are investigating the issue.
Resolution:  All Internet connectivity has been restored. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
6/16/2005 12:46:28 PM - NIHnet: Off-Campus Network Issues
Description:   Description: Users located in the Democracy Complex, the Gateway Building and B5 Research Court are experiencing issues with network connectivity. CIT/DNST engineers are investigating the problem.
Resolution:  The network issues have been resolved and all network connectivity has been restored. Please contact the NIH Help Desk at 301-496-HELP, 301-496-8294 (TTY) or helpdesk@nih.gov if you have any questions.
6/15/2005 3:03:16 AM - NIHnet: Rockledge Power Issue
Description:   Due to a thunderstorm that moved through the area, Rockledge is experiencing a total power outage and is on emergency lighting only. Pepco has been contacted. Please contact the NIH Help Desk at 301-496-HELP or helpdesk@nih.gov if you have any questions.
Resolution:  The power outage at Rockledge is over and all power has been restored.
6/14/2005 2:10:32 PM - ITAS wireless will be unavailable 6/15/2005 4:00pm - 6:00pm
Description:   ITAS wireless service will be temporarily unavailable this Wednesday, June 15, from 4:00pm - 6:00pm for a security upgrade. The normal ITAS Web service https://itas.nih.gov will be uneffected. Sorry for the inconvenience this may cause you. ITAS Technical Team, DECA/CIT
Resolution:  The problem with ITAS Wirless service has been corrected and all ITAS services are now functioning normally.
6/13/2005 9:43:33 AM - NIHnet: Executive Boulevard and Rockledge
Description:   Description: Users on Executive Boulevard and Rockledge are currently experiencing a loss of connectivity. DNST engineers are investigating the issue. Please contact the NIH Help Desk at 301-496-HELP or helpdesk@nih.gov if you have any questions.
Resolution:  Network connectivity has been restored to Executive Boulevard and Rockledge. CIT/DNST engineers will continue to investigate the root cause of the problem.
6/11/2005 9:17:41 AM - BlackBerry servers down- srp connection to RIM down
Description:   NIH network maintenance work this morning cut a fiber line. this is afffecting DNS. This will probably affect many things, not just BlackBerry... but the BlackBerry servers are unable to resolve the connection to RIM due to this. If they can't make an SRP connection to RIM, they can't rout user mail to devices. Reference Hot News HN05719
Resolution:  SRP connections restored to RIM. Email flowing to BlackBerry devices. DNS queries to RIM SRP host are successful.
6/11/2005 9:15:40 AM - NIHnet: Internet Connectivity
Description:   While performing a fiber maintenance this morning, our fiber provider inadvertently cut the wrong fiber which has caused a loss of connectivity to the Internet. They are working to restore service as soon as possible.
Resolution:  The fiber has been repaired and all network connectivity has been restored.
6/10/2005 8:51:02 AM - Nihexchange Maintenance
Description:   Nihexchange27 will be rebooted and unavailable from 9 to 9:15PM this evening. EMIB is taking this action to correct a connection time-out issue currently experienced by NIA POP3 clients. EMIB is ware of the source of this issue and is working along with Microsoft to resolve this bug in their software. However, until such time as the problem is fixed, we must reboot the server to clear the close_wait sessions.
Resolution:  Reboot of Nihexchange27 successfully cleared close_wait sessions. Exchange server, services, queues, server performance and event logs are fine. Also, able to send/receive email on server.
6/8/2005 1:22:48 PM - IMS is Inaccessible
Description:   The NIH Help Desk has just been informed that IMS (most commonly used to access the ADB system) is currently unavailable. The service providers are aware of the problem and are currently working on a resolution. This HN item will be update as additional information becomes available. Thank you for you patience
Resolution:  IMS is now accessible. Thanks for your patience and understanding
6/6/2005 12:06:07 PM - NIHnet: The NS2.nih.gov server is down
Description:   The NS2.nih.gov server is down. DNST engineers are working to resolve this issue, but do not have an Estimated Time of Repair, at this time. Please contact the NIH Help Desk at 301-496-HELP or helpdesk@nih.gov if you have any questions.
Resolution:  The NS2.nih.gov server is back up and all connectivity has been restored.
6/2/2005 3:43:13 PM - Portal Collaboration Server Experiencing Difficulties
Description:   The NIH Help Desk has been informed that the NIH Portal Collaboration server is currently experiencing difficulties. The Portal Admin is diligently working to resolve this issue. There is no ETR at this time. This Hot New will be updated as additional information becomes available Thank you for your patience
Resolution:  At approximately 5:15 PM Portal collaboration service was restored. Thank you for you patience
6/1/2005 3:49:05 PM - Public IM Service Outage
Description:   At 9PM on 6/3/05, CIT engineers will be performing maintenance on the system that provides public IM connectivity to AOL, Yahoo, and MSN. Internal NIH user access to these public services will be offline for approximately 30 minutes. No user action is required, and connectivity will be automatically restored when the maintenance is complete.
Resolution:  Maintenance was successfully completed within the established service, and public IM connectivity was fully restored.
6/1/2005 1:53:57 PM - New HHS security certificate for ITAS.
Description:   The ITAS Technical team will be deploying a new HHS security certificate for ITAS web later this afternoon. Although no downtime is expected, users could experience warnings about changes to the ITAS web certificate in their web browsers when using ITAS. If users receive a message in their browsers asking if they wish to accept the new certificate, they should. If this warning recurs, users should ensure that the HHS certificate authority is properly configured in the security settings of their web browsers. Very Best Regards, CIT/DECA ITAS Technical Team
Resolution:  The certificate deployment is now complete and all ITAS systems are operating normally.
5/30/2005 2:02:01 PM - NIHnet: VPN Problems
Description:   Users are currently reporting problems with VPN access to NIHnet. DNST Engineers are currently investigating the issue. There is no ETR at this time.
Resolution:  Per CIT DNST engineers, the VPN service has been restored. Please report any further problems to the NIH Help Desk -- 301-496-4357 or ithelpdesk@nih.gov
5/28/2005 1:58:15 PM - OD Exec Blvd Network Unavailable
Description:   The NIH Help Desk was informed by OD network engineers that the network for OD Exec Blvd is currently unavailable. OD engineers are working on the problem. The ETTR is approx 1 hour.
Resolution:  The OD Executive Boulevard network issues have been resolved and all network connectivity has been restored.
5/27/2005 7:33:21 AM - NIHEXCHANGE19 not authenticating mailboxes
Description:   CES19 is not allowing users' to authenticate properly to mailboxes via POP, IMAP, MAPI or HTML. EMIB is currently working to get server operational.
Resolution:  CES19 is fully functional again.
5/25/2005 7:17:09 AM - OD Network
Description:   OD Network is experiencing connectivity problem on all off-campus buildings which includes Democracy Blv. , Executive and Rockledge. Techs on site are working on the problem. There is no estimated resolution time when updates are recieved the Hot News will be updated.
Resolution:  Connectivity issues have been resolved. We were unable to recreate the issue with the problem at the bunker last evening.
5/21/2005 8:07:06 AM - NIHnet: Power Issues in Building 13
Description:   Description: Currently there are power problems in Building 13 causing a loss of network connectivity for networks that connect through that building. They have switched over to generator power and the Division of Engineering Services personnel are onsite working on the issue. Estimated Time of Repair: There is no estimated time of repair. Please contact the NIH Help Desk at 301-496-HELP or helpdesk@nih.gov if you have any questions.
Resolution:  The power issues in Building 13 have been resolved and all network connectivity has been restored.
5/20/2005 11:30:03 AM - Resolved: NIHCESMLBX mail server in distress
Description:   Resolved: NIHCESMLBX mail server in distress. The latency/connectivity issue on NIHCESMLBX has been resolved. There were some disk related latency alerts that have cleared up, therefore restoring access to the DCS and DCSS databases. Databases that DCS and DCSS on NIHCESMLBX are currently in distress EMIB are currently looking into the problem. Other databases on NIHCESMLBX are working successfully.
Resolution:  The latency/connectivity issue on NIHCESMLBX has been resolved. There were some disk related latency alerts that have cleared up, therefore restoring access to the DCS and DCSS databases.
5/20/2005 10:09:32 AM - NIHnet: Print Share Service is Unavailable
Description:   The Print Share Service is currently unavailable. DNST Engineers are currently working to resolve this issue. Please contact the NIH Help Desk at 301-496-HELP or helpdesk@nih.gov if you have any questions.
Resolution:  The Print Share Service is now available, all problems have been resolved.
5/18/2005 2:52:09 PM - CIT Customer Support Report Server Data Problems
Description:   The CIT Customer Support Report Server (CITDCSRPT) does not currently have ticketing system data available due to a failure in last night's mirroring job. We are currently copying the data across from another mirrored server, though we do not have an ETA for return to service due to the volume of data involved. We apologize for the inconvenience and appreciate your patience as we work to resolve this issue.
Resolution:  The wayward ticketing system data has been coped to the server and service has been restored. Thanks again for your patience.
5/17/2005 4:48:32 PM - ITAS Web Server is down
Description:   The ITAS Web Server is non-responsive. We are working to resolve the problem, and hope to resume normal ITAS operations in a few minutes. Thanks for your patience. CIT/DECA ITAS Technical Suport Team.
Resolution:  The ITAS Web server is now back up and all ITAS systems are operating normally. We appologize for this inconvenience.
5/16/2005 9:25:42 AM - OWA
Description:   Users are currently experiencing problems with login onto OWA. EMIB is aware of the problem and are looking into remedying the situation ASAP.
Resolution:  OWA issues have been resolved, the problem occurred due to an issues with Nihcesgc. Called the helpdesk to verify that users were able to get back into OWA and they can.
5/12/2005 2:18:10 PM - EOS.NIH.GOV experiencing system problems
Description:   EOS.NIH.GOV is current experiencing a disk issue and is non-responsive. We are investigating; no estimated time to resolution. Affected applications include NED, EGG, CRISP, and ACF databases.
Resolution:  EOS.NIH.GOV required a system reboot to resolve the situation. It was brought back into service at approximately 3pm and database are currently being brought back up. We apologize for any inconvenience.
5/12/2005 9:01:45 AM - CIT Customer Support Report Server Work 5/14/2005 7:00 PM
Description:   The CIT Customer Support Report Server (CITDCSRPT) will be taken out of service for a brief period at 7:00 PM on Saturday 5/14/2005 so that we may bring a new server online. We do not expect that the outage will last longer than one hour, and we anticipate that the actual service outage will likely not be more than a few minutes. During this time, customers will be unable to use ODBC connections to the server to run custom reports, and any dynamic reports run from the CIT Customer Support Website will be queued until the server is restored to service. We appreciate your patience while we work to maintain and enhance the quality of our reporting services.
Resolution:  New server is in place and working.
5/11/2005 11:20:42 AM - NIHEXCHANGE11
Description:   Nihexchange11 was in distress this morning. We got a phone call from the Helpdesk at 10:45 am informing EMIB that pop/imap users could not receive email.
Resolution:  So by stopping the MTA for 5 mins to all the paging to settle, than restarting the MTA the exchange services were restored in full
5/10/2005 6:21:45 PM - NIHEXCHANGE11
Description:   This afternoon, the NIHEXCHANGE11 server showed symptoms of distress. eMIB(formerly BOSB) immediately responded to the issue and was able to return the email server to full service within 45 minutes. Estimated Time of Repair: Complete Please contact the NIH Help Desk at 301-496-HELP or helpdesk@nih.gov if you have any questions.
Resolution:  Power to server was recycled and Exchange functionality was fully restored.
5/9/2005 1:29:36 PM - Resolved-NIHEXCHANGE18.nih.gov connectivity problems resolved
Description:   The sever was rebooted and is back online. CES18 experienced memory/network issues and went offline at 11:50.
Resolution:  BOSB engineers brought system back up. The sever was rebooted and is back online. It's operational as of 12:23
5/7/2005 7:55:54 AM - ITAS application update Tuesday evening.
Description:   Dear ITAS User, The ITAS Technical Team will be performing a routine update to the ITAS application on Tuesday May 9th between 6PM and 6:30PM. During the update the ITAS application will not be available. We appreciate your patience with this maintenance, as we strive to improve ITAS. Regards, CIT/DECA ITAS Technical Support Team.
Resolution:  The scheduled ITAS update is now complete and all ITAS services are again available.
5/1/2005 2:00:48 PM - ITAS Time card problem
Description:   Dear ITAS Users, Due to unexpected problems with the pay period close process for pay period 10, timecards have not yet been created for pay period 11 (the current pay period). We are working to resolved the problem and anticipate that timecards will be available by noon today. Until timecards are available, users may have difficultiy with some operations in ITAS. We will update this hot news when the problem has been resolved. Many thanks for your patience. CIT/DECA ITAS Technical Support Team. " Thanks!
Resolution:  ITAS Time card problem has been resolved.
4/30/2005 2:11:29 PM - NIHnet: Power Issues in Democracy I and II
Description:   Currently there is a power outage affecting Democracy I and II, causing a loss of network connectivity for Democracy users. PEPCO is on site at this time.
Resolution:  The power outage affecting Democracy I and II has been resolved and all network connectivity has been restored.
4/26/2005 1:22:47 PM - Difficulty Accessing http://my.nih.gov
Description:   Some customers are currently having trouble accessing the NIH Portal and we are working on a solution. During this brief period when the NIH Portal is unavailable, please try using these direct-links to access the applications: ITAS https://itas.nih.gov NBS Travel (Production) https://nbrssprod.cit.nih.gov:8920/servlet/com.GelcoSSO.GelcoSignIn nVision https://nvisionreports.nih.gov/start
Resolution:  The NIH Portal is now available.
4/26/2005 10:19:12 AM - NIH Intramural Database (NIDB) is down
Description:   There is no connectivity to the NIDB server. NIDB staff is investigating now
Resolution:  Hardware issues
4/26/2005 10:01:50 AM - Video Service Problems
Description:   Videocast video playback performance degraded. IP Video Conferencing is also affected.
Resolution:  The problem was found & corrected. Videocast video playback & IP Video Conferencing performance has been returned to normal.
4/25/2005 1:11:18 PM - DNST Video Services are unavailable
Description:   DNST Video Service team reports all video services including Videocast, IP Video conferencing services are unavailable at this time. No estimated time of return to service is available and updates will be posted when they are received.
Resolution:  CIT DNST Videocast reports that all services have been restored.
4/18/2005 5:47:36 PM - NIH Portal Maintenance
Description:   Between 8-8:30 this evening (4/18/2005), the NIH Portal must undergo a critical patch. During this time, the emergency page (http://temp.my.nih.gov) will be posted whenever a customer attempts to access the Portal. Service should remain uninterrupted for enterprise applications. Normal service will be restored no later than 9pm.
Resolution:  The NIH Portal was successfully patched.
4/18/2005 10:08:29 AM - Lost ODBC TCP/IP connectivity to NIHSQLPROD
Description:   DCSS DBAs working on ODBC connectivity via TCP/IP pipies
Resolution:  Server rebooted after registry changes.
4/17/2005 9:45:16 AM - CIT ITAS Users unable to login to ITAS
Description:   CIT Users of ITAS are experienceing problems logging in. We are reviewing the problem and hope to have the issue resolved shortly.
Resolution:  ITAS Service for CIT users has now been restored. All ITAS services are functioning normally.
4/16/2005 10:28:36 AM - NIHnet: Latency Issues
Description:   Currently there is extreme latency for some networks and a loss of connectivity for other local networks on the NIHnet. CIT/DNST engineers are on-site working on the issue.
Resolution:  All network connectivity has been restored at this time. Please contact the NIH Help Desk at 301-496-HELP or helpdesk@nih.gov if you have any further issues.
4/15/2005 4:39:55 PM - IP Video calls may be unsuccessful
Description:   Description: Customers who are attempting to make IP Video calls may be unsuccessful. CIT DNST is currently working to resolve the issue. Please contact the NIH Help Desk at 301-496-HELP or helpdesk@nih.gov if you have any questions.
Resolution:  The problem was found & corrected. Videocast video playback & IP Video Conferencing performance has been returned to normal.
4/11/2005 4:46:12 PM - ITAS upgrade Wednesday evening, April 13 from 6:00PM - 6:30PM
Description:   ITAS is scheduled for upgrade Wednesday evening, April 13 from 6:00PM - 6:30PM in preparation of the DFAS rollout scheduled by DHHS on April 17. During the upgrade, ITAS will be unavailable. If you have any questions, please contact Helpdesk at 301-594-6248 or via email at nihhelpdesk@mail.nih.gov <<<<<<>>>>>> We apologize for any inconvenience this may cause you. Thank you for your patience. Very Best Regards, ITAS Technical Team, DECA/CIT
Resolution:  The scheduled upgrade to ITAS Web is now complete, and all ITAS services are fully functional. Thank you for your patience. Very Best Regards, ITAS Technical Team, DECA/CIT
4/11/2005 2:01:08 PM - Helix Systems Downtime, Wednesday April 13, 6 PM-Midnight
Description:   Helix, Nimbus, Quasar and the Biowulf cluster will be unavailable to users on Wednesday April 13 starting at 6 PM. We anticipate having the systems available to users by midnight.
Resolution:  System Maintenance has successfully concluded. All systems returned to users by 12:30 AM.
4/11/2005 10:03:33 AM - NIHnet: Listserv Server
Description:   We are currently experiencing a problem with the Listserv Server. DNST engineers are investigating the issue and there is no estimated time of repair at this time. Please contact the NIH Help Desk at 301-496-HELP or helpdesk@nih.gov if you have any questions.
Resolution:  The NIH Listserv server is operational at this time.
4/6/2005 10:33:19 AM - CIT Customer Support Report Server unavailable.
Description:   The CIT Customer Support report server is experiencing data issues and will be unavailable for the balance of the business day today, Wednesday April 6, 2005. This will prevent customers from running their own reports against the server and will also prevent dynamic reports on the CIT Customer Support Reports website from running. Static reports on the website will be unaffected. The problems will be resolved after hours this evening. We apologize for the inconvenience and appreciate your patience while we work to resolve this issue. If there are any mission-critical reports that are impacted by this and that must be generated today, please contact the NIH Help Desk for assistance.
Resolution:  The CIT Customer Support report server has been returned to service. Data is now available and dynamic reports on the Customer Support Reports site are functioning. Thank you for your patience while we worked to resolve this issue.
4/5/2005 4:14:06 PM - Remedy/NIH Enterprise Directory (NED) Integration Release CIT
Description:   The NED support staff will begin the initial NED/Remedy data synchronization on April 9th, 2005 from 6:00 AM to 6:00 PM. During this time the Remedy system may experience some minor degradation in performance. Should you experience any major difficulties with Remedy please contact the NIH Help Desk at 301-496-4357 or email them at nihhelpdesk@mail.nih.gov. We apologize for any inconvenience this may cause.
Resolution:  The initial NED/Remedy synchronization has been completed successfully.
4/4/2005 8:49:12 AM - CIT Customer Support Web Server Temporarily Unavailable
Description:   The CIT Customer Support Web Server is temporarily unavailable due to a system issue. This impacts the IT Help Desk site, the CIT Training site, the CIT Knowledge Base, and other CIT customer support sites. Staff are working to resolve the problem as soon as possible. While there is no ETA, the service outage is not anticipated last for an extended period of time.
Resolution:  The CIT Customer Support Web Server has been returned to service. We appreciate your patience while we addressed this issue.
4/1/2005 10:44:43 AM - MSN Messenger network issues
Description:   This customer advisory applies to IM customers using the MSN Messenger service. The MSN Messenger network is currently experiencing difficulties. Many users are unable to log on. We will issue another advisory when we have an estimated time of resolution from Microsoft.
Resolution:  No resolution. Service just returned.
3/26/2005 1:17:34 PM - Blackberry Connectivity Issues.
Description:   Research in Motion (RIM) is having infrastructure issues which is impacting the Blackberry services that the Central Email Service provide. Expect intermittent connectivity. There is no ETA on resolution.
Resolution:  RIM restored full functionality at approximately 2:20PM.
3/25/2005 6:11:23 PM - ITAS is scheduled for maintenance Tuesday evening 6-6:30PM.
Description:   Dear ITAS User, ITAS is scheduled for maintenance Tuesday evening, March 29 from 6:00PM - 6:30PM. During this time, ITAS will be unavailable. If you have any questions, please contact Helpdesk at 301-594-6248 or via email at nihhelpdesk@mail.nih.gov <<<<<>>>>> Thank you for your patience. Very Best Regards, ITAS Technical Team, DECA/CIT
Resolution:  ITAS scheduled maintenance is now complete.
3/25/2005 9:28:42 AM - Remedy/NIH Enterprise Directory (NED) Integration Release
Description:   The Support Technologies Group (STG) will be working with the NED support staff on March 31, 2005 from 6:00 to 9:00 PM to release the initial integration between the two systems. During this time the Remedy system may experience some degradation in performance as new code is added to the production Remedy environment and the initial data synchronization is done with NED. We apologize for any inconvenience this may cause.
Resolution:  Resolving per Goeff. We have completed the Remedy portion of the NED upgrade. We expect the actual NED Link to occur next Saturday April
3/24/2005 11:05:40 AM - NED Join Engine Service Not Operational
Description:   The NED Join Engine is currently unavailable. New records can be created in NEDWeb and existing records updated, but updates will not be propagated to external connected systems (e.g., the ID badge system and ITAS) until the Join Engine service has been restored. Still awaiting word from BOSB regarding status of the server that runs the Join Engine.
Resolution:  The NED Join Engine service has been restored. All updates entered in NEDWeb have been propagated to external connected systems (e.g., ID badge system, ITAS).
3/23/2005 2:27:55 AM - NIHnet: Power Outage on Executive Boulevard
Description:   There is currently a power outage on Executive Boulevard causing a loss of network connectivity for many of the buildings located along that road. Per PEPCO, a tree limb fell on power pole and there is no estimated time of repair.
Resolution:  The power outage has been resolved on Executive Boulevard and all network connectivity restored.
3/21/2005 5:04:13 PM - NIHnet: Guest Wireless Service
Description:   There is currently a problem with the NIHnet guest wireless service preventing guest users from using the wireless network. CIT/DNST engineers are working on the issue. Please note that this does not affect NIHnet wireless users who are using the wireless VPN service.
Resolution:  The NIHnet guest wireless services have been completely restored. Please contact the NIH Help Desk at 301-496-HELP or helpdesk@nih.gov if you have any further issues.
3/21/2005 8:07:23 AM - NIH Portal Temporarily Unavailable
Description:   The NIH Portal is temporarily unavailable due to issues stemming from other sources. The engineers are currently working to resolve this issue ASAP. Thanks for you patience.
Resolution:  The NIH Portal and all applications accessible through the portal are now available. We do apologize for any inconvenience this may have caused. We thank you for you patience and understanding.
3/18/2005 11:51:01 AM - Restarting ITAS Web services
Description:   The ITAS Technical Team will be restarting the ITAS Web servers in an attempt to remedy a problem some users are having logging in. We anticipate the web service will be unavailable for 5 minutes.
Resolution:  The problem that some users have sporadically experienced in logging into ITAS have now been corrected.
3/17/2005 4:15:27 PM - NIHnet: Mailfwd Issue
Description:   NIHnet users are reporting problems with mailfwd. CIT/DNST engineers are currently working on the issue. There is no estimated time of repair at this time.
Resolution:  The Mailfwd issue has been resolved.
3/15/2005 4:56:48 PM - NIH Portal Upgrade (Scheduled Maintenance): NIH Portal;
Description:   The NIH Portal Administrators will be performing a scheduled upgrade to the NIH Portal covering the period of 3/15/2005 through 3/21/2005. Below is a schedule of the series of events: 3/15/2005 COB to 3/18/2005 my.nih.gov will be available but no changes will be saved (e.g. gadget preferences and document uploads will be lost) a message saying this will be prominently displayed in the portal header area 3/19-3/20: my.nih.gov should be up and running over this weekend, if users do NOT see the message regarding changes not being saved, they can assume this is the production database and to proceed as usual, but portal staff will be busily changing database features (recreating portlet taxonomy, changing the admin hierarchy, creating roles for community leaders, help desk staff, etc.) at this time 3/21: portal go live with new version Estimated Return to Service = 3/18/2005 12:00:00 AM
Resolution:  Outage Completed (automatically updated by CIT Outages form). The NIH Portal Administrators will be performing a scheduled upgrade to the NIH Portal covering the period of 3/15/2005 through 3/21/2005. Below is a schedule of the series of events: 3/15/2005 COB to 3/18/2005 my.nih.gov will be available but no changes will be saved (e.g. gadget preferences and document uploads will be lost) a message saying this will be prominently displayed in the portal header area 3/19-3/20: my.nih.gov should be up and running over this weekend, if users do NOT see the message regarding changes not being saved, they can assume this is the production database and to proceed as usual, but portal staff will be busily changing database features (recreating portlet taxonomy, changing the admin hierarchy, creating roles for community leaders, help desk staff, etc.) at this time 3/21: portal go live with new version Estimated Return to Service = 3/18/2005 12:00:00 AM
3/14/2005 10:22:47 PM - NIHnet: Power Outage in Building 9
Description:   Currently there is a power outage in Building 9 causing a loss of network connectivity for users in that building. There is no estimated time of repair at this time.
Resolution:  The power issues in Building 9 have been resolved and all network connectivity has been restored.
3/14/2005 10:53:26 AM - NIHnet: Internet Network Slowness for Building 10 Users
Description:   CIT is getting reports that some users in Building 10 are experiencing network slowness getting to web sites outside of NIHnet. CIT/DNST engineers are investigating the issue.
Resolution:  The problem with Internet slowness for Building 10 users has been resolved. Please contact the NIH Help Desk at 301-496-HELP if you have any questions.
3/11/2005 8:36:14 AM - ADB Printing Problems (IMS Precipitated)
Description:   The help desk has received a number of calls this morning from customers reporting problems printing from the ADB. This does not appear to be a MVS/VPS problem, but an ADB one. Please ask customers for their printer NIHC number and the ADB module they were working in when they received the error message "Terminal not defined to system" At this time, this is an Internal HN until we get additional information from ADB Tier 3 Support. Please relate all ticket to this HN. Thanks
Resolution:  All, I’m happy to announce that the problem printing from the ADB has been resolved and confirmed by some customers who reported the problem. Please note the slight modification to the Event title. The problem precipitated within IMS and affected ADB customers. The NIH Help Desk and the service provider(s) thank you for your patience and understanding.
3/10/2005 4:23:29 PM - NIHExchange15 Emergency Reboot
Description:   Greetings, We need to schedule an emergency reboot of NIHEXCHANGE15 tonight at 10PM to clear a hung backup agent. We have worked with the backup software vendor to try and resolve this with out a reboot, but they have exhausted all other possibilities.
Resolution:  Exchange 15 is in service. Work is continuing with vendor on updates for identified service.
3/8/2005 7:21:46 AM - Mail addressed to @nih.gov addresses delayed
Description:   Due to hardware issues, mail addressed to @nih.gov addresses is currently not being delivered due to hardware problems. Staff is currently working to correct the situation.
Resolution:  Problem has been resolved. Email addressed to @nih.gov is being delivered. Expect some delays this morning as the system processes queued up messages. We apologize for the inconvenience and the delays.
3/7/2005 1:19:02 PM - NIH Login - sporadic 500 errors
Description:   NIH Login team has received reports from Unix based applications that their users are reporting sporadic 500 errors when trying to access their applications. Our engineers are aware of the problem and are are working with vendors to find a resolution.
Resolution:  The NIH Help Desk was informed that this problem had been resolved.
3/3/2005 11:00:18 AM - Resolved: NIHExchange13 server reboot/issues
Description:   Resolved: Nihexchange13 is back up and functioning. Exchange services are running. Queued messages to the store are all processed and outbound email from the server is working fine. NIHExchange13 server reboot in progress to resolve issues with the store on NIHExchange13. BOSB engineers must reboot to restore normal activity...
Resolution:  Nihexchange13 reboot. Nihexchange13 is back up. Exchange services are running. Queued messages to the store are all processed and outbound email from the server is working fine.
2/28/2005 10:45:40 AM - Remedy System Giving License Errors
Description:   The Remedy ticketing system is currently generating license errors. Users who attempt to log in interactively or access the system via the Support Website or API may receive messages indicating that there are insufficient licenses on the server. Support staff are currently working with the vendor to get the problem resolved ASAP. There is no ETA at this time but we hope and expect that the issue will resolved very shortly. Note that there is no true license shortage, this is a system error.
Resolution:  The issue has been resolved and all licenses are now available on the system. We apologize for the inconvenience and appreciate your patience while we worked to get the issue corrected.
2/28/2005 8:12:07 AM - ITAS Wireless service is in distress
Description:   The wireless ITAS service for BlackBerry is not responding after login attempts. Other ITAS services are functioning normally. The ITAS Technical Team is working on the problem and hopes to have it resolved very sson.
Resolution:  The problem with ITAS Wireless BlackBerry service has been resolved. All ITAS systems are now operating normally.
2/25/2005 4:06:47 PM - NIHnet Maintenance: DNS and Listserv Server Relocation
Description:   On Saturday, 3/5/2005, from 7:00 a.m. - 11:00 a.m., CIT/DNST will be moving the hardware that hosts our primary Domain Name Server (DNS), ns.nih.gov, and our Listserv server, list.nih.gov. From 7:00 am - 9:30 a.m., the Listserv website, http://list.nih.gov, will not be available. Please note that all listserv e-mail will be queued and will be processed on completion of the maintenance. From 9:30 am - 11:00 a.m. the primary DNS server, ns.nih.gov, will not be available. During the maintenance window the backup DNS server, ns2.nih.gov, will continue to service all DNS requests. Please note, any DNS change requests made via the DNS website, http://www.net.nih.gov/dns, will not take effect until the completion of the maintenance. Please contact the NIH Help Desk at 301-496-HELP or helpdesk@nih.gov if you have any questions.
Resolution:  The maintenance to move the hardware that hosts Listserv and DNS was successfully completed. Please contact the NIH Help Desk at 301-496-HELP or helpdesk@nih.gov if you have any questions.
2/25/2005 3:30:21 PM - ITAS Maintenance Today, March 1 from 6:00 PM - 6:30 PM
Description:   Dear ITAS User, Next Tuesday, March 1 from 6:00 PM - 6:30 PM, ITAS is scheduled for an upgrade. This ITAS-Web upgrade consists of changes in ITAS T&A rules to accommodate the new DFAS payroll system, changes that will allow all us to begin phasing out the administrative officer role from the old ITAS Client Server application, and corrections to a few minor problems identified with the current version. During this upgrade, ITAS will NOT available for 30 minutes. After the upgrade, the access to the Client/Server version for the vast majority of the Administrative Officers will be removed. We apologize for any inconvenience this may cause you. If you have any questions, please contact Helpdesk at 301-594-6248 or via email at nihhelpdesk@mail.nih.gov <<<<>>>>. Best Regards, ITAS Technical Team, DECA/CIT
Resolution:  The ITAS upgrade is now complete and all ITAS systems should be fully functional. Thanks for your patience as we work to improve ITAS. Best Regards, ITAS Technical Team, DECA/CIT
2/22/2005 8:03:23 PM - BlackBerry services down
Description:   BlackBerry services went down approximately 7:15 PM eastern time. RIM sent this info: A component of the BlackBerry network infrastructure is experiencing a service interruption. Service Affected: BlackBerry Relay service for some *NA Network subscribers in the following locations: The Americas Summary: BlackBerry Infrastructure Service Outage Impact: Customers will experience delays in receiving messages and an "x" when sending messages. Undelivered messages may be re-sent after the incident has cleared. New SRP clients and other services cannot authenticate. PINs without recent activity cannot be assigned, so their traffic cannot be processed. Customers will lose SRP connections. BES clients will not be able to reconnect and new users will not be able to register their devices until after the incident has cleared. Subscribers will be unable to receive messages and will receive an "x" when sending messages. Subscribers will be unable to use Relay Roaming. GRX roaming is unaffected. Subscribers who attempt to register will receive a message indicating that the request has failed. These subscribers will not be able to send or receive from their handheld until it is registered, after the incident has cleared. Ticket Number: BB000026223 Start Date and Time: 22/02/2005 7:26:00 AM (EST) Event Duration: Ongoing % of Subscribers Affected: 100.00 (estimated) Cause: To Be Determined
Resolution:  SRP connections slowly coming back... RIM says up to 2 hours, but we already have over over half the BES's back online.
2/16/2005 11:46:54 AM - Remedy System Maintenance 2-16-05 10:00 PM
Description:   We will be performing system maintenance on the Remedy system tonight which will require us to reboot the Remedy server. During the reboot time, approximately 15 minutes, the following system will be affected: Remedy, Remedy Web client and Remedy API. We expect full service to be returned by 10:15 PM. Thank you for your understanding.
Resolution:  System maintenance has been completed, the server rebooted and back in operation. All services have returned to normal.
2/14/2005 11:39:41 AM - BlackBerry-Degradation of GPRS Cingular service
Description:   Update: Issue location isolated to Alabama, Florida, Louisiana, and Texas. Some RIM BlackBerry users will not be able to connect to the Cingular GPRS network. Location of Issue: See update No estimated time of repair was given. Updates will be posted as available.
Resolution:  Location of Issue: Issue isolated to Alabama, Florida, Louisiana, and Texas.
2/11/2005 9:23:45 PM - Customer Support Web Server Experiencing Problems
Description:   The CIT Customer Support Web Server is experiencing problems. We are currently investigating and will provide updates as we gain more information.
Resolution:  We have isolated the problem to code for the Outage Calendar. We have isolated the Outage Calendar site and removed the Outage Calendar code from the Support Site and replaced it with a link to the stand alone Outage Calendar web page. The Customer Support Web Server is up and functional.
2/11/2005 5:48:53 PM - Customer Support Web Server Being Restarted
Description:   The CIT Customer Support Web Server is experiencing problems. Staff is currently working on the issue and will be restarting the server. Estimated return to service is before 6:00 PM.
Resolution:  The server has been restarted. Staff continues to monitor the server status to ensure continuity of service.
2/11/2005 4:46:33 PM - Remedy System Maintenance 2-12-05 10:00 PM
Description:   We will be performing system maintenance on the Remedy system Saturday Feb. 12 at 10:00 PM which will require us to reboot the Remedy server. During the reboot time, approximately 15 minutes, the following system will be affected: Remedy, Remedy Web client and Remedy API. We expect full service to be returned by 10:15 PM. Thank you for your understanding.
Resolution:  Reboot was cancelled.
2/11/2005 4:28:32 PM - Customer Support Web Server Being Restarted
Description:   The CIT Customer Support Web Server is experiencing problems. Staff is currently working on the issue and will be restarting the server. Estimated return to service is before 4:45 PM.
Resolution:  The server has been restarted. Staff continues to monitor the server status to ensure continuity of service.
2/10/2005 4:40:10 PM - Customer Support Web Server Being Restarted
Description:   The CIT Customer Support Web Server is experiencing problems. Staff is currently working on the issue and will be restarting the server. Estimated return to service is before 5:00 PM.
Resolution:  The server has been restarted. Staff continues to monitor the server status to ensure continuity of service.
2/10/2005 4:33:14 PM - ITAS Scheduled Maintenance
Description:   ITAS is scheduled for maintenance this Friday evening, Feb 11, 2005 from 6:00PM - 6:30PM. During this time ITAS will not be available. We apologize for any inconvenience this may cause you. If you have any questions, please contact Helpdesk at 301-594-6248 or via email at nihhelpdesk@mail.nih.gov <<<<<>>>>>
Resolution:  ITAS scheduled for maintenance is now complete.
2/10/2005 8:08:01 AM - LISTSERV e-mail delays
Description:   There is currently an issue with LISTSERV causing delays in e-mails. DNST is currently working to resolve the issue. E-mail is still being queued and will be sent upon resolution. No ETR given. Updates will be posted when available.
Resolution:  The LISTSERV issues have been resolved.
2/9/2005 7:15:42 PM - ITAS Wireless BlackBerry Service temporarily unavailable.
Description:   Hello ITAS Wirless Blackberry User, We have suspended access to the ITAS Wireless Application in order to troubleshoot intermittent connectivity problems that we have monitored in this service. We will resume access to this service as soon as the problem has been diagnosed and corrected. This will not effect normal ITAS Web users, only ITAS Wireless Blackberry. We regret this inconvenience, and appreciate your patience while we work to resolve the problem. Very Best Regards, The ITAS Technical Team.
Resolution:  The problem with ITAS Wireless service connectivity has now been resolved. Users can again utilize the ITAS Wireless service via supported BlackBerry handheld devices by going to https://itas.nih.gov.
2/9/2005 4:29:09 PM - Remedy System Maintenance 2-9-05 10:00 PM
Description:   We will be performing system maintenance on the Remedy system tonight which will require us to reboot the Remedy server. During the reboot time, approximately 15 minutes, the following system will be affected: Remedy, Remedy Web client and Remedy API. We expect full service to be returned by 10:15 PM. Thank you for your understanding.
Resolution:  System maintenance has been completed, the server rebooted and back in operation. All services have returned to normal.
2/8/2005 3:05:09 PM - MSN Messenger outage
Description:   Microsoft has notified NIH of an outage of the Microsoft .NET Messenger Service. They do not yet have an ETA for when it will be restored. Microsoft will post updates on their status page: http://messenger.msn.com/Status.aspx We will update when MSN service is restored.
Resolution:  MSN Service has been restored by Microsoft as of shortly after 8:30 PM this evening. The MSN status page at http://messenger.msn.com/Status.aspx has been updated to show the corrected system status
2/8/2005 2:00:22 PM - Resolved Hot News-------nVision System Maintenance
Description:   nVision will be down one hour earlier today (5pm, instead of 6pm) for necessary system maintenance.
Resolution:  Maintenance completed.
2/6/2005 9:42:12 AM - Restarting the ITAS Database Server
Description:   The ITAS Technical Team has identified an anomoly in the ITAS database server. We are restarting the service in hopes of restoring the system to normal operation. Users can expect a brief lapse in service (a few minutes).
Resolution:  After normal password rotation procedures the SQL Agent password was not correctly updated. In the previous version of SQL Server, this had not been needed. The password change control document has been properly amended to reflect this SQL 2000 specific change.
2/2/2005 11:45:29 AM - ITAS Wireless Issue
Description:   We had several calls concerning ITAS wireless service from NIAID. We are currently investigating the problem and will send an update as soon as possible. Also, another issue appears to be that users are going to the NIH Portal instead of the specified ITAS URL, http://itas.nih.gov . Additionally, only specific Blackberry models (7xxx and 6xxx) will be able to use the ITAS wireless service
Resolution:  We anticipate the NIAID users will be able to access ITAS wireless service as soon as the NIH Active Directory consolidation is complete.
1/24/2005 4:42:54 PM - Remedy Server Outage 8 PM 1-24-05
Description:   We will be rebooting the citdcsrx server at approximately 8 PM tonight. Remedy services will be down for the period of time it takes the server to reboot. We estimate that full service will be restored by 8:15 PM 1-24-05.
Resolution:  Reboot complete. All Remedy services are up and running.
1/14/2005 8:25:42 AM - NBS Travel Production Currently Unavailable
Description:   The NIH Help Desk received the following email from the NBS Management Center: We are currently experiencing a problem with the NBS Travel Production. Anyone attempting to login will see a "Webspeed" error message. The NBS DBA group is working on a resolution to the problem. At this time we do not have timeframe for when the system will be back online. We will notify you as soon as more information becomes available.
Resolution:  The NIH Help Desk has been informed that the NBS (Travel, and Budget and Finance) is back online. Thanks for patience and understanding
1/12/2005 2:55:13 PM - HHS Careers website experiencing problems
Description:   HHS Careers is currently experiencing problems with the Applicant website, https://jobs.quickhire.com/scripts/hhs.exe, at HHS Careers (QuickHire). They are also having problems with the human resources tools including the Question Library and Vacancy Preview. QuickHire is working to resolve the problem, however, this may result in system outages today. If you are trying to view or apply to an NIH vacancy and experience problems, please try again later. If the vacancy closes today, please call HR Systems Support Help Desk at NIH at 301-451-1436 or e-mail hrsystemssupport@od.nih.gov and provide them with your name and the vacancy announcement number. We apologize for any inconvenience.
Resolution:  Problems with HHS Careers website have been resolved.
1/11/2005 6:47:10 PM - DCS Web Server Emergency Reboot
Description:   We will be rebooting the DCS Web server at 9:00PM on January 11, 2005. The outage should last for no longer than 30 minutes and will affect all DCS web services. We apologize for any inconvenience this may cause and will provide an update when the reboot is complete.
Resolution:  The Server has been rebooted successfully. We apologize for any inconvenience this may have caused.
1/10/2005 11:03:07 AM - CIT Web Server Not Responding
Description:   Currently, the CITDCSWEB server is not responding, and is therefore affecting a variety of sites, including support.nih.gov; remoteaccess.nih.gov, and the knowledgebase. ISG staff is aware of the problem and are working to resolve it.
Resolution:  Web Server is now operating normally and all sites are available.
1/7/2005 12:36:54 PM - Report Server Database Problems
Description:   The CIT Reports server that supports reporting for the NIH Helpdesk is currently experiencing database issues. We are currently researching the cause of the problem and do not have an estimated time to correct the database. We apologize for any inconvenence this may cause. If you have an urgent requirement to generate a report please contact the NIH Helpdesk at 301-496-4357.
Resolution:  The issues with the CIT Reports server database have been corrected. We apologize for any inconvenience this may have caused.
1/6/2005 1:24:31 PM - OWA Access -- NIH Wide
Description:   The Outlook Web Access at the NIH is experiencing sporadic access issues. In order to correct the issue BOSB will be reseating the load balancer module for OWA. During this process all access to OWA will be lost, down time will be minimal.
Resolution:  Correction: There was no problem with the OWA service. This service outage was a result of Hot News HN05552 - Datecenter Netowork issues.
1/6/2005 12:53:52 PM - NIHnet: Data Center Router Module Errors
Description:   CIT/DNST is seeing intermittent connectivity issues on one module on one of the routers in the CIT Data Center. CIT/DNST engineers are on site investigating the issue.
Resolution:  The faulty module was replaced on the CIT Data Center router and all connectivity has been restored. Please contact the NIH Help Desk at (301) 496-HELP or helpdesk@nih.gov if you have any questions.
1/5/2005 10:40:07 AM - NIHnet: Power Outage in Building 9
Description:   There was a power outage in Building 9 causing a loss of connectivity for Building 9 users. The power is back on and all network connectivity has been restored.
Resolution:  Power was restored to building 9.
12/30/2004 2:10:55 PM - CIT Report Server (CITDCSRPT) Emergency Maintenance
Description:   The transactional log files on the CIT report server have grown to maximum size constraints. Attempts to reduce the size of the Transactional Log file have not been successful. Therefore the Database services will need to be restarted. This activity will occur at 6:00 PM on December 30, 2004. It is anticipated that this activity will take no longer than 15 minutes to complete. We apologize for any inconvenience this may cause.
Resolution:  The database restart is complete. We apologize for any inconvenience this may have caused.
12/27/2004 1:39:42 PM - NIHnet Fiber Cut
Description:   The Off Campus fiber provider is experiencing a fiber cut affecting network connectivity for the below two sites. There is no estimated time of repair. - 5 Research Court - 9610 Medical Center Drive
Resolution:  The off campus fiber provider repaired the fiber. Connectivity for the users at 5 Research Court and 9610 Medical Center Drive has been restored.
12/20/2004 2:34:33 PM - NIHEXCHANGE15: Customers unable to access email
Description:   Customers on NIHEXCHANGE15 are unable to connect to send or receive email. This will affect several OD customers. BOSB is aware of the problem and is researching a solution. More updates as they become available.
Resolution:  The issues with NIHEXCHANGE15 have all been resolved. It was back online as of 2:35 PM and was being monitored until now.
12/20/2004 10:55:07 AM - Outlook Web Access: Inaccessible outside of the NIH network
Description:   Outlook Web Access is not accesible to customers outside of the NIH network. BOSB is researching the issue.
Resolution:  Tested and verified that OWA is currently accessible to customers outside the NIH network.
12/17/2004 8:40:25 AM - NIHnet: Poolesville Power Issues
Description:   Poolesville is currently experiencing power issues, causing a loss of connectivity to NIHnet for our Poolesville customers. There is no estimated time of repair at this time.
Resolution:  The Poolesville power issues have been resolved, and all network connectivity has been restored.
12/13/2004 1:31:05 PM - Resolved-nVision Upgrade
Description:   The NIH nVision be down today Monday, December 13, beginning at 5 PM due to an upgrade and will be available tomorrow morning.
Resolution:  Upgrade completed.
12/13/2004 1:24:07 PM - T-mobile problems with Incoming Voice Calls
Description:   No apparent resolution from service provider. Problems seems to be resolved. NIH Helpdesk confirmed with CIT Blackberry Administrators who have tested T-mobile incoming calls and they are receiving these incoming calls. T-mobile customers may experience problems with the following: Voice - Incoming Calls Estimated time of resolution is 1 hour. Updates will be posted when available.
Resolution:  No apparent resolution from service provider. NIH Helpdesk confirmed with CIT Blackberry Administrators who have tested T-mobile incoming calls and they are receiving these incoming calls.
12/10/2004 9:46:36 AM - Helix Systems Scheduled Downtime, Tuesday December 14, 6 PM
Description:   As previously announced to users, the Helix Systems - Helix, Nimbus, Quasar and the Biowulf cluster - will be unavailable to users on Tuesday December 14 beginning at 6 PM for system maintenance. It is anticipated that the systems will be available to users again no later than 10 PM.
Resolution:  Helix, Nimbus and Quasar were made available to users at 8:10 pm. The Biowulf cluster was available at 1 am on Wednesday, 15 December.
12/9/2004 7:20:38 PM - DCSS-SAN server access issue.
Description:   DCSS-SAN server access issue. There is a problem with the SAN server which is affecting access to certain file systems. We are experiencing problems with the DCSS SAN. Staff is on the way to NIH to investigate. We will provide details and updates as we have them. Affected server systems that we know of are VPDBS, VPDBS, VPICRP, VSERVE1, CSRFILER, NIHFILER. There is no ETA.
Resolution:  System in service and servers can connect. Engineers doing post mortem
12/8/2004 3:03:19 PM - NIHExchange27 e-mail issues
Description:   Customers are experiencing e-mail issues connecting to NIHExchange27. These problems are affecting parts of NIAID, NIA/IRP and FIC. The E-mail Exchange Administrators are aware of the problem and working on resolving these problems. No estimated time of repair as of yet. Updates will be posted when available.
Resolution:  NIHExchange27 has been fully restored to service. All affected customers should now be able to access their email.
12/7/2004 5:03:35 PM - Emergency Remedy Outage 7:00 to 7:30 PM Tuesday, 12/7/2004
Description:   The Remedy system is currently experiencing problems with the escalation processes. DCS/ISG has been working with the vendor to correct the problems. The vendor has recommended that the systems supporting Remedy be rebooted. DCS/ISG is scheduling an reboot of the Remedy Server and supporting database server for 7:00 PM on Tuesday, December 7, 2004. The outage should last no longer than 30 minutes and will affect all access to the Remedy system.   We apologize for any inconvenience this may cause.
Resolution:  The Remedy System has been successfully rebooted and returned to service. The reboot process has cleared the problems with the escalation processes. We appreciate your patience and apologize for any inconvenience this may have caused.
12/7/2004 3:21:14 PM - Mainframe to Network Printing Issue (ADB)
Description:   The NIH Help Desk has been informed that jobs spooling from the ADB system are not being passed to the JES/VPS system on the Titan system. This will affect some users printing from the ADB system. The MVS Support team is working to resolve this issue and will provide additional updates as they become available. Thanks for your patience and understanding
Resolution:  Print queues have been cleared. Cause of the print spooling from the ADB is still under investigation.
12/1/2004 2:05:54 PM - Resolved: T-Mobile Blackberry data problems.
Description:   The problems with T-Mobile Blackberry Data service has been resolved. Please be advised T-Mobile customers may experience problems with the following: Blackberry Data services. Estimated time to resolution is 2 hours. Updates will be posted when available. If customers are experiencing problems with data services, it is possible that resetting the Blackberry device will resolve the issue.
Resolution:  The problems with T-Mobile Blackberry Data service has been resolved.
11/30/2004 3:01:40 PM - Resolved: Blackberry RIM delays receiving e-mail
Description:   Resolved: Blackberry RIM delays receiving e-mail At&t/Cingular stated that RIM BlackBerry users may experience delays receiving Email. Users should be able to send Emails without problems. There is a Degradation of RIM BlackBerry services. No estimated time of repair was given. Updates will be posted when available.
Resolution:  Resolved: Blackberry RIM delays receiving e-mail
11/30/2004 10:49:04 AM - ECARES Currently is Unavailable
Description:   The NIH Help Desk has been informed that the ECARES system is currently experiencing connectivity issues. The ECARES technical support staff is working with the appropriate parties to resolve this issue ASAP. This posting will be updated as additional information becomes available. Thanks for your patience and understanding.
Resolution:  The ECARES system is provisionally back on line. Several application server, database server, and network related events that occurred this morning within a fairly brief period of time. As a result, the ECARES server lost its connection with the database server. That event apparently triggered additional problems on the ECARES server that ultimately required a rebuild of the search
11/30/2004 7:23:54 AM - Resolved: Blackberry GPRS network degradation
Description:   Resolved: The GPRS network degradation has been resolved. Total Duration: 4 hours and 8 minutes Technicians are working towards resolution as quickly as possible. AT&T/Cingular is experiencing degradation of their GPRS network. Location of issue: National. This degradation started at 2:15 a.m. PST 11/30/04. There is no estimated time of repair. Updates will be posted when available.
Resolution:  Resolved: The GPRS network degradation has been resolved. Total Duration: 4 hours and 8 minutes
11/18/2004 7:47:07 PM - NIHnet: ORF Electrical Shutdown in CRC - CANCELLED
Description:   The Office of Research Facilities has just notified CIT that there will be an electrical shutdown in the CRC Building on Saturday, November 20, 2004, from 8:00 a.m. until 12:00 p.m. This power maintenance will cause a loss of connectivity for NIHnet users within the Clinical Research Center. Information on the power maintenance can be found at the following link: http://des.od.nih.gov/iWeb/Utility/scripts/utility_shutdown.asp?seq=4151 or by contacting the PWB Office of Resource Management at (301) 594-9802. If you have any questions regarding NIHnet connectivity, please contact the NIH Help Desk at (301) 496-4357.
Resolution:  CIT has been advised that ORF has cancelled the electrical maintenance in the CRC.
11/18/2004 5:56:23 PM - NIHnet: Building 10 Switch Issue
Description:   There is currently an issue with one of the switches servicing some of the Building 10 local networks. CIT/DNST engineers are onsite investigating the issue.
Resolution:  Building 10 switch issue has been resolved and local network connectivity restored.
11/18/2004 4:09:13 PM - NIHnet: Internet Connectivtiy Issues
Description:   NIHnet is currently experiencing connectivity issues getting to the Internet. CIT/DNST Engineers are investigating the issue now.
Resolution:  Internet connectivity has been fully restored.
11/18/2004 1:22:14 PM - NIHnet Emergency Maintenance: Perimeter Network
Description:   On Friday, 11/19/2004, from 6:30am -7:30am, CIT/DNST engineers will be performing maintenance on the NIHnet perimeter network to return traffic to our primary, redundant network path. There may be very brief packet loss and data retransmission as the routing tables recalculate during the failover process. Please contact the NIH Help Desk at (301) 496-HELP or helpdesk@nih.gov if you have any questions.
Resolution:  The perimeter network maintenance will not be performed as it is no longer necessary.
11/18/2004 2:57:32 AM - NIHnet: Building 10 Network Issues
Description:   There are currently network connectivity problems for some local networks in Building 10. CIT/DNST engineers are onsite investigating the issue.
Resolution:  All network connectivity for local networks in Building 10 has been restored.
11/15/2004 1:33:40 PM - Access issue to EDM (Enterprise Directory Management)
Description:   Some customers may have issues accessing EDM (Enterprise Directory Management a web based active directory application). There is a licensing issue that is causing the problem. The licensing issue is being investigated. Updates will be posted as they are available.
Resolution:  New licenses have been applied and access has been restored.
11/10/2004 7:16:31 AM - Resolved: NHGRI network outage building 12A
Description:   Resolved: The network outage at NHGRI building 12A has been resolved. All network resources are now available to the NHGRI users in building 12A. NHGRI is experiencing a network outage in building 12A. All network resources are unavailable. NHGRI Network engineers are working on resolving the problem. Updates will be posted as they come available.
Resolution:  Resolved: The network outage at NHGRI building 12A has been resolved. All network resources are now available to the NHGRI users in building 12A.
11/8/2004 2:39:08 PM - Resolved: CIT printing issues to CITPRINT server
Description:   Resolved: The printing problems with CITPRINT server have been resolved. The server was rebooted. CIT customers are experiencing problems printing to CIT printers. CITPRINT server administrators are aware of the issue and are working on a resolution. Updates will be posted when available.
Resolution:  Resolved: The printing problems with CITPRINT server have been resolved. The server was rebooted.
11/8/2004 9:32:02 AM - Titan/MVS System upgrade causing ADB printing issues
Description:   Due to the recent update to the Titan/MVS system, VPS printing is experiencing an issue passing output to the Administrative Database (ADB). This is preventing all output from this application to be handed off to the ADB printers. The issue is currently being addressed by the MVS Systems Team. No estimated time of resolution as of the time of this notice.
Resolution:  All remaining ADB printing issues (in this case, output formatting) were resolved at approximately 3:30PM yesterday, November 8th.
11/4/2004 12:16:46 PM - Citrix Upgrade-RESOLVED
Description:   The DataWarehouse, nVision-Adhoc, ITAS-Terminal Server version and ADBWeb will be down on Thursday, November 4 from 7 PM to 8 PM and on Friday, November 5 from 5:30 AM to 7 AM due to a Citrix upgrade.-Resolved, upgrade complete.
Resolution:  Upgrade complete.
11/4/2004 7:33:42 AM - Blackberry Cingular/ATT network degradation issue
Description:   Update: When attempting to switch your data service on your BlackBerry go to "options", "network" change the "Scan Mode" to manual and select "scan for networks". The available networks will be listed. If Cingular is an available network, select Cingular, and then select "register now". If the registration is successful customers will be able to receive and send e-mail messages. Note: If the Cingular network is not available this will not work. You may try again later to see if the Cingular network is available. Update: With regards to this outage, you may still be able to receive email on your BlackBerry by selecting a roaming carrier . To get data service on your BlackBerry go to "options", "network", "scan for networks", and choose Cingular (if available). Once registered with Cingular, you should be able to receive Emails on your BlackBerry device. Technicians are working towards resolution as quickly as possible. Cingular/ATT has reported degradation of Blackberry CDPD and GPRS Networks. Customers may experience difficulties accessing the CDPD and GPRS Networks to send and receive e-mail messages. No estimated time of repair given. Updates will be posted as available.
Resolution:  Cingular/AT&T has resolved the Degradation of CDPD and GPRS Networks which my have affected Blackberry customers. Cingular/AT&T reported a time of resolution at: 11/04/04 11:11 PM EDT. Customers should reset their Blackberry devices to re-attach to the Cingular/AT&T network.
11/2/2004 3:21:03 PM - Portal User Migrations
Description:   On Wednesday evening, 11/3/2004 from 7 PM till start of business on Thursday, the portal will be in the process of migrating users to a new authentication mechanism. Any users logging in and out during this time may notice slowness and possibly will be logged out or receive an error message when logging in. Upon completion, users will notice no difference in portal behavior.
Resolution:  This activity was completed successfully. At this time we are still cleaning up the database, removing all non "-AD" user/groups. As indicated in an earlier email to Community Managers, please ignore any non -AD users/groups. These should be gone by Friday
11/1/2004 1:37:14 PM - Resolved: Blackberry GPRS service degradation
Description:   Resolution: The problem AT&T/Cingular reported with degradation of GPRS service has been resolved. Network problems in Baltimore, MD; Washington D.C. and surrounding areas MAY affect your Mobile Internet Service. Users may experience a loss, or degradation of GPRS service. No estimated time of repair given. Updates will be posted as they are available.
Resolution:  Resolution: The problem AT&T/Cingular reported with degradation of GPRS service has been resolved.
10/29/2004 3:23:43 PM - ITAS scheduled for maintenance Monday, 11/1/04 6:00- 6:30PM
Description:   Dear ITAS User, ITAS is scheduled for maintenance this coming Monday, Nov 1, 2004, from 6:00PM - 6:30PM. This maintenance includes fixes of identified system defects. During this time ITAS will not be available. We apologize for any inconvenience this may cause you. If you have any questions, please contact Helpdesk at 301-594-6248 or via email at nihhelpdesk@mail.nih.gov <<<<>>>> As always, we appreciate your patience as we strive to maintain high reliability of the system. Very Best Regards, ITAS Technical Team, DECA/CIT
Resolution:  ITAS scheduled maintenance is now complete.
10/29/2004 11:00:05 AM - NIH Portal Exchange Gadgets
Description:   At approximately 1 pm today, the Exchange gadgets will be moved to a different server. Some users may need to re-authenticate as a result.
Resolution:  Gadgets moved to new server, all back to normal. Again, some users may need to enter password again for these gadgets.
10/27/2004 1:23:56 PM - NIHEXCHANGE7
Description:   NIHEXCHANGE7 is backup, and we are Sorry for any problems this may have caused.
Resolution:  We are still looking into the cause with MS.
10/25/2004 8:51:07 AM - Web of Science literature search currently unavailable
Description:   The Web of Science literature search service (publishorperish.nih.gov) is currently down due to a system failure. Systems staff are working to resolve the problem. At this time there is no estimate for when service will resume.
Resolution:  The Web of Science hardware problem was resolved by noon Monday, October 25. Weekly Porpoise mailings were being delivered by 3:00 pm Monday. We apologize for any inconvenience this may have caused.
10/21/2004 5:57:07 PM - ITAS Security Upgrade (Oct 21, 6:00 PM - 6:30 PM): ITAS;
Description:   Dear ITAS User, ITAS is scheduled for a security upgrade this evening, Oct 21, from 6:00 PM - 6:30 PM. During the upgrade, the service will not be available. We apologize for any inconvenience this may cause you. If you have any questions, please contact Helpdesk at 301-594-6248 or via email at nihhelpdesk@mail.nih.gov <<<<<<>>>>>> Very Best Regards, ITAS Technical Team, DECA/CIT Estimated Return to Service = 10/21/2004 6:30:00 PM
Resolution:  Outage Completed (automatically updated by CIT Outages form). Dear ITAS User, ITAS is scheduled for a security upgrade this evening, Oct 21, from 6:00 PM - 6:30 PM. During the upgrade, the service will not be available. We apologize for any inconvenience this may cause you. If you have any questions, please contact Helpdesk at 301-594-6248 or via email at nihhelpdesk@mail.nih.gov <<<<<<>>>>>> Very Best Regards, ITAS Technical Team, DECA/CIT Estimated Return to Service = 10/21/2004 6:30:00 PM
10/18/2004 11:31:38 AM - NIHnet: Problems with Internet Access
Description:   One of our diverse Internet Service Providers is currently experiencing problems on their backbone network. As a result, we have shut down the NIHnet connection to the one provider and are now running all our traffic through our other provider.
Resolution:  Our Internet Service Provider resolved the problem and NIHnet traffic is once again traversing both providers networks.
10/18/2004 10:13:22 AM - NIH Portal Exchange Gadgets
Description:   Exchange Inbox, Calendar and Contacts gadgets will be migrated to a different server this evening. There should be no downtime. Users may need to re-authenticate: just click the edit button in any gadget and enter password if requested.
Resolution:  Gadgets should be working properly at this time. Again, some users may need to click the edit button in any one of them and re-enter password.
10/18/2004 9:03:18 AM - CIT Knowledge Base - COTS Content Unavailable
Description:   The knowledge provider for information about Commercial Off-The-Shelf (COTS) products is experiencing problems, therefore the connection to their servers has been temporarily disabled. We are notifying the vendor. At this time we have no ETA for return to service.
Resolution:  This problem appears to have been related to the network issues reported earlier. Services have been restored and everything is funtional at this time.
10/15/2004 8:51:45 AM - NIHnet: Fernwood Wireless Issues
Description:   There are reports of some problems authenticating into the Fernwood wireless network. CIT/DNST engineers are investigating the issue.
Resolution:  The Fernwood wireless issues have been resolved.
10/14/2004 7:42:20 AM - T-Mobile Blackberry Data Service Outage
Description:   T-Mobile Blackberry devices with the error "data connection refused" and/or gprs (in lowercase) in the top right corner are having problems sending and receiving messages. Please be advised you may experience problems with the following: Ticket Number: T000844356 Service(s): Data - GPRS - All data services impacted Market(s): WASHINGTON DC, VIRGINIA, PHILADELPHIA PA, CONNECTICUT Geographic Location(s): WASHINGTON DC, VIRGINIA, PHILADELPHIA PA, CONNECTICUT. The Estimated Time to Resolution is: 2 hours. Sent: 06:32AM 10/14/2004
Resolution:  Please be advised that the outage for: Ticket Number: T000844356 Service(s): Data - GPRS - All data services impacted Market(s): WASHINGTON DC, VIRGINIA, PHILADELPHIA PA, CONNECTICUT Geographic Location(s): WASHINGTON DC, VIRGINIA, PHILADELPHIA PA, CONNECTICUT has been resolved. Sent: 10/14/2004 9:59 AM
10/7/2004 12:38:55 PM - NIHnet Maintenance: Listserv Upgrade
Description:   Reminder: On Saturday, 10/16/2004, from 7:00 a.m. - 8:00 a.m., CIT/DNST will be performing a hardware and software upgrade on the server that hosts Listserv. During this maintenance window, the Listserv website, http://list.nih.gov, will not be available. Please note that all listserv e-mail will be queued and will be processed on completion of the maintenance. Please contact the NIH Help Desk at 301-496-HELP or helpdesk@nih.gov if you have any questions.
Resolution:  Listserv Upgrade maintenance completed successfully.
10/6/2004 11:48:26 AM - ECARES Emergency Restart at 11:45 AM
Description:   The NIH Help Desk has been informed that ECARES will be shut down temporarily at 11:45 for an emergency restart of the system. Downtime should be approximately 30 minutes. Thank you for you patience and understanding
Resolution:  This was an intermittent problem that has been resolved
10/5/2004 11:11:23 AM - eCares System Unavailable for Maintenace
Description:   The eCares system will be unavailable for maintenance from 11:45am for 30-60 minutes to improve performance.
Resolution:  The problem has been resolved.
10/4/2004 7:49:52 PM - NIHnet: HHS Website Issues
Description:   The www.hhs.gov website is accessible at this time. Original posting: The website www.hhs.gov is currently inaccessible. CIT engineers are investigating.
Resolution:  .
10/4/2004 11:40:43 AM - NIH Badging Issue Resolved 10/6/04
Description:   Update: NIH Badging resolved prior difficulties on Weds 10/6/04. Employees are now able to obtain NIH ID Badges as operations resumes back to normal. Original Message: NIH Badging is experiencing difficulties with authorizing new NIH ID Badges. Problems are being investigated and the NIH Help Desk will provide updates as new information becomes available. Currently there is no estimated time of resolution (ETR).
Resolution:  Update: NIH Badging resolved prior difficulties on Weds 10/6/04. Employees are now able to obtain NIH ID Badges as operations resumes back to normal.
10/4/2004 11:08:11 AM - Phone Problems at NIH Helpdesk
Description:   Currently the NIH Helpdesk is experiencing problems with the incoming phone lines. We are currently working to identify the cause of the outage. There is no time to resolutions currently available. Users can still submit service requests to the NIH Helpdesk through the web at http://ithelpdesk.nih.gov/ or via electronic mail at ithelpdesk@nih.gov.
Resolution:  The phones have been in service and functioning normally since the just after 11:30 AM. No root cause has been determined at present. We will continue to review the incident to determine the root cause.
10/3/2004 6:02:37 PM - The ITAS database server is undergoing emergency maintenance.
Description:   The ITAS database server is undergoing emergency maintenance. The CIT/BOSB staff is working on the problem now and hopes to resolve the issue soon.
Resolution:  After much effort the ITAS Database Server has been restored to full operation. Users who executed any ITAS transactions between 4:30-5PM yesterday (Sunday 10/3/2004), should review those transactions to ensure the recovery process captured them. We regret any inconvenience this problem may have caused.
10/1/2004 2:46:04 PM - Fernwood Scheduled Power Outage starting Oct. 9th
Description:   The Fernwood Building will have an extended scheduled power outage beginning on Saturday, October 9th at 8:30PM and is expected to be restored at approximately 4:00AM Sunday morning, Oct. 10th. As a result of this power outage, the CIT Local Area Network in the Fernwood Building will be unavailable from 6:30 p.m.on 10/9 until 7:00 a.m. on 10/10. Interim measures will be taken to ensure that the 24x7 operations (NIH Help Desk and Network Operations Center) located in the Fernwood building is not interrupted in any way. If you experience any difficulty reaching the NIH Help Desk via phone for some unforeseen reason, you are encouraged to submit requests via email (ithelpdesk@nih.gov) or Internet (http://ithelpdesk.nih.gov). Thank you.
Resolution:  Power was restored successfully according to schedule.
10/1/2004 12:50:05 AM - NIHnet: Internet and ATM Connectivity Issues
Description:   The circuit provider isolated the problem and at this time all NIHnet traffic to the Internet and remote ATM connections are currently running on their primary paths. Original Posting: ******************* The NIHnet primary connection to the Internet as well as our remote ATM connections are currently down. No service is impacted as all traffic is currently traversing backup connections. The circuit provider is currently en route to investigate the cause of the outage.
Resolution:  .
9/29/2004 12:25:09 PM - NIHnet Emergency Maintenance: Routing modification
Description:   In order to meet an HHS directive, CIT has been instructed to perform an emergency maintenance this evening in order to accept new HHS routes. Normally, CIT avoids changes to the perimeter network during the fiscal year moratorium. However, due to this HHS directive, CIT will be performing this maintenance tonight, 9/29/2004, at 5:00 p.m. Please note that this maintenance is expected to have no impact on NIHnet users. Please contact the NIH Help Desk at 301-496-HELP or helpdesk@nih.gov if you have any questions.
Resolution:  Routing modification completed successfully.
9/28/2004 5:15:56 PM - Power Outage in bldg 6011 (Executive Blvd)
Description:   The NIH Help Desk has been informed that there is a power outage in building 6011 Executive blvd. Building Engineers have informed Pepco of this outage. There is no Estimated Time of Repair (ETR). We will continue to provide updates as they become available.
Resolution:  Pepco has restored power to bldg 6011.
9/27/2004 11:39:04 AM - (update) ITAS maintenance this evening from 6:00 - 6:30 PM
Description:   ITAS maintenance has been completed. ITAS is scheduled for maintenance this evening from 6:00 - 6:30 PM for deployment of an enhancement to the Web-based Administrative Officer Module. During this time, ITAS will not be available. We apologize for any inconvenience this may cause you. If you have any questions, please contact Helpdesk at 301-594-6248 or via email at nihhelpdesk@mail.nih.gov <<<<>>>> As always, we appreciate your patience as we strive to improve the system and maintain high reliability. Very Best Regards, ITAS Technical Team, DECA/CIT
Resolution:  ITAS Maintenance is now complete.
9/26/2004 6:53:50 AM - COMPLETED - CIT Aspect Maintenance
Description:   CIT Aspect Callcenter system will be down for maintenance between 7:00 AM and Noon. All calls have been routed to alternate numbers.
Resolution:  Maintenance Completed. All lines routed back to the ACD.
9/24/2004 2:29:01 PM - ITAS maintenance Monday, September 27, 2004, 6 - 6:30 PM
Description:   ITAS is scheduled for maintenance this coming Monday, September 27, 2004, from 6:00 - 6:30 PM for deployment of an enhancement to the Web-based Administrative Officer Module. During this time, ITAS will not be available. We apologize for any inconvenience this may cause you. If you have any questions, please contact Helpdesk at 301-594-6248 or via email at nihhelpdesk@mail.nih.gov <<<<>>>> As always, we appreciate your patience as we strive to improve the system and maintain high reliability. Very Best Regards, ITAS Technical Team, DECA/CIT
Resolution:  To be completed soon.
9/17/2004 7:50:28 AM - (Update) NIH Login is Temporary Inaccessible (Resolved).
Description:   This issue has been resolved. Thanks for your patience and understanding. It has been determined the problem is with NIH Login, not the Portal. The problem also seems to be intermittent. A resolution is being sought. This Hot News will be updated as additional information becomes available. The NIH Portal is temporary inaccessible at the moment. The Portal Admin has been notified and is working to resolve this issue. Thanks for your patience and understanding as a resolution is being sought.
Resolution:  This problem has been resolved. It appears that rebooting of NIHSSOWEB2 resolved the issue, but that has not been confirmed as yet.
9/16/2004 5:56:37 PM - NIH Login servers updated with Windows Service Pack 4
Description:   NIH Login Servers will be updated with Windows Service Pack 4 during the BOSB maintenance window on Friday September 17, 2004.
Resolution:  Problem has been resolved and incident closed.
9/15/2004 9:15:04 PM - Aspect Emergency Maintenance - COMPLETED
Description:   Maintenance Completed: Aspect phone lines will be redirected to alternate numbers begining at 9:00 PM. Customers should be unaffected.
Resolution:  Maintenance Completed
9/13/2004 3:16:15 PM - Resolved: NIH Internet issues
Description:   Resolved: The NIH internet connectivity issues have been resolved. NIH customers may experience Internet connectivity issues. CIT/DNST is aware of the issue and is working to resolve the issue. Updates will be posted when available.
Resolution:  NIH connectivity issues have been resolved
9/13/2004 10:21:59 AM - DCS Web Server Not Responding
Description:   The DCS Web Server is currently in distress. As a result, all DCS hosted sites, including support.nih.gov, training.nih.gov and statsoft.cit.nih.gov are not accessible. ISG staff will reboot the server momentarily.
Resolution:  The DCS Web Server has been rebooted. All sites appear to be working fine.
9/12/2004 1:47:04 AM - BlackBerry outage
Description:   National network outage. I believe we received a warning of this dated incorrectly. (which I copied in th event work log). All BlackBerry Servers have down SRP connections to RIM. Confirmed with RIM this is a planned outage, should last an hour or so. Servers should come back up automatically once their maintenance is complete as the servers constantly ping the SRP connection.
Resolution:  RIM concluded update/repair 6:53 AM
9/10/2004 9:23:36 AM - NIH Survey URL issues
Description:   The NIH Survey URL that was sent in the email from the NIH Director showed in emails as: http://surveysolutions.com/cgi-bin/perseus6.pl/?P=cgstrategy/2004.htm&Qu=NI H The "H" dropped to the second line causing users to be unable to get to the second page of the survey. The URL needs to be entered into the address line as follows: http://surveysolutions.com/cgi-bin/perseus6.pl/?P=cgstrategy/2004.htm&Qu=NIH
Resolution:  Both links now allow the users to advance correctly through the site until completion of the survey: http://surveysolutions.com/cgi-bin/perseus6.pl/?P=cgstrategy/2004.htm&Qu=NIH and http://surveysolutions.com/cgi-bin/perseus6.pl/?P=cgstrategy/2004.htm&Qu=NI H
9/10/2004 7:43:34 AM - RESOLVED: NIH Wireless Network Issues
Description:   The NIH wireless authentication server was reset and connectivity to the wireless network has been restored. ================== Users have reported having problems connecting to the NIH Wireless Network. DNST engineers are currently working to resolve this issue.
Resolution:  Reboot of authentication server restored wireless service.
9/9/2004 8:35:46 AM - NIH Wireless Network Issues - RESOLVED
Description:   The NIH wireless authentication server was reset and connectivity to the wireless network has been restored. ====== Users have reported having problems connecting to the NIH Wireless Network. DNST engineers are currently working to resolve this issue.
Resolution:  Reboot of authentication server restored wireless service.
9/9/2004 7:32:46 AM - UPDATE: Poolesville Network Issues
Description:   The Poolesville circuit is backup. ===== The NIHnet connection to Poolesville is currently down due to a problem with the circuit. The circuit provider has been notified and is dispatching an engineer. There is no estimated time of repair at this time.
Resolution:  The circuit provider restored service to the circuit from Bethesda to Poolesville.
9/8/2004 7:58:14 PM - Done-Remedy System Maintenance 9-8-2004 8:15 Duration 15 Minutes
Description:   ISG will be performing a required maintenance procedure this evening on the Remedy System. The maintenance should take approximately 15 minutes to complete. The impact to users of the Remedy system will be minimal; however, any notifications generated during the maintenance will no contain the correct data. ISG will monitor the notifications during this period to insure that any invalid notifications are updated with the correct information
Resolution:  Maintenance completed successfully. No outgoing notifications were affected. Thank you for your patience
9/2/2004 11:10:26 PM - NIHnet Maintenance: Perimeter Network (REMINDER)
Description:   The CIT/DNST NIHnet Perimeter Network Maintenance completed successfully. ***************************************************************************************** The CIT/DNST NIHnet Perimeter Network Maintenance starts in the next 10mins. ********************************************* Reminder: The CIT/DNST NIHnet Perimeter Network Maintenance will take place this evening, 9/9/2004, from 7:00-11:00 PM. See below for the original posting. Please contact the NIH Help Desk at 301-496-HELP if you have any questions. ****************************************** CIT/DNST engineers will be performing maintenance on the perimeter network on Thursday, 9/9/2004, from 7:00 - 11:00 pm, in support of our efforts to enhance the redundancy and reliability of the NIHnet perimeter network. During this maintenance window, there may be intermittent loss of connectivity as the NIHnet routing converges. This loss of connectivity is expected to be brief with minimum packet loss and data retransmission. Please contact the NIH Help Desk at 301-496-HELP or helpdesk@nih.gov if you have any questions.
Resolution:  Perimeter maintenance was completed successfully.
9/1/2004 1:59:48 PM - (Update) CRIS System Maintenance Completed
Description:   This maintenance has been completed. The NIH Help Desk has been informed that the CRIS system will be unavailable Thursday September 2, 2004 from 1:00am to 3:00am for maintenance.
Resolution:  Maintenance has been completed.

Center for Information Technology
National Institutes of Health
Bethesda, Maryland 20892		 U.S. Department of Health and Human Services
Washington, D.C. 20201