On December 9th we received alerts that our Lenoir network was offline. Upon attempting to contact the data center, we found that their whole network was offline due to a problem with their upstream providers. In short, it appears that their primary upstream provider had their fiber lines cut at the same time their backup upstream was having maintenance performed. Below are the updates posted publicly by our data center:
Update #1: Multiple upstream providers originating in Charlotte, NC have been affected by a region-wide fiber cut. Our alternate provider originating in Atlanta is also experiencing an issue at present. We are currently using a limited capacity local circuit which has restored some connectivity to the Data Center. We are working with the Atlanta provider to restore connectivity.
UPDATE 2: Our Atlanta based upstream provider has advised us that they are conducting a transport upgrade maintenance that was previously thought not to include the CentriLogic circuit. CentriLogic was not notified of the maintenance window, which was scheduled to last until 06:00 AM EST. We have escalated with the upstream to restore service as soon as practical, but the maintenance is currently underway. They are aware of the unintended impact and will expedite the work as much as possible to restore service.
Outage resolved: Dec 09 2015 03:00:05 GMT-5.0
-The Secure Dragon Staff
Wednesday, December 9, 2015
