As of 2:00 pm, Merit reports that service has been restored, but
there are still problems on the Extreme Aspen switches. One of
the redundant network links between Michigan State University and
Wayne State University has been disabled until repairs are made,
but service should remain stable.
Doug
----- Forwarded message from Doug Nelson <[log in to unmask]> -----
Date: Tue, 21 Aug 2007 12:52:20 -0400
From: Doug Nelson <[log in to unmask]>
Subject: Re: [MSUNAG] Merit about yesterday's trouble
To: [log in to unmask]
For those of you who didn't attend Merit's web conference, here's a
short update. Merit engineers traced the source of the Merit Network
problems to the failure of an optical amplifier within the MiLR network,
following scheduled maintenance by the fiber carrier. This failure
created a marginal optical fiber path, and accounted for the network
difficulties seen in the original outages between Thursday and Sunday,
and again this morning, after a second scheduled fiber maintenance
window. The fiber path failure was compounded by a software failure
which repeatedly attempted to bypass the marginal link, and then rapidly
restore the link to service. This process also caused the errant
switches to exhaust their memory due to a second software failure,
leading to additional packet switching and routing failures, cascading
throughout the Merit Network.
Once the problematic link was disabled, service returned to a somewhat
stable state, but on limited bandwidth.
As of 12:00 noon today, the Merit network appears to be operating
normally and at full capacity, but we are awaiting final word from
the Merit field and operations engineers on what was done and whether
service is fully restored.
I will pass on additional details once Merit releases them.
Doug
--
Doug Nelson, Network Manager | [log in to unmask]
Academic Computing and Network Services | Ph: (517) 353-2980
Michigan State University | http://www.msu.edu/~nelson/
----- End forwarded message -----
--
Doug Nelson, Network Manager | [log in to unmask]
Academic Computing and Network Services | Ph: (517) 353-2980
Michigan State University | http://www.msu.edu/~nelson/
|
