My Aunt is an Operator and sent this to my brother who is overweight. He sent it to my sister who is having guests on Labor Day. She sent it to me so I would stay home.
======================================================================= HURRICANE KATRINA - UPDATE #18 =======================================================================
The NRRT (Network Response Recovery Team) is monitoring the recovery and restoration activities in the Gulf Region. Here is the latest status on Hurricane Katrina from the Customer Relations/NSM Event Notification team.
Subject - Hurricane Katrina Advisory # 18, (Please note all new updates are in red.)
Date/Time ? Friday, 13:00 CDT, September 2, 2005
Customer Impact -
Out of Service and impaired conditions on the wireline and wireless networks as a result of the 2nd landfall of Katrina in the LA, MS, AL Gulf Coast.
Event Synopsis ?
Sprint Nextel has a wireline switch site offline in New Orleans, a loss of a POP in Biloxi, MS and multiple cell site towers offline due to Hurricane Katrina.
Current Status ?
Effective September 1st, the NRRT team will meet two times per day. An update message to these distribution lists will be sent before
13:00 CDT each day unless conditions warrant otherwise. For additional status updates please access.
EVN5 has been opened for tickets related to the second landfall for Katrina, tracking 1851 tickets and EVN4 is related to SONET node isolation and remains open to track 567 tickets. LEC1 has 9 tickets and has TSP circuits.
General updates and statuses ?
Business Service Assurance Center responsible for wireline service assurance BSAC has a team set up to take internal calls. The purpose is to prevent calls from Sales and Sales Support from competing with the timely servicing of actual customer calls.
IP proactive tickets generation has been off.
The NRRT team is meeting via a conference bridge approximately two times each day.
Sprint NEXTEL's Business Continuity Office, (BCO) is currently monitoring storm, network, and employee impacts.
The CMC Scheduled Maintenance freeze for the following: New Orleans, Baton Rouge, Shreveport, Montgomery, Birmingham, Jackson, Louisville, and the Ft. Worth Distribution Center - in effect until further notice.
The NETS team is also assisting with prioritization on all CDMA site recovery and COW deployment efforts.
Current Network Impact
Please see wireline/wireless summary information below
The most significant network impacts:
The Sprint New Orleans wireline switch is offline. The site has experienced water intrusion. The Switch, Digital Cross Connects, and other network elements are offline. There are 600+ DS3s down, multiple SONET ring nodes isolated, and switched access services from numerous NPA/NXXs impaired. Sprint Nextel 1+ customers and Sprint Local customers home out of New Orleans (Gulf States as far east as the Panhandle and even parts of mid Florida are experiencing fast busies and reorder tones). For dedicated access, customers homed to or through this switch for ATM, IP, Frame Relay etc. are out of service. Sprint has reviewed the facility assessment and has made the decision to move forward with the re-homing of 13 of 14 POP?s currently homed to New Orleans to other switch locations.
The Biloxi POP was damaged and non recoverable. Access Services through and from this POP are out of service. Two SONET rings are down. The replacement building is scheduled to be onsite September
The Kenner CDMA wireless switch is back up as of Wednesday afternoon, August 31st. Sprint continues to experience loss of visibility to the switch and SS7 availability. Local, roaming and LD will fail intermittently until this is fully stabilized. We understand 3G text messaging is working at this time.
Sprint Nextel has numerous cell towers out of service due to power, T1 (or both) issues.
1+ Long distance calling from the local territories in the Panhandle are impacted because of the New Orleans situation.
iDEN reports additional problems in the New Orleans area. Bell South has lost a main Central Office due to generator failure. This has increased loss of cell coverage in the region.
Restoration Efforts
The situation in New Orleans continues to deteriorate with severe flooding, looting, and rioting. Sprint has teams in the area and has secured access into the city to make onsite evaluations. Efforts are being hampered by the rapid changes. Security and safety has become a critical concern for Sprint Nextel personnel in the region. As a result further physical site restoration has been temporarily suspended. Sprint Security is working to provide additional protection.
The wireline Circuit Restoration team is fully activated and prioritizing circuits along these general guidelines:
§ TSP
§ Critical Services
§ High priority business sites
Sprint?s Circuit Restoration Team, along with ASA, BSAC and NSM organization are in the process of identifying TSP and other critical services in the effort to prioritize restoration in the region for all circuits, not just pass through.
o The first Restoration Priority list has been delivered to the field and is posted on the BSAC web site. Priority is based on TSP, Public Safety, ticket severity levels, and field input. The list will be updated COB each day and will be used by BSAC and Network in restoration. The opportunity to restore services is in three areas: Circuits that traverse through New Orleans but do not originate or terminate there Circuits homed to New Orleans that are not in areas affected by the storm. For example the Panhandle of Florida. Re-home efforts to other network elements like Frame, ATM, IP or Voice switches have their own set of challenges with this effort. Circuits that are in the affected areas that are in the impacted Gulf state area will take the longest to restore. Karen Gordey, NSM Network Notification Team, is the lead for the Legacy Sprint wireline and wireless circuits. Account teams can send priority circuits to Karen and include the trouble ticket number, circuit number, and justification for prioritization. (The NETS organization is lead for Legacy Nextel IDEN services, as noted above.) TSP restoral - 93 TSP?s, 56 restored, 37 non-workable. 248 critical circuits have been identified as of this time. Progress has not been captured. These circuits are largely non-TSP health and safety with a few extremely large business circuits. Medium and Routine should be prioritized next. No ETR can be given for circuit restoral at this time. Because of the volume of requests being submitted for priority restoration, new regional / segment POC?s will be established for submissions and escalations. More details forthcoming later today. Trunk re-homing for local territories long distances services to AT&T is reported to have been finished and calls are reporting to be completing at this time. We are in the process of augmenting long distance trunks within the BellSouth territory with multiple IXC carriers. It is anticipated that incremental relief should be felt on 1+ dialing in the days ahead, Toll Free origination relief will not be felt until new access routes are built. We have contacted approximately 2/3 of the independent telephone operating companies on the issue of 1+ dialing. There have been varying degrees of response ranging from the LEC already moving traffic to other carriers and / or performing capacity analysis. The circuit restoration team is meeting hourly for updates, they have assigned a project manager for each of the key areas for restoration. Sales and Sales Support teams should not use order activity to attempt to restore service. These resources are receiving numerous inquiries which is taking away from their daily responsibilities. Additionally this capacity needs to be reserved for TSP and critical services.
Summary Network Impact Data
No new updates on Bell South at this time.
Wireline Network:
Sprint has lost visibility into multiple sites (monitoring/alarm streaming) in the region including because of the Switch outage in New Orleans.
Louisiana
New Orleans Switch has been powered down to protect network integrity.
o 7 New Orleans DACS affecting 602 DS3 are out of service.
o 4 SONET ring node isolation
o 13 SONET rings switch failed
o WATS traffic enabled through IXC carriers, however some impairment still exist .
o Numerous NPA?s and POP?s supported out of New Orleans.
o Physical testing of the fiber plant appears to be intact in New Orleans.
Baton Rouge POP is on commercial power.
Hammond POP is on generator
Lafayette POP is on generator, no monitoring visibility
Kentwood REGEN is on generator, no monitoring visibility
Krotz Springs REGEN is on generator, no monitoring visibility
Alabama
Mobile POP is on generator
Montevallo REGEN is on generator
Mississippi
Biloxi POP / Regen has been declared destroyed.
o 2 Sonet rings down affecting 32 DS3?s 4 SONET rings successfully switched
Jackson POP is on commercial power.
Brookhaven REGEN is on generator
Wireless CDMA Network:
Sprint has lost command and control (monitoring, alarms, etc.) for many of the CDMA access facilities in the impacted area.
There are 324 sites down across the affected area with 182 Sprint and 142 partners.
Wireless iDEN Network:
There are 204 sites down across the affected area with 152 iDEN and
52 partners.
Next Status Update - By 13:00 CDT Saturday, September 3rd.