We have recently performed some upgrades on our Cat6000, which serves as the core router in our campus network. It now has dual Sup1A with MSFCs, and the Sup cards now have 128Mb, as we ran out of memory when our VMPS database exceeded about 675 entries. Anyway, as part of the upgrades, I took the opportunity to enable single-router-mode failover on the MSFCs after upgrading the secondary Sup card to 128Mb. I then tested the switchover by pulling the primary Sup card, and upgraded it to 128Mb. The Sup and MSFC failover appeared to work as expected, although there was about 5-7 seconds of interruption in routing, a bit more than I had hoped. The next day I learned that some of our VLANs were not being routed anymore. After checking, I could not see a problem - all VLANs were active on the MSFC. I reset the secondary Sup card to force a failover back to the slot1 sup/msfc. The problem continued. I then undid the single-router-mode configuration and reset both sup cards. At this point everything returned to normal. After some thought, I realized the common link between the VLANs which stopped working is that they all used time-based ACLs. The VLANS without time-based ACLs worked fine after the failover. Also, doing a show access-lists indicated that the VLANs with time-based ACLs *did* work for a period of time after the failover, as there were matches shown on the ACL. Once the VLAN stopped being routed, the match counters stopped incrementing. Here is an example of one of the ACLs applied to a VLAN using time-based entries:
Extended IP access list std-120 permit udp any eq bootpc any (7306 matches) permit udp any any eq 1975 permit udp any any eq 1976 permit udp any any eq 1977 permit tcp any host 10.9.0.2 eq www (4 matches) permit tcp any host 72.2.0.3 eq www time-range Web-120 (active) (160 matches) permit udp any 72.2.0.0 0.0.0.255 eq domain (7109 matches) permit tcp any host 72.2.0.11 eq www time-range Web-120 (active) (209956 matches) permit tcp any host 10.4.0.6 eq 139 time-range Intra-120 (active) permit tcp any host 10.4.0.5 eq 139 time-range Intra-120 (active) permit udp any host 10.4.0.5 range netbios-ns netbios-dgm time-range Intra-120 (active) permit tcp any host 72.2.0.4 eq www time-range Intra-120 (active) (1832 matches) permit tcp any host 72.2.0.4 eq pop3 time-range Intra-120 (active) (20 matches) permit tcp any host 72.2.0.4 eq smtp time-range Intra-120 (active) permit tcp any host 10.4.0.7 eq 139 time-range Intra-120 (active) (1356 matches) permit tcp any host 10.4.0.8 eq 139 time-range Intra-120 (active) permit tcp any host 72.2.0.5 eq 139 time-range Intra-120 (active) (169 matches) permit udp any host 72.2.0.5 range netbios-ns netbios-dgm time-range Intra-120 (active) (3764 matches) permit tcp any host 10.4.0.11 eq 139 time-range Intra-120 (active) (841 matches)
Here are the time ranges: time-range entry: Intra-120 (active) periodic weekdays 6:00 to 23:00 periodic Saturday 6:00 to 23:00 periodic Sunday 6:00 to 23:00 used in: IP ACL entry used in: IP ACL entry used in: IP ACL entry used in: IP ACL entry used in: IP ACL entry used in: IP ACL entry used in: IP ACL entry used in: IP ACL entry used in: IP ACL entry used in: IP ACL entry used in: IP ACL entry time-range entry: Web-120 (active) periodic weekdays 6:00 to 23:00 periodic Saturday 6:00 to 23:00 periodic Sunday 6:00 to 23:00 used in: IP ACL entry used in: IP ACL entry
And finally, here is one of the ACLs applied to a VLAN unaffected by this problem (no time-ranges):
Extended IP access list Staff-200 permit udp any eq bootpc any (189 matches) permit udp any any eq 1975 (10 matches) permit udp any any eq 1976 permit udp any any eq 1977 permit tcp any host 10.9.0.2 eq www permit tcp any eq 139 10.2.210.0 0.0.0.255 permit tcp any 10.2.210.0 0.0.0.255 eq 139 (1 match) permit tcp any any eq pop3 permit icmp any any (38 matches) permit tcp any host 72.2.0.3 eq www permit tcp any host 72.2.0.11 eq www (881 matches) permit tcp any host 10.4.0.6 eq 139 permit tcp any host 10.4.0.5 eq 139 permit udp any host 10.4.0.5 range netbios-ns netbios-dgm permit udp any 72.2.0.0 0.0.0.127 eq domain (107 matches) permit tcp any host 72.2.0.4 eq www (992 matches) permit tcp any host 72.2.0.4 eq pop3 permit tcp any host 72.2.0.4 eq smtp permit tcp any host 10.4.0.7 eq 139 (681 matches) permit tcp any eq 445 72.2.0.0 0.0.0.127 permit tcp any eq 139 72.2.0.0 0.0.0.127 permit tcp any host 72.2.0.4 eq 143 permit tcp any host 72.2.0.4 eq 139 (338 matches) permit tcp any host 72.2.0.5 eq 139 (30 matches) permit udp any host 72.2.0.5 range netbios-ns netbios-dgm (460 matches) permit tcp any host 10.4.0.11 eq 139 permit tcp any host 10.4.0.12 eq 139 (276 matches)
Here is the show modules output from the switch showing the CatOS/IOS versions in use (secondary Sup card removed):
Catalyst 6006> (enable) sh mod Mod Slot Ports Module-Type Model Sub Status
--- ---- ----- ------------------------- ------------------- ---
--------
1 1 2 1000BaseX Supervisor WS-X6K-SUP1A-2GE yes ok 15 1 1 Multilayer Switch Feature WS-F6K-MSFC no ok 3 3 8 1000BaseX Ethernet WS-X6408A-GBIC no ok 4 4 24 100BaseFX MM Ethernet WS-X6224-100FX-MT no okMod Module-Name Serial-Num
--- -------------------- -----------
1 SAD04260GEA 15 SAD04260JH6 3 SAD0429002D 4 SAD03310103Mod MAC-Address(es) Hw Fw Sw
--- -------------------------------------- ------ ----------
-----------------
1 00-02-7e-21-0e-a2 to 00-02-7e-21-0e-a3 3.2 5.3(1) 7.6(3a) 00-02-7e-21-0e-a0 to 00-02-7e-21-0e-a1 00-02-fc-4e-20-00 to 00-02-fc-4e-23-ff 15 00-02-7e-21-0e-a4 to 00-02-7e-21-0e-e3 1.4 12.1(8b)E2 12.1(8b)E20 3 00-30-b6-3c-d2-38 to 00-30-b6-3c-d2-3f 1.3 5.4(2) 7.6(3a) 4 00-30-80-05-1e-28 to 00-30-80-05-1e-3f 2.0 4.2(0.24)V 7.6(3a)Mod Sub-Type Sub-Model Sub-Serial Sub-Hw Sub-Sw
--- ----------------------- ------------------- ----------- ------
------
1 L3 Switching Engine WS-F6K-PFC SAD041701CC 1.1Has anyone seen this before?