Hi, since 4 weeks we have much trouble in our mixed switched network. We have Cisco 2950, 2960 and Enterasys VH124-24 switches. All was normal, but then the Cisco 2950 switches have massive performance problems (round trip time > 600ms) between the other 2950 switches. The Enterasys have this problems only from time to time. The 2960 switches are only affected when the 2950 did not respond anymore.
I have the same problem . i.e. loads of throttles in VLAN1. But I still need to figure out what causes this, and if it has any affect/impact of the switchs performance.
Do you have any other model that you could try to replace it with ?
Clear the counters, wait 24 hours:- post sh int vlan1 sh int (busiest interface or few)
Throttles in VLAN 1 will be the result of excessive traffic getting to the vlan 1 interface, i.e. the management interface.
This traffic could be broadcast traffic, or multicast traffic or unicast traffic directed at the mac of VLAN 1.
On a 2950 the CPU is pretty puny and it will not be very hard to overwhelm it.
Check the CPU utilisation. The risk is that the switch will not be able to do housekeeping and will for example fail to process BPDUs which could result in a spanning tree loop. Also the switch may not be able to manage it's MAC table. This will affect learning and forgetting. Otherwise, basic forwarding is done in hardware and will be unaffected.
I recently dealt with a case where every IP phone port had all vlans configured. This resulted in about a thousand STP virtual ports that had to be managed. i.e. BPDU sent every
2 seconds etc. the CPU fell quite a bit when that was fixed:-)
Also - plug in a sniffer (ethereal or whatever) and see what the broadcast traffic looks like.
All you want is a bit of a graph to get an idea if there is anything odd going on.
Do NOT use VLAN 1 for data traffic!!!! I won't say NEVER because if you only have a single switch with a just one VLAN, and just a few megabits/sec of traffic THROUGH the switch then you should be ok. If you have more than one VLAN defined on the switch, NEVER use VLAN 1 for anything. VLAN 1 is used for the control plane, and when you start mixing control and data plane traffic, weird things can happen.
Here is a link to the Cisco LAN Switching Best Practices Guide:
formatting link
Although this document references "Catalyst Networks", 6000/6500 and
4000/4500's, this document applies to ALL switches running IOS.
I am completely aware of the use and recommendation og use for native vlan. I am not using vlan1 for anything but management. I use around 50 VLAN in VTP domain, so it is in the clear of the limits of
64 on 2950's
I just get the throttles in vlan1 on C2950 IOS switches, ONLY on C2950 switches ! i.e. NOT on other models, like 2940's, 2900Xl, 3500XL, 3550's, 2960's 3560's
4500's and 6500's - Just on C2950. Since the original topic/thread was around c2950's, I posted my observations around this model. And again, I am not sure this behavoir has any effect on the switch and/or performance of the switch. I have only observed it, and still need to verify what the root cause is.
So if you guys, have any input, as to what causes throttels in vlan1 on c2950's, please speak up, but please keep the thread on focus. I am running latest IOS.
Throttles are the result of excessive taffic arriving at an interface such that there is insufficient CPU or possibly other resource to deal with it. My understanding is that Interrupts are turned off 'throttled' for a period to give the chance for the CPU do do essential housekeeping.
It will probably take a LOT of digging to get to the bottom of it.
The root cause may not be traffic on VLAN 1 but that the CPU is busy with other things. For example how many STP instances are you running?
How many STP ports are there? sh span summ ! < -- the bottom line has the info.
Are you doing a lot of logging, snmp?
I would start by plotting CPU and throttles via say snmp. If there was a correlation I would then worry about what was causing the CPU.
Here comes sh int vlan 1 and sh int fastethernet 0/25 (the uplink interface):
swXX#sh int vlan 1 Vlan1 is up, line protocol is up Hardware is CPU Interface, address is 000b.fd76.9900 (bia
000b.fd76.9900) Internet address is xxx.xxx.xxx.xxx/24 MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec, reliability 255/255, txload 1/255, rxload 1/255 Encapsulation ARPA, loopback not set ARP type: ARPA, ARP Timeout 04:00:00 Last input 00:00:00, output never, output hang never Last clearing of "show interface" counters 19:45:37 Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0 Queueing strategy: fifo Output queue: 0/40 (size/max) 5 minute input rate 32000 bits/sec, 47 packets/sec 5 minute output rate 33000 bits/sec, 48 packets/sec 4570366 packets input, 373567217 bytes, 0 no buffer Received 1624174 broadcasts (0 IP multicast) 0 runts, 0 giants, 54974 throttles 0 input errors, 0 CRC, 0 frame, 0 overrun, 3485 ignored 3400123 packets output, 290914884 bytes, 0 underruns 0 output errors, 0 interface resets 0 output buffer failures, 0 output buffers swapped out swXX#sh int faste 0/25 FastEthernet0/25 is up, line protocol is up (connected) Hardware is Fast Ethernet, address is 000b.fd76.9919 (bia
000b.fd76.9919) Description: Uplink EIII MTU 1500 bytes, BW 100000 Kbit, DLY 100 usec, reliability 255/255, txload 1/255, rxload 1/255 Encapsulation ARPA, loopback not set Keepalive set (10 sec) Full-duplex, 100Mb/s, media type is 100BaseFX input flow-control is unsupported output flow-control is unsupported ARP type: ARPA, ARP Timeout 04:00:00 Last input 00:00:01, output 00:00:02, output hang never Last clearing of "show interface" counters never Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0 Queueing strategy: fifo Output queue: 0/40 (size/max) 5 minute input rate 35000 bits/sec, 49 packets/sec 5 minute output rate 29000 bits/sec, 41 packets/sec 206884483 packets input, 1775484953 bytes, 36084 no buffer Received 109154222 broadcasts (0 multicast) 0 runts, 0 giants, 0 throttles 0 input errors, 0 CRC, 0 frame, 0 overrun, 36084 ignored 0 watchdog, 5606682 multicast, 0 pause input 0 input packets with dribble condition detected 92983878 packets output, 1445024690 bytes, 0 underruns 0 output errors, 0 collisions, 2 interface resets 0 babbles, 0 late collision, 0 deferred 0 lost carrier, 0 no carrier, 0 PAUSE output 0 output buffer failures, 0 output buffers swapped out
and sh spanning-tree:
swXX#sh spanning-tree
VLAN0001 Spanning tree enabled protocol ieee Root ID Priority 32768 Address 0000.1d36.26ce Cost 200068 Port 25 (FastEthernet0/25) Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Bridge ID Priority 32769 (priority 32768 sys-id-ext 1) Address 000b.fd76.9900 Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec Aging Time 300
Here comes sh int vlan 1 and sh int fastethernet 0/25 (the uplink interface):
swXX#sh int vlan 1 Vlan1 is up, line protocol is up Hardware is CPU Interface, address is 000b.fd76.9900 (bia
000b.fd76.9900) Internet address is xxx.xxx.xxx.xxx/24 MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec, reliability 255/255, txload 1/255, rxload 1/255 Encapsulation ARPA, loopback not set ARP type: ARPA, ARP Timeout 04:00:00 Last input 00:00:00, output never, output hang never Last clearing of "show interface" counters 19:45:37 Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0 Queueing strategy: fifo Output queue: 0/40 (size/max) 5 minute input rate 32000 bits/sec, 47 packets/sec 5 minute output rate 33000 bits/sec, 48 packets/sec 4570366 packets input, 373567217 bytes, 0 no buffer Received 1624174 broadcasts (0 IP multicast) 0 runts, 0 giants, 54974 throttles 0 input errors, 0 CRC, 0 frame, 0 overrun, 3485 ignored 3400123 packets output, 290914884 bytes, 0 underruns 0 output errors, 0 interface resets 0 output buffer failures, 0 output buffers swapped out swXX#sh int faste 0/25 FastEthernet0/25 is up, line protocol is up (connected) Hardware is Fast Ethernet, address is 000b.fd76.9919 (bia
000b.fd76.9919) Description: Uplink EIII MTU 1500 bytes, BW 100000 Kbit, DLY 100 usec, reliability 255/255, txload 1/255, rxload 1/255 Encapsulation ARPA, loopback not set Keepalive set (10 sec) Full-duplex, 100Mb/s, media type is 100BaseFX input flow-control is unsupported output flow-control is unsupported ARP type: ARPA, ARP Timeout 04:00:00 Last input 00:00:01, output 00:00:02, output hang never Last clearing of "show interface" counters never Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0 Queueing strategy: fifo Output queue: 0/40 (size/max) 5 minute input rate 35000 bits/sec, 49 packets/sec 5 minute output rate 29000 bits/sec, 41 packets/sec 206884483 packets input, 1775484953 bytes, 36084 no buffer Received 109154222 broadcasts (0 multicast) 0 runts, 0 giants, 0 throttles 0 input errors, 0 CRC, 0 frame, 0 overrun, 36084 ignored 0 watchdog, 5606682 multicast, 0 pause input 0 input packets with dribble condition detected 92983878 packets output, 1445024690 bytes, 0 underruns 0 output errors, 0 collisions, 2 interface resets 0 babbles, 0 late collision, 0 deferred 0 lost carrier, 0 no carrier, 0 PAUSE output 0 output buffer failures, 0 output buffers swapped out
and sh spanning-tree:
swXX#sh spanning-tree
VLAN0001 Spanning tree enabled protocol ieee Root ID Priority 32768 Address 0000.1d36.26ce Cost 200068 Port 25 (FastEthernet0/25) Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Bridge ID Priority 32769 (priority 32768 sys-id-ext 1) Address 000b.fd76.9900 Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec Aging Time 300
The No buffer is not good. Keep an eye on it as well, and have the switch replaced if possible, to an other model, like 2960. I am sure your problem will go away ...
There are two people with similar issues with 2950.
"boert" has in fact two seperate issues.
(a) VPN interface reporting throttles.
(b) snmp cpu.
Fast Ethernet no buffers.
and 2. are COMPLETELY seperate (maybe)
The reason for this is that the Fast Ethernet no-buffers issue is due to the /hardware/ switching engine being overwhelmed.
The VPN interface throttles is entirely due to the fairly puny management CPU being offered too much traffic.
I put the (maybe) in above when I realised that if the CP?U is not able to manage the forwarding database then the backplane may become stressed. I do not fully understand the
2950 architecture.
Boert,
Reduce your snmp polling. Whatever it is that is going on is taking up more resource than the 2950 can reasonably provide.
Maybe there are devices doing snmp that you do not know about.
Put in Access-lists.
The one thing that you have not mentined is whether you are observing any actual user problems. It is all very well to worry about this and that, but if the users are not seeing problems then spending money may be difficult to justify.
This sort of problem is very hard to deal with and not so many or the people that are available are able to make progress. It may well be best to update the hardware.
In particular the CPU on more recent Cisco proucts tends to have a lot more capacity.
There was no excessive snmp traffic seen in etherreal, but after disabling snmp on the 2950 switches, the ping response and performance is much better. The throttles still apear. This confuses me.
The users had performance- and connection problems with snmp enabled. After disabling I didn't heard about problems.
Cabling-Design.com Forums website is not affiliated with any of the manufacturers or service providers discussed here.
All logos and trade names are the property of their respective owners.