Desktop switch kills routing

Hello all,

I run a network where three different lan:s are used. Between the buildings at every site the traffic flows through tagged ports in layer-2 switches. (ASCII-art and switchmodels below) When traffic need to go somewhere outside that site a layer-3 switch routes it onto a carrier network kept separated from the three other vlan:s.

Enabled spanning-tree on all switches to kill off nasty loops.

So far so good.

Then some student connected a simple desktop-switch and made a loop within that little switch. Somehow the spanning tree did not work correctly in that situation. The entire student-vlan stopped dead. While searching for what was going on, the administration people started complaining too; They could reach the local servers, but remote servers and internet was unreachable.

Set up lab to study things a little closer.

Found out that when one of the vlan:s was looping, the other vlan:s worked within that site, but routing soon stopped in the layer-3 switch. The very second i disconnected the offending desktop-switch everything went back to normal again.

Any ideas how to stop this from happening and keep the routing going? The admin-network Must Always Be Reachable, so I dont like the idea that some lousy desktop-switch can wreak such havoc...

TIA

------------

layer-2 switches are D-Link DES-3526 layer-3 switches are D-Link DES-3326S, DGS-3324SR, DGS-3312SR

vlan-10: link-net that connect all sites togehter. vlan-110: students vlan-120: administration vlan-130: public hotspots etc.

(carrier network) | | vlan-10 | __|_________________ | | | switch-1 (layer 3) | |____________________| | | | tagged link with vlans-110,120,130 | ________________|___ | | | switch-2 (layer 2) | |____________________| | | | | | | | | 110 120 130 | | tagged link with vlans-110,120,130

Reply to
nntp chip
Loading thread data ...
Reply to
glen herrmannsfeldt

My first guess is that the student VLAN was completely swamped with a broadcast or multicast packet being repeated at full wire speed by the rogue switch (effectively a traffic generator). And I guess that this broadcast flood drove the router (switch 1) into total CPU overload where it couldn't function properly.

It's difficult to know what to do about that, other than perhaps using a separate router / L3 switch to route between the non-admin LANS and the carrier network, just to separate and protect the admin router. Perhaps with rate limiting.

Steve

Reply to
Steve Horsley

Agreed - i have seen this happen before as well.

it may be the broadcast level generally, or just IP broadcasts and multicasts depending on your layer 3 kit and the traffic profile in the network.

Some layer 2 switches support broadcast rate limiting. The idea here is to limit the number of broadcasts propagated to the rest of the network to limit the spread of this kind of fault.

This does work, but probably isnt turned on by default if available on your hardware, or the level is set so high that the limited stream still chokes the layer 3 switch - experimenting may be the only way to choose a compromise setting.

Alternatively, some layer 3 switches support control plane limiting, intended to limit this kind of denial of service attack

These are the kind of "bells and whistles" features that isnt universal, so may not be useful for your current kit.

Reply to
stephen

Broadcast rate limiting exists, but even when set to the lowest possible limit it doesn't seem to work as intended. Looks like the excessive traffic is not classified as broadcast and thus never triggers the limiting behaviour.

I'll try experimenting with rate limiting for those locations where bandwith is not an issue.

Still some places require huge bandwidh, so another router or layer

3-switch should do the trick there.

Thanks for the ideas.

Do you know any brands/models that use control plane limiting? Might be nice to read up on that.

Reply to
nntp chip

cisco have put this on a lot of their recent stackables, as well as the chassis switches - they also support broadcast and multicast storm control.

here is a link to the Catalyst 3550 with some info in the feature list (layer 3 enet stackable)

formatting link

Reply to
stephen

Cabling-Design.com Forums website is not affiliated with any of the manufacturers or service providers discussed here. All logos and trade names are the property of their respective owners.