Ethernet switch flooding packets?

(snip)

Certainly 5 or 10 packets/second won't overload the network software.

You might check the MAC address cache size in the switches. Even if you have more hosts, it is likely that they won't all be in heavy use at the same time.

It might be that the backbone links have somewhat more load, but

I would probably suggest more study before deciding on L3 switches.

-- glen

Define "huge" :) The PDF datasheet for the 2610:

I think you should call them routers :)

rick jones not actually in hp procurve...

BTW, a web search for "procurve mac table size" seems to be reasonably effective at getting decent hits across the procurve family.

rick jones

So it's OK (and, I guess normal and accepted) that a switch is (probably) flooding unicast packets (that are, of course not broadcasts)? I thought this was a no-no? How common is this?

Hi Rick,

Thanks for your reply. I'm aware of the 8K address table, but I'm being a little vague because I'm not part of the networks team and don't have all the specifics; - I believe we use mainly 26xx at the edge, but I'm unsure what we use at the distribution layer - I know roughly how many hosts we have on the main VLAN, but not across all the (many and varied) secondary VLANs

Rick, is it normal and acceptable for a network to be flooding unicast packets? (I accept it's how it's designed, but didn't realised it was OK)

snipped-for-privacy@googlemail.com wrote: (snip)

I think 10/sec is not considered a flood. Cache entries time out, and packets are sent to all ports until the entry is restored.

If the destination is down, and doesn't reply, the entry will never be added to the cache. Broadcasts are sent to all ports, by definition.

-- glen

Depends.

A switch will flood unicast when the destination address is not in the learned address table of the switch.

An address enters those tables when it is seen as a source address, and leaves the tables after some time when not seen again as a source address of other packets.

Common problem 1, in my experience: the timeout for the switch address table is shorter than the ARP timeouts used by some of the communicating hosts, and such hosts do unidirectional messaging. Initially, the destination host for such messages will show itself with an ARP reply, populating the switch address table, and messages to it will not be flooded. But with unidirectional traffic, that destination host does not send anything back. So its address will timeout in the switch address table eventually. From then on, the unidirectional messages will be flooded. A periodic ARP request from the source host, even unicast, will elicit further ARP replies from the destination, again populating the address table.

I've seen two different cases of such unidirectional messaging causing flooding. One was Cisco netflow export data. The other was in the context of a load balancer setup with triangular dataflow, i.e. one VLAN transports requests from Internet through loadbalancer to some server, but the replies of the server flow back to the Internet on a different VLAN or LAN. I think that similar things could happen with host based ethernet adapter "teaming" with two host ports connected to different switches in the same switch cloud.

Such problems are fixed by either making the sending hosts periodically ARP more often (the loadbalancer, in my scenario above), or by making the timeouts of the switch address tables larger (than a common timeout on all the hosts. we set it to 999 seconds).

best regards Patrick

Hi Everyone,

Thanks again for your replies.

Glen, I acknowledge 10 frames/sec wouldn't be considered a "flood", I was merely using the Ethernet switching terminology - if a switch doesn't know which individual port to push a frame out to, it will "flood" them to all ports. I've only done a short capture at one location at one time, so the magnitude of the problem is certainly inconclusive.

Patrick, I don't think this is a switch address timeout. In the frames that I saw, I saw some packets from my PC to a server on the other end of a router, and the corresponding replies. Hence "my" switch has "learnt" that my MAC address is on my port, and the router's MAC address is on the uplink port. If I exchanged no more traffic for the timeout period, then my address would of course timeout and I would be flooded. HOWEVER, just a few packets (milliseconds) later, the switch is forwarding unicast packets on my port that aren't for me. I think this is because - the switch as 8K MAC address table. - every PC on each port gets added to the table when it transmits (as it should) - however, because there are many thousands of MAC addresses on the Ethernet fabric, they are all being added as belonging to the "uplink" port - these are replacing the entries in the MAC address table about which PCs are on the "local/edge" ports - this is happening so fast that I'm recieving unicasts addressed to other MAC addresses on my computer, even though I just transmitted milliseconds ago

Hope this is making sense!

Kind regards,

Anwar

snipped-for-privacy@googlemail.com wrote: (snip)

Yes, I was just trying to indicate the scale involved. It would sound funny to call it a trickle in the ethernet standards.

Yes, but so are all the other ports. Note that the switch, by design, doesn't assume one host per port, even though, as you say, it isn't an uplink port. When the MAC address times out for another port, even the uplink port, those packets are sent to you.

Are there more than 8000 on the subnet (or VLAN)? In the first post, it sounds closer to 1000 than 8000.

Putting your MAC in the cache doesn't stop others from being sent to you. Only putting the other host in the MAC cache for a different port will do that. As I said, one way that happens is if the host goes down and stops replying. Until it is out of the ARP cache, the packets will still be sent. When it is out of the ARP cache, ARP broadcasts will be sent, and you will see those.

The real answer is that if you really have 8000 hosts then yes, you should go for L3 switches or routers to subnet the network. Many people like to keep it below 250 (for a convenient netmask) but for reasonably traffic levels and speeds 1000 is probably fine. Maybe 2000, but not 8000.

The reason for the change is not that a few packets leak through, but that it really is too crowded.

-- glen

Hi Glen,

Thanks for your reply. The number of Ethernet hosts is around 5K on the primary VLAN, but each switch is managed (via a separate VLAN, of course) so will have it's own MAC address. I don't know how many other VLANs there are are or how many hosts each has.

Interestingly, I've just done another capture (it's about 10:15am, so a peak time on the network; yesterday I did it around 6.50pm) and I'm seeing no "foreign" unicasts *whatsoever* - very peculiar.

Kind regards,

Anwar

Glen Herrmannsfeldt wrote:

Firstly:-

- a few leaked unicasts a second are of no concern to me at all.

- a network of 5000 hosts in a broadcast domain *is* of concern. It is not *necessarily* incorrect and may be well designed taking into acccount the known behaviour of the hosts and the network equipment however it would be rather unusual to say the least.

So:- Yes it is OK to see some flooded unicast packets - in the sense that it is inevitable in some cases. It happens by design.

Some considerations with regard to the number of hosts in a broadcast domain are -

Resultant 'background' broadcast level -verses- - capacity of weakest device to process the 'unwanted' broadcasts - capacity of slowest link to carry 'unnecessary' broadcasts

If you are using a network with spanning-tree controlled redundant links then the behaviour when a link fails might be of concern. On a topology change: - Forwarding database (fdb) aging time is reduced to 15 seconds for a time.

- Switches may have to re-learn the whole fdb and switches have a learning rate that may be dependent on the CPU. During learning unknown unicasts are flooded. With such a large number of hosts it may take some time to re-establish the fdb.

Finally: Such a design is on current terms not normally 'good' practise however without knowing much, much *much* more about it is not possible to say whether the design is satisfactory or not. The best measure of all of course is *does it work*.

My guess would be that if it does work every day then it has been well designed. If such a network had been thrown together there seems a decent chance to me that it might break from time to time.