Jumbo frames?

In article , John Rowe wrote: :I'm trying to get to the bottom of using jumbo frames on :gigabit. Specifically:

:1. Does it really make a difference to performance?

It can, but it depends a lot on how the applications use data transfers. For example, if 90% of your traffic is telnet and web surfing to outside your LAN, then that last 10% might not be worth the trouble and expense.

There are papers available that discuss the performance of TCP as packet size changes. You can make calculations yourself based upon the size of your data bursts and upon the sizes of all the TCP overheads -- but when you do the calculations, be sure to take into account how systems in *your* network contend for the bottleneck link, together with the packet buffer sizes on the switches / routers, and be sure you use the actual TCP windowing and selective-ack model that you will be using in real life. (e.g., if you have not already enabled selective acks and the larger window sizes, then you might find that turning those on would have a greater benefit than going to a larger frame would have.)

The calculations of theoretical maximum throughput usually are dominated by the latency instead of the bandwidth. Jumbo frames allow you to have about six times as much payload in transit as regular frame sizes, which makes a *big* difference if you have a high latency. If, though, your latency is low enough that the ACK to the first packet would not still be in transit back by the time the sender has reached the end of the TCP window, then frame overheads become the more important factor: the less overhead you have [including intrapacket gaps] to send a given payload size, the better your efficiency.

But... if you are using an OS that does not have an efficient IP driver, or you are using dumb NICs that require the main CPU handle all the fragmentation and checksums and so on, or if you have an inefficient internal architecture, then you will not be able to reach the theoretical maximum. For example, a standard old PCI bus cannot (in practice) handle a full gigabit feed, but the wider and faster PCI busses can. And your disk drive together with your device drivers might not be able to load data from disk at full gigabit speeds.

To use the full capacity of gigabit links, you have to pay attention to the entire system architecture, as -most- architectures have not caught up yet. For an examination of this topic, please look on tomsnetworking.com, which in roughly the April timeframe had an investigation of how various NICs and motherboards combined for gigabit networking.

:2. How to handle older 100BaseT machines? Is there anything better : than having a designated jumbo gigabit VLAN?

As noted above, quite possibly yes. If you haven't turned on selective acks and larger windows, then do so: the performance increase can be substantial.

:3. Do the HP Procurve 28xx/26xx switches support jumbo frames?

Sorry, I don't know. To find out, I would have to google HP's technical documents, which I don't really have time for. I suggest you check out the technical documents yourself.

it may do - apps that move lots of data (i came across this for workgroup video editing for a broadcaster) are often set to use 9k frame size.

be wary of just saying "jumbo frames" - there doesnt seem to be any agreed standards.

For example different cisco switches support max frames sizes from Ethernet up to 9k - but there are 3 or 4 different size limits.

no reason a 100M interface cant support jumbo, but it doesnt seem as common.

most switches only seemed to support 9k frame sizes on Gig ports until recently. Some support different frame sizes on different module types (eg Cisco catalyst 6000)

just about any useful network will have devices in it that dont use / support jumbo frames, so any jumbo frame device needs to work in a "mixed" environment

what this means in practice is that you should have consistent frame size limits in a subnet, and interconnect by a router / layer 3 switch that can do fragmentation, and make sure any device set to use jumbo frames does MTU size discovery, or uses some other mechanism to avoid router based fragmentation wherever possible.

No idea.

Nortel Passport 8600, Cisco Cat 6000, and various Extreme and foundry boxes all support 9k frame sizes though.

A lot of application vendors who insist they must have jumbo frame support to make their software work / efficient / supported - have absolutely no idea what happens when they have to fall back to 1514 byte frames

i suspect this means that the requirement is more an issue of faith than engineering - and as for actually testing it....

As far as I know there is no connection between jumbo frames and gigabit. It can be done it 10 megabit or 100 megabit, too.

If you do frame bursting doesn't that come close, and yet remain standard, though with a little more overhead for the processor.

-- glen

No, not as i remember. This is however a question to be asked to the sales critter, any "slidings" received should be interpreted as "No".

Given that JumboFrames in and of themselves do not increase the TCP window it might be better to put that as "about six times as much payload in transit with the same CPU overhead..."

JumboFrame is (IMO) all about minimizing CPU overhead, although it can also help with masking stuff like DMA latency (as contrasted with stuff like WAN latencies)

If one happens to have switches that do not support JumboFrame, there is a "poor man's" version called "large send" or TSO depending on the context. Some NICs (and OSes) can take larger-than-MTU TCP segments and further segment them based on the link MTU. For the sender, the benefit is just about the same as JumboFrame as JumboFrame end-to-end also minimizes ACKs per byte. However, the receiver still sees a "standard" stream of normally sized segments (which is why it works with switches that don't do JumboFrame and also with clients that don't also exchange the large MSS of JumboFrame).

rick jones

Unless perhaps the stack selects its default window by taking a multiple of the MSS.

rick jones