concerns about jumbo frames

All you need to do is make sure that your gateway router a) enforces a 1500 mtu on outgoing traffic and b) supports PMTU properly. Then the jumbo frames will stop at the door.

In article , Damian Menscher wrote: :Since our server will also need to communicate with the outside world, :I need to make sure that PMTU discovery can't break. I understand :that it should get back an ICMP fragmentation-needed packet if a :packet encounters a link with a smaller MTU. But how reliable is :this? Are there routers that don't send those back? (I'd expect it

There are, and they do.

There few routers that -cannot- send back the appropriate ICMPs, but there is an unfortunate trend to turn the ICMPs off or to filter them out.

The reasons vary, I'm sure, but one of the reasons is that people are tending to believe that their firewalls or routers should be "invisible to the outside world", so they block outbound icmp. This is of course nothing more than "security through obscurity", but over the course of several years of working in the Cisco newsgroups, I can only recall one time that someone said "Oops, I didn't think about that" and agreed to abandon the idea of "invisibility".

Sometimes the problem is just that people haven't heard about PMTU Discovery, or that it has slipped their mind. If it's just slipped their mind then they usually turn on the icmp as soon as they are reminded, but if they haven't heard about PMTU Discovery then it can be a Problem to convince them that it's a real problem -- *they* have never experienced the problem and so to them the problem must not exist or else must academic (sort of like being told that there is still a law on the books saying that all "horseless carriages" must be proceeded by a person on foot waving a red flag.")

:Is this :something to worry about?

In your situation, I'm not sure it's worth worrying about. I would expect that your border routers are likely to reject the jumbo packets before they get onto the Internet [or Internet II]. In the more general case, though, it continues to be a problem that plagues many people: ADSL providers who use PPPoE are (historically) particularily bad at ensuring that the proper icmp makes it back. It isn't uncommon for ADSL providers to simply tell all their customers that they must lower their MTU, usually to between 1360 and 1492. For example, my residential provider used to use a higher encapsulation overhead than they do now, and at that time if one didn't lower one's MTU drastically, then one could not even successfully load the provider's home page.

You must be aware that _all_ nodes participating in this LAN will have to be configured for (the same) jumbo frame MTU.

Yes, it is something to worry about. And the worries are not under your control - it's routers, loadbalancers, and firewalls in remote places, e.g. in front of

The "normal" 1500 byte ethernet MTU almost always works in those icmp-broken cases. Thus, the easiest way to circumvent problems, is to have a second ethernet link on all machines where you need to communicate with the outside world, running that link with the ethernet standard MTU. If you can't easily put a second interface on the machines, maybe VLAN trunking would work, with one (internal) VLAN configured for jumbo frames, and the other (external) VLAN configured with the standard MTU.

best regards Patrick

I would start some measurment to assure that the current status and sort out any packet drops and other misbehavior. Also arrange a "measurment client" where scripted "typical transactions" are performed. With this as a base you might start testing if jumboframes gives anything. You should also dump the networking gears counters and monitor them during traffic. Any packet drops, buffer shortages or simular will affect your performace. Aslo look out for networking-layer wierdness ( wrong netmasks etc)

Yes, the world outside your LAN will be affected as jumboframes is a "layer-2" thingy, and not standardized in any way.

NFS will be using your jumboframes automatically. This both for nfs/UDP and nfs/TCP.

But as i read that you use linux, you might find the sources of your performace trouble here, depending on kernel version.

and beyond that, the MSS exchange at the beginning of a TCP connection will "cover" 90+% of the situations to begin with - all those remote sites will _probably_ still be on standard MTUs and so will be sending MSS options of ~1460 bytes, which your TCP will be honoring anyway. The time you really need PMTU to work is when you connect to a remote site that is also useing jumbo frame. Then you have what has been called a "dumbell" network - larger on the ends than in the middle - precisely the situation for which PMTU Discovery was invented in the first place.

As others point-out, some folks filter the ICMPs - probably concerned about them being spoofed or something I guess. That tends to break PMTU Discovery - or at least slow things down by forcing extra retransmissions while TCP's walk their PTMU tables.

rick jones