I'm hoping to deploy jumbo frames on my linux boxes, in the hope of improving NFS performance. My basic plan is to enable jumbo frames on my gigabit machines, making sure that they all have a gigabit path all the way through to the router. My 100BaseT machines would stay with ordinary frames on the same LAN trusting on the OS to negotiate the frame size.
Has anybody any experience of this and did they find any problems?
Thanks
John
I recall reading a posting indicating that jumbo frames for NFS can noticably improve performance.
You will be relying on Path MTU Detection (PMTUD); I expect that should work well in any modern Linux, but I don't have any information as to whether it is enabled by default.
If you happen to start mixing in Windows boxes, then chances are that you will run into problems, as PMTUD does not default to being set. And somehow Windows users end up modifying their TCP settings... possibly as a side-effect of "You are having a problem with Windows or a Windows application? Simply Reinstall Windows And All Will Be Well (SM)"
(snip)
That would make sense for NFS/UDP but it isn't so obvious for NFS/TCP. I am not sure which one is more popular by now.
I have done some, though not this particular one, by having multiple interfaces and using the appropriate address. That is, NFS would address the other end of the gigabit link, where other services would address the 100baseT link address.
I am not so sure about windows, but I think multiple interfaces, each with a different MTU and appropriately addressed, should work.
-- glen
Doubleplusungood. If they are on the same LAN (ie no intervening routers) then UDP traffic from the JF machines to the 100BT machines will magically stop working - there is no negotiation of frame size in Ethernet, and it is only the TCP MSS exchange that kludges around the mismatch of MTU between machines on the same LAN.
And even if there are intervening routers, many times the ICMP messages used in PathMTU discovery are filtered, and even if not, are not always used for UDP communications.
If you happen to have a path with JF on the ends and a 1500 byte link in the middle (a so called "dumb-bell network") of the LAN then even the TCP MSS exchange will not protect you.
In short - _all_ devices (end stations, switches and such...) in a given (v)LAN/broadcast domain must have the same MTU.
rick jones
It makes no sense to use so-called 'jumbo frames' over only part of the path between communicating partners, as you suggest. Your environment would have the clients (100BASE-T machines) using 1500 byte frames, the servers using jumbo frames, and the routers performing fragmentation.
Fragmentation is highly performance-intensive. Few (if any) routers fragment in hardware; even within the software, fragmentation is considered an exception/boundary condition, and is generally not coded as part of the 'fast path'. The idea is that fragmentation should be performed *where absolutely necessary* to provide connectivity, but not in the normal course of events.
Thus, any performance advantage achieved in the servers (due to the use of jumbo frames) would be more than offset by the performance degradation in the routers (due to fragmentation). A number of other factors should be considered:
-By using jumbo frames and fragmentation, you are pushing the performance demands from the servers to the routers. In all likelihood, your servers are higher-performance machines than your routers, thus would be able to handle the issue more efficiently.
-Fragmentation exposes the 'jumbo frame' to increased performance degradation due to packet errors. A single error in any of the fragments will require retransmission of (at a minimum) all fragments comprising the original jumbo frame. Had the message instead been sent as a set of independent IP packets, loss of a single packet would only mandate retransmission of that one packet.
-- Rich Seifert Networks and Communications Consulting 21885 Bear Creek Way (408) 395-5700 Los Gatos, CA 95033 (408) 228-0803 FAX
Send replies to: usenet at richseifert dot com
i see lots of quotes about "jumbo = performance" - but hard numbers seem sort of thin on the ground.
have you got any info from say a manufacturer or a 3rd party test that makes you think this actually will be any faster? if so, it would be very useful.....
i have a similar scheme set up - broadcaster app suppliers seem wedded to jumbo frames.
you need to route between subnets, and as other posters have warned keep any "jumbo" sections of the network fully interconnected - dont forget any backup paths, or the connectivity breaks during a fault for no apparent reason.
it works sort of and can be reliable once you sort it out - but tends to break in all sorts of creative ways.
a couple of the "gotchas": no standard for jumbo frame size (router / switch manufacturers usually something around 9 Kbytes+ (9216 is popular, but not universal), but a lot of Telco suppliers think jumbo means 4 to 5 Kbytes). whenever someonesays they "support jumbo frames" - get it in writing and check they arent just referring to "baby giants" or "baby jumbo frame" - somewhere around 1550 or 1600 bytes. even a single manufacturer may have 4 or 5 different frame size limits on different kit, (or even blades in the same box). interworking between manufacturers is "interesting" - and documentation, support, tools etc are awful. some systems will choke on jumbo frames - printers, sniffers and various NMS have been a problem for me.
Cabling-Design.com Forums website is not affiliated with any of the manufacturers or service providers discussed here. All logos and trade names are the property of their respective owners.