Anyone used Iperf or Netperf w/GigE?

Rick,

I spent a few more hours testing this weekend, including various different sizes for the "RWIN". Increasing it up to 64KB or so made no noticeable difference.

I also tried enabling the "TCP Scaling" (Tcp1323Opts) which should allow the RWIN to be set to greater than 64KB, and then tried various sizes for RWIN. Again, no difference.

I began running Windows Performance Monitor, monitoring "Total Bytes/sec" on the sending machine, and what I was seeing was that:

- There was very low CPU utilization throughout the test (

Hi,

After much testing, I was finally able to get some reasonable results from all 3 network test tools that I had been working with, Iperf, Netperf, and PCATTCP.

What I had to do was to include command line parameters for the following:

MSS: 100000 TcpWindowSize: 64K Buffer Size: 24K

For example, for Iperf sending end, I used:

iperf -c 10.1.1.24 -M 100000 -w 64K -l 24K -t 30

and for Netperf, I used:

netperf-2.1pl1 -H 10.1.1.24 -l 30 -- -s 24000,24000 -m 100000 -M 100000

With these command line parameters, I am now getting results in the 900+ Mbits/sec range, both via the GigE switch and via a cross-over cable.

I'm posting this in case anyone needs this info, and to close off this thread. I'll be posting another msg to start a thread re. "What now?", i.e., what are the implications of these test results.

Thanks for all those who replied to this thread!!

Yours, Jim Lum

Those are truely odd. FWIW, and I suspect the same is true for iperf, what you are calling the MSS is really the size of the buffer being presented to the transport at one time. TCP then breaks that up into MSS-sized segments.

Windows TCP has a bit of a disconnect between SO_SNDBUF/SO_RCVBUF (what netperf sets with -s and -S on either side) and the TCP window doesn't it.

rick jones

Rick,

What did you mean in your last sentence when you said "and the TCP window doesn't it"?

Re. you comments, I'm a bit confused (not by your comments, but just in general).

I was able to get the higher speed results with Iperf first. I found these parameters at:

Then, I proceeded to try to duplicate these results with netperf and PCATTCP, i.e., I did the best that I could to try to use the equivalent parameters to the ones that I used with Iperf. Granted, now that I go back and review the parameters, some of the "translations" were somewhat unclear.

According to the Iperf docs, the "-M" parameter is:

"Attempt to set the TCP maximum segment size (MSS) via the TCP_MAXSEG option. The MSS is usually the MTU - 40 bytes for the TCP/IP header. For ethernet, the MSS is 1460 bytes (1500 byte MTU). This option is not implemented on many OSes."

The "-l" parameter is:

"The length of buffers to read or write. Iperf works by writing an array of len bytes a number of times. Default is 8 KB for TCP, 1470 bytes for UDP. Note for UDP, this is the datagram size and needs to be lowered when using IPv6 addressing to 1450 or less to avoid fragmentation. See also the -n and -t options."

The "-w" parameter is: "Sets the socket buffer sizes to the specified value. For TCP, this sets the TCP window size. For UDP it is just the buffer which datagrams are received in, and so limits the largest receivable datagram size."

It sounds like what you describe in your post as "really the size of the buffer presented to the transport at one time" corresponds to the Iperf "-l" parameter, rather than the "-M", which the Iperf docs say are for "attempting" to set the MSS using TCP_MAXSEG?

Jim

:> Windows TCP has a bit of a disconnect between SO_SNDBUF/SO_RCVBUF :> (what netperf sets with -s and -S on either side) and the TCP window :> doesn't it.

:What did you mean in your last sentence when you said "and the TCP :window doesn't it"?

That confused me a moment too, but I then re-parsed it and understood.

"A has a bit of a disconnect between B and C, does it not?"

In other words,

"I think you will agree that in system A, element B and element C are not related as strongly as you would normally think they would be."

Rick,

I haven't had a chance to try adjust "Description: The number of receive bytes that AFD buffers on a connection before imposing flow control. For some applications, a larger value here gives slightly better performance at the expense of increased resource utilization. Applications can modify this value on a per-socket basis with the SO_RCVBUF socket option."

There's also a "DefaultSendWindow" just below that.

It looks like, from this description, that the SO_RCVBUF is equivalent to the DefaultReceiveWindow and the SO_SNDBUF is equivalent to the DefaultSendWindow?

Jim

On a different note, but related to Iperf....

I found a Linux distribution called Knoppix-STD (security tool distribution). The CD is bootable and it contains the Linux version of Iperf. The host does not even need a hard drive. The CD can breathe life into old workstations and servers by transforming them into traffic generators.

-mike

Rick J>

Yeah, what he said :) Basically, I am accustomed to having setsockopt() calls for SO_RCVBUF, when made before the call to connect() or listen(), controlling the size of the offered TCP window. My understanding is that is not _particularly_ the case under Windows.

rick jones

It might - _if_ the "flow control" being mentioned is between TCP endpoints (at least in the receive case) however, if it is an intra-stack flow control it would be different. One possible experiement is to take the code, hack it a bit to open a socket, connected it to chargen on some other system and see in tcpdump just how many bytes flow before the zero window advertisements start to flow.

rick jones