Cisco Wireless -N Home Router WRT120N

Good idea. I'll setup something on the neighbors cable modem in the next few days using WireShark to sniff the traffic.

Look again. If the packets went to the ISP gateway and back, I would expect to see an additional delay to 192.168.100.1. I would also expect to see the default gateway IP in the traceroute path. It would be similar to what I saw with my DSL modem as in:

That's not quite standard behavior, methinks. Some else is happening, but I don't know what. Maybe sniffing will help.

Cool, I'll be very interested to see what you find.

[63.249.85.1]

I've already mentioned that the lack of an entry in a traceroute is not a red flag and doesn't mean anything by itself. The most likely explanation is that the router is configured to leave the TTL alone rather than decrementing it. Many routers are configured that way, so there's nothing unusual there.

That leaves the quick RTT that you're seeing. I don't think that's a valid indicator of "something else". Bottom line, it still looks like standard IP behavior to me. Removing the modem's coax showed it, and I expect sniffing to show it, as well.

Then you don't understand. It's not going to get to the ISPs router and back to the customer in 2ms, is it?

So far, I'm the only one with a plausible theory. Jeff's still working on his, and I haven't heard yours. Care to share?

Jeff, just how far away do you expect the IPS's responder to be? You may recall something called a "radar mile" which is 12.359 usec/nm or

6.673 usec/km out and back. Even if you include the cable propagation factor, the distance delay is lost in the noise. The main factor in the delay is in the operating system, traversing the network stack. The operating system or routing application is doing a lot of other things and can't respond ``instantly'' to an ICMP echo request. When the packet arrives, the CPU has to stop watching the computer pr0n-show it's watching, look at the source and destination addresses, look at the TTL in the IP header, make the "appropriate" routing decision and either forward the packet or create the appropriate reply packet, look at it's routing table, and finally shove the packet out the door. All this takes time. Or do you think that 192.168.1.1 is 1 msec = 1000 usec = 1000/12.36 = 81 nautical miles / 150 km away from this source. (I'm ignoring the time it takes the originating computer to get the packet from the 'tracert' application out onto the wire, and the reply from the wire back up to the application.)

So the gateway is 1200 miles away (well, maybe 840 miles through wet string)? Most of that delay is the network stack. As to why hop 2 appears between hop 1 and 3 here and not above, that's a function of the O/S and routing application. Did the gateway decrement the TTL above? The "Discussion" section of RFC1122 section 3.2.1.7 says it should happen (based on RFC1812 section 5.3.1), but is that happening? Maybe, maybe not. You might get a hint by looking at the arriving TTL on "this" box, but the only way to find out for sure is to put sniffers either side of the gateway. The fact that you're going from a RFC1918 address to a routable one and back to RFC1918 may also delay things.

Old guy

I would expect the latency for traceroute to anywhere to be a minimum of the latency between the local DSL modem and the delays introduced by the Byzantine path the packet follows through the ISP's hardware. I have yet to see it even close to the theoretical propagation delay. For DSL, it has to go through the RT (remote terminal) concentrator, get converted and compressed into whatever fiber or copper protocol is used to get to the DSLAM. At DSLAM, it goes to the network edge Redback router, which eventually hits the ISP's gateway router. In my case, I think it's from Ben Lomond to San Francisco via ATM and from San Francisco to Pentaluma (sonic.net) via ATM. From SF to Santa Cruz on probably ATM where it hits the Cruzio gateway router. I have an Efficient 5260 DSL modem, which can do an ATM ping. Right now, I'm showing 14 msec latency on a 3Mbit/sec DSL line. As I recall, the ATM ping shows about 4 msec, most of which is ATM encoding/decoding. The rest is IP overhead, which you so nicely described (thanks).

My guess(tm) is much of the observed latency is tied up in the gateway router trying to do some manner of intelligent filtering and prioritization (QoS or MPLS QoS). It also takes some time to assemble ATM packets into IP packets and back again. I recall reading a detailed breakdown on where the delays are hiding, but need to get otto here immediately so I separate my customers from their money. More later (if I survive).

I'd tend to agree, although I don't see really long delays.

Heck, I haven't seen that on a cross-over cable between systems, let alone a tunnel from client to local backbone.

6349 Framework for TCP Throughput Testing. B. Constantine, G. Forget, R. Geib, R. Schrage. August 2011. (Format: TXT=62494 bytes) (Status: INFORMATIONAL)

Old guy

2ms is implausibly quick for getting from a computer, to the ISPs router and back again.

My best guess is that the cable modem somehow intercepts traffic destined for 192.168.100.1 [gratuitous ARP? Something non-standard?].

hints at this:

# Some specific brands of cable modem (e.g. Motorola SURFboard, 3Com # Tailfin) take special action to sniff passing traffic from the user's PC, # and intercept packets addressed to 192.168.100.1, so they appear to do the # right thing even when the rules of IP routing are broken, but even this # magic fails unless those cable modems have successfully connected to the # cable operator's network.

I agree. The short latency is sufficient proof that it's not going to the ISP default gateway router and back.

Googling for "modem intercept 192.168.100.1" yields quite a few hits that suggest the same thing. I haven't had time to sniff the traffic yet, but I suspect it won't show anything useful. If the cable modem is intercepting the outbound traffic, then no amount of sniffing between the router and modem will show anything useful.

I tried testing with a Motorola 2210-02 DSL modem. No problems getting to the modem web config through the router. However, the earlier Speedstream 4100 (my favorite DSL modem) seems to have it half way working. I can ping or traceroute to the config IP address (192.168.0.1), but the web browser doesn't show the modem config pages. I have other 4100 modems with other firmware mutations, which might fix the problem:

It might also be an artifact of my static IP address configurations.

More testing....(groan).

[selene ~]$ whatis p0f p0f (1) - identify remote systems passively [selene ~]$

That's really an O/S fingerprint tool - but one of the things it does is to try to guess how far away the remote is by hop-count. Briefly, it looks at the arriving TTL, and guesses from that. RFC0791 Section

3.2 and RFC1122 Section 3.2.1.7 point to the obsolete "Assigned Numbers" RFC (the last being RFC1700) which RFC3232 says has been replaced with an on-line data base. The one you want is which says

IP TIME TO LIVE PARAMETER

The current recommended default time to live (TTL) for the Internet Protocol (IP) is 64 [RFC791, RFC1122].

In fact, many O/S wiggle on this, but with few exceptions, the starting TTL is a binary number - 32, 64, 128 or 255 (yeah). The exceptions are older UNIX versions. So, what's the arriving TTL of packets in question (compared to packets from the modem, and elsewhere)? And what's the TTL and source IP on the ICMP "Network Unreachable" when you try to reach some /other/ non-routable address like 192.168.222.222?

Well, it's better than being picked up by the CHP for DWI 'cause you found this neat winery up the road... ;-)

Old guy

Sufficient proof for you, not for me. In my own case, running a continuous ping to my Comcast gateway shows some results coming back in 3mS, which is not a whole lot different from your 2mS example. (The average is 5mS.)

Obviously.

At least we can put to rest the theories that NAT router make/model have anything to do with it.

I only read a few paragraphs of the article at the link you provided and stopped reading after stumbling across multiple factual and technical errors. Despite the shadow of doubt that places over everything on the page, I concede that the sniffing described there satisfies all of my expectations, conditions, and requirements and may well be exactly what happens.

Good find, thanks for posting.