Traceroute anomaly

Cisco's traceroute command is like Unix traceroute, it sends UDP packets. When you get to the target device, the response is an ICMP Port Unreachable. ICMP error packets are normally sent with the source address of the interface they're sent through. So the above behavior is normal if you're tracing to a different interface than the one used to send back to you.

This doesn't happen with Windows TRACERT, because it sends ICMP Echo packets. The rule for replying to Echo packets is to swap the source and destination addresses, so the reply will always come from the address you sent to.

Barry

Thanks for spotting the answer. However, I think "newbie123", as a "newbie"[1], may like a bit more explanation.

As I understand "traceroute", the original "traceroute" is the one you call the "UNIX" traceroute. Back in about 1993, I was tasked to teach a network management program on the IBM UNIX, AIX, and I found the source of this traceroute - in C, of course - on the system. Thus I was sure precisely how it worked. I used to teach an optional ICMP module, essentially rather dry but given greater relevance by explaining the "traceroute" program as the last topic and describing it as "an ICMP application".

Control of the fields of the IP header is required[2] so that the "time to live" field can be set with progressively increasing values starting with 1. Of course it is important to know that the "time" in the "time to live" field is reinterpreted by the IP protocol as each unit representing not a second but a stage between adjacent IP nodes on the packet path over the IP network.

The packet sent is an UDP packet with a to-be-hoped port number which is not open on the destination IP node.

At each IP node where the "time to live" field is decremented to zero, an ICMP packet is returned containing a code which identifies that the destination was unreachable because the "time to live" field had expired. The ICMP packet contains the address of the receiving interface on the IP node, necessarily a router (because the destination behaves differently as we shall see).

The ICMP packet is queued to all processes with an open socket for the ICMP protocol. The traceroute program can pick up the ICMP packet based on the fact that this is an *unsolicited* flavour of ICMP packet and it contains the IP header and 64 bits (8 octets or bytes) of the protocol header of the packet which gave rise to the ICMP packet.

I seem to recall that the IP header "identification" field is seeded with the traceroute program UNIX process identifier so that each traceroute - or any ICMP - program can pick up the ICMP packets which "belong" to that program in the absence of ICMP having a port number for that purpose.

The traceroute program can now display the source address of the router interface together with the time delay.

Recall that the "time to live" field is set to 1 in order to detect the first router, 2 in order to detect the second router and so on.

The traceroute program waits only so long for a response with a particular "time to live" value set. If a response is not received after a certain time, an asterisk is shown in place of the router IP interface address. It's usual to try three times.

When the destination is reached, the UDP packet is recognised as being destined for this node because the destination IP address is matched against one of the node's IP interface addresses, this being a comparison which is always performed before - in nodes which allow onward routing - the packet is passed to the routing table, the "time to live" field having been decremented and compared to zero, of course.

The protocol code and port number are now examined. An ICMP packet is returned containing a code which again identifies that the destination was unreachable but now the code is qualified with the explanation that the destination port for the specified protocol was not open.

As I mentioned above the source IP address of the ICMP packet is the interface where the UDP packet was received as, in the case of the final ICMP packet (port not open), was noted in the original post as supposedly anomalous behaviour.

Ironically, a supposed "traceroute" which does *not* return the IP address of the interface where the traceroute probe packet is received is a "traceroute" which is behaving anomalously!!!

I say "supposed" "traceroute" because, from what you say, it would appear that Windows TRACERT is actually probably PING with the route recording option set - as I learned from exploring the possibilities of the PING command on AIX and happily informing my most interested students - well one in particular whose gratitude I recall!

I believe this "route recording" option corresponds to RFC 1393 dated January 1993. The RFC may be read in order to get a description of the "traceroute" I described above, identified as "Traceroute Today" in the RFC and the newer technique identified as "Traceroute Tomorrow".

The reason that the traceroute program I described may be preferred is that it relies on basic ICMP behaviour rather than relying on routers which understand the proposed RFC 1393 enhancement. The reason the newer technique may be preferred is that it involves less network traffic - but the effect on today's networks is trivial.

I'm told there are other flavours of "traceroute" out there which rely purely on UDP or TCP packets in an attempt to dodge the depredations of overzealous firewall administrators.

Without going through RFC 1393 in detail, from your explanation and reflecting that the response from the destination IP node is directly related to the original packet by having the source IP address and the destination IP address exchanged, it would appear that the received source IP address is the only reference the supposed "traceroute" program has to the destination.

I do hope that helps rather than hinders understanding.

[1] I mused that he/she may have chosen the moniker say 10 years ago and hadn't bothered to change it. I'll take the name at face value and, in any case, maybe the fuller explanation is useful to the silent watching "newbies" if any. [2] I remember this specifically because it stopped me being able to "port" the traceroute program to MVS.

Chris Mas> In article ,

Barry has it right (as he almost always does). Unix traceroute and Windows tracert both operate in essentially the same way.

They send a series of probe packets out with restricted TTL's They look for ICMP "time exceeded" responses from the intermediate gateways. And they look for a different indication from the final destination.

A UDP-based traceroute looks for an ICMP "port unreachable" from the final destination.

An ICMP-based tracert looks for an ICMP echo reply from the final destination.

I'm looking at a packet capture from a Windows tracert. No IP options are set anywhere. The IP record route option is most definitely not in use.

[Binary capture dumbed down to just the essentials]

January 1993. The RFC may be read in order to get a description of the

No. Nothing to do with RFC 1393. "Traceroute tomorrow" appears to have been relegated to the dustbin of history.

Microsoft doesn't need a reason to do things differently than everyone else. But again, RFC 1393 isn't involved.

Google for tcptraceroute. It operates exactly like traceroute or tracert. It sends TCP SYNs as probes, looks for ICMP time exceeded datagrams from the intermediate hops and expects a TCP RST or TCP SYN+ACK from the final destination.

My first attempt to respond didn't appear in the Google archive so I'm trying again. My apologies if this appears twice.

Bri - short for Brian I guess

Hm - checking back on previous exchanges I have had over traceroute I see that I have had something like this discussion before.

I believe my description of what seems universally to be called the UNIX traceroute - to my mind, the "original" traceroute - is solid, explains the reported behaviour with the UNIX traceroute[1] but classifies it correctly as proper rather than anomalous behaviour.

I'm sorry I "muddied the water" with RFC 1393 and the IP "route recording" option.

Do remember that I said I used to teach ICMP and what seems to have happened is that I was so outraged that the ICMP-based "traceroute" violated one of the principle dictates of ICMP that, although I have had this ICMP-based "traceroute" explained to me before, I must just have put it out of my mind.

The cardinal ICMP rule which is violated by this ICMP-based "traceroute" is that an ICMP packet is never - supposed to be - generated when the packet which might give rise to the ICMP packet is itself an ICMP packet.

A clear illustration of the application of this rule is that the author of the "original" UNIX traceroute chose to use an UDP packet as his "probe" packet rather than an ICMP packet - and than had the problem of how to try to be sure that the UDP packet would be rejected at the destination IP node.

RFC 792 states as the last paragraph of the introduction:

The ICMP messages typically report errors in the processing of datagrams. To avoid the infinite regress of messages about messages etc., no ICMP messages are sent about ICMP messages. Also ICMP messages are only sent about errors in handling fragment zero of fragmented datagrams. (Fragment zero has the fragment offset equal zero).

I constructed my presentation on ICMP in 1993 based on information I had to hand which I imagined was the then current description of how ICMP was supposed to behave. It may be that there is some later approved RFC which somehow permits the ICMP-based "traceroute" to operate as you describe - and illustrate.

The most recent RFC regarding ICMP I can find is RFC 4884 (one author being from Cisco) dated April 2007. It specifies that it updates RFC

792 (which applies to ICMP for IP v4 and RFC 4443 which applies to IP v6.)

RFC 4884 mentions qualifying RFCs later than RFC 792. It mentions RFC

1191, the Path MTU Discovery RFC, and RFC 1812, the requirements for IP v4 routers RFC.

In reviewing RFC 1191 I noticed that I did not describe the source address in the ICMP with the code for "destination unreachable" correctly - and Barry did, although his use of the word "through" rather than "from" confused me a bit. It is the IP address of the interface over which the ICMP packet is *sent*, not the IP address of the interface over which the "problem" packet was received. The interface will tend to be - but may not be - the same. This is a key consideration for the operation of the/a traceroute program and relates directly to the subject of the original post.

Interestingly, if the interface is "unnumbered", the router's "router- id" is used, the router-id being the IP address of another interface - if I have understood section 2.2.7 correctly, Again this relates directly to the subject of the original post.

In effect RFC 1812 appears to redefine the ICMP "cardinal rule" - and may well perform the same "service" for other protocols for all I know

- by, when describing when an ICMP packet should *not* be sent - section 4.2.3.7, slipping in the adjective "error" - in reference to

*unsolicited* ICMP packets - in

An ICMP error message MUST NOT be sent as the result of receiving:

o An ICMP error message, or ...

Interestingly, nowhere does it appear to justify this exception with reference to the requirements of traceroute - but I expect that is what is in mind. I imagine that it is logically considered sufficient that the requirement to avoid ICMP "storms" is still met by allowing

*unsolicited* ICMP packets - described as "error" ICMP packets in RFC 1812 - to be sent in response to problems with *solicited* ICMP packets - described as "query" ICMP packets in RFC 1812.

I guess it's just possible that there are still routers out there which are not aware of Mr Baker's embellishment of RFC 792 as hidden away in RFC 1812 - despite the June 1995 date. Such a router will not feature in an ICMP-based "traceroute".

[1] Just to state again in order to "clarify the water":

The specified destination address is used as the actual destination address with the ICMP-based "traceroute" because it is taken from an ICMP response packet, which has had the source and destination addresses exchanged, to the "probe" ICMP packet - that managed to reach the destination IP node.

The specified destination address *may be* used as the actual destination address with the UNIX traceroute if it happens to be the IP address of the interface over which the final ICMP packet with the code for "destination unreachable" is sent.

This is just a restatement of the key point in what Barry said but I hope it now has a context which was the purpose for my earlier post.

Chris Mas> >

target. TTL = 1

target. TTL = 1

request to target. TTL = 1

request to target. TTL = 2

to target. TTL = 2

request to target. TTL = 2

The change in ICMP processing requirements appears in RFC 1122, October

1989, with explanation. See section 3.2.2. RFC 1122 is part of the requirements spec for all Internet hosts, so everything designed in the last 18 years or so should behave this way.

Sam

Sam

RFC 1122, "Requirements for Internet Hosts - Communication Layers", concerns the behaviour of IP end nodes - "hosts" - rather than IP network nodes - "routers" - which is what concerns RFC 1812[1], "Requirements for IP Version 4 Routers". The behaviour which is of interest regarding this disputed "change" to ICMP processing concerns the IP *network* nodes. However thanks for the reference.

I scanned the ICMP portions of both RFCs and there seems to be quite a lot of "copy and paste" here, presumably from RFC 1122 to RFC 1812. What *both* RFCs are doing it seems to me is summarising what a good "host" (RFC 1122) or a good "router" (RFC 1812) should do - indeed may be *required* to do - with reference to a spectrum of protocols, without actually claiming to redefine any of the protocols.

Thus I now believe I have allowed myself to be distracted by the apparent revision of RFC 792 by RFC 1812 (and 1122). RFC 1812 doesn't necessarily redefine RFC 792 at all, it merely states that a router ought not to create an ICMP packet for a fault with an ICMP "error" packet. It does not say anything in that bullet - obviously - or the remaining bullets in the list describing the conditions under which an ICMP packet ought not to be created - about whether or not an ICMP packet should be created for a fault with ICMP packets other than an ICMP "error" packet. This being so, RFC 792 still applies where it states that an ICMP packet should not be created for a fault with

*any* ICMP packet.

So you see I was wrong and you are wrong - although I guess I may have led you into this error - I'm having to apologise yet again - in considering that ICMP had *changed* in some way - officially anyhow.

And so my original objection to the ICMP-based traceroute stands despite RFC 1812 (or RFC 1122).

The ICMP-based traceroute relies on undocumented behaviour no matter how many different types of routers actually implement this undocumented behaviour.

It may be sensible for anyone who normally uses an ICMP-based traceroute to get hold of an implementation of the "original" UDP- based traceroute so that, if irritating asterisks appear in some of the router "hop" positions using the ICMP-based traceroute, the UDP- based traceroute can be employed in order to try to fill in the gaps. It may even be more sensible *always* to try the UDP-based traceroute first since that program's design follows "standards" which are more likely to be being observed.

Putting some effort - if not my money - where my mouth is, I'd better check the various - free - traceroutes I've managed to discover on the web for use with my home Windows PC to see which is the best of the UDP-based breed.

Chris Mason

[1] Incidentally, I *do* like the "Acknowledgement" in RFC 1812. Already the number recalls the Battle of Borodino, Napoleon's retreat from Moscow and Tchaikovsky's "pop" "Overture", the Russian anthem subduing the French - to be compared with Beethoven's equally "pop" "Battle Symphony"[2] commemorating the Battle of Vit(t)oria the following year, "Rule Brittania" subduing "Malbrouque"[3], the famous

- then - French martial tune - although it actually refers to the British victor of Blenheim a century before.

But I digress. The text from Henry V is presumably the RFC author's favourite motivational speech - mine too or I wouldn't bother to mention it. How delightfully the word "gentleman" is defined towards the end. Note also what a well constructed structure the speech has.

Blenheim, Borodino, Vitoria and Agincourt in one footnote! I'd better confess I live in Waterloo.

[2] Originally composed for a "panharmonicon", a "mechanical orchestra" invented by Johann Nepomuk Maelzel - who also invented the metronome. Thus it can be coded as a MIDI file quite successfully: - Wellington's Victory Op. 91[3] [3] If you think you hear "For he's a jolly good fellow"[4], you're right but it was known as "Malbrouque" (or "Malbrouk") in 1813 and was associated with the French army. [4] Also known as "The bear went over the mountain" in some parts of the world.

Well, then let me direct you to section 2.2 of RFC 1009 "Requirements for Internet Gateways", the precursor to RFCs 1716 and 1812. This section states:

... The ICMP messages are grouped into two classes: error messages and information messages. ICMP error messages are never sent about ICMP error messages, nor about broadcast or multicast datagrams.

That dates from June 1987.

See above wording of RFC 1009. Whilst 1009 (and indeed the other RFCs we've referenced) predates the MUST/SHOULD/MAY [NOT] wording standard set by RFC 2119 it's clear that in modern parlance ICMP messages MUST NOT be sent in response to ICMP error messages. The silence on what else can happen implies that ICMP error messages MAY or SHOULD be generated in response to ICMP information messages. If 1009 and its successors meant that ICMP error messages should not be sent in response to any ICMP message they would have said so.

From the above it should be obvious that I disagree. :-) But I'll leave it there.

Sam

Sam

You may be prepared to leave it there but I can't possibly let a mistaken approach to how standards should be defined go unchallenged. "Silence", "implies", "nudges", "winks, "know what I mean, 'squire" isn't enough.

RFC 1812 states specifically that ICMP is defined in RFC 792, its internal reference being [INTERNET:8]. It follows this with the sentence "A router MUST support ICMP". I'm now going to rely on "silence" in a positive way: it does *not* say that the current RFC changes RFC 792 in any way.[1]

Reading some of the ICMP section in RFC 1812, I get the impression that it is concerned to prevent mistaken interpretations of RFC 792 by dealing with misinterpretations of how to handle ICMP. For example, it would appear that the "bright sparks" responsible for one implementation of ICMP decided to "copy" the TTL field from the incoming packet.

In the section which mentions when an ICMP packet MUST NOT be sent, there is a "Discussion" paragraph or two which states that the concern addressed by the "MUST NOT" rules is to avoid "broadcast storms". This concern is addressed in the case of ICMP messages by insisting that the "error" type of ICMP is included in the rules. This is perhaps again to deal with "bright sparks" who have misinterpreted RFC 792. The "information" type of ICMP is not of concern when "broadcast storms" are to be avoided[2] and so no mention is made of the "information" type of ICMP.

Just to be sure what I'm saying is understood, you CAN NOT use RFC

1812 as a "green light" for describing a problem with ICMP "information" packets using ICMP "error" packets.

I also checked RFC 1122 briefly and, being much the same as RFC 1812, it also doesn't change RFC 792 in the matter of describing a problem with ICMP "information" packets using ICMP "error" packets.

Don't you think it strange that the vendor best known for implementing routers, namely Cisco, should implement its traceroute using the UDP- based version rather than the - if the RFC 792 rule can really be ignored - more logical ICMP-based version? I say "more logical" because the UDP-based version is obliged to cross its fingers and hope that the UDP port it uses does not happen to be in use by an application at the destination IP node.[3]

Let mew direct *you* to the following thought regarding RFC 1009. I am prepared to accept that the rule in RFC 792 is, in retrospect, a mistake. However, unless, it is *explicitly* revoked in the case of ICMP "information" messages, relying upon the router vendor having

*assumed* it has been revoked is - using legal terminology - "unsafe". RFC 1009 missed the rather obvious opportunity to introduce the revoking text in order to rectify the mistake.

Chris Mason

[1] Except in the sense that some new codes are *added* as I found was the case with the "partner" RFC 1122. [2] I have discovered that I can create a bit of a "storm" of ICMP "information" packets by pinging to a "broadcast" address on a LAN and thereby oblige every active IP node on the LAN to respond! [3] Perhaps there is a standardised port number now def> >

I'm sorry you see it that way.

So you can use silence to support your point of view, but I can't? Doesn't seem entirely fair.

Indeed, that's exactly what it was trying to do - standardise the handling of ICMP messages. If you see the original traceroute man pages (e.g. ) and look at the paragraph beginning "The silent gateway 12..." you'll see an example of the work of the "bright sparks" you refer to above.

It was to prevent a storm of error messages being raised in response to error messages. The same kind of thinking has to go into the design of mail systems.

Sorry, I really don't understand that point.

"Some of the text in this memo has been (shamelessly) plagiarized from earlier documents, most notably RFC-1122 by Bob Braden and the Host Requirements Working Group, and RFC-1009 by Bob Braden and Jon Postel. The work of these earlier authors is gratefully acknowledged." (RFC 1812, Acknowledgements)

Absolutely not. It's a matter of history. The original traceroute (for Unix, so far as I know) was and is a shameless hack - it uses features of the protocol for purposed for which they were not designed. Good for it (and Steve Deering and Van Jacobson - see the man page). The ICMP-based tracert appeared with some version of Windows. It arrived at a time when firewalls were beginning to appear and UDP was less likely to arrive at its intended destination. Whether that was the intention, or whether Microsoft was having an attack of NIH syndrome (remember Bill thought the Internet was a passing fad) I don't know. There are now TCP-based traceroutes too, so you can point them at 80/tcp and they'll go almost everywhere. Cisco's implementation predates the Windows one by many years so it's not surprising it's more like the original Unix version.

I'm sorry, but I think you're just being silly. Read RFC 1812, section

4.3.2.7. It specifically calls out ICMP error messages as ones to which a further ICMP error message MUST NOT be sent. If the authors had meant to say that ICMP error messages MUST NOT be sent in response to ANY ICMP message they would have said so. Therefore it is permissible to send ICMP error messages in response to ICMP information messages. Q.E.D.

Gosh, you must be the first to have noticed that. :-) IIRC Novell's LAN Workplace product used to respond with the broadcast address as the source, which was marginally confusing the first few times you came across it.

Not that I know of either. Anyway traceroute uses a range of ports.

Sam