Frustrated that I don't UNDERSTAND why my network times out

Have a question or want to start a discussion? Post it! No Registration Necessary.  Now with pictures!

Threaded View
Why can't I connect (via port 80 or any port) to a certain web site?

For more than a year I've had the same problem, and, it's NOT
the way I'm running traceroute! (e.g., it's not ICMP vs TCP, etc.).
It's also not that the server I'm pinging is down, or slow.

There's something wrong with "my" home networking setup.  
But what?

I just want to UNDERSTAND the problem. That's it.
It makes NO sense what I've been seeing over the past year.

Basically, for months at a time, I can't connect to centos.org
and, for months at a time, I can connect to the web site.

When I can't connect, traceroute (ICMP or TCP) fails to connect;
when I can connect, traceroute also connects.  

So, it isn't HOW I'm running traceroute, as traceroute is telling
me exactly what Firefox is telling me.

This happens for months at a time, and has happened about five
times in the past two years.  

I change NOTHING (not my router firewall, not my computer firewall,  
not my networking setup, etc.) in the interim.

When this happens, I switch to TOR, and I can EASILY connect to  
centos.org via the proxy Firefox - so there's nothing wrong with
my firewall or with my home broadband router (as far as I can tell).

When I can't connect, I ask my NEIGHBORS who "can" get to centos.org
to show me their traceroute, and it looks the same as mine except
for the fact that their times are slightly faster and they get  
past that last hop - whereas mine dies at the penultimate hop.

So, THAT would implicate something on "my" side (but what?).

I switch to Knoppix 7, and I get the same result.
I go to a Windows PC, and I get the same result.
So, it's NOT the PC!

If I knew how to get around my router, I would, but it has
all the setup for the ISP (it's a WISP, not cable or DSL).

My question?  

How can I debug WHY (for months at a time), I can't get to a web site?

Here's a traceroute run just now:
knoppix@Microknoppix:~$ traceroute www.centos.org
traceroute to www.centos.org (72.232.194.162), 30 hops max, 60 byte  
packets
 1  192.168.1.1 (192.168.1.1)  2.835 ms  2.809 ms  20.293 ms
 2  REDACTED_WISP.net (xxx.xxx.xxx.xxx)  20.280 ms  20.265 ms  20.248  
ms
 3  10.50.0.1 (10.50.0.1)  29.973 ms  29.959 ms  29.943 ms
 4  10.25.0.1 (10.25.0.1)  39.067 ms  42.759 ms  42.745 ms
 5  10.20.0.1 (10.20.0.1)  82.295 ms  82.280 ms  82.265 ms
 6  10.0.0.1 (10.0.0.1)  122.956 ms  159.675 ms  159.654 ms
 7  69.36.226.193 (69.36.226.193)  198.537 ms  201.445 ms  201.433 ms
 8  vl2.core1.scl.layer42.net (69.36.225.129)  201.423 ms  201.412 ms  
201.388 ms
 9  216.156.84.141.ptr.us.xo.net (216.156.84.141)  201.377 ms  201.361  
ms  201.346 ms
10  207.88.14.233.ptr.us.xo.net (207.88.14.233)  239.215 ms  239.185 ms  
239.171 ms
11  vb15.rar3.dallas-tx.us.xo.net (207.88.12.45)  239.137 ms  239.122  
ms  239.061 ms
12  207.88.14.34.ptr.us.xo.net (207.88.14.34)  239.030 ms  123.544 ms  
178.276 ms
13  207.88.185.74.ptr.us.xo.net (207.88.185.74)  178.261 ms  178.264 ms  
178.231 ms
14  border1.pc2-bbnet2.dal004.pnap.net (216.52.191.81)  178.234 ms  
border1.pc1-bbnet1.dal004.pnap.net (216.52.191.19)  178.187 ms  
border1.pc2-bbnet2.dal004.pnap.net (216.52.191.81)  178.199 ms
15  layered-11.border1.dal004.pnap.net (63.251.44.74)  178.171 ms  
178.139 ms  178.123 ms
16  * * *
17  * * *
18  * * *
19  * * *
20  * * *
21  * * *
22  * * *
23  * * *
24  * * *
25  * * *
26  * * *
27  * * *
28  * * *
29  * * *
30  * * *

I know, from two years of experiencing this, that the hop after the  
last hop showing resuls "is" Centos.org! So, when it works, it gets to  
the last hop; but when it dies, it always dies at just before the last  
hop. But why?

Can you help me UNDERSTAND why/how this situation can be happening?
Note: All other web sites work just fine.

NOTE: I already know that YOU will be able to access this same site  
with much lower ping times (you're not on a WISP either) - but that  
doesn't help ME figure out what the problem is.

Is there freeware extant to help me UNDERSTAND why this happens to me?

Re: Frustrated that I don't UNDERSTAND why my network times out
On 8/10/2013 2:32 PM, billy wrote:
Quoted text here. Click to load it
[SNIP]

There is no freeware, or any sort of software available to you, that can  
help with your problem.

There is a "black hole" between you and centos.org.

Packets go in, but do not come out, that's what the traceroute is  
telling you.

Contact your ISP, and provide them with the traceroute, they then need  
to pass that to their (various) upstream connections to get the problem  
solved.

I would assume that the problem lies with "pnap.net", who- or what-ever  
they are, but they probably won't talk to you.

Your WISP appears to be connecting to "layer42" (69.36.226.193) as their  
gateway to the internet, again, they won't talk to you, but your ISP  
should be able to get them off their arses.

    Cheers,
        Gary    B-)

--  
When men talk to their friends, they insult each other.
They don't  really mean it.
When women talk to their friends, they compliment each other.
They don't mean it either.

Re: Frustrated that I don't UNDERSTAND why my network times out
Gary R. Schmidt wrote:

Quoted text here. Click to load it

All this makes perfect sense, except ...  

Except my neighbors, on the same WISP, can get to centos.org.

So, it must be something in 'my' setup; but where?

More specifically, how do I diagnose to pinpoint where?


Re: Frustrated that I don't UNDERSTAND why my network times out
billy wrote:

Quoted text here. Click to load it
Is it an MTU/fragmentation issue? (Check out ping -M)

Dave


Re: Frustrated that I don't UNDERSTAND why my network times out
Quoted text here. Click to load it


This is exactly what I would have suggested. Data packets have a maximum
size dependent on the transport layer carrying them. The default size is
typically 1500 for ethernet, and a little less for connections running
over PPP and/or VPN. Some long distance WAN links can have even lower
maximum packet sizes. If a packet cannot be transmitted in its entirety,
it can be split (fragmented) unless the sender has specified that it
must not be split. If it can't be split then the sender is responsible
for transmitted the data in smaller sized packets, but obviously the
sender needs to be informed that the packet size must be reduced. If
there's a dubious firewall somewhere between you and the target -
one that (incorrectly) eats the ICMP fragmentation request packets -
then your sender can't realise that it needs to reduce the packet size,
and such packets inevitably get dropped.

You can test this with ping -M, as David Hough has suggested. You can
also reduce your own MTU and see whether this "fixes" the problem. Try
"ifconfig eth0 mtu 1400", and experiment with different values.

Chris

Re: Frustrated that I don't UNDERSTAND why my network times out
On 8.10.13 9:32 , Chris Davies wrote:
Quoted text here. Click to load it


TCP should be able to find a suitable segment size, but it needs
an ICMP message for the functionality. There are sysadmins killing
all ICMP, in an attempt to hide from ICMP echo (ping). This could
be the cause here.

--  

Tauno Voipio


Re: Frustrated that I don't UNDERSTAND why my network times out
On Tue, 08 Oct 2013 23:16:05 +0300, Tauno Voipio wrote:

Quoted text here. Click to load it

What I don't understand is whether the web browser, which is the  
problem observed, is using ICMP or TCP.

Q: What is the web browser using? (ICMP? TCP?)


Re: Frustrated that I don't UNDERSTAND why my network times out
billy wrote:
Quoted text here. Click to load it

TCP.  But the 'process' involves TCP/IP.

It would be of value for you to learn about the protocols.

http://en.wikipedia.org/wiki/Internet_protocol_suite

The article breaks down the different layers of the suite.

But your 'breakdown' is more than just the browser, so you should also  
understand the protocols used by your tools of investigation.

Standard ping = ICMP  (also there is UDP pinger)
traceroute and tracert also can come in 'flavors' ICMP, UDP, and TCP

Some hop obstructions or 'holes' are related to protocol.


--  
Mike Easter

Re: Frustrated that I don't UNDERSTAND why my network times out
On Wed, 09 Oct 2013 11:28:22 -0700, Mike Easter wrote:

Quoted text here. Click to load it

Hi Mike,

So, if the web browser is what is failing, and if the  
debug commands using TCP also fail, what value is there
in everyone saying that ICMP is dropped?

What does ICMP have to do with the web browser failing?

That is, why is everyone suggesting I test ICMP when the  
failure of the web browser is the problem I'm trying to  
test?

I don't understand why ICMP has 'anything' directly to do
with the web browser failing (other than ICMP also fails).


Re: Frustrated that I don't UNDERSTAND why my network times out
On 9.10.13 9:52 , billy wrote:
Quoted text here. Click to load it


PLEASE do get a text on TCP, read and understand it.
It explains the usage.

ICMP is a helper protocol inside the TCP/IP protocol suite.
TCP uses ICMP to detect a suitable segment (transmission
block) size.

Your web browser needs UDP for name resolution, TCP for
page transfer and ICMP to help TCP work, and all run
using IP.

Can you ping the target from your computer?

It works perfectly from here:

--- clip clip ---

tauno-voipios-macbook-pro-2:~ tauno$ ping centos.org
PING centos.org (72.232.194.162): 56 data bytes
64 bytes from 72.232.194.162: icmp_seq=0 ttl=52 time=161.925 ms
64 bytes from 72.232.194.162: icmp_seq=1 ttl=52 time=164.736 ms
c64 bytes from 72.232.194.162: icmp_seq=2 ttl=52 time=161.303 ms
64 bytes from 72.232.194.162: icmp_seq=3 ttl=52 time=165.196 ms
64 bytes from 72.232.194.162: icmp_seq=4 ttl=52 time=165.483 ms
^C

--- clip clip ---

I can also access the web site with a browser.

--  

Tauno Voipio

--  

Tauno Voipio


Re: Frustrated that I don't UNDERSTAND why my network times out
On Wed, 09 Oct 2013 22:47:22 +0300, Tauno Voipio wrote:

Quoted text here. Click to load it

I know it works for you as it works for everyone but me.
1. ping fails every time on www.centos.org
2. traceroute shows the penultimate hop is where it dies
It makes no sense.  

And, it works for my neighbor, who is on the same WISP.
He says my times are longer than his, which is the only delta.

The only thing my neighbor could think of was maybe my  
transit times are so long that packets are discarded
at some point.

I'll bet your transit times are much better than mine, right?

Does *that* transit-time theory make any sense?


Re: Frustrated that I don't UNDERSTAND why my network times out
Quoted text here. Click to load it

But he has stated that they all work on other web sites. Only on
centos.org do they fail, apparently.  

Quoted text here. Click to load it

He says he cannot, although he keeps showing the output of traceroute
rather than ping.  

Quoted text here. Click to load it

Yes, he also says that his neighbors have no problem either. So it is
something peculiar to the interaction between his machine and that
particular site. -- eg firewall on centos which rejects him. Firewall on
his side which rejects the far response. (probably he should make sure
that all firewalls are disabled and try to contact them, just to rule
out the firewall on his machines.)



Re: Frustrated that I don't UNDERSTAND why my network times out
unruh wrote:

Quoted text here. Click to load it

IIRC, when I had asked my neighbors, they had fewer hops to  
the same destination - so - I'm beginning to wonder if it's  
the sheer number of hops (or the time involved); but that's
just a guess.

I'd be glad to post traceroute or ping results to ANY server
you suggest, so we can compare with you.

I'll also ask a neighbor to send me his traceroutes and
post them when/if I get them.

Re: Frustrated that I don't UNDERSTAND why my network times out
unruh wrote:

Quoted text here. Click to load it

The ping is useless. Either that or I'm using it wrong.

Here's the cut and paste for the pings on Knoppix:
# ping -M icmp centos.org
ping: wrong value for -M: do, dont, want are valid ones.

# ping --help
ping: invalid option -- '-'
Usage: ping [-LRUbdfnqrvVaAD] [-c count] [-i interval] [-w deadline]
            [-p pattern] [-s packetsize] [-t ttl] [-I interface]
            [-M pmtudisc-hint] [-m mark] [-S sndbuf]
            [-T tstamp-options] [-Q tos] [hop1 ...] destination

# # ping www.centos.org
PING www.centos.org (72.232.194.162) 56(84) bytes of data.

^C
--- www.centos.org ping statistics ---
286 packets transmitted, 0 received, 100% packet loss, time 285212ms

(strangely it took minutes to time out on Knoppix so I killed it).




Re: Frustrated that I don't UNDERSTAND why my network times out
Quoted text here. Click to load it

BEcause you were also worried about the problem of traceroute failing
and/or ping failing.  

You could try
telnet centos.com 80
(or whatever the address is)
to see if there is any response  
(QUIT should get  you out)

eg
info:0.0[unruh]>telnet centos.com 80
Trying 87.106.187.200...
Connected to centos.com (87.106.187.200).
Escape character is '^]'.
QUIT
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>501 Method Not Implemented</title>
</head><body>
<h1>Method Not Implemented</h1>
<p>QUIT to /index.html not supported.<br />
</p>
</body></html>
Connection closed by foreign host.



Quoted text here. Click to load it

Re: Frustrated that I don't UNDERSTAND why my network times out
On Wed, 09 Oct 2013 19:51:42 +0000, unruh wrote:

Quoted text here. Click to load it

This is very true.

Maybe it's my fault for bringing up traceroute & ping
in the first place; but they're the only freeware tools
for debugging the network that I know of.

Clearly they're doing the *same* thing as the web is,  
which is they're all dying at the penultimate hop
before www.centos.org

I did say, in my very first post, that I was pretty sure
the problem had nothing to do with the *way* I was running
the tools (since the problem is independent of those tools).

Quoted text here. Click to load it

$ telnet www.centos.org 80
Trying 72.232.194.162... <==== it hangs here for a minute or two
telnet: connect to address 72.232.194.162: Connection timed out

The problem is that NOTHING will connect to www.centos.org
that comes from me. If I use TOR, everything works fine.

THAT is what I'm trying to debug.
But, I don't know how.


Re: Frustrated that I don't UNDERSTAND why my network times out
billy wrote:
Quoted text here. Click to load it

Have you been in touch here as suggested by Pooh?

Technical Contact:
       Internap Network Services Corporation
       Domain Administrator
       One Ravinia Drive Suite 1300
       Atlanta, GA 30346
       US
       Phone: +1.8778434662
       Email: noc@internap.com

If not, WHY not?

Re: Frustrated that I don't UNDERSTAND why my network times out
~BD~ wrote:

Quoted text here. Click to load it

I have sent all the email addresses on that list an email.
I will write back if/when they respond.  
I don't have high hopes that they will - but - they might.

I gave them the unedited traceroute results just like I gave you.

Re: Frustrated that I don't UNDERSTAND why my network times out
Quoted text here. Click to load it

Well, a firewall might be rejecting you or the reply.
Switch off all firewalls-- on your machine, your router, etc to see if
that might be the problem.

(switch them on again after the test)


Quoted text here. Click to load it

Re: Frustrated that I don't UNDERSTAND why my network times out
billy wrote:
Quoted text here. Click to load it

Personally when I'm troubleshooting something I take into account the  
fact that some hops handle ICMP, UDP, and TCP differently.  One of my  
preferred tools for testing access to a server is Steve Gibson's IDServe  
which runs under windows or WINE in linux and doesn't require  
installation in either.

For this target I would be using TCP port 80 on/at the webserver at  
centos.  The tool shows my system resolving the IP and connecting to the  
webserver there.

Initiating server query ...
Looking up IP address for domain: www.centos.org
The IP address for the domain is: 72.232.194.162
Connecting to the server on standard HTTP port: 80
[Connected]  Requesting the server's default page.
The server returned the following response headers:
[redact html from the centos Apache server]
Query complete.

Quoted text here. Click to load it

 From the other info you've posted here, your system when not reaching  
centos on port 80 TCP IDServe would show you resolving the name but not  
connecting, which IDServe interprets as the server being stealthed or  
not online, so its use wouldn't add anything to this particular  
problem's solution; but I mention it as an indication that 'typically' I  
don't use an ICMP tool to test some TCP-related server issues.


--  
Mike Easter

Site Timeline