I have an instance where a remote ADSL site (PPPOA) wants to talk to a server in a Co-Lo but the 3 x way handshake doesn't fully complete. PC at remote site is Linux as is the Server in the Co-Lo.
When the traffic traverses the ADSL it comes into the clients main office and jumps across a couple of Ethernet circuits to the Co-Lo.
I've taken off the remote acl and firewall from the site router. The Hosts gets all the way to Established but the server stays in the SYN_REC state indicating that:
The originating PC sent the ACK to the server. The server sent the SYN Ack back to to client. The client moves to Established but the last ACK never hists the server to open up the communication.
I don't think it's a fragmentation issue. We can ping with DF bit set to 1500 bytes (1472 + headers) between source and destination (haven't tested the other way though). There are no other filtering devices in the way. The path (traceroute) from site to site isn't exactly the same. Could out of sequence traffic somehow be the issue, or maybe fragmentation could be the issue on the way back as I haven't tested the return path.
We have tried a HTTP request, FTP and SSH and none work, so it isn't an issue restricted to HTTP.
Any helpful debugs - I have seen 'debug tcp transactions' to test the 3 x way handshake. I know this will help but the Co-Lo router crunches a lot of data and I am concerned that the debug may have a negative effect. Anyone got any other good debugs or traps they can suggest so that I can determine where the 3 x way communication is breaking down.