Debugging an IPSec tunnel on PIX515

I need to set up an IPSec tunnel between a PIX 515E and a NetScreen unit (which I don't have access to). This is what I need to do:

- All traffic from the inside network to two specific IP addresses should go through the IPSec tunnel

- All traffic going through the tunnel must be NATed behind the IP address "10.38.1.1"

So far, the tunnel actually seems to be working. Pinging one of the IP addresses initiates ISAKMP negotiations, and the tunnel changes status to "up". Also, according to the guy managing the NetScreen unit, my packets do indeed reach the other side, NATed and all, and reply packets are sent back to the NetScreen box, but I'm not seeing any replies at my end. How can I find out exactly where the packets are being dropped?

Here's the output from "show crypto ipsec sa" on the PIX, after I've tried to send a bunch of ping packets to 192.168.71.34. Something seems to be wrong, but what?

------------------------------------------------------------ local ident (addr/mask/prot/port): (10.38.1.1/255.255.255.255/0/0) remote ident (addr/mask/prot/port): (192.168.71.34/255.255.255.255/0/0) current_peer: 172.18.0.1:500 PERMIT, flags={origin_is_acl,} #pkts encaps: 117, #pkts encrypt: 117, #pkts digest 117 #pkts decaps: 109, #pkts decrypt: 125, #pkts verify 125 #pkts compressed: 0, #pkts decompressed: 0 #pkts not compressed: 0, #pkts compr. failed: 0, #pkts decompress failed: 0 #send errors 3, #recv errors 16

local crypto endpt.: 172.16.0.1, remote crypto endpt.: 172.18.0.1 path mtu 1500, ipsec overhead 56, media mtu 1500 current outbound spi: be2585c8

inbound esp sas: spi: 0xc4fa66b0(3304744624) transform: esp-des esp-sha-hmac , in use settings ={Tunnel, } slot: 0, conn id: 4, crypto map: external-ipsec sa timing: remaining key lifetime (k/sec): (4607997/1222) IV size: 8 bytes replay detection support: Y

inbound ah sas:

inbound pcp sas:

outbound esp sas: spi: 0xbe2585c8(3190130120) transform: esp-des esp-sha-hmac , in use settings ={Tunnel, } slot: 0, conn id: 3, crypto map: external-ipsec sa timing: remaining key lifetime (k/sec): (4607998/1195) IV size: 8 bytes replay detection support: Y

outbound ah sas:

outbound pcp sas:

------------------------------------------------------------

....and this is (part of) the PIX config:

------------------------------------------------------------ : Access list for the outside interface : access-list external permit icmp any any echo access-list external permit icmp any any echo-reply access-list external permit icmp any any unreachable access-list external permit icmp any any time-exceeded access-list external permit icmp any any parameter-problem access-list external permit udp any any eq isakmp access-list external permit esp any any access-list external permit ah any any access-list external deny ip any any : : 192.168.11.0/24 is a network used by PPTP VPN clients : access-list NoNAT permit ip any 192.168.11.0 255.255.255.0 access-list NoNAT permit ip 192.168.11.0 255.255.255.0 192.168.0.0

255.255.0.0 : : This access list is used to NAT packets going through : the IPSec tunnel : access-list Tunnel-NAT permit ip 192.168.10.0 255.255.255.0 host 192.168.48.211 access-list Tunnel-NAT permit ip 192.168.10.0 255.255.255.0 host 192.168.71.34 : : ...and this is the access list that triggers creation : of the IPSec tunnel : access-list Tunnel permit ip host 10.38.1.1 host 192.168.48.211 access-list Tunnel permit ip host 10.38.1.1 host 192.168.71.34 : ip address outside 172.16.0.1 255.255.255.0 ip address inside 192.168.10.1 255.255.255.0 : : The 2nd global statement is for packets going through : the tunnel : global (outside) 2 interface global (outside) 1 10.38.1.1 : nat (inside) 0 access-list NoNAT nat (inside) 1 access-list Tunnel-NAT 0 0 nat (inside) 2 192.168.0.0 255.255.0.0 0 0 access-group external in interface outside : route outside 0.0.0.0 0.0.0.0 172.16.0.2 : sysopt connection permit-ipsec sysopt connection permit-pptp sysopt ipsec pl-compatible : crypto ipsec transform-set default esp-des esp-sha-hmac : crypto map external-ipsec 10 ipsec-isakmp crypto map external-ipsec 10 match address Tunnel crypto map external-ipsec 10 set pfs group2 crypto map external-ipsec 10 set peer 172.18.0.1 crypto map external-ipsec 10 set transform-set default crypto map external-ipsec interface outside : isakmp enable outside isakmp key ******** address 172.18.0.1 netmask 255.255.255.255 : isakmp identity address isakmp policy 21 authentication pre-share isakmp policy 21 encryption des isakmp policy 21 hash sha isakmp policy 21 group 2 isakmp policy 21 lifetime 86400 :

------------------------------------------------------------

Any help would be much appreciated.

:BTW, I tried "debug packet inside proto icmp" on the PIX, but it seemed :to ignore the "proto icmp" part, and dumped everything to the console, :including my telnet connection to the PIX.

Yeah, as far as I can tell, the 'proto' qualifier is ignored for 'debug packet'. Two alternative debug methods you might wish to explore are debug icmp trace and the 'capture' command.

Capture is applied -after- decapsulation, by the way, so try capturing against the outside interface. Also, I find that the capture ACL doesn't seem to be automatically read "in reverse" for incoming packets, so as well as permitting X to Y, permit Y to X.

In article , KR wrote: :You are right; the counters increment exactly as one would expect, and :"send errors" and "recv errors" remain static. But where do the packets :get dropped, and by what?

I would tend to suspect that the received packets have the wrong destination IP. "debug icmp trace" would help with tracking that.

If, though, the destination was wildly wrong, then you would see complaints in the logs about the recieved packet unexpectedly being an IP packet, so I'm not confident in my suspicion.

Not really. This is what I got with "logging monitor 7" and "term mon" while pinging 192.168.71.34 from an inside host:

305011: Built dynamic ICMP translation from inside:192.168.10.3/49 to outside(Tunnel-NAT):10.38.1.1/155 305011: Built dynamic ICMP translation from inside:192.168.10.3/50 to outside(Tunnel-NAT):10.38.1.1/156 305011: Built dynamic ICMP translation from inside:192.168.10.3/51 to outside(Tunnel-NAT):10.38.1.1/157 305011: Built dynamic ICMP translation from inside:192.168.10.3/52 to outside(Tunnel-NAT):10.38.1.1/158 305011: Built dynamic ICMP translation from inside:192.168.10.3/53 to outside(Tunnel-NAT):10.38.1.1/159

(I tore down the SA and tried again, and no errors were reported from ISAKMP or IPSEC either. I got both "sa created" and "ISAKMP Phase 2 exchange completed".)

You are right; the counters increment exactly as one would expect, and "send errors" and "recv errors" remain static. But where do the packets get dropped, and by what?

BTW, I tried "debug packet inside proto icmp" on the PIX, but it seemed to ignore the "proto icmp" part, and dumped everything to the console, including my telnet connection to the PIX.

Fixed. But as you said, it shouldn't have any effect on the IPSec tunnel.

Here's output from "debug icmp trace":

5: ICMP echo-request from inside:192.168.10.3 to 192.168.71.34 ID=5 112 seq=485 length=80 6: ICMP echo-request: translating inside:192.168.10.3/5112 to outside:10.38.1.1/172 7: ICMP echo-request from inside:192.168.10.3 to 192.168.71.34 ID=5113 seq=485 length=80 8: ICMP echo-request: translating inside:192.168.10.3/5113 to outside:10.38.1.1/173

....and so on. There's no corresponding "ICMP echo-reply: untranslating outside". It is as if the reply packets never arrive.

This is what I did:

access-list capt permit icmp any any capture test access-list capt interface outside circular-buffer

I then tried to ping 192.168.71.34 from a host on the inside, and afterwards, this was the output from "sh capt test":

0 packet captured 0 packet shown

If I ping a host other than those on the other end of the IPSec tunnel, the packets get captured. I guess the capture must happen before decapsulation after all? I tried changing the ACL to "permit 50 any any", and sure enough, every ping packet results in two esp packets being captured, one in each direction.

A capture against the inside interface showed the outgoing echo-requests, but no replies.

Tired of all of this nonsense, I removed the "nat (inside) 1 access-list Tunnel-NAT" and "global (outside) 1 10.38.1.1" statements. I then added a host route to 10.38.1.1 pointing to a cisco router on the inside, and did this on the router:

interface lo0 ip address 10.38.1.1 255.255.255.255 no shutdown

Then I tried an extended ping to 192.168.71.34 from that router, using loopback0 as the source interface.

It worked.

It it unreasonable to conclude that The NAT mechanism on the PIX is broken when it comes to IPSec tunnels?

And we've just spent money upgrading the PIX to 6.3(4). The customer is going to be thrilled when I tell him this.