I have two 6509 switches with SUP720's in them. They are both supporting multiple subnets that are using HSRP. I have 5 internal subnets that are each in different HSRP groups. I have one external facing subnet that is also in an HSRP group.
I use a policy route (ip default next hop) to direct all the internal subnets to a proxy server on the external subnet. All works great when all the sbunets are running on the primary router. If one of the subnets fails over to the secondary though, they can reach all the internal networks, but the route map no long seems to work.
Please paste 'show standby', the relevant part of the routing configs (both routers), and I'm assuming these are vtp peers with trunking between them. Do both have access to the VLAN that has the next hop router? Can you source pings to that hop?
Thanks for the reply. I am using the 6509 like a straight router. There are no VLAN's. IP's are directly assigned to the interfaces. There is a Gig interface connecting the 2 boxes (no trunking). I can source pings from all interfaces to the proxy on the standby router and in a failed over scenario the proxy can ping internal workstations. It is as if the policy route is being ignored. Below is all the relevent information.
Primary Router
GigabitEthernet6/2 - Group 2 *** Link to Checkpoint cluster at
10.10.1.11 *** Local state is Active, priority 105, may preempt Hellotime 3 sec, holdtime 10 sec Next hello sent in 1.999 Virtual IP address is 10.10.1.10 configured Active router is local Standby router is 10.10.1.101 expires in 9.164 Virtual mac address is 0000.0c07.ac02 Authentication text "abc" 24 state changes, last state change 19:18:28 IP redundancy name is "hsrp-Gi6/2-2" (default)
GigabitEthernet6/5 - Group 5 *** Internal Subnet *** Local state is Active, priority 105, may preempt Hellotime 3 sec, holdtime 10 sec Next hello sent in 1.845 Virtual IP address is 10.5.1.1 configured Active router is local Standby router is 10.5.1.4 expires in 7.972 Virtual mac address is 0000.0c07.ac05 Authentication text "abc" 26 state changes, last state change 18:50:53 IP redundancy name is "hsrp-Gi6/5-5" (default)
interface GigabitEthernet6/2 description 10.10.1.0/24 Subnet *** Link to Checkpoint cluster at
10.10.1.11 *** ip address 10.10.1.9 255.255.255.0 ip route-cache flow ip policy route-map firewall-proxy standby 2 ip 10.10.1.10 standby 2 priority 105 standby 2 preempt standby 2 authentication abc
interface GigabitEthernet6/5 description 10.5.1.1/24 Subnet *** Internal Subnet *** ip address 10.5.1.3 255.255.255.0 ip route-cache flow ip policy route-map firewall-proxy standby 5 ip 10.5.1.1 standby 5 priority 105 standby 5 preempt standby 5 authentication abc
ip route 0.0.0.0 0.0.0.0 204.48.238.57 200 ip route 10.0.0.0 255.0.0.0 10.11.1.5 210
Standby Router
GigabitEthernet6/37 - Group 2 *** Link to Checkpoint cluster at
10.10.1.11 *** Local state is Standby, priority 100, may preempt Hellotime 3 sec, holdtime 10 sec Next hello sent in 2.763 Virtual IP address is 10.10.1.10 configured Active router is 10.10.1.9, priority 105 expires in 7.908 Standby router is local Authentication text "abc" 19 state changes, last state change 18:43:38 IP redundancy name is "hsrp-Gi6/37-2" (default)
GigabitEthernet6/46 - Group 5 *** Internal Subnet *** Local state is Standby, priority 100, may preempt Hellotime 3 sec, holdtime 10 sec Next hello sent in 1.496 Virtual IP address is 10.5.1.1 configured Active router is 10.5.1.3, priority 105 expires in 9.040 Standby router is local Authentication text "abc" 10 state changes, last state change 18:35:29 IP redundancy name is "hsrp-Gi6/46-5" (default)
interface GigabitEthernet6/37 description 10.10.1.0/24 Subnet *** Link to Checkpoint cluster at
10.10.1.11 *** ip address 10.10.1.101 255.255.255.0 ip route-cache flow ip policy route-map firewall-proxy standby 2 ip 10.10.1.10 standby 2 preempt standby 2 authentication abc
interface GigabitEthernet6/46 *** Internal Subnet *** description 10.5.1.1/24 Subnet (Backup) ip address 10.5.1.4 255.255.255.0 ip route-cache flow ip policy route-map firewall-proxy standby 5 ip 10.5.1.1 standby 5 preempt standby 5 authentication abc
ip route 0.0.0.0 0.0.0.0 204.48.238.57 200 ip route 10.0.0.0 255.0.0.0 10.11.1.5 210
Core-01#ping Protocol [ip]: Target IP address: 10.10.1.11 Repeat count [5]: Datagram size [100]: Timeout in seconds [2]: Extended commands [n]: y Source address or interface: GigabitEthernet6/46 Type of service [0]: Set DF bit in IP header? [no]: Validate reply data? [no]: Data pattern [0xABCD]: Loose, Strict, Record, Timestamp, Verbose[none]: Sweep range of sizes [n]: Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 10.10.1.11, timeout is 2 seconds: Packet sent with a source address of 10.5.1.4 !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/4 ms
One thing I forgot. This is the case if one internal subnet fails over to the secondary box and everything else is still on the primary. My assumption is that if the entire primary box were to fail, all would work fine on the secondary (I have not had a chance to test this).
I don't see how HSRP is even working for the 5.1 subnet without trunking between the two MSFCs? You can see in your config that both routers think they are the local owner of the 5.1 subnet, therefore HSRP is not working for that access vlan because they are not connected to each other in that VLAN to exchange HSRP packets. This is your issue.
They are not running as VLAN's so there is nothing to trunk. HSRP is working flawlessly, it is the policy route that is messing me up. The ports are running as physical L3 ports ("no switchport" command which is default in the 6509 on the SUP720). The box is running more like a
48 port router than a switch. There is also no STP enabled since there is no need for it.
I should also mention that the reason I need the policy route is that these boxes also peer with BGP to my 2 ISP's (one on each box). Right now I am taking only default route from the ISP's but I guess if I can't get this policy route to work I can take full routes from the ISPs and setup a default route to the Checkpoint cluster. I was trying to avoid doing that if possible though.
So I'm assuming you have a switch in the 10.X network that goes to the checkpoint cluster, and another that goes to the 5.X network. These two 6509s have router interfaces in each, the connections on these interfaces you pasted are just layer 3 based on your comments. When you fail over, are you simply raising the standby cost and failing over that way, or actually shutting down an interface on one or the other? The nodes on 5.X that you are testing are all off one switch downstream that is connected to both routers? I know you aren't trying to focus on L2, but I'm trying to understand traffic in and out of the subnet as you are saying that when Core 2 owns the 5.X network, and Core 1 still owns the 10.X network returning from the firewall, traffic fails. Additionally, you have passive interfaces on these routers, so is the checkpoint stuff static routed back? A diagram would help tremendously......I'm not sold that it is your policy map yet.
Lastly, and in regards to your internet configuration, how are you advertising out to the internet world? Firewalls definitely do not like single direction traffic, and if you are going out one firewall to one ISP, but back in another, your traffic will be put into the bit bucket. This would explain why going out one core and therefore ISP1 would work, but if it fails over and goes out the other path, it could very well return through the primary path and cause issues. Just some shots in the dark.
I really appreciate all your assitance with this. I think you are right on now with our config. From the 6509 I have runs out to simple stackable Linksys switches with no vlans. So the 10.5.1 subnet goes to one switch (with both routers on it) and the 10.10.1.x subnet goes to another. The firewalls are in a cluster with a VIP both inside and outside. They are using static routes to point back to the HSRP VIPs. There should be no asymetric traffic through the firewalls since there is only one path in and out.
When I failover for testing I shutdown the other interface to simulate a true failure. So the 10.5.1 subnet will be failed over but the
10.10.1 subnet will not be failed over. When I do failover and to a traceroute out from 10.5.1 I go one hop (10.5.1.1) then stars the rest of the way.
I would be happy to send you a diagram in a PDF. Where should I send it?
I'm fairly certain that is your issue. When 5.1 fails over but 10.x does not, the traffic still returns to Core1 because he owns your layer 3 HSRP and you are not running dynamic routing protocols for it to go to Core 2. Right then, he does not know how to route to 5.X because his interface is down, and you aren't exchanging routes with your peer over those networks. Perhaps you are exchanging routes some other way, but I think this is where you need to focus.
I do have a route to it via OSPF. The 2 boxes exchange routes that way. The gig link that connects the 2 boxes is not passive (I did not include that in the config snippet I sent). They should both be totally aware of available routes between them.
On those two links in the config you originally sent, they were configured as passive. What does the route table on Core1 look like when its ethernet in that network is down?
I got it all working. I did some more reading last night as I could not sleep bacause I could not stop thinking about this issue and I overlooked a minor but critical detail.
hop are similar commands but have a different order of operations. Configuring the set ip next-hop command causes the system to use policy routing first and then use the routing table. Configuring the set ip default next-hop command causes the system to use the routing table first and then policy route the specified next hop.
So I modified my route map to use the routing table for local - local and changed set ip default next-hop to set ip next-hop like this:
route-map test permit 10 - This should make traffic from local to local use the routing table (I hope) match ip address 125
route-map test permit 20 - If not local, go to the firewall proxy match ip address 50 set ip next-hop 10.10.1.11
access-list 125 remark Source-Destination Pairs that will not use PBR access-list 125 permit ip 10.0.0.0 0.255.255.255 10.0.0.0
0.255.255.255 access-list 125 permit ip 10.0.0.0 0.255.255.255 172.16.0.0
0.0.255.255 access-list 125 permit ip 10.0.0.0 0.255.255.255 192.168.0.0
0.0.255.255
access-list 50 remark Everyone that will use PBR access-list 50 permit 10.0.0.0 0.255.255.255
Thank you for all your assistance. I really do appreciate it.
Cabling-Design.com Forums website is not affiliated with any of the manufacturers or service providers discussed here.
All logos and trade names are the property of their respective owners.