Here is my layout in it's basic form;
I have a large number of web servers in my DMZ. I obviously can't just re-route this setup to default gateway through the 100Mb/s side. All replies to inbound traffic would then route incorrectly.
Because it was easy & I knew how to do it, I setup a transparent proxy in the DMZ and did a re-write rule on the firewall so all outbound port 80 traffic from the LAN gets send to this proxy. This proxy's default gateway is out through another firewall on the 100Mb/s side. There is a peer which it can talk to which defaults out the slower pipes. I can also just disable the re-write rule if there are problems.
I can't transparently do SSL or other traffic. I could just setup another firewall, configure my DMZ networks to route specific through the old firewall, and have my PC's default gateway through this new firewall, hooked up to the 100Mb/s connection. We have a very restrictive firewall, so I don't desire recreating that on another firewall. Nor do I desire two sets of logs.
Routers are not my specialty - we have another guy configure those. But it is mostly basic stuff. I'm learning about policy based routing.
Could I setup the clients I want to go out over the 100Mb/s connection to NAT from a specific address on the main firewall, connect the two Cisco routers connected to the Internet together (along with all the public routing that entails) and then configure policy based routing so that if the source IP is that NAT address, it defaults to route out to the router on the 100Mb/s Internet and then a higher metric through my BGP peers?
Does that make sense? Any pitfalls with this approach?