Firewall change management

In article , James Coldwind wrote: :I'm finding that I have to make changes to my PIX firewall on a regular :basis. I just wanted to get a feel of how everyone is managing changes on :their firewall. Is there software that will help with change control :management?

If you have several PIX that are under active control (e.g., the security policy is to deny by default and you regularily have to go around opening new services once the need for them is known), then unless you are more concientious then most, two PIX is about the realistic limit for manual updates: by the time you get to three PIX, chances are that you'll update them pairwise as things come up between two specific sites, and forget to (or not have time to) update the remaining PIXen to keep them consistant.

We recently hit 6 PIX, and integrating the 6th PIX has turned out to be a major several-months-long effort, because of the differences that had been allowed to grow up between the various configurations.

Change Management Hint #1:

Strongly avoid trying to -remotely- do major ruleset rewrites on non-trivial PIX configurations. You will spend 95% of your time or more on transitional configurations, taking one step at a time closer to the unified configuration, but moving glacially slowly to avoid locking yourself out, and to avoid having the remote offices losing contact during the time you are updating their counterpoint VPN devices to match the new configs.

Instead, either pre-configure a replacement firewall and lab-bench test it and send out that one, or else insist that your company send you on-site to do the configuration there. That might -sound- expensive, but the alternative is to have the configurations drag on and on and on. There are some reconfiguration changes that *require* that you clear the active connections: you *will* lose contact with the remote site... and you have to hope that it comes back again afterwards :(

#2:

If you can avoid it, do not treat the live configuration on the PIX as being the authorative configuration. Instead, tftp off an initial configuration, and edit that configuration, and tftp that configuration into place when you are ready. The text version on your tftp server becomes the master configuration, and any change not made to the master configuration is "expendable". Now, apply standard revision control system (e.g., rcs or cvs) to the master configuration.

#3:

When you produce a master configuration, you want to know that after you tftp in that configuration that that is what the configuration will -become-. But when you tftp a configuration off of the PIX, it will contain only the configuration as-is, and when you tftp in a configuration, it -adds- the new configuration in, rather than -replacing- the old configuration.

The way to solve this is to sprinkle your configuration strategically with 'no' and 'clear' commands -- e.g., 'clear names' just before the 'names' section, 'clear username' before 'clear aaa' before setting hew AAA, and so on.

#4:

When you start accumulating PIX, start thinking using some manner of policy editor to generate the configurations consistantly instead of managing them by hand. See #1. Cisco used to have a stand-alone policy editor; they integrated it into CiscoWorks, which is (I gather) not the least expensive of software.

When we hit our 6th PIX, I took the route of writing my own configuration generator (from templates)... not a true policy editor really. It turned out to take a long time to get all the software pieces in place -- considering my time, buying one would likely have been less expensive. But the old Cisco Policy editor was kinda clunky and awkward to work with... try before you buy!

#5:

It gets worse. Firewall configurations get uglier, not cleaner. Though the PIX 6.2 "object" technology helps a lot -- use object groups liberally. If you have non-trivial configurations, then you should think of writing the configurations as being not just "a little bit of editing": for non-trivial configurations with multiple PIX, you should think of the configurations as being more in the line of being "programs" in an unusual programming language -- programs have to be debugged; and programs have to be -structured- to be understandable; and if you aren't a fiend for consistancy in your naming schemes and in your access-lists, then you -will- get "bugs" that you will waste endless time debugging.

Hi,

I am amazed that Walter only manages 6 Pix. Seems to know rather a lot about them:-))

I have done a little bit with PIX and a bit more with Checkpoint fw1. The interesting thing is that we have about 30 or 40 fw1's out there and we do loads of remote changes and have never, I don't think, lost connectivity in about 3 years. We do little planning usually, just get in and hack away. It's the corporate culture and who am I to argue? Most of the boxes are not too far away so we can take more risks, however the fact is that we do not get locked out. We also usually do this hacking in production hours and have almost no, or even exactly no, outages.

It is interesting that someone with so much understanding is so concerned about this possibility.

It is clear that the checkpoint is easier to deal with but I didn't perhaps realise that such a knowledgeable and I assume experienced Pix administrator would effectively rule out remote changes.

For those not in the know the checkpoint has a GUI for pretty much all administrative purposes. When compared to the Pix It is a bit like the difference between writing Fortran to do some calculations and using Excel.

I think that fw1 costs more though.

Interesting stuff, hope I don't get too many flames:-(

Thanks for the comments. Has anyone used software such as Firemon? If so, is it worth the pricetag?

In article , wrote: :I am amazed that Walter only manages 6 Pix. Seems to :know rather a lot about them:-))

Like the old punchline goes: "Practice, Practice, Practice!"

:I have done a little bit with PIX and a bit more with Checkpoint :fw1. The interesting thing is that we have about 30 or 40 :fw1's out there and we do loads of remote changes and :have never, I don't think, lost connectivity in about 3 years. :We do little planning usually, just get in and hack away. :It's the corporate culture and who am I to argue? Most of :the boxes are not too far away so we can take more risks,

We have a rather unofficial remote office at my house (~10 minutes from work), but our official remote offices are 850 (1 timezone) and 2550 miles away (2 timezones). And unfortunately in both locations the local ISP-equivilents filter a number of protocols, so our only practical access is ssh or an access through the live VPN (e.g., log in to one of the systems there and connect from it to the inside of the firewall.) We do not have an appropriate inside system at one of the locations, so reconfiguration is tftp over the live VPN... and as each tftp'd line is processed as it is read, mistakes in the configuration can drop the VPN connection while we are in the middle of (say) creating the ACL that defines the VPN.

:It is interesting that someone with so much understanding :is so concerned about this possibility.

We are a default-deny shop ... that's a requirement from a government level *far* above ours. So we have to fail-closed even for internal traffic. The PIX has an option to turn off ACL processing for IPSec traffic, but we are not permitted to use it [and do not want to use it anyhow, as we aren't fond of the idea of a laptop brought into one office poisoning everything through all of our offices.]

default-deny unfortunately leads to a proliferation of rules. My current template-generated configuration is over 21000 lines, even for our PIX 501's. And there are thousands more lines I haven't had a chance to integrate yet.

About 3/4 of the lines are there as part of a strategy I had to come up with in order to minimize the risk of breaking the very VPN connection that I'm using to load in the new configuration. The PIX has a shortcoming in 6.x that when you add new entries to a crypto ACL that is used to define an active VPN, that the new entries do not necessarily become active immediately, and the old ACL entries now removed do not necessarily disappear immediately. If one has redefined the VPN ACLs, one must effectively clear the VPN in order to put the new ACLs into effect.. but clearing the VPN breaks the link I'm using to push in the config. I found a strategy to clear away the old ACLs with no window of configuration commands needed to activate the new ACLs... but the cost was a configuration file size explosion. [Keep in mind that the remote PIX must be treated as having an unknown configuration at the start, so we can't just do a bit of selective pruning here and there...]

:For those not in the know the checkpoint has a GUI :for pretty much all administrative purposes.

The PIX has GUI interfaces available too, and those go in through a dedicated port, potentially not needing the VPN once launched through https. But you don't keep thousands of rules aligned by clicking them in one-by-one into GUI interfaces.

This does, though, lead to an interesting possibility I should test: if I lost the VPN then I might be able to go in through the GUI (especially if I started up the GUI before I started the changes.)

:Interesting stuff, hope I don't get too many flames:-(

No, no, your comments are fair. Different firewalls do things differently, and different people use them differently or under different constraints.

One doesn't get to be a technical expert by refusing to see the shortcomings of a device or the advantages of a different one.