I have an old Pentium 166 with 16M of RAM. Is that enough machine to run a simple Linux distro (maybe Damn Small Linux) and Smoothwall?
Also, does anybody make a "real" modem any more? I've looked for hours trying to find a new 56K dialup modem that's not a "Winmodem" without success. Sometimes the application data seems to be worded to make you think it's not a Winmodem, and then you find out that it is, when you read something in the fine print like "peripheral software is required to operate this product."
My guess is that it depends on the Linux install and kernel you run it on top of, and how it's been configured. A gentoo build with only the required drivers, kernel swapping allowed, and -Os as optimization might increase the available memory quite a bit. On the other hand, I wouldn't want to wait around for a gentoo build on a P166 - it'd probably take weeks of compile time.
I wouldn't recommend it, no; however, I find it is very easy to scrounge up some pretty decent (well, for their age anyway) Pentium boxes with a lot more RAM (my own box had a whopping 128 MB!). Just ask around.
On a side note, 16 MB just *might* be enough if you're willing to do with a toned-down kernel and Busybox for all applications. Would not give any fancy interfaces though, probably just a single terminal featuring a Bourne shell-lookalike. Not good if you're looking for a web based solution, but could be used as an ultra-spartan environment - include only init, mount and iptables, and run some sort of script from a read-only floppy - or go all the way and build a floppy containing just grub, a minimal kernel, and an initrd containing iptables with some `script' (preferably a static binary) to set up the firewall (and possibly a minimal k/syslogd). Hmm, sounds nice... I'll have to look that one up on freshmeat, or build one myself.
If you're going the less extreme route, just getting an upgraded Pentium may be sufficient.
Also make sure anything that is used as a server/router is pretty damn stable - Knoppix includes a memory test (memtest86, you could also just look it up on freshmeat.net), also verify the disk, etc etc.
The P1 166 is fine. I have a smoothwall running on one just fine but you'll need more ram.
Quote from their installation guide:
"Minimum 32 Mbytes RAM, ideally 64 Mbytes or more. With less than 64 Mbytes RAM it may not be possible to run all services especially the Web Proxy Server and the Intrusion Detection System. Additional RAM may be needed for optimal performance if the Proxy Cache and Intrusion Detection features are utilised."
64MB is ideally minimum.
The only modems I have ever used or ever will use are U.S.R./3Com modems. They are rock steady/stable & can be obtained as true hardware modems (non-winmodems). There may be others but not as good quality. If you can't find a good internal one, an external 57.6 will work great.
Isn't that what distcc is for? You might not wish to use it on a firewall that is actually in use - though, if properly firewalled, it might not compare badly with a web or SSH server - but it will allow you to actually build the firewall.
However, do go the easy route and pry some memory from a dead box, or just get a new fifth-hand one.
As for your modem, pretty much any serial port modem is a hardware modem. Winmodems are typically USB or internal modems. You do have serial ports don't you? It'll be a shame when they disappear forever... Try finding them on a laptop these days.