Sounds as if this is the one single security concious client you have. It would seem all the others allow outgoing connections to any port, which is BAD.
The client you are having problems with probably has all outgoing ports blocked expecpt for the ones that are most commonly used (ie: 80, 8080,20, 21, maybe 23, 119, etc) - which is how ANY internet connection to a windoze box should be, IMO.
Norton Firewall should be configurable to specifically allow port 81 outgoing. However I am afraid I don't know the details on how, since I do not use that product.