Share bandwidth evenly....

Must I say it? Can't I just write it? Is it 5Mbits/sec symmetrical or is the outgoing bandwidth somewhat less? If so, how much less?

Can I presume that most of the users are connected via wireless?

DD-WRT v24 SP1 is the current stable release. Pre-SP2 is out. I tried it and had problems. Grrrr....

Which bandwidth? LAN bandwidth, wireless bandwidth, or ethernet bandwidth? Each one can be independently controlled according to a variety of possible rules. The search for the ultimate fair bandwidth sharing algorthm continues forever.

You could do that. Search Google for "bandwidth manager". There are various commerical and open source products that will do that. The problem is that adding such a device between your hypothetical

5Mbit/sec unspecified internet connection will not be able to seperate the various users on the router. All you can see from there is one IP address and tangled mess of IP ports. The only way to use an external bandwidth manager is to have the users connect to one side of the bandwidth manager, while the other side goes to the router.

I limit the amount of outgoing bandwidth for all users (total outgoing bandwidth) to about 80% of the outgoing capacity. I've found that it's the outgoing bandwidth that saturates first, especially if users are running portable servers or variouis file sharing nightmares.

It's really difficult to distribute the bandwidth equally when some wireless users are connected at 1Mbits/sec, while others are connected at 54Mbits/sec. If I count bytes going by, the slow user will have allocated to it about 50 times the air time of the fast user. Not very useful. The trick is to force users to go as fast as possible or better yet, run the access point at a fixed speed. Faster connections use less air time. I've done this by fixing the speed to 12 or

24Mbits/sec. This also has a desireable side effect in that distant users (i.e. neighbors and hackers) can't reliably connect. With everyone connecting at the same speed, you don't have to do anything more than reserve outgoing bandwidth. The ethernet "fair share" algorithm will take care of distributing the bandwidth fairly. However, if you want to give priority to time sensitive applications (i.e. VoIP) and downgrade the priority of undesireable applications (file sharing), you'll need to configure QoS appropriately by application:

I can send you my settings, but my situation is probably different from yours.

Ok, end of lecture. So, what problem are you trying to solve? Let me guess.... you have one user doing file sharing (and denying it), which is hogging the entire system. Right?

There's a setting on the QoS page to block *ALL* P2P protocols.

Drivel: I want one:

(size=Large)

Jeff Liebermann skrev:

Ok Jeff,

Sorry about being too short... I'll give you much more details. I just didn't want to overwhelm you in my initial post... :-|

This scenario is actually in a dorms area of a boarding high school. We have several wrt54g* running dd-wrt v24 as access points in the different areas. These wrts are all connected to a pfsense router/firewall/gateway. Students connect their computers wirelessly (802.11G only) to the wrts.

The ISP provide the pfsense gateway with 15Mbit downlink and 5Mbit uplink. All WRTs are cabled together and connected to the pfsense box with 100Mb ethernet.

When looking at pfsense's bandwidth graphs before, during and after one student has downloaded a huge file, we see that during the download he gets almost all of the 'free' bandwidth (which is ok).

However, unfortunately all the other students experience significantly longer response times (ping / latency) and their downloads / browsing only seems to share a small bandwidth available to those opening sessions WHILE there are other huge downloads already underway.

I would like to impose a bandwidth limit of 1Mb or less for each user / IP so single users don't grab too much of our valuable internet bandwidth. If this need to be done at the wrt level I wonder if there are tools available now which enable us to config multiple wrts at a time either via some kind of centralized gui or by using scripts centrally.

For us it's the internet bandwidth which is precious and I concentrate on that. I have never heard students complaining that they have slow access to our intranet's downloading area (which is also available through the wrt network).

When using wireshark we see relatively little torrent and gnutella traffic, but in any case we have tried to stop p2p at the wrt level (QoS as you suggest).

We'll also take the steps to avoid the 'Router Slowdown' you refer to.

So it cooks down to ways to limit internet bandwidth (focusing on outgoing as you suggest). Any suggestions to this are welcome. I've also heard that there is a commercial dd-wrt version available, but haven't yet had time to look into it. Does that version have features which can help us with our desire to distribute bandwidth evenly?

I'll gladly get you the thinkgeek item mentioned at the end of your post if you can help us find good solutions and configurations to solve this problem :-D

Take care :-)

rgds Geir

Are the WRT54G boxes acting as access points (no router section) or as routers (with double NAT)? The QoS and traffic management sections of the WRT54G are in the router section. If this section is disabled or bypassed, as is common with multiple AP systems, all the bandwidth limiting must be done at the Pfsense (M0n0wall) router.

That's normal if there is only one active connection.

The problem may have several causes, each of which has to be addressed with traffic management.

1. Over the air (RF) traffic. The heavy user may be monopolizing all the air time, leaving little for the other users. This is a common problem if the wireless speeds are too slow. For example, a

15Mbits/sec full WAN bandwidth download, with a wireless rate of 30Mbits/sec, will monopolize all the available air time as the radio will be on the air almost full time. (The reason why it's 30Mbits/sec is that 802.11b/g has about 100% overhead).

1. Radical variations in wireless speeds. If a user connects at

1Mbits/sec, they will only get 0.5Mbits/sec of WAN bandwidth during a download. If a user connects at 54Mbits/sec, they will get all the 15Mbits/sec of download bandwidth. However, the slow user will get 50 times the air time of the fast user, thus preventing anyone else from using the system due to lack of available air time.

1. Upstream bandwidth saturation. If the download also includes an upload channel (i.e. P2P file sharing), then the 5Mbits/sec upstream bandwidth might be the problem. There will be available download bandwidth, but if the ACK's don't get through, response time and downloads will stall.
2. Buffer saturation in the access points and router. Bitorrent likes to open a huge number of simultaneous streams in order to maximize it's ability to grab as much bandwidth as possible. The AP and router only have a finite amount of buffering, which can be overrun. A year ago, the result was a hung AP or router. I haven't checked what happens in current versions of DD-WRT (may have been mitigated or fixed). Note that it's not just Bitorrent that causes this problem as there are several "download accelerators" that use similar technology.

There are some others, but those are the major culprits. If you have P2P under control, I think 1 and 2 need to be addressed before you add a bandwidth manager. I suggest you record the connection speed of the various users and build some data as to what is the highest speed that will work for everyone. Then, try setting the AP speed to this value. My guess is about 12Mbit/sec. That will also give each connected user an automatic 6Mbit/sec download speed limit and should reserve sufficient bandwidth for at least 3 simultaneous downloaders.

The problem will be if you have users with marginal connections, that can only work at 802.11b speeds. They won't be able to connect and will surely complain that the "range" or "coverage" of the system has been drastically reduced by this change. I suspect this may be in the minority and limited to PDA's and IP phones, which have generally minimal antenna systems.

Again, the basic idea is to minimize the air time used by each user. If you limited the wireless connection speed to 2Mbit/sec, thus giving each user 1Mbit/sec of download and upload bandwidth, you will not be able to use the full WAN 15Mbits/sec download bandwidth because each user is hogging far too much air time. Therefore, the trick is to minimize the air time for each user or at least set a lower limit on connection speeds.

If you're using the WRT54G AP's as AP's, the router section is disabled and no QoS can be performed in the AP's. It has to be done in the M0n0wall router. However, we may have a problem. What you want is "bandwidth management" and that appears to be on the wish list for PfSense:

My Googling found some clues as to how to do QoS (by service) but nothing for allocating bandwidth by user or IP. I'll dig some more when I have time.

I have some ideas on how to do it, but nothing definitive. It can't be done on the WAN port because the traffic manager has no way to (easily) separate the traffic by user, unless each user gets a routable IP address, which is improbable. I can be done on the LAN side, by moving the NAT over to the bandwidth manager, but that means almost totally replacing the PFsense router.

If that's true, then it's unlikely that my recommended fixed RF solution is going to do anything useful. It will need to be a traffic shaper of some sort on the LAN side, probably by replacing the PFsense router with something more feature intensive, such as a Cisco router that supports bandwidth management.

Where are you blocking P2P? At the router or in the AP's with dd-wrt?

There's some confusing notes on the commercial version at:

I don't know the current status or feature set.

No, don't bother. I just thought it was cool. The tee shirt has been around for a while, but this is the first time I've seen it advertised.

Have a look at installing the ntop module in pfSense. It's buggy as hell and uses lots of RAM, but other than that it gives lots of insight into network activity.

Have a look at:

If pfSense won't do what you want, you could always look at IPCop instead:

I built a WAN emulator from a linux box with 2 NICs in. Rate limiting was the main aim, with latency and packet loss being optional extras. ISTR 'tc qdisc ...' played a large part in that:

Jeff's suggestion of per-station rate limiting by reducing the wireless connection rate is an ingenious one :-)

Actually, I think Jeff is lamenting that this appears to be a solution but would not be a good idea because lowering the rate would increase air time consumed and thus further clog the airways.

BUT, seeing that there are multiple APs going, it seems like if 3-5 users share one AP, then limiting the bandwidth to 2 or 6 Mbps (thruput of 1 or 3) might do the job to spread the load, yet not clog up the airwaves nearly as much as if one 2 Mbps radio was serving all

15 users. In addition, you would tend to localize any problems so that only those sharing the saturated radio (if so) would be affected. Also, you could learn where it tends to happen.

Problem I see is that DD-WRT only limits the transmission rate, not the total link ? If I look at our DD-WRT router, which is set for

12 Mbps, it shows 12 Mbps for the rate in "Status". But on the pc end, it shows 54 Mbps as the rate. If that is not actually true, then connection rate might work for you.

QOS: If you set up your network so that your APs are also serving as local routers, connecting through their WAN side, then you could perhaps control bandwidth through the QOS settings in DD-WRT. I have not used it, but see that under NAT/QOS>QOS, when you activate it, there are settings for uplink and downlink max speeds. Again, to do this, you would have to run each Linksys as it's own router and there may be consequential problems with that setup...or maybe not? I would think that intra-networking would be complicated, but perhaps it's workable.

QOS is still unexplored territory to me, but maybe there's something there...

Steve

You don't have to run NAT to use it as a router. Are the QoS features available when using it as a router in the 'classic' sense? You can avoid the DHCP hassles by forwarding the requests to a centralized server. Which would have to be configured for scopes supporting each subnet of course. An option which would certainly be easier than babysitting DHCP servers on multiple low-end routers. You'd also gain a bit more logging and filtering options based on the subnet (easier, say, to set up on the fly filtering based on IP addresses).

-Bill Kearney

Bill Kearney skrev:

If I uncheck NAT in the wrt boxes , do I need to enter WAN IPs then, or how's eveything working together in that mode (which I suspect is 'transparent fw mode'. Please elaborate or hint me towards some informative urls.

/geir

Jeff Liebermann skrev:

Yes the wrts are plain access points with no router configured (wan IF disabled). We have one central dhcp server and all wrts are wired together.

What would the config be if we want to use their router section but transparently? (We want to still use our dhcp server, if possible).

I'm going to do some tests against our intranet from clients experienced saturation / slow access during the nights to confirm if the bottleneck is at the wireless level or a pure internet bandwidth problem.

Results will be posted here in a couple of days...

Finally, does anyone here have comments on centralized management of several wrt boxes? Does it exist scripting solutions or other approaches available...?

best regards

geir

That's what I guessed. That means that the bandwidth management has to be done in the central PFsense router and not the dd-wrt boxes. Start thinking about a new router with bandwidth management features.

Ugh. We're at the limit of my competence here. I think I could probably conjure a template for real NAT (network address translation), instead of PAT (port address translation), where each wireless user gets an IP address from a pool on the PFsense server which goes transparently through the DD-WRT router. However, I don't know exactly how to do this with IPtables or with DD-WRT. I'll do some digging and see what Google can find (later).

Good plan. Try to get the connection speeds at the same time. Note the the initial connection speed is usually 54Mbits/sec. However, once traffic is moving, the connection speed usually slows down. It's the slower "loaded" speed that you want.

Oh-oh. I tried to find what arguments to the "wl" command will yield the current connected client information. "wl status" seems to be the likely argument, but it only seems to supply the AP information, not the clients. The "wireless status" page on the HTML menus has speeds in both directions listed, but all 3 wireless devices that I just tried to use shows N/A for the speed. Argh.

Well, at least speed reporting can be done with SNMP:

I think I have a better way. I'll tinker and post anything that works (later).

I use Nagios, MRTG, RRDTool, and PRTG (free demo version) for monitoring. They're all quite different in terms of installation and user interface, but all use SNMP. Do you have SNMP access to the AP's?

You might consider that what you're apparently doing is building is a "wireless switch". These are brain dead access points attached to an overly intelligence central switch. It treats each radio as if it were an ethernet device. Assorted articles:

The basic idea is central management of the system from the central switch. Everything in the switch gets automagically propagated to the brain dead AP's. Similarly, all performance data is sent to the central switch. The big advantage is central management. The big problem is the price. These things are rather expensive unless you have a large system, where the central switch cost is distributed among many AP's or locations.

If each AP was routing its own subnet rather than bridging, the OP might get more insight into who is using all the bandwidth.

I wouldn't rely on the rate reported by your PC. I'm pretty sure DD-WRT limits the rate at the 802.11 layer.

I've managed to get >25Mbps [ethernet, not wifi] out of a Buffalo WHR-HP-54G running DD-WRT with NAT turned off, ie just routing. Don't know what model of APs the OP is using but if it's capable of running DD-WRT then it's probably capable of delivering at least his internet bandwidth.

It's as complicated as 'several' static routes on the pfSense gateway, which would only need to be done once. Networks upstream of it need not know about each individual wireless LAN.

If you turn off NAT, you will need to have one subnet per AP. You should be able to get DD-WRT to forward DHCP requests to your DHCP server. You will then need to tell your pfSense gateway how to get to each network, and to make sure pfSense knows to NAT traffic coming from each wireless subnet before sending it out to the internet. It's really not as scary as it sounds.

For example:

Name LAN WAN NAT? pfSense 192.168.1.1/24 x.x.x.x Yes AP1 10.0.1.1/24 192.168.1.101/24 No AP2 10.0.2.1/24 192.168.1.102/24 No APn 10.0.n.1/24 192.168.1.10n/24 No

with the WAN of each AP connected to the same switch as the pfSense's LAN.

You would then need to tell pfSense how to reach each wireless LAN with a 'static route':

So, to get to AP1's LAN [10.0.1.0/24], set a static route pointing to

192.168.1.101 as the gateway, and so on for each AP. That's how I'd do it anyway. You don't need to take a big-bang approach to this; you can move one AP at a time and see how it goes. Once you've got the static routing sorted, you could play with RIPv2 if you're feeling brave!

If you absolutely want to stick with a bridged LAN/WLAN, you can traffic shape by MAC with ebtables on linux generally, don't know about DD-WRT specifically: