Normally, I’d go for some kind of Linux solution, but after doing a bit of research it looked like one of the BSD’s with pf/ALTQ would be the way to go. After trying out a couple of popular ready-made “appliances” (m0n0wall, pfsense) I decided that the best, most flexible way, would be to get my hands dirty and roll my own.

This post is just a quick summary of the active configuration at the time of writing, mainly for my future reference. So, without further ado, here’s what I came up with…

[Updated]

I decided to go for OpenBSD, grabbed the ISO, burnt it to CD and made a standard installation (minus the games.tgz package). The server I’m using has 2 on-board NIC’s and a 4-port PCI NIC card. During installation I configured one of the on-board ports as my servers main Internet/management interface.

Next I configured a couple of the ports on the PCI card to act as bridged interfaces:

/etc/hostname.vr0 :
up

/etc/hostname.vr3 :
up

/etc/bridgename.bridge0 :
add vr0
add vr3
up

Now it was time to configure pf, the ALTQ traffic queues and some rules to assign packets to queues. The server is going to act as a transparent bridge between the internet-facing 100Mb switchport and the LAN. The “LAN” is all public IP addressable, so no NAT rules/translation etc is needed here. The LAN should also only be allowed 10Mb of Internet bandwidth. Here’s the current configuration after a bit of playing around and tuning:

/etc/pf.conf :

# Define the interface aliases
ext_if=”vr3″ # External WAN-facing interface
int_if=”vr0″ # Internal LAN-facing interface

# Leecher IP addresses – These are punished with a shared low-bandwidth queue
# Separate addresses/CIDR ranges with commas.
#
table <leechers> persist { }

# Enable ALTQ on the internal interface, assign the root queue and
# ultimate bandwidth limit
altq on $int_if hfsc bandwidth 10Mb queue { int_root }

# Define the interface queues
queue int_root bandwidth 100% priority 0 hfsc {default_out, penalty_out, acks_out, normal_out, high_out}
queue default_out bandwidth 1% priority 1 qlimit 500 hfsc (default realtime 1% red ecn)
queue penalty_out bandwidth 1% priority 2 qlimit 500 hfsc (upperlimit 1Mb red ecn)
queue acks_out bandwidth 30% priority 7 qlimit 500 hfsc (realtime 10% red ecn)
queue normal_out bandwidth 20% priority 3 qlimit 500 hfsc (realtime 1Kb red ecn)
queue high_out bandwidth 45% priority 4 qlimit 500 hfsc (realtime 1Kb red ecn)

# Enable ALTQ on the external interface, assign the root queue and
# ultimate bandwidth limit
altq on $ext_if hfsc bandwidth 10Mb queue { ext_root }

# Define the interface queues
queue ext_root bandwidth 100% priority 0 hfsc {default_in, penalty_in, acks_in,normal_in, high_in}
queue default_in bandwidth 1% priority 1 qlimit 500 hfsc (default realtime 1% red ecn)
queue penalty_in bandwidth 1% priority 2 qlimit 500 hfsc (upperlimit 1Mb red ecn)
queue acks_in bandwidth 30% priority 7 qlimit 500 hfsc (realtime 10% red ecn)
queue normal_in bandwidth 20% priority 3 qlimit 500 hfsc (realtime 1Kb red ecn)
queue high_in bandwidth 45% priority 4 qlimit 500 hfsc (realtime 1Kb red ecn)

###
# Packet matching rules – A match will assign a packet to a given queue
###

# Put leeching mofo’s into the penalty queue
pass in quick on $int_if from { <leechers> } to any queue penalty_out
pass in quick on $ext_if from any to { <leechers> } queue penalty_in

# Give ICMP packets high(er) priority
pass in quick on $int_if proto icmp all queue high_out
pass in quick on $ext_if proto icmp all queue high_in

# Give VPN, RDP, SSH & DNS packets high(er) priority
pass in quick on $int_if proto ah all queue (high_out, acks_out)
pass in quick on $ext_if proto ah all queue (high_in, acks_in)
pass in quick on $int_if proto esp all queue (high_out, acks_out)
pass in quick on $ext_if proto esp all queue (high_in, acks_in)
pass in quick on $int_if proto gre all queue (high_out, acks_out)
pass in quick on $ext_if proto gre all queue (high_in, acks_in)

pass in quick on $int_if proto tcp from any to any port 3389 queue (high_out, acks_out)
pass in quick on $ext_if proto tcp from any to any port 3389 queue (high_in, acks_in)
pass in quick on $int_if proto tcp from any to any port 22 queue (high_out, acks_out)
pass in quick on $ext_if proto tcp from any to any port 22 queue (high_in, acks_in)
pass in quick on $int_if proto tcp from any to any port 53 queue (high_out, acks_out)
pass in quick on $ext_if proto tcp from any to any port 53 queue (high_in, acks_in)
pass in quick on $int_if proto udp from any to any port 53 queue (high_out, acks_out)
pass in quick on $ext_if proto udp from any to any port 53 queue (high_in, acks_in)

# Assign standard web & mail traffic to the “normal” queue
pass in quick on $int_if proto tcp from any to any port 80 queue (normal_out, acks_out)
pass in quick on $ext_if proto tcp from any to any port 80 queue (normal_in, acks_in)
pass in quick on $int_if proto tcp from any to any port 443 queue (normal_out, acks_out)
pass in quick on $ext_if proto tcp from any to any port 443 queue (normal_in, acks_in)
pass in quick on $int_if proto tcp from any to any port 25 queue (normal_out, acks_out)
pass in quick on $ext_if proto tcp from any to any port 25 queue (normal_in, acks_in)
pass in quick on $int_if proto tcp from any to any port 110 queue (normal_out, acks_out)
pass in quick on $ext_if proto tcp from any to any port 110 queue (normal_in, acks_in)
pass in quick on $int_if proto tcp from any to any port 143 queue (normal_out, acks_out)
pass in quick on $ext_if proto tcp from any to any port 143 queue (normal_in, acks_in)

# Stick everything else into the default queue
pass in quick on $int_if all queue (default_out, acks_out)
pass in quick on $ext_if all queue (default_in, acks_in)

A quick check with “pftop” shows that traffic is being queued & shaped correctly…

QUEUE               BW SCH  PR  PKTS BYTES DROP_P DROP_B QLEN BORR SUSP P/S  B/S
root_vr0           10M hfsc  0     0     0      0      0    0             0    0
 int_root          10M hfsc  0     0     0      0      0    0             0    0
  default_out     100K hfsc     565K  358M      0      0    0           284 316K
  penalty_out     100K hfsc  2     0     0      0      0    0             0    0
  acks_out       3000K hfsc  7  454K   25M      0      0    0           113 6417
  normal_out     2000K hfsc  3 1119K 1405M      0      0    0           277 364K
  high_out       4500K hfsc  4  219K   65M      0      0    0            78  42K
root_vr3           10M hfsc  0     0     0      0      0    0             0    0
 ext_root          10M hfsc  0     0     0      0      0    0             0    0
  default_in      100K hfsc    1839K  878M     15  17364    4           566 301K
  penalty_in      100K hfsc  2     0     0      0      0    0             0    0
  acks_in        3000K hfsc  7 56271 3425K      0      0    0             5  396
  normal_in      2000K hfsc  3 76830   81M     10  15140    0             7 4698
  high_in        4500K hfsc  4  163K   39M      0      0    0            71  22K

Hopefully someone somewhere will find this useful. If anyone has any ideas to fine-tune my config, please feel free to let me know

UPDATE 1: The above installation was made onto a standard IDE harddisk, but I intend to replace the disk with a 256MB IDE flash card. If anyone has any advice for getting a small OpenBSD installation onto one of these, please drop me a line. Thanks

UPDATE 2: It turns out installing onto flash was easier than I expected it to be. I fired up the installation CD, partitioned the flash drive with one large 256MB partition and then only installed the bsd, base41 & etc41 packages. Once that lot installed I created /proto/var & /proto/dev, re-created the device nodes in /proto/dev and copied /var into /proto/var. All that was left was to edit /etc/fstab to mount /var and /dev from the prototype dirs as ramdisks. Job’s a goodun!…

/etc/fstab :

/dev/wd0a / ffs rw,noatime 1 1
swap /var mfs rw,-P/proto/var,-s=65535,noexec,nosuid,nodev,noauto 0 0
swap /dev mfs rw,-P/proto/dev,-s=1200,-i=128,noexec,nosuid 0 0

This last piece of advice was taken from this website: http://www.kaschwig.net/projects/openbsd/wrap/