documentation:examples:fair_traffic_shaping_per_ip_with_ipfw-dummynet
no way to compare when less than two revisions
Differences
This shows you the differences between two versions of the page.
Previous revisionNext revision | |||
— | documentation:examples:fair_traffic_shaping_per_ip_with_ipfw-dummynet [2020/05/14 23:11] – [User PC configuration] olivier | ||
---|---|---|---|
Line 1: | Line 1: | ||
+ | ====== Fair traffic shaping per ip with ipfw and dummynet ====== | ||
+ | |||
+ | This lab shows an example of fair sharing asymmetric Internet access between multiple users (one user = one IP address). | ||
+ | This feature is called [[http:// | ||
+ | |||
+ | ===== Network diagram ==== | ||
+ | |||
+ | And here is this lab detailed diagram: | ||
+ | |||
+ | {{: | ||
+ | |||
+ | ===== Virtual Lab setp ===== | ||
+ | |||
+ | This chapter will describe how to start each routers and configuring the 4 hosts. | ||
+ | |||
+ | More information on these BSDRP lab scripts available on [[documentation: | ||
+ | |||
+ | Start the Virtual lab (example using bhyve): | ||
+ | < | ||
+ | BSDRP-lab-bhyve.sh -i BSDRP-1.60-full-amd64-serial.img.xz -n 5 -l 1 | ||
+ | BSD Router Project (http:// | ||
+ | Setting-up a virtual lab with 5 VM(s): | ||
+ | - Working directory: /tmp/BSDRP | ||
+ | - Each VM have 1 core(s) and 256M RAM | ||
+ | - Switch mode: bridge + tap | ||
+ | - 1 LAN(s) between all VM | ||
+ | - Full mesh Ethernet links between each VM | ||
+ | VM 1 have the following NIC: | ||
+ | - vtnet0 connected to VM 2. | ||
+ | - vtnet1 connected to VM 3. | ||
+ | - vtnet2 connected to VM 4. | ||
+ | - vtnet3 connected to VM 5. | ||
+ | - vtnet4 connected to LAN number 1 | ||
+ | VM 2 have the following NIC: | ||
+ | - vtnet0 connected to VM 1. | ||
+ | - vtnet1 connected to VM 3. | ||
+ | - vtnet2 connected to VM 4. | ||
+ | - vtnet3 connected to VM 5. | ||
+ | - vtnet4 connected to LAN number 1 | ||
+ | VM 3 have the following NIC: | ||
+ | - vtnet0 connected to VM 1. | ||
+ | - vtnet1 connected to VM 2. | ||
+ | - vtnet2 connected to VM 4. | ||
+ | - vtnet3 connected to VM 5. | ||
+ | - vtnet4 connected to LAN number 1 | ||
+ | VM 4 have the following NIC: | ||
+ | - vtnet0 connected to VM 1. | ||
+ | - vtnet1 connected to VM 2. | ||
+ | - vtnet2 connected to VM 3. | ||
+ | - vtnet3 connected to VM 5. | ||
+ | - vtnet4 connected to LAN number 1 | ||
+ | VM 5 have the following NIC: | ||
+ | - vtnet0 connected to VM 1. | ||
+ | - vtnet1 connected to VM 2. | ||
+ | - vtnet2 connected to VM 3. | ||
+ | - vtnet3 connected to VM 4. | ||
+ | - vtnet4 connected to LAN number 1 | ||
+ | For connecting to VM' | ||
+ | - VM 1 : cu -l /dev/nmdm1B | ||
+ | - VM 2 : cu -l /dev/nmdm2B | ||
+ | - VM 3 : cu -l /dev/nmdm3B | ||
+ | - VM 4 : cu -l /dev/nmdm4B | ||
+ | - VM 5 : cu -l /dev/nmdm5B | ||
+ | </ | ||
+ | |||
+ | ==== User PC configuration ==== | ||
+ | |||
+ | Each user PC will be configured as simple DHCP clients. | ||
+ | |||
+ | === Router 1 === | ||
+ | |||
+ | < | ||
+ | sysrc hostname=R1 \ | ||
+ | gateway_enable=no \ | ||
+ | ipv6_gateway_enable=no \ | ||
+ | ifconfig_vtnet4=" | ||
+ | ifconfig_vtnet4_ipv6=" | ||
+ | rtsold_enable=" | ||
+ | service hostname restart | ||
+ | service netif restart | ||
+ | service routing restart | ||
+ | service rtsold start | ||
+ | config save | ||
+ | </ | ||
+ | |||
+ | === Router 2 === | ||
+ | |||
+ | < | ||
+ | sysrc hostname=R2 \ | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | service hostname restart | ||
+ | service netif restart | ||
+ | service routing restart | ||
+ | service rtsold start | ||
+ | config save | ||
+ | </ | ||
+ | |||
+ | === Router 3 === | ||
+ | |||
+ | < | ||
+ | sysrc hostname=R3 \ | ||
+ | gateway_enable=no \ | ||
+ | ipv6_gateway_enable=no \ | ||
+ | ifconfig_vtnet4=" | ||
+ | ifconfig_vtnet4_ipv6=" | ||
+ | rtsold_enable=" | ||
+ | service hostname restart | ||
+ | service netif restart | ||
+ | service routing restart | ||
+ | service rtsold start | ||
+ | config save | ||
+ | </ | ||
+ | |||
+ | ==== Internet server configuration ==== | ||
+ | |||
+ | It's a simple static host | ||
+ | |||
+ | < | ||
+ | sysrc hostname=R5 | ||
+ | sysrc gateway_enable=no | ||
+ | sysrc ipv6_gateway_enable=no | ||
+ | sysrc ifconfig_vtnet3=" | ||
+ | sysrc ifconfig_vtnet3_ipv6=" | ||
+ | sysrc defaultrouter=" | ||
+ | sysrc ipv6_defaultrouter=" | ||
+ | service netif restart | ||
+ | service routing restart | ||
+ | </ | ||
+ | |||
+ | ==== Traffic shaper configuration ==== | ||
+ | |||
+ | It's a DHCPv4 server and traffic shaper. | ||
+ | |||
+ | < | ||
+ | sysrc hostname=R4 | ||
+ | sysrc ifconfig_vtnet4=" | ||
+ | sysrc ifconfig_vtnet4_ipv6=" | ||
+ | sysrc ifconfig_vtnet3=" | ||
+ | sysrc ifconfig_vtnet3_ipv6=" | ||
+ | sysrc rtadvd_enable=yes | ||
+ | sysrc rtadvd_interfaces=vtnet4 | ||
+ | sysrc dhcpd_enable=YES | ||
+ | sysrc dhcpd_flags=" | ||
+ | sysrc dhcpd_conf="/ | ||
+ | sysrc dhcpd_ifaces=" | ||
+ | sysrc firewall_enable=YES | ||
+ | sysrc firewall_script="/ | ||
+ | |||
+ | cat > / | ||
+ | option domain-name " | ||
+ | default-lease-time 600; | ||
+ | max-lease-time 7200; | ||
+ | ddns-update-style none; | ||
+ | subnet 10.0.5.0 netmask 255.255.255.0 { | ||
+ | } | ||
+ | subnet 10.0.0.0 netmask 255.255.255.0 { | ||
+ | range 10.0.0.1 10.0.0.3; | ||
+ | option routers 10.0.0.4; | ||
+ | } | ||
+ | EOF | ||
+ | |||
+ | cat > / | ||
+ | #!/bin/sh | ||
+ | fwcmd="/ | ||
+ | if ! kldstat -q -m dummynet; then | ||
+ | kldload dummynet | ||
+ | fi | ||
+ | # Flush out the list before we begin. | ||
+ | ${fwcmd} -f flush | ||
+ | oif=vtnet3 | ||
+ | bwu=50Mbit/ | ||
+ | bwd=100Mbit/ | ||
+ | # Declare hard-limit of our links (2 because bidirectional) | ||
+ | ${fwcmd} pipe 1 config bw $bwu | ||
+ | ${fwcmd} pipe 2 config bw $bwd | ||
+ | # per-ip fair queueing | ||
+ | ${fwcmd} queue 1 config pipe 1 mask src-ip 0xffffffff | ||
+ | ${fwcmd} queue 2 config pipe 2 mask dst-ip 0xffffffff | ||
+ | # Assing outgoing traffic to upload queue and incoming to download queue | ||
+ | ${fwcmd} add queue 1 ip from any to any xmit $oif out | ||
+ | ${fwcmd} add queue 2 ip from any to any recv $oif in | ||
+ | # We don't want to block traffic, only shape some | ||
+ | ${fwcmd} add 3000 allow ip from any to any | ||
+ | EOF | ||
+ | |||
+ | service netif restart | ||
+ | service routing restart | ||
+ | service isc-dhcpd start | ||
+ | service rtadvd start | ||
+ | service ipfw start | ||
+ | config save | ||
+ | </ | ||
+ | |||
+ | ===== Shapping Tests ===== | ||
+ | |||
+ | Start 3 iperf3 servers on the " | ||
+ | < | ||
+ | [root@R5]~# iperf3 -s -p 9091 | ||
+ | ----------------------------------------------------------- | ||
+ | Server listening on 9091 | ||
+ | ----------------------------------------------------------- | ||
+ | </ | ||
+ | < | ||
+ | [root@R5]~# iperf3 -s -p 9092 | ||
+ | ----------------------------------------------------------- | ||
+ | Server listening on 9092 | ||
+ | ----------------------------------------------------------- | ||
+ | </ | ||
+ | < | ||
+ | [root@R5]~# iperf3 -s -p 9093 | ||
+ | ----------------------------------------------------------- | ||
+ | Server listening on 9093 | ||
+ | ----------------------------------------------------------- | ||
+ | </ | ||
+ | ==== With only one user ==== | ||
+ | |||
+ | If there is only one user it should have the full bandwith. | ||
+ | From only one client (R1, R2 or R3) start by generating traffic toward the iperf3 server for checking maximum upload bandwith is correctly shapped to 50Mb/s: | ||
+ | |||
+ | < | ||
+ | [root@R3]~# iperf3 -c 10.0.5.5 -t 60 -i 10 -f m -p 9093 | ||
+ | Connecting to host 10.0.5.5, port 5201 | ||
+ | [ 4] local 10.0.0.3 port 10049 connected to 10.0.5.5 port 9093 | ||
+ | [ ID] Interval | ||
+ | [ 4] | ||
+ | [ 4] 10.00-20.00 | ||
+ | [ 4] 20.00-30.00 | ||
+ | [ 4] 30.00-40.00 | ||
+ | [ 4] 40.00-50.00 | ||
+ | [ 4] 50.00-60.00 | ||
+ | - - - - - - - - - - - - - - - - - - - - - - - - - | ||
+ | [ ID] Interval | ||
+ | [ 4] | ||
+ | [ 4] | ||
+ | |||
+ | iperf Done. | ||
+ | </ | ||
+ | |||
+ | => Upload is correctly shaped to 50Mb/s. | ||
+ | |||
+ | Now a " | ||
+ | |||
+ | < | ||
+ | [root@r3]~# iperf3 -c 10.0.5.5 -t 60 -i 10 -f m -R -p 9093 | ||
+ | Connecting to host 10.0.5.5, port 5201 | ||
+ | Reverse mode, remote host 10.0.5.5 is sending | ||
+ | [ 4] local 10.0.0.3 port 10280 connected to 10.0.5.5 port 9093 | ||
+ | [ ID] Interval | ||
+ | [ 4] | ||
+ | [ 4] 10.00-20.00 | ||
+ | [ 4] 20.00-30.00 | ||
+ | [ 4] 30.00-40.00 | ||
+ | [ 4] 40.00-50.00 | ||
+ | [ 4] 50.00-60.00 | ||
+ | - - - - - - - - - - - - - - - - - - - - - - - - - | ||
+ | [ ID] Interval | ||
+ | [ 4] | ||
+ | [ 4] | ||
+ | </ | ||
+ | |||
+ | ⇒ Download is correctly shaped to 100Mb/ | ||
+ | |||
+ | ==== With two users ==== | ||
+ | |||
+ | Now start iperf clients on the same time on 2 clients and check that upload is equally share (25Mb/s each): | ||
+ | |||
+ | < | ||
+ | [root@R3]~# iperf3 -c 10.0.5.5 -t 60 -i 10 -f m -p 9093 | ||
+ | Connecting to host 10.0.5.5, port 9093 | ||
+ | [ 4] local 10.0.0.3 port 36202 connected to 10.0.5.5 port 9093 | ||
+ | [ ID] Interval | ||
+ | [ 4] | ||
+ | [ 4] 10.00-20.00 | ||
+ | [ 4] 20.00-30.00 | ||
+ | [ 4] 30.00-40.00 | ||
+ | [ 4] 40.00-50.00 | ||
+ | [ 4] 50.00-60.00 | ||
+ | - - - - - - - - - - - - - - - - - - - - - - - - - | ||
+ | [ ID] Interval | ||
+ | [ 4] | ||
+ | [ 4] | ||
+ | |||
+ | iperf Done. | ||
+ | </ | ||
+ | |||
+ | < | ||
+ | [root@R2]~# iperf3 -c 10.0.5.5 -t 60 -i 10 -f m -p 9092 | ||
+ | Connecting to host 10.0.5.5, port 9092 | ||
+ | [ 4] local 10.0.0.1 port 25306 connected to 10.0.5.5 port 9092 | ||
+ | [ ID] Interval | ||
+ | [ 4] | ||
+ | [ 4] 10.00-20.00 | ||
+ | [ 4] 20.00-30.00 | ||
+ | [ 4] 30.00-40.00 | ||
+ | [ 4] 40.00-50.00 | ||
+ | [ 4] 50.00-60.00 | ||
+ | - - - - - - - - - - - - - - - - - - - - - - - - - | ||
+ | [ ID] Interval | ||
+ | [ 4] | ||
+ | [ 4] | ||
+ | |||
+ | iperf Done. | ||
+ | </ | ||
+ | |||
+ | => Upload bandwidth is fair-shaped-shared between these two users. | ||
+ | |||
+ | Now the download speed should be 50Mb/s each too: | ||
+ | |||
+ | < | ||
+ | [root@R3]~# iperf3 -c 10.0.5.5 -t 60 -i 10 -f m -p 9093 -R | ||
+ | Connecting to host 10.0.5.5, port 9093 | ||
+ | Reverse mode, remote host 10.0.5.5 is sending | ||
+ | [ 4] local 10.0.0.3 port 14377 connected to 10.0.5.5 port 9093 | ||
+ | [ ID] Interval | ||
+ | [ 4] | ||
+ | [ 4] 10.00-20.00 | ||
+ | [ 4] 20.00-30.00 | ||
+ | [ 4] 30.00-40.00 | ||
+ | [ 4] 40.00-50.00 | ||
+ | [ 4] 50.00-60.00 | ||
+ | - - - - - - - - - - - - - - - - - - - - - - - - - | ||
+ | [ ID] Interval | ||
+ | [ 4] | ||
+ | [ 4] | ||
+ | |||
+ | iperf Done. | ||
+ | </ | ||
+ | |||
+ | < | ||
+ | [root@R2]~# iperf3 -c 10.0.5.5 -t 60 -i 10 -f m -p 9092 -R | ||
+ | Connecting to host 10.0.5.5, port 9092 | ||
+ | Reverse mode, remote host 10.0.5.5 is sending | ||
+ | [ 4] local 10.0.0.1 port 56926 connected to 10.0.5.5 port 9092 | ||
+ | [ ID] Interval | ||
+ | [ 4] | ||
+ | [ 4] 10.00-20.00 | ||
+ | [ 4] 20.00-30.00 | ||
+ | [ 4] 30.00-40.00 | ||
+ | [ 4] 40.00-50.00 | ||
+ | [ 4] 50.00-60.00 | ||
+ | - - - - - - - - - - - - - - - - - - - - - - - - - | ||
+ | [ ID] Interval | ||
+ | [ 4] | ||
+ | [ 4] | ||
+ | |||
+ | iperf Done. | ||
+ | </ | ||
+ | |||
+ | => Same correct behavior here. | ||
+ | |||
+ | During this bench, on the router, queue 1 (upload) and queue 2 (download) status: | ||
+ | |||
+ | < | ||
+ | [root@R4]~# ipfw queue 1 show | ||
+ | q00001 | ||
+ | mask: 0x00 0xffffffff/ | ||
+ | BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/ | ||
+ | 2 ip 10.0.0.1/ | ||
+ | 6 ip 10.0.0.3/ | ||
+ | [root@R4]~# ipfw queue 2 show | ||
+ | q00002 | ||
+ | mask: 0x00 0x00000000/ | ||
+ | BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/ | ||
+ | 1 ip | ||
+ | 3 ip | ||
+ | </ | ||
+ | |||
+ | |||
+ | ==== With three users ==== | ||
+ | |||
+ | Same correct behavior with three users, here is the upload bandwith of one of the three users: | ||
+ | < | ||
+ | [root@R1]~# iperf3 -c 10.0.5.5 -t 60 -i 10 -f m -p 9091 | ||
+ | Connecting to host 10.0.5.5, port 9091 | ||
+ | [ 4] local 10.0.0.2 port 35814 connected to 10.0.5.5 port 9091 | ||
+ | [ ID] Interval | ||
+ | [ 4] | ||
+ | [ 4] 10.00-20.00 | ||
+ | [ 4] 20.00-30.00 | ||
+ | [ 4] 30.00-40.00 | ||
+ | [ 4] 40.00-50.00 | ||
+ | [ 4] 50.00-60.00 | ||
+ | - - - - - - - - - - - - - - - - - - - - - - - - - | ||
+ | [ ID] Interval | ||
+ | [ 4] | ||
+ | [ 4] | ||
+ | |||
+ | iperf Done. | ||
+ | |||
+ | </ | ||
+ | |||
+ | => Only 50M/3 = 16.66 Mbs for each user regarding upload. | ||
+ | |||
+ | Queues status on the router during this three-users bench: | ||
+ | < | ||
+ | [root@R4]~# ipfw queue 1 show | ||
+ | q00001 | ||
+ | mask: 0x00 0xffffffff/ | ||
+ | BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/ | ||
+ | 2 ip 10.0.0.1/ | ||
+ | 4 ip 10.0.0.2/ | ||
+ | 6 ip 10.0.0.3/ | ||
+ | [root@R4]~# ipfw queue 2 show | ||
+ | q00002 | ||
+ | mask: 0x00 0x00000000/ | ||
+ | BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/ | ||
+ | 1 ip | ||
+ | 2 ip | ||
+ | 3 ip | ||
+ | </ | ||
documentation/examples/fair_traffic_shaping_per_ip_with_ipfw-dummynet.txt · Last modified: 2020/05/14 23:32 by olivier