BSD Router Project

User Tools

Site Tools

Trace:
documentation:examples:aggregating_multiple_isp_links_with_mlvpn

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
documentation:examples:aggregating_multiple_isp_links_with_mlvpn [2020/02/21 21:24] – [Links bandwidth] olivierdocumentation:examples:aggregating_multiple_isp_links_with_mlvpn [2020/02/21 21:42] (current) – [Aggregated bandwidth] olivier
Line 13: Line 13:
 {{:documentation:examples:bsdrp-lab-mlvpn-details.png|}} {{:documentation:examples:bsdrp-lab-mlvpn-details.png|}}
  
-===== Virtual Lab setp =====+===== Virtual Lab setup =====
  
-This chapter will describe how to start each routers and configuring the 3 centrals routers.+This chapter will describe how to start each routers and configuring the 4 centrals routers.
  
 More information on these BSDRP lab scripts available on [[documentation:examples:How to build a BSDRP router lab]]. More information on these BSDRP lab scripts available on [[documentation:examples:How to build a BSDRP router lab]].
Line 22: Line 22:
  
 <code> <code>
-# ./tools/BSDRP-lab-bhyve.sh -n 5+# ./tools/BSDRP-lab-bhyve.sh -n 6
 BSD Router Project (http://bsdrp.net) - bhyve full-meshed lab script BSD Router Project (http://bsdrp.net) - bhyve full-meshed lab script
-Setting-up a virtual lab with 5 VM(s): +Setting-up a virtual lab with 6 VM(s): 
-- Working directory: /tmp/BSDRP +- Working directory: /root/BSDRP-VMs 
-- Each VM have 1 core(s) and 256M RAM+- Each VM has a total of 1 (1 cores and 1 threads) and 512M RAM 
 +- Emulated NIC: virtio-net
 - Switch mode: bridge + tap - Switch mode: bridge + tap
 - 0 LAN(s) between all VM - 0 LAN(s) between all VM
 - Full mesh Ethernet links between each VM - Full mesh Ethernet links between each VM
-VM 1 have the following NIC: +VM 1 has the following NIC: 
-- vtnet0 connected to VM 2. +- vtnet0 connected to VM 2 
-- vtnet1 connected to VM 3. +- vtnet1 connected to VM 3 
-- vtnet2 connected to VM 4. +- vtnet2 connected to VM 4 
-- vtnet3 connected to VM 5. +- vtnet3 connected to VM 5 
-VM 2 have the following NIC: +- vtnet4 connected to VM 6 
-- vtnet0 connected to VM 1. +VM 2 has the following NIC: 
-- vtnet1 connected to VM 3. +- vtnet0 connected to VM 1 
-- vtnet2 connected to VM 4. +- vtnet1 connected to VM 3 
-- vtnet3 connected to VM 5. +- vtnet2 connected to VM 4 
-VM 3 have the following NIC: +- vtnet3 connected to VM 5 
-- vtnet0 connected to VM 1. +- vtnet4 connected to VM 6 
-- vtnet1 connected to VM 2. +VM 3 has the following NIC: 
-- vtnet2 connected to VM 4. +- vtnet0 connected to VM 1 
-- vtnet3 connected to VM 5. +- vtnet1 connected to VM 2 
-VM 4 have the following NIC: +- vtnet2 connected to VM 4 
-- vtnet0 connected to VM 1. +- vtnet3 connected to VM 5 
-- vtnet1 connected to VM 2. +- vtnet4 connected to VM 6 
-- vtnet2 connected to VM 3. +VM 4 has the following NIC: 
-- vtnet3 connected to VM 5. +- vtnet0 connected to VM 1 
-VM 5 have the following NIC: +- vtnet1 connected to VM 2 
-- vtnet0 connected to VM 1. +- vtnet2 connected to VM 3 
-- vtnet1 connected to VM 2. +- vtnet3 connected to VM 5 
-- vtnet2 connected to VM 3. +- vtnet4 connected to VM 6 
-- vtnet3 connected to VM 4. +VM 5 has the following NIC: 
-For connecting to VM'serial console, you can use: +- vtnet0 connected to VM 1 
-- VM 1 : cu -l /dev/nmdm1B +- vtnet1 connected to VM 2 
-- VM 2 : cu -l /dev/nmdm2B +- vtnet2 connected to VM 3 
-- VM 3 : cu -l /dev/nmdm3B +- vtnet3 connected to VM 4 
-- VM 4 : cu -l /dev/nmdm4B +- vtnet4 connected to VM 6 
-- VM 5 : cu -l /dev/nmdm5B+VM 6 has the following NIC: 
 +- vtnet0 connected to VM 1 
 +- vtnet1 connected to VM 2 
 +- vtnet2 connected to VM 3 
 +- vtnet3 connected to VM 4 
 +To connect VM'serial console, you can use: 
 +- VM 1 : cu -l /dev/nmdm-BSDRP.1B 
 +- VM 2 : cu -l /dev/nmdm-BSDRP.2B 
 +- VM 3 : cu -l /dev/nmdm-BSDRP.3B 
 +- VM 4 : cu -l /dev/nmdm-BSDRP.4B 
 +- VM 5 : cu -l /dev/nmdm-BSDRP.5B 
 +- VM 6 : cu -l /dev/nmdm-BSDRP.6B
 </code> </code>
  
Line 67: Line 79:
 === Router 2 === === Router 2 ===
  
-Router 2 is configured for rate-limiting traffic at 10 Mb/s on interface to/from R1.+Router 2 is configured for rate-limiting traffic at 10 Mb/s on interface to/from VM1.
  
 <code> <code>
-sysrc hostname=R2 +sysrc hostname=VM2 \ 
-sysrc ifconfig_vtnet0="10.0.12.2/24" +        ifconfig_vtnet0="inet 10.0.12.2/24" \ 
-sysrc ifconfig_vtnet2="10.0.24.2/24" +        ifconfig_vtnet3="inet 10.0.25.2/24" \ 
-sysrc static_routes="R5" +        defaultrouter="10.0.25.5" \ 
-sysrc route_R5="-net 10.0.45.0/24 10.0.24.4" +        firewall_enable=YES \ 
-sysrc firewall_enable=YES +        firewall_script="/etc/ipfw.rules" 
-sysrc firewall_script="/etc/ipfw.rules" +cat > /etc/ipfw.rules <<EOF
- +
-cat > /etc/ipfw.rules <<'EOF'+
 #!/bin/sh #!/bin/sh
 fwcmd="/sbin/ipfw" fwcmd="/sbin/ipfw"
 kldstat -q -m dummynet || kldload dummynet kldstat -q -m dummynet || kldload dummynet
 # Flush out the list before we begin. # Flush out the list before we begin.
-${fwcmd} -f flush +\${fwcmd} -f flush 
-#Create pipes (one for each direction) +\${fwcmd} pipe 10 config bw 10Mbit/s 
-${fwcmd} pipe 10 config bw 10Mbit/s +\${fwcmd} pipe 20 config bw 10Mbit/s
-${fwcmd} pipe 20 config bw 10Mbit/s+
 #Traffic getting out vtnet0 is limited to 10Mbit/s #Traffic getting out vtnet0 is limited to 10Mbit/s
-${fwcmd} add 1000 pipe 10 all from any to any out via vtnet0+\${fwcmd} add 1000 pipe 10 all from any to any out via vtnet0
 #Traffic getting int vtnet0 is limited to 10Mbit/s #Traffic getting int vtnet0 is limited to 10Mbit/s
-${fwcmd} add 2000 pipe 20 all from any to any in via vtnet0+\${fwcmd} add 2000 pipe 20 all from any to any in via vtnet0
 #We don't want to block traffic, only shape some #We don't want to block traffic, only shape some
-${fwcmd} add 3000 allow ip from any to any +\${fwcmd} add 3000 allow ip from any to any 
-'EOF'+EOF
  
 service netif restart service netif restart
 service routing restart service routing restart
 service ipfw start service ipfw start
 +hostname VM2
 config save config save
 </code> </code>
Line 103: Line 113:
 === Router 3 === === Router 3 ===
  
-Router 3 is configured for rate-limiting traffic at 10 Mb/s on interface to/from R1.+Router 3 is configured for rate-limiting traffic at 10 Mb/s on interface to/from VM1.
  
 <code> <code>
-sysrc hostname=R3 +sysrc hostname=VM3 \ 
-sysrc ifconfig_vtnet0="10.0.13.3/24" +        ifconfig_vtnet0="inet 10.0.13.3/24" \ 
-sysrc ifconfig_vtnet2="10.0.34.3/24" +        ifconfig_vtnet3="inet 10.0.35.3/24" \ 
-sysrc static_routes="R5" +        defaultrouter="10.0.35.5" \ 
-sysrc route_R5="-net 10.0.45.0/24 10.0.34.4" +        firewall_enable=YES \ 
-sysrc firewall_enable=YES +        firewall_script="/etc/ipfw.rules"
-sysrc firewall_script="/etc/ipfw.rules"+
  
-cat > /etc/ipfw.rules <<'EOF'+cat > /etc/ipfw.rules <<EOF
 #!/bin/sh #!/bin/sh
 fwcmd="/sbin/ipfw" fwcmd="/sbin/ipfw"
 kldstat -q -m dummynet || kldload dummynet kldstat -q -m dummynet || kldload dummynet
 # Flush out the list before we begin. # Flush out the list before we begin.
-${fwcmd} -f flush +\${fwcmd} -f flush 
-#Create pipes (one for each direction) +\${fwcmd} pipe 10 config bw 10Mbit/s 
-${fwcmd} pipe 10 config bw 10Mbit/s +\${fwcmd} pipe 20 config bw 10Mbit/s
-${fwcmd} pipe 20 config bw 10Mbit/s+
 #Traffic getting out vtnet0 is limited to 10Mbit/s #Traffic getting out vtnet0 is limited to 10Mbit/s
-${fwcmd} add 1000 pipe 10 all from any to any out via vtnet0+\${fwcmd} add 1000 pipe 10 all from any to any out via vtnet0
 #Traffic getting int vtnet0 is limited to 10Mbit/s #Traffic getting int vtnet0 is limited to 10Mbit/s
-${fwcmd} add 2000 pipe 20 all from any to any in via vtnet0+\${fwcmd} add 2000 pipe 20 all from any to any in via vtnet0
 #We don't want to block traffic, only shape some #We don't want to block traffic, only shape some
-${fwcmd} add 3000 allow ip from any to any +\${fwcmd} add 3000 allow ip from any to any 
-'EOF'+EOF
  
 service netif restart service netif restart
 service routing restart service routing restart
 service ipfw start service ipfw start
 +hostname VM3
 config save config save
 </code> </code>
Line 139: Line 148:
 === Router 4 === === Router 4 ===
  
-Router 4 is the aggregating server's default gateway.+Router 4 is configured for rate-limiting traffic at 10 Mb/s on interface to/from VM1.
  
 <code> <code>
-sysrc hostname=R4 +sysrc hostname=VM4 \ 
-sysrc ifconfig_vtnet1="10.0.24.4/24" +        ifconfig_vtnet0="inet 10.0.14.4/24" \ 
-sysrc ifconfig_vtnet2="10.0.34.4/24" +        ifconfig_vtnet3="inet 10.0.45.4/24" \ 
-sysrc ifconfig_vtnet3="10.0.45.4/24" +        defaultrouter="10.0.45.5" \ 
-sysrc static_routes="R2 R3" +        firewall_enable=YES \ 
-sysrc route_R2="-net 10.0.12.0/24 10.0.24.2" +        firewall_script="/etc/ipfw.rules" 
-sysrc route_R3="-net 10.0.13.0/24 10.0.34.3"+ 
 +cat > /etc/ipfw.rules <<EOF 
 +#!/bin/sh 
 +fwcmd="/sbin/ipfw" 
 +kldstat -q -m dummynet || kldload dummynet 
 +# Flush out the list before we begin. 
 +\${fwcmd} -f flush 
 +\${fwcmd} pipe 10 config bw 10Mbit/s 
 +\${fwcmd} pipe 20 config bw 10Mbit/s 
 +#Traffic getting out vtnet0 is limited to 10Mbit/s 
 +\${fwcmd} add 1000 pipe 10 all from any to any out via vtnet0 
 +#Traffic getting int vten0 is limited to 10Mbit/s 
 +\${fwcmd} add 2000 pipe 20 all from any to any in via vtnet0 
 +#We don't want to block traffic, only shape some 
 +\${fwcmd} add 3000 allow ip from any to any 
 +EOF 
 service netif restart service netif restart
 service routing restart service routing restart
 +service ipfw start
 +hostname VM4
 +config save
 +</code>
 +
 +=== Router 5 ===
 +
 +Router 5 is the aggregating server's default gateway.
 +
 +<code>
 +sysrc hostname=R5 \
 +        ifconfig_vtnet1="inet 10.0.25.5/24" \
 +        ifconfig_vtnet2="inet 10.0.35.5/24" \
 +        ifconfig_vtnet3="inet 10.0.45.5/24" \
 +        ifconfig_vtnet4="inet 10.0.56.5/24" \
 +        static_routes="ISP1 ISP2 ISP3" \
 +        route_ISP1="-host 10.0.12.1 10.0.25.2" \
 +        route_ISP2="-host 10.0.13.1 10.0.35.3" \
 +        route_ISP3="-host 10.0.14.1 10.0.45.4"
 +service netif restart
 +service routing restart
 +hostname VM5
 config save config save
 </code> </code>
Line 156: Line 203:
  
 Router 1 is configured as a  MLVPN client router connected to 3 different Internet links. Router 1 is configured as a  MLVPN client router connected to 3 different Internet links.
-The big difference with MLPPP: We can't use 3 differents IP addresses on our server, then can't simply install 3 differents static routes. We need a 3 default routes, then a minimum of 4 differents routing table. 
  
-<code> +We need a default routes for each ISP links, then a minimum of 4 different routing tables.
-sysrc hostname=R1 +
-sysrc cloned_interfaces="lo1" +
-sysrc ifconfig_lo1="inet 10.1.1.1/32" +
-sysrc ifconfig_vtnet0="10.0.12.1/24 fib 2" +
-sysrc ifconfig_vtnet1="10.0.13.1/24 fib 3" +
-sysrc static_routes="ISP1 ISP2" +
-sysrc route_ISP1="-fib 2 default 10.0.12.2" +
-sysrc route_ISP2="-fib 3 default 10.0.13.3" +
-sysrc mlvpn_enable=YES+
  
 +<code>
 +sysrc hostname=VM1 \
 +        cloned_interfaces="lo1" \
 +        ifconfig_lo1="inet 10.1.1.1/32" \
 +        ifconfig_vtnet0="inet 10.0.12.1/24 fib 2" \
 +        ifconfig_vtnet1="inet 10.0.13.1/24 fib 3" \
 +        ifconfig_vtnet2="inet 10.0.14.1/24 fib 4" \
 +        static_routes="ISP1 ISP2 ISP3" \
 +        route_ISP1="-fib 2 default 10.0.12.2" \
 +        route_ISP2="-fib 3 default 10.0.13.3" \
 +        route_ISP3="-fib 4 default 10.0.14.4"
 cat <<EOF > /usr/local/etc/mlvpn/mlvpn.conf cat <<EOF > /usr/local/etc/mlvpn/mlvpn.conf
 [general] [general]
Line 175: Line 223:
 mtu = 1452 mtu = 1452
 tuntap = "tun" tuntap = "tun"
-ip4 = "10.0.15.1/30" +ip4 = "10.0.16.1/30" 
-ip4_gateway = "10.0.15.5" +ip4_gateway = "10.0.16.2" 
-ip4_routes = "10.5.5.5/32"+ip4_routes = "10.6.6.6/32"
 timeout = 30 timeout = 30
 password = "pleasechangeme!" password = "pleasechangeme!"
Line 187: Line 235:
 bindport = 5082 bindport = 5082
 bindfib = 2 bindfib = 2
-remotehost = "10.0.45.5"+remotehost = "10.0.56.6"
 remoteport = 5082 remoteport = 5082
- 
 [dsl3] [dsl3]
 bindhost = "10.0.13.1" bindhost = "10.0.13.1"
 bindport = 5083 bindport = 5083
 bindfib = 3 bindfib = 3
-remotehost = "10.0.45.5"+remotehost = "10.0.56.6"
 remoteport = 5083 remoteport = 5083
  
-EOF+[dsl4] 
 +bindhost = "10.0.14.1" 
 +bindport = 5084 
 +bindfib = 4 
 +remotehost = "10.0.56.6" 
 +remoteport = 5084
  
 +EOF
 +service mlvpn enable
 service netif restart service netif restart
 service routing restart service routing restart
 service mlvpn start service mlvpn start
 +hostname VM1
 config save config save
 </code> </code>
  
-==== Router 5 : MLVPN server ====+==== Router 6 : MLVPN server ====
  
-Router 5 is configured as a aggregating server.+Router 6 is configured as a aggregating server.
  
 <code> <code>
-sysrc hostname=R5 +sysrc hostname=VM6 \ 
-sysrc cloned_interfaces="lo1" +        cloned_interfaces="lo1" \ 
-sysrc ifconfig_lo1="inet 10.5.5.5/32" +        ifconfig_lo1="inet 10.6.6.6/32" \ 
-sysrc ifconfig_vtnet3="10.0.45.5/24" +        ifconfig_vtnet4="inet 10.0.56.6/24" \ 
-sysrc defaultrouter=10.0.45.4 +        defaultrouter="10.0.56.5" 
-sysrc mlvpn_enable=YES +cat > /usr/local/etc/mlvpn/mlvpn.conf <<EOF
- +
-cat <<'EOF' > /usr/local/etc/mlvpn/mlvpn.conf+
 [general] [general]
 statuscommand = "/usr/local/etc/mlvpn/mlvpn_updown.sh" statuscommand = "/usr/local/etc/mlvpn/mlvpn_updown.sh"
 tuntap = "tun" tuntap = "tun"
 mode = "server" mode = "server"
-ip4 = "10.0.15.5/30" +ip4 = "10.0.16.2/30" 
-ip4_gateway = "10.0.15.1"+ip4_gateway = "10.0.16.1"
 ip4_routes = "10.1.1.1/32" ip4_routes = "10.1.1.1/32"
 timeout = 30 timeout = 30
Line 231: Line 284:
  
 [adsl2] [adsl2]
-bindhost = "10.0.45.5"+bindhost = "10.0.56.6"
 bindport = 5082 bindport = 5082
  
 [adsl3] [adsl3]
-bindhost = "10.0.45.5"+bindhost = "10.0.56.6"
 bindport = 5083 bindport = 5083
-'EOF' 
  
 +[adsl4]
 +bindhost = "10.0.56.6"
 +bindport = 5084
 +
 +EOF
 +
 +service mlvpn enable
 service netif restart service netif restart
 service routing restart service routing restart
 service mlvpn start service mlvpn start
 +hostname VM6
 config save config save
 </code> </code>
Line 313: Line 373:
 MLVPN can be started in debug mode:  MLVPN can be started in debug mode: 
 <code> <code>
-[root@R1]# mlvpn --debug -n mlvpn -u mlvpn +[root@VM1]~# mlvpn --debug -n mlvpn -u mlvpn --config /usr/local/etc/mlvpn/mlvpn.conf 
-2016-04-19T23:48:21 [INFO/config] new password set +2020-02-21T21:25:12 [INFO/config] new password set 
-2016-04-19T23:48:21 [INFO/config] dsl2 tunnel added +2020-02-21T21:25:12 [INFO/config] dsl2 tunnel added 
-2016-04-19T23:48:21 [INFO/config] dsl3 tunnel added +2020-02-21T21:25:12 [INFO/config] dsl3 tunnel added 
-2016-04-19T23:48:21 [INFO] created interface `tun0' +2020-02-21T21:25:12 [INFO/config] dsl4 tunnel added 
-2016-04-19T23:48:21 [INFO] dsl2 bind to 10.0.12.1 +2020-02-21T21:25:12 [INFO] created interface `tun0' 
-2016-04-19T23:48:21 [INFO] dsl3 bind to 10.0.13.1 +2020-02-21T21:25:12 [INFO] dsl2 bind to 10.0.12.1 
-2016-04-19T23:48:21 [INFO/protocol] dsl3 authenticated +2020-02-21T21:25:12 [INFO] dsl3 bind to 10.0.13.1 
-2016-04-19T23:48:21 [INFO/protocol] dsl2 authenticated+2020-02-21T21:25:12 [INFO] dsl4 bind to 10.0.14.1 
 +2020-02-21T21:25:12 [INFO/protocol] dsl2 authenticated 
 +2020-02-21T21:25:12 [INFO/protocol] dsl3 authenticated 
 +2020-02-21T21:25:12 [INFO/protocol] dsl4 authenticated
 </code> </code>
  
 tun interface need to be check (correct IP address and non-1500 MTU): tun interface need to be check (correct IP address and non-1500 MTU):
 <code> <code>
-[root@R1]# ifconfig tun0+[root@VM1]~# ifconfig tun0
 tun0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1452 tun0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1452
         options=80000<LINKSTATE>         options=80000<LINKSTATE>
-        inet6 fe80::5a9c:fcff:fe01:201%tun0 prefixlen 64 scopeid 0x7 +        inet6 fe80::5a9c:fcff:fe01:201%tun0 prefixlen 64 scopeid 0x9 
-        inet 10.0.15.1 --> 10.0.15.5 netmask 0xfffffffc+        inet 10.0.16.1 --> 10.0.16.2 netmask 0xfffffffc 
 +        groups: tun
         nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>         nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
-        Opened by PID 2326+        Opened by PID 92891
 </code> </code>
  
 And static route(s) needs to be installed (10.5.5.5/32 in this example): And static route(s) needs to be installed (10.5.5.5/32 in this example):
 <code> <code>
-[root@R1]~# netstat -rn4 +[root@VM1]~# route get 10.6.6.6 
-Routing tables +   route to: 10.6.6.6 
- +destination: 10.6.6.6 
-Internet: +       mask: 255.255.255.255 
-Destination        Gateway            Flags      Netif Expire +    gateway: 10.0.16.2 
-10.0.12.0/24       link#1             U        vtnet0 +        fib: 0 
-10.0.13.0/24       link#2             U        vtnet1 +  interface: tun0 
-10.0.15.1          link#7             UHS         lo0 +      flags: <UP,GATEWAY,DONE,STATIC> 
-10.0.15.5          link#7             UH         tun0 + recvpipe  sendpipe  ssthresh  rtt,msec    mtu        weight    expire 
-10.5.5.5/32        10.0.15.5          UGS        tun0 +       0         0         0         0      1452         1         0
-127.0.0.1          link#6             UH          lo0+
 </code> </code>
 ==== Aggregated bandwidth ==== ==== Aggregated bandwidth ====
  
-Check that aggregated bandwitdh is 10+10 = 20Mbit/s on this lab.+Check that aggregated bandwitdh is 10+10+10 = 30Mbit/s on this lab.
  
 <code> <code>
-[root@R1]# iperf --bind 10.1.1.1 -c 10.5.5.5 -t 60 +[root@VM1]~# iperf3 -B 10.1.1.1 -c 10.6.6.6 
------------------------------------------------------------- +(...) 
-Client connecting to 10.5.5.5, TCP port 5001 +[ ID] Interval           Transfer     Bitrate         Retr 
-Binding to local address 10.1.1.1 +[  5]   0.00-10.00  sec  7.89 MBytes  6.62 Mbits/sec  428             sender 
-TCP window size: 32.3 KByte (default) +[  5]   0.00-10.01  sec  7.85 MBytes  6.58 Mbits/sec                  receiver
------------------------------------------------------------- +
-[  3] local 10.1.1.1 port 5001 connected with 10.5.5.5 port 5001 +
-[ ID] Interval       Transfer     Bandwidth +
-[  3]  0.0-60.0 sec   129 MBytes  18.1 Mbits/sec+
 </code> </code>
 +
 +Ouch, not the expected performance :-(
documentation/examples/aggregating_multiple_isp_links_with_mlvpn.1582316679.txt.gz · Last modified: 2020/02/21 21:24 by olivier
Except where otherwise noted, content on this wiki is licensed under the following license: BSD 2-Clause
Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki