User Tools

Site Tools


documentation:examples:aggregating_multiple_isp_links_with_mlvpn

This is an old revision of the document!


Aggregating multiple ISP links

This lab shows an example of aggregating multiple independent ISP links with MLVPN.

Network diagram

Here is the concept:

And here is this lab detailed diagram:

Virtual Lab setp

This chapter will describe how to start each routers and configuring the 3 centrals routers.

More information on these BSDRP lab scripts available on How to build a BSDRP router lab.

Start the Virtual lab (example using bhyve):

# ./tools/BSDRP-lab-bhyve.sh -n 5
BSD Router Project (http://bsdrp.net) - bhyve full-meshed lab script
Setting-up a virtual lab with 5 VM(s):
- Working directory: /tmp/BSDRP
- Each VM have 1 core(s) and 256M RAM
- Switch mode: bridge + tap
- 0 LAN(s) between all VM
- Full mesh Ethernet links between each VM
VM 1 have the following NIC:
- vtnet0 connected to VM 2.
- vtnet1 connected to VM 3.
- vtnet2 connected to VM 4.
- vtnet3 connected to VM 5.
VM 2 have the following NIC:
- vtnet0 connected to VM 1.
- vtnet1 connected to VM 3.
- vtnet2 connected to VM 4.
- vtnet3 connected to VM 5.
VM 3 have the following NIC:
- vtnet0 connected to VM 1.
- vtnet1 connected to VM 2.
- vtnet2 connected to VM 4.
- vtnet3 connected to VM 5.
VM 4 have the following NIC:
- vtnet0 connected to VM 1.
- vtnet1 connected to VM 2.
- vtnet2 connected to VM 3.
- vtnet3 connected to VM 5.
VM 5 have the following NIC:
- vtnet0 connected to VM 1.
- vtnet1 connected to VM 2.
- vtnet2 connected to VM 3.
- vtnet3 connected to VM 4.
For connecting to VM'serial console, you can use:
- VM 1 : cu -l /dev/nmdm1B
- VM 2 : cu -l /dev/nmdm2B
- VM 3 : cu -l /dev/nmdm3B
- VM 4 : cu -l /dev/nmdm4B
- VM 5 : cu -l /dev/nmdm5B

Backbone routers configuration

Router 2

Router 2 is configured for rate-limiting traffic at 10 Mb/s on interface to/from R1.

sysrc hostname=R2
sysrc ifconfig_vtnet0="10.0.12.2/24"
sysrc ifconfig_vtnet2="10.0.24.2/24"
sysrc static_routes="R5"
sysrc route_R5="-net 10.0.45.0/24 10.0.24.4"
sysrc firewall_enable=YES
sysrc firewall_script="/etc/ipfw.rules"

cat > /etc/ipfw.rules <<'EOF'
#!/bin/sh
fwcmd="/sbin/ipfw"
kldstat -q -m dummynet || kldload dummynet
# Flush out the list before we begin.
${fwcmd} -f flush
#Create pipes (one for each direction)
${fwcmd} pipe 10 config bw 10Mbit/s
${fwcmd} pipe 20 config bw 10Mbit/s
#Traffic getting out vtnet0 is limited to 10Mbit/s
${fwcmd} add 1000 pipe 10 all from any to any out via vtnet0
#Traffic getting int vtnet0 is limited to 10Mbit/s
${fwcmd} add 2000 pipe 20 all from any to any in via vtnet0
#We don't want to block traffic, only shape some
${fwcmd} add 3000 allow ip from any to any
'EOF'

service netif restart
service routing restart
service ipfw start
config save

Router 3

Router 3 is configured for rate-limiting traffic at 10 Mb/s on interface to/from R1.

sysrc hostname=R3
sysrc ifconfig_vtnet0="10.0.13.3/24"
sysrc ifconfig_vtnet2="10.0.34.3/24"
sysrc static_routes="R5"
sysrc route_R5="-net 10.0.45.0/24 10.0.34.4"
sysrc firewall_enable=YES
sysrc firewall_script="/etc/ipfw.rules"

cat > /etc/ipfw.rules <<'EOF'
#!/bin/sh
fwcmd="/sbin/ipfw"
kldstat -q -m dummynet || kldload dummynet
# Flush out the list before we begin.
${fwcmd} -f flush
#Create pipes (one for each direction)
${fwcmd} pipe 10 config bw 10Mbit/s
${fwcmd} pipe 20 config bw 10Mbit/s
#Traffic getting out vtnet0 is limited to 10Mbit/s
${fwcmd} add 1000 pipe 10 all from any to any out via vtnet0
#Traffic getting int vtnet0 is limited to 10Mbit/s
${fwcmd} add 2000 pipe 20 all from any to any in via vtnet0
#We don't want to block traffic, only shape some
${fwcmd} add 3000 allow ip from any to any
'EOF'

service netif restart
service routing restart
service ipfw start
config save

Router 4

Router 4 is the aggregating server's default gateway.

sysrc hostname=R4
sysrc ifconfig_vtnet1="10.0.24.4/24"
sysrc ifconfig_vtnet2="10.0.34.4/24"
sysrc ifconfig_vtnet3="10.0.45.4/24"
sysrc static_routes="R2 R3"
sysrc route_R2="-net 10.0.12.0/24 10.0.24.2"
sysrc route_R3="-net 10.0.13.0/24 10.0.34.3"
service netif restart
service routing restart
config save

Router 1 : MLVPN client

Router 1 is configured as a MLVPN client router connected to 3 different Internet links. The big difference with MLPPP: We can't use 3 differents IP addresses on our server, then can't simply install 3 differents static routes. We need a 3 default routes, then a minimum of 4 differents routing table.

sysrc hostname=R1
sysrc cloned_interfaces="lo1"
sysrc ifconfig_lo1="inet 10.1.1.1/32"
sysrc ifconfig_vtnet0="10.0.12.1/24 fib 2"
sysrc ifconfig_vtnet1="10.0.13.1/24 fib 3"
sysrc static_routes="ISP1 ISP2"
sysrc route_ISP1="-fib 2 default 10.0.12.2"
sysrc route_ISP2="-fib 3 default 10.0.13.3"
sysrc mlvpn_enable=YES

cat <<EOF > /usr/local/etc/mlvpn/mlvpn.conf
[general]
statuscommand = "/usr/local/etc/mlvpn/mlvpn_updown.sh"
mode = "client"
mtu = 1452
tuntap = "tun"
ip4 = "10.0.15.1/30"
ip4_gateway = "10.0.15.5"
ip4_routes = "10.5.5.5/32"
timeout = 30
password = "pleasechangeme!"
#reorder_buffer_size = 64
loss_tolerence = 10

[dsl2]
bindhost = "10.0.12.1"
bindport = 5082
bindfib = 2
remotehost = "10.0.45.5"
remoteport = 5082

[dsl3]
bindhost = "10.0.13.1"
bindport = 5083
bindfib = 3
remotehost = "10.0.45.5"
remoteport = 5083

EOF

service netif restart
service routing restart
service mlvpn start
config save

Router 5 : MLVPN server

Router 5 is configured as a aggregating server.

sysrc hostname=R5
sysrc cloned_interfaces="lo1"
sysrc ifconfig_lo1="inet 10.5.5.5/32"
sysrc ifconfig_vtnet3="10.0.45.5/24"
sysrc defaultrouter=10.0.45.4
sysrc mlvpn_enable=YES

cat <<'EOF' > /usr/local/etc/mlvpn/mlvpn.conf
[general]
statuscommand = "/usr/local/etc/mlvpn/mlvpn_updown.sh"
tuntap = "tun"
mode = "server"
ip4 = "10.0.15.5/30"
ip4_gateway = "10.0.15.1"
ip4_routes = "10.1.1.1/32"
timeout = 30
password = "pleasechangeme!"
#reorder_buffer_size = 64
loss_tolerence = 10

[adsl2]
bindhost = "10.0.45.5"
bindport = 5082

[adsl3]
bindhost = "10.0.45.5"
bindport = 5083
'EOF'

service netif restart
service routing restart
service mlvpn start
config save

Basic Tests

FIB test

Start by checking that R5 is reacheable from each R1's fib (2, 3):

[root@VM1]~# setfib 2 ping -c 2 10.0.56.6
PING 10.0.56.6 (10.0.56.6): 56 data bytes
64 bytes from 10.0.56.6: icmp_seq=0 ttl=62 time=16.473 ms
64 bytes from 10.0.56.6: icmp_seq=1 ttl=62 time=20.017 ms

--- 10.0.56.6 ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 16.473/18.245/20.017/1.772 ms
[root@VM1]~# setfib 3 ping -c 2 10.0.56.6
PING 10.0.56.6 (10.0.56.6): 56 data bytes
64 bytes from 10.0.56.6: icmp_seq=0 ttl=62 time=18.202 ms
64 bytes from 10.0.56.6: icmp_seq=1 ttl=62 time=11.193 ms

--- 10.0.56.6 ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 11.193/14.698/18.202/3.504 ms
[root@VM1]~# setfib 4 ping -c 2 10.0.56.6
PING 10.0.56.6 (10.0.56.6): 56 data bytes
64 bytes from 10.0.56.6: icmp_seq=0 ttl=62 time=10.973 ms
64 bytes from 10.0.56.6: icmp_seq=1 ttl=62 time=14.465 ms

--- 10.0.56.6 ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 10.973/12.719/14.465/1.746 ms

Test bandwidth of each link by starting an iperf on MLVPN server:

[root@VM6]# iperf3 -s

Then from the MLVPN client, test bandwidth for each ISP links:

[root@VM1]~# setfib 2 iperf3 -c 10.0.56.6
(...)
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  11.5 MBytes  9.62 Mbits/sec    0             sender
[  5]   0.00-10.06  sec  11.4 MBytes  9.53 Mbits/sec                  receiver

[root@VM1]~# setfib 3 iperf3 -c 10.0.56.6
(...)
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  11.4 MBytes  9.57 Mbits/sec    3             sender
[  5]   0.00-10.06  sec  11.4 MBytes  9.47 Mbits/sec                  receiver


[root@VM1]~# setfib 4 iperf3 -c 10.0.56.6
Connecting to host 10.0.56.6, port 5201
(...)
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  11.5 MBytes  9.62 Mbits/sec    0             sender
[  5]   0.00-10.06  sec  11.4 MBytes  9.53 Mbits/sec                  receiver

MLVPN tests

tunnel

MLVPN can be started in debug mode:

[root@R1]# mlvpn --debug -n mlvpn -u mlvpn
2016-04-19T23:48:21 [INFO/config] new password set
2016-04-19T23:48:21 [INFO/config] dsl2 tunnel added
2016-04-19T23:48:21 [INFO/config] dsl3 tunnel added
2016-04-19T23:48:21 [INFO] created interface `tun0'
2016-04-19T23:48:21 [INFO] dsl2 bind to 10.0.12.1
2016-04-19T23:48:21 [INFO] dsl3 bind to 10.0.13.1
2016-04-19T23:48:21 [INFO/protocol] dsl3 authenticated
2016-04-19T23:48:21 [INFO/protocol] dsl2 authenticated

tun interface need to be check (correct IP address and non-1500 MTU):

[root@R1]# ifconfig tun0
tun0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1452
        options=80000<LINKSTATE>
        inet6 fe80::5a9c:fcff:fe01:201%tun0 prefixlen 64 scopeid 0x7
        inet 10.0.15.1 --> 10.0.15.5 netmask 0xfffffffc
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        Opened by PID 2326

And static route(s) needs to be installed (10.5.5.5/32 in this example):

[root@R1]~# netstat -rn4
Routing tables

Internet:
Destination        Gateway            Flags      Netif Expire
10.0.12.0/24       link#1             U        vtnet0
10.0.13.0/24       link#2             U        vtnet1
10.0.15.1          link#7             UHS         lo0
10.0.15.5          link#7             UH         tun0
10.5.5.5/32        10.0.15.5          UGS        tun0
127.0.0.1          link#6             UH          lo0

Aggregated bandwidth

Check that aggregated bandwitdh is 10+10 = 20Mbit/s on this lab.

[root@R1]# iperf --bind 10.1.1.1 -c 10.5.5.5 -t 60
------------------------------------------------------------
Client connecting to 10.5.5.5, TCP port 5001
Binding to local address 10.1.1.1
TCP window size: 32.3 KByte (default)
------------------------------------------------------------
[  3] local 10.1.1.1 port 5001 connected with 10.5.5.5 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-60.0 sec   129 MBytes  18.1 Mbits/sec
documentation/examples/aggregating_multiple_isp_links_with_mlvpn.1582316679.txt.gz · Last modified: 2020/02/21 21:24 by olivier