This lab test a cool OpenVPN's patch: lowest-latency-server. This mean when multiple remote servers are configured, we measure their latency and connect in priority to the lowest latency.
Lab build following How to build a BSDRP router lab: 5 routers with full-meshed link.
Here is the logical and physical view:
We start a 5 routers full-mesh lab with one shared LAN:
root@lab:~ # /tools/BSDRP-lab-bhyve.sh -i BSDRP-1.591-full-amd64-vga.img.xz -n 5 -l 1 vmm module not loaded. Loading it... if_tap module not loaded. Loading it... BSD Router Project (http://bsdrp.net) - bhyve full-meshed lab script Setting-up a virtual lab with 5 VM(s): - Working directory: /tmp/BSDRP - Each VM have 1 core(s) and 256M RAM - Switch mode: bridge + tap - 1 LAN(s) between all VM - Full mesh Ethernet links between each VM VM 1 have the following NIC: - vtnet0 connected to VM 2. - vtnet1 connected to VM 3. - vtnet2 connected to VM 4. - vtnet3 connected to VM 5. - vtnet4 connected to LAN number 1 VM 2 have the following NIC: - vtnet0 connected to VM 1. - vtnet1 connected to VM 3. - vtnet2 connected to VM 4. - vtnet3 connected to VM 5. - vtnet4 connected to LAN number 1 VM 3 have the following NIC: - vtnet0 connected to VM 1. - vtnet1 connected to VM 2. - vtnet2 connected to VM 4. - vtnet3 connected to VM 5. - vtnet4 connected to LAN number 1 VM 4 have the following NIC: - vtnet0 connected to VM 1. - vtnet1 connected to VM 2. - vtnet2 connected to VM 3. - vtnet3 connected to VM 5. - vtnet4 connected to LAN number 1 VM 5 have the following NIC: - vtnet0 connected to VM 1. - vtnet1 connected to VM 2. - vtnet2 connected to VM 3. - vtnet3 connected to VM 4. - vtnet4 connected to LAN number 1 For connecting to VM'serial console, you can use: - VM 1 : cu -l /dev/nmdm1B - VM 2 : cu -l /dev/nmdm2B - VM 3 : cu -l /dev/nmdm3B - VM 4 : cu -l /dev/nmdm4B - VM 5 : cu -l /dev/nmdm5B
sysrc hostname=R1 sysrc ifconfig_em0="inet 10.0.1.1/24" sysrc ifconfig_em0_ipv6="inet6 2001:db8:1::1 prefixlen 64" sysrc ifconfig_em3="inet 192.168.15.1/24" sysrc ifconfig_em3_ipv6="inet6 2001:db8:15::1 prefixlen 64" sysrc defaultrouter=192.168.15.5 sysrc ipv6_defaultrouter=2001:db8:15::5 ifconfig -l | grep -q vtnet && sed -i "" 's/em/vtnet/g' /etc/rc.conf service netif restart service routing restart config save
sysrc hostname=R2 sysrc ifconfig_em4="inet 10.0.2.2/24" sysrc ifconfig_em4_ipv6="inet6 2001:db8:2::2 prefixlen 64" sysrc ifconfig_em3="inet 192.168.25.2/24" sysrc ifconfig_em3_ipv6="inet6 2001:db8:25::2 prefixlen 64" sysrc defaultrouter="192.168.25.5" sysrc ipv6_defaultrouter="2001:db8:25::5" ifconfig -l | grep -q vtnet && sed -i "" 's/em/vtnet/g' /etc/rc.conf service netif restart service routing restart config save
sysrc hostname=R3 sysrc ifconfig_em4="inet 10.0.2.3/24" sysrc ifconfig_em4_ipv6="inet6 2001:db8:2::3 prefixlen 64" sysrc ifconfig_em3="inet 192.168.35.3/24" sysrc ifconfig_em3_ipv6="inet6 2001:db8:35::3 prefixlen 64" sysrc defaultrouter="192.168.35.5" sysrc ipv6_defaultrouter="2001:db8:35::5" ifconfig -l | grep -q vtnet && sed -i "" 's/em/vtnet/g' /etc/rc.conf service netif restart service routing restart config save
Router 4 base configuration, like R2: A simple connected-network router with a default route pointing to R3.
sysrc hostname=R4 sysrc ifconfig_em4="inet 10.0.2.4/24" sysrc ifconfig_em4_ipv6="inet6 2001:db8:2::4 prefixlen 64" sysrc ifconfig_em3="inet 192.168.45.4/24" sysrc ifconfig_em3_ipv6="inet6 2001:db8:45::4 prefixlen 64" sysrc defaultrouter="192.168.45.5" sysrc ipv6_defaultrouter="2001:db8:45::5" ifconfig -l | grep -q vtnet && sed -i "" 's/em/vtnet/g' /etc/rc.conf service netif restart service routing restart config save
Router 5 is the central router simulating Internet and low latency link.
sysrc hostname=R5 sysrc ifconfig_em0="inet 192.168.15.5/24" sysrc ifconfig_em1="inet 192.168.25.5/24" sysrc ifconfig_em2="inet 192.168.35.5/24" sysrc ifconfig_em3="inet 192.168.45.5/24" sysrc ifconfig_em0_ipv6="inet6 2001:db8:15::5 prefixlen 64" sysrc ifconfig_em1_ipv6="inet6 2001:db8:25::5 prefixlen 64" sysrc ifconfig_em2_ipv6="inet6 2001:db8:35::5 prefixlen 64" sysrc ifconfig_em3_ipv6="inet6 2001:db8:45::5 prefixlen 64" sysrc firewall_enable=YES sysrc firewall_script="/etc/ipfw.rules" ifconfig -l | grep -q vtnet && sed -i "" 's/em/vtnet/g' /etc/rc.conf cat > /etc/ipfw.rules <<EOF #!/bin/sh fwcmd="/sbin/ipfw" kldstat -q -m dummynet || kldload dummynet # Flush out the list before we begin. \${fwcmd} -f flush #Create pipes (one for each direction) \${fwcmd} pipe 52 config delay 100ms \${fwcmd} pipe 25 config delay 100ms \${fwcmd} pipe 53 config delay 50ms \${fwcmd} pipe 35 config delay 50ms \${fwcmd} add pipe 25 all from any to any in via em1 \${fwcmd} add pipe 52 all from any to any out via em1 \${fwcmd} add pipe 35 all from any to any in via em2 \${fwcmd} add pipe 53 all from any to any out via em2 #We don't want to block traffic, only shape some \${fwcmd} add allow ip from any to any EOF ifconfig -l | grep -q vtnet && sed -i "" 's/em/vtnet/g' /etc/ipfw.rules service netif restart service routing restart config save
All these step will be done on R2 (OpenVPN server & CA)
Start by copying easyrsa3 configuration folder and define new configuration file:
cp -r /usr/local/share/easy-rsa /usr/local/etc/ setenv EASYRSA /usr/local/etc/easy-rsa
Initialize PKI and generate a DH:
easyrsa init-pki easyrsa gen-dh
Build a root certificate:
[root@R2]~# easyrsa build-ca nopass Note: using Easy-RSA configuration from: /usr/local/etc/easy-rsa/vars Generating a 2048 bit RSA private key ...............................................+++ ..................................................................................+++ writing new private key to '/usr/local/etc/easy-rsa/pki/private/ca.key.EvwYAl9tEs' ----- You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ----- Common Name (eg: your user, host, or server name) [Easy-RSA CA]: CA creation complete and you may now import and sign cert requests. Your new CA certificate file for publishing is at: /usr/local/etc/easy-rsa/pki/ca.crt
Make a server certificate called R2, R3 and R4. Then client certificate called R1:
easyrsa build-server-full R2 nopass easyrsa build-server-full R3 nopass easyrsa build-server-full R4 nopass easyrsa build-client-full R1 nopass config save
Create the openvpn configuration file for server mode as /usr/local/etc/openvpn/openvpn.conf:
mkdir /usr/local/etc/openvpn cat > /usr/local/etc/openvpn/openvpn.conf <<'EOF' dev tun tun-ipv6 ca /usr/local/etc/easy-rsa/pki/ca.crt cert /usr/local/etc/easy-rsa/pki/issued/R2.crt key /usr/local/etc/easy-rsa/pki/private/R2.key dh /usr/local/etc/easy-rsa/pki/dh.pem server 10.0.21.0 255.255.255.0 server-ipv6 2001:db8:21::/64 keepalive 5 60 ifconfig-pool-persist ipp.txt client-config-dir ccd push "route 10.0.2.0 255.255.255.0" push "route-ipv6 2001:db8:2::/64" route 10.0.1.0 255.255.255.0 route-ipv6 2001:db8:1::/64 'EOF'
Create the Client-Configuration-dir and declare the volatile route to the subnet behind the client R1:
mkdir /usr/local/etc/openvpn/ccd cat > /usr/local/etc/openvpn/ccd/R1 <<'EOF' iroute 10.0.1.0 255.255.255.0 iroute-ipv6 2001:db8:1::/64 'EOF'
Enable and start openvpn and sshd (we will get certificates files by SCP later):
sysrc sshd_enable=YES sysrc openvpn_enable=YES service openvpn start service sshd start
And set a password for root account (mandatory for next SCP file copy):
passwd
Create the openvpn configuration file for server mode as /usr/local/etc/openvpn/openvpn.conf:
mkdir /usr/local/etc/openvpn cat > /usr/local/etc/openvpn/openvpn.conf <<'EOF' dev tun tun-ipv6 ca ca.crt cert R3.crt key R3.key dh dh.pem server 10.0.31.0 255.255.255.0 server-ipv6 2001:db8:31::/64 keepalive 5 60 ifconfig-pool-persist ipp.txt client-config-dir ccd push "route 10.0.2.0 255.255.255.0" push "route-ipv6 2001:db8:2::/64" route 10.0.1.0 255.255.255.0 route-ipv6 2001:db8:1::/64 'EOF'
Create the Client-Configuration-dir and declare the volatile route to the subnet behind the client R1:
mkdir /usr/local/etc/openvpn/ccd cat > /usr/local/etc/openvpn/ccd/R1 <<'EOF' iroute 10.0.1.0 255.255.255.0 iroute-ipv6 2001:db8:1::/64 'EOF'
Then get CA and your own certificate from R2
scp 192.168.25.2:/usr/local/etc/easy-rsa/pki/ca.crt /usr/local/etc/openvpn scp 192.168.25.2:/usr/local/etc/easy-rsa/pki/dh.pem /usr/local/etc/openvpn scp 192.168.25.2:/usr/local/etc/easy-rsa/pki/issued/R3.crt /usr/local/etc/openvpn scp 192.168.25.2:/usr/local/etc/easy-rsa/pki/private/R3.key /usr/local/etc/openvpn
Enable and start openvpn:
sysrc openvpn_enable=YES service openvpn start
Create the openvpn configuration file for server mode as /usr/local/etc/openvpn/openvpn.conf:
mkdir /usr/local/etc/openvpn cat > /usr/local/etc/openvpn/openvpn.conf <<'EOF' dev tun tun-ipv6 ca ca.crt cert R4.crt key R4.key dh dh.pem server 10.0.41.0 255.255.255.0 server-ipv6 2001:db8:41::/64 keepalive 5 60 ifconfig-pool-persist ipp.txt client-config-dir ccd push "route 10.0.2.0 255.255.255.0" push "route-ipv6 2001:db8:2::/64" route 10.0.1.0 255.255.255.0 route-ipv6 2001:db8:1::/64 'EOF'
Create the Client-Configuration-dir and declare the volatile route to the subnet behind the client R1:
mkdir /usr/local/etc/openvpn/ccd cat > /usr/local/etc/openvpn/ccd/R1 <<'EOF' iroute 10.0.1.0 255.255.255.0 iroute-ipv6 2001:db8:1::/64 'EOF'
Then get CA and your own certificate from R2
scp 192.168.25.2:/usr/local/etc/easy-rsa/pki/ca.crt /usr/local/etc/openvpn scp 192.168.25.2:/usr/local/etc/easy-rsa/pki/dh.pem /usr/local/etc/openvpn scp 192.168.25.2:/usr/local/etc/easy-rsa/pki/issued/R4.crt /usr/local/etc/openvpn scp 192.168.25.2:/usr/local/etc/easy-rsa/pki/private/R4.key /usr/local/etc/openvpn
Enable and start openvpn:
sysrc openvpn_enable=YES service openvpn start
As OpenVPN client, R1 should get these files from R2 and put them in /usr/local/etc/openvpn:
On this lab, scp can be used for getting these files:
mkdir /usr/local/etc/openvpn scp 192.168.25.2:/usr/local/etc/easy-rsa/pki/ca.crt /usr/local/etc/openvpn scp 192.168.25.2:/usr/local/etc/easy-rsa/pki/issued/R1.crt /usr/local/etc/openvpn scp 192.168.25.2:/usr/local/etc/easy-rsa/pki/private/R1.key /usr/local/etc/openvpn
Configure openvpn as a client:
cat > /usr/local/etc/openvpn/openvpn.conf <<'EOF' client dev tun #Declare servers with bigger latency first remote 192.168.25.2 remote 192.168.35.3 remote 192.168.45.4 remote-cert-tls server ca ca.crt cert R1.crt key R1.key 'EOF'
Enable and start openvpn:
sysrc openvpn_enable=YES service openvpn start
Test the current setup by checking if with unpatched OpenVPN it's works but connect only to the first declared OpenVPN server (192.168.25.2):
[root@R1]# grep openvpn /var/log/messages Jun 11 14:38:41 R1 openvpn[2499]: OpenVPN 2.3.11 amd64-portbld-freebsd10.3 [SSL (OpenSSL)] [LZO] [MH] [IPv6] built on May 31 2016 Jun 11 14:38:41 R1 openvpn[2499]: library versions: OpenSSL 1.0.1s-freebsd 1 Mar 2016, LZO 2.09 Jun 11 14:38:41 R1 openvpn[2500]: UDPv4 link local (bound): [undef] Jun 11 14:38:41 R1 openvpn[2500]: UDPv4 link remote: [AF_INET]192.168.25.2:1194 Jun 11 14:38:42 R1 openvpn[2500]: [R2] Peer Connection Initiated with [AF_INET]192.168.25.2:1194 Jun 11 14:38:45 R1 openvpn[2500]: TUN/TAP device /dev/tun0 opened Jun 11 14:38:45 R1 openvpn[2500]: do_ifconfig, tt->ipv6=1, tt->did_ifconfig_ipv6_setup=1 Jun 11 14:38:45 R1 openvpn[2500]: /sbin/ifconfig tun0 10.0.21.6 10.0.21.5 mtu 1500 netmask 255.255.255.255 up Jun 11 14:38:45 R1 openvpn[2500]: /sbin/ifconfig tun0 inet6 2001:db8:21::1000/64 Jun 11 14:38:45 R1 openvpn[2500]: add_route_ipv6(2001:db8:2::/64 -> 2001:db8:21::1 metric -1) dev tun0 Jun 11 14:38:45 R1 openvpn[2500]: Initialization Sequence Completed [root@R1]# ping -c 4 10.0.2.2 PING 10.0.2.2 (10.0.2.2): 56 data bytes 64 bytes from 10.0.2.2: icmp_seq=0 ttl=64 time=199.854 ms 64 bytes from 10.0.2.2: icmp_seq=1 ttl=64 time=199.922 ms 64 bytes from 10.0.2.2: icmp_seq=2 ttl=64 time=199.921 ms 64 bytes from 10.0.2.2: icmp_seq=3 ttl=64 time=199.925 ms --- 10.0.2.2 ping statistics --- 4 packets transmitted, 4 packets received, 0.0% packet loss round-trip min/avg/max/stddev = 199.854/199.906/199.925/0.030 ms
For this test, we start by:
server unpatched | server patched | |
---|---|---|
client unpatched | OK | OK |
client patched | OK | OK |
Now we still didn't upgrade OpenVPN on the 2 last servers (R3 and R4) but we add remote-best-latency option on client:
service openvpn stop echo "remote-best-latency" >> /usr/local/etc/openvpn/openvpn.conf
OpenVPN.conf should be like this one:
client dev tun #Declare servers with bigger latency first remote 192.168.25.2 remote 192.168.35.3 remote 192.168.45.4 remote-cert-tls server ca ca.crt cert R1.crt key R1.key remote-best-latency
Then we start openvpn for checking the new behavior:
[root@R1]/usr/local/etc/openvpn# openvpn openvpn.conf Fri Jun 17 07:04:44 2016 OpenVPN 2.3.11 amd64-portbld-freebsd10.3 [SSL (OpenSSL)] [LZO] [MH] [IPv6] built on Jun 16 2016 Fri Jun 17 07:04:44 2016 library versions: OpenSSL 1.0.1s-freebsd 1 Mar 2016, LZO 2.09 Timeout reached Timeout reached Timeout reached Timeout reached Timeout reached Timeout reached Timeout reached Timeout reached Timeout reached Timeout reached Timeout reached Fri Jun 17 07:04:54 2016 UDPv4 link local (bound): [undef] Fri Jun 17 07:04:54 2016 UDPv4 link remote: [AF_INET]192.168.45.4:1194 Fri Jun 17 07:04:54 2016 [R4] Peer Connection Initiated with [AF_INET]192.168.45.4:1194 Fri Jun 17 07:04:56 2016 TUN/TAP device /dev/tun0 opened Fri Jun 17 07:04:56 2016 do_ifconfig, tt->ipv6=1, tt->did_ifconfig_ipv6_setup=1 Fri Jun 17 07:04:56 2016 /sbin/ifconfig tun0 10.0.41.6 10.0.41.5 mtu 1500 netmask 255.255.255.255 up Fri Jun 17 07:04:56 2016 /sbin/ifconfig tun0 inet6 2001:db8:41::1000/64 add net 10.0.2.0: gateway 10.0.41.5 fib 0 add net 10.0.41.1: gateway 10.0.41.5 fib 0 Fri Jun 17 07:04:56 2016 add_route_ipv6(2001:db8:2::/64 -> 2001:db8:41::1 metric -1) dev tun0 add net 2001:db8:2::/64: gateway tun0 fib 0 Fri Jun 17 07:04:56 2016 Initialization Sequence Completed
We notice timeout message: 2 OpenVPN servers are still not upgraded and still didn't support the OpenVPN-latency-ping packets, then they timeout. But the client no more connect to the first declared server but on R4 here (why?).