====== OpenVPN performance lab of an IBM System x3550 M3 with Intel 82580 ======
{{description>OPenVPN performance lab of a quad cores Xeon 2.13GHz and quad-port gigabit Intel 82580}}
===== Hardware detail =====
This lab will test an [[IBM System x3550 M3]] with **quad** cores (Intel Xeon L5630 2.13GHz, hyper-threading disabled) and a quad NIC 82580 connected to the PCI-Express Bus.
This CPU includes AES-NI: AES-CBC,AES-XTS,AES-GCM,AES-ICM.
===== Method used =====
The benchmarking method used here is detailed in [[documentation:examples:Setting up a VPN (IPSec, GRE, etc...) performance benchmark lab]] and [[documentation/examples/gre_ipsec_and_openvpn#openvpn|OpenVPN example lab]].
==== Diagram ====
+---------------------+ +-------------------------------------+ +----------------------------------------+
| R1 | | IBM x3550 M3 | | R3 |
| Packet generator | | Device under Test | | IPSec endpoint |
| and receiver | | | | (AES-NI) |
| | | | | |
|igb2: 198.18.0.201/24|=>=| igb2: 198.18.0.202/24 | | |
| 2001:2::201/64| | 2001:2::202/64 | | |
| 00:1b:21:d4:3f:2a| | 00:1b:21:d3:8f:3e | | |
| | | | | |
| | | igb3: 198.18.1.202/24 |==>=| igb2: 198.18.1.203/24 |
| | | 2001:2:0:1::202/64 | | 2001:2:0:1::203/64 |
| | | 00:1b:21:d3:8f:3f | | 00:1b:21:c4:95:7a |
| | | | | |
| | | static routes | | static routes |
| | | 198.19.0.0/16 => 198.18.1.203 | | 198.19.0.0/16 => 198.19.0.201 |
| | | 198.18.0.0/16 => 198.18.0.201 | | 198.18.0.0/16 => 198.18.1.202 |
| | | 2001:2::/49 => 2001:2::201 | | 2001:2::/49 => 2001:2:0:1::202 |
| | |2001:2:0:8000::/49 => 2001:2:0:1::203| | 2001:2:0:8000::/49=>2001:2:0:8000::201 |
| | | | | |
|igb3: 198.19.0.201/24| | | | igb3: 198.19.0.203/24 |
|2001:2:0:8000::201/64| | | | 2001:2:0:8000::203/64 |
| 00:1b:21:d4:3f:2b | | | | 00:1b:21:c4:95:7b |
+---------------------+ +-------------------------------------+ +----------------------------------------+
|| ||
==================================<============================================
===== Devices configuration =====
Almost the same as on the forwarding performance lab but with fastforwarding disabled (not compatible with IPsec).
==== R2 (DUT) ====
Disable fastforwarding (not compliant with IPSec), configure IP address, routes and static IPSec.
/etc/rc.conf:
# IPv4 router
gateway_enable="YES"
ifconfig_igb2="198.18.0.202/24 -tso4 -tso6 -lro"
ifconfig_igb3="198.18.1.202/24 -tso4 -tso6 -lro"
# Route toward receiver will be pushed by OpenVPN client
static_routes="generator"
route_generator="-net 198.18.0.0/16 198.18.0.201"
static_arp_pairs="receiver generator"
static_arp_generator="198.18.0.201 00:1b:21:d4:3f:2a"
static_arp_receiver="198.18.1.203 00:1b:21:c4:95:7a"
# IPv6 router
ipv6_gateway_enable="YES"
ipv6_activate_all_interfaces="YES"
ifconfig_igb2_ipv6="inet6 2001:2::202 prefixlen 64"
ifconfig_igb3_ipv6="inet6 2001:2:0:1::202 prefixlen 64"
ipv6_static_routes="generator"
ipv6_route_generator="2001:2:: -prefixlen 49 2001:2::201"
static_ndp_pairs="receiver generator"
static_ndp_generator="2001:2::201 00:1b:21:d4:3f:2a"
static_ndp_receiver="2001:2:0:1::203 00:1b:21:c4:95:7a"
# Enabling OpenVPN
openvpn_enable="YES"
# Enabling AES-NI
kld_list="aesni"
/usr/local/etc/openvpn/openvpn.conf
client
dev tun
remote 198.18.1.203
ca ca.crt
cert client.crt
key client.key
ncp-disable
cipher AES-128-GCM
engine cryptodev
fast-io
==== R3 (reference) ====
Disable fastforwarding (not compliant with IPSec), configure IP address, routes and static IPSec:
# IPv4 router
gateway_enable="YES"
ifconfig_igb2="inet 198.18.1.203/24"
ifconfig_igb3="inet 198.19.0.203/24"
# routes toward generator will be installed by OpenVPN
static_routes="receiver"
route_receiver="-net 198.19.0.0/16 198.19.0.201"
static_arp_pairs="receiver generator"
static_arp_generator="198.18.1.202 00:1b:21:d3:8f:3f"
static_arp_receiver="198.19.0.201 00:1b:21:d4:3f:2b"
# IPv6 router
ipv6_gateway_enable="YES"
ipv6_activate_all_interfaces="YES"
ifconfig_igb2_ipv6="inet6 2001:2:0:1::203 prefixlen 64"
ifconfig_igb3_ipv6="inet6 2001:2:0:8000::203 prefixlen 64"
ipv6_static_routes="receiver"
ipv6_route_receiver="2001:2:0:8000:: -prefixlen 49 2001:2:0:8000::201"
static_ndp_pairs="receiver generator"
static_ndp_generator="2001:2:0:1::202 00:1b:21:d3:8f:3f"
static_ndp_receiver="2001:2:0:8000::201 00:1b:21:d4:3f:2b"
# Enabling OpenVPN
kld_list="aesni"
openvpn_enable="YES"
/usr/local/etc/openvpn/openvpn.conf:
dev tun
ca /usr/local/etc/easy-rsa/pki/ca.crt
cert /usr/local/etc/easy-rsa/pki/issued/server.crt
key /usr/local/etc/easy-rsa/pki/private/server.key
dh /usr/local/etc/easy-rsa/pki/dh.pem
server 172.16.254.0 255.255.255.0
server-ipv6 2001:db8:172::/64
ncp-disable
cipher AES-128-GCM
engine cryptodev
ifconfig-pool-persist ipp.txt
client-config-dir ccd
push "route 198.19.0.0 255.255.0.0"
push "route-ipv6 2001:2:0:8000::/49"
route 198.18.0.0 255.255.0.0
route-ipv6 2001:2::/49
keepalive 60 720
fast-io
===== Equilibrium throughput benchmark methodology =====
Once done, we start using a fast method for measuring the "equilibrium throughput" of the DUT.
From the packet generator/receiver a simple script that use netmap-pktgen will do the job:
[root@R1]/tmp# equilibrium -u -d 00:1b:21:d3:8f:3e -t igb2 -r igb3 -o 0.001
Benchmark tool using equilibrium throughput method
- Benchmark mode: Bandwitdh (bps) for VPN gateway
- UDP load = 500B, IPv4 packet size=528B, Ethernet frame size=542B
- Link rate = 1000 Mb/s
- Tolerance = 0.001
Iteration 1
- Offering load = 500 Mb/s
- Step = 250 Mb/s
- Measured forwarding rate = 500 Mb/s
Iteration 2
- Offering load = 750 Mb/s
- Step = 250 Mb/s
- Trend = increasing
- Measured forwarding rate = 539 Mb/s
Iteration 3
- Offering load = 625 Mb/s
- Step = 125 Mb/s
- Trend = decreasing
- Measured forwarding rate = 543 Mb/s
Iteration 4
- Offering load = 563 Mb/s
- Step = 62 Mb/s
- Trend = decreasing
- Measured forwarding rate = 545 Mb/s
Iteration 5
- Offering load = 532 Mb/s
- Step = 31 Mb/s
- Trend = decreasing
- Measured forwarding rate = 532 Mb/s
Iteration 6
- Offering load = 547 Mb/s
- Step = 15 Mb/s
- Trend = increasing
- Measured forwarding rate = 545 Mb/s
Iteration 7
- Offering load = 540 Mb/s
- Step = 7 Mb/s
- Trend = decreasing
- Measured forwarding rate = 540 Mb/s
Iteration 8
- Offering load = 543 Mb/s
- Step = 3 Mb/s
- Trend = increasing
- Measured forwarding rate = 543 Mb/s
Iteration 9
- Offering load = 544 Mb/s
- Step = 1 Mb/s
- Trend = increasing
- Measured forwarding rate = 543 Mb/s
Estimated Equilibrium Ethernet throughput= 543 Mb/s (maximum value seen: 545 Mb/s)
=> OpenVPN (userspace) reach about 454Mb/s which is about half IPSec performance (kernel space)
==== Encryption algorithms ====
{{documentation:examples:openvpn-ibm3550-fbsd11.0.png|Impact of OpenVPN encryption algorithms on 4 cores Xeon 2.13GHz with Intel 82580 NIC}}