Table of Contents
OpenVPN performance lab of an IBM System x3550 M3 with Intel 82580
OPenVPN performance lab of a quad cores Xeon 2.13GHz and quad-port gigabit Intel 82580
Hardware detail
This lab will test an IBM System x3550 M3 with quad cores (Intel Xeon L5630 2.13GHz, hyper-threading disabled) and a quad NIC 82580 connected to the PCI-Express Bus.
This CPU includes AES-NI: AES-CBC,AES-XTS,AES-GCM,AES-ICM.
Method used
The benchmarking method used here is detailed in Setting up a VPN (IPSec, GRE, etc...) performance benchmark lab and OpenVPN example lab.
Diagram
+---------------------+ +-------------------------------------+ +----------------------------------------+ | R1 | | IBM x3550 M3 | | R3 | | Packet generator | | Device under Test | | IPSec endpoint | | and receiver | | | | (AES-NI) | | | | | | | |igb2: 198.18.0.201/24|=>=| igb2: 198.18.0.202/24 | | | | 2001:2::201/64| | 2001:2::202/64 | | | | 00:1b:21:d4:3f:2a| | 00:1b:21:d3:8f:3e | | | | | | | | | | | | igb3: 198.18.1.202/24 |==>=| igb2: 198.18.1.203/24 | | | | 2001:2:0:1::202/64 | | 2001:2:0:1::203/64 | | | | 00:1b:21:d3:8f:3f | | 00:1b:21:c4:95:7a | | | | | | | | | | static routes | | static routes | | | | 198.19.0.0/16 => 198.18.1.203 | | 198.19.0.0/16 => 198.19.0.201 | | | | 198.18.0.0/16 => 198.18.0.201 | | 198.18.0.0/16 => 198.18.1.202 | | | | 2001:2::/49 => 2001:2::201 | | 2001:2::/49 => 2001:2:0:1::202 | | | |2001:2:0:8000::/49 => 2001:2:0:1::203| | 2001:2:0:8000::/49=>2001:2:0:8000::201 | | | | | | | |igb3: 198.19.0.201/24| | | | igb3: 198.19.0.203/24 | |2001:2:0:8000::201/64| | | | 2001:2:0:8000::203/64 | | 00:1b:21:d4:3f:2b | | | | 00:1b:21:c4:95:7b | +---------------------+ +-------------------------------------+ +----------------------------------------+ || || ==================================<============================================
Devices configuration
Almost the same as on the forwarding performance lab but with fastforwarding disabled (not compatible with IPsec).
R2 (DUT)
Disable fastforwarding (not compliant with IPSec), configure IP address, routes and static IPSec.
/etc/rc.conf:
# IPv4 router gateway_enable="YES" ifconfig_igb2="198.18.0.202/24 -tso4 -tso6 -lro" ifconfig_igb3="198.18.1.202/24 -tso4 -tso6 -lro" # Route toward receiver will be pushed by OpenVPN client static_routes="generator" route_generator="-net 198.18.0.0/16 198.18.0.201" static_arp_pairs="receiver generator" static_arp_generator="198.18.0.201 00:1b:21:d4:3f:2a" static_arp_receiver="198.18.1.203 00:1b:21:c4:95:7a" # IPv6 router ipv6_gateway_enable="YES" ipv6_activate_all_interfaces="YES" ifconfig_igb2_ipv6="inet6 2001:2::202 prefixlen 64" ifconfig_igb3_ipv6="inet6 2001:2:0:1::202 prefixlen 64" ipv6_static_routes="generator" ipv6_route_generator="2001:2:: -prefixlen 49 2001:2::201" static_ndp_pairs="receiver generator" static_ndp_generator="2001:2::201 00:1b:21:d4:3f:2a" static_ndp_receiver="2001:2:0:1::203 00:1b:21:c4:95:7a" # Enabling OpenVPN openvpn_enable="YES" # Enabling AES-NI kld_list="aesni"
/usr/local/etc/openvpn/openvpn.conf
client dev tun remote 198.18.1.203 ca ca.crt cert client.crt key client.key ncp-disable cipher AES-128-GCM engine cryptodev fast-io
R3 (reference)
Disable fastforwarding (not compliant with IPSec), configure IP address, routes and static IPSec:
# IPv4 router gateway_enable="YES" ifconfig_igb2="inet 198.18.1.203/24" ifconfig_igb3="inet 198.19.0.203/24" # routes toward generator will be installed by OpenVPN static_routes="receiver" route_receiver="-net 198.19.0.0/16 198.19.0.201" static_arp_pairs="receiver generator" static_arp_generator="198.18.1.202 00:1b:21:d3:8f:3f" static_arp_receiver="198.19.0.201 00:1b:21:d4:3f:2b" # IPv6 router ipv6_gateway_enable="YES" ipv6_activate_all_interfaces="YES" ifconfig_igb2_ipv6="inet6 2001:2:0:1::203 prefixlen 64" ifconfig_igb3_ipv6="inet6 2001:2:0:8000::203 prefixlen 64" ipv6_static_routes="receiver" ipv6_route_receiver="2001:2:0:8000:: -prefixlen 49 2001:2:0:8000::201" static_ndp_pairs="receiver generator" static_ndp_generator="2001:2:0:1::202 00:1b:21:d3:8f:3f" static_ndp_receiver="2001:2:0:8000::201 00:1b:21:d4:3f:2b" # Enabling OpenVPN kld_list="aesni" openvpn_enable="YES"
/usr/local/etc/openvpn/openvpn.conf:
dev tun ca /usr/local/etc/easy-rsa/pki/ca.crt cert /usr/local/etc/easy-rsa/pki/issued/server.crt key /usr/local/etc/easy-rsa/pki/private/server.key dh /usr/local/etc/easy-rsa/pki/dh.pem server 172.16.254.0 255.255.255.0 server-ipv6 2001:db8:172::/64 ncp-disable cipher AES-128-GCM engine cryptodev ifconfig-pool-persist ipp.txt client-config-dir ccd push "route 198.19.0.0 255.255.0.0" push "route-ipv6 2001:2:0:8000::/49" route 198.18.0.0 255.255.0.0 route-ipv6 2001:2::/49 keepalive 60 720 fast-io
Equilibrium throughput benchmark methodology
Once done, we start using a fast method for measuring the “equilibrium throughput” of the DUT.
From the packet generator/receiver a simple script that use netmap-pktgen will do the job:
[root@R1]/tmp# equilibrium -u -d 00:1b:21:d3:8f:3e -t igb2 -r igb3 -o 0.001 Benchmark tool using equilibrium throughput method - Benchmark mode: Bandwitdh (bps) for VPN gateway - UDP load = 500B, IPv4 packet size=528B, Ethernet frame size=542B - Link rate = 1000 Mb/s - Tolerance = 0.001 Iteration 1 - Offering load = 500 Mb/s - Step = 250 Mb/s - Measured forwarding rate = 500 Mb/s Iteration 2 - Offering load = 750 Mb/s - Step = 250 Mb/s - Trend = increasing - Measured forwarding rate = 539 Mb/s Iteration 3 - Offering load = 625 Mb/s - Step = 125 Mb/s - Trend = decreasing - Measured forwarding rate = 543 Mb/s Iteration 4 - Offering load = 563 Mb/s - Step = 62 Mb/s - Trend = decreasing - Measured forwarding rate = 545 Mb/s Iteration 5 - Offering load = 532 Mb/s - Step = 31 Mb/s - Trend = decreasing - Measured forwarding rate = 532 Mb/s Iteration 6 - Offering load = 547 Mb/s - Step = 15 Mb/s - Trend = increasing - Measured forwarding rate = 545 Mb/s Iteration 7 - Offering load = 540 Mb/s - Step = 7 Mb/s - Trend = decreasing - Measured forwarding rate = 540 Mb/s Iteration 8 - Offering load = 543 Mb/s - Step = 3 Mb/s - Trend = increasing - Measured forwarding rate = 543 Mb/s Iteration 9 - Offering load = 544 Mb/s - Step = 1 Mb/s - Trend = increasing - Measured forwarding rate = 543 Mb/s Estimated Equilibrium Ethernet throughput= 543 Mb/s (maximum value seen: 545 Mb/s)
⇒ OpenVPN (userspace) reach about 454Mb/s which is about half IPSec performance (kernel space)