User Tools

Site Tools


documentation:examples:ipsec_performance_of_a_superserver_5018a-ftn4

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

documentation:examples:ipsec_performance_of_a_superserver_5018a-ftn4 [2017/10/23 12:17] (current)
Line 1: Line 1:
 +====== IPSec performance lab of SuperServer 5018A-FTN4 ======
 +{{description>​IPSec performance lab of a 8 cores Atom}}
 +
 +===== Hardware detail =====
 +
 +This lab will test a [[http://​www.supermicro.com/​products/​system/​1U/​5018/​SYS-5018A-FTN4.cfm|SuperMicro]] [[SuperServer 5018A-FTN4]]:​
 +   * Intel Rangeley: [[http://​ark.intel.com/​products/​77988/​Intel-Atom-Processor-C2758-4M-Cache-2_40-GHz|Atom C2758 (8 cores) at 2.4GHz]]
 +   * 8Gb of RAM
 +   * Quad port Chelsio 10-Gigabit T540-CR and OPT SFP (SFP-10G-LR)
 +
 +This CPU includes AES-NI: AES-CBC,​AES-XTS,​AES-GCM,​AES-ICM.
 +
 +===== Method used =====
 +
 +The benchmarking method used here is detailed in [[documentation:​examples:​Setting up a VPN (IPSec, GRE, etc...) performance benchmark lab]].
 +==== Diagram ====
 +
 +<​code>​
 ++--------------------+ ​  ​+-------------------------------------+ ​  ​+------------------------------------+
 +|         ​r630 ​      ​| ​  ​| ​         Atom C2758-Chelsio ​        ​| ​  ​| ​                 HP                |
 +|  Packet generator ​ |   ​| ​          ​Device under Test         ​| ​  ​| ​          IPSec endpoint ​          |
 +|     and receiver ​  ​| ​  ​| ​                                    ​| ​  ​| ​             (AES-NI) ​             |
 +|                    |   ​| ​                                    ​| ​  ​| ​                                   |
 +|vcxl0: 198.18.0.2/​24|=>​=| cxl0: 198.18.0.208/​24 ​              ​| ​  ​| ​                                   |
 +|       ​2001:​2::​2/​64 |   | 2001:​2::​208/​64 ​                     |   ​| ​                                   |
 +|  00:​07:​43:​2f:​fe:​b2 |   | 00:​07:​43:​2e:​e5:​90 ​                  ​| ​  ​| ​                                   |
 +|                    |   ​| ​                                    ​| ​  ​| ​                                   |
 +|                    |   ​| ​              cxl1: 198.18.1.208/​24 |=>=| cxl0: 198.18.1.210/​24 ​             |
 +|                    |   ​| ​                 2001:​2:​0:​1::​208/​64 |   ​| ​   2001:​2:​0:​1::​210/​64 ​             |
 +|                    |   ​| ​                  ​00:​07:​43:​2e:​e5:​98 |   ​| ​    ​00:​07:​43:​2e:​e4:​70 ​             |
 +|                    |   ​| ​                                    ​| ​  ​| ​                                   |
 +|                    |   ​| ​             static routes ​         |   ​| ​           static routes ​          |
 +|                    |   ​| ​    ​198.19.0.0/​16 => 198.18.1.210 ​  ​| ​  ​| ​   198.19.0.0/​16 => 198.19.0.2 ​    |
 +|                    |   ​| ​    ​198.18.0.0/​16 => 198.18.0.2 ​    ​| ​  ​| ​   198.18.0.0/​16 => 198.18.1.208 ​  |
 +|                    |   ​| ​      ​2001:​2::/​49 => 2001:​2::​2 ​     |   ​| ​     2001:2::/49 => 2001:​2:​0:​1::​208|
 +|                    |   ​|2001:​2:​0:​8000::/​49 => 2001:​2:​0:​1::​210| ​  ​|2001:​2:​0:​8000::/​49=>​2001:​2:​0:​8000::​2|
 +|                    |   ​| ​                                    ​| ​  ​| ​                                   |
 +|vcxl1: 198.19.0.2/​24| ​  ​| ​                                    ​| ​  ​| ​       cxl1: 198.19.0.210/​24 ​      |
 +| 2001:​2:​0:​8000::​2/​64| ​  ​| ​                                    ​| ​  ​| ​       2001:​2:​0:​8000::​210/​64 ​      |
 +| 00:​07:​43:​2f:​fe:​ba ​ |   ​| ​                                    ​| ​  ​| ​        ​00:​07:​43:​2e:​e4:​78 ​         |
 ++--------------------+ ​  ​+-------------------------------------+ ​  ​+------------------------------------+
 +          ||                                                                          ||
 +          ==================================<​===========================================
 +</​code>​
 +
 +===== Devices configuration =====
 +
 +Almost the same as on the forwarding performance lab.
 +
 +==== DUT ====
 +
 +Configure IP address, routes and static IPSec.
 +
 +/​etc/​rc.conf:​
 +<​code>​
 +# IPv4 router
 +gateway_enable="​YES"​
 +static_routes="​generator receiver"​
 +route_generator="​-net 198.18.0.0/​16 198.18.0.2"​
 +route_receiver="​-net 198.19.0.0/​16 198.18.1.210"​
 +ifconfig_cxl0="​inet 198.18.0.208/​24 -tso4 -tso6 -lro"
 +ifconfig_cxl1="​inet 198.18.1.208/​24 -tso4 -tso6 -lro"
 +static_arp_pairs="​generator receiver"​
 +static_arp_generator="​198.18.0.2 00:​07:​43:​2f:​fe:​b2"​
 +static_arp_receiver="​198.18.1.210 00:​07:​43:​2e:​e4:​70"​
 +
 +# IPv6 router
 +ipv6_gateway_enable="​YES"​
 +ipv6_activate_all_interfaces="​YES"​
 +ipv6_static_routes="​generator receiver"​
 +ipv6_route_generator="​2001:​2::​ -prefixlen 49 2001:​2::​2"​
 +ipv6_route_receiver="​2001:​2:​0:​8000::​ -prefixlen 49 2001:​2:​0:​1::​210"​
 +ifconfig_cxl0_ipv6="​inet6 2001:2::208 prefixlen 64"
 +ifconfig_cxl1_ipv6="​inet6 2001:​2:​0:​1::​208 prefixlen 64"
 +static_ndp_pairs="​generator receiver"​
 +static_ndp_generator="​2001:​2::​2 00:​07:​43:​2f:​fe:​b2"​
 +static_ndp_receiver="​2001:​2:​0:​1::​210 00:​07:​43:​2e:​e4:​70"​
 +
 +# Enabling IPSec
 +kld_list="​aesni"​
 +ipsec_enable="​YES"​
 +</​code>​
 +
 +/​etc/​ipsec.conf
 +
 +<​code>​
 +flush;
 +spdflush;
 +spdadd 198.18.0.0/​16 198.19.0.0/​16 any -P out ipsec esp/​tunnel/​198.18.1.208-198.18.1.210/​require;​
 +spdadd 198.19.0.0/​16 198.18.0.0/​16 any -P in ipsec esp/​tunnel/​198.18.1.210-198.18.1.208/​require;​
 +add 198.18.1.208 198.18.1.210 esp 0x1000 -E aes-gcm-16 "​12345678901234567890";​
 +add 198.18.1.210 198.18.1.208 esp 0x1001 -E aes-gcm-16 "​12345678901234567890";​
 +spdadd 2001:2::/49 2001:​2:​0:​8000::/​49 any -P out ipsec esp/​tunnel/​2001:​2:​0:​1::​208-2001:​2:​0:​1::​210/​require;​
 +spdadd 2001:​2:​0:​8000::/​49 2001:2::/49 any -P in ipsec esp/​tunnel/​2001:​2:​0:​1::​210-2001:​2:​0:​1::​208/​require;​
 +add 2001:​2:​0:​1::​208 2001:​2:​0:​1::​210 esp 0x1002 -E aes-gcm-16 "​12345678901234567890";​
 +add 2001:​2:​0:​1::​210 2001:​2:​0:​1::​208 esp 0x1003 -E aes-gcm-16 "​12345678901234567890";​
 +</​code>​
 +
 +==== Reference Endpoint ====
 +
 +Configure IP address, routes and static IPSec:
 +<​code>​
 +# IPv4 router
 +gateway_enable="​YES"​
 +ifconfig_cxl0="​inet 198.18.1.210/​24 -tso4 -tso6 -lro -vlanhwtso"​
 +ifconfig_cxl1="​inet 198.19.0.210/​24 -tso4 -tso6 -lro -vlanhwtso"​
 +static_routes="​generator receiver"​
 +route_generator="​-net 198.18.0.0/​16 198.18.1.208"​
 +route_receiver="​-net 198.19.0.0/​16 198.19.0.2"​
 +static_arp_pairs="​generator receiver"​
 +static_arp_generator="​198.18.1.208 00:​07:​43:​2e:​e5:​98"​
 +static_arp_receiver="​198.19.0.2 00:​07:​43:​2f:​fe:​ba"​
 +
 +# IPv6 router
 +ipv6_gateway_enable="​YES"​
 +ipv6_activate_all_interfaces="​YES"​
 +ifconfig_cxl0_ipv6="​inet6 2001:​2:​0:​1::​210 prefixlen 64"
 +ifconfig_cxl1_ipv6="​inet6 2001:​2:​0:​8000::​210 prefixlen 64"
 +ipv6_static_routes="​generator receiver"​
 +ipv6_route_generator="​2001:​2::​ -prefixlen 49 2001:​1::​208"​
 +ipv6_route_receiver="​2001:​2:​0:​8000::​ -prefixlen 49 2001:​2:​0:​8000::​2"​
 +static_ndp_pairs="​generator receiver"​
 +static_ndp_generator="​2001:​2:​0:​1::​208 00:​07:​43:​2e:​e5:​98"​
 +static_ndp_receiver="​2001:​2:​0:​8000::​2 00:​07:​43:​2f:​fe:​ba"​
 +
 +# Enabling IPSec
 +kld_list="​aesni"​
 +ipsec_enable="​YES"​
 +</​code>​
 +
 +/​etc/​ipsec.conf:​
 +
 +<​code>​
 +flush;
 +spdflush;
 +spdadd 198.18.0.0/​16 198.19.0.0/​16 any -P in ipsec esp/​tunnel/​198.18.1.208-198.18.1.210/​require;​
 +spdadd 198.19.0.0/​16 198.18.0.0/​16 any -P out ipsec esp/​tunnel/​198.18.1.210-198.18.1.208/​require;​
 +add 198.18.1.208 198.18.1.210 esp 0x1000 -E aes-gcm-16 "​12345678901234567890";​
 +add 198.18.1.210 198.18.1.208 esp 0x1001 -E aes-gcm-16 "​12345678901234567890";​
 +spdadd 2001:2::/49 2001:​2:​0:​8000::/​49 any -P in ipsec esp/​tunnel/​2001:​2:​0:​1::​208-2001:​2:​0:​1::​210/​require;​
 +spdadd 2001:​2:​0:​8000::/​49 2001:2::/49 any -P out ipsec esp/​tunnel/​2001:​2:​0:​1::​210-2001:​2:​0:​1::​208/​require;​
 +add 2001:​2:​0:​1::​208 2001:​2:​0:​1::​210 esp 0x1002 -E aes-gcm-16 "​12345678901234567890";​
 +add 2001:​2:​0:​1::​210 2001:​2:​0:​1::​208 esp 0x1003 -E aes-gcm-16 "​12345678901234567890";​
 +</​code>​
 +
 +===== IPSec benchmark "​Equilibrium throughput"​ method =====
 +
 +Once done, we start using a fast method for measuring the "IPsec equilibrium throughput"​ of the DUT.
 +
 +From the packet generator/​receiver a simple script that use netmap-pktgen will do the job:
 +
 +<​code>​
 +[root@pkt-gen]~#​ equilibrium -4 -d 00:​07:​43:​2e:​e5:​90 -t vcxl0 -r vcxl1 -l 10000
 +Benchmark tool using equilibrium throughput method
 +- Benchmark mode: Bandwitdh (bps) for VPN gateway
 +- UDP load = 500B, IPv4 packet size=528B, Ethernet frame size=542B
 +- Link rate = 10000 Mb/s
 +- Tolerance = 0.01
 +Iteration 1
 +  - Offering load = 5000 Mb/s
 +  - Step = 2500 Mb/s
 +  - Measured forwarding rate = 1383 Mb/s
 +  - Forwared rate too low, forcing OLOAD=FWRATE and STEP=FWRATE/​2
 +Iteration 2
 +  - Offering load = 1383 Mb/s
 +  - Step = 691 Mb/s
 +  - Trend = decreasing
 +  - Measured forwarding rate = 1384 Mb/s
 +  - forwarding rate greater than offering load! (forcing FWRATE=OLOAD)
 +Iteration 3
 +  - Offering load = 1728 Mb/s
 +  - Step = 345 Mb/s
 +  - Trend = increasing
 +  - Measured forwarding rate = 1383 Mb/s
 +Iteration 4
 +  - Offering load = 1556 Mb/s
 +  - Step = 172 Mb/s
 +  - Trend = decreasing
 +  - Measured forwarding rate = 1386 Mb/s
 +Iteration 5
 +  - Offering load = 1470 Mb/s
 +  - Step = 86 Mb/s
 +  - Trend = decreasing
 +  - Measured forwarding rate = 1384 Mb/s
 +Iteration 6
 +  - Offering load = 1427 Mb/s
 +  - Step = 43 Mb/s
 +  - Trend = decreasing
 +  - Measured forwarding rate = 1385 Mb/s
 +Iteration 7
 +  - Offering load = 1406 Mb/s
 +  - Step = 21 Mb/s
 +  - Trend = decreasing
 +  - Measured forwarding rate = 1384 Mb/s
 +Estimated Equilibrium Ethernet throughput= 1384 Mb/s (maximum value seen: 1386 Mb/s)
 +</​code>​
 +
 +=> We reach about 1.386Gb/s of encrypted traffic (notice the equilibrium script bug at step 2 that could stop here).
 +
 +==== Encryption algorithms ====
 +
 +TO DO:
 +
 +<​code>​
 +~/​netbenches/​Atom_C2758_8Cores-Chelsio_T540-CR % ../​scripts/​bench-lab.sh -f bench-lab-3nodes.config -c ipsec/​configs/​ -p ../​pktgen.configs/​dualstack-vpn/​ -d ipsec/​results/​fbsd11.1/​
 +</​code>​
 +
 +
 +
 +
  
documentation/examples/ipsec_performance_of_a_superserver_5018a-ftn4.txt · Last modified: 2017/10/23 12:17 (external edit)