documentation:examples:forwarding_performance_lab_of_a_pc_engines_apu2
Differences
This shows you the differences between two versions of the page.
— | documentation:examples:forwarding_performance_lab_of_a_pc_engines_apu2 [2019/03/25 12:37] (current) – created - external edit 127.0.0.1 | ||
---|---|---|---|
Line 1: | Line 1: | ||
+ | ====== Forwarding performance lab of a PC Engines APU 2 ====== | ||
+ | {{description> | ||
+ | ===== Hardware detail ===== | ||
+ | |||
+ | This lab will test a [[http:// | ||
+ | * Quad core [[https:// | ||
+ | * 3 with Intel i210AT Gigabit | ||
+ | * 4Gb of RAM | ||
+ | |||
+ | ===== Lab set-up ===== | ||
+ | |||
+ | For more information about full setup of this lab: [[documentation: | ||
+ | |||
+ | BSDRP release 1.92 based on FreeBSD 12-STABLE is used. | ||
+ | ==== Diagram ==== | ||
+ | |||
+ | < | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | | 2001: | ||
+ | | ||
+ | | ||
+ | | 198.18.0.203 | ||
+ | | 2001: | ||
+ | | ||
+ | | 198.19.0.203 | ||
+ | | 2001: | ||
+ | | ||
+ | |||
+ | </ | ||
+ | |||
+ | The generator **MUST** generate lot's of IP flows (multiple source/ | ||
+ | |||
+ | Multiple source/ | ||
+ | < | ||
+ | pkt-gen -U -i igb2 -f tx -n 80000000 -l 60 -d 198.19.10.1: | ||
+ | </ | ||
+ | |||
+ | And the same with IPv6 flows: | ||
+ | < | ||
+ | pkt-gen -N -f tx -i igb2 -n 1000000000 -l 60 -6 -d " | ||
+ | </ | ||
+ | |||
+ | |||
+ | Receiver will use these commands: | ||
+ | < | ||
+ | pkt-gen -i igb3 -f rx -w 4 | ||
+ | </ | ||
+ | ===== Basic configuration ===== | ||
+ | |||
+ | ==== Disabling Ethernet flow-control === | ||
+ | |||
+ | < | ||
+ | echo " | ||
+ | echo " | ||
+ | service sysctl restart | ||
+ | </ | ||
+ | |||
+ | ====Enabling Tx abdicate (iflib drivers) === | ||
+ | |||
+ | Second, if NIC drivers is iflib based you should enable TX abdicate (automatically done with BSDRP but not on a generic FreeBSD): | ||
+ | < | ||
+ | cat <<EOF >> / | ||
+ | # Enabling Tx abdicate | ||
+ | dev.igb.1.iflib.tx_abdicate=1 | ||
+ | dev.igb.2.iflib.tx_abdicate=1 | ||
+ | EOF | ||
+ | service sysctl restart | ||
+ | </ | ||
+ | |||
+ | ==== Disabling ICMP redirect === | ||
+ | |||
+ | ICMP redirect is enabled by default on a generic FreeBSD (it's disabled by default on BSDRP), and this feature disable the fast tryforward code path. | ||
+ | |||
+ | < | ||
+ | cat <<EOF >> / | ||
+ | # Enabling fastforwarding by disabling ICMP redirect | ||
+ | net.inet.ip.redirect=0 | ||
+ | net.inet6.ip6.redirect=0 | ||
+ | EOF | ||
+ | service sysctl restart | ||
+ | </ | ||
+ | |||
+ | |||
+ | |||
+ | ==== Static routes and ARP/NDP entries ==== | ||
+ | |||
+ | Configure static routes, configure IP addresses, static ARP and some entropy sources are excluded: | ||
+ | |||
+ | A router [[Documentation: | ||
+ | |||
+ | < | ||
+ | sysrc gateway_enable=" | ||
+ | sysrc static_routes=" | ||
+ | sysrc route_generator=" | ||
+ | sysrc route_receiver=" | ||
+ | sysrc static_arp_pairs=" | ||
+ | sysrc static_arp_generator=" | ||
+ | sysrc static_arp_receiver=" | ||
+ | sysrc ifconfig_igb1=" | ||
+ | sysrc ifconfig_igb2=" | ||
+ | |||
+ | sysrc ipv6_gateway_enable=" | ||
+ | sysrc ipv6_activate_all_interfaces=" | ||
+ | sysrc ipv6_static_routes=" | ||
+ | sysrc ipv6_route_generator=" | ||
+ | sysrc ipv6_route_receiver=" | ||
+ | sysrc ifconfig_igb1_ipv6=" | ||
+ | sysrc ifconfig_igb2_ipv6=" | ||
+ | sysrc static_ndp_pairs=" | ||
+ | sysrc static_ndp_generator=" | ||
+ | sysrc static_ndp_receiver=" | ||
+ | |||
+ | # Reduce number of source for entropy device harvests | ||
+ | harvest_mask=" | ||
+ | </ | ||
+ | |||
+ | ===== Default forwarding rate ===== | ||
+ | |||
+ | We start the first test by starting one packet generator at gigabit line-rate (1.488Mpps) and found: | ||
+ | * APU2 is not responsive during this test: NIC multiqueue correcly distribute load to all cores; | ||
+ | * About 794Kpps are accepted by the igb(4) Ethernet interface. | ||
+ | |||
+ | < | ||
+ | 110.375986 main_thread [2514] 794985 pps (795780 pkts 381974400 bps in 1001000 usec) 7.63 avg_batch 1001 min_space | ||
+ | 111.377986 main_thread [2514] 794692 pps (796281 pkts 382214880 bps in 1002000 usec) 7.58 avg_batch 1001 min_space | ||
+ | 112.378985 main_thread [2514] 794606 pps (795400 pkts 381792000 bps in 1000999 usec) 7.07 avg_batch 1005 min_space | ||
+ | 113.379987 main_thread [2514] 796200 pps (796998 pkts 382559040 bps in 1001002 usec) 6.78 avg_batch 999 min_space | ||
+ | 114.380986 main_thread [2514] 800465 pps (801265 pkts 384607200 bps in 1000999 usec) 6.88 avg_batch 1001 min_space | ||
+ | </ | ||
+ | ===== Firewalls impact ===== | ||
+ | |||
+ | This test will generate 2000 different flows by using 2000 different UDP destination ports. | ||
+ | |||
+ | pf and ipfw configurations used are detailed on the previous [[documentation: | ||
+ | |||
+ | {{bench.forwarding.and.firewalling.rate.on.pc.engines.apu2.png|forwarding and firewalling rate with a PC Engines APU running FreeBSD 10.3}} | ||
+ | |||
+ | |||
+ | |||
+ | |||
documentation/examples/forwarding_performance_lab_of_a_pc_engines_apu2.txt · Last modified: 2019/03/25 12:37 by 127.0.0.1