documentation:examples:forwarding_performance_lab_of_a_superserver_5018a-ftn4_with_10-gigabit_chelsio_t540-cr
no way to compare when less than two revisions
Differences
This shows you the differences between two versions of the page.
Next revision | |||
— | documentation:examples:forwarding_performance_lab_of_a_superserver_5018a-ftn4_with_10-gigabit_chelsio_t540-cr [2017/09/04 09:03] – external edit 127.0.0.1 | ||
---|---|---|---|
Line 1: | Line 1: | ||
+ | ====== Forwarding performance lab of a SuperServer 5018A-FTN4 with 10-Gigabit Chelsio T540-CR ====== | ||
+ | {{description> | ||
+ | ===== Hardware detail ===== | ||
+ | |||
+ | This lab will test a [[http:// | ||
+ | * Intel Rangeley: | ||
+ | * 8Gb of RAM | ||
+ | * Quad port Chelsio 10-Gigabit T540-CR and OPT SFP (SFP-10G-LR) | ||
+ | |||
+ | ===== Lab set-up ===== | ||
+ | |||
+ | For more information about full setup of this lab: [[documentation: | ||
+ | |||
+ | BSDRP used for this bench is based on FreeBSD 11.1. | ||
+ | |||
+ | ==== Diagram ==== | ||
+ | |||
+ | < | ||
+ | +------------------------------------------+ +-------+ +------------------------------+ | ||
+ | | Device under test | |Juniper| | Packet generator & receiver | ||
+ | | | | QFX | | | | ||
+ | | cxl0: 198.18.0.8/ | ||
+ | | 2001: | ||
+ | | (00: | ||
+ | | | | | | | | ||
+ | | cxl1: 198.19.0.8/ | ||
+ | | 2001: | ||
+ | | (00: | ||
+ | | | | ||
+ | | static routes | ||
+ | | 192.18.0.0/ | ||
+ | | 192.19.0.0/ | ||
+ | | 2001: | ||
+ | | 2001: | ||
+ | | | | ||
+ | | static arp and ndp | | / | ||
+ | | 198.18.0.10 | ||
+ | | 2001: | ||
+ | | | | ||
+ | | 198.19.0.10 | ||
+ | | 2001: | ||
+ | +------------------------------------------+ | ||
+ | </ | ||
+ | |||
+ | The generator **MUST** generate lot's of smallest IP flows (multiple source/ | ||
+ | |||
+ | Here is an example for generating 2000 IPv4 flows (100 destination IP addresses * 20 source IP addresses) with a Chelsio NIC: | ||
+ | < | ||
+ | pkt-gen -i vcxl0 -f tx -n 1000000000 -l 60 -d 198.19.10.1: | ||
+ | </ | ||
+ | |||
+ | And the same with IPv6 flows (minimum frame size of 62 here): | ||
+ | < | ||
+ | pkt-gen -f tx -i vcxl0 -n 1000000000 -l 62 -6 -d " | ||
+ | </ | ||
+ | |||
+ | <note warning> | ||
+ | This version of pkt-gen is improved with: IPv6 support, software checksum and optional unit normalization. [[https:// | ||
+ | </ | ||
+ | Receiver will use this command: | ||
+ | < | ||
+ | pkt-gen -i vcxl1 -f rx -w 4 | ||
+ | </ | ||
+ | ===== Basic configuration ===== | ||
+ | |||
+ | ==== Disabling Ethernet flow-control === | ||
+ | |||
+ | First, disable Ethernet flow-control on both servers. Chelsio T540 are configured like this: | ||
+ | < | ||
+ | echo " | ||
+ | echo " | ||
+ | service sysctl reload | ||
+ | </ | ||
+ | |||
+ | === Disabling LRO and TSO === | ||
+ | |||
+ | A router [[Documentation: | ||
+ | ==== IP Configuration on DUT ==== | ||
+ | |||
+ | Configure static routes: | ||
+ | < | ||
+ | sysrc static_routes=" | ||
+ | sysrc route_generator=" | ||
+ | sysrc route_receiver=" | ||
+ | sysrc ifconfig_cxl0=" | ||
+ | sysrc ifconfig_cxl1=" | ||
+ | sysrc static_arp_pairs=" | ||
+ | sysrc static_arp_HPncxl0=" | ||
+ | sysrc static_arp_HPncxl1=" | ||
+ | </ | ||
+ | ===== Default forwarding rate ===== | ||
+ | |||
+ | Behaviour in front of a multi-flow traffic generator at line-rate 14.8Mpps (thanks Chelsio!), pkt-gen on the receiver reports: | ||
+ | |||
+ | < | ||
+ | 705.046821 main_thread [2277] 3173758 pps (3176351 pkts 1524648480 bps in 1000817 usec) 19.66 avg_batch 0 min_space | ||
+ | 706.048593 main_thread [2277] 3179960 pps (3185595 pkts 1529085600 bps in 1001772 usec) 19.65 avg_batch 823 min_space | ||
+ | 707.050593 main_thread [2277] 3180102 pps (3186459 pkts 1529500320 bps in 1001999 usec) 19.63 avg_batch 1000 min_space | ||
+ | 708.052592 main_thread [2277] 3182707 pps (3189072 pkts 1530754560 bps in 1002000 usec) 19.69 avg_batch 768 min_space | ||
+ | 709.054593 main_thread [2277] 3166689 pps (3173022 pkts 1523050560 bps in 1002000 usec) 19.59 avg_batch 1001 min_space | ||
+ | 710.055593 main_thread [2277] 3126576 pps (3129706 pkts 1502258880 bps in 1001001 usec) 19.33 avg_batch 999 min_space | ||
+ | 711.057592 main_thread [2277] 3138706 pps (3144980 pkts 1509590400 bps in 1001999 usec) 19.41 avg_batch 768 min_space | ||
+ | 712.058592 main_thread [2277] 3147761 pps (3150909 pkts 1512436320 bps in 1001000 usec) 19.47 avg_batch 768 min_space | ||
+ | 713.059592 main_thread [2277] 3156042 pps (3159195 pkts 1516413600 bps in 1000999 usec) 19.51 avg_batch 768 min_space | ||
+ | 714.060592 main_thread [2277] 3174469 pps (3177647 pkts 1525270560 bps in 1001001 usec) 19.58 avg_batch 768 min_space | ||
+ | 715.061593 main_thread [2277] 3172119 pps (3175291 pkts 1524139680 bps in 1001000 usec) 19.60 avg_batch 1001 min_space | ||
+ | </ | ||
+ | |||
+ | About 3.1M packets-per-seconds forwarded. | ||
+ | |||
+ | ===== Firewall impact on forwarding rate ===== | ||
+ | |||
+ | {{: | ||
documentation/examples/forwarding_performance_lab_of_a_superserver_5018a-ftn4_with_10-gigabit_chelsio_t540-cr.txt · Last modified: 2019/12/16 20:48 by olivier