documentation:examples:dropping_packets_at_high_rate
Differences
This shows you the differences between two versions of the page.
| Both sides previous revisionPrevious revisionNext revision | Previous revision | ||
| documentation:examples:dropping_packets_at_high_rate [2020/02/07 09:01] – [Performance impact] olivier | documentation:examples:dropping_packets_at_high_rate [2024/02/09 09:42] (current) – [NIC level configuration] olivier | ||
|---|---|---|---|
| Line 1: | Line 1: | ||
| ====== Dropping packets at high rate ====== | ====== Dropping packets at high rate ====== | ||
| - | ===== Goal ===== | + | ===== Objective |
| {{: | {{: | ||
| Line 10: | Line 10: | ||
| The configuration file of an IPFW in standard mode is this one: | The configuration file of an IPFW in standard mode is this one: | ||
| - | | + | |
| - | | + | |
| - | | + | |
| < | < | ||
| Line 28: | Line 28: | ||
| </ | </ | ||
| ==== NIC level configuration ==== | ==== NIC level configuration ==== | ||
| - | (A FreeBSD 13 / head) only feature. | ||
| - | Only [[https:// | + | Currently the [[https:// |
| The configuration file of an IPFW-at-NIC-level is this one: | The configuration file of an IPFW-at-NIC-level is this one: | ||
| - | | + | |
| - | | + | |
| - | | + | |
| - | | + | |
| < | < | ||
| Line 56: | Line 55: | ||
| </ | </ | ||
| - | ==== Performance | + | ==== Performance |
| Hardware: | Hardware: | ||
| Line 64: | Line 63: | ||
| * FreeBSD 13.0-CURRENT r357572 | * FreeBSD 13.0-CURRENT r357572 | ||
| - | Here is the rate of inet4 packets-per-second forwarded while dropping | + | Here is the rate of inet4 (legitimate) |
| < | < | ||
| x ipfw-standard | x ipfw-standard | ||
| Line 85: | Line 84: | ||
| (Student' | (Student' | ||
| </ | </ | ||
| + | |||
| + | On the 14Mpps of legitimate traffic, this generic (ie: supported by multi drivers) software firewall is still able to forward 12Mpps while droping 42Mpps of denied packets. | ||
| ===== Using Chelsio' | ===== Using Chelsio' | ||
| - | Chelsio NIC allows to configure hardware firewall with the use of cxgbetool(8). | + | Chelsio NIC allows to configure hardware firewall with the use of cxgbetool(8): The [[https:// |
| A Chelsio NIC is defined by its family name + id and the port id (if it's a 4 ports, port 0 to 4). | A Chelsio NIC is defined by its family name + id and the port id (if it's a 4 ports, port 0 to 4). | ||
| + | |||
| Example with only one Chelsio (t5nex0) with 2 ports (0 and 1) | Example with only one Chelsio (t5nex0) with 2 ports (0 and 1) | ||
| < | < | ||
| - | # grep t5nex / | + | # grep t.nex / |
| t5nex0: <Chelsio T580-LP-CR> | t5nex0: <Chelsio T580-LP-CR> | ||
| cxl0: <port 0> on t5nex0 | cxl0: <port 0> on t5nex0 | ||
| Line 98: | Line 101: | ||
| </ | </ | ||
| - | Here is the same rules translated | + | Translating |
| - | * cxl0 is then port 0 of t5nex0 | + | * Add a filter to drop packets incoming from Chelsio NIC 0 (t5nex0) |
| - | * Add a filter to drop packets incoming from port 0 matching source IP range 198.18.2.0/ | + | |
| < | < | ||
| # cxgbetool t5nex0 filter 0 iport 0 sip 198.18.2.0/ | # cxgbetool t5nex0 filter 0 iport 0 sip 198.18.2.0/ | ||
| Line 108: | Line 110: | ||
| </ | </ | ||
| - | Checking | + | To check the packet dropping |
| < | < | ||
| #!/bin/sh | #!/bin/sh | ||
| Line 115: | Line 117: | ||
| echo "Need Chelsio nexus name (examble: t5nex0)" | echo "Need Chelsio nexus name (examble: t5nex0)" | ||
| echo "List of Nexus detected:" | echo "List of Nexus detected:" | ||
| - | grep t5nex / | + | grep t.nex / |
| exit 1 | exit 1 | ||
| fi | fi | ||
| Line 141: | Line 143: | ||
| </ | </ | ||
| - | The NIC is filtering at a rate of 32Mpps: Where are the 10Mpps | + | The script report an hardware dropping |
| - | Need to tune the TCAM for a filtering usage: | + | Let's read the [[https:// |
| - | * Let only 2 occupied " | + | < |
| + | # TCAM has 8K cells; each region must start at a multiple of 128 cell. | ||
| + | # Each entry in these categories takes 4 cells each. nhash will use the | ||
| + | # TCAM iff there is room left (that is, the rest don't add up to 2048). | ||
| + | nroute = 32 | ||
| + | nclip = 32 | ||
| + | nfilter = 1008 | ||
| + | nserver = 512 | ||
| + | nhash = 524288 | ||
| + | </ | ||
| - | Here the default | + | And we can display |
| < | < | ||
| # sysctl -n dev.t5nex.0.misc.devlog | grep -w le | # sysctl -n dev.t5nex.0.misc.devlog | grep -w le | ||
| Line 152: | Line 163: | ||
| 16 | 16 | ||
| </ | </ | ||
| - | For that we need to download a [[https://svnweb.freebsd.org/ | + | |
| + | To improve the TCAM performance for a filtering usage, all unused " | ||
| + | |||
| + | For that we need to download a [[https://cgit.freebsd.org/ | ||
| < | < | ||
| - | # fetch -o / | + | # fetch -o / |
| # sed -i "" | # sed -i "" | ||
| - | # sed -i "" | + | # sed -i "" |
| # sed -i "" | # sed -i "" | ||
| # sed -i "" | # sed -i "" | ||
| # echo ' | # echo ' | ||
| # cxgbetool t5nex0 loadcfg / | # cxgbetool t5nex0 loadcfg / | ||
| + | # reboot | ||
| </ | </ | ||
| Line 168: | Line 183: | ||
| 12 | 12 | ||
| </ | </ | ||
| - | We confirm that server|hash are at 0, | + | We confirm that regions |
| Now the packet drop rate by the TCAM firewall match the generator' | Now the packet drop rate by the TCAM firewall match the generator' | ||
| Line 184: | Line 199: | ||
| </ | </ | ||
| - | And the firewall able to forward all packets without being too busy in the same time: | + | And the firewall |
| < | < | ||
| [root@firewall]~# | [root@firewall]~# | ||
| Line 196: | Line 211: | ||
| | | ||
| | | ||
| + | |||
| [root@firewall]~# | [root@firewall]~# | ||
| | | ||
documentation/examples/dropping_packets_at_high_rate.1581062507.txt.gz · Last modified: 2020/02/07 09:01 by olivier