documentation:examples:dropping_packets_at_high_rate
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revision | ||
documentation:examples:dropping_packets_at_high_rate [2020/02/07 08:55] – [Using Chelsio's TCAM firewall] olivier | documentation:examples:dropping_packets_at_high_rate [2024/02/09 09:42] (current) – [NIC level configuration] olivier | ||
---|---|---|---|
Line 1: | Line 1: | ||
====== Dropping packets at high rate ====== | ====== Dropping packets at high rate ====== | ||
- | ===== Goal ===== | + | ===== Objective |
{{: | {{: | ||
Line 10: | Line 10: | ||
The configuration file of an IPFW in standard mode is this one: | The configuration file of an IPFW in standard mode is this one: | ||
- | | + | |
- | | + | |
- | | + | |
< | < | ||
Line 28: | Line 28: | ||
</ | </ | ||
==== NIC level configuration ==== | ==== NIC level configuration ==== | ||
- | (A FreeBSD 13 / head) only feature. | ||
- | Only [[https:// | + | Currently the [[https:// |
The configuration file of an IPFW-at-NIC-level is this one: | The configuration file of an IPFW-at-NIC-level is this one: | ||
- | | + | |
- | | + | |
- | | + | |
- | | + | |
< | < | ||
Line 56: | Line 55: | ||
</ | </ | ||
- | ==== Performance | + | ==== Performance |
- | Here is the rate of inet4 packets-per-second forwarded while filtering the 42Mpps of DDoS using the different configuration sets: | + | Hardware: |
+ | * Intel Xeon CPU E5-2697A v4 @ 2.60GHz (16 cores, 32 threads) | ||
+ | * Input NIC (filtering): | ||
+ | * Output NIC: Mellanox ConnectX-4 MCX416A-CCAT (QSFP28 100GBASE-SR4) | ||
+ | * FreeBSD 13.0-CURRENT r357572 | ||
+ | |||
+ | Here is the rate of inet4 (legitimate) | ||
< | < | ||
x ipfw-standard | x ipfw-standard | ||
Line 79: | Line 84: | ||
(Student' | (Student' | ||
</ | </ | ||
+ | |||
+ | On the 14Mpps of legitimate traffic, this generic (ie: supported by multi drivers) software firewall is still able to forward 12Mpps while droping 42Mpps of denied packets. | ||
===== Using Chelsio' | ===== Using Chelsio' | ||
- | Chelsio NIC allows to configure hardware firewall with the use of cxgbetool(8). | + | Chelsio NIC allows to configure hardware firewall with the use of cxgbetool(8): The [[https:// |
A Chelsio NIC is defined by its family name + id and the port id (if it's a 4 ports, port 0 to 4). | A Chelsio NIC is defined by its family name + id and the port id (if it's a 4 ports, port 0 to 4). | ||
+ | |||
Example with only one Chelsio (t5nex0) with 2 ports (0 and 1) | Example with only one Chelsio (t5nex0) with 2 ports (0 and 1) | ||
< | < | ||
- | # grep t5nex / | + | # grep t.nex / |
t5nex0: <Chelsio T580-LP-CR> | t5nex0: <Chelsio T580-LP-CR> | ||
cxl0: <port 0> on t5nex0 | cxl0: <port 0> on t5nex0 | ||
Line 92: | Line 101: | ||
</ | </ | ||
- | Here is the same rules translated | + | Translating |
- | * cxl0 is then port 0 of t5nex0 | + | * Add a filter to drop packets incoming from Chelsio NIC 0 (t5nex0) |
- | * Add a filter to drop packets incoming from port 0 matching source IP range 198.18.2.0/ | + | |
< | < | ||
# cxgbetool t5nex0 filter 0 iport 0 sip 198.18.2.0/ | # cxgbetool t5nex0 filter 0 iport 0 sip 198.18.2.0/ | ||
Line 102: | Line 110: | ||
</ | </ | ||
- | Checking | + | To check the packet dropping |
< | < | ||
#!/bin/sh | #!/bin/sh | ||
Line 109: | Line 117: | ||
echo "Need Chelsio nexus name (examble: t5nex0)" | echo "Need Chelsio nexus name (examble: t5nex0)" | ||
echo "List of Nexus detected:" | echo "List of Nexus detected:" | ||
- | grep t5nex / | + | grep t.nex / |
exit 1 | exit 1 | ||
fi | fi | ||
Line 135: | Line 143: | ||
</ | </ | ||
- | The NIC is filtering at a rate of 32Mpps: Where are the 10Mpps | + | The script report an hardware dropping |
- | Need to tune the TCAM for a filtering usage: | + | Let's read the [[https:// |
- | * Let only 2 occupied " | + | < |
+ | # TCAM has 8K cells; each region must start at a multiple of 128 cell. | ||
+ | # Each entry in these categories takes 4 cells each. nhash will use the | ||
+ | # TCAM iff there is room left (that is, the rest don't add up to 2048). | ||
+ | nroute = 32 | ||
+ | nclip = 32 | ||
+ | nfilter = 1008 | ||
+ | nserver = 512 | ||
+ | nhash = 524288 | ||
+ | </ | ||
- | Here the default | + | And we can display |
< | < | ||
# sysctl -n dev.t5nex.0.misc.devlog | grep -w le | # sysctl -n dev.t5nex.0.misc.devlog | grep -w le | ||
Line 146: | Line 163: | ||
16 | 16 | ||
</ | </ | ||
- | For that we need to download a [[https://svnweb.freebsd.org/ | + | |
+ | To improve the TCAM performance for a filtering usage, all unused " | ||
+ | |||
+ | For that we need to download a [[https://cgit.freebsd.org/ | ||
< | < | ||
- | # fetch -o / | + | # fetch -o / |
# sed -i "" | # sed -i "" | ||
- | # sed -i "" | + | # sed -i "" |
# sed -i "" | # sed -i "" | ||
# sed -i "" | # sed -i "" | ||
# echo ' | # echo ' | ||
# cxgbetool t5nex0 loadcfg / | # cxgbetool t5nex0 loadcfg / | ||
+ | # reboot | ||
</ | </ | ||
Line 162: | Line 183: | ||
12 | 12 | ||
</ | </ | ||
- | We confirm that server|hash are at 0, | + | We confirm that regions |
Now the packet drop rate by the TCAM firewall match the generator' | Now the packet drop rate by the TCAM firewall match the generator' | ||
Line 178: | Line 199: | ||
</ | </ | ||
- | And the firewall able to forward all packets without being too busy in the same time: | + | And the firewall |
< | < | ||
[root@firewall]~# | [root@firewall]~# | ||
Line 190: | Line 211: | ||
| | ||
| | ||
+ | |||
[root@firewall]~# | [root@firewall]~# | ||
| |
documentation/examples/dropping_packets_at_high_rate.txt · Last modified: 2024/02/09 09:42 by olivier