Both sides previous revisionPrevious revisionNext revision | Previous revisionNext revisionBoth sides next revision |
documentation:examples:dropping_packets_at_high_rate [2020/02/07 15:47] – [NIC level configuration] olivier | documentation:examples:dropping_packets_at_high_rate [2020/02/07 16:36] – [Performance impact] olivier |
---|
====== Dropping packets at high rate ====== | ====== Dropping packets at high rate ====== |
| |
===== Goal ===== | ===== Objective ===== |
| |
{{:documentation:examples:labs.examples.ddos.png}} | {{:documentation:examples:labs.examples.ddos.png}} |
| |
The configuration file of an IPFW in standard mode is this one: | The configuration file of an IPFW in standard mode is this one: |
* First rule is a blacklist of denied IP addresses | - First rule is to deny a blacklist table (IP addresses) |
* Second rule is to allow all the rest | - Second rule is to allow all the rest |
* Disabling the outgoing [[https://www.freebsd.org/cgi/man.cgi?query=pfil&apropos=0&sektion=0&manpath=FreeBSD+12.1-RELEASE+and+Ports&arch=default&format=html|pfil(9)]] hook because we don't need to filter outgoing traffic | - Disable the outgoing [[https://www.freebsd.org/cgi/man.cgi?query=pfil&apropos=0&sektion=0&manpath=FreeBSD+12.1-RELEASE+and+Ports&arch=default&format=html|pfil(9)]] hook at IP level because we don't need to filter outgoing traffic |
| |
<code> | <code> |
(A FreeBSD 13 / head) only feature. | (A FreeBSD 13 / head) only feature. |
| |
Only [[https://svnweb.freebsd.org/changeset/base/346632|iflib]], [[https://svnweb.freebsd.org/changeset/base/356613|vtnet]], [[https://svnweb.freebsd.org/changeset/base/346247|Mellanox]] and [[https://svnweb.freebsd.org/changeset/base/357483|Chelsio]] drivers are "[[https://svnweb.freebsd.org/changeset/base/343631|Pfil Memory Pointer Hooks]]" compliant. | Currently the [[https://svnweb.freebsd.org/changeset/base/343631|Pfil Memory Pointer Hooks]] feature is supported by [[https://svnweb.freebsd.org/changeset/base/346632|iflib]], [[https://svnweb.freebsd.org/changeset/base/356613|vtnet]], [[https://svnweb.freebsd.org/changeset/base/346247|Mellanox]] and [[https://svnweb.freebsd.org/changeset/base/357483|Chelsio]] drivers. |
| |
The configuration file of an IPFW-at-NIC-level is this one: | The configuration file of an IPFW-at-NIC-level is this one: |
(Student's t, pooled s = 29598.4) | (Student's t, pooled s = 29598.4) |
</code> | </code> |
| |
| On the 14Mpps of legitimate traffic, this generic (ie: supported by multi drivers) software firewall is still able to forward 12Mpps while droping 42Mpps of denied packets. |
===== Using Chelsio's TCAM firewall ===== | ===== Using Chelsio's TCAM firewall ===== |
| |
To improve the TCAM performance for a filtering usage, all unused "regions" will be disabled to kept only the route and filter. | To improve the TCAM performance for a filtering usage, all unused "regions" will be disabled to kept only the route and filter. |
| |
| For that we need to download a [[https://svnweb.freebsd.org/base/head/sys/dev/cxgbe/firmware/t5fw_cfg_hashfilter.txt?view=co|default TCAM firmware configuration file for our T5 NIC]] to modify its parameters then load the modified configuration into the NIC flash and instruct the NIC to use the file from its flash. |
For that we need to download a [[https://svnweb.freebsd.org/base/head/sys/dev/cxgbe/firmware/t5fw_cfg_hashfilter.txt?view=co|default TCAM firmware configuration file for our T5 NIC]] to modify its parameters then load the modified configuration into the NIC flash. | |
<code> | <code> |
# fetch -o /etc/t5fw.txt 'https://svnweb.freebsd.org/base/head/sys/dev/cxgbe/firmware/t5fw_cfg_hashfilter.txt?view=co' | # fetch -o /etc/t5fw.txt 'https://svnweb.freebsd.org/base/head/sys/dev/cxgbe/firmware/t5fw_cfg_hashfilter.txt?view=co' |
# sed -i "" -e "s/nclip.*/nclip = 0/" /etc/t5fw.txt | # sed -i "" -e "s/nclip.*/nclip = 0/" /etc/t5fw.txt |
# sed -i "" -e "s/nfilter.*/nfilter = 2000/" /etc/t5fw.txt | # sed -i "" -e "s/nfilter.*/nfilter = 2016/" /etc/t5fw.txt |
# sed -i "" -e "s/nserver.*/nserver = 0/" /etc/t5fw.txt | # sed -i "" -e "s/nserver.*/nserver = 0/" /etc/t5fw.txt |
# sed -i "" -e "s/nhash.*/nhash = 0/" /etc/t5fw.txt | # sed -i "" -e "s/nhash.*/nhash = 0/" /etc/t5fw.txt |
12 690716 INFO RES le configuration: nentries 2048 route 32 clip 32 filter 1024 server 0 active 960 hash 0 nserversram 0 | 12 690716 INFO RES le configuration: nentries 2048 route 32 clip 32 filter 1024 server 0 active 960 hash 0 nserversram 0 |
</code> | </code> |
We confirm that server|hash are at 0, | We confirm that regions server and hash are at 0 (disabled): Notice that region clip is not disabled and filter didn't have the size we've instructed, but the filtering performance expectation are matched. |
| |
Now the packet drop rate by the TCAM firewall match the generator's 42Mpps: | Now the packet drop rate by the TCAM firewall match the generator's 42Mpps: |
</code> | </code> |
| |
And the firewall able to forward all packets without being too busy in the same time: | And the firewall is now able to forward all packets **without being too busy** in the same time: |
<code> | <code> |
[root@firewall]~# netstat -ihw 1 | [root@firewall]~# netstat -ihw 1 |
46M 0 422 2.7G 14M 0 801M 0 | 46M 0 422 2.7G 14M 0 801M 0 |
46M 0 3.9k 2.7G 14M 0 801M 0 | 46M 0 3.9k 2.7G 14M 0 801M 0 |
| |
[root@firewall]~# nstat -I cxl0 | [root@firewall]~# nstat -I cxl0 |
InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree | InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree |