documentation:examples:freebsd_performance_regression_lab
no way to compare when less than two revisions
Differences
This shows you the differences between two versions of the page.
Last revision | |||
— | documentation:examples:freebsd_performance_regression_lab [2017/03/29 09:32] – external edit 127.0.0.1 | ||
---|---|---|---|
Line 1: | Line 1: | ||
+ | ====== FreeBSD performance regression lab ====== | ||
+ | {{description> | ||
+ | ===== Concepts ===== | ||
+ | |||
+ | This article presents the method used for spotting some performance regression problem during the evolution of FreeBSD code (regarding different svn numbers). | ||
+ | |||
+ | Here is an example of a final bench graphic measuring the impact of forwarding performance: | ||
+ | {{: | ||
+ | |||
+ | This method follow these steps: | ||
+ | - A lab of one or more servers is set-up. [[documentation/ | ||
+ | - A [[https:// | ||
+ | - A first [[https:// | ||
+ | - [[https:// | ||
+ | - A second [[https:// | ||
+ | - Upgrading the server to the nanobsd release to be tested; | ||
+ | - ..Uploading the configuration set to be tested (in this example: forwarding-only, | ||
+ | - ....Start the bench test and collecting the result; | ||
+ | - ....Reboot the DUT; | ||
+ | - ....Loop 5 times (for having usable [[http:// | ||
+ | - ..Loop to next configuration set; | ||
+ | - Loop to next nanobsd release | ||
+ | - [[https:// | ||
+ | ===== Physical lab ===== | ||
+ | |||
+ | The physical lab used by BSDRP targets forwarding network speed. And for this purpose it's composed of minimum 3 servers and 1 switches, and maximum 4 servers and 2 switchs. | ||
+ | Common to all these setups: | ||
+ | * The controller server that runs the bench script; | ||
+ | * The admin switch (send SSH commands) | ||
+ | |||
+ | Specific: | ||
+ | * One server with netmap compliant NIC for being used as packet generator/ | ||
+ | * One server as DUT | ||
+ | * And for network crypto (IPSec, OpenVPN) bench, a referent server | ||
+ | * A dedicated switch with ([[documentation/ | ||
+ | |||
+ | ==== 2 nodes lab examples ==== | ||
+ | |||
+ | === simplest lab === | ||
+ | |||
+ | This simple lab is great for benching 10Gigabit performance without the need of 10Gigabit switch: | ||
+ | |||
+ | {{documentation: | ||
+ | |||
+ | === with switch lab === | ||
+ | |||
+ | This simple lab is usefull for cross-checking NIC statistics counters with switch statistics counters: | ||
+ | |||
+ | {{documentation: | ||
+ | |||
+ | ==== 3 nodes lab example ==== | ||
+ | |||
+ | This is the setup used for a VPN benchmark: | ||
+ | |||
+ | {{documentation: | ||
+ | |||
+ | ===== Generating FreeBSD nanobsd images ===== | ||
+ | |||
+ | The objective is to generate BSDRP nanobsd images regarding a list of FreeBSD' | ||
+ | |||
+ | You need to [[documentation: | ||
+ | |||
+ | Then you have to create or customize a BSDRP project for obtaining a small image: Try to remove maximum [[http:// | ||
+ | |||
+ | Take care of the kernel configuration file: the head branch can impose kernel configuration parameters incompatible between different code revisions. The best method for avoiding this incompatibility is to start the kernel configuration by " | ||
+ | |||
+ | Once your BSDRP bench-project ready, edit the script [[https:// | ||
+ | < | ||
+ | root@dev:/ | ||
+ | root@dev:/ | ||
+ | Building image matching revision 257506...done | ||
+ | Building image matching revision 257657...done | ||
+ | Building image matching revision 257715...done | ||
+ | All images were put in / | ||
+ | </ | ||
+ | |||
+ | At the end, you will find all the nanobsd images in your selected directory. | ||
+ | |||
+ | You can find an example of finals images generated here: [[http:// | ||
+ | |||
+ | ===== Configuration sets ===== | ||
+ | |||
+ | The creation of different configuration sets is pretty simple: just create a main folder and inside this folder, creating different folder for each configuration sets. | ||
+ | |||
+ | Inside each configuration sets folder will be placed the configuration files that need override default parameters. | ||
+ | |||
+ | Here is a very simple example regarding [[documentation/ | ||
+ | |||
+ | We want 3 configuration sets: | ||
+ | * First called " | ||
+ | * Second called " | ||
+ | * Third called " | ||
+ | |||
+ | < | ||
+ | mkdir -p / | ||
+ | cat > / | ||
+ | hw.igb.rxd=" | ||
+ | hw.igb.txd=" | ||
+ | hw.igb.rx_process_limit=" | ||
+ | EOF | ||
+ | mkdir -p / | ||
+ | cat > / | ||
+ | hw.igb.rxd=" | ||
+ | hw.igb.txd=" | ||
+ | hw.igb.rx_process_limit=" | ||
+ | EOF | ||
+ | mkdir -p / | ||
+ | cat > / | ||
+ | hw.igb.rxd=" | ||
+ | hw.igb.txd=" | ||
+ | hw.igb.rx_process_limit=" | ||
+ | EOF | ||
+ | </ | ||
+ | |||
+ | ===== Testing Lab ===== | ||
+ | |||
+ | Start by configuring all servers, and the DUT with the configurations parameters common to all configuration sets (but remember that files in configuration sets will totally overwrite existing file). | ||
+ | Install your ssh public-key on each servers for permitting bench script to automatically connect to them. | ||
+ | |||
+ | There are 2 examples of configuration files available for a bench lab: | ||
+ | * [[https:// | ||
+ | * [[https:// | ||
+ | Put on one of these files, all the IP adresses, testing command lines, etc corresponding to your bench test. | ||
+ | |||
+ | Then start [[https:// | ||
+ | Then start it: | ||
+ | < | ||
+ | ./ | ||
+ | </ | ||
+ | |||
+ | And here is an example: | ||
+ | < | ||
+ | olivier@lame4: | ||
+ | s/ -p ../ | ||
+ | BSDRP automatized upgrade/ | ||
+ | |||
+ | This script will start 210 bench tests using: | ||
+ | - Multiples images to test: yes | ||
+ | - Multiples configuration-sets to test: yes | ||
+ | - Multiples pkt-gen configuration to test: yes | ||
+ | - Number of iteration for each set: 5 | ||
+ | - Results dir: forwarding-pf-ipfw/ | ||
+ | |||
+ | Do you want to continue ? (y/n): y | ||
+ | Testing ICMP connectivity to each devices: | ||
+ | 192.168.1.8...OK | ||
+ | 192.168.1.8...OK | ||
+ | 192.168.1.10...OK | ||
+ | Testing SSH connectivity with key to each devices: | ||
+ | 192.168.1.8...OK | ||
+ | 192.168.1.8...OK | ||
+ | 192.168.1.10...OK | ||
+ | Starting the benchs | ||
+ | Start firmware image set: / | ||
+ | Upgrading...done | ||
+ | Rebooting 192.168.1.10 and waiting device return online...done | ||
+ | Start configuration set: forwarding | ||
+ | Uploading cfg forwarding-pf-ipfw/ | ||
+ | Rebooting 192.168.1.10 and waiting device return online...done | ||
+ | Start pkt-gen set: ../ | ||
+ | Start bench serie bench.311014.forwarding.inet4.1 | ||
+ | Waiting for end of bench 1/ | ||
+ | Rebooting 192.168.1.10 and waiting device return online...done | ||
+ | Start bench serie bench.311014.forwarding.inet4.2 | ||
+ | Waiting for end of bench 2/ | ||
+ | Rebooting 192.168.1.10 and waiting device return online...done | ||
+ | Start bench serie bench.311014.forwarding.inet4.3 | ||
+ | Waiting for end of bench 3/ | ||
+ | Rebooting 192.168.1.10 and waiting device return online...done | ||
+ | Start bench serie bench.311014.forwarding.inet4.4 | ||
+ | Waiting for end of bench 4/ | ||
+ | Rebooting 192.168.1.10 and waiting device return online...done | ||
+ | Start bench serie bench.311014.forwarding.inet4.5 | ||
+ | Waiting for end of bench 5/ | ||
+ | Rebooting 192.168.1.10 and waiting device return online...done | ||
+ | Start pkt-gen set: ../ | ||
+ | Start bench serie bench.311014.forwarding.inet6.1 | ||
+ | Waiting for end of bench 6/ | ||
+ | Rebooting 192.168.1.10 and waiting device return online...done | ||
+ | Start bench serie bench.311014.forwarding.inet6.2 | ||
+ | Waiting for end of bench 7/ | ||
+ | Rebooting 192.168.1.10 and waiting device return online...done | ||
+ | Start bench serie bench.311014.forwarding.inet6.3 | ||
+ | Waiting for end of bench 8/ | ||
+ | Rebooting 192.168.1.10 and waiting device return online...done | ||
+ | Start bench serie bench.311014.forwarding.inet6.4 | ||
+ | Waiting for end of bench 9/ | ||
+ | Rebooting 192.168.1.10 and waiting device return online...done | ||
+ | Start bench serie bench.311014.forwarding.inet6.5 | ||
+ | Waiting for end of bench 10/ | ||
+ | Rebooting 192.168.1.10 and waiting device return online...done | ||
+ | Start configuration set: ipfw-statefull | ||
+ | Uploading cfg forwarding-pf-ipfw/ | ||
+ | Rebooting 192.168.1.10 and waiting device return online...done | ||
+ | ... | ||
+ | Rebooting DUT and waiting device return online...done | ||
+ | Waiting for end of bench 209/ | ||
+ | Rebooting DUT and waiting device return online...done | ||
+ | Waiting for end of bench 210/ | ||
+ | All bench tests were done, results in forwarding-pf-ipfw/ | ||
+ | </ | ||
+ | ===== Results generation ===== | ||
+ | |||
+ | Once bench finished, there are lot's of raw result files with these names: | ||
+ | * bench.NANOBSD-SVN.CONFIGURATION-SETS.info: | ||
+ | * bench.NANOBSD-SVN.CONFIGURATION-SETS.ITERATION.receiver: | ||
+ | * bench.NANOBSD-SVN.CONFIGURATION-SETS.ITERATION.sender: | ||
+ | |||
+ | Here is [[https:// | ||
+ | |||
+ | These raw results, closely dependents of the bench tool used, need to be filtered (and synthesized) in a usable data file. | ||
+ | |||
+ | BSDRP use the script [[https:// |
documentation/examples/freebsd_performance_regression_lab.txt · Last modified: 2020/12/30 08:29 by olivier