User Tools

Site Tools


documentation:examples:ecmp

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision
Previous revision
documentation:examples:ecmp [2019/11/01 08:49] – [VM 2 (ECMP router)] olivierdocumentation:examples:ecmp [2022/12/11 18:40] (current) – [Checking routes installed] olivier
Line 1: Line 1:
 ====== Equal-cost multi-path routing (ECMP) ====== ====== Equal-cost multi-path routing (ECMP) ======
  
-This lab show an ECMP (feature called RADIX_MPATH on FreeBSD) routing example. +This lab need a FreeBSD 13-head r366390 minimum ([[https://reviews.freebsd.org/D26449|new ECMP routing code ]])
 ===== Presentation ===== ===== Presentation =====
  
 +<note warning>Bhyve doesn't support emulating multiqueue NIC, so RSS flow-id could not be tested using a bhyve based lab: Had to use physical lab</note>
 ==== Network diagram ==== ==== Network diagram ====
  
 Here is the logical and physical view: Here is the logical and physical view:
  
-{{:documentation:examples:labs.examples.ecmp.png|}}+{{:documentation:examples:bsdrp-lab-mpath.png|}}
  
 ===== Setting-up the lab ===== ===== Setting-up the lab =====
Line 15: Line 15:
 ==== Downloading BSD Router Project images ==== ==== Downloading BSD Router Project images ====
  
-Download BSDRP serial image (prevent to have to use an X display) on Sourceforge.+Download BSDRP serial image on Sourceforge and upload them to the 2 ECMP routers.
  
-==== Download Lab scripts ===== 
- 
-More information on these BSDRP lab scripts available on [[documentation:examples:How to build a BSDRP router lab]]. 
- 
-Start the lab with 6 routers. 
-With bhyve under FreeBSD: 
-<code> 
-user:~ # BSDRP-lab-bhyve.sh -i BSDRP.amd64/BSDRP-1.71-full-amd64-serial.img.xz -n 6 
-BSD Router Project (http://bsdrp.net) - bhyve full-meshed lab script 
-Setting-up a virtual lab with 6 VM(s): 
-- Working directory: /tmp/BSDRP 
-- Each VM have 1 core(s) and 256M RAM 
-- Switch mode: bridge + tap 
-- 0 LAN(s) between all VM 
-- Full mesh Ethernet links between each VM 
-VM 1 have the following NIC: 
-- vtnet0 connected to VM 2. 
-- vtnet1 connected to VM 3. 
-- vtnet2 connected to VM 4. 
-- vtnet3 connected to VM 5. 
-- vtnet4 connected to VM 6. 
-VM 2 have the following NIC: 
-- vtnet0 connected to VM 1. 
-- vtnet1 connected to VM 3. 
-- vtnet2 connected to VM 4. 
-- vtnet3 connected to VM 5. 
-- vtnet4 connected to VM 6. 
-VM 3 have the following NIC: 
-- vtnet0 connected to VM 1. 
-- vtnet1 connected to VM 2. 
-- vtnet2 connected to VM 4. 
-- vtnet3 connected to VM 5. 
-- vtnet4 connected to VM 6. 
-VM 4 have the following NIC: 
-- vtnet0 connected to VM 1. 
-- vtnet1 connected to VM 2. 
-- vtnet2 connected to VM 3. 
-- vtnet3 connected to VM 5. 
-- vtnet4 connected to VM 6. 
-VM 5 have the following NIC: 
-- vtnet0 connected to VM 1. 
-- vtnet1 connected to VM 2. 
-- vtnet2 connected to VM 3. 
-- vtnet3 connected to VM 4. 
-- vtnet4 connected to VM 6. 
-VM 6 have the following NIC: 
-- vtnet0 connected to VM 1. 
-- vtnet1 connected to VM 2. 
-- vtnet2 connected to VM 3. 
-- vtnet3 connected to VM 4. 
-- vtnet4 connected to VM 5. 
-For connecting to VM'serial console, you can use: 
-- VM 1 : cu -l /dev/nmdm1B 
-- VM 2 : cu -l /dev/nmdm2B 
-- VM 3 : cu -l /dev/nmdm3B 
-- VM 4 : cu -l /dev/nmdm4B 
-- VM 5 : cu -l /dev/nmdm5B 
-- VM 6 : cu -l /dev/nmdm6B 
-</code> 
  
 ===== Static routing setup ===== ===== Static routing setup =====
  
-==== VM 1 (client) ====+==== Client ====
  
-VM1 is configured as a simple host with multiples aliases:+simple host with static routes:
  
 <code> <code>
-sysrc hostname=VM1 +sysrc hostname=client \ 
-sysrc gateway_enable=NO +  gateway_enable=NO \ 
-sysrc ipv6_gateway_enable=NO +  ipv6_gateway_enable=NO \ 
-sysrc ifconfig_vtnet0="inet 10.0.12.1/24" +  ifconfig_igb1="inet 10.0.12.1/24" \ 
-sysrc ifconfig_vtnet0_alias1="inet 10.0.12.11/32+  ifconfig_igb1_ipv6="inet6 2001:db8:12::3 prefixlen 64\ 
-sysrc ifconfig_vtnet0_alias2="inet 10.0.12.21/32+  static_routes="LAB\ 
-sysrc ifconfig_vtnet0_alias3="inet 10.0.12.31/32" +  route_LAB="-net 10.0.0.0/16 10.0.12.2\ 
-sysrc ifconfig_vtnet0_alias4="inet 10.0.12.41/32" +  ipv6_static_routes="LAB\ 
-sysrc ifconfig_vtnet0_alias5="inet 10.0.12.51/32+  ipv6_route_LAB="2001:db8:: -prefixlen 32 2001:db8:12::2"
-sysrc ifconfig_vtnet0_ipv6="inet6 2001:db8:12::1 prefixlen 64+
-sysrc ifconfig_vtnet0_alias61="inet6 2001:db8:12::11 prefixlen 64" +
-sysrc ifconfig_vtnet0_alias62="inet6 2001:db8:12::21 prefixlen 64" +
-sysrc ifconfig_vtnet0_alias63="inet6 2001:db8:12::31 prefixlen 64" +
-sysrc ifconfig_vtnet0_alias64="inet6 2001:db8:12::41 prefixlen 64" +
-sysrc ifconfig_vtnet0_alias65="inet6 2001:db8:12::51 prefixlen 64" +
-sysrc defaultrouter="10.0.12.2" +
-sysrc ipv6_defaultrouter="2001:db8:12::2"+
 service hostname restart service hostname restart
 service netif restart service netif restart
Line 107: Line 40:
 </code> </code>
  
-==== VM 2 (ECMP router) ====+==== R1 (ECMP router) ====
  
-VM 2 is a router with ECMP: 2 static routes toward the same destination but using 2 different next-hop+R1 is a router with ECMP: 2 static routes toward the same destination but using 2 different next-hop.
-Need to disable fastforward by re-enabling ICMP redirect (disabled on BSDRP by default).+
  
 <code> <code>
-sysrc hostname=VM2 + 
-sysrc ifconfig_vtnet0="inet 10.0.12.2/24" +sysrc hostname=R1 \ 
-sysrc ifconfig_vtnet1="inet 10.0.23.2/24+  gateway_enable=YES \ 
-sysrc ifconfig_vtnet2="inet 10.0.24.2/24" +  ipv6_gateway_enable=YES \ 
-sysrc ifconfig_vtnet0_ipv6="inet6 2001:db8:12::2 prefixlen 64" +  ifconfig_igb0="inet 10.0.12.2/24" \ 
-sysrc ifconfig_vtnet1_ipv6="inet6 2001:db8:23::prefixlen 64+  ifconfig_igb0_ipv6="inet6 2001:db8:12::prefixlen 64\ 
-sysrc ifconfig_vtnet2_ipv6="inet6 2001:db8:24::2 prefixlen 64" +  ifconfig_igb1="inet 10.0.231.2/24" \ 
-sysrc static_routes="VM3VM6 VM3VM5 VM4VM6 VM4VM5+  ifconfig_igb1_ipv6="inet6 2001:db8:231::2 prefixlen 64" \ 
-sysrc route_VM3VM6="-net 10.0.56.0/24 10.0.23.3" +  ifconfig_igb2="inet 10.0.232.2/24\ 
-sysrc route_VM3VM5="-net 10.0.35.0/24 10.0.23.3" +  ifconfig_igb2_ipv6="inet6 2001:db8:232::2 prefixlen 64" \ 
-sysrc route_VM4VM6="-net 10.0.56.0/24 10.0.24.4" +  static_routes="MPATH1 MPATH2\ 
-sysrc route_VM4VM5="-net 10.0.45.0/24 10.0.24.4+  route_MPATH1="-net 10.0.0.0/16 10.0.231.3" \ 
-sysrc ipv6_static_routes="VM3VM6 VM3VM5 VM4VM6 VM4VM5+  route_MPATH2="-net 10.0.0.0/16 10.0.232.3\ 
-sysrc ipv6_route_VM3VM6="2001:db8:56:: -prefixlen 64 2001:db8:23::3" +  ipv6_static_routes="MPATH1 MPATH2\ 
-sysrc ipv6_route_VM3VM5="2001:db8:35:: -prefixlen 64 2001:db8:23::3" +  ipv6_route_MPATH1="2001:db8:: -prefixlen 32 2001:db8:231::3" \ 
-sysrc ipv6_route_VM4VM6="2001:db8:56:: -prefixlen 64 2001:db8:24::4" +  ipv6_route_MPATH2="2001:db8:: -prefixlen 32 2001:db8:232::3"
-sysrc ipv6_route_VM4VM5="2001:db8:45:: -prefixlen 64 2001:db8:24::4" +
-sed -i '' -e '/net.inet.ip.redirect=0/d' /etc/sysctl.conf +
-sed -i '' -e '/net.inet6.ip6.redirect=0/d' /etc/sysctl.conf +
-sysctl net.inet.ip.redirect=1 +
-sysctl net.inet6.ip6.redirect=1+
 service hostname restart service hostname restart
 service netif restart service netif restart
Line 140: Line 67:
 </code> </code>
  
-And first problem using static route:+Checking static route with multiple next-hop:
  
 <code> <code>
-add net 10.0.56.0: gateway 10.0.23.3 fib +root@R1:~ # netstat -rn4 | grep 10.0.0.0/16 
-add net 10.0.35.0: gateway 10.0.23.3 fib +10.0.0.0/16        10.0.231.3         UGS        igb1 
-add net 10.0.56.0: gateway 10.0.24.4 fib 0: route already in table +10.0.0.0/16        10.0.232.3         UGS        igb2 
-add net 10.0.45.0: gateway 10.0.24.4 fib + 
-add net 2001:db8:56::: gateway 2001:db8:23::3 fib +root@R1:~ # netstat -4onW 
-add net 2001:db8:35::: gateway 2001:db8:23::3 fib +Nexthop data 
-add net 2001:db8:56::: gateway 2001:db8:24::4 fib 0: route already in table + 
-add net 2001:db8:45::: gateway 2001:db8:24::4 fib +Internet: 
-[root@VM2]~# route get 10.0.56.0/24 +Idx   Type         IFA                Gateway             Flags      Use Mtu         Netif     Addrif Refcnt Prepend 
-   route to10.0.56.0 +1       v4/resolve 127.0.0.1          lo0/resolve        H              16384        lo0               2 
-destination10.0.56.0 +2       v4/resolve 10.0.12.2          igb0/resolve                       1500       igb0               2 
-       mask255.255.255.0 +      v4/resolve 127.0.0.1          lo0/resolve        HS            0  16384        lo0      igb0     2 
-    gateway10.0.23.3 +4       v4/resolve 10.0.231.2         igb1/resolve                       1500       igb1               2 
-        fib+5       v4/resolve 127.0.0.1          lo0/resolve        HS            0  16384        lo0      igb1     2 
-  interfacevtnet1 +6       v4/resolve 10.0.232.2         igb2/resolve                       1500       igb2               2 
-      flags<UP,GATEWAY,DONE,STATIC> +7       v4/resolve 127.0.0.1          lo0/resolve        HS            0  16384        lo0      igb2     2 
- recvpipe  sendpipe  ssthresh  rtt,msec    mtu        weight    expire +8            v4/gw 10.0.231.2         10.0.231.3         GS            0   1500       igb1               1 
-               0                      1500                 0+9            v4/gw 10.0.232.2         10.0.232.3         GS            0   1500       igb2               1 
 + 
 +root@R1:~ # netstat -6onW 
 +Nexthop data 
 + 
 +Internet6: 
 +Idx   Type         IFA                           Gateway                        Flags      Use Mtu       Netif   Addrif Refcnt Prepend 
 +1       v6/resolve ::1                           lo0/resolve                   HS            0  16384      lo0             2 
 +2       v6/resolve fe80::1%lo0                   lo0/resolve                   HS            0  16384      lo0             2 
 +3       v6/resolve fe80::1%lo0                   lo0/resolve                                  16384      lo0             2 
 +4       v6/resolve ::1                           lo0/resolve                   HS             16384      lo0    igb0     3 
 +5       v6/resolve fe80::20d:b9ff:fe41:ca3c%igb0 igb0/resolve                                0   1500     igb0             3 
 +6       v6/resolve ::1                           lo0/resolve                   HS            0  16384      lo0    igb1     3 
 +7       v6/resolve fe80::20d:b9ff:fe41:ca3d%igb1 igb1/resolve                                0   1500     igb1             3 
 +8       v6/resolve ::1                           lo0/resolve                   HS             16384      lo0    igb2     3 
 +9       v6/resolve fe80::20d:b9ff:fe41:ca3e%igb2 igb2/resolve                                0   1500     igb2             3 
 +10           v6/gw ::1                           ::1                           GRS            16384      lo0             5 
 +11           v6/gw 2001:db8:231::2               2001:db8:231::3               GS              1500     igb1             1 
 +12           v6/gw 2001:db8:232::2               2001:db8:232::3               GS              1500     igb2             1
 </code> </code>
-==== VM 3 (router) ====+==== R2 (ECMP router) ====
  
-VM 3 is a simple router.+R2 is like R1, a router with ECMP: 2 static routing toward the same destination but using 2 different next-hop..
  
 <code> <code>
-sysrc hostname=VM3 +sysrc hostname=R2 \ 
-sysrc ifconfig_vtnet1="inet 10.0.23.3/24" +  gateway_enable=YES \ 
-sysrc ifconfig_vtnet3="inet 10.0.35.3/24" +  ipv6_gateway_enable=YES \ 
-sysrc ifconfig_vtnet1_ipv6="inet6 2001:db8:23::3 prefixlen 64" +  ifconfig_igb0="inet 10.0.34.3/24" 
-sysrc ifconfig_vtnet3_ipv6="inet6 2001:db8:35::3 prefixlen 64" +  ifconfig_igb0_ipv6="inet6 2001:db8:34::3 prefixlen 64" \ 
-sysrc static_routes="VM6 VM1+  ifconfig_igb1="inet 10.0.231.3/24" \ 
-sysrc route_VM1="-net 10.0.12.0/24 10.0.23.2" +  ifconfig_igb1_ipv6="inet6 2001:db8:231::3 prefixlen 64" \ 
-sysrc route_VM6="-net 10.0.56.0/24 10.0.35.5+  ifconfig_igb2="inet 10.0.232.3/24"
-sysrc ipv6_static_routes="VM6 VM1+  ifconfig_igb2_ipv6="inet6 2001:db8:232::3 prefixlen 64" \ 
-sysrc ipv6_route_VM1="2001:db8:12:: -prefixlen 64 2001:db8:23::2" +  static_routes="DST1 DST2 SRC1 SRC2\ 
-sysrc ipv6_route_VM6="2001:db8:56:: -prefixlen 64 2001:db8:35::5"+  route_DST1="-net 10.0.0.0/16 10.0.231.4"
 +  route_DST2="-net 10.0.0.0/16 10.0.232.4"
 +  route_SRC1="-net 10.0.12.0/24 10.0.231.2" \ 
 +  route_SRC2="-net 10.0.12.0/24 10.0.232.2\ 
 +  ipv6_static_routes="DST1 DST2 SRC1 SRC2\ 
 +  ipv6_route_MPATH1="2001:db8:: -prefixlen 32 2001:db8:231::4"
 +  ipv6_route_MPATH2="2001:db8:: -prefixlen 32 2001:db8:231::4"
 +  ipv6_route_SRC1="2001:db8:12:: -prefixlen 64 2001:db8:231::2" \ 
 +  ipv6_route_SRC2="2001:db8:12:: -prefixlen 64 2001:db8:231::2"
 service hostname restart service hostname restart
 service netif restart service netif restart
Line 183: Line 136:
 config save config save
 </code> </code>
 +==== Server ====
  
-==== VM 4 (router) ==== +simple host with some static routes:
- +
-VM 4 is like VM 3, a simple router.+
  
 <code> <code>
-sysrc hostname=VM4 +sysrc hostname=server \ 
-sysrc ifconfig_vtnet1="inet 10.0.24.4/24" +  gateway_enable=NO \ 
-sysrc ifconfig_vtnet3="inet 10.0.45.4/24" +  ipv6_gateway_enable=NO \ 
-sysrc ifconfig_vtnet1_ipv6="inet6 2001:db8:24::4 prefixlen 64" +  ifconfig_igb1="inet 10.0.34.4/24" \ 
-sysrc ifconfig_vtnet3_ipv6="inet6 2001:db8:45::4 prefixlen 64+  ifconfig_igb1_ipv6="inet6 2001:db8:34::4 prefixlen 64" \ 
-sysrc static_routes="VM6 VM1+  static_routes="12 231 232\ 
-sysrc route_VM1="-net 10.0.12.0/24 10.0.24.2+  route_12="-net 10.0.12.0/24 10.0.34.3\ 
-sysrc route_VM6="-net 10.0.56.0/24 10.0.45.5+  route_231="-net 10.0.231.0/24 10.0.34.3\ 
-sysrc ipv6_static_routes="VM6 VM1+  route_232="-net 10.0.232.0/24 10.0.34.3\ 
-sysrc ipv6_route_VM1="2001:db8:12:: -prefixlen 64 2001:db8:24::2+  ipv6_static_routes="12 231 232\ 
-sysrc ipv6_route_VM6="2001:db8:56:: -prefixlen 64 2001:db8:45::5"+  ipv6_route_12="2001:db8:12:: -prefixlen 64 2001:db8:34::3" \ 
 +  ipv6_route_231="2001:db8:231:: -prefixlen 64 2001:db8:34::3\ 
 +  ipv6_route_232="2001:db8:232:: -prefixlen 64 2001:db8:34::3"
 service hostname restart service hostname restart
 service netif restart service netif restart
Line 206: Line 160:
 </code> </code>
  
-==== VM 5 (ECMP router) ====+===== FRR Multipath setup =====
  
-VM 5 is like VM 2, a router with ECMP: 2 static routing toward the same destination but using 2 different next-hop..+Replacing static routes by FRR (OSPF) compiled with MULTIPATH option. 
 + 
 +==== R1 (ECMP router) ==== 
 + 
 +In place of static routes, OSPF with FRR is used:
  
 <code> <code>
-sysrc hostname=VM5 +sysrc frr_vtysh_boot="YES\ 
-sysrc ifconfig_vtnet2="inet 10.0.35.5/24+  frr_enable="YES\ 
-sysrc ifconfig_vtnet3="inet 10.0.45.5/24+  frr_daemons="zebra ospfd ospf6d\ 
-sysrc ifconfig_vtnet4="inet 10.0.56.5/24+  watchfrr_flags=" -d -r /usr/sbin/servicebBfrrbBrestartbB%s -s /usr/sbin/servicebBfrrbBstartbB%s -k /usr/sbin/servicebBfrrbBstopbB%s -b bB -t 30 zebra ospfd ospf6d\ 
-sysrc ifconfig_vtnet2_ipv6="inet6 2001:db8:35::5 prefixlen 64+  watchfrr_enable="YES
-sysrc ifconfig_vtnet3_ipv6="inet6 2001:db8:45::5 prefixlen 64+ 
-sysrc ifconfig_vtnet4_ipv6="inet6 2001:db8:56::5 prefixlen 64" +cat > /usr/local/etc/frr/frr.conf <EOF 
-sysrc static_routes="VM3VM1 VM3VM2 VM4VM1 VM4VM2" +frr version 8.4.1 
-sysrc route_VM3VM1="-net 10.0.12.0/24 10.0.35.3" +frr defaults traditional 
-sysrc route_VM3VM2="-net 10.0.23.0/24 10.0.35.3" +hostname R1 
-sysrc route_VM4VM1="-net 10.0.12.0/24 10.0.45.4" +
-sysrc route_VM4VM2="-net 10.0.24.0/24 10.0.45.4" +interface igb0 
-sysrc ipv6_static_routes="VM3VM1 VM3VM2 VM4VM1 VM4VM2" + ip ospf passive 
-sysrc ipv6_route_VM3VM1="2001:db8:12:: -prefixlen 64 2001:db8:35::3" + ipv6 ospf6 area 0.0.0.
-sysrc ipv6_route_VM3VM2="2001:db8:23:: -prefixlen 64 2001:db8:35::3" + ipv6 ospf6 passive 
-sysrc ipv6_route_VM4VM1="2001:db8:12:: -prefixlen 64 2001:db8:45::4" +exit 
-sysrc ipv6_route_VM4VM2="2001:db8:24:: -prefixlen 64 2001:db8:45::4" +
-sed -i '' -e '/net.inet.ip.redirect=0/d' /etc/sysctl.conf +interface igb1 
-sed -i '' -e '/net.inet6.ip6.redirect=0/d' /etc/sysctl.conf + ipv6 ospf6 area 0.0.0.0 
-sysctl net.inet.ip.redirect=1 +exit 
-sysctl net.inet6.ip6.redirect=1 +
-service hostname restart +interface igb2 
-service netif restart + ipv6 ospf6 area 0.0.0.0 
-service routing restart +exit 
-config save+
 +router ospf 
 + ospf router-id 1.1.1.1 
 + network 10.0.12.0/24 area 0 
 + network 10.0.231.0/24 area 0 
 + network 10.0.232.0/24 area 
 +exit 
 +! 
 +router ospf6 
 +exit 
 +! 
 +EOF 
 +service frr start 
 +service watchfrr start
 </code> </code>
  
-==== VM 6 (server) ====+==== R2 (ECMP router) ====
  
-VM 6, like VM 1 is configured as a simple host with multiple aliases:+Same as R1 with OSPF and FRR: 
 +<code> 
 +sysrc frr_vtysh_boot="YES"
 +  frr_enable="YES"
 +  frr_daemons="zebra staticd ospfd ospf6d"
 +  watchfrr_flags=" -d -r /usr/sbin/servicebBfrrbBrestartbB%s -s /usr/sbin/servicebBfrrbBstartbB%s -k /usr/sbin/servicebBfrrbBstopbB%s -b bB -t 30 zebra ospfd ospf6d"
 +  watchfrr_enable="YES"
  
 +cat > /usr/local/etc/frr/frr.conf <EOF
 +frr version 8.4.1
 +frr defaults traditional
 +hostname R2
 +!
 +ip route 10.0.0.0/16 10.0.34.4
 +ipv6 route 2001:db8::/32 2001:db8:231::4
 +!
 +interface igb0
 + ip ospf passive
 + ipv6 ospf6 area 0.0.0.0
 + ipv6 ospf6 passive
 +exit
 +!
 +interface igb1
 + ipv6 ospf6 area 0.0.0.0
 +exit
 +!
 +interface igb2
 + ipv6 ospf6 area 0.0.0.0
 +exit
 +!
 +router ospf
 + ospf router-id 2.2.2.2
 + redistribute static
 + network 10.0.34.0/24 area 0
 + network 10.0.231.0/24 area 0
 + network 10.0.232.0/24 area 0
 +exit
 +!
 +router ospf6
 + redistribute static
 +exit
 +!
 +EOF
 +service frr start
 +service watchfrr start
 +</code>
 +
 +==== Checking routes installed ====
 +
 +On R1:
 <code> <code>
-sysrc hostname=VM6 +root@R1:~ # vtysh 
-sysrc gateway_enable=NO +Hello, this is FRRouting (version 8.4.1)
-sysrc ipv6_gateway_enable=NO +Copyright 1996-2005 Kunihiro Ishiguro, et al
-sysrc ifconfig_vtnet4="inet 10.0.56.6/24" + 
-sysrc ifconfig_vtnet4_alias1="inet 10.0.56.16/32" +R1# sh ip route 10.0.0.0 
-sysrc ifconfig_vtnet4_alias2="inet 10.0.56.26/32" +Routing entry for 10.0.0.0/16 
-sysrc ifconfig_vtnet4_alias3="inet 10.0.56.36/32+  Known via "ospf", distance 110, metric 20, best 
-sysrc ifconfig_vtnet4_alias4="inet 10.0.56.46/32" +  Last update 00:02:26 ago 
-sysrc ifconfig_vtnet4_alias5="inet 10.0.56.56/32" +  * 10.0.231.3, via igb1, weight 1 
-sysrc ifconfig_vtnet4_ipv6="inet6 2001:db8:56::6 prefixlen 64" +  10.0.232.3, via igb2, weight 1 
-sysrc ifconfig_vtnet4_alias61="inet6 2001:db8:56::16 prefixlen 64" + 
-sysrc ifconfig_vtnet4_alias62="inet6 2001:db8:56::26 prefixlen 64+R1# sh ipv6 route 2001:db8:: 
-sysrc ifconfig_vtnet4_alias63="inet6 2001:db8:56::36 prefixlen 64" +Routing entry for 2001:db8::/32 
-sysrc ifconfig_vtnet4_alias64="inet6 2001:db8:56::46 prefixlen 64" +  Known via "ospf6", distance 110, metric 20, best 
-sysrc ifconfig_vtnet4_alias65="inet6 2001:db8:56::56 prefixlen 64" +  Last update 00:02:39 ago 
-sysrc defaultrouter="10.0.56.5" +  * fe80::20d:b9ff:fe45:7ad5, via igb1, weight 1 
-sysrc ipv6_defaultrouter="2001:db8:56::5" +  * fe80::20d:b9ff:fe45:7ad6, via igb2, weight 1
-service hostname restart +
-service netif restart +
-service routing restart +
-config save+
 </code> </code>
  
-==== Testing Load balancing ====+===== Test Load balancing IP packets =====
  
-Flows from VM1 to VM6 should be load-balanced by VM2 toward VM3 and VM4. Let's check using multiple sources and destination IP addresses. +Flows from the client to the server should be "flow-id shared" between the 2 paths. Let's check using multiple sources and destination IP addresses using pkt-gen on client and server, then using systat on R1 and R2 to check their load-distribution
-Using ping record feature like with this example:+ 
 +On server:
 <code> <code>
-[root@VM1]~# ping -c1 --S 10.0.12.21 10.0.56.26 +root@server:~ # pkt-gen -i igb1 -f rx 
-PING 10.0.56.26 (10.0.56.26) from 10.0.12.21: 56 data bytes +</code>
-64 bytes from 10.0.56.26: icmp_seq=0 ttl=61 time=0.358 ms +
-RR:     10.0.23.2 +
-        10.0.35.3 +
-        10.0.56.5 +
-        10.0.56.26 +
-        10.0.35.5 +
-        10.0.23.3 +
-        10.0.12.2 +
-        10.0.12.21+
  
---- 10.0.56.26 ping statistics --- +On client: 
-1 packets transmitted, packets received, 0.0% packet loss +<code> 
-round-trip min/avg/max/stddev = 0.358/0.358/0.358/0.000 ms+root@client:~ # pkt-gen -i igb1 -f tx -n 8000000 -l 60 -10.0.255.1:2000-10.0.255.254 -D 00:0d:b9:41:ca:3c -s 10.0.254.1:2000-10.0.254.254 -S 00:0d:b9:45:7f:b0 -w 4 -R 20000
 </code> </code>
  
-^ Source ^ Destination ^ Ongoing path selected by VM2 ^ Returning path selected by VM5^ +On R1: 
-| 10.0.12.1 | 10.0.56.6 | VM3 | VM3 | +<code> 
-| 10.0.12.11 | 10.0.56.16 | VM3 | VM3 | +systat -ifstat -match igb0,igb1,igb2 -pps 
-| 10.0.12.21 | 10.0.56.26 | VM3 | VM3 | + 
-| 10.0.12.31 10.0.56.36 | VM3 | VM3 | +                    /  /  /2   /  /  /  /  /  /  /  /10 
-| 10.0.12.41 | 10.0.56.46 | VM3 | VM3 | +     Load Average   | 
-| 10.0.12.51 | 10.0.56.56 | VM3 | VM3 | + 
-| 2001:db8:12::1 | 2001:db8:56::6 | VM3 | VM3 | +      Interface           Traffic               Peak                Total 
-| 2001:db8:12::11 | 2001:db8:56::16 | VM3 | VM3 | +           igb2  in      0.000 Kp/s          0.000 Kp/s           71.247 Mp 
-| 2001:db8:12::21 | 2001:db8:56::26 | VM3 | VM3 | +                 out     9.762 Kp/s          9.777 Kp/s           76.892 Mp 
-| 2001:db8:12::31 | 2001:db8:56::36 | VM3 | VM3 | + 
-| 2001:db8:12::41 | 2001:db8:56::46 | VM3 | VM3 | +           igb1  in      0.000 Kp/s          0.000 Kp/s           71.392 Mp 
-| 2001:db8:12::51 | 2001:db8:56::56 | VM3 | VM3 | +                 out     9.770 Kp/s          9.771 Kp/s           80.341 Mp 
-| 10.0.12.1 | 10.0.45.5 | VM4 | VM3 |+ 
 +           igb0  in     19.533 Kp/s         19.534 Kp/s           90.007 Mp 
 +                 out     0.000 Kp/s          0.000 Kp/s            0.243 Kp 
 +                  
 +</code> 
 + 
 +=> We confirm that 20 Kps entering igb0 and are equally split by exiting by igb1 and igb2 
 + 
 +On R2: 
 +<code> 
 +systat -ifstat -match igb0,igb1,igb2 -pps 
 + 
 +                    /0   /  /  /  /  /  /  /  /  /  /10 
 +     Load Average   | 
 + 
 +      Interface           Traffic               Peak                Total 
 +           igb2  in      9.768 Kp/s          9.771 Kp/s          300.830 Kp 
 +                 out     0.000 Kp/s          0.000 Kp/s            0.000 Kp 
 + 
 +           igb1  in      9.763 Kp/s          9.768 Kp/s          300.785 Kp 
 +                 out     0.000 Kp/s          0.000 Kp/s            0.006 Kp 
 + 
 +           igb0  in      0.000 Kp/s          0.001 Kp/s            0.240 Kp 
 +                 out    19.530 Kp/s         19.531 Kp/s          601.615 Kp 
 +</code>
  
-<note>There is no flow load-balancing with RADIX MPATH since FreeBSD 11[[https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225792|seems broken with tryforward]], put proposed patch disable tryforward(fastforward) if RADIX_MPATH is enabled.</note>+=R2 has no choice than receiving packets from igb1 and igb2and forwarding them through igb0.
  
documentation/examples/ecmp.1572594552.txt.gz · Last modified: 2019/11/01 08:49 by olivier

Except where otherwise noted, content on this wiki is licensed under the following license: BSD 2-Clause
Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki