BSD Router Project

User Tools

Site Tools

Trace:
documentation:examples:ecmp

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
Last revisionBoth sides next revision
documentation:examples:ecmp [2020/01/14 01:23] – [VM 5 (ECMP router)] olivierdocumentation:examples:ecmp [2022/12/11 18:40] – [Checking routes installed] olivier
Line 1: Line 1:
 ====== Equal-cost multi-path routing (ECMP) ====== ====== Equal-cost multi-path routing (ECMP) ======
  
-This lab show an ECMP (feature called RADIX_MPATH on FreeBSD) routing example. +This lab need a FreeBSD 13-head r366390 minimum ([[https://reviews.freebsd.org/D26449|new ECMP routing code ]])
 ===== Presentation ===== ===== Presentation =====
  
 +<note warning>Bhyve doesn't support emulating multiqueue NIC, so RSS flow-id could not be tested using a bhyve based lab: Had to use physical lab</note>
 ==== Network diagram ==== ==== Network diagram ====
  
 Here is the logical and physical view: Here is the logical and physical view:
  
-{{:documentation:examples:labs.examples.ecmp.png|}}+{{:documentation:examples:bsdrp-lab-mpath.png|}}
  
 ===== Setting-up the lab ===== ===== Setting-up the lab =====
Line 15: Line 15:
 ==== Downloading BSD Router Project images ==== ==== Downloading BSD Router Project images ====
  
-Download BSDRP serial image (prevent to have to use an X display) on Sourceforge.+Download BSDRP serial image on Sourceforge and upload them to the 2 ECMP routers.
  
-==== Download Lab scripts ===== 
- 
-More information on these BSDRP lab scripts available on [[documentation:examples:How to build a BSDRP router lab]]. 
- 
-Start the lab with 6 routers. 
-With bhyve under FreeBSD: 
-<code> 
-user:~ # BSDRP-lab-bhyve.sh -i BSDRP.amd64/BSDRP-1.71-full-amd64-serial.img.xz -n 6 
-BSD Router Project (http://bsdrp.net) - bhyve full-meshed lab script 
-Setting-up a virtual lab with 6 VM(s): 
-- Working directory: /tmp/BSDRP 
-- Each VM have 1 core(s) and 256M RAM 
-- Switch mode: bridge + tap 
-- 0 LAN(s) between all VM 
-- Full mesh Ethernet links between each VM 
-VM 1 have the following NIC: 
-- vtnet0 connected to VM 2. 
-- vtnet1 connected to VM 3. 
-- vtnet2 connected to VM 4. 
-- vtnet3 connected to VM 5. 
-- vtnet4 connected to VM 6. 
-VM 2 have the following NIC: 
-- vtnet0 connected to VM 1. 
-- vtnet1 connected to VM 3. 
-- vtnet2 connected to VM 4. 
-- vtnet3 connected to VM 5. 
-- vtnet4 connected to VM 6. 
-VM 3 have the following NIC: 
-- vtnet0 connected to VM 1. 
-- vtnet1 connected to VM 2. 
-- vtnet2 connected to VM 4. 
-- vtnet3 connected to VM 5. 
-- vtnet4 connected to VM 6. 
-VM 4 have the following NIC: 
-- vtnet0 connected to VM 1. 
-- vtnet1 connected to VM 2. 
-- vtnet2 connected to VM 3. 
-- vtnet3 connected to VM 5. 
-- vtnet4 connected to VM 6. 
-VM 5 have the following NIC: 
-- vtnet0 connected to VM 1. 
-- vtnet1 connected to VM 2. 
-- vtnet2 connected to VM 3. 
-- vtnet3 connected to VM 4. 
-- vtnet4 connected to VM 6. 
-VM 6 have the following NIC: 
-- vtnet0 connected to VM 1. 
-- vtnet1 connected to VM 2. 
-- vtnet2 connected to VM 3. 
-- vtnet3 connected to VM 4. 
-- vtnet4 connected to VM 5. 
-For connecting to VM'serial console, you can use: 
-- VM 1 : cu -l /dev/nmdm1B 
-- VM 2 : cu -l /dev/nmdm2B 
-- VM 3 : cu -l /dev/nmdm3B 
-- VM 4 : cu -l /dev/nmdm4B 
-- VM 5 : cu -l /dev/nmdm5B 
-- VM 6 : cu -l /dev/nmdm6B 
-</code> 
  
 ===== Static routing setup ===== ===== Static routing setup =====
  
-==== VM 1 (client) ====+==== Client ====
  
-VM1 is configured as a simple host with multiples aliases:+A simple host with static routes:
  
 <code> <code>
-sysrc hostname=VM1 \ +sysrc hostname=client \ 
- gateway_enable=NO \ +  gateway_enable=NO \ 
- ipv6_gateway_enable=NO \ +  ipv6_gateway_enable=NO \ 
- ifconfig_vtnet0="inet 10.0.12.1/24" \ +  ifconfig_igb1="inet 10.0.12.1/24" \ 
- ifconfig_vtnet0_alias1="inet 10.0.12.11/32" \ +  ifconfig_igb1_ipv6="inet6 2001:db8:12::3 prefixlen 64" \ 
- ifconfig_vtnet0_alias2="inet 10.0.12.21/32" \ +  static_routes="LAB" \ 
- ifconfig_vtnet0_alias3="inet 10.0.12.31/32" \ +  route_LAB="-net 10.0.0.0/16 10.0.12.2" \ 
- ifconfig_vtnet0_alias4="inet 10.0.12.41/32" \ +  ipv6_static_routes="LAB" \ 
- ifconfig_vtnet0_alias5="inet 10.0.12.51/32" \ +  ipv6_route_LAB="2001:db8:: -prefixlen 32 2001:db8:12::2"
- ifconfig_vtnet0_ipv6="inet6 2001:db8:12::1 prefixlen 64" \ +
- ifconfig_vtnet0_alias61="inet6 2001:db8:12::11 prefixlen 64" \ +
- ifconfig_vtnet0_alias62="inet6 2001:db8:12::21 prefixlen 64" \ +
- ifconfig_vtnet0_alias63="inet6 2001:db8:12::31 prefixlen 64" \ +
- ifconfig_vtnet0_alias64="inet6 2001:db8:12::41 prefixlen 64" \ +
- ifconfig_vtnet0_alias65="inet6 2001:db8:12::51 prefixlen 64" \ +
- defaultrouter="10.0.12.2" \ +
- ipv6_defaultrouter="2001:db8:12::2"+
 service hostname restart service hostname restart
 service netif restart service netif restart
Line 107: Line 40:
 </code> </code>
  
-==== VM 2 (ECMP router) ====+==== R1 (ECMP router) ====
  
-VM 2 is a router with ECMP: 2 static routes toward the same destination but using 2 different next-hop. +R1 is a router with ECMP: 2 static routes toward the same destination but using 2 different next-hop.
-Need to disable fastforward by re-enabling ICMP redirect (disabled on BSDRP by default).+
  
 <code> <code>
-sysrc hostname=VM2 \ + 
- ifconfig_vtnet0="inet 10.0.12.2/24" \ +sysrc hostname=R1 \ 
- ifconfig_vtnet1="inet 10.0.23.2/24" \ +  gateway_enable=YES \ 
- ifconfig_vtnet2="inet 10.0.24.2/24" \ +  ipv6_gateway_enable=YES \ 
- ifconfig_vtnet0_ipv6="inet6 2001:db8:12::2 prefixlen 64" \ +  ifconfig_igb0="inet 10.0.12.2/24" \ 
- ifconfig_vtnet1_ipv6="inet6 2001:db8:23::2 prefixlen 64" \ +  ifconfig_igb0_ipv6="inet6 2001:db8:12::2 prefixlen 64" \ 
- ifconfig_vtnet2_ipv6="inet6 2001:db8:24::2 prefixlen 64" \ +  ifconfig_igb1="inet 10.0.231.2/24" \ 
- static_routes="VM3VM6 VM3VM5 VM4VM6 VM4VM5" \ +  ifconfig_igb1_ipv6="inet6 2001:db8:231::2 prefixlen 64" \ 
- route_VM3VM6="-net 10.0.56.0/24 10.0.23.3" \ +  ifconfig_igb2="inet 10.0.232.2/24" \ 
- route_VM3VM5="-net 10.0.35.0/24 10.0.23.3" \ +  ifconfig_igb2_ipv6="inet6 2001:db8:232::2 prefixlen 64" \ 
- route_VM4VM6="-net 10.0.56.0/24 10.0.24.4" \ +  static_routes="MPATH1 MPATH2" \ 
- route_VM4VM5="-net 10.0.45.0/24 10.0.24.4" \ +  route_MPATH1="-net 10.0.0.0/16 10.0.231.3" \ 
- ipv6_static_routes="VM3VM6 VM3VM5 VM4VM6 VM4VM5" \ +  route_MPATH2="-net 10.0.0.0/16 10.0.232.3" \ 
- ipv6_route_VM3VM6="2001:db8:56:: -prefixlen 64 2001:db8:23::3" \ +  ipv6_static_routes="MPATH1 MPATH2" \ 
- ipv6_route_VM3VM5="2001:db8:35:: -prefixlen 64 2001:db8:23::3" \ +  ipv6_route_MPATH1="2001:db8:: -prefixlen 32 2001:db8:231::3" \ 
- ipv6_route_VM4VM6="2001:db8:56:: -prefixlen 64 2001:db8:24::4" \ +  ipv6_route_MPATH2="2001:db8:: -prefixlen 32 2001:db8:232::3"
- ipv6_route_VM4VM5="2001:db8:45:: -prefixlen 64 2001:db8:24::4" +
-sed -i '' -e '/net.inet.ip.redirect=0/d' /etc/sysctl.conf +
-sed -i '' -e '/net.inet6.ip6.redirect=0/d' /etc/sysctl.conf +
-sysctl net.inet.ip.redirect=1 +
-sysctl net.inet6.ip6.redirect=1+
 service hostname restart service hostname restart
 service netif restart service netif restart
Line 140: Line 67:
 </code> </code>
  
-And first problem using static route:+Checking static route with multiple next-hop:
  
 <code> <code>
-add net 10.0.56.0: gateway 10.0.23.3 fib 0 +root@R1:~ # netstat -rn4 | grep 10.0.0.0/16 
-add net 10.0.35.0: gateway 10.0.23.3 fib 0 +10.0.0.0/16        10.0.231.3         UGS        igb1 
-add net 10.0.56.0: gateway 10.0.24.4 fib 0: route already in table +10.0.0.0/16        10.0.232.3         UGS        igb2 
-add net 10.0.45.0: gateway 10.0.24.4 fib 0 + 
-add net 2001:db8:56::: gateway 2001:db8:23::3 fib 0 +root@R1:~ # netstat -4onW 
-add net 2001:db8:35::: gateway 2001:db8:23::3 fib 0 +Nexthop data 
-add net 2001:db8:56::: gateway 2001:db8:24::4 fib 0: route already in table + 
-add net 2001:db8:45::: gateway 2001:db8:24::4 fib 0 +Internet: 
-[root@VM2]~# route get 10.0.56.0/24 +Idx   Type         IFA                Gateway             Flags      Use Mtu         Netif     Addrif Refcnt Prepend 
-   route to: 10.0.56.0 +1       v4/resolve 127.0.0.1          lo0/resolve        H             0  16384        lo0               2 
-destination: 10.0.56.0 +2       v4/resolve 10.0.12.2          igb0/resolve                     0   1500       igb0               2 
-       mask: 255.255.255.0 +3       v4/resolve 127.0.0.1          lo0/resolve        HS            0  16384        lo0      igb0     2 
-    gateway: 10.0.23.3 +4       v4/resolve 10.0.231.2         igb1/resolve                     0   1500       igb1               2 
-        fib: 0 +5       v4/resolve 127.0.0.1          lo0/resolve        HS            0  16384        lo0      igb1     2 
-  interface: vtnet1 +6       v4/resolve 10.0.232.2         igb2/resolve                     0   1500       igb2               2 
-      flags: <UP,GATEWAY,DONE,STATIC> +7       v4/resolve 127.0.0.1          lo0/resolve        HS            0  16384        lo0      igb2     2 
- recvpipe  sendpipe  ssthresh  rtt,msec    mtu        weight    expire +8            v4/gw 10.0.231.2         10.0.231.3         GS            0   1500       igb1               1 
-       0         0         0         0      1500         1         0+9            v4/gw 10.0.232.2         10.0.232.3         GS            0   1500       igb2               1 
 + 
 +root@R1:~ # netstat -6onW 
 +Nexthop data 
 + 
 +Internet6: 
 +Idx   Type         IFA                           Gateway                        Flags      Use Mtu       Netif   Addrif Refcnt Prepend 
 +1       v6/resolve ::1                           lo0/resolve                   HS            0  16384      lo0             2 
 +2       v6/resolve fe80::1%lo0                   lo0/resolve                   HS            0  16384      lo0             2 
 +3       v6/resolve fe80::1%lo0                   lo0/resolve                                 0  16384      lo0             2 
 +4       v6/resolve ::1                           lo0/resolve                   HS            0  16384      lo0    igb0     3 
 +5       v6/resolve fe80::20d:b9ff:fe41:ca3c%igb0 igb0/resolve                                0   1500     igb0             3 
 +6       v6/resolve ::1                           lo0/resolve                   HS            0  16384      lo0    igb1     3 
 +7       v6/resolve fe80::20d:b9ff:fe41:ca3d%igb1 igb1/resolve                                0   1500     igb1             3 
 +8       v6/resolve ::1                           lo0/resolve                   HS            0  16384      lo0    igb2     3 
 +9       v6/resolve fe80::20d:b9ff:fe41:ca3e%igb2 igb2/resolve                                0   1500     igb2             3 
 +10           v6/gw ::1                           ::1                           GRS           0  16384      lo0             5 
 +11           v6/gw 2001:db8:231::2               2001:db8:231::3               GS            0   1500     igb1             1 
 +12           v6/gw 2001:db8:232::2               2001:db8:232::3               GS            0   1500     igb2             1
 </code> </code>
-==== VM 3 (router) ====+==== R2 (ECMP router) ====
  
-VM 3 is a simple router.+R2 is like R1, a router with ECMP: 2 static routing toward the same destination but using 2 different next-hop..
  
 <code> <code>
-sysrc hostname=VM3 \ +sysrc hostname=R2 \ 
- ifconfig_vtnet1="inet 10.0.23.3/24" \ +  gateway_enable=YES \ 
- ifconfig_vtnet3="inet 10.0.35.3/24" \ +  ipv6_gateway_enable=YES \ 
- ifconfig_vtnet1_ipv6="inet6 2001:db8:23::3 prefixlen 64" \ +  ifconfig_igb0="inet 10.0.34.3/24" \ 
- ifconfig_vtnet3_ipv6="inet6 2001:db8:35::3 prefixlen 64" \ +  ifconfig_igb0_ipv6="inet6 2001:db8:34::3 prefixlen 64" \ 
- static_routes="VM6 VM1" \ +  ifconfig_igb1="inet 10.0.231.3/24" \ 
- route_VM1="-net 10.0.12.0/24 10.0.23.2" \ +  ifconfig_igb1_ipv6="inet6 2001:db8:231::3 prefixlen 64" \ 
- route_VM6="-net 10.0.56.0/24 10.0.35.5" \ +  ifconfig_igb2="inet 10.0.232.3/24" \ 
- ipv6_static_routes="VM6 VM1" \ +  ifconfig_igb2_ipv6="inet6 2001:db8:232::3 prefixlen 64" \ 
- ipv6_route_VM1="2001:db8:12:: -prefixlen 64 2001:db8:23::2" \ +  static_routes="DST1 DST2 SRC1 SRC2" \ 
- ipv6_route_VM6="2001:db8:56:: -prefixlen 64 2001:db8:35::5"+  route_DST1="-net 10.0.0.0/16 10.0.231.4" \ 
 +  route_DST2="-net 10.0.0.0/16 10.0.232.4" \ 
 +  route_SRC1="-net 10.0.12.0/24 10.0.231.2" \ 
 +  route_SRC2="-net 10.0.12.0/24 10.0.232.2" \ 
 +  ipv6_static_routes="DST1 DST2 SRC1 SRC2" \ 
 +  ipv6_route_MPATH1="2001:db8:: -prefixlen 32 2001:db8:231::4" \ 
 +  ipv6_route_MPATH2="2001:db8:: -prefixlen 32 2001:db8:231::4" \ 
 +  ipv6_route_SRC1="2001:db8:12:: -prefixlen 64 2001:db8:231::2" \ 
 +  ipv6_route_SRC2="2001:db8:12:: -prefixlen 64 2001:db8:231::2"
 service hostname restart service hostname restart
 service netif restart service netif restart
Line 183: Line 136:
 config save config save
 </code> </code>
 +==== Server ====
  
-==== VM 4 (router) ==== +A simple host with some static routes:
- +
-VM 4 is like VM 3, a simple router.+
  
 <code> <code>
-sysrc hostname=VM4 \ +sysrc hostname=server \ 
- ifconfig_vtnet1="inet 10.0.24.4/24" \ +  gateway_enable=NO \ 
- ifconfig_vtnet3="inet 10.0.45.4/24" \ +  ipv6_gateway_enable=NO \ 
- ifconfig_vtnet1_ipv6="inet6 2001:db8:24::4 prefixlen 64" \ +  ifconfig_igb1="inet 10.0.34.4/24" \ 
- ifconfig_vtnet3_ipv6="inet6 2001:db8:45::4 prefixlen 64" \ +  ifconfig_igb1_ipv6="inet6 2001:db8:34::4 prefixlen 64" \ 
- static_routes="VM6 VM1" \ +  static_routes="12 231 232" \ 
- route_VM1="-net 10.0.12.0/24 10.0.24.2" \ +  route_12="-net 10.0.12.0/24 10.0.34.3" \ 
- route_VM6="-net 10.0.56.0/24 10.0.45.5" \ +  route_231="-net 10.0.231.0/24 10.0.34.3" \ 
- ipv6_static_routes="VM6 VM1" \ +  route_232="-net 10.0.232.0/24 10.0.34.3" \ 
- ipv6_route_VM1="2001:db8:12:: -prefixlen 64 2001:db8:24::2" \ +  ipv6_static_routes="12 231 232" \ 
- ipv6_route_VM6="2001:db8:56:: -prefixlen 64 2001:db8:45::5"+  ipv6_route_12="2001:db8:12:: -prefixlen 64 2001:db8:34::3" \ 
 +  ipv6_route_231="2001:db8:231:: -prefixlen 64 2001:db8:34::3" \ 
 +  ipv6_route_232="2001:db8:232:: -prefixlen 64 2001:db8:34::3"
 service hostname restart service hostname restart
 service netif restart service netif restart
Line 206: Line 160:
 </code> </code>
  
-==== VM 5 (ECMP router) ====+===== FRR Multipath setup =====
  
-VM 5 is like VM 2, a router with ECMP: 2 static routing toward the same destination but using 2 different next-hop..+Replacing static routes by FRR (OSPF) compiled with MULTIPATH option. 
 + 
 +==== R1 (ECMP router) ==== 
 + 
 +In place of static routes, OSPF with FRR is used:
  
 <code> <code>
-sysrc hostname=VM5 \ +sysrc frr_vtysh_boot="YES" \ 
- ifconfig_vtnet2="inet 10.0.35.5/24" \ +  frr_enable="YES" \ 
- ifconfig_vtnet3="inet 10.0.45.5/24" \ +  frr_daemons="zebra ospfd ospf6d" \ 
- ifconfig_vtnet4="inet 10.0.56.5/24" \ +  watchfrr_flags=" -d -r /usr/sbin/servicebBfrrbBrestartbB%s -s /usr/sbin/servicebBfrrbBstartbB%s -k /usr/sbin/servicebBfrrbBstopbB%s -b bB -t 30 zebra ospfd ospf6d" \ 
- ifconfig_vtnet2_ipv6="inet6 2001:db8:35::5 prefixlen 64" \ +  watchfrr_enable="YES" 
- ifconfig_vtnet3_ipv6="inet6 2001:db8:45::5 prefixlen 64" \ + 
- ifconfig_vtnet4_ipv6="inet6 2001:db8:56::5 prefixlen 64" \ +cat > /usr/local/etc/frr/frr.conf <EOF 
- static_routes="VM3VM1 VM3VM2 VM4VM1 VM4VM2" \ +frr version 8.4.1 
- route_VM3VM1="-net 10.0.12.0/24 10.0.35.3" \ +frr defaults traditional 
- route_VM3VM2="-net 10.0.23.0/24 10.0.35.3" \ +hostname R1 
- route_VM4VM1="-net 10.0.12.0/24 10.0.45.4" \ +! 
- route_VM4VM2="-net 10.0.24.0/24 10.0.45.4" \ +interface igb0 
- ipv6_static_routes="VM3VM1 VM3VM2 VM4VM1 VM4VM2" \ + ip ospf passive 
- ipv6_route_VM3VM1="2001:db8:12:: -prefixlen 64 2001:db8:35::3" \ + ipv6 ospf6 area 0.0.0.0 
- ipv6_route_VM3VM2="2001:db8:23:: -prefixlen 64 2001:db8:35::3" \ + ipv6 ospf6 passive 
- ipv6_route_VM4VM1="2001:db8:12:: -prefixlen 64 2001:db8:45::4" \ +exit 
- ipv6_route_VM4VM2="2001:db8:24:: -prefixlen 64 2001:db8:45::4" +! 
-sed -i '' -e '/net.inet.ip.redirect=0/d' /etc/sysctl.conf +interface igb1 
-sed -i '' -e '/net.inet6.ip6.redirect=0/d' /etc/sysctl.conf + ipv6 ospf6 area 0.0.0.0 
-sysctl net.inet.ip.redirect=1 +exit 
-sysctl net.inet6.ip6.redirect=1 +! 
-service hostname restart +interface igb2 
-service netif restart + ipv6 ospf6 area 0.0.0.0 
-service routing restart +exit 
-config save+! 
 +router ospf 
 + ospf router-id 1.1.1.1 
 + network 10.0.12.0/24 area 0 
 + network 10.0.231.0/24 area 0 
 + network 10.0.232.0/24 area 0 
 +exit 
 +! 
 +router ospf6 
 +exit 
 +! 
 +EOF 
 +service frr start 
 +service watchfrr start
 </code> </code>
  
-==== VM 6 (server) ====+==== R2 (ECMP router) ====
  
-VM 6, like VM 1 is configured as a simple host with multiple aliases:+Same as R1 with OSPF and FRR: 
 +<code> 
 +sysrc frr_vtysh_boot="YES" \ 
 +  frr_enable="YES" \ 
 +  frr_daemons="zebra staticd ospfd ospf6d" \ 
 +  watchfrr_flags=" -d -r /usr/sbin/servicebBfrrbBrestartbB%s -s /usr/sbin/servicebBfrrbBstartbB%s -k /usr/sbin/servicebBfrrbBstopbB%s -b bB -t 30 zebra ospfd ospf6d" \ 
 +  watchfrr_enable="YES"
  
 +cat > /usr/local/etc/frr/frr.conf <EOF
 +frr version 8.4.1
 +frr defaults traditional
 +hostname R2
 +!
 +ip route 10.0.0.0/16 10.0.34.4
 +ipv6 route 2001:db8::/32 2001:db8:231::4
 +!
 +interface igb0
 + ip ospf passive
 + ipv6 ospf6 area 0.0.0.0
 + ipv6 ospf6 passive
 +exit
 +!
 +interface igb1
 + ipv6 ospf6 area 0.0.0.0
 +exit
 +!
 +interface igb2
 + ipv6 ospf6 area 0.0.0.0
 +exit
 +!
 +router ospf
 + ospf router-id 2.2.2.2
 + redistribute static
 + network 10.0.34.0/24 area 0
 + network 10.0.231.0/24 area 0
 + network 10.0.232.0/24 area 0
 +exit
 +!
 +router ospf6
 + redistribute static
 +exit
 +!
 +EOF
 +service frr start
 +service watchfrr start
 +</code>
 +
 +==== Checking routes installed ====
 +
 +On R1:
 <code> <code>
-sysrc hostname=VM6 +root@R1:~ # vtysh 
-sysrc gateway_enable=NO +Hello, this is FRRouting (version 8.4.1). 
-sysrc ipv6_gateway_enable=NO +Copyright 1996-2005 Kunihiro Ishiguro, et al. 
-sysrc ifconfig_vtnet4="inet 10.0.56.6/24" + 
-sysrc ifconfig_vtnet4_alias1="inet 10.0.56.16/32" +R1# sh ip route 10.0.0.0 
-sysrc ifconfig_vtnet4_alias2="inet 10.0.56.26/32" +Routing entry for 10.0.0.0/16 
-sysrc ifconfig_vtnet4_alias3="inet 10.0.56.36/32" +  Known via "ospf", distance 110, metric 20, best 
-sysrc ifconfig_vtnet4_alias4="inet 10.0.56.46/32" +  Last update 00:02:26 ago 
-sysrc ifconfig_vtnet4_alias5="inet 10.0.56.56/32" +  * 10.0.231.3, via igb1, weight 1 
-sysrc ifconfig_vtnet4_ipv6="inet6 2001:db8:56::6 prefixlen 64" +  * 10.0.232.3, via igb2, weight 1 
-sysrc ifconfig_vtnet4_alias61="inet6 2001:db8:56::16 prefixlen 64" + 
-sysrc ifconfig_vtnet4_alias62="inet6 2001:db8:56::26 prefixlen 64" +R1# sh ipv6 route 2001:db8:: 
-sysrc ifconfig_vtnet4_alias63="inet6 2001:db8:56::36 prefixlen 64" +Routing entry for 2001:db8::/32 
-sysrc ifconfig_vtnet4_alias64="inet6 2001:db8:56::46 prefixlen 64" +  Known via "ospf6", distance 110, metric 20, best 
-sysrc ifconfig_vtnet4_alias65="inet6 2001:db8:56::56 prefixlen 64" +  Last update 00:02:39 ago 
-sysrc defaultrouter="10.0.56.5" +  * fe80::20d:b9ff:fe45:7ad5, via igb1, weight 1 
-sysrc ipv6_defaultrouter="2001:db8:56::5" +  * fe80::20d:b9ff:fe45:7ad6, via igb2, weight 1
-service hostname restart +
-service netif restart +
-service routing restart +
-config save+
 </code> </code>
  
-==== Testing Load balancing ====+===== Test Load balancing IP packets =====
  
-Flows from VM1 to VM6 should be load-balanced by VM2 toward VM3 and VM4. Let's check using multiple sources and destination IP addresses. +Flows from the client to the server should be "flow-id shared" between the 2 paths. Let's check using multiple sources and destination IP addresses using pkt-gen on client and server, then using systat on R1 and R2 to check their load-distribution. 
-Using ping record feature like with this example:+ 
 +On server:
 <code> <code>
-[root@VM1]~# ping -c1 -R -S 10.0.12.21 10.0.56.26 +root@server:~ # pkt-gen -i igb1 -f rx 
-PING 10.0.56.26 (10.0.56.26) from 10.0.12.21: 56 data bytes +</code>
-64 bytes from 10.0.56.26: icmp_seq=0 ttl=61 time=0.358 ms +
-RR:     10.0.23.2 +
-        10.0.35.3 +
-        10.0.56.5 +
-        10.0.56.26 +
-        10.0.35.5 +
-        10.0.23.3 +
-        10.0.12.2 +
-        10.0.12.21+
  
---- 10.0.56.26 ping statistics --- +On client: 
-1 packets transmitted, 1 packets received, 0.0% packet loss +<code> 
-round-trip min/avg/max/stddev = 0.358/0.358/0.358/0.000 ms+root@client:~ # pkt-gen -i igb1 -f tx -n 8000000 -l 60 -d 10.0.255.1:2000-10.0.255.254 -D 00:0d:b9:41:ca:3c -s 10.0.254.1:2000-10.0.254.254 -S 00:0d:b9:45:7f:b0 -w 4 -R 20000
 </code> </code>
  
-^ Source ^ Destination ^ Ongoing path selected by VM2 ^ Returning path selected by VM5^ +On R1: 
-| 10.0.12.1 | 10.0.56.6 | VM3 | VM3 | +<code> 
-| 10.0.12.11 | 10.0.56.16 | VM3 | VM3 | +systat -ifstat -match igb0,igb1,igb2 -pps 
-| 10.0.12.21 | 10.0.56.26 | VM3 | VM3 | + 
-| 10.0.12.31 | 10.0.56.36 | VM3 | VM3 | +                    /0   /1   /2   /3   /4   /5   /6   /7   /8   /9   /10 
-| 10.0.12.41 | 10.0.56.46 | VM3 | VM3 | +     Load Average   | 
-| 10.0.12.51 | 10.0.56.56 | VM3 | VM3 | + 
-| 2001:db8:12::1 | 2001:db8:56::6 | VM3 | VM3 | +      Interface           Traffic               Peak                Total 
-| 2001:db8:12::11 | 2001:db8:56::16 | VM3 | VM3 | +           igb2  in      0.000 Kp/s          0.000 Kp/s           71.247 Mp 
-| 2001:db8:12::21 | 2001:db8:56::26 | VM3 | VM3 | +                 out     9.762 Kp/s          9.777 Kp/s           76.892 Mp 
-| 2001:db8:12::31 | 2001:db8:56::36 | VM3 | VM3 | + 
-| 2001:db8:12::41 | 2001:db8:56::46 | VM3 | VM3 | +           igb1  in      0.000 Kp/s          0.000 Kp/s           71.392 Mp 
-| 2001:db8:12::51 | 2001:db8:56::56 | VM3 | VM3 | +                 out     9.770 Kp/s          9.771 Kp/s           80.341 Mp 
-| 10.0.12.1 | 10.0.45.5 | VM4 | VM3 |+ 
 +           igb0  in     19.533 Kp/s         19.534 Kp/s           90.007 Mp 
 +                 out     0.000 Kp/s          0.000 Kp/s            0.243 Kp 
 +                  
 +</code> 
 + 
 +=> We confirm that 20 Kps entering igb0 and are equally split by exiting by igb1 and igb2 
 + 
 +On R2: 
 +<code> 
 +systat -ifstat -match igb0,igb1,igb2 -pps 
 + 
 +                    /0   /1   /2   /3   /4   /5   /6   /7   /8   /9   /10 
 +     Load Average   | 
 + 
 +      Interface           Traffic               Peak                Total 
 +           igb2  in      9.768 Kp/s          9.771 Kp/s          300.830 Kp 
 +                 out     0.000 Kp/s          0.000 Kp/s            0.000 Kp 
 + 
 +           igb1  in      9.763 Kp/s          9.768 Kp/s          300.785 Kp 
 +                 out     0.000 Kp/s          0.000 Kp/s            0.006 Kp 
 + 
 +           igb0  in      0.000 Kp/s          0.001 Kp/s            0.240 Kp 
 +                 out    19.530 Kp/s         19.531 Kp/s          601.615 Kp 
 +</code>
  
-<note>There is no flow load-balancing with RADIX MPATH since FreeBSD 11, [[https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225792|seems broken with tryforward]], put proposed patch disable tryforward(fastforward) if RADIX_MPATH is enabled.</note>+=> R2 has no choice than receiving packets from igb1 and igb2, and forwarding them through igb0.
  
documentation/examples/ecmp.txt · Last modified: 2024/03/28 23:19 by olivier
Except where otherwise noted, content on this wiki is licensed under the following license: BSD 2-Clause
Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki