BSD Router Project

User Tools

Site Tools

Trace:
documentation:examples:ecmp

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
documentation:examples:ecmp [2022/12/11 17:56] – [Server] olivierdocumentation:examples:ecmp [2024/03/28 23:19] (current) – [Equal-cost multi-path routing (ECMP)] olivier
Line 1: Line 1:
 ====== Equal-cost multi-path routing (ECMP) ====== ====== Equal-cost multi-path routing (ECMP) ======
  
-This lab need a FreeBSD 13-head r366390 minimum ([[https://reviews.freebsd.org/D26449|new ECMP routing code ]]) 
 ===== Presentation ===== ===== Presentation =====
  
Line 22: Line 21:
 ==== Client ==== ==== Client ====
  
-A simple host with multiples aliases:+A simple host with static routes:
  
 <code> <code>
Line 29: Line 28:
   ipv6_gateway_enable=NO \   ipv6_gateway_enable=NO \
   ifconfig_igb1="inet 10.0.12.1/24" \   ifconfig_igb1="inet 10.0.12.1/24" \
-  ifconfig_igb1_alias1="inet 10.0.12.11/32" \ 
-  ifconfig_igb1_alias2="inet 10.0.12.21/32" \ 
-  ifconfig_igb1_alias3="inet 10.0.12.31/32" \ 
-  ifconfig_igb1_alias4="inet 10.0.12.41/32" \ 
-  ifconfig_igb1_alias5="inet 10.0.12.51/32" \ 
   ifconfig_igb1_ipv6="inet6 2001:db8:12::3 prefixlen 64" \   ifconfig_igb1_ipv6="inet6 2001:db8:12::3 prefixlen 64" \
-  ifconfig_igb1_alias61="inet6 2001:db8:12::11 prefixlen 64" \ 
-  ifconfig_igb1_alias62="inet6 2001:db8:12::21 prefixlen 64" \ 
-  ifconfig_igb1_alias63="inet6 2001:db8:12::31 prefixlen 64" \ 
-  ifconfig_igb1_alias64="inet6 2001:db8:12::41 prefixlen 64" \ 
-  ifconfig_igb1_alias65="inet6 2001:db8:12::51 prefixlen 64" \ 
   static_routes="LAB" \   static_routes="LAB" \
   route_LAB="-net 10.0.0.0/16 10.0.12.2" \   route_LAB="-net 10.0.0.0/16 10.0.12.2" \
Line 132: Line 121:
   ifconfig_igb2_ipv6="inet6 2001:db8:232::3 prefixlen 64" \   ifconfig_igb2_ipv6="inet6 2001:db8:232::3 prefixlen 64" \
   static_routes="DST1 DST2 SRC1 SRC2" \   static_routes="DST1 DST2 SRC1 SRC2" \
-  route_DST1="-net 10.0.0.0/16 10.0.231.3" \ +  route_DST1="-net 10.0.0.0/16 10.0.231.4" \ 
-  route_DST2="-net 10.0.0.0/16 10.0.232.3" \+  route_DST2="-net 10.0.0.0/16 10.0.232.4" \
   route_SRC1="-net 10.0.12.0/24 10.0.231.2" \   route_SRC1="-net 10.0.12.0/24 10.0.231.2" \
   route_SRC2="-net 10.0.12.0/24 10.0.232.2" \   route_SRC2="-net 10.0.12.0/24 10.0.232.2" \
   ipv6_static_routes="DST1 DST2 SRC1 SRC2" \   ipv6_static_routes="DST1 DST2 SRC1 SRC2" \
-  ipv6_route_MPATH1="2001:db8:: -prefixlen 32 2001:db8:231::3" \ +  ipv6_route_MPATH1="2001:db8:: -prefixlen 32 2001:db8:231::4" \ 
-  ipv6_route_MPATH2="2001:db8:: -prefixlen 32 2001:db8:231::3" \+  ipv6_route_MPATH2="2001:db8:: -prefixlen 32 2001:db8:231::4" \
   ipv6_route_SRC1="2001:db8:12:: -prefixlen 64 2001:db8:231::2" \   ipv6_route_SRC1="2001:db8:12:: -prefixlen 64 2001:db8:231::2" \
   ipv6_route_SRC2="2001:db8:12:: -prefixlen 64 2001:db8:231::2"   ipv6_route_SRC2="2001:db8:12:: -prefixlen 64 2001:db8:231::2"
Line 148: Line 137:
 ==== Server ==== ==== Server ====
  
-A simple host with multiple aliases:+A simple host with some static routes:
  
 <code> <code>
Line 155: Line 144:
   ipv6_gateway_enable=NO \   ipv6_gateway_enable=NO \
   ifconfig_igb1="inet 10.0.34.4/24" \   ifconfig_igb1="inet 10.0.34.4/24" \
-  ifconfig_igb1_alias1="inet 10.0.34.14/32" \ 
-  ifconfig_igb1_alias2="inet 10.0.34.24/32" \ 
-  ifconfig_igb1_alias3="inet 10.0.34.34/32" \ 
-  ifconfig_igb1_alias4="inet 10.0.34.44/32" \ 
-  ifconfig_igb1_alias5="inet 10.0.34.54/32" \ 
   ifconfig_igb1_ipv6="inet6 2001:db8:34::4 prefixlen 64" \   ifconfig_igb1_ipv6="inet6 2001:db8:34::4 prefixlen 64" \
-  ifconfig_igb1_alias61="inet6 2001:db8:34::14 prefixlen 64" \ 
-  ifconfig_igb1_alias62="inet6 2001:db8:34::24 prefixlen 64" \ 
-  ifconfig_igb1_alias63="inet6 2001:db8:34::34 prefixlen 64" \ 
-  ifconfig_igb1_alias64="inet6 2001:db8:34::44 prefixlen 64" \ 
-  ifconfig_igb1_alias65="inet6 2001:db8:34::54 prefixlen 64" \ 
   static_routes="12 231 232" \   static_routes="12 231 232" \
   route_12="-net 10.0.12.0/24 10.0.34.3" \   route_12="-net 10.0.12.0/24 10.0.34.3" \
Line 186: Line 165:
 ==== R1 (ECMP router) ==== ==== R1 (ECMP router) ====
  
-In place of static routes, FRR is enabled:+In place of static routes, OSPF with FRR is used:
  
 <code> <code>
Line 196: Line 175:
  
 cat > /usr/local/etc/frr/frr.conf <EOF cat > /usr/local/etc/frr/frr.conf <EOF
-frr version 7.4+frr version 8.4.1
 frr defaults traditional frr defaults traditional
-hostname homelab1+hostname R1
 ! !
 interface igb0 interface igb0
 + ip ospf passive
 + ipv6 ospf6 area 0.0.0.0
  ipv6 ospf6 passive  ipv6 ospf6 passive
 +exit
 +!
 +interface igb1
 + ipv6 ospf6 area 0.0.0.0
 +exit
 +!
 +interface igb2
 + ipv6 ospf6 area 0.0.0.0
 +exit
 ! !
 router ospf router ospf
  ospf router-id 1.1.1.1  ospf router-id 1.1.1.1
- passive-interface igb0 + network 10.0.12.0/24 area 0 
- network 10.0.31.0/24 area 0 + network 10.0.231.0/24 area 0 
- network 10.0.112.0/24 area 0 + network 10.0.232.0/24 area 0 
- network 10.0.212.0/24 area 0+exit
 ! !
 router ospf6 router ospf6
- interface igb0 area 0.0.0.0 +exit
- interface igb1 area 0.0.0.0 +
- interface igb2 area 0.0.0.0 +
-! +
-line vty+
 ! !
 EOF EOF
Line 224: Line 210:
 ==== R2 (ECMP router) ==== ==== R2 (ECMP router) ====
  
-In place of static routes, FRR is enabled:+Same as R1 with OSPF and FRR:
 <code> <code>
 sysrc frr_vtysh_boot="YES" \ sysrc frr_vtysh_boot="YES" \
   frr_enable="YES" \   frr_enable="YES" \
-  frr_daemons="zebra ospfd ospf6d" \+  frr_daemons="zebra staticd ospfd ospf6d" \
   watchfrr_flags=" -d -r /usr/sbin/servicebBfrrbBrestartbB%s -s /usr/sbin/servicebBfrrbBstartbB%s -k /usr/sbin/servicebBfrrbBstopbB%s -b bB -t 30 zebra ospfd ospf6d" \   watchfrr_flags=" -d -r /usr/sbin/servicebBfrrbBrestartbB%s -s /usr/sbin/servicebBfrrbBstartbB%s -k /usr/sbin/servicebBfrrbBstopbB%s -b bB -t 30 zebra ospfd ospf6d" \
   watchfrr_enable="YES"   watchfrr_enable="YES"
  
 cat > /usr/local/etc/frr/frr.conf <EOF cat > /usr/local/etc/frr/frr.conf <EOF
-frr version 7.4+frr version 8.4.1
 frr defaults traditional frr defaults traditional
-hostname labhome2+hostname R2 
 +! 
 +ip route 10.0.0.0/16 10.0.34.4 
 +ipv6 route 2001:db8::/32 2001:db8:231::4
 ! !
 interface igb0 interface igb0
 + ip ospf passive
 + ipv6 ospf6 area 0.0.0.0
  ipv6 ospf6 passive  ipv6 ospf6 passive
 +exit
 +!
 +interface igb1
 + ipv6 ospf6 area 0.0.0.0
 +exit
 +!
 +interface igb2
 + ipv6 ospf6 area 0.0.0.0
 +exit
 ! !
 router ospf router ospf
  ospf router-id 2.2.2.2  ospf router-id 2.2.2.2
- passive-interface igb0 + redistribute static 
- network 10.0.24.0/24 area 0 + network 10.0.34.0/24 area 0 
- network 10.0.112.0/24 area 0 + network 10.0.231.0/24 area 0 
- network 10.0.212.0/24 area 0+ network 10.0.232.0/24 area 0 
 +exit
 ! !
 router ospf6 router ospf6
- interface igb0 area 0.0.0.0 + redistribute static 
- interface igb1 area 0.0.0.0 +exit
- interface igb2 area 0.0.0.0 +
-! +
-line vty+
 ! !
 EOF EOF
Line 263: Line 261:
 On R1: On R1:
 <code> <code>
-[root@router1]~# vtysh +root@R1:~ # vtysh 
- +Hello, this is FRRouting (version 8.4.1).
-Hello, this is FRRouting (version 7.4).+
 Copyright 1996-2005 Kunihiro Ishiguro, et al. Copyright 1996-2005 Kunihiro Ishiguro, et al.
  
-router1# sh ip route 10.0.24.0/24 +R1# sh ip route 10.0.0.0 
-Routing entry for 10.0.24.0/24+Routing entry for 10.0.0.0/16
   Known via "ospf", distance 110, metric 20, best   Known via "ospf", distance 110, metric 20, best
-  Last update 00:06:39 ago +  Last update 00:02:26 ago 
-  * 10.0.112.2, via igb1, weight 1 +  * 10.0.231.3, via igb1, weight 1 
-  * 10.0.212.2, via igb2, weight 1+  * 10.0.232.3, via igb2, weight 1
  
-route1# sh ipv6 route 2001:db8:24::/64 +R1# sh ipv6 route 2001:db8:: 
-Routing entry for 2001:db8:24::/64+Routing entry for 2001:db8::/32
   Known via "ospf6", distance 110, metric 20, best   Known via "ospf6", distance 110, metric 20, best
-  Last update 00:02:42 ago+  Last update 00:02:39 ago
   * fe80::20d:b9ff:fe45:7ad5, via igb1, weight 1   * fe80::20d:b9ff:fe45:7ad5, via igb1, weight 1
   * fe80::20d:b9ff:fe45:7ad6, via igb2, weight 1   * fe80::20d:b9ff:fe45:7ad6, via igb2, weight 1
 +</code>
  
 +===== Test Load balancing IP packets =====
  
-router1# exit+Flows from the client to the server should be "flow-id shared" between the 2 paths. Let's check using multiple sources and destination IP addresses using pkt-gen on client and server, then using systat on R1 and R2 to check their load-distribution.
  
-[root@router1]~# netstat -rn4 | grep 10.0.24.0/24 +On server: 
-10.0.24.0/24       10.0.212.2         UG1        igb2 +<code> 
-10.0.24.0/24       10.0.112.2         UG1        igb1+root@server:~ # pkt-gen -i igb1 -f rx 
 +</code>
  
-[root@router1]~# netstat -rn6 | grep 2001:db8:24::/64 +On client: 
-2001:db8:24::/64                  fe80::20d:b9ff:fe45:7ad6%igb2 UG1        igb2 +<code> 
-2001:db8:24::/64                  fe80::20d:b9ff:fe45:7ad5%igb1 UG1        igb1+root@client:~ # pkt-gen -i igb1 -f tx -n 8000000 -l 60 -d 10.0.255.1:2000-10.0.255.254 -D 00:0d:b9:41:ca:3c -s 10.0.254.1:2000-10.0.254.254 -S 00:0d:b9:45:7f:b0 -w 4 -R 20000 
 +</code>
  
 +On R1:
 +<code>
 +systat -ifstat -match igb0,igb1,igb2 -pps
 +
 +                    /0   /1   /2   /3   /4   /5   /6   /7   /8   /9   /10
 +     Load Average   |
 +
 +      Interface           Traffic               Peak                Total
 +           igb2  in      0.000 Kp/s          0.000 Kp/s           71.247 Mp
 +                 out     9.762 Kp/s          9.777 Kp/s           76.892 Mp
 +
 +           igb1  in      0.000 Kp/s          0.000 Kp/s           71.392 Mp
 +                 out     9.770 Kp/s          9.771 Kp/s           80.341 Mp
 +
 +           igb0  in     19.533 Kp/s         19.534 Kp/s           90.007 Mp
 +                 out     0.000 Kp/s          0.000 Kp/s            0.243 Kp
 +                 
 </code> </code>
  
-===== Test Load balancing IP packets =====+=> We confirm that 20 Kps entering igb0 and are equally split by exiting by igb1 and igb2 
 + 
 +On R2: 
 +<code> 
 +systat -ifstat -match igb0,igb1,igb2 -pps 
 + 
 +                    /0   /1   /2   /3   /4   /5   /6   /7   /8   /9   /10 
 +     Load Average   | 
 + 
 +      Interface           Traffic               Peak                Total 
 +           igb2  in      9.768 Kp/s          9.771 Kp/s          300.830 Kp 
 +                 out     0.000 Kp/s          0.000 Kp/s            0.000 Kp 
 + 
 +           igb1  in      9.763 Kp/s          9.768 Kp/s          300.785 Kp 
 +                 out     0.000 Kp/s          0.000 Kp/s            0.006 Kp 
 + 
 +           igb0  in      0.000 Kp/s          0.001 Kp/s            0.240 Kp 
 +                 out    19.530 Kp/s         19.531 Kp/s          601.615 Kp 
 +</code>
  
-Flows from the client to the server should be "flow-id shared" between the 2 paths. Let's check using multiple sources and destination IP addresses.+=> R2 has no choice than receiving packets from igb1 and igb2, and forwarding them through igb0.
  
-^ Source ^ Destination ^ Ongoing path selected by R1 ^ Returning path selected by R2^ 
-| 10.0.31.3 | 10.0.24.4 | igb1 | igb2 | 
-| 10.0.31.3 | 10.0.24.14 | igb2 | igb1 | 
-| 10.0.31.13 | 10.0.24.14 | igb2 | igb1 | 
-| 10.0.31.23 | 10.0.24.24 | igb1 | igb2 | 
-| 10.0.31.33 | 10.0.24.34 | igb2 | igb2 | 
-| 10.0.31.43 | 10.0.24.44 | igb1 | igb2 | 
-| 10.0.31.53 | 10.0.24.54 | igb1 | igb1 | 
-| 2001:db8:31::3 | 2001:db8:24::4 | igb1 | igb1 | 
-| 2001:db8:31::13 | 2001:db8:24::14 | igb1 | igb2 | 
-| 2001:db8:31::23 | 2001:db8:24::24 | igb2 | igb2 | 
-| 2001:db8:31::33 | 2001:db8:24::34 | igb2 | igb1 | 
-| 2001:db8:31::43 | 2001:db8:24::44 | igb2 | igb1 | 
documentation/examples/ecmp.1670777796.txt.gz · Last modified: 2022/12/11 17:56 by olivier
Except where otherwise noted, content on this wiki is licensed under the following license: BSD 2-Clause
Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki