documentation:examples:ecmp
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revisionNext revisionBoth sides next revision | ||
documentation:examples:ecmp [2020/01/26 19:46] – [Testing Load balancing] olivier | documentation:examples:ecmp [2020/09/23 15:01] – [R2 (ECMP router)] olivier | ||
---|---|---|---|
Line 1: | Line 1: | ||
====== Equal-cost multi-path routing (ECMP) ====== | ====== Equal-cost multi-path routing (ECMP) ====== | ||
- | This lab is testing new ECMP routing code. | + | This lab is testing new ECMP routing code from [[https:// |
===== Presentation ===== | ===== Presentation ===== | ||
+ | <note warning> | ||
==== Network diagram ==== | ==== Network diagram ==== | ||
Here is the logical and physical view: | Here is the logical and physical view: | ||
- | {{: | + | {{: |
===== Setting-up the lab ===== | ===== Setting-up the lab ===== | ||
Line 15: | Line 15: | ||
==== Downloading BSD Router Project images ==== | ==== Downloading BSD Router Project images ==== | ||
- | Download BSDRP serial image (prevent to have to use an X display) | + | Download BSDRP serial image on Sourceforge |
- | ==== Download Lab scripts ===== | ||
- | |||
- | More information on these BSDRP lab scripts available on [[documentation: | ||
- | |||
- | Start the lab with 6 routers. | ||
- | With bhyve under FreeBSD: | ||
- | < | ||
- | user:~ # BSDRP-lab-bhyve.sh -i BSDRP.amd64/ | ||
- | BSD Router Project (http:// | ||
- | Setting-up a virtual lab with 6 VM(s): | ||
- | - Working directory: /tmp/BSDRP | ||
- | - Each VM have 1 core(s) and 256M RAM | ||
- | - Switch mode: bridge + tap | ||
- | - 0 LAN(s) between all VM | ||
- | - Full mesh Ethernet links between each VM | ||
- | VM 1 have the following NIC: | ||
- | - vtnet0 connected to VM 2. | ||
- | - vtnet1 connected to VM 3. | ||
- | - vtnet2 connected to VM 4. | ||
- | - vtnet3 connected to VM 5. | ||
- | - vtnet4 connected to VM 6. | ||
- | VM 2 have the following NIC: | ||
- | - vtnet0 connected to VM 1. | ||
- | - vtnet1 connected to VM 3. | ||
- | - vtnet2 connected to VM 4. | ||
- | - vtnet3 connected to VM 5. | ||
- | - vtnet4 connected to VM 6. | ||
- | VM 3 have the following NIC: | ||
- | - vtnet0 connected to VM 1. | ||
- | - vtnet1 connected to VM 2. | ||
- | - vtnet2 connected to VM 4. | ||
- | - vtnet3 connected to VM 5. | ||
- | - vtnet4 connected to VM 6. | ||
- | VM 4 have the following NIC: | ||
- | - vtnet0 connected to VM 1. | ||
- | - vtnet1 connected to VM 2. | ||
- | - vtnet2 connected to VM 3. | ||
- | - vtnet3 connected to VM 5. | ||
- | - vtnet4 connected to VM 6. | ||
- | VM 5 have the following NIC: | ||
- | - vtnet0 connected to VM 1. | ||
- | - vtnet1 connected to VM 2. | ||
- | - vtnet2 connected to VM 3. | ||
- | - vtnet3 connected to VM 4. | ||
- | - vtnet4 connected to VM 6. | ||
- | VM 6 have the following NIC: | ||
- | - vtnet0 connected to VM 1. | ||
- | - vtnet1 connected to VM 2. | ||
- | - vtnet2 connected to VM 3. | ||
- | - vtnet3 connected to VM 4. | ||
- | - vtnet4 connected to VM 5. | ||
- | For connecting to VM' | ||
- | - VM 1 : cu -l /dev/nmdm1B | ||
- | - VM 2 : cu -l /dev/nmdm2B | ||
- | - VM 3 : cu -l /dev/nmdm3B | ||
- | - VM 4 : cu -l /dev/nmdm4B | ||
- | - VM 5 : cu -l /dev/nmdm5B | ||
- | - VM 6 : cu -l /dev/nmdm6B | ||
- | </ | ||
===== Static routing setup ===== | ===== Static routing setup ===== | ||
- | ==== VM 1 (client) | + | ==== Client |
- | VM1 is configured as a simple host with multiples aliases: | + | A simple host with multiples aliases: |
< | < | ||
- | sysrc hostname=VM1 \ | + | sysrc hostname=client |
- | gateway_enable=NO \ | + | gateway_enable=NO \ |
- | ipv6_gateway_enable=NO \ | + | ipv6_gateway_enable=NO \ |
- | ifconfig_vtnet0="inet 10.0.12.1/24" \ | + | |
- | ifconfig_vtnet0_alias1="inet 10.0.12.11/32" \ | + | |
- | ifconfig_vtnet0_alias2="inet 10.0.12.21/32" \ | + | |
- | ifconfig_vtnet0_alias3="inet 10.0.12.31/ | + | |
- | ifconfig_vtnet0_alias4="inet 10.0.12.41/32" \ | + | |
- | ifconfig_vtnet0_alias5="inet 10.0.12.51/32" \ | + | |
- | ifconfig_vtnet0_ipv6=" | + | |
- | ifconfig_vtnet0_alias61=" | + | |
- | ifconfig_vtnet0_alias62=" | + | |
- | ifconfig_vtnet0_alias63=" | + | |
- | ifconfig_vtnet0_alias64=" | + | |
- | ifconfig_vtnet0_alias65=" | + | |
- | defaultrouter=" | + | |
- | ipv6_defaultrouter=" | + | route_LAB=" |
+ | | ||
+ | ipv6_route_LAB=" | ||
service hostname restart | service hostname restart | ||
service netif restart | service netif restart | ||
Line 107: | Line 50: | ||
</ | </ | ||
- | ==== VM 2 (ECMP router) ==== | + | ==== R1 (ECMP router) ==== |
- | VM 2 is a router with ECMP: 2 static routes toward the same destination but using 2 different next-hop. | + | R1 is a router with ECMP: 2 static routes toward the same destination but using 2 different next-hop. |
< | < | ||
- | sysrc hostname=VM2 \ | + | sysrc hostname=R1 \ |
- | ifconfig_vtnet0="inet 10.0.12.2/24" \ | + | |
- | ifconfig_vtnet1="inet 10.0.23.2/ | + | ipv6_gateway_enable=YES \ |
- | ifconfig_vtnet2="inet 10.0.24.2/24" \ | + | ifconfig_igb0="inet 10.0.31.1/24" \ |
- | ifconfig_vtnet0_ipv6=" | + | |
- | ifconfig_vtnet1_ipv6="inet6 2001: | + | |
- | ifconfig_vtnet2_ipv6=" | + | |
- | static_routes=" | + | |
- | route_VM3VM6="-net 10.0.56.0/24 10.0.23.3" | + | |
- | route_VM3VM5=" | + | static_routes=" |
- | route_VM4VM6="-net 10.0.56.0/24 10.0.24.4" | + | |
- | route_VM4VM5=" | + | |
- | ipv6_static_routes=" | + | ipv6_static_routes=" |
- | ipv6_route_VM3VM6=" | + | |
- | ipv6_route_VM3VM5=" | + | |
- | ipv6_route_VM4VM6=" | + | |
- | ipv6_route_VM4VM5=" | + | |
service hostname restart | service hostname restart | ||
service netif restart | service netif restart | ||
Line 138: | Line 79: | ||
< | < | ||
- | [root@VM2]~# netstat -rn4 | grep 10.0.56.0 | + | [root@R1]~# # netstat -rn4 | grep 10.0.0.0/16 |
- | 10.0.56.0/24 10.0.24.4 | + | 10.0.0.0/16 |
- | 10.0.56.0/24 10.0.23.3 | + | 10.0.0.0/16 |
- | [root@VM2]~# route -n get 10.0.56.0/24 | + | |
- | route to: 10.0.56.0 | + | [root@R1]~# route -n get 10.0.0.0/16 |
- | destination: | + | route to: 10.0.0.0 |
- | mask: 255.255.255.0 | + | destination: |
- | gateway: 10.0.24.4 | + | mask: 255.255.0.0 |
+ | gateway: 10.0.112.2 | ||
fib: 0 | fib: 0 | ||
- | interface: | + | interface: |
flags: < | flags: < | ||
| | ||
- | | + | |
- | [root@VM2]~# netstat -4onW | + | |
+ | [root@R1]~# netstat -4onW | ||
Nexthop data | Nexthop data | ||
Internet: | Internet: | ||
Idx | Idx | ||
- | 4 v4/ | + | 1 v4/ |
- | 6 v4/resolve 10.0.24.2 vtnet2/ | + | 2 v4/resolve 10.0.31.1 igb0/ |
- | 2 v4/ | + | 3 v4/ |
- | 1 v4/ | + | 4 v4/ |
- | 7 v4/resolve 127.0.0.1 | + | 5 v4/resolve 127.0.0.1 |
- | 5 v4/ | + | 6 v4/ |
- | 3 v4/resolve 127.0.0.1 | + | 7 v4/resolve 127.0.0.1 |
- | 9 v4/gw 10.0.24.2 | + | 8 v4/gw 10.0.112.1 10.0.112.2 GS |
- | 8 v4/gw 10.0.23.2 | + | 9 v4/gw 10.0.212.1 10.0.212.2 GS 0 |
- | Nexthop groups data | + | |
- | BUF: 192 | + | [root@R1]~# netstat -rn6 | grep 2001: |
+ | 2001: | ||
+ | 2001: | ||
+ | |||
+ | [root@R1]~# route -n6 get 2001:db8::/32 | ||
+ | route to: 2001: | ||
+ | destination: | ||
+ | mask: ffff: | ||
+ | gateway: 2001: | ||
+ | fib: 0 | ||
+ | interface: igb2 | ||
+ | flags: < | ||
+ | | ||
+ | | ||
- | Internet: | + | [root@R1]~# netstat -6onW |
- | MpIdx NHIdx Weigh Slots | + | Nexthop data |
- | 1 | + | |
- | | + | Internet6: |
- | 8 100 1 10.0.23.3 vtnet1 | + | Idx |
+ | 1 v6/resolve ::1 | ||
+ | 2 | ||
+ | 3 | ||
+ | 4 | ||
+ | 5 | ||
+ | 6 | ||
+ | 7 | ||
+ | 8 v6/resolve ::1 | ||
+ | 9 | ||
+ | 10 | ||
+ | 11 v6/resolve ::1 | ||
+ | 12 v6/gw ::1 :: | ||
+ | 13 v6/gw 2001: | ||
+ | 14 v6/gw 2001: | ||
+ | 15 v6/resolve 2001: | ||
</ | </ | ||
- | ==== VM 3 (router) ==== | ||
- | VM 3 is a simple router. | ||
- | < | + | Look like this router will prefer igb1 for inet4 and igb2 for inet6. |
- | sysrc hostname=VM3 \ | + | |
- | ifconfig_vtnet1=" | + | |
- | ifconfig_vtnet3=" | + | |
- | ifconfig_vtnet1_ipv6=" | + | |
- | ifconfig_vtnet3_ipv6=" | + | |
- | static_routes=" | + | |
- | route_VM1=" | + | |
- | route_VM6=" | + | |
- | ipv6_static_routes=" | + | |
- | ipv6_route_VM1=" | + | |
- | ipv6_route_VM6=" | + | |
- | service hostname restart | + | |
- | service netif restart | + | |
- | service routing restart | + | |
- | config save | + | |
- | </ | + | |
- | ==== VM 4 (router) ==== | + | ==== R5 (ECMP router) ==== |
- | VM 4 is like VM 3, a simple | + | R5 is like R2, a router |
< | < | ||
- | sysrc hostname=VM4 \ | + | sysrc hostname=R2 \ |
- | ifconfig_vtnet1="inet 10.0.24.4/24" \ | + | |
- | ifconfig_vtnet3="inet 10.0.45.4/24" \ | + | ipv6_gateway_enable=YES \ |
- | ifconfig_vtnet1_ipv6=" | + | ifconfig_igb0="inet 10.0.24.2/24" \ |
- | ifconfig_vtnet3_ipv6=" | + | |
- | static_routes=" | + | ifconfig_igb1="inet 10.0.112.2/24" \ |
- | route_VM1="-net 10.0.12.0/24 10.0.24.2" \ | + | |
- | route_VM6="-net 10.0.56.0/24 10.0.45.5" \ | + | |
- | ipv6_static_routes=" | + | ifconfig_igb2_ipv6=" |
- | ipv6_route_VM1=" | + | static_routes=" |
- | ipv6_route_VM6=" | + | |
+ | | ||
+ | ipv6_static_routes=" | ||
+ | | ||
+ | | ||
service hostname restart | service hostname restart | ||
service netif restart | service netif restart | ||
Line 218: | Line 175: | ||
</ | </ | ||
- | ==== VM 5 (ECMP router) ==== | + | And checking: |
+ | < | ||
+ | # route get 10.0.0.0/ | ||
+ | route to: 10.0.0.0 | ||
+ | destination: | ||
+ | mask: 255.255.0.0 | ||
+ | gateway: 10.0.212.1 | ||
+ | fib: 0 | ||
+ | interface: igb2 | ||
+ | flags: < | ||
+ | | ||
+ | | ||
- | VM 5 is like VM 2, a router with ECMP: 2 static routing toward the same destination but using 2 different next-hop.. | ||
- | < | + | # route -6 get 2001:db8::/32 |
- | sysrc hostname=VM5 \ | + | route to: 2001: |
- | ifconfig_vtnet2=" | + | destination: 2001: |
- | ifconfig_vtnet3=" | + | mask: ffff:ffff:: |
- | ifconfig_vtnet4=" | + | |
- | ifconfig_vtnet2_ipv6=" | + | fib: 0 |
- | ifconfig_vtnet3_ipv6=" | + | interface: igb1 |
- | ifconfig_vtnet4_ipv6=" | + | flags: < |
- | static_routes=" | + | recvpipe |
- | route_VM3VM1=" | + | 0 |
- | route_VM3VM2=" | + | |
- | route_VM4VM1=" | + | |
- | route_VM4VM2=" | + | |
- | ipv6_static_routes=" | + | |
- | ipv6_route_VM3VM1=" | + | |
- | ipv6_route_VM3VM2=" | + | |
- | ipv6_route_VM4VM1=" | + | |
- | ipv6_route_VM4VM2=" | + | |
- | service hostname restart | + | |
- | service netif restart | + | |
- | service routing restart | + | |
- | config save | + | |
</ | </ | ||
- | ==== VM 6 (server) | + | Look like this router will prefer to use igb2 for inet4 and igb1 for inet6. |
+ | |||
+ | ==== Server | ||
- | VM 6, like VM 1 is configured as a simple host with multiple aliases: | + | A simple host with multiple aliases: |
< | < | ||
- | sysrc hostname=VM6 \ | + | sysrc hostname=server |
- | gateway_enable=NO \ | + | gateway_enable=NO \ |
- | ipv6_gateway_enable=NO \ | + | ipv6_gateway_enable=NO \ |
- | ifconfig_vtnet4="inet 10.0.56.6/24" \ | + | |
- | ifconfig_vtnet4_alias1="inet 10.0.56.16/32" \ | + | |
- | ifconfig_vtnet4_alias2="inet 10.0.56.26/32" \ | + | |
- | ifconfig_vtnet4_alias3="inet 10.0.56.36/32" \ | + | |
- | ifconfig_vtnet4_alias4="inet 10.0.56.46/32" \ | + | |
- | ifconfig_vtnet4_alias5="inet 10.0.56.56/32" \ | + | |
- | ifconfig_vtnet4_ipv6=" | + | |
- | ifconfig_vtnet4_alias61=" | + | |
- | ifconfig_vtnet4_alias62=" | + | |
- | ifconfig_vtnet4_alias63=" | + | |
- | ifconfig_vtnet4_alias64=" | + | |
- | ifconfig_vtnet4_alias65=" | + | |
- | defaultrouter=" | + | |
- | ipv6_defaultrouter=" | + | route_LAB=" |
+ | | ||
+ | ipv6_route_LAB=" | ||
service hostname restart | service hostname restart | ||
service netif restart | service netif restart | ||
Line 274: | Line 234: | ||
</ | </ | ||
- | ==== Testing Load balancing | + | ===== FRR Multipath setup ===== |
- | Flows from VM1 to VM6 should be " | + | Replacing static routes |
- | Using ping record feature like with this example: | + | |
- | < | + | |
- | [root@VM1]~# | + | |
- | PING 10.0.56.26 | + | |
- | 64 bytes from 10.0.56.26: icmp_seq=0 ttl=61 time=0.455 ms | + | |
- | RR: | + | |
- | 10.0.45.4 | + | |
- | 10.0.56.5 | + | |
- | 10.0.56.26 | + | |
- | 10.0.45.5 | + | |
- | 10.0.24.4 | + | |
- | 10.0.12.2 | + | |
- | 10.0.12.21 | + | |
- | --- 10.0.56.26 ping statistics --- | + | ==== R1 (ECMP router) ==== |
- | 1 packets transmitted, | + | |
- | round-trip min/ | + | |
- | </ | + | |
- | ^ Source ^ Destination ^ Ongoing path selected by VM2 ^ Returning path selected by VM5^ | + | In place of static routes, FRR is enabled: |
- | | 10.0.12.1 | 10.0.35.5 | VM3 | VM4 | | + | |
- | | 10.0.12.1 | 10.0.45.5 | VM4 | VM4 | | + | |
- | | 10.0.12.1 | 10.0.56.6 | VM3 | VM4 | | + | |
- | | 10.0.12.11 | 10.0.56.16 | VM3 | VM4 | | + | |
- | | 10.0.12.21 | 10.0.56.26 | VM3 | VM4 | | + | |
- | | 10.0.12.31 | 10.0.56.36 | VM3 | VM4 | | + | |
- | | 10.0.12.41 | 10.0.56.46 | VM3 | VM4 | | + | |
- | | 10.0.12.51 | 10.0.56.56 | VM3 | VM4 | | + | |
- | | 2001:db8:12::1 | 2001: | + | |
- | | 2001: | + | |
- | | 2001: | + | |
- | | 2001: | + | |
- | | 2001: | + | |
- | | 2001: | + | |
- | |||
- | Notice inet6 bug: Cannot ping itself neither neighbor | ||
< | < | ||
- | [root@VM1]~# | + | sysrc frr_vtysh_boot=" |
- | vtnet0: flags=8943< | + | |
- | | + | |
- | inet6 fe80:: | + | |
- | inet6 2001: | + | |
- | inet6 2001: | + | |
- | inet6 2001: | + | |
- | inet6 2001: | + | |
- | inet6 2001: | + | |
- | inet6 2001: | + | |
- | nd6 options=21< | + | |
- | [root@VM1]~# | + | |
- | PING6(56=40+8+8 bytes) fe80:: | + | |
- | + | ||
- | --- fe80:: | + | |
- | 1 packets transmitted, | + | |
- | [root@VM1]~# | + | |
- | ping6: illegal number of packets | + | |
- | [root@VM1]~# | + | |
- | PING6(56=40+8+8 bytes) 2001: | + | |
- | --- 2001: | + | cat > / |
- | 1 packets transmitted, | + | frr version 7.4 |
+ | frr defaults traditional | ||
+ | hostname homelab1 | ||
+ | ! | ||
+ | interface igb0 | ||
+ | ipv6 ospf6 passive | ||
+ | ! | ||
+ | router ospf | ||
+ | ospf router-id 1.1.1.1 | ||
+ | | ||
+ | network 10.0.31.0/24 area 0 | ||
+ | | ||
+ | | ||
+ | ! | ||
+ | router ospf6 | ||
+ | | ||
+ | | ||
+ | | ||
+ | ! | ||
+ | line vty | ||
+ | ! | ||
+ | EOF | ||
+ | service frr start | ||
+ | service watchfrr start | ||
</ | </ | ||
- | ==== Testing Fail Over ==== | + | ==== R2 (ECMP router) |
- | VM4 is the preferred hop, so shutdown this VM to check behavior: | + | In place of static routes, FRR is enabled: |
+ | < | ||
+ | sysrc frr_vtysh_boot=" | ||
+ | frr_enable=" | ||
+ | frr_daemons=" | ||
+ | watchfrr_flags=" | ||
+ | watchfrr_enable=" | ||
- | <code> | + | cat > / |
- | [root@VM4]~# | + | frr version 7.4 |
- | Shutdown NOW! | + | frr defaults traditional |
+ | hostname labhome2 | ||
+ | ! | ||
+ | interface igb0 | ||
+ | ipv6 ospf6 passive | ||
+ | ! | ||
+ | router ospf | ||
+ | ospf router-id 2.2.2.2 | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | ! | ||
+ | router ospf6 | ||
+ | | ||
+ | | ||
+ | | ||
+ | ! | ||
+ | line vty | ||
+ | ! | ||
+ | EOF | ||
+ | service frr start | ||
+ | service watchfrr start | ||
</ | </ | ||
+ | ==== Checking routes installed ==== | ||
+ | |||
+ | On R1: | ||
< | < | ||
- | [root@VM2]~# netstat -4onW | + | [root@router1]~# vtysh |
- | Nexthop data | + | |
- | Internet: | + | Hello, this is FRRouting (version 7.4). |
- | Idx | + | Copyright 1996-2005 Kunihiro Ishiguro, et al. |
- | 4 v4/resolve 10.0.23.2 vtnet1/ | + | |
- | 6 | + | homelab1# sh ip route 10.0.24.0/24 |
- | 2 | + | Routing entry for 10.0.24.0/24 |
- | 1 | + | |
- | 7 | + | |
- | 5 | + | |
- | 3 | + | |
- | 9 v4/gw 10.0.24.2 | + | |
- | 8 v4/gw 10.0.23.2 | + | |
- | Nexthop groups data | + | |
- | BUF: 192 | + | |
- | Internet: | + | router1# exit |
- | MpIdx NHIdx Weigh Slots Gateway Netif Refcnt | + | [root@router1]~# |
- | 1 ---- ---- ---- ---- ---- 1 | + | 10.0.24.0/24 |
- | | + | 10.0.24.0/ |
- | | + | |
</ | </ | ||
- | < | + | ===== Test Load balancing IP packets ===== |
- | [root@VM1]~# | + | |
- | PING 10.0.56.26 (10.0.56.26) from 10.0.12.21: 56 data bytes | + | |
- | --- 10.0.56.26 ping statistics --- | + | Flows from the client to the server should be "flow-id shared" |
- | 1 packets transmitted, | + | |
- | </ | + | ^ Source ^ Destination ^ Ongoing path selected by R1 ^ Returning path selected by R2^ |
+ | | 10.0.31.3 | 10.0.24.4 | igb1 | igb2 | | ||
+ | | 10.0.31.3 | 10.0.24.14 | igb2 | igb1 | | ||
+ | | 10.0.31.13 | 10.0.24.14 | igb2 | igb1 | | ||
+ | | 10.0.31.23 | 10.0.24.24 | igb1 | igb2 | | ||
+ | | 10.0.31.33 | 10.0.24.34 | igb2 | igb2 | | ||
+ | | 10.0.31.43 | 10.0.24.44 | igb1 | igb2 | | ||
+ | | 10.0.31.53 | 10.0.24.54 | igb1 | igb1 | | ||
+ | | 2001: | ||
+ | | 2001: | ||
+ | | 2001: | ||
+ | | 2001: | ||
+ | | 2001: |
documentation/examples/ecmp.txt · Last modified: 2022/12/11 18:40 by olivier