documentation:examples:ecmp
- en
- fr
This is an old revision of the document!
Table of Contents
Equal-cost multi-path routing (ECMP)
This lab is testing new ECMP routing code from FreeBSD's review D26449
Presentation
Bhyve doesn't support emulating multiqueue NIC, so RSS flow-id could not be tested using a bhyve based lab: Had to use physical lab
Network diagram
Here is the logical and physical view:
Setting-up the lab
Downloading BSD Router Project images
Download BSDRP serial image on Sourceforge and upload them to the 2 ECMP routers.
Static routing setup
VM 1 (client)
VM1 is configured as a simple host with multiples aliases:
sysrc hostname=VM1 \ gateway_enable=NO \ ipv6_gateway_enable=NO \ ifconfig_igb1="inet 10.0.31.3/24" \ ifconfig_igb1_alias1="inet 10.0.31.13/32" \ ifconfig_igb1_alias2="inet 10.0.31.23/32" \ ifconfig_igb1_alias3="inet 10.0.31.33/32" \ ifconfig_igb1_alias4="inet 10.0.31.43/32" \ ifconfig_igb1_alias5="inet 10.0.31.53/32" \ ifconfig_igb1_ipv6="inet6 2001:db8:31::3 prefixlen 64" \ ifconfig_igb1_alias61="inet6 2001:db8:31::13 prefixlen 64" \ ifconfig_igb1_alias62="inet6 2001:db8:31::23 prefixlen 64" \ ifconfig_igb1_alias63="inet6 2001:db8:31::33 prefixlen 64" \ ifconfig_igb1_alias64="inet6 2001:db8:31::43 prefixlen 64" \ ifconfig_igb1_alias65="inet6 2001:db8:31::53 prefixlen 64" \ static_routes="LAB" \ route_LAB="-net 10.0.0.0/16 10.0.31.1" \ ipv6_static_routes="LAB" \ ipv6_route_LAB="2001:db8:: -prefixlen 32 2001:db8:31::1" service hostname restart service netif restart service routing restart config save
VM 2 (ECMP router)
VM 2 is a router with ECMP: 2 static routes toward the same destination but using 2 different next-hop.
sysrc hostname=VM2 \ gateway_enable=YES \ ipv6_gateway_enable=YES \ ifconfig_igb0="inet 10.0.31.1/24" \ ifconfig_igb0_ipv6="inet6 2001:db8:31::1 prefixlen 64" \ ifconfig_igb1="inet 10.0.112.1/24" \ ifconfig_igb1_ipv6="inet6 2001:db8:112::1 prefixlen 64" \ ifconfig_igb2="inet 10.0.212.1/24" \ ifconfig_igb2_ipv6="inet6 2001:db8:212::1 prefixlen 64" \ static_routes="MPATH1 MPATH2" \ route_MPATH1="-net 10.0.0.0/16 10.0.112.2" \ route_MPATH2="-net 10.0.0.0/16 10.0.212.2" \ ipv6_static_routes="MPATH1 MPATH2" \ ipv6_route_MPATH1="2001:db8:: -prefixlen 32 2001:db8:112::2" \ ipv6_route_MPATH2="2001:db8:: -prefixlen 32 2001:db8:212::2" service hostname restart service netif restart service routing restart config save
Checking static route with multiple next-hop:
[root@VM2]~# # netstat -rn4 | grep 10.0.0.0/16
10.0.0.0/16 10.0.112.2 UGS igb1
10.0.0.0/16 10.0.212.2 UGS igb2
[root@VM2]~# route -n get 10.0.0.0/16
route to: 10.0.0.0
destination: 10.0.0.0
mask: 255.255.0.0
gateway: 10.0.112.2
fib: 0
interface: igb1
flags: <UP,GATEWAY,DONE,STATIC>
recvpipe sendpipe ssthresh rtt,msec mtu weight expire
0 0 0 0 1500 0 0
[root@VM2]~# netstat -4onW
Nexthop data
Internet:
Idx Type IFA Gateway Flags Use Mtu Netif Addrif Refcnt Prepend
1 v4/resolve 127.0.0.1 lo0/resolve HS 0 16384 lo0 1
2 v4/resolve 10.0.31.1 igb0/resolve 0 1500 igb0 1
3 v4/resolve 127.0.0.1 lo0/resolve HS 0 16384 lo0 igb0 1
4 v4/resolve 10.0.112.1 igb1/resolve 0 1500 igb1 1
5 v4/resolve 127.0.0.1 lo0/resolve HS 0 16384 lo0 igb1 1
6 v4/resolve 10.0.212.1 igb2/resolve 0 1500 igb2 1
7 v4/resolve 127.0.0.1 lo0/resolve HS 0 16384 lo0 igb2 1
8 v4/gw 10.0.112.1 10.0.112.2 GS 0 1500 igb1 1
9 v4/gw 10.0.212.1 10.0.212.2 GS 0 1500 igb2 1
[root@VM2]~# netstat -rn6 | grep 2001:db8::/32
2001:db8::/32 2001:db8:212::2 UGS igb2
2001:db8::/32 2001:db8:112::2 UGS igb1
[root@VM2]~# route -n6 get 2001:db8::/32
route to: 2001:db8::
destination: 2001:db8::
mask: ffff:ffff::
gateway: 2001:db8:212::2
fib: 0
interface: igb2
flags: <UP,GATEWAY,DONE,STATIC>
recvpipe sendpipe ssthresh rtt,msec mtu weight expire
0 0 0 0 1500 0 0
[root@VM2]~# netstat -6onW
Nexthop data
Internet6:
Idx Type IFA Gateway Flags Use Mtu Netif Addrif Refcnt Prepend
1 v6/resolve ::1 lo0/resolve H 0 16384 lo0 1
2 v6/resolve fe80::1%lo0 lo0/resolve HS 0 16384 lo0 1
3 v6/resolve fe80::1%lo0 lo0/resolve 0 16384 lo0 1
4 v6/resolve ::1 lo0/resolve HS 0 16384 lo0 igb0 2
5 v6/resolve fe80::20d:b9ff:fe45:79e4%igb0 igb0/resolve 0 1500 igb0 1
6 v6/resolve 2001:db8:31::1 igb0/resolve 0 1500 igb0 1
7 v6/resolve fe80::20d:b9ff:fe45:79e5%igb1 igb1/resolve 0 1500 igb1 1
8 v6/resolve ::1 lo0/resolve HS 0 16384 lo0 igb1 2
9 v6/resolve 2001:db8:112::1 igb1/resolve 0 1500 igb1 1
10 v6/resolve fe80::20d:b9ff:fe45:79e6%igb2 igb2/resolve 0 1500 igb2 1
11 v6/resolve ::1 lo0/resolve HS 0 16384 lo0 igb2 2
12 v6/gw ::1 ::1 GRS 0 16384 lo0 4
13 v6/gw 2001:db8:112::1 2001:db8:112::2 GS 0 1500 igb1 1
14 v6/gw 2001:db8:212::1 2001:db8:212::2 GS 0 1500 igb2 1
15 v6/resolve 2001:db8:212::1 igb2/resolve 0 1500 igb2 1
Look like this router will prefer igb1 for inet4 and igb2 for inet6.
VM 5 (ECMP router)
VM 5 is like VM 2, a router with ECMP: 2 static routing toward the same destination but using 2 different next-hop..
sysrc hostname=VM5 \ gateway_enable=YES \ ipv6_gateway_enable=YES \ ifconfig_igb0="inet 10.0.24.2/24" \ ifconfig_igb0_ipv6="inet6 2001:db8:24::2 prefixlen 64" \ ifconfig_igb1="inet 10.0.112.2/24" \ ifconfig_igb1_ipv6="inet6 2001:db8:112::2 prefixlen 64" \ ifconfig_igb2="inet 10.0.212.2/24" \ ifconfig_igb2_ipv6="inet6 2001:db8:212::2 prefixlen 64" \ static_routes="MPATH1 MPATH2" \ route_MPATH1="-net 10.0.0.0/16 10.0.112.1" \ route_MPATH2="-net 10.0.0.0/16 10.0.212.1" \ ipv6_static_routes="MPATH1 MPATH2" \ ipv6_route_MPATH1="2001:db8:: -prefixlen 32 2001:db8:112::1" \ ipv6_route_MPATH2="2001:db8:: -prefixlen 32 2001:db8:212::1" service hostname restart service netif restart service routing restart config save
VM 6 (server)
VM 6, like VM 1 is configured as a simple host with multiple aliases:
sysrc hostname=VM6 \ gateway_enable=NO \ ipv6_gateway_enable=NO \ ifconfig_vr0="inet 10.0.24.4/24" \ ifconfig_vr0_alias1="inet 10.0.24.14/32" \ ifconfig_vr0_alias2="inet 10.0.24.24/32" \ ifconfig_vr0_alias3="inet 10.0.24.34/32" \ ifconfig_vr0_alias4="inet 10.0.24.44/32" \ ifconfig_vr0_alias5="inet 10.0.24.54/32" \ ifconfig_vr0_ipv6="inet6 2001:db8:24::4 prefixlen 64" \ ifconfig_vr0_alias61="inet6 2001:db8:24::14 prefixlen 64" \ ifconfig_vr0_alias62="inet6 2001:db8:24::24 prefixlen 64" \ ifconfig_vr0_alias63="inet6 2001:db8:24::34 prefixlen 64" \ ifconfig_vr0_alias64="inet6 2001:db8:24::44 prefixlen 64" \ ifconfig_vr0_alias65="inet6 2001:db8:24::54 prefixlen 64" \ static_routes="LAB" \ route_LAB="-net 10.0.0.0/16 10.0.24.2" \ ipv6_static_routes="LAB" \ ipv6_route_LAB="2001:db8:: -prefixlen 32 2001:db8:24::2" service hostname restart service netif restart service routing restart config save
Testing Load balancing
Flows from the client to the server should be “flow-id shared” between the 2 paths. Let's check using multiple sources and destination IP addresses.
</code>
| Source | Destination | Ongoing path selected by VM2 | Returning path selected by VM5 |
|---|---|---|---|
| 10.0.31.3 | 10.0.24.4 | igb1 | igb2 |
| 10.0.31.13 | 10.0.24.14 | igb1 | igb1 |
| 10.0.31.23 | 10.0.24.24 | igb1 | igb2 |
| 10.0.31.33 | 10.0.24.34 | igb1 | igb2 |
| 10.0.31.43 | 10.0.24.44 | igb1 | igb1 |
| 10.0.31.43 | 10.0.24.44 | igb1 | igb2 |
| 2001:db8:12::1 | 2001:db8:56::6 | VM4 | VM4 |
| 2001:db8:12::11 | 2001:db8:56::16 | VM4 | VM4 |
| 2001:db8:12::21 | 2001:db8:56::26 | VM4 | VM4 |
| 2001:db8:12::31 | 2001:db8:56::36 | VM4 | VM4 |
| 2001:db8:12::41 | 2001:db8:56::46 | VM4 | VM4 |
| 2001:db8:12::51 | 2001:db8:56::56 | VM4 | VM4 |
documentation/examples/ecmp.1600685907.txt.gz · Last modified: 2020/09/21 12:58 by olivier