User Tools

Site Tools


documentation:examples:ecmp

Equal-cost multi-path routing (ECMP)

This lab show an ECMP (feature called RADIX_MPATH on FreeBSD) routing example using BSDRP (FreeBSD 11).

Presentation

Network diagram

Here is the logical and physical view:

Setting-up the lab

Downloading BSD Router Project images

Download BSDRP serial image (prevent to have to use an X display) on Sourceforge.

Download Lab scripts

More information on these BSDRP lab scripts available on How to build a BSDRP router lab.

Start the lab with 6 routers. With bhyve under FreeBSD:

user:~ # BSDRP-lab-bhyve.sh -i BSDRP.amd64/BSDRP-1.71-full-amd64-serial.img.xz -n 6
BSD Router Project (http://bsdrp.net) - bhyve full-meshed lab script
Setting-up a virtual lab with 6 VM(s):
- Working directory: /tmp/BSDRP
- Each VM have 1 core(s) and 256M RAM
- Switch mode: bridge + tap
- 0 LAN(s) between all VM
- Full mesh Ethernet links between each VM
VM 1 have the following NIC:
- vtnet0 connected to VM 2.
- vtnet1 connected to VM 3.
- vtnet2 connected to VM 4.
- vtnet3 connected to VM 5.
- vtnet4 connected to VM 6.
VM 2 have the following NIC:
- vtnet0 connected to VM 1.
- vtnet1 connected to VM 3.
- vtnet2 connected to VM 4.
- vtnet3 connected to VM 5.
- vtnet4 connected to VM 6.
VM 3 have the following NIC:
- vtnet0 connected to VM 1.
- vtnet1 connected to VM 2.
- vtnet2 connected to VM 4.
- vtnet3 connected to VM 5.
- vtnet4 connected to VM 6.
VM 4 have the following NIC:
- vtnet0 connected to VM 1.
- vtnet1 connected to VM 2.
- vtnet2 connected to VM 3.
- vtnet3 connected to VM 5.
- vtnet4 connected to VM 6.
VM 5 have the following NIC:
- vtnet0 connected to VM 1.
- vtnet1 connected to VM 2.
- vtnet2 connected to VM 3.
- vtnet3 connected to VM 4.
- vtnet4 connected to VM 6.
VM 6 have the following NIC:
- vtnet0 connected to VM 1.
- vtnet1 connected to VM 2.
- vtnet2 connected to VM 3.
- vtnet3 connected to VM 4.
- vtnet4 connected to VM 5.
For connecting to VM'serial console, you can use:
- VM 1 : cu -l /dev/nmdm1B
- VM 2 : cu -l /dev/nmdm2B
- VM 3 : cu -l /dev/nmdm3B
- VM 4 : cu -l /dev/nmdm4B
- VM 5 : cu -l /dev/nmdm5B
- VM 6 : cu -l /dev/nmdm6B

Static routing setup

Router 1 (client)

R1 is configured as a simple host with multiples aliases:

sysrc hostname=R1
sysrc gateway_enable=NO
sysrc ipv6_gateway_enable=NO
sysrc ifconfig_vtnet0="inet 10.0.12.1/24"
sysrc ifconfig_vtnet0_alias1="inet 10.0.12.11/32"
sysrc ifconfig_vtnet0_alias2="inet 10.0.12.21/32"
sysrc ifconfig_vtnet0_alias3="inet 10.0.12.31/32"
sysrc ifconfig_vtnet0_alias4="inet 10.0.12.41/32"
sysrc ifconfig_vtnet0_alias5="inet 10.0.12.51/32"
sysrc ifconfig_vtnet0_ipv6="inet6 2001:db8:12::1 prefixlen 64"
sysrc ifconfig_vtnet0_alias61="inet6 2001:db8:12::11 prefixlen 64"
sysrc ifconfig_vtnet0_alias62="inet6 2001:db8:12::21 prefixlen 64"
sysrc ifconfig_vtnet0_alias63="inet6 2001:db8:12::31 prefixlen 64"
sysrc ifconfig_vtnet0_alias64="inet6 2001:db8:12::41 prefixlen 64"
sysrc ifconfig_vtnet0_alias65="inet6 2001:db8:12::51 prefixlen 64"
sysrc defaultrouter="10.0.12.2"
sysrc ipv6_defaultrouter="2001:db8:12::2"
service hostname restart
service netif restart
service routing restart
config save

Router 2 (ECMP router)

R2 is a router with ECMP: 2 static routes toward the same destination but using 2 different next-hop.

sysrc hostname=R2
sysrc ifconfig_vtnet0="inet 10.0.12.2/24"
sysrc ifconfig_vtnet1="inet 10.0.23.2/24"
sysrc ifconfig_vtnet2="inet 10.0.24.2/24"
sysrc ifconfig_vtnet0_ipv6="inet6 2001:db8:12::2 prefixlen 64"
sysrc ifconfig_vtnet1_ipv6="inet6 2001:db8:23::2 prefixlen 64"
sysrc ifconfig_vtnet2_ipv6="inet6 2001:db8:24::2 prefixlen 64"
sysrc static_routes="R3R6 R3R5 R4R6 R4R5"
sysrc route_R3R6="-net 10.0.56.0/24 10.0.23.3"
sysrc route_R3R5="-net 10.0.35.0/24 10.0.23.3"
sysrc route_R4R6="-net 10.0.56.0/24 10.0.24.4"
sysrc route_R4R5="-net 10.0.45.0/24 10.0.24.4"
sysrc ipv6_static_routes="R3R6 R3R5 R4R6 R4R5"
sysrc ipv6_route_R3R6="2001:db8:56:: -prefixlen 64 2001:db8:23::3"
sysrc ipv6_route_R3R5="2001:db8:35:: -prefixlen 64 2001:db8:23::3"
sysrc ipv6_route_R4R6="2001:db8:56:: -prefixlen 64 2001:db8:24::4"
sysrc ipv6_route_R4R5="2001:db8:45:: -prefixlen 64 2001:db8:24::4"
service hostname restart
service netif restart
service routing restart
config save

Router 3 (simple router)

R3 is a simple router.

sysrc hostname=R3
sysrc ifconfig_vtnet1="inet 10.0.23.3/24"
sysrc ifconfig_vtnet3="inet 10.0.35.3/24"
sysrc ifconfig_vtnet1_ipv6="inet6 2001:db8:23::3 prefixlen 64"
sysrc ifconfig_vtnet3_ipv6="inet6 2001:db8:35::3 prefixlen 64"
sysrc static_routes="R6 R1"
sysrc route_R1="-net 10.0.12.0/24 10.0.23.2"
sysrc route_R6="-net 10.0.56.0/24 10.0.35.5"
sysrc ipv6_static_routes="R6 R1"
sysrc ipv6_route_R1="2001:db8:12:: -prefixlen 64 2001:db8:23::2"
sysrc ipv6_route_R6="2001:db8:56:: -prefixlen 64 2001:db8:35::5"
service hostname restart
service netif restart
service routing restart
config save

Router 4 (simple router)

R4 is like R3, a simple router.

sysrc hostname=R4
sysrc ifconfig_vtnet1="inet 10.0.24.4/24"
sysrc ifconfig_vtnet3="inet 10.0.45.4/24"
sysrc ifconfig_vtnet1_ipv6="inet6 2001:db8:24::4 prefixlen 64"
sysrc ifconfig_vtnet3_ipv6="inet6 2001:db8:45::4 prefixlen 64"
sysrc static_routes="R6 R1"
sysrc route_R1="-net 10.0.12.0/24 10.0.24.2"
sysrc route_R6="-net 10.0.56.0/24 10.0.45.5"
sysrc ipv6_static_routes="R6 R1"
sysrc ipv6_route_R1="2001:db8:12:: -prefixlen 64 2001:db8:24::2"
sysrc ipv6_route_R6="2001:db8:56:: -prefixlen 64 2001:db8:45::5"
service hostname restart
service netif restart
service routing restart
config save

Router 5 (ECMP router)

R5 is like R2, a router with ECMP: 2 static routing toward the same destination but using 2 different next-hop..

sysrc hostname=R5
sysrc ifconfig_vtnet2="inet 10.0.35.5/24"
sysrc ifconfig_vtnet3="inet 10.0.45.5/24"
sysrc ifconfig_vtnet4="inet 10.0.56.5/24"
sysrc ifconfig_vtnet2_ipv6="inet6 2001:db8:35::5 prefixlen 64"
sysrc ifconfig_vtnet3_ipv6="inet6 2001:db8:45::5 prefixlen 64"
sysrc ifconfig_vtnet4_ipv6="inet6 2001:db8:56::5 prefixlen 64"
sysrc static_routes="R3R1 R3R2 R4R1 R4R2"
sysrc route_R3R1="-net 10.0.12.0/24 10.0.35.3"
sysrc route_R3R2="-net 10.0.23.0/24 10.0.35.3"
sysrc route_R4R1="-net 10.0.12.0/24 10.0.45.4"
sysrc route_R4R2="-net 10.0.24.0/24 10.0.45.4"
sysrc ipv6_static_routes="R3R1 R3R2 R4R1 R4R2"
sysrc ipv6_route_R3R1="2001:db8:12:: -prefixlen 64 2001:db8:35::3"
sysrc ipv6_route_R3R2="2001:db8:23:: -prefixlen 64 2001:db8:35::3"
sysrc ipv6_route_R4R1="2001:db8:12:: -prefixlen 64 2001:db8:45::4"
sysrc ipv6_route_R4R2="2001:db8:24:: -prefixlen 64 2001:db8:45::4"
service hostname restart
service netif restart
service routing restart
config save

Router 6 (server)

R6, like R1 is configured as a simple host with multiple aliases:

sysrc hostname=R6
sysrc gateway_enable=NO
sysrc ipv6_gateway_enable=NO
sysrc ifconfig_vtnet4="inet 10.0.56.6/24"
sysrc ifconfig_vtnet4_alias1="inet 10.0.56.16/32"
sysrc ifconfig_vtnet4_alias2="inet 10.0.56.26/32"
sysrc ifconfig_vtnet4_alias3="inet 10.0.56.36/32"
sysrc ifconfig_vtnet4_alias4="inet 10.0.56.46/32"
sysrc ifconfig_vtnet4_alias5="inet 10.0.56.56/32"
sysrc ifconfig_vtnet4_ipv6="inet6 2001:db8:56::6 prefixlen 64"
sysrc ifconfig_vtnet4_alias61="inet6 2001:db8:56::16 prefixlen 64"
sysrc ifconfig_vtnet4_alias62="inet6 2001:db8:56::26 prefixlen 64"
sysrc ifconfig_vtnet4_alias63="inet6 2001:db8:56::36 prefixlen 64"
sysrc ifconfig_vtnet4_alias64="inet6 2001:db8:56::46 prefixlen 64"
sysrc ifconfig_vtnet4_alias65="inet6 2001:db8:56::56 prefixlen 64"
sysrc defaultrouter="10.0.56.5"
sysrc ipv6_defaultrouter="2001:db8:56::5"
service hostname restart
service netif restart
service routing restart
config save

Testing Load balancing

Flows from R1 to R6 should be load-balanced by R2 toward R3 and R4. Let's check using multiple sources and destination IP addresses:

Source Destination Ongoing path selected by R2 Returning path selected by R5
10.0.12.1 10.0.56.6 R3 R3
10.0.12.11 10.0.56.16 R3 R3
10.0.12.21 10.0.56.26 R3 R3
10.0.12.31 10.0.56.36 R3 R3
10.0.12.41 10.0.56.46 R3 R3
10.0.12.51 10.0.56.56 R3 R3
2001:db8:12::1 2001:db8:56::6 R3 R3
2001:db8:12::11 2001:db8:56::16 R3 R3
2001:db8:12::21 2001:db8:56::26 R3 R3
2001:db8:12::31 2001:db8:56::36 R3 R3
2001:db8:12::41 2001:db8:56::46 R3 R3
2001:db8:12::51 2001:db8:56::56 R3 R3
10.0.12.1 10.0.45.5 R4 R3
There is no flow load-balancing with RADIX MPATH on FreeBSD 11, seems broken with tryforward, put proposed patch disable tryforward(fastforward) if RADIX_MPATH is enabled.
documentation/examples/ecmp.txt · Last modified: 2018/07/13 10:31 by olivier