====== Multicast with PIM-SM ======
This lab show a multicast routing example using PIM in Sparse Mode.
===== Presentation =====
==== Network diagram ====
Here is the logical and physical view:
{{:documentation:examples:bsdrp.lab.pim-sm.png|}}
===== Setting-up the lab =====
==== Downloading BSD Router Project images ====
Download BSDRP serial image (prevent to have to use an X display) on Sourceforge.
==== Download Lab scripts =====
More information on these BSDRP lab scripts available on [[documentation:examples:How to build a BSDRP router lab]].
Start the lab with 4 routers simulating an e1000 NIC (vtnet interface and they didn't support mcast routing on FreeBSD):
tools/BSDRP-lab-bhyve.sh -n 4 -e -i BSDRP-1.96-full-amd64-serial.img.xz
BSD Router Project (http://bsdrp.net) - bhyve full-meshed lab script
Setting-up a virtual lab with 4 VM(s):
- Working directory: /root/BSDRP-VMs
- Each VM has 1 core and 512M RAM
- Emulated NIC: e1000
- Switch mode: bridge + tap
- 0 LAN(s) between all VM
- Full mesh Ethernet links between each VM
VM 1 has the following NIC:
- em0 connected to VM 2
- em1 connected to VM 3
- em2 connected to VM 4
VM 2 has the following NIC:
- em0 connected to VM 1
- em1 connected to VM 3
- em2 connected to VM 4
VM 3 has the following NIC:
- em0 connected to VM 1
- em1 connected to VM 2
- em2 connected to VM 4
VM 4 has the following NIC:
- em0 connected to VM 1
- em1 connected to VM 2
- em2 connected to VM 3
To connect VM'serial console, you can use:
- VM 1 : cu -l /dev/nmdm-BSDRP.1B
- VM 2 : cu -l /dev/nmdm-BSDRP.2B
- VM 3 : cu -l /dev/nmdm-BSDRP.3B
- VM 4 : cu -l /dev/nmdm-BSDRP.4B
===== Routers configuration =====
==== Router 1 ====
Configuration:
sysrc hostname=VM1 \
gateway_enable=no \
ipv6_gateway_enable=no \
ifconfig_em0="inet 10.0.12.1/24" \
defaultrouter=10.0.12.254
service hostname restart
service netif restart
service routing restart
config save
==== Router 2 ====
VM2 is a PIM router that announce itself (10.0.23.2) as Canditate RP with and adv period of 10 seconds and high priority (will be the rendez-vous point).
sysrc hostname=VM2 \
ifconfig_em0="inet 10.0.12.254/24" \
ifconfig_em1="inet 10.0.23.2/24" \
defaultrouter=10.0.23.3 \
pimd_enable=yes
cat > /usr/local/etc/pimd.conf <
==== Router 3 ====
We would VM3 annonces hitself (10.0.23.3) as a Canditate BootStrap Router with high priority.
sysrc hostname=VM3 \
ifconfig_em1="inet 10.0.23.3/24" \
ifconfig_em2="inet 10.0.34.254/24" \
defaultrouter=10.0.23.2 \
pimd_enable=yes
cat > /usr/local/etc/pimd.conf <
==== Router 4 ====
sysrc hostname=VM4 \
gateway_enable=no \
ipv6_gateway_enable=no \
ifconfig_em2="inet 10.0.34.4/24" \
defaultrouter=10.0.34.254
service hostname restart
service netif restart
service routing restart
config save
===== Checking NIC drivers and Bhyve compatibility with multicast =====
Before to star with advanced routing setup, just start to test simple multicast between 2 relatives host: Some NIC (vtnet) or some hypervisors network setup aren't compliant with very simple multicast.
On VM1, start a mcast generator (client emitting mcast):
[root@VM1]~# iperf -c 239.1.1.1 -u -T 32 -t 3000 -i 1
------------------------------------------------------------
Client connecting to 239.1.1.1, UDP port 5001
Sending 1470 byte datagrams, IPG target: 11215.21 us (kalman adjust)
Setting multicast TTL to 32
UDP buffer size: 9.00 KByte (default)
------------------------------------------------------------
[ 3] local 10.0.12.1 port 46636 connected with 239.1.1.1 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0- 1.0 sec 131 KBytes 1.07 Mbits/sec
[ 3] 1.0- 2.0 sec 128 KBytes 1.05 Mbits/sec
[ 3] 2.0- 3.0 sec 128 KBytes 1.05 Mbits/sec
[ 3] 0.0- 3.5 sec 446 KBytes 1.05 Mbits/sec
[ 3] Sent 311 datagrams
(...)
On the direct connected VM2, start to check if in non-promiscious mode it sees mcast packets comming:
[root@VM2]~# tcpdump -pni em0 -c 2
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on em0, link-type EN10MB (Ethernet), capture size 262144 bytes
15:22:32.517270 IP 10.0.12.1.33482 > 239.1.1.1.5001: UDP, length 1470
15:22:32.528668 IP 10.0.12.1.33482 > 239.1.1.1.5001: UDP, length 1470
2 packets captured
2 packets received by filter
0 packets dropped by kernel
=> VM2 is receiving mcast packets from 10.0.12.1 to mcast group 239.1.1.1.
Now on VM2 start a mcast listener (server receiving), it should receive multicast flow
[root@VM2]~# iperf -s -u -B 239.1.1.1%em0 -i 1
------------------------------------------------------------
Server listening on UDP port 5001
Binding to local address 239.1.1.1
Joining multicast group 239.1.1.1
Receiving 1470 byte datagrams
UDP buffer size: 41.1 KByte (default)
------------------------------------------------------------
[ 3] local 239.1.1.1 port 5001 connected with 192.168.100.149 port 35181
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 3] 0.0- 1.0 sec 129 KBytes 1.06 Mbits/sec 0.038 ms 107/ 197 (54%)
[ 3] 1.0- 2.0 sec 128 KBytes 1.05 Mbits/sec 0.054 ms 0/ 89 (0%)
[ 3] 2.0- 3.0 sec 128 KBytes 1.05 Mbits/sec 0.021 ms 0/ 89 (0%)
[ 3] 3.0- 4.0 sec 128 KBytes 1.05 Mbits/sec 0.025 ms 0/ 89 (0%)
[ 3] 4.0- 5.0 sec 128 KBytes 1.05 Mbits/sec 0.024 ms 0/ 89 (0%)
[ 3] 5.0- 6.0 sec 129 KBytes 1.06 Mbits/sec 0.024 ms 0/ 90 (0%)
[ 3] 6.0- 7.0 sec 128 KBytes 1.05 Mbits/sec 0.024 ms 0/ 89 (0%)
(...)
=> Notice the mcast receiver is correctly receiving at 1Mb/s.
Here is a non working example (here because source interface not given, and it uses the other one):
[root@VM2]~# iperf -s -u -B 239.1.1.1 -i 1
------------------------------------------------------------
Server listening on UDP port 5001
Binding to local address 239.1.1.1
Joining multicast group 239.1.1.1
Receiving 1470 byte datagrams
UDP buffer size: 41.1 KByte (default)
------------------------------------------------------------
(...)
=> Here it doesn't receive traffic and stay in "waiting" mode forever.
===== Checking pimd behavior =====
==== PIM neighbors ====
Does the PIM routers see each others ?
[root@VM2]~# pimd -r
Virtual Interface Table ======================================================
Vif Local Address Subnet Thresh Flags Neighbors
--- --------------- ------------------ ------ --------- -----------------
0 10.0.12.254 10.0.12/24 1 DR NO-NBR
1 10.0.23.2 10.0.23/24 1 PIM 10.0.23.3
2 10.0.12.254 register_vif0 1
Vif SSM Group Sources
Multicast Routing Table ======================================================
--------------------------------- (*,*,G) ------------------------------------
Number of Groups: 0
Number of Cache MIRRORs: 0
------------------------------------------------------------------------------
=> VM2 sees VM3 as PIM neighbor
[root@VM3]~# pimd -r
Virtual Interface Table ======================================================
Vif Local Address Subnet Thresh Flags Neighbors
--- --------------- ------------------ ------ --------- -----------------
0 10.0.23.3 10.0.23/24 1 DR PIM 10.0.23.2
1 10.0.34.254 10.0.34/24 1 DR NO-NBR
2 10.0.23.3 register_vif0 1
Vif SSM Group Sources
Multicast Routing Table ======================================================
--------------------------------- (*,*,G) ------------------------------------
Number of Groups: 0
Number of Cache MIRRORs: 0
------------------------------------------------------------------------------
=> VM3 sees VM2 as PIM Designated Router neighbor.
==== Does PIM daemon locally register to PIM mcast group ? ====
PIM router need to register to 224.0.0.13 mcast group, check if all PIM routers correctly display this group on their enabled interfaces:
[root@VM2]~# ifmcstat
em0:
inet 10.0.12.2
igmpv2
group 224.0.0.22 refcnt 1 state lazy mode exclude
mcast-macaddr 01:00:5e:00:00:16 refcnt 1
group 224.0.0.2 refcnt 1 state lazy mode exclude
mcast-macaddr 01:00:5e:00:00:02 refcnt 1
group 224.0.0.13 refcnt 1 state lazy mode exclude
mcast-macaddr 01:00:5e:00:00:0d refcnt 1
group 224.0.0.1 refcnt 1 state silent mode exclude
mcast-macaddr 01:00:5e:00:00:01 refcnt 1
inet6 fe80:1::a8aa:ff:fe00:212
mldv2 flags=2 rv 2 qi 125 qri 10 uri 3
group ff01:1::1 refcnt 1
mcast-macaddr 33:33:00:00:00:01 refcnt 1
group ff02:1::2:54c6:805c refcnt 1
mcast-macaddr 33:33:54:c6:80:5c refcnt 1
group ff02:1::2:ff54:c680 refcnt 1
mcast-macaddr 33:33:ff:54:c6:80 refcnt 1
group ff02:1::1 refcnt 1
mcast-macaddr 33:33:00:00:00:01 refcnt 1
group ff02:1::1:ff00:212 refcnt 1
mcast-macaddr 33:33:ff:00:02:12 refcnt 1
em1:
inet 10.0.23.2
igmpv2
group 224.0.0.22 refcnt 1 state sleeping mode exclude
mcast-macaddr 01:00:5e:00:00:16 refcnt 1
group 224.0.0.2 refcnt 1 state lazy mode exclude
mcast-macaddr 01:00:5e:00:00:02 refcnt 1
group 224.0.0.13 refcnt 1 state lazy mode exclude
mcast-macaddr 01:00:5e:00:00:0d refcnt 1
group 224.0.0.1 refcnt 1 state silent mode exclude
mcast-macaddr 01:00:5e:00:00:01 refcnt 1
inet6 fe80:2::a8aa:ff:fe02:202
mldv2 flags=2 rv 2 qi 125 qri 10 uri 3
group ff01:2::1 refcnt 1
mcast-macaddr 33:33:00:00:00:01 refcnt 1
group ff02:2::2:54c6:805c refcnt 1
mcast-macaddr 33:33:54:c6:80:5c refcnt 1
group ff02:2::2:ff54:c680 refcnt 1
mcast-macaddr 33:33:ff:54:c6:80 refcnt 1
group ff02:2::1 refcnt 1
mcast-macaddr 33:33:00:00:00:01 refcnt 1
group ff02:2::1:ff02:202 refcnt 1
mcast-macaddr 33:33:ff:02:02:02 refcnt 1
[root@VM3]~# ifmcstat
em0:
em1:
inet 10.0.23.3
igmpv2
group 224.0.0.22 refcnt 1 state sleeping mode exclude
mcast-macaddr 01:00:5e:00:00:16 refcnt 1
group 224.0.0.2 refcnt 1 state sleeping mode exclude
mcast-macaddr 01:00:5e:00:00:02 refcnt 1
group 224.0.0.13 refcnt 1 state lazy mode exclude
mcast-macaddr 01:00:5e:00:00:0d refcnt 1
group 224.0.0.1 refcnt 1 state silent mode exclude
mcast-macaddr 01:00:5e:00:00:01 refcnt 1
inet6 fe80:2::a8aa:ff:fe00:323
mldv2 flags=2 rv 2 qi 125 qri 10 uri 3
group ff01:2::1 refcnt 1
mcast-macaddr 33:33:00:00:00:01 refcnt 1
group ff02:2::2:1124:9296 refcnt 1
mcast-macaddr 33:33:11:24:92:96 refcnt 1
group ff02:2::2:ff11:2492 refcnt 1
mcast-macaddr 33:33:ff:11:24:92 refcnt 1
group ff02:2::1 refcnt 1
mcast-macaddr 33:33:00:00:00:01 refcnt 1
group ff02:2::1:ff00:323 refcnt 1
mcast-macaddr 33:33:ff:00:03:23 refcnt 1
em2:
inet 10.0.34.3
igmpv2
group 224.0.0.22 refcnt 1 state lazy mode exclude
mcast-macaddr 01:00:5e:00:00:16 refcnt 1
group 224.0.0.2 refcnt 1 state lazy mode exclude
mcast-macaddr 01:00:5e:00:00:02 refcnt 1
group 224.0.0.13 refcnt 1 state lazy mode exclude
mcast-macaddr 01:00:5e:00:00:0d refcnt 1
group 224.0.0.1 refcnt 1 state silent mode exclude
mcast-macaddr 01:00:5e:00:00:01 refcnt 1
inet6 fe80:3::a8aa:ff:fe03:303
mldv2 flags=2 rv 2 qi 125 qri 10 uri 3
group ff01:3::1 refcnt 1
mcast-macaddr 33:33:00:00:00:01 refcnt 1
group ff02:3::2:1124:9296 refcnt 1
mcast-macaddr 33:33:11:24:92:96 refcnt 1
group ff02:3::2:ff11:2492 refcnt 1
mcast-macaddr 33:33:ff:11:24:92 refcnt 1
group ff02:3::1 refcnt 1
mcast-macaddr 33:33:00:00:00:01 refcnt 1
group ff02:3::1:ff03:303 refcnt 1
mcast-macaddr 33:33:ff:03:03:03 refcnt 1
We correctly sees mcast group 224.0.0.13 subscribed on PIM enabled interfaces.
===== Testing =====
==== 1. Sart a mcast generator (IPerf client) on VM1 ====
Start an iperf client to 239.1.1.1.
[root@VM1]~# iperf -c 239.1.1.1 -u -T 32 -t 3000 -i 1
------------------------------------------------------------
Client connecting to 239.1.1.1, UDP port 5001
Sending 1470 byte datagrams
Setting multicast TTL to 32
UDP buffer size: 9.00 KByte (default)
------------------------------------------------------------
[ 3] local 10.0.12.1 port 41484 connected with 239.1.1.1 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0- 1.0 sec 129 KBytes 1.06 Mbits/sec
[ 3] 1.0- 2.0 sec 128 KBytes 1.05 Mbits/sec
[ 3] 2.0- 3.0 sec 128 KBytes 1.05 Mbits/sec
[ 3] 3.0- 4.0 sec 128 KBytes 1.05 Mbits/sec
==== 2. Check VM2 updates its mrouting table with discovered mcast source =====
PIM daemon should be updated:
[root@VM2]~# pimd -r
Virtual Interface Table ======================================================
Vif Local Address Subnet Thresh Flags Neighbors
--- --------------- ------------------ ------ --------- -----------------
0 10.0.12.254 10.0.12/24 1 DR NO-NBR
1 10.0.23.2 10.0.23/24 1 PIM 10.0.23.3
2 10.0.12.254 register_vif0 1
Vif SSM Group Sources
Multicast Routing Table ======================================================
----------------------------------- (S,G) ------------------------------------
Source Group RP Address Flags
--------------- --------------- --------------- ---------------------------
10.0.12.1 239.1.1.1 10.0.23.2 SG
Joined oifs: ...
Pruned oifs: ...
Leaves oifs: ...
Asserted oifs: ...
Outgoing oifs: ...
Incoming : I..
TIMERS: Entry JP RS Assert VIFS: 0 1 2
0 0 0 0 0 0 0
--------------------------------- (*,*,G) ------------------------------------
Number of Groups: 1
Number of Cache MIRRORs: 0
------------------------------------------------------------------------------
And mcast routing table too:
[root@VM2]~# netstat -g
IPv4 Virtual Interface Table
Vif Thresh Local-Address Remote-Address Pkts-In Pkts-Out
0 1 10.0.12.254 0 0
1 1 10.0.23.2 0 0
2 1 10.0.12.254 0 0
IPv4 Multicast Forwarding Table
Origin Group Packets In-Vif Out-Vifs:Ttls
10.0.12.1 239.1.1.1 0 65535
IPv6 Multicast Interface Table is empty
IPv6 Multicast Forwarding Table is empty
VM2 had update its mroute table for adding a source for group 239.1.1.1 comming from '65535'??.
==== 3. Start a mcast receiver (IPerf server) on VM4 ====
IPerf server will subscribe to 239.1.1.1 multicast group and receiving mcast traffic:
[root@VM4]~# iperf -s -u -B 239.1.1.1 -i 1
------------------------------------------------------------
Server listening on UDP port 5001
Binding to local address 239.1.1.1
Joining multicast group 239.1.1.1
Receiving 1470 byte datagrams
UDP buffer size: 41.1 KByte (default)
------------------------------------------------------------
[ 3] local 239.1.1.1 port 5001 connected with 10.0.12.1 port 41484
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 3] 0.0- 1.0 sec 128 KBytes 1.05 Mbits/sec 0.313 ms 16336/16425 (99%)
[ 3] 1.0- 2.0 sec 128 KBytes 1.05 Mbits/sec 0.250 ms 0/ 89 (0%)
[ 3] 2.0- 3.0 sec 128 KBytes 1.05 Mbits/sec 0.307 ms 0/ 89 (0%)
[ 3] 3.0- 4.0 sec 128 KBytes 1.05 Mbits/sec 0.262 ms 0/ 89 (0%)
[ 3] 4.0- 5.0 sec 128 KBytes 1.05 Mbits/sec 0.188 ms 0/ 89 (0%)
[ 3] 5.0- 6.0 sec 129 KBytes 1.06 Mbits/sec 0.347 ms 0/ 90 (0%)
[ 3] 6.0- 7.0 sec 128 KBytes 1.05 Mbits/sec 0.238 ms 0/ 89 (0%)
[ 3] 7.0- 8.0 sec 128 KBytes 1.05 Mbits/sec 0.234 ms 0/ 89 (0%)
[ 3] 8.0- 9.0 sec 128 KBytes 1.05 Mbits/sec 0.241 ms 0/ 89 (0%)
[ 3] 9.0-10.0 sec 128 KBytes 1.05 Mbits/sec 0.210 ms 0/ 89 (0%)
[ 3] 10.0-11.0 sec 128 KBytes 1.05 Mbits/sec 0.289 ms 0/ 89 (0%)
[ 3] 11.0-12.0 sec 129 KBytes 1.06 Mbits/sec 0.309 ms 0/ 90 (0%)
==== 4. Check VM3 correctly notice this mcast subscriber ====
Now the mrouting table of VM3 is updated and know it has a customer:
[root@VM3]~# pimd -r [9/367]
Virtual Interface Table ======================================================
Vif Local Address Subnet Thresh Flags Neighbors
--- --------------- ------------------ ------ --------- -----------------
0 10.0.23.3 10.0.23/24 1 DR PIM 10.0.23.2
1 10.0.34.254 10.0.34/24 1 DR NO-NBR
2 10.0.23.3 register_vif0 1
Vif SSM Group Sources
Multicast Routing Table ======================================================
----------------------------------- (*,G) ------------------------------------
Source Group RP Address Flags
--------------- --------------- --------------- ---------------------------
INADDR_ANY 239.1.1.1 10.0.23.2 WC RP CACHE
Joined oifs: ...
Pruned oifs: ...
Leaves oifs: .l.
Asserted oifs: ...
Outgoing oifs: .o.
Incoming : I..
TIMERS: Entry JP RS Assert VIFS: 0 1 2
0 50 0 0 0 0 0
----------------------------------- (S,G) ------------------------------------
Source Group RP Address Flags
--------------- --------------- --------------- ---------------------------
10.0.12.1 239.1.1.1 10.0.23.2 SG
Joined oifs: ...
Pruned oifs: ...
Leaves oifs: .l.
Asserted oifs: ...
Outgoing oifs: .o.
Incoming : I..
TIMERS: Entry JP RS Assert VIFS: 0 1 2
200 55 0 0 0 0 0
--------------------------------- (*,*,G) ------------------------------------
Number of Groups: 1
Number of Cache MIRRORs: 1
------------------------------------------------------------------------------
And its mcast routing is updated too:
[root@VM3]~# netstat -g
IPv4 Virtual Interface Table
Vif Thresh Local-Address Remote-Address Pkts-In Pkts-Out
0 1 10.0.23.3 157 0
1 1 10.0.34.254 0 157
2 1 10.0.23.3 0 0
IPv4 Multicast Forwarding Table
Origin Group Packets In-Vif Out-Vifs:Ttls
10.0.12.1 239.1.1.1 157 0 1:1
IPv6 Multicast Interface Table is empty
IPv6 Multicast Forwarding Table is empty
VM3 correctly learn that there is a subscriber to group 239.1.1.1 on interface vif1 (toward VM4) and the source is on vif0 (toward VM2).