Great to hear that the problem has been resolved. It’s often stressful when you’ve got a production network not working and you’re trying to troubleshoot. Now that everything is OK, you have the time to go over issues calmly and cooly to learn for the future!
Anyway, now that I’ve said that, here goes:
Troubleshooting multicast packet loss should not be different than troubleshooting packet loss in general. Multicast doesn’t inherently add any particular characteristics to packet loss that would not be present under other circumstances. Both the Mstat and the Mtrace utilities are useful for troubleshooting multicast routing failures and to examine the behavior of a multicast topology including statistics and choice of path, however, they don’t deal with all types of packet loss. Only packet loss that may be due to faulty multicast routing.
Unless your packet loss is due exclusively to faulty multicast routing, packet loss is typically due to other issues independent of multicast, that have to do with congestion, corrupted packets, MTU issues, or misconfigurations in ACLs route maps, or other policies.
Indeed, Wireshark is not so helpful here because of the reasons you give, but also because of the fact that you need to know where to collect packets on the network. You have to first zero in on the problem and then choose what packets to capture.
However, for voice applications including protocols such as SIP, RTP etc, Wireshark can be VERY helpful because it has a great telephony utility where you can follow a whole conversation and see the SIP exchanges and the voice packets of a particular conversation. It analyzes in detail these exchanges In such a case, you can examine the quality of the voice conversation. All of these tools are found under the “Telephony” menu option in the application.
I have a question regards to how a receiver can join a mcast group during register suppression window.
Lets assume a publisher is trying to register the source with RP and currently no active recipients are there.
So RP sends a register STOP message to source and a register suppression timer starts.
During this period, if an active receiver sends a PIM join message for that group to RP, how long the receiver need to wait to receive the MAST stream? Does it need to wait till the end of register suppression timer window ? Does Source sits idle til the timer expiry before it cant start sending the stream?
This is an excellent question. If a subscriber sends a join message to its local router, this will be forwarded as expected to the RP. If a PIM Register-Stop message had previously been sent to the first-hop router (the router connected to the multicast source), the RP will typically wait for the first-hop router to send a new PIM Register message. This is assuming a strict adherence to the “rules” of this operation. This means that the host wanting to join will have to wait for this timer to elapse so that the first-hop router will start sending multicast traffic to the RP once again.
However, as you have suggested in your post, this seems inefficient. So I tried labbing this up, and found that if a join message reaches the RP during that timer, the RP will automatically send a join message to the first hop router, informing it that it wants to receive traffic. The first hop router in turn will ignore its timer and will begin sending multicast traffic.
This is highly vendor and platform specific, and it may be that some IOS versions or other vendors don’t do this while others do. It is likely that most modern implementations do this to avoid the inefficiencies involved.
i work in a low latency trading environment where multicast is heavily used.I was giving a presentation on PIM JOIN and REGISTER and one of the dev guys asked this question and i was clueless for some time.I will get back to him with this explanation
Thanks again
Bejoy
I am practicing configuring PIM parse-mode on multi layer switches and had a few questions regarding this type of implementation. I am using Cisco modeling labs.
Assume we were using a single multi-layer core switch (Core-1), and 2 access layer switches with a multicast source, and a few multicast receivers. The multicast Source and the first multicast receiver are connected to the first access switch, on the same VLAN, lets call this VLAN 5. The second multicast receiver is connected to the second access switch on a separate VLAN, lets call it VLAN 10. The multicast source is sending multicast packets with a destination IP of 239.0.0.8 that both receivers would like to receive.
Assume the default gateway for VLAN 5 is an SVI of 192.168.5.254/24 on the core switch, and an SVI of 192.168.10.254/24 for VLAN 10’s gateway.
I am using two routers as my receivers, I turned off routing on them, configured an igmp join group for 239.0.0.8 on their interfaces, and assigned them a default gateway so that they can communicate with other subnets.
Heres my config for the core:
Core-1:
ip multicast-routing
int vlan 5
ip address 192.168.5.254 255.255.255.0
ip pim sparse-mode
int vlan 10
ip address 192.168.10.254 255.255.255.0
ip pim sparse-mode
ip pim rp-address 192.168.10.254
The configuration on the access switches are just access ports on each respective VLAN for the Multicast source and receivers, and the uplinks are configured properly to allow frames from the proper VLANS across to the core. IGMP Snooping is also configured for each VLAN in the access layer switches.
I am not able to route multicast traffic with this current configuration, and I dont even see what I was expecting to see when I do sh ip mroute on the core, but at layer 2 multicast traffic flows within the same VLAN fine, and I see the receivers respond when I use a ping to 239.0.0.8 on my source. It may just be the cisco modeling labs itself, but my questions to verify my work are the following:
Do i have to do anything on the access layer switches for this to work? My thoughts are that as long as they each have their mrouter port from receiving PIM hellos or igmp membership queries, they are able to forward multicast traffic to the core.
Does all multicast traffic regardless of the igmp snooping mechanism get forwarded towards the mrouter port?
What are you using to determine if you are able to route multicast traffic? Also, what do you actually see as the output of the show ip mroute command?
Multicast will flow just fine in Layer 2 switches, even if you don’t configure the mrouter port, it will simply flood all multicast traffic as if it was broadcast.
Now having said that, your configuration seems fine at first glance. Here are the responses to your questions:
In a typical scenario, you don’t need to do anything on the access layer switches for multicast routing to work. They should be able to forward multicast traffic to the core switch as long as they have their mrouter port from receiving PIM hellos or IGMP membership queries.
In a typical IGMP Snooping setup, all multicast traffic is forwarded towards the mrouter port, assuming it is configured. IGMP Snooping is a mechanism to constrain the flooding of multicast traffic by dynamically configuring Layer 2 interfaces so that multicast traffic is forwarded to only those interfaces associated with IP multicast devices. Therefore, when a switch receives the IGMP membership report from a host for a particular multicast group, it adds that host’s port number to the associated multicast table entry. It then forwards future frames for that multicast group directly to that host by forwarding the frames out the port connecting to that host.
As you suggested as well, one thing to note is that CML may not fully support all multicast routing features and functionalities, so the problem may be due to CML’s limitations.
Let us know how you get along, and if we can be of further help!
@lagapidis
im somewhat confused about packet encap level
i think i have good grasp about the overall process igmp + pim but i’d like to focus on the pim register msg :
S1 forwards mcast traffic to 239.1.1.1 grp , its received by the DR, R1 who sends a PIM register msg to the RP and this is the phase i’d like to do a pause… 1st the DR previously had to discover the RP (static, auto-rp (sparse-dense or auto-rp listener) or bsr) so when the DR already knows how to reach the RP it can send the PIM Register msg to this RP … and for what i know and read on the correspondent lessons, this msg is unicast (https://www.cloudshark.org/captures/611435f7d00d)
Src: 192.168.12.1 (DR), Dst: 2.2.2.2 (RP)
So in a nutshell the encap here is : ICMP → IPv4 (SRC 192.168.1.1 DST 239.1.1.1) → PIM (if we expand it we can see the type is a register (1) , and flags where we can see “0100 … = IP Version: IPv4 (4)” as one of its flags , does it mean PIM is encapsulating the previous IPv4 mcast packt ( SRC 192.168.1.1 DST 239.1.1.1) ? → IPv4 unicast (Src: 192.168.12.1 (DR), Dst: 2.2.2.2 (RP) → L2 Ethernet → L1
This was for Register (1) , Register-stop (2) is basically the same except as the lesson mentioned is does not contain any payload (ICMP for this example) so Flags is hex Flags: 0x40000000 and the Null-Register Bit is 1 (yes)
EDIT : I also have a question about Tunnel0 and Tunnel1. Tunnel0 is only on non-RP Routers for forwarding this kind of message ? which kind of tunnel is this ? Tunnel0 and Tunnel1 is in RPs for both, encap and decap … but i think im missing something here.
Now the interesting thing is that I have been going through some of the PIM RFCs, and looking at the header structure of the PIM register message (which is type 1 as seen in the Type field). I haven’t been able to find any information about the PIM Options fields! The flags are not defined, and they don’t seem to exist in the RFC. If anyone has any info about this, please feel free to post!
I personally believe that the flags in the PIM message indicate the IP version of the original multicast packet, so it means that the PIM is encapsulating an IPv4 multicast packet.
The Register-Stop message, as you mentioned, doesn’t contain any payload. It’s sent by the RP to the DR to signal that it should stop sending Register messages for a particular source/group pair. This happens when the RP has learned about the source via the normal multicast routing process.
As for your question about Tunnel0 and Tunnel1, these are commonly used in PIM implementations to handle PIM Register messages. Tunnel0 is typically used on the DR to encapsulate the original multicast packet within the PIM Register message, while Tunnel1 is used on the RP to decapsulate the original packet from the PIM Register message.
I have a scenario that requires configuring multicast with multiple vlans using only L3 switches. I am using a Cisco C9300X-24Y-A fiber switch as the core. The access or leaf switches are all Cisco C9200L-48P-4X switches. The C9300X is the FHR and LHR and no traffic needs outside of these L3 switches. My solution is this:
Enable ip routing and ip multicast routing on the Cisco C9300X
Create the required VLANs and SVIs for the multicast traffic on the C9300X.
create vlan number
name vlan name
interface vlan number* ip address ip address* subnet mask
Enable PIM sparse mode on each of the SVIs ip pim sparse-mode
Configure an IP address on each switch in an Management VLAN with a default route configured on the C9200L switches pointing to the C9300X switch.
Create and configure trunk links from the C9300X to the C9200L with the required VLAN
Enable IGMP Snooping per vlan or per port on each C9200L switch.
Configure each access port on the C9200L to be in its respective VLAN.
Your solution seems to be on point and should work for your scenario. I assume that the 9200 switches are only functioning at Layer 2, correct? Only the 9300 is operating at layer 3, and that’s why it is the FHR and LHR, right?
Assuming that is the case, I would only suggest that after enabling PIM sparse mode on each of the SVIs, you should also designate the 9300 as the rendezvous point (RP). This is necessary for PIM sparse mode to work correctly. You can use the ‘ip pim rp-address’ command to do this. Otherwise, it all looks good! Let us know how your implementation goes.
Thank you for the reply. So, yes the 9300 is L3 and the 9200s are L2. My follow up question is that if I designate the 9300 as the RP, since this network is small enough I can manually configure the RP on each switch, but I would have to create a route to the RP?
Yes, you are correct. If you are designating the 9300 as the RP, you can manually configure the RP on each switch. That way, because your network is small, you can avoid using AutoRP or BSR which automatically pick and advertise the RP. This will simplify your config.
As for your question about creating a route to the RP, yes, you have to make sure that the IP address that is used as the RP is reachable from all devices. Just make sure that the routing protocol or the static routing you configure allows all devices to be able to route to that address.
