MTU Troubleshooting on Cisco IOS

Lessons Discussion
41

Hi Davis,

R1 will have no problem sending this large packet but R2 will drop it since it exceeds its MTU.

Rene

42

Hi Rene,

So can i say that the Interface MTU we have to match at both R1 and R2 in order to use the jumbo frame. Else it will cause the packet drop.

Thanks

Davis

43

Hi Davis,

It doesn’t really have to match on both ends but when you look at it end-to-end, the interface with the lowest MTU will always be the “bottleneck”.

On a LAN, it’s best to use the same MTU everywhere if you want to use jumbo frames.

Rene

44

Ok. Thanks Rene.

Davis

45

Based on your picture what is the difference between IP MTU and MTU. I change the MTU to say 1000 but the IP MTU remains at 1500 bytes.

46

Hi Michael,

The difference is that MTU is the L2 MTU and IP MTU is the L3 MTU.

The L2 MTU is the maximum packet size that the interface supports. This could be an IPv4 packet, IPv6 packet or anything else. It the interface receives something that exceeds the L2 MTU, then it will be dropped.

The IP MTU is the maximum size of an IP packet that can be sent. When you receive a packet that exceeds the IP MTU, the packet will be fragmented.

If you set the (L2) to 1000 then your interface won’t accept any packet larger than 1000 bytes. Give it a try :slight_smile:

Rene

5 Likes
47

Hi Rene,
So you are saying there is a Layer 2 MTU (mtu that includes the ethernet frame and transport headers and data) and there is a Layer 3 MTU (that includes just the transport headers and data?) If thats the case why do I feel this a little known concept - that there are two MTUs technically. I always thought there was 1

48

Hi Michael,

That’s right, there are two MTUs…the L2 and L3 MTU. On a switch that supports Jumbo frames, there’s even a third MTU for that.

The L2 MTU doesn’t include the Ethernet frame header/trailer, it only specifies the size of the payload. This is probably why it is confusing since both MTUs are 1500 bytes by default.

The main difference is that the L2 MTU specifies what the interface is able to send, the L3 MTU specifies when we fragment the packet.

Rene

5 Likes
49

Hi Rene,

What are the uses of these ping options and when do we use them with examples :-
1 - Validate reply data.
2 - Data pattern.
3 - Loose.
4 - Strict.
5 - Record.
6 - Timestamp.

Best Regards
Hussein.

50

Hello Hussein!

Cisco gives a very good explanation of the extended ping options here:

In this article you’ll find a clear explanation about Loose, Strict, Record and Timestamp. However for the “Validate reply data” and “Data Pattern” the explanations are lacking in detail, so here’s the detail for you:

If the “Validate reply data” is checked, then this means that the router that originated the ICMP-Echo will check the ICMP-CRC in the Echo-Reply to confirm the integrity of the PDU being received. This would be used when you are testing a network (or a route within a network) for which you have a suspicion that errors are being introduced to your PDUs such as bad cabling, electrical wiring running parallel to your UTP or UTP runs of greater than 100m for example.

As for the Data Pattern, this is useful for testing serial connections/T1 and T3 circuits and links that require a clocking mechanism. The type of encoding used on T1 and T3 circuits replaces a certain series of zeros with something called Bi-Polar violations. This substitution, called B8ZS and B3ZS respectively, ensures that timing slips do not occur on the circuit. By using the Data Pattern option, you can test the ability of a WAN circuit to perform this zero substitution properly.

I went on my production Catalyst 6506 and performed an extended ping using Validate, Timestamp and Record. You can see the results below:

CORE_6506_A#ping      
Protocol [ip]: 
Target IP address: 10.96.28.10
Repeat count [5]: 
Datagram size [100]: 
Timeout in seconds [2]: 
Extended commands [n]: y
Source address or interface: 10.96.4.1
Type of service [0]: 
Set DF bit in IP header? [no]: 
Validate reply data? [no]: yes
Data pattern [0xABCD]: 
Loose, Strict, Record, Timestamp, Verbose[none]: timestamp
Number of timestamps [ 9 ]: 5
Loose, Strict, Record, Timestamp, Verbose[TV]: record
Number of hops [ 3 ]: 
Loose, Strict, Record, Timestamp, Verbose[RTV]: 
Sweep range of sizes [n]: 
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.96.28.10, timeout is 2 seconds:
Packet sent with a source address of 10.96.4.1 
Reply data will be validated
Packet has IP options:  Total option bytes= 39, padded length=40
 Timestamp: Type 0.  Overflows: 0 length 24, ptr 5
  >>Current pointer<<
  Time= 03:00:00.000 EET2DST (00000000)
  Time= 03:00:00.000 EET2DST (00000000)
  Time= 03:00:00.000 EET2DST (00000000)
  Time= 03:00:00.000 EET2DST (00000000)
  Time= 03:00:00.000 EET2DST (00000000)
 Record route: <*>
   (0.0.0.0)
   (0.0.0.0)
   (0.0.0.0)

Reply to request 0 (1 ms).  Received packet has options
 Total option bytes= 40, padded length=40
 Timestamp: Type 0.  Overflows: 0 length 24, ptr 21
  Time= 07:49:52.400 EET2DST (01096310)
  Time=*01:58:36.714 EET2DST (84EE282A)
  Time=*01:58:36.714 EET2DST (84EE282A)
  Time= 07:49:52.401 EET2DST (01096311)
  >>Current pointer<<
  Time= 03:00:00.000 EET2DST (00000000)
 Record route:
   (192.168.101.253)
   (192.168.101.2)
   (192.168.100.2)
   <*>
 End of list

Reply to request 1 (1 ms).  Received packet has options
 Total option bytes= 40, padded length=40
 Timestamp: Type 0.  Overflows: 0 length 24, ptr 21
  Time= 07:49:52.402 EET2DST (01096312)
  Time=*01:58:36.716 EET2DST (84EE282C)
  Time=*01:58:36.716 EET2DST (84EE282C)
  Time= 07:49:52.402 EET2DST (01096312)
  >>Current pointer<<
  Time= 03:00:00.000 EET2DST (00000000)
 Record route:
   (192.168.101.253)
   (192.168.101.2)
   (192.168.100.2)
   <*>
 End of list

Reply to request 2 (1 ms).  Received packet has options
 Total option bytes= 40, padded length=40
 Timestamp: Type 0.  Overflows: 0 length 24, ptr 21
  Time= 07:49:52.403 EET2DST (01096313)
  Time=*01:58:36.718 EET2DST (84EE282E)
  Time=*01:58:36.718 EET2DST (84EE282E)
  Time= 07:49:52.404 EET2DST (01096314)
  >>Current pointer<<
  Time= 03:00:00.000 EET2DST (00000000)
 Record route:
   (192.168.101.253)
   (192.168.101.2)
   (192.168.100.2)
   <*>
 End of list

Reply to request 3 (1 ms).  Received packet has options
 Total option bytes= 40, padded length=40
 Timestamp: Type 0.  Overflows: 0 length 24, ptr 21
  Time= 07:49:52.405 EET2DST (01096315)
  Time=*01:58:36.720 EET2DST (84EE2830)
  Time=*01:58:36.720 EET2DST (84EE2830)
  Time= 07:49:52.406 EET2DST (01096316)
  >>Current pointer<<
  Time= 03:00:00.000 EET2DST (00000000)
 Record route:
   (192.168.101.253)
   (192.168.101.2)
   (192.168.100.2)
   <*>
 End of list

Reply to request 4 (1 ms).  Received packet has options
 Total option bytes= 40, padded length=40
 Timestamp: Type 0.  Overflows: 0 length 24, ptr 21
  Time= 07:49:52.407 EET2DST (01096317)
  Time=*01:58:36.721 EET2DST (84EE2831)
  Time=*01:58:36.722 EET2DST (84EE2832)
  Time= 07:49:52.408 EET2DST (01096318)
  >>Current pointer<<
  Time= 03:00:00.000 EET2DST (00000000)
 Record route:
   (192.168.101.253)
   (192.168.101.2)
   (192.168.100.2)
   <*>
 End of list

Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/1 ms
CORE_6506_A#

If you have access to a Cisco router, I suggest experimenting with these commands and seeing the results.

I hope this has been helpful!!

Laz

51

Hi Sirs

Confused, is adjust-mss configured egressing out, and usually will face the provider PE router?
Reason being that is where the syn will transit from my router.
Is my understanding correct?

52

Virgilio,
Adjust-mss will work in either direction. So long as the TCP traffic that needs to be reduced traverses an interface where this is enabled, the adjust-mss will work. Obviously, there are good choices and poor choices as to where to implement this command when considering what kinds of traffic is flowing through. For example, supposing the whole reason you need to adjust the MSS is because of limitations with your provider, you might not want to put this command on an internal interface since all traffic flowing through that interface would be subjected to it. In this case, of course, putting it on an interface as close to the provider as possible would be a good idea.

53

Hi Sirs

Which utility in the webserver or host can i use to check for fragmentation?
Cant use icmp due to fragments being dropped at provider end.
Need your insights on this

I used ping x.x.x.x -s 1472 -c 10 towards the webserver.
Didnt receive and packet capture of the second offset in the packet capture
It then times out after the router having the gre sends an icmp reply to fragment

Provider advised me icmp fragments are dropped.
Looking for tools to check whether the fragmentation are working on the higher layers.
Thoughts ?

54

Hi Virgilio,

I would use Wireshark to capture the traffic. It will allow you to filter IP fragments. On Linux servers you can also use the tcpdump command and import the capture file later in Wireshark.

With the ping command you used, you can keep decreasing the MTU until you get an ICMP reply. This will tell you the maximum MTU size that is permitted. You also might want to check the MTU of your (Windows) computer:

C:\Users\info>netsh interface ipv4 show subinterfaces

   MTU  MediaSenseState   Bytes In  Bytes Out  Interface
------  ---------------  ---------  ---------  -------------
  1500                5          0          0  Wi-Fi
  1380                1    9209492    4846728  Local Area Connection* 4
  1500                1  278130242   18461304  Ethernet 2
  1500                1       8200     237624  VMware Network Adapter VMnet1
  1500                1       8200     237236  VMware Network Adapter VMnet8
  1500                5          0          0  Ethernet 4

Rene

55

Hi Rene,

Can you pls answer below question

Ethernet adds a minimum of 18 bytes, does it not? smac + dmac + type + FCS = 6 + 6 + 2 + 4 = 18. I only bring this up because of the conversation we had recently about some inconsistencies with fragmentation … and of course, the 8 bytes preamble added by the interface drivers. Wondering if Wireshark simply discards the FCS, but I am pretty sure the FCS is part of the actual Ethernet II frame and is associated with L2 encapsulation, not L1 like the preamble is.

56

Hello Rohitendu.

Yes, you are correct, that an ethernet header adds 18 bytes. This can increase if you’re using QinQ for example where you’re adding another VLAN tag that’s worth 4 more bytes so you’d go up to 22 bytes. If you add more QinQ layers, you go up by 4 bytes for each layer. Fragmentation will occur if you have a frame with an MTU that’s larger than the maximum allowed, so it’s a good idea to make sure all devices have MTUs large enough to prevent unnecessary fragmentation.

Keep in mind that when refering to the MTU in Rene’s configuration, we are NOT including the Ethernet header or the FCS, but just the payload.

I hope this has been helpful.

Laz

1 Like
57

Great write up, I’ve never really understood this and now I do.

Thanks man.

58

19 posts were merged into an existing topic: MTU Troubleshooting on Cisco IOS

59
It doesn’t include the Ethernet header of 14 bytes. The total “packet” size of 1514 bytes is the complete Ethernet frame, the IP packet itself is 1500 bytes.

this parh was mostly hard for me

in non-cisco equipment situation can be another, so this makes me missunderstood

thanks a lot

60

Hello Mikhail.

When you configure the MTU on a switch and set it to 1500, you are referring ONLY to the payload of the ethernet frame. The payload of the ethernet frame is an IP packet including the layer 3 headers. The header of the ethernet frame, or its layer 2 header is an additional 14 bytes. So the size of an ethernet frame with an MTU of 1500 is 1514.

I hope this has been helpful.

Laz

2 Likes