MTU Troubleshooting on Cisco IOS

Hello Neves

By the context of your words, I assume you mean trunk interface and not tunnel interface?

The ip mtu and ip tcp adjust-mss commands can only be applied to a Layer 3 interface. This is because the first of these changes the way a router will re-encapsualte a packet into a frame. If the IP MTU is set to a value smaller than the size of the IP packet, that packet will be fragmented into two or more frames during encapsulation from Layer 3 to Layer 2.

If we’re talking about a layer 2 switch with a trunk interface, then neither of these commands are relevant. The only MTU-related parameter for such a situation is the interface MTU. This is the size of the largest allowed frame on the interface. If any larger frames attempt to traverse the interface, they will be dropped.

I hope this has been helpful!

Laz

Hello Marcus

Your explanation seems correct… Take a look at these posts for additional information:

I hope this has been helpful!

Laz

1 Like

without trunking the total MTU size is 1518Byte,including FCS…RIght …
and with Trunking total MTU would be 1518+4=1522Byte…Right ??

Why have you mentioned that the normal MTU size is 1514 byte …why are you excluding FCS ??

Hello Narad

When we talk about MTU, we have to make sure that we’re always using the right context. Sometimes it includes the Ethernet header, sometimes it does not. For example, when you configure the IP MTU or the interface MTU on a Cisco IOS device, it never includes the Ethernet header. This diagram shows this clearly:

image

Whenever we configure these values, we only take into account the payload. So a “normal” Ethernet frame will have an MTU of 1500, but the extra 14 bytes of the header are assumed.

Now if you have a trunk port, the MTU size still remains 1500, because the extra 4 bytes of the VLAN tag are considered part of the Ethernet header. So you would still have an MTU of 1500, but the interface knows to allow for a header size of 18, so the total frame size will be 1518.

However, if you are using a feature such as QinQ, which adds a second VLAN tag, you must change the MTU of the QinQ switches to at least 1504 to accommodate that additional VLAN tag. Why? Because they don’t detect the internal tag of 4bytes, so they expect a size of 1500+18=1518, but they receive a frame of 1500+18+4 = 1522 so they drop it.

To add confusion to the whole mix, Cisco IOS XR software includes the L2 header in the interface mtu command. So in such a case, you would use the value of 1514 for a “normal” frame on a Cisco IOS XR device, while you would use a value of 1500 on a Cisco IOS device. More info about this is found in the MTU size in Cisco IOS XER and IOS software NetworkLessons note.

In all cases, however, the FCS is never included in the calculation of the MTU. The only time this would be taken into account is if an Ethernet frame is encapsulated within another type of frame (MPLS for example) where the FCS is included in that encapsulation.

I hope this has been helpful!

Laz

1 Like

Hi,

I need to know header for ethernet packet is 22 bytes but u mentioned 14 bytes & 4 additional bytes for 802.1 tagging so then also it will be 18 bytes not 22 bytes.
Pl. explain

Hello Nitin

This is a good point you bring up. If you take a look at the header as described in the Introduction to Ethernet lesson, you will find the following fields:

image

If you count up all of the green fields, you will indeed get 22 bytes. However, for the purposes of the MTU, the Preamble (7 bytes) and SFD (1 byte) are not counted in the total bytes of the frame. Why? Because strictly speaking, the Preamble and SFD are considered to be part of the Physical layer encapsulation. So really, at Layer 2, which is the Datalink Layer, as far as the MTU calculations are concerned, you only have the following:

image

So when doing your calculations, you use a value of 14 bytes plus 4 additional bytes if you have an 802.1q tagging involved.

I hope this has been helpful!

Laz

The IP MTU 1400 command is talking about the IP payload correct? I was pretty confused because I was wondering if the IP header was fragmented that would break everything.

Hello Justin

The ip mtu command includes the IP header and the payload. For example, ip mtu 1400 will limit the size of the IP packet to 1400 including the header. However, this is not a problem because the range of allowed values is from 68 bytes to the interface MTU size. So, if you have set your MTU interface size to 1496 for example, then the allowable range of size for the ip mtu command will be between 68 and 1496 bytes, as seen below:

R1(config-if)#mtu 1496
R1(config-if)#ip mtu ?
  <68-1496>  MTU (bytes)

A minimum of 68 bytes will never cause an IP packet to be fragmented so much that the header itself would be fragmented because as you say, that would indeed break everything!

I hope this has been helpful!

Laz

1 Like

Why we use mss adjust?
When another header need to be added, will take place from mss and mss will not pass more than 1360, am i right?

I mean if i have big data, will the mss pass more than 1460 and take from header size? I think no, mss size will still fixed (1360) and headers 40= 1400,

Hello Ali

If you are adding more headers, like when you are using a GRE tunnel or QinQ, then there is no way that you can decrease the MSS directly. Remember when you add headers you are adding them during encapsulation. The MSS is defined at Layer 4 (Transport) so if you are encapsulating in an IP packet, and then a GRE header, and then QinQ where you add four more bits for the second tag, then you are not taking any bytes away from the TCP segment. You are adding more to the whole size of the frame.

The adjust-mss feature allows you to limit the size of the MSS on any TCP session that is traversing that particular interface. If you know what headers are added, you can easily make the adjustments here.

Alternatively, a TCP host can use Path MTU Discovery (PMTUD) to determine the proper MTU to use. More about this can be found at the following Cisco community thread:

PMTUD is not always applicable because some routers in the path may have disabled ICMP responses so PMTUD will fail in such cases.

Keep in mind that TCP hosts are able to adjust their MSS during the initial negotiation of TCP as well as during the TCP session as needed. You can find out more about adjusting the MSS at this NetworkLessons note.

I hope this has been helpful!

Laz

I have a question here . I was running RFC2544 test( that test checks throughput, latency and jitter) using TBERT equipments from the CORE router to the router on customer side . when i run the test i add vlan on both side and I create tunnel through the leased line (1G ) and i run the test for all different frame size . all frames passed except jumbo frame (9000 MB) it is failed . the leased line provider is configured as a Transparent tunnel . My question here is what would be the main problem ? Also what failed in test is latency , throughput and jitter ?. the RFC2544 test is only L2 test

Hello Ramy

You say that all frames passed except for jumbo frames at 9000 bytes, correct? Does that mean that frame sizes of 8999 pass through? What is the maximum frame size that is able to pass?

Secondly, if you have latency and jitter problems, you have to examine the possibility of congestion on the network, or any high CPU usage on any of the network devices. Generally, the jitter and latency problems are independent of the MTU issue you are facing.

Let us know about the above questions so that we can help you further in your troubleshooting.

I hope this has been helpful!

Laz

Hello Laz,

the device settings(RFC 2544 tester) runs test on several frame size which are (74,132,260,516,772,1028,1284,1522,1600 and 9000)MB.9000 frame result failed totally so i do not see the options of 8999 frame size . As a customer, we are provided a Dedicated 1G Leased Line, so this Leased Line is not shared with any other customer.

Is it possible that there is a Mismatch in MTU between RFC 2544 tester and provider devices?
Also, is this something related to PMTU (which might not be supported or disabled by provider equipment)?

Thanks for your response

Hello Ramy

It may be helpful to find out what the largest MTU size permissible on the link is. It may give a clue as to what is causing the smaller than expected size. To determine this, you can use an extended ping feature called Sweep range of sizes. For more info on how to do that, take a look at this NetworkLessons note on the topic.

Once you determine the maximum MTU, you can then examine where on the path the restriction takes place. However, in order for us to help you more, you’ll have to provide us with some more information about your setup. What devices are actually set up between the tester and the destination? Let us know so we can help you further.

I hope this has been helpful!

Laz

Hello Laz,
I solved the issue . there was a mismatch between the ethernet service MTU and the tunnel that is carrying the service (physical ports between both ends MTU ) . after change the MTU size the test has passed .
ethernet service set to be smaller than port MTU due to overhead

Thanks for your response and your additional steps for troubleshooting

Regards

1 Like

Hi,

Am I right in saying that if you have two routers connected via a GRE tunnel, using ip tcp adjust-mss command on router interfaces that traffic passes through will mean no fragmentation occurs? i.e. if you set the mss to 1436 (1460 - 24 GRE and new IP header).

For example:
image

Thanks,

Sam

Hello Samir

This configuration of the ip tcp adjust-mss command will cause the S1 and H1 devices to negotiate a maximum segment size (on the Transport layer) of 1436 bytes for any TCP session they create. Indeed, any TCP session that is negotiated that traverses these interfaces will have their MSS values changed on the TCP SYN packets, resulting in a maximum segment size of 1436.

The maximum MTU size of the GRE tunnel when we take into account the inside IP header of 20 bytes, the GRE header of 4 bytes, and the outside IP header of another 20 bytes (total 44) is 1500-44 = 1456.

However it is also common practice to allow for another 20 bytes of “wiggle room” resulting in the use of 1456-20 = 1436 bytes.

So to answer your question, this means that there will be no fragmentation of the IP packets in such a scenario assuming there are no other mechanisms in place that will create an even smaller MTU for the link between R1 and R2.

I hope this has been helpful!

Laz

Hi Laz,

Thanks for replying.

I came to 1436 because I also deducted 20 bytes for the TCP header as I thought the mss setting was TCP payload only.

1500 - 20 bytes (outside IP header) - 4 bytes (GRE header) - 20 Bytes (inside IP header)- 20 bytes (TCP header) = 1436.

Thanks.

Sam

1 Like

Hello Guys,

I have on my ASA setup a routed-based vpn by using VTI interfaces. So i would like to know what is best practice to setup the connection tcp mss voor ipsec tunnel. Do i even need to adjust the TCP MSS, because By default the ASA sets the TCP MSS option in the SYN packets to 1380 for s2s IPsec tunnel. And i know that tcpmss forces the tcp connection to have a maximum segment size not larger than 1380 bytes. So should i leave it as it is, or is it necessary to adjust it.

sysopt connection tcpmss 1350 (1360)

regards,
Sul

Hello Sulemani

When using a route-based VPN with VTI interfaces on your ASA, it’s important to consider the impact of the MSS on the performance of the VPN tunnel. The default MSS value of 1380 bytes set by the ASA for site-to-site IPsec tunnels is typically sufficient for most scenarios. This value takes into consideration the overhead introduced by the IPsec encapsulation and helps prevent issues like fragmentation and inefficient use of bandwidth.

You should only make any adjustments if you perceive problems on the network. If you’re experiencing performance issues or connectivity problems on that particular VPN, then investigating the values of the MSS may be a good troubleshooting step. Otherwise, leaving the default values should be perfectly fine.

I hope this has been helpful!

Laz