When configuring jumbo frames, it is imperative that all devices within a network be configured with the same jumbo frame size. Implementing different frame sizes can cause extensive fragmentation and intermittent connectivity issues that are often difficult and time-consuming to troubleshoot. Typically, ISPs will enable jumbo frames on all of their internal networks, and the networks that they have administrative privileges over. It’s easy for them to ensure such consistency.
But at locations where they interconnect with other networks, including customer networks, it is a good idea to use the standard 1500 bytes. This way, you are sure that all other devices, regardless of vendor, will be compatible.
ISPs want to minimize compatibility problems, especially ones that are so difficult to troubleshoot, and such a policy achieves this.
Now if you have a long-standing relationship with the ISP, and they trust you, they may make exceptions to accommodate your needs. But this is on a per-user and per-requirement basis.
Thanks Laz - I understand that wanting to ensure compatibility is the main reason for enabling the default of 1500 on network interconnects. Are there any extra resource demands on the ISP routers (memory/CPU etc) when passing jumbo frames on their internal networks? Thanks - Gareth.
There are no extra resource demands on the ISP networks when processing jumbo frames on their networks. Indeed, quite the opposite! Processing the contents of a single large frame rather than many smaller frames saves on CPU processing time, as well as on network overhead since you have fewer Ethernet headers. If configured consistently, you can get up to a 6% increase in network efficiency assuming you are using an MTU of 9000 and the majority of your frames are indeed 9000 bytes in length.
The only area that jumbo sized frames may affect has to do with QoS. Switches have a specific amount of buffer size for outgoing frames, and with jumbo frames, any QoS feature will be able to queue fewer frames due to their size. For example, the Cisco catalyst 2960 has a 2MB egress buffer, which means it will be able to hold just over 220 jumbo frames (at 9000 bytes each) while it can hold over 1330 normal sized frames. This may slightly affect CPU as fewer buffered frames will need to be kept track of, but again, that’s assuming all frames will be that size, something that is very rare. In such a network, there will still be frames of smaller sizes, and indeed, most will remain small. I’m just thinking out loud here…
The root of the problem is that jumbo frames are not part of the IEEE 802.3 Ethernet standard, and thus may be implemented in different ways by different vendors, and even by the same vendor in differing switch models! In general, these variations can be dealt with, but must be done so with care to avoid the problems mentioned before.
Sending 1514 bytes is no problem for Ethernet but some other mediums might have issues with large MTU values.
With the above statement, If the default ethernet mtu is usually 1500 bytes and from what I gather 1514 bytes is the normal frame size then is every regular default frame not 14 bytes over size ?
When referring to the MTU of an interface, or the IP MTU, this value never includes the 14 bytes of the Ethernet header. This is just a convention that is used. The value of 1514 should be kept in mind when you are encapsulating Ethernet within other protocols such as when you tunnel Ethernet over GRE, or over MPLS.
After some Reading Further I have understood that Fragmentation actually Happens at L3 Only. Fragmentation is Not Possible At Layer 2 Am I right? So Switches will not be able to fragment at L2
If I change the L2 MTU and L3 MTU, will that Impact Inbound traffic or Outbound traffic?
e.g if MTU = 1500 and Ip MTU -1500 and if a ping request comes with 2014 Frame size then will the router process it ? or drop in in ingress queue as MTU Size is higher than what is configured.?
Fragmentation takes place during the encapsulation of Layer 3 IP packets into Layer 2 Ethernet frames. If a Layer 3 IP packet is too large to fit into one Layer 2 Ethernet frame, it must be fragmented and encapsulated into two or more frames. The frames cannot be fragmented further, because they are at the bottom of the OSI model, and only the Physical layer is below them. So frames are simply encoded onto the medium.
Think of the frames as trucks of a specific size, and the IP packet as a shipment of wood. If the shipment is too long to fit in one truck, it will be split in two, and put into two separate trunks. The trucks themselves cannot be split further, but are simply put on the road to transport their payload.
Take a look at this post, I believe you will find your answer there:
So ,the ip tcp adjust-mss 1360 applied to R2 f0/1 interface changes the MSS so that when S1 sends back traffic to H1 its responses wont exceed the MTU on R2 f0/0. However, if S1 initiates the connection to H1, the MTU issue will still occur - is that correct? Or will the MSS be adjusted in the return syn/ack?
I’m also unsure how the command below fixes the issue:
R2(config)#interface fastEthernet 0/0
R2(config-if)#ip mtu 1400
The MTU command that you describe here configures the interface MTU of the SVI interface. Remember that an SVI interface may be virtual, but it too has the same configuration parameters as a physical interface. The mtu 1490 command causes the interface to allow Ethernet frames with up to 1490 bytes in size. Anything larger will be dropped.
For more information about the MTU and its meaning, take a look at this lesson:
Hi all,
im trying to make some test on my LAB, i have customer switch model WS-C2960L-24PS-LL, if i try to change mtu size to 1504 the only possibilities it offers are:
1522 Set MTU to 1522
2048 Set MTU to 2048
jumbo Set MTU to 10240
While at the core switch L3 C9400-LC-48U it gives me the possibility to change it to 1504.
my question is, if i put 1504 on the core and 1522 on the customer side the qinq should work correct?
First of all, the fact that it only gives you specific values for MTU is interesting. I’ve found documentation that shows similar options for MTU sizes. Just for verification, can you share your IOS version with us?
Secondly, this is probably the system MTU that you are configuring, right? I don’t believe the 2960 has the capability of changing the interface MTU. Having said that, the system MTU should be set to 1500 on the customer switch. When you connect a trunk from the customer equipment to the ISP (core), you add a 4 byte tag, so the MTU of the core network should be 1500+4 = 1504.
From your post, it seems that 1500 is not an option? Is that the case? Well, I suggest you try it out in your lab and check to see what works and what doesn’t, and if you need any more troubleshooting help, let us know.
Hello,
thank you for your respons, the version of customer switch is: Cisco IOS Software, C2960L Software (C2960L-UNIVERSALK9-M), Version 15.2(6)E, RELEASE SOFTWARE (fc4)
You are correct, its a system MTU, at the moment is :
show system mtu
System MTU size is 1522 bytes
The only options are:
system mtu ?
1522 Set MTU to 1522
2048 Set MTU to 2048
jumbo Set MTU to 10240
Since the default is 1522, I believe that this is including the header of the ethernet frame as well. In any case, leaving the MTU of the customer devices at their defaults, and making sure that the interface MTU of the ISP switches is at least 1504, you should be OK to pass the QinQ frames. You can implement it and test it out, and check with pings of 1500 bytes in size to ensure that they are indeed going through.
An interface that has the L2 MTU set to 1000 won’t accept any frame larger than 1000 bytes. Incoming frames cannot be fragmented. In such a situation, an interface will accept the frame if it is 1000 bytes or smaller, and will drop it if it is 1001 bytes or larger. This is the behaviour will be seen for incoming and outgoing frames.
Note that it is the IP MTU configuration that tells the device if and when to fragment IP packets. This value should be set correctly so that the router will know when to fragment packets in order to accommodate any L2 MTU settings of the interfaces used.
Hey Laz, hope you are well.
GRE tunnel would add one additional 20 bytes outer IP packet (tunnel source and destination) other than 4 bytes GRE header. Original 20 bytes IP header + 20 bytes TCP header
If I have config as such,
interface TunnelXXX
ip mtu 1400
ip tcp adjust-mss 1360
To avoid fragmentation, should the IP tcp adjust-mss on my GRE tunnel intf be 1360-20 (additional IP header introduced by GRE)= 1340 if I want to keep ip mtu of 1400? Or this additional 20 bytes header is excluded from this GRE interface so 1360 would be still good for mss value? Thanks
Yes you are correct, there would be the “outer” IP header outside of the GRE header with the source and destination addresses of the GRE tunnel. Here is the revised diagram:
For the scenario that you are describing, if you want to achieve an IP MTU of 1400, you would still only need to configure the MSS to 1360. The outer IP header does not come into the calculation. That extra 20 bytes is beyond the 1400 bytes of the IP MTU, and since typically your interface MTU is at 1500, you have plenty of room for those 20 bytes.
