Thx for your reply, Actually i know all of these info, I will ask my Q’s in another way
could you please answer each one independently ?
during the handshake, for example both agreed on mss 9860 ( Header =40 bytes )
Q1:How did they agreed on 9860? what are the criteria that they took? is it just the interface mtu in all the path and the header? or there is somethings to be considered also?
i know that mss= mtu-header.
Q2: is it mandatory that the packet should be 9000 in our case ( if the header =40)? or it may be less and this depending on the payload size?
i mean if the payload is small, will the segment be small as well in our case? or it is mandatory to be as aggreged ( fix size ) always.
Q3:
Q4:When my payload is just 1460 and the mss is 9860, will the padding take place here mandatory?
Sure, no problem, I can respond to each question specifically.
The MSS is determined primarily based on the MTUs of the NICs of the sender and receiver. This process is further described in this NetworkLessons note. However, as mentioned in the note, the MSS can be influenced by using the TCP adjust-mss feature, where the MSS value in a SYN packet can be replaced with another value by an intervening router. So if a TCP sender and receiver agree upon 8960 bytes as the MSS, that means that their NICs support an MTU size of 9000. But this does not guarantee that the intervening network infrastructure supports this. The network designer must ensure that this is the case.
No it is not mandatory. The MSS defines only the maximum. The segment size can actually be anything up to the MSS value. So you don’t even need padding, you can have segments of varying sizes without any problem.
No there will be no padding necessary. The segment size of that particular transmission will be limited to the payload you have. Again, the MSS only defines the maximum size, not the size of all segments. Does that make sense?
clear, thx
but recently i faced issue which is strange for me, i thought that maybe i understood something worng, anyway, simply i have network with 9000 mtu in all the path ( Spine, Leaf and FW), but from server side 1500, my edge FW is Mikrotik and each server can contact other servers normally, when i shifted the service to the new FW which is Fortigate ( I have experience in Security), SSH and some services freezes between Servers, but they can reach internet normally, again all the path 9000, just servers 1500, Freeze issue mostly related to MTU but as far as i know it should be no issue as long as they will communicate the mss, but when we changed the MTU from server side to 9000, All pass and cleared with no issue, what do u think the issue? i couldn’t perform packet capture due to the time, because we had production services during the operation.
1500 pass with DF from server to the FW and old FW ( Mikrotik ) as well.
The first question that comes to mind is, are you sure that the problem is MTU related? You mention that the freeze issue is mostly related to MTU. The first thing to do is to determine conclusively that this is the case. Some ways you can do this is to use a ping sweep of various MTU sizes from one server to the other, to see if you do get a drop after a certain size.
This is the most obvious quick and dirty test you can do, but you may not be able to if your firewall drops ICMP (ping) packets. If this is the case, and you have control over the FW, during a maintenance window, you can temporarily allow ICMP to traverse the firewall to do that test.
Another thing that comes to mind with your description is the Path MTU Discovery (PMTUD) feature. This is a mechanism used by endpoints in a network to determine the largest MTU size that can be used for data transmission between them. If the network path between two endpoints supports an MTU size of 9000, but one endpoint is set to an MTU size of 1500, the smaller MTU size will be used for data transmission.
The problem arises when PMTUD fails, which can happen if a network device (like your Fortigate firewall) in the path between the endpoints is blocking ICMP messages. PMTUD uses ICMP to perform its function, and it would fail resulting in intermittent connectivity. Since in your case it is only happening using SSH (as far as you can see), it may be that SSH is using PMTUD and the FW is blocking ICMP packet with the results you see.
These are just some thoughts that may help you out in your troubleshooting. Doing a packet capture will give you even more information, so that will be helpful when you get a chance to do it.
Let us know how you get along, and how you ultimately got to the bottom of the problem and solved it.
In this lesson if we ping R1 without setting Df bit to set we will get reply to ping packet packet up to 1505 this is because R2 egress interface will fragment packet so the maximum packet that will leave R2 will have ethernet payload of 1500 bytes and R1 has MTU of 1500 bytes everything will work since R1 will see maximum ethernet payload of 1500 bytes.
Reply to request 0 (1 ms) (size 1495)
Reply to request 1 (4 ms) (size 1496)
Reply to request 2 (1 ms) (size 1497)
Reply to request 3 (4 ms) (size 1498)
Reply to request 4 (1 ms) (size 1499)
Reply to request 5 (4 ms) (size 1500)
Request 6 timed out (size 1501)
Request 7 timed out (size 1502)
Request 8 timed out (size 1503)
Request 9 timed out (size 1504)
Request 10 timed out (size 1505)
with df bit set packet above 1500 bytes will not leave R2 because its own interface mtu is 1500 bytes and df is set. This understanding is based on the concept I tried between two windows machine. for example If I change one windows host mtu to 1000 bytes and send ping packet to other windows host with ping size 2000 bytes with df bit set it would not leave the nic and prompt will show fragmentation needed message. when sent without df bit set pc will fragment the packet and the ethernet frame payload that will leave the nic will be less than or equal to mtu configured on the nic [1000 byes for example]
Let’s consider the below scenario on the same topology you have used in the lesson to understand how receiving router R1 will behave.
if we change the MTU on R2 to say 2000 bytes and keep 1500 bytes on R1.
will ping from R2 to R1 with size 1800 bytes without df bit set will succeed ?
will ping from R2 to R1 with size 1800 bytes with df bit set will succeed ?
I am guessing without df bit set R2 still will not need to fragment the packet because it has mtu of 2000 byte .With Df bit set R2 cannot do fragmentation so in both cases R1 will see a ethernet frame that would have payload of 1800 bytes.
How R1 with mtu of 1500 bytes would react in two scenario.
R2 is sending ping with 1800 bytes with df bit set >>> will ping successful ? I think no. R2 is sending ping with 1800 bytes without df bit set >>> will ping successful? I guess yes.
In both cases R1 is seeing ethernet frame from R2 with payload that is exceeding R1 mtu but how it responds will be different in my opinion.
Kindly share your thoughts if my understanding is correct.
This is an interesting question. I went into the lab and tried a few scenarios and packet captures using CML. I found the following:
Both scenarios seem to fail. I thought the reason was because of the IP MTU setting which has a maximum size of 1500 on the CML device I’m using. So I changed your scenario a bit.
I set the MTU on Fa0/0 of R1 to 1000 and the MTU of Fa0/0 of R2 to 1300. I kept the IP MTU at 1500 on both.
I tried pinging from R2 to R1 using a size of 1100 without the DF bit set, and it still failed. I was only able to ping up to a size of 1010. From 1011 and higher it failed. I did a packet capture and found that as long as the DF bit was not set, I could send out echo requests of any size, and for sizes over 1000 fragmentation took place. I tried 1000, 1200, 1500, even 1700 which is above the IP MTU, and all of them successfully sent out an echo request. However, the receiving router seems to fail to respond for any size larger than 1010 (with these settings).
However, with the DF bit set, I am only able to ping up to a size of 1000. For a size of 1001 I am unable to. The packet captures show that the echo requests are sent but the replies are not received.
I believe this has to do with how ping interprets the size and the DF bit setting for the echo reply. I believed the size and the DF bit settings should be the same for both request and reply, but this may not be true. It could also have to do with CML as an emulator. It would be interesting to see this on real devices as well…
I will ask Rene to take a look as well and see what he can interpret from these results.
Hello Laz,
can you please trying doing the original scenario on real router ?
I performed test in packet tracer. In packet tracer there is no option to set Df bit so without setting DF bit I was able to successfully ping 1300/1500 size from router with mtu of 1500 to destination router with mtu of 1000. Furthermore, I was able to ping a neighboring router which was connected to a destination router (R3 in diagram below) on the other end that means without df bit set router with lower mtu was accepting packet regardless whether the packet is destinated for its interface or it is destined remote router it is successfully passing the ping packet.
R 1500 mtu >>>>>>>>>> R 1000 MTU>>>>>>>>>>>>>>> R3 15000 MTU
I also tested the MTU concept between two windows devices and the results were interesting.
I reduced MTU on destination host to 1000 and kept 1500 on source host.
In both cases df bit set and unset I was getting reply for ping packet those were exceeding the destination host mtu for example 1200, 1300 1400 1500.
It is important to note that in ping response packet destination host will not put any thing on wire that is greater than it’s own mtu . so ethernet frame payload from destination will be less than or equal to 1000 bytes and it would do fragmentation. Destination is not dropping ping packet.
I wanted to see how real router would behave since ping is successful in packet tracer without df bit set. In your test in CML even without DF bit set it is not successful.
you do not need to change the IP mtu once you reduce the mtu, IP mtu will automatically change and in real world IP mtu should be equal to or less than layer 2 MTU.
Thanks for the additional information, all of it is quite useful. The results you got from packet tracer are as expected as you suggest, since regardless of the MTU settings, if the DF bit is not set, fragmentation takes place and the ping is successful.
My CML results are somewhat strange, so wouldn’t put too much emphasis on them. After chatting with Rene, a couple of 4331 routers were set up to do some tests. For Cisco IOS the following has been confirmed:
When you set the ping size, the size is the same for both echo request and echo reply.
When you set the DF bit, both the echo request and the echo reply have the DF bit set.
So knowing this, the original tests you suggested were performed:
Doing the original tests that you requested, we found the following results:
When pinging with the DF bit set, the ping is successfully sent because it fits in the L2 MTU of the egress interface. However, R1 will drop the frame upon ingress because its L2 MTU is 1500 bytes.
When pinging without the DF bit set, the echo request is sent as a whole frame just like in the first case, but again, R1 will still drop the packet upon ingress because the L2 MTU is too large.
The idea here is that even if the DF bit is not set, you can’t fragment an incoming frame. you can only fragment it in an outgoing direction. Does that make sense?
It is possible that other operating systems like Windows or Linux behave differently, but Cisco IOS 4331 routers drop it. You can find more detailed information in this NetworkLessons note that details the test.
