Hello team,
For a L2 Etherchannel, if I have 4 links configured for an etherchannel and the 3rd link goes down than how would the ratio change. If it is 4 link than it is 1:1:1:1, if the last link is down, it is 2:1:1, but what if one of the middle link is down and not the last link.
Sincerely,
Jugal Modi
Hello Jugal
The ratios that you are suggesting here are those used by the 6500/6000 series Catalyst switches using the CatOS operating system. In this system, the hashing algorithm is used to compute a value between 0 and 7. Using this value as a basis, a specific port in the PortChannel is chosen to be used. When you set up the ports, you also use what is known as a mask, which is the ratio notation that you shared.
Now these switches use specific load balancing ratios depending upon the number of ports in the EtherChannel. CatOS supports up to 8 ports, so the corresponding load balancing masks are as follows:
Number of Ports in the EtherChannel Load Balancing Mask
8 1:1:1:1:1:1:1:1
7 2:1:1:1:1:1:1
6 2:2:1:1:1:1
5 2:2:2:1:1
4 2:2:2:2
3 3:3:2
2 4:4
Note that the CatOS is rarely used today.
The more modern devices that use Cisco IOS and IOS XE, load balancing algorithms donât follow this logic of using such load balancing masks, but rather on a hash result of the packetâs header. This means that if a link goes down, the traffic that would have been sent over that link is redistributed generally evenly over the remaining links.
So, if you have a 4-link EtherChannel and one link goes down, regardless of whether itâs the first, second, third, or fourth link, the traffic will be redistributed over the remaining three links according to the hash algorithm.
For more information on how load balancing is achieved, and how it differs from platform to platform, take a look at this Cisco documentation:
I hope this has been helpful!
Laz
Hello, everyone.
I have a few rather complex questions regarding EC (Etherchannel).
With PAGP, what exactly does the PAGP priority do? From my understanding, a port with a higher priority will transmit PAGP messages? But this doesnât make much sense to me. Shouldnât all ports do this regardless of the priority? Since thatâs how you know whether the port and the link are still active/whether they need to be removed from the bundle.
The PAGP timer is weird, too.
Iâve configured it to be 45 yet the neighbors are still alive even after this timer expires?? Itâs always a bit higher than what I configure it to be.
Thatâs all, thank you.
David
Hello David
You have some great questions regarding EtherChannel and PAGP. Letâs dive into them:
PAGP Priority: The PAGP priority is used to determine which end of the EtherChannel will become the master. The device with the lower priority value will become the master. The master device is responsible for sending PAGP packets to its partner device to negotiate EtherChannel formation. While itâs true that all ports should be transmitting PAGP messages, the priority is used to decide which end takes the active role in the formation of the EtherChannel.
PAGP Timer: The pagp timer command is used to determine after how long PAgP informational packets are considered valid.
The age value indicates the time in seconds since the last PAgP packet was received from the neighbor. Now it does seem strange that your Age value should exceed the PAgP timer. I labbed this up and did some experimentation and in no case did the value exceed 30 seconds. And this makes sense, because there is another value that is involved here, which is the hello timer. This has a value of 30 seconds, and cannot be changed. You can see this value in the output of the show pagp internal command. Can you share with us the full configuration of the physical interfaces and the portchannel interface of your configuration? For your reference, here is the command reference for the pagp timer command:
PAGP Learning Modes: Youâre correct in your understanding of the two learning modes. In physical mode, MAC addresses are learned on the physical port, while in aggregate mode, MAC addresses are learned on the Port-Channel interface. I tried recreating your scenario by configuring the learning to physical on the interfaces in CML and found I got the same results as you. I donât know why, but Iâd be interested to try this out on real devices at some point.
I wish I could have been more helpful with your questions, but in the meantime, I hope this helped you to further understand the technology of EtherChannel.
I hope this has been helpful!
Laz
Hello Laz
PAGP Priority: The PAGP priority is used to determine which end of the EtherChannel will become the master. The device with the lower priority value will become the master. The master device is responsible for sending PAGP packets to its partner device to negotiate EtherChannel formation
I might have gotten this part wrong. Here is what a Cisco doc says:
You also can configure a single port within the group for all transmissions and use other ports for hot-standby. The unused ports in the group can be swapped into operation in just a few seconds if the selected single port loses hardware-signal detection. You can configure which port is always selected for packet transmission by changing its priority with the pagp port-priority interface configuration command. The higher the priority, the more likely that the port will be selected.
So it basically determines which port will forward.. all of the data? Iâve tried to lab this up in CML but I saw other ports being used as well. Is it possible for you to verify this too, please?
Now it does seem strange that your Age value should exceed the PAgP timer.
I was running a default PAGP configuration and just shutdown the link in a way where it wouldnât go down (otherwise the timer would expire immediately). I think this is a CML thing because CML is not as fast as real hardware when it comes to timers
I tried recreating your scenario by configuring the learning to physical on the interfaces in CML and found I got the same results as you. I donât know why, but Iâd be interested to try this out on real devices at some point.
Yeah, I am thinking that this is just a CML thing. L2 and such works a bit worse on virtual IOS.
Thatâs all, thank you!
Hello David
To be honest, the Cisco documentation is not that clear. My understanding is that the priority command for PAgP tells the switch to use a particular port for all traffic, and causes the other ports to remain idle in a hot standby status. I tried labbing this up and found that the priority I set did not seem to affect the port that was chosen to be used. So Iâve reproduced your results in CML.
It look like for some of these features, CML is not quite the best solution. This has been interesting, thanks!
Laz
what are the load balance algorithm used for the L3 etherchannel? How the traffic will be distributed among the physical links?
Hello Tom
For either L2 or L3 EtherChannel, the load-balancing algorithms function in the same manner. To get more detail on how they operate, take a look at this NetworkLessons note on EtherChannel load balancing algorithms.
I hope this has been helpful!
Laz
Hello Laz,
Please find some questions I have regarding the topic below:
My understanding is while an etherchannel will increase the bandwidth between switches, or between a switch and a server that have an etherchannel configured between them, that the speed across the the etherchannel will not change. For example, configuring 2 x 1G interfaces in an etherchannel will not give the etherchannel a speed of 2G. What it would do is be able to potentially load balance across 2 x 1G interfaces for increased bandwidth, but any single flows or connections exceeding 1G will still experience bottlenecks and packet loss across the etherchannel due to both links being 1G. To illustrate this, if I had the same etherchannel I mentioned above (2 x 1G interfaces), and we tried to forward a single 2G stream across, we would still see a bottleneck and packet loss occur due to each link within the etherchannel being rated for 1G. Now, since etherchannels increase bandwidth, a situation in which we could use an etherchannel appropriately would be if we still had the same etherchannel of 2 x 1G interfaces between two switches, and we wanted to forward a total of 4 x 500 Mbps streams across the etherchannel. Since no single stream/flow exceeds 1G, we could forward two of them over a single 1G link, and load balancing between them, we could forward all 4 of them since we have 2G worth of bandwidth. Another example would be if we wanted to use 2 x 10G links in an etherchannel between switches to forward lets say 20 x 500 Mbps streams between them. Is my understanding correct?
My understanding is that we use a hashing algorithm to determine which link within an etherchannel we use to forward traffic. The default is the source MAC address of the frames. I also understand that the traffic going through the etherchannel is not necessarily evenly spread between the links in the etherchannel, so one link may be forwarding all of the traffic where as the other links may not be forwarding any traffic. Would the hashing algorithm automatically recognize when an individual link within the etherchannel is saturated and start using another link to load balance?
Consider the following hypothetical situation: Lets say that in a cisco switched environment, we had an etherchannel with 2 x 10G interfaces that was showing a good amount of output drops. Upon further investigation, we notice that only one of the interfaces that forms part of the etherchannel is showing those output drops, the other interface shows no drops. Could these be a result of improper load balancing based on the network requirements of the system? I understand using the default load balancing with the source MAC address may not be the best option for every network. I have experienced this issue I described before in which changing the load balancing method to something like src-dst-mixed-ip-port alleviated the output drops.
Thank You Laz
Hello Paul
These are excellent questions that help to clarify how EtherChannel actually works. We often say that EtherChannel will increase the bandwidth to the sum of the speeds of the links, but this is not quite accurate. It can potentially reach this maximum, but only if certain other conditions are met. In all of your questions, I believe that you have a very good grasp of how things work. Let me just comment on each one.
For number 1, your description is correct. Because EtherChannel will not load balance a single flow across multiple physical links, each flow cannot exceed the maximum speed of one physical link. So if you have only two hosts communicating across that EtherChannel bundle, even if you have eight physical links, only one will be used.
For this reason, EtherChannel is best applied when you have many hosts communicating over the EtherChannel bundle. Your examples show very clearly that if you increase the number of hosts communicating, the flows will be more evenly distributed across the available physical links. As you can imagine, if you have hundreds of hosts, probabilistically, the traffic will be distributed relatively evenly.
For question 2, indeed, the hashing algorithm determines which link the traffic will take. There are various load balancing algorithms you can use, and they should be used depending on your setup. More info about the way load balancing takes place in EtherChannel and how the specifics of your network can affect this can be found at this post:
No, the hashing algorithm is not intelligent enough to do this, nor will it proactively adjust the load balancing algorithm to accommodate different types of traffic flow. Typically, it is sufficient to apply the algorithm that is most appropriate to your setup.
Concerning question 3:
Yes absolutely. For example, imagine you have dozens of hosts on one end of the EtherChannel, and a single default gateway used by all hosts on the other end. If youâre hashing based on the source MAC address, you may find that traffic from the default gateway to all hosts may take only a single physical link (since the source MAC address is always the same!) And of course, you would get oversubscription on that one link, with congestion and packet drops⌠this, of course, is undesirable.
Let me know if you require any further clarification!
I hope this has been helpful!
Laz
Thank you for the response!
Just to make sure, when you mentioned âWe often say that EtherChannel will increase the bandwidth to the sum of the speeds of the links, but this is not quite accurate. It can potentially reach this maximum, but only if certain other conditions are metâ, you mean that we can potentially increase the bandwidth to the sum of the links as long as we load balance appropriately to achieve this, right? In other words, if we bundled 2 x 10G links, then in order to actually increase the bandwidth of the etherchannel to 20G we would have to load balance correctly across the two links, otherwise we could still experience congestion and output drops if all of our traffic uses the same link within the etherchannel, effectively still giving us a single 10G link in terms of bandwidth. Is my understanding here correct?
One more question. I have heard that etherchannels work better in pairs/even numbers. For example an etherchannel with 2,4, or 6 links will work better than an etherchannel that has 3 links on it, or 5 links for example. Is this true?
Thank You
Paul
ââ
ââ
Hello Paul
For the most part, yes. Part of it is ensuring that the load-balancing algorithm is appropriately configured. But even then, itâs not 100% sure that you will be using the full bandwidth all the time.
Using your example, if you have bundled 2 X 10G links, then you would be able to use the full 20G capacity only if the hashing algorithm splits the flows 50% 50% exactly, and only if those hosts are using the same bandwidth all the time. For example:
However, as you can imagine, the throughput per flow, and even the number of flows per host continually changes. The hashing mechanism wonât split the flows exactly at 50%. It works with probabilities, so you will get fluctuating split percentages over time, but will average out to relatively close to 50%.
Also, you may find that more hosts that have been relegated to one link using the hash algorithm may be using applications with higher bandwidths for a period of time, while those on the other link may be using applications with lower bandwidths. So you may temporarily oversubscribe one link while the other is only half used.
So you see, it is highly probabilistic in nature. In general, it approaches maximum efficiency, but it will depend on the traffic patterns occurring at any particular time.
This is an excellent question and one that often comes up. The quick answer is no. Practically speaking, there is no difference in efficiency if you use an even or odd number of links. The misconception comes from the mathematics involved in Ciscoâs hashing mechanism.
EtherChannelâs distribution efficiency can be affected by how the hash algorithm maps to the number of links. Cisco uses a hash algorithm that produces values, typically 0-7 for 3-bit hash, or 0-16 for a 4-bit hash. These values are mapped to available links using modulo operation.
Î For a number of links that is a âpower of 2â (i.e. 2, 4, 8, 16), the hash distribution is more mathematically uniform. With other values, including odd numbers, you can potentially get slightly uneven distribution.
This is the mathematical analysis. Practically speaking, in real-world scenarios, traffic is diverse enough that the difference is negligible. A 3-link EtherChannel (or 5 or 7) works just fine and is fully supported.
I hope this has been helpful!
Laz
