Hi LAZ,
Still trying to get hold of it.
Can you please give me an example for this.I mean what will happen if we dint remove equal tokens from the be bucket.
T
Hello Ajai
Hereās an example I hope will help.
Letās use the same values used by Rene. We have a CIR of 128000 bps and a PIR of 256000 bps. One second elapses between packets arriving, so the buckets will have:
- Bc bucket = 1 * 128000/8 = 16000 tokens
- PIR bucket = 1 * 256000/8 = 32000 tokens
The whole key to this algorithm is the fact that the Bc bucket and the PIR bucket are replenished at the same time, but at different rates, so you will always have a difference in the number of tokens in each bucket. Specifically, the difference between them is in direct proportion to the difference between the CIR and PIR.
Letās assume that at a specific point in time, the buckets have the number of tokens calculated above. Letās look at three scenarios:
- Letās say that the very next packet arriving is 12000 bytes. Compare this value to the number of tokens in the buckets and we see that packet < Bc. Since this is true, the packet is considered conforming.
- Instead of 12000 bytes, letās say the very next packet arriving is 24000 bytes. Here we see that Bc < packet < PIR, by definition this means that the packet is exceeding.
- Instead, letās say the very next packet arriving is 36000 bytes. In this case PIR < packet so the packet is violating.
I hope this has been helpful!
Laz
1 Like
Dear Rene,
I have some thing to clarify,
Take the single rate, 2 color configured to CIR = 96 kbps
when we configure it, will the bucket be filled with 12K tokens before even start receiving the 1st packet?
Letās say the 1st packet of 50 Bytes received after 0.1 seconds.
As per my understanding;
all 12k tokens are removed and 1200 tokens are put in to the bucket and if it conforms 50 tokens will be removed, so now only 1150 tokens available? am i ryt?
1 Like
Hello Roshan
At the very beginning, when the device is turned on and QoS policing mechanisms take place, the token bucket is full. Letās say that the maximum capacity of the bucket is 24K tokens for argumentās sake.
Tokens are removed from the bucket every time a packet arrives. The size of the packet in bytes is the number of tokens that are removed.
So if a packet of 50 bytes arrives, it will remove (50x8=400 bits) 400 tokens from the bucket, resulting in 23600 tokens at that specific moment in time.
Remember that tokens are replenished based on the amount of time that has passed between packets. This is why we use the (packet arrival time - previous packet arrival time) formula. This can also be described as āas long as the circuit is idleā. If you set a CIR of 96kbps, then you are setting your token replenishing rate at 96000 bits per second as long as the circuit is idle.
Every time a packet comes, we remove tokens, and every time interval where the circuit is idle passes, the tokens are replenished at a rate of (idle time) * CIR.
I hope this has been helpful!
Laz
1 Like
Dear Laz,
Thanks for the reply.
In policing, 1 token = 1 byte ryt? I mean not a bit ryt? So if a packet of 50 bytes arrives, it will remove 50 tokens not 400 tokens ryt?
Also if the bit rate is set for 96kbps, the maximum capacity of the token bucket should be 12000 tokens ryt?
1 Like
Hello Roshan
Yes, you are correct, one token corresponds to 1 byte, not one bit. And yes the capacity should be 12000 tokens. Thanks for correcting that. The logic behind the example still stands.
Thanks again!
Laz
1 Like
Dear Laz,
Thanks for your replies. My doubt still exists as I am trying to visualize this process of replenishing to bit rate. Let me be clear of what I think whats happening step by step. Please correct me where I am misunderstood. Assume this is for Single rate- Single bucket.
(1) Assume I configured the policing rate to be 96 Kbps. As soon as I applied it to an interface, the token bucket full of tokens (12,000) is created and waiting for packets.
(2) Letās assume 1st Packet which has a packet size of 100 bytes comes after 0.5 seconds
(3) So 100 tokens will be removed from the bucket and at the same time it will put 6,000 tokens back into the bucket.
(4) But because the bucket capacity is 12,000, new 5,900 tokens will be spilled and discarded.
(5) Assume 2nd packet of 500 bytes comes after 0.2 seconds from the last packet received time
(6) 500 tokens removed from 12,000 and 2,400 tokens are added same time which will again full the bucket discarding new 1,900 tokens.
(7) Assume 3rd packet comes after 0.01 seconds from the second packet of 300 bytes
(8) Now 300 tokens will be removed and 120 tokens will be put into the bucket
(9) Now total number of tokens in the bucket is 11,820 and waiting for the next packet
Is this ryt or wrong?
Hello Roshan
I believe that youāve got the idea!! You can really see how this functions especially if you imagine an example where you have large numbers of packets arriving and the bucket gets empty. If you imagine 1500 byte packets arriving every 0.05 seconds, youād have a situation where 1500 tokens would be removed and be replenished with 600 every timeā¦ the bucket would eventually become emptyā¦
I hope this has been helpful!
Laz
1 Like
Policer in the topic with 128 000 bps. Packet will be dropped when If the number of bytes in the packet is more than the number of tokens in the bucket, does it means, that it allowes maximum half of the bitrate? That is 64 000 bps? for example 1 second no traffic 1 second traffic, whateverā¦
Hello Jan
The purpose of the token bucket and of all of these different algorithms is to allow for a smooth distribution of traffic over time. This means that if you are policing at 128000 bps, then over a long period of time, say 10 minutes, the maximum average of throughput you can achieve is 128000 bps. If your traffic is smooth and consistent, then you will achieve this. If it is bursty and irregular, then you will achieve something much smaller.
If your traffic is so irregular that the bucket is often emptied completely, then you will have many packets that are dropped. If your traffic is smooth enough so that the bucket is replenished at regular intervals, then fewer packets will be dropped.
Depending on the algorithm you use, you will have a hard upper limit (single rate two colour) or will allow for bursting (single rate, three colour and dual rate three colour) to accommodate this bursty nature of traffic.
I hope this has been helpful!
Laz
Hi, iām a bit confused with the Single rate, two color (one token bucket) explanation, this is what Iāve gathered so farā¦
Every time a packet comes in the formula adds more tokens (bytes) to the bucket, if the bucket is full then the packets are dropped.
Each packet that goes into the router the bytes from the formula are added to the bucket in bytes (NOT the actual size of the packet in bytes itself). If the number of bytes in the ACTUAL packet is equal to or less than the number of tokens (bytes) in the bucket then conforming is done and the policer takes the tokens out of the bucket (? What does it take out and what in bytes?)
For the sizes that are being put into the bucket, 1 single packet cannot be that big as well so no packets will ever be dropped so how does the traffic dropping occur then? When the bucket is full? How does it get full and what is the limit on the bucket, is it the CIR?
Thanks
Hello Michael
First of all, at some specific rate (which we will clarify shortly) tokens are replenished into the bucket. Tokens are replenished as a function of time. As time goes on, tokens are added. Iāll talk a little more about this shortly.
When a packet arrives, one of two things take place:
- If the bucket has a number of tokens greater than or equal to the number of bytes of the packet, then:
- The packet is considered conforming
- A number of tokens equal to the number of bytes of the packet, are removed from the bucket.
- If the bucket has a number of tokens less than the number of bytes of the packet, then
- The packet is considered exceeding
- The policer doesnāt remove any packets from the bucket
The above process only classifies a packet as conforming or exceeding. What happens to the packet when it is conforming or exceeding depends upon the action that is configured to be taken.
Now replenishing takes place over time. Replenishing essentially takes place at the rate at which the policer has been set. If it is set to 128000 bps, then the replenishing rate is 128000/8 = 16000 bytes or tokens per second.
I hope this has been helpful! Stay safe and healthy!
Laz
Hi Laz
I understand what youāre saying, but by this logic the bucket would always be empty or have bytes lower than the packet size coming in because if every second 16,000 bytes (which is an average packet transmission rate of 10 ms per packet) is going into the bucket and you have a max packet size of 1514 bytes then every 10ms that means 160 bytes is going into the bucket and itās taking 1514 bytes out of the bucket that means that within 1 second 151,400 bytes will be taken out the bucket and only 16,000 bytes put in so how would anything get allowed/conformed?
Also one more thing, the replenishing formula that is on the page is slightly confusing to and is a bit different to yours, you donāt mention anything about āpacket time arrival - previous packet arrival timeā, on the one on the page what if youāve a packet that is 10ms apart? That would be (10 -10) x 128,000 Ć· 8 = 0. So thatās a bit confusing as well. Hopefully you can shed some light on these things because I donāt think Iām getting it at the moment.
Thanks and stay safe too
Iāve been reading a bit more and this is what Iāve gathered so far, maybe Iām right in what Iām saying here Iām not sure.
A packet flow of every 5-10ms which means around 80-160 bytes are going into the bucket (on a 128kbps line) regardless of what is taken out if the packet is conformed because of the replenishing formula. And if you have a packet that is coming in that is 1514 bytes in size then depending on if there is enough tokens in the bucket from the replenishing formula, the packet will either be dropped or conformed and those packets are taken out of the bucket if the packet is conformed. If the packet is dropped then the policer doesnāt remove any packets and the packet is more than likely dropped.
So youāre putting less into the bucket on slower lines and taking more out if you hammer the bandwidth with large packets to the maximum packet size whereas with a 500mb line for example if you had one packet coming in with 4ms and the next one came in at 6ms then that would be (0.002) Ć 500000000 Ć· 8 = 125,000 bytes into the bucket and only taking out 1514 bytes which means that most packets will be allowed/conformed.
But then I have to ask if that is correct then how would that 500Mb line ever drop packets since most packets would be conformed.
If the ISP has set a CIR of 500Mb and the user is on a physical 1Gb link with no shaping their end then surely there would be more packets sent at that speed than should be allowed to be conformed at the ISP end so how does the ISP and a policer of 500mb drop packets if itās getting to many off the 1Gb link from the customers end? How does it work it out?
Thanks again Laz
Hello Michael
I apologize, I wasnāt quite clear in my previous posts, and I was missing an important piece of information. Let me try to respond to both of your posts:
Yes, let me clarify. The replenishing of the token bucket occurs over time, but I neglected to indicate that the replenishing takes place during idle time. So replenishing will take place at a rate of 16000 bytes per second of idle time. This is another way of looking at the formula. Notice the replenishing shown in the diagram in the lesson (as seen below) occurs during the interval between the arrival of packets:
Think about it this way. If the bucket is at maximum capacity (16000 bytes), and a steady flow of packets arrive, with no idle time at all, then eventually (after about 10 packets) 16000 bytes would be emptied from the bucket. The next packet would be exceeding, and thus would be dropped. This creates some idle time, giving the opportunity for the bucket to be refilled. Once it is refilled enough, the next packet that arrives will be conforming. The corresponding number of tokens will be removed. The next packet that arrives will be evaluated in the same way, and so onā¦
Note here that the key to policing is that this process, the dropping of the packets, creates idle time. It is this idle time that limits the traffic to the appropriate level even if the physical interface is capable of transmitting at higher throughputs.
The same is the case for a 1 Gbps physical link that is policed at 500 Mbps, itās simply a matter of scale. The token bucket will have a capacity of about 62.5 million tokens. If the stream of packets is steady, the bucket will eventually empty, forcing idle time between incoming packets (by dropping the exceeding ones). Once the bucket refills sufficiently, subsequent packets will conform, removing the appropriate number of tokens from the bucket, and so on, resulting in a policed link.
(Once again, remember, that this whole procedure is a matter of classification. Packets are classified as either conforming or exceeding. If they conform, tokens are removed from the bucket, if they exceed, tokens are not removed. If they are actually dropped or forwarded depends on the configured action).
I hope this has been helpful! Stay healthy and safe!
Laz
1 Like
Right I think (finally! haha) Iām starting to understand it a bit more now after your last message Laz. Iāll take another go at explaining it too see if I understand it now.
The police rate of 128,000kbps will create a 16000 byte bucket size and if there are a steady stream of packets sizes 1514 bytes and no packet time interval time over 1 second then this will empty the bucket after around 10 packets due to the packet sizes being deducted from the bucket. When the bucket has been emptied the bucket then refills during the idle time (1 second) were no packets are being sent because the bucket is empty and has been used up. The dropping of packets creates idle time for the bucket to be replenished up to its CIR again.
The bucket does NOT fill up during per packet intervals unless the interval is 1 second.
Thanks again Laz, I look forward to your response
Hello Michael
Great to hear that itās becoming clearer. What you describe looks good, let me just point out one thing. The bucket will be replenished during any amount of idle time. The idle time does not have to be 1 second, it could be 0.5, 0.2 or 0.1 for example. The replenishing rate however is 16000 tokens per second. So if you have idle time of 0.2 seconds, you will have replenished 0.2*16000=3200 tokens. In practice, there is a minimum amount of time that allows replenishing simply because this is implemented using a CPU that has finite or limited capabilities, but it is more on the order of several milliseconds rather than one second.
I hope this has been helpful!
Laz
Hi Laz,
Sadly after you mentioning that the bucket replenishes after ANY amount of idle time it has become confusing again so to test this out i went and labbed up the exact same topology and config as is on the lesson and what i found was odd, if you look at the picture youāll find that there is packets being dropped even before there is enough time to deplete the bucket isnāt there?
I went for the max packet size and different amount of packets sent, if you look at the first one the first packet is dropped after the 8th ping (not including the first initial drop for the ARP) which means thatās only 12,112 bytes and isnāt even the close to the 16,000 bytes in the bucket and i havent included the idle time replenish tokens to (which came to around 3200 bytes after some careful adding up in wireshark in GNS3) so youāve 16,000 bytes, 12,112 of which have been deducted on the 8th packet which comes to 3,888 bytes plus around 3,200 bytes for the replenishment to which comes to around 7,088 left in the bucket but yet the packet is dropped on the 8th packet.
So i ran another group of pings all 20 packets and the same size again and got the same pattern pretty much every time to. Dropping on the 4th packet now each time doesnāt sound right to when the bucket is full again to.
Is all what has been explained in the lesson the ātheoryā of it but in reality there are other things at work here to that havenāt been mentioned that could cause this?
Thanks again Laz and stay safe
Hello Michael
Your determination and attention to detail is commendable! Using Wireshark in this manner to determine the number of tokens in the bucket at any one time is not quite accurate. You are getting different results because there are varying idle times between packets, if any. When a ping is sent from a Cisco device, there are ARP lookups taking place, as well as encapsulation, transmission and processing, all of which take up some amount of variable time. Also, pings from a Cisco device are not sent out at regular intervals, but are sent out whenever the echo reply of the previous request was sent. As a result, continuous ping will not result in a steady stream of traffic at predictable data rates, or predictable intervals. Even if you look at the timestamps of wireshark, you are looking at the times as seen from your physical computer and its CPU, and not the internal clock of the router itself.
As a result, you cannot use pings to accurately determine when the bucket is full and when it is empty, and this is why you are getting different results every time. The only thing that you can indeed determine from your output is that policing is indeed functioning.
Iām sorry I have confused you again 
As stated in the lesson as well, replenishing can take place during any amount of idle time. If it only took place during a whole second of idle time, then the bucket would all refill in one go (assuming a bucket size of 16000 tokens).
If itās still not making sense, Iāll ask @ReneMolenaar to step in, as a fresh approach may be helpful in further explaining the concept.
I hope this has been helpful!
Laz
Hi Laz
Sorry it has taken me a while to wrap my head around it, it was a bit of a confusing topic overall thatās all haha. But thank you for the patience and clarity regarding my last question. I understand now why it hasnāt behaved the way i thought it would.
Thanks again for all your help