Jun
26

The goal of this article is to discuss how would the following configuration work in the 3560 series switches:

interface FastEthernet0/13
 switchport mode access
 load-interval 30
 speed 10
 srr-queue bandwidth shape 50 0 0 0
 srr-queue bandwidth share 33 33 33 1
 srr-queue bandwidth limit 20

Before we begin, let’s recap what we know so far about the 3560 egress queuing:

1) When SRR scheduler is configured in shared mode, bandwidth allocated to each queue is based on relative weight. E.g. when configuring “srr-queue bandwidth share 30 20 25 25″ we obtain the weight sum as 30+20+25+25 = 100 (could be different, but it’s nice to reference to “100”, as a representation of 100%). Relative weights are therefore “30/100”, “20/100”, “25/100”, “25/100” and you can calculate the effective bandwidth *guaranteed* to a queue multiplying this weight by the interface bandwidth: e.g. 30/100*100Mbps = 30Mbps for the 100Mbps interface and 30/100*10Mbps=3Mbps for 10Mbps interface. Of course, the weights are only taken in consideration when interface is oversubscribed, i.e. experiences a congestion.

2) When configured in shaped mode, bandwidth restriction (policing) for each queue is based on inverse absolute weight. E.g. for “srr-queue bandwidth shape 30 0 0 0” we effectively restrict the first queue to “1/30” of the interface bandwidth (which is approximately 3,3Mbps for 100Mbps interface and approximately 330Kbps for a 10Mbps interface). Setting SRR shape weight to zero effectively means no shaping is applied. When shaping is enabled for a queue, SRR scheduler does not use shared weight corresponding to this queue when calculating relative bandwidth for shared queues.

3) You can mix shaped and shared settings on the same interface. For example two queues may be configured for shaping and others for sharing:

interface FastEthernet0/13
 srr-queue bandwidth share 100 100 40 20
 srr-queue bandwidth shape  50  50  0  0

Suppose the interface rate is 100Mpbs; then queues 1 and 2 will get 2 Mbps, and queues 3 and 4 will share the remaining bandwidth (100-2-2=96Mbps) in proportion “2:1”. Note that queues 1 and 2 will be guaranteed and limited to 2Mbps at the same time.

4) The default “shape” and “share” weight settings are as follows: “25 0 0 0” and “25 25 25 25”. This means queue 1 is policed down to 4Mbps on 100Mpbs interfaces by default (400Kbps on 10Mbps interface) and the remaining bandwidth is equally shared among the other queues (2-4). So take care when you enable “mls qos” in a switch.

5) When you enable “priority-queue out” on an interface, it turns queue 1 into priority queue, and scheduler effectively does not account for the queue’s weight in calculations. Note that PQ will also ignore shaped mode settings as well, and this may make other queues starve.

6) You can apply “aggregate” egress rate-limitng to a port by using command “srr-queue bandwidth limit xx” at interface level. Effectively, this command limits interface sending rate down to xx% of interface capacity. Note that range starts from 10%, so if you need speeds lower than 10Mbps, consider changing port speed down o 10Mbps.

How will this setting affect SRR scheduling? Remember, that SRR shared weights are relative, and therefore they will share the new bandwidth among the respective queues. However, shaped queue rates are based on absolute weights calculated off interface bandwidth (e.g. 10Mbps or 100Mbps) and are subtracted from interface “available” bandwidth. Consider the example below:

interface FastEthernet0/13
 switchport mode access
 speed 10
 srr-queue bandwidth shape 50 0 0 0
 srr-queue bandwidth share 20 20 20 20
 srr-queue bandwidth limit 20

Interface sending rate is limited to 2Mbps. Queue 1 is shaped to 1/50 of 10Mps, which is 200Kbps of bandwidth. The remaining bandwidth 2000-200=1800Kbps is divided equally among other queues in proportion 20:20:20=1:1:1. That is, in case on congestion and all queues filled up, queue 1 will get 200Kbps, and queues 2-4 will get 600Kbps each.

Quick Questions and Answers

Q: How would I determine which queue will the packet go to? What if my packet has a CoS and DSCP value set at the same time?
A: That depends on what you are trusting at classification stage. If you trust CoS value, then QoS to Output Queue map will be used. Likewise, if you trust DSCP value, then DSCP to Output Queue map will determine the outgoing queue. Use “show mls qos map” commands to find out the current mappings.

Q: What if I’ve configured “shared” and “shaped” srr-queue settings for the same queue?
A: The shaped queue settings will override shared weight. Effectively, shared weight will also be exempted from SRR calculations. Note that in shaped mode queue is still guaranteed it’s bandwidth, but at the same time is not allowed to send above the limit.

Q: What if priority-queue is enabled on the interface? Can I restrict the PQ sending rate using “shaped” weight?
A: No you can’t. Priority-queue will take all the bandwidth if needed, so take care with traffic admission.

Q: How will a shaped queue compete with shared queues on the same interface?
A: Shared queues share the bandwidth remaining from the shaped queues. At the same time, shaped queues are guaranteed the amount of bandwidth allowed by their absolute weight.

Q: How is SRR shared mode different from WRR found in the Catalyst 3550?
A: SRR in shared mode essentially behaves similar to WRR, but is designed to be more efficient. Where WRR would empty the queue up to it’s credit in single run, SRR will take series of quick runs among all the queues, providing more “fair” share and smoother traffic behavior.

Verification scenario diagram and configs

3560 Egress Queuing

For the lab scenario, we configure R1, R3 and R5 to send traffic down to R2 across two 3560 switches saturating the link between them. All routers share one subnet 172.16.0.X/24 where X is the router number. SW1 will assign CoS/IP Precedence values of 1, 3 and 5 respectively to traffic originated by R1, R3 and R5. At the same time, SW1 will apply egress scheduling on it’s connection to SW2. R2’s function is to meter the incoming traffic, by matching the IP precedence values in packets. Note that SW2 has mls qos disabled by default.

We will use the default CoS to Output Queue mappings with CoS 1 mapped to Queue 2, CoS 3 mapped to Queue 3 and CoS 5 mapped to Queue 1. Note that by the virtue of the default mapping tables, CoS 0-7 map to IP Precedence 0-7 (which become overwritten), so we can match IP precedence’s on R2.

SW1#show mls qos maps cos-output-q
   Cos-outputq-threshold map:
              cos:  0   1   2   3   4   5   6   7
              ------------------------------------
  queue-threshold: 2-1 2-1 3-1 3-1 4-1 1-1 4-1 4-1

SW1’s connection to SW2 is set to 10Mbps port rate, and further limited down to 2Mps by the use of “srr bandwidth limit” command. We will apply different scenarios and see how SRR behaves. Here comes the configurations for SW1 and R2:

SW1:
interface FastEthernet0/1
 switchport mode access
 load-interval 30
 mls qos cos 1
 mls qos trust cos
 spanning-tree portfast
!
interface FastEthernet0/3
 switchport mode access
 load-interval 30
 mls qos cos 3
 mls qos trust cos
 spanning-tree portfast
!
interface FastEthernet0/5
 load-interval 30
 mls qos cos 5
 mls qos trust cos
 spanning-tree portfast

R2:
class-map match-all PREC5
 match ip precedence 5
class-map match-all PREC1
 match ip precedence 1
class-map match-all PREC3
 match ip precedence 3
!
!
policy-map TEST
 class PREC5
 class PREC3
 class PREC1
!
access-list 100 deny   icmp any any
access-list 100 permit ip any any
!
interface FastEthernet0/0
 ip address 172.16.0.2 255.255.255.0
 ip access-group 100 in
 load-interval 30
 duplex auto
 speed auto
 service-policy input TEST

To simulate traffic flow we execute the following command on R1, R3 and R5:

R1#ping 172.16.0.2 repeat 100000000 size 1500 timeout 0

In the following scenarios port speed is locked to 10Mbps and additionally port is limited to 20% of the bandwidth, with the effective sending rate of 2Mbps.

First scenario: Queue 1 (prec 5) is limited to 200Kbps while Queue 2 (prec 1) and Queue 3 (prec 3) share the remaining bandwidth in equal proportions:

SW1:
interface FastEthernet0/13
 switchport mode access
 load-interval 30
 speed 10
 srr-queue bandwidth share 33 33 33 1
 srr-queue bandwidth shape  50  0  0  0
 srr-queue bandwidth limit 20

R2#sh policy-map interface fastEthernet 0/0 | inc bps|Class
    Class-map: PREC5 (match-all)
      30 second offered rate 199000 bps
    Class-map: PREC3 (match-all)
      30 second offered rate 886000 bps
    Class-map: PREC1 (match-all)
      30 second offered rate 887000 bps
    Class-map: class-default (match-any)
      30 second offered rate 0 bps, drop rate 0 bps

Second Scenario: Queue 1 (prec 5) is configured as priority and we see it leaves other queues starving for bandwidth:

SW1:
interface FastEthernet0/13
 switchport mode access
 load-interval 30
 speed 10
 srr-queue bandwidth share 33 33 33 1
 srr-queue bandwidth shape  50  0  0  0
 srr-queue bandwidth limit 20
 priority-queue out

R2#sh policy-map interface fastEthernet 0/0 | inc bps|Class
    Class-map: PREC5 (match-all)
      30 second offered rate 1943000 bps
    Class-map: PREC3 (match-all)
      30 second offered rate 11000 bps
    Class-map: PREC1 (match-all)
      30 second offered rate 15000 bps
    Class-map: class-default (match-any)
      30 second offered rate 0 bps, drop rate 0 bps

Third Scenario: Queues 1 (prec 5) and 2 (prec 1) are shaped to 200Kbps, while Queue 3 (prec 3) takes all the remaining bandwidth:

SW1:
interface FastEthernet0/13
 switchport mode access
 load-interval 30
 speed 10
 srr-queue bandwidth share 33 33 33 1
 srr-queue bandwidth shape  50  50  0  0
 srr-queue bandwidth limit 20

R2#sh policy-map interface fastEthernet 0/0 | inc bps|Class
    Class-map: PREC5 (match-all)
      30 second offered rate 203000 bps
    Class-map: PREC3 (match-all)
      30 second offered rate 1569000 bps
    Class-map: PREC1 (match-all)
      30 second offered rate 199000 bps
    Class-map: class-default (match-any)
      30 second offered rate 0 bps, drop rate 0 bps
About Petr Lapukhov, 4xCCIE/CCDE:

Petr Lapukhov's career in IT begain in 1988 with a focus on computer programming, and progressed into networking with his first exposure to Novell NetWare in 1991. Initially involved with Kazan State University's campus network support and UNIX system administration, he went through the path of becoming a networking consultant, taking part in many network deployment projects. Petr currently has over 12 years of experience working in the Cisco networking field, and is the only person in the world to have obtained four CCIEs in under two years, passing each on his first attempt. Petr is an exceptional case in that he has been working with all of the technologies covered in his four CCIE tracks (R&S, Security, SP, and Voice) on a daily basis for many years. When not actively teaching classes, developing self-paced products, studying for the CCDE Practical & the CCIE Storage Lab Exam, and completing his PhD in Applied Mathematics.

Find all posts by Petr Lapukhov, 4xCCIE/CCDE | Visit Website


You can leave a response, or trackback from your own site.

21 Responses to “Quick Notes on the 3560 Egress Queuing”

 
  1. bill nigh says:

    Thanks for the article. You lost me when you said “(eg. 300/100 * 100Mbps = 30Mbps)”. I know the fastethernet interface has speed set to 10. However, don’t gloss over that in your formulas. You also said 1/30 of 100Mbps is 3Mbps. That’s not accurate, either. Does SRR round the number out from 3.333.. Mbps??

  2. To Bill Nigh

    Sorry, there were a couple of typos (fixed now):

    for SRR shared weights should be 30/100*100 =30Mps for 100Mbps; and the effective bandwidth is 3Mbps for 10Mbps interface.

    As for 1/30, SRR shaped mode is pretty accurate, so you will end up with a 3300Kbps on 100Mbps interface, plus minus 5-10Kbps maybe – you can never get the exact precise 3,333Mbps with non-ideal scheduler

  3. Tony says:

    Once again a wonderful Blog. Thanks Petr!!

  4. Jo says:

    Is this a glimpse of the v5 workbook lab diagrams? ;)

  5. Gianluca says:

    Hi Petr,

    In the quick questions and answers section you say that it is not possible to restrict the PQ sending rate using “shaped” weight.

    However in the CCIE Routing and Switching Exam Certification Guide (Third Edition, page 462) it is written that when the PQ still has many frames to send, but queues 2, 3, and 4 are empty, in shared mode the PQ would send at full line rate while in shaped mode the switch would simply not service the PQ part of the time so that its overall rate would be the bandwidth configured for that queue.
    According to the book it looks like it is possible to limit the PQ sending rate configuring a shaped weight but your verification scenario demonstrates that it is not true.

    Have I understood wrong what is written in the certification guide?

  6. To Gianluca

    Obviously, shared mode weight does not affect PQ behavior. Up until the moment when I first verified PQ functionality, I was sure that “shaped” weight does affect PQ sending rate. However, as you can see (and run a verification yourself just to be sure), shaped weight has no effect on PQ either. Actually, this is what they briefly mention in the DocCD:


    When you configure this command [priority-queue out], the SRR weight and queue size ratios are affected because there is one fewer queue participating in SRR. This means that weight1 in the srr-queue bandwidth shape or the srr-queue bandwidth share command is ignored (not used in the ratio calculation).

    I’ll try to make another post on ingress queueing in policing in near future and investigate how ingress PQ actually works.

  7. To Jo

    Sort of :) But version 5 diagrams will probably have extra pictures of medieval castles, dragons, knights and fairies :)

  8. Pavel Stefanov says:

    Another great article!

    Just a quick question – when there is no congestion present on the interface, the srr-queue bandwidth shaped command would not limit traffic that matches a queue that is shaped, correct?

    Thank you!

  9. To: Pavel Stefanov

    Shaping is always active, no matter whether the interface is congested or not. Thus, it performs function similar to egress policing on the older 3550 models.

  10. [...] you read this Quick Notes on the 3560 Egress Queuing by Petr Lapukhov, this table will complete the picture to show you how calculation [...]

  11. hobbs says:

    First, thanks for a wonderful explanation. I have followed this lab, and when I enable priority queuing in your second scenario, I get the same load distribution as in the first scenario. In other words, prec 5 is still being limited. I have verified that cos 5 maps to queue 1. Is there something else I am missing? I made sure to clear stats on R2 before testing. thanks

  12. Daniel says:

    Hobbs: Your router platform could be the culprit. Like you, I got puzzling results at first when I used c3725 Routers. Then when I changed to the older C2600 routers, I was able to reproduce all of Petr’s above scenarios. Also manually set all interfaces to full-duplex, especially when your the router interfaces are Ethernet (i.e have max speed of 10MBps).

  13. Goyheneix Didier CCIP France says:

    Thanks for the explication “egress queues shared and shaped”.

    but for the ingress queues, i don’t understand the notion of 2 queues when there is a priority queue.
    Doc Cisco says, by default 90% q2 and 10% q1 whith a priority 10% for q2.
    if q2 is priority with 10%, why q2 is too 10% in the SRR process ?

    Thanks you

  14. Didier from France says:

    Hi Petr,

    an another question please
    if i enable q1 PQ (30% for example) but if there is no trafic for the PQ;
    the bandwidth of other queues q2/q3/q4 is limited at 70%
    or the bandwidth reserved PQ can be used by the others queues?

    i think, the answer will be goog for ingress queues too ?

    Thanks

  15. K says:

    It is excellent article.And the examples is good for me to understand SRR.
    thanks you very much again!

  16. Tmo says:

    Hi Petr

    srr-queue bandwidth limit 20 – does this do policing or shaping?
    Until recently i was pretty sure it does policing, but fallowing a post on some forum someone claims this is command actually does shaping.

    Thank you

  17. Hugh says:

    Hi Petr,

    Thank you for sharing such a wonderful article!
    Your explanation is spot on and crystal clear.
    So useful for any IE Lab preparation.

    Hugh

  18. Albert Suwandhi says:

    Great article Petr. It makes me understand SRR easily.

  19. lee maynard says:

    Hi Petr – what is the diag a type of visio theme ? – looks great ;-)

  20. [...] http://blog.ine.com/2008/06/26/quick-notes-on-the-3560-egress-queuing/ Share this:TwitterFacebookLike this:Like Loading… By botches, on April 7, 2013 at 3:47 pm, under QoS. No Comments Post a comment or leave a trackback: Trackback URL. « CUE with CCM [...]

  21. maxwell says:

    I really loved reading this. Very good work.

    Cheers

 

Leave a Reply

Categories

CCIE Bloggers