Nov
08

Abstract

This publication discusses the spectrum of problems associated with transporting Constant Bit Rate (CBR) circuits over packet networks, specifically focusing VoIP services. It provides guidance on practical calculation for voice bandwidth allocation in IP networks, including the maximum bandwidth proportion allocation and LLQ queue settings. Lastly, the publication discusses the benefits and drawbacks of transporting CBR flows over packet switched networks and demonstrates some effectiveness criteria.

Introduction

Historically, the main design goal of Packet Switched Networks (PSNs) was optimum bandwidth utilization for low-speed links. Compared to their counterpart, circuit-switched networks (CSNs such as SONET/SDH networks), PSNs use statistical as opposed to deterministic (synchronous) multiplexing. This feature allows PSNs to be very effective for bursty traffic sources, i.e. those that send traffic sporadically. Indeed, with many sources this allows the transmission channel to be optimally utilized by sending traffic only when necessary. Statistical multiplexing is only possible if every node in the network implements packet queueing, because PSNs introduce link contention. One good historical example is ARPANET: the network theoretical foundation has been developed in Kleinrock's work on distributed queueing systems (see [1]).

In PSNs, it is common for the traffic from multiple sources to be scheduled for sending out the same link at the same moment. In such case of contention for the shared resource, exceeding packets are buffered, delayed and possibly dropped. In addition to this, packets could be re-ordered, i.e. packets sent earlier may arrive behind packets that have been sent after them. The latter is normally a result of packets taking different paths in the PSN as a due to routing decisions. Such behavior is OK with bursty, delay insensitive data traffic, but completely inconsistent with the behavior of constant bit rate (CBR), delay/jitter sensitive traffic sources, such as emulated TDM traffic. Indeed, transporting CBR flows over PSNs poses significant challenges. Firstly, emulating a circuit service requires that every node should not buffer the CBR packets (i.e. should not introduce delay or packet drops) and be "flow-aware" to avoid re-ordering. The other challenge is the "packet overhead" tax imposed on emulated CBR circuits. Per their definition, CBR sources produce relatively small burst of data at regular periodic intervals. The more frequent are the intervals, the typically smaller are the bursts. In turn, PSNs apply a header to every transmitted burst of information to implement network addressing and routing, with the header size being often comparable to the CBR payload. This significantly decreases link utilization efficiency when transporting CBRtraffic.

Emulating CBR services over PSN

At first, it may seem that changing queuing discipline in every node will resolve the buffering problem. Obviously, if we distinguish CBR flow packets and service them ahead of all other packets using priority-queue then they would never get buffered. This assumes that link speed is fast enough so that serialization delay is negligible in the context of the given CBR flow. Such delay may vary depending on the CBR source: for example, voice flows typically produce one codec sample every 10ms and based on this, serialization delay at every node should not exceed 10ms, or preferably be less than that (otherwise, the next produced packet will "catch up" the previous one). Serialization problem on slow links could be solved using fragmentation and interleaving mechanics, e.g. as demonstrated in [6]. Despite of priority queueing and fragmentation, situation becomes more complicated with multiple CBR flows transported over the same PSN. The reason is that there is now contention among the CBR flows, since all of them should be serviced on priority basis. This creates queueing issues and intolerable delays. There is only one answer to reduce resource contention PSNs - over-provisioning.

Following the work [2], let's review how the minimum over-provisioning rate could be calculated. We first define the CBR flows as those that cannot tolerate a single delay of their packet in the link queue. Assume there are r equally behaving traffic flows contending for the same link. Pick up a designated flow out of the set. When we randomly "look" at the link, the probability that we see a packet from the designated flow is 1/r, since we assume that all flows are serviced equally by the connected PSN node. Then, the probability that the selected packet does NOT belong to our flow is 1-1/r respectively. If the link can accept at maximum t packets per millisecond, then during an interval of k milliseconds the probability that our designated flow may send a packet over the link without blocking is: P=1-(1-1/r)^tk, where (1-1/r)^tk is the probability of NOT seeing our flows designated packet amount the tk packets. The value P is the probability of any given packet NOT being delayed due to contention. It is important to understand that every delayed packet will cause flow behavior deviation from CBR. Following [2], we define u=tk as the "over-provisioning ratio" where u=1 when the channel can only send one flow packet during a time unit that take a flow to generate the same packet, i.e. when channel rate = flow rate. When u=2 the link is capable of sending twice as much packets during a unit of time compared to the number of packets sent by a single flow during the same interval. With the new variable, the formula becomes P=1-(1-1/r)^u. Fixing the value of P in this equation we obtain:

u=ln(1-P)/ln(1-1/r). (*)

which the minimum over-provision ratio to achieve the desired P probability of successfully transmitting the designated flow's packet when r equal flows are contending for the link. For example, with P=99.9% and r=10 we end up having u=ln(0.001)/ln(0.9)=65.5. That is, in order to provide the guarantee of not delaying 99.9% packets for 10 CBR flows we need to have at least 66 times more bandwidth than a single flow requires. Lowering P to 99% results in u=44 over-provisioning coefficient. It is interesting to look at the r/u ratio, which demonstrate what portion of minimally over-provisioned link bandwidth would be occupied by the "sensitive" flows when they all are transmitted in parallel. If we take the ratio r/u=ln((1-1/r)^r)/ln(1-P) then with large number of r we can replace (1-1/r)^r with 1/e and the link utilization ratio becomes approximated by:

r/u=-1/ln(1-P). (**)

For P=99% we get the ratio of 21%, for P=99.9% the ratio is 14% and for P=90% the ratio becomes 43%. In practice, this means that for moderately large amount of concurrent CBR flows, e.g. over 30, you may allocate no more than specified percentage of the link's bandwidth to CBR traffic based on the target QoS requirement.

Now that we are done with buffering delays, what about packet reordering and packet overhead tax? Reordering problem could be solved at the routing level in PSNs: if every routing node is aware of the flow state it may ensure that all packets belonging to the same flow are sent across the same path. This is typically implemented by deep packet inspection (which by the way violates the end-to-end principle as stated in RFC 1958) and classifying the packets based on the higher-level information. Such implementations are, however, rather effective as inspection and flow classification is typically performed in forwarding path using hardware acceleration. The overhead tax problem has two solution. The first one is, again, over-provisioning. By using high-capacity, low-utilized link we may ignore the bandwidth wastage due to the overhead. The second solution requires adding some state to network nodes: by performing flow inspection at the both ends of a single link we may strip the header information and replace it with a small flow ID. The other end of the link will reconstruct the original headers by matching the flow ID to the locally stored state information. This solution violates the end-to-end principle and has poor scalability as the number of flows grow. Typically it is used on slow-speed links. For example, VoIP services utilize IP/RTP/UDP and possibly TCP header compression to reduce the packet overhead tax.

Practical Example: VoIP Bandwidth Consumption

Let's say we have a 2Mbps link and we want to know how to provision priority-queue settings for G.729 calls. Firstly, we need to know per-flow bandwidth consumption. You may find enough information on this topic referring to [3]. Assuming we are using header compression over Frame-Relay, the per-flow bandwidth is 11.6Kbps and maximum link capacity is roughly 2000Kbps the theoretical maximum over-subscription rate is 2000/11.6=172. We can find the maximum number of flows allowed under the condition of P as

r=1/(1-exp(ln(1-p)/u)) = 1/(1-(1-P)^(1/u)) (***)

setting u=170 and P=0.99. This yields the theoretical limit of 37 concurrent flows. The total bandwidth for that many flows is 37*11.6=429Kbps or about 21% of the link capacity, as predicted by the asymptotic formula (**) above. The remaining bandwidth could be used by other non-CBR applications, as it should be expected from a PSN exhibiting high link utilization efficiency.

Knowing the aggregate bandwidth and maximum number of flows provides us the parameters for admission control tools (e.g. policer rate, RSVP bandwidth and so froth). However, what is left to define yet are the burst settings and the queue depth for LLQ. The maximum theoretical burst size equals to the maximum number of flows multiplied by the voice frame size. From [3] we readily obtain that a compressed G.729 frame size for Frame-Relay connection is 29 bytes. This gives us the burst of 1073 bytes, which we could round up to 1100 bytes for safety. The maximum queue depth could be defined as number of flows minus one, since in the worst case at least one priority packet would be scheduled for serializing while others held in the priority queue for processing. This means the queue depth would be at maximum 36 packets. The resulting IOS configuration would look like:

policy-map TEST
class TEST
priority 430 1100
queue-limit 36 packets


Circuits vs Packets

It is interesting to compare voice call capacity for a digital TDM circuit vs the same circuit being used for packet mode transport. Talking of a E1 circuit, we can transport as many as 30 calls, if one channel is used for associated signaling (e.g. ISDN). Compare this to the 37 G.729 VoIP calls we may obtain if the same circuit is channelized and runs IP - about 20% increase in call capacity. However, it is important to point out the quality of G.729 calls is degraded as compared to digital 64Kbps bearer channels, not to mention that other services could not be delivered over a compressed emulated voice channel. It might be more fair comparing the digital E1 to the packetized E1 circuit carrying G.711 VoIP calls. In this case, the bit rate for a single call running over Frame-Relay encapsulation with IP/RTP/UDP header compression would be (160+2+7)*50*8=67600bps or 67.6Kbps. Maximum over-provision rate is 29 in such case, which ends up with only six (6) VoIP calls allowed for the packetized link with P=99% in-time delivery! Therefore, if you try providing digital call quality over a packet network you end up with extremely inefficient implementation. Finally, consider an intermediate case - G.729 calls without IP/RTP/UDP header compression. This case assumes that complexity belongs to the network edge, as any transit links are not required to implement the header compression procedure. We end up with the following: uncompressed G.729 call over Frame-Relay generates (20+40+7)*50*8=26.8Kbps which results in over-provisioning coefficient of u=74 and r=16 flows - slightly over the half of the number that a E1 could carry.

Using the asymptotic formula (**) we see that for P=99% no more than 21% of the packetized link could be used for CBR services. This implies that the packet compression scheme should reduce the bandwidth of pure CBR flow by more than 5 times to be effectively compared with circuit-switched transport. Based on this, we conclude that PSNs could be more efficient for CBR transportation compared to "native" circuit-networks only if they utilize advanced processing features such as payload/header compression yielding compression coefficient over 5 times. However, we should keep in mind that such compression is not possible for all CBR services, e.g. relaying T1/E1 over IP has to maintain the full bandwidth of the original TDM channels, which is extremely inefficient in terms of resource utilization. Furthermore, the advanced CODEC features require complex equipment at the network edge and possibly additional complexity in the other parts of the network, e.g. in order to implement link header compression.

It could be argued that the remaining 79% of the packetized link could be used for data transmission, but the same is possible with circuit switched networks, provided that packet routers are attached to the edges. All the data packet switching routers need to do is dynamically request transmission circuit from the CSN based on traffic demands and used them for packet transmissions. This approach has been implemented, among others, in GMLPS ([5]).

Conclusions

The above logic demonstrates that PSNs were not really designed to be good at emulating true CBR services. Naturally, as the original intent of PSNs was maximizing the use of scarce link bandwidth. Transporting CBR services not only requires complex queueing disciplines but also ultimately over-provisioning the link bandwidth, thus somewhat defeating the main purpose of PSNs. Indeed if all that a PSN is used for is CBR service emulation, the under-utilization remains very significant. Some cases, like VoIP, allows for effective payload transformation and significant bandwidth reduction, which allows for more efficient use of network resources. On the other hand, such payload transformation requires introducing extra complexity to networking equipment. All this in addition to the fact that packet-switching equipment is inherently more complex and expensive compared to circuit-switched networks especially for very high-speed links. Indeed, packet switching logic requires complex dynamic lookups, large buffer memory and internal interconnection fabric. Memory requirements and dynamic state grow proportionally to the link speed, making high-speed packet-switching routers extremely expensive not only in hardware but also in software, due to advanced control plane requirement and proliferating services. More on this subject could be found in [4]. It is worth mentioning that the packet-switching inefficiency in network core has been realized long time ago, and there have been attempts for integrating circuit-switching core networks with packet-switching networks, most notable being GMPS ([5]). However, so far, the industry inertia did not make any of the proposed integration solutions viable.

Despite of all arguments, VoIP implementations have been highly successful so far, most likely akin to the effectiveness of VoIP codecs. Of course, no one can yet say than VoIP over Internet provides quality comparable to digital phone lines, but at least it is cheap, and that's what market is looking for. VoIP has been highly successful in enterprises, so far, mainly due to the fact that enterprise campus networks are mostly high-speed based on ethernet switched technology, that demonstrates very low general link utilization ratio, within 1-3% of available bandwidth. In such over-provisioned conditions, deploying VoIP should not pose major QoS challenges.

Further Reading

[1] Information Flow in Large Communication Nets, L. Kleinrock
[2] The Case for Service Overlays, Brassil, J.; McGeer, R.; Sharma, P.; Yalagandula, P.; Mark, B.L.; Zhang, S.; Schwab, S.
[3] Voice Bandwidth Consumption, INE Blog
[4] Circuit Switching in the Internet, Pablo Molinero Fernandez
[5] RFC 3945
[6] PPP Multilink Interleaving over Frame-Relay

Oct
17

Computing voice bandwidth is usually required for scenarios where you provision LLQ queue based on the number of calls and VoIP codec used. You need to account for codec rate, Layer 3 overhead (IP, RTP and UDP headers) and Layer 2 overhead (Frame-Relay, Ethernet, HDLC etc. headers). Accounting for Layer 2 overhead is important, since the LLQ policer takes this overhead in account when enforcing maximum rate.

We are going to consider two codecs for bandwidth computation: G.729 and G.711. By default, both codecs generate 50 VoIP packets per second. However, the codec framing rate is 10ms (100 packets per second). Therefore, each VoIP packet carries two frames with VoIP samples. The frame sizes are 10 bytes and 80 bytes for G.729 and G.711 codecs respectively.

Based on this, G.729 generates [10*2]*50*8=8000bps and G.711 generates [80*2]*50*8=64000bps of “payload” rate – no Layer 3 or Layer 2 overheads.

RTP header size is 12 bytes and UDP header size is 8 bytes. Typical IP header (no options) is 20 bytes in lenght. Therefore, the Layer 3 overhead is 40 bytes, if we don’t use header compression.

The following are the formats for WAN frames commonly used to transport voice. Note that these formats remain the same with or without FRF.12/MLP fragmentation schemes, since voice packets are never fragmented with good design.

Highlighted in green are the portions of Layer 2 frames that Cisco IOS queue scheduler accounts for when computing actual frame size. Note that the scheduler does not account for full Layer 2 overhead, but you need to provision more bandwidth for LLQ so that other class may not be configured with too much bandwidth. As we can see, both Cisco and IETF Frame-Relay encapsulations add 7 bytes of Layer 2 overhead to VoIP packets. The same holds true for HDLC encapsulation (which is not very common but added here for sake of completeness). PPP over Frame-Relay adds 9 bytes of overhead – the maximum overhead of all presented encapsulation types.

Using the information above, you can compute bandwidth usage for uncompressed voice traffic flow across any WAN connection. For example, let’s compute bandwidth consumption for a G.729 call across Frame-Relay link with FRF.12 fragmentation. First, FRF.12 does not fragment voice packets if configured properly. Next, the size of payload + Layer 3 overhead is 2x10 bytes + 40 bytes = 60 bytes. Based on the 50 pps rate and adding the 7 bytes overhead we end up with the bandwidth value of:

(20+40+7)*50*8=26800bps.

If you want to use G.711 codec, then replace the 20 bytes payload with 160 bytes. The result is:

(160+40+7)*50*8=82800bps.

Another thing to consider is IP/RTP/UDP headers compression. Cisco’s implementation reduces the total overhead of 40 bytes (12+8+20) down to 2 bytes (no UDP checksum). This limits the Layer 3 overhead to just 2 bytes. Let’s compute the bandwidth usage for G.729 call over MLPoFR with UDP header compression (9 bytes of Layer 2 overhead):

(20+2+9)*50*8=12400bps.

The same computations for compressed G.729 over Frame-Relay with or without FRF.12 bring the following result:

(20+2+7)*50*8=11600bps.

Now a few words about running VoIP traffic across Ethernet. Usually you don’t use CBWFQ/LLQ on fast connections on small to mid.range routers to guarantee bandwidth to VoIP traffic. Most of these routers are not capable of sending traffic at such rate that they oversubscribe 100Mbs interface. However, you may occasionally use Ethernet as "WAN" connection using Class-Based Shaping for sub-rate access. So just in case, Layer 2 overhead for typical Ethernet frame is 18 bytes – 14 bytes for Ethernet header and 4 bytes for FCS (32 bits). If the frame carries VLAN tag, add another 4 bytes here for 22 bytes of total overhead. Note that you will typically see G.711 codec used over LAN links.

Sep
16

The security appliance supports two kinds of priority queuing - standard priority queuing and hierarchical priority queuing. Let's configure each in this third part of our blog.

Standard Priority Queuing

This queuing approach allows you to place your priority traffic in a priority queue, while all other traffic is placed in a best effort queue. You can police all other traffic if needed.

Step 1: Create the priority queue on the interface where you want to configure the standard priority queuing. This is done in global configuration mode with the priority-queue interface_name command. Notice this will place you in priority queue configuration mode where you can optionally manipulate the size of the queue with the queue-limit number_of_packets command. You can also optionally set the depth of the hardware queue with the tx-ring-limit number_of_packets command. Remember that the hardware queue forwards packets until full, and then queuing is handled by the software queue (composed of the priority and best effort queues).

pixfirewall(config)# priority-queue outside
pixfirewall(config-priority-queue)#

Step 2: Use the Modular Policy Framework (covered in Part 2 of these blogs) to configure the prioritized traffic.

pixfirewall(config-priority-queue)# exit
pixfirewall(config)# class-map CM-VOICE
pixfirewall(config-cmap)# match dscp ef
pixfirewall(config-cmap)# exit
pixfirewall(config)# class-map CM-VOICE-SIGNAL
pixfirewall(config-cmap)# match dscp af31
pixfirewall(config-cmap)# exit
pixfirewall(config)# policy-map PM-VOICE-TRAFFIC
pixfirewall(config-pmap)# class CM-VOICE
pixfirewall(config-pmap-c)# priority
pixfirewall(config-pmap-c)# exit
pixfirewall(config-pmap)# class CM-VOICE-SIGNAL
pixfirewall(config-pmap-c)# priority
pixfirewall(config-pmap-c)# exit
pixfirewall(config-pmap)# exit
pixfirewall(config)# service-policy PM-VOICE-TRAFFIC interface outside
pixfirewall(config)# end

Hierarchical Priority Queuing

This queuing approach allows you to shape traffic and allow a subset of the shaped traffic to be prioritized. I have cleared the configuration from the security appliance in preparation for this new configuration. Notice with this approach, you do not configure a priority queue on the interface. Also notice with this approach the nesting of the Policy Maps.

pixfirewall(config)# class-map CM-VOICE
pixfirewall(config-cmap)# match dscp ef
pixfirewall(config-cmap)# exit
pixfirewall(config)# class-map CM-VOICE-SIGNAL
pixfirewall(config-cmap)# match dscp af31
pixfirewall(config-cmap)# exit
pixfirewall(config)# policy-map PM-VOICE-TRAFFIC
pixfirewall(config-pmap)# class CM-VOICE
pixfirewall(config-pmap-c)# priority
pixfirewall(config-pmap-c)# exit
pixfirewall(config-pmap)# class CM-VOICE-SIGNAL
pixfirewall(config-pmap-c)# priority
pixfirewall(config-pmap-c)# exit
pixfirewall(config-pmap)# exit
pixfirewall(config)# policy-map PM-ALL-TRAFFIC-SHAPE
pixfirewall(config-pmap)# class class-default
pixfirewall(config-pmap-c)# shape average 2000000 16000
pixfirewall(config-pmap-c)# service-policy PM-VOICE-TRAFFIC
pixfirewall(config-pmap-c)# exit
pixfirewall(config-pmap)# service-policy PM-ALL-TRAFFIC-SHAPE interface outside
pixfirewall(config)# end

Verifications for Priority Queuing

These verification commands can be used for both forms of priority queuing. Obviously, you can examine portions of the running configuration to confirm your Modular Policy Framework components. For example:

pixfirewall# show run policy-map
!
policy-map PM-VOICE-TRAFFIC
 class CM-VOICE
  priority
 class CM-VOICE-SIGNAL
  priority
 class class-default
policy-map PM-ALL-TRAFFIC-SHAPE
 class class-default
  shape average 2000000 16000
  service-policy PM-VOICE-TRAFFIC
!

Another example:

pixfirewall# show run class-map
!
class-map CM-VOICE-SIGNAL
 match dscp af31
class-map CM-VOICE
 match dscp ef
!

To verify the statistics of the standard priority queuing configuration, use the following:

pixfirewall# show service-policy priority
Interface outside:
  Service-policy: PM-VOICE-TRAFFIC
   Class-map: CM-VOICE
      Priority:
        Interface outside: aggregate drop 0, aggregate transmit 0
    Class-map: CM-VOICE-SIGNAL
      Priority:
        Interface outside: aggregate drop 0, aggregate transmit 0

You can also view the priority queue statistics for an interface using the following:

pixfirewall# show priority-queue statistics outside
Priority-Queue Statistics interface outside
Queue Type         = BE
Tail Drops         = 0
Reset Drops        = 0
Packets Transmit   = 0
Packets Enqueued   = 0
Current Q Length   = 0
Max Q Length       = 0
Queue Type         = LLQ
|Tail Drops         = 0
Reset Drops        = 0
Packets Transmit   = 0
Packets Enqueued   = 0
Current Q Length   = 0
Max Q Length       = 0

To verify the statistics on the shaping you have done with the hierarchical priority queuing, use the following:

pixfirewall# show service-policy shape
Interface outside:
  Service-policy: PM-ALL-TRAFFIC-SHAPE
    Class-map: class-default
      shape (average) cir 2000000, bc 16000, be 16000
      (pkts output/bytes output) 0/0
      (total drops/no-buffer drops) 0/0
      Service-policy: PM-VOICE-TRAFFIC

The next blog entry on this subject will focus on the shape tool available on the PIX/ASA.

Thanks so much for reading!

Sep
12

This blog is focusing on QoS on the PIX/ASA and is based on 7.2 code to be consistent with the CCIE Security Lab Exam as of the date of this post. I will create a later blog regarding new features to 8.X code for all of you non-exam biased readers :-)

NOTE: We have already seen thanks to our readers that some of these features are very model/license dependent! For example, we have yet to find an ASA that allows traffic shaping. 

One of the first things that you discover about QoS for PIX/ASA when you check the documentation is that none of the QoS tools that these devices support are available when you are in multiple context mode. This jumped out at me as a bit strange and I just had to see for myself. Here I went to a PIX device, switched to multiple mode, and then searched for the priority-queue global configuration mode command. Notice that, sure enough, the command was not available in the CUSTA context, or the system context.

pixfirewall# configure terminal
pixfirewall(config)# mode multiple
WARNING: This command will change the behavior of the device
WARNING: This command will initiate a Reboot
Proceed with change mode? [confirm]
Convert the system configuration? [confirm]
pixfirewall> enable
pixfirewall# show mode
Security context mode: multiple
pixfirewall# configure terminal        
pixfirewall(config)# context CUSTA
Creating context 'CUSTA'... Done. (2)
pixfirewall(config-ctx)# context CUSTA
pixfirewall(config-ctx)# config-url flash:/custa.cfg
pixfirewall(config-ctx)# allocate-interface e2 
pixfirewall(config-ctx)# changeto context CUSTA
pixfirewall/CUSTA(config)# pri?     
configure mode commands/options:
privilege
pixfirewall/CUSTA# changeto context system
pixfirewall# conf t
pixfirewall(config)# pr?
configure mode commands/options:
privilege 

OK, so we have no QoS capabilities when in multiple context mode. :-| What QoS capabilities do we possess on the PIX/ASA when we are behaving in single context mode? Here they are:

  • Policing – you will be able to set a “speed limit” for traffic on the PIX/ASA. The policer will discard any packets trying to exceed this rate. I always like to think of the Soup Guy on Seinfeld with this one - "NO BANDWIDTH FOR YOU!" 
  • Shaping – again, this tool allows you to set a speed limit, but it is “kinder and gentler”. This tool will attempt to buffer traffic and send it later should the traffic exceed the shaped rate.
  • Priority Queuing – for traffic (like VoIP that rely hates delays and variable delays (jitter), the PIX/ASA does support priority queuing of that traffic. The documentation refers to this as a Low Latency Queuing (LLQ).

Now before we get too excited about these options for tools, we must understand that we are going to face some pretty big limitations with their usage compared to shaping, policing, and LLQ on a Cisco router. We will detail these limitations in future blogs on the specific tools, but here is an example. We might get very excited when we see LLQ in relation to the PIX/ASA, but it is certainly not the LLQ that we are accustomed to on a router. On a router, LLQ is really Class-Based Weighted Fair Queuing (CBWFQ) with the addition of strict Priority Queuing (PQ). On the PIX/ASA, we are just not going to have that type of granular control over many traffic forms. In fact, with the standard priority queuing approach on the PIX/ASA, there is a single LLQ for your priority traffic and all other traffic falls into a best effort queue.

If you have been around QoS for a while, you are going to be very excited about how we set these mechanisms up on the security appliance. We are going to use the Modular Quality of Service Command Line Interface (MQC) approach! The MQC was invented for CBWFQ on the routers, but now we are seeing it everywhere. In fact, on the security appliance it is termed the Modular Policy Framework. This is because it not only handles QoS configurations, but also traffic inspections (including deep packet inspections), and can be used to configure the Intrusion Prevention and Content Management Security Service Modules. Boy, the ole’ MQC sure has come a long way.

While you might be frustrated with some of the limitations in the individual tools, at least there are a couple of combinations that can feature the tools working together. Specificaly, you can:

  • Use standard priority queueing (for example for voice) and then police for all of the other traffic.
  • You can also use traffic shaping for all traffic in conjunction with hierarchical priority queuing for a subset of traffic. Again, in later blogs we will educate you more fully on each tool.

Thanks for reading and I hope you are looking forward to future blog entries on QoS with the ASA/PIX.

Aug
17

Try assessing your understanding of Cisco's CBWFQ by looking at the following example:

class-map match-all HTTP_R6
match access-group name HTTP_R6
!
policy-map CBWFQ
class HTTP_R6
bandwidth remaining percent 5
!
interface Serial 0/1
bandwidth 128
clock rate 128000
service-policy output CBWFQ

and answering a question on the imaginable scenario: Two TCP flows (think of them as HTTP file transfers) are going across Serial 0/1 interface. One of the flows matches the class HTTP_R6, and another flow, marked with IP Precedence of 7 (pretty high), does not match any class. The traffic flow overwhelms the interface, so the system engages CBWFQ. Now the question is: how CBWFQ will share the interface bandwidth among the flows.

This is not an easy one - you can't asnwer correctly if you stick with the bandwidth allocation logic described on DocCD. First, look at the "answer":

Rack1R4#show policy-map interface serial 0/1
Serial0/1

Service-policy output: CBWFQ

Class-map: HTTP_R6 (match-all)
10982 packets, 6368035 bytes
30 second offered rate 95000 bps, drop rate 0 bps
Match: access-group name HTTP_R6
Queueing
Output Queue: Conversation 41
Bandwidth remaining 5 (%)Max Threshold 64 (packets)
(pkts matched/bytes matched) 10973/6363227
(depth/total drops/no-buffer drops) 8/0/0

Class-map: class-default (match-any)
3429 packets, 1765978 bytes
30 second offered rate 29000 bps, drop rate 0 bps
Match: any

The bandwidth is shared approximately in proportions “3,3:1”. The user configured class has more bandwidth than “class-default”, even though we reserved just 5% of available bandwidth to it. This does not look deterministric - what about the idea that the unused bandwidth goes to “class-default”? The below is the explanation of this behavior, inherent to CBWFQ's design:

The following are the condensed facts about CBWFQ:

1) CBWFQ works the same was as WFQ! You just have the option to use flexible criteria for flow classification using MQC syntax. If you feel lost thinking about WFQ, you may find more information here QoS Teaser: Hold-Queue and WFQ.
2) CBWFQ shares interface bandwidth inversely proportional to flow "weights" (these weights are based on the bandwidth settings as we see later). If you have N flows, where flow “i” has weight value of Weight(i). The CBWFQ will guarantee the flow “i” the following share of bandwidth: Share(i)=(Weight(1)+...+Weight(i)+...+Weight(N))/Weight(i). Thus, flows with smaller weights get more bandwidth. Note that you should treat those values as relative to each other, not as absolute shares.
3) CBWFQ assigns weights to dynamic conversation (flows that don’t match any user-defined class) using the formula Weight(i) = 32384/(IP_Precedence(i)+1) - the same logic found in WFQ.
4) CBWFQ assigns weight to a user-defined class using either of the following formulas:

4.1) Weight(i) = Const*Interface_BW/Class_BW if the class is configured with explicit bandwidth value.
4.2) Weight(i)=Const*100/Bandwidth_Percent if the class is configured with either bandwidth percent or bandwidth remaining percent

Here Const is a special constant that depends on the number of flow queues in WFQ. Cisco never gave any explicit formula, but it looks like the constants are chosen according to the following table (a result of some sweaty modeling experiments):

Number of flows

Constant

16

64

32

64

64

57

128

30

256

16

512

8

1024

4

2048

2

4096

1

That’s all the magic behind the meaning of the “bandwidth” statement. As you can see, user-configurable classes are nothing else than separate conversations within the CBWFQ flows pool. The flow is simply a FIFO queue, scheduled according to its sequence number, which is proportional to flow weight (btw the formula for sequence number is the same as with WFQ!). All flows share the buffer pool that system allocates to CBWFQ, using the hold-queue N out interface-level command where N is the number of buffers. In addition to that, you can even specify WFQ Congestive Discard Threshold using the command queue-limit under the “class-deafult” of your policy map.

!
! Implementing pure WFQ using MQC syntax
!
policy-map WFQ
class-map class-default
!
! number of dynamic flows
!
fair-queue 256
!
! WFQ Congestive Discard Threshold
!
queue-limit 32
!
interface Serial 0/1
no fair-queue
service-policy output WFQ
!
! WFQ total size
!
hold-queue 4096 out

Now look at the following table:

Flow/Conversation Numbers

Weight

Description

Below 2^N

Weight(i)=32384/(IP_Precedence(i)+1)

Dynamic flows, unclassified traffic. This is the classic “fair-queue”.

2^N…2^N+7

Weight(i)=1024

Link Queues. Routing updates, Layer 2 Keepalives etc. Basically it’s the traffic marked as PAK_PRIORITY inside the router.

2^N+8

Weight(i)=0

LLQ or the priority queue. CBWFQ always service this queue first, but de-queued packets are policed using the defined token bucket parameters.

Above 2^N+8

Weight(i) = Const*Interface_BW/Class_BW
OR
Weight(i)=Const*100/Bandwidth_Percent

User-defined classes. Those classes are treated by CBWFQ as the RSVP flows, with relatively low weights. Their weights are almost all the time better than the weights of dynamic flows.

A few notes here. The value of “N” is the base parameter that defines the number of dynamic flows for CBWFQ. Remember you can only specify the number of flows as power of 2, and that “N” is this power value. You configure the number of dynamic flows using the command fair-queue under “class-default”. Next, CBWFQ uses special hash function to distribute unclassified packets in dynamic conversations. They have the same weights as they would have with classic WFQ. Now the Link Queues – we remember they were with WFQ as well. System uses those queues to send critical control plane traffic. The link queues has weight values of 1024, which is much better than any dynamic flows weights, and as we see later are almost on par with user-defined classes weights. Since control plane traffic is intermittent (unless you pump huge BGP tables ;) those flows do not affect bandwidth distribution too much. By the way, the well-known max-reserved-bandwidth 75% rule specifically ensures that link queues will not starve, by preventing a user from allocating too much "weight" to the user-defined classes.

Now, take a quick look at the user-defined classes. Think of two extreme cases:

a) We assigned all interface bandwidth to the class (you may need max-reserved-bandwidth 100). Then the weight value is 64 in the worst case of just 16 flow queues, which is better than almost any other possible weight. This class will get what it wants (almost), no matter what :)
b) We assign small amount of interface bandwidth to the class, e.g. 2%. Then, the class weight is 64*100/2=3200 in the worst case of 16 or 32 flow queues. This is getting close to 32384/(7+1)=4048 which is the weight value for the “best” dynamic queue with IP Precedence value of 7.

From those two facts, we may conclude that user-defined classes dominate dynamic flows almost all the time, unless they have small shares of bandwidth configured. Of course, priority queue beats all, but it is rate-limited (conditionally!), so it can’t starve other conversations, unless you set policer rate to the whole interface bandwidth. By the way, you need to account for layer 2 overhead when setting the rate-limit bandwidth for a priority class. This is important when you are working with voice traffic flows, that has small packet sizes and layer 2 size is significant compared to the payload.

Keeping all those facts in mind, let’s look at the following output – the CBWFQ queue contents from the first configuration sample:

Rack1R4#show queueing interface serial 0/1
Interface Serial0/1 queueing strategy: fair
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: Class-based queueing
Output queue: 12/1000/64/0 (size/max total/threshold/drops)
Conversations 2/5/32 (active/max active/max total)
Reserved Conversations 1/1 (allocated/max allocated)
Available Bandwidth 96 kilobits/sec

(depth/weight/total drops/no-buffer drops/interleaves) 5/1280/0/0/0
Conversation 41, linktype: ip, length: 580
source: 155.1.146.6, destination: 155.1.108.10, id: 0x47F9, ttl: 254,
TOS: 0 prot: 6, source port 80, destination port 11003

(depth/weight/total drops/no-buffer drops/interleaves) 7/4048/0/0/0
Conversation 17, linktype: ip, length: 580
source: 155.1.146.1, destination: 155.1.45.5, id: 0xADC9, ttl: 254,
TOS: 224 prot: 6, source port 80, destination port 56546

Note the weight values for each flow. The weight for user-defined HTTP conversation is 64*100/5=1280, while the weight for dynamic flow is 32384/(7+1)=4048. Thus, using the formula for bandwidth shares, we obtain the following:

Share(1)=(4048+1280)/1280=4,1
Share(2)=(4048+1280)/4048=1,3

Normalize that, dividing by the smallest number which is 1,3, and you will get the proportion “3,1:1” which is pretty close to the distribution we've see above. Some unfairness is probably due to the slow line and large serialization delays.

To summarize what we learned so far:

1) CBWFQ is nothing else than WFQ on steroids ;)
2) User-defined classes have much better scheduling weights than any dynamic flow queue. Therefore, the bandwidth allocated to dynamic queue usually is small compared to any user-defined class.
3) Scheduler shares interface bandwidth in relative proportions. For example, if you have two classes configured with bandwidth values of “32” and “64” and interface bandwidth 128 that does not mean system will allocate classes 32Kbps and 64Kbps. That means: in case on congestion CBWFQ will share bandwidth in proportions 32:64=1:2 between the two classes, plus some small amount to class-default. If you want bandwidth to be "realistic", ensure your entire bandwidth values sum to interface bandwidth. The same goes to bandwidth percents. 4) If you want the scheduler to honor class-default traffic, assign it an explicit bandwidth value. This will effectively disable dynamic flow queues (though preserve Link Queues) and assign all unclassified traffic to a single FIFO queue.

Now a few simple rules to understand how various CBWFQ commands syntax applies in case of interface congestion. All those rules assume that bandwidth weights are large enough to make dynamic flows weights negligible.

1) If you have priority bandwidth configured in your policy map, subtract this value from total interface bandwidth to yield the amount of bandwidth available to other classes. The priority queue is only rate-limited under interface congestion, and in such case, it cannot get more bandwidth than configured with priority statement. Note that in the following text we will refer to priority bandwidth as configured in Kbps, but you may replace its value with priority-percent*interface-bandwidth if you configured rate in percent.

2) Suppose that you configured user-defined classes with bandwidth statement. First, IOS CLI will check that that:

bandwidth(1)+…+bandwidth(N) + priority <= max_reserved_bandwidth*interface_bandwdith/100.

In case of congestion, the scheduler allocates the following amount of bandwidth to class “k”.

share(k)=(interface_bandwidth - priority) * bandwidth(k)/(bandwidth(1)+…+bandwidth(N))  Kbps.

Therefore, as mentioned above, if you want the share to be equal to the bandwidth you set for the class, make sure all bandwidth settings sum to the interface bandwidth.

3) Another case: you configured your classes with bandwidth percent. The IOS CLI performs the following assertion:

[bw_percent(1)+…+bw_percent(N)]*interface_bandwidth + priority <= max_reserved_bandwidth*interface_bandwidth/100

In case of congestion, the scheduler allocates the following amount of bandwidth to class “k”.

share(k)= (interface_bandwidth-priority) * bw_percent(k)/(bw_percent(1)+…+bw_percent(N)).

3) Final case: you configured your classes with bandwidth remaining percent. The IOS CLI performs the following assertion:

bw_rem_percent(1)+…+bw_rem_percent(N) <= 100%

In case of congestion, the scheduler allocates the following amount of bandwidth to class “k”.

share(k)= (interface_bandwidth - priority) * bw_rem_percent(k)/(bw_rem_percent(1)+…+bw_rem_percent(N)).

The funniest thing is that this is the same formula as in the case of simple bandwidth percent. However, the verification is rather simple, and it lets you forget about all those bandwidth computations.

Now you know how to answer that tricky CBWFQ questions :) Cisco never published (at least I never seen that) the information on CBWFQ algorithm. We got all information in this post based on simulations and information available on Cisco WFQ. Hope it helps!

Jan
26

To begin with, why whould anyone need to run Multilink PPP (MLPPP or MLP) with Interleaving over Frame-Relay? Well, back in days, when Frame-Relay and ATM were really popular, there was a need to interwork the two technologies: that is, transparently pass encapsulated packets between FR and ATM PVCs. (This is similar in concept with modern L2 VPN interworking, however it was specific to ATM and Frame-Relay). Let's imagine a situation where we have slow ATM and Frame-Relay links, used to transport a mix of VoIP and data traffic. As we know, some sort of fragmentation and interleaving scheme should be implemented, in order to keep voice quality under control. Since there was no fragmentation scheme common to both ATM and Frame-Relay, people came with idea to run PPP (yet another L2 tech) over Frame-Relay and ATM PVCs and use PPP multilink and interleave feature to implement fragmentation. (Actually there was no good scheme for native fragmentation and interleaving with VoIP over ATM - the cell mode technology - how ironic!)

Before coming up with a configuration example, let's discuss briefly how PPP Multilink and Interleave works. MLPPP is defined under RFC 1990, and it's purpose is to group a number of physical links into one logical channel with larger "effective" bandwidth. As we discussed before, MLPPP uses a fragmentation algorithm, where one large frame is being split at Layer2 and replaced with a bunch of sequenced (by the use of additional MLPPP header) smaller frames which are then being sent over multiple physical links in parallel. The receiving side will then accept fragments, reorder some of them if needed, and assemble the pieces into complete frame using the sequence numbers.

So here comes the interleave feature: small voice packets are not fragmented by MLPPP (no MLPPP header and sequence number added) and are simply inserted (intermixed) among the fragments of large data packet. Of course, a special interleaving priority queue is used for this purpose, as we have discussed before.

To summarize:

1) MLPPP uses fragmentation scheme where large packets are sliced in pieces and sequence numbers are added using special MLPPP headers
2) Small voice packets are interleaved with fragments of large packets using a special priority queue

We see that MLPPP was originally designed to work with multiple physical links at the same time. However, PPP Multilink Interleave only works with one physical link. The reason is that voice (small) packets are being sent without sequence numbers. If we were using multiple physical links, the receiving side may start accepting voice packets out of their original order (due to different physical link latencies). And since voice packets bear no fragmentation headers, there is no way to reorder them. In effect, packets may arrive to their final destination out of order, degrading voice quality.

To overcome this obstacle, Multiclass Multilink PPP (MCMLPPP or MCMLP) has been introduced in RFC 2886. Under this RFC, different "fragment streams" or classes are supported at sending and receiving sides, using independent sequence numbers. Therefore, with MCMLPPP voice packets may be sent using MLPPP header with separate sequence numbers space. In result, MCMPPP permits the use of fragmentation and interleaving over multiple physical links at time.

Now back to our MLPPPoFR example. Let's imagine the situation where we have two routers (R1 and R2) connected via FR cloud, with physical ports clocked at 512Kpbs and PVC CIR values equal to 384Kbps (There is no ATM interworking in this example). We need to provide priority treatment to voice packets and enable PPP Multilink and Interleave to decrease serialization delays.

[R1]---[DLCI 112]---[Frame-Relay]---[DLCI 211]---[R2]

Start by defining MQC policy. We need to make sure that software queue gives voice packets priority treatmet, or else interleaving will be useless

R1 & R2:

!
! Voice bearer
!
class-map VOICE
match ip dscp ef

!
! Voice signaling
!
class-map SIGNALING
match ip dscp cs3

!
! CBWFQ: priority treatment for voice packets
!
policy-map CBWFQ
class VOICE
priority 48
class SIGNALING
bandwidth 8
class class-default
fair-queue

Next create a Virtual-Template interface for PPPoFR. We need to calculate the fragment size for MLPPP. Since physical port speed is 512Kpbs, and required serialization delay should not exceed 10ms (remember, fragment size is based on physical port speed!), the fragment size must be set to 512000/8*0,01=640 bytes. How is the fragment size configured with MLPPP? By using command ppp multilink fragment delay - however, IOS CLI takes this delay value (in milliseconds) and multiplies it by configured interface (virtual-template) bandwidth (in our case 384Kbps). We can actually change the virtual-template bandwidth to match the physical interface speed, but this would affect the CBWFQ weights! Therefore, we take the virtual-template bandwidth (384Kpbs) and adjust the delay to make sure the fragment size matches the physical interace rate is 512Kpbs. This way, the "effective" delay value would be set to "640*8/384 = 13ms" (Fragment_Size/CIR*8) to accomodate the physical and logical bandwidth discrepancy. (This may be unimportant if our physical port speed does not differ much from PVC CIR. However, if you have say PVC CIR=384Kbps and port speed 768Kbps you may want to pay attention to this issue)

R1:
interface Loopback0
ip address 177.1.101.1 255.255.255.255
!
interface Virtual-Template 1
encapsulation ppp
ip unnumbered Loopback 0
bandwidth 384
ppp multilink
ppp multilink interleave
ppp multilink fragment delay 13
service-policy output CBWFQ

R2:
interface Loopback0
ip address 177.1.102.1 255.255.255.255
!
interface Virtual-Template 1
encapsulation ppp
ip unnumbered Loopback 0
bandwidth 384
ppp multilink
ppp multilink interleave
ppp multilink fragment delay 13
service-policy output CBWFQ

Next we configure PVC shaping settings by using legacy FRTS configuration. Note that Bc is set to CIR*10ms.

R1 & R2:
map-class frame-relay SHAPE_384K
frame-relay cir 384000
frame-relay mincir 384000
frame-relay bc 3840
frame-relay be 0

Finally we apply all the settings to the Frame-Relay interfaces:

R1:
interface Serial 0/0/0:0
encapsulation frame-relay
frame-relay traffic-shaping
!
! Virtual Template bound to PVC
!
interface Serial 0/0/0:0.1 point-to-point
no ip address
frame-relay interface-dlci 112 ppp virtual-template 1
class SHAPE_384K

R2:
interface Serial 0/0/1:0
encapsulation frame-relay
frame-relay traffic-shaping
!
! Virtual Template bound to PVC
!
interface Serial 0/0/1:0.1 point-to-point
no ip address
no frame-relay interface-dlci 221
frame-relay interface-dlci 211 ppp virtual-Template 1
class SHAPE_384K

Verification

Two virtual-access interfaces have been cloned. First for the member link:

R1#show interfaces virtual-access 2
Virtual-Access2 is up, line protocol is up
Hardware is Virtual Access interface
Interface is unnumbered. Using address of Loopback0 (177.1.101.1)
MTU 1500 bytes, BW 384 Kbit, DLY 100000 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation PPP, LCP Open, multilink Open
Link is a member of Multilink bundle Virtual-Access3 <---- MLP bundle member
PPPoFR vaccess, cloned from Virtual-Template1
Vaccess status 0x44
Bound to Serial0/0/0:0.1 DLCI 112, Cloned from Virtual-Template1, loopback not set
Keepalive set (10 sec)
DTR is pulsed for 5 seconds on reset
Last input 00:00:52, output never, output hang never
Last clearing of "show interface" counters 00:04:17
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo <---------- FIFO is the member link queue
Output queue: 0/40 (size/max)
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 0 bits/sec, 0 packets/sec
75 packets input, 16472 bytes, 0 no buffer
Received 0 broadcasts, 0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
86 packets output, 16601 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets
0 output buffer failures, 0 output buffers swapped out
0 carrier transitions

Second for the MLPPP bundle itself:

R1#show interfaces virtual-access 3
Virtual-Access3 is up, line protocol is up
Hardware is Virtual Access interface
Interface is unnumbered. Using address of Loopback0 (177.1.101.1)
MTU 1500 bytes, BW 384 Kbit, DLY 100000 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation PPP, LCP Open, multilink Open
Open: IPCP
MLP Bundle vaccess, cloned from Virtual-Template1 <---------- MLP Bundle
Vaccess status 0x40, loopback not set
Keepalive set (10 sec)
DTR is pulsed for 5 seconds on reset
Last input 00:01:29, output never, output hang never
Last clearing of "show interface" counters 00:03:40
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: Class-based queueing <--------- CBWFQ is the bundle queue
Output queue: 0/1000/64/0 (size/max total/threshold/drops)
Conversations 0/1/128 (active/max active/max total)
Reserved Conversations 1/1 (allocated/max allocated)
Available Bandwidth 232 kilobits/sec
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 0 bits/sec, 0 packets/sec
17 packets input, 15588 bytes, 0 no buffer
Received 0 broadcasts, 0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
17 packets output, 15924 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets
0 output buffer failures, 0 output buffers swapped out
0 carrier transitions

Verify the CBWFQ policy-map:

R1#show policy-map interface
Virtual-Template1

Service-policy output: CBWFQ

Service policy content is displayed for cloned interfaces only such as vaccess and sessions
Virtual-Access3

Service-policy output: CBWFQ

Class-map: VOICE (match-all)
0 packets, 0 bytes
5 minute offered rate 0 bps, drop rate 0 bps
Match: ip dscp ef (46)
Queueing
Strict Priority
Output Queue: Conversation 136
Bandwidth 48 (kbps) Burst 1200 (Bytes)
(pkts matched/bytes matched) 0/0
(total drops/bytes drops) 0/0

Class-map: SIGNALING (match-all)
0 packets, 0 bytes
5 minute offered rate 0 bps, drop rate 0 bps
Match: ip dscp cs3 (24)
Queueing
Output Queue: Conversation 137
Bandwidth 8 (kbps) Max Threshold 64 (packets)
(pkts matched/bytes matched) 0/0
(depth/total drops/no-buffer drops) 0/0/0

Class-map: class-default (match-any)
17 packets, 15554 bytes
5 minute offered rate 0 bps, drop rate 0 bps
Match: any
Queueing
Flow Based Fair Queueing
Maximum Number of Hashed Queues 128
(total queued/total drops/no-buffer drops) 0/0/0

Check PPP multilink status:

R1#ping 177.1.102.1 source loopback 0 size 1500

Type escape sequence to abort.
Sending 5, 1500-byte ICMP Echos to 177.1.102.1, timeout is 2 seconds:
Packet sent with a source address of 177.1.101.1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 64/64/64 ms

R1#show ppp multilink

Virtual-Access3, bundle name is R2
Endpoint discriminator is R2
Bundle up for 00:07:49, total bandwidth 384, load 1/255
Receive buffer limit 12192 bytes, frag timeout 1000 ms
Interleaving enabled <------- Interleaving enabled
0/0 fragments/bytes in reassembly list
0 lost fragments, 0 reordered
0/0 discarded fragments/bytes, 0 lost received
0x34 received sequence, 0x34 sent sequence <---- MLP sequence numbers for fragmented packets
Member links: 1 (max not set, min not set)
Vi2, since 00:07:49, 624 weight, 614 frag size <------- Fragment Size
No inactive multilink interfaces

Verify the interleaving queue:

R1#show interfaces serial 0/0/0:0
Serial0/0/0:0 is up, line protocol is up
Hardware is GT96K Serial
MTU 1500 bytes, BW 1536 Kbit, DLY 20000 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation FRAME-RELAY, loopback not set
Keepalive set (10 sec)
LMI enq sent 10, LMI stat recvd 11, LMI upd recvd 0, DTE LMI up
LMI enq recvd 0, LMI stat sent 0, LMI upd sent 0
LMI DLCI 1023 LMI type is CISCO frame relay DTE
FR SVC disabled, LAPF state down
Broadcast queue 0/64, broadcasts sent/dropped 4/0, interface broadcasts 0
Last input 00:00:05, output 00:00:02, output hang never
Last clearing of "show interface" counters 00:01:53
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: dual fifo <--------- Dual FIFO
Output queue: high size/max/dropped 0/256/0 <--------- High Queue
Output queue: 0/128 (size/max) <--------- Low (fragments) queue
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 0 bits/sec, 0 packets/sec
47 packets input, 3914 bytes, 0 no buffer
Received 0 broadcasts, 0 runts, 0 giants, 0 throttles
1 input errors, 1 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
47 packets output, 2149 bytes, 0 underruns
0 output errors, 0 collisions, 4 interface resets
0 output buffer failures, 0 output buffers swapped out
1 carrier transitions
Timeslot(s) Used:1-24, SCC: 0, Transmitter delay is 0 flags

Further Reading

Reducing Latency and Jitter for Real-Time Traffic Using Multilink PPP
Multiclass Multilink PPP
Using Multilink PPP over Frame Relay

Subscribe to INE Blog Updates

New Blog Posts!