Jan
26

To begin with, why whould anyone need to run Multilink PPP (MLPPP or MLP) with Interleaving over Frame-Relay? Well, back in days, when Frame-Relay and ATM were really popular, there was a need to interwork the two technologies: that is, transparently pass encapsulated packets between FR and ATM PVCs. (This is similar in concept with modern L2 VPN interworking, however it was specific to ATM and Frame-Relay). Let's imagine a situation where we have slow ATM and Frame-Relay links, used to transport a mix of VoIP and data traffic. As we know, some sort of fragmentation and interleaving scheme should be implemented, in order to keep voice quality under control. Since there was no fragmentation scheme common to both ATM and Frame-Relay, people came with idea to run PPP (yet another L2 tech) over Frame-Relay and ATM PVCs and use PPP multilink and interleave feature to implement fragmentation. (Actually there was no good scheme for native fragmentation and interleaving with VoIP over ATM - the cell mode technology - how ironic!)

Before coming up with a configuration example, let's discuss briefly how PPP Multilink and Interleave works. MLPPP is defined under RFC 1990, and it's purpose is to group a number of physical links into one logical channel with larger "effective" bandwidth. As we discussed before, MLPPP uses a fragmentation algorithm, where one large frame is being split at Layer2 and replaced with a bunch of sequenced (by the use of additional MLPPP header) smaller frames which are then being sent over multiple physical links in parallel. The receiving side will then accept fragments, reorder some of them if needed, and assemble the pieces into complete frame using the sequence numbers.

So here comes the interleave feature: small voice packets are not fragmented by MLPPP (no MLPPP header and sequence number added) and are simply inserted (intermixed) among the fragments of large data packet. Of course, a special interleaving priority queue is used for this purpose, as we have discussed before.

To summarize:

1) MLPPP uses fragmentation scheme where large packets are sliced in pieces and sequence numbers are added using special MLPPP headers
2) Small voice packets are interleaved with fragments of large packets using a special priority queue

We see that MLPPP was originally designed to work with multiple physical links at the same time. However, PPP Multilink Interleave only works with one physical link. The reason is that voice (small) packets are being sent without sequence numbers. If we were using multiple physical links, the receiving side may start accepting voice packets out of their original order (due to different physical link latencies). And since voice packets bear no fragmentation headers, there is no way to reorder them. In effect, packets may arrive to their final destination out of order, degrading voice quality.

To overcome this obstacle, Multiclass Multilink PPP (MCMLPPP or MCMLP) has been introduced in RFC 2886. Under this RFC, different "fragment streams" or classes are supported at sending and receiving sides, using independent sequence numbers. Therefore, with MCMLPPP voice packets may be sent using MLPPP header with separate sequence numbers space. In result, MCMPPP permits the use of fragmentation and interleaving over multiple physical links at time.

Now back to our MLPPPoFR example. Let's imagine the situation where we have two routers (R1 and R2) connected via FR cloud, with physical ports clocked at 512Kpbs and PVC CIR values equal to 384Kbps (There is no ATM interworking in this example). We need to provide priority treatment to voice packets and enable PPP Multilink and Interleave to decrease serialization delays.

[R1]---[DLCI 112]---[Frame-Relay]---[DLCI 211]---[R2]

Start by defining MQC policy. We need to make sure that software queue gives voice packets priority treatmet, or else interleaving will be useless

R1 & R2:

!
! Voice bearer
!
class-map VOICE
match ip dscp ef

!
! Voice signaling
!
class-map SIGNALING
match ip dscp cs3

!
! CBWFQ: priority treatment for voice packets
!
policy-map CBWFQ
class VOICE
priority 48
class SIGNALING
bandwidth 8
class class-default
fair-queue

Next create a Virtual-Template interface for PPPoFR. We need to calculate the fragment size for MLPPP. Since physical port speed is 512Kpbs, and required serialization delay should not exceed 10ms (remember, fragment size is based on physical port speed!), the fragment size must be set to 512000/8*0,01=640 bytes. How is the fragment size configured with MLPPP? By using command ppp multilink fragment delay - however, IOS CLI takes this delay value (in milliseconds) and multiplies it by configured interface (virtual-template) bandwidth (in our case 384Kbps). We can actually change the virtual-template bandwidth to match the physical interface speed, but this would affect the CBWFQ weights! Therefore, we take the virtual-template bandwidth (384Kpbs) and adjust the delay to make sure the fragment size matches the physical interace rate is 512Kpbs. This way, the "effective" delay value would be set to "640*8/384 = 13ms" (Fragment_Size/CIR*8) to accomodate the physical and logical bandwidth discrepancy. (This may be unimportant if our physical port speed does not differ much from PVC CIR. However, if you have say PVC CIR=384Kbps and port speed 768Kbps you may want to pay attention to this issue)

R1:
interface Loopback0
ip address 177.1.101.1 255.255.255.255
!
interface Virtual-Template 1
encapsulation ppp
ip unnumbered Loopback 0
bandwidth 384
ppp multilink
ppp multilink interleave
ppp multilink fragment delay 13
service-policy output CBWFQ

R2:
interface Loopback0
ip address 177.1.102.1 255.255.255.255
!
interface Virtual-Template 1
encapsulation ppp
ip unnumbered Loopback 0
bandwidth 384
ppp multilink
ppp multilink interleave
ppp multilink fragment delay 13
service-policy output CBWFQ

Next we configure PVC shaping settings by using legacy FRTS configuration. Note that Bc is set to CIR*10ms.

R1 & R2:
map-class frame-relay SHAPE_384K
frame-relay cir 384000
frame-relay mincir 384000
frame-relay bc 3840
frame-relay be 0

Finally we apply all the settings to the Frame-Relay interfaces:

R1:
interface Serial 0/0/0:0
encapsulation frame-relay
frame-relay traffic-shaping
!
! Virtual Template bound to PVC
!
interface Serial 0/0/0:0.1 point-to-point
no ip address
frame-relay interface-dlci 112 ppp virtual-template 1
class SHAPE_384K

R2:
interface Serial 0/0/1:0
encapsulation frame-relay
frame-relay traffic-shaping
!
! Virtual Template bound to PVC
!
interface Serial 0/0/1:0.1 point-to-point
no ip address
no frame-relay interface-dlci 221
frame-relay interface-dlci 211 ppp virtual-Template 1
class SHAPE_384K

Verification

Two virtual-access interfaces have been cloned. First for the member link:

R1#show interfaces virtual-access 2
Virtual-Access2 is up, line protocol is up
Hardware is Virtual Access interface
Interface is unnumbered. Using address of Loopback0 (177.1.101.1)
MTU 1500 bytes, BW 384 Kbit, DLY 100000 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation PPP, LCP Open, multilink Open
Link is a member of Multilink bundle Virtual-Access3 <---- MLP bundle member
PPPoFR vaccess, cloned from Virtual-Template1
Vaccess status 0x44
Bound to Serial0/0/0:0.1 DLCI 112, Cloned from Virtual-Template1, loopback not set
Keepalive set (10 sec)
DTR is pulsed for 5 seconds on reset
Last input 00:00:52, output never, output hang never
Last clearing of "show interface" counters 00:04:17
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo <---------- FIFO is the member link queue
Output queue: 0/40 (size/max)
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 0 bits/sec, 0 packets/sec
75 packets input, 16472 bytes, 0 no buffer
Received 0 broadcasts, 0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
86 packets output, 16601 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets
0 output buffer failures, 0 output buffers swapped out
0 carrier transitions

Second for the MLPPP bundle itself:

R1#show interfaces virtual-access 3
Virtual-Access3 is up, line protocol is up
Hardware is Virtual Access interface
Interface is unnumbered. Using address of Loopback0 (177.1.101.1)
MTU 1500 bytes, BW 384 Kbit, DLY 100000 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation PPP, LCP Open, multilink Open
Open: IPCP
MLP Bundle vaccess, cloned from Virtual-Template1 <---------- MLP Bundle
Vaccess status 0x40, loopback not set
Keepalive set (10 sec)
DTR is pulsed for 5 seconds on reset
Last input 00:01:29, output never, output hang never
Last clearing of "show interface" counters 00:03:40
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: Class-based queueing <--------- CBWFQ is the bundle queue
Output queue: 0/1000/64/0 (size/max total/threshold/drops)
Conversations 0/1/128 (active/max active/max total)
Reserved Conversations 1/1 (allocated/max allocated)
Available Bandwidth 232 kilobits/sec
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 0 bits/sec, 0 packets/sec
17 packets input, 15588 bytes, 0 no buffer
Received 0 broadcasts, 0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
17 packets output, 15924 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets
0 output buffer failures, 0 output buffers swapped out
0 carrier transitions

Verify the CBWFQ policy-map:

R1#show policy-map interface
Virtual-Template1

Service-policy output: CBWFQ

Service policy content is displayed for cloned interfaces only such as vaccess and sessions
Virtual-Access3

Service-policy output: CBWFQ

Class-map: VOICE (match-all)
0 packets, 0 bytes
5 minute offered rate 0 bps, drop rate 0 bps
Match: ip dscp ef (46)
Queueing
Strict Priority
Output Queue: Conversation 136
Bandwidth 48 (kbps) Burst 1200 (Bytes)
(pkts matched/bytes matched) 0/0
(total drops/bytes drops) 0/0

Class-map: SIGNALING (match-all)
0 packets, 0 bytes
5 minute offered rate 0 bps, drop rate 0 bps
Match: ip dscp cs3 (24)
Queueing
Output Queue: Conversation 137
Bandwidth 8 (kbps) Max Threshold 64 (packets)
(pkts matched/bytes matched) 0/0
(depth/total drops/no-buffer drops) 0/0/0

Class-map: class-default (match-any)
17 packets, 15554 bytes
5 minute offered rate 0 bps, drop rate 0 bps
Match: any
Queueing
Flow Based Fair Queueing
Maximum Number of Hashed Queues 128
(total queued/total drops/no-buffer drops) 0/0/0

Check PPP multilink status:

R1#ping 177.1.102.1 source loopback 0 size 1500

Type escape sequence to abort.
Sending 5, 1500-byte ICMP Echos to 177.1.102.1, timeout is 2 seconds:
Packet sent with a source address of 177.1.101.1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 64/64/64 ms

R1#show ppp multilink

Virtual-Access3, bundle name is R2
Endpoint discriminator is R2
Bundle up for 00:07:49, total bandwidth 384, load 1/255
Receive buffer limit 12192 bytes, frag timeout 1000 ms
Interleaving enabled <------- Interleaving enabled
0/0 fragments/bytes in reassembly list
0 lost fragments, 0 reordered
0/0 discarded fragments/bytes, 0 lost received
0x34 received sequence, 0x34 sent sequence <---- MLP sequence numbers for fragmented packets
Member links: 1 (max not set, min not set)
Vi2, since 00:07:49, 624 weight, 614 frag size <------- Fragment Size
No inactive multilink interfaces

Verify the interleaving queue:

R1#show interfaces serial 0/0/0:0
Serial0/0/0:0 is up, line protocol is up
Hardware is GT96K Serial
MTU 1500 bytes, BW 1536 Kbit, DLY 20000 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation FRAME-RELAY, loopback not set
Keepalive set (10 sec)
LMI enq sent 10, LMI stat recvd 11, LMI upd recvd 0, DTE LMI up
LMI enq recvd 0, LMI stat sent 0, LMI upd sent 0
LMI DLCI 1023 LMI type is CISCO frame relay DTE
FR SVC disabled, LAPF state down
Broadcast queue 0/64, broadcasts sent/dropped 4/0, interface broadcasts 0
Last input 00:00:05, output 00:00:02, output hang never
Last clearing of "show interface" counters 00:01:53
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: dual fifo <--------- Dual FIFO
Output queue: high size/max/dropped 0/256/0 <--------- High Queue
Output queue: 0/128 (size/max) <--------- Low (fragments) queue
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 0 bits/sec, 0 packets/sec
47 packets input, 3914 bytes, 0 no buffer
Received 0 broadcasts, 0 runts, 0 giants, 0 throttles
1 input errors, 1 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
47 packets output, 2149 bytes, 0 underruns
0 output errors, 0 collisions, 4 interface resets
0 output buffer failures, 0 output buffers swapped out
1 carrier transitions
Timeslot(s) Used:1-24, SCC: 0, Transmitter delay is 0 flags

Further Reading

Reducing Latency and Jitter for Real-Time Traffic Using Multilink PPP
Multiclass Multilink PPP
Using Multilink PPP over Frame Relay

Petr Lapukhov, 4xCCIE/CCDE
About Petr Lapukhov, 4xCCIE/CCDE

Subscribe to INE Blog Updates