blog
    OER/PfR... it's always wa ...
    31 December 09

    OER/PfR... it's always watching.

    Posted byINE
    facebooktwitterlinkedin
    news-featured

    Benjamin Franklin was quoted as saying "You may delay but time will not".  We may also say that "Email may tolerate delay, but VOIP will not".  Performance Routing (PfR), previously called Optimized Edge Routing (OER), is designed to solve network performance issues, such as delay, that vanilla IP routing can’t.  In this blog, we will use the term OER as that is still the command set used.

    Petr and the team have created some awesome new labs on OER, including Volume 1, v5 in the IP Routing section. Our workbook content is extensive, has great explanations for everything and is highly recommended it to anyone preparing for the v4 CCIE RS lab.

    One of the big challenges for the OER newcomer is knowing what to expect after configuring OER, and how to see measurable results. My intention in this blog post, is to assist the learner by providing a jump start into OER/PfR, by walking you through a solid configuration, and sharing a few critical show and debug commands to verify that OER is working. For you dynamips/GNS3 fans, the configuration may be replicated using (C3725-ADVENTERPRISEK9-M), Version 12.4(15)T10, RELEASE SOFTWARE (fc3).

    So, how does it work?   It is actually very simple.  In the world of OER, there are 2 roles. The brains of the operation, called the master controller, and the border router who owns the exit interfaces. There may be multiple border routers. The key is there needs to be at least 2 or more exit points across the borders being managed by a single controller. Although the controller and border can actually run on the same router, I broke it up into separate devices for this blog post. R1 will be the OER Master Controller, and R2 will be the OER Border. Here is the topology.

    Single Border, with 2 exit interfaces.

    First we will create key chain on the controller R1.  The key is used for communications between the controller and the border.

    R1(config)# key chain cisco
    R1(config-keychain)# key 1
    R1(config-keychain-key)# key-string cisco

    Next, on R1 we will configure the master controller portion. About 98 % of all the configuration for OER is done on the controller. Here we will enable logging, and specify who the border(s) are, along with which interfaces on the border are exit interfaces to the outside world.

    R1(config)#oer master
    R1(config-oer-mc)#logging
    R1(config-oer-mc)#border 136.1.12.2 key-chain cisco
    R1(config-oer-mc-br)#interface Tunnel1 external
    R1(config-oer-mc-br-if)#interface Tunnel2 external
    R1(config-oer-mc-br-if)#interface FastEthernet0/0 internal
    R1(config-oer-mc-br)#exit

    Next on the controller, we  specify that we will learn prefixes based on throughput and delay, set time periods regarding learning, and tell the controller the network lengths to learn. In this example, due to the small network, we are going to have the controller learn /32 routes. We could also specify a maximum number of prefixes to learn as well. (Note: In a production network, you would not want thousands of static routes for sites visited by users. There is a default maximum, and the default aggregation is /24).

    R1(config-oer-mc)#learn
    R1(config-oer-mc-learn)#throughput
    R1(config-oer-mc-learn)#delay
    R1(config-oer-mc-learn)#periodic-interval 1
    R1(config-oer-mc-learn)#monitor-period 2
    R1(config-oer-mc-learn)#expire after time 30
    R1(config-oer-mc-learn)#aggregation-type prefix-length 32
    R1(config-oer-mc-learn)#exit

    Next, now that we have exited out of “learn mode”, we can configure some of the other properties of the controller, such as how often to make prefix decisions (the backoff period), allow it to put routes in the routing table (mode route control), and choose the best exit interface (mode select-exit best). OER will create /32 static routes that use that use the “best” interface to reach the destination network.

    R1(config-oer-mc)#backoff 180 360
    R1(config-oer-mc)#mode route control
    R1(config-oer-mc)#mode select-exit best

    A few additional options include how often to make policy decisions, (periodic command), and which policy items are the most important regarding making those policy decisions (resolve commands).  Highest priority is 1, and lowest is 10.

    R1(config-oer-mc)#periodic 90
    R1(config-oer-mc)#resolve loss priority 1 variance 1
    R1(config-oer-mc)#resolve delay priority 2 variance 1
    R1(config-oer-mc)#resolve utilization priority 3 variance 1
    R1(config-oer-mc)#resolve range priority 4

    That is it for now on the controller. Let’s visit the border router R2 next. We’ll create the key chain, and tell R2 who the controller is. We will also tell R2 which interface to use for sending an active probe, (in the event the controller requests one).  Active probes are one method (from many available within OER) to determine delay.

    R2(config)# key chain cisco
    R2(config-keychain)# key 1
    R2(config-keychain-key)# key-string cisco

    R2(config)#oer border
    R2(config-oer-br)#local FastEthernet0/0
    R2(config-oer-br)#master 136.1.12.1 key-chain cisco
    R2(config-oer-br)#active-probe address source interface Loopback0

    So, how do we verify this? Since we are at R2, let’s start by issuing a show oer border on R2.

    R2#show oer border
    OER BR 136.1.12.2 ACTIVE, MC 136.1.12.1 UP/DOWN: UP 00:00:03,
    Auth Failures: 0
    Conn Status: SUCCESS, PORT: 3949
    Version: 2.1 MC Version: 2.1
    Exits
    Fa0/0 INTERNAL Tu1 EXTERNAL Tu2 EXTERNAL
    R2#

    From this we see that we have successfully connected with the controller (R1), and that the controller has told R2 which interfaces are internal and which are external, based on what we configured on R1.

    On R1, we can issue a few commands to confirm our configuration and connectivity to our border as well. Any elements shown in the output that we didn’t configure are based on the default settings.   Lets jump over to R1 and take a look.

    R1#show oer master
    OER state: ENABLED and ACTIVE
    Conn Status: SUCCESS, PORT: 3949
    Version: 2.1
    Number of Border routers: 1 Number of Exits: 2
    Number of monitored prefixes: 0 (max 5000)
    Max prefixes: total 5000 learn 2500
    Prefix count: total 0, learn 0, cfg 0

    Border Status UP/DOWN AuthFail Version 136.1.12.2 ACTIVE UP 00:00:12 0 2.1
    ! Note: Below, items in bold are derived from our configuration.
    Global Settings:
    max-range-utilization percent 0 recv 0
    mode route metric bgp local-pref 5000
    mode route metric static tag 5000
    trace probe delay 1000
    logging

    Default Policy Settings:
    backoff 180 360 180
    delay relative 50
    holddown 300
    periodic 90
    probe frequency 56
    mode route control
    mode monitor both
    mode select-exit best
    loss relative 10
    jitter threshold 20
    mos threshold 3.60 percent 30
    unreachable relative 50
    resolve loss priority 1 variance 1 resolve delay priority 2 variance 1 resolve utilization priority 3 variance 1 resolve range priority 4 variance 0

    Learn Settings:
    current state : RETRY
    time remaining in current state : 26 seconds
    throughput delay
    no inside bgp
    no protocol
    monitor-period 2 periodic-interval 1
    aggregation-type prefix-length 32
    prefixes 200
    expire after time 30
    R1#

    As a reference below, on R2, here are the IP addresses and Interfaces in use.   Note, that Serial 0/0 is used only as a transport for the GRE tunnels.

    R2#show ip int brief | ex una
    Interface IP-Address OK? Method Status Protocol
    FastEthernet0/0 136.1.12.2 YES NVRAM up up
    Serial0/0 136.1.23.2 YES NVRAM up up
    Loopback0 150.1.2.2 YES NVRAM up up
    Tunnel1 10.0.0.2 YES NVRAM up up
    Tunnel2 20.0.0.2 YES NVRAM up up
    R2#

    Here is the initial routing table on R2, (before OER has dynamically learned routes). We have only 2 static routes, which originated from the startup configuration.

    R2#show ip route
    Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
    D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
    N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
    E1 - OSPF external type 1, E2 - OSPF external type 2
    i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
    ia - IS-IS inter area, * - candidate default, U - per-user static route
    o - ODR, P - periodic downloaded static route

    Gateway of last resort is 20.0.0.3 to network 0.0.0.0

    136.1.0.0/24 is subnetted, 2 subnets
    C 136.1.12.0 is directly connected, FastEthernet0/0
    C 136.1.23.0 is directly connected, Serial0/0

    20.0.0.0/24 is subnetted, 1 subnets
    C 20.0.0.0 is directly connected, Tunnel2

    10.0.0.0/24 is subnetted, 1 subnets
    C 10.0.0.0 is directly connected, Tunnel1

    150.1.0.0/24 is subnetted, 1 subnets
    C 150.1.2.0 is directly connected, Loopback0

    S* 0.0.0.0/0 [1/0] via 20.0.0.3 [1/0] via 10.0.0.3
    R2#

    R2#show run | inc route
    ip route 0.0.0.0 0.0.0.0 10.0.0.3 ip route 0.0.0.0 0.0.0.0 20.0.0.3

    Now for the fun part. (Really!:) Lets generate some traffic in the background, and allow OER to dynamically learn the specific routes, as well as learn the best path to those routes and watch those routes be added to the routing table on R2. We’ll create two SLAs from R1 directed to the loopbacks of R4 and R5.

    R1

    R1(config)#ip sla 1
    R1(config-ip-sla)#tcp-connect 150.1.4.4 4000
    R1(config-ip-sla-tcp)#timeout 100
    R1(config-ip-sla-tcp)#frequency 1
    R1(config-ip-sla-tcp)#exit
    R1(config)#ip sla schedule 1 life forever start-time now
    R1(config)#
    R1(config)#ip sla 2
    R1(config-ip-sla)#tcp-connect 150.1.5.5 5000
    R1(config-ip-sla-tcp)#timeout 100
    R1(config-ip-sla-tcp)#frequency 1
    R1(config-ip-sla-tcp)#exit
    R1(config)#ip sla schedule 2 life forever start-time now

    On R4 and R5, we need to set up the SLA responder, so they will respond correctly to the tcp-connect requests sourced from R1.

    R4 and R5

    R4(config)#ip sla responder
    R5(config)#ip sla responder

    On R2, lets see what networks have been learned. (Note: Based on the timers, it may take a ~minute or so for the /32 networks of 150.1.4.4 and 150.1.5.5 to be learned.   I waited at least a couple minutes before issuing the commands below.   You may want to verify that your SLA traffic is reaching R4 and R5 if you need to troubleshoot).

    R2#show oer border passive cache prefix
    

    OER Passive Prefix Cache, State: enabled, 278544 bytes
    4 active, 4092 inactive, 1222 added
    67316 ager polls, 0 flow alloc failures
    Active flows timeout in 1 minutes
    Inactive flows timeout in 15 seconds
    IP Sub Flow Cache, 25800 bytes
    12 active, 1012 inactive, 3666 added, 1222 added to flow
    0 alloc failures, 0 force free
    1 chunk, 30 chunks added
    0 flows not aggregated due to main subcache starvation

    Prefix NextHop Src If Dst If Flows
    Pkts B/Pk Active sDly #Dly PktLos #UnRch
    ------------------------------------------------------------------------
    150.1.4.4/32 0.0.0.0 Tu1 Fa0/0 24
    34 38 18.5 0 0 0 0
    150.1.5.5/32 0.0.0.0 Tu1 Fa0/0 24
    34 38 18.1 0 0 0 0
    150.1.4.4/32 20.0.0.3 Fa0/0 Tu2 23
    35 62 18.5 64 5 0 0
    150.1.5.5/32 20.0.0.3 Fa0/0 Tu2 23
    35 62 18.1 84 5 0 0
    R2#

    Now, lets turn on a debug of active probes on the border, and see what we can learn from it.

    R2#debug oer border active-probes detail
    <snip>
    OER BR APE detail: Completed retrieving Probe Statistics.
    probeType = echo, probeTarget = 150.1.4.4, probeTargetPort = 0
    probeSource = 150.1.2.2, probeSourcePort = 0, probeNextHop = 10.0.0.3
    probeIfIndex = 10, SAA index = 339 probeToS = 0 policy_seq = 0
    OER BR APE detail: Completions 2, Sum of rtt 40, Max rtt 24, Min rtt 16
    <snip>
    OER BR APE detail: Completed retrieving Probe Statistics.
    probeType = echo, probeTarget = 150.1.4.4, probeTargetPort = 0
    probeSource = 150.1.2.2, probeSourcePort = 0, probeNextHop = 20.0.0.3
    probeIfIndex = 11, SAA index = 340 probeToS = 0 policy_seq = 0
    OER BR APE detail: Completions 2, Sum of rtt 52, Max rtt 28, Min rtt 24

    Based on this, the rtt values from Tunnel 1 (using 10.0.0.3 as the next hop to reach 150.1.4.4) are lower than Tunnel 2 (which is using 20.0.0.3 as the next hop). As a result, OER believes that Tunnel 1 is the best path and will inject a static /32 route for destination 150.1.4.4, using Tunnel 1 as the exit interface and 10.0.0.3 as the next hop. A similar result was discovered for 150.1.5.5 as well. As a result, both /32 routes are dynamically added to the routing table on R2.

    R2#show ip route
    Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
    D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
    N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
    E1 - OSPF external type 1, E2 - OSPF external type 2
    i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
    ia - IS-IS inter area, * - candidate default, U - per-user static route
    o - ODR, P - periodic downloaded static route

    Gateway of last resort is 20.0.0.3 to network 0.0.0.0

    136.1.0.0/24 is subnetted, 2 subnets
    C 136.1.12.0 is directly connected, FastEthernet0/0
    C 136.1.23.0 is directly connected, Serial0/0
    20.0.0.0/24 is subnetted, 1 subnets
    C 20.0.0.0 is directly connected, Tunnel2
    10.0.0.0/24 is subnetted, 1 subnets
    C 10.0.0.0 is directly connected, Tunnel1
    150.1.0.0/16 is variably subnetted, 3 subnets, 2 masks
    C 150.1.2.0/24 is directly connected, Loopback0
    ! OER dynamically added the 2 static routes below
    S 150.1.5.5/32 [1/0] via 10.0.0.3 S 150.1.4.4/32 [1/0] via 10.0.0.3
    S* 0.0.0.0/0 [1/0] via 20.0.0.3
    [1/0] via 10.0.0.3
    R2#

    Also, on R1, the controller, there were a couple messages logged:

    %OER_MC-5-NOTICE: Route changed Prefix 150.1.5.5/32, BR 136.1.12.2, i/f Tu1, Reason Delay, OOP Reason Timer Expired
    %OER_MC-5-NOTICE: Route changed Prefix 150.1.4.4/32, BR 136.1.12.2, i/f Tu1, Reason Delay, OOP Reason Timer Expired

    So now that OER has decided that Tunnel 1, using 10.0.0.3 is the best path for those 2 networks, lets tip the scale in the other direction by using generic  traffic shaping on Tunnel 1, so that the delay there will be worse than Tunnel 2, and watch the results. (Note: in addition to the generic traffic shaping on R2 Tunnel 1, I also ran an extended ping sourced from 150.1.4.4 and 150.1.5.5 to increase the overall delay).

    R2(config-if)#traffic-shape rate 8000 1000 0
    R2(config-if)#do show traffic-shape

    Interface Tu1
    Access Target Byte Sustain Excess Interval Increment Adapt
    VC List Rate Limit bits/int bits/int (ms) (bytes) Active
    - 8000 125 1000 0 125 125 -

    R2#
    ! Below is debug output showing Tunnel 2 with next hop of 20.0.0.3 ! as having the least delay for destination 150.1.4.4 ...

    OER BR APE detail: Completed retrieving Probe Statistics.
    probeType = echo, probeTarget = 150.1.4.4, probeTargetPort = 0
    probeSource = 150.1.2.2, probeSourcePort = 0, probeNextHop = 10.0.0.3
    probeIfIndex = 10, SAA index = 363 probeToS = 0 policy_seq = 0
    OER BR APE detail: Completions 1, Sum of rtt 88, Max rtt 88, Min rtt 88
    OER BR APE detail: Completed retrieving Probe Statistics.
    probeType = echo, probeTarget = 150.1.4.4, probeTargetPort = 0
    probeSource = 150.1.2.2, probeSourcePort = 0, probeNextHop = 20.0.0.3
    probeIfIndex = 11, SAA index = 364 probeToS = 0 policy_seq = 0
    OER BR APE detail: Completions 1, Sum of rtt 8, Max rtt 8, Min rtt 8
    OER BR APE detail: Completed retrieving Probe Statistics.

    ! Now the debug output for the destination 150.1.5.5 showing ! that Tunnel 2 with next hop of 20.0.0.3 has the least delay ....
    probeType = echo, probeTarget = 150.1.5.5, probeTargetPort = 0
    probeSource = 150.1.2.2, probeSourcePort = 0, probeNextHop = 10.0.0.3
    probeIfIndex = 10, SAA index = 365 probeToS = 0 policy_seq = 0
    OER BR APE detail: Completions 1, Sum of rtt 72, Max rtt 72, Min rtt 72
    OER BR APE detail: Completed retrieving Probe Statistics.
    probeType = echo, probeTarget = 150.1.5.5, probeTargetPort = 0
    probeSource = 150.1.2.2, probeSourcePort = 0, probeNextHop = 20.0.0.3
    probeIfIndex = 11, SAA index = 366 probeToS = 0 policy_seq = 0
    OER BR APE detail: Completions 1, Sum of rtt 12, Max rtt 12, Min rtt 12

    Back on the master controller (R1), we notice the messages:

    %OER_MC-5-NOTICE: Route changed Prefix 150.1.4.4/32, BR 136.1.12.2, i/f Tu2, Reason Delay, OOP Reason Timer Expired
    %OER_MC-5-NOTICE: Route changed Prefix 150.1.5.5/32, BR 136.1.12.2, i/f Tu2, Reason Delay, OOP Reason Timer Expired

    On R2 (the border), here is the updated routing table:

    R2#show ip route
    Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
    D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
    N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
    E1 - OSPF external type 1, E2 - OSPF external type 2
    i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
    ia - IS-IS inter area, * - candidate default, U - per-user static route
    o - ODR, P - periodic downloaded static route

    Gateway of last resort is 20.0.0.3 to network 0.0.0.0

    136.1.0.0/24 is subnetted, 2 subnets
    C 136.1.12.0 is directly connected, FastEthernet0/0
    C 136.1.23.0 is directly connected, Serial0/0

    20.0.0.0/24 is subnetted, 1 subnets
    C 20.0.0.0 is directly connected, Tunnel2

    10.0.0.0/24 is subnetted, 1 subnets
    C 10.0.0.0 is directly connected, Tunnel1

    150.1.0.0/16 is variably subnetted, 3 subnets, 2 masks
    C 150.1.2.0/24 is directly connected, Loopback0

    ! Note: the next hop has changed, and will use Tunnel 2 as the exit
    S 150.1.5.5/32 [1/0] via 20.0.0.3
    S 150.1.4.4/32 [1/0] via 20.0.0.3

    S* 0.0.0.0/0 [1/0] via 20.0.0.3
    [1/0] via 10.0.0.3
    R2#

    For detailed information regarding OER and the hundreds of options available with it, please leverage the content in our updated V5 RS content.

    OER is very fun, and very important. Enjoy your studies, and good luck!

     

    Hey! Don’t miss anything - subscribe to our newsletter!

    © 2022 INE. All Rights Reserved. All logos, trademarks and registered trademarks are the property of their respective owners.
    instagram Logofacebook Logotwitter Logolinkedin Logoyoutube Logo