Dec
31

Benjamin Franklin was quoted as saying “You may delay but time will not“.  We may also say that “Email may tolerate delay, but VOIP will not“.  Performance Routing (PfR), previously called Optimized Edge Routing (OER), is designed to solve network performance issues, such as delay, that vanilla IP routing can’t.  In this blog, we will use the term OER as that is still the command set used.

Petr and the team have created some awesome new labs on OER, including Volume 1, v5 in the IP Routing section. Our workbook content is extensive, has great explanations for everything and is highly recommended it to anyone preparing for the v4 CCIE RS lab.

One of the big challenges for the OER newcomer is knowing what to expect after configuring OER, and how to see measurable results. My intention in this blog post, is to assist the learner by providing a jump start into OER/PfR, by walking you through a solid configuration, and sharing a few critical show and debug commands to verify that OER is working. For you dynamips/GNS3 fans, the configuration may be replicated using (C3725-ADVENTERPRISEK9-M), Version 12.4(15)T10, RELEASE SOFTWARE (fc3).

So, how does it work?   It is actually very simple.  In the world of OER, there are 2 roles. The brains of the operation, called the master controller, and the border router who owns the exit interfaces. There may be multiple border routers. The key is there needs to be at least 2 or more exit points across the borders being managed by a single controller. Although the controller and border can actually run on the same router, I broke it up into separate devices for this blog post. R1 will be the OER Master Controller, and R2 will be the OER Border. Here is the topology.

Single Border, with 2 exit interfaces.

Single Border (R2), with 2 exit interfaces Tunnel 1 and Tunnel 2.

First we will create key chain on the controller R1.  The key is used for communications between the controller and the border.

R1(config)# key chain cisco
R1(config-keychain)#  key 1
R1(config-keychain-key)#  key-string cisco

Next, on R1 we will configure the master controller portion. About 98 % of all the configuration for OER is done on the controller. Here we will enable logging, and specify who the border(s) are, along with which interfaces on the border are exit interfaces to the outside world.

R1(config)#oer master
R1(config-oer-mc)#logging
R1(config-oer-mc)#border 136.1.12.2 key-chain cisco
R1(config-oer-mc-br)#interface Tunnel1 external
R1(config-oer-mc-br-if)#interface Tunnel2 external
R1(config-oer-mc-br-if)#interface FastEthernet0/0 internal
R1(config-oer-mc-br)#exit

Next on the controller, we  specify that we will learn prefixes based on throughput and delay, set time periods regarding learning, and tell the controller the network lengths to learn. In this example, due to the small network, we are going to have the controller learn /32 routes. We could also specify a maximum number of prefixes to learn as well. (Note: In a production network, you would not want thousands of static routes for sites visited by users. There is a default maximum, and the default aggregation is /24).

R1(config-oer-mc)#learn
R1(config-oer-mc-learn)#throughput
R1(config-oer-mc-learn)#delay
R1(config-oer-mc-learn)#periodic-interval 1
R1(config-oer-mc-learn)#monitor-period 2
R1(config-oer-mc-learn)#expire after time 30
R1(config-oer-mc-learn)#aggregation-type prefix-length 32
R1(config-oer-mc-learn)#exit

Next, now that we have exited out of “learn mode”, we can configure some of the other properties of the controller, such as how often to make prefix decisions (the backoff period), allow it to put routes in the routing table (mode route control), and choose the best exit interface (mode select-exit best). OER will create /32 static routes that use that use the “best” interface to reach the destination network.

R1(config-oer-mc)#backoff 180 360
R1(config-oer-mc)#mode route control
R1(config-oer-mc)#mode select-exit best

A few additional options include how often to make policy decisions, (periodic command), and which policy items are the most important regarding making those policy decisions (resolve commands).  Highest priority is 1, and lowest is 10.

R1(config-oer-mc)#periodic 90
R1(config-oer-mc)#resolve loss priority 1 variance 1
R1(config-oer-mc)#resolve delay priority 2 variance 1
R1(config-oer-mc)#resolve utilization priority 3 variance 1
R1(config-oer-mc)#resolve range priority 4

That is it for now on the controller. Let’s visit the border router R2 next. We’ll create the key chain, and tell R2 who the controller is. We will also tell R2 which interface to use for sending an active probe, (in the event the controller requests one).  Active probes are one method (from many available within OER) to determine delay.

R2(config)# key chain cisco
R2(config-keychain)#  key 1
R2(config-keychain-key)#  key-string cisco

R2(config)#oer border
R2(config-oer-br)#local FastEthernet0/0
R2(config-oer-br)#master 136.1.12.1 key-chain cisco
R2(config-oer-br)#active-probe address source interface Loopback0

So, how do we verify this? Since we are at R2, let’s start by issuing a show oer border on R2.

R2#show oer border
OER BR 136.1.12.2 ACTIVE, MC 136.1.12.1 UP/DOWN: UP 00:00:03,
Auth Failures: 0
Conn Status: SUCCESS, PORT: 3949
Version: 2.1  MC Version: 2.1
Exits
Fa0/0 INTERNAL Tu1 EXTERNAL Tu2 EXTERNAL
R2#

From this we see that we have successfully connected with the controller (R1), and that the controller has told R2 which interfaces are internal and which are external, based on what we configured on R1.

On R1, we can issue a few commands to confirm our configuration and connectivity to our border as well. Any elements shown in the output that we didn’t configure are based on the default settings.   Lets jump over to R1 and take a look.

R1#show oer master
OER state: ENABLED and ACTIVE
Conn Status: SUCCESS, PORT: 3949
Version: 2.1
Number of Border routers: 1 Number of Exits: 2
Number of monitored prefixes: 0 (max 5000)
Max prefixes: total 5000 learn 2500
Prefix count: total 0, learn 0, cfg 0

Border Status UP/DOWN AuthFail Version 136.1.12.2 ACTIVE UP 00:00:12 0 2.1
! Note:  Below, items in bold are derived from our configuration.
Global Settings:
max-range-utilization percent 0 recv 0
mode route metric bgp local-pref 5000
mode route metric static tag 5000
trace probe delay 1000
logging

Default Policy Settings:
backoff 180 360 180
delay relative 50
holddown 300
periodic 90
probe frequency 56
mode route control
mode monitor both
mode select-exit best
loss relative 10
jitter threshold 20
mos threshold 3.60 percent 30
unreachable relative 50
resolve loss priority 1 variance 1 resolve delay priority 2 variance 1 resolve utilization priority 3 variance 1 resolve range priority 4 variance 0

Learn Settings:
current state : RETRY
time remaining in current state : 26 seconds
throughput delay
no inside bgp
no protocol
monitor-period 2 periodic-interval 1
aggregation-type prefix-length 32
prefixes 200
expire after time 30
R1#

As a reference below, on R2, here are the IP addresses and Interfaces in use.   Note, that Serial 0/0 is used only as a transport for the GRE tunnels.

R2#show ip int brief | ex una
Interface                  IP-Address      OK? Method Status                Protocol
FastEthernet0/0            136.1.12.2      YES NVRAM  up                    up
Serial0/0                  136.1.23.2      YES NVRAM  up                    up
Loopback0                  150.1.2.2       YES NVRAM  up                    up
Tunnel1 10.0.0.2       YES NVRAM  up                    up
Tunnel2 20.0.0.2       YES NVRAM  up                    up
R2#

Here is the initial routing table on R2, (before OER has dynamically learned routes). We have only 2 static routes, which originated from the startup configuration.

R2#show ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route

Gateway of last resort is 20.0.0.3 to network 0.0.0.0

136.1.0.0/24 is subnetted, 2 subnets
C       136.1.12.0 is directly connected, FastEthernet0/0
C       136.1.23.0 is directly connected, Serial0/0

20.0.0.0/24 is subnetted, 1 subnets
C       20.0.0.0 is directly connected, Tunnel2

10.0.0.0/24 is subnetted, 1 subnets
C       10.0.0.0 is directly connected, Tunnel1

150.1.0.0/24 is subnetted, 1 subnets
C       150.1.2.0 is directly connected, Loopback0

S* 0.0.0.0/0 [1/0] via 20.0.0.3 [1/0] via 10.0.0.3
R2#

R2#show run | inc route
ip route 0.0.0.0 0.0.0.0 10.0.0.3 ip route 0.0.0.0 0.0.0.0 20.0.0.3

Now for the fun part. (Really!:) Lets generate some traffic in the background, and allow OER to dynamically learn the specific routes, as well as learn the best path to those routes and watch those routes be added to the routing table on R2. We’ll create two SLAs from R1 directed to the loopbacks of R4 and R5.

R1

R1(config)#ip sla 1
R1(config-ip-sla)#tcp-connect 150.1.4.4 4000
R1(config-ip-sla-tcp)#timeout 100
R1(config-ip-sla-tcp)#frequency 1
R1(config-ip-sla-tcp)#exit
R1(config)#ip sla schedule 1 life forever start-time now
R1(config)#
R1(config)#ip sla 2
R1(config-ip-sla)#tcp-connect 150.1.5.5 5000
R1(config-ip-sla-tcp)#timeout 100
R1(config-ip-sla-tcp)#frequency 1
R1(config-ip-sla-tcp)#exit
R1(config)#ip sla schedule 2 life forever start-time now

On R4 and R5, we need to set up the SLA responder, so they will respond correctly to the tcp-connect requests sourced from R1.

R4 and R5

R4(config)#ip sla responder
R5(config)#ip sla responder

On R2, lets see what networks have been learned. (Note: Based on the timers, it may take a ~minute or so for the /32 networks of 150.1.4.4 and 150.1.5.5 to be learned.   I waited at least a couple minutes before issuing the commands below.   You may want to verify that your SLA traffic is reaching R4 and R5 if you need to troubleshoot).

R2#show oer border passive cache prefix

OER Passive Prefix Cache, State: enabled, 278544 bytes
4 active, 4092 inactive, 1222 added
67316 ager polls, 0 flow alloc failures
Active flows timeout in 1 minutes
Inactive flows timeout in 15 seconds
IP Sub Flow Cache, 25800 bytes
12 active, 1012 inactive, 3666 added, 1222 added to flow
0 alloc failures, 0 force free
1 chunk, 30 chunks added
0 flows not aggregated due to main subcache starvation

Prefix              NextHop         Src If         Dst If          Flows
Pkts   B/Pk  Active   sDly   #Dly  PktLos  #UnRch
------------------------------------------------------------------------
150.1.4.4/32 0.0.0.0        Tu1            Fa0/0             24
34      38    18.5      0      0       0       0
150.1.5.5/32 0.0.0.0         Tu1            Fa0/0             24
34      38    18.1      0      0       0       0
150.1.4.4/32 20.0.0.3        Fa0/0          Tu2               23
35      62    18.5     64      5       0       0
150.1.5.5/32 20.0.0.3        Fa0/0          Tu2               23
35      62    18.1     84      5       0       0
R2#

Now, lets turn on a debug of active probes on the border, and see what we can learn from it.

R2#debug oer border active-probes detail
<snip>
OER BR APE detail: Completed retrieving Probe Statistics.
probeType = echo, probeTarget = 150.1.4.4, probeTargetPort = 0
probeSource = 150.1.2.2, probeSourcePort = 0, probeNextHop = 10.0.0.3
probeIfIndex = 10, SAA index = 339 probeToS = 0 policy_seq = 0
OER BR APE detail: Completions 2, Sum of rtt 40, Max rtt 24, Min rtt 16
<snip>
OER BR APE detail: Completed retrieving Probe Statistics.
probeType = echo, probeTarget = 150.1.4.4, probeTargetPort = 0
probeSource = 150.1.2.2, probeSourcePort = 0, probeNextHop = 20.0.0.3
probeIfIndex = 11, SAA index = 340 probeToS = 0 policy_seq = 0
OER BR APE detail: Completions 2, Sum of rtt 52, Max rtt 28, Min rtt 24

Based on this, the rtt values from Tunnel 1 (using 10.0.0.3 as the next hop to reach 150.1.4.4) are lower than Tunnel 2 (which is using 20.0.0.3 as the next hop). As a result, OER believes that Tunnel 1 is the best path and will inject a static /32 route for destination 150.1.4.4, using Tunnel 1 as the exit interface and 10.0.0.3 as the next hop. A similar result was discovered for 150.1.5.5 as well. As a result, both /32 routes are dynamically added to the routing table on R2.

R2#show ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route

Gateway of last resort is 20.0.0.3 to network 0.0.0.0

136.1.0.0/24 is subnetted, 2 subnets
C       136.1.12.0 is directly connected, FastEthernet0/0
C       136.1.23.0 is directly connected, Serial0/0
20.0.0.0/24 is subnetted, 1 subnets
C       20.0.0.0 is directly connected, Tunnel2
10.0.0.0/24 is subnetted, 1 subnets
C       10.0.0.0 is directly connected, Tunnel1
150.1.0.0/16 is variably subnetted, 3 subnets, 2 masks
C       150.1.2.0/24 is directly connected, Loopback0
! OER dynamically added the 2 static routes below
S 150.1.5.5/32 [1/0] via 10.0.0.3 S 150.1.4.4/32 [1/0] via 10.0.0.3
S*   0.0.0.0/0 [1/0] via 20.0.0.3
               [1/0] via 10.0.0.3
R2#

Also, on R1, the controller, there were a couple messages logged:

%OER_MC-5-NOTICE: Route changed Prefix 150.1.5.5/32, BR 136.1.12.2, i/f Tu1, Reason Delay, OOP Reason Timer Expired
%OER_MC-5-NOTICE: Route changed Prefix 150.1.4.4/32, BR 136.1.12.2, i/f Tu1, Reason Delay, OOP Reason Timer Expired

So now that OER has decided that Tunnel 1, using 10.0.0.3 is the best path for those 2 networks, lets tip the scale in the other direction by using generic  traffic shaping on Tunnel 1, so that the delay there will be worse than Tunnel 2, and watch the results. (Note: in addition to the generic traffic shaping on R2 Tunnel 1, I also ran an extended ping sourced from 150.1.4.4 and 150.1.5.5 to increase the overall delay).

R2(config-if)#traffic-shape rate 8000 1000 0
R2(config-if)#do show traffic-shape

Interface   Tu1
Access Target    Byte   Sustain   Excess    Interval  Increment Adapt
VC     List   Rate      Limit  bits/int  bits/int  (ms)      (bytes)   Active
-             8000      125    1000      0         125       125       -

R2#
! Below is debug output showing Tunnel 2 with next hop of 20.0.0.3 ! as having the least delay for destination 150.1.4.4 ...

OER BR APE detail: Completed retrieving Probe Statistics.
probeType = echo, probeTarget = 150.1.4.4, probeTargetPort = 0
probeSource = 150.1.2.2, probeSourcePort = 0, probeNextHop = 10.0.0.3
probeIfIndex = 10, SAA index = 363 probeToS = 0 policy_seq = 0
OER BR APE detail: Completions 1, Sum of rtt 88, Max rtt 88, Min rtt 88
OER BR APE detail: Completed retrieving Probe Statistics.
probeType = echo, probeTarget = 150.1.4.4, probeTargetPort = 0
probeSource = 150.1.2.2, probeSourcePort = 0, probeNextHop = 20.0.0.3
probeIfIndex = 11, SAA index = 364 probeToS = 0 policy_seq = 0
OER BR APE detail: Completions 1, Sum of rtt 8, Max rtt 8, Min rtt 8
OER BR APE detail: Completed retrieving Probe Statistics.

! Now the debug output for the destination 150.1.5.5 showing ! that Tunnel 2 with next hop of 20.0.0.3 has the least delay .... 
probeType = echo, probeTarget = 150.1.5.5, probeTargetPort = 0
probeSource = 150.1.2.2, probeSourcePort = 0, probeNextHop = 10.0.0.3
probeIfIndex = 10, SAA index = 365 probeToS = 0 policy_seq = 0
OER BR APE detail: Completions 1, Sum of rtt 72, Max rtt 72, Min rtt 72
OER BR APE detail: Completed retrieving Probe Statistics.
probeType = echo, probeTarget = 150.1.5.5, probeTargetPort = 0
probeSource = 150.1.2.2, probeSourcePort = 0, probeNextHop = 20.0.0.3
probeIfIndex = 11, SAA index = 366 probeToS = 0 policy_seq = 0
OER BR APE detail: Completions 1, Sum of rtt 12, Max rtt 12, Min rtt 12

Back on the master controller (R1), we notice the messages:

%OER_MC-5-NOTICE: Route changed Prefix 150.1.4.4/32, BR 136.1.12.2, i/f Tu2, Reason Delay, OOP Reason Timer Expired
%OER_MC-5-NOTICE: Route changed Prefix 150.1.5.5/32, BR 136.1.12.2, i/f Tu2, Reason Delay, OOP Reason Timer Expired

On R2 (the border), here is the updated routing table:

R2#show ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route

Gateway of last resort is 20.0.0.3 to network 0.0.0.0

136.1.0.0/24 is subnetted, 2 subnets
C       136.1.12.0 is directly connected, FastEthernet0/0
C       136.1.23.0 is directly connected, Serial0/0

20.0.0.0/24 is subnetted, 1 subnets
C       20.0.0.0 is directly connected, Tunnel2

10.0.0.0/24 is subnetted, 1 subnets
C       10.0.0.0 is directly connected, Tunnel1

150.1.0.0/16 is variably subnetted, 3 subnets, 2 masks
C       150.1.2.0/24 is directly connected, Loopback0

! Note: the next hop has changed, and will use Tunnel 2 as the exit
S       150.1.5.5/32 [1/0] via 20.0.0.3
S       150.1.4.4/32 [1/0] via 20.0.0.3

S*   0.0.0.0/0 [1/0] via 20.0.0.3
               [1/0] via 10.0.0.3
R2#

For detailed information regarding OER and the hundreds of options available with it, please leverage the content in our updated V5 RS content.

OER is very fun, and very important. Enjoy your studies, and good luck!

 


You can leave a response, or trackback from your own site.

15 Responses to “OER/PfR… it’s always watching.”

 
  1. Seifeddine says:

    Awesomeee thanks Guys :)
    u r so great i was actually waiting for something related to PfR :)

  2. Gary says:

    Very informative. Thx. One question for you. Why did you set the active-probe interfaces if you are doing PASSIVE learning via netflow and not ACTIVE via ip sla?

  3. Bruno says:

    Very nice,

    I think the “ip sla” feature is there only to generate traffic. Do you confirm this?

    Bruno

  4. Bruno- Yes. The 2 IP SLAs that were set up, were to only generate traffic.

  5. Hello Gary- great question!

    From the OER learning perspective, prefixes are learned on the border routers through monitoring the traffic flow using the embedded NetFlow capability.

    From the monitoring perspective, for each aggregated prefix, (in our case, /32 prefixes) up to five host addresses are selected to use as active probe targets. (In our case, it is the 32 bit host address that represents the /32 prefix that is being used for the active probe.)

    In our example, I used a couple SLAs to generate some traffic (separate from OER) so the learning would begin. NetFlow and IP SLAs are enabled automatically within OER. We can use both active and passive monitoring methods at the same time.

    The master controller uses both passive and active monitoring by default (mode monitor both). All traffic classes are passively monitored using integrated NetFlow functionality. Out-of-policy traffic classes are actively monitored using IP SLA functionality. Out of policy (OOP) can be due to the timer expiring, loss, delay, utilization etc.

    Thanks for the question.

  6. Chris says:

    Hi,

    great article, finding good information on how to configure OER seems to be pretty rare!

    One question though, whenever I have configured OER in my lab, I have always had to manually enable netflow on the border routers in order for it to identify prefixes.

    How come you have not done this in this example? is it just ingress netflow on the external intefaces you need and do you also need to enable top talkers on the border routers?

  7. Gary says:

    Hey,

    I guess I am still a bit confused. Lets pretend that you never specified a source interface to send active probes from on the BR’s, would we only be doing PASSIVE monitoring on the prefixes that you have learned via NETFLOW?

    I thought that to do ACTIVE you actually have to do: “active-probe tcp-conn 150.1.1.1 target-port 23″ on the MC.

    If I am understanding you correctly, then your saying that since “monitor mode both” is on, we are indeed doing ACTIVE and PASSIVE even though no active-probes have been configured?

    Thanks for the clarification.

  8. Maher / CCIE No. 21032 says:

    Hi, Really great article. Thank you.

  9. Gary- Which interface will your router use to source PING, TACACS sessions, Telnet, SSH, etc?

    The answer is, without external forces, the closest interface to the destination. By removing the command “active-probe address source interface Loopback0″ from the border router won’t stop the active probes, it will simply source the traffic from the 2 tunnel interfaces (1 per probe) instead of the loopback 0.

    So, yes, we are indeed doing active and passive even though no active probes have been configured.

  10. Chris- The OER process internally uses netflow, without requiring us to administratively turn netflow on.

    I looked at my config on R2, and I did have top-talkers enabled, but it was for another project I was working in. I turned off top-talkers, shut the border and master, restarted both, and everything still worked the same.

    The netflow feature used by OER is “automagic”.

  11. Roy Waterman says:

    Hi

    Really good article, v well written, thanks!

  12. martin says:

    Hi,

    i have problem about implementing the OER. i can’t inject the static route into routing table. FYI, i use EIGRP routing protocol.

    When i type sh oer master traffic-class, the status is uncontrolled (DEFAULT*)

    what should i do to solve this problem.

    sh oer master traffic-class
    OER Prefix Statistics:
    Pas – Passive, Act – Active, S – Short term, L – Long term, Dly – Delay (ms),
    P – Percentage below threshold, Jit – Jitter (ms),
    MOS – Mean Opinion Score
    Los – Packet Loss (packets-per-million), Un – Unreachable (flows-per-million),
    E – Egress, I – Ingress, Bw – Bandwidth (kbps), N – Not applicable
    U – unknown, * – uncontrolled, + – control more specific, @ – active probe all
    # – Prefix monitor mode is Special, & – Blackholed Prefix
    % – Force Next-Hop, ^ – Prefix is denied

    DstPrefix Appl_ID Dscp Prot SrcPort DstPort SrcPrefix
    Flags State Time CurrBR CurrI/F Protocol
    PasSDly PasLDly PasSUn PasLUn PasSLos PasLLos EBw IBw
    ActSDly ActLDly ActSUn ActLUn ActSJit ActPMOS ActSLos ActLLos
    ——————————————————————————–
    10.14.199.56/30 N defa N N N N
    DEFAULT* 53 10.14.254.209 Fa0/0 U
    U U 0 1000000 0 0 0 0
    U U 0 0 N N N N

    10.14.167.92/30 N defa N N N N
    DEFAULT* 10 10.14.254.209 Fa0/0 U
    U U 0 1000000 0 0 0 0
    U U 0 0 N N N N

    10.14.207.0/30 N defa N N N N
    DEFAULT* 54 10.14.254.209 Fa0/0 U
    U U 1000000 1000000 0 0 1 0
    U U 0 0 N N N N

    10.14.119.100/30 N defa N N N N
    DEFAULT* 30 10.14.254.209 Fa0/0 U
    U U 0 1000000 0 0 0 0
    U U 0 0 N N N N

    10.14.246.244/30 N defa N N N N
    DEFAULT* 63 10.14.254.209 Fa0/0 U
    U U 1000000 1000000 0 0 1 0
    U U 0 0 N N N N

    10.14.63.216/30 N defa N N N N
    DEFAULT* 25 10.14.254.209 Fa0/0 U
    U U 0 1000000 0 0 0 0
    U U 0 0 N N N N

    10.14.95.132/30 N defa N N N N
    DEFAULT* 26 10.14.254.209 Fa0/0 U
    U U 1000000 1000000 0 0 1 0
    –More–   U U 0 0 N N N N

    10.14.39.196/30 N defa N N N N
    DEFAULT* 59 10.14.254.209 Fa0/0 U
    U U 0 1000000 0 0 0 0
    U U 0 0 N N N N

    10.14.199.48/30 N defa N N N N
    DEFAULT* 25 10.14.254.209 Fa0/0 U
    U U 1000000 1000000 0 0 1 0
    U U 0 0 N N N N

    10.14.207.56/30 N defa N N N N
    DEFAULT* 21 10.14.254.209 Fa0/0 U
    U U 0 1000000 0 0 0 0
    U U 0 0 N N N N

    10.14.143.112/30 N defa N N N N
    DEFAULT* 2 10.14.254.209 Fa0/0 U
    U U 0 1000000 0 0 0 0
    U U 0 0 N N N N

    10.14.55.212/30 N defa N N N N
    DEFAULT* 85 10.14.254.209 Fa0/0 U
    U U 0 1000000 0 0 0 0
    U U 0 0 N N N N

    10.14.95.184/30 N defa N N N N
    DEFAULT* 70 10.14.254.209 Fa0/0 U
    U U 0 1000000 0 0 0 0
    U U 0 0 N N N N

    10.14.135.108/30 N defa N N N N
    DEFAULT* 21 10.14.254.209 Fa0/0 U
    U U 1000000 1000000 0 0 1 0
    U U 0 0 N N N N

    10.14.223.12/30 N defa N N N N
    DEFAULT* 7 10.14.254.209 Fa0/0 U
    U U 1000000 1000000 0 0 1 0
    U U 0 0 N N N N

    10.14.31.244/30 N defa N N N N
    DEFAULT* 89 10.14.254.209 Fa0/0 U
    U U 0 1000000 0 0 0 0
    U U 0 0 N N N N

    i want to ask about injecting the static route into routing t

  13. crownade says:

    Pls I could do with some help and apologies if it’s a stupid question. I’ve loaded the initial configs for OER from ATC Vol1 and haven’t been able to make it past this first task. After configuring the OER Master (R5) and Borders (R2,R3 & R5), my R3 Border router keeps flapping and I keep getting the following message on the Master:

    Rack1R5#
    *Mar 12 15:15:44.857: %OER_MC-5-NOTICE: BR 150.1.3.3 UP
    *Mar 12 15:15:45.273: %OER_MC-5-NOTICE: BR 150.1.3.3 DOWN
    *Mar 12 15:15:45.273: %OER_MC-5-NOTICE: Uncontrol prefixes, Clear exit, BR 150.1.3.3 i/f
    *Mar 12 15:15:45.273: %OER_MC-5-NOTICE: Uncontrol prefixes, Clear exit, BR 150.1.3.3 i/f
    Rack1R5#
    *Mar 12 15:15:49.981: %OER_MC-5-NOTICE: BR 150.1.3.3 UP
    *Mar 12 15:15:50.397: %OER_MC-5-NOTICE: BR 150.1.3.3 DOWN
    *Mar 12 15:15:50.397: %OER_MC-5-NOTICE: Uncontrol prefixes, Clear exit, BR 150.1.3.3 i/f
    *Mar 12 15:15:50.397: %OER_MC-5-NOTICE: Uncontrol prefixes, Clear exit, BR 150.1.3.3 i/f
    Rack1R5#
    *Mar 12 15:15:55.097: %OER_MC-5-NOTICE: BR 150.1.3.3 UP
    *Mar 12 15:15:55.513: %OER_MC-5-NOTICE: BR 150.1.3.3 DOWN
    *Mar 12 15:15:55.513: %OER_MC-5-NOTICE: Uncontrol prefixes, Clear exit, BR 150.1.3.3 i/f
    *Mar 12 15:15:55.513: %OER_MC-5-NOTICE: Uncontrol prefixes, Clear exit, BR 150.1.3.3 i/f
    Rack1R5#
    *Mar 12 15:16:00.221: %OER_MC-5-NOTICE: BR 150.1.3.3 UP
    *Mar 12 15:16:00.633: %OER_MC-5-NOTICE: BR 150.1.3.3 DOWN
    *Mar 12 15:16:00.637: %OER_MC-5-NOTICE: Uncontrol prefixes, Clear exit, BR 150.1.3.3 i/f
    *Mar 12 15:16:00.637: %OER_MC-5-NOTICE: Uncontrol prefixes, Clear exit, BR 150.1.3.3 i/f
    Rack1R5#
    *Mar 12 15:16:05.337: %OER_MC-5-NOTICE: BR 150.1.3.3 UP
    *Mar 12 15:16:05.753: %OER_MC-5-NOTICE: BR 150.1.3.3 DOWN
    *Mar 12 15:16:05.753: %OER_MC-5-NOTICE: Uncontrol prefixes, Clear exit, BR 150.1.3.3 i/f
    *Mar 12 15:16:05.757: %OER_MC-5-NOTICE: Uncontrol prefixes, Clear exit, BR 150.1.3.3 i/f
    Rack1R5#

    I’ve also read this wonderful blog, but unfortunately, I still haven’t been able to figure out what the issue is.

    Pls help!!

  14. Abhishek says:

    Hi,
    Thanks for this article, I am a little rusty on Loopbacks, from this tutorial its not clear which all routers need a loopback address and how to configure them. Would you have a note somewhere describing how the initial setup was made ?
    Thanks,
    ABhishek
    abhishekm@aryaka.com

  15. Juan says:

    Hello,
    Thank you very much for the good article about OER. I configured your sample using GNS3 version 0.8.3.1 running on CentOS 64bits and I am unable to get your results. I am using the same IOS you are using adventerprisek9-mz.12.4-15.T10

    Bottom line is the output of “debug oer border active-probes detail” displays rtt’s=0 as shown below (If I send pings from R1 to R4 and R5 I do get the normal ping output showing rtt’s not zero):

    R2# Probe Statistics.
    probeType = echo, probeTarget = 150.1.5.5, probeTargetPort = 0
    probeSource = 150.1.2.2, probeSourcePort = 0, probeNextHop = 10.0.0.3
    probeIfIndex = 7, SAA index = 10 probeToS = 0 policy_seq = 0
    *Mar 1 00:20:07.279: OER BR APE detail: Completions 0, Sum of rtt 0, Max rtt 0, Min rtt 0
    *Mar 1 00:20:07.287: OER BR APE detail: Attempting to retrieve Probe Statistics.
    probeType = echo, probeTarget = 150.1.4.4, probeTargetPort = 0
    probeSource = Default, probeSourcePort = 0, probeNextHop = Default
    probeIfIndex = 8 probeToS = 0 policy_seq = 0
    *Mar 1 00:20:07.299: OER BR APE detail: Completed retrieving Probe Statistics.
    probeType = echo, probeTarget = 150.1.4.4, probeTargetPort = 0
    probeSource = 150.1.2.2, probeSourcePort = 0, probeNextHop = 20.0.0.3
    probeIfIndex = 8, SAA index = 11 probeToS = 0 policy_seq = 0
    *Mar 1 00:20:07.315: OER BR APE detail: Completions 0, Sum of rtt 0, Max rtt 0, Min rtt 0

    And from the beginning the output of “sho ip route” does not display the injected routes neither before or after configuring traffic-shape.

    And R1 displays:
    *Mar 1 00:21:40.667: %OER_MC-5-NOTICE: Uncontrol Prefix 150.1.5.5/32, Couldn’t find the best exit
    *Mar 1 00:21:40.671: %OER_MC-5-NOTICE: Uncontrol Prefix 150.1.5.5/32, Couldn’t choose exit in prefix timeout
    *Mar 1 00:21:40.679: %OER_MC-5-NOTICE: Uncontrol Prefix 150.1.4.4/32, Couldn’t find the best exit
    *Mar 1 00:21:40.679: %OER_MC-5-NOTICE: Uncontrol Prefix 150.1.4.4/32, Couldn’t choose exit in prefix timeout

    I am missing something. I would really appreciate if you can go over the following configuration files and point out where is the problem.

    Kind regards.

    R1#sho run
    Building configuration…

    Current configuration : 1802 bytes
    !
    version 12.4
    service timestamps debug datetime msec
    service timestamps log datetime msec
    no service password-encryption
    !
    hostname R1
    !
    boot-start-marker
    boot-end-marker
    !
    !
    no aaa new-model
    memory-size iomem 5
    ip cef
    !
    no ip domain lookup
    ip domain name lab.local
    !
    multilink bundle-name authenticated
    !
    !
    key chain cisco
    key 1
    key-string cisco
    !
    !
    oer master
    logging
    !
    border 136.1.12.2 key-chain cisco
    interface FastEthernet0/0 internal
    interface Tunnel2 external
    interface Tunnel1 external
    !
    learn
    throughput
    delay
    periodic-interval 1
    monitor-period 2
    expire after time 30
    aggregation-type prefix-length 32
    no max range receive
    backoff 180 360
    mode route control
    mode monitor active
    mode select-exit best
    periodic 90
    resolve loss priority 1 variance 1
    resolve delay priority 2 variance 1
    resolve utilization priority 3 variance 1
    resolve range priority 4
    !
    archive
    log config
    hidekeys
    !
    !
    interface Loopback0
    ip address 150.1.2.1 255.255.255.0
    !
    interface FastEthernet0/0
    ip address 136.1.12.1 255.255.255.0
    speed 100
    full-duplex
    !
    interface FastEthernet0/1
    ip address 192.168.32.104 255.255.255.0
    speed 100
    full-duplex
    !
    ip forward-protocol nd
    ip route 0.0.0.0 0.0.0.0 136.1.12.2
    !
    !
    no ip http server
    no ip http secure-server
    !
    ip sla 1
    icmp-echo 150.1.4.4 source-interface FastEthernet0/0
    timeout 2000
    ip sla schedule 1 life forever start-time now
    ip sla 2
    icmp-echo 150.1.5.5 source-ip 136.1.12.1
    timeout 2000
    frequency 10
    ip sla schedule 2 life forever start-time now
    !
    !
    control-plane
    !
    line con 0
    exec-timeout 0 0
    privilege level 15
    logging synchronous
    line aux 0
    exec-timeout 0 0
    privilege level 15
    logging synchronous
    line vty 0 4
    login
    !
    !
    end

    R1#
    ++++++++++++++++++++++++++++++++++++++++++++++++++++

    R2#sho run
    Building configuration…

    Current configuration : 1547 bytes
    !
    version 12.4
    service timestamps debug datetime msec
    service timestamps log datetime msec
    no service password-encryption
    !
    hostname R2
    !
    boot-start-marker
    boot-end-marker
    !
    !
    no aaa new-model
    memory-size iomem 5
    ip cef
    !
    !
    no ip domain lookup
    ip domain name lab.local
    !
    multilink bundle-name authenticated
    !
    !
    key chain cisco
    key 1
    key-string cisco
    !
    oer border
    local FastEthernet0/0
    master 136.1.12.1 key-chain cisco
    active-probe address source interface Loopback0
    !
    !
    archive
    log config
    hidekeys
    !
    interface Loopback0
    ip address 150.1.2.2 255.255.255.0
    !
    interface Tunnel1
    ip address 10.0.0.2 255.255.255.0
    traffic-shape rate 8000 1000 0 1000
    tunnel source 136.1.13.1
    tunnel destination 136.1.13.2
    !
    interface Tunnel2
    ip address 20.0.0.2 255.255.255.0
    tunnel source 136.1.13.1
    tunnel destination 136.1.13.2
    !
    interface FastEthernet0/0
    ip address 136.1.12.2 255.255.255.0
    speed 100
    full-duplex
    !
    interface FastEthernet0/1
    no ip address
    shutdown
    duplex auto
    speed auto
    !
    interface FastEthernet1/0
    ip address 136.1.13.1 255.255.255.0
    speed 100
    full-duplex
    !
    ip forward-protocol nd
    ip route 0.0.0.0 0.0.0.0 10.0.0.3
    ip route 0.0.0.0 0.0.0.0 20.0.0.3
    !
    !
    no ip http server
    no ip http secure-server
    !
    !
    control-plane
    !
    !
    line con 0
    exec-timeout 0 0
    privilege level 15
    logging synchronous
    line aux 0
    exec-timeout 0 0
    privilege level 15
    logging synchronous
    line vty 0 4
    login
    !
    !
    end

    R2#
    ++++++++++++++++++++++++++++++++++++++++++++++++++++

    R3#sho run
    Building configuration…

    Current configuration : 1405 bytes
    !
    version 12.4
    service timestamps debug datetime msec
    service timestamps log datetime msec
    no service password-encryption
    !
    hostname R3
    !
    boot-start-marker
    boot-end-marker
    !
    !
    no aaa new-model
    memory-size iomem 5
    ip cef
    !
    !
    no ip domain lookup
    ip domain name lab.local
    !
    multilink bundle-name authenticated
    !
    !
    archive
    log config
    hidekeys
    !
    !
    interface Tunnel1
    ip address 10.0.0.3 255.255.255.0
    tunnel source 136.1.13.2
    tunnel destination 136.1.13.1
    !
    interface Tunnel2
    ip address 20.0.0.3 255.255.255.0
    tunnel source 136.1.13.2
    tunnel destination 136.1.13.1
    !
    interface FastEthernet0/0
    ip address 136.1.34.1 255.255.255.0
    speed 100
    full-duplex
    !
    interface FastEthernet0/1
    no ip address
    shutdown
    duplex auto
    speed auto
    !
    interface FastEthernet1/0
    ip address 136.1.13.2 255.255.255.0
    speed 100
    full-duplex
    !
    router rip
    network 136.1.0.0
    no auto-summary
    !
    ip forward-protocol nd
    ip route 136.1.12.0 255.255.255.0 136.1.13.1
    ip route 150.1.4.4 255.255.255.255 136.1.34.2
    ip route 150.1.5.5 255.255.255.255 136.1.34.2
    ip route 192.168.32.170 255.255.255.255 136.1.13.1
    !
    !
    no ip http server
    no ip http secure-server
    !
    !
    control-plane
    !
    !
    line con 0
    exec-timeout 0 0
    privilege level 15
    logging synchronous
    line aux 0
    exec-timeout 0 0
    privilege level 15
    logging synchronous
    line vty 0 4
    login
    !
    !
    end

    R3#

    +++++++++++++++++++++++++++++++++++++++++++++++++++

    R4#sho run
    Building configuration…

    Current configuration : 1168 bytes
    !
    version 12.4
    service timestamps debug datetime msec
    service timestamps log datetime msec
    no service password-encryption
    !
    hostname R4
    !
    boot-start-marker
    boot-end-marker
    !
    !
    no aaa new-model
    memory-size iomem 5
    ip cef
    !
    !
    no ip domain lookup
    ip domain name lab.local
    !
    multilink bundle-name authenticated
    !
    !
    archive
    log config
    hidekeys
    !
    !
    interface Loopback0
    ip address 150.1.4.4 255.255.255.0
    !
    interface FastEthernet0/0
    ip address 136.1.34.2 255.255.255.0
    speed 100
    full-duplex
    !
    interface FastEthernet0/1
    ip address 136.1.45.1 255.255.255.0
    speed 100
    full-duplex
    !
    router rip
    network 136.1.0.0
    no auto-summary
    !
    ip forward-protocol nd
    ip route 0.0.0.0 0.0.0.0 136.1.34.1
    ip route 136.1.45.0 255.255.255.0 136.1.45.2
    ip route 150.1.5.5 255.255.255.255 136.1.45.2
    i
    !
    no ip http server
    no ip http secure-server
    !
    ip sla responder
    !
    !
    control-plane
    !
    !
    line con 0
    exec-timeout 0 0
    privilege level 15
    logging synchronous
    line aux 0
    exec-timeout 0 0
    privilege level 15
    logging synchronous
    line vty 0 4
    login
    !
    !
    end

    R4#

    ++++++++++++++++++++++++++++++++++++++++++++++++++

    R5#sho run
    Building configuration…

    Current configuration : 1189 bytes
    !
    version 12.4
    service timestamps debug datetime msec
    service timestamps log datetime msec
    no service password-encryption
    !
    hostname R5
    !
    boot-start-marker
    boot-end-marker
    !
    !
    no aaa new-model
    memory-size iomem 5
    ip cef
    !
    !
    no ip domain lookup
    ip domain name lab.local
    !
    multilink bundle-name authenticated
    !
    !
    archive
    log config
    hidekeys
    !
    !
    interface Loopback0
    ip address 150.1.5.5 255.255.255.0
    !
    interface FastEthernet0/0
    no ip address
    shutdown
    duplex auto
    speed auto
    !
    interface FastEthernet0/1
    ip address 136.1.45.2 255.255.255.0
    speed 100
    full-duplex
    !
    router rip
    network 136.1.0.0
    no auto-summary
    !
    ip forward-protocol nd
    ip route 0.0.0.0 0.0.0.0 136.1.45.1
    !
    !
    no ip http server
    no ip http secure-server
    !
    ip sla responder
    ip sla responder udp-echo ipaddress 136.1.12.1 port 9
    ip sla responder tcp-connect ipaddress 136.1.12.1 port 5000
    ip sla responder tcp-connect ipaddress 136.1.12.1 port 5050
    !
    !
    control-plane
    !
    !
    line con 0
    exec-timeout 0 0
    privilege level 15
    logging synchronous
    line aux 0
    exec-timeout 0 0
    privilege level 15
    logging synchronous
    line vty 0 4
    login
    !
    !
    end

    R5#

    ++++++++++++++++++++++++++++++++++++++++++++++++

    All routers:
    Cisco IOS Software, 3700 Software (C3725-ADVENTERPRISEK9-M), Version 12.4(15)T10, RELEASE SOFTWARE (fc3)
    Technical Support: http://www.cisco.com/techsupport
    Copyright (c) 1986-2009 by Cisco Systems, Inc.
    Compiled Mon 14-Sep-09 15:53 by prod_rel_team

    ROM: ROMMON Emulation Microcode
    ROM: 3700 Software (C3725-ADVENTERPRISEK9-M), Version 12.4(15)T10, RELEASE SOFTWARE (fc3)

 

Leave a Reply

Categories

CCIE Bloggers