Oct
03

Our bootcamps are a great study resource for CCIE candidates. No matter whether you're just starting out on your CCIE training journey, or have been studying for months, an INE bootcamp can help you gauge where you're at in the study process and what you need to focus on before attempting your CCIE Exams.

What is a Bootcamp?

Bootcamps are intensive, live classes that typically last from 5-7 days. Bootcamps allow you to dive further into your study path in a small classroom environment with an in-person, expert INE instructor leading the way. Each bootcamp class will cover a specific list of topics tailored to the Cisco track and certification level you are studying. Our instructors will customize the training to focus on certain topics and technologies that best meet the individual requests of the students in your bootcamp.

What's a Bootcamp Class Actually Like?

In our Written Routing and Switching Bootcamp you will learn about Switching, RIP & EIGRP, OSPF, BGP, MPLS, Redistribution and Evolving Technologies. Our 7 time CCIE, expert-instructor, Rohit Pardasani, will not only help you understand all of the essential R&S topics for this exam, he will also share study techniques and prepare you for what the actual exam environment will be like.

Bootcamp participants will find that our in person classes are laid back, private and comfortable study environments, where questions are encouraged and each student is afforded the attention they deserve. Our small class sizes (less than 20 students per bootcamp) allow our instructors to customize curriculum and focus on the specific needs and requirements of the individuals in your class.

Typically, our R&S Bootcamps run 10 hours a day, with a 1 hour lunch break, and ample snack and coffee breaks to fuel your studying. Each day will be a combination of lectures, instructor-led demonstrations, Q&A sessions, hands-on exercises, and frequent quizzes. Students will also have full access to lab equipment throughout the bootcamp and in the evenings to continue practicing on their own or as groups.

Starting in 2018, our bootcamps will always take place in one of three locations; Durham, North Carolina, Salt Lake City, Utah, or Amsterdam, Netherlands. For those that can't, or don't want to travel, we also offer online live options.

What Our Students Have to Say About the Routing and Switching Bootcamp Experience:

Don't believe INE has the best in-person training? Take it from our students, our bootcamps are the best on the market!

"I would like to thank INE and Rohit Pardasani in particular for the great experience I had. This course has provided me with a good idea on how the CCIE exam should be approached in terms of study and preparation. I will highly recommend this training to anybody that is on the same journey as I am. Also, many thanks to the Coordinator in providing all the information needed and answering my questions, this made everything as smooth as I hoped."
Berry Batist

"I enjoyed Rohit personally. I would look for him if seeking another workshop. The content/material informed and challenged me. I feel better prepared."
John Wattenbarger

Ready to Join Thousands of Students Who Passed Their CCIE Written Exam After Participating in an INE Bootcamp?

Check out our current Routing and Switching Written Exam Bootcamp Dates:

November 13 - November 17, 2018 ~ Durham, NC or Online Live

January 7 - January 11, 2019 ~ Online Live

April 15 - April 19, 2019 ~ Durham, NC or Online Live

June 24 - June 28, 2019 ~ Amsterdam, NL or Online Live

You can find information about all of the bootcamps we offer on our bootcamps website, or by contacting our bootcamps training advisor.

Aug
31

Cisco just rolled out the Evolving Technologies v1.1 update, which will affect anyone taking their CCIE certification exams on, or after, August 30, 2018. Fortunately, the v1.1 updates are fairly minor. The CCIE/CCDE Evolving Technologies section still includes three overall categories; Cloud, Network Programmability and Internet of things, and still makes up 10% of all CCIE/CCDE written exams. However, changes can be found in the specific topics tested in each of the evolving technologies categories.

Cloud - v1.1

Compare and contrast public, private, hybrid, and multi-cloud design considerations

  • Infrastructure, platform, and software as a service (XaaS)
  • Performance, scalability, and high availability
  • Security implications, compliance, and policy
  • Workload migration

Describe cloud infrastructure and operations

  • Compute virtualization (containers and virtual machines)
  • Connectivity (virtual switches, SD-WAN and SD-Access)
  • Virtualization functions (NFVi, VNF, and L4/L1)
  • Automation and orchestration tools (cloud center, DNA-center, and Kubernetes)

Network Programmability (SDN) - v1.1

Describe architectural and operational Considerations for a programmable network

  • Data models and structures (YANG, JSON and XML)
  • Device programmability (gRPC, NETCONF and RESTCONF)
  • Controller based network design (policy driven configuration and northbound/ southbound APIs)
  • Configuration management tools (agent and agent-less) and version control systems (Git and SVN)

Internet of Things (IOT) - v1.1

Describe architectural framework and deployment considerations for Internet of Things (IoT)

  • IoT technology stack (IoT Network Hierarchy, data acquisition and flow)
  • IoT standards and protocols (characteristics within IT and OT environment)
  • IoT security (network segmentation, device profiling, and secure remote access)
  • IoT edge and fog computing (data aggregation and edge intelligence)

To Learn more about the changes between v1.0 and v1.1 of Cisco's Evolving Technologies, read the full article on Cisco's Website.



Aug
15


Since 2003 we've been helping IT professionals reach their career goals with help from top notch instructors and training materials. One of our most popular training resources - INE Bootcamps, continue to wow students and are a major step in the journey towards earning your certification. Thinking about signing up but aren't sure what to expect? Take it from our current students, participating in an INE Bootcamp is the best way to ensure you'll succeed in passing your certification exams.



CCNA Routing & Switching

I would arguably say that Keith is the best CCNA instructor in the nation. The interaction in this class is key. Listening to a lesson doesn't ensure comprehension, so Keith offered periodic quizzes; not only did this make the course increasingly interactive but also verified your understanding of the technologies discussed.

I very much look forward to going to the CCNP bootcamps. Thank you again Keith and staff for making my learning experience a great one!
Thomas Osborne - CCNA



CCNP ROUTE

The instructor Keith is very knowledgeable, patient, and polite. He covered everything possible with the amount of time we had. I also like the format of having to take routing and switching separately.  
Sirak Zewdie - CCNA



CCNP SWITCH

Keith is a very detailed instructor and kept the class engaged throughout the week.  He goes through material in the workbooks, but he also makes sure you have a real world understanding by either showing us through examples on the hardware or drawing it out.  He makes sure there is an understanding before moving forward.    
Theresa



CCNP Routing & Switching

Keith is a very good mentor. This was my first time attending an in class session and it was a great, informative experience for me. Keith is one gem of an instructor. He was kind, patient and very informative all through the course. The way he explains concepts with his hand gestures and whiteboard diagrams helped me understand them with ease. The lab sessions were the best. I personally loved the way the course was structured and would definitely recommend it for students who are planning to take it. Overall it was more than a lifetime experience for the money I paid.
Satish Devan

Keith was an amazing instructor. Luckily my class had so few people that he knew us by first name. Keith was knowledgeable in every aspect of the CCNP R&S track. You could even throw "scenarios" at him and he would work out the problem in his head. Then he would show you how the logic of the protocols would behave. Keith was able to go very in depth on the routing protocols and that really helped me feel more comfortable with large networks!  
Victor Halili ~ CCNA 2x & CCNP



CCIE Routing & Switching Written

I would like to thank INE and Rohit Pardasani in particular for the great experience I had. This course has provided me with a good idea on how the CCIE exam should be approached in terms of study and preparation. I will highly recommend this training to anybody that is on the same journey as I am. Also, many thanks to the Coordinator in providing all the information needed and answering my questions, this made everything as smooth as I hoped.
Berry Batist

I enjoyed Rohit personally. I would look for him if seeking another workshop. The content/material informed and challenged me. I feel better prepared. 
John Wattenbarger ~ CCNP R&S and Passed CCIE Written October 2017



CCIE Routing & Switching Lab

Dave Smith exceeded my expectations concerning his depth of knowledge of the material covered. I especially appreciated his incorporation of first-hand experience and historical context of the technologies. Relevant anecdotes help to solidify "why" something does what it does, rather than just memorizing. His enthusiasm and passion for networking really do help keep you focused on and listening to everything he says. 
Scott Bridges ~ CCIE #55860, VCP-NV, RCSP-WAN

This will open your eyes to the way you have to be prepared. Knowing the technology is one thing, but knowing different ways to resolve issues with the technologies and why is another! I would strongly recommend that every CCIE candidate take this class!  
Vernado Deal ~ CCNP and CCNAx3 (Voice, R&S, Security)



CCIE Routing & Switching Graded Practice Labs

I thought this class was very beneficial in helping me prepare for my first CCIE lab attempt. It not only tested my knowledge on many aspects within the CCIE blueprint, but also included some of the question semantics and "gotchas" that should be expected on the lab. I do feel like I am better prepared for what I should expect on the actual lab, and I have identified some weak areas that I need to work on before I attempt my lab.
Scott Charles Kitchen - CCNP route & switch and CompTIA Security+



CCIE Collaboration Lab Exam

For my last CCIE, I finally decided to go to a real bootcamp. I've made a smart choice with INE's Collaboration bootcamp taught by 5xCCIE Rohit Pardasani. I highly recommend going to this class when you feel you are about 80-90% ready to take the lab. Rohit will set you on the right path to fill in the remaining gaps!  
Roman Rodichev ~ 8xCCIE #7927, Founder & Chief Solution Architect @ ieMentor



CCIE Security Lab

The class has taught me how to approach the CCIE Lab Exam in a whole different way by showing me better ways to understand technical concepts instead of just memorizing commands, which at the end will be key to continue my certification and career goals. This is by far the best technical training I have received and will certainly be back for more.
Borman Bravo



CCIE Data Center Lab

Brian McGahan is an amazing instructor. He not only explained difficult concepts with ease but also helped in strengthening my basics of networking. He is very friendly to students and addresses even the silliest doubt without any hesitation, which instills confidence to speak your doubts and know that there is someone there for true guidance. I have enjoyed every moment of the class. I would love to attend future bootcamps whose instructor would be Brian McGahan.
Zain Nachiz

The boot camp was outstanding, and Brian was very patient and informative. This one-week boot camp was the equivalent of months of studying on my own.
Matt - CCNP route/switch



CCIE Service Provider Lab

Rohit is an extremely knowledgeable professional. He gave me insight into routing protocols that I never had before.
Doug Gluntz ~ CCNP R&S, BCNP, ACWA



Apr
28

It’s about that time of year again - with Cisco Live US 2014 just a few weeks away, INE has several new and exciting developments to talk about.

First, I’d like to invite all of you to join us at INE Rewired, our 2014 Customer Appreciation Party at Cisco Live. The party is on Tuesday, May 20th, 2014, at 6:00pm at the Mezzanine Nightclub San Francisco. We’ll be serving complimentary cocktails and appetizers, there will be over $25,000 in prizes and giveaways, and most importantly we'll have a sneak peak at INE’s revolutionary new training platform, which will completely change the way you learn – but more on that coming soon. Second, we have two important announcements regarding new and updated products for CCIE R&Sv5.

On May 9th & 10th, 2014, I will be running an online CCIE RSv4 to RSv5 Transition Technologies Class. This online course is for candidates who have already invested a significant amount of time preparing for the CCIE Routing & Switching version 4 blueprint, but will be taking the lab exam in the new version 5 blueprint format that begins on June 3rd, 2014. This class focuses on technologies that have been newly introduced in the v5 blueprint, such as IPsec VPNs, DMVPN, GETVPN, and Embedded Packet Capture, as well as technology enhancements such as VTPv3, EIGRP Multi-AF Mode & Wide Metrics, EIGRP HMAC & OSPF SHA Authentication, Bidirectional Forwarding Detection (BFD), and OSPF Loop Free Alternative (LFA) just to name a few. All Access Pass members can attend the class for free, but customers who purchase the CCIE RSv4 to RSv5 Transition Technologies Class will have real-time access to ask me questions via chat and will also be granted early beta access to our new new training platform, which is scheduled for release shortly after Cisco Live.

Last but not least, this week we will begin releasing our updated CCIE Routing & Switching Version 5 Workbook. INE’s CCIE Routing & Switching v5 Workbook will collapse the previous four volume format used in our CCIE Routing & Switching v4 Workbooks into a single product. Customers who previously purchased either INE’s CCIE Routing & Switching Workbook Volume I or Volume II will have the new Version 5 Workbook automatically added to their INE Members account before the end of the week. Anyone who purchased just Volume III or Volume IV as standalone products can get in touch with sales@ine.com for special upgrade pricing to the new Version 5 workbook.

The new CCIE Routing & Switching Version 5 Workbook will be priced at $499, or $299 for AAP Members. However, if you purchase either the current Volume 1 or Volume 2 workbook before Thursday, you'll still be eligible for the free upgrade to the Version 5 Workbook, and essentially save $300 on it.  Just don't tell my sales team that I told you about this loophole ;)

 

Apr
24

Introduction

In a continuing effort to protect the integrity of the CCIE program, Cisco has announced a major change regarding the retake policy of the CCIE Written and Practical Lab exams. These changes take effect on August 1, 2014. Assuming a candidate happens not to pass on their first attempt at either a written or a practical "lab" exam within a given track, the frequency with which they will be allowed to retake the exam will change dramatically from past allowances, effectively not allowing the candidate virtually 'unlimited' retakes within a single calendar year (more specifically, within 12 calendar months from the date of the first attempt).

Changes to CCIE Practical Lab Exam

Perhaps the most interest for most people will be the frequency with which one will be allowed to re-sit for a CCIE Lab exam. Assuming a candidate does not pass on their first attempt at a given lab exam, they will still be allowed to attempt to retake the exam after 30 days has elapsed. The major change comes with the possibility that the candidate does not pass on their second attempt - after this attempt they must now wait for another 90 days to make their third attempt. Unlikely, but assuming a failure on attempt three, and a need to sit for attempt four, the candidate must wait another 90 days. Same goes for attempt four to attempt five. After a very, very bad year whereby a need to appear a sixth time becomes necessary, the wait period goes up to a full six months between attempts. The changes can be seen in a screenshot from a recent webinar below (after the jump).


Changes to CCIE Written Lab Exam

Next up are the changes to the CCIE Written exam. These changes are a bit more straightforward - wait time between failures increases from 5 days to 15 days, and you simply may not attempt the written exam more than 4 times in a single calendar year. Again, the changes can be seen in a screenshot from the same webinar.


Resources

The webinar discussed above can be found here, and it goes on to discuss a large number of other very interesting topics such as the new role of the Network Programability Engineer (SDN, OnePK, ACI/APIC) and the dedicated DevNet lounge that Cisco will have coming up in a few weeks at Cisco Live in San Francisco. Some other smaller changes are also announced in the webinar, such as the new re-scoring policy. The major changes announced above start around the 35 minute mark into the webinar. The re-scoring policy change pertains mainly to the new fact that a candidate may now pay $400USD to have any relevant output (show, debug, etc) obtained by scripts or by proctors during the grading of your CCIE lab exam (any CCIE lab) reviewed again to decide if they may possibly reverse your previous 'Fail' to a 'Pass'. This is different than the reread (which is still only available for the R&S and SP tracks at the cost of $600USD), whereby your full lab configurations are reloaded onto the relevant devices and the entire grading performed a second time. Also of interest and discussed in the webinar are the current policies of rescheduling labs both outside of the 90 days for free and also within the 90 day window for a small fee (if your credit card or wire transfer has already been processed).

Takeaway

The changes Cisco is making are done with one purpose in mind - namely to protect the integrity of the CCIE Program while maintaining that candidates apply themselves aptly in a rigorous manner leading up to their attempt at an examination. Anyone properly prepared, with a thorough understanding of every possible permutation of every technology listed in the blueprint for the track they are studying for, still has only to go and apply the technologies they've studied hard to learn, and expect success.

May
17

Edit: The INE party will be at the Hard Rock *Hotel*, not the Hard Rock *Cafe*.

I would like to thank the over 600 people who RSVP'd for INE's 2013 Party at the Hard Rock Hotel in Orlando during Cisco Live. Registration is closed as of today for our party but I wanted to be the first to let everyone know about the grand prize giveaway we are doing. On top of the standard giveaway prizes (iPads, MacBook Airs, AAP Memberships, Bootcamps, etc) we are giving away a Harley Davidson 2013 XL 1200X Forty-Eight to a lucky winner during our party.

Sportster Forty Eight

On top of the Harley Davidson 2013 XL 1200X Forty-Eight we're having a second grand prize giveaway. Details on the second grand prize giveaway will be revealed after the drawing for the winner of the Harley Davidson at the party.

As a side note I don't personally ride anymore but that bike really does look cool when it's all blacked out.

May
02

The top contributors in May for the following forums will receive their choice of either an iPad or Samsung Galaxy Note 10.1 tablet. The top overall contributor for IEOC for May will receive their choice of an Apple Macbook Air (13" 256GB) or Google Pixel with 4G LTE.

CCIE Routing & Switching Technical

CCIE Service Provider Technical

CCIE Security Technical

CCIE Voice Technical

CCIE Data Center Technical

Additionally the best CCIE success story (most details, inspirational, etc) post on IEOC in May will also receive their choice of either an iPad or Samsung Galaxy Note 10.1 tablet.

Good Luck!

Mar
23

As many of you already know I've been teaching our brand new Routing and Switching CCIE bootcamps for the past couple months. I've recently restructured the bootcamp in that mock labs are not done during the second week to allow for expanded coverage of the CCIE blueprint topics. The mock labs are now done after the bootcamp and I give every student 1500 tokens to schedule them. The new structure allows for roughly 100+ hours of hands-on demonstration using my own rack over the course of the two weeks. I start at 9am and go until roughly 8pm each night (Friday's included). After 8pm the students usually go for dinner and start working on the labs and/or review for the next day. Every topic is covered hands-on by me during the two weeks. Students are given the R&S CCIE ATC Videos free of charge when scheduling the bootcamp because even with 100+ hours we do not have time to cover the basics.

As far as my teaching style goes I do not use PowerPoint slides or just draw on a whiteboard. I break the technologies down, explain how they work and then do everything hands-on. Students learn by actually seeing me configure, verify, and troubleshoot the topics in the CCIE Lab Exam Blueprint. Most students follow along with me using their own dedicated rack that we of course provide. That is the differentiating factor of my style. I do not just tell you how something works using a PowerPoint slide or whiteboard, I show you on the routers and switches. My logic is that if your CCIE instructor can't do everything hands-on from the CCIE Lab Exam Blueprint right there in the bootcamp, then they shouldn't be your CCIE instructor. If you walk in Monday morning and they throw you labs to do while they sit there checking their email or browsing the web, then they shouldn't be your CCIE instructor. If your CCIE instructor wants to waste your time talking about meditation, exercise or what fruits to eat when preparing for your exam, then they shouldn't be your CCIE instructor. These people may be nice guys but they're wasting your time and money. At the end of the day you're in the bootcamp to learn so before buying any CCIE bootcamp make sure you know what exactly will be covered and how much time the instructor will spend teaching. I literally stand up for 10 to 12 hours a day covering the topics not because I like to be hard on my feet but there is so much that I want to cover in the two weeks. In fact I've recently added a R&S CCIE Troubleshooting Bootcamp to spend an additional week focusing purely on the troubleshooting portion of the CCIE Lab Exam. You even get to use the same awful desktop like you will have in the real lab during the Troubleshooting Bootcamp. As you can tell I’m deeply passionate about teaching networking and Cisco in general plus I really enjoy helping people.

Here are a couple reviews from a student in my last London bootcamp to get an idea of my style: (review 1 and review 2).

In addition to the new structure I've built 24 new R&S CCIE racks for the bootcamps. These racks are identical to the Cisco 360 topology. The racks all have 4 x 3560's and a mix of 1841s, 2811s & 3825s. I of course do not use the Cisco 360 course materials as I have a totally different teaching methodology as mentioned above. During the bootcamp I VPN into my rack to use tools like Wireshark along with EEM Packet Capture to help students get a better understanding of how technologies like PIM, DHCP Relay, etc work in addition to show and debugging commands. I've also just added in virtual machines to my rack to generate traffic for the testing and verification of topics like QoS, WCCP, AAA, etc. Just talking about these technologies isn't enough to truly understand them. Seeing a topic like WCCP implemented (filter and/or caching webpages) helps students get a better understanding of not only how to configure, verify and troubleshoot it but how the technology is used in the real world.

We’ve had a lot of growth recently here at INE. In addition to making Inc 5000’s list of fastest growing education companies (we'll also make the list next year) we’ve built new classrooms in Bellevue, WA (greater Seattle area). Our new offices and classrooms are on the same floor in the Bellevue City Center building as Cisco. The Cisco employees in my last bootcamp liked being located next to the Cisco office. We've added a whole new video production department here in Bellevue and the quality of our new videos really shows.

If anyone has attended an INE R&S CCIE Bootcamp in the past that wasn't taught by me, I highly recommend that you comeback and sit one of my bootcamps. It's no cost to you of course to resit the bootcamp. All you need to do is contact sales to get your seat reserved. I would also like to encourage potential students to come audit one of my bootcamps before purchasing. I will be in Dubai, Washington DC, RTP, NYC, London, Seattle and Amsterdam.

Lastly I would like to give back by starting up the CCIE Scholarship Program again for 2012. I will personally select one person from each of the following regions to receive everything they need to pass the CCIE Lab Exam (bootcamp, tokens, workbooks, videos, etc) including paying for their lab exam fee. The regions are Americas, Europe, Africa, Asia and Oceania as defined by Wikipedia. The details about the Scholarship Program will be posted next week.

Dec
14

A new update to INE’s CCIE Routing & Switching Written Exam Bootcamp is now available in streaming format for All Access Pass subscribers, and available for purchase as a download.  This completely new video series, taught by me – Brian McGahan, 3 x CCIE #8593 (Routing & Switching, Security, Service Provider) – is specifically designed for students looking to focus on the topics and technologies covered in the CCIE Routing and Switching Written Exam version 4 blueprint.

As a precursor to our CCIE Routing & Switching Advanced Technologies Class and our CCIE Routing & Switching Lab Workbook Volume 1, the Written Exam Bootcamp helps to create a solid foundation of the concepts covered in the CCIE Routing & Switching version 4 Written Exam, as well to give students the knowledge they need to continue straight into their hands-on CCIE Lab Exam preparation. This bootcamp will also benefit current CCIEs who need to re-affirm their knowledge from a theoretical standpoint in order to recertify on the various technologies covered on the CCIE Routing & Switching Written Exam blueprint.

Samples of the class can be found here.

Nov
01

This blog post is the first in a series covering Performance Routing (PfR) formerly known as Optimized Edge Routing (OER) that I will be publishing over the coming weeks.  I decided to cover PfR in a series of blog posts contrary to a single post as PfR is a very powerful and useful feature that leverages the power of Cisco’s IOS but at the same time PfR is potentially very complicated and often confusing feature to configure and troubleshoot.  Trying to cover PfR in a single blog post would be the equivalent of trying to cover OSPF in a single blog post.  In fact if you compare the IOS 12.4T OSPF Configuration Guide against the Optimized Edge Routing (OER) Configuration Guide you will notice that OER documentation is nearly 35% larger.

NOTE
In this blog post the term PfR will be used in place of OER wherever possible as Cisco has started to depricate the OER terminology and commands as of IOS 15.0.

The primary focus of these blog posts will be how PfR relates to the Routing and Switching CCIE Lab Exam (PfR v2.2).  The first couple posts will cover basic scenarios (static routing and BGP) with PfR, while introducing the reader to the PfR specific terminology and features as we use then and/or run across them. After we cover the basic scenarios I will get into more complicated scenarios using PfR to optimize routing based upon DSCP values, inbound routing optimization using BGP, routing based upon application response time and voice call quality. A final post will cover PfR in IOS 15.1 (PfR v3.0) and will focus on some of the newer PfR features. I will try to keep the details and complexities of PfR out of the first couple blog posts so that the reader can gain a solid grasp of PfR before moving forward.  I spend roughly half a day in my new Routing and Switching CCIE 10 Day Bootcamp covering PfR as it's important for the R&S CCIE candidate has a solid understanding before attempting the CCIE Lab exam. Additionally, I personally believe that in the future the concept of centralized route control and/or route manipulation as with PfR could become more common, similar to the concept of OpenFlow. With that being said lets get started.

Performance Routing (PfR), previously called Optimized Edge Routing (OER), introduces a new concept into IP routing. With traditional routing, path selection decisions do not consider a particular path's traffic characteristics be it throughput, actual delay, packet loss, voice mean opinion score (MOS), monetary cost of a given path, or jitter. PfR enhances the classic destination-based routing concept centered on the shortest path (lowest-cost metric) by adding into the routing selection process, network performance and/or application performance aware intelligence.  In the past when routing protocols were implemented in large-scale networks, routers did not have the resources to calculate the best path based upon anything other than a simple metric. Additionally, many of these networks would be considered simplistic in regard to the number of primary and redundant links compared to today's networks. With the increase in CPU power and memory available in routers today, routing based solely upon a simple metric (hop count, cost, as-path length, etc.) is not the best use of these available resources. PfR will leverage these available resources to allow routing decisions based upon additional factors namely the networks performance and/or application performance across the network.  Getting the most out of your network’s available bandwidth and/or the best possible performance for your applications across the network should be one of the primary goals of any network implementation. Let's look at an example of how we can do this using PfR.

A site consisting of a single router has two connections to the Internet.  One from service provider A and the other from service provider B.  The link through provider A is a 10Mbps Ethernet link while the link through provider B is a 1.5Mbps T1 link.  The Ethernet link is the primary path used to reach the Internet, and the T1 is used as a backup in the event the Ethernet link is down.  Routing is done using static default route pointing out the Ethernet link and an additional static with a higher administrative distance (floating static) pointing out the T1 link.  With this configuration, the T1 link is only used in the case the Ethernet link is down.  This means the T1 link primarily sits idle for the vast majority of the time.  The administrator could implement additional static routes pointing out the T1 link for networks that provider B originates to at least get some use out of the T1 but this is not an ideal solution.  Ideally, the router itself will automatically select the best path for traffic based upon performance through the different service providers and/or allow traffic to overflow to the T1 link when the Ethernet link starts to become fully utilized.  For simplify we will configure the latter case in this blog post.

OER / PfR

In our scenario, the requirements are for the router to automatically move traffic flows to the T1 link when the Ethernet link becomes 75% utilized.  Using traditional static routes, activating another path based upon a link's utilization is not possible without some sort of hacked up solution (EEM, TCL, packet marking with policy based routing, etc).  PfR will moving these traffic flows for us.  As the primary Ethernet link’s utilization rises above the 75% utilization threshold, PfR will start to look for an alternate path to route traffic flows across in order to bring down the utilization on the Ethernet link.  In our scenario, PfR will automatically use NetFlow, the interface’s utilization, along with active probing with IP SLA to determine the availability of an alternate path.  Knowledge of NetFlow and IP SLA is not needed as PfR manages the configuration of these technologies.

Now that we know the problem we are trying to solve let's start looking at PfR itself.  For PfR, we have the concept of a Master Controller (MC) and Border Router/Routers (BR). The MC is the centralized decision maker in the PfR network.  It is responsible for controlling the BRs and acts as a central location to store and analyze data collected from the BRs.  The MC does not need to be in the path of the traffic flows but does need basic IP reachability to the BRs.  Often in real-world medium and large-scale PfR deployments, the MC will be a dedicated router. Traditionally, the BRs are the edge routers with links to external networks (i.e. the Internet). This is where the term Optimized Edge Routing (OER) came from as we are optimizing the routing at the edge of our network but as we will see later, the added ability in the newer IOS versions for routing based upon network and/or application performance, Cisco renamed it Performance Routing (PfR). This is why today, PfR is suited for used in enterprise networks as we will see in future blog posts. The BRs are in the traffic flow paths and are responsible for collecting the NetFlow data and IP SLA probe results along with influencing the traffic flow paths by executing policy change instructions issued by the MC.

MC and BRs

The communication between the MC and BRs uses an authenticated TCP connection on port 3949. The BR is responsible for initiating the TCP connection. In the case of a firewall or any traffic filters between the MC and BR, TCP port 3949 needs to be open in the path of the BR towards the MC. The default TCP port can be changed and the default interface used for the connection (based upon the outbound interface used to route to the MC) can be changed. The password used for authentication is done similar to how EIGRP and RIP handle authentication in IPv4, and that is through the use of key chains. Being that PfR is a path selection technology we will need to define at least one internal and two external interfaces. In our PfR setup along with the requirement of having a MC and BR will need to have at least one internal and two external interfaces on the BR. These interfaces will be defined on the MC.

NOTE
The two external interfaces are not required to be located on the same BR. One external interface could be located on one BR, and additional external interfaces could be located on another BR or even several BRs. External interfaces are not required to be physical interface and can be logical interfaces (Tunnels, Dialer, etc).

The PfR will enable NetFlow on the BR's interfaces to allow for the collection of traffic statistics to be sent back to the MC. For our first basic example, we will have the MC and BR located on the same router. Below is the diagram for this scenario:

OER PrF CCIE

Here is the relevant configuration for this scenario prior to configuring PfR. To ease the creation of congestion, the T1 has been clocked down to 64k, and the Ethernet link is being shaped to 256k. You will notice the floating static route pointing out the Serial link in the event the Ethernet interface goes down. This additional static route is what PfR will term a parent route for the Serial link. Since it's not important at this point, the concept of a parent route will be covered at the end of the blog post. We just need to understand that an additional route that encompasses our traffic is needed over the alternate external interface. Lastly, the load interval for the interfaces has been set to 30 seconds contrary to the default of 300 seconds. This allows PfR to react quicker to changes in the load (throughput) of the interface.

Rack1R3#show run
Building configuration...

Current configuration : 2236 bytes
!
version 12.4
!
hostname Rack1R3
!
policy-map 256_SHAPE
class class-default
shape average 256000 8000 0
!
interface Loopback0
ip address 150.1.3.3 255.255.255.255
!
interface FastEthernet0/0
description Internal
ip address 204.12.1.3 255.255.255.0
!
interface FastEthernet0/1
description External
bandwidth 256
ip address 192.10.1.3 255.255.255.0
load-interval 30
service-policy output 256_SHAPE
!
interface Serial1/0
encapsulation frame-relay
load-interval 30
!
interface Serial1/0.1 point-to-point
description External
bandwidth 64
ip address 192.10.2.3 255.255.255.0
frame-relay interface-dlci 301
!
ip route 0.0.0.0 0.0.0.0 192.10.1.1
ip route 0.0.0.0 0.0.0.0 192.10.2.1 5
!
end

Rack1R3#show ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route

Gateway of last resort is 192.10.1.1 to network 0.0.0.0

C 204.12.1.0/24 is directly connected, FastEthernet0/0
C 192.10.1.0/24 is directly connected, FastEthernet0/1
C 192.10.2.0/24 is directly connected, Serial1/0.1
150.1.0.0/32 is subnetted, 3 subnets
O 150.1.7.7 [110/2] via 204.12.1.7, 2d22h, FastEthernet0/0
O 150.1.6.6 [110/2] via 204.12.1.6, 2d23h, FastEthernet0/0
C 150.1.3.3 is directly connected, Loopback0
S* 0.0.0.0/0 [1/0] via 192.10.1.1
Rack1R3#

 

NOTE
With static routes it's recommended in production environments to use IP SLA and Enhanced Object Tracking to monitor the primary static route's next-hop availability if the interface's state does not indicate availability of the next-hop. This is common when a layer 2 switch is located between the local router and the next-hop.

We will start with the BR configuration. First, we need to define a key chain in the global configuration:

Rack1R3(config)#key chain OER
Rack1R3(config-keychain)#key 1
Rack1R3(config-keychain-key)#key-string CISCO
Rack1R3(config-keychain-key)#exit
Rack1R3(config-keychain)#exit
Rack1R3(config)#exit
Rack1R3#show run | section key chain
key chain OER
key 1
key-string CISCO
Rack1R3#

The key chain name is arbitrary and in this case, we used the name OER. We used the key numbered 1 with a key string of CISCO. For this example since the MC and BR will be located on a single router the same key chain will be used. In fact, if you change the key chain name used for the MC under the BR configuration sub-mode it will automatically change it for the BR under the MC configuration sub-mode. Some sort of idiot-proofing in the IOS ;-) Next we will configure the actual BR. On the BR we will configure the interface used to source the TCP connection off of and the IP address of the MC along with the key chain used to authenticate the session. The BR configuration is done in the OER border configuration sub-mode. To enter this mode type oer border in the global configuration.

Rack1R3(config)#oer border
Rack1R3(config-oer-br)#local Loopback0
Rack1R3(config-oer-br)#master 150.1.3.3 key-chain OER
Rack1R3(config-oer-br)#exit
Rack1R3(config)#exit
Rack1R3#show run | section oer border
oer border
local Loopback0
master 150.1.3.3 key-chain OER
Rack1R3#

 
The local command is used to control the interface the TCP session is sourced off of. This commonly is a loopback but could be any interface that has reachability to the MC. The master command is used to define the IP address of the master along with the key chain. The only other requirement for the BR is that CEF is enabled, which should be by default but remember to check that it is enabled when troubleshooting a PfR issue.

NOTE
As of IOS version 15.0 the oer keyword is changed to pfr and the oer keyword will eventually be removed from the IOS.

Rack1R1(config)#do show version | include Version 15.1
Cisco IOS Software, 2800 Software (C2800NM-ADVENTERPRISEK9-M), Version 15.1(3)T2, RELEASE SOFTWARE (fc1)
Rack1R1(config)#oer ?
border Enter PfR border router configuration submode
master Enter PfR master controller configuration submode

Rack1R1(config)#pfr ?
border Enter PfR border router configuration submode
master Enter PfR master controller configuration submode

Rack1R1(config)#pfr border
Rack1R1(config-pfr-br)#exit
Rack1R1(config)#oer border
Rack1R1(config-pfr-br)#

 
Now we will configure the MC. As with the BR configuration being done in the config-oer-br configuration sub-mode, the MC will be done in similar configuration sub-mode. We will start off by entering the MC configuration sub-mode by issuing the oer master command from the global configuration. Next we will define the BR's IP address, key chain, internal and external interfaces.

Rack1R3(config)#oer master
Rack1R3(config-oer-mc)#border 150.1.3.3 key-chain OER
Rack1R3(config-oer-mc-br)#interface FastEthernet0/0 internal
Rack1R3(config-oer-mc-br)#interface Serial1/0.1 external
Rack1R3(config-oer-mc-br-if)#interface FastEthernet0/1 external
Rack1R3(config-oer-mc-br-if)#exit
Rack1R3(config-oer-mc-br)#exit
Rack1R3(config-oer-mc)#exit
Rack1R3(config)#exit
Rack1R3#show run | section oer master
oer master
!
border 150.1.3.3 key-chain OER
interface FastEthernet0/0 internal
interface Serial1/0.1 external
interface FastEthernet0/1 external
Rack1R3#

We can now verify that the MC and BR processes on the router are communicating:

Rack1R3#show oer master
OER state: ENABLED and ACTIVE
Conn Status: SUCCESS, PORT: 3949
Version: 2.2
Number of Border routers: 1
Number of Exits: 2
Number of monitored prefixes: 0 (max 5000)
Max prefixes: total 5000 learn 2500
Prefix count: total 0, learn 0, cfg 0
PBR Requirements met
Nbar Status: Inactive

Border Status UP/DOWN AuthFail Version
150.1.3.3 ACTIVE UP 00:00:52 0 2.2

Global Settings:
max-range-utilization percent 20 recv 0
mode route metric bgp local-pref 5000
mode route metric static tag 5000
trace probe delay 1000
no logging
exit holddown time 60 secs, time remaining 0

Default Policy Settings:
backoff 300 3000 300
delay relative 50
holddown 300
periodic 0
probe frequency 56
number of jitter probe packets 100
mode route observe
mode monitor both
mode select-exit good
loss relative 10
jitter threshold 20
mos threshold 3.60 percent 30
unreachable relative 50
resolve delay priority 11 variance 20
resolve range priority 12 variance 0
resolve utilization priority 13 variance 20

Learn Settings:
current state : DISABLED
time remaining in current state : 0 seconds
no throughput
no delay
no inside bgp
no protocol
monitor-period 5
periodic-interval 120
aggregation-type prefix-length 24
prefixes 100
expire after time 720
Rack1R3#show oer border
OER BR 150.1.3.3 ACTIVE, MC 150.1.3.3 UP/DOWN: UP 00:01:00,
Auth Failures: 0
Conn Status: SUCCESS
OER Netflow Status: ENABLED, PORT: 3949
Version: 2.2 MC Version: 2.2
Exits
Fa0/0 INTERNAL
Fa0/1 EXTERNAL
Se1/0.1 EXTERNAL
Rack1R3#show oer master border detail
Border Status UP/DOWN AuthFail Version
150.1.3.3 ACTIVE UP 01:25:10 0 2.2
Fa0/0 INTERNAL UP
Se1/0.1 EXTERNAL UP
Fa0/1 EXTERNAL UP

External Capacity Max BW BW Used Load Status Exit Id
Interface (kbps) (kbps) (kbps) (%)
--------- -------- ------ ------- ------- ------ ------
Se1/0.1 Tx 64 48 0 0 UP 2
Rx 64 0 0
Fa0/1 Tx 256 192 0 0 UP 1
Rx 256 0 0
Rack1R3#

 

NOTE
For the PfR versions, there is a major release and minor release number. For example, PfR v2.2 in IOS 12.4T or PfR v3.0 in IOS 15.1. The "2.x" and "3.x" are the major version numbers and "x.2" and "x.0" are the minor version numbers. The PfR major version number must match between the MC and BRs. Additionally the minor release version on the MC must be equal to or greater than the BR's minor release version. The version used can be determined by issuing the show oer master on the MC and show oer border on the BR.

Now that we have our MC and BR defined along with our internal and external interfaces, we need to configure the MC to start monitoring the Ethernet link's throughput rate so that traffic can be moved to the Serial link once the 75% threshold has been breached. To do this we need to enter the learn configuration sub-mode under the MC configuration sub-mode. If you look back at the output from the show oer master command, you will notice that the default state for learning phase is disabled. Once we are under the learning configuration sub-mode we will configure the MC to starting monitoring the throughput of the traffic flows from the internal interfaces out to the external interfaces.

Rack1R3(config-oer-mc)#learn
Rack1R3(config-oer-mc-learn)#throughput
Rack1R3(config-oer-mc-learn)#do sho oer master | begin Learn
Learn Settings:
current state : STARTED
time remaining in current state : 358 seconds
throughput
no delay
no inside bgp
no protocol
monitor-period 5
periodic-interval 120
aggregation-type prefix-length 24
prefixes 100
expire after time 720
Rack1R3(config-oer-mc-learn)#

The default interface utilization threshold is 75% when monitoring throughput. This can be changed under the interface configuration sub-mode for a particular BR (config-oer-mc-br-if). The value can be entered as either a percentage or absolute value. The configuration flow is as follows:

Rack1R3(config)#oer master
Rack1R3(config-oer-mc)#border 150.1.3.3 key-chain OER
Rack1R3(config-oer-mc-br)#interface FastEthernet0/1 external
Rack1R3(config-oer-mc-br-if)#max-xmit-utilization ?
absolute Specify the utilization as an absolute value
percentage Specify the utilization as a percentage of the exit's bandwidth

The MC can generate log messages containing information in regard to its operation. By default, this behavior is disabled but can be enabled by issuing the logging command under the OER master configuration sub-mode. This will help us get a better understanding of what PfR is doing without the need for show commands or enabling debugging:

Rack1R3(config)#oer master
Rack1R3(config-oer-mc)#logging

By default the MC is in a route control mode of observe. This means monitoring and reporting will occur but no policy changes will be issued by the MC to the BRs. In this mode we can view what the MC is observing and what actions it would take assuming it wasn't in observe mode. Below are the log messages from the MC when the 75% threshold is breached and the MC is in observe mode.

Oct 31 11:50:05: %OER_MC-5-NOTICE: Range OOP BR 150.1.3.3, i/f Fa0/1, percent 100. Other BR 150.1.3.3, i/f Se1/0.1, percent 0
Oct 31 11:50:05: %OER_MC-5-NOTICE: Load OOP BR 150.1.3.3, i/f Fa0/1, load 256 policy 192
Oct 31 11:50:05: %OER_MC-5-NOTICE: Exit 150.1.3.3 intf Fa0/1 OOP, Tx BW 256, Rx BW 256, Tx Load 100, Rx Load 100
Oct 31 11:50:27: %OER_MC-5-NOTICE: NO route change, Observe mode, Prefix 114.0.1.0/24, BR 150.1.3.3, i/f Se1/0.1, Reason Delay, OOP reason Range

For our scenario, we want the BR to actually move traffic flows by injecting a more specific static route over the T1 link when the Ethernet's link utilization threshold of 75% is breached. In order to do this we need PfR to determine what is called the traffic classes from the traffic flows that are flowing through BRs from the internal interface to an external interface. When OER was first introduced in IOS 12.3(8)T only layer 3 network prefix based traffic classes were supported. In newer IOS versions, we can define traffic classes based upon the layer 4 port number, DSCP value or even application using NBAR. In this scenario, we will use automatically learned layer 3 prefix traffic classes but in future blog posts, we will manually define the traffic flows that we are interested in PfR monitoring.

Since we are going to allow PfR to learn the traffic classes, these classes will be based upon the Netflow Top Talkers and then aggregated into /24 traffic classes. By default, PfR aggregates traffic flows into /24 traffic classes before sending them to the MC. The level of aggregation can be changed using the aggregation-type prefix-length <1-32> command in the learn configuration sub-mode. Once received by the MC these traffic classes are what is called Monitored Traffic Classes or MTC for short. We will be able to view these by using the show oer master traffic-class command. What this /24 aggregation means for our scenario is that when we have two traffic flows that are not in the same /24 address space (i.e. traffic destined for 114.0.0.1 and traffic destined for 114.0.1.1) the BR will aggregate these into two traffic classes: 114.0.0.0/24 and 114.0.1.0/24. Once the threshold is breached the traffic class will be considered to be what is called Out of Policy (OOP). When this happens the MC will try to move the one of the traffic classes to the T1 link by injecting a static route for the specific traffic class (i.e. 114.0.1.0/24) out the Serial1/0.1 external interface.

To have PfR automatically learn the traffic classes and alter the static routing for our scenario, we will need to enable the learning of the prefixes and trigger the route policy change based upon the external interfaces' throughput. We could base the trigger upon delay, packet loss and/or reachability but for this simplest of cases, we're just going to use throughput. Additionally, we will need to change the MC's route control policy settings from observe (mode route observe) to route control (mode route control) so that the MC can instruct the BR to inject static routes allowing it to move traffic from the Ethernet link to the T1 link. This mode can be seen using the show oer master command.

Rack1R3#show oer master | include ^Default|mode route control
Default Policy Settings:
mode route control
Rack1R3#conf t
Enter configuration commands, one per line. End with CNTL/Z.
Rack1R3(config)#oer master
Rack1R3(config-oer-mc)#mode route control
Rack1R3(config-oer-mc)#learn
Rack1R3(config-oer-mc-learn)#throughput
Rack1R3(config-oer-mc-learn)#exit
Rack1R3(config-oer-mc)#exit
Rack1R3(config)#exit

 

NOTE
The MC will learn up to 100 of these traffic classes by default. This can be changed by using the prefixes <1-100000> command in the learn configuration sub-mode. Before IOS 12.2(15)T the IOS was limited to a maximum of 5000 prefixes, but this limitation was bumped up to 100000 in the more recent IOS versions.

Before we actually start to generate traffic flows for testing, let's take a look at our final configuration (relevant commands):

Rack1R3#show run
Building configuration...

Current configuration : 2283 bytes
!
version 12.4
!
hostname Rack1R3
!
ip cef
!
key chain OER
key 1
key-string CISCO
!
oer master
logging
!
border 150.1.3.3 key-chain OER
interface FastEthernet0/0 internal
interface Serial1/0.1 external
interface FastEthernet0/1 external
!
learn
throughput
mode route control
!
!
oer border
local Loopback0
master 150.1.3.3 key-chain OER
!
policy-map 256_SHAPE
class class-default
shape average 256000 8000 0
!
interface Loopback0
ip address 150.1.3.3 255.255.255.255
!
interface FastEthernet0/0
description Internal
ip address 204.12.1.3 255.255.255.0
!
interface FastEthernet0/1
description External
bandwidth 256
ip address 192.10.1.3 255.255.255.0
load-interval 30
service-policy output 256_SHAPE
!
interface Serial1/0
encapsulation frame-relay
load-interval 30
!
interface Serial1/0.1 point-to-point
description External
bandwidth 64
ip address 192.10.2.3 255.255.255.0
frame-relay interface-dlci 301
!
ip route 0.0.0.0 0.0.0.0 192.10.1.1
ip route 0.0.0.0 0.0.0.0 192.10.2.1 5
!
end

 
Next we're going to start with the actual testing. First off, we're going to reset the MC's state by issuing the clear oer master * command. Next we will generate a large ICMP flow by pinging from behind R3 (204.12.1.6) out to 114.0.0.1. The packet size is set to 1400, repeat 10000000 and timeout is set to 0. This will cause the BR's external Ethernet link to exceed the 75% threshold of 192k (256k * .75). Next we will telnet to 114.0.1.1 from behind R3 (204.12.1.5). This traffic flow will be aggregated to 114.0.1.0/24 and be moved by the MC, using a static route injected on the BR, to the T1 link.

Rack1R3#clear oer master *
Rack1R3#
Oct 31 17:55:22: %OER_MC-5-NOTICE: Uncontrol prefixes, Clear all
Oct 31 17:55:22: %OER_MC-5-NOTICE: BR 150.1.3.3 DOWN
Oct 31 17:55:22: %OER_MC-5-NOTICE: BR 150.1.3.3 IF Fa0/0 Unverified
Oct 31 17:55:22: %OER_MC-5-NOTICE: BR 150.1.3.3 IF Se1/0.1 Unverified
Oct 31 17:55:22: %OER_MC-5-NOTICE: BR 150.1.3.3 IF Fa0/1 Unverified
Oct 31 17:55:22: %OER_MC-5-NOTICE: MC Inactive
Oct 31 17:55:23: %OER_MC-5-NOTICE: Uncontrol prefixes, Exit down, BR 150.1.3.3 i/f Se1/0.1
Oct 31 17:55:23: %OER_MC-5-NOTICE: Uncontrol prefixes, Exit down, BR 150.1.3.3 i/f Fa0/1
Oct 31 17:55:23: %OER_MC-5-NOTICE: Uncontrol prefixes, Clear Application all
Oct 31 17:55:23: %OER_MC-5-NOTICE: Uncontrol prefixes, Clear prefix all
Rack1R3#
Oct 31 17:55:28: %OER_MC-5-NOTICE: BR 150.1.3.3 UP
Oct 31 17:55:29: %OER_MC-5-NOTICE: BR 150.1.3.3 IF Fa0/0 UP
Oct 31 17:55:29: %OER_MC-5-NOTICE: BR 150.1.3.3 IF Se1/0.1 UP
Oct 31 17:55:29: %OER_MC-5-NOTICE: BR 150.1.3.3 Active
Oct 31 17:55:29: %OER_MC-5-NOTICE: BR 150.1.3.3 IF Fa0/1 UP
Oct 31 17:55:29: %OER_MC-5-NOTICE: MC Active
Rack1R3#
Oct 31 17:55:59: %OER_MC-5-NOTICE: Prefix Learning STARTED
Rack1R3#
Oct 31 17:56:28: %OER_MC-5-NOTICE: Range OOP BR 150.1.3.3, i/f Fa0/1, percent 100. Other BR 150.1.3.3, i/f Se1/0.1, percent 0
Oct 31 17:56:28: %OER_MC-5-NOTICE: Load OOP BR 150.1.3.3, i/f Fa0/1, load 256 policy 192
Oct 31 17:56:28: %OER_MC-5-NOTICE: Exit 150.1.3.3 intf Fa0/1 OOP, Tx BW 256, Rx BW 8, Tx Load 100, Rx Load 3

Oct 31 18:00:49: %OER_MC-5-NOTICE: Range OOP BR 150.1.3.3, i/f Fa0/1, percent 100. Other BR 150.1.3.3, i/f Se1/0.1, percent 0
Oct 31 18:00:49: %OER_MC-5-NOTICE: Load OOP BR 150.1.3.3, i/f Fa0/1, load 256 policy 192
Oct 31 18:00:49: %OER_MC-5-NOTICE: Exit 150.1.3.3 intf Fa0/1 OOP, Tx BW 256, Rx BW 8, Tx Load 100, Rx Load 3
Rack1R3#
Oct 31 18:00:59: %OER_MC-5-NOTICE: Prefix Learning WRITING DATA
Rack1R3#
Oct 31 18:01:09: %OER_MC-5-NOTICE: Range OOP BR 150.1.3.3, i/f Fa0/1, percent 100. Other BR 150.1.3.3, i/f Se1/0.1, percent 0
Oct 31 18:01:09: %OER_MC-5-NOTICE: Load OOP BR 150.1.3.3, i/f Fa0/1, load 256 policy 192
Oct 31 18:01:09: %OER_MC-5-NOTICE: Exit 150.1.3.3 intf Fa0/1 OOP, Tx BW 256, Rx BW 8, Tx Load 100, Rx Load 3
Rack1R3#
Oct 31 18:01:26: %OER_MC-5-NOTICE: Discovered Exit for Prefix 114.0.1.0/24, BR 150.1.3.3, i/f Fa0/1
Oct 31 18:01:26: %OER_MC-5-NOTICE: Discovered Exit for Prefix 114.0.0.0/24, BR 150.1.3.3, i/f Fa0/1
Rack1R3#
Oct 31 18:01:29: %OER_MC-5-NOTICE: Range OOP BR 150.1.3.3, i/f Fa0/1, percent 100. Other BR 150.1.3.3, i/f Se1/0.1, percent 0
Oct 31 18:01:29: %OER_MC-5-NOTICE: Load OOP BR 150.1.3.3, i/f Fa0/1, load 256 policy 192
Oct 31 18:01:29: %OER_MC-5-NOTICE: Exit 150.1.3.3 intf Fa0/1 OOP, Tx BW 256, Rx BW 8, Tx Load 100, Rx Load 3
Rack1R3#
Oct 31 18:02:29: %OER_MC-5-NOTICE: Uncontrol Prefix 114.0.0.0/24, Couldn't find the best exit
Oct 31 18:02:29: %OER_MC-5-NOTICE: Uncontrol Prefix 114.0.0.0/24, Couldn't choose exit in prefix timeout
Oct 31 18:02:29: %OER_MC-5-NOTICE: Range OOP BR 150.1.3.3, i/f Fa0/1, percent 100. Other BR 150.1.3.3, i/f Se1/0.1, percent 0
Oct 31 18:02:29: %OER_MC-5-NOTICE: Load OOP BR 150.1.3.3, i/f Fa0/1, load 256 policy 192
Oct 31 18:02:29: %OER_MC-5-NOTICE: Exit 150.1.3.3 intf Fa0/1 OOP, Tx BW 256, Rx BW 8, Tx Load 100, Rx Load 3
Oct 31 18:02:29: %OER_MC-5-NOTICE: Route changed Prefix 114.0.1.0/24, BR 150.1.3.3, i/f Se1/0.1, Reason Delay, OOP Reason Range
Rack1R3#

 
From the output of the MC logging, we can see that at 18:02:29 a static route is injected into the BR's routing table for the 114.0.1.0/24 MTC which took roughly 7 minutes from the time we restarted the MC (clear oer master *). This is the MTC for the telnet traffic flow from 204.12.1.5 to 114.0.1.1. Later, we will speed this process up and even configure PfR to switch within a matter of just a few seconds.

Rack1R3#show oer master traffic-class
OER Prefix Statistics:
Pas - Passive, Act - Active, S - Short term, L - Long term, Dly - Delay (ms),
P - Percentage below threshold, Jit - Jitter (ms),
MOS - Mean Opinion Score
Los - Packet Loss (packets-per-million), Un - Unreachable (flows-per-million),
E - Egress, I - Ingress, Bw - Bandwidth (kbps), N - Not applicable
U - unknown, * - uncontrolled, + - control more specific, @ - active probe all
# - Prefix monitor mode is Special, & - Blackholed Prefix
% - Force Next-Hop, ^ - Prefix is denied

DstPrefix Appl_ID Dscp Prot SrcPort DstPort SrcPrefix
Flags State Time CurrBR CurrI/F Protocol
PasSDly PasLDly PasSUn PasLUn PasSLos PasLLos EBw IBw
ActSDly ActLDly ActSUn ActLUn ActSJit ActPMOS ActSLos ActLLos
--------------------------------------------------------------------------------
114.0.0.0/24 N defa N N N N
DEFAULT* 21 150.1.3.3 Fa0/1 U
U U 0 0 0 0 8204 0
U U 0 1000000 N N N N

114.0.1.0/24 N defa N N N N
INPOLICY 0 150.1.3.3 Se1/0.1 STATIC
20 20 0 0 0 0 1 1
U U 0 0 N N N N

Rack1R3#
Rack1R3#show ip route static
114.0.0.0/24 is subnetted, 1 subnets
S 114.0.1.0 [1/0] via 192.10.2.1
S* 0.0.0.0/0 [1/0] via 192.10.1.1
Rack1R3#
Rack1R3#show oer border routes static

Flags: C - Controlled by oer, X - Path is excluded from control,
E - The control is exact, N - The control is non-exact

Flags Network Parent Tag
CE 114.0.1.0/24 0.0.0.0/0 5000
Rack1R3#

 

NOTE
The static route injected on the BR will not show up in the running configuration which means it will not be saved upon a reload. This is done to ensure if ever communication is lost between the MC and BR that any policy changes issued by the MC cannot be saved and are quickly removed.

Rack1R3#show run | include ip route
ip route 0.0.0.0 0.0.0.0 192.10.1.1
ip route 0.0.0.0 0.0.0.0 192.10.2.1 5

 
We can now see that everything worked as expected and the traffic destined for the 114.0.1.0/24 prefix is being routed across the T1 using the static route injected by PfR over the less specific static default route. As mentioned earlier PfR uses NetFlow for collecting statistics about the traffic flows. This can be seen by using the show ip cache flow command.

Rack1R3#show ip cache flow
IP packet size distribution (65263711 total packets):
1-32 64 96 128 160 192 224 256 288 320 352 384 416 448 480
.000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000

512 544 576 1024 1536 2048 2560 3072 3584 4096 4608
.000 .000 .000 .000 .999 .000 .000 .000 .000 .000 .000

IP Flow Switching Cache, 278544 bytes
3 active, 4093 inactive, 3747 added
174735 ager polls, 0 flow alloc failures
Active flows timeout in 1 minutes
Inactive flows timeout in 15 seconds
IP Sub Flow Cache, 34056 bytes
8 active, 1016 inactive, 6691 added, 3747 added to flow
0 alloc failures, 0 force free
1 chunk, 20 chunks added
last clearing of statistics never
Protocol Total Flows Packets Bytes Packets Active(Sec) Idle(Sec)
-------- Flows /Sec /Flow /Pkt /Sec /Flow /Flow
TCP-Telnet 385 0.0 108 143 0.0 31.3 7.3
ICMP 2379 0.0 27404 1463 102.5 59.8 0.4
Total: 2764 0.0 23602 1462 102.6 55.9 1.4

SrcIf SrcIPaddress DstIf DstIPaddress Pr SrcP DstP Pkts
Fa0/0 204.12.1.6 Fa0/1 114.0.0.1 01 0000 0800 26K
Fa0/1 114.0.1.1 Fa0/0 204.12.1.5 06 0017 3893 174
Fa0/0 204.12.1.5 Se1/0.1 114.0.1.1 06 3893 0017 256
Rack1R3#

 
It was mentioned earlier that by default PfR uses IP SLA along with NetFlow. Although we will go into more detail on IP SLA with PfR later, how PfR used ICMP echo probes to assist with path selection for our scenario can be seen below:

Rack1R3#show oer master active-probes
OER Master Controller active-probes
Border = Border Router running this Probe
State = Un/Assigned to a Prefix
Prefix = Probe is assigned to this Prefix
Type = Probe Type
Target = Target Address
TPort = Target Port
How = Was the probe Learned or Configured
N - Not applicable

The following Probes exist:

State Prefix Type Target TPort How Codec
Assigned 114.0.1.0/24 echo 114.0.1.1 N Lrnd N
Assigned 114.0.0.0/24 echo 114.0.0.1 N Lrnd N

The following Probes are running:

Border State Prefix Type Target TPort
150.1.3.3 ACTIVE 114.0.1.0/24 echo 114.0.1.1 N
150.1.3.3 ACTIVE 114.0.1.0/24 echo 114.0.1.1 N
150.1.3.3 ACTIVE 114.0.0.0/24 echo 114.0.0.1 N
150.1.3.3 ACTIVE 114.0.0.0/24 echo 114.0.0.1 N

Rack1R3#show oer border active-probes
OER Border active-probes
Type = Probe Type
Target = Target IP Address
TPort = Target Port
Source = Send From Source IP Address
Interface = Exit interface
Att = Number of Attempts
Comps = Number of completions
N - Not applicable

Type Target TPort Source Interface Att Comps
DSCP
echo 114.0.1.1 N 192.10.1.3 Fa0/1 1 1
0
echo 114.0.1.1 N 192.10.2.3 Se1/0.1 1 1
0
echo 114.0.0.1 N 192.10.1.3 Fa0/1 1 0
0
echo 114.0.0.1 N 192.10.2.3 Se1/0.1 1 0
0

 
Although this completes our relatively simple scenario, I will like to come back to the concept of the parent route briefly covered earlier. As mentioned the MC will, by default, try to aggregate traffic into /24 prefixes (traffic classes). In order for the MC to instruct the BR to inject a new static route for a particular flow out an alternate external interface, a parent route needs to already exist pointing out the alternate external interface. In our scenario, we have a floating static route pointing out the Serial1/0.1 interface. Although this static route isn't in the routing table due to the higher administrative distance, it is still needed for PfR to use the alternate interface. PfR will need to verify that the interface is useable and reachability can be obtained across the alternate external interface. In our case, PfR used IP SLA because by default PfR operates in a prefix monitoring mode of what is termed as both. The term both is referring to both active and passive monitoring. Active monitoring means that PfR will generate test traffic (probes), ICMP ECHOs in our case, to verify reachability across the alternate path out the Serial1/0.1 interface. As we will see in future blog posts, we can specify the type of test traffic we want generated. For instance, if we are using PfR to optimize VoIP traffic in our internal network, we could use VoIP voice packets with IP SLA for the active probing.

So the point here is to remember that a parent route, which is defined as a route that is equal to or less specific than the monitored traffic class (MTC), is needed but does not actually have to be in the routing table when static routes are used. Just being in the global configuration is adequate to be considered a parent route.

Valid due to the fact a parent route does exist for the alternate external interface:

Rack1R3#show run | include ip route
ip route 0.0.0.0 0.0.0.0 192.10.1.1
ip route 0.0.0.0 0.0.0.0 192.10.2.1 5
Rack1R3#show ip route static
S* 0.0.0.0/0 [1/0] via 192.10.1.1
Rack1R3#

Invalid due to the fact a parent route does not exist for the alternate external interface:

Rack1R3#show run | include ip route
ip route 0.0.0.0 0.0.0.0 192.10.1.1
Rack1R3#show ip route static
S* 0.0.0.0/0 [1/0] via 192.10.1.1
Rack1R3#

 
Although this blog post my seem long there are a lot of details that were left out but will be covered in future posts on PfR. In the next blog post we will define our own traffic classes based upon the DSCP values and applications for PfR to monitor along with a PfR load balancing scenario.

Subscribe to INE Blog Updates