Apr
08

One of our students in the INE RS bootcamp today, asked about an OSPF sham-link. I thought it would make a beneficial addition to our blog, and here it is.  Thanks for the request Christian!

Reader’s Digest version: MPLS networks aren’t free. If a customers is using OSPF to peer between the CE and PE routers, and also has an OSPF CE to CE neighborship, the CE’s will prefer the Intra-Area CE to CE routes (sometimes called the “backdoor” route in this situation), instead of using the Inter-Area CE to PE learned routes that use the MPLS network as a transit path. OSPF sham-links correct this behavior.

This blog post walks through the problem and the solution, including the configuration steps to create and verify a sham-link.

To begin, MPLS is set up in the network as shown with R2 and R4 acting as Provider Edge (PE) routers, and MPLS is enabled throughout R2-R3-R4.

R1 and R5 are Customer Edge (CE) routers, and the Serial0/1.15 interfaces of R1 and R5 are temporarily shut down, (this means the backdoor route isn’t in place yet, and at the moment, there is no problem).

mpls-ospf sham

Currently, R1 and R5 see the routes to each others local networks through the VPNv4 MPLS network, and the routes show up as Inter-Area OSPF routes with the PE routers as the next hop.

Let’s do some testing and verification of what is currently in place. Notice that R1 and R5 can see each others Fa0/0 and Fa0/1 connected networks. These routes show up as Inter-Area (IA) routes.

R1#show ip route ospf
10.0.0.0/24 is subnetted, 2 subnets
O IA 10.45.0.0 [110/2] via 10.12.0.2, 00:00:58, FastEthernet0/0 O IA 192.168.1.0/24 [110/3] via 10.12.0.2, 00:00:43, FastEthernet0/0

R5#show ip route ospf
172.16.0.0/24 is subnetted, 1 subnets
O IA 172.16.0.0 [110/3] via 10.45.0.4, 00:01:49, FastEthernet0/1
10.0.0.0/24 is subnetted, 2 subnets
O IA 10.12.0.0 [110/2] via 10.45.0.4, 00:01:49, FastEthernet0/1

Next, we will enable the Serial0/1.15 interfaces of R1 and R5. When we enable these interfaces, R1 and R5 will become neighbors, and see each others routes to the Fa0/0 and Fa0/1 networks as Intra-Area routes. Even though the OSPF cost will be worse via the serial interfaces, take a close look at what happens and which routes end up in the routing table.

R1(config)#int ser 0/1.15
R1(config-subif)#no shut

R5(config)#int ser 0/1.15
R5(config-subif)#no shut

We’ll wait a few moments, to give the network  time to converge, then take a look at the OSPF routes on the CE routers R1 and R5, just as we did earlier, and see if the routes are different.

R1#show ip route ospf
10.0.0.0/24 is subnetted, 3 subnets
O 10.45.0.0 [110/65] via 10.15.0.5, 00:02:52, Serial0/1.15 O 192.168.1.0/24 [110/65] via 10.15.0.5, 00:02:52, Serial0/1.15

R5#show ip route ospf
172.16.0.0/24 is subnetted, 1 subnets
O 172.16.0.0 [110/65] via 10.15.0.1, 00:03:19, Serial0/1.15
10.0.0.0/24 is subnetted, 3 subnets
O 10.12.0.0 [110/65] via 10.15.0.1, 00:03:19, Serial0/1.15

Notice, that the remote customer networks attached to Fa0/0 and Fa0/1 are now reachable via the serial 0/1.15 interface, and they appear as Intra-Area routes. Even though the metric of 65 is worse than before, and using the slower serial link, the routers prefer these routes instead of using the PE learned routes, because Intra-Area routes are preferred over  Inter-Area routes. Now the Service Provider’s MPLS network will only be used as a backup in the event the serial connection fails. (I don’t think they will be providing a price break either). ;)

To train the network to use the MPLS network as the primary transit path, we need to make the remote Ethernet customer networks look like Intra-Area routes via the PE routers, with a better metric than the serial interfaces, so they can be used instead of the slower serial link. We are actually going to pull a fast one, or a “sham”, on OSPF because the MPLS network is really acting as a “superbackbone” for OSPF, and therefore routes between the CEs are indeed Inter-Area by default. To create the illusion of the CEs not being separated by a backbone, we will create an OSPF sham-link. We will create a couple loopback interfaces in the VRFs on both PEs, and make sure those loopbacks are originated and advertised via BGP. We will use those loopbacks as the source/destination of the OSPF sham-link.

Because the sham-link is seen as an Intra-Area link between PE routers (R2 and R4), an OSPF adjacency is created and database exchange takes place across the sham-link. The two PE routers can then flood LSAs between sites from across the MPLS VPN backbone. As a result, the desired Intra-Area routes are created.

Enough chat, lets create this sham-link!

R2(config)#int loop 100
R2(config-if)#ip vrf forwarding Vrf1
R2(config-if)#ip address 11.11.11.2 255.255.255.255
R2(config-if)#router bgp 24
R2(config-router)#address-family ipv4 vrf Vrf1
R2(config-router-af)#network 11.11.11.2 mask 255.255.255.255
R2(config-router-af)#exit
R2(config-router)#router ospf 1 vrf Vrf1
R2(config-router)#area 1 sham-link 11.11.11.2 11.11.11.4 cost 5

R4(config)#int loop 100
R4(config-if)#ip vrf forwarding Vrf1
R4(config-if)#ip address 11.11.11.4 255.255.255.255
R4(config-if)#router bgp 24
R4(config-router)#address-family ipv4 vrf Vrf1
R4(config-router-af)#network 11.11.11.4 mask 255.255.255.255
R4(config-router-af)#exit
R4(config-router)#router ospf 1 vrf Vrf1
R4(config-router)#area 1 sham-link 11.11.11.4 11.11.11.2 cost 5
%OSPF-5-ADJCHG: Process 1, Nbr 10.12.0.2 on OSPF_SL0 from LOADING to FULL, Loading Done

Looks like the sham-link came up.  Let’s take a closer look at the sham link with a show command made just for that purpose.

R4#show ip ospf sham-links
Sham Link OSPF_SL0 to address 11.11.11.2 is up
Area 1 source address 11.11.11.4
Run as demand circuit
DoNotAge LSA allowed. Cost of using 5 State POINT_TO_POINT,
Timer intervals configured, Hello 10, Dead 40, Wait 40,
Hello due in 00:00:06
Adjacency State FULL (Hello suppressed)
Index 2/2, retransmission queue length 0, number of retransmission 0
First 0x0(0)/0x0(0) Next 0x0(0)/0x0(0)
Last retransmission scan length is 0, maximum is 0
Last retransmission scan time is 0 msec, maximum is 0 msec

Looks like it is in place, but is it creating the desired result, of having the CE routers R1 and R5 see the Ethernet remote networks as reachable through the PE routers R2 and R4? Let’s go to R1 and see!

R1#show ip route ospf
10.0.0.0/24 is subnetted, 3 subnets
O 10.45.0.0 [110/7] via 10.12.0.2, 00:06:02, FastEthernet0/0
11.0.0.0/32 is subnetted, 2 subnets
O E2    11.11.11.2 [110/1] via 10.12.0.2, 00:06:43, FastEthernet0/0
O E2    11.11.11.4 [110/1] via 10.12.0.2, 00:06:13, FastEthernet0/0
O 192.168.1.0/24 [110/8] via 10.12.0.2, 00:06:02, FastEthernet0/0

That looks perfect! How about R5?

R5#show ip route ospf
172.16.0.0/24 is subnetted, 1 subnets
O 172.16.0.0 [110/8] via 10.45.0.4, 00:06:27, FastEthernet0/1
10.0.0.0/24 is subnetted, 3 subnets
O 10.12.0.0 [110/7] via 10.45.0.4, 00:06:27, FastEthernet0/1
11.0.0.0/32 is subnetted, 2 subnets
O E2    11.11.11.2 [110/1] via 10.45.0.4, 00:07:05, FastEthernet0/1
O E2    11.11.11.4 [110/1] via 10.45.0.4, 00:06:45, FastEthernet0/1

And just to be sure, a ping to verify connectivity. We will ping the remote Fa0/1 interface of CE router R1 from CE router R5.

R5#ping 172.16.0.1

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 172.16.0.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 120/130/148 ms

That’s cool, so we know we have connectivity, and based on the routing table output, we believe it is going through the SP MPLS network. Let’s do one more test to prove that as well. A traceroute.

R5#trace 172.16.0.1

Type escape sequence to abort.
Tracing the route to 172.16.0.1

1 10.45.0.4 48 msec 92 msec 12 msec 2 10.34.0.3 [MPLS: Labels 16/24 Exp 0] 136 msec 180 msec 228 msec 3 10.12.0.2 [MPLS: Label 24 Exp 0] 124 msec 80 msec 88 msec 4 10.12.0.1 112 msec * 176 msec

Tags and all!  I still love it when a plan comes together.   Now our transit traffic is moving through the MPLS network, and the serial 0/1.15 interfaces are available as a backup.

More fun times regarding MPLS, OSPF and MPBGP can be found in our workbooks for RS and SP.

Best wishes, and enjoy the journey!


You can leave a response, or trackback from your own site.

33 Responses to “A Sham-Link, Really? Yes, and it’s not used for Phishing!”

 
  1. Bilal Hansrod says:

    Thank you again for wonderful article.

    I have the same challenge to prefer External routes over Intra-Area routes in one OSPF domain. If all the routers were in area 0 and I were redistributing static routes from R2 to R4 and same route was learned via R5, it will prefer Intra-Area route via R5 than Extrenal route via R2.

    Example: R2 redistributing 1.1.1.0/24 and sent to R4, it will treat as External route at the same time R5 propagate 1.1.1.0/24 to R4 as Inter-Area route, the R5 route will be prefered over R2. Can we change this preference, as Cisco states:
    “According to section 11 of RFC 2328 leavingcisco.com, the order of preference for OSPF routes is:

    intra-area routes, O
    interarea routes, O IA
    external routes type 1, O E1
    external routes type 2, O E2

    This rule of preference cannot be changed. However, it applies only within a single OSPF process.

    Can this riddle be solved:)

    Thanks and Regards,

    Bilal Hansrod

  2. ob says:

    Great post (although I already knew this). One thing though, bolding sentences like that really interrupts my reading flow. But maybe that’s just me.

  3. Mathew Abraham says:

    This was really interesting, learned few stuff thanks for the post guys!!

  4. Steve says:

    Hi Kieth,

    Thank you for your post. I was just wondering, how flexible carriers are with configuring sham-links?

  5. Bilal Hansrod says:

    Thanks, you are right, it will work.

  6. Mat says:

    Nice one, however there is one thing I didn’t get:

    how do routes get propagated through the MPLS network?

    If the routes are redistributed from OSPF to BGP and vice-versa on the PEs, then the routes should show as External, not as Intra-Area?

    Mat

  7. Malick Ndiaye says:

    I just have one question. I thought the OSPF Sham link is always configured over the MPLS network (OSPF PID 234 AREA0) and not over the backup link (PID 1 AREA 1). Can you clear that up for me please.

    Thanks in advance.

  8. Mat-

    For the control plane, routes begin at the CE, are advertised to the PE (which has them in a VRF for that customer), are exported from that VRF to the BGP process (an export from that VRF), advertised via BGP to another PE, then imported from BGP into the VRF at the remote PE, and then advertised from the PE via OSPF to adjacent CE.
    Looks like you are really thinking. Great work! When both CEs are using OSPF, the PE routers implement a modified version of the BGP/OSPF interaction. The end result is that if both CEs are in the same OSPF domain, the routes from one side, will be presented to the far side as OSPF inter-area routes using type 3 LSAs.

    If you have a couple minutes, take a look at RFC 4577. I think you would enjoy it. If a person has a grasp of the fundamentals of BGP/MPLS, this RFC is very useful, and explains the modifications that are and can be done, if using OSPF is used on both PE-CE connections. It also explains the DN bit, sham-links and routing loop prevention as well.

  9. Malick-

    The sham link endpoint address is an address in the VPN’s address space, not the SP’s address space. The endpoints also need to be distributed by BGP as a VPN-IPv4 address that is 32 bits. The same endpoint addresses can’t be originated from OSPF.

    So in short, the providers OSPF 234 is advertising 5 routes only:
    Loopback of 2, 3 and 4 use for LDP and R2-R4 iBGP
    Link between 2-3
    Link between 3-4

    The sham-link network was added to BGP within the address-family of our VRF, which puts the endpoint for the sham-link in our VPN address space.

    Let me know if you have other questions, and thanks for your participation!

  10. Mat says:

    Thanks.

    I did not know there was a specific RFC for using OSPF as a PE-CE protocol. It is indeed interesting to see the superbackbone concept, and all the “tweaks” to OSPF that are needed.
    In term of implementation/configuration, I didn’t find any document that shows what is needed… is it all “automagic” in the cisco world?

    Cheers,
    Mat

  11. Mat- Regarding implementation, I used the info shown in the diagram, with standard MPLS/BGP configuration for this type of scenario without any special commands to make the routes appear as IA routes originally. “automagic” is about right.

  12. Steav says:

    How did guys know that Sham link is added to the configuration section of the exam? :)

  13. Steav- all the more convenient to know! Hope you are enjoying the blog.

  14. Ted says:

    Hi,

    It is stated that the endpoints should not be advertised by OSPF. Since most of the time BGP routes are redistributed into OSPF in MPLS VPN scenarios, would using route-map to filter the /32 prefixes be the easiest way?

    Ted

  15. Ted- yes. A route-map could do the trick. I recommend you practice that, observe the results to verify it is working correctly. Best wishes on your lab date in the next few months!

  16. A Different Steve says:

    To Steve re: a service provider’s flexibility in doing sham links–I’d say if the SP is willing to peer OSPF to you, they should be willing to put in sham links. But they shouldn’t peer OSPF to you in the first place–peering between AS’s is what BGP is for. With an AD of 20, eBGP will be your preferred route to the backdoor networks. Sham links in this situation are a band aid, not good design…in my ever so humble opinion. ;)

  17. Honzhu says:

    great~~~

  18. baloo says:

    “The end result is that if both CEs are in the same OSPF domain, the routes from one side, will be presented to the far side as OSPF inter-area routes using type 3 LSAs. ”

    Is this a typo? Type 3 LSA are summary routes, no? so why is it advertised as a type 3 LSA?

    Reading RFC 4577 it’s basically saying becuase both PE-CE are in the same OSPF domain the special case (sham-link) comes into effect, correct?

  19. Baloo- The MPLS acts as a “super-backbone” for OSPF. Without the sham-link, if the OSPF domain-id is the same on both PE routers, the advertisements to the CEs will be LSA type 3s (no typo), and those far side CE routes will show up on the other side as IntER-area routes.

    (Side note: If we changed the OSPF domain-id on one of the PE routers, the routes would then begin to show up on the CE routers as External routes.)

    The Inter-Area routes are pretty cool, but they would lose out to a backdoor connection that is advertising those same route(s) as intRA-area routes. That is where the sham link comes in, we can use a sham link to make the MPLS/BGP/OSPF routes appear as intRA-area routes, and then compete with backdoor routes.

    Here is a great link that can provides even greater detail: https://www.racf.bnl.gov/Facility/TechnologyMeeting/Archive/06-30-04-CISCO/Using-OSPF-in-MPLS-VPN-Environment.pdf

    Thanks for the question. The best thing you can do for yourself, would be to lab this up and take a look!

    Best wishes.

  20. Ted-
    Had a chance to visit another MPLS config, and added the filter to stop the CEs from seeing the endpoints of the sham link.

    conf t

    ip prefix-list sham-end seq 20 permit 66.66.66.66/32
    ip prefix-list sham-end seq 30 permit 44.44.44.44/32

    route-map deny-sham-endpoints deny 10
    match ip address prefix-list sham-end
    exit

    route-map deny-sham-endpoints permit 20
    exit

    router ospf 1 vrf VPN_CustA
    no redistribute bgp 46 subnets
    redistribute bgp 46 subnets route-map deny-sham-endpoints
    end

    best wishes.

  21. Tam says:

    Hi,

    Can a ospf sham link be formed with a loop back interface of /24 subnet. All the examples i saw has the loopback address as /32.

    Please suggest.

  22. paks kal says:

    I tried the same topology in dynamips, but instead of serial form CE to CE i used Fast ethernet and i created a loopback at CE2 and CE1 instead of physical interface
    after sham link creation
    My sham link was up and working fine

    but when i trace it from CE2 to CE1 and vice cersa it uses the CE-to-CE link

    not MPLS backbone

    and when i changed the bandwidth on phyisical interface CE-to CE
    it traces route form MPLS back bone

    so does it require the smaller bandwitdh on CE to CE for using the MPLS backbone

    http://prakashkalsaria.wordpress.com

  23. MO says:

    Thanks for the post.

    What if from one side we have PE-CE with OSPF area different from the area on the other side? Will this solution work?

    Thanks

    • Mo –

      If the R1 CE was using OSPF area 9, and CE R5 was using OSPF area 1, the MPLS backbone would allow for connectivity between the CEs, so that part would work. What wouldn’t be needed is the sham-link, as R1 and R5 are in separate areas for the backdoor, would never build and OSPF adjacency.

      R1#show ip ospf int brief
      Interface PID Area IP Address/Mask Cost State Nbrs F/C
      Se0/1.15 1 9 10.15.0.1/24 64 P2P 0/0
      Fa0/1 1 9 172.16.0.1/24 1 DR 0/0
      Fa0/0 1 9 10.12.0.1/24 1 BDR 1/1
      Lo0 1 9 1.1.1.1/24 1 LOOP 0/0

      R1#show ip route ospf | begin 1
      5.0.0.0/32 is subnetted, 1 subnets
      O IA 5.5.5.5 [110/3] via 10.12.0.2, 00:06:25, FastEthernet0/0
      10.0.0.0/24 is subnetted, 3 subnets
      O IA 10.45.0.0 [110/2] via 10.12.0.2, 00:06:25, FastEthernet0/0
      O IA 192.168.1.0/24 [110/3] via 10.12.0.2, 00:06:25, FastEthernet0/0
      R1#

      R5#show ip ospf int brief
      Interface PID Area IP Address/Mask Cost State Nbrs F/C
      Se0/1.15 1 1 10.15.0.5/24 64 P2P 0/0
      Fa0/0 1 1 192.168.1.5/24 1 DR 0/0
      Fa0/1 1 1 10.45.0.5/24 1 BDR 1/1
      Lo0 1 1 5.5.5.5/24 1 LOOP 0/0

      R5#show ip route ospf | begin 1
      1.0.0.0/32 is subnetted, 1 subnets
      O IA 1.1.1.1 [110/3] via 10.45.0.4, 00:05:24, FastEthernet0/1
      172.16.0.0/24 is subnetted, 1 subnets
      O IA 172.16.0.0 [110/3] via 10.45.0.4, 00:05:24, FastEthernet0/1
      10.0.0.0/24 is subnetted, 3 subnets
      O IA 10.12.0.0 [110/2] via 10.45.0.4, 00:05:24, FastEthernet0/1
      R5#

      Best wishes,

  24. paks kal -

    Once the CE routers see the MPLS backbone routes as OSPF intra-area routes, it is then up to cost. So we would want to make sure the MPLS OSPF routes appear as a better cost, than the backdoor OSPF routes.

  25. TechnologyBoy says:

    Thanks for great explanation,

    What if from one side we have PE-CE with OSPF area different from the area on the other side? But still I must prefer MPLS backbone?
    Can I use sham-link even CEs in deffirent areas?
    I understand from RFC 4577 : CEs in the same area prerequest to use sham-link.

  26. Ahmad says:

    There is a requirement to allow only OSPF intra and inter area routes throug backbone link. ON PE, we redistributed ospf math only internal routes, but sham-link leak that policy and on CE routes can be seen through backbone link but not reachable as we are not redistributing into MBGP,

  27. Ahmad says:

    There is a requirement to allow only OSPF intra and inter area routes throug backbone link. ON PE, we redistributed ospf math only internal routes, but sham-link leak that policy and on CE routes can be seen through backbone link but not reachable as we are not redistributing into MBGP, How can we filter LSA5 in that case.

  28. Daniel Valente says:

    In this scenario, if we reload R2 or R4, the sham-link will go down and stay down. It will be up only after we reenter the command “area x sham-link…..”. After researching a little bit, i wasnt able to find a workaround. Is there a solution for it?!?
    Thanks!

  29. dan says:

    hi i configured the ospf Sham -link lab but it did no work can some please help me out here is my config

    topology ce>pe1>p>pe2>ce2 | ce also have connectivity to ce

    pe1 :

    interface Loopback0
    ip address 2.2.2.2 255.255.255.255
    !
    interface Loopback10
    ip vrf forwarding abc
    ip address 192.168.60.9 255.255.255.255
    !
    interface FastEthernet0/0
    ip vrf forwarding abc
    ip address 192.168.60.1 255.255.255.252
    duplex full
    speed 100
    !
    interface FastEthernet0/1
    ip address 150.50.50.1 255.255.255.0
    duplex full
    speed 100
    mpls ip
    !
    router ospf 2 vrf abc
    log-adjacency-changes
    area 1 sham-link 192.168.60.9 192.168.60.10 cost 5
    redistribute bgp 65000 subnets
    network 192.168.60.0 0.0.0.3 area 1
    !
    router ospf 1
    router-id 20.20.20.1
    log-adjacency-changes
    redistribute bgp 65000 subnets
    network 2.2.2.0 0.0.0.255 area 0
    network 150.50.50.0 0.0.0.255 area 0
    !
    router bgp 65000
    no synchronization
    bgp log-neighbor-changes
    neighbor 4.4.4.4 remote-as 65000
    neighbor 4.4.4.4 update-source Loopback0
    no auto-summary
    !
    address-family vpnv4
    neighbor 4.4.4.4 activate
    neighbor 4.4.4.4 send-community both
    neighbor 4.4.4.4 next-hop-self
    exit-address-family
    !
    address-family ipv4 vrf abc
    redistribute ospf 2 vrf abc
    neighbor 4.4.4.4 remote-as 65000
    neighbor 4.4.4.4 activate
    neighbor 4.4.4.4 send-community both
    no synchronization
    network 192.168.60.8 mask 255.255.255.255
    exit-address-family
    !
    no ip http server

    pe2

    interface Loopback0
    ip address 4.4.4.4 255.255.255.255
    !
    interface Loopback10
    ip vrf forwarding abc
    ip address 192.168.60.10 255.255.255.252
    !
    interface FastEthernet0/0
    ip address 160.60.60.2 255.255.255.0
    duplex full
    speed 100
    mpls ip
    !
    interface FastEthernet0/1
    ip vrf forwarding abc
    ip address 192.168.60.6 255.255.255.252
    duplex full
    speed 100
    !
    router ospf 2 vrf abc
    log-adjacency-changes
    area 1 sham-link 192.168.60.10 192.168.60.9 cost 5
    redistribute bgp 65000 subnets
    network 192.168.60.4 0.0.0.3 area 1
    !
    router ospf 1
    router-id 40.40.40.1
    log-adjacency-changes
    redistribute bgp 65000 subnets
    network 4.4.4.0 0.0.0.255 area 0
    network 160.60.60.0 0.0.0.255 area 0
    !
    router bgp 65000
    no synchronization
    bgp log-neighbor-changes
    neighbor 2.2.2.2 remote-as 65000
    neighbor 2.2.2.2 update-source Loopback0
    no auto-summary
    !
    address-family vpnv4
    neighbor 2.2.2.2 activate
    neighbor 2.2.2.2 send-community both
    neighbor 2.2.2.2 next-hop-self
    exit-address-family
    !
    address-family ipv4 vrf abc
    redistribute ospf 2 vrf abc
    neighbor 2.2.2.2 remote-as 65000
    neighbor 2.2.2.2 activate
    neighbor 2.2.2.2 send-community both
    no synchronization
    network 192.168.60.8 mask 255.255.255.252
    exit-address-family

    • Mustafa Nassef says:

      address-family ipv4 vrf abc
      redistribute ospf 2 vrf abc
      neighbor 4.4.4.4 remote-as 65000
      neighbor 4.4.4.4 activate
      neighbor 4.4.4.4 send-community both
      no synchronization

      network 192.168.60.8 mask 255.255.255.255

      the last line is wrong:
      it should be network 192.168.60.9 mask 255.255.255.255

 

Leave a Reply

Categories

CCIE Bloggers