One of our students in the INE RS bootcamp today, asked about an OSPF sham-link. I thought it would make a beneficial addition to our blog, and here it is. Thanks for the request Christian!
Reader’s Digest version: MPLS networks aren’t free. If a customers is using OSPF to peer between the CE and PE routers, and also has an OSPF CE to CE neighborship, the CE’s will prefer the Intra-Area CE to CE routes (sometimes called the “backdoor” route in this situation), instead of using the Inter-Area CE to PE learned routes that use the MPLS network as a transit path. OSPF sham-links correct this behavior.
This blog post walks through the problem and the solution, including the configuration steps to create and verify a sham-link.
To begin, MPLS is set up in the network as shown with R2 and R4 acting as Provider Edge (PE) routers, and MPLS is enabled throughout R2-R3-R4.
R1 and R5 are Customer Edge (CE) routers, and the Serial0/1.15 interfaces of R1 and R5 are temporarily shut down, (this means the backdoor route isn’t in place yet, and at the moment, there is no problem).
Currently, R1 and R5 see the routes to each others local networks through the VPNv4 MPLS network, and the routes show up as Inter-Area OSPF routes with the PE routers as the next hop.
Let’s do some testing and verification of what is currently in place. Notice that R1 and R5 can see each others Fa0/0 and Fa0/1 connected networks. These routes show up as Inter-Area (IA) routes.
R1#show ip route ospf 10.0.0.0/24 is subnetted, 2 subnets O IA 10.45.0.0 [110/2] via 10.12.0.2, 00:00:58, FastEthernet0/0 O IA 192.168.1.0/24 [110/3] via 10.12.0.2, 00:00:43, FastEthernet0/0 R5#show ip route ospf 172.16.0.0/24 is subnetted, 1 subnets O IA 172.16.0.0 [110/3] via 10.45.0.4, 00:01:49, FastEthernet0/1 10.0.0.0/24 is subnetted, 2 subnets O IA 10.12.0.0 [110/2] via 10.45.0.4, 00:01:49, FastEthernet0/1
Next, we will enable the Serial0/1.15 interfaces of R1 and R5. When we enable these interfaces, R1 and R5 will become neighbors, and see each others routes to the Fa0/0 and Fa0/1 networks as Intra-Area routes. Even though the OSPF cost will be worse via the serial interfaces, take a close look at what happens and which routes end up in the routing table.
R1(config)#int ser 0/1.15 R1(config-subif)#no shut R5(config)#int ser 0/1.15 R5(config-subif)#no shut
We’ll wait a few moments, to give the network time to converge, then take a look at the OSPF routes on the CE routers R1 and R5, just as we did earlier, and see if the routes are different.
R1#show ip route ospf 10.0.0.0/24 is subnetted, 3 subnets O 10.45.0.0 [110/65] via 10.15.0.5, 00:02:52, Serial0/1.15 O 192.168.1.0/24 [110/65] via 10.15.0.5, 00:02:52, Serial0/1.15 R5#show ip route ospf 172.16.0.0/24 is subnetted, 1 subnets O 172.16.0.0 [110/65] via 10.15.0.1, 00:03:19, Serial0/1.15 10.0.0.0/24 is subnetted, 3 subnets O 10.12.0.0 [110/65] via 10.15.0.1, 00:03:19, Serial0/1.15
Notice, that the remote customer networks attached to Fa0/0 and Fa0/1 are now reachable via the serial 0/1.15 interface, and they appear as Intra-Area routes. Even though the metric of 65 is worse than before, and using the slower serial link, the routers prefer these routes instead of using the PE learned routes, because Intra-Area routes are preferred over Inter-Area routes. Now the Service Provider’s MPLS network will only be used as a backup in the event the serial connection fails. (I don’t think they will be providing a price break either). 
To train the network to use the MPLS network as the primary transit path, we need to make the remote Ethernet customer networks look like Intra-Area routes via the PE routers, with a better metric than the serial interfaces, so they can be used instead of the slower serial link. We are actually going to pull a fast one, or a “sham”, on OSPF because the MPLS network is really acting as a “superbackbone” for OSPF, and therefore routes between the CEs are indeed Inter-Area by default. To create the illusion of the CEs not being separated by a backbone, we will create an OSPF sham-link. We will create a couple loopback interfaces in the VRFs on both PEs, and make sure those loopbacks are originated and advertised via BGP. We will use those loopbacks as the source/destination of the OSPF sham-link.
Because the sham-link is seen as an Intra-Area link between PE routers (R2 and R4), an OSPF adjacency is created and database exchange takes place across the sham-link. The two PE routers can then flood LSAs between sites from across the MPLS VPN backbone. As a result, the desired Intra-Area routes are created.
Enough chat, lets create this sham-link!
R2(config)#int loop 100 R2(config-if)#ip vrf forwarding Vrf1 R2(config-if)#ip address 11.11.11.2 255.255.255.255 R2(config-if)#router bgp 24 R2(config-router)#address-family ipv4 vrf Vrf1 R2(config-router-af)#network 11.11.11.2 mask 255.255.255.255 R2(config-router-af)#exit R2(config-router)#router ospf 1 vrf Vrf1 R2(config-router)#area 1 sham-link 11.11.11.2 11.11.11.4 cost 5 R4(config)#int loop 100 R4(config-if)#ip vrf forwarding Vrf1 R4(config-if)#ip address 11.11.11.4 255.255.255.255 R4(config-if)#router bgp 24 R4(config-router)#address-family ipv4 vrf Vrf1 R4(config-router-af)#network 11.11.11.4 mask 255.255.255.255 R4(config-router-af)#exit R4(config-router)#router ospf 1 vrf Vrf1 R4(config-router)#area 1 sham-link 11.11.11.4 11.11.11.2 cost 5 %OSPF-5-ADJCHG: Process 1, Nbr 10.12.0.2 on OSPF_SL0 from LOADING to FULL, Loading Done
Looks like the sham-link came up. Let’s take a closer look at the sham link with a show command made just for that purpose.
R4#show ip ospf sham-links Sham Link OSPF_SL0 to address 11.11.11.2 is up Area 1 source address 11.11.11.4 Run as demand circuit DoNotAge LSA allowed. Cost of using 5 State POINT_TO_POINT, Timer intervals configured, Hello 10, Dead 40, Wait 40, Hello due in 00:00:06 Adjacency State FULL (Hello suppressed) Index 2/2, retransmission queue length 0, number of retransmission 0 First 0x0(0)/0x0(0) Next 0x0(0)/0x0(0) Last retransmission scan length is 0, maximum is 0 Last retransmission scan time is 0 msec, maximum is 0 msec
Looks like it is in place, but is it creating the desired result, of having the CE routers R1 and R5 see the Ethernet remote networks as reachable through the PE routers R2 and R4? Let’s go to R1 and see!
R1#show ip route ospf 10.0.0.0/24 is subnetted, 3 subnets O 10.45.0.0 [110/7] via 10.12.0.2, 00:06:02, FastEthernet0/0 11.0.0.0/32 is subnetted, 2 subnets O E2 11.11.11.2 [110/1] via 10.12.0.2, 00:06:43, FastEthernet0/0 O E2 11.11.11.4 [110/1] via 10.12.0.2, 00:06:13, FastEthernet0/0 O 192.168.1.0/24 [110/8] via 10.12.0.2, 00:06:02, FastEthernet0/0
That looks perfect! How about R5?
R5#show ip route ospf 172.16.0.0/24 is subnetted, 1 subnets O 172.16.0.0 [110/8] via 10.45.0.4, 00:06:27, FastEthernet0/1 10.0.0.0/24 is subnetted, 3 subnets O 10.12.0.0 [110/7] via 10.45.0.4, 00:06:27, FastEthernet0/1 11.0.0.0/32 is subnetted, 2 subnets O E2 11.11.11.2 [110/1] via 10.45.0.4, 00:07:05, FastEthernet0/1 O E2 11.11.11.4 [110/1] via 10.45.0.4, 00:06:45, FastEthernet0/1
And just to be sure, a ping to verify connectivity. We will ping the remote Fa0/1 interface of CE router R1 from CE router R5.
R5#ping 172.16.0.1 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 172.16.0.1, timeout is 2 seconds: !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 120/130/148 ms
That’s cool, so we know we have connectivity, and based on the routing table output, we believe it is going through the SP MPLS network. Let’s do one more test to prove that as well. A traceroute.
R5#trace 172.16.0.1 Type escape sequence to abort. Tracing the route to 172.16.0.1 1 10.45.0.4 48 msec 92 msec 12 msec 2 10.34.0.3 [MPLS: Labels 16/24 Exp 0] 136 msec 180 msec 228 msec 3 10.12.0.2 [MPLS: Label 24 Exp 0] 124 msec 80 msec 88 msec 4 10.12.0.1 112 msec * 176 msec
Tags and all! I still love it when a plan comes together. Now our transit traffic is moving through the MPLS network, and the serial 0/1.15 interfaces are available as a backup.
More fun times regarding MPLS, OSPF and MPBGP can be found in our workbooks for RS and SP.
Best wishes, and enjoy the journey!
About INE Instructor:
You can leave a response, or trackback from your own site.
33 Responses to “A Sham-Link, Really? Yes, and it’s not used for Phishing!”
Leave a Reply
Thank you again for wonderful article.
I have the same challenge to prefer External routes over Intra-Area routes in one OSPF domain. If all the routers were in area 0 and I were redistributing static routes from R2 to R4 and same route was learned via R5, it will prefer Intra-Area route via R5 than Extrenal route via R2.
Example: R2 redistributing 1.1.1.0/24 and sent to R4, it will treat as External route at the same time R5 propagate 1.1.1.0/24 to R4 as Inter-Area route, the R5 route will be prefered over R2. Can we change this preference, as Cisco states:
“According to section 11 of RFC 2328 leavingcisco.com, the order of preference for OSPF routes is:
intra-area routes, O
interarea routes, O IA
external routes type 1, O E1
external routes type 2, O E2
This rule of preference cannot be changed. However, it applies only within a single OSPF process.
Can this riddle be solved:)
Thanks and Regards,
Bilal Hansrod
Bilal-
Perhaps a couple of instances of OSPF, and differing ADs for each would do the trick. We could redistribute between the OSPF processes as needed.
Great post (although I already knew this). One thing though, bolding sentences like that really interrupts my reading flow. But maybe that’s just me.
ob- thanks for the feedback. I reduced the bold elements of the text, and appreciate you letting us know!
This was really interesting, learned few stuff thanks for the post guys!!
Hi Kieth,
Thank you for your post. I was just wondering, how flexible carriers are with configuring sham-links?
Thanks, you are right, it will work.
Nice one, however there is one thing I didn’t get:
how do routes get propagated through the MPLS network?
If the routes are redistributed from OSPF to BGP and vice-versa on the PEs, then the routes should show as External, not as Intra-Area?
Mat
I just have one question. I thought the OSPF Sham link is always configured over the MPLS network (OSPF PID 234 AREA0) and not over the backup link (PID 1 AREA 1). Can you clear that up for me please.
Thanks in advance.
Mat-
For the control plane, routes begin at the CE, are advertised to the PE (which has them in a VRF for that customer), are exported from that VRF to the BGP process (an export from that VRF), advertised via BGP to another PE, then imported from BGP into the VRF at the remote PE, and then advertised from the PE via OSPF to adjacent CE.
Looks like you are really thinking. Great work! When both CEs are using OSPF, the PE routers implement a modified version of the BGP/OSPF interaction. The end result is that if both CEs are in the same OSPF domain, the routes from one side, will be presented to the far side as OSPF inter-area routes using type 3 LSAs.
If you have a couple minutes, take a look at RFC 4577. I think you would enjoy it. If a person has a grasp of the fundamentals of BGP/MPLS, this RFC is very useful, and explains the modifications that are and can be done, if using OSPF is used on both PE-CE connections. It also explains the DN bit, sham-links and routing loop prevention as well.
Malick-
The sham link endpoint address is an address in the VPN’s address space, not the SP’s address space. The endpoints also need to be distributed by BGP as a VPN-IPv4 address that is 32 bits. The same endpoint addresses can’t be originated from OSPF.
So in short, the providers OSPF 234 is advertising 5 routes only:
Loopback of 2, 3 and 4 use for LDP and R2-R4 iBGP
Link between 2-3
Link between 3-4
The sham-link network was added to BGP within the address-family of our VRF, which puts the endpoint for the sham-link in our VPN address space.
Let me know if you have other questions, and thanks for your participation!
Thanks.
I did not know there was a specific RFC for using OSPF as a PE-CE protocol. It is indeed interesting to see the superbackbone concept, and all the “tweaks” to OSPF that are needed.
In term of implementation/configuration, I didn’t find any document that shows what is needed… is it all “automagic” in the cisco world?
Cheers,
Mat
Mat- Regarding implementation, I used the info shown in the diagram, with standard MPLS/BGP configuration for this type of scenario without any special commands to make the routes appear as IA routes originally. “automagic” is about right.
How did guys know that Sham link is added to the configuration section of the exam?
Steav- all the more convenient to know! Hope you are enjoying the blog.
Hi,
It is stated that the endpoints should not be advertised by OSPF. Since most of the time BGP routes are redistributed into OSPF in MPLS VPN scenarios, would using route-map to filter the /32 prefixes be the easiest way?
Ted
Ted- yes. A route-map could do the trick. I recommend you practice that, observe the results to verify it is working correctly. Best wishes on your lab date in the next few months!
To Steve re: a service provider’s flexibility in doing sham links–I’d say if the SP is willing to peer OSPF to you, they should be willing to put in sham links. But they shouldn’t peer OSPF to you in the first place–peering between AS’s is what BGP is for. With an AD of 20, eBGP will be your preferred route to the backdoor networks. Sham links in this situation are a band aid, not good design…in my ever so humble opinion.
great~~~
“The end result is that if both CEs are in the same OSPF domain, the routes from one side, will be presented to the far side as OSPF inter-area routes using type 3 LSAs. ”
Is this a typo? Type 3 LSA are summary routes, no? so why is it advertised as a type 3 LSA?
Reading RFC 4577 it’s basically saying becuase both PE-CE are in the same OSPF domain the special case (sham-link) comes into effect, correct?
Baloo- The MPLS acts as a “super-backbone” for OSPF. Without the sham-link, if the OSPF domain-id is the same on both PE routers, the advertisements to the CEs will be LSA type 3s (no typo), and those far side CE routes will show up on the other side as IntER-area routes.
(Side note: If we changed the OSPF domain-id on one of the PE routers, the routes would then begin to show up on the CE routers as External routes.)
The Inter-Area routes are pretty cool, but they would lose out to a backdoor connection that is advertising those same route(s) as intRA-area routes. That is where the sham link comes in, we can use a sham link to make the MPLS/BGP/OSPF routes appear as intRA-area routes, and then compete with backdoor routes.
Here is a great link that can provides even greater detail: https://www.racf.bnl.gov/Facility/TechnologyMeeting/Archive/06-30-04-CISCO/Using-OSPF-in-MPLS-VPN-Environment.pdf
Thanks for the question. The best thing you can do for yourself, would be to lab this up and take a look!
Best wishes.
Ted-
Had a chance to visit another MPLS config, and added the filter to stop the CEs from seeing the endpoints of the sham link.
conf t
ip prefix-list sham-end seq 20 permit 66.66.66.66/32
ip prefix-list sham-end seq 30 permit 44.44.44.44/32
route-map deny-sham-endpoints deny 10
match ip address prefix-list sham-end
exit
route-map deny-sham-endpoints permit 20
exit
router ospf 1 vrf VPN_CustA
no redistribute bgp 46 subnets
redistribute bgp 46 subnets route-map deny-sham-endpoints
end
best wishes.
Hi,
Can a ospf sham link be formed with a loop back interface of /24 subnet. All the examples i saw has the loopback address as /32.
Please suggest.
I tried the same topology in dynamips, but instead of serial form CE to CE i used Fast ethernet and i created a loopback at CE2 and CE1 instead of physical interface
after sham link creation
My sham link was up and working fine
but when i trace it from CE2 to CE1 and vice cersa it uses the CE-to-CE link
not MPLS backbone
and when i changed the bandwidth on phyisical interface CE-to CE
it traces route form MPLS back bone
so does it require the smaller bandwitdh on CE to CE for using the MPLS backbone
http://prakashkalsaria.wordpress.com
Thanks for the post.
What if from one side we have PE-CE with OSPF area different from the area on the other side? Will this solution work?
Thanks
Mo –
If the R1 CE was using OSPF area 9, and CE R5 was using OSPF area 1, the MPLS backbone would allow for connectivity between the CEs, so that part would work. What wouldn’t be needed is the sham-link, as R1 and R5 are in separate areas for the backdoor, would never build and OSPF adjacency.
R1#show ip ospf int brief
Interface PID Area IP Address/Mask Cost State Nbrs F/C
Se0/1.15 1 9 10.15.0.1/24 64 P2P 0/0
Fa0/1 1 9 172.16.0.1/24 1 DR 0/0
Fa0/0 1 9 10.12.0.1/24 1 BDR 1/1
Lo0 1 9 1.1.1.1/24 1 LOOP 0/0
R1#show ip route ospf | begin 1
5.0.0.0/32 is subnetted, 1 subnets
O IA 5.5.5.5 [110/3] via 10.12.0.2, 00:06:25, FastEthernet0/0
10.0.0.0/24 is subnetted, 3 subnets
O IA 10.45.0.0 [110/2] via 10.12.0.2, 00:06:25, FastEthernet0/0
O IA 192.168.1.0/24 [110/3] via 10.12.0.2, 00:06:25, FastEthernet0/0
R1#
R5#show ip ospf int brief
Interface PID Area IP Address/Mask Cost State Nbrs F/C
Se0/1.15 1 1 10.15.0.5/24 64 P2P 0/0
Fa0/0 1 1 192.168.1.5/24 1 DR 0/0
Fa0/1 1 1 10.45.0.5/24 1 BDR 1/1
Lo0 1 1 5.5.5.5/24 1 LOOP 0/0
R5#show ip route ospf | begin 1
1.0.0.0/32 is subnetted, 1 subnets
O IA 1.1.1.1 [110/3] via 10.45.0.4, 00:05:24, FastEthernet0/1
172.16.0.0/24 is subnetted, 1 subnets
O IA 172.16.0.0 [110/3] via 10.45.0.4, 00:05:24, FastEthernet0/1
10.0.0.0/24 is subnetted, 3 subnets
O IA 10.12.0.0 [110/2] via 10.45.0.4, 00:05:24, FastEthernet0/1
R5#
Best wishes,
paks kal -
Once the CE routers see the MPLS backbone routes as OSPF intra-area routes, it is then up to cost. So we would want to make sure the MPLS OSPF routes appear as a better cost, than the backdoor OSPF routes.
Thanks for great explanation,
What if from one side we have PE-CE with OSPF area different from the area on the other side? But still I must prefer MPLS backbone?
Can I use sham-link even CEs in deffirent areas?
I understand from RFC 4577 : CEs in the same area prerequest to use sham-link.
There is a requirement to allow only OSPF intra and inter area routes throug backbone link. ON PE, we redistributed ospf math only internal routes, but sham-link leak that policy and on CE routes can be seen through backbone link but not reachable as we are not redistributing into MBGP,
There is a requirement to allow only OSPF intra and inter area routes throug backbone link. ON PE, we redistributed ospf math only internal routes, but sham-link leak that policy and on CE routes can be seen through backbone link but not reachable as we are not redistributing into MBGP, How can we filter LSA5 in that case.
In this scenario, if we reload R2 or R4, the sham-link will go down and stay down. It will be up only after we reenter the command “area x sham-link…..”. After researching a little bit, i wasnt able to find a workaround. Is there a solution for it?!?
Thanks!
hi i configured the ospf Sham -link lab but it did no work can some please help me out here is my config
topology ce>pe1>p>pe2>ce2 | ce also have connectivity to ce
pe1 :
interface Loopback0
ip address 2.2.2.2 255.255.255.255
!
interface Loopback10
ip vrf forwarding abc
ip address 192.168.60.9 255.255.255.255
!
interface FastEthernet0/0
ip vrf forwarding abc
ip address 192.168.60.1 255.255.255.252
duplex full
speed 100
!
interface FastEthernet0/1
ip address 150.50.50.1 255.255.255.0
duplex full
speed 100
mpls ip
!
router ospf 2 vrf abc
log-adjacency-changes
area 1 sham-link 192.168.60.9 192.168.60.10 cost 5
redistribute bgp 65000 subnets
network 192.168.60.0 0.0.0.3 area 1
!
router ospf 1
router-id 20.20.20.1
log-adjacency-changes
redistribute bgp 65000 subnets
network 2.2.2.0 0.0.0.255 area 0
network 150.50.50.0 0.0.0.255 area 0
!
router bgp 65000
no synchronization
bgp log-neighbor-changes
neighbor 4.4.4.4 remote-as 65000
neighbor 4.4.4.4 update-source Loopback0
no auto-summary
!
address-family vpnv4
neighbor 4.4.4.4 activate
neighbor 4.4.4.4 send-community both
neighbor 4.4.4.4 next-hop-self
exit-address-family
!
address-family ipv4 vrf abc
redistribute ospf 2 vrf abc
neighbor 4.4.4.4 remote-as 65000
neighbor 4.4.4.4 activate
neighbor 4.4.4.4 send-community both
no synchronization
network 192.168.60.8 mask 255.255.255.255
exit-address-family
!
no ip http server
pe2
interface Loopback0
ip address 4.4.4.4 255.255.255.255
!
interface Loopback10
ip vrf forwarding abc
ip address 192.168.60.10 255.255.255.252
!
interface FastEthernet0/0
ip address 160.60.60.2 255.255.255.0
duplex full
speed 100
mpls ip
!
interface FastEthernet0/1
ip vrf forwarding abc
ip address 192.168.60.6 255.255.255.252
duplex full
speed 100
!
router ospf 2 vrf abc
log-adjacency-changes
area 1 sham-link 192.168.60.10 192.168.60.9 cost 5
redistribute bgp 65000 subnets
network 192.168.60.4 0.0.0.3 area 1
!
router ospf 1
router-id 40.40.40.1
log-adjacency-changes
redistribute bgp 65000 subnets
network 4.4.4.0 0.0.0.255 area 0
network 160.60.60.0 0.0.0.255 area 0
!
router bgp 65000
no synchronization
bgp log-neighbor-changes
neighbor 2.2.2.2 remote-as 65000
neighbor 2.2.2.2 update-source Loopback0
no auto-summary
!
address-family vpnv4
neighbor 2.2.2.2 activate
neighbor 2.2.2.2 send-community both
neighbor 2.2.2.2 next-hop-self
exit-address-family
!
address-family ipv4 vrf abc
redistribute ospf 2 vrf abc
neighbor 2.2.2.2 remote-as 65000
neighbor 2.2.2.2 activate
neighbor 2.2.2.2 send-community both
no synchronization
network 192.168.60.8 mask 255.255.255.252
exit-address-family
address-family ipv4 vrf abc
redistribute ospf 2 vrf abc
neighbor 4.4.4.4 remote-as 65000
neighbor 4.4.4.4 activate
neighbor 4.4.4.4 send-community both
no synchronization
network 192.168.60.8 mask 255.255.255.255
the last line is wrong:
it should be network 192.168.60.9 mask 255.255.255.255