May
25

It isn’t my fault, they configured it that way before I got here! That was the entry level technician’s story Monday morning, and he was sticking to it.  :)

Here is the rest of the story.   Over the weekend, some testing had been done regarding a proposed BGP configuration.   The objective was simple, R1 and R3 needed to ping each others loobacks at 1.1.1.1 and 3.3.3.3 respectively, with those 2 networks, being carried by BGP.  R2 is performing NAT.    The topology diagram looks like this:

3 routers in a row-NO-user

The ping between loopbacks didn’t work, but R1 and R3 had these console messages:

R1#
%TCP-6-BADAUTH: No MD5 digest from 10.0.0.3(179) to 10.0.0.1(28556) (RST)

R1#
%TCP-6-BADAUTH: No MD5 digest from 10.0.0.3(179) to 10.0.0.1(28556) (RST)
R1#

R3#
%TCP-6-BADAUTH: No MD5 digest from 23.0.0.1(179) to 23.0.0.3(59922) (RST)
R3#
%TCP-6-BADAUTH: No MD5 digest from 23.0.0.1(179) to 23.0.0.3(59922) (RST)
R3#

The senior engineer looked at the configurations for R1, R2 and R3 and found 5 specific items, each of which was independently causing a failure.

Here is the challenge:  Can you find 1 or more of them?

Let us know what your troubleshooting skills can find, and post your comments here on the blog.

Here are the configurations for the 3 routers:

R1#show run
version 12.4
hostname R1
!
interface Loopback0
 ip address 1.1.1.1 255.255.255.0
!
interface FastEthernet0/0
 ip address 10.0.0.1 255.255.255.0
!
router ospf 1
 network 10.0.0.0 0.0.0.255 area 0
!
router bgp 1
 no synchronization
 bgp log-neighbor-changes
 network 1.1.1.1 mask 255.255.255.255
 neighbor 10.0.0.3 remote-as 3
 neighbor 10.0.0.3 password cisco
 no auto-summary
!
end
R1#

R2#show run
version 12.4
hostname R2
!
interface Loopback0
 ip address 2.2.2.2 255.255.255.0
!
interface FastEthernet0/0
 ip address 10.0.0.2 255.255.255.0
 ip nat inside
 ip virtual-reassembly
!
interface FastEthernet0/1
 ip address 23.0.0.2 255.255.255.0
 ip nat outside
 ip virtual-reassembly
!
router ospf 1
 network 2.2.2.2 0.0.0.0 area 0
 network 10.0.0.2 0.0.0.0 area 0
 network 23.0.0.2 0.0.0.0 area 0
!
ip nat inside source static 10.0.0.1 23.0.0.1
ip nat outside source static 23.0.0.3 10.0.0.3
!
end

R3#show run
version 12.4
hostname R3
!
interface Loopback0
 ip address 3.3.3.3 255.255.255.0
!
interface FastEthernet0/1
 ip address 23.0.0.3 255.255.255.0
!
router ospf 1
 log-adjacency-changes
 network 23.0.0.0 0.0.0.255 area 0
!
router bgp 3
 no synchronization
 bgp log-neighbor-changes
 network 3.3.3.3 mask 255.255.255.255
 neighbor 23.0.0.1 remote-as 1
 neighbor 23.0.0.1 password cisco123
 no auto-summary
!
end
R3#

Let us know what you find!

Best wishes.

 

 

 

UPDATE:   ANSWERS

Your contributions and input is great.  You ROCK!

I have summarized the 5 specific errors/issues with the configuration, and here they are:

  • R2: NAT isn’t fully baked. Can fix with  “ip nat outside source static 23.0.0.3 10.0.0.3 add-route” (or we could manually add the route as well).
  • R1 & R3: The BGP passwords don’t match, but it doesn’t matter. BGP authentication doesn’t work between NAT’d BGP neighbors, so it would have to be removed. :)
  • R1 & R3: Incorrect network statements for loopback addresses on both BGP routers (incorrect mask)
  • R1 & R3: Ebgp-multihop statements are needed on both neighbors (not directly connected EBGP)
  • R2: R2 doesn’t know how to reach 1.1.1.1 or 3.3.3.3 (non-BGP routing issue)

Again, thanks for the time and effort invested in this solution, and in learning in general.   I appreciate you!

Best wishes.


You can leave a response, or trackback from your own site.

66 Responses to “BGP: The Big Gory Protocol (Can you troubleshoot it?)”

 
  1. Chris Bloomfield says:

    Off the top of my head R1 and R3 are both advertising the loopback address in BGP with a /32 mask but that won’t match the routing table entry as the loopback interface is configured with a /24 mask.

    Maybe ebgp-multihop needs specifying.

    The BGP peering passwords don’t match.

    OSPF not running on R2 as 0.0.0.0 wildcard mask does not match any interface.

    That’s it for now :-)

  2. Robert Juric says:

    Feeling completely under-qualified, I’ll go ahead and try to answer:

    Could one reason be that R1 and R3 do not have OSPF networks configured for their Loopback interfaces?

  3. jo says:

    0. bgp passwords on both routers are different :)

    1. incorrect network statement for both loopbacks(alternatively subnet mask on loopbacks theirself)

    may be we have to make one of the peers active via transport mode modification

  4. Kevin says:

    Hi Keith, how about:

    1. Incorrect subnet mask on the bgp network statement for network 1.1.1.1 – must match the route in the rib.
    2. Wrong password on R3 BGP neighbor 23.0.0.1 password cisco123, should match R1.
    3. Network mask on R3 network statement 3.3.3.3 is wrong – must match the route in the rib.
    4. eBGP multi-hop or disable connected check or ttl security 2 change required on R1 & R3.
    5. No route on R1 to 3.3.3.3 and no route on R3 to 1.1.1.1. Redistribute Lo0 into ospf.

  5. Robbie says:

    1. Router one is advertising incorrect network into BGP – should be mask 255.255.255.0
    2. Router three is advertising incorrect mask – (should be 255.255.255.0) into BGP
    3. Password mismatch for BGP authentication.(One peer is using “password cisco123″ the other is using password “cisco”
    4. BGP peering requires multihop as eBGP peers are not directly connected.
    5. The Nat statements on R2 look a little iffy….

  6. Mark Lah says:

    What I could find:

    -BGP passwords not matching
    -R2 OSPF (3) network statements have the wrong wildcard string
    -R1 BGP loopback network statement has wrong mask
    -R3 BGP loopback network statement has wrong mask
    -R2 NAT statements are wrong. When R1 initially sends out the BGP packet to neighbor 10.0.0.3, it will have a source IP of 10.0.0.1 (IP addy of interface it left from), and when that packet is received on F0/0 of R2 it will hit the ‘ip nat inside’ rule to translate that to have a destination IP of 23.0.0.1. With that destination IP R2 will route it out the F0/1 interface (can’t ARP that IP, but it has a route for that network from R3), R3 will see the packet, and explains the 23.0.0.1 address seen above in the debug output. Vise-versa for the R3-R1 BGP setup.

  7. It’s 12:53 Am hence no time for labbing. Here are some problems I picked up:
    1.Password Mismatch on the BGP peering
    2.Wrong network mask on R1′s loopback under BGP (neighbor 23.0.0.3 remote-as 3)
    3.Wrong network mask on R3′s loopback under BGP (neighbor 23.0.0.1 remote-as 1)
    4.Wrong neighbor statements. It should be on R1 (nei 23.0.0.3 remote-as 3) and on R3 (neighbor 23.0.0.1 remote-as 1)
    5.Last but not least, you are missing the EBGP multihop after the neighbor statement on both routers. On R1 (neighbor 23.0.0.3 ebgp-multihop 255) and on R3 (neighbor 23.0.0.1 ebgp-multihop 255)
    6. For reachability, you might want to add some static routes on R2:
    ip route 1.1.1.1 255.255.255.255 10.0.0.1
    ip route 3.3.3.3 255.255.255.255 23.0.0.3
    or redistribute bgp routes into OSPF
    :-)

  8. Woops, I told you it’s late here. 1:21AM. Small mistakes happens. Discard first one. So here we go again:
    1.Password Mismatch on the BGP peering
    2.Wrong network mask on R1’s loopback under BGP (network 1.1.1.0 mask 255.255.255.0)
    3.Wrong network mask on R3’s loopback under BGP (network 3.3.3.0 mask 255.255.255.0)
    4.Wrong neighbor statements. It should be on R1 (nei 23.0.0.3 remote-as 3) and on R3 (neighbor 23.0.0.1 remote-as 1). You can also use the 10.0.0.0/24 network and it will work fine ;)
    5.Last but not least, you are missing the EBGP multihop after the neighbor statement on both routers. On R1 (neighbor 23.0.0.3 ebgp-multihop 255) and on R3 (neighbor 23.0.0.1 ebgp-multihop 255)
    6. For reachability, you might want to add some static routes on R2:
    ip route 1.1.1.1 255.255.255.255 10.0.0.1
    ip route 3.3.3.3 255.255.255.255 23.0.0.3
    or redistribute bgp routes into OSPF
    :)

  9. Name says:

    The BGP password is cisco on R1, cisco123 on R3

  10. Andre says:

    - network statements on R1 and R3 are using wrong network and mask since loopbacks are /24 networks:

    network 1.1.1.1 mask 255.255.255.255
    network 3.3.3.3 mask 255.255.255.255

    - BGP neighbor passwords dont match

    neighbor 10.0.0.3 password cisco
    neighbor 23.0.0.1 password cisco123

    - EBGP multihop is needed since neighbors are not directly connected

  11. DarrellEscola says:

    R1 & R3 – BGP passwords do not match – cisco vs. cisco123.

    R1 – network statement for BGP has a different mask than Lo0 (/32 vs. /24)

    R3 – network statement for BGP has a different mask than Lo0 (/32 vs. /24)

  12. Shafeeq Shaikh says:

    - mismatched bgp passwords
    - ebgp multihop command missing

  13. BGP is not enable on R2

    Passswords are wrong
    R1 password cisco so R2 should have a password cisco

    R3 password cisco123 so R2 should have a password cisco123

  14. BGP is not enable on R2

    R1 password cisco so, R2 should have a password cisco for neighbor R2
    R3 password cisco123, so R2 should have a password cisco123 for Neighbor R2

    http://prakashkalsaria,wordpress.com

  15. BGP is not enable on R2
    R1 password cisco so, R2 should have a password cisco for neighbor R1
    R3 password cisco123, so R2 should have a password cisco123 for Neighbor R3

    Corrected and final one from me

    http://prakashkalsaria,wordpress.com

  16. Alexei Monastyrnyi says:

    Keith,
    you should have outlined the constrains if any… Like if NAT has to stay and if yes, wether it has to be the same sort of inside source and outside source.

    A.

  17. Jason Munns says:

    I can see a few possible issues.

    1. The network statements in bgp for the loopbacks have the wrong mask, should be a /24.
    2. The bgp passwords don’t match.
    3. I’m not sure about this one, but the ebgp session might need to have ebgp multihop enabled because of the router in between.
    4. Another problem that im not 100% sure on is that R2 may need a /32 static route for 10.0.0.3 pointing to 23.0.0.3. (I remember a recent INE troubleshooting lab had something similar).

    I cant find the 5th fault.

    Jason

  18. sujit says:

    According to me,the five problems associated with this configuration are :-

    1) when no-autosummary is enabled in bgp, the mask advertised with the network command should match with the mask associated with the corresponding route in the ip routing table.

    means loopback 1 has been advertised with /32 instead of the correct mask /24.

    2) bgp configuration has “ebgp multihop ” command missing for respective ebgp-neighbors R1 & R3.

    deault ttl =1 won’t work here.

    3) bgp password mismatch on both neighbors R1 & R3.

    4) In the ospf configuration on R2, 10.0.0.3 & 23.0.0.1 is not been advertised with the “NETWORK command” because of which nat is not working properly.

    5)bgp synchronization problem leading to packets been dropped on R2 as it is unaware of the destination loopback interace .

    A black hole is created on R2.

    either enable bgp synchronization on R2 or run bgp on R2

  19. 1. Both BGP neighbors have different MD5 passwords. However, we can not do NAT and MD5 authentication at the same time in BGP. So if we continue to use NAT we have to remove the MD5 authentication under the BGP process.
    2. Both BGP neighbors should have “ebgp-multihop” (or “ttl-security”)
    3. The loopback networks advertised by both BGP peers are incorrect. They are advertising loopback networks with /32 masks instead of /24 masks.
    4. R2 should have “ip route 10.0.0.3 255.255.255.255 23.0.0.3″, otherwise NAT will not function properly as the router (R2) does route lookup first before doing NAT translation for the traffic received on the inside interface.
    5. R2 will blackhole the traffic between the loopback networks as it has no knowledge of the loopback networks advertised by the BGP peers. One possible solution could be adding static routes for loopback networks in R2 as following:
    ip route 1.1.1.0 255.255.255.0 10.0.0.1
    ip route 3.3.3.0 255.255.255.0 23.0.0.3

  20. Baad says:

    R1: change password to cisco123, or, on R3 change password to cisco.
    R2 :run bgp, advertise loopback, define bgp neighbors.

  21. Zeeshan Ashraf says:

    Neighbor address and password is not correct in BGP protocol configuration. Password should be same on both sides when you are configure authentication in BGP between two neighbors.

  22. cisco'n cie says:

    1 – NAT on R2
    inside to outside NAT performed after routing

    http://www.cisco.com/en/US/tech/tk648/tk361/technologies_tech_note09186a0080133ddd.shtml
    needs a static for host 10.0.0.3 towards R3

    ip route 10.0.0.3 255.255.255.255 23.0.0.3 (or Fa0/1)

    2 – Loopbacks on R1 and R3 are /24 – BGP nets are /32

    change BGP nets to /24
    On R1
    router bgp 1
    network 1.1.1.0 mask 255.255.255.0
    On R3
    router bgp 3
    network 3.3.3.0 mask 255.255.255.0

    3 – Due to the NAT the eBGP peers are not directly connected, needs to configure ebgp-multihop

    On R1
    router bgp 1
    neighbor 10.0.0.3 ebgp-multihop 2
    On R3
    router bgp 3
    neighbor 23.0.0.1 ebgp-multihop 2

    4- TCP MD5 authentication doesn’t work with NAT as the hash is partly based on the original IP header
    Remove MD5 authentication

    On R1
    router bgp 1
    no neighbor 10.0.0.3 password cisco

    On R3
    router bgp 3
    no neighbor 23.0.0.1 password cisco123

    5 – there is no route for R1 and R3 loopbacks on R2 – traffic is blackholed
    Add static routes for R1 and R3 loopbacks

    On R2
    ip route 3.3.3.3 255.255.255.255 23.0.0.3
    ip route 1.1.1.1 255.255.255.255 10.0.0.1

    Alternately advertise the loopback into OSPF on R1 and R3 with ospf network type point-to-point (so that R2 knows the loopbacks via OSPF but R1 and R3 still prefer the routes via ebgp)

  23. Sherif says:

    I think the problem in R2.. below is the result of debug ip tcp on R2

    *May 26 10:36:51.679: MD5 received, but NOT expected from 10.0.0.1:59566 to 10.0.0.3:179
    *May 26 10:36:51.679: TCP: connection attempt to port 0
    *May 26 10:36:51.679: TCP: sending RST, seq 0, ack 686446852
    *May 26 10:36:51.683: TCP: sent RST to 10.0.0.1:59566 from 10.0.0.3:179
    *May 26 10:36:51.687: TCP0: state was LISTEN -> CLOSED [0 -> UNKNOWN(0)]

    This is because When R1 tries to open a tcp connection to 10.0.0.3, R1 send open message contains MD5 key, but R2 don’t listen on that port as no BGP is configured on it.

  24. MO says:

    ebgp-multihop should be configured
    neighbor password mismatch

  25. Krzysztof says:

    - BGP passwords are not the same
    - loopback networks are not in the OSPF – they won’t ping
    - loopback network masks added to BGP process are not the same as on the interface, so they won’t enter the BGP process

  26. alan says:

    I found 4, not convinced on number 5

    1 – obviously BGP passwords would be incorrect on R1 and R3 for peers
    2 – We would need a STATIC route on R2 (fixup rule) pointing 10.0.0.3 to 23.0.0.3
    3 – INCLUDE ebgpmultihop 2 in your neighbour statement on R1 and R3 (took me a while to figure that out, thanks google)
    4 – change the the NETWORK statment in BGP on both R1 and R3 to 3.3.3.0
    mask 255.255.255.0
    5 – goto be routing, is adding Loopback interfaces on R1 and R3 into OSPF cheating? cuz thats what i did.

  27. Ronald says:

    Of one I’m really sure:
    1) The passwords are different: R1 has cisco. R3 has cisco123.

    About these 2 I’m not so sure:
    2) This line has to be deleted on R2:
    “ip nat outside source static 23.0.0.3 10.0.0.3″

    3) On R1 this has to change:
    ” neighbor 10.0.0.3 remote-as 3
    neighbor 10.0.0.3 password cisco”

    should be:

    ” neighbor 23.0.0.3 remote-as 3
    neighbor 23.0.0.3 password cisco”
    ———-

    Please forgive me if it’s wrong.. I’m working on my CCNP! :-)

  28. Ian Finlayson says:

    Hi Keith,

    Quickly off the top of my head and I hope these are right :) ???

    1) BGP Password is not the same on R1 and R3
    2) eBGP Hop Count will be an issue
    3) R1 is expecting to see neighbor coming from 23.0.0.1 which it wont
    4) R3 is expecting to see neighbor coming from 10.0.0.3 which it wont
    5) Neighbor update-source statements are needed on R1 & R3

    Just a quick scan, hopefully I got some of them right :)

    Rgds,
    Ian.

  29. MO says:

    as a follow up on my first comment:

    Configure a static route on R2 as follows:
    ip route 10.0.0.3 255.255.255.255 23.0.0.3

    This is a more specific route. The reason we do this is that with NAT when the packet travels from outside to inside, translation occurs first, and then the routing table is checked for the destination.

  30. Konstantin Zalyan says:

    Obvious one – BGP passwords missmatch (cisco on R1 and cisco123 on R3)

    Looking further for other problems in config :)

  31. Ven says:

    1. R1# neighbor 10.0.0.3 password cisco
    R2# neighbor 23.0.0.1 password cisco123

    2. R1# interface Loopback0
    ip address 1.1.1.1 255.255.255.0
    !
    router bgp 1
    network 1.1.1.1 mask 255.255.255.255

    3. R3# interface Loopback0
    ip address 3.3.3.3 255.255.255.0
    !
    router bgp 1
    network 3.3.3.3 mask 255.255.255.255

    4. R1# router bgp 1
    neighbor 10.0.0.3 ebgp-multihop 2

    5. R3# router bgp 3
    neighbor 23.0.0.1 ebgp-multihop 2

  32. Mc says:

    - Missing the neighbor x.x.x.x ebgp-multihop command
    - The BGP network command are bad, as it must match the exact network a mask of 255.255.255.0 would be better
    - The static route of 10.0.0.3 toward R3 is missing for the outside nat. (could be done with route-add keyword)

    - For the authentification the passwords are differents and I’m not sure md5 could work thrue NAT as bgp header is not modified, so i would remove authentication

  33. Ritesh Malviya says:

    1 is

    On R1 & R3 the MD5 Auth password is different ,

  34. fulopa7 says:

    Hi,

    I think the first problem is that the bgp passwords do not match..

    Attila

  35. Maciej Wisnirwski says:

    Hello,

    First of all there is no connectivity between R1 and R3 peer addresses. The nat outside command on R2 have to be changed to work properly in this scenario to:

    R2:
    ip nat outside source static 23.0.0.3 10.0.0.3 add-route

    The second issue is a bgp password mismatch. R1 has cisco, R3 cisco123 configured.

    But BGP usses TPC MD5 to authenticate peer and TCP MD5 uses TCP pseudo header to calculate hash value. TCP pseudo header contains source and destination IP addresses so it will not work with natted peers. Source and destination IP addresses will change so computed hash value will not match with those from packets.

    Next issue is wrong network statement for ip subnets of loopback addresses on both R1 nad R3. There is no network 1.1.1.1/32 in R1 (and 3.3.3.3/32 on R3) routing table so BGP will not install that route in BGP table. It should be changed to:
    R1:
    router bgp 1

    network 1.1.1.0 mask 255.255.255.0

    R3:
    router bgp 3

    network 3.3.3.0 mask 255.255.255.0

    R1 and R3 are eBGP neighbors that are not adjacent. One has to add ebgp-multihop statement to build proper BGP session.
    R1:
    router bgp 1

    neighbor 10.0.0.3 ebgp-multihop

    R3:
    router bgp 3

    neighbor 23.0.0.1 ebgp-multihop

    With this new configuration R1 and R3 will build valid BGP sessions and exchange NLRI about 1.1.1.0/24 and 3.3.3.0/24. Unfortunately there will be no connectivity between 1.1.1.1 and 3.3.3.3. R2 is a transit router but it has no knowledge about networks 1.1.1.0 and 3.3.3.0 so it will discard all traffic between those addresses.

  36. Gavin says:

    BGP Password incorrect
    Neighbor statements using their local subnets and not the remote

    R3: SHOULD BE
    router bgp 3
    neighbor 10.0.0.1 remote-as 1
    neighbor 10.0.0.1 password cisco123

    R1: SHOULD BE
    router bgp 1
    neighbor 23.0.0.3 remote-as 3
    neighbor 23.0.0.3 password cisco123

  37. Maciej Wisniewski says:

    The issue with blackholing traffic at R2 could be solved in numerous ways. The clue is that R2 has to have knowledge about routes to 1.1.1.0/24 and 3.3.3.0/24. One can for example add static routes to 1.1.1.1 and 3.3.3.3 on R2, run OSPF on R1 and R3 loopback interfaces, run BGP on R2 etc.

  38. John Batty says:

    Hmm,

    I’ve seen a few:
    Passwords are not the same between both BGP Speaking routers- so would fail
    Network statements on both BGP routers would not match the loopback address in the routing table as the mask is wrong so the looopbacks wouldn’t get advertised
    EBGP Multihop would be required as we are more than one hop away from each other.

    The last is possibly to do with the NAT/OSPF, I’d have to lab it up to see for certain, but not sure if it there is something wrong there as well.

  39. Mat says:

    Under BGP,
    network 1.1.1.1 mask 255.255.255.255 should be network 1.1.1.1 mask 255.255.255.0 (need an exact match in the routing table)

    network 3.3.3.3 mask 255.255.255.255 should be network 3.3.3.3 mask 255.255.255.0 for the same reason
    (do those count as 2 errors?)

    The configured password on the BGP neighbors don’t match.

    I think you would also need the ebgp-multihop command on the BGP neighbors (or disable-connected-check)

  40. Kirill says:

    1. Different passwords
    2. Wrong masks in BGP network statements
    3. TTL has to be decreased when packet is traversing NAT. So, “ebgp multihop” is needed.

  41. Ritesh Malviya says:

    1st ) under config on R1 & R3 MDF authentication password is different .
    Thats the reason its giving error .

  42. Mahmoud says:

    1- BGP network Mask in R1 and R3 for Loopback is different than what is configured in the interface.

    2- Passwords is different between R1 and R3.

    3- Multihop isn’t configured.

  43. D says:

    One of the mistake is the password mismatch in BGP
    R3 – neighbor 23.0.0.1 password cisco123
    R1 – neighbor 10.0.0.3 password cisco

  44. D says:

    The other one is the network command in BGP
    R1 – network 1.1.1.1 mask 255.255.255.255
    R3 – network 3.3.3.3 mask 255.255.255.255

    should be
    R1 – network 1.1.1.1 mask 255.255.255.0
    R3 – network 3.3.3.3 mask 255.255.255.0
    or
    configure static route pointing to NULL0

  45. d_esTin_y says:

    R1
    router bgp 1
    network 1.1.1.0 mask 255.255.255.0
    neighbor 23.0.0.3 remote-as 3
    neighbor 23.0.0.3 ebgp-multihop 2
    neighbor 23.0.0.3 password cisco

    R3
    router bgp 3
    network 3.3.3.0 mask 255.255.255.0
    neighbor 23.0.0.1 ebgp-multihop 2
    neighbor 23.0.0.2 password cisco

    R2
    ip route 1.1.1.1 255.255.255.255 10.0.0.1
    ip route 3.3.3.3 255.255.255.255 23.0.0.3

    it was not easy ;)

  46. D says:

    The other one is because the ebgp neighbor is not one hop away, ebgp-multihop is needed to reach the bgp peer.

  47. Owen Rodgers says:

    I’ll take the easy one and point out that the R1 and R3 passwords don’t match.

  48. Adam says:

    1. BGP passwords don’t match.
    2. The network statement for the Loopback on R1 is a /32 but the interface is configured for a /24. One needs to change so they match.
    3. The network statement for the Loopback on R3 is a /32 but the interface is configured for a /24. One needs to change so they match.
    4. BGP Multihop needs to be configured as there is another router in the path of the BGP session.
    5. ?

  49. Michael Kartvelishvili says:

    Hello,

    Here are 5 problems (only actually 4 resolved)

    1. Incorrect mask in network statement under BGP.

    on R1:
    router bgp 1
    network 1.1.1.0 mask 255.255.255.0
    on R3:
    router bgp 3
    network 3.3.3.0 mask 255.255.255.0

    2. incorrect BGP password
    on R3:
    router bgp 3
    neighbor 23.0.0.1 password cisco

    3. Inconsistent NAT config. NAT is not operational as BGP session destination address is routed out the same interface as source (as RIB lookup is triggered before actual NAT) . Here R2 seems to answer with an RST without MD5 option on behalf of the destination creating confusing message shown above.

    On R2:
    ip route 10.0.0.3 255.255.255.255 FastEthernet0/1
    ip route 23.0.0.1 255.255.255.255 FastEthernet0/0

    4. When (3) is corrected we need to add ebgp-multihop as peers are 2 hops away from each other.

    On R1:
    router bgp 1
    neighbor 10.0.0.3 ebgp-multihop 2

    On R3:
    router bgp 3
    neighbor 23.0.0.1 ebgp-multihop 2

    5. The last problem is related to the fact that TCP MD5 option (Option 19) is protecting not only TCP segment, but IP header as well (according to RFC2385). I cannot see how this one can be resolved without either disabling NATing of BGP session or disabling BGP session authentication itself. As any modification of IP header performed by NAT (like IP address rewrite) will automatically invalidate TCP MD5 option, resulting the message like

    %TCP-6-BADAUTH: Invalid MD5 digest from 10.0.0.3(49500) to 10.0.0.1(179)

    Best Regards,
    Michael

  50. Alejandro says:

    1. Remove static NAT statements
    2. Reconfigure the “neighbor remote-as” statements
    3. Passwords must be the same
    4. Enable eBGP multihop capability
    5. Correct subnet mask on “network” statements

  51. Tim says:

    1) In router 2, the NAT needs to be removed. The hash in the IP headers are being modified by the NAT process. As such, the password being shared by both neighbors are being altered and thus creating one cause of authentication failure.

    2) In router 1 and 3, without the NAT. The neighbor statements must now point to the ‘real’ remote peers to establish the ebgp peering.
    R1 needs to point to 23.0.0.3 for its peer statements
    R3 needs to point to 10.0.0.1 for its peer statements

    3) Both router 1 and 3 need to agree on the same password.

    4) For the ebgp peer to establish, a BGP multihop neighbor statement is needed between R1 and R3.

    5) The network statements have the wrong mask in both R1 and R3. They need to be /24 because no-auto is enabled.

  52. schoutentl says:

    I’m seeing missing routes to let BGP pull them in, password set for one an not another. But this is just a glance have to go to work will look again later.

    Terry

  53. minty says:

    I have been playing with this. I think these are the 5 errors/issues
    1 password do not match in the neighbor statements
    2 lo0 require advertising in ospf or r2 will blackhole the traffic
    3 network statements under bgp x should match the lo0 subnet mask
    4 use loopback peering and ebgp-multihop
    5 ip nat outside source static is not required

    Please advise…..

    Ian

  54. Jason Beltrame says:

    password between the 2 are incorrect.

  55. Nendy says:

    The errors that I found are;
    1- wrong password for BGP neighbors
    2- wrong mask in network statement in BGP in R3 or wrong mask in loopback 0 .. they have to match
    3- eBGP multihop is needed between neighbors
    4- Static route is needed in R2 because NAt inside to outside first route and then translate (ip route 10.0.0.3/32 fas0/1)
    5- R2 doesnt know about 1.1.1.1/32 or 3.3.3.3/32 .. (redistribution will fix that)
    Thanks,

    Nendy

  56. Arturas Lapiene says:

    Hi,

    1. BGP md5 password aren’t the same
    2. NAT breaks md5 digest fields because it changes src/dst addresses.
    3. Because EBGP default ttl is 1, in this case bgp-multihop is requried
    4. Network statements are incorrect, because there is no exact match in routing table
    5. Blackhole routing occurs on R2 for R1 and R3 lookbacks, because R2 does not know anything about their loopbacks.

  57. Thanks for all the input! I updated the original post with the answer regarding the 5 errors/issues we were looking for. I appreciate all the contributions made everyone.

    Keep up the great studies.

  58. Eugene K says:

    1) Passwords on R1 and R3 do not match. R1 password is “cisco”, R3 password is “cisco123″.
    2) If we make the password the same, problem still remains because nat breaks md5 digest.
    3) network statement for loopback do not match on r1 and r3
    At this point:
    “ip nat outside source static” is not needed, but this is not point of failure.
    we can remove bgp completely and use “ip ospf 1 area 0″ on both loopbacks on R1 and R3.
    we can remove nat completely and because of that:
    4) ebgp multihop needed because this is eBGP session between two not directly connected neighboors in different AS.
    5) neighboor statement will be changed because now we are peering betheen 10.0.0.1 and 23.0.0.3
    6) and finally blackholing occures on R2, because R2 does’n know anything about 1.1.1.1 and 3.3.3.3.

    Oh… too late. Answers already posted =(

    Hi, can we use MPLS and “BGP-free core” concept to fix 6th issue?

  59. Nadeem Rafi says:

    Thanks for such a nice “teaser”.

  60. Thanks,

    you come up every week a scenario and test us

    its great improving my skills out here
    keepi it up

    how about MPLS TE using PBR reaching the destination and source through PBR (loopbacks which is not advertised in IGP and no static route)
    can you come up with this scenario

  61. Prakash -

    Thanks for the kind words and the request. I will put that on the list of possibilities for a future post.

    Thanks again.

  62. Eugene -

    Yes, if we setup MPLS and VPNv4 services, and the 1.1.1.1 and 2.2.2.2 networks were customer routes, we could easily label-switch the transit traffic across R2.

    Best wishes.

  63. Dear, Password mismatch is problem is here

    router1#
    router bgp 1
    no synchronization
    bgp log-neighbor-changes
    network 1.1.1.1 mask 255.255.255.255
    neighbor 10.0.0.3 remote-as 3
    neighbor 10.0.0.3 password cisco
    no auto-summary
    ________________________________________
    router bgp 3
    no synchronization
    bgp log-neighbor-changes
    network 3.3.3.3 mask 255.255.255.255
    neighbor 23.0.0.1 remote-as 1
    neighbor 23.0.0.1 password cisco123
    no auto-summary
    !
    end
    R3#

  64. Bromden says:

    ‘add-route’ in NAT command is not necessary as the outside local address is already in the RT. It would be needed if we chose an address outside of 10.0.0.0/24 in order to redistribute it to OSPF so that R1 can learn it.

  65. Tammy Burley says:

    hummm…wondering why the “route” portion of the nat config is required if the subnets that are being used are directly connected to R2? Even with a route the order of operation for outside nat is route first then nat. routing to the destination in the packet 23.0.0.1 would turn it right back out the interface it came in on. the return packed from R3 with a source of 23.0.0.3 destined for 23.0.0.1 would get routed to R2 fa0/1 first, then natted. Then that’s where the 23.0.0.1 would get translated to 10.0.0.1 right? Or would the route that is added be ip route 23.0.0.1 255.255.255.255 10.0.0.1? thanks!

  66. Khan says:

    Hi,

    If its a normal IP packet and nat enabled we can think of authentication not working, but BGP packet in OPEN msg contains OPTIONS field which is not mandat.. then how does “BGP authentication doesn’t work between NAT’d BGP neighbors, so it would have to be removed.” this statement clarify…

 

Leave a Reply

Categories

CCIE Bloggers