IE Vol 1: Inter-AS MPLS VPNs with MP-eBGP for VPNv4 Exchange Complete

Wow, that title sure is a mouthful. This lab seriously threw me for a loop. No matter what I did, my provider PE routers would not load the vpnv4 routes advertised by the customer PE routers.

After playing with enough debug commands, I finally came across the following:

R2#debug ip bgp vpnv4 uni upd
*Dec 31 03:03:41.943: BGP(2): 150.1.3.3 rcvd UPDATE w/ attr: nexthop 150.1.3.3, origin ?, localpref 100, metric 0, extended community RT:1:100
*Dec 31 03:03:41.943: BGP(2): 150.1.3.3 rcvd 1:100:10.1.37.0/24 -- DENIED due to: extended community not supported;

Huh? What does it mean extended community not supported?

As it turns out, extended communities, by default, are not accepted from eBGP peers. This is the first lab that I've tried to exchange vpnv4 routes across eBGP, so it's the first time I've run into this.

The command to enable extended communities across eBGP peers is no bgp default route-target filter. The details can be found here.

IE Vol 1: Inter-AS MPLS VPNs with Back-to-Back VRF Complete

Things are starting to get really complex. I was stumped a few times on this one and had to look at the answers. Most of my problem involved incorrect assumptions about bgp vpnv4.

Prior to this lab, the general rule has been to only peer bgp vpnv4 with PE routers. But in this lab, two separate bgp ASs are peering separate vrf's over separate interfaces. Rather than two providers sharing routes over vpnv4, they split them over the interfaces as ipv4 neighbors.

So it actually ends up with each provider's edge router actually acting like a CE router and a PE router at the same time. Each provider is each other's customer.

I guess this is what back-to-back VRF means. Rather than sharing the vpnv4 information across bgp, we strip the rd information out, but dedicate a subinterface for each vrf. Then on the other side of the link, the router loads the routes back into bgp and reconstructs the vpnv4 routes.

Why do it this way? One reason seems to make sense--what if you need to carry MPLS VPNs across separate providers but each provider is using different route descriptors? This would allow each provider to load their own rd when the routes are imported. If vpnv4 routes were shared, rd's would need to be consistent across the entire path.

IE Vol 1: Carrier Supporting Carrier - IP Only

In this lab, the goal is to segment one AS (something like a tier-2 carrier) with another AS (a tier-1 carrier). It's quite interesting how this ends up working out. The Tier-1 carrier becomes the P network, and the tier-2 carrier becomes the C network. iBGP relationships can still be formed by CE routers across the P network, since as far as the CE routers are concerned, the routes are all being learned via OSPF. The entire separate AS is completely transparent to the C network.

There really wasn't anything particularly challenging about this lab, there was just a lot going on. It's definitely required to build each piece step by step and verify along the way. A simple missing mpls adjacency, ospf neighbor, or bgp next-hop-self missing could make troubleshooting really difficult.

The mpls verification was quite interesting. On the PE routers, a separate MPLS adjancency is actually formed inside of the vrf. To check this, the commands sh mpls ldp neigh vrf VPN_A and sh mpls for vrf VPN_A are required.

Aside from that, I had to go back and do a couple of redistribute connected's into bgp. This was required so that I could ping from the outgoing interface rather than the loopback of the true customer routers.

IE Vol 1: VPNv4 Route Reflection Complete

This was an interesting lab that took a heck of a lot of time to just reconfigure the topology.

Essentially, the configuration ends up with two separate route reflectors--one for ipv4 routes and one for vpnv4 routes. Amazingly, no real complex configuration is required. There's no need to mess with clusters or anything like that. Instead, simply configuring the route-reflector-clients under vpnv4 instead of directly under the bgp process takes care of it.

I did run into two issues due to simple mistakes. First, one of my PE routers showed all MPLS labels as being untagged. This was due to me forgetting to turn on cef on one of the P routers.

Second, one of my CE routers was not learning any RIP routes. This issue actually threw me for a loop for a bit. It turned out that on the PE router I didn't set RIP to version 2 under the appropriate address-family. By default, rip sends version 2 and receives versions 1 and 2. But when version 2 is specified, rip only sends and receives version 2.

Since the PE router did not specify, it received the version 2 routes from the CE router and entered them into the routing table. It also redistributed its bgp routes into RIP and sent them as version 1.

Since the CE router specified version 2, it ignored the version 1 routes sent by the PE router.

IE Vol 1 OSPF Domain-id complete

This was another interesting lab. Typically in MPLS, redistribution is done from OSPF to BGP and back to OSPF in order to connect OSPF together across the P network. Because of the redistribution, OSPF sees the remote sites as External (LSA type 5) networks.

But since these remote sites are likely to be within one company's control, having them marked external may not be the best option. Instead, the domain-id can be used on the PE router under the ospf process. When the PE router advertises the ospf route to the CE router, it looks at the domain ids of both routes. If they match, the ospf route is advertised as an inter-area (IA) route instead of an External route.

This seems like it would be important for a couple of reasons. IA routes are always preferred over E routes. Also, E routes are not advertised into stubby areas, while IA summaries are. In more complex scenerios with a mix of nssa areas and redistribution, a company may prefer to use its provider's MPLS network rather than another non-MPLS link. The domain-id allows for this much easier than filtering routes from the edges.

IE Vol 1 OSPF Sham Links Complete

This was an interesting lab on two levels. It was the first time I have configured isis and ospf sham links are an interesting subject.

I was really amazed at how easy it is to configure a simple instance of isis. Just configure a net address and throw isis on the interfaces, optionally set the interface type, and it's complete. No issues there.

OSPF sham links, on the other hand, I found to be a a much more complicated subject. Its purpose is to deal with a customer that has non-MPLS links neighboring via OSPF, yet the customer still wants to route this traffic over MPLS.

Once these OSPF routes hit BGP, they are seen as iBGP routes, which as an AD of 200, as opposed to the OSPF AD of 110. This means if the prefix-length and default AD is the same, the OSPF route is going to be entered into the routing table.

To prevent this, an ospf sham link must be created. This is configured by setting up an additional loopback interface on the PE router and adding it to the customer VRF. Next, these loopbacks need to be redistributed into the bgp ipv4 vrf. Once the loopbacks are reachable to each other via the P network, under the ospf vrf process, enter area 0 sham-link source-ip dest-ip. This, somewhat similar to a virtual-link, creates a logical link between the newly created loopback interfaces. Then, the path is seen via ospf rather than bgp and the MPLS route can be preferred over the customer link.

Study Plan

The following is a rough draft of my SP Lab study plan. I don't have a date set yet so this will become more precise once I do. I plan (so far) to use Internetwork Expert for all of my studying. The basic timeline that was successful last time was 1 month of studying one practice lab per week, and then two months of two practice labs per week leading up to the actual lab.

Already completed: Volume 1 MPLS VPN labs.

Week of 12/21: Remaining Volume 1 MPLS labs.
Week of 12/28: Volume 1 Multilcast labs.

1/4: Volume II lab 1
1/11: Volume II lab 2
1/18: Volume II lab 3
1/25: Volume II lab 4
1/31: Volume II lab 5

2/1: Volume II lab 6
2/7: Volume II lab 7
2/8: Volume II lab 8
2/14: R&S Graded Mock lab
2/15: Volume II lab 9
2/21: CCIE Assessor R&S
2/22: Volume II lab 10
2/28: R&S Graded mock lab
2/29: Repeat of one of the Volume II labs

So this puts me taking the lab sometime in March. Hopefully I can find a date around there. I'm still waiting for my exam to clear the CCIE site so I can see what dates are available. I should know within the next day or two.