MPLS L3VPN’s – Part 1

MPLS Layer 3 VPN’s (MPLS L3VPN) or also called MPLS IP VPN, is one of the most used application of MPLS in today’s networks. This service is most popular with Service Provider’s but can also be found in Enterprise WAN and Data-center environments. This post will focus on an overview of the different technologies used to create an MPLS IP VPN network and how they interact with each other.

MPLS L3 VPN is, like the name says, a way to create a Layer 3 VPN by harnessing the power of MPLS. The terminology “L3” or Layer 3 refers to the third layer of the OSI model and it is basically just a fancy way of saying that we are segregating at the network layer by creating a separate routing and forwarding table for each VPN. This means that for each VPN we can have an overlapping IP addressing topology running on a shared backbone network.

MPLS VPN Connection Model:

Let’s start by looking at a common topology for an MPLS VPN network. This is what a simplified Service Provider network would look like:

2015-11-14 12_40_05-untitled

The purpose of the network in this example is to send traffic for each customer from one site to another over the routed Service Provider network while maintaining a separate routing and forwarding instance for each customer. In an enterprise environment, the same topology could be used but with the goal being instead of separating traffic between different departments, applications, groups, services or any other logical domain that might be a requirement of the business. There are different components and technologies used in this network and we will explore them one by one separately.

The first part of this network is the Core MPLS Network, made of Provider (P) devices.

2015-11-14 14_29_57-untitled

P devices sit inside of the MPLS network and run a link-state routing protocol (OSPF or IS-IS) with an MPLS distribution protocol on all of their interfaces. There are a couple of MPLS distribution protocols that can be used: Label Distribution Protocol (LDP), Resource Reservation Protocol with Traffic Engineering Extensions (RSVP-TE), Constraint-based routed LDP (CR-LDP), Multiprotocol BGP (MP-BGP) and Segment Routing, but we will focus on LDP for simplicity and because it is possibly the most widely used.

A quick overview of LDP:

Label Distribution Protocol (LDP) is a protocol used to form Label-Switched Paths (LSP’s) by using the existing routing table made by an IGP. LSP’s are sequences of MPLS enabled devices that forward packets of a certain Forwarding Equivalence Class (FEC). A FEC is a set of packets that a single device forwards the same next hop, with the same interface and the same treatment. If this all sounds confusing and unclear, just think of an LSP as a unidirectional tunnel between two devices that share common characteristics. Without going into too much details on this technology, LDP will create an LSP using P devices and it will be used to forward customer traffic across the MPLS network.

The second part of this network is the Provider Edge (PE) devices.

2015-11-14 14_31_29-untitled

PE devices sit at the edge of the MPLS network in between Provider (P) and Customer Edge (CE) devices. They use MPLS/LDP to the P devices and IP to the CE devices. By running LDP with the P devices, the PE’s are able to create an LSP to the other PE’s. The labels forming this LSP will be used for forwarding packets over the MPLS core and are more commonly called the IGP label or Transport Label.

PE’s also need to create an MP-BGP connection to other PE’s to exchange VPNv4 information as follow:

2015-11-14 15_40_24-untitled

We will see later on what is contained exactly in these BGP updates but one of the key information exchanged will be the VPN address that will be imposed as a label, more commonly known as the VPN label.

PE devices also have another very important role and that is to hold a separate routing instance for every CE. This is made possible by using a separate Virtual Routing and Forwarding (VRF) instances for each customer.

A quick overview of VRF’s:

Virtual Routing and Forwarding (VRF) is a technology that allows multiple routing and forwarding tables to exist on a single device. It is basically the equivalent of a VLAN for Layer 3 segmentation. The use of a VRF alone is called VRF-lite, but within the context of MPLS it is just called a VRF. The difference really is that VRF-lite does not make use of two critical components used in MPLS IP VPN’s: the Route Distinguisher’s (RD’s) and Route Target’s (RT’s).

Route Distinguishers (RD’s) are defined within a VRF’s configuration and their only purpose in L3VPN’s is to make an IPv4 prefix globally unique for route exchange. This is important so that the service provider can distinguish between two same prefixes from different customers.

Let’s say Customer 1 and Customer 2 both have the network and are advertising this prefix through the Service Provider network. How would the Service Provider know from what customer this route came from? It can’t, unless there is something to differentiate both routes. That is exactly why the route distinguisher exists. During the route exchange process, the route distinguisher will be appended on the IPv4 address as follows:

2015-11-14 16_49_08-Untitled - Notepad

Route Target’s (RT’s) are also defined within a VRF configuration and in a similar format as the Route Distinguisher but they have a completely different purpose. Their role is to tell PE device which prefixes should be exported or imported in the VPNv4 routing table using a BGP extended Community Attribute.

Again, we will see how this whole process works later on this post but for now just remember that the Route Distinguisher makes an IPv4 address unique across the MPLS network and the Route Target defines which prefixes get imported or exported on the PE devices.

The third and final part of the network that we need to look at are the CE devices:

2015-11-14 17_22_57-untitled

CE devices sit at the edge of the network and they exchange IP routes with the PE devices using an IP routing protocol (BGP, OSPF, EIGRP, IS-IS, RIP). On the CE devices, the routing is done in the global routing table (normal non-VRF routing table) but on the PE, the routing is done in a separate VRF for each customer. VRF’s are locally significant and this means that the CE does not need to know if the PE interface is in a VRF or not. From the perspective of the customer, the routes advertised to the PE device are tunneled transparently across the service provider network and advertised to the CE at the other end without any special configuration.

To better understand the whole process, let’s look at the life of an IP packet as it moves across the Service Provider network from its signaling (control-plane) to it’s forwarding (forwarding plane). For simplicity, we will only look at Customer1 in a single direction (uni-directional traffic).

  1. The first step is to have LDP to do it’s magic. LDP Label Binding and signaling will be done hop by hop from the loopback’s of a PE to the other PE device, forming an LSP.

2015-11-15 12_37_42-untitled

In this case the LSP is created for Loopback0 ( prefix) and the labels signaled are: Null -> 33 -> 16 -> 40. The first label is always signaled as Null because of the Penultimate hop popping (PHP) function. This process is simply an optimization to avoid a double lookup of the label at the last hop device of the LSP; the Label Edge Router (LER). You can read more on this feature if you check RFC3031 section 3.16

  1. In the second step, the customer sends a route update from the CE device and it is advertised through the routing protocol configured between the CE and PE device. When it reaches the PE device, it is signaled through MP-BGP to the other PE as a VPNv4 route with a Route-Distinguisher attached to the prefix, a Route-Target and a Label. In this case, the route 200:1: is sent with Route-Target 200:1 and Label 21.

2015-11-15 12_57_24-untitled

The update packet in Wireshark  looks like this:


In the first box you can see the Route-Target value of 200:1. You can also see in the third box the Label value sent of 21 for the IPv4 route Finally, in the last box you can see the Route-Distinguisher value that makes the route globally unique.

  1. Finally, the customer at the opposite side sends packets for the signaled route. These packets are forwarded from the CE using IPv4 to the PE. From the PE, the IPv4 packets will be encapsulated with two labels to be forwarded over the MPLS backbone, the VPN and the IGP label. Again, the VPN label was signaled by MP-BGP and tells us in which VRF of the opposing PE to send the traffic into. The IGP label was signaled by LDP and is used to forward packets hop by hop across the MPLS backbone.

2015-11-15 13_15_33-untitled

In this example, when the IPv4 packet for destination enters the ingress PE device from the CE, the PE device looks up the destination IP address ( in its forwarding table and finds the correct VRF to send the traffic by looking at which interface the route entered the PE device in the first place. It encapsulates the packet with both IGP and VPN labels (21 and 40) and sends it to the first P device. Hop by hop, the P devices will change the IGP label based on the BGP next-hop IP address ( and finally remove it at the last P before entering the PE device because of the Null label. The last hop PE device looks up the VPN label in its forwarding table and forwards the packet without any labels (as an IP packet) towards the CE device.

Again, this example only showed unidirectional forwarding. If bi-directional traffic was required, Step 1 and Step 2 would have to happen again in the reversed direction for the route back towards the source to be installed in the control-plane of the remote CE device.

In conclusion, MPLS L3VPN is a mix of several different protocols forming a highly scalable and flexible VPN solution. In Part 2, I will be looking at different use cases for MPLS L3VPN’s and some of the most common deployment scenarios.

Hour 80: Routing Traffic Isolation using VRF’s

A lot of engineers when they hear Virtual Routing and Forwarding (VRF’s) they think MPLS VPN’s and associate it to a Service Provider feature. Although VRF’s are widely used with MPLS in SP environments, in today’s networks, a lot of companies will opt to use the VRF feature alone in order to obtain L3 isolation. Similar to VLAN’s that isolate broadcast domains at L2, VRF’s (also called VRF-lite) isolate routing information at L3 by creating local virtual routing tables.

The main reason, that I’ve personally implemented VRF’s in networks, was in order to join several private overlapping address topologies. Even though VRF-lite is only locally significant, GRE tunnels can be used to extend VRF information. In fact, you can make a GRE tunnel VRF-aware with the use of the “tunnel vrf <vrf name>” command. While VRF’s offer a layer of security by isolating routing information locally, it does not offer any kind of security when passing data from one point to the other of the network, and for that reason, I would always suggest coupling GRE tunnels with IPSEC.

A common misconception I’ve heard from network engineers is that since a network is already using VRF’s you might as well implement MPLS to increase the forwarding speed of the network. This is a bogus argument as with CEF enabled on all routers, MPLS will not offer any speed increase at the forwarding level. In fact, you will lose speed as the overhead of the MPLS + IP calculation factor will make your forwarding decision process a little slower. It’s very important to note that MPLS and VRF are two different separate technologies that aren’t interdependent of each other at all but rather compliment one another in certain situations.