MPLS L3VPN
Talking about MPLS L3VPNs involves to understand three terms to describe the roles of routers when building L3VPNs. The names used for the routers in most of this post followed the convention of identifying the type of router as CE, PE, or P, as explained here:
- Customer edge (CE): A router that has no knowledge of MPLS protocols and doesn’t send any labeled packets but is directly connected to an LSR (PE) in the MPLS VPN.
- Provider edge (PE): An LSR that shares a link with at least one CE router, thereby providing a function particular to the edge of the MPLS VPN, including internal BGP (iBGP) and VRF tables.
- Provider (P): An LSR that doesn’t have a direct link to a CE router, which allows the router to just forward labeled packets, and allows the LSR to ignore customer VPNs routes.
The key to understanding the general idea of how MPLS L3VPNs work is to focus on the control plane distinctions between PE routers and P routers. Both P and PE routers run LDP and an IGP to support unicast IP routing. However, the IGP advertises routes only for subnets inside the MPLS network, with no customer routes included. As a result, the P and PE routers can together label-switch packets from the ingress PE to the egress PE.
PEs have several other duties as well, all oriented toward the issue of learning customer routes and keeping track of which routes belong to which customers. PEs exchange routes with the connected CE routers from various customers, using either external BGP (eBGP), RIPv2, OSPF, or EIGRP, noting which routes are learned from which customers.
To keep track of the possibly overlapping prefixes, PE routers don’t put the routes in the normal IP routing table. Instead, PEs store those routes in separate per-customer routing tables, called VRFs (Virtual Routing and Forwarding). Then the PEs use iBGP to exchange these customer routes with other PEs through an address-family called VPNv4.
An extra work for the PE relates to the fact that the MPLS L3VPN data plane causes the ingress PE to place two labels on the packet, as follows:
- An outer MPLS header (S-bit = 0), with a label value that causes the packet to be label switched to the egress PE.
- An inner MPLS header (S-bit = 1), with a label that identifies the egress VRF on which to base the forwarding decision.
In addition, MPLS L3VPN introduces two new concepts that are related to VRFs and are explained as follows:
- Route Distinguishers (RD): While VRFs allow for the same IP space to be reused by distinct routing domains, it is the route distinguisher (RD) that distinguishes a set of routes that are part of one VRF from another. An RD is a unique number that is added to each route within a VRF, which helps other routers to identify the routes as belonging to that specific VRF or customer.
- Route Targets (RT): Route Targets are identifiers added to routes to enable a router to know which routes must be inserted into which VRFs. A route target allows the router to control the import and export of routes among different VRFs.
Demonstrating: MPLS L3VPN
The following set of commands will configure the MPLS backbone previously shown on the scenario with:
- OSPF for the MPLS backbone (PEs and Ps) to achieve IP connectivity and use MP-iBGP to exchange VPNv4 routing information.
- RR-P4 will be the Route-Reflector for the backbone.
- LDP (Label Distribution Protocol) which will be responsible for generate and distribute labels between routers.
PE1
hostname PE1
!
interface Loopback0
ip address 10.255.255.1 255.255.255.255
!
interface Ethernet0/0
ip address 10.1.4.1 255.255.255.0
ip ospf network point-to-point
!
interface Ethernet0/1
ip address 10.1.2.1 255.255.255.0
ip ospf network point-to-point
!
interface Ethernet0/2
ip address 10.1.5.1 255.255.255.0
ip ospf network point-to-point
!
router ospf 65512
mpls ldp autoconfig
network 10.0.0.0 0.255.255.255 area 0
!
mpls ldp router-id Loopback0
!
router bgp 65512
bgp log-neighbor-changes
no bgp default ipv4-unicast
neighbor 10.255.255.4 remote-as 65512
neighbor 10.255.255.4 update-source Loopback0
!
address-family ipv4
exit-address-family
!
address-family vpnv4
neighbor 10.255.255.4 activate
neighbor 10.255.255.4 send-community extended
exit-address-family
!
P2
hostname P2
!
interface Loopback0
ip address 10.255.255.2 255.255.255.255
!
interface Ethernet0/0
ip address 10.2.5.2 255.255.255.0
ip ospf network point-to-point
!
interface Ethernet0/1
ip address 10.1.2.2 255.255.255.0
ip ospf network point-to-point
!
interface Ethernet0/2
ip address 10.2.4.2 255.255.255.0
ip ospf network point-to-point
!
interface Ethernet1/1
ip address 10.2.3.2 255.255.255.0
ip ospf network point-to-point
!
interface Ethernet1/2
ip address 10.2.6.2 255.255.255.0
ip ospf network point-to-point
!
router ospf 65512
mpls ldp autoconfig
network 10.0.0.0 0.255.255.255 area 0
!
mpls ldp router-id Loopback0
!
router bgp 65512
bgp log-neighbor-changes
no bgp default ipv4-unicast
neighbor 10.255.255.4 remote-as 65512
neighbor 10.255.255.4 update-source Loopback0
!
address-family ipv4
exit-address-family
!
address-family vpnv4
neighbor 10.255.255.4 activate
neighbor 10.255.255.4 send-community extended
exit-address-family
!
PE3
hostname PE3
!
interface Loopback0
ip address 10.255.255.3 255.255.255.255
!
interface Ethernet0/0
ip address 10.3.6.3 255.255.255.0
ip ospf network point-to-point
!
interface Ethernet1/1
ip address 10.2.3.3 255.255.255.0
ip ospf network point-to-point
!
interface Ethernet1/2
ip address 10.3.5.3 255.255.255.0
ip ospf network point-to-point
!
router ospf 65512
mpls ldp autoconfig
network 10.0.0.0 0.255.255.255 area 0
!
mpls ldp router-id Loopback0
!
router bgp 65512
bgp log-neighbor-changes
no bgp default ipv4-unicast
neighbor 10.255.255.4 remote-as 65512
neighbor 10.255.255.4 update-source Loopback0
!
address-family ipv4
exit-address-family
!
address-family vpnv4
neighbor 10.255.255.4 activate
neighbor 10.255.255.4 send-community extended
exit-address-family
!
!
RR-P4
hostname RR-P4
!
interface Loopback0
ip address 10.255.255.4 255.255.255.255
!
interface Ethernet0/0
ip address 10.1.4.4 255.255.255.0
ip ospf network point-to-point
!
interface Ethernet0/1
ip address 10.4.5.4 255.255.255.0
ip ospf network point-to-point
!
interface Ethernet0/2
ip address 10.2.4.4 255.255.255.0
ip ospf network point-to-point
!
router ospf 65512
mpls ldp autoconfig
network 10.0.0.0 0.255.255.255 area 0
!
mpls ldp router-id Loopback0
!
router bgp 65512
bgp log-neighbor-changes
no bgp default ipv4-unicast
neighbor 10.255.255.1 remote-as 65512
neighbor 10.255.255.1 update-source Loopback0
neighbor 10.255.255.2 remote-as 65512
neighbor 10.255.255.2 update-source Loopback0
neighbor 10.255.255.3 remote-as 65512
neighbor 10.255.255.3 update-source Loopback0
neighbor 10.255.255.5 remote-as 65512
neighbor 10.255.255.5 update-source Loopback0
neighbor 10.255.255.6 remote-as 65512
neighbor 10.255.255.6 update-source Loopback0
!
address-family ipv4
exit-address-family
!
address-family vpnv4
neighbor 10.255.255.1 activate
neighbor 10.255.255.1 send-community extended
neighbor 10.255.255.1 route-reflector-client
neighbor 10.255.255.2 activate
neighbor 10.255.255.2 send-community extended
neighbor 10.255.255.2 route-reflector-client
neighbor 10.255.255.3 activate
neighbor 10.255.255.3 send-community extended
neighbor 10.255.255.3 route-reflector-client
neighbor 10.255.255.5 activate
neighbor 10.255.255.5 send-community extended
neighbor 10.255.255.5 route-reflector-client
neighbor 10.255.255.6 activate
neighbor 10.255.255.6 send-community extended
neighbor 10.255.255.6 route-reflector-client
exit-address-family
!
P5
hostname P5
!
interface Loopback0
ip address 10.255.255.5 255.255.255.255
!
interface Ethernet0/0
ip address 10.2.5.5 255.255.255.0
ip ospf network point-to-point
!
interface Ethernet0/1
ip address 10.4.5.5 255.255.255.0
ip ospf network point-to-point
!
interface Ethernet0/2
ip address 10.1.5.5 255.255.255.0
ip ospf network point-to-point
!
interface Ethernet1/1
ip address 10.5.6.5 255.255.255.0
ip ospf network point-to-point
!
interface Ethernet1/2
ip address 10.3.5.5 255.255.255.0
ip ospf network point-to-point
!
router ospf 65512
mpls ldp autoconfig
network 10.0.0.0 0.255.255.255 area 0
!
mpls ldp router-id Loopback0
!
router bgp 65512
bgp log-neighbor-changes
no bgp default ipv4-unicast
neighbor 10.255.255.4 remote-as 65512
neighbor 10.255.255.4 update-source Loopback0
!
address-family ipv4
exit-address-family
!
address-family vpnv4
neighbor 10.255.255.4 activate
neighbor 10.255.255.4 send-community extended
exit-address-family
!
P6
hostname P6
!
interface Loopback0
ip address 10.255.255.6 255.255.255.255
!
interface Ethernet0/0
ip address 10.3.6.6 255.255.255.0
ip ospf network point-to-point
!
interface Ethernet1/1
ip address 10.5.6.6 255.255.255.0
ip ospf network point-to-point
!
interface Ethernet1/2
ip address 10.2.6.6 255.255.255.0
ip ospf network point-to-point
!
router ospf 65512
mpls ldp autoconfig
network 10.0.0.0 0.255.255.255 area 0
!
mpls ldp router-id Loopback0
!
router bgp 65512
bgp log-neighbor-changes
no bgp default ipv4-unicast
neighbor 10.255.255.4 remote-as 65512
neighbor 10.255.255.4 update-source Loopback0
!
address-family ipv4
exit-address-family
!
address-family vpnv4
neighbor 10.255.255.4 activate
neighbor 10.255.255.4 send-community extended
exit-address-family
!
Once the devices are properly configured with OSPF, LDP and MP-iBGP. It will be possible to verify connectivity between PEs, learned routes in the backbone, LDP adjacencies, and verify the MPLS Label Forwarding Information Base.
Verifying connectivity from PE1
PE1#tclsh
PE1(tcl)#foreach address {
+>10.255.255.2
+>10.255.255.3
+>10.255.255.4
+>10.255.255.5
+>10.255.255.6
+>} { ping $address source loop 0
+>}
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.255.255.2, timeout is 2 seconds:
Packet sent with a source address of 10.255.255.1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/2/5 ms
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.255.255.3, timeout is 2 seconds:
Packet sent with a source address of 10.255.255.1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/2/3 ms
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.255.255.4, timeout is 2 seconds:
Packet sent with a source address of 10.255.255.1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/2 ms
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.255.255.5, timeout is 2 seconds:
Packet sent with a source address of 10.255.255.1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/3 ms
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.255.255.6, timeout is 2 seconds:
Packet sent with a source address of 10.255.255.1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/2 ms
Verifying LDP adjacencies and the MPLS LFIB
PE1#show mpls ldp neighbor
Peer LDP Ident: 10.255.255.5:0; Local LDP Ident 10.255.255.1:0
TCP connection: 10.255.255.5.23778 - 10.255.255.1.646
State: Oper; Msgs sent/rcvd: 44/43; Downstream
Up time: 00:21:13
LDP discovery sources:
Ethernet0/2, Src IP addr: 10.1.5.5
Addresses bound to peer LDP Ident:
10.2.5.5 10.4.5.5 10.1.5.5 10.5.6.5
10.3.5.5 10.255.255.5
Peer LDP Ident: 10.255.255.4:0; Local LDP Ident 10.255.255.1:0
TCP connection: 10.255.255.4.33898 - 10.255.255.1.646
State: Oper; Msgs sent/rcvd: 44/44; Downstream
Up time: 00:21:12
LDP discovery sources:
Ethernet0/0, Src IP addr: 10.1.4.4
Addresses bound to peer LDP Ident:
10.1.4.4 10.4.5.4 10.2.4.4 10.255.255.4
Peer LDP Ident: 10.255.255.2:0; Local LDP Ident 10.255.255.1:0
TCP connection: 10.255.255.2.51933 - 10.255.255.1.646
State: Oper; Msgs sent/rcvd: 43/44; Downstream
Up time: 00:21:10
LDP discovery sources:
Ethernet0/1, Src IP addr: 10.1.2.2
Addresses bound to peer LDP Ident:
10.2.5.2 10.1.2.2 10.2.4.2 10.2.3.2
10.2.6.2 10.255.255.2
PE1#show mpls forwarding-table
Local Outgoing Prefix Bytes Label Outgoing Next Hop
Label Label or Tunnel Id Switched interface
16 16 10.255.255.6/32 0 Et0/1 10.1.2.2
16 10.255.255.6/32 0 Et0/2 10.1.5.5
17 Pop Label 10.255.255.5/32 0 Et0/2 10.1.5.5
18 Pop Label 10.255.255.4/32 0 Et0/0 10.1.4.4
19 19 10.255.255.3/32 0 Et0/1 10.1.2.2
18 10.255.255.3/32 0 Et0/2 10.1.5.5
20 Pop Label 10.255.255.2/32 0 Et0/1 10.1.2.2
21 Pop Label 10.5.6.0/24 0 Et0/2 10.1.5.5
22 Pop Label 10.4.5.0/24 0 Et0/0 10.1.4.4
Pop Label 10.4.5.0/24 0 Et0/2 10.1.5.5
23 23 10.3.6.0/24 0 Et0/1 10.1.2.2
21 10.3.6.0/24 0 Et0/2 10.1.5.5
24 Pop Label 10.3.5.0/24 0 Et0/2 10.1.5.5
25 Pop Label 10.2.6.0/24 0 Et0/1 10.1.2.2
26 Pop Label 10.2.5.0/24 0 Et0/1 10.1.2.2
Pop Label 10.2.5.0/24 0 Et0/2 10.1.5.5
27 Pop Label 10.2.4.0/24 0 Et0/1 10.1.2.2
Pop Label 10.2.4.0/24 0 Et0/0 10.1.4.4
28 Pop Label 10.2.3.0/24 0 Et0/1 10.1.2.2
Our MPLS backbone is already configured, the next step will be the configuration of VRF FARMA (our customer that appear on the scenario) on PE1 and PE3, the associated interfaces with the VRF, the routing protocol used between CEs and PEs to exchange routing information (in this case EIGRP with AS101).
Defining the VRF and the associated interfaces on PE1 and PE3.
! On PE1
vrf definition FARMA
rd 65512:30012
!
address-family ipv4
route-target export 65512:30012
route-target import 65512:30012
exit-address-family
!
!
interface Ethernet0/3
vrf forwarding FARMA
ip address 172.16.11.1 255.255.255.0
!
! On PE3
vrf definition FARMA
rd 65512:30012
!
address-family ipv4
route-target export 65512:30012
route-target import 65512:30012
exit-address-family
!
!
interface Ethernet0/3
vrf forwarding FARMA
ip address 172.16.23.1 255.255.255.0
!
Configuring the CEs and PEs with EIGRP
! PE1
router eigrp AS65512
!
address-family ipv4 unicast vrf FARMA autonomous-system 101
!
topology base
redistribute connected
exit-af-topology
network 172.16.11.0 0.0.0.255
exit-address-family
!
! CE1
interface Loopback18
ip address 172.18.11.2 255.255.255.0
!
interface Ethernet0/3
ip address 172.16.11.2 255.255.255.0
!
router eigrp 101
network 0.0.0.0
!
! PE3
router eigrp AS65512
!
address-family ipv4 unicast vrf FARMA autonomous-system 101
!
topology base
redistribute connected
exit-af-topology
network 172.16.23.0 0.0.0.255
exit-address-family
!
! CE2
interface Loopback18
ip address 172.18.23.2 255.255.255.0
!
interface Ethernet0/3
ip address 172.16.23.2 255.255.255.0
!
router eigrp 101
network 0.0.0.0
!
At this point, the EIGRP adjacencies are established between CEs and PEs, but the exchange of routing information through the two PEs is not occurring.
Verifying the PEs
PE1#show ip eigrp vrf FARMA neighbors
EIGRP-IPv4 VR(AS65512) Address-Family Neighbors for AS(101)
VRF(FARMA)
H Address Interface Hold Uptime SRTT RTO Q Seq
(sec) (ms) Cnt Num
0 172.16.11.2 Et0/3 12 01:12:28 7 100 0 4
PE1#show ip eigrp vrf FARMA topology
EIGRP-IPv4 VR(AS65512) Topology Table for AS(101)/ID(172.16.11.1)
Topology(base) TID(0) VRF(FARMA)
Codes: P - Passive, A - Active, U - Update, Q - Query, R - Reply,
r - reply Status, s - sia Status
P 172.18.11.0/24, 1 successors, FD is 458752000
via 172.16.11.2 (458752000/327761920), Ethernet0/3
P 172.16.11.0/24, 1 successors, FD is 131072000
via Connected, Ethernet0/3
PE3#show ip eigrp vrf FARMA neighbors
EIGRP-IPv4 VR(AS65512) Address-Family Neighbors for AS(101)
VRF(FARMA)
H Address Interface Hold Uptime SRTT RTO Q Seq
(sec) (ms) Cnt Num
0 172.16.23.2 Et0/3 11 01:13:55 1023 5000 0 4
PE3#show ip eigrp vrf FARMA topology
EIGRP-IPv4 VR(AS65512) Topology Table for AS(101)/ID(172.16.23.1)
Topology(base) TID(0) VRF(FARMA)
Codes: P - Passive, A - Active, U - Update, Q - Query, R - Reply,
r - reply Status, s - sia Status
P 172.18.23.0/24, 1 successors, FD is 458752000
via 172.16.23.2 (458752000/327761920), Ethernet0/3
P 172.16.23.0/24, 1 successors, FD is 131072000
via Connected, Ethernet0/3
At the beginning of the explanation we had configured the address-family VPNv4 between PEs, Ps and the RR, so to establish the exchange of routing information between the two PEs (PE1 and PE3), will be necessary to configure an address-family IPv4 associated with the VRF FARMA within the BGP, and make the redistribution between EIGRP and BGP.
The missing part
!
router eigrp AS65512
!
address-family ipv4 unicast vrf FARMA autonomous-system 101
!
topology base
redistribute bgp 65512 metric 1000 100 255 100 1500
redistribute connected
exit-af-topology
exit-address-family
!
router bgp 65512
!
address-family ipv4 vrf FARMA
redistribute connected
redistribute eigrp 101
exit-address-family
!
Finally, is it possible to verify that the two PEs are exchanging routing information associated with the VRF FARMA, and inclusive it is possible to verify connectivity between the two sites through a traceroute.
Verifying the VPNv4 routes on the PEs and RR
PE1#show ip bgp vpnv4 all
BGP table version is 17, local router ID is 10.255.255.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
x best-external, a additional-path, c RIB-compressed,
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 65512:30012 (default for vrf FARMA)
*> 172.16.11.0/24 0.0.0.0 0 32768 ?
*>i 172.16.23.0/24 10.255.255.3 0 100 0 ?
*> 172.18.11.0/24 172.16.11.2 3584000 32768 ?
*>i 172.18.23.0/24 10.255.255.3 3584000 100 0 ?
PE3#show ip bgp vpnv4 all
BGP table version is 17, local router ID is 10.255.255.3
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
x best-external, a additional-path, c RIB-compressed,
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 65512:30012 (default for vrf FARMA)
*>i 172.16.11.0/24 10.255.255.1 0 100 0 ?
*> 172.16.23.0/24 0.0.0.0 0 32768 ?
*>i 172.18.11.0/24 10.255.255.1 3584000 100 0 ?
*> 172.18.23.0/24 172.16.23.2 3584000 32768 ?
RR-P4#show ip bgp vpnv4 all
BGP table version is 13, local router ID is 10.255.255.4
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
x best-external, a additional-path, c RIB-compressed,
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 65512:30012
*>i 172.16.11.0/24 10.255.255.1 0 100 0 ?
*>i 172.16.23.0/24 10.255.255.3 0 100 0 ?
*>i 172.18.11.0/24 10.255.255.1 3584000 100 0 ?
*>i 172.18.23.0/24 10.255.255.3 3584000 100 0 ?
Verifying connectivity between two sites
CE1#show ip route
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
a - application route
+ - replicated route, % - next hop override
Gateway of last resort is not set
172.16.0.0/16 is variably subnetted, 3 subnets, 2 masks
C 172.16.11.0/24 is directly connected, Ethernet0/3
L 172.16.11.2/32 is directly connected, Ethernet0/3
D 172.16.23.0/24 [90/307200] via 172.16.11.1, 00:22:11, Ethernet0/3
172.18.0.0/16 is variably subnetted, 3 subnets, 2 masks
C 172.18.11.0/24 is directly connected, Loopback18
L 172.18.11.2/32 is directly connected, Loopback18
D 172.18.23.0/24 [90/435200] via 172.16.11.1, 00:22:11, Ethernet0/3
CE1#ping 172.18.23.2
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 172.18.23.2, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 2/2/3 ms
CE1#traceroute 172.18.23.2 source loopback 18
Type escape sequence to abort.
Tracing the route to 172.18.23.2
VRF info: (vrf in name/id, vrf out name/id)
1 172.16.11.1 4 msec 1 msec 1 msec
2 10.1.2.2 [MPLS: Labels 19/30 Exp 0] 2 msec 2 msec 2 msec
3 172.16.23.1 [MPLS: Label 30 Exp 0] 2 msec 2 msec 4 msec
4 172.16.23.2 3 msec * 5 msec
__
References