MPLS L3VPN: Carrier Supporting Carrier

Friday. April 26, 2019

CCIE

The previous posts were used to explain the different options available to establish an Inter-AS MPLS L3VPN. There is another technology called Carrier Supporting Carrier (CSC) that takes inter-AS L3VPN to the next level.

Suppose that you have different ASes that belongs to the same overall authority (i.e. Telefonica), such as Service Provider with ASes in different regions of a country or the world. CSC enables to use the cloud of another large service provider (i.e. CenturyLink) in order to connect parts of their own networks, using the transport services provided by another service provider in this manner eliminates the need for the small service providers to build and maintain their own MPLS backbone.

Topology

With Carrier Supporting Carrier (CSC), there are a two new terms to deal with:

Carrier Supporting Carrier - Customer Edge (CSC-CE): Is the device located in the customer carriers network, connecting to the backbone carrier.
Carrier Supporting Carrier - Provider Edge (CSC-PE): Is the device located in the backbone carriers network connecting to the customer carrier.

This is much like option C in the inter-AS in which two PE routers peer with each other even though they are not directly connected in the same AS. In order to do so we need each of them to have a valid route to each other. As each CSC-CE on our topology have routes to their local PE device, they need to learn routes from the other side through the core carrier.

Carrier Supporting Carrier (CSC) has some characteristics that define it:

Route Reflectors exchange VPNv4 routes + next-hop-unchanged.
Labelled eBGP IPv4 between CSC-CE and CSC-PE. Remembering that we need to use the as-override command on the CSC-PE if the CSC-CE routers are using the same AS.
The eBGP between the CSC-PE and CSC-CE routers is like normal MPLS L3VPN PE-CE eBGP under the address-family ipv4 vrf on the CSC-PE side but adding the neighbor <address> send-label command and the mpls bgp forwarding interface command on both sides.
CSC-CEs only contain the addresses of remote PEs for BGP next-hop reachability of VPN routes.
The loopback prefixes received from eBGP must be redistributed into IGP (in each Service Provider) for other PEs to resolved the VPNv4 next-hop between ASes.
End-to-end LSP.
The Carrier Supporting Carrier (CSC) can accommodate unplanned growth and changes. CSC enables tens of thousands of VPNs to be set up over the same network, and it allows a service provider to offer both VPN and Internet services.
The backbone carrier can accommodate customer carriers that require security and different bandwidths.

Demonstrating: Carrier Supporting Carrier

The following set of commands will be used to demonstrate the basic configuration to establish Carrier Supporting Carrier, and assuming that the two MPLS backbones of each Service Providers are already configured, as well as, the backbone carrier.

PEs in both Service Providers are configured with the same Route Distinguisher (RD) and Route Target (RT) for the customer FARMA.

On AS6306

hostname CE1
!
interface Loopback0
 ip address 192.168.100.101 255.255.255.255
!
interface Ethernet0/3
 description CE-PE
 ip address 192.168.21.2 255.255.255.0
!
!
router eigrp 101
 network 0.0.0.0
 eigrp router-id 192.168.100.101
!

hostname PE1
!
vrf definition FARMA
 rd 6306:12956
 !
 address-family ipv4
  route-target export 6306:12956
  route-target import 6306:12956
 exit-address-family
!
!
interface Ethernet0/3
 vrf forwarding FARMA
 ip address 192.168.21.1 255.255.255.0
!
router eigrp AS6306
!
!
address-family ipv4 unicast vrf FARMA autonomous-system 101
  !
  topology base
   redistribute bgp 6306 metric 1000 10 255 1 1500
  exit-af-topology
  network 192.168.21.0
 exit-address-family
!
router bgp 6306
 !
 address-family ipv4 vrf FARMA
  redistribute connected
  redistribute eigrp 101
 exit-address-family
!

hostname RR1
!
interface Loopback0
 ip address 10.255.255.1 255.255.255.255
!
!
router bgp 6306
 bgp router-id 10.255.255.1
 bgp log-neighbor-changes
 no bgp default ipv4-unicast
 neighbor 10.255.255.2 remote-as 6306
 neighbor 10.255.255.2 update-source Loopback0
 neighbor 10.255.255.3 remote-as 6306
 neighbor 10.255.255.3 update-source Loopback0
 neighbor 10.255.255.10 remote-as 12956
 neighbor 10.255.255.10 ebgp-multihop 255
 neighbor 10.255.255.10 update-source Loopback0
 !
 address-family ipv4
 exit-address-family
 !
 address-family vpnv4
  neighbor 10.255.255.2 activate
  neighbor 10.255.255.2 send-community extended
  neighbor 10.255.255.2 route-reflector-client
  neighbor 10.255.255.3 activate
  neighbor 10.255.255.3 send-community extended
  neighbor 10.255.255.3 route-reflector-client
  neighbor 10.255.255.10 activate
  neighbor 10.255.255.10 send-community extended
  neighbor 10.255.255.10 next-hop-unchanged
 exit-address-family
!

hostname CSC-CE1
!
interface Ethernet0/3
 description CSC-CEAS6306-CSC-PEAS3549
 ip address 200.71.188.2 255.255.255.252
 mpls bgp forwarding
!
!
router ospf 6306
 mpls ldp autoconfig
 redistribute bgp 6306 subnets route-map REDISTRIBUTE_IN_OSPF
!
!
router bgp 6306
 bgp log-neighbor-changes
 no bgp default ipv4-unicast
 no bgp default route-target filter
 neighbor 200.71.188.1 remote-as 3549
 !
 address-family ipv4
  network 10.255.255.1 mask 255.255.255.255
  network 10.255.255.2 mask 255.255.255.255
  neighbor 200.71.188.1 activate
  neighbor 200.71.188.1 send-label
 exit-address-family
!
ip prefix-list FOREIGN_PREFIXES seq 10 permit 10.255.255.10/32
ip prefix-list FOREIGN_PREFIXES seq 20 permit 10.255.255.13/32
!
route-map REDISTRIBUTE_IN_OSPF permit 10
 match ip address prefix-list FOREIGN_PREFIXES
!

On AS12956

hostname CE2
!
interface Loopback0
 ip address 192.168.200.101 255.255.255.255
!
interface Ethernet0/3
 description CE-PE
 ip address 192.168.132.2 255.255.255.0
!
!
router eigrp 101
 network 0.0.0.0
 eigrp router-id 192.168.200.101
!

hostname PE2
!
vrf definition FARMA
 rd 6306:12956
 !
 address-family ipv4
  route-target export 6306:12956
  route-target import 6306:12956
 exit-address-family
!
!
interface Ethernet0/3
 vrf forwarding FARMA
 ip address 192.168.132.1 255.255.255.0
!
router eigrp AS12956
!
!
address-family ipv4 unicast vrf FARMA autonomous-system 101
  !
  topology base
   redistribute bgp 12956 metric 1000 10 255 1 1500
  exit-af-topology
  network 192.168.132.0
 exit-address-family
!
router bgp 12956
 !
 address-family ipv4 vrf FARMA
  redistribute connected
  redistribute eigrp 101
 exit-address-family
!

hostname RR2
!
interface Loopback0
 ip address 10.255.255.10 255.255.255.255
!
!
router bgp 12956
 bgp log-neighbor-changes
 no bgp default ipv4-unicast
 neighbor 10.255.255.1 remote-as 6306
 neighbor 10.255.255.1 ebgp-multihop 255
 neighbor 10.255.255.1 update-source Loopback0
 neighbor 10.255.255.12 remote-as 12956
 neighbor 10.255.255.12 update-source Loopback0
 neighbor 10.255.255.13 remote-as 12956
 neighbor 10.255.255.13 update-source Loopback0
 !
 address-family ipv4
 exit-address-family
 !
 address-family vpnv4
  neighbor 10.255.255.1 activate
  neighbor 10.255.255.1 send-community extended
  neighbor 10.255.255.1 next-hop-unchanged
  neighbor 10.255.255.12 activate
  neighbor 10.255.255.12 send-community extended
  neighbor 10.255.255.12 route-reflector-client
  neighbor 10.255.255.13 activate
  neighbor 10.255.255.13 send-community extended
  neighbor 10.255.255.13 route-reflector-client
 exit-address-family
!

hostname CSC-CE2
!
interface Ethernet0/3
 description CSC-CEAS12956-CSC-PEAS3549
 ip address 190.66.4.2 255.255.255.252
 mpls bgp forwarding
!
!
router ospf 12956
 mpls ldp autoconfig
 redistribute bgp 12956 subnets route-map REDISTRIBUTE_IN_OSPF
!
router bgp 12956
 bgp log-neighbor-changes
 no bgp default ipv4-unicast
 no bgp default route-target filter
 neighbor 190.66.4.1 remote-as 3549
 !
 address-family ipv4
  network 10.255.255.10 mask 255.255.255.255
  network 10.255.255.13 mask 255.255.255.255
  neighbor 190.66.4.1 activate
  neighbor 190.66.4.1 send-label
 exit-address-family
!
ip prefix-list FOREIGN_PREFIXES seq 10 permit 10.255.255.1/32
ip prefix-list FOREIGN_PREFIXES seq 20 permit 10.255.255.2/32
!
route-map REDISTRIBUTE_IN_OSPF permit 10
 match ip address prefix-list FOREIGN_PREFIXES
!

On AS3549

hostname CSC-PE
!
vrf definition CSC-FARMA
 rd 3549:12956
 !
 address-family ipv4
  route-target export 3549:12956
  route-target import 3549:12956
 exit-address-family
!
!
interface Ethernet0/3
 vrf forwarding CSC-FARMA
 ip address 190.66.4.1 255.255.255.252
 mpls bgp forwarding
!
router bgp 3549
 bgp log-neighbor-changes
 no bgp default ipv4-unicast
 neighbor 10.255.255.8 remote-as 3549
 neighbor 10.255.255.8 update-source Loopback0
 !
 address-family ipv4
 exit-address-family
 !
 address-family vpnv4
  neighbor 10.255.255.8 activate
  neighbor 10.255.255.8 send-community extended
  neighbor 10.255.255.8 next-hop-self
 exit-address-family
 !
 address-family ipv4 vrf CSC-FARMA
  neighbor 190.66.4.2 remote-as 12956
  neighbor 190.66.4.2 activate
  neighbor 190.66.4.2 send-label
 exit-address-family
!

Both CSC-PE will act as PE routers for the backbone carrier. They will have their interfaces in a VRF and running a regular BGP VPNv4 with each other.

The other CSC-PE has a similar configuration on the other side, so it makes no sense to put it here.

Verifying end-to-end reachability between CEs

CE1# show ip route
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
       a - application route
       + - replicated route, % - next hop override

Gateway of last resort is not set

      192.168.21.0/24 is variably subnetted, 2 subnets, 2 masks
C        192.168.21.0/24 is directly connected, Ethernet0/3
L        192.168.21.2/32 is directly connected, Ethernet0/3
      192.168.100.0/32 is subnetted, 1 subnets
C        192.168.100.101 is directly connected, Loopback100
D     192.168.132.0/24 [90/307200] via 192.168.21.1, 00:53:45, Ethernet0/3
      192.168.200.0/32 is subnetted, 1 subnets
D        192.168.200.101 [90/435200] via 192.168.21.1, 00:53:45, Ethernet0/3

CE1# ping 192.168.200.101 source 192.168.100.101
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.200.101, timeout is 2 seconds:
Packet sent with a source address of 192.168.100.101
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 3/3/4 ms

CE1# traceroute 192.168.200.101 source 192.168.100.101
Type escape sequence to abort.
Tracing the route to 192.168.200.101
VRF info: (vrf in name/id, vrf out name/id)
  1 192.168.21.1 2 msec 5 msec 8 msec
  2 10.1.2.1 [MPLS: Labels 23/23 Exp 0] 5 msec 5 msec 3 msec
  3 10.1.3.3 [MPLS: Labels 23/23 Exp 0] 3 msec 5 msec 6 msec
  4 10.3.4.4 [MPLS: Labels 24/23 Exp 0] 4 msec 6 msec 5 msec
  5 200.71.188.1 [MPLS: Labels 25/23 Exp 0] 3 msec 2 msec 4 msec
  6 10.6.7.7 [MPLS: Labels 16/23/23 Exp 0] 4 msec 4 msec 4 msec
  7 10.7.8.8 [MPLS: Labels 16/23/23 Exp 0] 2 msec 3 msec 4 msec
  8 10.8.9.9 [MPLS: Labels 23/23 Exp 0] 4 msec 2 msec 3 msec
  9 190.66.4.2 [MPLS: Labels 17/23 Exp 0] 2 msec 7 msec 3 msec
 10 10.10.11.10 [MPLS: Labels 16/23 Exp 0] 3 msec 7 msec 5 msec
 11 10.10.12.12 [MPLS: Labels 16/23 Exp 0] 3 msec 3 msec 3 msec
 12 192.168.132.1 [MPLS: Label 23 Exp 0] 5 msec 3 msec 2 msec
 13 192.168.132.2 2 msec *  8 msec

With a simple traceroute between CEs we can see there is a single LSP (Label Switched Path) used to provide end-to-end reachability, this LSP is using the label 23. Also, at this point we are up to a three label stack in AS3549.

Verifying LSP between PE1 and PE2

With these types of configurations, we need to ensure the control plane is working properly. Verifying the update of CE1’s loopback over to CE2. The PE1 router should be learning this as a VRF prefix.

PE1# show ip route vrf FARMA 192.168.100.101

Routing Table: FARMA
Routing entry for 192.168.100.101/32
  Known via "eigrp 101", distance 90, metric 3584000, type internal
  Redistributing via eigrp 101, bgp 6306
  Advertised by bgp 6306
  Last update from 192.168.21.2 on Ethernet0/3, 01:10:51 ago
  Routing Descriptor Blocks:
  * 192.168.21.2, from 192.168.21.2, 01:10:51 ago, via Ethernet0/3
      Route metric is 3584000, traffic share count is 1
      Total delay is 6000 microseconds, minimum bandwidth is 10000 Kbit
      Reliability 255/255, minimum MTU 1500 bytes
      Loading 1/255, Hops 1

The prefix is converted into a VPNv4 prefix and advertised over to PE2, and should end up in the correct VRF table.

PE2# show bgp vpnv4 unicast vrf FARMA 192.168.100.101
BGP routing table entry for 6306:12956:192.168.100.101/32, version 7
Paths: (1 available, best #1, table FARMA)
  Not advertised to any peer
  Refresh Epoch 1
  6306
    10.255.255.2 (metric 1) from 10.255.255.10 (10.255.255.10)
      Origin incomplete, metric 0, localpref 100, valid, internal, best
      Extended Community: RT:6306:12956 0x8800:32768:0 0x8801:101:153600
        0x8802:65281:256000 0x8803:65281:1500 0x8806:0:3232261221
      mpls labels in/out nolabel/25
      rx pathid: 0, tx pathid: 0x0

PE2# show ip route vrf FARMA 192.168.100.101

Routing Table: FARMA
Routing entry for 192.168.100.101/32
  Known via "bgp 12956", distance 200, metric 0
  Tag 6306, type internal
  Redistributing via eigrp 101
  Advertised by eigrp 101 metric 1000 10 255 1 1500
  Last update from 10.255.255.2 01:21:06 ago
  Routing Descriptor Blocks:
  * 10.255.255.2 (default), from 10.255.255.10, 01:21:06 ago
      Route metric is 0, traffic share count is 1
      AS Hops 1
      Route tag 6306
      MPLS label: 25
      MPLS Flags: MPLS Required

PE2# show bgp vpnv4 unicast all labels
   Network          Next Hop      In label/Out label
Route Distinguisher: 6306:12956 (FARMA)
   192.168.21.0       10.255.255.2    nolabel/24
   192.168.100.101/32 10.255.255.2    nolabel/25
   192.168.132.0      0.0.0.0         22/nolabel(FARMA)
   192.168.200.101/32 192.168.132.2   23/nolabel

References

RFC4364 BGP/MPLS IP Virtual Private Networks - Carrier Supporting Carrier