ITU-T G.8032 Between DM2100 and ASR903

Thursday. July 27, 2017

DATACOM

Ethernet Ring Protection Switching (ERPS) is based on ITU-T G.8032 recommendation and is one of the most emerging and proven technology to prevent loop formation in Ethernet ring. It enables ring to converge in 50 ms from node or link failure.

An Ethernet ring consists of multiple Ethernet ring nodes. Each Ethernet ring node is connected to adjacent Ethernet ring nodes using two independent ring links. A ring link prohibits formation of loops that affect the network. The Ethernet ring uses a specific link to protect the entire Ethernet ring. This specific link is called the Ring Protection Link (RPL). A ring link is bound by two adjacent Ethernet ring nodes and a port for a ring link (also known as a ring port). There must be at least two Ethernet ring nodes in a Ethernet ring.

How ITU-T G.8032 Ethernet Ring Protection Switching protocol works?

The Ethernet Ring Protection Switching (ERPS) provide the following:

Loop avoidance
Utilization of learning, forwarding and address table mechanisms as defined in the Ethernet flow forwarding function

Loop avoidance in an Ethernet ring is achieved by ensuring that, at any time, traffic flows on all interfaces except the Ring Protection Link (RPL). Exists different types of RPL nodes, and as is shown below with their functions:

RPL owner - Responsible for blocking traffic over the RPL so that no loops are formed in the Ethernet traffic and can be only one RPL owner in a ring.
RPL neighbor node - An Ethernet ring node adjacent to the RPL. It is responsible for blocking its end of the RPL under normal conditions. This node type is optional and prevents RPL usage when protected, several times this configuration depends on vendors implementation.

All Nodes that belongs to the ring use a mechanism control called Ring Automatic Protection Switching (R-APS) messages to coordinate the activities of switching the ring protection link (RPL) on and off. Any failure along the ring triggers a R-APS Signal Failure (R-APS SF) message in both directions of the nodes adjacent to the failed link, after the nodes have blocked the port facing the failed link. On obtaining this message, the RPL owner unblocks the RPL port.

After a failed link is detected and when this is restored, the nodes adjacent to the restored link send Ring Automatic Protection Switching (R-APS) No Request (R-APS NR) messages. On obtaining this message, the ring protection link (RPL) owner blocks the RPL port and sends R-APS NR and R-APS RPL (R-APS NR, RB) messages. These messages cause all other nodes, other than the RPL owner in the ring, to unblock all blocked ports. The Ethernet Ring Protection Switching (ERPS) protocol works for both unidirectional failure and multiple link failure scenarios in a ring topology.

The ITU-T G.8032 Ethernet Ring Protection Switching (ERPS) can support multiple instances. An instance is a logical ring running over a physical ring. Such instances are used for various reasons, such as load-balancing of protected VLANs over a ring. For example, odd-numbered VLANs may go in one direction of the ring, and even-numbered VLANs may go in the other direction. Specific VLANs can be configured under only one instance. They cannot overlap multiple instances. Otherwise, data traffic or Ring Automatic Protection Switching (R-APS) messages may cross logical rings, which is not desirable.

ITU-T G.8032 Timers

The ITU-T G.8032 Ethernet Ring Protection Switching (ERPS) protocol specifies the use of different timers to avoid race conditions and unnecessary switching operations:

After a signal failure (SF) condition, a Wait-to-Restore (WTR) timer is used to verify that the SF is not intermittent. The WTR timer can be configured by the operator. The default time interval is 5 minutes, depending on vendors implementations the time interval ranges can be configured from 1 to 12 minutes.
Guard Timer - It’s the time period which is used to prevent reception of outdated R-APS messages by nodes in ring. This prevents the switching of traffic to protected path due to erroneous interpretation of outdated R-APS messages. It can be configured in 10 ms steps between 10 ms and 2 seconds, with a default value of 500 ms.
Holdoff Timer - Used by the underlying Ethernet layer to filter out intermittent link faults. The hold-off timer can be configured. The default time interval is 0 seconds, the time interval ranges is from 0 to 10 seconds. Faults are reported to the ring protection mechanism only if this timer expires.

Interoperability: ITU-T G.8032 Between DATACOM DM2100 and ASR903

Topology

In normal condition, the RPL owner of the ring, blocks traffic on the protected VLANs by blocking its either port0 (GigabitEthernet 0/0/6) or port1 (GigabitEthernet 0/0/7). In figure shown, the RPL owner of ASR903 has blocked its port0 for traffic to prevent loop formation and allows traffic to flow in clockwise direction.

Configuring RPL Owner

The following set of commands will configure the ASR 903 with:

Control VLAN id 500 (used for R-APS messages)
VLANs 50 to 100 as protected VLANs for data traffic
Interface GigabitEthernet 0/0/6 as RPL.

!
ethernet cfm ieee
ethernet cfm global
!
ethernet evc e500
!
interface GigabitEthernet0/0/6
 no ip address
 negotiation auto
 service instance trunk 50 ethernet
  encapsulation dot1q 50-100
  rewrite ingress tag pop 1 symmetric
  bridge-domain from-encapsulation
 !
 service instance 500 ethernet e500
  encapsulation dot1q 500
  bridge-domain 500
 !
 interface GigabitEthernet0/0/7
 no ip address
 negotiation auto
 service instance trunk 50 ethernet
  encapsulation dot1q 50-100
  rewrite ingress tag pop 1 symmetric
  bridge-domain from-encapsulation
 !
 service instance 500 ethernet e500
  encapsulation dot1q 500
  bridge-domain 500
 !
 !
ethernet ring g8032 DATACOM
 port0 interface GigabitEthernet0/0/6
 port1 interface GigabitEthernet0/0/7
 instance 1
  rpl port0 owner
  inclusion-list vlan-ids 50-100,500
  aps-channel
   level 0
   port0 service instance 500
   port1 service instance 500
  !
 !
!

Note: All nodes in the Ethernet ring must be configured with the same level of R-APS messages, by default DATACOM use the R-APS messages with level 0

Configuring Transit Nodes

The following set of commands will configure the both DM2100 EDD SII with:

Control VLAN id 500 (used for R-APS messages)
VLANs 50 to 100 as protected VLANs for data traffic
ERPS Instance configured as transit node

!
interface vlan range 50 100
 set-member tagged ethernet 1/1
 set-member tagged ethernet 1/6
!
interface vlan 500
 set-member tagged ethernet 1/1
 set-member tagged ethernet 1/6
!
vlan-group 1
vlan-group 1 vlan range 50 100
!
interface ethernet 1/1
 no switchport storm-control dlf
 no loopback-detection
!
interface ethernet 1/6
 no switchport storm-control dlf
 no loopback-detection
!
erps 1
erps 1 control-vlan id 500
erps 1 name DATACOM
erps 1 port0 node ethernet 1/1
erps 1 port1 node ethernet 1/6
erps 1 protected-vlans vlan-group 1
!

Verifying the ITU-T G.8032 Status

Once devices are properly configured with ITU-T G.8032 Ethernet Ring Protection Switching, protocol status should be PROTECTION, PENDING or IDLE on the RPL Owner (ASR 903), on the transit nodes EDD-Node-A and EDD-Node-B should be PROTECTION or IDLE

ASR903# show ethernet ring g8032 status
Ethernet ring DATACOM instance 1 is RPL Owner node in Idle State
 Port0: GigabitEthernet0/0/6 (Monitor: GigabitEthernet0/0/6)
  APS-Channel: GigabitEthernet0/0/6
  Status: RPL, blocked
  Remote R-APS NodeId: 0000.0000.0000, BPR: 0
 Port1: GigabitEthernet0/0/7 (Monitor: GigabitEthernet0/0/7)
  APS-Channel: GigabitEthernet0/0/7
  Status: Non-RPL
  Remote R-APS NodeId: 0000.0000.0000, BPR: 0
 APS Level: 0
 Profile:
  WTR interval: 5 minutes
  Guard interval: 500 milliseconds
  HoldOffTimer: 0 seconds
  Revertive mode

In normal condition the protocol status is IDLE, but If the status is PROTECTED, that indicate a link failure in some place over the Ethernet ring. When the protocol status is PENDING, that indicate that is waiting to finish the WTR timer to recover the link and block the RPL interface.

EDD-Node-A# show erps detail
Domain ID:                        1
 Domain Name:                     DATACOM
 State:                           IDLE
 Mode:                            Transit
 HW Forwarding:                   Disabled
 Guard Timer (ms):                500
 WTR Timer (min):                 5
 Holdoff Timer (ms):              0
 Port 0:                          Eth1/1                Port status: Unblocked
 Port 1:                          Eth1/6                Port status: Unblocked
 Control VLAN ID:                 500
 Protected VLAN group IDs:        0
 Accept topology change of domains:

EDD-Node-B# show erps detail
Domain ID:                        1
 Domain Name:                     DATACOM
 State:                           IDLE
 Mode:                            Transit
 HW Forwarding:                   Disabled
 Guard Timer (ms):                500
 WTR Timer (min):                 5
 Holdoff Timer (ms):              0
 Port 0:                          Eth1/1                Port status: Unblocked
 Port 1:                          Eth1/6                Port status: Unblocked
 Control VLAN ID:                 500
 Protected VLAN group IDs:        0
 Accept topology change of domains:

Demonstrating the convergence time of sub 50ms

To demonstrate the convergence time of sub 50ms, it’s necessary to configure an additional interface with a VLAN belonging to the group of protected VLANs on the equipments where the traffic generator (IXIA N2X) will be connected, in this case, VLAN id 50 with the interfaces GigabitEthernet 0/0/5 on the ASR 903 and Ethernet 1/5 on the EDD SII DM2100 Node-A, to then generate 1000 fps, and monitor how many packets are dropped during the link failure, as well as, after the WTR timer when the link is recovered.

Topology

When the nodes A and B (as shown in figure) senses link failure in between them, they will block their Ethernet 1/1 respectively for traffic and trigger signal fail (SF) to their adjacent node, in this case the ASR 903.

Jul  27 00:19:39 : ERPS sending a SF event on <Eth1/1>
Jul  27 00:19:39 : ERPS Event: domain 0 (DATACOM) started Holdoff timer on <Eth1/1>
Jul  27 00:19:39 : ERPS Event: domain 0 (DATACOM) detected Local Signal Failure on <Eth1/1>
Jul  27 00:19:39 : ERPS State change: domain 0 (DATACOM) Blocked on <Eth1/1>
Jul  27 00:19:39 : ERPS TX: 3 R-APS(SF)  for ERP id = 0 DST = 01:19:A7:00:00:01
Jul  27 00:19:39 : ERPS Flush: ERP domain 0 (DATACOM) clearing L2/L3 FDB
Jul  27 00:19:39 : ERPS Flush: ERP domain 0 (DATACOM) clearing L2/L3 FDB
Jul  27 00:19:39 : ERPS State change: domain 0 (DATACOM) Performed State IDLE with Input LOCAL_SF
Jul  27 00:19:39 : ERPS State change: domain 0 (DATACOM) now at State PROTECTION

EDD-Node-A(config-if-eth-1/1)# show erps detail
Domain ID:                        1
 Domain Name:                     DATACOM
 State:                           PROTECTION
 Mode:                            Transit
 HW Forwarding:                   Disabled
 Guard Timer (ms):                500
 WTR Timer (min):                 5
 Holdoff Timer (ms):              0
 Port 0:                          Eth1/1                Port status: Blocked
 Port 1:                          Eth1/6                Port status: Unblocked
 Control VLAN ID:                 500
 Protected VLAN group IDs:        1
 Accept topology change of domains:

Topology Convergence time during the link failure was 39ms

Through debugging commands of the ITU-T G.8032 protocol, is possible observe the SF messages arriving to the RPL Owner

ASR903# debug ethernet ring g8032 packets
*Jul 27 00:19:39.554: PKT: Read VLAN 500 from VLAN header on GigabitEthernet0/0/7
*Jul 27 00:19:39.558: PKT: GigabitEthernet0/0/7 rcvd v1 R-APS(SF)[0]: req B0 status 00, node_id 0004.DFC9.E918
*Jul 27 00:19:39.558: PKT: Read VLAN 500 from VLAN header on GigabitEthernet0/0/7
*Jul 27 00:19:39.559: PKT: GigabitEthernet0/0/7 rcvd v1 R-APS(SF)[0]: req B0 status 00, node_id 0004.DFC9.E918
*Jul 27 00:19:39.559: PKT: Read VLAN 500 from VLAN header on GigabitEthernet0/0/7
*Jul 27 00:19:39.566: PKT: port0:GigabitEthernet0/0/6, encap: 11, vlan1: 500, vlan2: 0
*Jul 27 00:19:39.566: PKT: port1:GigabitEthernet0/0/7, encap: 11, vlan1: 500, vlan2: 0
*Jul 27 00:19:39.566: PKT: sent 3 APS packet R-APS(SF): req_state B0, status 00, node_id 1CE8.5DC4.B6BF over interfaces GigabitEthernet0/0/6 and GigabitEthernet0/0/7
*Jul 27 00:19:39.572: ERP_G8032-6-STATE_CHANGED: Ethernet ring DATACOM instance 1 changed state to Protection
*Jul 27 00:19:39.573: PKT: GigabitEthernet0/0/6 rcvd v1 R-APS(SF)[0]: req B0 status 00, node_id 0004.DFC9.E93F
*Jul 27 00:19:39.573: PKT: Read VLAN 500 from VLAN header on GigabitEthernet0/0/6
*Jul 27 00:19:39.581: PKT: GigabitEthernet0/0/6 rcvd v1 R-APS(SF)[0]: req B0 status 00, node_id 0004.DFC9.E93F
*Jul 27 00:19:39.581: PKT: Read VLAN 500 from VLAN header on GigabitEthernet0/0/6
*Jul 27 00:19:39.581: PKT: GigabitEthernet0/0/6 rcvd v1 R-APS(SF)[0]: req B0 status 00, node_id 0004.DFC9.E93F
*Jul 27 00:19:39.581: PKT: Read VLAN 500 from VLAN header on GigabitEthernet0/0/6

After the link recovers from failure, Node A and B starts sending NR message on both ends and starts their Guard Timer.

EDD-Node-A# debug erps
Jul  27 00:19:49 : ERPS TX: 1 R-APS(SF)  for ERP id = 0 DST = 01:19:A7:00:00:01
Jul  27 00:19:49 : ERPS sending a Clear SF event on <Eth1/1>
Jul  27 00:19:49 : ERPS Event: domain 0 (DATACOM) issued Local Clear SF on <Eth1/1>
Jul  27 00:19:49 : ERPS TX: 3 R-APS(NR)  for ERP id = 0 DST = 01:19:A7:00:00:01
Jul  27 00:19:49 : ERPS State: domain 0 (DATACOM) Clearing Request = IDLE
Jul  27 00:19:49 : ERPS RX: DST = 01:19:A7:00:00:01 SRC = 00:04:DF:C9:E9:18 VID = 500 port = 6 msdulen = 42 opcode 40
Jul  27 00:19:49 : ERPS RX: domain 0 received R-APS at Eth1/6 vlan 500
Jul  27 00:19:49 : ERPS RX: Guard timer running, don't process received R-APS
Jul  27 00:19:49 : ERPS RX: DST = 01:19:A7:00:00:01 SRC = 00:04:DF:C9:E9:18 VID = 500 port = 6 msdulen = 42 opcode 40

The ASR 903 on receiving RAPS (NR) messages starts WTR Timer and change their protocol status to PENDING. After WTR Timer expiry, the ASR 903 (RPL owner) blocks the RPL interface and starts sending RAPS (NR, RB) message on both its ends.

*Jul 27 00:19:49.339: PKT: GigabitEthernet0/0/6 rcvd v0 R-APS(NR)[0]: req 00 status 00, node_id 0004.DFC9.E93F
*Jul 27 00:19:49.339: PKT: Read VLAN 500 from VLAN header on GigabitEthernet0/0/6
*Jul 27 00:19:49.340: PKT: GigabitEthernet0/0/6 rcvd v0 R-APS(NR)[0]: req 00 status 00, node_id 0004.DFC9.E93F
*Jul 27 00:19:49.340: PKT: Read VLAN 500 from VLAN header on GigabitEthernet0/0/6
*Jul 27 00:19:49.340: ERP_G8032-6-STATE_CHANGED: Ethernet ring DATACOM instance 1 changed state to Pending
*Jul 27 00:19:49.342: PKT: GigabitEthernet0/0/6 rcvd v0 R-APS(NR)[0]: req 00 status 00, node_id 0004.DFC9.E93F
*Jul 27 00:19:49.342: PKT: Read VLAN 500 from VLAN header on GigabitEthernet0/0/6
*Jul 27 00:19:49.342: PKT: GigabitEthernet0/0/7 rcvd v0 R-APS(NR)[0]: req 00 status 00, node_id 0004.DFC9.E918
.
.
.
*Jul 27 00:24:48.337: PKT: GigabitEthernet0/0/6 rcvd v1 R-APS(NR)[0]: req 00 status 00, node_id 0004.DFC9.E93F
*Jul 27 00:24:48.337: PKT: Read VLAN 500 from VLAN header on GigabitEthernet0/0/6
*Jul 27 00:24:49.338: PKT: port0:GigabitEthernet0/0/6, encap: 11, vlan1: 500, vlan2: 0
*Jul 27 00:24:49.338: PKT: port1:GigabitEthernet0/0/7, encap: 11, vlan1: 500, vlan2: 0
*Jul 27 00:24:49.340: PKT: sent 3 APS packet R-APS(NR,RB): req_state 00, status 80, node_id 1CE8.5DC4.B6BF over interfaces GigabitEthernet0/0/6 and GigabitEthernet0/0/7
*Jul 27 00:24:49.342: ERP_G8032-6-STATE_CHANGED: Ethernet ring DATACOM instance 1 changed state to Idle
*Jul 27 00:24:49.342: PKT: GigabitEthernet0/0/7 rcvd v1 R-APS(NR,RB)[0]: req 00 status 80, node_id 1CE8.5DC4.B6BF
*Jul 27 00:24:49.342: PKT: Read VLAN 500 from VLAN header on GigabitEthernet0/0/7
*Jul 27 00:24:49.342: PKT: GigabitEthernet0/0/6 rcvd v1 R-APS(NR,RB)[0]: req 00 status 80, node_id 1CE8.5DC4.B6BF

Finally, when Node A and B detects RAPS (NR, RB) message they unblocks their interfaces Ethernet 1/1. All the nodes again flushes the addresses tables and then the traffic is again routed back on the path on which it was previously flowing as shown in figure of the sample scenario, at this point all the nodes changes to IDLE.

Jul 27 00:24:49 : ERPS RX: domain 0 received R-APS at Eth1/6 vlan 500
Jul 27 00:24:49 : ERPS State change: domain 0 (DATACOM) Unblocked on <Eth1/1>
Jul 27 00:24:49 : ERPS Flush: ERP domain 0 (DATACOM) clearing L2/L3 FDB
Jul 27 00:24:49 : ERPS Flush: ERP domain 0 (DATACOM) clearing L2/L3 FDB
Jul 27 00:24:49 : ERPS RX: DST = 01:19:A7:00:00:01 SRC = 1C:E8:5D:C4:B6:86 VID = 500 port = 6 msdulen = 42 opcode 40
Jul 27 00:24:49 : ERPS RX: DST = 01:19:A7:00:00:01 SRC = 1C:E8:5D:C4:B6:86 VID = 500 port = 6 msdulen = 42 opcode 40
Jul 27 00:24:49 : ERPS State change: domain 0 (DATACOM) Performed State PROTECTION with Input RAPS_NR_RB
Jul 27 00:24:49 : ERPS State change: domain 0 (DATACOM) now at State IDLE
Jul 27 00:24:49 : ERPS RX: DST = 01:19:A7:00:00:01 SRC = 1C:E8:5D:C4:B6:87 VID = 500 port = 1 msdulen = 42 opcode 40
Jul 27 00:24:49 : ERPS RX: DST = 01:19:A7:00:00:01 SRC = 1C:E8:5D:C4:B6:87 VID = 500 port = 1 msdulen = 42 opcode 40

EDD-Node-A#show erps detail
Domain ID:                        1
 Domain Name:                     DATACOM
 State:                           IDLE
 Mode:                            Transit
 HW Forwarding:                   Disabled
 Guard Timer (ms):                500
 WTR Timer (min):                 5
 Holdoff Timer (ms):              0
 Port 0:                          Eth1/1                Port status: Unblocked
 Port 1:                          Eth1/6                Port status: Unblocked
 Control VLAN ID:                 500
 Protected VLAN group IDs:        1
 Accept topology change of domains:

Topology Convergence time after links recovers was 27ms

References