RFC 9722: Fast Recovery for EVPN Designated Forwarder Election

1.2. Terminology

PE:

Provider Edge¶

DF:

Designated Forwarder. A PE that is currently forwarding (encapsulating/decapsulating) traffic for a given VLAN in and out of a site.¶

NDF:

Non-Designated Forwarder. A PE that is currently blocking traffic (see DF above).¶

EVI:

EVPN Instance. It spans the PE devices participating in that EVPN.¶

HRW:

Highest Random Weight algorithm [HRW98]¶

Service carving:

This refers to DF election, as defined in [RFC7432].¶

SCT:

Service Carving Time. Defined in this document as the time at which all nodes participating in an Ethernet Segment perform DF Election.¶

1.3. Challenges with Existing Mechanism

In EVPN technology, multiple PE devices encapsulate and decapsulate data belonging to the same VLAN. Under certain conditions, this may cause duplicated Ethernet packets and potential loops if there is a momentary overlap in forwarding roles between two or more PE devices, potentially also leading to broadcast storms of frames forwarded back into the VLAN.¶

EVPN [RFC7432] currently specifies timer-based synchronization among PE devices within an Ethernet Segment redundancy group. This approach can lead to duplications and potential loops due to multiple DFs if the timer interval is too short or can lead to packet drops if the timer interval is too long.¶

Split-horizon filtering, as described in Section 8.3 of [RFC7432], can prevent loops but does not address duplicates. However, if there are overlapping DFs of two different sites simultaneously for the same VLAN, the site identifier will differ when the packet re-enters the Ethernet Segment. Consequently, the split-horizon check will fail, resulting in Layer 2 loops.¶

The updated DF procedures outlined in [RFC8584] use the well-known HRW algorithm to prevent the reshuffling of VLANs among PE devices within the Ethernet Segment redundancy group during failure or recovery events. This approach minimizes the impact on VLANs not assigned to the failed or recovered ports and eliminates the occurrence of loops or duplicates during such events.¶

However, upon PE insertion or a port being newly added to a multihomed Ethernet Segment, the HRW cannot help either, as a transfer of the DF role to the new port must occur while the old DF is still active.¶

                                  +---------+
               +-------------+    |         |
               |             |    |         |
             / |    PE1      |----|         |   +-------------+
            /  |             |    |  MPLS/  |   |             |---CE3
           /   +-------------+    |  VxLAN/ |   |     PE3     |
      CE1 -                       |  Cloud  |   |             |
           \   +-------------+    |         |---|             |
            \  |             |    |         |   +-------------+
             \ |     PE2     |----|         |
               |             |    |         |
               +-------------+    |         |
                                  +---------+

In Figure 1, when PE2 is inserted in the Ethernet Segment or its CE1-facing interface is recovered, PE1 will transfer the DF role of some VLANs to PE2 to achieve load-balancing. However, because there is no handshake mechanism between PE1 and PE2, overlapping of DF roles for a given VLAN is possible, which leads to duplication of traffic as well as Layer 2 loops.¶

Current EVPN specifications [RFC7432] and [RFC8584] rely on a timer-based approach for transferring the DF role to the newly inserted device. This can cause the following issues:¶

1.4. Design Principles for a Solution

The clock-synchronization solution for fast DF recovery presented in this document follows several design principles and offers multiple advantages, namely:¶

The solution in this document relies on nodes in the topology, more specifically the peering nodes of each Ethernet-Segment, to be clock-synchronized and to advertise the Time Synchronization capability. When this is not the case, or when clocks are badly desynchronized, network convergence and DF Election is no worse than that described in [RFC7432] due to the timestamp range checking (Section 2.2).¶

2.1. BGP Encoding

A BGP extended community, with Type 0x06 and Sub-Type 0x0F, is defined to communicate the SCT for each Ethernet Segment:¶

                     1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Type = 0x06   | Sub-Type(0x0F)|      Timestamp Seconds        ~
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
~  Timestamp Seconds            |      Timestamp Fraction       |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

The timestamp exchanged uses the NTP prime epoch of 0 h 1 January 1900 UTC [RFC5905] and an adapted form of the 64-bit NTP timestamp format.¶

The 64-bit NTP timestamp format consists of a 32-bit unsigned seconds field and a 32-bit fraction field, which are encoded in the Service Carving Time as follows:¶

Timestamp Seconds:

32-bit NTP seconds are encoded in this field.¶

Timestamp Fraction:

The high-order 16 bits of the NTP "Fraction" field are encoded in this field.¶

When rebuilding a 64-bit NTP timestamp format using the values from a received SCT BGP extended community, the lower-order 16 bits of the NTP "Fraction" field are set to 0. The use of a 16-bit fractional seconds value yields adequate precision of 15 microseconds (2^-16 s).¶

The format of the DF Election Extended Community that is used in this document is:¶

                     1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Type = 0x06   | Sub-Type(0x06)| RSV |  DF Alg |    Bitmap     ~
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
~     Bitmap    |            Reserved                           |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

The Bitmap field (2 octets) encodes "capabilities" [RFC8584], where this document introduces a new Time Synchronization capability indicated by "T".¶

                     1         1
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |A| |T|                       |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Bit 3:

Time Synchronization (corresponds to Bit 27 of the DF Election Extended Community). When set to 1, it indicates the desire to use the Time Synchronization capability with the rest of the PEs in the Ethernet Segment.¶

This capability is utilized in conjunction with the agreed-upon DF Election Type. For instance, if all the PE devices in the Ethernet Segment indicate the desire to use the Time Synchronization capability and request the DF Election Type to be the HRW, then the HRW algorithm is used in conjunction with this capability. A PE that does not support the procedures set out in this document or that receives a route from another PE in which the capability is not set MUST NOT delay DF election as this could lead to duplicate traffic in some instances (overlapping DFs).¶

2.2. Timestamp Verification

The NTP Era value is not exchanged, and participating PEs may consider the timestamps to be in the same Era as their local value. A DF Election operation occurring exactly at the next Era transition will be some time on February 7, 2036. Implementors and operators may address credible cases of rollover ambiguity (adjacent Eras n and n+1) as well as the security issue of unreasonably large or unreasonably small NTP timestamps in the following manner.¶

The procedures in this document address implicitly what occurs with receiving an SCT value in the past. This would be a naturally occurring event with a large BGP propagation delay: the receiving PE treats the DF Election at the peer as having already occurred and proceeds without starting any timer to further delay service carving, effectively falling back on behavior as specified in [RFC7432]. A PE that receives an SCT value smaller than its current time MUST discard the Service Carving Time and SHALL treat the DF Election at the peer as having occurred already.¶

The more problematic scenario is the PE in Era n+1 that receives an SCT advertised by the PE still in Era n, with a very large SCT value. To address this Era rollover as well as the large values attack vector, implementations MUST validate the received SCT against an upper bound.¶

It is left to implementations to decide what constitutes an "unreasonably large" SCT value. A recommended approach, however, is to compare the received offset to the local peering timer value. In practice, peering timer values are configured uniformly across Ethernet Segment peers and may be treated as an upper bound on the offset of received SCT values. A PE that receives an SCT representing an offset larger than the local peering timer MUST discard the SCT and SHALL treat the DF Election at the peer as having already occurred, as above.¶

2.3. Updates to RFC 8584

This document introduces an additional delay to the events and transitions defined for the default DF election algorithm FSM in Section 2.1 of [RFC8584] without changing the FSM state or event definitions themselves.¶

Upon receiving an RCVD_ES message, the peering PE's FSM transitions from the DF_DONE state (indicating the DF election process was complete) to the DF_CALC state (indicating that a new DF calculation is needed). Due to the SCT included in the Ethernet Segment update, the completion of the DF_CALC state and the subsequent transition back to the DF_DONE state are delayed. This delay ensures proper synchronization and prevents conflicts. Consequently, the accompanying forwarding updates to the DF and NDF states are also deferred.¶

Item 9 in Section 2.1 of [RFC8584], in the list "Corresponding actions when transitions are performed or states are entered/exited", is changed as follows:¶

9. DF_CALC on CALCULATED: Mark the election result for the VLAN or VLAN bundle.¶

   9.1

   If no Service Carving Time is present during the RCVD_ES event of Action 11, proceed to step 9.4¶

   9.2

   If a Service Carving Time is present during the RCVD_ES event of Action 11, wait until the time indicated by the SCT minus skew before proceeding to step 9.3.¶

   9.3

   Assume the role of NDF for the local PE concerning the VLAN or VLAN bundle. Wait the remaining skew time before proceeding to step 9.4.¶

   9.4

   Assume the election result's role (DF or NDF) for the local PE concerning the VLAN or VLAN bundle and transition to the DF_DONE state.¶

This revised approach ensures proper timing and synchronization in the DF election process, avoiding conflicts and ensuring accurate forwarding updates.¶

3.1. Concurrent Recoveries

In the eventuality that two or more PEs in a peering Ethernet Segment group are recovering concurrently or roughly at the same time, each will advertise a SCT. This SCT value would correspond to what each recovering PE considers the "end time" for DF Election. A similar situation arises in sequentially recovering PEs, when a second PE recovers approximately at the time of the first PE's advertised SCT expiry and with its own new SCT-2 outside of the initial SCT window.¶

In the case of multiple concurrent DF elections, each initiated by one of the recovering PEs, the SCTs must be ordered chronologically. All PEs SHALL execute only a single DF Election at the service carving time corresponding to the largest (latest) received timestamp value. This DF Election will lead peering PEs into a single coordinated DF Election update.¶

Example:¶

In the eventuality that a PE in an Ethernet Segment group recovers during the discovery window specified in Section 8.5 of [RFC7432] and does not support or advertise the T-bit, all PEs in the current peering sequence SHALL immediately revert to the default behavior described in [RFC7432].¶