Internet-Draft LSN November 2026
Camarillo, et al. Expires 6 May 2027 [Page]
Workgroup:
FANN
Internet-Draft:
draft-camarillo-fann-lsn-00
Published:
Intended Status:
Standards Track
Expires:
Authors:
P. Camarillo, Ed.
Cisco
C. Filsfils
Cisco
N. Chachmon
Cisco
O. Iny
Cisco
Y. Su
Alibaba
R. Jiang
Alibaba

Lightspeed Notification Protocol

Abstract

This document defines the Lightspeed Notification Protocol (LSN), a hardware-accelerated signaling mechanism designed for sub-100 microsecond network convergence in AI/ML data center fabrics. By remaining on chip and never punting to the host CPU, LSN bypasses traditional CPU-based latencies to propagate link failures and congestion via a hardware-efficient encoding. It serves as a high-speed complement to routing protocols such as BGP, providing an immediate hardware veto that prunes failed or congested paths while maintaining control-plane stability for path recovery.

The same mechanism applies inside the DC Clos and, in some deployments, across the Data Center Interconnect (DCI) and WAN. This document defines two encoding modes: a dense encoding for DC fabrics, where traffic is load-balanced over many ECMP spines, and a sparse encoding for the DCI and WAN.

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."

This Internet-Draft will expire on 6 May 2027.

▲

Table of Contents

1. Introduction

Artificial Intelligence (AI) and Machine Learning (ML) workloads impose stringent demands on data center fabrics, and in some deployments on the Data Center Interconnect (DCI) and WAN that join those fabrics. As outlined in [I-D.clad-rtgwg-ipfrr-aiml], these environments require network convergence times in the sub-100 microsecond range to avoid performance degradation caused by packet loss and jitter.

Traditional routing protocols and convergence mechanisms rely on control-plane intervention, where link state changes are processed by the CPU. This introduces activation latencies in the tens of milliseconds, which is orders of magnitude too slow for these requirements. Furthermore, existing mechanisms typically operate on a binary "up/down" model and do not signal capacity degradation or congestion.

This document defines the Lightspeed Notification Protocol (LSN), a hardware-accelerated notification mechanism that operates on chip to handle both network failures and congestion events. LSN is complementary to traditional routing protocols (for example, BGP). The interaction between the routing protocol and LSN is functionally equivalent to a boolean AND operation: a path is eligible for forwarding only if permitted by the routing protocol AND confirmed healthy by the hardware notification.

This enforces an asymmetric reaction. "Bad news" (failure or congestion) acts as an immediate hardware veto, disabling the path in sub-microseconds. "Good news" (recovery) is gated by the routing protocol, ensuring the control plane has fully converged before traffic resumes. This approach combines sub-microsecond protection with routing coherence and stability.

LSN defines two encoding modes. Dense encoding (Section 4) is used in DC fabrics, where ECMP is always present through many spines. Sparse encoding (Section 5) is used in the DCI and WAN. The encodings differ due to the different network characteristics of those environments.

1.1. Requirements Language

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.

1.2. Terminology

  • LSN: Lightspeed Notification Protocol.

  • Dense encoding: The LSN encoding used in DC fabrics, where the destination set is the other leaves and forwarding is ECMP across many spines.

  • Sparse encoding: The LSN encoding used in the DCI and WAN, where the destination set is sparse and large and forwarding may be ECMP, weighted ECMP, or primary/backup.

  • DC: Data Center fabric, typically a Clos spine-leaf topology.

  • DCI: Data Center Interconnect, the network that joins DC fabrics.

  • WAN: The wide-area network that carries traffic between regions or sites of an AI/ML fabric.

  • uN: The uSID function defined in [RFC9800]. In the context of LSN, uN identifies the node advertising the LSN event.

  • uA: The uSID function defined in [RFC9800]. In the context of LSN, uA identifies the link or bundle member associated with the LSN event.

  • Efficient Remote Protection: The IP Fast Reroute mechanism specified in [I-D.clad-rtgwg-efficient-remote-protection], also known as remote TI-LFA. It activates a pre-computed backup at a node upstream of the failure to avoid hairpinning.

2. Problem Statement

Current IP Fast Reroute (IPFRR) and resiliency mechanisms face fundamental limitations when a remote hop fails or congests behind a next-hop that remains up.

Latency of CPU-Based Processing

Existing failure detection and propagation mechanisms generally trigger interrupts that must be processed by a line-card or system CPU. As noted in [I-D.clad-rtgwg-ipfrr-aiml], this host-CPU path introduces an activation delay typically in the range of 10-50 milliseconds. For AI training workloads requiring barrier synchronization, this latency results in unacceptable flow disruption. To achieve the required sub-100 microsecond convergence, the propagation and processing of network state changes MUST occur on chip and MUST NOT be punted to the host CPU.

Remote Failure Behind a Live Next-Hop

A common failure mode is that the ingress node's immediate next-hop remains reachable while a downstream link has failed. In a DC Clos fabric, traditional BFD/TWAMP liveness of the BGP session to the spine remains up, so local detection and PIC-core at the ingress leaf do not fire. In a DCI or WAN, traditional BFD/TWAMP liveness of the BGP session to the next-hop likewise remains up, so local FRR does not activate. In both cases the control plane will eventually withdraw the affected prefixes, but traffic blackholes for the entire control-plane window. The final FIB produced by BGP is already correct; the issue is that the dataplane sits wrong until that reconvergence completes.

Inability to Handle Congestion and Partial Failures

Standard routing protocols and ECMP operate on a binary failure model: a link is either available or unavailable. AI fabrics, including the DCI and the WAN, frequently experience brownout scenarios, such as partial capacity reduction (for example, loss of a single member in a bundle) or acute congestion. Current mechanisms lack the granularity to signal these states, causing traffic to be blackholed or hashed onto congested paths. When the BGP next-hop represents a LAG, the safe reaction without per-destination visibility is to disable the whole bundle, taking remaining healthy members out of service.

Limited Visibility of Remote Network State

In multi-stage Clos topologies and in DCI and WAN topologies, a failure or congestion event often occurs multiple hops away from the ingress point. Local protection at the point of failure is often insufficient, or it leads to suboptimal hairpin routing that increases latency and wastes bandwidth. Efficient Remote Protection [I-D.clad-rtgwg-efficient-remote-protection] addresses the hairpin by activating a backup at an upstream node, but it depends on a timely forwarding-plane notification of the remote event. Current routing protocols do not provide that notification in the required time budget.

3. Common Operation

LSN is not a routing protocol and is not a BGP attribute. It is an on-chip notification, generated and processed without the host CPU, that overlays a health signal on the paths already installed by the routing protocol.

The following properties apply to both the dense and the sparse encodings:

Two encodings are defined, due to the different network characteristics of DC fabrics and of the DCI and WAN. Section 4 specifies the dense encoding and illustrates it in a DC Clos. Section 5 specifies the sparse encoding and illustrates it in a DCI/WAN topology.

4. Dense Encoding

This section specifies the dense encoding. Dense encoding is used in DC fabrics, where ECMP is always present through many spines and the destination set is the other leaves of the fabric. The notification carries a bitmap of destination reachability or congestion, and the ingress leaf overlays that bitmap on the spine next-hops already installed by BGP.

4.1. Dense Notification Packet Format

The dense notification is an Ethernet MAC Control frame. The packet header is the following:

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                   MAC Destination Address (DA)                |
+-------------------------------+-------------------------------+
|      MAC DA (continued)       |    MAC Source Address (SA)    |
+-------------------------------+-------------------------------+
|                   MAC Source Address (continued)              |
+-------------------------------+-------------------------------+
|        EtherType (TBD1)       |         OpCode (TBD2)         |
+-------------------------------+-------------------------------+
|                                                               |
.                         LSN Payload                           .
.                                                               .
|                                                               |
+---------------------------------------------------------------+
|                           CRC (FCS)                           |
+---------------------------------------------------------------+
Figure 1: Dense LSN Ethernet header

Notes:

  • MAC Destination Address: 01-80-C2-00-00-01 (IEEE MAC-specific control protocols)

  • Ethertype = TBD1

  • Opcode = TBD2

  • MAC Source Address: the neighbor that originated the notification. The receiver uses the source address to identify which spine next-hop (column) to update.

The LSN Payload is defined as follows:

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Type  |R|Msg| Rsv | Dev-Range |                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               +
|                                                               |
|                   Reachable-Devices-Bitmap                    |
|                          (256 bits)                           |
|                                                               |
+                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 2: Dense LSN payload

Field Descriptions:

  • Header Type (4 bits): Value '12' - Identifies this as a topology reachability packet.

  • R (1 bit): Reserved. It MUST be set to zero upon transmission and ignored upon receipt.

  • Msg-type (2 bits): Indicates the information conveyed by the bitmap. It may represent reachability or congestion status. The congestion status is defined as 3 different congestion levels, which are implementation specific. The following Msg-Types are defined:

    • 0x0: Reachable

    • 0x1: Reachable and not congested (Congestion Level 1)

    • 0x2: Reachable and not congested (Congestion Level 2)

    • 0x3: Reachable and not congested (Congestion Level 3)

  • Reserved (3 bits): Reserved. It MUST be set to zero upon transmission and ignored upon receipt.

  • Reachable-Devices-Range (6 bits): Determines the range of the reachable devices bitmap. The range per packet is within [Reachable-Devices-Range*256, Reachable-Devices-Range*256 + 255].

    • '0': Devices 0-255

    • '1': Devices 256-511

    • '2': Devices 512-767

    • ... (Covers up to 16K devices).

  • Reachable-Devices-Bitmap (256 bits): One bit per destination leaf. Bit i indicates whether destination leaf i is healthy through the device sending this message.

4.2. Dense Node Identification

To facilitate bitmap-based signaling, every leaf node within the fabric is assigned a unique Global Node ID. This identifier maps the node to a specific bit position within the notification bitmap.

The allocation, synchronization, and lifecycle management of these Global Node IDs are performed by an external central controller or management plane. The specific mechanisms for ID allocation are outside the scope of this document.

4.3. Dense Origination

Each spine is responsible for generating notification messages that convey the direct reachability or congestion state of the leaves attached to it.

Message Scope and Fragmentation

A single notification message supports a payload covering up to 256 destination devices.

  • The message header MUST specify the Reachable-Devices-Range (or Base ID), indicating the starting Global Node ID for the bitmap payload.

  • If the fabric scale exceeds 256 leaves, the originating node MUST generate multiple distinct notification messages. Each message corresponds to a different, non-overlapping Reachable-Devices-Range to cover the full topology.

MSG-Type and State Encoding

The interpretation of the bitmap payload is determined by the MSG-Type field in the header.

  • Reachability Information: When the MSG-Type indicates reachability, the bitmap encodes the binary status of the downlinks:

    • Bit value 1: Indicates the destination leaf is directly reachable.

    • Bit value 0: Indicates the destination leaf is not reachable.

  • Congestion Information: When the MSG-Type indicates congestion, the bitmap conveys the congestion status of the associated downlinks. The mapping of queue congestion levels to the various congestion values is implementation-specific, provided it supports the hardware logic required for path pruning or de-prioritization.

Notifications are event-driven. One is emitted when an interface changes state or a path's congestion level changes significantly; the originating node does not restate the whole fabric on every tick. A low-rate periodic refresh sits underneath those events, so that a switch that missed a notification resynchronizes without waiting for the next change.

4.4. Dense Hardware Processing

Upon receipt of a dense notification message, the receiving leaf MUST process the packet on chip. The packet MUST NOT be punted to the host CPU.

State Aggregation

The receiving leaf maintains a local health table with one row per destination leaf and one column per spine next-hop. When a message is received:

  1. The hardware identifies the column from the MAC Source Address (the advertising spine) and the row-range from the Reachable-Devices-Range.

  2. The received bitmap is overlaid onto that column, updating the reachability or congestion status for the corresponding destination leaves.

Forwarding Decision Logic

To derive the final forwarding decision, the hardware performs a bitwise operation between the standard routing table and the aggregated notification state.

  • For reachability, this is functionally a Bitwise AND operation: a path is valid if and only if the routing protocol has installed it (Logical 1) AND the notification protocol reports it as reachable (Logical 1).

  • If the result of this operation is 0 for a specific next-hop, the hardware immediately removes that next-hop from the Equal-Cost Multi-Path (ECMP) set for that destination. Next-hops used for other destinations are unchanged.

When a bundle (LAG) is used toward a spine, the health table MAY be maintained per bundle member. A single member failure then reduces capacity toward the affected destination (for example from 100% to 75% on a 4-member bundle) rather than withdrawing the entire BGP next-hop. The BGP adjacency remains up; LSN only changes membership or weight for the destinations that use the failed member.

4.5. Illustration: Dense Encoding in a DC Fabric

This section illustrates dense encoding in a DC Clos fabric. The example uses three spines and three leaves so that the topology is readable; the same procedure applies to a fabric of 256 spines and 256 leaves.

4.5.1. DC Topology

Spines are identified by letters. Leaves are identified by numbers. Every leaf is connected to every spine.

  • Spines: A, B, C.

  • Leaves: 1, 2, 3. Each leaf's Global Node ID equals its number.

  • Traffic: leaf 1 sends traffic to leaf 3 (and also to leaf 2).

  • Routing state: BGP advertises that leaf 3 is reachable via all spines. Leaf 1 maintains ECMP {A, B, C} for destination 3, and the same ECMP for destination 2.

DC Clos topology with spines A B C and leaves 1 2 3 Two-tier Clos fabric. Spines A, B, and C on the top row. Leaves 1, 2, and 3 on the bottom row. Every leaf is connected to every spine. The link from spine B to leaf 3 is marked as failed. A B C 1 2 3 Spine Spine Spine Leaf Leaf Leaf B-3 down
Figure 3: DC Clos fabric used in the illustration

4.5.2. DC Failure Scenario

A physical link failure occurs between spine B and destination leaf 3.

  1. Detection and Generation: Spine B detects that the port connected to leaf 3 is down. The hardware at spine B immediately generates a notification packet and transmits it to all connected leaves, including leaf 1.

    • MAC Destination Address: 01-80-C2-00-00-01 (MAC Control)

    • MAC Source Address: spine B

    • Reachable-Devices-Range (Base ID): 0 (covering leaves 0-255, here 1, 2, and 3).

    • MSG-Type: Reachability.

    • Bitmap: three entries, one per attached leaf. 1 and 2 remain active (1). The third entry, Leaf 3, is set to inactive (0).

  1. Hardware Processing at Ingress: Leaf 1 receives the notification from spine B. The forwarding engine overlays column B of its health table and performs a logical AND with the BGP ECMP set.

    • Destination 3 via spine B:

      • Routing table state for leaf 3: 1 (the control plane has not yet converged; leaf 3 is still considered reachable via B).

      • Notification state from spine B for leaf 3: 0.

      • Result: 1 AND 0 = 0 (path invalid).

    • Destination 3 via spines A and C:

      • Routing table state for leaf 3: 1.

      • Notification state from A and from C for leaf 3: 1.

      • Result: 1 AND 1 = 1 (path valid).

    • Destination 2 via spine B:

      • The bit for leaf 2 in the same notification is 1. Traffic from leaf 1 to leaf 2 continues to use B. Spine B is not drained.

  2. Forwarding Result: Because the result for spine B is 0 only for destination 3, the forwarding hardware excludes B from the ECMP group for leaf 3. Traffic to leaf 3 is rebalanced across {A, C}. Balancing of traffic to leaf 2 is unchanged: {A, B, C}.

This mechanism provides sub-microsecond protection. When the link B-3 is restored, spine B sends an updated bitmap with the leaf-3 bit set to 1. Traffic does not resume via B until BGP also re-installs the route, satisfying the 1 AND 1 condition.

5. Sparse Encoding

This section specifies the sparse encoding. Sparse encoding is used in the DCI and WAN, where the destination set is sparse and large. Forwarding may be an ECMP set, a weighted-ECMP (wECMP) set, or a primary/backup pair; it is not limited to the wide spine ECMP of a Clos fabric. The notification identifies the event type, the node that detected the event, and the affected adjacency. A node that has registered interest in that adjacency either eliminates a member from an ECMP set, changes the wECMP weights, or activates a pre-programmed backup, most commonly an Efficient Remote Protection (remote TI-LFA) path [I-D.clad-rtgwg-efficient-remote-protection].

LSN does not compute that ECMP change or that backup. An agent with visibility of the topology programs the ECMP membership, the wECMP weights, and the backup pointers before any failure. LSN only sets the protection state that the agent has already installed.

5.1. Sparse Notification Packet Format

The sparse notification is an Ethernet frame carrying an IPv6 header and no payload.

+------------------+------------------+
| Ethernet header  | IPv6 header      |
|                  | (no payload)     |
+------------------+------------------+
Figure 5: Sparse LSN packet

The IPv6 header [RFC8200] is used as follows:

  • Source Address: an IPv6 address of the node originating the notification.

  • Destination Address: the LSN Notification SID of the domain. 2001:db8::/32 is the documentation prefix [RFC3849] standing in for the SRv6 uSID block of the domain [RFC8986]. This SID is associated with the LSN Endpoint Behavior (TBD3). The LSN Endpoint Behavior takes three 16-bit arguments, in order:

    • Event-Type: the kind of event being advertised.

    • uN: the uSID function defined in [RFC9800]; in LSN it identifies the advertising node.

    • uA: the uSID function defined in [RFC9800]; in LSN it identifies the affected link or bundle member.

    In this document's illustrations, FF00 is a SID associated with the LSN Endpoint Behavior. The resulting SID is structured as 2001:db8:FF00:Event-Type:uN:uA:: (see Section 8).

  • Payload Length: 0.

  • Next Header: 59 (IPv6 No Next Header).

  • Hop Limit: the blast radius of the notification.

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                         uSID block                            |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|     LSN SID                   |         Event-Type            |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|              uN               |              uA               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                            0                                  |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 6: LSN Notification SID

The Event-Type argument indicates the information conveyed by the notification. Value 0x0000 is reserved. The following values are defined:

  • 0x0001: Down / not reachable

  • 0x0002: Up / recovered

  • 0x0003: Congestion Level 1

  • 0x0004: Congestion Level 2

  • 0x0005: Congestion Level 3

Congestion levels are implementation specific. When Event-Type indicates congestion, a registered node MAY change wECMP weights or de-prefer the adjacency; it is not required to treat the adjacency as failed.

This document uses the following identifier convention in its illustrations:

  • Event-Type uses the values defined above. The failure illustration uses Event-Type = 0001 (Down).

  • uN is 0X00, where X is the node number of the advertising node. For example, node 2 uses uN = 0200.

  • uA is F0YZ, where Y is the source node of the adjacency and Z is the destination node of the adjacency. For example, the link from node 2 to node 3 uses uA = F023.

The allocation of uN and uA values in a deployment is performed by the operator or by a controller. The specific allocation protocol is outside the scope of this document.

5.2. Sparse Origination

Every node monitors its local links. Upon detecting a local link failure, a link-up event, a congestion-level change, or a change of link properties (for example a bundle capacity change that must be reflected in weighted ECMP), the node originates a sparse LSN notification.

The originating hardware keeps one pre-crafted IPv6 notification ready to send. Distinct packets per neighbor MUST NOT be required. When the local interface-status or congestion logic detects the event, it writes the LSN Notification SID (a SID associated with the LSN Endpoint Behavior, together with the three arguments Event-Type, uN, and uA) and sends that same packet out on all interfaces.

Local origination is based on interface status registers (on the order of 10 microseconds), not on traditional BFD/TWAMP liveness. Traditional BFD/TWAMP liveness continues to serve the BGP session; it is not the trigger for LSN.

5.3. Sparse Processing and Registration

Upon receipt, the packet is processed in the dataplane.

A hardware classifier matches the LSN Notification SID (domain uSID block and the SID associated with the LSN Endpoint Behavior). The match triggers two actions:

  1. If the Hop Limit is greater than 1, decrement the Hop Limit and forward the notification on the interfaces indicated by the local LSN forwarding policy, so that registered upstream nodes can receive it.

  2. Process a copy of the packet on chip. The LSN Endpoint Behavior (TBD3) is invoked with its three arguments (Event-Type, uN, uA) to activate any locally registered protection for that tuple.

an agent, before any failure, programs an exact-match table:

  • Key: the three arguments of the LSN Endpoint Behavior (Event-Type, uN, uA).

  • Value: a linked list of local protection pointers. Each pointer identifies a FIB protection bit to set: elimination of an ECMP member, a wECMP weight update, or activation of an Efficient Remote Protection (remote TI-LFA) backup.

A single DCI or WAN adjacency often covers many destinations. The linked list allows one notification to enable every locally relevant protection entry without a host-CPU walk of the RIB. Failure and congestion on the same adjacency use different Event-Type values and therefore different table entries, so they MAY install different protection lists.

The lookup on (Event-Type, uN, uA) returns the first protection pointer and the next-member link. The protection pointer is set on chip and the list is walked until it terminates. The packet MUST NOT be punted to the host CPU.

LSN does not compute the ECMP change or the backup path. The agent has already installed the ECMP members, the wECMP weights, and any Efficient Remote Protection (remote TI-LFA) backup [I-D.clad-rtgwg-efficient-remote-protection]. LSN sets the protection state that applies them.

5.4. Illustration: Sparse Encoding in the DCI and WAN

This section illustrates sparse encoding in a DCI/WAN topology. The example uses a primary/backup pair and Efficient Remote Protection. The same notification can instead eliminate a member of an ECMP set or change wECMP weights. Routers are identified by numbers. Numbers are not reused.

5.4.1. DCI/WAN Topology

Consider the topology in Figure 7.

  • Node 1 is the ingress / Point of Remote Repair.

  • The primary path from 1 to destination 7 is 1-2-3-7.

  • The backup path from 1 to destination 7 is 1-4-5-6-7.

  • The protected resource is the adjacency 2-3.

Identifier assignment for the illustration:

  • Node 1: uN = 0100

  • Node 2: uN = 0200

  • Node 3: uN = 0300

  • Node 4: uN = 0400

  • Node 5: uN = 0500

  • Node 6: uN = 0600

  • Node 7: uN = 0700

  • Adjacency 2-3: uA = F023

  • Failure of adjacency 2-3: Event-Type = 0001

DCI topology with routers 1 through 7 Primary path 1-2-3-7 and backup path 1-4-5-6-7. The link from node 2 to node 3 is marked as failed. Node 2 sends an LSN notification toward node 1 with Event-Type 0001, uN 0200, and uA F023. Node 1 activates the backup via 4, 5, and 6. 1 2 3 7 4 5 6 primary 2-3 down primary LSN (ETYP=0001, uN=0200, uA=F023) backup 1-4-5-6-7 (Efficient Remote Protection)
Figure 7: DCI/WAN topology used in the illustration

5.4.2. DCI/WAN Failure Scenario

Traffic from node 1 to destination 7 uses next-hop 2. Local FRR at node 1 does not fire, because the protected resource is the remote adjacency 2-3.

If only local TI-LFA were used at node 2, traffic already at node 1 would continue toward node 2 and would then be repaired in a way that hairpins back through the upstream path. Efficient Remote Protection programs the repair at node 1: if adjacency 2-3 fails, traffic from 1 to 7 is steered on 1-4-5-6-7.

  1. Pre-failure programming: The agent installs at node 1:

    • An Efficient Remote Protection backup for destination 7, protecting adjacency 2-3, via 4-5-6.

    • An exact-match entry for (Event-Type=0001, uN=0200, uA=F023) whose protection pointer enables that backup.

    • On node 2, a Hop Limit of 1.

  2. Detection and Generation: Node 2 detects that the local port toward node 3 is down. The hardware originates one IPv6 LSN notification:

    • Source Address: node 2.

    • Destination: LSN Notification SID 2001:db8:FF00:0001:0200:F023::.

    • Hop Limit: 1.

    • Payload Length: 0; Next Header: 59. The packet is replicated in hardware on the interfaces of node 2, including the interface toward node 1.

  3. Processing at node 1: The classifier matches the LSN Notification SID. Node 1 invokes the LSN Endpoint Behavior (TBD3) with arguments (0001, 0200, F023), walks the protection linked list, and sets the Efficient Remote Protection pointer for destination 7. Traffic from 1 to 7 is switched onto 1-4-5-6-7. The packet MUST NOT be punted to the host CPU.

  4. AND with routing: The backup is used only while BGP still authorizes a path from 1 to 7 and LSN reports adjacency 2-3 as failed. When 2-3 is restored, node 2 may originate a corresponding recovery notification with Event-Type = 0002 (Up). Traffic returns to 1-2-3-7 only after BGP has reconverged, satisfying the AND condition.

The same notification can enable more than one protection pointer. If adjacency 2-3 is also a weighted-ECMP input for other destinations at node 1, those destinations are chained from the same (0001, 0200, F023) entry.

A congestion event on the same adjacency uses a different Event-Type (0x0003, 0x0004, or 0x0005) and therefore a different exact-match entry, which MAY update wECMP weights without activating the backup.

6. Security Considerations

The Lightspeed Notification Protocol (LSN) is designed to achieve sub-100 microsecond convergence on chip. To achieve this performance, LSN messages do not include cryptographic authentication or integrity checks. Consequently, the protocol introduces several security considerations that must be mitigated by the deployment architecture.

Spoofing and Denial of Service (DoS): Because LSN messages are unauthenticated, an attacker capable of injecting forged LSN frames into the fabric could advertise false congestion or failure notifications. This would cause receiving nodes to prune valid paths or to activate backups, potentially leading to forced congestion, suboptimal routing, or a denial of service.

Mitigation via Boundary Filtering: LSN is intended exclusively for internal use within a single administrative domain (a DC fabric and, where deployed, that operator's DCI and WAN). To prevent spoofing, boundary nodes MUST implement strict port-level filtering. LSN notification packets MUST be dropped unconditionally on any port facing a server, host, or external network. LSN processing MUST only be enabled on trusted infrastructure links. In the sparse encoding, the Hop Limit further constrains how far a notification can travel.

Fail-Safe Routing Logic: As defined in this document, LSN acts only as a hardware veto (a logical AND operation with the routing protocol). While an attacker can maliciously prune a path or force a pre-authorized backup, they cannot force the network to forward traffic into a blackhole or an invalid topology segment, because reachability still requires the routing protocol (for example, BGP) to authorize the path. The control plane remains the source of truth for topology validation.

7. IANA Considerations

This document requests the following allocations.

IANA is requested to allocate a codepoint in the "SRv6 Endpoint Behaviors" registry as follows:

Table 1: SRv6 Endpoint Behavior allocation
Value Hex Endpoint Behavior Reference Change Controller
TBD3   LSN Notification This document IETF

Upon assignment of these values, the RFC Editor is requested to replace all instances of TBD1, TBD2, and TBD3 with the allocated values.

8. Running Code

The dense encoding is running on Cisco Silicon One G300. Notification generation and notification processing are performed entirely on chip, at linerate. LSN notifications are not processed on the host CPU and are not handed to the control plane.

The prototype instantiates the IANA codepoints of this document as follows:

9. Acknowledgements

The authors would like to acknowledge Kris Michielsen.

10. References

10.1. Normative References

[RFC2119]
Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, , <https://www.rfc-editor.org/rfc/rfc2119>.
[RFC8174]
Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, , <https://www.rfc-editor.org/rfc/rfc8174>.
[RFC8200]
Deering, S. and R. Hinden, "Internet Protocol, Version 6 (IPv6) Specification", STD 86, RFC 8200, DOI 10.17487/RFC8200, , <https://www.rfc-editor.org/rfc/rfc8200>.
[RFC8986]
Filsfils, C., Ed., Camarillo, P., Ed., Leddy, J., Voyer, D., Matsushima, S., and Z. Li, "Segment Routing over IPv6 (SRv6) Network Programming", RFC 8986, DOI 10.17487/RFC8986, , <https://www.rfc-editor.org/rfc/rfc8986>.
[RFC9800]
Cheng, W., Ed., Filsfils, C., Li, Z., Decraene, B., and F. Clad, Ed., "Compressed SRv6 Segment List Encoding", RFC 9800, DOI 10.17487/RFC9800, , <https://www.rfc-editor.org/rfc/rfc9800>.

10.2. Informative References

[I-D.clad-rtgwg-efficient-remote-protection]
Clad, F., Filsfils, C., Su, Y., and D. Cai, "Efficient Remote Protection", , <https://datatracker.ietf.org/doc/draft-clad-rtgwg-efficient-remote-protection/>.
[I-D.clad-rtgwg-ipfrr-aiml]
Clad, F., Filsfils, C., Jiang, R., and D. Cai, "IP Fast Reroute for AI/ML Fabrics", , <https://datatracker.ietf.org/doc/draft-clad-rtgwg-ipfrr-aiml/>.
[IEEE802.3]
IEEE Computer Society, "IEEE Standard for Ethernet; IEEE Std 802.3-2022", , <https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9844436>.
[RFC3849]
Huston, G., Lord, A., and P. Smith, "IPv6 Address Prefix Reserved for Documentation", RFC 3849, DOI 10.17487/RFC3849, , <https://www.rfc-editor.org/rfc/rfc3849>.

Authors' Addresses

Pablo Camarillo Garvia (editor)
Cisco
Spain
Clarence Filsfils
Cisco
Belgium
Nadav Chachmon
Cisco
Israel
Ofer Iny
Cisco
Israel
Yuanchao Su
Alibaba
China
Roy Jiang
Alibaba
China