RIFT Working Group A. Przygienda, Ed. Internet-Draft Juniper Intended status: Standards Track A. Sharma Expires: November 27, 2020 Comcast P. Thubert Cisco Bruno. Rijsman Individual Dmitry. Afanasiev Yandex May 26, 2020 RIFT: Routing in Fat Trees draft-ietf-rift-rift-12 Abstract This document defines a specialized, dynamic routing protocol for Clos and fat-tree network topologies optimized towards minimization of configuration and operational complexity. The protocol o deals with no configuration, fully automated construction of fat- tree topologies based on detection of links, o minimizes the amount of routing state held at each level, o automatically prunes and load balances topology flooding exchanges over a sufficient subset of links, o supports automatic disaggregation of prefixes on link and node failures to prevent black-holing and suboptimal routing, o allows traffic steering and re-routing policies, o allows loop-free non-ECMP forwarding, o automatically re-balances traffic towards the spines based on bandwidth available and finally o provides mechanisms to synchronize a limited key-value data-store that can be used after protocol convergence to e.g. bootstrap higher levels of functionality on nodes. Przygienda, et al. Expires November 27, 2020 [Page 1] Internet-Draft RIFT May 2020 Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at https://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on November 27, 2020. Copyright Notice Copyright (c) 2020 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Table of Contents 1. Authors . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 6 2.1. Requirements Language . . . . . . . . . . . . . . . . . . 8 3. Reference Frame . . . . . . . . . . . . . . . . . . . . . . . 8 3.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . 8 3.2. Topology . . . . . . . . . . . . . . . . . . . . . . . . 13 4. RIFT: Routing in Fat Trees . . . . . . . . . . . . . . . . . 15 4.1. Overview . . . . . . . . . . . . . . . . . . . . . . . . 16 4.1.1. Properties . . . . . . . . . . . . . . . . . . . . . 16 4.1.2. Generalized Topology View . . . . . . . . . . . . . . 17 4.1.2.1. Terminology . . . . . . . . . . . . . . . . . . . 17 4.1.2.2. Clos as Crossed Crossbars . . . . . . . . . . . . 18 4.1.3. Fallen Leaf Problem . . . . . . . . . . . . . . . . . 28 4.1.4. Discovering Fallen Leaves . . . . . . . . . . . . . . 30 Przygienda, et al. Expires November 27, 2020 [Page 2] Internet-Draft RIFT May 2020 4.1.5. Addressing the Fallen Leaves Problem . . . . . . . . 31 4.2. Specification . . . . . . . . . . . . . . . . . . . . . . 32 4.2.1. Transport . . . . . . . . . . . . . . . . . . . . . . 33 4.2.2. Link (Neighbor) Discovery (LIE Exchange) . . . . . . 33 4.2.2.1. LIE FSM . . . . . . . . . . . . . . . . . . . . . 36 4.2.3. Topology Exchange (TIE Exchange) . . . . . . . . . . 46 4.2.3.1. Topology Information Elements . . . . . . . . . . 46 4.2.3.2. South- and Northbound Representation . . . . . . 46 4.2.3.3. Flooding . . . . . . . . . . . . . . . . . . . . 49 4.2.3.4. TIE Flooding Scopes . . . . . . . . . . . . . . . 56 4.2.3.5. 'Flood Only Node TIEs' Bit . . . . . . . . . . . 59 4.2.3.6. Initial and Periodic Database Synchronization . . 60 4.2.3.7. Purging and Roll-Overs . . . . . . . . . . . . . 60 4.2.3.8. Southbound Default Route Origination . . . . . . 61 4.2.3.9. Northbound TIE Flooding Reduction . . . . . . . . 62 4.2.3.10. Special Considerations . . . . . . . . . . . . . 67 4.2.4. Reachability Computation . . . . . . . . . . . . . . 67 4.2.4.1. Northbound SPF . . . . . . . . . . . . . . . . . 67 4.2.4.2. Southbound SPF . . . . . . . . . . . . . . . . . 68 4.2.4.3. East-West Forwarding Within a non-ToF Level . . . 69 4.2.4.4. East-West Links Within ToF Level . . . . . . . . 69 4.2.5. Automatic Disaggregation on Link & Node Failures . . 69 4.2.5.1. Positive, Non-transitive Disaggregation . . . . . 69 4.2.5.2. Negative, Transitive Disaggregation for Fallen Leaves . . . . . . . . . . . . . . . . . . . . . 73 4.2.6. Attaching Prefixes . . . . . . . . . . . . . . . . . 75 4.2.7. Optional Zero Touch Provisioning (ZTP) . . . . . . . 84 4.2.7.1. Terminology . . . . . . . . . . . . . . . . . . . 85 4.2.7.2. Automatic SystemID Selection . . . . . . . . . . 86 4.2.7.3. Generic Fabric Example . . . . . . . . . . . . . 87 4.2.7.4. Level Determination Procedure . . . . . . . . . . 88 4.2.7.5. ZTP FSM . . . . . . . . . . . . . . . . . . . . . 89 4.2.7.6. Resulting Topologies . . . . . . . . . . . . . . 95 4.2.8. Stability Considerations . . . . . . . . . . . . . . 97 4.3. Further Mechanisms . . . . . . . . . . . . . . . . . . . 98 4.3.1. Overload Bit . . . . . . . . . . . . . . . . . . . . 98 4.3.2. Optimized Route Computation on Leaves . . . . . . . . 98 4.3.3. Mobility . . . . . . . . . . . . . . . . . . . . . . 98 4.3.3.1. Clock Comparison . . . . . . . . . . . . . . . . 100 4.3.3.2. Interaction between Time Stamps and Sequence Counters . . . . . . . . . . . . . . . . . . . . 100 4.3.3.3. Anycast vs. Unicast . . . . . . . . . . . . . . . 101 4.3.3.4. Overlays and Signaling . . . . . . . . . . . . . 101 4.3.4. Key/Value Store . . . . . . . . . . . . . . . . . . . 101 4.3.4.1. Southbound . . . . . . . . . . . . . . . . . . . 101 4.3.4.2. Northbound . . . . . . . . . . . . . . . . . . . 102 4.3.5. Interactions with BFD . . . . . . . . . . . . . . . . 102 4.3.6. Fabric Bandwidth Balancing . . . . . . . . . . . . . 103 Przygienda, et al. Expires November 27, 2020 [Page 3] Internet-Draft RIFT May 2020 4.3.6.1. Northbound Direction . . . . . . . . . . . . . . 103 4.3.6.2. Southbound Direction . . . . . . . . . . . . . . 106 4.3.7. Label Binding . . . . . . . . . . . . . . . . . . . . 106 4.3.8. Leaf to Leaf Procedures . . . . . . . . . . . . . . . 106 4.3.9. Address Family and Multi Topology Considerations . . 106 4.3.10. Reachability of Internal Nodes in the Fabric . . . . 107 4.3.11. One-Hop Healing of Levels with East-West Links . . . 107 4.4. Security . . . . . . . . . . . . . . . . . . . . . . . . 107 4.4.1. Security Model . . . . . . . . . . . . . . . . . . . 107 4.4.2. Security Mechanisms . . . . . . . . . . . . . . . . . 109 4.4.3. Security Envelope . . . . . . . . . . . . . . . . . . 110 4.4.4. Weak Nonces . . . . . . . . . . . . . . . . . . . . . 113 4.4.5. Lifetime . . . . . . . . . . . . . . . . . . . . . . 114 4.4.6. Key Management . . . . . . . . . . . . . . . . . . . 114 4.4.7. Security Association Changes . . . . . . . . . . . . 114 5. Examples . . . . . . . . . . . . . . . . . . . . . . . . . . 115 5.1. Normal Operation . . . . . . . . . . . . . . . . . . . . 115 5.2. Leaf Link Failure . . . . . . . . . . . . . . . . . . . . 117 5.3. Partitioned Fabric . . . . . . . . . . . . . . . . . . . 118 5.4. Northbound Partitioned Router and Optional East-West Links . . . . . . . . . . . . . . . . . . . . . . . . . . 119 6. Implementation and Operation: Further Details . . . . . . . . 120 6.1. Considerations for Leaf-Only Implementation . . . . . . . 120 6.2. Considerations for Spine Implementation . . . . . . . . . 121 6.3. Adaptations to Other Proposed Data Center Topologies . . 121 6.4. Originating Non-Default Route Southbound . . . . . . . . 122 7. Security Considerations . . . . . . . . . . . . . . . . . . . 122 7.1. General . . . . . . . . . . . . . . . . . . . . . . . . . 122 7.2. ZTP . . . . . . . . . . . . . . . . . . . . . . . . . . . 122 7.3. Lifetime . . . . . . . . . . . . . . . . . . . . . . . . 123 7.4. Packet Number . . . . . . . . . . . . . . . . . . . . . . 123 7.5. Outer Fingerprint Attacks . . . . . . . . . . . . . . . . 123 7.6. TIE Origin Fingerprint DoS Attacks . . . . . . . . . . . 123 7.7. Host Implementations . . . . . . . . . . . . . . . . . . 124 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 124 8.1. Requested Multicast and Port Numbers . . . . . . . . . . 124 8.2. Requested Registries with Suggested Values . . . . . . . 125 8.2.1. Registry RIFT_v4/common/AddressFamilyType . . . . . . 125 8.2.1.1. Requested Entries . . . . . . . . . . . . . . . . 125 8.2.2. Registry RIFT_v4/common/HierarchyIndications . . . . 125 8.2.2.1. Requested Entries . . . . . . . . . . . . . . . . 125 8.2.3. Registry RIFT_v4/common/IEEE802_1ASTimeStampType . . 125 8.2.3.1. Requested Entries . . . . . . . . . . . . . . . . 126 8.2.4. Registry RIFT_v4/common/IPAddressType . . . . . . . . 126 8.2.4.1. Requested Entries . . . . . . . . . . . . . . . . 126 8.2.5. Registry RIFT_v4/common/IPPrefixType . . . . . . . . 126 8.2.5.1. Requested Entries . . . . . . . . . . . . . . . . 126 8.2.6. Registry RIFT_v4/common/IPv4PrefixType . . . . . . . 126 Przygienda, et al. Expires November 27, 2020 [Page 4] Internet-Draft RIFT May 2020 8.2.6.1. Requested Entries . . . . . . . . . . . . . . . . 126 8.2.7. Registry RIFT_v4/common/IPv6PrefixType . . . . . . . 126 8.2.7.1. Requested Entries . . . . . . . . . . . . . . . . 127 8.2.8. Registry RIFT_v4/common/PrefixSequenceType . . . . . 127 8.2.8.1. Requested Entries . . . . . . . . . . . . . . . . 127 8.2.9. Registry RIFT_v4/common/RouteType . . . . . . . . . . 127 8.2.9.1. Requested Entries . . . . . . . . . . . . . . . . 127 8.2.10. Registry RIFT_v4/common/TIETypeType . . . . . . . . . 128 8.2.10.1. Requested Entries . . . . . . . . . . . . . . . 128 8.2.11. Registry RIFT_v4/common/TieDirectionType . . . . . . 128 8.2.11.1. Requested Entries . . . . . . . . . . . . . . . 128 8.2.12. Registry RIFT_v4/encoding/Community . . . . . . . . . 128 8.2.12.1. Requested Entries . . . . . . . . . . . . . . . 128 8.2.13. Registry RIFT_v4/encoding/KeyValueTIEElement . . . . 129 8.2.13.1. Requested Entries . . . . . . . . . . . . . . . 129 8.2.14. Registry RIFT_v4/encoding/LIEPacket . . . . . . . . . 129 8.2.14.1. Requested Entries . . . . . . . . . . . . . . . 129 8.2.15. Registry RIFT_v4/encoding/LinkCapabilities . . . . . 130 8.2.15.1. Requested Entries . . . . . . . . . . . . . . . 130 8.2.16. Registry RIFT_v4/encoding/LinkIDPair . . . . . . . . 130 8.2.16.1. Requested Entries . . . . . . . . . . . . . . . 131 8.2.17. Registry RIFT_v4/encoding/Neighbor . . . . . . . . . 131 8.2.17.1. Requested Entries . . . . . . . . . . . . . . . 131 8.2.18. Registry RIFT_v4/encoding/NodeCapabilities . . . . . 131 8.2.18.1. Requested Entries . . . . . . . . . . . . . . . 132 8.2.19. Registry RIFT_v4/encoding/NodeFlags . . . . . . . . . 132 8.2.19.1. Requested Entries . . . . . . . . . . . . . . . 132 8.2.20. Registry RIFT_v4/encoding/NodeNeighborsTIEElement . . 132 8.2.20.1. Requested Entries . . . . . . . . . . . . . . . 132 8.2.21. Registry RIFT_v4/encoding/NodeTIEElement . . . . . . 132 8.2.21.1. Requested Entries . . . . . . . . . . . . . . . 133 8.2.22. Registry RIFT_v4/encoding/PacketContent . . . . . . . 133 8.2.22.1. Requested Entries . . . . . . . . . . . . . . . 133 8.2.23. Registry RIFT_v4/encoding/PacketHeader . . . . . . . 133 8.2.23.1. Requested Entries . . . . . . . . . . . . . . . 134 8.2.24. Registry RIFT_v4/encoding/PrefixAttributes . . . . . 134 8.2.24.1. Requested Entries . . . . . . . . . . . . . . . 134 8.2.25. Registry RIFT_v4/encoding/PrefixTIEElement . . . . . 134 8.2.25.1. Requested Entries . . . . . . . . . . . . . . . 135 8.2.26. Registry RIFT_v4/encoding/ProtocolPacket . . . . . . 135 8.2.26.1. Requested Entries . . . . . . . . . . . . . . . 135 8.2.27. Registry RIFT_v4/encoding/TIDEPacket . . . . . . . . 135 8.2.27.1. Requested Entries . . . . . . . . . . . . . . . 135 8.2.28. Registry RIFT_v4/encoding/TIEElement . . . . . . . . 135 8.2.28.1. Requested Entries . . . . . . . . . . . . . . . 136 8.2.29. Registry RIFT_v4/encoding/TIEHeader . . . . . . . . . 136 8.2.29.1. Requested Entries . . . . . . . . . . . . . . . 137 8.2.30. Registry RIFT_v4/encoding/TIEHeaderWithLifeTime . . . 137 Przygienda, et al. Expires November 27, 2020 [Page 5] Internet-Draft RIFT May 2020 8.2.30.1. Requested Entries . . . . . . . . . . . . . . . 137 8.2.31. Registry RIFT_v4/encoding/TIEID . . . . . . . . . . . 137 8.2.31.1. Requested Entries . . . . . . . . . . . . . . . 138 8.2.32. Registry RIFT_v4/encoding/TIEPacket . . . . . . . . . 138 8.2.32.1. Requested Entries . . . . . . . . . . . . . . . 138 8.2.33. Registry RIFT_v4/encoding/TIREPacket . . . . . . . . 138 8.2.33.1. Requested Entries . . . . . . . . . . . . . . . 138 9. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 138 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 139 10.1. Normative References . . . . . . . . . . . . . . . . . . 139 10.2. Informative References . . . . . . . . . . . . . . . . . 141 Appendix A. Sequence Number Binary Arithmetic . . . . . . . . . 143 Appendix B. Information Elements Schema . . . . . . . . . . . . 144 B.1. common.thrift . . . . . . . . . . . . . . . . . . . . . . 146 B.2. encoding.thrift . . . . . . . . . . . . . . . . . . . . . 152 Appendix C. Constants . . . . . . . . . . . . . . . . . . . . . 161 C.1. Configurable Protocol Constants . . . . . . . . . . . . . 161 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 163 1. Authors This work is a product of a list of individuals which are all to be considered major contributors independent of the fact whether their name made it to the limited boilerplate author's list or not. Tony Przygienda, Ed. | Alankar Sharma | Pascal Thubert Juniper Networks | Comcast | Cisco Bruno Rijsman | Ilya Vershkov | Dmitry Afanasiev Individual | Mellanox | Yandex Don Fedyk | Alia Atlas | John Drake Individual | Individual | Juniper Table 1: RIFT Authors 2. Introduction Clos [CLOS] and Fat-Tree [FATTREE] topologies have gained prominence in today's networking, primarily as result of the paradigm shift towards a centralized data-center based architecture that is poised to deliver a majority of computation and storage services in the future. Today's current routing protocols were geared towards a network with an irregular topology and low degree of connectivity originally but given they were the only available options, consequently several attempts to apply those protocols to Clos have been made. Most successfully BGP [RFC4271] [RFC7938] has been extended to this purpose, not as much due to its inherent suitability Przygienda, et al. Expires November 27, 2020 [Page 6] Internet-Draft RIFT May 2020 but rather because the perceived capability to easily modify BGP and the immanent difficulties with link-state [DIJKSTRA] based protocols to optimize topology exchange and converge quickly in large scale densely meshed topologies. The incumbent protocols precondition normally extensive configuration or provisioning during bring up and re-dimensioning. This tends to be viable only for a set of organizations with according networking operation skills and budgets. For many IP fabric builders a desirable protocol would be one that auto-configures itself and deals with failures and mis-configurations with a minimum of human intervention only. Such a solution would allow local IP fabric bandwidth to be consumed in a 'standard component' fashion, i.e. provision it much faster and operate it at much lower costs than today, much like compute or storage is consumed already. In looking at the problem through the lens of data center requirements, RIFT addresses challenges in IP fabric routing not through an incremental modification of either a link-state (distributed computation) or distance-vector (diffused computation) but rather a mixture of both, colloquially best described as "link- state towards the spine" and "distance vector towards the leaves". In other words, "bottom" levels are flooding their link-state information in the "northern" direction while each node generates under normal conditions a "default route" and floods it in the "southern" direction. This type of protocol allows naturally for highly desirable aggregation. Alas, such aggregation could blackhole traffic in cases of misconfiguration or while failures are being resolved or even cause partial network partitioning and this has to be addressed by some adequate mechanism. The approach RIFT takes is described in Section 4.2.5 and is basically based on automatic, sufficient disaggregation of prefixes in case of link and node failures. For the visually oriented reader, Figure 1 presents a first level simplified view of the resulting information and routes on a RIFT fabric. The top of the fabric is holding in its link-state database the nodes below it and the routes to them. In the second row of the database table we indicate that partial information of other nodes in the same level is available as well. The details of how this is achieved will be postponed for the moment. When we look at the "bottom" of the fabric, the leaves, we see that the topology is basically empty and they only hold a load balanced default route to the next level under normal conditions. The balance of this document details a dedicated IP fabric routing protocol, fills in the specification details and ultimately includes resulting security considerations. Przygienda, et al. Expires November 27, 2020 [Page 7] Internet-Draft RIFT May 2020 . [A,B,C,D] . [E] . +-----+ +-----+ . | E | | F | A/32 @ [C,D] . +-+-+-+ +-+-+-+ B/32 @ [C,D] . | | | | C/32 @ C . | | +-----+ | D/32 @ D . | | | | . | +------+ | . | | | | . [A,B] +-+---+ | | +---+-+ [A,B] . [D] | C +--+ +-+ D | [C] . +-+-+-+ +-+-+-+ . 0/0 @ [E,F] | | | | 0/0 @ [E,F] . A/32 @ A | | +-----+ | A/32 @ A . B/32 @ B | | | | B/32 @ B . | +------+ | . | | | | . +-+---+ | | +---+-+ . | A +--+ +-+ B | . 0/0 @ [C,D] +-----+ +-----+ 0/0 @ [C,D] Figure 1: RIFT Information Distribution 2.1. Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 8174 [RFC8174]. 3. Reference Frame 3.1. Terminology This section presents the terminology used in this document. It is assumed that the reader is thoroughly familiar with the terms and concepts used in OSPF [RFC2328] and IS-IS [ISO10589-Second-Edition], [ISO10589] as well as the according graph theoretical concepts of shortest path first (SPF) [DIJKSTRA] computation and DAGs. Crossbar: Physical arrangement of ports in a switching matrix without implying any further scheduling or buffering disciplines. Clos/Fat Tree: This document uses the terms Clos and Fat Tree interchangeably whereas it always refers to a folded spine-and- leaf topology with possibly multiple Points of Delivery (PoDs) and one or multiple Top of Fabric (ToF) planes. Several modifications Przygienda, et al. Expires November 27, 2020 [Page 8] Internet-Draft RIFT May 2020 such as leaf-2-leaf shortcuts and multiple level shortcuts are possible and described further in the document. Directed Acyclic Graph (DAG): A finite directed graph with no directed cycles (loops). If links in Clos are considered as either being all directed towards the top or vice versa, each of such two graphs is a DAG. Folded Spine-and-Leaf: In case Clos fabric input and output stages are analogous, the fabric can be "folded" to build a "superspine" or top which we will call Top of Fabric (ToF) in this document. Level: Clos and Fat Tree networks are topologically partially ordered graphs and 'level' denotes the set of nodes at the same height in such a network, where the bottom level (leaf) is the level with lowest value. A node has links to nodes one level down and/or one level up. Under some circumstances, a node may have links to nodes at the same level. As footnote: Clos terminology uses often the concept of "stage" but due to the folded nature of the Fat Tree we do not use it to prevent misunderstandings. Superspine vs. Aggregation and Spine vs. Edge/Leaf: Traditional level names in 5-stages folded Clos for Level 2, 1 and 0 respectively. We normalize this language to talk about top-of- fabric (ToF), top-of-pod (ToP) and leaves. Zero Touch Provisioning (ZTP): Optional RIFT mechanism which allows to derive node levels automatically based on minimum configuration (only ToF property has to be provisioned on according nodes). Point of Delivery (PoD): A self-contained vertical slice or subset of a Clos or Fat Tree network containing normally only level 0 and level 1 nodes. A node in a PoD communicates with nodes in other PoDs via the Top-of-Fabric. We number PoDs to distinguish them and use PoD #0 to denote "undefined" PoD. Top of PoD (ToP): The set of nodes that provide intra-PoD communication and have northbound adjacencies outside of the PoD, i.e. are at the "top" of the PoD. Top of Fabric (ToF): The set of nodes that provide inter-PoD communication and have no northbound adjacencies, i.e. are at the "very top" of the fabric. ToF nodes do not belong to any PoD and are assigned "undefined" PoD value to indicate the equivalent of "any" PoD. Spine: Any nodes north of leaves and south of top-of-fabric nodes. Multiple layers of spines in a PoD are possible. Przygienda, et al. Expires November 27, 2020 [Page 9] Internet-Draft RIFT May 2020 Leaf: A node without southbound adjacencies. Its level is 0 (except cases where it is deriving its level via ZTP and is running without LEAF_ONLY which will be explained in Section 4.2.7). Top-of-fabric Plane or Partition: In large fabrics top-of-fabric switches may not have enough ports to aggregate all switches south of them and with that, the ToF is 'split' into multiple independent planes. Introduction and Section 4.1.2 explains the concept in more detail. A plane is subset of ToF nodes that see each other through south reflection or E-W links. Radix: A radix of a switch is basically number of switching ports it provides. It's sometimes called fanout as well. North Radix: Ports cabled northbound to higher level nodes. South Radix: Ports cabled southbound to lower level nodes. South/Southbound and North/Northbound (Direction): When describing protocol elements and procedures, we will be using in different situations the directionality of the compass. I.e., 'south' or 'southbound' mean moving towards the bottom of the Clos or Fat Tree network and 'north' and 'northbound' mean moving towards the top of the Clos or Fat Tree network. Northbound Link: A link to a node one level up or in other words, one level further north. Southbound Link: A link to a node one level down or in other words, one level further south. East-West Link: A link between two nodes at the same level. East- West links are normally not part of Clos or "fat-tree" topologies. Leaf shortcuts (L2L): East-West links at leaf level will need to be differentiated from East-West links at other levels. Routing on the host (RotH): Modern data center architecture variant where servers/leaves are multi-homed and consecutively participate in routing. Northbound representation: Subset of topology information flooded towards higher levels of the fabric. Southbound representation: Subset of topology information sent towards a lower level. Przygienda, et al. Expires November 27, 2020 [Page 10] Internet-Draft RIFT May 2020 South Reflection: Often abbreviated just as "reflection" it defines a mechanism where South Node TIEs are "reflected" from the level south back up north to allow nodes in the same level without E-W links to "see" each other's node TIEs. TIE: This is an acronym for a "Topology Information Element". TIEs are exchanged between RIFT nodes to describe parts of a network such as links and address prefixes, in a fashion similar to ISIS LSPs or OSPF LSAs. A TIE has always a direction and a type. We will talk about North TIEs (sometimes abbreviated as N-TIEs) when talking about TIEs in the northbound representation and South-TIEs (sometimes abbreviated as S-TIEs) for the southbound equivalent. TIEs have different types such as node and prefix TIEs. Node TIE: This stands as acronym for a "Node Topology Information Element" that contains all adjacencies the node discovered and information about node itself. Node TIE should NOT be confused with a North TIE since "node" defines the type of TIE rather than its direction. Prefix TIE: This is an acronym for a "Prefix Topology Information Element" and it contains all prefixes directly attached to this node in case of a North TIE and in case of South TIE the necessary default routes the node advertises southbound. Key Value TIE: A South TIE that is carrying a set of key value pairs [DYNAMO]. It can be used to distribute information in the southbound direction within the protocol. TIDE: Topology Information Description Element, equivalent to CSNP in ISIS. TIRE: Topology Information Request Element, equivalent to PSNP in ISIS. It can both confirm received and request missing TIEs. De-aggregation/Disaggregation: Process in which a node decides to advertise more specific prefixes Southwards, either positively to attract the corresponding traffic, or negatively to repel it. Disaggregation is performed to prevent black-holing and suboptimal routing to the more specific prefixes. LIE: This is an acronym for a "Link Information Element", largely equivalent to HELLOs in IGPs and exchanged over all the links between systems running RIFT to form three way adjacencies. Flood Repeater (FR): A node can designate one or more northbound neighbor nodes to be flood repeaters. The flood repeaters are responsible for flooding northbound TIEs further north. They are Przygienda, et al. Expires November 27, 2020 [Page 11] Internet-Draft RIFT May 2020 similar to MPR in OSLR. The document sometimes calls them flood leaders as well. Bandwidth Adjusted Distance (BAD): Each RIFT node can calculate the amount of northbound bandwidth available towards a node compared to other nodes at the same level and can modify the route distance accordingly to allow for the lower level to adjust their load balancing towards spines. Overloaded: Applies to a node advertising `overload` attribute as set. The semantics closely follow the meaning of the same attribute in [ISO10589-Second-Edition]. Interface: A layer 3 entity over which RIFT control packets are exchanged. Three-Way Adjacency: RIFT tries to form a unique adjacency over an interface and exchange local configuration and necessary ZTP information. An adjacency is only advertised in node TIEs and used for computations after it achieved three-way state, i.e. both routers reflected each other in LIEs including relevant security information. LIEs before three-way state is reached may carry ZTP related information already. Bi-directional Adjacency: Bidirectional adjacency is an adjacency where nodes of both sides of the adjacency advertised it in the node TIEs with the correct levels and system IDs. Bi- directionality is used to check in different algorithms whether the link should be included. Neighbor: Once a three-way adjacency has been formed a neighborship relationship contains the neighbor's properties. Multiple adjacencies can be formed to a remote node via parallel interfaces but such adjacencies are NOT sharing a neighbor structure. Saying "neighbor" is thus equivalent to saying "a three-way adjacency". Cost: The term signifies the weighted distance between two neighbors. Distance: Sum of costs (bound by infinite distance) between two nodes. Shortest-Path First (SPF): A well-known graph algorithm attributed to Dijkstra that establishes a tree of shortest paths from a source to destinations on the graph. We use SPF acronym due to its familiarity as general term for the node reachability calculations RIFT can employ to ultimately calculate routes of which Dijkstra algorithm is one. Przygienda, et al. Expires November 27, 2020 [Page 12] Internet-Draft RIFT May 2020 North SPF (N-SPF): A reachability calculation that is progressing northbound, as example SPF that is using South Node TIEs only. Normally it progresses a single hop only and installs default routes. South SPF (S-SPF): A reachability calculation that is progressing southbound, as example SPF that is using North Node TIEs only. Security Envelope RIFT packets are flooded within an authenticated security envelope that allows to protect the integrity of information a node accepts. 3.2. Topology Przygienda, et al. Expires November 27, 2020 [Page 13] Internet-Draft