]> Precise Priority-based Flow Control Notification with RoCEv2 ZTE Corporation
xiong.quan@zte.com.cn
ZTE Corporation
zhu.xiangyang@zte.com.cn
rtgwg This document defines a format for Precise Priority-based Flow Control (PPFC) notifications within RoCEv2 (RDMA over Converged Ethernet version 2) networks. The proposed format enables network devices experiencing congestion to generate explicit congestion signals that can be efficiently carried back to the source clients. This facilitates fast, fine-grained flow control, complementing traditional end-to-end congestion control and improving performance in high-throughput, low-latency environments.
Introduction RoCEv2 (RDMA over Converged Ethernet version 2) is widely deployed in data centers and high-performance computing clusters to provide low-latency, kernel-bypass networking. However, managing congestion in these environments remains a challenge. End-to-end congestion control mechanisms, such as those using ECN or CNP (Congestion Notification Packets), can incur Round-Trip Time (RTT) delays before a sending host adjusts its rate. Precise Priority-based Control (PPFC) is a technique to instantly pause the traffic sources, based on explicit notifications from the network. The PPFC notification can be also sent to the traffic source directly with the message format which is designed to be:
  • Efficient: Adds minimal overhead.
  • Explicit: Carries detailed information about the congestion event (e.g., destination address, recommended action).
  • Interoperable: Can be processed by both network and hosts.
This document defines a lightweight, generic encapsulation format for PPFC notifications within RoCEv2 networks.
Conventions Used in This Document
Abbreviations RTT: Round-Trip Time TCP: Transfer Control Protocol RDMA: Remote Direct Memory Access Round-Trip Time QUIC: Quick UDP Internet Connections ECN: Explicit Congestion Notification CNP: Congestion Notification Packet GRH: Global Routing Header (InfiniBand) PPFC: Source Flow Control RoCEv2: RDMA over Converged Ethernet version 2
Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.
PPFC Notification with RoCEv2 The PPFC Notification message reuses the basic RoCEv2 encapsulation but is identified as a new extension. The proposed packet format is as follows: where:
  • UDP Destination Port: SHOULD be used to distinguish the RoCEv2 traffic. It indicates the RoCEv2 Base Transport Header (BTH).
  • P (1 bit): A new bit MUST be set to 1 for PPFC notification. This bit indicates that PPFC Extension Fields follow the BTH.
  • Congested Destination IP (variable) : Indicates the IPv4 or IPv6 address of the congested network node.
  • PT (2bits): indicates the action type of the PPFC notification for the ingress to take actions of traffic. It can be set to 00 "stop", 01 "resume", 10 "alarm", 11 "hold".
  • Congestion Port (16bits): Identifier of the congested port.
  • Pause Duration (16bits): Recommended pausing interval, in microseconds.
The other fields are identical to the BTH fields.
Security Considerations To be discussed in future versions of this document.
IANA Considerations TBD.
Normative References User Datagram Protocol Key words for use in RFCs to Indicate Requirement Levels In many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words RFC 2119 specifies common key words that may be used in protocol specifications. This document aims to reduce the ambiguity by clarifying that only UPPERCASE usage of the key words have the defined special meanings. Precise Priority-based Flow Control Notification ZTE Corporation ZTE Corporation This document specifies the notification mechanism for Precise Priority-based Flow Control (PPFC), defining the message formats and network actions for communicating congestion information among network nodes. The PPFC notification enables rapid congestion signaling, allowing per-flow flow control for traffic from source.