Token Service Flow Awareness: Use Cases and Requirements

In recent years, there has been a significant increase in the use of generative AI, LLM MaaS platforms, and autonomous AI Agents as mainstream internet services. Unlike traditional web, video, and file transfer traffic, AI services generate unique token-bearing flows. These include prompt/output token streaming, cross-Agent A2A collaborative token packets, and authorization tokens carried in JWT/OAuth headers. AI token flows have distinct business attributes, such as ultra-low Time-To-First-Token (TTFT) latency for interactive Agent dialogue, longer transmission delay tolerance for offline batch inference, and stable cross-domain connectivity for cross-industry multi-Agent collaboration. However, current network traffic classification systems only identify generic application domains through SNI and DNS fingerprints, lacking fine-grained awareness of token volume, task priority, and Agent identity. This lack of dedicated token flow identification capabilities hinders operators from deploying differentiated QoS strategies, implementing token-based metering billing, and executing compute-aware routing optimization. As a result, there are consequences such as inconsistent dialogue latency, unfair bandwidth resource contention, inability to quantify AI service revenue, and unmonitored abnormal token consumption attacks. Against this background, this document outlines the use cases and requirements for token service flow awareness, providing the IETF working group with a standardized reference to better support the assurance of the user’s end-to-end experience..

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.

This document categorizes use cases into service identification, operation & management and routing,

The core goal of sevice identification is to map raw packet flows to standardized token service types to deliver differentiated user experience and tiered service packages. For example, personal multi-modal assistants, vehicle-mounted MUI Agents and customer service Agents generate low-latency interactive token streams. After identification, the network can implement various optimization measures to reduce TTFT jitter.

The use case of operation management focus on full lifecycle monitoring, risk control and statistical analysis based on identified token flow tags. Network management platforms aggregate identified token flow data to analyze industry distribution of Agent services, and forecast token traffic growth for capacity planning of edge computing and backbone links.

The use case of routing realize compute-network collaborative steering by leveraging token flow business attributes. For example, after marking interactive token flows, routing planes dynamically select nearby edge computing gateway links, avoiding long-distance backbone transmission to cut end-to-end latency. This enables latency-priority routing of real-time token streams.

Corresponding to the three use cases, this chapter defines mandatory and optional requirements for service identification, operation management and routing steering respectively.

Identification equipment SHALL parse plaintext token count headers in SSE/gRPC/MCP streaming protocols, and extract Agent ID, task type and authorization token fields. For fully encrypted TLS traffic, the system MUST implement traffic fingerprint recognition based on packet interval, burst small-packet characteristics and TTFT statistical features to classify token flows without payload decryption.

To ensure the smooth operation of the service, the operations management platform should support the ability to record service logs. All identified token flows SHALL generate structured logs containing five-tuple, token label, total token volume, start/end timestamp and Agent identity information, , stored for no less than user service period for audit.

It should support different routing policies for different services to provide differentiated service guarantees and meet the needs of different users. In addition, routing nodes supporting KV context cache reuse shall match token flow labels to cache corresponding prompt data, reducing repeated token transmission volume and link load.

This document outlines the use cases and requirements for token service flow awareness and provides an overview from three perspectives: service identification, operation management and routing,

TBD.