TEMPO: Timing Extension for Media Playout Orchestration over MoQ

Introduction This document specifies TEMPO (Timing Extension for Media Playout Orchestration), a synchronization mechanism for the Media over QUIC Transport (MoQT) protocol . MoQT enables distribution of media content through relay networks. When multiple subscribers receive the same track objects through different network paths, achieving synchronized playback is challenging due to:

Clock skew between subscriber devices: Even with NTP synchronization, consumer devices can drift by 10-100ms from reference time.
Variable network delays through different relay paths: Network jitter and routing differences introduce additional delay variance in delivery times.
Lack of shared timing reference across the distribution network: Publishers and relays operate on different clock domains without coordination.

Motivation and Use Cases

Live Sports Event Synchronization Multiple viewers watching the same live sports event should see goals, touchdowns, or key moments simultaneously to maintain the shared experience and prevent spoilers through social media. Challenge: Viewers connected through different CDN nodes experience 50-300ms delivery variance, causing spoilers when some viewers see goals before others.

Interactive Streaming Synchronization Streamers conducting live Q&A sessions or interactive games require synchronized playback so that viewer reactions and comments align temporally with the content. For example, Twitch-style interactive streams with real-time polls and reactions. Challenge: Streamer asks question at T=0, but viewers see it at T+100ms to T+400ms due to network variance, causing response timing issues.

Multi-Room Audio System Distributed audio systems playing the same audio across multiple speakers in the same room or adjacent rooms need precise synchronization to avoid audio artifacts and maintain coherent sound. Challenge: Different network paths to each room cause echo effects.

Emergency Broadcasting Natural disaster alerts can play out on many devices at the same locations and if they are not synchronized, it can make it harder to understand the alert. Challenge: Multiple unsynchronized playouts in the same room make it harder to understand audio.

Collaborative Viewing Watch parties, online classrooms, and virtual meetings benefit from synchronized playback to enable meaningful real-time discussion. Challenge: When participants can discuss what they are seeing in real time, even small timing differences become noticeable and disruptive.

Solution Overview TEMPO addresses these challenges by adding timing metadata to each MoQT object. The Original Publisher (OrigPub) attaches two kinds of information: a capture timestamp indicating when the media was captured, paired with a playout delay that tells the subscriber how long after capture time to present the object, and a hop timestamp indicating when the object was last forwarded. As relays forward the object through the network, each relay that has accurate time overwrites the hop timestamp with its own current clock value. This means the subscriber always receives a fresh timestamp from its closest relay, which it can compare against its own local receive time and transport delay to estimate if its clock is accurate or use for determining playout time. Subscribers use the capture timestamp, playout delay, and hop timestamp to compute exactly when to render each object. If a subscriber finds itself consistently falling behind the target, it reports this directly to a PlaySyncServer — a coordination entity that receives feedback from all subscribers and decides whether the playout delay needs to be adjusted. If most subscribers are struggling, the PlaySyncServer tells the OrigPub to increase the delay on future objects, giving everyone more time to buffer and catch up. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.

TEMPO Object Extension Properties This section defines three TEMPO Object Extension properties that carry timing information on each object as it flows through the relay network. Together, they give end subscribers enough information to determine when to present each object and to detect when they are falling out of sync.

CaptureTimestamp The CaptureTimestamp tells the end subscriber the time at which the start of the media in the object was captured. If the object contains 20 ms of audio, this time indicates the time of the first sample. The Original Publisher sets this value when it creates the object. It is an immutable object property. This property is more accurate when NTP support is available at the original publisher.

PlayoutDelay The PlayoutDelay tells the end subscriber how long to wait after the capture time of an object before presenting it. For example, if the PlayoutDelay is 200ms, the subscriber should buffer the object until the current time is 200 ms greater than the capture time. The Original Publisher sets this value when it creates the object. It is an immutable object property. If it is omitted, the value is assumed to be the same as the previous object on the same track.

HopTime The HopTime records when the object was last forwarded. The Original Publisher initially sets it to its own current time when creating the object. As the object passes through each relay, that relay overwrites the HopTime with its own current clock value before forwarding. This means the end subscriber always sees the timestamp of its closest relay — the last node that touched the object before it arrived. By comparing the HopTime to its own local receive time, an end subscriber can estimate the one-way network delay on the final hop. This is valuable because it isolates the most variable part of the path (the last mile to the subscriber) from the rest of the relay chain. Comparing this to the QUIC RTT estimate can often allow a subscriber to detect if its local clock is wrong. Unlike CaptureTimestamp and PlayoutDelay, HopTime is mutable. Every relay that supports this extension MUST overwrite it with its own current timestamp of approximately when it started sending the object.

Sync Support Parameter Subscribers indicate their sync capabilities when they subscribe to a track. This allows relays to know whether they need to perform HopTime stamping.

Architecture

Topology The system consists of an Original Publisher, a tree of relays, end subscribers, and an optional application-defined PlaySyncServer. Media objects flow from the OrigPub down through relays to subscribers. Subscribers report their sync state directly to the PlaySyncServer (not through the relay network). The PlaySyncServer communicates delay adjustments to the OrigPub which it uses to update the PlayoutDelay. ─── Solid lines: media flow (OrigPub → Relays → Subscribers) via MOQ ─ ─ Dashed lines: subscriber feedback directly to PlaySyncServer ─ ─ PlaySyncServer → OrigPub: timing adjustments (signaling)

Scope The mechanism by which end subscribers discover the PlaySyncServer is out of scope for this specification. The signaling protocol used by subscribers to report feedback directly to the PlaySyncServer (e.g., MoQ Metrics, HTTP, or other mechanism) is also out of scope. Similarly, the specific signaling protocol between the PlaySyncServer and the OrigPub is also out of scope.

NTP Capability Scenarios Not all deployments have NTP available on every node. This specification supports three scenarios depending on which nodes have synchronized clocks: Scenario 1: OrigPub, Relays, and End Subscribers are all NTP capable. This is the ideal case. The OrigPub sets a CaptureTimestamp and PlayoutDelay on each object, and subscribers simply present the object at the playout time which is the capture time plus the playout delay. HopTime is still useful for monitoring delay and detecting that the end subscriber has NTP synchronized time but is not strictly needed for playout decisions. Scenario 2: OrigPub and Relays are NTP capable, but End Subscribers are not. The OrigPub still sets CaptureTimestamp and PlayoutDelay to compute a playout time. The end subscriber estimates NTP time using the HopTime and adding half of the RTT estimate from the QUIC transport that provided the HopTime. These values can be smoothed and, along with an unsynchronized local clock, can be used to get a close estimate of NTP time. Scenario 3: Only Relays are NTP capable. The OrigPub itself has no synchronized clock but it can subscribe to its own media to estimate NTP time in the same way the end subscriber does in Scenario 2.

Original Publisher (OrigPub) The Original Publisher is responsible for creating media objects and attaching the timing metadata that drives synchronization. When it creates each object, it sets the CaptureTimestamp and PlayoutDelay to indicate when the object should be presented. It also sets the initial HopTime to its own current time.

Steady-State Timing During normal operation, the OrigPub increments CaptureTimestamp on each successive object by the media frame duration. For example, if the track carries 20ms audio frames, each object's CaptureTimestamp is 20ms later than the previous one. The PlayoutDelay remains constant unless adjusted by the PlaySyncServer. This produces a continuous, evenly spaced playout schedule that subscribers follow.

PlayoutDelay updates from PlaySyncServer The OrigPub receives timing adjustment instructions from the PlaySyncServer. When the PlaySyncServer decides (based on subscriber feedback it receives directly) that the playout target needs to change — for example, increasing delay by 100ms — the OrigPub applies that correction to the next object it produces and continues incrementing by the media frame duration from the new baseline onward. The effect is a one-time shift in the playout schedule; all subsequent objects maintain the same spacing relative to the corrected point.

Relays When forwarding an object, a relay MUST replace the HopTime value with its own current timestamp of approximately when it started sending the object. Relays MUST use NTP-synchronized clocks when generating HopTime values. Relays SHOULD only perform HopTime stamping when at least one downstream subscriber has indicated sync support in its SUBSCRIBE parameters, to avoid unnecessary processing overhead.

PlaySyncServer The PlaySyncServer is a logical service defined by the application that receives feedback directly from all subscribers and communicates with the OrigPub to issue timing adjustments. It is responsible for observing the health of synchronization across all subscribers and making decisions about whether timing needs to change. The PlaySyncServer MUST implement the following decision logic based on aggregated subscriber feedback:

If all subscribers report sync state within acceptable bounds, the PlaySyncServer MUST NOT issue any timing adjustments.
If a majority of subscribers report falling behind by a consistent amount, the PlaySyncServer SHOULD instruct the OrigPub to increase the PlayoutDelay on subsequent objects by the reported deficit.

The PlaySyncServer SHOULD implement hysteresis to avoid oscillation.

Optimized Forms

Short Timestamps The timestamps will typically encode to about a 6 byte integer in MoQ. In many applications, there is no requirement to sync media that is delivered more than a few seconds outside of the desired playout time. These applications can send just the low order 14 bits of the timestamps which are referred to as short timestamps. If the application receives a timestamp less than 16384, it is assumed to be a short timestamp. The receiver computes the full timestamp by taking its current time, replacing the lower 14 bits with the received short timestamp, then if that result is greater than the current time, subtracting 16384.

Periodic Marking There is no need to send markings with every object for applications that send objects at a constant rate. Receivers can look at two existing timestamps and extrapolate the timestamps of other objects that are near the same time. Typically these applications also know the rate of the objects. An original publisher that uses periodic marking needs to consider:

If the objects are over datagrams, consider sending information in multiple objects in case one is lost.
If subscribers join mid track, how quickly will they discover the synchronization information.

A typical strategy might be to send the timestamp and playout delay information once every 200ms.

Time Aligned Groups Many applications can have a direct linear function between the MoQ group ID and the capture timestamps. The relationship is defined by an object rate per 720 seconds and the time offset of the first object in the first group. CaptureTimestamp = GroupID * 720000 / objectsPer720s + startTime Where:

objectsPer720s is the number of objects produced in 720 seconds (e.g., for 30 fps video: 30 × 720 = 21600).
startTime is the capture timestamp of the first object in group 0, expressed in milliseconds since the NTP epoch.
CaptureTimestamp is the resulting capture time in the same units as startTime.

The constant 720000 (= 2^7 × 3^2 × 5^4) was chosen to allow a wide range of common frame rates to be exactly represented as integer divisors. RTP often uses 90000 but high frame rates for slow motion video such as 480 fps do not evenly divide into 90000, whereas they do divide evenly into 720000. The objectsPer720s and startTime can be communicated out of band, for example, in the catalog. If the publisher uses Time Aligned Groups, it MUST choose the group number to correctly represent the capture time of the first object in the group based on the out of band rate and start time. End Subscribers can use the group ID to recreate the capture time of the first object in the group. The capture times for subsequent objects within a group can be derived by adding 720000 / objectsPer720s for each object offset. If end subscribers are aware that groups are time-aligned, there is no need to include the CaptureTimestamp property. TODO - add extension to catalog to carry this information

Catalog Based Playout Delay For many applications the Catalog can be used to deliver the PlayoutDelay for a track. If the logical PlaySyncServer wants to update the PlayoutDelay, a new version of the catalog can be pushed with delta encoding on a catalog track that all the clients subscribe to. TODO - define extension to catalog to carry PlayoutDelay.

End Subscribers End subscribers compute playout times from the timing metadata and report sync state to the PlaySyncServer.

Computing Playout Target The subscriber estimates one-way delay from its closest relay: The playout target depends on the NTP scenario (see ): Scenario 1 (all nodes have NTP): Scenario 2 (subscribers lack NTP): Scenario 3 (only relays have NTP):

Sync Feedback Subscribers SHOULD periodically report their sync state to the PlaySyncServer. The PlaySyncServer uses these reports to determine whether a systemic issue exists and, if so, instructs the OrigPub to adjust PlayoutDelay on future objects.

Drift and Jump Handling End subscribers MUST be prepared to handle clock drift (gradual divergence between local and relay clocks) and delay jumps (sudden shifts from route changes or congestion events). Implementations SHOULD apply local compensation to maintain playout continuity and report persistent drift or jumps to the PlaySyncServer.

Security Considerations Subscribers MAY implement reasonableness checks — for example, verifying that HopTime values are monotonically increasing and that the delta between consecutive objects' HopTime values is bounded. The connection between the OrigPub and PlaySyncServer MUST be authenticated and integrity-protected to prevent injection of false playout targets. Malicious subscribers could send fabricated feedback reports claiming to be far behind, tricking the PlaySyncServer into unnecessarily increasing playout delay for everyone. The PlaySyncServer SHOULD implement rate limiting on reports and outlier detection to identify and filter anomalous feedback.

IANA Considerations This document requests the following IANA registrations:

TBD1: PlayoutDelay Object Extension Type
TBD2: CaptureTimestamp Object Extension Type
TBD3: HopTime Object Extension Type
TBD6: Sync Support Subscribe Parameter Type