Benchmarking Methodology for Route Origin Validation (ROV)

1.1. Requirements Language

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.¶

4.1. Test Topology

+-------------------+    RTR    +----------------------+
|    RTR Emulator   |---------->|          DUT         |
|(RTR Update Source)|           |     (ROV Enabled)    |
+-------------------+           +----------------------+
                                 /\          /\
                                                          |           | Data-plane Traffic
                              BGP |  +-----------------+
+---------------------+           |  |      Tester     |
|BGP Traffic Generator|-----------+  |(Data-plane Load)|
+---------------------+              +-----------------+

The test topology consists of four primary components: the DUT, an RPKI-RTR update source, a BGP traffic generator, and a tester for generating data-plane load. The DUT is a router equipped with ROV capabilities, supporting the RPKI-RTR protocol and applying ROV policies to received BGP routes. The RPKI-RTR update source may be either a real RPKI cache implementation running in isolated mode or a dedicated emulator capable of producing arbitrary VRP sets and update patterns. This RTR source connects directly to the DUT using the RPKI-RTR protocol and provides precisely controlled VRP updates, including serial increments, cache resets, and bursty or delayed update sequences.¶

The BGP traffic generator establishes one or more BGP peering sessions with the DUT and is responsible for delivering a full global routing table, on the order of 800,000 to 1,000,000 prefixes, along with controlled withdrawal or re-announcement events. The generator should be capable of presenting both stable baseline routing conditions and timed ROV-affected prefixes whose validation status will change in response to VRP updates. A tester is connected to the DUT to introduce controlled data-plane load during benchmarking. When present, the tester SHOULD generate stable and deterministic traffic loads so that the impact of forwarding load on ROV processing can be evaluated. When data-plane load is applied, its rate, frame size, and traffic profile MUST be documented in the test report.¶

4.2. DUT Configuration Requirements

The DUT must be configured with ROV enabled on all BGP sessions receiving test routes. The router must establish a stable and fully functional RPKI-RTR session with the RTR emulator. To ensure that performance results are attributable solely to ROV behavior, all non-essential features on the DUT, such as additional routing protocols, unnecessary telemetry mechanisms, and unused services, should be disabled. Logging related to ROV may remain enabled for debugging purposes but must be rate-limited to avoid skewing CPU measurements or affecting test repeatability. All system parameters relevant to routing performance, such as multipath behavior or maximum-prefix limits, must be documented prior to testing.¶

4.3. RTR Data Source Emulation

The RTR emulator must be capable of generating synthetic VRP data sets with user-defined characteristics. This includes the ability to create arbitrary combinations of prefixes and ASNs, overlapping VRPs, conflicting VRPs, and other edge cases relevant to validation logic. The VRP datasets should mimic realistic global distributions where appropriate, but must also support scaling tests where VRP volumes are substantially higher than today's norm. The data source must further support generating controlled bursts of VRP updates, ranging from 100 to 10,000 VRP changes per second, and must allow for both additive updates and withdrawals. These capabilities are essential for evaluating the DUT's scalability and robustness under high churn.¶

4.4. BGP Traffic Generation Requirements

The BGP traffic generator must present the DUT with a stable baseline routing table prior to initiating any benchmark. This ensures that the DUT begins each test run in a known, converged state with predictable CPU and memory utilization. The generator must also provide a set of ROV-affected prefixes whose origin AS can be manipulated in concert with VRP updates from the RTR emulator. These prefixes should span a range of prefix lengths and originate from diverse ASes to reflect realistic routing conditions. The traffic generator must support deterministic convergence triggers, such as the precise injection of BGP updates following a VRP change or the simultaneous application of both BGP and VRP events.¶

4.5. Traffic Profile Parameters

When data-plane traffic is used, the following parameters SHOULD be specified:¶

• Frame size(s) used (e.g., 64, 512, 1518 bytes).¶

• Traffic rate (percentage of line rate or packets per second).¶

• Traffic pattern (constant rate, burst, IMIX).¶

• Source and destination IP address ranges.¶

• Whether traffic matches ROV-affected prefixes.¶

Each frame size and traffic rate combination SHOULD be reported separately.¶

5.1. General Considerations

Before any measurements are taken, the DUT must reach a well-defined steady state in which the RPKI-RTR session is fully established, the VRP set has been completely synchronized, and the BGP control plane has converged. A warm-up period is recommended to eliminate any cold-start effects that could bias measurement results.¶

All sources of measurement noise should be avoided. Features such as logging, real-time telemetry export, or periodic background tasks can interfere with timing-sensitive measurements; therefore, such features should be disabled or rate-limited during benchmarking. CPU clock scaling, thermal throttling, or other variable-performance modes should be minimized if the test setup allows it.¶

5.2. Test Control and Input Conditions

Accurate benchmarking depends on precise control of the input conditions applied to the DUT. All tests should begin from a consistent baseline consisting of:¶

• A predefined VRP set size (e.g., tens of thousands to millions of entries).¶

• A stable and realistic baseline BGP RIB-in (e.g., ~1M global routes).¶

From this baseline, input variables may be modified to stress different aspects of ROV behavior. These variables include the VRP churn rate, ranging from steady incremental updates to high-intensity bursts, and the type of RPKI-RTR updates provided to the DUT, such as incremental updates versus full-table refreshes. Each of these conditions may trigger different processing strategies within the DUT, and therefore must be explicitly controlled and documented.¶

5.3. Metrics and Measurements

Benchmarking ROV behavior requires collecting quantitative performance metrics that reflect how the DUT processes validation information and incorporates it into the BGP decision process. Therefore, this document proposes key performance metrics including ROV update processing latency, ROV validation latency, BGP convergence time, VRP storage size, CPU and memory utilization, and ROV state rebuild time.¶

ROV update processing latency measures the time from receipt of an RTR update (incremental or full) until the DUT has fully updated its internal validation state. This metric captures the efficiency of ROV data structures and algorithms.¶

ROV validation latency measures the time interval between a router's receipt of a BGP UPDATE message that contains a new or changed route, and the completion of the ROV procedure for that route, producing a validation state of Valid, Invalid, or NotFound. This metric isolates the internal validation step, excluding the larger BGP convergence process, and provides insight into the responsiveness of the DUT's validation engine.¶

BGP convergence time with ROV enabled measures how long the DUT takes to converge on BGP prefixes whose validation states change due to VRP updates. This reflects the real operational behavior of ROV as it interacts with the control plane.¶

The VRP storage size inside the DUT should also be recorded to evaluate the scalability of the implementation when operating with large VRP datasets. Alongside this, CPU and memory utilization should be monitored to identify performance limits or resource-intensive operations triggered by ROV.¶

A recovery-related measurement, ROV state rebuild time after RTR session reset, quantifies the time needed for the DUT to re-establish a complete and correct ROV validation state after an RTR session reset or cache outage. This metric reflects robustness and recovery behavior under fault or restart scenarios.¶

Finally, the DUT should be evaluated under high-pressure scenarios by measuring its behavior when processing VRP bursts, such as surges of 100-10,000 VRPs per second. This measurement reveals whether the implementation can sustain abrupt workload increases without dropping updates, stalling, or entering unstable states.¶

6.1. ROV Update Processing Latency

Objective: Measure the latency from the arrival of an RTR PDU until the new VRP information is installed in the DUT's internal ROV tables.¶

The test procedures for ROV update processing latency are listed below:¶

1. Prepare baseline state¶

   • Establish RTR session between DUT and the RTR emulator.¶

   • Preload DUT with a selected baseline VRP size (e.g., 100k VRPs).¶

   • Ensure BGP is fully converged.¶

2. Inject controlled RTR update¶

   • From the emulator, send a single incremental update modifying a known VRP.¶

   • Alternatively, for full-refresh tests, send a full VRP set replacement PDU sequence.¶

3. Timestamp PDU transmission¶

   • Record the exact moment the first update PDU is sent.¶

4. Monitor DUT internal state¶

   • Use device instrumentation (API, CLI, or telemetry) to detect the exact moment the VRP table reflects the update.¶

   • Confirm the VRP entry has been added, removed, or modified as expected.¶

5. Calculate latency¶

   • Latency = (VRP applied timestamp) − (RTR PDU sent timestamp).¶

6. Repeat for multiple VRP table sizes¶

   • E.g., 50k, 100k, 500k, and 1M VRPs.¶

7. Repeat at least 10 times per condition¶

   • Compute mean and standard deviation.¶

6.2. ROV Validation Latency

Objective: Measure how long the DUT takes to apply updated VRPs to the validation states of affected BGP prefixes.¶

The test procedures for ROV validation latency are listed below:¶

1. Establish baseline¶

   • Load BGP full table (e.g., 1M routes).¶

   • Ensure all prefixes have a known baseline validation state.¶

2. Select a controlled prefix set¶

   • Pick a set of prefixes (e.g., 1,000) whose origin AS is tied to specific VRPs.¶

3. Trigger validation update¶

   • Modify VRPs so that these prefixes change validation state (Valid->Invalid or Invalid->Valid).¶

4. Timestamp VRP installation completion¶

   • As measured in Section 6.1.¶

5. Monitor DUT validation table¶

   • Continuously query validation state for selected prefixes.¶

   • Note the timestamp when all prefixes reflect the updated state.¶

6. Compute latency¶

   • Validation Latency = (all validation updated) − (VRP installed).¶

7. Repeat with varying set sizes¶

   • E.g., 10 prefixes, 100 prefixes, 1,000 prefixes.¶

6.3. BGP Convergence with ROV Enabled

Objective: Measure BGP convergence time for routes impacted by ROV state changes, and compare to BGP-only convergence without ROV.¶

The test procedures for BGP convregence with ROV enabled are listed below:¶

1. Prepare baseline¶

   • Establish full-table BGP adjacency.¶

   • Enable ROV on DUT.¶

   • Ensure stable initial convergence.¶

2. Select test prefixes¶

   • Choose prefixes that will transition from Valid to Invalid once VRP updates are applied.¶

3. Trigger VRP state change¶

   • Send VRP modifications via RTR.¶

4. Monitor BGP behavior¶

   • Observe best-path selection changes.¶

   • Timestamp withdrawal or replacement of Invalid prefixes.¶

5. Measure convergence¶

   • Convergence Timer Starts: The convergence timer SHOULD start at the timestamp when the DUT completes installation of the relevant VRP update.¶

   • Convergence Timer Ends: The convergence timer SHOULD end when both the BGP RIB and FIB reach stable state.¶

6. Repeat test with ROV disabled¶

   • Use identical routing changes for baseline comparison.¶

7. Record:¶

   • Time to withdraw Invalid prefixes.¶

   • Time until new best paths stabilize.¶

   • Differences relative to ROV-disabled baseline.¶

6.4. VRP Scalability Tests

Objective: Evaluate DUT performance with varying VRP table sizes.¶

The test procedures for VRP scalability tests are listed below:¶

1. Generate VRP datasets at sizes:¶

   • E.g., 50k, 100k, 500k, 1M.¶

2. Load each dataset into the RTR emulator.¶

3. For each dataset, measure:¶

   • Full-table synchronization time.¶

   • VRP update processing latency (from Section 6.1).¶

   • ROV validation latency (from Section 6.2).¶

   • Memory consumption¶

   • CPU utilization during sync and steady state.¶

4. Record failures¶

   • Session drops¶

   • Timeouts¶

   • Missing VRPs¶

   • ROV process crashes¶

5. Repeat 10 times per size for statistical stability.¶

6.5. VRP Churn and Stress Tests

Objective: Stress-test the DUT under rapid VRP changes to measure stability, performance, and correctness.¶

The test procedures for VRP churn and stress tests are listed below:¶

1. Baseline setup¶

   • Load a stable VRP table (e.g., 500k).¶

   • Establish full BGP table (e.g., 1M prefixes).¶

2. Generate controlled churn patterns¶

   • Rapid add or remove spikes: 100-10,000 VRPs per second.¶

   • Sustained churn: continuous modifications for 5-10 minutes.¶

   • Mixed churn: adds, removes, and changes simultaneously.¶

3. Measure DUT behavior¶

   • VRP update backlog or queueing.¶

   • ROV validation delays.¶

   • CPU spikes.¶

   • BGP convergence degradation.¶

   • Missed or dropped VRP updates.¶

4. Check correctness¶

   • Verify that no stale or inconsistent ROV states remain.¶

5. Record crash, stall, or throttling events.¶

6.6. Resource Utilization

Objective: Measure resource consumption under various ROV workloads.¶

The test procedures for resource utilization are listed below:¶

1. Establish monitoring tools¶

   • CPU sampling (100-500 ms interval).¶

   • Memory usage tracking.¶

   • Hardware counters if available.¶

2. Measure under conditions¶

   • Idle ROV.¶

   • Full VRP sync.¶

   • VRP churn.¶

   • BGP convergence triggered by ROV events.¶

3. Record¶

   • CPU load curves¶

   • Peak memory consumption¶

   • Any evidence of saturation (e.g., 100% CPU, memory exhaustion)¶

4. Identify thresholds¶

   • Points where performance degrades or ROV processing becomes unstable.¶

6.7. RTR Session Behavior Tests

Objective: Evaluate robustness and recovery of DUT under RTR failure and failover scenarios.¶

The test procedures for resource utilization are listed below:¶

1. Session reset test¶

   • Establish normal RTR session.¶

   • Trigger forced session reset from emulator.¶

   • Measure time to reestablish RTR session, ROV state rebuild time, time until validation state becomes consistent again.¶

2. Cache failover test¶

   • Configure DUT with two RTR servers (primary + secondary).¶

   • Terminate primary RTR connection.¶

   • Measure failover time and data consistency after switch.¶

3. Full resynchronization timing¶

   • From emulator, force a full Reset Query sequence.¶

   • Measure full VRP reload time.¶

   • Compare across different VRP scales.¶

4. Incremental update performance¶

   • Send controlled incremental PDUs.¶

   • Measure processing latency and correctness.¶

   • Introduce occasional malformed or unexpected PDUs to test robustness.¶