Add csv output for latency benchmarks to local_video example - #1320
Add csv output for latency benchmarks to local_video example#1320chenosaurus wants to merge 10 commits into
Conversation
chenosaurus
commented
Aug 6, 2026
chenosaurus
commented
- add logging output of the video pipeline timing metrics to csv
- add python script to generate report from logged csv
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| if updated_sample.is_complete() { | ||
| self.latest_complete_sample = Some(updated_sample); | ||
| if let Some(frame_log) = self.frame_log.as_mut() { | ||
| if let Err(error) = frame_log.record(updated_sample) { | ||
| warn!("Publisher CSV logging disabled after write failure: {error}"); | ||
| self.frame_log = None; | ||
| } | ||
| } |
There was a problem hiding this comment.
🟡 Publisher benchmark log records the same video frame several times when multi-layer publishing is on
Each already-finished frame is written to the log again (frame_log.record(updated_sample) at examples/local_video/src/publisher.rs:741) every time another encoded copy of that same frame is reported, so the log contains repeated rows for one frame.
Impact: With simulcast enabled, the CSV and the generated report show inflated frame counts, zero inter-frame intervals and blank frame-gap values, making the latency benchmark misleading.
Why simulcast produces repeated completions
PacketTrailerTransformer::TransformSend runs once per encoded frame, i.e. once per simulcast layer, and emits EncoderOutput/WebrtcPacketize publish-timing events for each layer (webrtc-sys/src/packet_trailer.cpp:262, webrtc-sys/src/packet_trailer.cpp:289). All layers share the same capture_timestamp_us, so PublisherTimingState::get_or_insert_sample returns the same PublisherTimingSample, which is already is_complete() after the first layer. Every subsequent layer event therefore re-enters the is_complete() branch and calls PublisherCsvLogger::record again with the same frame_id. In record, frame_id.checked_sub(previous) yields 0, so frame_id_gap is empty and packetize_interval_ms is 0.000, while sample_count keeps incrementing.
A guard such as tracking the last logged frame_id (or last logged sensor_exposure_timestamp_us) and skipping repeats would fix this.
Was this helpful? React with 👍 or 👎 to provide feedback.
| Generate a PDF report from the publisher log, subscriber log, or both: | ||
| ``` | ||
| python3 -m pip install reportlab | ||
|
|
||
| python3 examples/local_video/scripts/generate_frame_report.py \ | ||
| --publisher publisher.csv \ | ||
| --subscriber subscriber.csv \ | ||
| --output frame-report.pdf | ||
| ``` |
There was a problem hiding this comment.
🟡 Pull request is missing the required change-documentation entry
The change adds new cross-crate functionality (new subscribe-timing and decode-timestamp entry points at webrtc-sys/src/video_frame.rs:46-49) without adding the change note the repository requires for every pull request, so the release notes and version bumps will omit it.
Impact: Released crates will not document or bump versions for this behavioral change.
Repository rule and current state
AGENTS.md states under "Documenting changes": "Every PR needs a changeset" and "Changeset must list any crates which need to be bumped stemming from the change". git diff --name-only <merge-base> HEAD -- .changeset/ returns nothing for this PR; the only file in /.changeset comes from an earlier commit (0aea78ff). This PR touches libwebrtc, webrtc-sys (new FFI surface decode_start_timestamp_us, decode_finish_timestamp_us, emit_subscribe_timing_at) and changes receive/decode timing semantics, so those crates need to be listed.
Was this helpful? React with 👍 or 👎 to provide feedback.
| if let Some(frame_id) = completed_frame_id { | ||
| info!("Publisher completed --log-end-frame-id {frame_id}; shutting down..."); | ||
| shutdown_on_log_end.store(true, Ordering::Release); | ||
| break; | ||
| } |
There was a problem hiding this comment.
🔴 Publisher window stays open forever after the automatic stop, so the program never exits
The publisher is told to stop (shutdown_on_log_end.store(true) at examples/local_video/src/publisher.rs:1348) without waking the preview window, which now only redraws when a new camera frame arrives, so the window can never notice the stop request and the program keeps running.
Impact: With --display-video --log-csv --log-end-frame-id, the publisher writes the last row but the window stays open and the process must be killed manually; the same can happen on Ctrl-C.
Mechanism: root viewport lost its periodic repaint
This PR removed the unconditional ui.ctx().request_repaint() and ctx.request_repaint_after(viewport_aspect::VIDEO_REPAINT_INTERVAL) from the publisher preview app (examples/local_video/src/video_display.rs:780-796). The root viewport is now repainted only from pack_i420_into_shared (examples/local_video/src/video_display.rs:141-143) when the capture loop delivers a frame.
VideoApp::ui is the only place that observes ctrl_c_received and issues ViewportCommand::Close. Once the shutdown flag is set, run_capture_loop breaks at its top-of-loop check (examples/local_video/src/publisher.rs:1585-1587) and stops producing frames, so no further root repaint is ever requested. The diagnostics window is a deferred viewport and repaints itself independently (examples/local_video/src/video_display.rs:759), so it does not drive the root, and eframe::run_native never returns.
The subscriber handles exactly this case by calling repaint_ctx.request_repaint_of(egui::ViewportId::ROOT) right after setting the shutdown flag (examples/local_video/src/subscriber.rs:2522-2527); the publisher has no equivalent.
Prompt for agents
The publisher preview window no longer repaints periodically (the continuous repaint calls were removed from VideoApp::ui in examples/local_video/src/video_display.rs); it now repaints only when pack_i420_into_shared receives a new frame. VideoApp::ui is the only code that checks ctrl_c_received and closes the window. When the --log-end-frame-id shutdown path sets the flag (examples/local_video/src/publisher.rs, publish timing event task) or when Ctrl-C is pressed, the capture loop stops emitting frames, so no repaint is ever requested and eframe::run_native never returns, leaving the process hung.
Possible approaches: mirror what the subscriber does (examples/local_video/src/subscriber.rs paint callback) by requesting a repaint of egui::ViewportId::ROOT whenever the shutdown flag is set - the egui Context is already stored in SharedYuv via register_repaint_context, so the shutdown path can grab it; alternatively keep a low-frequency request_repaint_after in VideoApp::ui purely so the shutdown flag is polled.
Was this helpful? React with 👍 or 👎 to provide feedback.
No changeset foundThis PR modifies versioned packages but doesn't include a changeset. The following packages require a version bump:
A package must be bumped when its own files change, and whenever a package it depends on is bumped (so downstream consumers get a matching release). Click here to create a changeset for the missing packages The link pre-populates a changeset file with If this change doesn't require a version bump, add the |
1 participant
