feat(render): render an arbitrary frame range, with the segment's own audio - #169
Merged
Conversation
… audio Closes the Critical gap that gates the whole distributed/serverless axis: until now a scenario could only be rendered whole, or one frame at a time. The internals were already close. `build_frame_tasks` produces the complete ordered task list, `render_frame_task` renders one in isolation, and `--frame` already indexed into it. What was missing sat downstream. - `--frames a-b` (inclusive, 0-indexed). Malformed input fails at the clap layer; out-of-range fails against the scenario's real total, naming both the range and that total. Mutually exclusive with `--frame` and `--watch`. `png-seq`/`gif`/`raw` refuse explicitly rather than silently ignoring the range — those encoders live outside this change's file scope. - Audio was the real hazard, and the reason a naive `--frames` would have been worse than none. `mix_audio_tracks` had no offset parameter: every segment would have received the audio from the top of the scenario, so the video would have cut cleanly while the sound was wrong, with nothing to signal it. `mix_audio_tracks_segment` translates each sample into absolute scenario time and reprojects it into the segment's buffer. The scenario's total duration stays a separate parameter from the segment's, so fades and unbounded track ends remain anchored to the whole scenario rather than to a segment edge. Measured on the red phase: without the offset, 99.9% of a second segment's samples were wrong. - `rustmotion concat` joins segments through ffmpeg's concat demuxer with `-c copy`. Raw Annex-B bitstream joining was rejected deliberately: it requires every segment boundary to land on an independently decodable keyframe, which holds on the native openh264 path but *not* on the default ffmpeg path, where libx264 manages its own GOP structure. Joining bitstreams there would produce a silently corrupt stream at some cuts. Verified end to end, not just in unit tests: the same scenario rendered whole and as two concatenated segments gives 90 frames and 3.000000s either way. The segment mixer is byte-for-byte identical to the whole-scenario mix when its segments are concatenated. Also fixes a pre-existing race found while testing this: the ffmpeg audio scratch directory was named by PID alone, so concurrent encodes in one process shared it and one call's cleanup deleted a directory another was still writing to. It now carries an atomic counter as well.
This was referenced Aug 11, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes the Critical gap "frame-range rendering and concatenable segments" from the re-scored Remotion differential. It is the single lock between today's state and the whole distributed/serverless axis, which is scored XL and which several High-value gaps depend on strictly.
The internals were already close
build_frame_tasksproduces the complete ordered task list,render_frame_taskrenders one in isolation with no cross-frame state, and--framealready indexed into it. Rendering an arbitrary subset was, in principle, available. What was missing sat downstream.What this adds
--frames a-b— inclusive, 0-indexed. Malformed input fails at the clap layer; out-of-range fails against the scenario's real total, naming both:Mutually exclusive with
--frameand--watch.png-seq/gif/rawrefuse explicitly rather than silently ignoring the range.Audio offset — the real hazard. This is why a naive
--frameswould have been worse than none at all.mix_audio_trackshad no offset parameter: every segment would have received the audio from the top of the scenario. The video would have cut cleanly while the sound was wrong, with nothing to signal it.Red phase, measured: without the offset, 99.9% of a second segment's samples were wrong (229176 of 229320 bytes).
mix_audio_tracks_segmenttranslates each sample into absolute scenario time and reprojects it into the segment's buffer. The scenario's total duration stays a separate parameter from the segment's, so fades and unbounded track ends stay anchored to the whole scenario rather than to a segment edge.rustmotion concatjoins segments through ffmpeg's concat demuxer with-c copy.Why not join the bitstreams
Raw Annex-B joining requires every segment boundary to land on an independently decodable keyframe. That holds on the native openh264 path — which forces an intra frame on every frame, a pre-existing property unrelated to this change — but not on the default path, where ffmpeg is auto-detected and libx264/libx265 manage their own GOP structure with no per-frame control. Joining bitstreams there would produce a silently corrupt or undecodable stream at some cuts.
The concat demuxer sidesteps it: it trusts each segment's container and touches no pixels. Assumed implication:
rustmotion concatdepends hard on ffmpeg being onPATH, with no software fallback — consistent with the default render path already depending on it.Verification
The end-to-end check, reproduced with the real binary rather than only in tests:
And the segment mixer is byte-for-byte identical to the whole-scenario mix once its segments are concatenated.
cargo test --workspace: 24 targets, 999 tests, 0 failurescargo fmt --all --checkandcargo clippy --workspace --all-targets -- -D warnings: cleanmainFixed along the way
A pre-existing race in
encode_with_ffmpeg_hw: the audio scratch directory was named by PID alone, so concurrent encodes within one process shared it and one call's cleanup deleted a directory another was still writing to. Surfaced by adding four audio renders to the test suite; it now carries an atomic counter alongside the PID.What distributed rendering still needs on top of this
a-bfrom the total (rustmotion inforeports it).mix_audio_tracks_segmentre-decodes each source track in full per segment. Correct, but N segments means N full decodes.concatlets ffmpeg fail on codec/resolution mismatch rather than diagnosing it first.