Fix first-frame RTX synchronization after tensor pose writes

[StriverAlex]

Description

PhysX tensor pose writers can change simulation state without advancing the public physics step. RTX/Fabric may therefore render the previous transform in the first frame after a write.

This change keeps the normal physics and clean rendering paths lightweight. It introduces a step-stamped tensor-pose write barrier that is consumed only at an actual Kit/RTX frame boundary.

Fixes #6394.

Root cause and design

• Native tensor pose writes bypass the propagation needed for the next RTX frame.
• PhysicsManager.before_kit_app_update() -> bool provides a backend-neutral frame-boundary hook. The default implementation is a no-op.
• PhysxManager records a private pending token only after a tensor pose mutation succeeds.
• At the next Kit update, the PhysX backend drops a stale token if a real physics step already consumed the change; otherwise it runs one native update_simulation() and consumes the token.
• Multiple writes in the same public physics step coalesce into one synchronization.
• A failed native synchronization is requeued so a transient failure does not silently lose the pending write.
• The boolean result tells the frame owner whether one final Fabric forward() is required before app.update().
• Kit simulation playback state is disabled only around this synchronization and restored with try/finally.

The resulting invariants are:

• SimulationContext.forward() does not call native update_simulation().
• A clean Kit update performs no additional Fabric forward.
• A dirty Kit update performs one native synchronization followed by one final Fabric forward.
• The renderer utility retains its existing single unconditional Fabric forward on both clean and dirty frames.
• Normal physics steps and clean frames do not inspect or clear generic dirty state on their hot paths.

Tensor writer coverage

The barrier is notified by all six native tensor pose implementations:

• rigid object root-link pose
• rigid object root center-of-mass pose
• rigid object collection root-link pose
• rigid object collection root center-of-mass pose
• articulation root-link pose
• articulation root center-of-mass pose

Root-pose, indexed, and mask-based public APIs are covered through their existing delegation to these implementations.

Review concerns addressed

The previously reviewed performance regression came from placing the full native PhysX update in forward(), which taxed every call. This revision removes that behavior entirely. Synchronization is local to the PhysX backend, causally tied to a successful tensor pose write, keyed to the public physics-step epoch, and consumed only by a frame-producing Kit update.

This is deliberately not a general-purpose manager dirty flag: the state models one backend-specific synchronization obligation and remains private to PhysxManager.

Validation

Rebased onto current develop at 3b736feb04a98f708bb69ed1f69eb0f8e81b2adf.

• 25 writer, renderer, and Kit visualizer regression tests passed.
• 2 native PhysX callback and coalescing integration tests passed.
• 4 CUDA articulation and rigid-object-collection writer cases passed.
• Real Kit RTX first-frame pixel regression passed on the candidate.
• The same regression mounted over clean develop failed as expected: first-frame red fraction 0.000000, stable red fraction 0.010468.
• Ruff lint passed for all 12 changed Python files; all 12 were already correctly formatted.
• Standards review: 0 findings.
• Specification review: 0 findings.
• Full repository suite was not run.

The local Git LFS pre-push hook was skipped because its SSH authentication helper hung in the validation environment. The 16 PR files are ordinary text files and contain no LFS pointers.

Performance regression check

The same Isaac Sim container command was run in develop-candidate-develop order with Isaac-Cartpole-Direct, 4096 environments, 500 measured steps, 100 warmup steps, and the Kit visualizer.

Run	FPS	Iteration time
develop before	169,223.41	24.2047 ms
candidate	159,664.78	25.6537 ms
develop after	150,679.47	27.1835 ms

The two develop endpoints differed by 11.59%, showing monotonic host drift during the run. The candidate landed essentially at the midpoint: -0.18% FPS and -0.16% iteration time versus the arithmetic mean of the bracketing develop runs. This does not reproduce a candidate-specific +11% regression. An earlier, more stable ABA run likewise placed the candidate at +1.13% FPS versus the develop mean with only 1.30% baseline spread.

Type of change

• Bug fix with targeted regression coverage

Checklist

• Rebased onto current develop.
• Preserved clean forward() and normal physics-step behavior.
• Covered all PhysX tensor root-pose writer implementations.
• Added changelog fragments.
• Added real first-frame RTX regression coverage.
• Reproduced the failure on clean develop.
• Ran targeted functional, callback, CUDA, lint, and performance checks.
• Added the contributor entry.
• Full repository test suite run.

[greptile-apps]

Greptile Summary

This PR fixes a rendering stale-pose bug where rigid-body positions written through the PhysX tensor API were not visible to RTX cameras until after the next public physics step. The fix adds an omni.physx.get_physx_interface().update_simulation() call inside PhysxManager.forward() to flush tensor state before the Fabric update, while a new _suppress_step_callbacks class flag prevents registered PRE/POST physics-step callbacks from firing during this internal synchronization.

• Core fix (physx_manager.py): forward() now calls update_simulation() wrapped in a save/restore guard for _suppress_step_callbacks before the existing Fabric update path; the callback guarded() wrapper gains a not cls._suppress_step_callbacks check.
• Regression tests: an RTX camera test verifies the tensor-written pose is visible before any public physics step, and a SimulationContext unit test verifies forward() leaves the step count and callback counts unchanged.

Confidence Score: 4/5

Safe to merge for the common case; the suppression flag uses a correct save/restore pattern and the fix was validated against both the current develop branch and v3.0.0-beta2.

The new update_simulation() call runs unconditionally without checking _view_created, so forward() called before reset() would invoke update_simulation() before start_simulation() has been called. The callback suppression also only covers callbacks registered through PhysxManager, not direct PhysX event subscribers.

source/isaaclab_physx/isaaclab_physx/physics/physx_manager.py — the unconditional update_simulation() placement and the documented scope of _suppress_step_callbacks.

Important Files Changed

Filename	Overview
source/isaaclab_physx/isaaclab_physx/physics/physx_manager.py	Core fix: adds update_simulation() call in forward() to flush tensor-written poses before Fabric update; suppresses PRE/POST step callbacks via a new class-level flag.
source/isaaclab/test/sensors/test_first_frame_textured_rendering.py	Adds RTX regression test asserting the tensor-written pose red pixel fraction in the first camera frame is at least 50% of a post-step reference frame.
source/isaaclab/test/sim/test_simulation_context.py	Adds unit test verifying forward() leaves physics step count and PRE/POST callback counts unchanged, while step() increments each exactly once.
source/isaaclab_physx/changelog.d/striveralex-fix-physx-forward-sync.rst	New changelog fragment describing the fix.
CONTRIBUTORS.md	Adds StriverAlex to the contributors list in alphabetical order.

Sequence Diagram

sequenceDiagram
    participant User
    participant SimCtx as SimulationContext
    participant Mgr as PhysxManager
    participant PhysX as IPhysX
    participant Fabric

    User->>SimCtx: reset()
    SimCtx->>Mgr: warmup + start_simulation()
    Mgr-->>SimCtx: "_view_created=True"

    User->>User: write_root_pose_to_sim_index()
    note over User: tensor pose written

    User->>SimCtx: forward()
    SimCtx->>Mgr: forward()
    Mgr->>Mgr: "_suppress_step_callbacks=True"
    Mgr->>PhysX: update_simulation(dt, 0.0)
    note over PhysX: PRE/POST suppressed
    PhysX-->>Mgr: poses flushed
    Mgr->>Mgr: "_suppress_step_callbacks=False"
    Mgr->>Fabric: _update_fabric(0.0, 0.0)
    Fabric-->>SimCtx: RTX sees updated pose

    User->>SimCtx: step()
    SimCtx->>Mgr: simulate() + fetch_results()
    note over Mgr: callbacks fire, step count incremented

Loading

Comments Outside Diff (1)

1. source/isaaclab_physx/isaaclab_physx/physics/physx_manager.py, line 345-357 (link)

   update_simulation called before PhysX is started

   omni.physx.get_physx_interface().update_simulation() is invoked unconditionally on every call to forward(), including calls made before reset() triggers _warmup_and_create_views(). That warmup sequence calls force_load_physics_from_usd() → start_simulation() → update_simulation() in order; skipping start_simulation() and going straight to update_simulation() (as the new code does when forward() is called pre-reset) may behave unexpectedly or silently fail depending on the PhysX version. The pre-existing _view_created guard in the callback wrapper already knows about this timing requirement — adding the same guard or checking cls._view_created before the update_simulation() call would keep the two paths consistent.

_{Reviews (1): Last reviewed commit: "Ensure tensor pose resets reach the firs..." | Re-trigger Greptile}

[greptile-apps]

Suppression scope is limited to PhysxManager-registered callbacks

The _suppress_step_callbacks flag only silences callbacks that went through PhysxManager._subscribe_isaac(). Any code that registers step-event listeners directly via omni.physx.get_physx_interface().subscribe_physics_on_step_events() will still fire during forward(). Worth documenting as a known limitation — third-party or user code that bypasses the manager will observe a step event during each forward() call.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[AntoineRichard]

AI review — Request changes\n\nI am requesting changes because this moves a scene-wide PhysX update into the general path, including calls that have no pending rigid-body tensor pose writes.\n\n calls before visualizer updates, and returns . Consequently, Kit-rendered workloads execute on every rendered frame, not only for the reset/first-camera-frame case addressed by this PR.\n\nI reproduced the proposed behavior locally on an RTX 5090 with 4,096 environments, including the PR's callback suppression:\n\n| Measurement | Current path | Proposed path | Difference |\n|---|---:|---:|---:|\n| | 0.00247 ms | 2.2366 ms | +2.234 ms/call |\n| Rendered environment step | 21.93 ms | 24.35 ms | +11.0% |\n\nThere is also a callback-contract gap. only suppresses callbacks wrapped by . A callback registered directly with still fires. In a local probe, PRE and POST counts remained zero after the current , but each incremented once with the proposed path. This can introduce observable behavior and additional extension callback work on every render.\n\nCould you provide more information about the workflow that needs this synchronization?\n\n- Is the stale RTX pose limited to reset-time writes, or can tensor pose writes occur before arbitrary camera frames?\n- Which tensor write APIs need to become visible immediately?\n- How often does the affected workflow call , and at what typical environment counts and render intervals?\n- Does the workflow use Kit visualization, camera-only rendering, or both?\n- Were any narrower PhysX flush/update APIs evaluated, and what specifically remained stale with them?\n\nIf the full call is necessary, I think it should be behind a dirty flag or a dedicated render synchronization path that runs only after an affected tensor pose write and immediately before an RTX frame consumes it. The existing lightweight articulation/Fabric path should remain cheap when no such synchronization is required.\n\nPlease also add coverage for direct native PhysX step subscribers if is intended to preserve callback-free semantics.

[AntoineRichard]

AI review — Request changes

I am requesting changes because this moves a scene-wide PhysX update into the general PhysxManager.forward() path, including calls that have no pending rigid-body tensor pose writes.

SimulationContext.update_visualizers() calls forward() before visualizer updates, and KitVisualizer.requires_forward_before_step() returns True. Consequently, Kit-rendered workloads execute update_simulation() on every rendered frame, not only for the reset/first-camera-frame case addressed by this PR.

I reproduced the proposed behavior locally on an RTX 5090 with 4,096 Isaac-Cartpole-Direct environments, including the PR's callback suppression:

Measurement	Current path	Proposed path	Difference
forward()	0.00247 ms	2.2366 ms	+2.234 ms/call
Rendered environment step	21.93 ms	24.35 ms	+11.0%

There is also a callback-contract gap. _suppress_step_callbacks only suppresses callbacks wrapped by PhysxManager._subscribe_isaac(). A callback registered directly with omni.physx.get_physx_interface().subscribe_physics_on_step_events() still fires. In a local probe, PRE and POST counts remained zero after the current forward(), but each incremented once with the proposed update_simulation() path. This can introduce observable behavior and additional extension callback work on every render.

Could you provide more information about the workflow that needs this synchronization?

• Is the stale RTX pose limited to reset-time writes, or can tensor pose writes occur before arbitrary camera frames?
• Which tensor write APIs need to become visible immediately?
• How often does the affected workflow call sim.forward(), and at what typical environment counts and render intervals?
• Does the workflow use Kit visualization, camera-only rendering, or both?
• Were any narrower PhysX flush/update APIs evaluated, and what specifically remained stale with them?

If the full update_simulation() call is necessary, I think it should be behind a dirty flag or a dedicated render synchronization path that runs only after an affected tensor pose write and immediately before an RTX frame consumes it. The existing lightweight articulation/Fabric forward() path should remain cheap when no such synchronization is required.

Please also add coverage for direct native PhysX step subscribers if SimulationContext.forward() is intended to preserve callback-free semantics.

Superseded by the correctly formatted AI review submitted immediately afterward.

[StriverAlex]

Thank you @AntoineRichard for the detailed review, and especially for measuring the rendering-path regression at 4096 environments. Your feedback made the performance and callback constraints much clearer.

I reworked the fix in commit 94c93f52d to follow your suggested direction:

• PhysxManager.forward() is cheap again and no longer calls update_simulation().
• Tensor pose writes mark a render-sync dirty flag at the actual RigidObject write sites.
• The pending synchronization is consumed once in pre_render(), immediately before RTX rendering.
• Clean renders and forward() do not trigger native PhysX step callbacks.
• The tests now subscribe directly to the native PhysX step events and cover dirty render, clean render, and regular simulation-step behavior.

I also evaluated the narrower APIs before retaining update_simulation() behind the dirty gate: both flush_changes() and update_transformations() left the first rendered frame stale in the regression test. The targeted simulation and RTX tests pass with the revised implementation.

Thanks again for the concrete guidance and performance data. They helped narrow the change to the render boundary and avoid adding cost to the normal forward path.

[StriverAlex]

Restored as #7138 after the original StriverAlex fork was deleted and recreated, which left this PR without a recoverable head repository. The replacement preserves the reviewed implementation, rebases it onto current develop, and includes fresh RTX, CUDA, callback, Ruff, and PhysX ABA performance validation.