Skip to content

Skip RTX scene partitioning without spectator-view support - #25

Closed
mataylor-nvidia wants to merge 2 commits into
pr-7053-basefrom
mataylor/scene-partition-spectator-fallback
Closed

Skip RTX scene partitioning without spectator-view support#25
mataylor-nvidia wants to merge 2 commits into
pr-7053-basefrom
mataylor/scene-partition-spectator-fallback

Conversation

@mataylor-nvidia

Copy link
Copy Markdown
Owner

Description

Follow-up to isaac-sim#7053. On that branch the Kit viewport renders completely black for any
camera task when running against the Isaac Sim version this repo's pyproject.toml pins
(isaacsim==6.0.1.0).

_apply_viewport_camera_scene_partition removes the viewport camera's omni:scenePartition
token whenever /rtx/scenePartitioning/showAllPartitionsByDefault reads back as true, relying
on RTX to then show the camera every partition. Carb settings are schemaless, so the experience
files create that key on every runtime and reading it back cannot distinguish a renderer that
implements the spectator view from one that ignores it. The check passes on Isaac Sim 6.0, the
token is stripped, and nothing implements the fallback — so the camera matches no partition and
the viewport renders nothing.

Reproduced with 4 environments and the Isaac RTX renderer:

HEADLESS=1 python scripts/reinforcement_learning/train.py --rl_library rsl_rl \
  --task Isaac-Cartpole-Camera --num_envs 4 --video --video_length 60 \
  --max_iterations 2 --viz kit \
  env.viewer.eye=[38.0,38.0,26.0] env.viewer.lookat=[0.0,0.0,0.0]
Build Non-black pixels Envs visible
develop, partitioning on 736 1 / 4
isaac-sim#7053, partitioning on 0 0 / 4
isaac-sim#7053, partitioning off (control) 3962 4 / 4
isaac-sim#7053 + this change 3962 4 / 4

Confirmed against the installed runtime: /renderer/scenePartitioning/enabled appears in
rtx.scenedb.plugin.dll and carb.scenerenderer-rtx.plugin.dll, while no showAllPartitions
string exists in any RTX binary shipped with Isaac Sim 6.0.0-rc.59 (Kit 6312fa25).

Changes

  • Add show_all_partitions_supported() resolving the capability from the Isaac Sim version
    (SHOW_ALL_PARTITIONS_MIN_ISAAC_SIM_VERSION = 6.1).
  • IsaacRtxRenderer.prepare_stage now skips scene-partition authoring when the spectator view
    is requested but unsupported, and logs an actionable warning, rather than half-applying the
    feature. The existing visualizer guard then leaves the viewport camera untouched.
  • Document the Isaac Sim requirement and the fallback in the renderers guide.

Behavior on Isaac Sim 6.1+ is unchanged. Setting
IsaacRtxRendererCfg.global_settings.show_all_partitions_by_default=False keeps per-environment
isolation on older runtimes, with the viewport bound to a single environment.

Trade-off

On Isaac Sim 6.0 this trades tiled-camera partition isolation for a working viewport. Isolation
does work on 6.0 when the spectator view is not requested, so the alternative is to keep
partitioning and have the visualizer fall back to tagging env_0 — a single-environment viewport
instead of an empty one. Happy to switch if that is preferred.

Note on the Isaac Sim pin

isaac-sim#7053 bumps only the CI container to 6.1.0-alpha.50; pyproject.toml still pins
isaacsim[all,extscache]==6.0.1.0. Until those move together, anyone installing from the wheels
rather than the CI image is on a runtime without the spectator view. This change makes that
configuration degrade to the pre-isaac-sim/IsaacLab#7053 behavior instead of an empty viewport.

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • Documentation update

Screenshots

Before (isaac-sim#7053) After (this change)
0 of 4 environments — viewport fully black 4 of 4 environments

Checklist

  • I have read and understood the contribution guidelines
  • I have run the pre-commit checks with ./isaaclab.sh --format
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • I have added a changelog fragment under source/<pkg>/changelog.d/ for every touched package (do not edit CHANGELOG.rst or bump extension.toml — CI handles that)
  • I have added my name to the CONTRIBUTORS.md or my name already exists there

Testing

  • New test_partitioning_skipped_when_spectator_view_unsupported fails without the gate and
    passes with it; test_partitioning_kept_without_spectator_view_on_unsupported_runtime covers
    the opt-out path.
  • The two existing default-behavior tests now pin the capability so they no longer depend on the
    Isaac Sim build running them.
  • source/isaaclab_physx/test/renderers/ — 36 passed.
  • test_kit_visualizer_scene_partitioning.py + test_renderers.py — 10 passed.

test_partitioning_isolates_rigid_object fails on Isaac Sim 6.0 both with and without this
change (byte-identical pixel matrix) — the per-instance PointInstancer marker tokens added in
isaac-sim#7053 do not isolate on that runtime either. Untouched here.

Per-environment scene partitioning is now on by default, and the Kit viewport
relies on /rtx/scenePartitioning/showAllPartitionsByDefault to see partitioned
geometry. That RTX setting arrived in Isaac Sim 6.1. Carb settings are
schemaless, so the experience files create the key on every runtime and reading
it back cannot tell an implementing renderer from one that ignores it.

On Isaac Sim 6.0 the viewport camera therefore ended up with no partition token
and no spectator fallback, matching no partition and rendering an empty image.

Resolve the capability from the Isaac Sim version and leave the stage
unpartitioned when the spectator view is requested but unsupported, rather than
half-applying the feature. Setting show_all_partitions_by_default=False keeps
partitioning on those runtimes, with the viewport bound to one environment.
Record that /rtx/scenePartitioning/showAllPartitionsByDefault needs Isaac Sim
6.1 or newer, and that Isaac Lab skips scene-partition authoring on older
releases so the Kit viewport keeps showing every environment.
@github-actions github-actions Bot added documentation Improvements or additions to documentation isaac-lab labels Aug 15, 2026
@mataylor-nvidia

Copy link
Copy Markdown
Owner Author

Superseded: keeping show_all_partitions_by_default=True unconditionally, with no version-gated fallback for older Isaac Sim releases.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation isaac-lab

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant