Fixes Kit camera rendering when running headless#5015
Fixes Kit camera rendering when running headless#5015peterd-NV wants to merge 7 commits intoisaac-sim:developfrom
Conversation
Greptile SummaryThis PR fixes two related bugs that caused stale camera observations when running headless with cameras:
Key changes:
Confidence Score: 4/5
Important Files Changed
Sequence DiagramsequenceDiagram
participant User
participant AppLauncher
participant SimulationContext
participant RTXUtils as isaac_rtx_renderer_utils
participant KitViz as KitVisualizer
participant OmniApp as omni.kit.app
User->>AppLauncher: launch(headless=True, viz=kit)
AppLauncher->>AppLauncher: _resolve_visualizer_settings()<br/>cli_visualizer_types=[kit]
AppLauncher->>AppLauncher: _resolve_headless_settings()<br/>(no longer clears visualizer list)
AppLauncher->>SimulationContext: create with KitVisualizer active
Note over SimulationContext,KitViz: Episode runs...
User->>SimulationContext: reset()
SimulationContext->>KitViz: viz.reset(soft)
SimulationContext->>RTXUtils: invalidate_render_update()<br/>_last_render_update_key = (0,-1)
SimulationContext->>SimulationContext: physics_manager.play()
User->>SimulationContext: step() → render()
SimulationContext->>KitViz: update(dt)
KitViz->>OmniApp: app.update() [pumps RTX]
KitViz-->>RTXUtils: pumps_app_update() = True → dedup stamp updated
User->>Camera: get_data()
Camera->>RTXUtils: ensure_isaac_rtx_render_update()
RTXUtils->>RTXUtils: key == _last_render_update_key?<br/>True → skip (already pumped by Kit)
RTXUtils-->>Camera: fresh frame returned ✓
Last reviewed commit: f1eb1eb |
| if self._cli_visualizer_explicit: | ||
| logger.warning( | ||
| "Both '--headless' and '--visualizer/--viz' were provided. " | ||
| "Deprecated '--headless' takes precedence and disables all visualizers." | ||
| "Visualizers are still enabled so viewport/camera pipeline runs correctly in headless." | ||
| ) |
There was a problem hiding this comment.
Misleading warning fires for --headless --viz none
The second warning says "Visualizers are still enabled so viewport/camera pipeline runs correctly in headless", but this fires whenever self._cli_visualizer_explicit is True — including when the user passes --headless --viz none, which sets self._cli_visualizer_disable_all = True and does disable all visualizers. The guard should exclude the "disable all" case so the message only fires when visualizers are actually being kept active.
| if self._cli_visualizer_explicit: | |
| logger.warning( | |
| "Both '--headless' and '--visualizer/--viz' were provided. " | |
| "Deprecated '--headless' takes precedence and disables all visualizers." | |
| "Visualizers are still enabled so viewport/camera pipeline runs correctly in headless." | |
| ) | |
| if self._cli_visualizer_explicit and not self._cli_visualizer_disable_all: | |
| logger.warning( | |
| "Both '--headless' and '--visualizer/--viz' were provided. " | |
| "Visualizers are still enabled so viewport/camera pipeline runs correctly in headless." | |
| ) |
Description
Previously to get correct camera renderings when running headless the Kit visualizer needed to be specified together with headless. This runs the script without the GUI but still initializes the Kit visualizer.
The Kit visualizer pumps the render update: https://github.com/isaac-sim/IsaacLab/blob/develop/source/isaaclab_visualizers/isaaclab_visualizers/kit/kit_visualizer.py#L173
With PR 4948 the use of headless now overwrites any visualizers which results in incorrect camera observations (particularly after a reset) when running workflows that have cameras headlessly.
Fixes # (issue)
This PR fixes the camera rendering problem in headless mode by pumping the render as Kit would do.
Type of change
Checklist
pre-commitchecks with./isaaclab.sh --formatconfig/extension.tomlfileCONTRIBUTORS.mdor my name already exists there