Fix CUDA 719 crash from unpumped Kit marker backend on non-Kit visualizers

[AntoineRichard]

Description

Fixes a crash when using the Newton GL visualizer on a PhysX-backed task with debug-vis markers enabled: env.step() crashes on the very first call with CUDA error: unspecified launch failure (719) from PhysX's GpuArticulationView, preceded by FabricManager::initializePointInstancer mismatched prototypes warnings for the active debug-vis markers.

Root cause

VisualizationMarkers._ensure_backends_initialized() stood up a Kit/USD marker backend whenever sim.is_rendering was True — which is also true for non-Kit visualizers like newton_gl, since that flag is set by any active visualizer, not just Kit. With no real Kit visualizer instance to pump app.update() every frame, the Kit backend's raw USD marker writes (position/prototype-index attrs) were never digested by Fabric, desyncing its point-instancer prototype table. That desync corrupts shared Fabric/GPU-tensor interop state, and the very next PhysX GPU articulation step then crashes with CUDA error 719.

Reproduced on an independent machine/GPU (Linux, RTX 5000 Ada / sm_89) — different architecture than the originally reported RTX 5090 (sm_120) — with the identical crash signature and warning sequence, confirming this is architecture-independent. Confirmed causally: disabling the task's debug-vis markers eliminated the crash entirely.

Fix

_ensure_backends_initialized() now checks the concrete conditions that actually need a Kit-pumped marker backend (native GUI, RTX sensor rendering, XR, offscreen capture) plus the existing Kit-pumping-visualizer check, instead of the broader sim.is_rendering.

Fixes #(issue)

Type of change

• Bug fix (non-breaking change which fixes an issue)

Checklist

• I have read and understood the contribution guidelines
• I have run the pre-commit checks with ./isaaclab.sh --format
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• I have added a changelog fragment under source/<pkg>/changelog.d/ for every touched package
• I have added my name to the CONTRIBUTORS.md or my name already exists there

[greptile-apps]

Greptile Summary

The PR narrows marker backend initialization so non-Kit visualizers no longer accidentally create an unpumped USD backend, while preserving Kit markers for GUI, RTX sensor, XR, offscreen, and Kit-pumping modes.

• Replaces the broad SimulationContext.is_rendering check with concrete Kit-backend requirements.
• Adds backend-selection regression coverage across rendering modes and supported visualizers.
• Documents the Fabric point-instancer desynchronization and CUDA crash fix in the changelog.

Confidence Score: 5/5

The PR appears safe to merge, with no concrete blocking or independently actionable non-blocking issues identified.

The new predicates preserve the established Kit-rendering modes while excluding non-Kit-pumping visualizers, and the added tests directly cover each backend-selection branch introduced by the change.

Important Files Changed

Filename	Overview
source/isaaclab/isaaclab/markers/visualization_markers.py	Replaces broad rendering-state backend selection with explicit Kit-pumping/rendering predicates and retains independent Newton backend initialization.
source/isaaclab/test/markers/test_visualization_markers.py	Expands integration coverage for GUI, RTX, XR, offscreen, Kit, Newton, Rerun, and Viser backend-selection cases.
source/isaaclab/changelog.d/fix-newton-gl-marker-cuda-crash.rst	Records the unpumped Kit marker backend failure mechanism and the corrected selection behavior.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Marker visualization requested] --> B{SimulationContext exists?}
    B -- No --> K[Initialize Kit backend]
    B -- Yes --> C{GUI, RTX sensors, XR, offscreen, or Kit-pumping marker visualizer?}
    C -- Yes --> K
    C -- No --> D[Do not initialize Kit backend]
    K --> E{Enabled non-Kit-pumping marker visualizer?}
    D --> E
    E -- Yes --> N[Initialize Newton marker backend]
    E -- No --> F[Use selected backends]
    N --> F

Loading

_{Reviews (1): Last reviewed commit: "Fix CUDA 719 crash from unpumped Kit mar..." | Re-trigger Greptile}

[isaaclab-review-bot]

Isaac Lab Review Bot

The backend-selection change was traced through GUI, RTX sensor, XR, offscreen capture, Kit-pumping visualizer, and non-Kit visualizer paths. The patch directly addresses the Newton GL regression and adds focused selection coverage plus the required changelog fragment.

• Design and architecture: Separating generic rendering activity from conditions that actually pump Kit is consistent with the backend architecture. Direct setting-key lookups create some coupling to SimulationContext configuration, but the candidate review does not establish an existing accessor or a concrete compatibility failure requiring action before merge.
• API: No public marker API, configuration type, or export changes. The behavioral change is limited to backend creation, preserving Kit creation for the explicitly covered rendering modes while avoiding it for a Newton-only visualizer.
• Implementation: The implementation and parametrized test consistently model the intended selection matrix. The claim that has_gui is necessarily a method is not supported by the supplied patch or current-file context, so the alleged always-truthy bound-method failure is not established.

No blocking issues. No inline issue met the actionable-evidence threshold; the assessment above records the review feedback.

Automated review; human maintainers own approval decisions.

[matthewtrepte]

approved, thanks!