Add --video recording for all renderer backends (Newton+OVRTX, Newton+Warp)

[bdilinila]

Description

Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context.
List any dependencies that are required for this change.

Fixes # (issue)

Type of change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (existing functionality will not work without user modification)
• Documentation update

Screenshots

Please attach before and after screenshots of the change if applicable.

Checklist

• I have read and understood the contribution guidelines
• I have run the pre-commit checks with ./isaaclab.sh --format
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• I have updated the changelog and the corresponding version in the extension's config/extension.toml file
• I have added my name to the CONTRIBUTORS.md or my name already exists there

[ooctipus]

Does this return from all environments? usually when rendering training video, we only care about 1 environment, or, a handful environment that is small enough for human ease to parse.

[bdilinila]

Currently, yes. Per your comment and the meeting notes I will add an option to specify number of environments to view.

[greptile-apps]

Greptile Summary

This PR extends the --video flag across all RL training/playing scripts and simulation backends by introducing a new VideoRecorder / VideoRecorderCfg pair that routes frame capture to either a Newton GL viewer (Newton+Warp backends), the Kit /OmniverseKit_Persp camera via omni.replicator, or a TiledCamera sensor (tiled mode). All env config classes (DirectRLEnvCfg, DirectMARLEnvCfg, ManagerBasedEnvCfg) gain a video_recorder field with a sensible default, and every script's --video argument is upgraded from a boolean to an optional string mode (perspective / tiled).

Key issues found:

• render() crashes with AttributeError when video_recorder=None: All three env render() implementations call self.video_recorder.render_rgb_array() without a null guard, despite the docs stating cfg.video_recorder = None is a valid way to disable recording.
• _find_video_camera() permanently caches None: The hasattr-based cache unconditionally marks the result as settled on the first call. If called before the fallback TiledCamera is initialized, tiled-mode recording fails for the entire run.
• render_rgb_array() returns None on errors: Both perspective render paths return None on exception; gym.wrappers.RecordVideo will throw an opaque TypeError when given None instead of an array.
• rlinf/play.py help text and const are contradictory: Help says "'tiled' (default)" but const="perspective" makes perspective the actual no-arg default; the mode value is also never forwarded to the env config.

Confidence Score: 2/5

• The PR introduces several logic bugs that will cause crashes in documented use-cases — needs fixes before merging.
• Three separate render() implementations have an unguarded None dereference that crashes when cfg.video_recorder=None is used as documented. The permanent None caching bug in _find_video_camera() can silently break all tiled recording. The None-returning render helpers will produce opaque errors in the RecordVideo gym wrapper. Together these represent real correctness issues for a newly introduced feature.
• source/isaaclab/isaaclab/envs/utils/video_recorder.py, source/isaaclab/isaaclab/envs/direct_rl_env.py, source/isaaclab/isaaclab/envs/direct_marl_env.py, source/isaaclab/isaaclab/envs/manager_based_rl_env.py

Important Files Changed

Filename	Overview
source/isaaclab/isaaclab/envs/utils/video_recorder.py	New VideoRecorder class with two bugs: (1) _find_video_camera() permanently caches None on the very first call if the fallback TiledCamera isn't initialized yet; (2) render_rgb_array() in perspective mode silently returns None on error — the gym RecordVideo wrapper may crash when given None instead of a valid array.
source/isaaclab/isaaclab/envs/utils/video_recorder_cfg.py	New VideoRecorderCfg dataclass; well-structured with clear docstrings. No validation of video_mode against allowed values ("perspective"/"tiled").
source/isaaclab/isaaclab/envs/direct_rl_env.py	Delegates render to VideoRecorder, but if cfg.video_recorder=None (as documented: "Set to None to disable") and render_mode="rgb_array" is active, self.video_recorder.render_rgb_array() crashes with AttributeError.
source/isaaclab/isaaclab/envs/direct_marl_env.py	Same pattern and same null-safety bug as direct_rl_env.py — self.video_recorder.render_rgb_array() is called without a None check.
source/isaaclab/isaaclab/envs/manager_based_rl_env.py	Same null-safety issue in render(). Also correctly forwards render_mode/kit_cam_prim_path/kit_resolution to VideoRecorderCfg before calling super().init().
source/isaaclab/isaaclab/envs/manager_based_env.py	Creates VideoRecorder but never uses it directly (no render() method). The recorder is used by ManagerBasedRLEnv through the shared self.video_recorder attribute. Structurally fine.
scripts/reinforcement_learning/rlinf/play.py	The --video flag is parsed but its mode value is never forwarded to env config (only truthiness matters). The help text says 'tiled' is the default mode, but const="perspective" makes perspective the actual default — documentation inconsistency.
scripts/reinforcement_learning/rsl_rl/train.py	Correctly updates argparse for --video and forwards video_mode to VideoRecorderCfg before env creation.
scripts/benchmarks/benchmark_non_rl.py	Consistent --video argparse change and video_mode forwarding — no issues.

Sequence Diagram

sequenceDiagram
    participant Script as Train/Play Script
    participant Gym as gym.make()
    participant Env as DirectRLEnv / ManagerBasedRLEnv
    participant VRCfg as VideoRecorderCfg
    participant VR as VideoRecorder
    participant RV as RecordVideo Wrapper

    Script->>VRCfg: set video_mode, render_mode, kit_cam_prim_path
    Script->>Gym: gym.make(task, render_mode="rgb_array")
    Gym->>Env: __init__(cfg, render_mode)
    Env->>VRCfg: forward render_mode / kit settings
    Env->>VR: class_type(cfg, scene)
    Note over VR: If tiled+fallback: spawn VideoCamera prims<br/>Enable pyglet headless
    Env->>Env: sim.reset() → physics init
    Gym-->>Script: env (unwrapped)
    Script->>RV: wrap env with RecordVideo
    loop Each step
        Script->>RV: step(action)
        RV->>Env: step(action)
        RV->>Env: render()
        Env->>VR: render_rgb_array()
        alt perspective mode
            VR->>VR: _try_init_gl_viewer() [once]
            alt Newton backend
                VR-->>Env: Newton GL frame (ndarray)
            else Kit backend
                VR-->>Env: Kit /OmniverseKit_Persp frame (ndarray)
            end
        else tiled mode
            VR->>VR: _find_video_camera() [cached after 1st call]
            VR->>VR: _render_tiled_camera_rgb_array()
            VR-->>Env: (G*H × G*W × 3) tile grid
        end
        Env-->>RV: ndarray frame
        RV->>RV: write frame to video
    end

Loading

_{Last reviewed commit: ed55c9e}