Skip to content

Report resolved backend for play benchmarks - #7173

Open
AntoineRichard wants to merge 1 commit into
isaac-sim:developfrom
AntoineRichard:fix/play-resolved-backend
Open

Report resolved backend for play benchmarks#7173
AntoineRichard wants to merge 1 commit into
isaac-sim:developfrom
AntoineRichard:fix/play-resolved-backend

Conversation

@AntoineRichard

Copy link
Copy Markdown
Collaborator

Description

Fixed benchmark play adapters to pass the resolved environment configuration when building RunConfig. This makes run.config.physics_backend report the backend that ran, matching training, runtime, and startup behavior.

For isaacsim_physx, play results now report physx; the requested preset remains in run.config.presets.

No additional dependencies are required.

Type of change

  • Bug fix (non-breaking change which fixes an issue)

Screenshots

Not applicable.

Checklist

  • I have read and understood the contribution guidelines
  • I have run the pre-commit checks
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective
  • I have added a changelog fragment under source/<pkg>/changelog.d/ for every touched package
  • My name already exists in CONTRIBUTORS.md

@AntoineRichard
AntoineRichard requested a review from a team August 19, 2026 14:33
@github-actions github-actions Bot added bug Something isn't working isaac-lab Related to Isaac Lab team labels Aug 19, 2026

@isaaclab-review-bot isaaclab-review-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isaac Lab Review Bot

The four benchmark play adapters now pass their resolved env_cfg to capture.run_config_from_presets, aligning reported run.config.physics_backend with the backend actually used while retaining requested tokens in run.config.presets.

  • Design and architecture: Backend identification remains centralized in capture.run_config_from_presets; the rl_games, rsl_rl, sb3, and skrl adapters reuse their existing resolved environment configuration rather than duplicating backend-resolution logic.
  • API: The change uses the existing env_cfg keyword without altering the function signature, exports, or defaults. The intentional metadata behavior change for isaacsim_physx is documented in the isaaclab changelog fragment.
  • Implementation: All four play paths pass env_cfg before constructing benchmark metadata, and the focused capture test verifies that an isaacsim_physx preset with a resolved PhysX configuration reports physics_backend as physx. The requested preset remains available through cfg.presets; the only compatibility tradeoff is for metadata consumers that previously interpreted the requested preset name as the resolved backend.

No blocking issues. No inline issue met the actionable-evidence threshold; the assessment above records the review feedback.

Automated review; human maintainers own approval decisions.

@greptile-apps

greptile-apps Bot commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR makes all four benchmark play adapters derive run metadata from the resolved environment configuration, aligning play reporting with training, runtime, and startup behavior.

  • Passes env_cfg to run_config_from_presets for RL-Games, RSL-RL, Stable-Baselines3, and SKRL play runs.
  • Adds a regression assertion that an Isaac Sim PhysX configuration reports physx.
  • Adds the required changelog fragment.

Confidence Score: 5/5

The PR appears safe to merge, with the changed play adapters consistently using the resolved configuration already followed by sibling benchmark paths.

The four adapters pass a native resolved environment configuration whose backend fields are finalized before metadata capture, and no changed-code-triggered failure remains.

Important Files Changed

Filename Overview
source/isaaclab/isaaclab/benchmark/entrypoints/backends/rl_games/benchmark_play_rl_games.py Passes the resolved environment configuration into benchmark run metadata construction, matching existing training behavior.
source/isaaclab/isaaclab/benchmark/entrypoints/backends/rsl_rl/benchmark_play_rsl_rl.py Uses the resolved environment configuration when recording play-run backend metadata.
source/isaaclab/isaaclab/benchmark/entrypoints/backends/sb3/benchmark_play_sb3.py Aligns Stable-Baselines3 play metadata capture with the resolved simulation configuration.
source/isaaclab/isaaclab/benchmark/entrypoints/backends/skrl/benchmark_play_skrl.py Aligns SKRL play metadata capture with the resolved simulation configuration.
source/isaaclab/test/benchmark/test_capture.py Extends backend-resolution coverage to verify that a PhysX manager is normalized to the physx metadata value.
source/isaaclab/changelog.d/play-resolved-backend.rst Documents the corrected play benchmark metadata behavior.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
    CLI[Play benchmark arguments] --> Resolve[Resolve task environment config]
    Resolve --> Launch[Resolve and launch simulation backend]
    Launch --> Capture[Build RunConfig from presets and env_cfg]
    Capture --> Metadata[Report resolved physics backend]
Loading

Reviews (1): Last reviewed commit: "Report resolved backend for play benchma..." | Re-trigger Greptile

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working isaac-lab Related to Isaac Lab team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant