Skip to content

Report checkpoint paths for RL-Games and SKRL benchmarks - #7171

Open
AntoineRichard wants to merge 1 commit into
isaac-sim:developfrom
AntoineRichard:antoine/fix-training-checkpoint-path
Open

Report checkpoint paths for RL-Games and SKRL benchmarks#7171
AntoineRichard wants to merge 1 commit into
isaac-sim:developfrom
AntoineRichard:antoine/fix-training-checkpoint-path

Conversation

@AntoineRichard

Copy link
Copy Markdown
Collaborator

Description

RL-Games and SKRL training benchmark bundles now report the checkpoint produced by the completed run instead of leaving TrainingBundle.checkpoint_path unset.

This reuses the canonical get_checkpoint_path resolver rather than adding a second filesystem search mechanism. A small benchmark helper supplies the backend-specific layout and filename patterns:

  • RL-Games resolves nn/*.pth under run_log_dir.
  • SKRL resolves checkpoints/agent_.pt and checkpoints/best_agent.pt under log_dir.
  • ValueError and a missing checkpoint subdirectory are converted to None, preserving the optional bundle field and preventing an otherwise successful training run from failing during final bundle assembly.
  • Current develop naturally sorts numeric filename components, so checkpoint 10 is selected after checkpoint 9.

The change intentionally leaves RSL-RL, SB3, video handling, and Odin tooling untouched.

Type of change

  • Bug fix (non-breaking change which fixes an issue)

Testing

  • uv run --frozen --extra test python -m pytest source/isaaclab/test/benchmark — 334 passed
  • uv run --frozen isaaclab -f — passed
  • uv run --frozen --extra test python tools/changelog/cli.py check develop — passed

Checklist

  • I have read and understood the contribution guidelines.
  • I have run the repository formatting and pre-commit checks.
  • No public documentation changes are required; the user-visible fix has a changelog fragment.
  • My changes generate no new warnings.
  • I have added behavior tests that prove the fix is effective.
  • I have added a changelog fragment for the touched package.
  • My name already exists in CONTRIBUTORS.md.

@AntoineRichard
AntoineRichard requested a review from a team August 19, 2026 13:03
@github-actions github-actions Bot added bug Something isn't working isaac-lab Related to Isaac Lab team labels Aug 19, 2026
@greptile-apps

greptile-apps Bot commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR populates RL-Games and SKRL training bundles with checkpoint paths by reusing the canonical checkpoint resolver.

  • Adds backend-specific checkpoint directory and filename patterns.
  • Resolves checkpoints after successful training while preserving None when no checkpoint is available.
  • Adds tests for supported filenames, absent checkpoints, and natural numeric ordering.
  • Adds a changelog fragment for the behavior change.

Confidence Score: 5/5

The PR appears safe to merge with no concrete blocking or non-blocking defects identified.

The resolver targets each completed run exactly, uses checkpoint layouts consistent with the two backends, and preserves the optional bundle field when no matching artifact exists.

Important Files Changed

Filename Overview
source/isaaclab/isaaclab/benchmark/entrypoints/training.py Adds a shared backend-aware checkpoint resolver with exact run-directory matching and graceful handling of missing checkpoints.
source/isaaclab/isaaclab/benchmark/entrypoints/backends/rl_games/benchmark_train_rl_games.py Populates the completed RL-Games training bundle with the resolved checkpoint path.
source/isaaclab/isaaclab/benchmark/entrypoints/backends/skrl/benchmark_train_skrl.py Populates the completed SKRL training bundle with the resolved checkpoint path.
source/isaaclab/test/benchmark/test_train_checkpoint_path.py Covers backend filename matching, natural checkpoint ordering, and the no-checkpoint fallback.

Reviews (1): Last reviewed commit: "Report training benchmark checkpoint pat..." | Re-trigger Greptile

@isaaclab-review-bot isaaclab-review-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isaac Lab Review Bot

The change correctly populates RL-Games and SKRL training bundle checkpoint paths by reusing the existing checkpoint resolver, while retaining None when no checkpoint can be resolved.

  • Design and architecture: The shared helper centralizes backend-specific checkpoint layouts and avoids duplicating filesystem search logic. Its placement in the training entrypoint creates an internal dependency from backend adapters to that module, but the patch shows no circular import, public stability requirement, or concrete maintenance failure requiring relocation before merge.
  • API: TrainingBundle.checkpoint_path retains its optional str | None contract. Missing directories and unmatched checkpoints remain non-fatal, and the user-visible behavior change is covered by the package changelog fragment.
  • Implementation: The RL-Games and SKRL adapters pass their established run log directories to the resolver. Exact run-directory matching avoids selecting sibling runs, backend patterns cover the documented checkpoint layouts, and tests cover matching filenames, absent checkpoints, and natural numeric ordering. Selection among different SKRL checkpoint forms continues to follow the canonical resolver’s ordering.

No blocking issues. No inline issue met the actionable-evidence threshold; the assessment above records the review feedback.

Automated review; human maintainers own approval decisions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working isaac-lab Related to Isaac Lab team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant