Document nuPlan download, conversion, and replay usage#519
Closed
eugenevinitsky wants to merge 4 commits into
Closed
Document nuPlan download, conversion, and replay usage#519eugenevinitsky wants to merge 4 commits into
eugenevinitsky wants to merge 4 commits into
Conversation
Add docs/nuplan_data.md covering the mini-split download, the external py123d converter (repo URL and invocation left as maintainer TODOs, with a reduced-parallelism RAM note), the expected .bin layout, an example replay training command, and how to enable the shipped-disabled nuPlan evaluators. Link it from the README Data section alongside the WOMD download scripts. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Replaces the maintainer placeholders with the real two-stage flow: py123d downloads/parses nuPlan into arrow, 123Drive converts arrow to PufferDrive bins. Includes the RAM guidance (--workers, and the nuplan preset's 20 s log chunking). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Author
|
Filled in the converter placeholders with the real pipeline: py123d (download + arrow parse, per its nuPlan guide) → 123Drive ( |
Fetching nuplan_mini_train/val from the lab buckets is the default path; the py123d + 123Drive pipeline stays as the do-it-yourself route. Depends on the fetch script from the s3-data-fetch PR. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Author
|
Folded into #525 — the nuPlan doc now lives there alongside the fetch script it references (single reviewable unit for data onboarding). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
docs/nuplan_data.md: how to get the nuPlan mini split (maps + mini DBs, no sensor blobs), how conversion to PufferDrive.binworks (external py123d converter — repo URL and exact invocation left as clearly-flagged maintainer TODOs, with a reduced-parallelism RAM note), the expected flatnuplan__<token>.binoutput layout, an example replay training command (--env.map-dir / --env.num-maps / --env.simulation-mode replay / --env.control-mode control_sdc_only / --env.scenario-length 200, mirroring[eval.validation_replay]), and how to enable the nuPlan evaluators (inline viaenabled = true+ localenv.map_dir, or standalone viapuffer eval --evaluator).## Datasection with bullets for the WOMD download scripts (data_utils/download_womd_data.py,data_utils/download_database_gcp.sh) and a pointer todocs/nuplan_data.md.AGENTIC_PRper repo convention for agent-created PRs.Why
There was no documentation for obtaining nuPlan data or converting it to PufferDrive
.binfiles; the converter is external tooling referenced only in commit messages, and its default parallelism OOMs on small-RAM machines. This writes down what is verifiable from the repo and flags the rest as maintainer-supplied.Notes
[eval.validation_replay]and[eval.behaviors_full_dir]as shipping disabled with placeholderenv.map_dirvalues — that is the state after the sibling PRfix/drive-ini-remove-hardcoded-scratch-pathslands. On this branch's base,drive.inistill hasenabled = trueand/scratch/ev2237paths. Do not merge this PR before the sibling PR.c2667356); the nuPlan behavior-category eval bins referenced by the config were a py123d v0.2.1 reconvert (2b1d3ecb). The checked-in sample nuPlan bin's converter version is not recorded, so the doc does not claim one. Converter repo URL and invocation are explicit TODO placeholders — nothing invented.drive.py:268-273(map_dir dir-or-single-.bin) and:368-369(num_maps fail-fast) exact; all example flags exist indrive.ini[env]; sample bin, WOMD scripts, anddocs/evaluation.mdevaluator flow confirmed. Docs-only change, no runtime risk.🤖 Generated with Claude Code
🤖 Generated with Claude Code