Skip to content

feat(examples): zarr dataset configs and .pdmsh converter for the unified external aero recipe#1828

Draft
negin513 wants to merge 1 commit into
NVIDIA:mainfrom
negin513:feat/zarr-mesh-recipe
Draft

feat(examples): zarr dataset configs and .pdmsh converter for the unified external aero recipe#1828
negin513 wants to merge 1 commit into
NVIDIA:mainfrom
negin513:feat/zarr-mesh-recipe

Conversation

@negin513

@negin513 negin513 commented Jul 10, 2026

Copy link
Copy Markdown
Member

Description

Split out of #1798 per review discussion (scope proposal): the recipe integration for the mesh-zarr storage path, held as a separate decision from the library core until the team settles whether to migrate curated datasets.

What's included

  • drivaer_ml_surface_zarr.yaml / drivaer_ml_volume_zarr.yaml: DrivAerML dataset configs reading physicsnemo-domainmesh-zarr groups. They differ from their .pdmsh twins only in the reader: block — the full transform stack and all five recipe models run unchanged.
  • src/convert_pdmsh_to_zarr.py: transitional CLI for datasets already curated to .pdmsh (new curation should emit the schema directly from PhysicsNeMo-Curator; companion PR: feat(mesh): DomainMeshZarrSink emitting the physicsnemo mesh-zarr schema physicsnemo-curator#64).
  • dataset_paths.yaml: adds the drivaer_ml_zarr path key.

Evidence (carried over from #1798, measured with these configs)

  • Matched training A/Bs (seed 42): surface — final train loss 0.1266 (.pdmsh) vs 0.1265 (zarr); volume — 0.2648 vs 0.2680 train / 0.0998 vs 0.0987 val at epoch 3; trajectories track epoch-for-epoch within subsample-RNG noise.
  • End-to-end: GeoTransolver surface and volume both train from zarr through the recipe's train.py (volume incl. on-the-fly SDF against the in-group full-resolution STL boundary).
  • DataLoader (benchmark_io, 8 DrivAerML cases, Lustre): 3.66 cold / 7.96 warm batches/s (.pdmsh) vs 9.23 / 23.64 (zarr) — raw logs.

Dependencies

Merge-blocks on #1798 (imports save_domain_mesh_to_zarr / to_cell_soup; configs target ${dp:ZarrMeshReader} / ${dp:ZarrDomainMeshReader}). Draft until #1798 lands and the dataset-migration decision is made.

Checklist

🤖 Generated with Claude Code

… aero recipe

Split out of NVIDIA#1798 per review: the drivaer_ml_surface_zarr /
drivaer_ml_volume_zarr dataset configs (identical to their .pdmsh twins
except the reader: block) and the transitional convert_pdmsh_to_zarr.py
CLI. Depends on NVIDIA#1798 for ZarrMeshReader / ZarrDomainMeshReader /
save_domain_mesh_to_zarr. Kept out of the flagship recipe until the
dataset-migration decision lands.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@copy-pr-bot

copy-pr-bot Bot commented Jul 10, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant