fix(deepmd/hdf5): skip empty optional frame arrays on dump (#996)

[CedricConday]

Fixes #996.

The deepmd/hdf5 writer reshapes and writes every optional frame array, so an empty optional property (e.g. forces/virials when cal_force/cal_stress is disabled) is stored as a (nframes, 0) dataset. On load, to_system_data tries to reshape that zero-width dataset back to (nframes, natoms, 3) and raises ValueError: cannot reshape array of size 0.

Reproducer (fails on current master):

import dpdata, numpy as np
s = dpdata.LabeledSystem(data={
    "atom_names": ["H"], "atom_numbs": [1], "atom_types": np.array([0]),
    "cells": np.eye(3).reshape(1, 3, 3), "coords": np.zeros((1, 1, 3)),
    "energies": np.zeros(1), "orig": np.zeros(3),
    "forces": np.zeros((1, 0, 3)),
})
s.to("deepmd/hdf5", "t.h5")
dpdata.LabeledSystem("t.h5", fmt="deepmd/hdf5")   # ValueError before this change

The deepmd/raw and deepmd/npy writers already skip this case; this makes the HDF5 writer consistent with them by skipping an optional array whose size is 0 while the system still has frames. Non-empty arrays are written and reloaded exactly as before.

Added TestHDF5EmptyOptionalArray covering both the previously-failing empty round-trip and a non-empty round-trip to guard against regression. The deepmd test suites pass (test_deepmd_hdf5, test_deepmd_raw, test_deepmd_comp, test_deepmd_mixed) and ruff is clean.

Disclosure: written with AI assistance; reviewed and tested before submitting.

Summary by CodeRabbit

• Bug Fixes

  • Improved HDF5 export/import handling for optional empty per-frame arrays, avoiding invalid data files that could fail to load.
  • Empty optional fields are now skipped during save, while valid data continues to round-trip correctly.

• Tests

  • Added regression coverage for empty optional array handling.
  • Added verification that real force data is still preserved after save and reload.

[codspeed-hq]

Merging this PR will not alter performance

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

✅ 2 untouched benchmarks

---

_{Comparing CedricConday:fix/hdf5-empty-optional-arrays (642a745) with master (221f254)}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.87%. Comparing base (221f254) to head (642a745).

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #1011   +/-   ##
=======================================
  Coverage   86.87%   86.87%           
=======================================
  Files          89       89           
  Lines        8264     8266    +2     
=======================================
+ Hits         7179     7181    +2     
  Misses       1085     1085           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 6375cf7e-99ea-438a-a168-f97b7916de4d

📥 Commits

Reviewing files that changed from the base of the PR and between 221f254 and 642a745.

📒 Files selected for processing (2)

• dpdata/formats/deepmd/hdf5.py
• tests/test_deepmd_hdf5.py

---

📝 Walkthrough

Walkthrough

The DeepMD HDF5 dump function now skips writing optional per-frame property datasets (e.g., forces) when their data is empty but the system has frames, avoiding an unreshapeable zero-width dataset on reload. Regression tests were added to verify this behavior.

Changes

Empty optional array fix

Layer / File(s)	Summary
Skip empty optional dataset on dump dpdata/formats/deepmd/hdf5.py	Added a guard in dump that skips creating a dataset for an optional per-frame property when its data has zero total size but nframes > 0, preventing unusable (nframes, 0)-shaped datasets.
Regression tests for round-trip behavior tests/test_deepmd_hdf5.py	Added TestHDF5EmptyOptionalArray test class checking that empty forces are omitted on reload while coords still round-trip, and that real forces data is preserved.

Estimated code review effort: 1 (Trivial) | ~5 minutes

Possibly related PRs

• deepmodeling/dpdata#981: Also modifies the DeepMD HDF5 dump logic in the same file for dataset-writing eligibility.
• deepmodeling/dpdata#982: Applies the same fix of skipping empty optional per-frame properties when nframes > 0 to prevent unusable datasets.

Suggested labels: `dpdata`

Suggested reviewers: `iProzd`, `wanghan-iapcm`, `njzjz`

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly describes the main fix in DeepMD HDF5 dump behavior and matches the PR scope.
Linked Issues check	✅ Passed	The change skips empty optional HDF5 frame arrays and adds regression tests for empty and non-empty round trips, matching #996.
Out of Scope Changes check	✅ Passed	The diff is limited to the HDF5 writer and its tests, with no obvious unrelated or extra-scope changes.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}