Skip to content

ENH: Per-organ tutorial parameters and distance-map registration fixes #119

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
aylward wants to merge 2 commits into
base: main
Choose a base branch
from
Open

ENH: Per-organ tutorial parameters and distance-map registration fixes #119

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
a34fc1d
ENH: Per-organ tutorial parameters and distance-map registration fixes
aylward Aug 8, 2026
18d225e
ENH: Label-aware meshing, labeled USD export, Duke heart tutorial track
aylward Aug 11, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .gitattributes
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,7 @@ tutorials/network_weights/physicsnemo_mgn_heart/*.pt filter=lfs diff=lfs merge=l
tutorials/network_weights/physicsnemo_mgn_heart/*.vtu filter=lfs diff=lfs merge=lfs -text
tutorials/network_weights/physicsnemo_mgn_heart/*.vtp filter=lfs diff=lfs merge=lfs -text
tutorials/network_weights/physicsnemo_mgn_heart/pca_model.json filter=lfs diff=lfs merge=lfs -text
tutorials/network_weights/physicsnemo_mgn_lung_motion/*.pt filter=lfs diff=lfs merge=lfs -text
tutorials/network_weights/physicsnemo_mgn_lung_motion/*.vtu filter=lfs diff=lfs merge=lfs -text
tutorials/network_weights/physicsnemo_mgn_lung_motion/*.vtp filter=lfs diff=lfs merge=lfs -text
tutorials/network_weights/physicsnemo_mgn_lung_motion/pca_model.json filter=lfs diff=lfs merge=lfs -text
3 changes: 2 additions & 1 deletion .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,10 +34,11 @@ docs/assets/*.mp4
# Network weights
network_weights

# Track the MGN heart model weights (other network_weights stay ignored)
# Track the MGN heart and lung model weights (other network_weights stay ignored)
!tutorials/network_weights/
tutorials/network_weights/*
!tutorials/network_weights/physicsnemo_mgn_heart/
!tutorials/network_weights/physicsnemo_mgn_lung_motion/

# Data files
*.gz
Expand Down
1 change: 1 addition & 0 deletions data/Duke-Heart-4DLabelmaps/.gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
pm*
31 changes: 31 additions & 0 deletions data/Duke-Heart-4DLabelmaps/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# Duke-Heart-4DLabelmaps

Gated 4D cardiac labelmaps acquired at Duke University by Dr. Paul Segars.

## Availability

This dataset is **not currently available**. It is being considered for public
release; until that happens it cannot be downloaded, and it is not distributed
with this repository.

## Effect on the tutorials

Tutorials that depend on this dataset are named with a `duke_heart` prefix in
their organ field, for example:

- `tutorials/tutorial_02_duke_heart_distancemap_finetune_icon.py`

These `duke_heart` tutorials will not run without the data. Every other
tutorial uses a publicly available dataset and is unaffected — see
[../README.md](../README.md) for download instructions.

Downstream tutorials that consume `duke_heart` outputs (such as the finetuned
distance-map ICON weights used by
`tutorials/tutorial_07_heart_fit_statistical_model_to_patient.py`) fall back to
stock weights and still run, with reduced accuracy.

## Expected layout

When available, the data is expected under `data/Duke-Heart-4DLabelmaps/` as
one directory per case (`pm0002/`, `pm0003/`, ...), each holding one labelmap
per gated frame.
4 changes: 2 additions & 2 deletions docs/api/registration/chained.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@ Chained Image Registration
Coarse-to-fine registration composes two registrars: a fast, robust method
recovers the large motion, then a deformable method refines it.
``RegisterImagesChain`` is the general composition;
``RegisterImagesGreedyICON`` is the Greedy-then-ICON pairing the tutorials use
for 4D CT.
``RegisterImagesGreedyICON`` is the Greedy-then-ICON pairing, used by Tutorial 2
and by the distance-map stage of the statistical-model fit.
Comment on lines +12 to +13

@coderabbitai coderabbitai Bot Aug 8, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "Files matching chained.rst:"
fd -a 'chained\.rst$' . || true

echo
echo "Target file excerpt:"
if [ -f docs/api/registration/chained.rst ]; then
  nl -ba docs/api/registration/chained.rst | sed -n '1,80p'
fi

echo
echo "Search for RegisterImagesGreedyICON/RegisterModelsDistanceMaps/Tutorial 2:"
rg -n "RegisterImagesGreedyICON|RegisterModelsDistanceMaps|RegisterImagesGreedy|RegisterImagesICON|Tutorial 2|Tutorial.*2|distance-map|distance map" . || true

Repository: Project-MONAI/physiotwin4d

Length of output: 308

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "Repository files relevant to registration:"
git ls-files | rg 'registration|tutorial|models|fit|distance' || true

echo
echo "All definitions/usages with some context:"
rg -n "class RegisterImagesGreedyICON|RegisterImagesGreedyICON\s*=|class RegisterModelsDistanceMaps|RegisterModelsDistanceMaps\s*=|RegisterImagesGreedy|RegisterImagesICON" . || true

echo
echo "Candidate files by name:"
git ls-files | rg 'registration|tutorial|statistical|model|distance' | head -200 || true

Repository: Project-MONAI/physiotwin4d

Length of output: 40451

Correct the documented consumers of RegisterImagesGreedyICON.

Tutorial 2 distance-map stages use separate RegisterImagesGreedy and RegisterImagesICON instances; they do not use RegisterImagesGreedyICON. Replace the Tutorial 2 claim with a consumer that actually uses the chained registrar, or keep the wording as the documented Greedy-then-ICON pattern.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/api/registration/chained.rst` around lines 12 - 13, Update the
documentation for RegisterImagesGreedyICON to remove the incorrect claim that
Tutorial 2 uses it. Either name a verified consumer of the chained registrar or
describe it only as the Greedy-then-ICON pairing, while preserving the accurate
statistical-model fit reference.


Both implement :class:`RegisterImagesBase`, so they drop into any workflow that
takes a ``registration_method``.
Expand Down
3 changes: 3 additions & 0 deletions docs/assets/tutorial_04_duke_heart.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
3 changes: 3 additions & 0 deletions docs/assets/tutorial_04_heart-2png.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
3 changes: 3 additions & 0 deletions docs/assets/tutorial_04_lung-2.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
74 changes: 38 additions & 36 deletions docs/tutorials.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Tutorials
<p class="pt4d-kicker">PhysioTwin4D tutorials</p>
<h1>From a CT scan to an animated digital twin</h1>
<p>
Ten numbered stages across 16 runnable Python scripts.
Ten numbered stages across 17 runnable Python scripts.
Comment thread
aylward marked this conversation as resolved.
Each one drives the real workflow classes end-to-end on downloadable
data, shows what it produced, and ends with the handful of constants
to change so it runs on your own scans.
Expand Down Expand Up @@ -132,13 +132,15 @@ pipeline on top.

1. **Tutorial 1** — after downloading Slicer-Heart-CT.
2. **Tutorial 2** — after obtaining DIR-Lab. It writes the finetuned ICON
weights that Tutorials 3 (lung) and 8 use.
3. **Tutorial 3** — after Tutorial 2, whose weights it registers with.
weights Tutorial 8 uses when present; optional if stock weights are
acceptable.
3. **Tutorial 3** — after obtaining its dataset; it registers with Greedy and
needs no finetuned weights.
4. **Tutorial 4** — after downloading Slicer-Heart-CT.
5. **Tutorial 5** — after Tutorial 4, whose surfaces it converts.
6. **Tutorial 6** — heart needs KCL-Heart-Model, lung needs DIR-Lab.
7. **Tutorial 7** — after Tutorial 6; the lung variant also needs Chest-CT.
8. **Tutorial 8** — after Tutorials 2 and 6 (lung).
8. **Tutorial 8** — after Tutorial 6 (lung); Tutorial 2 optional.
9. **Tutorial 9** — after Tutorial 8, whose fitted meshes it trains on.
10. **Tutorial 10** — after Tutorial 9, whose checkpoint it loads.

Expand All @@ -152,7 +154,7 @@ Script

Workflow
:class:`~physiotwin4d.WorkflowConvertImageToUSD`, driving
:class:`~physiotwin4d.RegisterImagesICON` and a
:class:`~physiotwin4d.RegisterImagesGreedy` and a
:class:`~physiotwin4d.SegmentAnatomyBase` subclass.

Dataset
Expand All @@ -161,9 +163,8 @@ Dataset
registration reference.

Requirements
GPU strongly recommended — ICON registers every phase against the
reference. Swap in :class:`~physiotwin4d.RegisterImagesGreedy` or
:class:`~physiotwin4d.RegisterImagesANTS` for CPU-only environments.
Greedy registers every phase against the reference on the CPU; a GPU is
still needed for segmentation.

Preview
.. figure:: assets/tutorial_01_heart_4d.gif
Expand Down Expand Up @@ -219,11 +220,21 @@ Tutorial 2: Finetune ICON Registration
Script
``tutorials/tutorial_02_lung_finetune_icon.py``

``tutorials/tutorial_02_lung_distancemap_finetune_icon.py`` — the
``tutorials/tutorial_02_lung_distancemap_finetune_icon.py`` — the lung
distance-map variant, which finetunes on distance maps rather than image
intensities so the labelmap-to-labelmap stage of Tutorials 7 and 8 has
in-distribution weights.

``tutorials/tutorial_02_duke_heart_distancemap_finetune_icon.py`` — the same
for the heart, on Duke-Heart-4DLabelmaps. The heart needs its own run because
it registers with a much tighter mask than the lungs, so its distance maps
saturate over a shorter radius and do not share an intensity distribution
with lung ones. The per-organ values live in
``tutorials/parameters_lung_ct_dirlab.py`` and
``tutorials/parameters_heart_ct_kcl.py``. This is a ``duke_heart`` tutorial:
Duke-Heart-4DLabelmaps is not publicly available yet, so it cannot be run —
see ``data/Duke-Heart-4DLabelmaps/README.md``.

Workflow
:class:`~physiotwin4d.WorkflowFinetuneICONRegistration`, then
:class:`~physiotwin4d.RegisterImagesGreedy` and
Expand Down Expand Up @@ -301,20 +312,15 @@ Script

Workflow
:class:`~physiotwin4d.WorkflowReconstructHighres4DCT` with
:class:`~physiotwin4d.RegisterImagesGreedyICON`.
:class:`~physiotwin4d.RegisterImagesGreedy`.

Dataset
Slicer-Heart-CT for the heart; DIR-Lab for the lung, which reconstructs
against its T70 (end-exhale) phase — the same reference Tutorial 8 fits to.
The lung variant registers with **Tutorial 2's finetuned ICON weights** when
they exist, and logs a warning and falls back to the stock uniGradICON
weights when they do not.

Requirements
GPU recommended. One coarse-to-fine registration per phase, greedy schedule
``[30, 15, 7, 3]``. The lung variant enables mass preservation for
non-contrast CT; the heart variant does not and uses the stock uniGradICON
weights.
CPU is enough. One coarse-to-fine registration per phase, greedy schedule
``[30, 15, 7, 3]``.

Preview
.. figure:: assets/Tutorial_03_heart_original.gif
Expand All @@ -338,11 +344,8 @@ Preview
Inner API usage
.. code-block:: python

registration_method = RegisterImagesGreedyICON()
registration_method.greedy.set_number_of_iterations([30, 15, 7, 3])
registration_method.icon.set_mass_preservation(True)
if icon_weights_path.exists(): # Tutorial 2 output, optional
registration_method.icon.set_weights_path(str(icon_weights_path))
registration_method = RegisterImagesGreedy()
registration_method.set_number_of_iterations([30, 15, 7, 3])

workflow = WorkflowReconstructHighres4DCT(
time_series_images=time_series,
Expand All @@ -366,10 +369,7 @@ Outputs

Adapt to your data
Set ``case_glob`` and ``data_dir`` to your series and pick the reference
with ``reference_time_frame``. Point ``icon_weights_path`` at weights you
finetuned on your own cohort with Tutorial 2, or leave it missing to
register with the stock uniGradICON weights. If you have a separate
breath-hold or
with ``reference_time_frame``. If you have a separate breath-hold or
contrast-enhanced volume, pass it as ``reference_image`` instead of one of
the phases — that is what the workflow is really designed for. Tune
``number_of_iterations_greedy`` down for a fast smoke test. The saved
Expand Down Expand Up @@ -417,7 +417,7 @@ Inner API usage
)
result = workflow.process(
input_image=ct_image,
surface_target_reduction=0.5,
surface_reduction_rate=HEART_CT_KCL.surface_reduction_rate,
extract_label_surfaces=save_label_surfaces,
)

Expand All @@ -437,7 +437,8 @@ Adapt to your data
Change the input volume path, then choose the segmenter matching your scan:
contrast versus non-contrast CT, or
:class:`~physiotwin4d.SegmentNVSegmentCTMRI` for CT **and** MRI. Raise
``surface_target_reduction`` toward ``1.0`` for lighter meshes. Every
``surface_reduction_rate`` in the tutorial's parameter module toward ``1.0``
for lighter meshes. Every
segmenter declares its own labels through
:class:`~physiotwin4d.AnatomyTaxonomy`, so downstream grouping and USD
materials follow automatically — see :doc:`api/segmentation/index`.
Expand Down Expand Up @@ -657,9 +658,9 @@ Workflow
the fitted surface through every other phase.

Dataset
DIR-Lab, plus Tutorial 6 (lung)'s model. Tutorial 2's finetuned ICON weights
are used when present; without them the tutorial warns and registers with the
stock uniGradICON weights.
DIR-Lab, plus Tutorial 6 (lung)'s model. Tutorial 2's finetuned distance-map
ICON weights are used by the model fit when present; without them the
tutorial warns and fits with the stock uniGradICON weights.

Requirements
GPU required, and the heaviest registration workload in the set: one
Expand Down Expand Up @@ -766,10 +767,11 @@ Run
python tutorials/tutorial_09_lung_train_physicsnemo_mgn.py

Outputs
``mgn_stage_model.pt``, its metadata and loss/RMSE logs, the per-case
manifests, and the held-out evaluation under ``eval_mgn/`` — in the
directory training used: ``tutorials/output/tutorial_09_lung_mgn/``
normally, or a fresh sibling when resuming.
``mgn_stage_model.pt``, its metadata and loss/RMSE logs, in the shared
weights directory Tutorial 10 reads
(``tutorials/network_weights/physicsnemo_mgn_lung_motion/``, a fresh sibling
of it when resuming). The per-case manifests and the held-out evaluation
under ``eval_mgn/`` stay in ``tutorials/output/tutorial_09_lung_mgn/``.

Adapt to your data
The contract is the manifest, not the tutorial. Each JSON names a reference
Expand Down Expand Up @@ -829,7 +831,7 @@ Run
Outputs
The predicted surface, its error statistics against the ground-truth phase
in millimetres, and a USD scene, under
``tutorials/output/tutorial_09_lung_mgn/tutorial_10_lung_mgn/<case>/``.
``tutorials/output/tutorial_10_lung_mgn/<case>/``.

Adapt to your data
Change ``case_id`` and ``stage_fraction`` to predict a different subject, or
Expand Down
11 changes: 11 additions & 0 deletions pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,7 @@ dependencies = [
"pyvista[all]>=0.47.0",
"usd-core>=23.11",
"trimesh>=4.0.0",
"pyacvd>=0.4.0",

# Utilities
"ipykernel>=6.0.0",
Expand Down Expand Up @@ -289,6 +290,8 @@ module = [
"physicsnemo.*",
"pxr",
"pxr.*",
"pyacvd",
"pyacvd.*",
"pyvista",
"pyvista.*",
"simpleware",
Expand Down Expand Up @@ -325,17 +328,25 @@ module = [
# pre-commit mypy environment and ships no py.typed marker when installed.
# mypy per-module patterns match whole components only, so each tutorial is
# listed by name; add a line here when adding or renaming a tutorial.
"parameters_duke_heart_labelmaps",
"parameters_heart_ct_kcl",
"parameters_lung_ct_dirlab",
"tutorial_01_heart_gated_ct_to_usd",
"tutorial_01_lung_gated_ct_to_usd",
"tutorial_02_duke_heart_distancemap_finetune_icon",
"tutorial_02_lung_distancemap_finetune_icon",
"tutorial_02_lung_finetune_icon",
"tutorial_03_heart_reconstruct_highres_4d_ct",
"tutorial_03_lung_reconstruct_highres_4d_ct",
"tutorial_04_duke_heart_labelmap_to_vtk",
"tutorial_04_heart_ct_to_vtk",
"tutorial_04_lung_ct_to_vtk",
"tutorial_05_duke_heart_vtk_to_usd",
"tutorial_05_heart_vtk_to_usd",
"tutorial_06_duke_heart_create_statistical_model",
"tutorial_06_heart_create_statistical_model",
"tutorial_06_lung_create_statistical_model",
"tutorial_07_duke_heart_fit_statistical_model_to_patient",
"tutorial_07_heart_fit_statistical_model_to_patient",
"tutorial_07_lung_fit_statistical_model_to_patient",
"tutorial_08_lung_fit_model_to_4d_patients",
Expand Down
4 changes: 2 additions & 2 deletions src/physiotwin4d/cli/convert_image_to_vtk.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ def main() -> int:
),
)
parser.add_argument(
"--surface-target-reduction",
"--surface-reduction-rate",
type=float,
default=0.0,
help=(
Expand Down Expand Up @@ -177,7 +177,7 @@ def main() -> int:
result = workflow.process(
input_image=input_image,
anatomy_groups=args.anatomy_groups,
surface_target_reduction=args.surface_target_reduction,
surface_reduction_rate=args.surface_reduction_rate,
extract_label_surfaces=(args.output_mode == "label"),
)
except (ValueError, RuntimeError, OSError) as exc:
Expand Down
Loading
Loading