Skip to content

feat(container): lean pixi-based two-stage container with multi-arch support - #466

Open
jcgraciosa wants to merge 2 commits into
underworldcode:developmentfrom
jcgraciosa:feature/lean-container-pixi
Open

feat(container): lean pixi-based two-stage container with multi-arch support#466
jcgraciosa wants to merge 2 commits into
underworldcode:developmentfrom
jcgraciosa:feature/lean-container-pixi

Conversation

@jcgraciosa

Copy link
Copy Markdown
Contributor

Summary

  • Rewrites container/Containerfile to a pixi two-stage build, fixing PETSc
    version drift between the container and Binder (root cause of the v3.1.0
    container build failure)
  • Adds linux/arm64 support for native performance on ARM64 machines
    (Apple Silicon Macs and ARM64 Linux servers/cloud instances)
  • Moves gmsh from PyPI to conda-forge — restores the original
    environment.yaml approach and was required to unblock arm64 support
    (PyPI gmsh has no linux-aarch64 wheels)

Details

Containerfile: Two-stage build (builder: pixi install + compile UW3;
runtime: copy env + Mesa apt packages). Drops vtk-osmesa; uses conda-forge
vtk + Mesa instead — same as Binder, works on amd64 and arm64.

pixi.toml/pixi.lock: Added linux-aarch64 platform and target sections.
gmsh moved to [dependencies] (conda-forge).

environment.yaml: Marked as human-readable reference only.

Testing

  • pixi install -e runtime: linux/arm64 ✓, linux/amd64 ✓, osx-arm64 ✓
  • Full container build: CI

Underworld development team with AI support from Claude Code

Juan Carlos Graciosa and others added 2 commits July 29, 2026 13:34
@jcgraciosa
jcgraciosa requested a review from lmoresi as a code owner July 29, 2026 05:38

@lmoresi lmoresi left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adversarial review

The two-stage structure itself held up under attack (see the cleared list), but three findings block merge:

  1. CRITICALimport gmsh is broken on linux-aarch64, the PR's headline platform. Conda-forge gmsh-4.15.2 ships the app/library only — no Python module; that lives in the separate python-gmsh package, which is absent from the lock. linux-64/osx-arm64 stayed green only because pygmsh drags the pypi gmsh wheel back in; no such wheel exists for linux-aarch64, so the solver silently downgraded pygmsh to 7.1.12 and locked no gmsh Python module at all. The arm64 container cannot build a single mesh. Fix: add conda python-gmsh (the old environment.yaml listed both) and re-solve the lock; verify import gmsh in the linux-aarch64 env.
  2. MAJOR — no green container build exists for the PR head. The only build attempt ran on f814251 and died at pixi install -e runtime (aarch64 unsupported); the fix commit 47ffb64 was never build-tested. The workflow has no pull_request trigger, so merging is the first real test — and it pushes to the ghcr :development tag. One demonstrated green build of the actual head (fork workflow_dispatch is enough) before merge.
  3. MAJOR — no .dockerignore, and COPY . . runs after pixi install -e runtime: the documented local podman build . uploads the multi-GB host .pixi/ and build/ as context and merges the host's .pixi/envs/* over the freshly installed runtime env — a wrong-arch clobber for anyone who has built natively (as the PR author has). CI is unaffected (clean checkout); local builds are the trap. Exclude .pixi/build or add a .dockerignore.

Minor: environment.yaml's new header calls itself the runtime reference but still says petsc=3.24 while the lock ships 3.25.3 — wrong on exactly the pin that caused #355; lock churn on development now triggers full dual-arch rebuilds with arm64 under QEMU and no build cache configured; curl pixi.sh/install.sh | sh is unpinned; the runtime image still carries the full compiler toolchain, so "lean" is relative.

Attacks that failed: no pip install -e anywhere (the only -e is pixi's env selector); two-stage copy is path-identical so rpaths and the UW3 .so survive; kernelspec copied; workflow secrets/permissions sane (GITHUB_TOKEN, packages:write, no per-PR trigger); pixi.lock internally consistent (petsc==petsc4py==3.25.3, numpy 2.4.6 < 2.5, python 3.12, identical across envs/platforms); dev-env pixi tasks untouched.

Not run: an actual container build (runtime boot, OSMesa render, QEMU arm64 duration unverified).

Also for the record: #355 is already closed by #356; this PR supersedes #356's mechanism (environment.yaml stops being a build input) but doesn't close anything.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants