Skip to content

fix(vm): sync the new runtime into var instead of destroying it - #212

Merged
mobileoverlord merged 5 commits into
mainfrom
jschneck/vm-seed-sync
Aug 25, 2026
Merged

fix(vm): sync the new runtime into var instead of destroying it#212
mobileoverlord merged 5 commits into
mainfrom
jschneck/vm-seed-sync

Conversation

@mobileoverlord

Copy link
Copy Markdown
Contributor

Stacked on #196 — targets fix-vm-update-reset-var so the diff reads as a delta. Squashing into #196 is fine; the two are one change in intent.

Why this replaces the reset

#196 deletes var.btrfs on a seed change. That also deletes /var/lib/docker, which holds the SDK image layers and the avo-<uuid> volume that is $AVOCADO_PREFIX — hence the release note telling people to re-run avocado install and avocado build.

That was never necessary. The avocado-owned half of var is already content-addressed and versioned (images/, runtimes/, active), and avocadoctl already installs a runtime alongside the current one and switches active only once it is staged and verified. The guest can adopt the release's runtime out of the new seed and leave everything else alone.

The btrfs work has to happen guest-side regardless: this VM exists because the host is macOS or Windows, neither of which can mount a btrfs image.

Change

  • vm update records runtime.pending_var_seed_sha instead of unlinking the disk.
  • vm start attaches that seed read-only and names its sha and filesystem UUID on the cmdline. The guest echoes the sha into a stamp, which is what lets the marker be cleared for this sync. Every failure leaves it set, so the next start retries — that is the whole recovery story.
  • Never attached when the live disk was byte-copied from that same seed. seed_var_disk is a std::fs::copy, so the two carry an identical btrfs fsid and the kernel would read them as one multi-device filesystem. A correctness gate, not an optimisation.

The ordering bug this found

The first version addressed the seed by serial=, which needs if=none plus an explicit -device. An explicit -device virtio-blk-pci takes a lower PCI slot than the devices qemu creates for if=virtio — so the seed became /dev/vda, the rootfs and var disks shifted, and the VM dropped into emergency mode with root=/dev/vda pointing at a btrfs seed. Fixing the guest's lookup broke the two consumers that address disks positionally: the kernel cmdline and the guest fstab.

Every drive now stays on if=virtio so attaching one is purely additive, and the guest finds the seed by filesystem UUID. Reading that UUID is a 16-byte read at a fixed superblock offset rather than shelling out to blkid — the hosts this VM exists for have neither blkid nor btrfs. Nothing is cfg-gated; it behaves the same on macOS, Linux and Windows.

Tests pin the ordering property directly: no seed emits one if=virtio drive, a seed emits exactly two, the seed never precedes the rootfs, and no explicit -device appears.

Also included

ext/build.rs now reads [Install] from drop-ins, not just the unit body. systemd honours a WantedBy= declared in a drop-in, so the standard way to enable a vendor unit shipping without an [Install] was invisible to the build. Parsed one file at a time — a single sed over several files shares an input stream, so the [Install] range would run past a file boundary. Test runs the generated fragment with sh against a real sysroot and asserts the .wants symlink appears; it fails against the previous parser.

What this removes from #196

Nothing is destroyed, so --reset-var, the typed-update prompt and the var_reset event all go. The JSON event becomes var_seed_sync_pending, which host applications can ignore rather than act on. record_var_size_floor goes too — it existed only so a grown disk survived the delete.

min_cli_version — the most valuable part of #196 — is untouched.

Testing

1412 unit tests, clippy clean.

End to end on a real VM: stock 0.3.0 migrated to a locally built release, 8/8 assertions — var.btrfs the same inode throughout, active runtime changed, all extensions merged, and the user's Docker volume, image layers and /var marker intact. Kernel 6.6.123 → 6.18.35, extensions 2024.1.0 → 0.1.0.

Depends on

avocado-vm#3 (guest unit) and avocado-vm#4. Without #3 the drive is attached and nothing reads it — exactly today's behaviour, so they can land in either order.


Reopened as a new PR: #206 was auto-closed when its stack base (#196, now merged) was deleted, and GitHub refuses to reopen across a deleted base. Branch rebased onto main (3259 tests pass); the #196 commits dropped out as already-merged.

Copilot AI lite review requested due to automatic review settings August 25, 2026 11:22

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR changes the VM update path to preserve the live /var disk on seed changes by scheduling a guest-side sync of Avocado-managed state (via a read-only attached seed) instead of deleting/reseeding var.btrfs. It also fixes systemd unit enabling in extension builds by honoring [Install] directives coming from drop-ins.

Changes:

  • Record runtime.pending_var_seed_sha during vm update and attach the corresponding seed read-only on vm start, passing both seed sha and btrfs fsid to the guest via kernel cmdline.
  • Add host-side btrfs fsid extraction (fixed-offset read) and best-effort clearing of the pending marker only after the guest confirms the seed was applied.
  • Update extension build service-enabling logic to read [Install] from drop-ins (and add an integration-style test that runs the emitted shell fragment).

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
src/utils/vm/qemu.rs Adds VarSeed and appends a read-only seed -drive; passes seed identifiers on the kernel cmdline; adds ordering/shape tests.
src/utils/vm/lifecycle.rs Resolves pending seed attachment, reads btrfs fsid from the seed image, and clears pending_var_seed_sha after guest confirmation.
src/utils/vm/config.rs Adds persisted runtime.pending_var_seed_sha config field and updates config round-trip tests.
src/main.rs Removes --reset-var flag plumbing and updates CLI help text to reflect non-destructive updates.
src/commands/vm/update.rs Switches from destructive var reset to recording a pending seed sync; updates prompt behavior and JSON event.
src/commands/ext/build.rs Parses [Install] from unit drop-ins when enabling services and adds a test that executes the generated shell fragment.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/utils/vm/qemu.rs Outdated
Comment thread src/utils/vm/lifecycle.rs
Comment thread src/commands/vm/update.rs Outdated
Comment thread src/commands/ext/build.rs
Builds on the var-reset branch and replaces the reset.

Deleting var.btrfs on a seed change also deletes /var/lib/docker, which
holds the SDK image layers and the avo-<uuid> volume that *is*
$AVOCADO_PREFIX — so every update cost a full SDK re-pull and sysroot
rebuild, and the release notes had to tell people to re-run `avocado
install` and `avocado build`.

That was never necessary. The avocado-owned half of var is already
content-addressed and versioned (images/, runtimes/, active), and
avocadoctl already installs a runtime alongside the current one and
switches `active` only once it is staged and verified. So the guest can
adopt the release's runtime out of the new seed and leave everything
else untouched. The btrfs work has to happen guest-side regardless:
this VM exists because the host is macOS or Windows, neither of which
can mount a btrfs image.

- `vm update` records runtime.pending_var_seed_sha rather than
  unlinking the disk.
- `vm start` attaches that seed read-only and names its sha on the
  cmdline; the guest echoes it into a stamp, which is what lets the
  marker be cleared for this sync specifically. Every failure leaves it
  set, so retry is the whole recovery story.
- The seed is addressed by serial rather than /dev/vdN: the var and
  data drives are both conditional, so enumeration order is not stable
  across boots or arches.
- Never attached when the live disk was byte-copied from that same
  seed — identical btrfs fsid, which the kernel reads as one
  multi-device filesystem.

Nothing is destroyed, so --reset-var, the typed-'update' prompt and the
var_reset event go away; the JSON event becomes var_seed_sync_pending,
which host applications can ignore rather than acting on.
record_var_size_floor goes with them — it existed only so a grown disk
survived the delete.

Requires the guest-side unit in avocado-vm; without it the drive is
attached and nothing reads it, which is exactly today's behaviour.
`enable_services` parsed WantedBy=/RequiredBy= out of the unit file
alone. systemd also reads [Install] from drop-ins, and `systemctl
enable` honours a WantedBy= declared there — so the standard way to
enable a vendor unit that ships without an [Install] was invisible
here, and the build failed with the fix already sitting in the overlay.

Hit while building avocado-vm against distro 2026, where
qemu-guest-agent.service no longer carries an [Install] section.

Parsed one file at a time: a single sed over several files shares an
input stream, so the /^\[Install\]/,/^\[/ range would run past the end
of one file and match keys in the next.

Test runs the generated fragment with sh against a real sysroot holding
a unit with no [Install] plus a drop-in supplying one, and asserts the
.wants symlink appears — string assertions can't catch a shell bug in
generated code. Fails against the previous parser.
… UUID

Found by running the upgrade on a real VM: it dropped into emergency
mode, with btrfs on /dev/vda where the erofs rootfs belongs.

`serial=` needs `if=none` plus an explicit `-device`, and an explicit
`-device virtio-blk-pci` takes a lower PCI slot than the devices qemu
creates for `if=virtio`. So the seed became /dev/vda and pushed the
rootfs and var disks along, breaking both `root=/dev/vda` on the cmdline
and `/dev/vdb /var` in the guest fstab. Addressing the seed by serial
fixed the guest's lookup and broke the two consumers that address disks
by position.

Keep every drive on `if=virtio` so attaching one is purely additive, and
identify the seed by its btrfs filesystem UUID instead, passed as
avocado.seed_fsid= alongside the sha.

Reading that UUID is a 16-byte read at a fixed superblock offset rather
than a shell out to blkid: the hosts this VM exists for are macOS and
Windows, which have neither blkid nor btrfs. Nothing here is cfg-gated,
so it behaves the same on all three.

Tests pin the ordering property directly — no seed emits one if=virtio
drive, a seed emits exactly two, the seed never precedes the rootfs, and
no explicit -device appears.
All four from Copilot, all real:

- pending_seed_sha was derived before the btrfs_fsid filter, so a seed
  whose UUID cannot be read (truncated, corrupt) was never attached yet
  still interrogated the guest for an application that could not have
  happened, silently, on every start. The sha now comes from the
  VarSeed actually attached, and the unreadable-seed case warns loudly
  instead of retrying quietly forever.

- The [Install]-parsing shell accumulated file paths into one
  space-separated string and re-split it, breaking on any path with a
  space. Each file is now parsed as encountered through a quoted "$1".

- vm/update.rs still carried the delete-the-disk comment from the
  superseded reset approach, describing destruction the code no longer
  performs.

- The seed-drive test claimed addressing by virtio serial; the
  mechanism is (deliberately) the filesystem UUID. Renamed to pin what
  it actually asserts.
The no_stdout_on_the_vm_path guard landed on main after this branch
forked and rightly flags this site: it runs on the vm start path where
the following command may own stdout.
@mobileoverlord
mobileoverlord merged commit 80c818c into main Aug 25, 2026
9 checks passed
@mobileoverlord
mobileoverlord deleted the jschneck/vm-seed-sync branch August 25, 2026 11:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants