release: 0.40.0 — vm download UX, lifecycle-on-CLI, config tree, deploy JSON#145
Merged
Conversation
`avocado vm update` was failing mid-download on real-world links:
Error: error decoding response body
Caused by: operation timed out
Two underlying causes:
1. `ClientBuilder::timeout(Duration::from_secs(30))` set a global
request timeout that fires across the body download too — 30s
is hopeless for a 458 MB artifact on any link slower than
~16 MB/s. Fix: switch to `connect_timeout` (still 30s — fail
fast on a stalled handshake) but leave the overall timeout
unset so large downloads can run to completion.
2. `Response::bytes()` buffered the entire body in memory before
writing — 458 MB of heap per artifact, plus the wasted RTT
before disk writes started. Fix: stream via
`Response::bytes_stream()` and write each chunk straight to
`std::fs::File`.
UX:
- Human mode now renders an indicatif progress bar per artifact:
[1/4] downloading avocado-image-rootfs-qemuarm64.erofs-lz4 (...)
===>... 65.0 MB/65.0 MB 10.2 MB/s ETA 0s
- JSON mode (--output json) emits structured NDJSON events
throttled to ~10 Hz so a polling consumer (e.g. avocado-desktop's
VMInstallController) can drive its own progress UI without
being flooded:
{"event":"download_started","file":"...","size":N,"index":i,"total":n}
{"event":"download_progress","file":"...","bytes":N,"total":N}
{"event":"download_completed","file":"...","bytes":N,"index":i,"total":n}
Verified locally end-to-end: clean cache + state, ran
`avocado vm update --yes` against the live v0.1.0 channel, all
four artifacts downloaded, sha256-verified, atomic-swapped into
~/.avocado/vm/install/. Sha hashing still happens after write
(unchanged from before — keeps the staging-dir verify path the
same).
User flow that hit this:
$ avocado vm update # downloads to ~/.avocado/vm/install/
$ avocado vm start
Error: --vm-source not given and AVOCADO_VM_DIR is unset…
After `avocado vm update` the artifacts ARE on disk in the canonical
location — `vm start` shouldn't refuse to find them. Resolution order
is now:
1. --vm-source flag
2. $AVOCADO_VM_DIR env var
3. ~/.avocado/vm/install/ (managed install, populated by vm update)
4. ~/.avocado/vm/artifact-dir (last `vm start --vm-source <dev>`
or `vm rebuild`, dev workflow)
5. Helpful error pointing at `avocado vm update`
VmPaths::default_vm_source() centralises the layered fallback. The
existing `vm rebuild` flow still writes the artifact-dir pointer for
backwards compat with the dev workflow, but end-users who only ever
`avocado vm update` + `avocado vm start` no longer need to think
about env vars.
main.rs help text + start.rs error message both updated to reflect
the new behaviour.
Pre-allocate a ProgressBar per artifact via MultiProgress (same as
connect upload at src/commands/connect/upload.rs:563-585), and reuse
the exact template:
" {msg} [{wide_bar:.cyan/blue}] {bytes}/{total_bytes} ({bytes_per_sec})"
with progress_chars "#>-". Bars finish_with_message("X (done)") so
the queue stays visible at 100% after each download — matches the
connect upload completion behaviour.
The earlier per-file style (green/black bar + ETA, drawn one at a
time after a "[1/4] downloading…" header) is replaced; users now see
all four artifacts queued from the start, filling sequentially. One
visual idiom across the CLI.
JSON mode unchanged — emits the same three download_* events
(started / progress / completed).
Adds `~/.avocado/vm/config.yaml` as a shared host/guest config file with a typed `VmConfig` reader and a `vm config get/set/unset/list` CLI surface. avocado-desktop reads/writes the same file so settings stay in sync across the two clients. First consumer is DNS: persisted `network.dns` (+ optional one-shot `--dns` override on `vm start`) is pushed into the guest via `resolvectl` once SSH comes up. Falls back to SLIRP's 10.0.2.3 when nothing is configured. Resolves the macOS-VPN case where scoped resolvers on the host are invisible to the slirp DNS proxy.
New `windows-check` PR job that runs `cargo check --target x86_64-pc-windows-msvc`. To make that build pass without functional regressions on unix targets: - `qga` (AF_UNIX socket client) and `qmp` (QEMU monitor over AF_UNIX) are now `#[cfg(unix)]` on the module declarations; their call sites in `lifecycle::stop` and `boot_sync::wait_for_guest_ready` get the same gating with a non-unix fallback path that waits for SSH only. - `libc::SIGTERM` / `libc::SIGKILL` are not exposed on Windows — replace the call-site uses in `lifecycle` and `forward` with local POSIX-value constants (`send_signal` itself is already `#[cfg(unix)]`). No behavior change on macOS/Linux.
Adds `utils::runtime_extension::RuntimeExtensionSpec::parse_entry` as the one place that knows how to read a runtime config's `extensions:` list. The list previously accepted only string entries; now it also accepts a single-key map whose value carries per-extension flags (initial flag: `enabled`). Every iterator that walks the list — `Config::extension_deps`, `Config::find_active_extensions`, runtime/build, runtime/deps, build, install — now funnels through the same parser, so any future per-extension knob (`merge_index`, `runs_on`, …) lands in one spot without silently dropping entries. `runtime build` propagates the `enabled: false` flag into the runtime manifest's `AVOCADO_EXT_DISABLED` env var so avocadoctl skips activation at refresh time.
…present Mirror the existing sysusers.d / ld.so.conf.d detection. When an extension ships config under `usr/(local/)lib/tmpfiles.d/` or `etc/tmpfiles.d/`, the generated release file gets `AVOCADO_ON_MERGE="systemd-tmpfiles --create"` so the extension's tmpfiles take effect after merge without a reboot.
`compute_file_hash` previously read the whole file into memory before feeding it to the hasher. Image artifacts can be tens of GB (`var.btrfs` alone is ~458 MB now and growing), so reading them fully into RAM is a footgun waiting to happen — and on a 16 GB Mac it can push the process into swap thrash before any sign of progress. Replace with a fixed-size 1 MiB-chunk streaming loop for both Sha256 and Blake3. No behavior change for callers; the resulting digest is identical.
Two remaining call sites in build.rs (collect_extension_dependencies' nested loop and the per-extension dep-resolution loop) and the runtime input-hash compute in stamps.rs were still treating each entry as a bare string. After 3a7c528 added support for `extensions: - foo: {enabled: false}` map syntax, those paths silently skipped any map-form entry, so disabled extensions could still pull deps via the nested traversal and the input hash didn't account for them.
Previously the macOS path delegated `vm start` / `vm stop` to Avocado.app via a synchronous JSON-line client, which made the CLI unusable on its own and stalled `vm stop` whenever the app's main actor was busy. Now the CLI spawns and signals qemu directly on every platform; Avocado.app (when installed) adopts the pid via its existing pidfile reconciler. The remaining IPC (`vm.notify.starting` / `.running` / `.stopping` / `.stopped`) is pure dashboard hinting: 100 ms timeouts, silent no-op when the desktop isn't reachable, and self-heals via the reconciler within ~2 s if a notification is dropped. `Client::connect_or_launch` and `delegate_start_to_app` are gone along with their wait loops. Also adds a second virtio-serial port (`avocado.control` -> control.sock) so Avocado.app's USBHostBridge / ControlPlane can talk to avocado-vm-agent without waiting for the app to spawn qemu itself.
avocado config show --detail emits a `detail` block alongside the existing narrow projection: per-runtime extension references (with defined/enabled flags and node_paths for cross-ref navigation), per-extension types/packages/services/used_by_runtimes, and an SDK image+packages summary. Default output is byte-stable so the desktop app's project-list scan keeps working unchanged; the new payload only appears when --detail is passed. Six unit tests cover map/list package shapes, broken runtime->extension refs, used_by inversion, and empty configs. The desktop app consumes this to render a config tree per project without having to parse avocado.yaml itself.
f96a00d to
e7d7a93
Compare
The previous code called `lock_file.save()` after clearing entries, which merges with the on-disk state and re-adds them. The desktop app's Unlock button hit this and silently no-op'd. Add a regression test.
`avocado vm start --memory-mib` / `--cpus` now make the flags optional,
resolve to `runtime.{cpus,memory_mib}` in the VM config (also written by
Avocado.app's settings UI), and fall back to DEFAULT_CPUS / DEFAULT_MEMORY_MIB.
When the user passes a flag, the value is written back to the config so the
next flag-less `vm start` (and the desktop app) converge on the same value.
`vm reset` / `vm update` / route-on-demand callers pass `None` so they pick
up the persisted settings instead of hardcoded 4096 MiB / 4 CPUs.
When `--output json` is set, deploy skips TUI rendering and emits NDJSON `task_registered` / `step` / `step_error` events for each phase (stamps, hash-collection, metadata-sign, deploy). Names mirror the human-facing labels so the desktop can render them directly.
e7d7a93 to
6d4c265
Compare
6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Bumps avocado-cli to 0.40.0. 17 commits since 0.39.0.
Three themes:
avocado vmpolish — streaming downloads with progress, smarter defaults, persistent cpu/memory/DNS config, and avm configsubcommand shared with Avocado.app. macOS lifecycle moved back into the CLI itself (Avocado.app adopts the pid via its existing reconciler) sovm start/vm stopno longer depend on the desktop being installed or responsive.runtime deploy --output jsonandconfig show --detailgive the desktop the structured project + deploy events it needs to render its UI without parsing prose.cargo check --target x86_64-pc-windows-msvcnow runs in CI, with#[cfg(unix)]gating on the AF_UNIX-bound modules.Plus a
clean --unlockcorrectness fix (regression test included), runtime-extension map syntax everywhere it should have been, anAVOCADO_ON_MERGE=systemd-tmpfiles --createextension hook, and streaming hashing for multi-GB image signing.avocado vmUXfeb9b1e,8cfc381) —vm updatewas timing out mid-download on the 458 MBvar.btrfsartifact:ClientBuilder::timeout(30s)applied to the body download, andResponse::bytes()buffered the entire body in RAM before disk writes started. Replaced withconnect_timeoutonly +bytes_stream()writing each chunk straight to disk. Human mode now renders an indicatifMultiProgressqueue matchingavocado connect upload's style (one cyan/blue bar per artifact, finishes with(done)); JSON mode emits throttleddownload_{started,progress,completed}NDJSON events at ~10 Hz.vm startdefaults (170324c) — afteravocado vm updatepopulates~/.avocado/vm/install/,vm startnow finds it without needing--vm-sourceorAVOCADO_VM_DIR. Resolution order: flag → env var →install/(managed) →artifact-dir(dev workflow) → helpful error pointing atvm update. End-users on the managed path no longer have to think about env vars.d06cf42) —vm start --memory-mib/--cpusare now optional and resolve toruntime.{cpus,memory_mib}in~/.avocado/vm/config.yaml(also written by Avocado.app's settings UI), with persistence: passing a flag writes it back so the next flag-lessvm startand the desktop app converge.vm reset/vm update/ route-on-demand callers passNoneto pick up persisted settings instead of hardcoded 4096 MiB / 4 CPUs.vm config get/set/unset/list+ persistent network config (7d41154) —~/.avocado/vm/config.yamlis the shared host/guest config file Avocado.app reads/writes. First consumer:network.dns(+ optional one-shot--dnsonvm start) pushed into the guest viaresolvectlonce SSH is up. Resolves the macOS-VPN case where scoped resolvers on the host are invisible to SLIRP's DNS proxy.avocado vmlifecycle7afc946) —vm start/vm stoppreviously delegated to Avocado.app over a synchronous JSON-line client, which made the CLI unusable when the app wasn't installed and stalledvm stopwhenever the app's main actor was busy. CLI now spawns and signals qemu directly on every platform; Avocado.app (when present) adopts the pid via its pidfile reconciler. Remaining IPC (vm.notify.{starting,running,stopping,stopped}) is pure dashboard hinting — 100 ms timeouts, silent no-op when unreachable, self-heals via the reconciler within ~2s. Also adds a second virtio-serial port (avocado.control→ control.sock) so Avocado.app's USBHostBridge / ControlPlane can talk to avocado-vm-agent without spawning qemu itself.Desktop integration (JSON output)
runtime deploy --output json(f96a00d) — skips TUI rendering and emits NDJSONtask_registered/step/step_errorevents per phase (stamps,hash-collection,metadata-sign,deploy). Names mirror the human labels so the desktop can render them directly.config show --detail(a9633a9) — adds adetailblock alongside the existing narrow projection: per-runtime extension references (withdefined/enabledflags + node_paths for cross-ref navigation), per-extension types/packages/services/used_by_runtimes, and an SDK image+packages summary. Default output is byte-stable so the desktop's project-list scan keeps working unchanged. Six unit tests cover map/list package shapes, broken cross-refs, used_by inversion, and empty configs.Cross-platform
7e43410) — newwindows-checkPR job runscargo check --target x86_64-pc-windows-msvc.qga(AF_UNIX qemu-guest-agent client) andqmp(QEMU monitor over AF_UNIX) become#[cfg(unix)]at the module declaration; call sites inlifecycle::stopandboot_sync::wait_for_guest_readyget matching gates with a non-unix fallback that waits for SSH only.libc::SIG{TERM,KILL}replaced with local POSIX-value constants at the two call sites that referenced them (thesend_signalfunction itself is already#[cfg(unix)]). No behavior change on macOS/Linux.Runtime extension map form
extensions: - foo: {enabled: false}syntax (3a7c528) — addsutils::runtime_extension::RuntimeExtensionSpec::parse_entryas the single source of truth. Every list-walker (Config::extension_deps,Config::find_active_extensions, runtime/build, runtime/deps, build, install) now funnels through it, so any future per-extension knob (merge_index,runs_on, …) lands in one spot.runtime buildpropagatesenabled: falseinto the manifest'sAVOCADO_EXT_DISABLEDso avocadoctl skips activation at refresh time.a97ba18) —collect_extension_dependencies(nested loop) and the runtime input-hash compute instamps.rswere still treating each entry as a bare string and silently skipping map-form entries. Fixed.Extension build
AVOCADO_ON_MERGE=systemd-tmpfiles --createfortmpfiles.d/(7519489) — mirrors the existing sysusers.d / ld.so.conf.d detection. Extensions shipping config underusr/(local/)lib/tmpfiles.d/oretc/tmpfiles.d/get the merge hook automatically.Reliability
509faca) —compute_file_hashpreviously read the whole file into memory before feeding it to Sha256/Blake3. Image artifacts can be tens of GB; on a 16 GB Mac that pushed the process into swap thrash. Replaced with a fixed 1 MiB-chunk streaming loop. Digest is identical.clean --unlockactually unlocks (0a141ed) — previous code calledlock_file.save()after clearing entries, which merges with on-disk state and re-adds them. The desktop app's Unlock button hit this and silently no-op'd. Switched tosave_replacing; regression test included.Maintenance
2936f8d—cargo fmtsweep on update.rs.fca9e79— docs link update.48680dd— version bump to 0.40.0.Test plan
cargo fmt --all -- --checkcleancargo clippy --all-targets --all-features -- -D warningscleancargo buildcleancargo test— all 9 test binaries green (993 + 1000 + 126 + 49 + 20 + 7 + 6 + 6 + 5 lib/integration tests pass)cargo audit— no advisories on the lockfilewindows-checkruns on this PR for the first time on a release branch — verifycargo check --target x86_64-pc-windows-msvccleanavocado vm updateend-to-end against the live channel renders the new progress bars, downloads stream to disk, atomic-swap completesavocado vm startfrom a clean~/.avocado/vm/install/(post-update) finds artifacts without--vm-source/AVOCADO_VM_DIRavocado vm config set network.dns 1.1.1.1, restart guest,resolvectl statusinside the guest shows the configured resolveravocado vm start --memory-mib 8192persists; nextavocado vm start(no flags) launches with 8192 MiBavocado runtime deploy --output json -r <runtime> -d <device-ip>emits the four task_registered events then step/success transitions in orderavocado config show --detail --output jsonreturns thedetailblock with runtime/extension cross-refs populatedavocado clean --unlock, verify on-disk lockfile no longer contains the cleared entry