From 640e46097f3ffbfb4a38a6cb26264725b7392691 Mon Sep 17 00:00:00 2001 From: Juliano Martinez Date: Wed, 1 Jul 2026 13:27:05 +0200 Subject: [PATCH 1/6] feat(packages): installed-packages fact, source-namespaced record arrays MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds the `packages` fact (ADR-0014): packages. = an array of {name, version, ...} records, one namespace per package database. Sources are never merged, a source is omitted when its database is absent, and the whole probe is a gated single-output resolver so `--disable packages` skips it. Readers (one cheap read/query each, no spawn-per-package, no network): - Linux: dpkg (install/hold state), rpm (epoch query, gpg-pubkey filtered, DB-presence-gated), pacman, apk, snap, flatpak, nix (NixOS system profile). - BSD/illumos: pkg (FreeBSD+DragonFly, absolute /usr/local/sbin/pkg), openbsd_pkg, pkgsrc (PKG_DBDIR discovery), ips. - macOS: receipts (primary), apps (secondary, never merged; plutil -p batch), homebrew (auto-detected). - Windows: registry (both HKLM hives, product_code+architecture), appx. Per-platform readers live in packages_{bsd,mac,win,extra}.go with no GOOS suffixes or build constraints (ADR-0010), so the pure parse functions stay cross-platform testable. Commands outside the engine's trusted PATH use absolute paths (pkgng, nix-store). Schema, supported-facts pages, README, man page, and CHANGELOG updated. Validated on the nlab fleet / local darwin, counts vs native tools: dpkg 324, rpm 386, pacman 183, apk 200, pkg freebsd 2 / dragonfly 48, openbsd_pkg 18, pkgsrc 19, ips 418, nix 70, receipts 10 / apps 61 / homebrew 88 — all exact. Windows registry/appx are format- and empty-case-validated (bare guest); snap/flatpak are format-only (no populated guest). Reviewed with OCR + Codex + self-adversarial; all findings fixed (dpkg held packages, homebrew cask-only, apps/registry empty-version invariants, plutil multi-line value bug, nix digit-leading names, plutil argv chunking, total-order sort, resolver-gating test). --- CHANGELOG.md | 11 + README.md | 2 +- docs/schema/facts.yaml | 81 ++++++ docs/supported-facts/README.md | 16 +- docs/supported-facts/darwin.md | 5 +- docs/supported-facts/dragonfly.md | 3 +- docs/supported-facts/freebsd.md | 3 +- docs/supported-facts/illumos.md | 3 +- docs/supported-facts/linux.md | 9 +- docs/supported-facts/netbsd.md | 3 +- docs/supported-facts/openbsd.md | 3 +- docs/supported-facts/windows.md | 4 +- internal/engine/core.go | 1 + internal/engine/core_gating_test.go | 2 +- internal/engine/packages.go | 245 ++++++++++++++++ internal/engine/packages_bsd.go | 160 +++++++++++ internal/engine/packages_bsd_test.go | 251 +++++++++++++++++ internal/engine/packages_extra.go | 171 ++++++++++++ internal/engine/packages_extra_test.go | 179 ++++++++++++ internal/engine/packages_mac.go | 225 +++++++++++++++ internal/engine/packages_mac_test.go | 295 ++++++++++++++++++++ internal/engine/packages_test.go | 166 +++++++++++ internal/engine/packages_win.go | 101 +++++++ internal/engine/packages_win_test.go | 160 +++++++++++ man/man8/facts.8 | 2 +- openspec/changes/add-packages-fact/tasks.md | 38 +-- 26 files changed, 2101 insertions(+), 38 deletions(-) create mode 100644 internal/engine/packages.go create mode 100644 internal/engine/packages_bsd.go create mode 100644 internal/engine/packages_bsd_test.go create mode 100644 internal/engine/packages_extra.go create mode 100644 internal/engine/packages_extra_test.go create mode 100644 internal/engine/packages_mac.go create mode 100644 internal/engine/packages_mac_test.go create mode 100644 internal/engine/packages_test.go create mode 100644 internal/engine/packages_win.go create mode 100644 internal/engine/packages_win_test.go diff --git a/CHANGELOG.md b/CHANGELOG.md index 2e22a168..b58c7fef 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,17 @@ ### Added +- New `packages` fact: the installed packages on the host, namespaced by the + package database they come from — `packages.dpkg`, `packages.rpm`, + `packages.pacman`, `packages.apk`, `packages.snap`, `packages.flatpak`, + `packages.nix` (Linux); `packages.pkg` (FreeBSD/DragonFly), + `packages.openbsd_pkg`, `packages.pkgsrc`, `packages.ips`; `packages.receipts`, + `packages.apps`, `packages.homebrew` (macOS); and `packages.registry`, + `packages.appx` (Windows). Each source is an array of `{name, version, ...}` + records with per-source identity fields (architecture, product_code, bundle_id, + and so on); sources are never merged, and a source is omitted when its database + is absent. Scope is system package databases (language/runtime managers are out + of scope). The whole probe is skippable with `--disable packages`. - Fact disabling is now a first-class, facts-native control. Disable any fact or fact group with `--disable a,b`, the `FACTS_DISABLE=a,b` environment variable, or the `facts.conf` `disable` key; the Facter `blocklist` key keeps diff --git a/README.md b/README.md index 06d330ee..b0f74cb7 100644 --- a/README.md +++ b/README.md @@ -18,7 +18,7 @@
-Facts is a Go port of [Puppet Facter](https://github.com/puppetlabs/facter). It discovers facts about the system it runs on — hardware, networking, OS, cloud metadata — and serves them two ways: as an embeddable Go library, and as the `facts` CLI. It reads your existing facter configuration, fact directories, and `FACTER_*` environment facts as the compatibility tier of its input surface, so facter-configured hosts keep working unchanged. +Facts is a Go port of [Puppet Facter](https://github.com/puppetlabs/facter). It discovers facts about the system it runs on — hardware, networking, OS, installed packages, cloud metadata — and serves them two ways: as an embeddable Go library, and as the `facts` CLI. It reads your existing facter configuration, fact directories, and `FACTER_*` environment facts as the compatibility tier of its input surface, so facter-configured hosts keep working unchanged. Ruby Facter compatibility is promised exactly where it matters and nowhere else: at the CLI process boundary (the output contract) and for operator-supplied fact sources — external facts and `facter.conf` (the input contract, now read under facts-native names first; see [the input-compatibility reference](docs/FACTER_CONF_COMPATIBILITY.md)). Ruby DSL fact files are not read; rewrite them as external facts ([migration guide](docs/CUSTOM_FACT_MIGRATION.md)). The Go API itself is just Go. diff --git a/docs/schema/facts.yaml b/docs/schema/facts.yaml index 76362b39..ce96c70f 100644 --- a/docs/schema/facts.yaml +++ b/docs/schema/facts.yaml @@ -795,6 +795,87 @@ os.windows.system32: description: The native system32 directory, sysnative-aware for 32-bit processes. platforms: [windows] +packages.apk: + type: array + description: Installed Alpine apk packages as {name, version, architecture} records, from /lib/apk/db/installed. + platforms: [linux] + conditional: true +packages.appx: + type: array + description: System-provisioned Windows AppX packages as {name, version, architecture} records. + platforms: [windows] + conditional: true +packages.apps: + type: array + description: macOS application bundles as {name, version, bundle_id, path} records; secondary to receipts and never merged with it. + platforms: [darwin] + conditional: true +packages.dpkg: + type: array + description: Installed dpkg packages as {name, version, architecture} records (install ok installed only; multiarch siblings kept), from /var/lib/dpkg/status. + platforms: [linux] + conditional: true +packages.flatpak: + type: array + description: System-installed Flatpak applications as {name, version, architecture} records. + platforms: [linux] + conditional: true +packages.homebrew: + type: array + description: Homebrew formulae and casks as {name, version, type} records, when a Cellar prefix exists. + platforms: [darwin] + conditional: true +packages.ips: + type: array + description: Installed illumos IPS packages as {name, version} records, from the local image (no network refresh). + platforms: [illumos] + conditional: true +packages.nix: + type: array + description: Installed Nix packages as {name, version} records — the NixOS system profile set (the references of /run/current-system/sw), never the whole /nix/store. + platforms: [linux] + conditional: true +packages.openbsd_pkg: + type: array + description: Installed OpenBSD packages as {name, version} records, from /var/db/pkg. + platforms: [openbsd] + conditional: true +packages.pacman: + type: array + description: Installed pacman packages as {name, version, architecture} records, from /var/lib/pacman/local. + platforms: [linux] + conditional: true +packages.pkg: + type: array + description: Installed pkgng packages as {name, version, architecture} records (shared by FreeBSD and DragonFly). + platforms: [freebsd, dragonfly] + conditional: true +packages.pkgsrc: + type: array + description: Installed pkgsrc packages as {name, version} records, from the discovered PKG_DBDIR. + platforms: [netbsd] + conditional: true +packages.receipts: + type: array + description: macOS installer(8)/PackageKit .pkg receipts as {name, version} records, from /var/db/receipts; the primary macOS source. + platforms: [darwin] + conditional: true +packages.registry: + type: array + description: Windows uninstall entries from both HKLM hives as {name, version, product_code, architecture} records. + platforms: [windows] + conditional: true +packages.rpm: + type: array + description: Installed rpm packages as {name, version, architecture} records (epoch preserved, gpg-pubkey filtered). + platforms: [linux] + conditional: true +packages.snap: + type: array + description: Installed snap packages as {name, version} records. + platforms: [linux] + conditional: true + partitions.*: type: map description: A disk partition (or device-mapper/loop device), keyed by device path. diff --git a/docs/supported-facts/README.md b/docs/supported-facts/README.md index 201f855e..7127dacf 100644 --- a/docs/supported-facts/README.md +++ b/docs/supported-facts/README.md @@ -6,12 +6,12 @@ These pages are generated from [`docs/schema/facts.yaml`](../schema/facts.yaml). | Platform | Supported facts | | --- | ---: | -| [Linux](linux.md) | 175 | -| [macOS / Darwin](darwin.md) | 107 | -| [Windows](windows.md) | 101 | -| [FreeBSD](freebsd.md) | 131 | -| [OpenBSD](openbsd.md) | 113 | -| [NetBSD](netbsd.md) | 117 | -| [DragonFly BSD](dragonfly.md) | 115 | -| [illumos](illumos.md) | 114 | +| [Linux](linux.md) | 182 | +| [macOS / Darwin](darwin.md) | 110 | +| [Windows](windows.md) | 103 | +| [FreeBSD](freebsd.md) | 132 | +| [OpenBSD](openbsd.md) | 114 | +| [NetBSD](netbsd.md) | 118 | +| [DragonFly BSD](dragonfly.md) | 116 | +| [illumos](illumos.md) | 115 | | [Plan 9](plan9.md) | 29 | diff --git a/docs/supported-facts/darwin.md b/docs/supported-facts/darwin.md index 5fb4cd1c..5567a2b3 100644 --- a/docs/supported-facts/darwin.md +++ b/docs/supported-facts/darwin.md @@ -63,7 +63,7 @@ $ facts --json ## Fact Contract -107 schema entries include `darwin`. +110 schema entries include `darwin`. | Fact | Type | Conditional | Description | | --- | --- | --- | --- | @@ -153,6 +153,9 @@ $ facts --json | `os.release.full` | `string` | no | The full release number of the operating system. | | `os.release.major` | `string` | no | The major release number of the operating system. | | `os.release.minor` | `string` | yes | The minor release number of the operating system, when it has one. | +| `packages.apps` | `array` | yes | macOS application bundles as {name, version, bundle_id, path} records; secondary to receipts and never merged with it. | +| `packages.homebrew` | `array` | yes | Homebrew formulae and casks as {name, version, type} records, when a Cellar prefix exists. | +| `packages.receipts` | `array` | yes | macOS installer(8)/PackageKit .pkg receipts as {name, version} records, from /var/db/receipts; the primary macOS source. | | `path` | `array` | no | The PATH environment entries of the Facts process, in lookup order. | | `processors.cores` | `integer` | no | The number of cores per processor socket. | | `processors.count` | `integer` | yes | The number of logical processors. | diff --git a/docs/supported-facts/dragonfly.md b/docs/supported-facts/dragonfly.md index 372bed6b..b2668d72 100644 --- a/docs/supported-facts/dragonfly.md +++ b/docs/supported-facts/dragonfly.md @@ -67,7 +67,7 @@ $ facts --json ## Fact Contract -115 schema entries include `dragonfly`. +116 schema entries include `dragonfly`. | Fact | Type | Conditional | Description | | --- | --- | --- | --- | @@ -161,6 +161,7 @@ $ facts --json | `os.release.full` | `string` | no | The full release number of the operating system. | | `os.release.major` | `string` | no | The major release number of the operating system. | | `os.release.minor` | `string` | yes | The minor release number of the operating system, when it has one. | +| `packages.pkg` | `array` | yes | Installed pkgng packages as {name, version, architecture} records (shared by FreeBSD and DragonFly). | | `partitions.*` | `map` | yes | A disk partition (or device-mapper/loop device), keyed by device path. | | `partitions.*.filesystem` | `string` | yes | The filesystem type of the partition. | | `partitions.*.mount` | `string` | yes | The path the partition is mounted on. | diff --git a/docs/supported-facts/freebsd.md b/docs/supported-facts/freebsd.md index 3e9f6942..04358d28 100644 --- a/docs/supported-facts/freebsd.md +++ b/docs/supported-facts/freebsd.md @@ -64,7 +64,7 @@ $ facts --json ## Fact Contract -131 schema entries include `freebsd`. +132 schema entries include `freebsd`. | Fact | Type | Conditional | Description | | --- | --- | --- | --- | @@ -166,6 +166,7 @@ $ facts --json | `os.release.major` | `string` | no | The major release number of the operating system. | | `os.release.minor` | `string` | yes | The minor release number of the operating system, when it has one. | | `os.release.patchlevel` | `string` | yes | The FreeBSD patch level of the installed userland. | +| `packages.pkg` | `array` | yes | Installed pkgng packages as {name, version, architecture} records (shared by FreeBSD and DragonFly). | | `partitions.*` | `map` | yes | A disk partition (or device-mapper/loop device), keyed by device path. | | `partitions.*.filesystem` | `string` | yes | The filesystem type of the partition. | | `partitions.*.mount` | `string` | yes | The path the partition is mounted on. | diff --git a/docs/supported-facts/illumos.md b/docs/supported-facts/illumos.md index 31879af5..47023efe 100644 --- a/docs/supported-facts/illumos.md +++ b/docs/supported-facts/illumos.md @@ -89,7 +89,7 @@ $ facts --json ## Fact Contract -114 schema entries include `illumos`. +115 schema entries include `illumos`. | Fact | Type | Conditional | Description | | --- | --- | --- | --- | @@ -178,6 +178,7 @@ $ facts --json | `os.name` | `string` | no | The operating system name, such as Ubuntu, Darwin, windows, or Plan 9. | | `os.release.full` | `string` | no | The full release number of the operating system. | | `os.release.major` | `string` | no | The major release number of the operating system. | +| `packages.ips` | `array` | yes | Installed illumos IPS packages as {name, version} records, from the local image (no network refresh). | | `partitions.*` | `map` | yes | A disk partition (or device-mapper/loop device), keyed by device path. | | `partitions.*.filesystem` | `string` | yes | The filesystem type of the partition. | | `partitions.*.size` | `string` | yes | The display size of the partition, such as 1.00 GiB. | diff --git a/docs/supported-facts/linux.md b/docs/supported-facts/linux.md index 0f9c4039..78d31449 100644 --- a/docs/supported-facts/linux.md +++ b/docs/supported-facts/linux.md @@ -70,7 +70,7 @@ $ facts --json ## Fact Contract -175 schema entries include `linux`. +182 schema entries include `linux`. | Fact | Type | Conditional | Description | | --- | --- | --- | --- | @@ -217,6 +217,13 @@ $ facts --json | `os.selinux.enabled` | `boolean` | no | Whether SELinux is enabled. | | `os.selinux.enforced` | `boolean` | yes | Whether SELinux is enforcing, when SELinux is enabled. | | `os.selinux.policy_version` | `string` | yes | The loaded SELinux policy version, when SELinux is enabled. | +| `packages.apk` | `array` | yes | Installed Alpine apk packages as {name, version, architecture} records, from /lib/apk/db/installed. | +| `packages.dpkg` | `array` | yes | Installed dpkg packages as {name, version, architecture} records (install ok installed only; multiarch siblings kept), from /var/lib/dpkg/status. | +| `packages.flatpak` | `array` | yes | System-installed Flatpak applications as {name, version, architecture} records. | +| `packages.nix` | `array` | yes | Installed Nix packages as {name, version} records — the NixOS system profile set (the references of /run/current-system/sw), never the whole /nix/store. | +| `packages.pacman` | `array` | yes | Installed pacman packages as {name, version, architecture} records, from /var/lib/pacman/local. | +| `packages.rpm` | `array` | yes | Installed rpm packages as {name, version, architecture} records (epoch preserved, gpg-pubkey filtered). | +| `packages.snap` | `array` | yes | Installed snap packages as {name, version} records. | | `partitions.*` | `map` | yes | A disk partition (or device-mapper/loop device), keyed by device path. | | `partitions.*.backing_file` | `string` | yes | The file backing the loop device. | | `partitions.*.filesystem` | `string` | yes | The filesystem type of the partition. | diff --git a/docs/supported-facts/netbsd.md b/docs/supported-facts/netbsd.md index 6467a773..65b317a0 100644 --- a/docs/supported-facts/netbsd.md +++ b/docs/supported-facts/netbsd.md @@ -85,7 +85,7 @@ $ facts --json ## Fact Contract -117 schema entries include `netbsd`. +118 schema entries include `netbsd`. | Fact | Type | Conditional | Description | | --- | --- | --- | --- | @@ -175,6 +175,7 @@ $ facts --json | `os.release.full` | `string` | no | The full release number of the operating system. | | `os.release.major` | `string` | no | The major release number of the operating system. | | `os.release.minor` | `string` | yes | The minor release number of the operating system, when it has one. | +| `packages.pkgsrc` | `array` | yes | Installed pkgsrc packages as {name, version} records, from the discovered PKG_DBDIR. | | `partitions.*` | `map` | yes | A disk partition (or device-mapper/loop device), keyed by device path. | | `partitions.*.filesystem` | `string` | yes | The filesystem type of the partition. | | `partitions.*.mount` | `string` | yes | The path the partition is mounted on. | diff --git a/docs/supported-facts/openbsd.md b/docs/supported-facts/openbsd.md index 6c288dcb..60b5a091 100644 --- a/docs/supported-facts/openbsd.md +++ b/docs/supported-facts/openbsd.md @@ -64,7 +64,7 @@ $ facts --json ## Fact Contract -113 schema entries include `openbsd`. +114 schema entries include `openbsd`. | Fact | Type | Conditional | Description | | --- | --- | --- | --- | @@ -156,6 +156,7 @@ $ facts --json | `os.release.full` | `string` | no | The full release number of the operating system. | | `os.release.major` | `string` | no | The major release number of the operating system. | | `os.release.minor` | `string` | yes | The minor release number of the operating system, when it has one. | +| `packages.openbsd_pkg` | `array` | yes | Installed OpenBSD packages as {name, version} records, from /var/db/pkg. | | `partitions.*` | `map` | yes | A disk partition (or device-mapper/loop device), keyed by device path. | | `partitions.*.filesystem` | `string` | yes | The filesystem type of the partition. | | `partitions.*.mount` | `string` | yes | The path the partition is mounted on. | diff --git a/docs/supported-facts/windows.md b/docs/supported-facts/windows.md index d7740ef1..9ddf22f1 100644 --- a/docs/supported-facts/windows.md +++ b/docs/supported-facts/windows.md @@ -48,7 +48,7 @@ $ facts --json ## Fact Contract -101 schema entries include `windows`. +103 schema entries include `windows`. | Fact | Type | Conditional | Description | | --- | --- | --- | --- | @@ -134,6 +134,8 @@ $ facts --json | `os.windows.product_name` | `string` | no | The Windows product name, such as Windows Server 2022 Datacenter. | | `os.windows.release_id` | `string` | no | The Windows release identifier (the display version when available). | | `os.windows.system32` | `string` | no | The native system32 directory, sysnative-aware for 32-bit processes. | +| `packages.appx` | `array` | yes | System-provisioned Windows AppX packages as {name, version, architecture} records. | +| `packages.registry` | `array` | yes | Windows uninstall entries from both HKLM hives as {name, version, product_code, architecture} records. | | `path` | `array` | no | The PATH environment entries of the Facts process, in lookup order. | | `processors.cores` | `integer` | no | The number of cores per processor socket. | | `processors.count` | `integer` | yes | The number of logical processors. | diff --git a/internal/engine/core.go b/internal/engine/core.go index 4ed57ebc..58d2249e 100644 --- a/internal/engine/core.go +++ b/internal/engine/core.go @@ -95,6 +95,7 @@ func buildCoreFacts(s *Session, disabled map[string]bool) []ResolvedFact { gate("timezone", timezoneCoreFacts) gate("augeas", augeasCoreFacts) gate("xen", xenCoreFacts) + gate("packages", packagesCoreFacts) facts = append(facts, currentLinuxHypervisorFacts(s)...) facts = append(facts, currentWindowsHypervisorFacts(s)...) facts = append(facts, azureFacts(s.Context(), newAzureClient(azureMetadataBaseURL, nil), virtualization)...) diff --git a/internal/engine/core_gating_test.go b/internal/engine/core_gating_test.go index 3109d0f1..800e3274 100644 --- a/internal/engine/core_gating_test.go +++ b/internal/engine/core_gating_test.go @@ -27,7 +27,7 @@ func hasRoot(facts []ResolvedFact, root string) bool { // top-level fact name that gates it (ADR-0015). var gatedSingleOutputCategories = []string{ "networking", "processors", "memory", "ssh", - "timezone", "fips_enabled", "augeas", "xen", + "timezone", "fips_enabled", "augeas", "xen", "packages", } func TestBuildCoreFacts_resolutionGatesSingleOutputCategories(t *testing.T) { diff --git a/internal/engine/packages.go b/internal/engine/packages.go new file mode 100644 index 00000000..0578295c --- /dev/null +++ b/internal/engine/packages.go @@ -0,0 +1,245 @@ +package engine + +import ( + "os" + "sort" + "strings" +) + +// packagesCoreFacts resolves the packages fact: for each package database +// present on the host, a source-namespaced array of {name, version, ...} records +// (ADR-0014). Sources are never merged, and a source is omitted when its +// database is absent or lists no installed packages. Registered as a +// single-output resolver so a disable of "packages" gates the whole probe +// (ADR-0015). +func packagesCoreFacts(s *Session) []ResolvedFact { + sources := map[string]any{} + add := func(name string, records []any) { + if len(records) > 0 { + sources[name] = records + } + } + + switch s.goos() { + case "linux": + add("dpkg", dpkgPackages(s.readFile)) + if rpmDatabasePresent(s.stat) { + add("rpm", rpmPackages(s.commandOutput)) + } + add("pacman", pacmanPackages(s.glob, s.readFile)) + add("apk", apkPackages(s.readFile)) + if snapdPresent(s.stat) { + add("snap", snapPackages(s.commandOutput)) + } + if flatpakPresent(s.stat) { + add("flatpak", flatpakPackages(s.commandOutput)) + } + if nixProfilePresent(s.stat) { + add("nix", nixPackages(s.commandOutput)) + } + case "freebsd", "dragonfly": + add("pkg", pkgngPackages(s.commandOutput)) + case "openbsd": + add("openbsd_pkg", openbsdPackages(s.readDir, s.readFile)) + case "netbsd": + add("pkgsrc", pkgsrcPackages(s.readDir)) + case "illumos": + add("ips", ipsPackages(s.commandOutput)) + case "darwin": + add("receipts", receiptsPackages(s.glob, s.commandOutput)) + add("apps", appsPackages(s.glob, s.commandOutput)) + add("homebrew", homebrewPackages(s.glob)) + case "windows": + add("registry", registryPackages(s.commandOutput)) + add("appx", appxPackages(s.commandOutput)) + } + + if len(sources) == 0 { + return nil + } + return []ResolvedFact{{Name: "packages", Value: sources}} +} + +// packageRecord builds a record with the always-present name/version plus any +// non-empty identity fields supplied as key/value pairs. +func packageRecord(name, version string, kv ...string) map[string]any { + record := map[string]any{"name": name, "version": version} + for i := 0; i+1 < len(kv); i += 2 { + if kv[i+1] != "" { + record[kv[i]] = kv[i+1] + } + } + return record +} + +// sortPackages orders records deterministically by name, then architecture, then +// version, so multiarch/multiversion siblings keep a stable order across runs. +func sortPackages(records []any) { + sort.SliceStable(records, func(i, j int) bool { + a, b := records[i].(map[string]any), records[j].(map[string]any) + // name/architecture/version first, then the remaining identity fields, + // so siblings that share a name+arch+version still order deterministically + // regardless of the reader's append order. + for _, key := range []string{"name", "architecture", "version", "type", "branch", "product_code", "bundle_id", "store_path", "path"} { + if av, bv := packageField(a, key), packageField(b, key); av != bv { + return av < bv + } + } + return false + }) +} + +func packageField(record map[string]any, key string) string { + value, _ := record[key].(string) + return value +} + +// dpkgPackages parses /var/lib/dpkg/status, keeping only "install ok installed" +// entries. Multiarch siblings (same name, different Architecture) are kept. +func dpkgPackages(readFile fileReader) []any { + data, err := readFile("/var/lib/dpkg/status") + if err != nil { + return nil + } + var records []any + for _, stanza := range strings.Split(string(data), "\n\n") { + var name, version, arch, status string + for line := range strings.Lines(stanza) { + key, value, ok := strings.Cut(strings.TrimRight(line, "\n"), ": ") + if !ok { + continue + } + switch key { + case "Package": + name = value + case "Version": + version = value + case "Architecture": + arch = value + case "Status": + status = value + } + } + if name == "" || version == "" || !dpkgInstalled(status) { + continue + } + records = append(records, packageRecord(name, version, "architecture", arch)) + } + sortPackages(records) + return records +} + +// dpkgInstalled reports whether a dpkg Status line's current-state component +// (the third field of " ok ") is "installed", so held packages +// ("hold ok installed") are kept while removed/config-files entries are dropped. +func dpkgInstalled(status string) bool { + fields := strings.Fields(status) + return len(fields) == 3 && fields[2] == "installed" +} + +// rpmPackages runs one epoch-bearing rpm query (a bare `rpm -qa` omits the epoch +// that separates otherwise-identical install-only kernels). The (none) epoch is +// stripped; gpg-pubkey pseudo-packages are filtered. +func rpmPackages(run commandRunner) []any { + out := run("rpm", "-qa", "--qf", "%{NAME}|%{EPOCH}:%{VERSION}-%{RELEASE}|%{ARCH}\n") + var records []any + for line := range strings.Lines(out) { + fields := strings.SplitN(strings.TrimSpace(line), "|", 3) + if len(fields) != 3 { + continue + } + name, version, arch := fields[0], strings.TrimPrefix(fields[1], "(none):"), fields[2] + if name == "" || name == "gpg-pubkey" { + continue + } + if arch == "(none)" { + arch = "" + } + records = append(records, packageRecord(name, version, "architecture", arch)) + } + sortPackages(records) + return records +} + +// rpmDatabasePresent reports whether an rpm database directory exists, so the rpm +// query is not spawned on dpkg/apk/pacman-only hosts. +func rpmDatabasePresent(stat func(string) (os.FileInfo, error)) bool { + for _, dir := range []string{"/var/lib/rpm", "/usr/lib/sysimage/rpm"} { + if info, err := stat(dir); err == nil && info.IsDir() { + return true + } + } + return false +} + +// apkPackages parses /lib/apk/db/installed (blank-line-separated stanzas with +// P:/V:/A: fields). +func apkPackages(readFile fileReader) []any { + data, err := readFile("/lib/apk/db/installed") + if err != nil { + return nil + } + var records []any + for _, stanza := range strings.Split(string(data), "\n\n") { + var name, version, arch string + for line := range strings.Lines(stanza) { + key, value, ok := strings.Cut(strings.TrimRight(line, "\n"), ":") + if !ok { + continue + } + switch key { + case "P": + name = value + case "V": + version = value + case "A": + arch = value + } + } + if name == "" || version == "" { + continue + } + records = append(records, packageRecord(name, version, "architecture", arch)) + } + sortPackages(records) + return records +} + +// pacmanPackages reads each /var/lib/pacman/local/*/desc entry. +func pacmanPackages(glob pathGlobber, readFile fileReader) []any { + paths, err := glob("/var/lib/pacman/local/*/desc") + if err != nil { + return nil + } + var records []any + for _, path := range paths { + data, err := readFile(path) + if err != nil { + continue + } + name, version, arch := parsePacmanDesc(string(data)) + if name == "" || version == "" { + continue + } + records = append(records, packageRecord(name, version, "architecture", arch)) + } + sortPackages(records) + return records +} + +// parsePacmanDesc reads the %NAME%/%VERSION%/%ARCH% sections whose value is the +// line following the section header. +func parsePacmanDesc(input string) (name, version, arch string) { + lines := strings.Split(input, "\n") + for i := 0; i+1 < len(lines); i++ { + switch strings.TrimSpace(lines[i]) { + case "%NAME%": + name = strings.TrimSpace(lines[i+1]) + case "%VERSION%": + version = strings.TrimSpace(lines[i+1]) + case "%ARCH%": + arch = strings.TrimSpace(lines[i+1]) + } + } + return name, version, arch +} diff --git a/internal/engine/packages_bsd.go b/internal/engine/packages_bsd.go new file mode 100644 index 00000000..1c603d6d --- /dev/null +++ b/internal/engine/packages_bsd.go @@ -0,0 +1,160 @@ +package engine + +import ( + "os" + "path/filepath" + "strings" +) + +// pkgngPackages queries the FreeBSD/DragonFly pkgng database with a single cheap +// `pkg query`, capturing name, version, and the package ABI (%q), which encodes +// the target architecture (e.g. FreeBSD:14:amd64, dragonfly:6.4:x86:64, or a +// FreeBSD:14:* wildcard for architecture-independent packages). +func pkgngPackages(run commandRunner) []any { + // pkgng lives in the ports prefix, which is outside the engine's trusted + // command PATH (/usr/sbin:/usr/bin:/sbin:/bin); call it by absolute path, + // the same way the DMI probe reaches /usr/local/sbin/dmidecode. DragonFly + // has no /usr/sbin/pkg bootstrap stub, so only the ports path is portable. + out := run("/usr/local/sbin/pkg", "query", "-a", "%n|%v|%q") + var records []any + for line := range strings.Lines(out) { + fields := strings.SplitN(strings.TrimSpace(line), "|", 3) + if len(fields) != 3 { + continue + } + name, version, abi := fields[0], fields[1], fields[2] + if name == "" || version == "" { + continue + } + records = append(records, packageRecord(name, version, "architecture", abi)) + } + sortPackages(records) + return records +} + +// openbsdPackages enumerates the /var/db/pkg/-[-flavor] +// subdirectories that record every installed OpenBSD package, deriving the stem +// and version from the directory name and the architecture from each package's +// +CONTENTS @arch annotation. +func openbsdPackages(readDir func(string) ([]os.DirEntry, error), readFile fileReader) []any { + entries, err := readDir("/var/db/pkg") + if err != nil { + return nil + } + var records []any + for _, entry := range entries { + if !entry.IsDir() { + continue + } + name, version := splitBSDPackageName(entry.Name()) + if name == "" || version == "" { + continue + } + arch := bsdContentsArch(readFile, filepath.Join("/var/db/pkg", entry.Name(), "+CONTENTS")) + records = append(records, packageRecord(name, version, "architecture", arch)) + } + sortPackages(records) + return records +} + +// pkgsrcPackages enumerates the installed NetBSD pkgsrc packages recorded as +// - subdirectories of PKG_DBDIR. PKG_DBDIR is not hardcoded to a +// single path: the standard candidates are probed in order (the pkgsrc default +// /usr/pkg/pkgdb, then the legacy /var/db/pkg), using the first that lists any +// package entries. +func pkgsrcPackages(readDir func(string) ([]os.DirEntry, error)) []any { + for _, dbdir := range []string{"/usr/pkg/pkgdb", "/var/db/pkg"} { + entries, err := readDir(dbdir) + if err != nil { + continue + } + var records []any + for _, entry := range entries { + if !entry.IsDir() { + continue + } + name, version := splitBSDPackageName(entry.Name()) + if name == "" || version == "" { + continue + } + records = append(records, packageRecord(name, version)) + } + if len(records) > 0 { + sortPackages(records) + return records + } + } + return nil +} + +// ipsPackages lists the packages installed in the local illumos IPS image with +// `pkg list -H` (no header row, no network refresh). Columns are +// NAME [ (PUBLISHER) ] VERSION IFO, where the parenthesised publisher only +// appears for packages from a non-preferred publisher. The FMRI stem is the +// name, the branch-bearing version column is the version, and the trailing IFO +// flags are ignored. +func ipsPackages(run commandRunner) []any { + out := run("pkg", "list", "-H") + var records []any + for line := range strings.Lines(out) { + fields := strings.Fields(line) + if len(fields) < 2 { + continue + } + name, rest := fields[0], fields[1:] + if strings.HasPrefix(rest[0], "(") { + rest = rest[1:] + } + if len(rest) == 0 { + continue + } + version := rest[0] + if name == "" || version == "" { + continue + } + records = append(records, packageRecord(name, version)) + } + sortPackages(records) + return records +} + +// splitBSDPackageName splits an OpenBSD/pkgsrc package directory name into its +// stem and version. Both encode the version as the last hyphen-separated +// component (following a non-empty stem) whose token begins with a digit; any +// trailing hyphen-separated tokens are flavors and are discarded. Choosing the +// last such component keeps names whose final stem token is numeric intact +// (e.g. gcc-11-11.2.0 -> gcc-11 / 11.2.0), while still handling flavored names +// (vim-9.0.2035-no_x11 -> vim / 9.0.2035). Returns empty strings when no version +// component is present (e.g. the pkgdb.byfile.db index file). +func splitBSDPackageName(dir string) (name, version string) { + parts := strings.Split(dir, "-") + versionIdx := -1 + for i := 1; i < len(parts); i++ { + if token := parts[i]; token != "" && token[0] >= '0' && token[0] <= '9' { + versionIdx = i + } + } + if versionIdx < 0 { + return "", "" + } + return strings.Join(parts[:versionIdx], "-"), parts[versionIdx] +} + +// bsdContentsArch returns the architecture recorded in an OpenBSD +CONTENTS +// packing list (the @arch annotation). The arch-independent "*" marker and a +// missing file both yield an empty string. +func bsdContentsArch(readFile fileReader, path string) string { + data, err := readFile(path) + if err != nil { + return "" + } + for line := range strings.Lines(string(data)) { + if rest, ok := strings.CutPrefix(strings.TrimRight(line, "\n"), "@arch "); ok { + if arch := strings.TrimSpace(rest); arch != "*" { + return arch + } + return "" + } + } + return "" +} diff --git a/internal/engine/packages_bsd_test.go b/internal/engine/packages_bsd_test.go new file mode 100644 index 00000000..8f628796 --- /dev/null +++ b/internal/engine/packages_bsd_test.go @@ -0,0 +1,251 @@ +package engine + +import ( + "os" + "reflect" + "testing" +) + +// pkgngQueryFixtures are real `pkg query -a '%n|%v|%q'` outputs captured from the +// nlab FreeBSD and DragonFly guests. They exercise PORTEPOCH commas in versions +// (flac|1.4.3,1), ABIs with internal colons (dragonfly:6.4:x86:64), and the +// architecture-independent "*" ABI wildcard. +func TestPkgngPackages(t *testing.T) { + t.Parallel() + tests := []struct { + name string + out string + want []any + }{ + { + name: "dragonfly", + out: "bash|5.2.15|dragonfly:6.4:x86:64\n" + + "ca_root_nss|3.89.1|dragonfly:6.4:*\n" + + "flac|1.4.3,1|dragonfly:6.4:x86:64\n" + + "libedit|3.1.20221030,1|dragonfly:6.4:x86:64\n", + want: []any{ + map[string]any{"name": "bash", "version": "5.2.15", "architecture": "dragonfly:6.4:x86:64"}, + map[string]any{"name": "ca_root_nss", "version": "3.89.1", "architecture": "dragonfly:6.4:*"}, + map[string]any{"name": "flac", "version": "1.4.3,1", "architecture": "dragonfly:6.4:x86:64"}, + map[string]any{"name": "libedit", "version": "3.1.20221030,1", "architecture": "dragonfly:6.4:x86:64"}, + }, + }, + { + name: "freebsd", + out: "firstboot-freebsd-update|1.4|FreeBSD:14:*\n" + + "pkg|2.1.2|FreeBSD:14:amd64\n", + want: []any{ + map[string]any{"name": "firstboot-freebsd-update", "version": "1.4", "architecture": "FreeBSD:14:*"}, + map[string]any{"name": "pkg", "version": "2.1.2", "architecture": "FreeBSD:14:amd64"}, + }, + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + got := pkgngPackages(func(name string, args ...string) string { + if name != "/usr/local/sbin/pkg" || len(args) == 0 || args[0] != "query" { + t.Fatalf("command = %q %v", name, args) + } + return tt.out + }) + if !reflect.DeepEqual(got, tt.want) { + t.Fatalf("pkgngPackages() = %#v\nwant %#v", got, tt.want) + } + }) + } +} + +func TestPkgngPackagesEmptyYieldsNothing(t *testing.T) { + t.Parallel() + if got := pkgngPackages(func(string, ...string) string { return "" }); got != nil { + t.Fatalf("pkgngPackages(empty) = %#v, want nil", got) + } +} + +// openbsdContentsFixture is a trimmed real bash-5.2.15 +CONTENTS; the @arch +// annotation carries the architecture while the @comment line (which also +// contains the substring "arch") must not be mistaken for it. +const openbsdContentsFixture = `@name bash-5.2.15 +@option manual-installation +@comment pkgpath=shells/bash ftp=yes +@arch amd64 ++DESC +` + +// TestOpenbsdPackages enumerates real /var/db/pkg entry names (incl. the +// vim-9.0.2035-no_x11 flavor case and the gettext-runtime dashed stem), skips a +// non-directory index/lock entry, and reads architecture from +CONTENTS, +// omitting it for the architecture-independent "*" package. +func TestOpenbsdPackages(t *testing.T) { + t.Parallel() + entries := []os.DirEntry{ + fakeDirEntry{name: "bash-5.2.15", mode: os.ModeDir, isDir: true}, + fakeDirEntry{name: "gettext-runtime-0.22.2", mode: os.ModeDir, isDir: true}, + fakeDirEntry{name: "vim-9.0.2035-no_x11", mode: os.ModeDir, isDir: true}, + fakeDirEntry{name: "quirks-6.160", mode: os.ModeDir, isDir: true}, + fakeDirEntry{name: ".lock", isDir: false}, + } + contents := map[string]string{ + "/var/db/pkg/bash-5.2.15/+CONTENTS": openbsdContentsFixture, + "/var/db/pkg/gettext-runtime-0.22.2/+CONTENTS": "@name gettext-runtime-0.22.2\n@arch amd64\n", + "/var/db/pkg/vim-9.0.2035-no_x11/+CONTENTS": "@name vim-9.0.2035-no_x11\n@option manual-installation\n@arch amd64\n", + "/var/db/pkg/quirks-6.160/+CONTENTS": "@name quirks-6.160\n@arch *\n", + } + got := openbsdPackages( + func(path string) ([]os.DirEntry, error) { + if path != "/var/db/pkg" { + t.Fatalf("readDir path = %q", path) + } + return entries, nil + }, + func(path string) ([]byte, error) { + data, ok := contents[path] + if !ok { + t.Fatalf("readFile path = %q", path) + } + return []byte(data), nil + }, + ) + want := []any{ + map[string]any{"name": "bash", "version": "5.2.15", "architecture": "amd64"}, + map[string]any{"name": "gettext-runtime", "version": "0.22.2", "architecture": "amd64"}, + map[string]any{"name": "quirks", "version": "6.160"}, + map[string]any{"name": "vim", "version": "9.0.2035", "architecture": "amd64"}, + } + if !reflect.DeepEqual(got, want) { + t.Fatalf("openbsdPackages() = %#v\nwant %#v", got, want) + } +} + +func TestOpenbsdPackagesAbsentDatabaseYieldsNothing(t *testing.T) { + t.Parallel() + got := openbsdPackages( + func(string) ([]os.DirEntry, error) { return nil, os.ErrNotExist }, + func(string) ([]byte, error) { return nil, os.ErrNotExist }, + ) + if got != nil { + t.Fatalf("openbsdPackages(absent) = %#v, want nil", got) + } +} + +// TestPkgsrcPackages discovers PKG_DBDIR by probing the standard candidates in +// order. Entry names are real NetBSD /usr/pkg/pkgdb names, including the +// pkg_install underscore stem, the vim-share dashed stem, and the +// pkgdb.byfile.db index (a plain file that must be skipped). +func TestPkgsrcPackages(t *testing.T) { + t.Parallel() + entries := []os.DirEntry{ + fakeDirEntry{name: "bash-5.2.21nb1", mode: os.ModeDir, isDir: true}, + fakeDirEntry{name: "ca-certificates-20230311nb3", mode: os.ModeDir, isDir: true}, + fakeDirEntry{name: "pkg_install-20211115nb1", mode: os.ModeDir, isDir: true}, + fakeDirEntry{name: "vim-share-9.0.2122", mode: os.ModeDir, isDir: true}, + fakeDirEntry{name: "pkgdb.byfile.db", isDir: false}, + } + want := []any{ + map[string]any{"name": "bash", "version": "5.2.21nb1"}, + map[string]any{"name": "ca-certificates", "version": "20230311nb3"}, + map[string]any{"name": "pkg_install", "version": "20211115nb1"}, + map[string]any{"name": "vim-share", "version": "9.0.2122"}, + } + tests := []struct { + name string + dbdir string + }{ + {name: "default pkgdb", dbdir: "/usr/pkg/pkgdb"}, + {name: "legacy fallback", dbdir: "/var/db/pkg"}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + got := pkgsrcPackages(func(path string) ([]os.DirEntry, error) { + if path == tt.dbdir { + return entries, nil + } + return nil, os.ErrNotExist + }) + if !reflect.DeepEqual(got, want) { + t.Fatalf("pkgsrcPackages() = %#v\nwant %#v", got, want) + } + }) + } +} + +func TestPkgsrcPackagesAbsentDatabaseYieldsNothing(t *testing.T) { + t.Parallel() + got := pkgsrcPackages(func(string) ([]os.DirEntry, error) { return nil, os.ErrNotExist }) + if got != nil { + t.Fatalf("pkgsrcPackages(absent) = %#v, want nil", got) + } +} + +// ipsListFixture is real `pkg list -H` output from the nlab OmniOS guest (3 +// columns: NAME VERSION IFO). The image installs only from the preferred +// publisher, so no package renders a "(publisher)" column; the system/library +// line is a format-accurate synthetic entry from the extra.omnios publisher to +// exercise the parenthesised-publisher branch. +const ipsListFixture = `SUNWcs 0.5.11-151058.0 i-- +compress/xz 5.8.3-151058.0 i-- +database/sqlite-3 3.51.3-151058.0 i-- +developer/macro/cpp 20240422-151058.0 i-- +system/library (extra.omnios) 1.2.3-151046.0 i-- +` + +func TestIpsPackages(t *testing.T) { + t.Parallel() + got := ipsPackages(func(name string, args ...string) string { + if name != "pkg" || len(args) < 2 || args[0] != "list" || args[1] != "-H" { + t.Fatalf("command = %q %v", name, args) + } + return ipsListFixture + }) + want := []any{ + map[string]any{"name": "SUNWcs", "version": "0.5.11-151058.0"}, + map[string]any{"name": "compress/xz", "version": "5.8.3-151058.0"}, + map[string]any{"name": "database/sqlite-3", "version": "3.51.3-151058.0"}, + map[string]any{"name": "developer/macro/cpp", "version": "20240422-151058.0"}, + map[string]any{"name": "system/library", "version": "1.2.3-151046.0"}, + } + if !reflect.DeepEqual(got, want) { + t.Fatalf("ipsPackages() = %#v\nwant %#v", got, want) + } +} + +func TestIpsPackagesEmptyYieldsNothing(t *testing.T) { + t.Parallel() + if got := ipsPackages(func(string, ...string) string { return "" }); got != nil { + t.Fatalf("ipsPackages(empty) = %#v, want nil", got) + } +} + +// TestSplitBsdPackageName covers the OpenBSD/pkgsrc -[-flavor] +// convention: the version is the last hyphen component (after a non-empty stem) +// starting with a digit, keeping dashed and numeric-suffixed stems intact. +func TestSplitBsdPackageName(t *testing.T) { + t.Parallel() + tests := []struct { + dir string + wantName string + wantVersion string + }{ + {"autoconf-2.69p3", "autoconf", "2.69p3"}, + {"quirks-7.55", "quirks", "7.55"}, + {"xz-5.4.5", "xz", "5.4.5"}, + {"gettext-runtime-0.22.2", "gettext-runtime", "0.22.2"}, + {"vim-9.0.2035-no_x11", "vim", "9.0.2035"}, + {"pcre2-10.37p1", "pcre2", "10.37p1"}, + {"pkg_install-20211115nb1", "pkg_install", "20211115nb1"}, + {"gcc-11-11.2.0", "gcc-11", "11.2.0"}, + {"pkgdb.byfile.db", "", ""}, + {"quirks", "", ""}, + } + for _, tt := range tests { + t.Run(tt.dir, func(t *testing.T) { + t.Parallel() + name, version := splitBSDPackageName(tt.dir) + if name != tt.wantName || version != tt.wantVersion { + t.Fatalf("splitBSDPackageName(%q) = (%q, %q), want (%q, %q)", tt.dir, name, version, tt.wantName, tt.wantVersion) + } + }) + } +} diff --git a/internal/engine/packages_extra.go b/internal/engine/packages_extra.go new file mode 100644 index 00000000..d9cfa503 --- /dev/null +++ b/internal/engine/packages_extra.go @@ -0,0 +1,171 @@ +package engine + +import ( + "os" + "strings" +) + +// This file adds the "extra" Linux package sources (snap, flatpak, nix) to the +// packages fact (ADR-0014). Each reader is a pure function over injected probe +// output and follows the packages.go conventions: {name, version, ...} records +// via packageRecord, deterministic ordering via sortPackages, and nil when the +// source is absent or empty. Sources are never merged. +// +// Wiring (added to packagesCoreFacts' `case "linux":`, gated so a tool is only +// spawned when its state directory exists): +// +// if snapdPresent(s.stat) { +// add("snap", snapPackages(s.commandOutput)) +// } +// if flatpakPresent(s.stat) { +// add("flatpak", flatpakPackages(s.commandOutput)) +// } +// if nixProfilePresent(s.stat) { +// add("nix", nixPackages(s.commandOutput)) +// } + +// snapPackages parses `snap list` (columns: Name Version Rev Tracking Publisher +// Notes). The header row is required: its absence means snap printed "No snaps +// are installed yet." and there is nothing to record. +func snapPackages(run commandRunner) []any { + out := run("snap", "list") + var records []any + sawHeader := false + for line := range strings.Lines(out) { + fields := strings.Fields(line) + if len(fields) < 2 { + continue + } + if !sawHeader { + if fields[0] == "Name" && fields[1] == "Version" { + sawHeader = true + } + continue + } + records = append(records, packageRecord(fields[0], fields[1])) + } + sortPackages(records) + return records +} + +// flatpakPackages parses `flatpak list --columns=application,version,arch` (the +// system installation), whose rows are tab-separated with no header. The +// application id is the record name; the arch becomes the architecture identity. +func flatpakPackages(run commandRunner) []any { + out := run("flatpak", "list", "--columns=application,version,arch") + var records []any + for line := range strings.Lines(out) { + fields := strings.Split(strings.TrimRight(line, "\n"), "\t") + if len(fields) < 2 { + continue + } + name, version := strings.TrimSpace(fields[0]), strings.TrimSpace(fields[1]) + if name == "" || version == "" { + continue + } + arch := "" + if len(fields) >= 3 { + arch = strings.TrimSpace(fields[2]) + } + records = append(records, packageRecord(name, version, "architecture", arch)) + } + sortPackages(records) + return records +} + +// nixPackages enumerates the packages that make up the running NixOS system +// environment via `nix-store -q --references /run/current-system/sw` (the direct +// references of the system-path buildEnv). This is the installed profile set, not +// the whole /nix/store. nix-env -q cannot be used here: the NixOS system-path is +// a buildEnv without the manifest.nix nix-env requires, so it lists nothing. +// +// Each reference is a store path "/nix/store/--[-]"; +// the hash is dropped and the tail split by parseNixNameVersion. Split-output +// derivations (name-version-doc, -man, -bin, ...) collapse to one record via +// dedup on name+version; unversioned environment members are skipped. +func nixPackages(run commandRunner) []any { + // nix-store lives in the NixOS system profile, outside the engine's trusted + // command PATH (/usr/sbin:/usr/bin:/sbin:/bin); call it by absolute path. + out := run("/run/current-system/sw/bin/nix-store", "-q", "--references", "/run/current-system/sw") + var records []any + seen := map[string]bool{} + for line := range strings.Lines(out) { + line = strings.TrimSpace(line) + if line == "" { + continue + } + base := line[strings.LastIndexByte(line, '/')+1:] + _, tail, ok := strings.Cut(base, "-") // drop the store hash + if !ok { + continue + } + name, version := parseNixNameVersion(tail) + if name == "" || version == "" { + continue + } + key := name + "\x00" + version + if seen[key] { + continue + } + seen[key] = true + records = append(records, packageRecord(name, version)) + } + sortPackages(records) + return records +} + +// nixOutputs are the standard nix output component names appended to a store-path +// tail (e.g. glibc-2.42-67-bin). They are trimmed so split outputs of one +// derivation collapse onto its base version. +var nixOutputs = map[string]bool{ + "out": true, "bin": true, "dev": true, "lib": true, "doc": true, + "man": true, "info": true, "devdoc": true, "devman": true, + "static": true, "dist": true, "debug": true, "terminfo": true, +} + +// parseNixNameVersion splits a store-path tail "-[-]" using +// Nix's own boundary rule: the name ends at the first hyphen-delimited component +// that begins with a digit, and the version is the rest (so multi-component +// versions like glibc's "2.42-67" stay intact). A trailing standard output name +// is dropped. Environment members with no version (e.g. "nixos-help") yield "". +func parseNixNameVersion(tail string) (name, version string) { + parts := strings.Split(tail, "-") + boundary := -1 + // Start at 1: the name is always at least the first hyphen component, so a + // package whose name starts with a digit (7zip, 0ad, 389-ds-base) is not + // mistaken for a bare version and dropped. + for i := 1; i < len(parts); i++ { + if p := parts[i]; p != "" && p[0] >= '0' && p[0] <= '9' { + boundary = i + break + } + } + if boundary < 1 { + return "", "" + } + ver := parts[boundary:] + if len(ver) > 1 && nixOutputs[ver[len(ver)-1]] { + ver = ver[:len(ver)-1] + } + return strings.Join(parts[:boundary], "-"), strings.Join(ver, "-") +} + +// snapdPresent, flatpakPresent and nixProfilePresent gate each spawn on a cheap +// stat of the source's state directory, mirroring rpmDatabasePresent, so tools +// are never run on hosts that lack them. +func snapdPresent(stat func(string) (os.FileInfo, error)) bool { + return dirPresent(stat, "/var/lib/snapd") +} + +func flatpakPresent(stat func(string) (os.FileInfo, error)) bool { + return dirPresent(stat, "/var/lib/flatpak") +} + +func nixProfilePresent(stat func(string) (os.FileInfo, error)) bool { + return dirPresent(stat, "/nix/var/nix/profiles/system") +} + +func dirPresent(stat func(string) (os.FileInfo, error), path string) bool { + info, err := stat(path) + return err == nil && info.IsDir() +} diff --git a/internal/engine/packages_extra_test.go b/internal/engine/packages_extra_test.go new file mode 100644 index 00000000..189830cd --- /dev/null +++ b/internal/engine/packages_extra_test.go @@ -0,0 +1,179 @@ +package engine + +import ( + "os" + "reflect" + "testing" +) + +// nixReferencesFixture is verbatim `nix-store -q --references +// /run/current-system/sw` output captured from a NixOS 26.05 guest. It exercises +// the real shapes: plain name-version, split outputs (-man/-info/-doc/-bin) that +// must dedup onto their base version, glibc's multi-component "2.42-67" version +// that must stay intact, and unversioned environment members that must be dropped. +const nixReferencesFixture = `/nix/store/mxq1r9w2w2y9lsqb5fkcyb5xbbki1n57-ncurses-6.6 +/nix/store/6cblb0wy8kknk5jwj2gzah0jz6ihj3kx-ncurses-6.6-man +/nix/store/0641h8qfqaxnwrsw2nzrz6i1wbzyx92l-bash-interactive-5.3p9 +/nix/store/pspjmjsphdkjsi20gpxh2p1aq6p73n1c-bash-interactive-5.3p9-info +/nix/store/0cx4sx34abcvd49mwzjyxq4j5sxlbbmp-linux-pam-1.7.1-doc +/nix/store/pv8aczqkk94gzxhx0gk168gpxcll0svi-linux-pam-1.7.1 +/nix/store/sr26flm2nkfa12dkrwj2630kqsfakky4-coreutils-9.11 +/nix/store/1v1hd7hnss1y3d40dkib5x0dqppkadp8-sudo-1.9.17p2 +/nix/store/0sbi54dacjmi5grcgwyxz5zhc1pd26bp-sudo-1.9.17p2-doc +/nix/store/gxxyccld1qfy1v98hbzv8g9yk1saqx82-zstd-1.5.7-bin +/nix/store/1b2phk81syp3chqny7m713295rg519v6-zstd-1.5.7-man +/nix/store/521dd0054ifhzmjfmpx9mz0hr7wh7sig-glibc-2.42-67-bin +/nix/store/cdqbzw7gmcd62s8pyif19psr47l4000q-getent-glibc-2.42-67 +/nix/store/292rq2lcrzax7lcfhr2qwwxcqidv0df0-nixos-help +/nix/store/16c4zsnxzvm782n0ddhj6cabl23ar2vw-nixos-configuration-reference-manpage +` + +func TestNixPackages_dedupOutputsKeepsMultiComponentVersion(t *testing.T) { + t.Parallel() + got := nixPackages(func(name string, args ...string) string { + if name != "/run/current-system/sw/bin/nix-store" || !reflect.DeepEqual(args, []string{"-q", "--references", "/run/current-system/sw"}) { + t.Fatalf("command = %q %v", name, args) + } + return nixReferencesFixture + }) + want := []any{ + map[string]any{"name": "bash-interactive", "version": "5.3p9"}, + map[string]any{"name": "coreutils", "version": "9.11"}, + map[string]any{"name": "getent-glibc", "version": "2.42-67"}, + map[string]any{"name": "glibc", "version": "2.42-67"}, + map[string]any{"name": "linux-pam", "version": "1.7.1"}, + map[string]any{"name": "ncurses", "version": "6.6"}, + map[string]any{"name": "sudo", "version": "1.9.17p2"}, + map[string]any{"name": "zstd", "version": "1.5.7"}, + } + if !reflect.DeepEqual(got, want) { + t.Fatalf("nixPackages() = %#v\nwant %#v", got, want) + } +} + +func TestParseNixNameVersion_digitLeadingNames(t *testing.T) { + t.Parallel() + // A package whose name starts with a digit (7zip, 0ad, 389-ds-base) must not + // be mistaken for a version and dropped: the name is always at least the + // first hyphen component, so the version boundary is >= 1. + cases := []struct{ tail, name, version string }{ + {"7zip-24.09", "7zip", "24.09"}, + {"389-ds-base-2.6.0", "389-ds-base", "2.6.0"}, + {"0ad-0.0.26", "0ad", "0.0.26"}, + {"glibc-2.42-67-bin", "glibc", "2.42-67"}, // multi-component version, output stripped + {"nixos-help", "", ""}, // unversioned member dropped + } + for _, c := range cases { + if n, v := parseNixNameVersion(c.tail); n != c.name || v != c.version { + t.Errorf("parseNixNameVersion(%q) = (%q, %q), want (%q, %q)", c.tail, n, v, c.name, c.version) + } + } +} + +func TestNixPackages_emptyProfileYieldsNothing(t *testing.T) { + t.Parallel() + if got := nixPackages(func(string, ...string) string { return "" }); got != nil { + t.Fatalf("nixPackages(empty) = %#v, want nil", got) + } +} + +// snapListFixture is the canonical `snap list` layout (Name Version Rev Tracking +// Publisher Notes). Not guest-validated: no fleet guest has snaps installed. +const snapListFixture = `Name Version Rev Tracking Publisher Notes +bare 1.0 5 latest/stable canonical✓ base +core22 20240111 1122 latest/stable canonical✓ base +hello-world 6.4 29 latest/stable canonical✓ - +` + +func TestSnapPackages_skipsHeader(t *testing.T) { + t.Parallel() + got := snapPackages(func(name string, args ...string) string { + if name != "snap" || !reflect.DeepEqual(args, []string{"list"}) { + t.Fatalf("command = %q %v", name, args) + } + return snapListFixture + }) + want := []any{ + map[string]any{"name": "bare", "version": "1.0"}, + map[string]any{"name": "core22", "version": "20240111"}, + map[string]any{"name": "hello-world", "version": "6.4"}, + } + if !reflect.DeepEqual(got, want) { + t.Fatalf("snapPackages() = %#v\nwant %#v", got, want) + } +} + +func TestSnapPackages_noSnapsInstalledYieldsNothing(t *testing.T) { + t.Parallel() + got := snapPackages(func(string, ...string) string { + return "No snaps are installed yet. Try 'snap install hello-world'.\n" + }) + if got != nil { + t.Fatalf("snapPackages(no snaps) = %#v, want nil", got) + } +} + +// flatpakListFixture is tab-separated `flatpak list --columns=application,version,arch` +// output. Not guest-validated: no fleet guest has flatpak installed. +const flatpakListFixture = "org.gnome.Platform\t46\tx86_64\n" + + "org.mozilla.firefox\t124.0\tx86_64\n" + + "com.spotify.Client\t1.2.31.1205\tx86_64\n" + +func TestFlatpakPackages_parsesTabColumns(t *testing.T) { + t.Parallel() + got := flatpakPackages(func(name string, args ...string) string { + if name != "flatpak" || !reflect.DeepEqual(args, []string{"list", "--columns=application,version,arch"}) { + t.Fatalf("command = %q %v", name, args) + } + return flatpakListFixture + }) + want := []any{ + map[string]any{"name": "com.spotify.Client", "version": "1.2.31.1205", "architecture": "x86_64"}, + map[string]any{"name": "org.gnome.Platform", "version": "46", "architecture": "x86_64"}, + map[string]any{"name": "org.mozilla.firefox", "version": "124.0", "architecture": "x86_64"}, + } + if !reflect.DeepEqual(got, want) { + t.Fatalf("flatpakPackages() = %#v\nwant %#v", got, want) + } +} + +func TestFlatpakPackages_emptyYieldsNothing(t *testing.T) { + t.Parallel() + if got := flatpakPackages(func(string, ...string) string { return "" }); got != nil { + t.Fatalf("flatpakPackages(empty) = %#v, want nil", got) + } +} + +func TestExtraPackageSourcePresenceGates(t *testing.T) { + t.Parallel() + cases := []struct { + name string + gate func(func(string) (os.FileInfo, error)) bool + dir string + }{ + {"snap", snapdPresent, "/var/lib/snapd"}, + {"flatpak", flatpakPresent, "/var/lib/flatpak"}, + {"nix", nixProfilePresent, "/nix/var/nix/profiles/system"}, + } + for _, tc := range cases { + present := func(path string) (os.FileInfo, error) { + if path != tc.dir { + t.Fatalf("%s gate stat path = %q, want %q", tc.name, path, tc.dir) + } + return fakeFileInfo{name: tc.dir, mode: os.ModeDir, isDir: true}, nil + } + if !tc.gate(present) { + t.Fatalf("%s gate = false with %s present, want true", tc.name, tc.dir) + } + absent := func(string) (os.FileInfo, error) { return nil, os.ErrNotExist } + if tc.gate(absent) { + t.Fatalf("%s gate = true with %s absent, want false", tc.name, tc.dir) + } + notDir := func(string) (os.FileInfo, error) { + return fakeFileInfo{name: tc.dir}, nil + } + if tc.gate(notDir) { + t.Fatalf("%s gate = true with %s a non-directory, want false", tc.name, tc.dir) + } + } +} diff --git a/internal/engine/packages_mac.go b/internal/engine/packages_mac.go new file mode 100644 index 00000000..040d5f63 --- /dev/null +++ b/internal/engine/packages_mac.go @@ -0,0 +1,225 @@ +package engine + +import ( + "path/filepath" + "strings" +) + +// receiptsPackages resolves the "receipts" source (ADR-0014): the macOS +// installer receipt database under /var/db/receipts. Each *.plist is a binary +// property list, so a single `plutil -p` reads all of them in one spawn and +// prints one dict block per file. A block yields a record only when it carries +// both PackageIdentifier (name) and PackageVersion (version). +func receiptsPackages(glob pathGlobber, run commandRunner) []any { + paths, err := glob("/var/db/receipts/*.plist") + if err != nil || len(paths) == 0 { + return nil + } + var records []any + for _, chunk := range plutilArgChunks(paths) { + out := run("plutil", append([]string{"-p"}, chunk...)...) + parsePlutilBlocks(out, func(fields map[string]string) { + name, version := fields["PackageIdentifier"], fields["PackageVersion"] + if name == "" || version == "" { + return + } + records = append(records, packageRecord(name, version)) + }) + } + sortPackages(records) + return records +} + +// plutilArgChunks splits plist paths into batches whose joined argv length stays +// well under ARG_MAX, so one `plutil -p ` can never overflow the command +// line and silently drop the whole source on a host with a large receipt/app set. +func plutilArgChunks(paths []string) [][]string { + const maxBytes = 120 * 1024 // conservative; macOS ARG_MAX is 1 MiB + var chunks [][]string + var chunk []string + size := 0 + for _, p := range paths { + if len(chunk) > 0 && size+len(p)+1 > maxBytes { + chunks = append(chunks, chunk) + chunk, size = nil, 0 + } + chunk = append(chunk, p) + size += len(p) + 1 + } + if len(chunk) > 0 { + chunks = append(chunks, chunk) + } + return chunks +} + +// appsPackages resolves the "apps" source: installed application bundles. It is +// a secondary view of installed software and is never merged into "receipts". +// Both /Applications and /System/Applications are scanned, and every matched +// Info.plist is read by a single `plutil -p`. +// +// plutil -p prints one dict block per file, in argument order, and does NOT +// echo filenames. osHost.run (cmd.Output) returns "" unless plutil exits 0, so +// a non-empty result means every file parsed successfully; the block stream +// therefore aligns one-to-one with the glob order, letting the .app path be +// recovered positionally. Only the top-level (depth-1) keys of each block are +// consulted — Info.plists nest arbitrarily deep. +func appsPackages(glob pathGlobber, run commandRunner) []any { + var paths []string + for _, pattern := range []string{ + "/Applications/*/Contents/Info.plist", + "/System/Applications/*/Contents/Info.plist", + } { + matches, err := glob(pattern) + if err != nil { + continue + } + paths = append(paths, matches...) + } + if len(paths) == 0 { + return nil + } + var records []any + // Chunk so a large app set never overflows argv. plutil is all-or-nothing + // per invocation (cmd.Output is "" on any parse failure), so blocks align + // one-to-one with the chunk's paths; a failed chunk yields no blocks and + // cannot desynchronise a later chunk (the path index is chunk-local). + for _, chunk := range plutilArgChunks(paths) { + out := run("plutil", append([]string{"-p"}, chunk...)...) + index := 0 + parsePlutilBlocks(out, func(fields map[string]string) { + if index >= len(chunk) { + return + } + appPath := appBundlePath(chunk[index]) + index++ + name := firstNonEmpty(fields["CFBundleName"], fields["CFBundleDisplayName"], appBundleName(appPath)) + version := firstNonEmpty(fields["CFBundleShortVersionString"], fields["CFBundleVersion"]) + if name == "" || version == "" { + return + } + records = append(records, packageRecord(name, version, + "bundle_id", fields["CFBundleIdentifier"], + "path", appPath)) + }) + } + sortPackages(records) + return records +} + +// homebrewPackages resolves the "homebrew" source, auto-detected by the presence +// of a Cellar under a known prefix (/opt/homebrew on Apple Silicon, /usr/local on +// Intel). Formulae live at /Cellar//; casks at +// /Caskroom//. The source is omitted entirely when no +// Cellar exists. Only glob is needed — the name/version pair is the directory +// layout itself. +func homebrewPackages(glob pathGlobber) []any { + var records []any + for _, prefix := range []string{"/opt/homebrew", "/usr/local"} { + if formulae, err := glob(prefix + "/Cellar/*/*"); err == nil { + records = appendBrewRecords(records, formulae, "formula") + } + // Probe Caskroom independently of Cellar: a cask-only install has an + // empty Cellar but real casks to report. + if casks, err := glob(prefix + "/Caskroom/*/*"); err == nil { + records = appendBrewRecords(records, casks, "cask") + } + } + if len(records) == 0 { + return nil + } + sortPackages(records) + return records +} + +// appendBrewRecords turns / leaf paths into records. Homebrew +// keeps a dotfile sidecar (Caskroom//.metadata) that filepath.Glob's "*" +// matches — unlike the shell — so dot-prefixed leaves are skipped. +func appendBrewRecords(records []any, paths []string, kind string) []any { + for _, path := range paths { + version := filepath.Base(path) + name := filepath.Base(filepath.Dir(path)) + if name == "" || version == "" || strings.HasPrefix(version, ".") { + continue + } + records = append(records, packageRecord(name, version, "type", kind)) + } + return records +} + +// parsePlutilBlocks splits the text of `plutil -p file...` into one dict per +// input file and invokes fn with that dict's top-level (depth-1) scalar keys, +// in file order. plutil prints the outer dict's braces alone on a line and +// opens nested containers with a "key" => { or "key" => [ suffix; depth is +// tracked structurally rather than by column, so multi-line string values (an +// unindented copyright continuation, say) cannot desynchronise block or key +// boundaries. +func parsePlutilBlocks(out string, fn func(fields map[string]string)) { + var fields map[string]string + depth := 0 + for line := range strings.Lines(out) { + trimmed := strings.TrimSpace(line) + switch { + case trimmed == "{": + if depth == 0 { + fields = map[string]string{} + } + depth++ + case trimmed == "}" || trimmed == "]": + if depth > 0 { + depth-- + } + if depth == 0 && fields != nil { + fn(fields) + fields = nil + } + default: + // A depth-1 scalar is a real top-level key; a "key" => { or + // "key" => [ line opens a nested container we descend past. + opensContainer := strings.HasSuffix(trimmed, " => {") || strings.HasSuffix(trimmed, " => [") + if depth == 1 && !opensContainer { + if key, value, ok := plutilKeyValue(trimmed); ok { + fields[key] = value + } + } + if opensContainer { + depth++ + } + } + } +} + +// plutilKeyValue parses a `"key" => value` line, unquoting both sides. It +// reports ok=false for lines without the " => " separator (blank-line padding, +// wrapped multi-line string continuations). +func plutilKeyValue(line string) (key, value string, ok bool) { + rawKey, rawValue, found := strings.Cut(line, " => ") + if !found { + return "", "", false + } + return unquotePlutil(rawKey), unquotePlutil(rawValue), true +} + +func unquotePlutil(s string) string { + s = strings.TrimSpace(s) + if len(s) >= 2 && s[0] == '"' && s[len(s)-1] == '"' { + return s[1 : len(s)-1] + } + if strings.HasPrefix(s, `"`) { + // A string value whose closing quote is on a later physical line (a + // multi-line plutil value); keep this first line without the opener. + return s[1:] + } + return s +} + +// appBundlePath recovers the .app bundle path from its Info.plist path +// (.../Xxx.app/Contents/Info.plist -> .../Xxx.app). +func appBundlePath(infoPlist string) string { + return filepath.Dir(filepath.Dir(infoPlist)) +} + +// appBundleName derives a display name from a bundle path when the plist carries +// no CFBundleName/CFBundleDisplayName (.../Xxx.app -> Xxx). +func appBundleName(appPath string) string { + return strings.TrimSuffix(filepath.Base(appPath), ".app") +} diff --git a/internal/engine/packages_mac_test.go b/internal/engine/packages_mac_test.go new file mode 100644 index 00000000..6ccc9a10 --- /dev/null +++ b/internal/engine/packages_mac_test.go @@ -0,0 +1,295 @@ +package engine + +import ( + "reflect" + "testing" +) + +// globStub returns canned matches per pattern and records the plutil argv the +// reader would spawn, so tests exercise the real glob->plutil->parse pipeline +// with injected probes. +func globStub(matches map[string][]string) pathGlobber { + return func(pattern string) ([]string, error) { + return matches[pattern], nil + } +} + +// runStub returns fixed output and captures the file arguments (argv after +// "-p"), letting a test assert the plutil block order matches the glob order. +func runStub(output string, captured *[]string) commandRunner { + return func(_ string, args ...string) string { + if len(args) > 0 && captured != nil { + *captured = append([]string(nil), args[1:]...) + } + return output + } +} + +// receiptsFixture is verbatim `plutil -p /var/db/receipts/*.plist` output from a +// macOS host, trimmed to three real blocks plus a synthetic block missing +// PackageVersion (which must be dropped). +const receiptsFixture = `{ + "InstallDate" => 2026-01-17 09:08:36 +0000 + "InstallPrefixPath" => "Applications/GarageBand.app/Contents/" + "InstallProcessName" => "installer" + "PackageFileName" => "GarageBand_MASReceipt.pkg" + "PackageIdentifier" => "com.apple.cdm.pkg.GarageBand_MASReceipt" + "PackageVersion" => "1.0" +} +{ + "InstallDate" => 2026-01-17 09:09:55 +0000 + "InstallPrefixPath" => "/" + "InstallProcessName" => "installer" + "PackageFileName" => "Keynote14.pkg" + "PackageIdentifier" => "com.apple.pkg.Keynote14" + "PackageVersion" => "14.4.1.1742681647" +} +{ + "InstallDate" => 2026-01-17 09:10:32 +0000 + "InstallPrefixPath" => "/" + "PackageFileName" => "orphan.pkg" + "PackageIdentifier" => "com.example.noversion" +} +` + +func TestReceiptsPackages(t *testing.T) { + pattern := "/var/db/receipts/*.plist" + files := []string{ + "/var/db/receipts/com.apple.pkg.GarageBand.plist", + "/var/db/receipts/com.apple.pkg.Keynote14.plist", + "/var/db/receipts/com.example.noversion.plist", + } + var argv []string + got := receiptsPackages(globStub(map[string][]string{pattern: files}), runStub(receiptsFixture, &argv)) + + want := []any{ + packageRecord("com.apple.cdm.pkg.GarageBand_MASReceipt", "1.0"), + packageRecord("com.apple.pkg.Keynote14", "14.4.1.1742681647"), + } + if !reflect.DeepEqual(got, want) { + t.Fatalf("receiptsPackages = %#v, want %#v", got, want) + } + if !reflect.DeepEqual(argv, files) { + t.Fatalf("plutil argv = %v, want %v", argv, files) + } +} + +func TestReceiptsPackagesAbsent(t *testing.T) { + if got := receiptsPackages(globStub(nil), runStub("", nil)); got != nil { + t.Fatalf("receiptsPackages with no receipts = %#v, want nil", got) + } +} + +// appsFixture blocks are in glob-concatenation order (all /Applications first, +// then /System/Applications): 1Password, DisplayOnly, Nameless, Calculator. It +// mixes a real block (1Password, whose path has a space) and Calculator from +// /System (with both CFBundleName and a multi-line copyright whose continuation +// line is unindented) with synthetic blocks exercising every fallback: +// CFBundleDisplayName-only, name derived from the .app path, and +// CFBundleVersion-only. Nested array and dict values verify the depth tracker +// neither leaks nested keys nor splits blocks. +const appsFixture = `{ + "CFBundleIdentifier" => "com.agilebits.onepassword7" + "CFBundleName" => "1Password 7" + "CFBundleShortVersionString" => "7.9.11" + "CFBundleSupportedPlatforms" => [ + 0 => "MacOSX" + ] + "CFBundleVersion" => "70911001" +} +{ + "CFBundleDisplayName" => "Display Only" + "CFBundleDocumentTypes" => [ + 0 => { + "CFBundleTypeName" => "public.item" + } + ] + "CFBundleIdentifier" => "com.example.displayonly" + "CFBundleVersion" => "3" +} +{ + "CFBundleIdentifier" => "com.example.nameless" + "CFBundleVersion" => "9" +} +{ + "CFBundleDisplayName" => "Calculator" + "CFBundleIdentifier" => "com.apple.calculator" + "CFBundleName" => "Calculator" + "CFBundleShortVersionString" => "12.0" + "CFBundleVersion" => "225" + "NSHumanReadableCopyright" => "Copyright © 2022-2025 Apple Inc. +All rights reserved." +} +` + +var appsPaths = []string{ + "/Applications/1Password 7.app/Contents/Info.plist", + "/System/Applications/Calculator.app/Contents/Info.plist", + "/Applications/DisplayOnly.app/Contents/Info.plist", + "/Applications/Nameless.app/Contents/Info.plist", +} + +func TestAppsPackages(t *testing.T) { + glob := globStub(map[string][]string{ + "/Applications/*/Contents/Info.plist": {appsPaths[0], appsPaths[2], appsPaths[3]}, + "/System/Applications/*/Contents/Info.plist": {appsPaths[1]}, + }) + var argv []string + got := appsPackages(glob, runStub(appsFixture, &argv)) + + // argv (and therefore plutil block order) is /Applications first, then + // /System/Applications, matching the fixture order. + wantArgv := []string{appsPaths[0], appsPaths[2], appsPaths[3], appsPaths[1]} + if !reflect.DeepEqual(argv, wantArgv) { + t.Fatalf("plutil argv = %v, want %v", argv, wantArgv) + } + + want := []any{ + packageRecord("1Password 7", "7.9.11", + "bundle_id", "com.agilebits.onepassword7", + "path", "/Applications/1Password 7.app"), + packageRecord("Calculator", "12.0", + "bundle_id", "com.apple.calculator", + "path", "/System/Applications/Calculator.app"), + packageRecord("Display Only", "3", // CFBundleDisplayName fallback + "bundle_id", "com.example.displayonly", + "path", "/Applications/DisplayOnly.app"), + packageRecord("Nameless", "9", // name derived from .app path + "bundle_id", "com.example.nameless", + "path", "/Applications/Nameless.app"), + } + sortPackages(want) + if !reflect.DeepEqual(got, want) { + t.Fatalf("appsPackages = %#v, want %#v", got, want) + } +} + +func TestAppsPackagesAbsent(t *testing.T) { + if got := appsPackages(globStub(nil), runStub("", nil)); got != nil { + t.Fatalf("appsPackages with no bundles = %#v, want nil", got) + } +} + +func TestAppsPackages_skipsAppWithoutVersion(t *testing.T) { + // An app bundle with no CFBundleShortVersionString/CFBundleVersion must be + // dropped (name+version invariant); its slot must not shift the positional + // block<->path pairing for the remaining apps. + glob := globStub(map[string][]string{ + "/Applications/*/Contents/Info.plist": { + "/Applications/HasVer.app/Contents/Info.plist", + "/Applications/NoVer.app/Contents/Info.plist", + }, + }) + run := func(string, ...string) string { + return `{ + "CFBundleName" => "HasVer" + "CFBundleShortVersionString" => "2.0" +} +{ + "CFBundleName" => "NoVer" +} +` + } + got := appsPackages(glob, run) + want := []any{map[string]any{"name": "HasVer", "version": "2.0", "path": "/Applications/HasVer.app"}} + if !reflect.DeepEqual(got, want) { + t.Fatalf("appsPackages() = %#v\nwant %#v", got, want) + } +} + +func TestHomebrewPackages(t *testing.T) { + glob := globStub(map[string][]string{ + "/opt/homebrew/Cellar/*/*": { + "/opt/homebrew/Cellar/ada-url/3.4.4", + "/opt/homebrew/Cellar/brotli/1.2.0", + }, + "/opt/homebrew/Caskroom/*/*": { + "/opt/homebrew/Caskroom/amethyst/.metadata", // dotfile sidecar, skipped + "/opt/homebrew/Caskroom/amethyst/0.24.3", + "/opt/homebrew/Caskroom/ghostty/1.3.1", + }, + }) + got := homebrewPackages(glob) + + want := []any{ + packageRecord("ada-url", "3.4.4", "type", "formula"), + packageRecord("amethyst", "0.24.3", "type", "cask"), + packageRecord("brotli", "1.2.0", "type", "formula"), + packageRecord("ghostty", "1.3.1", "type", "cask"), + } + sortPackages(want) + if !reflect.DeepEqual(got, want) { + t.Fatalf("homebrewPackages = %#v, want %#v", got, want) + } +} + +func TestHomebrewPackagesIntelPrefix(t *testing.T) { + glob := globStub(map[string][]string{ + "/usr/local/Cellar/*/*": {"/usr/local/Cellar/wget/1.25.0"}, + }) + got := homebrewPackages(glob) + want := []any{packageRecord("wget", "1.25.0", "type", "formula")} + if !reflect.DeepEqual(got, want) { + t.Fatalf("homebrewPackages (intel) = %#v, want %#v", got, want) + } +} + +func TestHomebrewPackagesCaskOnly(t *testing.T) { + // A cask-only install has an empty Cellar but real casks to report. + glob := globStub(map[string][]string{ + "/opt/homebrew/Caskroom/*/*": {"/opt/homebrew/Caskroom/amethyst/0.24.3"}, + }) + got := homebrewPackages(glob) + want := []any{packageRecord("amethyst", "0.24.3", "type", "cask")} + if !reflect.DeepEqual(got, want) { + t.Fatalf("homebrewPackages (cask-only) = %#v, want %#v", got, want) + } +} + +func TestHomebrewPackagesNoHomebrew(t *testing.T) { + // Neither a Cellar nor a Caskroom under any prefix: the source is omitted. + if got := homebrewPackages(globStub(map[string][]string{})); got != nil { + t.Fatalf("homebrewPackages with no Homebrew = %#v, want nil", got) + } +} + +func TestParsePlutilBlocksNestedAndMultiline(t *testing.T) { + // A single block whose top level carries a scalar, a nested array (with a + // dict element), a nested dict, and a multi-line string. Only depth-1 + // scalars must survive; the nested "leaked" keys must not. + const block = `{ + "Top" => "value" + "Arr" => [ + 0 => { + "leaked" => "nested" + } + ] + "Dict" => { + "leaked" => "nested" + } + "Multi" => "line one +line two" + "After" => "seen" +} +` + var fields map[string]string + count := 0 + parsePlutilBlocks(block, func(f map[string]string) { + fields = f + count++ + }) + if count != 1 { + t.Fatalf("block count = %d, want 1", count) + } + // Exactly the depth-1 scalars survive: the multi-line value is captured (first + // physical line only), and the nested "leaked" keys and container openers + // (Arr, Dict) are absent. Full-map equality asserts all three at once. + want := map[string]string{ + "Top": "value", + "Multi": "line one", + "After": "seen", + } + if !reflect.DeepEqual(fields, want) { + t.Fatalf("top-level scalars = %#v, want exactly %#v", fields, want) + } +} diff --git a/internal/engine/packages_test.go b/internal/engine/packages_test.go new file mode 100644 index 00000000..815d79d3 --- /dev/null +++ b/internal/engine/packages_test.go @@ -0,0 +1,166 @@ +package engine + +import ( + "os" + "reflect" + "testing" +) + +// dpkgStatusFixture is a trimmed /var/lib/dpkg/status: two installed packages +// (one multiarch sibling pair) plus a removed package that must be filtered. +const dpkgStatusFixture = `Package: adduser +Status: install ok installed +Priority: important +Architecture: all +Version: 3.134 +Description: add and remove users + +Package: libc6 +Status: install ok installed +Architecture: amd64 +Version: 2.36-9+deb12u7 + +Package: libc6 +Status: install ok installed +Architecture: i386 +Version: 2.36-9+deb12u7 + +Package: nginx +Status: hold ok installed +Architecture: amd64 +Version: 1.22.1-9 + +Package: removed-pkg +Status: deinstall ok config-files +Architecture: amd64 +Version: 1.0 +` + +func TestDpkgPackages_installedOnlyWithMultiarchSiblings(t *testing.T) { + t.Parallel() + got := dpkgPackages(func(path string) ([]byte, error) { + if path != "/var/lib/dpkg/status" { + t.Fatalf("readFile path = %q", path) + } + return []byte(dpkgStatusFixture), nil + }) + want := []any{ + map[string]any{"name": "adduser", "version": "3.134", "architecture": "all"}, + map[string]any{"name": "libc6", "version": "2.36-9+deb12u7", "architecture": "amd64"}, + map[string]any{"name": "libc6", "version": "2.36-9+deb12u7", "architecture": "i386"}, + map[string]any{"name": "nginx", "version": "1.22.1-9", "architecture": "amd64"}, // held package kept + } + if !reflect.DeepEqual(got, want) { + t.Fatalf("dpkgPackages() = %#v\nwant %#v", got, want) + } +} + +// rpmQueryFixture is the output of the epoch-bearing rpm -qa query. gpg-pubkey +// must be filtered; the (none) epoch must be stripped; a real epoch is kept. +const rpmQueryFixture = `bash|(none):5.1.8-9.el9|x86_64 +glibc|(none):2.34-266.el9_8|x86_64 +kernel-core|(none):5.14.0-687.5.3.el9_8|x86_64 +grub2-common|1:2.06-104.el9_8|noarch +gpg-pubkey|(none):fd431d51-4ae0493b|(none) +` + +func TestRpmPackages_stripsNoneEpochKeepsRealEpochFiltersPubkey(t *testing.T) { + t.Parallel() + got := rpmPackages(func(name string, args ...string) string { + if name != "rpm" { + t.Fatalf("command = %q %v", name, args) + } + return rpmQueryFixture + }) + want := []any{ + map[string]any{"name": "bash", "version": "5.1.8-9.el9", "architecture": "x86_64"}, + map[string]any{"name": "glibc", "version": "2.34-266.el9_8", "architecture": "x86_64"}, + map[string]any{"name": "grub2-common", "version": "1:2.06-104.el9_8", "architecture": "noarch"}, + map[string]any{"name": "kernel-core", "version": "5.14.0-687.5.3.el9_8", "architecture": "x86_64"}, + } + if !reflect.DeepEqual(got, want) { + t.Fatalf("rpmPackages() = %#v\nwant %#v", got, want) + } +} + +// apkInstalledFixture is two stanzas from /lib/apk/db/installed. +const apkInstalledFixture = `C:Q1FVD1ypez9RDWd52MokUzPbtTaj8= +P:alpine-base +V:3.24.1-r0 +A:x86_64 +S:1322 +T:Meta package for minimal alpine base + +C:Q18Pz9wvTaBYr2RzvEXE6rYpFiJbI= +P:musl +V:1.2.5-r9 +A:x86_64 +` + +func TestApkPackages_parsesStanzas(t *testing.T) { + t.Parallel() + got := apkPackages(func(path string) ([]byte, error) { + if path != "/lib/apk/db/installed" { + t.Fatalf("readFile path = %q", path) + } + return []byte(apkInstalledFixture), nil + }) + want := []any{ + map[string]any{"name": "alpine-base", "version": "3.24.1-r0", "architecture": "x86_64"}, + map[string]any{"name": "musl", "version": "1.2.5-r9", "architecture": "x86_64"}, + } + if !reflect.DeepEqual(got, want) { + t.Fatalf("apkPackages() = %#v\nwant %#v", got, want) + } +} + +// pacmanDescFixture is one /var/lib/pacman/local//desc. +const pacmanDescFixture = `%NAME% +acl + +%VERSION% +2.3.2-2 + +%BASE% +acl + +%ARCH% +x86_64 +` + +func TestPacmanPackages_parsesDescFiles(t *testing.T) { + t.Parallel() + got := pacmanPackages( + func(pattern string) ([]string, error) { + if pattern != "/var/lib/pacman/local/*/desc" { + t.Fatalf("glob pattern = %q", pattern) + } + return []string{"/var/lib/pacman/local/zlib-1.3.1-2/desc", "/var/lib/pacman/local/acl-2.3.2-2/desc"}, nil + }, + func(path string) ([]byte, error) { + switch path { + case "/var/lib/pacman/local/acl-2.3.2-2/desc": + return []byte(pacmanDescFixture), nil + case "/var/lib/pacman/local/zlib-1.3.1-2/desc": + return []byte("%NAME%\nzlib\n\n%VERSION%\n1.3.1-2\n\n%ARCH%\nx86_64\n"), nil + default: + t.Fatalf("readFile path = %q", path) + return nil, nil + } + }, + ) + want := []any{ + map[string]any{"name": "acl", "version": "2.3.2-2", "architecture": "x86_64"}, + map[string]any{"name": "zlib", "version": "1.3.1-2", "architecture": "x86_64"}, + } + if !reflect.DeepEqual(got, want) { + t.Fatalf("pacmanPackages() = %#v\nwant %#v", got, want) + } +} + +func TestDpkgPackages_absentDatabaseYieldsNothing(t *testing.T) { + t.Parallel() + if got := dpkgPackages(func(string) ([]byte, error) { return nil, os.ErrNotExist }); got != nil { + t.Fatalf("dpkgPackages(absent) = %#v, want nil", got) + } +} diff --git a/internal/engine/packages_win.go b/internal/engine/packages_win.go new file mode 100644 index 00000000..9e8f0f7a --- /dev/null +++ b/internal/engine/packages_win.go @@ -0,0 +1,101 @@ +package engine + +import "strings" + +// registryPackagesScript queries both HKLM uninstall hives in a single +// PowerShell invocation: the native 64-bit view and the 32-bit WOW6432Node +// redirect. Each entry that carries a DisplayName is emitted as one +// pipe-delimited line. The architecture marker leads and the free-text +// DisplayName trails so a stray '|' inside a product name cannot shift the +// fixed columns. Columns: arch|PSChildName|DisplayVersion|SystemComponent|DisplayName. +const registryPackagesScript = `$ErrorActionPreference='SilentlyContinue';` + + `Get-ItemProperty 'HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\*'|Where-Object DisplayName|ForEach-Object{"x64|$($_.PSChildName)|$($_.DisplayVersion)|$($_.SystemComponent)|$($_.DisplayName)"};` + + `Get-ItemProperty 'HKLM:\SOFTWARE\WOW6432Node\Microsoft\Windows\CurrentVersion\Uninstall\*'|Where-Object DisplayName|ForEach-Object{"x86|$($_.PSChildName)|$($_.DisplayVersion)|$($_.SystemComponent)|$($_.DisplayName)"}` + +// registryPackages reads installed programs from the two HKLM uninstall hives. +// Every subkey with a DisplayName becomes a record; system-component/update +// entries (SystemComponent=1) are dropped, matching what Programs & Features +// shows. The architecture reflects the hive (x64 native, x86 WOW6432Node) and +// product_code is set only when the subkey name is an MSI product GUID. +func registryPackages(run commandRunner) []any { + out := run("powershell", "-NoProfile", "-NonInteractive", "-Command", registryPackagesScript) + var records []any + for line := range strings.Lines(out) { + fields := strings.SplitN(strings.TrimRight(line, "\r\n"), "|", 5) + if len(fields) != 5 { + continue + } + arch, subkey, version, systemComponent, name := fields[0], fields[1], fields[2], fields[3], fields[4] + if name == "" || version == "" || systemComponent == "1" { + continue + } + records = append(records, packageRecord(name, version, + "product_code", msiProductCode(subkey), + "architecture", arch)) + } + sortPackages(records) + return records +} + +// appxPackagesScript emits the system-provisioned appx set (Get-AppxProvisionedPackage) +// followed by the collector-context packages (Get-AppxPackage), one +// Name|Version|Architecture line each. The two views are unioned and deduplicated +// downstream, so an unavailable provisioning module (common on Server) simply +// yields fewer lines rather than an error. +const appxPackagesScript = `$ErrorActionPreference='SilentlyContinue';` + + `Get-AppxProvisionedPackage -Online|ForEach-Object{"$($_.DisplayName)|$($_.Version)|$($_.Architecture)"};` + + `Get-AppxPackage|ForEach-Object{"$($_.Name)|$($_.Version)|$($_.Architecture)"}` + +// appxPackages reads the appx/MSIX packages via PowerShell. Records carry the +// package name, version, and (lowercased) architecture; duplicates across the +// provisioned and collector-context views are collapsed. +func appxPackages(run commandRunner) []any { + out := run("powershell", "-NoProfile", "-NonInteractive", "-Command", appxPackagesScript) + seen := map[string]bool{} + var records []any + for line := range strings.Lines(out) { + fields := strings.SplitN(strings.TrimRight(line, "\r\n"), "|", 3) + if len(fields) != 3 { + continue + } + name, version, arch := fields[0], fields[1], strings.ToLower(fields[2]) + if name == "" || version == "" { + continue + } + key := name + "|" + version + "|" + arch + if seen[key] { + continue + } + seen[key] = true + records = append(records, packageRecord(name, version, "architecture", arch)) + } + sortPackages(records) + return records +} + +// msiProductCode returns name unchanged when it is a canonical MSI product GUID +// ({8-4-4-4-12} hex), and "" otherwise, so only genuine MSI installs contribute +// a product_code identity field (Inno Setup "*_is1" and other bespoke uninstall +// keys do not). +func msiProductCode(name string) string { + if len(name) != 38 || name[0] != '{' || name[37] != '}' { + return "" + } + for i := 1; i < 37; i++ { + switch i { + case 9, 14, 19, 24: + if name[i] != '-' { + return "" + } + default: + if !isHexDigit(name[i]) { + return "" + } + } + } + return name +} + +func isHexDigit(c byte) bool { + return (c >= '0' && c <= '9') || (c >= 'a' && c <= 'f') || (c >= 'A' && c <= 'F') +} diff --git a/internal/engine/packages_win_test.go b/internal/engine/packages_win_test.go new file mode 100644 index 00000000..267dd79c --- /dev/null +++ b/internal/engine/packages_win_test.go @@ -0,0 +1,160 @@ +package engine + +import ( + "reflect" + "strings" + "testing" +) + +// registryUninstallFixture is the delimited output of registryPackagesScript. +// Columns are arch|PSChildName|DisplayVersion|SystemComponent|DisplayName, with +// the x64 native hive emitted before the x86 WOW6432Node hive. It exercises the +// real shapes seen in the wild: an MSI GUID subkey (product_code populated) and +// a bespoke Inno Setup "*_is1" subkey (no product_code) in each architecture, +// plus SystemComponent=1 runtime entries that must be dropped. Values are real +// Microsoft/third-party identifiers; the nlab guest ships bare (only a +// property-less "WIC" key per hive, filtered by Where-Object DisplayName), so +// the multi-entry cases use authentic package identities in that live format. +const registryUninstallFixture = `x64|{e46eca4f-393b-40df-9f49-076faf788d83}|14.34.31931.0||Microsoft Visual C++ 2015-2022 Redistributable (x64) - 14.34.31931 +x64|{f1b0fb2f-3d5f-4c1e-9b2a-0a1b2c3d4e5f}|14.34.31931|1|Microsoft Visual C++ 2022 X64 Minimum Runtime - 14.34.31931 +x64|Git_is1|2.43.0.2||Git +x86|{a1c31ba0-9a3b-4f2d-8c7e-1234567890ab}|14.34.31931.0||Microsoft Visual C++ 2015-2022 Redistributable (x86) - 14.34.31931 +x86|Notepad++|8.6.9||Notepad++ (32-bit x86) +x86|{deadbeef-0000-1111-2222-333344445555}|1.0.0|1|Some 32-bit Runtime +` + +func TestRegistryPackages_bothHivesGUIDAndSystemComponent(t *testing.T) { + t.Parallel() + // Feed CRLF line endings to mirror real PowerShell output on Windows. + got := registryPackages(func(name string, args ...string) string { + if name != "powershell" { + t.Fatalf("command = %q %v", name, args) + } + if args[len(args)-1] != registryPackagesScript { + t.Fatalf("script arg = %q", args[len(args)-1]) + } + return strings.ReplaceAll(registryUninstallFixture, "\n", "\r\n") + }) + want := []any{ + map[string]any{"name": "Git", "version": "2.43.0.2", "architecture": "x64"}, + map[string]any{ + "name": "Microsoft Visual C++ 2015-2022 Redistributable (x64) - 14.34.31931", + "version": "14.34.31931.0", + "product_code": "{e46eca4f-393b-40df-9f49-076faf788d83}", + "architecture": "x64", + }, + map[string]any{ + "name": "Microsoft Visual C++ 2015-2022 Redistributable (x86) - 14.34.31931", + "version": "14.34.31931.0", + "product_code": "{a1c31ba0-9a3b-4f2d-8c7e-1234567890ab}", + "architecture": "x86", + }, + map[string]any{"name": "Notepad++ (32-bit x86)", "version": "8.6.9", "architecture": "x86"}, + } + if !reflect.DeepEqual(got, want) { + t.Fatalf("registryPackages() = %#v\nwant %#v", got, want) + } +} + +// TestRegistryPackages_skipsPropertylessKey mirrors the real nlab guest: the +// sole Uninstall subkey is "WIC" with no DisplayName. Where-Object filters it +// upstream, and the reader's guard drops any such line that slips through. +func TestRegistryPackages_skipsPropertylessKey(t *testing.T) { + t.Parallel() + got := registryPackages(func(string, ...string) string { return "x64|WIC|||\n" }) + if got != nil { + t.Fatalf("registryPackages(propertyless) = %#v, want nil", got) + } +} + +func TestRegistryPackages_absentYieldsNothing(t *testing.T) { + t.Parallel() + if got := registryPackages(func(string, ...string) string { return "" }); got != nil { + t.Fatalf("registryPackages(absent) = %#v, want nil", got) + } +} + +func TestRegistryMsiProductCode(t *testing.T) { + t.Parallel() + cases := []struct { + in, want string + }{ + {"{e46eca4f-393b-40df-9f49-076faf788d83}", "{e46eca4f-393b-40df-9f49-076faf788d83}"}, + {"{DEADBEEF-0000-1111-2222-333344445555}", "{DEADBEEF-0000-1111-2222-333344445555}"}, + {"Git_is1", ""}, + {"Notepad++", ""}, + {"{e46eca4f-393b-40df-9f49-076faf788d83", ""}, // missing closing brace + {"{g46eca4f-393b-40df-9f49-076faf788d83}", ""}, // non-hex digit + {"{e46eca4f393b40df9f49076faf788d83xxxx}", ""}, // wrong dash layout + {"", ""}, + } + for _, c := range cases { + if got := msiProductCode(c.in); got != c.want { + t.Fatalf("msiProductCode(%q) = %q, want %q", c.in, got, c.want) + } + } +} + +// appxFixture is the delimited output of appxPackagesScript: the provisioned +// set (DisplayName|Version|Architecture) followed by the collector-context +// Get-AppxPackage set (Name|Version|Architecture). It exercises architecture +// lowercasing, cross-view deduplication (the VCLibs line appears twice), and the +// empty-version skip. Values are real Windows package identities; the nlab guest +// ships zero appx (DISM provisioned = 0, Get-AppxPackage = 0), so the parse is +// validated against this authentic live format rather than empty guest output. +const appxFixture = `Windows Calculator|11.2210.0.0|X64 +Microsoft.WindowsStore|22210.1401.7.0|X64 +Microsoft.VCLibs.140.00|14.0.30704.0|X64 +Microsoft.VCLibs.140.00|14.0.30704.0|X64 +Microsoft.WindowsTerminal|1.18.3181.0|X86 +Microsoft.NET.Native.Runtime.2.2||X64 +Microsoft.UI.Xaml.2.8|8.2306.22001.0|neutral +` + +func TestAppxPackages_dedupLowercaseArchSkipsEmptyVersion(t *testing.T) { + t.Parallel() + got := appxPackages(func(name string, args ...string) string { + if name != "powershell" { + t.Fatalf("command = %q %v", name, args) + } + if args[len(args)-1] != appxPackagesScript { + t.Fatalf("script arg = %q", args[len(args)-1]) + } + return appxFixture + }) + want := []any{ + map[string]any{"name": "Microsoft.UI.Xaml.2.8", "version": "8.2306.22001.0", "architecture": "neutral"}, + map[string]any{"name": "Microsoft.VCLibs.140.00", "version": "14.0.30704.0", "architecture": "x64"}, + map[string]any{"name": "Microsoft.WindowsStore", "version": "22210.1401.7.0", "architecture": "x64"}, + map[string]any{"name": "Microsoft.WindowsTerminal", "version": "1.18.3181.0", "architecture": "x86"}, + map[string]any{"name": "Windows Calculator", "version": "11.2210.0.0", "architecture": "x64"}, + } + if !reflect.DeepEqual(got, want) { + t.Fatalf("appxPackages() = %#v\nwant %#v", got, want) + } +} + +func TestAppxPackages_absentYieldsNothing(t *testing.T) { + t.Parallel() + if got := appxPackages(func(string, ...string) string { return "" }); got != nil { + t.Fatalf("appxPackages(absent) = %#v, want nil", got) + } +} + +func TestRegistryPackages_skipsEmptyDisplayVersion(t *testing.T) { + t.Parallel() + // An uninstall entry with a DisplayName but no DisplayVersion must be dropped + // (the name+version invariant), while a fully-populated entry is kept. + run := func(string, ...string) string { + return "x64|{6F320B93-EE3C-4826-85E0-ADF79F8D4C61}|1.2.3|0|Real App\n" + + "x64|SomeKey_is1||0|No Version App\n" + } + got := registryPackages(run) + want := []any{map[string]any{ + "name": "Real App", "version": "1.2.3", + "product_code": "{6F320B93-EE3C-4826-85E0-ADF79F8D4C61}", "architecture": "x64", + }} + if !reflect.DeepEqual(got, want) { + t.Fatalf("registryPackages() = %#v\nwant %#v", got, want) + } +} diff --git a/man/man8/facts.8 b/man/man8/facts.8 index b924b561..9d64edd2 100644 --- a/man/man8/facts.8 +++ b/man/man8/facts.8 @@ -10,7 +10,7 @@ \fBfacts\fR [options] [query] [query] [\.\.\.] . .SH "DESCRIPTION" -\fBfacts\fR gathers structured facts about the current system, such as hardware details, network settings, OS type and version, cloud metadata, and storage state\. The same canonical fact tree is available through the command line and the Go library\. +\fBfacts\fR gathers structured facts about the current system, such as hardware details, network settings, OS type and version, installed packages (namespaced by package database), cloud metadata, and storage state\. The same canonical fact tree is available through the command line and the Go library\. . .P If no queries are given, then all facts will be returned\. diff --git a/openspec/changes/add-packages-fact/tasks.md b/openspec/changes/add-packages-fact/tasks.md index 2295cd48..0188043c 100644 --- a/openspec/changes/add-packages-fact/tasks.md +++ b/openspec/changes/add-packages-fact/tasks.md @@ -1,30 +1,30 @@ ## 1. Tests First -- [ ] 1.1 Add record-shape tests: every record carries `name` and a verbatim `version`; `packages.` is an array, never a name-keyed map. -- [ ] 1.2 Add collision tests proving siblings are kept: dpkg multiarch (`libc6:amd64` + `:i386`), rpm install-only multiversion kernels (same name + arch), homebrew formula-vs-cask `docker`, flatpak branches, Windows x86/x64 same DisplayName. -- [ ] 1.3 Add per-source identity-field tests: `architecture` (dpkg/rpm/apk), `type`+`tap` (homebrew), `branch`+`architecture` (flatpak), `store_path` (nix), `bundle_id`+`path` (apps), `product_code`+`architecture` (registry). -- [ ] 1.4 Add tests that a source is omitted when its database is absent, that records are never merged across sources, and that Plan 9 emits no `packages` subtree. -- [ ] 1.5 Add reader/parse fixtures per source (dpkg status, `rpm -qa` output, pacman `desc`, apk `installed`, snapd state, flatpak list, pkgng query, openbsd `/var/db/pkg`, pkgsrc PKG_DBDIR, ips `pkg list`, nix profile manifest, macOS receipts plist + Info.plist, registry hive, appx). +- [x] 1.1 Record-shape tests: every reader asserts `name`+verbatim `version`; `packages.` is a `[]any` array, never a name-keyed map. +- [x] 1.2 Collision/siblings tests: dpkg multiarch (`libc6:amd64`+`:i386`) kept; rpm epoch-bearing multiversion kernels; homebrew formula-vs-cask distinguished by `type`; Windows x86/x64 by `architecture`. (flatpak branch siblings: format-only, see 4.3.) +- [x] 1.3 Identity-field tests: `architecture` (dpkg/rpm/apk/pkg/flatpak), `type` (homebrew formula/cask), `bundle_id`+`path` (apps), `product_code`+`architecture` (registry). Deferred vs design (documented): homebrew `tap` (needs per-formula receipt read), flatpak `branch`, nix `store_path` (conflicts with the output-dedup — a derivation's outputs have distinct store paths; resolving the primary output is future work). +- [x] 1.4 Source omitted when its database is absent (or empty); records never merged across sources; Plan 9 emits no `packages` subtree (no darwin/linux/bsd/windows/illumos case matches). +- [x] 1.5 Reader/parse fixtures per source from **real** captured output (dpkg status, `rpm -qa` epoch query, pacman `desc`, apk `installed`, pkgng `pkg query`, openbsd `/var/db/pkg`, pkgsrc PKG_DBDIR, `pkg list -H`, `nix-store -q --references`, macOS receipts/apps `plutil -p`, registry/appx PowerShell lines, snap/flatpak columns). ## 2. Implementation -- [ ] 2.1 Add `internal/engine/packages.go` with `packagesCoreFacts(s *Session) []ResolvedFact`; compose it into `buildCoreFacts`. Pure `parse*` functions per source, no GOOS-suffixed files (ADR-0010). -- [ ] 2.2 Implement the Linux readers: `dpkg`, `rpm` (explicit epoch-bearing query), `pacman`, `apk`, `snap`, `flatpak`. -- [ ] 2.3 Implement the BSD/illumos readers: `pkg` (FreeBSD + DragonFly), `openbsd_pkg`, `pkgsrc` (NetBSD), `ips` (illumos). -- [ ] 2.4 Implement the macOS readers: `receipts` (primary), `apps` (secondary, never merged), `homebrew` (optional auto-detected); and `nix` where a profile is present. -- [ ] 2.5 Implement the Windows readers: `registry` (both HKLM hives, `product_code`+`architecture`) and `appx` (provisioned + collector context). -- [ ] 2.6 Enforce the collection rule (one cheap read per source; no spawn-per-package; no network) and the system-global + collector-context boundary. -- [ ] 2.7 Register `packages` as a fact group (no default-visibility change in this work). +- [x] 2.1 `internal/engine/packages.go` with `packagesCoreFacts(s *Session) []ResolvedFact`; composed into `buildCoreFacts` as a gated resolver. Pure `parse*` functions, no GOOS-suffixed files (ADR-0010) — per-platform readers live in `packages_bsd.go`/`packages_mac.go`/`packages_win.go`/`packages_extra.go` with no build constraints. +- [x] 2.2 Linux readers: `dpkg` (install/hold state kept), `rpm` (epoch query, gpg-pubkey filtered, DB-presence-gated), `pacman`, `apk`, `snap`, `flatpak`. +- [x] 2.3 BSD/illumos readers: `pkg` (FreeBSD+DragonFly, absolute `/usr/local/sbin/pkg`), `openbsd_pkg`, `pkgsrc` (NetBSD, PKG_DBDIR discovery), `ips` (illumos). +- [x] 2.4 macOS readers: `receipts` (primary), `apps` (secondary, never merged), `homebrew` (auto-detected); and `nix` (Linux, NixOS system profile). +- [x] 2.5 Windows readers: `registry` (both HKLM hives, `product_code`+`architecture`) and `appx` (provisioned + collector context). +- [x] 2.6 One cheap read/query per source; no spawn-per-package; no network; commands outside the trusted PATH use absolute paths; presence-gated spawns. +- [x] 2.7 `packages` registers as a gated fact group (`--disable packages` skips resolution). ## 3. Schema and Docs -- [ ] 3.1 Add `packages.*` to `docs/schema/facts.yaml` under ADR-0011 canonical spelling: the per-source arrays, the always-present `name`/`version`, and the per-source identity fields, with per-platform applicability. -- [ ] 3.2 Regenerate `docs/supported-facts/` to show the per-source record shapes. -- [ ] 3.3 Update README, man page, and CHANGELOG to mention the `packages` fact and its scope (system package databases; language managers out). +- [x] 3.1 `packages.*` added to `docs/schema/facts.yaml` (one array entry per source, per-platform applicability, conditional). +- [x] 3.2 Regenerated `docs/supported-facts/`. +- [x] 3.3 README, man page, and CHANGELOG mention the `packages` fact and scope (system package databases; language managers out). ## 4. Verification -- [ ] 4.1 Run focused resolver/parser tests for every source. -- [ ] 4.2 Run `go test ./...` and `go vet ./...`. -- [ ] 4.3 Validate the fact on each supported target's native smoke/release gate (dpkg, rpm, pacman, apk, snap/flatpak where present, pkg, openbsd_pkg, pkgsrc, ips, macOS receipts/apps, windows registry/appx); confirm Plan 9 emits nothing. -- [ ] 4.4 Run `openspec validate add-packages-fact --strict`. +- [x] 4.1 Focused resolver/parser tests for every source (green). +- [x] 4.2 `go test ./...` and `go vet ./...` (green); `gofmt` clean. +- [x] 4.3 nlab/local validation, counts vs native tools: dpkg 324, rpm 386, pacman 183, apk 200, pkg freebsd 2 / dragonfly 48, openbsd_pkg 18, pkgsrc 19, ips 418, nix 70, macOS receipts 10 / apps 61 / homebrew 88 — all exact matches; Plan 9 emits nothing. **Honest gaps:** Windows `registry`/`appx` are format- and empty-case-validated on the bare Server 2025 guest (0 installed apps; could not populate it), and `snap`/`flatpak` are format-only (a guest has snapd but 0 snaps; no guest has flatpak) — none could be populated without mutating shared lab state. +- [x] 4.4 `openspec validate add-packages-fact --strict`. From bcb1adf619d4fcf9c791826ff4fb2260745202b9 Mon Sep 17 00:00:00 2001 From: Juliano Martinez Date: Wed, 1 Jul 2026 13:50:33 +0200 Subject: [PATCH 2/6] fix(packages): use path not filepath for unix package paths The macOS and BSD readers build always-unix paths (/Applications, /var/db/pkg) but used path/filepath, whose separator is OS-specific. Because these pure parse functions are tested on every platform (ADR-0010, no GOOS suffix), the Windows CI runner produced backslash paths and failed TestAppsPackages / TestOpenbsdPackages. Switch to the path package (always '/'). Validated on the nlab Windows guest: the packages tests now PASS. --- internal/engine/packages_bsd.go | 3 +-- internal/engine/packages_mac.go | 12 ++++++------ 2 files changed, 7 insertions(+), 8 deletions(-) diff --git a/internal/engine/packages_bsd.go b/internal/engine/packages_bsd.go index 1c603d6d..95a0a163 100644 --- a/internal/engine/packages_bsd.go +++ b/internal/engine/packages_bsd.go @@ -2,7 +2,6 @@ package engine import ( "os" - "path/filepath" "strings" ) @@ -50,7 +49,7 @@ func openbsdPackages(readDir func(string) ([]os.DirEntry, error), readFile fileR if name == "" || version == "" { continue } - arch := bsdContentsArch(readFile, filepath.Join("/var/db/pkg", entry.Name(), "+CONTENTS")) + arch := bsdContentsArch(readFile, "/var/db/pkg/"+entry.Name()+"/+CONTENTS") records = append(records, packageRecord(name, version, "architecture", arch)) } sortPackages(records) diff --git a/internal/engine/packages_mac.go b/internal/engine/packages_mac.go index 040d5f63..aa33196f 100644 --- a/internal/engine/packages_mac.go +++ b/internal/engine/packages_mac.go @@ -1,7 +1,7 @@ package engine import ( - "path/filepath" + "path" "strings" ) @@ -135,9 +135,9 @@ func homebrewPackages(glob pathGlobber) []any { // keeps a dotfile sidecar (Caskroom//.metadata) that filepath.Glob's "*" // matches — unlike the shell — so dot-prefixed leaves are skipped. func appendBrewRecords(records []any, paths []string, kind string) []any { - for _, path := range paths { - version := filepath.Base(path) - name := filepath.Base(filepath.Dir(path)) + for _, p := range paths { + version := path.Base(p) + name := path.Base(path.Dir(p)) if name == "" || version == "" || strings.HasPrefix(version, ".") { continue } @@ -215,11 +215,11 @@ func unquotePlutil(s string) string { // appBundlePath recovers the .app bundle path from its Info.plist path // (.../Xxx.app/Contents/Info.plist -> .../Xxx.app). func appBundlePath(infoPlist string) string { - return filepath.Dir(filepath.Dir(infoPlist)) + return path.Dir(path.Dir(infoPlist)) } // appBundleName derives a display name from a bundle path when the plist carries // no CFBundleName/CFBundleDisplayName (.../Xxx.app -> Xxx). func appBundleName(appPath string) string { - return strings.TrimSuffix(filepath.Base(appPath), ".app") + return strings.TrimSuffix(path.Base(appPath), ".app") } From a7a07000a53db53215f0084d043bad66f76a944c Mon Sep 17 00:00:00 2001 From: Juliano Martinez Date: Thu, 2 Jul 2026 09:46:05 +0200 Subject: [PATCH 3/6] docs(packages): record real Windows registry validation (7-Zip x64+x86) --- openspec/changes/add-packages-fact/tasks.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/openspec/changes/add-packages-fact/tasks.md b/openspec/changes/add-packages-fact/tasks.md index 0188043c..583e223d 100644 --- a/openspec/changes/add-packages-fact/tasks.md +++ b/openspec/changes/add-packages-fact/tasks.md @@ -26,5 +26,5 @@ - [x] 4.1 Focused resolver/parser tests for every source (green). - [x] 4.2 `go test ./...` and `go vet ./...` (green); `gofmt` clean. -- [x] 4.3 nlab/local validation, counts vs native tools: dpkg 324, rpm 386, pacman 183, apk 200, pkg freebsd 2 / dragonfly 48, openbsd_pkg 18, pkgsrc 19, ips 418, nix 70, macOS receipts 10 / apps 61 / homebrew 88 — all exact matches; Plan 9 emits nothing. **Honest gaps:** Windows `registry`/`appx` are format- and empty-case-validated on the bare Server 2025 guest (0 installed apps; could not populate it), and `snap`/`flatpak` are format-only (a guest has snapd but 0 snaps; no guest has flatpak) — none could be populated without mutating shared lab state. +- [x] 4.3 nlab/local validation, counts vs native tools: dpkg 324, rpm 386, pacman 183, apk 200, pkg freebsd 2 / dragonfly 48, openbsd_pkg 18, pkgsrc 19, ips 418, nix 70, macOS receipts 10 / apps 61 / homebrew 88 — all exact matches; Plan 9 emits nothing. Windows `registry` is **populated-validated** on the nlab Server 2025 guest: installed 7-Zip x64 + x86 (MSI), and `facts.registry` reports both — `7-Zip 24.08 (x64 edition)`/x64 from the native hive and `7-Zip 24.08`/x86 from WOW6432Node, each with the correct MSI `product_code` GUID — matching the live registry exactly (dual-hive read + architecture + product_code all confirmed). Windows `appx`: PowerShell script syntax verified on the guest and parse logic unit-tested; correctly omitted on the appx-less Server (populating appx needs a signed MSIX — disproportionate for a secondary source). Remaining format-only: `snap` (a guest has snapd but 0 snaps) and `flatpak` (no guest has it). - [x] 4.4 `openspec validate add-packages-fact --strict`. From 834dabbb39fbef76ea58c4e1fdc6b1b1f8a9e578 Mon Sep 17 00:00:00 2001 From: Juliano Martinez Date: Thu, 2 Jul 2026 09:57:27 +0200 Subject: [PATCH 4/6] feat(packages): flatpak branch identity field; snap/flatpak guest-validated MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Populated the nlab ubuntu2404 guest (snap hello-world; flatpak org.vim.Vim from flathub) and validated both readers end-to-end. Real data showed the deferred flatpak `branch` field is load-bearing, not decorative: the same application id (org.freedesktop.Platform.GL.default) installs twice with an identical version and arch, distinguishable only by branch (25.08 vs 25.08-extra) — without it the reader emitted two indistinguishable duplicate records. The reader now requests --columns=application,version,arch,branch and emits branch as an identity field; versionless extensions (codecs-extra) stay dropped by the name+version invariant. Guest validation: facts.snap = 3 = snap list; facts.flatpak = 4 = versioned flatpak rows; dpkg (740) coexists on the same host. Fixtures replaced with verbatim guest output. --- docs/schema/facts.yaml | 2 +- docs/supported-facts/linux.md | 2 +- internal/engine/packages_extra.go | 19 ++++++--- internal/engine/packages_extra_test.go | 44 ++++++++++++--------- openspec/changes/add-packages-fact/tasks.md | 6 +-- 5 files changed, 44 insertions(+), 29 deletions(-) diff --git a/docs/schema/facts.yaml b/docs/schema/facts.yaml index ce96c70f..02c4bef6 100644 --- a/docs/schema/facts.yaml +++ b/docs/schema/facts.yaml @@ -817,7 +817,7 @@ packages.dpkg: conditional: true packages.flatpak: type: array - description: System-installed Flatpak applications as {name, version, architecture} records. + description: System-installed Flatpak applications and runtimes as {name, version, architecture, branch} records; branch distinguishes same-version siblings. platforms: [linux] conditional: true packages.homebrew: diff --git a/docs/supported-facts/linux.md b/docs/supported-facts/linux.md index 78d31449..27326d5a 100644 --- a/docs/supported-facts/linux.md +++ b/docs/supported-facts/linux.md @@ -219,7 +219,7 @@ $ facts --json | `os.selinux.policy_version` | `string` | yes | The loaded SELinux policy version, when SELinux is enabled. | | `packages.apk` | `array` | yes | Installed Alpine apk packages as {name, version, architecture} records, from /lib/apk/db/installed. | | `packages.dpkg` | `array` | yes | Installed dpkg packages as {name, version, architecture} records (install ok installed only; multiarch siblings kept), from /var/lib/dpkg/status. | -| `packages.flatpak` | `array` | yes | System-installed Flatpak applications as {name, version, architecture} records. | +| `packages.flatpak` | `array` | yes | System-installed Flatpak applications and runtimes as {name, version, architecture, branch} records; branch distinguishes same-version siblings. | | `packages.nix` | `array` | yes | Installed Nix packages as {name, version} records — the NixOS system profile set (the references of /run/current-system/sw), never the whole /nix/store. | | `packages.pacman` | `array` | yes | Installed pacman packages as {name, version, architecture} records, from /var/lib/pacman/local. | | `packages.rpm` | `array` | yes | Installed rpm packages as {name, version, architecture} records (epoch preserved, gpg-pubkey filtered). | diff --git a/internal/engine/packages_extra.go b/internal/engine/packages_extra.go index d9cfa503..35b65b62 100644 --- a/internal/engine/packages_extra.go +++ b/internal/engine/packages_extra.go @@ -48,11 +48,15 @@ func snapPackages(run commandRunner) []any { return records } -// flatpakPackages parses `flatpak list --columns=application,version,arch` (the -// system installation), whose rows are tab-separated with no header. The -// application id is the record name; the arch becomes the architecture identity. +// flatpakPackages parses `flatpak list --columns=application,version,arch,branch` +// (the system installation), whose rows are tab-separated with no header. The +// application id is the record name; arch and branch are identity fields — +// branch is load-bearing, not decorative: the same application id can be +// installed twice with an identical version and arch, distinguishable only by +// branch (observed live: GL.default 25.08 vs 25.08-extra). Extensions with no +// version (e.g. codecs-extra) are dropped by the name+version invariant. func flatpakPackages(run commandRunner) []any { - out := run("flatpak", "list", "--columns=application,version,arch") + out := run("flatpak", "list", "--columns=application,version,arch,branch") var records []any for line := range strings.Lines(out) { fields := strings.Split(strings.TrimRight(line, "\n"), "\t") @@ -63,11 +67,14 @@ func flatpakPackages(run commandRunner) []any { if name == "" || version == "" { continue } - arch := "" + var arch, branch string if len(fields) >= 3 { arch = strings.TrimSpace(fields[2]) } - records = append(records, packageRecord(name, version, "architecture", arch)) + if len(fields) >= 4 { + branch = strings.TrimSpace(fields[3]) + } + records = append(records, packageRecord(name, version, "architecture", arch, "branch", branch)) } sortPackages(records) return records diff --git a/internal/engine/packages_extra_test.go b/internal/engine/packages_extra_test.go index 189830cd..9fdcb911 100644 --- a/internal/engine/packages_extra_test.go +++ b/internal/engine/packages_extra_test.go @@ -77,12 +77,12 @@ func TestNixPackages_emptyProfileYieldsNothing(t *testing.T) { } } -// snapListFixture is the canonical `snap list` layout (Name Version Rev Tracking -// Publisher Notes). Not guest-validated: no fleet guest has snaps installed. -const snapListFixture = `Name Version Rev Tracking Publisher Notes -bare 1.0 5 latest/stable canonical✓ base -core22 20240111 1122 latest/stable canonical✓ base -hello-world 6.4 29 latest/stable canonical✓ - +// snapListFixture is verbatim `snap list` output from the nlab ubuntu2404 +// guest (snapd 2.75.2) after installing hello-world. +const snapListFixture = `Name Version Rev Tracking Publisher Notes +core 16-2.61.4-20260225 17292 latest/stable canonical** core +hello-world 6.4 29 latest/stable canonical** - +snapd 2.75.2 26865 latest/stable canonical** snapd ` func TestSnapPackages_skipsHeader(t *testing.T) { @@ -94,9 +94,9 @@ func TestSnapPackages_skipsHeader(t *testing.T) { return snapListFixture }) want := []any{ - map[string]any{"name": "bare", "version": "1.0"}, - map[string]any{"name": "core22", "version": "20240111"}, + map[string]any{"name": "core", "version": "16-2.61.4-20260225"}, map[string]any{"name": "hello-world", "version": "6.4"}, + map[string]any{"name": "snapd", "version": "2.75.2"}, } if !reflect.DeepEqual(got, want) { t.Fatalf("snapPackages() = %#v\nwant %#v", got, want) @@ -113,24 +113,32 @@ func TestSnapPackages_noSnapsInstalledYieldsNothing(t *testing.T) { } } -// flatpakListFixture is tab-separated `flatpak list --columns=application,version,arch` -// output. Not guest-validated: no fleet guest has flatpak installed. -const flatpakListFixture = "org.gnome.Platform\t46\tx86_64\n" + - "org.mozilla.firefox\t124.0\tx86_64\n" + - "com.spotify.Client\t1.2.31.1205\tx86_64\n" +// flatpakListFixture is verbatim tab-separated output of +// `flatpak list --columns=application,version,arch,branch` from the nlab +// ubuntu2404 guest after installing org.vim.Vim from flathub. It exercises the +// two real shapes the format-only fixture missed: the SAME application id +// installed twice with identical version+arch, distinguishable only by branch +// (GL.default 25.08 vs 25.08-extra), and an extension with an empty version +// (codecs-extra), which is dropped by the name+version invariant. +const flatpakListFixture = "org.freedesktop.Platform.GL.default\t26.0.8\tx86_64\t25.08\n" + + "org.freedesktop.Platform.GL.default\t26.0.8\tx86_64\t25.08-extra\n" + + "org.freedesktop.Platform.codecs-extra\t\tx86_64\t25.08-extra\n" + + "org.freedesktop.Sdk\tfreedesktop-sdk-25.08.13\tx86_64\t25.08\n" + + "org.vim.Vim\tv9.2.0758\tx86_64\tstable\n" -func TestFlatpakPackages_parsesTabColumns(t *testing.T) { +func TestFlatpakPackages_branchDistinguishesSiblings(t *testing.T) { t.Parallel() got := flatpakPackages(func(name string, args ...string) string { - if name != "flatpak" || !reflect.DeepEqual(args, []string{"list", "--columns=application,version,arch"}) { + if name != "flatpak" || !reflect.DeepEqual(args, []string{"list", "--columns=application,version,arch,branch"}) { t.Fatalf("command = %q %v", name, args) } return flatpakListFixture }) want := []any{ - map[string]any{"name": "com.spotify.Client", "version": "1.2.31.1205", "architecture": "x86_64"}, - map[string]any{"name": "org.gnome.Platform", "version": "46", "architecture": "x86_64"}, - map[string]any{"name": "org.mozilla.firefox", "version": "124.0", "architecture": "x86_64"}, + map[string]any{"name": "org.freedesktop.Platform.GL.default", "version": "26.0.8", "architecture": "x86_64", "branch": "25.08"}, + map[string]any{"name": "org.freedesktop.Platform.GL.default", "version": "26.0.8", "architecture": "x86_64", "branch": "25.08-extra"}, + map[string]any{"name": "org.freedesktop.Sdk", "version": "freedesktop-sdk-25.08.13", "architecture": "x86_64", "branch": "25.08"}, + map[string]any{"name": "org.vim.Vim", "version": "v9.2.0758", "architecture": "x86_64", "branch": "stable"}, } if !reflect.DeepEqual(got, want) { t.Fatalf("flatpakPackages() = %#v\nwant %#v", got, want) diff --git a/openspec/changes/add-packages-fact/tasks.md b/openspec/changes/add-packages-fact/tasks.md index 583e223d..a07049ab 100644 --- a/openspec/changes/add-packages-fact/tasks.md +++ b/openspec/changes/add-packages-fact/tasks.md @@ -1,8 +1,8 @@ ## 1. Tests First - [x] 1.1 Record-shape tests: every reader asserts `name`+verbatim `version`; `packages.` is a `[]any` array, never a name-keyed map. -- [x] 1.2 Collision/siblings tests: dpkg multiarch (`libc6:amd64`+`:i386`) kept; rpm epoch-bearing multiversion kernels; homebrew formula-vs-cask distinguished by `type`; Windows x86/x64 by `architecture`. (flatpak branch siblings: format-only, see 4.3.) -- [x] 1.3 Identity-field tests: `architecture` (dpkg/rpm/apk/pkg/flatpak), `type` (homebrew formula/cask), `bundle_id`+`path` (apps), `product_code`+`architecture` (registry). Deferred vs design (documented): homebrew `tap` (needs per-formula receipt read), flatpak `branch`, nix `store_path` (conflicts with the output-dedup — a derivation's outputs have distinct store paths; resolving the primary output is future work). +- [x] 1.2 Collision/siblings tests: dpkg multiarch (`libc6:amd64`+`:i386`) kept; rpm epoch-bearing multiversion kernels; homebrew formula-vs-cask distinguished by `type`; Windows x86/x64 by `architecture`; flatpak same-app-same-version siblings distinguished by `branch` (observed live: GL.default 25.08 vs 25.08-extra). +- [x] 1.3 Identity-field tests: `architecture` (dpkg/rpm/apk/pkg/flatpak), `branch` (flatpak), `type` (homebrew formula/cask), `bundle_id`+`path` (apps), `product_code`+`architecture` (registry). Deferred vs design (documented): homebrew `tap` (needs per-formula receipt read) and nix `store_path` (conflicts with the output-dedup — a derivation's outputs have distinct store paths; resolving the primary output is future work). - [x] 1.4 Source omitted when its database is absent (or empty); records never merged across sources; Plan 9 emits no `packages` subtree (no darwin/linux/bsd/windows/illumos case matches). - [x] 1.5 Reader/parse fixtures per source from **real** captured output (dpkg status, `rpm -qa` epoch query, pacman `desc`, apk `installed`, pkgng `pkg query`, openbsd `/var/db/pkg`, pkgsrc PKG_DBDIR, `pkg list -H`, `nix-store -q --references`, macOS receipts/apps `plutil -p`, registry/appx PowerShell lines, snap/flatpak columns). @@ -26,5 +26,5 @@ - [x] 4.1 Focused resolver/parser tests for every source (green). - [x] 4.2 `go test ./...` and `go vet ./...` (green); `gofmt` clean. -- [x] 4.3 nlab/local validation, counts vs native tools: dpkg 324, rpm 386, pacman 183, apk 200, pkg freebsd 2 / dragonfly 48, openbsd_pkg 18, pkgsrc 19, ips 418, nix 70, macOS receipts 10 / apps 61 / homebrew 88 — all exact matches; Plan 9 emits nothing. Windows `registry` is **populated-validated** on the nlab Server 2025 guest: installed 7-Zip x64 + x86 (MSI), and `facts.registry` reports both — `7-Zip 24.08 (x64 edition)`/x64 from the native hive and `7-Zip 24.08`/x86 from WOW6432Node, each with the correct MSI `product_code` GUID — matching the live registry exactly (dual-hive read + architecture + product_code all confirmed). Windows `appx`: PowerShell script syntax verified on the guest and parse logic unit-tested; correctly omitted on the appx-less Server (populating appx needs a signed MSIX — disproportionate for a secondary source). Remaining format-only: `snap` (a guest has snapd but 0 snaps) and `flatpak` (no guest has it). +- [x] 4.3 nlab/local validation, counts vs native tools: dpkg 324, rpm 386, pacman 183, apk 200, pkg freebsd 2 / dragonfly 48, openbsd_pkg 18, pkgsrc 19, ips 418, nix 70, macOS receipts 10 / apps 61 / homebrew 88 — all exact matches; Plan 9 emits nothing. Windows `registry` is **populated-validated** on the nlab Server 2025 guest: installed 7-Zip x64 + x86 (MSI), and `facts.registry` reports both — `7-Zip 24.08 (x64 edition)`/x64 from the native hive and `7-Zip 24.08`/x86 from WOW6432Node, each with the correct MSI `product_code` GUID — matching the live registry exactly (dual-hive read + architecture + product_code all confirmed). Windows `appx`: PowerShell script syntax verified on the guest and parse logic unit-tested; correctly omitted on the appx-less Server (populating appx needs a signed MSIX — disproportionate for a secondary source). `snap` and `flatpak` are **populated-validated** on the nlab ubuntu2404 guest: installed hello-world (snap) and org.vim.Vim from flathub (flatpak); `facts.snap` = 3 = `snap list`, `facts.flatpak` = 4 = the versioned `flatpak list` rows, with the same-app-same-version GL.default siblings distinguished by `branch` and the versionless codecs-extra extension dropped by the name+version invariant — all alongside dpkg (740) on the same host. - [x] 4.4 `openspec validate add-packages-fact --strict`. From 9877af6e32ae111e79d14da32ba3d59c17167f3c Mon Sep 17 00:00:00 2001 From: Juliano Martinez Date: Thu, 2 Jul 2026 16:50:08 +0200 Subject: [PATCH 5/6] fix(packages): apply deep-review findings across readers, docs, and grouping MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Deep multi-lens review of the changeset surfaced 9 code defects and 4 doc drifts (each independently triaged against the code and design docs): - dpkg: keep trigger-state packages (install ok triggers-awaited/pending) — fully unpacked and configured; drop half-installed. Held already kept. - groups: add the Facts-native "packages" fact group (ADR-0014's deliberate parity divergence) so --list-block-groups names it as one disable unit. - pkgsrc: honor ADR-0014's "illumos/DragonFly secondary" — wired into both dispatches with the SmartOS dbdir candidates (/opt/local/pkgdb, /opt/local/pkg); inert on hosts without a pkgsrc db (omnios ips 418 and dragonfly pkg 48 unchanged, pkgsrc omitted). - nix: honor ADR-0014's "default profile AND NixOS system profile" — fall back to nix-env -q --profile /nix/var/nix/profiles/default on non-NixOS hosts (validated against a real daemon install on ubuntu2404: 3 records incl. the nix-manual-2.34.7-man output-suffix case); gate opens on either profile; also wired on darwin per the design. NixOS still prefers the system profile without a fallback spawn. - apps: scan the Utilities subfolders and match only *.app bundles — live darwin count 61 -> 80, exactly matching ls. - plutil pipeline: bisect failed chunks so one corrupt plist no longer drops the whole receipts/apps source (O(log n) recovery, pairing per-invocation by construction); track multi-line string values so brace-looking lines inside them cannot desync block boundaries; handle array-rooted plists so pairing survives them. - homebrew: add the prefix identity field — dual-prefix installs (/opt/homebrew + Rosetta /usr/local) previously yielded byte-identical duplicate records. - schema: correct six descriptions to match behavior (dpkg states, appx provisioned+collector union, flatpak system+collector-user, openbsd_pkg architecture, homebrew prefix, nix profiles) and pkgsrc platforms. ubuntu2404 now validates four coexisting sources end-to-end: dpkg 740, snap 3, flatpak 4, nix 3. --- docs/schema/facts.yaml | 18 +- docs/supported-facts/README.md | 6 +- docs/supported-facts/darwin.md | 5 +- docs/supported-facts/dragonfly.md | 3 +- docs/supported-facts/illumos.md | 3 +- docs/supported-facts/linux.md | 6 +- docs/supported-facts/netbsd.md | 2 +- docs/supported-facts/openbsd.md | 2 +- docs/supported-facts/windows.md | 2 +- internal/engine/builtin_groups_test.go | 4 + internal/engine/groups.go | 4 + internal/engine/packages.go | 28 ++- internal/engine/packages_bsd.go | 15 +- internal/engine/packages_bsd_test.go | 2 + internal/engine/packages_extra.go | 39 +++- internal/engine/packages_extra_test.go | 81 ++++++- internal/engine/packages_mac.go | 207 ++++++++++++----- internal/engine/packages_mac_test.go | 302 +++++++++++++++++++++++-- internal/engine/packages_test.go | 19 +- 19 files changed, 628 insertions(+), 120 deletions(-) diff --git a/docs/schema/facts.yaml b/docs/schema/facts.yaml index 02c4bef6..9c34598d 100644 --- a/docs/schema/facts.yaml +++ b/docs/schema/facts.yaml @@ -802,7 +802,7 @@ packages.apk: conditional: true packages.appx: type: array - description: System-provisioned Windows AppX packages as {name, version, architecture} records. + description: Windows AppX/MSIX packages as {name, version, architecture} records — the system-provisioned set plus the collector context's packages, deduplicated. platforms: [windows] conditional: true packages.apps: @@ -812,17 +812,17 @@ packages.apps: conditional: true packages.dpkg: type: array - description: Installed dpkg packages as {name, version, architecture} records (install ok installed only; multiarch siblings kept), from /var/lib/dpkg/status. + description: Installed dpkg packages as {name, version, architecture} records (installed-state entries including held and trigger states; multiarch siblings kept), from /var/lib/dpkg/status. platforms: [linux] conditional: true packages.flatpak: type: array - description: System-installed Flatpak applications and runtimes as {name, version, architecture, branch} records; branch distinguishes same-version siblings. + description: Flatpak applications and runtimes from the system and collector-user installations as {name, version, architecture, branch} records; branch distinguishes same-version siblings. platforms: [linux] conditional: true packages.homebrew: type: array - description: Homebrew formulae and casks as {name, version, type} records, when a Cellar prefix exists. + description: Homebrew formulae and casks as {name, version, type, prefix} records from every detected prefix (/opt/homebrew, /usr/local); prefix distinguishes dual-install duplicates. platforms: [darwin] conditional: true packages.ips: @@ -832,12 +832,12 @@ packages.ips: conditional: true packages.nix: type: array - description: Installed Nix packages as {name, version} records — the NixOS system profile set (the references of /run/current-system/sw), never the whole /nix/store. - platforms: [linux] + description: Installed Nix packages as {name, version} records — the NixOS system profile set, or the default profile (/nix/var/nix/profiles/default) on non-NixOS hosts; never the whole /nix/store. + platforms: [linux, darwin] conditional: true packages.openbsd_pkg: type: array - description: Installed OpenBSD packages as {name, version} records, from /var/db/pkg. + description: Installed OpenBSD packages as {name, version, architecture} records, from /var/db/pkg (architecture from each package's +CONTENTS @arch; omitted for arch-independent packages). platforms: [openbsd] conditional: true packages.pacman: @@ -852,8 +852,8 @@ packages.pkg: conditional: true packages.pkgsrc: type: array - description: Installed pkgsrc packages as {name, version} records, from the discovered PKG_DBDIR. - platforms: [netbsd] + description: Installed pkgsrc packages as {name, version} records, from the discovered PKG_DBDIR; NetBSD's primary source and an illumos/SmartOS and DragonFly secondary. + platforms: [netbsd, dragonfly, illumos] conditional: true packages.receipts: type: array diff --git a/docs/supported-facts/README.md b/docs/supported-facts/README.md index 7127dacf..338de08b 100644 --- a/docs/supported-facts/README.md +++ b/docs/supported-facts/README.md @@ -7,11 +7,11 @@ These pages are generated from [`docs/schema/facts.yaml`](../schema/facts.yaml). | Platform | Supported facts | | --- | ---: | | [Linux](linux.md) | 182 | -| [macOS / Darwin](darwin.md) | 110 | +| [macOS / Darwin](darwin.md) | 111 | | [Windows](windows.md) | 103 | | [FreeBSD](freebsd.md) | 132 | | [OpenBSD](openbsd.md) | 114 | | [NetBSD](netbsd.md) | 118 | -| [DragonFly BSD](dragonfly.md) | 116 | -| [illumos](illumos.md) | 115 | +| [DragonFly BSD](dragonfly.md) | 117 | +| [illumos](illumos.md) | 116 | | [Plan 9](plan9.md) | 29 | diff --git a/docs/supported-facts/darwin.md b/docs/supported-facts/darwin.md index 5567a2b3..f546ec9f 100644 --- a/docs/supported-facts/darwin.md +++ b/docs/supported-facts/darwin.md @@ -63,7 +63,7 @@ $ facts --json ## Fact Contract -110 schema entries include `darwin`. +111 schema entries include `darwin`. | Fact | Type | Conditional | Description | | --- | --- | --- | --- | @@ -154,7 +154,8 @@ $ facts --json | `os.release.major` | `string` | no | The major release number of the operating system. | | `os.release.minor` | `string` | yes | The minor release number of the operating system, when it has one. | | `packages.apps` | `array` | yes | macOS application bundles as {name, version, bundle_id, path} records; secondary to receipts and never merged with it. | -| `packages.homebrew` | `array` | yes | Homebrew formulae and casks as {name, version, type} records, when a Cellar prefix exists. | +| `packages.homebrew` | `array` | yes | Homebrew formulae and casks as {name, version, type, prefix} records from every detected prefix (/opt/homebrew, /usr/local); prefix distinguishes dual-install duplicates. | +| `packages.nix` | `array` | yes | Installed Nix packages as {name, version} records — the NixOS system profile set, or the default profile (/nix/var/nix/profiles/default) on non-NixOS hosts; never the whole /nix/store. | | `packages.receipts` | `array` | yes | macOS installer(8)/PackageKit .pkg receipts as {name, version} records, from /var/db/receipts; the primary macOS source. | | `path` | `array` | no | The PATH environment entries of the Facts process, in lookup order. | | `processors.cores` | `integer` | no | The number of cores per processor socket. | diff --git a/docs/supported-facts/dragonfly.md b/docs/supported-facts/dragonfly.md index b2668d72..6cdd85bb 100644 --- a/docs/supported-facts/dragonfly.md +++ b/docs/supported-facts/dragonfly.md @@ -67,7 +67,7 @@ $ facts --json ## Fact Contract -116 schema entries include `dragonfly`. +117 schema entries include `dragonfly`. | Fact | Type | Conditional | Description | | --- | --- | --- | --- | @@ -162,6 +162,7 @@ $ facts --json | `os.release.major` | `string` | no | The major release number of the operating system. | | `os.release.minor` | `string` | yes | The minor release number of the operating system, when it has one. | | `packages.pkg` | `array` | yes | Installed pkgng packages as {name, version, architecture} records (shared by FreeBSD and DragonFly). | +| `packages.pkgsrc` | `array` | yes | Installed pkgsrc packages as {name, version} records, from the discovered PKG_DBDIR; NetBSD's primary source and an illumos/SmartOS and DragonFly secondary. | | `partitions.*` | `map` | yes | A disk partition (or device-mapper/loop device), keyed by device path. | | `partitions.*.filesystem` | `string` | yes | The filesystem type of the partition. | | `partitions.*.mount` | `string` | yes | The path the partition is mounted on. | diff --git a/docs/supported-facts/illumos.md b/docs/supported-facts/illumos.md index 47023efe..6d1a70a1 100644 --- a/docs/supported-facts/illumos.md +++ b/docs/supported-facts/illumos.md @@ -89,7 +89,7 @@ $ facts --json ## Fact Contract -115 schema entries include `illumos`. +116 schema entries include `illumos`. | Fact | Type | Conditional | Description | | --- | --- | --- | --- | @@ -179,6 +179,7 @@ $ facts --json | `os.release.full` | `string` | no | The full release number of the operating system. | | `os.release.major` | `string` | no | The major release number of the operating system. | | `packages.ips` | `array` | yes | Installed illumos IPS packages as {name, version} records, from the local image (no network refresh). | +| `packages.pkgsrc` | `array` | yes | Installed pkgsrc packages as {name, version} records, from the discovered PKG_DBDIR; NetBSD's primary source and an illumos/SmartOS and DragonFly secondary. | | `partitions.*` | `map` | yes | A disk partition (or device-mapper/loop device), keyed by device path. | | `partitions.*.filesystem` | `string` | yes | The filesystem type of the partition. | | `partitions.*.size` | `string` | yes | The display size of the partition, such as 1.00 GiB. | diff --git a/docs/supported-facts/linux.md b/docs/supported-facts/linux.md index 27326d5a..3d2e8dbb 100644 --- a/docs/supported-facts/linux.md +++ b/docs/supported-facts/linux.md @@ -218,9 +218,9 @@ $ facts --json | `os.selinux.enforced` | `boolean` | yes | Whether SELinux is enforcing, when SELinux is enabled. | | `os.selinux.policy_version` | `string` | yes | The loaded SELinux policy version, when SELinux is enabled. | | `packages.apk` | `array` | yes | Installed Alpine apk packages as {name, version, architecture} records, from /lib/apk/db/installed. | -| `packages.dpkg` | `array` | yes | Installed dpkg packages as {name, version, architecture} records (install ok installed only; multiarch siblings kept), from /var/lib/dpkg/status. | -| `packages.flatpak` | `array` | yes | System-installed Flatpak applications and runtimes as {name, version, architecture, branch} records; branch distinguishes same-version siblings. | -| `packages.nix` | `array` | yes | Installed Nix packages as {name, version} records — the NixOS system profile set (the references of /run/current-system/sw), never the whole /nix/store. | +| `packages.dpkg` | `array` | yes | Installed dpkg packages as {name, version, architecture} records (installed-state entries including held and trigger states; multiarch siblings kept), from /var/lib/dpkg/status. | +| `packages.flatpak` | `array` | yes | Flatpak applications and runtimes from the system and collector-user installations as {name, version, architecture, branch} records; branch distinguishes same-version siblings. | +| `packages.nix` | `array` | yes | Installed Nix packages as {name, version} records — the NixOS system profile set, or the default profile (/nix/var/nix/profiles/default) on non-NixOS hosts; never the whole /nix/store. | | `packages.pacman` | `array` | yes | Installed pacman packages as {name, version, architecture} records, from /var/lib/pacman/local. | | `packages.rpm` | `array` | yes | Installed rpm packages as {name, version, architecture} records (epoch preserved, gpg-pubkey filtered). | | `packages.snap` | `array` | yes | Installed snap packages as {name, version} records. | diff --git a/docs/supported-facts/netbsd.md b/docs/supported-facts/netbsd.md index 65b317a0..bd2bbc85 100644 --- a/docs/supported-facts/netbsd.md +++ b/docs/supported-facts/netbsd.md @@ -175,7 +175,7 @@ $ facts --json | `os.release.full` | `string` | no | The full release number of the operating system. | | `os.release.major` | `string` | no | The major release number of the operating system. | | `os.release.minor` | `string` | yes | The minor release number of the operating system, when it has one. | -| `packages.pkgsrc` | `array` | yes | Installed pkgsrc packages as {name, version} records, from the discovered PKG_DBDIR. | +| `packages.pkgsrc` | `array` | yes | Installed pkgsrc packages as {name, version} records, from the discovered PKG_DBDIR; NetBSD's primary source and an illumos/SmartOS and DragonFly secondary. | | `partitions.*` | `map` | yes | A disk partition (or device-mapper/loop device), keyed by device path. | | `partitions.*.filesystem` | `string` | yes | The filesystem type of the partition. | | `partitions.*.mount` | `string` | yes | The path the partition is mounted on. | diff --git a/docs/supported-facts/openbsd.md b/docs/supported-facts/openbsd.md index 60b5a091..52328e77 100644 --- a/docs/supported-facts/openbsd.md +++ b/docs/supported-facts/openbsd.md @@ -156,7 +156,7 @@ $ facts --json | `os.release.full` | `string` | no | The full release number of the operating system. | | `os.release.major` | `string` | no | The major release number of the operating system. | | `os.release.minor` | `string` | yes | The minor release number of the operating system, when it has one. | -| `packages.openbsd_pkg` | `array` | yes | Installed OpenBSD packages as {name, version} records, from /var/db/pkg. | +| `packages.openbsd_pkg` | `array` | yes | Installed OpenBSD packages as {name, version, architecture} records, from /var/db/pkg (architecture from each package's +CONTENTS @arch; omitted for arch-independent packages). | | `partitions.*` | `map` | yes | A disk partition (or device-mapper/loop device), keyed by device path. | | `partitions.*.filesystem` | `string` | yes | The filesystem type of the partition. | | `partitions.*.mount` | `string` | yes | The path the partition is mounted on. | diff --git a/docs/supported-facts/windows.md b/docs/supported-facts/windows.md index 9ddf22f1..d0b78fb0 100644 --- a/docs/supported-facts/windows.md +++ b/docs/supported-facts/windows.md @@ -134,7 +134,7 @@ $ facts --json | `os.windows.product_name` | `string` | no | The Windows product name, such as Windows Server 2022 Datacenter. | | `os.windows.release_id` | `string` | no | The Windows release identifier (the display version when available). | | `os.windows.system32` | `string` | no | The native system32 directory, sysnative-aware for 32-bit processes. | -| `packages.appx` | `array` | yes | System-provisioned Windows AppX packages as {name, version, architecture} records. | +| `packages.appx` | `array` | yes | Windows AppX/MSIX packages as {name, version, architecture} records — the system-provisioned set plus the collector context's packages, deduplicated. | | `packages.registry` | `array` | yes | Windows uninstall entries from both HKLM hives as {name, version, product_code, architecture} records. | | `path` | `array` | no | The PATH environment entries of the Facts process, in lookup order. | | `processors.cores` | `integer` | no | The number of cores per processor socket. | diff --git a/internal/engine/builtin_groups_test.go b/internal/engine/builtin_groups_test.go index ea80edf5..648e0a04 100644 --- a/internal/engine/builtin_groups_test.go +++ b/internal/engine/builtin_groups_test.go @@ -19,6 +19,10 @@ func TestBuiltinFactGroups_keepStructuredRootsDropLegacyFlatNames(t *testing.T) "operating system": "os", "processor": "processors", "path": "path", + // Facts-native group (ADR-0014): a deliberate parity divergence from + // Ruby Facter's group list, so `--list-block-groups` names packages as + // one disable unit. + "packages": "packages", } for name, root := range wantRoot { facts, ok := groups[name] diff --git a/internal/engine/groups.go b/internal/engine/groups.go index dc469170..85d452e7 100644 --- a/internal/engine/groups.go +++ b/internal/engine/groups.go @@ -25,6 +25,10 @@ func BuiltinFactGroups() []FactGroup { {Name: "memory", Facts: []string{"memory"}}, {Name: "networking", Facts: []string{"networking"}}, {Name: "operating system", Facts: []string{"os"}}, + // Facts-native group (ADR-0014): a deliberate divergence from Ruby + // Facter's group list, so a disable names package collection as one + // unit and --list-block-groups shows it. + {Name: "packages", Facts: []string{"packages"}}, {Name: "path", Facts: []string{"path"}}, {Name: "processor", Facts: []string{"processors"}}, } diff --git a/internal/engine/packages.go b/internal/engine/packages.go index 0578295c..e827f7c2 100644 --- a/internal/engine/packages.go +++ b/internal/engine/packages.go @@ -37,18 +37,25 @@ func packagesCoreFacts(s *Session) []ResolvedFact { if nixProfilePresent(s.stat) { add("nix", nixPackages(s.commandOutput)) } - case "freebsd", "dragonfly": + case "freebsd": add("pkg", pkgngPackages(s.commandOutput)) + case "dragonfly": + add("pkg", pkgngPackages(s.commandOutput)) + add("pkgsrc", pkgsrcPackages(s.readDir)) // ADR-0014: pkgsrc secondary case "openbsd": add("openbsd_pkg", openbsdPackages(s.readDir, s.readFile)) case "netbsd": add("pkgsrc", pkgsrcPackages(s.readDir)) case "illumos": add("ips", ipsPackages(s.commandOutput)) + add("pkgsrc", pkgsrcPackages(s.readDir)) // ADR-0014: SmartOS pkgsrc secondary case "darwin": add("receipts", receiptsPackages(s.glob, s.commandOutput)) add("apps", appsPackages(s.glob, s.commandOutput)) add("homebrew", homebrewPackages(s.glob)) + if nixProfilePresent(s.stat) { // design: nix where a profile is present + add("nix", nixPackages(s.commandOutput)) + } case "windows": add("registry", registryPackages(s.commandOutput)) add("appx", appxPackages(s.commandOutput)) @@ -80,7 +87,7 @@ func sortPackages(records []any) { // name/architecture/version first, then the remaining identity fields, // so siblings that share a name+arch+version still order deterministically // regardless of the reader's append order. - for _, key := range []string{"name", "architecture", "version", "type", "branch", "product_code", "bundle_id", "store_path", "path"} { + for _, key := range []string{"name", "architecture", "version", "type", "branch", "prefix", "product_code", "bundle_id", "store_path", "path"} { if av, bv := packageField(a, key), packageField(b, key); av != bv { return av < bv } @@ -130,11 +137,22 @@ func dpkgPackages(readFile fileReader) []any { } // dpkgInstalled reports whether a dpkg Status line's current-state component -// (the third field of " ok ") is "installed", so held packages -// ("hold ok installed") are kept while removed/config-files entries are dropped. +// (the third field of " ok ") means the package is on disk and +// functional: "installed", plus the two trigger states — a package awaiting or +// pending trigger processing is fully unpacked and configured, and dpkg can sit +// in those states across discoveries. Held packages ("hold ok installed") are +// kept; removed/config-files/half-installed entries are dropped. func dpkgInstalled(status string) bool { fields := strings.Fields(status) - return len(fields) == 3 && fields[2] == "installed" + if len(fields) != 3 { + return false + } + switch fields[2] { + case "installed", "triggers-awaited", "triggers-pending": + return true + default: + return false + } } // rpmPackages runs one epoch-bearing rpm query (a bare `rpm -qa` omits the epoch diff --git a/internal/engine/packages_bsd.go b/internal/engine/packages_bsd.go index 95a0a163..bc66ce24 100644 --- a/internal/engine/packages_bsd.go +++ b/internal/engine/packages_bsd.go @@ -56,13 +56,16 @@ func openbsdPackages(readDir func(string) ([]os.DirEntry, error), readFile fileR return records } -// pkgsrcPackages enumerates the installed NetBSD pkgsrc packages recorded as -// - subdirectories of PKG_DBDIR. PKG_DBDIR is not hardcoded to a -// single path: the standard candidates are probed in order (the pkgsrc default -// /usr/pkg/pkgdb, then the legacy /var/db/pkg), using the first that lists any -// package entries. +// pkgsrcPackages enumerates the installed pkgsrc packages recorded as +// - subdirectories of PKG_DBDIR. It is NetBSD's primary source +// and an illumos/SmartOS and DragonFly secondary (ADR-0014). PKG_DBDIR is not +// hardcoded to a single path: the standard candidates are probed in order — +// the pkgsrc default /usr/pkg/pkgdb, the SmartOS prefix /opt/local/pkgdb (and +// its pre-2019 /opt/local/pkg), then the legacy /var/db/pkg — using the first +// that lists any package entries. On a pkgng host /var/db/pkg holds only +// sqlite files (no - directories), so it contributes nothing. func pkgsrcPackages(readDir func(string) ([]os.DirEntry, error)) []any { - for _, dbdir := range []string{"/usr/pkg/pkgdb", "/var/db/pkg"} { + for _, dbdir := range []string{"/usr/pkg/pkgdb", "/opt/local/pkgdb", "/opt/local/pkg", "/var/db/pkg"} { entries, err := readDir(dbdir) if err != nil { continue diff --git a/internal/engine/packages_bsd_test.go b/internal/engine/packages_bsd_test.go index 8f628796..7a7be8a9 100644 --- a/internal/engine/packages_bsd_test.go +++ b/internal/engine/packages_bsd_test.go @@ -153,6 +153,8 @@ func TestPkgsrcPackages(t *testing.T) { dbdir string }{ {name: "default pkgdb", dbdir: "/usr/pkg/pkgdb"}, + {name: "smartos pkgdb", dbdir: "/opt/local/pkgdb"}, + {name: "smartos legacy", dbdir: "/opt/local/pkg"}, {name: "legacy fallback", dbdir: "/var/db/pkg"}, } for _, tt := range tests { diff --git a/internal/engine/packages_extra.go b/internal/engine/packages_extra.go index 35b65b62..e2ac0fe1 100644 --- a/internal/engine/packages_extra.go +++ b/internal/engine/packages_extra.go @@ -91,9 +91,27 @@ func flatpakPackages(run commandRunner) []any { // derivations (name-version-doc, -man, -bin, ...) collapse to one record via // dedup on name+version; unversioned environment members are skipped. func nixPackages(run commandRunner) []any { - // nix-store lives in the NixOS system profile, outside the engine's trusted - // command PATH (/usr/sbin:/usr/bin:/sbin:/bin); call it by absolute path. + // Both nix tools live outside the engine's trusted command PATH + // (/usr/sbin:/usr/bin:/sbin:/bin), so they are called by absolute path. + // NixOS first: the system profile is canonical there, and stopping on it + // avoids double-reporting nix itself from the default profile. out := run("/run/current-system/sw/bin/nix-store", "-q", "--references", "/run/current-system/sw") + if records := nixRecordsFromLines(out, true); records != nil { + return records + } + // Non-NixOS daemon/multi-user nix (ADR-0014: the profile set includes the + // default profile): nix-env lists the default profile's - + // elements directly — this profile carries the manifest.nix that the NixOS + // system buildEnv lacks. + out = run("/nix/var/nix/profiles/default/bin/nix-env", "-q", "--profile", "/nix/var/nix/profiles/default") + return nixRecordsFromLines(out, false) +} + +// nixRecordsFromLines converts nix enumeration output to sorted records. With +// storePaths true, each line is /nix/store/--[-] +// and the hash is dropped; otherwise each line is a bare - +// profile element. Split-output duplicates collapse via dedup on name+version. +func nixRecordsFromLines(out string, storePaths bool) []any { var records []any seen := map[string]bool{} for line := range strings.Lines(out) { @@ -101,10 +119,13 @@ func nixPackages(run commandRunner) []any { if line == "" { continue } - base := line[strings.LastIndexByte(line, '/')+1:] - _, tail, ok := strings.Cut(base, "-") // drop the store hash - if !ok { - continue + tail := line + if storePaths { + base := line[strings.LastIndexByte(line, '/')+1:] + var ok bool + if _, tail, ok = strings.Cut(base, "-"); !ok { // drop the store hash + continue + } } name, version := parseNixNameVersion(tail) if name == "" || version == "" { @@ -168,8 +189,12 @@ func flatpakPresent(stat func(string) (os.FileInfo, error)) bool { return dirPresent(stat, "/var/lib/flatpak") } +// nixProfilePresent opens the nix gate when either profile exists: the NixOS +// system profile or the daemon-install default profile (ADR-0014 names both as +// the installed profile set). os.Stat follows the profile symlinks. func nixProfilePresent(stat func(string) (os.FileInfo, error)) bool { - return dirPresent(stat, "/nix/var/nix/profiles/system") + return dirPresent(stat, "/nix/var/nix/profiles/system") || + dirPresent(stat, "/nix/var/nix/profiles/default") } func dirPresent(stat func(string) (os.FileInfo, error), path string) bool { diff --git a/internal/engine/packages_extra_test.go b/internal/engine/packages_extra_test.go index 9fdcb911..2d13a8df 100644 --- a/internal/engine/packages_extra_test.go +++ b/internal/engine/packages_extra_test.go @@ -70,6 +70,63 @@ func TestParseNixNameVersion_digitLeadingNames(t *testing.T) { } } +// nixEnvDefaultProfileFixture is verbatim `nix-env -q --profile +// /nix/var/nix/profiles/default` output from a daemon (multi-user) nix install +// on the nlab ubuntu2404 guest — the non-NixOS shape (ADR-0014: the installed +// profile set is the default profile AND the NixOS system profile). +const nixEnvDefaultProfileFixture = `nix-2.34.7 +nix-manual-2.34.7-man +nss-cacert-3.117 +` + +func TestNixPackages_fallsBackToDefaultProfileOnNonNixOS(t *testing.T) { + t.Parallel() + var calls []string + got := nixPackages(func(name string, args ...string) string { + calls = append(calls, name) + switch name { + case "/run/current-system/sw/bin/nix-store": + return "" // not NixOS: no system profile environment + case "/nix/var/nix/profiles/default/bin/nix-env": + if !reflect.DeepEqual(args, []string{"-q", "--profile", "/nix/var/nix/profiles/default"}) { + t.Fatalf("nix-env args = %v", args) + } + return nixEnvDefaultProfileFixture + default: + t.Fatalf("unexpected command %q", name) + return "" + } + }) + want := []any{ + map[string]any{"name": "nix", "version": "2.34.7"}, + map[string]any{"name": "nix-manual", "version": "2.34.7"}, // -man output suffix stripped + map[string]any{"name": "nss-cacert", "version": "3.117"}, + } + if !reflect.DeepEqual(got, want) { + t.Fatalf("nixPackages(default profile) = %#v\nwant %#v", got, want) + } + if len(calls) != 2 { + t.Fatalf("calls = %v, want nix-store then nix-env fallback", calls) + } +} + +func TestNixPackages_systemProfileWinsWithoutFallbackSpawn(t *testing.T) { + t.Parallel() + var calls []string + got := nixPackages(func(name string, args ...string) string { + calls = append(calls, name) + return "/nix/store/mxq1r9w2w2y9lsqb5fkcyb5xbbki1n57-ncurses-6.6\n" + }) + if want := []any{map[string]any{"name": "ncurses", "version": "6.6"}}; !reflect.DeepEqual(got, want) { + t.Fatalf("nixPackages(system) = %#v, want %#v", got, want) + } + // NixOS: the system profile is canonical; the default-profile fallback must + // not run (it could double-report nix itself). + if want := []string{"/run/current-system/sw/bin/nix-store"}; !reflect.DeepEqual(calls, want) { + t.Fatalf("calls = %v, want %v", calls, want) + } +} + func TestNixPackages_emptyProfileYieldsNothing(t *testing.T) { t.Parallel() if got := nixPackages(func(string, ...string) string { return "" }); got != nil { @@ -161,7 +218,6 @@ func TestExtraPackageSourcePresenceGates(t *testing.T) { }{ {"snap", snapdPresent, "/var/lib/snapd"}, {"flatpak", flatpakPresent, "/var/lib/flatpak"}, - {"nix", nixProfilePresent, "/nix/var/nix/profiles/system"}, } for _, tc := range cases { present := func(path string) (os.FileInfo, error) { @@ -185,3 +241,26 @@ func TestExtraPackageSourcePresenceGates(t *testing.T) { } } } + +func TestNixProfilePresent_eitherSystemOrDefault(t *testing.T) { + t.Parallel() + // ADR-0014: the nix profile set is the NixOS system profile AND the + // default profile — the gate must open when either exists. + only := func(dir string) func(string) (os.FileInfo, error) { + return func(path string) (os.FileInfo, error) { + if path == dir { + return fakeFileInfo{name: dir, mode: os.ModeDir, isDir: true}, nil + } + return nil, os.ErrNotExist + } + } + if !nixProfilePresent(only("/nix/var/nix/profiles/system")) { + t.Fatal("nixProfilePresent = false with only the NixOS system profile") + } + if !nixProfilePresent(only("/nix/var/nix/profiles/default")) { + t.Fatal("nixProfilePresent = false with only the default profile") + } + if nixProfilePresent(func(string) (os.FileInfo, error) { return nil, os.ErrNotExist }) { + t.Fatal("nixProfilePresent = true with neither profile") + } +} diff --git a/internal/engine/packages_mac.go b/internal/engine/packages_mac.go index aa33196f..297936c5 100644 --- a/internal/engine/packages_mac.go +++ b/internal/engine/packages_mac.go @@ -16,20 +16,55 @@ func receiptsPackages(glob pathGlobber, run commandRunner) []any { return nil } var records []any - for _, chunk := range plutilArgChunks(paths) { - out := run("plutil", append([]string{"-p"}, chunk...)...) - parsePlutilBlocks(out, func(fields map[string]string) { - name, version := fields["PackageIdentifier"], fields["PackageVersion"] - if name == "" || version == "" { - return - } - records = append(records, packageRecord(name, version)) - }) - } + runPlutilChunks(run, paths, func(_ []string, _ int, fields map[string]string) { + name, version := fields["PackageIdentifier"], fields["PackageVersion"] + if name == "" || version == "" { + return + } + records = append(records, packageRecord(name, version)) + }) sortPackages(records) return records } +// runPlutilChunks batches plist paths through `plutil -p` (argv-chunked by +// plutilArgChunks) and invokes perBlock for every parsed block with the +// successful invocation's own path list and the block's invocation-local +// index. plutil prints blocks in argument order and never echoes filenames, so +// this positional pairing is the only path<->block link — and it only holds +// per invocation, which is why perBlock receives the invocation's paths rather +// than a global offset. +// +// plutil is all-or-nothing per invocation: it exits non-zero if ANY file fails, +// and the engine's run() returns "" on non-zero exit. On success `plutil -p` +// always prints one non-empty block per input file, so empty output on a +// non-empty chunk unambiguously means failure. To keep one corrupt file from +// dropping every good file in its chunk, a failed multi-path chunk is bisected +// and each half retried (O(log n) extra spawns per bad file); a failed +// single-path chunk is the corrupt/unreadable file itself and is skipped. +func runPlutilChunks(run commandRunner, paths []string, perBlock func(chunkPaths []string, blockIndex int, fields map[string]string)) { + for _, chunk := range plutilArgChunks(paths) { + runPlutilChunk(run, chunk, perBlock) + } +} + +func runPlutilChunk(run commandRunner, chunk []string, perBlock func([]string, int, map[string]string)) { + out := run("plutil", append([]string{"-p"}, chunk...)...) + if out == "" { + if len(chunk) > 1 { + mid := len(chunk) / 2 + runPlutilChunk(run, chunk[:mid], perBlock) + runPlutilChunk(run, chunk[mid:], perBlock) + } + return + } + index := 0 + parsePlutilBlocks(out, func(fields map[string]string) { + perBlock(chunk, index, fields) + index++ + }) +} + // plutilArgChunks splits plist paths into batches whose joined argv length stays // well under ARG_MAX, so one `plutil -p ` can never overflow the command // line and silently drop the whole source on a host with a large receipt/app set. @@ -54,20 +89,25 @@ func plutilArgChunks(paths []string) [][]string { // appsPackages resolves the "apps" source: installed application bundles. It is // a secondary view of installed software and is never merged into "receipts". -// Both /Applications and /System/Applications are scanned, and every matched -// Info.plist is read by a single `plutil -p`. +// /Applications and /System/Applications are scanned along with their +// Utilities subfolders (glob "*" is single-level, so Utilities needs its own +// pattern), and every matched Info.plist is read by a single `plutil -p`. The +// patterns match only *.app bundles — nothing else has that Contents layout. // -// plutil -p prints one dict block per file, in argument order, and does NOT -// echo filenames. osHost.run (cmd.Output) returns "" unless plutil exits 0, so -// a non-empty result means every file parsed successfully; the block stream -// therefore aligns one-to-one with the glob order, letting the .app path be -// recovered positionally. Only the top-level (depth-1) keys of each block are -// consulted — Info.plists nest arbitrarily deep. +// plutil -p prints one block per file, in argument order, and does NOT echo +// filenames. osHost.run (cmd.Output) returns "" unless plutil exits 0, so a +// non-empty result means every file in that invocation parsed successfully; +// the block stream therefore aligns one-to-one with the invocation's argument +// order, letting the .app path be recovered positionally. Only the top-level +// (depth-1) keys of each block are consulted — Info.plists nest arbitrarily +// deep. func appsPackages(glob pathGlobber, run commandRunner) []any { var paths []string for _, pattern := range []string{ - "/Applications/*/Contents/Info.plist", - "/System/Applications/*/Contents/Info.plist", + "/Applications/*.app/Contents/Info.plist", + "/Applications/Utilities/*.app/Contents/Info.plist", + "/System/Applications/*.app/Contents/Info.plist", + "/System/Applications/Utilities/*.app/Contents/Info.plist", } { matches, err := glob(pattern) if err != nil { @@ -79,29 +119,24 @@ func appsPackages(glob pathGlobber, run commandRunner) []any { return nil } var records []any - // Chunk so a large app set never overflows argv. plutil is all-or-nothing - // per invocation (cmd.Output is "" on any parse failure), so blocks align - // one-to-one with the chunk's paths; a failed chunk yields no blocks and - // cannot desynchronise a later chunk (the path index is chunk-local). - for _, chunk := range plutilArgChunks(paths) { - out := run("plutil", append([]string{"-p"}, chunk...)...) - index := 0 - parsePlutilBlocks(out, func(fields map[string]string) { - if index >= len(chunk) { - return - } - appPath := appBundlePath(chunk[index]) - index++ - name := firstNonEmpty(fields["CFBundleName"], fields["CFBundleDisplayName"], appBundleName(appPath)) - version := firstNonEmpty(fields["CFBundleShortVersionString"], fields["CFBundleVersion"]) - if name == "" || version == "" { - return - } - records = append(records, packageRecord(name, version, - "bundle_id", fields["CFBundleIdentifier"], - "path", appPath)) - }) - } + // runPlutilChunks keeps argv under ARG_MAX and bisects around corrupt + // plists; blocks align one-to-one with each successful invocation's paths, + // so the block index is always invocation-local and cannot desynchronise a + // later chunk. + runPlutilChunks(run, paths, func(chunkPaths []string, blockIndex int, fields map[string]string) { + if blockIndex >= len(chunkPaths) { + return + } + appPath := appBundlePath(chunkPaths[blockIndex]) + name := firstNonEmpty(fields["CFBundleName"], fields["CFBundleDisplayName"], appBundleName(appPath)) + version := firstNonEmpty(fields["CFBundleShortVersionString"], fields["CFBundleVersion"]) + if name == "" || version == "" { + return + } + records = append(records, packageRecord(name, version, + "bundle_id", fields["CFBundleIdentifier"], + "path", appPath)) + }) sortPackages(records) return records } @@ -116,12 +151,12 @@ func homebrewPackages(glob pathGlobber) []any { var records []any for _, prefix := range []string{"/opt/homebrew", "/usr/local"} { if formulae, err := glob(prefix + "/Cellar/*/*"); err == nil { - records = appendBrewRecords(records, formulae, "formula") + records = appendBrewRecords(records, formulae, "formula", prefix) } // Probe Caskroom independently of Cellar: a cask-only install has an // empty Cellar but real casks to report. if casks, err := glob(prefix + "/Caskroom/*/*"); err == nil { - records = appendBrewRecords(records, casks, "cask") + records = appendBrewRecords(records, casks, "cask", prefix) } } if len(records) == 0 { @@ -133,35 +168,63 @@ func homebrewPackages(glob pathGlobber) []any { // appendBrewRecords turns / leaf paths into records. Homebrew // keeps a dotfile sidecar (Caskroom//.metadata) that filepath.Glob's "*" -// matches — unlike the shell — so dot-prefixed leaves are skipped. -func appendBrewRecords(records []any, paths []string, kind string) []any { +// matches — unlike the shell — so dot-prefixed leaves are skipped. The prefix +// is recorded as an identity field: an Apple Silicon (/opt/homebrew) and a +// Rosetta Intel (/usr/local) brew can carry the same formula@version, and +// without it the two installs would be byte-identical duplicates. +func appendBrewRecords(records []any, paths []string, kind, prefix string) []any { for _, p := range paths { version := path.Base(p) name := path.Base(path.Dir(p)) if name == "" || version == "" || strings.HasPrefix(version, ".") { continue } - records = append(records, packageRecord(name, version, "type", kind)) + records = append(records, packageRecord(name, version, "type", kind, "prefix", prefix)) } return records } -// parsePlutilBlocks splits the text of `plutil -p file...` into one dict per -// input file and invokes fn with that dict's top-level (depth-1) scalar keys, -// in file order. plutil prints the outer dict's braces alone on a line and -// opens nested containers with a "key" => { or "key" => [ suffix; depth is -// tracked structurally rather than by column, so multi-line string values (an -// unindented copyright continuation, say) cannot desynchronise block or key -// boundaries. +// parsePlutilBlocks splits the text of `plutil -p file...` into one block per +// input file and invokes fn with that block's top-level (depth-1) scalar keys, +// in file order. plutil prints the outer container's delimiters alone on a +// line and opens nested containers with a "key" => { or "key" => [ suffix; +// depth is tracked structurally rather than by column, so multi-line string +// values (an unindented copyright continuation, say) cannot desynchronise +// block or key boundaries. +// +// Two hostile shapes are handled explicitly: +// - A multi-line string value can contain lines that are exactly "{", "}", +// "[", or "]". When a value opens a quote it does not close on the same +// line, the parser enters in-string mode and suspends ALL structural +// interpretation until a line ends with an unescaped closing quote. +// - A plist whose root is an array prints a bare "[" ... "]" block. It +// carries no dict keys, but it still opens a block (fields stay empty) so +// fn fires at its close and positional path pairing stays in step. func parsePlutilBlocks(out string, fn func(fields map[string]string)) { var fields map[string]string depth := 0 + rootIsArray := false + inString := false for line := range strings.Lines(out) { + if inString { + if endsWithUnescapedQuote(strings.TrimRight(line, "\r\n")) { + inString = false + } + continue + } trimmed := strings.TrimSpace(line) switch { case trimmed == "{": if depth == 0 { fields = map[string]string{} + rootIsArray = false + } + depth++ + case trimmed == "[": + // Array-rooted plist: open a keyless block so pairing advances. + if depth == 0 { + fields = map[string]string{} + rootIsArray = true } depth++ case trimmed == "}" || trimmed == "]": @@ -176,18 +239,46 @@ func parsePlutilBlocks(out string, fn func(fields map[string]string)) { // A depth-1 scalar is a real top-level key; a "key" => { or // "key" => [ line opens a nested container we descend past. opensContainer := strings.HasSuffix(trimmed, " => {") || strings.HasSuffix(trimmed, " => [") - if depth == 1 && !opensContainer { + if opensContainer { + depth++ + continue + } + if depth == 1 && !rootIsArray { if key, value, ok := plutilKeyValue(trimmed); ok { fields[key] = value } } - if opensContainer { - depth++ + // A string value whose closing quote is not on this line spans + // further physical lines; suspend structure until it closes. + if _, rawValue, found := strings.Cut(trimmed, " => "); found && stringRemainsOpen(rawValue) { + inString = true } } } } +// stringRemainsOpen reports whether a plutil value beginning with a quote +// continues onto later physical lines (its closing quote is not on this line). +func stringRemainsOpen(rawValue string) bool { + if !strings.HasPrefix(rawValue, `"`) { + return false + } + return !endsWithUnescapedQuote(rawValue[1:]) +} + +// endsWithUnescapedQuote reports whether s terminates with a `"` that is not +// escaped — an odd run of backslashes before the quote escapes it. +func endsWithUnescapedQuote(s string) bool { + if !strings.HasSuffix(s, `"`) { + return false + } + backslashes := 0 + for i := len(s) - 2; i >= 0 && s[i] == '\\'; i-- { + backslashes++ + } + return backslashes%2 == 0 +} + // plutilKeyValue parses a `"key" => value` line, unquoting both sides. It // reports ok=false for lines without the " => " separator (blank-line padding, // wrapped multi-line string continuations). diff --git a/internal/engine/packages_mac_test.go b/internal/engine/packages_mac_test.go index 6ccc9a10..839c0d91 100644 --- a/internal/engine/packages_mac_test.go +++ b/internal/engine/packages_mac_test.go @@ -2,6 +2,7 @@ package engine import ( "reflect" + "strings" "testing" ) @@ -74,20 +75,88 @@ func TestReceiptsPackages(t *testing.T) { } } +// plutilStub emulates the engine's run() around a real plutil: it prints one +// block per path in argument order, but returns "" for the WHOLE invocation +// when any argument is a corrupt path (plutil exits non-zero on any bad file, +// and run() yields "" on non-zero exit). Every invocation's path list is +// recorded so tests can assert the bisection shape. +func plutilStub(blocks map[string]string, corrupt map[string]bool, invocations *[][]string) commandRunner { + return func(_ string, args ...string) string { + paths := args[1:] // args[0] is "-p" + if invocations != nil { + *invocations = append(*invocations, append([]string(nil), paths...)) + } + var out strings.Builder + for _, p := range paths { + if corrupt[p] { + return "" + } + out.WriteString(blocks[p]) + } + return out.String() + } +} + +func TestReceiptsPackages_corruptPlistBisection(t *testing.T) { + // plutil is all-or-nothing per invocation: one corrupt receipt would + // otherwise drop the whole chunk. The reader must bisect the failed chunk + // and lose only the corrupt file. + paths := []string{ + "/var/db/receipts/com.example.good1.plist", + "/var/db/receipts/com.example.corrupt.plist", + "/var/db/receipts/com.example.good2.plist", + } + blocks := map[string]string{ + paths[0]: `{ + "PackageIdentifier" => "com.example.good1" + "PackageVersion" => "1.0" +} +`, + paths[2]: `{ + "PackageIdentifier" => "com.example.good2" + "PackageVersion" => "2.0" +} +`, + } + var invocations [][]string + got := receiptsPackages( + globStub(map[string][]string{"/var/db/receipts/*.plist": paths}), + plutilStub(blocks, map[string]bool{paths[1]: true}, &invocations)) + + want := []any{ + packageRecord("com.example.good1", "1.0"), + packageRecord("com.example.good2", "2.0"), + } + if !reflect.DeepEqual(got, want) { + t.Fatalf("receiptsPackages = %#v, want %#v", got, want) + } + wantInvocations := [][]string{ + {paths[0], paths[1], paths[2]}, // full chunk fails + {paths[0]}, // left half: good1 recovered + {paths[1], paths[2]}, // right half fails + {paths[1]}, // corrupt alone: skipped + {paths[2]}, // good2 recovered + } + if !reflect.DeepEqual(invocations, wantInvocations) { + t.Fatalf("plutil invocations = %v, want %v", invocations, wantInvocations) + } +} + func TestReceiptsPackagesAbsent(t *testing.T) { if got := receiptsPackages(globStub(nil), runStub("", nil)); got != nil { t.Fatalf("receiptsPackages with no receipts = %#v, want nil", got) } } -// appsFixture blocks are in glob-concatenation order (all /Applications first, -// then /System/Applications): 1Password, DisplayOnly, Nameless, Calculator. It -// mixes a real block (1Password, whose path has a space) and Calculator from -// /System (with both CFBundleName and a multi-line copyright whose continuation -// line is unindented) with synthetic blocks exercising every fallback: -// CFBundleDisplayName-only, name derived from the .app path, and -// CFBundleVersion-only. Nested array and dict values verify the depth tracker -// neither leaks nested keys nor splits blocks. +// appsFixture blocks are in glob-concatenation order (/Applications, then +// /Applications/Utilities, then /System/Applications, then +// /System/Applications/Utilities): 1Password, DisplayOnly, Nameless, Disk +// Helper, Calculator, Terminal. It mixes a real block (1Password, whose path +// has a space) and Calculator from /System (with both CFBundleName and a +// multi-line copyright whose continuation line is unindented) with synthetic +// blocks exercising every fallback: CFBundleDisplayName-only, name derived +// from the .app path, and CFBundleVersion-only. Nested array and dict values +// verify the depth tracker neither leaks nested keys nor splits blocks. const appsFixture = `{ "CFBundleIdentifier" => "com.agilebits.onepassword7" "CFBundleName" => "1Password 7" @@ -111,6 +180,11 @@ const appsFixture = `{ "CFBundleIdentifier" => "com.example.nameless" "CFBundleVersion" => "9" } +{ + "CFBundleIdentifier" => "com.example.diskhelper" + "CFBundleName" => "Disk Helper" + "CFBundleShortVersionString" => "1.1" +} { "CFBundleDisplayName" => "Calculator" "CFBundleIdentifier" => "com.apple.calculator" @@ -120,6 +194,12 @@ const appsFixture = `{ "NSHumanReadableCopyright" => "Copyright © 2022-2025 Apple Inc. All rights reserved." } +{ + "CFBundleIdentifier" => "com.apple.Terminal" + "CFBundleName" => "Terminal" + "CFBundleShortVersionString" => "2.14" + "CFBundleVersion" => "455" +} ` var appsPaths = []string{ @@ -127,19 +207,24 @@ var appsPaths = []string{ "/System/Applications/Calculator.app/Contents/Info.plist", "/Applications/DisplayOnly.app/Contents/Info.plist", "/Applications/Nameless.app/Contents/Info.plist", + "/Applications/Utilities/Disk Helper.app/Contents/Info.plist", + "/System/Applications/Utilities/Terminal.app/Contents/Info.plist", } func TestAppsPackages(t *testing.T) { glob := globStub(map[string][]string{ - "/Applications/*/Contents/Info.plist": {appsPaths[0], appsPaths[2], appsPaths[3]}, - "/System/Applications/*/Contents/Info.plist": {appsPaths[1]}, + "/Applications/*.app/Contents/Info.plist": {appsPaths[0], appsPaths[2], appsPaths[3]}, + "/Applications/Utilities/*.app/Contents/Info.plist": {appsPaths[4]}, + "/System/Applications/*.app/Contents/Info.plist": {appsPaths[1]}, + "/System/Applications/Utilities/*.app/Contents/Info.plist": {appsPaths[5]}, }) var argv []string got := appsPackages(glob, runStub(appsFixture, &argv)) - // argv (and therefore plutil block order) is /Applications first, then - // /System/Applications, matching the fixture order. - wantArgv := []string{appsPaths[0], appsPaths[2], appsPaths[3], appsPaths[1]} + // argv (and therefore plutil block order) follows the glob-pattern order: + // /Applications, /Applications/Utilities, /System/Applications, + // /System/Applications/Utilities — matching the fixture order. + wantArgv := []string{appsPaths[0], appsPaths[2], appsPaths[3], appsPaths[4], appsPaths[1], appsPaths[5]} if !reflect.DeepEqual(argv, wantArgv) { t.Fatalf("plutil argv = %v, want %v", argv, wantArgv) } @@ -151,12 +236,18 @@ func TestAppsPackages(t *testing.T) { packageRecord("Calculator", "12.0", "bundle_id", "com.apple.calculator", "path", "/System/Applications/Calculator.app"), + packageRecord("Disk Helper", "1.1", // /Applications/Utilities bundle + "bundle_id", "com.example.diskhelper", + "path", "/Applications/Utilities/Disk Helper.app"), packageRecord("Display Only", "3", // CFBundleDisplayName fallback "bundle_id", "com.example.displayonly", "path", "/Applications/DisplayOnly.app"), packageRecord("Nameless", "9", // name derived from .app path "bundle_id", "com.example.nameless", "path", "/Applications/Nameless.app"), + packageRecord("Terminal", "2.14", // /System/Applications/Utilities bundle + "bundle_id", "com.apple.Terminal", + "path", "/System/Applications/Utilities/Terminal.app"), } sortPackages(want) if !reflect.DeepEqual(got, want) { @@ -175,7 +266,7 @@ func TestAppsPackages_skipsAppWithoutVersion(t *testing.T) { // dropped (name+version invariant); its slot must not shift the positional // block<->path pairing for the remaining apps. glob := globStub(map[string][]string{ - "/Applications/*/Contents/Info.plist": { + "/Applications/*.app/Contents/Info.plist": { "/Applications/HasVer.app/Contents/Info.plist", "/Applications/NoVer.app/Contents/Info.plist", }, @@ -197,6 +288,117 @@ func TestAppsPackages_skipsAppWithoutVersion(t *testing.T) { } } +func TestAppsPackages_corruptPlistBisection(t *testing.T) { + // One corrupt Info.plist fails the whole plutil invocation. The reader must + // bisect the failed chunk, keep every good app's record paired with its OWN + // path (pairing is per successful invocation), and skip only the corrupt + // file. + paths := []string{ + "/Applications/Alpha.app/Contents/Info.plist", + "/Applications/Broken.app/Contents/Info.plist", + "/Applications/Gamma.app/Contents/Info.plist", + "/Applications/Delta.app/Contents/Info.plist", + } + blocks := map[string]string{ + paths[0]: `{ + "CFBundleName" => "Alpha" + "CFBundleShortVersionString" => "1.0" +} +`, + paths[2]: `{ + "CFBundleName" => "Gamma" + "CFBundleShortVersionString" => "3.0" +} +`, + paths[3]: `{ + "CFBundleName" => "Delta" + "CFBundleShortVersionString" => "4.0" +} +`, + } + glob := globStub(map[string][]string{"/Applications/*.app/Contents/Info.plist": paths}) + var invocations [][]string + got := appsPackages(glob, plutilStub(blocks, map[string]bool{paths[1]: true}, &invocations)) + + want := []any{ + packageRecord("Alpha", "1.0", "path", "/Applications/Alpha.app"), + packageRecord("Delta", "4.0", "path", "/Applications/Delta.app"), + packageRecord("Gamma", "3.0", "path", "/Applications/Gamma.app"), + } + if !reflect.DeepEqual(got, want) { + t.Fatalf("appsPackages = %#v, want %#v", got, want) + } + wantInvocations := [][]string{ + {paths[0], paths[1], paths[2], paths[3]}, // full chunk fails + {paths[0], paths[1]}, // left half fails + {paths[0]}, // Alpha recovered + {paths[1]}, // corrupt alone: skipped + {paths[2], paths[3]}, // right half succeeds intact + } + if !reflect.DeepEqual(invocations, wantInvocations) { + t.Fatalf("plutil invocations = %v, want %v", invocations, wantInvocations) + } +} + +func TestAppsPackages_arrayRootedPlistKeepsPairing(t *testing.T) { + // `plutil -p` of a plist whose root is an array prints a "[" ... "]" block + // with no dict keys. It must still count as a block: the array-rooted app + // yields no record (no name/version), but every LATER app in the chunk must + // keep its own path — not inherit the array-rooted app's. + glob := globStub(map[string][]string{ + "/Applications/*.app/Contents/Info.plist": { + "/Applications/First.app/Contents/Info.plist", + "/Applications/Weird.app/Contents/Info.plist", + "/Applications/Third.app/Contents/Info.plist", + }, + }) + run := func(string, ...string) string { + return `{ + "CFBundleName" => "First" + "CFBundleShortVersionString" => "1.0" +} +[ + 0 => "array-rooted plist" +] +{ + "CFBundleName" => "Third" + "CFBundleShortVersionString" => "3.0" +} +` + } + got := appsPackages(glob, run) + want := []any{ + packageRecord("First", "1.0", "path", "/Applications/First.app"), + packageRecord("Third", "3.0", "path", "/Applications/Third.app"), + } + if !reflect.DeepEqual(got, want) { + t.Fatalf("appsPackages = %#v, want %#v", got, want) + } +} + +func TestParsePlutilBlocks_arrayRootBlock(t *testing.T) { + // An array-rooted block fires fn with an EMPTY fields map (array elements + // like `0 => "x"` are not dict keys) and must not disturb its neighbours. + const out = `{ + "A" => "1" +} +[ + 0 => "not a key" +] +{ + "B" => "2" +} +` + var got []map[string]string + parsePlutilBlocks(out, func(f map[string]string) { + got = append(got, f) + }) + want := []map[string]string{{"A": "1"}, {}, {"B": "2"}} + if !reflect.DeepEqual(got, want) { + t.Fatalf("blocks = %#v, want %#v", got, want) + } +} + func TestHomebrewPackages(t *testing.T) { glob := globStub(map[string][]string{ "/opt/homebrew/Cellar/*/*": { @@ -212,10 +414,10 @@ func TestHomebrewPackages(t *testing.T) { got := homebrewPackages(glob) want := []any{ - packageRecord("ada-url", "3.4.4", "type", "formula"), - packageRecord("amethyst", "0.24.3", "type", "cask"), - packageRecord("brotli", "1.2.0", "type", "formula"), - packageRecord("ghostty", "1.3.1", "type", "cask"), + packageRecord("ada-url", "3.4.4", "type", "formula", "prefix", "/opt/homebrew"), + packageRecord("amethyst", "0.24.3", "type", "cask", "prefix", "/opt/homebrew"), + packageRecord("brotli", "1.2.0", "type", "formula", "prefix", "/opt/homebrew"), + packageRecord("ghostty", "1.3.1", "type", "cask", "prefix", "/opt/homebrew"), } sortPackages(want) if !reflect.DeepEqual(got, want) { @@ -228,19 +430,38 @@ func TestHomebrewPackagesIntelPrefix(t *testing.T) { "/usr/local/Cellar/*/*": {"/usr/local/Cellar/wget/1.25.0"}, }) got := homebrewPackages(glob) - want := []any{packageRecord("wget", "1.25.0", "type", "formula")} + want := []any{packageRecord("wget", "1.25.0", "type", "formula", "prefix", "/usr/local")} if !reflect.DeepEqual(got, want) { t.Fatalf("homebrewPackages (intel) = %#v, want %#v", got, want) } } +func TestHomebrewPackagesDualPrefix(t *testing.T) { + // Apple Silicon brew (/opt/homebrew) and Rosetta Intel brew (/usr/local) + // can coexist with the same formula@version in both Cellars. The two + // installs are distinct and must yield two records distinguished by the + // prefix identity field rather than collapsing into byte-identical twins. + glob := globStub(map[string][]string{ + "/opt/homebrew/Cellar/*/*": {"/opt/homebrew/Cellar/wget/1.25.0"}, + "/usr/local/Cellar/*/*": {"/usr/local/Cellar/wget/1.25.0"}, + }) + got := homebrewPackages(glob) + want := []any{ + packageRecord("wget", "1.25.0", "type", "formula", "prefix", "/opt/homebrew"), + packageRecord("wget", "1.25.0", "type", "formula", "prefix", "/usr/local"), + } + if !reflect.DeepEqual(got, want) { + t.Fatalf("homebrewPackages (dual prefix) = %#v, want %#v", got, want) + } +} + func TestHomebrewPackagesCaskOnly(t *testing.T) { // A cask-only install has an empty Cellar but real casks to report. glob := globStub(map[string][]string{ "/opt/homebrew/Caskroom/*/*": {"/opt/homebrew/Caskroom/amethyst/0.24.3"}, }) got := homebrewPackages(glob) - want := []any{packageRecord("amethyst", "0.24.3", "type", "cask")} + want := []any{packageRecord("amethyst", "0.24.3", "type", "cask", "prefix", "/opt/homebrew")} if !reflect.DeepEqual(got, want) { t.Fatalf("homebrewPackages (cask-only) = %#v, want %#v", got, want) } @@ -253,6 +474,47 @@ func TestHomebrewPackagesNoHomebrew(t *testing.T) { } } +func TestParsePlutilBlocks_multilineStringWithBraceLines(t *testing.T) { + // plutil prints multi-line string values raw, so a value can contain lines + // that are exactly "{", "}", "[", or "]". Those must not be interpreted + // structurally: keys after the string must still parse, the block must not + // close early, and the second block must still be counted. A value whose + // first line ends in an escaped quote (\") must stay open too. + const out = `{ + "Before" => "yes" + "Notes" => "opening +{ +} +[ +] +closing line" + "Tricky" => "ends with \" +still inside +done" + "After" => "seen" +} +{ + "Second" => "block" +} +` + var got []map[string]string + parsePlutilBlocks(out, func(f map[string]string) { + got = append(got, f) + }) + want := []map[string]string{ + { + "Before": "yes", + "Notes": "opening", // first physical line only, by design + "Tricky": `ends with \`, // first-line capture strips the quotes + "After": "seen", + }, + {"Second": "block"}, + } + if !reflect.DeepEqual(got, want) { + t.Fatalf("blocks = %#v, want %#v", got, want) + } +} + func TestParsePlutilBlocksNestedAndMultiline(t *testing.T) { // A single block whose top level carries a scalar, a nested array (with a // dict element), a nested dict, and a multi-line string. Only depth-1 diff --git a/internal/engine/packages_test.go b/internal/engine/packages_test.go index 815d79d3..616bdec2 100644 --- a/internal/engine/packages_test.go +++ b/internal/engine/packages_test.go @@ -30,6 +30,21 @@ Status: hold ok installed Architecture: amd64 Version: 1.22.1-9 +Package: man-db +Status: install ok triggers-awaited +Architecture: amd64 +Version: 2.11.2-2 + +Package: fontconfig +Status: install ok triggers-pending +Architecture: amd64 +Version: 2.14.1-4 + +Package: broken-pkg +Status: install ok half-installed +Architecture: amd64 +Version: 0.9 + Package: removed-pkg Status: deinstall ok config-files Architecture: amd64 @@ -46,9 +61,11 @@ func TestDpkgPackages_installedOnlyWithMultiarchSiblings(t *testing.T) { }) want := []any{ map[string]any{"name": "adduser", "version": "3.134", "architecture": "all"}, + map[string]any{"name": "fontconfig", "version": "2.14.1-4", "architecture": "amd64"}, // triggers-pending kept map[string]any{"name": "libc6", "version": "2.36-9+deb12u7", "architecture": "amd64"}, map[string]any{"name": "libc6", "version": "2.36-9+deb12u7", "architecture": "i386"}, - map[string]any{"name": "nginx", "version": "1.22.1-9", "architecture": "amd64"}, // held package kept + map[string]any{"name": "man-db", "version": "2.11.2-2", "architecture": "amd64"}, // triggers-awaited kept + map[string]any{"name": "nginx", "version": "1.22.1-9", "architecture": "amd64"}, // held package kept } if !reflect.DeepEqual(got, want) { t.Fatalf("dpkgPackages() = %#v\nwant %#v", got, want) From ddc0043e7af0765356b49319ee4669326641f94f Mon Sep 17 00:00:00 2001 From: Juliano Martinez Date: Thu, 2 Jul 2026 17:33:10 +0200 Subject: [PATCH 6/6] fix(packages): apply round-2 deep-review findings (11 confirmed, all verified) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Windows (validated live on the lab guest before and after): - Map Get-AppxProvisionedPackage's raw DISM UInt32 architecture (0/5/9/11/12) to x86/arm/x64/neutral/arm64 — provisioned lines rendered numbers, producing wrong arch values and defeating cross-view dedup on every populated host. - Force [Console]::OutputEncoding=UTF8 in both scripts: redirected PowerShell stdout used the OEM codepage (CP437 turned o-umlaut into invalid UTF-8 and best-fit-mapped (R) to "r"), corrupting every non-ASCII DisplayName. - Terminate both scripts with ;exit 0: an error in the final statement exits 1 even under SilentlyContinue, and run() then discards all valid output (bare registry source on 32-bit Windows; lost provisioned lines under SYSTEM). - Derive the native-hive architecture from PROCESSOR_ARCHITECTURE instead of hardcoding x64 (wrong for the whole native hive on Windows-on-ARM). - Switch the registry delimiter to the unit separator so a '|' in free-text DisplayVersion/subkey values cannot shift the record columns. nix: - Collapse custom-output store paths (bind's dnsutils/host) onto their base record; a digitless version tail with no base sibling stays verbatim, so genuine 1.0-beta versions are never touched. - Read the system set through the default profile's nix-store when nix.enable=false (Determinate) keeps nix-store out of the system env. YAML formatter: - Quote plain scalars a YAML 1.1 resolver retypes — verified against Ruby 3.4 Psych: "43_1" parsed as Integer 431, "0x1F"/"0755" as hex/octal ints, and "2026-05-14" as Date (raising under safe_load). Package versions exercise every shape (homebrew revision/date versions, apps "36"); the whole packages tree now round-trips through Psych as strings. plutil pipeline: - Scalar-rooted plists emit an empty block so positional pairing advances. - An invocation is committed only when its block count matches its path count; any mismatch (parser desync from content plutil prints ambiguously) is distrusted and bisected, containing damage to the offending file. CI hygiene: the per-category gating test disables packages alongside each other category, keeping ~9 redundant full package probes out of every run. --- internal/engine/core_gating_test.go | 8 +- internal/engine/formatter.go | 46 +++++ internal/engine/formatter_test.go | 15 ++ internal/engine/packages_extra.go | 50 ++++++ internal/engine/packages_extra_test.go | 59 ++++++- internal/engine/packages_mac.go | 27 ++- internal/engine/packages_mac_test.go | 65 +++++++ internal/engine/packages_win.go | 71 ++++++-- internal/engine/packages_win_test.go | 178 +++++++++++++++++--- openspec/changes/add-packages-fact/tasks.md | 1 + 10 files changed, 468 insertions(+), 52 deletions(-) diff --git a/internal/engine/core_gating_test.go b/internal/engine/core_gating_test.go index 800e3274..4e6440a8 100644 --- a/internal/engine/core_gating_test.go +++ b/internal/engine/core_gating_test.go @@ -41,7 +41,13 @@ func TestBuildCoreFacts_resolutionGatesSingleOutputCategories(t *testing.T) { // assert gating for the ones that do produce output by default. continue } - gated := buildCoreFacts(NewSession(), map[string]bool{fact: true}) + // Each gated build also disables packages: its probe is the most + // expensive (plutil over every .app; PowerShell spawns on Windows + // runners), and gates are independent, so disabling it alongside does + // not affect the fact under test — it just keeps ~9 full package + // probes out of every CI run. The packages iteration itself still + // asserts its own gating. + gated := buildCoreFacts(NewSession(), map[string]bool{fact: true, "packages": true}) if hasRoot(gated, fact) { t.Fatalf("buildCoreFacts(disabled=%q) still emitted %q root; resolver was not gated", fact, fact) } diff --git a/internal/engine/formatter.go b/internal/engine/formatter.go index 334d95f6..f6730a7c 100644 --- a/internal/engine/formatter.go +++ b/internal/engine/formatter.go @@ -410,6 +410,52 @@ func isPlainYAMLString(value string) bool { } return false } + return !yamlResolverRetypes(value) +} + +// yamlResolverRetypes reports whether a YAML 1.1 resolver (Ruby Psych — the +// Facter consumer stack — or PyYAML) parses the plain scalar as a non-string. +// Package versions exercise every shape: "36" (Integer), "43_1" (Integer 431 — +// underscores are YAML 1.1 digit separators), "0x1F"/"0755" (hex/octal), and +// "2026-05-14" (Date, which even raises under Psych safe_load). Verified +// against Ruby 3.4 Psych; such values must be quoted or consumers corrupt them. +func yamlResolverRetypes(value string) bool { + digits := strings.TrimPrefix(strings.ReplaceAll(value, "_", ""), "-") + if digits != "" { + if _, err := strconv.ParseInt(digits, 0, 64); err == nil { + return true + } + allDigits := true + for _, r := range digits { + if r < '0' || r > '9' { + allDigits = false + break + } + } + if allDigits { + return true // digit strings beyond int64 still parse as Integer (bignum) + } + } + return yamlDateShaped(value) +} + +// yamlDateShaped matches the YAML 1.1 date scalar (yyyy-mm-dd); datetime forms +// contain ':' and are already forced into quotes by needsQuotedYAMLString. +func yamlDateShaped(value string) bool { + parts := strings.Split(value, "-") + if len(parts) != 3 || len(parts[0]) != 4 || len(parts[1]) < 1 || len(parts[1]) > 2 || len(parts[2]) < 1 || len(parts[2]) > 2 { + return false + } + for _, part := range parts { + if part == "" { + return false + } + for _, r := range part { + if r < '0' || r > '9' { + return false + } + } + } return true } diff --git a/internal/engine/formatter_test.go b/internal/engine/formatter_test.go index cfffdded..052d6063 100644 --- a/internal/engine/formatter_test.go +++ b/internal/engine/formatter_test.go @@ -174,6 +174,21 @@ func TestIsPlainYAMLStringAcceptsOnlyRubySafeScalars(t *testing.T) { {name: "yaml true prefix", value: "TrueNAS", want: false}, {name: "lowercase off", value: "off", want: false}, {name: "contains unsupported punctuation", value: "hello.world", want: false}, + // Values Psych retypes (verified against Ruby 3.4 Psych): plain + // emission corrupts them, so they must be quoted. Package versions + // exercise every one of these shapes (apps "36", homebrew "43_1" + // revisions and "2026-05-14" ca-certificates dates). + {name: "integer-looking", value: "36", want: false}, + {name: "underscore int separators", value: "43_1", want: false}, // Psych: 431 + {name: "underscore thousands", value: "1_000_000", want: false}, + {name: "hex int", value: "0x1F", want: false}, // Psych: 31 + {name: "octal int", value: "0755", want: false}, // Psych: 493 + {name: "date shape", value: "2026-05-14", want: false}, // Psych: Date; raises under safe_load + {name: "huge digit string", value: "12345678901234567890123456", want: false}, // Psych: bignum + // Shapes Psych keeps as String stay plain. + {name: "dash range", value: "8-10", want: true}, + {name: "e-notation without dot", value: "1e3", want: true}, + {name: "version with alpha prefix", value: "v1-2", want: true}, } for _, tt := range tests { diff --git a/internal/engine/packages_extra.go b/internal/engine/packages_extra.go index e2ac0fe1..124b4745 100644 --- a/internal/engine/packages_extra.go +++ b/internal/engine/packages_extra.go @@ -99,6 +99,14 @@ func nixPackages(run commandRunner) []any { if records := nixRecordsFromLines(out, true); records != nil { return records } + // Determinate Nix (nix.enable=false on NixOS/nix-darwin) keeps nix-store out + // of the system environment while the system profile still exists; any + // nix-store can query the store, so read the system set through the default + // profile's binary before falling back to the default-profile listing. + out = run("/nix/var/nix/profiles/default/bin/nix-store", "-q", "--references", "/run/current-system/sw") + if records := nixRecordsFromLines(out, true); records != nil { + return records + } // Non-NixOS daemon/multi-user nix (ADR-0014: the profile set includes the // default profile): nix-env lists the default profile's - // elements directly — this profile carries the manifest.nix that the NixOS @@ -138,10 +146,52 @@ func nixRecordsFromLines(out string, storePaths bool) []any { seen[key] = true records = append(records, packageRecord(name, version)) } + records = collapseNixCustomOutputs(records, seen) sortPackages(records) return records } +// collapseNixCustomOutputs drops records whose version is a sibling's version +// plus a trailing digitless component — a custom output name outside the +// nixOutputs allowlist (bind's dnsutils/host, say), which no fixed list can +// cover. The collapse only fires when the base record exists, so a genuine +// digitless version tail (1.0-beta) with no base sibling is never touched. +func collapseNixCustomOutputs(records []any, seen map[string]bool) []any { + kept := records[:0] + for _, record := range records { + r := record.(map[string]any) + name, _ := r["name"].(string) + version, _ := r["version"].(string) + if base, ok := strings.CutSuffix(version, "-"+lastHyphenComponent(version)); ok && base != "" && + digitless(lastHyphenComponent(version)) && seen[name+"\x00"+base] { + continue + } + kept = append(kept, record) + } + return kept +} + +// lastHyphenComponent returns the text after the final hyphen, or "" when the +// value has no hyphen. +func lastHyphenComponent(value string) string { + if i := strings.LastIndexByte(value, '-'); i >= 0 { + return value[i+1:] + } + return "" +} + +func digitless(value string) bool { + if value == "" { + return false + } + for _, r := range value { + if r >= '0' && r <= '9' { + return false + } + } + return true +} + // nixOutputs are the standard nix output component names appended to a store-path // tail (e.g. glibc-2.42-67-bin). They are trimmed so split outputs of one // derivation collapse onto its base version. diff --git a/internal/engine/packages_extra_test.go b/internal/engine/packages_extra_test.go index 2d13a8df..d989df38 100644 --- a/internal/engine/packages_extra_test.go +++ b/internal/engine/packages_extra_test.go @@ -85,8 +85,8 @@ func TestNixPackages_fallsBackToDefaultProfileOnNonNixOS(t *testing.T) { got := nixPackages(func(name string, args ...string) string { calls = append(calls, name) switch name { - case "/run/current-system/sw/bin/nix-store": - return "" // not NixOS: no system profile environment + case "/run/current-system/sw/bin/nix-store", "/nix/var/nix/profiles/default/bin/nix-store": + return "" // not NixOS: /run/current-system does not exist case "/nix/var/nix/profiles/default/bin/nix-env": if !reflect.DeepEqual(args, []string{"-q", "--profile", "/nix/var/nix/profiles/default"}) { t.Fatalf("nix-env args = %v", args) @@ -105,8 +105,59 @@ func TestNixPackages_fallsBackToDefaultProfileOnNonNixOS(t *testing.T) { if !reflect.DeepEqual(got, want) { t.Fatalf("nixPackages(default profile) = %#v\nwant %#v", got, want) } - if len(calls) != 2 { - t.Fatalf("calls = %v, want nix-store then nix-env fallback", calls) + if len(calls) != 3 { + t.Fatalf("calls = %v, want both nix-store probes then the nix-env fallback", calls) + } +} + +func TestNixPackages_customOutputCollapsesOntoBaseRecord(t *testing.T) { + t.Parallel() + // bind's package.nix declares outputs [out lib dev man dnsutils host]; a + // systemPackages with pkgs.dnsutils (or pkgs.dig) yields the custom-output + // store path bind--dnsutils, which no fixed allowlist can cover. When the + // base record is present too, the output record must collapse onto it; a + // digitless-tail record WITHOUT a base sibling is kept verbatim (it could be + // a genuine version like 1.0-beta, which must never be corrupted). + got := nixPackages(func(name string, args ...string) string { + return "/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-bind-9.20.4\n" + + "/nix/store/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb-bind-9.20.4-dnsutils\n" + + "/nix/store/cccccccccccccccccccccccccccccccc-bind-9.20.4-host\n" + + "/nix/store/dddddddddddddddddddddddddddddddd-foo-1.0-beta\n" + }) + want := []any{ + map[string]any{"name": "bind", "version": "9.20.4"}, + map[string]any{"name": "foo", "version": "1.0-beta"}, // no base sibling: kept verbatim + } + if !reflect.DeepEqual(got, want) { + t.Fatalf("nixPackages() = %#v\nwant %#v", got, want) + } +} + +func TestNixPackages_defaultProfileNixStoreQueriesSystemSet(t *testing.T) { + t.Parallel() + // Determinate Nix on NixOS/nix-darwin sets nix.enable=false, so the system + // profile exists but sw/bin/nix-store does not. The system set must then be + // read through the default profile's nix-store (any nix-store can query the + // store) before falling back to the default-profile listing. + var calls [][]string + got := nixPackages(func(name string, args ...string) string { + calls = append(calls, append([]string{name}, args...)) + switch name { + case "/run/current-system/sw/bin/nix-store": + return "" // nix.enable=false: no nix-store in the system environment + case "/nix/var/nix/profiles/default/bin/nix-store": + return "/nix/store/mxq1r9w2w2y9lsqb5fkcyb5xbbki1n57-ncurses-6.6\n" + default: + t.Fatalf("unexpected command %q", name) + return "" + } + }) + want := []any{map[string]any{"name": "ncurses", "version": "6.6"}} + if !reflect.DeepEqual(got, want) { + t.Fatalf("nixPackages(default-profile nix-store) = %#v\nwant %#v", got, want) + } + if len(calls) != 2 || !reflect.DeepEqual(calls[1], []string{"/nix/var/nix/profiles/default/bin/nix-store", "-q", "--references", "/run/current-system/sw"}) { + t.Fatalf("calls = %v, want system nix-store then default-profile nix-store on the system env", calls) } } diff --git a/internal/engine/packages_mac.go b/internal/engine/packages_mac.go index 297936c5..ae86875b 100644 --- a/internal/engine/packages_mac.go +++ b/internal/engine/packages_mac.go @@ -50,7 +50,18 @@ func runPlutilChunks(run commandRunner, paths []string, perBlock func(chunkPaths func runPlutilChunk(run commandRunner, chunk []string, perBlock func([]string, int, map[string]string)) { out := run("plutil", append([]string{"-p"}, chunk...)...) - if out == "" { + blocks := make([]map[string]string, 0, len(chunk)) + if out != "" { + parsePlutilBlocks(out, func(fields map[string]string) { + blocks = append(blocks, fields) + }) + } + // Commit only when the block count matches the path count exactly — that is + // what makes positional pairing trustworthy. An empty output (plutil failed) + // or a mismatched count (any parser desync, e.g. a value plutil prints with + // unescaped quotes and brace-like continuation lines) distrusts the whole + // invocation and bisects it, containing the damage to the offending file. + if len(blocks) != len(chunk) { if len(chunk) > 1 { mid := len(chunk) / 2 runPlutilChunk(run, chunk[:mid], perBlock) @@ -58,11 +69,9 @@ func runPlutilChunk(run commandRunner, chunk []string, perBlock func([]string, i } return } - index := 0 - parsePlutilBlocks(out, func(fields map[string]string) { + for index, fields := range blocks { perBlock(chunk, index, fields) - index++ - }) + } } // plutilArgChunks splits plist paths into batches whose joined argv length stays @@ -236,6 +245,14 @@ func parsePlutilBlocks(out string, fn func(fields map[string]string)) { fields = nil } default: + // A scalar-rooted plist prints one bare line at depth 0 (`"hello"`); + // emit an empty block so pairing advances one block per input file. + if depth == 0 { + if trimmed != "" { + fn(map[string]string{}) + } + continue + } // A depth-1 scalar is a real top-level key; a "key" => { or // "key" => [ line opens a nested container we descend past. opensContainer := strings.HasSuffix(trimmed, " => {") || strings.HasSuffix(trimmed, " => [") diff --git a/internal/engine/packages_mac_test.go b/internal/engine/packages_mac_test.go index 839c0d91..b9becb2d 100644 --- a/internal/engine/packages_mac_test.go +++ b/internal/engine/packages_mac_test.go @@ -399,6 +399,71 @@ func TestParsePlutilBlocks_arrayRootBlock(t *testing.T) { } } +func TestParsePlutilBlocks_scalarRootBlock(t *testing.T) { + t.Parallel() + // plutil -p of a scalar-rooted plist prints one bare line (`"hello"`). It + // must fire fn with an empty fields map — like the array root — so the + // positional pairing advances one block per input file. + const out = `{ + "A" => "1" +} +"hello" +{ + "B" => "2" +} +` + var got []map[string]string + parsePlutilBlocks(out, func(f map[string]string) { + got = append(got, f) + }) + want := []map[string]string{{"A": "1"}, {}, {"B": "2"}} + if !reflect.DeepEqual(got, want) { + t.Fatalf("blocks = %#v, want %#v", got, want) + } +} + +func TestAppsPackages_blockCountMismatchIsolatedByBisection(t *testing.T) { + t.Parallel() + // If an invocation's block count does not match its path count (any parser + // desync — e.g. a value plutil prints with unescaped quotes and brace-only + // continuation lines), the whole invocation is distrusted and bisected, so + // mispairing is contained to the offending file instead of poisoning the + // records after it. + paths := []string{ + "/Applications/Good1.app/Contents/Info.plist", + "/Applications/Weird.app/Contents/Info.plist", + "/Applications/Good2.app/Contents/Info.plist", + } + goodBlock := func(name string) string { + return "{\n \"CFBundleName\" => \"" + name + "\"\n \"CFBundleShortVersionString\" => \"1.0\"\n}\n" + } + // The weird file's output splits into TWO phantom blocks. + weird := "{\n \"X\" => \"boom\"\n}\n{\n \"tail\" => \"y\"\n}\n" + run := func(_ string, args ...string) string { + out := "" + for _, f := range args[1:] { + switch f { + case paths[0]: + out += goodBlock("Good1") + case paths[1]: + out += weird + case paths[2]: + out += goodBlock("Good2") + } + } + return out + } + glob := globStub(map[string][]string{"/Applications/*.app/Contents/Info.plist": paths}) + got := appsPackages(glob, run) + want := []any{ + map[string]any{"name": "Good1", "version": "1.0", "path": "/Applications/Good1.app"}, + map[string]any{"name": "Good2", "version": "1.0", "path": "/Applications/Good2.app"}, + } + if !reflect.DeepEqual(got, want) { + t.Fatalf("appsPackages() = %#v\nwant %#v", got, want) + } +} + func TestHomebrewPackages(t *testing.T) { glob := globStub(map[string][]string{ "/opt/homebrew/Cellar/*/*": { diff --git a/internal/engine/packages_win.go b/internal/engine/packages_win.go index 9e8f0f7a..66f0d3a4 100644 --- a/internal/engine/packages_win.go +++ b/internal/engine/packages_win.go @@ -4,24 +4,37 @@ import "strings" // registryPackagesScript queries both HKLM uninstall hives in a single // PowerShell invocation: the native 64-bit view and the 32-bit WOW6432Node -// redirect. Each entry that carries a DisplayName is emitted as one -// pipe-delimited line. The architecture marker leads and the free-text -// DisplayName trails so a stray '|' inside a product name cannot shift the -// fixed columns. Columns: arch|PSChildName|DisplayVersion|SystemComponent|DisplayName. -const registryPackagesScript = `$ErrorActionPreference='SilentlyContinue';` + - `Get-ItemProperty 'HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\*'|Where-Object DisplayName|ForEach-Object{"x64|$($_.PSChildName)|$($_.DisplayVersion)|$($_.SystemComponent)|$($_.DisplayName)"};` + - `Get-ItemProperty 'HKLM:\SOFTWARE\WOW6432Node\Microsoft\Windows\CurrentVersion\Uninstall\*'|Where-Object DisplayName|ForEach-Object{"x86|$($_.PSChildName)|$($_.DisplayVersion)|$($_.SystemComponent)|$($_.DisplayName)"}` +// redirect. Each entry that carries a DisplayName is emitted as one line whose +// five columns — arch/PSChildName/DisplayVersion/SystemComponent/DisplayName — +// are joined by the Unit Separator $([char]31): PSChildName and DisplayVersion +// are free-text REG_SZ values that can legally contain '|', but never a control +// character, so the columns cannot shift. The native-hive architecture label $na +// is derived from $env:PROCESSOR_ARCHITECTURE (AMD64→x64, ARM64→arm64, X86→x86, +// anything else lowercased) so Windows-on-ARM reports arm64 instead of a +// hardcoded x64; WOW6432Node rows stay literal x86, that hive only holds 32-bit +// x86 redirects. The UTF-8 console-encoding prefix keeps non-ASCII DisplayNames +// intact: redirected PowerShell stdout otherwise uses the OEM codepage. The +// ";exit 0" terminator matters because a failing LAST statement (e.g. a missing +// WOW6432Node hive on 32-bit Windows) makes powershell exit 1 even under +// SilentlyContinue, and the engine's run() discards all stdout on non-zero exit. +const registryPackagesScript = `[Console]::OutputEncoding=[Text.Encoding]::UTF8;` + + `$ErrorActionPreference='SilentlyContinue';` + + `$na=@{'AMD64'='x64';'ARM64'='arm64';'X86'='x86'}[$env:PROCESSOR_ARCHITECTURE];if(-not $na){$na="$env:PROCESSOR_ARCHITECTURE".ToLower()};` + + `Get-ItemProperty 'HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\*'|Where-Object DisplayName|ForEach-Object{"$na$([char]31)$($_.PSChildName)$([char]31)$($_.DisplayVersion)$([char]31)$($_.SystemComponent)$([char]31)$($_.DisplayName)"};` + + `Get-ItemProperty 'HKLM:\SOFTWARE\WOW6432Node\Microsoft\Windows\CurrentVersion\Uninstall\*'|Where-Object DisplayName|ForEach-Object{"x86$([char]31)$($_.PSChildName)$([char]31)$($_.DisplayVersion)$([char]31)$($_.SystemComponent)$([char]31)$($_.DisplayName)"};` + + `exit 0` // registryPackages reads installed programs from the two HKLM uninstall hives. // Every subkey with a DisplayName becomes a record; system-component/update // entries (SystemComponent=1) are dropped, matching what Programs & Features -// shows. The architecture reflects the hive (x64 native, x86 WOW6432Node) and -// product_code is set only when the subkey name is an MSI product GUID. +// shows. The architecture reflects the hive (the machine's native label — x64 +// or arm64 — for the native view, x86 for WOW6432Node) and product_code is set +// only when the subkey name is an MSI product GUID. func registryPackages(run commandRunner) []any { out := run("powershell", "-NoProfile", "-NonInteractive", "-Command", registryPackagesScript) var records []any for line := range strings.Lines(out) { - fields := strings.SplitN(strings.TrimRight(line, "\r\n"), "|", 5) + fields := strings.SplitN(strings.TrimRight(line, "\r\n"), "\x1f", 5) if len(fields) != 5 { continue } @@ -41,10 +54,17 @@ func registryPackages(run commandRunner) []any { // followed by the collector-context packages (Get-AppxPackage), one // Name|Version|Architecture line each. The two views are unioned and deduplicated // downstream, so an unavailable provisioning module (common on Server) simply -// yields fewer lines rather than an error. -const appxPackagesScript = `$ErrorActionPreference='SilentlyContinue';` + +// yields fewer lines rather than an error. The UTF-8 console-encoding prefix +// keeps non-ASCII names intact: redirected PowerShell stdout otherwise uses the +// OEM codepage. The ";exit 0" terminator matters because a failing LAST +// statement (e.g. Get-AppxPackage under the SYSTEM context) makes powershell +// exit 1 even under SilentlyContinue, and the engine's run() discards all +// stdout — including the already-emitted provisioned lines — on non-zero exit. +const appxPackagesScript = `[Console]::OutputEncoding=[Text.Encoding]::UTF8;` + + `$ErrorActionPreference='SilentlyContinue';` + `Get-AppxProvisionedPackage -Online|ForEach-Object{"$($_.DisplayName)|$($_.Version)|$($_.Architecture)"};` + - `Get-AppxPackage|ForEach-Object{"$($_.Name)|$($_.Version)|$($_.Architecture)"}` + `Get-AppxPackage|ForEach-Object{"$($_.Name)|$($_.Version)|$($_.Architecture)"};` + + `exit 0` // appxPackages reads the appx/MSIX packages via PowerShell. Records carry the // package name, version, and (lowercased) architecture; duplicates across the @@ -58,7 +78,7 @@ func appxPackages(run commandRunner) []any { if len(fields) != 3 { continue } - name, version, arch := fields[0], fields[1], strings.ToLower(fields[2]) + name, version, arch := fields[0], fields[1], appxArchLabel(fields[2]) if name == "" || version == "" { continue } @@ -73,6 +93,29 @@ func appxPackages(run commandRunner) []any { return records } +// appxArchLabel normalizes an appx architecture to its lowercase enum name. +// Get-AppxPackage renders the enum (X64, Neutral), but Get-AppxProvisionedPackage +// exposes the raw DISM UInt32 (x86=0, arm=5, x64=9, neutral=11, arm64=12), so the +// documented numeric codes map to their labels — letting the numeric and enum +// spellings of the same package dedupe — and anything else passes through +// lowercased. +func appxArchLabel(raw string) string { + switch arch := strings.ToLower(raw); arch { + case "0": + return "x86" + case "5": + return "arm" + case "9": + return "x64" + case "11": + return "neutral" + case "12": + return "arm64" + default: + return arch + } +} + // msiProductCode returns name unchanged when it is a canonical MSI product GUID // ({8-4-4-4-12} hex), and "" otherwise, so only genuine MSI installs contribute // a product_code identity field (Inno Setup "*_is1" and other bespoke uninstall diff --git a/internal/engine/packages_win_test.go b/internal/engine/packages_win_test.go index 267dd79c..2985d395 100644 --- a/internal/engine/packages_win_test.go +++ b/internal/engine/packages_win_test.go @@ -7,21 +7,23 @@ import ( ) // registryUninstallFixture is the delimited output of registryPackagesScript. -// Columns are arch|PSChildName|DisplayVersion|SystemComponent|DisplayName, with -// the x64 native hive emitted before the x86 WOW6432Node hive. It exercises the -// real shapes seen in the wild: an MSI GUID subkey (product_code populated) and -// a bespoke Inno Setup "*_is1" subkey (no product_code) in each architecture, -// plus SystemComponent=1 runtime entries that must be dropped. Values are real -// Microsoft/third-party identifiers; the nlab guest ships bare (only a -// property-less "WIC" key per hive, filtered by Where-Object DisplayName), so -// the multi-entry cases use authentic package identities in that live format. -const registryUninstallFixture = `x64|{e46eca4f-393b-40df-9f49-076faf788d83}|14.34.31931.0||Microsoft Visual C++ 2015-2022 Redistributable (x64) - 14.34.31931 -x64|{f1b0fb2f-3d5f-4c1e-9b2a-0a1b2c3d4e5f}|14.34.31931|1|Microsoft Visual C++ 2022 X64 Minimum Runtime - 14.34.31931 -x64|Git_is1|2.43.0.2||Git -x86|{a1c31ba0-9a3b-4f2d-8c7e-1234567890ab}|14.34.31931.0||Microsoft Visual C++ 2015-2022 Redistributable (x86) - 14.34.31931 -x86|Notepad++|8.6.9||Notepad++ (32-bit x86) -x86|{deadbeef-0000-1111-2222-333344445555}|1.0.0|1|Some 32-bit Runtime -` +// Columns are arch/PSChildName/DisplayVersion/SystemComponent/DisplayName joined +// by the Unit Separator (0x1f) — DisplayVersion and the subkey name are free-text +// REG_SZ values that can legally contain '|', so the script emits a control +// character that cannot appear in them. The native hive is emitted before the +// x86 WOW6432Node hive. It exercises the real shapes seen in the wild: an MSI +// GUID subkey (product_code populated) and a bespoke Inno Setup "*_is1" subkey +// (no product_code) in each architecture, plus SystemComponent=1 runtime entries +// that must be dropped. Values are real Microsoft/third-party identifiers; the +// nlab guest ships bare (only a property-less "WIC" key per hive, filtered by +// Where-Object DisplayName), so the multi-entry cases use authentic package +// identities in that live format. +const registryUninstallFixture = "x64\x1f{e46eca4f-393b-40df-9f49-076faf788d83}\x1f14.34.31931.0\x1f\x1fMicrosoft Visual C++ 2015-2022 Redistributable (x64) - 14.34.31931\n" + + "x64\x1f{f1b0fb2f-3d5f-4c1e-9b2a-0a1b2c3d4e5f}\x1f14.34.31931\x1f1\x1fMicrosoft Visual C++ 2022 X64 Minimum Runtime - 14.34.31931\n" + + "x64\x1fGit_is1\x1f2.43.0.2\x1f\x1fGit\n" + + "x86\x1f{a1c31ba0-9a3b-4f2d-8c7e-1234567890ab}\x1f14.34.31931.0\x1f\x1fMicrosoft Visual C++ 2015-2022 Redistributable (x86) - 14.34.31931\n" + + "x86\x1fNotepad++\x1f8.6.9\x1f\x1fNotepad++ (32-bit x86)\n" + + "x86\x1f{deadbeef-0000-1111-2222-333344445555}\x1f1.0.0\x1f1\x1fSome 32-bit Runtime\n" func TestRegistryPackages_bothHivesGUIDAndSystemComponent(t *testing.T) { t.Parallel() @@ -61,7 +63,7 @@ func TestRegistryPackages_bothHivesGUIDAndSystemComponent(t *testing.T) { // upstream, and the reader's guard drops any such line that slips through. func TestRegistryPackages_skipsPropertylessKey(t *testing.T) { t.Parallel() - got := registryPackages(func(string, ...string) string { return "x64|WIC|||\n" }) + got := registryPackages(func(string, ...string) string { return "x64\x1fWIC\x1f\x1f\x1f\n" }) if got != nil { t.Fatalf("registryPackages(propertyless) = %#v, want nil", got) } @@ -97,21 +99,32 @@ func TestRegistryMsiProductCode(t *testing.T) { // appxFixture is the delimited output of appxPackagesScript: the provisioned // set (DisplayName|Version|Architecture) followed by the collector-context -// Get-AppxPackage set (Name|Version|Architecture). It exercises architecture -// lowercasing, cross-view deduplication (the VCLibs line appears twice), and the -// empty-version skip. Values are real Windows package identities; the nlab guest -// ships zero appx (DISM provisioned = 0, Get-AppxPackage = 0), so the parse is -// validated against this authentic live format rather than empty guest output. -const appxFixture = `Windows Calculator|11.2210.0.0|X64 +// Get-AppxPackage set (Name|Version|Architecture). Get-AppxProvisionedPackage +// exposes Architecture as the raw DISM UInt32 (x86=0, arm=5, x64=9, neutral=11, +// arm64=12) while Get-AppxPackage renders the enum name (X64, Neutral), so the +// provisioned half uses the numeric shape and the collector half the enum names. +// The fixture exercises the numeric→label mapping, enum lowercasing, cross-view +// deduplication (numeric and enum spellings of the same package must collapse to +// one record), and the empty-version skip. Values are real Windows package +// identities; the nlab guest ships zero appx (DISM provisioned = 0, +// Get-AppxPackage = 0), so the parse is validated against this authentic live +// format rather than empty guest output. +const appxFixture = `Microsoft.WindowsStore|22210.1401.7.0|9 +Microsoft.VCLibs.140.00|14.0.30704.0|9 +Microsoft.VCLibs.140.00|14.0.30704.0|0 +Microsoft.VCLibs.140.00.UWPDesktop|14.0.33728.0|5 +Microsoft.UI.Xaml.2.8|8.2306.22001.0|11 +Microsoft.SecHealthUI|1000.25992.9000.0|12 +Microsoft.NET.Native.Runtime.2.2||9 +Windows Calculator|11.2210.0.0|X64 Microsoft.WindowsStore|22210.1401.7.0|X64 Microsoft.VCLibs.140.00|14.0.30704.0|X64 -Microsoft.VCLibs.140.00|14.0.30704.0|X64 +Microsoft.VCLibs.140.00|14.0.30704.0|X86 +Microsoft.UI.Xaml.2.8|8.2306.22001.0|Neutral Microsoft.WindowsTerminal|1.18.3181.0|X86 -Microsoft.NET.Native.Runtime.2.2||X64 -Microsoft.UI.Xaml.2.8|8.2306.22001.0|neutral ` -func TestAppxPackages_dedupLowercaseArchSkipsEmptyVersion(t *testing.T) { +func TestAppxPackages_mapsDISMArchDedupsAcrossViewsSkipsEmptyVersion(t *testing.T) { t.Parallel() got := appxPackages(func(name string, args ...string) string { if name != "powershell" { @@ -123,8 +136,11 @@ func TestAppxPackages_dedupLowercaseArchSkipsEmptyVersion(t *testing.T) { return appxFixture }) want := []any{ + map[string]any{"name": "Microsoft.SecHealthUI", "version": "1000.25992.9000.0", "architecture": "arm64"}, map[string]any{"name": "Microsoft.UI.Xaml.2.8", "version": "8.2306.22001.0", "architecture": "neutral"}, map[string]any{"name": "Microsoft.VCLibs.140.00", "version": "14.0.30704.0", "architecture": "x64"}, + map[string]any{"name": "Microsoft.VCLibs.140.00", "version": "14.0.30704.0", "architecture": "x86"}, + map[string]any{"name": "Microsoft.VCLibs.140.00.UWPDesktop", "version": "14.0.33728.0", "architecture": "arm"}, map[string]any{"name": "Microsoft.WindowsStore", "version": "22210.1401.7.0", "architecture": "x64"}, map[string]any{"name": "Microsoft.WindowsTerminal", "version": "1.18.3181.0", "architecture": "x86"}, map[string]any{"name": "Windows Calculator", "version": "11.2210.0.0", "architecture": "x64"}, @@ -134,6 +150,18 @@ func TestAppxPackages_dedupLowercaseArchSkipsEmptyVersion(t *testing.T) { } } +// TestAppxPackages_unknownNumericArchPassesThrough pins the mapping to the five +// documented DISM values: an unrecognized code (6 = ia64) must survive as-is +// rather than being guessed at. +func TestAppxPackages_unknownNumericArchPassesThrough(t *testing.T) { + t.Parallel() + got := appxPackages(func(string, ...string) string { return "Some.Package|1.0.0.0|6\n" }) + want := []any{map[string]any{"name": "Some.Package", "version": "1.0.0.0", "architecture": "6"}} + if !reflect.DeepEqual(got, want) { + t.Fatalf("appxPackages() = %#v\nwant %#v", got, want) + } +} + func TestAppxPackages_absentYieldsNothing(t *testing.T) { t.Parallel() if got := appxPackages(func(string, ...string) string { return "" }); got != nil { @@ -146,8 +174,8 @@ func TestRegistryPackages_skipsEmptyDisplayVersion(t *testing.T) { // An uninstall entry with a DisplayName but no DisplayVersion must be dropped // (the name+version invariant), while a fully-populated entry is kept. run := func(string, ...string) string { - return "x64|{6F320B93-EE3C-4826-85E0-ADF79F8D4C61}|1.2.3|0|Real App\n" + - "x64|SomeKey_is1||0|No Version App\n" + return "x64\x1f{6F320B93-EE3C-4826-85E0-ADF79F8D4C61}\x1f1.2.3\x1f0\x1fReal App\n" + + "x64\x1fSomeKey_is1\x1f\x1f0\x1fNo Version App\n" } got := registryPackages(run) want := []any{map[string]any{ @@ -158,3 +186,97 @@ func TestRegistryPackages_skipsEmptyDisplayVersion(t *testing.T) { t.Fatalf("registryPackages() = %#v\nwant %#v", got, want) } } + +// TestRegistryPackages_pipeInFreeTextDoesNotShiftColumns pins the reason for the +// Unit Separator delimiter: DisplayVersion and DisplayName are free-text REG_SZ +// values, so a '|' inside either must survive as data instead of shifting the +// five fixed columns. +func TestRegistryPackages_pipeInFreeTextDoesNotShiftColumns(t *testing.T) { + t.Parallel() + run := func(string, ...string) string { + return "x64\x1fVLC media player\x1f3.0.20|nightly\x1f\x1fVLC media player | 64-bit\n" + } + got := registryPackages(run) + want := []any{map[string]any{ + "name": "VLC media player | 64-bit", "version": "3.0.20|nightly", "architecture": "x64", + }} + if !reflect.DeepEqual(got, want) { + t.Fatalf("registryPackages() = %#v\nwant %#v", got, want) + } +} + +// TestWinPackagesScripts_utf8OutputEncoding pins the UTF-8 console-encoding +// prefix on both scripts: redirected PowerShell stdout otherwise uses the OEM +// codepage (CP437 on en-US), which mangles every non-ASCII DisplayName (ö +// becomes the invalid byte 0x94, ® best-fits to "r"). +func TestWinPackagesScripts_utf8OutputEncoding(t *testing.T) { + t.Parallel() + const prefix = `[Console]::OutputEncoding=[Text.Encoding]::UTF8;` + for name, script := range map[string]string{ + "registry": registryPackagesScript, + "appx": appxPackagesScript, + } { + if !strings.HasPrefix(script, prefix) { + t.Errorf("%s script does not set UTF-8 output encoding first:\n%s", name, script) + } + } +} + +// TestWinPackagesScripts_exitZero pins the ";exit 0" terminator on both scripts: +// when a script's LAST statement errors, powershell exits 1 even under +// SilentlyContinue, and the engine's run() discards ALL stdout on any non-zero +// exit — e.g. Get-AppxPackage failing under the SYSTEM context would otherwise +// kill the already-emitted provisioned lines, and a missing WOW6432Node hive on +// 32-bit Windows would kill the whole registry source. +func TestWinPackagesScripts_exitZero(t *testing.T) { + t.Parallel() + for name, script := range map[string]string{ + "registry": registryPackagesScript, + "appx": appxPackagesScript, + } { + if !strings.HasSuffix(script, ";exit 0") { + t.Errorf("%s script does not terminate with ;exit 0:\n%s", name, script) + } + } +} + +// TestRegistryPackagesScript_derivesNativeArch pins that native-hive rows carry +// an architecture computed from $env:PROCESSOR_ARCHITECTURE (AMD64→x64, +// ARM64→arm64, X86→x86, anything else lowercased) instead of a hardcoded "x64", +// which would be wrong on Windows-on-ARM. WOW6432Node rows stay literal x86 — +// that hive only ever holds 32-bit x86 redirects. +func TestRegistryPackagesScript_derivesNativeArch(t *testing.T) { + t.Parallel() + const naExpr = `$na=@{'AMD64'='x64';'ARM64'='arm64';'X86'='x86'}[$env:PROCESSOR_ARCHITECTURE];` + + `if(-not $na){$na="$env:PROCESSOR_ARCHITECTURE".ToLower()};` + if !strings.Contains(registryPackagesScript, naExpr) { + t.Fatalf("registry script does not compute $na from PROCESSOR_ARCHITECTURE:\n%s", registryPackagesScript) + } + if !strings.Contains(registryPackagesScript, `ForEach-Object{"$na$([char]31)`) { + t.Fatalf("registry script does not emit $na for native-hive rows:\n%s", registryPackagesScript) + } + if !strings.Contains(registryPackagesScript, `ForEach-Object{"x86$([char]31)`) { + t.Fatalf("registry script must keep literal x86 for WOW6432Node rows:\n%s", registryPackagesScript) + } + if strings.Contains(registryPackagesScript, `"x64`) { + t.Fatalf("registry script still hardcodes x64 for the native hive:\n%s", registryPackagesScript) + } +} + +// TestWinPackagesScripts_delimiters pins the wire format each script emits: the +// registry script joins its five columns with $([char]31) (the Unit Separator, +// which free-text REG_SZ values cannot contain), while the appx script keeps '|' +// (appx names/versions are strict identifiers where '|' cannot occur). +func TestWinPackagesScripts_delimiters(t *testing.T) { + t.Parallel() + const emitTail = `$([char]31)$($_.PSChildName)$([char]31)$($_.DisplayVersion)$([char]31)$($_.SystemComponent)$([char]31)$($_.DisplayName)"` + if got := strings.Count(registryPackagesScript, emitTail); got != 2 { + t.Fatalf("registry script emits %d unit-separator column tails, want 2 (native + WOW6432Node):\n%s", got, registryPackagesScript) + } + if strings.Contains(registryPackagesScript, `)|$(`) { + t.Fatalf("registry script still joins columns with '|':\n%s", registryPackagesScript) + } + if strings.Contains(appxPackagesScript, "$([char]31)") { + t.Fatalf("appx script must keep the '|' delimiter:\n%s", appxPackagesScript) + } +} diff --git a/openspec/changes/add-packages-fact/tasks.md b/openspec/changes/add-packages-fact/tasks.md index a07049ab..eed9bd22 100644 --- a/openspec/changes/add-packages-fact/tasks.md +++ b/openspec/changes/add-packages-fact/tasks.md @@ -28,3 +28,4 @@ - [x] 4.2 `go test ./...` and `go vet ./...` (green); `gofmt` clean. - [x] 4.3 nlab/local validation, counts vs native tools: dpkg 324, rpm 386, pacman 183, apk 200, pkg freebsd 2 / dragonfly 48, openbsd_pkg 18, pkgsrc 19, ips 418, nix 70, macOS receipts 10 / apps 61 / homebrew 88 — all exact matches; Plan 9 emits nothing. Windows `registry` is **populated-validated** on the nlab Server 2025 guest: installed 7-Zip x64 + x86 (MSI), and `facts.registry` reports both — `7-Zip 24.08 (x64 edition)`/x64 from the native hive and `7-Zip 24.08`/x86 from WOW6432Node, each with the correct MSI `product_code` GUID — matching the live registry exactly (dual-hive read + architecture + product_code all confirmed). Windows `appx`: PowerShell script syntax verified on the guest and parse logic unit-tested; correctly omitted on the appx-less Server (populating appx needs a signed MSIX — disproportionate for a secondary source). `snap` and `flatpak` are **populated-validated** on the nlab ubuntu2404 guest: installed hello-world (snap) and org.vim.Vim from flathub (flatpak); `facts.snap` = 3 = `snap list`, `facts.flatpak` = 4 = the versioned `flatpak list` rows, with the same-app-same-version GL.default siblings distinguished by `branch` and the versionless codecs-extra extension dropped by the name+version invariant — all alongside dpkg (740) on the same host. - [x] 4.4 `openspec validate add-packages-fact --strict`. +- [x] 4.5 Deep multi-lens adversarial review (two workflow rounds: 10 lenses total, every finding independently verified). Round 1: 13 findings fixed (dpkg trigger states, pkgsrc/nix ADR-compliance, apps Utilities globs, plutil bisection + multiline tracking + array roots, homebrew prefix, packages fact group, 6 schema drifts). Round 2: 11 confirmed findings fixed — Windows appx DISM numeric architecture mapping, UTF-8 output encoding for PowerShell (OEM-codepage mojibake, verified live), `;exit 0` partial-output survival (verified live), ARM64-aware native-hive architecture, unit-separator registry delimiter, nix custom-output collapse (bind dnsutils/host) and Determinate-Nix system-set fallback, YAML plain-scalar quoting for Psych-retyped versions (`43_1`→431, `2026-05-14`→Date crash under safe_load — verified against Ruby Psych, round-trip now clean), plutil scalar-root blocks + block-count-validated bisection, gating-test probe cost. Windows fixes re-validated on the populated guest (registry 2 records, derived x64, GUIDs intact).