Skip to content

fix: four defects an adversarial audit found in the merged v1.44 work - #392

Merged
JasonYeYuhe merged 2 commits into
mainfrom
fix/v144-audit-findings
Jul 29, 2026
Merged

fix: four defects an adversarial audit found in the merged v1.44 work#392
JasonYeYuhe merged 2 commits into
mainfrom
fix/v144-audit-findings

Conversation

@JasonYeYuhe

Copy link
Copy Markdown
Collaborator

A 5-lens adversarial audit of f9f15bc..HEAD (#387#390) raised 8 findings; 4 survived two rounds of refutation. All four are mine.

1 — Twenty-six healthy collectors were labelled "No data returned" (high)

CollectorDataKind has three cases and I applied a counter predicate to all of them.

  • .credits — "a credit-based balance"
  • .statusOnly — "local status only, no quota model"

Those collectors put their payload in status_text — Groq returns "12.5 req/min · 4300 tok/min", OpenRouter "$12.34 / $50.00" — and hardcode today/week/quota/remaining to zero-or-nil on their success path.

So classify called them .ranButEmpty, and the card rendered a permanent orange "No data returned — the check ran but came back empty" directly underneath the live numbers it had just fetched. 16 .statusOnly + 10 .credits.

Two knock-ons from the same call: producedValue needs a .producedData, so a user whose providers are all credits/status-only would see real numbers and never get the W5 login item; and every report logged them "empty", poisoning the silent-zero diagnostic this release exists to power.

I excluded status_text from the predicate on purpose — for quota providers it is always populated and made the check a tautology. Right for .quota, wrong to generalise. The fix trusts the collector's declared kind rather than reintroducing a string test.

2 — A cancelled refresh published its results (medium)

requestRefresh cancels the in-flight task → propagates into every collector → URLSession throws URLError(.cancelled) (-999) → categorize correctly reads .network. Neither publish site checked cancellation, so a refresh the user themselves superseded (⌘R, which is not gated on isLoading) painted every network-backed provider red. The replacement pass returns early on isLoading, so nothing repaired it for 60–120s.

The telemetry Task {} is unstructured and does not inherit cancellation — it uploaded failed_network for the whole fleet row and cached it as last-reported.

3 — brew uninstall --zap left everything behind (medium)

The cask listed ~/Library/Application Support/CLI Pulse. The app writes CLIPulse, no spaceDailyUsageArchive.defaultRoot() appends that literal and the pet ledger/state share the directory.

Verified on this machine: the real path holds 60K of usage history and pet state; the path the cask named does not exist at all. --zap reported success and removed nothing, and a reinstall came back with every day of history.

Also missing ~/Library/Preferences/group.yyh.CLI-Pulse.plist — the app-group defaults suite backs to a plist next to the container, not inside it, and that is where provider configs and helper status live. All four new paths verified present on disk.

This one is already live in the published tap, so the cask needs re-pushing after merge.

4 — The Settings launch-at-login switch could invert (high)

.onChange { _ in LaunchAtLogin.toggle() } threw the binding value away and inferred direction from live status, while the switch renders from a @State snapshot. Once anything changes registration behind that snapshot: switch reads OFF, reality is ON, user taps to turn it "on", it unregisters instead — and the same tap writes userTouchedToggleKey, locking W5 out from ever repairing it.

The inference is pre-existing (both refuters correctly said so). What is new is that W5 registers in the background mid-session, so a user opening Settings during the first collector pass hits it without doing anything wrong. setEnabled(_:) now takes the desired state.

Verification

Gate Result
CLIPulseCore, default 2273 passed / 0 failed
CLIPulseCore, -DDEVID_BUILD 2308 passed / 0 failed
macOS / iOS / watchOS BUILD SUCCEEDED
brew style --cask clean

Mutation-verified: removing the dataKind guard reddens exactly the two new classification tests.

🤖 Generated with Claude Code

JasonYeYuhe and others added 2 commits July 29, 2026 01:27
All three are mine, from #388/#389.

1. TWENTY-SIX HEALTHY COLLECTORS WERE LABELLED "No data returned"

`CollectorDataKind` has three cases and I applied a counter predicate to all
of them. `.credits` is "a credit-based balance"; `.statusOnly` is "local status
only, no quota model". Those collectors put their payload in `status_text` —
Groq returns "12.5 req/min · 4300 tok/min", OpenRouter "$12.34 / $50.00" — and
hardcode today/week/quota/remaining to zero-or-nil on their SUCCESS path.

So `classify` called them `.ranButEmpty`, and the provider card rendered a
permanent orange "No data returned — the check ran but came back empty"
directly underneath the live numbers it had just fetched. 16 `.statusOnly` +
10 `.credits` collectors.

Two knock-ons from the same call: `producedValue` requires a `.producedData`,
so a user whose enabled providers are all credits/status-only would be shown
real numbers and never get the W5 login item; and every report to
`devices.collector_status` logged them "empty", poisoning the silent-zero
diagnostic this release exists to power.

I excluded `status_text` from the predicate on purpose — for quota providers
it is always populated and made the check a tautology. That was right for
`.quota` and wrong to generalise. The fix trusts the collector's own declared
kind rather than reintroducing a string test.

2. A CANCELLED REFRESH PUBLISHED ITS RESULTS

`requestRefresh` cancels the in-flight task; that propagates into every
collector and `URLSession` throws `URLError(.cancelled)` (-999), which
`categorize` correctly reads as `.network`. Neither publish site checked
cancellation, so a refresh the user themselves superseded — ⌘R, which is not
gated on `isLoading`, or the helper's sync notification — painted every
network-backed provider red. The replacement pass then returns early on
`isLoading`, so nothing repaired it until the next 60-120s tick.

The telemetry Task spawned from the same callback is unstructured and does NOT
inherit cancellation, so it uploaded `failed_network` for the whole fleet row
and cached it as last-reported.

3. `brew uninstall --zap` LEFT EVERYTHING BEHIND

The cask listed "~/Library/Application Support/CLI Pulse". The app writes
"CLIPulse", no space — `DailyUsageArchive.defaultRoot()` appends that literal,
and the pet ledger/state share the directory. Verified on this machine: the
real path holds 60K of usage history and pet state; the path the cask named
does not exist at all. So `--zap` reported success and removed nothing, and a
reinstall came back with every day of history intact.

Also missing: ~/Library/Preferences/group.yyh.CLI-Pulse.plist. The app-group
defaults suite backs to a plist NEXT TO the group container, not inside it,
and unsandboxed (which is what brew installs) that is where provider configs,
helper status and cached collector results live. All four paths in the new
list verified present on disk.

Mutation-verified: removing the `dataKind` guard reddens exactly the two new
classification tests.

CLIPulseCore 2273 passed / 0 failed (default), 2308 / 0 (-DDEVID_BUILD);
macOS + iOS + watchOS BUILD SUCCEEDED; `brew style --cask` clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…inding 4)

`.onChange(of: launchAtLogin) { _ in LaunchAtLogin.toggle() }` threw the new
binding value away, and `toggle()` inferred direction from `isEnabled` — the
LIVE status — while the switch renders from a `@State` snapshot taken when
SettingsTab was built (SettingsTab.swift:13).

Once anything changes registration behind that snapshot the control inverts:
switch reads OFF, reality is ON, user taps to turn it "on", inferred direction
unregisters instead. And since the tap also writes `userTouchedToggleKey`, W5
is locked out and never repairs it.

The inference is pre-existing — SettingsTab and AdvancedSection are unchanged
since f9f15bc, and both refuters correctly said so. What is new is that W5
registers in the background mid-session, so a user who opens Settings during
the first collector pass (5-30s: ~48 providers, 8 concurrent, network
timeouts) can hit it without doing anything wrong. Shipping a feature that
makes a latent inversion reachable, and then permanent, is not a defensible
place to leave it.

`setEnabled(_:)` takes the desired state, so tapping a stale switch converges
on what its label says instead of doing the opposite. `toggle()` remains as
`setEnabled(!isEnabled)` for callers with no desired-state to pass.

macOS + iOS BUILD SUCCEEDED.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@JasonYeYuhe
JasonYeYuhe merged commit 2664204 into main Jul 29, 2026
26 of 28 checks passed
@JasonYeYuhe
JasonYeYuhe deleted the fix/v144-audit-findings branch July 29, 2026 09:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant