diff --git a/docs/DASHBOARD.md b/docs/DASHBOARD.md index 6f52df3..73bcd13 100644 --- a/docs/DASHBOARD.md +++ b/docs/DASHBOARD.md @@ -23,7 +23,7 @@ Its position does not move when the primary area changes; only the choices insid | Overview | Hosts & Routing | `#overview/hosts` | Hosts & routing | Enabled execution hosts, activity assignments, primary-host policy, and escalation paths | | Overview | Providers | `#overview/providers` | Inference providers | Provider bindings, availability, provenance, and configuration health | | Overview | Runtime | `#overview/runtime` | Runtime health | Local services, MCP connections, processes, and operational readiness | -| Overview | Intelligence | `#overview/intelligence` | Intelligence & learning | Memory, learned patterns, quality feedback, and improvement signals | +| Overview | Intelligence | `#overview/intelligence` | Intelligence & learning | Memory, learned patterns, reasoning-graph growth, and improvement signals, updated near-live while the view is open | | Usage | Scorecard | `#usage/score` | Usage scorecard | Token consumption, API-equivalent cost, efficiency, and trends | | Usage | Limits | `#usage/limits` | Provider limits | Current provider windows, reset timing, and available capacity | | Usage | Findings | `#usage/findings` | Usage findings | Actionable anomalies, efficiency opportunities, and evidence-backed recommendations | @@ -61,7 +61,13 @@ Overview keeps status and routing in one health-first area: of which inference provider served a particular session. - **Providers** presents inference-provider bindings and their configuration provenance. - **Runtime** presents operational services, processes, and MCP readiness. -- **Intelligence** presents memory, learning, and quality-improvement signals. +- **Intelligence** presents memory, learning, and quality-improvement signals: the neural pattern + store's current size, its separate lifetime patterns-learned counter, reasoning-graph growth, and + the route-learner's improvement delta. It reads files ruflo/agentic-qe already write under + `.claude-flow/` and updates near-live over its own SSE stream while the view is open, falling back + to the general status poll otherwise. See [Project intelligence](ddd/project-intelligence.md) and + [ADR-0024](adr/0024-project-intelligence-telemetry.md) for the full model and the two learning + metrics' load-bearing distinction. ## Usage diff --git a/docs/adr/0024-project-intelligence-telemetry.md b/docs/adr/0024-project-intelligence-telemetry.md new file mode 100644 index 0000000..8becf7e --- /dev/null +++ b/docs/adr/0024-project-intelligence-telemetry.md @@ -0,0 +1,164 @@ +# ADR-0024 — Project intelligence: live learning telemetry from ruflo/agentic-qe's own state + +- **Status:** Implemented +- **Date:** 2026-08-05 +- **Deciders:** agentic-kit maintainers +- **Related:** [ADR-0005](0005-dashboard-in-page-routing-reveal.md), + [ADR-0009](0009-usage-scorecard-local-transcript-analytics.md), + [ADR-0012](0012-observability.md) + +## Context + +Overview's **Intelligence** destination (`#overview/intelligence`, added by +[ADR-0005](0005-dashboard-in-page-routing-reveal.md)'s 2026-08-04 information-architecture +amendment) has always advertised "memory, learned patterns, quality feedback, and improvement +signals." Until now its "learning over time" strip rendered only two sparklines: + +- **patterns learned**, sourced from `.claude-flow/health-history.json` — a ring `dashboard-server.mjs` + itself appended to, and only while a dashboard happened to be running to observe + `.claude-flow/neural/stats.json`. On a machine (or CI checkout) where the dashboard had never + polled long enough to accumulate a ring, this file simply did not exist and the panel showed + `no data`; +- **improvement Δpp**, sourced from the route-learner's existing `.claude-flow/improvement.json` + (unchanged by this decision). + +Meanwhile ruflo and agentic-qe were already writing two richer, always-present sources that the +dashboard never read: `.claude-flow/neural/patterns.json` (the neural pattern store — a live JSON +array of pattern entries, each carrying `createdAt`/`type`) and +`.claude-flow/data/intelligence-snapshot.json` (point-in-time samples of the reasoning/knowledge +graph's size: `nodes`, `edges`, `pageRankSum`). `src/commands/status.mjs`'s CLI `learning` row +already reads `.claude-flow/neural/stats.json` for the same lifetime `patternsLearned` counter the +health ring's samples happened to carry — but the *store's own current inventory* (how many +patterns are on disk right now, which shrinks under pruning/compaction even as the lifetime +counter only ever climbs) had no reader at all. The panel's "no data" state was therefore +avoidable, not fundamental: real trend data already existed on disk. + +Separately, every value on this panel only ever refreshed on the dashboard's general ~30s +`/api/status` poll (the same cadence used for unrelated subsystem-health cards), so a user +actively watching learning happen — a pattern being stored, the graph growing — waited up to that +long to see it. + +This telemetry is a different kind of fact than anything [ADR-0012](0012-observability.md)/ +[Observability](../ddd/observability.md) models. It carries no session, actor, host, provider, or +model identity; no lifecycle (`queued → running → completed`); no per-field evidence confidence +(`observed`/`correlated`/`inferred`/`assumed`/`planned`); and no transcript content requiring +redaction. It is four scalar/array-shaped reads over this project's own `.claude-flow/` state — +the same local trust boundary `ak status` already reads directly, with no host-specific +anti-corruption adapter needed because there is only ever one shape, ak's own. The panel that +surfaces it also lives under **Overview**, not under **Observability**'s Live/History scope. The +implementation deliberately did not construct an `ObservedSession`, did not flow through the +canonical event normalizer, and did not extend the replay/snapshot cursor — it reuses only +source-agnostic transport plumbing (`JsonlTailer`, `sseChannel`, `reserveClientSlot`/`clientGone`, +`transcriptSseFrame`) that Dashboard delivery already shares across contexts. + +## Decision + +### 1. A new bounded context: Project intelligence + +Project intelligence is its own bounded context (see the updated +[context map](../ddd/context-map.md) and [Project intelligence](../ddd/project-intelligence.md)), +not an Observability extension and not folded into ADR-0005's navigation-shell amendments. Its +model, invariants, and the reasoning for keeping it separate from `ObservedSession` are specified +in that document; this ADR records the decision and its consequences. + +### 2. One read-only history module composes four existing sources + +`src/lib/dashboard/intel-history.mjs` adds: + +- `readNeuralPatternStoreHistory(cwd)` — every entry currently on disk in + `.claude-flow/neural/patterns.json`, as `{ createdAt, type }`; +- `readGraphHistory(cwd)` — every sample in `.claude-flow/data/intelligence-snapshot.json`, as + `{ timestamp, nodes, edges, pageRankSum }`; +- `readGlobalStats(cwd)` — the current cumulative counters in `.claude-flow/neural/stats.json` + (`patternsLearned`, `trajectoriesRecorded`, `signalsProcessed`, `lastAdaptation`), reading via the + same `readJson` helper and `?? 0` defaulting `status.mjs`'s `learning` row already uses, so the two + call sites cannot drift apart; +- `readHealthRing(cwd)` and `appendHealthSnapshot(cwd, snapshot)` — moved (not duplicated) from + `dashboard-server.mjs`, unchanged behavior, now capped at 500 samples with field-level dedup + (a repeated poll of unchanged stats writes nothing); +- `readIntelHistory(cwd)` — the combinator `collectData()` and the SSE route both call, returning + `{ patternStore, graph, healthRing, globalStats }`. + +The **patterns-learned counter** (`globalStats.patternsLearned`, a lifetime total) and the +**pattern-store size** (`patternStore.length`, entries actually present right now) are +independent metrics that may legitimately diverge as the store is pruned or compacted. This +project's own repository demonstrates the divergence today: 28 live pattern-store entries against +a 1,337 lifetime counter. Every reader, doc comment, and rendered label keeps the two separate; +none averages, sums, or substitutes one for the other. + +### 3. Push updates over a new SSE route, additive to the existing poll + +`src/lib/live/intelligence-watch.mjs` adds `IntelligenceWatch`, which polls the three source files' +`mtime` every second (default), corroborated by a change-only tail of +`.claude-flow/data/pending-insights.jsonl` (via the existing `JsonlTailer`, whose line *contents* +are never read — a record arriving at all is the signal), and flushes a debounced +(2.5s trailing-edge, measured from the most recent detected change) `onUpdate(readIntelHistory(cwd))` +call. `dashboard-server.mjs` exposes this as `GET /api/live/intelligence`: one `event: init` frame +with the current `readIntelHistory(cwd)` on connect, then an `event: update` frame per flush, +fanned out to every connected client. It reuses Dashboard delivery's proven SSE discipline — +reservation-before-await client-cap tracking, the forwarding-cleanup idiom, `sseChannel` backpressure +and heartbeat — but sends `transcriptSseFrame` payloads, not `sseFrame`'s session-privacy-redacted +ones, because this payload was never session or transcript content in need of that redaction +pipeline. + +`GET /api/live/intelligence` shares Dashboard delivery's transport primitives with, but is not part +of, [Observability](../ddd/observability.md)'s `/api/live`, `/api/live/events`, +`/api/live/transcripts/:host/:id/events`, and `/api/live/playback/:host/:id` family documented in +[OBSERVABILITY.md](../OBSERVABILITY.md); it is not covered by that document's evidence, privacy, or +capability-coverage contract, and unlike Observability's ruflo/agentic-qe sources, it requires no +`--live-source` registration — the four files it reads are always this project's own. + +### 4. `/api/status` keeps working as the fallback path + +The SSE route is additive. `collectData()`'s existing return gains `globalStats`, `patternStore`, +and `graph` alongside the unchanged `health` field (still `intel.healthRing`); a client without +`EventSource` support, or one that has not yet opened the stream, still gets the full picture on the +next poll. The `/api/status` error-fallback payload was extended with the same three keys +(`globalStats: null, patternStore: [], graph: null`) so its shape never diverges from the success +path. + +## Consequences + +### Positive + +- The Intelligence panel shows real trend data — pattern-store growth and reasoning-graph growth — + sourced from files that already existed, at zero new collection cost and no new write path beyond + the existing, now-relocated `appendHealthSnapshot`. +- Users watching active learning see updates within the debounce window (≤ ~3.5s) instead of waiting + out the general status poll. +- The lifetime-counter-vs-store-size divergence is now visible and labeled instead of silently + absent or conflated. +- A genuinely different domain got its own bounded context instead of stretching + `ObservedSession`'s Session/Actor/Activity model, or ADR-0005's navigation-shell decision, to cover + facts neither was designed to grade. + +### Negative + +- A second `/api/live/*`-prefixed SSE endpoint and client-cap surface to operate, alongside + `/api/live/events` and `/api/live/transcripts/...`. +- `mtime`-based polling can in principle miss two rewrites that land on the same filesystem-reported + millisecond; the `pending-insights.jsonl` tail is a second, independent trigger but not a formal + guarantee. +- No per-field confidence grading exists for this data, unlike Observability's evidence model — + accepted because every source is this project's own local file, not another host's evidence + requiring provenance. + +### Risks and mitigations + +| Risk | Mitigation | +|------|------------| +| Pattern-store size and the lifetime counter get conflated in a future edit or display | Documented domain invariant, shared header comment across both modules, and separate rendered figures with distinct captions | +| Debounce coalesces a burst into a stale window | Trailing-edge debounce measured from the most recently detected change, not a fixed interval | +| A reader is called on this project's own untrusted-shaped JSON (a partially written file, a schema drift) | Malformed or non-array data degrades to `[]`/`null`, matching `readJsonSafe`'s existing null-on-absent convention; individual malformed entries are skipped rather than throwing | +| New SSE route reintroduces a client-cap race | Reuses the same `reserveClientSlot`/forwarding-cleanup idiom already hardened for `/api/live/events` | +| Readers drift from `status.mjs`'s CLI `learning` row over time | `readGlobalStats` calls the same `readJson` helper and `?? 0` default `status.mjs` uses, not a hand-copied reimplementation | + +## References + +- `src/lib/dashboard/intel-history.mjs`, `tests/kit/intel-history.test.mjs` +- `src/lib/live/intelligence-watch.mjs`, `tests/kit/intelligence-watch.test.mjs` +- `src/lib/dashboard-server.mjs` (`collectData`, `GET /api/live/intelligence`, `lazyIntelWatch`) +- `src/lib/dashboard/client.mjs`, `src/lib/dashboard/page.mjs` (Intelligence panel rendering, SSE + subscription) +- [Project intelligence domain](../ddd/project-intelligence.md) +- [Dashboard guide](../DASHBOARD.md) diff --git a/docs/adr/README.md b/docs/adr/README.md index 79886a8..e77cd4a 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -32,6 +32,7 @@ Consequences**, and cites the grounded source it rests on where relevant. | [0021](0021-inference-provider-provenance.md) | Inference-provider provenance for live sessions | Accepted | | [0022](0022-metaharness-as-optional-assurance-companion.md) | MetaHarness as an optional assurance companion | Proposed | | [0023](0023-fail-closed-operations-and-explicit-degradation.md) | Fail-closed mutations and explicit degraded operation evidence | Implemented | +| [0024](0024-project-intelligence-telemetry.md) | Project intelligence: live learning telemetry from ruflo/agentic-qe's own state | Implemented | Theme: ADRs **0001–0006** define **dual-host LLM routing and leadership** — how `ak` lets ruflo route each development activity (architecture, implementation, testing, review, …) to the right host (Claude @@ -121,3 +122,12 @@ managed fallbacks report degradation, SQLite retains classified failure evidence usage, promised backups fail closed before atomic replacement, status-line failures gain redacted opt-in diagnostics, process discovery is current-user and argv-minimized, setup discloses and verifies its project auto-approve manifest, and clean-machine tests isolate every mutable path. + +**0024** gives Overview's Intelligence view real trend data instead of a permanently-empty strip: +a new `intel-history.mjs` module reads the neural pattern store, its lifetime learned-pattern +counter, and reasoning-graph size samples that ruflo/agentic-qe already write under +`.claude-flow/`, while a debounced `IntelligenceWatch` pushes near-instant updates over a new +`GET /api/live/intelligence` SSE route additive to the existing status poll. It establishes Project +intelligence as its own bounded context rather than an Observability extension — this telemetry +carries no session, actor, host, provider, or lifecycle identity and needs no per-field evidence +confidence, and the panel it feeds lives under Overview, not Observability's Live/History scope. diff --git a/docs/ddd/README.md b/docs/ddd/README.md index f683d7c..6e86f39 100644 --- a/docs/ddd/README.md +++ b/docs/ddd/README.md @@ -12,6 +12,7 @@ describe the current system unless a section is explicitly marked as future work | [Integration management](integration-management.md) | Hosts, inference providers, bindings, capabilities, lifecycle, facts, and ownership | | [Routing and orchestration](routing-and-orchestration.md) | Activities, routes, leadership, escalation, projections, and canonical `ak run` execution | | [Observability](observability.md) | Evidence acquisition, observed-session aggregates, replay, and dashboard delivery | +| [Project intelligence](project-intelligence.md) | Pattern store, learning counters, reasoning-graph size, and live delivery for Overview's Intelligence view | ## Relationship to other documentation diff --git a/docs/ddd/context-map.md b/docs/ddd/context-map.md index 0039397..3e0beee 100644 --- a/docs/ddd/context-map.md +++ b/docs/ddd/context-map.md @@ -20,6 +20,8 @@ Native Evidence ----> Evidence Acquisition ----> Canonical Evidence | +----> Workspace Snapshot Cache | | | | +-----------------------+----> Dashboard Delivery <----+ + ^ +Project State (.claude-flow/*) ----> Project Intelligence ┘ Maintainer Administration is a separate, deliberately-egressing context. ``` @@ -74,11 +76,22 @@ an advisory read-model cache, not the append-only Evidence Archive and not a sou Restoration supplies inert History context using the original capture time; it cannot query a current checkout and present that state as historical. +### Project intelligence + +Owns read-only trend projections over ruflo/agentic-qe's own project-level learning state: the +neural pattern store, its lifetime learned-pattern counter, reasoning-graph size samples, and the +machine-health sample ring. It reads `.claude-flow/*` files directly — there is only ever one +shape, ak's own, so no anti-corruption adapter is required — and is independent of Evidence +Acquisition and Observability's canonical event model. It carries no session, actor, host, +provider, or lifecycle identity and grades no per-field evidence confidence. + +See [Project intelligence](project-intelligence.md). + ### Dashboard delivery Owns protected HTTP/SSE delivery, browser DTOs, filters, presentation, and interaction state. It -may combine read models from Observability, Historical Usage, routing, and integration facts. It -cannot manufacture or strengthen domain facts. +may combine read models from Observability, Historical Usage, Project Intelligence, routing, and +integration facts. It cannot manufacture or strengthen domain facts. ### Maintainer administration @@ -99,6 +112,8 @@ and credential policy is distinct from the offline-first dashboard and integrati | Observability | Workspace snapshot cache | Last safe metadata-only session workspace capture | | Workspace snapshot cache | Dashboard delivery | Inert last-recorded History context after restart | | Historical usage | Dashboard delivery | Historical aggregates and findings | +| Project state (`.claude-flow/*`) | Project intelligence | Direct local file reads; no anti-corruption adapter needed | +| Project intelligence | Dashboard delivery | Read-model projection, delivered by poll (`/api/status`) and SSE push (`/api/live/intelligence`) | ## Boundary rules @@ -108,3 +123,6 @@ and credential policy is distinct from the offline-first dashboard and integrati - Dashboard presentation cannot upgrade provenance. - Historical usage and live topology share identifiers, not aggregate ownership. - Network egress occurs only in commands and contexts whose contract explicitly permits it. +- Project intelligence reads local project state directly; it never enters Evidence Acquisition's + anti-corruption layer or Observability's canonical event model, and it establishes no session, + actor, host, provider, or lifecycle identity. diff --git a/docs/ddd/project-intelligence.md b/docs/ddd/project-intelligence.md new file mode 100644 index 0000000..a47df66 --- /dev/null +++ b/docs/ddd/project-intelligence.md @@ -0,0 +1,145 @@ +# Project Intelligence Domain + +This document describes the domain implemented by [ADR-0024](../adr/0024-project-intelligence-telemetry.md), +`src/lib/dashboard/intel-history.mjs`, and `src/lib/live/intelligence-watch.mjs`. + +## Purpose + +Project intelligence surfaces trend data about ruflo/agentic-qe's own project-level learning +subsystem — the neural pattern store, its lifetime learned-pattern counter, the reasoning graph's +structural growth, and a machine-health sample ring — inside the dashboard's Overview → +**Intelligence** view (`#overview/intelligence`). It is a read-only projection over files those +tools already write under `.claude-flow/`. It owns no session, actor, or activity identity; it +grades no per-field evidence confidence; and it cannot steer, retrain, or mutate the learning +subsystem it reads. + +The shared terms in [Ubiquitous language](ubiquitous-language.md) are normative. + +## Why this is a separate context, not Observability + +[Observability](observability.md) exists to grade evidence about concurrent, cross-host **session** +execution: an `ObservedSession` aggregate keyed by `(host, sessionId)`, a canonical event envelope +with per-field confidence (`observed`/`correlated`/`inferred`/`assumed`/`planned`), a lifecycle +state machine, and a protected transcript-content plane. None of that applies here: + +- Every value in this domain is a scalar count, a timestamp, or a flat historical array read from + this project's own `.claude-flow/` state — the same local trust boundary `ak status` already + reads directly. There is no other host's evidence to normalize through an anti-corruption + adapter, because there is only ever one shape: ak's own. +- There is no session, actor, host, provider, or model identity anywhere in this domain's data, and + therefore no capability-coverage matrix, no actor lens, and no court membership. +- There is no lifecycle (`queued → running → completed`); sources are either a live inventory + (the pattern store), a monotonic lifetime counter, an append-only sample history (the graph), or + a capped, deduplicated ring (machine health). +- The panel is a permanent secondary view under **Overview**, never a mode of **Observability**'s + mutually exclusive Live/History scope (see [ADR-0005](../adr/0005-dashboard-in-page-routing-reveal.md)). + +The implementation reuses only source-agnostic transport plumbing that Dashboard delivery already +shares across contexts — `JsonlTailer`, `sseChannel`, `reserveClientSlot`/`clientGone`, and +`transcriptSseFrame` — never Observability's canonical normalizer, `ObservedSession` aggregate, or +replay/snapshot cursor. `GET /api/live/intelligence` shares the `/api/live/*` path prefix with +Observability's endpoints by transport convention only; it is not covered by +[OBSERVABILITY.md](../OBSERVABILITY.md)'s evidence, privacy, or capability-coverage contract, and it +needs no `--live-source` registration because its four sources are always this project's own. + +## Model + +```text +.claude-flow/neural/patterns.json -> PatternStoreEntry[] { createdAt, type } +.claude-flow/neural/stats.json -> GlobalLearningStats { patternsLearned, trajectoriesRecorded, + signalsProcessed, lastAdaptation } +.claude-flow/data/intelligence-snapshot.json -> GraphSample[] { timestamp, nodes, edges, pageRankSum } +.claude-flow/health-history.json -> HealthSample[] (capped ring, deduped, appended here) +.claude-flow/data/pending-insights.jsonl -> change signal only (line contents never read) +.claude-flow/improvement.json -> ImprovementEval (pre-existing; unchanged by this domain) + +readIntelHistory(cwd) -> { patternStore, graph, healthRing, globalStats } + | + +--> collectData() (Dashboard delivery) --> GET /api/status (poll, ~30s) + | + +--> IntelligenceWatch --> broadcastIntel --> GET /api/live/intelligence (SSE push, debounced) + | + v + Overview -> Intelligence (#overview/intelligence): five sparklines + improvement verdict badge +``` + +### Pattern-store size vs. patterns-learned counter + +These are the domain's two central, easily-confused facts and must never be conflated: + +- **Pattern-store size** (`patternStore.length`, bucketed by day from `createdAt`) counts entries + currently present in `.claude-flow/neural/patterns.json` — the store's live inventory right now. +- **Patterns-learned counter** (`globalStats.patternsLearned`) is a cumulative total persisted in + `.claude-flow/neural/stats.json` — patterns learned over the store's whole lifetime, including + ones since pruned, compacted, or replaced. + +The store can be pruned while the counter keeps climbing; that divergence is expected, not a bug. +This repository's own `.claude-flow/` state demonstrates it directly: 28 pattern-store entries +against a 1,337 lifetime counter. No reader, computation, or rendered label treats one as a +substitute display for the other. + +### Reasoning graph size + +`graph` is a point-in-time series of the reasoning/knowledge graph's structural size +(`nodes`, `edges`, `pageRankSum`) sampled into `.claude-flow/data/intelligence-snapshot.json` by +existing ruflo/agentic-qe tooling. It is a structural-growth series, independent of both pattern +metrics above and of the machine-health ring below. + +### Machine-health ring + +`healthRing` is the existing capped (500-entry), field-level-deduplicated sample ring in +`.claude-flow/health-history.json`. Its reader and writer (`readHealthRing` / +`appendHealthSnapshot`) moved into this domain from `dashboard-server.mjs` with unchanged behavior; +this document is now their domain home. A repeated snapshot whose fields are identical to the last +stored sample except `ts` is a no-op — polling unchanged stats does not grow the ring. + +### Improvement delta + +`improvement.json` (the route-learner's held-out-accuracy evaluation: curve, cold/warm accuracy, +`deltaPP`, significance) predates this domain and is unchanged by it. It continues to be read +verbatim by `collectData()` and rendered as the existing Δpp sparkline and verdict badge. + +## Live delivery + +`IntelligenceWatch` polls the three source files' `mtime` on an interval (default 1,000 ms), +corroborated by a change-only tail of `pending-insights.jsonl` via the existing `JsonlTailer` — +line *contents* are never read; a record arriving at all is the signal. Detected changes accumulate +against a trailing-edge debounce (default 2,500 ms, measured from the most recently detected +change) so a burst of writes during an active session collapses into one flush. A flush re-reads +`readGlobalStats(cwd)`; if it differs from the watcher's own last-seen value, it appends a health +snapshot (independent of, and in addition to, `intel-history.mjs`'s own on-disk dedup) and then +calls `readIntelHistory(cwd)` and forwards the combined result to every connected +`GET /api/live/intelligence` client. + +The route follows Dashboard delivery's existing SSE discipline exactly: reservation-before-await +client-cap tracking (default cap 32, clamped to 256), the forwarding-cleanup idiom for a close that +races connection setup, and `sseChannel`'s bounded per-client queue (default 256 frames, clamped to +4,096) with heartbeat. `/api/status` remains a fully sufficient fallback for a client that has not +opened, or does not support, the stream — both paths return the identical `readIntelHistory(cwd)` +shape. + +## Invariants + +1. Pattern-store size and the patterns-learned counter are computed, labeled, and rendered + independently; neither substitutes for the other. +2. Every reader degrades to an honest empty/`null` result on a missing, unreadable, or + wrong-shaped source file rather than throwing or fabricating a value; individual malformed + entries are skipped rather than corrupting a bucket. +3. `readGlobalStats` and `status.mjs`'s CLI `learning` row read the same file through the same + helper and default logic, so the two cannot silently drift apart. +4. The health-history ring is capped and deduplicated; unchanged repeated snapshots do not grow it. +5. `pending-insights.jsonl` line contents are never read or trusted as data — only "a record + arrived" is a signal. +6. This domain introduces no session, actor, host, provider, model, or lifecycle identity, and no + per-field evidence-confidence grading. +7. `GET /api/live/intelligence` requires no `--live-source` registration; its sources are always + this project's own `.claude-flow/` state. +8. `/api/status` and `GET /api/live/intelligence` return the same `readIntelHistory(cwd)` shape, so + client rendering has one code path regardless of delivery route. + +## References + +- [ADR-0024](../adr/0024-project-intelligence-telemetry.md) +- [Observability](observability.md) — the bounded context this domain is deliberately distinct from +- [Context map](context-map.md) +- [Dashboard guide](../DASHBOARD.md) diff --git a/docs/ddd/ubiquitous-language.md b/docs/ddd/ubiquitous-language.md index 59a7628..46e7ebb 100644 --- a/docs/ddd/ubiquitous-language.md +++ b/docs/ddd/ubiquitous-language.md @@ -60,6 +60,20 @@ missing price. `Dual-host` describes two enabled peer hosts, not an execution command and not evidence that two inference vendors served a workflow. Generalized execution belongs to `ak run`. +## Project intelligence language + +| Term | Meaning | +|------|---------| +| Pattern store | The neural pattern store's current on-disk inventory (`.claude-flow/neural/patterns.json`); shrinks under pruning or compaction | +| Patterns-learned counter | A cumulative lifetime total (`.claude-flow/neural/stats.json`'s `patternsLearned`); only ever climbs | +| Reasoning graph sample | A point-in-time structural-size measurement (`nodes`, `edges`, `pageRankSum`) of the reasoning/knowledge graph | +| Health-history ring | The capped, deduplicated sample ring recording learning-stat snapshots over time | +| Project intelligence | Read-only trend telemetry over ruflo/agentic-qe's own local learning state, distinct from Observability evidence | + +Pattern-store size and the patterns-learned counter are never interchangeable displays of "how many +patterns exist" — the store can be pruned while the counter keeps climbing. See +[Project intelligence](project-intelligence.md). + ## Usage rules - Say **host** when referring to Claude Code, Codex, OpenCode, session drivers, leadership, or diff --git a/src/lib/dashboard-server.mjs b/src/lib/dashboard-server.mjs index 95d1c50..2a4d115 100644 --- a/src/lib/dashboard-server.mjs +++ b/src/lib/dashboard-server.mjs @@ -5,9 +5,15 @@ // no external fetches — offline-first, matches the kit ethos) // GET /api/status → JSON: the same subsystem rows `ak status --json` emits, // PLUS version drift, the project's .claude-flow/improvement.json -// (if present), and the health-history ring (if present). +// (if present), the health-history ring, and the intel-history +// (pattern store / graph / global stats) series — see +// dashboard/intel-history.mjs and collectData()'s own field +// comments for the exact, frozen shape of each. // GET /api/live → bounded, privacy-safe live session projection // GET /api/live/events → resumable Server-Sent Events stream +// GET /api/live/intelligence → SSE stream of intel-history.mjs's combined +// read; one initial frame on connect, then a fresh frame +// whenever IntelligenceWatch detects a real change // GET /api/usage → the usage Aggregate MINUS sessions[] (ADR-0009) // GET /api/sessions → the session list, filtered + paginated // GET /api/session/:id → one transcript, secrets masked SERVER-side @@ -34,6 +40,11 @@ import { renderPage } from './dashboard/page.mjs'; import { requestRejection } from './dashboard/request-security.mjs'; import { tokenMatches } from './admin-server.mjs'; import { sseChannel, reserveClientSlot, clientGone } from './dashboard/sse.mjs'; +// readHealthRing itself was moved (not duplicated) into intel-history.mjs — +// dashboard-server.mjs no longer defines it locally. It isn't called directly +// here because readIntelHistory() already composes it (as `.healthRing`, +// forwarded verbatim below); a bare `readHealthRing` import would be unused. +import { readIntelHistory } from './dashboard/intel-history.mjs'; import { TRANSCRIPT_ROOTS, maskMeta, @@ -70,15 +81,6 @@ function readJsonSafe(file) { try { return JSON.parse(fs.readFileSync(file, 'utf8')); } catch { return null; } } -/** The health-history ring: an array of point samples over time. Accepts either - * a bare array or `{ samples: [...] }`. Returns null when absent/unreadable. */ -function readHealthRing(cwd) { - const raw = readJsonSafe(path.join(cwd, '.claude-flow', 'health-history.json')); - if (!raw) return null; - const arr = Array.isArray(raw) ? raw : Array.isArray(raw.samples) ? raw.samples : null; - return arr && arr.length ? arr : null; -} - /** Default status provider: shell out to the installed CLI and parse its JSON. * Resilient — a spawn/parse failure resolves to an honest empty payload rather * than rejecting, so /api/status always answers with valid JSON. */ @@ -132,6 +134,13 @@ async function collectData({ cwd, fetchStatus }) { } catch { /* banner is best-effort — the subsystem card still carries the ruvector row */ } } + // Learning/intelligence history (src/lib/dashboard/intel-history.mjs) — one + // combined read powering the four DISTINCT series exposed below. See that + // module's header for the authoritative statement of why patternStore and + // globalStats.patternsLearned must never be conflated; the payload shape + // here preserves that separation instead of collapsing it. + const intel = readIntelHistory(cwd); + return { generatedAt: new Date().toISOString(), kit: { name: '@pacphi/agentic-kit', version: kitVersion() }, @@ -140,7 +149,37 @@ async function collectData({ cwd, fetchStatus }) { rows, drift, improvement: readJsonSafe(path.join(cwd, '.claude-flow', 'improvement.json')), - health: readHealthRing(cwd), + // health — UNCHANGED contract: the machine-health snapshot RING from + // .claude-flow/health-history.json (an array of point samples, each + // typically carrying a `patternsLearned` counter value and/or an + // `improvement`/`deltaPP` field — see appendHealthSnapshot's callers), or + // null if that file is absent. This is intel.healthRing forwarded + // verbatim; client.mjs's renderHistory() keeps reading it exactly as + // before this integration. + health: intel.healthRing, + // globalStats — the CURRENT (not historical) cumulative counters read + // straight from .claude-flow/neural/stats.json right now: + // { patternsLearned, trajectoriesRecorded, signalsProcessed, + // lastAdaptation }, or null if that file is absent. patternsLearned here + // is a LIFETIME counter and can legitimately be higher than + // patternStore.length below — the store gets pruned/compacted over time + // while this counter only ever climbs. Do not treat the two as + // interchangeable displays of "how many patterns exist". + globalStats: intel.globalStats, + // patternStore — every entry CURRENTLY PRESENT in the neural pattern + // store (.claude-flow/neural/patterns.json), as `{ createdAt, type }` + // pairs, NOT pre-bucketed by day (the client buckets). A point-in-time + // inventory of the store's live contents — distinct from + // globalStats.patternsLearned (a lifetime counter, see above) and + // distinct from health[] (machine-health samples, not pattern-store + // entries). + patternStore: intel.patternStore, + // graph — point-in-time samples of the reasoning graph's size over time + // (.claude-flow/data/intelligence-snapshot.json), as + // `{ timestamp, nodes, edges, pageRankSum }`, or null if that file is + // absent. A structural-growth series, independent of the three + // learning/pattern-count metrics above. + graph: intel.graph, routing: routingPayload(), }; } @@ -334,13 +373,29 @@ function lazyLive(liveOptions = {}) { }; } +/** Load the intelligence watcher only when /api/live/intelligence is first + * requested — same "pay only when used" rationale as lazyLive. Unlike + * LiveSessionsService's subscribe/replay pub-sub, IntelligenceWatch wires its + * `onUpdate` once, at construction; the route below fans that single callback + * out to every currently-connected client itself (see `broadcastIntel`). */ +function lazyIntelWatch(cwd, onUpdate) { + let instancePromise; + return async () => { + instancePromise ||= import('./live/intelligence-watch.mjs').then(({ IntelligenceWatch }) => ( + new IntelligenceWatch({ cwd, onUpdate }) + )); + return instancePromise; + }; +} + /** * Start the dashboard HTTP server, bound to loopback only. * @param {{ port?: number, cwd?: string, fetchStatus?: () => Promise, usage?: any, * limits?: () => Promise, live?: any, liveHeartbeatMs?: number, * liveClientBuffer?: number, liveMaxClients?: number, liveOptions?: any, * liveIdleMs?: number, transcripts?: any, transcriptOptions?: any, - * transcriptClientBuffer?: number, transcriptMaxClients?: number }} [opts] + * transcriptClientBuffer?: number, transcriptMaxClients?: number, + * intelWatch?: any, intelClientBuffer?: number, intelMaxClients?: number }} [opts] * @returns {Promise<{ url: string, urlWithToken: string, port: number, token: string, close: () => Promise }>} */ export function startDashboard({ @@ -348,6 +403,7 @@ export function startDashboard({ liveHeartbeatMs = 15_000, liveClientBuffer = 256, liveMaxClients = 32, liveOptions = {}, liveIdleMs = 30_000, transcripts, transcriptOptions = {}, transcriptClientBuffer = 64, transcriptMaxClients = 16, + intelWatch, intelClientBuffer = 256, intelMaxClients = 32, } = {}) { const provide = fetchStatus || shellOutStatus(cwd); const usageApi = usage || lazyUsage(); @@ -426,6 +482,40 @@ export function startDashboard({ .catch((error) => { liveStartPromise = null; throw error; })); return service; }; + + // ── /api/live/intelligence singleton plumbing ──────────────────────────── + // `intelClients` mirrors `liveClients`/`transcriptClients`: cap-tracking + // AND the set force-closed on shutdown. `intelWriters` is separate — the + // subset of those clients' raw `write` functions the watcher's single + // `onUpdate` callback broadcasts to, since (unlike LiveSessionsService) + // IntelligenceWatch has no built-in per-subscriber pub-sub of its own. + const intelClients = new Set(); + const intelWriters = new Set(); + const broadcastIntel = (combined) => { + const frame = transcriptSseFrame('update', combined); + for (const write of intelWriters) { + try { write(frame); } catch { /* a dead writer is reaped by its own cleanup */ } + } + }; + const provideIntelWatch = typeof intelWatch === 'function' + ? intelWatch : intelWatch ? async () => intelWatch : lazyIntelWatch(cwd, broadcastIntel); + let intelWatchPromise; + let intelWatchStartPromise; + let intelWatchStarted = false; + const getIntelWatch = async () => { + if (shuttingDown) throw new Error('dashboard is closing'); + const watch = await (intelWatchPromise ||= Promise.resolve().then(provideIntelWatch)); + if (shuttingDown) throw new Error('dashboard is closing'); + if (!watch || typeof watch.start !== 'function' || typeof watch.stop !== 'function') { + throw new TypeError('intelligence watch must implement start and stop'); + } + if (!intelWatchStarted) await (intelWatchStartPromise ||= Promise.resolve() + .then(() => watch.start()) + .then(() => { intelWatchStarted = true; }) + .catch((error) => { intelWatchStartPromise = null; throw error; })); + return watch; + }; + const html = renderPage({ name: '@pacphi/agentic-kit', version: kitVersion() }); // Per-session auth secret (ADR-0014, mirrors admin's ADR-0007 §2): 256-bit, @@ -473,7 +563,7 @@ export function startDashboard({ if (url === '/api/status') { let payload; try { payload = await collectData({ cwd, fetchStatus: provide }); } - catch (e) { payload = { generatedAt: new Date().toISOString(), overall: 'unknown', rows: [], drift: null, improvement: null, health: null, error: String(e && e.message || e) }; } + catch (e) { payload = { generatedAt: new Date().toISOString(), overall: 'unknown', rows: [], drift: null, improvement: null, health: null, globalStats: null, patternStore: [], graph: null, error: String(e && e.message || e) }; } sendJson(res, 200, payload); return; } @@ -626,6 +716,76 @@ export function startDashboard({ return; } + if (url === '/api/live/intelligence') { + // Same reservation-before-await discipline as /api/live/events above + // (sse.mjs's reserveClientSlot doc comment): getIntelWatch() may await a + // dynamic import() + watch.start() on the very first connection, and + // concurrent requests arriving during that gap must not all observe the + // same pre-reservation size and all pass the cap. + const maxClients = Math.max(1, Math.min(256, Number(intelMaxClients) || 32)); + const slot = reserveClientSlot(intelClients, maxClients); + if (!slot) { + sendJson(res, 503, { error: 'too many intelligence clients' }); + return; + } + // Same forwarding-cleanup pattern as /api/live/events and the transcript + // route: catches a close that fires during the awaits below, before the + // real cleanup (which needs the channel/write) can be constructed. + let earlyClosed = false; + let realCleanup = null; + const cleanup = (terminate) => { + if (realCleanup) { realCleanup(terminate); return; } + earlyClosed = true; + }; + req.once('close', cleanup); + res.once('close', cleanup); + + try { await getIntelWatch(); } catch { + intelClients.delete(slot); + sendJson(res, 503, { error: 'intelligence telemetry unavailable' }); + return; + } + if (earlyClosed || clientGone(req, res)) { intelClients.delete(slot); return; } + + res.writeHead(200, { + 'content-type': 'text/event-stream; charset=utf-8', + 'cache-control': 'no-store', + connection: 'keep-alive', + 'x-accel-buffering': 'no', + }); + res.flushHeaders?.(); + + const limit = Math.max(1, Math.min(4096, Number(intelClientBuffer) || 256)); + // Reuses transcriptSseFrame (plain id/event/data lines, no publicLivePayload + // redaction) rather than sseFrame — this payload is aggregate learning + // metrics, not live session/transcript content, so the session-privacy + // scrubbing sseFrame applies is not the right tool here. + const channel = sseChannel(res, { + limit, heartbeatMs: liveHeartbeatMs, + onOverflow: () => transcriptSseFrame('init', readIntelHistory(cwd)), + }); + const write = channel.write; + + realCleanup = (terminate = false) => { + if (channel.isClosed()) return; + channel.cleanup(terminate); + intelWriters.delete(write); + intelClients.delete(cleanup); + }; + intelClients.delete(slot); + intelClients.add(cleanup); + intelWriters.add(write); + if (earlyClosed) { cleanup(true); return; } + + // One initial frame with the current combined read so a fresh page load + // doesn't have to wait out the watcher's own debounce window; every + // frame after this is pushed by IntelligenceWatch's onUpdate via + // broadcastIntel. + write(transcriptSseFrame('init', readIntelHistory(cwd))); + channel.startHeartbeat(); + return; + } + const playbackMatch = /^\/api\/live\/playback\/([^/]+)\/([^/]+)$/.exec(url); if (playbackMatch) { const host = playbackMatch[1]; @@ -899,9 +1059,11 @@ export function startDashboard({ cancelLiveIdle(); for (const cleanup of [...liveClients]) cleanup(true); for (const cleanup of [...transcriptClients]) cleanup(true); + for (const cleanup of [...intelClients]) cleanup(true); await new Promise((res) => server.close(() => res(undefined))); await stopLive({ force: true }); try { await (await transcriptServicePromise)?.close?.(); } catch {} + try { (await intelWatchPromise)?.stop?.(); } catch {} }, }); }); diff --git a/src/lib/dashboard/client.mjs b/src/lib/dashboard/client.mjs index ca324f3..f96a6ee 100644 --- a/src/lib/dashboard/client.mjs +++ b/src/lib/dashboard/client.mjs @@ -131,6 +131,7 @@ export const JS = ` if(panel)panel.hidden=!on; } if(!skipHash&&activeTab==="overview")syncHash(); + syncIntelStream(); } function setTab(id,focus,skipHash){ if(TABS.indexOf(id)<0)return; @@ -153,6 +154,7 @@ export const JS = ` for(var j=0;j1?sparkline(pats):flat(pats.length?String(pats[0])+" (one sample)":"no data"); + + document.getElementById("spark-pattern-store").innerHTML=storeSeries.length>1?sparkline(storeSeries):flat(storeSeries.length?String(storeTotal)+" entries (one day)":"no data"); + + document.getElementById("spark-graph").innerHTML=nodesSeries.length>1?sparkline(nodesSeries):flat(nodesSeries.length?String(nodesSeries[0])+" nodes (one sample)":"no data"); + var graphMeta=document.getElementById("graph-meta"); + if(graphMeta)graphMeta.textContent=lastGraph?("latest: "+fmtNum(lastGraph.nodes)+" nodes · "+fmtNum(lastGraph.edges)+" edges"):""; + document.getElementById("spark-delta").innerHTML=deltas.length>1?sparkline(deltas):flat(deltas.length?(deltas[0]>=0?"+":"")+deltas[0]+"pp (one sample)":"no data"); + var deltaMeta=document.getElementById("delta-meta"); + if(deltaMeta){ + var verdict=imp&&typeof imp.verdict==="string"?imp.verdict:null; + var pVal=imp&&typeof imp.pValue==="number"?imp.pValue:null; + var dVal=imp&&typeof imp.cohensD==="number"?imp.cohensD:null; + if(!verdict&&pVal==null&&dVal==null){deltaMeta.hidden=true;deltaMeta.innerHTML="";} + else{ + var lvl=verdict==="PASS"?"ok":"warn"; + var pTxt=pVal==null?"—":(pVal<0.001?"<.001":"="+pVal); + var dTxt=dVal==null?"—":dVal.toFixed(2); + deltaMeta.hidden=false; + deltaMeta.innerHTML=(verdict?''+esc(verdict)+"":"") + +'p'+esc(pTxt)+" · d="+esc(dTxt)+""; + } + } + + document.getElementById("spark-curve").innerHTML=curveVals.length>1?sparkline(curveVals):flat(curveVals.length?(curveVals[0]*100).toFixed(0)+"% (one sample)":"no data"); } function renderRouting(rt){ @@ -388,6 +446,46 @@ export const JS = ` document.getElementById("model-list").innerHTML=html; } + // ── /api/live/intelligence (SSE) ── pushes fresh intel-history frames so the + // Intelligence panel repaints immediately instead of waiting out the ~30s + // poll tick. Mirrors live/client.mjs's openStream() for /api/live/events: + // same dashSseUrl token-in-query-param bridge (EventSource cannot set + // headers), same named-event addEventListener wiring, and no manual + // reconnect loop — EventSource retries natively on error, same as there. + function dashSseUrl(u){return DASH_TOKEN?u+(u.indexOf("?")<0?"?":"&")+"token="+encodeURIComponent(DASH_TOKEN):u;} + var intelSource=null; + function closeIntelStream(){ + if(intelSource){intelSource.close();intelSource=null;} + } + function receiveIntel(d){ + // readIntelHistory()'s own field names (healthRing) differ from + // collectData()'s renamed "health" on /api/status — reshape here, then + // funnel through the SAME renderHistory() the poll path uses, so there is + // exactly one code path for drawing the panel, not two. "improvement" isn't + // part of this stream's payload, so whatever the last poll saw stays put. + if(!d||typeof d!=="object"||!LAST)return; + LAST.health=d.healthRing; + LAST.globalStats=d.globalStats; + LAST.patternStore=d.patternStore; + LAST.graph=d.graph; + renderHistory(LAST); + } + function openIntelStream(){ + if(intelSource||!window.EventSource)return; + var src=new EventSource(dashSseUrl("/api/live/intelligence")); + intelSource=src; + src.addEventListener("init",function(e){try{receiveIntel(JSON.parse(e.data));}catch(e){}}); + src.addEventListener("update",function(e){try{receiveIntel(JSON.parse(e.data));}catch(e){}}); + src.onerror=function(){}; // native retry — nothing else to do here, same as openStream() + } + // Connected only while the Intelligence view is actually visible, closed the + // moment it isn't — the same "activate while shown, deactivate on hide" + // discipline AKLive.activate()/deactivate() applies to the Observability tab. + function syncIntelStream(){ + if(activeTab==="overview"&&overviewView==="intel")openIntelStream(); + else closeIntelStream(); + } + function render(data){ if(!data)return; LAST=data; diff --git a/src/lib/dashboard/intel-history.mjs b/src/lib/dashboard/intel-history.mjs new file mode 100644 index 0000000..b62288f --- /dev/null +++ b/src/lib/dashboard/intel-history.mjs @@ -0,0 +1,156 @@ +// intel-history.mjs — readers for the "learning intelligence" history the +// dashboard's intel views chart: the neural pattern store, the reasoning +// graph's point-in-time snapshots, the machine-health ring, and the neural +// global stats counters. All four sources are files this project's own +// ruflo/agentic-qe tooling already writes under .claude-flow/ — this module +// only reads (and, for the health ring, appends to) them; it invents nothing. +// +// IMPORTANT — two metrics that look alike but are NOT the same thing: +// - readNeuralPatternStoreHistory() counts ENTRIES actually present in +// .claude-flow/neural/patterns.json (the pattern store on disk right now). +// - readGlobalStats().patternsLearned is a cumulative COUNTER persisted in +// .claude-flow/neural/stats.json (patterns learned over the store's whole +// lifetime, including ones since pruned/compacted/replaced). +// These can legitimately diverge — the store can be pruned while the counter +// keeps climbing — and that divergence is not a bug. Do not conflate the two, +// and do not treat one as a substitute display for the other. +import path from 'node:path'; +import { readJson, writeJsonWithBackup } from '../settings.mjs'; + +const HEALTH_RING_CAP = 500; + +/** Coerce a raw createdAt value (epoch-ms number, ISO/parseable string) to an + * ISO-8601 string for day-bucketing by the caller. Returns null when the + * value can't be resolved to a real instant, so malformed entries are + * dropped rather than corrupting a bucket. */ +function toIsoTimestamp(value) { + if (typeof value === 'number' && Number.isFinite(value)) return new Date(value).toISOString(); + if (typeof value === 'string' && value.trim()) { + const parsed = Date.parse(value); + if (!Number.isNaN(parsed)) return new Date(parsed).toISOString(); + } + return null; +} + +/** + * Read .claude-flow/neural/patterns.json — a top-level JSON ARRAY of pattern + * entries (NOT an object with a `patterns` key). Returns [] if the file is + * missing, unreadable, or not shaped as an array; individual entries lacking + * a resolvable createdAt are skipped rather than throwing. + * @returns {Array<{ createdAt: string, type: string|null }>} + */ +export function readNeuralPatternStoreHistory(cwd) { + const data = readJson(path.join(cwd, '.claude-flow', 'neural', 'patterns.json')); + if (!Array.isArray(data)) return []; + const out = []; + for (const entry of data) { + if (!entry || typeof entry !== 'object') continue; + const createdAt = toIsoTimestamp(entry.createdAt); + if (createdAt == null) continue; + out.push({ createdAt, type: typeof entry.type === 'string' ? entry.type : null }); + } + return out; +} + +/** + * Read .claude-flow/data/intelligence-snapshot.json — already a JSON array of + * point-in-time graph samples on disk. Projects each sample down to the four + * scalar fields the dashboard charts. Returns null if missing, unreadable, or + * not shaped as an array, matching the null-on-absent convention readJsonSafe + * already uses elsewhere in dashboard-server.mjs. + * @returns {Array<{ timestamp: number, nodes: number, edges: number, pageRankSum: number }>|null} + */ +export function readGraphHistory(cwd) { + const data = readJson(path.join(cwd, '.claude-flow', 'data', 'intelligence-snapshot.json')); + if (!Array.isArray(data)) return null; + return data.map((entry) => ({ + timestamp: Number(entry?.timestamp) || 0, + nodes: Number(entry?.nodes) || 0, + edges: Number(entry?.edges) || 0, + pageRankSum: Number(entry?.pageRankSum) || 0, + })); +} + +/** + * Read .claude-flow/neural/stats.json — the SAME file src/commands/status.mjs + * reads for its 'learning' status row. Uses the shared readJson helper (the + * same one status.mjs imports from ./settings.mjs) and the same `?? 0` + * default logic, so the two call sites cannot drift apart. + * @returns {{ patternsLearned: number, trajectoriesRecorded: number, signalsProcessed: number, lastAdaptation: number }|null} + */ +export function readGlobalStats(cwd) { + const stats = readJson(path.join(cwd, '.claude-flow', 'neural', 'stats.json')); + if (!stats) return null; + return { + patternsLearned: stats.patternsLearned ?? 0, + trajectoriesRecorded: stats.trajectoriesRecorded ?? 0, + signalsProcessed: stats.signalsProcessed ?? 0, + lastAdaptation: stats.lastAdaptation ?? 0, + }; +} + +/** The health-history ring: an array of point samples over time. Accepts + * either a bare array or `{ samples: [...] }`. Returns null when absent, + * empty, or unreadable. Moved verbatim from dashboard-server.mjs (formerly + * a private function there) — behavior is unchanged. */ +export function readHealthRing(cwd) { + const raw = readJson(path.join(cwd, '.claude-flow', 'health-history.json')); + if (!raw) return null; + const arr = Array.isArray(raw) ? raw : Array.isArray(raw.samples) ? raw.samples : null; + return arr && arr.length ? arr : null; +} + +/** Deep-equal on plain JSON-shaped values (objects/arrays/primitives) — + * key-order independent, unlike a naive JSON.stringify comparison. */ +function deepEqual(a, b) { + if (a === b) return true; + if (typeof a !== typeof b || a === null || b === null) return false; + if (typeof a !== 'object') return false; + const aKeys = Object.keys(a); + const bKeys = Object.keys(b); + if (aKeys.length !== bKeys.length) return false; + return aKeys.every((k) => Object.hasOwn(b, k) && deepEqual(a[k], b[k])); +} + +/** Two snapshots are "the same" for dedup purposes when every field except + * `ts` matches. */ +function sameSnapshot(a, b) { + const { ts: _tsA, ...restA } = a ?? {}; + const { ts: _tsB, ...restB } = b ?? {}; + return deepEqual(restA, restB); +} + +/** + * Append `snapshot` to .claude-flow/health-history.json's samples ring, + * creating the file (seeded with just this snapshot) if it doesn't exist yet. + * A no-op — nothing is written — when `snapshot` is identical to the last + * stored row on every field except `ts` (dedup: repeated polling of unchanged + * stats must not grow the ring). The ring is capped at 500 entries, oldest + * dropped first. Writes via settings.mjs's writeJsonWithBackup, which reuses + * file-write.mjs's atomic backup-first replace rather than hand-rolling a + * tmp-file-then-rename. + * @returns {void} + */ +export function appendHealthSnapshot(cwd, snapshot) { + const file = path.join(cwd, '.claude-flow', 'health-history.json'); + const raw = readJson(file); + const existing = Array.isArray(raw) ? raw : Array.isArray(raw?.samples) ? raw.samples : []; + const last = existing.length ? existing[existing.length - 1] : null; + if (last && sameSnapshot(last, snapshot)) return; + const next = [...existing, snapshot]; + const capped = next.length > HEALTH_RING_CAP ? next.slice(next.length - HEALTH_RING_CAP) : next; + writeJsonWithBackup(file, { samples: capped }); +} + +/** + * Convenience combinator — everything collectData() needs for the intel + * history views in one call. + */ +export function readIntelHistory(cwd) { + return { + patternStore: readNeuralPatternStoreHistory(cwd), + graph: readGraphHistory(cwd), + healthRing: readHealthRing(cwd), + globalStats: readGlobalStats(cwd), + }; +} diff --git a/src/lib/dashboard/page.mjs b/src/lib/dashboard/page.mjs index ab65691..7b29a17 100644 --- a/src/lib/dashboard/page.mjs +++ b/src/lib/dashboard/page.mjs @@ -191,11 +191,28 @@ export function renderPage({ name, version }) {
patterns learned
+
lifetime counter (neural/stats.json) — only ever climbs, even as the store below is pruned
-
improvement Δpp
+
pattern store size
+
+
entries currently on disk (neural/patterns.json), by day created — a different number from the lifetime counter
+
+
+
reasoning graph size
+
+
+
+
+
improvement Δpp
+
+
+
learning curve (cold→warm)
+
+
held-out accuracy within this eval run, at each k-step checkpoint
+
diff --git a/src/lib/live/intelligence-watch.mjs b/src/lib/live/intelligence-watch.mjs new file mode 100644 index 0000000..ca6782f --- /dev/null +++ b/src/lib/live/intelligence-watch.mjs @@ -0,0 +1,227 @@ +// intelligence-watch.mjs — detects when this project's own learning/ +// intelligence data actually changes on disk, and reacts by (a) persisting a +// health-history snapshot (via intel-history.mjs's appendHealthSnapshot) and +// (b) handing a fresh combined read to a caller-supplied onUpdate callback, +// so a dashboard SSE route can push near-instant updates instead of waiting +// out its own slower poll loop. +// +// Two independent, complementary change signals feed one debounced pipeline: +// 1. `.claude-flow/data/pending-insights.jsonl` is tailed with JsonlTailer +// purely as a cheap, low-latency "something just happened" trigger — +// its line CONTENTS are never read or trusted as data. See +// jsonl-tailer.mjs's own doc comment for why polling (not fs.watch) is +// deliberate there; the same reasoning applies here to the direct mtime +// polling below: fs.watch is only a hint on several filesystems, while +// stat reconciliation is honest about what actually changed. +// 2. `.claude-flow/neural/stats.json`, `.claude-flow/neural/patterns.json`, +// and `.claude-flow/data/intelligence-snapshot.json` mtimes are polled +// directly via fs.statSync, as a fallback AND a primary signal in their +// own right — not every real change to these files produces a +// pending-insights line. +// Either signal marks a pending change; the poll loop itself measures the +// quiet gap since the most recent one against `debounceMs` (a trailing-edge +// debounce built entirely from the injected `now`/`setInterval` primitives — +// no extra timer primitive needed) so a burst of edits during an active +// session collapses into a single onUpdate call per debounce window. +// +// Two DIFFERENT metrics ride through here and must not be conflated (see +// intel-history.mjs's own header for the authoritative statement): the +// pattern-STORE history (readNeuralPatternStoreHistory, sourced from +// patterns.json — entries actually present right now) and the +// patternsLearned COUNTER inside globalStats (sourced from stats.json — a +// cumulative lifetime count). They can legitimately diverge; this module +// forwards both, unmodified, and treats neither as a stand-in for the other. +// +// Design note on readers: intel-history.mjs's contract is frozen, but this +// module intentionally never hard-depends on module-load timing — the three +// reader functions are constructor-injectable, defaulting to the real +// intel-history.mjs exports. Production callers get real reads for free; +// tests inject fakes and never touch the filesystem. +import fs from 'node:fs'; +import path from 'node:path'; +import { projectClaudeFlowDir } from '../paths.mjs'; +import { JsonlTailer } from './index.mjs'; +import { + readGlobalStats as defaultReadGlobalStats, + readIntelHistory as defaultReadIntelHistory, + appendHealthSnapshot as defaultAppendHealthSnapshot, +} from '../dashboard/intel-history.mjs'; + +/** Shallow, key-order-independent equality over the flat numeric objects + * readGlobalStats returns (or null). Good enough for change detection — + * this never needs to compare nested structures. */ +function statsEqual(a, b) { + if (a === b) return true; + if (!a || !b || typeof a !== 'object' || typeof b !== 'object') return false; + const aKeys = Object.keys(a); + const bKeys = Object.keys(b); + if (aKeys.length !== bKeys.length) return false; + return aKeys.every((key) => Object.hasOwn(b, key) && Object.is(a[key], b[key])); +} + +/** + * Watches this project's on-disk learning/intelligence sources for real + * changes and pushes fresh combined reads through `onUpdate` — debounced, + * so a burst of writes collapses into one push. + */ +export class IntelligenceWatch { + #options; + #cwd; + #watchedFiles; + #tailer; + #timer = null; + #started = false; + #mtimes = new Map(); + #tailerDirty = false; + #pendingChange = false; + #lastChangeAt = 0; + #lastGlobalStats; + + /** + * `onUpdate` is typed optional only so the `options = {}` default below + * type-checks (TS requires a bare-object default to satisfy every + * non-optional property); it is still functionally REQUIRED — the + * constructor throws a TypeError immediately below when it's missing or + * not a function. + * @param {{ + * cwd?: string, + * onUpdate?: (combined: { patternStore: unknown[], graph: unknown[]|null, + * healthRing: unknown[]|null, globalStats: object|null }) => void, + * onError?: (error: unknown) => void, + * pendingInsightsFile?: string, + * watchedFiles?: string[], + * pollIntervalMs?: number, + * debounceMs?: number, + * setInterval?: typeof globalThis.setInterval, + * clearInterval?: typeof globalThis.clearInterval, + * now?: () => number, + * fsImpl?: Pick, + * readGlobalStats?: (cwd: string) => object|null, + * readIntelHistory?: (cwd: string) => object, + * appendHealthSnapshot?: (cwd: string, snapshot: object) => void, + * }} options + */ + constructor(options = {}) { + if (typeof options.onUpdate !== 'function') throw new TypeError('onUpdate is required'); + const cwd = options.cwd ?? process.cwd(); + const claudeFlowDir = projectClaudeFlowDir(cwd); + this.#cwd = cwd; + this.#watchedFiles = Array.isArray(options.watchedFiles) ? options.watchedFiles : [ + path.join(claudeFlowDir, 'neural', 'stats.json'), + path.join(claudeFlowDir, 'neural', 'patterns.json'), + path.join(claudeFlowDir, 'data', 'intelligence-snapshot.json'), + ]; + this.#options = { + onUpdate: options.onUpdate, + onError: options.onError ?? (() => {}), + pollIntervalMs: options.pollIntervalMs ?? 1_000, + debounceMs: options.debounceMs ?? 2_500, + setInterval: options.setInterval ?? globalThis.setInterval, + clearInterval: options.clearInterval ?? globalThis.clearInterval, + now: options.now ?? (() => Date.now()), + fsImpl: options.fsImpl ?? fs, + readGlobalStats: options.readGlobalStats ?? defaultReadGlobalStats, + readIntelHistory: options.readIntelHistory ?? defaultReadIntelHistory, + appendHealthSnapshot: options.appendHealthSnapshot ?? defaultAppendHealthSnapshot, + }; + const pendingInsightsFile = options.pendingInsightsFile + ?? path.join(claudeFlowDir, 'data', 'pending-insights.jsonl'); + // Lines are never inspected — reconcile() firing onRecord at all IS the + // signal. Started at end so pre-existing history doesn't manufacture a + // change on the very first poll (the mtime baseline below already covers + // "first observation" for the three data files themselves). + this.#tailer = new JsonlTailer(pendingInsightsFile, { + onRecord: () => { this.#tailerDirty = true; }, + onError: (error) => this.#options.onError(error), + startAtEnd: true, + }); + } + + start() { + if (this.#started) return this; + this.#started = true; + this.#poll(); + this.#timer = this.#options.setInterval(() => this.#poll(), this.#options.pollIntervalMs); + this.#timer?.unref?.(); + return this; + } + + stop() { + if (this.#timer != null) this.#options.clearInterval(this.#timer); + this.#timer = null; + this.#tailer.close(); + this.#started = false; + return this; + } + + #poll() { + let changed = false; + for (const file of this.#watchedFiles) { + let mtimeMs; + try { + mtimeMs = this.#options.fsImpl.statSync(file).mtimeMs; + } catch (error) { + if (error?.code !== 'ENOENT') this.#options.onError(error); + mtimeMs = null; + } + const seen = this.#mtimes.has(file); + if (!seen || this.#mtimes.get(file) !== mtimeMs) { + changed = true; + this.#mtimes.set(file, mtimeMs); + } + } + try { + this.#tailer.reconcile(); + } catch (error) { + this.#options.onError(error); + } + if (this.#tailerDirty) { + changed = true; + this.#tailerDirty = false; + } + const now = this.#options.now(); + if (changed) { + this.#pendingChange = true; + this.#lastChangeAt = now; + } + if (this.#pendingChange && now - this.#lastChangeAt >= this.#options.debounceMs) { + this.#pendingChange = false; + this.#flush(); + } + } + + #flush() { + let globalStats = null; + try { + globalStats = this.#options.readGlobalStats(this.#cwd); + } catch (error) { + this.#options.onError(error); + } + if (!statsEqual(globalStats, this.#lastGlobalStats)) { + this.#lastGlobalStats = globalStats; + if (globalStats) { + try { + this.#options.appendHealthSnapshot(this.#cwd, { + ts: this.#options.now(), + ...globalStats, + }); + } catch (error) { + this.#options.onError(error); + } + } + } + let combined = null; + try { + combined = this.#options.readIntelHistory(this.#cwd); + } catch (error) { + this.#options.onError(error); + } + if (combined) { + try { + this.#options.onUpdate(combined); + } catch (error) { + this.#options.onError(error); + } + } + } +} diff --git a/tests/kit/dashboard-intel-integration.test.mjs b/tests/kit/dashboard-intel-integration.test.mjs new file mode 100644 index 0000000..1b97eb4 --- /dev/null +++ b/tests/kit/dashboard-intel-integration.test.mjs @@ -0,0 +1,181 @@ +// dashboard-intel-integration.test.mjs — end-to-end coverage for the seam +// between dashboard-server.mjs's collectData() (served at GET /api/status) +// and dashboard/intel-history.mjs's readers. Boots the REAL HTTP server +// (startDashboard) over an isolated mkdtempSync() fixture dir shaped like a +// real project's .claude-flow/ tree — no real project files are ever +// touched — and asserts the four documented fields (`health`, `globalStats`, +// `patternStore`, `graph`) survive the collectData → JSON round trip exactly +// as intel-history.mjs's own readers would produce them, INCLUDING the +// file-does-not-exist-yet case, which must 200 rather than throw. +// +// fetchStatus is injected (a stub, never `ak status --json`) and its stub +// `drift` is always a real array so collectData takes the caller-supplied +// drift path and never falls through to the network-touching self-computed +// path (driftReport / the brain and ruvector drift folds) — see +// tests/dashboard.test.cjs's own comment on the same hazard. +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import fs from 'node:fs'; +import os from 'node:os'; +import path from 'node:path'; +import http from 'node:http'; +import { startDashboard } from '../../src/lib/dashboard-server.mjs'; +import { + readNeuralPatternStoreHistory, + readGraphHistory, + readGlobalStats, + readHealthRing, +} from '../../src/lib/dashboard/intel-history.mjs'; + +const STUB_STATUS = { overall: 'ok', rows: [], drift: [] }; + +function mkFixture(files) { + const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'ak-dash-intel-')); + for (const [rel, data] of Object.entries(files)) { + const fp = path.join(dir, rel); + fs.mkdirSync(path.dirname(fp), { recursive: true }); + fs.writeFileSync(fp, typeof data === 'string' ? data : JSON.stringify(data)); + } + return dir; +} + +function get(url, token) { + return new Promise((resolve, reject) => { + const opts = token ? { headers: { 'x-dash-token': token } } : {}; + http.get(url, opts, (res) => { + let body = ''; + res.setEncoding('utf8'); + res.on('data', (c) => { body += c; }); + res.on('end', () => resolve({ status: res.statusCode, headers: res.headers, body })); + }).on('error', reject); + }); +} + +test('GET /api/status assembles globalStats/patternStore/graph/health exactly as intel-history.mjs would read them off a full fixture', async () => { + const fixture = mkFixture({ + '.claude-flow/neural/stats.json': { + patternsLearned: 1337, trajectoriesRecorded: 1400, signalsProcessed: 1266, lastAdaptation: 1785915702033, + }, + '.claude-flow/neural/patterns.json': [ + { id: 'a', type: 'action', createdAt: 1780024063013, embedding: [1, 2], content: 'x' }, + { id: 'b', type: 'result', createdAt: 1783106501523, embedding: [3, 4], content: 'y' }, + ], + '.claude-flow/data/intelligence-snapshot.json': [ + { timestamp: 1785257673755, nodes: 110, edges: 1568, pageRankSum: 1, confidences: [0.5], topPatterns: [{ id: 'x' }] }, + ], + '.claude-flow/health-history.json': [ + { ts: 1700000000, patternsLearned: 10, deltaPP: 5 }, + { ts: 1700000600, patternsLearned: 22, deltaPP: 18 }, + ], + }); + + const { url, close, token } = await startDashboard({ + port: 0, cwd: fixture, fetchStatus: async () => STUB_STATUS, + }); + try { + const r = await get(`${url}api/status`, token); + assert.equal(r.status, 200, `expected 200, got ${r.status}`); + const body = JSON.parse(r.body); + + // Cross-checked against the SAME readers intel-history.mjs's own unit + // tests exercise directly, over the same fixture dir — the seam under + // test is collectData()'s wiring, not the readers' own field logic + // (that's intel-history.test.mjs's job). + assert.deepEqual(body.globalStats, readGlobalStats(fixture)); + assert.deepEqual(body.patternStore, readNeuralPatternStoreHistory(fixture)); + assert.deepEqual(body.graph, readGraphHistory(fixture)); + assert.deepEqual(body.health, readHealthRing(fixture)); + + // Field-by-field shape assertions per the frozen contract (dashboard-server.mjs's + // own doc comments above collectData's return). + assert.deepEqual(body.globalStats, { + patternsLearned: 1337, trajectoriesRecorded: 1400, signalsProcessed: 1266, lastAdaptation: 1785915702033, + }); + assert.deepEqual(body.patternStore, [ + { createdAt: new Date(1780024063013).toISOString(), type: 'action' }, + { createdAt: new Date(1783106501523).toISOString(), type: 'result' }, + ]); + assert.deepEqual(body.graph, [ + { timestamp: 1785257673755, nodes: 110, edges: 1568, pageRankSum: 1 }, + ]); + assert.equal(body.health.length, 2); + + // patternsLearned (a lifetime counter) and patternStore.length (entries + // currently on disk) must remain independently reported, never collapsed + // into one figure — the exact divergence intel-history.mjs's header + // comment documents and this fixture deliberately exercises (1337 vs 2). + assert.notEqual(body.globalStats.patternsLearned, body.patternStore.length); + } finally { + await close(); + } +}); + +test('GET /api/status does not throw when NONE of the intel-history sources exist yet — the fresh-project path', async () => { + const fixture = mkFixture({}); // no .claude-flow/ at all + const { url, close, token } = await startDashboard({ + port: 0, cwd: fixture, fetchStatus: async () => STUB_STATUS, + }); + try { + const r = await get(`${url}api/status`, token); + assert.equal(r.status, 200, `expected 200 (never a throw), got ${r.status}`); + const body = JSON.parse(r.body); + assert.equal(body.health, null); + assert.equal(body.globalStats, null); + assert.deepEqual(body.patternStore, []); + assert.equal(body.graph, null); + } finally { + await close(); + } +}); + +test('GET /api/status tolerates a partial fixture — health-history.json absent while the other three sources are present', async () => { + // Exercises the exact "create fresh" path Module A's report called out: + // health-history.json genuinely does not exist yet in a real project even + // once neural/graph data does, and that must read as null, not a crash. + const fixture = mkFixture({ + '.claude-flow/neural/stats.json': { patternsLearned: 5 }, + '.claude-flow/neural/patterns.json': [{ id: 'a', type: 'action', createdAt: 1 }], + '.claude-flow/data/intelligence-snapshot.json': [{ timestamp: 1, nodes: 2, edges: 3, pageRankSum: 0.5 }], + }); + assert.equal(fs.existsSync(path.join(fixture, '.claude-flow', 'health-history.json')), false); + + const { url, close, token } = await startDashboard({ + port: 0, cwd: fixture, fetchStatus: async () => STUB_STATUS, + }); + try { + const r = await get(`${url}api/status`, token); + assert.equal(r.status, 200); + const body = JSON.parse(r.body); + assert.equal(body.health, null, 'health-history.json absent -> null, not a throw'); + assert.deepEqual(body.globalStats, { + patternsLearned: 5, trajectoriesRecorded: 0, signalsProcessed: 0, lastAdaptation: 0, + }); + assert.equal(body.patternStore.length, 1); + assert.equal(body.graph.length, 1); + } finally { + await close(); + } +}); + +test('GET /api/status tolerates malformed JSON in every intel source at once without a 500', async () => { + const fixture = mkFixture({ + '.claude-flow/neural/stats.json': '{ not json', + '.claude-flow/neural/patterns.json': 'not json at all', + '.claude-flow/data/intelligence-snapshot.json': '[unterminated', + '.claude-flow/health-history.json': 'also not json', + }); + const { url, close, token } = await startDashboard({ + port: 0, cwd: fixture, fetchStatus: async () => STUB_STATUS, + }); + try { + const r = await get(`${url}api/status`, token); + assert.equal(r.status, 200, 'malformed on-disk JSON must degrade gracefully, never 500'); + const body = JSON.parse(r.body); + assert.equal(body.health, null); + assert.equal(body.globalStats, null); + assert.deepEqual(body.patternStore, []); + assert.equal(body.graph, null); + } finally { + await close(); + } +}); diff --git a/tests/kit/intel-history.test.mjs b/tests/kit/intel-history.test.mjs new file mode 100644 index 0000000..4c02f6f --- /dev/null +++ b/tests/kit/intel-history.test.mjs @@ -0,0 +1,249 @@ +// intel-history.mjs — readers for the neural pattern store, the graph +// snapshot history, the neural global stats counters, and the machine-health +// ring, plus the append-with-dedup writer for that ring. Fixtures are written +// under an isolated mkdtempSync() dir shaped like a real project's +// .claude-flow/ tree; no real project files are ever touched. +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import fs from 'node:fs'; +import os from 'node:os'; +import path from 'node:path'; +import { + readNeuralPatternStoreHistory, + readGraphHistory, + readGlobalStats, + readHealthRing, + appendHealthSnapshot, + readIntelHistory, +} from '../../src/lib/dashboard/intel-history.mjs'; + +const tmp = () => fs.mkdtempSync(path.join(os.tmpdir(), 'ak-intel-history-')); + +/** Write `content` (object → JSON.stringify'd, string → written verbatim so + * malformed-JSON fixtures are easy to express) at cwd-relative `relPath`, + * creating parent dirs as needed. */ +function writeFixture(cwd, relPath, content) { + const file = path.join(cwd, relPath); + fs.mkdirSync(path.dirname(file), { recursive: true }); + fs.writeFileSync(file, typeof content === 'string' ? content : JSON.stringify(content)); + return file; +} + +// ── readNeuralPatternStoreHistory ─────────────────────────────────────────── + +test('readNeuralPatternStoreHistory reads the top-level array shape and maps createdAt/type', () => { + const cwd = tmp(); + writeFixture(cwd, '.claude-flow/neural/patterns.json', [ + { id: 'a', type: 'action', createdAt: 1780024063013, embedding: [1, 2], content: 'x' }, + { id: 'b', type: 'result', createdAt: 1783106501523, embedding: [3, 4], content: 'y' }, + ]); + const rows = readNeuralPatternStoreHistory(cwd); + assert.deepEqual(rows, [ + { createdAt: new Date(1780024063013).toISOString(), type: 'action' }, + { createdAt: new Date(1783106501523).toISOString(), type: 'result' }, + ]); +}); + +test('readNeuralPatternStoreHistory defaults a missing type to null and drops entries with no resolvable createdAt', () => { + const cwd = tmp(); + writeFixture(cwd, '.claude-flow/neural/patterns.json', [ + { id: 'no-type', createdAt: 1780024063013 }, + { id: 'no-timestamp', type: 'action' }, + { id: 'bad-timestamp', type: 'action', createdAt: 'not-a-date' }, + ]); + const rows = readNeuralPatternStoreHistory(cwd); + assert.deepEqual(rows, [ + { createdAt: new Date(1780024063013).toISOString(), type: null }, + ]); +}); + +test('readNeuralPatternStoreHistory returns [] for a missing file', () => { + const cwd = tmp(); + assert.deepEqual(readNeuralPatternStoreHistory(cwd), []); +}); + +test('readNeuralPatternStoreHistory returns [] for malformed JSON text', () => { + const cwd = tmp(); + writeFixture(cwd, '.claude-flow/neural/patterns.json', '{not valid json'); + assert.deepEqual(readNeuralPatternStoreHistory(cwd), []); +}); + +test('readNeuralPatternStoreHistory returns [] when the file is an object, not a top-level array', () => { + const cwd = tmp(); + writeFixture(cwd, '.claude-flow/neural/patterns.json', { patterns: [{ id: 'a', type: 'action', createdAt: 1 }] }); + assert.deepEqual(readNeuralPatternStoreHistory(cwd), []); +}); + +// ── readGraphHistory ───────────────────────────────────────────────────── + +test('readGraphHistory projects each sample down to the four scalar fields', () => { + const cwd = tmp(); + writeFixture(cwd, '.claude-flow/data/intelligence-snapshot.json', [ + { timestamp: 1785257673755, nodes: 55, edges: 128, pageRankSum: 1, confidences: [0.5], topPatterns: [{ id: 'x' }] }, + ]); + const rows = readGraphHistory(cwd); + assert.deepEqual(rows, [ + { timestamp: 1785257673755, nodes: 55, edges: 128, pageRankSum: 1 }, + ]); +}); + +test('readGraphHistory returns null for a missing file', () => { + const cwd = tmp(); + assert.equal(readGraphHistory(cwd), null); +}); + +test('readGraphHistory returns null when the file is not a JSON array', () => { + const cwd = tmp(); + writeFixture(cwd, '.claude-flow/data/intelligence-snapshot.json', { not: 'an array' }); + assert.equal(readGraphHistory(cwd), null); +}); + +test('readGraphHistory returns null for malformed JSON text', () => { + const cwd = tmp(); + writeFixture(cwd, '.claude-flow/data/intelligence-snapshot.json', 'not json at all'); + assert.equal(readGraphHistory(cwd), null); +}); + +// ── readGlobalStats ────────────────────────────────────────────────────── + +test('readGlobalStats reads .claude-flow/neural/stats.json with the same ?? 0 defaulting status.mjs uses', () => { + const cwd = tmp(); + writeFixture(cwd, '.claude-flow/neural/stats.json', { + trajectoriesRecorded: 1400, + patternsLearned: 1337, + signalsProcessed: 1266, + lastAdaptation: 1785915702033, + }); + assert.deepEqual(readGlobalStats(cwd), { + patternsLearned: 1337, + trajectoriesRecorded: 1400, + signalsProcessed: 1266, + lastAdaptation: 1785915702033, + }); +}); + +test('readGlobalStats defaults missing numeric fields to 0', () => { + const cwd = tmp(); + writeFixture(cwd, '.claude-flow/neural/stats.json', { patternsLearned: 5 }); + assert.deepEqual(readGlobalStats(cwd), { + patternsLearned: 5, + trajectoriesRecorded: 0, + signalsProcessed: 0, + lastAdaptation: 0, + }); +}); + +test('readGlobalStats returns null for a missing file', () => { + const cwd = tmp(); + assert.equal(readGlobalStats(cwd), null); +}); + +test('readGlobalStats returns null for malformed JSON text', () => { + const cwd = tmp(); + writeFixture(cwd, '.claude-flow/neural/stats.json', '{ broken'); + assert.equal(readGlobalStats(cwd), null); +}); + +// ── readHealthRing ─────────────────────────────────────────────────────── + +test('readHealthRing accepts a bare array', () => { + const cwd = tmp(); + const samples = [{ ts: 1, ok: true }, { ts: 2, ok: false }]; + writeFixture(cwd, '.claude-flow/health-history.json', samples); + assert.deepEqual(readHealthRing(cwd), samples); +}); + +test('readHealthRing accepts an object with a samples array', () => { + const cwd = tmp(); + const samples = [{ ts: 1, ok: true }]; + writeFixture(cwd, '.claude-flow/health-history.json', { samples }); + assert.deepEqual(readHealthRing(cwd), samples); +}); + +test('readHealthRing returns null for a missing file', () => { + const cwd = tmp(); + assert.equal(readHealthRing(cwd), null); +}); + +test('readHealthRing returns null for an empty array', () => { + const cwd = tmp(); + writeFixture(cwd, '.claude-flow/health-history.json', []); + assert.equal(readHealthRing(cwd), null); +}); + +test('readHealthRing returns null for malformed JSON text', () => { + const cwd = tmp(); + writeFixture(cwd, '.claude-flow/health-history.json', 'not json'); + assert.equal(readHealthRing(cwd), null); +}); + +// ── appendHealthSnapshot ───────────────────────────────────────────────── + +test('appendHealthSnapshot creates the file fresh, seeded with just this snapshot', () => { + const cwd = tmp(); + const file = path.join(cwd, '.claude-flow', 'health-history.json'); + assert.equal(fs.existsSync(file), false); + appendHealthSnapshot(cwd, { ts: 100, patternsLearned: 5, trajectoriesRecorded: 2, signalsProcessed: 1 }); + assert.equal(fs.existsSync(file), true); + const onDisk = JSON.parse(fs.readFileSync(file, 'utf8')); + assert.deepEqual(onDisk, { + samples: [{ ts: 100, patternsLearned: 5, trajectoriesRecorded: 2, signalsProcessed: 1 }], + }); + assert.deepEqual(readHealthRing(cwd), [{ ts: 100, patternsLearned: 5, trajectoriesRecorded: 2, signalsProcessed: 1 }]); +}); + +test('appendHealthSnapshot dedups: an identical snapshot (aside from ts) appended twice leaves the ring length unchanged', () => { + const cwd = tmp(); + appendHealthSnapshot(cwd, { ts: 1, patternsLearned: 5, trajectoriesRecorded: 2, signalsProcessed: 1 }); + appendHealthSnapshot(cwd, { ts: 2, patternsLearned: 5, trajectoriesRecorded: 2, signalsProcessed: 1 }); + const ring = readHealthRing(cwd); + assert.equal(ring.length, 1); + assert.equal(ring[0].ts, 1); // second call was a no-op; the original row stands +}); + +test('appendHealthSnapshot appends a new row when a field actually changed', () => { + const cwd = tmp(); + appendHealthSnapshot(cwd, { ts: 1, patternsLearned: 5, trajectoriesRecorded: 2, signalsProcessed: 1 }); + appendHealthSnapshot(cwd, { ts: 2, patternsLearned: 6, trajectoriesRecorded: 2, signalsProcessed: 1 }); + const ring = readHealthRing(cwd); + assert.equal(ring.length, 2); + assert.equal(ring[1].patternsLearned, 6); +}); + +test('appendHealthSnapshot caps the ring at 500 entries, dropping the oldest first', () => { + const cwd = tmp(); + for (let i = 0; i <= 500; i++) { + appendHealthSnapshot(cwd, { ts: i, patternsLearned: i, trajectoriesRecorded: 0, signalsProcessed: 0 }); + } + const ring = readHealthRing(cwd); + assert.equal(ring.length, 500); + assert.equal(ring[0].patternsLearned, 1); // entry 0 was evicted + assert.equal(ring[ring.length - 1].patternsLearned, 500); +}); + +// ── readIntelHistory (combinator) ──────────────────────────────────────── + +test('readIntelHistory combines all four readers', () => { + const cwd = tmp(); + writeFixture(cwd, '.claude-flow/neural/patterns.json', [{ id: 'a', type: 'action', createdAt: 1780024063013 }]); + writeFixture(cwd, '.claude-flow/data/intelligence-snapshot.json', [{ timestamp: 1, nodes: 2, edges: 3, pageRankSum: 1 }]); + writeFixture(cwd, '.claude-flow/neural/stats.json', { patternsLearned: 9 }); + writeFixture(cwd, '.claude-flow/health-history.json', [{ ts: 1 }]); + + assert.deepEqual(readIntelHistory(cwd), { + patternStore: readNeuralPatternStoreHistory(cwd), + graph: readGraphHistory(cwd), + healthRing: readHealthRing(cwd), + globalStats: readGlobalStats(cwd), + }); +}); + +test('readIntelHistory tolerates every source being absent', () => { + const cwd = tmp(); + assert.deepEqual(readIntelHistory(cwd), { + patternStore: [], + graph: null, + healthRing: null, + globalStats: null, + }); +}); diff --git a/tests/kit/intelligence-watch.test.mjs b/tests/kit/intelligence-watch.test.mjs new file mode 100644 index 0000000..e07cbe0 --- /dev/null +++ b/tests/kit/intelligence-watch.test.mjs @@ -0,0 +1,367 @@ +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import fs from 'node:fs'; +import os from 'node:os'; +import path from 'node:path'; +import { IntelligenceWatch } from '../../src/lib/live/intelligence-watch.mjs'; + +const sandbox = () => fs.mkdtempSync(path.join(os.tmpdir(), 'ak-intel-watch-')); + +/** A controllable fake clock: `now()` returns the current fake ms, `advance` + * moves it forward without any real wall-clock wait. */ +const makeClock = (startMs = 0) => { + let ms = startMs; + return { now: () => ms, advance: (delta) => { ms += delta; } }; +}; + +/** A fake interval: captures the callback `start()` registers so the test can + * fire poll ticks manually, deterministically, with no real timers. */ +const fakeTimer = () => { + let tick = null; + let handle = null; + return { + setInterval: (fn) => { + tick = fn; + handle = { unref() {} }; + return handle; + }, + clearInterval: () => {}, + fire: () => tick(), + }; +}; + +test('constructor requires an onUpdate callback', () => { + assert.throws(() => new IntelligenceWatch({ cwd: sandbox() }), TypeError); +}); + +test('flushes once, only after debounceMs of quiet following a watched-file change', () => { + const dir = sandbox(); + const file = path.join(dir, 'stats.json'); + fs.writeFileSync(file, '{}'); + const clock = makeClock(0); + const timer = fakeTimer(); + const updates = []; + const watcher = new IntelligenceWatch({ + cwd: dir, + watchedFiles: [file], + pendingInsightsFile: path.join(dir, 'pending.jsonl'), + debounceMs: 2_500, + setInterval: timer.setInterval, + clearInterval: timer.clearInterval, + now: clock.now, + readGlobalStats: () => null, + readIntelHistory: () => ({ n: updates.length }), + appendHealthSnapshot: () => {}, + onUpdate: (combined) => updates.push(combined), + }); + + watcher.start(); // t=0: file exists for the first time -> a detected change + assert.equal(updates.length, 0, 'must not flush on the same tick a change is first observed'); + + clock.advance(1_000); + timer.fire(); // t=1000 < debounceMs since the change + assert.equal(updates.length, 0); + + clock.advance(1_000); + timer.fire(); // t=2000 < debounceMs since the change + assert.equal(updates.length, 0); + + clock.advance(600); + timer.fire(); // t=2600 >= debounceMs(2500) since the change -> flush + assert.equal(updates.length, 1); + + clock.advance(10_000); + timer.fire(); // no new change -> must not flush again + assert.equal(updates.length, 1); +}); + +test('coalesces a burst of file changes into a single debounced flush', () => { + const dir = sandbox(); + const file = path.join(dir, 'stats.json'); + fs.writeFileSync(file, '{}'); + const clock = makeClock(0); + const timer = fakeTimer(); + const updates = []; + const watcher = new IntelligenceWatch({ + cwd: dir, + watchedFiles: [file], + pendingInsightsFile: path.join(dir, 'pending.jsonl'), + debounceMs: 2_500, + setInterval: timer.setInterval, + clearInterval: timer.clearInterval, + now: clock.now, + readGlobalStats: () => null, + readIntelHistory: () => ({ n: updates.length }), + appendHealthSnapshot: () => {}, + onUpdate: (combined) => updates.push(combined), + }); + + watcher.start(); // t=0: baseline change, lastChangeAt=0 + + clock.advance(1_000); + fs.utimesSync(file, new Date(1_000), new Date(1_000)); // burst edit #2 + timer.fire(); // t=1000: another change resets the quiet window + assert.equal(updates.length, 0); + + clock.advance(1_000); + fs.utimesSync(file, new Date(2_000), new Date(2_000)); // burst edit #3 + timer.fire(); // t=2000: another change resets the quiet window again + assert.equal(updates.length, 0); + + clock.advance(2_500); + timer.fire(); // t=4500: 2500ms quiet since the LAST edit (t=2000) -> exactly one flush + assert.equal(updates.length, 1, 'three edits within the window must collapse into one onUpdate call'); +}); + +test('a pending-insights.jsonl append triggers a flush even when watched files are unchanged', () => { + const dir = sandbox(); + const pendingFile = path.join(dir, 'pending.jsonl'); + // The file must already exist before start() so the tailer's first + // reconcile() baselines its offset at the CURRENT size (startAtEnd:true) -- + // otherwise content written after the baseline would look like history. + fs.writeFileSync(pendingFile, ''); + const clock = makeClock(0); + const timer = fakeTimer(); + const updates = []; + const watcher = new IntelligenceWatch({ + cwd: dir, + watchedFiles: [], // isolate the jsonl-tailer signal from mtime polling + pendingInsightsFile: pendingFile, + debounceMs: 1_000, + setInterval: timer.setInterval, + clearInterval: timer.clearInterval, + now: clock.now, + readGlobalStats: () => null, + readIntelHistory: () => ({ combined: true }), + appendHealthSnapshot: () => {}, + onUpdate: (combined) => updates.push(combined), + }); + + watcher.start(); // baselines at offset 0 -> no change yet + assert.equal(updates.length, 0); + + fs.appendFileSync(pendingFile, `${JSON.stringify({ type: 'edit' })}\n`); + clock.advance(200); + timer.fire(); // tailer reconciles the new line -> change detected + assert.equal(updates.length, 0); + + clock.advance(1_000); + timer.fire(); // quiet window elapsed -> flush + assert.equal(updates.length, 1); + assert.deepEqual(updates[0], { combined: true }); +}); + +test('a malformed pending-insights.jsonl line is reported via onError but does not by itself trigger a flush', () => { + const dir = sandbox(); + const pendingFile = path.join(dir, 'pending.jsonl'); + fs.writeFileSync(pendingFile, ''); // exists before start() so the tailer baselines at offset 0 + const clock = makeClock(0); + const timer = fakeTimer(); + const updates = []; + const errors = []; + const watcher = new IntelligenceWatch({ + cwd: dir, + watchedFiles: [], + pendingInsightsFile: pendingFile, + debounceMs: 500, + setInterval: timer.setInterval, + clearInterval: timer.clearInterval, + now: clock.now, + readGlobalStats: () => null, + readIntelHistory: () => ({ ok: true }), + appendHealthSnapshot: () => {}, + onUpdate: (combined) => updates.push(combined), + onError: (error) => errors.push(error), + }); + + watcher.start(); + // JsonlTailer calls onRecord only for a line that parses; a malformed line + // routes to the tailer's onError instead (see jsonl-tailer.mjs), so this + // watcher's tailerDirty flag -- set only from onRecord -- is intentionally + // NOT raised here: garbage bytes alone must not push a dashboard update. + fs.appendFileSync(pendingFile, 'not-json\n'); + clock.advance(600); + timer.fire(); + assert.equal(errors.length, 1, 'the malformed line must still be reported via onError'); + assert.equal(updates.length, 0, 'a malformed line alone must not trigger a flush'); +}); + +test('appendHealthSnapshot is only called when globalStats genuinely changes between flushes', () => { + const dir = sandbox(); + const file = path.join(dir, 'stats.json'); + fs.writeFileSync(file, '{}'); + const clock = makeClock(0); + const timer = fakeTimer(); + const appended = []; + let stats = { patternsLearned: 1 }; + const watcher = new IntelligenceWatch({ + cwd: dir, + watchedFiles: [file], + pendingInsightsFile: path.join(dir, 'pending.jsonl'), + debounceMs: 500, + setInterval: timer.setInterval, + clearInterval: timer.clearInterval, + now: clock.now, + readGlobalStats: () => stats, + readIntelHistory: () => ({}), + appendHealthSnapshot: (cwd, snapshot) => appended.push(snapshot), + onUpdate: () => {}, + }); + + watcher.start(); // t=0: baseline change + clock.advance(600); + timer.fire(); // t=600: flush #1 -- stats differ from "never seen" -> appended + assert.equal(appended.length, 1); + assert.deepEqual(appended[0], { patternsLearned: 1, ts: 600 }); + + // File rewritten with identical stats content (e.g. a no-op re-save). + fs.utimesSync(file, new Date(1_000), new Date(1_000)); + clock.advance(600); + timer.fire(); // t=1200: change detected, quiet window starts + clock.advance(600); + timer.fire(); // t=1800: flush #2 -- stats unchanged -> must NOT append again + assert.equal(appended.length, 1); + + // Now the stats genuinely change. + stats = { patternsLearned: 2 }; + fs.utimesSync(file, new Date(2_000), new Date(2_000)); + clock.advance(600); + timer.fire(); + clock.advance(600); + timer.fire(); // flush #3 -- stats changed -> appended again + assert.equal(appended.length, 2); + assert.equal(appended[1].patternsLearned, 2); +}); + +test('reader errors are caught, routed to onError, and never reach onUpdate', () => { + const dir = sandbox(); + const pendingFile = path.join(dir, 'pending.jsonl'); + fs.writeFileSync(pendingFile, ''); // exists before start() so the tailer baselines at offset 0 + const clock = makeClock(0); + const timer = fakeTimer(); + const errors = []; + const updates = []; + const boom = new Error('boom'); + const watcher = new IntelligenceWatch({ + cwd: dir, + watchedFiles: [], + pendingInsightsFile: pendingFile, + debounceMs: 500, + setInterval: timer.setInterval, + clearInterval: timer.clearInterval, + now: clock.now, + readGlobalStats: () => { throw boom; }, + readIntelHistory: () => { throw boom; }, + appendHealthSnapshot: () => {}, + onUpdate: (combined) => updates.push(combined), + onError: (error) => errors.push(error), + }); + + watcher.start(); + fs.appendFileSync(pendingFile, `${JSON.stringify({ type: 'edit' })}\n`); + clock.advance(600); + timer.fire(); // t=600: change detected this tick -> quiet window not elapsed yet + clock.advance(600); + timer.fire(); // t=1200: quiet window elapsed since the change -> flush + + assert.equal(updates.length, 0, 'onUpdate must not fire when readIntelHistory threw'); + assert.equal(errors.filter((error) => error === boom).length, 2, + 'both the readGlobalStats and readIntelHistory failures must reach onError'); +}); + +test('a statSync failure other than ENOENT is routed to onError but does not stop polling', () => { + const dir = sandbox(); + const clock = makeClock(0); + const timer = fakeTimer(); + const errors = []; + const updates = []; + const denied = Object.assign(new Error('denied'), { code: 'EACCES' }); + const fsImpl = { statSync: () => { throw denied; } }; + const watcher = new IntelligenceWatch({ + cwd: dir, + watchedFiles: [path.join(dir, 'unreadable.json')], + pendingInsightsFile: path.join(dir, 'pending.jsonl'), + debounceMs: 500, + setInterval: timer.setInterval, + clearInterval: timer.clearInterval, + now: clock.now, + fsImpl, + readGlobalStats: () => null, + readIntelHistory: () => ({ ok: true }), + appendHealthSnapshot: () => {}, + onUpdate: (combined) => updates.push(combined), + onError: (error) => errors.push(error), + }); + + watcher.start(); // first observation of a permanently-failing stat still counts as a baseline change + assert.ok(errors.includes(denied)); + + clock.advance(600); + timer.fire(); // debounce window elapsed since the baseline -> exactly one flush + assert.equal(updates.length, 1); + + clock.advance(600); + timer.fire(); // stat keeps failing the same way -> no further "change" -> no second flush + assert.equal(updates.length, 1); +}); + +test('start() is idempotent and stop() invokes the injected clearInterval exactly once', () => { + const dir = sandbox(); + let intervalCalls = 0; + let clearedWith; + const handle = { unref() {} }; + const watcher = new IntelligenceWatch({ + cwd: dir, + watchedFiles: [], + pendingInsightsFile: path.join(dir, 'pending.jsonl'), + setInterval: () => { intervalCalls += 1; return handle; }, + clearInterval: (h) => { clearedWith = h; }, + now: () => 0, + readGlobalStats: () => null, + readIntelHistory: () => ({}), + appendHealthSnapshot: () => {}, + onUpdate: () => {}, + }); + + watcher.start(); + watcher.start(); // idempotent: must not register a second interval + assert.equal(intervalCalls, 1); + + watcher.stop(); + assert.equal(clearedWith, handle); + watcher.stop(); // safe no-op when already stopped +}); + +test('end-to-end: real intel-history.mjs readers wired against the default project paths', () => { + const dir = sandbox(); + const statsFile = path.join(dir, '.claude-flow', 'neural', 'stats.json'); + fs.mkdirSync(path.dirname(statsFile), { recursive: true }); + fs.writeFileSync(statsFile, JSON.stringify({ + patternsLearned: 3, trajectoriesRecorded: 5, signalsProcessed: 7, lastAdaptation: 123, + })); + const clock = makeClock(0); + const timer = fakeTimer(); + const updates = []; + const watcher = new IntelligenceWatch({ + cwd: dir, + debounceMs: 1_000, + setInterval: timer.setInterval, + clearInterval: timer.clearInterval, + now: clock.now, + onUpdate: (combined) => updates.push(combined), + }); + + watcher.start(); + clock.advance(1_500); + timer.fire(); + + assert.equal(updates.length, 1); + assert.equal(updates[0].globalStats.patternsLearned, 3); + assert.deepEqual(updates[0].patternStore, []); + + const ringFile = path.join(dir, '.claude-flow', 'health-history.json'); + const ring = JSON.parse(fs.readFileSync(ringFile, 'utf8')); + assert.equal(ring.samples.length, 1); + assert.equal(ring.samples[0].patternsLearned, 3); + assert.equal(typeof ring.samples[0].ts, 'number'); +});