Skip to content

fix(telemetry): send the usage event directly instead of via posthog-node - #1476

Merged
clay-good merged 3 commits into
mainfrom
fix/telemetry-without-posthog-node
Jul 29, 2026
Merged

fix(telemetry): send the usage event directly instead of via posthog-node#1476
clay-good merged 3 commits into
mainfrom
fix/telemetry-without-posthog-node

Conversation

@clay-good

@clay-good clay-good commented Jul 29, 2026

Copy link
Copy Markdown
Collaborator

Status: LGTM — resolves alfred's CHANGES_REQUESTED on the v1.7.0 Version Packages PR (#1380); this is the release-gating fix for #1390.

What was wrong: Installing OpenSpec shipped posthog-node's transitive tree (@posthog/core, @posthog/types) to every downstream consumer. Those packages publish several times a day, so freshly resolved installs tripped supply-chain age policies — pnpm's minimumReleaseAge: 1440 failed with ERR_PNPM_MINIMUM_RELEASE_AGE_VIOLATION on entries younger than the window (#1390, reproduced by alfred against the 1.7.0 candidate).

Why no form of pinning works (each verified, not assumed):

  • Exact-pinning posthog-node leaves its own @posthog/core range floating (alfred's point).
  • npm overrides apply only at the consumer's root, never from a dependency.
  • pnpm ignores a dependency's npm-shrinkwrap.json — and the failing consumers are pnpm users.
  • bundledDependencies under pnpm-managed node_modules packs the .pnpm virtual-store layout: the bundled posthog-node resolves but @posthog/core does not, and the installed CLI crashes on import. Empirically tested; transcript available.

How it was fixed: The SDK's only remaining job was the wire format — the client was already configured to send exactly one event, immediately, time-bounded, no retries, through an injected fetch that never throws. The telemetry module now posts the same capture payload to the same /batch/ endpoint with that fetch directly. Same event name, properties, distinct_id, opt-out guards (OPENSPEC_TELEMETRY=0, DO_NOT_TRACK, CI), and first-run notice; shutdown() still flushes in-flight events, each bounded by the 1s request timeout. posthog-node is removed from dependencies.

Replication / proof:

  • Packed tarball: zero posthog files.
  • The exact PNPM: Failed pipelines with the "ERR_PNPM_MINIMUM_RELEASE_AGE_VIOLATION" error #1390 scenario: a pnpm consumer with minimumReleaseAge: 1440 installs the tarball cleanly with zero posthog lockfile entries and a working CLI — the failure is now structurally impossible, not just currently avoided.
  • Live wire check: POST https://edge.openspec.dev/batch/ → 200 OK with the new payload.
  • Regression coverage (the requested packed-consumer guarantee): tests pin that the published manifest (dependencies/optionalDependencies/peerDependencies) contains no posthog package and that no src/ module imports one — the invariant that makes any age-policy violation from our tree impossible. A full registry-resolution-under-policy test would be network- and clock-dependent in CI; the invariant test enforces the same guarantee deterministically.
  • Full suite: 115 files / 3,384 tests green. Telemetry tests now assert against the real network boundary (fetch spy) instead of an SDK mock: payload shape, timeout signal, flush-before-exit, and silent failure on error/abort/non-2xx.

Notes: flake.nix's pnpmDeps hash will need its usual bump for the lockfile change — will follow up with the CI-reported hash on this branch. One net effect users may notice: npm install -g @fission-ai/openspec gets ~160 packages lighter.

Fixes #1390

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Bug Fixes
    • Updated telemetry to send events directly via HTTP instead of using the PostHog SDK.
    • Telemetry now flushes any in-flight events safely during shutdown.
    • Network, timeout, and non-success responses are handled silently to avoid impacting command execution.
  • Tests
    • Refreshed telemetry tests to mock HTTP calls (no SDK mocking) and verify shutdown/flush behavior.
    • Added checks to prevent posthog-named dependencies from being included.
  • Documentation
    • Revised release notes to reflect the streamlined telemetry flow and reduced install-time dependency impact.

…node

Installing OpenSpec shipped posthog-node's transitive tree
(@posthog/core, @posthog/types) to every consumer. Those packages
release several times a day, so any freshly resolved install tripped
supply-chain age policies — pnpm's minimumReleaseAge failed with
ERR_PNPM_MINIMUM_RELEASE_AGE_VIOLATION on entries younger than the
policy window (#1390). No pinning fixes this: exact-pinning posthog-node
leaves its own ranges floating, npm overrides only apply at a consumer's
root, pnpm ignores a dependency's npm-shrinkwrap, and bundledDependencies
under a pnpm-managed node_modules packs the virtual-store layout and
breaks module resolution (verified: the bundled CLI crashes on import).

The SDK's only remaining job here was the wire format: the client was
already configured to send one event immediately, time-bounded, with no
retries, through an injected fetch that never throws. Post the same
capture payload to the same /batch/ endpoint with that fetch directly.
Same event name, properties, distinct id, and opt-out guards; shutdown
still flushes in-flight events, each bounded by the request timeout.

Verified end to end: the packed tarball contains zero posthog files, a
pnpm consumer with minimumReleaseAge: 1440 installs cleanly with zero
posthog lockfile entries, and the live endpoint answers 200 OK to the
new payload. Regression tests pin the manifest and src free of posthog.

Fixes #1390

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@clay-good
clay-good requested a review from a team as a code owner July 29, 2026 00:40
@clay-good
clay-good requested review from alfred-openspec and removed request for a team July 29, 2026 00:40
@coderabbitai

coderabbitai Bot commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 4d114c1a-c407-461c-8a42-5927ec7dfde9

📥 Commits

Reviewing files that changed from the base of the PR and between 4a1c6fd and c1b83b3.

📒 Files selected for processing (2)
  • src/telemetry/index.ts
  • test/telemetry/index.test.ts
🚧 Files skipped from review as they are similar to previous changes (2)
  • test/telemetry/index.test.ts
  • src/telemetry/index.ts

📝 Walkthrough

Walkthrough

Telemetry no longer uses posthog-node. It sends usage events through PostHog’s batch endpoint with fetch, tracks pending requests for shutdown flushing, removes the dependency, updates tests and the Nix hash, and documents the release change.

Changes

Telemetry transport replacement

Layer / File(s) Summary
Direct batch telemetry transport
src/telemetry/index.ts
Telemetry sends batch requests through fetch, tracks in-flight promises, dispatches command metadata, and waits for pending requests during shutdown.
Fetch transport and lifecycle validation
test/telemetry/index.test.ts
Tests intercept fetch calls and cover opt-out behavior, payloads, timeouts, failures, response disposal, shutdown flushing, and PostHog import/dependency absence.
Published dependency and release metadata
package.json, flake.nix, .changeset/telemetry-without-posthog-node.md
The published dependency is removed, the Nix dependency hash is updated, and the changeset records the fetch-based telemetry implementation and dependency-tree changes.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant trackCommand
  participant sendEvent
  participant safeTelemetryFetch
  participant pendingEvents
  participant shutdown
  trackCommand->>sendEvent: command, version, surface, $ip
  sendEvent->>safeTelemetryFetch: POST batch payload with timeout
  sendEvent->>pendingEvents: register request promise
  shutdown->>pendingEvents: await all settled promises
  pendingEvents-->>shutdown: flush completed
Loading
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title is concise and accurately describes the main telemetry change.
Linked Issues check ✅ Passed The PR removes posthog-node and sends telemetry directly, which addresses the dependency-tree violation in #1390.
Out of Scope Changes check ✅ Passed The other changes are supporting updates for the telemetry and dependency fix, with no clear unrelated scope creep.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/telemetry-without-posthog-node

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
test/telemetry/index.test.ts (1)

225-241: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Regex misses non-from import forms.

require('posthog-node'), await import('posthog-node'), and bare import 'posthog-node' would all slip past this guard, weakening the regression test's purpose.

♻️ Broaden the match
-            if (/from\s+['"](posthog|`@posthog`)/.test(content)) {
+            if (/(?:from|import|require)\s*\(?\s*['"]@?posthog/.test(content)) {
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/telemetry/index.test.ts` around lines 225 - 241, Broaden the
module-detection regex in the “imports no posthog module anywhere in src” test
to catch all import forms, including require(), dynamic import(), and
side-effect imports, while retaining matches for posthog and `@posthog` package
names. Keep the existing recursive source scan and empty hits assertion
unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/telemetry/index.ts`:
- Around line 122-130: Update the request promise in the telemetry batching flow
around safeTelemetryFetch so the returned Response body is deterministically
consumed or cancelled before the promise resolves. Preserve the existing
pendingEvents tracking and cleanup, while ensuring successful requests release
their underlying response resources before completion.

---

Nitpick comments:
In `@test/telemetry/index.test.ts`:
- Around line 225-241: Broaden the module-detection regex in the “imports no
posthog module anywhere in src” test to catch all import forms, including
require(), dynamic import(), and side-effect imports, while retaining matches
for posthog and `@posthog` package names. Keep the existing recursive source scan
and empty hits assertion unchanged.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 5058fe7f-da63-4bcf-8c9f-719837a78909

📥 Commits

Reviewing files that changed from the base of the PR and between 17af60c and 07cda25.

⛔ Files ignored due to path filters (1)
  • pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
📒 Files selected for processing (4)
  • .changeset/telemetry-without-posthog-node.md
  • package.json
  • src/telemetry/index.ts
  • test/telemetry/index.test.ts
💤 Files with no reviewable changes (1)
  • package.json

Comment thread src/telemetry/index.ts
Value taken from the CI mismatch report.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

@alfred-openspec alfred-openspec left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The dependency-policy fix is sound, but the direct transport drops the SDK's explicit response-body disposal: both successful and non-2xx responses leave Undici bodies unread, so connections can remain occupied after shutdown() returns. Please consume or cancel each response body before the tracked promise resolves, with coverage for both paths.

undici keeps the connection occupied until the response body is consumed
or canceled, and telemetry never reads it — on both the success and
non-2xx paths the socket could linger after shutdown() returned. Cancel
the body before the tracked promise resolves, with coverage for both
paths (bodyUsed asserted after shutdown), and the live endpoint
re-verified with disposal in place.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

@alfred-openspec alfred-openspec left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The response-disposal regression is fixed: every body is canceled before the pending event settles, with coverage for both successful and non-2xx responses. Build, lint, focused telemetry tests, and the hosted CI/security/Nix matrix all pass.

@clay-good
clay-good added this pull request to the merge queue Jul 29, 2026
Merged via the queue into main with commit 8731290 Jul 29, 2026
16 checks passed
@clay-good
clay-good deleted the fix/telemetry-without-posthog-node branch July 29, 2026 01:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

PNPM: Failed pipelines with the "ERR_PNPM_MINIMUM_RELEASE_AGE_VIOLATION" error

2 participants