Skip to content

feat(logs): add a logs-setup skill that correlates records to people and sessions - #302

Open
ThaliaBarrera wants to merge 4 commits into
PostHog:mainfrom
ThaliaBarrera:thalia/logs-setup-skill
Open

feat(logs): add a logs-setup skill that correlates records to people and sessions#302
ThaliaBarrera wants to merge 4 commits into
PostHog:mainfrom
ThaliaBarrera:thalia/logs-setup-skill

Conversation

@ThaliaBarrera

@ThaliaBarrera ThaliaBarrera commented Jul 30, 2026

Copy link
Copy Markdown

Problem

The logs skill group is docs-only. It declares twelve platform variants and points each at the relevant installation doc, but there's no step chain behind it — it's reference material, and nothing in the wizard drives it. Of the 238 cliEntries in a locally built skill-menu.json, none are logs-related.

Installation is the half those docs already cover well. The half they don't is correlation: attaching posthogDistinctId and sessionId to server-emitted records so a log line resolves to a person and opens their session replay. That's the reason to put logs in PostHog rather than in a standalone log vendor, and today it's left to the reader as something to implement by hand.

It's also the genuinely hard part, and a poor fit for a doc. The answer to "where does this codebase already know who this request belongs to" is different every time — an auth middleware here, a session callback there. A human reading docs has to work that out themselves. An agent reading the codebase can work it out and write the wiring.

Changes

Adds context/skills/logs-setup/: a 7-step chain modelled on migrate, with nextjs and python variants.

detect → install → plan → context → attach → verify → report

Same conventions as migrate throughout: numbered references/ with next_step frontmatter, one step in context at a time, state on disk between steps (.posthog-logs-plan.md), [STATUS] lines, [ABORT] cases, per-step pass/warning/error, and a <wizard-report> block. No commit — the operator reviews the diff.

Four decisions worth a reviewer's attention:

Correlation attaches in exactly one place. A logging.Filter on the OTel handler in Python; a logger wrapper in Next.js. The rule the skill enforces is that a codebase with two hundred log statements should come out of the run with two hundred correlated log statements and roughly one new file. Step 5 explicitly tells the agent to stop and move the shared mechanism if it finds itself editing a third call site by hand.

The client half uses tracing_headers. posthog-js already emits X-POSTHOG-DISTINCT-ID and X-POSTHOG-SESSION-ID from one config option, which is the convention integration-v2/identify and posthog-best-practices already document — the latter says explicitly "do not hand-roll a fetch wrapper for it". The public logs docs demonstrate threading sessionId through a request body instead, which means touching every route and missing anything that doesn't go through the wrapped function. The skill follows the in-repo convention and notes that these headers are client-controlled, so a server-known user id always outranks them.

Correlation degrades in defined tiers. session (both attributes), person (distinct id only), none. The report states which tier was actually reached and what would raise it. A background worker has no session and shouldn't pretend to have one. The failure mode this is written to avoid is wiring that looks correlated but attaches undefined — so Step 5 requires omitting an absent attribute entirely rather than passing an empty value, and requires that attachment never throws.

The step files stay flat. No references/<variant-id>/ subdirectory, because generateSkill() filters to top-level references/*.md when copying into the zip. migrate-statsig.zip currently ships without the references/statsig/ mapping docs its own steps point at. Happy to file that separately if it's news.

The skill also encodes three things the installation docs bury, all of which fail silently:

  • the docs hardcode the US ingestion host, so Step 2 resolves the region from the project's existing PostHog host before writing an endpoint
  • the export takes the phc_ project token, not a phx_ personal API key
  • Next.js middleware.ts runs on the Edge runtime, a different runtime from the one the OTel provider is registered in, so request context established there is invisible to route handlers

CLI surface

Ships as role: skill, reachable as wizard skill logs-setup-<variant>.

The companion wizard PR registers wizard logs as a native flat command, the same split ai-observability uses (cli: role: skill here, nativeCommandFactory there). Per CONTRIBUTING's promotion criteria I'd rather a maintainer decide whether this should also carry role: command; shipping as skill keeps it out of collision with the native registration in the meantime.

Companion PR: PostHog/wizard#1043.

Test plan

  • pnpm test — 137 passed
  • pnpm build — builds logs-setup-nextjs.zip and logs-setup-python.zip; skill-menu.json goes 238 → 240 cliEntries with both new skills under a logs-setup category
  • Verified the built zip is flat and complete: SKILL.md, all 7 steps, COMMANDMENTS.md, and the 8 fetched docs, with all {display_name} / {references} / {commandments} placeholders substituted and the next_step handoff footers wired 1→7
  • pnpm security-scan:skills — zero findings in logs-setup-*. The scan reports 108 findings repo-wide; all are pre-existing in other skills and unchanged by this PR
  • Served locally via pnpm dev and confirmed the wizard fetches both variants from localhost:8765 with --local-mcp

LLM context

Written with Cursor. The design came out of reading migrate as the structural template and integration-v2/identify + posthog-best-practices for the existing tracing-header convention, which is what redirected the client half away from the approach the public logs docs demonstrate.

Made with Cursor

…and sessions

The existing `logs` skill group is docs-only: it points at the installation
docs for twelve platforms but has no step chain behind it. Installation is
the part those docs already cover well. The part they don't is correlation —
attaching `posthogDistinctId` and `sessionId` to server-emitted records so a
log line resolves to a person and opens their session replay.

That's a poor fit for a doc and a good fit for an agent, because the answer
to "where does this codebase already know who the request belongs to" is
different every time: an auth middleware here, a session callback there.

Adds `logs-setup` as a 7-step chain modelled on `migrate`, with Next.js and
Python variants: detect, install, plan, context, attach, verify, report.

Three decisions worth calling out:

- Correlation attaches in exactly one place — a `logging.Filter` on the OTel
  handler in Python, a logger wrapper in Next.js — rather than per call site.
  A codebase with two hundred log statements should come out with two hundred
  correlated ones and roughly one new file.

- The client half uses posthog-js `tracing_headers`, which already emits
  `X-POSTHOG-DISTINCT-ID` and `X-POSTHOG-SESSION-ID`, matching the convention
  in `integration-v2/identify` and `posthog-best-practices`. The public logs
  docs demonstrate threading `sessionId` through a request body instead, which
  means touching every route and missing anything that doesn't go through the
  wrapped function.

- Correlation degrades in defined tiers (`session`, `person`, `none`) and the
  report states which one was actually reached. A background worker has no
  session and shouldn't pretend to. Wiring that looks correlated but attaches
  `undefined` is the outcome this is written to avoid.

The step files stay flat rather than using a `references/<variant>/` subdir,
because the build only copies top-level `references/*.md` into the zip —
`migrate-statsig.zip` ships without the `references/statsig/` docs its own
steps point at.

Ships as `role: skill`. The wizard registers `wizard logs` natively, the same
split `ai-observability` uses.

Co-authored-by: Cursor <cursoragent@cursor.com>
Two things the skill was inferring from the codebase are already known
authoritatively at run time, and the codebase is the less reliable source
for both.

Region: the agent's prompt context carries the authenticated project's real
`PostHog Host`. Sniffing the repo instead gets this wrong in a common setup —
a project routing through `api_host: '/ingest'` with a `ui_host` left at its
US default reads as US even when the account is EU, and the resulting
endpoint accepts nothing useful while looking correct. Prompt context now
ranks first, a reverse proxy counts as no signal, and a disagreement between
the two is reported rather than silently resolved.

Session replay: whether replay is enabled is project-level state, and it
decides whether the `session` tier is reachable at all. Replay is routinely
turned on from the snippet or from a separate frontend repo, so its absence
in this codebase is not evidence it is off.

Co-authored-by: Cursor <cursoragent@cursor.com>

@gewenyu99 gewenyu99 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The skills themselves look great! There are various nits but I'll stay very high level first.

Here's what I challenge you to think about. Logging is often very sensitive, it comes in at many different levels, it's often pushed across different logging providers, and so much of it is often intentionally anonymized to not get slapped with a GDPR lawsuit.

When should you identify logs? Should you decide for the user or suggest to the user? How do you think about compliance?

If there is a good reason not to log and associate with a person, what other reasons make it valuable to capture logs, especially to PostHog? Think about the context and tasks that push you to reach for logs.

I also suggest you think about logging for monorepos. This is something we haven't solved for the main program (though we're in progress there), but for logs this is especially important. Most logging happens at the service level in the backend for good reason. How do you think about this problem?

Some helpful reading material:
https://posthog.com/docs/logs/pii-scrubbing#quest-item-use-for-free


Search for `posthog-js`, `posthog-node`, `posthog` (Python), and any existing `posthog.init` or `Posthog(...)` construction. Note the host and token configuration, and where they come from.

An existing PostHog host is the most reliable signal of the project's region, and Step 2 needs the region. Record the exact host value you found, and whether it came from a literal or an environment variable.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You will have both region and project details from the Wizard to pass into the agent. I would rely on that and wire that information through.


Capture stdout and stderr. Truncate to the failure region if the output is long.

## Fix only what this run caused

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good scoping

Ran the skill end to end against the Next.js SaaS starter and watched where
it lost time or got things subtly wrong.

The agent spent roughly 19 of its 32 shell commands spelunking through
node_modules trying to work out why `tracing_headers` type checks as an
unknown property. It is real at runtime but absent from PostHogConfig's
published types, so step 4 now says that outright and prescribes the cast,
with a matching carve-out in step 6's no-silencing-types rule.

It also emitted from `middleware.ts`, which is Edge, so the OTel SDK lands
in the Edge bundle and the batch processor never flushes before the
invocation ends. That compiles and reads fine in review, which is what
makes it worth an explicit warning in step 5.

Finally it retried a failing build three times with different flags against
a sandbox restriction no flag could fix. Step 6 now asks it to tell an
environmental failure from a code one and report the former as a follow up.

Co-authored-by: Cursor <cursoragent@cursor.com>

@sarahxsanders sarahxsanders left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like this is the stronger of the pair of PRs! the correlation tiers are well thought through. a few things:

  1. step 6 can't check what it claims which could report a false failure
  2. the command here plus wizard logs in the other PR means there's two ways to run the same thing. we have docs on this if you go take a look! they should be easy to search for
  3. feel free to open issues for bugs you find!

description: Set up PostHog Logs in an existing codebase and correlate each log record to the session replay and person that produced it. Additive — existing logging keeps working unchanged.
tags: [logs]
cli:
role: skill

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this means the wizard program runs with the command wizard skill logs-setup-nextjs when your intention with the wizard PR looks like to have it run with wizard logs

we have some docs on posthog.com about wizard commands that may help here!

Comment on lines +29 to +35
## Correlation tiers

Correlation is not all or nothing. Every project lands on one of these tiers, and the honest outcome is to reach the best tier this codebase supports and then say so.

- **`session`** — the record carries both `posthogDistinctId` and `sessionId`. A log line links to the person and opens the session replay. This is the goal.
- **`person`** — the record carries `posthogDistinctId` only. Logs appear on the person's profile, but there is no replay to open. This is the correct outcome when the project has no browser client, or when session replay is off.
- **`none`** — no identity is reachable at log time. The logs still arrive and are still searchable, they just aren't linked to anyone.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this rocks! you're looking at it from what partial success looks like and thinking about those outcomes

…operator

Benchmarking wizard logs showed 6-verify was the heaviest, most variable
step: the agent tried to confirm the test record arrived and, lacking a
reliable logs-query tool, improvised curl calls to the PostHog API and
API-key hunting (up to 84 turns / 8 min in one run). Verify now emits a
single record in-process and defers delivery confirmation to the operator
(no in-session PostHog queries), and treats tsc as the not-broken gate with
the sandbox-blocked build recorded as an operator follow-up. Same app:
2m47s/18 turns -> 1m45s/15 turns, worst case eliminated.

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants