feat(logs): add a logs-setup skill that correlates records to people and sessions - #302
feat(logs): add a logs-setup skill that correlates records to people and sessions#302ThaliaBarrera wants to merge 4 commits into
Conversation
…and sessions The existing `logs` skill group is docs-only: it points at the installation docs for twelve platforms but has no step chain behind it. Installation is the part those docs already cover well. The part they don't is correlation — attaching `posthogDistinctId` and `sessionId` to server-emitted records so a log line resolves to a person and opens their session replay. That's a poor fit for a doc and a good fit for an agent, because the answer to "where does this codebase already know who the request belongs to" is different every time: an auth middleware here, a session callback there. Adds `logs-setup` as a 7-step chain modelled on `migrate`, with Next.js and Python variants: detect, install, plan, context, attach, verify, report. Three decisions worth calling out: - Correlation attaches in exactly one place — a `logging.Filter` on the OTel handler in Python, a logger wrapper in Next.js — rather than per call site. A codebase with two hundred log statements should come out with two hundred correlated ones and roughly one new file. - The client half uses posthog-js `tracing_headers`, which already emits `X-POSTHOG-DISTINCT-ID` and `X-POSTHOG-SESSION-ID`, matching the convention in `integration-v2/identify` and `posthog-best-practices`. The public logs docs demonstrate threading `sessionId` through a request body instead, which means touching every route and missing anything that doesn't go through the wrapped function. - Correlation degrades in defined tiers (`session`, `person`, `none`) and the report states which one was actually reached. A background worker has no session and shouldn't pretend to. Wiring that looks correlated but attaches `undefined` is the outcome this is written to avoid. The step files stay flat rather than using a `references/<variant>/` subdir, because the build only copies top-level `references/*.md` into the zip — `migrate-statsig.zip` ships without the `references/statsig/` docs its own steps point at. Ships as `role: skill`. The wizard registers `wizard logs` natively, the same split `ai-observability` uses. Co-authored-by: Cursor <cursoragent@cursor.com>
Two things the skill was inferring from the codebase are already known authoritatively at run time, and the codebase is the less reliable source for both. Region: the agent's prompt context carries the authenticated project's real `PostHog Host`. Sniffing the repo instead gets this wrong in a common setup — a project routing through `api_host: '/ingest'` with a `ui_host` left at its US default reads as US even when the account is EU, and the resulting endpoint accepts nothing useful while looking correct. Prompt context now ranks first, a reverse proxy counts as no signal, and a disagreement between the two is reported rather than silently resolved. Session replay: whether replay is enabled is project-level state, and it decides whether the `session` tier is reachable at all. Replay is routinely turned on from the snippet or from a separate frontend repo, so its absence in this codebase is not evidence it is off. Co-authored-by: Cursor <cursoragent@cursor.com>
gewenyu99
left a comment
There was a problem hiding this comment.
The skills themselves look great! There are various nits but I'll stay very high level first.
Here's what I challenge you to think about. Logging is often very sensitive, it comes in at many different levels, it's often pushed across different logging providers, and so much of it is often intentionally anonymized to not get slapped with a GDPR lawsuit.
When should you identify logs? Should you decide for the user or suggest to the user? How do you think about compliance?
If there is a good reason not to log and associate with a person, what other reasons make it valuable to capture logs, especially to PostHog? Think about the context and tasks that push you to reach for logs.
I also suggest you think about logging for monorepos. This is something we haven't solved for the main program (though we're in progress there), but for logs this is especially important. Most logging happens at the service level in the backend for good reason. How do you think about this problem?
Some helpful reading material:
https://posthog.com/docs/logs/pii-scrubbing#quest-item-use-for-free
|
|
||
| Search for `posthog-js`, `posthog-node`, `posthog` (Python), and any existing `posthog.init` or `Posthog(...)` construction. Note the host and token configuration, and where they come from. | ||
|
|
||
| An existing PostHog host is the most reliable signal of the project's region, and Step 2 needs the region. Record the exact host value you found, and whether it came from a literal or an environment variable. |
There was a problem hiding this comment.
You will have both region and project details from the Wizard to pass into the agent. I would rely on that and wire that information through.
|
|
||
| Capture stdout and stderr. Truncate to the failure region if the output is long. | ||
|
|
||
| ## Fix only what this run caused |
Ran the skill end to end against the Next.js SaaS starter and watched where it lost time or got things subtly wrong. The agent spent roughly 19 of its 32 shell commands spelunking through node_modules trying to work out why `tracing_headers` type checks as an unknown property. It is real at runtime but absent from PostHogConfig's published types, so step 4 now says that outright and prescribes the cast, with a matching carve-out in step 6's no-silencing-types rule. It also emitted from `middleware.ts`, which is Edge, so the OTel SDK lands in the Edge bundle and the batch processor never flushes before the invocation ends. That compiles and reads fine in review, which is what makes it worth an explicit warning in step 5. Finally it retried a failing build three times with different flags against a sandbox restriction no flag could fix. Step 6 now asks it to tell an environmental failure from a code one and report the former as a follow up. Co-authored-by: Cursor <cursoragent@cursor.com>
sarahxsanders
left a comment
There was a problem hiding this comment.
I feel like this is the stronger of the pair of PRs! the correlation tiers are well thought through. a few things:
- step 6 can't check what it claims which could report a false failure
- the command here plus
wizard logsin the other PR means there's two ways to run the same thing. we have docs on this if you go take a look! they should be easy to search for - feel free to open issues for bugs you find!
| description: Set up PostHog Logs in an existing codebase and correlate each log record to the session replay and person that produced it. Additive — existing logging keeps working unchanged. | ||
| tags: [logs] | ||
| cli: | ||
| role: skill |
There was a problem hiding this comment.
this means the wizard program runs with the command wizard skill logs-setup-nextjs when your intention with the wizard PR looks like to have it run with wizard logs
we have some docs on posthog.com about wizard commands that may help here!
| ## Correlation tiers | ||
|
|
||
| Correlation is not all or nothing. Every project lands on one of these tiers, and the honest outcome is to reach the best tier this codebase supports and then say so. | ||
|
|
||
| - **`session`** — the record carries both `posthogDistinctId` and `sessionId`. A log line links to the person and opens the session replay. This is the goal. | ||
| - **`person`** — the record carries `posthogDistinctId` only. Logs appear on the person's profile, but there is no replay to open. This is the correct outcome when the project has no browser client, or when session replay is off. | ||
| - **`none`** — no identity is reachable at log time. The logs still arrive and are still searchable, they just aren't linked to anyone. |
There was a problem hiding this comment.
this rocks! you're looking at it from what partial success looks like and thinking about those outcomes
…operator Benchmarking wizard logs showed 6-verify was the heaviest, most variable step: the agent tried to confirm the test record arrived and, lacking a reliable logs-query tool, improvised curl calls to the PostHog API and API-key hunting (up to 84 turns / 8 min in one run). Verify now emits a single record in-process and defers delivery confirmation to the operator (no in-session PostHog queries), and treats tsc as the not-broken gate with the sandbox-blocked build recorded as an operator follow-up. Same app: 2m47s/18 turns -> 1m45s/15 turns, worst case eliminated. Co-authored-by: Cursor <cursoragent@cursor.com>
Problem
The
logsskill group is docs-only. It declares twelve platform variants and points each at the relevant installation doc, but there's no step chain behind it — it's reference material, and nothing in the wizard drives it. Of the 238cliEntriesin a locally builtskill-menu.json, none are logs-related.Installation is the half those docs already cover well. The half they don't is correlation: attaching
posthogDistinctIdandsessionIdto server-emitted records so a log line resolves to a person and opens their session replay. That's the reason to put logs in PostHog rather than in a standalone log vendor, and today it's left to the reader as something to implement by hand.It's also the genuinely hard part, and a poor fit for a doc. The answer to "where does this codebase already know who this request belongs to" is different every time — an auth middleware here, a session callback there. A human reading docs has to work that out themselves. An agent reading the codebase can work it out and write the wiring.
Changes
Adds
context/skills/logs-setup/: a 7-step chain modelled onmigrate, withnextjsandpythonvariants.detect → install → plan → context → attach → verify → reportSame conventions as
migratethroughout: numberedreferences/withnext_stepfrontmatter, one step in context at a time, state on disk between steps (.posthog-logs-plan.md),[STATUS]lines,[ABORT]cases, per-steppass/warning/error, and a<wizard-report>block. No commit — the operator reviews the diff.Four decisions worth a reviewer's attention:
Correlation attaches in exactly one place. A
logging.Filteron the OTel handler in Python; a logger wrapper in Next.js. The rule the skill enforces is that a codebase with two hundred log statements should come out of the run with two hundred correlated log statements and roughly one new file. Step 5 explicitly tells the agent to stop and move the shared mechanism if it finds itself editing a third call site by hand.The client half uses
tracing_headers.posthog-jsalready emitsX-POSTHOG-DISTINCT-IDandX-POSTHOG-SESSION-IDfrom one config option, which is the conventionintegration-v2/identifyandposthog-best-practicesalready document — the latter says explicitly "do not hand-roll a fetch wrapper for it". The public logs docs demonstrate threadingsessionIdthrough a request body instead, which means touching every route and missing anything that doesn't go through the wrapped function. The skill follows the in-repo convention and notes that these headers are client-controlled, so a server-known user id always outranks them.Correlation degrades in defined tiers.
session(both attributes),person(distinct id only),none. The report states which tier was actually reached and what would raise it. A background worker has no session and shouldn't pretend to have one. The failure mode this is written to avoid is wiring that looks correlated but attachesundefined— so Step 5 requires omitting an absent attribute entirely rather than passing an empty value, and requires that attachment never throws.The step files stay flat. No
references/<variant-id>/subdirectory, becausegenerateSkill()filters to top-levelreferences/*.mdwhen copying into the zip.migrate-statsig.zipcurrently ships without thereferences/statsig/mapping docs its own steps point at. Happy to file that separately if it's news.The skill also encodes three things the installation docs bury, all of which fail silently:
phc_project token, not aphx_personal API keymiddleware.tsruns on the Edge runtime, a different runtime from the one the OTel provider is registered in, so request context established there is invisible to route handlersCLI surface
Ships as
role: skill, reachable aswizard skill logs-setup-<variant>.The companion wizard PR registers
wizard logsas a native flat command, the same splitai-observabilityuses (cli: role: skillhere,nativeCommandFactorythere). Per CONTRIBUTING's promotion criteria I'd rather a maintainer decide whether this should also carryrole: command; shipping asskillkeeps it out of collision with the native registration in the meantime.Companion PR: PostHog/wizard#1043.
Test plan
pnpm test— 137 passedpnpm build— buildslogs-setup-nextjs.zipandlogs-setup-python.zip;skill-menu.jsongoes 238 → 240cliEntrieswith both new skills under alogs-setupcategorySKILL.md, all 7 steps,COMMANDMENTS.md, and the 8 fetched docs, with all{display_name}/{references}/{commandments}placeholders substituted and thenext_stephandoff footers wired 1→7pnpm security-scan:skills— zero findings inlogs-setup-*. The scan reports 108 findings repo-wide; all are pre-existing in other skills and unchanged by this PRpnpm devand confirmed the wizard fetches both variants fromlocalhost:8765with--local-mcpLLM context
Written with Cursor. The design came out of reading
migrateas the structural template andintegration-v2/identify+posthog-best-practicesfor the existing tracing-header convention, which is what redirected the client half away from the approach the public logs docs demonstrate.Made with Cursor