Add privacy-conscious playground analytics

Author: a-baran-orhan

docs/product-analytics.md

@@ -1,8 +1,9 @@
 # Product usage analytics
 
-Posecode keeps Vercel pageviews and product-usage events separate. Pageviews
-answer “which routes were visited?”; the events below answer “did someone use
-the product?”
+Posecode keeps Vercel pageviews and product-usage events separate. One initial
+pageview per loaded HTML entry point answers “which route was visited?”; the
+deduplicated events below answer whether someone completed a meaningful product
+step.
 
 ## Provider and production configuration
 
@@ -12,6 +13,13 @@ types, failure isolation, and session deduplication live in
 `playground/src/vercel-analytics.ts`.
 
 Vercel Web Analytics pageviews remain enabled without extra configuration.
+The adapter disables soft-navigation tracking because the playground uses
+`history.replaceState()` while editing; those source-address updates are not
+new visits. Its `beforeSend` hook strips query strings and hashes, collapses
+`/play/:movement` to `/play/[movement]`, and normalizes `.html` aliases before
+the event leaves the browser. This prevents encoded movement source in a share
+hash from becoming analytics URL data.
+
 Vercel's current plan table says custom events are **not available on Hobby**;
 they are available on Pro and Enterprise. Pro allows at most two properties per
 custom event. The schema below deliberately stays within that limit.
@@ -40,7 +48,9 @@ Sources:
 | `preset_opened` | A bundled movement is actually opened at initial load or selected in the library. | `source`: `library`, `direct_url`, `shared_link`, or `landing_cta`; `preset_id`: bundled stable ID |
 | `editor_changed` | The first real CodeMirror user edit in the page session. Programmatic preset loads do not count. | `document_kind`: `preset`, `shared`, or `custom` |
 | `render_succeeded` | `viewer.load()` successfully accepts a new meaningful document revision. Lazy boot and repeated recompiles of the same revision are deduplicated. The animation frame loop never emits this event. | `trigger`: `initial`, `preset_open`, `shared_link`, or `editor_change`; `document_kind` |
-| `share_created` | The generated preset/encoded URL has successfully been written to the clipboard. | `share_kind`: `preset` or `encoded` |
+| `prompt_copied` | The authoring guide is successfully copied on the landing page or playground. Repeated copies in one page session are deduplicated. | `location`: `landing` or `playground` |
+| `movement_attempted` | The first user-initiated editor revision in the page session parses without errors, loads in the viewer, and does not exactly equal a bundled preset. Initial preset/shared loads and invalid edits do not count. | none |
+| `share_created` | The generated preset/encoded URL has successfully been written to the clipboard. Repeated successful copies in one page session are deduplicated. | `share_kind`: `preset` or `encoded` |
 | `embed_docs_clicked` | The embed documentation CTA on `/for-products` is clicked. | `location`: `for_products` |
 | `install_command_copied` | An npm/npx command on `/for-products` is successfully written to the clipboard. | `command`: `embed`, `packages`, or `mcp`; `location`: `for_products` |
 
@@ -51,20 +61,29 @@ Useful readings include:
 
 - `preset_opened` grouped by `source` separates library discovery from direct,
   shared, and landing-page entry.
-- Compare `editor_changed` and `render_succeeded` counts to see whether editing
-  reaches a valid renderer update. They are intentionally not a strict funnel:
-  initial and preset renders also count.
-- `share_created` is a confirmed clipboard outcome, not a button-click count.
+- Compare route visitors with `prompt_copied`, `movement_attempted`, and
+  `share_created` for the focused authoring funnel. These are aggregate counts,
+  not joined user records.
+- `movement_attempted` excludes invalid edits and unchanged presets.
+- `prompt_copied` and `share_created` are confirmed clipboard outcomes, not
+  button-click counts.
 - Group `install_command_copied` by `command` to compare integration intent.
 
 Vercel reports aggregate events rather than a user-level funnel. Do not attempt
 to join individual visitors or reconstruct sessions from these payloads.
 
 ## Privacy and resilience
 
-Events never contain Posecode source text, authoring prompts, personal data,
-full share tokens, query strings, referrers, or sensitive URLs. `preset_id` is a
-bounded public catalogue identifier; all other values are closed enums.
+Custom event properties never contain Posecode source text, authoring prompts,
+personal data, full share tokens, query strings, referrers, or sensitive URLs.
+`preset_id` is a bounded public catalogue identifier; all other values are
+closed enums.
+
+Funnel deduplication is deliberately page-session-only and uses in-memory sets,
+not cookies, local storage, user IDs, or source hashes. Reloading the page starts
+a new page session. Vercel pageviews can still include Vercel's standard
+anonymous dimensions and an incoming referrer under its Web Analytics privacy
+model; the application does not add identifying fields.
 
 Every analytics call is best-effort and catches provider failures. Ad blockers,
 network failures, a missing provider configuration, or plan limitations do not

playground/src/analytics.ts

@@ -9,6 +9,8 @@ export const USAGE_EVENT_NAMES = {
   presetOpened: "preset_opened",
   editorChanged: "editor_changed",
   renderSucceeded: "render_succeeded",
+  promptCopied: "prompt_copied",
+  movementAttempted: "movement_attempted",
   shareCreated: "share_created",
   embedDocsClicked: "embed_docs_clicked",
   installCommandCopied: "install_command_copied",
@@ -26,12 +28,15 @@ export type RenderTrigger =
   | "shared_link"
   | "editor_change";
 export type ShareKind = "preset" | "encoded";
+export type PromptLocation = "landing" | "playground";
 export type InstallCommand = "embed" | "packages" | "mcp";
 
 export interface UsageEventMap {
   preset_opened: { source: PresetOpenSource; preset_id: string };
   editor_changed: { document_kind: DocumentKind };
   render_succeeded: { trigger: RenderTrigger; document_kind: DocumentKind };
+  prompt_copied: { location: PromptLocation };
+  movement_attempted: Record<string, never>;
   share_created: { share_kind: ShareKind };
   embed_docs_clicked: { location: "for_products" };
   install_command_copied: {
@@ -70,6 +75,35 @@ export function trackUsageEvent<Name extends UsageEventName>(
 export class UsageSession {
   private firstEditTracked = false;
   private renderedRevisions = new Set<number>();
+  private funnelEvents = new Set<
+    "prompt_copied" | "movement_attempted" | "share_created"
+  >();
+
+  private trackFunnelEventOnce<Name extends
+    | "prompt_copied"
+    | "movement_attempted"
+    | "share_created">(
+    name: Name,
+    properties: UsageEventMap[Name],
+  ): void {
+    if (this.funnelEvents.has(name)) return;
+    this.funnelEvents.add(name);
+    trackUsageEvent(name, properties);
+  }
+
+  trackPromptCopied(location: PromptLocation): void {
+    this.trackFunnelEventOnce(USAGE_EVENT_NAMES.promptCopied, { location });
+  }
+
+  trackFirstValidCustomMovement(): void {
+    this.trackFunnelEventOnce(USAGE_EVENT_NAMES.movementAttempted, {});
+  }
+
+  trackSuccessfulShare(shareKind: ShareKind): void {
+    this.trackFunnelEventOnce(USAGE_EVENT_NAMES.shareCreated, {
+      share_kind: shareKind,
+    });
+  }
 
   trackFirstEdit(documentKind: DocumentKind): void {
     if (this.firstEditTracked) return;

playground/src/landing.ts

@@ -6,15 +6,19 @@
  */
 
 import { parse } from "posecode-parser";
-import { inject } from "@vercel/analytics";
+import { UsageSession } from "./analytics.js";
+import { initializeAnalytics } from "./vercel-analytics.js";
 import { PRESETS } from "./presets.js";
 import llmPrompt from "../../spec/llm-authoring.md?raw";
 
-inject();
+const usageSession = new UsageSession();
 
 // Preserve permalinks shared before the tool moved from `/` to `/play`.
 if (location.hash.startsWith("#doc=")) {
   location.replace(`/play${location.hash}`);
+} else {
+  // The redirect target records the visit; do not double-count the legacy URL.
+  initializeAnalytics();
 }
 
 const prefersReducedMotion = matchMedia("(prefers-reduced-motion: reduce)").matches;
@@ -155,6 +159,7 @@ for (const copyBtn of document.querySelectorAll<HTMLButtonElement>("[data-copy-p
     lbl.textContent = "Copying…";
     try {
       await writeClipboard(llmPrompt);
+      usageSession.trackPromptCopied("landing");
       lbl.textContent = "Copied ✓";
     } catch {
       lbl.textContent = "Copy failed";

playground/src/main.ts

@@ -352,6 +352,13 @@ function recompile(): void {
   if (ir && viewer) {
     viewer.load(ir);
     updateFloorGuideKey();
+    if (
+      errors.length === 0 &&
+      pendingRenderTrigger === "editor_change" &&
+      documentKind() === "custom"
+    ) {
+      usageSession.trackFirstValidCustomMovement();
+    }
     usageSession.trackSuccessfulRender(
       documentRevision,
       pendingRenderTrigger,
@@ -702,6 +709,7 @@ async function copyPrompt(btn: HTMLButtonElement): Promise<void> {
   flash(btn, "Copying…", "pending", 0);
   try {
     await navigator.clipboard.writeText(llmPrompt);
+    usageSession.trackPromptCopied("playground");
     flash(btn, "Copied ✓", "success");
   } catch {
     flash(btn, "Copy failed", "error");
@@ -722,9 +730,9 @@ async function shareLink(): Promise<void> {
     const url = `${location.origin}${path}${hash}`;
     history.replaceState(null, "", `${path}${hash}`);
     await navigator.clipboard.writeText(url);
-    trackUsageEvent(USAGE_EVENT_NAMES.shareCreated, {
-      share_kind: path === "/play" ? "encoded" : "preset",
-    });
+    usageSession.trackSuccessfulShare(
+      path === "/play" ? "encoded" : "preset",
+    );
     flash(shareBtn, "Link copied ✓", "success");
   } catch (err) {
     const message =

playground/src/vercel-analytics.ts

@@ -1,15 +1,57 @@
-import { inject, track } from "@vercel/analytics";
+import {
+  inject,
+  track,
+  type BeforeSendEvent,
+} from "@vercel/analytics";
 import {
   configureUsageAnalytics,
   type UsageEventSink,
 } from "./analytics.js";
 
+/** Collapse public aliases and dynamic movement paths into bounded route names. */
+export function analyticsRoute(pathname: string): string {
+  if (/^\/(?:index\.html)?\/?$/.test(pathname)) return "/";
+  if (/^\/play(?:\.html)?\/?$/.test(pathname)) return "/play";
+  if (pathname.startsWith("/play/")) return "/play/[movement]";
+  if (/^\/for-products(?:\.html)?\/?$/.test(pathname)) {
+    return "/for-products";
+  }
+  return pathname;
+}
+
+/**
+ * Page URLs can contain an encoded Posecode document in the hash. Redact all
+ * query/hash data and normalize dynamic movement paths before Vercel sees it.
+ */
+export function redactAnalyticsUrl(
+  event: BeforeSendEvent,
+): BeforeSendEvent | null {
+  try {
+    const absolute = /^[a-z][a-z\d+.-]*:\/\//i.test(event.url);
+    const url = new URL(event.url, "https://analytics.posecode.invalid");
+    url.pathname = analyticsRoute(url.pathname);
+    url.search = "";
+    url.hash = "";
+    return {
+      ...event,
+      url: absolute ? `${url.origin}${url.pathname}` : url.pathname,
+    };
+  } catch {
+    // Fail closed rather than risk forwarding an unrecognized URL shape.
+    return null;
+  }
+}
+
 /**
- * Pageviews remain enabled exactly as before. Product events are opt-in because
- * Vercel Hobby accepts pageviews but does not expose custom events.
+ * Each HTML entry point records one pageview. The playground mutates history
+ * as the source changes, so soft-navigation auto-tracking is disabled to avoid
+ * treating edits as visits.
  */
 export function initializeAnalytics(): void {
-  inject();
+  inject({
+    beforeSend: redactAnalyticsUrl,
+    disableAutoTrack: true,
+  });
   if (import.meta.env.VITE_PRODUCT_ANALYTICS_PROVIDER !== "vercel") return;
   configureUsageAnalytics(((name, properties) => {
     track(name, properties);

playground/test/analytics-wiring.test.ts

@@ -5,6 +5,7 @@ import { describe, expect, it } from "vitest";
 const root = resolve(import.meta.dirname, "../..");
 const main = readFileSync(resolve(root, "playground/src/main.ts"), "utf8");
 const editor = readFileSync(resolve(root, "playground/src/editor.ts"), "utf8");
+const landing = readFileSync(resolve(root, "playground/src/landing.ts"), "utf8");
 const products = readFileSync(
   resolve(root, "playground/src/for-products.ts"),
   "utf8",
@@ -24,9 +25,49 @@ describe("product analytics UI wiring", () => {
     expect(tracked).toBeGreaterThan(load);
   });
 
+  it("tracks a movement attempt only after a valid custom editor render", () => {
+    const load = main.indexOf("viewer.load(ir)");
+    const valid = main.indexOf('errors.length === 0', load);
+    const editorChange = main.indexOf(
+      'pendingRenderTrigger === "editor_change"',
+      valid,
+    );
+    const custom = main.indexOf('documentKind() === "custom"', editorChange);
+    const tracked = main.indexOf(
+      "usageSession.trackFirstValidCustomMovement()",
+      custom,
+    );
+
+    expect(load).toBeGreaterThan(-1);
+    expect(valid).toBeGreaterThan(load);
+    expect(editorChange).toBeGreaterThan(valid);
+    expect(custom).toBeGreaterThan(editorChange);
+    expect(tracked).toBeGreaterThan(custom);
+  });
+
+  it("tracks prompt copy only after clipboard success on both entry points", () => {
+    const playgroundCopy = main.indexOf(
+      "await navigator.clipboard.writeText(llmPrompt)",
+    );
+    const playgroundTracked = main.indexOf(
+      'usageSession.trackPromptCopied("playground")',
+      playgroundCopy,
+    );
+    const landingCopy = landing.indexOf("await writeClipboard(llmPrompt)");
+    const landingTracked = landing.indexOf(
+      'usageSession.trackPromptCopied("landing")',
+      landingCopy,
+    );
+
+    expect(playgroundCopy).toBeGreaterThan(-1);
+    expect(playgroundTracked).toBeGreaterThan(playgroundCopy);
+    expect(landingCopy).toBeGreaterThan(-1);
+    expect(landingTracked).toBeGreaterThan(landingCopy);
+  });
+
   it("tracks share only after the link reaches the clipboard", () => {
     const copied = main.indexOf("await navigator.clipboard.writeText(url)");
-    const tracked = main.indexOf("USAGE_EVENT_NAMES.shareCreated", copied);
+    const tracked = main.indexOf("usageSession.trackSuccessfulShare", copied);
     expect(copied).toBeGreaterThan(-1);
     expect(tracked).toBeGreaterThan(copied);
   });

playground/test/analytics.test.ts

@@ -20,6 +20,8 @@ describe("product usage analytics", () => {
       presetOpened: "preset_opened",
       editorChanged: "editor_changed",
       renderSucceeded: "render_succeeded",
+      promptCopied: "prompt_copied",
+      movementAttempted: "movement_attempted",
       shareCreated: "share_created",
       embedDocsClicked: "embed_docs_clicked",
       installCommandCopied: "install_command_copied",
@@ -46,6 +48,23 @@ describe("product usage analytics", () => {
     expect(sink).toHaveBeenCalledTimes(2);
   });
 
+  it("deduplicates each confirmed funnel outcome per page session", () => {
+    const session = new UsageSession();
+
+    session.trackPromptCopied("landing");
+    session.trackPromptCopied("playground");
+    session.trackFirstValidCustomMovement();
+    session.trackFirstValidCustomMovement();
+    session.trackSuccessfulShare("encoded");
+    session.trackSuccessfulShare("preset");
+
+    expect(sink.mock.calls).toEqual([
+      ["prompt_copied", { location: "landing" }],
+      ["movement_attempted", {}],
+      ["share_created", { share_kind: "encoded" }],
+    ]);
+  });
+
   it("does not let a blocked provider break product behavior", () => {
     configureUsageAnalytics((() => {
       throw new Error("blocked");