Implement ATJSON canonical content storage

[mdroidian]

Summary

This PR implements the ATJSON canonical content rollout described in:

• docs/atjson-canonical-storage-scope.md
• docs/atjson-canonical-storage-plan.md
• docs/atjson-port-plan.md

It stores DG ATJSON as the canonical structured representation in Content.metadata.content, keeps Content.text as derived plain text, and preserves the existing plain-text/Markdown rows used by current Obsidian and Roam workflows.

What changed

Database/storage

• Adds Content.content_type as the representation discriminator.
• Keeps variant as the semantic content slice.
• Updates content uniqueness and upsert_content conflict behavior to use (space_id, source_local_id, variant, content_type).
• Backfills existing variant = 'full' content as text/markdown; all other existing rows default to text/plain.
• Updates content_local_input, my_contents, embedding view, generated DB types, and shared content query fields.
• Updates FileReference with generated content_type = 'text/markdown' so existing Markdown asset references continue to target the Markdown full row.
• Prevents upsert_content from writing inline embeddings for non-text/plain rows.

Shared content model

• Adds @repo/content-model.
• Defines the DG-owned DgDocument model and content-type constants.
• Adds validation, derived plain-text generation, and metadata.content helpers.
• Adds parser/renderer helpers inspired by the useful SamePage ATJSON patterns without bringing over SamePage sync, protocol, Automerge, IPFS, or networking runtime code.

Converters/renderers

• Implements Obsidian Markdown -> DG ATJSON.
• Implements DG ATJSON -> Obsidian Markdown.
• Implements Roam native tree/text -> DG ATJSON.
• Implements DG ATJSON -> Roam Markdown and materialized block tree.
• Implements DG ATJSON -> HTML rendering with basic URL sanitization.
• Adds representative round-trip/parity tests for Obsidian and Roam fixtures.

App integration

• Obsidian writes:
  • direct/text/plain
  • full/text/markdown
  • full/application/vnd.discourse-graph.atjson+json; version=1
• Roam preserves existing text behavior and also writes full/application/vnd.discourse-graph.atjson+json; version=1.
• Existing Obsidian import/publish/read paths continue to select plain/Markdown rows and do not prefer ATJSON yet.
• Embedding batches only include plain-text rows.

Verification

Passed:

• pnpm --filter @repo/content-model test
• pnpm --filter @repo/content-model check-types
• pnpm --filter @repo/content-model lint
• pnpm --filter @repo/database test:atjson-contract
• pnpm --filter @repo/database check-types
• pnpm --filter @repo/database lint
• pnpm --filter @repo/database test
• pnpm --filter @discourse-graphs/obsidian test
• pnpm --filter @discourse-graphs/obsidian check-types
• pnpm --filter @discourse-graphs/obsidian lint
• pnpm --filter roam test
• pnpm --filter roam check-types
• pnpm --filter roam lint
• pnpm run build
• git diff --check

Additional local verification:

• Applied 20260517172000_atjson_content_type.sql against local Supabase.
• Ran a direct local Supabase ATJSON storage contract check covering:
  • plain/Markdown/ATJSON coexistence for one source node
  • ATJSON stored in metadata.content
  • Content.text storing derived plain text rather than serialized JSON
  • Markdown FileReference FK behavior
  • non-plain embedding guard

Notes:

• Lint commands pass with existing warning volume.
• git diff --check only reports Windows LF/CRLF warnings.

Reviewer notes

This is intentionally draft-sized and should be reviewed around the main contract boundaries first:

• SQL migration semantics and backfill behavior.
• The content_type + variant uniqueness model.
• The metadata.content ATJSON storage shape.
• Reader fallback behavior that keeps the current Obsidian sink working until the full read-path swap happens.

Out of scope

This PR does not implement:

• native export storage
• SamePage runtime sync or protocol behavior
• Automerge
• IPFS
• websocket transport
• destination-reader preference for ATJSON as the default read path

[supabase]

Updates to Preview Branch (init-atjson) ↗︎

Deployments	Status	Updated
Database	✅	Mon, 18 May 2026 03:32:11 UTC
Services	✅	Mon, 18 May 2026 03:32:11 UTC
APIs	✅	Mon, 18 May 2026 03:32:11 UTC

Tasks are run on every commit but only new migration files are pushed.
Close and reopen this PR if you want to apply changes from existing seed or migration files.

Tasks	Status	Updated
Configurations	✅	Mon, 18 May 2026 03:32:13 UTC
Migrations	✅	Mon, 18 May 2026 03:32:14 UTC
Seeding	✅	Mon, 18 May 2026 03:32:14 UTC
Edge Functions	✅	Mon, 18 May 2026 03:32:16 UTC

---

View logs for this Workflow Run ↗︎.
Learn more about Supabase for Git ↗︎.