You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
FAVA Trails will become easier to read through the Rich Views work in #51, but readability alone will not solve findability. Operators still need to answer questions like "where is the thought I half-remember?", "which scope contains the relevant decision?", and "what source records should an agent inspect before acting?"
The attached recommendation report argues that search and retrieval are a separate layer from rendering. That is directionally useful, but this PRD is intentionally only a skeleton. Requirements must be gathered first using Lima Discovery before this becomes an implementation-ready PRD.
The most important unresolved product question is not which search engine to use. It is the accountable end: who needs retrieval, in what moment, what they are currently doing instead, and what result would prove the layer is useful. Without that Lima, technical choices like Pagefind, SQLite FTS5, Quartz, Starlight, aliases, or semantic search would be premature.
Solution
Create a follow-up search and retrieval layer for FAVA Trails after the Rich Views reader in #51 has proven the local readability surface.
The intended end state has two retrieval surfaces:
Human retrieval over human artifacts: search and filters over the rendered local reader pages, dashboard rows, thought detail pages, lineage views, and relationship surfaces.
Agentic retrieval over source records: structured access to FAVA Markdown records and frontmatter so agents can retrieve exact source material without relying on HTML scraping or ad hoc Markdown parsing.
This skeleton does not choose the implementation stack. The technical implementation depends completely on how PRD #51 pans out: the routes it creates, the generated metadata it emits, the local server shape it establishes, the title/ULID model it proves, and the fixture seam it leaves behind.
Before implementation, run Lima Discovery to establish:
the living human or sharply typed operator being served;
the exact retrieval moment;
the current workaround;
the evidence that search is painful enough to build now;
whether human search, agent retrieval, or both are required first;
the stop condition that would kill or defer this layer.
User Stories
As a team operator, I want to find a remembered thought without knowing its scope, so that I can regain context without asking an agent to search for me.
As a team operator, I want to search rendered human-readable titles and excerpts, so that I can search the same artifacts I read.
As a team operator, I want search results to preserve source provenance, so that I can open the exact FAVA record behind a result.
As a team operator, I want search results to distinguish current and superseded thoughts, so that I do not rely on stale conclusions.
As a team operator, I want filters for scope, namespace, source type, validation status, confidence, tags, and agent, so that I can narrow results without remembering exact words.
As a team operator, I want drafts and proposed records to be explicitly controllable in search, so that unfinished memory does not silently pollute or disappear from results.
As a team operator, I want current decisions to rank or filter distinctly, so that durable commitments are easy to find.
As a team operator, I want lineage-aware results, so that if I find an old thought I can navigate to the current replacement.
As a team operator, I want relationship-aware retrieval, so that I can find nearby records that depend on, reference, revise, or supersede the result.
As a team operator, I want search to run locally over private FAVA data, so that private trails are not exposed to public infrastructure.
As a team operator, I want search freshness to be visible, so that I know whether the index reflects the current trail state.
As a team operator, I want a manual way to rebuild the index, so that I can recover from stale or suspicious search results.
As a team operator, I want search results to use human-readable display titles while preserving ULID identity, so that the interface is readable and stable.
As a team operator, I want same-title records to remain distinct, so that superseded and current versions do not collapse into one result.
As a team operator, I want partial-word or prefix matching if Lima Discovery shows that operators remember fragments, so that search matches real recall behavior.
As a team operator, I want phrase or exact matching if Lima Discovery shows that operators search by quoted snippets, so that precise recall works.
As a team operator, I want tag and metadata facets if Lima Discovery shows operators browse by project shape, so that search supports exploration rather than only keyword lookup.
As an agent, I want a source-record retrieval surface, so that I can retrieve exact Markdown and frontmatter without scraping rendered HTML.
As an agent, I want stable identifiers and content hashes in retrieval results, so that I can detect drift and avoid stale references.
As an agent, I want retrieval results to include source type, validation status, supersession state, confidence, tags, and scope, so that I can decide whether a record is safe to use.
As an agent, I want retrieval to hide or demote superseded records by default unless asked, so that current truth is easier to consume.
As an agent, I want retrieval to expose unresolved or broken relationships clearly, so that I can avoid treating incomplete graph context as reliable.
As a FAVA maintainer, I want the search layer to reuse the reader artifacts from PRD: FAVA Rich Views semantic renderer #51 where appropriate, so that search does not create another competing representation.
As a FAVA maintainer, I want source Markdown to remain the durable source of truth, so that search indexes can be deleted and rebuilt.
As a FAVA maintainer, I want implementation choices deferred until PRD: FAVA Rich Views semantic renderer #51 is complete enough to inspect, so that search does not force premature architecture into the reader.
As a FAVA maintainer, I want candidate engines evaluated against the discovered retrieval job, so that Pagefind, SQLite FTS5, or another option is chosen for evidence-backed reasons.
As a FAVA maintainer, I want any canonical title or alias migration to be justified by retrieval requirements, so that readability work does not accidentally mutate storage conventions.
As a FAVA maintainer, I want optional external projections such as Obsidian or Quartz treated as downstream consumers, so that they do not define canonical FAVA identity.
As a FAVA maintainer, I want the search index to be rebuildable and disposable, so that corruption or stale state does not damage the trail.
As a FAVA maintainer, I want a clear separation between human search and agentic retrieval, so that each surface can optimize for its caller without confusing HTML reading with source-grounded recall.
Implementation Decisions
This is a PRD skeleton, not an implementation-ready PRD.
Requirements must be gathered first using Lima Discovery before implementation issues are created.
The search and retrieval PRD must not become a commitment-class artifact until a passing Lima Card exists or the project is explicitly kept as a reversible spike.
The first implementation seam should be chosen only after PRD: FAVA Rich Views semantic renderer #51 establishes the local reader's routes, generation metadata, fixture data shape, title/ULID behavior, and CLI serving model.
The likely highest-level future test seam is: FAVA source records plus rendered reader artifacts become queryable human results and source-grounded agent retrieval results.
The layer should distinguish human retrieval over rendered artifacts from agentic retrieval over source Markdown and frontmatter.
Candidate human-artifact search options from the report include Pagefind over static rendered HTML and search features inherited from a future Starlight, Quartz, or other projection.
Candidate source-record retrieval options from the report include SQLite FTS5 with facet columns over frontmatter and body, plus later semantic/vector retrieval only if Lima Discovery proves a need.
Pagefind, SQLite FTS5, Quartz, Starlight, MkDocs Material, Obsidian projection, Orama, Tantivy, Meilisearch, and Typesense are research inputs, not decisions.
Public website serving is not part of this PRD skeleton; FAVA data is private by default.
Private hosted serving should remain deferred unless Lima Discovery identifies real users and a delivery need beyond local usage.
Local privacy and rebuildability should remain baseline assumptions unless Lima Discovery proves otherwise.
ULIDs remain durable identity unless a later PRD explicitly changes the storage model.
Human-readable titles and aliases are an open requirements question, not a committed canonical migration.
Optional Obsidian or Quartz projections should be treated as generated downstream artifacts, not as canonical FAVA storage.
Search freshness, regeneration triggers, and index rebuild behavior must be specified after PRD: FAVA Rich Views semantic renderer #51 establishes its generation and local server mechanics.
Tests should verify that current, superseded, draft, proposed, rejected, and tombstoned records behave according to the discovered retrieval requirements.
Tests should verify that same-title records preserve ULID identity in search results.
Tests should verify that result metadata includes scope, namespace, source type, validation status, confidence, tags, agent, source reference, and freshness information where required.
Tests should verify index rebuild or refresh behavior once the freshness trigger is chosen.
Tests should verify that private FAVA data is served locally by default if the search layer plugs into the local CLI reader.
Prior art for tests includes PRD: FAVA Rich Views semantic renderer #51's planned fixture seam, existing FAVA model parsing tests, recursive scope discovery tests, lifecycle tests for promotion and supersession, and local CLI verification patterns.
Choosing Pagefind, SQLite FTS5, Quartz, Starlight, Obsidian, or another stack in this skeleton.
Public website hosting for private FAVA data.
Private hosted service deployment without identified users.
Semantic or vector search unless Lima Discovery proves full-text and facet retrieval are insufficient.
Canonical storage migration to Obsidian conventions.
Renaming FAVA source files away from ULID identity.
Building write paths, edit surfaces, or alternative thought lifecycle operations.
Replacing MCP recall.
Further Notes
This PRD skeleton is based on the attached report, "Adding a Read-Only Reading & Search Layer to FAVA Trails: Recommendation Report."
The report's technical recommendations should be treated as research inputs, not binding decisions. In particular:
Pagefind may be appropriate for search over rendered human artifacts.
SQLite FTS5 may be appropriate for local faceted source-record retrieval.
Quartz, Starlight, MkDocs Material, or optional Obsidian projections may become useful later, but they should not drive canonical FAVA identity.
Title and alias frontmatter may become useful for retrieval, but should be justified by requirements rather than assumed.
The first required next step is Lima Discovery. The Lima should establish the accountable end before the skeleton becomes an implementation PRD.
The second dependency is #51. The search and retrieval implementation should be designed only after the FAVA Rich Views reader shows what human artifacts exist, how they are generated, how they are served locally, and which metadata they expose.
Problem Statement
FAVA Trails will become easier to read through the Rich Views work in #51, but readability alone will not solve findability. Operators still need to answer questions like "where is the thought I half-remember?", "which scope contains the relevant decision?", and "what source records should an agent inspect before acting?"
The attached recommendation report argues that search and retrieval are a separate layer from rendering. That is directionally useful, but this PRD is intentionally only a skeleton. Requirements must be gathered first using Lima Discovery before this becomes an implementation-ready PRD.
The most important unresolved product question is not which search engine to use. It is the accountable end: who needs retrieval, in what moment, what they are currently doing instead, and what result would prove the layer is useful. Without that Lima, technical choices like Pagefind, SQLite FTS5, Quartz, Starlight, aliases, or semantic search would be premature.
Solution
Create a follow-up search and retrieval layer for FAVA Trails after the Rich Views reader in #51 has proven the local readability surface.
The intended end state has two retrieval surfaces:
This skeleton does not choose the implementation stack. The technical implementation depends completely on how PRD #51 pans out: the routes it creates, the generated metadata it emits, the local server shape it establishes, the title/ULID model it proves, and the fixture seam it leaves behind.
Before implementation, run Lima Discovery to establish:
User Stories
Implementation Decisions
Testing Decisions
Out of Scope
Further Notes
This PRD skeleton is based on the attached report, "Adding a Read-Only Reading & Search Layer to FAVA Trails: Recommendation Report."
The report's technical recommendations should be treated as research inputs, not binding decisions. In particular:
The first required next step is Lima Discovery. The Lima should establish the accountable end before the skeleton becomes an implementation PRD.
The second dependency is #51. The search and retrieval implementation should be designed only after the FAVA Rich Views reader shows what human artifacts exist, how they are generated, how they are served locally, and which metadata they expose.