feat(search): narrow a reference’s facet to the keys its target admits

docs/decisions/0022-key-a-root-type-on-a-declared-field.md

@@ -6,6 +6,11 @@ Date: 2026-08-21
 
 Accepted
 
+Amended 2026-08-22: gains the **facet policy** – `facetKeys` on a Root Type,
+inherited by every facetable reference that names it, along the same boundary
+as re-keying – as the index-side half of the same design. See “The facet
+policy” under Decision and the two consequences it adds.
+
 Extends [ADR 20](./0020-resolve-a-references-fields-from-the-targets-own-collection.md),
 whose contract – a reference holds ids of documents in the target’s collection –
 is what makes reference rewriting a consequence rather than a new rule. Relates
@@ -71,14 +76,30 @@ reference dangling.
 needs a node’s key before the projection runs reads the same answer the
 projection will.
 
-### The boundary, for keys and for joins alike
+### The boundary, for keys, facets and joins alike
 
 Only a reference that **names** its target – a `lookup`’s `target`, an
-`idOnly`’s `labelSource` – is re-keyed. That is the same line a join draws, and
-for the same reason: naming the target is what asserts that the field holds ids
-of that collection’s documents. An `idOnly` reference with no label source, and
-a `derive`d reference over a raw internal path, never claimed as much, so
-nothing rewrites them.
+`idOnly`’s `labelSource` – is re-keyed, and only such a reference inherits the
+target’s facet policy. That is the same line a join draws, and for the same
+reason: naming the target is what asserts that the field holds ids of that
+collection’s documents. An `idOnly` reference with no label source, and a
+`derive`d reference over a raw internal path, never claimed as much, so nothing
+rewrites them and no policy narrows them.
+
+### The facet policy
+
+A Root Type may declare which of its documents get a facet bucket –
+`facetKeys: { only: isCovered }`, a predicate over the document key – and every
+facetable reference naming the type inherits it. _Which ids deserve a bucket_
+is a fact about the target, not about each field pointing at it, so it is
+declared once; the alternative, a query-time facet variant, adds a facets entry
+type, a validation rule and a surface enum for a choice the deployment already
+made when it built a cross-dataset index. Only the facet narrows: the field
+keeps every value, so a local place still displays and still filters exactly.
+The mechanism is a second physical field per inheriting facet – `${name}_facet`,
+the admitted subset, written by the projection from the already-keyed values –
+which the engine facets instead of the field, so the facet is exact under any
+bucket cap.
 
 ## Consequences
 
@@ -113,16 +134,40 @@ nothing rewrites them.
 - **The key is assigned before any `derive` runs**, so a derive sees the key and
   never the node IRI. A deployment that wants the node IRI declares a plain
   `idOnly` reference over the same path.
+- **A facet policy is a second physical field in the engine** – a subset of the
+  same ids stored twice – and a collection-definition change for every type
+  referencing the policy’s type. Under a blue-green rebuild that is the next
+  run; under an in-place rebuild it is a loud failure at run open, after the
+  lock and before any write, asking for the collection to be dropped once:
+  rotating a pipeline version reprocesses datasets, it does not recreate
+  collections. The Typesense adapter keys facet results off the engine’s
+  physical field name, so the companion’s name is mapped back to the declared
+  field once, at the response boundary, before labels are resolved and counts
+  filed; and a membership filter follows the field’s _engine_ facet status
+  rather than its declaration, or a filter on an excluded value – the very case
+  the policy promises stays whole – would compile to the tokenised operator and
+  partial-match on a shared path prefix.
+- **An over-strict predicate empties a facet instead of erroring.** Applied to
+  a type whose keys are not what it tests for – or to one keyed after its
+  references were indexed, so the stored values are still node IRIs – it admits
+  nothing and every facet referencing the type comes back empty, silently. A
+  documented footgun on `facetKeys` for now; a writer noticing a companion that
+  stayed empty across a run while its field did not is a cheap tell to add
+  later.
 - **A cross-dataset node reference does not resolve.** A work in dataset A
   pointing at a local node in dataset B gets no candidates – the hop runs against
   A’s distribution – so it stores the node IRI and dangles against B’s keyed
   document. Publishers reference other publishers through `sameAs` rather than
   directly, and such a reference is already unresolvable today for every purpose
   but labels.
-- `@lde/pipeline`, `@lde/search-indexer` and the API packages are untouched, and
-  `@lde/search-typesense` only adopts the shared `rootTypeNamed` in place of a
-  by-name map of its own: the change is a schema member, the projection, and one
-  hop in the extraction generator. A schema declaring no `key` extracts,
+- `@lde/pipeline`, `@lde/search-indexer` and the API packages are untouched.
+  The key is a schema member, the projection, and one hop in the extraction
+  generator, with `@lde/search-typesense` only adopting the shared
+  `rootTypeNamed` in place of a by-name map of its own; the facet policy is a
+  second schema member, the companion in the projection, and – in the adapter –
+  the companion in the collection definition, the facet and membership clauses
+  in the query compiler, the name mapping at the response boundary and the
+  in-place check. A schema declaring neither `key` nor `facetKeys` extracts,
   projects, indexes and queries exactly as before.
 
 ## Rejected

docs/reference/search-typesense.md

@@ -98,6 +98,21 @@ labels. Pass the `schema` option for a type that surfaces one – it is what
 resolves the Reference Type; a type declaring one without it throws here, rather
 than building a collection whose documents would all fail to import.
 
+A **reference inheriting a [facet policy](./search#facet-policy)** – a facetable
+`lookup`/`labelSource` reference to a type declaring `facetKeys` – is declared
+as a plain stored field (`facet: false`) plus a `${name}_facet` companion
+(`facet: true`, optional) holding the admitted subset, and the query compiler
+facets the companion and filters the field itself with the exact `:=`
+operator. Only the `schema` option can resolve the policy: built without it,
+the field itself is declared the facet and no companion is – the projection
+makes the same reading without a schema – so a deployment declaring a policy
+passes the schema to its writers, as `createSearchIndexer` does.
+
+An `InPlaceRebuild` additionally refuses a type whose declared
+[dataset field](#provenance) inherits a policy: it enumerates the indexed
+datasets by faceting that field, and a facet narrowed to the admitted keys
+would hide every dataset the policy excludes from the membership sweep.
+
 ### Joins across collections
 
 A reference declaring [`joinable: true`](./search#filtering-across-collections)
@@ -309,10 +324,14 @@ Document ids must be unique per (source, entity) – the caller keys them.
 `openRun` creates the collection on demand and otherwise leaves an existing one
 alone – with one exception. If the collection exists but does not carry every
 [reference field](#joins-across-collections) the declaration asks for (a
-`joinable` added to a schema whose index predates it), the run **fails**, naming
-the drop-and-rebuild that fixes it. Without that it would index and commit
-happily and then 400 on every join query: the values would be there, the
-reference would not. Scoped to reference fields only – every other schema
+`joinable` added to a schema whose index predates it), or every
+[facet companion](#collection-schema-and-engine) its facets read (a `facetKeys`
+policy added to a type this one references), the run **fails** at open – after
+the lock, before any write – naming the drop-and-rebuild that fixes it. Without
+that it would index and commit happily and then fail on every join query, or
+on every facet over that reference: the values would be there, the reference
+or the facet field would not. Rotating a pipeline version reprocesses datasets;
+it does not recreate collections. Scoped to those two – every other schema
 difference is self-correcting.
 
 ### The join component is the unit of rebuild

docs/reference/search.md

@@ -72,7 +72,9 @@ Exports are stratified by audience:
   - `assertTypeInSchema` – the port membership guard (the exact declaration
     object must be in the schema);
   - `physicalFields` / `PhysicalFields` – the indexed physical fanout a field
-    produces (per-locale search/sort keys);
+    produces (per-locale search/sort keys, and the field a facet reads – the
+    field itself, or the companion of a reference inheriting a
+    [facet policy](#facet-policy), which is why it takes the schema);
   - `physicalNameTokens` – the neutral name tokens an engine formats its own
     collection/index names from;
   - `irAlias` – the minted extraction predicate
@@ -170,9 +172,9 @@ graph. A field that
 declares **no** role is an **internal field**: projected so a later `derive` can
 read it, then pruned before the writer and absent from the collection definition
 – not stored, not indexed, no RAM. The physical field names a declaration fans
-out to (per-locale search/sort keys) come from `physicalFields`, the single
-convention projection, the collection definition and the query compiler all
-share.
+out to (per-locale search/sort keys, the field a facet reads) come from
+`physicalFields`, the single convention projection, the collection definition
+and the query compiler all share.
 
 ```ts
 import { defineSearchType, projectRoots, searchSchema } from '@lde/search';
@@ -614,6 +616,62 @@ later:
   straight into the `is` filter that selects it;
 - a range-facet bin carries `min`/`max` instead – see below.
 
+### Facet policy
+
+A Root Type may declare **`facetKeys`**: which of its documents get a facet
+bucket, as a predicate over the [document key](#document-key). It is declared
+once, on the type, and inherited by every facetable reference that _names_ it –
+a `lookup`’s `target`, an `idOnly`’s `labelSource` – the same boundary along
+which a reference is re-keyed and a join is drawn (so a `derive`d reference,
+which reads no referent, is narrowed by nothing either):
+
+```ts
+const place = defineSearchType({
+  name: 'Place',
+  class: `${SCHEMA}Place`,
+  key: { field: '_sameAs', pick: (candidates) => candidates.find(isCovered) },
+  // Cross-dataset index: only a key in a covered source gets a bucket, on every
+  // facet that references Place – a publisher’s local place is never one.
+  facetKeys: { only: isCovered },
+  fields: [/* … */],
+});
+```
+
+_Which `Place` ids deserve a bucket_ is a fact about `Place`, not about each of
+the fields that point at it (`CreativeWork.locationCreated`, `Person.birthPlace`,
+`Organization.location`, …), so the policy lives on the target: a per-field
+declaration would state one rule several times, and forgetting one would
+silently reintroduce the buckets on that facet alone. The typical reason is a
+**cross-dataset index**: two publishers who both left the same-named place
+unaligned yield two buckets with one label, which a consumer cannot tell apart –
+while a single-dataset app on its own index wants every bucket. The deployment
+knows which it is building, so the choice is an indexing policy, not a query
+option; a consumer sees one facet that simply has no local buckets.
+
+**Only the facet narrows.** The referring field keeps every value: a document
+still displays the excluded place, and `where: { locationCreated: { in:
+[kessel] } }` still matches it exactly. _Facets are discovery, filters are
+exact_ – the one place the two deliberately disagree, so state it where your
+consumers read.
+
+How it works: for a reference inheriting a policy, the projection writes a
+`${name}_facet` companion holding the admitted subset of the field’s values –
+taken after the field’s own `transform`, so the policy sees what the field
+stores, which for a keyed target is the key – and the engine facets the
+companion instead of the field (`physicalFields(field, schema).facet`). The
+facet is therefore exact under any bucket cap: the engine never sees an
+excluded value. Declaring a policy changes the collection definition of every
+type that references the policy’s type; see the
+[Typesense adapter](./search-typesense#collection-schema-and-engine) for what
+that means for a live collection.
+
+**The failure mode is silence.** A predicate that admits none of a type’s keys
+empties every facet referencing that type, with no error: a `Term` aligned to
+AAT given the places’ `isCovered`, or – the likelier trap – a type that was
+`key`ed _after_ its references were indexed, so the stored values are still
+node IRIs. Key a type before declaring a policy over its keys, and give each
+type its own predicate rather than reuse another’s.
+
 ### Range facets
 
 A facetable numeric field (`integer`/`number`/`date`) may declare

packages/search-typesense/package.json

@@ -31,6 +31,7 @@
     "typesense": "^3.0.6"
   },
   "devDependencies": {
+    "n3": "^2.1.2",
     "testcontainers": "^12.1.0"
   },
   "peerDependencies": {

packages/search-typesense/src/collection-definition.ts

@@ -39,11 +39,19 @@ export interface CollectionDefinitionOptions {
   /**
    * The Search Schema the type belongs to – required when the type surfaces an
    * inline reference (whose nested fields are declared from the
-   * {@link ReferenceType} the schema resolves) or declares a joinable
+   * {@link ReferenceType} the schema resolves), declares a joinable
    * reference (whose target collection the schema’s {@link joinGraph}
    * resolves). A type doing neither needs no schema, so a caller passes none;
    * a type that does fails here rather than building a collection that
    * silently omits the nesting or the reference.
+   *
+   * Also what resolves the facet policy a facetable reference inherits from
+   * the type it names ({@link physicalFields}): without the schema the field
+   * itself is declared the facet and no `${name}_facet` companion is, the same
+   * reading the projection makes without one. Whether a policy applies cannot
+   * be told without the schema, so this is not guarded: a deployment that
+   * declares one and builds its collections schema-less meets the engine’s own
+   * error on the first facet query, not a silently narrowed-to-nothing facet.
    */
   readonly schema?: SearchSchema;
   /** Snowball stemming locale for non-localized searchable fields (e.g. `en`).
@@ -233,7 +241,7 @@ function typesenseFields(
   if (nested !== undefined) {
     return nestedFields(field.name, field, nested, schema as SearchSchema);
   }
-  const names = physicalFields(field);
+  const names = physicalFields(field, schema);
   if (field.kind === 'text') {
     const locales = field.locales;
     const displayPattern = displayFieldPattern(field);
@@ -281,7 +289,10 @@ function typesenseFields(
     {
       name: field.name,
       type: valueType,
-      facet: field.facetable ?? false,
+      // A reference inheriting a facet policy facets its companion below, and
+      // the field itself stays a plain stored value – which is also what keeps
+      // a membership filter on it exact (`compileMembership`).
+      facet: names.facet === field.name,
       sort: field.sortable ?? false,
       // A `required` field is non-optional; so is the `default_sorting_field`,
       // which Typesense requires to be present. Everything else may be absent.
@@ -302,6 +313,18 @@ function typesenseFields(
       }),
     });
   }
+  // The `${name}_facet` companion of a reference inheriting a facet policy:
+  // the admitted subset of the field’s values, and the only one of the two
+  // the engine facets. Optional, because a document none of whose values the
+  // policy admits carries no companion at all.
+  if (names.facet !== undefined && names.facet !== field.name) {
+    fields.push({
+      name: names.facet,
+      type: valueType,
+      facet: true,
+      optional: true,
+    });
+  }
   return fields;
 }