Skip to content

feat(search): key a root type on a declared field - #764

Merged
ddeboer merged 4 commits into
mainfrom
worktree-key-root-type-on-declared-field
Aug 22, 2026
Merged

feat(search): key a root type on a declared field#764
ddeboer merged 4 commits into
mainfrom
worktree-key-root-type-on-declared-field

Conversation

@ddeboer

@ddeboer ddeboer commented Aug 21, 2026

Copy link
Copy Markdown
Member

PR 1 of #763 – the write side. The facet policy (facetKeys: { only }, PR 2)
is not in here.

What changes

A Root Type is keyed on the node’s IRI unless it names a key field to
read the key from. A reference to such a type stores the target’s key.

key: {
  field: '_sameAs',
  pick: (candidates) => candidates.find(isGeoNames) ?? candidates.find(isCovered),
}

Non-breaking and opt-in: a schema declaring no key extracts, projects, indexes
and queries exactly as before. @lde/pipeline, @lde/search-indexer and the API
packages are untouched; the selector is unchanged.

@lde/search

  • KeyField / RootType.key. field names a declared field of the type: a
    path-bearing, array reference field that is not inline. Naming a declared
    field is the point – its extraction branch already exists, a reader transform
    that repairs reference values covers the candidates, and the field’s own
    transform is where IRI normalisation lives, so two spellings of one IRI
    become one candidate before anything chooses between them.
  • documentKeyOf (exported from /adapter) is the whole rule in one
    function – transform, IRI filter, dedupe, sort, pick – so the projection
    and a deployment’s transform read the same answer. Candidates reach pick
    sorted, so the default (the first) is deterministic whatever order the
    CONSTRUCT returned them in; pick must return one of them or undefined, and
    anything else throws naming the node and the candidates.
  • rootTypeNamed resolves the type a declaration points at, replacing the
    by-name map assertResolvableLabelSources kept for itself (and the one in
    @lde/search-typesense’s projection lookup).
  • Projection. A root is keyed through documentKeyOf; a reference that
    names a keyed target (a lookup’s target, an idOnly’s labelSource)
    stores the referent’s key. A reference naming no target, a derived one, and a
    projection run without a schema are left alone – that is the same line a join
    draws, and for the same reason.
  • Validation rejects a key naming an unknown, non-reference, path-less,
    single-valued or inline field, a non-function pick, and a key on a
    Reference Type (typed never, so it also fails to compile).

Also fixes an error message this refactor would otherwise have made misleading:
a lookup naming a declared Reference Type used to complain about a missing
label field and now says the name is unknown – so it names it for what it is
instead, and says why only a Root Type can serve labels.

@lde/search-pipeline

One OPTIONAL hop per reference into a keyed target, reading the referent’s key
field and emitting it under the target’s IR Alias. Inside that reference’s
own UNION branch, so it multiplies against nothing; OPTIONAL, so an unaligned
referent keeps its row and its own IRI instead of dropping out of the CONSTRUCT.
The root side is unchanged.

Consequences worth reading

  • Several nodes with one key are one document. That is what a document key
    means; the writer upserts by id. The projection still emits one document per
    distinct root – folding them is the writer’s upsert.
  • A transform that replaces a root’s quads must re-emit the key field – the
    existing rule that a field the document needs must be in the stream, applied to
    one more field. Documented on key and in the indexer reference; left a
    convention rather than a guarantee, since the structural alternative touches
    @lde/pipeline’s runner.
  • A transform that supplies key candidates must reach every referring type.
    A transform is attached to one type’s reader, and a reference’s key is read in
    the referring type’s extraction query – so candidates minted on the target
    alone key the target’s own document while every reference to it still stores
    the node IRI. Repairing candidates the graph already carries is unaffected.
  • The referring field’s own transform runs on the key, not on the
    referent’s node IRI, since a transform transforms what the field stores.
    Pinned by a test so the ordering is deliberate.
  • The key is assigned before any derive runs, so a derive sees the key.
  • A cross-dataset node reference does not resolve (the hop runs against the
    referring dataset’s distribution). Recorded in the docs and the ADR.

Documentation

ADR 22 Key a root type on a declared field, a “Document key” section in the
@lde/search reference beside labelField, the key hop in the
@lde/search-pipeline extraction section, and the re-emit rule in
@lde/search-indexer’s “Add a transform”.

Follow-ups (not done here)

Per the issue: #754 is no longer a precondition, #760 stays latent, and the
multi-valued provenance stamp (#697 / #709) is worth un-deferring since this
makes shared documents common. Left for you to decide on the tracker.

Part of #763 – deliberately not Fix, so the issue stays open for PR 2.

A Root Type is keyed on the node’s IRI unless it names a `key` field to read
the key from. A reference to such a type stores the target’s key, which is
what a `lookup`’s `target` and an `idOnly`’s `labelSource` already promise.

- `KeyField` and `RootType.key`: `field` names a declared, path-bearing,
  array, non-inline reference field whose values are the candidates; `pick`
  chooses among them, defaulting to the first, and `undefined` keeps the
  node’s own IRI.
- `documentKeyOf` is that rule in one exported function – transform, IRI
  filter, dedupe, sort, pick – so the projection and a deployment’s transform
  read the same answer. A `pick` returning a value no candidate offered
  throws, naming the node and the candidates.
- `rootTypeNamed` resolves the type a declaration points at, replacing the
  by-name map `assertResolvableLabelSources` kept for itself.
- The projection keys a root through `documentKeyOf`, and re-keys the values
  of a reference that names a keyed target. A reference naming no target, a
  derived one, and a projection run without a schema are left alone.
- Validation rejects a key naming an unknown, non-reference, path-less,
  single-valued or inline field, a non-function `pick`, and a key on a
  Reference Type.
- ADR 22, plus a “Document key” section in the search reference and the
  re-emit rule for a replacing transform in the indexer reference.
…amed

The projection lookup kept a by-name map of its own, rebuilt per call. Read
the schema’s own index instead, so the projection, the label-source
validation and this lookup resolve a target the same way.

- Re-anchor the function-coverage threshold: dropping one covered arrow
  lowers the ratio without uncovering anything.
A reference naming a target that declares a `key` needs the referent’s key
field in the frame, or the projection has nothing to re-key it from.

- Extend such a reference’s UNION branch with an `OPTIONAL` hop reading the
  key field, emitted under the target’s own IR Alias. Inside the branch, so
  it multiplies against nothing; `OPTIONAL`, so a referent with no candidate
  keeps its row and its own IRI.
- The root side is unchanged: a key field is a declared field, so its branch
  and template triple are already there.
- Extend the extraction ⟷ projection alias drift guard to the key hop, and
  add an end-to-end run over a fixture of aligned, unaligned, multiply
  aligned and declined places with a work referencing each.
Resolving a label source through the Root Type index turned a name that IS
declared, just not as a Root Type, into “unknown label source … declare a
SearchType with that name” – which sends an author looking in the wrong
place. Say it is a Reference Type, and why only a Root Type can serve.

Documentation, from review of the key feature:

- A transform SUPPLYING key candidates must reach every referring type: a
  transform is attached to one type’s reader, and a reference’s key is read
  in the referring type’s query, so candidates minted on the target alone
  leave every reference keyed on the node IRI. Repairing candidates the graph
  already carries is unaffected.
- The referring field’s own `transform` runs on what it stores, which for a
  keyed target is the key rather than the referent’s node IRI – pinned by a
  test, so the ordering is deliberate.
- `documentKeyOf` states which shape its `rawValues` are, so a transform that
  reads them off quads cannot compute a key from a literal-valued object the
  projection drops.
- The key hop repeats a reference’s template triple once per candidate –
  linearly, like an inline reference’s link triple – rather than “multiplying
  against nothing”.
- ADR 22 no longer claims `@lde/search-typesense` is untouched.
@ddeboer
ddeboer merged commit ad64b95 into main Aug 22, 2026
4 checks passed
@ddeboer
ddeboer deleted the worktree-key-root-type-on-declared-field branch August 22, 2026 12:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant