feat(search): key a root type on a declared field - #764
Merged
Conversation
A Root Type is keyed on the node’s IRI unless it names a `key` field to read the key from. A reference to such a type stores the target’s key, which is what a `lookup`’s `target` and an `idOnly`’s `labelSource` already promise. - `KeyField` and `RootType.key`: `field` names a declared, path-bearing, array, non-inline reference field whose values are the candidates; `pick` chooses among them, defaulting to the first, and `undefined` keeps the node’s own IRI. - `documentKeyOf` is that rule in one exported function – transform, IRI filter, dedupe, sort, pick – so the projection and a deployment’s transform read the same answer. A `pick` returning a value no candidate offered throws, naming the node and the candidates. - `rootTypeNamed` resolves the type a declaration points at, replacing the by-name map `assertResolvableLabelSources` kept for itself. - The projection keys a root through `documentKeyOf`, and re-keys the values of a reference that names a keyed target. A reference naming no target, a derived one, and a projection run without a schema are left alone. - Validation rejects a key naming an unknown, non-reference, path-less, single-valued or inline field, a non-function `pick`, and a key on a Reference Type. - ADR 22, plus a “Document key” section in the search reference and the re-emit rule for a replacing transform in the indexer reference.
…amed The projection lookup kept a by-name map of its own, rebuilt per call. Read the schema’s own index instead, so the projection, the label-source validation and this lookup resolve a target the same way. - Re-anchor the function-coverage threshold: dropping one covered arrow lowers the ratio without uncovering anything.
A reference naming a target that declares a `key` needs the referent’s key field in the frame, or the projection has nothing to re-key it from. - Extend such a reference’s UNION branch with an `OPTIONAL` hop reading the key field, emitted under the target’s own IR Alias. Inside the branch, so it multiplies against nothing; `OPTIONAL`, so a referent with no candidate keeps its row and its own IRI. - The root side is unchanged: a key field is a declared field, so its branch and template triple are already there. - Extend the extraction ⟷ projection alias drift guard to the key hop, and add an end-to-end run over a fixture of aligned, unaligned, multiply aligned and declined places with a work referencing each.
Resolving a label source through the Root Type index turned a name that IS declared, just not as a Root Type, into “unknown label source … declare a SearchType with that name” – which sends an author looking in the wrong place. Say it is a Reference Type, and why only a Root Type can serve. Documentation, from review of the key feature: - A transform SUPPLYING key candidates must reach every referring type: a transform is attached to one type’s reader, and a reference’s key is read in the referring type’s query, so candidates minted on the target alone leave every reference keyed on the node IRI. Repairing candidates the graph already carries is unaffected. - The referring field’s own `transform` runs on what it stores, which for a keyed target is the key rather than the referent’s node IRI – pinned by a test, so the ordering is deliberate. - `documentKeyOf` states which shape its `rawValues` are, so a transform that reads them off quads cannot compute a key from a literal-valued object the projection drops. - The key hop repeats a reference’s template triple once per candidate – linearly, like an inline reference’s link triple – rather than “multiplying against nothing”. - ADR 22 no longer claims `@lde/search-typesense` is untouched.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR 1 of #763 – the write side. The facet policy (
facetKeys: { only }, PR 2)is not in here.
What changes
Non-breaking and opt-in: a schema declaring no
keyextracts, projects, indexesand queries exactly as before.
@lde/pipeline,@lde/search-indexerand the APIpackages are untouched; the selector is unchanged.
@lde/searchKeyField/RootType.key.fieldnames a declared field of the type: apath-bearing,
arrayreference field that is notinline. Naming a declaredfield is the point – its extraction branch already exists, a reader transform
that repairs reference values covers the candidates, and the field’s own
transformis where IRI normalisation lives, so two spellings of one IRIbecome one candidate before anything chooses between them.
documentKeyOf(exported from/adapter) is the whole rule in onefunction – transform, IRI filter, dedupe, sort,
pick– so the projectionand a deployment’s transform read the same answer. Candidates reach
picksorted, so the default (the first) is deterministic whatever order the
CONSTRUCT returned them in;
pickmust return one of them orundefined, andanything else throws naming the node and the candidates.
rootTypeNamedresolves the type a declaration points at, replacing theby-name map
assertResolvableLabelSourceskept for itself (and the one in@lde/search-typesense’s projection lookup).documentKeyOf; a reference thatnames a keyed target (a
lookup’starget, anidOnly’slabelSource)stores the referent’s key. A reference naming no target, a
derived one, and aprojection run without a schema are left alone – that is the same line a join
draws, and for the same reason.
single-valued or
inlinefield, a non-functionpick, and a key on aReference Type (typed
never, so it also fails to compile).Also fixes an error message this refactor would otherwise have made misleading:
a
lookupnaming a declared Reference Type used to complain about a missinglabel field and now says the name is unknown – so it names it for what it is
instead, and says why only a Root Type can serve labels.
@lde/search-pipelineOne
OPTIONALhop per reference into a keyed target, reading the referent’s keyfield and emitting it under the target’s IR Alias. Inside that reference’s
own UNION branch, so it multiplies against nothing;
OPTIONAL, so an unalignedreferent keeps its row and its own IRI instead of dropping out of the CONSTRUCT.
The root side is unchanged.
Consequences worth reading
means; the writer upserts by
id. The projection still emits one document perdistinct root – folding them is the writer’s upsert.
existing rule that a field the document needs must be in the stream, applied to
one more field. Documented on
keyand in the indexer reference; left aconvention rather than a guarantee, since the structural alternative touches
@lde/pipeline’s runner.A transform is attached to one type’s reader, and a reference’s key is read in
the referring type’s extraction query – so candidates minted on the target
alone key the target’s own document while every reference to it still stores
the node IRI. Repairing candidates the graph already carries is unaffected.
transformruns on the key, not on thereferent’s node IRI, since a
transformtransforms what the field stores.Pinned by a test so the ordering is deliberate.
deriveruns, so a derive sees the key.referring dataset’s distribution). Recorded in the docs and the ADR.
Documentation
ADR 22 Key a root type on a declared field, a “Document key” section in the
@lde/searchreference besidelabelField, the key hop in the@lde/search-pipelineextraction section, and the re-emit rule in@lde/search-indexer’s “Add a transform”.Follow-ups (not done here)
Per the issue: #754 is no longer a precondition, #760 stays latent, and the
multi-valued provenance stamp (#697 / #709) is worth un-deferring since this
makes shared documents common. Left for you to decide on the tracker.
Part of #763 – deliberately not
Fix, so the issue stays open for PR 2.