Add @lde/resolver: resolve external references with bounds, persistence and isolation

[ddeboer]

Problem Statement

Enrichment that draws on data outside the graph – an HTTP service, a store, a file –
already has a supported place to run. A QuadTransform attached to a reader sees the
extraction CONSTRUCT’s output for a batch of roots, can tell an absent value from a
present one, and may mint IR Aliases through the exported irAlias(); the prohibition in
CONTEXT.md is on hand-written alias strings that can drift, not on the function. #675
settled that, and closed for it. Contributing before projection is an advantage rather
than a workaround: the enriched value is shaped by the schema like any other, instead of
the enricher re-implementing physicalFields() and its per-locale display, search and
sort variants.

What has no home is everything around the fetch. Every deployment that writes such a
transform re-solves the same five problems:

• A durable place for a resolved fact, keyed on something stable, that survives an
  index rebuild. The indexer reads publisher RDF live per dataset (or via an ephemeral
  per-dataset QLever),
  ADR 16
  rules out an LDE-owned store, and the search index is rebuilt from its sources – so a
  fact fetched from outside has nowhere to live between runs.
• Outbound work bounded, globally and per host. The host-limited concurrent map is
  still private to distribution-probe (Share the host-limited concurrent map #688).
• Backoff and retry.
• Per-item failure isolation, so one bad reference degrades one record instead of
  aborting a run, and the run is told which degraded and why.
• A cap on how much a single run may fetch, so a cold store does not hand an upstream
  service its whole backlog at once.

That is complexity pushed up into every consumer. #675 closed for the right reason – a
slot absorbs nothing. A library does.

Solution

A deep module – @lde/resolver – with a narrow interface:

createResolver({ key, fetch, store, limits, report })
  .resolveAll(refs: readonly Reference[]): Promise<Map<Key, Result<Fact>>>

The caller selects what needs resolving and applies what comes back. The resolver knows
only references, keys and facts – no item generic, no select, no apply, and therefore
nothing about quads, documents, stages or pipelines. That matters concretely: inside a
QuadTransform, “applying” means minting quads, which no apply: (item, facts) => item
can express.

Inside, and never surfaced unless overridden:

• durable fact store keyed by key, behind an interface (Move provenance RDF/vocabulary to the consumer; keep ProcessingRecord + ProvenanceStore as the TS seam #473 keeps ProvenanceStore as
  a TS seam with the vocabulary moved to the consumer – same shape here)
• bounded concurrency, global and per host, promoting the map out of distribution-probe
  (Share the host-limited concurrent map #688). Worth noting that libraries often get this wrong and cannot be relied on: the
  IIIF consumer’s client throttles on a single global counter borrowed from an unrelated
  config key (ImageServiceLoader: verificationsRequired doubles as the fetch concurrency cap IIIF-Commons/iiif-helpers#39), so the resolver has to own bounding rather
  than delegate it.
• a per-run volume cap, so a cold or purged store warms over successive runs
• backoff and retry
• per-key failure isolation, surfaced as Result rather than a throw
• failure reporting through an injected callback

No dependency on @lde/pipeline. Testable without constructing a pipeline and usable
by anything doing bounded outbound work. The two things that pull toward coupling are both
injectable: reporting is a callback rather than a ProgressReporter import, and the batch
memory bound is satisfied by the caller passing one batch at a time.

Because it is standalone, a deployment calls it from inside its own QuadTransform.
No pipeline adapter, no second extension point, no ADR amendment – the transform is the
seam, and this is a library the transform calls.

One constraint on #688 that follows: the concurrency map must land in a leaf package,
not in @lde/pipeline. Promoting it into the pipeline would give the resolver a pipeline
dependency through the back door and lose the property above.

Naming

Transforms rewrite what the graph said; resolvers fetch what it only pointed at.
That rule distinguishes this from ADR 2’s quad transforms without needing to read an
ADR, and it is why this is not called “enrichment” – which says only that something
got better, not where the data came from.

User Stories

1. As a Deployment, I want to fill fields from an external service without re-solving
   storage, bounding, retry and isolation each time.
2. As a Deployment, I want resolved facts to survive an index rebuild, so that a
   rebuild does not re-fetch every external reference.
3. As a Deployment, I want a failing resolution to degrade one reference rather than
   abort the run, and to be told which degraded and why.
4. As a Deployment, I want outbound work bounded globally and per host, so that one
   slow host cannot starve the run and no host is hammered.
5. As a Deployment, I want a cold store to warm over several runs rather than in one
   burst against someone else’s infrastructure.
6. As a maintainer, I want the resolver testable without a pipeline, so its edge cases
   are covered by unit tests rather than integration runs.

Open questions

• Invalidation. The store is the only place the fact lives, so a wrong fact is
  durable. Neither known consumer has a validator to revalidate against – the Network of
  Terms is GraphQL, and IIIF info.json responses are inconsistently conditional – so v1
  should expose purge-by-key plus a TTL and let definitions layer policy. Modelling
  ETag/Last-Modified revalidation, and multiple key spaces for facts of differing
  volatility, should wait for a consumer that actually has validators.
• Store backend. FileProvenanceStore (Ship a file-backed ProvenanceStore in @lde/pipeline #634) is the precedent for the seam, but not
  for the default: it reads the whole file into memory on every access and is
  single-writer, which suits one record per dataset and not one per external identifier
  accumulating across every dataset and run. Node 24 ships node:sqlite, so a SQLite
  backend costs no dependency and gives keyed reads, a TTL query and atomic writes.
  Proposed: interface as the deliverable, SQLite as the default, in-memory for tests.

First consumer

netwerk-digitaal-erfgoed/stack#1 – @ndes/iiif-images, resolving IIIF image
dimensions and sizes ladders for the search index. It exercises every part: one
manifest fetch per record, info.json per image service, per-host bounds against
publisher infrastructure, facts that must outlive rebuilds, and records that must
still index when a fetch fails.

Second consumer

https://codeberg.org/limburg/lol/issues/110 – Linked Open Limburg, resolving GeoNames
coordinates for places through the Network of Terms. It pulls the design in usefully
different directions from the first:

• One host, batched. lookup(uris: […]) takes a list, so a batch of roots is a single
  request and the stage’s maxConcurrency already bounds what is in flight. Per-host
  limiting buys it little – and cannot see the host that matters, since the Network of
  Terms fans out to NDE’s GeoNames store behind the API. What protects that store is the
  volume cap, not per-host concurrency.
• No validators. GraphQL, so nothing to revalidate against, and a place’s coordinates
  do not drift. Purge-by-key plus a TTL is sufficient – evidence for deferring validator
  support rather than designing it now.
• A register-wide store. The key is the canonical term IRI, so entries accumulate
  across every dataset and every run; shared places are the point, and a newly selected
  dataset referencing a place already resolved must cost no outbound call. This is the
  consumer that makes a whole-file-in-memory default the wrong one.

Between them the two consumers cover many hosts vs one, per-record vs per-term, validators
vs none, and dataset-scoped vs register-wide storage – which is a better basis for the
interface than either alone.