Feature config

[ascottDI]

di.config — TorQ Modularisation PR

Summary

Extracts TorQ's in-process configuration handling (torq.q: loadf, loadconfig,
loadaddconfig, overrideconfig) into a standalone kdb-x module: di.config. The module
loads a process's q settings files from disk, resolves the override cascade
(default → proctype → procname, framework → app), and exposes the resolved config for
querying. di.torq will use it to partition config per module and inject each slice into
that module's init. It satisfies the di.* contract: a required injected logger, a minimal
exported API, no hard module dependencies, and a full k4unit suite.

Status: complete and tested in isolation, not yet deployable in a live stack. See
Caveats below — di.config needs the (unmerged) di.log logger and the (not-yet-built)
di.torq orchestrator before it runs in a real process.

---

Background

In TorQ, torq.q reads a cascade of settings files at start-up and every module pulls its
own values out of global namespaces via @[value;\var;default]`. Config resolution was
entangled with the monolithic bootstrap.

Per the modularisation plan, config is now a hybrid: di.config handles the mechanics
(load files, resolve the cascade, expose a queryable store); di.torq handles distribution
(partition by module, inject via each module's init). Modules receive a config dictionary
and don't know or care where it came from.

---

Changes

New files

File	Description
di/config/config.q	Implementation — init, loadcascade, overrideconfig, getmodule, and internal helpers loadconfig, applyoverride, raiseerror
di/config/init.q	Module entry point — loads config.q, declares the export
di/config/test.csv	k4unit suite (45 assertions)
di/config/config.md	Module README — API, design notes, and cross-module follow-ups

Differences from the TorQ original

Aspect	TorQ	di.config
Logging	Hard-coded .lg.o/.lg.e	Injected binary log dep, required at init, no fallback and no adaptation
Config values	@[value;\var;default]` per module	Loaded into namespaces; read via getmodule; distributed by di.torq
Cascade order	Dir-major (per the actual torq.q code)	Name-major — specificity wins over directory layer (matches the plan's documented file order)
Env / identity	Reads getenv + process.csv internally	Env-free — caller supplies the settings dirs and the name sequence
Directory loading	loaddir (loads code dirs via order.txt)	Not included — whole-directory/code loading is di.torq's concern, never used by the config cascade
Command-line overrides	.proc.override (reads .z.x)	overrideconfig[params] — caller passes the params; per-variable log-and-skip
Module contract	None	kdb-x use singleton, init[deps], export: list

---

Logging contract (injected dependency)

Logging is an injected dependency, wired via init — there is no default logger and the
module does no adaptation of the logger. init must be called before any other
function.

• init[deps] takes a dict carrying the required log dependency. It errors immediately
  (plain signal, di.config: prefix) if deps is not a dict, is missing `log, log is
  not a dict, or log lacks any of `info`warn`error.
• The log value must already be a binary `info`warn`error!{[c;m]} dict — each
  function takes a context symbol c and a message string m. Build it from di.log (the
  standard logger, which exports binary info/warn/error directly) or hand-roll one:

  logger:use`di.log
  config.init[enlist[`log]!enlist `info`warn`error!(logger.info;logger.warn;logger.error)]

  A logger that is not already binary {[c;m]} (e.g. a raw monadic kx.log instance) must be
  wrapped by the caller first — di.config does not wrap it. (This is the settled no-normlog
  direction: adaptation belongs in di.log, not in every consumer.)
• init fans the injected log dict out into three module-local functions —
  .z.m.loginfo/.z.m.logwarn/.z.m.logerr — and every call site uses
  .z.m.loginfo[`ctx;"msg"] (the three-flat-var convention from consistency.md and
  di.compression; maps 1:1 with TorQ's .lg.o[`ctx;"msg"]).
• Errors route through an internal raiseerror[ctx;msg] that logs via .z.m.logerr
  then signals '"di.config: ",ctx,": ",msg — so every failure is observable in the log as
  well as thrown.

---

Exported API

config:use`di.config

config.init[deps]                      / wire the required log dependency (call first)
config.loadcascade[dirs;names]         / load+resolve the cascade over dirs x names (name-major)
config.overrideconfig[params]          / command-line layer: di.torq parses .Q.opt .z.x and applies it after the cascade
config.getmodule[namespace]            / a whole namespace's config as a dict; index it inline for one value

• dirs — a settings-directory path or list of paths (low→high priority).
• names — a config name or list (least→most specific), e.g. `default`proctype`procname.
• namespace — a bare symbol, no leading dot (getmodule[\rdb]`); index the returned dict for one value.

Typical caller (di.torq): config.loadcascade[(kdbconfig;appconfig);\default,proctype,procname]→config.overrideconfig[cliparams]→config.getmodule[`rdb]→rdb.init[…]`.

---

Key design decisions

• Name-major cascade. For each name (least→most specific) load dir/{name}.q from each
  directory in turn, so a more specific name wins over the directory layer, and within a name the
  higher-priority directory overrides. This matches the plan's documented file order and is a
  deliberate departure from the actual torq.q code (which is dir-major).
• Config store = live namespace globals (1a). Settings files assign globals when executed, so
  those namespaces are the store; getmodule reads them (and reflects overrideconfig
  changes). An explicit store with provenance / cross-source precedence (1b) is deferred until a
  second config source (env vars, k8s) exists — the getmodule contract is
  storage-independent, so that swap won't touch consumers. Flagged as an EXTENSION POINT in code.
• Env-free by design. di.config reads no environment variables or process identity — di.torq
  resolves KDBCONFIG/KDBAPPCONFIG → dirs and proctype/procname → names and passes them
  explicitly. Deliberately no zero-arg load[] (config loads once; a blank call would only add
  hidden state and temporal coupling).
• overrideconfig is the command-line layer, not a manual step. It ports TorQ's
  overrideconfig/override[] (torq.q), which TorQ applies at startup off .Q.opt .z.x so
  launch-time flags (-loglevel, -tablelist, …) beat the settings files. di.torq owns the
  command-line parse and calls this after loadcascade; di.config just does the type-aware apply
  (staying env-free). It improves on TorQ on error handling: per-variable problems (undefined
  name, non-basic type, value that won't parse) are logged and skipped — a single bad override
  never aborts the batch and a null is never written into config.
• Minimal API. loaddir was dropped (code-directory loading, not config). loadfile was merged
  into the internal loadconfig (its string-check was dead and its existence check duplicated once
  the loader was internal-only). A single-key get[namespace;key;default] getter was also dropped —
  getmodule returns the slice and callers index it inline, and every config boots with a default so
  a fallback arg was dead weight (a missing key is now a caller bug, surfaced as a q null). Public
  surface is 4 functions; loadconfig/applyoverride/raiseerror stay internal.
• KDB-X reserved-word workaround (an 'assign trap not listed in .Q.res): the cascade driver
  is loadcascade (not load, which is reserved and throws 'assign at parse time).

---

Test coverage — test.csv (k4unit), 45 assertions, all green

Area	Coverage
init — dependency validation	rejects non-dict deps, missing log, non-dict log, log dict missing a key
loadcascade	dir-name grid loaded; name-major precedence (specific name beats dir layer); returned paths include skipped/missing; missing cascade file skipped at info; dedup on re-run; single-string/single-symbol accepted; type-error rejection
overrideconfig	bool/long/symbol-list parsed from strings and set; returns the vars overridden; undefined / non-basic-type / unparseable logged and skipped; empty params no-op; non-dict and non-symbol-key rejection
getmodule	whole-namespace dict; slice carries the values; empty dict for unknown namespace; reflects overrideconfig changes; non-symbol-namespace rejection
logger emission	a real logger (kx.log wrapped caller-side into the binary contract) receives di.config's init/cascade/load messages via a file sink — proves the injected-logger path end to end

Running the tests

export QPATH=/path/to/kx/mod:/path/to/kdbx-modules

k4unit:use`di.k4unit;
k4unit.moduletest`di.config;   / prints the results table; "All tests passed" on success

---

Caveats — not deployable in a live process until two things land

di.config is complete and tested as a standalone unit, and its logging path was verified
end-to-end against the real di.log with no code change. It cannot yet run inside a live
process because:

1. di.log (the standard logger) is not merged and not final. di.config requires a conforming
   binary `info`warn`error logger injected at init; di.log provides exactly that (verified),
   but it currently lives on the feature-logging branch and its contract isn't approved. If that
   contract changes, re-verify — no di.config code change is expected (di.config only needs the
   three binary functions). Interim callers can hand-roll a logger.
2. di.torq does not exist yet. The orchestrator that reads process identity, resolves the
   config dirs, calls loadcascade, applies overrideconfig, and distributes config via getmodule
   hasn't been built. getmodule is unit-tested for that role, but the real partition-and-inject flow
   is unproven until di.torq lands.

Deferred follow-ups (documented in config.md)

• Add a committed di.log integration test once di.log merges (verified manually for now; a
  use`di.log test can't be committed on this branch yet).
• Add a version export + deps.q when di.depcheck ships (repo-wide versioning rollout — no
  module carries these yet).
• Implement the 1b explicit config store (provenance / cross-source precedence) if/when a second
  config source (env vars, k8s) is added — behind the unchanged getmodule contract.

Test results screenshot

[DI-Software-Engineering]

DIReview Summary

2 critical | 4 warning(s) | 0 suggestion(s)

⚠️ Spec check skipped — tracker lookup failed (NO_REF_FOUND). Standards axis only.

[DI-Software-Engineering]

DIReview Summary

1 critical | 3 warning(s) | 0 suggestion(s)

⚠️ Spec check skipped — tracker lookup failed (NO_REF_FOUND). Standards axis only.

[DI-Software-Engineering]

DIReview Summary

0 critical | 1 warning(s) | 0 suggestion(s)

⚠️ Spec check skipped — tracker lookup failed (NO_REF_FOUND). Standards axis only.

[DI-Software-Engineering]

The /-comment filter lines like "/*" matches any line whose first character is /, but the glob pattern "/*" means 'slash followed by zero or more characters' — in q, * in a like pattern is a wildcard, so this correctly matches lines starting with /. However, the check uses not lines like "/*" which also drops lines whose first character is a forward-slash that is part of a valid value (e.g. a path string /opt/foo on the RHS of a name:value pair). More importantly, the filter is applied to the whole raw line, not to lines that lack a :, so a value line like path:/some/dir is kept (correctly), but a line like / comment with : colon would also be dropped only if it starts with / — that part is fine. The real bug is that blank-line and comment-line filtering happens BEFORE the split, but the split lambda {[ln] i:first where ln=":"; ...} will signal with a null i (and thus 0N#ln) on any line that survived filtering but contains no :. If a settings file has a line that is neither blank, nor a comment, nor a name:value pair (e.g. a continuation, a stray word, or a line whose only : is inside a string), first where ln=":" returns 0N, and 0N#ln / (0N+1)_ln will error or return unexpected results. Add a guard: lines:lines where ":" in' lines after the comment filter to skip non-pair lines safely.

[DI-Software-Engineering]

cascade builds names as `default,proctype,procname — if proctype and procname happen to be the same symbol (e.g. both `rdb), two identical tier bases are generated and parsetier is called twice on the same files, causing those files to be parsed and merged twice. The later (duplicate) merge overwrites the earlier one's keys with identical values, so the result is technically correct, but parsefile is called redundantly. More critically, if proctype~default(a caller passes the literal name ``default `` as proctype), the default tier is processed twice with the same precedence slot, which is misleading and could produce unexpected results if `parsetier` or `parsefile` has side-effects in future. Deduplicate `names` after construction: `names:distinct names`.

[DI-Software-Engineering]

dirs,\:"/",string nm concatenates each root with "/" and the string form of nm (e.g. "rdb"), but string on a symbol strips any leading backtick and returns the bare name. However if proctype or procname is the null symbol `, string[]returns"", producing paths like "/tmp/builtin/"— a directory, not a settings file base.parsefileon"/tmp/builtin/.q"would then attempt to read a file named literally.q(or.toml) in the root, which is almost certainly not intended. Validate that proctypeandprocnameare non-null symbols before use, or guard incascadewithif[(null proctype)|null procname; ...]`.

[DI-Software-Engineering]

In the overrideconfig fold :{[config;params;name] res:applyoverride[...]; $[res 0; @[config;name;:;res 1]; config]}[;params;]/[config;defined], the fold is written as f/[seed;list] but the lambda is a 3-argument function projected as f[;params;] — making it effectively binary when partially applied. The seed (config) is the accumulator and defined is the list. This is the correct q f/[seed;list] idiom. However, applyoverride is called as applyoverride[name; config name; params name] where config is the current accumulator from the fold, which is correct. The real risk is that defined is derived from the original config (vars where vars in key config), but if an earlier iteration of the fold changes the keys of config (which @[config;name;:;res 1] does not — it only updates a value for an existing key), this is safe. Confirmed no key addition occurs. No bug here — this finding is withdrawn.

[DI-Software-Engineering]

The kx.log emission test seeds the config dict inline: enlist[kxrows]!enlist 5(the first argument tooverrideconfig). This dict has a single long value 5for key ``kxrows , and the override applies `enlist"7"` for the same key. The test then asserts `7~kxres`kxrows`. Because `applyoverride` uses `type cur` (which is `-7h` for a long scalar `5`) and `"J"$'enlist"7"` → `enlist 7j`, then `if[t<0; vals:first vals]` reduces to scalar `7`, the assert is correct. However `enlist 5` is a 1-element long list (type `7h`), not a scalar long (type `-7h`): `enlist 5` has type `7h`, so `t:type cur` is `7h` (positive, a list), and `if[t<0; vals:first vals]` is NOT triggered — `vals` stays as `enlist 7j`. The returned dict would have kxrows `` bound to enlist 7j(a 1-item list), so7~kxreskxrows`` would fail (`` 7~enlist 7j `` is 0b). Fix the seed to use a scalar: (enlistkxrows)!enlist 5j should be (enlistkxrows)!enlist[5j]... actually the issue isenlist 5produces a list. Change to(enlistkxrows)!5j (scalar value, no enlist on the value side) so t is -7h.

[DI-Software-Engineering]

DIReview Summary

1 critical | 4 warning(s) | 0 suggestion(s)

⚠️ Spec check skipped — tracker lookup failed (NO_REF_FOUND). Standards axis only.