A self-governing quantitative-research loop: LLM agents propose strategies, a sealed harness evaluates them, and a machine-gated memory guarantees that what accumulates is bounded, honest, and reusable. This repo is a curated extract — the loop machinery in full, plus one complete worked example (an intraday mean-reversion research arc, 11 rounds, from skeleton to vault certification), with the real run logs.
The failure mode of LLM-driven research is not bad ideas — it is unaccountable memory: agents re-deriving dead ends, transcribing numbers wrong, and grading their own homework. Every component here exists to close one of those holes.
┌────────────────────────────────────────────────────────────────────┐
│ ONE RESEARCH ROUND │
│ │
│ hypothesis_synth ──► bounded brief (ledger + dead-ends, ~12KB │
│ │ flat no matter how many rounds ran) │
│ ▼ │
│ PROPOSER AGENT (blind tier-0: sees the brief, not the data) │
│ │ │
│ ▼ │
│ novelty_gate ──► semantic archive: has this idea been tried? │
│ │ (embedding search over every prior proposal) │
│ ▼ │
│ SEALED HARNESS (propose.py: blind proposer ─► AST dedup ─► │
│ │ exprc gate ─► score ─► disclosure) ──► run_<x>.json │
│ │ walk-forward gates, cost model, placebo controls │
│ ▼ │
│ CONTROLLER ADJUDICATION ── re-derives the agent's headline │
│ │ claims on data before adoption │
│ ▼ │
│ postrun process ──► ledger.py GENERALIZER GATE: │
│ │ • lesson ≤ 280 chars (bounded memory by construction) │
│ │ • NO-FABRICATED-DIGITS: every number in the prose must │
│ │ appear in machine-extracted metrics — agents cannot │
│ │ transcribe numbers, only cite them │
│ │ • dead_ends recorded so no future round repeats them │
│ ▼ │
│ postrun check ──► next round BLOCKED until every run is ledgered │
└────────────────────────────────────────────────────────────────────┘
Eleven ledgered rounds, real logs (example/runs/): residual-reversal skeleton →
feature generation → PCA collinearity audit → intraday substrate (volume-clock) →
micro-structure control → execution research (vol-scaled limit grids with a
fill-realism triple gate) → ensemble → one-shot vault certification.
Start with example/FINDINGS.md — the one-page funnel of all 11 rounds with the headline numbers. Then read the raw trail in ledger order (example/runs/LEDGER.jsonl) — each entry is one round's
gated lesson, with the dead-ends that later rounds were banned from repeating. The
point of the example is not the strategy (it is a deliberately vanilla reversion
family); it is the discipline trail: every number in every lesson is traceable
to a machine-extracted metric from a completed run.
| invariant | enforcement |
|---|---|
| No fabricated numbers in memory | ledger.validate_entry digit-guard: any 2+-digit number in prose must appear in machine metrics — hard reject |
| Bounded context growth | 280-char lessons; the synthesizer brief stays ~12KB after any number of rounds |
| No skipped generalization | postrun check exits nonzero while any run is un-ledgered → next round cannot start |
| No lookahead | every trailing gate is past-only; future-perturbation invariance is a self-test, not a review comment |
| No self-graded agents | the controller re-derives headline claims on data; agent numbers are never transcribed into memory |
| Prompt = render of the contract | promptrender builds the proposer prompt FROM the target's column contract + exprc's exported grammar; a bidirectional test (loop/tests/test_promptrender.py) fails if prompt and enforcement diverge |
| Blind cross-round generation | proposer sees prior FORMULAS only (novelty channel); outcomes reach generation only in explicit --mode adaptive, and every scored trial lands in a disclosure ledger whose length is the honest N for selection-corrected inference |
| No fantasy fills | triple gate: touch-fill vs close-fill floor, scrambled-signal placebo, penetration margin |
| Sealed hold-out | the vault window is touched once, at certification, or never |
loop/TASTE_PROTOCOL.md: three proposal contracts (effect-size-first
triage, falsification-first, distribution-not-point) plus a 15-pattern failure library distilled
from this loop's own measured kills — the controller adjudicates every agent claim against it and
must name which patterns it checked. The library grows only from measured failures.
loop/ the machinery: hypothesis_synth, novelty_gate (semcat-partitioned: cosine
compared only within an idea's concept cell), semcat (the object_type x
direction pre-classifier), featgate (per-direction feature-admission ladder:
shape -> anchor-calibrated tail-concordance redundancy -> DEV-internal
increment -> stability), ledger (gated memory), archive (semantic dedup),
postrun (write path + loop gate), propose (sealed harness), devtools
(window discipline), target (the machine strategy-contract + registry),
promptrender (prompt-as-render), costmodel, exprc
example/ the mean-reversion arc: revsig/revcore/revgate/revvault/revexec/revloop,
residual, volclock, ctaclass + per-module tests
example/runs/ 11 real run logs + the filtered LEDGER.jsonl + ARCHIVE.jsonl
docs/ design notes
Register a ResearchTarget (loop/target.py): name, 3-sentence framing, the PIT column
dictionary (role="input"|"outcome"), the reply schema, and an optional horizon range.
The proposer prompt, forbidden-column list, safety gate, and dedup all derive from it —
there is no prompt to write. propose.py --target <name> runs the loop against it.
Python 3.11+, numpy pandas scipy. Market data is expected at $MARKET_DATA_1M
(1-minute OHLCV parquet per symbol); the run logs are readable without any data.
# the memory gate, live:
python loop/ledger.py render # the bounded brief
python loop/postrun.py check # loop gate (CLEAN or blocking list)
# module self-tests (each refuses to emit numbers until its self-test passes):
python example/revsig.py