Multimode batched evaluation of factorized CC (cost model + placement + dry-run) - #583
Open
evaleev wants to merge 120 commits into
Open
Multimode batched evaluation of factorized CC (cost model + placement + dry-run)#583evaleev wants to merge 120 commits into
evaleev wants to merge 120 commits into
Conversation
CostProfile (peak/flops/exec) over the factorized IR via a zero-data dry-run evaluation, driving the batched cost model's predictions.
Perf-first (DenseTimeSpace) objective with peak_threshold as a ceiling; role-split (contracted/external) batchability; order-aware placement over the combined nest; per-node batch annotations consumed by the evaluator.
External-mode scatter + contracted accumulate; cache scope chain with fall-through; slice-on-use; per-level placement driven by a per-canonical lifetime mask (cross-occurrence meet) unioned with contracted residency; iterative (stack-safe) tree traversal.
Index-space occupancy predicates robust to non-physical spaces; logger; is_valid accepts Power; convention.
The multimode-batched-eval tests had only run under Release/IGNORE, so several Debug-only asserts (and one ASan bug) were masked. Fix them so the suite is green under Debug (SEQUANT_ASSERT_BEHAVIOR=ABORT + AddressSanitizer): - Covariant tensor forms in the synthetic batched-eval tests: contracted indices had been placed in the same bra/ket slot, tripping create_graph's strict-braket invariant. Reorient contractions Einstein-properly and move Hadamard/external indices to the aux slot (test_lifetime_mask, test_eval_ta, test_eval_dryrun). Physical-tensor tests were unaffected (Symm braket). - test_eval_dryrun: a rank-4 CSV composite used a duplicate proto index; use a distinct fourth occ index. - test_eval_ta: rand_tensor_yield now sizes the m (mu~) space; and copy the compared tiles by value in shape_spike_ToT_inner_contraction_to_flat_T (it bound a reference into a temporary Future -> ASan stack-use-after-scope). - test_cache_manager: the batch-axis veto is phase-2 -- a node carrying a batch mode free on its own result is batch-variant (External or Contracted) and correctly refused run-scope caching; update the stale case-3 expectation. - cost_model: seeded_root_peak_batched must admit the seed via the external-role predicate too. The seed is an external mode, so build_context's role filter gates it through is_batchable_external_index; overriding only the contracted-role predicate dropped the seed and tripped the k_seed assert. - Hide ([.]) the all-C60-terms perf-first cost diagnostic: it runs optimize() on every summand (tens of minutes in Debug) and has no correctness checks. - Add a hidden ([.]) water-20 occ-batching overcompute dry-run diagnostic.
order_aware_recompute=false selects the set-keyed DP that ignores the per-batch-block replay recompute, which under-costs every batched schedule and is never the more realistic default. Default it to true in BatchPolicy and CostParams (MPQC and other callers inherit it via optimize()). Pin it false in the two cases that specifically characterize the legacy set-keyed behavior: reconstruct_batched_modes_emits_external_per_node (its own comment documents the order-aware-off emit_external regime) and the C60 objective-determines-factorization case (peak-first forms the fully-sliceable 4-PAO only under the set-keyed peak model; under the realistic resident-scan model the contrast collapses -- peak-first also avoids it and perf-first flops then exceed peak-first because the recompute is charged).
OptimizeOptions::inner_pow and PeakBatchedModel::inner_pow already have no
default (empty + composite indices -> inner_aware_volume throws, so the old
silent mis-sizing fallback cannot recur). But 9 OptimizeOptions{...} designated
initializers omitted inner_pow, which g++ -Wextra -Werror flags as
missing-field-initializers (clang does not, so macOS CI and local clang builds
missed it) -- breaking every Linux Debug job.
Add an explicit .inner_pow = {} (composite-free no-op) at all 9 sites
(optimize.cpp compatibility_opts + 8 in test_optimize). Also finish the
removal that had missed CostParams::inner_pow: drop its stale = {} default and
its stale "sized by idxsz (k=1)" fallback comment so all three inner_pow fields
are uniformly no-default (all 21 CostParams{...} sites already set it).
The static per-node walk in cost_profile() prices each node once, so its
flops/exec are order- and batching-blind and never reflect the per-occ-block
REPLAY recompute the batched evaluator does at runtime -- the reason a dry-run
could not predict occ-batching being slower than aux-only.
Split the reported cost:
- Rename CostProfile::{flops,exec_cost,n_ops} -> model_{flops,exec,n_ops} (the
static DP-model quantities, unchanged).
- Add dryrun_{flops,exec,n_ops}, tallied from the existing Trace::On replay: an
optional CostSink is attached to the shared dry-run CostModel, and every
actual product-op execution (DryRunOps::prod) folds its own SLICED-extent
flops/exec (the same numbers already computed for the per-op OpCost log) into
it. Because a sliced occ-dependent op run N times does ~1/N work per pass, its
sliced-cost sum is work-neutral; only occ-INDEPENDENT work re-executed at full
size once per block inflates -- so dryrun_* isolates exactly the recompute.
The sink is opt-in (nullptr default) and lives on the dry-run CostModel, which
is constructed only inside cost_profile(); eval.hpp / make_evaluator are
untouched, so mpqc's production evaluator path is byte-identical.
water-20 [.][dryrun-water20-overcompute], heavy occ batching: model_flops
ratio 1.0 (flat), dryrun_flops/exec ratio ~1.98, dryrun_n_ops ~55x -- the
recompute is now visible where the model walk saw nothing.
Extend [.][dryrun-water20-overcompute] to three configs -- aux-only, occ+aux
order_aware=false (MPQC production / root-level forest seed), occ+aux
order_aware=true (node-level placement) -- and report the recompute-aware
dryrun_{exec,n_ops} per config plus the OA-true-vs-false ratio.
Diagnoses the water-20 order_aware=true slowdown: under occ batching,
order_aware=true does ~2x the dryrun_exec and ~12x the op-executions of
order_aware=false, and (via its resident-scan peak model reporting higher
peaks) also tips more terms over peak_threshold so occ batching engages where
order_aware=false leaves them un-batched -- a double hit that matches the
observed runtime regression.
Flipping the default to true (previous commit on this branch) turned on the order-aware cost model AND, together with batch_spectator_indices, node-level external placement. On water-20 that path is a runtime regression -- the water-20 dryrun diagnostic measures ~2x the traffic and ~12x the op-executions of the root-level forest seed for the same term -- and, worse, was reported on Owl (job 649250) to produce a WRONG schedule: incorrect PNO-CCSD iteration energies and malformed eval ops. Restore the known-good default (false = legacy set-keyed DP + root-level forest seed) until node-level placement is root-caused and fixed. The two tests that pin order_aware off explicitly keep passing (the pin now merely matches the default).
order_aware_recompute conflated two orthogonal concerns: the order-aware recompute COST MODEL (which factorization the DP selects) and the node-level external-mode EMISSION placement (per-node External stamps vs the root-level forest seed). node_level_placement was defined as order_aware_recompute && batch_spectator_indices, so enabling the more-realistic cost model forced the node-level emission along with it. A water-8 A/B (holding the cost model fixed, toggling only emission) shows the regression is the EMISSION, not the cost model: node-level placement runs ~6x slower (~124 vs ~20 s/iter) and emits ~8x more batch scopes than the root-level seed, because it nests a batch scope at every carrying node and the batched evaluator replays each. The order-aware cost model with root-seed emission is correct and cheap. Node-level placement also produces a wrong residual on water-20 (size-dependent; not reproduced at water-8/he10). Split node_level_placement into its own BatchPolicy/CostParams/CostModel flag (default false), threaded alongside order_aware_recompute. order_aware_recompute now drives only selection; node_level_placement drives only emission. Both default false, so this is behavior-neutral. A gated SEQUANT_NODE_LEVEL_PLACEMENT env knob forces the placement for A/B diagnostics without recompiling a flag through the caller. Tests that engaged node-level placement via order_aware_recompute=true now set node_level_placement=true explicitly; the node-level correctness sweep now drives the emission via its sweep variable.
Now that node-level emission is separately gated by node_level_placement (default off), the order-aware recompute cost model is selection-only and safe to default on: it charges recompute realistically and picks better-batching factorizations, while emission stays the correct, cheap root-level forest seed. Verified no churn across the [optimize] suite (628 assertions) and the batched emission tests. The internal CostModel/oracle-helper member defaults stay false (documented seeded-probe reason); the public BatchPolicy/CostParams path threads the true default through.
…le recompute
Add a batched-evaluation schedule-visualizer pipeline and make avoidable
recompute a first-class per-node output of cost_profile().
- schedule_dump.hpp (new): per-term IR schedule-record emitter
(schedule_ir_json) and a shared cost_op_signature() join key (result index
labels + the sorted operand pair). One definition, used by three producers
so a DAG node, its runtime Build event, and cost_profile's per-node number
all carry the identical key -- no signature is reconstructed downstream.
- eval.hpp: runtime schedule-dump hooks (SCHEDULE_RUN_EVENT / RUN_GROUP),
each internal Build event stamped with the same signature and per-loop
dependent-mode flags, all gated by SEQUANT_SCHED_DUMP (production path
byte-identical when unset).
- CostProfile gains per-node avoidable_nodes {label,count,exec,flops} plus
avoidable_exec / avoidable_ops and avoidable_time(). DryRunOps::prod tallies
each build's necessary = product of block counts of its touched (dependent)
modes, so builds - necessary is the avoidable recompute (a node rebuilt once
per block of a mode it does not touch). The cost model is dense, so necessary
is exact and no empirical correction is needed. avoidable_nodes_from_sink()
is the shared rollup, reused by the schedule-dump test to emit these numbers.
- Consolidate the two dryrun avoidable witnesses (occ-veto, extmode) onto
cost_profile's structural per-node rollup, replacing the BatchGroup/BatchIter
trace-parse string-match reconstruction with a read of cp.avoidable_* (the
structural signature is relabeling-proof; the trace is still parsed only for
the scatter/group Begin markers cost_profile does not expose).
…uckets The per-node avoidable-recompute rollup keyed only by the label signature (result+operand indices), so a node built at many slice sizes landed in one bucket. Because the exec model is roofline-like (nonlinear in slice size), those builds span orders of magnitude (a 257x spread on the C60 external-occ arm); pricing avoidable as count * a single per-build exec then exceeded the whole replay's exec -- the impossible avoidable_time > 100%. Fix: key the sink by the EXTENDED signature (label + the touched modes' realized extents), so every build in a bucket ran at the same slice-context and hence the same roofline cost. Within a cost-homogeneous bucket avoidable_exec = total_exec * (builds - necessary)/builds is exact; the buckets of one DAG node aggregate back to a per-label AvoidableNode (what the visualizer joins on by hash->sig). Verified: buckets are homogeneous (count*last == total*frac per bucket), all arms bounded <= 100% (C60 external-occ 390% -> 0.005%), and the giant term reads 77%, matching an independent dependent-mode analysis (78%). NodeCost now accumulates total_exec/total_flops; min/max/last are kept only for the SEQUANT_AVOIDABLE_DEBUG homogeneity dump.
Redefine the per-value avoidable-recompute metric: it is now measured in FLOPs
against the batching-free (unlimited-memory) ideal -- the arithmetic the batched
replay repeats beyond building each value once at full extent -- rather than in
roofline exec against a within-scheme "necessary" reference.
Two problems with the exec-weighted metric drove this:
- roofline exec is nonlinear in slice size, so one value's differently-sized
builds spanned ~257x; pricing avoidable as count x a single per-build exec
exceeded the whole replay's exec (avoidable_time > 100%). The prior
cost-homogeneous slice-context bucketing removed the >100% but stayed
exec-weighted;
- referencing "necessary = distinct slices the scheme produces" is circular --
it scores an un-hoisted value's per-block rebuilds as necessary, so it cannot
see the recompute hoisting exists to avoid.
FLOPs is linear in extents, hence additive across slices: disjoint per-block
slices that tile a value sum to exactly full_flops (0 avoidable), while a value
rebuilt full per block sums to N*full ((N-1)*full avoidable). So avoidable =
max(0, total_flops - full_flops) per value, bounded in [0, dryrun_flops] by
construction, needs no slice-context bucketing, and answers "what does batching
cost vs. infinite memory". NodeCost drops to {builds, total_flops, full_flops};
CostProfile.avoidable_exec -> avoidable_flops, avoidable_time() = avoidable_flops
/ dryrun_flops.
Witnesses re-baselined (nterms=55, FLOPs): occ-veto 1.8/6.5/15.4%; the extmode
witness shows external-occ (~1.95%) and contracted-occ (~1.97%) essentially
equal -- external-mode batching is NOT a recompute fix on the C60 forest (its
original conclusion, now with honest magnitudes). The [cost_profile] giant reads
77.8%, matching an independent dependent-mode analysis (78%).
The batch-variant caching veto in cache_manager had two disjuncts: (a) a node whose own batched_here() carries a Contracted, batchable mode FREE in its own result, and (b) a non-empty cross-occurrence lifetime mask. Disjunct (a) is structurally dead: a Contracted mode is summed AT the node, so it can never be free in that node's result, and post-role-split a free index is stamped External, never Contracted -- so the condition never holds (the occ-veto test's [veto-reach] probe read 0, structurally, not by forest accident). It only ever guarded a malformed emission. Remove disjunct (a) and the `is_batchable_contracted_index` parameter from the cache_manager factory (the only functional caller, build_dryrun_cache, drops the arg; no production caller passed it), the `CacheConfig::is_batchable_index` field and the cost_profile() overwrite that fed it, and the now-obsolete [veto-reach]/[veto-hazard] probes plus the disjunct-(a) sub-test. Disjunct (b) (the cross-occurrence lifetime-mask veto -- the load-bearing F1 correctness guard) is unchanged. Behavior-preserving: [cache_manager] (200), [lifetime_mask] (76), [dryrun], and [eval] (TA production path, 457) all green. Does NOT touch BatchPolicy::is_batchable_contracted_index (the batching decision) or BatchPolicy::is_batchable_index() (the eval accept union) -- both stay.
Design note reframing batched-eval cache placement as register allocation. Three identities -- value (hash), instance (use-site), cell (a materialized copy serving a subset of instances). A value's instances partition into cells; perfect CSE = one cell/value, no CSE = one cell/instance, and the peak budget chooses the granularity in between (a partial un-CSE / materialization DAG). Cell identity = (value, home-scope, split-index): home-scope is the loop level (the axis batching adds), split-index names a same-scope peak split (the RA live-range-split rename). Placement is register allocation + loop-invariant code motion + rematerialization: hoisting a shared value lengthens its live range (peak) to save recompute; slicing adds partial-hoist granularity. Objective: minimize recompute (the rational, batching-aware reuse count W-1 times build cost) subject to the whole-forest peak profile <= peak_threshold. Peak is a placement (post-CSE, whole-forest) constraint, not a factorizer one; cost_profile()'s replay peak is the detection safety net, and a peak that survives full splitting is factorization-inherent. Includes a prior-art section (rematerialization/checkpointing -- Checkmate; electronic-structure space-time tradeoff -- Cociorva/Sadayappan PLDI 2002; register allocation; pebble games), four worked cases, and open items (group-scoped cache keying, the greedy split move, per-placement footprint, W's fixed point).
O1 (cell keying) resolved as a router + dumb stores, not a wider cache key. Add
§7a "Runtime realization": one value-keyed store per (home-scope, split-index)
-- the cache stays TreeNode-keyed unchanged -- plus an explicit router
{value, use-site} -> (home-scope, split-index) that is the placement pass's
output and replaces the implicit parent_ fall-through search. Reads route via
the map then reuse the EXISTING Enter-stage slicer, (use-scope - home-scope)
INTERSECT carried(N), fed the home scope directly instead of via hops; default
{value} -> (home, 0) is byte-identical. Standardize terminology on "home scope"
(= the code's "lifetime scope" = store scope; consumer's is "use scope"). Update
§4, §9, and O1 accordingly; residual O1 sub-items are the use-site/occurrence id,
the parent_/hops audit, and the naming standardization.
Add §7b: the placement pass as a register-allocation spill loop. Seed = perfect CSE (recompute-minimal, peak-maximal); walk up the recompute axis to walk down peak until peak <= threshold. Objective and constraint are exactly cost_profile()'s avoidable_flops and peak_bytes -- no new measurement. Moves: SHRINK (slice a carried mode a cell holds full -- the existing external-slice / node_level_placement, now driven off the true whole-forest peak) and EVICT (delay/un-hoist an invariant cell held idle, or split a long-lived cell's instances into short-lived groups -- the new CSE-aware move the per-term DP cannot see). Greedy: candidates = cells alive at the binding peak point, prefer free shrinks then max ΔPeak/ΔRecompute (the spill metric), apply, incrementally re-cost, repeat; terminate on fit or on a factorization-inherent peak. Residual sub-items O2a (incremental profile update), O2b (per-move estimator/lookahead), O2c (subsume vs run-after the DP external-slice pass).
Clarify §7b: O2 runs after the per-term min-time factorizer and takes the factorization AND batch-loop assignments (batched_here) as FIXED, deciding only the whole-forest eval/placement strategy (home-scope + router); it never adds, removes, or re-assigns a batch loop. Reframe "shrink" from "slice a carried mode" to "re-home a cell into an EXISTING carried loop" -- a placement choice on the fixed nest, not a batching change; deciding to batch an un-batched mode (adding a loop) is the factorizer's lever. Split the termination boundary into two non-O2 failure modes: factorization-inherent (a single intermediate > budget) vs. re-batch-needed (fixed batching left placement too little room, e.g. a shared cell needing slicing on a mode no single term batched) -- both detected via peak_bytes and fed back, giving the structure factorize+batch -> O2 place -> if infeasible re-batch.
Add §7c. Cell footprint is home-relative: a carried mode is sliced (block extent) iff its fixed batch loop encloses the cell's home, else held full -- the existing moment-aware memsize with home-relative extent overrides, so O2's shrink ΔPeak is just the footprint delta. The peak profile is max weighted-interval overlap: each cell is a [first-use, last-use] interval (from the router's use-sites + the static schedule order) weighted by footprint; peak = max over static points of the sum of live cells' footprints (a sweep line), and the argmax is O2's binding peak point. Because it SUMS co-resident live cells it corrects today's peak_bytes = max(scratch, cache) under-count (a lower bound per §1); the replay stays the oracle (must sum, not max, co-residency). The weighted-interval form updates incrementally under an O2 move (feeds O2a). Residual O3a-c: the sweep structure, the summed- co-residency replay oracle, composite/proto sizing.
Add §7d. Define home_scope(value) = deepest scope enclosing the loops of (sliced_modes ∪ demoted_external_modes). sliced_modes is the cross-occurrence meet (max-reuse upper bound); the demotion fold adds the External batched_here stamps the meet demoted (has_demoted_external) -- occurrences bind them to incompatible blocks, so the value can't be a single full value above those loops and its home must be inside them. The fold is exactly what unifies the current cache-veto-vs-has_demoted_external disagreement into one authority both the cache and the runtime read. Per-block is temporal (one external-loop-homed cell re-instantiated per iteration), so no split-index -- that stays reserved for O2's peak-driven same-scope splits. Structural and computed from the meet before O2, which only lowers homes further for peak; consistent with W (the demoted mode is free tiling). Residual O5a-b: confirm the exact signal / edge cases and the seed router construction. Also tie O6 to §7b's two failure modes.
O4 (W's computation order) is not a fixed point: W is a function of the current placement, well-defined at the home_scope seed and re-costed incrementally per O2 move -- seed-then-refine, subsumed by §7b/§7d. O6 (feedback) scoped to a minimal detect-and-report step (surface the binding cell + failure mode so a schedule fails loudly, not silent OOM), with the re-batch/re-factorize hint as a follow-on that the detect step precedes. All major open items (O1-O6) now designed or resolved; the spec is design-complete.
Phased plan for the placement-as-register-allocation design. Phase 1 (detailed, bite-sized TDD) corrects cost_profile()'s peak_bytes from max(scratch, cache) hwmarks to the instant-resolved co-resident SUM across the scope chain (spec 7c/O3b) -- adds CacheManager current_residency()/chain_residency(), threads the chain sum into note_working_set, simplifies the fold, and re-baselines the documented-RED peak figures from measurement. Phases 2-5 (router+home_scope seed, static peak sweep, the O2 greedy, feedback) are a roadmap, each a future plan. Global constraints: no en-dashes, clang-format, byte-identical perfect-CSE default, replay stays the peak oracle.
Move index_position into a new SeQuant/core/eval/slicing_signature.hpp and add slicing_signature (vectorized index_position over a mode set) and signatures_consistent (equal signature across occurrences). This is the single criterion for whether a CSE-folded value's occurrences may share one sliced materialization or must be split. make_batched_scratch's ext_sig_of now calls the helper (behavior-identical). Foundation for the hoist-path split and the router-read guard.
The occurrence key is already DAG-global: canonicalize_slots colors with distinct_named_indices=false and RouterKeyEqual compares only the canonical graph, so two occurrences whose batched slot binds different labels (the g.C legs i_3 vs i_4) already map to the SAME key. Add a regression test pinning this, and fix the occurrence_key.hpp doc comment (it wrongly claimed space+label coloring). No coloring change is needed; the design spec's proposed occurrence -key change is a no-op.
HomeTarget now carries canonical slot_positions (home modes on the value's own slots) plus free_modes (modes it is invariant to but homed within). home_depth resolves slot_positions through the USE occurrence's canon_indices, so one DAG-global overlay places two occurrences whose batched slot binds different labels (the g.C legs i_3 vs i_4) at DIFFERENT depths -- removing the shared-slice collision. remat_to_router computes positions ONCE from the cell's canonical carried frame (stable across occurrences), classifying each home mode as a slot position or a free mode. Full suite green (313458 assertions).
place_at_this_level's collect dedup now, under a LIVE router, refuses to share one hoist materialization across canonically-equal occurrences that are slicing-signature INCONSISTENT over the enclosing batch modes -- they bind different physical labels to the sliced slot (the g.C legs i_3 vs i_4), so each is built PER OCCURRENCE at its own router-resolved depth (via Task 3's home_depth), closing the shared-wrong-slice corruption. Gated on a live router: with no router the meet homes a divergent value FULL at the root where sharing is safe, so every no-router path stays byte-identical (full suite green). Adds a cost_profile test proving the divergent value is built twice under the router.
On a router-directed fetch, SEQUANT_ASSERT that the resolved home scope is consistent with THIS occurrence -- when it resolves to a live loop, that loop's mode must be one the overlay names for f.node (its canon_indices at an overlay slot position, or a declared free_mode). The hoist split already keeps signature-inconsistent occurrences at different scopes, so this holds on every correct schedule; it is a live tripwire (elided in release) for a future home_depth/overlay bug that would otherwise serve a wrong-slice entry keyed by canonical hash. Full suite green (313466 assertions).
A cell whose home slices a mode that occurrences relabel divergently (the g.C legs binding i_3 vs i_4 to the same canonical slot) cannot share one sliced materialization -- the runtime splits it into two co-resident per-occurrence copies. Model that in the peak profile: ValueCell carries divergent_modes (carried union minus intersection across occurrences), and cell_footprint doubles the base footprint when the home slices a divergent mode. This makes remat's shrink pricing reflect the split it will actually pay for.
…indices A DAG value has no intrinsic labels -- only ops bind labels to it, meaningful only within that op. The DryRun per-op flops computation derived the contracted index set from the operands' stored indices_ (the labels their PRODUCER used) instead of THIS op's einsum annotations (lannot/rannot). For any CSE-shared value consumed under a different binding -- e.g. both legs of a (g.C)(g.C) contraction are the same cached value -- the stored indices are the producer's labels, unrelated to how this op binds the value, so contracted unioned modes from different label contexts and inflated the cost. On the C60 residual this made one bogus op read 6.65e16 (120^4*4320*42^3) vs the correct ~1.3e13 (120^3*4320*42^2) -- 80% of the whole dryrun_flops, corrupting the recompute metric. Derive contracted from lannot & rannot so out U contracted == lannot U rannot == the real contraction volume, all in one label context.
Guards the invariant that canonicalization does NOT collapse composite (PNO) proto pairs onto one canonical pair -- a leaf tensor and a binary contraction both bearing composites on distinct occ pairs (a<i_1,i_2>, b<i_2,i_3>) keep all three distinct occ in canon_indices. (This invariant held; the dryrun cost discrepancy was an annotation-space bug, not a canonicalization one.)
Roots the two annotation-space cost bugs in one design flaw -- the DryRun value stores labeled indices_, but a DAG value has no intrinsic labels; only ops bind labels via their annotation. Bug 1 (per-op FLOPs from stored indices) is fixed; bug 2 (slice overrides keyed by producer labels, which also drive mode_batches counts) cannot be spot-fixed. Spec: make the value label-free (positional shape), resolve every per-op quantity positionally from the op annotation; compiler- driven migration by flipping ExtentOverrides to position-keyed. DryRun-only; wet backends unaffected.
- Drop split_index: a cell is (value, home-scope); split cells of one value land at distinct home-scopes, so no separate discriminator. - Rewrite Design section 3 / plan Task 5: the split un-folds into two real same-hash cells; price each cell's recompute by its replication factor (build_cost times the product of block counts of enclosing loops it is homed-within but does not carry), NOT a flat 2x. Canonical case costs (1 + N3) x full V, equal to 2x only at N3 = 1. - State the hard dependency: this pricing needs the DAG-scope home (Task 3), which alone can name a homed-within-but-not-carried level; slot_positions cannot. Task 3 must land before Task 5. - Note the recompute term is report-only under the current pure-DeltaPeak objective; making it a secondary objective is a separate decision. - Add the DAG-scope coordinate caveat: a space-sequence names a nest-tree node only when no level has same-space siblings; our forest-fusion construction guarantees that, but a future branching nest would need a path/instance disambiguator.
A DAG value has no intrinsic labels; only an op's annotation binds labels to it, so a mode's only stable handle across ops is its POSITION. Flip ExtentOverrides from map<Index,size_t> to map<size_t,size_t> (mode position) and resolve every per-op quantity positionally: - memsize resolves positional overrides against its own index list; flops takes an annotation-label-keyed extent map built by prod from each operand's overrides via that operand's annotation. - prod/sum project each operand's positional overrides onto the result annotation (dropping contracted-away modes) before merging; permute remaps across the reorder; slice_mode/mode_batches/write_into_slice/ pre_sized_zeros key by position; cell_footprint maps home modes to positions in carried. Fixes the batched over-sizing where a CSE-shared value bound under different labels (the (g.C)(g.C) legs, i_3 vs i_4) mis-sized the contraction by reading producer labels as if stable across ops. Unbatched avoidable stays 0; the nested occ+aux arm is now correctly sized (total flops and peak both drop). Cost prediction only: the wet backends contract via annotation-driven einsum and are unaffected. Tests updated to positional keys; the split-test assertion is made direction-free (the pre-refactor over-sizing made split.dryrun_flops > baseline pass for the wrong reason). Spec: doc/dev/specs/2026-08-08-dryrun-labelless-value-design.md.
HomeTarget's value-relative coordinate (slot_positions + free_modes + split_index) becomes a label-free DAG-scope: an ordered svector<IndexSpace> naming a batch-loop nest prefix. home_depth(home, ctx, key) resolves each scope space to THIS occurrence's physical batched index of that space (via the occurrence key's get_indices()), maps it to a live-loop depth, and returns the deepest -- so one overlay places the g.C legs (i_3 vs i_4) at different depths per use. remat_to_router builds the DAG-scope from a moved cell's home_modes spaces, ordered by nest position (enclosing_modes), one overlay per value hash. The two eval.hpp home_depth call sites pass the occurrence key; the Enter-stage tripwire assert is rewritten (not weakened) to validate the resolved loop's mode is one of the occurrence's batched indices whose space the DAG-scope names. free_modes is dropped, not folded in: a cell's home_modes are always a subset of its carried modes, so real remat never homed a value in a loop it does not carry; a homed-within-but-invariant level (leg B's i_3) is a nest fact priced as recompute in Task 5, not a home coordinate. Empty-router paths stay byte-identical. Prerequisite for Task 5 (replication-factor split pricing).
Replace the flat 2x pricing fudge (46b495e) with a real two-cell split model. A divergent value remat homes at a sub-scope is un-folded by apply_split into two same-hash, non-divergent ValueCells, one per physical binding of the relabeled mode (leg A -> i_3, leg B -> i_4), each with subset-local carried/home_modes/enclosing/liveness and a fresh value_id. - peak_profile.hpp: retain per-occurrence records (OccurrenceRec on ValueCell, populated by compute_dag_boulevard instead of discarded at grouping) so the split can partition occurrences and re-derive each cell. DELETE cell_footprint's divergent?2*base:base branch: split cells are non-divergent, priced once, and peak co-residency is left to peak_profile_sweep (keyed on value_id, so two cells of one hash need no change). - shrink_candidates excludes divergent modes (in-place shrink is shared- label only); split_candidates offers a relabeled enclosing mode as a split. apply_split grows the cell container and returns the replication recompute. - The split's recompute = sum over split cells of cell_footprint x replication_factor, where the replication factor is the product of the block COUNTs (ceil(extent/block_size), via cm.regime().extent -- NOT the block size) of every enclosing loop the cell is homed-within but does not carry. Canonical case (1 + N3)x, not a flat 2x. - rematerialize_to_budget trials shrinks AND splits, selects by PEAK-ONLY DeltaPeak, and accumulates split recompute into RematResult:: modeled_recompute -- a report-only forecast, NOT an objective term (spec section 3, 'Does this recompute term change the schedule?'). - remat_to_router moved-detection is now hash-keyed (split cells carry fresh value_ids a value_id-keyed map would drop); both split cells yield the same DAG-scope, so one overlay per hash, asserted to agree. Empty-router paths byte-identical; existing shrink-only remat unchanged (no divergent modes -> split_candidates empty).
The router-read tripwire assert (Task 3) calls key.get_indices<...>() where key is dependent in the enclosing evaluate_impl template; gcc-13 needs the .template disambiguator (clang accepts it without). Matches the same call in PlacementRouter::home_depth. Header-only, no behavior change.
Task 6: replace the debug-only SEQUANT_ASSERT tripwire on the Enter-stage router read with a live, release-safe guard. Extract the resolution- consistency predicate into PlacementRouter::home_resolution_consistent (hd == 0, or the live loop at hd is one of THIS occurrence's own batched indices whose space the overlay's DAG-scope names), and gate the routed fetch on it: if inconsistent, 'routed' stays false and the default access_at path serves this occurrence its OWN value (recompute) rather than a wrong- slice entry -- in release, not only under an assert that is elided there. By construction home_depth returns only a consistent hd, so on every correct schedule the guard holds and the read is byte-identical (eval/dryrun suites unchanged). It is defense-in-depth: a future home_depth/overlay regression that collapsed two divergently-relabeled occurrences onto one shared entry -- the hole the DAG-scope per-use resolution closes -- is caught and recomputed instead of silently corrupting a release run.
…budget) The dry-run cost_profile's per-value cfg.max_footprint gate is the wrong instrument: per-value not peak, a second budget re-judging a schedule the DP already shaped for peak_threshold, and pre-empting the remat placement pass. Two distinct budgets were conflated (both 100 GB in the witnesses): policy.peak_threshold (the DP's per-node subtree_peak batching budget, KEPT) vs cfg.max_footprint (the replay hold-gate, RETIRED). Sweeps prove they are independent axes. Splits the fix: path A (cheap) disables the replay gate and MEASURES the DP schedule's true peak/avoidable -- no remat, one replay, already the library default; path B (opt-in) runs remat once on the final fused DAG to FIT a budget B, already paid in the MPQC pre-pass. Remat never runs per-DP- candidate (combinatorial); the DP keeps its cheap per-node proxy. Records the crux (modelled vs replayed peak equivalence, path B only) and the honest limit (remat refines placement, not batching-loop location).
Test-side plan (no library change; max_footprint already defaults to 0): env-gate the C60 witnesses' hold-gate defaulting to off, re-baseline the occ-veto and extmode bands to the true gate-off profile, and pin the confound-free invariants (perfect-CSE floor, gate-independence). Executes Design path A of 2026-08-09-remat-into-cost-profile-design.md.
… GB gate) Path A of 2026-08-09-remat-into-cost-profile-design.md. The C60 witnesses hardcoded cfg.max_footprint = 1e11 (100 GB), the replay hold-gate, which distorted the measured avoidable via evictions and conflated with the DP's separate batching budget (policy.peak_threshold). Make the gate env- configurable (SEQUANT_UT_DRYRUN_MAXFP_GB) defaulting to 0 = OFF, so the witnesses measure the schedule's TRUE peak and recompute. occ-veto, gate-off true profile (was 60.8% / 19.2% / 25.2% gated): unbatched 0% avoidable, 50814 GB peak, 1.58e16 flops (perfect-CSE floor) aux-only 0% avoidable, 18585 GB peak aux+occ ~76% avoidable, 563 GB peak (inherent per-block re-forms) Pins the perfect-CSE floor (base.total_flops == 1.5798e16) and the true tradeoff (base.avoidable == 0 proves no eviction gate governs; batching lowers peak). extmode gate-off (contracted ~0.7%, external ~2.2%) now satisfies its existing < 0.05 bands, which the gate had inflated to failing. Test-side only: no library change (CacheConfig::max_footprint already defaults to 0). Real suites byte-identical.
The equivalence probe [.][dryrun-remat-equiv] measures the remat MODELLED peak
against the replay REALIZED peak on the C60 aux+occ forest across B in
{2000,1000,500} GB. Finding: the peaks diverge (modelled up to 3.5x the
realized) and the realized peak is INVARIANT to remat placement -- flat at the
563 GB occ-veto floor for B=2000/1000, with a false RebatchNeeded at B=500. The
realized peak is DP-batching-governed, not placement-governed; the uniform
block_of the model uses also cannot match non-uniform model tiling. Recorded in
the spec evidence section; path B hard-bound enforcement does not hold on C60 as
written. Probe asserts only pipeline sanity (both peaks positive), not a bound.
…ectrum Investigation of the remat/replay peak mismatch on C60, concluded. Instrumentation: [.][dryrun-remat-equiv] probe (modelled vs replayed peak sweep, binding-composition + [remat-fix] enclosing-vs-home dumps) and a peak-composition diagnostic in CacheManager::note_working_set (env-gated SEQUANT_UT_PEAK_COMPOSE, harmless -- fires only on new global maxima). The replay's 563 GB peak = 120 GB co-resident cache (13 entries, max 24 GB) + 443 GB ONE transient contraction result: transient-working-set-bound, no 2930 GB giant ever resident. Finding: the current batched runtime is RECURSIVE FOREST DESCENT (cross-tree CSE only at the top scope), while the static peak oracle (peak_profile_sweep) models WHOLE-SCOPE / DAG descent -- two execution models, not two estimates of one peak. They are the two ends of ONE placement spectrum: enclosing-blocked footprint (633 GB, max recompute) = Forest/current runtime, validatable vs the replay; home-blocked (17667 GB, min recompute) = DAG/shared, remat-navigable to fit budget B. The forest runtime is pinned at the enclosing endpoint, which is why placement looked inert and the replay lands at 563 ~ 633. Retracts the intermediate 'block enclosing_modes is THE fix / placement is peak-inert' conclusion (that was measuring the current runtime). Neither home nor enclosing is a bug fix; they are the two execution-model footprints. Actionable: parameterize the peak oracle by ExecutionModel (Forest = enclosing, cross-checked vs the replay; Dag = home + remat); the enabling change is upgrading the runtime to whole-scope descent, which also collapses the ~76% avoidable recompute / ~5x per-group-replay penalty. Spec updated with the full account.
…t claim Rewrites the Task 1b resolution. RETRACTS the enclosing_modes mechanism entirely: it is a DAG union over occurrences, not a placement, and in forest execution each node's residency is governed by home_modes (the location variable in BOTH runtimes). 633 vs 563 was a numerical coincidence (they do not match, ~12% apart, no principled reason). Keeps the two-execution-models framing (recursive forest descent vs whole-scope DAG descent, E.V.'s insight) and the peak-composition evidence (563 = 443 GB single transient result + 120 GB cache; no giant ever resident). Correct account of why the static sweep mismodels the forest replay: it FOLDS occurrences to one meet-homed cell and prices every cell as a PERSISTENT resident, whereas the forest peak is a per-op TRANSIENT working set -- neither error is about home-vs-enclosing. Records the cheap static Forest peak model as an OPEN problem (transient working set + per-occurrence homing + per-tree timeline); the replay (cost_profile) is today's only faithful forest peak. peak_profile_sweep models the DAG runtime.
Faithful SeQuant surrogate for the MPQC water-20 pVDZ-F12 PNO-CCSD aux-batching regression (job 658937 vs 631196, 2.2x slowdown). Same csv doubles residual + df_regime(kWater20_pVDZF12) (extents/moments verified against the job log) + the exact aux-only batch config (DenseTimeSpaceBatched, K contracted-batchable, target 256, peak_threshold 1e11). Reproduces the fragmentation: 36 K-contracted batched-member annotations (~ the MPQC log's 35 distinct batched-member shapes). Confirms the root cause via SEQUANT_DP_RECOMPUTE_DEBUG: ALL 91 K-carrying gC-class composites (largest 34 GB) are priced rf==1 (K is carried -> not in the escaped set -> zero recompute charged), while the 46 K-escaping nodes are charged rf==7. The DP prices slicing the gC giants as free and batches them across the board; the runtime rebuilds each per consumer batch group (the 2.5x regression). Exactly the cost_model.hpp:1811 warning.
…t knobs) Two sweeps on the water-20 aux-batched forest. (1) rematerialize_to_budget budget sweep: modeled_recompute=0 and near-zero placement change at EVERY budget (seed 925 GB -> final 915-925 GB; RebatchNeeded at low B = no shrink candidates). remat is DEMOTE-ONLY and finds nothing to demote here, so its router is a near-no-op -- cost_profile runs on the DEFAULT placement, which homes the Kcon-batched gC INSIDE the K loop; realized avoidable 77.6%. NOTE: this does NOT test hoisting -- remat never PROMOTES a value to the top scope, so the build-once-at-top path is unexercised (a hoist pass, or not batching gC, is needed). (2) DP peak_threshold sweep (rebuild forest): Kcon-member count is a FLAT 36 across 100 GB -> 20 TB -- the DP batches every K-carrying gC across the board; the threshold does not gate it. Neither knob recovers the old ~12-member schedule.
…d-independent) Adds a seed-placement diagnostic to the water-20 remat sweep: of 21 K-carrying gC-class cells, remat_cells homes 17 INSIDE the K loop and only 4 at the top scope. The meet-home sits in the K loop because the DP batched the gC consumers Kcon across the board, so every gC occurrence is K-enclosed. remat_cells takes no peak_threshold, so this seed is threshold-INDEPENDENT; rematerialize_to_budget only DEMOTES from it (never promotes), so no peak_threshold hoists the gC to the top scope where the top cache would build each once. The lever is not peak_threshold -- it is either not batching the consumers across the board (so the meet-home rises to top) or a promote-capable placement pass (which remat lacks). (A direct hoist experiment -- force gC home_modes to top -- was removed as inconclusive: the hand-poked router was a no-op, peak unchanged; a proper hoist needs the router occurrence-keyed placement API.)
… + executor) Design from brainstorming the fundamental fix the water-20 investigation converged on. Recursive forest descent cannot share a K-slice across trees, so shared K-carrying composites are rebuilt per group. Whole-scope descent walks the fused forest scope-by-scope so a value homed at a scope node is built once per block and reused by its whole subtree. Design: (A) ahead-of-time scope-major scheduler + a pure-realizer executor that walks the scope tree (root=top scope, children=batch loops, path=one loop nest). Key insight: swap the DRIVER (per-tree recursion -> whole-forest scope-tree walk) while REUSING the existing accumulate/scatter/slice-on-use/scratch primitives. The executor honors placement exactly (no ad hoc spill); peak/recompute policy stays upstream in remat/the budget/the cost model -- which finally binds, since under whole-scope descent placement IS the dominant peak lever (resolving the placement-inert paradox) and the home_modes co-residency oracle becomes accurate. Scoped: general executor + scheduler IN, validated on the narrow canonical-chain placement the current machinery emits. OUT (explicit): general branching-tree placement (tree-LCA vs set-meet), and a scope-tree-aware optimizer that prices intraloop CSE (the current DP is per-tree and cannot see cross-tree sharing). Honest framing: this RELOCATES the peak/recompute tradeoff into principled placement, it does not eliminate it.
…dering, drop malloc aside) Three review refinements: (1) correctness is agreement to small numerical noise, NOT byte-identical -- the schedule changes contraction order so FP non-associativity yields a tiny diff; (2) cross-scope value ordering can be LAZY first -- reuse the existing compute-at-home-on-first-hit + slice-on-access mechanism (a mid-loop reference to an above-homed value is built full at its home and sliced thereafter), with explicit ahead-of-time topo sort a later optimization; the initial scheduler deliverable is just the scope tree + per-value home; (3) drop the MALLOC_ARENA confounder aside from the motivation -- state plainly that the analysis showed recompute of shared intermediates forced by forest descent that a DAG schedule eliminates.
…the executor's Per review: whether a placement fits (e.g. C60 top-scope residency) is decided by the placement/budget/optimizer, not the executor -- the executor evaluates a fixed schedule, it does not produce or change it. Moved the C60-feasibility note from the executor's open questions into the relocated-tradeoff section, framed as a future-optimizer concern. Removed a duplicate open-questions item.
Six-task plan realizing the 2026-08-10 spec. Task 1: scope-tree schedule builder from existing ValueCell placement. Task 2: executor skeleton (top-scope-only, proven == forest descent for unbatched). Task 3: single aux loop + lazy home-materialization -- the CSE win (water-20 equivalence-to-tolerance + shared gC built ONCE not per group). Task 4: nested aux+occ general walk. Task 5: coexistence flag + cost-model co-residency peak selection (flag-OFF byte-identical to forest descent). Task 6: water-20 + C60 witnesses. Pure-realizer throughout; general branching placement + intraloop-CSE optimizer explicitly out of scope; each increment validated by an equivalence test against forest descent so the primitive-composition open questions surface at the smallest increment.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Multimode batched evaluation of factorized coupled-cluster equations
Adds cost-model-driven multimode batching to the SeQuant evaluator so
large-system CSV/PNO-CC residuals can be evaluated without forming their
largest transients whole. Six squashed commits (dry-run backend, optimizer,
evaluator, supporting core, tests, docs).
What it does
-> scattered into disjoint slices) and contracted (DF aux, summed ->
accumulated). Loops nest external-outside-contracted.
DenseTimeSpace): minimizes flops withpeak_thresholdas a ceiling; role-split (contracted/external) batchability;order-aware placement over the combined nest.
(a cached intermediate fetched from an outer scope is sliced to the current
block) decouples correctness from placement; per-level placement driven by
a per-canonical lifetime mask (cross-occurrence proto-aware meet) unioned
with contracted residency; iterative (stack-safe) tree traversal.
C60 PNO-CCSD dry run (55-term residual, aux K@256, occ@8, 100 GB budget)
The DP selects the same factorization regardless of what is batchable
(flops are unchanged); batching only slices modes to lower the peak. The
roofline-time column moves because a giant intermediate executed whole is
memory-bound (
machine_balance x traffic) but compute-bound when sliced -- thecache-blocking win of the same schedule, not a cheaper one. Recompute overhead
(
avoidable_time) is 1.8% -> 6.5% -> 39.8% as slicing gets more aggressive.Validation
[eval]449,[lifetime_mask]76,[optimize]628 assertions green;OFF (order-blind) path byte-identical.
events) matches unbatched to < 1e-9, within the 1e-7 precision, no aborts.
Follow-ups (non-blocking, from the final review): dedup the proto-expansion
helper; add a real-forest hidden-tag hash-regression test; revisit the
stamp_lifetime_masksconst_cast.