From e2dedd409e4496510c1a4e13e572657ccafeec8e Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Sat, 6 Jun 2026 12:12:36 +0000 Subject: [PATCH] docs: add issue write-ups for multi-victim steal chain and wakeFirstIdle ABA bias --- docs/issues/multi-victim-sequential-chain.md | 113 +++++++++++ .../wakefirstidle-lowest-id-bias-aba.md | 179 ++++++++++++++++++ 2 files changed, 292 insertions(+) create mode 100644 docs/issues/multi-victim-sequential-chain.md create mode 100644 docs/issues/wakefirstidle-lowest-id-bias-aba.md diff --git a/docs/issues/multi-victim-sequential-chain.md b/docs/issues/multi-victim-sequential-chain.md new file mode 100644 index 0000000..4c8b676 --- /dev/null +++ b/docs/issues/multi-victim-sequential-chain.md @@ -0,0 +1,113 @@ +# Sequential chain for multi-victim steal: Go's soft approach for load imbalance at 8+ carriers + +## Problem + +⚠️ **Still open — no burst data at 8+ carriers with load imbalance** + +With 8+ carriers and load imbalance, the current `wakeFirstIdle` approach lacks burst-handling data. When a single carrier in a cluster of 4 cores slows down, the remaining idle carriers need to help — but our current sequential scan has limitations that may not scale. + +The core scenario: in a cluster of 4 cores, 1 slows down (GC pause, OS scheduling, thermal throttling, etc.). You need the other idle cores to absorb the burst. But if you have 2 slowing down, you need the remaining 2 idle ones to help — can you afford soft limits that exceed half your cores? + +## Go's Approach: Sequential Victim Chain (Soft Limits) + +Go's runtime (`src/runtime/proc.go`) uses a **sequential chain** for multi-victim work-stealing: + +### How `findRunnable()` works + +``` +// Pseudocode from Go runtime +func findRunnable() { + // 1. Check local run queue + // 2. Check global run queue (every 61st schedule) + // 3. Poll network + // 4. Steal from other P's: + // - Pick random start via fastrand() + // - Iterate ALL P's sequentially (wrapping around) + // - Steal up to half the victim's run queue +} +``` + +### Key mechanisms + +1. **No explicit utilization measurement**: Go does NOT measure per-P utilization. Instead: + - Global `npidle` counter tracks number of idle P's + - `needspinning()` heuristic limits spinning M's to `GOMAXPROCS / 2` + - At most `sched.nmspinning` goroutines actively searching + +2. **Throttling searchers, not victims**: Go limits **how many threads can be in search state simultaneously**: + ``` + func wakep() { + if sched.nmspinning.Load() != 0 || !sched.nmspinning.CompareAndSwap(0, 1) { + return // Someone already searching, don't wake more + } + // ... wake an idle M + } + ``` + The `nmspinning` limit prevents thundering-herd effects without per-core utilization. + +3. **Randomized victim selection**: `fastrand()` picks a starting victim index, then sequential iteration through all P's distributes steal pressure evenly. + +4. **Steal half the queue**: Amortizes the cost of stealing — one wake-up can move multiple runnable goroutines. + +### Why `GOMAXPROCS / 2`? + +Go's reasoning: if more than half of processors are already spinning/searching, adding more searchers increases contention without proportionally increasing throughput. The remaining active processors are either: +- Productively executing goroutines, or +- About to finish and become victims themselves + +## What We Can Do Without Utilization Measurement + +Our current implementation in `EventLoopSchedulerGroup` uses `ClusterState.tryStartSearcher()` which already implements a searcher count — analogous to Go's `nmspinning`. The question is: **what should the cap be?** + +### Option A: Cap at `clusterSize / 2` (Go's approach) + +``` +// In ClusterState.tryStartSearcher(): +if (nSearching >= clusterSize / 2) return false; +``` + +**Pro**: Prevents thundering herd, matches Go's battle-tested heuristic. +**Con**: With cluster size 4, this means at most 2 searchers. If 2 carriers are overloaded, only 2 can help — but they're also the ones that are supposed to be idle helpers. + +### Option B: Cap at `clusterSize - 1` (aggressive) + +Always allow all-but-one to search. Maximizes steal bandwidth. + +**Con**: High contention on victim queues, wasted cycles on failed steals. + +### Option C: Adaptive cap based on idle count + +``` +int cap = Math.max(1, idleCount / 2); +``` + +Scale the searcher cap based on how many carriers are actually idle. More idle = more search bandwidth allowed. + +### Option D: Sequential victim chain with rotating start + +Instead of bitmap scan from word 0 (lowest-ID first), rotate the start position: +``` +int start = (lastWakeIndex.getAndIncrement()) % capacity; +// scan from start, wrap around +``` + +This distributes which carrier gets woken, preventing lowest-ID starvation. + +## Relationship to the Half-Cores Question + +> You cannot afford to make soft limits exceed half of your cores (?) or not. + +The tension is: +- **Too few searchers** (< half): Under bursty load, available help sits idle while one carrier drowns +- **Too many searchers** (> half): Wasted wake-ups, CAS contention, cache pollution + +Go's answer: cap at half, but make each steal move MORE work (steal-half semantics). This means fewer wake-ups are needed to redistribute work. + +**Our answer should likely be**: cap searchers at `clusterSize / 2`, BUT when we do wake a searcher, direct it precisely to the overloaded victim (which we already do via `SEARCHING + victimId`). This is actually better than Go because our directed steal avoids the random walk. + +## References + +- Go runtime: `src/runtime/proc.go` — `findRunnable()`, `wakep()`, `stealWork()` +- Go's `nmspinning` invariant: `src/runtime/proc.go:3133` +- Current implementation: `IdleCarrierTracker.wakeFirstIdle()`, `EventLoopScheduler.signalWorkFor()` +- Cluster state: `EventLoopSchedulerGroup.ClusterState.tryStartSearcher()` diff --git a/docs/issues/wakefirstidle-lowest-id-bias-aba.md b/docs/issues/wakefirstidle-lowest-id-bias-aba.md new file mode 100644 index 0000000..fdc4bcb --- /dev/null +++ b/docs/issues/wakefirstidle-lowest-id-bias-aba.md @@ -0,0 +1,179 @@ +# `wakeFirstIdle` lowest-ID bias: ABA problem and bitmap vs. stack trade-off + +## Problem + +The current `IdleCarrierTracker.wakeFirstIdle()` uses a bitmap scan starting from word 0, bit 0. This creates a **lowest-ID bias**: carrier 0 is always woken first, then carrier 1, etc. This bias creates an ABA problem pattern and unfair distribution of work-stealing duties. + +**Constraint**: We want ONE data structure (bitmap OR stack), not both. The question is which one, and how to avoid the ABA problem inherent in each. + +## The ABA Problem in Detail + +### With Bitmap (current implementation) + +```java +// IdleCarrierTracker.wakeFirstIdle() — always starts from word 0 +for (int w = 0; w < words.length; w++) { + long word = (long) WORDS.getVolatile(words, w); + while (word != 0) { + int bit = Long.numberOfTrailingZeros(word); // always picks lowest bit! + int carrierId = (w << 6) + bit; + if (group.scheduler(carrierId).wakeupAsSearcher(victim)) { + return true; + } + word &= ~(1L << bit); + } +} +``` + +**The ABA scenario:** + +1. **Time T0**: Carriers 0, 1, 2, 3 are idle (bitmap = `0b1111`) +2. **Time T1**: Signal arrives → wakes carrier 0 (lowest ID) +3. **Time T2**: Carrier 0 steals work, runs it, finishes, parks again → bitmap = `0b1111` +4. **Time T3**: Another signal → wakes carrier 0 AGAIN +5. **Repeat**: Carrier 0 is constantly hot-pathed, carriers 1-3 remain cold + +This is the "ABA" because the bitmap returns to the same state (`A → B → A`), and the scan algorithm cannot distinguish "carrier 0 was just active and re-parked" from "carrier 0 has been idle for a long time." + +**Consequences:** +- Carrier 0's cache is constantly churned (steal from varying victims) +- Carriers 1-3 never build cache affinity for their assigned work +- Under moderate load, one carrier does all the stealing while others sleep +- Unfair power distribution — carrier 0's core runs hotter + +### With Stack (LIFO idle list) + +A stack (Treiber stack or similar) gives **most-recently-parked** semantics: + +``` +// Push on park +push(carrierId) // most recent parked is on top + +// Pop on wake +carrier = pop() // wakes the LAST one that parked +``` + +**The ABA scenario with stack:** + +1. **Time T0**: Stack = [3, 2, 1, 0] (carrier 3 parked most recently) +2. **Time T1**: Signal → pop carrier 3, wake it +3. **Time T2**: Carrier 3 finishes, parks → Stack = [3, 2, 1, 0] +4. **Time T3**: Signal → pop carrier 3 AGAIN + +Same problem, different polarity! Now it's the **most-recently-parked** carrier that gets all the work. + +But the stack ABA is actually WORSE because: +- The "hot" carrier just finished work and re-parked → its cache is warm for ITS OWN work, not for stealing +- Waking it to steal from a different victim pollutes that warm cache +- Stack operations require pointer chasing (slower than bitmap scan on small sets) + +### The Core ABA Issue + +Both structures suffer from ABA because they lack **temporal ordering** — they cannot distinguish: +- "This carrier has been idle for 1µs" (just re-parked) +- "This carrier has been idle for 10ms" (genuinely cold, good steal candidate) + +## Why Not Both? + +Having bitmap AND stack creates consistency problems: +- Must update both atomically on park/wake — ordering bugs +- Bitmap says idle, stack doesn't contain it (or vice versa) +- Double the memory barriers and CAS operations +- Complexity explosion for minimal gain + +## Solutions: Single Data Structure + +### Option 1: Bitmap with Rotating Scan Start + +Keep the bitmap but eliminate lowest-ID bias: + +```java +boolean wakeFirstIdle(EventLoopSchedulerGroup group, EventLoopScheduler victim) { + int startWord = (int)(scanCounter++ % words.length); // rotate start + for (int i = 0; i < words.length; i++) { + int w = (startWord + i) % words.length; + long word = (long) WORDS.getVolatile(words, w); + while (word != 0) { + int bit = Long.numberOfTrailingZeros(word); + int carrierId = (w << 6) + bit; + if (group.scheduler(carrierId).wakeupAsSearcher(victim)) { + return true; + } + word &= ~(1L << bit); + } + } + return false; +} +``` + +**Pro**: Simple, keeps O(1) per-word scan, distributes wakes across carriers. +**Con**: `scanCounter` is a shared mutable — adds contention. Could use thread-local or victim-ID-derived offset instead. + +### Option 2: Bitmap with Victim-ID-Derived Start + +Use the victim's ID to deterministically pick which idle carrier to wake: + +```java +int startBit = victim.id % capacity; // start near the victim +int startWord = startBit >>> 6; +``` + +**Pro**: No shared mutable counter. Naturally distributes — different victims wake different helpers. Good for cache locality (nearby carriers share cache hierarchy). +**Con**: If victims are always the same carrier (the slow one), this degrades back to fixed-start. + +### Option 3: FIFO Queue (Ring Buffer) + +Replace bitmap with a bounded FIFO ring buffer of idle carrier IDs: + +``` +// On park: enqueue(carrierId) at tail +// On wake: dequeue from head → oldest idle carrier +``` + +**Pro**: Wakes the **longest-idle** carrier — maximally cold, no ABA, fair distribution. +**Con**: +- Requires CAS on head/tail pointers (more expensive than bit-set/clear) +- Cannot do O(1) "is carrier X idle?" check (need separate flag or scan) +- Ring buffer sized to carrier count, but memory is fine + +### Option 4: Bitmap + Per-Carrier Epoch (Lightweight Temporal Tag) + +Keep bitmap for fast "who is idle?" query, but add a per-carrier generation counter: + +```java +// On park: generation[carrierId]++; markIdle(carrierId) +// On wakeFirstIdle: among idle carriers, prefer one with LOWEST generation +// (has been idle longest / woken least recently) +``` + +This is still ONE logical structure (bitmap for set membership, parallel array for priority), but gives temporal ordering without a separate stack. + +**Pro**: O(N) scan but N ≤ cluster size (typically 4-8), so negligible cost. +**Con**: Adds a VarHandle read per idle carrier during selection. + +## Recommendation + +For clusters of 4-8 carriers (the common case), **Option 4 (bitmap + epoch)** or **Option 1 (rotating scan)** are the best trade-offs: + +- The bitmap remains the authoritative "who is idle?" structure (single source of truth) +- Rotating scan is simplest — just change the start position +- Epoch-based adds fairness but at the cost of one extra read per idle carrier + +The key insight: **at cluster sizes of 4-8, the scan cost is negligible** — we're talking about scanning 1 `long` word. The ABA unfairness matters more than scan efficiency. + +## The Half-Cores Constraint + +> You usually have in a cluster of 4 cores, 1 slowing down. If you have 2 slow you need the other 2 idle to help. + +This constrains the solution: if we rotate wakes too aggressively, we might wake a carrier that's about to receive its own new work. The safe heuristic: +- At most `clusterSize / 2` carriers should be in search/steal state simultaneously +- The rotation ensures that among the idle set, each gets fair turns being the helper +- But we never wake MORE than half, preventing the case where all cores are stealing and none are serving + +## References + +- Current implementation: `IdleCarrierTracker.wakeFirstIdle()` (line 92) +- `findIdle()` has the same lowest-ID bias (line 73) +- `markIdle`/`markActive` are correct (no ABA in the marking itself) +- The ABA manifests only in the **selection** logic (`numberOfTrailingZeros` always picks lowest) +- Go's equivalent: `pidleget()`/`pidleput()` use a linked list (stack-based), but Go's `findRunnable()` randomizes victim selection to compensate