dasLLAMA: dig the last two board reds green — lcpp-shape mx4 kernels + GPU PLE pre-step, fresh M1 board

[borisbat]

Digs the two remaining public-board reds — gpt-oss (tg 0.963 / pp 0.88) and E4B (pp 0.856) — with measurement hygiene as position 0. Both cells flip green; every number below is a paired same-process run vs llama.cpp stock on M1 Max at llama-bench shapes.

Position 0 — hygiene

• dasMetal per-site leak attribution: every live handle now carries a type[bytes] @ file:line tag (metal_live_object_report(), metal_log_leaks_if_any() in das_metal_boost); the 5 dasLLAMA metal test gates assert through it. It named the historic 51-object leak in one shot: the single-decode MoE release ledger forgot u_moe_ss/u_moe_kdim/u_moe_tot — 3 pool_release lines, fam-gptoss 15 leaks -> 0.
• 27B cpu record refresh (gen_bench_records) + the dn-plane fix in the exec_fmt receipt.
• q8q8 crown restored on the 12B-Q8 tune sidecar (stale kstep2 pin beat by mr8_budget +6.8%).
• pdflatex: declare θ (U+03B8) and − (U+2212) so the doc PDF gates keep passing with the tutorial revision.

gpt-oss: mx4 kernels to the lab-winning shape

bench_metal_moe_lab grew a gmx4 section at exact gpt-oss shapes (E=32, nfe=dim=2880, k=4, CPU-ref bit-exact) and laddered the routed expert GEMV from the production 239 wGB/s to 301:

rung	wGB/s	change
production	239	8 threads/block, 4B uint loads (each loaded twice), device LUTs
tgmem LUT	257	llama.cpp's 16-value LUT copied to threadgroup memory
lcpp mapping	272	16 blocks in flight x 2 threads/block, yl[4] float4 x prefetch, float4 accumulators
+ uint2 loads	275	each weight byte loads exactly once
+ computed e8m0	301	as_type<float> in ALU — kills the mxe->etab dependent device-load chain

Refuted along the way: ALU e2m1 decode (2.5x worse — tables win on AGX), etab-to-tgmem (broadcast reads are already on the uniform path).

Both production mx4 kernels get the winning shape — MetalMoeGemvMx4 (decode GEMV) and MetalMoeMulMmMx4's A-stage (uint4 block load + tgmem LUT + computed e8m0). Both e8m0 device tables are deleted; the ALU form is bit-exact for all 256 exponent bytes including the e<2 denormals.

• tg128: 86.58 vs 82 = 1.06 GREEN (was 0.963 red)
• pp512: 986.9 vs 1024 = 0.963 (was 0.88); steady das GPU 515ms vs lcpp wall 500ms — the remaining ~3.5% needs its own prefill chase

msl_emit grows the standing-rule capability behind this: the daslib/math_bits bitcast family lowers to as_type (plus exp2/log2 in the math whitelist), with real-GPU coverage in tests/metal/test_metal_math.das.

E4B: PLE pre-step on GPU

The E-series prefill pre-step (gather 512 token rows off the 3GB Q8_0 per_layer_token_embd, the 14-GMAC bf16 model_proj GEMM, rms/combine) ran on CPU — 126ms beside a 529ms GPU window that already beat lcpp's whole wall. It now runs as three dispatches at the head of the prefill graph:

• MetalPleGatherQ8 — dequants token-row slices straight off the blob's q8 views (zero-copy on mapped images) into the layer-major side panel, sqrt(ple) folded; the CPU-side transpose memcpy is gone with it
• MetalBf16MulMm — the q8 mul_mm with its A stage swapped to native BF16 rows (widen = exact bit shift via the new bitcast); model_proj streams from the kept-bf16 plane, no resident f32 copy
• MetalPleFinish — per (layer, position) slice rms + pnorm + combine, in place

Plumbing: register_ple_gpu_gate — forward_prefill asks the driver before the CPU pre-step and stashes the token rows on yes; the driver falls back to ple_pre_prefill on a late decline, and ple_block_prefill panics loud if a promise ever leaks to the CPU body. Decode PLE (1 row/step) stays CPU by design.

• pp512: 959.2 vs 916 = 1.05 GREEN (was 0.856 red); steady prefill wall 655 -> 530ms, GPU window +1-3ms
• tg128: 0.98 (path untouched)

Verification: fam-gptoss tolerance cells bit-identical pre/post the kernel swap; fam-gemma4e green including the token-for-token generate vs CPU truth; model-less kernel-unit gates bit-exact (odd-shape tails, bias and batch-stream arms); llama prefill base arm green; full preflight green.

The fresh board

Full M1 re-sweep after the fixes (all 8 catalog models, cpu+metal, refs adjacent per cell, quiet window, r=5) — records + the hidden site section regenerated. das leads or ties 41 of 48 cells:

model	metal pp/tg	cpu vs clean pp/tg	cpu vs stock pp/tg
Qwen3.6-35B-A3B-MTP	1.02 / 1.20	1.39 / 1.12	1.45 / 1.12
Qwen3.6-27B-MTP	0.91 / 1.04	1.03 / 1.05	1.02 / 1.05
gemma-4-12B	1.04 / 1.16	1.11 / 1.05	1.11 / 1.04
gemma-4-E4B	1.06 / 1.00	1.58 / 1.08	0.96 / 1.11
gpt-oss-20b	0.92 / 1.06	1.74 / 0.98	1.75 / 0.98
Qwen3-30B-A3B	1.00 / 1.14	1.60 / 1.10	1.62 / 1.02
Mistral-Small-24B	1.01 / 1.16	1.06 / 1.00	1.04 / 1.01
gemma-4-26B-A4B	1.07 / 0.98	1.10 / 0.97	0.86 / 0.97

Every sweep-era window-skew red re-paired clean (30B metal tg 0.95 -> 1.14, Mistral pp cells, 27B cpu tg, 12B cpu pp). Remaining sub-parity, honest: 27B metal pp 0.91 (dense k4 mul_mm A-stage — next chase), gpt-oss metal pp 0.92 (0.96 quiet-paired; a fresh-process first-rep PSO-compile stall skews the sweep mean — warmup-rep fix is a tool follow-up), and the AMX-stock cpu-pp class on 26B/E4B (0.86/0.96 — both lead the kernel-fair clean-cpu ref).

Ride-along: the tune confirm pass now echoes its child's output tail when an arm scores 0 (a swallowed 'cannot open gguf' cost a debug round mid-sweep).

🤖 Generated with Claude Code

https://claude.ai/code/session_01XLduse7eHSS1iJ2BG2P4LT

[Copilot]

Pull request overview

This PR targets the remaining public-board Metal performance regressions for dasLLAMA on Apple Silicon by (1) improving measurement/leak hygiene, (2) reshaping the gpt-oss MXFP4 MoE kernels to the lab-proven mapping, and (3) moving gemma-4 E-series PLE prefill pre-step onto the GPU.

Changes:

• Add per-creation-site Metal live-handle reporting and integrate it into existing Metal leak gates.
• Extend Metal MSL codegen to support exp2/log2 and daslib/math_bits bitcasts via MSL as_type, with a new GPU-vs-CPU math/bitcast test.
• Implement gpt-oss MXFP4 kernel updates and an E-series GPU PLE pre-step (gather + BF16 GEMM + finish), plus refreshed M1 performance records and harness diagnostics.

Reviewed changes

Copilot reviewed 20 out of 20 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
tests/metal/test_metal_math.das	Expands GPU-vs-CPU math test coverage to include exp2/log2 and bitcast round-trips via daslib/math_bits.
modules/dasMetal/src/dasMetal.mm	Tracks live Metal handles with a tagged registry and exposes metal_live_object_report() for leak attribution.
modules/dasMetal/src/dasMetal.h	Declares the new metal_live_object_report API for module/AOT integration.
modules/dasMetal/metal/msl_emit.das	Allows exp2/log2 and lowers math_bits bitcasts to MSL as_type<>().
modules/dasMetal/metal/das_metal_boost.das	Enhances leak assertions/logging using metal_live_object_report() and adds metal_log_leaks_if_any().
modules/dasLLAMA/tests/test_metal_support_matrix.das	Switches leak-gate checks to metal_log_leaks_if_any() for better diagnostics on failures.
modules/dasLLAMA/tests/test_metal_prefill_parity.das	Uses metal_log_leaks_if_any() for more actionable leak test failures.
modules/dasLLAMA/tests/test_metal_prefill_kernels.das	Updates the mx4 GEMV kernel gate for tgmem LUT usage and removes now-obsolete e8m0 table plumbing.
modules/dasLLAMA/tests/test_metal_gemm.das	Uses metal_log_leaks_if_any() for improved Metal leak reporting in GEMM tests.
modules/dasLLAMA/performance/records/m1.json	Refreshes M1 performance record snapshots and tune metadata.
modules/dasLLAMA/performance/profile_common.das	Fixes/extends exec-format “receipt” reporting to include deltanet projection formats for recurrent layers.
modules/dasLLAMA/harness/gen_tune_probe.das	Improves tune-confirm diagnostics by printing the child output tail when parsing yields no score.
modules/dasLLAMA/dasllama/dasllama_metal_prefill.das	Adds BF16 mul_mm variant, implements GPU PLE pre-step dispatches, and updates mx4 kernels to the new mapping/bitcast path.
modules/dasLLAMA/dasllama/dasllama_metal_llama.das	Fixes MoE step resource releases by returning missing uniform buffers to the pool.
modules/dasLLAMA/dasllama/dasllama_metal_kernels.das	Removes e8m0 device-table usage and updates mx4 GEMV dispatch to set required threadgroup memory.
modules/dasLLAMA/dasllama/dasllama_metal_common.das	Adds bf16_of helper to bind kept-BF16 planes for zero-copy or cached residency.
modules/dasLLAMA/dasllama/dasllama_common.das	Introduces a PLE GPU gate mechanism and exposes ple_pre_prefill for late-decline fallback.
modules/dasLLAMA/benchmarks/matmul/bench_metal_moe_lab.das	Adds a gpt-oss mx4 “lab” section to validate and time multiple kernel rungs at exact shapes.
doc/source/conf.py	Adds LaTeX unicode declarations for θ and typographic minus to keep PDF builds passing.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 22 out of 22 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 22 out of 22 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (2)

modules/dasMetal/src/dasMetal.mm:88

• If g_metalLiveTags is updated to track retain counts per handle, release_handle should decrement that count and only erase when it reaches zero. Otherwise the tag entry can disappear while the handle is still logically live (multiple retains).

    static void release_handle ( void * h ) {
        if ( !h ) return;
        g_metalLiveObjects.fetch_sub(1, std::memory_order_relaxed);
        {
            std::lock_guard<std::mutex> guard(g_metalLiveTagsMx);
            g_metalLiveTags.erase(h);
        }

modules/dasMetal/src/dasMetal.mm:560

• metal_live_object_report currently assumes g_metalLiveTags values are plain strings. If g_metalLiveTags tracks (tag,count) per handle, aggregate bySite using the stored count so the report reflects multiple retains of the same ObjC object address.

    char * metal_live_object_report ( Context * ctx, LineInfoArg * at ) {
        std::map<std::string, int64_t> bySite;
        {
            std::lock_guard<std::mutex> guard(g_metalLiveTagsMx);
            for ( auto & kv : g_metalLiveTags ) bySite[kv.second]++;

[Copilot]

Pull request overview

Copilot reviewed 22 out of 22 changed files in this pull request and generated no new comments.