feat: nested closed-loop metrics (object/rb/redlight/brake + tdigest)

[xtk8532704]

What this PR does

Restructures closed-loop evaluation metrics into a nested schema and adds road-border, red-light, and strong-brake scorers, plus clearance t-digest sidecars. Per-step scoring moves into scenario_generation/metrics/; closed_loop_eval.aggregate / summary.json / segments.jsonl consume the new layout.

Base: #279 (kem/refactor-reproducer-test-PR2). Retarget to tier4-main after #279 merges.

Nested summary schema

Each segment row (and the run summary.json) groups metrics by family instead of flat n_collision_steps-style keys:

Block	Contents
object	collision / miss (near-miss) steps·count·segments·rates, clearance min/mean/p5
road_border	same shape for curb distance (collision ≤ 0, miss ≤ thresh)
red_light_violation	steps·count·segments·rates
strong_brake	threshold, steps·count·segments·rates, strongest accel
reproducer	expand_count, snap_count, normal_steps, repeat_steps, repeat_step_rate
terminated_counts	end-reason histogram

Event *_count uses rising-edge detection with a short falling-edge debounce so threshold flicker does not re-count the same event.

Breaking: old flat keys (n_collision_steps, n_near_miss_steps, n_snaps, …) are gone. In-repo callers (train.py, valid_predictor_closed_loop.py, R2LPL / mining tools) are updated in this PR.

New scorers (scenario_generation/metrics/)

• object.py — OBB ego↔neighbor clearance / collision / miss (extracted from rollout)
• road_border.py — distance to line_strings curb geometry
• red_light.py + ego_traj.py — red-light violation using live ego history
• strong_brake.py — realized tangential accel at/below --strong_brake_mps2 (default −2.5)
• tdigest.py — clearance distribution helpers

Clearance t-digest sidecar

Clearance digests are written to tdigests.jsonl / tdigests_{rank}.jsonl so segments.jsonl stays human-readable. Multi-GPU merge reattaches sidecars before aggregate (p5 via merged digests). --strong_brake_mps2 is passed through sharded parent merge so sequential and multi-GPU summaries share the same schema.

Testing

• tests/test_closed_loop_metrics.py, test_reproducer_unstick.py, test_perception_reproducer_cursor.py, test_reproducer_collision_scope.py
• Spot-check a closed-loop run: summary.json, segments.jsonl, tdigests.jsonl

[danielsanchezaran]

Reviewed the nested-metric rewire. Good news first: the t-digest is the external tdigest==0.5.2.2 library (correct API usage — empty-guarded, percentile in [0,100], documented to_dict/update_from_dict merge), and the schema wiring is solid — train.py._flat_scalars, format_summary_lines, run_lifelong_r2lpl_rounds._run_closed_loop_probe's keep tuple, and both mining tools all read keys that aggregate actually emits (no KeyError/silent-zero), division guards short-circuit on 0, strong_brake accel source is real and aligned, and the red-light sign is correct. Two real bugs + a couple of nits inline.

Two findings worth blocking on the first: the road_border collision bucket is silently always zero, and the pooled clearance mean is weighted wrong.

[danielsanchezaran]

Bug — this bucket is structurally always 0. rb here is the per-timestep min border distance from compute_road_border_penalty(...)[5], which is an unsigned point-to-segment distance (planner_metrics/subscores.py _point_to_segments_min_dist, always ≥ 0, inf when no border data). So rb <= 0.0 is measure-zero — road_border.collision_steps/count/segments/*_rate report 0 even on a blatant curb crossing. The canonical crossing predicate in the same module is per_ts_min < rb_cross_thresh (0.20 m, config.py). The miss bucket (rb <= near_miss_thresh = 0.5) still fires, which hides this in casual inspection. Use rb < rb_cross_thresh (or a signed distance) for the collision bucket. Contrast object, whose collision correctly uses a signed-overlap < 0 test.

[danielsanchezaran]

Bug: clearance_mean_m per segment is the mean over finite clearance steps only, but here it's weighted by n_steps_run (all steps, including the no-neighbor inf steps that were excluded from that mean). Segments with many neighbor-free steps get over-weighted, biasing the pooled global mean. The correct weight is the per-segment count of finite clearance samples — which the segment row doesn't currently emit, so this needs a new per-segment field (or pool from the digests instead).

[danielsanchezaran]

Nit: {**_chunk_row(chunk), **result} dumps the full _finalize dict — including the _tdigest centroid blob from _clearance_stats — straight into segments.jsonl, bypassing metrics_for_json. Serializable (so not a crash), but it defeats the human-readable-jsonl goal and bloats every mined row with nested centroid dicts. Route through metrics_for_json(...). (mine_collisions_reproducer.py has the same pattern.)

[danielsanchezaran]

Flagging the road_border collision bucket as a blocking bug to fix before merge (details in the inline comment on reproducer_rollout.py): rb_coll = rb_finite & (rb <= 0.0) can never be true — rb (compute_road_border_penalty(...)[5], per_ts_min) is an unsigned point-to-segment distance (≥ 0, inf when no border data). So road_border.collision_steps/count/segments/*_rate are silently always 0, even on an actual curb crossing. The miss bucket (<= 0.5) still fires, which hides it. Please switch the collision predicate to rb < rb_cross_thresh (0.20 m, matching the canonical crossing test in planner_metrics/subscores.py) or use a signed distance, and add a curb-crossing fixture that asserts collision_steps > 0 so it can't regress.

[danielsanchezaran]

Re-reviewed the rebased/latest head. The lockfile follow-ups do not address the two correctness bugs on the existing inline discussions: road_border.collision_* still derives collision from an unsigned distance <= 0, so crossings are effectively reported as zero; and pooled clearance_mean_m is still weighted by all rollout steps instead of finite clearance samples. Both directly affect closed-loop safety comparisons and R2LPL probe summaries.

The broader workflow suite also exposes an incomplete result/state API migration: scenario_generation/tests/test_closed_loop_metrics.py, test_reproducer_collision_scope.py, both reproducer cursor/unstick modules, and full rlvr/autoresearch/tools/test_r2lpl_lifelong_replay.py yield 177 passed, 4 failed, 1 skipped. One failure still returns the removed SimpleNamespace(metrics=...) shape to mine_direct_reproducer_chunks; three fake rollout states lack the newly-required rb_dists/red_light buffers (and retain the old _finalize contract). Please update those harnesses and run the full R2LPL module, not only the four scenario-generation files listed in the PR.

Workflow impact after correction: this is an intentional breaking migration from flat metrics to nested object/road-border/red-light/brake/reproducer blocks; the in-repo mining and lifelong-probe consumers are updated, but historical segments.jsonl/summary consumers will need migration. The t-digest sidecar fixes sharded p5 aggregation, and the sharded summary now correctly persists both strong_brake_mps2 and final elapsed_sec.

[danielsanchezaran]

Heads up on merge order: we're going to land #285 before this one. It adds fail-to-resume mining (realized-lag) — something the reproducer couldn't do before (it was mining 0 of these events) and that the lvl4 campaign needs right now. It's a small change and its core lives in reproducer_danger_scorer.py, which this PR doesn't touch.

The reason to do it first rather than after: this PR reworks the per-chunk metrics and the summary that #285's --realized_reward writes into. If this merges first, that path has to be re-plumbed onto a schema that's still in flux here; if #285 goes first, you can just fold its one realized_cl_reward field into the new nested-metric schema as part of this refactor. #285 is already rebased, reviewed, and green, so the rebase on this side afterward should be small. Flagging so we don't collide on reproducer_rollout.py / the mining tools.

[xtk8532704]

Sounds good — I'll wait for #285 to land, then rebase this PR and merge after that.

[danielsanchezaran]

Update: #285 is merged (squashed into main as fda76074). So the realized-lag mining + its single realized_cl_reward metric field are now on main ahead of this PR, as planned. When you rebase this refactor, that field can just fold into the new nested-metric schema instead of being re-plumbed. Should be a small rebase on the reproducer_rollout.py / mining-tools side now.