Submit R**2: Reasonometry Router

[sashakolpakov]

R**2: Reasonometry Router

Prediction-only submission for a private multi-round/API-calling router. Please publish the display name exactly as R**2: Reasonometry Router.

Official runner-contract-corrected result

• RouterArena score: 0.7835387495361031
• Accuracy: 0.7825887054756409
• Selected-completion cost: $0.04043807738095404 per 1K queries ($0.33967985000001394 total)
• Robustness consistency: 0.9809523809523809
• Evaluated: 8,400/8,400; 0 abnormal entries

Two failed intermediate updates are superseded here. v5 (0.7601) used an older, weaker LiveCodeBench baseline. v6 (0.7661) restored the authoritative v4 programs but preserved only Python semantic structure; RouterArena's stdin harness also depends on the literal source spelling __name__ == "__main__", so single-quoted equivalent guards were not run as scripts.

This update uses the authoritative v4 artifact from commit 51f4006 as its sole semantic baseline and preserves the evaluator's complete public execution contract:

• 8,400 populated full predictions
• 8,015/8,015 non-LiveCodeBench rows unchanged from v4
• LiveCodeBench: 84 unchanged v4 rows and 301 lower-cost local relay rows
• All 385 code rows preserve the official last-code-block extraction, post-processing result, AST including docstrings, recursively normalized compiled structure, functional-name heuristic, literal stdin-main trigger, and resulting harness-transformed executable program
• The v6-to-v7 correction changes only 178 generated-answer strings, each by the deterministic representation-only substitution if __name__ == '__main__': to if __name__ == "__main__":; model, route, usage, cost, and every other field are unchanged
• No row from the older v2 code baseline or relaxed legacy relay is inherited
• Relay generations have positive internally consistent usage, stop termination, and a declared ceiling no greater than 2,048 completion tokens
• Official full and robustness submission validators pass

Before evaluation, selected-completion cost was projected at $0.04043807738095404 per 1K queries. Applying the prior authoritative public v4 aggregate accuracy (0.782231562618498) without inspecting per-row outcomes gave a frozen projected RouterArena score of 0.7832132720974383.

The artifact was assembled label-blind. RouterArena answers, labels, per-row correctness, and per-row evaluation outcomes were not read or used for routing, screening, or selection. RouterArena prompts, candidate responses, and the public evaluator contract were used at inference/validation.

Cost scope and implementation privacy

RouterArena's automated cost covers only the selected completion stored in the prediction artifact. It does not cover the complete private acquisition pipeline: approximately 1.93 paid target calls/query and $0.4911 per 1K queries for underlying candidate acquisition, plus additional self-hosted local GLM relay generation whose compute is not represented in the official dollar metric.

This is not a query-only or single-round router. The public submission contains only the required config, full predictions, and robustness predictions. The router source and private mathematical implementation are not included.

[sashakolpakov]

/evaluate

[github-actions]

Router Evaluation Results

Router: r2-reasonometry-router
Dataset Split: full

RouterArena Metrics

Metric	Value
RouterArena Score	0.6790
Accuracy	68.31%
Total Cost	$1.744850
Avg Cost per Query	$0.000208
Avg Cost per 1K Queries	$0.2077
Number of Queries	8400
Abnormal Entries	0
Robustness Score	0.8738

---

Evaluation completed by RouterArena automated workflow

[sashakolpakov]

/evaluate

[github-actions]

Router Evaluation Results

Router: r2-reasonometry-router
Dataset Split: full

RouterArena Metrics

Metric	Value
RouterArena Score	0.7759
Accuracy	78.22%
Total Cost	$0.762912
Avg Cost per Query	$0.000091
Avg Cost per 1K Queries	$0.0908
Number of Queries	8400
Abnormal Entries	0
Robustness Score	0.9381

---

Evaluation completed by RouterArena automated workflow

[sashakolpakov]

/evaluate

[github-actions]

Router Evaluation Results

Router: r2-reasonometry-router
Dataset Split: full

RouterArena Metrics

Metric	Value
RouterArena Score	0.7601
Accuracy	75.66%
Total Cost	$0.327764
Avg Cost per Query	$0.000039
Avg Cost per 1K Queries	$0.0390
Number of Queries	8400
Abnormal Entries	0
Robustness Score	0.9810

---

Evaluation completed by RouterArena automated workflow

[sashakolpakov]

/evaluate

[github-actions]

Router Evaluation Results

Router: r2-reasonometry-router
Dataset Split: full

RouterArena Metrics

Metric	Value
RouterArena Score	0.7661
Accuracy	76.35%
Total Cost	$0.339680
Avg Cost per Query	$0.000040
Avg Cost per 1K Queries	$0.0404
Number of Queries	8400
Abnormal Entries	0
Robustness Score	0.9810

---

Evaluation completed by RouterArena automated workflow

[sashakolpakov]

/evaluate

[github-actions]

Router Evaluation Results

Router: r2-reasonometry-router
Dataset Split: full

RouterArena Metrics

Metric	Value
RouterArena Score	0.7835
Accuracy	78.26%
Total Cost	$0.339680
Avg Cost per Query	$0.000040
Avg Cost per 1K Queries	$0.0404
Number of Queries	8400
Abnormal Entries	0
Robustness Score	0.9810

---

Evaluation completed by RouterArena automated workflow

[sashakolpakov]

Hello, @xufan866 @yl231 - just checking in on this PR. Please let me know if you need any additional input from me.