#1 Official DolphinBench · 91.17% ($24.54) · 81.50% Luna ($1.35) · 57.33% ($0 Local) 75 / 75 SHA-256 Gates (90/90 w/ DABstep) · 333K+ Queries · 48 Raw JSON + 1 CI = 49 Audit Logs 96 GB RTX PRO 6000 Blackwell · Local Arctic 1024d
Updated 2026-11-04

Sovereign LLM Router & Agentic Benchmark Arena

Cryptographically pinned (SHA-256), zero-leakage evaluation across 12 staged external benchmarks (333,000+ queries/traces · 48 raw JSON audit logs + 1 live PR #169 receipt bundle = 49 audit logs · 75/75 dataset gates / 90/90 ecosystem gates): Official DolphinBench 600-Task Suite (27 runs / 16,200 gpt-5.6-sol traces across 6 model families · #1 Overall 91.17% [547/600] Argon Low @ $24.54, +20.50 pp over #1 Mem0 70.67% @ $96.21 · 89.17% Argon Low + Grounded / Argon Med [Argon Low Oracle@2 = 94.00%] · Model-Matched GPT 5.6 Luna (High, + Grounded) 81.50% [489/600] @ $1.35 [30.03s = 1.57x faster than Luna High Std. 81.33%, +10.83 pp & 71.1x cheaper vs Mem0] & Luna (Low, Std.) 76.67% [460/600] @ $1.10 [87.4x cheaper; Luna Oracle@5 = 91.83%] · #1 Open-Weights 57.33% [344/600] krush420/gemma4-31b-sre-lora [Wave-6 QLoRA+DPO + Grounded v0.3] @ $0.00 [+3.17 pp over 54.17% Stock UUID 288671ad…, Gemma 4 Oracle@6 = 71.83%] beating all 8 non-Luna cloud stacks · 30B Wave-4 + Grounded v0.3 43.00% [42.00% + INV-STACK-10 2PC, -95.2% dup MCP calls 42→2, 58.50% Wave-4 Oracle@3, 61.17% 30B Oracle@5] · 97.83% [587/600] Oracle@27 & 83.17% [499/600] Open-Weights Oracle@17 · 8/8 queued + 19 validated controls [27/27 validated]), Martian RouterBench (73K ID + OOD), ACL 2026 LLMRouterBench (12.6K prompts × 13 frontier models), RouteWorks RouterArena (official CI PR #169 Run #780: 0.7263 · #13/34 · 74.44% Acc @ $0.3802/1K · 91.43% Rob. · 0 Abnormal + live SSE aborts), ETH SpyLab AgentDojo v1.2.2 (1,046 episodes), ACL 2025 CoIR (2.39M docs · 162K Qs), BIRD-SQL Mini-Dev (500/500), UC Berkeley BFCL v3 Live (2,251/2,251), Aider Polyglot (225/225), RepoBench-R (48K), LegalBench-RAG (6.9K), and ICLR 2026 BEAM (2,000 probes up to 10M tokens).

Paper #27: #1 DolphinBench (91.17% | 81.50% $1.35 | 57.33% $0) → Paper #26: Wave-4 Ablation → Dataset & 48 Raw JSON + 1 CI Logs → HF Adapter: gemma4-31b-sre-lora (57.33%) → GitHub: krusch-stack → GitHub: krusch-cascade-router → RouterArena CI (PR #169 · 0.7263) →
Official DolphinBench (600 Qs × 27 Runs · 6 Families)
#1 · 91.17% ($24.54)
81.50% Luna Grounded @ $1.35 (71.1x cheaper vs Mem0) · 57.33% gemma4-31b-sre-lora (#1 Open @ $0) · 97.83% Oracle@27
Martian RouterBench (73K ID + OOD)
#1 · 0.7488 / 0.7695
-72.63% cost @ 98% GPT-4 · #1 OOD (0.7268 / 0.7455) beating ZeroRouter where KNN/SVM/MLP collapse
ACL 2026 LLMRouterBench (n=5,563)
#1 · 0.4269
Lowest Pareto Dist · -83.95% cost vs GPT-5 · 2.89x RouteLLM on unseen ARC-AGI
ETH SpyLab AgentDojo v1.2.2
0.00% ASR (0/949)
19.28% → 0.00% Targeted ASR across 1,046 episodes · 0.00 pp benign utility loss (83 blocks)
ACL 2025 CoIR (10/10 · 2.39M Docs)
#1 · 56.40% NDCG@10
73.88% Macro R@10 across 162,216 Qs · +7.42 pp over BM25 · +8.00 pp codefeedback-mt
2PC SQL, Deferred MCP & Polyglot
40.2% EX · -80.7% Tok
1.55x BIRD-SQL (100% valid SQL) · 66.28% BFCL Live (2,251 Qs) · 51.56% Polyglot (2.64x)
Cost vs. Accuracy Pareto Frontier — Official ACL 2026 LLMRouterBench (13 Frontier Models)
Rank / Policy Pareto Dist ↓ Micro Acc ↑ Macro Acc ↑ Cost / 1K ↓ Cost Save AIQ AUC ↑ Unseen ARC-AGI Dispatch Latency
Martian RouterBench (arXiv:2403.12031) — 11-Model In-Distribution & Leave-One-Suite-Out OOD
Rank / AbstractRouter Micro AIQ ↑ Macro-7 AIQ ↑ Macro-9 AIQ ↑ Pareto Dist ↓ Cost @ 98% GPT-4 Cost @ 95% GPT-4 Cold Latency
Per-Suite AIQ Breakdown (9 Evaluation Suites): Honest OOD note — in 0shot_ood leave-chinese-out (n=236), monolingual English training sets std(cjk_ratio) ≈ 0 (1e-4 floor), scoring 0.0811 on krusch_cascade (vs. #1 0.4880 in 0shot_id and 0.3059 in 5shot_ood); zero post-hoc test tuning was applied.
Official RouteWorks/RouterArena CI Leaderboard (8,400 Queries + 420 Robustness)
Official PR #169 github-actions[bot] CI Run #780 (main @ 44cc1ff · 8,400/8,400 live OpenRouter gen-… receipts sha256:bede6659ebc8…): 0.7263 (72.63, #13 / 34) · 74.44% Acc @ $0.3802 / 1K ($3.1939 total, $0.00 router overhead) · 91.43% Robustness · 0 Abnormal — beating OrcaRouter-Adaptive (72.48 @ $0.83/1K, 22.62% rob.), Microsoft Azure-Model-Router (70.37 @ $0.75/1K, 2.0x cheaper), OpenAI GPT-5 (64.32 @ $10.02/1K, 26.4x cheaper), Martian (57.56), NotDiamond (57.29 @ $4.10/1K), and RouteLLM (48.07).
Rank Router Submission Arena Score Accuracy Cost / 1K
Hermetic 96GB G4 Live Token Generation & SSE Abort
809 live sub_10 generations + 420 noise-injected robustness runs (all aborted tokens billed)
Live Policy Acc Mean TTFT Cost / 1K SSE Aborts (Billed Tok)
6-Language Aider Polyglot Multi-Step Trajectory Routing (Pass@1 → 2PC Verify-on-Stop Pass@2)
60 Exercism repositories × 4 policies = 240 sandboxed compile/test trajectories (skipped tests forcibly activated)
Multi-Step Policy Pass@1 Pass@2 (2PC) Self-Healed Router Overhead / Step Total Cost Pass@2/$ vs 120B
Official DolphinBench 600-Task Suite (#1 Overall 91.17% · 81.50% Luna Grounded @ $1.35 · #1 Open-Weights 57.33% @ $0) & Sovereign Wave-2 Benchmarks
Unified KruschStack 16-Row Sovereign Scorecard (75 / 75 Dataset Gates · 90 / 90 Ecosystem Gates · 12 Staged Suites)
Click any SHA-256 badge to copy its full 64-hex digest, or open the raw JSON log / live CI receipt bundle
Stack Pillar External Benchmark Verified Result Controlled Comparison Raw Log & SHA-256
Complete 49-Artifact Cryptographic SHA-256 Provenance Ledger (48 Raw JSON Audit Logs + 1 Live PR #169 Receipt Bundle)
Every staged artifact in krush420/sovereign-g4-benchmarks (63 dataset files across 14 splits) + official RouteWorks/RouterArena PR #169 live receipt bundle
# Audit Artifact Suite / Configuration SHA-256 Digest