42.1%
BenchMaxxed.
How far private-test scores fall below predictions from public benchmarks.
Public scores. Private reality.
Click a model. See where the gap comes from.
Right: a bigger private-score gap. Up: more public wins become private losses.
Predicted from SciCode + Terminal-Bench 4.0. Below the line: actual private scores fall short.
Predicted from GDPval-AA v2.1 + GDP.pdf · all-pass. Below the line: actual private scores fall short.
Not enough evidence to score.
The public benchmarks do not predict this private test reliably enough.
Predicted from EnterpriseOps-Gym-AA + Terminal-Bench 4.0.
Predicted from SciCode + GDP.pdf · all-pass. Below the line: actual private scores fall short.
Predicted from GPQA Diamond + Humanity’s Last Exam. Below the line: actual private scores fall short.
Predicted from GDP.pdf · all-pass + AA-LCR v1.1. Below the line: actual private scores fall short.
Not enough evidence to score.
The public benchmarks do not predict this private test reliably enough.
Predicted from Terminal-Bench 4.0 + Terminal-Bench 2.1.
Not enough evidence to score.
The public benchmarks do not predict this private test reliably enough.
Predicted from AA-LCR v1.1 + GDP.pdf · all-pass.
BenchMaxxed = how far private-test scores fall below predictions from public benchmarks. A score gap, not proof of training intent.
Put the gap under a microscope.
Higher means a larger private-score shortfall.
Not “more guilty.” Open any row to inspect it.
01MiniMax-M3MiniMax · Published configuration · 4/8 private tests42.1% BenchMaxxed
Source variant: MiniMax-M3
| Private test | Actual | Predicted | BenchMaxxed |
|---|---|---|---|
| FrontierCode 1.1 ↗ | 23.0% | 32.6% | 29.5% |
| AA-Briefcase v1.1 · rubric ↗ | 30.3% | 37.0% | 18.2% |
| AutomationBench-AA ↗ | 21.3% | — | Unscored |
| AA-AnalystAgent · pass^5 ↗ | 10.0% | 25.9% | 61.4% |
| AA-Omniscience · accuracy ↗ | 16.7% | 41.2% | 59.5% |
| Harvey LAB-AA v1.1 ↗ | 0.0% | — | Unscored |
| ITBench-AA ↗ | — | — | Unscored |
| MLCR-AA · private holdout ↗ | 17.2% | — | Unscored |
Public benchmarks and calculation details
9.6 points below prediction. Rank reversals: 6.4%. Provider-removal sensitivity: 25.2–32.4%.
SciCode → 34.6% predicted. Correlation ρ 0.58. 21 models from 13 other providers.
Terminal-Bench 4.0 → 31.6% predicted. Correlation ρ 0.80. 21 models from 13 other providers.
6.7 points below prediction. Rank reversals: 1.9%. Provider-removal sensitivity: 16.9–19.9%.
GDPval-AA v2.1 → 33.7% predicted. Correlation ρ 0.87. 21 models from 13 other providers.
GDP.pdf · all-pass → 42.3% predicted. Correlation ρ 0.70. 21 models from 13 other providers.
Two supported public predictors required
Terminal-Bench 4.0 → 43.7% predicted. Correlation ρ 0.70. 21 models from 13 other providers.
15.9 points below prediction. Rank reversals: 9.1%. Provider-removal sensitivity: 53.2–64.7%. Prediction extends outside the reference range.
SciCode → 30.0% predicted. Correlation ρ 0.64. 15 models from 8 other providers.
GDP.pdf · all-pass → 22.5% predicted. Correlation ρ 0.70. 15 models from 8 other providers.
24.5 points below prediction. Rank reversals: 35.7%. Provider-removal sensitivity: 55.1–61.5%.
GPQA Diamond → 44.2% predicted. Correlation ρ 0.82. 12 models from 9 other providers.
Humanity’s Last Exam → 37.7% predicted. Correlation ρ 0.76. 21 models from 13 other providers.
Two supported public predictors required
GDP.pdf · all-pass → 0.0% predicted. Correlation ρ 0.67. 15 models from 9 other providers.
Private anchor unavailable
Two supported public predictors required
GDP.pdf · all-pass → 16.8% predicted. Correlation ρ 0.40. 20 models from 12 other providers.
Average of scored private tests. Missing tests never count as zero. Score stability shows provider-removal sensitivity, not confidence intervals. A zero does not establish clean training.
02Muse GlimmerMeta · High · 4/8 private tests27.6% BenchMaxxed
Source variant: Muse Glimmer (High)
| Private test | Actual | Predicted | BenchMaxxed |
|---|---|---|---|
| FrontierCode 1.1 ↗ | 13.9% | 32.0% | 56.5% |
| AA-Briefcase v1.1 · rubric ↗ | 11.5% | 22.3% | 48.3% |
| AutomationBench-AA ↗ | 6.8% | — | Unscored |
| AA-AnalystAgent · pass^5 ↗ | 20.0% | 21.2% | 5.5% |
| AA-Omniscience · accuracy ↗ | 27.0% | 1.7% | 0.0% |
| Harvey LAB-AA v1.1 ↗ | — | — | Unscored |
| ITBench-AA ↗ | — | — | Unscored |
| MLCR-AA · private holdout ↗ | 20.0% | — | Unscored |
Public benchmarks and calculation details
18.1 points below prediction. Rank reversals: 1.8%. Provider-removal sensitivity: 53.4–58.1%. Prediction extends outside the reference range.
SciCode → 33.1% predicted. Correlation ρ 0.61. 20 models from 13 other providers.
Terminal-Bench 4.0 → 31.4% predicted. Correlation ρ 0.82. 20 models from 13 other providers.
10.7 points below prediction. Rank reversals: 3.8%. Provider-removal sensitivity: 44.3–52.2%. Prediction extends outside the reference range.
GDPval-AA v2.1 → 10.0% predicted. Correlation ρ 0.86. 20 models from 13 other providers.
GDP.pdf · all-pass → 42.3% predicted. Correlation ρ 0.68. 20 models from 13 other providers.
Two supported public predictors required
Terminal-Bench 4.0 → 41.2% predicted. Correlation ρ 0.73. 20 models from 13 other providers.
1.2 points below prediction. Rank reversals: 0.0%. Provider-removal sensitivity: 0.0–18.5%. Prediction extends outside the reference range.
SciCode → 25.3% predicted. Correlation ρ 0.69. 14 models from 8 other providers.
GDP.pdf · all-pass → 18.2% predicted. Correlation ρ 0.82. 14 models from 8 other providers.
0.0 points below prediction. Rank reversals: 0.0%. Provider-removal sensitivity: 0.0–0.0%. Prediction extends outside the reference range.
GPQA Diamond → 0.0% predicted. Correlation ρ 0.66. 11 models from 9 other providers.
Humanity’s Last Exam → 2.9% predicted. Correlation ρ 0.76. 20 models from 13 other providers.
Private anchor unavailable
Private anchor unavailable
Two supported public predictors required
GDP.pdf · all-pass → 14.2% predicted. Correlation ρ 0.42. 19 models from 12 other providers.
Average of scored private tests. Missing tests never count as zero. Score stability shows provider-removal sensitivity, not confidence intervals. A zero does not establish clean training.
03InklingThinking Machines · Xhigh · 4/8 private tests26.0% BenchMaxxed
Source variant: Inkling (Xhigh)
| Private test | Actual | Predicted | BenchMaxxed |
|---|---|---|---|
| FrontierCode 1.1 ↗ | 15.8% | 32.4% | 51.2% |
| AA-Briefcase v1.1 · rubric ↗ | 19.3% | 33.4% | 42.3% |
| AutomationBench-AA ↗ | 5.0% | — | Unscored |
| AA-AnalystAgent · pass^5 ↗ | 23.8% | 26.6% | 10.7% |
| AA-Omniscience · accuracy ↗ | 41.5% | 10.9% | 0.0% |
| Harvey LAB-AA v1.1 ↗ | 0.0% | — | Unscored |
| ITBench-AA ↗ | — | — | Unscored |
| MLCR-AA · private holdout ↗ | 12.2% | — | Unscored |
Public benchmarks and calculation details
16.6 points below prediction. Rank reversals: 3.3%. Provider-removal sensitivity: 47.8–53.2%.
SciCode → 34.5% predicted. Correlation ρ 0.58. 21 models from 13 other providers.
Terminal-Bench 4.0 → 31.3% predicted. Correlation ρ 0.80. 21 models from 13 other providers.
14.1 points below prediction. Rank reversals: 6.0%. Provider-removal sensitivity: 40.7–43.9%.
GDPval-AA v2.1 → 25.6% predicted. Correlation ρ 0.87. 21 models from 13 other providers.
GDP.pdf · all-pass → 45.2% predicted. Correlation ρ 0.71. 21 models from 13 other providers.
Two supported public predictors required
Terminal-Bench 4.0 → 43.3% predicted. Correlation ρ 0.70. 21 models from 13 other providers.
2.8 points below prediction. Rank reversals: 0.0%. Provider-removal sensitivity: 0.0–21.6%.
SciCode → 28.3% predicted. Correlation ρ 0.63. 15 models from 8 other providers.
GDP.pdf · all-pass → 25.2% predicted. Correlation ρ 0.71. 15 models from 8 other providers.
0.0 points below prediction. Rank reversals: 0.0%. Provider-removal sensitivity: 0.0–0.0%.
GPQA Diamond → 3.7% predicted. Correlation ρ 0.82. 12 models from 9 other providers.
Humanity’s Last Exam → 18.3% predicted. Correlation ρ 0.82. 21 models from 13 other providers.
Two supported public predictors required
GDP.pdf · all-pass → 0.9% predicted. Correlation ρ 0.69. 15 models from 9 other providers.
Private anchor unavailable
Two supported public predictors required
GDP.pdf · all-pass → 21.5% predicted. Correlation ρ 0.39. 20 models from 12 other providers.
Average of scored private tests. Missing tests never count as zero. Score stability shows provider-removal sensitivity, not confidence intervals. A zero does not establish clean training.
04Qwen3.8 27BAlibaba · Xhigh · 4/8 private tests22.5% BenchMaxxed
Source variant: Qwen3.8 27B (Xhigh)
| Private test | Actual | Predicted | BenchMaxxed |
|---|---|---|---|
| FrontierCode 1.1 ↗ | 34.2% | 30.1% | 0.0% |
| AA-Briefcase v1.1 · rubric ↗ | 47.8% | 44.0% | 0.0% |
| AutomationBench-AA ↗ | 48.2% | — | Unscored |
| AA-AnalystAgent · pass^5 ↗ | 18.8% | 29.1% | 35.5% |
| AA-Omniscience · accuracy ↗ | 15.6% | 34.3% | 54.6% |
| Harvey LAB-AA v1.1 ↗ | — | — | Unscored |
| ITBench-AA ↗ | — | — | Unscored |
| MLCR-AA · private holdout ↗ | 21.7% | — | Unscored |
Public benchmarks and calculation details
0.0 points below prediction. Rank reversals: 0.0%. Provider-removal sensitivity: 0.0–0.0%.
SciCode → 29.1% predicted. Correlation ρ 0.61. 20 models from 13 other providers.
Terminal-Bench 4.0 → 30.7% predicted. Correlation ρ 0.81. 20 models from 13 other providers.
0.0 points below prediction. Rank reversals: 1.9%. Provider-removal sensitivity: 0.0–0.0%.
GDPval-AA v2.1 → 41.3% predicted. Correlation ρ 0.90. 20 models from 13 other providers.
GDP.pdf · all-pass → 48.5% predicted. Correlation ρ 0.71. 20 models from 13 other providers.
Two supported public predictors required
Terminal-Bench 4.0 → 39.6% predicted. Correlation ρ 0.74. 20 models from 13 other providers.
10.3 points below prediction. Rank reversals: 14.7%. Provider-removal sensitivity: 29.3–43.4%.
SciCode → 25.7% predicted. Correlation ρ 0.67. 14 models from 8 other providers.
GDP.pdf · all-pass → 32.0% predicted. Correlation ρ 0.71. 14 models from 8 other providers.
18.7 points below prediction. Rank reversals: 21.2%. Provider-removal sensitivity: 43.8–57.0%.
GPQA Diamond → 35.2% predicted. Correlation ρ 0.76. 11 models from 9 other providers.
Humanity’s Last Exam → 33.4% predicted. Correlation ρ 0.75. 20 models from 13 other providers.
Private anchor unavailable
Private anchor unavailable
Two supported public predictors required
GDP.pdf · all-pass → 27.4% predicted. Correlation ρ 0.44. 19 models from 12 other providers.
Average of scored private tests. Missing tests never count as zero. Score stability shows provider-removal sensitivity, not confidence intervals. A zero does not establish clean training.
05GLM-5.3-FlashZ AI · Published configuration · 3/8 private tests13.1% BenchMaxxed
Source variant: GLM-5.3-Flash
| Private test | Actual | Predicted | BenchMaxxed |
|---|---|---|---|
| FrontierCode 1.1 ↗ | 41.0% | 39.9% | 0.0% |
| AA-Briefcase v1.1 · rubric ↗ | 46.4% | 51.3% | 9.6% |
| AutomationBench-AA ↗ | 60.4% | — | Unscored |
| AA-AnalystAgent · pass^5 ↗ | — | — | Unscored |
| AA-Omniscience · accuracy ↗ | 27.5% | 39.0% | 29.6% |
| Harvey LAB-AA v1.1 ↗ | 0.8% | — | Unscored |
| ITBench-AA ↗ | 51.2% | — | Unscored |
| MLCR-AA · private holdout ↗ | 51.1% | — | Unscored |
Public benchmarks and calculation details
0.0 points below prediction. Rank reversals: 10.1%. Provider-removal sensitivity: 0.0–0.0%.
SciCode → 37.9% predicted. Correlation ρ 0.66. 20 models from 13 other providers.
Terminal-Bench 4.0 → 41.1% predicted. Correlation ρ 0.87. 20 models from 13 other providers.
4.9 points below prediction. Rank reversals: 20.2%. Provider-removal sensitivity: 7.5–10.8%.
GDPval-AA v2.1 → 54.4% predicted. Correlation ρ 0.92. 20 models from 13 other providers.
GDP.pdf · all-pass → 46.5% predicted. Correlation ρ 0.74. 20 models from 13 other providers.
Two supported public predictors required
Terminal-Bench 4.0 → 52.3% predicted. Correlation ρ 0.76. 20 models from 13 other providers.
Private anchor unavailable
11.5 points below prediction. Rank reversals: 12.1%. Provider-removal sensitivity: 16.3–32.5%.
GPQA Diamond → 38.1% predicted. Correlation ρ 0.74. 11 models from 9 other providers.
Humanity’s Last Exam → 39.9% predicted. Correlation ρ 0.77. 20 models from 13 other providers.
Two supported public predictors required
GDP.pdf · all-pass → 1.6% predicted. Correlation ρ 0.66. 14 models from 9 other providers.
Two supported public predictors required
Two supported public predictors required
GDP.pdf · all-pass → 20.2% predicted. Correlation ρ 0.61. 19 models from 12 other providers.
Average of scored private tests. Missing tests never count as zero. Score stability shows provider-removal sensitivity, not confidence intervals. A zero does not establish clean training.
06GLM-5.3Z AI · Max · 3/8 private tests9.5% BenchMaxxed
Source variant: GLM-5.3 (Max)
| Private test | Actual | Predicted | BenchMaxxed |
|---|---|---|---|
| FrontierCode 1.1 ↗ | 40.1% | 44.5% | 9.9% |
| AA-Briefcase v1.1 · rubric ↗ | 51.3% | 49.7% | 0.0% |
| AutomationBench-AA ↗ | 62.2% | — | Unscored |
| AA-AnalystAgent · pass^5 ↗ | — | — | Unscored |
| AA-Omniscience · accuracy ↗ | 33.9% | 41.5% | 18.5% |
| Harvey LAB-AA v1.1 ↗ | 0.3% | — | Unscored |
| ITBench-AA ↗ | 46.1% | — | Unscored |
| MLCR-AA · private holdout ↗ | 48.3% | — | Unscored |
Public benchmarks and calculation details
4.4 points below prediction. Rank reversals: 35.0%. Provider-removal sensitivity: 8.5–10.5%.
SciCode → 44.6% predicted. Correlation ρ 0.66. 20 models from 13 other providers.
Terminal-Bench 4.0 → 44.5% predicted. Correlation ρ 0.87. 20 models from 13 other providers.
0.0 points below prediction. Rank reversals: 9.1%. Provider-removal sensitivity: 0.0–0.0%.
GDPval-AA v2.1 → 54.7% predicted. Correlation ρ 0.92. 20 models from 13 other providers.
GDP.pdf · all-pass → 41.9% predicted. Correlation ρ 0.74. 20 models from 13 other providers.
Two supported public predictors required
Terminal-Bench 4.0 → 58.2% predicted. Correlation ρ 0.76. 20 models from 13 other providers.
Private anchor unavailable
7.7 points below prediction. Rank reversals: 12.1%. Provider-removal sensitivity: 6.4–21.4%.
GPQA Diamond → 40.2% predicted. Correlation ρ 0.74. 11 models from 9 other providers.
Humanity’s Last Exam → 42.8% predicted. Correlation ρ 0.77. 20 models from 13 other providers.
Two supported public predictors required
GDP.pdf · all-pass → 0.0% predicted. Correlation ρ 0.66. 14 models from 9 other providers.
Two supported public predictors required
Two supported public predictors required
GDP.pdf · all-pass → 14.9% predicted. Correlation ρ 0.61. 19 models from 12 other providers.
Average of scored private tests. Missing tests never count as zero. Score stability shows provider-removal sensitivity, not confidence intervals. A zero does not establish clean training.
07Muse Spark 1.3Meta · Max · 4/8 private tests8.4% BenchMaxxed
Source variant: Muse Spark 1.3 (Max)
| Private test | Actual | Predicted | BenchMaxxed |
|---|---|---|---|
| FrontierCode 1.1 ↗ | 40.6% | 42.8% | 5.0% |
| AA-Briefcase v1.1 · rubric ↗ | 58.7% | 55.9% | 0.0% |
| AutomationBench-AA ↗ | 57.9% | — | Unscored |
| AA-AnalystAgent · pass^5 ↗ | 37.5% | 50.2% | 25.3% |
| AA-Omniscience · accuracy ↗ | 43.6% | 45.1% | 3.4% |
| Harvey LAB-AA v1.1 ↗ | 8.9% | — | Unscored |
| ITBench-AA ↗ | 33.2% | — | Unscored |
| MLCR-AA · private holdout ↗ | 43.3% | — | Unscored |
Public benchmarks and calculation details
2.1 points below prediction. Rank reversals: 29.4%. Provider-removal sensitivity: 3.0–5.8%.
SciCode → 44.7% predicted. Correlation ρ 0.61. 20 models from 13 other providers.
Terminal-Bench 4.0 → 41.7% predicted. Correlation ρ 0.82. 20 models from 13 other providers.
0.0 points below prediction. Rank reversals: 5.7%. Provider-removal sensitivity: 0.0–0.0%.
GDPval-AA v2.1 → 54.5% predicted. Correlation ρ 0.86. 20 models from 13 other providers.
GDP.pdf · all-pass → 58.1% predicted. Correlation ρ 0.68. 20 models from 13 other providers.
Two supported public predictors required
Terminal-Bench 4.0 → 57.0% predicted. Correlation ρ 0.73. 20 models from 13 other providers.
12.7 points below prediction. Rank reversals: 47.5%. Provider-removal sensitivity: 18.3–27.8%.
SciCode → 46.9% predicted. Correlation ρ 0.69. 14 models from 8 other providers.
GDP.pdf · all-pass → 52.5% predicted. Correlation ρ 0.82. 14 models from 8 other providers.
1.5 points below prediction. Rank reversals: 14.2%. Provider-removal sensitivity: 2.6–9.8%.
GPQA Diamond → 44.5% predicted. Correlation ρ 0.66. 11 models from 9 other providers.
Humanity’s Last Exam → 45.6% predicted. Correlation ρ 0.76. 20 models from 13 other providers.
Two supported public predictors required
GDP.pdf · all-pass → 5.4% predicted. Correlation ρ 0.70. 15 models from 9 other providers.
Two supported public predictors required
Two supported public predictors required
GDP.pdf · all-pass → 33.0% predicted. Correlation ρ 0.42. 19 models from 12 other providers.
Average of scored private tests. Missing tests never count as zero. Score stability shows provider-removal sensitivity, not confidence intervals. A zero does not establish clean training.
08Qwen3.8 Max (0902)Alibaba · 0902 · 4/8 private tests7.2% BenchMaxxed
Source variant: Qwen3.8 Max (0902)
| Private test | Actual | Predicted | BenchMaxxed |
|---|---|---|---|
| FrontierCode 1.1 ↗ | 43.2% | 40.2% | 0.0% |
| AA-Briefcase v1.1 · rubric ↗ | 58.0% | 54.2% | 0.0% |
| AutomationBench-AA ↗ | 56.2% | — | Unscored |
| AA-AnalystAgent · pass^5 ↗ | 45.0% | 38.0% | 0.0% |
| AA-Omniscience · accuracy ↗ | 31.7% | 44.4% | 28.6% |
| Harvey LAB-AA v1.1 ↗ | — | — | Unscored |
| ITBench-AA ↗ | 40.3% | — | Unscored |
| MLCR-AA · private holdout ↗ | 20.0% | — | Unscored |
Public benchmarks and calculation details
0.0 points below prediction. Rank reversals: 11.4%. Provider-removal sensitivity: 0.0–0.0%.
SciCode → 35.3% predicted. Correlation ρ 0.61. 20 models from 13 other providers.
Terminal-Bench 4.0 → 43.0% predicted. Correlation ρ 0.81. 20 models from 13 other providers.
0.0 points below prediction. Rank reversals: 0.0%. Provider-removal sensitivity: 0.0–0.0%.
GDPval-AA v2.1 → 54.0% predicted. Correlation ρ 0.90. 20 models from 13 other providers.
GDP.pdf · all-pass → 54.6% predicted. Correlation ρ 0.71. 20 models from 13 other providers.
Two supported public predictors required
Terminal-Bench 4.0 → 58.0% predicted. Correlation ρ 0.74. 20 models from 13 other providers.
0.0 points below prediction. Rank reversals: 4.1%. Provider-removal sensitivity: 0.0–0.0%.
SciCode → 33.3% predicted. Correlation ρ 0.67. 14 models from 8 other providers.
GDP.pdf · all-pass → 42.2% predicted. Correlation ρ 0.71. 14 models from 8 other providers.
12.7 points below prediction. Rank reversals: 19.1%. Provider-removal sensitivity: 22.6–30.9%.
GPQA Diamond → 44.7% predicted. Correlation ρ 0.76. 11 models from 9 other providers.
Humanity’s Last Exam → 44.1% predicted. Correlation ρ 0.75. 20 models from 13 other providers.
Private anchor unavailable
Two supported public predictors required
Two supported public predictors required
GDP.pdf · all-pass → 35.0% predicted. Correlation ρ 0.44. 19 models from 12 other providers.
Average of scored private tests. Missing tests never count as zero. Score stability shows provider-removal sensitivity, not confidence intervals. A zero does not establish clean training.
09Gemini 3.8 FlashGoogle · High · 3/8 private tests5.8% BenchMaxxed
Source variant: Gemini 3.8 Flash (High)
| Private test | Actual | Predicted | BenchMaxxed |
|---|---|---|---|
| FrontierCode 1.1 ↗ | 36.4% | 38.9% | 6.6% |
| AA-Briefcase v1.1 · rubric ↗ | 42.1% | 47.3% | 10.9% |
| AutomationBench-AA ↗ | 59.9% | — | Unscored |
| AA-AnalystAgent · pass^5 ↗ | — | — | Unscored |
| AA-Omniscience · accuracy ↗ | 54.6% | 46.8% | 0.0% |
| Harvey LAB-AA v1.1 ↗ | 0.0% | — | Unscored |
| ITBench-AA ↗ | 52.5% | — | Unscored |
| MLCR-AA · private holdout ↗ | 21.7% | — | Unscored |
Public benchmarks and calculation details
2.6 points below prediction. Rank reversals: 18.8%. Provider-removal sensitivity: 2.1–8.7%.
SciCode → 42.9% predicted. Correlation ρ 0.66. 21 models from 13 other providers.
Terminal-Bench 4.0 → 36.4% predicted. Correlation ρ 0.82. 21 models from 13 other providers.
5.1 points below prediction. Rank reversals: 19.0%. Provider-removal sensitivity: 9.0–12.8%.
GDPval-AA v2.1 → 43.1% predicted. Correlation ρ 0.88. 21 models from 13 other providers.
GDP.pdf · all-pass → 52.8% predicted. Correlation ρ 0.76. 21 models from 13 other providers.
Two supported public predictors required
Terminal-Bench 4.0 → 47.4% predicted. Correlation ρ 0.76. 21 models from 13 other providers.
Private anchor unavailable
0.0 points below prediction. Rank reversals: 3.7%. Provider-removal sensitivity: 0.0–0.0%.
GPQA Diamond → 50.3% predicted. Correlation ρ 0.67. 12 models from 9 other providers.
Humanity’s Last Exam → 44.0% predicted. Correlation ρ 0.76. 21 models from 13 other providers.
Two supported public predictors required
GDP.pdf · all-pass → 4.0% predicted. Correlation ρ 0.76. 15 models from 9 other providers.
Two supported public predictors required
Two supported public predictors required
GDP.pdf · all-pass → 30.8% predicted. Correlation ρ 0.43. 20 models from 12 other providers.
Average of scored private tests. Missing tests never count as zero. Score stability shows provider-removal sensitivity, not confidence intervals. A zero does not establish clean training.
10Grok 4.7SpaceXAI · Xhigh · 3/8 private tests5.6% BenchMaxxed
Source variant: Grok 4.7 (Xhigh)
| Private test | Actual | Predicted | BenchMaxxed |
|---|---|---|---|
| FrontierCode 1.1 ↗ | 33.8% | 40.7% | 16.9% |
| AA-Briefcase v1.1 · rubric ↗ | 55.4% | 54.8% | 0.0% |
| AutomationBench-AA ↗ | 65.6% | — | Unscored |
| AA-AnalystAgent · pass^5 ↗ | — | — | Unscored |
| AA-Omniscience · accuracy ↗ | 47.5% | — | Unscored |
| Harvey LAB-AA v1.1 ↗ | 9.4% | 2.2% | 0.0% |
| ITBench-AA ↗ | 42.1% | — | Unscored |
| MLCR-AA · private holdout ↗ | 15.0% | — | Unscored |
Public benchmarks and calculation details
6.9 points below prediction. Rank reversals: 28.5%. Provider-removal sensitivity: 14.7–17.8%.
SciCode → 43.7% predicted. Correlation ρ 0.67. 21 models from 13 other providers.
Terminal-Bench 4.0 → 38.8% predicted. Correlation ρ 0.82. 21 models from 13 other providers.
0.0 points below prediction. Rank reversals: 8.5%. Provider-removal sensitivity: 0.0–0.8%.
GDPval-AA v2.1 → 57.1% predicted. Correlation ρ 0.89. 21 models from 13 other providers.
GDP.pdf · all-pass → 51.5% predicted. Correlation ρ 0.74. 21 models from 13 other providers.
Two supported public predictors required
Terminal-Bench 4.0 → 50.9% predicted. Correlation ρ 0.79. 21 models from 13 other providers.
Private anchor unavailable
Two supported public predictors required
Humanity’s Last Exam → 36.6% predicted. Correlation ρ 0.76. 21 models from 13 other providers.
0.0 points below prediction. Rank reversals: 0.0%. Provider-removal sensitivity: 0.0–0.0%. Prediction extends outside the reference range.
GDP.pdf · all-pass → 2.8% predicted. Correlation ρ 0.88. 15 models from 9 other providers.
AA-LCR v1.1 → 0.0% predicted. Correlation ρ 0.48. 15 models from 9 other providers.
Two supported public predictors required
Two supported public predictors required
GDP.pdf · all-pass → 29.7% predicted. Correlation ρ 0.44. 20 models from 12 other providers.
Average of scored private tests. Missing tests never count as zero. Score stability shows provider-removal sensitivity, not confidence intervals. A zero does not establish clean training.
11GPT-6 LunaOpenAI · Max · 3/8 private tests5.1% BenchMaxxed
Source variant: GPT-6 Luna (Max)
| Private test | Actual | Predicted | BenchMaxxed |
|---|---|---|---|
| FrontierCode 1.1 ↗ | 41.9% | 35.0% | 0.0% |
| AA-Briefcase v1.1 · rubric ↗ | 41.2% | 48.6% | 15.2% |
| AutomationBench-AA ↗ | 53.2% | — | Unscored |
| AA-AnalystAgent · pass^5 ↗ | 43.8% | 40.7% | 0.0% |
| AA-Omniscience · accuracy ↗ | 43.8% | — | Unscored |
| Harvey LAB-AA v1.1 ↗ | 3.3% | — | Unscored |
| ITBench-AA ↗ | — | — | Unscored |
| MLCR-AA · private holdout ↗ | 16.1% | — | Unscored |
Public benchmarks and calculation details
0.0 points below prediction. Rank reversals: 5.0%. Provider-removal sensitivity: 0.0–0.0%.
SciCode → 37.3% predicted. Correlation ρ 0.78. 19 models from 13 other providers.
Terminal-Bench 4.0 → 33.0% predicted. Correlation ρ 0.82. 19 models from 13 other providers.
7.4 points below prediction. Rank reversals: 32.9%. Provider-removal sensitivity: 12.1–16.3%.
GDPval-AA v2.1 → 41.8% predicted. Correlation ρ 0.93. 19 models from 13 other providers.
GDP.pdf · all-pass → 57.1% predicted. Correlation ρ 0.83. 19 models from 13 other providers.
Two supported public predictors required
Terminal-Bench 4.0 → 43.4% predicted. Correlation ρ 0.68. 19 models from 13 other providers.
0.0 points below prediction. Rank reversals: 8.0%. Provider-removal sensitivity: 0.0–0.0%.
SciCode → 35.1% predicted. Correlation ρ 0.79. 13 models from 8 other providers.
GDP.pdf · all-pass → 46.1% predicted. Correlation ρ 0.80. 13 models from 8 other providers.
Two supported public predictors required
Humanity’s Last Exam → 29.1% predicted. Correlation ρ 0.75. 19 models from 13 other providers.
Two supported public predictors required
GDP.pdf · all-pass → 4.2% predicted. Correlation ρ 0.66. 13 models from 9 other providers.
Private anchor unavailable
Two supported public predictors required
GDP.pdf · all-pass → 39.4% predicted. Correlation ρ 0.54. 18 models from 12 other providers.
Average of scored private tests. Missing tests never count as zero. Score stability shows provider-removal sensitivity, not confidence intervals. A zero does not establish clean training.
12Step 5 PreviewStepFun · Published configuration · 3/8 private tests4.9% BenchMaxxed
Source variant: Step 5 Preview
| Private test | Actual | Predicted | BenchMaxxed |
|---|---|---|---|
| FrontierCode 1.1 ↗ | 38.4% | 42.6% | 9.8% |
| AA-Briefcase v1.1 · rubric ↗ | 47.8% | 48.8% | 2.0% |
| AutomationBench-AA ↗ | 51.0% | — | Unscored |
| AA-AnalystAgent · pass^5 ↗ | 35.0% | 36.0% | 2.8% |
| AA-Omniscience · accuracy ↗ | 41.5% | — | Unscored |
| Harvey LAB-AA v1.1 ↗ | — | — | Unscored |
| ITBench-AA ↗ | 55.6% | — | Unscored |
| MLCR-AA · private holdout ↗ | 16.7% | — | Unscored |
Public benchmarks and calculation details
4.2 points below prediction. Rank reversals: 30.0%. Provider-removal sensitivity: 8.0–10.9%.
SciCode → 45.0% predicted. Correlation ρ 0.66. 21 models from 13 other providers.
Terminal-Bench 4.0 → 41.2% predicted. Correlation ρ 0.84. 21 models from 13 other providers.
1.0 points below prediction. Rank reversals: 10.6%. Provider-removal sensitivity: 1.1–4.0%.
GDPval-AA v2.1 → 50.5% predicted. Correlation ρ 0.90. 21 models from 13 other providers.
GDP.pdf · all-pass → 46.0% predicted. Correlation ρ 0.72. 21 models from 13 other providers.
Two supported public predictors required
Terminal-Bench 4.0 → 56.4% predicted. Correlation ρ 0.77. 21 models from 13 other providers.
1.0 points below prediction. Rank reversals: 20.6%. Provider-removal sensitivity: 0.0–11.5%.
SciCode → 43.2% predicted. Correlation ρ 0.73. 15 models from 8 other providers.
GDP.pdf · all-pass → 28.4% predicted. Correlation ρ 0.71. 15 models from 8 other providers.
Two supported public predictors required
Humanity’s Last Exam → 47.0% predicted. Correlation ρ 0.78. 21 models from 13 other providers.
Private anchor unavailable
Two supported public predictors required
Two supported public predictors required
GDP.pdf · all-pass → 23.9% predicted. Correlation ρ 0.42. 20 models from 12 other providers.
Average of scored private tests. Missing tests never count as zero. Score stability shows provider-removal sensitivity, not confidence intervals. A zero does not establish clean training.
13MiMo-V2.6-ProXiaomi · Published configuration · 3/8 private tests3.5% BenchMaxxed
Source variant: MiMo-V2.6-Pro
| Private test | Actual | Predicted | BenchMaxxed |
|---|---|---|---|
| FrontierCode 1.1 ↗ | 45.4% | 43.5% | 0.0% |
| AA-Briefcase v1.1 · rubric ↗ | 54.7% | 53.5% | 0.0% |
| AutomationBench-AA ↗ | 58.6% | — | Unscored |
| AA-AnalystAgent · pass^5 ↗ | 37.5% | 42.0% | 10.6% |
| AA-Omniscience · accuracy ↗ | 34.8% | — | Unscored |
| Harvey LAB-AA v1.1 ↗ | — | — | Unscored |
| ITBench-AA ↗ | — | — | Unscored |
| MLCR-AA · private holdout ↗ | 18.3% | — | Unscored |
Public benchmarks and calculation details
0.0 points below prediction. Rank reversals: 3.3%. Provider-removal sensitivity: 0.0–0.0%.
SciCode → 46.8% predicted. Correlation ρ 0.60. 21 models from 13 other providers.
Terminal-Bench 4.0 → 41.7% predicted. Correlation ρ 0.81. 21 models from 13 other providers.
0.0 points below prediction. Rank reversals: 8.5%. Provider-removal sensitivity: 0.0–0.0%.
GDPval-AA v2.1 → 55.5% predicted. Correlation ρ 0.89. 21 models from 13 other providers.
GDP.pdf · all-pass → 50.6% predicted. Correlation ρ 0.74. 21 models from 13 other providers.
Two supported public predictors required
Terminal-Bench 4.0 → 55.8% predicted. Correlation ρ 0.75. 21 models from 13 other providers.
4.5 points below prediction. Rank reversals: 22.0%. Provider-removal sensitivity: 0.0–16.8%.
SciCode → 46.6% predicted. Correlation ρ 0.73. 15 models from 8 other providers.
GDP.pdf · all-pass → 37.0% predicted. Correlation ρ 0.71. 15 models from 8 other providers.
Two supported public predictors required
Humanity’s Last Exam → 48.2% predicted. Correlation ρ 0.81. 21 models from 13 other providers.
Private anchor unavailable
Private anchor unavailable
Two supported public predictors required
GDP.pdf · all-pass → 29.0% predicted. Correlation ρ 0.43. 20 models from 12 other providers.
Average of scored private tests. Missing tests never count as zero. Score stability shows provider-removal sensitivity, not confidence intervals. A zero does not establish clean training.
14DeepSeek V4.1 FlashDeepSeek · Max · 2/8 private tests3.3% BenchMaxxed
Source variant: DeepSeek V4.1 Flash (Max)
| Private test | Actual | Predicted | BenchMaxxed |
|---|---|---|---|
| FrontierCode 1.1 ↗ | 44.0% | 38.0% | 0.0% |
| AA-Briefcase v1.1 · rubric ↗ | 45.2% | 48.4% | 6.7% |
| AutomationBench-AA ↗ | 68.9% | — | Unscored |
| AA-AnalystAgent · pass^5 ↗ | — | — | Unscored |
| AA-Omniscience · accuracy ↗ | 46.4% | — | Unscored |
| Harvey LAB-AA v1.1 ↗ | 1.7% | — | Unscored |
| ITBench-AA ↗ | 46.9% | — | Unscored |
| MLCR-AA · private holdout ↗ | 22.8% | — | Unscored |
Public benchmarks and calculation details
0.0 points below prediction. Rank reversals: 0.0%. Provider-removal sensitivity: 0.0–0.0%.
SciCode → 36.7% predicted. Correlation ρ 0.66. 21 models from 13 other providers.
Terminal-Bench 4.0 → 38.8% predicted. Correlation ρ 0.84. 21 models from 13 other providers.
3.2 points below prediction. Rank reversals: 16.5%. Provider-removal sensitivity: 5.3–8.6%.
GDPval-AA v2.1 → 51.3% predicted. Correlation ρ 0.91. 21 models from 13 other providers.
GDP.pdf · all-pass → 43.9% predicted. Correlation ρ 0.72. 21 models from 13 other providers.
Two supported public predictors required
Terminal-Bench 4.0 → 51.4% predicted. Correlation ρ 0.80. 21 models from 13 other providers.
Private anchor unavailable
Two supported public predictors required
Humanity’s Last Exam → 30.5% predicted. Correlation ρ 0.79. 21 models from 13 other providers.
Two supported public predictors required
GDP.pdf · all-pass → 0.0% predicted. Correlation ρ 0.70. 15 models from 9 other providers.
Two supported public predictors required
Two supported public predictors required
GDP.pdf · all-pass → 19.0% predicted. Correlation ρ 0.45. 20 models from 12 other providers.
Average of scored private tests. Missing tests never count as zero. Score stability shows provider-removal sensitivity, not confidence intervals. A zero does not establish clean training.
15Claude Sonnet 5.5Anthropic · Max, Default Fallback · 3/8 private tests2.8% BenchMaxxed
Source variant: Claude Sonnet 5.5 (Max, Default Fallback)
| Private test | Actual | Predicted | BenchMaxxed |
|---|---|---|---|
| FrontierCode 1.1 ↗ | 45.8% | 50.1% | 8.5% |
| AA-Briefcase v1.1 · rubric ↗ | 65.2% | 60.0% | 0.0% |
| AutomationBench-AA ↗ | 71.8% | — | Unscored |
| AA-AnalystAgent · pass^5 ↗ | 57.5% | 43.8% | 0.0% |
| AA-Omniscience · accuracy ↗ | 54.0% | — | Unscored |
| Harvey LAB-AA v1.1 ↗ | 2.8% | — | Unscored |
| ITBench-AA ↗ | — | — | Unscored |
| MLCR-AA · private holdout ↗ | 75.0% | — | Unscored |
Public benchmarks and calculation details
4.3 points below prediction. Rank reversals: 11.1%. Provider-removal sensitivity: 6.7–11.5%. Prediction extends outside the reference range.
SciCode → 45.3% predicted. Correlation ρ 0.51. 18 models from 13 other providers.
Terminal-Bench 4.0 → 52.2% predicted. Correlation ρ 0.78. 18 models from 13 other providers.
0.0 points below prediction. Rank reversals: 0.0%. Provider-removal sensitivity: 0.0–0.0%. Prediction extends outside the reference range.
GDPval-AA v2.1 → 62.8% predicted. Correlation ρ 0.83. 18 models from 13 other providers.
GDP.pdf · all-pass → 55.5% predicted. Correlation ρ 0.67. 18 models from 13 other providers.
Two supported public predictors required
Terminal-Bench 4.0 → 75.8% predicted. Correlation ρ 0.72. 18 models from 13 other providers.
0.0 points below prediction. Rank reversals: 0.0%. Provider-removal sensitivity: 0.0–0.0%. Prediction extends outside the reference range.
SciCode → 40.5% predicted. Correlation ρ 0.46. 12 models from 8 other providers.
GDP.pdf · all-pass → 44.7% predicted. Correlation ρ 0.90. 12 models from 8 other providers.
Two supported public predictors required
Humanity’s Last Exam → 54.6% predicted. Correlation ρ 0.67. 18 models from 13 other providers.
Two supported public predictors required
GDP.pdf · all-pass → 6.0% predicted. Correlation ρ 0.69. 12 models from 9 other providers.
Private anchor unavailable
Two supported public predictors required
Average of scored private tests. Missing tests never count as zero. Score stability shows provider-removal sensitivity, not confidence intervals. A zero does not establish clean training.
16Kimi K3Kimi · Max · 4/8 private tests2.7% BenchMaxxed
Source variant: Kimi K3 (Max)
| Private test | Actual | Predicted | BenchMaxxed |
|---|---|---|---|
| FrontierCode 1.1 ↗ | 43.6% | 37.5% | 0.0% |
| AA-Briefcase v1.1 · rubric ↗ | 51.0% | 49.9% | 0.0% |
| AutomationBench-AA ↗ | 58.3% | — | Unscored |
| AA-AnalystAgent · pass^5 ↗ | 38.8% | 43.5% | 10.8% |
| AA-Omniscience · accuracy ↗ | 47.6% | 43.4% | 0.0% |
| Harvey LAB-AA v1.1 ↗ | 5.3% | — | Unscored |
| ITBench-AA ↗ | 47.7% | — | Unscored |
| MLCR-AA · private holdout ↗ | 38.3% | — | Unscored |
Public benchmarks and calculation details
0.0 points below prediction. Rank reversals: 4.8%. Provider-removal sensitivity: 0.0–0.0%.
SciCode → 45.6% predicted. Correlation ρ 0.61. 21 models from 13 other providers.
Terminal-Bench 4.0 → 33.4% predicted. Correlation ρ 0.87. 21 models from 13 other providers.
0.0 points below prediction. Rank reversals: 7.6%. Provider-removal sensitivity: 0.0–0.5%.
GDPval-AA v2.1 → 47.3% predicted. Correlation ρ 0.89. 21 models from 13 other providers.
GDP.pdf · all-pass → 53.8% predicted. Correlation ρ 0.73. 21 models from 13 other providers.
Two supported public predictors required
Terminal-Bench 4.0 → 43.4% predicted. Correlation ρ 0.74. 21 models from 13 other providers.
4.7 points below prediction. Rank reversals: 19.8%. Provider-removal sensitivity: 0.0–17.9%.
SciCode → 44.2% predicted. Correlation ρ 0.73. 15 models from 8 other providers.
GDP.pdf · all-pass → 42.7% predicted. Correlation ρ 0.74. 15 models from 8 other providers.
0.0 points below prediction. Rank reversals: 0.0%. Provider-removal sensitivity: 0.0–0.4%.
GPQA Diamond → 43.3% predicted. Correlation ρ 0.66. 12 models from 9 other providers.
Humanity’s Last Exam → 43.5% predicted. Correlation ρ 0.75. 21 models from 13 other providers.
Two supported public predictors required
GDP.pdf · all-pass → 3.6% predicted. Correlation ρ 0.72. 15 models from 9 other providers.
Two supported public predictors required
Two supported public predictors required
GDP.pdf · all-pass → 28.0% predicted. Correlation ρ 0.44. 20 models from 12 other providers.
Average of scored private tests. Missing tests never count as zero. Score stability shows provider-removal sensitivity, not confidence intervals. A zero does not establish clean training.
17GPT-6 AstraOpenAI · Max · 4/8 private tests2.3% BenchMaxxed
Source variant: GPT-6 Astra (Max)
| Private test | Actual | Predicted | BenchMaxxed |
|---|---|---|---|
| FrontierCode 1.1 ↗ | 47.0% | 45.8% | 0.0% |
| AA-Briefcase v1.1 · rubric ↗ | 52.0% | 57.0% | 8.8% |
| AutomationBench-AA ↗ | 68.5% | — | Unscored |
| AA-AnalystAgent · pass^5 ↗ | 51.2% | 51.6% | 0.6% |
| AA-Omniscience · accuracy ↗ | 62.6% | 53.5% | 0.0% |
| Harvey LAB-AA v1.1 ↗ | 8.6% | — | Unscored |
| ITBench-AA ↗ | 48.6% | — | Unscored |
| MLCR-AA · private holdout ↗ | 35.0% | — | Unscored |
Public benchmarks and calculation details
0.0 points below prediction. Rank reversals: 2.8%. Provider-removal sensitivity: 0.0–0.0%.
SciCode → 39.7% predicted. Correlation ρ 0.78. 19 models from 13 other providers.
Terminal-Bench 4.0 → 51.4% predicted. Correlation ρ 0.82. 19 models from 13 other providers.
5.0 points below prediction. Rank reversals: 18.6%. Provider-removal sensitivity: 5.4–10.6%. Prediction extends outside the reference range.
GDPval-AA v2.1 → 49.1% predicted. Correlation ρ 0.93. 19 models from 13 other providers.
GDP.pdf · all-pass → 67.1% predicted. Correlation ρ 0.83. 19 models from 13 other providers.
Two supported public predictors required
Terminal-Bench 4.0 → 69.3% predicted. Correlation ρ 0.68. 19 models from 13 other providers.
0.3 points below prediction. Rank reversals: 19.4%. Provider-removal sensitivity: 0.0–9.3%. Prediction extends outside the reference range.
SciCode → 38.3% predicted. Correlation ρ 0.79. 13 models from 8 other providers.
GDP.pdf · all-pass → 64.5% predicted. Correlation ρ 0.80. 13 models from 8 other providers.
0.0 points below prediction. Rank reversals: 3.7%. Provider-removal sensitivity: 0.0–0.0%. Prediction extends outside the reference range.
GPQA Diamond → 53.5% predicted. Correlation ρ 0.67. 12 models from 9 other providers.
Humanity’s Last Exam → 53.5% predicted. Correlation ρ 0.75. 19 models from 13 other providers.
Two supported public predictors required
GDP.pdf · all-pass → 6.9% predicted. Correlation ρ 0.66. 13 models from 9 other providers.
Two supported public predictors required
Two supported public predictors required
GDP.pdf · all-pass → 53.3% predicted. Correlation ρ 0.54. 18 models from 12 other providers.
Average of scored private tests. Missing tests never count as zero. Score stability shows provider-removal sensitivity, not confidence intervals. A zero does not establish clean training.
18GPT-6.1 SolOpenAI · Max · 3/8 private tests1.2% BenchMaxxed
Source variant: GPT-6.1 Sol (Max)
| Private test | Actual | Predicted | BenchMaxxed |
|---|---|---|---|
| FrontierCode 1.1 ↗ | 49.1% | 43.8% | 0.0% |
| AA-Briefcase v1.1 · rubric ↗ | 55.7% | 57.7% | 3.5% |
| AutomationBench-AA ↗ | 64.9% | — | Unscored |
| AA-AnalystAgent · pass^5 ↗ | 50.0% | 49.6% | 0.0% |
| AA-Omniscience · accuracy ↗ | 62.1% | — | Unscored |
| Harvey LAB-AA v1.1 ↗ | 6.9% | — | Unscored |
| ITBench-AA ↗ | — | — | Unscored |
| MLCR-AA · private holdout ↗ | 33.9% | — | Unscored |
Public benchmarks and calculation details
0.0 points below prediction. Rank reversals: 0.0%. Provider-removal sensitivity: 0.0–0.0%.
SciCode → 36.6% predicted. Correlation ρ 0.78. 19 models from 13 other providers.
Terminal-Bench 4.0 → 50.2% predicted. Correlation ρ 0.82. 19 models from 13 other providers.
2.0 points below prediction. Rank reversals: 11.6%. Provider-removal sensitivity: 0.0–5.3%. Prediction extends outside the reference range.
GDPval-AA v2.1 → 50.2% predicted. Correlation ρ 0.93. 19 models from 13 other providers.
GDP.pdf · all-pass → 67.1% predicted. Correlation ρ 0.83. 19 models from 13 other providers.
Two supported public predictors required
Terminal-Bench 4.0 → 67.6% predicted. Correlation ρ 0.68. 19 models from 13 other providers.
0.0 points below prediction. Rank reversals: 15.6%. Provider-removal sensitivity: 0.0–8.4%. Prediction extends outside the reference range.
SciCode → 34.3% predicted. Correlation ρ 0.79. 13 models from 8 other providers.
GDP.pdf · all-pass → 64.5% predicted. Correlation ρ 0.80. 13 models from 8 other providers.
Two supported public predictors required
Humanity’s Last Exam → 50.9% predicted. Correlation ρ 0.75. 19 models from 13 other providers.
Two supported public predictors required
GDP.pdf · all-pass → 6.9% predicted. Correlation ρ 0.66. 13 models from 9 other providers.
Private anchor unavailable
Two supported public predictors required
GDP.pdf · all-pass → 53.3% predicted. Correlation ρ 0.54. 18 models from 12 other providers.
Average of scored private tests. Missing tests never count as zero. Score stability shows provider-removal sensitivity, not confidence intervals. A zero does not establish clean training.
19Claude Fable 5.1Anthropic · Max, Default Fallback · 4/8 private tests0.0% BenchMaxxed
Source variant: Claude Fable 5.1 (Max, Default Fallback)
| Private test | Actual | Predicted | BenchMaxxed |
|---|---|---|---|
| FrontierCode 1.1 ↗ | 49.0% | 47.7% | 0.0% |
| AA-Briefcase v1.1 · rubric ↗ | 61.5% | 57.5% | 0.0% |
| AutomationBench-AA ↗ | 59.4% | — | Unscored |
| AA-AnalystAgent · pass^5 ↗ | 57.5% | 44.9% | 0.0% |
| AA-Omniscience · accuracy ↗ | 67.2% | 52.0% | 0.0% |
| Harvey LAB-AA v1.1 ↗ | 6.4% | — | Unscored |
| ITBench-AA ↗ | 49.5% | — | Unscored |
| MLCR-AA · private holdout ↗ | 71.1% | — | Unscored |
Public benchmarks and calculation details
0.0 points below prediction. Rank reversals: 1.7%. Provider-removal sensitivity: 0.0–0.0%. Prediction extends outside the reference range.
SciCode → 47.3% predicted. Correlation ρ 0.51. 18 models from 13 other providers.
Terminal-Bench 4.0 → 47.8% predicted. Correlation ρ 0.78. 18 models from 13 other providers.
0.0 points below prediction. Rank reversals: 0.0%. Provider-removal sensitivity: 0.0–0.0%. Prediction extends outside the reference range.
GDPval-AA v2.1 → 58.6% predicted. Correlation ρ 0.83. 18 models from 13 other providers.
GDP.pdf · all-pass → 55.9% predicted. Correlation ρ 0.67. 18 models from 13 other providers.
Two supported public predictors required
Terminal-Bench 4.0 → 68.7% predicted. Correlation ρ 0.72. 18 models from 13 other providers.
0.0 points below prediction. Rank reversals: 0.0%. Provider-removal sensitivity: 0.0–0.0%. Prediction extends outside the reference range.
SciCode → 43.1% predicted. Correlation ρ 0.46. 12 models from 8 other providers.
GDP.pdf · all-pass → 45.3% predicted. Correlation ρ 0.90. 12 models from 8 other providers.
0.0 points below prediction. Rank reversals: 0.0%. Provider-removal sensitivity: 0.0–0.0%. Prediction extends outside the reference range.
GPQA Diamond → 44.4% predicted. Correlation ρ 0.68. 12 models from 9 other providers.
Humanity’s Last Exam → 59.7% predicted. Correlation ρ 0.67. 18 models from 13 other providers.
Two supported public predictors required
GDP.pdf · all-pass → 6.1% predicted. Correlation ρ 0.69. 12 models from 9 other providers.
Two supported public predictors required
Two supported public predictors required
Average of scored private tests. Missing tests never count as zero. Score stability shows provider-removal sensitivity, not confidence intervals. A zero does not establish clean training.
20Claude Haiku 5.5Anthropic · Max · 3/8 private tests0.0% BenchMaxxed
Source variant: Claude Haiku 5.5 (Max)
| Private test | Actual | Predicted | BenchMaxxed |
|---|---|---|---|
| FrontierCode 1.1 ↗ | 41.8% | 40.3% | 0.0% |
| AA-Briefcase v1.1 · rubric ↗ | 52.3% | 51.4% | 0.0% |
| AutomationBench-AA ↗ | 35.4% | — | Unscored |
| AA-AnalystAgent · pass^5 ↗ | 53.8% | 35.9% | 0.0% |
| AA-Omniscience · accuracy ↗ | 36.4% | — | Unscored |
| Harvey LAB-AA v1.1 ↗ | 0.6% | — | Unscored |
| ITBench-AA ↗ | — | — | Unscored |
| MLCR-AA · private holdout ↗ | 63.3% | — | Unscored |
Public benchmarks and calculation details
0.0 points below prediction. Rank reversals: 19.0%. Provider-removal sensitivity: 0.0–0.0%.
SciCode → 39.5% predicted. Correlation ρ 0.51. 18 models from 13 other providers.
Terminal-Bench 4.0 → 40.7% predicted. Correlation ρ 0.78. 18 models from 13 other providers.
0.0 points below prediction. Rank reversals: 7.7%. Provider-removal sensitivity: 0.0–0.0%.
GDPval-AA v2.1 → 51.6% predicted. Correlation ρ 0.83. 18 models from 13 other providers.
GDP.pdf · all-pass → 51.2% predicted. Correlation ρ 0.67. 18 models from 13 other providers.
Two supported public predictors required
Terminal-Bench 4.0 → 57.0% predicted. Correlation ρ 0.72. 18 models from 13 other providers.
0.0 points below prediction. Rank reversals: 0.0%. Provider-removal sensitivity: 0.0–0.0%.
SciCode → 32.9% predicted. Correlation ρ 0.46. 12 models from 8 other providers.
GDP.pdf · all-pass → 36.7% predicted. Correlation ρ 0.90. 12 models from 8 other providers.
Two supported public predictors required
Humanity’s Last Exam → 41.8% predicted. Correlation ρ 0.67. 18 models from 13 other providers.
Two supported public predictors required
GDP.pdf · all-pass → 4.1% predicted. Correlation ρ 0.69. 12 models from 9 other providers.
Private anchor unavailable
Two supported public predictors required
Average of scored private tests. Missing tests never count as zero. Score stability shows provider-removal sensitivity, not confidence intervals. A zero does not establish clean training.
21Claude Opus 5.5Anthropic · Max, Default Fallback · 3/8 private tests0.0% BenchMaxxed
Source variant: Claude Opus 5.5 (Max, Default Fallback)
| Private test | Actual | Predicted | BenchMaxxed |
|---|---|---|---|
| FrontierCode 1.1 ↗ | 52.2% | 50.8% | 0.0% |
| AA-Briefcase v1.1 · rubric ↗ | 61.2% | 61.0% | 0.0% |
| AutomationBench-AA ↗ | 69.5% | — | Unscored |
| AA-AnalystAgent · pass^5 ↗ | 56.3% | 45.9% | 0.0% |
| AA-Omniscience · accuracy ↗ | 66.2% | — | Unscored |
| Harvey LAB-AA v1.1 ↗ | 4.2% | — | Unscored |
| ITBench-AA ↗ | 38.2% | — | Unscored |
| MLCR-AA · private holdout ↗ | 66.7% | — | Unscored |
Public benchmarks and calculation details
0.0 points below prediction. Rank reversals: 0.0%. Provider-removal sensitivity: 0.0–0.2%. Prediction extends outside the reference range.
SciCode → 51.0% predicted. Correlation ρ 0.51. 18 models from 13 other providers.
Terminal-Bench 4.0 → 50.7% predicted. Correlation ρ 0.78. 18 models from 13 other providers.
0.0 points below prediction. Rank reversals: 0.0%. Provider-removal sensitivity: 0.0–1.5%. Prediction extends outside the reference range.
GDPval-AA v2.1 → 64.3% predicted. Correlation ρ 0.83. 18 models from 13 other providers.
GDP.pdf · all-pass → 55.9% predicted. Correlation ρ 0.67. 18 models from 13 other providers.
Two supported public predictors required
Terminal-Bench 4.0 → 73.3% predicted. Correlation ρ 0.72. 18 models from 13 other providers.
0.0 points below prediction. Rank reversals: 0.0%. Provider-removal sensitivity: 0.0–0.0%. Prediction extends outside the reference range.
SciCode → 48.0% predicted. Correlation ρ 0.46. 12 models from 8 other providers.
GDP.pdf · all-pass → 45.3% predicted. Correlation ρ 0.90. 12 models from 8 other providers.
Two supported public predictors required
Humanity’s Last Exam → 62.4% predicted. Correlation ρ 0.67. 18 models from 13 other providers.
Two supported public predictors required
GDP.pdf · all-pass → 6.1% predicted. Correlation ρ 0.69. 12 models from 9 other providers.
Two supported public predictors required
Two supported public predictors required
Average of scored private tests. Missing tests never count as zero. Score stability shows provider-removal sensitivity, not confidence intervals. A zero does not establish clean training.
22K2 Horizon 375B A23BInstitute of Foundation Models · Published configuration · 3/8 private tests0.0% BenchMaxxed
Source variant: K2 Horizon 375B A23B
| Private test | Actual | Predicted | BenchMaxxed |
|---|---|---|---|
| FrontierCode 1.1 ↗ | 32.1% | 28.5% | 0.0% |
| AA-Briefcase v1.1 · rubric ↗ | 41.6% | 38.0% | 0.0% |
| AutomationBench-AA ↗ | 37.2% | — | Unscored |
| AA-AnalystAgent · pass^5 ↗ | — | — | Unscored |
| AA-Omniscience · accuracy ↗ | 18.2% | 17.7% | 0.0% |
| Harvey LAB-AA v1.1 ↗ | — | — | Unscored |
| ITBench-AA ↗ | — | — | Unscored |
| MLCR-AA · private holdout ↗ | — | — | Unscored |
Public benchmarks and calculation details
0.0 points below prediction. Rank reversals: 0.0%. Provider-removal sensitivity: 0.0–0.0%. Prediction extends outside the reference range.
SciCode → 25.8% predicted. Correlation ρ 0.58. 21 models from 13 other providers.
Terminal-Bench 4.0 → 29.8% predicted. Correlation ρ 0.80. 21 models from 13 other providers.
0.0 points below prediction. Rank reversals: 0.0%. Provider-removal sensitivity: 0.0–0.0%. Prediction extends outside the reference range.
GDPval-AA v2.1 → 38.3% predicted. Correlation ρ 0.88. 21 models from 13 other providers.
GDP.pdf · all-pass → 37.6% predicted. Correlation ρ 0.72. 21 models from 13 other providers.
Two supported public predictors required
Terminal-Bench 4.0 → 43.2% predicted. Correlation ρ 0.71. 21 models from 13 other providers.
Private anchor unavailable
0.0 points below prediction. Rank reversals: 13.2%. Provider-removal sensitivity: 0.0–26.2%.
GPQA Diamond → 7.3% predicted. Correlation ρ 0.77. 12 models from 9 other providers.
Humanity’s Last Exam → 28.5% predicted. Correlation ρ 0.75. 21 models from 13 other providers.
Private anchor unavailable
Private anchor unavailable
Private anchor unavailable
Average of scored private tests. Missing tests never count as zero. Score stability shows provider-removal sensitivity, not confidence intervals. A zero does not establish clean training.
No matching model. Try a shorter name.
Different models have different test coverage. Choose one private-test tab for a like-for-like comparison.
Same model. Two kinds of test.
Average the gaps across 4 scored private tests → 42.1% BenchMaxxed.
The calculation & its limits +
- Predict private scores from public benchmarks.Each private test has two capability-matched public predictors. Provider-excluded Theil–Sen fits use the same model variants and reasoning settings.
- Score only supported relationships.Each fit needs eight models, three other providers, a positive slope, and Spearman ρ ≥ 0.35. Both predictors must qualify.
- Measure the relative shortfall.Clamp predictions to 0–100. Combine them with ρ² weights. GDPval uses raw Elo as input, not as a percentage.
- Average the scored private tests.At least two must qualify. Include measured zero gaps. Never count missing tests as zero.
100 × max(0, predicted − actual) / predictedRank reversals count public wins that become private losses. Score stability removes each other provider in turn. These ranges are not confidence intervals.
This measures a score gap, not proof of benchmark-specific training. Task mix, harnesses, reasoning budgets, and grading can also cause gaps. No model has a verified “clean” label.
SciCode → FrontierCode
ρ = 0.63Coding example. Lines use the full cohort. Individual predictions exclude the model’s provider.
Terminal-Bench 4.0 → FrontierCode
ρ = 0.82Coding example. Lines use the full cohort. Individual predictions exclude the model’s provider.
Transparent by design: MIT scoring code ↗
A better test score.
Not a better model everywhere.
Qwen2.5-Coder → Openhands LM.
One specialized checkpoint. Different outcomes.
Optimized for SWE-bench: 7.0 → 37.2.
Django repair: 47.4 → 13.9.
The gain on one test did not transfer.
Study & context +
Shibaev et al., Table 1. Qwen2.5-Coder-32B and Openhands LM 32B. Separate from our 22-model snapshot. The study also reports format failures and task differences.
No mystery numbers.
Artificial Analysis · 22 model variants · 339 published scores · updated manually.
Private tests & their public benchmarks +
FrontierCode 1.1 ↗
Coding · 22/22 scoredAA-Briefcase v1.1 · rubric ↗
Knowledge work · 22/22 scoredAutomationBench-AA ↗
Workflows · 0/22 scoredAA-AnalystAgent · pass^5 ↗
Data analysis · 16/22 scoredAA-Omniscience · accuracy ↗
Knowledge · 13/22 scoredHarvey LAB-AA v1.1 ↗
Legal work · 1/22 scoredITBench-AA ↗
Operations · 0/22 scoredMLCR-AA · private holdout ↗
Medical reasoning · 0/22 scoredAll source scores +
| Exact model variant | FrontierCode 1.1 ↗ | AA-Briefcase v1.1 · rubric ↗ | AutomationBench-AA ↗ | AA-AnalystAgent · pass^5 ↗ | AA-Omniscience · accuracy ↗ | Harvey LAB-AA v1.1 ↗ | ITBench-AA ↗ | MLCR-AA · private holdout ↗ |
|---|---|---|---|---|---|---|---|---|
| Claude Opus 5.5 (Max, Default Fallback) | 52.2 | 61.2 | 69.5 | 56.3 | 66.2 | 4.2 | 38.2 | 66.7 |
| Claude Sonnet 5.5 (Max, Default Fallback) | 45.8 | 65.2 | 71.8 | 57.5 | 54.0 | 2.8 | — | 75.0 |
| Claude Fable 5.1 (Max, Default Fallback) | 49.0 | 61.5 | 59.4 | 57.5 | 67.2 | 6.4 | 49.5 | 71.1 |
| GPT-6 Astra (Max) | 47.0 | 52.0 | 68.5 | 51.2 | 62.6 | 8.6 | 48.6 | 35.0 |
| GPT-6.1 Sol (Max) | 49.1 | 55.7 | 64.9 | 50.0 | 62.1 | 6.9 | — | 33.9 |
| Muse Spark 1.3 (Max) | 40.6 | 58.7 | 57.9 | 37.5 | 43.6 | 8.9 | 33.2 | 43.3 |
| Grok 4.7 (Xhigh) | 33.8 | 55.4 | 65.6 | — | 47.5 | 9.4 | 42.1 | 15.0 |
| MiMo-V2.6-Pro | 45.4 | 54.7 | 58.6 | 37.5 | 34.8 | — | — | 18.3 |
| Qwen3.8 Max (0902) | 43.2 | 58.0 | 56.2 | 45.0 | 31.7 | — | 40.3 | 20.0 |
| GLM-5.3 (Max) | 40.1 | 51.3 | 62.2 | — | 33.9 | 0.3 | 46.1 | 48.3 |
| Step 5 Preview | 38.4 | 47.8 | 51.0 | 35.0 | 41.5 | — | 55.6 | 16.7 |
| Kimi K3 (Max) | 43.6 | 51.0 | 58.3 | 38.8 | 47.6 | 5.3 | 47.7 | 38.3 |
| Claude Haiku 5.5 (Max) | 41.8 | 52.3 | 35.4 | 53.8 | 36.4 | 0.6 | — | 63.3 |
| GLM-5.3-Flash | 41.0 | 46.4 | 60.4 | — | 27.5 | 0.8 | 51.2 | 51.1 |
| Gemini 3.8 Flash (High) | 36.4 | 42.1 | 59.9 | — | 54.6 | 0.0 | 52.5 | 21.7 |
| DeepSeek V4.1 Flash (Max) | 44.0 | 45.2 | 68.9 | — | 46.4 | 1.7 | 46.9 | 22.8 |
| GPT-6 Luna (Max) | 41.9 | 41.2 | 53.2 | 43.8 | 43.8 | 3.3 | — | 16.1 |
| Qwen3.8 27B (Xhigh) | 34.2 | 47.8 | 48.2 | 18.8 | 15.6 | — | — | 21.7 |
| K2 Horizon 375B A23B | 32.1 | 41.6 | 37.2 | — | 18.2 | — | — | — |
| MiniMax-M3 | 23.0 | 30.3 | 21.3 | 10.0 | 16.7 | 0.0 | — | 17.2 |
| Inkling (Xhigh) | 15.8 | 19.3 | 5.0 | 23.8 | 41.5 | 0.0 | — | 12.2 |
| Muse Glimmer (High) | 13.9 | 11.5 | 6.8 | 20.0 | 27.0 | — | — | 20.0 |
| Exact model variant | SciCode ↗ | Terminal-Bench 4.0 ↗ | Terminal-Bench 2.1 ↗ | Terminal-Bench-Science 0.1 ↗ | GDPval-AA v2.1 ↗RAW ELO | GDP.pdf · all-pass ↗ | EnterpriseOps-Gym-AA ↗ | GPQA Diamond ↗ | Humanity’s Last Exam ↗ | AA-LCR v1.1 ↗ |
|---|---|---|---|---|---|---|---|---|---|---|
| Claude Opus 5.5 (Max, Default Fallback) | 66.9 | 59.6 | — | 59.0 | 1866 | 26.2 | — | — | 61.4 | 84.7 |
| Claude Sonnet 5.5 (Max, Default Fallback) | 61.0 | 63.6 | — | 53.3 | 1838 | 25.8 | — | — | 55.0 | 82.7 |
| Claude Fable 5.1 (Max, Default Fallback) | 63.1 | 52.0 | 91.4 | 43.3 | 1756 | 26.2 | — | 93.7 | 59.1 | 85.3 |
| GPT-6 Astra (Max) | 56.5 | 59.1 | 88.4 | 63.3 | 1572 | 31.0 | — | 96.1 | 54.7 | 80.7 |
| GPT-6.1 Sol (Max) | 54.2 | 56.1 | — | 58.1 | 1593 | 31.0 | — | — | 52.9 | 83.0 |
| Muse Spark 1.3 (Max) | 58.8 | 33.3 | 84.3 | 11.0 | 1681 | 26.6 | — | 93.5 | 48.7 | 83.0 |
| Grok 4.7 (Xhigh) | 57.4 | 25.8 | — | — | 1712 | 20.0 | — | — | 43.1 | 76.7 |
| MiMo-V2.6-Pro | 60.9 | 34.8 | — | 5.7 | 1682 | 19.2 | — | — | 49.4 | 86.3 |
| Qwen3.8 Max (0902) | 52.1 | 38.9 | 88.8 | 11.9 | 1667 | 22.8 | 47.6 | 92.8 | 43.1 | 80.3 |
| GLM-5.3 (Max) | 59.0 | 41.9 | 83.9 | 9.5 | 1651 | 11.2 | 36.4 | 91.7 | 42.3 | 79.7 |
| Step 5 Preview | 58.9 | 33.3 | — | 2.4 | 1584 | 14.8 | 47.2 | — | 46.5 | 88.3 |
| Kimi K3 (Max) | 59.5 | 12.6 | 85.0 | — | 1536 | 22.0 | 45.3 | 93.5 | 46.9 | 88.7 |
| Claude Haiku 5.5 (Max) | 55.0 | 32.8 | — | 20.0 | 1620 | 20.8 | — | — | 44.4 | 82.7 |
| GLM-5.3-Flash | 51.6 | 32.8 | 84.3 | 4.8 | 1644 | 15.4 | 33.2 | 91.2 | 39.9 | 80.0 |
| Gemini 3.8 Flash (High) | 56.6 | 19.7 | 87.6 | 10.0 | 1436 | 21.0 | — | 95.3 | 47.8 | 81.3 |
| DeepSeek V4.1 Flash (Max) | 51.9 | 26.8 | — | 9.0 | 1600 | 12.8 | — | — | 39.2 | 84.0 |
| GPT-6 Luna (Max) | 54.6 | 12.6 | — | 8.6 | 1430 | 22.8 | — | — | 38.5 | 83.3 |
| Qwen3.8 27B (Xhigh) | 46.6 | 5.6 | 79.8 | 0.5 | 1419 | 16.6 | 44.2 | 90.5 | 33.9 | 82.0 |
| K2 Horizon 375B A23B | 42.9 | 1.5 | 71.9 | — | 1361 | 7.4 | — | 87.3 | 32.0 | 80.0 |
| MiniMax-M3 | 47.1 | 2.0 | 65.2 | 0.5 | 1242 | 9.8 | 32.1 | 92.9 | 39.0 | 83.0 |
| Inkling (Xhigh) | 47.0 | 1.0 | 55.1 | 0.0 | 1076 | 12.8 | 38.0 | 87.2 | 31.9 | 77.3 |
| Muse Glimmer (High) | 44.9 | 0.5 | 51.7 | 0.0 | 786 | 10.0 | 34.7 | 83.5 | 22.0 | 83.3 |
339/396 cells. — = unpublished. Briefcase: rubric pass rate. Omniscience: accuracy. MLCR: private holdout. Terminal-Bench-Science: context only.
FrontierCode’s 22-model cohort, not every Artificial Analysis model. SciCode is under review and affects coding and analysis predictions.
Sources, updates & code +
- Artificial Analysis — FrontierCode 1.1 · private dataset
- Artificial Analysis — AA-Briefcase v1.1 · rubric · private dataset
- Artificial Analysis — AutomationBench-AA · private dataset
- Artificial Analysis — AA-AnalystAgent · pass^5 · private dataset
- Artificial Analysis — AA-Omniscience · accuracy · private dataset · Accuracy over all questions, not the -100–100 hallucination-adjusted Omniscience Index.
- Artificial Analysis — Harvey LAB-AA v1.1 · private dataset · Hallucination-gated all-pass rate. Do not mix v1.0 scores.
- Artificial Analysis — ITBench-AA · private dataset
- Artificial Analysis — MLCR-AA · private holdout · private dataset · AA evaluates a private expert/compound held-out set, not the public MLCR dataset.
- Artificial Analysis — SciCode · public dataset · Under review by Artificial Analysis after a dataset audit.
- Artificial Analysis — Terminal-Bench 4.0 · public dataset
- Artificial Analysis — Terminal-Bench 2.1 · public dataset
- Artificial Analysis — Terminal-Bench-Science 0.1 · public dataset · Context only. Not used in the default pairings.
- Artificial Analysis — GDPval-AA v2.1 · public dataset · Raw Elo. Used as a predictor, never displayed as a percentage.
- Artificial Analysis — GDP.pdf · all-pass · public dataset
- Artificial Analysis — EnterpriseOps-Gym-AA · public dataset
- Artificial Analysis — GPQA Diamond · public dataset
- Artificial Analysis — Humanity’s Last Exam · public dataset
- Artificial Analysis — AA-LCR v1.1 · public dataset
- Artificial Analysis methodology
- MIT algorithm v2.0.0, tests, and output
Retrieved 13 October 2026. A maintainer reviews new scores, recalculates, and publishes a static release. No live scraper.
Only the algorithm is MIT licensed. The article, design, artwork, and website are not. Source-data rights remain with Artificial Analysis. No affiliation.
Did it learn the skill?
Or just the test?
Benchmaxxing means optimizing for the benchmark more than the ability it should measure. Private tests check what carries over.
Make your own
AI activity visible.
AI activity on your computer, shared on your terms.