Leaderboard
Every configuration we measured on the paper's four task sets, ranked by score, time, or cost.
Top score
94.2%
GPT-6 Astrahigh
1107.9 s · $11.11 per task
Fastest WR
806.8s
GPT-6 Astralow
Scores 90.3%
Cheapest
$8.425
GPT-6 Astramedium
Scores 86.7% in 810.5 s
Filter models and reasoning effort
Results
All included results, ordered by average performance.
| # | Model | Effort | Performance | Time / task | Cost / task | |
|---|---|---|---|---|---|---|
| 1 | GPT-6 Astra Cost / task: $11.1091 | high | 94.23% | 1107.9s | $11.1091 | |
| 2 | GPT-6 Astra Cost / task: $15.0610 | xhigh | 92.98% | 1515.4s | $15.0610 | |
| 3 | GPT-6 Astra Cost / task: $9.2869 | low | 90.29% | 806.8s | $9.2869 | |
| 4 | GPT-6 Astra Cost / task: $8.4246 | medium | 86.73% | 810.5s | $8.4246 |
Task performance
Select a measure to compare. Tap a point for its model, score, and timing.
Scatter plot comparing model performance with the selected horizontal metric.
Performance, time, and cost
Drag to rotate; select a point to inspect it. Only results with known costs appear here.
Rotatable three-dimensional scatter plot comparing cost, time, and performance.
How to read these numbers
Score is the mean verifier score over the task set, with partial credit. Time is the mean time per task, from when the agent receives the instruction until it finishes; setup and grading are not counted. Cost is the model API cost per task. A dash means usage was not recorded.
Every configuration ran on the same tasks within its benchmark.