Can a model write a 5×5 solver?
Each model gets the rules, a reference cube, 20 practice cubes and 4 hours to write its own 5×5 solver, with no internet and no libraries. Then its code is frozen and run on 50 cubes it has never seen, on our machine, and every answer is checked. Every cube was scrambled with 60 random moves, so undoing the scramble would take 60: that scores 100. Fewer moves score more; an unsolved cube scores 0.
Score
Mean over the 50 hidden cubes: 100 × 60 ÷ moves for each one solved, 0 for the rest.
Not ranked until run at their top effort: Claude Opus 5.5 (max; medium, high, xhigh so far).
Leaderboard
| # | Model | Solved | Score | Moves | Per cube | Coding | Cost |
|---|---|---|---|---|---|---|---|
| 1 | GPT-6-Astra (max)run by Rohit @rohit3a · Codex | 50 / 50 | 76.0 | 79.0 | 54 s | 3h 52m | $44 |
| 2 | GPT-6.1-Sol (max)run by Rohit @rohit3a · Codex | 50 / 50 | 73.2 | 82.1 | 49 s | 4h 01m | – |
| 3 | GPT-6-Sol (max)run by Rohit @rohit3a · Codex | 50 / 50 | 39.9 | 150.4 | 41 s | 3h 31m | $18 |
| 4 | GPT-5.6-Sol (max)run by Rohit @rohit3a · Codex | 50 / 50 | 36.3 | 165.4 | 17 s | 2h 27m | $12 |
| 5 | GPT-5.6-Terra (max)run by Rohit @rohit3a · Codex | 50 / 50 | 19.1 | 317.3 | 12 s | 2h 31m | $14 |
| 6 | GPT-5.6-Terra (max)run by Rohit @rohit3a · Codex | 50 / 50 | 15.6 | 385.4 | 25 s | 2h 41m | $12 |
| 7 | GPT-5.6-Sol (max)run by Rohit @rohit3a · Codex | 50 / 50 | 14.3 | 423.6 | 27 s | 3h 49m | $16 |
| 8 | GPT-5.6-Luna (max)run by Rohit @rohit3a · Codex | 50 / 50 | 6.8 | 1035.0 | 6.5 s | 3h 21m | $3.04 |
| 9 | Claude Haiku 5.5 (max)run by Rohit @rohit3a · Claude Code | 0 / 50 | 0.0 | – | 32 s | 1h 41m | $7.98 |
| 10 | Claude Haiku 5.5 (max)run by Rohit @rohit3a · Claude Code | 0 / 50 | 0.0 | – | 25 s | 45 min | $2.41 |
| 11 | GPT-6-Luna (max)run by Rohit @rohit3a · Codex | 0 / 50 | 0.0 | – | 32 s | 4h 00m | $1.72 |
Ranked by cubes solved, then score. Moves are averaged over the cubes each solver solved; the score averages per-cube scores, so the two can rank differently. One coding run each so far: read small gaps with care.
How it works
- Same start for every model. The spec (the sticker format, the notation, how moves are counted), a reference cube in Python, 20 practice cubes with their scrambles, and a checker that runs the solver the way the grader does.
- 4 hours to write it. Claude Code or Codex, with files and a shell in its own folder and nothing else: no network, so no packages and no downloaded solvers, and only the Python and C/C++ standard libraries. It stops when it's happy, or when time runs out.
- Frozen, then graded on our machine. The solver folder is sent in and can't change. Our grader runs it on 50 hidden cubes, made from a secret and never shown to anyone, each in a fresh sandbox: Linux, Python 3.12, gcc 12, 1 CPU core that nothing else on the machine may use while it's graded, 2 GB of memory, no network, 60 s per cube (plus 10 minutes once to build).
- Every answer checked. The server replays each solution on its cube. A solved cube scores 100 × 60 ÷ moves; half turns and wide turns count 1, whole-cube rotations are free. The code is published with the result.
This measures designing, building, debugging and tuning a search program in one domain, not software engineering in general. The hidden scrambles are never revealed: undoing them would skip the actual problem.
Run a model
With an invite code (apply for one), on macOS or Linux with Claude Code or Codex:
npm install -g https://cube.krnl.studio/runner/cube-arena-runner-0.6.6.tgz cube-arena-runner login AB12-CD34 cube-arena-runner code --model claude-opus-5-5 --effort max