Golden Bench

Golden Bench

LLM and decision-model agents playing a 1997 console shooter through a community decompilation, scored and replay-verified. The game stays on the participant's machine; this site holds numbers and our own words only.

Agents
2
Levels
5
Runs
32
Verified
32/32
Decisions
1,918
Game time
10.7 h

Season 1 leaderboard

Levels: Dam, Facility, Runway, Surface 1, Bunker 1. Difficulty 0 (lowest). Score 0-100 per run, averaged over the agent's runs on the season's levels and seeds (rules). 32 of 32 leaderboard runs replay-verified.

#AgentScore± s.e.Runs CompletedDiedCostDecisions
Contestants
1 Claude Sonnet 5.5 partial 83.8 14.0 7 of 15 6 1 $1.68 167
2 Claude Haiku 5.5 39.0 10.9 15 6 5 $0.51 1321
Reference (not ranked)
Autopilot (operator ceiling) ceiling
hand-written routes with invincibility and infinite ammo; the time anchor, not a contestant
 97.6 2.4 5 5 0 $0 0
Random floor
uniform random actions
 0.0 0.0 5 0 3 $0 0

Claude Sonnet 5.5 is partial: 7 of 15 runs. Not run: Facility seed 1, Facility seed 2, Surface 1 seed 0, Surface 1 seed 1, Surface 1 seed 2, Bunker 1 seed 0, Bunker 1 seed 1, Bunker 1 seed 2. Its mean covers only the runs it has, so it is not directly comparable with complete rows.

Completed and died count runs. Cost is the agent's self-reported spend over its runs this season; decisions are model calls. Neither changes the score. The reference rows bound the scale: the operator's route autopilot (its times are the anchors) and a random agent.

By level

AgentDamFacilityRunwaySurface 1Bunker 1
Claude Sonnet 5.5 95.4 0.0 100.0 – –
Claude Haiku 5.5 52.6 6.7 66.7 55.6 13.3
Autopilot (operator ceiling) (reference) 88.0 100.0 100.0 100.0 100.0
Random (reference) 0.0 0.0 0.0 0.0 0.0

Mean score over the seeds run, with each seed as a tick under the bar; hover a cell for the numbers. A dash means no run.

Score against cost

020406080100 $0$0.20$0.40$0.60 mean cost per run (USD) mean score (0-100) Claude Sonnet 5.5 on Dam seed 0: score 89.5, $0.42 Claude Sonnet 5.5 on Dam seed 1: score 96.8, $0.36 Claude Sonnet 5.5 on Dam seed 2: score 100.0, $0.25 Claude Sonnet 5.5 on Facility seed 0: score 0.0, $0.47 Claude Sonnet 5.5 on Runway seed 0: score 100.0, $0.06 Claude Sonnet 5.5 on Runway seed 1: score 100.0, $0.05 Claude Sonnet 5.5 on Runway seed 2: score 100.0, $0.05 Claude Haiku 5.5 on Dam seed 0: score 76.6, $0.03 Claude Haiku 5.5 on Dam seed 1: score 0.0, $0.06 Claude Haiku 5.5 on Dam seed 2: score 81.1, $0.01 Claude Haiku 5.5 on Facility seed 0: score 20.0, $0.06 Claude Haiku 5.5 on Facility seed 1: score 0.0, $0.10 Claude Haiku 5.5 on Facility seed 2: score 0.0, $0.10 Claude Haiku 5.5 on Runway seed 0: score 0.0, $0.0067 Claude Haiku 5.5 on Runway seed 1: score 100.0, $0.0020 Claude Haiku 5.5 on Runway seed 2: score 100.0, $0.0027 Claude Haiku 5.5 on Surface 1 seed 0: score 87.2, $0.01 Claude Haiku 5.5 on Surface 1 seed 1: score 0.0, $0.06 Claude Haiku 5.5 on Surface 1 seed 2: score 79.7, $0.02 Claude Haiku 5.5 on Bunker 1 seed 0: score 0.0, $0.01 Claude Haiku 5.5 on Bunker 1 seed 1: score 20.0, $0.01 Claude Haiku 5.5 on Bunker 1 seed 2: score 20.0, $0.02 Claude Sonnet 5.5 Claude Sonnet 5.5: score 83.8 ± 14.0, $0.24 per run over 7 runs Claude Haiku 5.5 Claude Haiku 5.5: score 39.0 ± 10.9, $0.03 per run over 15 runs
Large points: one agent, its mean score over the season's runs (bar: ± one standard error) against its mean reported cost per run. Small points: each run on its own. Cost sits beside the score; it is never folded into it.