Gym Battle
A benchmark gym for Luke’s team of AI agents (the swarm). It runs standard task benchmarks against the team, grades every answer with a mechanical referee, and keeps a skill reading that remembers failures and fades when a skill goes untested.
IN BUILD
What it is
Gym Battle wraps the swarm as an environment with the familiar make, reset and step interface, so any benchmark can plug in and every run is scored, recorded and comparable over time. The referee is never the agent's own judgement: the gym runs the answer through the benchmark's own checker and takes its exit code.
The scoreboard is not an average. Each benchmark task is a small belief in Umwelt (Luke’s belief engine) whose skill and confidence move with each result and drift back towards unknown when the task is not re-run. Three passes after one failure reads as high skill with less than full confidence, not a flat 100 percent.
For coding it compares builds (a model plus a strategy) across regimes of HumanEval, MBPP, LiveCodeBench and SWE-bench, and produces a map of which build wins where, with no combined score.
What exists today
Private repository, last pushed 31 July 2026: 47 modules and 37 test files. A paid benchmark-running service is built but not launched: metered runs, signed receipts with a standalone verifier, and a localhost-only page. One live receipt was produced and verified. Work paused on 23 July 2026 with its launch checklist open.
Not done
The bundled GAIA sample is synthetic, not GAIA. Real scores need the official sets.
The SWE-bench referee is parked until it has a repository and a container to run in.
Nothing is exposed to the internet. The launch checklist’s demo, payment and exposure steps are still open.
Next
Run the official benchmark sets through the gym and publish the regime map with its confidence.
Built with
Python Umwelt