Playtest Agent
An agent that plays your game from screenshots, looks for bugs, and checks every report before it counts.
How it works
- Play. A model sees a screenshot and picks keys or clicks.
- Report. While it plays, it writes down what it expected and what it saw.
- Replay. Each report is replayed twice on a fresh copy of the game. Reports that do not happen again are set aside.
- Judge. A second model looks at the frames and must quote what it saw. A yes with no quote counts as no.
- Report page. Each run ends with a page of screenshots, bugs, evidence and cost.
It plays different kinds of games
Freedoom, played by Muse Glimmer on one PC. No internet, no cost.
A phone game, played by Muse Wonder through its first screens.
Honest numbers
| Test | Result |
|---|---|
| A game it had never played, with 5 hidden bugs | 3 of 5 found |
| False reports on that game that got through | 1 |
| That new game with no bugs in it | 1 false report |
| Its first test game with no bugs in it | 0 reports, and it won |
| Muse Glimmer as judge on one PC, against Muse Spark | 19 of 22 agree |
| Cost of the never played run | $0.77 |
The never played game was made with its answer key kept away from the agent, and every setting was frozen before the run.