How to engineer an AI harness for game development

TL;DR Highlight

Making games deterministic and screenshot-friendly lets AI agents iterate faster

Who Should Read

A developer building a game or asset-heavy project with Claude Code who keeps watching the agent break things

Core Mechanics

LLMs struggle with games for structural reasons
- Games contain many non-code assets that are opaque to an LLM.
- The author frames this as a context problem, not an inherent trait of games.
Polluted context ruins agent work
- Every earlier message is resubmitted each turn within a session.
- Bad early content keeps influencing later work, so keep context clean.
Use one seeded randomness source
- Create a single seeded random source at game start and route all non-deterministic calls through it.
- Agents can then return to the same state from scratch repeatedly.
Quantify screenshot diffs with odiff
- The author says giving a model two before/after screenshots makes it hard to see the change or progress.
- The free CLI tool odiff outputs a diff image and a value like "85% similar".
- That number can serve as a stable evaluation baseline each turn.

Community Reactions

Determinism is hard across platforms
- One commenter said fixed randomness and determinism sound great on paper but are confining in practice.
- They said keeping determinism across many platforms is where it gets nasty.
Thing approach limits larger games
- One commenter said iterating thousands of Things across 30 subsystems cannot fit in frame time.
- They said that is why ECS and SOA exist and need a different kind of harness.
Screenshot and input automation worked well
- One commenter built an automated testing system that takes screenshots and simulates input, and Claude used it well to make games.
- They wrote a guide in their engine docs.
Letting the agent instrument the game
- One commenter had Claude Code run a 1997 game on modern hardware, and the first part took 30 minutes.
- It later mapped memory and game logic and built headless parallel runs.

How to Apply

Introduce one seeded random source
- Search your project for Random calls and route them through a single seeded random object today.
- Run twice with the same seed and confirm the same state results.
Build an agent eval loop with odiff
- Compare before/after screenshots with odiff and give the agent the diff image and similarity score.
- State in the instructions to iterate toward higher similarity.
Add UI and gameplay layer toggles
- When fixing UI, have the agent hide the gameplay layer before taking screenshots.
- This keeps irrelevant visuals from distracting the agent.
Put save/load in the early design
- In week one of a new project, implement saving and loading of exact state.
- Use it so the agent can experiment repeatedly from the same state.

Try It

copy & run

# Paste into Claude Code (the article's method turned into a prompt)

You are the agent for my game project. Work by these rules.
1. Find in the code whether all non-deterministic calls use one seeded random source, and report.
2. Check whether a save/load path exists that reproduces the same state from the same seed.
3. Compare before/after screenshots with odiff and judge progress by the similarity value.
4. Make the minimum change each turn and verify immediately.

Goal: {{feature or bug to fix}}
Screenshot command: {{how to take a screenshot}}

Next step: Ask it to convert the list of non-deterministic calls it found into the seeded source.

Terminology

HarnessA frame that bundles the environment and action patterns an agent works in. Like reins and a saddle, it helps the agent keep direction.

Context engineeringChoosing what goes into the model and reducing pollution. Like writing only the essentials on a cheat sheet.

DeterminismThe property that the same input reproduces exactly the same state. Like loading a save file and always getting the same scene.

odiffA free CLI tool that finds differences between two images. Think of it as code diff for images.

Thing-Driven DevelopmentMaking every game object a single Thing type separated by an enum. Like building everything from one kind of Lego brick.