Datamimic – don't let your coding agent invent its own test world | AI Paper Digest

TL;DR Highlight

Pair a seeded synthetic data engine with your coding agent to get reproducible test data.

Who Should Read

Backend or QA developers whose coding agents generate test data that changes every run and makes regression tests flaky

Core Mechanics

A fixed seed gives identical data
- Setting <setup rngSeed="N"> in the XML replays the same output on the tested runtime profiles.
- Without a seed, replay is not guaranteed.
A hash proves two runs match
- The Python facade generate_domain returns determinism_proof.content_hash.
- The same hash means the same data.
There is an agent-oriented CLI
- The datamimic authoring commands include schema, capabilities, reference, compile, build, lint, and dry-run.
- All authoring commands output JSON by default and accept files or stdin (-).
XML is the final artifact
- The persisted executable project is XML, while DM JSON is a transient agent representation.
- After Build is ready, save artifact.compiled_xml and run it with datamimic run model.xml.
The MCP adapter is optional

Community Reactions

Verification scope is explicitly bounded
- The CI compares seeded output to goldens across seven x64 cells (ubuntu Python 3.11/3.12/3.13, timezone variants, windows).
- Other Python versions and dependency resolutions are outside the guarantee.
Coverage numbers are published
- Facade domains 4/4, entities 23/23, literal generators 34/35.
- The database-backed SequenceTableGenerator is excluded from the hash check.
A commenter suggested that agents should write and run generator code rather than emit data directly.
A commenter asked whether foreign keys and schema constraints across relational databases work out of the box.

How to Apply

Add a fixed-seed request to your agent prompt
- Paste the prompt below the next time you ask an agent for test data, and state the seed and field requirements.
- Review the returned XML together with the verification evidence.
Compare content_hash in CI
- Call generate_domain with a fixed seed and clock, and compare the hash against a golden value in a test.
- Fail the build when the hash changes.
Lock the agent workflow to build then persist
- Tell the agent in its instructions to stop at outcome=ready and save artifact.compiled_xml.
- Allow real execution only when a human asks for it.

Try It

copy & run

# Paste into Claude Code or Codex after installing the DATAMIMIC plugin (pip install datamimic-ce)

Create the dataset I describe with the DATAMIMIC plugin's generate-test-data skill and the project CLI. Submit a BuildRequestV4 with transient DM JSON, expectations for my stated requirements, and identifier/FK roles where needed. Repair the reported paths before retrying. Stop on outcome=ready, persist artifact.compiled_xml, and return its path with the separate bounded verification evidence. Execute the XML only if I requested real execution. Return a human-readable model review with the XML and verification evidence.

Requirements: {{describe your data}}

Next to try: run datamimic run model.xml twice with the same seed and compare the outputs

Terminology

Deterministic generationThe same input and seed always produce the same output, like a recipe that always yields the same dish.

PseudonymizationReplacing real personal fields with other values by rule, like swapping name tags for nicknames.

MCPA standard protocol for agents to call external tools, like one USB port replacing many custom connectors.

Content hashA fingerprint (SHA-256) of the output. Same fingerprint means same content.

Datamimic – don't let your coding agent invent its own test world