DSLs Enable Reliable Use of LLMs

TL;DR Highlight

Small DSLs with validators let LLMs generate exactly what you intend

Who Should Read

A backend developer burdened with reviewing LLM-generated code that drifts from the intended design

Core Mechanics

The first spec is only a hypothesis
- Large systems involve many small design decisions that can't be known upfront.
- Constraints and edge cases appear during implementation, so specs must be revised iteratively.
Reviewing doesn't force design thinking
- Reading generated code only checks it against your intent.
- Writing code forces decisions about where responsibility lives and which boundaries to expose.
A few examples are enough for a DSL
- A general-purpose language like Java has many valid ways to express one intent.
- A DSL strips the variation, so a few in-context examples produce correct syntax.
The validator is the agent's harness
- DSLs usually ship with a parser, JSON schema, type checker, or compiler.
- An agent can generate, validate, and repair from errors without a human.
- Errors read in domain terms, like "you cannot select an action before choosing a client".

Community Reactions

The "DSL stays small" premise carries the argument, one commenter says
- PlantUML has huge training data, but a new DSL does not, so more combinations turn the system prompt into a language description.
- They argue a real compiler or executor is needed, not just a validator.
Short specs worked well, commenters report
- One got chatbots to work with a 200-line DSL spec.
- Another found that with a prompt and a checker tool, even small models like Gemma 12B QAT wrote good scripts in their DSL.
A SwiftUI counterexample
- One commenter says the DSL's formality is lost since it isn't what they communicate in with the LLM.
- They found regular compilers catch more mistakes, and exposing the full stack worked better than abstractions.
Correctness alone isn't enough, one commenter found
- Agent-generated GPU kernels compiled and ran but were slower than baseline.
- They say speed targets must be part of the check.

How to Apply

Paste your DSL/config spec into a prompt and test it
- Give an LLM the grammar plus 2-3 examples of the YAML or config format you use today and have it write a new entry.
- Run the output through your existing parser or schema to measure the pass rate.
Feed validator errors back to the agent
- In your agent loop, pass JSON schema or type-checker output straight into the retry prompt.
- Rephrase errors in domain terms where possible.
Name recurring domain concepts as types and functions
- Fix decisions like threading or retries in library types instead of re-explaining them in every prompt.
- Then prompt using only those names so the model writes only the logic.
Put non-correctness criteria in validation too
- Add targets like performance to the validation step.
- Filter out results that pass but are useless.

Try It

copy & run

# Paste into a Claude or ChatGPT chat as-is (simplified version of the article's method)

You are the natural-language interface to the DSL below. Output only results that match the DSL spec and examples, with no explanation.

[DSL spec]
{{DSL grammar description}}

[Examples]
{{request 1}} -> {{DSL output 1}}
{{request 2}} -> {{DSL output 2}}

Request: {{new request}}

Next step: run the output through your parser or schema validator; on an error, paste the message back and ask "fix this error and output again".

Terminology

DSLA language deliberately narrowed to express one domain's concepts. Like SQL or Mermaid, it is a single-purpose tool.

Semantic ModelA model that captures a domain's concepts and design decisions in code. It is the blueprint that defines how DSL syntax is executed.

Ubiquitous LanguageThe shared vocabulary a team uses in both code and conversation in DDD. Think of it as the team's dictionary.

HarnessA mechanism that keeps LLM output inside defined bounds. It works like reins that stop it from running wild.

In-context exampleAn example placed inside the prompt. It shows syntax without retraining the model.