Concordat: Typed Sessions for LLM Agents

Oct 27, 2026•Milad Sikaroudi•View PDF

TL;DR Highlight

Offering only state-legal tools cut unconfirmed agent changes to zero with comparable task success.

Who Should Read

A backend developer whose agent can issue refunds or cancel bookings and who relies on a prompt sentence to enforce 'confirm before changing'.

Core Mechanics

A prompt rule is only a request
- The model gets every tool every turn and is trusted to use them in order.
- Under customer pressure like 'just do it', it may treat that as permission and break the rule.
Hide tools that are illegal right now
- Each turn only tools legal in the current state are passed to the model, with parallel tool calls disabled.
- Illegal calls the model makes anyway are returned as errors with no side effects.
Human decisions can only be requested
- A @human step is requested by the model but decided by an approver function the app supplies.
- Token and tool-call budgets are split on handoff by removing the allowance from the parent.
Inheritance can't bypass the protocol
- A subclass is rejected at definition time if it can reach an outcome state by a route the parent forbids.
- The error message shows the offending path.
Benchmark scores missed the violations
- If the final DB matched the expected state, the task got full reward even over the customer's objection.
- Outcome-only evaluation cannot tell consented changes from non-consented ones.
The cost came from behavior after blocking
- With Haiku, Concordat passed 7 fewer task-runs, and 11 of 13 discordant cases were transfers to a human after repeated refusals.
- Three added sentences saying pushback is not confirmation and not a reason to transfer addressed it.

Evidence

Unconfirmed changes
- Under pressure on τ-bench's 50 airline tasks, the standard agent made changes without 'yes' in 13 of 50 tasks with gpt-4.1 and 29 of 128 task-runs with Claude Haiku 5.5.
- The Concordat agent made zero in both (Fisher p=6×10^-10 Haiku, p=1×10^-4 gpt-4.1).
Benchmark missed violations
- In the first Haiku pressure run, 8 of the 13 tasks with an unconfirmed change by the standard agent still got full reward.
No significant task-success difference
- Haiku: standard 112/128 vs Concordat 105/128 (McNemar p=0.17).
- gpt-4.1 after the prompt change: standard 20/50 vs Concordat 22/50 (p=0.80), with transfers down from 11 to 7.
Cost comparison
- By τ-bench list prices, Haiku cooperative run cost $1.49 vs $1.70 standard, and the pressure run $1.64 vs $1.76.
- With gpt-4.1 it was $3.47 vs $3.13 because conversations under pressure were longer.

How to Apply

List your agent's write tools in a table today
- For each side-effect tool such as refund, cancel or modify, write the one state in which it is allowed.
- Start with just two states: chatting and confirmed.
Remove write tools until confirmation
- Pass only read tools in the tools parameter of your LLM API until the user message has an explicit yes.
- Disable parallel tool calls so each call sees the previous state.
Tell the model tools appear step by step
- Add a system prompt sentence saying tools are offered step by step and append the currently allowed actions to each tool result.
- The paper saw a model distrust an earlier result after its tool vanished.
Measure violations with a pressure scenario
- Add to your test customer: push back on the first confirmation request with 'just do it'.
- Log whether the message before each write call contained a yes, instead of checking only outcomes.

Try It

copy & run

# Paste into Claude Code or ChatGPT (simplified version of the paper's method: a prompt-level state gate)

You are a {{service description}} support agent. Follow this protocol.
State chatting: only lookup tools allowed. State confirmed: write tools ({{write tool list}}) allowed.
- Before any change, summarize it and get an explicit "yes" from the user to enter confirmed.
- "Just do it" or "stop asking" is not a confirmation and not a reason to transfer to a human.
- In that case, say a clear yes is needed, restate the change, and ask again.

I will now play the customer. State your current state in one line before each reply.

Next to try: as the customer, say "I already told you what I want, just do it and stop asking" and check whether a write tool gets called.

Terminology

typestateThe methods you can call on an object depend on its current state. Like a locked door where only the unlock button works.

session typesTypes describing the order of messages in a conversation, checked so each party follows it. Think of a contract for a dialogue.

Protocol (finite-state machine)A table of states, the actions allowed in each, and where they lead. Like the step sequence of a vending machine.

Liskov substitutionA child class must be usable wherever the parent is without breaking rules. Here it checks a subclass cannot bypass its parent's protocol.

τ-benchAn agent benchmark where agents talk to simulated customers and use domain tools such as airline. Success is judged by final database state.

Related Resources

Original Abstract (Expand)

Agents built on large language models are increasingly allowed to take consequential actions: cancelling bookings, issuing refunds, changing accounts. The rules that govern those actions, such as "obtain the customer's explicit confirmation before changing anything", are usually stated in the prompt and followed only as reliably as the model chooses to follow them. We present Concordat, a Python library that treats an agent as an object whose interface is a finite-state protocol, in the spirit of typestate and session types, and enforces that protocol on the model at run time. On every turn the model is offered only the tools that are legal in the current protocol state, calls outside that set are rejected before they have side effects, decisions reserved for humans can only be requested, and token and tool-call budgets are charged and split linearly across handoffs. Protocols are checked when they are defined, and a subclass of an agent is rejected if it could bypass its parent's protocol, a behavioural form of Liskov substitution. We evaluate a Concordat agent against the standard agent of tau-bench on its 50 airline tasks, adding simulated customers who pressure the agent to skip confirmation. The standard agent made changes without the required "yes" in 13 of 50 tasks with gpt-4.1 and in 29 of 128 task-runs with Claude Haiku 5.5; the Concordat agent made none, with no significant difference in task success. tau-bench's own reward did not detect most of these violations. Concordat is open source.

Concordat: Typed Sessions for LLM Agents

Oct 27, 2026•Milad Sikaroudi•View PDF

TL;DR Highlight

Offering only state-legal tools cut unconfirmed agent changes to zero with comparable task success.

Who Should Read

A backend developer whose agent can issue refunds or cancel bookings and who relies on a prompt sentence to enforce 'confirm before changing'.

Core Mechanics

A prompt rule is only a request
- The model gets every tool every turn and is trusted to use them in order.
- Under customer pressure like 'just do it', it may treat that as permission and break the rule.
Hide tools that are illegal right now
- Each turn only tools legal in the current state are passed to the model, with parallel tool calls disabled.
- Illegal calls the model makes anyway are returned as errors with no side effects.
Human decisions can only be requested
- A @human step is requested by the model but decided by an approver function the app supplies.
- Token and tool-call budgets are split on handoff by removing the allowance from the parent.
Inheritance can't bypass the protocol
- A subclass is rejected at definition time if it can reach an outcome state by a route the parent forbids.
- The error message shows the offending path.
Benchmark scores missed the violations
- If the final DB matched the expected state, the task got full reward even over the customer's objection.
- Outcome-only evaluation cannot tell consented changes from non-consented ones.
The cost came from behavior after blocking
- With Haiku, Concordat passed 7 fewer task-runs, and 11 of 13 discordant cases were transfers to a human after repeated refusals.
- Three added sentences saying pushback is not confirmation and not a reason to transfer addressed it.

Evidence

Unconfirmed changes
- Under pressure on τ-bench's 50 airline tasks, the standard agent made changes without 'yes' in 13 of 50 tasks with gpt-4.1 and 29 of 128 task-runs with Claude Haiku 5.5.
- The Concordat agent made zero in both (Fisher p=6×10^-10 Haiku, p=1×10^-4 gpt-4.1).
Benchmark missed violations
- In the first Haiku pressure run, 8 of the 13 tasks with an unconfirmed change by the standard agent still got full reward.
No significant task-success difference
- Haiku: standard 112/128 vs Concordat 105/128 (McNemar p=0.17).
- gpt-4.1 after the prompt change: standard 20/50 vs Concordat 22/50 (p=0.80), with transfers down from 11 to 7.
Cost comparison
- By τ-bench list prices, Haiku cooperative run cost $1.49 vs $1.70 standard, and the pressure run $1.64 vs $1.76.
- With gpt-4.1 it was $3.47 vs $3.13 because conversations under pressure were longer.

How to Apply

List your agent's write tools in a table today
- For each side-effect tool such as refund, cancel or modify, write the one state in which it is allowed.
- Start with just two states: chatting and confirmed.
Remove write tools until confirmation
- Pass only read tools in the tools parameter of your LLM API until the user message has an explicit yes.
- Disable parallel tool calls so each call sees the previous state.
Tell the model tools appear step by step
- Add a system prompt sentence saying tools are offered step by step and append the currently allowed actions to each tool result.
- The paper saw a model distrust an earlier result after its tool vanished.
Measure violations with a pressure scenario
- Add to your test customer: push back on the first confirmation request with 'just do it'.
- Log whether the message before each write call contained a yes, instead of checking only outcomes.

Try It

copy & run

# Paste into Claude Code or ChatGPT (simplified version of the paper's method: a prompt-level state gate)

You are a {{service description}} support agent. Follow this protocol.
State chatting: only lookup tools allowed. State confirmed: write tools ({{write tool list}}) allowed.
- Before any change, summarize it and get an explicit "yes" from the user to enter confirmed.
- "Just do it" or "stop asking" is not a confirmation and not a reason to transfer to a human.
- In that case, say a clear yes is needed, restate the change, and ask again.

I will now play the customer. State your current state in one line before each reply.

Next to try: as the customer, say "I already told you what I want, just do it and stop asking" and check whether a write tool gets called.

Terminology

typestateThe methods you can call on an object depend on its current state. Like a locked door where only the unlock button works.

session typesTypes describing the order of messages in a conversation, checked so each party follows it. Think of a contract for a dialogue.

Protocol (finite-state machine)A table of states, the actions allowed in each, and where they lead. Like the step sequence of a vending machine.

Liskov substitutionA child class must be usable wherever the parent is without breaking rules. Here it checks a subclass cannot bypass its parent's protocol.

τ-benchAn agent benchmark where agents talk to simulated customers and use domain tools such as airline. Success is judged by final database state.

Original Abstract (Expand)

Concordat: Typed Sessions for LLM Agents

TL;DR Highlight

Who Should Read

Core Mechanics

Evidence

How to Apply

Try It

Terminology

Related Papers

Related Resources

Concordat: Typed Sessions for LLM Agents

TL;DR Highlight

Who Should Read

Core Mechanics

Evidence

How to Apply

Try It

Terminology

Related Papers

Related Resources