Tokenomics: Quantifying Where Tokens Are Used in Agentic Software Engineering

Who Should Read

Backend or platform developers struggling with unpredictable token costs in multi-agent coding pipelines

Core Mechanics

• Code review eats 59.4% of tokens
  • The authors analyzed execution traces of 30 development tasks run on ChatDev with a GPT-5 reasoning model.
  • The Code Review stage averaged 59.4% of all tokens, the largest share.
• Input tokens average 53.9%
  • Comparing input, output and reasoning tokens, input was the largest share.
  • The authors take this as evidence of possible inefficiencies in agent collaboration.
• Cost comes from verification, not generation
  • They conclude the main cost is in automated refinement and verification rather than initial code generation.
• They provide a stage-by-stage measurement method
  • ChatDev phases are mapped to Design, Coding, Code Completion, Code Review, Testing and Documentation.
  • The authors say this helps predict costs and optimize workflows.
• Results are preliminary
  • The paper itself calls them preliminary findings.
  • It covers one framework, ChatDev, and 30 tasks.

Community Reactions

• Experience that input dominates
  • One commenter sees roughly a 10:1 input-to-output ratio in their own usage.
  • They say an agent may read a million tokens just to patch one line.
• Opinion that small sample still matches practice
  • A commenter first thought 30 tasks was few, but said code review consuming most tokens matches what they have seen.
• Experience of unaccounted AI costs
  • A commenter saw a single simple query burn 250k tokens in a product demo.
  • The vendor could not say whether the license had a token budget cap.
• Experience of Copilot running out of tokens in two days
  • After a pricing change, one commenter ran out of tokens in two days and called token pricing arbitrary.
• One commenter pointed out that Tokenomics already refers to crypto economics.

How to Apply

• Break down tokens per stage from your agent logs today
  • Collect the usage fields from 20-30 recent agent runs and sum them by stage (generation, review, testing).
  • Record input, output and reasoning tokens separately to see which is largest.
• Cap review loop iterations
  • Set a maximum number of review-fix rounds and stop when exceeded.
  • Compare token usage and pass rate before and after the cap.
• Start optimizing by shrinking input context
  • Input is the largest share, so pass the review stage only changed parts of files and history.
  • Compare cost on the same tasks before and after.
• Add per-task token budget alerts
  • Log total tokens per task and send an alert when it exceeds a threshold.

Try It

# Paste into Claude/ChatGPT (simplified version of the paper's method: per-stage token analysis)

Below is my agent run log. Classify each call into Design, Coding, Code Completion, Code Review, Testing, or Documentation, and make a table of input/output/reasoning token totals and the percentage of overall usage per stage.
Then pick the stage that uses the most tokens and suggest 3 ways to reduce its input tokens.

Log:
{{agent run log or usage JSON}}

Terminology

Related Papers

• DeepInfer: High-performance inference framework for DeepSeek models

  DeepSeek open-sourced DeepInfer, its production inference engine, under Apache-2.0.

• Mercury 3: Diffusion at scale

  Scaling diffusion LLM 40x gives Mercury 3: 4–8x faster than speed-tier models, similar quality

• DeepSeek banned from U.S. federal civilian agencies under new DHS order

  DHS order bans DeepSeek from U.S. federal civilian agency IT systems

• Timestral: Mistral's time-series foundation models

  A 209M time-series model ranks first on three benchmarks and runs on CPU.

• DeepSeek V4.1 Flash 1017 (Max)

  DeepSeek V4.1 Flash 1017 Max scores 45 on the Intelligence Index at 211 tok/s and $0.30 input.

• If you don't run your own LLM proxy, how do you know whats going on?

  Running your own LLM proxy shows which keys, models, and cache misses drive your usage.

• DeepSeek: DeepSeek v4.1 Flash 1017

  DeepSeek V4.1 Flash 1017: $0.05/M input, 1M context, built for agents

• DeepSeek-v4.1-Flash-1017

  DeepSeek V4.1 Flash minor update lifts coding and agent benchmarks while cutting run time and output length

Related Resources

• https://arxiv.org/abs/2601.14470