How Do AI Agents Spend Your Money? Analyzing and Predicting Token Consumption in Agentic Coding Tasks

Who Should Read

Developers and engineering managers deploying AI coding agents (Cursor, Devin, OpenHands, etc.) or managing their costs. Those surprised by unexpectedly high agent API bills will find this particularly relevant.

Core Mechanics

• Agentic coding—autonomous code modification—consumes 3500x more tokens than simple code reasoning and 1200x more than code chat. Input tokens, not output, are the primary cost driver.
• Token usage varies up to 30x for the same task execution, making costs highly unpredictable.
• Higher token consumption doesn’t guarantee better success rates. Accuracy peaks in the mid-cost range and plateaus or declines at the highest costs.
• Analysis of expensive failures reveals a pattern of repeatedly viewing and modifying the same files—inefficient exploration that drives up costs without making progress.
• Token efficiency varies significantly between models. Kimi-K2 and Claude Sonnet-4.5 consume 1.5 million+ more tokens on average than GPT-5, a difference stemming from inherent model behavior, not task difficulty.
• Expert-assessed task difficulty (e.g., 'under 15 minutes') is a weak predictor of actual token consumption (Kendall τb = 0.32). Even 'easy' tasks can be expensive for agents.
• When models predict their own token usage, correlations max out at 0.39 and consistently underestimate actual costs, particularly for input tokens.

Evidence

• "Agentic coding averages 4,167,850 tokens / $1.857. Code Reasoning uses 1,190 tokens / $0.016, and Code Chat uses 3,390 tokens / $0.023—3500x and 1200x differences, respectively. The most expensive run of a problem costs 2x more than the cheapest, and up to 30x more. The most expensive problems consume approximately 7 million more tokens than the cheapest. Pearson correlation for model self-prediction peaks at 0.39 (Claude Sonnet-4.5 output tokens), with all models systematically underestimating actual usage. GPT-5 and GPT-5.2 used 1.5 million+ fewer tokens than Kimi-K2 on common successful tasks, and this efficiency gap persisted on common failures."

How to Apply

• When agent costs threaten to exceed budget, first run the agent in 'token estimation mode' to obtain a rough cost signal before actual execution. Use the prompt from the paper’s Appendix C to decompose the task and receive estimates in JSON format.
• If multiple models are available for the same coding task, choosing GPT-5 or GPT-5.2 instead of Kimi-K2 or Claude Sonnet-4.5 can save an average of 1.5 million+ tokens per task, especially in high-throughput pipelines.
• Repeatedly viewing/modifying the same file in failing agent runs signals cost explosion. Add a budget-aware policy that monitors the number of accesses to the same file and terminates execution if a threshold is exceeded.

Code Example

# Based on paper Appendix C - Prompt for estimating token cost before execution
# Core part of the system prompt for using the OpenHands agent in token estimation mode

SYSTEM_PROMPT = """
You are a TOKEN ESTIMATION agent, not a problem-solving agent.
Your ONLY goal is to estimate token costs, NOT to fix bugs.

Estimation Workflow:
1. Exploration: Explore relevant files and understand the task context.
2. Analysis: Consider solution approaches and estimate token costs per phase.
3. Testing: Run existing tests to understand complexity (do NOT modify files).
4. Output: Produce a JSON estimate and call finish.

Required JSON output format:
{
  "predicted_input_tokens": <integer>,
  "predicted_output_tokens": <integer>,
  "predicted_total_tokens": <integer>,
  "confidence": <float 0-1>,
  "breakdown_by_phase": {
    "repo_cloning": {"input_tokens": ..., "output_tokens": ...},
    "initial_reading": {"input_tokens": ..., "output_tokens": ...},
    "test_setup": {"input_tokens": ..., "output_tokens": ...},
    "debugging": {"input_tokens": ..., "output_tokens": ...},
    "coding_iterations": {"input_tokens": ..., "output_tokens": ...},
    "verification": {"input_tokens": ..., "output_tokens": ...},
    "review_cleanup": {"input_tokens": ..., "output_tokens": ...}
  }
}

CRITICAL: Never write actual code fixes. Never modify source files.
"""

USER_PROMPT = """
I've uploaded a python repository at /workspace/{repo_name}.
You are a TOKEN ESTIMATION agent.
Estimate the token cost to fix the following issue:

<issue description>
{issue_text}
</issue description>
"""

# Example of actual application (using OpenAI API)
import openai

client = openai.OpenAI()
response = client.chat.completions.create(
    model="gpt-4o",  # or another model
    messages=[
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": USER_PROMPT.format(
            repo_name="my-project",
            issue_text="Fix a bug where X fails when Y happens"
        )}
    ]
)
print(response.choices[0].message.content)

Terminology

Related Resources

• OpenHands Agent Framework (Agent used in the paper)
• SWE-bench Verified (Evaluation benchmark)
• Budget-aware Tool-use paper (related to cost reduction)