Gemini Decisions API
TL;DR Highlight
Scoring up to 1,000 questions in one pass without generating text: 150-250 ms end to end
Who Should Read
Backend developer struggling with latency and cost of LLM calls used for query routing, policy checks, or content grading
Core Mechanics
- It scores options instead of generating tokens
- gemini-decision-1 scores candidate options in a single forward pass and returns probabilities and confidence scores.
- End-to-end latency is 150-250 ms, with 50-100 ms of median model compute.
- Three question types in one request
- noul gives a yes/no probability, and choice picks a winner from 2 to 255 options with normalized probabilities.
- score grades against an ordered rubric starting at 0 and returns a weighted score plus per-level probabilities.
- One input, up to 1,000 questions
- You can mix text, images, audio, video, and PDFs up to 256K tokens.
- The shared input is billed once per request, and each question adds only its own instruction and criteria tokens.
- You pay only for input tokens
- It costs $0.10 per 1M input tokens and $0 for output.
- There are no thinking-token charges and no separate audio transcription or OCR costs.
- Nearly 5x cheaper than Gemini 3.5 Flash-Lite on the Interactions API
- This is on the shared benchmark suite described in the post.
- Google says it leads on agent routing, reasoning, ranking, and multimodal benchmarks with the lowest calibration error (gap between stated confidence and actual accuracy).
- Google uses it internally as a control layer
- Gemini Notebook routes queries to a model tier and thinking level, cutting latency by up to 40%.
- Google Search uses it to detect adversarial behavior in MCP tools, and CC uses it to pick from 200+ emoji.
Community Reactions
- Comment opinion that it is slightly pricier than other decision models but accepts audio and video
- They said it is especially useful in those specific cases.
- Commenter's quick evals showed 2-6x lower Brier score (probability error) than Jev
- Results varied by task, and p50 latency was over 100 ms lower.
- They said the whole eval cost twice as much.
- Skeptical comment asking why use a non-reasoning model for classification
- They asked how it differs from training your own classifier, other than pretrained general knowledge.
- Comment suggesting alternatives for local or CPU use
- jeffyclassify.com was mentioned, offering pretrained classifiers and training tools.
- Another comment raised concern about discriminatory decisions based on names or status.
How to Apply
- Move one routing prompt to a choice question
- Get an API key in AI Studio today and port one intent classifier using the free tier.
- Put each option in criteria and compare latency with your current generation call.
- Branch to human review on a confidence threshold
- Send low-confidence policy checks or grading results to a fallback model or reviewer queue.
- Pick the threshold using a labeled sample of your own.
- Batch questions on one input to cut calls
- Put all checks for the same document into one request's questions.
- Add variants of the same question to cross-check probabilities.
- Try dropping transcription/OCR steps in audio and PDF pipelines
- Feed call recordings or scanned PDFs directly as input.
- Compare accuracy and cost against the current pipeline before removing a step.
Try It
# Run in a terminal (pip install google-genai, GEMINI_API_KEY env var required)
from google import genai
client = genai.Client(http_options={"api_version": "v1beta"})
result = client.decisions.create(
model="gemini-decision-1",
input=[{"type": "text", "text": "{{text to classify}}"}],
questions={
"route": {
"type": "choice",
"instructions": "Which tier should handle this query?",
"criteria": {
"fast_model": "Simple everyday question.",
"reasoning_model": "Complex task needing deep reasoning.",
},
},
"needs_human": {
"type": "noul",
"instructions": "Does this need a human reviewer?",
},
},
)
print(result)
# Next: add a branch that sends low-confidence results to a fallbackTerminology
Related Papers
Gemini 4 Neon
Gemini 4 Neon cuts cost per task 32% versus Carbon, at $3 input and $15 output per million tokens
AI is killing All About Berlin
After Google AI Overviews, a Berlin guide site lost 70-75% of its traffic.
Not Everything Needs an LLM
PostHog used small decision models to skip 98.7% of expensive LLM calls
GPT‑6.2 Astra and Sol
GPT-6.2 Astra and Sol cut costs sharply, but over-engineering and early-stop reports are mixed
Show HN: TrnRT – An open-source LLM inference engine for AWS Trainium
One trn2.48xlarge serves GLM-5.3-Flash through an OpenAI-compatible API using NKI megakernels.
Know When to Zoom: How Small Is Too Small for Vision-Language Models?
Splitting large photos into tiles raised Claude and Gemini accuracy by 11.1–14.5 points
Understanding Positional Embeddings in Modern LLMs
A walkthrough of how LLMs encode token position, from GPT-2 to RoPE
Show HN: DeepSeek V3 in ~200 lines of pure Jax
DeepSeek V3's core design in 190 lines of JAX, trained on a MacBook Air in 2.6 hours