Qwen 3.8 27B available on Cerebras at 1500 tokens/s

TL;DR Highlight

Cerebras serves Qwen 3.8 27B at ~1850 tokens/s, but rate limits and costs bite

Who Should Read

A backend developer wiring an ultra-fast inference API into coding agents or sub-agents

Core Mechanics

Qwen 3.8 27B is now on Cerebras Shared Inference
- Model ID is qwen-3.8-27b with 27 billion parameters.
- Context is 64k on free and 128k on paid.
Catalog speed is about 1850 tokens/s
- The other listed model, gpt-oss-120b, is about 3000 tokens/s.
- The 1500 tokens/s in the post title is not from the docs page itself.
Shared Inference models are unpruned originals
- Pruning permanently removes layers or experts and changes the architecture.
- REAP-pruned models are only on Hugging Face, not served via the API.
Quantization is used only for storage
- Weights are stored in 16/8/4-bit, with sensitive layers at full precision and dequantized on the fly.
- Activations, attention, and kv cache stay in full precision.
They promise not to alter weights under existing model IDs

Community Reactions

Coding costs blew up for one user
- The same task on DeepSeek-V4-Flash reportedly took 172s and cost $0.024.
- Qwen 3.8 27B hit the 450,000 TPM limit in about 90 seconds and burned $1.10.
- Cached tokens count toward the limit.
No cache discount hurts long sessions, one commenter says
- A commit review with pi cost $1.60 in 5.1 min, p50 890 tok/s, 0.64s TTFT.
- OpenRouter averages would have been $0.29 in 14.4 min, so 5.6x pricier and 2.8x faster.
Fast output didn't fix the bottleneck for one user
- Input reading wasn't faster and tool calls failed more than DS4.
- Total wait time ended up about the same.
Rate limits and account issues drew complaints
- One says the 150k TPM public limit makes it unusable for many coding tasks.
- Another hit compaction trouble with 128k context, and a sub-agent exceeded the window in opencode.

How to Apply

Benchmark cost on one of your agent tasks
- Run the same task today on your current model and on Qwen 3.8 27B.
- Record total cost, time, and number of 429s.
- Check how cached tokens count against limits and cost.
Use it only for short-context sub-agents
- Don't use it for long-running main agents given 64k/128k context.
- Route small-input steps like code review or quick queries to it.
Add 429 retry and context-cap handling
- Add backoff retries on 429 responses.
- Count prompt tokens before sending and fall back to another model above 64k/128k.

Try It

copy & run

# Prompt for Claude or ChatGPT: build a cost and rate-limit checklist before adopting the model

I'm considering adding Cerebras qwen-3.8-27b to my coding agent.
Constraints: context 64k free/128k paid, speed ~1850 tokens/s, no cache discount, TPM limit (cached tokens count).

Our workload:
- Avg input tokens per session: {{avg input tokens}}
- Avg output tokens per session: {{avg output tokens}}
- Requests per minute: {{RPM}}
- Current model and pricing: {{model/price}}

Using this, make a table of (1) whether tokens per minute exceed the TPM limit, (2) cost difference vs. my current model without caching, (3) context overflow risk zones.

Next step: split which steps go to Cerebras and which stay on the current model, and write the routing rules.

Terminology

TPMThe cap on tokens processed per minute. Like how much water a faucet can pour per minute.

PruningPermanently removing layers or experts from a model to shrink it. Like cutting chapters out of a book, so it becomes a different model.

QuantizationLowering the numeric precision of weights to save memory. Like lowering photo resolution while keeping the composition.

TTFTTime to first token after a request. Like the wait before the first dish arrives.

Prompt CachingReusing a repeated prompt prefix to cut cost and time. Like not re-reading the start of a book you've already read.