Cerebras Deploys Trillion-Parameter AI Models

Oct 25, 2026•gmays•View Original

TL;DR Highlight

Cerebras fits Kimi K3 (3T) across 59 wafers in a pipeline, serving 1000-2000+ tokens per second

Who Should Read

A backend developer choosing an LLM inference API who wants to understand how Cerebras reaches its speed

Core Mechanics

One wafer has only 44GB of memory
- A WSE-3T wafer has 44GB of on-chip memory.
- Trillion-parameter models need terabytes for weights alone.
Each Kimi K3 layer is about 16GB
- That comes from 3T parameters at 4 bits split across 93 layers (69 linear-attention plus 24 dense).
- They assume an 8-bit KV cache and a 256k average request.
Wafer placement depends on layer type
- A dense (full-attention) layer needs KV cache room, so one goes per wafer.
- A linear-attention layer keeps a small fixed state per request, so two (32GB) fit per wafer.
- That totals 35 + 24 = 59 wafers.
Weights stay put, only activations move
- Whole layers fit on one wafer, so wafers exchange only activations.
- The link is RDMA over Ethernet at roughly 2 microseconds per hop.
The pipeline works like an assembly line
- Each wafer is a pipeline stage working on a different request at the same time.
- Requests in flight are roughly stages times micro-batch size.
KV cache memory caps concurrency
- The SRAM left on each dense-layer wafer is the KV cache budget.
- Smaller micro-batches make each request faster, larger ones raise total throughput.

Community Reactions

Outdated models were the top complaint
- One commenter said the models available on Cerebras, like GPT OSS 120b and Qwen 3.8 27B, aren't useful for them.
- Another argued that with a 6-9 month lag, better models should arrive soon.
Fast but inaccurate, per user experience
- One user ran GPT OSS 120b on Cerebras and found it flies but is mostly wrong.
- Another criticized citing full-precision benchmarks while serving 4-bit quants.
Cheap plans reportedly lack reliability
- One commenter said cheap plans are a sales funnel sharing limited resources, so they are unreliable.
- They also said plans starting around $1500/month are worth it for some uses.
Commenters asked about gaps in the post
- One noted the post doesn't explain how prefill is made fast on this interconnect.
- Another said 59 wafers looks like an architecture problem.
- One commenter estimated about 4.1 J/token from 20kW per wafer, 1500 tok/s and 190 concurrent requests.

How to Apply

Measure where your agent's time goes
- Run one agent task today and log LLM generation time separately from tool-call and build time.
- If generation dominates, comparing fast inference providers is worthwhile.
Compare providers on tok/s and accuracy together
- Send the same 20 prompts to each candidate and measure real tok/s and correctness.
- Check each provider's docs for quantization and record it in your table.
Estimate your concurrency limit for long contexts
- Write down your average context length and concurrent requests, then check whether KV cache memory is the cap.
- Account for whether the model uses dense or linear attention, since memory needs differ.

Try It

copy & run

# Paste into Claude or ChatGPT (simplified version of the article's calculation)

Model info:
- Parameters: {{parameter count}}
- Weight precision: {{bits}}
- Layers: {{layer count}}
- Memory per device: {{memory GB}}

Calculate in this order.
1. Total weight size and size per layer
2. How many layers fit on one device
3. Number of devices needed
Show the formula and assumptions for each step.

Next step: with an average request length of {{tokens}}, compute KV cache size per request, then how many concurrent requests the leftover memory supports.

Terminology

WSE-3TCerebras's wafer-sized chip. The whole wafer is one chip, so it has plenty of on-chip memory.

KV cacheA notepad of keys and values for tokens already processed. It grows with longer requests and more concurrent ones.

Linear attentionAttention that keeps a fixed-size state per request instead of storing every token. Memory doesn't grow with context length.

ActivationThe intermediate result a layer passes to the next. Like a half-finished part moving along an assembly line.

RDMA over EthernetA network method for machines to read each other's memory directly without going through the CPU.

Micro-batchA small group of requests a pipeline stage processes at once.

Related Resources

Cerebras Deploys Trillion-Parameter AI Models

Oct 25, 2026•gmays•View Original

TL;DR Highlight

Cerebras fits Kimi K3 (3T) across 59 wafers in a pipeline, serving 1000-2000+ tokens per second

Who Should Read

A backend developer choosing an LLM inference API who wants to understand how Cerebras reaches its speed

Core Mechanics

One wafer has only 44GB of memory
- A WSE-3T wafer has 44GB of on-chip memory.
- Trillion-parameter models need terabytes for weights alone.
Each Kimi K3 layer is about 16GB
- That comes from 3T parameters at 4 bits split across 93 layers (69 linear-attention plus 24 dense).
- They assume an 8-bit KV cache and a 256k average request.
Wafer placement depends on layer type
- A dense (full-attention) layer needs KV cache room, so one goes per wafer.
- A linear-attention layer keeps a small fixed state per request, so two (32GB) fit per wafer.
- That totals 35 + 24 = 59 wafers.
Weights stay put, only activations move
- Whole layers fit on one wafer, so wafers exchange only activations.
- The link is RDMA over Ethernet at roughly 2 microseconds per hop.
The pipeline works like an assembly line
- Each wafer is a pipeline stage working on a different request at the same time.
- Requests in flight are roughly stages times micro-batch size.
KV cache memory caps concurrency
- The SRAM left on each dense-layer wafer is the KV cache budget.
- Smaller micro-batches make each request faster, larger ones raise total throughput.

Community Reactions

Outdated models were the top complaint
- One commenter said the models available on Cerebras, like GPT OSS 120b and Qwen 3.8 27B, aren't useful for them.
- Another argued that with a 6-9 month lag, better models should arrive soon.
Fast but inaccurate, per user experience
- One user ran GPT OSS 120b on Cerebras and found it flies but is mostly wrong.
- Another criticized citing full-precision benchmarks while serving 4-bit quants.
Cheap plans reportedly lack reliability
- One commenter said cheap plans are a sales funnel sharing limited resources, so they are unreliable.
- They also said plans starting around $1500/month are worth it for some uses.
Commenters asked about gaps in the post
- One noted the post doesn't explain how prefill is made fast on this interconnect.
- Another said 59 wafers looks like an architecture problem.
- One commenter estimated about 4.1 J/token from 20kW per wafer, 1500 tok/s and 190 concurrent requests.

How to Apply

Measure where your agent's time goes
- Run one agent task today and log LLM generation time separately from tool-call and build time.
- If generation dominates, comparing fast inference providers is worthwhile.
Compare providers on tok/s and accuracy together
- Send the same 20 prompts to each candidate and measure real tok/s and correctness.
- Check each provider's docs for quantization and record it in your table.
Estimate your concurrency limit for long contexts
- Write down your average context length and concurrent requests, then check whether KV cache memory is the cap.
- Account for whether the model uses dense or linear attention, since memory needs differ.

Try It

copy & run

# Paste into Claude or ChatGPT (simplified version of the article's calculation)

Model info:
- Parameters: {{parameter count}}
- Weight precision: {{bits}}
- Layers: {{layer count}}
- Memory per device: {{memory GB}}

Calculate in this order.
1. Total weight size and size per layer
2. How many layers fit on one device
3. Number of devices needed
Show the formula and assumptions for each step.

Next step: with an average request length of {{tokens}}, compute KV cache size per request, then how many concurrent requests the leftover memory supports.

Terminology

WSE-3TCerebras's wafer-sized chip. The whole wafer is one chip, so it has plenty of on-chip memory.

KV cacheA notepad of keys and values for tokens already processed. It grows with longer requests and more concurrent ones.

Linear attentionAttention that keeps a fixed-size state per request instead of storing every token. Memory doesn't grow with context length.

ActivationThe intermediate result a layer passes to the next. Like a half-finished part moving along an assembly line.

RDMA over EthernetA network method for machines to read each other's memory directly without going through the CPU.

Micro-batchA small group of requests a pipeline stage processes at once.

Cerebras Deploys Trillion-Parameter AI Models

TL;DR Highlight

Who Should Read

Core Mechanics

Community Reactions

How to Apply

Try It

Terminology

Related Papers

Related Resources

Cerebras Deploys Trillion-Parameter AI Models

TL;DR Highlight

Who Should Read

Core Mechanics

Community Reactions

How to Apply

Try It

Terminology

Related Papers

Related Resources