Show HN: DeepSeek V3 in ~200 lines of pure Jax
TL;DR Highlight
DeepSeek V3's core design in 190 lines of JAX, trained on a MacBook Air in 2.6 hours
Who Should Read
An LLM app developer who wants to see how DeepSeek V3's MLA and MoE actually work in readable code
Core Mechanics
- DeepSeek V3 fits in 190 lines
- MLA, DeepSeekMoE, aux-loss-free balancing and MTP are implemented in pure JAX.
- No Flax, Equinox or PyTorch; params are plain dicts of arrays.
- MLA cuts the cache from 480 to 144
- MLA (Multi-head Latent Attention) caches one small latent per token instead of full keys and values.
- In this implementation that is 480 down to 144 numbers per token per layer.
- MoE is 1 shared plus top-4 of 16 routed
- The shared expert always runs.
- The model has 36.2M parameters, 13.6M active per token.
- Bias balancing cost nothing in quality
- At step 2000, expert load was 5.7-6.9% with bias and 0.09-18.6% without.
- Validation loss was 3.921 vs 3.923, which the author says matches the paper's claim.
- Training took about 2.6 hours on a MacBook Air M5 24GB
- 12,000 steps (~98M tokens) on 207 public-domain detective books, 21.7M training tokens.
- Median speed was ~10.7k tokens/s and validation loss went from 5.02 (step 250) to 3.51.
- It is an architecture demo, not a usable LM
- Every expert runs on every token and unpicked ones are zeroed, so there is no sparse dispatch.
- FP8 training, DualPipe, YaRN and post-training are missing, and scale is 36.2M vs 671B params, 256 vs 128K context.
Community Reactions
- A roadmap question for non-JAX devs
- A Java/Python SWE asked for a roadmap to understand this kind of build.
- No answer appeared in the thread.
- Comparability of the dataset choice
- One commenter said using datasets others have used makes results directly comparable.
- The author chose detective fiction on purpose for their own project.
- It ran end to end on an AMD RX 7600 XT
- A user on Fedora with ROCm said JAX worked without CUDA.
- It needed HSA_OVERRIDE_GFX_VERSION=11.0.0 and XLA_FLAGS=--xla_gpu_enable_triton_gemm=false, with container setup in PR #1.
- Plans to reuse it for small models
- A commenter training sub-10M models liked the detailed write-up and sample outputs.
- Another said visual diagrams of the architecture make it easier to reason about.
How to Apply
- Download the checkpoint and sample text
- Today, fetch the v1.0 checkpoint and tokenizer and generate 3 samples with sample.py.
- The checkpoint is a pickle, so only load it from trusted sources.
- Read deepseek_v3.py and compute the MLA cache size
- Open mla and init_cache and trace why it is 144 numbers per token.
- Compare with your service's KV cache memory math.
- Reproduce the bias on/off ablation
- Run train.py with --steps 2000, with and without --no-bias.
- Compare per-expert load in metrics.jsonl.
- Start with the small config on weaker machines
- On 8-16GB Macs or CPU, use JAX_PLATFORMS=cpu and --config small.
- Note it was only checked for a couple of steps, not trained to the end.
Try It
# Paste into a shell (Apple Silicon Mac, Python 3.12, uv required): generate text from the trained checkpoint
git clone https://github.com/Vaibhaav-Tiwari/deepseek-v3-jax.git && cd deepseek-v3-jax
uv venv -p 3.12 .venv
uv pip install --python .venv/bin/python -r requirements.txt
mkdir -p data runs/main
curl -L -o runs/main/ckpt.pkl https://github.com/Vaibhaav-Tiwari/deepseek-v3-jax/releases/download/v1.0/ckpt.pkl
curl -L -o data/tokenizer.json https://github.com/Vaibhaav-Tiwari/deepseek-v3-jax/releases/download/v1.0/tokenizer.json
.venv/bin/python sample.py --prompt "The fog came in off the" --num 3
# Next: run JAX_PLATFORMS=cpu .venv/bin/pytest tests.py -q to check the 39 testsTerminology
Related Papers
Gemini 4 Neon
Gemini 4 Neon cuts cost per task 32% versus Carbon, at $3 input and $15 output per million tokens
AI is killing All About Berlin
After Google AI Overviews, a Berlin guide site lost 70-75% of its traffic.
Not Everything Needs an LLM
PostHog used small decision models to skip 98.7% of expensive LLM calls
GPT‑6.2 Astra and Sol
GPT-6.2 Astra and Sol cut costs sharply, but over-engineering and early-stop reports are mixed
Show HN: TrnRT – An open-source LLM inference engine for AWS Trainium
One trn2.48xlarge serves GLM-5.3-Flash through an OpenAI-compatible API using NKI megakernels.
Know When to Zoom: How Small Is Too Small for Vision-Language Models?
Splitting large photos into tiles raised Claude and Gemini accuracy by 11.1–14.5 points
Understanding Positional Embeddings in Modern LLMs
A walkthrough of how LLMs encode token position, from GPT-2 to RoPE
Nvidia allegedly engineered scarcity by funding AI startups to buy its GPUs
WSJ alleges NVIDIA funded startups that then bought its own GPUs to prop up demand.