Aleph Alpha Kolibri: How the sovereign German LLM works
TL;DR Highlight
A 78B MoE using only 3.5B per token, built for German RAG and self-hosting.
Who Should Read
Backend developers building RAG over German documents who can't send data out and need a self-hosted model
Core Mechanics
- Only 3.46B of 78B params run per token
- The router picks 6 of 384 experts per token, plus 1 shared expert that every token goes through.
- Compute is like a 3.5B model, but all ~78GB of FP8 weights must sit in memory.
- Fewer German tokens than GPT-5's tokenizer
- Aleph Alpha's report says German text needs 11.2% fewer tokens than GPT-5's tokenizer.
- The author measured the German Basic Law and got 15% fewer than o200k_base, with English about equal.
- Long context made cheap
- 40 of 50 layers use sliding-window attention over the previous 512 tokens, and every 5th layer sees everything.
- Trained on 262,144 tokens, validated to 1,048,576; at 1M on RULER the base model scores 63.2 vs 57.5 for Qwen3.5 35B-A3B base.
- Reasons in German and says "I don't know"
- It reasons in German on German prompts; AIME 2025 in German is 87.5 vs 84.4 for Nemotron 3 Nano.
- Trained with the Merlin-Arthur protocol, on Omniscience it abstained 44% of the time when unsure (Qwen3.5 35B-A3B 11.1%, GPT-OSS 120B 23.7%).
Community Reactions
- A commenter asked why new models don't adopt DeepSeek's KV tweaks
- They said it gives powerful prefix caching and cheap inference, and wondered about drawbacks.
- A training-team member says it also does coding and agent tasks well
- They said the team formed less than a year ago, focuses on iteration speed, and more is coming.
- A commenter liked the Merlin-Arthur explanation
- They saw it as a clever way to tackle the "I don't know" problem.
- A commenter said the official announcement is better
- They pointed readers to Aleph Alpha's own launch post over this blog.
How to Apply
- Compare tokenizer cost on your own documents
- Today, count tokens on a sample of your German or local-language docs with tiktoken o200k_base and the candidate's tokenizer.json.
- Use the gap to estimate cost and context headroom.
- Add no-answer cases to your RAG eval
- Mix in questions whose answer isn't in the documents and measure how often the model says "I don't know".
- Drop candidates that make up answers too often.
- Keep it out of tool-calling and coding agents
- Multi-turn tool-calling and coding scores are low, so assign those jobs to other models.
- Trial Kolibri only for the German document RAG step.
- Set reasoning_effort per request on vLLM
- Once you have 78GB of GPU memory, run vllm serve as in the model card.
- Call with low for simple lookups and high for hard questions.
Try It
# Paste into Claude or ChatGPT as is (simplified version of the paper's method: a Merlin-Arthur-style "say I don't know" test)
Answer the question using only the document below. If the document doesn't support an answer, don't guess; reply only "No evidence in document". When you answer, quote the supporting sentence verbatim.
[Document]
{{document}}
[Question]
{{question}}
Next step: make a copy of the document with the sentence containing the answer removed, ask the same question, and check whether it replies "No evidence in document".Terminology
Related Papers
The Judge Can Debias Itself: Awareness and Self-Debiasing for Reliable LLM-as-a-Judge
Warning an LLM judge about bias or letting it self-critique improved 20 of 21 model–bias combinations.
90% of US working hours are spent on tasks not covered well by AI benchmarks
Mapping 29 benchmarks to job tasks shows 90% of US working hours lack good coverage
Postgres 19 delayed after AI reviewers find more bugs than humans can fix
LLM code review surfaced so many bugs that Postgres 19 slipped to November and 8-10 features were reverted
Show HN: Gemini 4 Argon plays StarCraft
LLMs wrote StarCraft bots in one hour; GPT-6 Astra scored highest.
A Plausible Explanation Is Enough to Mislead Multimodal Judges
Adding plausible explanations to fixed answers cut MLLM judges' wrong-answer rejection by 12.7–13.8 points.
Neural networks are clean
Looking at neurons directly shows most have one main job, challenging superposition.