German AI consortium releases Soofi S, an open 30B model that tops benchmarks

TL;DR Highlight

Soofi S, activating only 3.2B parameters, tops fully open models in English and German benchmarks

Who Should Read

A backend developer who serves long-context requests and wants a self-hostable open model with good German quality

Core Mechanics

Only 3.2B of 31.6B parameters run
- Soofi S is a mixture-of-experts model that activates about 3.2B parameters per token.
- Its compute cost is closer to a 3B model than a conventional 30B one.
Speed barely drops on long inputs
- The hybrid of Mamba-2 and attention layers keeps a KV cache in only 6 of 52 layers.
- At 40,000 tokens with 32 parallel requests, it generates about 8x more tokens per second per GPU than dense 14-24B models.
- Throughput stays nearly flat from 4,000 to 256,000 tokens.
German share was raised on purpose
- German is 7.2% of the first-phase mix and 15.3% of the second, out of about 27T tokens.
- Nvidia's Nemotron reference recipe gives all non-English languages combined about 5%.
Top among fully open models
- Per the report, it leads fully open models on aggregate German and English scores among 16 compared models.
- It scores 73.8% on HumanEval, 70.2 on MBPP and 84.2 on German MBPP, best among open peers.

Community Reactions

Claim that it tops benchmarks by training on them
- One commenter argued the model tops benchmarks because benchmark data was in its training set.
Comparison models are not the latest
- A commenter noted it compares against Qwen 3.5 and Gemma 3 though Qwen 3.6 and Gemma 4 have been out about three months.
Slow access approval
- A commenter said they requested access on Hugging Face and had no answer after a day.
More about infrastructure than model
- A commenter welcomed two open models from outside the US-China duopoly in the same week.

How to Apply

Measure long-context throughput
- Request access on Hugging Face today and, once approved, serve it on your stack such as vLLM.
- Compare 4K, 32K and 128K inputs against your current dense model on the same GPU and concurrency.
Test on your own German eval set
- Pick 50 real German summarization or QA cases and run them side by side with your current model.
- Decide on switching from these results, not public benchmark scores.
Keep your current model for weak areas
- Do not switch if math solving or word-frequency extraction in long documents is core.
- Test those separately against models like Qwen3.5 35B-A3B.
Auto-check training data for contamination
- Add a script to your pipeline that compares fine-tuning data against eval questions at the string level.
- Do not trust dataset split labels.

Try It

copy & run

# Paste into Claude or ChatGPT — simplified version of the article's lesson (compare content, not split labels)

I have a fine-tuning dataset and an eval set. Write a Python script that checks whether items overlap (contamination) between them.

Requirements:
- Training file: {{train.jsonl}}, eval file: {{eval.jsonl}}
- Text field name in each line: {{field_name}}
- Do not use split names or labels, compare text content only
- After lowercasing and whitespace cleanup, find suspicious items by exact match and n-gram overlap ratio
- Print the list and count of overlapping eval items

Next step: open the flagged items, remove them from the training data, re-run the evaluation, and show me how the score changes.

Terminology

MoE (Mixture-of-Experts)An architecture that uses only a few of many expert subnetworks per input. Like calling only the relevant department to each meeting.

Mamba-2 layerA layer that carries a fixed-size summary state of earlier content instead of attention. Memory does not grow much with longer inputs.

KV cacheMemory where attention stores earlier tokens so it need not recompute them. It grows like a pile of notes as a conversation lengthens.

Chinchilla scaling lawsDeepMind's 2022 rules for balancing model size and training data, with a sweet spot of about 20 tokens per parameter.

Data contaminationEval questions leaking into training data. Like a student scoring well after seeing the exam beforehand.