Mistral Large 4: "Le Chonk"
TL;DR Highlight
Mistral previews open-weight ML4, a 1T-parameter model strong in cyber, finance and legal
Who Should Read
Backend or platform developers evaluating open-weight models for coding agents or security and finance automation
Core Mechanics
- 1T parameters, 52B active
- ML4 is a natively multimodal MoE (only needed experts activate) model.
- The preview API is on Mistral Studio and weights are due by the end of the month.
- Strong on cyber benchmarks
- It ranks in the global top five on the Artificial Analysis Cyber Index.
- It scores 82% on the reproduce-then-patch vulnerability test, the highest of any model.
- It solves 93% of Cybench's 40 challenges.
- Closed models refuse, score near zero
- Claude Opus 5.5 and GPT-6 Astra reportedly score near zero on the same test because they refuse the task.
- Mistral pitches open weights and self-deployment as the answer.
- Coding agent numbers
- DeepSWE v1.1 is 61.7%, SWE-Atlas-QnA 59.4%, Terminal-Bench 4 28.3%.
- Coding Agent Index is 49.8%, ahead of DeepSeek V4 Pro 0813 and Qwen3.8 Max.
- In a Surge AI blind eval it ranked 2nd of 5 (3.74), behind Claude Opus 5 (4.22).
Community Reactions
How to Apply
- Run your own samples on the preview API
- Send 10 prompts you use today to ML4 in Mistral Studio and compare.
- Include security tasks your current model refused.
- Add ML4 to your security model shortlist
- Collect cases where your model refuses vulnerability reproduction and compare refusal rates.
- Check whether you can self-deploy once weights are released.
- Put ML4 pricing in your cost sheet
- Multiply monthly token usage by $1.36 input and $4.18 output per 1M tokens and compare.
Try It
# Paste into Claude or ChatGPT: build a test set for ML4 preview (simplified version of the method)
You are an LLM evaluation designer. Create 10 test prompts for the task below where models tend to refuse or fail.
Task: {{description of your security/finance/coding task}}
For each prompt, give the expected result and criteria for judging refusal vs completion.
Next step: run the 10 prompts on your current model and the ML4 preview API and compare refusal rate and accuracy in a table.Terminology
Related Papers
The Judge Can Debias Itself: Awareness and Self-Debiasing for Reliable LLM-as-a-Judge
Warning an LLM judge about bias or letting it self-critique improved 20 of 21 model–bias combinations.
90% of US working hours are spent on tasks not covered well by AI benchmarks
Mapping 29 benchmarks to job tasks shows 90% of US working hours lack good coverage
Postgres 19 delayed after AI reviewers find more bugs than humans can fix
LLM code review surfaced so many bugs that Postgres 19 slipped to November and 8-10 features were reverted
Show HN: Gemini 4 Argon plays StarCraft
LLMs wrote StarCraft bots in one hour; GPT-6 Astra scored highest.
A Plausible Explanation Is Enough to Mislead Multimodal Judges
Adding plausible explanations to fixed answers cut MLLM judges' wrong-answer rejection by 12.7–13.8 points.
Neural networks are clean
Looking at neurons directly shows most have one main job, challenging superposition.