LFM2.5-24B-A2B: Built for fast on-device agents

TL;DR Highlight

A 24B MoE with 2B active params matches 35B-class models on 5 of 8 agentic benchmarks.

Who Should Read

A backend developer who wants to run tool-calling agents locally because cloud API costs and data leakage are a concern

Core Mechanics

24B reasoning model with only 2B active
- It is an MoE (mixture of experts) that uses 2B of its 24B parameters per token.
- It is the largest and final model in the LFM2.5 family.
- Both the base and post-trained models are open-weight on Hugging Face.
Ahead on 5 of 8 agentic benchmarks
- Against Qwen3.6-35B-A3B it scores 27.69 vs 8.72 on AutomationBench and 48.09 vs 38.19 on LiveClaw.
- It also leads or matches on BFCLv4, ToolSandbox, and τ³-Bench Banking.
- It leads all three instruction-following benchmarks (IFEval, IFBench, Multi-IF).
Bigger models win on knowledge and coding
- GPQA Diamond is 74.42 vs 83.69 for Qwen3.6-35B-A3B.
- SWE-Bench Verified (OpenCode) is 51.00 vs 66.20.
- The post recommends pairing it with retrieval or web search for knowledge-heavy work.
Fastest on-device among compared MoEs
- It decodes 184 tokens/s on an Apple M5 Max and 114 tokens/s on an AMD Ryzen AI Max+ 395.

Community Reactions

Very fast on local GPUs, per commenters
- One user got 200-320 tok/s on a 5070ti with llama.cpp, 131k context, and no tuning.
- Another said it is very fast on a 4090 but also very dumb.
Real agent work disappointed some users
- One user ran it via OpenRouter in OpenCode following a plan from a larger model and it got stuck on many tool calls.
- Another asked it to implement a simple todo in the ghostty codebase and it looped on compile errors, while Qwen 3.8 finished at 40 vs 75 tok/s.
Knowledge is well below Qwen 35B, one commenter says
- In their private benchmark, general knowledge was much worse than Qwen 35B A3B.
- They still expect it to beat Qwen 9B at higher speed.
No prompt caching in Ollama, one user found
- On a Ryzen 7 8745HS with Radeon 780M via Ollama Vulkan, they saw about 45 tok/s generation and about 680 tok/s prompt processing.
- Ollama forced full prompt re-processing each time, so it was unusable in an agentic loop.

How to Apply

Run a local tool-calling smoke test with llama.cpp
- Download the GGUF checkpoint from Hugging Face and serve it with llama.cpp today.
- Try 3-5 tool schemas from your current agent and measure the call success rate.
- Check whether it gets stuck on tool calls, as commenters reported.
Attach a search tool for knowledge-heavy tasks
Check prompt caching on your serving stack
- Before using it in an agent loop, send the same system prompt twice and check that prefill time drops on the second call.
- If it does not, compare against other runtimes like vLLM or llama.cpp.
Check the license before adopting
- Read the commercial-use threshold in the model license.
- Ask legal whether internal local tooling counts.

Try It

copy & run

# Paste into Claude or ChatGPT to build a local agent eval set

Our agent uses the tools below.
{{tool list and JSON schemas}}

Create 20 test cases as JSON to verify whether a small on-device model calls tools correctly.
- Include 8 single-tool cases, 6 cases needing 2-3 tools in sequence, 3 questions that need no tool, and 3 cases where information is missing and the model should ask back
- Give each case user_message, expected_tool_calls, and pass criteria

Next step: run these cases on local LFM2.5-24B-A2B and on your current model, then compare pass rates.

Terminology

MoE (Mixture of Experts)An architecture that picks only some of many expert sub-networks to run. It is like calling in just the few people you need instead of the whole team.

Active parametersThe number of parameters actually used to process one token. Fewer means faster.

On-Policy DistillationA distillation method where a teacher grades answers the student model generated itself. It is like a tutor who marks your practice exam right after you take it.

Reward hackingWhen a model finds a shortcut to the reward instead of solving the task. It is like peeking at the answer key instead of taking the test.

GGUFA quantized model file format used by llama.cpp-style runtimes. It is a compressed model file made for running lightly on local machines.