Edge0-145B: Frontier AI Capabilities on Your Own Computer
TL;DR Highlight
A 145B MoE model reportedly runs on a laptop at 29GB peak memory and 80+ tokens/s.
Who Should Read
A backend developer weighing local coding agents because of cloud LLM API costs and data-leaving constraints.
Core Mechanics
- 145B parameters in 29GB peak memory
- The post says it runs on a compatible laptop such as a MacBook Pro without a server.
- Reported decode is 80+ tokens/s and prefill is 1,100+ tokens/s.
- Experts share matrices to save memory
- In this MoE (only some experts activate per input), each group of 64 experts shares two large matrices.
- Only about 10B of the 145B parameters are active per inference step.
- Memory is split across GPU, RAM, and SSD
- Hot experts stay in GPU memory, and a roughly 40GB expert library sits in system memory.
- About 48GB of vocabulary data is read on demand from the SSD.
- Strong coding and agent benchmarks (self-reported)
- DeepSWE 1.1: 64.6 vs Qwen3.8-Flash-Next 58.7 and DeepSeek-V4-Flash-0731 54.4.
- Terminal-Bench 2.1: 86.52 vs Claude-Opus-4.6 (Max) 78.2.
- GPQA-Diamond: 90.4, close to Claude-Opus-4.6 (Max) at 91.3.
- The pitch is quality per GB of memory
- Mean over six benchmarks is 72.8 vs 71.7 for Qwen3.8-Flash-Next.
- Qwen3.8-Flash-Next is estimated at about 70GB under 4-bit deployment, not measured.
- Local agent demos are included
- Demos cover Blender 3D scenes, games, real drone control code, and KiCad PCB design, all run locally.
- Weights, an inference framework, and one-click deployment tools are released for macOS, Windows, and Linux.
Community Reactions
- Narrow compatibility due to NVFP4 shipping
- One commenter said NVFP4 may make it Nvidia Blackwell only.
- Another found it odd that there are no GGUF files and the HF link leads to a directory of split files.
- Caution about benchmark claims
- A commenter wants third-party verification given the model size.
- Another said Terminal-Bench scores have matched real-world bash-heavy coding agent performance in their experience.
- Doubts about Blender output quality
- A commenter said agent-made designs are often messy and cleanup can cancel out the time saved.
- Open weights, not open source
- A commenter pointed out this is open weights, not open source.
- Another asked why the company site has so little information.
How to Apply
- Check your hardware memory first
- Today, read the model card and hardware guide for supported chips and memory needs.
- Check whether your laptop is 48GB class with an Apple M-series or RTX 4090/5090.
- Sort coding-agent tasks into local candidates
- List tasks like source code or design files that are hard to send outside.
- Run the same tasks on your current cloud model and on Edge0 and compare results.
- Re-verify benchmarks on your own tasks
- The numbers are Edge0's internal evaluations, so run 10–20 issues from your own repo.
- Record success rate, time, and number of fixes.
- Confirm the weight format before adopting
- Check whether weights are NVFP4 only or also available in formats like GGUF.
- Verify your inference engine is supported by the released framework.
Try It
# Paste into Claude or ChatGPT: local model feasibility check (simple checklist based on the article)
I am a {{role}}, and my laptop is {{chip/memory, e.g. M-series 48GB}}.
My current cloud LLM tasks are {{task list}}.
Build a table of tasks that could move to a local on-device model (e.g. 145B MoE, 29GB peak memory, 80+ tokens/s decode as reported), using these criteria:
1) Is there data that is hard to send externally?
2) Do many iterations make token cost high?
3) Is offline use needed?
4) Is lower quality than a cloud model unacceptable?
Next step: pick one candidate task, run it on both the local model and my current model, and compare whether each succeeds.Terminology
Related Papers
Gemini 4 Neon
Gemini 4 Neon cuts cost per task 32% versus Carbon, at $3 input and $15 output per million tokens
AI is killing All About Berlin
After Google AI Overviews, a Berlin guide site lost 70-75% of its traffic.
Not Everything Needs an LLM
PostHog used small decision models to skip 98.7% of expensive LLM calls
GPT‑6.2 Astra and Sol
GPT-6.2 Astra and Sol cut costs sharply, but over-engineering and early-stop reports are mixed
Show HN: TrnRT – An open-source LLM inference engine for AWS Trainium
One trn2.48xlarge serves GLM-5.3-Flash through an OpenAI-compatible API using NKI megakernels.
Know When to Zoom: How Small Is Too Small for Vision-Language Models?
Splitting large photos into tiles raised Claude and Gemini accuracy by 11.1–14.5 points
Understanding Positional Embeddings in Modern LLMs
A walkthrough of how LLMs encode token position, from GPT-2 to RoPE
Show HN: DeepSeek V3 in ~200 lines of pure Jax
DeepSeek V3's core design in 190 lines of JAX, trained on a MacBook Air in 2.6 hours