Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

TL;DR Highlight

Meta's 30B open-weight model, quantized to 4-bit with DFlash, runs local agents on a 24GB GPU

Who Should Read

A backend developer who wants to run function-calling and coding agents locally without a cloud API

Core Mechanics

A 30B model that fits on one consumer GPU
- Meta released the weights under Apache 2.0.
- It is optimized for local agents, function calling, local coding, and LLM-as-a-judge evaluation.
Distilled from a larger model
- Pre-training used logit distillation (learning the teacher's probability distribution) on Muse Spark's outputs.
- Post-training mixed SFT, on-policy distillation, and reinforcement learning.
Broad agent skills were trained in
- It calls tools with precise schemas and is trained to diagnose and retry when a call fails instead of halting.
- It supports image and text input, 100+ languages, and adjustable reasoning effort.
4-bit quantization shrinks 55GB to under 20GB
- At full precision a 30B model needs over 55GB, and quantizing to about 4-bit brings the language model under 20GB.
- The KV cache, perception encoder, and drafter also fit in a 24GB or 32GB envelope.
- Meta says agentic-task degradation is minimal to none.

Community Reactions

Unsloth GGUFs are already up
- One commenter says quantized builds change often in the weeks after release, so recheck in a few days.
- They say people with private test cases report better results than Qwen3.6 27B.
- Early reports say even 2-bit is usable, though the commenter advises caution.
Experience on a 32GB Mac mini
- One commenter ran it with Ollama and the pi coding harness and got good but slow results.
- They advise increasing the context size.
Experience on a 7900XT with 20GB
- Unsloth UD-Q4_K_XL barely fit at about 19GB of VRAM.
- Generation was about 36 tok/s, rising to about 60 tok/s with DFlash, per that commenter.
Pushback and limits
- One commenter running the full model on 120GB RAM says it looped while fixing broken code.
- Others note the comparison models are a generation back and expect Qwen3.8 to beat it.
- Some say the 24GB GPU and 32-64GB memory requirements are expensive.

How to Apply

Run it locally with Ollama or LM Studio
- Today, download an Unsloth GGUF or the LM Studio build.
- Raise the context size and run one or two of your existing agent prompts.
Compare on your function-calling test set
- Use the same tool schemas and questions you use for Qwen or Gemma and measure success rate.
- Add cases where a tool fails to check the retry behavior.
Move LLM-as-a-judge evaluation local
- If judge calls are costly, compare local verdicts on 100 samples against your current judge.
- Record the agreement rate to decide whether to switch.
Measure speed with the DFlash drafter
- After llama.cpp support lands, measure tok/s with and without the drafter.
- If VRAM is tight, turn off the multimodal projector for text-only tasks.

Try It

copy & run

# Paste into Claude/ChatGPT: design agent eval cases before adopting a local model (simple checklist inspired by Glimmer)

I plan to run a tool-calling agent on a local 30B model.
My tools: {{tool JSON schemas}}
My main tasks: {{task description}}

Create 5 test cases for each of these 4 categories.
1. Calling tools with exact schemas
2. Keeping a plan across multiple steps
3. Diagnosing and retrying when a tool errors or returns something unexpected
4. Quality difference between low and high reasoning effort

For each case, write the input, expected tool call, and pass criteria in a table.

Next step: run the same cases on your current model and the local model and compare pass rates.

Terminology

DistillationA large model acts as teacher and trains a smaller model on its answers and probability distributions. It is like studying from an answer key.

QuantizationLowering the precision of weight values to cut size. It is like compressing a high-res photo into a JPEG.

Speculative decodingA small model drafts several tokens and the big model checks them all at once. It is like an intern drafting and a senior approving.

KV cacheThe model's working memory of what it already read. It grows as the conversation gets longer.

ScaffoldThe agent framework around a model that manages tool calls and loops. The post cites OpenClaw as a compatible one.