How to setup a local coding agent on macOS

Who Should Read

A developer who wants a local coding agent on a 64GB-class Mac that keeps working when the internet goes down

Core Mechanics

• MTP gave 24% faster generation
  • Gemma 4 26B-A4B Q4 alone runs 58.2 tok/s; with a Q8 MTP draft model it hits 72.2 tok/s (M1 Max 64GB, llama.cpp Metal).
  • Prompt processing stayed about the same: 298.0 vs 295.6 tok/s.
• Best draft token count was 3
  • Sweeping --spec-draft-n-max from 1 to 6 gave 72.2 tok/s at 3 and 72.0 at 2.
  • 5 dropped to 63.7 and 6 to 61.2 tok/s.
• llama.cpp beat MLX
  • For Gemma 4, llama.cpp+MTP got 72.2, llama.cpp 58.2, and the best MLX-LM 45.8 tok/s.
  • The author expected Mac-native MLX to be fastest.
• Image input needs mmproj and a Pi setting
  • Pass --mmproj mmproj-BF16.gguf to llama-server to enable multimodal.
  • Set input to ["text","image"] in Pi's models.json so images are sent.
  • Loading the projector did not slow generation: 71.4 vs 72.2 tok/s.
• Qwen3.6 35B-A3B is better at coding but slower
  • The author says benchmarks he found rate Qwen higher as a coding agent.
  • Speed is 55 tok/s for Qwen3.6 vs 72 tok/s for Gemma 4.
• Final stack is one server plus Pi
  • llama-server exposes an OpenAI-compatible /v1 endpoint at 127.0.0.1:8080.
  • Pi connects to it via a local provider in ~/.pi/agent/models.json.

Community Reactions

• Concern that the benchmark is too short
  • One commenter says 128 tokens is not enough.
  • In their experience early output has higher draft acceptance, so short tests can inflate the speedup.
  • They point to a llama.cpp tool that sweeps arguments automatically.
• MTP isn't always a win
  • One commenter saw little speed gain from MTP on the QAT model on an M1 Max.
  • They also saw the Gemma 4 MTP head break markup in Opencode and miss stop tokens, so they turned it off there.
• Easier setup options
  • Commenters note llama.cpp's -hf flag and LLAMA_CACHE can download models without huggingface-cli.
  • Others say oMLX or LM Studio got them running in minutes.
• Other model experiences
  • One commenter got about 24 tok/s generation and 200 tok/s prefill with DeepSeek V4 Flash (ds4) on an M4 Max 128GB and found it useful for simple orchestration.
  • Another complains that local AI posts talk only about tokens per second, not answer quality.

How to Apply

• Build llama.cpp and measure a no-MTP baseline
  • Run brew install cmake git tmux, then build llama.cpp with -DGGML_METAL=ON.
  • Run the same prompt with llama-cli and note the tok/s baseline.
• Add the MTP draft model and sweep the draft value
  • Add --model-draft and --spec-type draft-mtp.
  • Try --spec-draft-n-max from 1 to 6 and pick the fastest on your machine.
• Serve with llama-server and connect your tools
  • Start llama-server on 127.0.0.1:8080 and check /v1/models with curl.
  • Put that address in baseUrl of an OpenAI-compatible client or Pi's models.json.
• Add mmproj and the input setting for screenshots
  • Add --mmproj mmproj-BF16.gguf to the server.
  • Add "image" to input in models.json and verify images shows yes with pi --offline --list-models.

Try It

# Paste into macOS terminal: build llama.cpp and run the MTP server (based on the article's commands)

brew install cmake git tmux python@3.11
git clone https://github.com/ggml-org/llama.cpp repos/llama.cpp
cd repos/llama.cpp && cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_METAL=ON -DGGML_ACCELERATE=ON && cmake --build build --config Release -j && cd ../..
pip install -U huggingface_hub hf_xet
huggingface-cli download unsloth/gemma-4-26B-A4B-it-GGUF gemma-4-26B-A4B-it-UD-Q4_K_XL.gguf mmproj-BF16.gguf MTP/gemma-4-26B-A4B-it-Q8_0-MTP.gguf --local-dir models/unsloth-gemma-4-26B-A4B-it-GGUF
D=models/unsloth-gemma-4-26B-A4B-it-GGUF
repos/llama.cpp/build/bin/llama-server -m $D/gemma-4-26B-A4B-it-UD-Q4_K_XL.gguf --model-draft $D/MTP/gemma-4-26B-A4B-it-Q8_0-MTP.gguf --mmproj $D/mmproj-BF16.gguf --spec-type draft-mtp --spec-draft-n-max 3 -ngl 999 -fa on -c 65536 --parallel 1 --host 127.0.0.1 --port 8080

Terminology

Related Papers

• DeepInfer: High-performance inference framework for DeepSeek models

  DeepSeek open-sourced DeepInfer, its production inference engine, under Apache-2.0.

• Mercury 3: Diffusion at scale

  Scaling diffusion LLM 40x gives Mercury 3: 4–8x faster than speed-tier models, similar quality

• DeepSeek banned from U.S. federal civilian agencies under new DHS order

  DHS order bans DeepSeek from U.S. federal civilian agency IT systems

• Timestral: Mistral's time-series foundation models

  A 209M time-series model ranks first on three benchmarks and runs on CPU.

• DeepSeek V4.1 Flash 1017 (Max)

  DeepSeek V4.1 Flash 1017 Max scores 45 on the Intelligence Index at 211 tok/s and $0.30 input.

• If you don't run your own LLM proxy, how do you know whats going on?

  Running your own LLM proxy shows which keys, models, and cache misses drive your usage.

• DeepSeek: DeepSeek v4.1 Flash 1017

  DeepSeek V4.1 Flash 1017: $0.05/M input, 1M context, built for agents

• DeepSeek-v4.1-Flash-1017

  DeepSeek V4.1 Flash minor update lifts coding and agent benchmarks while cutting run time and output length

Related Resources

• https://ikyle.me/blog/2026/how-to-setup-a-local-coding-agent-on-macos
• https://github.com/ggml-org/llama.cpp
• https://github.com/earendil-works/pi
• https://unsloth.ai/docs/models/mtp
• https://unsloth.ai/docs/models/gemma-4
• https://unsloth.ai/docs/models/qwen3.6
• https://github.com/jundot/omlx
• https://github.com/antirez/ds4
• https://blog.alexewerlof.com/p/local-llms-for-agentic-coding