Show HN: Llmspeed – feel how fast a local LLM runs before you buy the hardware | AI Paper Digest

Pick hardware and a model to preview local LLM speed before buying

A backend developer weighing GPU or Mac purchases to run an in-house local LLM

Pick hardware and model to feel the speed
- The site lets you browse by hardware and by model.
- It aims to show how fast a local LLM will feel before you buy.
Wide hardware range from RTX 3090 to Mac Studio
- The list includes NVIDIA RTX 3090/5090, AMD Strix Halo, Mac Studio M5 Ultra, DGX Spark and cloud H100/H200.
- A CPU-only setup (Ryzen 9 9950X, 96GB DDR5) is also listed.
Many models to choose from
- Options include Qwen 3.6/3.8 27B, Gemma 4 26B A4B and 31B, GPT-OSS 20B and 120B.
- DeepSeek V4 Flash, Llama 3.1 8B and GLM-5.3 are also there.
Numbers mix measured and estimated values
- Measured values come from community benchmarks (r/LocalLLaMA, localscore.ai and more).
- The rest are derived from similar setups or projected from specs.
Short prompt, single device assumption

Real agentic work is slower than benchmarks, one user says
- Running vLLM at work, they found real usage is 60k-140k prompts with a few k of output.
- Decode throughput there is half to a third of the benchmark, and cache-aware routing and KV cache management matter.
Many say the site's numbers don't match reality
- One user gets 60-70 tok/s on an RTX 5090 with Qwen3.6 27B 4-bit versus the site's 46 tok/s.
- Another gets 80 t/s on a 4090 with Q3.8 27B Q4_K_XL while the sim shows mid 20s.
- An M2 Ultra 128GB reportedly runs almost twice as fast as the figure for M3 Ultra 512GB.
Missing variables: speculative decoding and parallel prefill
- One user says ngram speculation lifts code generation to 100+ tok/s.
- Another says prefill parallelism matters more than output speed for agentic coding.
Models that don't fit in VRAM can still run, a commenter says
- The site says FP16 27B won't run on a 9070 XT, but llama.cpp offloads some layers to the CPU and it runs.
- Commenters also asked for multi-GPU, PCIe generation and NVLink options.

Compare candidate hardware and models on the site first
- Open llmspeed.dev now and pick 2-3 candidate GPUs plus your model and quantization.
- Write down the speeds and use them only as a rough ranking.
Measure with real prompt lengths on a rented GPU
- Rent each candidate GPU for a few minutes and measure TTFT and tok/s with your actual prompts.
- For agents, also test long prompts of 60k tokens or more.
Add prefill and long context to purchase criteria
- Write average input tokens, output tokens and concurrent requests in your requirements doc.
- Compare prefill time and KV cache memory, not just output tok/s.

tok/sTokens generated per second. Like typing speed: higher means answers appear faster.

prefillThe stage where the model first reads your prompt. Think of it as time spent reading a long document.

TTFTTime to first token. The wait between asking and the answer starting.

KV cacheMemory that stores earlier token computations. It grows with context length.

speculative decodingA small model drafts text and the large model verifies it, speeding up generation.

Show HN: Llmspeed – feel how fast a local LLM runs before you buy the hardware