Shapelearn Qwen 3.8 27B (13.1 GB VRAM)
TL;DR Highlight
Quantized Qwen 3.8 27B fits in 13.1GB and scores 99.63% of BF16
Who Should Read
Developers running 27B-class models locally on 16-24GB GPUs who must choose which quantized GGUF to use
Core Mechanics
- GPU-5 is the default pick
- GPU-5 (IQ4_XS, 3.84bpw, 13.1GB) reaches 99.63% of BF16's aggregate benchmark score.
- It runs at 90.4 tok/s on RTX Pro 6000 and 93.7 tok/s on RTX 5090.
- Use GPU-4 when memory is tight
- GPU-4 (IQ3_S) hits 98.72% of BF16 at 11.0 GB.
- It is faster than GPU-5 and leaves more room for context.
- All five models sit on the frontier
- Frontier means no other plotted model is both faster and more accurate.
- All five were on the frontier in all six GPU comparisons.
- KLD ranking disagreed with task scores
- KLD measures how far a quantized model's token distribution drifts from BF16.
- Unsloth UD-IQ3_S had about 20% lower KLD than Lite-1 (0.028759 vs 0.035875) but scored 95.55% vs Lite-1's 97.33%.
- Speculative decoding speeds things up
Community Reactions
- DFlash2 was slow on an AMD GPU
- One user on Vulkan with an AMD GPU said it was not faster than the Unsloth model.
- On a 7900XTX they got about 30 t/s with the DFlash2 draft and about 60 t/s with regular MTP.
- DFlash2 gave a wrong answer
- The same user said the DFlash2 draft answered a test question incorrectly.
- With MTP the answer was correct.
- MTP was usable on Strix Halo
- One user on Strix Halo with GPU-5 and MTP got 600 prefill and 30 tok/s generation.
- DFlash2 was very slow for them, under 10 tok/s.
- Charts helped pick a model
- One commenter said the graphs helped them find a faster quantization that fits their VRAM.
How to Apply
- Run GPU-5 with MTP first
- If you have around 24GB VRAM, run GPU-5 with MTP using the quick-start command today.
- Keep --mmproj-auto if you need image input.
- Switch to GPU-4 if VRAM is short
- If the 13.1GB model plus your context does not fit, change the tag to IQ3_S-3.23bpw.
- Compare speed and answer quality against GPU-5 on the same prompts.
- Benchmark DFlash2 on your own hardware
- If you are text-only and have spare memory, enable DFlash2 and compare tok/s with MTP.
- Commenters reported slowdowns on AMD (Vulkan), so measure in your own environment.
Try It
# Run in shell (needs llama.cpp b10658 or newer for DFlash2)
llama-server \
-hf byteshape/Qwen3.8-27B-GGUF:Qwen3.8-27B-IQ4_XS-3.84bpw \
--mmproj-auto \
--spec-type draft-mtp --spec-draft-n-max 3
# For max text-only speed (DFlash2)
llama-server \
-hf byteshape/Qwen3.8-27B-GGUF:Qwen3.8-27B-IQ4_XS-3.84bpw \
-hfd incoai/Qwen3.8-27B-DFlash2-GGUF:Q4_K_M \
--spec-type draft-dflash --spec-draft-n-max 7 \
--no-mmproj
# Next: compare tok/s and answers of both commands on the same promptTerminology
Related Papers
Mercury 3: Diffusion at scale
Scaling diffusion LLM 40x gives Mercury 3: 4–8x faster than speed-tier models, similar quality
Timestral: Mistral's time-series foundation models
A 209M time-series model ranks first on three benchmarks and runs on CPU.
DeepSeek V4.1 Flash 1017 (Max)
DeepSeek V4.1 Flash 1017 Max scores 45 on the Intelligence Index at 211 tok/s and $0.30 input.
If you don't run your own LLM proxy, how do you know whats going on?
Running your own LLM proxy shows which keys, models, and cache misses drive your usage.
DeepSeek: DeepSeek v4.1 Flash 1017
DeepSeek V4.1 Flash 1017: $0.05/M input, 1M context, built for agents
DeepSeek-v4.1-Flash-1017
DeepSeek V4.1 Flash minor update lifts coding and agent benchmarks while cutting run time and output length
Show HN: Llmspeed – feel how fast a local LLM runs before you buy the hardware