Qwen 3.8 27B

TL;DR Highlight

Qwen3.8-27B FP8 released; commenters praise quality but flag long thinking and heavy VRAM use

Who Should Read

Backend developers choosing an open model for coding agents on a 24-32GB local GPU or a MacBook

Core Mechanics

FP8 model published on Hugging Face
- Qwen/Qwen3.8-27B-FP8 is licensed under apache-2.0.
- It is an image-text-to-text model that accepts images and text.
- Safetensors show about 27.8B parameters and roughly 30.9GB of files.
Three reasoning effort levels
- The chat template supports xhigh (default), medium, and low.
- Any other value raises an error in the template.
- Setting enable_thinking to false turns thinking off.
Tool calls use an XML format
- A function call must nest a function block inside tool_call tags.
- Required parameters must be provided.
- Tool results are wrapped in tool_response tags inside a user turn.
Past thinking can be kept in the conversation
- The preserve_thinking option is on by default, so earlier reasoning_content is kept.
- The template handles image and video input but not in system messages.

Community Reactions

Fine on a 32GB card but memory hungry, per one user
- They said it solved a private benchmark only Gemma 4 had solved, using 5x the tokens and 12m30s with MTP on.
- They said 32K of context took 2.5GB of VRAM and 128k would not fit.
Pushback that thinking is too long
- One commenter said Gemma4:26b-a3b gets similar answers with about 1/10 the thinking tokens.
- In a WordPress plugin test, xhigh overthought into messy code while low wrote solid code, per another user.
- They said medium showed the wait/actually loops seen in 3.6.
Usable for local coding on an M5 Max MacBook, per users
- One user ran the 17GB GGUF for a pelican SVG, taking 21 minutes and 22,276 reasoning tokens.
- Another got 15 tok/s in power save and 30 tok/s in performance mode on a 48GB M5 Max with OpenCode.
Quant files and templates change quality, per users
- The unsloth Q8_K_XL looped in thinking while the bartowski build did not, according to one user.
- Another said the chat templates are broken and recommended froggeric's fixed templates.

How to Apply

Run with reasoning effort low first
- Run one of your usual coding prompts at low and at xhigh and compare results and time.
- Change only the reasoning_effort value in the code below.
Put a thinking budget guard on agent use
- Pin effort to low or medium and set a max token limit to avoid loops.
- You can also put an overthinking guard proxy in front.
Compare candidates by VRAM
- Run Qwen 3.8 27B and Gemma 4 at the same context length and measure VRAM and tok/s yourself.
- If you need long context, check whether KV cache quantization hurts accuracy.
Compare quant files from two sources
- If looping appears, switch to a GGUF from another publisher.
- If tool calling breaks, apply a fixed chat template.

Try It

copy & run

# Run in Python after serving Qwen3.8-27B on an OpenAI-compatible server (vLLM, llama.cpp, etc.) (pip install openai)

from openai import OpenAI
import time

c = OpenAI(base_url="http://localhost:8000/v1", api_key="x")
PROMPT = "{{질문}}"

for effort in ["low", "medium", "xhigh"]:
    t = time.time()
    r = c.chat.completions.create(
        model="Qwen/Qwen3.8-27B-FP8",
        messages=[{"role": "user", "content": PROMPT}],
        extra_body={"chat_template_kwargs": {"reasoning_effort": effort}},
    )
    print(effort, round(time.time() - t, 1), "s", r.usage.completion_tokens, "tokens")

# Next: compare answer quality and token counts per effort in a table

Terminology

FP8Stores numbers in 8 bits to cut file size and memory, like vacuum-packing luggage.

MTPMulti-Token Prediction. It guesses several next tokens ahead to speed up generation, used as a llama.cpp option in the comments.

KV cacheMemory that stores earlier token computations. The longer the context, the more VRAM it takes.

Dense modelA model that uses all parameters every time, unlike MoE which wakes only some experts, so it needs more memory and compute.

GGUFA quantized model file format for local runtimes such as llama.cpp.