Show HN: Qwen3-4B with the AI writing tics trained out
TL;DR Highlight
Distilling Qwen3-4B with LoRA cut its AI-writing slop score from 121 to 12.5 per 1000 words
Who Should Read
A developer whose LLM-generated content is full of em dashes and 'not X, but Y' tics and who wants cleaner output
Core Mechanics
- Slop score fell from 121 to 12.5
- On 40 unseen prompts, the score per 1000 words dropped from 121 (original Qwen3-4B-Instruct) to 12.5.
- 68% of answers land in the human-writing range.
- Em dashes went from 20.5 per 1000 words to zero.
- No system prompt needed
- The earlier approach needed steering (editing internal activations) during generation.
- Here a LoRA adapter learns the style via distillation (a student imitates a teacher's best answers).
- The adapter trained on 161 answers from 288 prompts.
- Chasing the score broke the writing
- The round-two model scored 9.9 but wrote opinion pieces without opinions, invented studies, and joined sentences with ', which'.
- The ', which' habit came from the teacher prompt telling the model to join sentences with because, so, but, which.
- Task fit was 60% versus 78% for the original.
- A task-fit filter fixed it
- Rules checked for an early stance, first person in reflections, a greeting in emails, and matching length.
Community Reactions
- Same model, same tics, in one view
- A commenter said that since everyone uses the same few models, their style becomes what people recognize as 'AI'.
- They noted that any widely adopted style would eventually get tiresome too.
- Prior work exists
- A commenter pointed to sam-paech's gemma-3-27b-it-antislop as an earlier full-size deslop model.
- The slopwash.com author said they treat this as an eval problem rather than post-training.
- Still looks like AI, and new tics may appear
- One commenter said the output is a slight improvement but still obviously AI-written.
- Another asked how the author can be sure it isn't developing new tics.
- Style depends on context
- A commenter argued the structured 'slop' style suits corporate writing better than the cleaned version.
- They saw the cleaned version as useful for things like reviews.
- Q4 build loops on long output
How to Apply
- Run a slop regex on your outputs
- Today, collect 50 recent LLM outputs and count em dashes, 'not X, but Y', 'delve', and 'here's the thing' per 1000 words.
- Keep this as a baseline and compare after each prompt change.
- Compare the deslop model via Ollama
- Run ollama run hf.co/erikkannike/qwen3-4b-deslop-v4-GGUF:Q4_K_M on 10 of your prompts.
- Put the output next to your current model's and check score and task fit.
- Add task-fit checks to your distillation filter
- When building fine-tuning data, check stance, person, greeting, and length alongside style score.
- Add a rule that counts invented studies and statistics.
- Audit teacher prompts for sentence-joining advice
- Remove instructions like 'join with because, so, which' since imitation amplifies them.
- Put the task ahead of style instructions.
Try It
# Paste into Claude or ChatGPT (simplified version of the paper's method: post-hoc AI-tic check)
Score the {{document}} below against these habits and rewrite it.
Habits: em dash, 'not X, but Y', lists of three, rhetorical questions answered on the spot, signposts like 'here's the thing', words like 'delve' and 'seamless', short fragments for emphasis.
1) Count each habit and convert to a score per 1000 words.
2) Rewrite keeping the same content and stance without those habits. Do not invent studies, numbers, or names.
3) Check it fits the task ({{task: opinion piece/email/reflection}}): stance, person, greeting, length.
{{document}}
Next, try: score the rewrite with the same criteria to see if it picked up new tics.Terminology
Related Papers
Dust: Pretraining Transformers Without Backpropagation
Adding per-token noise to activations lets transformers pretrain without backpropagation, rivaling backprop.
Show HN: TurboGPT: train 22KiB transformer in 13s
A CUDA C++ project trains a tiny byte-level GPT in under a minute.
I had Gemini train its own replacement for $9
Fine-tuning GLiNER on $9 of Gemini labels reached 0.83 F1 locally.
Learning to solve hard problems in RL for LLMs by never giving up
Cutting compute on easy problems and retrying hard ones in RL raised accuracy on the hardest problems
Training a 3.8B LLM to 0.384 CORE for $998
A 3.8B LLM trained on 8 B200s in 43 hours for $998 scored 0.384 on CORE.
I were 17, I'd learn how to build LLMs from scratch
PG says at 17 he'd skip startups and learn to build LLMs from scratch.
Show HN: Distilling DeepSeek into GPT-OSS doesn't transfer censorship. Try it