I had Gemini train its own replacement for $9
TL;DR Highlight
Fine-tuning GLiNER on $9 of Gemini labels reached 0.83 F1 locally.
Who Should Read
A backend developer whose per-call LLM entity extraction bill keeps growing with traffic
Core Mechanics
- Gemini's labels trained a small model
- Gemini 3.1 Pro labeled 4,290 comments once for $9.
- GLiNER large v2.5 (459M) was fine-tuned on them and runs locally.
- F1 against Gemini's answers went from about 0.65 zero-shot to 0.83.
- Code computes offsets, not the LLM
- The prompt asks only for the exact substring and a label.
- TypeScript finds the character offsets and drops and logs strings not found in the comment.
- No-product comments went in as negatives
- About 30% of the training set has false-positive triggers like gyuto, carbon, handle, patina but no product.
- Those were labeled empty.
- Going from 51 to 510 adversarial negatives dropped F1 to 0.799, so they went back to 51.
- Five of ten runs failed on words_mask
- words_mask is a word index, not an attention mask (0 for special tokens, then 1, 2, 3 for each word's first sub-token).
- Filled with ones, training gave no error and loss sat flat at 70-130 with eval F1 near zero.
Community Reactions
- Opinion that a dictionary might suffice
- One commenter said the model may just map knife-world words to classes, so a simple dictionary could fit.
- They suggested splitting train/validation by entity so unseen brands test generalization.
- Concern about small data and new brands
- A commenter noted the post-training dataset for GLiNER is pretty small.
- Another asked what happens when a new knife brand appears.
- Doubt about the $2.50 T4 cost
- A commenter calculated that 24 minutes at $2.50 implies $6.25/hour, which seems expensive for a T4.
- The article's $2.50 covers GPU time across all ten runs.
- Worry that bots could skew the data
- One commenter said advertisers use bots to praise products on Reddit.
- They argued the method might surface the biggest ad spender rather than the best knife.
- Support for swapping in specialized tools
How to Apply
- Pull label candidates from your LLM call logs today
- In 30 minutes, pick one repetitive NER or tagging call and export a few thousand inputs and outputs.
- Convert outputs to string+label JSON, compute offsets in code, and drop entries not in the source text.
- Lock a validation set first and hand-check some
- Hold out about 200 examples before training and never touch them again.
- Hand-verify part of it so you know you are not just copying LLM mistakes.
- Assert on every hand-built tensor
- For inputs like words_mask, read the library's training loop and assert value ranges and shapes.
- When loss is flat, print the input tensors before blaming the data.
- Tune per-class thresholds and negatives
- Sweep thresholds per class on the validation set.
- Include about 30% no-product comments with false-positive trigger words, and check scores before adding more.
Try It
# Paste into Claude/ChatGPT — simplified version of the article's method: LLM labeling prompt (return strings only, code computes offsets)
You are an NER labeler. Extract only items matching {{label types, e.g. brand, model, material}} from the comment below.
Rules:
- Never count character offsets. Return only the exact substring (text) as it appears in the comment, plus a label.
- Exclude generic non-product words ({{false-positive triggers, e.g. carbon, handle}}).
- If nothing matches, return {"entities": []}.
- Output JSON only: {"entities":[{"text":"...","label":"..."}]}
Comment: {{comment}}
Next, try this: ask for code that searches each returned text in the comment to compute the position and drops it if not found.Terminology
Related Papers
Show HN: Qwen3-4B with the AI writing tics trained out
Distilling Qwen3-4B with LoRA cut its AI-writing slop score from 121 to 12.5 per 1000 words
Dust: Pretraining Transformers Without Backpropagation
Adding per-token noise to activations lets transformers pretrain without backpropagation, rivaling backprop.
Show HN: TurboGPT: train 22KiB transformer in 13s
A CUDA C++ project trains a tiny byte-level GPT in under a minute.
Learning to solve hard problems in RL for LLMs by never giving up
Cutting compute on easy problems and retrying hard ones in RL raised accuracy on the hardest problems
Training a 3.8B LLM to 0.384 CORE for $998
A 3.8B LLM trained on 8 B200s in 43 hours for $998 scored 0.384 on CORE.
I were 17, I'd learn how to build LLMs from scratch
PG says at 17 he'd skip startups and learn to build LLMs from scratch.
Show HN: Distilling DeepSeek into GPT-OSS doesn't transfer censorship. Try it