MicroLLM Lab – Try 7 tiny LLM's in the browser | AI Paper Digest

TL;DR Highlight

Run 25M–360M LLMs in-browser via WebGPU and benchmark their speed and accuracy

Who Should Read

Frontend or full-stack developers who want to try on-device LLMs for lightweight tasks like classification and routing without server costs

Core Mechanics

Runs in the browser with no server
- It uses WebGPU to run GPU compute directly in the browser.
- Models are cached in the browser's IndexedDB.
- The page says no account is needed and prompts never leave the device.
Targets 25M–360M parameter models
- The page positions them as an edge layer, not general-knowledge engines.
- Suggested uses are query classification, spam filtering and intent extraction.
- They can also decide whether to call an expensive cloud LLM.
Q4 quantization cuts memory by 75%
- Weights shrink from 16-bit to 4-bit.
- Models over 100M parameters fit in about 50–84 MB of browser memory.
- The page claims near-lossless generation quality.
You can measure speed and accuracy yourself
- The Benchmarks tab measures tok/s and pass rate using regex and exact-token checks.

Community Reactions

Small models give poor answers, per users
- Several commenters got wrong answers and loops on basics like 2+2 and whether a cat is an animal.
- One user saw SmolLM2 360M give absurd California population figures, though it was very fast without a decent GPU.
Questions about usefulness
- One commenter asked what they are good for when they don't consistently pass the site's own benchmarks.
- Another asked whether the point is just to showcase WebGPU.
Firefox and UI problems reported
- One user got a GPUShaderStage is not defined error on Firefox 156.0.1 (Arch Linux), while Chromium worked.
- Another said the text is too small and there is too much filler before the actual interface.
Similar projects shared
- A TinyStories-trained small model demo was shared (tiny.tobelabs.com).
- HermitUI, a single-file WebGPU chat app that handles larger models, was shared.
- A Web Models API proposal for a browser-standard on-device model API was mentioned.

How to Apply

Load two models in Chromium and run the benchmark
- Open the site in Chrome today and Load SmolLM2 plus the smallest model.
- Click Run suite on every loaded model in the Benchmarks tab and compare tok/s and pass rate.
- Skip Firefox, since a load failure was reported.
Turn your classification task into a Custom eval
- Pick about 10 intent or spam examples and write them as JS checks.
- Measure the pass rate with exact match or regex to judge with numbers whether a small model suffices.
Try it as a routing layer before cloud calls
- Give only short classification tasks to the small model and send the rest to your existing cloud LLM.
- Drop tasks where the small model's pass rate is low.
Check browser support before shipping
- Add a WebGPU feature check and a fallback to a server API when it's missing.

Try It

copy & run

// Paste into browser dev console or a JS file: check WebGPU support (standard API, not code from the site)

async function checkWebGPU() {
  if (!navigator.gpu) return 'WebGPU unsupported: fall back to server API';
  const adapter = await navigator.gpu.requestAdapter();
  return adapter ? 'WebGPU available' : 'no adapter: fall back';
}
checkWebGPU().then(console.log);

// Next: load a model at https://stateofutopia.com/experiments/microllmlab/ and put {{your classification examples}} into Custom eval to measure the pass rate.

Terminology

SLMA small language model. Instead of an encyclopedia, it's like a neighborhood clerk that handles specific tasks quickly.

Q4 quantizationCompressing weights from 16-bit to 4-bit. Like saving a high-res photo at a smaller file size.

WebGPUA standard API that lets browsers use the GPU directly for compute.

TTFTTime to first token: how long until the first token appears after a request.

IndexedDBA local storage inside the browser. Model files are cached here so they aren't downloaded again.

MicroLLM Lab – Try 7 tiny LLM's in the browser