AI At Home Part 1: A Box Of Scraps

Who Should Read

A developer who wants to cut API dependence by building a local LLM server at home or in the office

Core Mechanics

• Four V620s give 128GB of VRAM
  • AMD overproduced the V620 for cloud gaming, and it has 32GB of VRAM and no video outputs.
  • Its poor software support keeps used prices low on eBay.
  • The author negotiated four cards at $350 each.
• The rest was old hardware
  • A 2017 X299 board (Supermicro C9X299-PGF), an i9-10900X and a 1600W PSU were used.
  • The work happens in GPU VRAM, so CPU, RAM and SSD requirements are low, per the author.
  • The case is a SilverStone RM4A ($262) with eight PCI slots.
• Fanless server cards need DIY cooling
  • Four cards side by side are about 160mm wide, so the author modeled and printed a shroud for 80mm fans.
  • The 10,000RPM Arctic S8038-10K fans were audible through the house walls.
• Booting needed BIOS changes
  • The board only started with four cards after enabling both Resizable BAR and MMIO High Size.
  • The two options sit in separate advanced menus.

Community Reactions

• Advice on cooling server cards
  • One commenter noted blower fans are not quieter but have higher static pressure for pushing air.
  • Another pointed out that cases fitting four dual-slot cards are rare.
• Experience with a similar 4x MI50 build
  • A single 14cm Noctua fan with a 3D-printed duct kept cards at 62C max at low speed, they said.
  • A bigger fan was quieter, and replacing the thermal pad with paste dropped temperatures a few degrees.
• Experience replacing cloud AI with DGX Sparks
  • One commenter runs DS4 Flash and OCR models on two Sparks linked over QSFP.
  • After long Claude use, they said they no longer need it.
• Pushback on printing material
  • One commenter said PLA softens around 55C and would fail here.
  • They suggested PETG-CF if ASA is not printable.

How to Apply

• Benchmark your candidate GPU with llama.cpp today
  • Build llama.cpp on your GPU or a candidate card and run a model that fits on one card like Gemma4.
  • Record speed and VRAM use and compare against your API cost.
• Check BIOS options before a multi-GPU build
  • If you plan many GPUs on an older board, confirm it has Resizable BAR and MMIO High Size.
  • Without them it may not boot.
• Plan cooling before buying server cards
  • Measure card width and case slots (8 or more) before buying fanless cards.
  • Pick fans with temperature-based PWM control in mind.
• Have a local model write ops scripts
  • Hand small utilities like fan control to a local model and check quality.
  • Expect to guide it continuously for usable results.

Try It

# Paste into Claude or ChatGPT as-is (a lite version of the article's approach of having the model write its own fan script)

I run a Linux server with {{number of GPUs}} GPUs. Write a Python script that reads GPU temperature, computes a fan speed (0-100%), and sends it over serial to {{device path}}.

Requirements:
- Read temperatures with {{temperature command}}.
- Base the speed on the hottest GPU.
- Show the temperature-to-speed table first.
- If a sensor read fails, fail safe to a high speed.

Explain the design first and write the code after I approve.

Next step: ask the model "find the risky parts of this code" on the finished script.

Terminology

Related Papers

• Mercury 3: Diffusion at scale

  Scaling diffusion LLM 40x gives Mercury 3: 4–8x faster than speed-tier models, similar quality

• Timestral: Mistral's time-series foundation models

  A 209M time-series model ranks first on three benchmarks and runs on CPU.

• DeepSeek V4.1 Flash 1017 (Max)

  DeepSeek V4.1 Flash 1017 Max scores 45 on the Intelligence Index at 211 tok/s and $0.30 input.

• If you don't run your own LLM proxy, how do you know whats going on?

  Running your own LLM proxy shows which keys, models, and cache misses drive your usage.

• DeepSeek: DeepSeek v4.1 Flash 1017

  DeepSeek V4.1 Flash 1017: $0.05/M input, 1M context, built for agents

• DeepSeek-v4.1-Flash-1017

  DeepSeek V4.1 Flash minor update lifts coding and agent benchmarks while cutting run time and output length

• Show HN: Llmspeed – feel how fast a local LLM runs before you buy the hardware

Related Resources

• https://jdagostino.github.io/ai-pt1-box-o-scraps/index.html