Llama.cpp on a jailbroken PS5: run any GGUF model natively on the console

TL;DR Highlight

A jailbroken PS5's GPU runs llama.cpp as a home LLM server

Who Should Read

Developers using OpenAI or Anthropic API clients who want a cloud-free local LLM server at home

Core Mechanics

A PS5 becomes a local LLM server
- It runs GGUF models on the console GPU via llama.cpp's Vulkan backend.
- 11.4GB of shared memory holds the model and its context.
Existing API clients plug right in
- It supports OpenAI's /v1/chat/completions and Anthropic's /v1/messages.
- The README says Claude Code works against it.
Measured speeds on a PS5 Slim
- Qwen 3.5 0.8B Q4_K_M (0.5GB) hits 207 tokens/s.
- Qwen 3.5 9B Q4_K_M (5.3GB) does 55 tokens/s, and Qwen 3.8 27B UD-Q2_K_XL (9.2GB) about 20 tokens/s at 64k context.
Context is sized automatically to fit memory
- A planner reads each GGUF and picks the longest context and best cache that fit free memory.
- The app also downloads from Hugging Face itself.
Decision models skip text generation
- The /v1/systemone API returns choice and yes/no answers with calibrated probabilities.

Community Reactions

2-bit 27B is still useful, one commenter argued
- It suits always-on jobs like monitoring server error rates or reviewing security footage.
- They calculated about 2M output tokens a day for 70 cents of power.
Too little VRAM to be practical, said a critic
- It is a cool feat, but the README oversells it as usable.
A newcomer asked how it compares to Codex or Claude
- Another guessed it would be slower and less capable.
Other hardware may work too, per commenters
- BC250 boards, a cut-down PS5 APU, go for $150-$200 used.
- One commenter also said the project should be hosted on its own domain, not GitHub.

How to Apply

Test the same API shape locally on your PC
- Today, point your OpenAI-compatible client's base URL at a local llama.cpp server.
- Then switch to PS5LM's http://<console>:8081 and the same code works.
Move always-on low-difficulty jobs to a local model
Try a decision model for classification
- For fixed-output tasks like intent classification or yes/no checks, call /v1/systemone.
- Measure whether latency is around 0.3s and compare it to an LLM call.

Try It

copy & run

# Paste into a terminal (when a console running PS5LM is on the same network)

curl http://{{console_ip}}:8081/v1/chat/completions -H 'Content-Type: application/json' -d '{"model":"any","messages":[{"role":"user","content":"{{question}}"}]}'

# If you loaded a decision model
curl http://{{console_ip}}:8081/v1/systemone -H 'Content-Type: application/json' -d '{"state":"I was charged twice this month. Refund me or I cancel.","questions":{"intent":{"type":"choice","instructions":"What does the customer want?","criteria":{"refund":null,"information":null,"cancel":null}}}}'

Terminology

GGUFThe model file format llama.cpp reads. It bundles weights into one file, like an install package.

QuantizationStoring model numbers in fewer bits to save space, like lowering image resolution.

Vulkan backendRuns model computation on the GPU through the general-purpose Vulkan graphics API.

Decision modelA model that gives choices, scores or yes/no answers in one pass without generating text.

JailbreakRemoving a maker's restrictions so a device can run arbitrary programs.