Ultrafast machine learning on FPGAs via Kolmogorov-Arnold Networks

Who Should Read

An embedded or hardware developer weighing FPGA acceleration for real-time inference systems that need microsecond or nanosecond latency.

Core Mechanics

• KANs fit LUTs naturally
  • A KAN puts a learnable univariate function on each edge instead of a scalar weight.
  • Each univariate function fits in one small lookup table (LUT).
  • The input domain is small, such as [-1, 1], so the table covers the whole range.
• Multivariate LUTs blow up exponentially
  • LUT entries for a multivariate function grow exponentially with input dimension.
  • KANs sum univariate functions, which avoids this and makes pruning easy.
• Inference is LUT lookups plus an adder tree
  • Trained KANs are trained in software (PyTorch), frozen, and deployed to the FPGA.
  • Activations are computed in parallel via LUTs, and sums use an adder tree of n_in-1 pairwise additions.
  • The paper reports a 2700x speedup over prior KAN-FPGA implementations.
• Online learning stores B-spline bases in LUTs
  • Coefficients change during training, so full activations cannot be precomputed.
  • The B-spline basis functions go into LUTs and are multiplied by the coefficients.
  • Forward pass, backward pass, and gradient updates all run on the FPGA.
• B-spline locality keeps compute light
  • Only S+1 of the G+S basis functions are nonzero for any input.
  • Hardware logic scales with S+1, not G.
  • Growing the grid size G adds expressivity with near-constant resources.
• Basis sum of 1 stabilizes fixed-point training
  • Because the basis functions sum to 1, outputs stay between the min and max coefficient.
  • Activation and gradient ranges are predictable regardless of input, which makes quantization ranges easier to choose.
• Online learning scales past 50,000 parameters
  • Forward and backward passes run at sub-microsecond latency.
  • The authors say this had not been achieved for gradient-based learning before.

Community Reactions

• Experience: it doesn't speed up LLM inference
  • One commenter tried targeting 100,000 tokens/s with a 3.28M-parameter model and found even that was an order of magnitude too large to benefit.
  • They felt the approach targets latency, not throughput.
• Pushback on model size limits
  • A commenter argued this limits you to very small models or very large FPGAs.
  • They said it only makes sense for simple ML tasks that truly need sub-microsecond latency.
• Concern that FPGA synthesis isn't trivial
  • A commenter noted many FPGA LUTs are 6:3 rather than 2:1 and also include flip-flops and muxes.
  • They suggested translating activations into LUTs may not be as easy as it sounds.
• Question on activation shape variety
  • One commenter suspected 90% of KAN's benefit may come from a small variety of function shapes.
  • They proposed sweeping input-weight and activation precision to find the best accuracy per parameter bit.

How to Apply

• Run pykan and compare against an MLP
  • Install pykan today and compare an MLP and a KAN with matched parameter counts on a small function-approximation task.
  • Plot the learned activations and check whether their shapes are simple.
• Check FPGA fit against your latency target
  • Confirm whether your inference latency target is at or below a microsecond.
  • Keep batch or large-model workloads on GPUs and rule them out for this approach.
• Quantize a trained KAN into LUTs and measure accuracy
  • Sample each activation at your input bit width to build a table, then measure accuracy loss per bit width.
  • Prune near-zero activations and check how the LUT count changes.

Try It

# Paste into Claude or ChatGPT (simplified version of the paper's method)

My problem: {{problem description: input dimension, output, required latency}}

Assume I implement a KAN (Kolmogorov-Arnold Network) on an FPGA using LUTs. Please:
1. Compute the number of entries and memory size when each edge activation becomes a LUT with b_i input bits and b_o output bits (b_i={{input bits}}, b_o={{output bits}}, edges={{number of edges}}).
2. Compare against the entry count of a multivariate LUT with the same input dimension.
3. Explain how much LUT and adder usage drops if I prune near-zero activations.
4. Tell me whether this approach suits my problem, and if not, why.

Terminology

Related Papers

• DeepInfer: High-performance inference framework for DeepSeek models

  DeepSeek open-sourced DeepInfer, its production inference engine, under Apache-2.0.

• Mercury 3: Diffusion at scale

  Scaling diffusion LLM 40x gives Mercury 3: 4–8x faster than speed-tier models, similar quality

• DeepSeek banned from U.S. federal civilian agencies under new DHS order

  DHS order bans DeepSeek from U.S. federal civilian agency IT systems

• Timestral: Mistral's time-series foundation models

  A 209M time-series model ranks first on three benchmarks and runs on CPU.

• DeepSeek V4.1 Flash 1017 (Max)

  DeepSeek V4.1 Flash 1017 Max scores 45 on the Intelligence Index at 211 tok/s and $0.30 input.

• If you don't run your own LLM proxy, how do you know whats going on?

  Running your own LLM proxy shows which keys, models, and cache misses drive your usage.

• DeepSeek: DeepSeek v4.1 Flash 1017

  DeepSeek V4.1 Flash 1017: $0.05/M input, 1M context, built for agents

• DeepSeek-v4.1-Flash-1017

  DeepSeek V4.1 Flash minor update lifts coding and agent benchmarks while cutting run time and output length

Related Resources

• https://aarushgupta.io/posts/kan-fpga/
• https://dx.doi.org/10.1145/3748173.3779202
• https://arxiv.org/abs/2602.02056
• https://github.com/KindXiaoming/pykan
• https://news.ycombinator.com/item?id=40219205
• https://arxiv.org/abs/2512.15742v2
• https://youtu.be/wgcSsJ69x1c?si=fiUl1YGTgaTt_bn9
• https://kindxiaoming.github.io/blog/