We rewrote JSONata with AI in a day, saved $500k/year
TL;DR Highlight
Reco saved $500K annually by rewriting their Node.js-based JSONata evaluation pipeline in Go using Claude AI — but the HN community fired back with criticism: 'Why did you let this linger so long?' and 'Why didn't you use an existing Go library?'
Who Should Read
Backend engineers whose data pipelines waste money on cross-language RPC calls, or engineers considering using AI code generation tools to replace legacy components.
Core Mechanics
- Reco's security policy engine evaluates JSONata expressions against billions of events, but since JSONata's reference implementation is in JavaScript, it couldn't be used directly in Go services. They ended up running a dedicated Node.js jsonata-js pod on Kubernetes and having Go services call it via RPC — an architecture they maintained for years.
- This architecture required a round-trip of serialization → network transfer → evaluation → result serialization → return for every event, costing roughly $300K per year in compute alone, with costs growing as customers and detection rules increased.
- The solution was to port JSONata directly to Go. The approach mirrored Cloudflare's rapid rewrite strategy: migrate the official jsonata-js test suite to Go, then implement the evaluator until all tests pass.
- Using Claude as the AI tool, the rewrite was completed in a single day at a cost of roughly $400 in Claude tokens. Switching to Go inline execution eliminated RPC overhead, bringing the jsonata-js pod cost from $25K/month down to zero.
- Total savings were announced at $500K annually, which appears to include not only compute costs ($300K) but also deployment complexity and operational overhead from running two language stacks simultaneously.
- JSONata's reference JavaScript implementation is approximately 5,500 lines, while the AI-generated Go implementation is approximately 13,000 lines — the increase likely reflects the effort of expressing JavaScript's dynamic characteristics and JSONata's JS-based grammar nuances in Go.
- The rewritten Go implementation was validated against the official JSONata test suite, and after production deployment the dramatic cost reduction provided immediate internal proof of ROI.
Evidence
- "The core criticism wasn't about the 'AI rewrite' itself, but rather 'why was this left alone for so long?' The fact that the rewrite cost only $400 in Claude tokens implies the codebase wasn't that large — meaning an engineer could have ported it manually at a comparable scale. Many commenters expressed disbelief at years of wasted spending at roughly $300K/year, the equivalent of one full-time employee. Critics also pointed out that two existing Go JSONata implementations already existed, yet Reco never explained why they didn't use them. One of those, `github.com/jsonata-go/jsonata`, was itself built with Claude Code and Codex in early 2025, passes the official test suite, and runs in production — confirmed by a contributor who commented on the post. Many felt it was a shortcoming for an engineering blog not to explain why existing libraries were unsuitable. The benchmarking approach also came under fire: measuring performance at the app level captures the effect of eliminating RPC overhead, not the actual performance of the Go JSONata implementation itself. Without an isolated, apples-to-apples comparison of the JS and Go implementations, concerns arose that 'the AI-generated implementation might actually be slower.' Long-term maintainability of AI-generated code was another concern raised. The JSONata open-source project currently has 15 open PRs and 150 open issues, and Reco now owns a self-maintained fork of 13,000 lines of Go code. Commenters questioned how upstream changes would be tracked and whether that maintenance cost was factored into the savings calculation. There were also cynical predictions that 'posts like this will become more common' — a pattern of reframing basic architectural fixes as AI wins, followed by a vicious cycle of shipping AI-generated code without fully understanding it and accumulating new bugs over time. One commenter warned of a future where 'code output increases 100x and token costs double developer salaries, but actual impact remains minimal.'"
How to Apply
- "If your Go services call out to a Node.js, Python, or other language runtime via RPC or a sidecar, start by calculating the monthly compute cost of that component. If the annual cost exceeds tens of thousands of dollars, it's worth exploring an inline port using an AI coding tool like Claude Code or Cursor. The key prerequisite: secure the official test suite before porting so you have a clear standard for verifying behavioral equivalence. If you need to use JSONata in a Go pipeline, evaluate `github.com/jsonata-go/jsonata` (ported with AI in early 2025, passes the official test suite, production-verified) before rewriting from scratch like Reco did. If an existing library meets your requirements, you can use it without taking on maintenance burden. When using AI to rewrite a legacy component, follow this sequence: port the official test suite to the target language → have AI implement until tests pass → run isolated benchmarks to verify performance before production deployment. The reason the benchmarking in this blog post drew criticism is precisely that it only looked at system-level metrics without isolated verification. When forking an open-source library with AI-generated code, define your maintenance strategy upfront. Either designate someone responsible for periodically syncing upstream updates, or explicitly decide to freeze features at the fork version and manually apply only security patches — failing to do this will create technical debt down the line."
Terminology
Related Papers
Claude Code sends 33k tokens before reading the prompt; OpenCode sends 7k
동일한 모델과 작업 환경에서 Claude Code와 OpenCode의 실제 토큰 사용량을 API 레벨에서 측정한 결과, Claude Code가 시스템 프롬프트 오버헤드만으로 OpenCode 대비 4.7배 더 많은 토큰을 소비한다는 것을 확인했다.
Mesh LLM: distributed AI computing on iroh
사무실, 집, 클라우드에 흩어진 GPU들을 하나의 OpenAI 호환 API로 묶어주는 분산 LLM 실행 시스템으로, 비싼 API 비용 없이 큰 모델을 직접 운영할 수 있다.
Show HN: Frugon – Find which LLM calls a cheaper model could handle (local, MIT)
내 LLM API 비용이 어디서 새는지 로컬에서 분석해주는 오픈소스 CLI 도구로, 비싼 모델 대신 저렴한 모델로 전환 가능한 호출을 골라낸다.
Jamesob's guide to running SOTA LLMs locally
2천 달러짜리 RTX 3090 한 장부터 4만 달러짜리 RTX PRO 6000 4장 셋업까지, 로컬에서 최신 LLM을 직접 돌리는 방법을 하드웨어 선택·구성·실행 설정까지 통째로 정리한 실전 가이드다.
Faster embeddings: how we rebuilt the ONNX path in Manticore
Manticore Search가 기존 SentenceTransformers/Candle 백엔드를 ONNX Runtime으로 교체해 텍스트 임베딩 생성 속도를 평균 14배 향상시켰다. 별도 모델 서비스 없이 DB 내부에서 직접 임베딩을 처리하는 구조에서 INSERT 속도가 곧 임베딩 속도이기 때문에 이 개선은 실질적인 ingest 처리량 향상으로 직결된다.
Asymmetric Quantization: Near-Lossless Retrieval with 97% Storage Reduction
멀티벡터 검색 모델의 문서 벡터를 1비트 이진값으로 압축하고 쿼리 벡터만 int8로 유지하는 비대칭 양자화 기법으로, 스토리지를 97% 줄이면서 검색 품질 손실을 0.61점(NDCG@10 기준)에 그치게 만든 실제 프로덕션 적용 사례다.