Further human + AI + proof assistant work on Knuth's "Claude Cycles" problem
TL;DR Highlight
A post sharing the process of solving the 'Claude Cycles' problem posed by mathematician Donald Knuth through collaboration between human experts, AI (LLMs), and formal proof assistants like Lean — demonstrating the real potential of AI to contribute meaningfully to mathematical research.
Who Should Read
Developers and researchers curious about how far AI can be used in mathematical reasoning or formal verification. Especially those interested in proof assistants like Lean or Coq, or AI for mathematics.
Core Mechanics
- This post covers a collaborative approach using human mathematicians + LLMs (large language models) + formal proof assistants (e.g., Lean) to solve a mathematical problem called 'Claude Cycles' posed by legendary computer scientist Donald Knuth.
- While the original tweet is inaccessible due to JavaScript being disabled, community comments and context indicate it reports further progress beyond previous work, showing that this kind of human-AI collaboration is producing real results in pure mathematics research.
- LLMs are characterized as strong at 'broad but shallow search' — meaning that when an expert sets the direction, LLMs excel at rapidly exploring a wide possibility space and proposing candidate ideas.
- Formal proof assistants like Lean and Coq are software tools that allow mathematical proofs to be written in a machine-verifiable form. Using these tools to verify proof ideas suggested by AI can reliably filter out errors.
- Some in the community predicted that in the future, applying AlphaGo-style reinforcement learning (RL) to Lean's syntax tree will prove more powerful than LLMs, since RL on the Lean syntax tree enables reasoning over much longer time scales.
- There was an observation that a professional mathematician's toolkit consists of roughly 10 core tricks, and if these tricks could be encoded as latent vectors (abstract representations inside AI models), AI could greatly accelerate mathematical research.
- Overall, a sober assessment also coexists: AI handles 'repetitive expert-level tasks' well when guided by specialists, but still has blind spots when it comes to truly difficult and complex problems.
Evidence
- "A witty comment went viral suggesting that 'AI will win a Fields Medal (the highest honor in mathematics) before it takes on the role of a McDonald's manager.' The argument is that while mathematics may seem like using a brain as a hammer to tighten a screw, LLMs' strength in 'broad and shallow search' actually makes them a good fit for mathematical research. There was also a prediction that AlphaGo-style reinforcement learning applied to Lean's syntax tree will become the dominant approach instead of LLMs, as RL-based methods can search over much longer time scales and are better suited for complex proofs. A realistic comment noted that it's unsurprising AI performs well when guided by experts — AI handles experts' 'lazy work' effectively, but still has blind spots on truly hard problems. One comment said it was hard to tell whether the thread participants were bots or humans, a meta-observation reflecting how deeply AI has become involved in mathematics community discussions, making it difficult to distinguish who is real. There were also comments wondering 'if anyone would tackle P≠NP this way,' and practical questions like 'what does this mean for ordinary people?' — reflecting that this type of research still largely remains within specialist communities."
How to Apply
- "When mathematical proof or algorithm correctness verification is needed, you can build a two-stage pipeline: generate draft proof ideas with an LLM, then verify them using a proof assistant like Lean or Coq to mechanically catch errors. Rather than trying to solve complex math problems with an LLM alone, design a role-sharing structure where a domain expert (or expert-level prompt) sets the direction and the LLM explores candidate paths — this yields far more reliable results. If you're interested in the AlphaGo-style RL + formal proof tool combination, use DeepMind's AlphaProof or related papers as references and experiment with reinforcement learning agents in the Lean environment. This field is currently advancing rapidly."
Terminology
Related Papers
Show HN: Mindwalk – Replay coding-agent sessions on a 3D map of your codebase
Claude Code나 Codex 같은 AI 코딩 에이전트가 세션 중 코드베이스의 어떤 파일을 탐색하고 수정했는지를 3D 지도 형태로 시각화해서 재생해주는 로컬 도구다. 에이전트가 작업을 어떻게 이해했는지 한눈에 파악할 수 있다.
Ghost Font: A font that humans can read but AI cannot
움직임(모션)을 이용해 글자를 표현해서 AI 모델이 정적 이미지 분석으로는 메시지를 해독하지 못하게 막는 실험적 프로젝트인데, 커뮤니티에서는 이미 GPT-5.6, Claude Opus 등으로 해독에 성공한 사례가 속출해 실효성 논쟁이 뜨겁다.
GPT-5.6, Grok 4.5, Claude, and Muse Spark build the same 4 apps
12개 LLM 모델에게 레이캐스터 미로, 루빅스 큐브, 계산기, Game of Life 앱을 각각 5번씩 만들게 해서 성공률·비용·속도를 비교한 실전 벤치마크다. GPT-5.6 Sol이 전반적으로 가장 일관된 결과를 냈고, Grok 4.5는 가성비 면에서 눈에 띄었다.
Benchmarking coding agents on Databricks' multi-million line codebase
Databricks가 자사 실제 코드베이스를 기반으로 여러 AI 코딩 에이전트의 성능과 비용을 직접 측정했고, 모델 토큰 가격과 실제 태스크 비용이 전혀 다르다는 점, 그리고 오픈소스 모델이 이제 최상위 수준에 도달했다는 점을 확인했다.
Estimating Uncertainty from Reasoning: A Large-Scale Study of Multi- and Crosslingual MCQA Performance in LLMs
LLM이 저자원 언어 질문을 받을 때 영어로 추론하게 하면 불확실성 추정 성능이 고자원 언어 수준으로 올라온다.
LLM-as-a-Verifier: A General-Purpose Verification Framework
LLM의 토큰 확률 분포를 활용해 discrete 점수 대신 continuous 점수를 뽑아내면, 추가 학습 없이 코딩·로봇·의료 에이전트 평가 정확도를 SOTA로 끌어올릴 수 있다.