The gap in shared understanding of LLM capability is widening

TL;DR Highlight

Karpathy says the LLM capability gap is now a four-tier funnel, from 6B barely exposed to ~5,000 with internal access.

Who Should Read

An LLM app developer struggling to communicate with non-technical colleagues and execs who use AI at a very different level

Core Mechanics

The gap is a four-tier funnel
- Napkin math: ~6B people (~75%) have barely touched LLMs.
- 1-2B (~20%) are casual users of free-tier chatbots.
- ~20M math and code professionals (0.2%) and ~5,000 with internal access sit at the top.
20M people finish weeks of work with a prompt
- Math and code professionals see agents complete large projects that took weeks or months.
- Examples include building, copying, translating apps and decompiling binaries.
- Karpathy says he was hand-typing code less than a year ago.
~5,000 insiders see agent swarms
- The public preview looks like thousands of agents collaborating for weeks on huge software projects.
- Examples include zero days, machine-speed cyber ops, new science and math advances.
- Human review and comprehension are starting to fall behind.
Three factors drive the funnel
- Impact scales with ambition, problem size and horizon; a paragraph-answer question barely stresses the system.

Community Reactions

Terminal access widened the divide, one view says
- Giving LLMs a terminal, CLIs and web access made developers feel far more powerful, per a commenter.
- Average ChatGPT users felt plateaued by comparison.
An LLM is a capability, not a product, one comment says
- Developers can build harnesses and automation themselves, so they stand at the front.
- Non-developers only get chatbots, so the funnel persists until products arrive.
Skepticism that 5,000 insiders are far ahead
- A commenter says internal models still need skilled users, and 90% vs 93% agent success is not a big difference.
- Another says each released model has been a modest increment.
Prompt-and-forget fell short for lasting code, one dev found
- Even with spec-driven prompts, edge cases, architecture issues, performance problems and duplicated code needed manual fixes.
- They say fully autonomous agents are not the road to quality unless it is throwaway software.

How to Apply

Map your team onto the four tiers
- Spend 30 minutes today placing teammates and stakeholders into the article's four tiers.
- Note beside each name whether they use a free chatbot or a paid agent.
Demo a big problem to non-dev colleagues
- Pick a real task that took days and solve it with an agent in front of them.
- Per factor 1, bigger problems make the gap visible.
Hand agents verifiable tasks first
- Choose tasks with tests or automatic checks and give them to an agent.
- Per factor 2, capability peaks in digital, verifiable domains.
Keep human review for agent output
- Merge agent code for long-lived systems only after PR review and passing tests.
- The article says human review is falling behind, so budget review time.

Try It

copy & run

# Paste into a Claude or ChatGPT chat (a simple version of the article's four-tier funnel for team diagnosis)

I want to diagnose the LLM usage gap on my team. Classify each person or stakeholder into these tiers.
Tier 1: barely touched LLMs
Tier 2: occasional free-tier chatbot use (search or writing aid)
Tier 3: professional use of frontier models and agents in math or code
Tier 4: internal-access level (skip if not applicable)

Team info: {{each person's role and the AI tools they use}}

Put each person's tier in a table, and for each tier suggest one concrete experiment they can run this week to move up. Use only the info I gave you, no guesses.

Next step: ask it to compare how a Tier 3 person and a Tier 2 person would each solve the same task.

Terminology

JaggednessAn LLM can be PhD-level in one area and clumsy in another. Like someone brilliant at math who is terrible at navigation.

RL on verifiable rewardsTraining a model on problems a machine can grade automatically. For example, scoring code by whether it passes tests.

Frontier-grade LLMThe highest-performing model available at the time.

Zero dayA security vulnerability with no patch yet.

Agent swarmMany agents splitting roles and working in parallel, like an ant colony on a big job.