No signs of AI payoff outside the tech sector

Who Should Read

A developer or team lead proposing LLM adoption who needs ROI evidence and faces token cost pressure

Core Mechanics

• No AI payoff outside tech
  • The article says AI has delivered no financial payoff to corporate America outside tech.
  • Revenue per worker and profit margins rise for the Mag 7 but are flat for the S&P 493.
• Productivity stats show nothing yet
  • The author sees no sign of AI boosting productivity growth.
  • The chart uses the San Francisco Fed's total factor productivity (TFP) series.
• Tokens are becoming a cost center
  • Mainstream companies increasingly see token spend as a cost that weighs on margins, not as growth.
  • VC-backed firms spend more on tokens, but mainstream firms won't pay for the latest model.
• Agent spend is confined to a small slice
  • If agentic AI and token spending stay in a small part of the economy, macro effects will be very limited.
  • Charts draw on the Ramp AI Index.
• Turning cheap intelligence into revenue is the test
  • The question is whether mainstream firms can turn commoditized, cheaper intelligence into revenue and margin growth.
  • The author says business intelligence is mostly judgment, which depends on context.
• AI lacks accountability and unwritten rules
  • The article lists accountability, trust, culture, unwritten rules, ethical trade-offs and fairness as things AI does not understand.
  • These shape reputational risk, revenue growth and margins.

Community Reactions

• Comparison to factory electrification
  • Dropping an electric motor into a water-powered mill didn't work; the gains came from rebuilding the building.
  • The view is that AI-native new companies will move the needle.
• It's early, plus the Solow paradox
  • One commenter notes Amazon took decades to overtake Walmart, so diffusion takes time.
  • Several comments quote Solow's 1987 line about computers and productivity statistics.
• Other industries lack test suites
  • Software can use non-deterministic systems because tests act as validation gates.
  • Other industries rely on manual checks, which can be slower and costlier than doing the work directly.
• Cutting work hours with a web chat
  • One commenter says a contracts manager cut 8-hour days to a few hours using Claude's web chat.
  • Such use isn't integrated into workflows, so it may not show up in statistics.
• Copilot check-processing was unreliable
  • A commenter had Copilot generate rename commands for scanned checks in batches of 20 but often got only 15 lines.
  • Failures worsened over long sessions and an upload limit eventually kicked in.

How to Apply

• Build an AI spend versus impact table
  • In 30 minutes, pull monthly LLM API cost per feature.
  • Add one time-saved or revenue metric next to each.
  • Mark features with no metric as cost items.
• Test whether the latest model is needed per task
  • Run 20 recurring tasks on a cheap model and the top model with identical inputs.
  • If the accuracy gap is small, downgrade to cut token cost.
• Add automatic validation gates to AI output
  • Put format checks and rule tests on LLM output in non-code workflows.
  • Send only failures to humans to reduce manual review cost.

Try It

# Paste into Claude or ChatGPT (the article is economic analysis with no code, so this is a simple audit prompt)

You are an analyst auditing AI ROI. Below are the features in our service that use an LLM and their monthly cost.
{{feature list, monthly token cost, expected benefit}}

For each feature, make a table with:
1) One metric that can measure its impact
2) A test method to check whether a cheaper model is enough instead of the latest model
3) Candidate rules for automatically validating the output

Next step: classify features with an empty metric as cost items and decide whether to keep them.

Terminology

Related Papers

• A 10-year-old CNN outperforms frontier LLMs for skin cancer diagnosis

  A fine-tuned Inception v3 beat GPT-6 Astra and Claude Fable 5.1 on 200 skin lesions

• America has lost 200k of jobs in media, film and the arts. How much is AI?

  US creative-industry jobs fell by over 200k in four years, but AI's share of the blame is hard to prove

• LLMs are making formal software verification feasible at a much larger scale

  Claude proved no-crash properties for 123 open-source libraries and found bugs along the way.

• PHP2Shell: AI uncovers critical RCE in PHP

  Wiz used AI to turn a decade-old PHP core flaw into a working RCE exploit

• Using LLM for confidence scoring is a bad idea

  An LLM's self-reported confidence is plausible text, not a measurement; build scores from outside rules.

• The Judge Can Debias Itself: Awareness and Self-Debiasing for Reliable LLM-as-a-Judge

  Warning an LLM judge about bias or letting it self-critique improved 20 of 21 model–bias combinations.

• 90% of US working hours are spent on tasks not covered well by AI benchmarks

  Mapping 29 benchmarks to job tasks shows 90% of US working hours lack good coverage

• Postgres 19 delayed after AI reviewers find more bugs than humans can fix

  LLM code review surfaced so many bugs that Postgres 19 slipped to November and 8-10 features were reverted

Related Resources

• https://www.apollo.com/wealth/insights-news/insights/daily-spark/no-signs-of-ai-payoff-outside-the-tech-sector
• https://www.brookings.edu/articles/the-solow-productivity-pa...