Why Large Language Models Fail at Tabular Prediction

TL;DR Highlight

LLM accuracy drops as tabular dimensionality grows, unlike every classical baseline tested.

Who Should Read

A backend or data developer who pastes whole CSVs into an LLM for classification or prediction and gets unreliable results.

Core Mechanics

Four of five failure hypotheses falsified
- Noisy or non-linear data, CSV format, numeric tokenization, and test points per query were not the cause in controlled experiments.
- Input dimensionality (number of columns) was decisive.
Only the LLM gets worse with dimension
- The authors applied random linear projections to 31 benchmark datasets and compared nine methods.
- The LLM was the only method whose accuracy fell as dimensionality grew, while every classical baseline stayed flat or improved.
Purest possible inference setup
- A single generation pass over a prompt containing the full training and test data.
- No tools, no agentic scaffolding, no fine-tuning.
In 2D it behaves like a distance-based method
- They compared its behavior against 252 configured classical models.
- In two dimensions the LLM predicts like a local, distance-based method, with up to 91.6% grid agreement.

Community Reactions

Opinion: build a harness and use proper tools first
- One commenter said you should ask what the best tool for the data is and build a proper harness instead of feeding raw data to an LLM.
Opinion: maybe tree boosting is just that good
- A commenter suggested the issue may be that tree boosting is spectacular on tabular data, not that LLMs are bad at it.
Experience: surprised CSV format was falsified
- A commenter who ran needle-in-haystack tests with gpt-3.5-turbo and gpt-4o found table format mattered a lot.
- They said models read CSV vertically very poorly but handled markdown well.
Criticism: reasoning level not reported
- A commenter noted the paper omits the reasoning level or budget, which could change results a lot.
- Another called the setup irrelevant to real use cases since it excludes tools and agents.

How to Apply

Audit where you feed whole tables to an LLM
- Search your code today for places that paste a CSV into an LLM for classification or prediction.
- Run the same data through logistic regression or a tree model and compare accuracy.
Plot accuracy against column count
- Increase columns to 5, 10, and 20 and check whether LLM accuracy drops.
- Compare against classical models over the same range.
Use the LLM to orchestrate, not predict
- Leave prediction to classical or tabular-specific models and limit the LLM to data prep and calling them.
- The paper tested a no-tools setup, so measure this structure's benefit separately.

Try It

copy & run

# Paste into Claude or ChatGPT — simplified version of the paper's method (plan an LLM-only vs classical model comparison)

I am asking an LLM directly to classify tabular data via a prompt. Design an experiment to check whether LLM accuracy drops as the number of columns grows.

Data description: {{data description and column count}}
Current prompt: {{your current prompt}}

Requirements:
1. Lay out experiment steps that increase the number of columns.
2. Write scikit-learn code that runs logistic regression, random forest, and kNN on the same data.
3. Explain how to compare LLM and classical model accuracy per column count in a table.

Next step: paste the results table and ask the LLM to interpret at which column count it starts to lose.

Terminology

Tabular predictionPredicting values from spreadsheet or database-style data, such as churn prediction or fraud detection.

Tabular foundation modelA pretrained model built specifically for tables, used instead of a general LLM.

Random linear projectionA transform that mixes data along random directions to change the number of dimensions. It tests how performance changes with column count alone.

Classical baselineAn older, traditional machine learning method used as the reference a new model must beat.

Grid agreementA metric that splits a 2D plane into a grid and measures how often two models predict the same thing.