From Out-of-Distribution Detection to Hallucination Detection: A Geometric View

Who Should Read

ML engineers who need real-time hallucination detection in LLM serving pipelines. Especially useful for reasoning tasks where traditional uncertainty metrics are unreliable.

Core Mechanics

• OOD (out-of-distribution) detection techniques transfer directly to hallucination detection with no modification
• Single-sample detection works — no need to generate multiple outputs for consistency checking
• The method works on any LLM without additional training or fine-tuning: just access to hidden states is required
• On reasoning tasks (math, logic), the OOD-based detector outperforms existing hallucination detectors by 15-20%
• Detection latency overhead is under 5ms per query — suitable for real-time serving

Evidence

• AUROC on hallucination detection: OOD method 0.847 vs. next best method 0.723 on reasoning benchmarks
• Single-sample OOD detection matches consistency-based methods (requiring 5+ samples) at 94% of their AUROC score with 1/5 the inference cost
• Detection latency: 3.2ms additional overhead per query on a standard A100 GPU setup
• Generalization: trained on one LLM family, tested on another — AUROC drops only 0.03 (3% relative)

How to Apply

• Extract hidden states from the LLM's last few layers during inference — most serving frameworks expose this via hooks
• Fit an OOD detector (e.g., Mahalanobis distance) on a reference set of known-correct outputs during calibration
• At inference time, compute the OOD score for each response and flag high-OOD-score outputs for human review or re-generation

Code Example

import torch

def compute_fDBD_score(hidden_states, lm_head_weight, k=1000):
    """
    fDBD-based hallucination detection score (higher = less likely hallucination)
    hidden_states: [seq_len, hidden_dim] - penultimate layer output
    lm_head_weight: [vocab_size, hidden_dim]
    """
    seq_scores = []

    for z in hidden_states:  # each token generation step
        logits = lm_head_weight @ z  # [vocab_size]
        c_hat = logits.argmax().item()
        w_hat = lm_head_weight[c_hat]

        # exclude c_hat from top k+1
        topk_vals, topk_idx = logits.topk(k + 1)
        alt_indices = topk_idx[topk_idx != c_hat][:k]

        # compute distance to decision boundary
        distances = []
        for c in alt_indices:
            w_c = lm_head_weight[c]
            logit_diff = logits[c_hat] - logits[c]
            w_diff_norm = (w_hat - w_c).norm()
            dist = logit_diff / (w_diff_norm + 1e-8)
            distances.append(dist.item())

        z_norm = z.norm().item() + 1e-8
        step_score = sum(distances) / (len(distances) * z_norm)
        seq_scores.append(step_score)

    return sum(seq_scores) / len(seq_scores)  # lower value suggests hallucination

# Usage example (based on Hugging Face model)
# outputs = model(input_ids, output_hidden_states=True)
# hidden = outputs.hidden_states[-2]  # penultimate layer
# weight = model.lm_head.weight
# score = compute_fDBD_score(hidden[0], weight, k=1000)
# threshold = 0.5  # tune with validation set

Terminology