Continuous Diffusion Language Models (CDLM's)

Who Should Read

An LLM API developer curious how far diffusion-based text generation has come compared to autoregressive models.

Core Mechanics

• Continuous diffusion vanished in 2023 and is returning
  • In 2022, papers like Diffusion-LM, SED and CDCD appeared in a flurry.
  • After 2023, nearly all new work moved to discrete diffusion.
  • The author says recent research activity suggests the tide is turning.
• CDLMs add noise to token embeddings
  • Tokens become continuous vectors, and Gaussian noise is added to them.
  • The only difference from discrete diffusion is the order: discrete corrupts tokens then embeds, continuous embeds then corrupts.
  • The appeal is reusing sampling and distillation tools from image diffusion.
• A training efficiency gap is cited as one reason for the decline
  • In May 2023, Gulrajani & Hashimoto measured Plaid-1B.
  • It was 64x less training-efficient than an autoregressive baseline.
  • The author also names the shift to performance-focused research after ChatGPT as a possible factor, but calls it speculative.
• CDLMs need four ingredients

Community Reactions

• Rebuttal that decoders already dominated before ChatGPT
  • One commenter says decoders were already dominant for chat in 2020.
  • They argue RL methods like RLHF and DPO drove mainstream adoption.
  • They argue diffusion never took off because it is a more complicated way to generate tokens.
• Hope for a thinking scratchpad alongside output
  • One commenter says current models rely on compressed CoT to get intelligence per token.
  • With diffusion, they say, models could instead output more tokens per step.
• Opinions that autoregressive sampling is fundamentally limited
  • One commenter finds diffusion more coherent.
  • Another says autoregressive LLMs still have low-hanging fruit.

How to Apply

• Check whether sequential generation is your LLM bottleneck
  • Spend 30 minutes today measuring output-token generation time as a share of response latency in your logs.
  • If it is large, diffusion-based parallel generation is worth tracking.
• List infilling and editing features as diffusion candidates
  • Pull out features like mid-text infill or constrained generation.
  • The article presents these as tasks where discrete diffusion has an edge, so shortlist them.
• Compare CDLM papers with a four-ingredient checklist
  • Fill a table with embedding, loss, noise schedule, and self-conditioning.
  • You can then see at a glance which part explains differences in performance.

Try It

# Paste into Claude or ChatGPT (simplified version of the article's ideas: build a paper comparison table)

I'm a developer using LLM APIs and not an ML specialist. Read the abstract below and summarize it as a table from a continuous diffusion language model perspective.

Paper abstract: {{paper abstract}}

Columns: embedding strategy (explicit/pre-trained/jointly learned), loss function (MSE/cross-entropy/likelihood bound), noise schedule handling, whether self-conditioning is used.
Write 'not mentioned' for anything missing from the abstract. Do not guess.

Next: do the same for a second paper and compare them side by side.

Terminology

Related Papers

• Mercury 3: Diffusion at scale

  Scaling diffusion LLM 40x gives Mercury 3: 4–8x faster than speed-tier models, similar quality

• Timestral: Mistral's time-series foundation models

  A 209M time-series model ranks first on three benchmarks and runs on CPU.

• DeepSeek V4.1 Flash 1017 (Max)

  DeepSeek V4.1 Flash 1017 Max scores 45 on the Intelligence Index at 211 tok/s and $0.30 input.

• If you don't run your own LLM proxy, how do you know whats going on?

  Running your own LLM proxy shows which keys, models, and cache misses drive your usage.

• DeepSeek: DeepSeek v4.1 Flash 1017

  DeepSeek V4.1 Flash 1017: $0.05/M input, 1M context, built for agents

• DeepSeek-v4.1-Flash-1017

  DeepSeek V4.1 Flash minor update lifts coding and agent benchmarks while cutting run time and output length

• Show HN: Llmspeed – feel how fast a local LLM runs before you buy the hardware

Related Resources

• https://sander.ai/2026/08/24/continuous-dlms.html