Fireworks AI security breach impacts Notion and their customers
TL;DR Highlight
A vendor breach at Fireworks AI exposed Notion AI chat transcripts for under 1% of workspaces.
Who Should Read
Backend or platform developers whose services send user conversations to external LLM inference vendors
Core Mechanics
- Under 1% of workspaces affected
- Vendor Fireworks AI notified Notion of a security incident.
- Notion contacted affected customers directly and the investigation is ongoing.
- Exposed data was Notion AI chat transcripts
- Any model input or output written to the chat is in the transcript.
- What an agent writes to the transcript varies by model.
- Fireworks powered Insights and open-weight serving
- Insights classified jobs to show customers their own AI usage.
- Fireworks was also a general inference provider for open-weight models.
- LEAP data was used for training experiments
- LEAP is a program where customers volunteer data for testing new AI.
- Notion previously experimented with training task-specific models on Fireworks with it.
- Notion cut Fireworks off on November 3
- It deleted data in Fireworks, stopped all traffic, and halted training runs.
- Insights was disabled during the investigation.
- Rotate pasted credentials right away
- Notion says to treat credentials pasted into Notion AI in the period as exposed.
- Affected data can be downloaded as CSV inside the workspace.
Community Reactions
- Pushback on the no-identifying-info claim
- Commenters said the conversation content itself is the critical information and makes identification easy.
- They argued that stressing missing metadata gives a false sense of security.
- Question about ZDR vs. stored data
- A comment noted OpenRouter labels Fireworks as ZDR (no prompt retention).
- Others asked whether Notion requested retention; the article does not answer.
- A user who checked found nothing sensitive
- They said their exposed transcripts held nothing particularly sensitive.
- They added that unlucky users could have sensitive content, or third-party info, exposed.
- Complaints about checking impact
- Some said seeing affected members requires an Enterprise plan.
- One said the CISO would not confirm access, which complicates GDPR notifications.
How to Apply
- List what your service sends to external LLMs
- Today, write down every field leaving your LLM call code.
- Mark input paths where passwords, keys, or personal data can appear.
- Add secret masking before sending prompts
- Mask API key and token patterns with regex before user input reaches the model.
- Apply the same filter before saving chat logs.
- Confirm retention policy with each inference vendor
- Get prompt retention and duration in writing per vendor.
- Keep data used for training or experiments separate from normal inference traffic.
- Prepare a vendor incident runbook
- Define the order: cut the connection, request deletion, notify customers.
- Put a vendor kill switch in config, as Notion disconnected Fireworks.
Try It
# Paste into Claude or ChatGPT as-is (a simplified checklist prompt based on this incident)
Our service sends user conversations to an external LLM vendor.
- Data sent: {{fields sent}}
- Vendor and purpose: {{vendor and purpose}}
- Chat log retention: {{retention approach}}
Using this, assess the risk if the vendor has a security incident.
1) Types of data that could be exposed
2) Paths where users may paste passwords or API keys
3) A checklist for cutting the vendor connection and deleting data
4) A draft customer notification
Next step: pick the riskiest path from the answer and ask for input-masking code.Terminology
Related Papers
LLM assisted reverse engineering
Pointing an agent at binaries with skills and MD notes made years-long hacks feasible
Metrics-in-the-Loop: Complexity Feedback Reshapes Agent-Written Code
Injecting per-turn complexity reports into coding agents cut code erosion without changing correctness
Why is Gemini 4 Carbon taking so long?
Carbon took nearly an hour on bug fixes that Astra and Opus finished in minutes, mostly from extra unrequested work.
How the Claude Code team thinks about verification
E2E, property and differential tests plus verification rules let Claude run in parallel.
Running coding agents without handing them my credentials
Credentials stay on the laptop; agents run commands there via Touch ID approval
Windows 11 Is Collecting Data on Which Coding Agents and Developer Tools You Use
Reverse engineering shows Windows 11 logs which coding agents you use and reports it as optional diagnostics
The Silent Cost of Conversation History in LLM Reasoning
Keeping full conversation history dropped accuracy from 0.6 to 0.1 within 20 turns; a verified state summary prevented it.
Anthropic upgrades Claude Code Projects to run parallel AI workers
Claude Code Projects now has a coordinator that splits work into parallel threads and opens PRs.