Notion AI chats leaked via Fireworks: the workspace owner's checklist

Notion says a breach at Fireworks AI, a company that ran some of its AI workloads, exposed Notion AI chat transcripts from fewer than 1% of Notion workspaces. The transcripts were the ones Notion used for Insights, a feature that shows administrators how their teams use AI. Notion says most were unattributable, meaning they carried no label naming the author or the workspace, and no other Notion features were affected.

That still means whatever people typed into Notion AI, and whatever it answered, is now in someone else's hands. Notion's advice is to treat the data as exposed. For anyone who pasted a password or API key into a chat, that is a credential problem today.

How it happened

Fireworks says a third-party cloud provider hosting its research machines was breached on October 23, and the attacker used that access to reach Fireworks development machines on October 24. Fireworks says it has seen no further access since October 25 and says the evidence links it to a financially motivated group that sells access to AI models.

Notion heard on the evening of November 2, when Fireworks flagged a dataset that held only synthetic test data. By the morning of November 3 Notion had disconnected from Fireworks, deleted its data there and halted training runs. On November 6 Fireworks warned that customer data may have been accessed, and on November 8 it confirmed the dataset. Notion contacted affected customers and published its advisory on November 9. On November 11 it revoked Notion API keys it found in affected users' messages, a step the BleepingComputer, VentureBeat and Record reports do not mention.

Why zero data retention did not help

Fireworks had a contractual commitment to zero data retention with Notion. Notion's FAQ says data was retained anyway, from the Fireworks endpoints it used.

The explanation sits in two documents. Notion's own zero-retention promise applies to the API requests it sends to model vendors, and the FAQ says it is not meant to cover features that store data, such as chat history and Insights. Insights sent stored requests and responses to a third party for classification. On the Fireworks side, its documentation says a few features do store customer content, among them stored responses, batch jobs and training, and its blocking zero-retention policy is Enterprise-only and off by default. Neither company has said which Fireworks feature held the transcripts.

What workspace owners should do

• If Notion has not contacted you or your workspace owner, it says no action is needed for now.
• If you were contacted, open the workspace sidebar, click ?, then Security incident review. It works only in a desktop browser on the latest version of Notion, not on mobile.
• Click Download affected members for a CSV of names, emails, status and user IDs.
• Use Copy link to send affected members a link where they can review their own affected data. Notion's FAQ says members in most workspaces can also find it themselves under the same Security incident review menu, but an in-app notice quoted on Hacker News says members were not emailed, so tell them it exists.
• If you would rather members did not download their own files, Remove access to affected content in the same menu withdraws it. It needs a verified domain and deletes nothing.
• If your workspace has a verified domain, download affected_content_annotated.csv. Its __analysis column flags possible sensitive information in each conversation, which is the fastest way to find pasted credentials.
• Rotate every password, API key or token that appears, at the service that issued it. Deleting the secret from Notion does not revoke it.
• Insights is paused while Notion investigates, and Notion has stopped serving open-weight models through Fireworks.

What to ask any SaaS vendor

Notion's subprocessor list named Fireworks, with a ten-day objection window for customers who subscribe to change notices. Customers who have not signed up get no such notice. Three questions are worth sending to every SaaS vendor with AI features: which features store prompts and outputs, which AI subprocessors receive that stored data, and whether a zero-retention promise covers those features or only live model requests. Notion says it is now auditing its own critical vendors, including inference providers.

Sources: Notion advisory, Notion FAQ, Fireworks incident update, Fireworks zero data retention docs, Notion subprocessors, BleepingComputer, VentureBeat, The Record. Related: Northflank breach: rotate keys, Trigger.dev breach and environment variables.