Private ChatGPT chats went public: Why OpenAI needs to be more careful

I won’t be wrong when I say that chatting with an AI feels as harmless as firing off an email to yourself – until it isn’t. If those chats are being indexed and appearing on Google search, that would send alarm bells ringing, as they rightly did in the case of a strange ChatGPT feature that frankly shouldn’t even have existed in the front place.

Survey

Last week, Fast Company revealed that ChatGPT users who opted into a new “Make this chat discoverable” feature were unknowingly sending their private conversations with the OpenAI chatbot straight into Google’s search index. A small, opt-in checkbox was all it took, buried beneath the share button, to transform a private one-on-one session with ChatGPT into a globally searchable web page. Within hours, search queries for “site:share.chat.openai.com” returned thousands of personal chats – some detailing therapy confessions, business secrets, and even criminal admissions, according to various reports.

Add As A Trusted Source For Google.

Add as a preferred source on Google

In full damage control mode after the initial revelation, OpenAI’s Chief Information Security Officer, Dane Stuckey, was quick to call the toggle a “short-lived experiment” – it was merely aimed at helping users showcase insightful exchanges. Yet the company underestimated how many people would assume “share” meant “share privately,” not “invite Google’s web crawlers.” By Friday morning, the feature was disabled, and OpenAI began coordinating with search engines to permanently delete the accidentally diverted conversations from their caches.

AI companies vs user trust

Imagine pouring your heart out to an AI – ChatGPT, Gemini, Copilot, Claude, or whoever – seeking career advice, drafting wedding vows, or hashing out a scandalous plot twist for your next novel. Only to discover a stranger could stumble upon every line with a simple Google search – not because of a hack or data breach, but a crazy feature. That sense of betrayal isn’t something to be taken lightly. Why such a chat sharing feature even existed in ChatGPT is beyond me, it makes no sense. 

Just think about it for a second, everyone who’s using AI to enhance their work or creativity is doing so in private, understandably hesitant to share their personal thoughts, ignorances and secrets with anyone. What made the OpenAI product team think it would be otherwise is beyond me – unless, of course, the idea was to test the limits of user behaviour. We are after all guinea pigs for big tech, for better or worse, aren’t we?

Also read: ChatGPT answers over 2.5 billion queries a day, shows internal data

This fresh ChatGPT debacle underscores a broader reality: default settings carry the weight of a company’s trust promise. A checkbox buried in fine print isn’t the same as informed consent. As Pieter Arntz of Malwarebytes aptly noted, “The friction for sharing potential private information should be greater than a checkbox – or not exist at all.” I wholeheartedly agree with this view.

User privacy needs to be paramount

This isn’t the first time AI’s privacy controls have flopped in the public eye. Only the outcry was swift, forcing OpenAI to correct the perceived mistake. This will only make more users grow wary of handing over their innermost thoughts to code. Therefore, OpenAI’s stumble is a timely reminder that transparency and clear UX design aren’t optional, but absolutely mission critical.

Also read: ChatGPT is changing the way we speak, study finds

Before surfacing any conversation, platforms should require a two-step confirmation – perhaps even a pop-quiz just to make it absolutely clear to the users of what they’re getting into. Needless to say, every conversation users have with AI chatbots should default to private, with discoverability toggled off. Period.

More importantly, governments should take these opportunities to tighten user privacy and data protection rules for AI, forcing companies to design features in compliance to begin with – rather than retrofit after a scandal erupts.

After all, the real test for AI isn’t how fast it can answer trivia – it’s how diligently it protects the private moments we entrust to it. When we talk to these systems, we’re not just generating text, but sharing fragments of our lives. Something that shouldn’t be trifled with.

Also read: MIT’s ChatGPT study says AI is making you dumber: Here’s how

Follow Us

Jayesh Shinde

Executive Editor at Digit. Technology journalist since Jan 2008, with stints at Indiatimes.com and PCWorld.in. Enthusiastic dad, reluctant traveler, weekend gamer, LOTR nerd, pseudo bon vivant. View Full Profile