Private AI chats were sold to the public as a safer, more contained alternative to posting on social media, yet a growing body of evidence shows those conversations are being quietly captured, traded and exposed. From browser extensions and VPNs to analytics vendors and court orders, the ecosystem around ChatGPT and other bots has turned intimate prompts into a lucrative data stream. What looks like a private box on your screen is, in practice, a funnel into a sprawling surveillance and monetization machine.
The result is a privacy crisis that touches everyone from casual users asking health questions to professionals feeding in client files and trade secrets. Millions of conversations are being harvested, bundled and sold, often without a clear way to opt out, while parallel legal and security shocks reveal just how widely those logs can spread once they leave the chat window.
The new market for your “private” prompts
I see the core problem as brutally simple: AI chat logs have become a commodity, and the people generating them are the last to know. Reporting on data brokers and ad-tech pipelines shows that millions of private ChatGPT conversations are being treated as raw material for profiling, targeting and resale, with users given little more than dense fine print instead of meaningful control. One investigation describes how “Millions of Private” ChatGPT logs are being packaged and “Conversations Are Being Harvested and Sold for Profit,” while warning that “There is no user-facing toggle to disable this,” a detail that undercuts the industry’s repeated assurances that privacy is just a settings change away, and that warning is captured in a report labeled Millions of Private.
What makes this market especially insidious is that it thrives on the illusion of intimacy. People pour into ChatGPT the kinds of details they would never post on Facebook or X, from relationship breakdowns to financial panic and workplace grievances. Yet the same reporting that exposes the sale of those logs also notes that “You know those memes about, an endorsement from Google itself,” a reminder that the AI boom is deeply entangled with the broader ad and analytics economy that already tracks users across the web. In practice, the chat box is not a vault, it is another sensor feeding a commercial surveillance stack that treats your prompts as just one more behavioral signal to be mined.
How Chrome extensions turned AI chats into a side hustle
One of the clearest examples of this quiet extraction comes from the browser itself. Security researchers have documented how popular Chrome add-ons marketed as productivity boosters or AI helpers were in fact siphoning off entire chat histories. A detailed warning urged users to “Uninstall Now” because “These Chrome Browser Extensions Are Stealing AI Chat Logs,” explaining that, according to cybersecurity firm Koi, the extensions were capturing prompts and responses from multiple bots and sending them to remote servers for analysis and resale, a pattern laid out in the alert titled Uninstall Now.
The problem is not limited to obscure tools with a few thousand installs. Another investigation found that a “popular” add-on in the Google Chrome ecosystem had been “silently stealing every AI prompt its users enter,” hoovering up anything typed into “most of the” major chatbots. The extension’s permissions looked routine, but under the hood it was logging prompts by default and routing them through its own infrastructure, effectively turning every AI query into a data point for someone else’s business model. For users, the distinction between the chatbot and the extension is invisible, yet that thin technical boundary is exactly where their privacy evaporates.
Urban VPN, BiScience and the proxy trap
Another front in this harvesting economy runs through VPNs and traffic anonymizers that promise protection while quietly doing the opposite. A recent security review of Urban VPN and related tools concluded that the service was not just routing traffic but also capturing AI chat content for downstream use. The report, produced by Tel Aviv based researchers, highlighted how a company called BiScience had previously been linked to large scale browsing data collection and quoted the finding that “Koi’s report notes that BiScience has previously been linked to large-scale browsing data collection” and that “Anyone who used” the proxy to access chatbots risked having their conversations repurposed for “marketing analytics purposes,” a pattern spelled out in the analysis of Koi.
What makes the Urban VPN case so troubling is that it collapses two layers of trust at once. Users turned to a VPN to shield their traffic from prying eyes, then used that protected channel to send deeply personal prompts to AI systems, assuming the combination would keep them safe. Instead, the proxy itself became the eavesdropper, with BiScience positioned to monetize the resulting trove. When “Anyone” who routed their chats through the service can have those logs folded into “marketing analytics purposes,” the idea of a private session becomes a fiction, and the VPN shifts from guardian to data broker.
Security firm Koi and the scale of AI chat scraping
Pulling these threads together, the work of Tel Aviv based security firm Koi shows just how systemic AI chat scraping has become. In a sweeping investigation, the firm traced a “massive data harvesting operation tied to a voluminous network of browser extensions and proxies,” concluding that AI conversations were being captured across multiple platforms and funneled into a shared commercial pipeline. The report stressed that the operation was not a one off but part of a broader pattern in which “Tel Aviv” researchers at Koi documented how chat logs from different bots were ending up in the same analytics and advertising ecosystems.
What stands out in Koi’s findings is the lack of meaningful user control. The firm noted that many of the tools involved either buried their data practices in vague language or failed to disclose them at all, while the platforms hosting them did not enforce clear standards for handling AI prompts. That gap leaves users exposed to a kind of ambient extraction, where every new extension or proxy can become another tap on the same stream of conversations. When a single research group can uncover such a wide network of harvesting, it suggests that the visible cases are only a fraction of what is actually happening behind the scenes.
OpenAI’s own privacy stumbles and the Mixpanel breach
The harvesting problem is not confined to third party tools. OpenAI itself has been pulled into a series of privacy shocks that reveal how fragile chat confidentiality can be once logs are stored and shared internally. One major incident involved The Mixpanel, an analytics platform used to track user behavior, which suffered a breach attributed to a group identified as ShinyHunters. In a detailed reconstruction of the event, investigators described “The Broader Context” and “Connection” between ShinyHunters and a wave of attacks on AI and SaaS platforms, noting that the Mixpanel compromise exposed data pipelines that touched OpenAI’s own systems, a chain of events laid out in the section labeled The Broader Context.
Even when there is no malicious breach, the way OpenAI handles chat logs has raised alarms. Legal filings and privacy complaints have argued that the company’s retention and internal use of conversations can “irreversibly harm” users’ privacy, especially when those logs are repurposed to train models or shared with partners. One analysis noted that, “Citing privacy concerns, artificial intelligence company OpenAI is a” target of growing regulatory scrutiny, with critics warning that its data practices could “irreversibly harm” users’ privacy, a concern captured in a piece by Wendy Davis, Staff Writer.
Courts ordering chat logs turned over
On top of commercial harvesting and security lapses, the legal system is now prying open AI chat archives in ways that could set far reaching precedents. In a closely watched case, a federal judge ordered OpenAI to hand over a vast trove of user conversations as part of discovery, rejecting the company’s attempt to shield the logs. The order, described as “The December” “Ruling” on chat data, required “Million Conversations Ordered Turned Over” after “On December” a court concluded that OpenAI’s objections were not “motivated by an improper purpose,” a sequence detailed in the account of The December 3 Ruling by Judge Wang.
For users, the implications are stark. Conversations they assumed were ephemeral or at least shielded from outside scrutiny are now subject to bulk disclosure in litigation they never signed up for. The fact that a single order can sweep in “20 Million” chats underscores how centralized and accessible these logs are once stored, and how little practical control individuals have over their fate. If one judge can compel such a handover, other courts and regulators will likely follow, turning AI chat archives into a routine evidentiary resource rather than a last resort.
When “opt in” quietly becomes “hybrid”
Even the basic question of whether your chats are used to train models has become murkier over time. One detailed critique of chatbot privacy noted that a major AI provider “once prided itself on being a fully opt-in system, but as of September 2025, it quietly flipped to a hybrid system,” blending explicit consent with default data collection. The analysis argued that this shift undermined earlier promises of user control and reinforced the idea that “chatbot privacy is an oxymoron,” urging people to “assume your data is always at risk” whenever they type into a bot, a warning laid out in a report dated Dec.
I see that change as emblematic of a broader pattern in the AI industry. Companies launch with strong privacy rhetoric, then gradually erode those protections as growth and monetization pressures mount. The move from pure opt in to “hybrid” collection means that even users who think they have declined data sharing may still find their prompts folded into training sets or analytics streams. When the default expectation becomes that “your data is always at risk,” the burden shifts unfairly onto individuals to navigate a maze of toggles and disclosures that can change without clear notice.
Search exposure and the Google indexing fiasco
Beyond covert harvesting, some of the most jarring exposures have come from features that were supposed to make AI chats more useful. Earlier this year, OpenAI quietly rolled out a function that allowed ChatGPT conversations to be indexed by search engines, only to discover that personal and sensitive chats were appearing in public search results. After a wave of backlash, the company “kills ChatGPT feature that exposed personal chats on Google,” acknowledging that “The feature was rolled out quietly earlier in the year but caused much uproar recently as many private conversations started showing up in search results,” and that this had put the “privacy and safety of users at risk,” a sequence described in a detailed breakdown of how The feature backfired.
The episode illustrates how even well intentioned features can collide with the reality of how people use AI. A tool designed to make helpful answers easier to find ended up turning therapy like confessions and workplace secrets into search snippets. It also shows how thin the line is between “private” and “public” in the AI context: a single configuration change can flip a chat from confidential to globally searchable, with users given little warning or recourse. Once indexed, those logs can be scraped, archived and fed into the same data markets that already trade on social media posts and browsing histories.
Warnings to users and the illusion of control
As these incidents pile up, consumer facing warnings have grown more blunt. One widely shared advisory told people that “ChatGPT users warned their private chats could be ‘sold for profit’” and stressed that the risk extended across multiple platforms, including tools from OpenAI, xAI and Meta AI. The piece, filed under “Home” and “News,” noted that it was “Published” at “10:54” “GMT” and framed the issue as a wake up call for anyone who assumed their prompts were off limits to commercialization, a framing captured in the alert that ChatGPT users warned their private chats could be sold for profit.
At the same time, legal and policy analysts have started to question whether existing privacy frameworks are equipped to handle this new category of data. One commentary argued that regulators should treat AI chat logs as especially sensitive, given their mix of behavioral, emotional and sometimes biometric information, and that companies should face higher burdens before they can repurpose them. Yet in practice, the burden still falls on individuals to read dense terms, hunt for toggles and avoid risky extensions or VPNs. The illusion of control persists in settings menus and marketing copy, even as the underlying systems treat every prompt as another asset to be captured and monetized.
Assume everything since July 2025 is on file
Perhaps the most sobering advice to emerge from recent investigations is also the simplest. One detailed report on AI data scraping urged users to “Assume any AI conversations you’ve had since July 2025 have been captured and shared with third parties,” warning that the combination of browser extensions, proxies, analytics tools and platform policies had effectively erased the boundary between private and shared. The same analysis cautioned that “Even if your” chats feel ephemeral, they are likely stored, copied and analyzed in ways you cannot see, a stark conclusion laid out in the section that begins with Assume and continues through “Even.”
I find that guidance harsh but realistic. In a world where ShinyHunters can breach The Mixpanel, where Urban VPN and BiScience can fold chats into “marketing analytics purposes,” where Koi can uncover sprawling scraping networks, and where courts like those overseen by Judge Wang can order “20 Million” conversations turned over, the safest mental model is that nothing you type into an AI is truly private. That does not mean abandoning these tools altogether, but it does mean treating them more like a public forum than a diary, and pushing regulators and companies alike to rebuild privacy protections from the ground up rather than treating them as an optional add on.
More from MorningOverview