Tag: AI

There's a version of this story where Anthropic was trying to protect itself from large-scale model theft. There's another where one of the AI industry's biggest privacy advocates quietly crossed a line its own users never expected. What makes this headline important isn't just that hidden tracking code existed. It's that the company behind it was Anthropic. Just months ago, Anthropic publicly refused to let the Trump administration use Claude to surveil American users. The company defended that position in court, arguing that AI companies shouldn't become tools for government surveillance. That stance became part of Anthropic's identity. Then came a very different decision. In March, Anthropic quietly added hidden tracking markers to Claude Code that flagged users' timezones, proxy connections, and potential ties to Chinese AI labs. The code remained unnoticed until security researcher Thereallo discovered it last week. After the discovery went public, an Anthropic engineer confirmed it on X, described it as an "experiment" intended to combat account abuse and model distillation, and said the company had already planned to remove it. The tracker was taken down shortly afterward. The bigger question isn't whether Anthropic had a reason. It's whether a company that built its reputation on privacy can afford to hide surveillance from the very developers it asks to trust its tools. Table of ContentsWhat the tracker actually didWhat Anthropic Was Actually Trying to StopWhy the Method Mattered More Than the IntentionWhat This Actually Costs AnthropicThe Lab That Asked for Trust What the tracker actually did To be clear about what this was and wasn't: the code wasn't stealing passwords or reading source files. It was using prompt steganography, hiding markers inside system prompts that most users would never notice, to quietly flag certain signals back to Anthropic. Timezone data. Proxy usage. Patterns that might indicate a connection to Chinese AI labs Anthropic has accused of running distillation attacks against Claude. It wasn't malicious in intent. It was, by Anthropic's own account, an abuse-prevention measure aimed at unauthorized resellers and labs that were allegedly prompting Claude millions of times to train competing models. The engineer who confirmed it said stronger mitigations had since been built, which is why removal was already planned. None of that changes the core problem: users didn't know it was there. A developer tool that can read your code, run commands, push commits, and install packages was quietly sending information home without telling anyone. Thereallo put it wellm coding agents already operate close to a line most users are uncomfortable thinking about. Hiding telemetry in the system prompt doesn't just cross that line. It makes every other privacy claim harder to believe going forward. What Anthropic Was Actually Trying to Stop The distillation threat Anthropic keeps raising isn't hypothetical. Researchers at Peking University and the Chinese Academy of Sciences published methods in February for detecting distillation in large language models and found that most major Chinese models showed substantial evidence of it, primarily from American ones. One of Alibaba's Qwen models reportedly mimicked Claude closely enough that in some tests it would identify itself as Claude when pushed. Anthropic has accused Chinese labs of running what it calls the largest distillation attack ever on Claude, allegedly using millions of queries to rapidly advance competing models. That kind of attack doesn't just cost compute, it potentially hands a competitor the capability gap Anthropic spent billions closing. On top of that, unauthorized resellers have been selling access to Claude's free tier for $1 a month and Pro subscriptions for as little as $12, cutting Anthropic out of revenue on its own product. The tracker was built to catch exactly this: timezone patterns, proxy usage, connection signatures that might indicate someone wasn't a normal developer but a lab running automated queries at scale. Whether it worked is a separate question. The intent wasn't to watch ordinary users. The problem is that ordinary users were watched anyway, and never told. You May Like: Open Source AI Coding Agents That Don’t Need a Subscription Why the Method Mattered More Than the Intention Thereallo's post wasn't arguing that Anthropic had no reason to act. It was arguing that the way Anthropic chose to act was the problem. Hiding detection logic inside system prompts using steganographic markers, encoding signals in ways most users would never notice is a choice. Anthropic could have disclosed the behavior in release notes. It could have added an explicit telemetry field with documentation. It could have put a line in the terms of service. Any of those paths would have left users informed. None of them happened. For a developer tool that already operates with significant access including reading code, running terminal commands, installing packages, pushing commits, the baseline expectation of transparency is higher, not lower. Thereallo flagged that Claude Code "already lives on the wrong side of a scary boundary." Hiding monitoring inside that tool, even for legitimate reasons, pushed further past it. The steganography detail is also worth sitting with. This wasn't a standard analytics call that a moderately technical user might catch in network traffic. It was specifically designed not to be noticed. That design choice is harder to explain as an oversight. What This Actually Costs Anthropic The tracker is gone. The damage isn't. Alibaba moved fastest. Within days of the disclosure, the company banned employees from using Claude Code for work entirely, citing "back-door risks" in an internal memo. For Anthropic, losing Alibaba's developer base isn't just a China problem, it's a signal about how enterprise customers respond when trust breaks, and enterprise customers talk to each other. The broader cost is harder to measure but more significant. Anthropic's competitive position in the frontier model race depends partly on being the lab that takes safety and user trust seriously. That positioning has real value: it shapes who builds on the platform, who advocates for it, and who defends it when the next controversy arrives. Thereallo put it plainly: "Hiding the signal in the system prompt makes every other privacy claim harder to believe." Anthropic is simultaneously asking developers to trust Claude Code with access to their codebases, asking regulators to trust it with AI policy, and asking the public to trust it as the responsible actor in the frontier race. Those asks get harder after this, not because the tracking was uniquely sinister, but because the gap between the company's stated values and its revealed behavior is now documented and public. The engineer who confirmed the tracker said Anthropic had been meaning to remove it for a while. That may be true. It would have been easier to believe before anyone had to find it themselves. The Lab That Asked for Trust Anthropic's entire pitch was: to users, to regulators, to the public, is that it's the AI company that takes these things seriously. Capable and trustworthy. That distinction is the product as much as the model is. Hidden tracking code doesn't end that pitch. But it makes it harder to deliver without footnotes.

Tag: AI

Most Read

Anthropic Secretly Tracked Claude Code Users. Then Called It an “Experiment.”

Leanstral 1.5: Mistral’s AI Built to Prove Math Ended Up Finding Real Software Bugs

Meetily: Privacy-First AI Meeting Assistant for Windows, macOS & Linux

7 Open Source AI Coding Agents That Don’t Need a Subscription