A pseudonymous researcher who goes by cereblab pointed mitmproxy at xAI's Grok Build coding CLI, planted fake secrets in a repository, and watched the tool ship the whole thing to a Google Cloud bucket named grok-code-session-traces. Not the files the agent read. The whole repository, git history included, on a normal consumer login. On a 12 GB test repo the storage channel moved 5.10 GiB across dozens of 75 MB chunks, every one accepted with an HTTP 200, while the actual model conversation moved 192 KB. That is a ratio of roughly 27,800 to one between what Grok needed to answer and what left the machine.

The finding everyone quoted was the secret: a canary credential in a .env file, transmitted verbatim and unredacted. The finding that matters is quieter. cereblab turned off the setting labelled "Improve the model" and ran it again. Grok uploaded the entire repository anyway, and the server's own settings response still came back with trace_upload_enabled: true. The opt-out a user is handed governs whether xAI trains on your data. It never governed whether your code leaves your computer.

xAI's answer told you where the real problem sat. A day or so later the uploads stopped, via a server-side flag flipped silently, plus a new disable_codebase_upload option in the CLI. No advisory. No changelog line. As of 13 July there was still no public statement on how many repositories were collected, how long they are kept, or whether any of it gets deleted. The behaviour was fixed the way you fix something you would rather nobody had measured.

The pitch, stated fairly

xAI's position, near as I can reconstruct it from the policy rather than a statement it declined to make, is the ordinary one for cloud AI: a coding agent has to send your code somewhere to act on it, this is disclosed in the terms of service, and users can opt out of training. All three points are true, and cereblab concedes the first outright. An agent that edits your files and runs your commands from a datacentre cannot do that without your files reaching the datacentre. Nobody serious disputes that a cloud tool sends context.

So the honest question isn't whether Grok sends code, but what it does that a reasonable user would never predict from "cloud coding tool." Three things, on the wire. It sends a secrets file unredacted rather than skipping or masking it. It persists the code to a named long-term bucket rather than processing it and dropping it. And it does both regardless of the one privacy control the product actually surfaces. The gap between the pitch and the capture is the manner of it: the data moves in ways the consent screen does not describe and the opt-out cannot touch.

The control was pointed at the wrong layer

Here is the argument. The privacy furniture we built for AI was designed for the chat era, when the sensitive thing you exposed was your prompt. "Don't train on my conversations" was a sensible control because the conversation was the asset at risk. The whole vocabulary of AI privacy settled there: model-improvement toggles, private-chat modes, training opt-outs. Every one of them governs what the model learns from what you typed.

Agents changed the asset. When you run a coding CLI in a repository, the thing at risk is no longer a sentence you chose to send. It is the working directory: every tracked file, the full commit history, the config with the API keys in it, whether or not you or the agent ever open them. Grok's own numbers make the point starkly, because the codebase upload continued after the model turn had been rate-limited. The storage pipeline does not sit downstream of the conversation; it runs on its own schedule.

The controls did not move with the asset. A user reaching for "Improve the model" is trying to answer the question "will my code be kept and reused?" and is handed a switch that answers a different question, "will my typed data train the model?" The two were the same thing in the chat era. They came apart the moment the tool started reading your disk, and nobody rewired the settings page to match. This is why the silent fix is the tell: if the upload had been a considered, disclosed feature, xAI could have defended it. Instead it shipped a disable_codebase_upload flag, which is a confession that "codebase upload" was a thing happening that users had no switch for.

The strongest case against reading it this way

Take the sceptical case seriously, because it has real weight. This is one pseudonymous researcher, testing one version (grok 0.2.93), on one operating system, and xAI has confirmed none of it. Wire captures get misread all the time; a local disk cache gets mistaken for an upload, a proxy artefact for exfiltration. And the loudest headline riding on the report, that Grok uploaded a user's entire home directory, is exactly the kind of claim that inflates in retelling.

Each of those deserves an answer. On the misread: cereblab pre-empted it, and in a way that raises rather than lowers my trust. An earlier version of the analysis had inferred an upload from a queue that drained locally, concluded the large blobs were not exfiltrated, and then retracted that conclusion when a proper wire capture with a mitmproxy certificate showed 75 MB request bodies leaving the host. Cloning the captured git bundle recovered a file the agent had been explicitly told not to read, marker string intact, and the whole thing replicated on a second unrelated repository. Someone fabricating a scare does not publish their own retraction and hand you the repro commands.

On the home-directory claim: I would separate it out and set it aside. The pitch of that headline is "Grok stole everything on my machine." What the rigorous capture demonstrates is narrower and more precise: the working repository, its git history, and any secrets living inside it, persisted to xAI's bucket regardless of the training toggle. That is less than the panic and considerably worse than the reassurance, and it is the version I would stand behind.

And on the single-researcher worry, the best corroboration is not another blogger agreeing. It is xAI's own hands. You do not quietly ship a flag named disable_codebase_upload for a behaviour that was not uploading codebases. The fix is the confirmation. Independent write-ups from Crypto Briefing and the security-trade coverage converged on the same reconstruction, and none of them, as of writing, could point to an xAI statement on retention. Silence about deletion, after a silent fix, is its own data point.

Why the default drifts toward keeping your code

Step back from xAI specifically, because the more useful read is structural. In a market where labs are visibly starved for training data and locked in a distillation arms race, a coding CLI installed on working machines is sitting on the single most valuable corpus that exists: real, private, functioning code that has never been scraped, paired with the git history that shows how it got that way. The gravitational pull on any product manager is toward retention, because the data is worth more kept than dropped. That is less an accusation against one firm than a description of the incentive every agent vendor now operates inside, which is why the safe assumption is that the default keeps drifting toward upload unless something structural holds it back.

Which is what makes the location of the control the whole game. A training opt-out sits at the wrong layer to hold that pull back; it constrains what the model learns, not what the servers store. The control that would matter is one at the storage layer, and preferably one enforced by a client that provably cannot retain your repository even if the server asks. Almost nobody offers that, because it is the opposite of the incentive.

The bet

So here is the falsifiable version. The Grok episode resolves in one of two ways, and they are easy to tell apart. Either xAI, or a competitor making it a selling point, ships a coding agent whose codebase upload is off by default, disclosed in plain language on the setup screen, with a retention window you can read and a deletion path you can verify. Or the resolution stays what it is today: a server flag flipped without announcement, a CLI option most users will never find, and no answer on the repositories already banked.

If it is the first, the control will have finally moved to the layer the asset moved to a year ago. If it is the second, then "don't train on me" remains the only lever on offer for a class of tool where training was never the exposure that mattered. My read is that the second is where it settles unless a customer with leverage forces the first, because the data is too valuable and the silence has been too comfortable. I would like to be wrong. The way to prove me wrong is a delete button that works, and a vendor willing to say what it deleted.

Google Cloud

Company Info

Funding

Related News