Publishers cannot escape Google AI training without sacrificing Search

It seems publishers are caught between a rock and a hard place when it comes to Google and AI. New testimony in the ongoing antitrust trial reveals a stark choice: let Google use your website’s content to train its Search AI, or disappear from Google Search results altogether. This really puts website owners in a bind.

The revelation came from Eli Collins, a VP at Google DeepMind, the company’s AI lab. As reported by Bloomberg, Collins testified that while publishers can use a specific control to opt out of having their content used for training Google DeepMind’s general AI models, this opt-out doesn’t apply to AI features within Google Search itself, like the AI Overviews that summarize answers at the top of results.

So, what does this mean? Google confirmed that the only way for a publisher to prevent their content from being used to train these Search-specific AI tools is to use the robots.txt web standard to block Google’s crawlers entirely. The catch? Doing so means your site won’t be indexed and won’t show up in regular Google Search results either. Ouch. For most websites, losing Google Search traffic is simply not an option.

This feels like a significant power play. Publishers are already worried about AI summaries potentially cannibalizing their traffic. We’ve seen concerns before, like how Google’s recent Search experiment could potentially destroy food recipe sites by providing answers directly on the results page. Now, they learn the very content generating that traffic is being used to power the AI features they fear, and opting out means sacrificing their main source of visibility.

This situation arises while Google is under scrutiny in a major antitrust case brought by the Department of Justice. The DOJ argues Google illegally monopolized the search market, potentially impacting competitors and even partners like Mozilla, which faces a massive revenue hit if the DOJ wins its case. The testimony highlighted the sheer scale of data involved, with one document suggesting half of a 160 billion token dataset was initially filtered out due to publisher opt-outs from DeepMind training, leaving a massive amount still accessible for Search AI.

All this makes it clear that AI is set to become deeply embedded in core search functions, a trend seen across platforms as both Google and Reddit are now baking AI directly into their core search boxes. For publishers, the message seems clear: resisting Google’s AI push might mean risking relevance in the world’s biggest search engine. It’s a tough spot, and one that highlights the immense power Google holds over the flow of information online.