Over ⅓ leading Indian news publishers block OpenAI as fair-share battle deepens

For nearly two decades, digital publishing operated on an uneasy but functional bargain. Search engines crawled news websites, indexed their journalism and sent users back to the original source. Publishers, in turn, monetised that traffic through advertising, subscriptions, branded content and audience data.

Generative AI has begun to disturb that compact.

AI assistants can now answer questions, summarise news developments, explain policy changes, interpret market events and produce context around public issues without always requiring users to visit the original publisher. For news companies, this is not merely another platform disruption. 

It is a deeper challenge to the economics of journalism: if AI platforms can use news content to train models, inform answers and build commercial products, what happens to the publishers who paid reporters, editors, photographers, lawyers and fact-checkers to create that information in the first place? That question is now moving from boardrooms to robots.txt files.

More than 30% of leading Indian news publishers have blocked OpenAI’s crawlers, according to the Digital News Publishers Association. The move is aimed at restricting OpenAI’s bots from accessing publisher content for training and other AI-led use cases without a commercial arrangement.

Globally, the pushback has been sharper. By the end of 2025, nearly two third of top news websites globally have blocked OpenAI’s crawlers, according to the Reuters Institute for the Study of Journalism. In the US, the proportion was significantly higher. This includes prominent publishers and news organisations, including The New York Times, CNN, Reuters, the Australian Broadcasting Corporation. 

The message from publishers is clear: journalism cannot become free input material for AI platforms while the commercial value accrues elsewhere.

From traffic loss to fair-share fight

The publisher's anxiety is being sharpened by a global traffic squeeze, e4m reported last month. This is why publishers argue that the issue is no longer only about copyright. It is also about market design.

Read earlier report: Why is news traffic falling across the globe?

In India, the dispute has already reached the Delhi High Court. ANI has sued OpenAI, alleging that the company used its published content without permission to train ChatGPT and provide responses to users. OpenAI has denied wrongdoing and has maintained that its models are built using publicly available data in a manner protected by fair-use and related principles. The verdict in the case is awaited. 

“For publishers, the concern is that news content may be used by AI platforms in India without a licensing framework, even as similar commercial arrangements have been signed with several publishers globally,” says Pradeep Gairola, Chief Digital Business Officer at The Hindu. 

Moreover, OpenAI has entered into partnerships with global media organisations such as Associated Press, Financial Times, Axel Springer, Le Monde, Prisa Media, Dotdash Meredith, News Corp, Condé Nast, The Atlantic, TIME and Hearst. Indian publishers argue that a comparable fair-share structure should emerge locally if their journalism is being used to build, improve or commercially support AI products.

Is traffic the real success metric for news publishers? Read more here

“The debate has become more urgent as AI platforms begin to develop advertising models of their own. OpenAI has started testing ads in ChatGPT in select markets, positioning advertising as a way to support broader access to its free and lower-cost tiers. For publishers, this changes the nature of the dispute,” says a DNPA member.  

This is the heart of the fair-share argument. Publishers are not saying AI should not exist. Many are already using AI for newsroom productivity, translation, video packaging, archive search, analytics and content workflows. Their objection is to a one-way value exchange in which journalism improves AI products while publishers absorb the cost of reporting and face declining traffic and ad revenue.

Gairola frames the issue as a question of scarcity and trust. “The more synthetic content floods the web, the more valuable trusted original knowledge becomes. Scarcity is not being destroyed by AI. Scarcity is being manufactured by it.”

According to him, the market has already started recognising this shift. In 2024, Google began penalising low-quality, scaled and unoriginal content, signalling that the correction had begun. For publishers, he said, the real question is whether they will position themselves to capture that correction — or watch someone else do it.

If AI makes average content abundant, credible journalism may become scarce. And scarcity, if publishers organise themselves well, can be priced.

Blocking is leverage, but not a complete answer

Crawler blocking, however, is not a complete solution.

Technically, a robots.txt block tells a crawler not to access a website. It is forward-looking. It does not remove content that may already have been collected before the block was put in place. It does not automatically apply to every AI company. A publisher that blocks OpenAI’s GPTBot has not necessarily blocked Google, Anthropic, Perplexity or the long tail of other AI crawlers. Each crawler has to be identified and addressed separately.

There is also an enforcement problem. Robots.txt is a convention, not a lock. It relies on crawlers respecting publisher instructions. Well-behaved crawlers may comply, but the mechanism itself does not prevent all forms of scraping or content reuse.

That creates a difficult trade-off.

Publishers rewrite playbook amid traffic loss. Read here why

For subscription-led publishers with valuable archives and strong direct brands, blocking may be a defensible negotiating tactic. For breaking-news organisations, the calculation is more complex because their value to AI platforms often lies in fresh, time-sensitive reporting. For digital-first publishers dependent on reach, total exclusion from AI answer engines could mean losing share of voice to rivals that remain accessible or have licensing arrangements.

Pricing the content

The next phase may resemble a wholesale content market. Publishers may price archives differently from real-time feeds. They may charge separately for summaries, citation rights, display rights, training access and commercial use in AI products. Premium publishers may offer verified datasets, fact-checking layers or domain-specific news feeds. Regional publishers may find value in language, local reporting and ground-level information that global AI systems cannot easily replicate.

Several firms are building AI-bot firewalls for publishers, allowing them to move beyond simple crawler blocks. Under such models, bots can be granted paid access to content based on its value, usage type and commercial purpose — whether for training, real-time retrieval, summaries or display inside AI products. 

A few publishers in India have already started experimenting with such AI payment walls, allowing crawler access only under defined commercial terms rather than offering unrestricted entry to their content.

 But the Indian market faces a challenge. Publishers remain fragmented, and bargaining power varies sharply by language, scale, category and brand trust, a leading publisher points out.