14 September 2026
Heard In AI

Idea

Can Shapley values pay publishers for what agents actually use?

Parag Agrawal, whose company Parallel builds web search for AI agents, argues that when an agent visits a page instead of a person, advertising and subscriptions both stop working. His proposed replacement borrows an idea from cooperative game theory: estimate how much each source improved an agent's answer, and pay for that contribution. This idea page explains the argument, the mathematics behind it, and what remains unproven.

An idea page explains a concept, theory or proposal: where it comes from, what supports it, the objections and the open questions. We update it when new material changes the explanation. How our formats work

Parag Agrawal says he likes advertising. Not as a reader — he notes that most people hate ads — but as an engineer who built ad systems and later ran Twitter. On the Training Data podcast, he called ads "the reason that so much amazing content and technology is available for free to all of us." Google Search wouldn't be free without ads, he said. Neither would Twitter.

What makes advertising work, in his telling, is that it charges wildly different amounts for different moments of attention. Most search queries lose Google money; a few make up the difference. He called this "differential pricing," and said it is why ads became the dominant business model of the web.

That model rests on an assumption: a human being shows up, with limited attention, and can be persuaded to do something. Agrawal's company, Parallel, builds search infrastructure for AI agents — software that browses and reads on a person's behalf. Earlier in the same conversation, he recalled seeing Cloudflare report that AI traffic to websites had reached roughly the level of human traffic in page reads, with the caveat that the measurement includes crawlers and other machine activity. His question follows from that shift: if the reader is a program, what is the page worth, and to whom?

The bargain that breaks

Agrawal described the awkward position publishers are already in. A site owner pays to be found by humans — search optimization, distribution, all of it — and then, when an agent arrives instead, blocks it. "Which seems confusing and disconcordant," he said, since the agent is ultimately acting on behalf of a human. "You can't monetize that visit."

Subscriptions fail differently. A publisher might know that a thousand human visitors reliably yield twenty monthly subscribers. With agents, Agrawal said, that loop is unmeasured: you cannot tell which visits might eventually convert and which are "nameless agents" taking content and leaving nothing.

Cloudflare, which sits in front of a large share of web traffic, has taken a different route to the same problem. In a July 2026 announcement, it split automated access into three categories — search, agent and training — so site owners can permit discovery while restricting the other two, and said that from 15 September newly onboarded domains would by default block agent and training crawlers on pages carrying advertisements while still allowing search. The stated reasoning is that ads on a page signal a business that depends on human visits, while search crawlers can help deliver those visits. It is a controls-and-permissions answer. Agrawal's is a payments answer.

Why the existing deals don't scale

Before Parallel, Agrawal said, a content owner with valuable material had essentially one option: be big enough to negotiate a fixed-fee contract with an AI lab, typically bundling training rights, liability protection for training already done, and access at inference time — the moment an answer is actually produced. Most of the web cannot get such a deal. He called it "a very head phenomenon."

Even for those who can, he argued the structure is wrong. If inference volumes grow sevenfold in a year and sevenfold again, the deal size does not grow fiftyfold with them. "None of them, after signing a two-year deal, believes that their share isn't going to decline materially at renewal." A fixed price inside a market growing by orders of magnitude, in his view, cannot sustain the businesses producing the content.

So he set out the properties any replacement would need: differential pricing on two dimensions — how distinctive the content is, and how valuable the work being done with it — and a method that scales without bespoke deals. Above all, what he kept returning to was incentive alignment: a scheme both sides would want to join. Asked why model companies should pay anything at all, he said the answer was theirs to give, and offered his understanding: they want fresh data at inference time to show in products like ChatGPT or Claude, they want training data, and they want some liability protection for training already done. How they weigh those three, he said, he cannot be sure.

The missing-source thought experiment

Asked how you attribute credit when a simple query triggers ten searches that collapse into one paragraph, Agrawal answered with a thought experiment rather than a product.

Take one source out of the corpus. Run the agent again. The quality of the output drops. Now ask what it would take to recover that quality some other way — more compute, a better model. Suppose it costs a cent. "I could get the quality I lost by not having this source. My alternative was to throw more compute, a cent worth of compute, to get that same quality." The end result might differ in wording, but for the customer it is equally good. So the source, he said, "is worth close to a cent."

That is the intuition. The formalization is Shapley values.

What a Shapley value is

A Shapley value comes from cooperative game theory. Agrawal put it plainly: three people collaborate, the whole is bigger than the sum of the parts, and the question is how to divide the larger pie so that all three have an incentive to collaborate. The Shapley value is a mathematical answer to that division.

It is not computed by removing one participant once. It requires simulating the worlds where some subset collaborated without the others, and working back from those alternative realities to a share for each participant today. That is why Agrawal called the concept "not that useful" in most settings: in practice you cannot play out those worlds.

Machine learning is one place where you can. In model explanation, the same math treats input features as the players: hold a feature back, run the model, see how well it did. The SHAP library's introductory tutorial works through a housing-price example in which each feature's contribution helps bridge the gap between a baseline expected price and the specific prediction, with the contributions summing to that difference. Agrawal's claim is that web search allows the same trick: you can rerun an agent without a given URL, domain or group of domains and observe how it performs.

With one large condition. "If you're good at evals, if you're good at assessing quality, you can build that data by running various scenarios," he said. Everything downstream depends on being able to judge whether the answer got worse and by how much.

Estimates, not computations

The honest limit, by Agrawal's own account, is cost. Running the full Shapley computation in Parallel's context is "way more expensive than the amount of dollars we spend on the agent" — never mind the dollars you want to hand a publisher. Determining that a content owner is owed a dollar might consume several dollars of compute. "So it doesn't make any sense, except you can really well estimate them if you build the right kind of data and models around it."

Parallel's approach, then, is to generate that simulation data and train models to predict the values rather than calculate them. Agrawal says he has confidence in the estimates and that the underlying theory is sound: if there were perfect information symmetry, participants would want to collaborate.

His analogy for how such a system settles is the second-price auction in advertising, where bidders were told — "for better or worse," he said — that revealing their true bid would leave them paying less than it. Feedback loops eventually took over: today most advertisers measure return on investment and let automated bidding set the number instead of making up bids. He expects something similar here, with a Shapley foundation maximizing participation by both content owners and the AI systems seeking content.

The SHAP tutorial flags a subtlety worth keeping in view alongside the estimation problem. Attribution depends on how you define the value being divided and how you handle information that is absent; computation is hard in general. Explaining a model's probability output can produce a different picture from explaining its log-odds. Deciding what "quality" means, and what the agent's world looks like when a source is missing, shapes who gets paid.

Paying differently for different reads

What Agrawal thinks sells the idea to publishers is not the math but its consequences. Explaining Shapley values to a content owner "takes a moment," he said. The properties are easier: unique, differentiated data earns more. And the same page can be worth different amounts depending on who is reading it. "If a banker in an expensive job reads your data versus my retired dad reads your data, the banker ends up paying more for that read," he said, because that read is part of high-value work.

The product form of this is Index, which Parallel announced on 19 May 2026. It offers site owners visibility into which agent queries draw on their content, how often it is referenced and how distinctive its contribution is, with compensation based on estimated Shapley value at inference time. The announced partners span large publishers and data companies — The Atlantic, Fortune, Enigma, Fiscal AI, PR Newswire, RocketReach, Tracxn and ZoomInfo — alongside independent publications including Exponential View and Not Boring.

The name came last. Agrawal incorporated the company as Shapley Inc. — "Shapley.ai happened to be available," and Shapley.com was parked — while knowing it was a terrible name for a business-to-business product. For six to eight months his conference badges said Nuco or Stealth Company. Parallel came from a different image: publishing to two audiences at once, humans and agents, and a parallel web taking shape for the second.

What is claimed and what is not

The most striking numbers in the conversation are projections, not receipts. Agrawal's macro argument is that if large sums are spent on inference for knowledge work, allocating something like 2 to 10 percent of it to web data would be far bigger than all of today's web-data business models outside walled gardens such as Facebook and LinkedIn. And as agents on the web grow by an order of magnitude year on year, "by my calculations, we're 12 to 24 months from this math being able to give meaningful dollars for a very wide range of content owners on the web."

That is a forecast by the founder of the company selling the mechanism. It leaves several things open. The estimates are only as good as the evaluations behind them, and those evaluations are built by the same party making the payments. A publisher cannot easily verify a number derived from simulations it cannot run. And a system that pays more for distinctive data offers less for the ordinary reporting that much distinctive analysis depends on.

Agrawal's own framing of the stakes was about incentives rather than fairness: with traffic to sites like Stack Overflow falling, the question is what keeps people publishing at all. His answer is that a source should be paid roughly what it would cost to do without it. Whether anyone can compute that cheaply enough, and credibly enough, to matter is the part still being tested.

Share this idea

Go to the original

Sources & further reading

  1. 01
  2. 02
  3. 03

From the conversation

Podcast episodes

Page history

How this idea page has changed

Tags

Parallel sold patient search agents before it could afford a web index

On Training Data, Parag Agrawal explains how his company Parallel entered web search without first building a giant index: it launched a search agent that crawled after a request arrived, replaced outsourced human data collection for insurance, sales and finance customers, and treated the index as a latency optimization to be grown later. He describes the agent-specific architecture behind it, the 200-millisecond Turbo mode Parallel announced in July, and a Google Cloud deal that puts Parallel Search beside Google Search as a grounding option.

7 min read

Parag Agrawal expects a web that calls the agent when something changes

On Training Data, Parallel Web Systems founder Parag Agrawal traces how agents multiply web searches — from a weekly credit-risk review across 10,000 small businesses to the meeting-prep agents that run hundreds of searches before his own calls — and forecasts a web that, in a couple of years, tells agents when something worth acting on has changed.

5 min read

Would she pay the real price? Zitron's test for AI adoption

On The Diary of a CEO, critic Ed Zitron praises a chatbot for reading a troubleshooting log and for helping fix his son's Minecraft mod, then argues that neither is worth a trillion dollars. The host counters with his fiancée's one-woman business and his chief of staff's inbox. The argument turns on tokens, subscription rate limits and who is paying the real bill.

7 min read

Why Graylin says distillation cannot explain all of China’s AI gains

Asked about allegations that Chinese labs extracted capabilities from Claude, Alvin Graylin argued that access to another model’s answers cannot explain every engineering advance. The Moonshots exchange turned on three distinctions: legitimate distillation versus prohibited extraction, query bills versus development costs, and learning from outputs versus improving the machinery behind them.

6 min read