Why AI Still Has No System for Paying for the Content It Uses

The AI industry has solved problems that looked impossible ten years ago. Models that reason. Systems that retrieve, synthesize, and answer in about a second. Infrastructure that runs all of it at planetary scale.

There is one problem it has not solved, and it is not a hard one. It is a plumbing problem.

When an AI system uses your article to answer someone’s question, there is no standard way for it to read your terms, no standard way to pay you, and no record that the use happened at all. Enormous capital has gone into building the intelligence. Almost none has gone into the part where the people who produced the source material get paid.

The AI content payment gap is the absence of shared infrastructure for licensing and compensating content used by AI systems. It has three parts: no widely adopted standard for expressing rights in machine-readable form, no transaction rail for settling payment at the speed AI operates, and no receipt proving a use occurred. Publishers can block, sue, or sign a one-off deal. None of those is a system, and none of them scales.

This post covers why the gap is structural rather than legal, what the standards bodies have and have not built, why usage measurement is the piece everything else is waiting on, and what closing the gap will take.

Three Things Have to Exist Before Anyone Gets Paid

Any market where one party uses another party’s property at scale rests on three pieces of infrastructure, and they have to arrive roughly in order.

First, a rights standard: a machine-readable way for the owner to state terms that the user can read without picking up a phone. Second, a transaction rail: a way to settle automatically, at the volume and speed of the use itself. Third, a receipt: a record both sides can point to afterward showing what was used, by whom, and under what terms.

Music has all three. So does advertising. So does electricity. The web has never had any of them, and for thirty years it did not need them, because the web ran on a different bargain. You gave the content away and the link brought a reader back. The exchange was implicit, it settled itself, and it worked well enough that nobody built the machinery.

AI retrieval broke that bargain faster than anyone built a replacement. The link came back less often. The content kept going out. Nothing in the stack was designed to notice, price, or record any of it.

It is worth being precise about the failure here, because the popular version gets it wrong. This is not a story about AI companies uniformly refusing to pay. Several have paid, some of them substantially. It is a story about there being no mechanism through which paying could become normal. Companies willing to license at scale have nowhere to do it. Publishers willing to license at scale have nothing to license through.

The Only Tool Publishers Had Was a Sign on the Door

For most of the web’s history, a publisher’s entire vocabulary for talking to automated visitors was robots.txt.

It is a good specification for what it does, and it is unusually candid about what it does not do. The IETF standardized it in 2022 as RFC 9309, the Robots Exclusion Protocol. The document describes rules that crawlers are asked to honor rather than required to obey, and it says outright that those rules do not amount to a form of access authorization. The security section goes further, noting that the protocol is no substitute for real access controls and that listing a path in the file makes that path publicly discoverable.

So the tool publishers were handed gives them two words, yes and no, applied to a path. There is no vocabulary for yes, for a fee. No vocabulary for yes to search indexing, no to model training. No way to attach a price. And nothing anywhere that records whether the sign was ever read.

A sign on a door is a reasonable thing to have. It is not a lock, a cash register, or a ledger. Publishers have been asked to run a licensing business with nothing else.

A Standard Is Arriving, and It Stops Short of the Money

That is finally changing, and the change deserves credit.

Really Simple Licensing, published as a 1.0 recommendation in December 2025, gives publishers the vocabulary robots.txt never had. The RSL 1.0 specification builds on RSS and on the Robots Exclusion Protocol, and it lets a publisher declare, in machine-readable form, which uses are permitted, which are prohibited, and what compensation applies. Its payment vocabulary covers per-crawl and per-use terms alongside subscription and one-time purchase, which is the first time the open web has had a standard way to say pay per crawl and mean something specific by it. It even defines reporting requirements, including event-level telemetry about how content was used.

That is real progress, and it is why NextNet AI built for compatibility with it rather than around it.

Read the specification closely, though, and the boundary shows up. The parts that make licensing declarative are required. The parts that make licensing enforceable are optional. The license-token protocol, the crawler authorization protocol that verifies those tokens at request time, and the encryption layer that protects the asset itself are all explicitly optional extensions. A publisher can adopt the standard, publish flawless terms, and still have nothing in the loop that checks whether anyone honored them.

Terms are a declaration. Payment is a system.

The standards process draws the same line, and draws it on purpose. The IETF chartered a working group on AI preferences to standardize how these signals are expressed and attached to content, with its first specifications due to the steering group in August 2026. The charter also lists what the group will not cover, and that list is the more useful read for a publisher: technical enforcement of preferences, protocols for authenticating or authorizing crawlers, registries of content preferences, and auditing or transparency measures.

Notice the word the charter uses throughout. Preferences. Not permissions, not terms, not contracts. The internet’s standards body is building a careful, well-designed way to state what you would prefer, and it has ruled the machinery for enforcing that preference out of scope deliberately, because enforcement is not a protocol problem.

That is the honest state of play. The vocabulary layer is arriving on schedule. The transaction layer is nobody’s charter.

Nobody Can Price What Nobody Measures

Here is the piece everything else is waiting on.

Before the electric meter, selling electricity meant selling a flat subscription for a lamp. Utilities charged per fixture per month because they had no way to know how much anyone used, so pricing was a guess wearing the costume of a rate. The meter did not simply enable billing. It created the market, because once both sides could see consumption, they could argue about price using the same numbers.

Publishers today are living in the pre-meter era.

Ask a publisher how many times their content was retrieved to ground an AI answer last month and the honest answer is that they do not know. Server logs show crawler traffic, which tells you something was fetched, not what it was used for or whether the answer it produced ever named you. They reveal nothing about retrieval by systems that already hold a copy. They cannot distinguish the article that was the load-bearing source in an answer from the one that appeared as the sixth link in a footer.

That blindness sets the price of every conversation that follows. A publisher walking into a licensing negotiation with no usage data is not negotiating. They are accepting an offer. It is a large part of why flat fees dominate the deals that exist, why those deals cluster among the biggest publishers, and why renewal starts from the same position of ignorance as the first signature did.

The meter comes first. Usage-based pricing, credible renewal leverage, an evidence record that survives a dispute: all of it is downstream of the same missing measurement.

Why the Three Available Options Are Not a System

Publishers have three moves available today. Each one is rational. None of them scales.

Block. Refusing AI crawlers at the network edge is the one lever that works immediately and requires nobody’s cooperation. It also removes you from the fastest-growing discovery surface in media, and it is increasingly partial: blocking a crawler does not remove content that was already ingested, and it does not touch systems that reach your material through other paths. You give up presence and you still do not get paid. For most publishers the goal is not absence from AI answers but standing inside them, which blocking forecloses.

Litigate. Court is where the biggest disputes have gone, and those cases matter for reasons well beyond money. They are also slow, expensive, and realistically available to a very small number of organizations. More to the point, litigation is a remedy for a use that already happened. Winning does not build a way to be paid for the next one.

Sign a deal. Bilateral licensing works, and where it exists it has been meaningful for the publishers involved. But look at the arithmetic. There are more than fifty AI products actively retrieving web content to ground their answers, and that number is climbing rather than consolidating. Bilateral negotiation between every publisher and every AI product describes a quantity of contracts nobody is going to write. The deals that exist cluster among publishers large enough to have both leverage and a legal budget, which leaves most of the industry with no path at all.

Three doors, all opening onto the same room. The industry keeps arguing about who ought to pay while the reason nobody can pay at scale sits one layer down, largely unaddressed.

What Closing the Gap Will Take

Everything above points in one direction. The gap is infrastructure, so the fix is infrastructure. Four things have to be true, and they have to be true together.

Rights have to travel with the content. Terms attached to a domain are too coarse, because the answer to whether AI can use this is different for a wire story, an original investigation, a photo, and a twenty-year archive. Rights belong on the object, so the terms move with the work wherever it goes.

Policy has to be checked at request time. A declaration nobody validates is a preference. Enforcement means the incoming request is evaluated against the publisher’s policy before any content is served, which is why the interesting engineering happens at the retrieval interface rather than in a file at the root of the domain.

Retrieval has to be a transaction, not a crawl. When an authorized system requests licensed content through a controlled interface, the exchange has two parties who both know what was agreed. An open crawl has one party who knows and one who finds out later, if at all.

Every use has to write a receipt. This is the meter. A signed, replayable record of what was retrieved, by whom, for what purpose, and under what terms turns usage into something a publisher can bill against, price from, and produce in a dispute. It is also, not incidentally, what the largest copyright fights are substantively arguing about.

That combination is what SAIL, the Standardized Agentic Intelligence Ledger, is built to provide, and it is why we describe it as a transaction, receipt, and market intelligence layer rather than as a licensing product. Content becomes rights-managed objects carrying their own terms. Policy is enforced on every incoming request. Authorized systems retrieve through a controlled interface instead of an open crawl. Every use lands in a verifiable ledger. It is designed for compatibility with the emerging machine-readable rights standards rather than as a replacement for them, because the vocabulary layer is not the missing piece. You can read how it works on our publisher page.

None of this is a promise that payment becomes universal on a date certain, and anyone offering you that date is guessing. The direction is what is knowable: the systems that get paid will be the ones that can prove what was used.

Frequently Asked Questions

Why doesn’t AI pay for content?

Mostly because there is no mechanism through which paying at scale is possible, not because AI companies uniformly refuse. Compensation requires three pieces of infrastructure the web has never had: machine-readable rights that travel with the content, a transaction rail that settles at machine speed, and a receipt proving a use occurred. Some AI companies have signed licensing agreements, but those are negotiated one at a time and are realistically available to a small number of large publishers. Without shared infrastructure, paying stays a bespoke project instead of becoming the default.

Is robots.txt enough to control how AI uses my content?

No, and the specification itself says so. RFC 9309, the IETF standard for the Robots Exclusion Protocol, describes rules that crawlers are asked to honor rather than required to obey, and states that those rules are not a form of access authorization. In practice robots.txt gives a publisher a yes or a no on a path. It cannot express a price, cannot distinguish search indexing from model training, and produces no record of whether anything was read or respected.

Does the RSL standard solve the payment problem?

It solves part of it. RSL 1.0, published in December 2025, gives publishers a machine-readable vocabulary for licensing terms, including per-crawl and per-use payment types and reporting requirements, which is a substantial advance on yes or no. But the specification makes its enforcement protocols optional extensions. A publisher can declare terms correctly and still have nothing that verifies compliance or records a use. The vocabulary layer is arriving. The transaction and receipt layers are a separate build.

What is pay per crawl?

Pay per crawl is a licensing model in which an AI company or automated agent compensates a publisher each time it fetches content, rather than paying a flat fee for open access. It appears as a defined payment type in the RSL standard alongside per-use pricing, which charges when content contributes to an AI-generated output. Both models rest on the same prerequisite: a reliable count of what was fetched or used, which is exactly what most publishers currently cannot produce.

Should publishers wait for the standards to settle before doing anything?

Waiting carries a cost, because content used during the wait is not compensated retroactively. The more useful posture is twofold: express your rights position in machine-readable form now, and separately get visibility into how AI systems are using your content today, since usage data is the input to every licensing conversation that follows. Our guide to reading which sources AI trusts in your coverage area is a reasonable starting point for the second half. Standards will keep evolving. The measurement problem is worth solving before they do.

The Meter Comes Before the Market

Every technology that eventually paid content owners went through some version of this. Radio played records for years before anyone counted plays. Cable retransmitted broadcast signals long before retransmission consent existed. In each case the argument about payment stayed unproductive until somebody built the thing that measured the use, and then the argument turned into a negotiation.

AI is at the same stage, with one difference. The volume is higher and the pace is faster, so the cost of another year without a meter is larger than it was for any of them.

The gap is not going to close because the argument gets louder. It closes when the infrastructure exists, and the infrastructure is buildable now.

If you want to see what setting your own terms and getting a receipt for every use looks like for your catalog, that is what SAIL was built to do.