Content monetization

51Degrees helps customers to monetize their content in two key ways, by identifying AI harvesters and making them pay for taking content, and by adding value to your inventory with rich data and analytics.

Talk to us

Make the AI scrapers pay

A free contract sets a default fee for each article AI uses beyond search, and four lines of robots.txt put it in front of every crawler that reads the file.

We write the file for free, and help you prove who took what. Search crawlers stay in for free, so readers still find you.

How it works. The Movement for an Open Web publishes a free Search Only Contract that any website can adopt. It allows search indexing and human, non-commercial use. Everything else needs a deal or pays you a default fee for each article, image, or video used.

However, a contract only pays when you can show who took what. 51Degrees helps you log every page request with the crawler's name and purpose, the network it came from, and the time.

AI companies that announce themselves are named for you, so their visits go straight onto an invoice. Scripts that hide their name show up in Human Confidence as Medium or Low, with the network behind them, so you can see how much is taken without a name on it.

You hold a dated record of the articles each named operator took, ready for an invoice. Non-payment can lead to enforcement through the County Court, because the contract is governed by the law of England and Wales.

UK publisher Trusted Reviews is already logging the bots that ignore its contract.

Increase the value of your inventory

By adding 51Degrees' rich data to your advertising inventory, you can increase the value of your traffic.

Human Confidence. Our Human Confidence tool helps you to filter out non-human traffic from your site, increasing the performance of your inventory and thus its long-term value to advertisers.
Device Detection and IP Intelligence. 51Degrees' Device Detection and IP Intelligence help you add rich and valuable data to your advertising traffic, from its location to the type and value of device being used. This can increase the value of inventory to advertisers, increasing your ad yields simply and effectively. The data can be added directly to Prebid.

3 ways to earn more from your content

Charge the AI scrapers

🔗

Set a fee for AI use of your articles

Set the terms. Four lines in robots.txt point crawlers at the contract before they read a page. Block the AI crawlers you never want, or let them in at the contract's price. Our free robots.txt service writes the rest of the file.

User-Agent: *
# Terms https://m4ow.uk/socw/2.txt
TDL: https://m4ow.uk/socw/2.txt
Allow: /

Who it serves. Publishers whose articles are used to train AI models or answer questions.

Talk to us about this

Pairs well with Evidence for the invoice, to show who took what once the price is set.

For engineersRecipe no. 24

Integration time. 10 minutes with the free robots.txt generator.

Ingredients, 3

Robots.txt

PlainText
AnnotatedText

Bots and Humans

CrawlerUsage

Hero ingredient. CrawlerUsage Use the crawler's purpose to allow free search and charge for other AI uses.

Allow the uses the contract permits for free, such as search
Decide whether each AI use is blocked or let in at the contract's price
Keep the Search Only Contract in, or add a link to your own terms
Publish the file at the root of your site as robots.txt

Chef's tip. Put the contract link in robots.txt, where crawlers read your terms before visiting a page.

Create a Resource Key Start free, then pick a plan

Evidence for the invoice

🔗

Record which AI company read each article, and when

Keep a record. Each visit from a named AI company is logged against the article it read, so the invoice can list the articles and the dates.

Who it serves. Publishers ready to invoice the AI companies that take their content.

Talk to us about this

Pairs well with Charge the AI scrapers, for the contract that sets the price of AI content use.

For engineersRecipe no. 25

Integration time. 30 minutes in any Pipeline API integration.

Ingredients, 7

Bots and Humans

IsCrawler
CrawlerName
CrawlerUsage
IsArtificialIntelligence
HumanConfidence

IP Intelligence

AsnName
IsHosted

Hero ingredient. CrawlerName Use the crawler name to group article visits by the company you plan to invoice.

Pass every page request through the Pipeline API
Log the crawler name, usage, page, network, and time
Log the HumanConfidence property to count visits that hide their name
Total the articles per named operator and send the invoice

Chef's tip. Start the log on the day the contract goes into robots.txt, so it covers every visit from then on.

Create a Resource Key Start free, then pick a plan

Prebid bidstream enrichment

🔗

Add device data to every bid request from Prebid

Add accurate device data, such as model, OS, and screen characteristics, to the bid requests from Prebid.js or Prebid Server. Publishers earn more, and advertisers get a better return on advertising spend (ROAS).

The Prebid integration. Our real-time data (RTD) module fills the device object and adds the 51Did as an EID on every request. Your adapters don't change.

pbjs.setConfig({
  realTimeData: {
    auctionDelay: 250,
    dataProviders: [{
      name: '51Degrees',
      waitForIt: true,
      params: { resourceKey: 'YOUR_RESOURCE_KEY' }
    }]
  }
});

Who it serves. Publishers, SSPs, and anyone running Prebid.js.

Talk to us about this

Pairs well with Bidstream enrichment, for more on the device data in each request. Use this recipe to add it through Prebid.

For engineersRecipe no. 12

Integration time. 20 minutes, one config block.

Ingredients, 15

Device Detection