Why Stack Overflow and Cloudflare launched a pay-per-crawl model

Why Stack Overflow and Cloudflare launched a pay-per-crawl model

In this episode of Leaders of Code, Stack Overflow’s Janice Manningham and Josh Zhang sit down with Cloudflare VP Will Allen to discuss the innovative pay-per-crawl model co-launched by their organizations. They explore how the rise of AI has disrupted the traditional “open versus block” internet model, creating a need for platforms to protect their content and data from commercial exploitation while maintaining community access.


The discussion also:

  • Explores the future of the bot ecosystem, emphasizing the importance of putting publishers back in the driver’s seat to decide how their content is accessed and monetized.
  • Explains the technical implementation of the pay-per-crawl system, which uses Cloudflare’s bot categorization and WAF rules to serve a 402 “Payment Required” message to specific crawlers.
  • Highlights the strategic value of data licensing, comparing comprehensive enterprise contracts with the more flexible, programmatic pay-per-use access enabled by the new model.

Notes

See Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.

Jaksot(913)

Data is the new oil, and your database is the only way to extract it

Data is the new oil, and your database is the only way to extract it

Ryan sits down with Shireesh Thota, CVP of Azure Databases at Microsoft, to discuss the evolution of databases at Microsoft; Azure’s comprehensive portfolio that includes SQL Server, CosmosDB, and Pos...

17 Helmi 40min

Even your voice is a data problem

Even your voice is a data problem

Recorded last December at AWS re:Invent, Ryan welcomes CEO and co-founder of Deepgram, Scott Stephenson, for a conversation on advancing voice AI technology. They cover how Deepgram is improving speec...

13 Helmi 35min

The logos, ethos, and pathos of your LLMs

The logos, ethos, and pathos of your LLMs

Ryan is joined by Professor Tom Griffiths, the head of Princeton University’s AI Lab, to dive into findings from his new book The Laws of Thought, which explores the history of the philosophy, mathema...

10 Helmi 34min

AI attention span so good it shouldn’t be legal

AI attention span so good it shouldn’t be legal

We have another two-for-one special this week, with two more interviews from the floor of re:Invent. First, Ryan welcomes Pathway CEO Zuzanna Stamirowska and CCO Victor Szczerba to dive into their dev...

6 Helmi 30min

Generating text with diffusion (and ROI with LLMs)

Generating text with diffusion (and ROI with LLMs)

Two guests for the price of one! This episode has two interviews recorded at AWS re:Invent back in December. In part 1, Ryan chats with the co-founder and CEO of Inception, Stefano Ermon, about diffus...

3 Helmi 30min

Wanna see a CSS magic trick?

Wanna see a CSS magic trick?

Ryan is joined by Chris Coyier, founder of CSS Tricks and CodePen, to talk all about what the state of the art of CSS is today, including new features like variables and scroll-driven animations. They...

30 Tammi 38min

Spy vs spy at scale

Spy vs spy at scale

Ryan welcomes Anthony Vinci, former senior intelligence officer and author of The Fourth Intelligence Revolution, to explore AI’s evolving role in intelligence in places like translation and image ana...

27 Tammi 35min

Suosittua kategoriassa Liike-elämä ja talous

sijotuskasti
mimmit-sijoittaa
psykopodiaa-podcast
rss-rahapodi
rss-draivi
rss-rahamania
ostan-asuntoja-podcast
rss-sami-miettinen-neuvottelija
pomojen-suusta
inderespodi
rss-seuraava-potilas
herrasmieshakkerit
taloudellinen-mielenrauha
oppimisen-psykologia
rss-h-asselmoilanen
rss-paasipodi
rss-inderes
asuntoasiaa-paivakirjat
rss-lahtijat
rss-bisnesta-bebeja