Why Stack Overflow and Cloudflare launched a pay-per-crawl model

Why Stack Overflow and Cloudflare launched a pay-per-crawl model

In this episode of Leaders of Code, Stack Overflow’s Janice Manningham and Josh Zhang sit down with Cloudflare VP Will Allen to discuss the innovative pay-per-crawl model co-launched by their organizations. They explore how the rise of AI has disrupted the traditional “open versus block” internet model, creating a need for platforms to protect their content and data from commercial exploitation while maintaining community access.


The discussion also:

  • Explores the future of the bot ecosystem, emphasizing the importance of putting publishers back in the driver’s seat to decide how their content is accessed and monetized.
  • Explains the technical implementation of the pay-per-crawl system, which uses Cloudflare’s bot categorization and WAF rules to serve a 402 “Payment Required” message to specific crawlers.
  • Highlights the strategic value of data licensing, comparing comprehensive enterprise contracts with the more flexible, programmatic pay-per-use access enabled by the new model.

Notes

See Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.

Denne episoden er hentet fra en åpen RSS-feed og er ikke publisert av Podme. Den kan derfor inneholde annonser.

Episoder(969)

Your MVP doesn’t need a Kubernetes cluster

Your MVP doesn’t need a Kubernetes cluster

Ryan welcomes Anurag Goel, CEO and co-founder of Render, to discuss why most startups shouldn’t start by managing their Kubernetes and cloud infrastructure. They discuss how frustration with spinning ...

4 Aug 31min

What happens to the internet when robots act like humans?

What happens to the internet when robots act like humans?

Ryan welcomes WPEngine CTO Ramadass Prabakar to the show to chat about what happens—and what we should do—when agents start acting like humans online, how our internet is evolving to serve both human ...

31 Jul 30min

You need reliable AI context for your site reliability

You need reliable AI context for your site reliability

Ryan is joined by Asaf Savich, Komodor’s AI Engineering Group Manager, to discuss why modern reliability work requires navigating massive cross-service context, what good context engineering actually ...

28 Jul 26min

Partnerships can keep open source sustainable

Partnerships can keep open source sustainable

Ryan welcomes VoidZero’s Evan You and Cloudflare’s Dane Knecht back to the show to discuss Cloudflare’s recent acquisition of VoidZero and what it means for JavaScript development, how partnerships li...

24 Jul 26min

The future of development is full-stack

The future of development is full-stack

Live from Snowflake Summit, Ryan talks with Snowflake’s Head of Developer Experience Umesh Unnikrishnan about the industry-wide shift from “vibe coding” for quick prototypes to agentic engineering for...

21 Jul 25min

Developers who move fast still need to do it together

Developers who move fast still need to do it together

At MS Build, Ryan is joined by Cassidy Williams, Senior Director of Developer Advocacy at GitHub and former Stack Overflow Podcast host, to discuss how agentic coding is shifting dev work towards high...

17 Jul 28min

Your AI is only as responsible as you are

Your AI is only as responsible as you are

Recorded at Microsoft Build, Ryan welcomes Sarah Bird, Microsoft’s Chief Product Officer for Responsible AI, about how we can build and use AI responsibly with the NIST approach, why most irresponsibl...

14 Jul 28min

Building more than just an agent harness

Building more than just an agent harness

Live from Microsoft Build, Ryan is joined by Jay Parikh, Microsoft’s VP of AI Core, for a conversation on what enterprises need to build, deploy, and run AI agents at scale with demonstrable ROI; how ...

10 Jul 31min

Populært innen Business og økonomi

stopp-verden
rss-penger-polser-og-politikk
dine-penger-pengeradet
e24-podden
lydartikler-fra-aftenposten
rss-pa-konto
pengepodden-2
tid-er-penger-en-podcast-med-peter-warren
morgenkaffen-med-finansavisen
finansredaksjonen
utbytte
livet-pa-veien-med-jan-erik-larssen
liberal-halvtime
rss-borsmorgen-okonominyhetene
lederpodden
pengesnakk
rss-kron-podden
rss-skravla-gar
rss-markedspuls-2
shifter