Interleaving
Linear Digressions22 Juli 2019

Interleaving

If you’re Google or Netflix, and you have a recommendation or search system as part of your bread and butter, what’s the best way to test improvements to your algorithm? A/B testing is the canonical answer for testing how users respond to software changes, but it gets tricky really fast to think about what an A/B test means in the context of an algorithm that returns a ranked list. That’s why we’re talking about interleaving this week—it’s a simple modification to A/B testing that makes it much easier to race two algorithms against each other and find the winner, and it allows you to do it with much less data than a traditional A/B test. Relevant links: https://medium.com/netflix-techblog/interleaving-in-online-experiments-at-netflix-a04ee392ec55 https://www.microsoft.com/en-us/research/publication/predicting-search-satisfaction-metrics-with-interleaved-comparisons/ https://www.cs.cornell.edu/people/tj/publications/joachims_02b.pdf

Det här avsnittet är hämtat från ett öppet RSS-flöde och publiceras inte av Podme. Det kan innehålla reklam.

Avsnitt(316)

Invisible LLM Failures and AI Fluency with Chris Potts (Stanford)

Invisible LLM Failures and AI Fluency with Chris Potts (Stanford)

What happens when a Stanford linguistics professor turns his attention to AI chatbots — and the surprisingly invisible ways humans misunderstand them? Chris Potts joins the show to unpack the hidden f...

20 Juli 41min

Still summer break: back next week

Still summer break: back next week

Still summer break: back next week by Katie Malone

13 Juli 25s

Summer break: back soon

Summer break: back soon

Summer break: back soon by Katie Malone

6 Juli 36s

Interviewing the Linear Digressions Agents (The Agents Season, Episode 11)

Interviewing the Linear Digressions Agents (The Agents Season, Episode 11)

After a five-year hiatus, the podcast that burned out partly over the tedium of writing episode descriptions is back — and using AI agents to handle exactly that task. The season-11 finale turns the l...

28 Juni 37min

Agent Economics (The Agents Season, Episode 10)

Agent Economics (The Agents Season, Episode 10)

What if building more highways made your commute *slower*? That's the paradox at the heart of AI agent economics: even as per-token inference costs have plummeted dramatically over the past two years,...

22 Juni 24min

Agent Trust, Oversight and Control (The Agents Season, Episode 9)

Agent Trust, Oversight and Control (The Agents Season, Episode 9)

Capabilities get all the attention when it comes to AI agents — but what happens when a highly capable agent makes a bad decision in the real world? Trust, oversight, and control are the unglamorous b...

15 Juni 25min

Many Agents, Many Problems (The Agents Season, Episode 8)

Many Agents, Many Problems (The Agents Season, Episode 8)

Whether you work best solo or thrive in a team, you know collaboration is complicated — and it turns out AI agents face the same tensions. This episode dives into multi-agent systems, exploring how ne...

8 Juni 28min

How Do You Evaluate An AI Agent? (The Agents Season, Episode 7)

How Do You Evaluate An AI Agent? (The Agents Season, Episode 7)

Knowing when an AI agent has failed sounds straightforward — until it isn't. Agents have a frustrating habit of finishing confidently while quietly doing the wrong thing, or looping endlessly without ...

1 Juni 31min

Populärt inom Teknik

uppgang-och-fall
elbilsveckan
market-makers
natets-morka-sida
rss-uppgang-och-fall
rss-technokratin
skogsforum-podcast
rss-elektrikerpodden
developers-mer-an-bara-kod
allt-du-behover-veta-om-ny-teknik
rss-sakerhetspodcasten
bilar-med-sladd
rss-fabriken-2
rss-veckans-ai
bli-saker-podden
rss-podden-om-dronare
rss-heja-framtiden
trevligt-snack
prova-programmering-av-distansakademin
rss-milpodden