Running experiments when there are network effects

Running experiments when there are network effects

Traditional A/B tests assume that whether or not one person got a treatment has no effect on the experiment outcome for another person. But that’s not a safe assumption, especially when there are network effects (like in almost any social context, for instance!) SUTVA, or the stable treatment unit value assumption, is a big phrase for this assumption and violations of SUTVA make for some pretty interesting experiment designs. From news feeds in LinkedIn to disentangling herd immunity from individual immunity in vaccine studies, indirect (i.e. network) effects in experiments can be just as big as, or even bigger than, direct (i.e. individual effects). And this is what we talk about this week on the podcast. Relevant links: http://hanj.cs.illinois.edu/pdf/www15_hgui.pdf https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2600548/pdf/nihms-73860.pdf

Denne episoden er hentet fra en åpen RSS-feed og er ikke publisert av Podme. Den kan derfor inneholde annonser.

Episoder(309)

How Do You Evaluate An AI Agent? (The Agents Season, Episode 7)

How Do You Evaluate An AI Agent? (The Agents Season, Episode 7)

Knowing when an AI agent has failed sounds straightforward — until it isn't. Agents have a frustrating habit of finishing confidently while quietly doing the wrong thing, or looping endlessly without ...

1 Jun 31min

AI Agent Failure Modes (The Agents Season, Episode 6)

AI Agent Failure Modes (The Agents Season, Episode 6)

Despite what the marketing hype might suggest, AI agents are far from infallible — and if you've ever actually used one, you already know this. Today's episode dives deep into the many, varied, and so...

25 Mai 32min

Agentic Planning (The Agents Season, Episode 5)

Agentic Planning (The Agents Season, Episode 5)

When tackling a complex, multi-step task, even the smartest AI agent can fail without a solid game plan. This episode dives into the research around agentic planning — how agents move beyond simply re...

18 Mai 24min

Memory Management for AI Agents (The Agents Season, Episode 4)

Memory Management for AI Agents (The Agents Season, Episode 4)

Context windows are powerful — but finite, and surprisingly easy to overwhelm. When an AI agent is tackling a long, complex task, the information it needs has to fit inside that limited real estate, a...

10 Mai 24min

Lost in the Middle (The Agents Season, Episode 3)

Lost in the Middle (The Agents Season, Episode 3)

Just like a memorable talk lives or dies by its opening and closing, LLMs have a surprisingly similar quirk: they pay close attention to what's at the beginning and end of their context window — and k...

4 Mai 19min

ReAct and Tool Usage (The Agents Season, Episode 2)

ReAct and Tool Usage (The Agents Season, Episode 2)

Before 2022, there was a wall between AI and the real world — models could reason impressively, but couldn't look anything up, run code, or check whether anything they said was actually true. This epi...

27 Apr 23min

What's an AI Agent? And Why's That Hard to Define? (The Agents Season, Episode 1)

What's an AI Agent? And Why's That Hard to Define? (The Agents Season, Episode 1)

AI agents are having a moment — and unpacking them properly takes more than a single conversation. This episode kicks off a dedicated multi-part season exploring AI agents from every angle, building u...

20 Apr 19min

Unfaithful Chain of Thought

Unfaithful Chain of Thought

What's actually happening when an LLM "thinks out loud"? Research on human decision-making suggests that much of the reasoning we believe drives our choices is actually post hoc rationalization — we d...

13 Apr 24min

Populært innen Teknologi

lydartikler-fra-aftenposten
romkapsel
teknisk-sett
energi-og-klima
tomprat-med-gunnar-tjomlid
nasjonal-sikkerhetsmyndighet-nsm
elektropodden
hans-petter-og-co
shifter
pedagogisk-intelligens
rss-anleggspraten
fornybaren
teknologi-og-mennesker
rss-snakk-om-sikkerhet
rss-plateprat
rss-ai-forklart
rss-ki-praten
plattformpodden
rss-devops
rss-30-minutter-inn-i-fremtiden