NerdOut@Spotify22 Jun 2023

20: The Rise and Fall of ABBA

Back in the day, Spotify built a custom A/B testing tool called ABBA. It was great. The platform enabled lots of teams to try out lots of ideas for new features to see what worked and what didn’t. With ABBA, we went from doing tens of experiments to hundreds of experiments. But we didn’t just learn what color button users liked better: the more tests we ran, the more we learned about our testing methods, including the limitations of ABBA itself — which eventually led us to a new, better way to test. Here’s the story of ABBA, our very first experimentation platform, and the lessons we learned about doing product experimentation at scale.

Host Dave Zolotusky talks with Mark Grey, a senior staff engineer and 10-year Spotify veteran. They discuss Spotify’s earliest efforts at product testing, our early infrastructure for data and data processing (using Hive and Hadoop), how migrating to the cloud unlocked more processing power (and more testing), the difference between using tests to design the color of a button and using tests to inform the very next user interaction via machine learning, feature flags and holdout groups, all the things we learned about conducting scientifically sound experiments, how we built a culture of experimentation among our software development teams, and what finally drove us to sunset ABBA and build its successor: a bigger, better internal experimentation platform. Plus, progress bars and lightsabers.

Read more about ABBA and how we do product experimentation at Spotify:

Read what else we’re nerding out about on the Spotify Engineering Blog: engineering.atspotify.com

You should follow us on Twitter @SpotifyEng and on LinkedIn!

Denne episoden er hentet fra en åpen RSS-feed og er ikke publisert av Podme. Den kan derfor inneholde annonser.

Episoder(39)

36: How the BBC Experiments at Scale: Learning Since 1922

How do you build a culture of experimentation inside a 100-year-old institution?This episode is part of our special miniseries on product experimentation where we go beyond A/B testing basics to explo...

11 Jun 31min

35: How Huel Experiments at Scale: Beyond A/B Tests

What happens when a seven-person data team restarts an experimentation program from scratch — and gets the whole company running experiments within months?Welcome to the first episode of our special m...

28 Mai 36min

34: Redesigning our Apple TV App Release Notes

What does it take to ship a video-first Spotify experience on the biggest screen in your house?In this Release Notes episode of the NerdOut@Spotify podcast, you’ll hear about the redesigned Spotify ap...

5 Mar 21min

33: Ship Happens (When Agents Code)

At Spotify, we're doubling down on background coding agents. The goal is to scale code migrations and boost developer productivity beyond our existing automation system. Built on top of Fleet Manageme...

15 Jan 49min

32: ARM-ing Ourselves with Google Axion, Part 2

In part two of ARM-ing ourselves with Google Axion, Dave is joined by Max Charas, a senior staff engineer at Spotify who’s been leading much of the migration work on our Google Axion journey, for a de...

19 Nov 202536min

31: ARM-ing Ourselves with Google Axion, Part 1

Spotify is in the middle of a big shift in modern computing: moving from x86 processors to Google’s new ARM-based Axion chips. This isn’t just a hardware swap — it’s a disruptive change with ripple ef...

8 Okt 202531min

30: Building AiKA: Spotify’s AI Knowledge Assistant

Hear about the journey behind building Spotify’s AI knowledge assistant, aka AiKA — from hack projects to internal chatbot, to enterprise developer tool.While general purpose chatbots are being used e...

14 Aug 202538min

29: Deploying Our New Typeface: Spotify Mix

Last year Spotify launched a big update to the app: a new typeface. For most of us, changing fonts is easy. It’s just a dropdown menu away. But creating a whole new typeface and then rolling it out ac...

3 Apr 202532min