NerdOut@Spotify19 Okt 2023

23: Searching for Neighbors with Voyager

How do you get a machine to find a song that’s similar to another song? What properties of the song should it look for? And then does it just compare each track to every other track, one by one, until it finds the closest match? When you have a catalog of 100 million different music tracks, like we do at Spotify, that would take a long time. So, for these kinds of problems, we use a technique known as nearest neighbor search (NNS). This past summer at Spotify, we built a new library for nearest neighbor search: It’s called Voyager — and we open sourced it.

Host and principal engineer Dave Zolotusky talks with Peter Sobot and Mark Koh, two of the machine learning engineers who developed Voyager. They discuss using nearest neighbor search for recommendations and personalization, how to go from searching for vectors in a 2D space to searching for them in a space with thousands of dimensions, the relative funkiness and danceability of Mozart and Bach, how to find a place on a map when you don’t have the exact coordinates, tricky acronyms (Annoy: “Approximate Nearest Neighbor Oh Yeah”) and initialisms (HNSW: “Hierarchical Navigable Small World”), why we stopped using our old NNS library, why we open sourced the new one, how it works for use cases beyond music (like LLMs), and looking for ducks in grass.

Learn more about Spotify Voyager:

Read what else we’re nerding out about on the Spotify Engineering Blog: engineering.atspotify.com

You should follow us on Twitter @SpotifyEng and on LinkedIn!

Denne episoden er hentet fra en åpen RSS-feed og er ikke publisert av Podme. Den kan derfor inneholde annonser.

Episoder(39)

36: How the BBC Experiments at Scale: Learning Since 1922

How do you build a culture of experimentation inside a 100-year-old institution?This episode is part of our special miniseries on product experimentation where we go beyond A/B testing basics to explo...

11 Jun 31min

35: How Huel Experiments at Scale: Beyond A/B Tests

What happens when a seven-person data team restarts an experimentation program from scratch — and gets the whole company running experiments within months?Welcome to the first episode of our special m...

28 Mai 36min

34: Redesigning our Apple TV App Release Notes

What does it take to ship a video-first Spotify experience on the biggest screen in your house?In this Release Notes episode of the NerdOut@Spotify podcast, you’ll hear about the redesigned Spotify ap...

5 Mar 21min

33: Ship Happens (When Agents Code)

At Spotify, we're doubling down on background coding agents. The goal is to scale code migrations and boost developer productivity beyond our existing automation system. Built on top of Fleet Manageme...

15 Jan 49min

32: ARM-ing Ourselves with Google Axion, Part 2

In part two of ARM-ing ourselves with Google Axion, Dave is joined by Max Charas, a senior staff engineer at Spotify who’s been leading much of the migration work on our Google Axion journey, for a de...

19 Nov 202536min

31: ARM-ing Ourselves with Google Axion, Part 1

Spotify is in the middle of a big shift in modern computing: moving from x86 processors to Google’s new ARM-based Axion chips. This isn’t just a hardware swap — it’s a disruptive change with ripple ef...

8 Okt 202531min

30: Building AiKA: Spotify’s AI Knowledge Assistant

Hear about the journey behind building Spotify’s AI knowledge assistant, aka AiKA — from hack projects to internal chatbot, to enterprise developer tool.While general purpose chatbots are being used e...

14 Aug 202538min

29: Deploying Our New Typeface: Spotify Mix

Last year Spotify launched a big update to the app: a new typeface. For most of us, changing fonts is easy. It’s just a dropdown menu away. But creating a whole new typeface and then rolling it out ac...

3 Apr 202532min