23: Searching for Neighbors with Voyager
NerdOut@Spotify19 Loka 2023

23: Searching for Neighbors with Voyager

How do you get a machine to find a song that’s similar to another song? What properties of the song should it look for? And then does it just compare each track to every other track, one by one, until it finds the closest match? When you have a catalog of 100 million different music tracks, like we do at Spotify, that would take a long time. So, for these kinds of problems, we use a technique known as nearest neighbor search (NNS). This past summer at Spotify, we built a new library for nearest neighbor search: It’s called Voyager — and we open sourced it.

Host and principal engineer Dave Zolotusky talks with Peter Sobot and Mark Koh, two of the machine learning engineers who developed Voyager. They discuss using nearest neighbor search for recommendations and personalization, how to go from searching for vectors in a 2D space to searching for them in a space with thousands of dimensions, the relative funkiness and danceability of Mozart and Bach, how to find a place on a map when you don’t have the exact coordinates, tricky acronyms (Annoy: “Approximate Nearest Neighbor Oh Yeah”) and initialisms (HNSW: “Hierarchical Navigable Small World”), why we stopped using our old NNS library, why we open sourced the new one, how it works for use cases beyond music (like LLMs), and looking for ducks in grass.

Learn more about Spotify Voyager:

Read what else we’re nerding out about on the Spotify Engineering Blog: engineering.atspotify.com

You should follow us on Twitter @SpotifyEng and on LinkedIn!

Tämä jakso on lisätty Podme-palveluun avoimen RSS-syötteen kautta eikä se ole Podmen omaa tuotantoa. Siksi jakso saattaa sisältää mainontaa.

Jaksot(37)

34: Redesigning our Apple TV App *Release Notes*

34: Redesigning our Apple TV App *Release Notes*

What does it take to ship a video-first Spotify experience on the biggest screen in your house?In this Release Notes episode of the NerdOut@Spotify podcast, you’ll hear about the redesigned Spotify ap...

5 Maalis 21min

33: Ship Happens (When Agents Code)

33: Ship Happens (When Agents Code)

At Spotify, we're doubling down on background coding agents. The goal is to scale code migrations and boost developer productivity beyond our existing automation system. Built on top of Fleet Manageme...

15 Tammi 49min

32: ARM-ing Ourselves with Google Axion, Part 2

32: ARM-ing Ourselves with Google Axion, Part 2

In part two of ARM-ing ourselves with Google Axion, Dave is joined by Max Charas, a senior staff engineer at Spotify who’s been leading much of the migration work on our Google Axion journey, for a de...

19 Marras 202536min

31: ARM-ing Ourselves with Google Axion, Part 1

31: ARM-ing Ourselves with Google Axion, Part 1

Spotify is in the middle of a big shift in modern computing: moving from x86 processors to Google’s new ARM-based Axion chips. This isn’t just a hardware swap — it’s a disruptive change with ripple ef...

8 Loka 202531min

30: Building AiKA: Spotify’s AI Knowledge Assistant

30: Building AiKA: Spotify’s AI Knowledge Assistant

Hear about the journey behind building Spotify’s AI knowledge assistant, aka AiKA — from hack projects to internal chatbot, to enterprise developer tool.While general purpose chatbots are being used e...

14 Elo 202538min

29: Deploying Our New Typeface: Spotify Mix

29: Deploying Our New Typeface: Spotify Mix

Last year Spotify launched a big update to the app: a new typeface. For most of us, changing fonts is easy. It’s just a dropdown menu away. But creating a whole new typeface and then rolling it out ac...

3 Huhti 202532min

28: The CNCF Turns 10

28: The CNCF Turns 10

Ever heard of Kubernetes? Envoy? Prometheus? You probably have. But what you may not know is that these projects are managed and sustained by the Cloud Native Computing Foundation, aka the CNCF. Cloud...

6 Maalis 202539min

Join us for Wizards, Portals, and Spotify for Backstage

Join us for Wizards, Portals, and Spotify for Backstage

Register for Spotify’s roadmap webinar on April 30, 2024 — and see what’s coming next from Spotify for Backstage, the open source platform for building internal developer portals. We’ll show you our l...

19 Huhti 20243min