72: Multimodal AI for Ray-Ban Meta glasses
Meta Tech Podcast28 Helmi 2025

72: Multimodal AI for Ray-Ban Meta glasses

In this episode of the Meta Tech Podcast, host Pascal sits down with Shane, a research scientist at Meta, to explore the cutting-edge research behind Ray-Ban Meta glasses. Shane shares insights from his seven-year journey at Meta, where he focuses on computer vision and multimodal AI within the Wearables AI organization.

Tune in to learn how Shane's team is pioneering foundational models for Ray-Ban Meta glasses, tackling unique challenges, and pushing the boundaries of AI-driven innovation. Discover how multimodal AI is transforming user experiences and get a glimpse into the future of wearable technology. Whether you're an engineer, a tech enthusiast, or simply curious about the latest advancements, there is something for everyone in this episode.

Got feedback? Send it to us on Threads (https://threads.net/@metatechpod), Instagram (https://instagram.com/metatechpod) and don't forget to follow our host Pascal (https://mastodon.social/@passy, https://threads.net/@passy_). Fancy working with us? Check out https://www.metacareers.com/.

Links

Timestamps

  • Intro 0:06

  • OSS News 0:56

  • Introduction Shane 1:30

  • The role of research scientist over time 3:03

  • What's Multimodal AI? 5:45

  • Applying Multimodal AI in Meta's products 7:21

  • Acoustic modalities beyond speech 9:17

  • AnyMAL 12:23

  • Encoder zoos 13:53

  • 0-shot performance 16:25

  • Iterating on models 17:28

  • LLM parameter size 19:29

  • How do we process a request from the glasses? 21:53

  • Processing moving images 23:44

  • Scaling to billions of users 26:01

  • Where lies the optimisation potential? 28:12

  • Incorporating feedback 29:08

  • Open-source influence 31:30

  • Be My Eyes Programme 33:57

  • Working with industry experts at Meta 36:18

  • Outro 38:55

Tämä jakso on lisätty Podme-palveluun avoimen RSS-syötteen kautta eikä se ole Podmen omaa tuotantoa. Siksi jakso saattaa sisältää mainontaa.

Jaksot(94)

BONUS: Send Us Your Questions About Engineering at Meta

BONUS: Send Us Your Questions About Engineering at Meta

Pascal is going on Recharge for August so there will be no regular interview episode. However, we want to hear your questions for our mailbag episode. Send them to us on Instagram (https://www.instagr...

29 Heinä 1min

87: Everything is Power: Battery Life Engineering for AI Glasses

87: Everything is Power: Battery Life Engineering for AI Glasses

What changes for you as a developer when the device moves from your pocket to your face? The battery shrinks by an order of magnitude, the thermal budget is whatever won't make your nose uncomfortable...

13 Heinä 41min

86: A Hard Cell: Engineering Ultra-Narrow Batteries for AI Glasses

86: A Hard Cell: Engineering Ultra-Narrow Batteries for AI Glasses

Your smart glasses run all day on a battery narrower than your pinky finger and building it required reinventing how batteries are made. In this episode, Pascal talks to Karthik and Myuran, the engine...

9 Kesä 42min

85: Reel Friends: Building Social Discovery that Scales to Billions

85: Reel Friends: Building Social Discovery that Scales to Billions

You've probably spotted those little circles of your friends' faces popping up on Facebook Reels. They look simple enough, but building them was a proper engineering challenge. In this episode, Pascal...

8 Touko 38min

84: Trust But Canary: Configuration Safety at Scale

84: Trust But Canary: Configuration Safety at Scale

Have you ever wondered how Meta makes config rollouts safe at scale? In this episode, Pascal sits down with Ishwari and Joe to discuss Meta's approach for propagating changes across services in second...

2 Huhti 37min

83: Patch Me If You Can: AI Codemods for Secure-by-Default Android Apps

83: Patch Me If You Can: AI Codemods for Secure-by-Default Android Apps

At Meta, even seemingly simple engineering tasks—like updating an API—become monumental undertakings when you're dealing with millions of lines of code and thousands of engineers, especially if the ch...

27 Helmi 47min

82: CSS at Scale with StyleX

82: CSS at Scale with StyleX

It's not just Not Invented Here Syndrome. Some technologies like CSS simply don't scale if you're building some of the largest websites on the planet with thousands of engineers committing to the same...

8 Tammi 44min

81: From Zero to Polish: Building Meta Ray-Ban Display

81: From Zero to Polish: Building Meta Ray-Ban Display

You've likely heard of Meta Ray-Ban Display by now — but what's it actually like to work on it? In this episode, Pascal talks to Kenan and Emanuel about the exciting features of Meta's First-Gen Displ...

12 Joulu 202547min