Meta Tech Podcast28 Feb 2025

72: Multimodal AI for Ray-Ban Meta glasses

In this episode of the Meta Tech Podcast, host Pascal sits down with Shane, a research scientist at Meta, to explore the cutting-edge research behind Ray-Ban Meta glasses. Shane shares insights from his seven-year journey at Meta, where he focuses on computer vision and multimodal AI within the Wearables AI organization.

Tune in to learn how Shane's team is pioneering foundational models for Ray-Ban Meta glasses, tackling unique challenges, and pushing the boundaries of AI-driven innovation. Discover how multimodal AI is transforming user experiences and get a glimpse into the future of wearable technology. Whether you're an engineer, a tech enthusiast, or simply curious about the latest advancements, there is something for everyone in this episode.

Got feedback? Send it to us on Threads (https://threads.net/@metatechpod), Instagram (https://instagram.com/metatechpod) and don't forget to follow our host Pascal (https://mastodon.social/@passy, https://threads.net/@passy_). Fancy working with us? Check out https://www.metacareers.com/.

Links

AnyMAL: An Efficient and Scalable Any-Modality Augmented Language Model - https://arxiv.org/abs/2309.16058
Be My Eyes Programme: https://www.forbes.com/sites/stevenaquino/2024/10/11/inside-the-be-my-eyes-meta-collaboration-and-the-allure-to--impact-humanity/
Meta Open Source on Threads: https://www.threads.net/@metaopensource
CacheLib: https://cachelib.org/
Meta's AI-Powered Ray-Bans Are Life-Enhancing for the Blind - Wall Street Journal: https://www.wsj.com/tech/ai/metas-ai-powered-ray-bans-are-life-enhancing-for-the-blind-3ae38026

Timestamps

Intro 0:06
OSS News 0:56
Introduction Shane 1:30
The role of research scientist over time 3:03
What's Multimodal AI? 5:45
Applying Multimodal AI in Meta's products 7:21
Acoustic modalities beyond speech 9:17
AnyMAL 12:23
Encoder zoos 13:53
0-shot performance 16:25
Iterating on models 17:28
LLM parameter size 19:29
How do we process a request from the glasses? 21:53
Processing moving images 23:44
Scaling to billions of users 26:01
Where lies the optimisation potential? 28:12
Incorporating feedback 29:08
Open-source influence 31:30
Be My Eyes Programme 33:57
Working with industry experts at Meta 36:18
Outro 38:55

Denne episoden er hentet fra en åpen RSS-feed og er ikke publisert av Podme. Den kan derfor inneholde annonser.

Episoder(94)

BONUS: Send Us Your Questions About Engineering at Meta

Pascal is going on Recharge for August so there will be no regular interview episode. However, we want to hear your questions for our mailbag episode. Send them to us on Instagram (https://www.instagr...

29 Jul 1min

87: Everything is Power: Battery Life Engineering for AI Glasses

What changes for you as a developer when the device moves from your pocket to your face? The battery shrinks by an order of magnitude, the thermal budget is whatever won't make your nose uncomfortable...

13 Jul 41min

86: A Hard Cell: Engineering Ultra-Narrow Batteries for AI Glasses

Your smart glasses run all day on a battery narrower than your pinky finger and building it required reinventing how batteries are made. In this episode, Pascal talks to Karthik and Myuran, the engine...

9 Jun 42min

85: Reel Friends: Building Social Discovery that Scales to Billions

You've probably spotted those little circles of your friends' faces popping up on Facebook Reels. They look simple enough, but building them was a proper engineering challenge. In this episode, Pascal...

8 Mai 38min

84: Trust But Canary: Configuration Safety at Scale

Have you ever wondered how Meta makes config rollouts safe at scale? In this episode, Pascal sits down with Ishwari and Joe to discuss Meta's approach for propagating changes across services in second...

2 Apr 37min

83: Patch Me If You Can: AI Codemods for Secure-by-Default Android Apps

At Meta, even seemingly simple engineering tasks—like updating an API—become monumental undertakings when you're dealing with millions of lines of code and thousands of engineers, especially if the ch...

27 Feb 47min

82: CSS at Scale with StyleX

It's not just Not Invented Here Syndrome. Some technologies like CSS simply don't scale if you're building some of the largest websites on the planet with thousands of engineers committing to the same...

8 Jan 44min

81: From Zero to Polish: Building Meta Ray-Ban Display

You've likely heard of Meta Ray-Ban Display by now — but what's it actually like to work on it? In this episode, Pascal talks to Kenan and Emanuel about the exciting features of Meta's First-Gen Displ...

12 Des 202547min