The Neuron: AI Explained7 Okt 2025

AI Inference: Why Speed Matters More Than You Think (with SambaNova's Kwasi Ankomah)

Everyone's talking about the AI datacenter boom right now. Billion dollar deals here, hundred billion dollar deals there. Well, why do data centers matter? It turns out, AI inference (actually calling the AI and running it) is the hidden bottleneck slowing down every AI application you use (and new stuff yet to be released).

In this episode, Kwasi Ankomah from SambaNova Systems explains why running AI models efficiently matters more than you think, how their revolutionary chip architecture delivers 700+ tokens per second, and why AI agents are about to make this problem 10x worse.

💡 This episode is sponsored by Gladia's Solaria - the speech-to-text API built for real-world voice AI. With sub-270ms latency, 100+ languages supported, and 94% accuracy even in noisy environments, it's the backbone powering voice agents that actually work. Learn more at gladia.io/solaria

🔗 Key Links:

• SambaNova Cloud: https://cloud.sambanova.ai

• Check out Solaria speech to text API: https://www.gladia.io/solaria

• Subscribe to The Neuron newsletter: https://theneuron.ai

🎯 What You'll Learn:

• Why inference speed matters more than model size

• How SambaNova runs massive models on 90% less power

• Why AI agents use 10-20x more tokens

• The best open source models right now

• What to watch for in AI infrastructure

➤ CHAPTERS

Timecode - Chapter Title

0:00 - Intro

2:14 - What is AI Inference?

3:19 - Why Inference is the Real Challenge

9:18 - A message from our sponsor, Gladia Solaria

10:16 - The 95% ROI Problem Discussion

13:47 - SambaNova's Revolutionary Chip Architecture

15:19 - Running DeepSeek's 670B Parameter Models

18:11 - Developer Experience & Platform

21:26 - AI Agents and the Token Explosion

24:33 - Model Swapping and Cost Optimization

31:30 - Energy Efficiency 10kW vs 100kW

36:13 - Future of AI Models Bigger vs Smaller

39:24 - Best Open Source Models Right Now

46:01 - AI Infrastructure Next 12 Months

47:09 - Agents as Infrastructure

50:28 - Human-in-the-Loop and Trust

52:55 - Closing and Resources

Article Written by: Grant Harvey

Hosted by: Corey Noles and Grant Harvey

Guest: Kwasi Ankomah

Published by: Manique Santos

Edited by: Adrian Vallinan

Denne episoden er hentet fra en åpen RSS-feed og er ikke publisert av Podme. Den kan derfor inneholde annonser.

Episoder(115)

AI Can Now Write DNA. What Does This Promise, And What Could Go Wrong? With Eric Nguyen, co-founder and CEO of Radical Numerics

AI can now read DNA like language, generate working biological sequences, and help scientists identify disease-related mutations. Radical Numerics CEO Eric Nguyen explains how genome language models w...

12 Aug 51min

BONUS: How to Use AI Agents for Total Beginners: A Crash Course w/ Agent Builder James McAulay

This is the #1 request we get every week: how to actually use agents to save time in your business.We’re bringing in James McAulay, founder of The Agent Accelerator, for a practical crash course on wh...

7 Aug 1h 59min

AI Agents Are About to Move Off the Cloud. Here’s What Changes

What happens when the most capable AI lives in the cloud, but the work you want it to do is too private, expensive, or repetitive to send there every time?Dr. Olena Zhu, Head of AI Solutions & Ecosyst...

5 Aug 54min

BONUS: How to Publish Your AI-Built App to the App Store

Update! The definitive follow-along walkthrough guide is ready! Check it out here: https://theneuron.ai/explainer-articles/how-to-publish-an-ai-built-iphone-app-to-the-app-store/Building an app with A...

31 Jul 1h 8min

AI Can Build It, But Can AI Ship It?

AI can now help almost anyone turn an idea into a working prototype. But what happens when that vibe-coded experiment needs to become secure, reliable software that real customers can trust—and pay fo...

29 Jul 23min

BONUS: We Built 15+ Apps With AI. Here’s What Actually Worked

Join Grant Harvey and Corey Noles live on Thursday, July 23, for a show-and-tell featuring the apps, games, extensions, and delightfully specific tools they’ve built with AI, plus tips to help you bui...

24 Jul 2h 20min

Will AI Robots Perform Surgery Better Than Humans?

Autonomous surgery may sound futuristic, but automation already plays an important role in procedures requiring extraordinary precision. Mathias Unberath of Inner Logic joins Grant Harvey to explain h...

22 Jul 1h 10min

BONUS: AI Agents Are Here. Now What?

Neuron Live is back and we are going deep on Microsoft’s big move in AI agents. 🤖This week, we’re joined by Bryan Goode, Corporate Vice President of Business Applications Marketing at Microsoft, to u...

17 Jul 2h 58min