#251 Sid Sheth: How d-Matrix is Disrupting AI Inference in 2025
Eye On A.I.30 Huhti 2025

#251 Sid Sheth: How d-Matrix is Disrupting AI Inference in 2025

This episode is sponsored by the DFINITY Foundation.

DFINITY Foundation's mission is to develop and contribute technology that enables the Internet Computer (ICP) blockchain and its ecosystem, aiming to shift cloud computing into a fully decentralized state.

Find out more at https://internetcomputer.org/

In this episode of Eye on AI, we sit down with Sid Sheth, CEO and Co-Founder of d-Matrix, to explore how his company is revolutionizing AI inference hardware and taking on industry giants like NVIDIA.

Sid shares his journey from building multi-billion-dollar businesses in semiconductors to founding d-Matrix—a startup focused on generative AI inference, chiplet-based architecture, and ultra-low latency AI acceleration.

We break down:

  • Why the future of AI lies in inference, not training

  • How d-Matrix's Corsair PCIe accelerator outperforms NVIDIA's H200

  • The role of in-memory compute and high bandwidth memory in next-gen AI chips

  • How d-Matrix integrates seamlessly into hyperscaler and enterprise cloud environments

  • Why AI infrastructure is becoming heterogeneous and what that means for developers

  • The global outlook on inference chips—from the US to APAC and beyond

  • How Sid plans to build the next NVIDIA-level company from the ground up.

Whether you're building in AI infrastructure, investing in semiconductors, or just curious about the future of generative AI at scale, this episode is packed with value.

Stay Updated:

Craig Smith on X:https://x.com/craigss

Eye on A.I. on X: https://x.com/EyeOn_AI

(00:00) Intro

(02:46) Introducing Sid Sheth

(05:27) Why He Started d-Matrix

(07:28) Lessons from Building a $2.5B Chip Business

(11:52) How d-Matrix Prototypes New Chips

(15:06) Working with Hyperscalers Like Google & Amazon

(17:27) What's Inside the Corsair AI Accelerator

(21:12) How d-Matrix Beats NVIDIA on Chip Efficiency

(24:10) The Memory Bandwidth Advantage Explained

(26:27) Running Massive AI Models at High Speed

(30:20) Why Inference Isn't One-Size-Fits-All

(32:40) The Future of AI Hardware

(36:28) Supporting Llama 3 and Other Open Models

(40:16) Is the Inference Market Big Enough?

(43:21) Why the US Is Still the Key Market

(46:39) Can India Compete in the AI Chip Race?

(49:09) Will China Catch Up on AI Hardware?

Tämä jakso on lisätty Podme-palveluun avoimen RSS-syötteen kautta eikä se ole Podmen omaa tuotantoa. Siksi jakso saattaa sisältää mainontaa.

Jaksot(353)

Every Enterprise Is About to Have a 100,000 Agent Problem | Oren Michaels of Barndoor AI

Every Enterprise Is About to Have a 100,000 Agent Problem | Oren Michaels of Barndoor AI

AI agents can now connect to every tool your employees use. The problem is that connecting them and trusting them are two completely different things, and most enterprises have figured out the first w...

6 Kesä 59min

More Customers Chose the AI Agent Than Anyone Expected | Tom Chen, Aircall

More Customers Chose the AI Agent Than Anyone Expected | Tom Chen, Aircall

Every time you hit a phone tree or a chatbot with canned answers, you're experiencing the gap between what AI can already do and what most companies are still delivering. Craig Smith sits down with To...

4 Kesä 56min

Why the Future of AI Isn't Just Bigger Models. It's Models That Evolve | Risto Miikkulainen of Cognizant

Why the Future of AI Isn't Just Bigger Models. It's Models That Evolve | Risto Miikkulainen of Cognizant

Most AI systems follow a gradient, a mathematical slope that tells them exactly how to improve, step by step, toward a known goal. Neuroevolution doesn't follow any gradient. Instead, it runs hundreds...

2 Kesä 1h 4min

How AI Is Reinventing Elder Care | Chia-Lin Simmons of LogicMark

How AI Is Reinventing Elder Care | Chia-Lin Simmons of LogicMark

One in four people over 65 will experience a fall, and for most of them, the technology designed to help is a device that hasn't meaningfully changed since the 1980s. Chia-Lin Simmons, CEO of LogicMar...

1 Kesä 53min

The App of the Future Is Voice — Not a Screen. Mitel's CTO Luiz Domingos Explains Why.

The App of the Future Is Voice — Not a Screen. Mitel's CTO Luiz Domingos Explains Why.

Luiz Domingos has spent 25 years watching enterprise communications evolve, from IP telephony to cloud to AI, and his assessment of where things stand now is unusually concrete. Companies have moved p...

28 Touko 54min

Is ChatGPT Conscious? A Pioneer of AI Explains | Dr. Terry Sejnowski

Is ChatGPT Conscious? A Pioneer of AI Explains | Dr. Terry Sejnowski

A fly with 100,000 neurons can fly, find food, and reproduce. A $100 million supercomputer cannot. Dr. Terry Sejnowski used that observation to silence a room full of MIT AI researchers in the 1980s, ...

28 Touko 56min

Your Child's Data Profile Starts Before They're Born | Eamonn Maguire of Proton

Your Child's Data Profile Starts Before They're Born | Eamonn Maguire of Proton

Your child's data profile doesn't start when they get their first phone. It starts before they're born, the moment a parent emails a gynecologist or visits a fertility clinic website. That's the core ...

28 Touko 55min

Training AI Models Without a Billion-Dollar Data Center | Steffen Cruz of Macrocosmos

Training AI Models Without a Billion-Dollar Data Center | Steffen Cruz of Macrocosmos

Training a frontier AI model today requires hundreds of thousands of GPUs, months of compute time, and a budget that only a handful of companies on earth can afford. Steffen Cruz, co-founder and CTO o...

25 Touko 47min