Deep Gradient Compression for Distributed Training with Song Han - TWiML Talk #146

Deep Gradient Compression for Distributed Training with Song Han - TWiML Talk #146

On today’s show I chat with Song Han, assistant professor in MIT’s EECS department, about his research on Deep Gradient Compression. In our conversation, we explore the challenge of distributed training for deep neural networks and the idea of compressing the gradient exchange to allow it to be done more efficiently. Song details the evolution of distributed training systems based on this idea, and provides a few examples of centralized and decentralized distributed training architectures such as Uber’s Horovod, as well as the approaches native to Pytorch and Tensorflow. Song also addresses potential issues that arise when considering distributed training, such as loss of accuracy and generalizability, and much more. The notes for this show can be found at twimlai.com/talk/146.

Det här avsnittet är hämtat från ett öppet RSS-flöde och publiceras inte av Podme. Det kan innehålla reklam.

Avsnitt(797)

Why Jev Is Changing How We Build With AI with Diogo Almeida - #779

Why Jev Is Changing How We Build With AI with Diogo Almeida - #779

In this episode, Diogo Almeida, co-founder and CEO of TypeSafe, joins us to discuss Jev, TypeSafe’s recently released model for bringing fast, reliable intelligence directly into software. We explore ...

6 Okt 1h 30min

From Math Olympiads to Navier-Stokes: How Fast Is AI Progressing? with Greg Burnham - #778

From Math Olympiads to Navier-Stokes: How Fast Is AI Progressing? with Greg Burnham - #778

AI systems have gone from struggling with grade-school math to helping solve research problems that have resisted mathematicians for decades, including Navier-Stokes. In this episode, Greg Burnham, w...

29 Sep 1h 7min

From Voice Agents to AI Avatars with Alexander Smola - #777

From Voice Agents to AI Avatars with Alexander Smola - #777

Voice AI has gotten remarkably good, but natural conversation remains a high bar. Small delays, awkward interruptions, or the wrong tone can quickly break the illusion—and adding vision and visual pre...

17 Sep 1h 4min

Do AI Tokenomics Matter More Than Model Benchmarks? with Chris Potts - #776

Do AI Tokenomics Matter More Than Model Benchmarks? with Chris Potts - #776

As reasoning models consume more tokens and AI systems become more expensive to run, understanding what those tokens actually buy is becoming increasingly important. In this episode, Stanford professo...

9 Sep 59min

World Models and the Future of Spatial AI with Justin Johnson - #775

World Models and the Future of Spatial AI with Justin Johnson - #775

In this episode, Justin Johnson, co-founder of World Labs, joins us to discuss world models and the emerging field of spatial AI. We explore why many researchers see capabilities beyond language as an...

1 Sep 1h 6min

Why the Next AI Breakthrough May Come from Physics with Max Welling - #774

Why the Next AI Breakthrough May Come from Physics with Max Welling - #774

The conventional wisdom in AI is that the next breakthrough will come from more compute, more data, and larger models. But what if the next leap comes from somewhere else? In this episode, Max Wellin...

26 Aug 58min

Why Image Generation Needs More Than Bigger Models with Fatih Porikli - #773

Why Image Generation Needs More Than Bigger Models with Fatih Porikli - #773

Text-to-image models have become remarkably good at producing realistic images. But realism isn’t the same as correctness. Ask for several distinct people, a specific composition, or a high-resolution...

12 Aug 56min

Why Models Are AI’s Next Training Dataset with Damian Borth - #772

Why Models Are AI’s Next Training Dataset with Damian Borth - #772

For more than a decade, AI has advanced by training ever-larger models on ever-larger datasets. But as high-quality training data becomes harder to find and pretraining grows increasingly expensive, r...

27 Juli 47min

Populärt inom Politik & nyheter

svenska-fall
fordomspodden
aftonbladet-krim
p3-krim
aftonbladet-daily
spar
rss-krimstad
flashback-forever
de-fyras-gang
svd-dokumentara-berattelser-2
rss-vad-fan-hande
rss-sanning-konsekvens
motiv
rss-krimreportrarna
svd-ledarredaktionen
rss-flodet
omni-podd
en-runda-till
ett-rent-noje
kungligt