Scaling Agentic Inference Across Heterogeneous Compute with Zain Asgar - #757

Scaling Agentic Inference Across Heterogeneous Compute with Zain Asgar - #757

In this episode, Zain Asgar, co-founder and CEO of Gimlet Labs, joins us to discuss the heterogeneous AI inference across diverse hardware. Zain argues that the current industry standard of running all AI workloads on high-end GPUs is unsustainable for agents, which consume significantly more tokens than traditional LLM applications. We explore Gimlet’s approach to heterogeneous inference, which involves disaggregating workloads across a mix of hardware—from H100s to older GPUs and CPUs—to optimize unit economics without sacrificing performance. We dive into their "three-layer cake" architecture: workload disaggregation, a compilation layer that maps models to specific hardware targets, and a novel system that uses LLMs to autonomously rewrite and optimize compute kernels. Finally, we discuss the complexities of networking in heterogeneous environments, the trade-offs between numerical precision and application accuracy, and the future of hardware-aware scheduling. The complete show notes for this episode can be found at https://twimlai.com/go/757.

Avsnitt(775)

This Week in ML & AI - 7/1/16: Fatal Tesla Autopilot Crash, EU Outlawing Machine Learning & CVPR

This Week in ML & AI - 7/1/16: Fatal Tesla Autopilot Crash, EU Outlawing Machine Learning & CVPR

This Week in Machine Learning & AI brings you the week’s most interesting and important stories from the world of machine learning and artificial intelligence. This week's show covers the first fatal Tesla autopilot crash, a new EU law that could prohibit machine learning, the AI that shot down a human fighter pilot (in simulation), the 2016 CVPR conference, 10 hot AI startups, the business implications of machine learning, cool chatbot projects and if you can believe it, even more. Show notes for this episode can be found at https://twimlai.com/7.

3 Juli 201635min

This Week in ML & AI - 6/24/16: Dueling Neural Networks at ICML, Plus Training a Robotic Housekeeper

This Week in ML & AI - 6/24/16: Dueling Neural Networks at ICML, Plus Training a Robotic Housekeeper

This Week in Machine Learning & AI brings you the week’s most interesting and important stories from the world of machine learning and artificial intelligence. This week's show covers the International Conference on Machine Learning (ICML), new research on "dueling architectures" for reinforcement learning, AI safety for robots, plus top AI business deals, tech announcement, projects and more.

25 Juni 201625min

This Week in Machine Learning & AI - 6/17/16: Apple's New ML APIs, IBM Brings Deep Learning Thunder

This Week in Machine Learning & AI - 6/17/16: Apple's New ML APIs, IBM Brings Deep Learning Thunder

This Week in Machine Learning & AI brings you the week’s most interesting and important stories from the world of machine learning and artificial intelligence. This week’s podcast digs into Apple's ML and AI announcements at WWDC, looks at IBM's new Deep Thunder offering, and discusses exciting new deep learning research from MIT, OpenAI and Google. Show notes available at https://twimlai.com/5.

18 Juni 201624min

This Week In Machine Learning & AI - 6/10/16: Self-Motivated AI, Plus A Kill-Switch for Rogue Bots

This Week In Machine Learning & AI - 6/10/16: Self-Motivated AI, Plus A Kill-Switch for Rogue Bots

This Week in Machine Learning & AI brings you the week’s most interesting and important stories from the world of machine learning and artificial intelligence. This week’s podcast looks at new research on intrinsic motivation for AI systems, a kill-switch for intelligent agents, "knu" chips for machine learning, a screenplay made by a neural net, and more. Show notes and subscribe links at https://cloudpul.se/twiml/4.

11 Juni 201624min

This Week In Machine Learning & AI - 6/3/16: Facebook's DeepText, ML & Art, Artificial Assistants

This Week In Machine Learning & AI - 6/3/16: Facebook's DeepText, ML & Art, Artificial Assistants

This Week in Machine Learning & AI brings you the week’s most interesting and important stories from the world of machine learning and artificial intelligence. This week’s podcast looks at Facebooks' new DeepText engine, creating music & art with deep learning and Google Magenta, how to build artificial assistants and bots, and applying economics to machine learning models. For show notes visit: https://cloudpul.se/posts/twiml-facebooks-deeptext-ml-art-artificial-assistants

4 Juni 201624min

This Week In Machine Learning & AI - 5/27/16: The White House on AI & Aggressive Self-Driving Cars

This Week In Machine Learning & AI - 5/27/16: The White House on AI & Aggressive Self-Driving Cars

This Week in Machine Learning & AI brings you the week's most interesting and important stories from the world of machine learning and artificial intelligence. This week's episode explores the White House workshops on AI, human bias in AI and machine learning models, a company working on machine learning for small datasets, plus the latest AI & ML news and a self-driving car that learned how to drive aggressively.

28 Maj 201625min

This Week In Machine Learning & AI - 5/20/16: AI at Google I/O, Amazon's Deep Learning DSSTNE

This Week In Machine Learning & AI - 5/20/16: AI at Google I/O, Amazon's Deep Learning DSSTNE

This Week In Machine Learning & AI - May 20, 2016. Google I/O, deep learning hardware and an AI to save you from conference call hell.

21 Maj 201619min

Populärt inom Politik & nyheter

aftonbladet-krim
svenska-fall
motiv
p3-krim
fordomspodden
blenda-2
rss-viva-fotboll
rss-krimstad
flashback-forever
aftonbladet-daily
rss-sanning-konsekvens
rss-vad-fan-hande
rss-krimreportrarna
rss-frandfors-horna
dagens-eko
sydsvenskan-dok
olyckan-inifran
rss-flodet
rss-svalan-krim
krimmagasinet