How to Engineer AI Inference Systems with Philip Kiely - #766

How to Engineer AI Inference Systems with Philip Kiely - #766

In this episode, Philip Kiely, head of AI education at Baseten, joins us to unpack the fast-evolving discipline of inference engineering. We explore why inference has become the stickiest and most critical workload in AI, how it blends GPU programming, applied research, and large-scale distributed systems, and where the line sits between inference and model serving. Philip shares how research-to-production can move in hours, not months, and why understanding “the knobs” of inference—batching, quantization, speculation, and KV cache reuse—lets teams design better products and SLAs. We trace the inference maturity journey from closed APIs to dedicated deployments and in-house platforms, discuss GPU lifecycles, and survey today’s runtime landscape, including vLLM, SGLang, and TensorRT LLM. Finally, we look ahead to agents and multimodality, making the case for specialized, workload-specific runtimes when performance and efficiency matter most. The complete show notes for this episode can be found at https://twimlai.com/go/766.

Det här avsnittet är hämtat från ett öppet RSS-flöde och publiceras inte av Podme. Det kan innehålla reklam.

Avsnitt(801)

Can AI Invent or Just Help Scientists Work Faster? with Richard Socher - #783

Can AI Invent or Just Help Scientists Work Faster? with Richard Socher - #783

AI can help scientists write code, analyze data, and draft papers. But can it generate the ideas that lead to new discoveries? Richard Socher, former chief scientist at Salesforce, founder of You.com,...

3 Nov 48min

The Brutal Reality of Building Agentic Products Today with Daniel Jeffries - #782

The Brutal Reality of Building Agentic Products Today with Daniel Jeffries - #782

AI agents are becoming accessible to everyone, raising the bar for developers building the next generation of agent products. When capable, general-purpose agents are available out of the box, differe...

28 Okt 54min

Airbnb Is Building a ‘Company Brain’ for AI Agents with Ahmad Al-Dahle - #781

Airbnb Is Building a ‘Company Brain’ for AI Agents with Ahmad Al-Dahle - #781

AI agents need the institutional knowledge experienced employees rely on: why a system works a certain way, which policies apply, and what never made it into the documentation. Capturing that knowledg...

21 Okt 51min

Your Ranking Model Is Optimizing the Wrong Thing with Zoe Weil - #780

Your Ranking Model Is Optimizing the Wrong Thing with Zoe Weil - #780

In this episode, Zoe Weil, co-founder and CEO of Sequen, joins us to discuss how ranking systems can use live behavioral signals to optimize for purchases rather than clicks or search relevance alone....

15 Okt 52min

Why Jev Is Changing How We Build With AI with Diogo Almeida - #779

Why Jev Is Changing How We Build With AI with Diogo Almeida - #779

In this episode, Diogo Almeida, co-founder and CEO of TypeSafe, joins us to discuss Jev, TypeSafe’s recently released model for bringing fast, reliable intelligence directly into software. We explore ...

6 Okt 1h 30min

From Math Olympiads to Navier-Stokes: How Fast Is AI Progressing? with Greg Burnham - #778

From Math Olympiads to Navier-Stokes: How Fast Is AI Progressing? with Greg Burnham - #778

AI systems have gone from struggling with grade-school math to helping solve research problems that have resisted mathematicians for decades, including Navier-Stokes. In this episode, Greg Burnham, w...

29 Sep 1h 7min

From Voice Agents to AI Avatars with Alexander Smola - #777

From Voice Agents to AI Avatars with Alexander Smola - #777

Voice AI has gotten remarkably good, but natural conversation remains a high bar. Small delays, awkward interruptions, or the wrong tone can quickly break the illusion—and adding vision and visual pre...

17 Sep 1h 4min

Do AI Tokenomics Matter More Than Model Benchmarks? with Chris Potts - #776

Do AI Tokenomics Matter More Than Model Benchmarks? with Chris Potts - #776

As reasoning models consume more tokens and AI systems become more expensive to run, understanding what those tokens actually buy is becoming increasingly important. In this episode, Stanford professo...

9 Sep 59min

Populärt inom Politik & nyheter

svenska-fall
fordomspodden
aftonbladet-krim
rss-krimstad
p3-krim
aftonbladet-daily
flashback-forever
motiv
rss-sanning-konsekvens
rss-vad-fan-hande
svd-ledarredaktionen
ett-rent-noje
rss-krimreportrarna
omni-podd
en-runda-till
kungligt
rss-politikrummet
politiken
rss-frandfors-horna
olyckan-inifran