Inside vLLM: The Engine Powering Open-Source AI

Elena Burger and Matt Bornstein are joined by Simon Mo, co-founder and CEO of Inferact, the open-source inference engine powering many of today's most advanced AI applications. Together, they explore how open-source AI evolved from a research project into critical infrastructure, why inference has become one of the most important layers of the AI stack, and what it takes to bring frontier intelligence to developers around the world.

The conversation covers vLLM's origins, the rise of open-weight models, why companies increasingly want control over their AI infrastructure, and how open-source inference enables the next generation of AI applications. They also discuss model licensing, the economics of open-weight AI, Kimi K3, distillation, AI infrastructure, and why Simon believes the gap between open and closed models is rapidly disappearing.

Resources:

Follow Simon Mo on X: https://x.com/simon_mo_

Follow Matt Bornstein on X: https://x.com/BornsteinMatt

Follow Elena Burger on X: https://x.com/VirtualElena

Follow Inferact: https://x.com/inferact

Check out everything a16z is doing with artificial intelligence here, including articles, projects, and more podcasts.

Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.

Tämä jakso on lisätty Podme-palveluun avoimen RSS-syötteen kautta eikä se ole Podmen omaa tuotantoa. Siksi jakso saattaa sisältää mainontaa.

Jaksot(102)

Building Self-Accelerating AI with Mirendil

Matt Bornstein speaks with Mirendil cofounders Behnam Neyshabur and Harsh Mehta about their vision for building self-accelerating AI. After leading research efforts at Google and Anthropic, the founde...

24 Kesä 40min

Ideogram’s Open-Weights Image Model and the Future of AI Design

Yoko Li and Justine Moore speak with Ideogram founder and CEO Mohammad Norouzi about image generation models, design workflows, and the evolving relationship between AI and creative work. The conversa...

15 Kesä 42min

Building Search for AI Agents with Exa CEO Will Bryk

Sarah Wang speaks with Exa cofounder and CEO Will Bryk about building search infrastructure for the AI era. The conversation covers Exa’s origins, why traditional search engines were not designed for ...

4 Kesä 49min

AI Agents and the Fight for Customer Data

Martin Casado speaks with George Fraser, cofounder and CEO of Fivetran, about the future of data infrastructure in the age of AI. The conversation covers Fivetran’s merger with dbt, the changing role ...

2 Kesä 50min

Ben Horowitz on AI Infrastructure, Economics and The New Laws of Software

Recorded live at the a16z Fintech Connect conference in Deer Valley, Alex Rampell speaks with Ben Horowitz, cofounder and general partner at a16z, about how AI has rewritten the fundamental rules of s...

19 Touko 29min

AI Infrastructure, Distribution, and the Next Wave of Software

Sophie Buonassisi speaks with Jennifer Li, general partner at a16z, about why infrastructure is becoming one of the most important areas in AI. They discuss how the shift to AI-native systems is resha...

12 Touko 38min

From Vector Databases to Knowledge Engines: The Next Layer of AI

Peter Levine speaks with Ash Ashutosh, CEO of Pinecone, about the launch of Nexus and the shift from vector databases to knowledge engines. As agents become the primary users of software, they discuss...

5 Touko 46min