Mamba, Mamba-2 and Post-Transformer Architectures for Generative AI with Albert Gu - #693

Mamba, Mamba-2 and Post-Transformer Architectures for Generative AI with Albert Gu - #693

Today, we're joined by Albert Gu, assistant professor at Carnegie Mellon University, to discuss his research on post-transformer architectures for multi-modal foundation models, with a focus on state-space models in general and Albert’s recent Mamba and Mamba-2 papers in particular. We dig into the efficiency of the attention mechanism and its limitations in handling high-resolution perceptual modalities, and the strengths and weaknesses of transformer architectures relative to alternatives for various tasks. We dig into the role of tokenization and patching in transformer pipelines, emphasizing how abstraction and semantic relationships between tokens underpin the model's effectiveness, and explore how this relates to the debate between handcrafted pipelines versus end-to-end architectures in machine learning. Additionally, we touch on the evolving landscape of hybrid models which incorporate elements of attention and state, the significance of state update mechanisms in model adaptability and learning efficiency, and the contribution and adoption of state-space models like Mamba and Mamba-2 in academia and industry. Lastly, Albert shares his vision for advancing foundation models across diverse modalities and applications. The complete show notes for this episode can be found at https://twimlai.com/go/693.

Avsnitt(781)

Multilingual LLMs and the Values Divide in AI with Sara Hooker - #651

Multilingual LLMs and the Values Divide in AI with Sara Hooker - #651

Today we’re joined by Sara Hooker, director at Cohere and head of Cohere For AI, Cohere’s research lab. In our conversation with Sara, we explore some of the challenges with multilingual models like p...

16 Okt 20231h 18min

Scaling Multi-Modal Generative AI with Luke Zettlemoyer - #650

Scaling Multi-Modal Generative AI with Luke Zettlemoyer - #650

Today we’re joined by Luke Zettlemoyer, professor at University of Washington and a research manager at Meta. In our conversation with Luke, we cover multimodal generative AI, the effect of data on mo...

9 Okt 202338min

Pushing Back on AI Hype with Alex Hanna - #649

Pushing Back on AI Hype with Alex Hanna - #649

Today we’re joined by Alex Hanna, the Director of Research at the Distributed AI Research Institute (DAIR). In our conversation with Alex, we discuss the topic of AI hype and the importance of tacklin...

2 Okt 202349min

Personalization for Text-to-Image Generative AI with Nataniel Ruiz - #648

Personalization for Text-to-Image Generative AI with Nataniel Ruiz - #648

Today we’re joined by Nataniel Ruiz, a research scientist at Google. In our conversation with Nataniel, we discuss his recent work around personalization for text-to-image AI models. Specifically, we ...

25 Sep 202344min

Ensuring LLM Safety for Production Applications with Shreya Rajpal - #647

Ensuring LLM Safety for Production Applications with Shreya Rajpal - #647

Today we’re joined by Shreya Rajpal, founder and CEO of Guardrails AI. In our conversation with Shreya, we discuss ensuring the safety and reliability of language models for production applications. W...

18 Sep 202340min

What’s Next in LLM Reasoning? with Roland Memisevic - #646

What’s Next in LLM Reasoning? with Roland Memisevic - #646

Today we’re joined by Roland Memisevic, a senior director at Qualcomm AI Research. In our conversation with Roland, we discuss the significance of language in humanlike AI systems and the advantages a...

11 Sep 202359min

Is ChatGPT Getting Worse? with James Zou - #645

Is ChatGPT Getting Worse? with James Zou - #645

Today we’re joined by James Zou, an assistant professor at Stanford University. In our conversation with James, we explore the differences in ChatGPT’s behavior over the last few months. We discuss th...

4 Sep 202342min

Why Deep Networks and Brains Learn Similar Features with Sophia Sanborn - #644

Why Deep Networks and Brains Learn Similar Features with Sophia Sanborn - #644

Today we’re joined by Sophia Sanborn, a postdoctoral scholar at the University of California, Santa Barbara. In our conversation with Sophia, we explore the concept of universality between neural repr...

28 Aug 202345min

Populärt inom Politik & nyheter

svenska-fall
aftonbladet-krim
p3-krim
rss-krimstad
spar
fordomspodden
flashback-forever
rss-sanning-konsekvens
aftonbladet-daily
rss-vad-fan-hande
motiv
rss-expressen-dok
rss-frandfors-horna
rss-krimreportrarna
dagens-eko
politiken
krimmagasinet
rss-flodet
rss-aftonbladet-krim
kungligt