Mamba, Mamba-2 and Post-Transformer Architectures for Generative AI with Albert Gu - #693

Mamba, Mamba-2 and Post-Transformer Architectures for Generative AI with Albert Gu - #693

Today, we're joined by Albert Gu, assistant professor at Carnegie Mellon University, to discuss his research on post-transformer architectures for multi-modal foundation models, with a focus on state-space models in general and Albert’s recent Mamba and Mamba-2 papers in particular. We dig into the efficiency of the attention mechanism and its limitations in handling high-resolution perceptual modalities, and the strengths and weaknesses of transformer architectures relative to alternatives for various tasks. We dig into the role of tokenization and patching in transformer pipelines, emphasizing how abstraction and semantic relationships between tokens underpin the model's effectiveness, and explore how this relates to the debate between handcrafted pipelines versus end-to-end architectures in machine learning. Additionally, we touch on the evolving landscape of hybrid models which incorporate elements of attention and state, the significance of state update mechanisms in model adaptability and learning efficiency, and the contribution and adoption of state-space models like Mamba and Mamba-2 in academia and industry. Lastly, Albert shares his vision for advancing foundation models across diverse modalities and applications. The complete show notes for this episode can be found at https://twimlai.com/go/693.

Avsnitt(781)

Patterns and Middleware for LLM Applications with Kyle Roche - #659

Patterns and Middleware for LLM Applications with Kyle Roche - #659

Today we’re joined by Kyle Roche, founder and CEO of Griptape to discuss patterns and middleware for LLM applications. We dive into the emerging patterns for developing LLM applications, such as off p...

11 Dec 202335min

AI Access and Inclusivity as a Technical Challenge with Prem Natarajan - #658

AI Access and Inclusivity as a Technical Challenge with Prem Natarajan - #658

Today we’re joined by Prem Natarajan, chief scientist and head of enterprise AI at Capital One. In our conversation, we discuss AI access and inclusivity as technical challenges and explore some of Pr...

4 Dec 202341min

Building LLM-Based Applications with Azure OpenAI with Jay Emery - #657

Building LLM-Based Applications with Azure OpenAI with Jay Emery - #657

Today we’re joined by Jay Emery, director of technical sales & architecture at Microsoft Azure. In our conversation with Jay, we discuss the challenges faced by organizations when building LLM-based a...

28 Nov 202343min

Visual Generative AI Ecosystem Challenges with Richard Zhang - #656

Visual Generative AI Ecosystem Challenges with Richard Zhang - #656

Today we’re joined by Richard Zhang, senior research scientist at Adobe Research. In our conversation with Richard, we explore the research challenges that arise when regarding visual generative AI fr...

20 Nov 202340min

Deploying Edge and Embedded AI Systems with Heather Gorr - #655

Deploying Edge and Embedded AI Systems with Heather Gorr - #655

Today we’re joined by Heather Gorr, principal MATLAB product marketing manager at MathWorks. In our conversation with Heather, we discuss the deployment of AI models to hardware devices and embedded A...

13 Nov 202338min

AI Sentience, Agency and Catastrophic Risk with Yoshua Bengio - #654

AI Sentience, Agency and Catastrophic Risk with Yoshua Bengio - #654

Today we’re joined by Yoshua Bengio, professor at Université de Montréal. In our conversation with Yoshua, we discuss AI safety and the potentially catastrophic risks of its misuse. Yoshua highlights ...

6 Nov 202348min

Delivering AI Systems in Highly Regulated Environments with Miriam Friedel - #653

Delivering AI Systems in Highly Regulated Environments with Miriam Friedel - #653

Today we’re joined by Miriam Friedel, senior director of ML engineering at Capital One. In our conversation with Miriam, we discuss some of the challenges faced when delivering machine learning tools ...

30 Okt 202344min

Mental Models for Advanced ChatGPT Prompting with Riley Goodside - #652

Mental Models for Advanced ChatGPT Prompting with Riley Goodside - #652

Today we’re joined by Riley Goodside, staff prompt engineer at Scale AI. In our conversation with Riley, we explore LLM capabilities and limitations, prompt engineering, and the mental models required...

23 Okt 202339min

Populärt inom Politik & nyheter

aftonbladet-krim
svenska-fall
p3-krim
rss-krimstad
fordomspodden
spar
flashback-forever
rss-sanning-konsekvens
rss-expressen-dok
aftonbladet-daily
motiv
rss-vad-fan-hande
rss-aftonbladet-krim
blenda-2
dagens-eko
rss-frandfors-horna
olyckan-inifran
grans
krimmagasinet
politiken