Mamba, Mamba-2 and Post-Transformer Architectures for Generative AI with Albert Gu - #693

Mamba, Mamba-2 and Post-Transformer Architectures for Generative AI with Albert Gu - #693

Today, we're joined by Albert Gu, assistant professor at Carnegie Mellon University, to discuss his research on post-transformer architectures for multi-modal foundation models, with a focus on state-space models in general and Albert’s recent Mamba and Mamba-2 papers in particular. We dig into the efficiency of the attention mechanism and its limitations in handling high-resolution perceptual modalities, and the strengths and weaknesses of transformer architectures relative to alternatives for various tasks. We dig into the role of tokenization and patching in transformer pipelines, emphasizing how abstraction and semantic relationships between tokens underpin the model's effectiveness, and explore how this relates to the debate between handcrafted pipelines versus end-to-end architectures in machine learning. Additionally, we touch on the evolving landscape of hybrid models which incorporate elements of attention and state, the significance of state update mechanisms in model adaptability and learning efficiency, and the contribution and adoption of state-space models like Mamba and Mamba-2 in academia and industry. Lastly, Albert shares his vision for advancing foundation models across diverse modalities and applications. The complete show notes for this episode can be found at https://twimlai.com/go/693.

Avsnitt(778)

AI for Athlete Optimization with Sinead Flahive - TWiML Talk #155

AI for Athlete Optimization with Sinead Flahive - TWiML Talk #155

This week we’re excited to kick off a series of shows on AI in sports. In this episode I'm joined by Sinead Flahive, data scientist at Dublin, Ireland based Kitman Labs to discuss Kitman’s Athlete Opt...

25 Juni 201840min

Omni-Channel Customer Experiences with Vince Jeffs - TWiML Talk #154

Omni-Channel Customer Experiences with Vince Jeffs - TWiML Talk #154

In this, the final episode of our PegaWorld series I’m joined by Vince Jeffs, Senior Director of Product Strategy for AI and Decisioning at Pegasystems. Vince and I had a great talk about the role AI ...

21 Juni 201843min

Workforce Intelligence for Automation & Productivity with Michael Kempe - TWiML Talk #153

Workforce Intelligence for Automation & Productivity with Michael Kempe - TWiML Talk #153

In this episode of our PegaWorld series, I’m joined by Michael Kempe, chief operating officer at global share registry and financial services provider Link Market Services. In the interview, Michael a...

20 Juni 201836min

Data Platforms for Decision Automation at Scotiabank with Jim Saleh - TWiML Talk #152

Data Platforms for Decision Automation at Scotiabank with Jim Saleh - TWiML Talk #152

In this show, part of our PegaWorld 18 series, I'm joined by Jim Saleh, Senior Director of process and decision automation at Scotiabank. Jim is tasked with helping the bank transition from a world wh...

19 Juni 201832min

Towards the Self-Driving Enterprise with Kirk Borne - TWiML Talk #151

Towards the Self-Driving Enterprise with Kirk Borne - TWiML Talk #151

In this show, the first of our PegaWorld 18 series, I'm joined by Kirk Borne, Principal Data Scientist at management consulting firm Booz Allen Hamilton. In our conversation, Kirk shares his views on ...

18 Juni 201841min

How a Global Energy Company Adopts ML & AI with Nicholas Osborn - TWiML Talk #150

How a Global Energy Company Adopts ML & AI with Nicholas Osborn - TWiML Talk #150

On today’s show I’m excited to share this interview with Nick Osborn, a longtime listener of the show and Leader of the Global Machine Learning Project Management Office at AES Corporation, a Fortune ...

14 Juni 201846min

Problem Formulation for Machine Learning with Romer Rosales - TWiML Talk #149

Problem Formulation for Machine Learning with Romer Rosales - TWiML Talk #149

In this episode, i'm joined by Romer Rosales, Director of AI at LinkedIn. We begin with a discussion of graphical models and approximate probability inference, and he helps me make an important connec...

11 Juni 201850min

AI for Materials Discovery with Greg Mulholland - TWiML Talk #148

AI for Materials Discovery with Greg Mulholland - TWiML Talk #148

In this episode I’m joined by Greg Mulholland, Founder and CEO of Citrine Informatics, which is applying AI to the discovery and development of new materials. Greg and I start out with an exploration ...

7 Juni 201842min

Populärt inom Politik & nyheter

p3-krim
svenska-fall
rss-krimstad
flashback-forever
motiv
rss-viva-fotboll
spar
rss-sanning-konsekvens
aftonbladet-daily
aftonbladet-krim
rss-krimreportrarna
olyckan-inifran
rss-frandfors-horna
rss-vad-fan-hande
fordomspodden
dagens-eko
rss-flodet
politiken
svd-ledarredaktionen
blenda-2