Mamba, Mamba-2 and Post-Transformer Architectures for Generative AI with Albert Gu - #693

Mamba, Mamba-2 and Post-Transformer Architectures for Generative AI with Albert Gu - #693

Today, we're joined by Albert Gu, assistant professor at Carnegie Mellon University, to discuss his research on post-transformer architectures for multi-modal foundation models, with a focus on state-space models in general and Albert’s recent Mamba and Mamba-2 papers in particular. We dig into the efficiency of the attention mechanism and its limitations in handling high-resolution perceptual modalities, and the strengths and weaknesses of transformer architectures relative to alternatives for various tasks. We dig into the role of tokenization and patching in transformer pipelines, emphasizing how abstraction and semantic relationships between tokens underpin the model's effectiveness, and explore how this relates to the debate between handcrafted pipelines versus end-to-end architectures in machine learning. Additionally, we touch on the evolving landscape of hybrid models which incorporate elements of attention and state, the significance of state update mechanisms in model adaptability and learning efficiency, and the contribution and adoption of state-space models like Mamba and Mamba-2 in academia and industry. Lastly, Albert shares his vision for advancing foundation models across diverse modalities and applications. The complete show notes for this episode can be found at https://twimlai.com/go/693.

Episoder(779)

Targeted Ticket Sales Using Azure ML with the Trail Blazers w/ Mike Schumacher & Chenhui Hu - TWiML Talk #156

Targeted Ticket Sales Using Azure ML with the Trail Blazers w/ Mike Schumacher & Chenhui Hu - TWiML Talk #156

In today’s episode of our AI in Sports series I'm joined by Mike Schumacher, director of business analytics for the Portland Trail Blazers, and Chenhui Hu, a data scientist at Microsoft to discuss how...

26 Jun 201837min

AI for Athlete Optimization with Sinead Flahive - TWiML Talk #155

AI for Athlete Optimization with Sinead Flahive - TWiML Talk #155

This week we’re excited to kick off a series of shows on AI in sports. In this episode I'm joined by Sinead Flahive, data scientist at Dublin, Ireland based Kitman Labs to discuss Kitman’s Athlete Opt...

25 Jun 201840min

Omni-Channel Customer Experiences with Vince Jeffs - TWiML Talk #154

Omni-Channel Customer Experiences with Vince Jeffs - TWiML Talk #154

In this, the final episode of our PegaWorld series I’m joined by Vince Jeffs, Senior Director of Product Strategy for AI and Decisioning at Pegasystems. Vince and I had a great talk about the role AI ...

21 Jun 201843min

Workforce Intelligence for Automation & Productivity with Michael Kempe - TWiML Talk #153

Workforce Intelligence for Automation & Productivity with Michael Kempe - TWiML Talk #153

In this episode of our PegaWorld series, I’m joined by Michael Kempe, chief operating officer at global share registry and financial services provider Link Market Services. In the interview, Michael a...

20 Jun 201836min

Data Platforms for Decision Automation at Scotiabank with Jim Saleh - TWiML Talk #152

Data Platforms for Decision Automation at Scotiabank with Jim Saleh - TWiML Talk #152

In this show, part of our PegaWorld 18 series, I'm joined by Jim Saleh, Senior Director of process and decision automation at Scotiabank. Jim is tasked with helping the bank transition from a world wh...

19 Jun 201832min

Towards the Self-Driving Enterprise with Kirk Borne - TWiML Talk #151

Towards the Self-Driving Enterprise with Kirk Borne - TWiML Talk #151

In this show, the first of our PegaWorld 18 series, I'm joined by Kirk Borne, Principal Data Scientist at management consulting firm Booz Allen Hamilton. In our conversation, Kirk shares his views on ...

18 Jun 201841min

How a Global Energy Company Adopts ML & AI with Nicholas Osborn - TWiML Talk #150

How a Global Energy Company Adopts ML & AI with Nicholas Osborn - TWiML Talk #150

On today’s show I’m excited to share this interview with Nick Osborn, a longtime listener of the show and Leader of the Global Machine Learning Project Management Office at AES Corporation, a Fortune ...

14 Jun 201846min

Problem Formulation for Machine Learning with Romer Rosales - TWiML Talk #149

Problem Formulation for Machine Learning with Romer Rosales - TWiML Talk #149

In this episode, i'm joined by Romer Rosales, Director of AI at LinkedIn. We begin with a discussion of graphical models and approximate probability inference, and he helps me make an important connec...

11 Jun 201850min

Populært innen Politikk og nyheter

giver-og-gjengen-vg
aftenpodden
aftenpodden-usa
forklart
stopp-verden
popradet
det-store-bildet
dine-penger-pengeradet
rss-gukild-johaug
bt-dokumentar-2
lydartikler-fra-aftenposten
hanna-de-heldige
fotballpodden-2
nokon-ma-ga
e24-podden
frokostshowet-pa-p5
aftenbla-bla
rss-ness
rss-penger-polser-og-politikk
rss-dannet-uten-piano