Mamba, Mamba-2 and Post-Transformer Architectures for Generative AI with Albert Gu - #693

Mamba, Mamba-2 and Post-Transformer Architectures for Generative AI with Albert Gu - #693

Today, we're joined by Albert Gu, assistant professor at Carnegie Mellon University, to discuss his research on post-transformer architectures for multi-modal foundation models, with a focus on state-space models in general and Albert’s recent Mamba and Mamba-2 papers in particular. We dig into the efficiency of the attention mechanism and its limitations in handling high-resolution perceptual modalities, and the strengths and weaknesses of transformer architectures relative to alternatives for various tasks. We dig into the role of tokenization and patching in transformer pipelines, emphasizing how abstraction and semantic relationships between tokens underpin the model's effectiveness, and explore how this relates to the debate between handcrafted pipelines versus end-to-end architectures in machine learning. Additionally, we touch on the evolving landscape of hybrid models which incorporate elements of attention and state, the significance of state update mechanisms in model adaptability and learning efficiency, and the contribution and adoption of state-space models like Mamba and Mamba-2 in academia and industry. Lastly, Albert shares his vision for advancing foundation models across diverse modalities and applications. The complete show notes for this episode can be found at https://twimlai.com/go/693.

Jaksot(778)

AI for Athlete Optimization with Sinead Flahive - TWiML Talk #155

AI for Athlete Optimization with Sinead Flahive - TWiML Talk #155

This week we’re excited to kick off a series of shows on AI in sports. In this episode I'm joined by Sinead Flahive, data scientist at Dublin, Ireland based Kitman Labs to discuss Kitman’s Athlete Opt...

25 Kesä 201840min

Omni-Channel Customer Experiences with Vince Jeffs - TWiML Talk #154

Omni-Channel Customer Experiences with Vince Jeffs - TWiML Talk #154

In this, the final episode of our PegaWorld series I’m joined by Vince Jeffs, Senior Director of Product Strategy for AI and Decisioning at Pegasystems. Vince and I had a great talk about the role AI ...

21 Kesä 201843min

Workforce Intelligence for Automation & Productivity with Michael Kempe - TWiML Talk #153

Workforce Intelligence for Automation & Productivity with Michael Kempe - TWiML Talk #153

In this episode of our PegaWorld series, I’m joined by Michael Kempe, chief operating officer at global share registry and financial services provider Link Market Services. In the interview, Michael a...

20 Kesä 201836min

Data Platforms for Decision Automation at Scotiabank with Jim Saleh - TWiML Talk #152

Data Platforms for Decision Automation at Scotiabank with Jim Saleh - TWiML Talk #152

In this show, part of our PegaWorld 18 series, I'm joined by Jim Saleh, Senior Director of process and decision automation at Scotiabank. Jim is tasked with helping the bank transition from a world wh...

19 Kesä 201832min

Towards the Self-Driving Enterprise with Kirk Borne - TWiML Talk #151

Towards the Self-Driving Enterprise with Kirk Borne - TWiML Talk #151

In this show, the first of our PegaWorld 18 series, I'm joined by Kirk Borne, Principal Data Scientist at management consulting firm Booz Allen Hamilton. In our conversation, Kirk shares his views on ...

18 Kesä 201841min

How a Global Energy Company Adopts ML & AI with Nicholas Osborn - TWiML Talk #150

How a Global Energy Company Adopts ML & AI with Nicholas Osborn - TWiML Talk #150

On today’s show I’m excited to share this interview with Nick Osborn, a longtime listener of the show and Leader of the Global Machine Learning Project Management Office at AES Corporation, a Fortune ...

14 Kesä 201846min

Problem Formulation for Machine Learning with Romer Rosales - TWiML Talk #149

Problem Formulation for Machine Learning with Romer Rosales - TWiML Talk #149

In this episode, i'm joined by Romer Rosales, Director of AI at LinkedIn. We begin with a discussion of graphical models and approximate probability inference, and he helps me make an important connec...

11 Kesä 201850min

AI for Materials Discovery with Greg Mulholland - TWiML Talk #148

AI for Materials Discovery with Greg Mulholland - TWiML Talk #148

In this episode I’m joined by Greg Mulholland, Founder and CEO of Citrine Informatics, which is applying AI to the discovery and development of new materials. Greg and I start out with an exploration ...

7 Kesä 201842min

Suosittua kategoriassa Politiikka ja uutiset

aikalisa
tervo-halme
rss-ootsa-kuullut-tasta
ootsa-kuullut-tasta-2
politiikan-puskaradio
viisupodi
rss-vaalirankkurit-podcast
rss-podme-livebox
et-sa-noin-voi-sanoo-esittaa
otetaan-yhdet
linda-maria
io-techin-tekniikkapodcast
rss-tasta-on-kyse-ivan-puopolo-verkkouutiset
rikosmyytit
rss-polikulaari-humanisti-vastaa-ja-muut-ts-podcastit
viela-yksi-sivu
rss-uusi-juttu
rss-aika-ankkuri
rss-kaikki-uusiksi
rss-merja-mahkan-rahat