Mamba, Mamba-2 and Post-Transformer Architectures for Generative AI with Albert Gu - #693

Mamba, Mamba-2 and Post-Transformer Architectures for Generative AI with Albert Gu - #693

Today, we're joined by Albert Gu, assistant professor at Carnegie Mellon University, to discuss his research on post-transformer architectures for multi-modal foundation models, with a focus on state-space models in general and Albert’s recent Mamba and Mamba-2 papers in particular. We dig into the efficiency of the attention mechanism and its limitations in handling high-resolution perceptual modalities, and the strengths and weaknesses of transformer architectures relative to alternatives for various tasks. We dig into the role of tokenization and patching in transformer pipelines, emphasizing how abstraction and semantic relationships between tokens underpin the model's effectiveness, and explore how this relates to the debate between handcrafted pipelines versus end-to-end architectures in machine learning. Additionally, we touch on the evolving landscape of hybrid models which incorporate elements of attention and state, the significance of state update mechanisms in model adaptability and learning efficiency, and the contribution and adoption of state-space models like Mamba and Mamba-2 in academia and industry. Lastly, Albert shares his vision for advancing foundation models across diverse modalities and applications. The complete show notes for this episode can be found at https://twimlai.com/go/693.

Avsnitt(779)

Graph Analytic Systems with Zachary Hanif - TWiML Talk #188

Graph Analytic Systems with Zachary Hanif - TWiML Talk #188

In this, the final episode of our Strata Data Conference series, we’re joined by Zachary Hanif, Director of Machine Learning at Capital One’s Center for Machine Learning. We start our discussion wit...

8 Okt 201854min

Diversification in Recommender Systems with Ahsan Ashraf - TWiML Talk #187

Diversification in Recommender Systems with Ahsan Ashraf - TWiML Talk #187

In this episode of our Strata Data conference series, we’re joined by Ahsan Ashraf, data scientist at Pinterest. We discuss his presentation, “Diversification in recommender systems: Using topical var...

4 Okt 201844min

The Fastai v1 Deep Learning Framework with Jeremy Howard - TWiML Talk #186

The Fastai v1 Deep Learning Framework with Jeremy Howard - TWiML Talk #186

In today's episode we're presenting a special conversation with Jeremy Howard, founder and researcher at Fast.ai. This episode is being released today in conjunction with the company’s announcement of...

2 Okt 20181h 11min

Federated ML for Edge Applications with Justin Norman - TWiML Talk #185

Federated ML for Edge Applications with Justin Norman - TWiML Talk #185

In this episode we’re joined by Justin Norman, Director of Research and Data Science Services at Cloudera Fast Forward Labs. In my chat with Justin we start with an update on the company before diving...

27 Sep 201847min

Exploring Dark Energy & Star Formation w/ ML with Viviana Acquaviva - TWiML Talk #184

Exploring Dark Energy & Star Formation w/ ML with Viviana Acquaviva - TWiML Talk #184

In today’s episode of our Strata Data series, we’re joined by Viviana Acquaviva, Associate Professor at City Tech, the New York City College of Technology. In our conversation, we discuss an ongoing p...

26 Sep 201840min

Document Vectors in the Wild with James Dreiss - TWiML Talk #183

Document Vectors in the Wild with James Dreiss - TWiML Talk #183

In this episode of our Strata Data series we’re joined by James Dreiss, Senior Data Scientist at international news syndicate Reuters. James and I sat down to discuss his talk from the conference “Doc...

24 Sep 201840min

Applied Machine Learning for Publishers with Naveed Ahmad - TWiML Talk #182

Applied Machine Learning for Publishers with Naveed Ahmad - TWiML Talk #182

In today’s episode we’re joined by Naveed Ahmad, Senior Director of data engineering and machine learning at Hearst Newspapers. In our conversation, we discuss into the role of ML at Hearst, including...

20 Sep 201839min

Anticipating Superintelligence with Nick Bostrom - TWiML Talk #181

Anticipating Superintelligence with Nick Bostrom - TWiML Talk #181

In this episode, we’re joined by Nick Bostrom, professor at the University of Oxford and head of the Future of Humanity Institute, a multidisciplinary institute focused on answering big-picture questi...

17 Sep 201844min

Populärt inom Politik & nyheter

motiv
p3-krim
spar
svenska-fall
flashback-forever
rss-krimstad
rss-viva-fotboll
rss-sanning-konsekvens
aftonbladet-daily
aftonbladet-krim
rss-vad-fan-hande
rss-krimreportrarna
olyckan-inifran
rss-frandfors-horna
fordomspodden
dagens-eko
rss-flodet
svd-ledarredaktionen
politiken
rss-aftonbladet-krim