Mamba, Mamba-2 and Post-Transformer Architectures for Generative AI with Albert Gu - #693

Mamba, Mamba-2 and Post-Transformer Architectures for Generative AI with Albert Gu - #693

Today, we're joined by Albert Gu, assistant professor at Carnegie Mellon University, to discuss his research on post-transformer architectures for multi-modal foundation models, with a focus on state-space models in general and Albert’s recent Mamba and Mamba-2 papers in particular. We dig into the efficiency of the attention mechanism and its limitations in handling high-resolution perceptual modalities, and the strengths and weaknesses of transformer architectures relative to alternatives for various tasks. We dig into the role of tokenization and patching in transformer pipelines, emphasizing how abstraction and semantic relationships between tokens underpin the model's effectiveness, and explore how this relates to the debate between handcrafted pipelines versus end-to-end architectures in machine learning. Additionally, we touch on the evolving landscape of hybrid models which incorporate elements of attention and state, the significance of state update mechanisms in model adaptability and learning efficiency, and the contribution and adoption of state-space models like Mamba and Mamba-2 in academia and industry. Lastly, Albert shares his vision for advancing foundation models across diverse modalities and applications. The complete show notes for this episode can be found at https://twimlai.com/go/693.

Jaksot(779)

Making Algorithms Trustworthy with David Spiegelhalter - TWiML Talk #212

Making Algorithms Trustworthy with David Spiegelhalter - TWiML Talk #212

Today we’re joined by David Spiegelhalter, Chair of Winton Center for Risk and Evidence Communication at Cambridge University and President of the Royal Statistical Society. David, an invited speaker ...

20 Joulu 201823min

Designing Computer Systems for Software with Kunle Olukotun - TWiML Talk #211

Designing Computer Systems for Software with Kunle Olukotun - TWiML Talk #211

Today we’re joined by Kunle Olukotun, Professor in the department of EE and CS at Stanford University, and Chief Technologist at Sambanova Systems. Kunle was an invited speaker at NeurIPS this year, p...

18 Joulu 201855min

Operationalizing Ethical AI with Kathryn Hume - TWiML Talk #210

Operationalizing Ethical AI with Kathryn Hume - TWiML Talk #210

Today we conclude our Trust in AI series with this conversation with Kathryn Hume, VP of Strategy at Integrate AI. We discuss her newly released white paper “Responsible AI in the Consumer Enterprise,...

14 Joulu 201853min

Approaches to Fairness in Machine Learning with Richard Zemel - TWiML Talk #209

Approaches to Fairness in Machine Learning with Richard Zemel - TWiML Talk #209

Today we continue our exploration of Trust in AI with this interview with Richard Zemel, Professor in the department of Computer Science at the University of Toronto and Research Director at Vector In...

12 Joulu 201845min

Trust and AI with Parinaz Sobhani - TWiML Talk #208

Trust and AI with Parinaz Sobhani - TWiML Talk #208

In today’s episode we’re joined by Parinaz Sobhani, Director of Machine Learning at Georgian Partners. In our conversation, Parinaz and I discuss some of the main issues falling under the “trust” um...

11 Joulu 201846min

Unbiased Learning from Biased User Feedback with Thorsten Joachims - TWiML Talk #207

Unbiased Learning from Biased User Feedback with Thorsten Joachims - TWiML Talk #207

In the final episode of our re:Invent series, we're joined by Thorsten Joachims, Professor in the Department of Computer Science at Cornell University. We discuss his presentation “Unbiased Learning f...

7 Joulu 201840min

Language Parsing and Character Mining with Jinho Choi - TWiML Talk #206

Language Parsing and Character Mining with Jinho Choi - TWiML Talk #206

Today we’re joined by Jinho Choi, assistant professor of computer science at Emory University. Jinho presented at the conference on ELIT, their cloud-based NLP platform. In our conversation, we discu...

5 Joulu 201847min

re:Invent Roundup Roundtable 2018 with Dave McCrory and Val Bercovici - TWiML Talk #205

re:Invent Roundup Roundtable 2018 with Dave McCrory and Val Bercovici - TWiML Talk #205

I’m excited to present our second annual re:Invent Roundtable Roundup. This year I’m joined by Dave McCrory, VP of Software Engineering at Wise.io at GE Digital, and Val Bercovici, Founder and CEO of ...

3 Joulu 20181h 7min

Suosittua kategoriassa Politiikka ja uutiset

aikalisa
tervo-halme
rss-ootsa-kuullut-tasta
ootsa-kuullut-tasta-2
politiikan-puskaradio
rss-vaalirankkurit-podcast
viisupodi
rss-podme-livebox
otetaan-yhdet
et-sa-noin-voi-sanoo-esittaa
rss-tasta-on-kyse-ivan-puopolo-verkkouutiset
rss-polikulaari-humanisti-vastaa-ja-muut-ts-podcastit
rss-kaikki-uusiksi
rss-merja-mahkan-rahat
rss-asiastudio
the-ulkopolitist
mtv-uutiset-polloraati
rss-aika-ankkuri
rss-hyvaa-huomenta-bryssel
rss-kuka-mina-olen