Quantizing Transformers by Helping Attention Heads Do Nothing with Markus Nagel - #663

Quantizing Transformers by Helping Attention Heads Do Nothing with Markus Nagel - #663

Today we’re joined by Markus Nagel, research scientist at Qualcomm AI Research, who helps us kick off our coverage of NeurIPS 2023. In our conversation with Markus, we cover his accepted papers at the conference, along with other work presented by Qualcomm AI Research scientists. Markus’ first paper, Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothing, focuses on tackling activation quantization issues introduced by the attention mechanism and how to solve them. We also discuss Pruning vs Quantization: Which is Better?, which focuses on comparing the effectiveness of these two methods in achieving model weight compression. Additional papers discussed focus on topics like using scalarization in multitask and multidomain learning to improve training and inference, using diffusion models for a sequence of state models and actions, applying geometric algebra with equivariance to transformers, and applying a deductive verification of chain of thought reasoning performed by LLMs. The complete show notes for this episode can be found at twimlai.com/go/663.

Avsnitt(779)

AI for Materials Discovery with Greg Mulholland - TWiML Talk #148

AI for Materials Discovery with Greg Mulholland - TWiML Talk #148

In this episode I’m joined by Greg Mulholland, Founder and CEO of Citrine Informatics, which is applying AI to the discovery and development of new materials. Greg and I start out with an exploration ...

7 Juni 201842min

Data Innovation & AI at Capital One with Adam Wenchel - TWiML Talk #147

Data Innovation & AI at Capital One with Adam Wenchel - TWiML Talk #147

In this episode I’m joined by Adam Wenchel, vice president of AI and Data Innovation at Capital One, to discuss how Machine Learning & AI are being integrated into their day-to-day practices, and how ...

4 Juni 201845min

Deep Gradient Compression for Distributed Training with Song Han - TWiML Talk #146

Deep Gradient Compression for Distributed Training with Song Han - TWiML Talk #146

On today’s show I chat with Song Han, assistant professor in MIT’s EECS department, about his research on Deep Gradient Compression. In our conversation, we explore the challenge of distributed traini...

31 Maj 201846min

Masked Autoregressive Flow for Density Estimation with George Papamakarios - TWiML Talk #145

Masked Autoregressive Flow for Density Estimation with George Papamakarios - TWiML Talk #145

In this episode, University of Edinburgh Phd student George Papamakarios and I discuss his paper “Masked Autoregressive Flow for Density Estimation.” George walks us through the idea of Masked Autoreg...

28 Maj 201834min

Training Data for Computer Vision at Figure Eight with Qazaleh Mirsharif - TWiML Talk #144

Training Data for Computer Vision at Figure Eight with Qazaleh Mirsharif - TWiML Talk #144

For today’s show, the last in our TrainAI series, I'm joined by Qazaleh Mirsharif, a machine learning scientist working on computer vision at Figure Eight. Qazaleh and I caught up at the TrainAI confe...

25 Maj 201821min

Agile Data Science with Sarah Aerni - TWiML Talk #143

Agile Data Science with Sarah Aerni - TWiML Talk #143

Today we continue our TrainAI series with Sarah Aerni, Director of Data Science at Salesforce Einstein. Sarah and I sat down at the TrainAI conference to discuss her talk “Notes from the Field: The Pl...

24 Maj 201838min

Tensor Operations for Machine Learning with Anima Anandkumar - TWiML Talk #142

Tensor Operations for Machine Learning with Anima Anandkumar - TWiML Talk #142

In this episode of our TrainAI series, I sit down with Anima Anandkumar, Bren Professor at Caltech and Principal Scientist with Amazon Web Services. Anima joined me to discuss the research coming out ...

23 Maj 201834min

Deep Learning for Live-Cell Imaging with David Van Valen - TWiML Talk #141

Deep Learning for Live-Cell Imaging with David Van Valen - TWiML Talk #141

In today’s show, I sit down with David Van Valen, assistant professor of Bioengineering & Biology at Caltech. David joined me after his talk at the Figure Eight TrainAI conference to chat about his re...

22 Maj 201837min

Populärt inom Politik & nyheter

aftonbladet-krim
motiv
p3-krim
rss-krimstad
fordomspodden
rss-viva-fotboll
flashback-forever
svenska-fall
rss-sanning-konsekvens
aftonbladet-daily
svd-dokumentara-berattelser-2
spar
rss-krimreportrarna
rss-vad-fan-hande
rss-frandfors-horna
krimmagasinet
olyckan-inifran
rss-aftonbladet-krim
dagens-eko
grans