Quantizing Transformers by Helping Attention Heads Do Nothing with Markus Nagel - #663

Quantizing Transformers by Helping Attention Heads Do Nothing with Markus Nagel - #663

Today we’re joined by Markus Nagel, research scientist at Qualcomm AI Research, who helps us kick off our coverage of NeurIPS 2023. In our conversation with Markus, we cover his accepted papers at the conference, along with other work presented by Qualcomm AI Research scientists. Markus’ first paper, Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothing, focuses on tackling activation quantization issues introduced by the attention mechanism and how to solve them. We also discuss Pruning vs Quantization: Which is Better?, which focuses on comparing the effectiveness of these two methods in achieving model weight compression. Additional papers discussed focus on topics like using scalarization in multitask and multidomain learning to improve training and inference, using diffusion models for a sequence of state models and actions, applying geometric algebra with equivariance to transformers, and applying a deductive verification of chain of thought reasoning performed by LLMs. The complete show notes for this episode can be found at twimlai.com/go/663.

Episoder(779)

Systems and Software for Machine Learning at Scale with Jeff Dean - TWiML Talk #124

Systems and Software for Machine Learning at Scale with Jeff Dean - TWiML Talk #124

In this episode I’m joined by Jeff Dean, Google Senior Fellow and head of the company’s deep learning research team Google Brain, who I had a chance to sit down with last week at the Googleplex in Mou...

2 Apr 201854min

Semantic Segmentation of 3D Point Clouds with Lyne Tchapmi - TWiML Talk #123

Semantic Segmentation of 3D Point Clouds with Lyne Tchapmi - TWiML Talk #123

In this episode I’m joined by Lyne Tchapmi, PhD student in the Stanford Computational Vision and Geometry Lab, to discuss her paper, “SEGCloud: Semantic Segmentation of 3D Point Clouds.” SEGCloud is a...

29 Mar 201836min

Predicting Cardiovascular Risk Factors from Eye Images with Ryan Poplin - TWiML Talk #122

Predicting Cardiovascular Risk Factors from Eye Images with Ryan Poplin - TWiML Talk #122

In this episode, I'm joined by Google Research Scientist Ryan Poplin, who recently co-authored the paper “Prediction of cardiovascular risk factors from retinal fundus photographs via deep learning.” ...

26 Mar 201842min

Reproducibility and the Philosophy of Data with Clare Gollnick - TWiML Talk #121

Reproducibility and the Philosophy of Data with Clare Gollnick - TWiML Talk #121

In this episode, i'm joined by Clare Gollnick, CTO of Terbium Labs, to discuss her thoughts on the “reproducibility crisis” currently haunting the scientific landscape. For a little background, a “Nat...

22 Mar 201838min

Surveying the Connected Car Landscape with GK Senthil - TWiML Talk #120

Surveying the Connected Car Landscape with GK Senthil - TWiML Talk #120

In this episode, I’m joined by GK Senthil, director & chief product owner for innovation at Toyota Connected. GK and I spoke about some of the potential opportunities and challenges for smart cars. We...

19 Mar 201830min

Adversarial Attacks Against Reinforcement Learning Agents with Ian Goodfellow & Sandy Huang

Adversarial Attacks Against Reinforcement Learning Agents with Ian Goodfellow & Sandy Huang

In this episode, I’m joined by Ian Goodfellow, Staff Research Scientist at Google Brain and Sandy Huang, Phd Student in the EECS department at UC Berkeley, to discuss their work on the paper Adversari...

15 Mar 201847min

Towards Abstract Robotic Understanding with Raja Chatila - TWiML Talk #118

Towards Abstract Robotic Understanding with Raja Chatila - TWiML Talk #118

In this episode, we're joined by Raja Chatila, director of Intelligent Systems and Robotics at Pierre and Marie Curie University in Paris, and executive committee chair of the IEEE global initiative o...

12 Mar 201847min

Discovering Exoplanets w/ Deep Learning with Chris Shallue - TWiML Talk #117

Discovering Exoplanets w/ Deep Learning with Chris Shallue - TWiML Talk #117

Earlier this week, I had a chance to speak with Chris Shallue, Senior Software Engineer on the Google Brain Team, about his project and paper on “Exploring Exoplanets with Deep Learning.” This is a gr...

8 Mar 201845min

Populært innen Politikk og nyheter

giver-og-gjengen-vg
aftenpodden-usa
aftenpodden
i-retten
forklart
stopp-verden
popradet
fotballpodden-2
rss-gukild-johaug
nokon-ma-ga
det-store-bildet
dine-penger-pengeradet
bt-dokumentar-2
aftenbla-bla
hanna-de-heldige
rss-penger-polser-og-politikk
rss-dannet-uten-piano
frokostshowet-pa-p5
rss-ness
e24-podden