Quantizing Transformers by Helping Attention Heads Do Nothing with Markus Nagel - #663

Quantizing Transformers by Helping Attention Heads Do Nothing with Markus Nagel - #663

Today we’re joined by Markus Nagel, research scientist at Qualcomm AI Research, who helps us kick off our coverage of NeurIPS 2023. In our conversation with Markus, we cover his accepted papers at the conference, along with other work presented by Qualcomm AI Research scientists. Markus’ first paper, Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothing, focuses on tackling activation quantization issues introduced by the attention mechanism and how to solve them. We also discuss Pruning vs Quantization: Which is Better?, which focuses on comparing the effectiveness of these two methods in achieving model weight compression. Additional papers discussed focus on topics like using scalarization in multitask and multidomain learning to improve training and inference, using diffusion models for a sequence of state models and actions, applying geometric algebra with equivariance to transformers, and applying a deductive verification of chain of thought reasoning performed by LLMs. The complete show notes for this episode can be found at twimlai.com/go/663.

Episoder(781)

Pytorch: Fast Differentiable Dynamic Graphs in Python with Soumith Chintala - TWiML Talk #70

Pytorch: Fast Differentiable Dynamic Graphs in Python with Soumith Chintala - TWiML Talk #70

This week, we’ll be featuring a series of shows recorded from Strange Loop, a great developer-focused conference that takes place every year right in my backyard! The conference is a multi-disciplinar...

21 Nov 201742min

Accessible Machine Learning for the Enterprise Developer with Ryan Sevey & Jason Montgomery

Accessible Machine Learning for the Enterprise Developer with Ryan Sevey & Jason Montgomery

This week, we’ll be featuring a series of shows recorded from Strange Loop, a great developer-focused conference that takes place every year right in my backyard! The conference is a multi-disciplinar...

20 Nov 201745min

Bridging the Gap Between Academic and Industry Careers with Ross Fadely - TWiML Talk #68

Bridging the Gap Between Academic and Industry Careers with Ross Fadely - TWiML Talk #68

We close out our NYU Future Labs AI Summit interview series with Ross Fadely, a New York based AI lead with Insight Data Science. Insight is an interesting company offering a free seven week post-doct...

16 Nov 201719min

The Limitations of Human-in-the-Loop AI with Dennis Mortensen - TWiML Talk #67

The Limitations of Human-in-the-Loop AI with Dennis Mortensen - TWiML Talk #67

We continue our NYU Future Labs AI Summit interview series with Dennis Mortensen, founder and CEO of X.ai, a company whose AI-based personal assistant Amy helps users with scheduling meetings. I caugh...

13 Nov 201735min

Nexus Lab Cohort 2 - Second Mind - TWiML Talk #66

Nexus Lab Cohort 2 - Second Mind - TWiML Talk #66

The podcast you’re about to hear is the fourth of a series of shows recorded at the NYU Future Labs AI Summit last week in New York City. In this show, I speak with Kul Singh, CEO and Founder of Secon...

9 Nov 201721min

Nexus Lab Cohort 2 - Bite.ai - TWiML Talk #65

Nexus Lab Cohort 2 - Bite.ai - TWiML Talk #65

The podcast you’re about to hear is the second of a series of shows recorded at the NYU Future Labs AI Summit last week in New York City.In this episode, you’ll hear from Bite.ai, a startup founded by...

8 Nov 201726min

Nexus Lab Cohort 2 - Bowtie - TWiML Talk #64

Nexus Lab Cohort 2 - Bowtie - TWiML Talk #64

The podcast you’re about to hear is the second of a series of shows recorded at the NYU Future Labs AI Summit last week in New York City. In this episode, I speak with Ron Fisher and Mike Wang, who, a...

7 Nov 201725min

AI Nexus Lab Cohort 2 - Mt. Cleverest - TWiML Talk #63

AI Nexus Lab Cohort 2 - Mt. Cleverest - TWiML Talk #63

The podcast you’re about to hear is the first of a series of shows recorded at the NYU Future Labs AI Summit last week in New York City. My guests this time around are James Villarrubia and Bernie Pra...

6 Nov 201732min

Populært innen Politikk og nyheter

giver-og-gjengen-vg
aftenpodden
forklart
i-retten
popradet
stopp-verden
aftenpodden-usa
lydartikler-fra-aftenposten
rss-gukild-johaug
det-store-bildet
fotballpodden-2
dine-penger-pengeradet
nokon-ma-ga
rss-ness
hanna-de-heldige
aftenbla-bla
frokostshowet-pa-p5
rss-penger-polser-og-politikk
e24-podden
rss-utenrikskomiteen-med-bogen-og-grasvik