Quantizing Transformers by Helping Attention Heads Do Nothing with Markus Nagel - #663

Quantizing Transformers by Helping Attention Heads Do Nothing with Markus Nagel - #663

Today we’re joined by Markus Nagel, research scientist at Qualcomm AI Research, who helps us kick off our coverage of NeurIPS 2023. In our conversation with Markus, we cover his accepted papers at the conference, along with other work presented by Qualcomm AI Research scientists. Markus’ first paper, Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothing, focuses on tackling activation quantization issues introduced by the attention mechanism and how to solve them. We also discuss Pruning vs Quantization: Which is Better?, which focuses on comparing the effectiveness of these two methods in achieving model weight compression. Additional papers discussed focus on topics like using scalarization in multitask and multidomain learning to improve training and inference, using diffusion models for a sequence of state models and actions, applying geometric algebra with equivariance to transformers, and applying a deductive verification of chain of thought reasoning performed by LLMs. The complete show notes for this episode can be found at twimlai.com/go/663.

Avsnitt(779)

Intelligent Autonomous Robots with Ilia Baranov - TWiML Talk #27

Intelligent Autonomous Robots with Ilia Baranov - TWiML Talk #27

Our first guest in the Industrial AI series is Ilia Baranov, engineering manager at Clearpath Robotics. Ilia is responsible for setting the engineering direction for all of Clearpath’s research platfo...

9 Juni 201753min

Global AI Trends with Ben Lorica - TWiML Talk #26

Global AI Trends with Ben Lorica - TWiML Talk #26

This week I’ve invited my friend Ben Lorica onto the show. Ben is Chief Data Scientist for O’Reilly Media, and Program Director of Strata Data & the O'Reilly A.I. conference. Ben has worked on analyti...

2 Juni 201754min

Offensive vs Defensive Data Science with Deep Varma - TWiML Talk #25

Offensive vs Defensive Data Science with Deep Varma - TWiML Talk #25

This week on the show my guest is Deep Varma, Vice President of Data Engineering at real estate startup Trulia. Deep has run data engineering teams in silicon valley for well over a decade, and is now...

26 Maj 201753min

Reinforcement Learning: The Next Frontier of Gaming with Danny Lange - TWiML Talk #24

Reinforcement Learning: The Next Frontier of Gaming with Danny Lange - TWiML Talk #24

My guest on the show this week is Danny Lange, VP for Machine Learning & AI at video game technology developer Unity Technologies. Danny is well traveled in the world of ML and AI, and has had a hand ...

20 Maj 201754min

Integrating Psycholinguistics into AI with Dominique Simmons - TWiML Talk #23

Integrating Psycholinguistics into AI with Dominique Simmons - TWiML Talk #23

I think you’re really going to enjoy today’s show. Our guest this week is Dominique Simmons, Applied research Scientist at AI tools vendor Dimensional Mechanics. Dominique brings an interesting backgr...

12 Maj 20171h

Deep Neural Nets for Visual Recognition with Matt Zeiler - TWiML Talk #22

Deep Neural Nets for Visual Recognition with Matt Zeiler - TWiML Talk #22

Today we bring you our final interview from backstage at the NYU FutureLabs AI Summit. Our guest this week is Matt Zeiler. Matt graduated from the University of Toronto where he worked with deep learn...

5 Maj 201722min

Engineering the Future of AI with Ruchir Puri - TWiML Talk #21

Engineering the Future of AI with Ruchir Puri - TWiML Talk #21

Today we bring you the second of three interviews we did backstage from the NYU FutureLabs AI Summit, this time with Ruchir Puri. Ruchir is the Chief Architect at IBM Watson as well as an IBM Fellow. ...

28 Apr 201720min

Selling AI to the Enterprise with Kathryn Hume - TWiML Talk #20

Selling AI to the Enterprise with Kathryn Hume - TWiML Talk #20

This week's guest is Kathryn Hume. Kathryn is the President of Fast Forward Labs, which is an independent machine intelligence research company that helps organizations accelerate their data science a...

21 Apr 201723min

Populärt inom Politik & nyheter

motiv
p3-krim
spar
svenska-fall
flashback-forever
rss-krimstad
rss-viva-fotboll
rss-sanning-konsekvens
aftonbladet-daily
aftonbladet-krim
rss-vad-fan-hande
rss-krimreportrarna
olyckan-inifran
rss-frandfors-horna
fordomspodden
dagens-eko
rss-flodet
svd-ledarredaktionen
politiken
rss-aftonbladet-krim