Quantizing Transformers by Helping Attention Heads Do Nothing with Markus Nagel - #663

Quantizing Transformers by Helping Attention Heads Do Nothing with Markus Nagel - #663

Today we’re joined by Markus Nagel, research scientist at Qualcomm AI Research, who helps us kick off our coverage of NeurIPS 2023. In our conversation with Markus, we cover his accepted papers at the conference, along with other work presented by Qualcomm AI Research scientists. Markus’ first paper, Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothing, focuses on tackling activation quantization issues introduced by the attention mechanism and how to solve them. We also discuss Pruning vs Quantization: Which is Better?, which focuses on comparing the effectiveness of these two methods in achieving model weight compression. Additional papers discussed focus on topics like using scalarization in multitask and multidomain learning to improve training and inference, using diffusion models for a sequence of state models and actions, applying geometric algebra with equivariance to transformers, and applying a deductive verification of chain of thought reasoning performed by LLMs. The complete show notes for this episode can be found at twimlai.com/go/663.

Avsnitt(781)

Supporting Food Security in Africa Using ML with Catherine Nakalembe - #611

Supporting Food Security in Africa Using ML with Catherine Nakalembe - #611

Today we conclude our coverage of the 2022 NeurIPS series joined by Catherine Nakalembe, an associate research professor at the University of Maryland, and Africa Program Director under NASA Harvest. ...

9 Jan 20231h 6min

Service Cards and ML Governance with Michael Kearns - #610

Service Cards and ML Governance with Michael Kearns - #610

Today we conclude our AWS re:Invent 2022 series joined by Michael Kearns, a professor in the department of computer and information science at UPenn, as well as an Amazon Scholar. In our conversation,...

2 Jan 202339min

Reinforcement Learning for Personalization at Spotify with Tony Jebara - #609

Reinforcement Learning for Personalization at Spotify with Tony Jebara - #609

Today we continue our NeurIPS 2022 series joined by Tony Jebara, VP of engineering and head of machine learning at Spotify. In our conversation with Tony, we discuss his role at Spotify and how the co...

29 Dec 202241min

Will ChatGPT take my job? - #608

Will ChatGPT take my job? - #608

More than any system before it, ChatGPT has tapped into our enduring fascination with artificial intelligence, raising in a more concrete and present way important questions and fears about what AI is...

26 Dec 202237min

Geospatial Machine Learning at AWS with Kumar Chellapilla - #607

Geospatial Machine Learning at AWS with Kumar Chellapilla - #607

Today we continue our re:Invent 2022 series joined by Kumar Chellapilla, a general manager of ML and AI Services at AWS. We had the opportunity to speak with Kumar after announcing their recent additi...

22 Dec 202236min

Real-Time ML Workflows at Capital One with Disha Singla - #606

Real-Time ML Workflows at Capital One with Disha Singla - #606

Today we’re joined by Disha Singla, a senior director of machine learning engineering at Capital One. In our conversation with Disha, we explore her role as the leader of the Data Insights team at Cap...

19 Dec 202243min

Weakly Supervised Causal Representation Learning with Johann Brehmer - #605

Weakly Supervised Causal Representation Learning with Johann Brehmer - #605

Today we’re excited to kick off our coverage of the 2022 NeurIPS conference with Johann Brehmer, a research scientist at Qualcomm AI Research in Amsterdam. We begin our conversation discussing some of...

15 Dec 202246min

Stable Diffusion & Generative AI with Emad Mostaque - #604

Stable Diffusion & Generative AI with Emad Mostaque - #604

Today we’re excited to kick off our 2022 AWS re:Invent series with a conversation with Emad Mostaque, Founder and CEO of Stability.ai. Stability.ai is a very popular name in the generative AI space at...

12 Dec 202242min

Populärt inom Politik & nyheter

aftonbladet-krim
svenska-fall
p3-krim
rss-krimstad
fordomspodden
rss-expressen-dok
flashback-forever
rss-sanning-konsekvens
motiv
aftonbladet-daily
spar
rss-vad-fan-hande
blenda-2
olyckan-inifran
rss-krimreportrarna
rss-frandfors-horna
rss-flodet
dagens-eko
svd-ledarredaktionen
grans