Quantizing Transformers by Helping Attention Heads Do Nothing with Markus Nagel - #663

Quantizing Transformers by Helping Attention Heads Do Nothing with Markus Nagel - #663

Today we’re joined by Markus Nagel, research scientist at Qualcomm AI Research, who helps us kick off our coverage of NeurIPS 2023. In our conversation with Markus, we cover his accepted papers at the conference, along with other work presented by Qualcomm AI Research scientists. Markus’ first paper, Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothing, focuses on tackling activation quantization issues introduced by the attention mechanism and how to solve them. We also discuss Pruning vs Quantization: Which is Better?, which focuses on comparing the effectiveness of these two methods in achieving model weight compression. Additional papers discussed focus on topics like using scalarization in multitask and multidomain learning to improve training and inference, using diffusion models for a sequence of state models and actions, applying geometric algebra with equivariance to transformers, and applying a deductive verification of chain of thought reasoning performed by LLMs. The complete show notes for this episode can be found at twimlai.com/go/663.

Episoder(781)

Multilingual LLMs and the Values Divide in AI with Sara Hooker - #651

Multilingual LLMs and the Values Divide in AI with Sara Hooker - #651

Today we’re joined by Sara Hooker, director at Cohere and head of Cohere For AI, Cohere’s research lab. In our conversation with Sara, we explore some of the challenges with multilingual models like p...

16 Okt 20231h 18min

Scaling Multi-Modal Generative AI with Luke Zettlemoyer - #650

Scaling Multi-Modal Generative AI with Luke Zettlemoyer - #650

Today we’re joined by Luke Zettlemoyer, professor at University of Washington and a research manager at Meta. In our conversation with Luke, we cover multimodal generative AI, the effect of data on mo...

9 Okt 202338min

Pushing Back on AI Hype with Alex Hanna - #649

Pushing Back on AI Hype with Alex Hanna - #649

Today we’re joined by Alex Hanna, the Director of Research at the Distributed AI Research Institute (DAIR). In our conversation with Alex, we discuss the topic of AI hype and the importance of tacklin...

2 Okt 202349min

Personalization for Text-to-Image Generative AI with Nataniel Ruiz - #648

Personalization for Text-to-Image Generative AI with Nataniel Ruiz - #648

Today we’re joined by Nataniel Ruiz, a research scientist at Google. In our conversation with Nataniel, we discuss his recent work around personalization for text-to-image AI models. Specifically, we ...

25 Sep 202344min

Ensuring LLM Safety for Production Applications with Shreya Rajpal - #647

Ensuring LLM Safety for Production Applications with Shreya Rajpal - #647

Today we’re joined by Shreya Rajpal, founder and CEO of Guardrails AI. In our conversation with Shreya, we discuss ensuring the safety and reliability of language models for production applications. W...

18 Sep 202340min

What’s Next in LLM Reasoning? with Roland Memisevic - #646

What’s Next in LLM Reasoning? with Roland Memisevic - #646

Today we’re joined by Roland Memisevic, a senior director at Qualcomm AI Research. In our conversation with Roland, we discuss the significance of language in humanlike AI systems and the advantages a...

11 Sep 202359min

Is ChatGPT Getting Worse? with James Zou - #645

Is ChatGPT Getting Worse? with James Zou - #645

Today we’re joined by James Zou, an assistant professor at Stanford University. In our conversation with James, we explore the differences in ChatGPT’s behavior over the last few months. We discuss th...

4 Sep 202342min

Why Deep Networks and Brains Learn Similar Features with Sophia Sanborn - #644

Why Deep Networks and Brains Learn Similar Features with Sophia Sanborn - #644

Today we’re joined by Sophia Sanborn, a postdoctoral scholar at the University of California, Santa Barbara. In our conversation with Sophia, we explore the concept of universality between neural repr...

28 Aug 202345min

Populært innen Politikk og nyheter

giver-og-gjengen-vg
aftenpodden
forklart
i-retten
popradet
stopp-verden
aftenpodden-usa
lydartikler-fra-aftenposten
rss-gukild-johaug
det-store-bildet
fotballpodden-2
dine-penger-pengeradet
nokon-ma-ga
rss-ness
hanna-de-heldige
aftenbla-bla
frokostshowet-pa-p5
rss-penger-polser-og-politikk
e24-podden
rss-utenrikskomiteen-med-bogen-og-grasvik