Quantizing Transformers by Helping Attention Heads Do Nothing with Markus Nagel - #663

Quantizing Transformers by Helping Attention Heads Do Nothing with Markus Nagel - #663

Today we’re joined by Markus Nagel, research scientist at Qualcomm AI Research, who helps us kick off our coverage of NeurIPS 2023. In our conversation with Markus, we cover his accepted papers at the conference, along with other work presented by Qualcomm AI Research scientists. Markus’ first paper, Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothing, focuses on tackling activation quantization issues introduced by the attention mechanism and how to solve them. We also discuss Pruning vs Quantization: Which is Better?, which focuses on comparing the effectiveness of these two methods in achieving model weight compression. Additional papers discussed focus on topics like using scalarization in multitask and multidomain learning to improve training and inference, using diffusion models for a sequence of state models and actions, applying geometric algebra with equivariance to transformers, and applying a deductive verification of chain of thought reasoning performed by LLMs. The complete show notes for this episode can be found at twimlai.com/go/663.

Avsnitt(781)

Assessing the Risks of Open AI Models with Sayash Kapoor - #675

Assessing the Risks of Open AI Models with Sayash Kapoor - #675

Today we’re joined by Sayash Kapoor, a Ph.D. student in the Department of Computer Science at Princeton University. Sayash walks us through his paper: "On the Societal Impact of Open Foundation Models...

11 Mars 202440min

OLMo: Everything You Need to Train an Open Source LLM with Akshita Bhagia - #674

OLMo: Everything You Need to Train an Open Source LLM with Akshita Bhagia - #674

Today we’re joined by Akshita Bhagia, a senior research engineer at the Allen Institute for AI. Akshita joins us to discuss OLMo, a new open source language model with 7 billion and 1 billion variants...

4 Mars 202432min

Training Data Locality and Chain-of-Thought Reasoning in LLMs with Ben Prystawski - #673

Training Data Locality and Chain-of-Thought Reasoning in LLMs with Ben Prystawski - #673

Today we’re joined by Ben Prystawski, a PhD student in the Department of Psychology at Stanford University working at the intersection of cognitive science and machine learning. Our conversation cente...

26 Feb 202425min

Reasoning Over Complex Documents with DocLLM with Armineh Nourbakhsh - #672

Reasoning Over Complex Documents with DocLLM with Armineh Nourbakhsh - #672

Today we're joined by Armineh Nourbakhsh of JP Morgan AI Research to discuss the development and capabilities of DocLLM, a layout-aware large language model for multimodal document understanding. Armi...

19 Feb 202445min

Are Emergent Behaviors in LLMs an Illusion? with Sanmi Koyejo - #671

Are Emergent Behaviors in LLMs an Illusion? with Sanmi Koyejo - #671

Today we’re joined by Sanmi Koyejo, assistant professor at Stanford University, to continue our NeurIPS 2024 series. In our conversation, Sanmi discusses his two recent award-winning papers. First, we...

12 Feb 20241h 5min

AI Trends 2024: Reinforcement Learning in the Age of LLMs with Kamyar Azizzadenesheli - #670

AI Trends 2024: Reinforcement Learning in the Age of LLMs with Kamyar Azizzadenesheli - #670

Today we’re joined by Kamyar Azizzadenesheli, a staff researcher at Nvidia, to continue our AI Trends 2024 series. In our conversation, Kamyar updates us on the latest developments in reinforcement le...

5 Feb 20241h 10min

Building and Deploying Real-World RAG Applications with Ram Sriharsha - #669

Building and Deploying Real-World RAG Applications with Ram Sriharsha - #669

Today we’re joined by Ram Sriharsha, VP of engineering at Pinecone. In our conversation, we dive into the topic of vector databases and retrieval augmented generation (RAG). We explore the trade-offs ...

29 Jan 202435min

Nightshade: Data Poisoning to Fight Generative AI with Ben Zhao - #668

Nightshade: Data Poisoning to Fight Generative AI with Ben Zhao - #668

Today we’re joined by Ben Zhao, a Neubauer professor of computer science at the University of Chicago. In our conversation, we explore his research at the intersection of security and generative AI. W...

22 Jan 202439min

Populärt inom Politik & nyheter

aftonbladet-krim
svenska-fall
p3-krim
rss-krimstad
fordomspodden
rss-expressen-dok
flashback-forever
rss-sanning-konsekvens
motiv
aftonbladet-daily
spar
rss-vad-fan-hande
blenda-2
olyckan-inifran
rss-krimreportrarna
rss-frandfors-horna
rss-flodet
dagens-eko
svd-ledarredaktionen
grans