Grokking, Generalization Collapse, and the Dynamics of Training Deep Neural Networks with Charles Martin - #734

Grokking, Generalization Collapse, and the Dynamics of Training Deep Neural Networks with Charles Martin - #734

Today, we're joined by Charles Martin, founder of Calculation Consulting, to discuss Weight Watcher, an open-source tool for analyzing and improving Deep Neural Networks (DNNs) based on principles from theoretical physics. We explore the foundations of the Heavy-Tailed Self-Regularization (HTSR) theory that underpins it, which combines random matrix theory and renormalization group ideas to uncover deep insights about model training dynamics. Charles walks us through WeightWatcher’s ability to detect three distinct learning phases—underfitting, grokking, and generalization collapse—and how its signature “layer quality” metric reveals whether individual layers are underfit, overfit, or optimally tuned. Additionally, we dig into the complexities involved in fine-tuning models, the surprising correlation between model optimality and hallucination, the often-underestimated challenges of search relevance, and their implications for RAG. Finally, Charles shares his insights into real-world applications of generative AI and his lessons learned from working in the field. The complete show notes for this episode can be found at https://twimlai.com/go/734.

Episoder(781)

AI Engineering Pitfalls with Chip Huyen - #715

AI Engineering Pitfalls with Chip Huyen - #715

Today, we're joined by Chip Huyen, independent researcher and writer to discuss her new book, “AI Engineering.” We dig into the definition of AI engineering, its key differences from traditional machi...

21 Jan 202557min

Evolving MLOps Platforms for Generative AI and Agents with Abhijit Bose - #714

Evolving MLOps Platforms for Generative AI and Agents with Abhijit Bose - #714

Today, we're joined by Abhijit Bose, head of enterprise AI and ML platforms at Capital One to discuss the evolution of the company’s approach and insights on Generative AI and platform best practices....

13 Jan 202558min

Why Agents Are Stupid & What We Can Do About It with Dan Jeffries - #713

Why Agents Are Stupid & What We Can Do About It with Dan Jeffries - #713

Today, we're joined by Dan Jeffries, founder and CEO of Kentauros AI to discuss the challenges currently faced by those developing advanced AI agents. We dig into how Dan defines agents and distinguis...

16 Des 20241h 8min

Automated Reasoning to Prevent LLM Hallucination with Byron Cook - #712

Automated Reasoning to Prevent LLM Hallucination with Byron Cook - #712

Today, we're joined by Byron Cook, VP and distinguished scientist in the Automated Reasoning Group at AWS to dig into the underlying technology behind the newly announced Automated Reasoning Checks fe...

9 Des 202456min

AI at the Edge: Qualcomm AI Research at NeurIPS 2024 with Arash Behboodi - #711

AI at the Edge: Qualcomm AI Research at NeurIPS 2024 with Arash Behboodi - #711

Today, we're joined by Arash Behboodi, director of engineering at Qualcomm AI Research to discuss the papers and workshops Qualcomm will be presenting at this year’s NeurIPS conference. We dig into th...

3 Des 202454min

AI for Network Management with Shirley Wu - #710

AI for Network Management with Shirley Wu - #710

Today, we're joined by Shirley Wu, senior director of software engineering at Juniper Networks to discuss how machine learning and artificial intelligence are transforming network management. We explo...

19 Nov 202453min

Why Your RAG System Is Broken, and How to Fix It with Jason Liu - #709

Why Your RAG System Is Broken, and How to Fix It with Jason Liu - #709

Today, we're joined by Jason Liu, freelance AI consultant, advisor, and creator of the Instructor library to discuss all things retrieval-augmented generation (RAG). We dig into the tactical and strat...

11 Nov 202458min

An Agentic Mixture of Experts for DevOps with Sunil Mallya - #708

An Agentic Mixture of Experts for DevOps with Sunil Mallya - #708

Today we're joined by Sunil Mallya, CTO and co-founder of Flip AI. We discuss Flip’s incident debugging system for DevOps, which was built using a custom mixture of experts (MoE) large language model ...

4 Nov 20241h 15min

Populært innen Politikk og nyheter

giver-og-gjengen-vg
aftenpodden
forklart
popradet
aftenpodden-usa
stopp-verden
lydartikler-fra-aftenposten
rss-gukild-johaug
det-store-bildet
fotballpodden-2
i-retten
dine-penger-pengeradet
rss-ness
nokon-ma-ga
aftenbla-bla
hanna-de-heldige
frokostshowet-pa-p5
rss-penger-polser-og-politikk
bt-dokumentar-2
rss-dannet-uten-piano