Machine Learning Street Talk (MLST)6 Dec 2020

#032- Simon Kornblith / GoogleAI - SimCLR and Paper Haul!

This week Dr. Tim Scarfe, Sayak Paul and Yannic Kilcher speak with Dr. Simon Kornblith from Google Brain (Ph.D from MIT). Simon is trying to understand how neural nets do what they do. Simon was the second author on the seminal Google AI SimCLR paper. We also cover "Do Wide and Deep Networks learn the same things?", "Whats in a Loss function for Image Classification?", and "Big Self-supervised models are strong semi-supervised learners". Simon used to be a neuroscientist and also gives us the story of his unique journey into ML.

00:00:00 Show Teaser / or "short version"

00:18:34 Show intro

00:22:11 Relationship between neuroscience and machine learning

00:29:28 Similarity analysis and evolution of representations in Neural Networks

00:39:55 Expressability of NNs

00:42:33 Whats in a loss function for image classification

00:46:52 Loss function implications for transfer learning

00:50:44 SimCLR paper

01:00:19 Contrast SimCLR to BYOL

01:01:43 Data augmentation

01:06:35 Universality of image representations

01:09:25 Universality of augmentations

01:23:04 GPT-3

01:25:09 GANs for data augmentation??

01:26:50 Julia language

@skornblith

https://www.linkedin.com/in/simon-kornblith-54b2033a/

https://arxiv.org/abs/2010.15327

Do Wide and Deep Networks Learn the Same Things? Uncovering How Neural Network Representations Vary with Width and Depth

https://arxiv.org/abs/2010.16402

What's in a Loss Function for Image Classification?

https://arxiv.org/abs/2002.05709

A Simple Framework for Contrastive Learning of Visual Representations

https://arxiv.org/abs/2006.10029

Big Self-Supervised Models are Strong Semi-Supervised Learners

Det här avsnittet är hämtat från ett öppet RSS-flöde och publiceras inte av Podme. Det kan innehålla reklam.

Avsnitt(252)

When AI Decides You're a Threat — Brad Carson

Brad Carson was the Army's General Counsel, served two terms in Congress and was Acting Under Secretary of Defense for Personnel and Readiness. He now heads Americans for Responsible Innovation, the A...

31 Maj 1h 20min

Intelligence is collective, not artificial — Prof. Michael I. Jordan (UC Berkeley / Inria)

Michael I. Jordan, described by Science magazine as the most influential computer scientist alive, has never thought of himself as an AI researcher. In this conversation he explains why that distincti...

21 Maj 1h 17min

The AI Models Smart Enough to Know They're Cheating — Beth Barnes & David Rein [METR]

Beth Barnes and David Rein on the one graph that ate the AI timelines discourse, and why the two people who built it are the most careful about how you read it.**SPONSOR**Prolific - Quality data. From...

4 Maj 1h 53min

When AI Discovers The Next Transformer - Robert Lange (Sakana)

Robert Lange, founding researcher at Sakana AI, joins Tim to discuss *Shinka Evolve* — a framework that combines LLMs with evolutionary algorithms to do open-ended program search. The core claim: syst...

13 Mars 1h 18min

"Vibe Coding is a Slot Machine" - Jeremy Howard

Dive into the realities of AI-assisted coding, the origins of modern fine-tuning, and the cognitive science behind machine learning with fast.ai founder Jeremy Howard. In this episode, we unpack why A...

3 Mars 1h 26min

Evolution "Doesn't Need" Mutation - Blaise Agüera y Arcas

What if life itself is just a really sophisticated computer program that wrote itself into existence?Blaise Agüera y Arcas presenting at ALife 2025 — the most technically detailed public walkthrough o...

16 Feb 55min

VAEs Are Energy-Based Models? [Dr. Jeff Beck]

What makes something truly *intelligent?* Is a rock an agent? Could a perfect simulation of your brain actually *be* you? In this fascinating conversation, Dr. Jeff Beck takes us on a journey through ...

25 Jan 46min

Abstraction & Idealization: AI's Plato Problem [Mazviita Chirimuuta]

Professor Mazviita Chirimuuta joins us for a fascinating deep dive into the philosophy of neuroscience and what it really means to understand the mind.*What can neuroscience actually tell us about how...

23 Jan 53min