The Startup Powering The Data Behind AGI

The Startup Powering The Data Behind AGI

In this episode of Gradient Dissent, Lukas Biewald talks with the CEO & founder of Surge AI, the billion-dollar company quietly powering the next generation of frontier LLMs. They discuss Surge's origin story, why traditional data labeling is broken, and how their research-focused approach is reshaping how models are trained.

You’ll hear why inter-annotator agreement fails in high-complexity tasks like poetry and math, why synthetic data is often overrated, and how Surge builds rich RL environments to stress-test agentic reasoning. They also go deep on what kinds of data will be critical to future progress in AI—from scientific discovery to multimodal reasoning and personalized alignment.


It’s a rare, behind-the-scenes look into the world of high-quality data generation at scale—straight from the team most frontier labs trust to get it right.


Timestamps:

00:00 – Intro: Who is Edwin Chen?

03:40 – The problem with early data labeling systems

06:20 – Search ranking, clickbait, and product principles

10:05 – Why Surge focused on high-skill, high-quality labeling

13:50 – From Craigslist workers to a billion-dollar business

16:40 – Scaling without funding and avoiding Silicon Valley status games

21:15 – Why most human data platforms lack real tech

25:05 – Detecting cheaters, liars, and low-quality labelers

28:30 – Why inter-annotator agreement is a flawed metric

32:15 – What makes a great poem? Not checkboxes

36:40 – Measuring subjective quality rigorously

40:00 – What types of data are becoming more important

44:15 – Scientific collaboration and frontier research data

47:00 – Multimodal data, Argentinian coding, and hyper-specificity

50:10 – What's wrong with LMSYS and benchmark hacking

53:20 – Personalization and taste in model behavior

56:00 – Synthetic data vs. high-quality human data


Follow Weights & Biases:

https://twitter.com/weights_biases

https://www.linkedin.com/company/wandb

Avsnitt(131)

Peter Skomoroch — Product Management for AI

Peter Skomoroch — Product Management for AI

👨🏻‍💻Our guest on this episode of Gradient Dissent is Peter Skomoroch! Peter is the former head of data products at Workday and LinkedIn. Previously, he was the cofounder and CEO of venture-backed deep learning startup SkipFlag, which was acquired by Workday, and a principal data scientist at LinkedIn. Check out his recent publication: What you need to know about product management for AI https://www.oreilly.com/radar/what-you-need-to-know-about-product-management-for-ai/ Follow Peter on Twitter: https://twitter.com/peteskomoroch And read some of his other work: Pangloss: Fast Entity Linking in Noisy Text Environments Large-Scale Hierarchical Topic Models Visit our podcasts homepage for transcripts and more episodes! www.wandb.com/podcast 🔊 Get our podcast on Soundcloud, Apple, and Spotify! YouTube: https://bit.ly/32NzZvI Apple Podcasts: https://bit.ly/2WdrUvI Spotify: https://bit.ly/2SqtadF We started Weights and Biases to build tools for Machine Learning practitioners because we care a lot about the impact that Machine Learning can have in the world and we love working in the trenches with the people building these models. One of the most fun things about these building tools has been the conversations with these ML practitioners and learning about the interesting things they’re working on. This process has been so fun that we wanted to open it up to the world in the form of our new podcast called Gradient Dissent. We hope you have as much fun listening to it as we had making it! 👩🏼‍🚀Weights and Biases: We’re always free for academics and open source projects. Email carey@wandb.com with any questions or feature suggestions. - Blog: https://www.wandb.com/articles - Gallery: See what you can create with W&B - https://app.wandb.ai/gallery - Continue the conversation on our slack community - http://bit.ly/wandb-forum 🎙Host: Lukas Biewald - https://twitter.com/l2k 👩🏼‍💻Producer: Lavanya Shukla - https://twitter.com/lavanyaai 📹Editor: Cayla Sharp - http://caylasharp.com/

22 Juli 20201h 27min

Josh Tobin — Productionizing ML Models

Josh Tobin — Productionizing ML Models

Josh Tobin is a researcher working at the intersection of machine learning and robotics. His research focuses on applying deep reinforcement learning, generative models, and synthetic data to problems in robotic perception and control. Additionally, he co-organizes a machine learning training program for engineers to learn about production-ready deep learning called Full Stack Deep Learning. https://fullstackdeeplearning.com/ Josh did his PhD in Computer Science at UC Berkeley advised by Pieter Abbeel and was a research scientist at OpenAI for 3 years during his PhD. Finally, Josh created this amazing field guide on troubleshooting deep neural networks: http://josh-tobin.com/assets/pdf/troubleshooting-deep-neural-networks-01-19.pdf Follow Josh on twitter: https://twitter.com/josh_tobin And on his website:http://josh-tobin.com/ Visit our podcasts homepage for transcripts and more episodes! www.wandb.com/podcast 🔊 Get our podcast on Youtube, Apple, and Spotify! Youtube: https://www.youtube.com/playlist?list=PLD80i8An1OEEb1jP0sjEyiLG8ULRXFob_ Apple Podcasts: https://bit.ly/2WdrUvI Spotify: https://bit.ly/2SqtadF We started Weights and Biases to build tools for Machine Learning practitioners because we care a lot about the impact that Machine Learning can have in the world and we love working in the trenches with the people building these models. One of the most fun things about these building tools has been the conversations with these ML practitioners and learning about the interesting things they’re working on. This process has been so fun that we wanted to open it up to the world in the form of our new podcast called Gradient Dissent. We hope you have as much fun listening to it as we had making it! 👩🏼‍🚀Weights and Biases: We’re always free for academics and open source projects. Email carey@wandb.com with any questions or feature suggestions. - Blog: https://www.wandb.com/articles - Gallery: See what you can create with W&B - https://app.wandb.ai/gallery - Continue the conversation on our slack community - http://bit.ly/wandb-forum 🎙Host: Lukas Biewald - https://twitter.com/l2k 👩🏼‍💻Producer: Lavanya Shukla - https://twitter.com/lavanyaai 📹Editor: Cayla Sharp - http://caylasharp.com/

8 Juli 202048min

Miles Brundage — Societal Impacts of Artificial Intelligence

Miles Brundage — Societal Impacts of Artificial Intelligence

Miles Brundage researches the societal impacts of artificial intelligence and how to make sure they go well. In 2018, he joined OpenAI, as a Research Scientist on the Policy team. Previously, he was a Research Fellow at the University of Oxford's Future of Humanity Institute and served as a member of Axon's AI and Policing Technology Ethics Board. Keep up with Miles on his website: https://www.milesbrundage.com/ and on Twitter: https://twitter.com/miles_brundage Visit our podcasts homepage for transcripts and more episodes! www.wandb.com/podcast 🔊 Get our podcast on Soundcloud, Apple, and Spotify! Apple Podcasts: https://bit.ly/2WdrUvI Spotify: https://bit.ly/2SqtadF We started Weights and Biases to build tools for Machine Learning practitioners because we care a lot about the impact that Machine Learning can have in the world and we love working in the trenches with the people building these models. One of the most fun things about these building tools has been the conversations with these ML practitioners and learning about the interesting things they’re working on. This process has been so fun that we wanted to open it up to the world in the form of our new podcast called Gradient Dissent. We hope you have as much fun listening to it as we had making it! 👩🏼‍🚀Weights and Biases: We’re always free for academics and open source projects. Email carey@wandb.com with any questions or feature suggestions. - Blog: https://www.wandb.com/articles - Gallery: See what you can create with W&B - https://app.wandb.ai/gallery - Continue the conversation on our slack community - http://bit.ly/wandb-forum 🎙Host: Lukas Biewald - https://twitter.com/l2k 👩🏼‍💻Producer: Lavanya Shukla - https://twitter.com/lavanyaai 📹Editor: Cayla Sharp - http://caylasharp.com/

1 Juli 20201h 2min

Hamel Husain — Building Machine Learning Tools

Hamel Husain — Building Machine Learning Tools

Hamel Husain is a Staff Machine Learning Engineer at Github. He has extensive experience building data analytics and predictive modeling solutions for a wide range of industries, including: hospitality, telecom, retail, restaurant, entertainment and finance. He has built large data science teams (50+) from the ground up and have extensive experience building solutions as an individual contributor. Follow Hamel on Twitter: https://twitter.com/HamelHusain And on his website: http://hamel.io/ Learn more about Github Actions: https://github.com/features/actions and the CodeSearchNet Challenge: https://github.blog/2019-09-26-introducing-the-codesearchnet-challenge/ Visit our podcasts homepage for transcripts and more episodes! www.wandb.com/podcast 🔊 Get our podcast on Apple, and Spotify! Apple Podcasts: https://bit.ly/2WdrUvI Spotify: https://bit.ly/2SqtadF We started Weights and Biases to build tools for Machine Learning practitioners because we care a lot about the impact that Machine Learning can have in the world and we love working in the trenches with the people building these models. One of the most fun things about these building tools has been the conversations with these ML practitioners and learning about the interesting things they’re working on. This process has been so fun that we wanted to open it up to the world in the form of our new podcast called Gradient Dissent. We hope you have as much fun listening to it as we had making it! 👩🏼‍🚀Weights and Biases: We’re always free for academics and open source projects. Email carey@wandb.com with any questions or feature suggestions. - Blog: https://www.wandb.com/articles - Gallery: See what you can create with W&B - https://app.wandb.ai/gallery - Continue the conversation on our slack community - http://bit.ly/wandb-forum 🎙Host: Lukas Biewald - https://twitter.com/l2k 👩🏼‍💻Producer: Lavanya Shukla - https://twitter.com/lavanyaai 📹Editor: Cayla Sharp - http://caylasharp.com/

24 Juni 202036min

Peter Welinder — Deep Reinforcement Learning and Robotics

Peter Welinder — Deep Reinforcement Learning and Robotics

Peter Welinder is a research scientist and roboticist at OpenAI. Before that, he was an engineer at Dropbox and ran the machine learning team, and before that, he co-founded Anchovi Labs a startup using Computer Vision to organize photos that was acquired by Dropbox in 2012. In this episode of our podcast, Peter shares his experiences and the challenges associated with building a robotic hand that can solve a rubix cube. Read some of Peter’s Articles: https://openai.com/blog/authors/peter/ Follow Peter on Twitter: https://twitter.com/npew Check out our podcasts homepage for transcripts and more episodes! www.wandb.com/podcast 🔊 Get our podcast on Apple, and Spotify! Apple Podcasts: https://bit.ly/2WdrUvI Spotify: https://bit.ly/2SqtadF We started Weights and Biases to build tools for Machine Learning practitioners because we care a lot about the impact that Machine Learning can have in the world and we love working in the trenches with the people building these models. One of the most fun things about these building tools has been the conversations with these ML practitioners and learning about the interesting things they’re working on. This process has been so fun that we wanted to open it up to the world in the form of our new podcast. We hope you have as much fun listening to it as we had making it. 👩🏼‍🚀Weights and Biases: We’re always free for academics and open source projects. Email carey@wandb.com with any questions or feature suggestions. - Blog: https://www.wandb.com/articles - Gallery: See what you can create with W&B - https://app.wandb.ai/gallery - Continue the conversation on our slack community - http://bit.ly/wandb-forum 🎙Host: Lukas Biewald - https://twitter.com/l2k 👩🏼‍💻Producer: Lavanya Shukla - https://twitter.com/lavanyaai 📹Editor: Cayla Sharp - http://caylasharp.com/

17 Juni 202054min

Vicki Boykis — Machine Learning Across Industries

Vicki Boykis — Machine Learning Across Industries

👩‍💻Today our guest is Vicki Boykis! Vicki is a senior consultant in machine learning and engineering and works with clients to build holistic data products used for decision-making. She's previously spoken at PyData, taught SQL for GirlDevelopIt, and blogs about data pipelines and open internet. Follow her on her website: vickiboykis.com On twitter: https://twitter.com/vboykis and subscribe to her newsletter: vicki.substack.com Check out our podcasts homepage for transcripts and more episodes! www.wandb.com/podcast 🔊 Get our podcast on Apple and Spotify! Apple Podcasts: https://bit.ly/2WdrUvI Spotify: https://bit.ly/2SqtadF We started Weights and Biases to build tools for Machine Learning practitioners because we care a lot about the impact that Machine Learning can have in the world and we love working in the trenches with the people building these models. One of the most fun things about these building tools has been the conversations with these ML practitioners and learning about the interesting things they’re working on. This process has been so fun that we wanted to open it up to the world in the form of our new podcast. We hope you have as much fun listening to it as we had making it. 👩🏼‍🚀Weights and Biases: We’re always free for academics and open source projects. Email carey@wandb.com with any questions or feature suggestions. - Blog: https://www.wandb.com/articles - Gallery: See what you can create with W&B - https://app.wandb.ai/gallery - Continue the conversation on our slack community - http://bit.ly/wandb-forum 🎙Host: Lukas Biewald - https://twitter.com/l2k 👩🏼‍💻Producer: Lavanya Shukla - https://twitter.com/lavanyaai 📹Editor: Cayla Sharp - http://caylasharp.com/

4 Juni 202034min

Angela & Danielle — Designing ML Models for Millions of Consumer Robots

Angela & Danielle — Designing ML Models for Millions of Consumer Robots

👩‍💻👩‍💻On this episode of Gradient Dissent our guests are Angela Bassa and Danielle Dean! Angela is an expert in building and leading data teams. An MIT-trained and Edelman-award-winning mathematician, she has over 15 years of experience across industries—spanning finance, life sciences, agriculture, marketing, energy, software, and robotics. Angela heads Data Science and Machine Learning at iRobot, where her teams help bring intelligence to a global fleet of millions of consumer robots. She is also a renowned keynote speaker and author, with credits including the Wall Street Journal and Harvard Business Review. Follow Angela on twitter: https://twitter.com/angebassa And on her website: https://www.angelabassa.com/ Danielle Dean, PhD is the Technical Director of Machine Learning at iRobot where she is helping lead the intelligence revolution for robots. She leads a team that leverages machine learning, reinforcement learning, and software engineering to build algorithms that will result in massive improvements in our robots. Before iRobot, Danielle was a Principal Data Scientist Lead at Microsoft Corp. in AzureCAT Engineering within the Cloud AI Platform division. Follow Danielle on Twitter: https://twitter.com/danielleodean Check out our podcasts homepage for transcripts and more episodes! www.wandb.com/podcast 🔊 Get our podcast on Apple and Spotify! Apple Podcasts: https://bit.ly/2WdrUvI Spotify: https://bit.ly/2SqtadF We started Weights and Biases to build tools for Machine Learning practitioners because we care a lot about the impact that Machine Learning can have in the world and we love working in the trenches with the people building these models. One of the most fun things about these building tools has been the conversations with these ML practitioners and learning about the interesting things they’re working on. This process has been so fun that we wanted to open it up to the world in the form of our new podcast. We hope you have as much fun listening to it as we had making it. 👩🏼‍🚀Weights and Biases: We’re always free for academics and open source projects. Email carey@wandb.com with any questions or feature suggestions. - Blog: https://www.wandb.com/articles - Gallery: See what you can create with W&B - https://app.wandb.ai/gallery - Continue the conversation on our slack community - http://bit.ly/wandb-forum 🎙Host: Lukas Biewald - https://twitter.com/l2k 👩🏼‍💻Producer: Lavanya Shukla - https://twitter.com/lavanyaai 📹Editor: Cayla Sharp - http://caylasharp.com/

6 Maj 202052min

Jack Clark — Building Trustworthy AI Systems

Jack Clark — Building Trustworthy AI Systems

Jack Clark is the Strategy and Communications Director at OpenAI and formerly worked as the world’s only neural network reporter at Bloomberg. Lukas and Jack discuss AI policy, ethics, and the responsibilities of AI researchers. Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims by OpenAI: https://arxiv.org/abs/2004.07213 Follow Jack Clark on Twitter: twitter.com/jackclarkSF Read more posts by Jack on his website: https://jack-clark.net/ Get our podcast on Apple and Spotify! https://podcasts.apple.com/us/podcast/gradient-dissent-weights-biases/id1504567418 https://open.spotify.com/show/7o9r3fFig3MhTJwehXDbXm 🤖Gradient Dissent by Weights and Biases Get a behind-the-scenes look at how industry leaders are using machine learning in the real world. While building experiment tracking tools, we’ve had the opportunity to learn about how different teams are building and deploying models. In this podcast, we share some of the insights and stories we’ve heard along the way. Follow Gradient Dissent for weekly machine learning updates, and be part of the conversation. 👩🏼‍🚀Weights and Biases: We’re always free for academics and open source projects. Email carey@wandb.com with any questions or feature suggestions. - Blog: https://www.wandb.com/articles - Gallery: See what you can create with W&B - https://app.wandb.ai/gallery - Continue the conversation on our slack community - http://bit.ly/wandb-forum 🎙Host: Lukas Biewald - https://twitter.com/l2k 👩🏼‍💻Producer: Lavanya Shukla - https://twitter.com/lavanyaai 📹Editor: Cayla Sharp - http://caylasharp.com/

22 Apr 202055min

Populärt inom Business & ekonomi

badfluence
framgangspodden
varvet
rss-svart-marknad
uppgang-och-fall
rss-borsens-finest
lastbilspodden
rss-jossan-nina
rss-kort-lang-analyspodden-fran-di
affarsvarlden
24fragor
rss-inga-dumma-fragor-om-pengar
rss-en-rik-historia
rss-dagen-med-di
avanzapodden
borsmorgon
tabberaset
fill-or-kill
bathina-en-podcast
dynastin