Ines & Sofie — Building Industrial-Strength NLP Pipelines

Jeremy Howard is a co-founder of fast.ai, the non-profit research group behind the popular massive open online course "Practical Deep Learning for Coders", and the open source deep learning library "fastai".Jeremy is also a co-founder of #Masks4All, a global volunteer organization founded in March 2020 that advocated for the public adoption of homemade face masks in order to help slow the spread of COVID-19. His Washington Post article "Simple DIY masks could help flatten the curve." went viral in late March/early April 2020, and is associated with the U.S CDC's change in guidance a few days later to recommend wearing masks in public.In this episode, Jeremy explains how diffusion works and how individuals with limited compute budgets can engage meaningfully with large, state-of-the-art models. Then, as our first-ever repeat guest on Gradient Dissent, Jeremy revisits a previous conversation with Lukas on Python vs. Julia for machine learning.Finally, Jeremy shares his perspective on the early days of COVID-19, and what his experience as one of the earliest and most high-profile advocates for widespread mask-wearing was like.Show notes (transcript and links): http://wandb.me/gd-jeremy-howard-2---⏳ Timestamps:0:00 Intro1:06 Diffusion and generative models14:40 Engaging with large models meaningfully20:30 Jeremy's thoughts on Stable Diffusion and OpenAI26:38 Prompt engineering and large language models32:00 Revisiting Julia vs. Python40:22 Jeremy's science advocacy during early COVID days1:01:03 Researching how to improve children's education1:07:43 The importance of executive buy-in1:11:34 Outro1:12:02 Bonus: Weights & Biases---📝 Links📍 Jeremy's previous Gradient Dissent episode (8/25/2022): http://wandb.me/gd-jeremy-howard📍 "Simple DIY masks could help flatten the curve. We should all wear them in public.", Jeremy's viral Washington Post article: https://www.washingtonpost.com/outlook/2020/03/28/masks-all-coronavirus/📍 "An evidence review of face masks against COVID-19" (Howard et al., 2021), one of the first peer-reviewed papers on the effectiveness of wearing masks: https://www.pnas.org/doi/10.1073/pnas.2014564118📍 Jeremy's Twitter thread summary of "An evidence review of face masks against COVID-19": https://twitter.com/jeremyphoward/status/1348771993949151232📍 Read more about Jeremy's mask-wearing advocacy: https://www.smh.com.au/world/north-america/australian-expat-s-push-for-universal-mask-wearing-catches-fire-in-the-us-20200401-p54fu2.html---Connect with Jeremy and fast.ai:📍 Jeremy on Twitter: https://twitter.com/jeremyphoward📍 fast.ai on Twitter: https://twitter.com/FastDotAI📍 Jeremy on LinkedIn: https://www.linkedin.com/in/howardjeremy/---💬 Host: Lukas Biewald📹 Producers: Riley Fields, Angelica Pan

5 Tammi 20231h 12min

Jerome Pesenti is the former VP of AI at Meta, a tech conglomerate that includes Facebook, WhatsApp, and Instagram, and one of the most exciting places where AI research is happening today.Jerome shares his thoughts on Transformers-based large language models, and why he's excited by the progress but skeptical of the term "AGI". Then, he discusses some of the practical applications of ML at Meta (recommender systems and moderation!) and dives into the story behind Meta's development of PyTorch. Jerome and Lukas also chat about Jerome's time at IBM Watson and in drug discovery.Show notes (transcript and links): http://wandb.me/gd-jerome-pesenti---⏳ Timestamps: 0:00 Intro0:28 Jerome's thought on large language models12:53 AI applications and challenges at Meta18:41 The story behind developing PyTorch26:40 Jerome's experience at IBM Watson28:53 Drug discovery, AI, and changing the game36:10 The potential of education and AI40:10 Meta and AR/VR interfaces43:43 Why NVIDIA is such a powerhouse47:08 Jerome's advice to people starting their careers48:50 Going back to coding, the challenges of scaling52:11 Outro---Connect with Jerome:📍 Jerome on Twitter: https://twitter.com/an_open_mind📍 Jerome on LinkedIn: https://www.linkedin.com/in/jpesenti/---💬 Host: Lukas Biewald📹 Producers: Riley Fields, Angelica Pan, Lavanya Shukla---Subscribe and listen to our podcast today!👉 Apple Podcasts: http://wandb.me/apple-podcasts​​👉 Google Podcasts: http://wandb.me/google-podcasts​👉 Spotify: http://wandb.me/spotify​

22 Joulu 202252min

D. Sculley is CEO of Kaggle, the beloved and well-known data science and machine learning community.D. discusses his influential 2015 paper "Machine Learning: The High Interest Credit Card of Technical Debt" and what the current challenges of deploying models in the real world are now, in 2022. Then, D. and Lukas chat about why Kaggle is like a rain forest, and about Kaggle's historic, current, and potential future roles in the broader machine learning community.Show notes (transcript and links): http://wandb.me/gd-d-sculley---⏳ Timestamps: 0:00 Intro1:02 Machine learning and technical debt11:18 MLOps, increased stakes, and realistic expectations19:12 Evaluating models methodically25:32 Kaggle's role in the ML world33:34 Kaggle competitions, datasets, and notebooks38:49 Why Kaggle is like a rain forest44:25 Possible future directions for Kaggle46:50 Healthy competitions and self-growth48:44 Kaggle's relevance in a compute-heavy future53:49 AutoML vs. human judgment56:06 After a model goes into production1:00:00 Outro---Connect with D. and Kaggle:📍 D. on LinkedIn: https://www.linkedin.com/in/d-sculley-90467310/📍 Kaggle on Twitter: https://twitter.com/kaggle---Links:📍 "Machine Learning: The High Interest Credit Card of Technical Debt" (Sculley et al. 2014): https://research.google/pubs/pub43146/---💬 Host: Lukas Biewald📹 Producers: Riley Fields, Angelica Pan, Anish Shah, Lavanya Shukla---Subscribe and listen to our podcast today!👉 Apple Podcasts: http://wandb.me/apple-podcasts​​👉 Google Podcasts: http://wandb.me/google-podcasts​👉 Spotify: http://wandb.me/spotify​

1 Joulu 20221h

Emad Mostaque is CEO and co-founder of Stability AI, a startup and network of decentralized developer communities building open AI tools. Stability AI is the company behind Stable Diffusion, the well-known, open source, text-to-image generation model.Emad shares the story and mission behind Stability AI (unlocking humanity's potential with open AI technology), and explains how Stability's role as a community catalyst and compute provider might evolve as the company grows. Then, Emad and Lukas discuss what the future might hold in store: big models vs "optimal" models, better datasets, and more decentralization.-🎶 Special note: This week’s theme music was composed by Weights & Biases’ own Justin Tenuto with help from Harmonai’s Dance Diffusion.-Show notes (transcript and links): http://wandb.me/gd-emad-mostaque-⏳ Timestamps:00:00 Intro00:42 How AI fits into the safety/security industry09:33 Event matching and object detection14:47 Running models on the right hardware17:46 Scaling model evaluation23:58 Monitoring and evaluation challenges26:30 Identifying and sorting issues30:27 Bridging vision and language domains39:25 Challenges and promises of natural language technology41:35 Production environment43:15 Using synthetic data49:59 Working with startups53:55 Multi-task learning, meta-learning, and user experience56:44 Optimization and testing across multiple platforms59:36 Outro-Connect with Jehan and Motorola Solutions:📍 Jehan on LinkedIn: https://www.linkedin.com/in/jehanw/📍 Jehan on Twitter: https://twitter.com/jehan/📍 Motorola Solutions on Twitter: https://twitter.com/MotoSolutions/📍 Careers at Motorola Solutions: https://www.motorolasolutions.com/en_us/about/careers.html-💬 Host: Lukas Biewald📹 Producers: Riley Fields, Angelica Pan, Lavanya Shukla, Anish Shah-Subscribe and listen to our podcast today!👉 Apple Podcasts: http://wandb.me/apple-podcasts​​👉 Google Podcasts: http://wandb.me/google-podcasts​👉 Spotify: http://wandb.me/spotify​

15 Marras 20221h 10min

Jehan Wickramasuriya is the Vice President of AI, Platform & Data Services at Motorola Solutions, a global leader in public safety and enterprise security.In this episode, Jehan discusses how Motorola Solutions uses AI to simplify data streams to help maximize human potential in high-stress situations. He also shares his thoughts on augmenting synthetic data with real data and the challenges posed in partnering with startups.Show notes (transcript and links): http://wandb.me/gd-jehan-wickramasuriya-⏳ Timestamps: 00:00 Intro00:42 How AI fits into the safety/security industry 09:33 Event matching and object detection14:47 Running models on the right hardware17:46 Scaling model evaluation23:58 Monitoring and evaluation challenges26:30 Identifying and sorting issues30:27 Bridging vision and language domains39:25 Challenges and promises of natural language technology41:35 Production environment43:15 Using synthetic data49:59 Working with startups53:55 Multi-task learning, meta-learning, and user experience56:44 Optimization and testing across multiple platforms59:36 Outro-Connect with Jehan and Motorola Solutions:📍 Jehan on LinkedIn: https://www.linkedin.com/in/jehanw/📍 Jehan on Twitter: https://twitter.com/jehan/📍 Motorola Solutions on Twitter: https://twitter.com/MotoSolutions/📍 Careers at Motorola Solutions: https://www.motorolasolutions.com/en_us/about/careers.html-💬 Host: Lukas Biewald📹 Producers: Riley Fields, Cayla Sharp, Angelica Pan, Lavanya Shukla-Subscribe and listen to our podcast today!👉 Apple Podcasts: http://wandb.me/apple-podcasts​​👉 Google Podcasts: http://wandb.me/google-podcasts​👉 Spotify: http://wandb.me/spotify​

6 Loka 20221h

Will Falcon is the CEO and co-founder of Lightning AI, a platform that enables users to quickly build and publish ML models.In this episode, Will explains how Lightning addresses the challenges of a fragmented AI ecosystem and reveals which framework PyTorch Lightning was originally built upon (hint: not PyTorch!) He also shares lessons he took from his experience serving in the military and offers a recommendation to veterans who want to work in tech.Show notes (transcript and links): http://wandb.me/gd-will-falcon---⏳ Timestamps: 00:00 Intro01:00 From SEAL training to FAIR04:17 Stress-testing Lightning07:55 Choosing PyTorch over TensorFlow and other frameworks13:16 Components of the Lightning platform17:01 Launching Lightning from Facebook19:09 Similarities between leadership and research22:08 Lessons from the military26:56 Scaling PyTorch Lightning to Lightning AI33:21 Hiring the right people35:21 The future of Lightning39:53 Reducing algorithm complexity in self-supervised learning42:19 A fragmented ML landscape44:35 Outro---Connect with Lightning📍 Website: https://lightning.ai📍 Twitter: https://twitter.com/LightningAI📍 LinkedIn: https://www.linkedin.com/company/pytorch-lightning/📍 Careers: https://boards.greenhouse.io/lightningai---💬 Host: Lukas Biewald📹 Producers: Riley Fields, Anish Shah, Cayla Sharp, Angelica Pan, Lavanya Shukla---Subscribe and listen to our podcast today!👉 Apple Podcasts: http://wandb.me/apple-podcasts​​👉 Google Podcasts: http://wandb.me/google-podcasts​👉 Spotify: http://wandb.me/spotify​

15 Syys 202245min

Aaron Colak is the Leader of Core Machine Learning at Qualtrics, an experiment management company that takes large language models and applies them to real-world, B2B use cases.In this episode, Aaron describes mixing classical linguistic analysis with deep learning models and how Qualtrics organized their machine learning organizations and model to leverage the best of these techniques. He also explains how advances in NLP have invited new opportunities in low-resource languages.Show notes (transcript and links): http://wandb.me/gd-aaron-colak---⏳ Timestamps: 00:00 Intro00:57 Evolving from surveys to experience management04:56 Detecting sentiment with ML10:57 Working with large language models and rule-based systems14:50 Zero-shot learning, NLP, and low-resource languages20:11 Letting customers control data25:13 Deep learning and tabular data28:40 Hyperscalers and performance monitoring34:54 Combining deep learning with linguistics40:03 A sense of accomplishment42:52 Causality and observational data in healthcare45:09 Challenges of interdisciplinary collaboration49:27 Outro---Connect with Aaron and Qualtrics📍 Aaron on LinkedIn: https://www.linkedin.com/in/aaron-r-colak-3522308/📍 Qualtrics on Twitter: https://twitter.com/qualtrics/📍 Careers at Qualtrics: https://www.qualtrics.com/careers/---💬 Host: Lukas Biewald📹 Producers: Riley Fields, Cayla Sharp, Angelica Pan, Lavanya Shukla---Subscribe and listen to our podcast today!👉 Apple Podcasts: http://wandb.me/apple-podcasts​​👉 Google Podcasts: http://wandb.me/google-podcasts​👉 Spotify: http://wandb.me/spotify​

26 Elo 202250min

Jordan Fisher is the CEO and co-founder of Standard AI, an autonomous checkout company that’s pushing the boundaries of computer vision.In this episode, Jordan discusses “the Wild West” of the MLOps stack and tells Lukas why Rust beats Python. He also explains why AutoML shouldn't be overlooked and uses a bag of chips to help explain the Manifold Hypothesis.Show notes (transcript and links): http://wandb.me/gd-jordan-fisher---⏳ Timestamps: 00:00 Intro00:40 The origins of Standard AI08:30 Getting Standard into stores18:00 Supervised learning, the advent of synthetic data, and the manifold hypothesis24:23 What's important in a MLOps stack27:32 The merits of AutoML30:00 Deep learning frameworks33:02 Python versus Rust39:32 Raw camera data versus video42:47 The future of autonomous checkout48:02 Sharing the StandardSim data set52:30 Picking the right tools54:30 Overcoming dynamic data set challenges57:35 Outro---Connect with Jordan and Standard AI📍 Jordan on LinkedIn: https://www.linkedin.com/in/jordan-fisher-81145025/📍 Standard AI on Twitter: https://twitter.com/StandardAi📍 Careers at Standard AI: https://careers.standard.ai/---💬 Host: Lukas Biewald📹 Producers: Riley Fields, Cayla Sharp, Angelica Pan, Lavanya Shukla---Subscribe and listen to our podcast today!👉 Apple Podcasts: http://wandb.me/apple-podcasts​​👉 Google Podcasts: http://wandb.me/google-podcasts​👉 Spotify: http://wandb.me/spotify​

4 Elo 202257min