Gradient Dissent: Conversations on AI

From Apache TVM to OctoML, Luis gives direct insight into the world of ML hardware optimization, and where systems optimization is heading. --- Luis Ceze is co-founder and CEO of OctoML, co-author of the Apache TVM Project, and Professor of Computer Science and Engineering at the University of Washington. His research focuses on the intersection of computer architecture, programming languages, machine learning, and molecular biology. Connect with Luis: 📍 Twitter: https://twitter.com/luisceze 📍 University of Washington profile: https://homes.cs.washington.edu/~luisceze/ --- ⏳ Timestamps: 0:00 Intro and sneak peek 0:59 What is TVM? 8:57 Freedom of choice in software and hardware stacks 15:53 How new libraries can improve system performance 20:10 Trade-offs between efficiency and complexity 24:35 Specialized instructions 26:34 The future of hardware design and research 30:03 Where does architecture and research go from here? 30:56 The environmental impact of efficiency 32:49 Optimizing and trade-offs 37:54 What is OctoML and the Octomizer? 42:31 Automating systems design with and for ML 44:18 ML and molecular biology 46:09 The challenges of deployment and post-deployment 🌟 Transcript: http://wandb.me/gd-luis-ceze 🌟 Links: 1. OctoML: https://octoml.ai/ 2. Apache TVM: https://tvm.apache.org/ 3. "Scalable and Intelligent Learning Systems" (Chen, 2019): https://digital.lib.washington.edu/researchworks/handle/1773/44766 4. "Principled Optimization Of Dynamic Neural Networks" (Roesch, 2020): https://digital.lib.washington.edu/researchworks/handle/1773/46765 5. "Cross-Stack Co-Design for Efficient and Adaptable Hardware Acceleration" (Moreau, 2018): https://digital.lib.washington.edu/researchworks/handle/1773/43349 6. "TVM: An Automated End-to-End Optimizing Compiler for Deep Learning" (Chen et al., 2018): https://www.usenix.org/system/files/osdi18-chen.pdf 7. Porcupine is a molecular tagging system introduced in "Rapid and robust assembly and decoding of molecular tags with DNA-based nanopore signatures" (Doroschak et al., 2020): https://www.nature.com/articles/s41467-020-19151-8 --- Get our podcast on these platforms: 👉 Apple Podcasts: http://wandb.me/apple-podcasts​​ 👉 Spotify: http://wandb.me/spotify​ 👉 Google Podcasts: http://wandb.me/google-podcasts​​ 👉 YouTube: http://wandb.me/youtube​​ 👉 Soundcloud: http://wandb.me/soundcloud​ Join our community of ML practitioners where we host AMAs, share interesting projects and meet other people working in Deep Learning: http://wandb.me/slack​​ Check out Fully Connected, which features curated machine learning reports by researchers exploring deep learning techniques, Kagglers showcasing winning models, industry leaders sharing best practices, and more: https://wandb.ai/fully-connected

24 Jun 202148min

Matthew explains how combining machine learning and computational biology can provide mainstream medicine with better diagnostics and insights. --- Matthew Davis is Head of AI at Invitae, the largest and fastest growing genetic testing company in the world. His research includes bioinformatics, computational biology, NLP, reinforcement learning, and information retrieval. Matthew was previously at IBM Research AI, where he led a research team focused on improving AI systems. Connect with Matthew: 📍 Personal website: https://www.linkedin.com/in/matthew-davis-51233386/ 📍 Twitter: https://twitter.com/deadsmiths --- ⏳ Timestamps: 0:00 Sneak peek, intro 1:02 What is Invitae? 2:58 Why genetic testing can help everyone 7:51 How Invitae uses ML techniques 14:02 Modeling molecules and deciding which genes to look at 22:22 NLP applications in bioinformatics 27:10 Team structure at Invitae 36:50 Why reasoning is an underrated topic in ML 40:25 Why having a clear buy-in is important 🌟 Transcript: http://wandb.me/gd-matthew-davis 🌟 Links: 📍 Invitae: https://www.invitae.com/en 📍 Careers at Invitae: https://www.invitae.com/en/careers/ --- Get our podcast on these platforms: 👉 Apple Podcasts: http://wandb.me/apple-podcasts​​ 👉 Spotify: http://wandb.me/spotify​ 👉 Google Podcasts: http://wandb.me/google-podcasts​​ 👉 YouTube: http://wandb.me/youtube​​ 👉 Soundcloud: http://wandb.me/soundcloud​ Join our community of ML practitioners where we host AMAs, share interesting projects and meet other people working in Deep Learning: http://wandb.me/slack​​ Check out Fully Connected, which features curated machine learning reports by researchers exploring deep learning techniques, Kagglers showcasing winning models, industry leaders sharing best practices, and more: https://wandb.ai/fully-connected

17 Jun 202143min

Clem explains the virtuous cycles behind the creation and success of Hugging Face, and shares his thoughts on where NLP is heading. --- Clément Delangue is co-founder and CEO of Hugging Face, the AI community building the future. Hugging Face started as an open source NLP library and has quickly grown into a commercial product used by over 5,000 companies. Connect with Clem: 📍 Twitter: https://twitter.com/ClementDelangue 📍 LinkedIn: https://www.linkedin.com/in/clementdelangue/ --- 🌟 Transcript: http://wandb.me/gd-clement-delangue 🌟 ⏳ Timestamps: 0:00 Sneak peek and intro 0:56 What is Hugging Face? 4:15 The success of Hugging Face Transformers 7:53 Open source and virtuous cycles 10:37 Working with both TensorFlow and PyTorch 13:20 The "Write With Transformer" project 14:36 Transfer learning in NLP 16:43 BERT and DistilBERT 22:33 GPT 26:32 The power of the open source community 29:40 Current applications of NLP 35:15 The Turing Test and conversational AI 41:19 Why speech is an upcoming field within NLP 43:44 The human challenges of machine learning Links Discussed: 📍 Write With Transformer, Hugging Face Transformer's text generation demo: https://transformer.huggingface.co/ 📍 "Attention Is All You Need" (Vaswani et al., 2017): https://arxiv.org/abs/1706.03762 📍 EleutherAI and GPT-Neo: https://github.com/EleutherAI/gpt-neo] 📍 Rasa, open source conversational AI: https://rasa.com/ 📍 Roblox article on BERT: https://blog.roblox.com/2020/05/scaled-bert-serve-1-billion-daily-requests-cpus/ --- Get our podcast on these platforms: 👉 Apple Podcasts: http://wandb.me/apple-podcasts​​ 👉 Spotify: http://wandb.me/spotify​ 👉 Google Podcasts: http://wandb.me/google-podcasts​​ 👉 YouTube: http://wandb.me/youtube​​ 👉 Soundcloud: http://wandb.me/soundcloud​ Join our community of ML practitioners where we host AMAs, share interesting projects and meet other people working in Deep Learning: http://wandb.me/slack​​ Check out Fully Connected, which features curated machine learning reports by researchers exploring deep learning techniques, Kagglers showcasing winning models, industry leaders sharing best practices, and more: https://wandb.ai/fully-connected

10 Jun 202146min

Wojciech joins us to talk the principles behind OpenAI, the Fermi Paradox, and the future stages of developments in AGI. --- Wojciech Zaremba is a co-founder of OpenAI, a research company dedicated to discovering and enacting the path to safe artificial general intelligence. He was also Head of Robotics, where his team developed general-purpose robots through new approaches to transfer learning, and taught robots complex behaviors. Connect with Wojciech: Personal website: https://wojzaremba.com// Twitter: https://twitter.com/woj_zaremba --- Topics Discussed: 0:00 Sneak peek and intro 1:03 The people and principles behind OpenAI 6:31 The stages of future AI developments 13:42 The Fermi paradox 16:18 What drives Wojciech? 19:17 Thoughts on robotics 24:58 Dota and other projects at OpenAI 33:42 What would make an AI conscious? 41:31 How to be succeed in robotics Transcript: http://wandb.me/gd-wojciech-zaremba Links: Fermi paradox: https://en.wikipedia.org/wiki/Fermi_paradox OpenAI and Dota: https://openai.com/projects/five/ --- Get our podcast on these platforms: Apple Podcasts: http://wandb.me/apple-podcasts​​ Spotify: http://wandb.me/spotify​ Google Podcasts: http://wandb.me/google-podcasts​​ YouTube: http://wandb.me/youtube​​ Soundcloud: http://wandb.me/soundcloud​ Join our community of ML practitioners where we host AMAs, share interesting projects and meet other people working in Deep Learning: http://wandb.me/slack​​ Check out Fully Connected, which features curated machine learning reports by researchers exploring deep learning techniques, Kagglers showcasing winning models, industry leaders sharing best practices, and more: https://wandb.ai/fully-connected

3 Jun 202144min

Phil shares some of the approaches, like sparsity and low precision, behind the breakthrough performance of Graphcore's Intelligence Processing Units (IPUs). --- Phil Brown leads the Applications team at Graphcore, where they're building high-performance machine learning applications for their Intelligence Processing Units (IPUs), new processors specifically designed for AI compute. Connect with Phil: LinkedIn: https://www.linkedin.com/in/philipsbrown/ Twitter: https://twitter.com/phil_s_brown --- 0:00 Sneak peek, intro 1:44 From computational chemistry to Graphcore 5:16 The simulations behind weather prediction 10:54 Measuring improvement in weather prediction systems 15:35 How high performance computing and ML have different needs 19:00 The potential of sparse training 31:08 IPUs and computer architecture for machine learning 39:10 On performance improvements 44:43 The impacts of increasing computing capability 50:24 The ML chicken and egg problem 52:00 The challenges of converging at scale and bringing hardware to market Links Discussed: Rigging the Lottery: Making All Tickets Winners (Evci et al., 2019): https://arxiv.org/abs/1911.11134 Graphcore MK2 Benchmarks: https://www.graphcore.ai/mk2-benchmarks Check out the transcription and discover more awesome ML projects: http://wandb.me/gd-phil-brown --- Get our podcast on these platforms: Apple Podcasts: http://wandb.me/apple-podcasts​​​ Spotify: http://wandb.me/spotify​​ Google Podcasts: http://wandb.me/google-podcasts​​​ YouTube: http://wandb.me/youtube​​​ Soundcloud: http://wandb.me/soundcloud​​ Join our community of ML practitioners where we host AMAs, share interesting projects and meet other people working in Deep Learning: http://wandb.me/slack​​​ Check out our Gallery, which features curated machine learning reports by researchers exploring deep learning techniques, Kagglers showcasing winning models, industry leaders sharing best practices, and more: https://wandb.ai/gallery

27 Mai 202157min

From working on COVID-19 vaccine rollout to writing a book on responsible ML, Alyssa shares her thoughts on meaningful projects and the importance of teamwork. --- Alyssa Simpson Rochwerger is as a Director of Product at Blue Shield of California, pursuing her dream of using technology to improve healthcare. She has over a decade of experience in building technical data-driven products and has held numerous leadership roles for machine learning organizations, including VP of AI and Data at Appen and Director of Product at IBM Watson. Connect with Sean: Personal website: https://seanjtaylor.com/ Twitter: https://twitter.com/seanjtaylor LinkedIn: https://www.linkedin.com/in/seanjtaylor/ --- Topics Discussed: 0:00 Sneak peak, intro 1:17 Working on COVID-19 vaccine rollout in California 6:50 Real World AI 12:26 Diagnosing bias in models 17:43 Common challenges in ML 21:56 Finding meaningful projects 24:28 ML applications in health insurance 31:21 Longitudinal health records and data cleaning 38:24 Following your interests 40:21 Why teamwork is crucial Transcript: http://wandb.me/gd-alyssa-s-rochwerger Links Discussed: My Turn: https://myturn.ca.gov/ "Turn the Ship Around!": https://www.penguinrandomhouse.com/books/314163/turn-the-ship-around-by-l-david-marquet/ --- Get our podcast on these platforms: Apple Podcasts: http://wandb.me/apple-podcasts​​ Spotify: http://wandb.me/spotify​ Google Podcasts: http://wandb.me/google-podcasts​​ YouTube: http://wandb.me/youtube​​ Soundcloud: http://wandb.me/soundcloud​ Join our community of ML practitioners where we host AMAs, share interesting projects and meet other people working in Deep Learning: http://wandb.me/slack​​ Check out Fully Connected, which features curated machine learning reports by researchers exploring deep learning techniques, Kagglers showcasing winning models, industry leaders sharing best practices, and more: https://wandb.ai/fully-connected

20 Mai 202145min

Sean joins us to chat about ML models and tools at Lyft Rideshare Labs, Python vs R, time series forecasting with Prophet, and election forecasting. --- Sean Taylor is a Data Scientist at (and former Head of) Lyft Rideshare Labs, and specializes in methods for solving causal inference and business decision problems. Previously, he was a Research Scientist on Facebook's Core Data Science team. His interests include experiments, causal inference, statistics, machine learning, and economics. Connect with Sean: Personal website: https://seanjtaylor.com/ Twitter: https://twitter.com/seanjtaylor LinkedIn: https://www.linkedin.com/in/seanjtaylor/ --- Topics Discussed: 0:00 Sneak peek, intro 0:50 Pricing algorithms at Lyft 07:46 Loss functions and ETAs at Lyft 12:59 Models and tools at Lyft 20:46 Python vs R 25:30 Forecasting time series data with Prophet 33:06 Election forecasting and prediction markets 40:55 Comparing and evaluating models 43:22 Bottlenecks in going from research to production Transcript: http://wandb.me/gd-sean-taylor Links Discussed: "How Lyft predicts a rider’s destination for better in-app experience"": https://eng.lyft.com/how-lyft-predicts-your-destination-with-attention-791146b0a439 Prophet: https://facebook.github.io/prophet/ Andrew Gelman's blog post "Facebook's Prophet uses Stan": https://statmodeling.stat.columbia.edu/2017/03/01/facebooks-prophet-uses-stan/ Twitter thread "Election forecasting using prediction markets": https://twitter.com/seanjtaylor/status/1270899371706466304 "An Updated Dynamic Bayesian Forecasting Model for the 2020 Election": https://hdsr.mitpress.mit.edu/pub/nw1dzd02/release/1 --- Get our podcast on these platforms: Apple Podcasts: http://wandb.me/apple-podcasts​​ Spotify: http://wandb.me/spotify​ Google Podcasts: http://wandb.me/google-podcasts​​ YouTube: http://wandb.me/youtube​​ Soundcloud: http://wandb.me/soundcloud​ Join our community of ML practitioners where we host AMAs, share interesting projects and meet other people working in Deep Learning: http://wandb.me/slack​​ Check out Fully Connected, which features curated machine learning reports by researchers exploring deep learning techniques, Kagglers showcasing winning models, industry leaders sharing best practices, and more: https://wandb.ai/fully-connected

13 Mai 202145min

Polly explains how microfluidics allow bioengineering researchers to create high throughput data, and shares her experiences with biology and machine learning. --- Polly Fordyce is an Assistant Professor of Genetics and Bioengineering and fellow of the ChEM-H Institute at Stanford. She is the Principal Investigator of The Fordyce Lab, which focuses on developing and applying new microfluidic platforms for quantitative, high-throughput biophysics and biochemistry. Twitter: https://twitter.com/fordycelab​ Website: http://www.fordycelab.com/​ --- Topics Discussed: 0:00​ Sneak peek, intro 2:11​ Background on protein sequencing 7:38​ How changes to a protein's sequence alters its structure and function 11:07​ Microfluidics and machine learning 19:25​ Why protein folding is important 25:17​ Collaborating with ML practitioners 31:46​ Transfer learning and big data sets in biology 38:42​ Where Polly hopes bioengineering research will go 42:43​ Advice for students Transcript: http://wandb.me/gd-polly-fordyce​ Links Discussed: "The Weather Makers": https://en.wikipedia.org/wiki/The_Wea...​ --- Get our podcast on these platforms: Apple Podcasts: http://wandb.me/apple-podcasts​​​ Spotify: http://wandb.me/spotify​​ Google Podcasts: http://wandb.me/google-podcasts​​​ YouTube: http://wandb.me/youtube​​​ Soundcloud: http://wandb.me/soundcloud​​ Join our community of ML practitioners where we host AMAs, share interesting projects and meet other people working in Deep Learning: http://wandb.me/slack​​​ Check out Fully Connected, which features curated machine learning reports by researchers exploring deep learning techniques, Kagglers showcasing winning models, industry leaders sharing best practices, and more: https://wandb.ai/fully-connected

29 Apr 202145min