Trends in NLP with John Bohannon - #550
Today we’re joined by friend of the show John Bohannon, the director of science at Primer AI, to help us showcase all of the great achievements and accomplishments in NLP in 2021! In our conversation, John shares his two major takeaways from last year, 1) NLP as we know it has changed, and we’re back into the incremental phase of the science, and 2) NLP is “eating” the rest of machine learning. We explore the implications of these two major themes across the discipline, as well as best papers, up and coming startups, great things that did happen, and even a few bad things that didn’t. Finally, we explore what 2022 and beyond will look like for NLP, from multilingual NLP to use cases for the influx of large auto-regressive language models like GPT-3 and others, as well as ethical implications that are reverberating across domains and the changes that have been ushered in in that vein. The complete show notes for this episode can be found at twimlai.com/go/550
6 Jan 20221h 18min
Trends in Computer Vision with Georgia Gkioxari - #549
Happy New Year! We’re excited to kick off 2022 joined by Georgia Gkioxari, a research scientist at Meta AI, to showcase the best advances in the field of computer vision over the past 12 months, and what the future holds for this domain. Welcome back to AI Rewind! In our conversation Georgia highlights the emergence of the transformer model in CV research, what kind of performance results we’re seeing vs CNNs, and the immediate impact of NeRF, amongst a host of other great research. We also explore what is ImageNet’s place in the current landscape, and if it's time to make big changes to push the boundaries of what is possible with image, video and even 3D data, with challenges like the Metaverse, amongst others, on the horizon. Finally, we touch on the startups to keep an eye on, the collaborative efforts of software and hardware researchers, and the vibe of the “ImageNet moment” being upon us once again. The complete show notes for this episode can be found at twimlai.com/go/549
3 Jan 202258min
Kids Run the Darndest Experiments: Causal Learning in Children with Alison Gopnik - #548
Today we close out the 2021 NeurIPS series joined by Alison Gopnik, a professor at UC Berkeley and an invited speaker at the Causal Inference & Machine Learning: Why now? Workshop. In our conversation with Alison, we explore the question, “how is it that we can know so much about the world around us from so little information?,” and how her background in psychology, philosophy, and epistemology has guided her along the path to finding this answer through the actions of children. We discuss the role of causality as a means to extract representations of the world and how the “theory theory” came about, and how it was demonstrated to have merit. We also explore the complexity of causal relationships that children are able to deal with and what that can tell us about our current ML models, how the training and inference stages of the ML lifecycle are akin to childhood and adulthood, and much more! The complete show notes for this episode can be found at twimlai.com/go/548
27 Des 202136min
Hypergraphs, Simplicial Complexes and Graph Representations of Complex Systems with Tina Eliassi-Rad - #547
Today we continue our NeurIPS coverage joined by Tina Eliassi-Rad, a professor at Northeastern University, and an invited speaker at the I Still Can't Believe It's Not Better! Workshop. In our conversation with Tina, we explore her research at the intersection of network science, complex networks, and machine learning, how graphs are used in her work and how it differs from typical graph machine learning use cases. We also discuss her talk from the workshop, “The Why, How, and When of Representations for Complex Systems”, in which Tina argues that one of the reasons practitioners have struggled to model complex systems is because of the lack of connection to the data sourcing and generation process. This is definitely a NERD ALERT approved interview! The complete show notes for this episode can be found at twimlai.com/go/547
23 Des 202135min
Deep Learning, Transformers, and the Consequences of Scale with Oriol Vinyals - #546
Today we’re excited to kick off our annual NeurIPS, joined by Oriol Vinyals, the lead of the deep learning team at Deepmind. We cover a lot of ground in our conversation with Oriol, beginning with a look at his research agenda and why the scope has remained wide even through the maturity of the field, his thoughts on transformer models and if they will get us beyond the current state of DL, or if some other model architecture would be more advantageous. We also touch on his thoughts on the large language models craze, before jumping into his recent paper StarCraft II Unplugged: Large Scale Offline Reinforcement Learning, a follow up to their popular AlphaStar work from a few years ago. Finally, we discuss the degree to which the work that Deepmind and others are doing around games actually translates into real-world, non-game scenarios, recent work on multimodal few-shot learning, and we close with a discussion of the consequences of the level of scale that we’ve achieved thus far. The complete show notes for this episode can be found at twimlai.com/go/546
20 Des 202152min
Optimization, Machine Learning and Intelligent Experimentation with Michael McCourt - #545
Today we’re joined by Michael McCourt the head of engineering at SigOpt. In our conversation with Michael, we explore the vast space around the topic of optimization, including the technical differences between ML and optimization and where they’re applied, what the path to increasing complexity looks like for a practitioner and the relationship between optimization and active learning. We also discuss the research frontier for optimization and how folks think about the interesting challenges and open questions for this field, how optimization approaches appeared at the latest NeurIPS conference, and Mike’s excitement for the emergence of interdisciplinary work between the machine learning community and other fields like the natural sciences. The complete show notes for this episode can be found at twimlai.com/go/545
16 Des 202145min
Jupyter and the Evolution of ML Tooling with Brian Granger - #544
Today we conclude our AWS re:Invent coverage joined by Brian Granger, a senior principal technologist at Amazon Web Services, and a co-creator of Project Jupyter. In our conversion with Brian, we discuss the inception and early vision of Project Jupyter, including how the explosion of machine learning and deep learning shifted the landscape for the notebook, and how they balanced the needs of these new user bases vs their existing community of scientific computing users. We also explore AWS’s role with Jupyter and why they’ve decided to invest resources in the project, Brian's thoughts on the broader ML tooling space, and how they’ve applied (and the impact of) HCI principles to the building of these tools. Finally, we dig into the recent Sagemaker Canvas and Studio Lab releases and Brian’s perspective on the future of notebooks and the Jupyter community at large. The complete show notes for this episode can be found at twimlai.com/go/544
13 Des 202157min
Creating a Data-Driven Culture at ADP with Jack Berkowitz - #543
Today we continue our 2021 re:Invent series joined by Jack Berkowitz, chief data officer at ADP. In our conversation with Jack, we explore the ever evolving role and growth of machine learning at the company, from the evolution of their ML platform, to the unique team structure. We discuss Jack’s perspective on data governance, the broad use cases for ML, how they approached the decision to move to the cloud, and the impact of scale in the way they deal with data. Finally, we touch on where innovation comes from at ADP, and the challenge of getting the talent it needs to innovate as a large “legacy” company. The complete show notes for this episode can be found at twimlai.com/go/543
9 Des 202134min
