Scalable Geometric Deep Learning on Molecular Graphs - Nathan C. Frey

If you enjoyed this talk, consider joining the Molecular Modeling and Drug Discovery (M2D2) talks live: https://valence-discovery.github.io/M...

Abstract: Deep learning in molecular and materials sciences is limited by the lack of integration between applied science, artificial intelligence, and high-performance computing. Bottlenecks with respect to the amount of training data, the size and complexity of model architectures, and the scale of compute infrastructure are all key factors limiting the scaling of deep learning for molecules and materials. In cases where design goals require explorations of vast areas of chemical/material space, or target properties are prohibitively expensive to compute, efficient use of resources and careful choice of method enable new capabilities for design. We explore interactive supercomputing for applying high-throughput virtual screening and machine learning to challenges in materials and chemistry. The abundance of data from first-principles calculations introduces a need to identify and investigate scalable neural network architectures that operate on graphs, which are a natural representation for atomistic systems. We present LitMatter, a lightweight framework for scaling geometric deep learning methods. We discuss scaling atomistic deep learning using key resources including compute, model and dataset sizes, and energy. We train four graph neural network architectures on over 400 GPUs and investigate the scaling behavior of these methods. Depending on the model architecture, training time speedups up to 60x are seen. Empirical neural scaling relations quantify the model-dependent scaling and enable optimal compute resource allocation and the identification of scalable geometric deep learning model implementations. Training speed estimation and energy monitoring are used to accelerate hyperparameter optimization for neural interatomic potentials, and quantify the efficiency of physics-informed architectures. We discuss applications of scalable ML to property prediction tasks, deep generative modeling, and neural force fields for fully differentiable simulations.

Speaker: Nathan C. Frey - https://ncfrey.github.io/

Twitter Prudencio: https://twitter.com/tossouprudencio

Twitter Therence: https://twitter.com/Therence_mtl