Introduction to Continual Learning

Speaker: Vincenzo Lomonaco, University of Pisa

Abstract: 
Learning continually from non-stationary data streams is a long-standing goal and a challenging problem in machine learning research. Naively fine-tuning prediction models only on the newly available data often incurs in Catastrophic Forgetting or Interference: a sudden erase of all the previously acquired knowledge. On the other hand, re-training prediction models from scratch on the accumulated data is not only inefficient but possibly unsustainable in the long-term and where fast, frequent model updates are necessary. In this lecture we will discuss recent progress and trends in making machines learn continually through architectural, regularization and replay approaches. We identify Deep Continual Learning as a promising approach and key technological enabler towards the design of systems compliant with the Sustainable AI principles. Then, we present Avalanche, an open-source end-to-end library for continual learning based on PyTorch and point out possible real-world applications. Finally, further interesting research directions will be discussed.

Biography:
Vincenzo Lomonaco is a 31 years old Assistant Professor at the University of Pisa, Italy and Co-Founding President of ContinualAI, a non-profit research organization and the largest open community on Continual Learning for AI. Currently, He is also a Co-founder and Board Member of AI for People, Director of the ContinualAI Lab and a proud member of the European Lab for Learning and Intelligent Systems (ELLIS). In Pisa, he works within the Pervasive AI Lab and the Computational Intelligence and Machine Learning Group, which is also part of the and the Confederation of Laboratories for Artificial Intelligence Research in Europe (CLAIRE). Previously, he was a Post-Doc @ University of Bologna (with: Davide Maltoni) where he also obtained his PhD in early 2019 with a dissertation titled “Continual Learning with Deep Architectures'' (on a topic he’s been working on for more than 8 years now) which was recognized as one of the top-5 AI dissertation of 2019 by the Italian Association for Artificial Intelligence. For more than 5 years he worked as a teaching assistant for the Machine Learning and Computer Architectures courses in the Department of Computer Science of Engineering (DISI) at UniBo. In the past Vincenzo have been a Visiting Research Scientist at AI Labs in 2020, at Numenta (with: Jeff Hawkins, Subutai Ahmad) in 2019, at ENSTA ParisTech (with: David Filliat) in 2018 and at Purdue University (with: Eugenio Culurciello) in 2017. Even before, he was a Machine Learning Software Engineer @ iDL in-line Devices and a Master Student @ UniBo. His main research interest and passion is about Continual Learning in all its facets. In particular, he loves to study Continual Learning under four main lights: Deep Learning, Distributed Learning and Practical Applications, all within an AI Sustainability developmental framework.

References:
1. Lifelong Machine Learning, Zhiyuan Chen and Bing Liu, 2018.
2. Continual Learning Course, PhD Course, University of Pisa, AIDA Academy and ContinualAI, Vicenzo Lomonaco, 2021.
3. Continual lifelong learning with neural networks: A review. Parisi, German I., et al. Neural Networks 113 (2019): 54-71.
4. Avalanche: an end-to-end library for continual learning. Lomonaco, Vincenzo, et al. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021.

Host: Prof. Cesare Alippi