Vierailuluento: Mohamed Elgharib "Neural Reconstruction and Rendering"
Neural Reconstruction and Rendering
Max Plack Institute, Saarbrücken, Germany
Note that we will begin a few minutes past 1pm.
Abstract: Digitising the world around us is of increasing importance, with several applications in Extended Reality, movie and media production, telecommunications, medicine, video games, robotics, and many more. Digitisation involves three main stages: modeling, reconstruction and rendering. Modeling is the process of describing the semantics of real world objects and scenes. Reconstruction is the task of fitting the learned model to unseen data during test. The final step is rendering, where the reconstructed model is projected onto a 2D plane that represents the medium of observation e.g. screen, image, etc… Here, rendering that seamlessly blends with the environment is important for high-quality photorealism. In this talk, I will discuss my work on digitising our world through neural based approaches. I will cover all stages of the digitisation pipeline and will discuss how to process different types of input data modalities. I will cover several means of model building including supervised and self-supervised learning. I will also highlight how generative models can boost the learning capabilities and reduce reliance on paired training data. I will discuss recent advances in implicit scene representations and show the advantages they can bring over traditional meshes in a number of problems. In the end, I will highlight event cameras and the potential they can bring to 3D scene reconstruction and rendering. The ultimate goal of my work is to fully digitise our surroundings to allow all of us to connect and interact with our friends, family, and loved ones, over a distance.
Bio: Mohamed Elgharib is a Research Group Leader at the Max Planck Institute for Informatics. His areas of expertise are computer vision, computer graphics and machine learning. His work is on building digital models of our world to allow novel applications in Extended Reality and VR/AR. Topics of interest include 3D scene modeling and reconstruction, deep generative modeling, neural rendering, 3D pose estimation, relighting and others. His work usually includes a heavy machine and deep learning component through supervised, self-supervised or unsupervised learning. He worked with different types of data, including monocular RGB, multiview RGB, audio, depth, and even with biologically inspired and neuromorphic based sensors such as event cameras. Mohamed Elgharib co-authored more than 40 peer-reviewed publications, has three granted US patents and has collaborated with a wide spectrum of academic and industrial institutes. Some of his publications were featured in media outlets such as BBC News and MIT News and a start-up was founded following one of his publications. Mohamed co-supervised three post-docs, six PhD students, nine Master’s thesis students, five long-term interns, and closely worked with many others. He also organized the Computer Vision and Machine Learning for Computer Graphics Seminar in the University of Saarland for 3 consecutive years and has strong experience in building and running sophisticated hardware capture devices.
This guest talk is hosted by Associate Professor Jaakko Lehtinen, Department of Computer Science.