CS Forum: George Drettakis, INRIA Sophia-Antipolis "Marrying Learning to Graphics: Rendering, 3D Geometry and Synthetic Training Data"
Marrying Learning to Graphics: Rendering, 3D Geometry and Synthetic Training Data
Dr George Drettakis
In recent years deep learning methods such as image-to-image translation have been used to develop neural rendering solutions that synthesize or render new images with stunning visual quality. Such methods are often ``end-to-end'' operating entirely on 2D photos, and are also trained on 2D image data. In this talk we discuss an alternative approach, that exploits the huge body of knowledge developed in traditional physically- and image-based rendering and uses it hand-in-hand with such learning methods to effectively solve both inverse and forward problems in graphics. In particular, we discuss the importance of three key elements in these solutions: the use of 3D data extracted from multi-view stereo, the use of rendering and rendered synthetic data for effective and efficient supervised learning. We illustrate these ideas on three recent projects: estimating material parameters from a single flash photo, image-based rendering with learned blending weights and a geometry-aware network for relighting captured outdoor 3D scenes.
Links to papers
Multi-view Relighting Using a Geometry-Aware Network, Philip et al. ACM TOG/SIGGRAPH 2019 http://www-sop.inria.fr/reves/Basilic/2019/PGZED19/ and https://repo-sam.inria.fr/fungraph/deep-relighting/
Deep Blending for Free-Viewpoint Image-Based Rendering, Hedman et al. ACM TOG/SIGGRAPH Asia 2018 http://www-sop.inria.fr/reves/Basilic/2018/HPPFDB18/ and https://repo-sam.inria.fr/fungraph/deep-blending/
Single-Image SVBRDF Capture with a Rendering-Aware Deep Network, Deschaintre et al. ACM TOG/SIGGRAPH 2018 http://www-sop.inria.fr/reves/Basilic/2018/DADDB18/ and https://team.inria.fr/graphdeco/fr/projects/deep-materials/
George Drettakis graduated in Computer Science from the University of Crete, Greece, and obtained an M.Sc.and a Ph.D., (1994) also in CS at the University of Toronto, Canada (Dynamic Graphics Project), supervised by Eugene Fiume. He was an ERCIM postdoctoral fellow in Grenoble, Barelona and Bonn (94-95), and became a permament Inria researcher in the iMAGIS group in Grenoble in 1995, obtaining the degree of "Habilitation" at the University of Grenoble in 1999. In 2000 he founded the REVES research group at INRIA Sophia-Antipolis, which he headed from 2002-2015. Since 2016 he heads the follow-up group GRAPHDECO. He became a INRIA Senior Researcher DR1 (full professor equivalent) in 2008. He received the Eurographics (EG) Outstanding Technical Contributions award in 2007 and is an EG fellow. He was an associate editor for ACM Transactions on Graphics, technical papers chair of SIGGRAPH Asia 2010, associate editor and co-editor in chief of IEEE Trans. on Computer Graphics and Visualization (2011-2015). He leads the EG working group on Rendering. He has worked on many different topics in computer graphics, with an emphasis on rendering. He initially concentrated on lighting and shadow computation and subsequently worked on 3D audio, perceptually-driven algorithms, virtual reality and 3D interaction. In recent years he has worked on image-based rendering and relighting, textures, weathering and perception for graphics, as well as learning for rendering and was awarded an ERC Advanced Grant, which started in Oct. 2018.
George currently serves on the Scientific Advisory Board (SAB) of Aalto University School of Science.
Professor Jaakko Lehtinen, Department of Computer Science