Machine Learning Coffee seminar: "Infinitely deep models with continuous-time flows" Markus Heinonen
Infinitely deep models with continuous-time flows
Academy of Finland Postdoctoral Fellow, Aalto University
Abstract: Bayesian formalism has recently entered the age of deep models. Deep generative models, such as VAEs, learn unimodal variational representations of complex data objects. Paired with Normalising flows these variational approximations can be successively transformed to the more powerful family of multimodal distributions. Similarly in predictive modelling stacking Gaussian processes into ‘layers’ produces a deep predictive models with increased capacity and multimodality. Both of these approaches are based on evolving random variables with discrete-time transformations, which are seriously hindered by the theoretical requirement of invertibility. We propose a novel paradigm of continuous time ‘flows’ that generalises the concept of discrete transformations into infinite continuous domain. We derive theory from fluid dynamics that does not require invertible transformations. We demonstrate initial results, which exceed the state-of-the-art.