Learning from the brain: Stéphane Deny uses insights from neuroscience for better artificial intelligence

Machine learning models typically need gigantic data sets and a lot of energy, whereas the brain consumes as much power as a single light bulb. Aalto’s new assistant professor uses neuroscience to make computer programs more efficient.
Stephane Deny
Stéphane Deny started as a new assistant professor in the beginning of December. Image: Mikko Raskinen / Aalto University

What do you research and why?

I try to understand how the brain works, and then to use that understanding to build better systems for machine learning and AI.

In the last decade, we have seen a big push in machine learning that has yielded a lot of impressive results in domains like visual recognition and language understanding. This push has been mostly driven by the industry, and basically with two engines: a lot of data and a lot of compute. It has not really been inspired by how the brain works.

I think this is a missed opportunity. The brain is much more efficient than the current systems used in machine learning: It requires much less data than current models that are trained on gigantic corpuses of text or images, and it needs much less energy. The brain consumes the same amount of power as a light bulb -- just 20 watts -- to function, and that is much more efficient than the forms of GPUs and algorithms that are currently used in any machine learning application. So I think there is still a lot to learn from the brain, and this is why I work at this interface.

How did you become a professor?

I did my PhD in France at the Vision Institute, where I studied the retina. I recorded from it and did experiments in neuroscience, and also modeled it with artificial neural networks. I realized that the retinal processing is very complex: Instead of sending just one message to the brain like a camera would do, the retina sends multiple images in parallel, and each is processed in a certain way. 

I was really intrigued by this complexity, and wanted to understand it from a more theoretical point-of-view. This is how I decided to move to a lab in Stanford, which was on the theoretical side of neuroscience. After this experience, I moved to do a postdoc at Facebook AI. There, I spent two years developing machine learning tools inspired by the brain. This led me to apply to professorships, and it is how I landed here at Aalto.

What is the high point of your career?

One work I am particularly proud of is where we understood from very simple principles some aspects of the organization of the retina. As I said earlier, the retina sends multiple images to the brain in parallel, each processed in its own way. There used to be no explanation for that. 

I had a collaboration at Stanford with a group of physicists, with whom we came up with an explanation of that structure with a simple principle of efficiency: If the goal of the retina is to send the visual information to the brain in the most efficient way, then it should be sent through these different pathways, with these different pre-processings. Some of our predictions were precisely aligned with the processing done by the primate retina. Here, then, we could explain, from first principles, a complex structure that was found in the brain. It was very exciting to me.

And then, this same principle turned out to be also useful in machine learning. The principle of efficiency behind our discovery in the retina has been described in the 1960s as the redundancy-reduction principle, and it tells you that each neuron should carry information as independently of other neurons as possible. This principle has been very useful for understanding the structure of the visual system, but it was largely ignored in machine learning. When I joined Facebook, one of my projects was to develop a model that was using this principle. That work resulted in a model for visual recognition without supervision which was state-of-the-art at the time, a year ago.

Stéphane Deny started as a new assistant professor in the beginning of December. His position is shared between the department of neuroscience and biomedical engineering and the department of computer science.

Stephane Deny

Finnish Center for Artificial Intelligence

The Finnish Center for Artificial Intelligence FCAI is a research hub initiated by Aalto University, the University of Helsinki, and the Technical Research Centre of Finland VTT. The goal of FCAI is to develop new types of artificial intelligence that can work with humans in complex environments, and help modernize Finnish industry. FCAI is one of the national flagships of the Academy of Finland.

Read more
  • Published:
  • Updated:

Read more news

Artistic illustration: Algorithms over a computer chip
Research & Art Published:

Aalto computer scientists in STOC 2024

Two papers from the field of computer science were accepted to STOC 2024.
A group of people walking along the linden alley in summer
Cooperation, Research & Art, University Published:

Strong overall result: 27.5 million euros from the Research Council of Finland

A total of 52 Aalto researchers received Academy Research Fellowship and Academy Project funding. The total funding awarded to Aalto University amounts to 27.5 million euros
Appointments Published:

Introducing four new professors at the School of Engineering

Professors of Practice Hannele Holttinen and Teemu Manderbacka and Assistant Professors Mikko Suominen and Risto Ojala were appointed to the career path of professors during January-June 2024.
A logo. Photo: Mikko Raskinen
Appointments, Cooperation Published:

Introducing three new professors at the School of Electrical Engineering

Assistant Professors Gopika Premsankar and Johannes Arend and Associate Professor (tenured) Marko Kosunen were appointed to the career path of professors during January-June 2024.