News

Learning from the brain: Stéphane Deny uses insights from neuroscience for better artificial intelligence

Machine learning models typically need gigantic data sets and a lot of energy, whereas the brain consumes as much power as a single light bulb. Aalto’s new assistant professor uses neuroscience to make computer programs more efficient.
Stephane Deny
Stéphane Deny started as a new assistant professor in the beginning of December. Image: Mikko Raskinen / Aalto University

What do you research and why?

I try to understand how the brain works, and then to use that understanding to build better systems for machine learning and AI.

In the last decade, we have seen a big push in machine learning that has yielded a lot of impressive results in domains like visual recognition and language understanding. This push has been mostly driven by the industry, and basically with two engines: a lot of data and a lot of compute. It has not really been inspired by how the brain works.

I think this is a missed opportunity. The brain is much more efficient than the current systems used in machine learning: It requires much less data than current models that are trained on gigantic corpuses of text or images, and it needs much less energy. The brain consumes the same amount of power as a light bulb -- just 20 watts -- to function, and that is much more efficient than the forms of GPUs and algorithms that are currently used in any machine learning application. So I think there is still a lot to learn from the brain, and this is why I work at this interface.

How did you become a professor?

I did my PhD in France at the Vision Institute, where I studied the retina. I recorded from it and did experiments in neuroscience, and also modeled it with artificial neural networks. I realized that the retinal processing is very complex: Instead of sending just one message to the brain like a camera would do, the retina sends multiple images in parallel, and each is processed in a certain way. 

I was really intrigued by this complexity, and wanted to understand it from a more theoretical point-of-view. This is how I decided to move to a lab in Stanford, which was on the theoretical side of neuroscience. After this experience, I moved to do a postdoc at Facebook AI. There, I spent two years developing machine learning tools inspired by the brain. This led me to apply to professorships, and it is how I landed here at Aalto.

What is the high point of your career?

One work I am particularly proud of is where we understood from very simple principles some aspects of the organization of the retina. As I said earlier, the retina sends multiple images to the brain in parallel, each processed in its own way. There used to be no explanation for that. 

I had a collaboration at Stanford with a group of physicists, with whom we came up with an explanation of that structure with a simple principle of efficiency: If the goal of the retina is to send the visual information to the brain in the most efficient way, then it should be sent through these different pathways, with these different pre-processings. Some of our predictions were precisely aligned with the processing done by the primate retina. Here, then, we could explain, from first principles, a complex structure that was found in the brain. It was very exciting to me.

And then, this same principle turned out to be also useful in machine learning. The principle of efficiency behind our discovery in the retina has been described in the 1960s as the redundancy-reduction principle, and it tells you that each neuron should carry information as independently of other neurons as possible. This principle has been very useful for understanding the structure of the visual system, but it was largely ignored in machine learning. When I joined Facebook, one of my projects was to develop a model that was using this principle. That work resulted in a model for visual recognition without supervision which was state-of-the-art at the time, a year ago.

Stéphane Deny started as a new assistant professor in the beginning of December. His position is shared between the department of neuroscience and biomedical engineering and the department of computer science.

 

Stephane Deny

Finnish Center for Artificial Intelligence

The Finnish Center for Artificial Intelligence FCAI is a research hub initiated by Aalto University, the University of Helsinki, and the Technical Research Centre of Finland VTT. The goal of FCAI is to develop new types of artificial intelligence that can work with humans in complex environments, and help modernize Finnish industry. FCAI is one of the national flagships of the Academy of Finland.

Read more
FCAI
  • Published:
  • Updated:
Share
URL copied!

Read more news

Markus Sunela
Appointments Published:

Markus Sunela appointed as Professor of Practice of Management of Water and Wastewater Networks

Sunela has over ten years of practical experience modelling water and district energy systems in addition to special expertise in topics such as heat capture from wastewater networks
Professori Anna Mikola
Appointments Published:

Assistant Professor Anna Mikola: Wastewater treatment is going through a transformation as the water treatment system is being overhauled to meet new requirements

Anna Mikola (b. 1973), D.Sc. (Tech.), has been appointed as assistant professor at the Aalto University Department of Built Environment for the period 1 June 2022–31 May 2026. The field of the professorship is water engineering in the circular economy.
Airi Nevalainen ja Joona Huikuri Aallon opiskelijaedustajiksi Unite-yliopistoverkostoon
Appointments, Cooperation, University Published:

Meet Airi and Joona, Aalto’s student representatives in the Unite! university alliance

We talked to Airi and Joona about their new roles as Unite!’s student representatives and got some great ideas and advice from them.
Pekka Pokela
Appointments Published:

Pekka Pokela appointed as Professor of Practice in the field of product development at Aalto University

His areas of expertise include product development, strategic management and sales, partnerships, and sustainable business.