News

Why can’t AI say ‘I don’t know’?

Overconfident AI systems can be dangerous, so researchers are teaching them humility
Doctoral researcher Lassi Meronen dressed in a blue shirt, photographed from the side with a studio light shining on the right side of the photo
Lassi Meronen defended his doctoral thesis at the Department of Computer Science on 15th December 2023. Image: Matti Ahlgren/Aalto University

Artificial intelligence models like ChatGPT can be powerful tools, but they’re also often overconfident. AI tools are poor at estimating the uncertainty in their predictions and most aren’t able to say ‘I don’t know’ in response to an unfamiliar situation.

‘This overconfidence becomes an issue in safety-critical applications, such as self-driving cars, where poor decision-making can lead to severe consequences,’ says Lassi Meronen, a doctoral researcher at Aalto University who’s working on ways to address this problem. ‘For example, a deep learning model could be helpful in automatic medical diagnosis from imaging data if it could accurately determine 90% of the cases, leaving only 10% as uncertain cases that require checking by a doctor. But if the model can’t tell which cases are uncertain, the doctor would need to go through all the cases anyway.

Meronen’s doctoral thesis focused on making deep learning models better at estimating the uncertainty in their predictions, which would let them be used more widely in safety-critical applications.

‘My main results stemmed from building mathematical connections that bring the benefits of principled probabilistic models into deep learning models,’ says Meronen.

Improving the uncertainty estimates would also allow deep learning models to distinguish between complex and simpler inputs, says Meronen. This would help save energy by using greater computational resources only when they’re needed – with the more complex inputs. This smart allocation of resources is important because deep learning models are getting bigger and take more and more energy to run.

Cooperation with Saab

Meronen’s thesis, which he recently defended, included several high-impact scientific articles, two of which were accepted to the most prestigious academic conference in AI research, the Conference on Neural Information Processing Systems (NeurIPS) in 2019 and 2021. His research also involved close collaboration with Saab under the auspices of a ten-year strategic research partnership with Aalto, which was launched in 2017 and expanded in 2021 to accommodate more research areas. The partnership is worth 23 million euros and has already resulted in dozens of scientific publications and several patent applications. 

‘When we were considering a new research partner, Aalto University stood out,’ says Petteri Alinikula CTO at Saab Finland. ‘Aalto has exactly the knowhow that Saab needs, and their attitude towards industrial cooperation is excellent. Our goal is to train at least 20 new PhDs during this program.’ 

Meronen’s collaboration with Saab has focused on safety-critical deep learning models. 

‘Saab deals with many applications that could benefit from the use of deep learning models. However, most of these applications are safety-critical, which makes research like mine quite relevant. In my collaboration with Saab, I’ve worked on automatic radar signal classification that can be used to recognise aircraft, for example. Such a system must be good at estimating uncertainty, since overconfident and incorrect classification of an aircraft could have severe consequences,’ says Meronen.
 

Lassi Meronen

Doctoral Reseacher
Saab Aalto Research Day 2019

Strategic corporate partner Saab

Saab's long-term commitment to cooperation provides good conditions for scientific breakthroughs and the creation of new innovations.

Corporate collaboration
Mahine Learning researchers working at Department of Computer Science in Aalto University

Department of Computer Science

To foster future science and society.

FCAI

Finnish Center for Artificial Intelligence (external link)

The Finnish Center for Artificial Intelligence FCAI is a research hub initiated by Aalto University, the University of Helsinki, and the Technical Research Centre of Finland VTT. The goal of FCAI is to develop new types of artificial intelligence that can work with humans in complex environments, and help modernize Finnish industry. FCAI is one of the national flagships of the Academy of Finland.

Students walking and working in the basement floor of the Harald Herlin Learning center / photo by Aalto University, Aino Huovio

Corporate collaboration

Research collaboration strengthens expertise and business development. New product and service ideas are created in study projects. Our campus offers an inspiring operating and networking environment. At Aalto's recruitment events, companies meet the top talents of the future.

How to apply for doctoral studies?

Doctoral students at Aalto University can pursue the degrees of Doctor of Science (Technology), Doctor of Arts (Art and Design), Doctor of Science (Economics and Business Administration), and Doctor of Science (Architecture). The target time for the doctoral degree is four years. The admissions are organised individually by each doctoral programme.

Read more
A man standing on stage with a doctor's hat on, having just being promoted as a doctor
  • Published:
  • Updated:

Read more news

Dr. Swarnalok De and logos of the Finnish Cultural Foundation, Aalto University and MMD group
Research & Art Published:

Dr. Swarnalok De receives a one-year grant from the Finnish Cultural Foundation

Awarded for research on the development of wearable healthcare sensors for autonomous health monitoring of the aging population
Unite! White paper - a new university open science and innovation governance model and policy for a sustainable world
Cooperation, University Published:

Unite!’s pathway for the management of open science and innovation in the European Universities Alliances

Unite! presents a new university open science and innovation governance model and five policy recommendations.
The magnetic properties of a material can affect how it interacts with light.
Press releases, Research & Art Published:

A new optical metamaterial makes true one-way glass possible

Researchers have discovered how to make a new optical metamaterial that would underpin a variety of new technologies.
Mikko Alava ja kolme muuta tutkimusryhmän jäsentä kuvattuna teknisen laitteen äärellä.
Research & Art Published:

Significant grants for science from the Finnish Cultural Foundation

A total of thirty individuals and research groups from Aalto University received grants from the Finnish Cultural Foundation for science and art. Professors Harri Lipsanen and Zhipei Sun and their team, were awarded a large grant to develop electronic components mimicking the structure and function of the human brain. Among the grantees is also Professor Mikko Alava, whose team is developing water-resistant foams as substitutes for plastic using AI-based methods.