News

Why can’t AI say ‘I don’t know’?

Overconfident AI systems can be dangerous, so researchers are teaching them humility
Doctoral researcher Lassi Meronen dressed in a blue shirt, photographed from the side with a studio light shining on the right side of the photo
Lassi Meronen defended his doctoral thesis at the Department of Computer Science on 15th December 2023. Image: Matti Ahlgren/Aalto University

Artificial intelligence models like ChatGPT can be powerful tools, but they’re also often overconfident. AI tools are poor at estimating the uncertainty in their predictions and most aren’t able to say ‘I don’t know’ in response to an unfamiliar situation.

‘This overconfidence becomes an issue in safety-critical applications, such as self-driving cars, where poor decision-making can lead to severe consequences,’ says Lassi Meronen, a doctoral researcher at Aalto University who’s working on ways to address this problem. ‘For example, a deep learning model could be helpful in automatic medical diagnosis from imaging data if it could accurately determine 90% of the cases, leaving only 10% as uncertain cases that require checking by a doctor. But if the model can’t tell which cases are uncertain, the doctor would need to go through all the cases anyway.

Meronen’s doctoral thesis focused on making deep learning models better at estimating the uncertainty in their predictions, which would let them be used more widely in safety-critical applications.

‘My main results stemmed from building mathematical connections that bring the benefits of principled probabilistic models into deep learning models,’ says Meronen.

Improving the uncertainty estimates would also allow deep learning models to distinguish between complex and simpler inputs, says Meronen. This would help save energy by using greater computational resources only when they’re needed – with the more complex inputs. This smart allocation of resources is important because deep learning models are getting bigger and take more and more energy to run.

Cooperation with Saab

Meronen’s thesis, which he recently defended, included several high-impact scientific articles, two of which were accepted to the most prestigious academic conference in AI research, the Conference on Neural Information Processing Systems (NeurIPS) in 2019 and 2021. His research also involved close collaboration with Saab under the auspices of a ten-year strategic research partnership with Aalto, which was launched in 2017 and expanded in 2021 to accommodate more research areas. The partnership is worth 23 million euros and has already resulted in dozens of scientific publications and several patent applications. 

‘When we were considering a new research partner, Aalto University stood out,’ says Petteri Alinikula CTO at Saab Finland. ‘Aalto has exactly the knowhow that Saab needs, and their attitude towards industrial cooperation is excellent. Our goal is to train at least 20 new PhDs during this program.’ 

Meronen’s collaboration with Saab has focused on safety-critical deep learning models. 

‘Saab deals with many applications that could benefit from the use of deep learning models. However, most of these applications are safety-critical, which makes research like mine quite relevant. In my collaboration with Saab, I’ve worked on automatic radar signal classification that can be used to recognise aircraft, for example. Such a system must be good at estimating uncertainty, since overconfident and incorrect classification of an aircraft could have severe consequences,’ says Meronen.
 

Lassi Meronen

Doctoral Reseacher
Saab Aalto Research Day 2019

Strategic corporate partner Saab

Saab's long-term commitment to cooperation provides good conditions for scientific breakthroughs and the creation of new innovations.

Corporate collaboration
Mahine Learning researchers working at Department of Computer Science in Aalto University

Department of Computer Science

To foster future science and society.

FCAI

Finnish Center for Artificial Intelligence (external link)

The Finnish Center for Artificial Intelligence FCAI is a research hub initiated by Aalto University, the University of Helsinki, and the Technical Research Centre of Finland VTT. The goal of FCAI is to develop new types of artificial intelligence that can work with humans in complex environments, and help modernize Finnish industry. FCAI is one of the national flagships of the Academy of Finland.

Students walking and working in the basement floor of the Harald Herlin Learning center / photo by Aalto University, Aino Huovio

Corporate collaboration

Research collaboration strengthens expertise and business development. New product and service ideas are created in study projects. Our campus offers an inspiring operating and networking environment. At Aalto's recruitment events, companies meet the top talents of the future.

How to apply for doctoral studies?

Doctoral students at Aalto University can pursue the degrees of Doctor of Science (Technology), Doctor of Arts (Art and Design), Doctor of Science (Economics and Business Administration), and Doctor of Science (Architecture). The target time for the doctoral degree is four years. The admissions are organised individually by each doctoral programme.

Read more
A man standing on stage with a doctor's hat on, having just being promoted as a doctor
  • Published:
  • Updated:

Read more news

An artistic rendition of a photo colliding with other particles.
Studies Published:

Bridging gaps in reactor damage modelling and embracing diversity in academia

With a keen interest in electron-ion interactions, doctoral researcher Evgeniia Ponomareva describes her experience in the Nuclear Materials and Engineering (NuME) research group led by Assistant Professor Andrea Sand in the Department of Applied Physics.
Lara Ejtehadian, Patrick Rinke, and Ilari Lähteenmäki sitting with coffee mugs and smiling to the camera.
Awards and Recognition, Research & Art Published:

Aalto Open Science Award Winner 2023 - Aalto Materials Digitalization Platform (AMAD)

We interviewed the AMAD team, winners of the first Aalto Open Science Award.
Lohmann, a white female, stands in front of a lit seaweed sculpture. Laakso, a white male, seated in a metallic room with wires.
Appointments, University Published:

Sparks of curiosity: two professors share what they wonder about

Two newly tenured Aalto professors talk about what drives them
People at the campus
Cooperation, Research & Art Published:

Aalto University to host CESAER Task Force Openness of Science and Technology

Aalto University is proud to host the CESAER Task Force Openness of Science and Technology on 16–17 April 2024.