The research puzzle of when humans and AI don’t see eye to eye

A sticker on a traffic sign can cause a self-driving car to crash. A few pixels of noise can make a neural network decide that an image of a dog actually shows a bird. These are just some examples of how AI systems can behave unreliably, displaying an inconsistency that is a bit hard for people to understand. “Why this happens is a mystery,” says Francesco Croce. “Features that are very important for humans are not important for models and vice versa, and why models solve tasks in odd ways is also a safety and a legal concern.”

Croce’s career so far has taken him from his native Italy, where he studied mathematics, to Germany for a PhD in machine learning and to London for an internship with Google DeepMind. “It’s lucky that I've been to several places where there were great researchers,” he says. Though ending up in Finland was a bit random, Croce admits, the establishment of the ELLIS Institute was a motivator to apply. “There’s a solid plan for working with AI research in Europe, the computing power of course, and they are building something that will hopefully last long and have an impact.”

At the ELLIS Institute, Croce is now set to recruit PhD students and postdocs who are interested in applying adversarial robustness to multi-modal models. He previously developed a benchmark for testing and tracking the progress of robustness, and is now keen to extend the work to other domains like generative AI.

Students should actually enjoy the research process, says Croce—“not just writing papers and getting lines on your CV”. Creativity is also an important trait for researchers: “To imagine something that doesn’t exist yet and find the good questions, to understand what’s currently missing and what’s relevant to push things forward.”