Artificial intelligence introduced to design

Professor Antti Oulasvirta has a novel idea: he wants to create self-optimising user interfaces that automatically adapt to the needs of individuals or groups by collecting and analysing data on them.

Nowadays, the usual approach is for a design team to first conduct an analysis and then come up with and test ideas before presenting the client with a solution. Oulasvirta points out a problem with this process.

“A mere eight side elements can be combined in 1080 ways, a number which is impossible for humans to comprehend. There is no way designers can consider all the different alternatives, so they often lean towards solutions with which they are already familiar.” 

Oulasvirta's research group seeks out the best possible solutions with the aid of algorithms. An algorithm is a detailed set of operating instructions that can be programmed to search for alternatives in an intelligent way. A search space, i.e. a set of possible designs such as all the ways in which a website's elements can be arranged, is defined for the algorithm.

“Enormous volumes are not a problem for an algorithm, and it can easily seek out and suggest suitable solutions from amongst an almost endless pool of alternatives. Designers have told us that they especially appreciate the fact that the designs proposed by our algorithm are different.”

Time, money and patience

First, the algorithm must be taught what people consider good and bad. Researchers define good and bad using mathematical and simulation models of, among other things, motor activity, learning, decision-making or visual attention. Oulasvirta, who started his career in the field of cognitive science, points to the bookcase that covers the wall of his office.

“If you had to find a specific black book from among all the black books there, it would take some time. If, however, you knew that the book is blue and stacked on its side, it would be much easier to find. Similarly, computers must know principles of human visual attention when proposing user interface designs," he illustrates.

As an ever-larger amount of working and free time is spent in front of a screen, even the most minor shortcoming in a user interface can present a problem for a great many people. If a site has too few functions, visitors will not find the things that they desire. And if there are too many, searching becomes frustrating and exhausting for visitors. The conceptualisation of the entire website can also be wrong, making it hard for visitors to deduce under which category the subjects they are looking for are located. This can lead to visitors becoming increasingly lost, drifting every further away from the object of their search and eventually abandoning the site, user interface or even the entire service.

“The problems with the Symbian user interface were a tough nut for Nokia,” Oulasvirta reminds.

“And user reactions forced Microsoft to recall its operating system, which applied a living room metaphor.”

Oulasvirta says algorithms can be harnessed to create more effective user interfaces in two ways. One option is to pass the ball to the users, allowing them to first define their objectives and then supplying them with a proposal tailored to their needs.

“This would represent a true fast-forward leap from the present situation in which we take a ready foundation and adjust our ideas to it,” he says with a smile.

“In another scenario, a watcher is added onto a finished user interface to observe users and, based on its observations, either automatically improve functionality – in the case of a website, for example, by changing the placement of links, image sizes and headers – or perhaps by suggesting changes to the structure of the site. We also perform design mining, i.e. algorithmically go through thousands of websites to identify a statistical average that can then be used as a basis for design.”

What can a machine learn?

A self-optimising web service is an example of artificial intelligence, which arouses expectations and even some fears. Usually it is taken to refer to computers or software that are capable of intelligent functions.

“In the beginning, the question was: chess is a really hard game, so could a computer learn to play it? After some pondering, researchers realised that the problem was in fact simple considering that, although the search space is expansive, it is also well defined,” Antti Oulasvirta says.

A historic moment came in 1997, when the IBM-designed AI Deep Blue beat chess superstar Garry Kasparov in a match held in New York. Accepting machine dominance in a game of logic is fairly easy to stomach, but seeing them move into design is an entirely different matter.

“Design is associated with visuality, creativeness and a lot of human understanding, and the belief has been that these lie in an area computers cannot reach. This is why design has long been considered a field, which cannot be automated,” Oulasvirta muses.

“Many fairly mundane chores, such as collecting the dishes, have proven difficult for computers. But when you think about it more closely, you'll see that designing a user interface more closely resembles chess than everyday household chores – the search space can be defined very accurately for it as well.”

The automation of design does not mean that designers are about to disappear, however.

“It would hardly represent a sensible use of time if a designer's job mostly involved just linking different pieces together. Automation allows humans to focus on higher-level thinking that cannot be assigned to a computer; the consideration of new service concepts, branding, researching use practices and so on. The situation somewhat resembles aeroplanes and the autopilot function: pilots no longer perform basic flight tasks, which are instead delegated to a computer.”

Diagnoses, art and architecture

Oulasvirta stresses that user-friendly information technology is much more than just smart menus and quick searches; use needs to feel meaningful, empowering and manageable. Improving services that are used by the masses can yield enormous benefits: a smoother user experience brings about reductions in on-the-job mistakes, improved ergonomics and frees up resources. Automating design also makes it possible to focus more attention to individual needs, such as blindness or ageing-related challenges that have perhaps not been prioritized up to now because of insufficient resources.

The development of automation and artificial intelligence has been breathtaking in recent years. IBM's Watson recently diagnosed a 60-year-old woman with a rare form of leukaemia, which her doctors had failed to spot. Machines are making rapid inroads into other creative industries besides design as well. For example, an algorithm has been taught to paint like van Gogh, and researchers at Aalto University's Department of Computer Science have been involved in the development of the DeepBeat rap generator, which produces lyrics with better rhyme factors than those written by professional rappers.

Oulasvirta also mentions Aalto Professor Toni Kotnik, an expert in computational architecture, who is busy developing an algorithm, which can propose creative architectural solutions that nevertheless also form viable structures.

“There is a wealth of possible applications in fields like robotics, making the study of artificial intelligence and the computational sciences a very safe bet for the future.”

• Antti Oulasvirta heads the User Interfaces research group at the Department of Communications and Networking. The European Research Council has awarded him a €1.5-million Starting Grant for studying the optimisation of user interfaces. The research project "Itseoptimoivat www-palvelut" conducted by Oulasvirta has got a €370,000 grant from the Jane and Aatos Erkko Foundation and the Technology Industries of Finland Centennial Foundation.

Text: Minna Hölttä. Photo: Jussi Särkilahti.

The article is originally published in Finnish in Aalto University Magazine 17 (issuu.com), October 2016.