News

Making computer-generated images look just right

Machine learning imperfections in materials
Smart Art - Machine learning imperfections in materials

Computer generated images are everywhere, from television, cinema and computer games. Doing a half-hearted job of it though can be worse off than not doing it at all. As MIT researcher and former Aalto doctoral student Miika Aittala points out, “Each detail on the screen must be in order so that nothing out of place sticks out. Viewers are really good at noticing if the graphics are done wrong.”

This requirement to make sure that all computer-generated items in a scene look convincing is why the list of credits at the end of films have grown substantially longer, especially when they come to the parts listing everyone responsible for the visual effects. Automation of the laborious process of making computer generated objects look “just right” would allow CGI artists to spend more time actually creating art, as opposed to making their creations look passable. To achieve this goal, researchers in professor Jaakko Lehtinen’s group in the Department of Computer Science at Aalto have turned their A.I. algorithms on the problem. 

One of the common ways a computer-generated image will stand out as clearly fake is when its surface looks unconvincing. In real like, the way the surface of an object appears depends on how light interacts with the material, and if the computer gets this wrong the object on screen may appear either too shiny, or too dull. 

The reason surfaces are hard to perfect is that their appearance is dominated by their microscopic imperfections. These are the lumps and bumps that are too small to perceive by the naked eye alone, but together add to what makes the surface of objects appear different to each other.  Currently artists creating computer generated replications have to manually play around with a variety of unintuitive parameters that simulate these things like bumpiness to get the surface to look right. If a computer could be made to be able to predict what these parameters should be from a sample image, it would make the whole process a lot quicker. For example, if an artist wants a CGI leather briefcase in a scene, it would be great if the artist could feed the computer a few images of a leather surface and have the computer generate what the CGI briefcase looked like straight from the images. Even more ideal would be if the artist could upload simple phone images, and not have to worry too much about high resolution and controlled lighting.

To solve this, the researchers have been training a machine learning algorithm on a series of images of real‑world surfaces and corresponding computer‑generated objects with the same surface. The computer works out what parameters in the CGI object make it a good match for the real‑world surface by trying to predict what the parameters should be, and then checking if the predictions it made match the values of the real‑world example. Once the computer is able to predict these parameters correctly, it is able to start producing CGI models for completely new real‑world photos, such as the photo of the leathers in the examples above. These results, including generating simple CGI surfaces from simple phone images continue to influence Dr Aittala’s current research work. 

Neural networks and machine learning have begun radically changing many areas of human/computer interaction in unpredictable ways, and computer art is no exception. “From a researcher’s perspective, things are happening rapidly and in entirely new ways. Neural networks allow us to tackle problems that were difficult to grasp in any way earlier.
The operating model as a whole will start to become clearer once neural networks have been utilised for a sufficiently long time.” Says Dr Aittala.

More from this series

Smart Art - Machine learning imperfections in materials
Research & Art Published:

Making computer-generated images look just right

Machine learning imperfections in materials
Smart Art - Finland's Art and Craft School Robotti
Research & Art Published:

Rewiring art education

Käsityökoulu Robotti is an extra-curricular school for kids that is as much about having fun bending circuits as about learning by breaking boundaries.
Smart Art - Quantum Garden
Research & Art Published:

Quantum Garden uses sculpture to get the public to train AI

The sculpture gathers data sequences to help solve a quantum computing problem.
Smart Art - AI needs art
Research & Art Published:

AI’s story won’t be complete without art

Aalto’s Kasperi Mäki-Reinikka wants art and technology to collide and make something new
  • Published:
  • Updated:
Share
URL copied!

Read more news

Logo for ABRA consisting of a grey a-shaped logo and the text ARTIFICIAL BIOLOGY, ROBOTICS & ART
Cooperation, Research & Art Published:

Call for interest to participate in the Artificial Biology, Robotics, and Art (ABRA) Workshop Series

ABRA is calling for applications to participate in a series of international transdisciplinary workshops between September 2021 and June 2023.
Kimchi and Chips
Research & Art Published:

Artist talk: Kimchi and Chips

We are pleased to welcome you to an artist talk by Kimchi and Chips!
02. sample holder-measurement-low temperature
Research & Art Published:

Data Vitality: Open Call

Exhibition Open Call and Reading Group
visualNoise.jpg
Research & Art Published:

Mind Meets Machine: Immersive Data Interaction

The exhibited projects showcase novel computational design methods to creatively interact with global and local data. Whereas the on-going general discussions in relation to data-handling stresses efficiency, productivity and often de-humanization, the exhibited projects set their focus on the exploration of immersive data-interaction design tools to enhance climate-smart solutions in the urban context. The projects aim for showcasing possibilities to make abstract data visible and tangible, to enter a new level of data-informed responsive design, that has public benefit.
Augmented data-interaction is merged with the tangible sandbox environment, in order to formulate sustainable design speculations for Ainonauki, an urban plaza located between Väre and Aalto Studios. The understanding of available technology to be composed and rearranged in order to support the field specific discourse led to innovative design speculations, based upon human-robotic interaction. The developed design methodology emphasizes the importance of design iterations through computationally informed feedback loops.