Making computer-generated images look just right

Smart Art - Machine learning imperfections in materials

Computer generated images are everywhere, from television, cinema and computer games. Doing a half-hearted job of it though can be worse off than not doing it at all. As MIT researcher and former Aalto doctoral student Miika Aittala points out, “Each detail on the screen must be in order so that nothing out of place sticks out. Viewers are really good at noticing if the graphics are done wrong.”

This requirement to make sure that all computer-generated items in a scene look convincing is why the list of credits at the end of films have grown substantially longer, especially when they come to the parts listing everyone responsible for the visual effects. Automation of the laborious process of making computer generated objects look “just right” would allow CGI artists to spend more time actually creating art, as opposed to making their creations look passable. To achieve this goal, researchers in professor Jaakko Lehtinen’s group in the Department of Computer Science at Aalto have turned their A.I. algorithms on the problem. 

One of the common ways a computer-generated image will stand out as clearly fake is when its surface looks unconvincing. In real like, the way the surface of an object appears depends on how light interacts with the material, and if the computer gets this wrong the object on screen may appear either too shiny, or too dull. 

The reason surfaces are hard to perfect is that their appearance is dominated by their microscopic imperfections. These are the lumps and bumps that are too small to perceive by the naked eye alone, but together add to what makes the surface of objects appear different to each other.  Currently artists creating computer generated replications have to manually play around with a variety of unintuitive parameters that simulate these things like bumpiness to get the surface to look right. If a computer could be made to be able to predict what these parameters should be from a sample image, it would make the whole process a lot quicker. For example, if an artist wants a CGI leather briefcase in a scene, it would be great if the artist could feed the computer a few images of a leather surface and have the computer generate what the CGI briefcase looked like straight from the images. Even more ideal would be if the artist could upload simple phone images, and not have to worry too much about high resolution and controlled lighting.

To solve this, the researchers have been training a machine learning algorithm on a series of images of real‑world surfaces and corresponding computer‑generated objects with the same surface. The computer works out what parameters in the CGI object make it a good match for the real‑world surface by trying to predict what the parameters should be, and then checking if the predictions it made match the values of the real‑world example. Once the computer is able to predict these parameters correctly, it is able to start producing CGI models for completely new real‑world photos, such as the photo of the leathers in the examples above. These results, including generating simple CGI surfaces from simple phone images continue to influence Dr Aittala’s current research work. 

Neural networks and machine learning have begun radically changing many areas of human/computer interaction in unpredictable ways, and computer art is no exception. “From a researcher’s perspective, things are happening rapidly and in entirely new ways. Neural networks allow us to tackle problems that were difficult to grasp in any way earlier.
The operating model as a whole will start to become clearer once neural networks have been utilised for a sufficiently long time.” Says Dr Aittala.

More from this series

Smart Art - Machine learning imperfections in materials
Research & Art Published:

Making computer-generated images look just right

Machine learning imperfections in materials
Smart Art - Finland's Art and Craft School Robotti
Research & Art Published:

Rewiring art education

Käsityökoulu Robotti is an extra-curricular school for kids that is as much about having fun bending circuits as about learning by breaking boundaries.
Smart Art - Quantum Garden
Research & Art Published:

Quantum Garden uses sculpture to get the public to train AI

The sculpture gathers data sequences to help solve a quantum computing problem.
Smart Art - AI needs art
Research & Art Published:

AI’s story won’t be complete without art

Aalto’s Kasperi Mäki-Reinikka wants art and technology to collide and make something new

Related news

Screenshot from Interactive Diorama - Rembrandt 1632
Cooperation, Press releases, Research & Art, University Published:

Two unique exhibitions by Aalto University at the Oodi Library to experience and play

The Finnish EU presidency is now in full swing and for September (when the EU level AV conference will take place), Aalto is offering two very different exhibitions to be on display at the Oodi library in Helsinki.
Kuva: Tuomas Uusheimo.
Research & Art Published:

EURO2022 conference to be hosted by Aalto University

Largest European conference in operations research and management science to be hosted in Otaniemi in July 2022
Falling Walls
Research & Art Published:

Win a trip to Berlin with ideas pitching competition

Falling Walls Lab Finland submission period closes soon, enter your research or business idea to be in a chance to compete in the international final
Matias Palva. Kuva: Mikko Raskinen.
Research & Art Published:

Matias Palva, Professor of Brain Signal Analytics, studies the mechanisms of the human mind and brain diseases

Understanding the brain mechanisms behind the diseases is important when developing targeted treatments
  • Published:
  • Updated:
Share
URL copied!