News

Researchers develop a new way to instruct dance in Virtual Reality

The new virtual reality technique makes it easier to join a dance routine without first having to learn choreography.
The WAVE technique developed by the researchers is based on anticipating future movement, such as a turn. Picture: Markus Laatta
The WAVE technique developed by the researchers is based on anticipating future movement, such as a turn. Picture: Markus Laattala.

Researchers at Aalto University were looking for better ways to instruct dance choreography in virtual reality. The new WAVE technique they developed will be presented in May at the CHI conference, a major venue for human-computer interaction research. 

Previous techniques have largely relied on pre-rehearsal and simplification. 

‘In virtual reality, it is difficult to visualise and communicate how a dancer should move. The human body is so multi-dimensional, and it is difficult to take in rich data in real time,’ says Professor Perttu Hämäläinen.

The researchers started by experimenting with visualisation techniques familiar from previous dance games. But after several prototypes and stages, they decided to try out the audience wave, familiar from sporting events, to guide the dance.

‘The wave-like movement of the model dancers allows you to see in advance what kind of movement is coming next. And you don't have to rehearse the movement beforehand,’ says PhD researcher Markus Laattala.

In general, one cannot follow a new choreography in real time because of the delay in human perceptual motor control. The WAVE technique developed by the researchers, on the other hand, is based on anticipating future movement, such as a turn.

‘No one had figured out how to guide a continuous, fluid movement like contemporary dance. In the choreography we implemented, making a wave is communication, a kind of micro-canon in which the model dancers follow the same choreography with a split-second delay,’ says Hämäläinen.

From tai chi to exaggerated movements

A total of 36 people took part in the one-minute dance test, comparing the new WAVE visualization to a traditional virtual version in which there was only one model dancer to follow. The differences between the techniques were clear.

‘This implementation is at least suitable for slow-paced dance styles. The dancer can just jump in and start dancing without having to learn anything beforehand. However, in faster movements, the visuals can get confusing, and further research and development is needed to adapt and test the approach with more dance styles’ says Hämäläinen.

In addition to virtual dance games, the new technique may be applicable to music videos, karaoke, and tai chi.

‘It would be optimal for the user if they could decide how to position the model dancers in a way that suits them. And if the idea were taken further, several dancers could send each other moves in social virtual reality. It could become a whole new way of dancing together’, says Laattala.

‘Current mainstream VR devices only track the movement of the headset and handheld controllers. On the other hand, machine learning data can sometimes be used to infer how the legs move,’ says Hämäläinen.

‘But in dance, inference is more difficult because the movements are stranger than, for example, walking,’ adds Laattala.

On the other hand, if you have a mirror in the real dance space, you can follow the movement of your feet using machine vision. The dancer's view could be modified using a virtual mirror. 

‘A dancer's virtual performance can be improved by exaggeration, for example by increasing flexibility, height of the jumps, or hip movement. This can make them feel that they are more skilled than they are, which research shows has a positive impact on physical activity motivation,’ says Hämäläinen.

The virtual dance game has been developed using the Magics infrastructure's motion capture kit, where the model dancer is dressed in a costume with sensors. These have been used to record the dance animation.

The WAVE dance game can be downloaded for Meta Quest 2 and 3 VR devices here.  The Github repository  also includes the open source code that anyone can use to develop the game further.

Reference:

Laattala, M., Piitulainen, R., Ady, N., Tamariz, M., & Hämäläinen, P. (2024). Anticipatory Movement Visualization for VR Dancing. ACM SIGCHI Annual Conference on Human Factors in Computing Systems

PDF of the study available here

Contact information:

Aalto computer scientists in CHI 2024

Ten papers from Aalto CS were accepted to the CHI 2024 conference

News
FCAI

Finnish Center for Artificial Intelligence (external link)

The Finnish Center for Artificial Intelligence FCAI is a research hub initiated by Aalto University, the University of Helsinki, and the Technical Research Centre of Finland VTT. The goal of FCAI is to develop new types of artificial intelligence that can work with humans in complex environments, and help modernize Finnish industry. FCAI is one of the national flagships of the Academy of Finland.

  • Published:
  • Updated:

Read more news

Nesteen polttoainejalostamon tankit Porvoossa
Cooperation, Research & Art Published:

Collaboration with Aalto brings significant financial benefits to Neste

As a result of the collaboration, Neste is now using digital tools in its fuel research alongside traditional testing.
Laboratory work at the School of Chemical Engineering
Cooperation Published:

Join the Unite! Engineering Biology matchmaking event on 11 September

The Unite! Engineering Biology initiative aims to unite researchers from all the nine partner universities in the Unite! alliance to advance key technologies and sustainable processes.
Nighttime picture of a snowy road lined with trees.
Press releases Published:

Snap, crackle, pop! — the sounds of frost actually come from the sky

Breakthrough research reveals that the popping sound associated with frost is in fact mostly coming from the atmosphere.
Comic-style illustration of Solip Park's research methods
Awards and Recognition Published:

Doctoral Researcher Solip Park's Paper Receives Honorable Mention at CHI 2024

Doctoral researcher Solip Park's paper has recently garnered attention at the prestigious CHI 2024 conference, earning an "honorable mention" distinction.