News

Gesture recognition technology shrinks to micro size

New resource-efficient gesture recognition can be embedded into smart clothing. The technology developed in collaboration between Aalto University and company HitSeed could be used in manufacturing and healthcare, for example.
A person with AR glasses and a smart glove compiling a demo set
Sensor-based smart gloves allow the employee to interact in real time with an augmented reality application that can provide visual and haptic feedback. Image: Mariela Urra Schiaffino

The use of augmented reality (AR) applications and wearable electronics is constantly increasing in the industry. For instance, smart glasses can show an employee real-time instructions on how to assemble a device or help find parts that need service. Smart textiles based on sensor technology, such as smart gloves, can convert physical movements into virtual equivalents and quite literally guide the employee by the hand.

The smart glove must be able to accurately detect hand and finger movements and grip force. This is often done using deep neural networks, machine learning methods that mimic the function of the human brain and traditionally require a lot of computational power. Researchers at Aalto University have collaborated with HitSeed, a company that specialises in intelligent sensor technology, to develop gesture recognition that can be used on even fingertip-sized microcontrollers.

‘Usually, sensor data collected by gloves needs to be sent over a network to a computer that processes it and sends the information back. The deep learning-based gesture recognition algorithms we have developed are so lightweight that they can do the same locally in an embedded system like smart gloves,’ says Yu Xiao, a researcher at Aalto University who is the leader of a research group that specialises in wearable systems development.

This means that the devices can be used anywhere, without the need for internet connection or an external computer. The information can be transferred between the smart gloves and AR glasses using the Bluetooth Low Energy network.

The technology could be used in a variety of embedded systems for sensor data in the future.

’We can apply the developed technology for several measurement types like keeping separate counts for multiple gestures, for measuring motion improvements in physiotherapy or for detecting the state of multiple machines running based on a vibration or sound spectrum,’ says HitSeed CTO Pertti Kasanen.

‘Smart sensors and augmented reality and virtual reality applications have endless opportunities in industry, healthcare and education,’ Xiao says.

The researchers used HitSeed’s fingertip-sized Sensor Computer, which supports Google’s Tensor Flow Lite software library, to run convolutional neural networks (CNNs) on smart gloves. CNN is a specialised type of neural network which is often used for image classification. A CNN is made up of neurons that have learnable weights and biases. As a next step, the system will be extended to support local execution of long short-term memory (LSTM), which is commonly used for processing entire sequences of data such as speech and video.

The research project received €100,000 seed funding from the European Union's Horizon 2020 ATTRACT project, which supports collaboration between research institutes and companies to develop technologies that change society. Next, researchers will seek partners for research aimed at commercialising the technology.

More information about the technology (pdf).

Contact

Professor Yu Xiao
Department of Communications and Networking
[email protected]

Pertti Kasanen
Partner, CTO/HitSeed 
[email protected]

  • Published:
  • Updated:
Share
URL copied!

Read more news

Arvo Kokkonen, kuva Ilona Savola, MLL
Appointments, Cooperation Published:

Director General Arvo Kokkonen to chair School of Engineering Stakeholder Advisory Board

The School of Engineering Stakeholder Advisory Board, launched in 2021, promotes the impact of the school's research and education among Finnish industries and society.
Omnia Makerspace ja studio
Campus, Cooperation Published:

Colliding students and startups in A Grid’s Omnia Makerspace

Omnia Makerspace was launched in startup hub A Grid’s first floor opposite Restaurant Factory in April 2019. Since then, the space has evolved from an empty basement into a flourishing workspace where students and startups collide.
Logo
Cooperation, Press releases, University Published:

The higher education community of the Helsinki metropolitan area gets together: EveryMoveCounts-campaign starts!

The goal of the EveryMoveCounts-campaign is to increase the students’ awareness of the benefits of daily movement for their well-being.
Paper packaging prototypes - miura ori technique. Photo: Valeria Azovskay
Cooperation Published:

Multidisciplinary research project based on origami seeks solutions for industrial needs

A joint project covering the fields of mathematics, engineering and arts is about to start at Aalto, where the aim is to study and develop folding technology for the needs of industry. In addition to Aalto, the project includes VTT as coordinator, and several companies from the forest industry to the machine design and cosmetics brands, such as Stora Enso, Metsä Group, Lumene, Mirka, Elomatic, Anpap, Orfer and Soften. Business Finland granted funding for the Co-Innovation project.