News

Acoustics researchers decompose sound accurately into its three basic components

Any sound can now be perfectly replicated by a combination of whistles, clicks, and hisses, with implications for sound processing across the media landscape

Researchers have been looking for ways to decompose sound into its basic ingredients for over 200 years. In the 1820s, French scientist Joseph Fourier proposed that any signal, including sounds, can be built using sufficiently many sine waves. These waves sound like whistles, each have their own frequency, level and start time, and are the basic building blocks of sound.

However, some sounds, such as the flute and a breathy human voice, may require hundreds or even thousands of sines to exactly imitate the original waveform. This comes from the fact that such sounds contain a less harmonical, more noisy structure, where all frequencies occur at once. One solution is to divide sound into two types of components, sines and noise, with a smaller number of whistling sine waves and combined with variable noises, or hisses, to complete the imitation.

Even this ‘complete’ two-component sound model has issues with the smoothing of the beginnings of sound events, such as consonants in voice or drum sounds in music. A third component, named transient, was introduced around the year 2000 to help model the sharpness of such sounds. Transients alone sound like clicks. From then on, sound has been often divided into three components: sines, noise, and transients.

The three-component model of sines, noise and transients has now been refined by researchers at Aalto University Acoustics Lab, using ideas from auditory perception, fuzzy logic, and perfect reconstruction.

Decomposition mirrors the way we hear sounds

Doctoral researcher Leonardo Fierro and professor Vesa Välimäki realized the way that people hear the different components and separate whistles, clicks, and hisses is important. If a click gets spread in time, it starts to ring and sound noisier; by contrast, focusing on very brief sounds might cause some loss of tonality.

Leonardo Fierro (vas.) ja professori Vesa Välimäki kokeilivat Sitranoa Aalto-yliopiston akustiikan laboratorion Otala-kuunteluhuoneessa. Kuva: Anna Berg / Aalto-yliopisto.
Leonardo Fierro (left) and Professor Vesa Välimäki demonstrated the enchanted decomposition method in the Otala listening room of Aalto Acoustics Lab. Photo: Anna Berg / Aalto University.

This insight from auditory perception was coupled with fuzzy logic: at any moment, part of the sound can belong to each of the three classes of sines, transients or noise, not just one of them. With the goal of perfect reconstruction, Fierro optimized the way sound is decomposed. 

In the enhanced method, sines and transients are two opposite characteristics of sound, and the sound is not allowed to belong to both classes at the same time. However, any of two opposite component types can still occur simultaneously with noise. Thus, the idea of fuzzy logic is present in a restricted way. The noise works as a fuzzy link between the sines and transients, describing all the nuances of the sound that are not captured by simple clicks and whistles. ‘It’s like finding the missing piece of a puzzle to connect those two parts that did not fit together before,’ says Fierro. 

This enhanced decomposition method was compared with previous methods in a listening test. Eleven experienced listeners were individually asked to audit several short music excepts and the components extracted from them using different methods. 

The new method emerged as the winning way to decompose most sounds, based on the listeners’ ratings. Only when there is a strong vibrato in a musical sound, such as in a singing voice or the violin, all decomposition methods struggle, and in these cases some previous methods are superior.

A test use case for the new decomposition method is the time-scale modification of sound, especially slowing down of music. This was tested in a preference listening test against the lab’s own previous method, which was selected as the best academic technique in a comparative study a few years ago. Again, Fierro’s new method was a clear winner.

‘The new sound decomposition method opens many exciting possibilities in sound processing,’ says professor Välimäki. ‘The slowing down of sound is currently our main interest. It is striking that for example in sports news, the slow-motion videos are always silent. The reason is probably that the sound quality in current slow-down audio tools is not good enough. We have already started developing better time-scale modification methods, which use a deep neural network to help stretch some components.’

The high-quality sound decomposition also enables novel types of music remixing techniques. One of them leads to distortion-free dynamic range compression. Namely, the transient component often contains the loudest peaks in the sound waveform, so simply reducing the level of the transient component and mixing it back with the others can limit the peak-to-peak value of audio.

Reference:

Fierro, L. & Välimäki, V. (2023). Enhanced Fuzzy Decomposition of Sound Into Sines, Transients, and Noise. Journal of the Audio Engineering Society. doi: 10.17743/jaes.2022.0077

Contact:

Leonardo Fierro
Doctoral Researcher, Aalto University, Department of Information and Communications Engineering, Acoustics Lab
[email protected] 

Vesa Välimäki
Professor, Aalto University, Department of Information and Communications Engineering, Acoustics Lab
[email protected] 
phone + 358 50 569 1176

 

Aalto University Acoustic Lab

Aalto Acoustics Lab

The Aalto Acoustics Lab is a multidisciplinary research center focusing on audio processing and spatial sound technologies. The laboratory gathers professors and research teams from three different units: Department of Information and Communications Engineering, Department of Computer Science, and Department of Art and Media.

  • Published:
  • Updated:

Read more news

Three photos on blue background showing adults and children standing around tables
Campus, Research & Art Published:

"Bring your child to work day" 2024 at the Department of Applied Physics

Find out about a fun morning spent making ice cream for children hosted by the Department of Applied Physics
Modern and Mesopotamian people experience love in a rather similar way. In Mesopotamia, love is particularly associated with the liver, heart and knees. Figure: Modern/PNAS: Lauri Nummenmaa et al. 2014, Mesopotamian: Juha Lahnakoski 2024.
Press releases Published:

We might feel love in our fingertips –– but did the Ancient Mesopotamians?

A multidisciplinary team of researchers studied a large body of texts to find out how people in the ancient Mesopotamian region (within modern day Iraq) experienced emotions in their bodies thousands of years ago, analysing one million words of the ancient Akkadian language from 934-612 BC in the form of cuneiform scripts on clay tablets.
Three white, folded paper structures of varying sizes and shapes arranged on a grey surface.
Cooperation, Press releases, Research & Art Published:

New origami packaging technology creates sustainable and eye-catching alternatives to conventional packing materials

Origami packaging enables completely new properties for cartonboard, making it an excellent alternative to, for example, plastic and expanded polystyrene in packaging. The aesthetics of the material have also garnered interest from designers.
Jose Lado.
Research & Art Published:

Quantum physics professor searches for exotic qubit alternatives with new European funding

Aalto University physics professor Jose Lado will use this funding to engineer a new type of topological quantum material that could have applications for quantum bit, or “qubit,” development for noise-resilient topological quantum computation.