Abstract: 

We will take the audience on a musical journey. The journey of an artificial intelligence learning how to reinvent musical instruments. Starting with digitally interpreting sounds, mimicing instruments using feed-forward neural nets, and ending with a metamorphosis of instruments using Generative Adversarial Networks.

Synthesizing techniques, such as subtractive and sample-based synthesis are used to create sounds of instruments we know but require a lot of tuning. A less used technique, additive synthesis, consists of combining basic sounds. This can also be used by an artificial intelligence to generate these sounds for us. And maybe more..

Computations and interpretations on digital signals, such as sound recordings, are typically complex and not very intuitive. Fourier transformations have allowed us to decompose complex sounds into combinations of sine waves. This exactly what we need for additive synthesis.

We started of simple by training a model to combine sine waves to mimic existing sounds. Using a large set of recorded sounds with known tone and instrument, we trained a supervised feed-forward neural net to generate the spectrograms based on tone and instrument. Based on the spectrogram we can easily replay the actual sound.

Obviously we did not go through all this effort to play the sounds that we already know. We want to explore the unexplored, travel the rimworld of sound space! Generative Adversarial Networks require us to set up two seperate networks. A generating network, inventing sounds, trying to please a second network, which is trained to recognize known instruments. At first this allows us to create sounds which are very similar to the instrument the second network is trained to recognize. However, as we tweak the settings of the generative network, we can morph to a new paradigm of sounds.

Bio: 

I am Laurens, Machine Learning Engineer at Dataworkz. I have a background in Artificial Intelligence and worked for several years as lead data scientist (at ProRail). I like to be on the edge where business meets tech, where AI and machine learning make an impact. Currently I am working as a data engineer / data architect at the Port of Rotterdam, via Dataworkz. I am intrigued by the creative side of artificial intelligence; can a machine show symptoms of creativity or can it only reproduce what it has seen before? How is that different from what we can do? The subject we are talking about is inspired by David's love for music and my urge to do cool stuff in the field of data science.