theharsh-artofsound

[theharsh-artofsound]

This is a demo of my project which aims to combine 2 different sounds and create a completely new sound from them. It uses phase vocoding and a spectral delay for creating unique sounds.

A commentary of what is going on in the video along with approximate time stamps is provided below for convenience:

0:04 -  A demo of the sample sounds used without any processing. Samples are of birds chirping, background noise, yawning and water in a shower. All samples were recorded during the sound walk activity.

0:38 - A demo of all created vocoding "voices" is shown.

1:52 - A demo of the 64-band delay is shown. There are a total of 64 horizontal bars in each graph area, which correspond to the 64-bands of delay. Since the figure is small, it looks like a drawing!

2:10 - This gain setting makes it so that only a few frequency bands are allowed into the delay. The resulting delay can be thought of as a filtered version of a normal delay.

2:19 - From here onwards, various possibilities of producing sound are explored by changing settings in the instrument.

5:34 - A demo for the instrument using live sound (microphone input) is shown. The microphone is connected to the computer using an audio interface.

6:16 - A demo for the instrument using live sound (electric guitar input) is shown. The guitar is connected to the computer using an audio interface.

8:42 - Just a quick overview of how the patching is done and looks inside the Max patch. The presentation mode UI was made to look simple and neat, but there is actually a lot going on behind the scenes!

I hope you liked what you heard, or if it at least intrigued you. Please take a look at my earlier posts in the blog for more information about the project.

[theharsh-artofsound]

This is the Max/MSP user interface that I created for the instrument. Currently, it has 4 samples loaded into it that I recorded during my sound walks. The green bo is the input stage, the blue one is the output stage, and the pink one is the delay stage.

The input stage has options to switch between different sound samples and the audio interface inputs (can be any 2 inputs, in my case a microphone and an electric guitar since my interface has only 2 inputs), and a spectrogram that shows the input signal.

The output stage has options to switch between different vocoding signals and trigger normalization of audio playback to ensure good volume of playback, and a spectrogram that shows the output signal.

The delay stage has options to change gain, delay time and feedback amount (%) of each of the 64 frequency bands of the audio. there is also an option to boost the delay level through the overdrive knob. the 2 spectrograms show the input signal to the delay and the output from the delay.

[theharsh-artofsound]

Random Pitch Vocoder Synth

Yes. This is what I have ultimately come up with for the name! Since the pitch detection is all over the place, I thought this would be the most fitting name.

My instrument is a phase vocoder that identifies the 3 most prominent frequencies instead of the traditional 1 frequency, and uses those identified frequencies to generate a sound using a vocoding signal consisting of those 3 frequencies. To give my instrument an even more uniques ound, I decided to add the spectral delay effect that I learnt some time back, with 64 delay frequency bands.

As of now, the instrument uses either a highly filtered noise signal, saw-wave signal, sine-wave signal or a combination of those 3 as the vocoding signal. It is capable of taking any kind of audio input, even another instrument, voice or random sound.

Since the frequency detection happens in almost real-time, any real-time input can be processed through it and vocoded successfully. This real-time detection is also the reason why the detected frequency fluctuates a lot for most sounds since most sounds change their “hottest” frequencies very frequently as they vary in time. This is what led to the instrument being called a random pitch vocoder synth. Now you know the lore!

[theharsh-artofsound]

Cross-Synthesis & Vocoder Tutorials

I have been looking at a number of sources for help with creating a cross-synthesizer, simple vocoder, LPC vocoder or a phase vocoder in Max/MSP, and it has been really hard to find very helpful resources, but it’s better than having nothing!

Here are a few links that I went through and tried to implement:

https://ccrma.stanford.edu/~ngang/220a/fp.html

https://cycling74.com/tutorials/a-basic-vocoder-tutorial-part-1

https://cycling74.com/tutorials/the-phase-vocoder-%E2%80%93-part-i

There were some which I could follow and implement but didn’t work the way I hoped they would, while some were really difficult to understand and I couldn’t help but leave them be since time isn’t exactly on my side right now!

My next update would hopefully be the final product since i am trying mybest to understand things, implement them and wrap the project up in time for the deadline... Look forward to it!

[theharsh-artofsound]

I found this pretty advanced yet really great tutorial which explains and guides us in developing a spectral delay effect! A spectral delay effect is just another delay effect, but applied separately to each frequency band the sound has been divided into whie processing.

I find the ability to control each frequency band’s delay separately really cool, as is the sound that comes out of it!

[theharsh-artofsound]

Phase Vocoder

It has been another hectic week with a lot of assignment submissions, but I managed to look up some more about vocoders and came across something known as a phase vocoder.

Pic source: https://www.researchgate.net/figure/Block-diagram-of-the-phase-vocoder-as-used-for-analysis_fig1_3333551

As per Wikipedia, “A phase vocoder is a type of vocoder-purposed algorithm which can interpolate information present in the frequency and time domains of audio signals by using phase information extracted from a frequency transform. The computer algorithm allows frequency-domain modifications to a digital sound file (typically time expansion/compression and pitch shifting).”

Essentially, a sound signal is converted from samples in time to samples in frequency using FFT (fast fourier transform), from where we can get the phase and amplitude data of the sound. The phase data is used to predict the dominant frequency in the sound.

I plan to test this method and see if I get any interesting results. I have started looking into Max/MSP software too, which is where I will be testing this since I plan on using the software to create my instrument anyway. It is a very powerful DSP software and equally hard to learn for me since I’m fairly new to it, but I will try my best!

[theharsh-artofsound]

Vocoder

Since my last update, I have been testing various sounds in place of my guitar sound since I wanted to mix 2 sounds from the sound walks, without adding an actual instrument’s sound. Apart from the distorted guitar sound, I haven’t had much success with other sounds in that the sound produced after cross-synthesis isn’t how I imagined the outcome to be.

I have now shifted focus to researching about vocoders since LPC analysis is also used in creating vocoders. In fact, LPC cross-synthesis is itself a form of vocoding. Since I have already been vocoding without even knowing it, I believe it’s best to try and understand it a bit better as it may reveal a path to my expectation with the sound I envisage.

The act of creating the new sound by replacing the original source is known as cross-synthesis, which is a form of vocoding.

[theharsh-artofsound]

This is a sample audio clip wherein I re-synthesized my guitar’s sound (with added distortion effect) and cross-synthesized it with a sample viola and violin sound using LPC analysis in MATLAB.

The audio contains re-synthesized guitar, cross-synthesized guitar with viola and cross-synthesized guitar with violin sounds (in said order), with a total of 4 different musical notes being played.

The cross-synthesized sounds are audibly very different from the original guitar sound, with a hint of either the viola’s or violin’s resonance.

[theharsh-artofsound]

LPC Analysis

LPC analysis involves breaking a sound into a “source” sound and a “residual” sound. The source can be thought of as an impulsive sound, which when combined with the residual sound (or the resonant filter), gives us the characteristic sound that we know it as.

This is a digital signal processing (DSP) concept that can be used to create synthesizers and vocoders. You can find more detailed information about this in DSP books.

[theharsh-artofsound]

Experimental Instrument

I am thinking of creating a software-based instrument since I don’t have access to a lot of hardware materials right now. Since I collected a lot of sounds for the schizophonia project as well as sound walks, I want to somehow find a way to combine them to create a new sound using software.

I have learnt about using LPC analysis to re-synthesize or cross-synthesize sounds in oneof my subjects. According to Wikipedia, “Linear predictive coding (LPC) is a method used mostly in audio signal processing and speech processing for representing the spectral envelope of a digital signal of speech in compressed form, using the information of a linear predictive model.LPC is the most widely used method in speech coding and speech synthesis.”

I plan to take a deeper look into LPC in order to possibly create new sounds by cross-synthesis. For proof-of-concept, I plan to use MATLAB software sinceI’m familiar with it.

Stay tuned for audio test samples!

[theharsh-artofsound]

Here is the final mix for the Schizophonia project. Below is a brief description about the sound work:

This work explores the idea of a soundscape inside a dream and tries to portray the mundaneness of daily life in the city, convoluted though hidden emotions of the person dreaming. The dream includes day-to-day activities such as walking, taking transportation and listening to people talking, as well as some nature sounds such as animals, rain, wind and thunder. Towards the end, the dream becomes more intense, when it is suddenly interrupted by the sound of the alarm. The person then wakes up to realize it was a dream, and proceeds to move on with daily life. All sounds used in the work are schizophonic in that they were derived from other sounds through audio processing. The work makes use of time stretching/contracting, pitch changes, reverb, delay, compression, distortion, de-noising, filters, equalizers, vocoding, slicing audio and time reversal.

[theharsh-artofsound]

Schizophonia Idea

I am currently exploring ideas and effects for the sounds that I have recorded. My idea for the schizophonia project is to create a city soundscape using the sounds that I have recorded on several occasions, including the sound walk recordings.

Since almost all the sounds are originally from the city, the challenge is to either re-create the soundscape with different sounds replacing the original sources while still giving a city feel, or to create an imaginary city-like soundscape using the existing source sounds and modulating them into something different using various effects.

[theharsh-artofsound]

Sound of the waterfall changing as I move around the structure, creating a filter/distance effect, even though I was standing right next to it!

[theharsh-artofsound]

Another unique sound effect I came across at my friend’s house was the sound of the waterfall in their pool. What was unique about the waterfall you may wonder? There was nothing unique about the sound of the waterfall itself, but the sound seemed to change when I walked around it. Moving closer to the back of the waterfall structure seemed to create a filtering effect on the sound of the waterfall.

Check the watefall sound clip to hear the effect!

[theharsh-artofsound]

While travelling to my friend’s place today, I happened to come across something really funny. Yes, the announcement made by the train guard!

The announcement sounded very different from what you usually expect - a synthetic female voice, not just because it was a real person, but also the animated way of talking.

I also plan to use the sounds from this clip in my schizophonia project.

[theharsh-artofsound]

Penultimate recording

[theharsh-artofsound]

First recording