lhui003-blog

[lhui003-blog]

Sensor setup and explanation

Each pad of this instrument comprises of two sensors layered on top of each other.  This is to try and capture as much of a single point of gesture as possible.  For the FSR, I am using a DIY velostat sensor.  This is to cover a wider area than a store bought FSR sensor, and allows flexibility in setup and sensing properties.  I’ve also created a layer between the FSR top plate and the capacitive sensor to eliminate the interference the capacitive sensor was experiencing from the conductive material of the FSR.

[lhui003-blog]

“Borrowing”

[lhui003-blog]

Improvements to gestures

After testing the working prototype on my roomate, it became clear that the timings for the gestures were not intuitive and would often miss or misfire unintentionally.  This prompted me to rethink them and make them more user friendly and natural to execute; being one of them main points of Paddr.

1-One tap gestures have a shorter timing (since it’s only one tap), where the system has to detect a clear note on, followed by a note off within 50ms.  Within this amount of time is not detectable.  The scaling for sample position and duration have also been changed so that the sensor data to start point scaling is narrower than it is on the granular synth’s sample position.  Although this sounds unintuitive since the position of the sample is displaced in the same gesture, it makes the sampler more playable, allowing full reach to the very beginning and end of the sample, and in practice (since hitting a one shot is more aggressive than playing the granular so there is naturally a bigger skin contact), it makes it much more intuitive to repeat the same gesture twice (makes playing with intention easier).

2-Double tap gesture to sample incoming audio has been improved using an object called dblbang, designed for the exact purpose of detecting two consecutive bangs.  I have changed the time frame to register a dblbng to be shorter though, as I did not want two intended one shots to accidentally register as a double tap.

3-Both of these gestures now send a noteoff to the granular part.  This ensures a level of seperation from the gestures and their intended sonic outputs.  

[lhui003-blog]

Tweaked mapping on force sensing resistors so they kick in later and widened the range of capacitive sensors.  The cap sensors are very hard to map, especially when confined to whole numbers between 1-127.  Trying to make the best of this situation by finding a sweet spot between wide range and detailed mapping in the low range.  As shown here, the sampling and playback functions have also been improved for fluidity and use on the fly

[lhui003-blog]

Bela problem

After building a (very) rough prototype of Paddr, I left it on the side to dedicate more time towards the writing aspect.  After needing to check a few things on the prototype, Bela suddenly stopped working, and started throwing serious error messages, not even being able to start the IDE.  This was worrying as I had not touched it for a while.

Luckily, after contacting Giulio Moro at Bela he was kind enough to give me a new one.  I loaded up the Paddr patch (that I luckily backed up) and everything works fine

[lhui003-blog]

Wekinator

At the recommendation of my supervisor, I looked into Wekinator.  This is an open source software that allows people to use machine learning to allow people to “build new musical instruments, gestural game controllers, computer vision or computer listening systems, and more” without having to write code.  You do this by training the software on gestures, and demonstrating what the output of those gestures should be.  

This use of machine learning is very interesting, as shown the trained inputs are continuous, and as the green box is moved around, the area between the trained inputs are a blend between them.  

http://www.wekinator.org/videos/

Although not directly linked to the topics in which I am exploring, this will still come in helpful when writing.

- Mapping strategies

- Reference works currently in the field

- It is taking technology typically used at a higher level and giving it to musicians and others, allowing anyone to build very complex interfaces without writing code

[lhui003-blog]

Related products continued

Sunhouse’s Sensory percussion is particularly interesting as it uses machine learning to learn gestures that you train the software with.  These gestures can then be used in a huge variety of ways since the physical gesture has been converted into data (MIDI, CC, Macro, Ableton clip launch etc.).  It’s also an interface for drumkit, like an augmented instrument; drum trigger-like sensors that clip onto a regular drum.  

https://sunhou.se/

The aspect of these instruments I find most interesting are that they are breathing fresh life into traditional an outdated designs.  They aren’t trying to create an entirely new interface/instrument, but improving and adapting them to broaden their use in the modern day.  Repurposing or “sampling” old concepts for now

[lhui003-blog]

Related products

Padd’s position in relation to other products available in the market is very important; Paddr’s purpose is to sit somewhere between consumer instruments/interfaces and the new musical interfaces being born out of academic research out of grasp for most consumers.  I have been researching consumer level/mass produced musical interfaces, that offer an interface that is a contemporary and expressive version of a pre existing (mostly acoustic) instrument(s).

Roli Seaboard has been very popular, and I know people who own them.  It is a piano style keyboard, but it has pressure sensing capabilities instead of note on/offs.  It’s whole keyboard is also one continuous sensor/material, therefore it’s as if the physical keyboard is more of a guide as opposed to a limitation.

https://roli.com/products/seaboard

[lhui003-blog]

Augmented instruments

Since Paddr’s design and functionality is so clearly inspired by samplers such as MPC, I have been researching papers looking at augmented instruments: modifying or adding electronic technology to instruments to give them functionality in interacting with electronic music, whilst retaining their core functions.  I have found there are points that help convey the idea I’m trying to make with Paddr

https://books.google.co.uk/books?hl=en&lr=&id=CGEwXZ7hcIoC&oi=fnd&pg=PP1&dq=modifying+acoustic+instruments+electronic+augmentation+of+acoustic+instruments&ots=r-muZKdNYv&sig=BvXNitIKu9ENXTWJUjyB393Y2yg&redir_esc=y#v=onepage&q=modifying%20acoustic%20instruments%20electronic%20augmentation%20of%20acoustic%20instruments&f=false

[lhui003-blog]

Research for writing

After meeting with my supervisor for this project, I’m preparing for the writing and documentation of this project.  In an effort to help me explain the concepts I’m trying to explore, I have been researching concepts in mapping strategy and instrument design:

https://cnmat.berkeley.edu/sites/default/files/attachments/1997_Instrumental_Gestural_Mapping_Strategies.pdf

https://www.researchgate.net/publication/221052698_Perceived_physicality_in_audio-enhanced_force_input

http://www.music.mcgill.ca/~mwanderley/Trends/Trends_in_Gestural_Control_of_Music/DOS/P.HunKir.pdf

[lhui003-blog]

Paddr test footage.  Playing the instrument in a rough way to test the robustness of the gesture detection on each pad. Sampling Meitei’s “Nami”

[lhui003-blog]

Sample player functionality

This abstraction allows Paddr alternate functionality to act more like an MPC in use.  The instrument can now play chopped samples similar to an MPC by single tapping the pad with different amounts of skin contact.  The amount of skin contact in the gesture defines sample start and duration. 

[lhui003-blog]

Bela mini problems

I encountered a problem with Bela mini freezing and not running in a stable way.  Since the patch has become quite big (due to running many grains per voice), the CPU has been overloading.  To try and remedy this, most of the GUI elements have been stripped from the patch, and I have been focusing on making all of the functions as concise and efficient as possible.

[lhui003-blog]

Case design

Here is the cad file for the case.  This will be used to laser cut each panel of the case to be assembled.  I am still unsure as to the positioning of audio in and out/ power ports position on the case as I will need to see where they run when the components are inside.  These ports can be drilled later on.

[lhui003-blog]

Problems with sensors

After testing the positioning of the sensors I have encountered a problem where since the capacitive sensors work by generating an electromagnetic field, and the FSRs are constructed of velostat, the proximity of the FSRs are affecting the capacitive sensor readings.  I will need to use a non conductive barrier between the two sensors in the case.

[lhui003-blog]

Recording functionality

Here is the subpatch I have made to allow Paddr to sample audio into it’s buffers/tables to be used for the granular synthesis.  It requires you to double tap the pad you want to record into, then begins recording into the buffer from the audio input.  The delay is used to apply an envelope to the start and end of the recording just as a safety measure incase grains try to play across them.

[lhui003-blog]

Granular synthesis

A start of a granular voice on Paddr.  This takes a sample and plays very small slices of it.  I have each on a random metro, so that hen a lot of them are played in unison they create a smooth almost frozen sound.  I’m doing this for all the parameters so that each grain is slightly different, and making the random range a parameter that the user can control.  Users can change how focused or chaotic the synth sounds