haru0402dac-blog

[haru0402dac-blog]

Exhibition day!

[haru0402dac-blog]

Testing worked. 

Had to position Kinect at a better angle so it is not getting extreme perspective. I also took the webcam to the back so it is capturing whole body length. 

[haru0402dac-blog]

This is the animation that will appear when two arms are raised. 

[haru0402dac-blog]

Working with Kinect...

This is a process of working with Kinect. I am checking all the values sent from Kinect so that I could use the values to create a primitive shape appear when one arm is raised which will move up and down and when both arms are raised the primitive shape will move between the arm. 

Also, I am checking the accelerometer value so when a person moves faster the outline of the colour changes to orange, 

[haru0402dac-blog]

This will be the set up for the exhibition. 

[haru0402dac-blog]

OSC

I managed to make 2 computers communicate with each other by using OSC messages. This will ensure to use send Kinect data to the windows PC with OpenCV buffer webcam image. 

Also, my most worry that was scaling the webcam image to 64x32 worked perfectly. At first, it didn't scale properly so the image was zoomed however, I solved the problem by using ofScale 640/64 and 480/32. I also increased the framerate as it was lagging too much due to the smaller window size. 

[haru0402dac-blog]

update

Now that openCV code is working, I will be using Kinect to pick up the average depth of position, depth value or proximity. This will change how much echo image or colour being displayed. 

There will be 2 inputs 1 for LED visual and 2 if for the user interaction, changing the visual parameters depending on the proximity of the person. 

I will openFramework code that displays the user’s silhouette using openCV, captured by a PS3 camera. Kinect code has 0-1 range. 0 being no detection of movements and 1 being movement found. I am still not sure whether this will all work on Windows 10 laptop. If it doesn’t work I will have windows and a Mac which will have OSC to work across the network

[haru0402dac-blog]

This is motion detection using OpenCV example. 

Changes colour as set by the timer. 

Image blur can be changed if It’s too much. 

[haru0402dac-blog]

I managed to send webcam data to the LEDShow T9 software which sends that data to the LED curtain itself. Although it worked, I will have to adjust the camera angle and how far away it has to be from the user as it has not yet captured the whole body. So far, it only captures the face. 

Also, I will have to work on how to create the webcam more interesting by adding some sort of effects, for example, optical flow... 

[haru0402dac-blog]

Screen capture

Now that the software is running properly, I envisioned how to send webcam data to the software. I figured the software provides screen capture mode ‘Extern video window’ it screen grabs whatever image/video that is playing on the other platform eg: processing sketch. 

My next step is to program with webcam data on 64x32 window size. Trying to put a webcam movement on 64x32 will be very difficult but I will try and scale it to fit in the window. 

[haru0402dac-blog]

LEDs working

After figuring out what was causing the issue, I explored and tried out every combination from every aspect for the LEDs to work for 4 days. It turns out the pixel width must be 48 by height 32. And X must be 612,  Y 94. 

[haru0402dac-blog]

Wednesday 14th March

Pete and I have found out what was causing the issue for the LED curtains. The software’s default setting on the number of LEDs was set to the width 256 and height 256. The real number should have been width 48 and height 32. This default setting confused the control box since it thinks there are 256 LEDs so, it only lights up at random position. 

We also found out that if we fiddle input and output pin din, it creates different patterns and colours, so we assumed we can definitely control the lights by connecting it to an Arduino. 

[haru0402dac-blog]

update on what’s happening so far...

There have been many issues with using Eurolite LED curtain. Firstly, the software to control LED lights is not compatible with Mac. Therefore, I had to Boot camp to install the software. However, the software did not run properly, therefore, I got to lend Windows 8 from the department. The software worked for about an hour but it suddenly kept crashing. 

So, I had a talk with Pete if there’s an alternative and he suggested that we can get around software bugs by simply making the laptop a Windows 98/vista. OR  he assumes that the LED controlling box is 5V so I could control it from Arduino and manually control the LED lights rather than using a outdated software. 

[haru0402dac-blog]

My initial idea was to use either LED strips or LED matrix panel, however, they both cost a fortune especially when the project involves visualising human body length. However, Pete found this great black curtain with LEDs installed which can be hung from the ceiling as well. 

[haru0402dac-blog]

I have created an audio sensor that detects sound which the LEDs flashes according to the beat or sound. The aim was to investigate how accurate can LED flash associating with a sensor. 

[haru0402dac-blog]

My project involves LED lights, therefore, I have created a digital simulator on Processing to check the pixel’s quality using the webcam. I initially created 200x200 pixels, however, it did not work well with the Webcam hence, I used a bigger grid to improve the visualisation. 

[haru0402dac-blog]

First Prototype

A simple processing sketch that allows interaction using Arduino photocell sensor. Using machine learning to send inputs and outputs I have used Wekinator as the model. The signal power of open sound control sensor data is digitalised then sends in OSC format to the machine learning Algorithm. For the algorithm, I have used Dynamic Time Warping to train gesture outputs which then Arduino captures the gesture that was trained from Wekinator to processing sketch to interact.