everywareproject

[everywareproject]

Presentation Link

https://prezi.com/view/cvW3ewB4za09OKP7Hktq/

[everywareproject]

Promotional Video

https://www.youtube.com/watch?v=xVB_mYe-LQw

[everywareproject]

The T-shirt Design

We may change this t-shirt design because of the issue we had with not being able to pre-programme the LED strips, meaning we had to try out different sorts of LED’s- we have finalised with an LED colour wheel, which is ace and can really show things occurring and can be programmed successfully. 

[everywareproject]

The APP

Initially the APP I designed was in PhoneGap, it was nice and easy to use to show people what we were thinking of as it is simple to set up a phone gap server and show its interactiveness. I used Bootstrap for the main theme to get the responsiveness and kept the README.txt file in the credentials section. I even put a simple GoogleMaps API in so that you could see fellow users potencially.

This worked really well, then I realised that PhoneGap cannot be linked with XAMPP or MySQL, meaning that I had to set up a SimpleHTTPServer and separate XAMPP Server for my Mysql. This took a while to get my head around but was successful in the end. 

Some print screens of it working are below:

Running on PhoneGap Mobile Server:

Running on PhoneGap Desktop:

The Backend (PHP and Mysql print screens)

[everywareproject]

Where we are

So far we have the t shirt connecting to the MQTT broker, the LED’s changing when anotherr user is within its vicinity. After our last presentation we were told that we should have two t-shirts for our final presentation to show it effectively working with two prototypes. We have a t-shirt design which was done by Gerrit, the LED’s which were done by Steph, the security and Geolocation that was done by Glenn and I designed the App, wrote the PHP and Mysql to that it could be saved and also connected in NODE-RED to Steph’s MQTT, so that data can be saved in forms and accessed in the T-shirt for each profile!

[everywareproject]

Backend

Security

In terms of security for our project we will plan on doing a number of things although we do acknowledge that not all things will be able to be completed it a short time frame but there are things that we have considered. Firstly we have to consider the security of the user database as it could have personal data in/ passwords. The way we will do this is by encrypting the sensitive data that we are inputting into the database. There are encryption API’s available for us to do this. This is useful because if the database gets hacked or leaked then the hacker won’t be able to understand the data without using the decryption key which they will not have available. We also have to consider the connection between the database and each of the user’s devices, for this we can use Secure Socket Layer (SSL) that encrypts the data. We will also make regular updates to the app to ensure that there is no security holes and to bring enhancements to our app.

Privacy and Preferences

For privacy we will be enable user preferences in the app so that the users can choose when they are being tracked but also what data is available to other users. Enabling a ghost mode for the tracking system is a good idea because it will make the user more comfortable about their location being shared, plus sometimes maybe they don’t want to get matches at an event or at a certain time so this gives the user the freedom of changing when and where they can get matches. Also as mentioned below we will not store the data of where the user has been or is, only use the real-time location if the app is open.

Ethics

We will not be tracking or storing data on where the user is or has been as we believe this is an invasion of privacy when the whole idea of our wearable and app is to connect more people together who they might not have met rather than to spy on people. We will also keep the data we ask the users very minimal as although we care about who uses our product we aren’t concerned about knowing all information about a user for them to use our service. We will make these points clear as soon as the user downloads our app so they feel confident and comfortable with what data they give to us to use our app.

[everywareproject]

Geolocation

[everywareproject]

How we see it working

Speech to text & Tone analysis built to work together

score/Tone ID 'simplified' and sent across MQTT to NodeMCU

Handling Multiples (in theory!)

Publish ID before sharing data across MQTT/Primary key for data

Data comparison stored online and handled server side

notification of match sent back via MQTT

built to be expandable to hundreds!

App/website:

input data, stored on server with primary key/ T-shirt 'ID'

[everywareproject]

The Next Step

Once the idea had been researched completely and the tutors were still happy for us to continue, we had to present what we had found and our initial concept

Above shows how the backend would work with Flowcharts, there is also a Trello Document- this is how we communicated throughout the module as its a brilliant platform for Group Work

[everywareproject]

Using IBM Watson

Speech to text:

Watson speech to text service is built to work with other services such as the Tone analyser.

The speech to text service can recognise and tell apart different speakers, which is useful when multiple wearers are together. The service by default includes keyword spotting and profanity filtering, which can be disabled if required. The speech to text service can be customized (beta acoustic model customisation).

The service takes input from either pre-recorded or live inputs; so you can speak to it much like you would speak to Google now or Siri.

The text is outputted as a text transcript in a JSON file, with highlighted recognised words if required.

The service is limited to 100 minutes per month on a student account.

 Source: https://www.ibm.com/watson/developercloud/speech-to-text/api/v1/

Tone Analysis:

The tone analyser service is built to work with the speech to text service as well as direct text inputs.

Unfortunately the tone analyser can’t be edited like the image recognition service, but this could be worked around.

The service takes input as a text file, which can be full sentences or not, and outputs both an ID for the tone and a percentage score. Both this ID and the percentage could be used to affect the colours used in the T-shirt’s response.

The service is limited to 2,500 api calls per month on a student account.

Source: https://www.ibm.com/watson/developercloud/tone-analyzer/api/v3/?python#post-tone

Watson general purpose tones:

The Watson tone analyser has a default output, this consists of both Emotional tone & Language tone outputs:

Anger (emotional tone)

Fear (emotional tone)

joy (emotional tone)

Analytical (Language Tone)

sadness (emotional tone)

confident (language tone)

tentative (Language tone)

[everywareproject]

Colour Changing Clothes Examples

https://www.theverge.com/2017/2/1/14476372/mit-research-wearable-app-detect-emotion-speech - Using artificial intelligence, the app can also figure out which part of the conversation was happy or sad, and tracks emotional changes in five-second intervals. Uses audio and psychological data

https://learn.adafruit.com/led-ampli-tie/overview - Colour changing tie, changes using conductive thread and a microphone amplifier, powered by a VBATT

http://www.dailymail.co.uk/femail/article-502889/The-dress-changes-colour-reflect-mood.html - Dress that changes colour depending on how you feel by monitoring body temperature as it changes depending on the mood you are in

https://www.newscientist.com/article/dn24232-colour-changing-clothes-could-make-tech-fashionable/ - “Lüme clothing can also change colours in time with the beat of music in a club. Bigger says the app could easily be tweaked to adjust the clothing colour according to the weather, or to sentiment in a wearer’s Twitter feed.”

[everywareproject]

Psychological and Semantic Linguistics

I found this part of the module really interesting, as I was researching the Psychological and Semantic Linguistics behind the scenes!

Determining the number and nature of the tonal contrasts is just one of many tasks undertaken in documenting and analyzing a language,

The service uses linguistic analysis and the correlation between the linguistic features of written text and emotional and language tones to develop scores for each of these tone dimensions.

For example, people exhibit various tones in their daily communications: joyful or sad, open or conservative, analytical or informal (Gou et al., 2014, and Jian et al., 2014). These tones can impact the perception of a person's online identity and the effectiveness of their communications in different contexts.

Research has shown a strong and statistically significant correlation between word choice and personality, emotions, attitudes, intrinsic needs, values, and thought processes.

When analyzing text, automatically detecting emotions such as joy, sadness, fear, anger, and surprise is useful for a number of purposes, including identifying blogs that express specific emotions towards the topic of interest, identifying what emotion a newspaper headline is trying to evoke, and devising automatic dialogue systems that respond appropriately to different emotional states of the user. Often different emotions are expressed through different words. For example, delightful and yummy indicate the emotion of joy, gloomy and cry are indicative of sadness, shout and boiling are indicative of anger, and so on. Therefore an emotion lexicon—a list of emotions and words that are indicative of each emotion—is likely to be useful in identifying emotions in text.

Emotional Lexicon- the idea of a list of a group of words meaning one emotion- so for sad, it would include unhappy, disappointed and gutted. This is something we will need to implement.

[everywareproject]

Meeting #4

Once this idea was generated we then split tasks between us to help with time organisation and most of all progression in the project!

We Split the tasks as follows:

Steph- Tone analyzer, watson, the settings behind it and how we could break it down, linguistics -

Glenn- Previous t shirt visuals/ideas concepts how its worked before, just the colour change reaction thing 

Henry- Psychological and semantic linguistics behind analyzing tones etc

Gerrit- General Design and how we could implement this

[everywareproject]

Meeting #3

And like that, the idea was born! As soon as we started talking about matching another user, getting some sort of reaction, all of us realised that this was the middle grounded-idea. Below is the 300 word Project Outline that we sent to Gianni so we could begin our work once we got the all-clear.

Personality Match t-shirt

The idea is to have a t-shirt that has a hidden profile to it. A wearer creates a profile, much like a facebook profile via an App we have created, with their likes, interests, hobbies, dislikes etc and they’re saved to that account. This then syncs to the user’s t-shirt, if the user comes into contact with another user who is also wearing a shirt and have the similar interests then the t-shirt will react and notify the person wearing the shirt that they have matched with someone close to them. Both shirts involved with the match will light up the same colour  so that each of the users involved know who they matched an interest with.

So for example if two people like dogs and they walk past each other then the t-shirt would start flashing to signal that those two people have the same interest. If no interests match then the t-shirt would remain idle.

To develop this further we thought about having it connect to other social media accounts a person may have, such as twitter. There is a lot of potential playing around with node-red and linking to twitter etc and posting/retrieving information.

A wearer could link their twitter profile to the t-shirt and it can be used as another signalling method by tweeting at the other wearer the same interest.

This idea could be pushed forward to be used by more than just having this idea of finding friends via t-shirts. Imagine if people in a workplace, or in a hospital, or in an old people’s home could all wear a t-shirt which would add interaction to people’s lives.

“17% of older people are in contact with family, friends and neighbours less than once a week and 11% are in contact less than once a month (Davidson and Rossall, 2014)”

--Purchase t shirt

--Create profile on simple interface online (website or web-based app) (free account with t shirt purchase)

--Update t shirt (wifi chip arduino)

--Wear t shirt

There could also be events where multiple wearers would get together in one place, such as meeting others in first year (e.g. Freshers fairs or events), conventions, open days and school induction days. This would enable people to talk and make new friends where they otherwise wouldn’t.

On the accompanying app you can see local events that are set up for people who also have the shirt, meaning that it adds a lot more value and that the shirt has multiple uses.

[everywareproject]

Meeting #2

The next step in the project was thinking about the environment that this could be applied in, the thought process went along the lines of :

-Hospitals? a smart hospital? With thermo-reacting gowns for example,  that could change with the users temperature or heart rate.

-Drumkit? A t shirt that could be interactive with other Users- a group of people would have to have some kind of interaction to play a piece of music or make a beat

-Proximity? GPS Sensor? glow when close? Interests could be linked into some kind of Profile

-You could get some king of notification, sound or visual when someone is in your vicinity? Beep Beep, and you've matched?

[everywareproject]

Meeting #1

Our idea initially was to create some kind of Interactive T-shirt, Using micro controllers to record certain fluxes around the User. For example local temp, wind temp, just general live data. We then thought about linking this to be displayed on LED’s using RGB Sheets etc. This then progressed onto using live data from Portland Square, using their wind data, importing in XML to Node-Red for example.

Following our initial brainstorming session above, we had a meeting with Chris and Stavros and they gave us a good idea on which direction to take- the social aspect of all of this. We wanted to have the t0shirt reacting to voices, audio around it. This then went onto the idea of a performance, with multiple t-shirts and some sort of reaction! The train of thought throughout this blog should be quite obvious, with each meeting our idea is progressing more and more. We thought abut having it to be individual or social- with constant sensing, to help with exploration. 

Most importantly from this meeting we took away what we wanted the t-shirt to mean to us. Is this going to be a brand, or is it going to mean something much more, different t-shirt for different people? Tailored to change in certain areas?

Above shows the initial thought process I had of linking the systems together in the back-end. Having an RTF stop words document so you couldn't swear into the tone analyser for example, and using DialogueFlow to get emotion percentages in peoples voices.

[everywareproject]

Everyware

This blog is to show the progression throughout my DAT 602 Module, from our first meeting as a Group to us Prototyping and Presenting in January