pebblecode - Tumblr blog

[pebblecode]

MAY LIGHTNING TALK

In May’s lightning talks, our designer Ian Black introduced us to Sow & Grow, a web app for folks looking to grow their own vegetables and have no idea where to start.  

He shares his journey on trying to grow vegetables whilst learning javascript.

pebble’s co-founder Toby Hunt introduces us to the concept of Kaizen the Japanese word for “change for better”, a system of “continuous improvement” and how we are using this beyond software development and improving everything from company culture, productivity, safety and leadership.

[pebblecode]

Hack Day - Hackball

What was problem you originally pitched?

In January 1815, Napolean and the Duke of Wellington met to settle their differences over a game of table football. It was a long and heated match with neither side having a clear upper hand. After a few hours of terrific shots and fantastique saves, an argument broke out: The Duke thought he had just scored his 10th and final goal, whereas Napoleon said it was the 9th and the Duke had one more to go. With no way of keeping track of the score, the fight turned ugly and 6 months later gave rise to the battle of Waterloo.

We’ll never know what the actual score was on that fateful day, but we aimed to prevent disagreements like this from happening again. So we set out to augment the office table football table; a device that’s very good at hitting balls but very bad at tracking scores and playing sound effects. We wanted to build something that would capture and log goals, and add a couple of nice extras to make the office kitchen feel like Wembley

What did you build?

Hackball! We made a server with a fancy front end (straight out of Sensible Soccer) that could be accessed by any device on the network. You could log in and start a game, and view the current score in a browser.

When goals were scored, you pressed some sweet arcade style buttons and the server updated the score in real time, as well as playing a couple of ‘goooaaallll!’ sound effects.

What tech / magic / sorcery did you use to make this awesomeness?

We had a node.js server running on the Raspberry pi, listening to button presses on the Raspberry pi GPIO ports, using the node GPIO library, and a custom extension named ‘Pinhead’. The server displayed the scores over the network, which was updated with Socket.io whenever a goal happened. Sounds were triggered both through the raspberry pi’s audio socket, and on the front end.

What were the big gotcha’s?

We ran into a couple of issues getting node to read the (analogue) sensors that were intending to register goals, so we had to emulate that by pressing buttons instead.

The rpi-gpio package was quite limited in detecting changes in the pins (which happens when a goal was scored). We ended up putting together a new npm package which allows users to easily monitor any raspberry pi pins for changes: https://www.npmjs.com/package/pinhead

What’s next?

Well we think we’ve solved reading the analog sensors on the pi, so hopefully there’ll be no need to log the goals manually with buttons anymore. A couple of other next steps are: • Log players scores and give a global league of top players in the company • Integrate it with Slack, our internal messaging system • Add a matchmaking option for solo players wanting to find someone to play against • Add a option to change ‘goal’ voice - Alan Partridge, Jon Motson

[pebblecode]

On average, the world is happy.

How far would you agree with the above statement? If you could visualise the emotional state of the world, what would it looks like?

I never used to find myself wondering the answers to these questions, but thanks to Art Hackathon 2015, I can now show you the answer.

Or, at least, a generalised, social-media-centric version of the answer. So let me take you on an emotional journey.

DISGUST

As the alarm bells rang by my bedside my mind was filled with loathing. This is not a sound one should hear on a weekend before 8am. But I clambered out and prepared myself for my first hack day outside of work. 50 artists and tech lovers were due to gather in North Greenwich for a weekend of collaboration and creation, and I sure as hell wasn't going to be late.

The crowds mingled over coffee before being ushered in to the workspace in Ravensbourne College. A brief welcome was followed by a set of talks on art and technology (such as Human Harp - below). Having been fed on a morning of creative ideas we got ready to being hacking.

Those with ideas to pitch lined up in front of the audience. After presenting, the crowd was left to gather in to teams for the weekend. I waited in line for my turn to spew out some muddled ideas about visualising the emotional state of twitter, and tracking its effect on a viewers. It wasn't my finest presentation, but tied in neatly with another designer's dream of creating an illuminated globe displaying human emotion. We banded together, along side a team of hardware fans and fashion designers. We were good to go.

A quick brainstorm and some sketching helped us pin down our concept. Psychological research has classified six (or 4) universally recognised emotions: happiness, sadness, disgust, surprise, fear and anger. However the team felt we should represent five of them. That's democracy for you, so out went disgust. Our idea was to get emotional data from twitter through their search widgets, and run this through a Raspberry Pi to power a strip of LED lights. The lights would be fashioned on to a perspex sphere to form the globe.

ANGER

Parkinson's law states that "work expands so as to fill the time available for its completion". So naturally, with 2 days to build this, my first few hours were not productive at all.

As I struggled to work out the javascript required to access the data from the twitter, I could feel infuriation creeping through my mind. Meanwhile, the hardware team were having issues of their own, in getting consistent results with our lights.

SURPRISE

However, with perseverance comes rewards, and it was with some astonishment that by the end of day one we had the bare bones of our project coming alive. We had the beginnings of a model for our globe, and a website showing live analysis of tweets from around the Twittersphere.

FEAR

As day two commenced, with the hardware issues looming over us from day one, doubt crept in over our ability to build our dream piece on schedule. Coupled with a limited supply of perspex, it was time to start looking for alternative solutions. We sketched out some designs utilising the refractive properties of what little acrylic we did have. By laying sheets vertically on a computer screen, the two dimensional web page could be brought to life as physical sculpture. The colours would then mutated and refract through the shards of material.

The acrylic was laser cut in to segments that came together as a ripple-like sculptural piece. Each slice sat within a careful engineered slot carved out of a sheet of plywood. This had the effect of blocking out any light not passing through the plastic.

HAPPINESS

As the room cleared for the exhibition and the final gong was called, we frantically pieced together our creation. In the last moments we haphazardly threw in sound effects that transitioned with the visuals. It was with sense of relief that we set down our tools and sat back to admire our installation. Emoti was born - An intimate, person to people piece, allowing a single viewer to immerse themselves in the emotional landscape of Twitter.

Whislt we were satisfied with our creation, the rest of the room blew us away. Digital magic, musical fruit salads, face-tracking forests and 3D museum pieces filled both physical and digital spaces.

SADNESS

But all good things must come to an end. With regret we dismantled our pieces, said our thank yous and goodbyes, and departed. The installations may be gone, but thankfully the internet is still going strong and you can check out the Emoti visualisation here, (or view the source code on Github.)

[pebblecode]

Hack Day - WFH Bot

What was problem you originally pitched?

It is hard for pebble staff to know where people are working from on each day. Some staff are in the office for some of the day and others work remotely for most of the time. We currently use a WFH email chain to let the company know if you are working from home but with the bigger number of people working at pebble this is getting less scalable.

What did you build

We created five streams of work.

The first was a RESTful API to allow different clients to update the status of a pebble employee.

The second was a Slack bot that would allow staff to update their status by using ‘wfh’ for working from home and ‘wfo’ for working from the office. An integration was created for TribeHR our HR system to pull in sickness and holidays and set the status correctly for that.

An email daemon was also created to send a single mail at 10 am to all of the company.

Finally a front end was created to allow the status of people to be shown on a big screen.

What tech / magic / sorcery did you use to make this awesomeness?

The most important part of the day for the team was creating the RESTful API. This allowed the work to be quickly divided up. The API was created using C# but the fact that the language was separated from the interface meant that other members of the team were free to use whatever they wanted to update it.

An experiment was made with Rust but it became apparent that this would take too long. The connector to TribeHR was created using Node.js as was the email cron.

The biggest strength of the day was that we had a multi-disciplinary team using different technologies working together because of the single API interface.

What were the big gotcha’s?

The team saw some friction between using different technology stacks. Once the RESTful API was bootstrapped we looked to make the updates real-time and the API team implemented SignalR to offer this. We then ran into problems with CORS in the browser and after an hour or so of tweaking settings we were unable to solve issues with headers.

What’s next?

The product is working in pebble now with many members of staff using it to up their status each day. The email job is working well and has been setup as a cron job on one of our servers. This bot had a brief malfunction when initially introduced thinking it was ok to send an update every minute. After a dressing down it is now working well.

To finish the project the team need to get the front-end hosted somewhere and for this to be displayed on a big screen in the office. We also need to resolve the SignalR issues either by using a proxy server to remove the CORS issues or to be able to set the headers correctly.

We also may look at creating a slack channel or a slack command that will list the status of staff since the company is gravitating heavily around Slack now.

[pebblecode]

APRIL LIGHTNING TALKS

In Aprils lightning talk, we learnt how to create dynamic animations using tools such as processing.js and open frameworks with Niall Henn.

George Ornbo gave us a high level introduction to using Docker as part of the development process for continuous deployment and integration.

[pebblecode]

Putting Pen to Paper

It’s another early morning in London. A select few members of pebble {code} are gathering on a shady corner in Islington - a far cry from our usual south London home. Some of us clasp at hot cups of coffee. Others stand buzzing from the adrenaline of cycling round the city’s most notoriously dangerous roundabout. But spirits are high all round at the thought of starting our first Design Thinking workshop on patient engagement, with one of the UK’s largest charities. Armed with our wits, stacks of paper and a large stash of 80’s neon Sharpie pens, we head inside to set up.

As 9:00 rolls around, the group from the charity pours through the door, armed with healthy snacks for the day. Moments later, hot beverages arrive, accompanied by a large silver platter of biscuits. I watched closely as willpower crumbled to dust and was swept away in the breeze. The apples and oranges barely stood a chance.

Let’s Begin.

The group was split in to four teams. We planned to work through a series of exercises throughout the day, with each team assisted by a Pebbler. Each activity would lead in to the next, developing, refining and challenging ideas throughout the morning, with a strong focus on drawing as a technique for documenting ideas. This was not about being an artist, but encouraging people to be explicit and specific about their ideas and decisions.

But first things first, and in the words of John Mildinhall, (our Creative Director) “What do we do before exercise? We warm up!” Luckily for those who forgot their gym kit, it was creative juices we wanted flowing, rather than blood pumping. No star jumps here thankfully. (Although obviously pebble {code} encourages a healthy lifestyle!)

Warm up part one asked everyone to create their Post-It Social Media Profile. A quick selfie sketch and a written bio functioned as an icebreaker between groups, while also ensuring everyone got in to the practice of sketching, writing and sharing ideas.

Warm up part 2 was drawing a tattoo. (No, not a real one.) The aim here was to get people to be a little more creative with their sketching, whilst also demonstrating what the charity meant to them as an employee.

Once everyone had unleashed his or her inner Michelangelo it was time for the first task.

Draw The Problem.

Whilst everyone was here for a common purpose, but being from different walks of life and sectors of the charity, it was likely they all saw things from slightly alternate viewpoints. Through discussion and sketching they were asked to visualise what they saw as the key concerns for the problem they were trying to solve. Individuals could work alone at first to describe problem through their eyes, but groups had to create a single vision by the end of the allotted time. The list of characters involved in each drawn process would lead us directly in to exercise number two.

Empathy Mapping.

By mapping out the problem the group had indirectly created a list of people who were involved or affected by the problem. Before a solution can be pieced together, you need to know your target audience. The room had amassed a broad list of individuals, from friends and co-workers to pharmacists and teachers, so it was clear that the groups were engaging with the tasks well. We assigned each group 5 people. They were asked to summarise 6 key actions for each persona: Hearing, Thinking, Seeing, Feeling, Saying and Doing. Once complete, each group could hope to have a much clearer idea of the problem they were looking to solve, and the roles that each person played in achieving the solution. But all this Monday morning thinking can be exhausting, which meant it was high time we stopped for refreshments.

A Quick Break.

We basked in the mid morning sun for 10 beautiful minutes. Spring had truly arrived. As the golden rays bounced off London’s rooftops I ... sorry ... time’s up ... we’ve got real work to be getting on with!

Put It To The Test.

With the teams reassembled, they were asked to pick out one specific issue from the morning’s session to think up (and sketch down) a solution to. When they thought they were done each team would be dealt a card. On this card would be a constraint. A constraint card might say “it ships today”, “keep it simple” or “take it outside”. The solution then had to be redrawn according to this restriction.

Once done, another card was dealt... And then another... And then another... Four were played in total, forcing the teams to tweak, refine, flip and distil their thoughts until they could draw no more.

Lessons Learned.

As coloured pens fell from cramping hands, we called the session to an end. A final presentation of ideas from each group rounded the day off. Perhaps we have taken the first steps down a path to a better future of charity care. But, if nothing else, we hope to have opened up a few people’s minds to the merits of sketching your ideas, because, in the words of David Hockney:

“Drawing makes you see things clearer, and clearer and clearer still, until your eyes ache.”

[pebblecode]

March Lightning Talks

Check out March’s Lightning talks, we had an intriguing talk by Vince on forthcoming ECMAScript6, and a very useful talk by our new face; Jonny on using powerpoint to create engaging slide presentations.

[00:38] Vince Martinez - Using ES6 Today

Some points and advice on using ECMAScript6 today

[15:35] Jonny Li - Powerpoint doesn't suck, you do

Quick run-down on how you can make more engaging slide presentations.

[pebblecode]

February Lightning Talks

Feburary's lightning talks were the usual eclectic mix and we were able to record them for the first time. Thanks to Mark, Cheng and Graham for their talks. Lapel microphones are on their way for next month!

[00:23] Mark Durrant - XYZ Machines

Experimenting with plotters and 3D printing.

[12:48] Cheng Cui - Chinese Mobile Internet

A tour of the Chinese Mobile Internet Market

[27:00] Graham Gilbert - Sal & Sal +

Showcasing pebble.it's Munki and Puppet based Mac Management Product

[pebblecode]

Angry wirds - Pebble Hackday Project

Demo Link: http://pebblecode.com/angry-wirds/ Github Link: https://github.com/pebblecode/angry-wirds For my first Pebble Hackday, I thought I'd try and see what unique skills I could bring to the table; a hard job given that Pebble has the cream-of-the-crop in developers and designers. With my background in motion design and creative coding, I thought I'd try and focus just on a technique for Visualising rather than the Visualisation itself.

Thinking about how I could visualise data in a new (and more importantly, beautiful) way, I moved to thinking about text and generative typography.

While text is arguably the most direct method to get a message across (whether it's data or not), the inherent need for readability means that visually the letters can often be a little flat and one dimensional. Not always conveying the emotional depth of the content.

As web typography comes of age, maybe it's worth asking if we can create type experiments along the line of guys like David Carson, and how dynamic media can bring new approaches.

I decided to create letterforms that have this extra dimension, reflecting the emotional content of the prose. I wanted to make angry words looks visually 'angry', and likewise for friendly words.

To give a usable example; A block of data depicting customer reviews could display particularly negative reviews as sharp angular letters, making the rage of the customer a lot more apparent.

Animated typography already has a history (Websites like animography do this very well), but playing back animations or displaying images would be too easy (this was a hackday after all, a time for experimenting). I decided to to generate the letters programatically, and I wanted to use the day to get better acquainted with paper.js, a javascript library for vector drawing that I'd only skimmed over previously.

A few sketches later, I had decided to start with the basics; A simple letter 'A', and designing the binary emotions of happiness and anger, imagining gentle floating outline for the former and sharp, jarring outline for the latter. I broke the letter down into a series of points, where I could place shapes at each point. The "happy" letter would have circles that gently pulsed giving a cloud-like appearance, The "Angry" letter would have sharp rectangles that vibrated randomly giving a sharp, scratchy look. I imagined a sliding scale between these two states, with other behaviours available at a later date (Sad words could droop, scared words could shake). If the letter was distilled into points, then their behaviour could take many forms.

After building the 'happy A' shape out of circles, then making the 'angry' shape out of rectangles, I focussed on how to move between them and hit an unexpected stumbling block. I imagined paperJS would have an easy function to switch between rectangles and circles, or to allow me to 'round off' a rectangle to look like a circle. Unfortunately it didn't.. There was only a one-way process to 'smooth' rectangles to round their corners. Running a little short of time, I elected to stick to my original intention of creating something beautiful, and focussed on the A and B, finessing them rather than focus on the whole alphabet.

All in all, I like what I made. The letters look nice, and I feel I've made a good starting point to build on for further experiments. I'm hoping to get some time going forward to re-visit this experiment and accomplish what I originally intended. The eventual aim would be to give the app a string ("Welcome back" / "Go away!") and it be displayed accordingly, with the sentiment analysis either happening in the app or externally. Looking forward to more Pebble Hackdays! P.S a hat tip must go to Jesse Zwaan for the name of the project.

[pebblecode]

Book Review: The Second Machine Age

In The Second Machine Age Erik Brynjolfsson and Andrew McAfee eloquently document many of the changes that increased Artificial Intelligence is bringing. They go on to make a series of policy recommendations that, whether you agree with them or not, will educate you further about the economic and social changes that increased Artificial Intelligence will bring.

Artificial Intelligence - the present and future

The Second Machine Age begins by documenting many of the changes that are happening to economics and society as a result of increased automation. Through examples of computers beating humans at Jeopardy!, unbeatable chess computers and cars that can drive themselves Brynjolfsson and McAfee demonstrate that the world is changing and will change radically over the years to come. The book is ambitious in that it initially seeks to document the impact of automation mostly through third-party research, theorise about the future impact of Artificial Intelligence and then make policy recommendations for legislators.

The book draws on a large body of academic research and cites several studies around the impact of Artificial Intelligence. It is not exploring what is possible, but rather showing (and sometimes in wonder) what AI is capable of. This includes boyish excitement of being driven around in a car with no driver or celebrating that Watson can beat humans at Jeopardy! every time. There is some reflection and analysis given to the social and economic impact of AI during these examples but predemoninantly the initial chapters of the book can be summarised by "this is what's happening, it is amazing, and we'll need to work out how to deal with it".

Productivity Challenged

Historically GDP has measured a country's productivity and to a large extent continues to do. GDP is inherently labour based and the idea that a nation's standard of living equates to GDP output still governs much of how the world is run. The book points out that because of digitisation and automation this is becoming increasingly less relevant and it is no longer the best way to measure growth.

Production in the second machine age depends less on physical equipment and structures and more on the four categories of intangible assets: intellectual property, organizational capital, user-generated content and human capital.

The authors suggest using consumer surplus to determine growth because it offers a better representation of improvements in quality of life as well as monetary growth. Many new technologies are free to users, such as Google, Wikipedia, and many digital technologies which drastically improve quality of life but are not accurately reflected in GDP numbers. This was a really interesting section of the book and effectively challenges many of the ways that we measure productivity and ultimately human happiness.

Reflection on Wealth

The authors reflect that far from the vision of a distributed, egalitarian platform the web has increased the gap between rich and poor. The cause of this is suggested to be a result of both digitisation and globalisation. The book notes that the top 1% of earners in the USA have seen large increases to their income whilst the lowest earners have seen their earnings decrease. The authors cite the example of digital music where more music is being listened to more than ever but artists have seeen a dramatic drop in revenue income. While it now costs almost nothing for distributors to sell digital music the music industry has had to accept that the new distributors like Spotify and Google Music will take a greater cut of the overall revenue pie and that record labels and artists will get less.

The authors describe this as bounty and spread, where distributors or technology providers command a bounty for having understood how to solve a particular problem but relatively everyone else's economic position in the problem will decrease.

Human Jobs will be automated

The book points out the inevitable truth that advances in AI will result in job losses. The example given in the book is of professionals that prepare tax returns. Software is already largely automating this task and it is cheaper and more efficient for the buyers of this service to use software services.

If you can give precise instructions to someone else on exactly what needs to be done, you can often write a precise computer program to do the same task. In other words offshoring is often only a way station on the road to automation.

In a largely utopian view the book's authors urge legislators to let AI and net effects like this to happen. The underlying message of the book encouraging humanity to run with machine rather than to fear it.

The book seems to position itself in a space that welcomes the advances that AI can bring to humanity but is acutely aware of the need to create regulatory and legislative environments for it to flourish. Indeed there are two entire chapters devoted to Policy Recommendations and Long-Term Recommendations. These recommendations support the underlying conclusion of the book that humans should focus on skills that compliment machines. They should not seek to fight automation and moreover the authors are hugely optimistic about what increased AI can bring. The most telling example in the book is of human chess players being able to beat a chess supercomputer with the support of an average chess computer. The authors see that combining the human consciousness with Artificial Intelligence can propel intelligence beyond what can be achieved without it. Far from running from machines the book recommends we should embrace Artificial Intelligence and recognise it as a compliment to our own human abilities.

Whether you agree with the liberal message to encouraging Artificial Intelligence and letting it run its course the book raises many issues that Technologists, Academics and Legislators should be considering. This book is not the only book to do this but if you are looking for a book to challenge your social and economic assumptions on Artificial Intelligence the book achieves well.

[pebblecode]

First hack for Mr Black.

It’s 9am on the last Friday of the month and all I can think about is hard liquor. Scottish Single Malt Whisky to be precise. Smokey, delicate, rich or light - today I’m just dying to get stuck in.

But don’t worry, I wont be knocking back any drams just yet. In fact I don’t plan on drinking it at all. All I care about this morning is information. It’s the first pebble {code} hackathon of the year and the theme is Data Visualisation. My stomach is full of pastry, and eight hours are put on the clock. We have one day to try and create something new with any dataset of our choosing. Armed with nothing but my mac and the infinite knowledge of the internet, I set to work.

My plan

To create an interactive version of the whisky flavour map that links each whisky to a map of distillery locations.

My tools

Html, css, D3.js (a JavaScript library for manipulating documents based on data) and Leaflet.js (a open-source JavaScript library for interactive maps).

The hack

Things got off to a good start when a fellow Pebbler revealed he already had a large database of whisky names, flavours and location data. Progress rapidly halted when the location data turned out to be an obscure grid referencing system I wasn’t sure how to work with. Keen to get coding rather than resolving the finer points of coordinate types, I opted for a manual approach to starting my data set. I selected 5 whiskeys from the map, searched their locations online and quickly shoved the information in to a CSV file.

I fleshed out a visual design in Illustrator. I started with a colour scheme, selected a typeface from Google Fonts and drew up a chart. A went for a simple single-page design, with flavour profiles on the left and the map on the right, knowing time was not on my side.

I extracted the SVG content and used this to map the flavour data. The right hand side would bring in a raster map of Scotland using leaflet. Using D3 I could geo-locate each distillery. I had to load the data in to each side separately. I had never tried to link two graphs before, and my JS knowledge is quite limited, so the greatest challenge was working out how I could read information from one side, and then search the other for a matching pair. By attaching id’s to each point on the data load I eventually managed to solve this, while brushing off the cobwebs from my knowledge of writing functions with variables in JS.

With a working prototype, I refined the page design some more, added a pile of extra data to my source file, and included an introductory page. I was then able to start adding finer details like transitions and colour changes to enhance the experience. I was just beginning to add in new variables, like whisky age, when the clock stopped.

Showtime 

I stepped up to the challenge of presenting first, hoping my extreme daring would reap rewards in the final voting. Luckily my design worked without a hitch first time. I was feeling confident, but after seeing what the rest of the contenders had produced, I knew any hopes of a win were pretty slim.

A leaderboard went up and people gathered round to cast their votes. The Whisky map was looking popular. I could see it was going to be a tight race. As the votes were counted I held my breath…

…but it’s the taking part that counts right?

(You can see the source coded here on githup)

[pebblecode]

Off Piste Baby Names

Off Piste Baby Names is a hack that I created for the pebble {code} data visualisation hackday on 30 January 2015. The ONS baby names dataset has been something that I have been aware of for some time. I had noticed some pretty unusual baby names in the data before, and I decided that I'd like to do something that exposes some of the themes that people seem to follow when they go "off piste", in other words when they select names for their children that are not considered to be traditional. 

Since I had the raw baby names data, it was up to me to generate the themes. I went through the 10,000 or so names for both boys and girls that were given at least 3 times to babies in 2013, extracting names that I thought were unusual, funny or note-worthy in some other way. There were no strict criteria for this, other than my own personal reaction to the names. These names seemed to cluster together in specific ways, so I started applying categories to these names. For example

Autumn-Willow  Summer-Rose  Winter-Rose

All combine a season with some sort of plant, so the theme "season/vegetation" category was born. Similarly, the names 

Isis  Lolita Berk Nadir Nimrod

all have negative connotations, and resulted in the creation of the category "doomed". Names can belong to more than one category, creating a complex graph between the names in the data set.

My initial intentions, since this was a data visualisation hack, was to create an interactive D3.js visualisation of the data, along the lines of the graph above. Unfortunately I ran out of time to get to grips with D3 before the day, so I limited myself to creating a way of navigating through the network on the basis of categories. 

I formatted the data as json, with three different data sets - Names, Categories, and Edges (specifying the relationship between the categories). Wrangling the data turned out to be rather easy - a set of functions that pulled out baby names on the basis of a category, and categories on the basis of baby names. I would probably have re-factored the code to separate the data layer from DOM manipulation more, but the constraints of a hack day were too much for this.

//Name { 'Index': '190', 'Name': 'JACK-DANIEL ', 'sex': 'boy', 'n': '3', 'rank': '4685' } //Category { "Number": "31", "Category": "booze" } //Edge { "number": "457", "name": "JACK-DANIEL ", "category": "booze" }

I'm hoping this will be the last hack day where I use jQuery. Not being a proper developer, I feel that it makes me more liable to write bad code, do things that just feel wrong, and it is hard to structure it sensibly and keep clean. I also really hope I have written my last $('.class').clone() function. This method of populating the DOM with your data is singularly loathsome, and nearly always results in hours of wrist-slitting debuggery. Maybe I just need to be a better programmer. Or perhaps angular.js is the answer.

[pebblecode]

The Periodic Table of Colours

The Periodic Table of Colours is a simple visualisation of all the colours present in the current HTML extended color keywords spec. I created it at pebble {code}’s recent data visualisation hack day.

I had originally planed to visualise an entire colour space such as RGB, HSL, or LAB. However these colour spaces have been expertly visualised several times before. For example Tauba Auerbach’s RGB Color Space Atlas.

The extended colour keywords spec is a list of all the named colours that can be used in HTML & CSS. I found the extra dimension of colour names to be an interesting twist on the standard mathematical models of colour.

Looking at all the colour name and hex code variations I was reminded of the periodic table. The combination of names and hex codes reminded me of the names and symbols for each of the elements, and the distinct colours reminded me of the periodic tables group.

After turning the 35 colour names and hex values from the spec into JSON I used TinyColor.js to get the RGB & HSL (hue, saturation, & lightness) values for each colour. This gave me analogue for the atomic number and mass for my colour table.

Next I spent a large part of the day learning about and using the new viewport based vw & vh CSS units. For a great overview of the vw & vh units checkout this article by Chris Coyier on css-tricks.com.

Chris talks in depth about using the new units for simple responsive typography. However I found another compelling use-case. Using viewport based units is a great way to create responsive shapes with a fixed ratio by using either vw or vh units for both the width and height of a shape.

In the The Periodic Table of Colours I’m using vw units to create responsive squares but the same technique could be used to create responsive shapes of any ratio. For example…

/* create a responsive square over half the viewport */.square { width: 50vw; height: 50vw } /* create a responsive rectangle using the golden ratio */ .gilded-rectangle { width: 16.18vh; height: 10vh; }

Using these new viewport based CSS units made it much easier to create the periodic table inspired design I was aiming for and make it fully responsive.

After finalising the design I used the great Isotype library by David DeSandro to give some basic sorting based on the name, hue, saturation, and lightness of each colour.

Whilst this wasn’t the most technically challenging hack I’ve ever undertaken it was great to work on a more design-focused project.

This just goes to show that hackathons aren’t just about technology. We can also use the hackathon structure to rapidly solve problems with design and produce high quality visual work.

[pebblecode]

The present and the future of Data Visualisation

Attractive versus Informative

"Beauty is truth, truth beauty," – that is all Ye know on earth, and all ye need to know.

John Keats. Ode on a Grecian Urn

Data visualisation stems from two broad traditions. The first tradition is academic, where economists, scientists and statisticians have sought to display complex findings and ideas in a succinct and easily digestible manner for an audience of academic peers. The second tradition is journalistic, where communicators initially sought additional colour and insight to add to their articles and arguments for an audience of the general public. Where these two differ significantly in their approach is in the focus of their efforts: Academics have valued minimalism, brevity and standardisation. Journalists have above all valued attention, but also clarity, impact and memorability. 

1. An example of academic visualisation - the Remote Distractor Effect (Walker, Deubel, Schneider & Findlay, 1997).

2. An example of journalistic visualisation - Bumps, Bruises and Breaks in the NFL. At a glance the viewer can determine that knees are most commonly injured in the NFL (Wall Street Journal).

There are two titans of academic visualisation, the statisticians John Tukey and Edward Tufte. Tukey advocated visualisation as a method for exploring data, rather than merely supporting or rejecting hypotheses. Among his inventions are the boxplot, and the use of the term 'bit' in computing. Tufte's seminal book Quantitative Data Visualisation (1983) emphasised absolute efficiency in visualising data, arguing that a minimal use of 'data-ink' is self-evidently beneficial.

3. A Tukey Boxplot showing the 1st, 2nd and 3rd quartiles, the 2nd and 98th percentiles, as well as outliers in the Michelson-Moreley experiment.

4. A Tukey visualisation displaying relative changes in magnitude and order between two dates. All extraneous information has been removed, leaving only 'data ink'.

Despite the many suggestions of novel and improved graphical representations of scientific, engineering and economic data, few of these have actually been taken up by the academic community. The inherent conservatism of scientific journals have upheld highly traditional visualisations that have not taken advantage of modern technology and that are essentially still stuck in the analogue age. 

The history of journalistic visualisation is a long one; The oldest examples date to the 18th and 19th centuries - such as Charles Joseph Minard's infographic of Napoleon's disastrous Russian campaign (see figure 6), the timelines of Joseph Priestley (discoverer of Oxygen and inventor of Soda Water) and the line and bar charts of William Playfair. More recent work such as Megan Jaegerman's work for the New York Times demonstrate how powerful journalistic visualisation can be for communicating complex issues.

The journalistic tradition was populist - some of the earliest published works of visualisation became best sellers. The producers vied with each other to become more and more bombastic, as they came to the realisation that they were in competition for the attention of readers. Transgressions of good visualisation were first held to account by Darrell Huff's 1954 work How to Lie with Statistics. Still a model of accessibility and clarity, Huff 's book categorised examples of how producers of visualisations misled their readers (sometimes intentionally, sometimes not) by violating principles of graphical data presentation. These included truncation of axes, inappropriately scaled two-dimensional objects representing one-dimensional data, and arbitrary choices of statistical averages that misrepresent the underlying data. 

5. A poor example of journalistic visualisation - an inappropriate choice of representing non-related data as a proportion of a whole, as well as depicting linear quantities as two-dimensional areas.

The excesses of journalistic visualisation are still with us today. Despite some excellent examples becoming more prevalent in recent years, we still see a seemingly unending stream of poorly labelled axes, inappropriate scalings and superfluous use of 3D elements. 

These two traditions in visualisations reflect the sometimes troubled relationship between truth and beauty. The truest representation is often diminished by attempts to make it beautiful, and aesthetic considerations can be ruined by the difficulty of using real-world data. Academic visualisation is not attractive, and as such has an extremely limited audience. Journalistic visualisation has promoted attention-grabbing flashiness at the expense of accuracy and fair representations of underlying data. 

Monologue versus Dialogue

Words! Pens are too light. Take a chisel to write. 

 Basil Bunting. Briggflats

Traditional visualisation is essentially an asynchronous medium. The producers of graphics and charts laboured long hours to create a clean version of their depiction of the data. Often hand-drawn, these would appear in print publications, or would be photographed and developed into slides. While this was simply a consequence of the technological limitations of the time, the barrier to creation also leant gravitas to the visualisations. Knowledge that the creation of a beautiful graph displaying rich data would have taken hours if not days implicitly communicated the notion that the creator would have taken the trouble to get the data right. 

6. Minard's depiction of Napoleon's Russian Campaign of 1812 is a classic example of the gravitas that a laboriously produced hand-drawn visualisation provided.

The arrival of graphical plotting software such as S, R, Matlab and Excel have reduced the amount off effort required to produce usable graphs to a trivial level. This in turn has led to a situation where the producer of a graph has great power (to produce an innumerable amount of graphs) but no responsibility to demonstrate how and why they used their data. As Tufte noted in his essay "The cognitive style of PowerPoint", the graph and its attendant presentation is still used in a primarily didactic mode where the presenter broadcasts their message to a passive audience. 

Graphs are and have always been talking points. They are a distillation of a message that would otherwise be hidden in raw data, but often the message is still too rich for useful communication to be one-way alone. We are now in the possession of design patterns that allow us to interact usefully and collaboratively with data, but the gold standard of truth in organisations is still PowerPoint. 

We are in a position to provide audiences with the means to co-explore with us what we have created, yet this is a relatively rare occurrence. Providing the means to make data exploration a dialogue represents a huge area of opportunity for those of us in the technology business. 

Conservative versus Progressive

The pervasiveness of Office as a visualisation tool (Excel to create, PowerPoint to communicate) has led to a standard language of line, bar and pie charts. Well used, these can be effective in certain cases, but the clarity provided can also cage the viewer's thought. Data often need to be manipulated heavily to fit into these tools, and the use of binning, indices of central tendency and surrogate measures all give plenty of opportunity to deceive the reader. 

7. An example of Excel "chart junk". The use of three dimensions inhibits the viewer's ability to understand the data. Data points are obscured by others, and fine discriminations are simply not possible. The use of labels is an admittance of the ineffectiveness of this visualisation, and adds further clutter. 

The standard visualisation tools are also not necessarily best at the job they were designed for. Angular differences are hard to distinguish, meaning that pie charts are a poor choice for judging relative proportions. Standard axes in line charts are arguably worse than the frame ranges and tick marks that Tufte proposed.

8. Tufte proposed adding tick marks to axes, allowing the user to judge distribution along a single dimension at the same time as visualising the correlation between two dimensions

A pervasive heuristic amongst data visualisation aficionados is that three-dimensional visualisation should be avoided. This is still the case for static visualisations, but the advances in technology in 3D visualisation are starting to make that heuristic look shaky. Virtual reality tech such the Oculus Rift, along with nascent attempts to extend HTML and CSS to the 3D realm raise the real possibility that the time has come for immersive visualisation. 

The advent of cheap 3D printing has also facilitated a more tangible sort of visualisation. A physical object brings many benefits such as effortless manipulation, a presence that draws attention to itself, and persistence (it's not hidden in a book, and doesn't need electricity to remain viewable). This is one of the ways that data visualisations are becoming more accessible to lay viewers.

8. 3D printed representation of 24 hours of physical activity (Susan Morris)

What needs to happen to data visualisation now

In the last few years technology has vastly reduced the barriers to producing fantastic visualisations. The possibilities of smooth and intuitive interaction models, adequate and mature 3D visualisation, and novel display methods combine to indicate a few ways forwards:

Stop relying on old-fashioned, static, 1-way methods of communicating data visualisations. The Excel/PowerPoint graphic should die soon.

Use interactions to let the viewer set the level of complexity at which they are comfortable. "Zoomable" data visualisation lets the user determine for themselves whether choices such as mean or median are appropriate to understanding the bigger picture, and they do this without forcing clutter on the user. 

Use technology to improve the attention-grabbing properties of visualisation. What is the use of a graphic if no-one looks at it in the first place? Academics have a moral obligation to disseminate their findings not just to other academics, but to the wider public. By maintaining current conservative and arcane data visualisation practices we are missing an opportunity for wider engagement with complex ideas.

Bring data visualisation into the physical realm. A physical presence simply has more of a shot at engaging people at depth. Harder to create maybe, but all the more impressive for it. 

Design for dialogue rather than monologue. We should not be foghorning our findings, but initiating conversations. The intellectually honest practitioner should provide a narrative and a commentary, but be sufficiently confident to not hide their working, or deny people access to the underlying data. There are always many ways to interpret data, and this should be reflected in the way we create, communicate and consume our visualisations.

[pebblecode]

Data Liberation 101

As part of pebble {code}'s upcoming Visualisation Hack Day we look at the life blood of Visualisations and how organisations can position themselves to take advantage of unlocking value and meaning through open data.

The end of the excel era

Historically data analysis has been conducted in Excel where an analyst will work in isolation and then present whatever meaning has been abstracted from the data in powerpoint. Increasingly this approach is looking long in the tooth. Users expect to be able to interact with data and to drill down into sections that interest us and potentially even build their own analysis.

The web browser is now a very capable means of delivering a rich and interactive data experience and opens up new possibilities for collaboration and interaction. Libraries like d3.js enable data to be delivered in a visual and interative manner and crucially for the user to explore data themselves.

Data analysis can now be collaborative too. Crowd science for example is just one way that innovative organisations like CRUK are using the wisdom of the crowd to propel their research.

All roads lead to data

For all of the new posibilities that browsers and JavaScript libraries like d3.js present the capabilities of a data visualisations are still bound by the ability to access and use data and the APIs that serve the data.

APIs are an increasingly competitive domain where attracting great developers and analysts can lead to significant value for the data owner.

How to prepare

When making your data available via an API you should focus on how easy the experience of using it will be. Is it easy to discover how to use the data? Is the data format the expected one? How is the data secured? Is the service fast enough for a web experience?

If you do not already have a web service this can be a significant undertaking. If you already have an API it is worth re-assessing these criteria. Often APIs were created before the browser became a viable means of visualising data and are woefully lacking.

Be developer friendly

Organisations that understand that open data can unlock business value and insight into data sets also need to realise that competition for talent is a key factor. How an API is documented is often the first interaction a developer has with a data set. If the API is poorly documented or not documented at all a developer may ignore it entirely. If that developer chooses to use their talents elsewhere this potentially a costly failing.

Some API providers go as far as providing API clients in the most popular programming languages to support the usage of the API.

Great documentation and potentially providing API clients in major languages means developers can get on with what they are interested in - interacting with the data and building on top of it.

Build a community

As well as investing in the presentation of the data tech savvy organisations are helping to support and build communities around data sets. Some hold meetups where developers and entrepreneurs can share ideas whilst others run hack events where real products can be created.

Particularly where the data is the product this can allow the data provider to respond to how third-parties want to use the data, developing the product and interacting with customers at the same time.

Unlock meaning and innovation

Visalisations can deliver meaning that would never be uncovered if data is left on a C drive or hidden in a database somewhere. A recent McKinsey report suggested that Open Data can help unlock $3-$5 trillion dollars in economic value. Open data and visualisations are entirely symbiotic here. Without open data it is impossible to create visualisations and without visualisations it is difficult to extract meaning from data sets.

pebble {code}'s Visualisation Hack Day is just a small example of how organisations can take advantage of highly skilled technologists around the world simply by publishing their data. For a relatively small investment organisations can open their data to a wealth of innovation outside of their organisation and give a serious boost to R&D efforts.

[pebblecode]

Pumpi - A raspberry pi in a pumpkin

pebble {code}'s latest hack day theme was hardware. It happened to fall on Halloween. So the natural thing to do was to combine Halloween and hardware. So why not put a Raspberry Pi in a pumpkin and make it make some scary sounds?

Making an LED flash

I was new to working with the GPIO on the Raspberry Pi and after a brief effort to get an analogue sensor working I decided to go back to basics and get an LED flashing. This turned out to be the hardest part of the hack. After briefly failing to compile python and node modules (not enough memory) I began to despair. Thankfully around about this time new employee Jaz Lalli turned up. We found Wiring Pi a GPIO interface written in C.

Taking the example code we knocked up a script to blink the LED

#include <wiringPi.h> int main (void) { wiringPiSetup () ; pinMode (0, OUTPUT) ; for (;;) { digitalWrite (0, HIGH) ; delay (500) ; digitalWrite (0, LOW) ; delay (500) ; } return 0 ; }

It didn't work. We soon realised that the GPIO number was wrong and the pin map we'd be referring to didn't match the GPIO library. After fixing that we were set and quickly added five more LEDs all blinking in unison.

The Pumpkin

Toby Hunt stepped up and went to our local supermarket and carved the pumpkin. By this stage we had flashing red lights inside the carved pumpkin.

Adding sound

The initial idea was to use a proximity sensor to trigger a sound whenever anyone walked by the pumpkin. We found some scary sounds and thought we were all set. There was one problem though - we didn't have a proximity sensor.

Instead we decided to make it possible to tweet at the pumpkin using a hashtag. The pumpkin would read out the tweet.

To listen for tweets was quite straightforward - a node script listened on the api for the hashtag #pumpkinpi. In terms of reading out the sound we first tried Festival to read out the tweets. This was ok but the voice was far too friendly. We needed spookiness.

Jaz discovered this script to convert text to speech via the Google translate api

#!/bin/bash say() { local IFS=+;/usr/bin/mplayer -ao alsa -really-quiet -noconsolecontrols "http://translate.google.com/translate_tts?tl=en&q=$*"; }

This worked really well but again the voice was too friendly. As far as our research went there is no way to change the voice on this api. After some research we discovered that mplayer can play the audio at half speed with the following flag -speed 0.5. This was terrifying!

The hack was complete.

Fruits of our labour @pebblecode hack day. Blinking jackolantern that reads out tweets. Natch.

A video posted by Jasdeep Lalli (@jazlalli) on Oct 10, 2014 at 8:44am PDT

[pebblecode]

The rise of peer-to-peer politics

The Scottish Independence Referendum has demonstrated the rise of peer-to-peer politics and will change New Labour style spin forever.

New Labour and Spin

When New Labour swept to power in the UK in 1997 they did so on a wave of energy for change. New Labour's relationship with the Media was one of the most radical features of the campaign. The idea that a political party can and should influence the news agenda was as central New Labour's philosophy as was moving away from the far left of politics. Under the direction of Director of Communications and Strategy Alistair Campbell and equally Peter Mandelson New Labour were able to push their agenda onto TV and Radio media channels and effectively make the news agenda for the day.

This was possible because the number of channels that news was broadcast to the mass population was relatively small. This comprised of a handful of national newspapers, television and radio channels. For the mainstream political parties in the United Kingdom this approach to the mainstream media has persisted. Intellectual debate is conducted on Radio 4's Today Programme, popular debate explodes on the front page of Murdoch's Sun Newspaper. Media executives carefully manage where politicians appear, the questions they ask and the responses they give.

A warning from UKIP

The problem with a post-New Labour media landscape is the electorate have become wise to the techniques and heavy management of the media. It is transparently evident when politicians from all sides of a debate refuse to engage in debate outside of the brief that media managers have given them. This is the point at which voter apathy materialises. When politicians only speak for their Communications Director and not for the man in the street something is rotten in the democratic state.

Perhaps though personality, although most likely recognising it Nigel Farage, the leader of the UK Independence Party spoke to the media in an entirely different way. He seldom ducks or evades difficult questions in interview and his personal branding reflects a man at home with pint of beer in the pub as opposed to a late night news chat show.

Progressive and traditional spin

Throughout the Scottish Referendum campaign a strong contrast in electioneering styles emerged. The Yes campaign ran a grass-roots campaign backed by social media. The No campaign largely stuck to a traditional style of electioneering, holding rallys and generating events that would generate press columns and TV coverage.

What is interesting about the Yes campaign is that the power of persuasion delivered the most compelling arguments through the activity streams and mouths of their supporters. The very basic idea of a one-on-one conversation with someone you know as opposed to the persuasion of a politician talking behind a lectern ignited this campaign. Granted this issue is central to Scotland's identity but the 84% turnout and vast sense of democratic engagement can be linked to a new technique of persuasion.

The hamster wheel of politicians delivering party press releases in the national media and evading questions outside of their agenda was not seen in peer-to-peer politics of the Yes campaign. Evidently the debate was heated, at times too much so, but in the user feeds of social media real debate happened without the need to follow a party line or push an agenda onto a media outlet.

Of course this is still spin. I'm under no illusion that large parts were orchestrated by a central campaign but by moving away from figurehead politicians and traditional media outlets voter apathy diminished and engagement soared. For anyone that has engaged with the Scottish Referendum campaign it felt completely different to the traditional political media merry-go-round and moreover profoundly democratic.

Power is devolving everywhere

I don't think it is an exaggeration to consider the Internet the greatest force for democracy the world has ever seen, even though some would swap the word democracy for surveillance. In the same way that the Industrial Revolution went beyond the mechanics of production to profoundly influence society the Internet is fundamentally influencing politics, economics and structures of power everywhere.

The Scottish Referendum debate has demonstrated that political influence and power can no longer be centralised but moreover that the strength of a crowd in electioneering can match and even usurp the media strategy of traditional politics. Those in positions of power in government and business would be wise to mark this election as a indicator that traditional hierarchies of power will be further challenged by the influence of the Internet and that new distributed, peer-to-peer models of information exchange and power are here to stay. For democracy at least this is a breath of fresh air.