Image Image Image Image Image Image Image Image Image

data visualisation



September 22, 2011

one tube minute

September 22, 2011 | By | No Comments

Click here to view the visualisation

When waiting on the platform a minute on the London Underground often feels longer than you’d expect. I decided to investigate this, and get some clarity on the length of one tube minute.

To do this I had to start by gathering the required data.

Data Gathering

Before I could record the duration of one minute according to the arrivals displays on the Underground, I needed to decide how I would measure this minute. Determining the start of the minute was easy. A minute starts when the arrival display switches to show that there is one minute remaining. The end of a minute was trickier to choose.

As far as I could tell there would be three options here:
– When the train enters the station
– When the train doors open
– When the train departs from the station

Because I wasn’t sure that I could always accurately record the point at which the train entered the station, and train departures can be delayed by people stuck in the tube doors, I chose to measure a minute as the time from when the arrivals display switches to one minute, and the time that the tube doors open.

With the method sorted I continued to record data using Reventer.


Having gathered the data I sent it to Google Docs (easy to do in Reventer) and started thinking about the best way to visualise this data. I’d recently read some good blogs about Processing.js, and having played with Processing in the past, I decided to use this to create the animation. I was also reading Nathan Yau’s Visualise This, which emphasizes using data to tell a story. To give readers the chance to experience the duration of the various tube minutes for themselves, I decided on using the animated balls to represent “minutes”.



April 23, 2011

massivedata on github

April 23, 2011 | By | No Comments

Nothing huge, but I’ve moved the massivedata demo to github. The project isn’t much more than the demo at the moment, but this is, if nothing else, a way to show my intent for massivedata as an open source project.



October 8, 2010


October 8, 2010 | By | No Comments

Over recent years data visualisation has become a topic I really enjoy. Being a DBA, I’m used to trying to make sense of large volumes of data, but there is something extremely satisfying about the immediacy of a clear visual representation. What I was looking for is a tool that would always be available (home, work wherever) and that can handle very large amounts of data.

That’s partly what sparked my interest in HTML5, which is promising a future with a blank canvas on every PC. From there my search took me to webgl (partly because of a game that I’ve been wanting to write for over a year now), and that’s when I had the idea for massivedata.

The idea is to have an easy and clear way to visualise large amounts of data. Taking into account that webgl harnesses the power of the GPU, it seems to be the perfect fit for working with large-scale data visualisation.

How it works

You start off in heatmap mode. This corresponds to a top-down view in 3D space. Here values are represented by colour, and you have the advantage of seeing a very large amount of data at once.

From heatmap mode you have the option of switching to a barchart view of either the columns or rows involved in the representation. A slider gives you control over which row/column you’re viewing, so you can compare values in different ways.

Finally there is explore mode, in which you have a full 3D view of your dataset.

This is still very much a work in progress, but find out how to get a webgl-capable browser, and then check out the demo.



May 26, 2010

Simple Heatmap Generator

May 26, 2010 | By | No Comments

The simple heatmap generator is available as a template in the google docs template gallery. It’s basically aimed at being a quick way to see what your data will look like as a heatmap. For the final product I would suggest you use a great tool like R.

Google spreadsheets do have advantages though, like allowing you to embed spreadsheets in webpages.

Give it a try and leave your feedback in the comments.