Home / admin / Visualize your Web Traffic & log-files

Visualize your Web Traffic & log-files

I had one of those day’s again where I driftet into the digital world with thought over thoughts how to create something cool. In the end I spend hours surfing without actually achiving anything. My initial goal was to add some cool feature to the blog showing some sort of real-time activity visualisation. Ok at this point not to show off how many people actually visiting as I am well aware that its only a nano proportion, I mean its a blog and I don’t write full-time … even if I would like to do so.

I remembered that some years ago, actually it was in 2008, a group of Facebook engineers (Jack Lindamood, Kevin Der and Dan Weatherford) created a project called Palantir at a Facebook Hackathon event if my memory serves me correct. The project with a nerdy touch was named after the palentir of Orthanc a crystal ball-like magical artefact you might remember from Lord of the Rings. Palentir is showing a visualisation of Planet Earth with real-time data usage.

That was in 2008, 5 years ago and that is a long time in IT development, so I thought along the line it would be raining of web-plugins offering a similar type of animation on a small scale, well I guess I was wrong or I may need to up my google search skills because I came up with nada.

Ok there are plenty web-based visualisations of real-time data feeds out there but not in a compact from. One I came across that I really liked is from the french web designer Franck Ernwein who shows the world in a new light. He drops a bright pixel on the location of tweets when the visiting the site tweetping. Pretty cool.

Something caught my eye…

I came across a website showing several different types of awesome visualisations @ pingdom.com If you look at the date this post is around 3 years old already… but here I found what I was looking for. My personal favourite (Apache Pong) was the first video on that page showing a visualisation of the website access log from logstalgia.

Now that I found something coming closer to what I wanted I checked out the software and saw it’s not quite up-to date. Last package found on the download site is from February 2011 and the code available for linux and windows. I decided without much thinking to give it a try.

Well it did not work out of the box and I had to tweak it a little to get it working on OS-X Mountain Lion. I have not installed in on Windows nor Linux yet but will do that as well and post the How-to – Guide next.

Here my first test


Mission accomplished?

Not quite but getting there. Finally I got a working visualisation of my web-traffic on my desktop. It is not real-time at this point as I don’t have a direct ssh connection to my web-server. I decided I am going to try the following 2 challenges which should not be too hard to get up and running:

  • write a script to download yesterday’s web-access logs (automatically)
  • combine all logs into 1 big file
  • try other log-visualisations like Firewall etc…
  • run the visualisation as Screensaver…

as always a how-to guide will follow

stay tuned…



Check Also


Mobile Payment Systems in Africa

A frica boasts of some of the fastest developing economies of the world, and just …