Nestoria Interview: Alan Smith of the ONS' Data Visualisation Centre

There's a restriction on how much you can fit into a title, but to be more precise, Alan Smith is the Principle Methodologist of the Office for National Statistics' Data Visualisation Centre.  Meaning he's got access to a lot of interesting statistical data to and it's up to him to present it in a visual fashion.  So you can see why he's something worth putting questions to if you're interested in the how/ why and what of data visualisation.

Before we get into the Qs and As, it's important to mention that Alan spoke to us from a personal perspective – these are his views not those of the ONS.  It's also worth noting that his background is in GIS and cartography.  ALAN: My masters is in GIS, though I confess I haven't fired up a big, proper, grown-up GIS for some while. My first maps at college in the States were ink on vellum! Now that was retro...

Ah, but there's often still a place for the old school.  Here are the rest of Alan's answers:

How would you describe your work and how did you come to do it?

The work I do now - pushing ONS forward in terms of data presentation – is a logical extension of what I first joined ONS to do, namely running the corporate mapping service.  There was a tipping point, about 8 or 9 years ago, where we looking at how traditional GIS software was being ported to the web because we wanted to move ONS into that space. We weren't too impressed with early online GIS solutions, so we starting looking at emerging web-specific technologies like SVG and, later, Flash and realised that if we worked with them, we could move beyond the restriction of 'just doing maps' and take on data visualisation in a broader sense. So in 2007 we made the move to set up a new team with the wider remit...

And how would you describe the Data Visualisation Centre and its aim/s and function/s? 

The role is really to provide a link between ONS' data producing teams and the wider world.  On the web front, that means trying to produce something engaging with entire ONS datasets. Behind the scenes, we also have a role to play defining standards and best practice for basic 'safety-first' presentation, which is very important - my team will never produce more than a small proportion of ONS' graphical outputs, there is a federated approach being adopted for most content creation in ONS.

How does your relationship with the ONS influence what you do and the way you do it?

I think we're naturally closer to the data and that's probably reflected in the form and content of our outputs. While that's usually a good thing, it means that we also work within the same constraints as the data producers - ie. most of our outputs go directly onto the ONS website - so we have a workflow that's currently working within the constraints of the organisation.

What are some of the Data Visualisation Centre projects that you think have been most effective?  And why would you class them as the most successful?

There are three that spring to mind. Firstly, our population pyramids, which were the first thing that I did which made me realise there was so much potential in looking at 'traditional' types of graphics and breathing new life into them with new media via animation and interactivity.  

Secondly, the CommuterView project which allowed us to interactively map commuting flows, showed just how powerful a modern web browser is at data handling and rendering, really powerful stuff.  Finally, the animated map of ageing, which has proven flexible as a mapping template for change over time and seems to have gone down very well with users. The idea of 'brushed' data displays - that is, more than one display of the same data linked to each other (such as a chart, a map, a histogram) is a natural thing to do on the web and this map was a good vehicle for that approach.

Do you even have moments mid way though a project where you realise there may be an even better way to show something?  Or do you come up with many structural options in the early stages?

We tend to produce small, initial prototypes and then iterate rapidly based on our discussions with clients (typically the data producers), so in terms of fundamental approach and symbology we rarely get too far in before changing things completely. Having said that, every project is a learning experience so there's very few projects we'd go back to and do exactly the same way again as you're always trying to move things on.

Where do you sit on the question of style over substance?  Do you believe that data visualisation sits perfectly in the middle, or do you think it still has a very scientific part to play?

The million dollar question! There really is no crime in making something look engaging, even pretty - there really isn't.  But too often it's at the expense of the information in the data, which is what we really care about.
ONS' expertise and insight is always going to be with the data so we're trying to reflect that in the presentation. The key decision for us is about whether visualisations are exploratory (user finds story - and with the way the web works now -shares it) or we want people to focus on a narrative (ONS experts present some insight).  There's room for both, but the risk at the moment is that everything becomes exploratory so I'm quite interested in what happens to the narrative.  Charts on their own can't tell you everything.  A good example recently is a promotional graphic we did for the 2011 Census  – we used hyperlinking between the text and the graphic to encourage people to genuinely 'read' the chart.

At the Data Visualisation Centre you get a lot your data though collaborative relationships with the people who actually collect the data, how closely do you work with the people who actually harvest the data and do they ever offer any input into the way the data is finally presented?

We work very closely with the data producers as it is generally their insight into the data that needs to be unlocked as part of a visualisation, so it's incredibly important.

That said, have you ever had any problem getting access to data that you're interested in working with, or do you usually start with the dataset/s and then decide how to present it/ them?

Yes, I think it's quite interesting that people see 'opendata' as a solution just for organisations outside of the public sector! More accessible data will make MY job easier (eg, discoverable data, less data prep by direct access through data APIs) and allow us to share our visualisations more widely.

Do you think that the situation re:data access is opening up or do you find the government, for example, are still very protective of their data.

See above - in my opinion, it's definitely opening up.  Most ONS outputs have always been free at point of use and so it's really a question about access mechanisms. I can't really comment on other departments other than to say my contacts in them are similarly like-minded people who are keen about getting their data out into the open and used positively by as many people as possible.

What are some of the interesting things you see happening at the convergence point of GIS, mobile and web technology?  And what about when you add social to the mix?

It's very, very exciting. Years ago, I remember getting excited when I saw maps in vending machines on the London Underground - now of course, everyone has a GIS in their pocket, which is extraordinary. This convergence is essentially delivering a connected, personalised web experience, which is great.

Do you think it's these elements (above) that have helped to breathe lots of life into the field of data visualisation or are there other factors that you think were more influential?

It definitely plays a part, but there's something a little more fundamental going on - and it's about data. There is so much data. Too much, really, if that's possible.  And visualisation is simply one of the things that can help abbreviate data into something meaningful. There's other things too - like better data analysis - and convergence between those fields (ie. visual analytics) is very interesting. But right now, it's kind of like a
frontier and we're in a stage of learning by doing.

And what particular trend or innovations in data visualisation do you find most exciting at the moment?  What are you watching out for, or hoping for in the future of the field?

Revisiting the concept of the narrative - the interpreted story – which will help balance things against the exploratory. I quite like the idea that things are getting easier to produce as long as we're not getting lazier at what we think of is acceptable to produce. I love Google Maps - but if everything became a Google pin map, regardless of whether a pin map is the best way of showing a particular piece of information, then that would be quite sad. The American cartographer Mark Harrower has written some wise cautionary words about this - we need to be enabled by the technology, not led by it necessarily.

Thanks for the cautionary closing, and for sharing some of your thoughts with us.  It's been great to get a perspective from someone who works in a more official data vis. role, and it's good to hear that you think there's many more interesting things to come in this field. 

G'day India!

Last year Nestoria expanded below the equator for the first time, into Australia, of course, and also Brazil.  In 2011 Nestoria is going into India . 

In order to make it happen some of the team got to visit India, more specifically Delhi, Mumbai and Chennai, where they met with our new partners 99acres, iProperty and Makaan, and learned valuable lessons about the Indian real estate market and the unique ways people live and do business there.  Naturally they also took the opportunity to learn a little about the local people and the local scene.  J2 reported some of the things he's learned from his first-hand experience of India when he announced the India launch on the UK blog:

We have now learnt that you shake hand with your right hand, we know that Indians never say no and that in Mumbai without a moped, a car, or a truck, you're nobody. Above all, we'd like to remind you all that if the menu says 'spicy food', in India it really means spicy. And even if the menu says the food is not spicy, the fact is that it will still be hot, hot, hot!

If that sounds like your kind of place then here's what you could trade into:

In beautiful Jaipur you can get a a three bed, three bath penthouse, built less than five years ago for RS59.9 lakhs, which works out to A$130,121 today.

Or in Mumbai you can get a flat in this new build with its own fully equipped gym and sauna, located near the building site of a new international airport for Rupee 54,20,500, which is A$117,750.

As usual the team have done their best to get things right from the start, but in such a different market of course there are going to be new things to learn, so we're aiming to evolve with ever evolving India.  This is the time in these welcome posts that I usually invite you to give us feedback if you have any, both on the new Indian product, or on Australia.

Welcoming our southern hemisphere sister: Nestoria Brazil

Until now Nestoria Australia has boasted the listings with the best weather (Ok, so Spain thinks it has the sunshine, but we all know that they'd lose in a battle of the beams.). But that may now change since the launch of Nestoria Brazil.

As well as the sunshine, Brazil has the Amazon (but we have the Daintree?)...

And they invented caipirinhas and churrasco...and we invented lamingtons...

And they samba and we...well sort of do a little groovy side step thing...

Ok, so there might be something in this idea of searching for property in Brazil thing.

Check out this property near Ipanema with ocean views:

And at 5,800,000.00 BRL , it's a snip at $A3,417,685.20.

As with all of the markets we launch in there's going to be a lot for us to learn about how people buy, sell and rent in Brazil, which is why the site has actually been live for a few weeks while we make tweeks and improvements to it. If you can, perchance, read Portuguese and you're having a little dream over the site and you see anything you think could do with improving just let us know through the usual feedback channels.

The caipirinhas that we're drinking to celebrate this launch wouldn't have been possible but for our initial Brazilian partners, who could see the value of Nestoria Brazil before there was even a Nestoria Brazil to see, and our own 'Team Brazil': Francesco Cardi and Anton Abreu, who'll be blogging all things Nestoria and Brazil.

We're going to South America!

Nestoria Interview: Steve Chilton - OpenStreetMap expert/author

This month the Nestoria UK blog spoke to Steve Chilton, co-editor of the recently published book:  "OpenStreetMap - Using and Enhancing the Free Map of the World".   Australia's OSM community might not be quite as big as the UK's yet, but I'm hoping that what he had to say may inspire a few more people to get involved.  So here it is re-edited for an Australian audience, but you can read the whole interview on the UK blog.

Steve manages the Educational Development Team within the Centre for Learning and Teaching Enhancement (CLTE) at Middlesex University, and is also one of the University's Teaching Fellows. By training Steve is a cartographer, having originally studied at Oxford Brookes, and is a research cartographer at the University, and Chair of the Society of Cartographers.  Steve has mapped large sections of Enfield/North London for OpenStreetMap, as well as many other places in the UK and abroad in his travels. He is the main designer of the look of (and additions to) the mapnik layer of the default slippy map.  Steve co-edited the English version of this definitive OSM book with Frederik Ramm and Jochen Topf.

Steve, thanks for talking with us (well, the UK blog readers and us by default). On to the questions!
The global OSM community has grown rapidly over the last few years. Nevertheless, newcomers often complain of the high barrier to entry. Does your book attempt to address this?

The book doesn't specifically set out with that in mind. To quote from the jacket blurb "The book explains the community, the data model and the software behind the endeavour. It enables you to use OpenStreetMap's ever growing body of geodata in your own projects." Expanding on this, there are sections within the book that cover how to use the two main map 'renderers' (Mapnik and Osmarender), with simple worked examples that explain everything in a stepwise manner. There is also a chapter that takes you through the basics of mapping practice. Some examples from within the book are also available on the accompanying website http://www.openstreetmap.info/ , where you can also see the full contents, plus a sample chapter - it is actually the 'Mapping practice' one just referred to. It is my hope that people within the project will soon apply their abilities to addressing the perceived high barrier to entry that you mention.

As a long time OSM contributor, what are some of the things that have surprised you the most about the project? What do you see as the biggest challenges ahead?

Well, when I first heard the other Steve C presenting his vision for OpenStreetMap at a Society of Cartographers conference in 2005 I was very sceptical, and thought to myself "well that will never happen". Well it did happen and it is now a big player in a seriously changing geodata landscape. All sorts of things surprise me about the project. Maybe I shouldn't be (knowing my own situation), but I am constantly surprised by the commitment some particular individuals make to the project. The mapping hours that folk put in are just amazing, as are the software development pathways that we have seen - all in people's 'spare' time remember. There have been a couple of supernovas that flared up and then disappeared, but many of the major players have been prominent from quite early days.

There are several challenges as I see it. Obviously there is the one of getting more comprehensive global data coverage. Then there is the maintenance of the existing data. Will the currently active mappers want to do the less 'exciting' mapping and routine checking that needs to be done to keep the data with its present currency of information advantage? Conversely, will new project members come along who love that aspect and develop new ways of ensuring it happens? Developing a really basic entry level portal for people to add or edit what they see in a non-techy environs is a challenge we must face too I feel, and also more simple data export/conversion capabilities. The planned change of licence from Creative Commons to ODBL has also created some serious potential divisiveness. Managing this process and any possible forks in the project are another big challenge that has to be faced.

In the past few months there have been more and more announcements of companies embracing OSM, be it start-ups or megacorps like Microsoft. How do you predict this will impact OSM, as a data repository, as a web service, and as a community?

I have said elsewhere that my ability to predict the future is about as good as Thomas Watson's of IBM was way back when, but I'll have a go. The interest in OSM shown by the likes of Microsoft and Mapquest recently is only to the good in my view. At the time of these two events I did predict the possibility of some head hunting of major players in the project with either software or project management skills, and to some extent we have seen a little of that already. But I firmly believe that there is such a large pool of fantastically skilled people in the project that I am confidant others will step up to the plate in any situation where this might happen. Also if I may cite the example of Mapquest, they do seem to have taken a pretty altruistic approach, and there seems to be areas of mutual interest between themselves and OSM. We have already seen that in the UK the freeing of some OS data last April has NOT had a dramatic effect on mappers or on data importing in OSM.

Having said that the project will have to make decisions as it goes along about what data is appropriate for the OSM database and what might not be. There is already discussion about keeping data in the DB for things that are no longer there (demolished buildings, dismantled railway lines, etc) - as this has real value for people studying historical geographic data. The availability of OSM data though Mapquest and Bing gives another possible source for people to access the data in mapped form. I think there will increasingly be an opportunity for people/companies to provide customisable map output using the data. Cloudmade already have this, up to a point - but the styles are not portable, the Mapquest ones are already. I would also like to see a major commitment from the members of the OSM community to engage with both the major map and data suppliers, and also governmental and other organisations - in both cases with a view to future possible collaborations.

You're not just a man of letters, but also a man of action - a few years back you headed into the "field" to map Antigua. Tell us a bit about your adventure. What opportunities and challenges are there for OSM in the developing world?

Antigua? I was SO naive and under-prepared for that. Knew it would be hot, but just took my un-reconstructed European mindset with me. Assumed I would just hire a bike, ride that around all day, mapping all the streets/names and POIs/addresses as I went, and cover the whole island in a week. Well - no bike hire to be found, too hot to map for more that about 3 hours in any one day, hardly any streets had nameplates and virtually no house numbers, or even names for POIs such as roadside cafes, etc. Also, I had some good contacts who seemed to promise geo-data from various local govt department sources. All saying yes/soon and stuff, but in the end not coming away with any. But lovely friendly people everywhere I went, including an amazingly laidback TV studio when I managed to swing a breakfast time slot for being a newsworthy novelty visitor - announced as "world renowned cartographer Professor Steve Chilton". Still living off that story! But seriously, there are immense opportunities for OSM to make a real difference to people's lives in the developing world. You only need to look at the effect that the OSM project furnishing up-to-date geodata in both Haiti and Kibera recently has had to see that.

Many thanks Steve for the interview and for your contribution to the OSM community. We highly recommend the new book for anyone looking to quickly grasp the key concepts and technologies behind OpenStreetMap.

Those who are interested should follow Steve on twitter (and of course you can follow all Nestoria interviewees via our twitter list.)

Nestoria hearts GeoNames & Scale Camp

Geonames iss an online database of locations and their co-ordinates – something like 10 million places are listed – that we're all welcome to use under a Creative Commons license.  If you haven't heard of it or used it check it out – it's got the basic text search front page, just like Nestoria does, and returns a list of places that could match your search, which you can then check out on a map. 

I knew there was a Perth in Scotland but I didn't know there was an Illawarra in South Africa.

As you can imagine it's a hugely valuable, and free, resource for any site developers wanting or needing map data.  As we're expanding into new, and less reliably mapped territories we're coming to appreciate these sorts of global, open source repositories of geo data even more, which is why we're sponsoring GeoNames to help them keep providing the service. 

They say they're receiving around 11 million web service requests each day, so that's quite a lot of service to be keeping up. 

In addition to hearting wiki style and user generated open source mapping sites and services, this week we're also hearting and sponsoring Scale Camp, which is an informal event (The Guardian who organise it call it an unconference) for people who're interested in scaling and performance.
Some Nestorians attended last year and reported back that they: “learned a few web scalability tricks, shared a few of our own, met some interesting people, and drank some tasty beer.”  So how could we not want to be part of it again.

Older is Better

One of the BST Nestoria interns has been beavering away on our API, and it's now been re-released in all its glory.  [Insert trumpet burst here] Welcome Nestoria API 1.18. 

In the UK turning 18 means the same thing that it does in Aus., more complex features – and you start getting asked for ID, but you actually have some.

By Flickr user ev0luti0nary

The two main additions are response fields that return API response status:

•     * "application_status_code"; and
•     * "application_status_text".

If you know what that means then you can find out the details on the Nestoria API  Return Codes page.  You could also set your imagination onto the spin cycle with it ands see what you come up with – we're always very pleased to hear about people using our API to do cooler things than we can think of ourselves. 

If you don't know what it means, but you have a vague idea what an API is then, the improvements mean that the API results have been brought into line with regular Nestoria results.  So if you've been using an app. which was using the Nestoria API to request listings by location, it will now receive notification of misspelled, ambiguous or unknown locations.

If you don't know what this means at all, then in a nutshell it means that we provide a way for other people to use the information in the Nestoria database to make other cool applications and websites.  There are some examples of what people have done with it in the past in our API Gallery.

A peek at global house prices

The housing market crash knew no national boundaries, so it can be interesting to examine Australia's recovery in terms of the global market as well as by comparing our local markets to each other.  One way to do that is to look at the Global Property Guide's latest summary of official house price statistics for Q2, 2010. 
The key point is the of the 36 countries they have statistics for, 18 countries showed property price increases, including Australia.  This is what the figures look like:

These prices have been adjusted for inflation.

At the end of the second quarter of last year the market slump was slowing, and prices in Australia had dropped just 2.4% on the previous quarter, but 12 months later and this table shows prices now definitely on the up. 
Generous price increases in Singapore and Hong Kong have lead people to speculate on the forming of another bubble in Asia.  Prices there fell as they did elsewhere, and then bottomed out around Q2 and Q3 2009, when the governments there cut interest rates and introduced stimulus packages as in other countries.  In Q4 things started to move more quickly and there's concern that there are now too many buyers speculating in these markets.  So maybe don't look into the legalities of buying there just yet...  Japan's recovery has been smoother, and Indonesian properties, while not gaining value, have lost only a little ground in Q2 2010.

Stimulus packages in Europe seem to be helping prices there, especially in export oriented Finland, and Norway, where interest rates have been kept low but are gradually starting to creep back up.  And UK prices, helped by healthy growth in London, are also much healthier. 

The U.S. is still struggling with a small reduction in the average value of homes, but Canada's housing market is in a growth phase again.  Canadian house prices stopped falling in July 2009, though there was a modest house price fall of 0.71% during the year to end-Q1 2010.

Ireland is really the only country where prices have fallen more dramatically than they were at this time last year.

When you're an international company like us it pays to look broadly as well as narrowly.  Come on Ireland...

All work makes for a dull property search engine...

Just because many of the Nestoria team would be happy to consider themselves a bit nerdy doesn't mean that we don't like fun.  Which is why, every now and then we rise, en mass, from the basement where we work (which is actually very nice), and come out into the fresh air to work together in a slightly different way.

Last year, as a nod to the seriousness of the financial times, the team went native, or the British equivalent there of, and did some survival training.  This year the challenge was a map based one.  A sort of information scavenger hunt if you will.  Which sounds a lot like the work some of us do on a day to day basis... 

The hunting ground was the area around Clerkenwell, and the quarry was all sorts of random sights and the facts and figures that go with them – all hidden from us with cryptic clues, as well as being difficult to spot.  Ed. noted on the UK blog that his team opted to magnanimously come last so that no one else would have to, which they're terming a 'morale-building last'  – this is until he's satisfied with the findings of the recount.  The team that I was involved in also didn't come first – despite several very competitive team members.  This was because we're so brilliant that we skipped a large section of clues which we thought were red herrings and reached the finish line long before anyone else.  Unfortunately this is apparently like correctly answering questions in a maths exam but failing to show any working... 

So, like Ed. I also have to congratulate the team who did win.  Congratulations.

After all the hunting it was nice to have a drink and a meal and then another drink etc.  The 'hunt' gave us all a lot to talk about.  It turns out that my team wasn't the only one with competitive team members, but with tasks like: 'photograph as many of your team in a phone box as you can', there wasn't space to do anything else but have some laughs.

Thanks to Nomsa, 'Voice of the German blog' for organising.

Bonjour Nestoria France!

Nestoria Australia is no longer everyone's little sibling, as this week Nestoria France has gone live! 

So now the French can search for properties using Nestoria's interface, as can anyone who wants to move to France.  Nestoria.fr works in the same way as the other Nestorias, so if you want to see how the French live pop on over to the site. 

If you're not sure where to look, François, who'll be writing the Nestoria.fr blog, has some suggestions: "Paris is a nice city, but to live, English people like Dordogne, I like Brittany, with Nantes and Rennes at the top. Obviously the South East: Nice and la Provence are really nice..."

This property in the Dordogne works out to be only AU$2,304,812.96

As you may have guessed the Nestoria team is pretty competitive, so I've set up a poll asking people if they'd rather live in London or in Paris... So far London is apparently more popular.

But it could just be that I didn't ask the question in French and the French contingent refuses to respond to a question directed to them in English.  Please lend your support either way.  Your city of choice needs you...

Top Pitch SOTM10

Ruben, one of the Nestoria team, attended State of the Map 2010 in Girona last week, and while he was there he was able to give the award for the Best Elevator Pitch which Nestoria sponsored.  Before the pitch-off started there was a panel discussion, chaired by Steve Feldman about 'Business models in turbulent times', featuring the thoughts of Christian O. Petersen and Roger Muller (CloudMade), Randy Meech (AOL Local), Robert Soden (Development seed) and Matthew Quinlan (Bing Maps).

The speakers then put on their judges hats to hear the 60 second pitches from the five pitching candidates: Raphael Volz with Nogago, Jaak Laineste with Nutiteq, Jonathan Harley with Spiffymap, Felix Lamouroux with Offmaps and Frederic Ramm with Geofabrik. 

Check out all the links for more about the products they were pitching, but pay closest attention to Geofabrik, which is what the winning pitch was about.  Basically the pitch was “pay me money to save a lot of time”.  Check out Ruben's update on the UK blog if you're interested in getting a more thorough picture of what he learned in Girona.