App. Update

The apps4nsw competition has closed which means you can now have a look at all the entries.  And while you're at it you might want to vote for your fave. NSW related app. in the People's Choice Award.

Perusing the current top scorers and you'll find an app. for ranting about public transport, and a couple that access health data, but not too many map-apps..  The most popular map based apps. as this is being typed are all for the iphone:

City Tag: which uses pretty much all the  geospatial data the NSW gov. has to offer – public toilets, ATMs, BBQs,playgrounds etc. - but also lets you 'tag' your own favourite locations for other people to use.  The data it collects when you tag is kept using the Creative Commons license.

That's Camping: lists campsites and their details, the most useful element of which is probably the map directions to get to said campsite from your current location once you've picked a place to set up your tent.  (Including a local weather report might have been a nice touch...)

That's Camping screen shot

And an idea called NSW Traffic Report, which is cooler than than the dry name sounds: a mash up of weather reports, traffic reports, RTA cameras and the latest public transport info.  Unfortunately you can't see it working but it sounds pretty dang useful.

Art, Money and Maps

Further proof that pinning information to a map can make it a lot more interesting comes from two very different sources:

First, a Mapize map with the billionaires from the Forbes list pinned on to it (not literally of course), which could also be used as fuel in ye olde Sydney vs. Melbourne debate, because it shows that Sydney has five resident billionaires against Melbourne's three.  Perth has one too, but surely the place with the greatest wealth per capital in Australia must be Humpty Doo, as there's a billionaire residing just outside of it – take that Sydney and Melbourne.

And second, a map to which great landscape paintings have been pinned.  This map is on a site called Geo-Coded Art.

It's supposed to bring attention to some of the world's greatest, but overlooked, landscape paintings, and you can browse by location, artist or painting name - it's good fun just to have a look at the map and see which areas are most painting-ogenic.

Competing to make government data more interesting

What do you do after you've mashed up some information with some maps?

Well you might want to create an application to help display it.  And after the interest shown in the MashUp Australia competition both the NSW and Victorian state governments are getting in on the free-ing up government data act by launching App building competitions, aiming to pique people's interest in doing new things with the government data they've released into the Creative Commons.

apps4nsw screen shot

The apps4nsw competition came first, offering a $100,000 prize pool for people with either the ideas, or ideas and development skills, to make good and interesting use of some of the government data available from the NSW Data Catalogue – which includes datasets like the Map of Fixed Speed Cameras in NSW or the Additional Transport Timetables set up for Major NSW Events.  The closing date for submissions of either app ideas, or prototypes, that will, as the website says: unlock the potential of NSW information, is the 22nd of March.

Never to be outdone by NSW, the Victorian State government announced the App my State competition, which invites people to create or imagine applications that: aim to make Victoria the best place to live, work, visit and raise a family. The prize money on offer is the same, and you're invited to use just Victorian government data, source your own or mash it all up together.

Vic apps screen shot

You can have a look at some of the Vic app entries so far here and the idea entries here. The competition closes on the 23rd of April.

It's interesting to see what kinds of things people come up with but it's more interesting to see what new data, if any, the state governments give people access to.

Nestoria Australia Interview: Andrew Naish

Feel free to accuse me of becoming a little bit too interested in mashing, but I can't help it, there are just too many cool mash-ups out there that once I started looking I couldn't stop 'accidentally' finding them...well, it's a legitimate part of my job, right...?  One mashup that's relevant enough to Nestoria's antics to justify it being mentioned here is 'How Safe Is Your Suburb?, a map based mashup of NSW police data, that could help you choose the safest place to live.  This is what Andrew Naish, one of the people who built it, had to say about it, and about other aspects of his work in the geospatial sphere, when I interviewed him for the blog:

Hi Andrew, welcome to the digital couch.  Let's start by talking about the couple of mashups you built for the MashUp Australia competition.

Space-Time Research, who I'm a GIS Technical Lead Developer for, entered two mashups: one was 'How Safe Is Your Suburb?' and one was the 'KML Cruncher'.

For 'How Safe Is Your Suburb', we had a choice of data sources from data.australia.gov.au and realized that most of the data sets did not contain geography variables. We wanted to be able to visualize data on both a chart and a map, and the NSW Crime data had interesting information both geographically and over time. We ended up mashing the NSW crime data with an ABS Local Government Area (LGA) mapping file to present a thematic map in Google Maps. We were able to categorize crime into different types, and our map shows how many crimes were committed in certain areas by category (e.g violent crime vs. white collar crime) and for either all years of the survey or individual years. Once you've queried the one you want, you can save it as a pdf, image or save the data associated with it for sharing, and you can comment on the data.

Rather than simply displaying the raw NSW Crime data, which is what many other competition entrants did, we wanted to come up with a story to do with the data that would be interesting for potential users of the application - citizens, researchers, policy makers, government and police. So we develop four key views:

* The first enables users to view the data on a map to see a visual representation of offence by type by year by Local Government Area.  The user can choose to hone in on different areas, types of crime or time periods depending on where their interest lies.

* The next is a crime breakdown by year pie chart that shows which crimes are most prominent for a given year.

* Crime in My Suburb shows a breakdown of crime over time and you can select the crime type(s) and suburb(s) to include.

* The last view shows the most dangerous suburbs, the top 10 suburbs showing number of offences. The user can select the offence type and the year of interest.

The KML Cruncher submission wasn't actually a map per se – it's basically a tool that allows people to easily create web maps from one of the most common geospatial digital formats - the ESRI Shapefile format. Most organizations dealing with geospatial data (especially government) use this file format, but unfortunately it's not very web friendly and can be a little difficult to work with. What the KML Cruncher does is two main things:

*  Converts an ESRI (polygon) shapefile format into KML (which is more web friendly format).

*  Generalises (simplifies) the polygons according to a specified tolerance so that the resulting KML is small. Smaller KML files mean faster loading and faster draw times.

We settled on the idea out of necessity -- at Space-Time Research we are always manipulating geospatial data in to make it work in Google Maps, but we were mainly using high-end GIS products like the ArcGIS suite to do it. You kind of need to know what you're doing in order to convert a Shapefile to KML using the ArcGIS suite, but KML Cruncher simplifies the process so that hopefully anyone dealing with the shapefile format can generate a KML file and use it in Google Maps or other mapping applications.

Cool ideas.  How difficult would it be to roll 'How Safe is your Suburb' out across the whole country and what would the challenges be?

It would be easy, if we can get the raw data. We would expand the application to mash in Australian Bureau of Statistics population data, and include socio-economic and demographic variables.

One of the aims of that competition was to promote open access to geospatial data - how much of an issue is access to data for people wanting to play around with maps?

It's a big issue - and one that hopefully the KML Cruncher will alleviate somewhat. As I mentioned earlier, most of the geospatial data people want to use on the web is stored in ESRI shapefile format inside internal networks, if you want to convert it into a web friendly format, and (more importantly) make it load nice and fast, there are no tools out there that can do it like the one we have created (trust me I've looked).

Thanks - I'm sure we'll find that useful too!

I think the other issue is the lack of a centralized datastore for geospatial data in a web friendly format - what I'd really like to see is access to any country's geographic hierarchy (like the Australian ASGC) in a wide variety of web formats (KML, JSON, GeoRSS, etc) – that's when we'll start to see some really interesting mashups. Unfortunately there are some hurdles though - namely data integrity (i.e. people need to know exactly where geospatial data has come from so that it can be verified as being genuine) and also there needs to be a simple (and I can't stress the word 'simple' enough) standard developed for the geospatial attributes (i.e the attributes describing the geographic boundaries) so that GIS engineers can interoperate with it seamlessly.

I often ask this question to the people I meet on my quest for interesting mashups: but what are some of the most interesting uses for geospatial data have you come across?

I'd say one of the weird cases was when I was working for Webraska Australia.  We were heavily involved in mobile mapping (handset navigation, maps for your mobile, etc) - one of the projects we went for was what we (internally) dubbed 'The Granny Tracker' - it was basically a mobile device with a button that an elderly person could push to alert a central office to exactly where the person was. Great idea, might have been a little intrusive though.

I've also got some really interesting stories around location intelligence when I used to work for ESRI Australia contracting to the Australian Defence Force, but I'm not going to share those!

What do you think people will be doing with geospatial data in the future?

I think mobile location intelligence is going to be very big in the future - we're going to see a lot of work involving geospatial awareness, and the first commercial use will be around advertising. For example, we'll see applications that track your location and determine target adverts based on your profile and where you are.

In terms of slightly more productive uses I think we'll see a general shift toward more openness of the geospatial data from government organizations, and ease of access, which will in turn allow us to really dig into more spatial analysis - I'm thinking mainly correlation analysis to start with. That is, we'll be able to determine answers to questions like, "Is there a correlation between the number of obese children in an area that also has a high number of fast food outlets?" Such questions can be answered easily with a map that is drawing from multiple data sets.

Also I am 100% sure there are many correlations between datasets that we simply don't know about, there are ways to determine correlation coefficients programmatically so we could, theoretically, find answers to questions we could never dreamed of asking ourselves.

I've got the impression from the people I've spoken to that the geospatial and mapping community in Australia is a pretty interesting space to be in, what are some of the projects the Australian community is engaged with?

Yes, the geospatial community is quite vibrant here, albeit small compared to other countries (everyone knows everyone else in the GIS sector!). There is wide range of projects being undertaken down here, which keeps me interested.

At Space-Time Research, mapping is one of the core outputs for our data dissemination and visualization products. We're finding more and more government agencies want to disseminate their data publicly, especially now that the Government 2.0  movement is in full swing. For example, we've worked with the Department of Transport on their VISTA 07 application, and the Australian Bureau of Statistics CDATA Online and Table Builder products.

I know that one big challenge agencies are having is dealing with privacy and 'confidentialising' data. We've worked out ways to handle that and all of our products incorporate the security methods, so hopefully as our products get used by more and more agencies it won't be too much of a challenge for too long.

Thanks very much for sharing, with me especially – hopefully that's my map fix for today!  I think you might have been overly hard on the Granny Tracker...I can see several different potential uses...

Andrew graduated from the RMIT Geomatics department 12 years ago and is now a GIS Technical Lead Developer at Space-Time Research.  He also dabbles in making online games and playing music (but says his music isn't quite popular enough to give up his day job just yet.).

Mashing up with the Australian Government

One of the things on the 'to-do' list leading up to the Nestoria Australia launch is to collect as much interesting geo-spatial data as possible to add to our maps.  Quite a lot of this information is owned by some facet of the Australian government and they're not always entirely trusting when it comes to entrusting people with it, despite the existence of the Australian Gov. 2.0 Taskforce, which is supposed to be all about online engagement between the public sector and public sector information, and the individual.  These are the guys who can help free up the information we'd like.

While part of the engagement process is about freeing up access to data, there's another side of it, proved by Google's Public Sector help page, and that's encouraging government agencies to use other web tools to add relevance to their sites and present their own data in interesting ways.  The 2.0 Taskforce were the organisers of the MashupAustralia Competition which was supposed to encourage people to use government data in an interesting way, so it's interesting to see some government agencies using Google Maps as templates for presenting their own data.

Mapping our Anzacs

The Anzac spirit is one of the few non sport related sentiments that the majority of Australians are united on, and this site, created by the National Archives of Australia, arranges the details of the 375,900 servicemen and women against a Google generated map of where each one was born or enlisted.

Australian's World Heritage List

The Department of the Environment, Water, Heritage and the Arts presents its information about Australia's 28 World heritage Listed sites (from Kakadu National Park to the Sydney Opera House) pinned onto a Google Map as well as in a list.

NSW Government Stimulus Snapshot Map

And the NSW Government has made some coloured warratah icons to use as pins to map out the different kinds of projects its $62.9 billion Economic Stimulus package will be spent on.  [Red is for education and purple is for transport.]

The Full Wiki: A Big Mash Up

There are so many different combinations of data to mash together that it makes sense that some of the really big ones might not be the most obvious choices.  Like Wikipedia for example.  It's already got geocoded destination links and numerous maps, but imagine all the possible things that could happen if it got a good mashing with Google Maps?  In the case of the full wiki you'd get a reference site where a map is displayed alongside the reference text, and each location mentioned in the text is marked with a map pin.

It's a genius site for looking at pages like the First voyage of James Cook 'cause not only can you see the pins marking his route, but you can also scroll over the pins to skip to the bit you're interested in.

Pages for The Ashes and The Sydney to Hobart are less interesting, but the stories of the ill-fated Burke and Wills Expedition and the exploits of Ned Kelly gain a lot from being mapped out.

Mashup Update

Here's a Mashup Update, the winners have been announced and they're both implementations with a similar feel and a relevance for Nestoria users:

Suburban Trends compares and contrasts suburbs on things like crime statistics, levels of acquired education, socio-economic standing and perceived safety standards,

and

Know Where You Live which aims give you as much information as possible about where you live, including historic photos, rental price data and various statistical data – the judges liked the citizen-centric 'common questions' section, which added to a user experience that could have been considered a little dull considering much of the data available at the moment is a bit on the dry side. This mashup was also created in 24 hours...what have you done with your day today...

Have a look at the full list of winners here.

We're not the only ones who love maps: Nestoria Interview with Julian Doherty

To justify the time I spent trawling over the MashupAustralia competition website I got in touch with one of the map mashers, Julian Doherty who is also a senior software developer at Lonely Planet.  At LP Julian is part of the community product team, who are responsible for developing all the social user interaction features, like discussions, groups and multi-user gadgets.

Julian also adds: I'm a geek through and through, I love playing around with technology and software development. In my spare time I play a lot of video games, and run a few websites about them (ubercharged and ihatevans are a couple).

Let's start with the map mashup competition, can you talk to us about the map you produced at the govhack event and how you settled on the idea?

The mashup we produced is called "it's buggered mate". It's a simple mashup that allows users to pinpoint maintenance tasks that their local council needs to attend to by picking a location on a map, drawing details, and annotating with text and icons. The app itself is pretty basic. It's got simple data entry and uses Google maps with some extra layers that Ken Hoetmer (another Lonely Planet person) wrote to allow drawing and annotation on the map.It's gotten a bit of media attention in the press releases etc - I think mainly because of the name. We intentionally made it a bit cheeky and very "Australian" so it would seem familiar and make users more comfortable. Which is ironic, because the LP team we had at govhack consisted of me (a New Zealander), a Canadian, and a Welshman :P

We settled on the concept because we couldn't think of anything better. The data feeds available at govhack weren't actually all that inspiring, and a lot were at an aggregate level without any geocoded information, so they didn't lend themselves to creating deep mashups with lots of user generated content. In the end we built that one because it *doesn't* actually read any data from any government sources, but rather allows data to be collected and fed *into* government rather than out of it to the public.

Later we integrated council lookup using geo2gov.com.au (another govhack project) to figure out which council was actually responsible for the area that users have identified (but I think they're changed their site, and the lookup isn't working right now, I'll have to fix that)

One of the aims of the competition was to promote open access to geospatial data - how much of an issue is access to data for people wanting to play around with maps?

Access to data was an issue. A lot of it is aggregated, and not presented in a format that is readily usable by 3rd parties (Excel spreadsheets, proprietary formats etc). And most of it doesn't contain a lot of geocoded information, so it makes it hard to cross reference it against your own data. There are a lot of government information sources that *are* available, but only via a web form that has to be used from a browser - which makes it difficult to consume from a mashup without resorting to screen scraping, which is a pain. Ideally, all these data feeds would be accessible via a simple web service API that other 3rd party applications can talk to automatically to pull down data to mash up.
That isn't because things are secret, or restricted, more that they just weren't designed to be used that way. Hopefully awareness gets raised through events like govhack, and data will become more available. Even in the last couple of months a lot of data sources have opened up.

What are some of the most interesting uses for this kind of geospatial data have you come across?  From both the practical to the weird and wonderful.

Some of the coolest ones I thought were the ones that layered a heap of data sources together to give you an interactive picture of what different areas are like. e.g Suburb Matchmaker , Know Where You Live, Suburban Trends and Lobby Lens which did a similar thing combining government lobbying data. There were a few "find the nearest toilet" apps that seemed to be crowd favourites as well :P

What kind of interesting implementations can you imagine on the horizon?  (Even if it's the far off horizon)

Big trends in internet development are real time, geo, mobile, and user generated content. These all relate to each other, and make 2-way communication between users and data easier and more useful. Getting real time updates on events happening around you, and providing feedback will be very cool. I think the user interfaces of mobile devices needs some time to develop, at least for the type of mashup we did, it's a pain to click through text fields to enter details on a mobile device. Having intelligent devices that can figure out what's going on around you and automatically send through that data for you would make these type of apps really useful.

e.g. for its-buggered-mate, it would be great to just point your mobile phone at a pot hole in the road, or a broken playground, take a picture, then have the phone figure out what you want to do - in this case report the problem along with the location and maybe some context information about it without any other user interaction.

You recently held a HackDay at LP Melbourne, can you talk about why you hosted the event and what kind of interesting ideas came out of it?

The LP hack day was to promote the LP places-of-interest API that we've recently opened up for public use. and also to join forces with govhack to host a Melbourne event following the Canberra event the weekend before.
In addition to promoting the API, it was also to test it out in the real world, and see what else needs to be added to it to make it useful.

The LP API exposes places of interest (POIs) collected as part of the travel guide production process (hotels, restaurants, sights etc) tagged with geo info. 3rd party applications can find POIs based on a place name, destination, or by specifying a lat/long bounding box.
Google developed a Wave application "Trippy" with us as part of the Google wave launch a few months back using this API.

There were a lot of cool apps that came out on the day. The winner was one that generated a road trip from a starting point to random interesting destinations in the area: A user enters a starting point (e.g. "Melbourne"), and a distance they want to travel, the app then goes and pulls down POIs from LP in the area, and plots the road trip with directions onto a Google map. Very cool - Although it uses the API in ways we hadn't anticipated and stresses it quite a lot. We had to bump the throttling limit WAY up to even let it work. Which is good, that's the kind of feedback we wanted to get, and it helps ongoing development going forward.

I've got the impression that the geospatial and mapping community in Australia is a pretty interesting place to be, can you give me an idea about current thoughts and trends in the field at the moment - what are people whispering about, what are people complaining about and what are people trying to crack - that sort of thing.

Yeah, there's a lot of interesting stuff happening. To be honest, I'm only closely familiar with what's happening at LP - although I think Google Maps was developed in Sydney, and there's some very cool stuff happening there.
Part of what we're working on is trying to link existing content more closely with geo data. We've got a heap of content on destinations that is generated by forum posts from users etc, but a lot of it has either no geo data, or only very coarse grained data based on country or maybe city, which can make it hard to link together. We're trying to figure out ways of grouping together by automatically tagging it based on general themes and location. Hopefully so we can better and present it in a way that it can be easily cross referenced by different parts of our site.

Is there anything else you'd like to add that I haven't quizzed you on yet?

Probably, but I can't think of anything off the top of my head :)

Thanks very much for your time Julian.

No probs

We love maps

There is a certain kind of person who gets a lot out of looking at maps and several in the Nestoria team qualify as this 'kind' of person.  This isn't a new thing, but at the point where maps and technology converge there are some fantastically interesting things going on – things that are turning regular folks into map geeks.  Yes I'm talking Google Street View, Aerial View, OpenStreetMap and more, but once the powers of these tools are harnessed by developers the world starts to look a lot more like a great big greeny-blue oyster of potential...

...that can be combined with other information to create...well almost anything: from a GPS system for thousands of the world's golf courses to a map of where the most twittering is coming from.  And other such practical ways to make the world a better place.

The Australian Government has waded into this map mash fest by launching a competition to MashupAustralia.  They've basically released a selection of interesting geo data and said “so do what you will with it”. Unfortunately I only found out about it as the competition was closing, but the winners have yet to be decided so you can still vote on your favourite from among the 82 mashups, which include the Suburb Matchmaker: which presents the NSW crime data for where you live now and where you may want to move to, My Solar Saver: which can help you save money on your hot water bills by location,  as well as saving the planet of course, and Bike Community Hub: which allows cyclists to share routes, hazards and crashes – like an eye-in-the-sky for bikers.  There are loads to wade though...and it's a most enjoyable way to procrastinate.