(Follow along in my process of creating a simple heat map – from initial idea, to brainstorming, trial and error, coding, designing, and more trial and error in between)
Before I start:
goal: create a heat map that displays all of my traces of where I was in 2013.
what’s the map’s purpose: – create something purdy and find out where I have been the most in 2013.
Context: In 2013, I had taken 140 gps traces* using osmtracker, my favorite gps logging software, on the android. Nearly all traceswere taken while I was driving or riding my bicyle.
(This does not include traces that I had taken in Haiti in May-June while with the Humanitarian OpenStreetMap Team.
Here’s some brainstorming:
– what tool to use? Most of my projects and cartographic exploration have been with Tilemill.
In this case, using tilemill isn’t a smart move here, because I had hundreds of layers that i’d have to manually add.
I thought of manually combining all of the traces together – which I did using this simple bash script and a
gpx file manipulator called gpsbabel:
gpsbabel -i gpx $(echo $* | for GPX; do echo " -f $GPX "; done) \
-o gpx -F appended.gpx
I added the merged gpx file as a new layer to a tilemill project and received an error:
“This datasource has multiple layers:
(pass layer= to the Advanced input to pick one)”
Instead of looking for a workaround for the error, I thought about, hey, let’s give cartodb a shot!
I have been looking for an excuse to use cartoDB for ages. I had played around with it minimally over the past year and a half, but my comfortability with Tilemill and the fact that I simply had not had a project where I thought it would be a
really appropriate tool for the job, hadn’t come up.
Let’s give cartodb a shot now…
uh-oh :( )
appended.gpx is too large (it’s 5.6mb). You can import files up to 4.53 MB.
Perhaps I should simplify the GPX traces I thought? I also thought, let’s just convert it to geoJSON and see if geoJSON would resize it’s size… so using ogr, I thought to do:
ogr2ogr -f "GeoJSON" traces.json appended.gpx
ERROR 6: "GeoJSON driver doesn't support creating more than one layer"
ERROR 1: Terminating translation prematurely after failed
translation of layer routes (use -skipfailures to skip errors)
What?! I thought “what do you mean by multiple layers?”
– Does ‘layers’ in the error message mean geometry types (points, linestrings) ? or multiple geometry collections? Some extensive google searching just led to the gdal source code which didn’t give me any more clues..
I walked away for an hour or 2, then thought something might have been wrong in my gpx file.
I opened the gpx file in qgis, and saw, hey it asked me which layer to open of the file and it referred to:
The layers created from my GPX file consisted of 5 layers, including points where I made voice recordings, text notes, or had taken a picture. The layer that I want – the linestrings that I recorded, ended up being the Tracks layer. (The other layers: routes was blank; track_points were the points that make up my linestring; waypoints were points where I had taken a voice recording, text notes, or a picture)
So, I thought I could just specify the layer within my geojson with the following
ogr2ogr -clipsrclayer tracks -f “GeoJSON” tracksonly.json appended.gpx
I thought this would select the layer that I’m exporting to, but it does not.
Alas, that didn’t work, I received an error as mentioned earlier in the post.
So, fellow readers:
Any suggestions for tools to use or workflow for this?
(Updates to this post will be made as I progress along)
What I plan to do :
– Some more google and gis.stackexchange searching for some inspiration.
– save the gpx file (selecting only the tracks layer) as JSON in qgis and open in
cartodb and/or Tilemill.
I also wonder:
Simply convert each of them individually to geojson files, and then add all of them as a layer in a leaflet instance?
Update 1, 2014/01/02, 7pm:
– I converted the tracks layer in my gpx file to JSON (only 1.3mb!) and created a cartodb project for it . Success!
– I’ve also created a tilemill project and loaded my JSON in there. (I’ll post links soon)
I thought of a new idea: increasing or changing a section of the line based on its proximity to another line…
For example, if there’s another line within 60 meters of it at a specific point, increase the intensity of color of the line for a few meters…
This cannot be done out of the box in tilemill or cartodb… but I have a hunch that this sort of calculation could be done in postgis… I would then reimport this data into cartodb and style there.
HMM, this also looks interesting – http://developers.cartodb.com/tutorials/intensity_map.html