I’ve been using RunKeeper to track my walking. I wanted to work with the 40+ GPS tracks they’ve logged for me and didn’t want to download them one at a time. Some quick notes on how to scrape them all out.
- Log into the website in your browser and go to your Activities page. It has a URL like http://runkeeper.com/user/USERNAME/activity/70000177
- Save that big HTML page to a file named activities.html
- Capture the IDs of all your activities from that page with a regexp like this:
printMatches ‘link=”/user/USERNAME/activity/([0-9]+)'< activities.html > ids
- You can now download a GPX for that ID with a URL like http://runkeeper.com/download/activity?activityId=70000177&downloadType=gpx. However, the download requires a URL, at least if your activities are private.
- Look in your browser for your JSESSIONID cookie. That’ll stand in for a login.
- Use wget with that cookie. Here’s my little shell script to get them all
cat ids | while read id; do wget --no-cookies \ --header 'Cookie: JSESSIONID=C~2BF2D2156284732CE8D8BDA900000000' \ -O $id.gpx \ "http://runkeeper.com/download/activity?activityId="$id"&downloadType=gpx"; done
Set up a PostGIS database with all the data
- createdb walks
- psql -d walks -f postgis.sql
- psql -d walks -f spatial_ref_sys.sql
- psql -d walks -f postgis_comments.sql
- for f in *gpx; do echo $f; ogr2ogr -append -f “PostgreSQL” PG:dbname=walks $f; done
ogr2ogr will create a fancy schema for you that contains every field ever seen in a GPX file. For RunKeeper data, the salient things are:
- tracks.wkb_geometry: one row per track, multiline string
- track_points: one row per point with a point, elevation, timestamp, etc.
QGIS did a fine job rendering the tracks table for me. I also pulled in a raster OSM map for a base layer from http://wms.latlon.org/.
Here’s a quicky query to get the length of a track in miles. Need to understand the ST_Transform better, also join to something else to get proper data about each imported track.
select name, ST_Length(ST_Transform(wkb_geometry, 2163)) /1609 as miles from tracks order by name;
A quicky view to simplify the tracks a bit, plus a query that compares the length of the original and simplified views.
create or replace view simple as select ogc_fid, ST_Simplify(wkb_geometry,0.00005) as track, name from tracks; select tracks.name, ST_Length(ST_Transform(simple.track, 2163)) /1609 as miles, ST_Length(ST_Transform(tracks.wkb_geometry, 2163)) /1609 as miles from simple inner join tracks on simple.name = tracks.name order by name;