VizWiz

Launch, grow, and unlock your career in data
Showing posts with label linear geometry. Show all posts

June 19, 2020

How to Get Your Data from Strava to Tableau

, , , , , , , , , , , , , , No comments
UPDATE: If you prefer video tutorials, there is now one on my YouTube channel here. The steps to get the links into the Simple Mass Downloader are done more efficiently in the video. There are several steps removed from the Excel and link uploads that are listed in the blog.



For a while now, I've been using this process for getting my data from Strava into Tableau like this:
  1. Download the data from Strava
  2. Prep it with Alteryx
  3. Visualize it in Tableau

In this post, I will outline the steps for getting your data from Strava into Tableau. I have also created a template with some sample vizzes for you to use. Download it from Tableau Public here.

I started revamping the process after wanting to create a small multiple view of all of my activities like this:


When I first started creating these, I took a process based on R and converted it into a workflow in Alteryx. This worked great, but there were two main drawbacks:

  1. My Alteryx workflow was a mess (I tinkered with it every month).
  2. Each of the routes would be adjusted to fit perfectly in a square.

This second problem meant that the routes were not scaled correct according to the curvature of the Earth. I then saw that Andy Cotgreave reached out to Ken Flerlage for help. They've written about their process here. Great! However, the calculations on this process make rendering the maps in Tableau very, very slow. Like REALLY slow.

That's where Alteryx comes in. I pushed all of the data processing into Alteryx, export each route as a single spatial object and every renders super fast. So if you're interested in getting all of your data out of Strava and visualizing it in Tableau, this process is for you. It does require Alteryx, but you could replicate the process in R.


TOOLS REQUIRED

  • A Strava account - NOTE: Make sure you are logged in to Strava throughout these steps.
  • Alteryx Designer
  • Tableau Desktop
  • Simple Mass Downloader Chrome extension

GET THE DATA FROM STRAVA

Step 1 - Login to Strava, click on your profile icon on the upper right, choose Settings.


Step 2 - On the My Profile page, click on My Account.


Step 3 - Scroll to the bottom and click on the Get Started button in the Download or Delete Your Account section.


Step 4 - In step 2, choose Request Your Archive. Don't do anything else on this page.




Step 5 - Check your email in a few minutes. When it arrives, click on the Download Archive button in the email.



Step 6 - Unzip the file that downloads. Open the folder that's created and you should see something like this.

The activities folder contains all of the raw files, but these won't all be in the same format, which means they're useless. The ONLY file we need is activities.csv. Open the CSV in Excel.

Step 7 - The only column we need in the file is the Activity ID column.


Delete all columns except Activity ID.


Step 8 - Insert a column to the left of Activity ID. I name it URL, but call it whatever you prefer. Then in cell A2 enter this formula: 

=CONCAT("https://www.strava.com/activities/",B2,"/export_gpx")

Then copy it down for all rows. This create a link to each activity in GPX format. If some of the activities don't have location data, don't worry about it. Those will simply not download in the process.

Choose the URL column header (the "A" above URL), choose Copy, then Paste Special => Values. Then delete column B (the column with the activity ids).



DOWNLOAD EACH ACTIVITY

Step 1 - Add the Simple Mass Downloader Chrome extension.


A new tab will open with a tutorial if you're interested to learn how it works.

Step 2 - Click on the extension button and you should see this screen.



Step 3 - Click on the hamburger on the right and choose Import URLs from local file.



Step 4 - Import the activities.csv file and the URLs will upload. This loads all of the files into the queue on the Download List tab. Choose the Select All box and they should all be highlighted.



Step 5 - Click on Start Selected and you'll see a bunch of files start downloading very quickly. Again, some of them will error out if the activity doesn't have location data. The files that download will have a green check next to them.



Phew! That might seem like a lot, but once you do it a couple times, the process is really quick.

Fantastic...all of the files are now downloaded. Onto Alteryx we go. 

ALTERYX PROCESS

Step 1 - Download the Strava Route Maps workflow from the Alteryx Gallery and open it in Alteryx Designer. If you can't download it from the Gallery, I have it on Google Drive here.


The Strava icon is a simple macro that will import all of the GPX files in the Directory you specify when you run the app. 

The workflow is split into four parts:

  1. Import the data, strip out the parts we need, then create points and lines.
  2. Normalize the data based on the calcs from Ken Flerlage, turn them into lines (one mark for each route rather than hundreds of point for each route) and export as a Tableau extract.
  3. For each point, calculate the distance, climb, etc. and extract all of the points.
  4. Take each of the points and turn them into lines.

Neither part 3 nor part 4 above normalize the data into squares. These are simply all of the data for each route.

Step 2 - Run the workflow as an App by clicking on the magic wand next to the Run button.


Step 3 - Choose the folder that contains all of the GPX files from the Simple Mass Downloader output. And click Finish.


This will generate three Tableau extracts that will load into the same directory as the GPX files. The workflow extracts them as TDE because Hyper files don't support polygons. Even though these aren't polygons, it makes me more comfortable that Tableau will read the files correctly.

Success!


Click the Clear button and then the OK button. Otherwise all three files will open in Tableau, which we don't want.

We're all done with Alteryx. Onto Tableau.


TABLEAU PROCESS

I've created a template for you to get started with. Download it from my Tableau Public profile here. Once you have done that, follow these steps.

Step 1- Open Tableau and right-click on each data source and pick Edit Data Source.


Step 2 - Click on the carrot next to the data source name and choose Edit Connection.


Step 3 - Navigate to the data source that downloaded from the Alteryx workflow. It will have the same name.



Step 4 - Go back to your worksheets and everything should update automatically. If not, right-click on the data source name and choose Refresh.

Step 5 - Customize the views as you see fit. That's it!

Here are the vizzes that I have included in the template.










January 17, 2018

Creating Maps With Linear Geometries in Tableau

, , , , No comments
This blog post is a couple months late as Tableau added support for linear geometries in version 10.4. So what are linear geometries? Essentially the are spatial files that are represented as a single line.

For example, let's say you have a series of locations that represent train stops. In Tableau you can draw the route by connecting the dots via the line shelf. This will result in marks for every station. If the route is a linear geometry (or linestring) instead, it is represented in Tableau as a single mark, meaning the viz will load much faster.

This is useful if you each point isn't important and you care more about the path itself. To help me understand how these work I downloaded shapefiles from the US Census, github and Transport for London. Each of these was a linear spatial file already, meaning I could connect with Tableau and go.

If I had a series of points, like the train stops example, I could use Alteryx to convert them to spatial points, create the path and export as a shapefile.

With that, here are a few example I've built using linear geometries and custom Mapbox maps. Note that the maps may be slow to load in Tableau Public. I'm not sure why because they're super fast in Desktop. Enjoy!

September 29, 2017

London Bus Routes - The Benefits of Linear Geometries in Tableau 10.4

, , , , , , , , , No comments
As I mentioned previously, one of my favorite new features in Tableau 10.4 is support for linear geometries. Think about it this way...let's say you want to plot a bus route. You can either:

  1. Plot each bus stop and connect the dots, or
  2. Create a single line for the entire path

What are the benefits of each?

Plotting Each Bus Stop

This method, which uses the Path shelf to connect the bus stops, allows you to indicate the name of each bus stop along the route. However, the drawback is that Tableau has to draw each bus stop. For London bus routes, this means drawing 28,270 marks then connecting each of those dots for the respective route.

Using Linear Geometry

This method, which creates a mark represented as a line for each bus route, results in only 736 marks (one for each route). That means you'll be significant performance gains. The drawback is that you lose the detail of each bus stop.

Which should you use?

It depends on the granularity you need. If plotting each point is important, the go with method A. If aggregating to the route is ok, then go with method B.

For this use case, I wanted to test both options. First, I went to the TFL website to download the data of every stop for every bus route (requires an account). The data comes as a CSV with eastings and northings, so I turned to my friend Alteryx for converting these into shapefiles.


I used this tip by Rob Suddaby from when he was in The Data School to convert the Eastings and Northings into a spatial object. From there it's simple to create either the linestrings for the entire route or points for each stop.

A quick Mapbox map added for context and we're done! One additional twist I added was to size the bus routes on the map based on the number of routes selected. This only works on the map with each individual stop. I'll be sharing this tip during my TC session in Vegas.

Have a play...enjoy!

September 27, 2017

Roads Across America - Tableau 10.4 Linear Geometries

, , , , 2 comments
Tableau 10.4 came out this week and with it support for linear geometry shapefiles. This feature makes visualizing routing data so much easier and faster. You no longer have to plot each point, change the mark type to line and play a game of connect the dots. Now, connect directly to your shapefiles and you're done!

Of course, I had to play with it. I went to the US Census website and downloaded shapefiles for every primary and secondary road for each State. With a couple clicks, I had an incredible map showing all of these roads. I wanted to highlight interstates, so I made them blue. 62,000 roads plotted with only a couple of clicks. Amazing!

Have a play with it yourself below! I'm excited about what boxes are now opened with this new capability.

July 25, 2017

Tableau Tip Tuesday: Using Alteryx, Mapbox & Tableau to Map Every Primary & Secondary Road in the US

, , , , , , No comments
Tableau 10.4 brings us support for linear geometry shapefiles. In this week's tip, I demonstrate how to create a map of all primary and secondary roads in the US by combining them in Alteryx and visualising them in Tableau. I also use a solid black US map with transparent oceans created in Mapbox for a better map experience.

Download the workbook I created to have a play yourself. Obviously this requires Tableau 10.4. Once Tableau Public is running 10.4, I'll upload it there as well.

In the video, you'll notice I used the Generalize tool in Alteryx. This took the file size from 500MB to 33MB, merely by aggregating up to 1 mile. Pretty sweet! Thanks to Chris Love for the tip!


Here's the how-to video. Enjoy!

Subscribe to: Posts ( Atom )