data visualisation workshop (govhack brisbane 2014)

Open Knowledge Brisbane Data Visualisation Workshop

Anna Gerber

Data Visualisation Workshop

GovHack Brisbane 2014

•  July 11 – 13 2014 •  The Cube, Level 6, P Block, QUT Gardens

Point Campus •  Register at

http://www.govhack.org/register-2014/ •  Tickets are almost sold out •  We need volunteers and mentors

Data Visualisation Workshop

GETTING STARTED

Data Visualisation Workshop

"Communica*ng  complex  data  and  intricate  ideas  through  clear  and  beau*ful  visual  storytelling."  

-­‐  Tobias  Sturt  –  Design  Manager,  The  Guardian  Digital  Agency  

Data Visualisation Workshop

If a picture paints 1000 words…

… what do you want your visualisation to say?

–  Insightful vs Infotainment – Exploratory vs Explanatory

•  What data will you use? •  Does the data need processing? •  What type(s) of visualisations will you use? •  What tools can you use to build it?

Data Visualisation Workshop

Data-Driven Journalism

Data Visualisation Workshop

http://en.wikipedia.org/wiki/File:Data_driven_journalism_process.jpg

DATA

Data Visualisation Workshop

Where to find Open Gov Data?

•  Open Data Portals: – http://data.gov.au/ – http://data.qld.gov.au/ – http://data.brisbane.qld.gov.au/

•  Clearing houses (reports) •  See http://www.govhack.org/data/ for more

Data Visualisation Workshop

Preparing the Data

•  Processing for machine readability – Data extraction, scraping –  Splitting / parsing

•  Cleaning: – Normalizing values / units – Correcting errors

•  Filtering •  Geocoding •  Connecting / mapping

Data Visualisation Workshop

Data Preparation Tools

•  DataWrangler http://vis.stanford.edu/wrangler/

•  OpenRefine (formerly Google Refine)http://openrefine.org/

•  CSVKit https://github.com/onyxfish/csvkit

•  Tabula http://source.mozillaopennews.org/en-US/articles/introducing-tabula/

Data Visualisation Workshop

Data Wrangler

Online  Data Visualisation Workshop

Open Refine

App  Data Visualisation Workshop

See  hEps://github.com/OpenRefine/OpenRefine/wiki/Recipes  

Geocoding

•  If data does not already include geographic co-ordinates it may be necessary to geocode (i.e. look up based on address, locality etc)

•  Google, Mapquest provide free services •  http://opensas.wordpress.com/

2013/06/30/using-openrefine-to-geocode-your-data-using-google-and-openstreetmap-api/

Data Visualisation Workshop

CREATING MAPS USING CARTODB VISUALISATION

Data Visualisation Workshop

CartoDB

•  Free and premium accounts available: http://cartodb.com/

•  Open Source, so you can host your own https://github.com/CartoDB/cartodb

•  Powered by PostgreSQL / PostGIS

Data Visualisation Workshop

CartoDB Dashboard

Data Visualisation Workshop

Drag  and  drop  CSV,  Excel,  KML  or  GIS  files  directly  into  the  browser  to  import  

Using  hEp://data.brisbane.qld.gov.au/index.php/dataset/public-­‐drinking-­‐fountain-­‐taps/    

Inspect & Edit Data

Data Visualisation Workshop

Customise Info Window

Data Visualisation Workshop

Select Visualisation Type

Data Visualisation Workshop

Supported Map Visualisations

Data Visualisation Workshop

Customise Category Icons

Data Visualisation Workshop

Customise CSS

Data Visualisation Workshop

Select Map Tiles

Data Visualisation Workshop

Customise Legend

Data Visualisation Workshop

Filtering

Data Visualisation Workshop

Share Visualisation

Data Visualisation Workshop

Public Visualisation

Data Visualisation Workshop

DEMO

Data Visualisation Workshop

Multiple Layers

Data Visualisation Workshop

Combining Datasets •  Combine Tables to:

–  join a dataset representing regional boundaries with a dataset that aggregates data by regions (e.g. Census data)

– mashup data from different sources •  CartoDB supports:

–  Joining on shared column via GUI –  Spatial joins via GUI – Creating new tables from custom

SQL queries

Data Visualisation Workshop

Filtering via Queries

Data Visualisation Workshop

Use SQL to filter values and save to new dataset

Edit Geometries

Data Visualisation Workshop

Animating Maps Using Torque

•  For example, display changing values over time

•  Only available for point-based data •  You can force polygons to points via SQL e.g.

SELECT ST_Centroid(the_geom) FROM {table_name}

Data Visualisation Workshop

Select Changing Column

Data Visualisation Workshop

Specify Aggregation Function

Data Visualisation Workshop

Custom CartoCSS for value ranges

Data Visualisation Workshop

CartoDB API

•  SQL API and CartoDB.js •  Allows development of custom web apps or

visualisations, e.g. in combination with other JavaScript libraries

•  http://developers.cartodb.com/documentation/apis-overview.html

Data Visualisation Workshop

More Map-Based Visualisation Tools

•  Point & Click •  Open Heat Map

http://www.openheatmap.com/ •  Google Maps and Fusion Tables

https://drive.google.com/ •  MapBox TileMill

https://www.mapbox.com/tilemill/ •  ColorBrewer (colour scheme generator)

http://colorbrewer2.com/

Data Visualisation Workshop

More Map-Based Visualisation Tools

•  Development libraries – Open Layers

http://openlayers.org/ – Leaflet

http://leafletjs.com/ – GeoExt

http://www.geoext.org/

Data Visualisation Workshop

CREATING WEB-BASED CHARTS VISUALISATION

Data Visualisation Workshop

Point & Click Chart Tools

•  Google Charts and Google Fusion Tables http://imagecharteditor.appspot.com/

•  Infogr.am http://infogr.am/

•  Data Wrapper http://datawrapper.de/

•  Tableau Public http://www.tableausoftware.com/public/

Data Visualisation Workshop

Data-Driven Documents (d3)

•  http://d3js.org/ •  Javascript Library for binding data to the

Document Object Model (DOM) and applying transformations

•  Not just for Charts e.g. 1900+ d3 visualisation examples: http://christopheviau.com/d3list/gallery.html

Data Visualisation Workshop

Why d3?

•  Interactive web-based charts •  Based on web standards (SVG, CSS) •  Works in modern browsers, including mobile HOWEVER •  Somewhat abstract •  Steep learning curve •  Tutorials:

http://alignedleft.com/tutorials/d3

Data Visualisation Workshop

Chart Libraries Built on d3

•  Simplify creating commonly used charts with d3: – NVD3

http://nvd3.org/ – d3.Chart

http://misoproject.com/d3-chart/ – Rickshaw

http://code.shutterstock.com/rickshaw/

Data Visualisation Workshop

Rickshaw Example

•  Example data: Average rainfall in mm per year per state

Anna Gerber Intro to JavaScript

Year NSW & ACT NT QLD SA TAS VIC WA 2005 498 477 478 206 1250 616 306

2004 493 637 610 214 1223 578 463

2003 484 686 518 260 1227 611 388

Data  source:hEp://www.water.gov.au/WaterAvailability/Wha*sourtotalwaterresource/Rainfalldistribu*on/index.aspx?Menu=Level1_3_1_2  

Rickshaw Example

Data Visualisation Workshop

Rickshaw Chart JS var chartData = [ { name: 'NSW & ACT', data: [ { x: 2003, y: 400 }, { x: 2004, y: 200 }, { x: 2005, y: 0 } ], color: 'steelblue' }, { name: 'NT', data: [ { x: 2003, y: 686 }, { x: 2004, y: 637 }, { x: 2005, y: 477 } ], color: 'orange' } // … ]

Data Visualisation Workshop

//  Use  the  Rickshaw  Library  to  create  a  graph  var  graph  =  new  Rickshaw.Graph(  {          //  aEach  the  graph  to  the  chart  element          element:  document.querySelector("#chart"),          //  render  as  a  stacked  area  chart          renderer:  'area',            series:  chartData  }  );  graph.render();  

Code: http://jsfiddle.net/zN8Eh/5/

Data Visualisation Workshop

NVD3  Examples  

Combine d3 + CartoDB

•  D3 can be used to draw maps using GeoJSON from the CartoDB API e.g. to overlay charts on maps

•  Examples: Bubble Map:http://bl.ocks.org/andrewxhill/4448162 Earthquakes visualisation:http://bl.ocks.org/andrewxhill/4455569

Data Visualisation Workshop

Data Visualisation Workshop

hEp://www.visual-­‐literacy.org/periodic_table/periodic_table.html    

Resources •  Videos:

–  http://datajournalism.stanford.edu/ –  http://teachingtosee.org/film/TeachingToSee.html

•  Podcast: –  http://datastori.es/

•  Websites: –  http://www.visualizing.org/ –  http://visual.ly/ –  http://flowingdata.com/ –  http://www.visualcomplexity.com/vc/ –  http://www.informationisbeautiful.net/

Data Visualisation Workshop