data visualisation workshop (govhack brisbane 2014)
DESCRIPTION
TRANSCRIPT
Open Knowledge Brisbane Data Visualisation Workshop
Anna Gerber
Data Visualisation Workshop
GovHack Brisbane 2014
• July 11 – 13 2014 • The Cube, Level 6, P Block, QUT Gardens
Point Campus • Register at
http://www.govhack.org/register-2014/ • Tickets are almost sold out • We need volunteers and mentors
Data Visualisation Workshop
GETTING STARTED
Data Visualisation Workshop
"Communica*ng complex data and intricate ideas through clear and beau*ful visual storytelling."
-‐ Tobias Sturt – Design Manager, The Guardian Digital Agency
Data Visualisation Workshop
If a picture paints 1000 words…
… what do you want your visualisation to say?
– Insightful vs Infotainment – Exploratory vs Explanatory
• What data will you use? • Does the data need processing? • What type(s) of visualisations will you use? • What tools can you use to build it?
Data Visualisation Workshop
Data-Driven Journalism
Data Visualisation Workshop
http://en.wikipedia.org/wiki/File:Data_driven_journalism_process.jpg
DATA
Data Visualisation Workshop
Where to find Open Gov Data?
• Open Data Portals: – http://data.gov.au/ – http://data.qld.gov.au/ – http://data.brisbane.qld.gov.au/
• Clearing houses (reports) • See http://www.govhack.org/data/ for more
Data Visualisation Workshop
Preparing the Data
• Processing for machine readability – Data extraction, scraping – Splitting / parsing
• Cleaning: – Normalizing values / units – Correcting errors
• Filtering • Geocoding • Connecting / mapping
Data Visualisation Workshop
Data Preparation Tools
• DataWrangler http://vis.stanford.edu/wrangler/
• OpenRefine (formerly Google Refine)http://openrefine.org/
• CSVKit https://github.com/onyxfish/csvkit
• Tabula http://source.mozillaopennews.org/en-US/articles/introducing-tabula/
Data Visualisation Workshop
Data Wrangler
Online Data Visualisation Workshop
Open Refine
App Data Visualisation Workshop
See hEps://github.com/OpenRefine/OpenRefine/wiki/Recipes
Geocoding
• If data does not already include geographic co-ordinates it may be necessary to geocode (i.e. look up based on address, locality etc)
• Google, Mapquest provide free services • http://opensas.wordpress.com/
2013/06/30/using-openrefine-to-geocode-your-data-using-google-and-openstreetmap-api/
Data Visualisation Workshop
CREATING MAPS USING CARTODB VISUALISATION
Data Visualisation Workshop
CartoDB
• Free and premium accounts available: http://cartodb.com/
• Open Source, so you can host your own https://github.com/CartoDB/cartodb
• Powered by PostgreSQL / PostGIS
Data Visualisation Workshop
CartoDB Dashboard
Data Visualisation Workshop
Drag and drop CSV, Excel, KML or GIS files directly into the browser to import
Using hEp://data.brisbane.qld.gov.au/index.php/dataset/public-‐drinking-‐fountain-‐taps/
Inspect & Edit Data
Data Visualisation Workshop
Customise Info Window
Data Visualisation Workshop
Select Visualisation Type
Data Visualisation Workshop
Supported Map Visualisations
Data Visualisation Workshop
Customise Category Icons
Data Visualisation Workshop
Customise CSS
Data Visualisation Workshop
Select Map Tiles
Data Visualisation Workshop
Customise Legend
Data Visualisation Workshop
Filtering
Data Visualisation Workshop
Share Visualisation
Data Visualisation Workshop
Public Visualisation
Data Visualisation Workshop
DEMO
Data Visualisation Workshop
Multiple Layers
Data Visualisation Workshop
Combining Datasets • Combine Tables to:
– join a dataset representing regional boundaries with a dataset that aggregates data by regions (e.g. Census data)
– mashup data from different sources • CartoDB supports:
– Joining on shared column via GUI – Spatial joins via GUI – Creating new tables from custom
SQL queries
Data Visualisation Workshop
Filtering via Queries
Data Visualisation Workshop
Use SQL to filter values and save to new dataset
Edit Geometries
Data Visualisation Workshop
Animating Maps Using Torque
• For example, display changing values over time
• Only available for point-based data • You can force polygons to points via SQL e.g.
SELECT ST_Centroid(the_geom) FROM {table_name}
Data Visualisation Workshop
Select Changing Column
Data Visualisation Workshop
Specify Aggregation Function
Data Visualisation Workshop
Custom CartoCSS for value ranges
Data Visualisation Workshop
CartoDB API
• SQL API and CartoDB.js • Allows development of custom web apps or
visualisations, e.g. in combination with other JavaScript libraries
• http://developers.cartodb.com/documentation/apis-overview.html
Data Visualisation Workshop
More Map-Based Visualisation Tools
• Point & Click • Open Heat Map
http://www.openheatmap.com/ • Google Maps and Fusion Tables
https://drive.google.com/ • MapBox TileMill
https://www.mapbox.com/tilemill/ • ColorBrewer (colour scheme generator)
http://colorbrewer2.com/
Data Visualisation Workshop
More Map-Based Visualisation Tools
• Development libraries – Open Layers
http://openlayers.org/ – Leaflet
http://leafletjs.com/ – GeoExt
http://www.geoext.org/
Data Visualisation Workshop
CREATING WEB-BASED CHARTS VISUALISATION
Data Visualisation Workshop
Point & Click Chart Tools
• Google Charts and Google Fusion Tables http://imagecharteditor.appspot.com/
• Infogr.am http://infogr.am/
• Data Wrapper http://datawrapper.de/
• Tableau Public http://www.tableausoftware.com/public/
Data Visualisation Workshop
Data-Driven Documents (d3)
• http://d3js.org/ • Javascript Library for binding data to the
Document Object Model (DOM) and applying transformations
• Not just for Charts e.g. 1900+ d3 visualisation examples: http://christopheviau.com/d3list/gallery.html
Data Visualisation Workshop
Why d3?
• Interactive web-based charts • Based on web standards (SVG, CSS) • Works in modern browsers, including mobile HOWEVER • Somewhat abstract • Steep learning curve • Tutorials:
http://alignedleft.com/tutorials/d3
Data Visualisation Workshop
Chart Libraries Built on d3
• Simplify creating commonly used charts with d3: – NVD3
http://nvd3.org/ – d3.Chart
http://misoproject.com/d3-chart/ – Rickshaw
http://code.shutterstock.com/rickshaw/
Data Visualisation Workshop
Rickshaw Example
• Example data: Average rainfall in mm per year per state
Anna Gerber Intro to JavaScript
Year NSW & ACT NT QLD SA TAS VIC WA 2005 498 477 478 206 1250 616 306
2004 493 637 610 214 1223 578 463
2003 484 686 518 260 1227 611 388
Data source:hEp://www.water.gov.au/WaterAvailability/Wha*sourtotalwaterresource/Rainfalldistribu*on/index.aspx?Menu=Level1_3_1_2
Rickshaw Example
Data Visualisation Workshop
Rickshaw Chart JS var chartData = [ { name: 'NSW & ACT', data: [ { x: 2003, y: 400 }, { x: 2004, y: 200 }, { x: 2005, y: 0 } ], color: 'steelblue' }, { name: 'NT', data: [ { x: 2003, y: 686 }, { x: 2004, y: 637 }, { x: 2005, y: 477 } ], color: 'orange' } // … ]
Data Visualisation Workshop
// Use the Rickshaw Library to create a graph var graph = new Rickshaw.Graph( { // aEach the graph to the chart element element: document.querySelector("#chart"), // render as a stacked area chart renderer: 'area', series: chartData } ); graph.render();
Code: http://jsfiddle.net/zN8Eh/5/
Data Visualisation Workshop
NVD3 Examples
Combine d3 + CartoDB
• D3 can be used to draw maps using GeoJSON from the CartoDB API e.g. to overlay charts on maps
• Examples: Bubble Map:http://bl.ocks.org/andrewxhill/4448162 Earthquakes visualisation:http://bl.ocks.org/andrewxhill/4455569
Data Visualisation Workshop
Data Visualisation Workshop
hEp://www.visual-‐literacy.org/periodic_table/periodic_table.html
Resources • Videos:
– http://datajournalism.stanford.edu/ – http://teachingtosee.org/film/TeachingToSee.html
• Podcast: – http://datastori.es/
• Websites: – http://www.visualizing.org/ – http://visual.ly/ – http://flowingdata.com/ – http://www.visualcomplexity.com/vc/ – http://www.informationisbeautiful.net/
Data Visualisation Workshop