visualising montioring and evaluation data

Data VisualizationTechnology for Monitoring and

Evaluation25th September 2015

Introduction

• Kwantu is a Cape Town social enterprise trying to make M&E easier

• We make affordable mobile and web apps to manage and monitor activities

• Includes integrated tools for analysis and reporting

Introduction

• This presentation was given to the TechChange Technology for Monitoring and Evaluation Diploma

• See https://www.techchange.org/online-courses/me-diploma-track/ for details

Overview

• Why visualize data?• Where to start?• Which tools?• Our approach• Our technology choices

Why visualize data?

• Unlock valuable insights• Efficient way to analyze large datasets• Easier to see patterns in the data• Can help to predict trends• Less intimidating• Great way to communicate with busy

people

What is data visualization?

Data visualization is the presentation of data in a pictorial or graphical

format

What is data visualization?

• Maps• Charts• Pictures• Infographics (combine several

formats to tell a story)

What is data visualization?

http://www.farmingfirst.org/women_infographic/

What is data visualization?

https://www.usaid.gov/infographics/50th/mobile-phones-tacking-poverty/

Interactive visualizations

A picture is worth a thousand words. An interface is worth a thousand

pictures.

- Ben Shneiderman

Interactive visualizations

• Goes one step further• Choose parameters that affect which

data you see and how it is visualised

Interactive visualizations

http://www.informationisbeautiful.net/visualizations/snake-oil-superfoods/

Interactive visualizations

https://docs.google.com/spreadsheets/d/1s4nak1vpPWRUm-RPdp-HeCfMIiuw1huS87qaTbGMOwY/pubchart?oid=8&format=interactive

Where to start?

• Clarify the questions you want to answer

• Start simple• Focus on what is really important• (your indicators?)

Where to start?

• Understand your data• Do you have the data you need to

answer this question?• How big is your dataset?• Qualitative or quantitative?

Where to start?

• Does it need to be:• Cleaned?• Reviewed?• Aggregated?• Transformed?• Coded?

Where to start?

Need help?

• Fix problems - Open Refine – www.openrefine.org

• Prevent problems – See our approach to data collection in a later slide

Where to start?

• Which type of visualization can best help my target audience answer the question?

• What type of visual will convey the information in the simplest and clearest way to your audience?

Tools - Scenario one – I’m new to data visualisation and on a low budget

• Take a look at:

• Excel• Google Spreadsheets• Google Map Maker• MapAList

Tools - Scenario two – I’ve got some experience and want to delve deeper

• Take a look at:

• Pentaho Business Analytics• Jasper Reports• Google Fusion Tablets• Tableau• Bamboo• R

Tools - Scenario two – I’ve got some experience and want to delve deeper

• These tools are designed to create a data warehouse (database) of your data

• They include tools to analyse and visualise data

• They can help you create interactive reports, charts and maps

Tools - Scenario three – I’m working with developers and want to extend our current

M&E systems

• Take a look at:

• D3• Kibana• Vega• Processing.js• Gephi (network analysis)

Tools - Scenario three – I’m working with developers and want to extend our current

M&E systems

• These libraries and frameworks are designed to integrate with other web applications

• They include pre-developed charts, maps and other visualisations that make it easy for a developer to display data visually

• They are typically also interactive

Our approach

• Our aim is to help partners implement real-time (and interactive) data visualizations

• Aim to avoid need to manually process data first

• This means integrating measures to validate and code the data from the outset

Our approach

• We do this by collecting data operationally, as part of day to day activities

• Data processing is built into the activity

• Caution! Avoid use of forms that collect pre-aggregated data sets

• Collect data at lowest level possible

Our approach

• This avoids the need for data reviews and cleaning

• It makes it possible to implement real-time reports and visualizations

Our technology choices

• Form builder tool:• Create forms for use on web or mobile

(offline) that collect data at the activity level

• Builds a ‘data dictionary’ for reporting• Allows for relational linkages between

forms• Based on Atom XML standard

• Collect well validated and coded data

Our technology choices

• Taxonomy tool:• Create taxonomies to help filter and

cross-reference data in your forms• Closed responses to questions• Gender• Age range• Location

• Standardize responses and filters

Our technology choices

• Query builder tool:• Select any form(s)• Filter by date range, taxonomy or

workflow step etc.• Create a dynamic data feed

• Separates the data layer from presentation layer

Our technology choices

• Workflow engine:• Guides staff through implementation

steps• Which forms to complete and when• Review steps to check data as it’s

being collected• Validate data operationally as

activities are implemented

Our technology choices

• Report builder: • Uses open source Jasper engine and also

Google Drive API to export data• Map fields from your data feed to your own

names• Select which data to show on your report• Export to Excel or PDF

• Enables you to answer questions as they arise

Our technology choices

• Visualizations: (Q1 2016) • Integrate with open source Kibana

framework• Has built in tools to create visualizations• Updates in real-time• Can include data from other databases

(integrate with other systems)• Create new visualizations to answer

questions as they arise

Do you need help with data visualization?

Rob Worthingtonrob@kwantu.netwww.kwantu.net