making sense of (big) data with visual analytics
TRANSCRIPT
Making Sense of (Big) Data
with Visual AnalyticsDr Kai Xu
Associate Professor in Data AnalyticsMiddlesex University, London, UK
[email protected] https://kaixu.me
Outline• What is Sensemaking• Why do we need Visual Analytics• Demo – SAVI: Social Analytics Visualisation• Demo – SenseMap: A ‘Map’ for Sensemaking
What is Sensemaking?• Making sense of data• Collecting, understanding, analysing, reasoning, and
making decisions
• It is something we do everyday:• Plan a holiday, buy a house, understand an illness, …• Defence, policing, investment, medical diagnosis, …
• How is it different from data analysis?• The task is usually not well defined
Example: what is the best camera for about £500?
What is the best camera for £500?
Pixel number
Sensor size
Image qualitychromatic aberration?!
Noise reduction
What does experts say?
Online reviews
What does my friend say? Smart phone
Compact
Full frame?
Micro 4/3?Sony RX100
Nikon D750Samsung Galaxy S7
What are the price?
How do I compare? Panasonic
LX100
Form factorModels
Camera LensAperture
This is usually what it looks like after one hour
• What is relevant and what is not?• Where is the information about image quality?• How to compare the models?• Where did I left off two days ago?• How do I explain to my wife?
Not just in browser
Making Sense of (Big) Data
Why IBM Watson or AlphaGo can’t do it• Watson is good at:
• Natural language processing, e.g., understand the Jeopardy! Questions• Find the (relevant) fact quickly
• However, the £500 camera task is• Every personal, Watson need all the information about me and understand it• No ‘best’ answer, so can’t just search it
• For AlphaGo, the Go game is very complex and difficult, but• The goal and rules are very well defined, and the results are easily measurable
• However, the £500 camera task is ill defined and not easily measurable• How many people have the knowledge and resource to build a deep neural network,
collect all the training data, and then train and tune it, just to find a camera?
Who is the best chess player in the world?• Deep Blue, was in 1997• Currently, probably a human-machine
team• And the two people on the team are not
even professional chess players
• The power of integrating the complementary strength of human and machine
Visual Analytics = Human + Computing Intelligence
Visualisation
Data Analysis Interaction
Information RetrievalMachine Learning
Data Mining
Information VisualisationScientific Visualisation
Computer Graphics
Human-Computer InteractionCognitive Psychology
Perception
Some work in the last five years
Example - SAVI: Social Analytics Visualisation• IEEE Visual Analytics Science & Technology (VAST) Challenge• Provide dataset and analysis tasks• Entry: visual analytics systems• Leading research groups and companies
• VAST Challenge 2014 – Mini Challenge 3• Data: tweets• Task: detect and describe a crime
The Data
The Final Findings
• Still a long time before AI can do such sensemaking• Difficult for human, too: almost impossible
without the tool
• Human leads, the tool supports• The tool does not provide answer, • Reveal pattern, help with organisation and
reasoning, and many more
A ‘Map’ of Sensemaking• Sensemaking is kind of like exploring a maze …• What may be helpful is something like this …
SenseMap – A ‘Map’ for Online Sensemaking
Browser enhancement
History Map
Knowledge Map
Takeaway Messages• Sensemaking is how people understand, reason, and make decisions
with data• It is important to Big Data, but there is limited support available• Visual Analytics combines data visualisation with analytics• A promising approach for sensemaking support
More details about SAVI and SenseMap: http://vis4sense.github.io/