![Page 1: Exploring Big Data using Visual Analytics · 2018. 12. 12. · 1 Exploring Big Data using Visual Analytics Daniel A. Keim Data Analysis and Information Visualization Group University](https://reader035.vdocuments.us/reader035/viewer/2022071415/6110600e7c0af9698e153869/html5/thumbnails/1.jpg)
1
Exploring Big Data using Visual Analytics
Daniel A. Keim Data Analysis and Information
Visualization Group University of Konstanz, Germany
Data Mining for Business Intelligence, Beer Sheva, Israel April 24, 2014
![Page 2: Exploring Big Data using Visual Analytics · 2018. 12. 12. · 1 Exploring Big Data using Visual Analytics Daniel A. Keim Data Analysis and Information Visualization Group University](https://reader035.vdocuments.us/reader035/viewer/2022071415/6110600e7c0af9698e153869/html5/thumbnails/2.jpg)
2
Automated Analysis
Good News
Automated Analysis of big data works!
(under certain preconditions)
Automated Analysis
Bad News
The preconditions are rarely met!
Preconditions: –! Data is clearly structured –! Data semantics is well-defined –! Data is complete, correct, and
not changing over time AND
–! Problem is well-defined
![Page 3: Exploring Big Data using Visual Analytics · 2018. 12. 12. · 1 Exploring Big Data using Visual Analytics Daniel A. Keim Data Analysis and Information Visualization Group University](https://reader035.vdocuments.us/reader035/viewer/2022071415/6110600e7c0af9698e153869/html5/thumbnails/3.jpg)
3
Automated Analysis
•! Network Security –! Data: IP Flows –! Task: Detection of Novel Viruses
•! Fraud Detection –! Data: Credit Card (or Phone Call) Data –! Task: Detection of Fraud
•! Business Analytics –! Data: Customer Records –! Task: Define Customer Target Groups
•! Molecular Biology –! Data: Patient DNA Records –! Task: Functional Root Cause Analysis for an Illness
The Role of Visualization
Visualization is needed in addition to analytics to
•! identify the structure (based on user knowledge)
•! bridge the semantic gap (bring in user knowledge)
•! help with incomplete or incorrect and changing data
•! understand the problem, generate hypotheses and define the problem
•! and steer the analysis process in dealing with massive data (local optimization)
![Page 4: Exploring Big Data using Visual Analytics · 2018. 12. 12. · 1 Exploring Big Data using Visual Analytics Daniel A. Keim Data Analysis and Information Visualization Group University](https://reader035.vdocuments.us/reader035/viewer/2022071415/6110600e7c0af9698e153869/html5/thumbnails/4.jpg)
4
Visualization
Bad News
Visualization of big data
does not work!
![Page 5: Exploring Big Data using Visual Analytics · 2018. 12. 12. · 1 Exploring Big Data using Visual Analytics Daniel A. Keim Data Analysis and Information Visualization Group University](https://reader035.vdocuments.us/reader035/viewer/2022071415/6110600e7c0af9698e153869/html5/thumbnails/5.jpg)
5
Search
Planning Diagnosis Prediction
Logic
Data Storage
Computing Power
General Knowledge Perception Creativity
Visual Analytics
![Page 6: Exploring Big Data using Visual Analytics · 2018. 12. 12. · 1 Exploring Big Data using Visual Analytics Daniel A. Keim Data Analysis and Information Visualization Group University](https://reader035.vdocuments.us/reader035/viewer/2022071415/6110600e7c0af9698e153869/html5/thumbnails/6.jpg)
6
"Computers are incredibly fast, accurate, and stupid; humans are incredibly slow, inaccurate, and
brilliant; together they are powerful beyond imagination."
attributed to Albert Einstein
Visual Analytics
www.visual-analytics.eu
Roadmap from the VisMaster EU Project
Video
![Page 7: Exploring Big Data using Visual Analytics · 2018. 12. 12. · 1 Exploring Big Data using Visual Analytics Daniel A. Keim Data Analysis and Information Visualization Group University](https://reader035.vdocuments.us/reader035/viewer/2022071415/6110600e7c0af9698e153869/html5/thumbnails/7.jpg)
7
Outline
•! The Role of Visualization and Analytics in Exploring Big Data –! Why automated analysis does not work
–! When visual exploration can help
•! Visual Analytics Examples –! Network Security
–! Document Analysis
–! Financial Analysis
–! Molecular Biology
•! Visual Analytics Perspectives
Visual Network Analysis
data: rzstat3 date: 29 Nov 2005 measure: outgoing connections
Hierarchy: •! Continents •! Countries •! Autonomous Systems •! Networks
![Page 8: Exploring Big Data using Visual Analytics · 2018. 12. 12. · 1 Exploring Big Data using Visual Analytics Daniel A. Keim Data Analysis and Information Visualization Group University](https://reader035.vdocuments.us/reader035/viewer/2022071415/6110600e7c0af9698e153869/html5/thumbnails/8.jpg)
8
NetFlow Records
Visual Analytics of Network Data
Honeypot
Intrusion Detection
Alerts
NetFlow Records
Aggr. NetFlows
IP Packet Stream Processing Processing
Global Internet
University Network Gateway
![Page 9: Exploring Big Data using Visual Analytics · 2018. 12. 12. · 1 Exploring Big Data using Visual Analytics Daniel A. Keim Data Analysis and Information Visualization Group University](https://reader035.vdocuments.us/reader035/viewer/2022071415/6110600e7c0af9698e153869/html5/thumbnails/9.jpg)
9
Visual Network Analysis
NetFlow Summary
IDS Summary
Summary of Input Data Netflow Visualization
Network Visual Analytics
Detail Analysis
Host Details
Netflow Details
![Page 10: Exploring Big Data using Visual Analytics · 2018. 12. 12. · 1 Exploring Big Data using Visual Analytics Daniel A. Keim Data Analysis and Information Visualization Group University](https://reader035.vdocuments.us/reader035/viewer/2022071415/6110600e7c0af9698e153869/html5/thumbnails/10.jpg)
10
![Page 11: Exploring Big Data using Visual Analytics · 2018. 12. 12. · 1 Exploring Big Data using Visual Analytics Daniel A. Keim Data Analysis and Information Visualization Group University](https://reader035.vdocuments.us/reader035/viewer/2022071415/6110600e7c0af9698e153869/html5/thumbnails/11.jpg)
11
Uni Konstanz Control Room
Outline
•! The Role of Visualization and Analytics in Exploring Big Data –! Why automated analysis does not work
–! When visual exploration can help
•! Visual Analytics Examples –! Network Security
–! Document Analysis
–! Financial Analysis
–! Molecular Biology
•! Visual Analytics Perspectives
![Page 12: Exploring Big Data using Visual Analytics · 2018. 12. 12. · 1 Exploring Big Data using Visual Analytics Daniel A. Keim Data Analysis and Information Visualization Group University](https://reader035.vdocuments.us/reader035/viewer/2022071415/6110600e7c0af9698e153869/html5/thumbnails/12.jpg)
12
Customer Feedback Analysis
© 2012 ACM
Customer Feedback Analysis
![Page 13: Exploring Big Data using Visual Analytics · 2018. 12. 12. · 1 Exploring Big Data using Visual Analytics Daniel A. Keim Data Analysis and Information Visualization Group University](https://reader035.vdocuments.us/reader035/viewer/2022071415/6110600e7c0af9698e153869/html5/thumbnails/13.jpg)
13
Customer Feedback Analysis
Customer Feedback Analysis
![Page 14: Exploring Big Data using Visual Analytics · 2018. 12. 12. · 1 Exploring Big Data using Visual Analytics Daniel A. Keim Data Analysis and Information Visualization Group University](https://reader035.vdocuments.us/reader035/viewer/2022071415/6110600e7c0af9698e153869/html5/thumbnails/14.jpg)
14
Customer Feedback Analysis
Outline
•! The Role of Visualization and Analytics in Exploring Big Data –! Why automated analysis does not work
–! When visual exploration can help
•! Visual Analytics Examples –! Network Security
–! Document Analysis
–! Financial Analysis
–! High-dim. & Subspace Analysis
–! Molecular Biology
•! Visual Analytics Perspectives
![Page 15: Exploring Big Data using Visual Analytics · 2018. 12. 12. · 1 Exploring Big Data using Visual Analytics Daniel A. Keim Data Analysis and Information Visualization Group University](https://reader035.vdocuments.us/reader035/viewer/2022071415/6110600e7c0af9698e153869/html5/thumbnails/15.jpg)
15
Future Visual Analytics Topics
Technical Challenges –! Streaming Data –! Complex Structure –! Data with Uncertainty –!!
Applications –! E-Humanities –! Molecular Biology –! Multimedia –!!
Evaluation
Visual Analytics
Data
Knowledge
DM-Algorithm
Result
Visualization of the data
Preceding Visualization
–!Visualization of –!the result
Data
DM-Algorithm
Knowledge
Visualization of the result
Result
Subsequent Visualization
Result
DM-Algorithm step 1
Data
Knowledge
DM-Algorithm step n
Visualization + Interaction
Visual Analytics
![Page 16: Exploring Big Data using Visual Analytics · 2018. 12. 12. · 1 Exploring Big Data using Visual Analytics Daniel A. Keim Data Analysis and Information Visualization Group University](https://reader035.vdocuments.us/reader035/viewer/2022071415/6110600e7c0af9698e153869/html5/thumbnails/16.jpg)
16
Questions?
infovis.uni-konstanz.de
“All truths are easy to understand once they are discovered; the point is to discover them.”
Galileo Galile (1564-1642)
Conclusion