data & visual analytics · logistics make sure you’re in the right piazza! ... this course...
TRANSCRIPT
![Page 1: Data & Visual Analytics · Logistics Make sure you’re in the right Piazza! ... This course helps you learn some important skills. Collection Cleaning Integration Visualization Analysis](https://reader033.vdocuments.us/reader033/viewer/2022042218/5ec4965fa26ed301f878a210/html5/thumbnails/1.jpg)
http://poloclub.gatech.edu/cse6242CSE6242 / CX4242
Data & Visual Analytics
Duen Horng (Polo) ChauAssociate Professor, College of Computing Associate Director, MS AnalyticsMachine Learning Area Leader, College of Computing Georgia Tech
![Page 2: Data & Visual Analytics · Logistics Make sure you’re in the right Piazza! ... This course helps you learn some important skills. Collection Cleaning Integration Visualization Analysis](https://reader033.vdocuments.us/reader033/viewer/2022042218/5ec4965fa26ed301f878a210/html5/thumbnails/2.jpg)
Google “Polo Chau” (easy to find; only one in the world)
![Page 3: Data & Visual Analytics · Logistics Make sure you’re in the right Piazza! ... This course helps you learn some important skills. Collection Cleaning Integration Visualization Analysis](https://reader033.vdocuments.us/reader033/viewer/2022042218/5ec4965fa26ed301f878a210/html5/thumbnails/3.jpg)
How to address Polo?Grammatically correct
Prof. Chau
Dr. Chau
Grammatically incorrect, but popular
Prof. Polo
Dr. Polo
![Page 4: Data & Visual Analytics · Logistics Make sure you’re in the right Piazza! ... This course helps you learn some important skills. Collection Cleaning Integration Visualization Analysis](https://reader033.vdocuments.us/reader033/viewer/2022042218/5ec4965fa26ed301f878a210/html5/thumbnails/4.jpg)
Course Registration
CSE 6242 A208/220 seats filled0/65 waitlist slots taken
CSE 6242 Q (distance-learning): 3 students
This class room seats 300. If you are on the waitlist, please wait for seats to released (some students typically “drop” after today). I’ll also increase the cap to close to 300.
![Page 5: Data & Visual Analytics · Logistics Make sure you’re in the right Piazza! ... This course helps you learn some important skills. Collection Cleaning Integration Visualization Analysis](https://reader033.vdocuments.us/reader033/viewer/2022042218/5ec4965fa26ed301f878a210/html5/thumbnails/5.jpg)
Course TAs Be very very nice to them!
Office hours and locations (TBD) on course homepagehttps://poloclub.github.io/cse6242-2020spring-campus/
Sushanto PraharajShrishti Aastha AgrawalApurv PriyamPriyank MadriaSaifil Nizarali Momin
![Page 7: Data & Visual Analytics · Logistics Make sure you’re in the right Piazza! ... This course helps you learn some important skills. Collection Cleaning Integration Visualization Analysis](https://reader033.vdocuments.us/reader033/viewer/2022042218/5ec4965fa26ed301f878a210/html5/thumbnails/7.jpg)
7
We work with (really) large data.
![Page 8: Data & Visual Analytics · Logistics Make sure you’re in the right Piazza! ... This course helps you learn some important skills. Collection Cleaning Integration Visualization Analysis](https://reader033.vdocuments.us/reader033/viewer/2022042218/5ec4965fa26ed301f878a210/html5/thumbnails/8.jpg)
8
Internet50 Billion Web Pages
www.worldwidewebsize.com www.opte.org
![Page 9: Data & Visual Analytics · Logistics Make sure you’re in the right Piazza! ... This course helps you learn some important skills. Collection Cleaning Integration Visualization Analysis](https://reader033.vdocuments.us/reader033/viewer/2022042218/5ec4965fa26ed301f878a210/html5/thumbnails/9.jpg)
9
Facebook2 Billion Users
![Page 10: Data & Visual Analytics · Logistics Make sure you’re in the right Piazza! ... This course helps you learn some important skills. Collection Cleaning Integration Visualization Analysis](https://reader033.vdocuments.us/reader033/viewer/2022042218/5ec4965fa26ed301f878a210/html5/thumbnails/10.jpg)
10
Citation Network
www.scirus.com/press/html/feb_2006.html#2 Modified from well-formed.eigenfactor.org
250 Million Articles
![Page 11: Data & Visual Analytics · Logistics Make sure you’re in the right Piazza! ... This course helps you learn some important skills. Collection Cleaning Integration Visualization Analysis](https://reader033.vdocuments.us/reader033/viewer/2022042218/5ec4965fa26ed301f878a210/html5/thumbnails/11.jpg)
TwitterWho-follows-whom (500 million users)
Who-buys-what (120 million users)
cellphone networkWho-calls-whom (100 million users)
Protein-protein interactions200 million possible interactions in human genome
11
Many More
Sources: www.selectscience.net www.phonedog.com www.mediabistro.com www.practicalecommerce.com/
![Page 12: Data & Visual Analytics · Logistics Make sure you’re in the right Piazza! ... This course helps you learn some important skills. Collection Cleaning Integration Visualization Analysis](https://reader033.vdocuments.us/reader033/viewer/2022042218/5ec4965fa26ed301f878a210/html5/thumbnails/12.jpg)
12
“Big Data” Analyzed
DATA ! INSIGHTS
Graph Nodes Edges
YahooWeb 1.4 Billion 6 Billion
Symantec Machine-File Graph 1 Billion 37 Billion
Twitter 104 Million 3.7 Billion
Phone call network 30 Million 260 Million
We also work with small data. Small data also needs love.
![Page 13: Data & Visual Analytics · Logistics Make sure you’re in the right Piazza! ... This course helps you learn some important skills. Collection Cleaning Integration Visualization Analysis](https://reader033.vdocuments.us/reader033/viewer/2022042218/5ec4965fa26ed301f878a210/html5/thumbnails/13.jpg)
7
![Page 14: Data & Visual Analytics · Logistics Make sure you’re in the right Piazza! ... This course helps you learn some important skills. Collection Cleaning Integration Visualization Analysis](https://reader033.vdocuments.us/reader033/viewer/2022042218/5ec4965fa26ed301f878a210/html5/thumbnails/14.jpg)
7Number of items an average human
holds in working memory
±2George Miller, 1956
![Page 15: Data & Visual Analytics · Logistics Make sure you’re in the right Piazza! ... This course helps you learn some important skills. Collection Cleaning Integration Visualization Analysis](https://reader033.vdocuments.us/reader033/viewer/2022042218/5ec4965fa26ed301f878a210/html5/thumbnails/15.jpg)
![Page 16: Data & Visual Analytics · Logistics Make sure you’re in the right Piazza! ... This course helps you learn some important skills. Collection Cleaning Integration Visualization Analysis](https://reader033.vdocuments.us/reader033/viewer/2022042218/5ec4965fa26ed301f878a210/html5/thumbnails/16.jpg)
7
![Page 17: Data & Visual Analytics · Logistics Make sure you’re in the right Piazza! ... This course helps you learn some important skills. Collection Cleaning Integration Visualization Analysis](https://reader033.vdocuments.us/reader033/viewer/2022042218/5ec4965fa26ed301f878a210/html5/thumbnails/17.jpg)
Data
Insights
![Page 18: Data & Visual Analytics · Logistics Make sure you’re in the right Piazza! ... This course helps you learn some important skills. Collection Cleaning Integration Visualization Analysis](https://reader033.vdocuments.us/reader033/viewer/2022042218/5ec4965fa26ed301f878a210/html5/thumbnails/18.jpg)
16
How to do that?
COMPUTATION +
HUMAN INTUITION
![Page 19: Data & Visual Analytics · Logistics Make sure you’re in the right Piazza! ... This course helps you learn some important skills. Collection Cleaning Integration Visualization Analysis](https://reader033.vdocuments.us/reader033/viewer/2022042218/5ec4965fa26ed301f878a210/html5/thumbnails/19.jpg)
17
Or, to ride the AI wave…
ARTIFICIAL INTELLIGENCE+
HUMAN INTELLIGENCE
![Page 20: Data & Visual Analytics · Logistics Make sure you’re in the right Piazza! ... This course helps you learn some important skills. Collection Cleaning Integration Visualization Analysis](https://reader033.vdocuments.us/reader033/viewer/2022042218/5ec4965fa26ed301f878a210/html5/thumbnails/20.jpg)
Both develop methods for making sense of network data
18
How to do that?
COMPUTATION INTERACTIVE VISAutomatic User-driven; iterative
Summarization, clustering, classification Interaction, visualization
>Millions of nodes Thousands of nodes
![Page 21: Data & Visual Analytics · Logistics Make sure you’re in the right Piazza! ... This course helps you learn some important skills. Collection Cleaning Integration Visualization Analysis](https://reader033.vdocuments.us/reader033/viewer/2022042218/5ec4965fa26ed301f878a210/html5/thumbnails/21.jpg)
18
How to do that?
COMPUTATION INTERACTIVE VISAutomatic User-driven; iterative
Summarization, clustering, classification Interaction, visualization
>Millions of nodes Thousands of nodes
![Page 22: Data & Visual Analytics · Logistics Make sure you’re in the right Piazza! ... This course helps you learn some important skills. Collection Cleaning Integration Visualization Analysis](https://reader033.vdocuments.us/reader033/viewer/2022042218/5ec4965fa26ed301f878a210/html5/thumbnails/22.jpg)
18
How to do that?
COMPUTATION INTERACTIVE VISAutomatic User-driven; iterative
Summarization, clustering, classification Interaction, visualization
>Millions of nodes Thousands of nodes
![Page 23: Data & Visual Analytics · Logistics Make sure you’re in the right Piazza! ... This course helps you learn some important skills. Collection Cleaning Integration Visualization Analysis](https://reader033.vdocuments.us/reader033/viewer/2022042218/5ec4965fa26ed301f878a210/html5/thumbnails/23.jpg)
18
How to do that?
COMPUTATION INTERACTIVE VISAutomatic User-driven; iterative
Summarization, clustering, classification Interaction, visualization
>Millions of nodes Thousands of nodes
![Page 24: Data & Visual Analytics · Logistics Make sure you’re in the right Piazza! ... This course helps you learn some important skills. Collection Cleaning Integration Visualization Analysis](https://reader033.vdocuments.us/reader033/viewer/2022042218/5ec4965fa26ed301f878a210/html5/thumbnails/24.jpg)
18
How to do that?
COMPUTATION INTERACTIVE VISAutomatic User-driven; iterative
Summarization, clustering, classification Interaction, visualization
>Millions of nodes Thousands of nodes
![Page 25: Data & Visual Analytics · Logistics Make sure you’re in the right Piazza! ... This course helps you learn some important skills. Collection Cleaning Integration Visualization Analysis](https://reader033.vdocuments.us/reader033/viewer/2022042218/5ec4965fa26ed301f878a210/html5/thumbnails/25.jpg)
18
How to do that?
COMPUTATION INTERACTIVE VISAutomatic User-driven; iterative
Summarization, clustering, classification Interaction, visualization
>Millions of nodes Thousands of nodes
![Page 26: Data & Visual Analytics · Logistics Make sure you’re in the right Piazza! ... This course helps you learn some important skills. Collection Cleaning Integration Visualization Analysis](https://reader033.vdocuments.us/reader033/viewer/2022042218/5ec4965fa26ed301f878a210/html5/thumbnails/26.jpg)
Our research combines the Best of Both Worlds
19
Our Approach for Big Data Analytics
DATA MINING HCIAutomatic User-driven; iterative
Summarization, clustering, classification Interaction, visualization
>Millions of items Thousands of items
Human-Computer Interaction
![Page 27: Data & Visual Analytics · Logistics Make sure you’re in the right Piazza! ... This course helps you learn some important skills. Collection Cleaning Integration Visualization Analysis](https://reader033.vdocuments.us/reader033/viewer/2022042218/5ec4965fa26ed301f878a210/html5/thumbnails/27.jpg)
20
Our mission & vision:
Scalable, interactive, usable tools for big data analytics
![Page 28: Data & Visual Analytics · Logistics Make sure you’re in the right Piazza! ... This course helps you learn some important skills. Collection Cleaning Integration Visualization Analysis](https://reader033.vdocuments.us/reader033/viewer/2022042218/5ec4965fa26ed301f878a210/html5/thumbnails/28.jpg)
“Computers are incredibly fast, accurate, and stupid.
Human beings are incredibly slow, inaccurate, and brilliant.
Together they are powerful beyond imagination.”
(Einstein might or might not have said this.)
![Page 29: Data & Visual Analytics · Logistics Make sure you’re in the right Piazza! ... This course helps you learn some important skills. Collection Cleaning Integration Visualization Analysis](https://reader033.vdocuments.us/reader033/viewer/2022042218/5ec4965fa26ed301f878a210/html5/thumbnails/29.jpg)
Course website (policies, syllabus, schedule, etc.)
https://poloclub.github.io/cse6242-2020spring-campus/(link also available on Canvas)
Discussion, Q&A, find teammates
Piazza (link/tab available on Canvas)
Assignment Submission
Canvas
Logistics
Make sure you’re in the right Piazza!(CSE-6242-O01, CSE-6242-OAN have
their Piazza forums too)
![Page 30: Data & Visual Analytics · Logistics Make sure you’re in the right Piazza! ... This course helps you learn some important skills. Collection Cleaning Integration Visualization Analysis](https://reader033.vdocuments.us/reader033/viewer/2022042218/5ec4965fa26ed301f878a210/html5/thumbnails/30.jpg)
Course HomepageFor syllabus, schedule, projects, datasets, etc.
If you Google “cse6242”, you will see many matches. Make sure you click the correct site!
![Page 32: Data & Visual Analytics · Logistics Make sure you’re in the right Piazza! ... This course helps you learn some important skills. Collection Cleaning Integration Visualization Analysis](https://reader033.vdocuments.us/reader033/viewer/2022042218/5ec4965fa26ed301f878a210/html5/thumbnails/32.jpg)
• Polo will announce events related to this class and data science in general
• Distinguished lectures
• Seminars
• Hackathons (free food, prizes)
• Company recruitment events (free food, swag)
Important to join Piazza because…
![Page 33: Data & Visual Analytics · Logistics Make sure you’re in the right Piazza! ... This course helps you learn some important skills. Collection Cleaning Integration Visualization Analysis](https://reader033.vdocuments.us/reader033/viewer/2022042218/5ec4965fa26ed301f878a210/html5/thumbnails/33.jpg)
Course Goals
26
![Page 34: Data & Visual Analytics · Logistics Make sure you’re in the right Piazza! ... This course helps you learn some important skills. Collection Cleaning Integration Visualization Analysis](https://reader033.vdocuments.us/reader033/viewer/2022042218/5ec4965fa26ed301f878a210/html5/thumbnails/34.jpg)
27
What is Data & Visual Analytics?
![Page 35: Data & Visual Analytics · Logistics Make sure you’re in the right Piazza! ... This course helps you learn some important skills. Collection Cleaning Integration Visualization Analysis](https://reader033.vdocuments.us/reader033/viewer/2022042218/5ec4965fa26ed301f878a210/html5/thumbnails/35.jpg)
27
What is Data & Visual Analytics?
No formal definition!
![Page 36: Data & Visual Analytics · Logistics Make sure you’re in the right Piazza! ... This course helps you learn some important skills. Collection Cleaning Integration Visualization Analysis](https://reader033.vdocuments.us/reader033/viewer/2022042218/5ec4965fa26ed301f878a210/html5/thumbnails/36.jpg)
27
Polo’s definition: the interdisciplinary science of combining computation techniques and interactive visualization to transform and model data to aid discovery, decision making, etc.
What is Data & Visual Analytics?
No formal definition!
![Page 37: Data & Visual Analytics · Logistics Make sure you’re in the right Piazza! ... This course helps you learn some important skills. Collection Cleaning Integration Visualization Analysis](https://reader033.vdocuments.us/reader033/viewer/2022042218/5ec4965fa26ed301f878a210/html5/thumbnails/37.jpg)
28
What are the “ingredients”?
![Page 38: Data & Visual Analytics · Logistics Make sure you’re in the right Piazza! ... This course helps you learn some important skills. Collection Cleaning Integration Visualization Analysis](https://reader033.vdocuments.us/reader033/viewer/2022042218/5ec4965fa26ed301f878a210/html5/thumbnails/38.jpg)
28
What are the “ingredients”?
Need to worry (a lot) about: storage, complex system design, scalability of algorithms, visualization techniques, interaction techniques, statistical tests, etc.
Wasn’t this complex before this big data era. Why?
![Page 39: Data & Visual Analytics · Logistics Make sure you’re in the right Piazza! ... This course helps you learn some important skills. Collection Cleaning Integration Visualization Analysis](https://reader033.vdocuments.us/reader033/viewer/2022042218/5ec4965fa26ed301f878a210/html5/thumbnails/39.jpg)
29http://spanning.com/blog/choosing-between-storage-based-and-unlimited-storage-for-cloud-data-backup/
![Page 40: Data & Visual Analytics · Logistics Make sure you’re in the right Piazza! ... This course helps you learn some important skills. Collection Cleaning Integration Visualization Analysis](https://reader033.vdocuments.us/reader033/viewer/2022042218/5ec4965fa26ed301f878a210/html5/thumbnails/40.jpg)
What is big data? Why care?Many businesses are based on big data.
Search engines: rank webpages, predict what you’re going to type
Advertisement: infer what you like, based on what your friends like; show relevant ads
E-commerce: recommends movies/products (e.g., Netfl ix, Amazon)
Health IT: patient records (EMR)
Finance
![Page 41: Data & Visual Analytics · Logistics Make sure you’re in the right Piazza! ... This course helps you learn some important skills. Collection Cleaning Integration Visualization Analysis](https://reader033.vdocuments.us/reader033/viewer/2022042218/5ec4965fa26ed301f878a210/html5/thumbnails/41.jpg)
Good news! Many jobs!
Most companies are looking for “data scientists”
The data scientist role is critical for organizations looking to extract insight from information assets for ‘big data’ initiatives and requires a broad combination of skills that may be fulfilled better as a team- Gartner (http://www.gartner.com/it-glossary/data-scientist)
Breadth of knowledge is important.This course helps you learn some important skills.
![Page 42: Data & Visual Analytics · Logistics Make sure you’re in the right Piazza! ... This course helps you learn some important skills. Collection Cleaning Integration Visualization Analysis](https://reader033.vdocuments.us/reader033/viewer/2022042218/5ec4965fa26ed301f878a210/html5/thumbnails/42.jpg)
Collection
Cleaning
Integration
Visualization
Analysis
Presentation
Dissemination
Course Schedule (Analytics Building Blocks)
![Page 43: Data & Visual Analytics · Logistics Make sure you’re in the right Piazza! ... This course helps you learn some important skills. Collection Cleaning Integration Visualization Analysis](https://reader033.vdocuments.us/reader033/viewer/2022042218/5ec4965fa26ed301f878a210/html5/thumbnails/43.jpg)
Building blocks. Not Rigid “Steps”.
Can skip some
Can go back (two-way street)
• Data types inform visualization design
• Data size informs choice of algorithms
• Visualization motivates more data cleaning
• Visualization challenges algorithm assumptionse.g., user finds that results don’t make sense
Collection
Cleaning
Integration
Visualization
Analysis
Presentation
Dissemination
![Page 44: Data & Visual Analytics · Logistics Make sure you’re in the right Piazza! ... This course helps you learn some important skills. Collection Cleaning Integration Visualization Analysis](https://reader033.vdocuments.us/reader033/viewer/2022042218/5ec4965fa26ed301f878a210/html5/thumbnails/44.jpg)
• Learn visual and computation techniques and use them in complementary ways
• Gain a breadth of knowledge
• Learn practical know-how by working on real data & problems
Course Goals
![Page 45: Data & Visual Analytics · Logistics Make sure you’re in the right Piazza! ... This course helps you learn some important skills. Collection Cleaning Integration Visualization Analysis](https://reader033.vdocuments.us/reader033/viewer/2022042218/5ec4965fa26ed301f878a210/html5/thumbnails/45.jpg)
• [50%] 4 homework assignments
• End-to-end analysis
• Techniques (computation and vis)
• “Big data” tools, e.g., Hadoop, Spark, etc.
• [50%] Group project -- 4 to 6 people
• [Tentative bonus points] In-class pop quizzes
• Each quiz is worth 1% course grade
• No exams
Grading
![Page 46: Data & Visual Analytics · Logistics Make sure you’re in the right Piazza! ... This course helps you learn some important skills. Collection Cleaning Integration Visualization Analysis](https://reader033.vdocuments.us/reader033/viewer/2022042218/5ec4965fa26ed301f878a210/html5/thumbnails/46.jpg)
Policies On website; we go through them now
Grading, plagiarism, collaboration,
late submission, and the “warnings” about the difficulty this course
![Page 47: Data & Visual Analytics · Logistics Make sure you’re in the right Piazza! ... This course helps you learn some important skills. Collection Cleaning Integration Visualization Analysis](https://reader033.vdocuments.us/reader033/viewer/2022042218/5ec4965fa26ed301f878a210/html5/thumbnails/47.jpg)
From Previous Classes…
• Class projects turned into papers at top conferences (KDD, IUI, etc.)
• Projects as portfolio pieces on CV
• Increased job and internship opportunities
• Former students sent me “thank you” notes
![Page 48: Data & Visual Analytics · Logistics Make sure you’re in the right Piazza! ... This course helps you learn some important skills. Collection Cleaning Integration Visualization Analysis](https://reader033.vdocuments.us/reader033/viewer/2022042218/5ec4965fa26ed301f878a210/html5/thumbnails/48.jpg)
IUI Full conference paper
![Page 49: Data & Visual Analytics · Logistics Make sure you’re in the right Piazza! ... This course helps you learn some important skills. Collection Cleaning Integration Visualization Analysis](https://reader033.vdocuments.us/reader033/viewer/2022042218/5ec4965fa26ed301f878a210/html5/thumbnails/49.jpg)
KDD Workshop paper
![Page 50: Data & Visual Analytics · Logistics Make sure you’re in the right Piazza! ... This course helps you learn some important skills. Collection Cleaning Integration Visualization Analysis](https://reader033.vdocuments.us/reader033/viewer/2022042218/5ec4965fa26ed301f878a210/html5/thumbnails/50.jpg)
IUI Poster paper
![Page 51: Data & Visual Analytics · Logistics Make sure you’re in the right Piazza! ... This course helps you learn some important skills. Collection Cleaning Integration Visualization Analysis](https://reader033.vdocuments.us/reader033/viewer/2022042218/5ec4965fa26ed301f878a210/html5/thumbnails/51.jpg)
“I feel like the concepts from your class are like a rite of passage for an aspiring data scientist. Assignments lead to a feelings of accomplishment and truly progressing in my area of passion.”
“I really get more intuition about how to deal with data with some powerful tools in HW3 [uses AWS]. That feeling is beyond description for me.”
“I would like to say thank you for your class! Thanks to the skills I got from the class and the project, I got the offer.”
41
![Page 52: Data & Visual Analytics · Logistics Make sure you’re in the right Piazza! ... This course helps you learn some important skills. Collection Cleaning Integration Visualization Analysis](https://reader033.vdocuments.us/reader033/viewer/2022042218/5ec4965fa26ed301f878a210/html5/thumbnails/52.jpg)
What Polo expects from you
• Actively participate throughout the course!
• Ask questions during class and on Piazza
• Help out whenever you can, e.g., help answer questions on Piazza
• Polo reserves last few minutes of every class for Q&A
![Page 53: Data & Visual Analytics · Logistics Make sure you’re in the right Piazza! ... This course helps you learn some important skills. Collection Cleaning Integration Visualization Analysis](https://reader033.vdocuments.us/reader033/viewer/2022042218/5ec4965fa26ed301f878a210/html5/thumbnails/53.jpg)
FREE After-class Coffee ☕• After (some) classes, Polo randomly selects
5 students (+2 volunteers) for FREE after-class coffee
• Polo’s treat. You can order coffee, tea, pastries — whatever you want
• Very casual — you can ask me ANYTHING
• Will try doing this starting next week!