visual intelligence @ ibs, hyderabad
DESCRIPTION
An introduction to the basics of Visual Intelligence with an illustration of Industry case studies.TRANSCRIPT
GANES KESARI, VP DELIVERY
VISUAL INTELLIGENCE IBS CONFERENCE, HYDERABAD (13 AUG ‘14)
2
AGENDA
Industry Case StudiesSome interesting applications of Visual Intelligence
Why VisualizeWhat is Data Visualization and what’s the need for it
Q&AOpen discussion
A DATA VISUALISATION CHALLENGE…
You will see 3 questions.You have 30 seconds.
Try it!
Your timerstarts now
HOW MANY NUMBERS ARE ABOVE 100?1
23 32 71 72 58 87 11 77 70 1617 21 56 44 68 51 84 20 60 40
37 8 107 14 12 41 69 14 18 71
62 55 59 64 33 55 71 58 103 92
101 56 45 34 43 15 73 78 6 93
39 53 22 26 26 94 60 82 99 74
11 12 36 67 70 71 97 59 73 99
75 74 69 69 51 48 2 66 92 98
15 10 41 58 104 94 92 84 74 82
12 52 10 57 33 77 88 81 81 91
15 56 25 30 21 7 66 66 78 87
29 23 5 34 11 96 74 99 99 88
37 10 43 15 50 71 65 60 101 98
46 34 19 102 57 70 95 84 63 91
3 34 39 37 60 81 65 63 9 71
48 46 25 50 22 64 91 76 71 79
HOW MANY NUMBERS ARE BELOW 10?2
23 32 71 72 58 87 11 77 70 1617 21 56 44 68 51 84 20 60 40
37 8 107 14 12 41 69 14 18 71
62 55 59 64 33 55 71 58 103 92
101 56 45 34 43 15 73 78 6 93
39 53 22 26 26 94 60 82 99 74
11 12 36 67 70 71 97 59 73 99
75 74 69 69 51 48 2 66 92 98
15 10 41 58 104 94 92 84 74 82
12 52 10 57 33 77 88 81 81 91
15 56 25 30 21 7 66 66 78 87
29 23 5 34 11 96 74 99 99 88
37 10 43 15 50 71 65 60 101 98
46 34 19 102 57 70 95 84 63 91
3 34 39 37 60 81 65 63 9 71
48 46 25 50 22 64 91 76 71 79
WHICH QUADRANT HAS THE HIGHEST TOTAL?
23 32 71 72 58 87 11 77 70 1617 21 56 44 68 51 84 20 60 40
37 8 107 14 12 41 69 14 18 71
62 55 59 64 33 55 71 58 103 92
101 56 45 34 43 15 73 78 6 93
39 53 22 26 26 94 60 82 99 74
11 12 36 67 70 71 97 59 73 99
75 74 69 69 51 48 2 66 92 98
15 10 41 58 104 94 92 84 74 82
12 52 10 57 33 77 88 81 81 91
15 56 25 30 21 7 66 66 78 87
29 23 5 34 11 96 74 99 99 88
37 10 43 15 50 71 65 60 101 98
46 34 19 102 57 70 95 84 63 91
3 34 39 37 60 81 65 63 9 71
48 46 25 50 22 64 91 76 71 79
3
A DATA VISUALISATION CHALLENGE…
We’ll answer the same questions again.But with simple visual cues.
See how long it takes.
Your timerstarts now
23 32 71 72 58 87 11 77 70 1617 21 56 44 68 51 84 20 60 40
37 8 107 14 12 41 69 14 18 71
62 55 59 64 33 55 71 58 103 92
101 56 45 34 43 15 73 78 6 93
39 53 22 26 26 94 60 82 99 74
11 12 36 67 70 71 97 59 73 99
75 74 69 69 51 48 2 66 92 98
15 10 41 58 104 94 92 84 74 82
12 52 10 57 33 77 88 81 81 91
15 56 25 30 21 7 66 66 78 87
29 23 5 34 11 96 74 99 99 88
37 10 43 15 50 71 65 60 101 98
46 34 19 102 57 70 95 84 63 91
3 34 39 37 60 81 65 63 9 71
48 46 25 50 22 64 91 76 71 79
HOW MANY NUMBERS ARE ABOVE 100?1
HOW MANY NUMBERS ARE BELOW 10?2
23 32 71 72 58 87 11 77 70 16
17 21 56 44 68 51 84 20 60 40
37 8 107 14 12 41 69 14 18 71
62 55 59 64 33 55 71 58 103 92
101 56 45 34 43 15 73 78 6 93
39 53 22 26 26 94 60 82 99 74
11 12 36 67 70 71 97 59 73 99
75 74 69 69 51 48 2 66 92 98
15 10 41 58 104 94 92 84 74 82
12 52 10 57 33 77 88 81 81 91
15 56 25 30 21 7 66 66 78 87
29 23 5 34 11 96 74 99 99 88
37 10 43 15 50 71 65 60 101 98
46 34 19 102 57 70 95 84 63 91
3 34 39 37 60 81 65 63 9 71
48 46 25 50 22 64 91 76 71 79
WHICH QUADRANT HAS THE HIGHEST TOTAL?3
23 32 71 72 58 87 11 77 70 16
17 21 56 44 68 51 84 20 60 40
37 8 107 14 12 41 69 14 18 71
62 55 59 64 33 55 71 58 103 92
101 56 45 34 43 15 73 78 6 93
39 53 22 26 26 94 60 82 99 74
11 12 36 67 70 71 97 59 73 99
75 74 69 69 51 48 2 66 92 98
15 10 41 58 104 94 92 84 74 82
12 52 10 57 33 77 88 81 81 91
15 56 25 30 21 7 66 66 78 87
29 23 5 34 11 96 74 99 99 88
37 10 43 15 50 71 65 60 101 98
46 34 19 102 57 70 95 84 63 91
3 34 39 37 60 81 65 63 9 71
48 46 25 50 22 64 91 76 71 79
Most discussions of decision-making assume, that only senior executives make decisions or that only senior executives’ decisions matter. This is a dangerous mistake…
- Peter F. Drucker
It's clearly a budget! Has a lot of numbers in it!
- George W. Bush
Information is the oil of the 21st century, and analytics is the combustion engine
-- Gartner
Business analytics software grew 14% in 2011 and will hit $50.7 bn by 2016
-- IDC
… the #1 trend is applying information & analytics to solve business problems
-- Deloitte
MARKETTransaction data
Increasing data being churned out by systems in information highway
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
Social network dataConsumers embracing Web 2.0 and the social media lifestyle
M2M dataPortable devices generating data for consumption by systems
Storage costMaterial science research has led to significant increase in data density
Bandwidth costDriven by massive investments in fibre capacity
Processing costMoore’s law has doubled the processing power per $ every 1.5 yrs
Growth in available data, and the potential for exploiting these, have grown exponentially in the last 10 years.
This changing data landscape heralds a radical shift in business decision-making approach, even for mere survival in this new age.Data growth to 7.9 ZB by 2015 posing a real ‘Data Tsunami’
Gartner’s BI Magic Quadrant Trends
• Emergence of data discovery/ visualization
• Increased willingness for new low-cost options
• Embedded low-cost purpose-built analytic apps
• Need for intuitive BI tools on mobile platforms
“
THE VISUAL INTELLIGENCE MARKET
Business Analytics software market is slated to grow at 10% annually and hit $50.7 Bn by 2016
By 2015:
• Big Data market would be $16.9 Bn
• Data Discovery market would be $ 1.6 Bn
Today, only strategic decisions are made at a rate slower than the speed of business. Tactical and operational decisions must increasingly be at a rate faster than humans are capable of.
…challenge of mastering big data analytics is the hardest, because big data technologies are not ready for enterprise demands, & skills to work with big data are scarce
“Data discovery and visualization vendor initiatives, and the rapid adoption from end users, have the potential to shake the BI market to its foundations.
“Visualization would be the platform on which Big Data consumption would happen”
WHO USES DATA VISUALISATION?
Internet giantsGoogleLinkedIn…..
NewspapersNew York TimesThe Guardian…..
TV ChannelsCNNBBC…..
SportsNFLNBA…..
BanksWorld BankCitibank…..
RetailersAmazonTesco…..
ManufacturersCokeAirbus…..
GovernmentUS GovernmentUK Government…..Telecom
AirtelBT…..
ScienceMolecule imagingStar formation…..
HealthcareGE HealthcareDetroit Medical C.…..
and pretty much anyone, today…
CASE STUDIES
EDUCATION
PREDICTING MARKS
What determines a child’s marks?
Do girls score better than boys?
Does the choice of subject matter?
Does the medium of instruction matter?
Does community or religion matter?
Does their birthday matter?
Does the first letter of their name matter?
0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48 51 54 57 60 63 66 69 72 75 78 81 84 87 90 93 96 990
5,000
10,000
15,000
20,000
25,000
30,000
35,000
40,000
TN CLASS X: ENGLISH
TN CLASS X: SOCIAL SCIENCE
0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48 51 54 57 60 63 66 69 72 75 78 81 84 87 90 93 96 990
5,000
10,000
15,000
20,000
25,000
30,000
35,000
40,000
TN CLASS X: LANGUAGE
0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48 51 54 57 60 63 66 69 72 75 78 81 84 87 90 93 96 990
5,000
10,000
15,000
20,000
25,000
30,000
35,000
40,000
TN CLASS X: SCIENCE
0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48 51 54 57 60 63 66 69 72 75 78 81 84 87 90 93 96 990
5,000
10,000
15,000
20,000
25,000
30,000
35,000
40,000
TN CLASS X: MATHEMATICS
0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48 51 54 57 60 63 66 69 72 75 78 81 84 87 90 93 96 990
5,000
10,000
15,000
20,000
25,000
30,000
35,000
40,000
CBSE 2013 CLASS XII: ENGLISH MARKS
Based on the results of the 20 lakh students taking the Class XII exams at Tamil Nadu over the last 3 years, it appears that the month you were born in can make a difference of as much as 120 marks out of 1,200.
June borns score the
lowest
The marks shoot up for Aug borns
… and peaks for Sep-borns
120 marks out of 1200
explainable by month of birth
An identical pattern was observed in 2009 and 2010…
… and across districts, gender, subjects, and class X & XII.
“It’s simply that in Canada the eligibility cutoff for age-class hockey is January 1. A boy who turns ten on January 2, then, could be playing alongside someone who doesn’t turn ten until the end of the year—and at that age, in preadolescence, a twelve-month gap in age represents an enormous difference in physical maturity.”
-- Malcolm Gladwell, Outliers
BIRTHDAYS IN THE US AND IN INDIA
htt
ps:
//g
ram
en
er.
com
/post
ers
/Bir
thd
ays
.pd
f
CRICKET
FASTEST SCORERS
“I’ve always been curious… who among India’s prolific one-day run-getters had the best strike rate?
Sachin?
Sehwag?
What about the rest of the world?
INDIA ODI BATTINGhttps://gramener.com/cricket/
https://gramener.com/cricket/
Shift Evening Morning Night
Weekday Fri Mon Sat Sun Thu Tue Wed
Product category FAH N70 RPP TDS ZDH
Part shipment 20-40% 40-60% 60-80% <20% Full
CARGO DELAYThis visualisation measures the recovery time (time from arrival of the flight until delivery), and identifies which factors most influence the recovery time.
Recovery times are neutral during the evening and morning shifts (mornings are slightly worse), night times are the best.
Recovery times are worst on Fridays, and best on Saturdays & Wednesdays.
Specifically, Friday mornings are particularly bad. So are Thursday mornings.
The FAH product category has the best recovery time, while ZDH is much worse.
However, RPP on Sundays is unusually slow.
Part shipped products tend to perform worse than full-shipments. Specifically the <20% and 40-60% part-shipments.
This is especially problematic for ZDH
This visualisation is part of a suite of analytical techniques we call “grouped means” that allows us to measure the impact of every parameter (shifts, weekdays, etc.) on any measure of interest – recovery time in this case, but this could be extended to revenue, operational efficiency, or ability to cross-sell.
It allows automatically detection of statistically significant flows and highlights only relevant ones to users.
The system therefore analyses all possible patterns, but users only see the insights that matter.
THE SOCIAL TALE OF TWO CITIES: BANGALORE & SINGAPORE
Recruiting top quality developers is always a problem. We decided to use an algorithmic approach and pulled out the social network of developers on Github (a social network for open source code).
In this visualisation, each circle is a person. The size of the circle represents the number of followers. Larger circles have more followers (but not in proportion – it’s a log scale.)
The circle’s colour represents the city the programmer’s live in. This visual is a slice showing the tale of two cities: Bangalore and Singapore
Two people are connected if one follows the other. This leads to a clustering of people in the form of a network.
Here, you can see that Bangalore and Singapore are reasonably well connected cities. Bangalore has more developers, but Singapore has more popular ones (larger circles).
However, the interaction between Bangalore and Singapore are few and far between. But for a few people across both cities, like:
… etc.
Sudar, Yahoo!Anand C, ConsultantKiran, HasgeekAnand S, Gramener
Mugunth, Steinlogic Honcheng, buUukSau Sheong, HP LabsLim Chee Aung
Bangalore
Singapore
1 follower
100 followers
A follows B (or)
B follows A
Most followed in Bangalore
Most followed in Singapore
Ciju CherianLin JunjieAmudhi Sebastian
There are, of course, a number of smaller independent circles – people who are not connected to others in the same city. (They may be connected to people in other cities.)
Apart from this, there are a few small networks of connected people – often people within the same company or start-up – who form a community of their own.
https://gramener.com/codersearch/
Can we visualize the results of every single Lok Sabha
election since Independence?
https://gramener.com/election/parliament
On the other hand, which party has the most losses in
Lok Sabha elections?
https://gramener.com/election/parliament#story.ddp
Which constituency had the most number of candidates
ever?
https://gramener.com/election/parliament#cartogram?YEAR=1996&METRIC=CANDIDATES
We handle terabyte-size data
via non-traditional analytics and visualise it in real-time.
Gramener visualises your data
Gramener transforms your data into concise dashboardsthat make your business problem & solution visually obvious.We help you find insights quickly, based on cognitive research,and our visualisations guide you towards actionable decisions.
A data analytics and visualisation company
www.gramener.comblog.gramener.com
FURTHER READING
EDWARD TUFTE: CLASSICS ON VISUALIZATION
STEPHEN FEW: DASHBOARD DESIGN