big data trap
DESCRIPTION
Don't fall in the big data trap key principlesTRANSCRIPT
Data/Big Data Knowledge Action
People care about Knowledge/actions not data
Agenda
● Big data dilemma
● When are we doing Big Data?
● Maturity/Evolution steps
● The big data trap
● Optimal design = real time data-mining
● Increase your chances of success
The Big Data Dilemma
Big Data = Data + IO bounded (disk)
CPU<100%Data
IO bounded
QABI
MaturityBarriers of entry Levels
Just another barrier of entry
Trap = no KPI ● No KPI -> batch processing -> big data● KPI -> real time -> no big data complexity
Optimal design = real-time data-mining
● Events -> everything is an event● + Rule -> create signal from events ● + KPIs -> selection of signals (top level)● + Incident = signal static/dynamic thresholds● + Root causes analysis
○ Bayesian inference (ratio signal)○ Signal correlation (std signal)○ Rule filtering (domain specific)
Increase chances of success
● Data driven culture● Data quality culture (Avoid logs)
● Reach Analytics/BI level● KISS
Recap
● Big Data = Small Data + IO bound● Big data->Data->Analytics->Mining->Predictive
○ Data Quality = BIGGEST PROBLEM
○ Big Data = another barrier of entry
● Big data trap = no KPI
● KISS = real time data mining
