mastering the 80% of analytics: what data scientists really do

Mastering the 80% of Analytics

What Data Scientists Really Do

Mik, PhD @AvrioAnalytics

Boss Parents

Me Reality

Caveat

•Data Science is very broad

•This is a particular perspective

•Mathematician

•Predictive algorithm developer

•Very brief

A Day in the LifeWrangling

Modeling

FeaturesR

esul

ts

What is “Wrangling”?•Data:

•Getting

•Formatting

•Cleaning

What is “Wrangling”?•Data:

•Getting

•Formatting

•Cleaning

Data Janitorial Work

Getting the Data

•Myriad of sources

Getting the Data


•Varying collection, storage and maintenance

Getting the Data



•Most people just don’t care

Getting the Data



•Most people just don’t care

•At least not soon enough

Got it. Now what?

•Structured: in a consistent and defined format

Got it. Now what?


•Unstructured: no consistent format

Got it. Now what?



•Text data

Got it. Now what?



•Text data

Movie Rating

Star Wars 5 StarsI loved the new Star Wars,

definitely 5/5 stars!

Formatting

•Alignment

Formatting

•Alignment

•Unions, intersections, grouping

Formatting

•Alignment

•Unions, intersections, grouping

•Transformations

FormattingTime Username Views

12:30 jsmith 32

12:45 mik 27

1:00 dmartin 8

1:15 jsmith 46

Time Username Views

12:20 gwarren 12

12:30 lpeabody 53

12:40 dmartin 20

12:50 hjohnson 5

Formatting

Username Views

jsmith 32, 46

Data is Dirty Business

•Duplicates


•Duplicates

•Missing values


•Duplicates

•Missing values

• Ill-formed values


•Duplicates

•Missing values


•Wrong values


•Duplicates

•Missing values


•Wrong values

Similar in effect


•Duplicates

•Missing values


•Wrong values

Types of Missing-ness

•MCAR: Missing Completely at Random



•MAR: Missing at Random




•MNAR: Missing Not at Random




•MNAR: Missing Not at Random

Bad

Worse

Dealing with Missing DataX Y Z

129 1 40110 3210 32

989 65

Dealing with Missing Data

•DeletionX Y Z

129 1 40110 3210 32

989 65


•Deletion

•Pairwise

•Listwise

X Y Z129 1 40110 3210 32

989 65


•Deletion

•Pairwise

•Listwise

X Y Z129 1 40110 3210 32

989 65

X Z129 40210 32

PairwiseX Y Z

129 40

Listwise


• Imputation


• Imputation

•Mean substitution

•Regression


•Multiple Imputation



•Stochastic simulation



•Stochastic simulation

•Must know distribution

Gotchas

•Sampling Error

Gotchas

•Sampling Error

•Statistical Power

Gotchas

•Sampling Error


•Population Parameters

Gotchas

•Sampling Error


•Population Parameters

•Propagation

So what do I do?

•Approaches vary quite a lot

So what do I do?


•MCAR, MAR hard to prove

So what do I do?


•MCAR, MAR hard to prove

•Principle of Least Harm

60% - 80% of Work

Cleaning Done! Now the fun!

•Almost…


•Almost…

•Clean data is still “raw”


•Almost…

•Clean data is still “raw”

•Features: pre-processed for modeling

Feature Engineering

•A lot of data is useless

Feature Engineering


•Filter, slice, transform

Feature Engineering


•Filter, slice, transform

•Singular idea: What’s the main driver?

Feature Engineering

•Considerations

•Relevance

•Redundancy

Feature Engineering

•Considerations

•Relevance

•Redundancy

•Curse of Dimensionality

Feature Engineering Methods

• PCA

• Edge Detection

• Blob Detection

• Auto encoding

• Kernel PCA

• Partial Least Squares

• Generalized Least Squares

• Direct Modeling

• Isomapping

• Mutual Information Theory

• Information Entropy Theory

• ICA

• MDR

• Latent Factors

• MPCA

• LSA

• Statistical Moments

• Random Projections

•De-Noising

•Weighting

•Patch Extraction

•Functional Mapping

•Discretization

•Filtering

•FFT

•Smoothing

•Density Mapping

Feature Engineering

• It’s hard

Feature Engineering

• It’s hard

•Analysis + Domain knowledge

Feature Engineering

• It’s hard


•…Deserves a presentation on its own

Feature Engineering

• It’s hard


•…Deserves a presentation on its own

•Features are input to machine learning

Now the fun stuff (finally)

•ML: computer acts without explicit program



•Utilizes empirical data to “teach” a process




•Pattern Rec. -> ML -> Deep Learning





•Buzzwords abound





•Buzzwords abound

•Fairly simple, lots of libraries

ML Approaches

•Classes of problems

ML Approaches


•Continuous (regression)

ML Approaches



•Discrete (classification)

ML Approaches




•Classes of solutions

ML Approaches





•Supervised

ML Approaches





•Supervised

•Unsupervised

ML Algorithms

•Neural Networks

ML Algorithms

•Neural Networks

•Genetic Algorithms

ML Algorithms

•Neural Networks


•Bayesian Classification

ML Algorithms

•Neural Networks



•Support Vector Machines

ML Algorithms

•Neural Networks



•Support Vector Machines

•Many used as type of feature extraction

Neural Networks

•Motivated by brain function

•Neurons fire, activate paths

•Non-linear

•Simplest: PerceptronX1

X2

Logic Layer

w1

w2

Neural Networks

• Inputs feed neuron with weight

Neural Networks


•Logic Layer: activation function

Neural Networks



•Fires (or not) based on inputs

Neural Networks




•Weights from minimizing cost function

Neural Networks




•Weights from minimizing cost function

•Backpropagation

Sigmoid Logic Layer

0

0.25

0.5

0.75

1

-10 -8 -6 -4 -2 0 2 4 6 8 10

w = 1 w = 2

1

1 + e�w

Tx

Neural Networks

•Most networks are bigger

X1

X2

A1

AM

Y1

YK

Machine Learning

•Got data, features and algorithm

Machine Learning


•Just plug in and profit!

Machine Learning



•Not quite

Machine Learning



•Not quite

•Tuning and training

Tuning

•What about N, M and K?

X1

X2

A1

AM

Y1

YK

Tuning


•Hyper-parameters

X1

X2

A1

AM

Y1

YK

Tuning


•Hyper-parameters

•Size of layers, thresholds, etc.

X1

X2

A1

AM

Y1

YK

Tuning


•Hyper-parameters

•Size of layers, thresholds, etc.

•Static specifics of the algorithm

X1

X2

A1

AM

Y1

YK

Training

• It’s all about the teaching

Training


•Representative data set

Training


•Representative data set

•Large, clean

Training

•Don’t teach to the test

Training


•Causes overfitting

Training



•Training (80%) and Testing (20%) data

Training



•Training (80%) and Testing (20%) data

•Cross-validation

With all the open source libraries, isn’t machine learning easy now?

I got results!

•Why doesn’t anyone care?

I got results!


•Kaggle vs. Real Life Syndrome

I got results!


•Kaggle vs. Real Life Syndrome

• It’s all in the presentation

It’s all in the presentation

•Complex topic


•Complex topic

•Non-technical audience


•Complex topic


•Several stakeholders


•Complex topic


•Several stakeholders

•Many likely skeptics


•Avoid buzzwords


•Avoid buzzwords

•Focus on a business problem


•Avoid buzzwords


•Show value


•Avoid buzzwords


•Show value

•Keep in mind cost

Is it actually science?

•Sometimes


•Sometimes

•…but often not


•Sometimes

•…but often not

•Data Sciences vs. Data Engineering


•Sometimes

•…but often not

•Data Sciences vs. Data Engineering

• It should be — focus on why


Applied Math

Computer Science

Domain Expertise


Applied Math

Computer Science

Domain Expertise

Applied Math

Computer Science

Physics

Physicist

Why Data Science?

•Big problems, fun challenges

Why Data Science?


•Both science and business

Why Data Science?


•Both science and business

•Consistently awesome

2012: Sexiest Job of the Century

2016: Best Job of the Year

2016: Hottest Job of the Year

2016: Best Career Opportunity

Why Data Science?S

alar

y

So want to get started?

•Theano


•Theano

•TensorFlow


•Theano

•TensorFlow

•Torch


•Theano

•TensorFlow

•Torch

•Pandas

Tomorrow is here

www.avrioanalytics.com

mastering the 80% of analytics: what data scientists really do

Data & Analytics