data driven decisions seminar

Veldkant 33A, Kontich ● [email protected] ● www.infofarm.be

Data Science Company

Data Driven Decisions withloosely structured data

InfoFarm seminar25/11/2015


Wrap up and lunch

9:30 9:40 10:00 10:30 11:45Coffee & welcome

Dark data

External data To structure or not to structure?

Log files

Text mining

Network analysis

Image processing

Overview

Veldkant 33A, Kontich ● [email protected] ● www.infofarm.beData Science Company

About us


Data Science Big Data

www.infofarm.be


Building (Big) Data (Science) solutions

– Recommendation engines, Prediction models, Automated classification, …

– Custom-made data applications


developmentDevelopment

Domain

knowledge

Data

Science

Visualization


Completing the puzzle


Our approach

•Explorative Data Analysis (EDA)

•Formulate hypotheses•Hypotheses testing

•Implement•Automate•Integrate•Add extra data gathering•Rollout

•Identify use cases•Clean data•Enrich data

•Gather the data you need

Acquire Prepare

AnalyzeAct


Our approach

“Don’t run before you can walk”

CollectDescribe

DiscoverPredict

Advise

This is were the hype

around Big Data and Data

Science generates

unrealistic expectations!


Our customers

10


Data Driven Decisions


Business KnowledgeAcquired by experience

(assumed) insights

RISK: too high bias on past experience and gut feeling

Data ScienceComplementary to business knowledge

Confirmative or new insightsData-driven decision taking

RISK: too naive data intepretation, disconnected from business

Versus business knowledge


It’s all about asking the right question!


Finding those questions

What do you want?

What do you have?

What is feasible?

What is implementable?What can you get?


Where to start?


The key point is spotting opportunities to outperform your

competitors!


What to dream?


How about the in the room?


Unused but valuable data sources


Hidden – Forgotten - Underestimated


Inaccessible


Terrifying


Or all of the above


Internal

ERP SYSTEM

Financial management

Supply Chain Management

Manufacturing Resource Planning

Human Resource

management

Customer Relationship Management

Secondary use of


External

And many,many more

….


When all comes together…

WEB SERVER LOGSWhich customers

looked at similar products?

ORDER HISTORYWhich

complementary products does the

customer own?

EXTERNAL DATAReviews or critics?

CRM INFORMATIONTypical profile of a

customer responsive on campaigns for a

similar product?


Analyzing non-relational data


To structureOr

Not to structure…


Count

Parse


Who’s connected to who? Who bought what?


Red

Green

Blue


Log files


From weblogs

How long?

When?What?

What will you buy???Will you buy???


Over usage logsUses of Google

When you're too lazy to type in ".com"Finding PornFinding useful informationSpell Checking


To performance logs


Demo: the use of external logs


External weather logs

Coordinates?

Missing data


Do not blindly trust your data

Outliers Missing


But do not give up to soonLa

titud

e

Longitude


As usage is still possible


Text mining


From named entity extraction


Over sentiment analysis


To topic extraction

Text

TopicWhat should we

communicate about?


Network analysis


From network optimization


Over mathematical mumbo-jumbo


To (predictive) network analysis


And recommenders


Image processing


From human taught


To self-learned

data driven decisions seminar

Data & Analytics