Download - Analytics demystified

Marc MOREAU – Développer la ville numérique

Analytic is not magic, it is mathematic

1



• Scope and definition

• Some examples• Forecasting (predicting the future)• Machine learning (learning from data and improving from experience)• Optimization (finding the right solution or set of solutions)

• Software and skills

2


What is analytics?

• Gartner :• Analytics has emerged as a catch-all term for a variety of different business intelligence (BI)

and application-related initiatives.• For some, it is the process of analyzing information from a particular domain, such as website

analytics. For others, it is applying the breadth of BI capabilities to a specific content area (for example, sales, service, supply chain and so on).

• In particular, BI vendors use the “analytics” moniker to differentiate their products from the competition.

• Increasingly, “analytics” is used to describe statistical and mathematical data analysis that clusters, segments, scores and predicts what scenarios are most likely to happen.

• Analytic is• about turning data into information for decision support and action• based on applied mathematics and algorithms

3


Analytics and Big Data

• Analytics and Big Data are often associated and even mixed

• Data mining can be performed and bring valuable insight without “big” data

• The characteristics of Big Data (Volume and/or Velocity and/or Variety) are a serious challenge for traditional algorithms which must be adapted (Map-Reduce…)

• Big Data is supposed to bring more information and value

4






5


Forecasting applications

• Forecast sales to organize production, manage budget…

• Forecast failures to organize predictive maintenance

• Forecast to detect anomalies and defaults

• …

6


GAM (Generalized additive models)

ARX(Auto Regressive model

with eXternal inputs)

Forecasting methods are just tools

7

ARIMAX(Auto Regressive

Integrated Moving Averagewith eXternal inputs)

PCA(Principal component analysis)

Nonlinear regression

Neural network


Forecasting methods are mathematical tools

8

Tomorrow = today

Linear regressionLinear trend

with seasonal adjustmentNon-linear regression

Moving average Arima

Arimax = Arima+ eXternal variables


Machine learning : applications

• Machine learning methods will discover hidden rules, correlation or information within data

• Clustering methods can be used for customer base segmentation

• Machine learning can help predict failures or detect frauds

• Machine learning can determine the probability a given customer will respond favorably to a certain interaction or will prefer a certain product (eg. Amazon)

9


Output value

Input parameters

Machine learning: neural networks

This method belongs to the family of supervised algorithms, because you have to “teach” the network with example inputs and correct answers.

Each « neuron » output is a function of the weighted sum of its inputs.

Based on a training set of data, the weights will be adapted until the network provides acceptable answers.

The nertwork learns a general rule that maps inputs to outputs.

10


Machine learning: clustering

• Wikipedia : Clustering is the task of grouping a set of objects in such a way that objects in the same group (called a cluster) are more similar (in some sense or another) to each other than to those in other groups (clusters).

• Algorithms can be bottom-upor top-down

• Example : k-means clustering

https://apandre.wordpress.com/visible-data/

11


Optimization applications

• Organize production (allocation between plants)

• Organize logistics (number and position of warehouses)

• Find the quickest or shortest path (GPS)

• Calibrate a model with the right parameters

• …

12


Optimisation: simplex algorithm

A method to minimize a linear objective function, subject to linear equality and linear inequality constraints.

A linear function to minimize can be the cost (fix cost + variable part)

If the objective function has a minimum value on the feasible regionthen it has this value on (at least) one of the extreme points.

The problem has to be written into a standard form (a matrix) and a sequence of matrix operations are applied to explore the feasible region.

http://en.wikipedia.org/wiki/Simplex_algorithm

13


Optimisation: genetic algorithms

Looking for the optimal set of parameters, in order to maximize a function, respecting a given set of constraints.

Example: number of units to be produced in each plant. Each plant has a production cost, a maximum production capacity and a maximum storage capacity

14

A set of parameters = a chromosome






15


Analytic is not cryptography

16

Algorithms and methods are mostly being developed openly and published…And even for cryptography, many prefer open methods rather than secret unchallenged algorithms.

There aren’t so many secret proprietarybreaking through methods.

Don’t get fooled by advertisers !A few start-ups have new innovative solutions,challenge them seriously.


Many toolboxes are available on the market

17

S+ (Tibco) R (Open Source)

SPSS (IBM)PI (OsiSoft)

Some BI platforms or software packages contain more tools than others, but 80% of their content is similar.The major ones are implementing Big Data strategies.For specific problems, experts will choose one specific toolbox because it contains a specific algorithm.Compatibility and integration with the existing IT system is the main issue.

Statistica (Statsoft)


The question is not about the tool but about the user

18

These tools can only be used by trained users who knows the limit of each method, can choose the right ones, can check if the result is consistent.Unskilled users can make big mistakes without even noticing.

The successful team:- a good end-user (with field experience to explain the problem)- a good data-scientist (to find, implement and maintain the solution)


Partnership or in-house expertise?

• In order to implement analytic solutions in your organization, you can either:• Rely on partners to define and implement the right methods:

• The software companies usually have consultants and experts (IBM…)• Specialized start-ups and SMEs (SenseWaves, :snips, Eurobios, Quantmetry…)

• Recruit and train data scientists• Students from ENSAE/ENSAI• Contact the Société Française de Statistique (SFdS)

• If you expect a regular analytic activity, or if you are in a changing environment it is recommended to have in-house expertise

19

Download - Analytics demystified

Top Related