enterprise miner: data exploration and visualisation · sas® enterprise miner™ sas® enterprise...
TRANSCRIPT
![Page 1: Enterprise Miner: Data Exploration and Visualisation · SAS® ENTERPRISE MINER™ SAS® ENTERPRISE MINER™ • Modern, collaborative, easy-to-use data mining workbench • Sophisticated](https://reader033.vdocuments.us/reader033/viewer/2022052520/6080dbfc432df679c57facae/html5/thumbnails/1.jpg)
Copyr i g ht © 2012, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
ENTERPRISE MINER:
1 – DATA EXPLORATION AND VISUALISATION
JOZEF MOFFAT, ANALYTICS & INNOVATION PRACTICE, SAS UK
10, MAY 2016
![Page 2: Enterprise Miner: Data Exploration and Visualisation · SAS® ENTERPRISE MINER™ SAS® ENTERPRISE MINER™ • Modern, collaborative, easy-to-use data mining workbench • Sophisticated](https://reader033.vdocuments.us/reader033/viewer/2022052520/6080dbfc432df679c57facae/html5/thumbnails/2.jpg)
Copyr i g ht © 2012, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
DATA EXPLORATION
AND VISUALISATIONAGENDA
• SAS Webinar – 10th May 2016 at 10:00 AM BST
• Enterprise Miner: Data Exploration and Visualisation
• The session looks at:
- Data Visualisation and Sampling
- Variable Selection
- Missing Value Imputation
- Outlier Detection
![Page 3: Enterprise Miner: Data Exploration and Visualisation · SAS® ENTERPRISE MINER™ SAS® ENTERPRISE MINER™ • Modern, collaborative, easy-to-use data mining workbench • Sophisticated](https://reader033.vdocuments.us/reader033/viewer/2022052520/6080dbfc432df679c57facae/html5/thumbnails/3.jpg)
Copyr i g ht © 2012, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
ROLE OF SAS ENTERPRISE MINER
![Page 4: Enterprise Miner: Data Exploration and Visualisation · SAS® ENTERPRISE MINER™ SAS® ENTERPRISE MINER™ • Modern, collaborative, easy-to-use data mining workbench • Sophisticated](https://reader033.vdocuments.us/reader033/viewer/2022052520/6080dbfc432df679c57facae/html5/thumbnails/4.jpg)
Copyr i g ht © 2012, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
THE ANALYTICS
LIFECYCLEPREDICTIVE ANALYTICS AND DATA MINING
IDENTIFY /
FORMULATE
PROBLEM
DATA
PREPARATION
DATA
EXPLORATION
TRANSFORM
& SELECT
BUILD
MODEL
VALIDATE
MODEL
DEPLOY
MODEL
EVALUATE /
MONITOR
RESULTS
Domain Expertise
Decision Making
Process and ROI Evaluation
Model Validation
Model Deployment
Model Monitoring
Data Preparation
Data Exploration
Data Visualization
Report Creation
Exploratory Analysis
Descriptive Segmentation
Predictive Modeling
![Page 5: Enterprise Miner: Data Exploration and Visualisation · SAS® ENTERPRISE MINER™ SAS® ENTERPRISE MINER™ • Modern, collaborative, easy-to-use data mining workbench • Sophisticated](https://reader033.vdocuments.us/reader033/viewer/2022052520/6080dbfc432df679c57facae/html5/thumbnails/5.jpg)
Copyr i g ht © 2012, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
THE ANALYTICS
LIFECYCLEPREDICTIVE ANALYTICS AND DATA MINING
Domain Expertise
Decision Making
Process and ROI Evaluation
Model Validation
Model Deployment
Model Monitoring
Data Preparation
Data Exploration
Data Visualization
Report Creation
Exploratory Analysis
Descriptive Segmentation
Predictive Modeling
IDENTIFY /
FORMULATE
PROBLEM
DATA
PREPARATION
DATA
EXPLORATION
TRANSFORM
& SELECT
BUILD
MODEL
VALIDATE
MODEL
DEPLOY
MODEL
EVALUATE /
MONITOR
RESULTS
![Page 6: Enterprise Miner: Data Exploration and Visualisation · SAS® ENTERPRISE MINER™ SAS® ENTERPRISE MINER™ • Modern, collaborative, easy-to-use data mining workbench • Sophisticated](https://reader033.vdocuments.us/reader033/viewer/2022052520/6080dbfc432df679c57facae/html5/thumbnails/6.jpg)
Copyr i g ht © 2012, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
SAS®
ENTERPRISE
MINER™
SAS® ENTERPRISE MINER™
• Modern, collaborative, easy-to-use data mining
workbench
• Sophisticated set of data preparation and exploration
tools
• Modern suite of modeling techniques and methods
• Interactive model comparison, testing and validation
• Automated scoring process delivers faster results
• Open, extensible design for ultimate flexibility
![Page 7: Enterprise Miner: Data Exploration and Visualisation · SAS® ENTERPRISE MINER™ SAS® ENTERPRISE MINER™ • Modern, collaborative, easy-to-use data mining workbench • Sophisticated](https://reader033.vdocuments.us/reader033/viewer/2022052520/6080dbfc432df679c57facae/html5/thumbnails/7.jpg)
Copyr i g ht © 2012, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
SAS®
ENTERPRISE
MINER™
SAS® ENTERPRISE MINER™
MODEL DEVELOPMENT PROCESS
Sample Explore Modify Model Assess
![Page 8: Enterprise Miner: Data Exploration and Visualisation · SAS® ENTERPRISE MINER™ SAS® ENTERPRISE MINER™ • Modern, collaborative, easy-to-use data mining workbench • Sophisticated](https://reader033.vdocuments.us/reader033/viewer/2022052520/6080dbfc432df679c57facae/html5/thumbnails/8.jpg)
Copyr i g ht © 2012, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
SAS®
ENTERPRISE
MINER™
SAS® ENTERPRISE MINER™
MODEL DEVELOPMENT PROCESS
Utility Apps.
Time
Series HPDM
Credit
Scoring
![Page 9: Enterprise Miner: Data Exploration and Visualisation · SAS® ENTERPRISE MINER™ SAS® ENTERPRISE MINER™ • Modern, collaborative, easy-to-use data mining workbench • Sophisticated](https://reader033.vdocuments.us/reader033/viewer/2022052520/6080dbfc432df679c57facae/html5/thumbnails/9.jpg)
Copyr i g ht © 2012, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
SAS®
ENTERPRISE
MINER™
SAS® ENTERPRISE MINER™
SEMMA IN ACTION – REPEATABLE PROCESS
![Page 10: Enterprise Miner: Data Exploration and Visualisation · SAS® ENTERPRISE MINER™ SAS® ENTERPRISE MINER™ • Modern, collaborative, easy-to-use data mining workbench • Sophisticated](https://reader033.vdocuments.us/reader033/viewer/2022052520/6080dbfc432df679c57facae/html5/thumbnails/10.jpg)
Copyr i g ht © 2012, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
DATA VISUALISATION AND SAMPLING
![Page 11: Enterprise Miner: Data Exploration and Visualisation · SAS® ENTERPRISE MINER™ SAS® ENTERPRISE MINER™ • Modern, collaborative, easy-to-use data mining workbench • Sophisticated](https://reader033.vdocuments.us/reader033/viewer/2022052520/6080dbfc432df679c57facae/html5/thumbnails/11.jpg)
Copyr i g ht © 2012, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
DATA
VISUALISATION AND
SAMPLING
VISUALISATION
“Quickly find related patterns within a set of data
via interactive pictures.”
![Page 12: Enterprise Miner: Data Exploration and Visualisation · SAS® ENTERPRISE MINER™ SAS® ENTERPRISE MINER™ • Modern, collaborative, easy-to-use data mining workbench • Sophisticated](https://reader033.vdocuments.us/reader033/viewer/2022052520/6080dbfc432df679c57facae/html5/thumbnails/12.jpg)
Copyr i g ht © 2012, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
DATA
VISUALISATION AND
SAMPLING
SAS® ENTERPRISE MINER™
SEMMA PROCESS
Sample Explore Modify Model Assess
![Page 13: Enterprise Miner: Data Exploration and Visualisation · SAS® ENTERPRISE MINER™ SAS® ENTERPRISE MINER™ • Modern, collaborative, easy-to-use data mining workbench • Sophisticated](https://reader033.vdocuments.us/reader033/viewer/2022052520/6080dbfc432df679c57facae/html5/thumbnails/13.jpg)
Copyr i g ht © 2012, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
DATA
VISUALISATION AND
SAMPLING
SAMPLE AND EXPLORE
Data selection
• Required & excluded fields
• Sample balancing
• Data partitioning
Data evaluation
• Statistical measures
• Visualization
• Identifying outliers
• Analytical segmentation
• Variable creation & selection
![Page 14: Enterprise Miner: Data Exploration and Visualisation · SAS® ENTERPRISE MINER™ SAS® ENTERPRISE MINER™ • Modern, collaborative, easy-to-use data mining workbench • Sophisticated](https://reader033.vdocuments.us/reader033/viewer/2022052520/6080dbfc432df679c57facae/html5/thumbnails/14.jpg)
Copyr i g ht © 2012, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
DATA
VISUALISATION AND
SAMPLING
SAMPLING
• Sampling
• Sample Node:
• Stratified / Simple Random Sampling
• Used for over/under sampling input data
• Data Partition Node:
• Random sampling into Training, Validation and Test sets
• Prevent model over fitting
• Filter Node:
• Select time period of interest
• Filter based on pre-defined flag
![Page 15: Enterprise Miner: Data Exploration and Visualisation · SAS® ENTERPRISE MINER™ SAS® ENTERPRISE MINER™ • Modern, collaborative, easy-to-use data mining workbench • Sophisticated](https://reader033.vdocuments.us/reader033/viewer/2022052520/6080dbfc432df679c57facae/html5/thumbnails/15.jpg)
Copyr i g ht © 2012, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
DATA
VISUALISATION AND
SAMPLING
SAMPLING
• Segmentation
• Cluster Node
• Unsupervised, k-means clustering algorithm
• Data driven
• Output tree based descriptions
![Page 16: Enterprise Miner: Data Exploration and Visualisation · SAS® ENTERPRISE MINER™ SAS® ENTERPRISE MINER™ • Modern, collaborative, easy-to-use data mining workbench • Sophisticated](https://reader033.vdocuments.us/reader033/viewer/2022052520/6080dbfc432df679c57facae/html5/thumbnails/16.jpg)
Copyr i g ht © 2012, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
DEMO
![Page 17: Enterprise Miner: Data Exploration and Visualisation · SAS® ENTERPRISE MINER™ SAS® ENTERPRISE MINER™ • Modern, collaborative, easy-to-use data mining workbench • Sophisticated](https://reader033.vdocuments.us/reader033/viewer/2022052520/6080dbfc432df679c57facae/html5/thumbnails/17.jpg)
Copyr i g ht © 2012, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
VARIABLE SELECTION
![Page 18: Enterprise Miner: Data Exploration and Visualisation · SAS® ENTERPRISE MINER™ SAS® ENTERPRISE MINER™ • Modern, collaborative, easy-to-use data mining workbench • Sophisticated](https://reader033.vdocuments.us/reader033/viewer/2022052520/6080dbfc432df679c57facae/html5/thumbnails/18.jpg)
Copyr i g ht © 2012, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
VARIABLE
SELECTIONVARIABLE SELECTION ROUTINES
• Variable Selection
• Variable Selection Node
• Relationship of independent variables to dependent target
• R-Square and Chi-square selection criteria
• Variable Clustering Node
• Identify correlations and covariance's between input variables
• Select Best variable from cluster or Cluster Component
• Interactive Grouping Node
• Computes Weights of Evidence
• GINI and Information Values for variable selection
![Page 19: Enterprise Miner: Data Exploration and Visualisation · SAS® ENTERPRISE MINER™ SAS® ENTERPRISE MINER™ • Modern, collaborative, easy-to-use data mining workbench • Sophisticated](https://reader033.vdocuments.us/reader033/viewer/2022052520/6080dbfc432df679c57facae/html5/thumbnails/19.jpg)
Copyr i g ht © 2012, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
DEMO
![Page 20: Enterprise Miner: Data Exploration and Visualisation · SAS® ENTERPRISE MINER™ SAS® ENTERPRISE MINER™ • Modern, collaborative, easy-to-use data mining workbench • Sophisticated](https://reader033.vdocuments.us/reader033/viewer/2022052520/6080dbfc432df679c57facae/html5/thumbnails/20.jpg)
Copyr i g ht © 2012, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
MISSING VALUE IMPUTATION
![Page 21: Enterprise Miner: Data Exploration and Visualisation · SAS® ENTERPRISE MINER™ SAS® ENTERPRISE MINER™ • Modern, collaborative, easy-to-use data mining workbench • Sophisticated](https://reader033.vdocuments.us/reader033/viewer/2022052520/6080dbfc432df679c57facae/html5/thumbnails/21.jpg)
Copyr i g ht © 2012, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
MISSING VALUE
IMPUTATIONIMPUTE
• Missing Value Imputation
• Impute Node
• Complete case required for models such as Regression
• Multiple imputation techniques e.g. Tree, Distribution,
Mean, ModeClass (categorical) variables Interval (numeric) variables
Input/Target Input/Target
Count Mean
Default Constant Value Median
Distribution Midrange
Tree (only for inputs) Distribution
Tree Surrogate (only for inputs) Tree (only for inputs)
Tree Surrogate (only for inputs)
Mid-Minimum Spacing
Tukey’s Biweight
Huber
Andrew’s Wave
Default Constant
![Page 22: Enterprise Miner: Data Exploration and Visualisation · SAS® ENTERPRISE MINER™ SAS® ENTERPRISE MINER™ • Modern, collaborative, easy-to-use data mining workbench • Sophisticated](https://reader033.vdocuments.us/reader033/viewer/2022052520/6080dbfc432df679c57facae/html5/thumbnails/22.jpg)
Copyr i g ht © 2012, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
DEMO
![Page 23: Enterprise Miner: Data Exploration and Visualisation · SAS® ENTERPRISE MINER™ SAS® ENTERPRISE MINER™ • Modern, collaborative, easy-to-use data mining workbench • Sophisticated](https://reader033.vdocuments.us/reader033/viewer/2022052520/6080dbfc432df679c57facae/html5/thumbnails/23.jpg)
Copyr i g ht © 2012, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
OUTLIER DETECTION
![Page 24: Enterprise Miner: Data Exploration and Visualisation · SAS® ENTERPRISE MINER™ SAS® ENTERPRISE MINER™ • Modern, collaborative, easy-to-use data mining workbench • Sophisticated](https://reader033.vdocuments.us/reader033/viewer/2022052520/6080dbfc432df679c57facae/html5/thumbnails/24.jpg)
Copyr i g ht © 2012, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
OUTLIER
DETECTIONDETECT AND TREAT
• Outlier Detection
• Filter Node
• Automated and Interactive filtering
• Identify and exclude extreme outliers
• Replacement Node
• Generates score code to process unknown levels when scoring
• Interactively specify replacement values for class and interval
levels
![Page 25: Enterprise Miner: Data Exploration and Visualisation · SAS® ENTERPRISE MINER™ SAS® ENTERPRISE MINER™ • Modern, collaborative, easy-to-use data mining workbench • Sophisticated](https://reader033.vdocuments.us/reader033/viewer/2022052520/6080dbfc432df679c57facae/html5/thumbnails/25.jpg)
Copyr i g ht © 2012, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
DEMO
![Page 26: Enterprise Miner: Data Exploration and Visualisation · SAS® ENTERPRISE MINER™ SAS® ENTERPRISE MINER™ • Modern, collaborative, easy-to-use data mining workbench • Sophisticated](https://reader033.vdocuments.us/reader033/viewer/2022052520/6080dbfc432df679c57facae/html5/thumbnails/26.jpg)
Copyr i g ht © 2012, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
SUMMARY
![Page 27: Enterprise Miner: Data Exploration and Visualisation · SAS® ENTERPRISE MINER™ SAS® ENTERPRISE MINER™ • Modern, collaborative, easy-to-use data mining workbench • Sophisticated](https://reader033.vdocuments.us/reader033/viewer/2022052520/6080dbfc432df679c57facae/html5/thumbnails/27.jpg)
Copyr i g ht © 2012, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
DATA EXPLORATION
AND VISUALISATIONSUMMARY
• Comprehensive data mining toolset
• Variety of visualisation and sampling methodologies
• Number of approaches to data and dimension reduction
• Importance of enhancing data prior to model development
• Garbage in = Garbage out (GIGO)
![Page 28: Enterprise Miner: Data Exploration and Visualisation · SAS® ENTERPRISE MINER™ SAS® ENTERPRISE MINER™ • Modern, collaborative, easy-to-use data mining workbench • Sophisticated](https://reader033.vdocuments.us/reader033/viewer/2022052520/6080dbfc432df679c57facae/html5/thumbnails/28.jpg)
Copyr i g ht © 2012, SAS Ins t i tu t e Inc . A l l r ights reser ve d . www.SAS.com
QUESTIONS AND ANSWERS
![Page 29: Enterprise Miner: Data Exploration and Visualisation · SAS® ENTERPRISE MINER™ SAS® ENTERPRISE MINER™ • Modern, collaborative, easy-to-use data mining workbench • Sophisticated](https://reader033.vdocuments.us/reader033/viewer/2022052520/6080dbfc432df679c57facae/html5/thumbnails/29.jpg)
Copyr i g ht © 2012, SAS Ins t i tu t e Inc . A l l r ights reser ve d . www.SAS.com