practicing data science a collection of case studies
TRANSCRIPT
![Page 1: Practicing Data Science A Collection of Case Studies](https://reader035.vdocuments.us/reader035/viewer/2022071613/6157d584ce5a9d02d46f9be4/html5/thumbnails/1.jpg)
© 2018 KNIME AG. All Right Reserved.
Practicing Data ScienceA Collection of Case Studies
@KNIME
Strata London , May 2 2019
![Page 2: Practicing Data Science A Collection of Case Studies](https://reader035.vdocuments.us/reader035/viewer/2022071613/6157d584ce5a9d02d46f9be4/html5/thumbnails/2.jpg)
© 2018 KNIME AG. All Rights Reserved.
A few Words about me
2
• I am Rosaria Silipo• Principal Data Scientist at KNIME• At least 20 years analyzing data
• Generally interesting projects become Case Studies• 22 case studies collected in a book• Almost 23
![Page 3: Practicing Data Science A Collection of Case Studies](https://reader035.vdocuments.us/reader035/viewer/2022071613/6157d584ce5a9d02d46f9be4/html5/thumbnails/3.jpg)
© 2018 KNIME AG. All Rights Reserved.
A Classic Data Science Project
It always starts with some data …
3
Data Preparation
Model Training
Model Optimization
Deployment
Data ManipulationData BlendingMissing Values HandlingFeature GenerationDimensionality ReductionFeature SelectionOutlier RemovalNormalizationPartitioning…
Model TrainingBag of ModelsModel SelectionEnsemble ModelsOwn Ensemble ModelExternal ModelsImport Existing ModelsModel Factory…
Parameter TuningParameter OptimizationRegularizationModel SizeNo. Iterations…
Performance MeasuresAccuracyROC CurveCross-Validation…
Files & DBsDashboardsREST APISQL Code ExportReporting…
Model Testing
![Page 4: Practicing Data Science A Collection of Case Studies](https://reader035.vdocuments.us/reader035/viewer/2022071613/6157d584ce5a9d02d46f9be4/html5/thumbnails/4.jpg)
© 2018 KNIME AG. All Rights Reserved. 4
Customer Intelligence: Churn Prediction
![Page 5: Practicing Data Science A Collection of Case Studies](https://reader035.vdocuments.us/reader035/viewer/2022071613/6157d584ce5a9d02d46f9be4/html5/thumbnails/5.jpg)
© 2018 KNIME AG. All Rights Reserved.
Churn Prediction: The Problem
CRM SystemData about your customer• Demographics• Behavior• Revenues
Model
• Churn Prediction• Upselling Likelihood• Product Propensity /NBO• Campaign Management• Customer Segmentation• …
5
![Page 6: Practicing Data Science A Collection of Case Studies](https://reader035.vdocuments.us/reader035/viewer/2022071613/6157d584ce5a9d02d46f9be4/html5/thumbnails/6.jpg)
© 2018 KNIME AG. All Rights Reserved.
Churn Prediction: The Training Workflow
6
![Page 7: Practicing Data Science A Collection of Case Studies](https://reader035.vdocuments.us/reader035/viewer/2022071613/6157d584ce5a9d02d46f9be4/html5/thumbnails/7.jpg)
© 2018 KNIME AG. All Rights Reserved.
Churn Prediction: The Deployment Workflow
7
YouTube: “Building a basic Model for Churn Prediction with KNIME” https://www.youtube.com/watch?v=RHsO10q7e2Y
EXAMPLES Server: 50_Applications/18_Churn_Prediction
![Page 8: Practicing Data Science A Collection of Case Studies](https://reader035.vdocuments.us/reader035/viewer/2022071613/6157d584ce5a9d02d46f9be4/html5/thumbnails/8.jpg)
© 2018 KNIME AG. All Rights Reserved. 9
Demand Prediction (Taxi)
![Page 9: Practicing Data Science A Collection of Case Studies](https://reader035.vdocuments.us/reader035/viewer/2022071613/6157d584ce5a9d02d46f9be4/html5/thumbnails/9.jpg)
© 2018 KNIME AG. All Rights Reserved.
Demand Prediction: The Problem
10
How many taxi do I need in NYC on Wednesday at 12:00?
How many customers?How many kW?How many diapers?
![Page 10: Practicing Data Science A Collection of Case Studies](https://reader035.vdocuments.us/reader035/viewer/2022071613/6157d584ce5a9d02d46f9be4/html5/thumbnails/10.jpg)
© 2018 KNIME AG. All Rights Reserved.
Demand Prediction: The Training Workflow
11
Training set
Test setR2 = 0.81
![Page 11: Practicing Data Science A Collection of Case Studies](https://reader035.vdocuments.us/reader035/viewer/2022071613/6157d584ce5a9d02d46f9be4/html5/thumbnails/11.jpg)
© 2018 KNIME AG. All Rights Reserved.
Demand Prediction: Deployment
12
On Wednesday at 12:00 we need 13k taxis
![Page 12: Practicing Data Science A Collection of Case Studies](https://reader035.vdocuments.us/reader035/viewer/2022071613/6157d584ce5a9d02d46f9be4/html5/thumbnails/12.jpg)
© 2018 KNIME AG. All Rights Reserved.
Automated Machine Learning
13
![Page 13: Practicing Data Science A Collection of Case Studies](https://reader035.vdocuments.us/reader035/viewer/2022071613/6157d584ce5a9d02d46f9be4/html5/thumbnails/13.jpg)
© 2018 KNIME AG. All Rights Reserved.
Interaction Points
Business analysts will simply access the KNIME WebPortal from any web browser..
![Page 14: Practicing Data Science A Collection of Case Studies](https://reader035.vdocuments.us/reader035/viewer/2022071613/6157d584ce5a9d02d46f9be4/html5/thumbnails/14.jpg)
© 2018 KNIME AG. All Rights Reserved.
Fraud/Anomaly Detection
15
![Page 15: Practicing Data Science A Collection of Case Studies](https://reader035.vdocuments.us/reader035/viewer/2022071613/6157d584ce5a9d02d46f9be4/html5/thumbnails/15.jpg)
© 2018 KNIME AG. All Rights Reserved.
Fraud Detection: The Problem
16
Transactions• Trx 1• Trx 2• Trx 3• Trx 4• Trx 5• Trx 6• …
Model
• Good• Good• Good
• Fraud• Good• Good• …
![Page 16: Practicing Data Science A Collection of Case Studies](https://reader035.vdocuments.us/reader035/viewer/2022071613/6157d584ce5a9d02d46f9be4/html5/thumbnails/16.jpg)
© 2018 KNIME AG. All Rights Reserved.
Fraud Detection: without Fraud Examples – Auto-encoder
17
• Trained with Back-Propagation on just “normal” transactions
• If distance > threshold => possible fraud
dis
tan
ce
![Page 17: Practicing Data Science A Collection of Case Studies](https://reader035.vdocuments.us/reader035/viewer/2022071613/6157d584ce5a9d02d46f9be4/html5/thumbnails/17.jpg)
© 2018 KNIME AG. All Rights Reserved.
Fraud Detection: without Fraud Examples
![Page 18: Practicing Data Science A Collection of Case Studies](https://reader035.vdocuments.us/reader035/viewer/2022071613/6157d584ce5a9d02d46f9be4/html5/thumbnails/18.jpg)
© 2018 KNIME AG. All Rights Reserved.
Fraud Detection deployed
Suspicious Transaction
![Page 19: Practicing Data Science A Collection of Case Studies](https://reader035.vdocuments.us/reader035/viewer/2022071613/6157d584ce5a9d02d46f9be4/html5/thumbnails/19.jpg)
© 2018 KNIME AG. All Rights Reserved. 20
Recommendation Engine
![Page 20: Practicing Data Science A Collection of Case Studies](https://reader035.vdocuments.us/reader035/viewer/2022071613/6157d584ce5a9d02d46f9be4/html5/thumbnails/20.jpg)
© 2018 KNIME AG. All Rights Reserved.
Recommendation Engines or Market Basket Analysis
21
Model Recommendation
IF =>
![Page 21: Practicing Data Science A Collection of Case Studies](https://reader035.vdocuments.us/reader035/viewer/2022071613/6157d584ce5a9d02d46f9be4/html5/thumbnails/21.jpg)
© 2018 KNIME AG. All Rights Reserved.
Market Basket Analysis: with Association Rules
![Page 22: Practicing Data Science A Collection of Case Studies](https://reader035.vdocuments.us/reader035/viewer/2022071613/6157d584ce5a9d02d46f9be4/html5/thumbnails/22.jpg)
© 2018 KNIME AG. All Rights Reserved.
Recommendation Engine: with Collaborative Filtering
![Page 23: Practicing Data Science A Collection of Case Studies](https://reader035.vdocuments.us/reader035/viewer/2022071613/6157d584ce5a9d02d46f9be4/html5/thumbnails/23.jpg)
© 2018 KNIME AG. All Rights Reserved.
Recommendation Engine/MBA: Deployment
![Page 24: Practicing Data Science A Collection of Case Studies](https://reader035.vdocuments.us/reader035/viewer/2022071613/6157d584ce5a9d02d46f9be4/html5/thumbnails/24.jpg)
© 2018 KNIME AG. All Rights Reserved. 25
Creative AI
![Page 25: Practicing Data Science A Collection of Case Studies](https://reader035.vdocuments.us/reader035/viewer/2022071613/6157d584ce5a9d02d46f9be4/html5/thumbnails/25.jpg)
© 2018 KNIME AG. All Rights Reserved.
Creative AI: The Problems
• Free Text Generation– Simulating a writing style
– Writing in different languages
– Providing an answer in a specific style
• Machine Translation
• Generating Candidates for Product Names
26
![Page 26: Practicing Data Science A Collection of Case Studies](https://reader035.vdocuments.us/reader035/viewer/2022071613/6157d584ce5a9d02d46f9be4/html5/thumbnails/26.jpg)
© 2018 KNIME AG. All Rights Reserved.
Deep Learning LSTM Network
27
On
e-ho
t enco
ded
character
Ch
aracter pro
bab
ilities
e - s- u – o - ho - u- s - e - <space>
![Page 27: Practicing Data Science A Collection of Case Studies](https://reader035.vdocuments.us/reader035/viewer/2022071613/6157d584ce5a9d02d46f9be4/html5/thumbnails/27.jpg)
© 2018 KNIME AG. All Rights Reserved.
Creative AI: The Training Workflow
28
![Page 28: Practicing Data Science A Collection of Case Studies](https://reader035.vdocuments.us/reader035/viewer/2022071613/6157d584ce5a9d02d46f9be4/html5/thumbnails/28.jpg)
© 2018 KNIME AG. All Rights Reserved.
Creative AI: The Deployment Workflow
29
![Page 29: Practicing Data Science A Collection of Case Studies](https://reader035.vdocuments.us/reader035/viewer/2022071613/6157d584ce5a9d02d46f9be4/html5/thumbnails/29.jpg)
© 2018 KNIME AG. All Rights Reserved.
Creative AI: Deployment and Results
Yo! This post is about generating free text with a deep learning networkparticularly it is about Brick X6,Phey, cabe, make you feel soom the way (I smoke good!) I probably make (What?)More money in six months,Than what's in your papa's safe (I'm serious)Look like I robbed a bank (Okay Okay)I set it off like Queen Latifah'Cause I'm living single I'm feeling cautiousI ain't scream when they served a subpoena (Can't go back to jail)I heard that he a leader(Who pood, what to be f*****' upThe baugerout Black alro Black X6, Phantom White X6 looks like a pandaGoin' out like I'm MontanaHundred killers, hundred hammers Black X6, Phantom White X6, pandaPockets swole, DannySellin' bar, candy Man I'm the macho like RandyThe choppa go Oscar for GrammyB**** n**** pull up ya pantyHope you killas understand meHey Panda, Panda Panda, Panda, Panda, Panda, PandaI got broads in AtlantaTwistin' dope, lean, and the FantaCredit cards and the scammersHittin' off licks in the bando
30
This License refers to version of the GNU General Public License. Copyright also means copyright-bick,Remade me any thing to his swordTo his salt and most hidden loose to be so for sings, but not in a libutt of his matter than that shall be sure as will be soldyeAs master compary, do not live in traitor.Bless thy five wits!-KentO pity!Sir, where is the patience now,That this is so far from the sea and some bidings to dismantleSo many folds of save and honest.-BrabantioI must not think the Turk of Cassio in the strange metting the cribles of a charmer be the reviling of libe to sayThat I can deceive him to the best advantage,In her prophetic fairs of a little to presently at your powers; whereof I thank you, sir.-AlbanyGloucester, I will prove upancy of his sport and first accuriorsand guard and talking on the white.-King LearWhere are the thief?Thou shalt never have the captains at the letterTo the Moor and thing we have not the better shall be sure as worth if he be anger—-ReganI pray you, have a countend more than think to do a proclaim’dthere of my heart, HotThe words save, honest, thief, master, traitor, and deceive seem to fit the context. Notice also that the dialogue sprouting from the start text of the license agreement interestingly involves mainly minor, less tragic characters from the plays.
Caro amico ti scrivo così mi distraggo un po'E siccome sei molto lontano più forte ti scriverò.Da quella prima folla strana, che aveva preso il suo nome, e di correre alla casa di don Abbondio, con un viso bene di non poterci andar la casa del padre Cristoforo, e gli disse che s'avvicinava all'uscio, e si mise a sparse di corsa, e di stare a sé, verso la strada di servizio, chiesto le parole che gli andavan dall'altra stanza, e con la sua condizione de' cappuccini, e di consigli ricerche di confidenza delle gride, nel suo passaggio, se non pensava con una certa ripugnanza a casa sua, che andavano a scomparire in un campo di buone ragioni che avevan potuto raccogliere i suoi pensieri, e di sopra non senza interrogare, che la sua avventura aveva fatto predicare, e con la forza d'un fatto come fuggitive che aveva preso il suo nome, e di correre alla casa di don Abbondio, con un cappuccino di quella sorte, con un certo sospiro, alzando le sue finestre, e le diede un'occhiata in carrozza. Si vendano a metter nelle mani di chi era stato a sedere sur una strada così fatta con le braccia in
![Page 30: Practicing Data Science A Collection of Case Studies](https://reader035.vdocuments.us/reader035/viewer/2022071613/6157d584ce5a9d02d46f9be4/html5/thumbnails/30.jpg)
© 2018 KNIME AG. All Rights Reserved.
Free Book as a Thank You
Free Copy of Practicing Data Science. A Collection of Case Studies Book from KNIME Press
https://www.knime.com/knimepress
with this code: STRATA-LONDON-2019
Expiration dateTue, 06/11/2019 - 23:59
34
![Page 31: Practicing Data Science A Collection of Case Studies](https://reader035.vdocuments.us/reader035/viewer/2022071613/6157d584ce5a9d02d46f9be4/html5/thumbnails/31.jpg)
© 2018 KNIME AG. All Rights Reserved.
Rate today ’s session
Session page on conference website O’Reilly Events App
![Page 32: Practicing Data Science A Collection of Case Studies](https://reader035.vdocuments.us/reader035/viewer/2022071613/6157d584ce5a9d02d46f9be4/html5/thumbnails/32.jpg)
© 2018 KNIME AG. All Rights Reserved. 37
The KNIME® trademark and logo and OPEN FOR INNOVATION® trademark are used by KNIME AG under license from KNIME GmbH, and are registered in the United States. KNIME® is also registered in Germany.