Download - Testing in the Age of Machine Learning
![Page 1: Testing in the Age of Machine Learning](https://reader034.vdocuments.us/reader034/viewer/2022042723/58ecf7551a28ab64718b4761/html5/thumbnails/1.jpg)
@vinaya1980 @hrishikarekar
TESTING IN THE AGE OF MACHINE LEARNING
March 2016
![Page 2: Testing in the Age of Machine Learning](https://reader034.vdocuments.us/reader034/viewer/2022042723/58ecf7551a28ab64718b4761/html5/thumbnails/2.jpg)
https://www.youtube.com/watch?v=DCgHsxISE0Q
W H E R E A R E W E H E A D E D
httpsweforum.org/agenda/2017/01/worried-about-ai-taking-your-job-its-already-happening-in-japan?utm_content=buffera14af&utm_medium=social&utm_source=twitter.com&utm_campaign=buffer
![Page 3: Testing in the Age of Machine Learning](https://reader034.vdocuments.us/reader034/viewer/2022042723/58ecf7551a28ab64718b4761/html5/thumbnails/3.jpg)
THE WORLD OF ARTIFICIAL INTELLIGENCE
NLP
Neural Networks
MACHINE LEARNING
DEEP LEARNING
…..
![Page 4: Testing in the Age of Machine Learning](https://reader034.vdocuments.us/reader034/viewer/2022042723/58ecf7551a28ab64718b4761/html5/thumbnails/4.jpg)
THE FUTURE IS ALREADY HERE
Google RankBrain
Assistants
![Page 5: Testing in the Age of Machine Learning](https://reader034.vdocuments.us/reader034/viewer/2022042723/58ecf7551a28ab64718b4761/html5/thumbnails/5.jpg)
HOW DOES IT WORK
Supervised
• Labelled data• Given new
data, predict outcome
• Classification
Unsupervised
• No labels• Find hidden
structures• Clustering
Reinforcement
• Decision process
• Actions are rewarded or punished
• Learns to optimize rewards
![Page 6: Testing in the Age of Machine Learning](https://reader034.vdocuments.us/reader034/viewer/2022042723/58ecf7551a28ab64718b4761/html5/thumbnails/6.jpg)
http://scikit-learn.org/
LEARNING FROM DATA
![Page 7: Testing in the Age of Machine Learning](https://reader034.vdocuments.us/reader034/viewer/2022042723/58ecf7551a28ab64718b4761/html5/thumbnails/7.jpg)
SUPERVISED LEARNING - CLASSIFCATION
http://www.mikedeff.in/MLIntro.PNG
![Page 8: Testing in the Age of Machine Learning](https://reader034.vdocuments.us/reader034/viewer/2022042723/58ecf7551a28ab64718b4761/html5/thumbnails/8.jpg)
INTO THE REALM OF PROBABILITIES
Y = f ( x ) Y ≈ f ( x )
What is scrum ?
{ "Prediction": { "details": { "Algorithm": "SGD", "PredictiveModelType": "MULTICLASS" }, "predictedLabel": "definition", "predictedScores": { "advantages": 0.0001860455668065697, "characteristics": 0.00006915141420904547, "compare": 0.00017757616296876222, "definition": 0.9970965385437012, "disadvantages": 0.0000534967657586094, } }}
Can you tell me about scrum ?
{ "Prediction": { "details": { "Algorithm": "SGD", "PredictiveModelType": "MULTICLASS" }, "predictedLabel": "definition", "predictedScores": { "advantages": 0.01977257989346981, "characteristics": 0.022757112979888916, "compare": 0.008386141620576382, "definition": 0.21092116832733154, "disadvantages": 0.04002799838781357 } }}
![Page 9: Testing in the Age of Machine Learning](https://reader034.vdocuments.us/reader034/viewer/2022042723/58ecf7551a28ab64718b4761/html5/thumbnails/9.jpg)
TOLERANCE LEVELS
Y ≈ f ( x )
Know the probability that is within acceptable limits
![Page 10: Testing in the Age of Machine Learning](https://reader034.vdocuments.us/reader034/viewer/2022042723/58ecf7551a28ab64718b4761/html5/thumbnails/10.jpg)
EVALUATE WITH DIFFERENT MODELS
Evaluate against a set of algorithms to iterate towards a model that’s closest representation and for further tuning
https://s3.amazonaws.com/MLMastery/MachineLearningAlgorithms.png?__s=h4reg8jqwyg4sz3bzdqf
![Page 11: Testing in the Age of Machine Learning](https://reader034.vdocuments.us/reader034/viewer/2022042723/58ecf7551a28ab64718b4761/html5/thumbnails/11.jpg)
EVALUATION – DATA SET APPROACHES
Random split• 70% train, 30% test
K-fold cross validation
Split into 3 datasets• #1 Train on 1 and 2, test on 3• #2 Train on 2 and 3, test on 1• #3 Train on 1 and 3, test on 2
Never use the same dataset for training and evaluating
![Page 12: Testing in the Age of Machine Learning](https://reader034.vdocuments.us/reader034/viewer/2022042723/58ecf7551a28ab64718b4761/html5/thumbnails/12.jpg)
ITERATIVE – LEARNING PROCESS
Be prepared to throw the model and start again
![Page 13: Testing in the Age of Machine Learning](https://reader034.vdocuments.us/reader034/viewer/2022042723/58ecf7551a28ab64718b4761/html5/thumbnails/13.jpg)
MODEL ACCURACY - CONFUSION MATRIX
Strive for better models, not 100% accurate.
![Page 14: Testing in the Age of Machine Learning](https://reader034.vdocuments.us/reader034/viewer/2022042723/58ecf7551a28ab64718b4761/html5/thumbnails/14.jpg)
OVERFITTING
![Page 15: Testing in the Age of Machine Learning](https://reader034.vdocuments.us/reader034/viewer/2022042723/58ecf7551a28ab64718b4761/html5/thumbnails/15.jpg)
MODEL IS AS GOOD AS THE TRAINING DATA
If all of the algorithms perform poorly,• it maybe worth considering if there is a lack of learning
structure in the data set
• some transformation needed to make the structure more learnable • remove unnecessary noise - stop words are typically
removed because they cause unnecessary noise)
![Page 16: Testing in the Age of Machine Learning](https://reader034.vdocuments.us/reader034/viewer/2022042723/58ecf7551a28ab64718b4761/html5/thumbnails/16.jpg)
SUMMARY
Machine Learning applications demand a shift in testing approach
• Use objective acceptance levels to evaluate the application
• Express test outcomes in statistical terms
• Have a high level understanding of the underlying working of the application
![Page 17: Testing in the Age of Machine Learning](https://reader034.vdocuments.us/reader034/viewer/2022042723/58ecf7551a28ab64718b4761/html5/thumbnails/17.jpg)
@vinaya1980 @hrishikarekar
THANK YOU