azure ml: from basic to integration with custom applications

Azure Machine Learningfrom basic to integration with custom application

Davide Mauri

Join the conversation on Twitter: @DevWeek // #DW2016 // #DevWeek


About MeMicrosoft SQL Server MVPWorks with SQL Server from 6.5, on BI from 2003Specialized in Data Solution Architecture, Database Design, Performance Tuning, High-Performance Data Warehousing, BI, Big DataPresident of UGISS (Italian SQL Server UG)Regular Speaker @ SQL Server eventsConsulting & Training, Mentor @ SolidQE-mail: [email protected]: @mauridb Blog: http://sqlblog.com/blogs/davide_mauri/default.aspx

mailto:[email protected]

http://sqlblog.com/blogs/davide_mauri/default.aspx


Agenda• Machine Learning, what’s that?

• Supervised & Unsupervised methods• Tools & Languages

• Experimenting On-Premises• IPython & R• SQL Server 2016 and R

• Azure Machine Learning• AzureML Studio• Jupyter Notebooks

• Integrating AzureML into custom applications

Machine LearningWhat’s that?


Machine Learning• Algorithms that learn from data

• Nothing really new from a scientific point of view• "Field of study that gives computers the ability to learn without being

explicitly programmed“ - 1959, Arthur Samuel

• Requires *a lot* of compute power (even for not-so-big-data)• Azure, here we come!


http://en.wikipedia.org/wiki/Arthur_Samuel

Machine Learning• Very useful for

• Identify unknown and complex pattern • Identify hidden correlations• Automatically classify data• Predict future trend and/or values basing on past knowledge


Machine Learning• Thanks to the cloud it’s now possible to integrate ML Algorithms into

Line-Of-Business applications• Choose the algorithm• Train it• Expose as a RESTful Web Service• Call it from you App• You’re Happy


Machine Learning• Two main categories (but sometimes are even divided in up to five

categories!)• Supervised• Unsupervised

• Supervised: humans (usually) teach to algorithms what is the expected result

• Unsupervised: algorithms tries to autonomously identify patterns and rules in given dataset


Languages• Most common languages used for machine learning

• R• Python

• Less common but on the rise• Julia• Scala• Go• Rust


Tools - Python• Python Packages

• Scikit-Learn• SciPy, NumPy, Pandas, Matplotlib, Seaborn

• Jupyter (was: IPython)• Anaconda

• Microsoft Data Science Virtual Machine• Pytools for Visual Studio


http://scikit-learn.org/stable/

http://scikit-learn.org/stable/

http://www.scipy.org/

http://www.numpy.org/

http://pandas.pydata.org/

http://matplotlib.org/

https://jupyter.org/

http://ipython.org/

https://www.continuum.io/why-anaconda

http://blogs.technet.com/b/machinelearning/archive/2015/11/23/announcing-the-availability-of-the-microsoft-data-science-virtual-machine.aspx

http://microsoft.github.io/PTVS/

http://microsoft.github.io/PTVS/

Tools - R• R• Microsoft Open R Portal

• Microsoft R Open (MRO)

• RStudio• R Tools for Visual Studio• Microsoft Data Science Virtual Machine• Anaconda

• https://www.continuum.io/conda-for-r


https://cran.r-project.org/

https://mran.revolutionanalytics.com/

https://www.rstudio.com/

https://www.visualstudio.com/en-us/features/rtvs-vs.aspx

http://blogs.technet.com/b/machinelearning/archive/2015/11/23/announcing-the-availability-of-the-microsoft-data-science-virtual-machine.aspx

https://www.continuum.io/why-anaconda

https://www.continuum.io/conda-for-r

Datasets• To learn ML, sample and well-known datasets are needed

• Here some places where nice Datasets can be found• http://archive.ics.uci.edu/ml/datasets.html • http://www.kdnuggets.com/datasets/index.html • http://homepages.inf.ed.ac.uk/rbf/IAPR/researchers/MLPAGES/mldat.htm • https://en.wikipedia.org/wiki/Data_set#Classic_datasets • https://mran.revolutionanalytics.com/documents/data/


http://archive.ics.uci.edu/ml/datasets.html

http://www.kdnuggets.com/datasets/index.html

http://homepages.inf.ed.ac.uk/rbf/IAPR/researchers/MLPAGES/mldat.htm

https://en.wikipedia.org/wiki/Data_set#Classic_datasets

https://mran.revolutionanalytics.com/documents/data/

IRIS Dataset• 150 instances of Iris Flowers

• 3 classes: Virginica, Versicolor, Setosa• 4 features: Sepal Width & Length, Petal Width & Length

• One of the most used for educational purposes• Simple, but….• Un class is linearly separable • Other two classes are NOT linearly separable

• Available at UC Irvine Machine Learning Repository• http://archive.ics.uci.edu/ml/datasets/Iris


http://archive.ics.uci.edu/ml/datasets/Iris

IRIS Datasethttp://www.anselm.edu/homepage/jpitocch/genbi101/diversity3Plants.html



DEMOExperiments On-Premises

SQL Server 2016 and R


Native support for R in SQL Server 2016!


DEMOExperiments with R and SQL Server 2016

Azure ML StudioOn the Cloud!


AzureML Studio• www.azureml.com

• Azure ML Studio• Web application (“Workspace”) for developing ML solutions• Part of the “Cortana Analytics Intelligence Suite”

• Development Process• Experiment -> Score -> Evaluate• Publish


http://www.azureml.com/

AzureML Studio• “Democratize Machine Learning”

• Free Tier Available• 10 GB Storage Space• 1h max experiment duration• Staging Web API

• Standard Tier• Costs per “Seat”, Studio and API Usage• https://azure.microsoft.com/en-us/pricing/details/machine-learning/


https://azure.microsoft.com/en-us/pricing/details/machine-learning/

AzureML Studio• Fully Interactive Environment

• Fully Integrated with Azure Ecosystem, but not only that • Very easy to use external data sources

• Support Jupyter/IPython Notebooks!• Even more Interative!



DEMOExperiments On-Premises

AzureML Web Services• Once the experiment has been created, tested and validated

• NOTE! AzureML Web Services *does not* support CORS right now• So it’s ok to call from a server-side script• If you want to call it directly from Javascript you need to go through Azure

Management API• And configure the Policy to allow CORS


AzureML Web Services• Can be created also on-premises with R and Python



DEMOExperiments With Web Services

What’s more• In addition to the “usual” Machine Learning objects Azure offers

specialized API and the “Cognitive Services”• https://www.microsoft.com/cognitive-services • https://gallery.cortanaintelligence.com/machineLearningAPIs


https://www.microsoft.com/cognitive-services

https://gallery.cortanaintelligence.com/machineLearningAPIs


Thanks!Questions?


Demos available on GitHubhttps://github.com/yorek/devweek2016

https://github.com/yorek/devweek2016

azure ml: from basic to integration with custom applications

Data & Analytics