microsoft technologies for data science -...

51
Microsoft Technologies for Data Science Architecture Edition Mark Tabladillo, Ph.D. Senior Data Scientist LogicBlox/Predictix

Upload: others

Post on 20-May-2020

1 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Microsoft Technologies for Data Science - Meetupfiles.meetup.com/1724872/MicrosoftIntegrationArchitects... · 2015-09-25 · SQL Server Analysis Services SQL Server Licensed Input:

Microsoft Technologies for Data ScienceArchitecture Edition

Mark Tabladillo, Ph.D.

Senior Data Scientist

LogicBlox/Predictix

Page 2: Microsoft Technologies for Data Science - Meetupfiles.meetup.com/1724872/MicrosoftIntegrationArchitects... · 2015-09-25 · SQL Server Analysis Services SQL Server Licensed Input:

Networking

Interactive

Page 3: Microsoft Technologies for Data Science - Meetupfiles.meetup.com/1724872/MicrosoftIntegrationArchitects... · 2015-09-25 · SQL Server Analysis Services SQL Server Licensed Input:
Page 4: Microsoft Technologies for Data Science - Meetupfiles.meetup.com/1724872/MicrosoftIntegrationArchitects... · 2015-09-25 · SQL Server Analysis Services SQL Server Licensed Input:
Page 5: Microsoft Technologies for Data Science - Meetupfiles.meetup.com/1724872/MicrosoftIntegrationArchitects... · 2015-09-25 · SQL Server Analysis Services SQL Server Licensed Input:

http://www.bizjournals.com/atlanta/subscriber-only/2014/07/11/top-employers-in-metro-atlanta.html

Page 6: Microsoft Technologies for Data Science - Meetupfiles.meetup.com/1724872/MicrosoftIntegrationArchitects... · 2015-09-25 · SQL Server Analysis Services SQL Server Licensed Input:
Page 7: Microsoft Technologies for Data Science - Meetupfiles.meetup.com/1724872/MicrosoftIntegrationArchitects... · 2015-09-25 · SQL Server Analysis Services SQL Server Licensed Input:
Page 8: Microsoft Technologies for Data Science - Meetupfiles.meetup.com/1724872/MicrosoftIntegrationArchitects... · 2015-09-25 · SQL Server Analysis Services SQL Server Licensed Input:
Page 9: Microsoft Technologies for Data Science - Meetupfiles.meetup.com/1724872/MicrosoftIntegrationArchitects... · 2015-09-25 · SQL Server Analysis Services SQL Server Licensed Input:
Page 10: Microsoft Technologies for Data Science - Meetupfiles.meetup.com/1724872/MicrosoftIntegrationArchitects... · 2015-09-25 · SQL Server Analysis Services SQL Server Licensed Input:

http://www.kdnuggets.com/polls/2015/analytics-

data-mining-data-science-software-used.html

http://products.office.com/en-us/excel

http://www.microsoft.com/en-us/server-cloud/products/sql-server/

http://pytools.codeplex.com/

http://azure.microsoft.com/en-us/services/hdinsight/

http://www.revolutionanalytics.com/

Page 11: Microsoft Technologies for Data Science - Meetupfiles.meetup.com/1724872/MicrosoftIntegrationArchitects... · 2015-09-25 · SQL Server Analysis Services SQL Server Licensed Input:
Page 12: Microsoft Technologies for Data Science - Meetupfiles.meetup.com/1724872/MicrosoftIntegrationArchitects... · 2015-09-25 · SQL Server Analysis Services SQL Server Licensed Input:
Page 13: Microsoft Technologies for Data Science - Meetupfiles.meetup.com/1724872/MicrosoftIntegrationArchitects... · 2015-09-25 · SQL Server Analysis Services SQL Server Licensed Input:

http://download.microsoft.com/download/F/C/2/FC21C981-

4351-4434-A78A-

3384CA7515BF/SQL_Server_2016_Deeper_Insights_Across_D

ata_White_Paper.pdf

Page 14: Microsoft Technologies for Data Science - Meetupfiles.meetup.com/1724872/MicrosoftIntegrationArchitects... · 2015-09-25 · SQL Server Analysis Services SQL Server Licensed Input:

SS

SQL

AS

NoSQL

Page 15: Microsoft Technologies for Data Science - Meetupfiles.meetup.com/1724872/MicrosoftIntegrationArchitects... · 2015-09-25 · SQL Server Analysis Services SQL Server Licensed Input:

SQL Server Data Mining: Analysis Serviceshttp://sqlserverdatamining.com

Page 16: Microsoft Technologies for Data Science - Meetupfiles.meetup.com/1724872/MicrosoftIntegrationArchitects... · 2015-09-25 · SQL Server Analysis Services SQL Server Licensed Input:
Page 17: Microsoft Technologies for Data Science - Meetupfiles.meetup.com/1724872/MicrosoftIntegrationArchitects... · 2015-09-25 · SQL Server Analysis Services SQL Server Licensed Input:

Data mining add-in for business analysts

• Ease of use

• Rich data mining

• Scalable

Page 18: Microsoft Technologies for Data Science - Meetupfiles.meetup.com/1724872/MicrosoftIntegrationArchitects... · 2015-09-25 · SQL Server Analysis Services SQL Server Licensed Input:
Page 19: Microsoft Technologies for Data Science - Meetupfiles.meetup.com/1724872/MicrosoftIntegrationArchitects... · 2015-09-25 · SQL Server Analysis Services SQL Server Licensed Input:
Page 20: Microsoft Technologies for Data Science - Meetupfiles.meetup.com/1724872/MicrosoftIntegrationArchitects... · 2015-09-25 · SQL Server Analysis Services SQL Server Licensed Input:
Page 21: Microsoft Technologies for Data Science - Meetupfiles.meetup.com/1724872/MicrosoftIntegrationArchitects... · 2015-09-25 · SQL Server Analysis Services SQL Server Licensed Input:

Rowset

Output

with Scores

Varchar

NVarchar

Office

PDF

Page 22: Microsoft Technologies for Data Science - Meetupfiles.meetup.com/1724872/MicrosoftIntegrationArchitects... · 2015-09-25 · SQL Server Analysis Services SQL Server Licensed Input:

Documents

Full-Text

Keyword

Index

“FTI”

iFilters

Semantic Document

Similarity Index “DSI”

Semantic

Database

Semantic

Key Phrase

Index –

Tag Index

“TI”

Page 23: Microsoft Technologies for Data Science - Meetupfiles.meetup.com/1724872/MicrosoftIntegrationArchitects... · 2015-09-25 · SQL Server Analysis Services SQL Server Licensed Input:

Simplified Chinese

British English

Portuguese

Chinese (Hong Kong SAR, PRC)

Spanish

Chinese (Singapore)

Chinese (Macau SAR)

Page 24: Microsoft Technologies for Data Science - Meetupfiles.meetup.com/1724872/MicrosoftIntegrationArchitects... · 2015-09-25 · SQL Server Analysis Services SQL Server Licensed Input:

Time in Seconds vs. Number of Documents

(2011 – K. Mukerjee, T. Porter, S. Gherman – Microsoft)

http://users.cis.fiu.edu/~lzhen001/activities/KDD2011Program/docs/p213.pdf

Page 25: Microsoft Technologies for Data Science - Meetupfiles.meetup.com/1724872/MicrosoftIntegrationArchitects... · 2015-09-25 · SQL Server Analysis Services SQL Server Licensed Input:
Page 26: Microsoft Technologies for Data Science - Meetupfiles.meetup.com/1724872/MicrosoftIntegrationArchitects... · 2015-09-25 · SQL Server Analysis Services SQL Server Licensed Input:
Page 27: Microsoft Technologies for Data Science - Meetupfiles.meetup.com/1724872/MicrosoftIntegrationArchitects... · 2015-09-25 · SQL Server Analysis Services SQL Server Licensed Input:

http://download.microsoft.com/download/3/B/9/3B9FBA69-8AAD-4707-830F-6C70A545C389/Introducing_Azure_Machine_Learning.pdf

Page 28: Microsoft Technologies for Data Science - Meetupfiles.meetup.com/1724872/MicrosoftIntegrationArchitects... · 2015-09-25 · SQL Server Analysis Services SQL Server Licensed Input:

http://datacamp.com

Page 29: Microsoft Technologies for Data Science - Meetupfiles.meetup.com/1724872/MicrosoftIntegrationArchitects... · 2015-09-25 · SQL Server Analysis Services SQL Server Licensed Input:

https://github.com/jakevdp/sklearn_pycon2015

Page 30: Microsoft Technologies for Data Science - Meetupfiles.meetup.com/1724872/MicrosoftIntegrationArchitects... · 2015-09-25 · SQL Server Analysis Services SQL Server Licensed Input:

http://azure.microsoft.com/en-us/services/machine-learning/

Page 31: Microsoft Technologies for Data Science - Meetupfiles.meetup.com/1724872/MicrosoftIntegrationArchitects... · 2015-09-25 · SQL Server Analysis Services SQL Server Licensed Input:
Page 32: Microsoft Technologies for Data Science - Meetupfiles.meetup.com/1724872/MicrosoftIntegrationArchitects... · 2015-09-25 · SQL Server Analysis Services SQL Server Licensed Input:
Page 33: Microsoft Technologies for Data Science - Meetupfiles.meetup.com/1724872/MicrosoftIntegrationArchitects... · 2015-09-25 · SQL Server Analysis Services SQL Server Licensed Input:
Page 34: Microsoft Technologies for Data Science - Meetupfiles.meetup.com/1724872/MicrosoftIntegrationArchitects... · 2015-09-25 · SQL Server Analysis Services SQL Server Licensed Input:
Page 35: Microsoft Technologies for Data Science - Meetupfiles.meetup.com/1724872/MicrosoftIntegrationArchitects... · 2015-09-25 · SQL Server Analysis Services SQL Server Licensed Input:

Mutable Immutable

Classic Open

Source

Java Scala

.NETNow Open Source

C#, C++,

VB.NET

F#

Page 36: Microsoft Technologies for Data Science - Meetupfiles.meetup.com/1724872/MicrosoftIntegrationArchitects... · 2015-09-25 · SQL Server Analysis Services SQL Server Licensed Input:

http://channel9.msdn.com/posts/Erik-Meijer-Functional-Programming-From-First-Principles

http://tomasp.net/

Page 37: Microsoft Technologies for Data Science - Meetupfiles.meetup.com/1724872/MicrosoftIntegrationArchitects... · 2015-09-25 · SQL Server Analysis Services SQL Server Licensed Input:
Page 38: Microsoft Technologies for Data Science - Meetupfiles.meetup.com/1724872/MicrosoftIntegrationArchitects... · 2015-09-25 · SQL Server Analysis Services SQL Server Licensed Input:

Technology Use Case

SQL Server Analysis

Services

SQL Server Licensed

Input: SQL Server, SSAS

Excel Add-In for SSAS

Semantic Search

Azure Machine

Learning

F#

Page 39: Microsoft Technologies for Data Science - Meetupfiles.meetup.com/1724872/MicrosoftIntegrationArchitects... · 2015-09-25 · SQL Server Analysis Services SQL Server Licensed Input:

Technology Use Case

SQL Server Analysis

Services

SQL Server Licensed

Input: SQL Server, SSAS

Excel Add-In for SSAS (Ditto) + Excellent Bridge for Excel Users

Semantic Search

Azure Machine

Learning

F#

Page 40: Microsoft Technologies for Data Science - Meetupfiles.meetup.com/1724872/MicrosoftIntegrationArchitects... · 2015-09-25 · SQL Server Analysis Services SQL Server Licensed Input:

Technology Use Case

SQL Server Analysis

Services

SQL Server Licensed

Input: SQL Server, SSAS

Excel Add-In for SSAS (Ditto) + Excellent Bridge for Excel Users

Semantic Search SQL Server: Volume Text Processing

Azure Machine

Learning

F#

Page 41: Microsoft Technologies for Data Science - Meetupfiles.meetup.com/1724872/MicrosoftIntegrationArchitects... · 2015-09-25 · SQL Server Analysis Services SQL Server Licensed Input:

Technology Use Case

SQL Server Analysis

Services

SQL Server Licensed

Input: SQL Server, SSAS

Excel Add-In for SSAS (Ditto) + Excellent Bridge for Excel Users

Semantic Search SQL Server: Volume Text Processing

Azure Machine

Learning

Students; Web Applications (R & Python)

Input: Azure-based Sources

F#

Page 42: Microsoft Technologies for Data Science - Meetupfiles.meetup.com/1724872/MicrosoftIntegrationArchitects... · 2015-09-25 · SQL Server Analysis Services SQL Server Licensed Input:

Technology Use Case

SQL Server Analysis

Services

SQL Server Licensed

Input: SQL Server, SSAS

Excel Add-In for SSAS (Ditto) + Excellent Bridge for Excel Users

Semantic Search SQL Server: Volume Text Processing

Azure Machine

Learning

Students; Web Applications (R & Python)

Input: Azure-based Sources

F# .NET Developers

Page 43: Microsoft Technologies for Data Science - Meetupfiles.meetup.com/1724872/MicrosoftIntegrationArchitects... · 2015-09-25 · SQL Server Analysis Services SQL Server Licensed Input:
Page 44: Microsoft Technologies for Data Science - Meetupfiles.meetup.com/1724872/MicrosoftIntegrationArchitects... · 2015-09-25 · SQL Server Analysis Services SQL Server Licensed Input:

Conference: http://www.kdd.org/

Page 45: Microsoft Technologies for Data Science - Meetupfiles.meetup.com/1724872/MicrosoftIntegrationArchitects... · 2015-09-25 · SQL Server Analysis Services SQL Server Licensed Input:
Page 46: Microsoft Technologies for Data Science - Meetupfiles.meetup.com/1724872/MicrosoftIntegrationArchitects... · 2015-09-25 · SQL Server Analysis Services SQL Server Licensed Input:
Page 47: Microsoft Technologies for Data Science - Meetupfiles.meetup.com/1724872/MicrosoftIntegrationArchitects... · 2015-09-25 · SQL Server Analysis Services SQL Server Licensed Input:
Page 48: Microsoft Technologies for Data Science - Meetupfiles.meetup.com/1724872/MicrosoftIntegrationArchitects... · 2015-09-25 · SQL Server Analysis Services SQL Server Licensed Input:
Page 49: Microsoft Technologies for Data Science - Meetupfiles.meetup.com/1724872/MicrosoftIntegrationArchitects... · 2015-09-25 · SQL Server Analysis Services SQL Server Licensed Input:

http://blogs.technet.com/b/machinelearning/

http://social.msdn.microsoft.com/forums/azure/en-US/home?forum=MachineLearning

http://sqlserverdatamining.com

http://marktab.net

http://curah.microsoft.com/342704/azure-machine-learning-videos-february-2015

Page 50: Microsoft Technologies for Data Science - Meetupfiles.meetup.com/1724872/MicrosoftIntegrationArchitects... · 2015-09-25 · SQL Server Analysis Services SQL Server Licensed Input:

http://datascience.sqlpass.org

Page 51: Microsoft Technologies for Data Science - Meetupfiles.meetup.com/1724872/MicrosoftIntegrationArchitects... · 2015-09-25 · SQL Server Analysis Services SQL Server Licensed Input: