the future of microsoft data strategy : iot, analytics, with a focus on power bi
Post on 16-Apr-2017
2.154 Views
Preview:
TRANSCRIPT
Meeting du Swiss User Group
www.swiss-user-group.org
Focus on IoT, Analytics and Power BI
The Future of Microsoft Data Strategy
Speaker : Vincent ThavonekhamMicrosoft Practice Manager @VISEO
Microsoft MVP Azure
V2.2
@vThavo
Speaker : Vincent Thavonekham, Microsoft Practice Manager @VISEOMVP Azure, P-Seller ALM/DevOps
www.thavo.com @vThavo
Microsoft User Group LyonAzure User Group France
Author & co-author DevOps et Azure(France & international)In 2015 :- White paper: Zend / Programmez!- Round table: live streaming
for Electric Cloud, with Gene Kim (The Phoenix project)
@vThavo
AGENDA (60 min ~ 1 slide per min)
• Description of the Data, traditional vs. new
• What are the problems ?• Solutions, with a focus on Power BI• Conclusion
@vThavo
CONTENT
• Description of the Data (traditional vs. new)
• What are the problems ?• Solutions (with a focus on Power BI)• Conclusion
@vThavoCredits : http://blogs.technet.com/b/dataplatforminsider/archive/2013/06/03/sql-server-2014-unlocking-real-time-insights.aspx
INTRODUCTIONThe Data is an “old” topic addressed by SQL Server
@vThavo
VALU
E
DIFFICULTY
INTRODUCTIONKey types of analysis in the Big DataTraditional BI has evolved. Microsoft has grouped them into 2 categories:
Descriptive and Advanced Analytics (part of the « Data platform »
Source :Gartner
Data Scientists (Senior Doctor in Maths,
Astrophysics, …)
Data Platform leaps in innovation in 2015 !2012
SQL Server 2012
2013
HDInsight
“Our industry does not respect tradition – it only respects innovation.” – Satya Nadella
2015
Azure Machine Learning
Azure Document DB
HDInsight on Linux, HBase, Storm
Azure SQL DB with In-memory OLTP
Cortana Analytics Preview
IoT Services
2014
SQL Server 2014
APS
Azure SQL Database
Power BI
+ BI DataZen+ Revolution R
Note : OLAP : On Line Analytic Processing OLTP : On Line Transaction Processing
Source : Microsoft
@vThavo
INTRODUCTIONKey actors in 2015
2015 2015
@vThavo
Traditional data warehouse approach
BI & analyticsETL
Staging
+ Sharepoint + Power BI + O365
@vThavo
Breaking points of the traditional approach
Staging
Increasing data volumes
1
Real-time data
2
New data types
3
Cloud-born data
4 $100B spend on cloud
50% large orgs have hybrid by 2017
40% CRM sold are SaaS
50x Data growth 2010-2020
40ZB Digital Universe 2020
1Trillion Web pages
Source : Microsoft Convergence
@vThavo
INTRODUCTIONKey actors in 2016
2015 2016
Spectacular progress of the TOP 3 Leaders !
@vThavo
CONTENT
• Description of the Data (traditional vs. new)
• What are the problems ?• Solutions (with a focus on Power BI)• Conclusion
@vThavo
What are the problems ? And how to address them ?
• Many innovations in 2015• Too many data• Too many data types• Cold and hot data (i.e. traditional and real-
time)• Need for Agility (don’t know yet how to use
data)• Need really powerful compute to process
@vThavo
(Continued) What are the problems ? And how to address them ?
• EXCEL is no more the key peace of Microsoft Agile BI,
• Reporting Service is less and less used,
• OLAP cubes are no more the automatic solution
Everyone
3rd waveEnd user BI
A new generation of BI Today, BI extends to everyone
Analyst to end user
2nd waveSelf-service BI
IT to end user
1st waveTechnical BI
@vThavo
FRUSTRATIONS : The old Power BI and its « WOW » effect• Beyond the “WOW effect”, you’ll find the « Ouch » effect !
Ouch : EXCEL 250 Mb
limitations!
Ouch : what ?!?PowerQuery, PowerPivot, PowerView, PowerMap
Ouch : Sharepoint ?Ouch : Silverlight ?
Ouch : Excel Plugins !
Ouch: Office licences 2010/2013/365 Pro
Ouch : Need admin rights ?
Ouch : … « and what else ?? »
@vThavo
Event Queuing System
Collection Presentation and action
Event producers Transformation Long-term storage
Event Hubs
Storage adapters
Stream processingCloud gateways
(web APIs)
Field gateways
Applications
Search and query
Data analytics (Excel)
Web/thick client dashboards
Live Dashboards
Apache Storm on
HDInsight
Devices to take action
Kafka /RabbitMQ /ActiveMQ
Web and Social
Devices
Sensors / IoT
Azure Stream Analytics
HDFS
Azure DBs
Azure storage
HBase
CURRENT BIG DATA ECOSYSTEM !
Source : Microsoft
IoT AS AN INPUT TO THE BIG DATAZOOM
@vThavo
CONTENT
• Description of the Data (traditional vs. new)
• What are the problems ?• Solutions (with a focus on
Power BI)• Conclusion
@vThavo
SOLUTIONS (some of them!)
• Microsoft is Opening even more to the world,• Reinventing Microsoft• Many Open Source solutions supported• Following standards
• SaaS + VERY quick Onboarding• Start small with Free offers• “Suites” to simplify usage: Azure IoT Suites, Cortona Analytics Suites,• Focus on PowerBI.com and PowerBI Desktop
MICROSOFT
ANSWERS
Robust Architecture : LambdaStandards + Generic architecture = OPEN to non Microsoft technologies
Storage Cold Path Analysis
Apache Spark
Apache StormHot Path AnalysisPresentation
ApacheKafka
IoT Hub
Apache Storm on HDInsight
Apache HBASEon HDInsight
Hybrid : MS + Open Source
“Lambda Architecture”
MICROSOFT
ANSWERS
Azure IoT SuiteZOOM
« Think big, start small » : Microsoft referenced architecture
• Asset management• Remote monitoring• Preventive maintenanceMICROSOFT
ANSWERS
Cortana Analytics Suite
http://pocavenet-azure-iot-dashboard.azurewebsites.net/
ZOOMMICROSOFT
ANSWERS
Global Vision of the Microsoft IoTMICROSOFT
ANSWERS
@vThavo
Event Queuing System
Collection Presentation and action
Event producers Transformation Long-term storage
Event Hubs
Storage adapters
Stream processingCloud gateways
(web APIs)
Field gateways
Applications
Search and query
Data analytics (Excel)
Web/thick client dashboards
Live Dashboards
Apache Storm on
HDInsight
Devices to take action
Kafka /RabbitMQ /ActiveMQ
Web and Social
Devices
Sensors / IoT
Azure Stream Analytics
HDFS
Azure DBs
Azure storage
HBase
CURRENT BIG DATA ECOSYSTEM !
Source : Microsoft
@vThavo
Tendency with reporting / BI• Reporting Services are “brittle” • and needs IT experts to modify
• No more Reporting Services within TFS/VSTS• Reports not based on OLAP cube anymore• Replaced by a lightweight real-time reports and Power BI.com (1)
• No more Power BI inside EXCEL,• Replaced by a lightweight standalone Power BI.com
(1) https://blogs.msdn.microsoft.com/visualstudioalm/2016/01/11/announcing-reporting-capabilities-for-visual-studio-team-services
@vThavo
Zoom on Power BIStarting with a definition
“Power BI is a suite of business analytics tools to analyze data and share insights.
Monitor your business and get answers quickly with rich dashboards available on every device.”
MICROSOFT
ANSWERS
@vThavo
Zoom on Power BIStarting with a new Portal
===== December 31th 2015 ===== https://powerbi.com
@vThavo
New and easy prices
Version pro
Version gratuite et pro
===== December 31th 2015 =====
@vThavo
IT challenges
Management of Data in the cloud Security, private data, refresh and data updates.
Need to ease the usage
Need an easy tool for both the IT and the business users, to decrease time spent to create reports and visuals.
Keep data up to date Complex to refresh data from various sources, and manage increasing data volumes due to real time.
Integrating with the existing Time and cost.
IT Administration tasks Time spent for high SLA. Cost of the servers and maintenance.
Various Data sources Complex to connect to both On Premise and Cloud (SaaS) Data.
@vThavo
Power BI – Key benefits and differentiatorsPre-built content packs, consisting of
dashboards and reports, for popular SaaS solutions
Integration with familiar Microsoft products, and commitment for scale and availability in Azure
Intuitive data exploration by using natural language query
Real-time dashboard updates
Fast deployment, hybrid configuration, secure, and integrated with existing IT systems
Secure, live connection to your data sources, on-premises and in the cloud
@vThavo
How does Power BI addresses both the IT challenges and the business needs ?
@vThavo
Data connections Version gratuite et pro
@vThavo
Connectorsto popular SaaS applications
Aujourd’hui :
Soon:
Version gratuite et pro
@vThavo
Quick start samples Version gratuite et pro
@vThavo
Create reports
Desktop
Version gratuite et pro
@vThavo
Power BI DesktopPower BI Desktop
Desktop application to explore data and create interactive reports.
Easy usage, with Drag & Drop experience.
Publish directly to PowerBI.com.
Version gratuite et pro
@vThavo
Reports Version gratuite et pro
Azure Stream Analytics
Azure SQL/DW
HDInsight
AS Tabular
PBI Desktop
Excel
@vThavo
New custom visuals: Gallery (subset)
Funnel
Combo Chart
Gauge
Tree Map
Fill Map
Version gratuite et pro
@vThavo
Dashboard
Azure Stream Analytics
Azure SQL/DW
HDInsight
AS Tabular
PBI Desktop
Excel
Version gratuite et pro
@vThavo
« Ready to go » content packs :to be used against popular SaaS applications
Version gratuite et pro
@vThavo
« Ready to go » content packs :to be used against your data sources
Version pro
@vThavo
Live data sources Version pro
Consume live data sources with full interactivity
@vThavo
Power BI Personnal Gateway Version pro
Dataset
Access your on-premise data using the Data Connectivity Gataways
(Personal and Data Management)
@vThavo
Data updates : Refresh strategies
Real time data streaming in your dashboard- Using Power BI REST API- Using Azure Stream Analytics
Live connectivity to Microsoft SSAS - Auto-refresh- The Data are updated when the user accesses a dashboard or a report
Planned or automatic refresh- SaaS connectors (updates every 24h or planned or on-demand)
- Personal Gateway : planned or on demand refresh
Data refresh architecture
Live query
Popular SaaS Solutions
Live dashboards and reports
Dynamics Marketing
Power BI
SQL Server Analysis Services (SSAS)
Secure Credential Store
PRO Version1 million rows/hour
FREE Version10 000 rows/hour
PRO VersionEvery hour
FREE VersionEvery day
https://support.powerbi.com/knowledgebase/articles/474669-refresh-data-in-power-bi
@vThavo
Sharing reports and personal Dashboards
Version gratuite et pro
Share dashboards with others within your orgianization. • Shared dashboards (and related reports) are Read-Only. • They cannot be saved volontarily• The users cannot access the underlaying datasource• The users always have an up to date view
@vThavo
Team working collaboration
Version pro
Team working with Office 365 groups within Power BI
Create, publish and display contents packs of the company
Manage access control and sharing via Active Directory
groupes
@vThavo
Stay connected from any mobile devices
• Support for iPad, iPhone, and Windows
• Get alerts when your data changes•
Share and collaborate with your team. Take collaborative décisions
Version gratuite et pro
@vThavo
Query your data by asking a question in Natural language
Version gratuite et pro
@vThavo
Push data
Push your data directly into Power BI from your apps.
Integration
Add Power BI to the workflows of your apps.
Develop a real-time connection from your data sources, applications, IoT devices, …
Real-time
{ name":{String}, "tables":[ ..." ]}
0110010100001010
“name”:[string],“columns”:[String]“daType”:[String]
{ "name":{String}, "columns":[ { "name":{String}, "dataType":{String} },
Visuals
Create your own visuals for Power BI
Power BI pour les developpeurs
Version gratuite et proDéveloppements spécifiques
@vThavo
Power BI REST APIsPower BI Desktop
Prepare Explore ShareReport
0110
001
0001
0110
Data sources
SaaS solutionse.g. Marketo, Salesforce, GitHub, Google analytics
On-premises datae.g. Analysis Services
Organizational content packsCorporate data sources or external data services
Azure servicesAzure SQL, Stream Analytics…
Excel filesWorkbook data / data models
Power BI Desktop filesData from files, databases, Azure, and other sources
Power BI
Power BI service
Data refresh
Visualizations
Live dashboards
Content packs Sharing & collaborationNatural language query
Reports
Datasets0100110101
Power BI Overview
@vThavo
Very frequent updates• Example in November 2015
@vThavo
Only ONE access point : Powerbi.com(for both Desktop and Cloud Service)
Sign up for a free Preview account
Take the Power BI Tour
@vThavo
EXAMPLE 1 : Integration with Azure servicesReal-time dashboards with Azure Stream Analytics
Use Azure Stream Analytics to push live, streaming data to Power BI• Enables real-time dashboards at scale, over data from devices and applications• Can stream millions of events per second• Can perform aggregation over time windows
@vThavo
EXAMPLE 2 : Integration with Azure servicesAdvanced analytics
Coalesce Azure services together to drive advanced analytics:
• Azure HDInsight: Big Data processing• Azure Machine Learning:
Predictive analytics• Azure Data Factory:
Orchestration at scale
@vThavo
CONCLUSION• STOP thinking Data the traditional way
• Nadella Satya : « Mobile First, Cloud First »
• “Microsoft: opening the Windows to the world” ((ref Edouard Danel, 2016)
• “From the inside to the outside” & “From the outside to the inside”
• Microsoft is facilitating (again) the adoption of the new technologies• The NEW Microsoft Power BI (Power BI .com and Power BI Desktop)• Azure Machine Learning, Cortona Analytics, …
@vThavo
Next eventWorldwidein Lyon, France,www.Thavo.com
April 16th 2016
MY SLIDES ARE AVAILABLE ON SLIDESHARE
REFERENCES
http://www.slideshare.net/VincentThavonekham/the-future-of-microsoft-data-strategy-iot-analytics-with-a-focus-on-power-bi
April 2015, Office Mix, IoT and Stream Analytics : https://mix.office.com/watch/12fv2cpp112bv
March 2015 http://www.widriksson.com/power-bi-limitations-considerations March 2015 http://sqlmag.com/power-bi/introducing-free-power-bi-designer-desktop-app Sept 2015 http://www.lemondeinformatique.fr/actualites/lire-pourquoi-power-bi-est-l-avenir-d-excel-62259.html
Meeting du Swiss User Group
www.swiss-user-group.org
top related