industrial iot solution architecture design – from …...azure stream analytics has built-in,...
TRANSCRIPT
Industrial IoT Solution Architecture Design –From Connectivity to Data
Cheryl Hsu
Program Manager
Strategic Engagement & Industrial IoT, Microsoft
OPERATIONS
PEOPLE
PRODUCTSCUSTOMERS
INTELLIGENCE
DATA
IoT Enables a Digital Feedback LoopThe benefits are profound
IoT enables a “digital feedback loop”
that connects
▪ Customers
▪ Operations
▪ Products/Assets
▪ Employees
Our vision is to help businesses take advantage
of the digital feedback loop
INSIGHTS
ACTIONSTHINGS
Digital Feedback Loop
A realtime
connection
enables new
breakthrough
levels of insights
that in turn drive
informed actions
With IoT Access devices remotely
to diagnose and resolve
issues
Complete corrections
within hours, including
rerouting processes and
reconfiguring machines
Access comprehensive data
immediately to perform
root-cause analysis
1 2 3
With
fragmented
solutions
Maintain technicians onsiteto determine and resolve issues
Take days or weeks to reroute and reconfigure devices
Search for data needed for root-cause analysis
1 2 3
Respond and recover quickly
OR OR
With
fragmented
solutions
Solve storage on your own using capacity planning, capital purchases and on-going maintenance
Connect new devices later after customizations and integration efforts are complete
Take weeks or months to modify and extend systems with custom connections
1 2 3
Expand, change and scale easily
+ +
With IoT Exploit cloud solutions to
scale instantly and pay for
only what you need
Connect new devices now
with little or no
configuration required
Add to and extend systems
faster by building on the
extensible architecture
1 2 3
INSIGHTS
ACTIONSTHINGS
Updating devices
Provisioning devices
Device updates
Data storage
Cold path analytics
Warm path analytics
Hot path analytics
On device analytics
Securing data
Business process integration
Solution scale
High availability
Disaster recovery
Transport protocols
Cost management
Operations monitoring
Device lifecycle
Data ownership
Data visualization
Cloud-to-devicecommands
< ---- End-to-End Security ---- >
Industry and government compliance
Enterprise integration
Device recoveryInternationalization
HW certificationManufacturing scale
CI/CD
Drivers
Device commercialization
Enabling the Digital Feedback Loop used to be challenging
Before working on Architecture Design
Understand your business needs and challenges
Define value
proposition
Understand
organizational
impact
Assess
organizational
capability
Secure
business
stakeholder
buy-in
Input to
Architectural
Design
Getting Started
IoT Reference Architectures
IoT Reference Architecture
The latest Azure IoT cloud native
recommended architecture and latest
technology implementation
recommendations.
Provides:
▪ Overview of the IoT space
▪ Recommended subsystem factoring for solutions
▪ Prescriptive technology recommendations per subsystem
▪ Proven production ready architecture
▪ Proven technology implementation choices
▪ Recommendations for scaling systems
▪ Reference architecture implementations such as Remote
Monitoring and Connected Factory.
https://aka.ms/iotrefarchitecture
ActionsThings Insights
• Every organization has unique skills and experience
• Every IoT application has unique needs and considerations
Core Subsystems
UI & Reporting
Tools
Storage
Provision and send data from device to cloud
Stream processing and rules
evaluation over data
Visualize data and learnings
Store dataIntegrate with Business processes
Cloud Gateway (IoT Hub)
Stream Processing
Business IntegrationDevice
Management
IoT DevicesIoT DevicesIoT Devices
Azure IoT reference architecture
All Subsystems – Lambda Architecture
UI & Reporting
Tools
Warm PathStore
Stream processing and rules evaluation over data
Visualize data and learnings
Integrate with Business processes
Cloud Gateway (IoT Hub)
Stream Processing
Business Integration
Device Management
IoT DevicesIoT DevicesIoT Devices
Machine Learning
Cold PathStore
User Management
IoT DevicesIoT DevicesIoT Edge Devices
Bulk Device Provisioning
Data Transformation
Store data
Fast Path – Real Time Processing
Slow Path – Batch Processing
All Subsystems and Cross-Cutting needs
UI & Reporting
Tools
Warm PathStore
2.1 Stream processing and rules evaluation is done for device telemetry records
and events.
Visualize data and learnings
Cloud Gateway
Stream Processing
Business Integration
Device Management
IoT DevicesIoT DevicesIoT Devices
Machine Learning
Data Transformation
Cold PathStore
User Management
IoT DevicesIoT DevicesIoT Edge Devices
Bulk Device Provisioning
Deployment
High Availability and Disaster Recovery
Security
Store data
1. Devices send telemetry records or events to the cloud gateway.
2.1 Device telemetry data is transformed if needed.
2.2 Device data telemetry is stored.
3. Business Process integration (email, CRM,
etc.) is executed.
4. Device information is visualized and shown in UI.
4. ML processing is done for data.
The industry’s most agile, comprehensive, and secure portfolio
Azure IoT (PaaS)
Partner repeatable solutions
Azure IoT Solution Accelerators
Data and Analytics
Azure Time Series
Insights
Azure
Machine Learning
Cosmos DB
Azure Stream
Analytics
Azure Data Lake
Azure Data Lake
Analytics
Azure HD Insight
Visualization and Integration
Azure Logic Apps
Notification Hubs
Azure Websites
Microsoft Flow
Microsoft
Power BI
Azure Monitor
Azure Active Directory
Device support
Azure IoT
Device SDK
Azure IoT
certified devices
Security Program for
Azure IoT
Windows 10 IoT
IoT
Edge
Azure IoT Hub
Azure IoT Edge
IoT Hub Device
Provisioning Service
Microsoft Dynamics
Connected Field Service
Azure IoT Central
IoT SaaS
Solutions (PaaS)
Technologies (PaaS)
Solutions (SaaS)
SaaS, PaaS, and IaaS Guidance
Azure IoT solution
accelerators
Predictive Maintenance
Connected FactoryRemote Monitoring
Device Simulation
End-to-end implementation
Completely customizable
Open-source microservices based architecture
Device connectivity and management
Dashboards, visualization, and insights
Workflow automation and integration
Command and control
Preconfigured solutions
Accelerate time to value
Start quickly for
common IoT scenarios
Get started in minutes
Modify existing rules and alerts
Add your devices and begin tailor to your needs
Finish with your
IoT application
Fine-tuned to specific assets and processes
Highly visual for your real-time operational data
Integrate with back-end systems
Microsoft Azure IoT Solution Accelerators
https://www.azureiotsolutions.com/Accelerators
Azure IoT Connected Factory
Azure IoT Gateway SDK
On-Premise: Device Connectivity Cloud: Data Ingestion & Processing, Command & Control Cloud: Presentation
Indu
stri
al D
evic
es(O
PC-U
A S
erve
rs)
Hot Path Analytics
Azure Stream Analytics, Azure Storm, …
Azure IoT Hub
OPC Clients, Servers, ERP Portals,
OPC Graph Database and OPC UA .NET Standard Stack
JSON/AMQPUA Binary
Other Devices
OPC UA Client Module
IoT Proxy Module
UA Binary/AMQP
UA Binary
JSON/AMQP
Any
Firewall
Cold Path Analytics & Storage
Azure HD Insight, Azure Storage, SQL, DocDB, …
Presentation & Business Connections
Websites, Mobile Services
Dynamics, BizTalk Services, Notification Hubs
Microsoft
Dynamics
OPC UA Integration into Azure IoT
Industrial IoT
Critical needs in Industrial IoT
• Interoperability
• Scalability
• Precision
• Programmability
• Low latency
Connecting your factory with IoT
Global PLC & Industrial network market share
Global PLC market share as of 2017, by manufacturer Industrial network market shares 2016 according to HMS
https://www.statista.com/statistics/897201/global-plc-market-share-by-manufacturer/ https://iebmedia.com/index.php?id=11451&parentid=74&themeid=255&hft=93&showdet
ail=true&bb=1
Siemens
Mitsubishi
Omron
Schneider
Rockwell
B&R
GE
ABB
Others
Connectivity Complexity
Machinery Equipment
• CNC
• Injection Molding
• More…
Connectivity
• Ethernet capable?
• Serial standard?
• Others…
PLC
• Brand
• Driver
• OPC UA supported?
P
Manufacturing
Communication Layer
Communication Layer
Communication Layer
Enterprise & plant transformationEnterprise & Plant
Topology
ERP-LevelEnterprise Resources
MES-LevelManufacturing Execution Systems
Control-LevelMachine Controllers
Device-LevelSensors, Devices
Finance, HR, SCM, SRM, CRM …
Plant Schedules, Products, BOMs,
Routings, Quality, Performance …
HMI, SCADA, Alarms, Events,
Historians …
Automation Control, Safety, …
Information Systems
Responsive and dynamic cross-
industry Value Networks
Cross-plant performance,
coordination & flexibility
Production optimization &
reduced process costs
Intelligent equipment with
reduced failures & downtime
Enhanced
Azure IoT Gateway SDK
On-Premise: Device Connectivity Cloud: Data Ingestion & Processing, Command & Control Cloud: Presentation
Indu
stri
al D
evic
es(O
PC-U
A S
erve
rs)
Hot Path Analytics
Azure Stream Analytics, Azure Storm, …
Azure IoT Hub
OPC Clients, Servers, ERP Portals,
OPC Graph Database and OPC UA .NET Standard Stack
JSON/AMQPUA Binary
Other Devices
OPC UA Client Module
IoT Proxy Module
UA Binary/AMQP
UA Binary
JSON/AMQP
Any
Firewall
Cold Path Analytics & Storage
Azure HD Insight, Azure Storage, SQL, DocDB, …
Presentation & Business Connections
Websites, Mobile Services
Dynamics, BizTalk Services, Notification Hubs
Microsoft
Dynamics
OPC UA Integration into Azure IoT
Azure IoT Connected Factory Architecture
IoT Hub
VM
Linux VM (with multiple assembly lines)
Web App hosting
Solution Dashboard &
OPC UA Client
OPC UA Server
OPC UA Server
OPC UA Server
Gateway SDK with
OPC Proxy &
OPC Publisher
Modules
MES
Simulation
(OPC UA Client)
Telemetry path
Command & Control path
Time Series Insights
Real-Time Analytics
Real-time Analytics
“We are trying to get insights from our devices in real-time, etc.”
Real-time Analytics (aka Stream Analytics) is the phenomenon of processing data as soon as it is generated, to derive very quick analysis/insight for timely action.
I N T E G R A T I O N W I T H A Z U R E E V E N T H U B & I O T H U B
Azure Stream Analytics has built-in, first class integration with Azure Event Hubs and IoT Hub
▪ Data from Azure Event Hubs and Azure IoT Hub can be sources of Streaming Data to Azure Stream Analytics.
• The connections can be established through the Azure Portal without any coding. (see next slide)
▪ Azure Blob Storage is supported as a source of Reference Data.
▪ Azure Stream Analytics supports compression across all data stream input sources (Event Hubs, IoT Hub, and Blob storage).
Azure Blob Storage
Azure IoT Hub
Azure Event Hubs
Reference Data
Streaming Data
Streaming Data
Azure Stream Analytics
S T R E A M I N G - C A N O N I C A L O P E R A T I O N S
B I G D A T A S T R E A M I N G P A T T E R N W I T H A Z U R E
REAL-TIME
APPLICATIONS
REAL-TIME
DASHBOARDSBUSINESS / CUSTOM
APPS
(STRUCTURED)
LOGS, FILES AND MEDIA
(UNSTRUCTURED)
r
SENSORS AND IOT
(UNSTRUCTURED)
EVENT HUBS IoT HUB KAFKA on HDINSIGHT STREAM
ANALYTICS
STORM on
HDINSIGHT AZURE DATABRICKS
(Spark Streaming)
AZURE ML STUDIO
R SERVER AZURE DATABRICKS
(Spark ML)
Advanced Analytics
Advanced Analytics
“We are trying to predict and prevent in advance”
Advanced Analytics is the process of applying machine learning and/or deep learning techniques to data for the purpose of creating predictive/prescriptive insights.
A D V A N C E D A N A L Y T I C S - C A N O N I C A L O P E R A T I O N S
Model Creation & Deployment Process
Collect Data
Prepare Data
Train Model
Evaluate Model
Deploy Model
Insights Actions
Azure IoT Hub
Cloud
GatewayThings
Azure Stream
Analytics
IoT Pattern: Gaining Insight
Consistency
Insights Actions
Azure IoT Hub
Cloud
GatewayThings
IoT Pattern + Edge
Insights
Actions
Azure IoT Edge Deployment
Azure
IoT HubIoT Edge Device
Azure
Machine
Learning
Azure
Stream
Analytics
Azure
Functions
Azure
Cognitive
Services
Azure Container Service
© 2018 Microsoft Corporation. All rights reserved.