cloudbrew 2016 - building iot solution with service fabric
TRANSCRIPT
Onsight
Building IoT solution with Service Fabric
Onsight
Teemu Tapanila Microsoft Azure MVP Architect, OnSight Helsinki oy Microsoft Certified Trainer
[email protected]@TapanilaT
Onsight
Topics Why Azure Service Fabric Device Registration Data Ingestion Data processing Data analysis
Onsight
Microsoft Azure Service FabricA platform for reliable, hyperscale, microservice-based applications
Microservices
Application Programming Models
Azure
WindowsServer Linux
Hosted Clouds
WindowsServer Linux
Service Fabric
Private Clouds
WindowsServer Linux
High Availability
Hyper-Scale
Hybrid Operations
High Density
Rolling Upgrades Stateful
services
Low Latency Fast startup & shutdown
Container Orchestration & lifecycle management Replication &
FailoverSimple
programming models
Load balancing
Self-healingData Partitioning
Automated Rollback
Health Monitoring
Placement Constraints
Onsight
Battle-hardened for over 5 years
Azure Core Infrastructure
thousands of machines
Power BI
Intune
800k devices
Azure SQL Database
1.4 million databases
Bing Cortana
500m evals/sec
Azure Document
DB
billions transactions/wee
k
Skype for Business
Hybrid Ops
Event Hubs
20bn events/day
Onsight
What is a microservice? Is (logic + state) that is independently versioned,
deployed, and scaled Has a unique name that can be resolved
e.g. fabric:/myapplication/myservice Interacts with other microservices over well defined
interfaces and protocols like REST Remains always logically consistent in the presence of
failures Hosted inside a “container” (code + config) Can be written in any language and framework Developed by a small engineering team
Onsight
Types of microservices Stateless microservice
Has either no state or it can be retrieved from an external store
There can be N instances e.g. web frontends, protocol gateways etc.
Stateful microservice Maintain hard, authoritative state N consistent copies achieved through replication and
local persistence e.g. database, documents, workflow, user profile,
shopping cart etc.
Stateless microservice
Demo
A set of independent machines; physical or virtual
Typical datacenter
A set of machines that Service Fabric stitches together to form a cluster
Clusters can scale to 1000s of machines
Cluster: A federation of machines
Node
Node
Node
Node
Node
Node
Machine failure detection
Time = t1
83 76 50 4664 New Node arrived61
Time = t2
8361 50 46
Failures Detected
cluster reconfigured
83 76 64 50 46
Time = t0
Nodes failed
Cluster: System viewSystem Services Failover manager
Cluster manager
Naming
Image store
Service Fabric Explorer
Demo
Queues Storage
3-Tier service pattern
Front End(StatelessWeb)
StatelessMiddle-tierCompute
Cache
Scale with partitioned storage
Increase reliability with queues
Reduce read latency with caches
Manage your own transactions for state consistency
Many moving parts each managed differently
Load Balancer
StatefulMiddle-tierCompute
Stateful services: Simplify design, reduce latency
Front End(StatelessWeb)
data stores used for analytics and disaster recovery
Application state lives in the compute tier
Low Latency reads and writes
Partitions are first class for scale-out
Built in transactions
Fewer moving parts
Load Balancer
Onsight
Device Registration Device needs to be registered when it gets
connected to the system. This includes normally authentication keys and
serial number When device gets registered you should assing
it value which you will later use for sharding
Onsight
Data Ingestion Gathered data from 1 data source is not so
important normally Real-time data ingestion vs batch ingestion
Onsight
Data Ingestion
Onsight
Data processing Visibility to data averages on grouping level
(Machine, Building, Floor or etc) Logic based on value changes
StatefulMiddle-tierCompute
Data ingestion & Processing
Front End(StatelessWeb)
data stores used for analytics and disaster recovery
Load Balancer
Onsight
Data processing
Room Stateful Service
Floor Stateful Service
Building Stateful Service
Campus Stateful Service
Onsight
Data Analysis Data is stored outside of service fabric for
processing Most cases makes sense of using other tools to
build the analysis platform than Service Fabric
Microsoft Azure Service FabricA platform for reliable, hyperscale, microservice-based applications
Microservices
Application Programming Models
Azure
WindowsServer Linux
Hosted Clouds
WindowsServer Linux
Service Fabric
Private Clouds
WindowsServer Linux
High Availability
Hyper-Scale
Hybrid Operations
High Density
Rolling Upgrades Stateful
services
Low Latency Fast startup & shutdown
Container Orchestration & lifecycle management Replication &
FailoverSimple
programming models
Load balancing
Self-healingData Partitioning
Automated Rollback
Health Monitoring
Placement Constraints
