serverless architecture - windows user group …...roman kiss microsoft integration mvp (2003...

Roman KissMicrosoft Integration MVP (2003 – 2013)

August 2017

Serverless ArchitectureWith

Azure FunctionsEvent driven model for modern applications

Serverless architecture with Azure Functions (AF)

• Concept & Design• Architecture • Triggers & Bindings • Runtime compile AF• Pre-compile AF• Continues integration• Templatizing & Declarative Programming

• Usage• Using AF in the Azure Backend processes• Using AF in the Azure Event Grid model• Using AF in the stream pipeline• Using AF in the non-telemetry pipeline• Using AF for workflow mediators

• Deploying, Testing, Monitoring, Optimizing and Pricing

Gen 3 - FaaS

Serverless architecture with Azure Functions (AF)Function as a Service

Thanks to https://www.opsgility.com/blog/2017/02/23/evolutions-of-serverless-compute-in-microsoft-azure/

Serverless architecture with Azure Functions (AF)Break down your monolithic application

Thanks to https://vkbiztalk.wordpress.com/tag/serverless-architecture/

Serverless Principles

• Abstract completely underlying infrastructure• Break business logic to functions• Focus only on application code• True auto scaling in seconds• Integrate with external systems

Serverless Advantages

• Rapid Development• Truly Polyglot• Event & Metadata Driven• Reduces the complexity of software• Minimal Maintenance• Very Scalable• Simplifies packaging and deployment• Pay Per Use

Serverless Disadvantages

• Not efficient for long-running apps• Architectural complexity (business decomposition)• Vendor lock-in (FaaS is not compatible with each other)• Additional overhead for function/microservices calls• Cold-start• Missing out-of-the-box tools• Multitenancy problem


Thanks to [email protected]

Serverless architecture with Azure Functions (AF)API Gateway (Proxy)

API Gateway

REST

Serverless architecture with Azure Functions (AF)API Gateway + FaaS (Host)

Blob Storage

API Gateway

REST

Host

Host

Event Driven Models


Event Driven – Pull Model


Event Driven – Push Model


Azure Event Grid

Event Sources

Azure Event Grid Event Message

[{"id": "1807","eventType": "recordInserted","subject": "myapp/vehicles/motorcycles","eventTime": "2017-08-10T21:03:07+00:00","data": {

"make": "Ducati","model": "Monster"

},"topic": "/subscriptions/{subscription-id}/resourceGroups/{resource-group}/providers/Microsoft.EventGrid/topics/{topic}"

}]

public static async Task<HttpResponseMessage> Run(HttpRequestMessage req, CloudBlobContainer container, CloudQueue queue, CloudTable table, TraceWriter log)

{

// code

return req.CreateResponse(HttpStatusCode.OK, “Done”);}

Functions Programming Model

Thanks to https://www.slideshare.net/TokyoAzureMeetup/tokyo-azure-meetup-7-introduction-to-serverless-architectures-with-azure-functions-july-2016

Pre-Compiled Functions Programming Model

Serverless architecture with Azure Functions (AF)Distributed State Machine

Fan-out

Fan-in

Event Grid

Serverless architectureEventing with Event Grid and Azure Functions

Telemetry Pre-Processing

Azure Function – Telemetry preprocessor

Azure IoT HubAzure Function

htt

ps

Service Bus

Queue

Http Response

Http Trigger

Message Exchange PatternsPost-Processing, etc.

Pull and Push Data Stream to Power BI using an Azure Function

Pre-Processing IoT Events

Azure IoT Hub Azure Event HubAzure Function

Stream Stream Stream

Azure IoT Hub Bridge to the MQTT Broker

Azure IoT Hub Azure Function

Stream

EventHub Trigger

MQTT Client MQTT Broker

Azure Function – Telemetry Pusher to AWS

Azure IoT Hub

AWS

Storage blob

Archive

Azure Event Hub

Routes

Azure Function

Trigger

Azure Storage Blob

(5 minutes)

REST API

Storing The Device Twins Changes

Azure IoT Hub Azure Event Hub

Routes

Azure Function

Trigger

Azure Storage Table

Storage table

TwinChangeEvents

Azure IoT Hub Source of InputsRouting for stream events (ESP/CEP) and message processing

Custom Endpoints

Default Event Hub

Event Hub

Service Bus Topic

R

IoT Hub

Stream Analytics

Azure Functions

/messages/events

Service Bus Queue

Stream Analytics

Azure Functions

S1: 10 endpoints100 rules

Pre-Processing IoT Events

Custom Endpoints

Default Event Hub

R

IoT Hub

/messages/events

Non-Telemetry Path

Azure Functions

…

Event Hub

Telemetry Path

Non-Telemetry

Path

https://myNamespace.azure-devices.net/devices/DeviceXYZ/messages/events?api-version=2016-11-14

Stream Stream

(Device Emulator)

Converting Telemetry Stream to the Messaging

Custom Endpoints

Default Event Hub

R

IoT Hub

/messages/eventsService Bus Queue

Telemetry Path (stream pipeline)

Non-Telemetry Path

Service Bus QueueAzure Functions

…

Azure Function - VETER Message Mediator https://xxxxxx.azurewebsites.net/api/microservice?code=xxxxx&name=HelloCloud

/messages/devicebound

Service Bus Queue

Stream Analytics

IoT Hub

Telemetry Path

C2D Message

Azure Function

IoT Hub

Routes

Event Hub AFNAFN

Validation

Enrichment

Transformation

Ingestion

Things

Downstream

Consumers

PowerBI,

Service Bus,

Storage,

etc.

Storage blob

ref

Storage blob

archive

stream

Non-

telemetry

Blob

StorageService

Bus Queue

Using AFN in the IoT Stream PipelineTelemetry Path

IoT Hub AFN

Validation

Enrichment

Transformation

Route

Ingestion

Things

Downstream

Consumers

PowerBI,

Service Bus,

Storage,

etc.

Storage blob

ref

Storage blob

archive

stream

Non-telemetry

Blob StorageService Bus

Queue

Lite IoT Stream PipelineTelemetry Path

Azure IoT Hub Telemetry and Non-Telemetry PathsDevice-To-Cloud, Cloud-to-Device, Device-To-Device

/messages/servicebound/feedback


/devices/{deviceId}/messages/devicebound

CustomEndpoi

nt

Custom Service Bus

Queue

Stream

Analytics

IoT Hub

Azure

Function

Telemetry Path

Non-Telemetry Path

Devic

e(T

hin

g)

Rule ABC

No-RulesNo-Match

DefaultEndpoin

t

MQ

TT

, A

MQ

P,

HT

TP

/devices/{deviceId}/messages/events/{propB

ag}

Example of the Device-To-Device Communications

/messages/servicebound/feedback


Device1

CustomEndpoi

nt

Custom Service Bus

Queue

Stream

Analytics

IoT Hub

Azure

Function

Telemetry Path

Non-Telemetry Path

Rule ABC

No-RulesNo-Match

DefaultEndpoin

t

MQTT, AMQP,

HTTP

Raspberry Pi3 + Windows IoT Core

/devices/Device2/messages/events/{propBag}

/devices/Device1/messages/deviceboundhtt

ps

/devices/Device1/messages/events

/devices/Device2/messages/deviceBound

Device2

1

2

3

4

Azure IoT Hub

Device/GW/Thing(MQTT, AMQP, Https)

DirectMethodsonDeviceMethod InvokeMethod

Telemetry Data

C2D C2DQueueing

Device (Thing) Data

Device Twin (State) Data

Direct Actuary

Stream Data

async

Queue/Topic

sync

Change notificationsvia Routes mechanism

Azure Stream AnalyticsServices

(AMQP, REST)

AzureFunctions

Updating Device twin reported properties

/messages/servicebound/feedb

ack

/devices/{deviceId}/messages/devicebound

CustomEndpoi

nt

Custom Service Bus

Queue

Stream

Analytics

IoT Hub

Azure

Function

Telemetry Path

Non-Telemetry Path

Devic

e(T

hin

g) Rule ABC

No-RulesNo-Match

DefaultEndpoin

t

MQ

T

T

/devices/{deviceId}/messages/events/{propB

ag}

Read propertiesRead/Write properties

Read, receive change notifications

1

2

3

5

4

6Read/Write properties


C2D Messaging

Note: Step 2 can be replaced by new featurein the Routes mechanism

Azure Function - VETER Message Mediator https://xxxxxx.azurewebsites.net/api/microservice?code=xxxxx&name=HelloCloud

Designer Tool

Microservice VETER

Azure Function – Telemetry VETER Message Mediator

WCF Workflow Channel

Blob Storage

Dynamic Activity(xaml definition)

HTTP Triger Bridge to WCF Client

Event Hub Triger Bridge to WCF Client

? Triger Bridge to WCF Client

VETER Definition

ModelingBusiness, Connectivity, Hosting, …

Telemetry Data Stream with VETER Mediator

AppendixA

http://www.bsp.sk/seminar

serverless architecture - windows user group …...roman kiss microsoft integration mvp (2003...

Documents