1

Tamir DresherSenior Software ArchitectJ

Cloud Patterns

June 2016

Cloud Patterns

http://en.wikipedia.org/wiki/List_of_cloud_types

2

333

• Software architect, consultant and instructor• Software Engineering Lecturer @ Ruppin Academic Center• Reactive Extensions in .NET (Manning)

@tamir_dreshertamirdr@codevalue.nethttp://www.TamirDresher.com.

About Me

cloudvirtualization

Elasticity

Self service

Automation

Pay-as-you-go

4

cloudvirtualization

Elasticity

Self service

Automation

Pay-as-you-go

5

cloudvirtualization

Elasticity

Self service

Automation

Pay-as-you-go

6

cloudvirtualization

Elasticity

Self service

Automation

Pay-as-you-go

7

cloudvirtualization

Elasticity

Self service

Automation

Pay-as-you-go

8

cloudvirtualization

Elasticity

Self service

Automation

Pay-as-you-go

9

cloudvirtualization

Elasticity

Self service

Automation

Pay-as-you-go

Scalability Cost effectiveness(Cost Oriented Architecture)

10

StorageStatic Content DeliveryDirect Upload PatternValet Key

ComputingQueue/Message CentricPoison MessagesServerless architecture

Failure ManagementRetry LogicCircuit BreakerExternal Configuration Store

Agenda

11

Storage ResourcesPatterns

12

Basic web application

13

Loading the static files

14

Static files on webserver antipattern

Load increaseCost cost costUpdate requires deploymentLimited storage on server

15

Static Content Hosting Pattern

Azure Blob/AWS S3 etc.

webserver

client

GET [URL]

Page with links

Static content from storage service

16

17

Storage Limits Mitigation - Replication

Multiple blobsMultiple containers\bucketsMultiple accounts Balancing?

Geolocation?

Maintenance?

18

Content Delivery Network (CDN) Pattern

storage

19

CDN Considerations

The GoodLow latencyCoping with peaks without application scale

The BadVersioning is hardDeployment is more complex

The UglySingle point of failureCost implications

20

Uploading to storage - traditional

webserver Azure Blob/AWS S3 etc.

21

Uploading to storage – Direct Upload Pattern

Azure Blob/AWS S3 etc.

webserver

client

Storage URL

22

GET\POST [metadata]

Valet Key Pattern

Azure Blob/AWS S3 etc.

webserver

client

GET\POST [metadata]

Storage URL +Create SAS/Pre-signed Object URL

23

Valet Key Pattern - Azure SAS (Shared Access Signature)

Azure Blob/AWS S3 etc.

webserver

client

GET [URL]

Storage URL

https://myaccount.blob.core.windows.net/container/blob.txt?sv=2015-04-05&st=2015-04-29T22%3A18%3A26Z&se=2015-04-30T02%3A23%3A26Z&sr=b&sp=rw&sip=168.1.5.60-168.1.5.70&spr=https&sig=Z%2FRHIX5Xcg0Mq2rqI3OlWTjEg2tYkboXr1P9ZUXDtkk%3D

VersionStart timeExpiry timeResource typePermissionIP rangeProtocol

Signature

24

Computing ResourcesPatterns

26

27

https://twitter.com/ZeldaDorant/status/71839063717172019228

G. Hohpe: Your Coffee Shop Doesn’t Use Two-Phase Commit, IEEE Software, 2005http://www.enterpriseintegrationpatterns.com/docs/IEEE_Software_Design_2PC.pdf

29

Synchronous coffee shop

coffee

30

Synchronous coffee shop under load

coffee#!$@

Cashier == ServerBasically, this is a scalability issueRunning in the cloud (potentially) makes scaling problem appear faster 31

This what happens when the coffee shop closes the doors

32

• Add More Nodes• More Cashiers

• Load Distribution• Round Robin• Performance• Other (location, expertise, etc)

• Add More Resources to a Node• Faster cashier

• Better CPU to the server• More Memory

• Faster Coffee machine• Use a better algorithm/service

• Limited

Scaling

Scale Up Scale Out

33

Autoscale

aws autoscaling create-auto-scaling-group --auto-scaling-group-name my-lb-asg --launch-configuration-name my-lc --availability-zones "us-west-2a" "us-west-2b" --load-balancer-names "my-lb" --max-size 5 --min-size 1 --desired-capacity 2

LOOK MA, NO CODE!34

Asynchronous Coffee Shop – Barista model

coffee.

36

Message Centric

Website worker::

workerWebservice

37

Message Centric – Variable Instance Size & Amount

38

Message Centric – Load Leveling

::

Request Received at variable rate

Messages Processed at consistent rate

39

Message Centric – Resilency

::

XX

40

Message Centric – Delayed Processing

::

41

Poison Messages (Dead Letters)

Messages that could not be processed by their receiversEventually exceeds the maximum number of delivery attempts to the application

42

1234Worker 1

Worker 2

43

1234Worker 1

Worker 2

1. GetMessage()

2. GetMessage()

44

34

1

2

Worker 1

Worker 2

1. GetMessage()3. Worker 1 crashed

2. GetMessage()4. Processed Successfully

45

1

2

34Worker 1

Worker 2

1. GetMessage()3. Worker 1 crashed6. After 30 seconds message becomes visible

2. GetMessage()4. Processed Successfully5. DeleteMessage()

46

Worker 2

134

Worker 1

1. GetMessage()3. Worker 1 crashed6. After 30 seconds message becomes visible

2. GetMessage()4. Processed Successfully5. DeleteMessage()7. GetMessage():Worker 2 crashed

47

Overcoming Poison Messages

Check the DequeueCount/DeliveryCount/ApproximateReceiveCount Set a Dead Letters Queue

Can be consumed by other consumers for logging or debugging or replaying

1234

1234

48

Priority Queue

The queue may hold message with different priorities (3-High, 1-Low)

Website worker::

workerWebservice

1 3 2 3 1:

49

Priority Queue

3 3 33Application

2 2

1

2

1

50

Priority Queue

3 3 33Application

2 2

1

2

1

51

AWS Spot Instances

AWS “Supply and Demand” marketBid your price -> get your instanceSupports AutoScale and NotificationCost-effective way to add capacity

52

Serverless Architecture

The new “Buzzword”Introduced by AWS Lambda but also available with Azure FunctionsWrite the code -> React to events

API Gateway

Notifications Broker

Cloud Notifications

53

54

Error HandlingPatterns

58

Dealing with errors – always have a fallback

Fallback logicFallback destinationThe “what-if” approach

59

Transient Faults

“Transient fault - a fault that is no longer present if power is disconnected for a short time and then restored.” (http://en.wikipedia.org/wiki/Transient_fault#Transient_fault)Many faults in connectivity to cloud are transient by natureCommonly occur when connecting to service or database

60

Retry Pattern

If at first you don’t succeed, try try again (William Edward Hickson)Retry Logic

Linear – every fixed amount of timeExponential – if the server is heavy-used (throttling) we don’t want to flood it immediate….1 sec….5 seconds….etc.Other

Cloud service

request

error

request

error

1

2

61

Operation With Basic Retry

int currentRetry = 0;while (currentRetry < MaxRetries){    try    {        // Calling external service.        await TransientOperationAsync();    }    catch (Exception ex)    {        currentRetry++;

        // Check if the exception thrown was a transient exception        if (!IsTransient(ex))        {            // If this is not a transient error             // or we should not retry re-throw the exception.             throw;        }    }     await Task.Delay(TimeBetweenRetries);}

62

Retry Pattern – Solutions

Cloud SDKs

Polly (https://github.com/App-vNext/Polly)

var policy = Policy    .Handle<TimeoutException>()    .WaitAndRetry(Enumerable.Repeat(TimeSpan.FromSeconds(10), 3),        (exception, timeSpan) =>        {            _logger.Warn($"{exception} trying again in {timespan}");        });

PolicyResult result = policy.ExecuteAndCapture(() => TransientOperation());

var requestOptions = new BlobRequestOptions(){    RetryPolicy = RetryExponential.Default,};

var s3Config = new AmazonS3Config(){ MaxErrorRetry = 2});

63

Circuit Breaker Pattern

Closed Open

Success

Fail [threshold reached]

Half Open

Retry Timeout

Fail

Success

Fail [under threshold]

Circuit Breaker pattern prevent repeatedly trying to execute an operation that is likely to failThe Circuit Breaker is a proxy that monitors the number of recent failuresPrevents wasting valuable resources because of the wait

64

public class CircuitBreaker{ ... public void ExecuteAction(Action action) { if (IsOpen) { // Check if state should become half-open and retry. ... throw new CircuitBreakerOpenException(LastException); } // The circuit breaker is Closed, execute the action. try { action(); } catch (Exception ex) { // Check if state should become Open. this.TrackException(ex); // Throw the exception so that the caller can tell the thrown exception type throw; } }} https://msdn.microsoft.com/en-us/library/dn589784.aspx

65

Circuit Breaker with Polly

CircuitBreakerPolicy breaker = Policy .Handle<TimeoutException>() .CircuitBreaker(2, TimeSpan.FromMinutes(1), (exception, timespan) => { ... } /* On Break */, () => { ... } /* On Reset */); breaker.Execute(() => { CloudOperation() });

66

Traditional Configuration Deployment

IOS

Android

Windows

67

External Configuration Store Pattern

Azure Blob/AWS S3 etc.

IOS

Android

Windows

ConfigurationStore

Local Store

GET If-Modified-Since

68

Summary

StorageStatic Content DeliveryDirect Upload PatternValet Key

ComputingQueue/Message CentricPoison MessagesServerless architecture

Failure ManagementRetry LogicCircuit BreakerExternal Configuration Store

69

References

P&P Cloud Design Patterns - http://msdn.microsoft.com/en-us/library/dn568099.aspxhttp://cloudpatterns.org/Cloud Architecture Patterns (Bill Wilder)

70

71 71

Presenter contact detailst: @tamir_dreshere: tamirdr@codevalue.netb: TamirDresher.comw: www.codevalue.net

manning.com/dresherDiscount code: ctwndcoslotw

72