![Page 1: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/1.jpg)
1
Adjusting Carbon Topology to Match High Availability Scenario
Requirements
Afkham Azeez
Director of Architecture
WSO2 Inc
![Page 2: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/2.jpg)
2
About Me
• PMC member Apache Axis, Committer Synapse & Web Services
• Member, Apache Software Foundation• Co-author, Axis2 Web Services• Director of Architecture, WSO2 Inc• Blog: http://blog.afkham.org• Twitter: afkham_azeez
![Page 3: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/3.jpg)
3
Agenda
• A brief look at the WSO2 platform
• Carbon clustering for availability
• Cost of availability & related topologies
![Page 4: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/4.jpg)
4
WSO2 Offerings• WSO2 Carbon
• Full platform of servers for deployment on-premise, in private or public cloud
• Products share a consistent architecture and core platform services (e.g. logging, management, security, identity, caching) through OSGi and the “Carbon Core”
• Includes ESB, AppServer, Data Services, Governance, Identity, Business Process, and more
• WSO2 Stratos• Platform-as-a-Service (PaaS) Foundation
• Supports running servers as elastic, metered, billed, multi-tenant with self-service
• Including all Carbon Servers, PHP, Jetty, and a growing list through a standard Cartridge model
• WSO2 StratosLive• http://stratoslive.wso2.com
• WSO2’s Public PaaS
• An instance of Stratos running in the cloud with all Carbon Servers available
![Page 5: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/5.jpg)
5
Consistent Architecture
• Carbon: A consistent set of class-leading enterprise servers
• The same products run either on-premise or in the cloud, single-tenant or multi-tenant
• Utilize the same Carbon core runtime for a seamless experience
• Stratos: A cloud platform for enterprise, hybrid and public deployment
• Extends the deployment to support full self-service, elastic scaling, metering and billing
• Supports Carbon and native server runtimes
• Including Java and non-Java servers such as Jetty and PHP
• Re-uses the same core Carbon architecture to offer core PaaS services including:
• Identity, Logging, File, Relational Storage, Column Storage, Code Deployment, etc
• Both projects share a common set of OSGi modules and a core runtime architecture
![Page 6: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/6.jpg)
6
WSO2 SOA Platform
![Page 7: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/7.jpg)
7
WSO2 Carbon
![Page 8: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/8.jpg)
8
Availability
The degree to which a system, subsystem, or equipment is in a specified operable and committable state at the start of a mission, when the mission is called for at an unknown, i.e., a random, time.
Simply put, availability is the proportion of time a system is in a functioning condition.
![Page 9: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/9.jpg)
9
Availability
![Page 10: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/10.jpg)
10
Availability
![Page 11: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/11.jpg)
11
High Availability (HA)
A system that is designed for continuous operation in the event of a failure of one or more components. However, the system may display some degradation of service, but will continue to perform correctly.
The proportion of time during which the service is accessible with reasonable response times should be close to 100%.
All single points of failure should be eliminated
![Page 12: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/12.jpg)
12
HA, CO & CA
• Continuous Operation (CO)
• Ability to avoid planned outages.
• hardware and software maintenance carried out while applications remains available users.
• Continuous Availability (CA)
• Combines the characteristics of HA and CO to keep the applications running without any noticeable downtime
• Hot update/ graceful round-robin restart
![Page 13: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/13.jpg)
13
High Availability Techniques
• Redundancy
• Time – retransmit
• Data – e.g. parity bits
• Processing – e.g. redundant nodes
• Diversity
• e.g. Hybrid deployments, do the same thing using different implementations
![Page 14: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/14.jpg)
14
How to decide required availability?
• Average throughput (TPS)
• Max throughput (TPS)
• Monetary value of a transaction
• Average loss & max loss per second of downtime
• Decide on how much to invest on availability based on cost vs. benefit tradeoff
![Page 15: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/15.jpg)
15
Patching Production DeploymentsPatch Distribution Coordinator
3. Push patch 3. Push patch
3. Push patch
3. Push patch
1. Check patch list2.Pull new patch
![Page 16: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/16.jpg)
16
Patching Production DeploymentsPatch Distribution Coordinator
4. Maintenance mode 5. Graceful restart
Round-robin
![Page 17: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/17.jpg)
17
Clustering
• Clustering for scalability
• Clustering for availability
![Page 18: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/18.jpg)
18
Clustering for scalability
18
![Page 19: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/19.jpg)
19
Clustering for availability
19
Group Communication Channel/State replication
![Page 20: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/20.jpg)
20
Carbon Clustering
• Membership types
• Static
• Dynamic
• Hybrid
• Membership modes
• Multicast
• Well-known address
![Page 21: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/21.jpg)
21
Static membership
• Predefined members
• Other (non-predefined) nodes cannot join
M2M1
M3 M4
Static group
N
![Page 22: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/22.jpg)
22
Dynamic membership
• No predefined members
• Nodes can join & leave
M2M1
M3 M4
Dynamic group
NJoin
![Page 23: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/23.jpg)
23
Hybrid membership• Some predefined (well-known) members, and some
dynamic members
• Nodes can join & leave
• Membership revolves around the static members
Hybrid group
N
M2M1
M3 M4
Static members
M6M5
M7
Dynamic members
Join (IP, Port)
![Page 24: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/24.jpg)
24
Multicast based membership management
N
M2
M1
M3
M4
Join (IP, Port)
![Page 25: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/25.jpg)
25
Well-known Address (WKA) based membership management
Hybrid group
NWK2
WK1
WK3 WK4
Static members
M6
M5
M7
Dynamic members
Join (IP, Port)
Notify
![Page 26: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/26.jpg)
26
Multicast vs. WKA
Multicast WKA
All nodes should be in the same subnet Nodes can be in different networks
All nodes should be in the same multicast domain No multicasting requirementMulticasting should not be blocked
No fixed IP addresses or hosts required At least one well-known IP address or host required
Failure of any member does not affect membership discovery
New members can join with some WKA nodes down, but not if all WKA nodes are down
Does not work on IaaSs such as Amazon EC2
IaaS-friendly
Requires keepalived, elastic IPs or some other mechanism for remapping IP addresses of WK members in cases of failure
![Page 27: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/27.jpg)
27
Multicast vs. WKA – how to decide?
• Multicast
• Cluster is going to be setup in a network where multicasting is allowed
• WKA
• Cloud based deployment
• Members are distributed across datacenters & regions
• Multicasting blocked
![Page 28: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/28.jpg)
28
HTTP Session Replication
• catalina-server.xml• <Cluster className="org.wso2.carbon.core.session.CarbonTomcatSimpleTcpCluster"/>
• <Valve className="org.wso2.carbon.tomcat.ext.valves.CarbonTomcatSessionReplicationValve"/>
• web.xml• <distributable/>
![Page 29: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/29.jpg)
29
State Replication
JSR-107/JCacheA standard Java Caching API for use by developers and a standard SPI ("Service Provider
Interface") for use by implementers.
import javax.cache.*
…
CacheManager cacheMgr = Caching.getCacheManager();
Cache<String, Integer> cache =cacheMgr .getCache(cacheName);
cache.put(“key”, sampleValue);
Integer i = cache.get(“key”);
![Page 30: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/30.jpg)
30
State Replication
CarbonContext based API
Cache cache = CarbonContext.getCurrentContext().getCache();
cache.put(“key”, sampleValue);
Integer i = cache.get(“key”);
Axis2 Contexts
Using Axis2 clustering StateManager – axis2.xml
<stateManager class="org.apache.axis2.clustering.state.DefaultStateManager” enable=”true">
![Page 31: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/31.jpg)
31
Elastic Load Balancer 2.0
• New sysadmin-friendly configuration language
• High performance PassThrough transport
• Tenant-aware load balancing
• Ability to dedicate clusters for tenants (private jet mode)
• Improved auto-scaler
• Separate IaaS-aware Cloud controller takes care of spawning new instances on different IaaSs
![Page 32: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/32.jpg)
32
Tenant-aware LB
![Page 33: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/33.jpg)
Private Jet mode
• Analogy• Economy class
• no SLA management, only elasticity
• Business class • elasticity plus SLA guarantees
• Private Jet • Guaranteed isolated VMs or machines for a specific
tenant• Still elastically scaled
![Page 34: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/34.jpg)
34
Private Jet Mode
![Page 35: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/35.jpg)
35
Topologies
• Single node
• Multi-node with LB
• Multi-node with elasticity using ELB
• Management & worker node separated
• Multi-zone or multi-datacenter deployment
• Multi-region
![Page 36: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/36.jpg)
36
Single nodeA
vailab
ilit
y
Cost
LOWEST
HIGHEST
![Page 37: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/37.jpg)
37
Primary-secondary
Primary Secondary
Availab
ilit
y
Cost
LOWEST
HIGHEST
![Page 38: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/38.jpg)
38
Primary-secondary, multiple LB
Primary Secondary
keepalived
Availab
ilit
y
Cost
LOWEST
HIGHEST
![Page 39: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/39.jpg)
39
Active cluster, multiple LB
Active Active
keepalived
Active
Availab
ilit
y
Cost
LOWEST
HIGHEST
![Page 40: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/40.jpg)
40
Management & Worker Node Separation
• Proper separation of concerns - management nodes specialize in management of the setup while worker nodes specialize in serving requests to deployment artifacts
• Only management nodes are authorized to add new artifacts into the system or make configuration changes
• Worker nodes can only deploy artifacts & read configuration
• Lower memory foot in the worker nodes because the management console related OSGi bundles are not loaded
• Improved security - management nodes can be behind the internal firewall & be exposed to clients running within the organization only, while worker nodes can be exposed to external clients.
• Isolation of failures
![Page 41: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/41.jpg)
41
Management & Worker Node SeparationA
vailab
ilit
y
Cost
LOWEST
HIGHEST
![Page 42: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/42.jpg)
42
Regions & Zones
![Page 43: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/43.jpg)
43
Stratos 2.0 Architecture
![Page 44: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/44.jpg)
44
Multi-zone or multi-datacenter Deployment
Region X
Zone 1
Zone 2
Cloud Controller
Availab
ilit
y
Cost
LOWEST
HIGHEST
![Page 45: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/45.jpg)
Multi-region deployment
45
Region X
Zone 1
Zone 2
Region Y
Zone 1
Zone 2
Availab
ilit
y
Cost
LOWEST
HIGHEST
![Page 46: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/46.jpg)
46
Multi-IaaS Deployment
Cloud Controller
![Page 47: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/47.jpg)
Multiple IaaS (hybrid) DeploymentA
vailab
ilit
y
Cost
LOWEST
HIGHEST
47
Private cloud (data center)
Zone 1
Zone 2
Amazon EC2
Zone 1
Zone 2
Rackspace Cloud
Zone 1
Zone 2
![Page 48: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/48.jpg)
48
Cost of Availability
Sin
gle
Nod
e
Prim
ary-
Se
con
dary
, si
ngle
LB
Mul
ti-no
de a
ctiv
e cl
uste
r-
Sin
gle
zon
e
Mul
ti-re
gion
Prim
ary-
Se
con
dary
, w
ith m
ultip
le L
Bs
Mul
ti-zo
ne Mul
ti-Ia
aS
![Page 49: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/49.jpg)
49
HA for the Load Balancer
• Load balancer cluster
• Keepalived
• Elastic IP address
• Round Robin DNS
![Page 50: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/50.jpg)
50
Monitoring Servers
• Monit
• Automatically provide alerts & restart processes when monitored items (e.g. latency) fall below certain thresholds.
• New Relic
• Nagios
![Page 51: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/51.jpg)
51
References
Information on tenant-aware load balancing http://sanjeewamalalgoda.blogspot.com/2012/03/tenant-aware-load-balancer-is-upcoming.html
http://sanjeewamalalgoda.blogspot.com/2012/05/tenant-aware-load-balancer.html
Scaling Stratoshttp://srinathsview.blogspot.com/2012/06/scaling-wso2-stratos.html
http://blog.afkham.org/2011/09/how-to-setup-wso2-elastic-load-balancer.html
http://blog.afkham.org/2011/09/wso2-load-balancer-how-it-works.html
![Page 52: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/52.jpg)
52
Auto-scaler service deployment http://nirmalfdo.blogspot.com/2012/07/autoscaler-service-deployment.html
Auto-scaler service http://nirmalfdo.blogspot.com/2012/07/wso2-autoscaler-service-part-i.html
Automatic failover for WSO2 ELBhttp://gonesimple.org/2012/09/24/automatic-fail-over-for-wso2-elb/
![Page 53: Adjusting carbon topology to match high availability scenario requirements](https://reader035.vdocuments.us/reader035/viewer/2022062616/54922a1cac795925288b4702/html5/thumbnails/53.jpg)
Questions?
53
http://www.flickr.com/photos/oberazzi/