agile, dynamic provisioning of multitier internet applications
DESCRIPTION
Agile, Dynamic Provisioning of Multitier Internet Applications. Bhuvan Urgaonkar , Prashant Shenoy , Abhishek Chandray , and Pawan Goyal ACM Transactions on Autonomous Adaptive Systems, 3(1), 2008. Agenda. Introduction System Overview Provisioning Algorithm How much When - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Agile, Dynamic Provisioning of Multitier Internet Applications](https://reader036.vdocuments.us/reader036/viewer/2022062410/56816102550346895dd0449a/html5/thumbnails/1.jpg)
1
AGILE, DYNAMIC PROVISIONING OF MULTITIERINTERNET APPLICATIONSBhuvan Urgaonkar, Prashant Shenoy, Abhishek Chandray, and Pawan GoyalACM Transactions on Autonomous Adaptive Systems, 3(1), 2008
![Page 2: Agile, Dynamic Provisioning of Multitier Internet Applications](https://reader036.vdocuments.us/reader036/viewer/2022062410/56816102550346895dd0449a/html5/thumbnails/2.jpg)
2
Agenda Introduction System Overview Provisioning Algorithm
How much When
Server Switching Evaluation Conclusion Comments
![Page 3: Agile, Dynamic Provisioning of Multitier Internet Applications](https://reader036.vdocuments.us/reader036/viewer/2022062410/56816102550346895dd0449a/html5/thumbnails/3.jpg)
3
Introduction (1/4) Internet applications employ a multi-tier
architecture, with each tier providing a certain functionality Such applications tend to see dynamically varying
workloads that contain long-term variations such as time-of-day effects short-term fluctuations due to flash crowds
Predicting the peak workload of an Internet application and capacity provisioning based on these worst case estimates is notoriously difficult
![Page 4: Agile, Dynamic Provisioning of Multitier Internet Applications](https://reader036.vdocuments.us/reader036/viewer/2022062410/56816102550346895dd0449a/html5/thumbnails/4.jpg)
4
Introduction (2/4) Since many single-tier provisioning
mechanisms have already been proposed a straightforward extension is to employ such an
approach at each tier of the application But….
Use single-tier provisioning mechanisms Bottleneck Shifting
Model all tiers as a black box and allocate servers whenever the observed response time exceed a threshold
Hard to determine how much servers and where the server should be allocated
![Page 5: Agile, Dynamic Provisioning of Multitier Internet Applications](https://reader036.vdocuments.us/reader036/viewer/2022062410/56816102550346895dd0449a/html5/thumbnails/5.jpg)
5
Introduction (3/4)
![Page 6: Agile, Dynamic Provisioning of Multitier Internet Applications](https://reader036.vdocuments.us/reader036/viewer/2022062410/56816102550346895dd0449a/html5/thumbnails/6.jpg)
6
Introduction (4/4) Research Contributions
Predictive and Reactive Provisioning Analytical modeling and incorporating tails
of workload distributions Virtual Machine based provisioning Handling session-based workloads
![Page 7: Agile, Dynamic Provisioning of Multitier Internet Applications](https://reader036.vdocuments.us/reader036/viewer/2022062410/56816102550346895dd0449a/html5/thumbnails/7.jpg)
7
System Overview (1/6) --Multi-tier Internet Application
A tier may be clustered or not the front-end tier can be a clustered Apache server that
runs on multiple machines the backend tier employs a database with shared-nothing
architecture, it cannot be replicated on-demand
Each clustered tier is also assumed to employ a load balancing element responsible for distributing requests to servers If a session is stateful, successive requests will need to be
serviced by the same server at each tier the load balancing element will need account for this server
state when redirecting requests
![Page 8: Agile, Dynamic Provisioning of Multitier Internet Applications](https://reader036.vdocuments.us/reader036/viewer/2022062410/56816102550346895dd0449a/html5/thumbnails/8.jpg)
8
System Overview (2/6) --Multi-tier Internet Application
Every application also runs a special component called a sentry polices incoming sessions to an application’s
server pool unlike systems that use per-tier admission control
makes a one-time admission decision when a session arrives
avoids resource wastage resulting from partially serviced requests that may be dropped at later tiers
Once a session has been admitted, none of its requests can be dropped at any intermediate tier
![Page 9: Agile, Dynamic Provisioning of Multitier Internet Applications](https://reader036.vdocuments.us/reader036/viewer/2022062410/56816102550346895dd0449a/html5/thumbnails/9.jpg)
9
System Overview (3/6) --Multi-tier Internet Application
![Page 10: Agile, Dynamic Provisioning of Multitier Internet Applications](https://reader036.vdocuments.us/reader036/viewer/2022062410/56816102550346895dd0449a/html5/thumbnails/10.jpg)
10
System Overview (4/6) --Hosting Platform Architecture
The hosting platform is a data center that consists of a cluster of commodity servers interconnected by gigabit Ethernet
Servers Hosting Application Components each application runs on a subset of the servers
and a server is allocated to at most one application at any given time
The component of an application that runs on a server is referred to as a capsule If the capsule is replicable – the server is called Elf If the capsule is non-replicable – the server is called Ent
![Page 11: Agile, Dynamic Provisioning of Multitier Internet Applications](https://reader036.vdocuments.us/reader036/viewer/2022062410/56816102550346895dd0449a/html5/thumbnails/11.jpg)
11
System Overview (5/6) --Hosting Platform Architecture
Nucleus a software component that performs online
measurements of the capsule workload, performance and resource usage
these statistics are periodically conveyed to the control plane
Control Plane responsible for dynamic provisioning of
servers to individual applications
![Page 12: Agile, Dynamic Provisioning of Multitier Internet Applications](https://reader036.vdocuments.us/reader036/viewer/2022062410/56816102550346895dd0449a/html5/thumbnails/12.jpg)
12
System Overview (6/6) --Hosting Platform Architecture
![Page 13: Agile, Dynamic Provisioning of Multitier Internet Applications](https://reader036.vdocuments.us/reader036/viewer/2022062410/56816102550346895dd0449a/html5/thumbnails/13.jpg)
13
Provisioning Algorithm --How much (1/3)
Model each server as a G/G/1 queuing model
Request arrival rate to tier i λi : the request arrival rate to tier i di : the mean response time for tier i si : the average service time for a request : the variance of inter-arrival time : the variance of service time
![Page 14: Agile, Dynamic Provisioning of Multitier Internet Applications](https://reader036.vdocuments.us/reader036/viewer/2022062410/56816102550346895dd0449a/html5/thumbnails/14.jpg)
14
=>
=>
=>
=>
=>
Wq : the waiting time in queue
X : the (random) service time
![Page 15: Agile, Dynamic Provisioning of Multitier Internet Applications](https://reader036.vdocuments.us/reader036/viewer/2022062410/56816102550346895dd0449a/html5/thumbnails/15.jpg)
15
Provisioning Algorithm --How much (2/3)
Observe that di is known the per-tier service time si the variance of inter-arrival and service
times and can be monitored online in the system. By substituting these values, a lower bound
on request rate λi that can serviced by a single server can be obtained.
![Page 16: Agile, Dynamic Provisioning of Multitier Internet Applications](https://reader036.vdocuments.us/reader036/viewer/2022062410/56816102550346895dd0449a/html5/thumbnails/16.jpg)
16
Provisioning Algorithm --How much (3/3)
ηi : The number of servers needed at tier i (output) Z : average session think-time : the rate that a session issues requests λ : the session arrival rate : the average session duration βi: the requests that triggered by a single
incoming request at tier i
![Page 17: Agile, Dynamic Provisioning of Multitier Internet Applications](https://reader036.vdocuments.us/reader036/viewer/2022062410/56816102550346895dd0449a/html5/thumbnails/17.jpg)
17
Provisioning Algorithm –When – Predictive Provisioning for Long Term(1/3) Predictive provisioning is motivated by long-
term variations such as time-of-day or seasonal effects exhibited by Internet workloads the workload seen by an Internet application
typically peaks around noon every day and is minimum in the middle of the night
The predictor uses past observations of the workload to predict peak demand that will be seen over a period of T hours For simplicity of exposition, assume that T = 1 hour
![Page 18: Agile, Dynamic Provisioning of Multitier Internet Applications](https://reader036.vdocuments.us/reader036/viewer/2022062410/56816102550346895dd0449a/html5/thumbnails/18.jpg)
18
Provisioning Algorithm –When – Predictive Provisioning for Long Term(2/3)
![Page 19: Agile, Dynamic Provisioning of Multitier Internet Applications](https://reader036.vdocuments.us/reader036/viewer/2022062410/56816102550346895dd0449a/html5/thumbnails/19.jpg)
19
Provisioning Algorithm –When – Predictive Provisioning for Long Term(3/3) λpred(t): the predicted arrival rate during a
particular hour denoted by t λobs(t): the actual arrival rate seen during
this hour λobs(t) - λpred(t): the prediction error h : the mean prediction error over the
past h hours
![Page 20: Agile, Dynamic Provisioning of Multitier Internet Applications](https://reader036.vdocuments.us/reader036/viewer/2022062410/56816102550346895dd0449a/html5/thumbnails/20.jpg)
20
Provisioning Algorithm –When – Reactive Provisioning for Short Term(1/3) sudden load spikes or flash crowds are
inherently unpredictable phenomena Reactive provisioning is used to swiftly
react to such unforeseen events operates on short time scales—on the order of
minutes—checking for workload anomalies
![Page 21: Agile, Dynamic Provisioning of Multitier Internet Applications](https://reader036.vdocuments.us/reader036/viewer/2022062410/56816102550346895dd0449a/html5/thumbnails/21.jpg)
21
Provisioning Algorithm –When – Reactive Provisioning for Short Term(2/3) Reactive provisioning is invoked once every few
minutes It can also be invoked on-demand by the application
sentry
Two approaches Recompute a new allocation of server for the various
tiers Increase the allocation of all tiers that are at or near
saturation by a constant amount
![Page 22: Agile, Dynamic Provisioning of Multitier Internet Applications](https://reader036.vdocuments.us/reader036/viewer/2022062410/56816102550346895dd0449a/html5/thumbnails/22.jpg)
22
Provisioning Algorithm –When – Reactive Provisioning for Short Term(3/3) If the free pool is empty or has
insufficient servers need to be borrowed from other
underloaded applications running on the hosting platform
An application is said to be underloaded if its observed workload is significantly lower than its provisioned capacity
![Page 23: Agile, Dynamic Provisioning of Multitier Internet Applications](https://reader036.vdocuments.us/reader036/viewer/2022062410/56816102550346895dd0449a/html5/thumbnails/23.jpg)
23
Server Switching (1/2) assume that each Elf server runs
multiple virtual machines and capsules of different applications within it Only one capsule and its virtual machine is
active at any time Other virtual machines are dormant—they
are allocated minimal server resources If the server belongs to the free pool, all of
its resident VMs are dormant
![Page 24: Agile, Dynamic Provisioning of Multitier Internet Applications](https://reader036.vdocuments.us/reader036/viewer/2022062410/56816102550346895dd0449a/html5/thumbnails/24.jpg)
24
Server Switching (2/2) switching an Elf server from one application to
another implies deactivating a VM by reducing its resource allocation to ε ε is a small value such that the VM consumes
negligible resources But, if the server retains state of existing sessions
Fixed rate ramp down Some long-lived residual session will be forced to
terminate Measurement-based ramp down
The server switching time is long
![Page 25: Agile, Dynamic Provisioning of Multitier Internet Applications](https://reader036.vdocuments.us/reader036/viewer/2022062410/56816102550346895dd0449a/html5/thumbnails/25.jpg)
25
Evaluation –Environment (1/3)
a prototype data center a cluster of 40 Pentium servers
An application capsule (2.8GHz, 512MB RAM) Load balancer Control plane (dual-processor 450MHz, 1GB RAM) Sentry (dual-processor 1GHz, 1GB RAM) Workload Generator
connected via a 1Gbps ethernet switch running Linux 2.4.20
Three tiers Apache Web server (2.0.48) Tomcat servlets container (4.1.29) Non-replicable Mysql database server (4.0.18)
![Page 26: Agile, Dynamic Provisioning of Multitier Internet Applications](https://reader036.vdocuments.us/reader036/viewer/2022062410/56816102550346895dd0449a/html5/thumbnails/26.jpg)
26
Evaluation –Environment (2/3)
Virtual Machine Monitor Xen 1.2 …..
Nucleus online measurements of resource usages and request performance real-time processing of logs provided by the application software
components offline measurements to determine various quantities needed by the
control plane Sentry and Load balancer
Use Kernel TCP Virtual Server (ktcpvs) version 0.0.14 for sentry and Apache layer
mod_jk: an Apache module that implement a varient of round robin request distribution for Tomcat layer
Control Plane A daemon running in a dedicated machine Implements the predictive and reactive provisioning
![Page 27: Agile, Dynamic Provisioning of Multitier Internet Applications](https://reader036.vdocuments.us/reader036/viewer/2022062410/56816102550346895dd0449a/html5/thumbnails/27.jpg)
27
Evaluation –Environment (3/3)
two open-source multi-tier applications Rubis
An eBay like auction site Three type of user sessions : selling, browsing, bidding 9 tables in the database 26 interactions that can be accessed from the clients’ Web
browsers Rubbos
A bulletin-board application Two different levels of access : regular user and moderator provides 24 Web interactions
SLA: the 95th percentile of the response time is no greater than 2 seconds
![Page 28: Agile, Dynamic Provisioning of Multitier Internet Applications](https://reader036.vdocuments.us/reader036/viewer/2022062410/56816102550346895dd0449a/html5/thumbnails/28.jpg)
28
Evaluation -- independent per-tier provisioning(1/3)
Use Rubbos application Workload increase every 10 minutes
![Page 29: Agile, Dynamic Provisioning of Multitier Internet Applications](https://reader036.vdocuments.us/reader036/viewer/2022062410/56816102550346895dd0449a/html5/thumbnails/29.jpg)
29
Evaluation -- independent per-tier provisioning(2/3) employ dynamic provisioning only at the most
compute-intensive tier of the application, since it is the most common bottleneck the Tomcat tier
The capacity of a Tomcat server was determined to be 40 simultaneous sessions, while Apache was configured with a connection limit of 256 sessions
![Page 30: Agile, Dynamic Provisioning of Multitier Internet Applications](https://reader036.vdocuments.us/reader036/viewer/2022062410/56816102550346895dd0449a/html5/thumbnails/30.jpg)
30
Evaluation -- independent per-tier provisioning(3/3)
Use multi-tier provisioning technique
![Page 31: Agile, Dynamic Provisioning of Multitier Internet Applications](https://reader036.vdocuments.us/reader036/viewer/2022062410/56816102550346895dd0449a/html5/thumbnails/31.jpg)
31
Evaluation -- the black box approach(1/2)
Use Rubis assume that two Tomcat servers and one
Apache server are added to the application every time a capacity increase is signaled
But database is not replicable
![Page 32: Agile, Dynamic Provisioning of Multitier Internet Applications](https://reader036.vdocuments.us/reader036/viewer/2022062410/56816102550346895dd0449a/html5/thumbnails/32.jpg)
32
![Page 33: Agile, Dynamic Provisioning of Multitier Internet Applications](https://reader036.vdocuments.us/reader036/viewer/2022062410/56816102550346895dd0449a/html5/thumbnails/33.jpg)
33
Evaluation -- the black box approach(2/2)
Use multi-tier provisioning technique
![Page 34: Agile, Dynamic Provisioning of Multitier Internet Applications](https://reader036.vdocuments.us/reader036/viewer/2022062410/56816102550346895dd0449a/html5/thumbnails/34.jpg)
34
Evaluation -- Predictive and Reactive Provisioning(1/4)
Use Rubis Workload
1998 Soccer World Cup Site 8 day period
Compressing the original 24-hr long trace to 1hr Picking every 24th minutes and discarding the
rest Day 6(typical day) Day 7(moderate overload) Day 8(extreme overload)
![Page 35: Agile, Dynamic Provisioning of Multitier Internet Applications](https://reader036.vdocuments.us/reader036/viewer/2022062410/56816102550346895dd0449a/html5/thumbnails/35.jpg)
35
Evaluation -- Predictive and Reactive Provisioning(2/4)
Day 6 Only predictive provisioning
![Page 36: Agile, Dynamic Provisioning of Multitier Internet Applications](https://reader036.vdocuments.us/reader036/viewer/2022062410/56816102550346895dd0449a/html5/thumbnails/36.jpg)
36
Evaluation -- Predictive and Reactive Provisioning(3/4)
Day 7 Predicted with/without recent trand Prediction failed during interval 2 Reactive must trigger after the SLA is
violated
![Page 37: Agile, Dynamic Provisioning of Multitier Internet Applications](https://reader036.vdocuments.us/reader036/viewer/2022062410/56816102550346895dd0449a/html5/thumbnails/37.jpg)
37
Evaluation -- Predictive and Reactive Provisioning(4/4)
Day 8 Prediction is failed The unpredictable workload consumes
all the server Using policing to drop sessions
![Page 38: Agile, Dynamic Provisioning of Multitier Internet Applications](https://reader036.vdocuments.us/reader036/viewer/2022062410/56816102550346895dd0449a/html5/thumbnails/38.jpg)
38
Evaluation –Switching of server resources
Scenario 1: New server taken from free pool; the application must be start
Scenario 2: as 1, but application is already running
Scenario 3: taken from another application, waiting for all residual sessions to finish
Scenario 4: as 3, let two VMs share the CPU equally until the session finish
Scenario 5: as 3, using “fixed rate ramp down”
![Page 39: Agile, Dynamic Provisioning of Multitier Internet Applications](https://reader036.vdocuments.us/reader036/viewer/2022062410/56816102550346895dd0449a/html5/thumbnails/39.jpg)
39
Conclusion a flexible queuing model to determine
how much resources to allocate to each tier of the application
a combination of predictive and reactive methods that determine when to provision these resources, both at large and small time scales
![Page 40: Agile, Dynamic Provisioning of Multitier Internet Applications](https://reader036.vdocuments.us/reader036/viewer/2022062410/56816102550346895dd0449a/html5/thumbnails/40.jpg)
40
Comments(1/2) A different thinking about resource provisioning
Which service should be allocated resource ? SLA must be violated first
How many resources and when to allocate to services ? The accuracy of prediction is key point
Can the two ways combine together? The evaluation result in the paper seems not so
good The prediction interval and reactive interval is too long
(15 min and few minutes) But frequently checking will make more loading
![Page 41: Agile, Dynamic Provisioning of Multitier Internet Applications](https://reader036.vdocuments.us/reader036/viewer/2022062410/56816102550346895dd0449a/html5/thumbnails/41.jpg)
41
Comments(2/2) Unpredictable workload is really
unpredictable ? Cooperate with news But its not automatic
Queuing theory…………
![Page 42: Agile, Dynamic Provisioning of Multitier Internet Applications](https://reader036.vdocuments.us/reader036/viewer/2022062410/56816102550346895dd0449a/html5/thumbnails/42.jpg)
42
Thanks The End