Integrating Resource Consumption and Allocation for Infrastructure Resources on-Demand

Resource Consumption and Allocation of MiddlewarePresenter: Longfei Zhang longfeiz@buffalo.eduOutlineIntroductionThe Integrated Control Loop-Based ApproachThe SmartRod FrameworkEvaluation and DiscussionConclusion and Future WorkIntroductionIntroduction-PaaSPlatform as a service (PaaS) is a category of cloud computing services that provide a computing platform and a solution stack as a service. In the classic layered model of cloud computing, the PaaS layer lies between the SaaS and the IaaS layers.

PaaS encapsulates a layer of software that includes middleware, and even development environment(perspective view of cloud computing)

Facilitate the building and deploying of cloud application

Google app engineamazon4Introduction-MiddlewareMiddleware is computer software that connects software components or people and their applications. The software consists of a set of services that allows multiple processes running on one or more machines to interact.

Introduction-Middleware(cont.)It includes web servers, application servers, and similar tools that support application development and delivery. Middleware is especially integral to modern information technology based on XML, SOAP, Web services, and service-oriented architecture

Introduction-IssueThousands of cloud applications share and compete for resources simultaneously

Sufficient and necessary resource provision

Ensure SLA, maximize the performanceImportant, should be on-demandAvoid excessive resources provision, wastePay-as-you-go7Introduction-SolutionIntegrate and coordinate the resource consumption and allocation management of a cloud application

Difficulties: maximize the resource utilization of an application and how to allocate resources to guarantee adequate resource provision for the system8Introduction-Resource Consumption ManagementWith a given amount of resource, adjusting the configurations of an application system to improve its resource utilization[1]

Challenge:Many methods, due to large number of different cloud applications coexistingFind an unified way maximize the resource utilization of an application Challenge: adapt a specific method for individual application is inefficient, cost huge9Introduction-Resource Allocation ManagementProvide or reclaim appropriate amounts of resources to/from a system to satisfy its special requirements or guarantee its performance[1]

Challenge:Should be done at the right time to proper extent, and should consider the features of a cloud application itself

Idlestatic model, cope with the peak workload as the very strart, even peak workload is rareFor instance, memory , if more critical then allocate firstConfiguration-adjusting becomes inefficient when workload is highAvoid excessive resources

10Resource Allocation ManagementAdjust the resources held by a VM Xen,Vmware,VirutalBox

11Introduction-IntegrationThe time to integrate

The execution frequency of each management should be adjusted dynamically according to the workload and available resourcesOutlineIntroductionThe Integrated Control Loop-Based ApproachThe SmartRod FrameworkEvaluation and DiscussionConclusion and Future WorkPaaS, middleware ,Resource consumption management, Resource allocation management, integration13The Integrated Control Loop-Based ApproachIBM autonomic computing[2]

The two loops run consecutively and repeatedly to provide infrastructure resources on-demand by first trying to improve resource utilization, and then allocating more resources when necessary

Monitor, exccute parts is a abstract management function ,map to specific function depending on certain middlewareThere will be policy for analyseThere will be algorithm for plan, talk about laterSo the method could adapt to different middleware and applications, and unify optimization solutions14The Integrated Control Loop-Based ApproachA control loop contains four parts:MonitorAnalyzePlanExecute

Introduction-SolutionLeverage two adaptive control loops:The resource consumption optimization loop[2]The resource allocation loop[2]

The two loops run consecutively and repeatedly to provide infrastructure resources on-demand by first trying to improve resource utilization, and then allocating more resources when necessaryThe Integrated Control Loop-Based Approach-StepsMonitorCollect data from the managed systemAnalyzeProcess and analyze the collected data based on some given policies or guides to determine if a change is necessaryPlanMake a proper plan for changingExecutePut the planned changes into practice

The Resource Consumption Optimization Loop Improve the resource utilization of a cloud application via management functions provided by the corresponding middleware layers of PaaSIF performance is unsatisfactory THEN adjusting middleware configurations

Different applications coexistingMiddlewareMiddleware provide many management functions18The Resource Consumption Optimization Loop-Monitor Tomcat management items:threadBusythreadCountrequestCounterrorCountmaxThreads

Jboss: QueueSizePerformance indicator(throughput, response time)Corresponding function 19The Resource Consumption Optimization Loop-Analyze Set and check the relationship between these monitoring functions or between these functions and preset threshold

Use relationships as system performance metrics:threadCount less than 85% of threadBusyerrorCount less than 30% of requestCountthreadCount less than 95% of manThreadsRelationships becomes false, denotes that current system performance is unsatisfacroryIf situation continues, the system performance will be greatly diminished20The Resource Consumption Optimization Loop-Plan Select the appropriate adjusting functions and determines the execution frequencies of the selected functions

Multi-objective optimization problem

Papers approach:Establish a quantitative functional relationship between an adjusting function and a performance metricUse the contribution to the improvement of a performance metricSeveral performance metricsA collection of adjusting functionsSelect appropriate adjusting functions is a challenge21The Resource Consumption Optimization Loop-Plan Definition: (Performance Improvement Contribution). PIC(Ai, Mj, Rt) is the contribution to the improvement of the performance metric Mj brought by executing the adjusting function Ai under a given amount of resource allocation Rt at time t. The value of PIC is calculated as follows:

Pt(Mj), + = 1, , (0, 1), (0, 0.5]

Pt(Mj) is the value of the performance metric Mj at time tIf Ai is executed at time t+1, then (1) will calculateelse equation 222The Resource Consumption Optimization Loop-Plan Tomcat = = = 0.5 Ai is an adjusting function that increases maxThreads by 10Mj is: threadAlert < 0, where threadAlert = threadBusy 0.85 threadCount P(Mj) is the value of threadAlert

At time t, PICt is 0.64, Pt(Mj) is 4.8. At time t+1, Ai is selected to execute, and Pt+1(Mj) is 2.4. Thus we use equation 1 to calculate the value of PICt+1, which is 0.41

23The Resource Consumption Optimization Loop-Plan Calculate the PIC values after each iteration of the optimization loop. In the next iteration of the loop, it uses these values as the input to the algorithms or methods

Output: sequence of ordered adjusting functions such as: (, , , )

Help select appropriate adjusting functionHelp address a specific multi-objective optimization problem24The Resource Consumption Optimization Loop-Plan Select appropriate optimization functions:The basic idea is to combine all of the objective functions into a single functional form, called AOF(Aggregate Objective Function)[4]

The AOF of the default algorithm is: AOF = weight(Mi) value(Mi). We can see that AOFmax is weight(Mi)

In the default algorithm, when a performance metric M is true, its value is set to 1, else 0

Weight means when M1 false, system performance is worse than M2 false25The Resource Consumption Optimization Loop-Plan TomcatAssume that there exist two performance metrics: M1 and M2. M1 is threadAlert < 0, where threadAlert=threadBusy 0.85 threadCountM2 is errorAlert < 0, where errorAlert=errorCount 0.3 requestCountM1s weight is set to 3 and M2s weight is set to 1

If AOFcurrent is smaller than AOFmax, the algorithm will select the adjusting function with maximum PIC value to be executed in the execute part of the optimization loop.

Plan made26The Resource Consumption Optimization Loop-Execute Perform the adjusting type of management functions provided by a middleware

TomcatIncrease maxThreadDisable dnsLookUpIncrease acceptCountAll for the RCO27The Resource Allocation LoopProvide or reclaim appropriate amounts of resources to/from the corresponding VM that hosts the cloud application and its supporting middleware

Describe the four steps through a algorithm28The Resource Allocation Loop

Greedy, probability

The resource reclaiming algorithm tries to reclaim resources from the system at regular intervals. It first checks if there are resource-provision requests arrived recently. If there has been a long time since the last request arrived, the algorithm will reclaim a specific amount of resources from the system based on the credit, and increase the credit to prepare for resource-provision in the future. 29The Resource Allocation LoopRequires system administrators to specify which kind of resources need to be allocated and the reference values of allocation-amount when executing the resource allocation loop30The Integration

ReclaimBased on performance of the application system, available resources, SLA between

31OutlineIntroductionThe Integrated Control Loop-Based ApproachThe SmartRod FrameworkEvaluation and DiscussionConclusion and Future WorkThe SmartRod FrameworkThe middleware targeted is java application servers Tomcat, JBoss, and JOnAS.

The VMs supported Xen and VirtualBox.

This framework contains two main modules: RCO implements the resource consumption optimization loop of our approachRA implements the resource allocation loop

The SmartRod FrameworkSmartRod has provided a collection of abstract management functions that includes two types: monitoring and adjusting. These abstract functions can be mapped to the specific management functions of a middleware.

System administrators only have to specify the class name, some parameters such as the amount of threads to be increased, and the mapping attribute (Tomcat), the SmartRod framework will automatically execute the corresponding management functions of the mapped middleware .

Thus this module can adapt to different middleware and applications, and unify optimization solutions

The SmartRod FrameworkTo leverage SmartRod, users (system administrators) only have to modify three configuration files: Middleware.xmlVM.xmlRod.xml

The SmartRod FrameworkThe Middleware.xml contains the mapping and connection information of the targeted middleware

The Middleware.xml contains the mapping and connection information of the targeted middleware (see Figure 4). Similarly, the VM.xml contains the mapping and connection information of the targeted VM and VMM (Virtual Machine Monitor or hypervisor).36The SmartRod Framework

(1) Monitors, which specifies the information to be collected from the targeted middleware and VM(2) Targetsperformance, RCO and RA modules of SmartRod (3) MwConf, adjusting functions(4) VmConf, which indentifies the kinds of resources to be allocated to the corresponding VM, and the reference values of allocation-amount when the RA module is running. 37OutlineIntroductionThe Integrated Control Loop-Based ApproachThe SmartRod FrameworkEvaluation and DiscussionConclusion and Future WorkSimulated PaaS EnvironmentCloud application:JEE benchmark application from the Web Performance Inc.[3]Executes the typical transactions of e-business operations

Container: Tomcat

Test driver: LoadRunner

Products browsing and ordering39Simulated PaaS EnvironmentTomcat configuration maxThreads, which limits the maximum number of threads uesd to process requests

LoadRunner has an adjustable parameter called Vuser, which determines the requests generation rateComing requests will be refused if their number exceeds the value of maxThreadIncrase maxThread for improving resource utilization and system performance when there are a large number of waiting requests

Vuser determines the request generation rate40Simulated PaaS EnvironmentTomcat and the benchmark are running on a Xen VM

LoadRuner is running on another Xen VM

OS: Ubuntu 9.10 server editionCPU: Intel Core Due 2.66GHzMemory: 4GBManual and Statical Configuration and AllocationMemory allocatedmaxThreads of TomcatTest 1256M75Test 2256M150Test 3384M75Test 4384M150300 Vusers, increases from 0 to 300 with the rate of starting 20 Vusers every 10 secondsEach Vuser performs a transaction of the benchmark application 10 times, and all the Vuers are scheduled to run 5 minutes.Limit cpu usage42Manual and Statical Configuration and AllocationThroughput Result

Manual and Statical Configuration and AllocationResponse Time Result

Manual and Statical Configuration and AllocationTest 4 almost always covers the Test 1Test 2 and Test 3 are intertwined

Average ThroughputAverage Response TimeTest 11977KB/s30.24sTest 22696KB/s24.16sTest 33021KB/s23.58sTest 43568KB/s20.32sStill have problem45Manual and Statical Configuration and AllocationIssueNot fully exploit the systems capability of optimizing resource utilizationLead to a waste of resources

MemorymaxThreadsThroughputResponse TimeTest 5320M3003426KB/s21.24sTest 4384M1503568KB/s20.32sDynamically and adaptively adjust and allocateGuarantee the performance according to SLA46Dynamic Configuration and AllocationWithout SmartRod (Test 4) allocate 384M memory to the VM hosting the benchmark; and set maxThreads of Tomcat to be 150.

With SmartRod: The RCO module runs on the VM that hosts the application and Tomcat.The RA module runs on the physical machine hosting the VMThe performance metric is threadAlert The adjusting function to be executed in Tomcat is increasing threadpool by 10 each time. The resources to provide and reclaim on the fly is 32M memory each time. The initial maxThreads is 75 and the initial memory allocated is 256M. Dynamic Configuration and AllocationAverage throughput With SmartRod: 3745KB/sWithout SmartRod: 3568KB/s

48Dynamic Configuration and AllocationAverage response and 90% response time With SmartRod: 19.27s and 29.66s Without SmartRod: 20.32s and 30.35s

In the With SmartRod test, the average response time and the 90% response time are 19.27s and 29.66s respectively, while in Without SmartRod test, these values are 20.32s and 30.35s respectively. These results further prove that With SmartRod performs better than Without SmartRod. 49Dynamic Configuration and AllocationMemory consumptionWith SmartRod : min: 256M; max: 384M; avg: 340.18M and ends in 352MWithout SmartRod: 384M

Bottle neck

50Dynamic Configuration and Allocation-ResultSmartRod indeed tries to exploit the systems capability of optimizing resource utilization for managing infrastructure resources on-demand.

Dynamic Configuration and Allocation-ResultDefect of SmartRod:The threadpool cannot be decreased correspondingly when the workload decreases.

There is no need to maintain a large threadpool when workload is relatively low.

52OutlineIntroductionThe Integrated Control Loop-Based ApproachThe SmartRod FrameworkEvaluation and DiscussionConclusion and Future WorkReferencesConclusion and Future workConclusionPaaSIBM autonomic computingResource consumption and allocation managementSmartRod

Future workThe next release of SmartRod will address this problem to enhance on-demand infrastructure resource management to be much more timely and adaptively.

There is no need to maintain a large threadpool when workload is relatively low.

54OutlineIntroductionThe Integrated Control Loop-Based ApproachThe SmartRod FrameworkEvaluation and DiscussionConclusion and Future WorkReferencesReferences[1] Hector A, Gordon S, Geoff C. Adaptive resource management in middleware: a survey. IEEE Distributed Systems Online, 2004 [2] IBM, An architectural blueprint for autonomic computing, 2006 [3]www.webperformanceinc.com/library/reports/ServletReport/ServletBenchmark.war [4] MMO, http://en.wikipedia.org/wiki/Multiobjective_optimization [5] Liang-Jie Zhang and Qun Zhou, CCOA: Cloud Computing Open Architecture, the IEEE International Conference on Web Services (ICWS), 2009, pp.607-616. [6] A Chandra, et al., Quantifying the Benefits of Resource Multiplexing in On-Demand Data Centers, ACM Workshop on Algorithms and Architectures for Self-Managing Systems, 2003 [7] Iganacio M. Liorente, Cloud Computing for on-Demand Resource Provisioning, 7th NRENs and Grids Workshop, 2008 Thank You!

Q&A