Distributed Monitoring and Adaptation of Multiple QoS in Service-Based Systems

Stephen S. Yau and Dazhi Huang Information Assurance Center and

School of Computing, Informatics and Decision Systems Engineering Arizona State University, Tempe, AZ 85287-8809, USA

E-mail: {yau, dazhi.huang}@asu.edu

Abstract— Systems based on service-oriented architecture are called service-based systems (SBS), and comprise of computing services offered by various organizations. SBS often require their services to be composed in complex workflows to perform their high-level tasks. The users usually have certain expectations on the overall QoS (quality of service) of their workflows. Due to the highly dynamic environments of SBS, in which temporary unavailability or quality degradation of services may occur frequently and unexpectedly, monitoring and controlling the execution of workflows adaptively in SBS to satisfy users’ QoS requirements is needed, and should be done in distributed and proactive manner. In this paper, a design of distributed QoS monitoring and adaptive (M/A) modules for multiple workflows in adaptive SBS (ASBS) is presented. The requirements on QoS M/A in ASBS, as well as the ASBS system architecture and the coordination protocol among software modules and services in ASBS for QoS M/A, are discussed.

Keywords: Adaptive service-based systems, multiple QoS monitoring and adaptation, system architecture, multiple workflows

I. INTRODUCTION Recent developments of service-oriented computing and

grid computing have led to rapid adoption of service-oriented architecture (SOA) in distributed computing systems, such as enterprise computing infrastructures, grid-enabled applications, and global information systems. Software systems based on SOA are called service-based systems (SBS). One of the most important advantages of SOA is the capability that enables rapid composition of needed computing services provided by various service providers through networks for distributed applications. These services provide well-defined interfaces for users to access certain capabilities, and are often hosted on geographically-dispersed computer systems. Workflows being composed of various services in SBS are often used to perform complicated tasks that cannot be done by an individual service, and often have certain QoS expectations. Service composition based on functionality has been studied extensively [1-3]. Some work on service composition based on QoS has been done [4-7]. Because the services’ QoS, such as timeliness, throughput, accuracy, security, dependability, and survivability, often vary depending on software designs, system capacities, active workloads, and network/system availabilities, a SBS which can facilitate runtime distributed execution monitoring and adaptation

(M/A) of QoS for multiple workflows to ensure the completion of workflow executions with satisfactory QoS, is needed. Furthermore, because satisfying the requirements of a QoS aspect may require sacrifice in other QoS aspects, adaptive control of the tradeoffs of require-ments among multiple QoS in the SBS is also needed.

A conceptual view of ASBS from software cybernetics perspective was presented in [8]. and the functional services used to compose the ASBS and the QoS M/A modules form a closed control loop for the dynamic control of selection and configuration of services. The QoS monitoring modules collect the measurements of various QoS aspects and system status concerning QoS, which are used in making QoS adaptation for adjusting the configurations and service operations of ASBS to satisfy various QoS requirements simultaneously. In this paper, we will summarize the requirements on QoS M/A in ASBS, and present our design of the ASBS system architecture and coordination protocol among distributed software modules and services in ASBS to support QoS M/A for multiple workflows in ASBS.

II. CURRENT STATE OF THE ART Substantial research has been done on workflow planning

and web service composition to construct and execute a workflow to satisfy users’ functional requirements [1-3]. Recently, designs of adaptive systems to satisfy various QoS requirements have been presented [4-7, 9, 10]. QoS-aware service composition [4-7] aims at finding optimal or suboptimal service composition to satisfy various QoS constraints within a reasonable amount of time. Various techniques have been presented for QoS-aware service composition, such as service routing [6] and genetic algorithms [7]. However, the QoS models considered in existing QoS-aware service composition methods are usually very simple, and runtime adaptation of service composition cannot be efficiently handled by most QoS-aware service composition methods. Self-tuning techniques [9] and autonomic middleware [10] have been developed for providing support for service adaptation and configuration management. However, these techniques and middleware are not designed for SBS, which is more loosely-coupled and more difficult to control.

Execution monitoring of workflows has been considered as an integral part of workflow management systems. Recently, Grid workflow systems operating in similar environments of SBS have been presented [11-14]. However, the workflow monitoring capabilities in these and other

2011 35th IEEE Annual Computer Software and Applications Conference Workshops

978-0-7695-4459-5/11 $26.00 © 2011 IEEE

DOI 10.1109/COMPSACW.2011.16

31

2011 35th IEEE Annual Computer Software and Applications Conference Workshops

978-0-7695-4459-5/11 $26.00 © 2011 IEEE

DOI 10.1109/COMPSACW.2011.16

31

workflow systems share a similar characteristic: They only focus on the status of the current workflow execution, and do not proactively acquire and analyze status information related to future workflow execution.

III. BACKGROUND In our approach in [8], we have presented an approach to

constructing Activity-State-QoS (ASQ) models for the cause-effect dynamics in ASBS. Many QoS M/A activities in ASBS are based on ASQ models. An ASQ model is a 6-tuple <A, S0, S, Q, RS, RQ>, where A is a set of possible user activities that can be performed on a particular service, S0 is a set of initial system resource states, S is the set of all possible system resource states, Q is the set of possible values for a particular aspect of QoS, RS is a relation defined on A × S0 → S representing how user activities and initial system resource states affect future system resource states, and RQ is a relation defined on A × S0 → Q representing how user activities and initial resource states affect QoS aspects.

The ASQ models of each service in ASBS are constructed through the analysis of the data related to user activities, system resource states, and the QoS aspects, which is collected from controlled experiments in SBS. The final result from the data analysis is a set of equations generated using Multivariate Adaptive Regression Splines (MARS) analysis for each service in SBS. Through MARS analysis, significant data variables related to system resource states and QoS for each service will be identified among all the data variables collected during controlled experiments. The set of equations are at two levels: Activity-State (A-S) and State-QoS (S-Q) levels, and used in our QoS M/A modules to estimate the resource consumption and resulting QoS respectively, given service and system configurations.

IV. REQUIREMENTS ON QOS M/A IN ASBS In ASBS, QoS M/A activities, including gathering

system status, measuring QoS, tracking workflow execution status, and dynamically configuring and invoking services, are conducted at server, service and workflow levels. The requirements on QoS M/A in ASBS are summarized below:

Requirements on QoS monitoring • M1: At the server level, collecting system resource

status, such as CPU, memory and bandwidth consumption, is needed for evaluating resource availability and estimating QoS based on our ASQ models [8]. The collected information will also be used to update ASQ models when necessary.

• M2: At service level, the measurements of service availability, and usage statistics, such as request arrival rate and service rate, are needed to provide useful information for making adaptation decisions. The measurements will also be used to update ASQ models.

• M3: At workflow level, the aggregation of workflow-level QoS based on measurements at the service level is needed for monitoring whether the executing workflows have satisfactory QoS.

• M4: At workflow level, detecting situations requiring adaptations, such as service unavailability and unsatisfied user requests, are needed to determine when adaptations should be made.

• M5: Workflow execution, i.e. the completed/remaining portion of an executing workflow, needs to be tracked to provide information for making adaptation decisions and controlling the execution when necessary.

Requirements on QoS adaptation • C1: At server level, the capability to dynamically

control resource allocation for the hosted services and for user requests with different priorities is needed for providing more fine-tuned QoS.

• C2: At service level, an execution control mechanism is needed to dynamically configure a service during its invocation to meet the QoS requirements of a user in system environments with fluctuating resource status.

• C3: A decision-making mechanism is needed to determine the proper service configuration given the user requirements and system resource status.

• C4: At workflow level, execution control mechanisms are needed to dynamically replace services used in a workflow in runtime.

• C5: At workflow level, execution control mechanisms are needed to allow changing the structure of a workflow in runtime.

• C6: The capability of suspending/resuming the execution of a workflow is needed. For example, the execution of a workflow for a low-priority user request may be suspended to release resources for a high-priority request, and resumed when more resources are available.

• C7: It is necessary to coordinate distributed decision-making at the service level to identify service instances configured in C3 for a workflow to meet the overall QoS expectations of the workflow.

V. AN OVERVIEW OF ASBS SYSTEM ARCHITECTURE Fig. 1 depicts the system architecture of our ASBS,

which contains all the necessary modules to form a closed control loop for QoS M/A. The rectangles with solid lines are basic software modules for QoS M/A. The rectangles with dashed lines are plug-ins for QoS optimization. The rounded rectangles are the functional services from various providers. Each workflow in an ASBS has a workflow agent (WA) and a workflow-level QoS coordinator (WQC) for M/A activities at the workflow level. Each server in an ASBS has a server-level QoS coordinator (SQC) and a server monitoring service (SMS) for M/A activities at the service and server levels. Basic software modules for QoS M/A are described below: • Workflow agent (WA): A WA provides the mechanisms for

controlling workflow execution (C2, C4-C6) and tracking workflow execution status (M5). A WA encodes the control structure of a workflow and several specified patterns of possible structural changes for QoS adaptation.

3232

• Workflow-level QoS Coordinator (WQC): A WQC aggregates workflow-level QoS (M3), detects situations requiring adaptation (M4), and provides basic decision support for QoS adaptation (C7), including identifying alternative service instances and selecting one of the specified patterns of possible structural changes. It also coordinates the activities of other M/A modules.

• Server-level QoS Coordinator (SQC): An SQC manages the information on services deployed on the server. It measures QoS of the hosted services (M2), and provides basic decision support for generating appropriate service configurations (C3) following a prioritized FCFS strategy.

• QoS Coordinator Directory (QCD): A QCD stores the information of all workflow-/server-level coordinators in an ASBS to facilitate the dynamic discovery and coordination of SQCs and WQCs. The stored information includes communication endpoints, services managed by each SQC, and types of services to be used in a workflow handled by a WQC.

• Server Monitoring Service (SMS): An SMS monitors system resource status of a server (M1), and provides remote access interface for retrieving such information.

Among all the requirements for QoS M/A, C1 relies on the support from the underlying operating system and service platform. M1-M5, C2, C4, and C6 are fully supported by our QoS M/A modules. C3, C5, and C7 are partially supported by our QoS M/A modules as follows: • For C3, a prioritized FCFS strategy based on our

ASQ models is incorporated in the design of SQCs. This strategy allows the generation of service configurations for a set of service requests to ensure that the minimum QoS requirements of users are satisfied with the available resources. An interface is designed to allow plugging in optimization modules with advanced optimization methods, such as genetic algorithms and mixed integer programming, to handle more sophisticated objectives, such as providing better QoS for users if possible, and optimally selecting a subset of user requests to be served when resources are not enough for all the requests.

• For C5, our current design of WAs and WQCs only supports switching between several specified patterns of possible structural changes when specific kinds of QoS adaptation are needed. For example, a data encryption service can be replaced by several data encryption services in a parallel split and join structure to increase the overall throughput when no server has sufficient resources to allow a single data encryption service to support the required throughput.

• For C7, WQCs coordinate themselves with the help of the QCD following a four-phase protocol: SQC discovery, resource withholding, service selection/ cancellation, and resource reservation. This protocol will be presented in Section VI. Conflicts among WQCs on attempting selection

of the same service instances for workflow execution, which cause the related servers to be overloaded, are resolved following a prioritized FCFS strategy. Similar to C3, more sophisticated objectives, such as optimal service selection to serve as many requests as possible, can be met by using plug-ins for QoS optimization modules.

VI. COORDINATION AMONG QOS M/A MODULES Coordination among distributed software modules in our

ASBS system architecture is crucial for proper decision-making and control of QoS adaptation. In this section, we will present how WAs, WQCs, SQCs, SMSs and the QCD shown in Fig. 1 are coordinated to execute workflows, and monitor and control workflow execution in ASBS.

System initialization Among the QoS M/A modules, the QCD does not require

any specific initialization process. But all the WQCs and SQCs need to be configured to connect to the QCD. The rest of the QoS M/A modules will be initialized as follows: • The SMSs need to be running as system services on the

servers in ASBS. ASQ models of the services deployed on a server will be provided as inputs for the SMS on this server to collect the specific data variables related to the hosted services. Once started, the SMSs will periodically acquire updates of the data variables through system APIs, such as Windows Performance Counters API.

• An SQC on a server needs to load the descriptions of the services hosted on the server. Once the service information

Figure 1. System Architecture of Our ASBS

3333

is loaded, the SQC will register to the QCD by sending the service information and the communication endpoint for the SQC to the QCD, and will notify the SMS on the same server to start sending data updates to the SQC.

• A WQC needs to load the workflow specification, which includes the workflow control structure and the descriptions of services used in the workflow. Once loaded, the WQC will register to the QCD by sending the descriptions of required services and the communication endpoint of the WQC to the QCD. Querying and updating the QCD

The QCD is the central point of sharing and exchanging information among SQCs and WQCs to facilitate the dynamic discovery and coordination among them. The QCD will receive the following queries and updates from the SQCs and WQCs: • From the SQCs: When an administrator adds new services

deployed or removes services no longer offered on the server of an SQC through its GUI, a message will be sent to the QCD to update the stored service information. An SQC will periodically send updates on system resource status, including the amount of remaining resources and the resource reservations made by WQCs for workflow execution, and service usage statistics to the QCD, which are required for making adaptation decisions.

• From the WQCs: The WQCs will query the QCD to find the SQCs providing the required services, and send resource withholding/ releasing requests to the QCD during the process of finding service instances with proper configurations for workflow execution. These queries and requests are designed to avoid the selection of service instances on heavily used servers, and hence reduce conflicts among WQCs causing system overload. Request handling in WAs

WAs are published as services, each of which has a browser-based interface for users to provide necessary input data and specify the minimum QoS requirements for workflow execution, such as the deadline for completing workflow execution, and the strength of the cryptographic algorithm to be used. A WA has information on the types of services to be used in the workflow. The specific service instances to be invoked during the workflow execution are determined at runtime by the corresponding WQC. To enable dynamic control and reconfiguration during workflow execution, checkpoints are added in a WA, where the execution

of the workflow will be paused for the WA to conduct reporting and reconfiguration. The process of handling a user request in a WA is shown in Fig. 2 and summarized in the following steps: WA1 When a user request is received, it will be forwarded

to the corresponding WQC. WA2 If the request can be served, the WQC will send the

WA a message containing the addresses of the service instances and their configurations. If the request cannot be served, the user will be notified so that the user can resubmit a request with different QoS requirements.

WA3 The WA starts executing the workflow following the instruction from the WQC.

WA4 The WA pauses the execution of the workflow at a checkpoint in the workflow. At each checkpoint, the WA measures QoS for the workflow, and reports the workflow progress and the measured QoS to the WQC.

WA5 If the workflow is done, the WA notifies the WQC and ends the process. Otherwise, continue with (WA6)

WA6 The WA checks its internal message queue to see whether any message is received from the WQC between the previous and current checkpoints. There are two possible types of messages here:

a) An UPDATE message containing new addresses and configurations of the remaining services in the workflow Upon receiving such a message, the WA will rebind to the new services and reconfigure the services.

Execute the workflow until the next check

point

Forward the request to the corresponding

OK to start execution?

Wait for the instruction from the

Intercept and parse a user

request

Notify the user that the request cannot

be served

Report the execution status

and certain QoS to the

Workflow complete?

Received an

message?

Report the completion of

workflow execution to the

Perform reconfiguration

Received amessage?

Wait for the instruction from the

Received amessage?

Received a

message?

Figure 2. Request handling in a WA

3434

b) A SUSPEND message indicating the workflow execution should not be continued until further notice. Upon receiving such a message, the WA will not proceed to the next checkpoint until receiving a RESUME message from the WQC.

WA7 The WA continues the execution until another checkpoint, and repeats (WA4-WA7)

The default locations for adding checkpoints are after the invocation of each service in the workflow since the invocation of a service is usually atomic. However, our design of the WAs allows the WAs to receive service update messages from the WQCs while a service is being executed. Hence, service developers may add more checkpoints during the execution of services with long service sessions, such as online streaming services and data processing services handling large size data.

Request handling in WQCs The WQCs process user requests periodically rather than

processing one request at a time to avoid initiating the decision making process for QoS adaptation too frequently. The process of handling user requests in a WQC is shown in Fig. 3 and summarized in the following steps: WQC1 The WQC periodically checks if any user requests

are received from the corresponding WA. WQC2 If some requests are received, the WQC queries the

QCD for an updated list of SQCs providing the required services in the workflow.

WQC3 The WQC receives the list of SQCs providing the required services along with the system resource status of these SQCs.

WQC4 The WQC estimates the resource consumption for invoking the required ser-vices on each SQC based on the minimum QoS requirements, the ASQ models of the services and the system resource status. Based on the estimations, the WQC se-lects N candidate SQCs for each required service such that the selected SQCs will have the lowest workload/ capacity ratios.

WQC5 The WQC sends a resource withholding re-quest to the QCD for the selected SQCs. The request contains the amount of estimated resource consumption for serving the user requests.

WQC6 The WQC receives the reply to the resource withholding request from the QCD. The reply may

contain an updated set of candidate SQCs due to the conflict resolution performed in the QCD when multiple WQCs are attempting to withhold resources on the same server causing possible system overload.

WQC7 The WQC forwards the user requests to the candidate SQCs to get service configurations for the required services.

WQC8 The WQC aggregates the replies from the candidate SQCs, and assigns the user requests to each SQC. The assignment of user requests is performed following a prioritized FCFS order to SQCs reporting the lowest workload/capacity ratios.

WQC9 If all the user requests are satisfied, the WQC notifies the WA of the selected service instances along with their configurations for each user request, sends a resource releasing message for the not-selected services to the QCD, and starts tracking the workflow execution.

WQC10 If some requests cannot be satisfied, the WQC checks whether any of the unsatisfied requests have higher priorities than requests currently being served:

If yes, the WQC combines the unsatisfied requests with low priority requests being served, notifies the WA to suspend the execution for those being served, and goes back to (WQC2).

If no, the WQC continues with (WQC11) WQC11 The WQC checks whether any suspended

execution for previous requests can be resumed based

Figure 3. Request handling in a WQC

3535

on the updated resource availability. If yes, the WQC notifies the WA to resume the execution for those requests.

If no, the WQC notifies the WA of the unsatisfied requests along with the results for the satisfied requests, and sends a resource releasing message for the not-selected services to the QCD.

In WQC6, the QCD may needs to resolve conflicting resource withholding requests from multiple WQCs before sending replies to the WQCs. The QCD will honor the resource withholding requests from WQCs based on a prioritized FCFS order. For an WQC that cannot withhold sufficient resources for a required service on any candidate SQCs in the request from the WQC, the QCD will select another SQC not in the WQC’s request based on the information stored in the QCD.

In WQC9 and WQC11, a resource releasing request will be sent to the QCD to remove the resource withholding on the SQCs with the not-selected services. Meanwhile, the QCD will also change the resource withholding on the SQCs with the selected services into resource reservation, which logically allocates the resources for workflow execution.

Request handling in SQCs In our design, an SQC may receive user requests from

multiple WQCs due to the sharing of the same services among multiple workflows. Similar to the WQCs, the SQCs will also process user requests periodically rather than processing one request at a time. In addition, an SQC provides a GUI interface for the administrator to manage the hosted services. When there is any change on the hosted services, the updates will be sent to the QCD. The updates will also be sent to the SMS to add or remove system counters to be monitored by the SMS. The process of handling user requests in an SQC is summarized in the following steps: SQC1 The SQC periodically checks if any user requests are

received from WQCs. SQC2 For each user request received, the corresponding

service configuration will be generated based on the ASQ models of the requested service and the QoS requirements in the request.

SQC3 The SQC starts to determine which user requests can be served in a prioritized FCFS order until all the requests are satisfied or resources are exhausted.

SQC4 The SQC sends the results to the WQCs, including the requests that can be satisfied along with the service configurations and the requests that cannot be satisfied. Then, go back to (SQC1).

VII. CONCLUSIONS AND FUTURE WORK In this paper, we have presented our design of distributed

QoS M/A modules for multiple workflows in ASBS, including a summary of the requirements on QoS M/A in ASBS, the ASBS system architecture, and the coordination protocol among software modules and services in ASBS for QoS M/A. Our QoS M/A modules provide proactive, autonomic, and efficient workflow monitoring and control in highly dynamic operation environments. A code generator for WAs and WQCs is under development to facilitate

developers to rapidly construct ASBS. Future work includes incorporation of task migration in our design, development of new methods for finding service configurations to achieve better QoS, and performing simulations and experiments to examine the overhead and benefits of our approach and compare it with other existing approaches.

ACKNOWLEDGMENT This research is supported by National Science

Foundation under grant number CCF-0725340.

REFERENCES [1] S. Ponnekanti and A. Fox, “Sword: A developer toolkit for web

service composition”, Proc. 11th Int’l World Wide Web Conf. (WWW 2002) Web Engineering Track, 2002. Available at: http://www2002.org/CDROM/alternate/786/

[2] J. Rao, P. Kungas and M. Matskin, “Application of linear logic to web service composition”, Proc. 1st Int’l Conf. on Web Services, 2003, pp. 3-9.

[3] E. Sirin, J. A. Hendler and B. Parsia, “Semi-automatic composition of web services using semantic descriptions”, Proc. Web Services: Modeling, Architecture and Infrastructure Workshop with the 5th Int’l Conf. on Enterprise Information Systems, 2003, pp. 17-24.

[4] X. T. Nguyen, R. Kowalczyk, and M. T. Phan, “Modeling and solving QoS composition problem using fuzzy DisCSP”, Proc. 2006 IEEE Int’l Conf. on Web Services, 2006, pp. 55-62.

[5] R. Berbner, et al., “Heuristics for QoS-aware web service composition”, Proc. 2006 IEEE Int’l Conf. on Web Services, 2006, pp. 72-82.

[6] J. Jin, and K. Nahrstedt, “On Exploring Performance Optimization in Web Service Composition,” Proc. ACM/IFIP/USENIX Int’l Middleware Conf., October 2004, pp. 115-134.

[7] G. Canfora, M. Di Penta, R. Esposito, and M.L. Villani, “An Approach for QoS-Aware Service Composition based on Genetic Algorithms,” Proc. 2005 Conf. on Genetic and Evolutionary Computation, 2005, pp. 1069-1075.

[8] S. S. Yau, N. Ye, H. Sarjoughian, D. Huang, A. Roontiva, M. Baydogan, and M. Muqsith, “Toward Development of Adaptive Service-Based Software Systems,” IEEE Trans. on Services Computing, Vol. 2(3), 2009, pp. 247-260.

[9] N. Kandasamy, S. Abdelwahed, and J.P. Hayes, “Self-Optimization in Computer Systems via On-line Control: Application to Power Management,” Proc. 1st Int’l Conf. on Autonomic Computing, May 2004, pp. 54-61.

[10] X. Dong, S. Hariri, L. Xue, et al., AUTONOMIA : an Autonomic Computing Environment, Proc. IEEE Int’l Conf. on Performance, Computing, and Communications, April 2003, pp. 61-68.

[11] T. Tannenbaum, D. Wright, K. Miller and M. Livny, “Condor – a distributed job scheduler”, Beowulf Cluster Computing with Linux, T. Sterling (eds.), MIT press, 2002. available at: http://www.cs.wisc.edu/condor/doc/beowulf-chapter-rev1.pdf

[12] J. Cao, S. A. Jarvis, S. Saini, and G. R. Nudd, “GridFlow: workflow management for Grid computing”, Proc. 3rd Int’l Symp. on Cluster Computing and the Grid (CCGrid 2003), 2003, pp. 198-205.

[13] B. Ludäscher, et al., “Scientific Workflow Management and the Kepler System”, Concurrency and Computation: Practice & Experience, vol. 18(10), 2006, pp. 1039-1065.

[14] J. Cao, et al., “ARMS: an Agent-based Resource Management System for Grid Computing”, Scientific Programming, vol. 10(2), 2002, pp. 135-148.

3636