a probabilistic quality of service model for nondeterministic … · 2020. 6. 20. ·...

A probabilistic quality of service model for nondeterministic service compositions

Doctoral thesis defense

Candidate: Adrian Satja Kurdija

Faculty of Electrical Engineering and Computing

Supervisor: Assistant professor Marin Šilić, PhD

June 2020

Outline

• Service-Oriented Architecture

• Quality of Service (QoS)

• QoS prediction

• Service selection

• Compositional QoS model

• Multi-Criteria Service Selection for Multi-User Composite Applications

• Evaluation

• Conclusion

2/50

Introduction

• Cloud computing applications contain various service invocations

• Example: an application such as Gmail uses various services (mail, chat/hangouts, calendar, tasks, translation)

• Multiple atomic services with different functionalities

• Application can be seen as a service composition

3/50

Service-Oriented Architecture

4/50

Fig. Multi-tenant travel booking service-based system (SBS) [1]


5/50



6/50



• Service Oriented Architecture (SOA): architectural style that assumes a variety of atomic reusable services which provide certain functionality through their publicly accessible interfaces

• Advanced functionality is achieved through atomic services composition into more complex composite services

7/50

Quality of Service (QoS)

• Each functionality → many service instances on servers worldwide → service class

• Functionally equivalent service candidates can have different non-functional properties, referred in literature as Quality of Service (QoS)

• QoS properties include:– availability

– reliability

– response time

– throughput

– reputation

– cost

8/50

Quality of Service (QoS)

• QoS properties depend on:– user-specific parameters

• location, network and device capabilities, usage profiles

– service-specific parameters

• location, computational complexity, system resources

– environment-specific parameters

• service provider load, network performance

• User requirements for QoS values:– maximal response time (e.g. ≤ 1 second)

– minimal reliability (e.g. ≥ 98%)

– minimal throughput (e.g. ≥ 1.5 requests per second)

– etc.

9/50

Service Composition

• Workflow of the composition - execution plan of tasks to perform

• Contains compositional structures:– sequence, branching, parallel execution, loop

• QoS of the composition depends on:– the actual execution path (probabilistic in case of e.g. branching)

– QoS of the selected service instances for each task

10/50

Fig. Example of a service composition (execution plan)

Research problem

• Which service instance to invoke at a particular time for a particular user?

• Two independent research tasks:– QoS prediction

• for atomic service instances

– Service selection

• for composite applications

11/50

QoS prediction

• Happens before the service selection as its prerequisite

• Most QoS(user, service) values are unknown (no data)

• Predictions are based on past invocation data for similar users and/or services (collaborative filtering)

12/50

QoS User 1 User 2 User 3 User 4

Service 1 ? ? 0.98 0.993

Service 2 ? 0.9 ? ?

Service 3 ? ? 0.99 ?

Service 4 0.97 ? 0.95 ?

Service 5 ? 0.96 ? 0.94

Service 6 0.992 ? ? ?

Fig. User-Service QoS data

QoS prediction

13/50

Fig. QoS Prediction (high-level)

QoS prediction model

• From Kurdija, Silic, Delac, Srbljic (2018): Real-time adaptive QoS prediction using approximate matrix multiplication [2]– Faster than standard UPCC/IPCC/Hybrid approaches with comparable

accuracy

14/50

Fig. QoS prediction model overview

Service selection problem

• Which service instance to select for a particular user and a particular task?

15/50

Fig. Multi-user service selection



16/50




17/50


Service selection constraints

• Optimize multiple application instances (users)

• Respect the throughput limit of each service instance– maximum number of requests it can process in a given time frame

• Each user might have constraints on QoS– total reliability / price / response time (…)

• A service instance might have good reliability, but high price, etc. → multicriteria service selection

18/50

Existing solutions

• Greedy approach: select a service with best QoS for each user?

• Problems:– QoS has many properties → possible trade-offs (e.g. price vs.

reliability)

– what matters is QoS of the whole composition, not just of a single task

– composition is non-deterministic

– service has a throughput limit → greedy will overload the most “popular” services

19/50

Existing solutions

• Reduction to Mixed Integer Programming (MIP) [4]– heuristic enhancements in case of large search space:

• heuristic ranking of services

• service clustering

• Reduction to Assignment Problem (AP) [5]– for single service class, 1 request per user and service

– solved by Hungarian (Kuhn-Munkres) algorithm

20/50

QoS User 1 User 2 User 3 User 4

Service 1 X

Service 2

Service 3 X

Service 3 X

Service 3

Service 3 X

Fig. Assignment problem

Processing capacity

(THR = 4)

Limitations of existing solutions

• Efficiency problems:– exponentially large search space (in case of MIP)

– large problem size for high throughputs (in case of AP)

• Simplistic treatment of QoS compositions– ignoring branching and/or probabilities

– aggregated QoS is calculated by simple sum/product or by taking worst-case execution path

• Non-personalized QoS– assumption that QoS values depend only on a service and not on a

user

21/50

Limitations of existing solutions

22/50

multi-user

multi-task (service

composition)

user-dependent

QoS

main approach

worst-case complexity

He et al.(2015)

✓ ✓ × mixed integerprogramming

(MIP)

exponential

Alrifai et al.(2015)

× ✓ × enhanced MIP exponential

He et al. (2015)

✓ ✓ × greedy + MIP exponential

Wang et al. (2015)

✓ ✓ × clustering + MIP

exponential

Wang and Cheng(2015)

✓ × × clustering + MIP

exponential

Wang et al.(2015)

✓ × ✓ assignment problem

polynomial

our work (2019)

✓ ✓ ✓ transportationproblem

polynomial

The proposed model

• From Kurdija et al. (2019): Fast Multi-Criteria Service Selection for Multi-User Composite Applications [6]

• Main aims: generality, time efficiency

• Support service composition with:– a large number of service-based applications

– for a large number of different users

– with the aim of meeting all (or as many as possible) QoS requirements

– respect the physical limitation of maximum throughput for each service in the cloud

– satisfy the assumption of QoS dependence on a particular user

– take into account the probabilistic aspect of a composite application (compositional QoS model)

23/50

Compositional QoS model

• Assume users with potentially different nondeterministic composition plans and QoS requirements

• Consider all common compositional structures (sequence, conditional branching, parallelism, and loops)

• Consider estimated probabilities of branching and the expected number of loop repetitions

• Calculate the expected number of invocations of each service class

• Calculate the expected compositional QoS values for a particular selection of atomic services

24/50

Compositional QoS model

• 𝐸𝐼(𝐶) = vector of expected number of invocations for each service class in composition 𝐶

• 𝑄𝑜𝑆(𝐶) = vector of expected QoS values for a specific service selection

• Calculated recursively – example (branching):– If 𝐶 is a composition which branches into 𝐶1, 𝐶2, … , 𝐶𝑚 with respective

probabilities 𝑝1, 𝑝2, … , 𝑝𝑚, then:

𝐸𝐼 𝐶 =

𝑗=1

𝑚

𝑝𝑗 𝐸𝐼 𝐶𝑗 ,

𝑄𝑜𝑆 𝐶 =

𝑗=1

𝑚

𝑝𝑗 𝑄𝑜𝑆 𝐶𝑗 .

25/50

Reduction to transportation problem

• Transportation problem (abstract):– each supplier has a given number of items to ship

– each demander has a given number of items to receive

– each supplier-demander connection has a cost

– goal: find shipping distribution (how many items to ship for each connection) to minimize total cost

• In service selection:– requests are items

– services are suppliers

– users are demanders

• demand is the expected

number of requests

– QoS-based cost of connections

26/50

Reduction to transportation problem

• Algorithm for solving TP:– Find an initial (heuristic) solution using Vogel Approximation

Method (VAM)

– Iteratively improve the solution until it is optimal, using Transportation Simplex Method (TSM)

• In our context:– multiple service classes → multiple transportation problems

– how to define transportation cost?

– how to enforce global (compositional) QoS requirements?

27/50

Iterative heuristic algorithm

• For each service class (task), reduce the service selection problem to a transportation problem

• Non-locality: define utility cost in a transportation problem by taking into account other tasks in user’s composition

• Transportation problems for different tasks can be solved in parallel

• Update cost (put more "weight") for non-satisfied QoS properties and repeat the procedure

28/50


29/50Fig. Selection example

Transportation problem utility cost

• How to define a global-aware cost of matching service 𝑖 to user 𝑢?

• Basic idea: If 𝑖 → 𝑢 is selected in task 𝑇𝑗, how difficult would it be to complete the selection?

• Namely, do we need low or high quality services in other tasks to satisfy the QoS requirement of user 𝑢?– Rank the services in each task by QoS

– If low-QoS services complete the requirement with 𝑖 → 𝑢, then 𝑖 → 𝑢 is

"easy" (low cost)

– If only high-QoS services complete the requirement with 𝑖 → 𝑢, then 𝑖 → 𝑢 is "difficult" (high cost)

32/50


33/50

Fig. Illustration of the concept of matching difficulty 𝑖 → 𝑢for a fixed user 𝑢, service 𝑖 and QoS property 𝑘


34/50



35/50


High-level algorithm illustration (1/3)

36/50


37/50


38/50

Evaluation

• Generating a testing dataset– mixture of actual and artificial test data

• Testing two variations of the proposed approach:– Service Selection using Vogel Approximation Method (SS-VAM)

– Service Selection using Transportation Simplex Method (SS-TSM)

• Types of experiments:– Single-Task Experiment

– User-Independent-QoS experiment

– General experiment

39/50

Evaluation

• Comparing with existing approaches according to:– QoS requirement satisfaction:

• accuracy = avg. percentage of satisfied QoS reqs.

– obtained QoS values:

• QoS improvement = 𝑎𝑣𝑔.obtained QoS value− required QoS value

required QoS value

– execution time (efficiency)

• Current cloud computing efforts deal with an increasing amount of users and service instances → focus on efficiency (more than accuracy)

40/50

Single-Task Experiment

• Application = single service, no compositions

• Comparison with AP model (reduction to assignment problem) and MIP

• 500 users, 100 services with low (1-50) or high (50-1000) throughput limits

41/50

Single-Task Experiment

42/50

User-Independent-QoS Experiment

• Assume that QoSu,i = QoSv,i for different users u, v

• Testing against enhanced-MIP models which depend on this assumption (Clus2-MIP and Clus3-MIP)

• 100 users, 8 tasks, 300 services, varied QoS requirement difficulty

43/50

User-Independent-QoS Experiment

44/50

General Experiment

• Without special assumptions

• Testing against general MIP and its enhancement (Greedy MIP)

• 100 users, 8 tasks, 300 services, varied QoS requirement difficulty

45/50

General Experiment: QoS

46/50

General Experiment: time and scalability

47/50

Conclusions

• The proposed SS-TSM model is the dominating approach in most experiments because of a significant reduction of execution time

• The proposed SS-VAM approach can be faster in a single-task scenario with high service throughput limits

• The alternative AP approach (based on the reduction to assignment problem) can be faster in a single-task scenario when throughputs are lower

• In the general scenario, the proposed SS-TSM model shows best results

48/50

References

1. Q. He, J. Han, F. Chen, Y. Wang, R. Vasa, Y. Yang, and H. Jin, "Qos-aware service selection for customisable multi-tenant service-based systems: Maturity and approaches," in 2015 IEEE 8th International Conference on Cloud Computing, pp. 237-244, June 2015.

2. A. Kurdija, M. Silic, and S. Srbljic, “Real-time adaptive qos prediction using approximate matrix multiplication,” Int. J. Web Grid Serv., vol. 14, pp. 200-235, Jan. 2018.

3. H. Jin, H. Zou, F. Yang, R. Lin, and X. Zhao, "A hybrid service selection approach for multi-user requests," in 2012 IEEE 9th International Conference on Embedded Software and Systems, Liverpool, 2012, pp. 1142-1149.

4. Y. Wang, Q. He and Y. Yang, "QoS-Aware Service Recommendation for Multi-tenant SaaS on the Cloud," 2015 IEEE International Conference on Services Computing, New York, NY, 2015

5. Wang, S., Hsu, C., Liang, Z. et al. Multi-user web service selection based on multi-QoS prediction. Inf Syst Front 16, 143–152 (2014).

6. A. S. Kurdija, M. Silic, G. Delac and K. Vladimir, "Fast Multi-Criteria Service Selection for Multi-User Composite Applications," in IEEE Transactions on Services Computing. doi: 10.1109/TSC.2019.2925614

49/50The rest of references can be found in the doctoral thesis.

Thank you!

Questions?

50/50

a probabilistic quality of service model for nondeterministic … · 2020. 6. 20. ·...

Documents