decision strategies for automated negotiation with limited … · decision strategies for automated...

Decision Strategies for AutomatedNegotiation with Limited Knowledge

Jan Richter

Submitted in fulfilment of the requirementsfor the degree of Doctor of Philosophy

Faculty of Information & Communication TechnologiesSwinburne University of Technology

December 2011

to my parents

Abstract

This thesis focuses on decision strategies in automated negotiation when only lim-ited knowledge about the negotiation partner or environment is available. Negoti-ation between self-interested agents is a key mechanism in distributed and autonomoussoftware systems which facilitates multi-stage decision-making between two or moreparties that are in conflict about their goals or preferences. In such systems, whenagents are competitive and act rationally, the agents do not disclose information abouttheir decision models and preferences, and behave in various ways to achieve theirgoals. Because of this, the available information for the decision-making of an agentis limited as it can only be derived from the current encounter or previous interactions.Therefore, an agent needs to find a decision strategy that obtains high payoffs while atthe same time reaches an agreement given this limited knowledge. Decision-making insuch situations is known to be hard and, while many approaches have been proposed,most assume that agents either have sufficient or precise knowledge about their oppon-ents in the form of empirical data, domain knowledge or the decision models of theircounterparts, or have enough time to learn it during their encounters.

The thesis proposes novel solutions to the above problem and, in particular, focuses onthe strategic concession behaviour of agents in competitive environments. The funda-mental setting considered is that of bilateral negotiation in which two agents bargainfor a product or service by exchanging offers alternately until one party agrees or with-draws from the encounter. In such a setting, the work first presents and investigates twodecision mechanisms, an existing heuristic-based approach and a novel decision modelbased on multistage fuzzy decision-making, that are suitable for situations in which anagent has only limited knowledge, and then proposes a mechanism for coordinatingthese strategies in more complex and realistic concurrent negotiation scenarios.

The heuristic-based approach linearly combines individual decision functions to cre-ate multi-tactic negotiation strategies that can react to a range of factors such as theopponent’s behaviour, time, or the state of a resource. While the advantage is that onlyobservable information from the current encounter is required, the mixing mechanismitself and its effect on the strategic concession behaviour of the agents has not been in-vestigated before. As the traditional linear combination can not guarantee monotonicconcession curves, even when all involved tactics are monotonic and weights are static,

agreements can be delayed and outcomes can differ significantly. We propose newmixing mechanisms based on linear combinations of individual negotiation threadsor single concessions, which guarantee monotonic concession curves for monotonictactics in static and dynamic strategies.

The second decision mechanism models the negotiation process as a multistage fuzzydecision problem in which fuzzy state transitions represent the limited knowledge ofthe opponent’s behaviour, for example, by using only a few reference cases. This en-ables the use of dynamic programming algorithms in order to find the best course ofactions that achieves a desired outcome. In this model, the preferences of an agent aremodelled using a fuzzy goal and fuzzy constraints that also allow an agent to com-bine a preferred strategy with the fuzzy state transitions in order to create differentstrategic concession behaviours. Due to the fuzzy transition model and the ability toimpose fuzzy constraints on the decision-making process, agents are able to negotiatecompetitively by utilizing their limited knowledge about their opponents.

The coordination of negotiation strategies in concurrent bilateral encounters is demon-strated using an example scenario with one-to-many negotiations in the domain ofservice-oriented computing. In this scenario, a number of service level agreementsneed to be negotiated with service providers in order to establish a workflow-basedcomposite service. It shows that the mechanism increases the number of compoundagreements by the method of utility boundary decomposition and surplus redistribu-tion of successfully finished negotiations, while simultaneously allowing the individualagents to use their own decision strategies for negotiation.

The major advantage of the proposed mechanisms is their ability to create negotiationstrategies that successfully cope with situations in which the available knowledge aboutthe opponents and the environment is limited. The example scenario also demonstratesthe applicability of the mechanisms in a more complex and realistic scenario. Bothdecision models and the coordination mechanism are validated experimentally.

iii

Acknowledgements

First and foremost my deepest gratitude goes to my supervisor Professor RyszardKowalczyk for taking me on as a PhD student and for the guidance on this researchtopic throughout these years. Our regular meetings and numerous discussions helpedme understand the field and discover my way of doing research. He gave me encour-agement and orientation in any moment of difficulty. My heartfelt thanks go to myassociate supervisor Professor Matthias Klusch for his kind support and the precisefeedback, which always found the core of a problem. I appreciate his way of findingarguments and good solutions, and his motivation that helped me focus on my research.

Special thanks goes to Mohan Baruwal Chhetri for his insights in writing publicationstogether, to Irene Moser for her kind support, to Bao Quoc Vo for the interesting andcritical discussions that encouraged me to look outside my research topic, and to allcurrent and former members of the Intelligent Agent Technology Group at SwinburneUniversity of Technology. Swinburne University has kindly supported me with a schol-arship during my PhD candidature, making it possible for me to concentrate my effortsfully on the research.

I also extend my thanks to Professor Bogdan Franczyk, Professor Dieter Ehrenberg,Professor Josef Noll and Professor Hans-Jürgen Kaftan, with whom I enjoyed workingbefore my PhD study, for encouraging me to do the PhD, providing references, and foralways having trust in my abilities as a computer scientist.

This research also benefited tremendously from the many friends whom I found duringthe PhD study at the University. In particular, I thank Tino Schlegel, Stefano Bernardiand Bjoern Stütz for hours spent discussing ideas and our research problems over cupsof coffee at the uni or at Mario’s cafe.

Most importantly, I thank my wife Franziska for her love and support throughout thoseyears, especially in the difficult times, and my son Philipp, who had to sacrifice somany playing hours with me. I am deeply thankful to my family and friends back inGermany for their motivation and encouragement, especially my parents, to whom Idedicate this thesis.

Declaration

This thesis contains no material which has been accepted for the award of any otherdegree or diploma, except where due reference is made. To the best of my knowledge,this thesis contains no material previously published or written by another person ex-cept where due reference is made in the text of the thesis.

Jan Richter

Date

Publications

Portions of the material in this thesis have previously appeared in the following pub-lications:

1. J. Richter and R. Kowalczyk. New Mechanisms for Mixing Time- and Behaviour-dependent Tactics in Negotiation Strategies. In Proceedings of the IEEE/WIC/

ACM International Conference on Web Intelligence and Intelligent Agent Tech-

nology (WI-IAT ’08), volume 2, 2008

2. J. Richter and R. Kowalczyk. Mixing Behaviour-dependent and -independentTactics in Multi-issue Negotiation. In Proceedings of the 8th International Joint

Conference on Autonomous Agents and Multiagent Systems (AAMAS ’09), 2009

3. J. Richter, R. Kowalczyk, and M. Klusch. Multistage Fuzzy Decision Making inBilateral Negotiation with Finite Termination Times. In Proceedings of the 22nd

Australasian Joint Conference on Advances in Artificial Intelligence, LectureNotes in Computer Science, Springer Berlin/Heidelberg, 2009.

4. J. Richter, M. Klusch, and R. Kowalczyk. On Monotonic Mixed Tactics andStrategies for Multi-issue Negotiation. In Proceedings of the 9th International

Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS ’10),2010.

5. J. Richter, M. Klusch, and R. Kowalczyk. A Multistage Fuzzy Decision Ap-proach for Modelling Adaptive Negotiation Strategies. In Proceedings of the

IEEE International Conference on Fuzzy Systems, 2010

6. J. Richter, M. B. Chhetri, R. Kowalczyk, Q. B. Vo, M. A. Talib, and A. W.Colman. Utility Decomposition and Surplus Redistribution in Composite SLANegotiation. In Proceedings of the IEEE International Conference on Services

Computing, 2010.

7. J. Richter, M. Baruwal Chhetri, R. Kowalczyk, and Q. Bao Vo. EstablishingComposite SLA’s through Concurrent QoS Negotiation with Surplus Redistribu-

tion. Concurrency and Computation: Practice and Experience, 2011 (acceptedfor publication)

8. J. Richter, M. Klusch, and R. Kowalczyk. Monotonic Mixing of Decision Strategiesfor Agent-based Bargaining. To appear in Proceedings of the Ninth German

Conference on Multi-Agent System Technologies (MATES ’11), Lecture Notes inArtificial Intelligence. Springer Berlin / Heidelberg, 2011.

vii

Contents

Abstract ii

Acknowledgements iv

Declaration v

Publications vi

1 Introduction 11.1 Research Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

1.2 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

1.3 Thesis Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2 Background and Preliminaries 92.1 Game-Theoretic Background . . . . . . . . . . . . . . . . . . . . . . 9

2.1.1 Cooperative Bargaining Theory . . . . . . . . . . . . . . . . 10

2.1.2 Non-Cooperative Bargaining Theory . . . . . . . . . . . . . . 12

2.2 Negotiation Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . 15

2.2.1 Negotiation Model . . . . . . . . . . . . . . . . . . . . . . . 18

2.2.2 Negotiation Thread . . . . . . . . . . . . . . . . . . . . . . . 19

2.2.3 Agents’ Preferences over Outcomes . . . . . . . . . . . . . . 20

2.3 Decision-Making in Automated Negotiation . . . . . . . . . . . . . . 23

2.3.1 Heuristic-based Negotiation Tactics . . . . . . . . . . . . . . 24

2.3.2 Mixing Negotiation Tactics . . . . . . . . . . . . . . . . . . . 27

2.3.3 Mechanisms for Pareto-Efficient Negotiations . . . . . . . . . 28

2.3.4 Learning and Reasoning in Negotiation . . . . . . . . . . . . 31

Contents

2.3.5 Fuzzy Logic-based Approaches in Negotiation . . . . . . . . 36

2.4 Multistage Fuzzy Decision-Making . . . . . . . . . . . . . . . . . . 38

2.4.1 Fuzzy Decision-Making . . . . . . . . . . . . . . . . . . . . 38

2.4.2 Multistage Fuzzy Decision Making in Deterministicand Stochastic Systems . . . . . . . . . . . . . . . . . . . . . 42

2.5 Application Areas for Automated Negotiation . . . . . . . . . . . . . 46

2.6 Simulation Environment and Experimental Evaluation . . . . . . . . 50

2.6.1 Simulation Environment . . . . . . . . . . . . . . . . . . . . 50

2.6.2 General Settings for Experiments . . . . . . . . . . . . . . . 50

2.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

3 Monotonic Mixing Mechanisms for Multi-Tactic Negotiation Strategies 553.1 Dynamic Behaviour of Multi-tactic Strategies . . . . . . . . . . . . . 56

3.1.1 Monotonicity of Negotiation Tactics . . . . . . . . . . . . . . 58

3.1.2 Monotonicity of Multi-tactic Negotiation Strategies . . . . . . 60

3.1.3 Constrained linear weighted combination . . . . . . . . . . . 68

3.2 Mixing based on Negotiation Threads . . . . . . . . . . . . . . . . . 69

3.3 Mixing based on Single Concessions . . . . . . . . . . . . . . . . . . 72

3.4 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

3.4.1 Experiment Settings . . . . . . . . . . . . . . . . . . . . . . 75

3.4.2 Non-Monotonicity of Concession Curves . . . . . . . . . . . 77

3.4.3 Scenario with Small Overlap and Equal Deadlines . . . . . . 79

3.4.4 Scenario with Small Overlap and Different Deadlines . . . . . 82

3.4.5 Scenario with Large Overlap and Equal Deadlines . . . . . . 85

3.4.6 Scenario with Large Overlap and Different Deadlines . . . . . 88

3.5 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

3.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

4 Multistage Fuzzy Decision-Making in Automated Negotiation 934.1 Model with Fuzzy State Transitions . . . . . . . . . . . . . . . . . . 94

4.2 Modelling Negotiation Strategies . . . . . . . . . . . . . . . . . . . . 97

4.2.1 States and Actions . . . . . . . . . . . . . . . . . . . . . . . 98

4.2.2 Fuzzy State Transitions . . . . . . . . . . . . . . . . . . . . . 99

4.2.3 Fuzzy Goal . . . . . . . . . . . . . . . . . . . . . . . . . . . 102

ix

Contents

4.2.4 Fuzzy Constraints . . . . . . . . . . . . . . . . . . . . . . . 103

4.2.5 Modelling Different Negotiation Strategies with FuzzyConstraints . . . . . . . . . . . . . . . . . . . . . . . . . . . 105

4.3 Decision Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . 106

4.4 Negotiation Examples . . . . . . . . . . . . . . . . . . . . . . . . . . 109

4.4.1 Agent using Reference Cases . . . . . . . . . . . . . . . . . . 109

4.4.2 Agent using Preferred Strategy . . . . . . . . . . . . . . . . . 111

4.4.3 Both Agents using Multistage Fuzzy Decision-Making . . . . 113

4.5 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114

4.5.1 Experiment Settings . . . . . . . . . . . . . . . . . . . . . . 114

4.5.2 Scenario with Small Overlap and Equal Deadlines . . . . . . 116

4.5.3 Scenario with Small Overlap and Different Deadlines . . . . . 119

4.5.4 Scenario with Large Overlap and Equal Deadlines . . . . . . 121

4.5.5 Scenario with Large Overlap and Different Deadlines . . . . . 123

4.6 Related Work and Discussion . . . . . . . . . . . . . . . . . . . . . . 126

4.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128

5 Coordinating Strategies in Concurrent Automated Negotiations 1305.1 Composite Service Provisioning . . . . . . . . . . . . . . . . . . . . 131

5.1.1 Definitions and Challenges . . . . . . . . . . . . . . . . . . . 133

5.1.2 Motivating Scenario of Specialized Property Search . . . . . 135

5.1.3 QOS Aggregation . . . . . . . . . . . . . . . . . . . . . . . . 137

5.2 SLA Negotiation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140

5.2.1 Strategic Adjustment of Boundary Values in Multi-tactic Ne-gotiation Strategies . . . . . . . . . . . . . . . . . . . . . . . 141

5.2.2 Strategic Adjustment of Multistage Fuzzy Decision Strategiesvia Fuzzy Constraints . . . . . . . . . . . . . . . . . . . . . . 142

5.3 Utility Boundary Decomposition and Surplus Redistribution . . . . . 143

5.4 Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144

5.5 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149

5.5.1 Experimental Settings . . . . . . . . . . . . . . . . . . . . . 149

5.5.2 Scenario with Two Services . . . . . . . . . . . . . . . . . . 152

5.5.3 Property Search Scenario . . . . . . . . . . . . . . . . . . . . 155

5.6 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159

x

Contents

5.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161

6 Conclusions 1626.1 Answers to Research Questions . . . . . . . . . . . . . . . . . . . . . 165

6.2 Outlook and Future Work . . . . . . . . . . . . . . . . . . . . . . . . 169

Bibliography 170

xi

List of Figures

2.1 Agreement zone for a single-issue negotiation example . . . . . . . . 21

2.2 Polynomial (left) and exponential (right) decision functions (βpoly ∈{9, 4, 2, 1, 0.5, 0.2, 0.05} and βexp ∈ {20, 9, 5, 3, 1.8, 1, 0.5, 0.2}) . . . 26

2.3 Fuzzy decision . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

2.4 Multistage fuzzy decision process . . . . . . . . . . . . . . . . . . . 43

2.5 Interface for single-issue negotiations between two agents . . . . . . . 51

3.1 Offer curves for Examples 3.2 and 3.3 when using the linear weightedcombination or pure tactics . . . . . . . . . . . . . . . . . . . . . . . 64

3.2 Outcomes for different buyer strategy parameters when using linearweighted combinations of tactics . . . . . . . . . . . . . . . . . . . . 65

3.3 Offer and utility curves for Example 3.4 using the traditional linearweighted combination or the negotiation thread-based mixing . . . . . 67

3.4 Offer curves for Examples 3.2 and 3.3 when using the constrained lin-ear weighted combination (compared to the traditional linear weightedcombination) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

3.5 Offer curves for examples 3.2 and 3.3 when using the negotiationthread-based mixing (compared to the traditional linear weighted com-bination) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

3.6 Offer curves for Example 3.3 when using the concession-based mixing(compared to the traditional linear weighted combination) . . . . . . . 74

3.7 Client (left) and provider (right) average utilities in the ’one-sidedwithout withdraw’ scenario (client uses different mixing mechanismswhile provider always uses traditional mixing) with small overlap andequal deadlines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

3.8 Client (left) and provider (right) average utilities in the ’one-sided withwithdraw’ scenario with small overlap and equal deadlines . . . . . . 80

List of Figures

3.9 Client (left) and provider (right) average utilities in the two-sided scen-ario (both agents use the same mixing mechanism) with small overlapand equal deadlines . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

3.10 Average negotiation length (left) and agreement rates (right) for the’one-sided without withdraw’ scenario with small overlap and equaldeadlines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

3.11 Average negotiation length (left) and agreement rates (right) for the’one-sided with withdraw’ scenario with small overlap and equal dead-lines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

3.12 Average negotiation length (left) and agreement rates (right) for thetwo-sided scenario with small overlap and equal deadlines . . . . . . 82

3.13 Client (left) and provider (right) average utilities in the ’one-sidedwithout withdraw’ scenario with small overlap and different deadlines 83

3.14 Client (left) and provider (right) average utilities in the ’one-sided withwithdraw’ scenario with small overlap and different deadlines . . . . 83

3.15 Client (left) and provider (right) average utilities in the two-sided scen-ario with small overlap and different deadlines . . . . . . . . . . . . . 83

3.16 Average negotiation length (left) and agreement rates (right) for the’one-sided without withdraw’ scenario with small overlap and differentdeadlines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

3.17 Average negotiation length (left) and agreement rates (right) for the’one-sided with withdraw’ scenario with small overlap and differentdeadlines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

3.18 Average negotiation length (left) and agreement rates (right) for thetwo-sided scenario with small overlap and different deadlines . . . . . 84

3.19 Client (left) and provider (right) average utilities in the ’one-sidedwithout withdraw’ scenario with large overlap and equal deadlines . . 85

3.20 Client (left) and provider (right) average utilities in the ’one-sided withwithdraw’ scenario with large overlap and equal deadlines . . . . . . 86

3.21 Client (left) and provider (right) average utilities in the two-sided scen-ario with large overlap and equal deadlines . . . . . . . . . . . . . . . 86

3.22 Average negotiation length (left) and agreement rates (right) for the’one-sided without withdraw’ scenario with large overlap and equaldeadlines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86

3.23 Average negotiation length (left) and agreement rates (right) for the’one-sided with withdraw’ scenario with large overlap and equal dead-lines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

xiii

List of Figures

3.24 Average negotiation length (left) and agreement rates (right) for thetwo-sided scenario with large overlap and equal deadlines . . . . . . . 87

3.25 Client (left) and provider (right) average utilities in the ’one-sidedwithout withdraw’ scenario with large overlap and different deadlines 88

3.26 Client (left) and provider (right) average utilities in the ’one-sided withwithdraw’ scenario with large overlap and different deadlines . . . . . 88

3.27 Client (left) and provider (right) average utilities in the two-sided scen-ario with large overlap and different deadlines . . . . . . . . . . . . . 89

3.28 Average negotiation length (left) and agreement rates (right) for the’one-sided without withdraw’ scenario with large overlap and differentdeadlines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89

3.29 Average negotiation length (left) and agreement rates (right) for the’one-sided with withdraw’ scenario with large overlap and differentdeadlines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89

3.30 Average negotiation length (left) and agreement rates (right) for thetwo-sided scenario with large overlap and different deadlines . . . . . 90

4.1 Multistage fuzzy decision process of a negotiation agent . . . . . . . 97

4.2 Example fuzzy goal (left) and utility function (right) for a partial over-lap of negotiation intervals . . . . . . . . . . . . . . . . . . . . . . . 103

4.3 Example fuzzy constraint . . . . . . . . . . . . . . . . . . . . . . . . 104

4.4 Examples for different time-dependent fuzzy constraints . . . . . . . 106

4.5 Inference example for expected fuzzy goal and fuzzy case constraints 110

4.6 Example offer curves for an agent using two reference cases and caseconstraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

4.7 Inference example for the expected fuzzy goal and fuzzy constraints ofa preferred strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . 112

4.8 Example offer curves for an agent using two reference cases and time-dependent fuzzy constraints . . . . . . . . . . . . . . . . . . . . . . . 112

4.9 Example offer curves when both agents use the multistage fuzzy de-cision approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114

4.10 Example cases for the multistage fuzzy strategy . . . . . . . . . . . . 116

4.11 Results for the multistage fuzzy strategy and the average mixed strategiesin the scenario with small overlap and equal deadlines . . . . . . . . . 117

4.12 Average utility (top) and agreement rate of the multistage fuzzy strategyusing boulware fuzzy constraints and the average mixed strategy usingthe traditional mechanism in the scenario with small overlap and equaldeadlines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118

xiv

List of Figures

4.13 Results for the multistage fuzzy strategy and the average mixed strategiesin the scenario with small overlap and different deadlines . . . . . . . 120

4.14 Average utility (top) and agreement rate of the multistage fuzzy strategyusing boulware fuzzy constraints and the average mixed strategy usingthe traditional mechanism in the scenario with small overlap and dif-ferent deadlines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121

4.15 Results for the multistage fuzzy strategy and the average mixed strategiesin the scenario with large overlap and equal deadlines . . . . . . . . . 122

4.16 Average utility (top) and agreement rate of the multistage fuzzy strategyusing boulware fuzzy constraints and the average mixed strategy usingthe traditional mechanism in the scenario with large overlap and equaldeadlines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

4.17 Results for the multistage fuzzy strategy and the average mixed strategiesin the scenario with large overlap and different deadlines . . . . . . . 124

4.18 Average utility (top) and agreement rate of the multistage fuzzy strategyusing boulware fuzzy constraints and the average mixed strategy usingthe traditional mechanism in the scenario with large overlap and dif-ferent deadlines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125

5.1 Composite service provisioning scenario . . . . . . . . . . . . . . . . 133

5.2 Service process fulfilling the business service of finding suitable prop-erties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136

5.3 Service process in tree form . . . . . . . . . . . . . . . . . . . . . . 139

5.4 Negotiation example with and without smoothing function for chan-ging boundaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141

5.5 Average end-to-end utility (top) and agreement rate (bottom) withoutand with surplus redistribution for the scenario with two services andagents using static mixed strategies with the traditional linear weightedcombination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152

5.6 Average end-to-end utility (top) and agreement rate (bottom) withoutand with surplus redistribution for the scenario with two services andagents using static mixed strategies with the negotiation thread-basedmechanism . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153

5.7 Average end-to-end utility (top) and agreement rate (bottom) withoutand with surplus redistribution for the scenario with two services andagents using static mixed strategies with the concession-based mech-anism . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154

xv

List of Figures

5.8 Average end-to-end utility (left) and agreement rate (right) withoutand with surplus redistribution for the scenario with two services andagents using the multistage fuzzy strategy with different time-dependentfuzzy constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155

5.9 Average end-to-end utility (top) and agreement rate (bottom) withoutand with surplus redistribution for the property search scenario witheight services and agents using static mixed strategies with the tradi-tional linear weighted combination . . . . . . . . . . . . . . . . . . . 156

5.10 Average end-to-end utility (top) and agreement rate (bottom) withoutand with surplus redistribution for the property search scenario witheight services and agents using static mixed strategies with the negoti-ation thread-based mechanism . . . . . . . . . . . . . . . . . . . . . 157

5.11 Average end-to-end utility (top) and agreement rate (bottom) withoutand with surplus redistribution for the property search scenario witheight services and agents using static mixed strategies with the concession-based mechanism . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158

5.12 Average end-to-end utility (left) and agreement rate (right) without andwith surplus redistribution for the property search scenario with eightservices and agents using the multistage fuzzy strategy with differentfuzzy constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159

xvi

List of Tables

2.1 Parameters for strategy groups . . . . . . . . . . . . . . . . . . . . . 52

3.1 Negotiation settings for example 3.4 . . . . . . . . . . . . . . . . . . 66

3.2 Non-monotonicity in negotiations . . . . . . . . . . . . . . . . . . . 78

5.1 Aggregation functions . . . . . . . . . . . . . . . . . . . . . . . . . 138

Chapter 1

Introduction

Negotiation is an important form of social interaction that facilitates conflict resolution

between individuals, organisations or any kind of human parties. For that reason, nego-

tiation has been studied extensively from different perspectives and in many research

areas, including the social sciences [106], economics [108, 83] and psychology [32].

The rapid development of computing systems and networks over the past few decades

and the emergence of complex and large distributed systems, such as the Grid [48],

the semantic web [10], service-oriented computing [40], or recently cloud computing

[22], has led to a demand for new types of interactions between software components

and between humans and computing systems.

A key mechanism for resolving conflicts in decentralized systems composed out of

computational, intelligent agents is automated negotiation. The agents, acting autonom-

ously on behalf of their users, interact with each other in order to fulfil certain goals or

objectives. Due to the characteristics of these environments in terms of their open and

dynamic architecture, computational distribution, lack of global or centralized know-

ledge and dispersed control of resources [48], automated negotiation has been studied

widely among the research fields of game theory [12], artificial intelligence [50] and

agent technology [64, 89], and for its potential in many real world applications. These

include e-commerce [137], resource allocation and scheduling [86], task distribution

[15] and lately service composition [28].

A fundamental setting is that of bilateral negotiation, wherein two agents bargain for

a product or service by alternately exchanging offers [122]. The preferences of an

agent are typically represented by its utility function, including its reservation limits

Chapter 1. Introduction

and deadlines. The utility function orders all possible outcomes by assigning a score

to each value in the outcome space. In order to find optimal decision strategies and

solutions, classical game-theoretic approaches make strict assumptions of complete

knowledge and unbounded rationality. When the agents have complete knowledge, the

preferences, or at least the beliefs about the preferences, are common knowledge. If an

agent unbounded rational, it is endued with an unlimited capacity for its reasoning and

decision-making. This enables the finding of optimal decision strategies and solutions,

though, in large problem domains, this might become intractable [78].

More realistic assumptions in open and distributed systems, however, are that agents

do not know the decision model and preferences of their opponents, i.e. that these are

private information, and that they have limited resources. For that reason, research in

artificial intelligence focuses on tractable and more realistic approaches for an agent’s

decision-making in order to find good, rather than optimal, solutions. In this field,

a common distinction based on the interaction behaviour of an agent is whether an

agent is competitive or cooperative. For example, in multi-issue negotiations agents

can submit partial preferences to a trusted, unbiased third party such as a mediator

who helps the agents to find efficient solutions, i.e. outcomes which maximize the

social welfare. Such a mediator, however, might not be available or trusted in such

environments. When agents are competitive they do not disclose any information about

their decision apparatus or their preferences nor use a third party to mediate. The

agents are rational in the sense that they aim to achieve outcomes with the highest

possible score, but at the same time have a common interest in finding an agreement

before an agent reaches its deadline. It is this characteristic that makes the process

of negotiation in this setting of incomplete information unique, due to the conflict in

which all the parties involved find themselves engaged as it incorporates competitive

and cooperative elements at the same time [64].

Decision-making in such situations is known to be hard [45, 81, 89] and a large number

of models have been proposed and investigated to solve this problem. These range from

If-then rules and heuristic-based approaches [44] to more advanced learning and reas-

oning techniques [15] such as Case-based Reasoning [132], Bayesian reasoning [148],

evolutionary algorithms [93], reinforcement learning [26], neural networks [103] or

non-linear regression analysis [18].

To negotiate effectively and efficiently, many of the proposed decision approaches are

2

built upon relatively strong assumptions. For example, in experience-based approaches

the agents are required to either have prior knowledge in the form of empirical data, or

to learn by exploring the environment and the negotiation partner’s behaviour. Probab-

ilistic approaches assume that the agent has partial knowledge in the form of probab-

ility distributions over some of the opponent’s parameters, e.g. derived from domain

knowledge or historical interactions, while regression-based approaches assume a set

of underlying decision models from which the opponent may choose. In many situ-

ations, these assumptions appear difficult to fulfil due to the above-mentioned char-

acteristics of such systems, the competitive interactions or the long learning times

required to gain precise knowledge. Most importantly, the decision models have to

cope with the dynamic nature of the system, since the agents expose different beha-

viours and may enter or leave the system at any time. As a result, the knowledge an

agent has at its disposal is limited. For example, it might be derived from the inform-

ation available of a few past interactions, the current encounters, or some states in the

environment.

This thesis focuses on decision-making strategies for automated negotiation in compet-

itive environments which are able to react to the dynamic nature of the system and the

different behaviours of opponents when only limited knowledge about the negotiation

partner or environment is available. Of particular interest is the strategic concession

behaviour of an agent, i.e. when and how much an agent should concede in order to

obtain the best outcomes. The work first presents and investigates two decision mech-

anisms, an existing heuristic-based approach and a novel decision model based on

multistage fuzzy decision-making, that are suitable for situations in which the agent

has only limited knowledge, and then proposes a mechanism for coordinating such

strategies in more complex and realistic concurrent negotiation scenarios.

We investigate the method of mixing heuristic-based decision functions, or tactics

[46, 44], in order to create multi-tactic negotiation strategies in the absence of prior

information. Despite having the advantage of being able to react to a range of different

factors simultaneously, such as the opponent’s behaviour, the remaining time or the

state of a resource, the mixing mechanism itself and its effect on the strategic conces-

sion behaviour of an agent has not been investigated previously. The traditional method

of a linear combination of offers can expose quite complex and dynamic behaviour, but

poses an important problem in that it can not guarantee monotonic concession curves

3


even in cases where all involved tactics are monotonic, i.e. they propose positive con-

cessions, and mixing weights are static. Since this behaviour might not be desirable in

many situations, alternative mixing mechanisms are proposed based on linear combin-

ations of individual negotiation threads for each imitative tactic or single concessions

which guarantee monotonic concession curves for monotonic tactics in both cases,

static and dynamic.

A novel decision model for an agent’s negotiation strategy based on multi-stage fuzzy

decision making is proposed. In this model, the agents’ individual preferences are

expressed via fuzzy goals and constraints whilst the dynamics of the negotiation are

modelled as a fuzzy Markov decision process which represents the relation between

the strategic concession behaviour between the two agents involved. The offers and

counteroffers of the agents correspond to state-action pairs in the negotiation process,

so that individual fuzzy state transitions enable an agent to utilize limited knowledge

about the concession behaviour of its opponent, for example, by using only a few ref-

erence cases. The problem of finding the best course of actions to achieve the desired

outcome can then be solved via fuzzy dynamic programming. By imposing the fuzzy

constraints on the decision-process, an agent is able to generate different concession

behaviours depending on the chosen preferences. Furthermore, the fuzzy representa-

tion of the decision process in this negotiation context allows the application the de-

cision strategy in many real world scenarios in which the available information about

agents’ behaviours, preferences and constraints is imprecise.

Finally, a mechanism for the coordination of negotiation strategies in one-to-many

bilateral concurrent negotiations is presented by using a more realistic and complex

example scenario in the domain of service-oriented computing. In this scenario, a

number of service level agreements need to be negotiated with service providers in

order to establish a workflow-based composite service. The mechanism uses the meth-

ods of utility boundary decomposition to derive the negotiation limits for each atomic

service agent and the consequent surplus redistribution of successfully finished nego-

tiations in order to increase the number of compound agreements. At the same time,

the agents on the service level remain in control of their concession behaviour and are

able to negotiate competitively. The example scenario further demonstrates the applic-

ability of the decision-making approaches presented in this thesis. An experimental

evaluation is presented to validate each of the decision mechanisms.

4

1.1. Research Questions

1.1 Research Questions

The work presented in this thesis is driven by the following research questions in

decision-making situations in which agents face the problem of limited knowledge

during negotiation:

Mixing mechanisms for multi-tactic negotiation strategies with static and dynamicweights guaranteeing monotonic concession curves for monotonic tactics

1. What mechanisms can generate monotonic concession curves in multi-tactic ne-

gotiation strategies when monotonic tactics are mixed using static or dynamic

weights?

2. How much do outcomes differ when using the alternative mixing mechanisms

compared to the traditional method, and in which scenarios can an agent improve

its utility?

A decision model for an agent’s strategic concession behaviour in automated nego-tiation based on multi-stage fuzzy decision making

1. How can an agent model the negotiation process as a multistage fuzzy decision

problem when only limited knowledge about the concession behaviour of the

opponent is available?

2. What are the advantages and disadvantages of the proposed multistage fuzzy de-

cision model compared to the heuristic-based or other approaches, and in which

scenarios can an agent gain in utility by using this approach?

A mechanism for coordinating negotiation strategies in concurrent negotiationsbased on utility boundary decomposition and surplus redistribution

1. How can negotiation strategies be efficiently coordinated in more complex ne-

gotiation scenarios with many bilateral concurrent negotiations?

2. Are the proposed negotiation decision-models and mechanisms applicable in real

world domains such as service-oriented computing?

The above research questions will be answered in Section 6.

5


1.2 Contributions

The main contributions of this thesis are as follows:

1. Mixing mechanisms for multi-tactic negotiation strategies with static and dynamicweights guaranteeing monotonic concession curves for monotonic tacticsWe investigate the dynamic behaviour of heuristic, multi-tactic negotiation strategies

created by linear weighted combinations, and demonstrate that non-monotonicity in the

concession curve of the agents can also occur when imitative and non-imitative tactics

are mixed using static weights and all tactics involved are monotonic. We discuss the

possible undesirable effects which can occur as a result of this mixing technique when

used in negotiation situations with limited knowledge, and propose new mixing mech-

anisms that solve this problem, the first based on individual negotiation threads and the

second based on single concessions of each tactic, and prove that these mechanisms

guarantee monotonic concession curves for monotonic tactics, the first for static and

the second also for dynamic weights. An experimental evaluation validates and com-

pares the proposed mechanisms against the traditional method. This work has been in

part published in [114, 115, 112, 113].

2. A decision model for an agent’s strategic concession behaviour in automated ne-gotiation based on multi-stage fuzzy decision makingWe propose a new decision model for the modelling of an agent’s strategic conces-

sion behaviour in automated negotiation based on multi-stage fuzzy decision making.

In this model, the agent’s preferences are modelled using a fuzzy goal and fuzzy con-

straints, while the fuzzy state transitions are created using limited and imprecise know-

ledge, for example, from only few reference cases. We show how the fuzzy constraints

enable an agent to impose different strategic preferences on the decision-making pro-

cess in order to create different concession behaviours. The decision algorithms for the

model are presented and the limitations and advantages compared to other approaches

are discussed. We validate the model in a series of experiments with different strategy

settings and negotiation deadlines, demonstrating that this modelling framework is

able to provide utility gains in many scenarios with limited and uncertain available

knowledge. This work has been in part published in [116, 111].

6

1.3. Thesis Overview

3. A mechanism for coordinating negotiation strategies in concurrent negotiationsbased on utility boundary decomposition and surplus redistributionUsing a more realistic example scenario in the domain of service-oriented comput-

ing, we present a decision mechanism which enables the coordination of negotiation

strategies in complex, one-to-many bilateral negotiations with limited knowledge. In

the chosen scenario, a number of agents concurrently negotiate service level agree-

ments with service providers in order to establish a workflow-based composite service.

We show that the mechanism can increase the number of compound agreements by the

method of utility boundary decomposition and surplus redistribution of successfully

finished negotiations while leaving the control over the concession behaviour to the

individual service agents in order to enable competitive negotiations. An experiment

using the example SLA-negotiation scenario demonstrates the applicability of the pro-

posed decision-making strategies in this thesis and the coordination mechanism. This

work has been in part published in [110, 109].

.

1.3 Thesis Overview

The thesis is further organised in six chapters.

Chapter 2 introduces basic notions of automated negotiation and related work in the

context of decision-making in single- and multi-issue negotiation, as well as various

learning and reasoning models studied in this research area. A number of potential

application areas and scenarios presented in the literature are pointed out.

Chapter 3 investigates the dynamic aspects of heuristic, multi-tactic strategies that are

suitable for decision-making situations in negotiations with limited available know-

ledge. It investigates the monotonic concession behaviour of such strategies when the

tactics involved are mixed using static or dynamic weights, and also when tactics are

of different types, such as imitative and non-imitative. The problem of the automatic

and uncontrolled occurrence of non-monotonic concession curves in static cases is

demonstrated, and new mixing mechanisms are presented which solve that problem

based on linear weighted combinations of single concessions or individual negotiation

threads of each of the imitative tactics involved. By means of descriptive examples

7


and negotiation experiments, we show that such undesired behaviour can change the

outcome or delay agreements significantly in many scenarios, and that the proposed

mixing mechanisms guarantee monotonic concession curves for monotonic tactics.

Chapter 4 presents the multi-stage fuzzy decision model and shows how dynamic ne-

gotiation strategies are modelled when only limited knowledge about the negotiation

partner’s concession behaviour is available, for example, in the form of reference cases

or a few past interactions. It is shown how the preferences of an agent are modelled

using a fuzzy goal and fuzzy constraints, and how the concession behaviour of an op-

ponent is represented by the fuzzy state transitions. Different negotiation strategies

are modelled using the time-dependent fuzzy constraints which allow the agent to in-

fluence the proposed course of actions of the reference cases. The solution method

of fuzzy dynamic programming is presented, in addition to the decision-making al-

gorithm for policy generation and the proposal of offers during the negotiation en-

counter. A number of negotiation scenarios with different deadlines, negotiation inter-

vals and strategies validate the proposed decision model, alongside a discussion of the

experimental results.

Chapter 5 presents the coordination mechanism for negotiation strategies in more

complex and realistic scenarios with multiple bilateral, concurrent negotiations. We il-

lustrate an example scenario situated within a service-oriented computing environment

in which a number of agents negotiate service-level agreements with service providers

in order to establish a composite service while having only limited knowledge about

the provider agents. The algorithms for the utility boundary decomposition and the

generation of reservation limits for each atomic service agent, and the surplus redis-

tribution of successfully finished negotiations among the remaining negotiations, are

shown. Finally, an experiment demonstrates the improvement in terms of the number

of compound agreements and utility gain that can be achieved by these methods and

that the decision mechanisms presented in this thesis are applicable in more complex

negotiation scenarios.

Chapter 6 draws conclusions about the decision mechanisms presented, answers the

research questions and discusses interesting future work in the area of decision-making

in automated negotiation with limited knowledge.

8

Chapter 2

Background and Preliminaries

This chapter provides the fundamentals in the area of automated negotiation and dis-

cusses the problem of strategic decision-making in bilateral negotiations. It presents

the game-theoretic background and the preliminaries to negotiation, such as the negoti-

ation model, thread, and an agent’s preferences, are presented, and the basic heuristic-

based decision model for the strategic concession-making of an agent in a negotiation is

also introduced. Various proposed approaches in related work for an agent’s decision-

making in different negotiation situations are reviewed. It also contains a discussion of

approaches in the field of Artificial Intelligence (AI) for learning and reasoning about

the opponent’s behaviour in order to make better decisions in negotiation situations

when preferences and decision models are private, and the available information is

limited or uncertain. The decision models of multistage fuzzy control that provide

the basis for the multistage fuzzy decision model in negotiation examined in Chapter

4 of this thesis is introduced. Important areas of potential application for distributed

negotiation are outlined, while the experimental environment for the evaluation of the

proposed decision strategies in this thesis is also presented.

2.1 Game-Theoretic Background

The field of game theory has laid the foundation for negotiation research from the

economics perspective, and provides insight into the decision-making process of the

parties, especially through its study of bargaining games. While the game-theoretic

Chapter 2. Background and Preliminaries

research focuses on finding unique solutions that are optimal given the preferences of

all agents and the set of their possible choices, it also aims at the analysis of equilibrium

solutions and the strategies that lead to them. The outcomes of a bargaining game are

often denoted in terms of utilities. The utility function of a player assigns a value to

each possible outcome of the game specifying how much the player prefers a particular

outcome. A bargaining solution is then typically represented by the set of utility pairs

of both players.

In order to enable the mathematical analysis, game theory makes often strict and sim-

plifying assumptions; the most common are that players have complete knowledge

about the game and its players, and that all players are unbounded rational. The as-

sumption of common knowledge implies that all players not only know the rules of

the game, but have full knowledge about the preferences of other players or at least

the beliefs about their preferences. The rationality assumption means that a player

selects the best strategy maximizing its payoff from the space of all strategies, given

all possible interactions (and strategies of the players) and the beliefs of the player.

This implies that the agents are endued with unlimited computational resources to al-

low such reasoning and calculation of optimal decision strategies. This intractability of

many game-theoretic approaches makes their application impractical for many realistic

negotiation situations [64].

Under the above assumptions bargaining theory distinguishes between cooperative and

non-cooperative approaches which, depending on whether an agreement is binding or

not, focus on different aspects of bargaining games, such as optimal solutions given a

set of axioms, or the equilibrium strategies given the decision-making process of the

parties during the game. The next sections give a brief overview of both theories. For

more details, we refer to the excellent surveys and introductions on bargaining theory

in relation to negotiation mechanisms in artificial intelligence in [80, 50, 64, 84].

2.1.1 Cooperative Bargaining Theory

In cooperative bargaining theory the parties are supposed to be able to discuss the situ-

ation and agree on some joint actions while the agreement is assumed to be binding

for both parties [97]. This means, that an agreement is enforceable, for example, by

a third party which can impose a penalty to any party deviating from the agreement.

10

2.1. Game-Theoretic Background

Under this assumption, cooperative bargaining theory is able to focus on the space of

possible outcomes of a bargaining game while leaving the process of negotiation un-

specified. In other words, it abstracts away from the actual details of the game and

the decision processes of all parties by defining a set of axioms which, by representing

desirable properties, uniquely define a rational solution. Such an approach was first

proposed by Nash [97] who defined the following axioms for a bargaining solution:

(a) independence of the utility scale, (b) each outcome pair is rational and Pareto-

efficient, (c) independence of alternatives, and (d) both parties get the same utility in

symmetric situations. The first axiom means that the final solution should not depend

on the scale of the player’s utility function (i.e. that the same outcome is obtained

after an affine transformation of the utility function), since players may use different

functions to represent their preferences. The second axiom states that a solutions is

rational if it obtains for each player a utility that is at least as large as the utility at the

disagreement point, and, that it is Pareto-efficient if no other solution can increase the

utility for a player without making any other player worse off. The third axiom relates

to alternative feasible agreements in that they are not considered if the current agreed

solution is also feasible. The last axiom holds for cases where the parties have the same

preference structure and, as a result, obtain the same utility. Nash proved that under

these properties there is a unique Nash-bargaining solution that corresponds to a pay-

off pair s = (x1, x2) that maximizes the so-called Nash-product (x1 − d1)(x2 − d2)

where x1 and x2 are the pay-offs for agent 1 and 2, respectively, for solution s, and

d1 and d2 are the payoffs in case of a disagreement (the conflict point). Especially

Nash’s third axiom of independent alternatives has been discussed controversially and,

as a result, other axiomatic bargaining solutions have been proposed such as the Kalai-

Smorodinski or utilitarian solution [70]. The former replaces Nash’s third axiom with

an axiom of individual monotonicity. This enables to use the maximum feasible utility

region of both parties based on their disagreements points to construct the final solu-

tion. The latter, utilitarian solution, aims at maximizing the social welfare, i.e. the

sum of the individual players utilities of an outcome. This implies that the first axioms

of the independence of the utility scales no longer holds and that the players’ utilities

are comparable. Bargaining solutions have also been considered from a more practical

viewpoint of an outcomes fairness given each partner’s preferences, for example, by

Raiffa [108]. Besides the assumption of a binding agreement in cooperative bargain-

ing theory, axiomatic approaches require full knowledge of the details of the game and

11


complete preferences of each party. This makes the rather theoretical approaches of

cooperative bargaining impractical for determining solutions in more realistic negoti-

ation situations with no common knowledge and limited rationality.

It should be noted that the notion of cooperativeness also appears in relation to Pareto-

efficiency in bargaining situations with incomplete information and multiple issues. In

such cases, the parties can aim to obtain an outcome at the Pareto-frontier, for example,

by making trade-off proposals. Such approaches are especially considered in the field

of artificial intelligence, which are discussed in more detail in Section 2.3.3.

2.1.2 Non-Cooperative Bargaining Theory

In non-cooperative bargaining theory the agreement is assumed to be non-binding,

i.e. it is not possible to enforce it, for example, by a third party. The players in the

game make decisions independently, and, while they may be able to cooperate, any

occurring cooperation is self-enforced. Non-cooperative bargaining theory therefore

focuses on the negotiation process and its determining factors such as the specific

rules of the game, or protocol, and the decision apparatus the parties may use. Given

the protocol and the set of players’ strategies, the aim is then to find the equilibrium

solutions that determine rational outcomes of the game. Generally, a strategy is said

to be an equilibrium strategy if there is no incentive for the player to deviate from it

given the strategies of all other players. Most common equilibria concepts are, for

example, dominant strategies, Nash equilibrium or the sub-game perfect equilibrium.

While a dominant strategy is optimal in every situation, i.e. for any of the strategies

of the other players, the Nash equilibrium represents a strategy combination in which

no player can gain by changing its strategy. The sub-game perfect equilibrium is a

particular equilibrium concepts for tree-like, extensive form games, in which each sub-

game resembles a Nash-equilibrium.

A fundamental setting in non-cooperative bargaining is that of dividing a surplus between

two parties, which has led to the study of different protocols and games such as the

Nash bargaining game, the ultimatum game, the monotonic concession or the alternat-

ing offers game [13]. In the Nash bargaining game two players simultaneously demand

a share of the surplus without knowing the other player’s demand. If the sum of the

demands does not exceed the surplus, both players get what they requested, otherwise

12

2.1. Game-Theoretic Background

they receive only the disagreement payoff (conflict outcome). In this game all Pareto-

efficient outcomes represent Nash equilibria [98] and also the case when both parties

ask for the whole surplus.

In the ultimatum game one player proposes a split of the surplus whereas the other

player can only accept or refuse this proposal. In the latter case, none of the players

gets a share. The proposer thus has more bargaining power than its partner. The game

has an infinite number of Nash-equilibria but only one sub-game perfect equilibrium

in which the proposing agent demands the whole surplus and the partner accepts [13].

The alternating offers game represents a multi-stage extension of the ultimatum game

in that after the first player proposed a share of the surplus, the second player can re-

fuse the offer or make a counterproposal at the next stage. Vice versa, if the first player

refuses the second player’s counteroffer, the first player proposes a new counteroffer

at the following stage and so on. The process finishes when one party agrees or a finite

deadline is reached. A version of this game for a single-issue with a finite number of

alternatives and a finite deadline has been studied in [128] where the theoretical ana-

lysis is simplified by the assumption that both parties can not increase their demands.

Under further assumptions of perfect rationality and complete information, optimal

strategies are obtained by backward induction starting with the last stage. Rubinstein

[122] presents a variant of the alternating-offer game with infinite horizon and continu-

ous alternatives. Because, in general, the game could go on forever if no player accepts

an offer, two models with different kinds of discounting are analysed, the first with a

fixed bargaining cost for each period, and the second with a fixed discounting factor for

each player. The discounting thus resembles how impatient a party is. It is shown that

in this alternating offers game the Nash-equilibrium is too weak to identify a unique

solution since every outcome of the surplus partitioning represents a Nash equilibrium

but that the concept of sub-game perfect equilibrium obtains a unique solution. How-

ever, the strong assumption of complete information still holds in this game.

The monotonic concession protocol represents a more restricted protocol compared to

the alternative-offers game. In this protocol, the two players announce their proposals

simultaneously. If both offer overlap in that they match or exceed the other agent’s

demand, an agreement is reached. If both proposals do not overlap, then the agents

either make a concession or repeat their proposal from the previous round. If neither

agent concedes, the negotiation ends and each party receives the conflict payoff. The

13


unique characteristic of this protocol is that each party is not allowed to make offers

with a lower utility to their counter player. Because of this property, at least one player

has to concede at each round or a disagreement is reached, such that the process is

finite if the minimum concession is fixed and larger than zero. In order to make mono-

tonic concessions possible the players need to have knowledge about the preferences of

each other, especially in cases with multiple issues, where also the relative importance

between the issues is important.

The above games and protocols provide the basis for the study of a wide range of

decision strategies. For example, Rosenschein and Zlotkin [121] discuss strategies

using the monotonic concession protocol in terms of stability and efficiency, where

a pair of strategies is considered efficient if it reaches an agreement, and stable, if it

resembles a Nash equilibrium. A protocol, such as in the alternating offers game, may

only define the rules of the interaction between the parties while permitting each party

to choose different kinds of decision strategies. On the other hand, a protocol may

also constrain a strategy in that the players have a limited range of choices in order to

enforce certain outcomes. An example is the Zeuthen strategy [150] which represents

a rational strategy when using the monotonic concession protocol [53]. Using this

strategy, a party concedes only if it has the more to lose than the opponent in a case of

an immediate negotiation failure. In other words, the party with the highest risk should

concede while the amount of the concession needs to change the balance of the risk

such that the opponent needs to concede next. Although, the Zeuthen strategies are

proven to achieve Pareto-optimal deals [53], they are not in equilibrium as the parties

have an incentive to deviate from the strategy at the last stage [121]. The bargaining

procedure may also change when more than one issue is involved in the game. The

issues may be negotiated simultaneously, separately, or issue-by-issue in a sequential

manner. However, most of the approaches for non-cooperative bargaining suggest to

negotiate issues sequentially.

The bargaining games above assume that all parties have complete knowledge about

the preferences of each other in order to reduce the complexity of the game for the

mathematical analysis in terms of equilibrium strategies. More realistic, however, is

that the parties do not know the preferences of the other players, such as the reservation

values, utility functions, risk attitudes or the individual evaluation of their issues. Such

games are also referred to bargaining with incomplete information, and are typically

14

2.2. Negotiation Preliminaries

modelled by using a limited number of player ’types’ that are associated with different

preference structures and beliefs that are unknown [50, 5]. Under this premise, there

are basically two general approaches, mechanism design and sequential bargaining.

Rather than modelling the game as a sequence of offers and counteroffers, mechanism

design focuses on the solution concepts of a game in an abstract way given the private

information and the space of possible outcomes. This allows to study the incentives

of the players, the attainable outcomes, and to identify equilibrium solutions such as

the Bayesian-Nash equilibrium [130]. Games modelled by the mechanism design ap-

proach are usually solved using mediated mechanisms [71] where the players disclose

their types. The second approach, sequential bargaining, considers the dynamic pro-

cess of the offer exchange between the parties when either one party or both parties

have private information about their preferences. In these settings, also referred to

as one-sided and two-sided incomplete information games, different equilibrium con-

cepts are studied such as the sequential or Bayes-Nash equilibrium, for example, by

Rubinstein for one-sided [123] and by Chatterjee and Samuelson [27] for two-sided

incomplete information games with infinite horizons. The common assumption for all

incomplete information games, however, is that the private information can be repres-

ented as a finite set of player types. The uncertainty over the particular type of a partner

in the form of a players’ beliefs is then represented by a probability distribution over

all types which is, again, common knowledge. Although this increases the complexity

of the game it still makes the computation of optimal solutions possible. In many real-

istic negotiations, however, the agents do not know the beliefs about the other agent’s

preferences when they are private. As a result, research in the field of artificial intel-

ligence (AI) aims at relaxing the strict assumptions of game theory in order to enable

automated negotiation in situations that are closer to the real world.

2.2 Negotiation Preliminaries

Negotiation with its broad range of characteristics, approaches and phenomena has

been studied extensively from different perspectives and in many research areas in-

cluding social sciences [106], economics [108, 83] and psychology [32]. There is a

particular interest in the automation of the negotiation process between self-interested

software agents in order to facilitate decision-making and conflict resolution in dis-

15


tributed and autonomous computing systems. Consequently, there is a potential for

automated negotiation in many real world applications such as e-commerce, task re-

distribution and scheduling, resource allocation, and recently service-oriented comput-

ing. While game theory provides insights into the decision-making process of agents,

its strict assumptions such as complete knowledge of preferences and beliefs, and the

full rationality of players limit the application in more realistic settings. The field of

artificial intelligence attempts to relax these assumptions in order to enable the design

of more practical mechanisms. Despite the variety of research topics in automated

negotiation, one can distinguish in general three areas with respect to the negotiation

protocol, the objects under negotiation and the decision models applied for the offer

proposal [64]:

• Negotiation protocol: The rules of the interaction between all agents involved

in the negotiation are determined by the negotiation protocol. In general, it spe-

cifies what types of messages can be sent to whom and when. This includes the

permissible types, or roles of participants such as the negotiators or any relev-

ant third parties, and the valid actions they may choose during the encounter. It

also defines the possible states of the negotiation and the events that may change

them, e.g. the acceptance of a proposal, no more bidders, or that a negotiation is

closed.

• Negotiation object: The negotiation object comprises the set of issues under

negotiation over which an agreement is to be met. Such issues may include for

example price, quality or response time. In conjunction with the protocol the

negotiation object also determines the types of operations that can be performed

on the object. For example, if the content of the agreement is fixed, agents can

only accept or reject, whereas in the non-fixed case, agents are able to make

counter-proposals in order to find a better fitting agreement. The number of

negotiation issues also influences the possible action of an agent as well as the

nature of the overall encounter. For example, in the case of a single issue the

negotiation is competitive since a gain for one agent represents a loss for the

other. In the case of multiple issues, on the other hand, agents can also cooperate

in that they search for joint gains, i.e. outcomes that are closer to the Pareto

frontier. In addition, it might also be allowed to change the structure of the object

by dynamically adding or removing issues. In general, agents have a preference

16


over the issues of the object and their possible outcomes, typically represented

by an utility function that also defines the acceptance region of the agent.

• Decision-Making Apparatus: The decision-making model determines what of-

fers and counter-offers an agent proposes in each stage of the negotiation and

thus specifies the negotiation strategy. The decision models can take into ac-

count a range of factors such as the behaviour of the negotiation partner or other

agents, the current time or negotiation round in the encounter, the state of a

particular resource in the environment or other outside options [85]. They may

further include learning and reasoning capabilities in that they utilize the agent’s

experience for the decision-making while exploring the other agent’s behaviours.

The decision model acts in line with the negotiation protocol and also depends

on the type of the negotiation object.

In relation to these topics some further classification are common in automated negoti-

ation. For example, two categories generally apply for protocols: bilateral negotiation

and auctions, based on the number of participants and the setting of their interactions

such as one-to-one (bargaining), one-to-many (bidding) or many-to-many (double ac-

tion). While in bargaining two parties, typically a buyer and a seller, exchange offers

over the set of issues [122], auctions allow bidding by more than two participants,

which involves request for proposals and interactions among a number of buyer(s) and

seller(s). For example, in open (e.g. English or Dutch) or sealed-bid (e.g. Vickrey or

First-sealed bid) auctions, many buyers compete by bidding for a product sold by an

auctioneer, whereas in double actions several sellers and buyers submit offers and bids

in order to find a match between them. Other scenarios include multilateral bargain-

ing, or one-to-many and many-to-many bilateral negotiations, which, however, usually

employ protocols similar to the purely bilateral case.

In the bilateral context, another typical distinction is whether the negotiating agents

are competitive or cooperative. Opposed to the notion of cooperativeness in game the-

ory (whether an agreement is enforceable or not), here the behaviour of an agent is

the dominant concern. For example, in multi-issue negotiations rational agents should

search for “win-win” situations, i.e. aim at solutions that are Pareto-optimal. A solu-

tion is Pareto-optimal if and only if no agent can increase its utility by deviating from

this solution without sacrificing the other’s utility. This type of negotiation in which the

agents are also cooperative by searching for joint gains is also referred to as integrat-

17


ive negotiation. In contrast, single-issue negotiations are similar to zero-sum games in

game theory as the gain of one party is the loss of the other. In this situation, also called

positional bargaining, the agents are only interested in increasing their own utility and

are therefore competitive. This type of negotiation is also referred to as distributive.

However, in most multi-issue negotiations agents are also interested in increasing their

own utility. Even when Pareto-optimality can be achieved (e.g. by means of a me-

diator), agents need to negotiate about the solutions along the Pareto-frontier. In that

sense, the behaviour of an agent in multi-issue negotiation can be cooperative and

competitive at the same time. This is more discussed in detail in Section 2.3.3.

The focus of research varies depending on the importance of the above topics in the

considered negotiation context. For example, mechanism design focuses on the nego-

tiation protocol and is concerned with the types of operations that can be performed

on the negotiation object while leaving out strategic behaviours of agents through their

decision models. In this thesis, we focus on the decision models of an agent in bilat-

eral negotiation, in which the agents are competitive and have no information about

the decision models and preferences of their opponents. Limited knowledge may be

only derived from the offers exchanged during the current encounter, from previous

interactions, or the knowledge of particular states in the environment. We also con-

sider a more complex one-to-many scenario with concurrent bilateral negotiations (cf.

Chapter 5). The next sections discuss required concepts and notations in this negoti-

ation context such as the underlying negotiation model, the negotiation thread, and the

preference structures of an agent as well as the assumptions drawn in this thesis.

2.2.1 Negotiation Model

This work focuses on bilateral negotiation where two agents a and b propose offers

and counteroffers xtna→b and xtn+1

b→a , respectively, at discrete time points tn, tn+1 ∈ Timeon a set of issues J = {1, 2, . . . , k} such as price or delivery time with k, n ∈ N.

In the case of multiple issues, an offer xtna→b represents a vector of values rather than

a single value. Similar to most of the existing work [44, 81] in bilateral negotiation

the negotiation mechanism is based on Rubinstein’s alternating offers bargaining pro-

tocol [122], where two agents exchange offers alternately until one party accepts or

withdraws from the encounter. However, we adopt the model and notation from the

18


service-oriented negotiation model introduced in [44] in which agents use scoring or

utility functions to evaluate their opponent’s proposals. If an agent a receives an offer

xtnb→a from agent b, agent a assesses the offer and decides whether to accept the offer

xtn+1

a→b , to withdraw, or to propose a new offer. The agents assesses the received offer

using its utility function Ua that assigns a degree of satisfaction to the offer value. If

the utility value Ua(xtnb→a) is higher than the potential counterproposal, i.e. the offer

agent a is going to propose at the next stage, then agent a accepts b’s proposal. Oth-

erwise, counteroffer xtn+1

a→b is proposed by agent a. An agent withdraws, if its deadline

tamax is reached or it has no incentive to continue the negotiation, for example, when

a similar agreement is already met with another provider. Based on this model [44] a

participant’s response to the opponent’s offer is formally written as follows:

Definition 2.1 (Faratin et al. [44]). Given an agent a and its associated utility function

Ua, a’s response at time tn+1 to agent b’s offer xtnb→a proposed at time tn < tn+1 is

defined as:

responsea(tn+1, xtnb→a) =

withdraw(a, b) if tn+1 > tamax

accept(a, b, xtnb→a) if Ua(xtnb→a) ≥ Ua(xtn+1

a→b)

offer(a, b, xtn+1

a→b) otherwise.

(2.1)

The response results in one of the three specified actions withdraw, accept or offer.

The actions accept and withdraw terminate the negotiation process, the former with

and the latter without an agreement.

2.2.2 Negotiation Thread

The sequence of offers exchanged between two agents a and b until time tk ∈ Timeis called the negotiation thread. It reflects the process of the negotiation encounter and

thus represents the time series of the alternating offer proposals of both parties until

the current negotiation stage. The negotiation thread is defined formally as follows:

Definition 2.2 (Faratin et al. [44]). A negotiation thread between two agents a and b

at time tn ∈ Time, denoted as X tna↔b, is any finite sequence of length n of the form

(xt1a→b, xt2b→a, x

t3a→b, . . . ) with t1, t2 ≤ tn, where:

19


1. ti+1 > ti, the sequence is ordered over time,

2. For each issue j, xia→b[j] ∈ Daj , where Da

j = [minaj ,maxai ] for quantitative

issues, xi+1b→a[j] ∈ Db

j with i = 1, 3, 5, . . ., and optionally the last element of the

sequence is one of the particles {accept, withdraw}.

The agents extend the negotiation thread with their alternating offer proposals until

one agent accepts or withdraws from the encounter (for example when a deadline is

reached). The negotiation thread is said to be active, if last(X tna↔b) /∈ {accept, withdraw},

where last() is a function that maps the sequence X tna↔b into the last element of this

sequence. The thread is assumed to be common knowledge, i.e. both parties are aware

of the exchanged offers from the current encounter. In many realistic situations, this

sequence of exchanged offers is the only precise knowledge the agents have about their

opponent.

2.2.3 Agents’ Preferences over Outcomes

In order to make decisions during the encounter about whether to accept a proposal

from the opponent or to make a new proposal, an agent needs to evaluate the received

offer and its own potential counteroffers by assigning scores, or utility values, accord-

ing to its preference structure. In general, a negotiation agent has a negotiation interval

Daj = [minaj ,max

aj ] (with Da

j ∈ Dj) assigned to each issue j under negotiation,

which is defined by the agent’s most and least preferred value. The latter is also called

reservation value and specifies the point until which an agent is willing to make con-

cessions. For instance, a client agent might want to negotiate a low price for a service

with a provider agent such that its reservation value would be RV cprice = maxcprice,

whereas it is the opposite for the provider with RV pprice = minpprice. The negotiation

interval hence determines the range of outcomes acceptable to an agent. For multiple

issues, this results in a k-dimensional space and describes the acceptance region of the

agent over the range of issues. If negotiation regions of both parties intersect then the

agents can possibly reach an agreement. This zone, or region, is also called the zone of

agreement. Figure 2.1 illustrates an example for the agreement zone of a single-issue

negotiation between two agents a and b with different deadlines. The figure also shows

the offer curves of the two agents and the obtained agreement if agent b proposes the

20


first offer. As shown, the agreement corresponds to the first offer after the two offer

curves overlap.

Figure 2.1: Agreement zone for a single-issue negotiation example

The preference over the values within the acceptance region is defined by an agent’s

utility function

Ua :∏j∈J

Daj → [0, 1] (2.2)

where J is the set of all issues under negotiation. The utility function orders all possible

outcomes in the negotiation region by mapping the Cartesian product of the negotiation

intervals of all issues to the unit interval. Given an offer, it assigns the degree of

satisfaction to the offer’s value within its acceptable region. A widely used method is

to assume that agents have an individual scoring or utility function assigned to each

issue under negotiation with

Uaj : [minaj ,max

aj ]→ [0, 1] (2.3)

that is monotonic increasing or decreasing over the negotiation interval. The aggreg-

ated utility over all issues is then simply given by a weighted additive utility function

Ua(x) =∑

1≤j≤p

waj · Uaj (xj) (2.4)

where the weight waj represents the relative importance of issue j to agent a with∑j waj = 1. For simplicity, it is often assumed that the individual utility functions are

linear. The additive utility treats each issue separately and thus assumes that all issues

are independent. Despite the advantage of its simple application and straightforward

21


creation by an agent’s user, it is often argued that in more realistic situations, agents

negotiate over multiple issues that are interdependent and thus require more complex

utility functions. Different types of such utility functions are proposed in the literature

such as quadratic, exponential interdependent or constant elasticity of substitution [81].

Other, non-linear utility models are also explored in [60, 74].

Although computing systems are fast and enable an agent to run thousands of iterations

in a very short amount of time, the encounters in automated negotiation are finite. The

deadline of the agent tamax hence becomes an important decision factor and is there-

fore also part of the preferences and thus, private. In addition, there may be situations

where an agent is interested in obtaining early agreements, or, similarly, in achieving

an agreement with a smaller number of messages exchanged. For that reason, the util-

ity function may include some form of negotiation cost that incorporate the point of

time when an agreement is reached into the evaluation of the outcome. A usual method

is to determine communication costs through the number of exchanged messages. For

example, based on the length of the negotiation thread, Faratin et al [42] proposes the

cost-adjusted utility as the difference between the intrinsic utility (e.g. (2.4) above) and

a cost function Ca = tanh(|Xa↔b| ∗ T ), such that Uacost(x) = Ua(x) − Ca where T

determines the rate of change of tanh(). However, this method can produce negative

utility values of outcomes before an agent’s deadline, which is counter-intuitive be-

cause an agent would stop negotiating if it realizes that the possible outcomes from the

current stage onward obtain negative utility values. Another method, producing non-

negative utility values throughout all stages, is to use simple discount factors, such that

Uacost(x, t) = δt ·Ua(x) where δ is the discount factor with 0� δ < 1, and t represents

negotiation rounds, or similarly, the number of messages sent by the agent. This is

similar to the discount method proposed in [122]. The cost-adjusted utility values the

same outcome higher when it is obtained in an earlier stage in the encounter than in

a future stage. It allows the assessment of outcomes not only in terms of the agent’s

subjective preference, but also in terms of its timely achievement.

According to the type of utility function different decision mechanisms may apply.

In the case of an additive utility function, for example, issues are negotiable sep-

arately (e.g. in a sequential manner) due to the independence between all issues.

Consequently, if agents use concession-making strategies, often simple linear utility

models are employed to enable investigation of the effects and performance of such

22

2.3. Decision-Making in Automated Negotiation

strategies. On the other hand, it has been shown by Ito et al [59] that strategies well-

suited for linear utility models do not cope well with non-linear utility spaces, in which

issues are highly interdependent. However, although more realistic in many situations,

non-additive functions are more difficult to elicit. Since the focus of this thesis is the

strategic concessions behaviour of agents, we use simple utility models with linear and

additive utility functions.

2.3 Decision-Making in Automated Negotiation

A large range of different AI approaches for an agent’s decision-making have been

proposed and investigated for automated bilateral negotiation, ranging from heuristic-

based decision functions to more complex learning and reasoning models. The aim of

those approaches is to overcome the strict assumptions of common knowledge and ra-

tionality of players in game theory by providing decision strategies that are applicable

in more realistic situations, for example, when the knowledge about the negotiation

partner is limited or uncertain.

In general, when an agent a generates an offer proposal xtna→b, it has to decide over

the amount of concession it makes. Agent a proposes a concession if Ua(xtn+1

a→b) <

Ua(xtn−1

a→b ), i.e. the agent’s own utility of its next offer is lower than the utility of its

previous offer. Since the agents do not know the deadlines of other agents, the decision

problem each agent faces is how much to concede at each negotiation stage in order

to obtain high payoffs without failing an agreement. In multi-issue negotiation, an

agent can further decide whether to make a trade-off proposal or to manipulate the

set of issues. A trade-off proposal of an agent has the same utility as its previous

offer (is on the same indifference curve), such that Ua(xtn+1

a→b) = Ua(xtn−1

a→b ). Agents

typically propose trade-offs that are more likely to be accepted by the opponent in

order to increase the chance of an agreement. Such trade-off proposals also support

the search for agreement solutions that are closer to the Pareto-frontier. When an agent

adds or removes issues, it attempts to change the utility function of its counterpart by

manipulating the set of issues [42]. The decision mechanisms may have a different

aim depending on the number of issues and the goal of the agent. For instance, while

heuristic-based concession mechanisms focus on the tractability of decision strategies

used by competitive agents, mediator-based approaches or trade-off mechanisms focus

23


on socially optimal outcomes. The next sections review and discuss some of those

approaches.

2.3.1 Heuristic-based Negotiation Tactics

Heuristic-based approaches provide approximate solutions to an agent’s rational deci-

sion-making. They attempt to overcome the limitation of computational intractability

of many decision and reasoning models by non-exhaustively searching the negotiation

space. Since in many, more realistic, negotiation settings the agents do not have com-

mon knowledge or beliefs about the preferences and decision models of each other,

optimal strategies are hard to find, such that the aim of heuristic-based approaches is to

obtain good rather than optimal outcomes. The realistic assumptions for this approach

allow the usage in a wider range of application domains and enable the creation of

a large range of different agent behaviours and architectures. A prominent approach,

introduced by Faratin et al [44], is to use decision functions, or so called negotiation

tactics, that enable an agent to generate offers and counter-offers at each stage of the

encounter based on various factors, such as time, the state of a resource in the environ-

ment, or the concession behaviour of the opponent. As a result, this model allows to

create different decision strategies which, for example, use the deadline of an agent as

a decision factor, or allow for some level of adaptation to the behaviour of the negoti-

ation partner by imitation. A tactic is usually applied for one issue and thus determines

the concession behaviour of an agent for this issue. The advantages of such decision

functions is their straightforward use and that they require only information that is

available during the current encounter. In addition, they provide the basis for obtaining

more complex concession behaviour by the simple method of mixing tactics using lin-

ear weighted combinations, which are detailed more in Section 2.3.2. In the following,

we denote a negotiation tactic of an agent a for issue j as τaj . A tactic can be interpreted

as a function mapping the mental state MSa of the agent to the issue domain Dj with

τaj : MSa → Dj . The mental state can represent different factors that corresponding

to the state of knowledge an agent has about its environment. The next sections review

examples for the negotiation tactics introduced by Faratin et al [44] such as the time-,

resource- or behaviour-dependent tactics.

24


2.3.1.1 Time-dependent Tactics

Time-dependent tactics generate offers based on the current time in the encounter and

the deadline of an agent. These tactics are completely independent from the opponent’s

behaviour and typically use polynomial (2.6) or exponential (2.7) monotonic decision

functions to propose the next offer.

xtn+1

a→b =

minaj + αaj (t)(maxaj −minaj ) if Ua

j decreasing

maxaj − αaj (t)(maxaj −minaj ) if Uaj increasing

(2.5)

αaj (t) = κaj + (1− κaj )(min(t, tamax)

tamax)

1β (2.6)

αaj (t) = e(1−

min(t, tamax)

tamax)β lnκaj

(2.7)

where αaj generates values between 0 and 1 which are mapped to the interval of the

issue using (2.5), and β and κai determine the concession behaviour of the tactics and

the first offer (at t = 0), respectively. Both decision functions map to values between

0 and 1 where β defines the concession behaviour of the agent, i.e. how fast the agent

reaches its reservation value with regard to its deadline. In general, three types of

concession behaviours are distinguished:

• Boulware for β < 1, the agent makes smaller concession in the beginning of the

encounter while reaching the reservation value quickly towards the deadline

• Linear β = 1 (only for polynomial decision function) and

• Conceder β > 1: large concessions are proposed in the beginning of the en-

counter and decreasing rapidly towards the agent’s negotiation deadline.

The first proposes larger concessions close to the deadline while the latter proposes

large concessions very fast by reaching its reservation value quickly, while κaj ∈ [0, 1]

specifies the initial offer in the agents’ negotiation intervals Daj . Figure 2.2 shows the

shape of the produced curves by two decisions functions for different β settings. As we

can see the two functions expose different curves for similar β settings. For example,

25


the linear setting β = 1 holds only for the polynomial function whereas it already

shows a boulware curve for the exponential one.

5 10 15 20 25 30t

0.2

0.4

0.6

0.8

1.0ΑHtL

5 10 15 20 25 30t

0.2

0.4

0.6

0.8

1.0ΑHtL

Figure 2.2: Polynomial (left) and exponential (right) decision functions (βpoly ∈{9, 4, 2, 1, 0.5, 0.2, 0.05} and βexp ∈ {20, 9, 5, 3, 1.8, 1, 0.5, 0.2})

2.3.1.2 Resource-dependent Tactics

Resource-dependent tactics make offer proposals based on the amount of available

resources. That is, based on the offer generating function:

αaj (t) = κaj + (1− κaj )e−resourcea(t) (2.8)

the function −resourcea(t) measures the quantity of a resource at time t for agent

a whereas resources can be of any kind such as the number |Na(t)| of negotiating

agents, or messages exchanged during the negotiation. Since time can be regarded as

a resource as well time-dependent tactics are a special type of this family of tactics.

2.3.1.3 Behaviour-dependent Tactics

Behaviour-dependent tactics imitate the opponent’s behaviour to some degree. The

counterproposal is calculated proportionally based on the previous offers of the oppon-

ent given by the negotiation thread. Three types of tit-for-tat (tft) tactics are proposed

[44]: relative (2.10), random absolute (2.11) and average tit-for-tat (2.12):

xtn+1

a→b [j] = min(max(ξtn+1

a→b [j],minaj ),maxaj ) (2.9)

26


ξtn+1

a→b [j] =xtn−2δ

b→a [j]

xtn−2δ+2

b→a [j]xtn−1

a→b [j] (2.10)

ξtn+1

a→b [j] = xtn−1

a→b [j] + (xtn−2δ

b→a [j]− xtn−2δ+2

b→a [j]) + (−1)s ·R(M) (2.11)

ξtn+1

a→b [j] =xtn−2δ

b→a [j]

xtnb→a[j]xtn−1

a→b [j] (2.12)

with random factor R(M) ∈ [0,M ] and s specifying whether the value is increased

or decreased by the random amount. The constraints δ ≥ 1 and n > 2δ define the

applicability of each of the tactics where at least two opponent offers and the agent’s

last offer are needed to suggest the next counteroffer. For average tit-for-tat, δ denotes

the window size rather than delay steps as in (2.10) and (2.11). If δ = 1 this tactic is

similar to relative tit-for-tat, whereas larger window sizes with δ > 1 result in larger

concessions. For absolute tit-for-tat in (2.11) s is zero or one if Uaj is decreasing or

increasing, respectively.

2.3.2 Mixing Negotiation Tactics

A common method to generate negotiation strategies with more complex concession

behaviour is to mix individual negotiation tactics by a linear weighted combination

[44]. In this context, an agent’s strategy determines which combination of tactics at

each stage during the encounter is used to generate offers and counter offers if the

opponent’s current offer is unsatisfactory. The weights assigned to the pure tactics

may change during the encounter depending on the mental state MSta of an agent a at

time t. The mental state is a concept, which represents the state of knowledge about the

environment and other agents at a particular stage, including the agents beliefs, goals

or obligations. The set of all possible mental states is denoted as MSa. The change of

an agent’s mental state may affect a negotiation strategy by the choice of the individual

tactics and the change of weights for the their mixture. The definition of a weighted

counterproposal is recalled according to Faratin et al [44] as follows:

Definition 2.3. Given a negotiation thread between agents a and b at time tn, X tna↔b

over domainD = D1× . . .×Dp, with last(X tna↔b) = xtnb→a, and a finite set ofm tactics

Ta = {τi|τi : MSa → D}i∈{1,...,m}, a weighted counter proposal, xtn+1

a→b , is a linear

27


weighted combination of the tactics given by a weight matrix Γtn+1

a→b

Γtn+1

a→b =

γ11 γ12 . . . γ1m

γ21 γ22 . . . γ2m...

... . . . ...

γp1 γp2 . . . γpm

(2.13)

defined as

xtn+1

a→b [j] = (Γtn+1

a→b ∗ Ta(MStn+1a ))[j] (2.14)

where (Ta(MStn+1a ))[i, j] = (τi(MStn+1

a ))[j] with γ ∈ [0, 1] and for all issues j,∑mi=1 γji = 1.

The weighted counterproposal extends the negotiation thread by appending xtn+1

a→b whereby

each row in the matrix represents a weighted linear combination of m tactics for one

issue. In simpler terms, the next counterproposal for a particular issue j can be written

as xtn+1

a→b [j] =∑m

i=1 γji · τji. Different types of negotiation behaviour can be obtained

by weighting a given set of tactics in different ways.

The advantage of this method is that such multi-tactic negotiation strategies can incor-

porate and respond to a large range of factors such as the behaviour of the opponent,

the agent’s deadline, or the state of a resource at the same time. The choice of differ-

ent tactics and the dynamic adjustment of weights enables a limited, but simple level

of adaptation to the agent’s environment and the other agent’s behaviour during the

encounter. An agent can generate complex concession behaviour by mixing simple

tactics in various ways. Moreover, because most pure tactics use only observable in-

formation from the current encounter, mixed tactics are also easier to apply in more

realistic negotiation situations with no common knowledge.

2.3.3 Mechanisms for Pareto-Efficient Negotiations

In multi-issue negotiations, when issues are negotiated simultaneously, it is possible

that a proposed solution by an agent increases the utility of at least one agent without

decreasing the utility of any other agent. Such proposals eventually lead to Pareto-

optimal outcomes, i.e. solutions from which no agent can deviate without sacrificing

the utility of any agent involved. Besides the goal of maximizing the utility of a negoti-

28


ation outcome, rational agents should therefore also seek for such “win-win” solutions

in multi-issue encounters as they increase the chance of an agreement. However, such

Pareto-optimal solutions are hard to find in situations without common knowledge.

A common method is to introduce a trusted, non-biased mediator that, acting as a third

party, supports the agents in finding Pareto-optimal settlements. While many mechan-

isms have been proposed in that realm, most are joint gain seeking methods, such as

the improving directions [38] or constraint proposal method [37]. In the first method,

the agents submit only local information based on a tentative agreement in the form of

their preferred directions (gradient directions) to the mediator, which then generates a

new tentative agreement using the set of jointly improving directions and some fairness

criteria. This process is repeated until a the agreement is Pareto-optimal. However, in

many cases the agents first need to decide upon such an initial agreement as it highly

influences the range on the Pareto frontier in which joint improvements are sought.

In constraint proposal methods, the mediator iteratively adjusts a hyperplane going

through a chosen reference point on which the agents announce their optimal alternat-

ives at each step until a joint tangent is found and alternatives coincide. Although such

methods are able to generate outcomes along the whole Pareto frontier [36] given a set

of reference points, distributive negotiation is still necessary to decide on the finally

chosen settlement. Other works on mediation include query learning for elicitation of

utility structures [79], bidding of participants with non-linear utility spaces [59], or a

simulated annealing-based mediator for intractable contract spaces with binary attrib-

ute values [73]. In many situations, however, the requirement of a mediator limits the

application in distributed and decentralized systems where such a mediator may not

be trusted or may simply not be available [81]. Although these approaches are able

to efficiently find solutions on the Pareto-frontier, the agents still require a distributive

mechanism in order to agree on a starting solution or the final Pareto settlement.

To overcome the requirement of a mediator, pure decentralized methods investigate

how agents should behave in order to achieve Pareto-optimality in the absence of com-

mon knowledge and any third parties. In such a setting, it is hard to even find close

Pareto-optimal outcomes, and agents have only limited means to find them, for ex-

ample, by iteratively making trade-off proposals that are more similar to the opponent’s

previous offers. In other words, when trading off issues, agents choose an offer on the

same indifference curve, or iso-curve, of their current aspiration level, defined as the set

29


of all offers isoa(θ) ∈ {x|Ua(x) = θ} with the same utility value θ, that they believe is

more preferable to the opponent. Faratin et al [43] uses the fuzzy similarity to choose

proposals on the iso-curve that are most similar to the last offer proposed by the oppon-

ent. Although this method also functions on qualitative issues, it assumes linear utility

spaces and requires a method for the estimation of the opponent’s issue weights. Con-

sequently, [29] uses kernel density estimation to estimate these weights, but requires

prior knowledge for good performance. Lai et al [81] proposes a trade-off mechanism

for complex utility spaces in which an agent proposes an offer on its current iso-curve

with the shortest distance to its partner’s previous offer. The authors show that, if both

parties iteratively use this mechanism outcomes are close to the Pareto frontier, and

even closer when iterations are high and agents propose a number of close offers on

their iso-curve from which the opponent chooses the offer that is closest to its current

aspiration level and in turn finds the closest counteroffer. Another approach is to es-

timate the opponent’s utility structure, which has been proposed in [118, 117] where

utility graphs for binary issues are updated during the encounter under the assump-

tion that the maximal structure of possible interdependencies is known beforehand.

In all of the above methods, however, the performance criteria is the achievement of

(or closeness to) Pareto-optimality. Since the agents are also interested in maximizing

their utility of an outcome, they still need a strategy for their concession-making along

their aspiration levels or indifference curves, or to negotiate over the Pareto set (along

the Pareto-frontier). To do this, the above methods usually assume simple bargaining

tactics, e.g. based on the time and the agents deadline (cf. Section 2.3.1.1), or do not

consider this distributive part of the negotiation. Consequently, in order to decide when

to use a concession or trade-off tactic, meta-strategies have been proposed [120, 119]

which only concede to a lower aspiration level when a deadlock occurs, i.e. the utility

of two consecutive offers from the opponent does not improve for an agent. Multi-

issue negotiations with incomplete information hence comprise cooperative elements,

in terms of reaching Pareto-efficient outcomes, and competitive elements, in that the

agents need a concession strategy which obtains high utilities while still reaching an

agreement. In this thesis, we focus on the distributive part of negotiation by assuming

that agents are competitive and do not disclose any information about their decision

models or preferences.

30


2.3.4 Learning and Reasoning in Negotiation

Various research fields have been investigated to enable reasoning and learning of

agents in automated negotiation, such as reinforcement learning [26], Bayesian learn-

ing [148], neural networks [103], regression analysis [19], case-based reasoning [93],

Markov decision processes [96], evolutionary approaches [102] or fuzzy logic based

approaches [76]. The main focus is on creating or finding negotiation strategies that

can anticipate some of the opponents strategic parameters and adapt to the opponents

behaviour in order to improve utilities of outcomes and agreement rates. Basically all

approaches can be classified based on the available information and assumptions made

before the negotiation starts:

• The agent has experience or information from past interactions, e.g. in the form

of historical cases. Typically, a large set of cases is required to enable reasoning.

• The agent has explicit (partial) knowledge about the negotiation environment

such as the domain or the negotiation partner. A general assumption is that the

agent has a probability distribution over different instances of unknown paramet-

ers, for example, the reservation value.

• A predefined set of strategies or models is assumed from which the partner might

choose its strategies.

Even though agents typically utilize information gathered during the current encounter,

in many scenarios they have no or only little and uncertain information about the envir-

onment and negotiation partners beforehand. However, many of the approaches require

one or two of the above assumptions in order to learn or reason properly about part-

ners. In the following we discuss briefly some popular approaches studied in this field.

We also refer to the excellent surveys discussing learning and reasoning approaches in

[15] and [50].

A widely applied approach is reinforcement learning where agents revise their strategies

based on observed failure or success. In Q-learning [26, 103] a reward function (Q-

function) provides feedback on actions taken in order to estimate a ranking of state-

action pairs. The Q-value is updated at every negotiation round for the chosen action

31


a taken in state s as follows:

Q(s, a)← Q(s, a) + α[r(s) + γmaxa′

Q(s′, a′)−Q(s, a)]

where maxa′ Q(s′, a′) is expected utility of the the next state s′, α is the learning rate

representing the impact of the update value and r(s) is the immediate reward for state

s. The factor γ specifies how much the Q-values are discounted at each stage. The

learning agent therefore has to explore the dynamic environment and the partner’s

behaviour by performing actions which are rewarded or punished. An agent applying

this mechanism is able to improve its performance by using its experience to learn what

tactics and actions should be used in what situations [15]. However, the disadvantage

of this approach is that the algorithm has a slow convergence to near-optimal solutions

and therefore needs a large number of trials. Furthermore, the agents need to determine

a balance between new actions and actions which already proved to be good.

In Bayesian learning the agent beliefs about the environment and the participating

agents are explicitly modelled by a probabilistic framework using Bayesian reasoning

for representation and updating [148, 147]. The beliefs are represented in the form of

probability distributions which are generated based on the acquired knowledge before

the negotiation starts. The knowledge can be gained from various sources such as the

domain (e.g. demand for a resource), previous experiences or second-hand knowledge

[15]. The beliefs about the partner can even contain payoff structure, reservation val-

ues or the negotiation style [148]. Based on the conditional probabilities for occurring

events such as received offers or changes in the environment the probability distri-

butions are updated using the Bayesian rule. This can be formally written as follows.

Suppose that an agent has a priori knowledge about the likelihood of a set of hypothesis

Hi with i = 1, 2, . . . , n, then, given the conditional knowledge about the probability

that an event e occurs the agent’s belief is updated with

P (Hi|e) =P (Hi)P (e|Hi)∑nk=1 P (e|Hk)P (Hk)

where P (Hi|e) is the posteriori probability of the hypothesis Hi, P (Hi) is the pri-

ori probability, and P (e|Hi) is the conditional probability that event e occurs given

the hypothesis Hi. The major disadvantage of this approach is that domain specific

knowledge or empirical data from other player’s behaviour are hardly available. Fur-

32


thermore, the agent still needs negotiation strategies which utilize the probabilities of

the belief structure. A further disadvantage of this approach is that it does not provide

the agent with a strategy for the negotiation process, e.g. in the form its concession

behaviour, rather than a probabilistic belief structure about particular parameters of

the opponent (such as the reservation value). It can therefore also not represent models

or beliefs about the negotiation partners explicit behaviour. The Bayesian inference

method has also been applied in many other works on negotiation. For example, in

[6] a Bayesian network models various negotiation contexts in order to aid an agent in

finding the best offer that is likely to be accepted. Hindriks and Tykhonov [56] apply

Bayesian learning to learn a model about issue preferences of the opponent based the

offers exchanged and the assumptions that some of the preference structure and the

rationality of the bidding process is known. Tesauro [133] uses Bayesian inference

and combinatorial search to estimate the expected value of a negotiation based on the

prior beliefs. The combinatorial search is required to search the game tree effectively

for the best actions. In all of those approaches the agent beliefs in the form of prior

probability distributions are assumed to be available.

Neural Networks have also been investigated [103] in order to learn and predict next

proposals using the series of historical offers. The approach provides good results in

medium to long term deadlines as the network needs to adapt to the new negotiation

context on-line. As the network needs a number of pre-training steps and initializations

in order to find optimal performing weights some knowledge about the opponent’s

behaviour is required in advance. For that reason the adaptation of the network takes

much longer in situations where the negotiation partner play strategies very different

to the ones in the training set.

Regression analysis aims at finding the parameters for a given model of a single or a

combination of negotiation decision functions based on the negotiation thread of the

current encounter. Under the assumption of a limited set of tactics or strategies the

agent may also assume a number of models in order to use the best fitting one for

the prediction. Brzostowski and Kowalczyk [19] applied parametric non-linear regres-

sion analysis to predict opponents behaviour for static mixed strategies with time- and

behaviour-dependent tactics. In the case of polynomial decision functions next offers

could be predicted after 5 to 6 rounds whereas for exponential ones at least 12 offers

were necessary. Hou [57] uses the same techniques to predict pure tactics and the

33


author claims that reservation points and deadlines can be predicted using regression

analysis. However, Brzostowski provided a formal proof in [21] demonstrating that

even in the case of pure tactics, such as simple time-dependent polynomial decision

functions it is impossible to derive reservation values or deadlines from the prediction.

Despite the fact that this approach does not need any empirical data before the nego-

tiation starts the disadvantages are the computationally expensive prediction and the

assumption of a number of predefined models which the partner may use to specify its

strategy.

Case-based reasoning (CBR) in negotiation captures and reuses previous negotiation

cases. One of the first to use case-based reasoning in multi-agent negotiation was Sy-

cara [131] where negotiation is performed through proposals and goal relaxations using

solutions provided by most similar cases. Wong et al uses concessions to capture epis-

odic strategies and applies filters to find best matches of buyer and seller concessions

between the cases and the current encounter [139]. Matos and Sierra apply case-based

reasoning in combination with fuzzy rules where the cases are used to adjust paramet-

ers and weights of combined decision functions [93]. This requires that not only the

negotiation thread is captured but also the applied negotiation strategies which in many

cases inhibits the use of cases from other agents, especially if they apply different de-

cision models. In all cases successful negotiations are added to the case base for later

retrievals. Typically a large number of cases is required to obtain good results, e.g. in

marketplace scenarios where different agents may expose completely different beha-

viours. Another approach is possibilistic case-based reasoning which is applied in [17]

to predict successful negotiations for potential partners. Based on the principle that

similar problems require similar solutions, similarity degrees are derived for each case

with regard to the current situation in order to obtain the qualitative expected utility for

each potential partner. This approach provides good results also for a small number of

cases. However, this method has not been applied to update and generate negotiation

strategies during a negotiation encounter.

Since negotiation can be regarded as a sequential decision problem negotiation has

been modelled using a Markov decision processes (MDP) in [96] and [134]. A MDP

is a stochastic process with observable states, which can change on the input of ac-

tions at discrete time steps. The Markovian property assumes that the state transition

of a system depends only on the current state and is independent of the history of

34


states. The state transition is defined via a transition function giving the probability for

the next state under the current state and action. Based on a reward scheme optimal

policies can be calculated which provide optimal actions to be taken in a particular

state. In automated negotiation the main difficulty appears to be how to define the

state space and the transition matrix. Narayanan and Jennings [96] model the agent’s

own behaviour by defining the states in terms of resource availability, deadlines and

reservation values. Depending on the opponent’s offers the algorithm proposes coun-

teroffers by considering changes in those three realms. It is shown that agreements

can be achieved much faster, but only when both agents use this algorithm. Teuteberg

[134] defined the state space by a finite set of predefined tactics reflecting the beha-

viour of the partner. During the encounter probabilities for the transition matrix are

derived from the frequency of applied tactics and their changes. The major drawback

is that a large number of negotiation rounds is needed to obtain sufficient empirical

data for a meaningful state transitions matrix.

Evolutionary Approaches provide a good means to empirically learn best negotiation

strategies [102, 93, 94] as many negotiation models operate in a wide range of envir-

onments with a large number of parameters. By using genetic algorithms the process

works as follows: the strategy parameters are typically encoded in the form of chromo-

somes, which can be interpreted as a representation of the solution. At first, a random

population of candidates is generated. By evaluating a fitness function the fittest can-

didates are selected as parents for generating a new population. While some parents

may be preserved new candidates are created by operations of cross-over and muta-

tion. The process then starts again with the newly created population. [102] applied

genetic algorithms to automated negotiation before the idea of tactics and decision

functions was proposed. The definition of strategies was hence defined in a simpler

way in the form of threshold decision rules. However, the outcomes were evaluated

against a number of different dimensions such as joint outcomes, nearness to the ef-

ficient frontier or similarity to outcomes of human negotiations. Matos et al [94, 93]

employed genetic algorithms to more advanced strategies, such as mixed strategies in

[94] or different architectures for automated negotiation such as case-based reasoning

or fuzzy rules in [93]. Despite the fact that genetic algorithms provide a good means

for evaluation their application in real world scenarios is limited due to the necessity

of searching a large part of the strategy space.

35


Other approaches include modelling the negotiation as a distributed constraint satis-

faction problem (DCSP) or using some form of fuzzy logic. When negotiation is mod-

elled as DSCP the negotiation domain is represented by a set of variables over which

the set of constraints is partitioned between the parties. The aim is to find a solution

by exchanging information in the form of offers until all constraints are satisfied. The

constraints are adjusted based on all offers and therefore guide the search for the solu-

tion [77]. The classical DCSP considers constraints that can be precisely defined and

fully satisfied [143] which may limit its applicability in many real-world negotiation

problems, where preferences and constraints are imprecise and soft. As a result, the

use of fuzzy constraints have been proposed, which is discussed among other fuzzy

logic approaches in the next section.

2.3.5 Fuzzy Logic-based Approaches in Negotiation

Two major approaches of fuzzy logic based reasoning have been employed for de-

cision making in automated negotiation: fuzzy if-then rules and modelling the negoti-

ation process as a fuzzy constraint satisfaction problem. Fuzzy if-then rules provide

a flexible means to model a negotiation strategy using changes in the environment, for

example the current market condition, or inputs from the negotiation partners such as

their offers. In particular, the fuzzy rules typically serve different purposes in the sense

that they may determine the behaviour of the agent directly, for example in form of an

agent’s concessions, or they may adjust parameters of an existing decision model such

as the weights and the parameters of a combination of tactics considering the agent’s

information and its mental state [93]. For example, a fuzzy rule acting on a number of

input variables x1, . . . , xn representing states in the environment or the mental state of

an agent can be formulated as follows [93]:

Rulei : IF x1 is Ai1 AND . . . AND xn is Ain THEN y is Bi (2.15)

where y is a parameter in the negotiation strategy and Ai1, . . . , Ain, Bi are the lin-

guistic numbers in the universe of discourse corresponding to particular settings of the

decision strategy parameters of the agent. Matos et al [93] also showed that fuzzy rules

may be employed to assist reasoning applied by other approaches such as case-based

reasoning in order to adapt the proposed strategy also to a change in the environment.

36


In [127] the fuzzy rules are used to flexibly react to changing market conditions such

as different trading options, competitions and deadlines, and accordingly adjust their

concession making strategies. In contrast to adjusting the strategy parameters that ap-

proach applies a fuzzy decision controller that uses the fuzzy rules to relax an agent’s

trading condition, i.e. its aspiration level, and hence applies different sets of rules un-

der certain conditions, for example when an agent is under time pressure. Even though

fuzzy rules provide a flexible and simple method for modelling negotiation behaviour,

in many cases the modelling process requires human input or a sufficient amount of

data for the initial rule generation, and is thus difficult to do automatically. This is even

more so in dynamic environments where the rules have to adapt quickly and automat-

ically to changes in the environment and different behaviours of other agents.

Negotiation has been modelled as a fuzzy constraint satisfaction problem in [91, 75,

76] where constraints, preferences and objectives are represented uniformly as fuzzy

constraints which are distributed among the parties. The fuzzy constraints are represen-

ted by membership functions which define the degree of satisfaction of the constraints

for a particular proposed solution. For example, in [76] a fuzzy relation Cj(xj) corres-

ponds to the set of constraints Cj = {Cjk} for the jth party with k = 1, . . . ,mj such

that

Cj(xj) = ∧k=1,...,mjCjk(x

j) (2.16)

where ∧ is a conjunctive combination operator. By exchanging their preferred solu-

tions according to the level of constraint satisfaction the agents iteratively relax. The

fuzzy constraints can be therefore considered as fuzzy relations over all issues between

the agents which are iteratively relaxed during the exchange of the preferred solutions

by the parties in the form of offers [15]. The agents therefore search for an agreement

which satisfies the constraints of all agents while it is guided by individual negotiation

strategies of each party. In that sense the fuzzy constraint based reasoning assists this

search process by ordering and pruning the search space of each party and maxim-

izes the satisfaction level of the final agreement for all agents [76]. Another fuzzy

constraint-based that also includes the fuzzy similarity to select the alternative that

may be accepted by the opponents is proposed in [82, 87]. This enables to select of-

fers based on various proposed concession strategies, which make it difficult to apply

different strategy models. Luo et al [92, 90] use prioritised fuzzy constraints in or-

der to express priority over issues and constraints such that fair deals can be found.

37


However, in their model the offers exchanged contain information about the particular

constraints which limits the application with traditional negotiation protocols where

only information about the negotiation issues is exchanged.

2.4 Multistage Fuzzy Decision-Making

The process of negotiation can be considered a multistage decision process in which

each agent needs to make a decision at each stage of the encounter in order to find

a solution that satisfies both agents. In such encounters, an agent also needs to be

able to incorporate limited knowledge about the opponent’s concession behaviour, for

example from a few reference cases, into its decision process while following a par-

ticular negotiation strategy at the same time. Multistage fuzzy control seems to be a

good candidate to model such decision problems. Therefore, we recall in the following

sections the basic concepts of fuzzy set theory and fuzzy decision making with goals

and constraints, and give a brief introduction of multistage fuzzy decision models for

deterministic and stochastic systems. For a more thorough introduction into the sub-

ject matter we refer to large amount of literature available, especially to [68], [144]

and [7]. An excellent overview and introduction into fuzzy set theory and multistage

fuzzy decision models can be found in [68].

2.4.1 Fuzzy Decision-Making

A fuzzy set A in the universe of discourse X is defined as a set of pairs

A = {(µA(x), x)} (2.17)

where µA : X → [0, 1] is the membership function of A and µA(x) determines the

grade of membership of an element x ∈ X in the fuzzy set A. While in conventional

set theory, the elements either belong to the set or not, elements in a fuzzy set can

belong to the set to some degree specified by the membership function. The universe

of discourse X is the set containing all possible elements. A fuzzy set therefore is a set

of pairs containing particular elements of the universe of discourse and their degrees

of membership. Similar to conventional sets, fuzzy set theory has the basic operations

38

2.4. Multistage Fuzzy Decision-Making

of complement, intersection and union. The complement of a fuzzy set A is written as

¬A and is defined as follows

µ¬A = 1− µA(x) (2.18)

for each x ∈ X .

The intersection A ∩ B of two fuzzy sets A and B is defined by a so called t-norm

which is defined as t : [0, 1] × [0, 1] → [0, 1]. The most widely used t-norm is the

minimum

µA∩B(x) = µA(x) ∧ µB(x) (2.19)

where the operator ∧ represents the minimum operation, i.e. a ∧ b = min(a, b). A

large number of other t-norms have been proposed in the literature, for example, the

algebraic product µA∩B(x) = µA(x) · µB(x) or the Lukasiewicz t-norm µA∩B(x) =

max(0, µA(x) + µB(x)− 1).

The union of two fuzzy sets A and B is written as A ∪ B and is defined in terms of

the s-norm (or t-conorm) with s : [0, 1] × [0, 1] → [0, 1]. The most widely used is the

maximum

µA∪B(x) = µA(x) ∨ µB(x) (2.20)

where the ∨ operator is the maximum operation, i.e. a∨b = max(a, b). Other s-norms

are, for example, the probabilistic product µA∪B(x) = µA(x) +µB(x)−µA(x) ·µB(x)

or the Lukasiewicz s-norm µA∪B(x) = min(µA(x) + µB(x), 0). Another important

concept in fuzzy set theory is the fuzzy relation between conventional sets. A fuzzy

relation R between two non-fuzzy sets X and Y is defined in the Cartesian product

space X × Y : R = {(µR(x, y), (x, y))} for each (x, y) ∈ X × Y and µR(x, y) :

X×Y → [0, 1]. A binary fuzzy relation is a fuzzy set specifying the fuzzy membership

of elements in the relation between two non-fuzzy sets. Similarly, any n-ary fuzzy

relation is defined in X1 × . . .×Xn. A Fuzzy composition R ◦ S combines two fuzzy

relations R in X × Y and S in Y × Z. For example, the max-min and max-product

compositions are written as

µR◦max−minS(x, z) = maxy∈Y

[µR(x, y) ∧ µS(y, z)]

µR◦max−prodS(x, z) = maxy∈Y

[µR(x, y) · µS(y, z)](2.21)

39


for each x ∈ X, z ∈ Z. The following example shall demonstrate the composition of

two binary fuzzy relations.

Example 2.1 Fuzzy compositionAssume that X = {1, 2}, Y = {1, 2, 3} and Z = {1, 2, 3, 4} with the fuzzy relations

R and S below the max-min composition R ◦ S is given by:

R ◦ S =y = 1 2 3

x = 1 0.2 0.7 0.92 1 0.6 0.3

◦

z = 1 2 3 4y = 1 0.9 0.5 0.3 0.4

2 0.3 1 0.7 0.13 0.6 0.8 0.5 1

=z = 1 2 3 4

x = 1 0.6 0.8 0.7 0.92 0.9 0.6 0.6 0.4

Throughout the thesis, we use the basic intersection and union aggregations of fuzzy

sets with simple operators such as∧ and∨, respectively. However, other t- and s-norms

can be applied depending on the context and the decision problem.

A fuzzy decision problem can now be defined using a fuzzy goal and a fuzzy constraint.

Assume that the set X contains the elements of the decision problem, such as actions,

options,etc. a fuzzy goal is defined as a fuzzy set G with the membership function

µG : X → [0, 1] that specifies the grade of membership of an option x ∈ X in the

fuzzy goal. Similarly, a fuzzy constraint is defined as a fuzzy set C in the set of options

X , such that µC(x) ∈ [0, 1] determines the membership grade of a particular option

x in the fuzzy constraint. Since the decision problem is typically attain C and satisfy

G, the decision can be found by aggregating the two fuzzy sets. The fuzzy decision

D is then also a fuzzy set in the set of options X that is the result of the aggregation

? : [0, 1]× [0, 1]→ [0, 1] of G and C such that

µD(x) = µC(x) ? µG(x) (2.22)

Because of the ’and’ connective in the decision problem ’attain G and satisfy C’ a

t-norm aggregation should be used here, such as the minimum. The min-type fuzzy

decision is then

µD(x) = µC(x) ∧ µG(x). (2.23)

40


Since the fuzzy decision is a fuzzy solution to the decision problem, we need the op-

timal non-fuzzy decision that is the solution that maximizes the degree of membership

in the fuzzy decision. The maximizing decision x∗ ∈ X is then defined as

µD(x∗) = maxx∈X

µD(x). (2.24)

The maximizing decision is basically a defuzzification of the fuzzy decision such that

the above maximum represents only a simple solution. Depending on the decision

problem other more suitable methods may be used, such as the center-of-area method

x∗ =

∑ni=1 xiµD(xi)∑ni=1 µD(xi)

. (2.25)

However, for the multistage fuzzy decision problems in this thesis the maximizing

decision is sufficient. The following example shall further demonstrate the fuzzy de-

cision:

Example 2.2 Fuzzy DecisionSuppose that X = R, the set of real numbers, the fuzzy goal is “x should much large

than 5” and the fuzzy constraint is “x should be about 6”. Both, the fuzzy constraint and

fuzzy goal are shown in Figure 2.3. The coloured area in the figure represents the min-

type fuzzy decision. The set of possible options is hence in the interval [5, 10] because

the membership degree of the fuzzy decision is zero outside of this interval. The

maximizing decision is then x∗ = 7.5. The value of the fuzzy decision µD(x) ∈ [0, 1]

can also be interpreted as the satisfaction level of how much the fuzzy goal and fuzzy

constraint are satisfied. Intuitively, the maximizing decision has the highest satisfaction

level. �

Similar to the decision problem with two fuzzy sets, a fuzzy decision problem can have

multiple fuzzy goals and constraints written as

D = C1 ? Cm ? . . . ? G1 ? Gn (2.26)

where D is the fuzzy decision in a fuzzy environment specified by n fuzzy goals

G1, . . . , Gn and m fuzzy constraints, C1, . . . , Cn. Both, fuzzy goals and constraints,

are fuzzy sets in the set of options X . The maximizing decision can then be used again

to find the optimal decision. In the next section we recall multistage fuzzy decision-

41


Figure 2.3: Fuzzy decision

making for different types of dynamic systems.

2.4.2 Multistage Fuzzy Decision Making in Deterministicand Stochastic Systems

We consider now decision problems that are more dynamic in that a sequence of de-

cisions has to be found that moves a system from the current state into a desired state.

The discrete time moments at which decisions are made are called stages, while the

input-output relationship of the system is also referred to as system under control. In

this context, the decisions are called controls or actions, where we use the latter inter-

changeably with decision in the following.

Assume that the state space of the system is X = {σ1, . . . , σn} and the action space

is U = {α1, . . . , αm}. The decision process starts with an initial state of the system

x0 ∈ X in which the action u0 ∈ U is subjected to a fuzzy constraint µC0(u0) and

applied to the system. The system then moves to the next state x1 ∈ X at stage 1 that

may be subjected to a fuzzy goal µG1(x1). The process repeats with state x1, where the

action u1 is subjected to the fuzzy constraint C1so that state x2 ∈ X is attained and so

on. Suppose that we have a deterministic system under control whose state transitions

are governed by the state transition function

xt+1 = f(xt, ut) (2.27)

42


where xt+1, xt ∈ X and ut ∈ U with t = 0, 1, . . . being the discrete time points or

stages. In addition, we assume that the decision process is finite and that a fuzzy goal

is only imposed at the last stage N . We only consider multistage fuzzy decision mod-

els with finite termination times here because in the automated negotiation context the

agents have deadlines. This means that the termination time is fixed and specified in

advance. Since only one fuzzy goal µGN (xN) at the final stage is used, the focus of

the decision process is to get the best possible state at the end of the process. The de-

cision process under these assumptions is illustrated in Figure 2.4. The fuzzy decision

Figure 2.4: Multistage fuzzy decision process

determines the performance of the multistage decision process being the aggregate of

the consecutive constraints at the stages and the fuzzy goal, such that

D(x0) = C0 ? . . . ? CN−1 ? GN , (2.28)

where ? is the aggregation operator. The consecutively attained states are given by the

state transition function 2.27 applied at each stage, i.e.

x1 = f(x0, u0)

x2 = f(x1, u1) = f(f(x0, u0)), u1)

. . .

xN = f(xN−1, uN−1) = f(f(. . . (f(x0, u0), . . . , uN−2), uN−1).

(2.29)

43


The fuzzy decision using the minimum is then given by

µD(u0, . . . , uN−1|x0) = µC0(u0) ∧ . . . ∧ µCN−1(uN−1) ∧ µGN (xN). (2.30)

where xN is uniquely determined by the initial state and the action trajectory (x0, u0, . . . uN−1)

via the state transition function. The problem is now to find the optimal sequence of

actions u∗0, . . . , u∗N−1 ∈ U that maximizes the fuzzy decision:

µD(u∗0, . . . , u∗N−1|x0) = max

u0,...,uN−1

µD(u0, . . . , uN−1|x0). (2.31)

A number of algorithms have been proposed in the literature that solve this problem,

such as dynamic programming, branch-and-bound, genetic algorithms and neural net-

works [69]. Among those dynamic programming the most widely used solution that

was proposed in the seminal paper of Bellman and Zadeh [7]. For that reason, we

briefly outline the the dynamic programming solution in the following and refer to [69]

for a more thorough discussion of other solution approaches. Using the state transition

function the above maximizing decision can also be written as

µD(u∗0, . . . , u∗N−1|x0) = max

u0,...,uN−1

[µC0(u0)∧. . .∧µCN−1(uN−1)∧µGN (f(xN−1, uN−1))]

(2.32)

The two right hand terms do only depend on the action uN−1 at stage N −1 and not on

any previous actions. This makes the application of dynamic programming possible

as the maximization can be divided into maximizing the action sequence u0, . . . , uN−2and maximizing the action uN−1. The same line of reasoning can be applied to the

next term µCN−2(uN−2) that depends only on action uN−2, such that the maximizing

decision can be written as

µD(u∗0, . . . , u∗N−1|x0) = max

u0,...,uN−3

[µC0(u0) ∧ . . . ∧ µCN−3(uN−3) ∧ . . .

∧maxuN−2

[µCN−2(uN−2)∧

∧maxuN−1

[µCN−1(uN−1) ∧ µGN (f(xN−1, uN−1))]]

(2.33)

This backward iteration can be repeated until u0 and therefore represents the dynamic

programming solution of this problem. Based on this iteration one can derive the

44


recurrence equations for the dynamic programming:

µGN−i(xN−i) = maxuN−i

[µCN−i(uN−i) ∧ µGN−i+1(xN−i+1)] (2.34)

xN−i+1 = f(xN−i, uN−i) (2.35)

where i = 0, 1, . . . , N and µGN−i(xN−i) can be regarded as a fuzzy goal at stage

t = N − i induced by the fuzzy goal at the next stage t = N − i + 1. The optimal

sequence of actions is therefore given by the successive maximization of actions uN−iwith i = 1, . . . , N . Since each optimal action u∗N−i depends on the state xN−i at the

same stage, the solution is expressed in terms of an optimal policy function a∗N−i :

X → U that assigns to each state the optimal action at stages i = 1, . . . N , such that

u∗N−i = a∗N−i(xN−i). (2.36)

An optimal solution only exists if there is at least one action sequence for which

µD(u0, . . . , uN−1|x0) > 0. The set A∗ = {a∗0, . . . , a∗N−1} then forms the optimal

action strategy.

Let us assume now that instead of a deterministic system the state transitions are gov-

erned by a conditional probability function

p(xt+1|xt, ut), (2.37)

where xt, xt+1 ∈ X and ut ∈ U with t = 0, 1, . . . , N − 1. This corresponds to a

Markov decision process where the fuzzy goals and constraints, imposed at the re-

spective stages of the process, represent the fuzzy environment in which a decision is

to be found given the time-invariant transition function and the fixed termination time.

Similar to the deterministic system we consider the case where the final outcome at

the last stage N is of most importance so that only one fuzzy goal GN is imposed.

The decision problem according to Bellman and Zadeh [7] is then to find an optimal

sequence of controls u∗0, . . . , uN−1 that maximizes the probability of attainment of the

fuzzy goal considering the fuzzy constraints, written as

µD(u∗0, . . . , u∗N−1|x0) = max

u0,...,uN−1

[µC0(u0)∧. . .∧µCN−1(uN−1)∧EµGN (xN)]. (2.38)

45


For the probability of attaining the fuzzy goal EµGN (xN) Zadeh’s definition of the

non-fuzzy probability of a fuzzy event is used. However, other definitions may be

applied instead [145, 140]. The fuzzy goal µGN (xN) is therefore regarded as a fuzzy

event in X such that the conditional probability given the action uN−1 and state xN−1of the previous stage is given by

EµGN (xN) = EµGN (xN |xN−1, uN−1) =∑xN∈X

p(xN |xN−1, uN−1)µGN (xN). (2.39)

Because this notion is similar to the notion of expected utility, EµGN (xN) may also

be called the expected fuzzy goal. Given this goal at stage N and the constraint at

stage N − 1, the fuzzy decision at stage N − 1 selects the optimal actions for each

state xN−1. Consequently, the fuzzy decision for each state at stage N − 1 may be

regarded as a fuzzy goal µGN−1 at stage N − 1 induced by the fuzzy goal µGN that

is used for the next iteration in order to find the optimal actions at stage N − 2. The

backward iteration, which is similar to the one for the deterministic system shown

above, is repeated until we find all optimal actions uN−1, uN−2, . . . , u0. Using (2.37)

to (2.39), the dynamic programming solution for this multistage decision problem is

given according to [68, 7] by the following recurrence equations:


[µCN−i(uN−i) ∧ EµGN−i+1(xN−i+1)] (2.40)

EµGN−i+1(xN−i+1) =∑

xN−i+1∈X

p(xN−i+1|xN−i, uN−i) · µGN−i+1(xN−i+1), (2.41)

for i = 1, . . . , N . The solution is again expressed in terms of a policy function u∗t =

a∗t (xt) with t = 0, 1, ..., N − 1 and A∗ = {a∗0, . . . , a∗N−1} being the optimal action

strategy.

2.5 Application Areas for Automated Negotiation

The rapid development of computing systems and networks over decades has led to the

emergence of more complex distributed and decentralized systems, such as the Grid

[48], service-oriented computing [40] or recently cloud computing [22], which increas-

ingly demand more intelligent and reliable interaction mechanisms between dispersed

46

2.5. Application Areas for Automated Negotiation

software components. Automated negotiation is considered as such a key mechanism

being able to resolve conflicts between self-interested software agents. Artificial Intel-

ligence research has thus focused on the study and development of negotiation mech-

anisms and decision models and their practical use in more realistic situations due to

their potential in many real world applications. Automated negotiation is primarily

useful for systems characterized by some of the following properties:

• Distribution: The system is composed of a number of software entities and re-

sources which are loosely coupled with dispersed ownership and control.

• Decentralization: There is no central instance managing the system or parts of

it. The individual software entities do not have global knowledge, except about

the underlying protocols that govern the interactions.

• Openness: Agents can enter and leave the system at any time. Entities have to

adhere to the offered protocol(s) of the system in order to be able to interact with

other entities.

• Dynamic: The behaviour of the individual entities of the system expose different

and changing behaviours as they interact and react to signals from the environ-

ment and other entities.

• Autonomic behaviour: Software entities act autonomously and are able to make

decisions (e.g. on behalf of their users). By doing so, they perceive their envir-

onment and utilize the available information to generate knowledge supporting

their decision-making.

The examples of distributed systems mentioned above share some of these properties.

In the following, we give a brief overview of the proposed application areas for auto-

mated negotiation, namely electronic commerce [137], supply chain management, task

distribution and scheduling, [15], resource allocation, and service composition and se-

lection in service-oriented environments [28].

An prevalent application domain is that of e-commerce and e-markets in which ne-

gotiation agents support sellers and buyers in finding trade agreements on economic

goods or services [26, 107, 54]. Different types of negotiation frameworks have been

47


proposed ranging from agent-mediated approaches [55] for flexible brokering and co-

ordination of markets to bilateral negotiation in which the agents act on behalf of their

users. In this context, negotiation is an efficient means to select consumers and pro-

viders, and also to avoid deadlocks as compared to fixed price systems. While a num-

ber of specific agent frameworks and architectures have been investigated for their

applicability in this domain in many-to-many and one-to-many negotiation settings

[102, 107], such techniques also serve as a basis for more advanced inter-organisational

relationships as, for example, the formation of virtual organisations [101] or the man-

agement of supply chains. In the bilateral negotiation context, learning and reas-

oning models have been in particular proposed in the area of e-commerce [96, 25,

103, 102] to make agents more adaptable to changing market conditions and differ-

ent buyer/seller behaviours, and to increase their performance based on the gathered

knowledge through their market interactions.

Similar to the e-commerce domain, multi-agent approaches with negotiation interac-

tions are proposed for the management of supply chains among different entities and

organisations [88]. While the aim is to provide infrastructures that dynamically react to

changes in the supply chain and synchronize supply and demands [30], proposed mech-

anisms support in particular the dynamic selection of suppliers and contracts [66, 47],

and the planning and scheduling of tasks [88]. For example, Jiao et al [66] present an

agent-based multi-contract negotiation system for the coordination of a global manu-

facturing supply chain, while also providing a case study in the area of mobile phone

manufacturing. The mediated negotiation of complex supply chain contracts with large

numbers of issues is considered in [47], whereas in the approach of Lopes et al [88],

the agents, representing supply chain activities, negotiate in a bilateral setting with

each other in order to execute their tasks. Similarly, Jennings et al [65] use agents to

support the negotiation between business units in order to manage tasks and resources

along a business process of providing a quote to a customer for installing a delivery

network for telecommunications services. The authors argue that the system is more

robust and flexible towards run-time changes, context-dependent exception handling,

and provisioning of resources as compared to existing workflow systems.

Another application example is resource allocation with autonomous negotiation agents

in a Grid computing or service-based environment. In [86], a strategic negotiation

mechanism is proposed to find agreements between resource providers and consumers

48

2.5. Application Areas for Automated Negotiation

in order to autonomously allocate and manage heterogeneous and decentralized Grid

resources. Also in a Grid environment, Streitberger et al. [129] compare centralized

market mechanisms such as auctions with the decentralized bargaining mechanism for

the allocation of resources and show that the decentralized methods perform better

when the Grid network reaches a certain threshold at the expense of a larger mes-

sage count. Furthermore, it is shown by An et. al [2] that automated negotiation for

the dynamic resource allocation in service-based systems with multiple buyers and

sellers perform better than combinatorial auction mechanism or fixed price models,

when agents are allowed to decommit from an agreement (at the cost of a penalty) and

individual negotiations are carried out concurrently.

In service-oriented systems a critical issue for service consumers and service providers

is to effectively achieve agreements on non-functional aspects, also called the quality

of service, of the service provision [67]. Moreover, when services are dynamically

composed together to form complex service workflows, such service level agreements

(SLAs) need to be attained in a flexible manner during run-time. Because of this, auto-

matic negotiation has been recently proposed and applied for the negotiation of quality

of service parameters, such as price, response time or throughput, in order to estab-

lish SLAs [100]. For example, a novel framework for agent-based SLA negotiation in

Web service compositions is proposed in [28, 67] where the individual agents negoti-

ate with service providers in order to select the best candidate provider for a service in

the composition while considering the end-to-end constraints on the quality of service

parameters. In a similar setting, Brzostowski et al [16] discuss the decision-making for

agents on the level of the coordination of agents, selection of partners and negotiation

strategies. Another work, focusing rather on the negotiation architecture for service

markets than for service compositions is proposed in [100]. In that work, the mar-

ket place supports the negotiation process with mediation based on search algorithms.

However, the automatic establishment of SLAs for complex service compositions is

vital for enabling the dynamic composition, selection and enactment of services given

the end-to-end requirements for the overall composite service. We focus in chapter

5 on such a scenario with the end-to-end QoS negotiation for SLA establishment in

composite services. This involves compound multi-party negotiations in which the

composite service provider concurrently negotiates with multiple candidates for each

atomic service in the composition, selecting the one that best satisfies the atomic ser-

vice QoS preferences while ensuring that the end-to-end QoS requirements are also

49


fulfilled. Using this scenario, we also demonstrate the applicability of the negotiation

strategies presented in this work.

2.6 Simulation Environment and Experimental Evalu-ation

This section describes the simulation environment and the general setup for the exper-

iments carried out in this work.

2.6.1 Simulation Environment

The negotiation system was implemented in Mathematica 7.01. It allows bilateral agent

negotiation in settings of one-to-one or one-to-many and testing of all strategies and

tactics presented in this thesis. To facilitate the investigation of the behaviour of ne-

gotiation strategies and tactics graphical user interfaces were created for single- and

multi-issue negotiations that allow to change strategy parameters and to observe the

offer and utility curves of encounters in real time (see Figure 2.5 for the single-issue

interface). In addition, the negotiation threads of individual interactions can be cap-

tured and stored such that the agents can use them as reference cases, for example, for

the creation of the fuzzy model of the opponent as presented in Chapter 4.

2.6.2 General Settings for Experiments

The aim of the negotiation experiments is to evaluate the proposed negotiation mech-

anisms in this thesis and to test them in different environment and strategic settings. In

order to provide more realistic settings we first need to simulate agents that can expose

different negotiation behaviours using various strategies. A common approach is to use

heuristic-based tactics and mix them to simulate different agent behaviours. Similar to

[42] and [21] we choose the heuristic-based time-dependent and behaviour-dependent

tactics presented in Section 2.3.1 to create mixed strategies. The advantage of such

strategies is that because of their imitative component they are able to partially adapt

1Wolfram Mathematica: www.wolfram.com

50

2.6. Simulation Environment and Experimental Evaluation

Show Lines Goal

Agree

Strategy p Trad Constr Thread Conc Multistage

Strategy c Trad

Process

constraints

Client

Time 25

Max 25

Min 10

Time dep. Poly Exp

Β 1.2

Beh. dep. Relative Absolute Average

Start 1

Steps 1

Weight 1

RV

Add 0

Time 10

Goal

Min 15

5 10 15 20 25 30t

15

20

25

30x

Provider

Time 30

Max 30

Min 15

Time dep. Poly Exp

Β 0.5

Beh. dep. Relative Absolute Average

Weight 1

Case 0

Case 2 0

Match Case Add Case Set Case Delete Case

maxcases 0

Figure 2.5: Interface for single-issue negotiations between two agents

to the opponent’s behaviour. In order to be able to distinguish between different types

of mixed strategies or strategy groups, we use different types of concession behaviour

of the time-dependent tactics for different strategy settings, such as conceder, linear,

and boulware, and mix the them with different sets of weights, such as such small,

medium or large, with imitative tactics. By this method, a mixed strategy can be clas-

sified not only based on the concession behaviour of the time-dependent tactics, but

also whether it is more reactive towards the opponent using the imitative tactics based

on the different weights sets. Table 2.1 shows the chosen tactics and the sets of weights.

Since the polynomial and exponential decision functions expose different offer curves

for similar settings, the parameter differs for the particular concession sets. A set of

mixed strategies can then be created by the Cartesian product of the individual sets of

51


Polynomial decision function: Conceder: PC = {β|β ∈ {3, 5, 7}}(Time-dependent) Linear: PL = {β|β ∈ {0.8, 1, 1.2}}

Boulware: PB = {β|β ∈ {0.1, 0.3, 0.5}}Exponential decision function: Conceder: EC = {β|β ∈ {5, 7, 9}}(Time-dependent) Linear: EL = {β|β ∈ {2, 3, 4}}

Boulware: EB = {β|β ∈ {0.3, 0.5, 0.7}}Behaviour-dependent Absolute TFT: a = δ = 1, R(M) = 0

Absolute TFT: r = δ = 1Weights Small: S = {γ|γ ∈ {0.1, 0.2, 0.3}}

Medium: M = {γ|γ ∈ {0.4, 0.5, 0.6}}Large: L = {γ|γ ∈ {0.7, 0.8, 0.9}}

Table 2.1: Parameters for strategy groups

the time-dependent behaviour-dependent and weights strategy settings. For example,

a set containing the polynomial time-dependent and imitative tactics is given by

ST = {PC, PL, PB} × {a, r} × {S,M,L} (2.42)

such that the set of possible strategy groups is

ST = {(PC × a× S), (PC × a×M), (PC × a× L), (PC × r × S),

(PC × r ×M), (PC × r × L), (PL× a× S), (PL× a×M),

(PL× a× L), (PL× r × S), (PL× r ×M), (PL× r × L)

(PB × a× S), (PB × a×M), (PB × a× L), (PB × r × S),

(PB × r ×M), (PB × r × L)}

(2.43)

The initial letters indicate the respective group of mixed strategies. For example,

‘PCaS’ denotes the strategy group containing conceder time-dependent and absolute

tit-for-tat tactics mixed by small weights. Each strategy group represents a particular

type of behaviour, and when agents play strategies from a particular strategy group,

their behaviour covers a similar range of behaviour in the space of all strategies. Using

the above method for creating mixed strategies therefore allows us to simulate a large

range of different concession behaviours.

Besides the negotiation strategies, the negotiation environment also strongly affects

the outcome of a negotiation. The negotiation environment is specified by the ne-

gotiation intervals and the deadlines of both agents. For example, if the intervals

52

2.7. Summary

overlap only to a small degree and the agents have different deadlines the zone of

agreement is also small such that a smaller number of agreements may be achieved.

We generate the negotiation intervals as follows. Given the interval for issue j of

a client agent maxcj = θj + mincj , the interval of the provider agent is given by

minpj = mincj + Φj(maxcj − mincj) and maxpj = minpj + (maxcj − mincj), where

Φj is the degree of overlap between the two negotiation intervals and θj is the size of

the intervals. In order to test the negotiation strategies within different environments

we choose interval settings with small and large overlaps, so that Φj ∈ {0.33, 0.66}.As described in Section 2.2.3 negotiations in automated software systems are supposed

to be finite that makes the deadline of an agent an important decision factor. Although

the deadline may be imposed by the system in that it has a pre-specified time limit for

an interaction, the agents may have their own limits. For that reason, we consider both

variants in this thesis, where agents have equal or different deadlines. The simulation

of real time in a software system is difficult due to the large number of possible con-

ditions of the communication channel between the agents. Without loss of generality,

the time measure used for all negotiation strategies can also be based on the numbers

of messages exchanged, which is a common approach in many of the research work

in automated negotiation [44, 21]. In this thesis, we use negotiation rounds for all ex-

periments in which one negotiation round consists of one offer proposal of each agent

with the first offer being from the agent that made the first proposal at the beginning of

the encounter.

2.7 Summary

This chapter has introduced important notions in bilateral automated negotiation and

has discussed related work for decision-making in negotiation from a game-theoretic

and AI perspective. It has shown that one of the key problems in bilateral encounters

is to decide when and how to make concessions when the decision models and pref-

erences of all parties are private and agents have only limited information available

for their decision-making derived from the current encounter or a few interactions.

It has also shown that even in the multi-issue case in which joint gains are possible,

and the negotiation is not entirely distributive, the agents need a decision-apparatus

in order to make concessions. For that reason, the heuristic-based model in which

53


tractable negotiation tactics are used to make concession decisions and mixed tactics

are used in order to be able to take into account a larger range of factors, such as the

opponent’s behaviour and the agent’s deadline at the same time, has been introduced.

Moreover, we have presented relevant existing work on the decision apparatus an agent

can apply to model its negotiation strategy, such as reasoning and learning models, in-

cluding Bayesian inference, evolutionary approaches and reinforcement learning. We

have also presented fuzzy logic-based approaches for negotiation and introduced the

model of multistage fuzzy decision-making, an extension of which is used in Chapter

4 to model negotiation strategies. Finally, the possible application areas studied and

presented in the literature were outlined. With the aim to investigate and propose

negotiation decision strategies when only limited knowledge is available, the next two

chapters investigate the heuristic-based approach of mixing tactics to create more com-

plex concession behaviour and propose a novel decision model for an agent’s negoti-

ation strategy based on multistage fuzzy decision-making. Then, Chapter 5 demon-

strates how such negotiation strategies can be coordinated in a more complex and real-

istic negotiation scenario with concurrent negotiations.

54

Chapter 3

Monotonic Mixing Mechanisms forMulti-Tactic Negotiation Strategies

This chapter presents and investigates heuristic approaches for creating multi-tactic

negotiation strategies for an agent by mixing a set of pure tactics, or decision functions,

at each stage of the negotiation using linear weighted combinations. This method, first

proposed in [44], is able to dynamically generate complex concession behaviour when

combining different types of decision functions while the individual functions typically

use only the limited information available in the current encounter, such as the offers

exchanged, number of available agents or the agent’s deadline. For that reason, the

heuristic multi-tactic negotiation strategies represent practical models for application

in situations when agents have no knowledge about the other parties decision models

and preferences, including their reservation values, deadlines and utility functions.

When an agent changes the weights of the linear combination during the encounter it

can create more dynamic negotiation strategies, which may also result in a sequence of

offers that is non-monotonic. However, in static strategy settings such non-monotonic

offer curves can also occur at any time as a result of the dynamic effects of an agent

system in which the agents use mixed strategies involving behaviour-dependent and -

independent tactics, even though all tactics individually generate offers in a monotonic

manner. Such effects are often undesirable as they can delay agreements, significantly

change outcomes as compared to monotonic offers curves, and may also become dif-

ficult to control due to a high sensitivity of the strategy parameters. The automatic

and uncontrolled occurrence of non-monotonicity in the offer curves of static mixed

Chapter 3. Monotonic Mixing Mechanisms for Multi-Tactic Negotiation Strategies

strategies with monotonic tactics is different from the case where an agent intends to

produce such non-monotonic behaviour by changing its strategy parameters during the

encounter, and should therefore be avoided. As such behaviour can occur dynamic-

ally, it also makes it difficult for an agent to anticipate whether the resulting effects are

beneficial or not.

After giving definitions of monotonic tactics, this chapter describes the dynamic effects

that can result from the mixing of different types of tactics, such as imitative and non-

imitative, using static or dynamic weights. It then proposes new mixing mechanisms

based on individual negotiation threads for all imitative tactics involved, or based on

single concessions, that avoid the undesirable dynamic effects by guaranteeing mono-

tonic concession behaviour, the first for static and the second also for dynamic mixing

weights. An evaluation also compares the mixing mechanisms with the traditional

linear weighted combination in situations in which either of the parties or both use

the new mixing mechanism. A number of examples throughout the chapter further

illustrate the concession behaviour of the presented mechanisms.

In the following sections, the terms mixed strategy and multi-tactic strategy are used

interchangeably. We also refer to an individual tactic as a pure or single tactic. The

concept of a mixed strategy combining multiple tactics at a particular negotiation stage

is similar to that of mixed strategies in many game theoretic models. It should be noted,

however, that in the context of this thesis the weights in the linear combination do not

represent probabilities, but rather an agent’s method of attaching importance levels to

the tactics involved in terms of their contribution to the resulting concession behaviour.

3.1 Dynamic Behaviour of Multi-tactic Strategies

Heuristic-based tactics represent tractable decision models for the concession-making

of agents, which typically have monotonic utility functions for the issues under nego-

tiation. Under this premise, an agent makes a concession if its utility value decreases

with the new offer proposal compared to its previous offer (cf. Section 2.3) thereby try-

ing to make the offer more attractive to the opponent. When combining different types

of tactics using a linear weighted combination as shown in Section 2.3.2, an agent can

create complex concession behaviour that is able to take into account a range of differ-

56

3.1. Dynamic Behaviour of Multi-tactic Strategies

ent factors, such as its deadline, the opponents behaviour or the state of a resource in the

environment. Moreover, by changing the weights for the individual tactics during the

encounter, the agent can create more dynamic negotiation strategies in order to adapt

the concession behaviour to different situations. However, when changing weights dy-

namically the sequence of proposed offers may become non-monotonic, for example,

in cases where the weight is changed in favour of a more stubborn strategy. Although

such behaviour may be expected for dynamic weights, non-monotonic offer curves can

also occur for static weights at any time as a result of the dynamic interrelation of an

agent system in which the agents use mixed strategies involving behaviour-dependent

and -independent tactics, even when all tactics individually generate offers in a mono-

tonic manner. In other words, the negotiating agents using such mixed strategies con-

stitute a dynamic system in which non-monotonic concession behaviour can emerge

even when the agents’ strategy settings and mixing weights are static and all involved

tactics are monotonic in the sense that they propose concessions if they had been in-

dividually applied. The resulting effects of such non-monotonic behaviour are often

undesirable as they can delay agreements, significantly change outcomes as compared

to monotonic offers curves, and also affect the sensitivity of the strategy parameters

in that a small change of a parameter may result in a sudden and large change of out-

comes. Furthermore, it is often argued [108, 42] that the process of negotiation should

be designed in a way that agents make concessions, or, if possible, seek for joint im-

provements, for example in the form of trade-off proposals, in a negotiation where the

agents have monotonic utility structures (cf. Section 2.2.3). This also implies mono-

tonic behaviour: an agent makes proposals so that its own utility of its next offer is

equal or lower than its own utility of its previous offer, i.e. Ua(xtn+1

a→b) ≤ Ua(xtn−1

a→b ).

In the following, we say that agents have monotonic behaviour if they propose offers

according to this principle.

In single-issue negotiations the agents typically have opposing utility structures, and

although the exact utility functions are unknown, an agent can easily detect when the

opponent tries to increase its utility by proposing a non-monotonic sequence of offers.

An agent behaving in such a way may therefore increase the risk of a withdrawal of the

opponent. In single-issue negotiations, we can say that an agent a acts rational if it con-

cedes towards the last offer of its opponent, thereby trying to increase the opponent’s

utility such that the sequence of its own utilities is monotonically decreasing.

57


In multi-issue negotiations, on the other hand, an offer of an agent a with a higher

aggregated utility for a as compared to its previous offer can not easily be detected

by the opponent as the utility structures and the importance attached to the issues are

unknown to each other. Consequently, if the opponent’s utility for a’s last offer is lower

as a’s previous offers, the opponent may assume that agent amade a trade-off proposal

and can therefore not detect the cause of such non-monotonic behaviour. However, in

order to reach agreements faster agents should behave monotonic, i.e. propose offers

such that the sequence of their own utilities of proposed offers is monotonic decreasing.

It is also argued that agents behaving non-monotonic under time-constraints can be

advantageous and the question whether automated negotiation should be designed in

a way that monotonic behaviour is ensured is widely discussed in the research literat-

ure [138]. However, the occurrence of non-monotonicity in the sequence of proposed

offers and their respective utilities can also be the result of the dynamic effects of

an agent system in which two interacting agents use static mixed strategies. In such

cases, the agent did not intend to produce this non-monotonic behaviour, as for ex-

ample, by changing their strategy parameters, that makes it undesirable because of its

possible and unexpected emergence at any time. For that reason, we investigate the

non-monotonic behaviour of multi-tactic negotiation strategies in the next sections by

first defining when a tactic is considered monotonic and then discussing the effects

on the negotiation outcome by means of examples. Without loss of generality, we re-

strict the discussion to linear utility spaces in order to simplify the illustration of the

dynamic effects of non-monotonic concession behaviour. However, similar effects can

be observed when using multi-tactic strategies for the concession-making with other

monotonic utility structures.

3.1.1 Monotonicity of Negotiation Tactics

In order to enable the discussion about the interrelated dynamic concession behaviour

of multi-tactic negotiation strategies we need to determine the concession behaviour

of a pure tactic in terms of its monotonicity. In general, a tactic or decision func-

tion is considered monotonic if it produces a monotonic sequence of offers such that

xti+1 ≥ xti−1 or xti−1 ≤ xti+1 , if the utility is monotonic decreasing or increasing, re-

spectively. This criterion can easily be applied to tactics which entirely depend on time

58


(cf. Section 2.3.1.1). In a similar way, a tactic depending on a resource in the environ-

ment (cf. Section 2.3.1.2) can be characterised by the state of the resource at a certain

time. The monotonicity criterion can then be used to determine whether the tactic pro-

duces a monotonic offer sequence when the state of the resource changes over time.

In cases where a tactic is imitative towards the opponent’s behaviour to some degree

the monotonicity of the opponent’s sequence of offers need also to be considered in

addition to the above monotonicity criterion. We then say that a pure imitative tactic is

monotonic if the sequence of offers it generates is monotonic in the above sense if the

sequence of offers from the opponent is monotonic as well. For example, a mirroring

tactic such as absolute tit-for-tat without a random factor copies the concessions of the

opponent to the same degree such that the sequence of proposed offers is monotonic

increasing if the sequence of copied offers is monotonic decreasing.

In order to determine whether a mixed strategy generates a monotonic sequence of of-

fers we need to distinguish only between two general types of pure tactics: monotonic

behaviour-dependent and -independent, which are formally defined as follows:

Definition 3.1. Given a negotiation between agents a and b, a monotonic behaviour-independent tactic τaj (tk) of agent a for issue j is a function generating offers at any

times tk, ti ∈ Tn such that τaj (tk) ≥ τaj (ti) if Ua is decreasing or τaj (tk) ≤ τaj (ti) if Ua

is increasing under the condition that k, i ∈ {1, 2, . . . , n} and k > i.

Definition 3.2. Given a negotiation between agents a and b at time tn, a monotonicbehaviour-dependent tactic τaj (X tn

a↔b) generates an offer using any sequence X tna↔b =

(xta↔b)t∈Tn where Tn 6= ∅ and Tn ⊆ Tn = {t1, . . . , tn} under the conditions that there

exists at least one offer xtib→a ∈ Dbj of agent b in the sequence such that

• τaj (X tna↔b) ≥ τaj (X

tn−2

a↔b ) if the sequence of opponent’s offers (xtb→a)t∈Tn and Ua

is monotonic decreasing or

• τaj (X tna↔b) ≤ τaj (X

tn−2

b↔a ) if the sequence of opponent’s offers (xtb→a)t∈Tn and Ua

is monotonic increasing.

As mentioned above, the first definition represents tactics that depend on a particu-

lar resource which state may change over time. We denote this class of tactics with

τj,time for issue j. In the simplest case the tactic may depend on time or the number

59


of negotiation rounds. For instance, the polynomial and exponential time-dependent

decision functions proposed by Faratin et al [44] represent such tactics as they generate

offers in a monotonically decreasing or increasing manner. However, in the case of a

resource-dependent tactic the resource may diminish and increase over time such that

a monotonic sequence of offers can not be guaranteed. A behaviour-dependent tactic

according to Definition 2 uses some of the historical offers from the opponent to pro-

pose counteroffers and preserves a monotonic offer sequence as long as the opponent’s

sequence of offers is monotonic as well. We denote the class of such imitative tactics as

τj,beh. For instance, the imitative tit-for-tat tactics from [44] shown in Section 2.3.1.3

fulfil this definition. However, once non-monotonicity is introduced by one partner it

can in turn cause a non-monotonic offer sequence of the opponent depending on the

degree of how much the concessions are copied. Nevertheless, if monotonic tactics are

mixed together, non-monotonic behaviour can emerge even when both agents apply

monotonic tactics as we investigate in the next section.

3.1.2 Monotonicity of Multi-tactic Negotiation Strategies

This section investigates the non-monotonic behaviour of negotiation agents using

mixed strategies, with static and dynamic weights. Intuitively, non-monotonic beha-

viour can occur when an agent changes its strategy, e.g. the mixing weights, during

the encounter. The emergence of non-monotonic behaviour can also be observed when

imitative and non-imitative tactics are mixed by a linear weighted combination without

the agent changing its strategy, i.e. even in the case of static strategy settings and mix-

ing weights. A simple example shall demonstrate this:

Example 3.1 Non-monotonic concession behaviour for a single issueAssume a single-issue negotiation between two agents a and b at time tn where agent

a applies a mixed strategy with static weights specified by γ, and one time-dependent

tactic τatime(tn+1) and one imitative tactic. The imitative tactic is a simple (absolute) tit-

for-tat tactic (cf. Section 2.3.1.3) that copies the concession behaviour of the partner:

τabeh(xtn−2

b→a , xtn−1

a→b , xtnb→a) = x

tn−2

b→a −xtnb→a+ x

tn−1

a→b . The next offer of agent a’s is hence

given by

xtn+1

a→b =γ · τatime(tn+1)+(1−γ) · τabeh(xtn−2

b→a , xtn−1

a→b , xtnb→a) (3.1)

60


Assume further that, given an ongoing negotiation with the thread

(. . . , xtn−2

b→a , xtn−1

a→b , xtnb→a)=(. . . , 30, 10, 20), (3.2)

agent a’s next time-dependent proposal is τtime(tn+1) = 11. With a mixing weight of

γ = 0.5, the next counteroffer is xtn+1

a→b = 0.5 · 11+0.5 · 20 = 15.5. If agent b replies

with a comparatively small concession xtn+2

b→a = 19 and agent a’s next time-dependent

proposal is τtime(tn+3) = 12, then agent a’s response is lower than its previous offer

and thus non-monotonic with xtn+3

a→b = 0.5 · 12 + 0.5 · 16.5 = 14.25. �

In the above example, agent a proposes an offer with a higher utility for herself than

its previous offer as a result of the mixed strategy. This non-monotonic behaviour

occurred even though the sequence of opponents’ offers is monotonic and all involved

tactics in the mix are monotonic according to definitions 1 and 2 as they individually

propose concessions. If both agents have imitative tactics in their mix a non-monotonic

sequence of offers is likely to be copied to some degree reproducing non-monotonicity

in the sequence of opponent’s offers and vice versa. In addition, if the agents have

opposing utility functions, a non-monotonic utility sequence of one agent then also

causes a non-monotonic utility sequence of the partner’s offers.

To answer the question of why and when this non-monotonicity occurs in such scen-

arios we need to investigate if a static mixed strategy can guarantee monotonic offer

sequences in static cases with monotonic tactics. In order to so, it is sufficient to

use a simple mixed strategy with two tactics, one behaviour-dependent and the other

behaviour-independent, in a single-issue negotiation. Suppose that a buyer agent b

uses the mixed strategy while negotiating with a seller agent s about a price and the

following conditions hold:

a) Agent b uses monotonic tactics τ btime and τ bbeh according to definitions 1 and 2:

τ btime(tn+1) ≥ τ btime(tn−1) and τ bbeh(xtn−2

s→b , xtn−1

b→s , xtns→b) = x

tn−2

s→b − xtns→b + x

tn−1

b→s

b) The opponent s proposes offers in a monotonic sequence: xtns→b ≤ xtn−2

s→b

The next offer for agent b using the static mixed strategy is written as follows:

xtn+1

b→s = γbτ btime(tn+1) + (1− γb)(xtn−2

s→b − xtns→b + x

tn−1

b→s ) (3.3)

61


with tn, tn+1 ∈ time. After transforming the above equation the concession of the

buyer agent is given by

xtn+1

b→s − xtn−1

b→s = γb(τ btime(tn+1)− xtn−1

b→s ) + (1− γb)(xtn−2

s→b − xtns→b). (3.4)

In order to see if the mixed strategy produces a monotonic offer sequence the client

concession has to be greater than or equal to zero, i.e. xtn+1

b→s − xtn−1

b→s ≥ 0, such that

γb(xtn−1

b→s − τbtime(tn+1)) ≤ (1− γb)(xtn−2

s→b − xtns→b). (3.5)

This condition, however, can not be guaranteed because it depends not only on the

tactics of the agent but also on the concession of the opponent in relation to its pre-

vious offer. Therefore, the agent’s imitative tactic is not independent from the other

non-imitative tactic since it uses the last offer of the current negotiation thread which is

a result of the mixed strategy rather than the pure imitative tactic. Example 3.1 demon-

strated this situation in which agent b’s concession xtnb→a − xtn+2

b→a = 20 − 19 = 1

was smaller than the difference xtn−1

a→b − τatime(tn+1) = 15.5 − 12 = −2, so that

0.5(3.5) � (1 − 0.5)(1). As we can see, the occurrence of non-monotonic behaviour

depends on a number of factors such as the agent’s mixing weights, the opponent’s

amount and change of concessions, as well as the agent’s behaviour-independent tac-

tics. In a similar way the condition at which non-monotonicity occurs can be found

for other combinations of tactics. For example, using relative tit-for-tat instead of the

above absolute tit-for-tat tactic the condition is given by

1− γb τbtime(tn+1)

xtn−1

b→s≤ (1− γb)x

tn−2

s→bxtns→b

. (3.6)

The automatic occurrence of non-monotonic concession behaviour can result in a num-

ber of undesirable effects, such as delayed or failed agreements, varying outcomes,

compared to monotonic offer sequences, and a high sensitivity of the strategy para-

meters. In the following, we demonstrate such effects by means of some examples,

which will also be used in the subsequent sections for a comparison between the mix-

ing mechanisms in the next sections and the traditional linear weighted combination.

Example 3.2 Mixed strategies with large agreement zoneWe assume a single-issue negotiation between a buyer b and a seller s with the in-

62


tervals minb = 10, maxb = 25, and mins = 15, maxs = 30, and equal deadlines

tbmax = tsmax = 20. The buyer applies a mixed strategy with one time-dependent and

one imitative tactic, whereas we consider two cases for the seller in which either a

pure tactic or a mixed strategy is applied. The settings for the agent’s strategies are as

follows:

Buyer: mixed strategy γ = 0.3; time-dependent tactic: polynomial β = 0.5; imitative:

absolute tit-for-tat δ = 1, R(M) = 0.

Seller: mixed strategy γ = 0.4; time dependent: polynomial β = 4; imitative: absolute

tit-for-tat δ = 1, R(M) = 0.

Figure 3.1a shows the offer curves for the cases where the seller uses either the pure

time-dependent tactic or the mixed strategy. The agreement is slightly delayed in

both cases due to the non-monotonic offer curve produced by the mixed strategy of

the buyer, compared to situations in which the buyer applies the individual tactics

only. This seems counter-intuitive, because the expected offer curve should indeed lie

between the offer curves of the pure tactics applied. In the case where the opponent

also applies a mixed strategy with imitative tactics, the non-monotonic behaviour is

reciprocated. The seller’s mixed strategy copies the negative concessions of the buyer

and thus reproduces the non-monotonicity in its sequence of offers. �

The offer curves in Figure 3.1a of Example 3.2 show that non-monotonic offer curves

can occur in simple negotiation scenarios with static mixed strategies. However, in

that example, the buyer’s non-monotonic behaviour results in a outcome with a higher

utility gain for the buyer since the seller’s strategy is static (time-dependent) and does

not change with different behaviours. On the other hand, in single-issue negotiations,

such behaviour may increase the risk of a failed agreement due to a withdrawal of the

opponent, or, at least, the opponent might also change its concession behaviour. The

following example illustrates the effect of the produced non-monotonic behaviour for

the case where the two agents have a smaller zone of agreement.

Example 3.3 Mixed strategies with small agreement zoneIn this example, the buyer and the seller use the same settings for the mixed strategy,

imitative and time-dependent tactics as in the previous Example 3.2, but with a smaller

overlap of the negotiation intervals with minb = 10, maxb = 25, mins = 15, and

maxs = 30, and different deadlines tsmax = 15 and tbmax = 20. The resulting overall

agreement zone is hence smaller than in the previous example. Figure 3.1b shows the

63


offer curves for the cases where the seller uses either the pure time-dependent tactic

or the mixed strategy. In both cases, no agreement can be obtained due to the the

non-monotonic offer curves produced by the mixed strategy of the buyer. �

ò

ò

ò

òò

ò

ò

ò

òò

ì

ì

ì

ìì

ìì

ìì

ìì

ìì ì ì

ì ì ì ì ìì

ìì

ìì

ì

ì

ì

ì ì

æ

æ

æ

æ

æ

ææ

ææ

ææ

ææ

ææ æ

æ

ææ

ææ

æ æ æ ææ

ææ

æ

æ

æ æ

0 5 10 15t

15

20

25

30x

ò

ò

ò

ò

ò

ò

ì

ìì

ì

ì

ìì

ìì

ìì

ìì

ì

ì ì ì ì ìì

ìì

ìì

ì

ì

ìì

æ

æ

æ æ

æ

ææ

æ

æ

ææ

ææ

ææ

æ

æ

æ

æ

æ

ææ

æ

ææ

ææ

æ

æ æ

0 5 10 15t

15

20

25

30x

(a) Example 3.2 with seller using pure time-dependent tactic (left) or mixed strategy (right)

ò

ò

ò

òò

òò

ò ò

ò

ò

ò

òò

òò

ò ò

ì

ì

ì

ìì

ìì

ìì

ìì ì ì ì ì ì

ì ì ì ì ì ìì

ìì

ìì

ìì

ì

ì

ì

ì

ì

ì

ì

ì

æ

æ

æ

æ

ææ

ææ

ææ

ææ

ææ æ æ

æ

ææ

ææ

æ æ æ æ ææ

ææ

æ

æ

æ

æ

æ

æ

æ

æ

0 5 10 15 20t

15

20

25

30

35x

ò

ò

ò

ò

ò

ò

ì

ìì

ìì

ìì

ìì

ìì

ìì ì ì ì

ì ì ì ì ì ìì

ìì

ìì

ìì

ì

ì

ì

ì

ì

ì

ì

ì

æ

æ

æ æ

æ

ææ

æ

æ

ææ

ææ

ææ æ

æ

æ

æ

æ

æ

æ æ

æ

ææ æ

ææ

æ

ææ

æ

æ

æ

æ

æ

0 5 10 15 20t

15

20

25

30

35x

(b) Example 3.3 with seller using pure time-dependent tactic (left) or mixed strategy (right)

Linear weighted combination Pure time-dependent Pure imitative

Figure 3.1: Offer curves for Examples 3.2 and 3.3 when using the linear weightedcombination or pure tactics

It is important to note that in both examples above the offer curve produced by the

buyer’s mixed strategy approaches the offer curve of the applied pure time-dependent

tactic towards the end of the negotiation (cf. Figure 3.1). This seems counter-intuitive,

since the buyer’s mixing weight is γ = 0.3, i.e. in favour of the imitative tactic, so

that the outcome of the mixed strategy should, in fact, be closer to the outcome of the

pure imitative tactic. Therefore, the linear weighted combination of tactics does not

represent a true mix of both imitative and non-imitative tactics in this scenario.

Another effect in mixed strategy scenarios is that the strategy parameters may become

highly sensitive as a result of the dynamic interrelation between the two agents in the

64


0.0

0.5

1.0

Γ

0

1

2

3

4

Β

20

25

30

35

x

(a) Outcomes for buyer strategy settings β ∈[0.01, 4] and γ ∈ [0, 1]

0.2 0.4 0.6 0.8 1.0Γ

26

27

28

29

30x

(b) Outcomes for buyer strategy settings β = 0.8and γ ∈ [0, 1]

(Seller: mixed strategy γ = 0.25; time dependent: polynomial β = 1; imitative: relative tit-for-tatδ = 1 / Buyer: mixed strategy γ ∈ [0, 1]; time-dependent: polynomial β = 1; imitative: absolute

tit-for-tat δ = 1, R(M) = 0)

Figure 3.2: Outcomes for different buyer strategy parameters when using linearweighted combinations of tactics

sense that little changes in the settings of one agent may result in a sudden and signi-

ficant change in the negotiation outcome. In such cases, the described dynamics of the

system makes it difficult for an agent to control such a mixed strategy and to determ-

ine whether a strategy preforms good or not. Figure 3.2 shows the outcome range for

different mixing weights and concession settings in the case of a time-dependent tac-

tic in the mix for a negotiation example in which both agents use mixed strategies. In

this example, the outcome range is almost similar for all buyer’s mixed strategies using

mixing weights of 0.6 and higher, whereas below this value the outcome may suddenly

change even for small changes. This high sensitivity makes it difficult for an agent to

apply such strategies in real world scenarios, because it does not depend solely on the

agent’s parameters, but also on the opponent’s settings, which are private information.

In single-issue negotiations, the non-monotonic concession behaviour also results in

a non-monotonic utility curve of the agent. In a similar manner, such effects can be

observed in multi-issue negotiations, which we demonstrate by the following example.

Example 3.4 Non-monotonic utility for multiple issuesAssume a negotiation between a buyer and a seller about two issues with static strategy

settings shown in Table 3.1. Figure 3.3 shows the offer curves and utility curves of both

agents for two different mixing weights of the provider for issue 1. The offer curve of

65


Buyer Agent Seller AgentIssue 1

min1=10, max1=25, w1=0.7 min1=15, max1=30, w1=0.5

Mixed Strategy (γ1 = 0.3): Mixed Strategy (γ1∈{0.1, 0.12}):τ1,time: polynomial, β1 = 5 τ1,time: polynomial, β1 = 1

τ1,beh: absolute tft, δ1=1, R1=0 τ1,beh: absolute tft, δ1 = 1, R1=0

Issue 2min2 = 20, max2 = 40, w2 = 0.3 min2 = 30, max2 = 50, w2 = 0.5

Mixed Strategy (γ2 = 0.4): Mixed Strategy (γ2 = 0.2):τ2,time: polynomial, β2 = 2 τ2,time: polynomial, β2 = 0.3

τ2,beh: absolute tft, δ2=1, R2=0 τ2,beh: relative tft, δ2 = 1

Table 3.1: Negotiation settings for example 3.4

the seller change rapidly when the seller changes its mixing weight for issue 1 by a

small amount and becomes non-monotonic (Figure 3.3a and 3.3b). As both agents use

imitative tactics in their mix and apply the traditional linear weighted combination, the

non-monotonic behaviour of one agent is reproduced by the other. As a consequence,

the offer and utility curves of both agents become non-monotonic, with the result that

the agreement is delayed. Figure 3.3 also illustrates that the agreement is delayed

in comparison to a monotonic mixing mechanism (negotiation thread-based) that is

introduced in Section 3.2. In situations, where the agents have different deadlines this

behaviour might also result in a failed agreement. The example demonstrates also the

high sensitivity of the parameters in such scenarios that makes it difficult for an agent

to find suitable strategy parameters as the outcome utility may change significantly for

slightly different settings. As shown in Figure 3.3b and 3.3d, the seller and buyer utility

changes from U s = 0.26 and U b = 0.26 to U s = 0.4 and U b = 0.16, respectively,

when the seller changes its weight γs1 from 0.12 to 0.1. �

The multi-issue negotiation example above demonstrates that a non-monotonic utility

sequence for the agent’s own offers can occur at any time when using static mixed

strategies with imitative and non-imitative tactics. Since the strategy parameters of the

opponent are private information, it is difficult for an agent to anticipate such behaviour

and detect if the effects are beneficial or not. Further, it should be noted that the

uncontrolled occurrence of the non-monotonic concession behaviour in static strategy

settings is different from the case in which an agent chooses to behave non-monotonic,

e.g. by changing its mixing weights, and should therefore be avoided.

66


æ

æ

æ

æ

æ

æ æ

æ

ææ æ

æ

æ

æ

æ

æ

æ

ææ

ææ

æ ææ

æ

æ

à

à

à

à

à

à

àà

à

à

à

à

à

à

à à

0 5 10 15t

15

20

25

30Issue 1

æ æ

æ

æ

ææ

æ

æ

ææ

æ

ææ

æ

æ

æ

æ

ææ

ææ

æ

æ

æ

æ

æ

à à

à

à

à

à

à

à

à

à

à

à

à

à

à à

0 5 10 15t20

25

30

35

40

45

50Issue 2

(a) Buyer’s and seller’s offer curves for issue 1 and 2 and seller’s mixing weight γ1 = 0.12

æ

æ

æ

æ

æ ææ

æ æ

æ

æ

æ æ

ææ

æ

æ

æ

æ

æ

æ

æ ææ

æ

æ

à

à

à

à

à

à

à à

àà

à

à

à

à

à

à

5 10 15t

-0.4

-0.2

0.2

0.4

0.6

0.8

1.0Ub

ææ

æ

æ

ææ

æ

æ

æ æ

æ

æ

æ

æ

æ

æ

æ

æ æ

ææ æ

æ

æ

æ

æ

àà

à

à

à

à

à

à

à

à

à

à

à

à

à à

5 10 15t

-0.4

-0.2

0.2

0.4

0.6

0.8

1.0U s

(b) Buyer’s and seller’s aggregated offer utilities for seller’s mixing weight γ1 = 0.12

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

æ

à

à

à

à

à

à

à à

à

à

à

à

à

à

à à

0 5 10 15t

15

20

25

30Issue 1

æ æ

æ

æ

ææ

æ

æ

æ

æ

æ

æ

à à

à

à

à

à

à

à

à

à

à

à

à

à

à à

0 5 10 15t20

25

30

35

40

45

50Issue 2

(c) Buyer’s and seller’s offer curves for issue 1 and 2 and seller’s mixing weight γ1 = 0.1

æ

æ

æ

æ

æ æ

ææ

æ

æ

æ

æ

à

à

à

à

à

à

à à

àà

à

à

à

à

àà

5 10 15t

-0.4

-0.2

0.2

0.4

0.6

0.8

1.0Ub

ææ

æ

æ

ææ

æ

æ

æ

æ

æ

æ

àà

à

à

à

à

àà

à

à

à

à

à

à

à à

5 10 15t

-0.4

-0.2

0.2

0.4

0.6

0.8

1.0U s

(d) Buyer’s and seller’s aggregated offer utilities for seller’s mixing weight γ1 = 0.1

Linear weighted combination Negotiation thread-based mixing

Figure 3.3: Offer and utility curves for Example 3.4 using the traditional linearweighted combination or the negotiation thread-based mixing

67


æ

æ

æ

æ

æ

ææ

ææ

ææ

ææ

ææ æ

æ

ææ

ææ

æ æ æ ææ

ææ

æ

æ

æ æ

ô

ô

ô

ôô

ôô

ôô

ôô

ôô ô ô

ô

ô ô ô ô ô ô ô ô ô ô ôô

ôô

0 5 10 15t

15

20

25

30x

æ

æ

æ æ

æ

ææ

æ

æ

ææ

ææ

ææ

æ

æ

æ

æ

æ

ææ

æ

ææ

ææ

æ

æ æ

ô

ô

ô ô ô ô ô ôô

ôô

ôô

ôô

ô

ô

ô ô ô ô ô ô ô ô ô ô ôô ô

0 5 10 15t

15

20

25

30x


æ

æ

æ

æ

ææ

ææ

ææ

ææ

ææ æ æ

æ

ææ

ææ

æ æ æ æ ææ

ææ

æ

æ

æ

æ

æ

æ

æ

æ

ô

ô

ô

ôô

ôô

ôô

ôô ô ô ô ô ô

ô

ô ô ô ô ô ô ô ô ô ô ô ôô

ôô

ô

ô

ô

ô

ô

0 5 10 15 20t

15

20

25

30

35x

æ

æ

æ æ

æ

ææ

æ

æ

ææ

ææ

ææ æ

æ

æ

æ

æ

æ

æ æ

æ

ææ æ

ææ

æ

ææ

æ

æ

æ

æ

æ

ô

ô

ô ô ô ô ô ô ôô

ô ô ô ôô

ô

ô

ô

ô ô ô ô ô ô ô ô ô ô ô ôô

ô

ô

ô

ô

ô

ô

0 5 10 15 20t

15

20

25

30

35x


Linear weighted combination Constrained linear weighted combination

Figure 3.4: Offer curves for Examples 3.2 and 3.3 when using the constrained linearweighted combination (compared to the traditional linear weighted combination)

3.1.3 Constrained linear weighted combination

The question remains of how the occurrence of non-monotonic concession behaviour

in static and dynamic mixed strategies can be avoided. Intuitively, a simple min- or

max-constraint could be applied to the next offer proposal, such that

xtn+1

a→b =

max(xtn+1

a→b , xtn−1

a→b ) if Ua is increasing

min(xtn+1

a→b , xtn−1

a→b ) if Ua is decreasing,.(3.7)

This ensures that the agent’s own utility does not increase compared to its previous

offer. However, the offer curve may then rapidly change to linear, so that the agent

proposes the same offer over a long time period, which may also increase the risk of

68

3.2. Mixing based on Negotiation Threads

the opponent’s withdrawal. This is demonstrated in Figure 3.4 which shows the offer

curves for Examples 3.2 and 3.3 when the constraint is applied to the mixed strategies

in comparison to the offer curves without the constraint. In all cases, the offer curve of

the mixed strategy with a constraint applied approaches the offer curve of the mixed

strategy without the constraint towards the end of negotiation, and leads to similar

outcomes. The constrained linear weighted combination still does not represent a true

mix of both imitative and non-imitative tactics in this scenario. For these reasons, we

present in the next sections two alternative mixing mechanisms that produce monotonic

offer and utility sequences in multi-tactic negotiation strategies based on individual

negotiation threads for each imitative tactic involved or single concession.

3.2 Mixing based on Negotiation Threads

To calculate the imitative tactics in mixed strategies using the traditional mixing method

the last offer in the current negotiation thread is used. The imitative part of the strategy

does therefore not represent an individually applied behaviour-dependent tactic. An-

other intuitive method is to use the last offers of each imitative tactic involved in the

mix. This can be interpreted as using individual negotiation threads X tna↔b[j, k] where

k denotes the k’th behaviour-dependent tactic τjk(X tna↔b[j, k]) for issue j. As a result,

offers from all imitative functions have to be stored in order to be used in the calcula-

tion of next proposals. Formally, the linear weighted combination of tactics can now

be written as follows:

xtn+1

a→b [j] =l∑

i=1

γji · τji(tn+1) +m∑

k=l+1

γjk · τjk(X tna↔b[j, k]) (3.8)

where m and l denote the total number and the number of behaviour-independent tac-

tics, respectively. Unlike the traditional mixing method described in Section 2.3.2 this

method can be regarded as a true linear weighted combination of tactics in which all

involved tactics are independent from each other.

Theorem 1. The mixing mechanism using individual negotiation threads for each

behaviour-dependent tactic results in a monotonic offer curve if monotonic tactics from

definitions 1 and 2 are used with static weights for all tactics.

69


Proof Let X tna↔b be the negotiation thread at time tn with xtnb→a[j] being the last of-

fer and xtn+1

a→b [j] being the next counteroffer of agent a for issue j, then, according to

Definition 1 and 2 γjk · τjk(X tna↔b[j, k]) ≥ γjk · τjk(X tn−2

a↔b [j, k]) and γji · τji(tn+1) ≥γji · τji(tn−1) if Ua is decreasing and all γji, γjk ≥ 0. Since each term of the sum

in (3.8) at tn is larger than the corresponding term of the sum at tn−2 it follows that

xtn+1a→b ≥ xtn−1a→b . The same line of reasoning can be followed for an increasing utility

function Ua. �

Taking the example 3.1 from Section 3.1.2 we can calculate the offer sequence using

the thread-based mechanism as follows. Given the thread (. . . , xtn−2

b→a , xtn−1

a→b , xtnb→a) =

(. . . , 30, 10, 20) agent a’s next offer is similar to the traditional mixing method with

γ · τatime(tn+1)+(1−γ) · τabeh(xtn−2

b→a , xtn−1

a→b , xtnb→a) = 0.5 · 11 + 0.5 · 20 = 15.5. After

b’s next offer of 19 the agent uses for the imitative part of the mixed strategy the

imitative offer 20 from the previous step instead of the actual offer 15.5, so that the

imitative part is 20 + 1 = 21 instead of 16.5. The new offer of agent a is then given

by xtn+3

a→b = 0.5 · 12 + 0.5 · 21 = 16.5 which is larger than the agent previous offer and

therefore results in a monotonic offer sequence.

Figure 3.5 shows the offer curves produced by the negotiation thread-based mechanism

for Examples 3.2 and 3.3 in comparison to the offer curves of the traditional linear

weighted combination. In Example 3.2 an agreement is reached at an earlier time

when using the thread-based mechanism, because the offer curve is now closer to the

offer curve of the imitative tactic, if it had been individually applied. This seems

intuitive as the buyer’s mixing weight is 0.3 (and therefore in favour of the imitative

tactic). As a result, the offer curves make larger concessions due to the partially copied

conceder time-dependent tactic used by the seller, which also leads to agreements in

Example 3.3, compared to the traditional mixing method (cf. Figure 3.5b). Figure

3.3 shows the monotonic offers curves and the resulting monotonic utility sequence

when both agents use this mixing mechanism for Example 3.4. The offer curves of

the negotiation thread-based mechanism also do not rapidly change for small changes

in the strategy settings, compared to the traditional mixing method. An agreement is

reached at an earlier time with a changed outcome favour of the seller. The system of

agents using this mixing mechanism does not expose the dynamic effects as described

in Section 3.1.2. However, the mechanism does not force the agent to propose offers

70

3.2. Mixing based on Negotiation Threads

ò

ò

ò

òò

ò

ò

ò

òò

ì

ì

ì

ìì

ìì

ìì

ìì

ìì ì ì

ì ì ì ì ìì

ìì

ìì

ì

ì

ì

ì ì

æ

æ

æ

æ

æ

ææ

ææ

ææ

ææ

ææ æ

æ

ææ

ææ

æ æ æ ææ

ææ

æ

æ

æ æ

à

à

à

à

àà

àà

à

à

à

àà

àà à

0 5 10 15t

15

20

25

30x

ò

ò

ò

ò

ò

ò

ì

ìì

ì

ì

ìì

ìì

ìì

ìì

ì

ì ì ì ì ìì

ìì

ìì

ì

ì

ìì

æ

æ

æ æ

æ

ææ

æ

æ

ææ

ææ

ææ

æ

æ

æ

æ

æ

ææ

æ

ææ

ææ

æ

æ æ

à

à

à

à

à

à

à à

0 5 10 15t

15

20

25

30x


ò

ò

ò

òò

òò

ò ò

ò

ò

ò

òò

òò

ò ò

ì

ì

ì

ìì

ìì

ìì

ìì ì ì ì ì ì

ì ì ì ì ì ìì

ìì

ìì

ìì

ì

ì

ì

ì

ì

ì

ì

ì

æ

æ

æ

æ

ææ

ææ

ææ

ææ

ææ æ æ

æ

ææ

ææ

æ æ æ æ ææ

ææ

æ

æ

æ

æ

æ

æ

æ

æ

à

à

à

àà

àà

àà

àà

à

à

à

àà

àà

àà

àà

àà

0 5 10 15 20t

15

20

25

30

35x

ò

ò

ò

ò

ò

ò

ì

ìì

ìì

ìì

ìì

ìì

ìì ì ì ì

ì ì ì ì ì ìì

ìì

ìì

ìì

ì

ì

ì

ì

ì

ì

ì

ì

æ

æ

æ æ

æ

ææ

æ

æ

ææ

ææ

ææ æ

æ

æ

æ

æ

æ

æ æ

æ

ææ æ

ææ

æ

ææ

æ

æ

æ

æ

æ

à

à

à

à à à à à à à

à

à

à

à à à à à à à

0 5 10 15 20t

15

20

25

30

35x


Traditional linear weighted combination Negotiation thread-basedPure time-dependent Pure imitative

Figure 3.5: Offer curves for examples 3.2 and 3.3 when using the negotiation thread-based mixing (compared to the traditional linear weighted combination)

in a monotonic manner. For instance, if the opponent still proposes offers in a non-

monotonic sequence, an imitative tactic in the mix might still copy it to some degree.

The agent may choose to strictly ensure monotonicity by applying a constraint C to

the imitative tactic:

C(τjk(Xtna↔b[j, k]), x

tn−1

a→b [j, k]) (3.9)

where C ≡ min if Ua decreasing and C ≡ max if Ua increasing. Although the

negotiation thread-based mechanism truly mixes the pure imitative and non-imitative

tactics in the mix according to their weights, the individual imitative threads used by

this method do not represent the actual negotiation thread. This seems counter-intuitive

as the offer curve and the outcome of the individually applied imitative tactics might

indeed be different from the mixed strategy. Therefore, we propose another mixing

mechanism based on combining single concession in the next section.

71


3.3 Mixing based on Single Concessions

Instead of mixing the individual offers generated by each tactic in the linear weighted

combination, one can use single concessions. Since the decision functions of most

heuristic negotiation strategies generate offers rather than concessions, such as the

tactic shown in Section 2.3.1, a concession-based form needs to be derived first. For

example, the concession proposed by the time-dependent functions in Section 2.3.1

can be written as

∆tn+1τji = τji(tn+1)− τji(tn−1) (3.10)

where i is a time-dependent tactic according to Definition 3.1. In the case of a behaviour-

dependent decision function the concession can be expressed as the difference between

the behaviour-dependent offer to be proposed and the last proposed offer of the agent,

∆tn+1τjk = τjk(Xtna↔b[j])− x

tn−1

a→b [j] (3.11)

with k being a imitative tactic according to Definition 3.2 (cf. imitative tactics in

Section 2.3.1.3). For example, in the case of the absolute tit-for-tat the concession-

based is written as

∆tn+1τabs−tft = xtn−2δ

b→a − xtn−2δ+2

b→a + (−1)s ·R(M) (3.12)

with δ, s and R(M) being the same as in Eq. (2.11) in Section 2.3.1.3. In a similar

way any decision function might be expressed in the concession-based form such that

the linear weighted combination of tactics in the concession-based form is given by

xtn+1

a→b [j] = xtn−1

a→b [j] +l∑

i=1

γji ·∆tn+1τji +m∑

k=l+1

γjk ·∆tn+1τjk (3.13)

with m and l denoting the total number and the number of behaviour-independent tac-

tics respectively. In order to use concessions at least two offers of the opponent are

necessary. Any of the former mechanisms can be used for initial offers as they pro-

pose the same offers in the first round. Concessions for behaviour-independent tactics

are, since they do not depend on opponent’ offers, the difference τji(tn+1)− τji(tn−1)between the calculated offer at tn+1 and the previous individual offer at tn−1. For the

imitative tactic we can not follow the same line of reasoning because, as described

72

3.3. Mixing based on Single Concessions

in the previous section, the last offer of the individually applied imitative tactic is

unknown. However, suppose that the agent changed its strategy to the pure imitat-

ive tactic at time tn+1 the last offer is still be xtn−1

a→b and hence the next offer is given

by τjk(X tna↔b[j]). We can hence calculate the behaviour-dependent concession by the

difference between the proposed imitative offer and the last offer of the agent. This ap-

proach provides monotonic offer curves similar to the negotiation thread-based mixing

and also avoids non-monotonic aggregated utilities over time. The major advantage,

however, is that a monotonic sequence of utilities is also never introduced if the agent

changes weights for tactics dynamically. This can be proven as follows:

Theorem 2. The mixing mechanism based on single concessions of pure tactics results

in a monotonic offer curve (and therefore preserves a monotonic sequence of utilities)

if monotonic tactics from Definitions 1 and 2 are used by both parties.

Proof Let X tna↔b be the negotiation thread at time tn with xtnb→a being the last offer

and xtn+1

a→b being the next counteroffer of agent a then according to Definition 1 the

behaviour-independent concession τaji(tn+1)− τaji(tn−1) is always greater zero if Ua is

increasing. The offer proposed by the pure behaviour-dependent tactics τajk(Xtna↔b[j])

for issue j is greater than the previous offer xtn−1

a→b [j] if monotonic tactics from Defin-

ition 2 are used and the opponent never introduces non-monotonicity. The behaviour-

dependent concession τajk(Xtna↔b[j])− x

tn−1

a→b [j] is therefore always greater zero. For all

weights γi, γk ≥ 0 follows that each term of the sum in Eq. (3.13) is greater zero and

hence xtn+1a→b [j] ≥ xtn−1a→b [j]. The same line of reasoning can be followed for an increas-

ing scoring function Ua. �

For example, we can calculate the offer sequence in Example 3.1 from Section 3.1.2

using the concession-based mechanism as follows. Given the thread (. . . , 30, 10, 20)

agent a’s next offer is similar to the traditional mixing method even though the agent

uses the individual concessions of each tactic, such that γ · τatime(tn+1) + (1 − γ) ·τabeh(x

tn−2

b→a , xtn−1

a→b , xtnb→a) = 10 + 0.5 · (11 − 10) + 0.5 · 10 = 15.5. After b’s next

offer of 19 agent a using the concessions for the mix generates a new offer xtn+3

a→b =

15.5 + 0.5 · 12− 11 + 0.5 · 1 = 16.5 which is larger than the agent previous offer

and represents a monotonic offer sequence. The result is also similar to the negotiation

thread-based method.

73


æ

æ

æ

æ

ææ

ææ

ææ

ææ

ææ æ æ

æ

ææ

ææ

æ æ æ æ ææ

ææ

æ

æ

æ

æ

æ

æ

æ

æ

ø

ø

øø

øø

øø ø ø ø ø

ø

øø

øø

øø

øø

øø

ø

0 5 10 15 20t

15

20

25

30

35x

(a) Seller using pure time-dependent tactic

æ

æ

æ æ

æ

ææ

æ

æ

ææ

ææ

ææ æ

æ

æ

æ

æ

æ

æ æ

æ

ææ æ

ææ

æ

ææ

æ

æ

æ

æ

æ

ø

ø

ø

ø

ø

ø

ø

ø

øø

0 5 10 15 20t

15

20

25

30

35x

(b) Seller using mixed strategy

Traditional linear weighted combination Concession-based mixing

Figure 3.6: Offer curves for Example 3.3 when using the concession-based mixing(compared to the traditional linear weighted combination)

The concession-based mixing mechanism produces similar offer curves as the nego-

tiation thread-based mechanism for the Example 3.2 with a single issue and Example

3.4 with multiple issues. For that reason, we do not show the offer curves here (see

Figure 3.5 and Figure 3.3 for the offer curves of the thread-based mechanism). How-

ever, in Example 3.3 the concession-based mixing obtains an earlier agreement for the

case where the seller uses a mixed strategy with a similar outcome to the thread-based

mixing. This is shown in Figure 3.6b. The concession-based mechanism also does

not expose the high sensitivity of strategy parameters in the examples above. In con-

trast to the thread-based mixing, this mechanism needs no separate negotiation threads

and produces monotonic offer curves even for dynamically changing weights. There-

fore, an agent can change its strategy dynamically during the negotiation encounter

and ensure that it never introduces a non-monotonic sequence of offers. Like the pre-

vious method the mechanism does not force the agent to propose offers in a monotonic

manner because an involved imitative tactic may still copy a non-monotonic sequence

of offers of the opponent. The agent can strictly avoid such imitation by applying a

constraint C to each imitative concession in (3.13) written as

C(τjk(Xtna↔b[j])− x

tn−1

a→b [j], 0) (3.14)

where C ≡ min if Ua decreasing or C ≡ max if Ua increasing. In the next section,

we evaluate the mixing mechanisms investigated in this chapter.

74

3.4. Evaluation

3.4 Evaluation

This section presents the results of a comparative evaluation of the discussed mixing

mechanisms in this chapter with respect to their non-monotonic concession behaviour

and the respective effects in different bilateral negotiation settings.

3.4.1 Experiment Settings

In order to enable an analysis of the individual concession behaviour of mixed strategies

we consider negotiations about a single-issue (for example price) between a client c and

a provider p and both agent using mixed strategies. Because the number of possible

mixes of tactics is infinite, we restrict the evaluation to an example mix of two tactics,

one behaviour- and one time-dependent, for each agent with static weights throughout

the encounter. The tactics chosen are the polynomial decision function and absolute

tit-for-tat, such the the set of strategy groups as detailed in Section 2.6.2 is generated

for the different settings and mixing weights as follows:

ST = {PC, PL, PB} × {a} × {S,M,L} (3.15)

The absolute tit-for-tat tactic is chosen because it is a symmetrical tactic, i.e. has the

same behaviour regardless whether a buyer or seller applies it, and is also independent

of the scale of the negotiation interval (cf. Section 2.3.1.3). The polynomial decision

function is a good choice as the concession behaviour can be clearly classified into

the groups conceder, linear and boulware. This simplifies the interpretation of the ex-

perimental results. The different mixing mechanisms are compared against each other

in a setting where the provider randomly selects a strategy of the set ST while the

client plays a particular strategy group from ST . This is similar to playing a partic-

ular strategy group by the client against the average of all strategies in the set ST by

the provider. Furthermore, it is interesting to see how the agents perform if both use

the same mixing mechanism or only one agent uses the monotonic mixing mechan-

isms. For that reason, we distinguish between two types of scenarios, one-sided and

two-sided, where in the former the provider uses the traditional linear weighted com-

bination of tactics while the client applies the different mixing mechanisms, and in the

latter both agents use the same mechanism for each strategy group. In addition, we

75


also consider an agent’s more rational behaviour in that it withdraws from the nego-

tiation if it detects a non-monotonic concession behaviour of the opponent for more

than two negotiation rounds. Since the non-monotonicity only occurs in the one-sided

scenarios, we only consider such ’rational’ behaviour for the one-sided case, and we

call the two scenario variants ’one-sided without withdraw’ or ’one-sided with with-

draw’ scenario. The performance of the mixed strategies is measured using the average

intrinsic utility Ua with a ∈ {c, p}, the negotiation length tn and the agreement rate A

(in %). The agents employ linear utility functions which enables the direct measure-

ment of the effects of the different mixing mechanisms in terms of their influence on

the concession behaviour. We use bar chart diagrams to illustrate the performance of

the individual mixing mechanisms (with the small dotted bars on top representing the

standard deviation). In each diagram a group of bars represents one strategy scenario,

where the different bars depict the mixing mechanism from left (light) to right (dark)

as follows:

1 - Linear weighted combination of tactics

2 - Constrained linear weighted combination

3 - Negotiation thread-based mechanism

4 - Concession-based mechanism

As described in Section 2.6.2 we focus on scenarios with more realistic settings. That

means that agents have only partial overlap of their negotiation intervals, i.e. that

the zone of agreement is either small or large (Φ ∈ {0.33, 0.66}). In addition, the

agents typically do not know their opponents deadlines as it part of their preferences,

but an agent system may also have a system-specific deadline for their negotiation

interactions. Because of this, we distinguish between scenarios with equal or different

deadlines in the evaluation. In the evaluation the negotiation environment settings are

as follows:

• Client: tcmax ∈ {20, 25, 30, 35, 40}, minc ∈ {10}, maxc ∈ {25}

• Provider: tpmax ∈ {20, 25, 30, 35, 40}, minp ∈ {10 + 25Φ|Φ ∈ {0.33, 0.66}},maxp ∈ {minp + 15}

76

3.4. Evaluation

Since it also important to see how often non-monotonic concession curves occur when

both agents use the traditional mixing mechanism we first consider a simple example

scenario and provide the percent of negotiations in which non-monotonicity occurred

as well as the degree of non-monotonicity measured in terms of utility. As a result, the

following types of negotiation scenarios are considered:

• Non-monotonicity of concession curves

• Small overlap and equal deadlines

• Small overlap and different deadlines

• Large overlap and equal deadlines

• Large overlap and different deadlines

It should be noted that, as described in Section 2.3, in multi-issue negotiations the

decision strategies using the heuristic-based tactics for the concession-making of an

agent are typically applied either for each issue individually or along the indifference

curves according to the agent’s utility function, potentially in combination with a trade-

off mechanism [120]. Therefore, it is sufficient to focus the evaluation on a single issue.

In addition, the space of possible combinations of mixed strategies for a number of

issues and their respectively used tactics becomes intractably large and is thus difficult

to evaluate. However, an initial evaluation for some examples can be found in [113].

The following sections present the experimental results and their discussion for the

settings described above.

3.4.2 Non-Monotonicity of Concession Curves

Before comparing the different mixing mechanisms we are interested in when and to

what degree non-monotonic behaviour emerges in static mixed strategies using the tra-

ditional linear weighted combination of tactics. In order to demonstrate this, we choose

an example scenario with equal deadlines and large overlap of negotiation intervals

(Φ = 0.33). Table 3.2 illustrates the rate (%) of negotiations where non-monotonic of-

fer curves occurred in the case of both agents applying the traditional linear weighted

combination of tactics for a particular strategy group from ST . The numbers below

77


p / c PCaS PCaM PCaL PLaS PLaM PLaL PBaS PBaM PBaLPCaS 5 % 4 % 4 % 32 % 85 % 53 % 37 % 98 % 83 %

0.14 0.12 0.14 0.33 0.3 0.25 0.4 0.34 0.320.11 0.07 0.13 0.42 0.36 0.33 0.47 0.39 0.43

PCaM 25 % 26 % 31 % 56 % 100 % 62 % 60 % 100 % 70 %0.08 0.1 0.09 0.21 0.21 0.16 0.23 0.23 0.190.08 0.07 0.08 0.35 0.31 0.24 0.42 0.39 0.33

PCaL 14 % 19 % 17 % 58 % 86 % 51 % 60 % 96 % 84 %0.05 0.12 0.09 0.26 0.3 0.23 0.34 0.32 0.30.04 0.07 0.06 0.35 0.32 0.25 0.45 0.36 0.36

PLaS 61 % 79 % 57 % 0 % 0 % 0 % 33 % 72 % 60 %0.22 0.23 0.24 - - - - - -0.11 0.08 0.12 - - - 0.04 0.04 0.03

PLaM 22 % 78 % 64 % 0 % 0 % 0 % 4 % 24 % 35 %0.09 0.14 0.15 - - - - - -

- 0.01 0.01 - - - 0.01 0.01 0.01PLaL 17 % 46 % 40 % 0 % 0 % 0 % 21 % 45 % 33 %

0.22 0.18 0.16 - - - - - -0.11 0.07 0.07 - - - 0.05 0.03 0.04

PBaS 89 % 86 % 90 % 0 % 0 % 0 % 0 % 0 % 0 %0.35 0.41 0.42 - - - - - -0.16 0.1 0.17 - - - - - -

PBaM 88 % 100 % 90 % 0 % 15 % 10 % 0 % 0 % 0 %0.17 0.28 0.27 - 0.01 0.01 - - -

- 0.01 0.01 - - - - - -PBaL 64 % 72 % 87 % 11 % 7 % 0 % 0 % 0 % 0 %

0.35 0.38 0.42 0.01 0.01 - - - -0.17 0.08 0.17 - - - - - -

Table 3.2: Non-monotonicity in negotiations

the rate correspond to the maximum variation in terms of non-monotonicity occurred

which is given as a utility measure for the provider (top) and client (bottom). As we

can see, the dynamically emerging non-monotonic behaviour in static strategy settings

is not a negligible side-effect in negotiation. In 38 % of all negotiations the agents

expose non-monotonic offer curves. We can observe that in such scenarios the vari-

ation of non-monotonicity is higher in the case of oppositional applied time-dependent

tactics in the mix, such as conceder against boulware. This corresponds to our ob-

servations in Section 3.1.2 where the non-monotonicity occurred, for example, when

one party uses a boulware tactic in the mix together with an imitative tactic and the

opponent applies a conceder tactic. Accordingly, almost no non-monotonicity occurs

78

3.4. Evaluation

in scenarios with both agents applying the same concession behaviour in the mix, e.g.

conceder against conceder or boulware against boulware. It should also be noted that,

since both agents apply the imitative absolute tit-for-tat tactic in their mix the degree

of non-monotonic concession behaviour is similar when both parties apply the same

mixed strategies.

3.4.3 Scenario with Small Overlap and Equal Deadlines

In this scenario, both agents, the client and the provider, have equal deadlines chosen

from the set {20, 25, 30, 35, 40} and small overlap with Φ = 0.66. The utilities shown

in Figure 3.7 to 3.9 suggest that the traditional linear weighted combination and the

constrained version perform similar as well as the thread-based and concession-based

mixing mechanisms. This corresponds to the observations in the examples earlier (cf.

Section 3.1.3, 3.2 and 3.3) in which the monotonicity constraint did not have a large

effect on the outcome of the traditional mechanism, and the thread- and concession-

based mechanisms had similar concession curves. When agents continue negotiation in

the one-sided scenario regardless whether they detect non-monotonic concession beha-

viour of their opponent, the results are similar for the one one-sided and the two-sided

scenario. This is a surprising result, since in the one-sided scenario non-monotonicity

still occurs due to the provider always applying the traditional method, whereas in

the two-sided scenario, no non-monotonic concession curves occur since both agents

apply the monotonic mixing mechanisms. This suggests that the monotonic conces-

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

ST

0.1

0.2

0.3

0.4Uc

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

ST

0.1

0.2

0.3

0.4Up

Figure 3.7: Client (left) and provider (right) average utilities in the ’one-sided withoutwithdraw’ scenario (client uses different mixing mechanisms while provider alwaysuses traditional mixing) with small overlap and equal deadlines

79


PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

ST

0.1

0.2

0.3

0.4Uc

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

ST

0.1

0.2

0.3

0.4Up

Figure 3.8: Client (left) and provider (right) average utilities in the ’one-sided withwithdraw’ scenario with small overlap and equal deadlines

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

ST

0.1

0.2

0.3

0.4Uc

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

ST

0.1

0.2

0.3

0.4Up

Figure 3.9: Client (left) and provider (right) average utilities in the two-sided scenario(both agents use the same mixing mechanism) with small overlap and equal deadlines

sion curve produced by the thread- and concession-based mechanisms already influ-

ences the negotiation outcome. In fact, the new mixing mechanisms generate different

monotonic concession curves than the constrained one, as the latter only constraints

the concession curve of the traditional linear weighted combination and therefore ob-

tains similar results. Since both agents have an imitative tactic in their mix the mono-

tonic concession curve of the client is copied to some degree resulting in different

outcomes for the new monotonic mixing mechanism already in the one-sided scenario.

The reason for the different monotonic concession behaviour is that the traditional and

constrained mechanisms approach the time-dependent tactic towards the end of the ne-

gotiation in many settings, such that the mixed strategy reaches the reservation value

of the agent. The thread- and concession-based mechanisms, on the other hand, truly

mix both tactics with the result that the reservation value of the mixed strategy is a

mix of the reservation values of both tactics (cf. Section 3.2 and 3.3). In the case of

80

3.4. Evaluation

the imitative tactic this depends on the amount of the opponent’s concessions and the

degree of how much they are copied. In our considered scenario with absolute tit-for-

tat the reservation value (or maximum possible amount of copied concession) is in the

middle of the negotiation range and therefore lower than the reservation value of the

time-dependent tactic. This means that the mixed strategies using the thread- and

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

ST

5

10

15

20

25

30

35

tn

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

ST

20

40

60

80

100A

Figure 3.10: Average negotiation length (left) and agreement rates (right) for the ’one-sided without withdraw’ scenario with small overlap and equal deadlines

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

ST

5

10

15

20

25

30

35

tn

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

ST

20

40

60

80

100A

Figure 3.11: Average negotiation length (left) and agreement rates (right) for the ’one-sided with withdraw’ scenario with small overlap and equal deadlines

concession-based mechanisms do not reach the reservation value of the agent towards

the end in some settings, which may give the agent higher utilities in some scenarios,

but also a slightly lower rate of agreements in others (see Figure 3.10 and 3.12). How-

ever, if we assume an agent withdraws from the negotiation if it detects non-monotonic

concession behaviour, the utilities are significantly lower for all strategy groups while

the agreement rate is larger for the monotonic mixing mechanisms (see Figure 3.11).

81


PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

ST

5

10

15

20

25

30

35

tn

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

ST

20

40

60

80

100A

Figure 3.12: Average negotiation length (left) and agreement rates (right) for the two-sided scenario with small overlap and equal deadlines

Compared to the traditional mixing the monotonic mechanisms obtain higher utilities

for the client in all strategy groups. The negotiation length is slightly smaller for the

thread- and concession-based mechanisms in many strategy groups in the one-sided

scenario without withdraw and the two-sided scenario. In general, it can be observed

that the thread- and concession-based mechanisms improve the client’s utilities for the

strategy groups with conceder time-dependent tactics in the mix compared to the tra-

ditional and constrained mechanism, whereas it is the opposite for the seller. This is

similar for the two-sided scenario where the utilities are higher for the provider and

lower for the client for the boulware tactics when using the thread- and concession-

based method.

3.4.4 Scenario with Small Overlap and Different Deadlines

In this scenario, both agents, the client and the provider, have equal deadlines chosen

from the set {20, 25, 30, 35, 40} and small overlap with Φ = 0.66. As a result the

agents fail to reach an agreement in many scenarios such that utilities obtained by both

agents are very low compared to the scenarios with equal deadlines or large overlap.

This is because the zone of agreement is very small, such that the performance is only

slightly lower (cf. Figure 3.14) in the one-sided scenario with withdraw.

82

3.4. Evaluation

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

ST

0.1

0.2

0.3

0.4Uc

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

ST

0.1

0.2

0.3

0.4Up

Figure 3.13: Client (left) and provider (right) average utilities in the ’one-sided withoutwithdraw’ scenario with small overlap and different deadlines

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

ST

0.1

0.2

0.3

0.4Uc

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

ST

0.1

0.2

0.3

0.4Up

Figure 3.14: Client (left) and provider (right) average utilities in the ’one-sided withwithdraw’ scenario with small overlap and different deadlines

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

ST

0.1

0.2

0.3

0.4Uc

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

ST

0.1

0.2

0.3

0.4Up

Figure 3.15: Client (left) and provider (right) average utilities in the two-sided scenariowith small overlap and different deadlines

83


PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

ST

10

20

30

tn

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

ST

20

40

60

80

100A

Figure 3.16: Average negotiation length (left) and agreement rates (right) for the ’one-sided without withdraw’ scenario with small overlap and different deadlines

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

ST

10

20

30

tn

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

ST

20

40

60

80

100A

Figure 3.17: Average negotiation length (left) and agreement rates (right) for the ’one-sided with withdraw’ scenario with small overlap and different deadlines

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

ST

10

20

30

tn

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

ST

20

40

60

80

100A

Figure 3.18: Average negotiation length (left) and agreement rates (right) for the two-sided scenario with small overlap and different deadlines

84

3.4. Evaluation

The client can benefit in almost all scenarios and strategy groups when using the

thread- or concession-based mechanism, whereas the provider obtains higher utilities

only for conceder time-dependent tactics in the mix using the traditional or constraint-

based mixing in the one-sided without withdraw and the two-sided scenario. In addi-

tion, the thread- and concession-based mechanisms achieve a higher agreement rate in

almost all scenarios and strategy groups except for the conceder time-dependent tactics

in the one-sided scenario without withdraw.

3.4.5 Scenario with Large Overlap and Equal Deadlines

In this scenario, the different mixing mechanisms are compared when both agents have

equal deadlines chosen from the set {20, 25, 30, 35, 40} and the overlap is large with

Φ = 0.33. Similar to the setting with small overlap and equal deadlines, the one-sided

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

ST

0.1

0.2

0.3

0.4

0.5

0.6Uc

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

ST

0.1

0.2

0.3

0.4

0.5

0.6Up

Figure 3.19: Client (left) and provider (right) average utilities in the ’one-sided withoutwithdraw’ scenario with large overlap and equal deadlines

scenario without withdraw and the two-sided scenario are similar. We can also see

that the new mixing mechanisms shift utility from one agent to the other when they

apply oppositional concession behaviour in their time-dependent tactics. The client

gains in utility in the one-sided without withdraw and two-sided scenario when it ap-

plies conceder time-dependent tactics in the mix with the thread- and concession-based

mechanisms (cf. Figure 3.19 and 3.19) whereas it the opposite for the seller. Similarly,

the client looses utility when using the boulware tactics and the new mechanisms while

the provider gains utility. This corresponds to the findings form Section 3.4.2 where

the highest rate of non-monotonicity occurred for strategy groups with oppositional

85


PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

ST

0.1

0.2

0.3

0.4

0.5

0.6Uc

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

ST

0.1

0.2

0.3

0.4

0.5

0.6Up

Figure 3.20: Client (left) and provider (right) average utilities in the ’one-sided withwithdraw’ scenario with large overlap and equal deadlines

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

ST

0.1

0.2

0.3

0.4

0.5

0.6Uc

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

ST

0.1

0.2

0.3

0.4

0.5

0.6Up

Figure 3.21: Client (left) and provider (right) average utilities in the two-sided scenariowith large overlap and equal deadlines

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

ST

5

10

15

20

25

30

tn

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

ST

20

40

60

80

100A

Figure 3.22: Average negotiation length (left) and agreement rates (right) for the ’one-sided without withdraw’ scenario with large overlap and equal deadlines

concession behaviour. The utility drops considerably for both agents when they with-

draw from the negotiation after they detected non-monotonic concession behaviour

86

3.4. Evaluation

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

ST

5

10

15

20

25

30

tn

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

ST

20

40

60

80

100A

Figure 3.23: Average negotiation length (left) and agreement rates (right) for the ’one-sided with withdraw’ scenario with large overlap and equal deadlines

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

ST

5

10

15

20

25

30

tnPC

aSPC

aM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

ST

20

40

60

80

100A

Figure 3.24: Average negotiation length (left) and agreement rates (right) for the two-sided scenario with large overlap and equal deadlines

while also the rate of agreements is lower. Due to the large overlap and the equal dead-

lines the agents a full rate of agreements is reached for the one-sided without withdraw

and the two sided scenario in the case of the traditional and constraint mixing method.

In the same scenarios, the agreements rate is only slightly lower for some strategy

groups when using the thread- or concession-based mechanisms. As explained in Sec-

tion 3.4.3 this is because the thread- and concession-based mechanisms treat the pure

tactics individually that may result in an overall lower reservation value for the mixed

strategy as compared to the traditional or constrained mixing methods.

87


3.4.6 Scenario with Large Overlap and Different Deadlines

In this scenario the agents have different deadlines chosen from the set {20, 25, 30, 35, 40}with a large overlap of the negotiation intervals (Φ = 0.33). Because of the different

deadlines the agents obtain lower average utilities as compared to the scenario with

equal deadlines. Similar to the other scenarios, the one-sided setting with withdraw

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

ST

0.1

0.2

0.3

0.4

0.5

0.6Uc

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

ST

0.1

0.2

0.3

0.4

0.5

0.6Up

Figure 3.25: Client (left) and provider (right) average utilities in the ’one-sided withoutwithdraw’ scenario with large overlap and different deadlines

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

ST

0.1

0.2

0.3

0.4

0.5

0.6Uc

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

ST

0.1

0.2

0.3

0.4

0.5

0.6Up

Figure 3.26: Client (left) and provider (right) average utilities in the ’one-sided withwithdraw’ scenario with large overlap and different deadlines

obtains lower utilities for both agents. Again, the client benefits from the thread- and

concession-based mechanisms for strategy groups with conceder time-dependent tac-

tics in the mix whereas it is the opposite for the provider. It is interesting to observe that

both agents gain in utility in the one-sided setting with and without withdraw when the

client applies boulware tactics (cf. Figure 3.25 and 3.26) with the new mechanisms.

This, however, is not the case in the two-sided setting in Figure 3.27. Accordingly, the

88

3.4. Evaluation

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

ST

0.1

0.2

0.3

0.4

0.5

0.6Uc

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

ST

0.1

0.2

0.3

0.4

0.5

0.6Up

Figure 3.27: Client (left) and provider (right) average utilities in the two-sided scenariowith large overlap and different deadlines

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

ST

5

10

15

20

25

30

35

tn

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

ST

20

40

60

80

100A

Figure 3.28: Average negotiation length (left) and agreement rates (right) for the ’one-sided without withdraw’ scenario with large overlap and different deadlines

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

ST

5

10

15

20

25

30

35

tn

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

ST

20

40

60

80

100A

Figure 3.29: Average negotiation length (left) and agreement rates (right) for the ’one-sided with withdraw’ scenario with large overlap and different deadlines

agreement rates are also significantly higher for the one-sided with and without with-

draw scenarios in the case of boulware tactics and the thread- and concession-based

89


PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

ST

5

10

15

20

25

30

35

tn

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

ST

20

40

60

80

100A

Figure 3.30: Average negotiation length (left) and agreement rates (right) for the two-sided scenario with large overlap and different deadlines

mixing, but not in the two sided scenario. As mentioned in the previous sections,

the reason for this is that the reservation value of the overall mixed strategy is lower

for the thread- and concession-based mechanisms in many settings as the traditional or

constraint-based methods as their mixed strategies approach the time-dependent tactics

in the mix towards the end of the negotiation.

3.5 Related Work

Based on the prominent negotiation tactics introduced in [44] different negotiation

strategies have been proposed, which focus primarily on single families of tactics. For

example, Fatima et al [45, 46] investigate scenarios of single- and multi-issue negoti-

ation where agents have only partial information about each other trying to find optimal

strategies that most exploit the opponent. While this work focuses on the effect of time,

information states and discounting factors on the outcome and comparisons are made

to equilibrium solutions, it is limited to time-dependent tactics and does not consider

mixed strategies. Faratin et al provides evaluation results for pure, static and dynamic

mixed strategies in [44, 42] with focus on the influence of long and short term dead-

lines, and initial offers. Although the initial idea of mixing tactics by a linear weighted

combination is proposed in that work, it does not investigate the resulting offer curves

of the mixed strategies nor the non-monotonic effects. Matos et al [94] propose the

application of genetic algorithms to determine most successful mixed strategies that

evolve depending on the environment and strategy of the opponent. Both approaches

90

3.6. Summary

demonstrate that mixed strategies perform better than pure tactics in terms of gained

utility and negotiation cycles, but do not investigate the mechanism of their mixing

with respect to non-monotonic behaviour. Cardoso et al [25], and Brzostowski et al

[19, 18] consider the mixing of different tactic families to evaluate adaptive strategies

based on reinforcement learning, respectively, heuristic predictive methods or regres-

sion analysis with a focus on their negotiation outcomes only. Sierra and Ros [120]

propose to let an agent make concessions through single or mixed tactics whenever a

deadlock occurs, i.e. the opponent’s last offer does not improve the utility of the offer

two steps before, otherwise a trade-off tactic is used. However, utilities of offers may

also decrease when single tactics are combined. The work presented here is different

in that it focuses on the analysis of the mixing mechanism itself, and proposes new

mechanisms that, in contrast to the commonly used linear weighted combination of

tactics, generate monotonic sequences of offers and utilities during the process of ne-

gotiation, thereby leading to different outcomes compared to the non-monotonic offer

curves.

3.6 Summary

In this chapter we have investigated a heuristic approach for creating multi-tactic nego-

tiation strategies by mixing a set of pure tactics at each stage of the negotiation using

linear weighted combinations. This approach was chosen because of its ability to dy-

namically generate complex concession behaviour when combining different types of

simple decision functions that require the agent to have no knowledge about the other’s

decision models and preferences, and may use only limited information available in the

current encounter. We have shown that when using the traditional linear weighted com-

bination, non-monotonic sequences of offers may occur even in static mixed strategies

that involve behaviour-dependent and -independent tactics which individually generate

offers in a monotonic manner. Such non-monotonic offers curves also lead to a non-

monotonic sequence of an agent’s own utilities, which is argued to be undesirable in

automated negotiation. Furthermore, it has been shown that such non-monotonic con-

cession behaviour can occur at any time as a result of the dynamic effects of an agent

system in which the agents use mixed strategies with imitative tactics, and that the

effects can delay agreements, significantly change outcomes, and result in high sensit-

91


ive strategy parameters compared to monotonic mixing mechanisms. Accordingly, we

have proposed two new mixing mechanisms, the first based on individual negotiation

threads of each imitative tactic involved and the second using single concessions of

each tactic. Based on definitions of monotonic behaviour-dependent and -independent

tactics, it has been proven that the new mechanisms produce monotonic concession

behaviour, the negotiation thread-based mechanism for static weights, and also the

concession-based mechanism for dynamic weights. A number of examples have been

used to demonstrate the different concession behaviours of the mixing mechanisms

in this chapter, and the evaluation has also shown that the proposed mechanisms can

improve an agent’s utilities in many negotiation scenarios.

92

Chapter 4

Multistage Fuzzy Decision-Making inAutomated Negotiation

This chapter presents a novel approach for modelling negotiation strategies based on

multistage fuzzy decision-making. The process of negotiation can be considered a

multistage decision process in which two parties make decisions over multiple stages

in order to find a mutually acceptable agreement that satisfies the preferences of both.

During the decision process the agents need to consider the concession behaviour of

the opponent as well as their own preferences and conditions while the decision model,

preferences and deadlines of the agents are unknown to each other. Although the

heuristic-based multi-tactic strategies are suitable in such situations and can gener-

ate partially adaptable behaviour when mixing behaviour-dependent and -independent

tactics, it is difficult for an agent to decide not only which tactics to choose but also

to find the appropriate set of strategy parameters in order to achieve a particular be-

haviour, due to the infinite possible combinations and their sometimes unknown be-

havioural effects (see previous chapter). The pure tactics, on the other hand, typically

represent only simple functions that react to time, a particular resource or simply copy

the opponent’s behaviour to some degree, but are not able to utilize any knowledge an

agent may have about the concession behaviour of the opponent. For that reason, we

consider a multistage fuzzy decision approach for an agent’s negotiation strategy in

which the agent can use its limited knowledge, for example, in the form of reference

cases from a few past interactions, to generate a model with fuzzy state transitions

about the possible concession behaviour of the opponent. In this decision model, the

Chapter 4. Multistage Fuzzy Decision-Making in Automated Negotiation

agent’s preferences are modelled using a fuzzy goal and fuzzy constraints that impose

the agent’s soft preferences on the decision process in order to obtain a strategy that

at the same time considers the limited knowledge about the opponent’s concession be-

haviour and the agent’s preferred strategy. The representation of the offer-response

pattern in the form of the fuzzy transition model allows the agent to use dynamic pro-

gramming algorithms to find the best course of actions during the encounter.

In the subsequent sections, we first present the model for the multistage decision-

making with fuzzy state transitions, and show how an agent can use this model to

create flexible negotiation strategies using limited knowledge in the form of a few ref-

erence cases in order to negotiate competitively in negotiation environments in which

agents can expose different strategic behaviours. It is also shown that an agent can use

the fuzzy constraints to impose further soft preferences or conditions on the decision

process that then finds a course of actions which takes into account both the limited

knowledge about the concession behaviour and the agent’s soft preferences. We use

some examples to demonstrate the modelling of a negotiation strategy, and also show

the decision algorithms of an agent applying this model. In a manner similar to that

in the previous chapter, the evaluation section validates the modelling approach and

shows experimental results for different negotiation behaviours.

4.1 Model with Fuzzy State Transitions

The multistage fuzzy decision models for deterministic and stochastic systems from

Section 2.4.2 assume that the underlying model of the state transitions is known. In our

negotiation context this appears to be a strong assumption, because in many scenarios

these probabilistic state transitions might not be obtainable by an agent due to the

limited amount of available information, or the required large number of negotiations

that is needed to derive such a model. For that reason, we consider the case where

the underlying state transitions are fuzzy and represent possible state transitions of

the system when choosing a particular action. This corresponds to the viewpoint of

possibility theory [146] where the possibility degree of a particular state transition

reflects how plausible it is to attain a succeeding state given the state and action at the

current stage [124]. The fuzzy representation of the state transitions allows an agent

to use the observed concession behaviour that after fuzzification can be utilized in the

94

4.1. Model with Fuzzy State Transitions

model for an agent’s decision-making. As a result, the knowledge required for the

creation of the fuzzy transition model may be limited, for example, taken from only a

few reference cases, which is shown in the next section. In such a fuzzy system, the

state transition function is a conditional fuzzy relation with the membership function

µF (xt+1|xt, ut) (4.1)

with µf : X × U × X → [0, 1], assigning for each xt ∈ X and ut ∈ U a fuzzy

value to the consecutive state xt+1 ∈ X . A similar model has been proposed by

Kacprzyk [68] in which the transition model as well as the the states and actions are

fuzzy. However, in our negotiation context, the agents exchange crisp offers which

makes the modelling with fuzzy states and actions inapplicable. In addition, it has

been shown that in that model, due to the infinite number of possible fuzzy states and

actions during the backward iteration, finding a solution may become intractable, and

therefore requires the use of interpolation between a limited number of fuzzy states

and actions. We follow the approach where the state transitions are fuzzy but the states

and actions in the decision process are crisp. Similar to the models described in 2.4.2

we assume that the transition matrix is time-invariant and the decision process has a

finite termination time N . Since in a negotiation an agent is most interested in the

final outcome of the encounter, one fuzzy goal GN is imposed at the final stage only

whereas fuzzy constraints Ct may be imposed at each stage of the decision process

with t = 1, . . . , N − 1. This model is thus a fuzzy or possibilistic Markov decision

process in a fuzzy environment. The decision problem is derived from the stochastic

system under control described in Section 2.4.2 where the optimal course of actions

u∗0, . . . , u∗N−1 is sought that maximizes the expected fuzzy goal given the initial state

x0 and the fuzzy constraints over all stages:

µD(u∗0, . . . , u∗N−1|x0) = max

u0,...,uN−1

[µC0(u0)∧ . . .∧µCN−1(uN−1)∧EµGN (xN)]. (4.2)

However, the expected goal EµGN (xN) clearly does not follow the same notion of the

probability of attainment of a fuzzy event as in the probabilistic case. In fact, the ex-

pected goal contains the expected fuzzy values for each possible action at each state

in the current stage based on the particular decision criterion chosen by the agent. For

example, in the case of the probabilistic system in a fuzzy environment the decision

95


criterion is the probability of the attainment of a fuzzy event, whereas in a pure Markov

decision process the decision criterion is the expected value or reward of future states

and actions. A number of decision criteria have been proposed and discussed for qual-

itative frameworks, especially from the perspective of possibility theory [125, 34, 141],

among which the most common are the optimistic and pessimistic qualitative expec-

ted utility represented by possibility or necessity of a fuzzy event, respectively, which

has been shown to be counterparts to the expected utility in standard decision theory

[35]. The optimistic criterion has also been applied by Kacprzyk [68] in the above

mentioned setting with fuzzy constraints, and we will use the same criterion in the

following. The expected goal can then be expressed using the optimistic qualitative

criterion as follows:

EµGN (xN |xN−1, uN−1) = maxxN∈X

[µF (xN |xN−1, uN−1) ∧ µGN (xN)], (4.3)

This corresponds to the max-min composition shown in Section 2.4.1. However, de-

pending on the decision problem and context different decision criteria may apply,

such that other s-t norm compositions could also be used instead [68]. With the de-

cision criterion above we can now formulate the recurrence equation similar to the

stochastic system in Section 2.4.2 as follows:

EµGN−i+1(xN−i+1) = maxxN−i+1∈X

[µF (xN−i+1|xN−i, uN−i) ∧ µGN−i+1(xN−i+1)], (4.4)


[µCN−i(uN−i) ∧ EµGN−i+1(xN−i+1)] (4.5)

for i = 1, . . . , N . Since the expected goal is conditioned on states and actions at stage

N − i it represents a fuzzy relation between xN−i and uN−i giving the maximum ex-

pected possibility over next states xN−i+1, the correct notation for the expected goal is

EµGN−i+1(xN−i, uN−i). However, throughout this chapter, we also use the simplified

notation introduced by Kacprzyk [68] interchangeably. Similar to the other models the

solution is expressed in terms of a policy function u∗t = a∗t (xt) with t = 0, 1, ..., N − 1

and A∗ = {a∗0, . . . , a∗N−1} being the optimal action strategy. Based on the above recur-

rence equations a dynamic programming approach can be chosen to generate the action

policies. For more details about decision-making in qualitative and semi-qualitative

frameworks we refer to the vast amount literature including [68, 124, 125, 35, 142]. In

96

4.2. Modelling Negotiation Strategies

the next sections, we will adapt this model to the bilateral negotiation process in or-

der to find the course of counteroffers an agent proposes given the limited knowledge

about the opponent’s concession behaviour.

4.2 Modelling Negotiation Strategies

The multistage fuzzy decision model from the previous section can be mapped dir-

ectly to the decision process of an agent in bilateral negotiation. Figure 4.1 shows

the decision process where the opponent offers correspond to the states, the modelling

agent’s offers to the actions, and the conditional fuzzy relation represents the underly-

ing model the agent created on the basis its limited knowledge about the opponent’s

behaviour. The agent’s fuzzy constraints are imposed at each stage of the encounter

whereas the fuzzy goal is imposed only at the last stage, since the agent is most in-

terested in the final outcome of the negotiation. In the next sections we describe in

Figure 4.1: Multistage fuzzy decision process of a negotiation agent

more detail how an agent models the state and action space, creates the fuzzy transition

model based on a few reference cases, and applies the fuzzy goal and fuzzy constraints.

In order to avoid a conflict in the notation of the negotiation model from Section 2.2.1

and the multistage fuzzy decision model from previous section, we slightly change the

notation for the offers exchanged, where an offer from agent a to b is denoted as okia→bwith ki ∈ time representing the discrete time points where the offers are proposed

during the encounter. The negotiation thread is denoted as NT instead of X , which

97


stands for the state space in this chapter. In addition, we use the term action instead of

control in the following since the opponent can not be ’controlled’ in the sense that the

agent has incomplete knowledge over the underlying decision structure.

4.2.1 States and Actions

The state and action spaces relate to the offers exchanged during the encounter. Despite

the fact that there are various ways for modelling the state and actions, for example

using a response or imitation rate [116], the most straightforward approach is to use

the offers directly. Both spaces have to be in discrete form where the state space covers

the complete negotiation range and the action space the negotiation interval of the agent

for the issue under negotiation. The discretization method for space X is given by

S =

{(l − 1)(uB − lB)

n− 1+ lB|l = 1, . . . , n

}. (4.6)

where n is the total number of states, and uB and lB represent the upper and lower

boundary of the state space, respectively. For example, if we assume that an agent a

applies this model, and agent b makes the first proposal, then for the state space, the

boundaries uB and lB correspond to the first offers ok1b→a and ok2a→b, respectively. For

the action space, the upper boundary of agent a is given by its reservation value RV a

while the lower boundary is the first offer of agent a. For simplicity, we use the same

discretization factor for both spaces such that the cardinality m of the action space is

given in relation to the total number of states n with

m = n · |RVa − ok2a→b|

|ok1b→a − ok2a→b|

(4.7)

where a can be a buyer or seller agent. Since the bilateral negotiation model in Section

2.2.1 and the multistage fuzzy decision model use different time intervals, the sequence

of offers in the negotiation thread is mapped into a state-action form such that offers

and counteroffers at time ki and ki+1 correspond to states and actions at stage t. The

sequence of the offers exchanged during the encounter is equivalent to the trajectory

98


TR of states and actions written as

TR =(x0, u0, x1, u1, . . . , xt−1, ut−1, xt) ≡

(ok1b→a, ok2a→b, o

k3b→a, o

k4a→b, . . . , o

ki−2

b→a, oki−1

a→b, okib→a),

(4.8)

where the next offer oki+1

a→b is the action ut by agent a for the given state xt, that cor-

responds to the offer okib→a from its opponent b. Thus, the action ut (offer oki+1

a→b) is the

action sought at stage t. Since the state and action spaces are in discrete form and

offers may be proposed in a different (continuous) space, offers are mapped to states

xt and actions ut with

xt = arg minσ∈X

|okib→a − σ|

ut = arg minα∈U

|oki+1

a→b − α|,(4.9)

if agent a is using the multistage fuzzy decision model. Agent a needs at least one offer

from its opponent to make a decision according to the responsive negotiation model

in Section 2.2.1 while the course of actions determined by its policy function then

represents its negotiation strategy in response to the opponent’s strategic concession

behaviour.

4.2.2 Fuzzy State Transitions

The fuzzy transition matrix of an individual agent encodes its fuzzy knowledge about

the possible concession behaviour of its opponent and the agent’s responses that may

lead to an agreement. Depending on the source and the amount of an agent’s know-

ledge, different methods may be used to create the fuzzy transition model. For ex-

ample, an agent may use fuzzy rules that it generated through a large number of in-

teractions or received from an expert or other agents. Although various methods have

been shown for the generation of such fuzzy rules in the setting of negotiation [3],

they require a large amount of data in order to generate a sufficient rule base. In most

realistic situations in automated negotiation, however, such knowledge is simply not

available due to the open and distributed nature of the agent system. Therefore, we

assume in this thesis that the agent uses a small number of reference cases, which the

agent may take from past interactions, to obtain the fuzzy state transitions. If the agent

99


has no prior knowledge the reference cases can also reflect an agent’s preferred course

of responses to a negotiation partner’s concessions. Such reference cases reflect the

range of actions over time as a response to the proposed offers of the opponent and,

in that sense, define the dynamic negotiation strategy over the agent’s possible offers.

Using only a limited number of reference cases, we use the similarity measure to create

and update the agent’s fuzzy transition matrix during the encounter. Let NT [h] be the

negotiation thread of case h, then the thread can be transformed into the state-action

form (cf. Section 4.2.1) obtaining the trajectory TR[h]:

TR[h] = (σl0[h], αv0[h] . . . , αvN [h]−1[h], σlN [h][h]) (4.10)

where N [h] is the last stage, and σli[h] and αvi[h] are states and actions at stages

i = 1, . . . , N [h] of case h, respectively. The indices li[h] ∈ {1, . . . , n} and vi[h] ∈{1, . . . ,m} correspond to the number of the states and actions of case h at stage i. The

trajectory of all states of case h can also be written as TRX [h] = (σl0[h], . . . , σlN [h][h])

and, respectively, of all actions as TRU [h] = (αv1 , . . . , αvN [h]). As described in Section

4.1, the policy function recommends at least one action for each state in the state space.

In order to create the necessary state transitions, we therefore need to interpolate the

trajectory of each case, such that it contains all states of the state space and each state

is assigned a particular action. This implies, that also the last state σlN [h]is assigned

an action αvN [h]since it represents the agreement of case h with σlN [h]

= αvN [h]. We

choose linear interpolation, and obtain the interpolated states σli,j [h] and actions αvi,j [h]for all i = 0, . . . , N [h]− 1 with

li,j[h] =

li[h] + j for li[h] < li+1[h]

li[h]− j for li[h] > li+1[h],(4.11)

vi,j[h] =

vi[h] + bj · δ[h]e for vi[h] < vi+1[h]

vi[h]− bj · δ[h]e for vi[h] > vi+1[h],(4.12)

where j = 0, . . . , |li[h] − li+1[h]| − 1 and δi[h] is the interpolation factor for two

consecutive actions in the trajectory:

δi[h] =|vi[h]− vi+1[h]| − 1

|li[h]− li+1[h]| − 1. (4.13)

100


Index j hence depends on the number of interpolated states between two consecut-

ive states in the state trajectory TR[h]. The interpolated state and action trajectories

TRX [h] and TRU [h] can then be written as

TRX [h] = (σl0,0[h], . . . , σl0,j [h], . . . , σl1,0[h], . . . , σlN [h],0[h])

TRU [h] = (αv0,0[h], . . . , αv0,j [h], . . . , αv1,0[h], . . . , αvN [h],0[h]).(4.14)

For the state transitions, we use the similarity between the trajectory of case h and the

current behaviour of the opponent represented by the current trajectory TRX [curr] at

time t:

simt(TRX [h], TRX [curr]) =1

t+ 1

t∑i=0

1− |σli[h] − xi|(maxh∈H(σli[h])−minh∈H σli[h])

(4.15)

for i ≤ N [h] and H being the set of all cases. The similarity values provide the neces-

sary fuzzy transitions for each case in comparison to the current negotiation and are

updated at each negotiation round. If during the negotiation the current stage exceeds

the last stage from a particular case its last offer is used instead. The transition matrix

is then created based on an initially zero transition matrix µ(xt+1, xt, ut) = 0n,m,n for

all m actions and n states using the similarity values:

µ(σli+1[h]|σli,j [h], αvi,j [h]) = max[simt(TRX [h], TRX [curr]), µ(σli+1[h]|σli,j [h], αvi,j [h])](4.16)

for all i = 1, . . . , N [h]− 1.

In scenarios where only reference cases are used for the state transitions, the expected

fuzzy goal at each stage can be derived directly from all cases:

EµGi+1(xi+1|x′i[h], ui[h]) = maxh∈H

(simt(h) ∧ µGi+1(xi+1[h])), (4.17)

for i = t, t+ 1, . . . , N [h]− 1, where t is the stage in the current negotiation.

This simplifies the recalculation of the expected goal at each stage with respect to

the current similarity value. Thus, the computational effort is reduced, especially in

scenarios where the fuzzy transition matrix is sparse due to a small number of cases.

To enable inference between the expected goal and the fuzzy constraint, the actions

holding zero value in the possibility distribution over all actions need to be interpolated

101


for each state σl as follows:

EµGt+1(xt+1|σl, αv) =

EµGt+1(σl, αv) =EµGt+1(σl, αv2)− EµGt+1(σl, αv1)

v2 − v1∗ (v − v1) + αv1 ,

(4.18)

under the condition that v1 < v < v2 and EµGt+1(σl, αv1), EµGt+1(σl, αv2) > 0 with

v, v1, v2 ∈ {1, . . . ,m}. If, however, the boundary actions α1 or αm are zero, we can re-

place them by very small values greater zero before applying the interpolation method

to obtain a non-zero possibility distribution over the whole actions space. The ra-

tionale behind is that a limited number of cases may be sufficient to propose an agent’s

response. Since the expected goal then holds values for all states and actions after

the interpolation, the model can also propose actions not covered by any of the refer-

ence cases. Therefore, this approach provides a great flexibility towards the creation

of adaptive negotiation strategies. As mentioned previously, the above method is used

when the agent has only a few reference cases at its disposal. Although also a large

number of cases can be used, other methods may be more efficient for the creation and

the update of the transition matrix if the agent has a larger amount of the pre-existing

knowledge or beliefs about the opponent, for example, in the form of a set of fuzzy

rules. In the next sections, the fuzzy goal and the fuzzy constraints are discussed as

they represent the preferences of an agent over its opponent’s offers (states) and its

own offers (actions), respectively, and constitute the means of an agent to direct the

decision process.

4.2.3 Fuzzy Goal

The negotiation agent uses the fuzzy goal to specify its preferences over all states

in the state space. The degree of membership in the fuzzy goal increases for states

closer to the initial value of the agent as they are more preferable to states close to the

initial offer of the opponent. The fuzzy goal is therefore similar to the utility function

of an agent in that it orders the possible outcomes of the negotiation by assigning

values from the interval [0, 1] to the possible offers in the negotiation space. However,

while the utility function orders all offers in the negotiation interval of the agent and

assign zero to all other offers (outside the interval), the membership degrees of the

fuzzy goal have to be non-zero for all states in the negotiation range, except the initial

102


state, as otherwise a state might never be reached as an intermediate or final state

during the backward iteration of the decision process. The fuzzy goal always covers

the whole discretized negotiation range, i.e. the range between the agent’s first offer

proposals. This difference between the utility function and the fuzzy goal is therefore

only important if the agents negotiation interval does not cover the whole negotiation

range, i.e. there is only a partial overlap of the agents intervals. Figure 4.2 shows an

example for a fuzzy goal and a utility function in the case where the agents negotiation

interval ([10, 25]) overlaps only partially with the other agents interval ([15, 30]). If the

agent’s negotiation interval, however, fully overlaps the negotiation range, its utility

function can be mapped directly to the fuzzy goal.

10 15 20 25 30xN0.0

0.2

0.4

0.6

0.8

1.0

ΜGN HxNL

15 20 25 30x0.0

0.2

0.4

0.6

0.8

1.0Uc

Figure 4.2: Example fuzzy goal (left) and utility function (right) for a partial overlapof negotiation intervals

4.2.4 Fuzzy Constraints

Whilst the fuzzy goal represents a preference over the states an agent can use the

fuzzy constraints to impose a preference over its actions at each stage of the decision

process. In this sense, the fuzzy constraints constitute a means to influence and direct

the decision process based on an agent soft preferences or any other factors, such as

a particular resource in the agent’s environment. Since time is an important factor in

negotiation, it is straightforward to represent the fuzzy constraints in a time-dependent

form (as shown in Section 4.1) where to each stage of the encounter a particular fuzzy

constraint is assigned. The fuzzy constraints constitute a time-dependent preference

over the range of possible offers of the modelling agent. For simplicity, we use the

103


triangle membership function in the following

µCt(x, a, b, c) =

0 x < a, x > c

x−ab−a a ≤ x ≤ b

c−xc−b b < x ≤ c.

(4.19)

However, any other type of membership function may be used instead. The fuzzy

constraints (an example is shown in Figure 4.3) does not need to cover the whole

action space because inference with the non-zero expected fuzzy goal always results in

a non-zero fuzzy set. The effect of a fuzzy constraint can vary depending on the shape

and the support of its fuzzy set. In general, the larger the support and area of the fuzzy

constraint the stronger the influence of the cases on the actions and vice versa. For

simplicity and easy specification, constraints are typically normalized. However, in a

10 12 14 16 18 20 22 24ut0.0

0.2

0.4

0.6

0.8

1.0

ΜCt HutL

Figure 4.3: Example fuzzy constraint

negotiation, the other agent may choose states close to its initial offer in the beginning

with small membership degrees in the fuzzy goal, such that membership degrees for

all state-action pairs in the expected goal relation (Eq. (4.3)) become also small. As a

result, constraints may have a low effect on the actions as they are normalized and may

completely overlay the expected goal distribution over the actions for a particular state.

The influence is increased by scaling the fuzzy constraints down, e.g. to the maximum

of the expected goal, before it is applied (cf. Eq. (4.4)):

µCt(α) = µCt(α) · maxα∈U,σ∈X

(EµGt+1(σ, α)) (4.20)

104


for all α ∈ U . This method ensures a high effect of the individual constraints on the

transition matrix and therefore on the cases during the encounter. The scaling factor

for the constraints depends on the preference of the agent and can be different from the

one shown above.

4.2.5 Modelling Different Negotiation Strategies with FuzzyConstraints

The possibility to impose time-dependent fuzzy constraints on the actions enables an

agent to apply its own soft preferences or conditions on the decision-making process.

In that sense, the generated course of actions during the backward induction considers

both the limited knowledge about the opponent’s concession behaviour in the fuzzy

transition relation and the agent’s own soft strategy. An agent can therefore use any

strategy or conditions based on time as, for example, the time-dependent decision func-

tions in Section 2.3.1.1 where µCt(α(t), a, b, c) represents the membership function

with t = 0, 1, . . . , N − 1 and α(t) corresponds to one of the decision functions. In

addition, the height and support of the fuzzy constraints allow an agent to determine

in what range and to what degree they influence the decision-making. For example, if

an agent needs to make sure that towards the end of the negotiation its own strategy

is prevalent, e.g. in order to approach the reservation value, the support of the fuzzy

constraints may be decreasing. Examples for such time-dependent soft strategies for

the boulware and conceder polynomial function are shown in Figure 4.4a and 4.4b,

and an example for fuzzy constraints with a decreasing support in Figure 4.4c. In the

case where the agent uses the reference cases only without any fuzzy constraints the

decision process may propose actions which, although they corresponds to the offer-

response patterns in the fuzzy transition relation, do not resemble the offers of the

reference cases at that particular stage of the encounter. This is because the fuzzy

transition matrix contains the knowledge about the relationship between the possible

concession behaviour of the opponent and the agent, but without the information at

what stage in the encounter the concession is made. In such situations, an agent can

use the reference cases to create the fuzzy constraints over time in order to ensure

concession behaviour that is more similar to the reference cases. Figure 4.4d shows

such an example in which the actions of two cases are used for generation of the fuzzy

105


0 5 10 15 20 25 30t10

15

20

25ut

(a) Boulware polynomial fuzzy constraints0 5 10 15 20 25 30

t10

15

20

25ut

(b) Conceder polynomial fuzzy constraints

0 5 10 15 20 25 30t10

15

20

25ut

(c) Fuzzy constraints with decreasing support0 5 10 15 20 25 30

t10

15

20

25ut

(d) Fuzzy constraints based on example cases

Figure 4.4: Examples for different time-dependent fuzzy constraints

constraints. The similarity of the two cases to the current encounter is then used to

determine the hight of the individual fuzzy constraints for each case.

4.3 Decision Algorithms

The question remains of how an individual agent applies the multistage fuzzy decision

model to specify its negotiation strategy. In the following, we provide the decision

algorithm for a client agent c negotiating with a provider agent p assuming that p

proposes the first offer. Algorithm 1 details the communication mechanism with the

opponent in terms of the offer exchange during negotiation (lines 6 to 28) according

to Section 2.2.1 after the agent created its state and action space and transformed the

reference cases into the respective form (cf. Section 4.2.1 and 4.2.2) using the first

offers of both parties (lines 1 to 5). For simplicity, the agent uses the number of ne-

gotiation rounds to specify its negotiation deadline (tbmax) for its withdrawal instead of

106

4.3. Decision Algorithms

using a real time measure. A negotiation round consists of one offer proposal of both

agents and thus corresponds to one stage in the multistage fuzzy decision process.

However, the agent may abort the negotiation after a timeout period (line 9), where it

receives no response after a predefined threshold time. This timeout period naturally

depends on the conditions and preferences of the system and the agent. Algorithm

Algorithm 1 Decision algorithm of the client agent c1: Exchange first offers ok1p→c (= x0) and ok2c→p (= u0)2: Create Action and State Space from x0 and u0 . cf. Eq.3: for all reference cases h ∈ H do4: TR[h]← Transform TR[h] into state-action form5: end for6: end→ False7: t← 18: while end 6= True do9: if t > tcmax or timeout then

10: Withdraw from negotiation11: end← True12: else13: xt ← Next offer from opponent s14: if s accepts last offer ut−1 of agent b then15: end← True16: else17: a∗ ← GETPOLICY(t, TRX [curr])18: u∗t ← a∗t (xt)19: if U c(xt) ≥ U c(u∗t ) then20: Accept last offer xt of s21: end← True22: else23: Propose counteroffer u∗t24: end if25: end if26: end if27: t← t+ 128: end while29: end algorithm

2 details how the multistage fuzzy decision model is applied to obtain action policies

throughout all stages of the negotiation encounter for the agent. It represents the fuzzy

dynamic programming method including the creation of the expected goal matrix from

107


Algorithm 2 Get action policy at stage t1: procedure GETPOLICY(t, TRX [curr])2: EµGN (xN−1, uN−1)← 0n,m3: k ← N4: while k > t do5: k = k − 16: for all cases h ∈ H do7: sim[h]← simt(TRX [h], TRX [curr])8: for all i = 1, . . . N [h]− 1 do9: g ← min(sim[h], µGk+1(σli+1[h])

10: for all j = 0, . . . , |li[h]− li+1[h]| − 1 do11: EµGk+1(σli,j [h], αvi,j [h])← . . .12: . . .max(EµGk+1(σli,j [h], αvi,j [h]), g)13: end for14: end for15: end for16: µCk(uk)← µCk(uk) · . . .17: . . .maxα∈U,σ∈X(EµGk+1(σ, α))18: for all l = 1, . . . , n do19: Interpolate EµGk+1(σl, uk)20: µσl(uk)← . . .21: . . .minα∈U(µCk(α), EµGk+1(σl, α))22: µGk(σl)← maxα∈U(µσl(α))23: a∗k(σl)← arg maxα∈U(µσl(α))24: end for25: end while26: a∗ ← {a∗k, . . . , a∗N−1}27: Return a∗28: end procedure

the reference cases (line 2 to 15) and its interpolation. It should be noted that the linear

interpolation to transform the reference cases into the state-action form (line 4, Al-

gorithm 1) and in the expected goal matrix (line 19, Algorithm 2) is straightforward

(cf. 4.2.2) and therefore not detailed here. Similar to the traditional dynamic program-

ming algorithms the complexity of finding the next offer at each round is O(n2). For a

detailed discussion about the complexity in terms of storage (space) and time of fuzzy

dynamic programming we refer to [41].

It should be noted that, as described in Section 2.3, in multi-issue negotiations the de-

cision strategies for the concession-making of an agent can be applied either for each

108

4.4. Negotiation Examples

issue individually or along the indifference curves according to the agent’s utility func-

tion in combination with a trade-off mechanism. In such cases, the multistage fuzzy

decision approach can be used for the concession-making of an agent. For simplicity,

we focus in the following on a single issue, and illustrate some example negotiations

with an agent using the multistage fuzzy approach in the next section.

4.4 Negotiation Examples

In this section, we discuss some examples where an agent uses the multistage fuzzy

decision approach with different fuzzy constraint setting to propose offers in a negoti-

ation encounter.

4.4.1 Agent using Reference Cases

First, the modelling agent uses two reference cases only to generate its fuzzy state

transitions. The cases, shown Figure 4.6a, need to be transformed into the state-action

form as described in Section 4.2.2 based on the discrete state and action space of the

agent. For example, assuming that the client and the provider have the following inter-

vals: minc = 10, maxc = 25 and minp = 15, maxp = 30, and the provider proposes

the first offer with ok1p→c = 30, the agent discretizes the negotiation range based on the

discretization factor (e.g. 0.25), such thatX = {10, 10.25, 10.5, . . . , 29.75, 30} and the

U = {10, 10.25, . . . , 24.75, 25}. The state-action form of a case then requires that each

case covers the whole state space. For example, for two consecutive offers okip→c = 29.8

and oki+2p→c = 28.2 and counteroffers oki+1

c→p = 10.3 and oki+3c→p = 11.9 of case h the state

and action trajectories would be TRX [h] = {. . . , 29.75, 29.5, . . . , 28.5, 28.25, . . .} and

TRU [h] = {. . . , 10.25, 10.5, . . . , 11.75, 12, . . .}, respectively. Each state-action pair is

assigned the next state from the original case. The cases in the state-action form and

their similarity to the current encounter is then used to generate the transition matrix,

or more simply, the expected fuzzy goal at each stage directly. For example, assume

for a particular current state σl that the next states, according to the cases, are σl1 and

σl2 for the actions αv1 and αv2 , respectively. Further assume that the fuzzy goal for

the two states is µGN (σl1) = 0.8 and µGN (σl2) = 0.6. Now, if the similarity of the

cases is simt(1) = 0.9 and simt(2) = 0.5, the expected fuzzy goal for state σl and the

109


12 14 16 18 20 22 24ut

0.2

0.4

0.6

0.8

1.0

Μ

(a) Example fuzzy set for one state in the ex-pected fuzzy goal based on the similarity oftwo cases

12 14 16 18 20 22 24ut

0.2

0.4

0.6

0.8

1.0

Μ

(b) Inference of normalized fuzzy case con-straints and the expected fuzzy goal (shown forone state)

Figure 4.5: Inference example for expected fuzzy goal and fuzzy case constraints

actions αv1 and αv2 is, according to (4.17), EµGN (xt+1|σl, αv1) = maxh∈H(simt(h)∧µGN (xt+1[h])) = 0.9 ∧ 0.8 = 0.8 and EµGN (xt+1|σl, αv2) = 0.5 ∧ 0.6 = 0.5. The

two singletons for state σl are then used to interpolate the distribution in the expected

fuzzy goal while the inference of the similarity with the fuzzy goal is carried out for

each state and all reference cases. Figure 4.5a shows such an example fuzzy set in

the expected fuzzy goal. In the situation, where the agent uses no fuzzy constraints

the similarity and the fuzzy goal determine which actions are chosen, so that the con-

cession behaviour of the agent is similar to either of the cases. This is illustrated in

Figure 4.6b and 4.6c. In these examples, the agent uses the two cases from Figure

4.6a to generate its course of actions. As we can see, in both examples, the outcome

is close to the outcome of the cases. Since in our model the fuzzy transition relation

does not contain information about the stage in which a concession is made, the agent

can use the original reference cases to generate fuzzy constraints (as described in Sec-

tion 4.2.5). Figure 4.5b shows the example where fuzzy case constraints are inferred

with the expected fuzzy goal for a particular state. As a result, the chosen actions are

closer to the actual actions in the same stage in the reference cases (Figure 4.6d). The

inference between the expected fuzzy goal and the fuzzy constraints, however, may

achieve different results depending on the height and support of the fuzzy sets. Figure

4.6d shows two action curves for different normalization levels of the fuzzy constraints

(for the lower curve the constraints have been normalized to the height of the expected

fuzzy goal). In addition, because during the backward induction the values in the fuzzy

goal decrease from stage to stage, it needs to be normalized, as otherwise the simil-

110


5 10 15 20 25 30t

15

20

25

30x

(a) Reference cases of client agent5 10 15 20 25 30

t

15

20

25

30x

(b) Multistage fuzzy strategy without fuzzycase constraints (bottom) when provider usespolynomial time-dependent tactic with β = 2(top)

5 10 15 20 25 30t

15

20

25

30x

(c) Multistage fuzzy strategy w/o fuzzy caseconstraints (bottom) when provider uses poly-nomial time-dependent tactic with β = 0.4(top)

5 10 15 20 25 30t

15

20

25

30x

(d) Multistage fuzzy strategy with fuzzy caseconstraints and different normalization levels(bottom) when provider uses polynomial time-dependent tactic with β = 0.4 (top)

Figure 4.6: Example offer curves for an agent using two reference cases and caseconstraints

arity of the cases and the fuzzy constraints are less effective. However, as mentioned

in [7] such normalization problems are not trivial as they can strongly influence the

calculated course of actions and therefore the negotiation strategy. The next section

will further demonstrate the effect of the fuzzy constraints.

4.4.2 Agent using Preferred Strategy

In addition to the reference cases the agent may also have a preferred soft strategy

over the course of actions. Using the same settings and reference cases from the pre-

vious section we show examples where the modelling agent uses the time-dependent

111


12 14 16 18 20 22 24ut

0.2

0.4

0.6

0.8

1.0

Μ

Figure 4.7: Inference example for the expected fuzzy goal and fuzzy constraints of apreferred strategy

5 10 15 20 25 30t

15

20

25

30x

(a) Multistage fuzzy strategy (bottom) withconceder fuzzy constraints (polynomial withβ = 2) and the provider using polynomialtime-dependent tactic with β = 2 (top)

5 10 15 20 25 30t

15

20

25

30x

(b) Multistage fuzzy strategy (bottom) withconceder fuzzy constraints (polynomial withβ = 2) and the provider using polynomialtime-dependent tactic with β = 0.4 (top)

5 10 15 20 25 30t

15

20

25

30x

(c) Multistage fuzzy strategy (bottom) withboulware fuzzy constraints (polynomial withβ = 0.4) and the provider using polynomialtime-dependent tactic with β = 2 (top)

5 10 15 20 25 30t

15

20

25

30x

(d) Multistage fuzzy strategy (bottom) withboulware fuzzy constraints (polynomial withβ = 0.4) and the provider using polynomialtime-dependent tactic with β = 0.4 (top)

Figure 4.8: Example offer curves for an agent using two reference cases and time-dependent fuzzy constraints

112


decision functions to create its fuzzy constraints as shown in Section 4.2.5. An infer-

ence example for the expected fuzzy goal with the time-dependent fuzzy constraints

is shown in Figure 4.7. Again, the agent can adjust the effect of the constraints via

the support and the height of the constraints. Figure 4.8 illustrate how conceder and

boulware fuzzy constraints change the offer proposals towards larger or smaller con-

cessions. Whereas the conceder constraints in Figure 4.8a and 4.8b show little effect,

as the reference cases are close to the constraints, the boulware fuzzy constraints at-

tempt to ’pull’ the offer curves towards the client (Figure 4.8c and 4.8d). Depending

on the aim of the agent, i.e. whether reaching an agreement quickly or negotiating

more competitively in order to gain higher outcome utilities, an agent can apply its

soft strategy to direct the encoded concession behaviour in the fuzzy transition relation

towards its own preferences.

4.4.3 Both Agents using Multistage Fuzzy Decision-Making

Another interesting question is how are the offer curves when both agents use the

multistage fuzzy decision approach, either with the same set or different sets of refer-

ence cases. Figure 4.9a shows an example in which the two agents use the same set

of cases from the previous section. The provider makes the first proposal and chooses

the offer from the case which potentially obtains the highest utility. Once the pro-

vider chose that boulware case, the client recognizes the similarity with that case and

chooses an action accordingly. As a result, both agents propose offers along that case.

In Figure 4.9b both agents have the same sets of cases but use boulware fuzzy con-

straints (polynomial with β = 0.4). Although both use the same fuzzy constraints the

outcome is still to the advantage of the provider due to the boulware reference case.

Figure 4.9c shows an example in which the provider has a different set of cases. As the

sets of cases of both agents are almost symmetrical, the outcome of the negotiation is

close to the middle point of the negotiation range (in this case the Nash point). This is

illustrated in Figure 4.9d. In general, however, the outcome depends on the reference

cases of the agents and their set of fuzzy constraints.

113


5 10 15 20 25 30t

15

20

25

30x

(a) Offers curves when both agents use thesame reference cases

5 10 15 20 25 30t

15

20

25

30x

(b) Offer curves when both agents use thesame reference cases and boulware fuzzy con-straints

5 10 15 20 25 30t

15

20

25

30x

(c) Two reference cases used by the provider5 10 15 20 25 30

t

15

20

25

30x

(d) Offers curves when agents use differentsets of reference cases

Figure 4.9: Example offer curves when both agents use the multistage fuzzy decisionapproach

4.5 Evaluation

This section evaluates the negotiation strategies generated by the model of multistage

fuzzy decision-making of this chapter.

4.5.1 Experiment Settings

Similar to the evaluation in Chapter 3, we consider single-issue negotiations between

client c and provider p in order to compare the performance of an agent using the

multistage fuzzy decision approach with an agent using mixed strategies. Because

the two decision-making approaches use entirely different techniques and settings, we

compare the multistage fuzzy strategy using two example reference cases with the av-

114

4.5. Evaluation

erage mixed strategy using static weights based on either the traditional linear weighted

combination or the concession-based mechanism from Section 3.3, while the opponent

applies different types of mixed strategies. In a similar manner, the mixed strategies

are created using one behaviour-dependent and one behaviour-independent tactic. For

the set of mixed strategies ST we choose the following tactics based on the settings

detailed in Section 2.6.2:

ST = {PC, PL, PB,EC,EL,EB} × {a, r} × {S,M,L} (4.21)

Playing the average mixed strategy is similar to playing all strategy groups from the

set ST . In order to ensure that the strategies are tested in a wide range of different

negotiation settings, the client agents with the multistage fuzzy strategy and the aver-

age mixed strategy play against providers with a particular strategy group of the set

ST . The strategies are hence compared for each strategy group in ST . Due to the

different concession behaviours of mixed strategies using different mixing mechan-

isms, the traditional and concession-based mixing methods are chosen for comparison

against the multitsage fuzzy strategy. For simplicity and easy analysis, the agent with

the multistage fuzzy strategy uses the same reference cases from the examples in Sec-

tion 4.4.1. The cases are shown again in Figure 4.10. In addition, we are interested in

how the performance changes when the multistage fuzzy agent applies boulware fuzzy

constraints in order to improve its utility gain by negotiating more competitively. The

boulware fuzzy constraints are created using the polynomial decision function with

boulware β settings as shown in Section 2.6.2. The performance of the strategies is

measured using the average intrinsic utility U c of the client agent and the agreement

rate A (in %). Similar to the evaluation section in Chapter 3 the agents employ again

the linear utility functions allowing the direct measurement of negotiation outcomes in

the negotiation interval, and we use bar chart diagrams to illustrate the performance

of the strategies (with the small dotted bars on top representing the standard devi-

ation). The dark bars correspond to the multistage fuzzy strategy and the light bars to

the average mixed strategy. As described in Section 2.6.2 we focus on scenarios with

more realistic settings in which agents have only partial overlap of their negotiation

intervals with Φ ∈ {0.33, 0.66}. As the agents typically do not know their opponents

deadlines as it part of their preferences, but an agent system may also have a system-

specific deadline for their negotiation interactions, we distinguish between scenarios

115


5 10 15 20 25 30t

15

20

25

30x

Figure 4.10: Example cases for the multistage fuzzy strategy

with equal or different deadlines. The considered types of negotiation scenarios are:

• Small overlap and equal deadlines

• Small overlap and different deadlines

• Large overlap and equal deadlines

• Large overlap and different deadlines

with the negotiation environment settings as follows:

• Client: tcmax = 30, minc ∈ {10}, maxc ∈ {25}

• Provider: tpmax ∈ {20, 25, 30, 35, 40}, minp ∈ {10 + 25Φ|Φ ∈ {0.33, 0.66}},maxp ∈ {minp + 15}

The following sections present the experimental results and their discussion.

4.5.2 Scenario with Small Overlap and Equal Deadlines

In this scenario, the agents have equal deadlines and intervals overlap only to a small

degree with the settings from the previous section. We compare the multistage fuzzy

strategy using the example cases shown in the previous section with the average mixed

strategy generated by either the traditional linear weighted combination (Figure 4.11a)

or the concession-based mechanism (Figure 4.11b).

116

4.5. Evaluation

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

PCrS

PCrM

PCrL

PLrS

PLrM

PLrL

PBrS

PBrM

PBrL

ST

0.1

0.2

0.3

0.4Uc

ECaS

ECaM

ECaL

ELaS

ELaM

ELaL

EBaS

EBaM

EBaL

ECrS

ECrM

ECrL

ELrS

ELrM

ELrL

EBrS

EBrM

EBrL

ST

0.1

0.2

0.3

0.4Uc

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

PCrS

PCrM

PCrL

PLrS

PLrM

PLrL

PBrS

PBrM

PBrL

ST

20

40

60

80

100A

ECaS

ECaM

ECaL

ELaS

ELaM

ELaL

EBaS

EBaM

EBaL

ECrS

ECrM

ECrL

ELrS

ELrM

ELrL

EBrS

EBrM

EBrL

ST

20

40

60

80

100A

(a) Average utility (top) and agreement rate when compared with the traditional mixed strategy

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

PCrS

PCrM

PCrL

PLrS

PLrM

PLrL

PBrS

PBrM

PBrL

ST

0.1

0.2

0.3

0.4Uc

ECaS

ECaM

ECaL

ELaS

ELaM

ELaL

EBaS

EBaM

EBaL

ECrS

ECrM

ECrL

ELrS

ELrM

ELrL

EBrS

EBrM

EBrL

ST

0.1

0.2

0.3

0.4Uc

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

PCrS

PCrM

PCrL

PLrS

PLrM

PLrL

PBrS

PBrM

PBrL

ST

20

40

60

80

100A

ECaS

ECaM

ECaL

ELaS

ELaM

ELaL

EBaS

EBaM

EBaL

ECrS

ECrM

ECrL

ELrS

ELrM

ELrL

EBrS

EBrM

EBrL

ST

20

40

60

80

100A

(b) Average utility (top) and agreement rate when compared with the concession-based mixed strategy

Figure 4.11: Results for the multistage fuzzy strategy and the average mixed strategiesin the scenario with small overlap and equal deadlines

117


In addition, Figure 4.12 shows the results when the agent with the multistage fuzzy

strategy imposes the boulware fuzzy constraints on its decision strategy. Figure 4.11

shows that the multistage fuzzy strategy performs better than the average mixed strategies

based on the traditional linear weighted combination or the concession-based mechan-

ism in most scenarios, except in cases where the opponent applies exponential boul-

ware tactics in its mix. The reason for this is that the exponential decision function used

with boulware settings proposes concessions towards the end of the encounter, and

when used in a mixed strategy with the traditional linear weighted combination may

cause a non-monotonic concession curve. Figure 4.11b shows the difference when the

monotonic concession-based mechanism is used. In this setting, the multistage fuzzy

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

PCrS

PCrM

PCrL

PLrS

PLrM

PLrL

PBrS

PBrM

PBrL

ST

0.1

0.2

0.3

0.4Uc

ECaS

ECaM

ECaL

ELaS

ELaM

ELaL

EBaS

EBaM

EBaL

ECrS

ECrM

ECrL

ELrS

ELrM

ELrL

EBrS

EBrM

EBrL

ST

0.1

0.2

0.3

0.4Uc

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

PCrS

PCrM

PCrL

PLrS

PLrM

PLrL

PBrS

PBrM

PBrL

ST

20

40

60

80

100A

ECaS

ECaM

ECaL

ELaS

ELaM

ELaL

EBaS

EBaM

EBaL

ECrS

ECrM

ECrL

ELrS

ELrM

ELrL

EBrS

EBrM

EBrL

ST

20

40

60

80

100A

Figure 4.12: Average utility (top) and agreement rate of the multistage fuzzy strategyusing boulware fuzzy constraints and the average mixed strategy using the traditionalmechanism in the scenario with small overlap and equal deadlines

strategy obtains the maximum agreement rate in all cases, whereas the utility is lower

only compared to some of the boulware time-dependent tactics in the mix. When the

agent imposes the boulware time-dependent fuzzy constraints the average utility as

well as the agreement rates for the exponential boulware tactics are improved (Figure

4.12), resulting in a better performance in all strategy groups. This result is intuitive,

as, due to the equal deadlines of both agents, the smaller concessions proposed by the

118

4.5. Evaluation

multistage fuzzy strategy with the boulware fuzzy constraints achieve higher utilities

without sacrificing the agreement rate.

4.5.3 Scenario with Small Overlap and Different Deadlines

In this scenario we compare the multistage fuzzy strategy with the average mixed

strategies using the traditional linear weighted combination and the concession-based

mechanism when the agents have different deadlines and only a small overlap of the

negotiation intervals. The Figures 4.13 and 4.14 demonstrate that the multistage fuzzy

strategy does not perform as well as compared to the scenario with equal deadlines. Al-

though the average utility and the agreement rate are still good compared to the average

mixed strategy using the traditional method in cases where the opponent uses conceder

or linear tactics in the mix, it is the opposite for the other strategy groups. The reason

is that if both agent have different deadlines, the boulware time-dependent tactics may

miss the zone of agreement even though this zone exists. In addition, even if the op-

ponent makes concessions similar to a case captured by the multistage fuzzy agent

but with a different negotiation deadline, it results in a different negotiation behaviour.

That emphasizes the fact that time is an important factor in automated negotiation. A

case captured by the multistage fuzzy agent not only represents the relation between

the concession behaviour of the two agents, but also when these concessions were

made in the process of the negotiation. Because the agent has no knowledge about

the opponent’s deadline it can only assume that the deadline is at least as long as the

time when the agreement was reached in that particular case. However, the individual

concessions and the agreement point is a result of the dynamic concession behaviour

of both agents, such that the deadline may indeed be different from the one assumed by

the multistage fuzzy agent. As a result, when both agents have different deadlines the

utility and agreement rate may indeed be lower as compared to the setting with equal

deadlines. This effect is also emphasized in Figure 4.14 where the multistage fuzzy

agent uses the boulware fuzzy constraints. As they result in smaller concessions, the

agent misses a significant amount of agreements. This is also reflected in the average

utility which decreases with a larger number of failed agreements.

119


PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

PCrS

PCrM

PCrL

PLrS

PLrM

PLrL

PBrS

PBrM

PBrL

ST

0.1

0.2

0.3

0.4Uc

ECaS

ECaM

ECaL

ELaS

ELaM

ELaL

EBaS

EBaM

EBaL

ECrS

ECrM

ECrL

ELrS

ELrM

ELrL

EBrS

EBrM

EBrL

ST

0.1

0.2

0.3

0.4Uc

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

PCrS

PCrM

PCrL

PLrS

PLrM

PLrL

PBrS

PBrM

PBrL

ST

20

40

60

80

100A

ECaS

ECaM

ECaL

ELaS

ELaM

ELaL

EBaS

EBaM

EBaL

ECrS

ECrM

ECrL

ELrS

ELrM

ELrL

EBrS

EBrM

EBrL

ST

20

40

60

80

100A


PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

PCrS

PCrM

PCrL

PLrS

PLrM

PLrL

PBrS

PBrM

PBrL

ST

0.1

0.2

0.3

0.4Uc

ECaS

ECaM

ECaL

ELaS

ELaM

ELaL

EBaS

EBaM

EBaL

ECrS

ECrM

ECrL

ELrS

ELrM

ELrL

EBrS

EBrM

EBrL

ST

0.1

0.2

0.3

0.4Uc

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

PCrS

PCrM

PCrL

PLrS

PLrM

PLrL

PBrS

PBrM

PBrL

ST

20

40

60

80

100A

ECaS

ECaM

ECaL

ELaS

ELaM

ELaL

EBaS

EBaM

EBaL

ECrS

ECrM

ECrL

ELrS

ELrM

ELrL

EBrS

EBrM

EBrL

ST

20

40

60

80

100A


Figure 4.13: Results for the multistage fuzzy strategy and the average mixed strategiesin the scenario with small overlap and different deadlines

120

4.5. Evaluation

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

PCrS

PCrM

PCrL

PLrS

PLrM

PLrL

PBrS

PBrM

PBrL

ST

0.1

0.2

0.3

0.4Uc

ECaS

ECaM

ECaL

ELaS

ELaM

ELaL

EBaS

EBaM

EBaL

ECrS

ECrM

ECrL

ELrS

ELrM

ELrL

EBrS

EBrM

EBrL

ST

0.1

0.2

0.3

0.4Uc

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

PCrS

PCrM

PCrL

PLrS

PLrM

PLrL

PBrS

PBrM

PBrL

ST

20

40

60

80

100A

ECaS

ECaM

ECaL

ELaS

ELaM

ELaL

EBaS

EBaM

EBaL

ECrS

ECrM

ECrL

ELrS

ELrM

ELrL

EBrS

EBrM

EBrL

ST

20

40

60

80

100A

Figure 4.14: Average utility (top) and agreement rate of the multistage fuzzy strategyusing boulware fuzzy constraints and the average mixed strategy using the traditionalmechanism in the scenario with small overlap and different deadlines

4.5.4 Scenario with Large Overlap and Equal Deadlines

In this scenario, the two agents have equal deadlines and negotiation intervals overlap

to a large degree. The multistage fuzzy strategy obtains full rate of agreements in all

strategy scenarios with high average utilities (Figures 4.15a, 4.15b, and Figure 4.16).

This result is not surprising as the the large overlap increases the size of the agreement

zone and therefore makes it easier for an agent to negotiate competitively while still

having a high chance of reaching an agreement. Only in the case where the opponent

makes large concessions, i.e. applies conceder time-dependent tactics in the mix, the

multistage fuzzy strategy has a lower average utility compared to the average mixed

strategy in Figure 4.15, as it also makes relatively large concessions here. However,

when the multistage fuzzy agent applies the boulware fuzzy constraints, the average

utility is improved to a large degree. Again, because of the equal deadlines and the

large agreement zone the multistage fuzzy agent gains utility in all strategy scenarios

when applying the fuzzy constraints.

121


PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

PCrS

PCrM

PCrL

PLrS

PLrM

PLrL

PBrS

PBrM

PBrL

ST

0.1

0.2

0.3

0.4

0.5

0.6Uc

ECaS

ECaM

ECaL

ELaS

ELaM

ELaL

EBaS

EBaM

EBaL

ECrS

ECrM

ECrL

ELrS

ELrM

ELrL

EBrS

EBrM

EBrL

ST

0.1

0.2

0.3

0.4

0.5

0.6Uc

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

PCrS

PCrM

PCrL

PLrS

PLrM

PLrL

PBrS

PBrM

PBrL

ST

20

40

60

80

100A

ECaS

ECaM

ECaL

ELaS

ELaM

ELaL

EBaS

EBaM

EBaL

ECrS

ECrM

ECrL

ELrS

ELrM

ELrL

EBrS

EBrM

EBrL

ST

20

40

60

80

100A


PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

PCrS

PCrM

PCrL

PLrS

PLrM

PLrL

PBrS

PBrM

PBrL

ST

0.1

0.2

0.3

0.4

0.5

0.6Uc

ECaS

ECaM

ECaL

ELaS

ELaM

ELaL

EBaS

EBaM

EBaL

ECrS

ECrM

ECrL

ELrS

ELrM

ELrL

EBrS

EBrM

EBrL

ST

0.1

0.2

0.3

0.4

0.5

0.6Uc

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

PCrS

PCrM

PCrL

PLrS

PLrM

PLrL

PBrS

PBrM

PBrL

ST

20

40

60

80

100A

ECaS

ECaM

ECaL

ELaS

ELaM

ELaL

EBaS

EBaM

EBaL

ECrS

ECrM

ECrL

ELrS

ELrM

ELrL

EBrS

EBrM

EBrL

ST

20

40

60

80

100A


Figure 4.15: Results for the multistage fuzzy strategy and the average mixed strategiesin the scenario with large overlap and equal deadlines

122

4.5. Evaluation

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

PCrS

PCrM

PCrL

PLrS

PLrM

PLrL

PBrS

PBrM

PBrL

ST

0.1

0.2

0.3

0.4

0.5

0.6

Uc

ECaS

ECaM

ECaL

ELaS

ELaM

ELaL

EBaS

EBaM

EBaL

ECrS

ECrM

ECrL

ELrS

ELrM

ELrL

EBrS

EBrM

EBrL

ST

0.1

0.2

0.3

0.4

0.5

0.6

Uc

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

PCrS

PCrM

PCrL

PLrS

PLrM

PLrL

PBrS

PBrM

PBrL

ST

20

40

60

80

100A

ECaS

ECaM

ECaL

ELaS

ELaM

ELaL

EBaS

EBaM

EBaL

ECrS

ECrM

ECrL

ELrS

ELrM

ELrL

EBrS

EBrM

EBrL

ST

20

40

60

80

100A

Figure 4.16: Average utility (top) and agreement rate of the multistage fuzzy strategyusing boulware fuzzy constraints and the average mixed strategy using the traditionalmechanism in the scenario with large overlap and equal deadlines

4.5.5 Scenario with Large Overlap and Different Deadlines

In this scenario both agents have different deadlines with a large overlap of the nego-

tiation intervals. Similar to the scenarios where the agents have small overlaps, the

different deadlines result in missed agreements and therefore lower average utilities as

compared to the scenario with equal deadlines. However, the multistage fuzzy strategy

still obtains higher agreement rates and average utilities as the average mixed strategy

in many strategy scenarios. The multistage fuzzy strategy obtains lower agreement

rates in cases where the opponent uses boulware time-dependent tactics in the mixed

strategies with the linear weighted combination (Figure 4.17a). This again is caused by

the traditional mixing mechanism that may produce non-monotonic concession curves

in such situations.

123


PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

PCrS

PCrM

PCrL

PLrS

PLrM

PLrL

PBrS

PBrM

PBrL

ST

0.1

0.2

0.3

0.4

0.5

0.6Uc

ECaS

ECaM

ECaL

ELaS

ELaM

ELaL

EBaS

EBaM

EBaL

ECrS

ECrM

ECrL

ELrS

ELrM

ELrL

EBrS

EBrM

EBrL

ST

-0.1

0.1

0.2

0.3

0.4

0.5

0.6Uc

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

PCrS

PCrM

PCrL

PLrS

PLrM

PLrL

PBrS

PBrM

PBrL

ST

20

40

60

80

100A

ECaS

ECaM

ECaL

ELaS

ELaM

ELaL

EBaS

EBaM

EBaL

ECrS

ECrM

ECrL

ELrS

ELrM

ELrL

EBrS

EBrM

EBrL

ST

20

40

60

80

100A


PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

PCrS

PCrM

PCrL

PLrS

PLrM

PLrL

PBrS

PBrM

PBrL

ST

0.1

0.2

0.3

0.4

0.5

0.6Uc

ECaS

ECaM

ECaL

ELaS

ELaM

ELaL

EBaS

EBaM

EBaL

ECrS

ECrM

ECrL

ELrS

ELrM

ELrL

EBrS

EBrM

EBrL

ST

0.1

0.2

0.3

0.4

0.5

0.6Uc

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

PCrS

PCrM

PCrL

PLrS

PLrM

PLrL

PBrS

PBrM

PBrL

ST

20

40

60

80

100A

ECaS

ECaM

ECaL

ELaS

ELaM

ELaL

EBaS

EBaM

EBaL

ECrS

ECrM

ECrL

ELrS

ELrM

ELrL

EBrS

EBrM

EBrL

ST

20

40

60

80

100A


Figure 4.17: Results for the multistage fuzzy strategy and the average mixed strategiesin the scenario with large overlap and different deadlines

124

4.5. Evaluation

As a comparison, the concession-based mechanism in Figure 4.17b produces higher

agreement rates and utilities for both the multistage fuzzy strategy and the average

mixed strategy for the strategy groups involving boulware time-dependent tactics (with

the former being better in almost all boulware strategy groups). Intuitively, the multistage

fuzzy agent obtains lower agreement rates in the case where it applies the boulware

fuzzy constraints (4.18). Similar to the scenario with small overlaps, the smaller con-

cessions of that strategy result in the agent missing the zone of agreement in many

cases due to the different deadlines of the agents.

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

PCrS

PCrM

PCrL

PLrS

PLrM

PLrL

PBrS

PBrM

PBrL

ST

0.1

0.2

0.3

0.4

0.5

0.6

Uc

ECaS

ECaM

ECaL

ELaS

ELaM

ELaL

EBaS

EBaM

EBaL

ECrS

ECrM

ECrL

ELrS

ELrM

ELrL

EBrS

EBrM

EBrL

ST

-0.1

0.1

0.2

0.3

0.4

0.5

0.6

Uc

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

PCrS

PCrM

PCrL

PLrS

PLrM

PLrL

PBrS

PBrM

PBrL

ST

20

40

60

80

100A

ECaS

ECaM

ECaL

ELaS

ELaM

ELaL

EBaS

EBaM

EBaL

ECrS

ECrM

ECrL

ELrS

ELrM

ELrL

EBrS

EBrM

EBrL

ST

20

40

60

80

100A

Figure 4.18: Average utility (top) and agreement rate of the multistage fuzzy strategyusing boulware fuzzy constraints and the average mixed strategy using the traditionalmechanism in the scenario with large overlap and different deadlines

It should be noted that the results shown in the evaluation of all four negotiation envir-

onment scenarios depend on the chosen cases for the multistage fuzzy strategy as well

as how the fuzzy constraints are modelled (cf. Section 4.2.5). In addition, the negoti-

ation strategy of an opponent may not only depend on the behaviour of its counterpart,

but also on different factors such as its deadline or the state of a resource in the envir-

onment. Because the multistage fuzzy strategy captures the relationship between the

modelling agent’s and the opponent’s concessions it attempts to model the behaviour

125


of the opponent in relation to the agent’s behaviour based on the limited knowledge it

observed. An opponent with a strategy that also depends on other factors may indeed

behave different than the modelled concession behaviour and lead to different results.

4.6 Related Work and Discussion

The problem of bilateral agent negotiation with limited or uncertain information about

the strategies of the opponent is known to be hard, and many solution approaches

have been proposed to cope with it ranging from simple If-then rules, heuristic tactics

to more advanced learning and reasoning techniques [15]. Such adaptive negotiation

mechanisms mostly assume agents to steadily explore their environment and other

agents’ behaviour to gain experience from past interactions, or maintain explicit be-

liefs about utilities, constraints and decision models of their opponents. There two

approaches, which are similar to our approach in that they use past cases to generate

a negotiation strategy. Matos and Sierra [93] present a case-based reasoning-driven

approach that lets agents use past successful interactions to negotiate similar agree-

ments by respectively (case-based) adjusting combined decision function parameters.

In fact, alongside the negotiation thread of each case the parameter values of the ap-

plied strategies are required. However, that inhibits the use of cases by agents with

different individual decision models. Wong et al [139] use observed concessions to

capture past negotiation cases and apply certain filters to select the best one. It differs

from our approach in that they do not allow for reasoning on and interpolation between

the cases and the preferences of an agent. The application of possibility theory to ne-

gotiation has been proposed in [17] where the decision on potentially beneficial ne-

gotiation partners bases on the expected qualitative utility but without modelling the

negotiation process itself as a fuzzy (or possibilistic) Markov decision process. In fact,

only a few approaches exist so far. For example, Narayanan and Jennings [96] model

the agent’s behaviour by defining the states in terms of resource availability, deadlines

and reservation values where counteroffers are proposed based on the opponent’s offers

and changes in those three realms. It is shown that agreements can be achieved much

faster when both agents use this algorithm, but no results for cases are provided where

only one agent uses this strategy. Similar to our method, Teuteberg [134] models the

behaviour of the opponent, but uses a probabilistic approach to generate the transition

126

4.6. Related Work and Discussion

matrix based on a predefined set of opponent tactics. The major disadvantage of such

an approach is the large number of negotiations required to obtain sufficient empir-

ical data for reliable state transitions. Negotiation has also been modelled as a fuzzy

constraint satisfaction problem [91] where constraints, preferences and objectives are

represented uniformly as fuzzy sets which are distributed among the agents and iter-

atively relaxed during the exchange of offers [15]. The search process is guided by

ordering and pruning the search space but still requires negotiation strategies for pro-

posing offers [76]. Based on the seminal paper of Bellmann and Zadeh [7] decision

making in fuzzy environments has been studied and extended by many researchers,

such as Kacprzyk [68], Iwamoto [61] and Dubois et al [33], and has been applied in

many areas including resource allocation, planning or scheduling [68]. The modelling

of agent-based negotiation strategies using multistage fuzzy decision-making repres-

ents a new application of the model in the domain of automated negotiation and further

demonstrates their ability to respond to opponent’s exposing different concession be-

haviours.

The advantages of the multistage fuzzy decision approach result from the fuzzy repres-

entation of the state transitions that allow an agent to use a limited number of reference

cases to generate the transition model for its multistage decision-making during ne-

gotiation. The fuzzy constraints further enable to impose a preferred soft strategy or

conditions of agent on the decision process, which provides flexibility in terms of its

application in different and more realistic negotiation scenarios. For example, an agent

can generate the fuzzy constraints based on the time-dependent decision functions as

shown in Section 4.2.5 to model different types of negotiation strategies in order to

make it more or less competitive. The state-action form and the modelling of the

decision problem as a fuzzy Markov decision process enables the use of traditional

dynamic programming techniques to find the best course of actions. An agent using

this decision model is able to simultaneously take different factors in the environment

and the opponent’s behaviour into account in order to create more adaptive negotiation

strategies for its concession-making during the encounter. On the other hand, a limita-

tion of the multistage fuzzy decision approach in this thesis is the high computational

cost, which increases for larger negotiation ranges and more negotiation issues due to

the requirement that the state space needs to cover the whole negotiation space. Added

to this, the cost of the algorithm also increases because of the required interpolation in

the expected fuzzy goal for each state in situations with a small number of reference

127


cases. It is therefore easier to fuzzify the cases to a form based on fuzzy rules when a

large number of cases is available, and use them to create the fuzzy transition model.

The similarity between the cases and the current encounter is then used to adjust the

fuzzy case constraints. Although the approach is able to successfully model the op-

ponent’s concession behaviour based on limited knowledge, it does not consider other

possible factors of a negotiation strategy. For example, in many realistic scenarios the

opponent’s strategy typically does not depend only on the behaviour of its counterpart,

but also on factors such as a resource in the environment, a users preference or other

outside options. The mixed strategies shown in Chapter 3 represent examples for such

strategies. An agent using the multistage fuzzy approach is not able to take such factors

into account as they are usually private information. Other important factors of a nego-

tiation strategy such as the reservation value and the negotiation deadline are unknown

to the negotiation partner. As a result, an agent with the multistage fuzzy strategy can

make concessions in order to obtain an agreement considering the possible concession

behaviour of the opponent, but without knowing if the opponent would make conces-

sions beyond the agreement point of the particular cases (as shown in the evaluation

in Section 4.5). Furthermore, if the negotiation partner changes its reservation value

or deadline, its concession behaviour changes even though it uses the same concession

strategy. It would be beneficial to the agent to have a mechanism that anticipates reser-

vation values or the negotiation deadlines of its opponents, which, however, requires

domain knowledge or a large amount of historical information. Another method is

to adjust the agents own reservation value depending on the negotiation scenario and

the domain. This is discussed in more detail in the next chapter with a more complex

example scenario in the domain of service-oriented computing.

4.7 Summary

In this chapter we have presented a novel approach for modelling an agent’s negoti-

ation strategy based on multistage fuzzy decision-making. In this approach, an agent’s

limited knowledge about the opponent’s concession behaviour, for example, in the

form reference cases from a few past interactions, is used to create a model with fuzzy

state transitions, while the agent’s soft preferences are represented using a fuzzy goal

and fuzzy constraints. While the fuzzy constraints allow an agent to model different

128

4.7. Summary

types of negotiation strategies by imposing its soft preferences on the decision process,

the fuzzy transition model in the form of states and actions enables the use of tradi-

tional dynamic programming algorithms for finding the best course of actions during

the encounter. The decision algorithms of an agent using the multistage fuzzy decision

model have been presented, as well as some negotiation examples, which have further

illustrated the approach using example reference cases and different fuzzy constraint

settings. The evaluation has validated the approach by comparing the mutistage fuzzy

decision approach with the mixed strategies using the traditional mixing mechanism

and the concession-based mechanism from Chapter 3. We have also discussed related

work and the limitations of this model. In the next chapter, we present a more complex

example scenario for automated concurrent negotiations that is used to present a new

coordination mechanism for negotiation strategies, and to demonstrate the applicability

of the decision strategies discussed in this thesis.

129

Chapter 5

Coordinating Strategies in ConcurrentAutomated Negotiations

In this chapter, we present a new decision mechanism which enables the coordination

of negotiation strategies in more complex, one-to-many bilateral concurrent negoti-

ations using an example scenario in the domain of service-oriented computing. In

this scenario, a number of agents concurrently negotiate with service providers about

the quality of service (QoS) parameters, such as delivery time, price or throughput,

in order to establish service level agreements for a number of atomic services within

in a workflow-based composite service. The composite service provider typically has

some end-to-end QoS constraints over the overall composite service given by the ser-

vice consumer, which the individual atomic service agents need to consider in their

encounters when negotiating towards their negotiation boundaries. In addition, the

structure of the composite service influences the aggregation of the particular QoS

parameters. Therefore, we propose in this chapter an algorithm for the utility bound-

ary decomposition based on the end-to-end QoS constraints provided by the service

consumer and the subsequent redistribution of surpluses from successfully finished ne-

gotiations among those remaining considering the structure of the composite service.

While the algorithm coordinates the negotiation boundaries of each atomic service

agent, it leaves control over the concession behaviour to the individual agents so that

they can use the limited knowledge they acquired. It is also shown that the mechan-

ism can increase the number of compound agreements through the method of surplus

redistribution of successfully finished negotiations while simultaneously negotiating

5.1. Composite Service Provisioning

competitively. An experiment using a two-service scenario and the SLA-negotiation

example demonstrates the applicability of the proposed decision-making strategies in

this thesis and the coordination mechanism.

5.1 Composite Service Provisioning

The Service Oriented Computing paradigm has paved the way for a new service-

oriented business model in dynamic business networks [11] that is referred to as Com-

posite Service Provisioning [8, 51] or Service Aggregation [72]. In this model, service

based applications are rarely built using a single service but are instead composed

by aggregating or bundling several component services to create dynamic business

processes that span organisations and computing platforms [104]. This model allows

service consumers, providers and composite service providers (CSP) to collaborate

in highly distributed environments, and establish on-demand, short-term and dynamic

business relationships based upon their requirements, constraints and capabilities. This

is particularly relevant in the context of Cloud Computing which is a technology that

aims to dynamically deliver on-demand IT resources based on Service Level Agree-

ments [136].

With increasingly competitive business environments, service providers are interested

in maximizing their profitability, and service requestors are interested in selecting ser-

vice providers that best meet their QoS standards. As a consequence, several possible

scenarios can arise in the services market. Several providers can offer functionally

equivalent services but at varying levels of quality; or any given service provider can

offer the same service at varying levels of quality [39]; and service requestors can have

varying QoS preferences over a requested service (both atomic and composite). Given

the diversity and complexity of end-user needs and the multitude of service offerings,

the mechanism chosen for establishment of Service Level Agreements (SLA) between

service requestors and service providers (including composite service providers) can

vary between simple design-time service selection (based upon static SLAs), dynamic

run-time service selection (based upon the latest but fixed SLA offerings) and dynamic

automated negotiations over the QoS requirements.

Given the unique characteristics of composite service provisioning including a dy-

131

Chapter 5. Coordinating Strategies in Concurrent Automated Negotiations

namic service landscape, varying non-functional service capabilities and requirements,

and changing user requests, service selection may not always be the best solution for

SLA establishment. Even manual human based negotiation may be inefficient for ne-

gotiating and establishing SLAs for on-demand composite service provisioning. How-

ever, automated negotiation is suited for the on-the-fly adjustment of QoS requirements

and offerings on the basis of the service consumers’ needs and the service providers’

load and current context. In the composite service provisioning process, negotiations

have to be held with multiple service providers for each atomic service within the

composition such that the end-to-end QoS requirements of the consumer are satisfied.

We present an algorithm for the initial global utility decomposition and subsequent

surplus redistribution, which takes into account the relative importance of each atomic

service in the composition, the appropriate aggregation function for each QoS attribute,

and the control flow pattern of the composite service. In scenarios involving multiple

concurrent service negotiations, the preference structure over negotiation outcomes,

the negotiation strategies and deadlines are unknown to both the service consumers and

providers. This increases the risk of failed agreements, especially with more complex

composite service structures. For that reason, we propose a mechanism to redistribute

the surplus of a successfully finished negotiation among the remaining concurrent ones

thereby increasing the chances of reaching agreements for all atomic services within

the composition. In addition, if negotiations are unsuccessful or services fail during

the provisioning, the atomic services can be renegotiated using our mechanism taking

into account already obtained agreements. The proposed mechanism is a practical ap-

proach to efficiently coordinate concurrent service negotiations within complex work-

flows, enabling the iterative and interactive adjustment of the negotiation boundaries

for each atomic service in a composition based on the performance and results of other

concurrent negotiations. We demonstrate the usefulness of our coordination algorithm

by evaluating it with the multi-tactic and multistage fuzzy negotiation strategies using

the Specialised Property Search Scenario and show that it obtains significantly higher

agreement rates and utilities for a large range of different negotiation behaviours. The

algorithm presented in this chapter can be equally applied to QoS negotiation for the

provisioning of resources, services and applications on any distributed environment

including the grid and the cloud.

132


S1A

S3A

S2A

Composite Service Provider

S1

S2S3

S4

S5

SLA1

SLA2

SLA3

SLA4

SLA5

Client SLA

S5A

S4A

S6S6ASLA6

S1B

S1C

S2B

S3B

S4B

S5B

S6B

Service Consumer

Service Providers

Com

posi

te

Ser

vice

Figure 5.1: Composite service provisioning scenario

5.1.1 Definitions and Challenges

Common Definitions. Figure 5.1 shows a composite service provisioning scenario

in which the CSP is a business entity that is adept at generating new business capab-

ilities by aggregating services in innovative ways. A composite service is a logical

collection of services that can collectively fulfil the functional requirements of the

end-consumers. Each indivisible and self-contained service in the composite service

is referred to as an atomic service. Each atomic service belongs to a particular ser-

vice type (a grouping of functionally equivalent services) and can either be abstract

(before negotiation) or concrete (after successful negotiation). There are a number of

service providers that can offer functionally equivalent services from which the CSP

can choose the best candidate. The concept of service types has been proposed in sev-

eral research projects and is widely published in [8, 28, 24, 31]. Composite service

provisioning generally occurs in two phases. The first phase involves the generation of

the abstract process definition that defines a new business capability. This definition

describes a logical collection of abstract services along with the control flow and data

flow between them. Each composite service is composed according to the various com-

position patterns and is in turn offered by the composite service provider to targeted

133


end-consumers. Such an abstract definition can be expressed as an abstract BPEL1

process. In the second phase, concrete services are chosen for each service type in the

composite service through automated negotiation, resulting in the transformation of

the abstract definition into an executable BPEL process.

Assumptions. We assume that the composite service is given as an abstract BPEL

process. Similarly the global or end-to-end utility function for the composite service

is also provided either directly by the end-consumer or is derived through the process

of preference elicitation2 [108]. We assume that the composite service provider and

the atomic service providers have appropriate middleware to support automated ne-

gotiation3. The abstract services can be regarded as placeholders for concrete services

which are selected from a set of candidate providers through the process of negotiation.

We make use of automated negotiation to select the best provider for each atomic ser-

vice in the composition. We have previously proposed the use of agents for enabling

automated negotiation of QoS for composite services in [28]. The aim is to obtain the

best possible agreement for each atomic service such that the global utility is maxim-

ized. Thus the objective of the problem can be regarded as the collective maximization

of the global utility function while ensuring agreements for all the atomic services in a

composite service.

Challenges. The negotiation of end-to-end QoS can involve a number of different

attributes such as price, response time, availability, throughput etc. The different at-

tributes are aggregated differently depending upon the type of the QoS attribute and

the control flow pattern as explained in Section 5.1.3. In order to carry out negoti-

ations with one or more providers, the composite service provider needs to determine

the preferences and desired outcomes (represented by utility functions) for each of the

atomic services. The service consumers generally provide the preferences and desired

1Business Process Execution Language (BPEL) is the de-facto industry standard for web servicecompositions

2Preference elicitation is the process of extracting information about user preferences to an extentnecessary to make good or even optimal decisions when evaluating requests and offers, e.g. by usingutility functions

3Negotiation is a process by which two or more parties (service consumers and service providers ina services environment), with different criteria, constraints and preferences, jointly reach an agreementon the terms of a business transaction. In automated negotiations, the task of reaching agreements isdelegated to computational agents that negotiate on behalf of humans

134


outcomes (i.e. overall utility function) for the composite service only, and do not expli-

citly specify the preferences for each atomic service, which is necessary for conducting

the concurrent one-to-many atomic service negotiations. Therefore, the first challenge

in the end-to-end QoS negotiation for composite services is the initial decomposition of

the end-to-end preferences for the composite service into atomic service preferences.

During the process of end-to-end QoS negotiation, the negotiations with service pro-

viders for all atomic services occur concurrently. However, the time taken to reach

an agreement will be different for the atomic services depending upon the negoti-

ation deadlines, strategies and behaviours of the different providers. During the phase

of concurrent negotiations, the higher the number of atomic agreements reached, the

greater the incentive for the CSP to reach agreements for the remaining services which

are still being negotiated. From the CSP’s perspective, the failure of finding a provider

service for even a single atomic service in the composition results in the failure of the

whole composition. Thus the second challenge is the coordinated and iterative update

of the individual utility functions and preferences for the remaining negotiations, so

that agreements can be reached in the subsequent rounds.

5.1.2 Motivating Scenario of Specialized Property Search

To illustrate the usefulness of the coordination approach, we present an example of

a composite service for Specialized Property Search. Buying a house is a very com-

plex and time-consuming process which involves a number of government and private

sector services. If we look at the high level business services involved in buying a

house, it includes searching for and shortlisting potential properties (finding suitable

property), inspecting the shortlisted properties and after making a final selection, pur-

chasing the selected property. These services could be offered on the cloud as user

level applications.

Each of these high-level business services or tasks can be fulfilled by an underlying

composite service which is an aggregation of different technical services that collect-

ively fulfil the high-level business service. In the following we focus on the business

service of ’Finding Suitable Properties’. Figure 5.2 shows an example of a composite

service which is required to realize this business capability. The input to this busi-

ness service is a set of functional requirements (related to the property) and the non-

135


��

Advanced Property Search

User Request

Search Result

��

Flood Plans

��

Bushfire Plans

��SLA

��

Crime Statistics

��

Map Service

��

Suburb Details

��

Merge Plans with Map

��

Report Generator

��SLA

��

SLA

��SLA

��SLA

��SLA

��SLA

��

SLA

Figure 5.2: Service process fulfilling the business service of finding suitable properties

functional requirements (delivery time and price). The output is a set of specialized

property reports based upon which the client can shortlist the properties that are to be

inspected. The Composite Service Provider could be a business entity which aims to

assist potential home buyers by providing seamless delivery of customized property

reports based on specific search criteria in an open and competitive services environ-

ment. In order to be able to filter out the best candidates (e.g. the top three candid-

ate properties which meet all user requirements), the user would be interested in the

following information about the properties and suburbs including (a)Bush fire plans,

(b)Flood plans, (c)Crime statistics, (d)Images/Videos of the property, (e)Street views,

(f)Transport services (closest train station, bus stop, tram stop), (g)Proximity of ba-

sic amenities (schools, post office, hospital, fire station, police station), and (h)Suburb

Real Estate Profile. This information is offered by disparate government and private

sector agencies through self-contained atomic services and hence the CSP provides

greater value to its clients by aggregating the different types of services and combining

the results into a single, unified report that it delivers to the clients. Figure 5.2 shows

the service workflow and the atomic services involved in the search for property, in-

formation collation and report generation within ’Finding Suitable Properties’. Each

atomic service might be offered by multiple providers while the composite service pro-

vider’s aim is to obtain a service level agreement (SLAs) for each atomic service. For

instance, there may be several providers offering specialized property search engines

which are more advanced than the current search engines4. Similarly, there can be one

4Australian property search engines are, for example, www.domain.com.au and www.realestate.com.au.

136

www.domain.com.au

www.realestate.com.au

www.realestate.com.au


or more potential providers for the Suburb Details service 5 which delivers reports on

suburb details and the Map service 6.

Although each of these services is offered as an IT enabled service, it may involve hu-

man resources as well. For example, the Suburb Details service might involve human

resources to prepare the details of a suburb. Thus the delivery time might vary de-

pending upon the price payable. The key negotiable attributes in this scenario include

delivery time (the time taken to deliver the final reports on the short-listed properties)

and the associated price for the service. When negotiating the end-to-end delivery

time and price for the final reports, the algorithm takes into consideration the control

flow pattern of the composite service, the appropriate aggregation functions for de-

livery time and price, and the relative importance of each service in the composition.

Throughout the rest of the chapter we will use the above scenario as a reference to

demonstrate the advantage of the coordination algorithm in Sections 5.4 and 5.5.

5.1.3 QOS Aggregation

In this section, we introduce the QoS Aggregation Model. A composite service is es-

sentially made up of a number of structural elements or compositional patterns that

integrate a number of atomic services into the workflow pattern of the composite ser-

vice. Such structural elements can be of different abstract types, such as ordered and

arbitrary sequences, loops or parallel patterns like AND, OR and XOR. Jaeger et al

[63] introduces a complete set of abstract composition patterns that comply with the

widely adopted BPEL4WS (Business Process Execution Language for Web Services)

standard. In this thesis, we assume that the negotiation with candidate service pro-

viders takes place before the composite service is executed. For that reason, we do not

need to distinguish between the different join (synchronization) patterns for parallel

structures as defined in [63]. As a simple example, in the case of a parallel XOR ag-

gregation the composite provider does not know which service will be finally executed

and hence needs to negotiate SLAs with the providers of all the atomic services. Table

5.1 shows all aggregation types needed for the SLA negotiation including the corres-

5Domains for obtaining suburb details in Australia are, for example, www.myrp.com.au, www.propertyvalue.com.au, www.homepriceguide.com.au or www.ozhomevalue.com.au.

6e.g. GoogleMaps,MicrosoftMaps or WherisMaps

137

www.myrp.com.au

www.propertyvalue.com.au

www.propertyvalue.com.au

www.homepriceguide.com.au

www.ozhomevalue.com.au

www.ozhomevalue.com.au

Google Maps, Microsoft Maps

Wheris Maps


Aggregation Required ExamplesType atomic services Price/Cost Execution Time Throughput

Sequence

Ordered All∑n

i=1 si∑n

i=1 si min{s1, . . . , sn}Arbitrary All

∑ni=1 si

∑ni=1 si min{s1, . . . , sn}

Loop All ls ls s

Parallel

AND All∑n

i=1 si max{s1, . . . , sn} min{s1, . . . , sn}XOR One max{s1, . . . , sn} max{s1, . . . , sn} min{s1, . . . , sn}OR At least one

∑ni=1 si max{s1, . . . , sn} min{s1, . . . , sn}

Table 5.1: Aggregation functions

ponding aggregation functions for the attributes price, execution time and throughput.

As we can see, for the QoS property price the aggregation function is the sum for the

AND pattern regardless of the synchronization point. In the case of the OR pattern,

the sum function also needs to be used because the service provider needs to find a

candidate for each atomic service as it is unknown which service in this pattern will

actually be executed. However, for the XOR-type, the maximum must be applied since

only one service needs to be executed but the provider does not know the exact ser-

vice in advance. The aggregation structure of the different QoS attributes according to

different compositional patterns imposes a representation in the form of a hierarchical

structure. For example, for the composite service presented in Section 5.1.2 the cor-

responding structured tree is shown in Figure 5.3 where the internal nodes represent

the structural elements or composition patterns with a certain aggregation type and the

leaf nodes are the atomic services that collectively make up the composite service. The

internal nodes serve as the sequential and parallel aggregation points (’Sequence’ and

’Flow’ represent the sequential and parallel composite patterns in BPEL) at which the

QoS attributes of all child nodes are aggregated according to the appropriate aggreg-

ation function (see Table 5.1). Each sub-tree hence represents a composite service by

itself contributing a service to the next higher level in the tree (the composite at the

next level).

We can formally define a composite service as follows. Let a composite service be

composed of n atomic services where sij is the value of the QoS attribute j, e.g. price,

for a service i with i = 1, . . . , n and Sj is the domain of the QoS attribute for service i

such that sij ∈ Sj . The compositional structure of the composite service is represented

138


Sequence

Flow

Sequence

Flow

��

Advanced Property Search

��

Flood Plans

��

Bushfire Plans

��

Crime Statistics

��

Map Service

��

Suburb Details

��

Merge Plans with Map

��

Report Generator

s1

s2 s3 s4 s5

s6

s7

s8

Figure 5.3: Service process in tree form

by the composite aggregation function fag,j : Snj → Sj for QoS attribute j with n being

the number of atomic services in the composite. This function is constructed based on

the aggregation types agtype with type ∈ {seq, loop, and, or, xor} and is applied in

accordance to the structure of the composite service, i.e. its parallel and sequential

aggregation types as shown in Table 5.1. Considering again our example scenario,

the aggregation types for the QoS attribute price are sequence and parallel AND such

that sj is the sum∑n

i=1 sij regardless of the arrangement of the individual atomic

services in the composition. Similarly, in the case of the QoS attribute throughput

the aggregation function is the minimum regardless of the aggregation pattern. In the

case of the QoS attribute delivery time, however, the maximum aggregation function

is required for the parallel (AND) and the sum for sequences. The overall aggregation

function fag can hence be written according to the structure shown in Figure 5.3 for

delivery time as follows:

fag(s1, . . . , s8) = agseq(s1, agand(agseq(agand(s2, s3, s4, s5), s6), s7), s8)

= s1 + max(max(s2, s3, s4, s5) + s6, s7) + s8.(5.1)

139


As shown above the composite aggregation function consists of nested aggregation

functions representing the structure of the composite service, starting from the inner-

most atomic service or service sequence. In that sense, each point of aggregation, i.e.

aggregation function, can be considered a composite service on its own, which in turn

contributes to the next higher level in the end-to-end composition.

5.2 SLA Negotiation

In order to find a provider for each atomic service in the composition, one negotiation

agent is assigned to each atomic service that negotiates with service providers about

the the quality of service parameters such as delivery time, price or throughput in or-

der to attain a service level agreement. The encounters between the atomic service

agents and the candidate providers are bilateral negotiations employing the simple

negotiation model described in Section 2.2.1. The bilateral negotiations are carried

out concurrently because the composite service provider is interested in establishing

the composite service by finding appropriate providers in a short amount of time. In

such concurrent multi-party negotiation scenarios a useful method to coordinate the

strategies of the individual agents is to change their strategy parameters based on the

performance of concurrent negotiations or the outcomes of agents after they reached

an agreement. Since in the considered scenario the preference over a QoS attribute

of the composite service is given only in the form of global QoS constraints (a global

utility boundary) the individual utility boundaries for each service agent are derived by

decomposing the global utility boundary while each agent may choose the individual

function, i.e. its tactic or negotiation strategy, by itself. Thus, a coordinating agent

may be used to change the strategies of the atomic service agents only by adjusting

their individual utility boundaries, which are equivalent to the negotiation boundaries

or reservation values. Such a coordinating agent communicates with the atomic ser-

vice agents and uses only the adjustment of the local negotiation boundaries of each

service agent to coordinate all atomic service negotiations according to the end-to-end

quality of service constraints. In the following, we show how the multi-tactic negoti-

ation strategies and the multistage fuzzy strategies can be coordinated by adjusting the

negotiation boundary.

140

5.2. SLA Negotiation

0 5 10 15 20t

15

20

25

30

35

40price

buyer with smoothing

buyer w�o smoothing

seller

Figure 5.4: Negotiation example with and without smoothing function for changingboundaries

5.2.1 Strategic Adjustment of Boundary Values in Multi-tactic Ne-gotiation Strategies

The adjustment of the negotiation boundary in a multi-tactic negotiation strategy de-

pends on the type of tactics involved in the mixed strategy. For example, while the

time-dependent tactics have a specified reservation value until which they negotiate

when the deadline is reached, the imitative and most of the resource-dependent tactics

from Section 2.3.1 have no adjustable reservation values. For example, an imitative

tactic proposes concessions to some degree based on the opponent’s concessions, but

independent of the negotiation boundary. Therefore, we focus on the time-dependent

tactics. When changing the negotiation boundaries of such a tactic during the en-

counter the change in the tactic may result in a abrupt change of the offer curve and

hence in the behaviour of the atomic service agent, for example, in the case of a time-

based tactic. In order to obtain a gradual change in the strategy we can replace αaj (t)

in (2.5) in Section 2.3.1.1 with function αaj (t, tinit) for agent a given by

αaj (t, tinit) =αaj (t)− αaj (tinit)

1− αaj (tinit)(5.2)

where the time tinit represents the time when the negotiation interval is reset, i.e. when

the change of the boundary value occurs. This requires that the other boundary value of

the interval also has to be reset to the current offer of the agent. For example, in case of

a decreasing utility function,maxaj is set to the new reservation value whileminaj is set

to agent a’s current offer and tinit to the current time tn. At the beginning of negotiation

141


tinit is set to 0. Figure 5.4 shows a simple single-issue negotiation example for the

price of a service where the client (e.g. buyer) changes its boundary value during

the encounter and generates the offers either with or without the above smoothing

function. The negotiation intervals of the provider agent p and the client agent c are

minc = 10, maxc = 30, and minp = 15, maxp = 40 and deadlines are the same

with tcmax = tpmax = 20. Both agents use simple time-dependent polynomial tactics

with βc = 2 and βp = 0.6. Agent c changes its boundary value maxc from 30 to 38 at

negotiation round 10 due to a change in the negotiation environment, or, as considered

in the example scenario, based on the outcome of a concurrent negotiation. As we can

see, in this example the smoothing function avoids the abrupt change and hence obtains

a better outcome. Furthermore, avoiding a large change in the concession behaviour

of the agent might be an advantage especially in cases where the opponent agent uses

more intelligent strategies that could reduce the amount of concessions immediately as

a response.

5.2.2 Strategic Adjustment of Multistage Fuzzy Decision Strategiesvia Fuzzy Constraints

In order to adjust the negotiation boundaries of the multistage fuzzy strategy the co-

ordinating agent can use the time-dependent fuzzy constraints to influence the decision

process of the agent. As shown in Section 4.2.5 the agent can use the time-dependent

decision functions to generate the set of fuzzy constraints. The coordinating agent

then only adjusts the boundary of time-dependent function based on the outcome of

a successfully finished concurrent negotiation which the agent then uses to generate a

new set of fuzzy constraints for the generation of its offer proposal. Moreover, sim-

ilar to the time-dependent tactics in multi-tactic negotiation strategies, the agent with

the multistage fuzzy strategy can choose the type of decision function and therefore

its own strategic preference. This gives the agent the ability to decide over its own

strategy based on its limited knowledge and the shape of the fuzzy constraints while

the coordinating agent only adjust the negotiation boundary.

142

5.3. Utility Boundary Decomposition and Surplus Redistribution

5.3 Utility Boundary Decomposition and Surplus Re-distribution

As described in Section 5.1.2 the end consumer specifies its preference over the QoS

attributes either by providing a utility function for the end-to-end composite service or

in the form of end-to-end QoS constraints like the most and least preferred QoS values.

However, before atomic negotiations can be carried out with the candidate providers,

it is necessary that the composite service provider equips each atomic service agent

with an individual (atomic) utility function based on the global utility function taking

into account the abstract structure of the composite service. A service agent then uses

its atomic utility function to specify the negotiation intervals in the form of the initial

and reservation values (as explained in Section 2.2.3) and to define its atomic nego-

tiation strategy. In many cases, however, it is sufficient to provide the atomic service

agents only with the boundaries of the utility function [20] while the agents can choose

different strategies for their concession behaviour to generate the offers and counterof-

fers depending on their experience or best practices. For this reason, we focus on the

decomposition of the boundaries of the global utility function. Let the global utility

function Uj represent the preference of a service consumer over the QoS attribute j

for the composite service mapping the QoS attribute space Sj to the unit interval. The

global utility function U that takes into account the aggregation structure fag,j of the

composite service can then be written as follows:

Uj : Snj → [0, 1]

Uj(s1j, s2j, . . . , snj) = Uj(fag,j(s1j, s2j, . . . , snj)).(5.3)

In the case where the service consumer provides its preference in the form of a utility

function instead of the end-to-end QoS constraints the upper and lower boundaries si,jand si,j , respectively, can be derived from Uj with

sj = sup{sj|sj ∈ arg maxsj∈Sj

(Uj(sj))}

sj = inf{sj|sj ∈ arg minsj∈Sj

(Uj(sj))}.(5.4)

143


In addition, during the end-to-end QoS negotiation for composite services, the nego-

tiations with service providers for all atomic services occur concurrently. Since the

time taken to reach an agreement can be different for different atomic services the

composite service provider can increase the chances of reaching an overall agreement

over the composite service by redistributing the surplus from successful negotiations

among the remaining negotiation threads in the composition. The surplus thereby is

the difference between the accepted negotiation outcome and the reservation value of

an agent. This can be mapped into a multi-party negotiation scenario where a coordin-

ator decomposes the QoS boundaries and iteratively redistributes the surplus during the

subsequent rounds. However, in order to enable the QoS decomposition and surplus

redistribution the coordinator requires an initial weight distribution over the atomic

services, which can be used in conjunction with the aggregation function to derive the

individual boundaries for each atomic service. This distribution represents the ’share’

or relative importance of each atomic service and can be expressed in the form of a

weight function ωij : Sj → [0, 1] taking into account the structure of the composite ser-

vice using the aggregation function such that ωj = fag,j(w1j, . . . , wij, . . . , wnj) = 1.

The weight distribution represents the proportion between all atomic services in re-

lation to the service composition and can be derived from various sources, such as

domain knowledge, outcomes of past negotiations or based on the prices the service

providers advertise for their services. In the case of no a priori information, the first

offers of each service provider can be used instead. Given the global QoS boundaries

and the tree-based structure of a composite service the utility boundary decomposition

and surplus redistribution can be done using the algorithm proposed in the next section.

5.4 Algorithms

As demonstrated in Section 5.1.3 the composite service in the example of Figure 5.2

can be represented as a tree (cf. Figure 5.3), where the internal nodes represent the

logical structure of the composition in the form of the parallel and sequential sub-

compositions and the leaf nodes correspond to the atomic services in the composite

service. We use a weighted tree approach where the edges hold the weights for child

nodes representing their importance or ’share’ in the composite service at the level

of the parent node, and the all inner nodes hold aggregation functions with respect

144

5.4. Algorithms

to the aggregation types of a particular QoS attribute. In this approach, the weights

for all edges are generated based on the initial values for all atomic services in the

composition that can be derived, as mentioned previously, from experience or represent

the first offers from service providers. The weighted tree T = (V,E) with nodes v ∈ Vand edges e ∈ E = V × V is further specified as follows:

• Inner nodes vl ∈ Vinner correspond to the structural elements of the composition

and hold the aggregation function agtype ∈ {plus,max,min, loopx} and the cur-

rent value of the node depending on the aggregation type and the QoS parameter

under negotiation. The aggregation function of an inner node is obtained with

type(vl).

• Leaf nodes vi ∈ Vleaf correspond to atomic services si holding the current

boundary value for the QoS parameter under negotiation.

• An edge e ∈ E connects a parent node vp ∈ Vinner with a child node vc ∈ C(vp)

where C(vp) is the set of child nodes of vp. An edge is thus uniquely defined

with evc = {vp, vc}. Each edge holds a weight evc representing the relational

proportion to all other nodes with respect to the end-to-end QoS attribute of the

parent node P (vc).

• Sub-trees Tvl with vl as the root node represent composite services by them-

selves, providing a service to a higher level such that inner nodes vl hold the

QoS values of the respective sub-trees and the root node vr of the whole tree

holds the end-to-end QoS value for the overall composite service. The value of

a node v is obtained by value(v).

The representation of the tree-based structure of the composition is used to derive the

required atomic utility boundaries for each atomic service agent and to redistribute the

surpluses to other negotiations still in progress after a negotiation reached an agree-

ment. Furthermore, it allows to renegotiate atomic services in cases when service

failures occur or no agreement with all potential providers for a particular service could

be reached. Algorithms 3, 4 and 5 present the main coordination algorithm with the

corresponding procedures to fulfil the utility boundary decomposition, set new global

constraints and redistribute the surpluses after a successfully finished negotiation. Al-

gorithm 6 details the method to update individual branches of the tree for renegotiating

individual services. All of these steps are described in more detail as follows:

145


Algorithm 3 Coordination Algorithm for SLA Negotiation1: for all si ∈ S do2: vi ← initial values3: end for4: INITIALIZENODE(vr)5: SETROOT(new boundary value)6: repeat7: if i reached agreement then8: evi ← 09: surplusi ← vi − si

10: REDISTRIBUTE(P (vi), surplusi)11: end if12: until all atomic negotiations finished

Utility boundary decomposition: Before individual utility boundaries can be ob-

tained from the global utility function we need to derive weights for all edges in the

tree based on the initial weight distribution over the atomic services. This requires

that each leaf node is initialized with a value (e.g. first offers) and each inner node

is initialized with the corresponding end-to-end QoS attribute for its subtree. Lines

1-4 in Algorithm 3 represent this phase. The procedure INITIALIZENODE(v) recurs-

ively updates the values of the inner nodes and the weights of all edges in the subtree

with v as the root node given the values of its leaf nodes and the operator types of its

inner nodes. Therefore, INITIALIZENODE (vr) initializes the whole tree. Using the

weights of all edges a new end-to-end constraint or boundary value is set for all nodes

representing the global QoS constraint or the boundary of the global utility function.

The procedure SETROOT in Algorithm 3 (line 5) represents this phase. The procedure

UPDATENODES(v, surplus) recursively updates the values of all nodes in the subtree

of v using the surplus value and the weights of edges. Even though the boundary de-

composition is executed before negotiations start or when first offers are proposed by

the service providers, new global constraints can be set at any time during the negoti-

ation by using SETROOT.

Surplus redistribution: During the negotiation encounters of the atomic service agents

with service providers a coordinating agent redistributes the surplus as soon as one ser-

vice agent reaches an agreement to other service agents (leaf nodes) still in negotiation.

The surplus is achieved when the value of an agreement obtains a higher utility than at

146

5.4. Algorithms

Algorithm 4 Procedures INITIALIZENODES, SETROOT and UPDATENODES

1: procedure INITIALIZENODE(v)2: if v ∈ Vinner then3: temp← ()4: for all vc ∈ C(v) do5: temp← temp • INITIALIZENODE(vc) . Concatenation of child value

after initialization6: end for7: agtype ← type(v)8: v ← agtype(temp)9: for all vc ∈ C(v) do

10: if agtype = plus then11: evc ← vc/v . Set weight for edge evc12: else13: evc ← 114: end if15: end for16: end if17: return v18: end procedure19: procedure SETROOT(value)20: for all vc ∈ C(vr) do21: UPDATENODES(vc, value− vr)22: end for23: vr ← value24: end procedure25: procedure UPDATENODES(v, surplus)26: v ← v + surplus · ev27: if v ∈ Vleaf then28: Notify negotiation agents for node v29: else30: for all vc ∈ C(v) do31: UPDATENODES(vc, surplus · ev)32: end for33: end if34: end procedure

the current final boundary point. This is represented by the REDISTRIBUTE(v, surplus)

procedure which recursively redistributes the surplus to all sibling nodes and their cor-

responding leaf nodes by using the procedure UPDATENODES. The weight of the edge

to the parent node of v is set to zero. In the case where all the weights from the parent

147


Algorithm 5 Procedure REDISTRIBUTE

1: procedure REDISTRIBUTE(v, surplus)2: agtype ← type(v)3: VC ← {vj ∈ C(v)|evj > 0}4: if VC = ∅ then5: temp← ()6: for all vc ∈ C(v) do7: temp← temp • vc8: end for9: surplusv ← v − agtype(temp)

10: v ← agtype(temp)11: if v 6= root node then12: ev ← 013: REDISTRIBUTE(P (v), surplusv)14: end if15: else if agtype = plus then16: wTv ←

∑vc∈VC evc

17: for all vc ∈ Vc do18: evc ← evc/wTv19: UPDATENODES(vc, surplus)20: end for21: end if22: end procedure

to its sibling nodes are zero, i.e. all negotiations at their leaves reached agreements, the

surplus of the parent node is calculated and redistribution is propagated to its parent

node (the grandparent of v). When all negotiations finish successfully all edges of the

tree have zero weights and the value at the root node is the resulting end-to-end QoS

value for the compound service.

Service renegotiation: In many service negotiation scenarios it can happen that an

atomic service has to be renegotiated, for example, when service failures occur dur-

ing provisioning or no agreement could be reached with all providers for a particular

atomic service. In such cases the particular leaf node has to be reinitialized by re-

setting the weights to its parent nodes recursively until the root node or a non-zero

edge is reached and the value of the leaf node accordingly updated. The procedure

RESETWEIGHTS(v) in Algorithm 6 iteratively sets the weights of all edges to the par-

ent nodes of v while taking into account the weights of its sibling nodes and further

148

5.5. Evaluation

Algorithm 6 Service Renegotiation1: procedure RESETWEIGHTS(v)2: if type(P (v)) = plus then3: for all vc ∈ {vj ∈ C(P (v))|evj > 0} ∩ {v} do4: evc ← vc/P (v)5: end for6: else7: ev ← 18: end if9: if P (v) = root node then

10: SETROOT(new boundary value)11: else if eP (v) 6= 0 then12: UPDATENODES(v,P (v) · ev − v)13: else14: RESETWEIGHTS(P (v))15: end if16: end procedure

updates all node values along this path in order to enable renegotiation of the atomic

service si. This procedure can be called at any time for an atomic service during or

after the negotiation phase of the composite service.

5.5 Evaluation

This section evaluates the proposed SLA negotiation framework including the coordin-

ation algorithm for the utility boundary decomposition and the surplus redistribution

using the discussed negotiation strategies and mechanisms in this thesis.

5.5.1 Experimental Settings

In order to evaluate the coordination mechanism using the surplus redistribution we

choose two scenarios, first, a simple two service composition and, second, the more

complex property search scenario from Section 5.1.2. In the first scenario, the two

services, for example a booking and a payment service, sequentially form the compos-

ite service. In the second scenario, we use the composite service example in Section

5.1.2 with the corresponding eight atomic services and the composite service structure

149


as shown in Figure 5.3: s1:Advanced Property Search, s2: Flood Plans, s3:Bushfire

Plans, s4:Crime Statistics, s5:Map Service, s6:Merge Plans with Map, s7: Suburb De-

tails, s8: Report Generator. In both scenarios, the composite service provider assigns

an agent to each atomic service that negotiates with different atomic service providers

about the issues (QoS parameters) under negotiation, in the first scenario about a single

issue price, and in the second scenario about the issues price and delivery time for an

individual atomic service. The number of service providers for each atomic service

is chosen randomly between two or three providers. For simplicity, we assume that

the initial distribution of the negotiation intervals’ min- and max-values for each is-

sue are derived from domain knowledge or the first offers of the atomic service pro-

viders. We can hence test our approach with randomly generated intervals based on

the method shown in Section 2.6.2, where mincj ∈ {5, 10, 15, 20, 25, 30, 35, 40} and

θj = {15, 20, 25, 30, 35, 40}7. Based on the generated initial interval distribution, the

weight distribution for the coordination algorithm is generated (shown in Algorithm

3 in Section 5.4). Because we are interested in more realistic negotiation environ-

ments the agent’s intervals overlap only partially, either to a small or large degree with

Φ ∈ {0.33, 0.66}. In addition, atomic service and provider agents can have different

deadlines. Since the composite service provider has global deadline this deadline is

imposed on all atomic service agents with tcmax = 30, such that we consider different

deadlines for the provider agents with tpmax ∈ {15, 20, 25, 30, 35, 40, 45}.

We test the negotiation strategies and mechanisms discussed in this thesis within the

two scenarios and compare the performance with and without the surplus redistribution

algorithm. The applied mechanisms are the linear weighted combination of tactics,

the negotiation thread-based mixing, the concession-based mixing, and the multistage

fuzzy decision strategy. In case of the first three multi-tactic mechanisms, we use static

mixed strategies with one time-dependent and one behaviour-dependent tactic, and

construct the set of all possible mixed strategies using the polynomial and exponential

decision functions, and the absolute and relative tit-for-tat tactics as follows:

ST = {PC, PL, PB,EC,EL,EB} × {a, r} × {S,M,L} (5.5)

7The numbers provided for the generation of the negotiation intervals represent examples and arenot taken from real commercial services. For example, the unit for delivery time could be minutes in thecase where humans are involved in the data provision or seconds in the case of full electronic services.

150

5.5. Evaluation

where the settings of the individual strategy groups are the same as in previous chapters

shown in Table 2.1 in Section 2.6.2. In case of the multistage fuzzy strategy, we use

time-dependent constraints to be able to influence the decision process based on the

adjusted reservation values of the atomic service agents by the coordinating agent. For

the reference cases, we use the same from the scenario with two cases in the evaluation

section 4.5.1 from the previous chapter. We test the different time-dependent fuzzy

constraints using the polynomial and exponential tactics, such that we obtain a set of

six types of fuzzy constraints {PC, PL, PB,EC,EL,EB}. We measure the perform-

ance of the concurrent negotiations using the global utility function U c and the overall

agreement rateA (in percent) for the composite service. The utility functions are linear

for the first scenario and linear additive for the second over the negotiation intervals as

described in Section 2.6.2. For simplicity, we assume that in the second scenario the

composite service provider values each of the issues equally (weights of 0.5 for both

attributes). Since the global utility is given based on the outcome of the overall com-

posite service, it is zero if only one atomic service agent fails to reach an agreement.

The agreement rate hence specifies the percentage of all negotiations where all atomic

service agents could reach an agreement. Similar to the evaluation sections from previ-

ous chapters we present the results in the form of a bar chart where each group of bars

represent a particular strategy group in case of a multi-tactic strategy or a particular

time-dependent constraint in case of the multistage fuzzy strategy. In each group of

bars the left bar (light) represents the negotiation without and the right bar (dark) with

the surplus redistribution algorithm. In our experiment, each provider agent applies a

mixed strategy from the set of all strategy groups ST (for both issues in scenario 2),

whereas all atomic service agents apply the same mixed strategy or multistage fuzzy

strategy during one negotiation cycle. The composite service provider negotiates the

whole composite service for each strategy group 200 times, which results in between

400 to 1200 negotiations for scenario 1 per strategy group (2 services with 200 nego-

tiations per service and provider, with 1 to 3 provider per service), and 1600 and 4800

negotiations for scenario 2 per strategy group (8 services with 200 negotiations per

service and provider, with 1 to 3 provider per service). The next sections present and

discuss the result found for the two scenarios.

151


5.5.2 Scenario with Two Services

In order to demonstrate the effect of the surplus distribution algorithm we first show

the results of a simple scenario with two atomic services where the atomic agents need

to negotiate about a single issue price only with the settings described in the previous

section. Figures 5.5 to 5.8 show the average end-to-end utility and the agreement rate

obtained by the atomic service agents when using the different negotiation mechan-

isms without and with surplus redistribution. The effect of the surplus redistribution

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

PCrS

PCrM

PCrL

PLrS

PLrM

PLrL

PBrS

PBrM

PBrL

ST

0.05

0.10

0.15

0.20

0.25

0.30Uc

ECaS

ECaM

ECaL

ELaS

ELaM

ELaL

EBaS

EBaM

EBaL

ECrS

ECrM

ECrL

ELrS

ELrM

ELrL

EBrS

EBrM

EBrL

ST

0.05

0.10

0.15

0.20

0.25

0.30Uc

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

PCrS

PCrM

PCrL

PLrS

PLrM

PLrL

PBrS

PBrM

PBrL

ST

20

40

60

80

A

ECaS

ECaM

ECaL

ELaS

ELaM

ELaL

EBaS

EBaM

EBaL

ECrS

ECrM

ECrL

ELrS

ELrM

ELrL

EBrS

EBrM

EBrL

ST

20

40

60

80

A

Figure 5.5: Average end-to-end utility (top) and agreement rate (bottom) without andwith surplus redistribution for the scenario with two services and agents using staticmixed strategies with the traditional linear weighted combination

for a composition with only two service negotiations is small, however, the rate of

agreements is slightly increased for all strategy groups of each considered negotiation

mechanisms, which in turn also slightly increases the average end-to-end utility for

most strategy scenarios. On the other hand, for some strategy groups the average end-

to-end utility is slightly lower when no surplus is redistributed. This emphasizes the

152

5.5. Evaluation

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

PCrS

PCrM

PCrL

PLrS

PLrM

PLrL

PBrS

PBrM

PBrL

ST

0.05

0.10

0.15

0.20

0.25

0.30Uc

ECaS

ECaM

ECaL

ELaS

ELaM

ELaL

EBaS

EBaM

EBaL

ECrS

ECrM

ECrL

ELrS

ELrM

ELrL

EBrS

EBrM

EBrL

ST

0.05

0.10

0.15

0.20

0.25

0.30Uc

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

PCrS

PCrM

PCrL

PLrS

PLrM

PLrL

PBrS

PBrM

PBrL

ST

20

40

60

80

A

ECaS

ECaM

ECaL

ELaS

ELaM

ELaL

EBaS

EBaM

EBaL

ECrS

ECrM

ECrL

ELrS

ELrM

ELrL

EBrS

EBrM

EBrL

ST

20

40

60

80

A

Figure 5.6: Average end-to-end utility (top) and agreement rate (bottom) without andwith surplus redistribution for the scenario with two services and agents using staticmixed strategies with the negotiation thread-based mechanism

relationship between the number of agreements and obtained utility when using dif-

ferent concession behaviours. For example, if an agent makes larger concessions, the

number of agreements increases while the gained utility for each agreement decreases.

Vice versa, if an agent is more competitive and makes smaller concessions, the ob-

tained utility for an agreement is larger, however, the number of agreements may be

lower instead.

Intuitively, when an agent redistributes its surplus after it successfully finished a nego-

tiation to a concurrent negotiation for the composite service, it increases the negotiation

boundary, i.e. the atomic reservation value of the second service. This, in turn, res-

ults in larger concessions of the second negotiation agent that received the redistributed

surplus. As a consequence, the agreement rate for the overall composite service (which

means that for each atomic service an SLA could be obtained) is increased. However,

153


the larger concessions may result in lower utilities, which also may be increased with

a larger amount of added agreements. Figures 5.5 to 5.8 demonstrate that the co-PC

aSPC

aMPC

aLPL

aSPL

aMPL

aLPB

aSPB

aMPB

aLPC

rSPC

rMPC

rLPL

rSPL

rMPL

rLPB

rSPB

rMPB

rL

ST

0.05

0.10

0.15

0.20

0.25

0.30Uc

ECaS

ECaM

ECaL

ELaS

ELaM

ELaL

EBaS

EBaM

EBaL

ECrS

ECrM

ECrL

ELrS

ELrM

ELrL

EBrS

EBrM

EBrL

ST

0.05

0.10

0.15

0.20

0.25

0.30Uc

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

PCrS

PCrM

PCrL

PLrS

PLrM

PLrL

PBrS

PBrM

PBrL

ST

20

40

60

80

A

ECaS

ECaM

ECaL

ELaS

ELaM

ELaL

EBaS

EBaM

EBaL

ECrS

ECrM

ECrL

ELrS

ELrM

ELrL

EBrS

EBrM

EBrL

ST

20

40

60

80

A

Figure 5.7: Average end-to-end utility (top) and agreement rate (bottom) without andwith surplus redistribution for the scenario with two services and agents using staticmixed strategies with the concession-based mechanism

ordination algorithm can be applied with any negotiation mechanism or strategy that

provides a means to adjust the negotiation boundary of the individual atomic service

agent for its negotiation. In the case of the heuristic-based strategies the coordination

algorithm only effects the time-dependent tactic, since the imitative tactic has no ad-

justable boundary. This implies that the surplus redistribution can only be effective

to the degree by which the time-dependent tactic is contributing to the mixed strategy

via its mixing weight. In the case of the multistage fuzzy strategy in Figure 5.8 the

coordination algorithm uses the time-dependent fuzzy constraints to influence the ne-

gotiation strategy and adjust the negotiation boundaries. The coordination mechanism

can hence directly influence the decision process of each atomic service agent.

154

5.5. Evaluation

PC PL PB EC EL EB

ST

0.05

0.10

0.15

0.20

0.25

0.30Uc

PC PL PB EC EL EB

ST

20

40

60

80

A

Figure 5.8: Average end-to-end utility (left) and agreement rate (right) without andwith surplus redistribution for the scenario with two services and agents using themultistage fuzzy strategy with different time-dependent fuzzy constraints

5.5.3 Property Search Scenario

In this section, we present the results for the more complex Property Search Scenario

shown in Section 5.1.2 where the atomic service agents negotiate with the candid-

ate service providers about two issues price and delivery time with the settings de-

scribed in Section 5.5.1. The figures below demonstrate that the surplus redistribution

is much more effective in the more complex scenario with 8 services as it significantly

increases the agreement rate and the average end-to-end utility for almost all mixed

strategies with the traditional linear weighted combination (Figure 5.9) , the negoti-

ation thread-based (Figure 5.10) and concession-based mixing (Figure 5.11), as well as

for the multistage fuzzy strategy under different fuzzy time-constraints (Figure 5.12).

This seems intuitive, because with larger and more complex service compositions the

chance of failing an agreement for a single atomic service is higher, which in turn also

increases the chance of failing the overall composite service. The surplus redistribu-

tion algorithm increases the amount of concessions an agent makes after it received the

redistributed surplus of successfully finished concurrent negotiations, and therefore in-

creases the chance of attaining an agreement for the atomic service. However, as only

the negotiation boundary is adjusted, the agent makes larger concessions only to some

degree depending on its strategy until the overall deadline is reached instead of making

an immediate large concession of the amount of the received surplus. This balances the

increase of the chance of an agreement and negotiating competitively in order to gain

high utilities at the same time. We can further observe that the increase in the agree-

155


PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

PCrS

PCrM

PCrL

PLrS

PLrM

PLrL

PBrS

PBrM

PBrL

ST

0.05

0.10

0.15

0.20

0.25

Uc

ECaS

ECaM

ECaL

ELaS

ELaM

ELaL

EBaS

EBaM

EBaL

ECrS

ECrM

ECrL

ELrS

ELrM

ELrL

EBrS

EBrM

EBrL

ST

0.05

0.10

0.15

0.20

0.25Uc

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

PCrS

PCrM

PCrL

PLrS

PLrM

PLrL

PBrS

PBrM

PBrL

ST

20

40

60

80

100A

ECaS

ECaM

ECaL

ELaS

ELaM

ELaL

EBaS

EBaM

EBaL

ECrS

ECrM

ECrL

ELrS

ELrM

ELrL

EBrS

EBrM

EBrL

ST

20

40

60

80

100A

Figure 5.9: Average end-to-end utility (top) and agreement rate (bottom) without andwith surplus redistribution for the property search scenario with eight services andagents using static mixed strategies with the traditional linear weighted combination

ment rate is larger for more cooperative strategy groups, i.e. mixed strategies with

conceder or linear time-dependent tactics in the mix, or for the same type of fuzzy

constraints in the case of the multistage fuzzy strategy. The reason is that the con-

ceder and linear time-dependent tactics (and fuzzy constraints) respond to the change

of the negotiation boundary earlier in the negotiation than the boulware tactics. The

figures also show that the coordination algorithm has a different effect on the negoti-

ation mechanisms. For example, the agreement rate for the mixed strategies with the

concession-based mechanism is lower compared to the other mechanisms, whereas the

agreement rate is highest for the traditional linear weighted combination with conceder

time-dependent tactics in the mix. As discussed in Chapter 3 this is a result of different

ways the mechanisms combine the single tactics. As the concession-based mechanism

truly mixes the concessions of each tactic, it makes overall smaller concessions due to

the imitative tactic in the mix as compared to the traditional mixing mechanism, be-

156

5.5. Evaluation

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

PCrS

PCrM

PCrL

PLrS

PLrM

PLrL

PBrS

PBrM

PBrL

ST

0.05

0.10

0.15

0.20

0.25

Uc

ECaS

ECaM

ECaL

ELaS

ELaM

ELaL

EBaS

EBaM

EBaL

ECrS

ECrM

ECrL

ELrS

ELrM

ELrL

EBrS

EBrM

EBrL

ST

0.05

0.10

0.15

0.20

0.25

Uc

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

PCrS

PCrM

PCrL

PLrS

PLrM

PLrL

PBrS

PBrM

PBrL

ST

20

40

60

80

100A

ECaS

ECaM

ECaL

ELaS

ELaM

ELaL

EBaS

EBaM

EBaL

ECrS

ECrM

ECrL

ELrS

ELrM

ELrL

EBrS

EBrM

EBrL

ST

20

40

60

80

100A

Figure 5.10: Average end-to-end utility (top) and agreement rate (bottom) without andwith surplus redistribution for the property search scenario with eight services andagents using static mixed strategies with the negotiation thread-based mechanism

cause towards the end of the negotiation the offers approach the time-dependent tactic

using the traditional method. It is also interesting to see, that the negotiation thread-

based method respond more effectively compared to the concession-based method.

In addition, the results in Figure 5.12 show that the fuzzy time-dependent constraints

provide an effective means to adjust and coordinate the negotiation strategies generated

by the multistage fuzzy decision approach.

It should be noted that the surplus redistribution algorithm ensures that the decomposed

end-to-end negotiation boundary for the overall composite service is used by all agents

until the end of all atomic service negotiations, i.e. the global negotiation deadline.

Therefore, the agents never propose offers such that their aggregated value exceeds

this global reservation value. For example, if an atomic service agent obtains an agree-

ment before the deadline with a positive surplus that is not redistributed, that surplus is

a gain for the overall utility of the composite service, but can not be used by any other

157


PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

PCrS

PCrM

PCrL

PLrS

PLrM

PLrL

PBrS

PBrM

PBrL

ST

0.05

0.10

0.15

0.20

0.25Uc

ECaS

ECaM

ECaL

ELaS

ELaM

ELaL

EBaS

EBaM

EBaL

ECrS

ECrM

ECrL

ELrS

ELrM

ELrL

EBrS

EBrM

EBrL

ST

0.05

0.10

0.15

0.20

0.25Uc

PCaS

PCaM

PCaL

PLaS

PLaM

PLaL

PBaS

PBaM

PBaL

PCrS

PCrM

PCrL

PLrS

PLrM

PLrL

PBrS

PBrM

PBrL

ST

20

40

60

80

100A

ECaS

ECaM

ECaL

ELaS

ELaM

ELaL

EBaS

EBaM

EBaL

ECrS

ECrM

ECrL

ELrS

ELrM

ELrL

EBrS

EBrM

EBrL

ST

20

40

60

80

100A

Figure 5.11: Average end-to-end utility (top) and agreement rate (bottom) without andwith surplus redistribution for the property search scenario with eight services andagents using static mixed strategies with the concession-based mechanism

agents to increase its chance of an agreement. Since the overall deadline for all other

atomic service agents was not reached at the time of the agreement of that one agent,

all atomic service negotiations would not negotiate using the global reservation value

until the global deadline. It is also important to note that the rate of agreements is more

important in more complex service compositions since negotiations that have already

reached an agreement might no longer be valid if a concurrent negotiation fails for an-

other atomic service in the composition. For that reason, the assumption drawn in this

experiment that the overall service composition obtains zero utility if no agreement can

be get for an atomic service therefore appears to be realistic. Moreover, if the agree-

ments already obtained between atomic service agents and service providers are not

tentative, then costs may be incurred for the already obtained agreements. Hence, the

more agreements are obtained for the atomic services the more the remaining agents

are willing to make concessions in order to ensure an overall agreement for the com-

158

5.6. Related Work

PC PL PB EC EL EB

ST

0.05

0.10

0.15

0.20

0.25Uc

PC PL PB EC EL EB

ST

20

40

60

80

100A

Figure 5.12: Average end-to-end utility (left) and agreement rate (right) without andwith surplus redistribution for the property search scenario with eight services andagents using the multistage fuzzy strategy with different fuzzy constraints

posite service. The surplus redistribution algorithm could be more effective when such

negotiation costs are considered, especially if the composite service structure is more

complex.

5.6 Related Work

The two main mechanisms for establishment of Service Level Agreements for com-

posite service provisioning include service selection and coordinated negotiation. The

problem of QoS based service selection and composition in service oriented applica-

tions is NP-hard [105] and has gained a lot of attention from researchers. In [58], the

authors propose heuristics based service selection algorithms for composite services

by transforming the problem into a multiple choice knapsack problem. Similarly [1]

also propose a heuristics based approach for QoS-based service selection. In [149]

and [4], the authors propose using mixed linear integer programming methods to per-

form service selection. In [24], the authors propose linear programming approach

to flow-based service selection for web service composition taking into account the

workflow pattern of the composite service. In [62] and [23], the authors propose

a genetic algorithm for QoS-based service selection. [49] use a dynamic program-

ming based algorithm for solving the service selection problem. Similarly, [14] and

[95] propose workflow-based SLA aggregation, assuming that all services exist at the

same level. [135] also proposes a model for SLA aggregation in the context of Busi-

159


ness Process Outsourcing while focusing on services at the same level. [52] proposes

a model for describing hierarchical Service Level Agreements in supply chain scen-

arios such as Business Value Networks. [31] also proposes a framework and formal

model for QoS-based Web Service Contracting. Thus, while extensive research has

been carried out on QoS-based Service selection, it may not always be the best solu-

tion for SLA establishment for composite services. This is particularly because of the

unique characteristics of composite service provisioning including a dynamic land-

scape, varying non-functional service capabilities and requirements and changing user

requirements. However, automated negotiation is suited for the on-the-fly adjustment

of QoS requirements and offerings on the basis of the service requestors’ needs and the

service providers’ load and current context. Most of the work on automated QoS nego-

tiation for SLA establishment for web services has been done in the agent community

[28, 95, 100]. However, most of the work has been restricted to QoS negotiation over

the provisioning of a single atomic service as opposed to a composite service. The

idea of using agreements of concurrent negotiations to adjust negotiation strategies of

agents still in negotiation has been explored in [99], but from the viewpoint of chan-

ging strategies to behave tougher in order to avoid unnecessary or to obtain better deals.

In contrast to our service composition scenario the buyer agents may withdraw from

already agreed negotiations in order to accept a better deal with another concurrent

service provider. It hence does not require an agreement for each atomic service as in

our scenario. Furthermore, the redistribution and the change of strategy do not con-

sider complex structures similar to the composite service addressed here. Concurrent

negotiation for multiple web services has also been investigated in [126] where linear

regression is used for coordination in order to predict possible next offer utilities of the

atomic service providers. Similar to [99] agents can withdraw from agreements but

may be penalized, which seems more realistic. Even though the authors claim their ap-

proach works for negotiating multiple QoS parameters, the structure of the composite

service is not taken into account, which is necessary, e.g. for QoS attribute response

time. Moreover, the coordination mechanisms do not consider the redistribution of the

surplus of a successfully finished negotiation to ensure better overall success rates for

the whole composite service. [28] is the first paper which proposed two-layered co-

ordinated negotiation architecture for QoS establishment for composite services. That

work was followed by [16] and [20] that address the problem of utility-decomposition

in compound multi-agent negotiations using a fuzzy projection based utility decom-

160

5.7. Summary

position approach. The limitation of that work is that the authors do not consider the

workflow pattern of the composite service during the initial utility decomposition and

the subsequent surplus redistribution during the process of end-to-end QoS negotiation

for SLA establishment. Our mechanism presents an improvement on the approach in

[16] by taking into consideration the workflow patterns and providing a practical ap-

proach that is applicable according to the existing industry standards such as BPEL for

web services.

5.7 Summary

In this chapter, we have presented a new algorithm for the end-to-end QoS fulfilment

for composite services through concurrent service negotiations, using the decomposi-

tion of global utility boundaries into atomic boundaries, and the redistribution of sur-

plus from successful atomic negotiations among the remaining negotiations. The pro-

posed approach uses a weighted tree-based representation of the composite service,

which enables the iterative and interactive adjustment of the atomic service negoti-

ation boundaries based on the performance of other atomic negotiations, while taking

into account the relative importance of each atomic service, the workflow pattern and

the appropriate aggregation function of the individual QoS attributes. Additionally, in

the case of unsuccessful negotiations or service failures during provisioning, atomic

services can be renegotiated taking into account the agreements already obtained. The

presented algorithm is a practical approach designed to efficiently coordinate mul-

tiple concurrent service negotiations, and can easily be used for service provisioning

on distributed platforms including the Grid and the Cloud. The experimental results

show that the surplus redistribution algorithm can be used with different negotiation

strategies, such as the those discussed in this thesis, and that it obtains significantly

higher agreement rates and utilities for a large range of different negotiation beha-

viours.

161

Chapter 6

Conclusions

This thesis has presented and investigated decision strategies for automated negoti-

ation when only limited knowledge about the negotiation partner or environment is

available. Within the fundamental setting of bilateral negotiation, it has focused on the

strategic concession behaviour in competitive environments in which the agent’s de-

cision models and preferences are private and knowledge can only be derived through

a limited number of interactions or from the environment. In such a setting, it has

first investigated an existing heuristic-based approach for creating multi-tactic negoti-

ation strategies while proposing new mechanisms, and then has presented a novel de-

cision model based on multistage fuzzy decision-making that is able to utilize limited

knowledge about the concession behaviour of the opponent. Because many realistic

negotiation situations are more complex, the thesis has also considered a scenario with

concurrent bilateral negotiations in the domain of service-oriented computing. It has

proposed an effective mechanism for coordinating negotiation strategies in such scen-

arios, and it has also demonstrated the usability of the decision strategies presented and

discussed in this thesis in the more complex and realistic negotiation scenario.

The first heuristic-based mechanism is able to create complex and dynamic concession

behaviour based on different factors, such as time, the behaviour of the opponent or a

resource in the environment, by a linear weighted combination of individual tactics or

decision functions. We have shown that this linear weighted combination of tactics can

expose non-monotonic offer curves when both behaviour-dependent and -independent

tactics are involved, even in cases with static mixing weights and monotonic tactics.

The non-monotonic concession behaviour may result in delayed agreements and signi-

ficantly different outcomes, compared to monotonic offer curves. It has been demon-

strated that the reason for the unpredicted occurrence of such behaviour is the depend-

ence of the imitative tactic on the previous offers of the linear weighted combination as

opposed to the previous offer of the single imitative tactic only. As a result, examples

with one imitative and one non-imitative tactic have shown that the offers generated

by the imitative tactic tend to approach the offers from the non-imitative tactic, with

the result that the linear weighted combination is closer to the non-imitative tactics

towards the end of the negotiation. Two new mixing mechanisms have been presen-

ted that solve this problem, the first based on individual negotiation threads of each

imitative tactic involved and the second on single concessions. We have proved that

both mechanisms produce monotonic concession behaviour, the first for static and the

second even for dynamic weights. It has been shown that both mechanisms treat their

tactics independently and, therefore, represent true linear combination of the tactics.

An experimental evaluation has compared the different mixing mechanisms in differ-

ent negotiation settings for an example with two tactics (imitative and non-imitative).

The results show that the outcomes differ especially in scenarios where both agents use

opposing concession behaviour in their time-dependent tactics.

The second decision mechanism represents a novel approach for modelling an agent’s

negotiation strategy based on multistage fuzzy decision-making. This approach al-

lows an agent to model the dynamics of the negotiation as a fuzzy Markov decision

process that represents the relation between the strategic concession behaviour of the

two agents, while the individual preferences of the agent are expressed via a fuzzy

goal and fuzzy constraints. The agent can use its limited knowledge about the oppon-

ent’s possible concession behaviour, for example, based on a few past interactions, to

create the fuzzy state transitions in which the state-action pairs correspond to the of-

fers and counteroffers of the agents in the negotiation process. We have shown that

by using this model an agent can find the best course of actions given the reference

cases and its fuzzy constraints via traditional dynamic programming techniques. It

has been demonstrated that, by imposing fuzzy constraints on the decision-process, an

agent is able to generate different concession behaviours that consider both the agent’s

soft preferences or conditions and its limited knowledge. The experimental evaluation

shows that the model is able to provide utility gains in many scenarios and negotiation

environments compared to the heuristic-based mixed strategy approach. The limita-

tions and advantages of the the approach have also been discussed. The approach has

163

Chapter 6. Conclusions

been demonstrated with a number of negotiation examples using reference cases and

different fuzzy constraint settings, which further demonstrate that an agent can create

competitive negotiation strategies by using this model based on limited knowledge.

A new coordination mechanism of negotiation strategies has been proposed for more

complex and realistic negotiation scenarios with one-to-many bilateral concurrent ne-

gotiations with limited knowledge. As an example, a scenario in the domain of service-

oriented computing has been used, in which a number of service agents negotiate with

service providers about quality of service (QoS) parameters, such as price or delivery

time, in order to establish service level agreements for a workflow-based compos-

ite service. In this scenario, the individual service agents need to consider the user

provided constraints over the particular QoS attributes in their negotiation strategy.

Therefore, the coordination mechanism uses utility boundary decomposition to derive

the negotiation limits for each atomic service agent, and redistributes the surplus of

successfully finished negotiations among the remaining agents in negotiation in order

to increase the number of compound agreements. By doing so, the mechanism con-

siders the structure of the composite service, the aggregation functions of the particular

QoS attribute and the global constraints. At the same time, it allows the individual ser-

vice agents to remain in control of their concession strategy with the result that they

are able to negotiate competitively and use their limited knowledge about their op-

ponents. The mechanism hence enables the use of any negotiation strategy on the

service level that allows the adjustment of the negotiation boundaries, which can be

done directly through the reservation values in the case of the multi-tactic strategies,

or through the fuzzy constraints in the case of the multistage fuzzy approach. The

experimental evaluation has tested the mechanism with all the decision strategies dis-

cussed in this thesis, by using a simple composite service with two services and the

complex composite service with eight services from the example scenario. Although

the mechanism already showed an effect with the simple composite service, in that

it increased the agreement rate slightly, the end-to-end utility and agreement rate for

the example scenario with eight services was significantly higher than without the sur-

plus redistribution for most strategy groups of all decision mechanisms. The use of all

decision strategies in the evaluation has further demonstrated the applicability of the

decision-making approaches presented in this thesis.

164

6.1. Answers to Research Questions

6.1 Answers to Research Questions

In the following, we answer the research questions stated in Section 1.1.

Mixing mechanisms for multi-tactic negotiation strategies with static and dy-namic weights guaranteeing monotonic concession curves for monotonic tactics

1. What mechanisms can generate monotonic concession curves in multi-tactic ne-

gotiation strategies when monotonic tactics are mixed using static or dynamic

weights?

Two mechanisms have been proposed that produce monotonic offer curves when

monotonic tactics are used. The first mixing mechanism uses individual negoti-

ation threads for each imitative tactic involved in the mix and, therefore, ensures

that the individual imitative tactics use the offers from these threads to calcu-

late their proposals instead of the actual negotiation thread. By doing so, the

tactics involved are independent from the offer proposals of other tactics in the

mix, which ensures monotonic concession behaviour. The negotiation thread-

based mechanism ensures this monotonic behaviour for static mixing weights,

whereas by using dynamic weights non-monotonic concession behaviour can

still occur automatically. The second mechanism combines the concessions of

each tactic involved compared to their individual previous offers. As this is

only possible for the behaviour-independent tactic, the previous offers for the

behaviour-dependent tactics are taken from the negotiation thread. By mixing

the single concessions, an agent always generates monotonic concession curves

for monotonic tactics even in cases with dynamic weights. We have proven these

cases for both mixing mechanisms.

2. How much do outcomes differ when using the alternative mixing mechanisms

compared to the traditional method, and in which scenarios can an agent im-

prove its utility?

The alternative mixing mechanisms produce significantly different outcomes

in scenarios in which both agents use opposing concession behaviour for the

behaviour-independent tactics in the mix. This has been shown by examples and

in the evaluation with two tactics, one imitative and one time-dependent, where

one agent applies a conceder time-dependent tactic and the other a boulware tac-

165


tic. Depending on whether the agent is a seller or a buyer, the utility is shifted

from one agent to the other. For example, when the buyer used a conceder tactic

and the seller a boulware tactic, the buyer gained in utility. Furthermore, we have

observed that both alternative mixing mechanisms preformed in a similar man-

ner in many scenarios, since they both treat their tactics independently. On the

other hand, if a constraint is applied to the traditional linear weighted combina-

tion, the outcomes are still similar to the traditional method without a constraint.

Since the alternative mixing mechanisms truly mix their tactics according to the

weights, the overall concessions are smaller in examples in which the imitative

tactics proposed smaller concessions than the non-imitative one. The imitative

tactic using the traditional method, however, approaches the non-imitative tactic

towards the end of the negotiation, therefore resulting in larger overall conces-

sions of the mixed strategy in such cases.

A decision model for an agent’s strategic concession behaviour in automated ne-gotiation based on multi-stage fuzzy decision making

1. How can an agent model the negotiation process as a multistage fuzzy decision

problem when only limited knowledge about the concession behaviour of the

opponent is available?

When an agent has only limited knowledge about the concession behaviour of

an opponent, for example, in the form of reference cases from a few past inter-

actions, it can use the offer-response patterns to create a model with fuzzy state

transitions, in which the offers and counteroffers correspond to state-action pairs

that lead to the next state. Based on this model an agent can impose its pref-

erences in the form of a fuzzy goal, which represents the preferred order of all

possible states, and fuzzy constraints on the decision process. The negotiation

process is hence modelled as a fuzzy Markov decision process which allows the

agents to use fuzzy dynamic programming to generate the best course of actions,

given the limited knowledge available and the soft preferences. In order to en-

able the use of a limited number of cases, the agent has to fuzzify the cases by

interpolating the missing states between the states of each case in the expec-

ted goal matrix. This ensures that a small number of cases can be used for the

decision-making during negotiation.

166

6.1. Answers to Research Questions

2. What are the advantages and disadvantages of the proposed multistage fuzzy

decision model compared to the heuristic-based or other approaches, and in

which scenarios can an agent gain in utility by using this approach?

The advantages of this approach result from the fuzzy representation of the state

transitions that allow the agents to use a limited number of reference cases to

generate the transition model for multistage decision-making during negotiation.

The fuzzy constraints further enable an agent to impose its preferred soft strategy

or conditions on the decision process, which provides flexibility in terms of its

application in different and more realistic negotiation scenarios. In addition, the

state-action form of the approach enables the use of traditional dynamic pro-

gramming to find the best course of actions. However, a limitation of the ap-

proach is the large number of states required for larger negotiation spaces, as it

increases the computational cost of the algorithm. The modelling of the con-

cession behaviour of the opponent may not be sufficient in some scenarios in

which the opponent’s negotiation strategy also depends on other factors, such as

a resource, or has changing reservation values or deadlines, which can not be

anticipated by this model. We have shown in the evaluation of the approach that

an agent can gain in utility significantly in many scenarios. This, however, de-

pends on the chosen cases and the negotiation environment. For example, when

the agent uses cases which suggest larger concessions, the resulting strategy is

more likely to achieve a higher number of agreements, but with lower utilit-

ies, especially when agents have different deadlines. On the other hand, if the

cases are competitive, in that they propose smaller concessions, the agent gains

in utility in scenarios with equal deadlines and larger overlaps of negotiation in-

tervals. We have also shown that by using fuzzy constraints generated from the

time-dependent decision functions, an agent can further make its strategy more

or less competitive. For example, in the scenario with equal deadlines the boul-

ware fuzzy constraints significantly increased the agent’s utility, whereas in the

scenario with different deadlines the same constraints led to a smaller number of

agreements due to the more competitive resulting negotiation strategy.

A mechanism for coordinating negotiation strategies in concurrent negotiationsbased on utility boundary decomposition and surplus redistribution

1. How can negotiation strategies be efficiently coordinated in more complex nego-

167


tiation scenarios with many bilateral concurrent negotiations?

In concurrent bilateral negotiation scenarios, the individual agents typically need

to consider global constraints during their negotiation encounters. This has been

shown by means of an example scenario in the domain of service-oriented com-

puting, in which a number of agents concurrently negotiate service level agree-

ments with service providers in order to establish a composite service. For each

agent, the negotiation boundary, i.e. the reservation value for their negotiation

strategy, is derived by decomposing the global utility boundary based on the

end-to-end quality of service constraints. The method of utility boundary de-

composition uses a tree-based mechanism that represents the structure of the

composite service including the aggregation functions for the parallel and se-

quential service flows for the particular quality of service parameter. The same

tree representation can then be used to redistribute the surplus of a successfully

finished negotiation among the remaining service agents still in negotiation, in

order to ensure that all agents consider the global reservation value until the end

of the negotiation. This increases the chance of achieving compound agreements

in order to increase the end-to-end utility for the composite service. Because the

mechanism only coordinates the negotiation boundaries, it allows the agents to

use their own concession strategies and use their available limited knowledge to

negotiate competitively at the same time.

2. Are the proposed negotiation decision-models and mechanisms applicable in

real world domains such as service-oriented computing?

We have shown by means of the example scenario entitled ’specialized prop-

erty search’ that the service agents can apply the multi-tactic strategies or the

multistage fuzzy strategies to negotiate about the quality of service paramet-

ers with potential service providers. In this scenario which incorporates eight

services, an agent is assigned to each service and negotiates with a number of

service providers using these negotiation strategies. In addition, the strategies

can also be coordinated by adjusting the reservation values of the multi-tactic

strategies or the fuzzy constraints of the multistage fuzzy strategy in order to

consider the global QoS constraints in the scenario. The evaluation further

demonstrated the applicability and usability of all decision strategies in the more

complex and realistic service negotiation scenario.

168

6.2. Outlook and Future Work

6.2 Outlook and Future Work

The proposed multistage fuzzy decision model represents a model-based approach that

requires the agent to have an initial model based on the opponent’s behaviour before

the negotiation starts. An interesting project for the future is to use learning mechan-

isms, such as fuzzy reinforcement learning [9], to learn the model during consecutive

interactions. Although such an approach based on Q-learning has already been pro-

posed for bilateral negotiation [26, 103], it has not been applied using the model of

multistage fuzzy decision-making. In addition, a further interesting approach would

be to model the negotiation environment, such as the reservation values and the dead-

lines of the opponent, instead of its concession behaviour. This, however, requires

domain knowledge or a larger number of interactions with the negotiation partner.

In the case of the coordination of one-to-many negotiation scenarios, the presented

approach of redistributing surpluses after successfully finished negotiations could be

extended in such a way that it coordinates the negotiation strategies at each offer pro-

posal of an atomic service agent. This means that not only the negotiation boundary but

also the concession behaviour of the individual service agents is provided by the co-

ordination mechanism. However, in such a scenario, the atomic service agents would

lose the ability to apply their own negotiation strategies. Moreover, it would be in-

teresting to investigate distributed coordination techniques for the individual service

agents, for instance, by using distributed constraints methods without the requirement

for a central coordination agent.

169

Bibliography

[1] M. Alrifai, T. Risse, P. Dolog, and W. Nejdl. A scalable approach for qos-basedweb service selection. In G. Feuerlicht and W. Lamersdorf, editors, Service-Oriented Computing - ICSOC 2008 Workshops, volume 5472 of Lecture Notesin Computer Science, pages 190–199. Springer Berlin / Heidelberg, 2009.

[2] B. An, V. Lesser, D. Irwin, and M. Zink. Automated negotiation with decommit-ment for dynamic resource allocation in cloud computing. In Proceedings of the9th International Conference on Autonomous Agents and Multiagent Systems,AAMAS ’10, pages 981–988, Richland, SC, 2010. International Foundation forAutonomous Agents and Multiagent Systems.

[3] R. Arapoglou, K. Kolomvatsos, and S. Hadjiefthymiades. Buyer agent decisionprocess based on automatic fuzzy rules generation methods. In Proceedings ofthe IEEE Int. Conf. on Fuzzy Systems, 2010.

[4] D. Ardagna and B. Pernici. Adaptive service composition in flexible processes.IEEE Trans. Softw. Eng., 33(6):369–384, 2007.

[5] L. M. Ausubel, P. Cramton, and R. J. Deneckere. Bargaining with incom-plete information. In R. j. Aumann and S. Hart, editors, Handbook of GameTheory, volume 3 of Handbook of Game Theory with Economic Applications,chapter 50, pages 1897 – 1945. Elsevier, 2002.

[6] B. Banerjee, S. Debnath, and S. Sen. Using bayesian network to aid negotiationsamong agents. In Working Notes of the AAAI-99 Workshop on Negotiation: Set-tling Conflicts and Identifying Opportunities, pages 44–49. AAAI Press, 1999.

[7] R. Bellman and L. Zadeh. Decision making in a fuzzy environment. Manage-ment Science, 17(4):141–164, December 1970.

Bibliography

[8] B. Benatallah, M. Dumas, and Z. Maamar. Definition and execution of com-posite web services: The self-serv project. Data Engineering Bulletin, 25:2002,2002.

[9] H. R. Berenji. Fuzzy q-learning: A new approach for fuzzy dynamic program-ming. In IEEE World Congress on Computational Intelligence., Proceedings ofthe Third IEEE Conference on Fuzzy Systems, pages 486–491 vol.1, June 1994.

[10] T. Berners-Lee, J. Hendler, and O. Lassila. The semantic web. Scientific Amer-ican, 284(5):34–43, May 2001.

[11] M. Bichler and K.-J. Lin. Service-oriented computing. Computer, 39(3):99–101, 2006.

[12] K. Binmore and N. Vulkan. Applying game theory to automated negotiation.Netnomics, 1(1):1–9, 1999.

[13] K. G. Binmore. Fun and Games, A Text on Game Theory. D.C. Heath andCompany, 1992.

[14] M. B. Blake and D. J. Cummings. Workflow composition of service level agree-ments. In IEEE International Conference on Services Computing, pages 138–145, July 2007.

[15] P. Braun, J. Brzostowski, G. Kersten, J. Kim, R. Kowalczyk, S. Strecker, andR. Vahidov. e-Negotiation Systems and Software Agents: Methods, Models, andApplications, chapter Intelligent Decision-making Support Systems. Springer,2006.

[16] J. Brzostowski, M. B. Chhetri, and R. Kowalczyk. Three decision-makingmechanisms to facilitate negotiation of service level agreements for web ser-vice compositions. In N. Jennings, G. Kersten, A. Ockenfels, and C. Weinhardt,editors, Negotiation and Market Engineering, number 06461 in Dagstuhl Sem-inar Proceedings, Dagstuhl, Germany, 2007. Internationales Begegnungs- undForschungszentrum für Informatik (IBFI), Schloss Dagstuhl, Germany.

[17] J. Brzostowski and R. Kowalczyk. On possibilistic case-based reasoning forselecting partners in multi-agent negotiation. In X. Y. Geoffrey I. Webb, editor,Proceedings of the 17th Australian Joint Conf. on Artificial Intelligence, LNCS.Springer, 2005.

[18] J. Brzostowski and R. Kowalczyk. Adaptive negotiation with on-line predic-tion of opponent behaviour in agent-based negotiations. In Proceedings of the

171

Bibliography

IEEE/WIC/ACM Int. Conf. on Intelligent Agent Technology, pages 263–269,Washington, DC, USA, 2006. IEEE Computer Society.

[19] J. Brzostowski and R. Kowalczyk. Predicting partner’s behaviour in agent nego-tiation. In Proceedings of the Fifth Int. Joint Conf. on Autonomous Agents andMultiagent Systems, pages 355–361, New York, NY, USA, 2006. ACM.

[20] J. Brzostowski and R. Kowalczyk. On fuzzy projection-based utility decompos-ition in compound multi-agent negotiations. In IFSA ’07: Proceedings of the12th international Fuzzy Systems Association world congress on Foundationsof Fuzzy Logic and Soft Computing, pages 757–766, Berlin, Heidelberg, 2007.Springer-Verlag.

[21] J. W. Brzostowski. Predictive Decision-making Mechanisms based on Off-lineand On-line Reasoning. PhD thesis, Swinburne University of Technology, 2007.

[22] R. Buyya, C. S. Yeo, S. Venugopal, J. Broberg, and I. Brandic. Cloud computingand emerging it platforms: Vision, hype, and reality for delivering computingas the 5th utility. Future Gener. Comput. Syst., 25:599–616, June 2009.

[23] G. Canfora, M. Di Penta, R. Esposito, and M. L. Villani. An approach for qos-aware service composition based on genetic algorithms. In Proceedings of the2005 conference on Genetic and evolutionary computation, GECCO ’05, pages1069–1075, New York, NY, USA, 2005. ACM.

[24] V. Cardellini, E. Casalicchio, V. Grassi, and F. Lo Presti. Flow-based serviceselection forweb service composition supporting multiple qos classes. In Pro-ceedings of the IEEE International Conference on Web Services, pages 743–750.IEEE Computer Society, July 2007.

[25] H. L. Cardoso and E. Oliveira. Using and evaluating adaptive agents for elec-tronic commerce negotiation. In M. Monard and J. Sichman, editors, Advancesin Artificial Intelligence, volume 1952 of Lecture Notes in Computer Science,pages 96–105. Springer Berlin / Heidelberg, 2000.

[26] H. L. Cardoso, M. Schaefer, and E. Oliveira. A multi-agent system for elec-tronic commerce including adaptive strategic behaviours. In Proceedings of the9th Portuguese Conference on Artificial Intelligence, EPIA ’99, pages 252–266,London, UK, 1999. Springer-Verlag.

[27] K. Chatterjee and L. Samuelson. Bargaining under two-sided incomplete in-formation: the unrestricted offers case. Operations Research, 36:605–618,1988.

172

Bibliography

[28] M. B. Chhetri, J. Lin, S. Goh, J. Y. Zhang, R. Kowalczyk, and J. Yan. A co-ordinated architecture for the agent-based service level agreement negotiationof web service composition. In Proceedings of the Australian Software Engin-eering Conference, ASWEC ’06, pages 90–99, Washington, DC, USA, 2006.IEEE Computer Society.

[29] R. M. Coehoorn and N. R. Jennings. Learning on opponent’s preferences tomake effective multi-issue negotiation trade-offs. In Proceedings of the 6thinternational conference on Electronic commerce, ICEC ’04, pages 59–68, NewYork, NY, USA, 2004. ACM.

[30] J. Collins, W. Ketter, and M. Gini. A multi-agent negotiation testbed for con-tracting tasks with temporal and precedence constraints. Int. J. Electron. Com-merce, 7:35–57, October 2002.

[31] M. Comuzzi and B. Pernici. A framework for qos-based web service contract-ing. ACM Transactions on the Web, 3(3):1–52, 2009.

[32] E. V. de Vliert. Complex interpersonal conflict behaviour: theoretical frontiers.Essays in social psychology. Psychology Press, East Sussex, UK, 1997.

[33] D. Dubois, H. Fargier, J. Lang, H. Prade, and R. Sabbadin. Qualitative decisiontheory and multistage decision making - a possibilistic approach. In Proceed-ings of the European Workshop on Fuzzy Decision Analysis for Management,Planning and Optimization, 1996.

[34] D. Dubois and H. Prade. Possibility theory as a basis for qualitative decisiontheory. In Proceedings of the International Joint Conference on Artificial Intel-ligence, IJCAI’95, 1995.

[35] D. Dubois and H. Prade. Towards possibilistic decision theory. In Selectedpapers from the Workshop on Fuzzy Logic in Artificial Intelligence, TowardsIntelligent Systems, pages 240–251, London, UK, 1997. Springer-Verlag.

[36] H. Ehtamo, R. P. Hämäläinen, P. Heiskanen, J. Teich, M. Verkama, andS. Zionts. Generating pareto solutions in a two-party setting: Constraint pro-posal methods. Manage. Sci., 45:1697–1709, December 1999.

[37] H. Ehtamo and R. P. Hämäläinen. Interactive multiple-criteria methods forreaching pareto optimal agreements in negotiations. Group Decision and Ne-gotiation, 10:475–491, 2001.

173

Bibliography

[38] H. Ehtamo, M. Verkama, and R. P. Hamalainen. How to select fair improvingdirections in a negotiation model over continuous issues. IEEE Transactions onSystems, Man, and Cybernetics - Part C: Applications and Reviews, 29(1):26–33, Feb. 1999.

[39] A. Elfatatry and P. Layzell. Negotiation in service-oriented environments. Com-munications of the ACM, 47(8):103–108, 2004.

[40] T. Erl. Service-Oriented Architecture: Concepts, Technology, and Design. Pren-tice Hall PTR, Upper Saddle River, NJ, USA, 2005.

[41] A. O. Esogbue. The computational complexity of some fuzzy dynamic pro-grams. Computers & Mathematics with Applications, 37(11-12):47 – 51, 1999.

[42] P. Faratin. Automated Service Negotiation Between Autonomous ComputationalAgents. PhD thesis, University of London, 2000.

[43] P. Faratin, M. Klein, H. Sayama, and Y. Bar-Yam. Simple negotiating agentsin complex games: Emergent equilibria and dominance of strategies. In J.-J.Meyer and M. Tambe, editors, Intelligent Agents VIII, volume 2333 of LNAI,pages 367–376, 2002.

[44] P. Faratin, C. Sierra, and N. R. Jennings. Negotiation decision functionsfor autonomous agents. Robotics and Autonomous Systems, 24(3-4):159–182,1998.

[45] S. S. Fatima, M. Wooldridge, and N. R. Jennings. Multi-issue negotiation undertime constraints. In Proceedings of the first international joint conference onAutonomous agents and multiagent systems, AAMAS ’02, pages 143–150, NewYork, NY, USA, 2002. ACM.

[46] S. S. Fatima, M. Wooldridge, and N. R. Jennings. Bargaining with incompleteinformation. Annals of Mathematics and Artificial Intelligence, 44(3):207–232,2005.

[47] A. Fink. Supply chain coordination by means of automated negotiations. InProceedings of the Proceedings of the 37th Annual Hawaii International Con-ference on System Sciences (HICSS’04) - Track 3 - Volume 3, HICSS ’04, pages30070.3–, Washington, DC, USA, 2004. IEEE Computer Society.

[48] I. Foster and C. Kesselman. The grid: blueprint for a new computing infrastruc-ture. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1999.

174

Bibliography

[49] Y. Gao, J. Na, B. Zhang, L. Yang, and Q. Gong. Optimal web services selectionusing dynamic programming. In Proceedings of the 11th IEEE Symposium onComputers and Communications, ISCC ’06, pages 365–370, June 2006.

[50] E. H. Gerding, D. D. Bragt, and J. A. L. Poutre. Scientific approaches and tech-niques for negotiation. a game theoretic and artificial intelligence perspective.Technical report, Centre for Mathematics and Computer Science, Amsterdam,The Netherlands, Amsterdam, The Netherlands, The Netherlands, 2000.

[51] V. Grassi and S. Patella. Reliability prediction for service-oriented computingenvironments. IEEE Internet Computing, 10(3):43–49, 2006.

[52] I. Haq, A. Huqqani, and E. Schikuta. Aggregating hierarchical service levelagreements in business value networks. In Proceedings of the 7th InternationalConference on Business Process Management, BPM ’09, pages 176–192, Ber-lin, Heidelberg, 2009. Springer-Verlag.

[53] J. C. Harsanyi. Approaches to the bargaining problem before and after the the-ory of games : A critical discussion of zeuthen’s , hicks’, and nash’s theories.Econometrica, 24(2):144–157, 1956.

[54] M. He and N. Jennings. Designing a successful trading agent: A fuzzy setapproach. IEEE Transactions on Fuzzy Systems, 12(3):389–410, 2004.

[55] M. He, N. Jennings, and H.-F. Leung. On agent-mediated electronic commerce.Knowledge and Data Engineering, IEEE Transactions on, 15(4):985 – 1003,july-aug 2003.

[56] K. Hindriks and D. Tykhonov. Opponent modelling in automated multi-issuenegotiation using bayesian learning. In Proceedings of the 7th internationaljoint conference on Autonomous agents and multiagent systems, AAMAS ’08,pages 331–338, Richland, SC, 2008. International Foundation for AutonomousAgents and Multiagent Systems.

[57] C. Hou. Predicting agents tactics in automated negotiation. In Proceedings ofthe IEEE/WIC/ACM International Conference on Intelligent Agent Technology,IAT ’04, pages 127–133, 2004.

[58] A. F. M. Huang, C.-W. Lan, and S. J. H. Yang. An optimal qos-based webservice selection scheme. Information Science, 179(19):3309–3322, 2009.

[59] T. Ito, H. Hattori, and M. Klein. Multi-issue negotiation protocol for agents:exploring nonlinear utility spaces. In Proceedings of the 20th international

175

Bibliography

joint conference on Artifical intelligence, pages 1347–1352, San Francisco, CA,USA, 2007. Morgan Kaufmann Publishers Inc.

[60] T. Ito, M. Klein, and H. Hattori. A multi-issue negotiation protocol amongagents with nonlinear utility functions. Multiagent Grid Syst., 4:67–83, January2008.

[61] S. Iwamoto, K. Tsurusaki, and T. Fujita. Conditional decision-making in fuzzyenvironment. Journal of the Operations Research, 42(2):198–218, 1999.

[62] M. C. Jaeger and G. Muehl. Qos-based selection of services: The implement-ation of a genetic algorithm. In T. Braun G. Carle and B. Stiller, editors, KiVS2007 Workshop: Service-Oriented Architectures und Service-Oriented Comput-ing, pages 359–370, Bern, Switzerland, Mar. 2007. VDE Verlag.

[63] M. C. Jaeger, G. Rojec-Goldmann, and G. Muehl. Qos aggregation for webservice composition using workflow patterns. In Proceedings of the 8th IEEEInternational Enterprise Distributed Object Computing Conference, EDOC ’04,2004.

[64] N. R. Jennings, P. Faratin, A. R. Lomuscio, S. Parsons, M. Wooldridge, andC. Sierra. Automated negotiation : Prospects, methods and challenges. GroupDecision and Negotiation, 10:199–215, 2001.

[65] N. R. Jennings, P. Faratin, T. J. Norman, P. O’Brien, B. Odgers, and J. L. Alty.Implementing a business process management system using adept: A real-worldcase study. Int. Journal of Applied Artificial Intelligence, 14(5):421–463, 2000.

[66] J. R. Jiao, X. You, and A. Kumar. An agent-based framework for collaborativenegotiation in the global manufacturing supply chain network. Robotics andComputer-Integrated Manufacturing, 22:239–255, 2006.

[67] Y. Jun, K. Ryszard, L. Jian, C. M. B., G. S. Keong, and Z. Jianying. Autonomousservice level agreement negotiation for service composition provision. FutureGener. Comput. Syst., 23(6):748–759, 2007.

[68] Kacprzyk. Multistage Fuzzy Control: A Model-based Approach to Fuzzy Con-trol and Decision Making. John Wiley & Sons, Inc., New York, NY, USA,1997.

[69] J. Kacprzyk. Control of a non-fuzzy system in a fuzzy environment with fuzzytermination time. Systems Science, 3:320–331, 1977.

176

Bibliography

[70] E. Kalai and M. Smorodinsky. Other solutions to nash ’ s bargaining problem.Society, The Econometric, 43(3):513–518, 2010.

[71] J. Kennan and R. Wilson. Bargaining with private information. Journal ofEconomic Literature, 31(1):45–104, 1993.

[72] R. Khalaf and F. Leymann. On web services aggregation. Lecture notes incomputer science, 2819:1–13, 2003.

[73] M. Klein, P. Faratin, H. Sayama, and Y. Bar-Yam. Negotiating complex con-tracts. SSRN Electronic Journal, 12(2):111–125, 2001.

[74] M. Klein, P. Faratin, H. Sayama, and Y. Bar-Yam. Negotiation algorithms forcollaborative design settings. In D. Braha, A. A. Minai, and Y. Bar-Yam, editors,Complex Engineered Systems, volume 14 of Understanding Complex Systems,pages 246–261. Springer Berlin / Heidelberg, 2006.

[75] R. Kowalczyk. On negotiation as a distributed fuzzy constraint satisfaction prob-lem. In Proceedings of the Third International Symposium on Soft Computingfor Industry of the World Automation Congress, pages 631–637, 2000.

[76] R. Kowalczyk. Fuzzy e-negotiation agents. Soft Computing - A Fusion of Found-ations, Methodologies and Applications, 6(5):337–347, 2002.

[77] R. Kowalczyk and V. Bui. On constraint-based reasoning in e-negotiationagents. In Agent-Mediated Electronic Commerce III, Current Issues in Agent-Based Electronic Commerce Systems (includes revised papers from AMEC 2000Workshop), pages 31–46, London, UK, 2001. Springer-Verlag.

[78] S. Kraus. Negotiation and cooperation in multi-agent environments. ArtificialIntelligence, 94(1-2):79–97, 1997.

[79] G. Lai, C. Li, and K. Sycara. Efficient multi-attribute negotiation with incom-plete information. Group Decision and Negotiation, 15(5):511–528, 2006.

[80] G. Lai, C. Li, K. Sycara, and J. A. Giampapa. Literature review on multi-attribute negotiations. Technical Report CMU-RI-TR-04-66, Robotics Institute,Carnegie Mellon University, Pittsburgh, PA, Dec. 2004.

[81] G. Lai, K. Sycara, and C. Li. A decentralized model for multi-attribute nego-tiations with incomplete information and general utility functions. In T. Ito,H. Hattori, M. Zhang, and T. Matsuo, editors, Rational, Robust, and SecureNegotiations in Multi-Agent Systems, volume 89 of Studies in ComputationalIntelligence. Springer Berlin / Heidelberg, 2008.

177

Bibliography

[82] R. Lai and L. Menq-Wen. Agent negotiation as fuzzy constraint processing. InProceedings of the IEEE International Conference on Fuzzy Systems, FUZZ-IEEE’02, 2002.

[83] R. Lewicki, B. Barry, D. Saunders, and J. Minton. Negotiation. McGraw Hill,New York, 2003.

[84] C. Li, J. A. Giampapa, and K. Sycara. A review of research literature on bi-lateral negotiations. Technical Report CMU-RI-TR-03-41, Robotics Institute,Pittsburgh, PA, November 2003.

[85] C. Li, J. A. Giampapa, and K. Sycara. Bilateral negotiation decisions withuncertain dynamic outside options. IEEE Transactions on Systems, Man, andCybernetics, Part C: Special Issue on Game-theoretic Analysis and StochasticSimulation of Negotiation Agents, 36(1):31–44, January 2006.

[86] J. Li and R. Yahyapour. Negotiation strategies for grid scheduling. In Proceed-ings of the First International Conference on Advances in Grid and PervasiveComputing, GPC ’06, Taichung, Taiwan, May 2006.

[87] M.-W. Lin, K. R. Lai, and T.-J. Yu. Fuzzy constraint-based agent negotiation.J. Comput. Sci. Technol., 20(3):319–330, 2005.

[88] F. Lopes and H. Coelho. Bilateral negotiation in a multi-agent supply chainsystem. In 11th International Conference on Electronic Commerce and WebTechnologies (EC-Web 10), pages 195–206, 30 August-3 September 2010.

[89] F. Lopes, M. Wooldridge, and A. Q. Novais. Negotiation among autonomouscomputational agents: principles, analysis and challenges. Artif. Intell. Rev.,29:1–44, March 2008.

[90] X. Luo, N. R. Jennings, and N. Shadbolt. Acquiring tradeoff preferences forautomated negotiations: A case study. In 5th International Workshop on Agent-Mediated E-Commerce, pages 41–48, 2003.

[91] X. Luo, N. R. Jennings, N. Shadbolt, H.-f. Leung, and J. H.-m. Lee. Afuzzy constraint based model for bilateral, multi-issue negotiations in semi-competitive environments. Artificial Intelligence, 148(1-2):53– 102, 2003.

[92] X. Luo, J. H. Lee, H. Leung, and N. R. Jennings. Prioritised fuzzy constraintsatisfaction problems: axioms, instantiation and validation. Int Journal of FuzzySets and Systems, 136(2):155–188, 2003.

178

Bibliography

[93] N. Matos and C. Sierra. Evolutionary computing and negotiating agents. In FirstInternational Workshop on Agent Mediated Electronic Trading on Agent Medi-ated E-Commerce, volume 1571, pages 126–150, London, UK, 1999. Springer.

[94] N. Matos, C. Sierra, and N. R. Jennings. Determining successful negotiationstrategies: An evolutionary approach. In 3rd Int. Conf. on Multi-Agent Systems,page 182, Washington, DC, USA, 1998. IEEE Computer Society.

[95] C. Napoli. Software agent negotiation for service composition. In Proceedingsof the 1st KES International Symposium on Agent and Multi-Agent Systems,KES-AMSTA ’07, pages 456–465, Berlin, Heidelberg, 2007. Springer-Verlag.

[96] V. Narayanan and N. R. Jennings. An adaptive bilateral negotiation model fore-commerce settings. In Proceedings of the 7th International IEEE Conferenceon E-Commerce Technology. IEEE Computer Society, 2005.

[97] J. Nash. Two-person cooperative games. Econometrica: Journal of the Econo-metric Society, 21(1):128–140, 1953.

[98] J. F. Nash. The bargaining problem. Econometrica, 18(2):155, Apr. 1950.

[99] T. D. Nguyen and N. R. Jennings. Concurrent bi-lateral negotiation in agentsystems. In Proceedings of the 14th International Workshop on Database andExpert Systems Applications, DEXA ’03, page 844, Washington, DC, USA,2003. IEEE Computer Society.

[100] E. Nitto, M. Penta, A. Gambi, G. Ripa, and M. L. Villani. Negotiation of servicelevel agreements: An architecture and a search-based approach. In Proceedingsof the 5th international conference on Service-Oriented Computing, ICSOC ’07,pages 295–306, Berlin, Heidelberg, 2007. Springer-Verlag.

[101] E. Oliveira and A. P. Rocha. Agents advanced features for negotiation in elec-tronic commerce and virtual organisations formation processes. In Agent Medi-ated Electronic Commerce, The European AgentLink Perspective., pages 78–97,London, UK, 2001. Springer-Verlag.

[102] J. R. Oliver. A machine learning approach to automated negotiation and pro-spects for electronic commerce. Journal of Management Information Systems,13(3), 1997.

[103] M. Oprea. An adaptive negotiation model for agent-based electronic commerce.Studies in Informatics and Control, 11(3):271–279, 2002.

179

Bibliography

[104] M. P. Papazoglou, P. Traverso, S. Dustdar, F. Leymann, and B. J. Krämer.Service-oriented computing: A research roadmap. In F. Cubera, B. J. Krämer,and M. P. Papazoglou, editors, Service Oriented Computing (SOC), number05462 in Dagstuhl Seminar Proceedings, Dagstuhl, Germany, 2006. Interna-tionales Begegnungs- und Forschungszentrum für Informatik (IBFI), SchlossDagstuhl, Germany.

[105] D. Pisinger. Algorithms for Knapsack Problems. PhD thesis, Dept. of ComputerScience, University of Copenhagen, 1995.

[106] D. Pruitt and S. Kim. Social conflict: escalation, stalemate, and settlement.McGraw Hill, New York, 2004.

[107] I. Rahwan, R. Kowalczyk, and H. H. Pham. Intelligent agents for auto-mated one-to-many e-commerce negotiation. In Proceedings of the twenty-fifth Australasian conference on Computer science, ACSC ’02, pages 197–204,Darlinghurst, Australia, Australia, 2002. Australian Computer Society, Inc.

[108] H. Raiffa. The art and science of negotiation. Harvard University Press Cam-bridge, Mass., Cambridge, Mass., 1982.

[109] J. Richter, M. Baruwal Chhetri, R. Kowalczyk, and Q. Bao Vo. Establishingcomposite slas through concurrent qos negotiation with surplus redistribution.Concurrency and Computation: Practice and Experience, pages n/a–n/a, 2011.

[110] J. Richter, M. B. Chhetri, R. Kowalczyk, B. Q. Vo, M. A. Talib, and A. Colman.Utility decomposition and surplus redistribution in composite sla negotiation. InProceedings of the 2010 IEEE International Conference on Services Comput-ing, SCC ’10, pages 627–630, Washington, DC, USA, 2010. IEEE ComputerSociety.

[111] J. Richter, M. Klusch, and R. Kowalczyk. A multistage fuzzy decision approachfor modelling adaptive negotiation strategies. In Proceedings of the IEEE Inter-national Conference on Fuzzy Systems, pages 1–8, 2010.

[112] J. Richter, M. Klusch, and R. Kowalczyk. On monotonic mixed tactics andstrategies for multi-issue negotiation. In Proceedings of the 9th InternationalConference on Autonomous Agents and Multiagent Systems, AAMAS ’10,pages 1609–1610. International Foundation for Autonomous Agents and Mul-tiagent Systems, 2010.

180

Bibliography

[113] J. Richter, M. Klusch, and R. Kowalczyk. Monotonic mixing of decisionstrategies for agent-based bargaining. In Proceedings of the Ninth German Con-ference on Multi-Agent System Technologies, Lecture Notes in Artificial Intelli-gence, pages 113–124. Springer Berlin / Heidelberg, 2011.

[114] J. Richter and R. Kowalczyk. New mechanisms for mixing time- and behaviour-dependent tactics in negotiation strategies. In IEEE/WIC/ACM InternationalConference on Web Intelligence and Intelligent Agent Technology (WI-IAT ’08),volume 2, pages 451–455, 2008.

[115] J. Richter and R. Kowalczyk. Mixing behaviour-dependent and -independenttactics in multi-issue negotiation. In Proceedings of the 8th International Con-ference on Autonomous Agents and Multiagent Systems, AAMAS ’09, pages1219–1220. International Foundation for Autonomous Agents and MultiagentSystems, 2009.

[116] J. Richter, R. Kowalczyk, and M. Klusch. Multistage fuzzy decision making inbilateral negotiation with finite termination times. In A. E. Nicholson and X. Li,editors, Proceedings of the 22nd Australasian Joint Conference on Advancesin Artificial Intelligence, volume 5866 of Lecture Notes in Computer Science,pages 21–30, Berlin, Heidelberg, 2009. Springer.

[117] V. Robu and H. Poutré. Learning the Structure of Utility Graphs Used in Multi-issue Negotiation through Collaborative Filtering, pages 192–206. Springer-Verlag, Berlin, Heidelberg, 2009.

[118] V. Robu, D. J. A. Somefun, and J. A. La Poutré. Modeling complex multi-issuenegotiations using utility graphs. In Proceedings of the fourth international jointconference on Autonomous agents and multiagent systems, AAMAS ’05, pages280–287, New York, NY, USA, 2005. ACM.

[119] R. Ros and C. Sierra. Experimental analysis of negotiation meta strategies. InProceedings of the 4th international joint conference on Autonomous agentsand multiagent systems, AAMAS ’05, pages 1209–1210, New York, NY, USA,2005. ACM.

[120] R. Ros and C. Sierra. A negotiation meta strategy combining trade-off andconcession moves. Autonomous agent and multiagent systems, 12:163–181,2006.

181

Bibliography

[121] J. S. Rosenschein and G. Zlotkin. Rules of encounter: designing conventions forautomated negotiation among computers. MIT Press, Cambridge, MA, USA,1994.

[122] A. Rubinstein. Perfect equilibrium in a bargaining model. Econometrica:Journal of the Econometric Society, 50(1):97–109, 1982.

[123] A. Rubinstein. A bargaining mo del under in omplete information. Economet-rica, 53:1151–1172, 1985.

[124] R. Sabbadin. Possibilistic markov decision processes. Engineering Applicationsof Artificial Intelligence, 14(3):287–300, June 2001.

[125] R. Sabbadin, H. Fargier, and J. Lang. Towards qualitative approaches to multi-stage decision making. International Journal of Approximate Reasoning, 19(3-4):441–471, October-November 1998.

[126] B. Shi and K. M. Sim. Coordination and concurrent negotiation for multiple webservices procurement. In S. I. Ao, O. Castillo, C. Douglas, D. D. Feng, and J.-A. Lee, editors, Proceedings of the International MultiConference of Engineersand Computer Scientists, pages 950–955, Hong Kong, Mar. 2009. InternationalAssociation of Engineers, Newswood Limited.

[127] K. M. Sim and S. Y. Wang. Flexible negotiation agent with relaxed decisionrules. Systems, Man, and Cybernetics, Part B: Cybernetics, IEEE Transactionson, 34(3):1602–1608, 2004.

[128] I. Stahl. Bargaining Theory. Stockholm School of Economics, Stockholm,Sweden, 1972.

[129] W. Streitberger, S. Hudert, T. Eymann, B. Schnizler, F. Zini, and M. Catalano.On the simulation of grid market coordination approaches. Journal of GridComputing, 6:349–366, 2008.

[130] J. Sutton. Non-cooperative bargaining theory: An introduction. The Review ofEconomic Studies, 53(5):709–724, 1986.

[131] K. Sycara. Resolving goal conflicts via negotiation. In Proceedings of theSeventh National Conference on Artificial Intelligence, 1988.

[132] K. Sycara. Utility theory in conflict resolution. Annals of Operations Research,12(1):65–83, Dec. 1988.

182

Bibliography

[133] G. Tesauro. Efficient search techniques for multi-attribute bilateral negotiationstrategies. In Proceedings of the 3rd International Symposium on ElectronicCommerce, pages 30 – 36, 2002.

[134] F. Teuteberg and K. Kurbel. Anticipating agents’ negotiation strategies in an e-marketplace using belief models. In W. Abramowicz, editor, Proceeding of the5th International Conference on Business Information Systems, Poznan, Poland,2002.

[135] T. Unger, F. Leymann, S. Mauchart, and T. Scheibler. Aggregation of servicelevel agreements in the context of business processes. In Proceedings of the12th International IEEE Enterprise Distributed Object Computing Conference,EDOC ’08, pages 43–52, Washington, DC, USA, 2008. IEEE Computer Soci-ety.

[136] C. Vecchiola, S. Pandey, and R. Buyya. High-performance cloud computing: Aview of scientific applications. In Proceedings of the 10th International Sym-posium on Pervasive Systems, Algorithms and Networks. IEEE CS Press, Dec.2009.

[137] X. Wang, X. Shen, and N. D. Georganas. A fuzzy logic based intelligent ne-gotiation agent (fina) in e-commerce. In Proceedings of the IEEE CanadianConference on Electrical and Computer Engineering, 2006.

[138] P. Winoto, G. I. McCalla, and J. Vassileva. Non-monotonic-offers bargain-ing protocol. Autonomous Agents and Multi-Agent Systems, 11(1):45–67, June2005.

[139] W. Y. Wong, D. M. Zhang, and M. Kara-Ali. Towards an experience-based nego-tiation agent. In Proceedings of the 4th International Workshop on CooperativeInformation Agents IV, 2000.

[140] R. R. Yager. A note on probabilities of fuzzy events. Information Sciences,18(2):113 – 129, 1979.

[141] R. R. Yager. Possibilistic decision making. IEEE Trans. on Systems, Man andCybernetics, 9:388–392, 1979.

[142] R. R. Yager. An approach to ordinal decision-making. International Journal ofApproximate Reasoning, 12:237–261, 1995.

183

Bibliography

[143] M. Yokoo, E. H. Durfee, T. Ishida, and K. Kuwabara. The distributed con-straint satisfaction problem: Formalization and algorithms. IEEE Transactionson knowledge and data engineering, 10(5):673–685, 1998.

[144] L. A. Zadeh. Fuzzy sets. Information and Control, 8:338–353, 1965.

[145] L. A. Zadeh. Probability measures of fuzzy events. Journal of Applied Math-ematical Analysis and Application, 23:421–427, 1968.

[146] L. A. Zadeh. Fuzzy sets as a basis for a theory of possibility. Fuzzy Sets andSystems, 100(1):9–34, 1999.

[147] D. Zeng and K. Sycara. Benefits of learning in negotiation. In Proceedings of thefourteenth national conference on artificial intelligence and ninth conference onInnovative applications of artificial intelligence, AAAI’97/IAAI’97, pages 36–41. AAAI Press, 1997.

[148] D. Zeng and K. Sycara. Bayesian learning in negotiation. International Journalof Human-Computer Studies, 48(1):125–141, 1998.

[149] L. Zeng, B. Benatallah, M. Dumas, J. Kalagnanam, and Q. Z. Sheng. Qualitydriven web services composition. In Proceedings of the 12th international con-ference on World Wide Web, WWW ’03, pages 411–421, New York, NY, USA,2003. ACM.

[150] F. L. B. Zeuthen. Problems of Monopoly and Economic Warfare. Routledge andSons, London, UK, 1930.

184

decision strategies for automated negotiation with limited … · decision strategies for automated...

Documents