Computer Networks 56 (2012) 927–939

Contents lists available at SciVerse ScienceDirect

Computer Networks

journal homepage: www.elsevier .com/ locate/comnet

A distributed resource management model for Virtual PrivateNetworks: Tit-for-Tat strategies q

Jean-Marc Robert a,⇑, Hadi Otrok b, Ahmad Nahar Quttoum a, Rihab Boukhris a

a Ecole de technologie supérieure, Département de génie logiciel et des TI, Montréal, QC, Canadab Department of Computer Engineering, Khalifa University of Science, Technology & Research (KUSTAR), Abu Dhabi, United Arab Emirates

a r t i c l e i n f o a b s t r a c t

Article history:Received 10 August 2010Received in revised form 8 November 2011Accepted 18 November 2011Available online 3 December 2011

Keywords:Autonomic service managementService level agreementQuality of ServiceBandwidth sharingNon-cooperative repeated games

1389-1286/$ - see front matter � 2011 Elsevier B.Vdoi:10.1016/j.comnet.2011.11.013

q Partially supported by Bell Canada and the NEngineering Research Council of Canada througResearch and Development grants.⇑ Corresponding author. Address: 1100 rue Notre

réal, QC, Canada H3C 2K3. Tel.: +1 514 396 8430; faE-mail addresses: jean-marc.robert@etsmtl.ca

otrok@kustar.ac.ae (H. Otrok), ahmad.qutt(A.N. Quttoum), rihab.boukhris.1@ens.etsmtl.ca (R.

In this paper, we address the autonomic resource management problem for Virtual PrivateNetworks (VPNs) in the presence of stochastic and selfish VPN operators. Resource man-agement is one of the most important problems that faces Internet Service Providers. Inthe literature, the Autonomic Service Architecture is proposed to provide a resource man-agement model that allows systems to manage themselves and aiming to utilize optimallythe unused resources. Unfortunately, this model suffers from two major limitations. First,unused resources from underloaded VPNs (lenders) are utilized over the overloaded ones(borrowers) without considering the unexpected changes of the VPNs’ state, which mayoften happen in the case of multimedia transmissions. This may affect negatively theQuality-of-Service (QoS) of the lenders while improving the QoS of the borrowers. Second,underloaded VPNs’ operators might behave selfishly and refuse to lend their unused(spare) resources to other overloaded VPNs. To overcome these limitations, we propose adistributed autonomic resource management, which is modeled as a repeated non-cooper-ative game with stochastic and selfish players. The classical Tit-for-Tat strategy is modifiedto cope with VPN operators who are not always able to lend some resources to others. Fourdifferent strategies are derived from Tit-for-Tat to motivate VPN operators to lend theirresources to others. As far as we know, our work is among the first efforts that usesrepeated non-cooperative game theory to motivate selfish participants to cooperate andto distinguish between stochastic and purely selfish VPNs’ operators. In our setting, thisresults in cooperative sharing of unused resources among VPNs. Simulation results showthat Tit-for-Tat strategy leads to deadlocks, while our strategies assure good gains to coop-erative VPN operators and punish the selfish ones.

� 2011 Elsevier B.V. All rights reserved.

1. Introduction

Virtual Private Networks (VPNs) are built on top of anInternet Service Provider’s (ISP) public infrastructure to

. All rights reserved.

atural Sciences andh its Collaborative

-Dame ouest, Mont-x: +1 514 396 8405.

(J.-M. Robert), hadi.oum.1@ens.etsmtl.caBoukhris).

establish secure and reliable connections with guaranteedQuality of Service (QoS) levels [9]. For this purpose, a flexi-ble, efficient and autonomous management model isneeded that is capable to satisfy the VPN operators’ needsand decrease the management operation expenses andthe request response time. Such an autonomicmanagement system should allow systems to managethemselves and demonstrate the following properties:self-configuration, self-optimization, self-protection andself-healing [19]. The importance of this approach has beenrecognized in the literature [1,17]. Frameworks have beenproposed very recently to facilitate the implementation of

928 J.-M. Robert et al. / Computer Networks 56 (2012) 927–939

self-management logic in network equipment [11,5]. Suchframeworks can increase the level of customisation in anetwork. This may have some beneficial impacts on thenetwork performance but may open the doors to misbehav-iors [30].

In the literature, an Autonomic Service Architecture(ASA) [6] is proposed to automate the utilization of re-sources by developing an autonomic bandwidth borrowingscheme among VPN operators. The bandwidth borrowingscheme utilizes the unused (spare) resources of underload-ed VPNs (lenders) to guarantee the Service Level Agree-ments (SLAs) of overloaded VPNs (borrowers). Suchagreements are usually conducted between VPN operatorsand ISPs to guarantee VPNs’ QoS. Unfortunately, the ASAmodel suffers from the following major limitations:

� Unused resources (spare) of underloaded VPNs (lend-ers) are distributed among all overloaded VPNs (bor-rowers) without any regard to the lenders’ short-termfuture needs. A centralized manager identifies the avail-able resources without any interaction with the poten-tial lenders. In this case, the lenders’ QoS might beaffected negatively while the QoS of the borrowers willbe guaranteed.� There is no incentive mechanism to motivate the oper-

ators of underloaded VPNs (lenders) to lend theirunused resources to operators of overloaded VPNs (bor-rowers). The absence of such a mechanism may leadsome rational lenders to behave selfishly and refuse toshare their unused resources. This may impact nega-tively the QoS of the overloaded VPNs.

To overcome the above limitations, we propose a dis-tributed autonomic resources management model thatcan motivate operators to be cooperative. In such a model,the VPN operators are responsible to identify their ownavailable resources. They should know their short-term fu-ture needs and could determine if they have unused re-sources to share with others. To motivate these operatorsto collaborate, we propose to use strategies based on re-peated non-cooperative games [4]. The main objectives ofthese strategies are to assure the self-protection and theself-configuration of the VPNs, as defined in [19].

Traditionally, classical [4] and evolutionary [20] gametheories mainly address the problem of evolution of coop-eration among non-cooperative participants in a determin-istic context. A cooperative participant should be able tocollaborate if he wants to do it. Tit-for-Tat [4] is one ofthe well-known strategies used in repeated non-coopera-tive games. The aim of such strategies is to enforce partic-ipants to develop a mutual cooperation to maximize theirpotential gains [4]. In such games, the participants aremotivated to cooperate under the threat of eventual pun-ishment. If a participant refuses to help some participantsat time t, these participants may refuse to help him back attime t0 > t. Therefore, participants should keep the historyof their received offers and respond to any new requestaccording to their historic records.

Unfortunately, applying the classical Tit-for-Tat strategyto the autonomic resource management problem will leadto mutual deadlocks which forbid any further cooperation.

In our problem, the VPN operators should demonstrate sto-chastic behaviors. Their decisions to cooperate or not arebased on (1) the past behavior of the requesting VPNoperators and (2) whether they have unused resources ornot. The latter condition is obvious but it represents a majordifference with the classical applications of repeated non-cooperative games. In our problem, a cooperativeparticipant may be forced to decline any collaboration atany given moment. In fact, a cooperative participant shouldbe able to collaborate with others only with probability atleast pa, for some 0 < pa < 1. Thus, in all the simulation sce-narios presented in this paper, the cooperative participantsare stochastic participants who can refuse to lend any re-sources when they do not have any unused ones. However,this stochastic behavior should still be distinguishable frompurely selfish participants – such participants never lendany resource.

This raises the need to modify the Tit-for-Tat strategy tocope with the stochastic behavior of the participants andpropose novel strategies. These strategies must either haveto demonstrate some forgiving/generous behavior or somealtruistic behavior. In this paper, we present four strategiesbased on Tit-for-Tat that can handle the case of stochasticparticipants. The strategies are: (1) generous Tit-for-Tat, (2)altruistic Tit-for-Tat, (3) reputation-based Tit-for-Tat, and (4)altruistic-reputation Tit-for-Tat. These strategies are able tomotivate selfish participants to cooperate with others andlend their unused resources under the threat of future pun-ishment. The difference between the four new strategieslies in the methods used to distinguish cooperative partic-ipants with stochastic behavior from purely selfish ones.These methods are formulated taking into considerationthe inability of cooperative participants to collaboratewhen they do not have any unused (spare) resources toshare. Simulations have been conducted to show the im-pacts of the different strategies on the total gain of cooper-ative participants over selfish ones.

The rest of this paper is organized as follows: Section 2presents the problem statement. Section 3 presents the re-source allocation model. Section 4 illustrates the Tit-for-Tatstrategies with simulation results. In Section 5, we presentthe related work. Finally, Section 6 concludes the paper.

2. Problem statement

The ASA model [6] proposes an autonomic managementsystem that is able to automate the utilization of resourcesin order to maximize the ISP profit. One of the most impor-tant objectives of the ASA model is to satisfy the VPN oper-ators that have signed a service level agreement (SLA) withthe ISP to guarantee their QoS level. To achieve this, themodel proposes an autonomic bandwidth borrowingscheme for efficient resource utilization to ensure VPNs’QoS. The core of ASA is in its borrowing scheme that sharesthe unused resources of underloaded VPNs among theoverloaded ones so that the QoS is always guaranteed. AVPN is underloaded at a time t if its measured traffic loadat this specific moment is below the capacity of its allo-cated bandwidth. In this case only, the rational VPN canlend some of its unused resources to others.

J.-M. Robert et al. / Computer Networks 56 (2012) 927–939 929

The borrowing scheme of the ASA model has the follow-ing limitations. First, the unused resources from under-loaded VPNs are utilized without considering the suddenand unpredictable changes in the network state and linksload. The quality of the transmitting streams varies withthe higher transmission rates, which requires more band-width resources. Such a borrowing scheme that requisi-tions the resources without contacting the eventuallenders would affect negatively the lender’s QoS whileguaranteeing the borrower’s QoS. As a solution, we pro-pose that the process of identifying the unused resourcesmust be distributed among all VPN operators. This willhelp to overcome the above limitation since VPN operatorsare the best ones to know their short-term needs andrequirements.

Traffic prediction is an important and very difficult to-pic for the provisioning of a network having to supportmany dynamic VPNs. Numerous models have been pro-posed in the literature [8,31]. These models have to predictthe traffic load behavior of a VPN according to its recentpast. Unfortunately, the past may not always predict thefuture. However, in many cases, the VPN operators mayhave some inputs to predict their short-term future re-source requirements. For example, in the Voice over IP(VoIP) video conferencing application [18], the VPN opera-tor can monitor the Session Initiation Protocol (SIP) mes-sages [32] which negotiate the parameters of multimediasession [6]. Therefore, it should be better to rely on theoperators to forecast their resource requirement. Such adistributed model should be more reliable.

Second, the other problem comes from the fact thatsome VPN operators may be tempted to behave selfishlyto preserve their QoS level. The VPN operators may refuseto lend their unused resources while consuming others’ re-sources once they are overloaded.

Overtime, such behaviors would impact negatively theQoS of the participants. In fact, if all the participants be-have selfishly and refuse to lend their unused resources,none of the temporarily requests for extra resources wouldbe satisfied. Therefore, rational participants should lendtheir resources and expect similar behaviors from others.

The distributed autonomic resource management prob-lem considered in this paper is reduced to motivate poten-tial selfish VPN operators to collaborate with others andoffer their unused resources when others are overloaded.Therefore, the question addressed in this paper can be sta-ted as follows: How can we enforce all the rational VPN oper-ators to collaborate and share resources in order to preservethe QoS of as many VPNs as possible? Participants have a ra-tional behavior if they prefer strategies yielding higherpayoffs and are indifferent between strategies yieldingequal payoffs [14]. To achieve our goal and solve this prob-lem, we propose to motivate selfish VPN operators to coop-erate and lend their unused resources under the threat ofpunishment which may result to reduced QoS levels.

3. The model

Traditionally, the interaction between two competingparticipants can be modeled as a game where each partic-

ipant can deterministically choose either to cooperate or todefect. In such a game, the participants are assumed to benon-collusive (i.e. they do not collude with each other forthe duration of the game) and play their correspondingmoves simultaneously. If both participants cooperate, theyreceive a reward, denoted R. If both of them defect, they re-ceive a punishment, denoted P. Otherwise, the participantwho cooperates receives a temptation, denoted T, and theother one receives only a sucker, denoted S. This game isknown as the prisoner’s dilemma, if the payoffs fulfill twoconditions: (1) T > R > P > S and (2) 2R > T + S. This problemhas been formalized by Tucker in 1950s [29].

If the prisoner’s dilemma game is played only once, thebest strategy for both participants is to defect (condition(1)). However, if this game is iterated and played an unde-termined number of rounds, there are some incentives forthe participants to cooperate (condition (2)). Many strate-gies have been proposed in the literature [3,4] to motivateparticipants to cooperate if they want to maximize theirgains. The Tit-for-Tat strategy is one of the well-knownstrategies. A participant, following this strategy, shouldcooperate in the first iteration of the game and then ‘‘coop-erate’’ or ‘‘defect’’ based on opponent’s previous move. Infact, a participant simply plays his opponent’s previousmove.

Unfortunately, the classical prisoner’s dilemma para-digm cannot be used for the distributed autonomic re-source management problem presented in this papersince resources are not offered and requested simulta-neously by all participants. At any given round, some par-ticipants may request more resources (borrowers) andsome others may offer some resources (lenders). A partic-ipant lends resources with the hope that he would recipro-cally receive in the future from the borrowers someresources when he would need them to guarantee hisown QoS level. Hence, the model used to characterize theparticipants’ behavior should be either the alternating[12,28] or, more generally, the asynchronous [24,25,15]prisoner’s dilemma.

Following the approaches presented by Frean [12] andby Nowak and Sigmund [28], two participants can alterna-tively request and offer some resources. In such a case, theborrower requests some resources and the lender eithercooperates and offers his unused resources or simply de-fects and refuses to share his resources. If a lender decidesto share some resources, he would have a negative pay-off,denoted a, which corresponds to his sharing costs. On theother hand, the borrower would have a positive payoff, de-noted b, which corresponds to his gain by using these extraresources (i.e., an higher QoS level). If a lender refuses toshare any resource, he would have a null payoff, denotedc, and the rejected borrower would have a negative payoff,denoted d, which corresponds to his lost due to his inca-pacity to achieve his desired QoS level.

For example, if we assume that c > a, then a lender wouldbe better to refuse to share his unused resources for the gi-ven iteration. Furthermore, if we assume that c � a < b � d,then the cost for the cooperative lender is less than the gainof the borrower. These two conditions are sufficient to en-force the two rational participants to mutually cooperateif they play an undetermined number of iterations.

930 J.-M. Robert et al. / Computer Networks 56 (2012) 927–939

The traditional methods to enforce rational participantsto collaborate cannot be used to solve the problem pre-sented in this paper. Cooperative participants may nothave always available resources to share at any given iter-ation. Their stochastic behavior represents a burden for theclassical methods. For instance, the classical Tit-for-Tatmethod could lead to deadlocks where no participantwould help any other participant, as Fig. 2 will showclearly. This is why our work can be considered as novel.

Therefore, we propose the following four novel strate-gies to cope with the haphazard behavior of theparticipants:

1. Generous Tit-for-Tat: to overcome the deadlock problemof the traditional Tit-for-Tat, participants will be gener-ous and offer resources from time to time to other par-ticipants regardless whether they were cooperative ornot in the recent past.

2. Altruistic Tit-for-Tat: since the generous strategy offersresources to all participants, selfish ones are still bene-fiting from such generosity. This raises the need for astrategy that can motivate selfish participants to coop-erate as any other participant under the threat of laterpunishment.

3. Reputation-based Tit-for-Tat: the generous and altruisticstrategies require that each participant keeps a longhistoric records for other participants. To reduce sucha requirement, a reputation-based strategy is proposed.In such a case, a participant decides to cooperate withanother participant and lends his unused resources ifthe latter has a good reputation. Unfortunately, thisstrategy requires a long initialization interval to reliablybuild the reputation table.

4. Altruistic-reputation Tit-for-Tat: to overcome the limita-tion of the last two strategies, we propose to mergethem in order to have a good gain for the cooperativeparticipants while being very restrictive with purelyselfish participants during the initialization phase.

4. Tit-for-Tat strategies

In this section, we first present the set-up and the sim-ulation parameters used to compare our four new strate-gies. Our objective is to show through simulations thatour strategies encourage rational VPN operators to collab-orate in order to optimize their long term gains.

4.1. Set-up and simulation scenarios

Consider a group of n participants. Each participant hasaccess to some allocated resources. With these resources, aparticipant may fulfill his requirements and obtain the de-sired QoS level. However, at any time, a participant hasthree mutual exclusive strategies. First, he consumes lessresources than allocated and thus can share unused re-sources with others. Second, he requires more resourceswhen network state becomes overloaded and thus borrowssome resources from others. Finally, he consumes exactlyall his allocated resources.

Unfortunately, some participants (s out of n) may dem-onstrate a selfish behavior. They can try to borrow re-

sources from others but are not ready to offer others anyof their unused resources. These participants can representa threat for the survival of the overall group.

In this paper, we have decided to simplify the simulatedscenarios. At each iteration, only one participant is allowedto request more resources. Thus, at time t, a given partici-pant i, that is selected randomly among all the participants,requests some resources Ri(t). The other participants caneither reject the request or cooperate and offer some re-sources. In the former case, a participant j can be selfishor, simply, does not have any unused resource with proba-bility 1 � pj. In the latter case, a participant j decides tocooperate and offer some resources Oi,j(t) according tothe offerings that i has proposed to j in the near past (i.e.Oj,i(t⁄), for t⁄ < t) – the historic records.

In order to compare the different strategies, we base oursimulations on the following scenario of the asynchronousprisoner’s dilemma game:

� In total, we have 12 VPN operators (participants) where8 of them are cooperative and 4 are selfish.– At each iteration, a non-requesting cooperative par-

ticipant j would offer one resource with a probabilityof pj ¼ pa ¼ 1

4.� 100000 sequential requests are made.

– At each iteration, a single participant is randomlychosen. Hence, the probability of requesting ispr ¼ 1

12, for any given participant.– This participant requests one resource.

In this scenario, each single request of a cooperativeparticipant can be fulfilled by 7

4 cooperative participants,on average. However, this request may be rejected simplydue to the lack of available resources with probability1� 1

4

� �7. In any simulated scenario, this should give anupper bound on the maximal number of requests whichcan be fulfilled. Hence, 100000� 8

12 ¼ 66666 requests aremade by cooperative participants, on average. From theserequests, 1� 1

4

� �7 � 13:3% would not be fulfilled simplydue to the lack of unused resources.

Some of the above restrictions could be relaxed. Eachparticipant j could publish his authenticated SLA contractestablishing his own offering probability pj. This latter va-lue should be eventually used to establish the reputation ofthe participant. If this SLA contract does not reflect any-more the real situation of a participant, this participantshould renegotiate another contract with the ISP.

A non-requesting cooperative participant j could offer oj

resources – instead of just one – and the requesting partic-ipant i could demand as much as di resources. As far asE½di� <

Pjpj � E½oj� i.e. the average demand is smaller than

the average total offering, the problem is the same. Therequesting participant needs the help of most of the coop-erative participants to fulfill his requirements.

To evaluate the performance of the different strategiesproposed in this paper, we formulate this prisoner’s dilem-ma game with the following parameters:

� Cost to share unused resources: a = �1� Cost to refuse any resource: c = 0� Gain from a fulfilled request: b ¼ 1

pr

0 2 4 6 8 10x 10

4

0

1

2

3

4

5

6

7x 10

5

Total number of requests

Tota

l gai

n fo

r the

eig

ht s

elec

ted

parti

cipa

nts Nb of offering participants n=12

Nb of offering participants n=8

Fig. 1. The optimal upper bounds.

J.-M. Robert et al. / Computer Networks 56 (2012) 927–939 931

� Loss from a non-fulfilled request: d = �1

Intuitively, we have chosen to assume that a VPN oper-ator requesting more resources aims for an higher QoS le-vel. In such a case, the gain associated to a fulfilled requestis high – a better quality of the service momentarily – andthe lost associated to a non-fulfilled one is negligible. No-tice that we could have used another approach. We couldsee these requests as if an operator would like to obtainhis negotiated QoS level. In such a case, the extra gain isnegligible (b = 1) and the lost is important (d ¼ �1

pr). How-

ever, this point of view would have given similar resultsthan those shown in this paper. In both cases, a similar ap-proach has been used. The gain associated to a fulfilled rarerequest or the lost associated to a non-fulfilled one shouldbe relatively high. This can be compared to the approachused to define the entropy in information theory [7]. In thislatter case, rare events are more informative and theirweight are inversely proportional to their probability.

These above parameters assure that the average gain fora cooperative participant is strictly positive:

pr � bþ ð1� prÞfpa � aþ ð1� paÞ � cg > 0:

The above inequality assumes implicitly that (1) theparticipant is recognized as cooperative by the other coop-erative participants and (2) the other cooperative partici-pants have enough available resources to fulfill therequests of the participant. If we remove the latter condi-tion, the average gain for a cooperative participant is givenby

prfð1� ð1� paÞ7Þ � bþ ð1� paÞ

7 � dgþ ð1� prÞfpa � aþ ð1� paÞ � cg > 0:

This inequality is obtained from the particular scenariodescribed above where the requesting participant asks fora single resource and the cooperative participants offer asingle resource with probability pa. In such a case, the ex-pected gain is still strictly positive.

To conclude this section, let consider the following twoscenarios:

� 8 cooperative participants among 12 participants shar-ing their available resources with any of them.� 12 cooperative participants sharing their available

resources with any of them.

These can be used to compare the effectiveness of thesolutions proposed in the incoming sections. The first sce-nario corresponds to the ideal one where the eight cooper-ative participants can perfectly recognize whether arequesting participant is cooperative or not. The secondscenario illustrates the gain which can be expected if allthe twelve participants behave cooperatively. In this latterscenario, only the total gain for the eight of those partici-pants should be considered. Otherwise, the comparisonwould be unfair.

Fig. 1 illustrates both scenarios. The first scenario is de-picted by the curve labeled n = 8. It gives the expectedupper bound for the total gain from fulfilled requests ofthe eight cooperative participants exchanging their unused

resources without any restriction. This upper corresponds

to 100000� 812� 1� 1� 1

4

� �7� �

� ðbþ aÞ þ 1� 14

� �7 � dh i

� 626500. This upper bound is particularly useful to eval-uate the performance of the incoming models. It can beseen as if the cooperative participants can have access toan oracle differentiating between cooperative and selfishparticipants.

In the second scenario, if the eight participants could beable to convince the other four selfish participants to be-have properly, all these participants would improve theirtotal gain from the fulfilled requests. In fact, the originaleight cooperative participants should receive offers fromall the twelve participants (curve labeled n = 12). Theupper bound of their expected total gain is given by

100 000 � 812 � 1 � 1 � 1

4

� �11� �

� ðb þ aÞ þ 1 � 14

� �11 � dh i

� 699 550. On the other hand, the four original selfish par-ticipants should be now recognized as cooperative partici-pants and receive resources from the others.

The gap between the two curves presented in Fig. 1can be seen as the potential gain for the original coopera-tive participants, if they can convince the other selfish par-ticipants to behave properly and share their unusedresources.

4.2. Model I: traditional Tit-for-Tat with fixed history

Suppose that each participant j keeps the historic recordof the last k ¼ coeff

paoffers of each other participant i, for

some coeff P 1. Let Hj, i(1), . . . ,Hj,i(k) correspond to the lastk offers of i to j i.e. Oj;iðti1 Þ; . . . ;Oj;iðtik Þ, for someti1 > � � � > tik .

In the first approach, a participant responds to arequesting participant according to what this latter offershim in the previous iterations. This behavior generalizesthe classical Tit-for-Tat (one-move history) and the Tit-for-two-tats (two-move history) strategies [4]. Formally,if i requests some resources, j who has Aj(t) unused

0 2 4 6 8 10x 10

4

−1

0

1

2

3

4

5

6

7x 10 5

Total number of requests

Tota

l gai

n fo

r the

par

ticip

ants

Cooperatives + coeff = 4 with 1 offerCooperatives + coeff = 4 with 2 offersSelfishes + coeff = 4

Fig. 3. The generous Tit-for-Tat.

0 2 4 6 8 10x 104

−1

0

1

2

3

4

5

6

7x 105

Total number of requests

Tota

l gai

n fo

r the

coo

pera

tive

parti

cipa

nts coeff = 16 with 1 offer

coeff = 16 with 8 offers

Fig. 4. The generous Tit-for-Tat with threshold.

0 2 4 6 8 10x 10

4

−1

−0.5

0

0.5

1

1.5

2x 10

5

Total number of requests

Tota

l gai

n fo

r the

coo

pera

tive

parti

cipa

nts coeff=4

coeff=3coeff=2

Fig. 2. The Tit-for-Tat with fixed history – maximum option.

932 J.-M. Robert et al. / Computer Networks 56 (2012) 927–939

resources can offer to i some resources depending of hisoffering historic record Hj,i(k). For example:

Oi;jðtÞ ¼minfAjðtÞ;max16h6k

Hj;iðhÞg; or

¼min AjðtÞ;1k

X16h6k

Hj;iðhÞ( )

:

Unfortunately, these two cases converge quickly towardmutual deadlocks where no participant collaborates withany other one.

Let us consider two cooperative participants requestingresources alternatively. Suppose that both participants canoffer resources only with probability pa. If a participantwould decline 2k consecutive requests, the historic recordsof both participants (Hi,j(�) and Hj,i(�)) should be filled onlywith 0s. The expected number of rounds before seeing sucha long run of refusals is H((1 � pa)�2k). Thus, these twocooperative participants should eventually enter into amutual deadlock. Ultimately, no cooperative participantwould be ready to lend any resource to any other partici-pant (cooperative or not).

Fig. 2 illustrates how the total gain of all the cooperativeparticipants progresses over time. The figure shows clearlythat the classical Tit-for-Tat strategy with a fixed history isinadequate for the resource borrowing problem.

4.3. Model II: the generous Tit-for-Tat

Cooperative participants would eventually enter ininterlocking deadlocks, if they may not have resources tolend occasionally. To thwart this problem, the cooperativeparticipants should be generous sporadically and forgivewhen a participant rejects their last requests.

Formally, a participant j who has some unused re-sources to lend can offer them as follows:

Oi;jðtÞ¼minfAjðtÞ;max

16h6kHj;iðhÞg if Hj;iðhÞ–;; for some h;

AjðtÞ �a bonus every k requests;0 otherwise:

8>><>>:

Hence, this cooperative participant would lend some un-used resources every k ¼ coeff

parequests with any participant

– cooperative or selfish – who has rejected his k latest re-quests. This generous behavior is sufficient to avoid anymutual deadlock among cooperative participants. As coeffincreases, the strategy is less generous where unused re-sources are offered only once per k requests. However, ascoeff increases, the probability to encounter a mutual dead-lock between two cooperative participants goes downexponentially fast – this follows the Chernoff inequality[22].

Fig. 3 shows the impact of the generous Tit-for-Tatstrategy. As shown in the figure, the total gain of the coop-erative participants (see the curve labeled Cooperativeswith 1 offer) is close to the optimal upper bound. In sucha case, any participant is recognized as cooperative if ithas offered some resources at least once in the short pasthistory (coeff

palast request).

Purely selfish participants who always refuse to sharetheir resources would benefit from such a behavior (seethe curve labeled Selfishes). In the simulated scenario,

J.-M. Robert et al. / Computer Networks 56 (2012) 927–939 933

the total expected gain of the selfish participants should

be 33333� 1� 1� 14

� �8� �

� bþ 33333� 1� 14

� �8� d �356600, if they could fool the cooperative participantsto fulfill their requests. As one can see from the figure,the selfish participants still got almost 25% of their poten-tial gain. To obtain a better performance, a rational selfishparticipant can adapt his behavior and simply offer onceevery k requests from any given participant. By doingso, selfish participants will be indistinguishable from afully cooperative participant who offers once every 1

pare-

quests, on average. This aspect would be important ifmultiple requests are done simultaneously. In such a case,participants should prefer to offer to most likely coopera-tive participants.

To respond to such an evolution of the selfish partici-pants, the cooperative participants can be more selectiveand offer resources to a requesting participant only if hishistoric record contains more than one offer. Unfortu-nately, the cooperative participants would be penalizedby such a behavior. See the curve labeled Cooperatives with1 offer vs. the curve labeled Cooperatives with 2 offers inFig. 3.

The impact of the threshold can be reduced by using alonger history. A cooperative participant should offer hisunused resources to a requesting participant if this latterhas made at least coeff

2 offers to the participant in the recentpast instead of a single one. As coeff increases, the impactof the threshold is less important, as shown in Fig. 4.

This generous strategy can ‘‘enforce’’ the rational selfishparticipants to offer their resources at least with probabil-ity pa

2 , if they do not want to be distinguished from thecooperative participants and see their total gain todecrease.

However, there is a major drawback. This strategy can-not always maximize the total gain even if selfish partici-pants started to be cooperative. Since a selfish participantis recognized as cooperative if he offers his resources onlywith probability pa

2 , rational cooperative participants mayadapt their behavior and offer their unused resources withthe same probability. In such a case they are still recog-nized as cooperative with less participation. Unfortunately,the total gain of the cooperative participants would be re-

duced significantly to 66666� 1� 1� 18

� �7� �

� ðbþ aÞþ

66666 � 1� 18

� �7 � d � 419200. Even if this behavior isgood for any single cooperative participant, this is extre-mely bad for the overall group.

4.4. Model III: the altruistic Tit-for-Tat

An altruistic strategy is proposed to avoid the mutualdeadlocks of the cooperative participants without helpingpurely selfish participants. To receive resources from acooperative participant, a participant should not have re-fused the latest k requests from this participant. Such a re-fusal would affect the total gain of any participant over thetime.

Hence, to avoid any future punishment, a cooperativeparticipant might accept to reduce his QoS level require-ment occasionally and lend some of his engaged resources

instead of refusing k consecutive requests from a partici-pant which he recognizes himself as cooperative. A partic-ipant is recognized as cooperative by another participant ifhe has offered resources at least coeff

2 times in the previous krequests of this latter participant.

Thus, the parameter a that corresponds to the sharingcost for the offering participant should be modified to re-flect the difference between offering available resourcesand offering engaged resources. In the former case, aA issimply set to �1 as before. In the latter case, aNA is definedas �1

pri.e. the same magnitude of the gain for a fulfilled re-

quest. These parameters assure that the average gain fora cooperative participant is still strictly positive:

pr � bþ ð1� prÞfpa � aA þ ð1� paÞð1� paÞk�1 � aNAg > 0:

This follows from the fact that an altruistic participantshould accept to reduce his QoS level with probability(1 � pa)(1 � pa)k�1 – i.e. he has no unused resource to offerand has denied the previous k � 1 requests. The assump-tion used for the previous strategy is also used in this case.The cooperative participants should be able to fulfill theirrequests.

Fig. 5 shows the impact of the altruistic Tit-for-Tatstrategy. For the cooperative participants, this new strat-egy is slightly better than the generous strategy (specially,for coeff = 4 with 1 offer or for coeff = 16 with 8 offers – notshown on the figure since both curves coincide).

However, the selfish participants would never abortthemselves and should not have good reputations for theother cooperative participants. Therefore, selfish partici-pants would see all their requests to be rejected by theother participants. This explains why their total gain isnot presented in the figure.

Rational selfish participants would be forced to collabo-rate with probability pa

2 if they want to have high total gainand be able to obtain resources from others. Therefore, theselfish participants would not obtain free payoffs as shownin the previous case.

4.5. Model IV: reputation-based Tit-for-Tat

The previous two strategies require to keep very longhistoric records of all the participants. This is especiallytrue if we want to (1) optimize the total gain of the coop-erative participants and (2) maximize the number of offersthat selfish participants must do in order to be indistin-guishable. In the previous section, each participant has tokeep 64 ¼ coeff � 1

pahistoric records for each other partici-

pant, if he wants that a selfish participant offers at least 8times in those 64 iterations, which is half the number of of-fers that a cooperative participant would do, in average. Ifwe want to have a better ratio that 1/2, the coefficientshould have to be much greater than 16. This would resultin very long historic records per participant.

To reduce the impact of the length of the historic re-cords on the participants, a reputation-based strategy isproposed. The objective of this strategy is the same as be-fore i.e. to avoid mutual deadlocks of the cooperative par-ticipants without helping purely selfish participants.

0 2 4 6 8 10x 10

4

0

1

2

3

4

5

6

7x 10

5

Total number of requests

Tota

l gai

n fo

r the

coo

pera

tive

parti

cipa

nts

δ = 0.5δ = 0.1

Fig. 6. The ratio-based reputation.

(a) (b)

Fig. 5. The altruistic Tit-for-Tat. (a) The total gain and (b) the number of victims.

934 J.-M. Robert et al. / Computer Networks 56 (2012) 927–939

However, this objective should be achieved with only O(1)memory instead of O 1

pa

� �per participant.

The participant i should compute the reputation of theparticipant j by evaluating the following ratio:

NOi;jðtÞNRiðtÞ

: ð1Þ

In this model, a participant simply has to keep NRi(t),which represents the total number of requests that i hasmade so far at time t and NOi,j(t), which represents the totalnumber of offers that j has offered so far to i at time t. Thisapproach can be seen as if a participant has kept an infinitehistory of the previous iterations. If the participants act aspurely random process, the ratio given by Eq. (1) should beclose to the probability pa. In fact, for 0 < d < 1,

Prob½NOi;jðtÞ < ð1� dÞpaNRiðtÞ� < e�d2paNRi ðtÞ

2 :

This follows from the Chernoff inequality [22].If a participant keeps his statistics for all the other par-

ticipants, eventually he would be able to distinguish accu-rately the cooperative participants from the selfish ones.The probability that the ratio (1) is below half the expecta-tion goes to zero exponentially fast in the number ofobservations.

Definition 4.1. A participant is declared selfish iff the ratio(1) is below (1 � d)pa, for some fixed 0 < d < 1.

After some initialization interval, where all the partici-pants are assumed to be cooperative, the participants usethe above definition to offer their unused resources onlyto participants which meet the lower bound. Therefore,

Oj;iðtÞ ¼ AiðtÞ ifNOi;jðtÞNRiðtÞ

> ð1� dÞpa:

Fig. 6 shows the impact of the reputation-based Tit-for-Tat.This last strategy is as good as the previous strategies forthe cooperative participants. On the other hand, if the self-ish participants do not offer their resources with a proba-

bility close to pa, they would eventually be recognized asselfish and no other participant would offer them any ofhis unused resources.

The choice of the initialization interval length with re-spect to d is important as shown in Fig. 7. The longer isthe interval the better is the total gain. However, thereshould be an optimal threshold above which the total gainof the cooperative participants does not increase signifi-cantly. Knowing that below this threshold, all the partici-pants are recognized as cooperative participants. Thus, itis important to use a minimum initialization interval, ifwe do not want to be too generous with selfishparticipants.

In Fig. 8, a different approach has been used. Instead ofusing a fixed d, this parameter can be chosen arbitrarilysmall as the number of iterations increases withoutimpacting the total gain. As d decreases, the strategy ismore and more strict with the selfish participants. If they

(a) (b)

Fig. 7. (a) The ratio with d = 0.5 and (b) the ratio with d = 0.1.

(a) (b)

Fig. 8. (a) Gain with adaptative d and (b) adaptative d.

J.-M. Robert et al. / Computer Networks 56 (2012) 927–939 935

want to be indistinguishable, then they have to offer re-sources with a probability close to pa.

At the beginning of the simulation presented in Fig. 8,the parameter d is fixed to a large value (e.g. 0.75). As thesimulation progresses, it is reduced very regularly. Every500 iterations, the parameter is decreased as follows:di+1 = di � 0.1 � (di � s), for some threshold s. In the simu-lation, s has been fixed to 0.1. This latter value has beenchosen arbitrarily. We could have chosen 0.05 as well.Hence, the parameter d converges exponentially fast to-ward the ultimate threshold s, as shown in Fig. 8(b). Thisbehavior can be explained by the Chernoff inequality pre-sented earlier.

This strategy represents a major improvement on thegenerous and altruistic strategies that are presented previ-

ously. In the two models, it is hard to go above d ¼ 12 with-

out using a long historic records for each participant.Hence, the rational selfish participants are forced by

such a strategy to behave similarly to the cooperative par-ticipants. On the other hand, rational cooperative partici-pants would not be tempted to diverge too much fromthe hypothetical pa. In fact, they cannot go below0.99 � pa, as in the previous example, if they still want tobe recognized as cooperative participants.

4.6. Model V: the altruistic-reputation-based Tit-for-Tat

If someone finds that the reputation-based strategy isstill too generous with the purely selfish participants dur-ing the initialization interval phase, it is possible to merge

936 J.-M. Robert et al. / Computer Networks 56 (2012) 927–939

the last two strategies and achieve the same gain for thecooperative participants while refusing any short-termgain to the purely selfish participants.

We first determine the length of the interval for a rela-tively large d = 0.5 – i.e. 1000 in the example presented inFig. 7(a). Then, we use the altruistic strategy during thisinterval. For this strategy, we use coeff = 16 and expect thata cooperative participant makes at least 8 offers for the lastcoeff

parequests of a participant. As we have already men-

tioned, by choosing a relatively small coefficient, the his-toric records of the participants would not have to be toolarge. Finally, the parameter d is decreased regularly as inthe Reputation-based strategy.

This final strategy has all the advantages of both strate-gies: (1) it produces a very good overall gain for the coop-erative participants, (2) it is very strict to recognize thecooperative participants i.e. these participants would haveto offer with probability arbitrarily close to pa, and (3) it re-quires a decent number of historic records to be kept. Theresult of this strategy is as in Fig. 8(b).

4.7. Simultaneous request scenario

The strategies proposed so far, in this paper, and espe-cially the reputation-based one, may have some problemsif multiple requests are made simultaneously. If the totaloffer of resources is less than the total need, the coopera-tive participants who have some resources to offer wouldhave to differentiate among the competing requestersand choose only some of them. In such cases, one obviouschoice is to select the participants that have the best repu-tations. Unfortunately, such a simple strategy can penalizethe cooperative participants in the case of simultaneousrequests.

Let us suppose that there is only one selfish participantwho can request one extra resource simultaneously withother cooperative participants. Such a participant could of-

(a)

Fig. 9. (a) Greedy and gene

fer his extra resource more often than others. On the otherhand, he could also request an extra resource more oftenthan others. The former case should improve his reputa-tion while the latter one should reduce it since he isrequesting more often than other cooperative participants.

In this new scenario, a selfish participant either offershis extra resource with probability of p�a, requests one extraresource with probability p�r , or does not offer nor requestany resource with probability 1� p�a � p�r . If p�r > p�a, thisparticipant is clearly selfish since he tries to use more re-sources than he is ready to offer. On the other hand, onecooperative participant is selected randomly and requestsone extra resource per iteration. The other cooperative par-ticipants offer their unique resource with probabilitypa ¼ 1

4. Hence, with probability p�r , there are two competingsimultaneous requests. If the available resources arescarce, the selfish participant can be very devastating forother participants, especially if his reputation is high com-pared to others. This can happen if the probabilityp�a > pa ¼ 1

4.To thwart the selfish participant, the strategy used by

the participants has to distinguish the selfish behaviorfrom the stochastic cooperative one. In such a case, the re-quest of the selfish participant could be discarded if thereare no enough resources available to fulfill the two re-quests – one from the selfish participant and one fromthe cooperative one. Otherwise, the selfish participantand cooperative one should have an equal chance to getthe available resource.

Two measures are proposed to differentiate betweenthe participants. The selfish participant (1) offers and/or(2) requests some resources more often than the coopera-tive participants.

The strategies presented so far force the selfish partici-pant to offer with a probability p�a > ð1� dÞpa. Thus, theonly rational choice to the selfish participant is to be gree-dy and asks more resources than the other cooperative

(b)

rous and (b) greedy.

J.-M. Robert et al. / Computer Networks 56 (2012) 927–939 937

participants. Consider the scenario where the selfish par-ticipant requests resources with probability p�r ¼ 2

3 – know-ing that the cooperative participants request with aprobability pr ¼ 1

8. Two cases are considered: the selfishparticipant offers his extra resource with the same proba-bility i.e. p�a ¼ pa ¼ 1

4 as the cooperative participants – theGreedy case – or with a higher probability i.e. p�a ¼ 1

3 > pa

– the Greedy & Generous case. These two cases are com-pared with the case where this selfish participant is notpresent.

In Fig. 9, we can observe that if the selfish participant isindistinguishable from the cooperative ones, the total gainof the latter participants decreases significantly in bothcases. When there is only one query which can be fulfilled,the choice between the two requests is random. On theother hand, if the selfish participant can be distinguished,the total gain of the cooperative participants is slightly bet-ter. This is because they benefit from the extra resource of-fered by the selfish participant while this latter does notsee any of his requests fulfilled – his total gain would benegative. Hence, his only rational choice is to reduce thefrequency of his requests and behave like any other coop-erative participant. In such a case, his presence should ben-efit to the overall group.

5. Related work

This section reviews the related work on autonomic re-source management and repeated non-cooperative gametheory.

5.1. Autonomic resource management

The idea of autonomic resource management have beenfirst introduced by Appleby et al. [2] who developed a pro-totype of a highly manageable infrastructure for an e-busi-ness computing. Their aim was to design and develop amanageable and scalable computing infrastructure thatconsists of a farm of massively parallel and densely pack-aged servers interconnected by high-speed, switched LANs.In this context, the concept of dynamic resource allocationhas been developed to accommodate both planned and un-planned fluctuation of network state under the constraintsof the contracted SLAs. The HP vision for the AdaptiveEnterprise [16,13] and the Microsoft Dynamic Systems ini-tiative [21] realize that the autonomic management for thecomputing components is critical for the future Informa-tion Technology (IT) industry. Their views reinforce the ap-proach presented by Appleby et al.

Farha and Leon-Garcia [10] expanded the autonomicview to include the computer telecommunication servicesand considered both computing and networking resources.In their work, they proposed the Autonomic Service Archi-tecture (ASA) model, which is a framework for automatedmanagement of Internet services and their underlying net-work resources. For this framework, they designed anAutonomic Resource Broker (ARB) to serve as the auto-nomic manager, which is the key component of the ASA.

The concept of autonomic resource broker has beenused in different contexts. Cheng et al. [6] proposed a

bandwidth sharing scheme among independent partici-pants for utilizing the available network resources. Thebandwidth borrowing scheme provides a way to managebandwidth resources that are already allocated for eachSLA. These resources may be automatically adjustedaccording to the network state, under the control of the de-fined policies for better resource utilization and QoS guar-antees. Another example of autonomic broker has beenpresented by Quttoum et al. [30]. These authors addressedthe problem of maximizing ISP profit in the presence ofexaggerating VPN operators that can ask for more unusedresources to ensure the QoS. Exaggeration behavior waseliminated under the threat of punishment of the non-cooperative participants. The authors used a model basedon mechanism design theory to achieve their goals.

Although the ASA model proposes a solution that canprovide an automatic management scheme and resourcessharing, we can note that such a model suffers from severallimitations that can decrease the QoS of the participants.Firstly, resources that are not used by a VPN operator areallocated to others without considering the short-term fu-ture needs of this operator – and without asking for hispermission. Secondly, VPN operators might not be readyto lend their resources in order to guarantee their ownQoS. Such a selfish behavior can have a major impact onthe overall group of participants. Some participants whocould not borrow enough resources would not be able toachieve their desired QoS level.

5.2. Repeated non-cooperative game theory

The problem of the evolution of cooperation in a groupof non-cooperative participants has a very long history andhas been study in biology, economics, mathematics, poli-tics and psychology, among others. The seminal book ofAxelrod [4] introduces this problem thoroughly. In hisbook, Axelrod presents the classical strategies proposedin the literature, among those the famous Tit-for-Tat pro-posed by the mathematical psychologist A. Rapoport. Healso presents two tournaments which have been used toevaluate the different strategies. The simple Tit-for-Tatstrategy won these tournaments brilliantly.

In the underlying prisoners’ dilemma, the participantsplay simultaneously their moves. Over the years, this mod-el of the participants has evolved. An axis of research intro-duces the concept of noise in the decision process. Aparticipant who wants to collaborate with his opponentmay see his move to be stochastically altered [23]. In sucha case, the performance of the classic Tit-for-Tat strategycan be very poor even with a very low level of noise. Tothwart this problem, Molander [23] proposed to introducean unconditionally generosity – with probability p. Nowack[26] characterized all the potential stochastic strategies.Another approach to solve this problem is to use the simpleWin-Stay/Lose-Shift (or Pavlov) strategy [27]. A participantusing this strategy cooperates if both participants agreed inthe previous move. This strategy introduces some forgive-ness in the decision process. These two approaches can cor-rect occasional errors without exposing the participant tobe exploited by another participant.

938 J.-M. Robert et al. / Computer Networks 56 (2012) 927–939

Another important axis of research relaxes the simulta-neity of the moves. This changes significantly the problem.Hence, the model used to characterize the participants’behavior is now either the alternating or, more generally,the asynchronous [24,25,15] model. These approaches tryto model the notion of reciprocal interaction. Markovianmodels have been developed to study such problems.Nowak and Sigmund [28] proposed stochastic strategieswhich cooperate generously with the opponents. Theirstrategies need longer historic records than the classicalsolutions. In fact, these strategies are different from thestrategies developed for the classical simultaneous prob-lem. For example, the Pavlov strategy performs well forsynchronous games but very poorly for asynchronous ones.On the other hand, a generous stochastic strategy like theFirm-but-fair strategy which retaliates after a non-cooper-ation (firm) and proposes cooperation (fair) just after witha given probability performs relatively well for asynchro-nous games [28].

These solutions introduced some of the concepts use inthis paper, like generosity and historic records. But, they donot consider stochastic cooperative participants who can-not participate when they do not have any unused re-source to share. In such cases, it is difficult todifferentiate partly cooperative from mostly selfishparticipants.

6. Conclusion

The Autonomic Service Architecture (ASA) model hasbeen proposed in the literature to automate the utilizationof resources. However, this model has the following twomajor limitations: (1) unused resources from underloadedVPNs are shared among overloaded ones without any re-gard for the lenders short-term needs and requirements;(2) operators of underloaded VPNs might behave selfishlyand refuse to share their unused resource with others over-loaded VPNs.

As a solution, we proposed a distributed autonomic re-source management model based on four novel strategiesthat are derived from Tit-for-Tat strategy to assure the evo-lution of cooperation among selfish and stochastic partici-pants. Participants might behave stochastically since theymay not have any unused (spare) resources to utilize overothers. Hence, more flexible strategies have to be devel-oped in order to see cooperation to emerge in a stochasticcontext.

From simulation results, we have noticed that the tradi-tional Tit-for-Tat cannot cope with our problem. A gener-ous strategy is proposed to overcome the limitation ofthe traditional one. Due to the generosity of the partici-pants, the strategy was unable to punish effectively theselfish ones. They receive some resources once in a while.To solve this problem, the altruistic strategy is proposedto punish purely selfish participants. As a drawback, thegenerous and the altruistic strategies require long historicrecords for each participant. Last but not least, the reputa-tion-based strategy is proposed to reduce the requirementson the historic records. Such a solution is interesting but itcan be seen as if the participants have a complete history of

the previous iterations. To overcome the generosity of thereputation model, we proposed to merge the altruistic andthe reputation models. The integration of both strategiescan improve the punishment during the initializationphase at the same time can maintain the same total gainas the reputation model.

In conclusion, the proposed strategies allow cooperativeparticipants to develop a mutual cooperation and achieveglobal acceptable gain and avoid mutual deadlocks. Onthe other hand, rational selfish participants would beforced to cooperate. They would have to offer their unusedresources almost as often as the cooperative participants ifthey need to be recognized as cooperative ones.

References

[1] N. Agoulmine, S. Balasubramaniam, D. Botvich, J. Strassner, E.Lehtihet, W. Donnelly, Challenges for autonomic networkmanagement, in: 1st IEEE International Workshop on ModellingAutonomic Communications Environments, IEEE, 2006.

[2] K. Appleby, S. Fakhouri, L. Fong, G. Goldszmidt, M. Kalantar, S.Krishnakumar, D.P. Pazel, J. Pershing, B. Rochwerger, Oceano: SLAbased management of a computing utility, in: Proceedings of theIEEE/IFIP International Symposium on Integrated NetworkManagement, IEEE, 2001.

[3] R. Axelrod, The Complexity of Cooperation, Princeton UniversityPress, Princeton, USA, 1997.

[4] R. Axelrod, The Evolution of Cooperation, Revised ed., Basic Books,New York, USA, 2006.

[5] G. Bouabene, C. Jelger, C. Tschudin, S. Schmid, A. Keller, M. May, Theautonomic network architecture (ANA), IEEE Journal on SelectedAreas in Communications 28 (2010) 4–14.

[6] Y. Cheng, R. Farha, M.S. Kim, A. Leon-Garcia, J.W.-K. Hong, A genericarchitecture for autonomic service and network management,Computer Communications 29 (2006) 3691–3709.

[7] T.M. Cover, J.A. Thomas, Elements of Information Theory, second ed.,Wiley-Interscience, Hoboken, USA, 2006.

[8] W. Cui, M.A. Bassiouni, Virtual private network bandwidthmanagement with traffic prediction, Computer Networks 42 (2003)765–778.

[9] N.G. Duffield, P. Goyal, A. Greenberg, P. Mishra, K. Ramakrishnan, J.van der Merive, A flexible model for resource management in virtualprivate networks, in: Proceedings of the SIGCOMM, ACM, 1999, pp.95–108.

[10] R. Farha, A. Leon-Garcia, Blueprint for an autonomic servicearchitecture, in: Proceedings of the 2nd ICAS 2006.

[11] C. Foley, S. Balasubramaniam, E. Power, M. Ponce de Leon, D. Botvich,D. Dudkowski, G. Nunzi, C. Mingardi, A framework for in-networkmanagement in heterogeneous future communication networks, in:Proceedings of the Third IEEE International Workshop ModellingAutonomic Communications Environments, LNCS, Springer, 2008,pp. 14–25.

[12] M.R. Frean, The prisoner’s dilemma without synchrony, Proceedingsof the Royal Society, Series B 257 (1994) 75–79.

[13] S. Graupner, A. Andrzejak, V. Kotov, H. Trinks, Adaptive controloverlay for service management, in: Proceedings of the Workshop onthe Design of Self-Managing Systems, IEEE, 2003.

[14] J.C. Harsanyi, A general theory of rational behavior in gamesituations, Econometrica 34 (1966) 613–634.

[15] C. Hauert, H.G. Schuster, Extending the iterated prisoner’s dilemmawithout synchrony, Journal of Theoretical Biology 192 (1998) 155–166.

[16] HP-Laboratories, The HP vision for the adaptive enterprise:achieving business agility, HP, July 2003.

[17] B. Jennings, S. van der Meer, S. Balasubramaniam, D. Botvich, M. ÓFoghlú, W. Donnelly, J. Strassner, Towards autonomic managementof communications networks, IEEE Communications Magazine 45(2007) 112–121.

[18] S. Karapantazis, F.-N. Pavlidou, VoIP: A comprehensive survey on apromising technology, Computer Networks 53 (2009) 2050–2090.

[19] J.O. Kephart, D.M. Chess, The vision of autonomic computing, IEEEComputer 36 (2003) 41–50.

[20] J. Maynard Smith, Evolution and the Theory of Games, CambridgeUniversity Press, Cambridge, UK, 1982.

J.-M. Robert et al. / Computer Networks 56 (2012) 927–939 939

[21] Microsoft-Corporation, Dynamic systems initiative overview,Microsoft.

[22] M. Mitzenmacher, E. Upfal, Probability and Computing: RandomizedAlgorithms and Probabilistic Analysis, Cambridge University Press,Cambridge, UK, 2005.

[23] P. Molander, The optimal level of genorisity in a selfish uncertainenvironment, Journal of Conflict Resolution 29 (1985) 611–618.

[24] D. Newth, Asynchronous iterated prisoner’s dilemma, AdaptiveBehavior 17 (2009) 175–183.

[25] D. Newth, D. Cornforth, The emergence of cooperation in the randomasynchronous prisoner’s dilemma, Artificial Life Robotics 12 (2008)329–334.

[26] M. Nowak, Stochastic strategies in the prisoner’s dilemma,Theoretical Population Biology 38 (1990) 93–112.

[27] M. Nowak, K. Sigmund, A strategy of win-stay, lose-shift thatoutperforms tit-for-tat in prisoner’s dilemma, Nature 364 (1993)56–58.

[28] M. Nowak, K. Sigmund, The alternating prisoner’s dilemma, Journalof Theoretical Biology 168 (1994) 219–226.

[29] W. Poundstone, Prisonner’s Dilemma, Anchor Books, New York, USA,1992.

[30] A. Quttoum, H. Otrok, Z. Dziong, A collusion-resistant mechanism forautonomic resource management in Virtual Private Networks,Computer Communications 33 (2010) 2070–2078.

[31] S. Raghunath, K.K. Ramakrishnan, Resource management for virtualprivate networks, IEEE Communications Magazine 45 (2007) 38–44.

[32] J. Rosenberg, H. Schulzrinne, G. Camarillo, A. Johnston, J. Peterson, R.Sparks, M. Handley, E. Schooler. Rfc 3261 – SIP: Session initiationprotocol. IETF, 2002.

Jean-Marc Robert holds an associate profes-sor position at ETS since June 2006 afterspending 10 years in the telecommunicationindustry. He worked at Gemplus as thedirector of the North American security groupand at Alcatel CTO Research and Innovation asa principal security researcher. His researchinterests include cryptography and computersecurity from telecommunication infrastruc-tures to computer hosts, from protocol secu-rity to embedded software security. Dr. Robertpublished numerous papers in international

conferences and academic journals. Finally, he is the author or co-authorof at least 12 issued patents.

Hadi Otrok holds an assistant professorposition in the department of computerengineering at Khalifa University. He receivedhis Ph.D. in Electrical and Computer Engi-neering (ECE) from Concordia University,Canada. His research interests are mainly onnetwork and computer security. Before join-ing Khalifa University, Dr. Otrok was holding apostdoctoral position at ETS. He is serving as atechnical program committee member fordifferent international conferences and regu-lar reviewer for different specialized journals.

Ahmad Nahar Quttoum is currently a Ph.D.candidate at ETS. His Ph.D. research topic is onthe resource management of Virtualized Net-works. He obtained his B.Eng. degree in 2006from Jordan University of Science and Tech-nology, Jordan. In 2007, he obtained a M.Sc.degree in Network Systems from the Univer-sity of Sunderland, UK His research interestsinclude virtualized networks, autonomicresource management, quality of service pro-visioning, and network security.

Rihab Boukhris is currently a Master’s can-didate at ETS. She obtained in 2006 a Master’sdegree from Institut Supérieur de Gestion ofTunis, Tunisia. Her research interests includerepetitive game theory and resource man-agement.