integrating a market-based model in trust-based service ... · challenges needs to be rigorously...

TSINGHUA SCIENCE AND TECHNOLOGYISSNll1007-0214ll02/10llpp554-567Volume 18, Number 6, December 2013

Integrating a Market-Based Model in Trust-Based Service Systems

Suronapee Phoomvuthisarn, Yan Liu�, Liming Zhu, and Ross Jeffery

Abstract: The reputation-based trust mechanism is a way to assess the trustworthiness of offered services, based

on the feedback obtained from their users. In the absence of appropriate safeguards, service users can still

manipulate this feedback. Auction mechanisms have already addressed the problem of manipulation by market-

trading participants. When auction mechanisms are applied to trust systems, their interaction with the trust systems

and associated overhead need to be quantitatively evaluated. This paper proposes two distributed architectures

based on centralized and hybrid computing for integrating an auction mechanism with the trust systems. The

empirical evaluation demonstrates how the architectures help to discourage users from giving untruthful feedback

and reduce the overhead costs of the auction mechanisms.

Key words: trust systems; software architecture; economic mechanisms

1 Introduction

Service oriented applications are highly distributedand loosely coupled with less central authority overparticipating for services. These services have a highdegree of autonomy, and hence most of them canarbitrarily claim service properties such as Quality ofService (QoS) in order to attract service users. Thisposes a number of risks to service users, who might endup selecting a poor-quality or even malicious serviceto cooperate. The Reputation-Based Trust mechanism(hereafter, the RBT) is one of the trust mechanismscommonly used for minimizing the risks of interactionsbetween participating services[1, 2]. This mechanismassumes that individual service users (or “raters” —� Suronapee Phoomvuthisarn is with the Faculty of Information

Sciences and Technology, Mahanakorn University ofTechnology, Bangkok, Thailand. E-mail: [email protected].�Yan Liu is with the Faculty of Engineering and Computer

Science, Concordia University, Montreal, Canada. E-mail:[email protected].� Liming Zhu and Ross Jeffery are with the National

ICT Australia and the School of Computer Scienceand Engineering, University of New South Wales,Sydney, Australia. E-mail: [email protected],[email protected].�To whom correspondence should be addressed.

Manuscript received: 2013-10-08; accepted: 2013-10-09

service users that rationally provide ratings to otherservices) can leave feedback such as ratings on servicesthat they have previously consumed[1]. These ratingscan be accumulated into a meaningful reputation toassess the trustworthiness of offered services.

There is, however, evidence that such feedback canbe manipulated[1, 3-5]. Some service users lie in theirfeedback intentionally to gain benefits from the biasedreputation that they establish, thus compromising thereputation produced by the RBT.

Economic mechanisms such as auctions are capableof addressing the problem of cheating behavior,even when the majority of market-trading participantslie. Unlike other approaches for preventing cheating,the auction mechanisms’ aim is to make sure lie doesnot gain, where the measure of gain for each bidder isindependent of the majority of bids from others[6]. Thisproperty enforces each participant to report the truthno matter how others act, and discourages dishonestraters from gaining benefits even when they arein a majority. Recognizing the merits of auctionmechanisms, this paper incorporates them into a trustnegotiation protocol. The aim is to provide the extracapability for trust systems to discourage raters fromcheating when they provide ratings to other services.

Supporting these additional auction mechanisms

Suronapee Phoomvuthisarn et al.: Integrating a Market-Based Model in Trust-Based Service Systems 555

within trust-based service systems necessitates asuitable distributed architecture for integration. Thisarchitecture ensures the following four trust systems’quality attributes:� Extensibility. The trust system can keep their

normal functionalities without incurring unaffordableoverhead when integrating with the auction mechanism;� Scalability. The overall architecture is highly

scalable when leveraging the auction mechanism thatwas originally designed for economics of a centralizednature;� Decentralization. The system can capture the

truth-telling property of the auction mechanism indistributed environments;� Performance. The architecture ensures an

affordable response time.Therefore, an architecture design to address abovechallenges needs to be rigorously evaluated.

This paper presents two architecture implementationsfor integrating an auction mechanism with trustsystems. The first architecture embeds the auctionmechanism within a trust system that coordinatesall the raters with a central coordinator. To makethe architecture further scalable, a hybrid architectureis designed to place the auction mechanism atdistributed services. Without a central coordinator,each service needs to calculate and share its trustratings to others. The quality of each architecturedesign is evaluated according to above four qualityattributes of extensibility, scalability, decentralization,and performance. The contribution of this paper istwofold. First, we provide an architectural approach tointegrating auction mechanisms with trust systems todiscourage raters from providing untruthful ratings toother services, even in scenarios where the majority ofthem are dishonest. Second, we evaluate this approachwith two architecture designs and compare qualityattributes with empirical experiments.

2 Background

2.1 Economic mechanisms

Economics is a social science that studies how humansmake choices on allocating scarce resources to satisfytheir unlimited wants[7]. Its mechanisms designed formarketplaces have already addressed the issue ofdiscouraging market-trading participants from cheatingbehavior[8]. One economic mechanism that yields truth-telling properties of market-trading participants is the

Vickrey Auction Mechanism (hereafter, the VAM)[9].The VAM’s rules are simple. Bidders place their bidsfor a particular item and hand them sealed to theauctioneer. The winner of the auction is the one whoplaces the highest bid and pays the exact amount of thesecond highest bid. After the auction, the reward andpunishment are conducted based on the utility gain (i.e.,the monetary gain after the auction) computed for eachbidder. The utility gain represented as money for thewinner is calculated by the difference between his orher true valuation for the item and the second highestbid in the auction, while the utility for other bidders iszero as follows:

ui D

(vi �Maxj¤i .bj /; if bi > Maxj¤i .bj /I

0; otherwise(1)

The VAM principle ensures that reporting the truevaluation maximizes the utility[9]. Bidders who submituntruthful bids will not gain, and are sometimespenalized. The property of the VAM encourages abidder to reveal the truth no matter how othersact[8]. Also, the VAM’s costs of computation areminimal because the modeling of participants’ behavioris simple and easy to implement. Furthermore, itdoes not require complex knowledge of how otherparticipants behave. This lightweight nature of the VAMis promising to satisfy the above four quality attributesrequired in the integration architecture.

In this paper, besides the VAM, we also incorporatethe Reverse Vickrey Auction Mechanism (hereafter, theRVAM) as a solution to the unfairness nature of theVAM. This generally happens when some raters whosubmit a very low but truthful bid have less chanceto win the auction. The measure of gain based on theRVAM is in a reverse order between the true valuationof the winning bidder who submits the lowest bid andthe second lowest bid in the auction.

2.2 Architectural challenges

The VAM is feasible to discourage raters from cheatingin trust-based domains; however, integrating the VAMwithin trust systems poses architectural challengeson balancing the quality attributes of extensibility,scalability, decentralization, and performance. From thearchitectural point of view, there are four concernsraised as follows.2.2.1 ExtensibilityThe first aspect is to achieve good separation ofconcerns. Auctioning messages incurred by the VAM

556 Tsinghua Science and Technology, December 2013, 18(6): 554-567

(e.g., sending, receiving, calculation, notification, etc.)might introduce tight couplings between the trustcomponents and the VAM components, thus degradingthe extensibility of the trust system itself. Trustand the VAM components should reduce the overlapin functionality so that modifications to the trustcomponents or replacement of the VAM componentscan be maintained without reengineering other parts ofthe architecture.

Besides, the VAM’s key characteristics need to becaptured in the deployed trust-based scenario. Eachrater has preferences to choose strategies that maximizeits utilities. Hence, the architecture should handlevarying preferences for all strategies to ensure thatsubmitting unfair ratings cannot provide benefits tounfair raters. In addition, the resource each participantputs into the auction depends on each strategythe participant performs. This requires the relationsbetween the strategies and their required resourcescaptured by relevant trust components and the VAMcomponents in the architecture.

2.2.2 ScalabilityThe original VAM auction mechanism assumes acentralized environment in which a central coordinatortakes care of all auctioning computation. Whenintegrated with the trust system, a centralizedarchitecture provides a common interface by whichraters can issue their bids through auctions. Such asystem therefore has limitations to scale and handle theincreased load of raters. This centralized infrastructureis not suitable for ultra-large scale service environmentwhere the number of potential services is huge. Thisnecessitates a distributed trust system that can handlebids from raters independently so that raters are notblocked while their bids are executed in the auctioningprocess.

2.2.3 DecentralizationTo achieve better scalability, the VAM can beembedded with distributed services without acentral coordinator. Consequently, each serviceneeds to calculate and share their informationto others. However, capturing the VAM’s truth-telling property in this environment is notstraightforward. Traditional VAM is suitable fora class of problems where bidders’ budgets areunlimited[10]. Without central authority, a distributedservice environment lacks the overall knowledge ofindividual’s budgets. Hence, services may take an

underbidding strategy to gain remote resources. Asa result, the VAM’s property to prevent lying maybe broken. Moreover, reporting the VAM outcomeis not trivial since the results need to be distributedto dynamic raters, which is not a known priori, ormay even change over time as a service joins orleaves. Obviously, explicit polling by the VAM fornotification incurs communication overhead. All leadto the challenge of capturing the VAM’s truth-tellingproperty in decentralized architecture.

2.2.4 PerformanceThe last aspect is to reduce overhead as discussed in theabove challenge of decentralization. When the numberof participating services involved is very high, thevolume of messages exchanged between the VAM andtrust components might incur significant overhead. Thisimposes a further research question in the architecturaldesign: how to leverage the VAM’s benefits to preventraters from cheating, while reducing the end-to-enddelay incurred by the VAM messages. The resultingarchitecture should be optimized to reduce this trafficoverhead significantly.

3 The Architecture

This section presents two VAM-based architectures todiscourage raters from cheating, namely centralizedand hybrid computing architecture. These VAM-based architectures are demonstrated through two casestudies. Each case has its own emphasis on specificquality attributes. They are then compared to findoptimal architectural solutions. The use of case studiesprovides insights on how these architectures can beextended with the VAM.

3.1 Centralized architecture

This centralized architecture relies on the system’s trustmanager embedding with the VAM’s capabilities to helpa requester find a set of potential providing services thathave high reputations so as to assist the choice of theircooperation.

3.1.1 Case descriptionThe case follows the supply chain scenario, wherea customer service (acting as a requester) intends tofind potential providing services in the line of supplychain including retailer, warehouse, and manufacturerservices to negotiate. To complete this line of business,the customer has to submit an order consisting of lineitems to a retailer service. Each line item identifies


a product and the corresponding quantity to beordered. To fulfill orders, the retailer goes through eachline item and finds a warehouse service with sufficientstock to ship them. The warehouse then ships the lineitem to the customer.

To manage stock levels in the warehouse, eachwarehouse needs to restock from the inventory of arelevant manufacturer service whenever its inventorylevels fall below the minimum of inventory levelsfor a particular product. To choose the best dealavailable among all the providing services (i.e., retailer,warehouse, and manufacturer services) that havesimilar functions, the requesting service (i.e., customer,chosen retailer, and chosen warehouse service) sendsa trust system the request asking for the reputationof relevant providers on how well they can providetheir QoS information as claimed. Since each service isindependent of the services or resources it provides[11],the interactions between participating services have therisks of failure, such as time delay and not alwaysaccessible within the promised timeline. Therefore, theoverall performance QoS attributes, including responsetime, waiting time, and availability, can be used in thisscenario.

The VAM comes into play in this context tomotivate each rater to reveal the rating faithfully. Inthe case of this supply chain, the customer serviceinvokes the central trust system to evaluate eachretailer’s trustworthiness before sending the list ofline items. Once having taken a decision for choosingone retailer, the chosen retailer then invokes the trustsystem to evaluate each warehouse’s trustworthinessbefore ordering a shipment from one warehouse. If thechosen warehouse’s stock is needed to be refilled, thewarehouse then invokes the trust system to evaluateeach manufacturer’s trustworthiness before ordering aproduct from the manufacturer.

3.1.2 Architecture layersConceptually, the architecture of a trust system hasthree layers to compose the key components, namelyservice layer, trust layer, and service metadata layer asdepicted in Fig. 1.

The service layer performs bootstrapping, whichreceives ratings from raters. The trust layer preventsaccumulation of unfair ratings. This layer consists of thetrust engine to calculate trust-related information, suchas a trust level of certain services, and the reputationengine to provide a robust reputation. The service

Fig. 1 Auction-based centralized architecture.

metadata layer contains the service registry to supportthe VAM’s reward and punishment process made toraters. The registry is extended to capture the reputationof registered services. In addition, a database is used tostore trust-related information such as trust parameters.

3.1.3 Auction-based trust negotiation protocolThe steps of the VAM are embedded in the trustnegotiation protocol, to guide the interactions betweenkey components across three service layers.

At Stage one, interrogation, a requester at theservice layer finds providing services that meet itsfunctional requirements. The requester checks the trustlevel of the discovered providers through Stage two,negotiation. Once receiving the requests from the trustengine for calculating the trust level of a provider, thereasoning manager then instructs the auction engine toperform the auction-based calculation. The calculationsteps are shown in Fig. 2.

(1) Auction engine initiates a new auction round(one auction per each provider requested) by setting anauction time and aggregates ratings from raters. Theauction engine encapsulates the auction logic in theVAM component, which computes the utility gain ofparticipating raters based on Eq. (1) (Section 2.1). Thisutility gain is input to the reasoning manager tomake the decision on a reward to fair raters or somepunishment to unfair raters. To calculate this utility,in addition to the second highest bid captured in theauction, the true value of the rating submitted by thewinning rater is required (see Eq. (1)).

(2) Auction engine terminates the auction.(3) Reasoning manager sends all values of ratings

already excluded to the auction engine.(4) Auction engine finds the winning rater and

instructs the controller to monitor the winner’s true


Fig. 2 Trust negotiation protocol in centralized architecture.

value. In this paper the term “the winning rater” refersto the raters who submit the highest rating (in the VAMprocess) and the lowest rating (in the RVAM process) inthe auction.

(5) Controller monitors the winning rater’s truevalue.

(6) Auction engine instructs the VAM component toperform utility computation.

(7) Auction engine then announces the winning raterand its utility through the reasoning manager.

(8) Reasoning manager performs the VAM’s rewardand punishment process and updates the newly raters’reputation to the service registry.

(9) Reasoning manager then calculates the provider’sreputation.

The trust engine then uses the provider’s reputationproduced by the reasoning manager to calculatethe provider’s trust level. Once the computation iscompleted, the calculated trust level is then returned tothe requester to determine whether each provider’s trustlevel exceeds the requester’s minimal trust threshold tofurther negotiate.

At Stage three, interaction, the requester negotiateswith a chosen provider to interact with. The requesterand the chosen provider use the trust system as amediator to establish the trust negotiation betweenthem. At Stage four, termination, once completelyconsuming the chosen provider’s service, the requesterends trust negotiation with the chosen provider.

3.2 Hybrid architecture

In the previous case study, the VAM-based centralizedarchitecture makes all the decisions in one place. This

poses a scalability issue when concurrent raters areparticipating with the central trust system at thesame time. Simply decentralizing the architecturalcomponents in the centralized architecture to eachparticipating service cannot solve this scalability issue,since query messages incurred by the auction-basedtrust negotiation protocol might induce a large volumeof traffic overhead due to the end-to-end delay. Oursolution is to the combination of them into a hybridcomputing architecture.

3.2.1 Case descriptionThe supply chain scenario is further extended to explorethe hybrid computing architecture. In Fig. 3, each ofregistry services maintains the centralized architecturepresented in Section 3.1. Each registry service has itsservice indexes and implements the VAM protocol inthe trust manager.

Each service registry acts as a broker andcommunicates with each other in a decentralizedmanner to find potential providers (i.e., retailer,warehouse, and manufacturer services) for arequester. After receiving a list of relevant providersfrom registries’ peers, the registry invokes its trustmanager to evaluate the trustworthiness of theseproviders.

3.2.2 Architecture layersFigure 4 shows the architecture of hybrid computing. Itmainly consists of a set of service registries, eachof which acts as a central server in resolving aquery message for a requesting service. These serviceregistries maintain a set of their own services registeredto the trust system using indexing scheme in the


Fig. 3 Supply chain scenario in hybrid computing.

Fig. 4 The VAM architecture for hybrid computing.

service description database, and communicate witheach other in a P2P manner. In this nature, propagationof messages between individual services only occurswithin a small population of service registries, whichhelp to reduce the messages communicated by the VAMprotocol.

Consequently, each service’s registry componentcommunicates with other services across four servicelayers, namely application layer, queuing layer, trustlayer, and service metadata layer.

The application layer handles an incoming requestfrom requesting services as well as an outgoing requestfrom a service registry. It interacts with the registry’sown registered services and other peers of the registryin resolving the query message of a requester. Thelayer also receives ratings submitted by raters in theauctioning process through the discovery manager.

The queuing layer aims to reduce a runtime overheadincurred by interchange messages passing between theapplication layer and the trust layer. It consists of (1)

the service caching to previously store evaluated resultsof a requester’s incoming request, (2) the queue list tostore the ratings of raters when the application cannotconcurrently process them, and (3) the scheduler engineto schedule the execution of these ratings according toa specific set of constraints.

The trust layer incorporates components to preventaccumulation of unfair ratings. This layer consists oftwo key components: (1) the trust manager from thecentralized architecture is reused to produce a trustlevel or reputation of providing services accumulatedfrom truthful ratings in the VAM process; while(2) the matchmaking manager is a broker for aregistry to interact with the discovery manager, theservice description database, and the trust manager forevaluating the trustworthiness of a providing service.

The service metadata layer stores trust data in thesystem. The repositories, including trust repository andreputation repository are reused from the centralizedarchitecture. Since each service registry interacts with


each other in a decentralized manner, the reputationrepositories consist of both (1) the local reputationrepository to store the reputation of raters in whichthe registry itself is responsible for, and (2) theglobal reputation repository to store the reputation ofother raters organized by its set of registries’ trustedneighbors (registries’ peers). The layer also contains aservice description database to store service informationindexed when services register to the system.

To propagate changes among shared data (e.g., raters’reputation credits), the proxy offers an observer-basedchange notification mechanism in which the globalreputation repository is the subject and each registry’strust manager as the observer. The proxy can triggerthe trust manager of each registry to be notified aboutthe changes of raters’ budget when they participate inmultiple auctions through its own reasoning manager.

3.2.3 Auction-based trust negotiation protocol

The auction-based trust negotiation protocol consistsof the following five stages: initiation, matching,interrogation, negotiation, and interaction. Thisprotocol interacts between a requester and a serviceregistry to choose a provider for the requester withwhich to cooperate as shown in Fig. 5.

At Stage one, initiation, the discovery managerreceives a requester’s query message, and then instructsthe matchmaking manager to resolve the request for arelevant provider.

At Stage two, matching, when the matchmaking

manager receives the query message, it first checksits local cache for relevant providing services thatcan meet the requester’s requirements in the querymessage. If the number of matching providers isnot enough for the requester, it then forwards thequery message to other registries’ peers. Once it hasreceived a sufficient number of relevant providers, thematchmaking manager then keeps the list of theseproviders for evaluating their trustworthiness.

At Stage three, interrogation, after receiving a listof relevant providers, if the service registry has enoughexperience with those providers, it can choose one or aset of the providers that it trusts most for the requester.Otherwise, the service registry requests raters’ opinionsabout the reputation of these providers. To computethese reputations, the reasoning manager instructs theauction engine to initialize auction services to gatherall values of ratings from raters. The auction-basedcalculation follows the similar steps of the protocolin the centralized architecture. After calculating thereputations of the potential providers, the trust enginethen uses these reputations to calculate their trust level,which is used to determine whether each provider’s trustlevel exceeds the requester’s trust threshold for furthernegotiation.

At Stage four, negotiation, the requester thenestablishes a trust negotiation with the chosen providerdirectly to get its service. At Stage five, interaction,after the trust negotiation is established, the requesterthen gets the chosen provider’s service.

Fig. 5 Trust negotiation protocol in hybrid architecture.


4 Architecture Deployment

In order to realize the VAM-based architectures, thispaper uses JXTA (2003), an open source peer-to-peerprotocol specification initiated by Sun Microsystems,as a platform for web service publication anddiscovery infrastructure. Compared to other distributedweb service discovery, JXTA provides service groupconcurrent searching by aggregating a search resultof each service group, thus its architecture is suitablefor web service discovery and selection processthat requires highly efficient searching[12]. The JXTAnetwork (Sun Microsystems, 2003) is built out offive key abstractions—uniform peer ID addressing,peer groups, advertisements, resolver, and pipes—thatprovide a generic infrastructure to deploy P2P servicesand applications. The deployment of the VAM-basedarchitectures is as follow.

4.1 Centralized architecture

The architecture deployment is shown in Fig. 6. Allservices including the all raters and all servicesare deployed as JXTA edge nodes hosted by theApache AXIS 1.0 Web Server. All raters and retailer,warehouse, and manufacturer services interact withthe trust-based service application via the Web servicedeployed in the IIS 5.0 Web Server that receives servicerequests and sends responses from the application.

The trust engine and the reputation engine aredeveloped as Java EJBs and deployed as the single trust-based application hosted by the Tomcat ApplicationServer.

This Application Server processes requests from theWeb Server and sends responses back to the WebServer.

Fig. 6 Deployment of centralized architecture.

The Application Server implements the auction-based trust negotiation protocol. It communicates withservices and the service registry using SOAP messagesand connects to the SQL Database Server using theJDBC driver. The service registry implements LDAPcomponents to support service registration.

4.2 Hybrid architecture

The architecture deployment is shown in Fig. 7. Allservices including customers, retailers, warehouses,manufacturers are built using JXTA relay nodes with 3friends per peer. They are randomly assigned to one ofthe service registry deployed as rendezvous nodes witheach having its own trust manager. With a rendezvouscapability, a service registry can maintain an indexof advertisement published by providing services todiscovery requests using a Shared Resource DistributedIndex (SRDI) (Sun Microsystems, 2003). These serviceregistries keep this random list of their registry’s peersranging from 1 to 10 peers and maintain a maximumnumber of 50 registered services per registry. Theseregistries interact with all raters through auctionsprocessed by their associated trust managers thatreceive ratings or any service requests.

The components of the trust managers and theApplication Server are implemented and deployed thesame way as the centralized architecture.

5 Evaluation

The architectural framework presented in Section 3 isnow applied to three test cases as follows:� The VAM property test Unfair raters should get

a penalty, especially when the majority of them lie abouttheir ratings.� The performance test evaluates overhead of the

devised architecture integrated with the VAM. Thetest implements two prototypes with and without the

Fig. 7 Deployment of hybrid architecture.


VAM deployed and observes the overhead cost for eachdeployment.� The scalability test evaluates the extent to which

the VAM-based architectures can scale in terms ofinteracting raters. The test is to compare the responsetime of two architectures (centralized vs hybrid) whenthe number of interacting raters increases.

5.1 Test setup

The test environment includes two identical WindowsXP machines with 3 GHz Core 2 Duo processors and3.25 GB of RAM. One is used for hosting all raters’services, and the other hosts the rest of the application.

The auction process initially involves 50 raters foreach case study. The raters are grouped into two groups:fair and unfair raters. A fair rater offers QoS ratingsto a provider just as it is perceived while an unfairrater randomly offers QoS ratings above or below whatit perceived. To simplify the problem of cheating byraters, the approach of this paper considers the casewhere all raters have their past experience with a certainprovider with 70% probability. Each of them has to rateeach provider with 50% probability.

At the end of each auction, the reputation ofeach participating rater is updated based on theutility gain/loss calculated by Eq. (1). To update thereputation of a rater with the calculated utility gain,the rater’s newly updated reputation is calculated byaccumulating the rater’s utility gain with the rater’scurrent reputation. For example, if a rater, whosereputation is 20 credits, gets a positive 0.6 utilitygain, the newly updated reputation is 20.6 credits. Thereputation of unfair raters is initially set to any randomnumbers between 5 to 15 credits while for the fair ratersit is between 10 to 20 credits based on 100 transactionspreviously conducted. The minimal credits of raters toparticipate in auctions are initially set as 5 credits.

In the case of centralized architecture, the newlyupdated reputation of a rater can be directly storedin the centralized service registry. However, to makethese newly updated reputations publicly known toothers in the hybrid architecture, a Distributed HashTables (DHT) technique[13] is applied to store anindex of raters’ reputation in each service registry’speers. This DHT method uses multiple hash functionsto map a single rater’s ID to corresponding trustedneighbors that calculate and store the reputation ofraters individually. DHT also provides a basic operationfor retrieving the reputation of raters. When services

need the reputation of raters, they can retrieve themusing DHT operation with their ID as a parameter.

In a service selection phase, after completing thecalculations of relevant providers’ reputation, thetrust system then ranks these providers based on therequester’s preferences. At the end, the requester willget a sorted list of potential providers with which tocooperate.

To select some of these providers for the requester,the trust system ranks them based on the overall scoretaking into account both (1) QoS score published bythe providers in service registries, and (2) reputationscore of the providers calculated by trust systems. Bothparameters’ thresholds are specified in the servicediscovery request by the requester. The details of eachparameter are in the following.

(1) The QoS score is an average real number in[0,1] specified by a provider when it publishes its QoSinformation to a service registry. In this paper, serviceproviders specify QoS information by using the conceptof embedded tModel in WSDL files.

(2) The reputation score is a measure of a provider’strustworthiness. It can be computed by collectingratings from other services that have previouslyinteracted with the provider. The reputation of oneprovider can be calculated by aggregating its ratingssubmitted by raters into one percentage measureReputation in [0,1], each of which is weighted by raters’reputation[1] as shown in Eq. (2).

Reputation =nX

iD1wi � ratingi (2)

where wi represents a weighted reputation in [0,1] of arater, ratingi represents a rating rated by a rater i , and nrepresents a number of raters.

After taking into account both parameters, the overallscore of a provider in the ranking process can becalculated as shown in Eq. (3) below.Overall score D ˛ �QoS scoreC ˇ � Reputation (3)

where ˛ and ˇ represent weights in [0,1], which issubjectively determined by the requesters depending onhow important each source is (˛ + ˇ = 1.0).

Based on the requesters’ preferences, the N numbersof providing services that satisfy the requesters’threshold are then returned in descending order basedon the overall score. Services whose overall scoreexceeds the trust level of 0.7, further proceed innegotiate.


5.2 VAM property test

Unfair raters should get a penalty, especially when themajority of them are dishonest. This can be measuredby the average reputation of unfair raters when theyconstantly provide unfairly high or low ratings topotential providing services.

5.2.1 Centralized architectureThe number of unfair raters is varied from 10% to 90%,with 10% increment per experiment. Each experimentinvolves 50 raters, each of which rates QoS ratings ofproviders based on given probabilities in Section 4. Atotal of 10, 300, and 500 auction rounds (equal to thenumber of the targeted providing services being rated)have been executed to observe the reputation changesincurred by cheating behavior of raters.

Figures 8 and 9 show that the average reputationof unfair raters decreases when the number of raterswho lie increases under two different strategies. Themain difference is that when unfair raters performan overbidding strategy, their average reputation isdecreasing much more when compared to the randombidding strategy. This is because in the overbid case,the unfair raters who rate others bids very high havemore chance to win the auction compared to the unfair

Fig. 8 Changes in the reputation of raters for the overbidcase, when an unfair rater issues its bid higher than itperceived.

Fig. 9 Changes in the reputation of raters for the randombid case, when an unfair rater issues its bid randomly eithera higher or lower bid than it perceived.

raters in the random bid case. However, their gainsof reputation are impaired by the VAM’s punishmentprocess made to the dishonest raters who submits thehighest QoS rating. In contrast, we can see that theaverage reputation of fair raters from both the overbidand random bid case increases as the result of the rewardgranted by the VAM process.

In the underbid case, an unfair rater issues its bidlower than it perceived. Before applying the RVAM’sutility functions with the VAM’s calculation logic, theaverage reputation of unfair raters are not getting muchdecreased when they issue their bids lower than theyactually perceived as shown in Fig. 10.

This is because the unfair raters who rate others bidsvery low have less chance to win the auction in the casethat they are in the minority. However, after applyingthe RVAM’s utility functions, their gains of reputationsare degraded due to the VAM’s punishment processmade to the dishonest raters who submits the lowestQoS rating as depicted in Fig. 11.

The results clearly demonstrate that the VAM helpspreventing cheating behavior by ensuring that unfairraters get a penalty when they simply lie. This is evidentby the significant decrease in the average reputationof unfair raters, especially when the majority of themare dishonest. Also, the approach promotes a direct

Fig. 10 Changes in the reputation of unfair raters for theunderbid case before integrating with RVAM.

Fig. 11 Changes in the reputation of unfair raters for theunderbid case after integrating with RVAM.


incentive for fair raters participating in an auction dueto the increasing of the average reputation of fair raterswhen they give ratings truthfully in the auction.

5.2.2 Hybrid architectureThe number of unfair raters varies from 10% to 90%,with 10% increment per experiment. Each experimentinvolves 100 000 raters, each of which rates QoS ratingsof providing services including retailers, warehouse,and manufacturers requested by requesting servicesbased on given probabilities in Section 5.1. The sametotal of auction rounds (10, 300, and 500 equalto the number of the providers rated) have beenexecuted to observe the effect of changes in theaverage reputation of raters. The providing serviceswere randomly grouped into 10 service registries, eachof which can contain the maximum of 50 providingservices.

The results of the overbid case and the random bidcase can discourage cheating by raters (the same as inthe case of centralized computing). However, in thecase of the underbid, unfair raters have slight gainsof reputation as depicted in Fig. 12. After integratingwith RVAM to detect the underbidding strategy ofunfair raters, the results demonstrate that the averagereputation of unfair raters decreases as depicted inFig. 13.

The results demonstrate that integrating RVAM in

Fig. 12 Changes in the reputation of raters for the underbidcase before integrating with RVAM.

Fig. 13 Changes in the reputation of raters for the underbidcase after integrating with RVAM.

the hybrid architecture discourages cheating behavior ofraters by ensuring that unfair raters get a penalty whenthey simply lie in a distributed environment.

5.3 Performance and scalability test

In the case of centralized architecture, thecomputational overhead is measured in terms ofCPU usage and memory usage as shown in Table1. The result shows the resource usage of deployingVAM is comparable to the deployment without VAM.

We further measure the application’s response time(second) with and without the VAM deployed. Byvarying a number of raters from 50 to 1000, theexperiments are performed with 10 concurrent auctions,each of which is conducted for one providing servicebeing rated.

The results show in the centralized architecture, theresponse time is almost identical with/without VAM(see Fig. 14). The performance overhead with VAM isapproximately 3.9% higher than without VAM, as thenumber of raters is up to 1000.

In the case of the hybrid architecture, theperformance overhead with VAM is approximately5.7% higher than without VAM for 10 000 raters (seeFig. 15).

Table 1 CPU and Memory Usage with/ without VAM for1000 raters.

CPU usage (%) Memory usage (%)With VAM 53 19.35

Without VAM 48 17.42

Fig. 14 Performance and scalability of centralizedarchitecture.

Fig. 15 Performance and scalability of hybrid architecture.


5.4 Discussions

This paper aims to support trust-based services on avery large scale to prevent unfair raters from cheating,especially when they are in the majority, in the serviceselection process. Based on the evaluation, integratingVAM in the service architecture (centralized or hybrid)can prevent benefits to dishonest raters. Each has itsown strengths and weaknesses that make it suitable fora certain service environment.

The centralized architecture is suitable for small-scale service systems where the number of raters is notvery high. This architecture has a simple deploymentsince all of the processing is controlled in a centrallocation. The hybrid architecture can scale to handlelarger number of raters and service providers.

6 Related Work

There are two main problems for the trust systems inthe use of the RBT. The first problem is that of unfairratings. A number of research studies have proposedtechniques to tackle the problem of unfair ratingsflowing from raters’ cheating behavior. These include:(1) detective techniques[2, 14-16], which detect unfairratings for predicting the trend of raters’ untruthfulbehavior with statistical measures (e.g., clustering);and (2) preventive techniques[3-5], such as the sidepayment schemes[17], which discourages raters fromlying by giving them some kind of incentives (e.g.,digital currency or credit), so that truthful reportingmaximizes the raters’ expected revenue.

Our approach generally belongs to the category ofpreventive techniques. Preventive techniques whoseeffectiveness replies on the majority of the participantsto be truthful are difficult to apply in the largescale service oriented computing. In service-orientedcomputing, participating services are loosely coupledand dynamic in nature[18], a large number ofservices are expected to grow very quickly inthe future[12, 18, 19]. Therefore the contribution of ourapproach is identifying a well-proven preventivemechanism that is suitable to integrate with thetrust-based services even majority of participants aredishonest.

Most research work related to theRBT[3-5, 14, 16, 17, 20, 21] has focused on devising amechanism to solve the problems of raters’ cheatingbehavior. Only a few studies[22-24] have investigated theoverhead costs incurred by these proposed mechanisms

when they are integrated with the original trustsystems. One study[23] evaluates the efficiency of itsproposed mechanism in terms of the consumptionof computing resources (i.e., CPU usage). Wanget al.[22] also dealed with the overhead cost ofintegrated mechanisms within trust systems. Theauthors attempted to investigate the cost and efficiencyof the trust systems where such mechanisms areapplied. By generalizing individual small groupsof networks, the authors quantitatively analyzedthe behavior of complex networks based on themathematical equations. Another study is from Zhaoet al.[24] that integrates the mechanism as a number ofexchange messages between participants, realized inrelevant layers in a message feedback protocol.

These approaches assume the existence of centralizedparty that maintains the digital currency or reputationscores of participants to enforce their mechanisms’truth-telling properties through rewarding or charging.Therefore, the computation of such approaches is allcentralized in the trust system[25]. This imposes researchquestions concerning their extension to support serviceoriented computing, while achieving performance andscalability.

All of the above demonstrate the need for a properarchitecture to integrate a preventive mechanismwith existing trust systems. This architecture needsto maintain quality attributes of the trust system:extensibility, decentralization, scalability, andperformance.

7 Conclusions

In this paper, we integrate an auction mechanismwith trust systems to discourage raters from providinguntruthful ratings to other services, even in scenarioswhere most of raters are dishonest. There are twomain contributions in this paper. The first contributionis an extension to existing trust systems, adding theextra capability to prevent cheating behavior on thepart of raters. The notion of the two architectureswith the VAM-based auction mechanism induces aneffective trust negotiation by preventing the trustsystems from being exploited by raters, even if themajority of them lie about their ratings. Second, thetwo architectures developed support the integrationof the VAM in term of optimized overhead costsregarding the quality attributes of: (1) extensibilityin terms of reusing software components across two


architectural designs; (2) scalability in terms of largeconcurrent raters interacting with trust systems; (3)decentralization for capturing the VAM’s truth-tellingproperty in distributed environments; and (4) nosignificant performance overhead due to the end-to-enddelay incurred by a significant amount of messages inhighly scalable trust systems.

The VAM can effectively discourage raters fromsubmitting untruthful ratings. However, the VAMassumes that each bidder submits his or her bidindependently. This assumption raises the issue ofhow far the VAM can be applied in situations whereplayers can communicate with others outside of thegame. Thus, a number of participating services (e.g.,raters) can communicate with each other and thusmanage their bids in an auction. Apart from biddingcollusion[26], the VAM rule that bids are sealed mightcause some difficulty in enforcing bid privacy inservice oriented computing. In a common auction, sometricky sellers might adopt a strategy of tracing thebids of others before submitting their own bids inthe auction. In this way, such a seller can managehis or her bid in order to gain benefits. Futurework involves providing architectural solutions to solvethese problems. Hence, in future work, the resultingarchitecture will be proposed to support protectionagainst bidding collusion, as well as integrating somesecurity mechanisms to protect bid privacy.

References

[1] A. Jøsang, R. Ismail, and C. Boyd, A survey of trust andreputation systems for online service provision, DecisionSupport Systems, vol. 43, no. 2, pp. 618-644, 2007.

[2] H. T. Nguyen, W. Zhao, and J. Yang, A trust and reputationmodel based on Bayesian network for web services, inProc. 8th IEEE Int. Conf. on Web Services (ICWS), Miami,Florida, USA, 2010, pp. 251-258.

[3] E. Gerding, K. Larson, and N. Jennings, Mechanismdesign for eliciting probabilistic estimates from multiplesuppliers with unknown costs and limited precision, inProc. 11th Int. Workshop on Agent-Mediated ElectronicCommerce, 2009, pp. 1-124.

[4] J. Miller, Game Theory at Work: How to Use Game Theoryto Outthink and Outmaneuver Your Competition. McGraw-Hill, 2003.

[5] M. Yang, Q. Feng, Y. Dai, and Z. Zhang, A multi-dimensional reputation system combined with trust andincentive mechanism in P2P file sharing systems, inProc. 27th Int. Conf. on Distributed Computing SystemsWorkshops, Toronto, Canada, 2007, pp. 29-35.

[6] L. V. Ahn, Auctions: Science of the web coursenotes, http://www.scienceoftheweb.org/15-396/lectures/

lecture09.pdf, 2008.[7] S. Shetty, P. Padala, and M. P. Frank, A survey of market-

based approaches to distributed computing, Technicalreport, no. TR03013, 2003.

[8] P. Klemperer, Auctions: Theory and Practice. PrincetonUniversity Press, New Jersey, USA, 2004.

[9] W. Vickrey, Counterspeculation, auctions, and competitivesealed tenders, Journal of Finance, vol. 16, no. 1, pp. 8-37,1961.

[10] A. Clayphan and T. Min’an, Mechanism simulationframework for large distributed systems, Captone projectreport, The University of New South Wales, Sydney,Australia, 2008.

[11] M. P. Papazoglou, P. Traverso, S. Dustdar, and F. Leymann,Service-oriented computing: State of the art and researchchallenges, IEEE Computer, vol. 40, no 11, pp. 38-45,2007.

[12] D. Wu, S. Ye, X. Wu, and J. Wei, Implementing service-correlation aware service selection, in Int. Conf. on ServiceSciences (ICSS), Hangzhou, China, 2010, pp. 383-387.

[13] G. Urdaneta, G. Pierre, and M. V. Steen, A survey of DHTsecurity techniques, ACM Computing Surveys, vol. 43,no. 2, 2011.

[14] K. Karta, An investigation on personalized collaborativefiltering for web service selection, Master degreedissertation, The University of Western Australia, Perth,Australia, 2005.

[15] Z. Li, S. Su, and F. Yang, WSrep: A novel reputationmodel for web services selection, in Agent and Multi-AgentSystems: Technologies and Applications; N.T. Nguyen,A. Grzech, R. J. Howlett, L. C. Jain, Eds., Springer-Verlag,2008, pp. 199-208.

[16] Z. Malik and A. Bouguettaya, A rater credibilityassessment in web services interactions, World Wide WebJournal, vol. 12, no. 1, pp. 3-25, 2009.

[17] R. Jurca and B. Faltings, An incentive-compatiblereputation mechanism, in Proc. the IEEE Conference onE-Commerce, Newport Beach, CA, USA, 2003.

[18] C. Wan, C. Ullrich, L. Chen, R. Huang, J. Luo, andZ. Shi, On solving QoS-aware service selection problemwith service composition, in Proc. 7th Int. Conf. on Gridand Cooperative Computing, 2008, pp. 467-474.

[19] Y. Wang and J. Vassileva, Towards trust and reputationbased web service selection: A survey, InternationalTransactions on Systems Science and Applications, vol. 3,no. 2, pp. 118-132, 2007.

[20] Y. Liu, A. Ngu, and L. Zheng, QoS computation andpolicing in dynamic web service selection, in Proc. 13thInt. World Wide Web Conference, 2004, pp. 66-73.

[21] L-H. Vu, M. Hauswirth, and K. Aberer, Towards P2P-based semantic web service discovery with QoS support,in Proc. Business Process Management Workshops, 2005,pp. 18-31.

[22] Y. Wang, Y. Hori, and K. Sakurai, Characterizingeconomic and social properties of trust and reputationsystems in P2P environment, Journal of Computer Scienceand Technology, vol. 23, no. 1, pp. 129-140, 2008.


[23] J. Witkowski, Elicit honest reputation feedback in aMarkov setting, in Proc. 21st Int. Joint Conf. onUncertainty in Artificial Intelligence, Pasadena, California,USA, 2009, pp. 330-335.

[24] H. Zhao, X. Yang, and X. Li, An incentive mechanism toreinforce truthful reports in reputation systems, Journal ofNetwork and Computer Applications (JNCA), vol. 35, no3, pp. 951-961, 2012.

[25] T. Moscibroda and S. Schmid, On mechanism designwithout payments for throughput maximization, inProc. 28th IEEE Conf. on Computer Communications(INFOCOM ‘09), Rio de Janeiro, Brazil, 2009.

[26] F. Brandt and T. Sandholm, Efficient privacy-preservingprotocols for multi-unit auctions, in FinancialCryptography and Data Security, A. Patrick, andM. Yung, Eds., Springer-Verlag, 2005, pp. 298-312.

Suronapee Phoomvuthisarn is a lecturerin the Faculty of Information Sciencesand Technology, Mahanakorn Universityof Technology, Bangkok, Thailand. Hereceived his MS degree in informationsciences from the University of Pittsburghin 2007 and PhD in computer sciences andengineering from the University of New

South Wales in 2011. His research interests include serviceeconomics, trust computing, and software architecture. He iscurrently the vice-dean for Public Relations and Student Affairsof the Faculty of Information Sciences and Technology atMahanakorn University.

Yan Liu is an associate professorin the Faculty of Engineering andComputer Science, Concordia University,Canada. Her current research involvescloud computing, distributed systems,software architecture, and model drivendevelopment. She has extensive researchexperience leading projects in domains

arranging from enterprise applications, to data intensiveworkflows, to energy systems and smart grids. She has publishedover sixty international journal and conference papers in theseareas since 2004. She was a Senor Scientist at Pacific NorthwestNational Laboratory from 2009 to 2012. She received her PhDin computer science from University of Sydney, Australia in2004. Until December 2009, she was a Senior Researcher andproject leader at NICTA (National ICT Australia).

Liming Zhu is a principal researcherat NICTA. NICTA is Australia’s centreof excellence for Information andCommunications Technology R&D. Healso holds conjoint positions at Universityof New South Wales (UNSW) andUniversity of Sydney. He received his PhDin software engineering from UNSW in

2007. His current research interests include dependable systemsand operation, software architecture, and software engineering.

Ross Jeffery is Emeritus Professor ofSoftware Engineering in the Schoolof Computer Science and Engineeringat UNSW and Research Leader inNICTA. He received his PhD in softwareengineering from the University of NewSouth Wales. His current research interestsare in software engineering process and

product modeling and improvement, electronic process guidesand software knowledge management, software quality, softwaremetrics, software technical and management reviews, andsoftware resource modeling and estimation. His research hasinvolved many government and industry organizations overa period of 30 years and has been funded from industry,government, and universities. He has co-authored four booksand over one hundred and thirty research papers. He has servedon the editorial board of the IEEE Transactions on SoftwareEngineering, and the Wiley International Series in InformationSystems and the Journal of Empirical Software Engineering. Heis a founding member of the International Software EngineeringResearch Network (ISERN). He was elected Fellow of theAustralian Computer Society for his contribution to softwareengineering research.

integrating a market-based model in trust-based service ... · challenges needs to be rigorously...

Documents