© 2006 the university of texas at austin 1 art testbed join the discussion group at:
TRANSCRIPT
1© 2006 THE UNIVERSITY OF TEXAS AT AUSTIN
ART Testbed
Join the discussion group at:
http://www.art-testbed.net
2© 2006 THE UNIVERSITY OF TEXAS AT AUSTIN
ART Testbed Questions Can agents request reputations about themselves? Can an agent produce an appraisal without purchasing opinions? Does the Testbed assume a common representation for
reputations? Does the Testbed prevent agents from winning via action-planning
skills, as opposed to trust-modeling skills? What if an agent can’t or won’t give a reputation value? Why does it cost more to generate an accurate opinion than an
inaccurate one? Why not have a centralized reputation broker? Isn’t it unrealistic to assume a true value of a painting can be
known? Is art appraisal a realistic domain? Why not design an incentive-compatible mechanism to enforce
truth-telling?
3© 2006 THE UNIVERSITY OF TEXAS AT AUSTIN
“Really Good” ART Testbed Questions Is there a consensus on the definitions of “trustworthiness” and
“reputation”? How can collusion be avoided? Is truth-telling a dominant strategy? Will the system reach equilibrium, at which point reputations are no
longer useful? What happens if client fee (100), opinion cost (10), and reputation
cost (1) are changed? Do any equilibria exist? What happens when agents enter or leave the system? When will agents seek out reputations? Space of experiments is underexplored—that’s a good thing!
4© 2006 THE UNIVERSITY OF TEXAS AT AUSTIN
Questions about the Paper What is a “trust model”? How does q-learning work? How related to reinforcement learning?
How do rewards tie in? What is lambda? How can experience- and reputation-based learning be combined to
overcome the weaknesses of each (intermediate lambda values)? What about different combinations of (more sophisticated) agents in
a game? Why the assumptions chosen? They seem too extreme. Reputation decisions weren’t examined very well.
The Agent Reputation and Trust Testbed: The Agent Reputation and Trust Testbed: Experimentation and Competition Experimentation and Competition
for Trust in Agent Societiesfor Trust in Agent Societies
Karen K. Fullam1, Tomas B. Klos2, Guillaume Muller3, Jordi Sabater4, Andreas Schlosser5, Zvi Topol6, K. Suzanne Barber1, Jeffrey S. Rosenschein6, Laurent Vercouter3, and Marco Voss5
1Laboratory for Intelligent Processes and Systems, University of Texas at Austin, USA2Center for Mathematics and Computer Science (CWI), Amsterdam, The Netherlands
3 Ecole Nationale Superieure des Mines, Saint-Etienne, France4Institute of Cognitive Science and Technology (ISTC), National Research Council (CNR), Rome, Italy
5IT Transfer Office, Darmstadt University of Technology, Darmstadt, Germany6Multiagent Systems Research Group—Critical MAS, Hebrew University, Jerusalem, Israel
The Agent Reputation and Trust Testbed, 2006
Appraiser Agent
Appraiser Agent
Client
Client
Client
Client Share
Opinions and Reputations
Appraiser Agent
Appraiser Agent
Appraiser Agent
Testbed Game RulesTestbed Game Rules
Agents function as art appraisers with varying expertise in
different artistic eras.
For a fixed price, clients ask appraisers to provide
appraisals of paintings from various eras.
If an appraiser is not very knowledgeable
about a painting, it can purchase "opinions"
from other appraisers.
Appraisers can also buy and sell reputation information about other
appraisers.
Appraisers whose appraisals are more
accurate receive larger shares of the client base
in the future. Appraisers compete to achieve the highest earnings by the end of the game.
The Agent Reputation and Trust Testbed, 2006
Step 1: Client and Expertise AssignmentsStep 1: Client and Expertise Assignments
Appraisers receive clients who pay a fixed price to request appraisals
Client paintings are randomly distributed across eras
As game progresses, more accurate appraisers receive more clients (thus more profit)
The Agent Reputation and Trust Testbed, 2006
Step 2: Reputation TransactionsStep 2: Reputation Transactions
Appraisers know their own level of expertise for each era
Appraisers are not informed (by the simulation) of the expertise levels of other appraisers
Appraisers may purchase reputations, for a fixed fee, from other appraisers
Reputations are values between zero and one • Might not correspond to
appraiser’s internal trust model
• Serves as standardized format for inter-agent communication
The Agent Reputation and Trust Testbed, 2006
Step 2: Reputation TransactionsStep 2: Reputation Transactions
ProviderRequester
Request
Accept
Payment
Reputation
Requester sends request message to a potential reputation provider, identifying
appraiser whose reputation is
requested
Potential reputation provider sends
“accept” message
Requester sends fixed payment to the
provider
Provider sends reputation
information, which may not be truthful
The Agent Reputation and Trust Testbed, 2006
Step 3: Opinion TransactionsStep 3: Opinion Transactions
For a single painting, an appraiser may request opinions (each at a fixed price) from as many other appraisers as desired
The simulation “generates” opinions about paintings for opinion-providing appraisers
Accuracy of opinion is proportional to opinion provider’s expertise for the era and cost it is willing to pay to generate opinion
Appraisers are not required to truthfully reveal opinions to requesting appraisers
The Agent Reputation and Trust Testbed, 2006
Step 3: Opinion TransactionsStep 3: Opinion Transactions
ProviderRequester
Request
Certainty
Payment
Opinion
Requester sends request message to a
potential opinion provider, identifying
painting
Potential provider sends a certainty
assessment about the opinion it can provide- Real number (0 – 1)
- Not required to truthfully report certainty
assessment
Requester sends fixed payment to the
providerProvider sends
opinion, which may not be truthful
The Agent Reputation and Trust Testbed, 2006
Step 4: Appraisal CalculationStep 4: Appraisal Calculation
Upon paying providers and before receiving opinions, requesting appraiser submits to simulation a weight (self-assessed reputation) for each other appraiser
Simulation collects opinions sent to appraiser (appraisers may not alter weights or received opinions)
Simulation calculates “final appraisal” as weighted average of received opinions
True value of painting and calculated final appraisal are revealed to appraiser
Appraiser may use revealed information to revise trust models of other appraisers
The Laboratory for Intelligent Processes and SystemsElectrical and Computer Engineering
The University of Texas at Austinhttp://www.lips.utexas.edu
Karen K. Fullam
2006 ART TestbedCompetition Results
14© 2006 THE UNIVERSITY OF TEXAS AT AUSTIN
Competition Organization
“Practice” Competition• Spanish Agent School, Madrid, April 2006 • 12 participants
International Competition • AAMAS, Hakodate, May 2006 • Preliminary Round
13 Participants 5 games each
• Final Round 5 Finalists 10 games with all finalists participating
15© 2006 THE UNIVERSITY OF TEXAS AT AUSTIN
Bank BalancesIam achieves highest bank
balances
16© 2006 THE UNIVERSITY OF TEXAS AT AUSTIN
Opinion Purchases
Joey and Neil do not purchases opinions
Sabatini purchases the most opinions
17© 2006 THE UNIVERSITY OF TEXAS AT AUSTIN
Opinion Earnings
Sabatini and Iam provide the most opinions
Neil and Frost do not
provide many
opinions
18© 2006 THE UNIVERSITY OF TEXAS AT AUSTIN
Opinion Sensing Costs
Iam invests the
most in opinions it generates
19© 2006 THE UNIVERSITY OF TEXAS AT AUSTIN
Expertise vs. Bank Balance
Iam’s average expertise was
not significantly higher than
others’
Greater Expertise
The Laboratory for Intelligent Processes and SystemsElectrical and Computer Engineering
The University of Texas at Austinhttp://www.lips.utexas.edu
Karen K. Fullam
K. Suzanne Barber
Learning Trust Strategies in Reputation Exchange Networks
21© 2006 THE UNIVERSITY OF TEXAS AT AUSTIN
Trust Decisions in Reputation Exchange Networks Agents perform transactions to obtain needed resources
• Transactions have risk because partners may be untrustworthy• Agents must learn whom to trust and how trustworthy to be
When agents can exchange reputations• Agents must also learn when to request reputations and what
reputations to tell• Agents’ trust decisions affect each other
Difficult to learn each decision independently
Resources (goods, services,
information)
How trustworthy should I be?
Reputations
Which reputations should I listen to?
What reputations should I tell?
Should I trust?
If I lie to others that C is bad, can I monopolize C’s
interactions?
If I cheat A, and A tells B, will it
hurt my interactions with B?
22© 2006 THE UNIVERSITY OF TEXAS AT AUSTIN
Enumerating Decisions in a Trust StrategyT
rust
erT
rust
ee
Ag
ent
Ro
le
Transaction
Fundamental Reputation
How trustworthy should I be?
Should I trust?
Should I tell an accurate
reputation?
Should I believe this reputation?
Truster Trustee
combinations aen2
combinations a en
combinations aen2
combinations a en
Num agents = aNum transaction types = eNum choices/decision = n
How to learn the best
strategy with so many choices?
If these decisions affect
each other, there are possible
strategies!
2 1ae an
23© 2006 THE UNIVERSITY OF TEXAS AT AUSTIN
Reinforcement Learning
Select a strategy
Strategy feedback influences expected reward
Strategies with higher expected
rewards are more likely to be selected
Strategy Expected RewardABCD
24© 2006 THE UNIVERSITY OF TEXAS AT AUSTIN
Learning In Reputation Exchange Networks
Strategy Expected RewardTr(A),Tr(B),Tr(C)…
. . .
⌐Tr(A),Tr(B),Tr(C)…
Tr(A),⌐Tr(B),Tr(C)…
⌐Tr(A),⌐Tr(B),Tr(C)…
Tr(A),Tr(B),⌐Tr(C)…
⌐Tr(A),Tr(B),⌐Tr(C)…
⌐Tr(A),⌐Tr(B),⌐Tr(C)…
Tr(A),⌐Tr(B),⌐Tr(C)…
Decision Expected RewardTr(A)
⌐Tr(A)
Decision Expected RewardTr(B)
⌐Tr(B)
Decision Expected RewardTr(C)
⌐Tr(C)Removing
interdepend-encies makes each decision in the strategy
learnable
Use the ART Testbed as a case study
Because decisions are interdependent,
there are . possible
strategies!
2 1ae an
25© 2006 THE UNIVERSITY OF TEXAS AT AUSTIN
Many Interdependent Decisions
Accuracy of Opinion
Requester’s appraisals
Reputation Requester’s
reputation costs
Opinion Requester’s
client revenue
Other Appraisers’
client revenue
Number of requests received
by Opinion Provider
Opinion Provider’s
opinion revenue
Opinion Provider’s
opinion order costs
Opinion Requester’s
opinion costs
Accuracy of Reputation
Requester’s trust models
Number of requests received
by Reputation Provider
Reputation Provider’s reputation revenue
Opinion Provider
Opinion Requester
Reputation Provider
Reputation Requester
When Reputation Requester is
Opinion Requester
26© 2006 THE UNIVERSITY OF TEXAS AT AUSTIN
Opinion Requester Feedback
Opinion Requester’s
client revenue
Opinion Requester’s
opinion costs
Opinion Requester
Assume: Client revenue feedback is wholly attributed to Opinion Requester
decision
Divide revenue (client revenue) among opinions
based on opinion accuracy
Opinion Requester’s
client revenue
Opinion Requester’s
opinion costs
Reward = –
Client Revenue
Opinion Purchase
Costs
27© 2006 THE UNIVERSITY OF TEXAS AT AUSTIN
Opinion Provider Feedback
Other Appraisers’
client revenue
Opinion Provider’s
opinion revenue
Opinion Provider’s
opinion order costs
Opinion Provider
Assume: Client revenue is not related to Opinion Provider
decision
Reward = –
Opinion Selling
Revenue
Opinion Generating
Costs
Opinion Provider’s
opinion revenue
Opinion Provider’s
opinion order costs
28© 2006 THE UNIVERSITY OF TEXAS AT AUSTIN
Reputation Provider Feedback
Other Appraisers’
client revenue
Reputation Provider’s reputation revenue
Assume: Client revenue is not
related to Reputation Provider decision
Reward =
Reputation Selling
Revenue
Reputation Provider’s reputation revenue
Reputation Provider
29© 2006 THE UNIVERSITY OF TEXAS AT AUSTIN
Reputation Requester
Reputation Requester Feedback
Reputation Requester’s
reputation costs
Opinion Requester’s
client revenue
Opinion Requester’s
opinion costs
determines influence of: past experience vs. reputations in
deciding to purchase opinions
= 0: Past experience only Opinion-requesting decision No reward for requesting reputations
= 1: Reputations only Reputation-requesting decision Full reward for requesting reputations
Opinion Requester
Reward = –
Opinion Requester
Reward
Reputation Purchase
Costs( )