reputation systems talk at ebay
DESCRIPTION
A variation of my Trust Symposium Keynote given at eBay, where I do poke a bit at the venerable "Feedback" system.TRANSCRIPT
Social (Reputation) System Design:Opportunities and Challenges
A Practitioners’ View
F Randall FarmerRosenfeld Media
(Coauthor: Bryce Glass)
@eBay, November 14th, 2013
Setting Context:Who am I to speak to eBay?
Definitions
Reputation is information used to make a value judgment about an object or a person – in a specific context.
Reputation about a person (physical or legal) has special features and pitfalls, and rates a special term: Karma
Why Web Reputation Systems?
Two words:
Information Overload
Searching, organizing, filtering, advertizing—all built on top of web reputation systems—is
driving an economy worth $ trillions.
Challenges and Fallacies
But, if web reputation systems are so important, why do so many of them suck?
– “Any content reputation system is better than none.”– “Content quality is a rich-man’s problem.”– “The other guys have thumbs-up/down, we should too.”– “What cool tools do we have on the shelf?”
Time-to-market trumps correctness, testing, and even research. Reputation is not, primarily, a technical problem.
Contexts for Reputation
Individual Context
Community Context
Corporate Context
Industrial Context
Challenges:Reputation is Individual
What does this mean?
What does it actually do?
Like
Personal Meaning & Utility of Ratings
Karma is Hard• Karma is useful for building trust between users, and between a user and the site• Karma can be an incentive for participation and contributions• Karma is contextual and has limited utility globally.• Karma comes in several flavors - Participation, Quality and Robust• Karma is complex (via indirect evaluations), and formulation is often opaque• Personal karma is displayed only to the owner, good for measuring progress• Corporate karma is used by the site to find the very best and very worst users• Public karma is displayed to other users, which makes it the hardest to get right
– It should be used sparingly– It is hard to understand, isn't expected, and easily confused with content
ratings.– It shouldn't be have a socially negative value.– It encourages competition in some users, and may discourage others
The FICO Fallacy: Universal Karma
Credit Score Factors
On-time PaymentsCapacity UsedLength of Credit HistoryTypes of Credit UsedPast Credit Applica-tions
Public Karma is Identity (and Personal)
Scott McCloud, Understanding Comics, pp. 38.
“We built a point system into Consumating because we thought giving direct feedback to people about their conduct on the site would encourage them to be nice to one another—you get a thumbs up when you are nice (treat!), and a thumbs down when you are a douche (electric shock!)”
Ben Brown, Internet Rockstar
Words are Weapons: Public Karma
However…
Dead
Challenges:Reputation is a Community
Communities Have Temperaments
Caring
Collaborative
Cordial
Competitive
Combative
ALTRUISTIC
COMMERCIAL
EGOCENTRIC
The Virtuous Circle
CONTRIBUTORS
THECOMMUNITY
THE SITE
CREATE CONTENT FOR
RANKSIT FOR
REWARDS AND INCENTIVIZES
GRINDING, POINT WHORING
POINTS!
ACTIVITY OVER QUALITY
The Vicious Spiral
POORINCENTIVES
WRONGFOCUS
ILL-BEHAVIOR
FACTIONS DEPARTURES
BAD NORMS
BEST USERS LEAVE
“HOW IS BABBY FORMED!?”
Tragedy of the Commons
Challenges:Reputation is Corporate
The Corporate Virtuous Circle (eBay)
SELLERS
BUYERS
EBAY(THE CORPORATION)
CREATE LISTINGS,SELL ITEMS
BUY,EVALUATESELLERS
REWARDS AND INCENTIVIZES
Yelp: Multi-sided Market…
Yelp: Multi-sided Market with Many Contexts
eBay as Multi-Sided Market:Drop-shippers and Feedback
When is this true? 100 – 5 < 0
What is a Negative Feedback worth?(in U.S. Dollars)
Google’s Virtuous Circle
PUBLISHERS
CONSUMERS(SEARCHERS &
LINKERS)
SEARCH RESULTS(VIA PAGERANK)
CREATE WEB PAGES FOR
PROVIDEINPUTSTO RANK
PROVIDES TRAFFIC BACK TO
ADVERTISING• SALES• ADS• ATTENTION• WHATEVER…
Challenges:Reputation is an Industry
Challenge: Systems that model real life
Honestly.com? Not so much.
Dead
Fallacy: Karma as influence & currency
Dead
Our Ultimate Challenge:Short Institutional Memory
From Fuzzy to Focused
A++++++ audience, would visit again…
Randy Farmer@frandallfarmer
[email protected] http://about.me/frandallfarmer