crowdsourcing and markets in the social media age

11 © Copyright 2009 Hewlett-Packard Development Company, L.P.

Thomas Sandholm—Research Scientist, HP Labs, Palo Alto

December 17, 2009

Crowdsourcing and Markets in the Social Media Age


Information Abundance and Attention Scarcity

– Information overflow on mobile devices•Small visual real-estate•Limited bandwidth•Limited input capabilities

– How can crowdsourcing, markets, and context improve the mobile Web experience?


Agenda

–Crowdsourcing

–Markets

–Social Media

–Examples

–SCL Projects

–HP Gloe


CROWDSOURCING


Wisdom of Crowds

– Surowiecki: The Wisdom of Crowds (2004)•Skepticism about dumb mobs•Crowds can be smarter than smartest individual if:

1.Diverse (otherwise no new ideas)

2.Independent (otherwise groupthink)

3.Decentralized (otherwise specialization is hard)

4.Aggregation (otherwise local knowledge is lost)


Crowdsourcing or Peer Production:Specialization and reputation drive long tail content production

– Business model•Open call•Parallel problem solving•Vetting of solutions

– Phenomenon•Historically: few publishers and millions of readers•Now: millions of publishers with few readers

– Examples•Wikipedia, YouTube, Linux, Digg


Dynamics of Peer ProductionWhy do people contribute to social media?

– Lack of attention reason to stop contributing

– Feedback loop of attention leads to power laws

– Negligible part of content produced by majority of users

– Negligible number of users produce majority of content

– Hubs of well-connected users vital to dissemination

– Persistence paradoxically inversely correlated to success


MARKETS


Market PropertiesSurowiecki’s fourth condition

– Aggregation of decentralized local knowledge

– Determine value of goods•Price varies until supply and demand are in equillibrium•Self-adjusting and decentralized

– Walras’ Law in General Equilibrium Theory•Sum of excess demand across all markets equals zero

– Market Mechanism:•Incentive compatible, individually rational, strategy proof•Incentive to participate truthfully, non-gameable•Selfish optimization leads to welfare •Effective high-quality, truthful information signals


Prediction or Information MarketsFocus on information aggregation and prediction

– Voting with costs and payoffs

– 1988 Forsythe, Nelson, Neumann: Iowa Electronic Markets•Predict outcome of political races•Polling less accurate due to phrasing of question•Real money forces truthful behavior

– 2003 Hanson: DoD PAM ―Terrorism Futures‖•Public outcry – unethical to bet on terrorism attacks•Publicity – Crowdcast, Inkling, HubDub, Bet2Give•Same games, some real for-profit markets

– Yahoo Tech Buzz, Google internal information flow markets•Cube neighbors influence betting behavior


SOCIAL MEDIA


Web 2.0 Social Media AppsDrove content production paradigm shift

– Long tail content •Unpopular content gets published

– Grass root reputation•meritocracy•open APIs for mashups

– Collaborative filtering•Amazon book recommendations

– The network effect •More users more quality opposite to Web 1.0

– Social networks are leveraged to disseminate information •Viral marketing•Friends determine what you see

– Flickr, MySpace, YouTube, Facebook, Twitter•Not technology leaders but grabbed critical social mass•Innovators in how people interact


Context-aware applicationsLocation, Social, Time, Weather Conditions

–Mobile devices more personal•Already used to interact with friends

–Sensors give more information to applications•No explicit user involvement

– iPhone paved the way for the mobile Web•Full-featured Web browsers on smartphones

–Affordable unlimited 3G data plans–Privacy opt-in

•Intrusive apps more accepted if they have a social value


Location Based ServicesPull or push content based on sensed location

– Geography-aware service on mobile device

– GIS for mobile phones

– GPS + Internet on smartphones

– Find what’s around you

– Map based, AR based or Distance/Popularity Listings

– Google Navigation/Maps, Wikitude.Me, Layar, Sherpa, Where, GeoVector, Loopt


EXAMPLES


Digg – a crowdsourced newspaper

– Crowd-rendered front page

– Novelty and popularity tradeoff

– Update frequency unmatched


Delicious – social bookmarking

– Share bookmarks with friends

– Find popular bookmarks

– Search by tags and tag clouds


Google AdWords – bidding for page prominence

– Market mechanisms used to leverage attention economy

– Key word auctions

– Non-intrusive, relevant presentation

– Virtually all of Google’s revenue


SOCIAL COMPUTING LAB PROJECTS


BRAIN

– Prediction of events in small groups

– Forecasting in teams biased towards ―loudest‖ member

– When members put a bet on their forecast they predict more carefully and accurately

– Risk attitudes of members matter, when normalized away the weighted prediction becomes more accurate


Tycoon

– Market based computational resource allocation

– Resources capacity allocated proportional to individual bids and inversely proportional to aggregatebids

– q = b/(b+y)

– Variable pricing, self adjusting to demand

– Similar properties to second bid auction but converges quicker and has no discrete clearing times


HP Gloe


Idea

– Popularity search by radius, aggregation by URL

– Social filtering

– Folksonomy of hierarchical channels

– Cyberspace notes

– Votes governed by market economy

Vote on web page to location mappings


Popularity Ranking and Social Filtering


Folksonomy of hierarchical channels

Pizza

Restaurants

Burgers

Restaurants

Pizza

Restaurants

Diners

Restaurants


Cyberspace notes


Votes governed by market economy

– Limited recommendation budget•American Idol type voting

– Contributions controlled by currency•Earn, pay, penalties, income streams

– Optimal budget allocation •Across regions in marketing campaigns•Across hierarchical key words

– Voting minimal effort but truth-telling enforced•Restaurant services forcing you to leave comments to vote

– Maximize information obtained from crowd•Value distribution beyond 5 star ratings


Demo

– Launched on Android market Dec 1st

– ~6 million recommendations, ~8000 channels, ~150 users

– Data from Wikipedia, TripAdvisor, Panoramio, WikiTravel, POI Plaza, IMDb, DBPedia, Geonames, Google, Twitter, Bing


Research Problems

– Design a market that gives incentive to users to contribute valuable mappings

– Visualization, control and optimization techniques to

trade off popularity (clicks and recs), novelty, and

distance

– Dynamic content provisioning, recommendation decay

– Optimized channel and location-based bidding strategies

– Data analytics, e.g. behavioral patterns in Web page recommendations


Summary

–Crowdsourcing can be a very effective technique to offer long tail content if certain conditions are met

–Markets can be used to control the quality of contributions and to aggregate local information concisely

–Your social network provides a way to effectively filter and discover new information


Q & A

crowdsourcing and markets in the social media age

Technology