robin hanson pm summit nyc 2009
TRANSCRIPT
![Page 1: Robin Hanson PM Summit NYC 2009](https://reader035.vdocuments.us/reader035/viewer/2022070315/5552491ab4c9059a4f8b4a81/html5/thumbnails/1.jpg)
Combinatorial Prediction Markets
Robin HansonEconomics, George Mason UniversityChief Scientist, Consensus Point
![Page 2: Robin Hanson PM Summit NYC 2009](https://reader035.vdocuments.us/reader035/viewer/2022070315/5552491ab4c9059a4f8b4a81/html5/thumbnails/2.jpg)
SpeculativeSpeculative
FutureFuture Currency
Currency
StockStockBetBet
Insurance
Insurance
HedgingHedging Gambling
Gambling
Decision
Decision
BondBond
SpotSpot
Market
ContextContext DirectDirect
SearchSearch MatchingMatching
“Prediction Markets”
![Page 3: Robin Hanson PM Summit NYC 2009](https://reader035.vdocuments.us/reader035/viewer/2022070315/5552491ab4c9059a4f8b4a81/html5/thumbnails/3.jpg)
To Evaluate Institutions
Institution A
When They UseSimilar Inputs
Compare QualityOf Outputs
Institution B
![Page 4: Robin Hanson PM Summit NYC 2009](https://reader035.vdocuments.us/reader035/viewer/2022070315/5552491ab4c9059a4f8b4a81/html5/thumbnails/4.jpg)
Collective Forecasting
ForecastsOn Requested
TopicsUser Contributions User Scores
Engagement
![Page 5: Robin Hanson PM Summit NYC 2009](https://reader035.vdocuments.us/reader035/viewer/2022070315/5552491ab4c9059a4f8b4a81/html5/thumbnails/5.jpg)
IssuesInput: Contributions
What info can express?How account for costs?Who let in where?Enough Incentives
T-shirts enough?Zero-sum scoring?
Limit CostsAwkward InterfaceWait for offer acceptRetribution
Output: Forecasts, Scores
What questions can ask?How account for value?Use or validate system?Should adjust outputs?Who let see outputs?Sabotage & manipulationLegal, P.R. risks?
![Page 6: Robin Hanson PM Summit NYC 2009](https://reader035.vdocuments.us/reader035/viewer/2022070315/5552491ab4c9059a4f8b4a81/html5/thumbnails/6.jpg)
Collective Forecasting Questions
ConsensusConsensus
My ForecastMy Forecast My ScoreMy Score
TruthTruth
What exactly is my influence?
What exactly are my
incentives?
How exactly do I express my
opinion?
![Page 7: Robin Hanson PM Summit NYC 2009](https://reader035.vdocuments.us/reader035/viewer/2022070315/5552491ab4c9059a4f8b4a81/html5/thumbnails/7.jpg)
Editing Interface Is Transparent
ConsensusConsensus
My EditsMy Edits My ScoreMy Score
TruthTruth
I directly change theconsensus
If my edit increasesthe consensus chance
of true state, I win. If decreases, I lose.
![Page 8: Robin Hanson PM Summit NYC 2009](https://reader035.vdocuments.us/reader035/viewer/2022070315/5552491ab4c9059a4f8b4a81/html5/thumbnails/8.jpg)
IssuesInput: Contributions
What info can express?How account for costs?Who let in where?Enough Incentives
T-shirts enough?Zero-sum scoring?
Limit CostsAwkward InterfaceWait for offer acceptRetribution
Output: Forecasts, Scores
What questions can ask?How account for value?Use or validate system?Should adjust outputs?Who let see outputs?Sabotage & manipulationLegal, P.R. risks?
![Page 9: Robin Hanson PM Summit NYC 2009](https://reader035.vdocuments.us/reader035/viewer/2022070315/5552491ab4c9059a4f8b4a81/html5/thumbnails/9.jpg)
Factors Might Influence Sales
Economy recovers fast?Competitors introduce new version?We do big promotion?We lower prices? They lower prices?We add distribution channel?We add feature F? They add feature F?Our defect rate very low?
E[Sales|Factor]P[Factor]
![Page 10: Robin Hanson PM Summit NYC 2009](https://reader035.vdocuments.us/reader035/viewer/2022070315/5552491ab4c9059a4f8b4a81/html5/thumbnails/10.jpg)
Combo Betting
Yoopick Facebook Application
WinPlace
Show
All outcomes
Win Place Show
Not Not Not
![Page 11: Robin Hanson PM Summit NYC 2009](https://reader035.vdocuments.us/reader035/viewer/2022070315/5552491ab4c9059a4f8b4a81/html5/thumbnails/11.jpg)
Sport Finals TicketsUEFA EURO 2008
Austria
Croatia
Czech
Germany
Poland
Portugal
Switzerl.
Turkey
France
Greece
Italy
Netherl.
Romania
Russia
Spain ActualGame
Sweden
Ticket if Greece in FinalsGreece v.Croatia
![Page 12: Robin Hanson PM Summit NYC 2009](https://reader035.vdocuments.us/reader035/viewer/2022070315/5552491ab4c9059a4f8b4a81/html5/thumbnails/12.jpg)
US President Decision Markets
Politimetrics.com
![Page 13: Robin Hanson PM Summit NYC 2009](https://reader035.vdocuments.us/reader035/viewer/2022070315/5552491ab4c9059a4f8b4a81/html5/thumbnails/13.jpg)
PAM Scenario
Saudi Arabian
100
75
125
103
203
303
403
104
204
65
35
70
30
30
35
25
10
15
40
35
10
10094
Economic Health
New Price <IQcs403 85
Return to Focus ?TradeSAum3 03105-125
No Trade
Exit Issue
Max Up
Max Dn
10% Dn
10% Up
You Pick
62.47%
Bu
y S
ell
>IQcs403 85
Payoffs:
68.72%
56.79%
48.54%
95.13%
22.98%
+$2.74
-$15.34
+$34.74
$0.00 $0.00
-$2.61
-2.04
-$3.28
-$85.18
-$120.74
+$2.74
65
+$26.02
+1.43
+$96.61
& If
65%
Ave. pay<85 Select
$0.00
+0.34
-$1.07
-$19.72
-$1.12
-$6.31
-$22.22
%
Update
Execute a Trade ?Return to Form
ExecuteAbort trade if price has changed?
If US military involvement in Saudi Arabia in 3rd Quarter 2003 is not between 105 and 125, this trade is null and void. Otherwise, if Iraq civil stability in 4th Quarter 2003 is below 85, then I will receive $1.43, but if it is not below 85, I will pay $2.04.
![Page 14: Robin Hanson PM Summit NYC 2009](https://reader035.vdocuments.us/reader035/viewer/2022070315/5552491ab4c9059a4f8b4a81/html5/thumbnails/14.jpg)
Imagine A DashboardUs Them A Them B
Base Price $240 $187 $320
Ship Date May ’09 Mar ’09 July ’09
Features Autozoop 38% 69% 15%
Fizzywoo 59% 8% 43%
Unit Sales Total 120K 148K 97K
Base model 82K 65K 88K
Via internet 43K 12K 73K
Promotion Magazine $30K $50K $3K
Circulars $45 $34K $39K
Ave. Worth:$12,459
![Page 15: Robin Hanson PM Summit NYC 2009](https://reader035.vdocuments.us/reader035/viewer/2022070315/5552491ab4c9059a4f8b4a81/html5/thumbnails/15.jpg)
Ask For DetailUs Them A Them B
Base Price $240 $187 $320
Ship Date May ’09 Mar ’09 July ’09
Features Autozoop 38% 69% 15%
Fizzywoo 59% 8% 43%
Unit Sales Total 120K 148K 97K
Base model 82K 65K 88K
Via internet 43K 12K 73K
Promotion Magazine $30K $50K $3K
Circulars $45 $34K $39K
J F M A M J J A S O N D
2009 2010
Them B Ship Date
Ave.Worth:$12,459
![Page 16: Robin Hanson PM Summit NYC 2009](https://reader035.vdocuments.us/reader035/viewer/2022070315/5552491ab4c9059a4f8b4a81/html5/thumbnails/16.jpg)
Make An EditUs Them A Them B
Base Price $240 $187 $320
Ship Date May ’09 Mar ’09 July ’09
Features Autozoop 40% 69% 15%
Fizzywoo 59% 8% 43%
Unit Sales Total 120K 148K 97K
Base model 82K 65K 88K
Via internet 43K 12K 73K
Promotion Magazine $30K $50K $3K
Circulars $45 $34K $39K
Ave. Worth:$12,459
If We Have Autozoop, you gain $53
But if We Don’t Have ItYou lose $78. OK?
![Page 17: Robin Hanson PM Summit NYC 2009](https://reader035.vdocuments.us/reader035/viewer/2022070315/5552491ab4c9059a4f8b4a81/html5/thumbnails/17.jpg)
Make an AssumptionUs Them A Them B
Base Price $240 $187 $253
Ship Date Apr ’09 Mar ’09 Assume Mar
Features Autozoop 38% 69% 4%
Fizzywoo 59% 8% 13%
Unit Sales Total 120K 148K 107K
Base model 82K 65K 94K
Via internet 43K 12K 84K
Promotion Magazine $30K $50K $17K
Circulars $45 $34K $49K
Ave. Worth:$10,724
Scenario:15%
![Page 18: Robin Hanson PM Summit NYC 2009](https://reader035.vdocuments.us/reader035/viewer/2022070315/5552491ab4c9059a4f8b4a81/html5/thumbnails/18.jpg)
Add 2nd AssumptionUs Them A Them B
Base Price $240 $187 $253
Ship Date Apr ’09 Mar ’09 Assume Mar
Features Autozoop 38% 69% 4%
Fizzywoo 59% 8% 13%
Unit Sales Total 185K 148K 107K
Base model 97K 65K 94K
Via internet 78K 12K 84K
Promotion Magazine Assume $40K
$50K $17K
Circulars $45 $34K $49K
Ave. Worth:$10,982
Scenario:2.3%
![Page 19: Robin Hanson PM Summit NYC 2009](https://reader035.vdocuments.us/reader035/viewer/2022070315/5552491ab4c9059a4f8b4a81/html5/thumbnails/19.jpg)
Edit As BeforeUs Them A Them B
Base Price $240 $187 $253
Ship Date Apr ’09 Mar ’09 Assume Mar
Features Autozoop 42% 69% 4%
Fizzywoo 59% 8% 13%
Unit Sales Total 185K 148K 107K
Base model 97K 65K 94K
Via internet 78K 12K 84K
Promotion Magazine Assume $40K
$50K $17K
Circulars $45 $34K $49K
Ave. Worth:$10,724
Scenario:2.3%
If we have Autozoop, you gain $53
But if we don’t have itYou lose $78. OK?
![Page 20: Robin Hanson PM Summit NYC 2009](https://reader035.vdocuments.us/reader035/viewer/2022070315/5552491ab4c9059a4f8b4a81/html5/thumbnails/20.jpg)
Combo Market Maker Best of 5 Mechs
3 Variables = 8 States
0.000
0.050
0.100
0.150
0.200
0.250
0.300
KL
Dis
tanc
e
8 Variables = 256 States
0.600
0.800
1.000
1.200
1.400
1.600
KL
Dis
tanc
e
3 subjects, 7 prices, 5 minutes 6 subjects, 256 prices, 5 minutes
![Page 21: Robin Hanson PM Summit NYC 2009](https://reader035.vdocuments.us/reader035/viewer/2022070315/5552491ab4c9059a4f8b4a81/html5/thumbnails/21.jpg)
-1
-0.5
0
0.5
1
0 5 10 15
MSR Info vs. Time – 7 Prices
0 5 10 15
Minutes
0
-1
1
% Info Agg. =
KL(prices,group)KL(uniform,group)
1-
![Page 22: Robin Hanson PM Summit NYC 2009](https://reader035.vdocuments.us/reader035/viewer/2022070315/5552491ab4c9059a4f8b4a81/html5/thumbnails/22.jpg)
MSR Info vs. Time – 255 prices
-1
-0.5
0
0.5
1
0 5 10 15
0 5 10 15
Minutes
0
-1
1
% Info Agg. =
KL(prices,group)KL(uniform,group)
1-
![Page 23: Robin Hanson PM Summit NYC 2009](https://reader035.vdocuments.us/reader035/viewer/2022070315/5552491ab4c9059a4f8b4a81/html5/thumbnails/23.jpg)
IssuesInput: Contributions
What info can express?How account for costs?Who let in where?Enough Incentives
T-shirts enough?Zero-sum scoring?
Limit CostsAwkward InterfaceWait for offer acceptRetribution
Output: Forecasts, Scores
What questions can ask?How account for value?Use or validate system?Should adjust outputs?Sabotage & manipulationCan keep results secret?Legal, P.R. risks?
![Page 24: Robin Hanson PM Summit NYC 2009](https://reader035.vdocuments.us/reader035/viewer/2022070315/5552491ab4c9059a4f8b4a81/html5/thumbnails/24.jpg)
States
Pri
ces
Use
r A
sset
s
$1 if A&B
$1 if B
+ q1
- q2
f1>1 f2<1
A B
+
-
A Simple Implementation
LISP: http://hanson.gmu.edu/mktscore-prototype.html
![Page 25: Robin Hanson PM Summit NYC 2009](https://reader035.vdocuments.us/reader035/viewer/2022070315/5552491ab4c9059a4f8b4a81/html5/thumbnails/25.jpg)
A Simple Implementation
States
A&B A&B
Pri
ces
Use
r A
sset
s
![Page 26: Robin Hanson PM Summit NYC 2009](https://reader035.vdocuments.us/reader035/viewer/2022070315/5552491ab4c9059a4f8b4a81/html5/thumbnails/26.jpg)
Environments: Goals, TrainingWant in Environment:
Many variables, few directly related Few people, each not see all variables Can compute rational group estimatesExplainable, fast, neutral
Training Environment: 3 binary variables X,Y,Z, 23 = 8 combosP(X=0) = .3, P(X=Y) = .2, P(Z=1)= .5 3 people, see 10 cases of: AB, BC, AC Random map XYZ to ABC
Case A B C
1 1 -
1 2 1 -
0
3 1 -
0 4 1 -
0
5 1 -
0 6 1 -
1
7 1 -
1 8 1 -
0
9 1 -
0 10 0 -
0
Sum: 9 -
3
Same A B C
A -- --
4 B -- --
-- C -- --
--
(Actually: X Z Y )
![Page 27: Robin Hanson PM Summit NYC 2009](https://reader035.vdocuments.us/reader035/viewer/2022070315/5552491ab4c9059a4f8b4a81/html5/thumbnails/27.jpg)
Experiment Environment8 binary vars: STUVWXYZ28 = 256 combinations20% = P(S=0) = P(S=T) = P(T=U) = P(U=V) = … = P(X=Y) = P(Y=Z)6 people, each see 10 cases: ABCD, EFGH, ABEF, CDGH, ACEG, BDFHrandom map STUVWXYZ to ABCDEFGH
Case A B C D E F G H 1 0 1 0 1 - - - - 2 1 0 0 1 - - - - 3 0 0 1 1 - - - - 4 1 0 1 1 - - - - 5 0 1 1 1 - - - - 6 1 0 0 1 - - - - 7 0 1 1 1 - - - - 8 1 0 0 1 - - - - 9 1 0 0 1 - - - - 10 1 0 0 1 - - - -
Sum 6 3 4 10 - - - -
Same A B C D E F G H A -- 1 2 6 -- -- -- -- B -- -- 7 3 -- -- -- -- C -- -- -- 4 -- -- -- -- D -- -- -- -- -- -- -- -- …
(Really: W V X S U Z Y T )
![Page 28: Robin Hanson PM Summit NYC 2009](https://reader035.vdocuments.us/reader035/viewer/2022070315/5552491ab4c9059a4f8b4a81/html5/thumbnails/28.jpg)
Ad Agency Decision Markets
$ Revenue if not Switch
E(R | not S)Compare!
$1 if not Switch
$ Revenue if Switch
E(R | S)
$1
P(S)
$1 if Switch
$ RevenueE(R)
![Page 29: Robin Hanson PM Summit NYC 2009](https://reader035.vdocuments.us/reader035/viewer/2022070315/5552491ab4c9059a4f8b4a81/html5/thumbnails/29.jpg)
Corporate Applications
E[ Revenue | Switch ad agency? ]E[ Revenue | Raise price 10%? ] E[ Project done date | Drop feature? ]E[ Project done date | Add personnel? ]E[ Stock price | Fire CEO? ]E[ Stock price | Acquire firm X? ]
![Page 30: Robin Hanson PM Summit NYC 2009](https://reader035.vdocuments.us/reader035/viewer/2022070315/5552491ab4c9059a4f8b4a81/html5/thumbnails/30.jpg)