michigania 02005 a logic of diversity ii scott e page complex systems, political science, economics...
TRANSCRIPT
Michigania 02005
A Logic of Diversity II
Scott E Page
Complex Systems, Political Science, Economics and
Institute for Social Research
University of MichiganSanta Fe Institute
Michigania 02005
Enlarging The Mantra
Identity,
Training,
Experiential
Diversity
Diverse
Perspectives
Better
Outcomes
Michigania 02005
Why Construct Models?Models allow us to provide conditions for
when a statement is true.
The Pythagorean Theorem: ``A-squared equals B-squared plus C squared’’ only holds for right triangles.
Michigania 02005
Finding the Conditions``Two heads are better than one!’’
``Too many cooks spoil the broth’’
Which one wins? Which do we apply in a given setting.
Michigania 02005
Finding the Conditions``Two heads are better than one!’’
``Too many cooks spoil the broth’’
Condition: For an irreversible process, too many cooks spoil the broth.
Michigania 02005
Swarm of BeesAlmost all of social science looks at
averages and changes in those averages.
Analogy: if you look at a swarm of bees, the path of any one bee is hard to predict and understand, but in the swarm all of those idiosyncratic behaviors cancel out and we can identify general trends.
Michigania 02005
The BuzzBee hives must stay around 96 degrees in
order for bees to reach maturation. Bees achieve this by genetic mechanisms that drive two behaviors:
When hot: fan out or leave the hiveWhen cool: huddle together
Michigania 02005
Diversity and HomeostasisGenetically homogeneous bees: All
get cool (or hot) at the same time. Temperature in hive fluctuates wildly. (1930’s heating system)
Genetically diverse bees: Get cool (or hot) at different temperatures. Temperature stabilizes.
Michigania 02005
Rethinking the SwarmThe logic of cancellation does not hold
because there are feedbacks between the bees. Those feedbacks imply we cannot look at averages.
Groups of people solving problems, making predictions, and making choices create feedbacks in abundance.
Michigania 02005
Performance
• Average Performance Given– solution in perspective– application of heuristics
• Ben and Jerry– average quality of solution = 82
• Consultant- average quality of solution = 74
Michigania 02005
Perspective Diversity
8375
7381
80
86 8380 74
Ben and Jerrystuck at 83
consultantGets to 86
Michigania 02005
Diversity or Ability: A Test
Create a bunch of artificial problem solving agents and rank these agents by their average performances on a difficult problem.
All of the agents must be “smart”
Michigania 02005
Two Groups• Group 1: Best 20 agents• Group 2: Random 20 agents
Have each group work collectively - when one agent gets stuck at a point, another agent tries to find a further improvement. Group stops when no one can find a better solution.
Michigania 02005
And the winner is..
“Most of the time” the diverse group outperforms the group of the best by a substantial margin.
See Lu Hong and Scott Page Proceedings of the National Academy of Sciences (2002)
Michigania 02005
The Toolbox View
ABD
EZ
AHK FD
Alpha Group Diverse Group
ADE BCD
ABC
BCD
ACD
BCD
AEG
IL
Michigania 02005
Formal Version
Theorem: Given a set of diverse problem solvers, a random collection outperforms a collection of the “best” individual problem solvers provided
-the set is large-the problem is hard-the problem solvers are smart
Michigania 02005
The Madness of Crowds
We tend to think of crowds of people as irrational mobs. And that can be true. When people hear the ideas and opinions of others, they often succumb to peer pressure rather than speaking their own minds.
Michigania 02005
The Wisdom of Crowds
If people do not hear the opinions of others, or if they render their true predictions anyway crowds can be incredibly wise.
Michigania 02005
Suroweicki’s Examples
Morton Thiokol’s stock plungePrediction Markets
Hollywood Stock ExchangeIowa Electronic MarketSports Betting Markets
Who Wants to be a Millionaire1906 West of England Fat Stock and
Poultry Exhibition
Michigania 02005
Two Separate Phenomena1. Information known by part of the crowd
2. Aggregative diverse predictive models
Michigania 02005
Revealing Known Information
Which of the following books would you NOT find in the Point o’ Pines Library
A. The Periwinkle Steamboat - LancasterB. Curtains - Agatha ChristieC. Unabridged Crossword Puzzle DictionaryD. I am Charlotte Simmons - Tom Wolfe
Michigania 02005
Information RisingSuppose that no one know the answer but
that 18 people know one of the books on the list is in the library and that 18 people know two of the books on the list are in the library. This means that 64 people guess randomly.
Michigania 02005
Information Rising
Of 64 Clueless: Correct answer gets 16Of 18 know one: Correct answer gets 6Of 18 know two: Correct answer gets 9
Total 31
Other answers get 23 (on average)
Michigania 02005
The Answer Is…Which of the following books would you NOT find in
the Point o’ Pines Library
B. Curtains - Agatha Christie
Michigania 02005
Aggregating Diverse Predictions
In most of the situations described, people do not know the answer yet. We can assume that people have diverse predictive models. We’d like to understand how that aggregation occurs and what roles diversity and ability play.
Michigania 02005
Experience Interpretation
HExperience MH
ML
L
G G G
G G G G
G
G
B
B
B
B
B
B
B
B
B
B
B
B
B G
G
75 % Correct
Michigania 02005
Charisma Interpretation
H MH ML L
75% Correct
G B
G BBBG
G G
BG G
BG G
BG G
G
G
B
B
B
B B
Michigania 02005
Balanced Interpretation
H MH ML L
75% Correct H
Good to beextreme on one MHmeasure, bad on other ML
L
G G G
G B G G
G
G
B
B
B
B
B
B
B
B
B
G
G
Michigania 02005
Voting OutcomeCharisma
H MH ML L
H
MH ML
L
GGB GGG GBG
GGG GGB G GBG
BGG
BGG
BGB
GBB
BBG
BBB
BBB
BBG
BBG
BGB
Michigania 02005
The Mathematics of Prediction
Prediction: # runs scored by winning softball team
Mon Tue WedBrad 8 10 9Matt 10 12 8
Michigania 02005
Squared Errors
Brad: (8-8)2 +(10-12)2 +(10-9)2 = 5
Matt : (10-8)2 +(12-12)2 +(8-9)2 = 5
Crowd: (9-8)2 +(11-12)2 +(9-9)2 = 2
Michigania 02005
Diversity of Predictions
(Brad-Crowd)2 = 1 + 1 + 1 = 3 (Matt-Crowd)2 = 1 + 1 + 1 = 3
Michigania 02005
Diversity Prediction Theorem
Crowd Error = Average Error - Diversity
(note: proven by statisticians, computer scientists, and economists)
Michigania 02005
Crowd = Average - Diversity
• Diversity as important as ability
• Limit to how much diversity(otherwise crowd error would be negative)
Michigania 02005
Experts on NFL DraftPlayer #1 #2 #3 #4 #5 #6 #7 #8 Alex Smith 1 1 1 1 1 1 1 2 Ronnie Brown 2 2 4 2 2 5 2 6 Braylon Edwards 3 3 2 7 3 2 3 3 Cedric Benson 4 4 13 4 8 4 8 4 Carnell Williams 8 5 5 5 4 13 4 8 Adam Jones 16 9 6 8 6 6 9 17
Error^2 158 89 210 235 112 82 39 300
Average Error: 153.13Diversity: 101.52
Michigania 02005
Crowd of Experts on NFL Draft
Player CrowdAlex Smith 1.13 Ronnie Brown 3.13Braylon Edwards 3.25Cedric Benson 6.13Carnell Williams 6.50Adam Jones 9.63
Error^2 51.61
Michigania 02005
Does Crowd Beat Best?
In the NFL draft example, the best predictor Pete Brisco had an error of only 39. He outperformed the crowd, which had an error of 51.6.
Michigania 02005
Novices and Experts
Novices: Base their models on only a few variables or a few boxes.
Experts: Base their models on many variables or many boxes.
Michigania 02005
In Praise of Experts
Theorem: If an expert contains every variable considered by any one of the novices, the expert predicts better than the crowd of novices.
Michigania 02005
Crowds vs Experts
Test Set: Linear functions defined over 20 variables.
Crowd: Each of 100 novices looks at N randomly chosen variables
Expert: Looks at E>N variablesTraining: 300 independent
variablesContest: 300 independent variables
Michigania 02005
What’s Happening
Expert: Getting best fit over all his variables.
Crowd: Getting an average of many fits over many distinct subsets of variables.
Michigania 02005
Diversity and Prediction
Diverse predictors generate better predictions unless someone’s head is large enough and data is sufficient enough for a complete model.
Michigania 02005
Instrumental vs Fundamental
Fundamental Preferences: Preferences over outcomes
Instrumental Preferences: Preferences over policies to attain outcomes
Michigania 02005
Instrumental Politics“I am the _____ candidate”
A.Pro crimeB.Anti childC.Anti environmentalD.Pro drug addictionE. Higher health care costs
Michigania 02005
Preference Cycle
A = Arts & Crafts, B = Boating, T = Tennis
Lindsey: A > B > T Samuel: B > T > ABecca : T > A > B
Michigania 02005
Preference Cycle
Lindsey: A > B > T Samuel: B > T > ABecca : T > A > B
• Majority Vote Outcome: A > B > T > A
Michigania 02005
ManipulationGiven any voting rule, people with
diverse preferences will always have an incentive to misrepresent themselves.
Implication: People in diverse groups will not trust one another as much.
Michigania 02005
Under Provision
If we want different outcomes and have a fixed budget, we are likely to spread our money too thin.
Idea: Rather than have a good car or a nice boat, we have a lousy car and a lousy boat.
Michigania 02005
Theoretical Summary
Tasks involve- Solution Generation (problem solving)- Evaluation (prediction)- Choice (preference aggregation)
Michigania 02005
“Diversity is Ability”
To be different is to be able to make a contribution.
Diversity Trumps Ability: Diverse group does better than “able” group at problem solving
Michigania 02005
“Diversity is Ability”
Diversity Prediction TheoremCrowd Error = individual error -
diversity
(ability and diversity enter equally)
Michigania 02005
ComplicationPreference diversity creates cycles.
It creates incentives to act strategically and to manipulate agendas.
At the same time, preference diversity may be a primary cause of the other types of diversity.
Michigania 02005
SummaryThe empirical evidence suggests
that diverse perspectives, mental models, and tools lead to “better outcomes” but that value diversity creates problems.
Michigania 02005
Quick Look at the ``Facts’’• Growth of modern civilization• National level GDP• City level productivity• Diverse team performance
Michigania 02005
Rise of Modern Civilization• Jared Diamond: diversity/easy
problems• Joel Mokyr: exploiting diversity• Michael Kremer: 1 million years of
data shows growth and population size correlated
Michigania 02005
National Level GDP• Paul Romer: Diversity crucial to
economic growth• Ethnic Linguistic Fractionalization
(ELF): strongly negatively correlated with economic growth
Michigania 02005
Performance of Cities (42)• Doubling of city size increases
productivity by 6% to 20%• Arrow, Lucas: spillovers within an
industry (silicon valley)• Jacobs, Auerbach: spillovers
between industries (just in time)*
Michigania 02005
Identity Diverse Teams• Generate more solutions (many
worse)• Thomas and Ely: do better if they
have diverse heuristics and perspectives
• People in diverse groups are less happy - world views are challenged - feel like outcomes were
manipulated
Michigania 02005
The End of Great Scientists First 10 Last 10
Physics Nobels: 14 28Chemistry Nobels: 10 27
(There’s a maximum of three)
Michigania 02005
Final ThoughtIndividual ability not likely to grow
much.
Collective diversity can grow.
Diversity is our best hope to solve problems and to create innovations.