![Page 1: "Let's leave that up to the computer" Chris Snijders](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d155503460f949eadd7/html5/thumbnails/1.jpg)
"Let's leave that up to the computer"
Chris Snijders
![Page 2: "Let's leave that up to the computer" Chris Snijders](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d155503460f949eadd7/html5/thumbnails/2.jpg)
www.chrissnijders.com/supercrunchers2012
![Page 3: "Let's leave that up to the computer" Chris Snijders](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d155503460f949eadd7/html5/thumbnails/3.jpg)
Chris Snijders, Super Crunchers, 2012
Chris [email protected]
Martijn [email protected]
Kees van [email protected]
![Page 4: "Let's leave that up to the computer" Chris Snijders](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d155503460f949eadd7/html5/thumbnails/4.jpg)
Chris Snijders, Super Crunchers, 2012
Passing the course …
• Presence and active participation, including presentation, evaluation of others, …
• Create “CaseFile” based on the SuperCrunchers book
• Write assignment about own “Super Cruncher” idea
![Page 5: "Let's leave that up to the computer" Chris Snijders](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d155503460f949eadd7/html5/thumbnails/5.jpg)
Chris Snijders, Super Crunchers, 2012
Overview of today
• A famous example: Cook county hospital• The science behind it• Computers as decision makers
• Warning! Upcoming assignment
![Page 6: "Let's leave that up to the computer" Chris Snijders](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d155503460f949eadd7/html5/thumbnails/6.jpg)
Chris Snijders, Super Crunchers, 2012
Case: Cook county hospital
Emergency Department
- 250.000 patients per year
- many persons without insurance
- not enough rooms, overworked staff
- 1996: Brendan Reilly director
(see Gladwell, 2005)
![Page 7: "Let's leave that up to the computer" Chris Snijders](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d155503460f949eadd7/html5/thumbnails/7.jpg)
Chris Snijders, Super Crunchers, 2012
Problem 1: acute chest pain
Diagnose through:
blood pressure, stethoscope: fluid in the lungs, how long have you been experiencing pain, how does it feel precisely, where does it hurt, does it always hurt or only when you exercise, have you had heart problems before, how about your cholesterol, do you have diabetes, let's look at your ECG, are there any heart problems in the family, do you use drugs, how old are you, are you in shape, do you smoke, do you drink, check appearance: stressed, overweight, ....
High risk : 8
Medium risk : 12
Go home30 p/day
![Page 8: "Let's leave that up to the computer" Chris Snijders](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d155503460f949eadd7/html5/thumbnails/8.jpg)
Chris Snijders, Super Crunchers, 2012
Reilly finds Goldman: obv 10,000 cases
Only 4 things matter
ECGBlood pressureFluid in your lungs"unstable angina"
![Page 9: "Let's leave that up to the computer" Chris Snijders](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d155503460f949eadd7/html5/thumbnails/9.jpg)
Chris Snijders, Super Crunchers, 2012
Great! So let's do that! Or not...
Implementation: … physicians protest …
A test: 20 cases were given to several physicians
Hardly any agreement between physicians!
![Page 10: "Let's leave that up to the computer" Chris Snijders](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d155503460f949eadd7/html5/thumbnails/10.jpg)
Chris Snijders, Super Crunchers, 2012
Reilly tests Goldman’s idea
vs
82% 95%
physician Goldman’s scheme
![Page 11: "Let's leave that up to the computer" Chris Snijders](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d155503460f949eadd7/html5/thumbnails/11.jpg)
Chris Snijders, Super Crunchers, 2012
A literature check …
Clinical versus statisticalprediction
For instance (zie Grove et al., 2000)
– Survival probabilities in medical procedures– Probability of recidivism– Probability of success of starting firms– Choice of job candidates– Diagnosing schizofrenia– Predicting school success– …
![Page 12: "Let's leave that up to the computer" Chris Snijders](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d155503460f949eadd7/html5/thumbnails/12.jpg)
Chris Snijders, Super Crunchers, 2012
What would a typicaltopic be, where you
would expect the humanto win?
![Page 13: "Let's leave that up to the computer" Chris Snijders](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d155503460f949eadd7/html5/thumbnails/13.jpg)
Chris Snijders, Super Crunchers, 2012
The results …
Over 160 studies
When given the same info,the number of cases in whichthe expert wins = ??
![Page 14: "Let's leave that up to the computer" Chris Snijders](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d155503460f949eadd7/html5/thumbnails/14.jpg)
Chris Snijders, Super Crunchers, 2012
Models beat Humans (often)How can this be?
![Page 15: "Let's leave that up to the computer" Chris Snijders](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d155503460f949eadd7/html5/thumbnails/15.jpg)
Chris Snijders, Super Crunchers, 2012
... we have some clues ...
We emphasize the improbable’ (Stickler)
Confirmation bias (Edwards, Wason)
Hindsight bias (Fischhoff)
Cognitive dissonance (Festinger)
“Dealing with probabilities / Base rate neglect”(Bar-Hillel)
Mental sets (Redelmayer, Tversky)
Our memory fools us (Wagenaar)
![Page 16: "Let's leave that up to the computer" Chris Snijders](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d155503460f949eadd7/html5/thumbnails/16.jpg)
Chris Snijders, Super Crunchers, 2012
And there are more of these
"Mental Floating Frankfurters"
![Page 17: "Let's leave that up to the computer" Chris Snijders](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d155503460f949eadd7/html5/thumbnails/17.jpg)
Chris Snijders, Super Crunchers, 2012
Restriction 1: “Mental sets”
Connect the 9 dots with at most 4 straight lines, without lifting your pen from the paper.
• • • • • • • • •
![Page 18: "Let's leave that up to the computer" Chris Snijders](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d155503460f949eadd7/html5/thumbnails/18.jpg)
Chris Snijders, Super Crunchers, 2012
Restriction 2: Memory
“Where were you, when …”
Shuttle Columbia Crew Lost Feb. 1, 2003
![Page 19: "Let's leave that up to the computer" Chris Snijders](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d155503460f949eadd7/html5/thumbnails/19.jpg)
Chris Snijders, Super Crunchers, 2012
Restriction 3: the “availability heuristic”
Which is more likely, a plane crash or a car crash?
![Page 20: "Let's leave that up to the computer" Chris Snijders](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d155503460f949eadd7/html5/thumbnails/20.jpg)
Chris Snijders, Super Crunchers, 2012
Restriction 4a: dealing with probabilities
Suppose: a manager has a good intuition in business:
– when a problem will arise: he gets a gut-feeling that something is wrong with probability 90%
– when no problem will arise: he gets a gut-feeling that something is wrong with probability 10%
On average, there is a problem in 5% of the transactions.
The manager starts a transaction, and he gets a gut-feeling that something might be wrong.
What is the probability that something is indeed wrong?
![Page 21: "Let's leave that up to the computer" Chris Snijders](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d155503460f949eadd7/html5/thumbnails/21.jpg)
Chris Snijders, Super Crunchers, 2012
Restriction 4a - solution
1,000 transactions
50 are problematic950 are not problematic
45xsignal
5xNo signal
855xNo signal
95xsignal
If a signal, then the probability of a problem = 45 / (45+95) = 0.32
![Page 22: "Let's leave that up to the computer" Chris Snijders](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d155503460f949eadd7/html5/thumbnails/22.jpg)
Chris Snijders, Super Crunchers, 2012
Restriction 4b: dealing with probabilities
A murder has been committed. The only evidence available is DNA, found at the murder scene. DNA-research shows a match with your DNA.
The probability that two persons are diagnosed as having the same DNA is about 1 in 100.000.
How likely is it that you are the murderer?
![Page 23: "Let's leave that up to the computer" Chris Snijders](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d155503460f949eadd7/html5/thumbnails/23.jpg)
Chris Snijders, Super Crunchers, 2012
Restriction 5: overconfidence
Trivial Pursuit: estimate how many questions you will know
Estimates are generally too high ... and this gets worse with expertise!
![Page 24: "Let's leave that up to the computer" Chris Snijders](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d155503460f949eadd7/html5/thumbnails/24.jpg)
Chris Snijders, Super Crunchers, 2012
Restriction 6:Finding non-existent patterns
![Page 25: "Let's leave that up to the computer" Chris Snijders](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d155503460f949eadd7/html5/thumbnails/25.jpg)
Chris Snijders, Super Crunchers, 2012
Restriction 7: the noble art of finding a broken leg
![Page 26: "Let's leave that up to the computer" Chris Snijders](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d155503460f949eadd7/html5/thumbnails/26.jpg)
Chris Snijders, Super Crunchers, 2012
Restriction 8: where is the feedback?
![Page 27: "Let's leave that up to the computer" Chris Snijders](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d155503460f949eadd7/html5/thumbnails/27.jpg)
Chris Snijders, Super Crunchers, 2012
Decision making =Store, retrieve, combine
![Page 28: "Let's leave that up to the computer" Chris Snijders](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d155503460f949eadd7/html5/thumbnails/28.jpg)
Chris Snijders, Super Crunchers, 2012
Where were we?
End of 1 set of reasons "why" humans (even expert humans) are often outperformed by computer models.
![Page 29: "Let's leave that up to the computer" Chris Snijders](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d155503460f949eadd7/html5/thumbnails/29.jpg)
Chris Snijders, Super Crunchers, 2012
The competition:
• “Naturalistic decision making” (cf. work by Shanteau)
• Fast and frugal heuristics (cf. Gigerenzer)
![Page 30: "Let's leave that up to the computer" Chris Snijders](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d155503460f949eadd7/html5/thumbnails/30.jpg)
Chris Snijders, Super Crunchers, 2012
The timelines of ideas
... and the science behind many more questions that you can ask in relation to such topics
• This is an innovation process • Can we find consistencies across topics?• Which kind(s) of crunchers are more likely to
be adopted?• etc...
idea implementation
![Page 31: "Let's leave that up to the computer" Chris Snijders](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d155503460f949eadd7/html5/thumbnails/31.jpg)
Chris Snijders, Super Crunchers, 2012
Reilly finds Goldman: obv 10,000 cases
Only 4 things matter
ECGBlood pressureFluid in your lungs"unstable angina"
![Page 32: "Let's leave that up to the computer" Chris Snijders](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d155503460f949eadd7/html5/thumbnails/32.jpg)
Chris Snijders, Super Crunchers, 2012
AND?are we using this today?
![Page 33: "Let's leave that up to the computer" Chris Snijders](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d155503460f949eadd7/html5/thumbnails/33.jpg)
Chris Snijders, Super Crunchers, 2012
NO!
![Page 34: "Let's leave that up to the computer" Chris Snijders](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d155503460f949eadd7/html5/thumbnails/34.jpg)
Chris Snijders, Super Crunchers, 2012
![Page 35: "Let's leave that up to the computer" Chris Snijders](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d155503460f949eadd7/html5/thumbnails/35.jpg)
Chris Snijders, Super Crunchers, 2012
When and why do the models win?
Can we use the experts’ knowledge somehow?
When are the models used, and when not (and why is that)?
What can/should you do when you want to have a model-based solution?
What prevents people from using models?
![Page 36: "Let's leave that up to the computer" Chris Snijders](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d155503460f949eadd7/html5/thumbnails/36.jpg)
Chris Snijders, Super Crunchers, 2012
The Supercrunchers book
Intro
Who's doing your thinking for you?
Creating your own data with the
flip of a coin
Government by chance
Evidence based medicine
Experts vs equations
Why now?
Are we having fun yet?
On the web site
[CaseFile]
[Example]
[Issue]
[Method]
![Page 37: "Let's leave that up to the computer" Chris Snijders](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d155503460f949eadd7/html5/thumbnails/37.jpg)
Chris Snijders, Super Crunchers, 2012
“Super Crunching” = using data
• Supercrunching = Using (lots of) data to predict something (think Twitter, Blogs, Airmiles, …) that we normally cannot predict
• Supercrunching = (also) Using data to predict something that humans normally tend to predict– Experts vs models– Experiment– “Natural experiments”
![Page 38: "Let's leave that up to the computer" Chris Snijders](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d155503460f949eadd7/html5/thumbnails/38.jpg)
Chris Snijders, Super Crunchers, 2012
<show website now>
![Page 39: "Let's leave that up to the computer" Chris Snijders](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d155503460f949eadd7/html5/thumbnails/39.jpg)
Chris Snijders, Super Crunchers, 2012
Passing the course …
• Presence and active participation, including presentation, evaluation of others, …
• Create “CaseFile” based on the SuperCrunchers book
• Write assignment about own “Super Cruncher” idea (or: we take a single topic and each do parts of it)
![Page 40: "Let's leave that up to the computer" Chris Snijders](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d155503460f949eadd7/html5/thumbnails/40.jpg)
Chris Snijders, Super Crunchers, 2012
Up next …
• Next week: no class (read! And start preparing the first Assignments)
• The week after that: we get creative…
![Page 41: "Let's leave that up to the computer" Chris Snijders](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d155503460f949eadd7/html5/thumbnails/41.jpg)
Chris Snijders, Super Crunchers, 2012
To do for next week
• Read the book – cover to cover
• Think about the Cook-county case, try to think about general patterns
• Upcoming assignment will be to create a “casefile” for one of the topics in the book: check for topics that interest you, if possible choose one already, and let me know