"evidence-based management" by tom coupe
TRANSCRIPT
![Page 2: "Evidence-Based Management" by Tom Coupe](https://reader035.vdocuments.us/reader035/viewer/2022070521/58f01f601a28ab48638b45d7/html5/thumbnails/2.jpg)
Evidence Based ManagementTom Coupé
Kyiv School of Economics
![Page 3: "Evidence-Based Management" by Tom Coupe](https://reader035.vdocuments.us/reader035/viewer/2022070521/58f01f601a28ab48638b45d7/html5/thumbnails/3.jpg)
KSE MBA PROGRAM
PurelyEnglish
Balanced Analytical Connected to Business
Internationally Connected
![Page 4: "Evidence-Based Management" by Tom Coupe](https://reader035.vdocuments.us/reader035/viewer/2022070521/58f01f601a28ab48638b45d7/html5/thumbnails/4.jpg)
Jeffrey Pfeffer and Robert Sutton (Stanford)
![Page 5: "Evidence-Based Management" by Tom Coupe](https://reader035.vdocuments.us/reader035/viewer/2022070521/58f01f601a28ab48638b45d7/html5/thumbnails/5.jpg)
Basic Idea• Decisions based on
• Sound analysis• Data• Research
• Not on• Conventional wisdom• Tradition• Expert opinion.
==> a manager should also be able to think like a researcher
![Page 6: "Evidence-Based Management" by Tom Coupe](https://reader035.vdocuments.us/reader035/viewer/2022070521/58f01f601a28ab48638b45d7/html5/thumbnails/6.jpg)
An Example from Medecine
![Page 7: "Evidence-Based Management" by Tom Coupe](https://reader035.vdocuments.us/reader035/viewer/2022070521/58f01f601a28ab48638b45d7/html5/thumbnails/7.jpg)
An Example from Business
![Page 8: "Evidence-Based Management" by Tom Coupe](https://reader035.vdocuments.us/reader035/viewer/2022070521/58f01f601a28ab48638b45d7/html5/thumbnails/8.jpg)
Pop Tarts
![Page 9: "Evidence-Based Management" by Tom Coupe](https://reader035.vdocuments.us/reader035/viewer/2022070521/58f01f601a28ab48638b45d7/html5/thumbnails/9.jpg)
Data Driven Decision Making and Ukraine • Algorithmic trading at major investment bank• Banks and CRM• Demand for Beer• Retail: loyalty programs• Telecommunication• “We are hiring a Business Insights Manager (BIM) with business analytics being a key source of growth”. (Big Fast Food Chain)
![Page 10: "Evidence-Based Management" by Tom Coupe](https://reader035.vdocuments.us/reader035/viewer/2022070521/58f01f601a28ab48638b45d7/html5/thumbnails/10.jpg)
![Page 11: "Evidence-Based Management" by Tom Coupe](https://reader035.vdocuments.us/reader035/viewer/2022070521/58f01f601a28ab48638b45d7/html5/thumbnails/11.jpg)
Why does not everybody practice evidence-based management?
• Cognitive Biases
• Changing Power Dynamics
• The market for ideas is messy
![Page 12: "Evidence-Based Management" by Tom Coupe](https://reader035.vdocuments.us/reader035/viewer/2022070521/58f01f601a28ab48638b45d7/html5/thumbnails/12.jpg)
Why does not everybody practice evidence-based management? Cognitive Biases• Intuitive Thinking versus reflective thinking (Thaler and Sunstein, Nudge – Kahneman, Thinking Slow and Fast)
Biased decision making
• Confirmation bias ‘is a tendency to search for or interpret information in a way that confirms one's preconceptions, leading to statistical errors’ (Wikipedia)
• Status Quo bias: ‘a preference for the current state of affairs. The current baseline (or status quo) is taken as a reference point, and any change from that baseline is perceived as a loss’ (Wikipedia)
![Page 13: "Evidence-Based Management" by Tom Coupe](https://reader035.vdocuments.us/reader035/viewer/2022070521/58f01f601a28ab48638b45d7/html5/thumbnails/13.jpg)
Behavioral Biases in Investment Decision MakingGervais, 2009
• Overconfidence: managers overestimate the precision of their knowledge and information
Overconfidence distorts investment decisions
![Page 14: "Evidence-Based Management" by Tom Coupe](https://reader035.vdocuments.us/reader035/viewer/2022070521/58f01f601a28ab48638b45d7/html5/thumbnails/14.jpg)
Measuring Overconfidence• Malmendier and Tate (2005a, 2008) - CEO own wealth decisions
• Malmendier and Tate (2005b, 2008) - popular press • Ben-David et al. (2008) - forecast of returns • Lin, Hu, and Chen (2005) - forecasts of company earnings
![Page 15: "Evidence-Based Management" by Tom Coupe](https://reader035.vdocuments.us/reader035/viewer/2022070521/58f01f601a28ab48638b45d7/html5/thumbnails/15.jpg)
Overconfidence and Investments• Firms that have more cash and rely less on debt financing tend to invest more than other firms, keeping investment opportunities fixed.• More so with overconfident managers?
• Malmendier and Tate (2005a): yes• Glaser, Schäfers, and Weber (2008): not for CFOs, yes for boards.
![Page 16: "Evidence-Based Management" by Tom Coupe](https://reader035.vdocuments.us/reader035/viewer/2022070521/58f01f601a28ab48638b45d7/html5/thumbnails/16.jpg)
Overconfidence and Mergers• Malmendier and Tate (2008):
• P(acquisition): +65%• three-day abnormal return: -0.90 percent vs. -0.29 percent)
• $2 to 4 billion loss • Ben-David et al. (2008): overconfident CFOs
• More acquisitions • 1.3 percent lower returns
![Page 17: "Evidence-Based Management" by Tom Coupe](https://reader035.vdocuments.us/reader035/viewer/2022070521/58f01f601a28ab48638b45d7/html5/thumbnails/17.jpg)
Overconfidence and Entrepreneurs• Cooper et al. (1988)
• 35 percent of entrepreneurs say 100 percent probability success
• half the ventures end up failing.
![Page 18: "Evidence-Based Management" by Tom Coupe](https://reader035.vdocuments.us/reader035/viewer/2022070521/58f01f601a28ab48638b45d7/html5/thumbnails/18.jpg)
Why does not everybody practice evidence-based management? Changing Power Dynamics
• Authority versus Knowledge
![Page 19: "Evidence-Based Management" by Tom Coupe](https://reader035.vdocuments.us/reader035/viewer/2022070521/58f01f601a28ab48638b45d7/html5/thumbnails/19.jpg)
Why does not everybody practice evidence-based management? Too much knowledge
• The market for ideas is messy
![Page 20: "Evidence-Based Management" by Tom Coupe](https://reader035.vdocuments.us/reader035/viewer/2022070521/58f01f601a28ab48638b45d7/html5/thumbnails/20.jpg)
Why does not everybody practice evidence-based management?
• Cognitive Biases• Changing Power Dynamics• The market for ideas is messy
• Next: How to be an evidence-based manager?
![Page 21: "Evidence-Based Management" by Tom Coupe](https://reader035.vdocuments.us/reader035/viewer/2022070521/58f01f601a28ab48638b45d7/html5/thumbnails/21.jpg)
How to be an evidence based manager?• 1. Formulate Answerable Questions• 2. Search for the best available evidence• 3. Evaluate the evidence• 4. Integrate this relevant evidence with your managerial experience and organizational concerns. If possible, generate evidence within your context.
• 5. Apply
![Page 22: "Evidence-Based Management" by Tom Coupe](https://reader035.vdocuments.us/reader035/viewer/2022070521/58f01f601a28ab48638b45d7/html5/thumbnails/22.jpg)
How to be an evidence based manager?• 1. Formulate Answerable Questions• 2. Search for the best available evidence• 3. Evaluate the evidence• 4. Integrate this relevant evidence with your managerial experience and organizational concerns. If possible, generate evidence within your context.
• 5. Apply
![Page 23: "Evidence-Based Management" by Tom Coupe](https://reader035.vdocuments.us/reader035/viewer/2022070521/58f01f601a28ab48638b45d7/html5/thumbnails/23.jpg)
Formulating Answerable Questions• PICOC• Problem or Population – who?• Intervention – what/how?• Comparison – compared to?• Outcome – what outcome?• Context – in what environment?
• Example: I am a consultant who advices a health care company that plans to merge with another company that has a different corporate culture – I wonder whether this difference in corporate culture will affect the outcome of the merger? (based on CEBMA)
![Page 24: "Evidence-Based Management" by Tom Coupe](https://reader035.vdocuments.us/reader035/viewer/2022070521/58f01f601a28ab48638b45d7/html5/thumbnails/24.jpg)
To be answerable, you need to know• P: What kind of Population are we talking about? Middle managers, back-office employees, medical staff, clerical staff?
• P/C: And how is the assumed cultural difference assessed? Is it the personal view of some managers or is it measured by a validated instrument?
![Page 25: "Evidence-Based Management" by Tom Coupe](https://reader035.vdocuments.us/reader035/viewer/2022070521/58f01f601a28ab48638b45d7/html5/thumbnails/25.jpg)
Extra Info Provided• According to the board the objective of the merger is to integrate the back-office of the two health care organizations (ICT, finance, purchasing, facilities, personnel administration, etc.) in order to create economy of scale. The front offices and primary process of the two organizations will remain separate.
• The cultural difference is not objectively assessed (it is the perception of the senior managers of both organizations).
![Page 26: "Evidence-Based Management" by Tom Coupe](https://reader035.vdocuments.us/reader035/viewer/2022070521/58f01f601a28ab48638b45d7/html5/thumbnails/26.jpg)
Already better• P =back office employees • I = merger, integration back office • C = status quo• O = economy of scale • C = healthcare
![Page 27: "Evidence-Based Management" by Tom Coupe](https://reader035.vdocuments.us/reader035/viewer/2022070521/58f01f601a28ab48638b45d7/html5/thumbnails/27.jpg)
Answerable Question• P = back office employees • I = merger, integration back office • C = status quo• O = economy of scale
• Employee productivity?• Job satisfaction?• Return on investment? • Market share?• Organizational commitment?
• C = healthcare
![Page 28: "Evidence-Based Management" by Tom Coupe](https://reader035.vdocuments.us/reader035/viewer/2022070521/58f01f601a28ab48638b45d7/html5/thumbnails/28.jpg)
How to be an evidence based manager?• 1. Formulate Answerable Questions• 2. Search for the best available evidence• 3. Evaluate the evidence• 4. Integrate this relevant evidence with your managerial experience and organizational concerns. If possible, generate evidence within your context.
• 5. Apply
![Page 29: "Evidence-Based Management" by Tom Coupe](https://reader035.vdocuments.us/reader035/viewer/2022070521/58f01f601a28ab48638b45d7/html5/thumbnails/29.jpg)
Searching for evidence• Own Experience+• Experience of Competitors+• Academic literature• Case studies • News • Google
![Page 30: "Evidence-Based Management" by Tom Coupe](https://reader035.vdocuments.us/reader035/viewer/2022070521/58f01f601a28ab48638b45d7/html5/thumbnails/30.jpg)
How to be an evidence based manager?• 1. Formulate Answerable Questions• 2. Search for the best available evidence• 3. Evaluate the evidence• 4. Integrate this relevant evidence with your managerial experience and organizational concerns. If possible, generate evidence within your context.
• 5. Apply
![Page 31: "Evidence-Based Management" by Tom Coupe](https://reader035.vdocuments.us/reader035/viewer/2022070521/58f01f601a28ab48638b45d7/html5/thumbnails/31.jpg)
Evaluate the evidence• Know analytical tools and concepts
• Econometrics, statistics!• Criteria to evaluate information
• Causality• Sample Size• Sample Selection• Explicit about shortcomings
![Page 32: "Evidence-Based Management" by Tom Coupe](https://reader035.vdocuments.us/reader035/viewer/2022070521/58f01f601a28ab48638b45d7/html5/thumbnails/32.jpg)
Causality vs Correlation
![Page 33: "Evidence-Based Management" by Tom Coupe](https://reader035.vdocuments.us/reader035/viewer/2022070521/58f01f601a28ab48638b45d7/html5/thumbnails/33.jpg)
The Simpsons: "Much Apu About Nothing“Homer: Not a bear in sight. The
"Bear Patrol" is working like a charm!
Lisa: That's specious reasoning, Dad.
Homer: [uncomprehendingly] Thanks, honey.
Lisa: By your logic, I could claim that this rock keeps tigers away.
Homer: Hmm. How does it work? Lisa: It doesn't work; it's just a
stupid rock! Homer: Uh-huh. Lisa: But I don't see any tigers
around, do you? Homer: (pause) Lisa, I want to
buy your rock.
![Page 34: "Evidence-Based Management" by Tom Coupe](https://reader035.vdocuments.us/reader035/viewer/2022070521/58f01f601a28ab48638b45d7/html5/thumbnails/34.jpg)
Possible Explanations for Correlation
![Page 35: "Evidence-Based Management" by Tom Coupe](https://reader035.vdocuments.us/reader035/viewer/2022070521/58f01f601a28ab48638b45d7/html5/thumbnails/35.jpg)
Causality versus correlation
•Bain: ‘our clients outperform the market 3 to 1’
![Page 36: "Evidence-Based Management" by Tom Coupe](https://reader035.vdocuments.us/reader035/viewer/2022070521/58f01f601a28ab48638b45d7/html5/thumbnails/36.jpg)
![Page 37: "Evidence-Based Management" by Tom Coupe](https://reader035.vdocuments.us/reader035/viewer/2022070521/58f01f601a28ab48638b45d7/html5/thumbnails/37.jpg)
Big Samples
•How do Belgians look like?• ‘case studies’
![Page 38: "Evidence-Based Management" by Tom Coupe](https://reader035.vdocuments.us/reader035/viewer/2022070521/58f01f601a28ab48638b45d7/html5/thumbnails/38.jpg)
Non-Random Samples - Selection• Selection by Researcher (reporter)
• ‘In Search of Excellence: Lessons from America’s Best-Run Companies’
• Self-Selection of Respondents
![Page 39: "Evidence-Based Management" by Tom Coupe](https://reader035.vdocuments.us/reader035/viewer/2022070521/58f01f601a28ab48638b45d7/html5/thumbnails/39.jpg)
Shortcomings• Analysis are stronger if shortcomings are made explicit
![Page 40: "Evidence-Based Management" by Tom Coupe](https://reader035.vdocuments.us/reader035/viewer/2022070521/58f01f601a28ab48638b45d7/html5/thumbnails/40.jpg)
Advanced: Critically Appraised Topic• Example-CAT.pdf
![Page 41: "Evidence-Based Management" by Tom Coupe](https://reader035.vdocuments.us/reader035/viewer/2022070521/58f01f601a28ab48638b45d7/html5/thumbnails/41.jpg)
How to be an evidence based manager?• 1. Formulate Answerable Questions• 2. Search for the best available evidence• 3. Evaluate the evidence• 4. Integrate this relevant evidence with your managerial experience and organizational concerns. If possible, generate evidence within your context.
• 5. Apply
![Page 42: "Evidence-Based Management" by Tom Coupe](https://reader035.vdocuments.us/reader035/viewer/2022070521/58f01f601a28ab48638b45d7/html5/thumbnails/42.jpg)
Exercise – what is the best book title?• Super Crunchers – why thinking by numbers is the new way to be smart
• The End of Intuition - why thinking by numbers is the new way to be smart
![Page 43: "Evidence-Based Management" by Tom Coupe](https://reader035.vdocuments.us/reader035/viewer/2022070521/58f01f601a28ab48638b45d7/html5/thumbnails/43.jpg)
![Page 44: "Evidence-Based Management" by Tom Coupe](https://reader035.vdocuments.us/reader035/viewer/2022070521/58f01f601a28ab48638b45d7/html5/thumbnails/44.jpg)
How to be an evidence based manager?• 1. Formulate Answerable Questions• 2. Search for the best available evidence• 3. Evaluate the evidence• 4. Integrate this relevant evidence with your managerial experience and organizational concerns. If possible, generate evidence within your context.
• 5. Apply
![Page 45: "Evidence-Based Management" by Tom Coupe](https://reader035.vdocuments.us/reader035/viewer/2022070521/58f01f601a28ab48638b45d7/html5/thumbnails/45.jpg)
Google’s Quest to Build a better Boss• What makes a good manager?• 10000 observations 8 rules • Trade off ‘technical expertise’ vs ‘being accessible’
• Improved performance: 75% of worst performing managers significantly improved
![Page 46: "Evidence-Based Management" by Tom Coupe](https://reader035.vdocuments.us/reader035/viewer/2022070521/58f01f601a28ab48638b45d7/html5/thumbnails/46.jpg)
Google’s Research on Successful teams• “Over two years we conducted 200+ interviews with Googlers
(our employees) and looked at more than 250 attributes of 180+ active Google teams. … Who is on a team matters less than how the team members interact, structure their work, and view their contributions.
• Psychological safety: Can we take risks on this team without feeling insecure or embarrassed?
• Dependability: Can we count on each other to do high quality work on time?
• Structure & clarity: Are goals, roles, and execution plans on our team clear?
• Meaning of work: Are we working on something that is personally important for each of us?
• Impact of work: Do we fundamentally believe that the work we’re doing matters?”
![Page 47: "Evidence-Based Management" by Tom Coupe](https://reader035.vdocuments.us/reader035/viewer/2022070521/58f01f601a28ab48638b45d7/html5/thumbnails/47.jpg)
Evidence Based Management and Organizational Design• Be committed to "fact based" decision making• build a culture in which people are encouraged to tell the truth
• encourage experimentation and learning by doing.• Look for the risks and drawbacks in what people recommend
![Page 48: "Evidence-Based Management" by Tom Coupe](https://reader035.vdocuments.us/reader035/viewer/2022070521/58f01f601a28ab48638b45d7/html5/thumbnails/48.jpg)
Exercise – bank deposits• You are the CEO of a bank and you want to increase the amount of money clients deposit on their savings account – what do you do?
![Page 49: "Evidence-Based Management" by Tom Coupe](https://reader035.vdocuments.us/reader035/viewer/2022070521/58f01f601a28ab48638b45d7/html5/thumbnails/49.jpg)
Exercise – Glass Windows• You are the CEO of a company that installs glass windows – your employees all are paid a fixed wage. Should you switch to a flexible wage, where wage and performance are linked?
![Page 50: "Evidence-Based Management" by Tom Coupe](https://reader035.vdocuments.us/reader035/viewer/2022070521/58f01f601a28ab48638b45d7/html5/thumbnails/50.jpg)
Important Lessons• 1. Decision making should be based on analysis• 2. Use the criteria to evaluate knowledge• 3. Use own data whenever possible • 4. Design your organization in such a way that 1 becomes possible
![Page 51: "Evidence-Based Management" by Tom Coupe](https://reader035.vdocuments.us/reader035/viewer/2022070521/58f01f601a28ab48638b45d7/html5/thumbnails/51.jpg)
The Best Question After this Talk?• Capgemini Survey - on average, business performance improvements of 26% in processes where big data analytics have been applied
• Brynjolfson et al (2011): ‘We examine whether firms that emphasize decision making based on data and business analytics (“data driven decision making” or DDD) show higher performance. Using detailed survey data on the business practices and information technology investments of 179 large publicly traded firms, we find that firms that adopt DDD have output and productivity that is 5-6% higher than what would be expected given their other investments and information technology usage. Furthermore, the relationship between DDD and performance also appears in other performance measures such as asset utilization, return on equity and market value.’
![Page 52: "Evidence-Based Management" by Tom Coupe](https://reader035.vdocuments.us/reader035/viewer/2022070521/58f01f601a28ab48638b45d7/html5/thumbnails/52.jpg)
Conclusion I• Practice Evidence Based Management
• Decision making should be based on analysis• Use the criteria to evaluate knowledge• Use own data whenever possible • Design your organization in such a way that 1 becomes possible
![Page 53: "Evidence-Based Management" by Tom Coupe](https://reader035.vdocuments.us/reader035/viewer/2022070521/58f01f601a28ab48638b45d7/html5/thumbnails/53.jpg)
Conclusion II• Who Will Benefit?
• Companies that create value from their data• Companies that help business to create value from their data
• Those who combine managerial skills with an understanding of data analytics!