evidence-based management evidence-based practice insights from key domains university of edinburgh,...
TRANSCRIPT
Evidence-Based Management
Evidence-Based Practice Insights from Key Domains
University of Edinburgh, May 2nd 2014
Eric Barends Rob Briner
A brief history
Current developments
Future developments
Challenges
Implications for practice & research
2
Evidence-Based Management
4
Manager > ‘Manage’
introduced mid 16th century: from Italian maneggiare, “to handle”, based on Latin manus, “hand”.
Management?
(we’re all managers)
History of management/business schools
Late 1800s - started as trade schools
1950s - Ford and Carnegie reports criticised business
schools for not being sufficiently academic or scientific
1980s onwards – criticised for producing research
irrelevant to practice and students without relevant skills
or knowledge
5
On the one hand they fear . . . the scorn of other, more
traditional academic subjects. On the other hand, they
often stand accused of being less than relevant to
business. (Grey, 2001)
2000s onwards – criticized for producing ideas and
students that lead to financial crisis and poor ethical
behaviour (e.g., Enron, only one US president has MBA,
MBA students cheat more than other graduate students)
6
History of management/business schools
Management schools are big business
Nat. Center for Education Statistics (USA, 2011-’12)
20% of all Batchelor’s Degrees
25% of all Master’s Degrees
Higher Education Statistics Agency (UK, 2012-’13)
27% of all Higher Degrees awarded
18% of all Higher Education qualifications obtained
UK Council for International Student Affairs (2011-’12)
36% of all business school students are international
30% of all international students studying business7
EBP in Management
Many similar ideas in past (e.g., Mode 1-Mode 2,
collaborative research, rigour and relevance)
Rousseau (2005) Presidential address
Peffer and Sutton (2006) Hard Facts, Dangerous Half-
Truths, and Total Nonsense: Profiting from Evidence-
based Management
8
EBP in Management
EBMgt Collaborative (2007-’08) Rousseau/Carnegie Mellon
Wiley Evidence-Based Management Insights (2007-’08)
(attempt to create Cochrane database for management)
Briner and Denyer
Presentation to AoM Board of Governors (2008) (attempt to
persuade to develop systematic reviews in management)
Center for Evidence-Based Management – more later
9
Evidence-based practice
Central Premise:
Decisions should be based on a
combination of critical thinking
and the ‘best available evidence‘.
12
Evidence?
findings from scientific research,
organizational facts & figures,
benchmarking, best practices,
professional experience
13
But…many managers pay
little or no attention to the
quality of the evidence they
base their decisions on
15
Teach managers how to
critically evaluate the validity,
and generalizability of the
evidence and help them find
‘the best available’ evidence
18
Evidence-based practice is about making decisions throughthe conscientious, explicit and judicious use of
the best available evidence from multiple sources by
1. Asking: translating a practical issue or problem into an answerable question
2. Acquiring: systematically searching for and retrieving the evidence
3. Appraising: critically judging the trustworthiness and relevance of the evidence
4. Aggregating: weighing and pulling together the evidence
5. Applying: incorporating the evidence in the decision-making process
6. Assessing: evaluating the outcome of the decision taken
to increase the likelihood of a favorable outcome.
Definition
19
Evidence based decision
Professional experience and
judgment
Organizational data, facts and figures
Stakeholders’ values and concerns
Scientific research findings
AskAcquire
AppraiseAggregate
ApplyAssess
20
Differences?
Research: low internal validity
Lack of evidence summaries
Focus on multiple sources
Organizational (BIG) data
Stakeholders’ concerns
How evidence-based are we (managers)?
“I’ve never thought I need more
evidence before making a
decision;
I know what needs to be done,
we get on with it and we get
results.”
25
How evidence-based are we?
959 (US) + 626 (Dutch) HR professionals
35 statements, based on an extensive body of
evidence
true / false / uncertain
HR Professionals' beliefs about effective human resource practices: correspondence between research and practice, (Rynes et al, 2002, Sanders et al 2008)
26
1. Incompetent people benefit more from feedback than
highly competent people.
2. Task conflict improves work group performance while
relational conflict harms it.
3. Encouraging employees to participate in decision
making is more effective for improving organizational
performance than setting performance goals.
True (likely) or false (not likely)?
27
Postgraduate Course
Current developments
Education
Access to research databases
REAs
Building a community
Postgraduate Course
CEBMa Database of Evidence Summaries
Online learning modules
Accreditational bodies
Future developments: practice
Postgraduate Course
Some move towards systematic review methods in management
Limited interest in teaching EBP in management schools
Management schools still a bit trapped: Cash cows for universities; focus on ‘top quality’ academic research; few incentives to be relevant
Future developments: academia
36
Postgraduate Course
Though issues around accountability, ethics, corporate social responsibility could in part be resolved by research and teaching EBP
Need for a professional doctorate?
Future developments: academia
In general
incentives to do non-EBP and punishment (or no incentives) for doing EBP
39
Challenges
(same for practitioners and academics)
Incentives for managers
Not rewarded for doing what ‘works’ – few evaluations
Speed and action valued more highly than accuracy and analysis
Managing and understanding power and politics to get things done more valued than understanding and using evidence to make decisions
It may be too late to change existing senior managers…
40
Barriers from academic / buss school context• In publishing and research
• In teaching content and teaching style
Barriers from organizational contexts• Managers love fads and quick fixes
• Power and politics
41
Other challenges
Ambivalence about the value and applicability of management research
Few incentives to get involved
Primary research (collecting new data) valued more highly than secondary research (reviewing existing data)
EBMgt not academics‘responsibility – this is about practice not research
Some concern that systematic reviews will expose the limited nature of management research
Some academics are like ‘gurus’ and feel that EBMgt might show their claims to be untrue
Why don’t academics like EBMgt?
Espoused and more implicit goals of management school educatorsESPOUSED GOALS To develop critical thinking To help students understand the
full body of knowledge To discuss gaps and limitations
of our knowledge To challenge thinking and
assumptions To educate To teach students how to think
for themselves To maintain quality standards
IMPLICIT GOALS To help students feel successful To select those bits of that are
interesting or digestible or ‘cutting edge’ fads
To reassure students that what we’re teaching them is solid
To make sure students are satisfied
To entertain (edutainment) Teach students what they need
to pass assessments To give ever-higher grades
44
ESPOUSED GOALS To advance scientific
understanding Using the best research
techniques Publishing all results and
replications – unbiased Focus on what’s important Being honest about existing
evidence To disseminate all our
evidence and make publically available
Collaboration & cooperation
IMPLICIT GOALS To advance career Use whatever techniques will
get you published Publishing (mostly) only
positive results, no replications Identifying ‘new’ or trendy
topics – creating empires Exaggerating how much we
know Locking up our evidence
behind publishers’ pay walls Competition for resources,
slots in journals, between universities 45
Espoused and more implicit goals of management school researchers
Implications for research
Less focus on collecting new primary data
More focus on systematic reviews and understanding what we know and don’t know
Focusing research efforts on what is academically and/or practically important without compromising ‘academic freedom’
Broader-based research training – management researchers highly and narrowly specialized
46
Undermines formal authority
They feel it constrains freedom to make managerial decisions
Speed valued and rewarded more than accuracy
Feel they cannot use their own experience and judgment (not true)
Managers not necessarily rewarded for doing what works (organizations rarely evaluate)
THEY LOVE FADS & QUICK FIXES
Why don’t managers like EBMgt?
Postgraduate Course
How are fads a problem?
“The main problem…is their lack of any solid
intellectual foundation. Implicit in each fad is a
cause effect statement that is rarely made
explicit and never properly supported.”
52(Donaldson & Hilmer, 1998)
QUICK FIXES
Because quick fixes / fads
Can be career-enhancing for managers
Speed is often valued over accuracy
Do we crave quick and easy solutions?
So who needs or wants academic research?
So why do managers love quick fixes?
Implications for management practice
Focus on accountability
Focus on next generation
Management = profession
EBP > Professional standard >Accreditation
57
Think about this
How can a profession / domain be evidence-based
if it is not managed in an evidence-based way?
58
“No job is more vital to our society than that of a manager. It is the manager who determines whether our social institutions serve us well or whether they squander our talents and resources.” Henry Mintzberg