measuring management practices€¦ · • middle east and north africa (with ebrd and eib) 2013/4...
TRANSCRIPT
![Page 1: MEASURING MANAGEMENT PRACTICES€¦ · • Middle East and North Africa (with EBRD and EIB) 2013/4 • In South America (2017, ongoing) • Upcoming: 44 countries in Europe & MENA,](https://reader034.vdocuments.us/reader034/viewer/2022051919/600b78a2160f3f0a7c2b1ed6/html5/thumbnails/1.jpg)
MEASURING MANAGEMENT PRACTICES:PRELIMINARY EVIDENCE FROM AN EXPERIMENT
Jorge Rodriguez MezaDavid C. Francis
Enterprise Analysis
![Page 2: MEASURING MANAGEMENT PRACTICES€¦ · • Middle East and North Africa (with EBRD and EIB) 2013/4 • In South America (2017, ongoing) • Upcoming: 44 countries in Europe & MENA,](https://reader034.vdocuments.us/reader034/viewer/2022051919/600b78a2160f3f0a7c2b1ed6/html5/thumbnails/2.jpg)
Overview
1
1. OVERVIEW OF THE ENTERPRISE SURVEY
2. MANAGEMENT PRACTICES IN THE ENTERPRISESURVEY
3. SOCIAL DESIRABILITY BIAS: experiment on reporting the use of manager bonuses
4. DISCUSSION
![Page 3: MEASURING MANAGEMENT PRACTICES€¦ · • Middle East and North Africa (with EBRD and EIB) 2013/4 • In South America (2017, ongoing) • Upcoming: 44 countries in Europe & MENA,](https://reader034.vdocuments.us/reader034/viewer/2022051919/600b78a2160f3f0a7c2b1ed6/html5/thumbnails/3.jpg)
Overview
2
1. OVERVIEW OF THE ENTERPRISE SURVEY
2. MANAGEMENT PRACTICES IN THE ENTERPRISESURVEY
3. SOCIAL DESIRABILITY BIAS: experiment on reporting the use of manager bonuses
4. DISCUSSION
![Page 4: MEASURING MANAGEMENT PRACTICES€¦ · • Middle East and North Africa (with EBRD and EIB) 2013/4 • In South America (2017, ongoing) • Upcoming: 44 countries in Europe & MENA,](https://reader034.vdocuments.us/reader034/viewer/2022051919/600b78a2160f3f0a7c2b1ed6/html5/thumbnails/4.jpg)
World Bank Enterprise Survey (ES)
3
• Global comparability: in ~130 countries of all incomelevels
• Measure data on: business environment + firmperformance + sensitive questions (on e.g. corruption)
• Administered face-to-face by trained enumerators
• Use of private contractors to implement fieldwork
• Use of visual tools (show cards)
• Standardized flow and question format
• Voluntary
![Page 5: MEASURING MANAGEMENT PRACTICES€¦ · • Middle East and North Africa (with EBRD and EIB) 2013/4 • In South America (2017, ongoing) • Upcoming: 44 countries in Europe & MENA,](https://reader034.vdocuments.us/reader034/viewer/2022051919/600b78a2160f3f0a7c2b1ed6/html5/thumbnails/5.jpg)
World Bank Enterprise Survey (ES), Cont’d
4
• Nationally representative
• Formal (non-agricultural, non-extractive) private sector
• 5+ employees
• Establishment level
• Stratified SRS design (size x sector x location x panel)
• Use of sampling weights
• Use of the most recent, accurate sampling frame
![Page 6: MEASURING MANAGEMENT PRACTICES€¦ · • Middle East and North Africa (with EBRD and EIB) 2013/4 • In South America (2017, ongoing) • Upcoming: 44 countries in Europe & MENA,](https://reader034.vdocuments.us/reader034/viewer/2022051919/600b78a2160f3f0a7c2b1ed6/html5/thumbnails/6.jpg)
Overview
5
1. OVERVIEW OF THE ENTERPRISE SURVEY
2. MANAGEMENT PRACTICES IN THE ENTERPRISESURVEY
3. SOCIAL DESIRABILITY BIAS: experiment on reporting the use of manager bonuses
4. DISCUSSION
![Page 7: MEASURING MANAGEMENT PRACTICES€¦ · • Middle East and North Africa (with EBRD and EIB) 2013/4 • In South America (2017, ongoing) • Upcoming: 44 countries in Europe & MENA,](https://reader034.vdocuments.us/reader034/viewer/2022051919/600b78a2160f3f0a7c2b1ed6/html5/thumbnails/7.jpg)
Inclusion of Management Practices in the ES
6
• Started from a relationship with MOI (EBRD) in 2008
• A sub-set of questions has been included in several regions to calculate a general management practices index
• ES (BEEPS with EBRD) in Europe and Central Asia 2012/3
• Middle East and North Africa (with EBRD and EIB) 2013/4
• In South America (2017, ongoing)
• Upcoming: 44 countries in Europe & MENA, Joint EBRD-EIB-WBG Enterprise Survey
• Upcoming: ES in selected countries in Africa
• TBD: testing to include (~11) variables necessary for management practices index calculation on in global module
![Page 8: MEASURING MANAGEMENT PRACTICES€¦ · • Middle East and North Africa (with EBRD and EIB) 2013/4 • In South America (2017, ongoing) • Upcoming: 44 countries in Europe & MENA,](https://reader034.vdocuments.us/reader034/viewer/2022051919/600b78a2160f3f0a7c2b1ed6/html5/thumbnails/8.jpg)
Inclusion of Management Practices in the ES
7
• Better management practices are generally correlated with greater productivity (EBRD Transition Report 2014)
• In MENA: poorly managed firms benefit more from improvements in management practices than from product innovation (EBRD-EIB-World Bank Group 2016)
![Page 9: MEASURING MANAGEMENT PRACTICES€¦ · • Middle East and North Africa (with EBRD and EIB) 2013/4 • In South America (2017, ongoing) • Upcoming: 44 countries in Europe & MENA,](https://reader034.vdocuments.us/reader034/viewer/2022051919/600b78a2160f3f0a7c2b1ed6/html5/thumbnails/9.jpg)
Inclusion of basic questions in the ES
8
TOPICS ON MANAGEMENT PRACTICES IN THE ES
1. Handling problems in the production processes (orprocess of providing services)
2. Monitoring performance indicators
3. Use of targets (term of focus, ability to achieve,awareness)
4. Incentives (bonuses, promotions, and dismissal)
![Page 10: MEASURING MANAGEMENT PRACTICES€¦ · • Middle East and North Africa (with EBRD and EIB) 2013/4 • In South America (2017, ongoing) • Upcoming: 44 countries in Europe & MENA,](https://reader034.vdocuments.us/reader034/viewer/2022051919/600b78a2160f3f0a7c2b1ed6/html5/thumbnails/10.jpg)
1. OVERVIEW OF THE ENTERPRISE SURVEY
2. MANAGEMENT PRACTICES IN THE ENTERPRISESURVEY
3. SOCIAL DESIRABILITY BIAS: experiment on reporting the use of manager bonuses
3. DISCUSSION
Overview
9
Q: Are there measurement effects (e.g. social desirabilitybias) to applying management practices questions to theES methodology?
![Page 11: MEASURING MANAGEMENT PRACTICES€¦ · • Middle East and North Africa (with EBRD and EIB) 2013/4 • In South America (2017, ongoing) • Upcoming: 44 countries in Europe & MENA,](https://reader034.vdocuments.us/reader034/viewer/2022051919/600b78a2160f3f0a7c2b1ed6/html5/thumbnails/11.jpg)
Social Desirability Bias
10
• Respondents are more (less) likely to report socially desirable (undesirable) behavior when interacting with an interviewer
• Response rates are known to be lower for self-administered (online or paper) surveys …
• … With the risk of losing representativeness
• This presents a trade-off: recovery of data on complex topics vs. interviewer effects (e.g. desirability bias)
![Page 12: MEASURING MANAGEMENT PRACTICES€¦ · • Middle East and North Africa (with EBRD and EIB) 2013/4 • In South America (2017, ongoing) • Upcoming: 44 countries in Europe & MENA,](https://reader034.vdocuments.us/reader034/viewer/2022051919/600b78a2160f3f0a7c2b1ed6/html5/thumbnails/12.jpg)
Social Desirability: Mode Effects
11
SOCIAL DESIRABILITY EFFECT
Online Face-to-face (F2F)CATI/phone
Less bias More bias
![Page 13: MEASURING MANAGEMENT PRACTICES€¦ · • Middle East and North Africa (with EBRD and EIB) 2013/4 • In South America (2017, ongoing) • Upcoming: 44 countries in Europe & MENA,](https://reader034.vdocuments.us/reader034/viewer/2022051919/600b78a2160f3f0a7c2b1ed6/html5/thumbnails/13.jpg)
An example for the holidays
12
Widmar et al. 2016
![Page 14: MEASURING MANAGEMENT PRACTICES€¦ · • Middle East and North Africa (with EBRD and EIB) 2013/4 • In South America (2017, ongoing) • Upcoming: 44 countries in Europe & MENA,](https://reader034.vdocuments.us/reader034/viewer/2022051919/600b78a2160f3f0a7c2b1ed6/html5/thumbnails/14.jpg)
Social Desirability: Innovation
13
• For business operators, some activities or topics are framed asdesirable (e.g. Cirera and Maloney, 2017)
• Example: innovation is framed as desirable for entrepreneurs andbusiness owners
• Bias may be exacerbated when topic is less understood
• Upshot: reinforces the tradeoff between complex topics andinterviewer effects
![Page 15: MEASURING MANAGEMENT PRACTICES€¦ · • Middle East and North Africa (with EBRD and EIB) 2013/4 • In South America (2017, ongoing) • Upcoming: 44 countries in Europe & MENA,](https://reader034.vdocuments.us/reader034/viewer/2022051919/600b78a2160f3f0a7c2b1ed6/html5/thumbnails/15.jpg)
Inclusion of basic questions in the ES
14
TOPICS ON MANAGEMENT PRACTICES IN THE LAC ES
Potential for social desirability:
• Performance bonuses (filter treatment in LAC)
• Production targets
• Monitoring of performance indicators
![Page 16: MEASURING MANAGEMENT PRACTICES€¦ · • Middle East and North Africa (with EBRD and EIB) 2013/4 • In South America (2017, ongoing) • Upcoming: 44 countries in Europe & MENA,](https://reader034.vdocuments.us/reader034/viewer/2022051919/600b78a2160f3f0a7c2b1ed6/html5/thumbnails/16.jpg)
LAC: experiment
15
• GOAL: test effect of treatment (filter question) on false positives ofmanagement activity (Type II error, use of managerial bonuses)
• GIVENS:• F2F enumeration (CAPI on tablet)• Include sub-set of questions to calculate management index• Use of filters throughout questionnaire• Use of show cards listing scale of desirable options
• IMPLEMENTATION:
• n ~4500• Seven South American countries• Fieldwork February – December 2017• Randomized treatment (not applied by strata) of Y/N filter question
![Page 17: MEASURING MANAGEMENT PRACTICES€¦ · • Middle East and North Africa (with EBRD and EIB) 2013/4 • In South America (2017, ongoing) • Upcoming: 44 countries in Europe & MENA,](https://reader034.vdocuments.us/reader034/viewer/2022051919/600b78a2160f3f0a7c2b1ed6/html5/thumbnails/17.jpg)
LAC: fieldwork
16
Country n WB Income Sampling Frame Yielda [Responseb]
Argentina* 1000 UMI Dun & Bradstreet Argentina 2010 y 2016 (N=46,688)
28.7% [62.9%]
Bolivia 360 LMI INEC, Censo Económico, updated 2016 (N=12,588)
40.3% [70.4%]
Colombia* 1000 UMI Confecamaras Colombia 2014/2016 (N=58,465)
15.9% [67.9%]
Ecuador 360 UMI Superintendencia de Compañías Valores y Seguros del Ecuador, 2016 (N=11,830)
22.5% [57.7%]
Paraguay 360 UMI Directorio General de Empresas y Establecimientos 2015 (N=5,624)
17.6% [76.8%]
Peru* 1000 UMI Registro MYPE, SUNAT (Hacienda), 2011 (N=15,720)
15.7% [46.8%]
Uruguay* 360 High Instituto Nacional de Estadística (INE),2015 (N=11,777)
10.4% [50.7%]
* Fieldwork ongoingUMI: Upper middle income a. Interviews per contacted firm/enterprise LMI: Lower middle income b. Interviews per confirmed eligible contact
![Page 18: MEASURING MANAGEMENT PRACTICES€¦ · • Middle East and North Africa (with EBRD and EIB) 2013/4 • In South America (2017, ongoing) • Upcoming: 44 countries in Europe & MENA,](https://reader034.vdocuments.us/reader034/viewer/2022051919/600b78a2160f3f0a7c2b1ed6/html5/thumbnails/18.jpg)
Implementation in MOPS
17
• Q11 is the comparable question • United States Census Bureau• Self-administered, mailed paper or electronic instrument• Stand-alone
![Page 19: MEASURING MANAGEMENT PRACTICES€¦ · • Middle East and North Africa (with EBRD and EIB) 2013/4 • In South America (2017, ongoing) • Upcoming: 44 countries in Europe & MENA,](https://reader034.vdocuments.us/reader034/viewer/2022051919/600b78a2160f3f0a7c2b1ed6/html5/thumbnails/19.jpg)
Implementation in LAC ES
18
CONTROL TREATMENT
![Page 20: MEASURING MANAGEMENT PRACTICES€¦ · • Middle East and North Africa (with EBRD and EIB) 2013/4 • In South America (2017, ongoing) • Upcoming: 44 countries in Europe & MENA,](https://reader034.vdocuments.us/reader034/viewer/2022051919/600b78a2160f3f0a7c2b1ed6/html5/thumbnails/20.jpg)
Results: treatment effect of filter
19
![Page 21: MEASURING MANAGEMENT PRACTICES€¦ · • Middle East and North Africa (with EBRD and EIB) 2013/4 • In South America (2017, ongoing) • Upcoming: 44 countries in Europe & MENA,](https://reader034.vdocuments.us/reader034/viewer/2022051919/600b78a2160f3f0a7c2b1ed6/html5/thumbnails/21.jpg)
Results: treatment effect of filter
20
![Page 22: MEASURING MANAGEMENT PRACTICES€¦ · • Middle East and North Africa (with EBRD and EIB) 2013/4 • In South America (2017, ongoing) • Upcoming: 44 countries in Europe & MENA,](https://reader034.vdocuments.us/reader034/viewer/2022051919/600b78a2160f3f0a7c2b1ed6/html5/thumbnails/22.jpg)
21
Results: treatment effect of filter
-.5-.4
-.3-.2
-.10
.1.2
Ave
rage
trea
tmen
t effe
ct
Poo
led
(20+
)
Poo
led
(50+
)
Uru
guay
(20+
)
Uru
guay
(50+
)
Arg
entin
a (2
0+)
Arg
entin
a (5
0+)
Col
ombi
a (2
0+)
Col
ombi
a (5
0+)
Ecu
ador
(20+
)
Ecu
ador
(50+
)
Bol
ivia
(20+
)
Bol
ivia
(50+
)
Per
u (2
0+)
Per
u (5
0+)
Par
agua
y (2
0+)
Par
agua
y (5
0+)
NOTE: 95% C.I. shown. Includes design-based and respondent controls, clustered S.E. by country-stratum-enumerator
On Probability of reporting use of bonusesEffect of the Use of Filter Question
![Page 23: MEASURING MANAGEMENT PRACTICES€¦ · • Middle East and North Africa (with EBRD and EIB) 2013/4 • In South America (2017, ongoing) • Upcoming: 44 countries in Europe & MENA,](https://reader034.vdocuments.us/reader034/viewer/2022051919/600b78a2160f3f0a7c2b1ed6/html5/thumbnails/23.jpg)
22
Results: treatment effect of filter
![Page 24: MEASURING MANAGEMENT PRACTICES€¦ · • Middle East and North Africa (with EBRD and EIB) 2013/4 • In South America (2017, ongoing) • Upcoming: 44 countries in Europe & MENA,](https://reader034.vdocuments.us/reader034/viewer/2022051919/600b78a2160f3f0a7c2b1ed6/html5/thumbnails/24.jpg)
23
Results: treatment effect of filter
![Page 25: MEASURING MANAGEMENT PRACTICES€¦ · • Middle East and North Africa (with EBRD and EIB) 2013/4 • In South America (2017, ongoing) • Upcoming: 44 countries in Europe & MENA,](https://reader034.vdocuments.us/reader034/viewer/2022051919/600b78a2160f3f0a7c2b1ed6/html5/thumbnails/25.jpg)
24
Results: treatment effect of filter
-.5-.4
-.3-.2
-.10
.1.2
Ave
rage
trea
tmen
t effe
ct
Poo
led
(20+
)
Poo
led
(50+
)
Ecu
ador
(20+
)
Ecu
ador
(50+
)
Arg
entin
a (2
0+)
Arg
entin
a (5
0+)
Uru
guay
(20+
)
Uru
guay
(50+
)
Col
ombi
a (2
0+)
Col
ombi
a (5
0+)
Bol
ivia
(20+
)
Bol
ivia
(50+
)
Per
u (2
0+)
Per
u (5
0+)
Par
agua
y (2
0+)
Par
agua
y (5
0+)
NOTE: 95% C.I. shown. Includes design-based and respondent controls, clustered S.E. by country-stratum-enumerator
On Probability of reporting use of bonusesEffect of the Use of Filter Question
MANUFACTURING
![Page 26: MEASURING MANAGEMENT PRACTICES€¦ · • Middle East and North Africa (with EBRD and EIB) 2013/4 • In South America (2017, ongoing) • Upcoming: 44 countries in Europe & MENA,](https://reader034.vdocuments.us/reader034/viewer/2022051919/600b78a2160f3f0a7c2b1ed6/html5/thumbnails/26.jpg)
25
Results: treatment effect of filter
![Page 27: MEASURING MANAGEMENT PRACTICES€¦ · • Middle East and North Africa (with EBRD and EIB) 2013/4 • In South America (2017, ongoing) • Upcoming: 44 countries in Europe & MENA,](https://reader034.vdocuments.us/reader034/viewer/2022051919/600b78a2160f3f0a7c2b1ed6/html5/thumbnails/27.jpg)
26
Results: treatment effect of filter
![Page 28: MEASURING MANAGEMENT PRACTICES€¦ · • Middle East and North Africa (with EBRD and EIB) 2013/4 • In South America (2017, ongoing) • Upcoming: 44 countries in Europe & MENA,](https://reader034.vdocuments.us/reader034/viewer/2022051919/600b78a2160f3f0a7c2b1ed6/html5/thumbnails/28.jpg)
27
Results: treatment effect of filterSERVICES
-.5-.4
-.3-.2
-.10
.1.2
Ave
rage
trea
tmen
t effe
ct
Poo
led
(20+
)
Poo
led
(50+
)
Uru
guay
(20+
)
Uru
guay
(50+
)
Arg
entin
a (2
0+)
Arg
entin
a (5
0+)
Col
ombi
a (2
0+)
Col
ombi
a (5
0+)
Per
u (2
0+)
Per
u (5
0+)
Bol
ivia
(20+
)
Bol
ivia
(50+
)
Ecu
ador
(20+
)
Ecu
ador
(50+
)
Par
agua
y (2
0+)
Par
agua
y (5
0+)
NOTE: 95% C.I. shown. Includes design-based and respondent controls, clustered S.E. by country-stratum-enumerator
On Probability of reporting use of bonusesEffect of the Use of Filter Question
![Page 29: MEASURING MANAGEMENT PRACTICES€¦ · • Middle East and North Africa (with EBRD and EIB) 2013/4 • In South America (2017, ongoing) • Upcoming: 44 countries in Europe & MENA,](https://reader034.vdocuments.us/reader034/viewer/2022051919/600b78a2160f3f0a7c2b1ed6/html5/thumbnails/29.jpg)
1. OVERVIEW OF THE ENTERPRISE SURVEY
2. SOCIAL DESIRABILITY BIAS: experiment on reporting the use of manager bonuses
3. DISCUSSION
Overview
28
![Page 30: MEASURING MANAGEMENT PRACTICES€¦ · • Middle East and North Africa (with EBRD and EIB) 2013/4 • In South America (2017, ongoing) • Upcoming: 44 countries in Europe & MENA,](https://reader034.vdocuments.us/reader034/viewer/2022051919/600b78a2160f3f0a7c2b1ed6/html5/thumbnails/30.jpg)
• Listing of certain management practices (e.g. use of bonuses) seems to imply desirable answers
• On its face: bias seems upward
• But, there may be downward effect (learning within the questionnaire)
• Need for broad comparability …
• … And feasible implementation
• Mode is given: are there similar effects using different modes?
• Conservative approach: application of filter
Discussion
29