perez.can experts.be.trusted

Can Experts be Trusted and what can be done about it?

Insights from the Biases and Heuristics Literature

Oren Perez, Bar Ilan University Faculty of

Law

oren.perez@biu.ac.il LSE-CARR, 14 October 2014

Motivation

O Experts play a critical role in the modern

regulatory system;

O Expert groups (Commission is advised by

about 1000 expert groups, which assemble

more than 30,000 experts);

O Regulatory Agencies - e.g., the European

Food Safety Authority (EFSA), the European

Securities and Markets Authority (ESMA) or

the European Integrated Pollution Prevention

and Control (IPPC) Bureau (EIPPCB)

2

Normative Background

O The objectivity ideal :

O "Experts should provide opinions which are

independent, regardless of the pressures of

litigation. In this context, a useful test of

‘independence’ is that the expert would express the

same opinion if given the same instructions by an

opposing party. Experts should not take it upon

themselves to promote the point of view of the

party instructing them or engage in the role of

advocates".

O UK Civil Justice Council. "Protocol for the Instruction of

Experts to give Evidence in Civil Claims." 2009

3

Can experts be Trusted (1)?

O Should I trust this guy?

4

So should experts be Trusted (2)?

O Misaligned incentives due to external

economic pressures (Conflict of Interests,

COI);

O cognitive failures.

O The key distinction between the two

categories of judgment failure is that the

latter can occur even in the absence of

COI

5

Current regulatory framework :

O Focus on conflict of interest:

O Register of Expert Groups

O the European Ombudsman investigation into

the Commission’s expert groups.

O Judicial review of expert advise (via

administrative law or tort law) is weak - strong

deference to expert opinion.

O informal rules of the scientific community -

vague + the normal checking mechanisms of

science are not applicable to the regulatory

context (e.g., peer review)

6

Changing the focus: Cognitive biases as a matter of regulatory concern

O Experts are not immune: their susceptibility

to bias originates in a fundamental human

tendency to couple Type 1 intuitive

processing and Type 2 analytical processing

in reasoning processes.

7

Key Cognitive Biases

O Motivated Reasoning:

O confirmation bias - "the seeking or

interpreting of evidence in ways that are

partial to existing beliefs, expectations, or

a hypothesis in hand"

O disconfirmation bias - people are unable

to ignore their prior beliefs when

processing counter-arguments or counter-

evidence

8

Examples of confirmation bias

O studies of diagnostic decision-making have

demonstrated that doctors are subject to

confirmation bias when they interpret evidence

(e.g., symptoms or lack of symptoms),

O "prosecutorial bias" - documented in the work of

forensic experts that work in laboratories located

in law enforcement agencies or prosecutors'

offices - and reflected in a selective treatment of

the evidence in ways that support the agenda of

the institution in which the laboratory is situated.

9

Hindsight and outcome biases

O Hindsight bias - finding out that an outcome

has occurred increases its perceived

likelihood.

O Outcome bias - the influence of outcome

knowledge upon evaluations of decision

quality and the potential responsibility or

culpability of the decision-maker to the

outcome.‏

10

The De-biasing Project (and nudging)

O De-biasing constitutes a more extensive

interventionist strategy than nudging.

O A nudge is "any aspect in the framing of a

decision problem that can affect people

decisions without changing economic

incentives" (changing, e.g., the way

information is presented or by changing

default rules).

11

De-biasing may go beyond Nudging

O E.g., by changing the agent's set of choices,

increasing the transaction costs associated

with a certain decision, or by requiring the

development of new institutional structures.

O In the context of expert-decision making the

literature on de-biasing has focused on ways

to shift cognitive processing from a System

1 mode of thinking (automatic, heuristic) to

a System 2 (controlled, rule-governed) mode

of thinking. 12

Debiasing techniques

(a) Introspective techniques, encouraging the

agent to self-reflect on his reasoning process,

revising it as needed (general bias awareness);

(b) Cognitive-forcing techniques - influencing the

decision-maker indirectly by changing some

features of the task or the decision-making

environment;

(c) Introducing deliberative elements into the

decision-making process - 'forcing' the expert to

consider and cope with other viewpoints.

13

Checklists (1)

14

Checklists (2)

15

Process-conditioning rules - Design of Automatic Teller Machines

16

Introspective, meta-cognitive techniques

17

The fallibility of introspective de-biasing

O State 1:{A, B, C, D};

O State 2: {Eva1{A, B, C, D}-> A, B, C, ~D};

O State 3: {Eva2{Eva1{A, B, C, D}-> A, B, C, ~D}-

>A, B, ~C, D};

O …

O (n) State n: {Evan-1{Eva n-2{ Eva n-3 …

18

Cognitive forcing techniques - some problems

O CFTs can generate new cognitive problems,

which potentially can undermine any

benefits achieved through the introduction

of the CFT.

19

The Design of ATMs

20

The pitfalls of checklists

21

E-deliberation and open-gov

O The Problem of Mass E-mails/mass petitions.

O https://petitions.whitehouse.gov/

22

From Theory to Policy

O The move from theory to policy is far from

trivial - requires differential approach ,

experimentation and realistic calibration of

our goals and expectations.

23

Some Examples

O Methodological guidelines - examples:

O GRADE system - Grading of Recommendations

Assessment, Development, and Evaluation;

O American Medical Association’s ("AMA") Guides to

the Evaluation of Permanent Impairment (Sixth

Edition, 2007)

O Forensic Science (Report of the House of Commons

Science and Technology Committee on Forensic Science

(Second Report of Session 2013–14)

O Challenges?

24

Encouraging perspective taking

O By creating a pluralistic decision-making

environment:

O revising the expert selection process

O IPCC Procedures for the Preparation, Review,

Acceptance, Adoption, Approval and

Publication of IPCC Reports,

O Blind selection and randomization

O Public deliberation (Regulation room;

regulation.gov)

25

The ultimate objective of the de-biasing project?

From cognitive sterility - to the creation of

reflexive and epistemologically complex

decision making structures.

26