Improving in-house legal reasoning

Dr Andrew Stranieri

Navigate

Decision making in a legal office

• How good is it ? Difficult to measure. Indications are that it is not so good.

• How to improve decision making?

– Relevant, up to date knowledge at decision makers finger tips

– Reason in deliberating groups– Make explicit the reasoning: coalesce – Re-use reasoning from a similar problem

Navigate

How is reasoning represented?

• Pre 4th millinnium BC Caveman did reason. Reasoning was transmitted verbally. No permanent representation. Still the quickest, easiest way for in-house reasoning.

• Writing was invented for tax collectors to record trader’s arguments 4th millennium BC

• Classical syllogism of Aristotle 500BC attempt to systematically describe an individual’s reasoning. Syllogism led to logics but too cumbersome in practice

• Judgements in written narrative. Expressive, flexible but unstructured

• Represents reasoning by a group

– Argument Visualisation eg Toulmin, GoReason, IBIS, gIBIS, Compendium, GAAM

Navigate

Improving in-house reasoning with up to date knowledge at decision makers finger tips

• We established a Wiki based knowledge management system for State Revenue Office in Victoria

• Officers easily access and add to organisational knowledge• One of the largest and most heavily used internal wiki based

knowledge management (now 000's of pages)

Navigate

Improving in-house reasoning by deliberating in groups

• Deliberation – dialogue devoid of power plays, information imbalances

• Wright (2001,2005), Rowe and Wright (2005) found that decision making in groups leads to better forecast decisions.

• Delphi – forecasts made by separate individuals and then anonymously fed back to eachother – leads to better decisions

• Delphi – forecasts with reasons even better• Our (yet to be published) study aimed to demonstrate that

Delphi with highly structured reasons was even better however, football this was not the case

Navigate

Improving in-house reasoning with up to date knowledge at decision makers finger tips

• We established a Wiki based knowledge management system for State Revenue Office in Victoria

• Officers easily access and add to organisational knowledge• One of the largest and most heavily used internal wiki based

knowledge management (now 000's of pages)

Navigate

Reasoning community: process

DecisionDecision

Make decisionMake decision

CoalescingCoalescing

Individual ReasoningE ach perso n m akestheir o wn sense o f

the issues, co m es upwith their o wn views

CoalescingCoalescing

S o m eo ne sum m arisesthe issues, v iews that

everyo ne has andgenerates a co -o perative

pro duct

Deliberation

Engagem entD eterm ine partic ipants

D ecide ho w, when to decideD ecide ho w to co m m unicate

Gather inform ation

Reflect

Navigate

Engagement• Select participants

– Patient- doctor community (friend?)– Multi-disciplinary cancer team (surgeon, oncologist, CAM?)– Jury (semi-random)– Supportive care cancer community (?)– Water allocation eDemocracy community – Family law litigation (judge, husband, wife, others?)

• Decide how to decide– Patient – doctor community. Doctor– Multi-disciplinary cancer team – Consensus?– Jury. Unanimous– Supportive care community ?– Water allocation community. Vote– Family law litigation. Judge

Navigate

Individual reasoning

• Patient- doctor community. Patient self diagnoses. (Currently controversial – shouldn't be) Doctor ind. reasoning to a diagnoses.

• Multi-disciplinary cancer team. Little individual reasoning prior to the group meeting

• Jury. Each member forms a view while hearing evidence before group dialogue

• Supportive care cancer community. Patient self assesses.

• Water allocation community. Each participant made an allocation prior to meeting

• Family Law case. Husband, wife, judge reason individually to form their views before the case

Navigate

Coalescing

• Summarise what every participants has asserted, Pools evidence. Produces co-operative product (McMahan)

• Academic community. Survey article• Patient- doctor community. Not done• Multi-disciplinary cancer team. Not done. GICS

research project

• Jury. Not done

• Supportive care cancer community. Question bank

• Water allocation community. Done with argument tree

• Family Law. Judgement provides summary in narrative form

• State Revenue Office Wiki

Navigate

Coalescing with narrative

• Eg Survey article or legal judgement provides summary in narrative form

– Lots 'between the lines'– Can be vague, difficult to interpret– Hard to retrieve a similar decision, legal

information retrieval

Navigate

Coalescing with IBIS (Kuntz and Rittel)

Ho w sho uld A us tralia d e al with unautho rise d e ntry b y b o at p e o p le ?

ISSU E

P ro ce ss the m us ingthe P ac if ic so lutio n

P O SITIO N

L e t the m land and b ep ro ce sse d within theco m m unity P ro ce ss

the m us ing the P ac if icso lutio n

T urn the b o ats awayL e t the m land andp ro ce ss the m in

d e te ntio n ce ntre s

P O SITIO N P O SITIO N P O SITIO N

(+ ) relatively c ost effec tive(+ ) disc ourages people smuggling

(+ ) disc ourages queue jumping(-) against the spir it of the UN

c onvention(-) Living c onditions are often poor

Argum e nts P ro /C o n

Navigate

Coalescing with Argument Tree (Yearwood and Stranieri)

is o fAu stra lia nco n ce rn o r

SafetyT h e sa fe ty o f th e

b o a t p e o p le : ·

·

is n o t o fAu stra lia nco n ce rn :

·

It is imp o rta n t o r

IllegalActivities ··

fo r Au stra lia toma in ta in o rd e r ly

ch a n n e ls o f e n trya n d n o t e n co u ra g e

ille g a l a ve n u e s su cha s p e o p lesmu g g lin g .

1.

The most appropriate w ay to de alw ith the ‘Boat Pe op le ’ is :

The Pacific So lu tion o r

2. De tain the m on Australian te rr ito ry o r3. Turn the m aw ay o r

4. Place the m in the community w h istthe y are p roce sse d

1.

1.

2. is a nAu stra lia nco n ce rn :

1.2.

It is imp o rta n tIt is n o t imp o rta n t

It is imp o rta n t o r

Deten tion ·

·1.

2.

sh o u ld n o tb e d e ta in e d

sh o u ldb e

d e ta in e d

It is imp o rta n t o r

Human itar ian ·

·1.

2.it is imp o rta n tit is n o t imp o rta n t

fo r Au stra lia totre a t th e b o a tp e o p le in th e

s imila r ma n n e r a sAu stra lia n c itize n s

It is imp o rta n t o r

In ternationalOb ligations ·

·

fo r Au stra lia a ct ina cco rd a n ce with th e

sp ir it o f itso b lig a tio n s u n d e r

th e UN C o n ve n tio no n R e fu g e e s.

1.

2.

It is imp o rta n t

It is n o t imp o rta n t

It is imp o rta n t o r

AustSovereign ty ·

·1.

2.

Au stra liash o u ld h a veu n fe tte re d

r ig h ts top ro te ct its

b o rd e rsAu stra lia sh o u ldn o t h a ve

u n fe tte re dr ig h ts to p ro te ct

its b o rd e rs

It is imp o rta n t o r

Economicconsiderations ·

·1.

2.

Eco n o micco n sid e ra tio ns a re o f so me

imp o rta n ce

Eco n o micco n sid e ra tio n

s a re o f n oimp o rta n ce

Navigate

Coalescion with Delphi

• Individual reasoning by participants is summarised by a Delphi facilitator

• Summary (as coalesced product) presented back to each participant

• Individuals review their reasoning. Facilitator summarises again

• etc.

Navigate

Coalescion leads to better reasoning• Many studies using Delphi for forecasting find

better forecasts than face to face • Many argument visualisation studies claim more

learning, better decisions but most have methodological problems (Prakken)

• Our 2 studies with GAAM, IBIS, Narrative, showed little difference in quality of deliberation or quality of decision

Navigate

School study 1 Yearwood, Stranieri, Bateman Do argument schemes facilitate group reasoning and

deliberation Groups of Yr 9 students at two schools Issue selected Govt policy on refugee determination Students provided with background information In small groups, discussed their views. Degree of deliberation

rated by two raters

• Independent variables; Using Toulmin, Using GAAM, Using Narrative, Using IBIS

• Dependent variables: Students wrote an essay, sat a test and the quality of their group deliberation was assessed

Family law litigation

Navigate

Findings Argument schemas were easily learnt but require a lot of practice for familiarity

Groups that used schemas showed some tendency toward quality reasoning

Groups that had access to the schemas without the template found it quite difficult to fill in the template

Dialogue quality was quite similar for all groups though the Control group readily reverted to combative exchanges

Navigate

School study 2

Yearwood, Stranieri, Mays 2008 Do argument schemes facilitate group reasoning and

deliberation Groups of Yr 8,9,10 students at DSC Teachers taught each group an argument scheme Issue selected: Random Drug Checks at School In small groups, discussed their views. Videotaped. Degree of

deliberation rated by two raters

• Independent variables; Using Toulmin, Using GAAM, Using IBIS

• Dependent variables: Students wrote an essay, sat pre & post critical thinking test and the quality of their group deliberation was assessed

Navigate

Findings Again, little evidence that any argument scheme leads to improved reasoning or critical thinking

Also, dialogue quality was quite similar for all groups though the Control group and groups using less structured schemes more readily reverted to combative exchanges

Navigate

Footy study

Wright, Yearwood, Stranieri 2009 Do reasons facilitate forecasts in group deliberation

settings Adults Forecast 16 AFL football matches In small groups, made tips, discussed or saw summaries of

other members tips.

• Independent variables; Face to face, Delphi, Delphi with reasons, Delphi with structured reasons

• Dependent variables: Tip success, Times tip changed

Navigate

Water study

Yearwood, Stranieri, McRae-Williams, Greymore, Thoms 2010 (HMS/UB-Deakin)

Do schemes facilitate novices to make expert decisions Adults Allocate scarce water to competing uses

• Independent variables; Novice group, Expert group

• Dependent variables: Allocation, Reasons

Navigate

Coalescion leads to re-use

• Reasoning of a higher court can be re-used by a lower court because judges have access to the higher judgement. Promotes consistency, transparency

• In medicine, evidence based reasoning involves a practitioner having access to assertions about a condition (backed by studies). The practioners reasoning is never coalesced so re-use is difficult to achieve

Navigate

References

Yearwood and Stranieri 2010 (Eds) Technologies for Supporting Reasoning Communities and Collaborative Decision making. IGI Sept

Yearwood, J. & Stranieri, A. (2009), 'Group structured reasoning for coalescing group decisions', Group Decision and Negotiation, 1-29.Yearwood, J and Stranieri, A., (2009) Deliberative dscourse and reasoning from generic argument structures. AI and Society 23(3): 353-377