ILP-based Reasoning for Weighted Abduction

Naoya Inoue, Kentaro Inui!Tohoku University

Introduction u Goal: Plan recognition from natural

language texts!

u Adopt abduction-based framework!– Hobbs et al. (93)ʼs Weighted abduction!

u No tools available for large-scale problem!

2

u Experiments with Mini-TACITUS (Mulkar-Mehta 07) on Ng & Mooney (92)ʼs story understanding dataset!– How many problems were solved in 10

minutes? How much did it take?!

Scalability Problem

3

shopping

go-to-store

rich

robbing

poor

42.0 % (21/50) 63.7 sec.

28.0 % (14/50) 30.3 sec.

28.0 % (14/50) 30.3 sec. capitalist homeless

100.0 % (50/50) 0.7 sec.

100.0 % (50/50) 0.4 sec.

100.0 % (50/50) 0.1 sec.

ILP-based Reasoning

þ Introduction!¨ Weighted Abduction!¨ ILP-based Reasoning!¨ Evaluation!

4

Hobbs+ (93)ʼs Weighted Abduction u Abduction-based framework of

natural language understanding!u  “Interpreting sentences is to prove the logical

forms of the sentences.”!– Merging redundancies where possible!– Making assumptions where necessary!

u  Important features!– Best explanation is selected by assumability costs!– Evaluating both likelihood and specificity

appropriateness of the best explanation!5

Abduction u  Inference to the best explanation!

u Formally,!

6

robbing ⇒ go-to-store shopping ⇒ go-to-store “go-to-store” is observed.

Background knowledge: B Observations: O

Hypothesis: H such that!Given Find

B, O, H: sets of logical formulae

H ∪B |= O

H ∪B �|=⊥

shopping

go-to-store

robbing

Itʼs necessary to quantify!(i)  likelihood of H (ii)  appropriateness of

specificity of H

robbing ⇒ go-to-store shopping⇒ go-to-store robbing ⇒ get-gun

Scheme of Weighted Abduction

7

1.4

shopping

go-to-store

robbing

get-gun

robbing

$10 $10

1.2

1.6

$12 $14 $16

UNIFICATION: smaller cost is taken

BACKCHAINED:cost is propagated

OBSERVATION:assumability cost is assigned

AXIOM:assumability weight is assigned

Hypothesis: H Background knowledge: B robbing1.2 → get-gun robbing1.5 → go-to-store hunting1.1 → get-gun shopping1.4 → go-to-store poor1.3 → robbing

8 Observations: O get-gun$10 go-to-store$10

Background knowledge: B robbing1.2 → get-gun robbing1.5 → go-to-store hunting1.1 → get-gun shopping1.4 → go-to-store poor1.3 → robbing

Hypothesis: H {hunting$11, shopping$14}

hunting$11 shopping$14 robbing$12

Hypothesis: H {hunting$11, shopping$14} {robbing$12}

Background knowledge: B robbing1.2 → get-gun robbing1.5 → go-to-store hunting1.1 → get-gun shopping1.4 → go-to-store poor1.3 → robbing

Hypothesis: H {hunting$11, shopping$14} {robbing$12} {poor$15.6}

Background knowledge: B robbing1.2 → get-gun robbing1.5 → go-to-store hunting1.1 → get-gun shopping1.4 → go-to-store poor1.3 → robbing

poor$15.6

Hypothesis: H Background knowledge: B robbing1.2 → get-gun robbing1.5 → go-to-store hunting1.1 → get-gun shopping1.4 → go-to-store poor1.3 → robbing

9 Observations: O get-gun$10 go-to-store$10

Background knowledge: B robbing1.2 → get-gun robbing1.5 → go-to-store hunting1.1 → get-gun shopping1.4 → go-to-store poor1.3 → robbing

Hypothesis: H {hunting$11, shopping$14} Hypothesis: H {hunting$11, shopping$14} {robbing$12}

Background knowledge: B robbing1.2 → get-gun robbing1.5 → go-to-store hunting1.1 → get-gun shopping1.4 → go-to-store poor1.3 → robbing

Hypothesis: H {hunting$11, shopping$14} {robbing$12} {poor$15.6} :

Background knowledge: B robbing1.2 → get-gun robbing1.5 → go-to-store hunting1.1 → get-gun shopping1.4 → go-to-store poor1.3 → robbing

- Best explanation is least-cost explanation#- Appropriate specificity is selected Implementation Issue:#

The combinatorial explosion of!candidate hypotheses.!

10

get-gun$10 go-to-store$10

hunting$11 shopping$14 robbing$12

poor$15.6

u  Assign 0-1 ILP variables over all literals potentially included in the best hypothesis for representing candidate hypotheses!

u  Cost of hypothesis is represented as the sum of variables!

u  There is efficient algorithms to find the optimal assignment of variables

Key ideas:

H1 = {robbing$12} H2 = {poor$15.6} H3 = {hunting$11, shopping$14} :

Naive approach

Our solution: ILP-based Reasoning

hget-gun hhunting hrobbing, ... rget-gun rhunting rrobbing, ... urobbing1,robbing2 ...

C(H) = 10・hget-gun, + 11・hhunting, ...

arg mini

C(Hi)

C(Hi) =�

h∈Hi

c(h)

arg minhget−gun,hhunting,...

C(H)

: hgs

: hs : hr2 : hr1

: hgg

: hh

u  P: set of literals potentially included in hypothesis!u  hp: 1 if literal p is included in hypothesis!

go-to-store$10 get-gun$10

shopping$14 robbing$12 robbing$15 hunting$11

ILP formulation (h → r → u)

11 P = {get-gun, go-to-store, hunting, robbing1, ...}

= 1

= 1 = 0 = 0

= 1

= 1

H = {hunting$11, shopping$14}

arg minh

�

p∈{p|p∈P,hp=1}

c(p)

= 1

= 0 = 1 = 1

= 1

= 0

H = {robbing$12}

u  P: set of literals potentially included in hypothesis!u  hp: 1 if literal p is included in hypothesis!u  rp: 1 if literal p doesnʼt pay its cost!

ILP formulation (h → r → u)

12

arg minh,r

�

p∈{p|p∈P,hp=1,rp=0}

c(p)

: hgs, rgs

: hs, rs : hr2, rr2 : hr1, rr1

: hgg, rgg

: hh, rh

go-to-store$10 get-gun$10

shopping$14 robbing$12 robbing$15 hunting$11

P = {get-gun, go-to-store, hunting, robbing1, ...}

= 1, 1

= 0, 0 = 1, 0 = 1, 0

= 1, 1

= 0, 0

H = {robbing$12}

u  P: set of literals potentially included in hypothesis!u  hp: 1 if literal p is included in hypothesis!u  rp: 1 if literal p doesnʼt pay its cost!u  up, q: 1 if literal p is unified with literal q!

ILP formulation (h → r → u)

13

ur1,r2 = 1

arg minh,r

�

p∈{p|p∈P,hp=1,rp=0}

c(p)

: hgs, rgs

: hs, rs : hr2, rr2 : hr1, rr1

: hgg, rgg

: hh, rh

go-to-store$10 get-gun$10

shopping$14 robbing$12 robbing$15 hunting$11

P = {get-gun, go-to-store, hunting, robbing1, ...}

= 1, 1

= 0, 0 = 1, 0 = 1, 0

= 1, 1

= 0, 0

H = {robbing$12}

ILP Constraints

14

go-to-store$10

robbing$12

poor$15.6

get-gun$10

robbing$15

poor$19.5

: hgs, rgs

: hr2, rr2

: hp2, rp2 : hp1, rp1

: hr1, rr1

: hgg, rgg

up1,p2

ur1,r2

Literals do not pay cost (r=1) only if they are!(i) explained by another literal (h=1), or!(ii) unified with another literal (u=1) of lesser cost

rr2 ≤ hp2 + ur1,r2

rgg ≤ hr1

hgs = 1 Literals in observations must be!included in a hypothesis

2*up1,p2 ≤ hp1+hp2

Literals can be unified (u=1) only if they are hypothesized (h=1)

Evaluation u How scalable is our approach?!– Plotted (depth, inference time)!•  The depth limit of back-chaining!•  Inference time averaged for all problems!

u Dataset!– 50 problems in Ng & Mooney (92)ʼs story

understanding corpus!

15

Results

u  The increase of inference time is not exponential to the number of candidate hypotheses!

Ø  Indicates the efficiency of our approach! 16

0!

20!

40!

60!

80!

100!

120!

140!

160!

180!

d=1! d=2! d=3! d=4! d=5!

Aver

aged

num

ber o

f! p

oten

tial e

lem

enta

l hyp

othe

ses

Depth parameter d

Figure 4: The complexity of each problem setting

a conjunction of literals. For example, in the problem t2, thefollowing observation literals are provided:(1) inst(get2, getting) ∧ agent get(get2, bob2) ∧

name(bob2, bob) ∧ patient get(get2, gun2) ∧inst(gun2, gun) ∧ precede(get2, getoff2) ∧inst(getoff2, getting off) ∧agent get off(getoff2, bob2) ∧patient get off(getoff2, bus2) ∧ inst(bus2, bus) ∧place get off(getoff2, ls2) ∧ inst(ls2, liquor store)

This logical form denotes a natural language sentence “Bob

got a gun. He got off the bus at the liquor store.” The planrecognition system requires to infer Bob’s plan from theseobservations using background knowledge. The backgroundknowledge base contains Horn-clause axioms such as:(2) inst(R, robbing) ∧ get weapon step(R,G) ⇒

inst(G, getting)

From this dataset, we created two types of testsets: (i)testset A: Ng and Mooney’s original dataset, (ii) testset B:a modified version of testset A. For both testsets, we as-signed uniform weights to antecedents in background ax-ioms so that the sum of those equals 1, and assigned $20to each observation literal. We created testset B so thatthe background knowledge base does not contain a con-stant in its arguments since Mini-TACITUS does not al-low us to use constants in background knowledge axiom.Specifically, we converted the predicate inst(X,Y ) that de-notes X is a instance of Y into a form of inst Y (X) (e.g.,inst(get2, getting) is converted into inst getting(get2)). We also converted an axiom involving a constant inits arguments into neo-Davidsonian style. For example,occupation(A, busdriver), where busdriver is a constant,is converted to busdriver(X) ∧ occupation(A,X). Thesetwo conversions did not affect the complexity of the prob-lems substantially.

5.2 Results and discussionWe first show the complexity of abduction problems in

our testset A. Figure 4 shows the number of potential el-emental hypotheses, P described in Section 4.2, averagedfor 50 problems. As d increases, the number of elementalhypotheses increases constantly. Recall that the number of

0!

500!

1000!

1500!

2000!

2500!

d=1! d=2! d=3! d=4! d=5!

Aver

aged

num

ber o

f!va

riabl

es/c

onst

rain

ts

Depth parameter d

variables!constraints!

Figure 5: The complexity of each ILP problem

!"!!!#

!"$!!#

!"%!!#

!"&!!#

!"'!!#

("!!!#

("$!!#

("%!!#

("&!!#

("'!!#

$"!!!#

)*(# )*$# )*+# )*%# )*,#

!"#$%$"&$'()

$'*+$&,"

-+.

/$012'03%3)$1$%'-

-./#

0120312#

Figure 6: The efficiency of the ILP-based formulation“prepare” and “ILP” denote the time required to convert aweighted abduction problem to ILP problem, and the time

required to solve the ILP problem respectively.

candidate hypotheses is O(2n), where n is the number ofpotential elemental hypotheses. Therefore, in our testset, weroughly have 2160 ≈ 1.46 · 1048 candidate hypotheses for apropositional case if we set d = 5. Figure 5 illustrates thenumber of variables and constraints of a ILP problem foreach parameter d, averaged for 50 problems. Although thecomplexity of the ILP problem increases, we can rely on anefficient algorithm to solve a complex ILP problem.

The results of inference time in our framework on test-

set A is given in Figure 6 in the two distinct measures: (i)the time of conversion to ILP problem, and (ii) the timeILP technique had took to find an optimal solution. Figure 6demonstrates that our framework is capable of coping withlarger scale problems, since the inference can be performedin polynomial time to the size of the problem.

Then we show the inference time of Mini-TACITUS ontestset B. The complexity of the testset was quite similarto the testset A since the modification affecting the origi-nal complexity occurred in only 2 axioms. On testset B, wehave confirmed that our framework had solved the 100%of the problems for 1 ≤ d ≤ 5, and it took 1.16 secondswhen averaged for the 50 problems of d = 5. The results

0!

20!

40!

60!

80!

100!

120!

140!

160!

180!

d=1! d=2! d=3! d=4! d=5!

Aver

aged

num

ber o

f! p

oten

tial e

lem

enta

l hyp

othe

ses

Depth parameter d

Figure 4: The complexity of each problem setting

a conjunction of literals. For example, in the problem t2, thefollowing observation literals are provided:(1) inst(get2, getting) ∧ agent get(get2, bob2) ∧

name(bob2, bob) ∧ patient get(get2, gun2) ∧inst(gun2, gun) ∧ precede(get2, getoff2) ∧inst(getoff2, getting off) ∧agent get off(getoff2, bob2) ∧patient get off(getoff2, bus2) ∧ inst(bus2, bus) ∧place get off(getoff2, ls2) ∧ inst(ls2, liquor store)

This logical form denotes a natural language sentence “Bob

got a gun. He got off the bus at the liquor store.” The planrecognition system requires to infer Bob’s plan from theseobservations using background knowledge. The backgroundknowledge base contains Horn-clause axioms such as:(2) inst(R, robbing) ∧ get weapon step(R,G) ⇒

inst(G, getting)

From this dataset, we created two types of testsets: (i)testset A: Ng and Mooney’s original dataset, (ii) testset B:a modified version of testset A. For both testsets, we as-signed uniform weights to antecedents in background ax-ioms so that the sum of those equals 1, and assigned $20to each observation literal. We created testset B so thatthe background knowledge base does not contain a con-stant in its arguments since Mini-TACITUS does not al-low us to use constants in background knowledge axiom.Specifically, we converted the predicate inst(X,Y ) that de-notes X is a instance of Y into a form of inst Y (X) (e.g.,inst(get2, getting) is converted into inst getting(get2)). We also converted an axiom involving a constant inits arguments into neo-Davidsonian style. For example,occupation(A, busdriver), where busdriver is a constant,is converted to busdriver(X) ∧ occupation(A,X). Thesetwo conversions did not affect the complexity of the prob-lems substantially.

5.2 Results and discussionWe first show the complexity of abduction problems in

our testset A. Figure 4 shows the number of potential el-emental hypotheses, P described in Section 4.2, averagedfor 50 problems. As d increases, the number of elementalhypotheses increases constantly. Recall that the number of

0!

500!

1000!

1500!

2000!

2500!

d=1! d=2! d=3! d=4! d=5!

Aver

aged

num

ber o

f!va

riabl

es/c

onst

rain

ts

Depth parameter d

variables!constraints!

Figure 5: The complexity of each ILP problem

!"!!!#

!"$!!#

!"%!!#

!"&!!#

!"'!!#

("!!!#

("$!!#

("%!!#

("&!!#

("'!!#

$"!!!#

)*(# )*$# )*+# )*%# )*,#

!"#$%$"&$'()

$'*+$&,"

-+.

/$012'03%3)$1$%'-

-./#

0120312#

Figure 6: The efficiency of the ILP-based formulation“prepare” and “ILP” denote the time required to convert aweighted abduction problem to ILP problem, and the time

required to solve the ILP problem respectively.

candidate hypotheses is O(2n), where n is the number ofpotential elemental hypotheses. Therefore, in our testset, weroughly have 2160 ≈ 1.46 · 1048 candidate hypotheses for apropositional case if we set d = 5. Figure 5 illustrates thenumber of variables and constraints of a ILP problem foreach parameter d, averaged for 50 problems. Although thecomplexity of the ILP problem increases, we can rely on anefficient algorithm to solve a complex ILP problem.

The results of inference time in our framework on test-

set A is given in Figure 6 in the two distinct measures: (i)the time of conversion to ILP problem, and (ii) the timeILP technique had took to find an optimal solution. Figure 6demonstrates that our framework is capable of coping withlarger scale problems, since the inference can be performedin polynomial time to the size of the problem.

Then we show the inference time of Mini-TACITUS ontestset B. The complexity of the testset was quite similarto the testset A since the modification affecting the origi-nal complexity occurred in only 2 axioms. On testset B, wehave confirmed that our framework had solved the 100%of the problems for 1 ≤ d ≤ 5, and it took 1.16 secondswhen averaged for the 50 problems of d = 5. The results

257 295 2115 2136

2157

Summary u Addressed the issue of scalability for

abductive reasoning!u Proposed ILP-based approach to Hobbs

et al. (93)ʼs weighted abduction!u Results of our experiments showed that:!– our approach efficiently finds the best

explanation!u Future work!– Exploring the semantics of weights, costs!– To handle negation in ILP-based approach

17 THANK YOU!