finding proofs for description logic entailments in practice · 2020. 9. 15. · based on finding...

Christian Alrabbaa Stefan Borgwardt Patrick Koopmann Alisa Kovtunova

Finding Proofs for Description LogicEntailments in PracticeBased on �Finding Small Proofs for Description Logic Entailments�Theory and

Practice� (LPAR'20) // Explainable Logic-Based Knowledge Representation

(XLoKR 2020), September 14, 2020

Page 2: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

Description Logics and Ontologies

Syntax of DL ALC

Concepts: C ::= A | ¬C | C u C | C t C | ∃r .C | ∀r .CAxioms: α ::= C v C | C ≡ C

Description Logics� Well-established formalism for specifying terminological knowledge inOntologies

� Used for many large-scale ontologies– SNOMED CT: over 300,000 concepts– BioPortal: repository of bio-medical ontologies, currently hosting 889

ontologies defining 12,084,317 terms– MOWLCorp: ontologies obtained by web-crawling, containing 21,000

ontologies

� With increasing complexity of the ontology, understanding entailmentsbecomes both crucial and difficult

� One typical reasoning task is classification– compute all entailed axioms of form A v B– obtain concept hierarchy

Finding Proofs for Description Logic Entailments in PracticeTechnische Universität Dresden // Christian Alrabbaa, Stefan Borgwardt, Patrick Koopmann, Alisa KovtunovaExplainable Logic-Based Knowledge Representation (XLoKR 2020), September 14, 2020

Slide 1 of 25

Page 3: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

Syntax of DL ALC

� Used for many large-scale ontologies� With increasing complexity of the ontology, understanding entailmentsbecomes both crucial and difficult

Slide 1 of 25

Page 4: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

Syntax of DL ALC

� Used for many large-scale ontologies� With increasing complexity of the ontology, understanding entailmentsbecomes both crucial and difficult

Slide 1 of 25

Page 5: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

Current Tool of Choice: Justi�cations

Slide 2 of 25

Page 6: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

Slide 2 of 25

Page 7: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

Slide 2 of 25

Page 8: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

Slide 2 of 25

Page 9: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

Justi�cations

Justifications: Minimal subsets entailing given subsumption

In practice often insufficient :� can be large� inferences often not obvious

Showing how to obtain the inference would be better� simple reasoning steps leading to conclusion� generally known as proof

Slide 3 of 25

Page 10: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

Proofs for ELK in Protege

Slide 4 of 25

Page 11: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

Slide 4 of 25

Page 12: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

Slide 4 of 25

Page 13: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

Slide 4 of 25

Page 14: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

Slide 4 of 25

Page 15: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

Proofs using Evonne (work in progress)

C. Alrabbaa, F. Baader, R. Dachselt, T. Flemisch, P. Koopmann: Visualizing Proofs andthe Modular Structure for Ontology Repair, DL 2020.

Slide 5 of 25

Page 16: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

Proofs with ELK

� ELK using a consequence-based reasoning procedure⇒ inferences performed using a calculus

R0C v C

R>C v > R−u

C v D u E

C v D C v E

R+uC v D C v E

C v D u ER∃

C v ∃r .D D v E

C v ∃r .E

R⊥C v ∃r .D D v ⊥

C v ⊥ RvC v D

C v E: D v E ∈ O

R◦C0 v ∃r1.C1,C1 v ∃r2.C2 . . . Cn−1 v ∃rn.Cn

C0 v ∃r .Cn: r1 ◦ . . . ◦ rn v r ∈ O

⇒ proofs generated as part of reasoning process

Slide 6 of 25

Page 17: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

Proofs For More Expressive DLs

� Currently, ELK is the only DL reasoner supporting proof generation

� ELK supports only a limited fragment of OWL, OWL EL

� More expressive reasoner often use other reasoning procedures– for a long time prominent: tableau reasoning– less convenient for understanding entailments

� Existing consequence-based reasoning for expressive DLs– often involved in complex systems– often combined with other reasoning paradigms– may use normal forms requiring different syntax⇒ generation of proofs not obvious

� Can we generate proofs without a calculus?

Slide 7 of 25

Page 18: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

Justi�cation Based Proofs

Justification-Based Proofs (Matthew Horridge 2011)� derive intermediate steps between conclusion and justification� consider all axioms of some predefined shapes� justification-relation between allows to construct a proof� involved ranking function allows to select axioms to be used in proof

J

A v B

Slide 8 of 25

Page 19: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

J

A v B

Slide 8 of 25

Page 20: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

J

A v B

Slide 8 of 25

Page 21: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

J

A v B

Slide 8 of 25

Page 22: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

J

A v B

Slide 8 of 25

Page 23: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

J

A v B

Slide 8 of 25

Page 24: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

J

A v B

Slide 8 of 25

Page 25: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

J

A v B

Slide 8 of 25

Page 26: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

Advantage of approach:� generates best proof according to ranking

Disadvantage of approach:� no clear inference principle� hard to implement� strongly depends on ranking function

Slide 8 of 25

Page 27: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

Forgetting-Based Proofs

Forgetting Based Proofs

� Idea from propositional resolution: q1 ∨ p q2 ∨ ¬pp1 ∨ p2

⇒ inference through elimination of p

� Decide satisfiability by eliminating names one after the other:� Idea: Use similar approach to prove axioms of form A v B

Slide 9 of 25

Page 28: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

⇒ inference through elimination of p� Decide satisfiability by eliminating names one after the other:

b a ∨ b ¬b ∨ c ¬b ∨ ¬c ¬a ∨ c

� Idea: Use similar approach to prove axioms of form A v B

Slide 9 of 25

Page 29: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

b a ∨ b ¬b ∨ c ¬b ∨ ¬c ¬a ∨ c

Slide 9 of 25

Page 30: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

b ¬b ∨ c ¬b ∨ ¬c b ∨ c

Slide 9 of 25

Page 31: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

b ¬b ∨ c ¬b ∨ ¬c b ∨ c

Slide 9 of 25

Page 32: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

c ¬c

Slide 9 of 25

Page 33: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

c ¬c

Slide 9 of 25

Page 34: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

⊥

Slide 9 of 25

Page 35: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

⊥

Slide 9 of 25

Page 36: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

⊥

A v C C v ∃r .D ∃r .> v B

Slide 9 of 25

Page 37: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

⊥

A v C C v ∃r .D ∃r .> v B

Slide 9 of 25

Page 38: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

⊥

A v C C v ∃r .> ∃r .> v B

Slide 9 of 25

Page 39: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

⊥

A v C C v ∃r .> ∃r .> v B

Slide 9 of 25

Page 40: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

⊥

A v C C v B

Slide 9 of 25

Page 41: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

⊥

A v C C v B

Slide 9 of 25

Page 42: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

⊥

A v B

Slide 9 of 25

Page 43: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

Forgetting

DefinitionLet O be an ontology and X a predicate name. Then, O−X is a result offorgetting X iff� X does not occur in O−X� for every axiom α in which X does not occur, O |= α iff O−X |= α

⇒ strongest ontology without X entailed by O� which α to preserve also depends on underlying DL

Slide 10 of 25

Page 44: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

Forgetting based proofs

� Use forgetting to produce sequence of ontologiesO0 O1 O2 O3 O4

A v B

−X0 −X1 −X2 −X3

� In each step, recompute justification for A v B

� Finally, reconstruct proof using justifications

Slide 11 of 25

Page 45: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

A v B

−X0 −X1 −X2 −X3

Slide 11 of 25

Page 46: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

A v B

Slide 11 of 25

Page 47: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

A v B

� In each step, recompute justification for A v B� Finally, reconstruct proof using justifications, skipping steps if it makessense

Slide 11 of 25

Page 48: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

Forgetting Based Proof

A v C

C v ∃r .D(D)

C v ∃r .>(C )

A v ∃r .> ∃r .> v B(r )A v B

Slide 12 of 25

Page 49: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

Forgetting Order

Forgetting order, as well as selection of justification, affects proof� Forgetting D first:

A v C

C v ∃r .D(D)

C v ∃r .>(C )

A v ∃r .> ∃r .> v B(r )A v B

� Forgetting r first:

A v C

C v ∃r .D ∃r .> v B(r )C v B

(C )A v B

Slide 13 of 25

Page 50: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

Forgetting Order

To obtain nicer proofs practically, we process names using the followingheuristics:� role with non-trivial fillers last:– otherwise may hide most of inference:

A v ∃r .B A v ∀r .C t D B v ∃s.D C v ∀s.¬D(r )

A v D

� unnested names first– delay complex inferences

� less frequent names first– delay expensive forgetting operations– (also used by existing forgetting procedures)

Slide 14 of 25

Page 51: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

Evaluation

� Implemented approach in modular fashion– easy exchange of different forgetting procedures, provided they produce OWL

ontologies– easy comparison with proofs generated by ELK

– Dijkstra-based search to extract shortest proof– use 2 forgetting tools in the evaluation

– ALCH variant of LETHE 0.6– ALCOI variant of FAME 1.01

1there is a much improved version FAME 2.0, but it often creates ontologies outside of theOWL-standard

Slide 15 of 25

Page 52: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

Evaluation: Corpus

� Focus on proofs in ELH– to be able to compare with ELK– easier extraction of justification patterns (see below)

� Use ontologies from the OWL Reasoner Evaluation 2015, OWL ELClassification Track– well-balanced mix of ontologies from different repositories

� Extracted 1,573 justification patterns– all entailments of form A v B or A ≡ B– all justifications for these entailments– abstract away concept and role names– remove resulting duplicates

Slide 16 of 25

Page 53: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

Evaluation: Metrics

� Hypergraph-size– number of distinct axioms used in the proof

� Tree-size– sub-proofs count as often as they are used

� Justification Complexity– Matthew et al. 2013: “Toward cognitive support for OWL justifications”– attempt to measure cognitive complexity of justification– provides value for each proof step– we measured maximum and sum for each proof

Slide 17 of 25

Page 54: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

Evaluation: Proof size

0 30 60 90 1200

30

60

90

120

LETHE (114)

FAME(410)

Tree Size

0 30 60 90 1200

30

60

90

120

ELK (468)

FAME(87)

Tree Size

0 30 60 90 1200

30

60

90

120

ELK (1087)

LETH

E(142)

Tree Size

0 10 20 30 40 50 600

10

20

30

40

50

60

LETHE (100)

FAME(398)

Hypergraph Size

0 10 20 30 40 50 600

10

20

30

40

50

60

ELK (486)

FAME(48)

Hypergraph Size

0 10 20 30 40 50 600

10

20

30

40

50

60

ELK (1146)

LETH

E(54)

Hypergraph Size

Slide 18 of 25

Page 55: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

Evaluation: ELK vs. Forgetting-Based Proofs

Slide 19 of 25

Page 56: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

Evaluation: ELK Proof

Slide 20 of 25

Page 57: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

Evaluation: LETHE Proof

Slide 21 of 25

Page 58: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

Evaluation: FAME Proof

Slide 22 of 25

Page 59: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

Evaluation: Justi�cation Complexity

0K 5K 10K0K

5K

10K

LETHE (101)

FAME(462)

Just. complexity (sum)

0K 5K 10K0K

5K

10K

ELK (411)FAME(214)

0K 5K 10K0K

5K

10K

ELK (669)

LETH

E(603)

0 250 500 7500

250

500

750

LETHE (37)

FAME(107)

Just. complexity (max)

0 250 500 7500

250

500

750

ELK (374)

FAME(189)

0 250 500 7500

250

500

750

ELK (493)

LETH

E(719)

Slide 23 of 25

Page 60: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

Evaluation: Complexity of Inferences

� Both LETHE and FAME may use logical operators outside of EL

� Large number of distinct “inference rules” was used:– LETHE: 362 different rules– FAME: 281 different rules

Slide 24 of 25

Page 61: Finding Proofs for Description Logic Entailments in Practice · 2020. 9. 15. · Based on Finding Small Proofs for Description Logic Entailments Theory and Practice (LPAR'20) //Explainable

Conclusion

� New proof generation procedure based on forgetting� Generate proof by repeated use of forgetting and justification� Proofs for expressive DLs without calculus� Can sometimes even compete with ELK� Several possibilities to improve:– better heuristics on forgetting order or when to skip steps– dynamic selection of forgetting order– use learned “rules” to shorten proof computation times– integrate newer version of FAME

Slide 25 of 25

finding proofs for description logic entailments in practice · 2020. 9. 15. · based on finding...

Documents