a novel approach to multimedia ontology engineering for automated reasoning over audiovisual lod...
TRANSCRIPT
A Novel Approach to Multimedia Ontology Engineering for
Automated Reasoning over Audiovisual LOD Datasets
Leslie F. Sikos, Ph.D. Flinders University, Australia ACIIDS 2016, 14 March 2016
Đà Nẵng, Vietnam
Reasoning over Video
• Vocabularies? Ontologies?
• Formalism with Description Logics
• ABox+TBox+RBox+DL-Safe Ruleset
• VidOnt, the most expressive video
ontology to date
• Reasoning over audiovisual datasets
Automated Reasoning over Audiovisual LOD Datasets ACIIDS’16
Outline
Ontology EngineeringVocabularies? Ontologies?
Thesaurus
Controlled
vocabulary
Ontology
Knowledge base
Dataset
Automated Reasoning over Audiovisual LOD Datasets ACIIDS’16
Ontology Engineering
• Description Logics (DLs): formal
knowledge representation languages
(decidable fragments of FOL)
DLs are suitable for the expressive
formalization of multimedia contents and the
semantic refinement of video segmentation.
Formalism with Description Logics
Automated Reasoning over Audiovisual LOD Datasets ACIIDS’16
Ontology EngineeringFormalism with Description Logics
FOL DL OWL
unary predicate concept class
binary predicate role property
constant individual individual
formula with two
free variables
role
expression
property
expression
Automated Reasoning over Audiovisual LOD Datasets ACIIDS’16
Ontology EngineeringDescription Logics
Automated Reasoning over Audiovisual LOD Datasets ACIIDS’16
Ontology EngineeringAvailable Constructors
Symbol Constructors Example
ALC ⊑ ≡ ⊓ ⊔ ¬ ∀∃ ⊤ ⊥
cartoon ≡ animatedMovie
⊓ ¬(liveAction ⊔computerAnimation)
S
+ transitivity
partOf ◦ partOf ⊑ partOf
Automated Reasoning over Audiovisual LOD Datasets ACIIDS’16
ALC
Ontology EngineeringAvailable Constructors
Symbol Constructors Example
R Limited complex
role inclusion
axioms, reflexivity
and irreflexivity,
role disjointness
partOf ◦ starredIn ⊑starredIn
Automated Reasoning over Audiovisual LOD Datasets ACIIDS’16
Ontology EngineeringAvailable Constructors
Symbol Constructors Example
O Nominals AnimationStudios ≡ {
DreamWorks, Walt Disney
Animation Studios, Pixar }
I Inverse roles directedBy ≈ directorOf-
Automated Reasoning over Audiovisual LOD Datasets ACIIDS’16
Ontology EngineeringAvailable Constructors
Symbol Constructors Example
Q Qualified
cardinality
restrictions
Actor ⊑= 1hasBirthplace.⊤
Automated Reasoning over Audiovisual LOD Datasets ACIIDS’16
Ontology EngineeringAvailable Constructors
Symbol Example(D) runningTime(Zambezia, 83)
Automated Reasoning over Audiovisual LOD Datasets ACIIDS’16
Ontology Engineering
copyrightHolder ≡ Organization ⊔ Person
copyrightHolder(DreamWorks)
DreamWorks ≡ Organization ⊔ Person
Reasoning Potential ( )
Automated Reasoning over Audiovisual LOD Datasets ACIIDS’16
ALC
Ontology Engineering
co-starredWith(JohnWayne, ClintEastwood) ⊑ ⊥starredIn(ClintEastwood, Unforgiven)
starredIn(JohnWayne, Unforgiven) ⊑ ⊥
Reasoning Potential ( )
Automated Reasoning over Audiovisual LOD Datasets ACIIDS’16
SROIQ(D)
Ontology Engineering
basedOn ◦ basedOn ⊑ basedOn
basedOn(LastManStanding, AFistfulOfDollars) ⊑ ⊤basedOn(AFistfulOfDollars, Yojimbo) ⊑ ⊤
basedOn(LastManStanding, Yojimbo) ⊑ ⊤
Reasoning Potential ( )
Automated Reasoning over Audiovisual LOD Datasets ACIIDS’16
SROIQ(D)
Ontology Engineering
releasedIn.PaleRider(1985)
retiredIn.JohnWayne(1979)
swrl:greaterThan(?releaseDate, ?retiredIn)
→ (inverse starredIn)(?a, ?m)
starredIn(JohnWayne, PaleRider) = ⊥
Reasoning Potential (Rules)
Automated Reasoning over Audiovisual LOD Datasets ACIIDS’16
Ontology Engineering
Example. The Visual Descriptor Ontology
(VDO) was published as an “ontology for
multimedia reasoning” while it features very
limited description logic expressivity
(corresponding to ALH) and reasoning potential
Description Logic Expressivity
Automated Reasoning over Audiovisual LOD Datasets ACIIDS’16
Multimedia Ontologies
Ontology Language DL
LinkedMDB RDFS AL
LSCOM OWL AL
M3O OWL SHIQ(D)
COMM OWL SHOIN(D)
Limited DL Expressivity
Automated Reasoning over Audiovisual LOD Datasets ACIIDS’16
Ontology Language DL
LinkedMDB RDFS AL
LSCOM OWL AL
M3O OWL SHIQ(D)
COMM OWL SHOIN(D)
Multimedia OntologiesFull DL Expressivity
Ontology Language DL
VidOnt OWL 2 SROIQ(D)
http://vidont.org
Automated Reasoning over Audiovisual LOD Datasets ACIIDS’16
Ontology EngineeringTBox Axioms
DL Syntax Turtle Syntax
liveAction ⊑ Movie :liveAction rdfs:subClassOf
:Movie .
Narrator ≡ Lector :Narrator owl:equivalentClass
:Lector .
Automated Reasoning over Audiovisual LOD Datasets ACIIDS’16
Ontology EngineeringABox Axioms
DL Syntax Turtle Syntax
actor(JackieChan) :JackieChan a :actor .
房仕龍 ≈ JackieChan :房仕龍 owl:sameIndividualAs
:JackieChan .
Automated Reasoning over Audiovisual LOD Datasets ACIIDS’16
Bridging the Semantic GapThe Missing Piece
• T TBox: terminological knowledge
• AABox: assertional knowledge
Automated Reasoning over Audiovisual LOD Datasets ACIIDS’16
Bridging the Semantic GapThe Missing Piece
• T TBox: terminological knowledge
• AABox: assertional knowledge
• R RBox: complex role inclusion
axioms + transitive roles
Automated Reasoning over Audiovisual LOD Datasets ACIIDS’16
Ontology EngineeringRBox Axioms
DL Syntax Turtle Syntax
partOf ◦ starredIn ⊑starredIn
:starredIn
owl:propertyChainAxiom
(:partOf :starredIn) .
basedOn ◦ basedOn ⊑basedOn
:basedOn a
owl:TransitiveProperty .
Automated Reasoning over Audiovisual LOD Datasets ACIIDS’16
Ontology Engineering
[ rdf:type swrl:Imp ; swrl:body [ rdf:type
swrl:AtomList ; rdf:rest [ rdf:type swrl:AtomList ;
rdf:first [ rdf:type swrl:DatavaluedPropertyAtom ;
swrl:argument2 "1080p" ; swrl:propertyPredicate:
videoMode ; swrl:argument1 :m ] ; rdf:rest rdf:nil ]
DL-Safe Ruleset
Automated Reasoning over Audiovisual LOD Datasets ACIIDS’16
…
…
Multimedia Reasoning
Actor(a), Actor(b), Actor(c), Actor(d)
Movie(m), Sequel(s), partOf(m, s)
partOf ◦ starredIn ⊑ starredIn
starredIn(a, m), starredIn(b, m), starredIn(c, m),
starredIn(d, s)
starredIn(?x, m) → co-starredWith(?x, d)
RBox and Rule-Based Reasoning
Automated Reasoning over Audiovisual LOD Datasets ACIIDS’16
Multimedia Reasoning
New object property assertions:
co-starredWith(a, d), co-starredWith(b, d),
co-starredWith(c, d)
starredIn(a, s), starredIn(b, s), starredIn(c, s)
Automatically Inferred Assertions
Automated Reasoning over Audiovisual LOD Datasets ACIIDS’16
Multimedia ReasoningReasoning Complexity
Automated Reasoning over Audiovisual LOD Datasets ACIIDS’16
DL Worst-Case Combined Complexity
EXPTIME
EXPTIME
NEXPTIME
N2EXPTIME
ALC
SHIF
SHOIN
SROIQ
VidOnt’s Knowledge DomainsVideo Production
Knowledge-Driven Video Information Retrieval with LOD ESAIR’15
VidOnt’s Knowledge DomainsBroadcasting
Knowledge-Driven Video Information Retrieval with LOD ESAIR’15
Reasoning over Audiovisual DataConclusions
• VidOnt, the first SROIQ(D) video ontology
• Multimedia ontology
engineering best practices
• Automatically infer new statements
with reasoners from audiovisual
LOD datasets
Automated Reasoning over Audiovisual LOD Datasets ACIIDS’16
Thank you.
Questions?
More on Description Logics
• Sikos, L. F. Description Logics in Multimedia
Reasoning. Springer (forthcoming)
• Sikos, L. F. Mastering Structured
Data on the Semantic Web: From
HTML5 Microdata to Linked Open
Data. Apress, New York, USA, 2015