The temporal representation and reasoning of complex events

F. Mele, A. Sorgente

C.N.R. Istituto di Cibernetica,Via Campi Flegrei, 34 – Pozzuoli (Napoli) Italia

CILC – Pescara 31 August - 2 September 2011

1

Outline

• Event and complex event

• Formalism for representing event:– simple event

– complex event

• Temporal reasoning for complex events:– the analysis of consistency,

– the discovery of new temporal relations, and

– the causal reasoning.

2

Event and Complex Event

• An Event is an action that happens over time (The church of Santa Chiara was built between 1310 and 1340) or a property that is true in a specific time interval (The church of Santa Chiara was Gothic style until 1744).

• The Complex Event is defined through a set of events (components) and relations between events (During the Second World War Allied bombing caused a fire)

3

Ontology of Complex Events (OntoCE)

• The formalism has been constructed to represent the complex events in explicit form, with the main aim that such a representation can be used as an ontological reference for various types of semantic annotations such as– The aggregation of multimedia elements (photos, video or texts) whose

contents represent events (historical events, news, cultural events, etc.)

– The annotation and aggregation of complex events in natural language

• The formalism has the aim of applying axiomatized theory for: the analysis of consistency; the discover of new temporal relations; the casual reasoning.

• The formalism for representing the events is based on “W's and one H“

4

Top Class of OntoCE

• AnyThingInTime is an abstract classs and it is the superclass of all entities that happen over time

• AnyThingInTime is defined by slot: hasWhen, hasWhere, hasParticipnats

• Event and complex event inherit all the attributes of AnyThingInTime which are characterized by attribute WHAT, respectively

5

When

• It describes when an event happens or property is true

• It is defined by – the effective symbolic interval, in which the event happens, don’t have values

in concrete time domain (we use generic value like ti, tj)

– the temporal modality of happening is described by one (or more) temporal order relations (before, after, during, etc.). These relations have the aim of anchoring an event on the chronological axis

w1:When[hasSymbolicHappenInterval*->symt1,

hasTemporalMode*->trel1]

symt1:SymbolicHappenInterval[stf->t1,sti->t2].

time1:DateCalendar[year->1980].

trel1:DuringEI[hasEntity1-> symt1, hasEntity2->time1].

6

Taxonomy of (Simple) Event

An example of representation

ev1:OccurenceEvent[ hasWhat->occ1, hasParticipants->{po1,po2}, hasWhen->wn1,

hasWhere->wr1].

occ1:Occurence[name->‘bombing'].

po1:PhisicalObject[ name->‘Basilica of S. Chiara'].po2:PhisicalObject[name->‘warplane'].

symt1:SymbolicHappenInterval[ stf->t1,sti->t2].wn1:When[ hasSymbolicHappenInterval->sym11]..

wr1:Where[hasLocations->loc1].loc1:Location[name->‘Naples'].

ph1:Photo[hasCreationTime->cl1,url->bombing.gif'].msea1:MediaStoryElementAnnotation[

hasAgent->ant1,hasAnnotationTime->cl1a,hasEvent->ev1, hasMedia->ph1].

Taxonomy of complex events

9

Complex Events(I)

• Narrative Events: the concept NarrativeEvent is represented as a set of events and temporal relations between events. The characterization of NarrativeEvent is given by the restriction of the attribute hasEventRelations which can only have instances of temporal relations as a value.

• Mental Events: the concept MentalEvent represents mental events of an agent (participant at the event), i.e., beliefs, desires, and intentions that occur over time. Example: The church and monastery of Santa Chiara was built between 1310 and 1340 for desire of Roberto d'Angiò and Queen Sancha of Aragon

10

Complex Events(II)

• Causal Events: the concept CausalEvent describes type of events that are in a cause-effect relationship.

• The concept CausalEvent is specialized in:– PhysicalByMentalEvent: “I think it's a good book, I’ll buy it”, and “I

would like something hot, I'll take a cup of tea”;

– MentalEventByPhysical: “He laughed and I thought he was joking”;

– PhysicalEventByPhysical: “He bumped the glass with his elbow and broke it”, “It’s raining and the road is wet”; and,

– MentalEventByMental:“I think it's the best team and I think it will win the championship”.

11

An example of Causal Event

During the Second World War Allied bombing caused a fireocc1:OccurranceNoun[annotated->"bombing"].ev1:OccurenceEvent[ hasWhat->occ1,hasParticipants->{po1,po2}, hasWhen->wn1,hasWhere->wr1].

occ2:OccurranceNoun[annotated->"fire"].ev2:EventNoun[

hasWhat->occ2,hasParticipants->{po1}, hasWhen->wn2].

physbyphis1:PhysicalByPhysicalRelation[hasCause->ev1,hasEffect->ev2].

cev1:PhysicalEventByPhysical[hasComplexWhat->{ev1,ev2},hasEventRelations->physbyphis1,hasParticipants->{po1,po2}, hasWhen->wn3].

12

Temporal reasoning for complex events

• The presented formalization has the advantage of applying axiomatized theory for:

– the analysis of consistency;

– the discover of new temporal relations;

– the casual reasoning.

13

The analysis of consistency

• The analysis of consistency checks if there are inconsistencies in the annotated temporal relations (i.e. there is a relation After[A,B] and is possible to infer After[B,A])

• The Russell and Kramp language, to check consistencies is not expressive enough to check consistencies of our instance of temporal relations(i.e. the during relation cannot be defined only through the primitive precev and overev)

• A set of axioms have been defined, that extend the Russel and Kramp language, with the order relations between time instants: prect(T1,T2) and eqt(T1,T2)

14

Axioms for the consistency control

1: prect(T1,T3) <- prect(T1,T2) prect(T2,T3).2: eqt(T1,T2) <- eqt(T1,T2) eqt(T2,T3).

3: eqt(Tx,Tx).4: eqt(T1,T2) <- eqt(T2,T1).5: prect(T1,T3) <- prect(T1,T2) eqt(T2,T3).6: ¬prect(T2,T1) <- prect(T1,T2).7: ¬eqt(T1,T2) <- prect(T1,T2).

15

• Axioms 1-4 define the properties of order relations:• the transitivity of relation prect(T1,T2) and eqt(T1,T2),

• the reflexivity and symmetry of the relation eqt(T1,T2).

• Axiom 5 defines the relationship between prect(T1,T2) and eqt(T1,T2): the substitution rule

• Axioms 6-7 define the exclusivity of the relationship

Implementation of consistency control axioms

16

prect(T1,T2) :- prectD(T1,T2), not prectN(T1,T2).prectN(T1,T2):- time(T1), time(T2), prect(T2,T1).prect(T1,T3) :- prect(T1,T2), prect(T2,T3),not prectN(T1,T3), time(T1),time(T2), time(T3).prectN(T1,T2):- time(T1), time(T2), eqt(T1,T2).eqt(T1,T2) :- eqtD(T1,T2), not eqtN(T1,T2).eqt(T1,T3) :- eqt(T1,T2), eqt(T2,T3), not eqtN(T1,T3), time(T1), time(T2),time(T3).eqt(T1,T1) :- time(T1).eqt(T2,T1) :- eqt(T1,T2), time(T1), time(T2).eqtN(T1,T2) :- time(T1), time(T2), prect(T1,T2).eqtN(T1,T2) :- time(T1), time(T2), prect(T2,T1).prect(T1,T3) :- prect(T1,T2), eqt(T2,T3),not prectN(T1,T3),

time(T1), time(T2),time(T3).

• A logic program for the axioms 1-7, through stable model semantics by using the SModels system [SYR], has been implemented:

Transformation of temporal relations in the primitive prect, eqt

17

BeforeEE3[E1,E2] <-> dt(E1,T1,T2), dt(E2,T3,T4),prect(T2,T3).

AfterEE[E1,E2] <-> BeforeEE[E2,E1].

MeetsEE[E1,E2] <-> dt(E1,T1,T2), dt(E2,T3,T4), eqt(T2,T3).

…

The discovery of new temporal relations

• The discovery of new temporal relations tries to infer new temporal relations starting by the annotated relations

• This approach uses the results of the process of the consistency check and the equivalence relation between the temporal relations and order relations defined on time points.

• It verifies the existence of conditions to discover a specific relation (i.e. BeforeEE relation) in the stable model

• A rule for each temporal relation (before, after, meets, etc.) has been defined.

18

Examples of a rules

1 findRel(E1,E2,SM) :- 2 not BeforeEE[E1,E2], 3 dt(E1,T1,T2), dt(E2,T3,T4), 4 subset({prect(T1,T2),prect(T3,T4),prect(T2,T3)},SM),

5 insert{BeforeEE[E1,E2]}.

• The rule checks if there already exists a BeforeEE relation between two events E1 and E2 (2),

• It reads the end points of occurrence intervals of events E1and E2 (3),

• Verify the existence of conditions to discover a BeforeEE relation (4),

• The rule asserts the relation discovered in the knowledge base (5).

19

Causal reasoning - axiomatization

A variant axiomatization defined by Bocham

20

An example of causal reasoning

21