semantic reasoning and verification for ambient assisted living · 2014-05-14 · semantic...
TRANSCRIPT
![Page 1: Semantic Reasoning and Verification for Ambient Assisted Living · 2014-05-14 · Semantic Reasoning and Verification for Ambient Assisted Living Thibaut Tiberghien, Mounir Mokhtari,](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8db8c75bef1f1f66403a77/html5/thumbnails/1.jpg)
Semantic Reasoning and Verification for Ambient Assisted Living
Thibaut Tiberghien, Mounir Mokhtari, Liu Yan & Dong Jin Song !!
FSFMA 2014 2nd French Singaporean Workshop on Formal Methods and Applications
13 May 2014, Singapore
![Page 2: Semantic Reasoning and Verification for Ambient Assisted Living · 2014-05-14 · Semantic Reasoning and Verification for Ambient Assisted Living Thibaut Tiberghien, Mounir Mokhtari,](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8db8c75bef1f1f66403a77/html5/thumbnails/2.jpg)
OutlineI. Introduction
II. Context reasoning
- Reasoning strategies
- Reasoning architecture
III. Validation & Verification
IV.Conclusion
!2
![Page 3: Semantic Reasoning and Verification for Ambient Assisted Living · 2014-05-14 · Semantic Reasoning and Verification for Ambient Assisted Living Thibaut Tiberghien, Mounir Mokhtari,](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8db8c75bef1f1f66403a77/html5/thumbnails/3.jpg)
- Major demographic changes *
Population ageing
2010
10
20
30
40
50
60
70
80
over 85Males Females
0
2050
10
20
30
40
50
60
70
80
over 85Males Females
00% 2% 4% 6%2%4%6% 0% 2% 4% 6%2%4%6%
Age Age
!3* United Nations, World Population Prospects: The 2010 Revision. http://esa.un.org/unpd/ wpp/index.htm
![Page 4: Semantic Reasoning and Verification for Ambient Assisted Living · 2014-05-14 · Semantic Reasoning and Verification for Ambient Assisted Living Thibaut Tiberghien, Mounir Mokhtari,](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8db8c75bef1f1f66403a77/html5/thumbnails/4.jpg)
- Elderly support ratio is shrinking
- e.g. in Singapore *
Population ageing
From 13.5 in 1970 to 2.6 in 2015!
!4* Resident Old-Age Support Ratio. Department of Statistics Singapore. http://www.singstat.gov.sg
![Page 5: Semantic Reasoning and Verification for Ambient Assisted Living · 2014-05-14 · Semantic Reasoning and Verification for Ambient Assisted Living Thibaut Tiberghien, Mounir Mokhtari,](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8db8c75bef1f1f66403a77/html5/thumbnails/5.jpg)
Progression of dementia- Projections of frail elderly population in Singapore *
!5* Dr. Philip Yap, Khoo Teck Puat Hospital, Singapore
Num
ber o
f fra
il el
derly
5000
15000
25000
35000
45000
2000 2010 2020 2030
7 000 beds available in nursing homes Need in 2015: 22 000 beds
![Page 6: Semantic Reasoning and Verification for Ambient Assisted Living · 2014-05-14 · Semantic Reasoning and Verification for Ambient Assisted Living Thibaut Tiberghien, Mounir Mokhtari,](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8db8c75bef1f1f66403a77/html5/thumbnails/6.jpg)
Sensing Context-aware framework Service provision
What is AAL?
!6
![Page 7: Semantic Reasoning and Verification for Ambient Assisted Living · 2014-05-14 · Semantic Reasoning and Verification for Ambient Assisted Living Thibaut Tiberghien, Mounir Mokhtari,](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8db8c75bef1f1f66403a77/html5/thumbnails/7.jpg)
Scientific challenges
!7
Sensing Context-aware framework Service provision
« Sensor prototyping »
« Micro context acquisition »
« Context awareness »
« Reasoning techniques »
« Modelling » « Service platform »
« User interface plasticity »
« Human factors »
Communication layer
System verification« Rules verification »
« Information consistency »
« Wireless communication »« Discovery protocols »
« Privacy »
![Page 8: Semantic Reasoning and Verification for Ambient Assisted Living · 2014-05-14 · Semantic Reasoning and Verification for Ambient Assisted Living Thibaut Tiberghien, Mounir Mokhtari,](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8db8c75bef1f1f66403a77/html5/thumbnails/8.jpg)
Scientific challenges
!8
Sensing Context-aware framework Service provision
« Sensor prototyping »
« Micro context acquisition »
« Context awareness »
« Reasoning techniques »
« Modelling » « Service platform »« User interface
plasticity »
« Human factors »
System verification« Rules verification »
« Information consistency »
« Wireless communication »« Discovery protocols »
« Privacy »
collaboration with Liu Yan, Dong Jin Song, NUS (PAT)
our research work
Communication layer
![Page 9: Semantic Reasoning and Verification for Ambient Assisted Living · 2014-05-14 · Semantic Reasoning and Verification for Ambient Assisted Living Thibaut Tiberghien, Mounir Mokhtari,](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8db8c75bef1f1f66403a77/html5/thumbnails/9.jpg)
OutlineI. Introduction
II. Context reasoning
- Reasoning strategies
- Reasoning architecture
III. Validation & Verification
IV.Conclusion
!9
![Page 10: Semantic Reasoning and Verification for Ambient Assisted Living · 2014-05-14 · Semantic Reasoning and Verification for Ambient Assisted Living Thibaut Tiberghien, Mounir Mokhtari,](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8db8c75bef1f1f66403a77/html5/thumbnails/10.jpg)
Related work
!10
Rule-based (Semantic Web)
Pattern matching (HMM, DBN…)
Sensor fusion and
binding to context
Detection of patterns in sequence of
preprocessed sensor data
+ no prior knowledge
data driven !-
labelled dataset for learning fine grain sensing
+ leverage prior knowledge
no learning !-
no implicit patterns sensitive to noise
![Page 11: Semantic Reasoning and Verification for Ambient Assisted Living · 2014-05-14 · Semantic Reasoning and Verification for Ambient Assisted Living Thibaut Tiberghien, Mounir Mokhtari,](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8db8c75bef1f1f66403a77/html5/thumbnails/11.jpg)
Semantic Web Technologies- Metadata-data graph model
- Make statements about resources
- Triple-based representation
!
- URI and namespaces
- RDF, RDF-S, OWL (TBox)
- Own namespaces (TBox + Abox)
- Serializations: XML, N3, ...
!11
« subject predicate object »
![Page 12: Semantic Reasoning and Verification for Ambient Assisted Living · 2014-05-14 · Semantic Reasoning and Verification for Ambient Assisted Living Thibaut Tiberghien, Mounir Mokhtari,](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8db8c75bef1f1f66403a77/html5/thumbnails/12.jpg)
Functional context model
!12Legend
datatypeProperty
Class
objectProperty
subClassOf
stageForAlertisAloneinRoomSinceinRoomFordoesActivitySincedoesActivityFor
Resident
liveIn/detectedIn/cameFrom
motionMeasured
Environment
lastUsed
Object
hasValuelastUpdatehasLastUpdate
Sensor
indicateLocationindicateUse
SensorState
Furniture
Door
SensorType
HouseRoomOutside
BedroomBathroom
LivingroomKitchenToilet
...
useNowlocatedIn
partOf
type
hasPossibleState/hasCurrentState
attachedTo
deployedIn
namesnoozeTime
believedToDo
nameneedHandsgetConfidenceScore
Context
busy
Caregiver
LocationActivity
solvedriskLevel
Deviance
Reminder
ackHandledacknowledgement
Notification
namemodelrepeatidtimeSentstage
Service
name
Device
watchesAfter
runningFor
solvedBy
helpsWith
usedBy
escalateTo
hasAckService
deployedIn
onDevice
Person
![Page 13: Semantic Reasoning and Verification for Ambient Assisted Living · 2014-05-14 · Semantic Reasoning and Verification for Ambient Assisted Living Thibaut Tiberghien, Mounir Mokhtari,](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8db8c75bef1f1f66403a77/html5/thumbnails/13.jpg)
Rule design: epistemology- Type I: Rules of reasoning
- Irrevocable knowledge
- Scientific models, common sense
- Type II: Rules of fact
- Domain knowledge
- Experts’ experience, intuition and insight
- Possible contradiction (scoring)
- Type III: Arbitration rules
!13
![Page 14: Semantic Reasoning and Verification for Ambient Assisted Living · 2014-05-14 · Semantic Reasoning and Verification for Ambient Assisted Living Thibaut Tiberghien, Mounir Mokhtari,](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8db8c75bef1f1f66403a77/html5/thumbnails/14.jpg)
Type I - Rules of reasoning
!14
∀ Sensor se; SensorState st; Room r, r2; House h; Resident u; Timestamp t; !
{se hasCurrentState st} ⋀ {se hasLastUpdate true} ⋀ {st indicateLocation true} ⋀ {se deployedIn r} ⋀ {r partOf h} ⋀ {u liveIn h} ⋀ {u detectedIn r2} ⋀ {r2 ≠ r}
⋀ {se lastUpdate t} ⟹ {u detectedIn r} ⋀ {u cameFrom r2} ⋀ {u inRoomSince t}
![Page 15: Semantic Reasoning and Verification for Ambient Assisted Living · 2014-05-14 · Semantic Reasoning and Verification for Ambient Assisted Living Thibaut Tiberghien, Mounir Mokhtari,](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8db8c75bef1f1f66403a77/html5/thumbnails/15.jpg)
Type II - Rules of fact
!15
![Page 16: Semantic Reasoning and Verification for Ambient Assisted Living · 2014-05-14 · Semantic Reasoning and Verification for Ambient Assisted Living Thibaut Tiberghien, Mounir Mokhtari,](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8db8c75bef1f1f66403a77/html5/thumbnails/16.jpg)
Type III - Arbitration rules
Compute the Rule-Based Confidence Score
!16
![Page 17: Semantic Reasoning and Verification for Ambient Assisted Living · 2014-05-14 · Semantic Reasoning and Verification for Ambient Assisted Living Thibaut Tiberghien, Mounir Mokhtari,](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8db8c75bef1f1f66403a77/html5/thumbnails/17.jpg)
OutlineI. Introduction
II. Context reasoning
- Reasoning strategies
- Reasoning architecture
III. Validation & Verification
IV.Conclusion
!17
![Page 18: Semantic Reasoning and Verification for Ambient Assisted Living · 2014-05-14 · Semantic Reasoning and Verification for Ambient Assisted Living Thibaut Tiberghien, Mounir Mokhtari,](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8db8c75bef1f1f66403a77/html5/thumbnails/18.jpg)
« La Pensée »A Cognitively Inspired Hybrid Reasoning Architecture
!18
Cogitation (conscious mind)
Cerebration (unconscious mind)
Knowledge formalisation
Stimulus (~ event)
Cerebral cortex
![Page 19: Semantic Reasoning and Verification for Ambient Assisted Living · 2014-05-14 · Semantic Reasoning and Verification for Ambient Assisted Living Thibaut Tiberghien, Mounir Mokhtari,](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8db8c75bef1f1f66403a77/html5/thumbnails/19.jpg)
Live event processing
!19
Legend
invokeservice
exchangedata
model.n3 home.n3 rules.n3 dump.n3
Stimulistener Cortex
Cerebration* Cogitation
EYE
DecisionInterpreter
NTriplestore
1.live event
0.loaded once when ubismart starts
2.update(live event)
3.think
i.think iv.think
ii.query iii.update v.dump
vii.think
x.update ix.inferred triples
vi.ontological dumpat each inference
viii.loaded in memoryat each inference
5.decision
xi.decision
![Page 20: Semantic Reasoning and Verification for Ambient Assisted Living · 2014-05-14 · Semantic Reasoning and Verification for Ambient Assisted Living Thibaut Tiberghien, Mounir Mokhtari,](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8db8c75bef1f1f66403a77/html5/thumbnails/20.jpg)
Hybrid reasoning
!20
Service Delivery Modules
Sensing Modules
Configuration Modules
Reasoning Modules
"X"
SensorFlowManager (Routing)
- merge all "X" signals- registration of SensApps- forward data to relevant SensApp- forward information to InfoConsumers- discovery of new sensors- configuration gate for Binder- sensors to SensApps bindings
SensApp*
- process data- produce information
BinderUI*
SensAppRepository*
- list SensApps per category of sensor- provide SensApp for download
"X" Gateway*
- low level- protocol dependent (code is specific)- normalization of data (e.g. HEX2DEC)- signal forwarding
D I K W?
- flow- no meaning- mostly useless
- in the semsea (ontology)- semantic (meaning)- useful & calculable
SensApp dev.> introduce libs in SensApp API> restructure data in a more meaningful way using processes
INFERENCE> process is not necessary semantic but I/O goes through semantic sea
Binder
- configuration tool (build, declarative)- choice, download & installation of SensApps- parametrization of SensApps- bind semantics (to sensor + its events)
Cortex
- handle reasoning cycle (until stable cogitative state)- alternate between cogitation pass & cerebration pass
start/stop & feedback
REST + Event Bus
handleEvent
processEvent
processThoughts
REST + download
Legend
SmartSpaceComposer (S2C): configuration
* this module can have several instances this module is semantic-aware, i.e. linked with NTriplestore
External Module
Internal Module
Abstract Class
EventBridge*
XMPPBridge
DPWSBridge
HTTPBridge
OSGiBridge
publish
subscribe
Stimulistener
- match sensor/state with semsea bindings- write context information in semseasubscribe
NTriplestore
- API to semantic sea (semsea)- provide CRUD access to the triplestore- implemented as a service
think
Cogitation
- high level rule-based reasoning- decision making- conscious process
Cerebration*
- threaded asynchronous inference- low level, specific, computational task- I/O in seamsea- subconscious process
Classification
Statistical Analysis
Numeric Computation
CoreEngine*
EYE
SVM
think
DecisionInterpreter
- interpret thoughts (reasoning results)- update triplestore- start/stop service- etc.
ServiceGateway
- registration of services- control over services- status update in triplestore
start/stop/update
UbiService*
BackEndService*FrontEndService*
AtomicService* CoupledService*
Reminder* CallCenter*
ActivityLog*Notification*
display(<ServiceSkin>) & ack
register
DeviceGateway
- registration & management of devices- semsea I/O
register
ComLayer*
XMPP
APNs
UbiDevice*
IPhone*
IPTV*
send & ack
ServiceSkin*
MobileGUI*
TextToSpeech*
...
- fused information- domain knowledge- higher level
subscribe
InfoConsumer*
- platform or application using the same data
![Page 21: Semantic Reasoning and Verification for Ambient Assisted Living · 2014-05-14 · Semantic Reasoning and Verification for Ambient Assisted Living Thibaut Tiberghien, Mounir Mokhtari,](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8db8c75bef1f1f66403a77/html5/thumbnails/21.jpg)
Ontology evolution breakdown- initial ontology
!21
Legend
object
datatype property
litteral
object
object property
update
query
johndoe
occupied livingroom
toilet
house
pir2
pir2_off
pir6
pir6_on
believedToDo detectedInpartOf
liveIn
deployedIn
deployedIn
hasCurrentState
hasCurrentState
motion
partOf
7
motion0
motion 7
id
A2
id
A6hasLastUpdate
2012-06-27 13:07:58
hasLastUpdate
2012-06-27 08:42:27
![Page 22: Semantic Reasoning and Verification for Ambient Assisted Living · 2014-05-14 · Semantic Reasoning and Verification for Ambient Assisted Living Thibaut Tiberghien, Mounir Mokhtari,](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8db8c75bef1f1f66403a77/html5/thumbnails/22.jpg)
Legend
object
datatype property
litteral
object
object property
update
query
johndoe
occupied livingroom
toilet
house
pir2
pir2_on
pir2_off
pir6
pir6_on
believedToDo detectedInpartOf
liveIn
deployedIn
deployedIn
hasCurrentState
hasCurrentState
motion
partOf
7
motion0
motion 7
id
A2
id
A6hasLastUpdate
2012-06-27 13:07:58
hasLastUpdate
2012-06-27 13:08:08
hasCurrentState
Ontology evolution breakdown- Stimulistener: new event update
!22
![Page 23: Semantic Reasoning and Verification for Ambient Assisted Living · 2014-05-14 · Semantic Reasoning and Verification for Ambient Assisted Living Thibaut Tiberghien, Mounir Mokhtari,](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8db8c75bef1f1f66403a77/html5/thumbnails/23.jpg)
Ontology evolution breakdown- Cerebration: motion estimation
!23
Legend
object
datatype property
litteral
object
object property
update
query
johndoe
occupied livingroom
toilet
house
pir2
pir2_onpir6
pir6_on
believedToDo detectedInpartOf
liveIn
deployedIn
deployedIn
hasCurrentState
motion
partOf
7
motion
1
motion 8
id
A2
id
A6hasLastUpdate
2012-06-27 13:07:58
hasLastUpdate
2012-06-27 13:08:08
hasCurrentState
![Page 24: Semantic Reasoning and Verification for Ambient Assisted Living · 2014-05-14 · Semantic Reasoning and Verification for Ambient Assisted Living Thibaut Tiberghien, Mounir Mokhtari,](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8db8c75bef1f1f66403a77/html5/thumbnails/24.jpg)
Ontology evolution breakdown- Cogitation: activity inference
!24
{?u qol:detectedIn ?r. ?r a qol:Bathroom. ?u qol:inRoomFor ?d. ?d math:lessThan 1800. ?a1 a qol:Hygiene. ?u qol:canDo ?a1} => {?a1 qol:getScore 9}.
Legend
object
datatype property
litteral
object
object property
update
query
johndoe
occupied
hygiene
livingroom
toilet
house
pir2
pir2_onpir6
pir6_on
believedToDo cameFrompartOf
liveIn
deployedIn
deployedIn
hasCurrentState
motion
partOf
7
motion1
motion 8
id
A2
id
A6hasLastUpdate
2012-06-27 13:07:58
hasLastUpdate
2012-06-27 13:08:08
believedToDo detectedIn
hasCurrentState
![Page 25: Semantic Reasoning and Verification for Ambient Assisted Living · 2014-05-14 · Semantic Reasoning and Verification for Ambient Assisted Living Thibaut Tiberghien, Mounir Mokhtari,](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8db8c75bef1f1f66403a77/html5/thumbnails/25.jpg)
OutlineI. Introduction
II. Context reasoning
- Reasoning strategies
- Reasoning architecture
III. Validation & Verification
IV.Conclusion
!25
![Page 26: Semantic Reasoning and Verification for Ambient Assisted Living · 2014-05-14 · Semantic Reasoning and Verification for Ambient Assisted Living Thibaut Tiberghien, Mounir Mokhtari,](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8db8c75bef1f1f66403a77/html5/thumbnails/26.jpg)
Traditional AAL validation- Difficulty to obtain relevant ground truth
- Synthetic dataset
- Role-played and labelled dataset (living lab)
!
- Irrelevant to real settings: is system usable?
- Does not validate economical "performance"
- Does not certify system behaviour
!26
![Page 27: Semantic Reasoning and Verification for Ambient Assisted Living · 2014-05-14 · Semantic Reasoning and Verification for Ambient Assisted Living Thibaut Tiberghien, Mounir Mokhtari,](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8db8c75bef1f1f66403a77/html5/thumbnails/27.jpg)
A 3-fold validation process- Performance: Load test
!
- Usability: In-situ system trial
- STARhome: controlled (technical validation)
- Singapore nursing home: semi-controlled (top-down)
- French individual homes: no control (bottom-up)
!
- Certification: Rules verification
!27
![Page 28: Semantic Reasoning and Verification for Ambient Assisted Living · 2014-05-14 · Semantic Reasoning and Verification for Ambient Assisted Living Thibaut Tiberghien, Mounir Mokhtari,](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8db8c75bef1f1f66403a77/html5/thumbnails/28.jpg)
Performance- Reasoning load test to 250 houses (40K+ triples)
!28
0
10
20
30
40
50
60
1 (34
4)
16 (2
804)
31 (5
264)
46 (7
724)
61 (1
0184
)
76 (1
2644
)
91 (1
5104
)
106 (
1756
4)
121 (
2002
4)
136 (
2248
4)
151 (
2494
4)
166 (
2740
4)
181 (
2986
4)
196 (
3232
4)
211 (
3478
4)
226 (
3724
4)
241 (
3970
4)
EYE
Rea
soni
ng D
urat
ion
(s)
Nb of houses (Nb of triples)
34s
200 homes
![Page 29: Semantic Reasoning and Verification for Ambient Assisted Living · 2014-05-14 · Semantic Reasoning and Verification for Ambient Assisted Living Thibaut Tiberghien, Mounir Mokhtari,](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8db8c75bef1f1f66403a77/html5/thumbnails/29.jpg)
STARhome
!29
Kitchen Livingroom Bedroom Bathroom
Pressure mat(ZigBee)
iPad(WiFi)
Ultrasound(ZigBee)
TV + Set top box(LAN)
Karotz(WiFi)
Karotz(WiFi)
Pressure mat(Web Service)
Water meter(Web Service)
Back-end
ZigBeegateway
Wi-Fi router
Tiny debian machine
Server with webservice repository
Serial/USBLAN
![Page 30: Semantic Reasoning and Verification for Ambient Assisted Living · 2014-05-14 · Semantic Reasoning and Verification for Ambient Assisted Living Thibaut Tiberghien, Mounir Mokhtari,](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8db8c75bef1f1f66403a77/html5/thumbnails/30.jpg)
Nursing home in Singapore
!30
-14 months trial -8 residents with dementia -2 caregivers
![Page 31: Semantic Reasoning and Verification for Ambient Assisted Living · 2014-05-14 · Semantic Reasoning and Verification for Ambient Assisted Living Thibaut Tiberghien, Mounir Mokhtari,](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8db8c75bef1f1f66403a77/html5/thumbnails/31.jpg)
Results- Recognition rate of events
!
!
!
- Early detection of the health issues
!31
events missed
correct recognition
events missed
correct recognition
Feb. 2012 (phase 2) Sep. 2012 (phase 3)
17%29%
71% 83%
0
3
6
9
12
24/02/12 27/02/12 01/03/12 04/03/12 07/03/12 10/03/12 13/03/12
Nb
of r
emin
ders
per
day
![Page 32: Semantic Reasoning and Verification for Ambient Assisted Living · 2014-05-14 · Semantic Reasoning and Verification for Ambient Assisted Living Thibaut Tiberghien, Mounir Mokhtari,](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8db8c75bef1f1f66403a77/html5/thumbnails/32.jpg)
Individual homes in France
!32
Bedroom 1
Livingroom
Bedroom 2
Kitchen
WC
Bath
door open/close sensor
motion sensor
internet-connected gateway (BeagleBone)
- 3 homes - 2 months trial in average
![Page 33: Semantic Reasoning and Verification for Ambient Assisted Living · 2014-05-14 · Semantic Reasoning and Verification for Ambient Assisted Living Thibaut Tiberghien, Mounir Mokhtari,](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8db8c75bef1f1f66403a77/html5/thumbnails/33.jpg)
Results- Activity inference
!33
Week 1
Week 2
Week 3
Week 4
Week 5
Week 6
00:00 06:00 12:00 18:00 00:00
Go out
Nap
Occupied
Sleep
time
day
![Page 34: Semantic Reasoning and Verification for Ambient Assisted Living · 2014-05-14 · Semantic Reasoning and Verification for Ambient Assisted Living Thibaut Tiberghien, Mounir Mokhtari,](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8db8c75bef1f1f66403a77/html5/thumbnails/34.jpg)
Results- Contextual dashboard
!34
2012-06-04
640640640640640
2012-06-05
824824824824824
2012-06-06339339339339339
2012-06-07
11111
2012-06-08362362362362362
2012-06-09
781781781781781
2012-06-10 470470470470470
2012-06-11
543543543543543
2012-06-12
835835835835835
2012-06-13388388388388388
2012-06-14340340340340340
2012-06-15
585585585585585
2012-06-16362362362362362
2012-06-17371371371371371
2012-06-18
590590590590590
2012-06-19 409409409409409 2012-06-20
134134134134134
2012-06-21
274274274274274
2012-06-22
44444
2012-06-23 394394394394394 2012-06-24
256256256256256
2012-06-25
288288288288288
2012-06-26 471471471471471 2012-06-27 422422422422422 2012-06-28 446446446446446 2012-06-29518518518518518
2012-06-30 418418418418418 2012-07-01
191191191191191
2012-07-02
309309309309309
2012-07-03
250250250250250
2012-07-04
299299299299299
2012-07-05
11111
2012-07-06
11111
2012-07-07
11111
2012-07-08
11111
2012-07-09
4242424242
2012-07-10
309309309309309
2012-07-11
172172172172172
2012-07-12
199199199199199
Outside
Living room
Kitchen
Bedroom3
Bedroom2
Bedroom
Mon. Tue. Wed. Thu. Fri. Sat. Sun.
![Page 35: Semantic Reasoning and Verification for Ambient Assisted Living · 2014-05-14 · Semantic Reasoning and Verification for Ambient Assisted Living Thibaut Tiberghien, Mounir Mokhtari,](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8db8c75bef1f1f66403a77/html5/thumbnails/35.jpg)
OutlineI. Introduction
II. Context reasoning
- Reasoning strategies
- Reasoning architecture
III. Validation & Verification
IV.Conclusion
!35
Liu Yan & Dong Jin Song (PAT)
@Peacehaven Nursing Home
![Page 36: Semantic Reasoning and Verification for Ambient Assisted Living · 2014-05-14 · Semantic Reasoning and Verification for Ambient Assisted Living Thibaut Tiberghien, Mounir Mokhtari,](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8db8c75bef1f1f66403a77/html5/thumbnails/36.jpg)
Model checking of AAL system- People + environment + system modelling
- Formalisms for concurrent interaction
!
- Comprehensive verification for certification
- Automatic verification and exhaustive search
!
- Counter-examples useful for bug tracking
- E.g. with rule conflict
!36
![Page 37: Semantic Reasoning and Verification for Ambient Assisted Living · 2014-05-14 · Semantic Reasoning and Verification for Ambient Assisted Living Thibaut Tiberghien, Mounir Mokhtari,](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8db8c75bef1f1f66403a77/html5/thumbnails/37.jpg)
Formal analysis approach
!37
Counterexamples
PAT Model Checker
Verification: Safety,
Liveness
Modelling: Environment,
System Design
Stakeholders: Nurses,
Engineers
Collecting Patient behaviours
Collecting Requirements
![Page 38: Semantic Reasoning and Verification for Ambient Assisted Living · 2014-05-14 · Semantic Reasoning and Verification for Ambient Assisted Living Thibaut Tiberghien, Mounir Mokhtari,](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8db8c75bef1f1f66403a77/html5/thumbnails/38.jpg)
Labelled transition systems- Modelling of people’s behaviours
!38
![Page 39: Semantic Reasoning and Verification for Ambient Assisted Living · 2014-05-14 · Semantic Reasoning and Verification for Ambient Assisted Living Thibaut Tiberghien, Mounir Mokhtari,](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8db8c75bef1f1f66403a77/html5/thumbnails/39.jpg)
Labelled transition systems- Modelling of system's behaviours
!39
![Page 40: Semantic Reasoning and Verification for Ambient Assisted Living · 2014-05-14 · Semantic Reasoning and Verification for Ambient Assisted Living Thibaut Tiberghien, Mounir Mokhtari,](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8db8c75bef1f1f66403a77/html5/thumbnails/40.jpg)
Properties specification- P1: Deadlock freeness
- P2: Guaranteed reminders
!
- P3: System inconsistency
!
!
- P4: Conflicted or erroneous reminders
!40
#assert SmartBedroom() |= [](UsingWrongBed -> <>RemindedWrongBed);
#define InConsistency (Pos_Person[1] == SHOWROOM
&& ShowerFlag && sensors[PIR] == SILENT);
# define ConflictReminder (ReminderStage[ShowerNoSoap * 2] != 0
&& ReminderStage[WanderingInSR * 2] != 0);
![Page 41: Semantic Reasoning and Verification for Ambient Assisted Living · 2014-05-14 · Semantic Reasoning and Verification for Ambient Assisted Living Thibaut Tiberghien, Mounir Mokhtari,](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8db8c75bef1f1f66403a77/html5/thumbnails/41.jpg)
Verification result- Run on Intel Xeon CPU E5506 2.13GHz, 32GB RAM
- Total runtime around 3.5hours
!41
Bugs? Property Result Nb of states Nb of transitions Execution time (s)
- P1.1 - - - OOM
J P1.2 True 1,433,654 2,038,064 815
J P1.3 True 10,783,353 15,832,370 7,045
J P2.1 True 1,599,797 2,430,351 1,945
L P2.2 False 68,178 130,734 39
L P2.3 False 2,192,251 4,531,005 12,414
L P2.4 False 832,144 1,663,779 729
L P2.5 False 4,314 5,150 1.6
J P2.6 True 1,579,579 2,377,381 1,913
L P3 True 572 745 0.3
L P4.1 True 14,675 20,615 6.1
L P4.2 True 2,446 3,036 1.11
L P4.3 True 2,332,744 3,001,756 1,047
![Page 42: Semantic Reasoning and Verification for Ambient Assisted Living · 2014-05-14 · Semantic Reasoning and Verification for Ambient Assisted Living Thibaut Tiberghien, Mounir Mokhtari,](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8db8c75bef1f1f66403a77/html5/thumbnails/42.jpg)
Verification result
!42
System part # Rules # Non-reachable # Redundant/ #Duplicated # Conflict
Bedroom 17 2 8/3 2
Shower Room 22 5 16/2 -
Avg. Time (s) - 2.05 3.05 -
- System split in two for memory purpose
![Page 43: Semantic Reasoning and Verification for Ambient Assisted Living · 2014-05-14 · Semantic Reasoning and Verification for Ambient Assisted Living Thibaut Tiberghien, Mounir Mokhtari,](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8db8c75bef1f1f66403a77/html5/thumbnails/43.jpg)
Bug report with error trace- System inconsistency
- enterShowerRoom.1 -> turnOnTap -> exitShowerRoom.1 -> port.PIRShowerRoom.Silent
!
- False alarm
- enterBedroom.2 -> sitOnBed.2.1 -> promptReminder.1
!43
![Page 44: Semantic Reasoning and Verification for Ambient Assisted Living · 2014-05-14 · Semantic Reasoning and Verification for Ambient Assisted Living Thibaut Tiberghien, Mounir Mokhtari,](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8db8c75bef1f1f66403a77/html5/thumbnails/44.jpg)
OutlineI. Introduction
II. Context reasoning
- Reasoning strategies
- Reasoning architecture
III. Validation & Verification
IV.Conclusion
!44
![Page 45: Semantic Reasoning and Verification for Ambient Assisted Living · 2014-05-14 · Semantic Reasoning and Verification for Ambient Assisted Living Thibaut Tiberghien, Mounir Mokhtari,](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8db8c75bef1f1f66403a77/html5/thumbnails/45.jpg)
Conclusion- Contribution of model checking to AAL
- Formal verification and certification of system
- Useful bug tracking tool at design level
- Save time on costly deployments
- Avoid patient disturbance of early testing
!
- An excellent verification tool during system design
- To be used in conjunction with real deployments
!45