monitoring and testing for reliable smart city applications€¦ · reliable smart city application...
TRANSCRIPT
![Page 1: Monitoring and Testing for Reliable Smart City Applications€¦ · Reliable Smart City Application Travel Planner application Asks user for destination in Aarhus, selects near parking](https://reader033.vdocuments.us/reader033/viewer/2022043003/5f818ce4d1943e0191532b65/html5/thumbnails/1.jpg)
Monitoring and Testing for Reliable Smart City Applications
21. VDE/ITG Fachtagung Mobilkommunikation
Osnabrück, 12.05.2016
Thorben Iggena
Tel.: +49 541/969-2723
E-Mail: [email protected]
![Page 2: Monitoring and Testing for Reliable Smart City Applications€¦ · Reliable Smart City Application Travel Planner application Asks user for destination in Aarhus, selects near parking](https://reader033.vdocuments.us/reader033/viewer/2022043003/5f818ce4d1943e0191532b65/html5/thumbnails/2.jpg)
Hochschule OsnabrückUniversity of Applied Sciences
12.05.2016, 21. VDE/ITG Fachtagung Mobilkommunikation Osnabrück
Agenda
Introduction
• CityPulse Framework
Reliable Smart City Application
• Travel Planner
Testing Concept
Monitoring Concept
• Atomic Monitoring
• Composite Monitoring
Conclusion
2
![Page 3: Monitoring and Testing for Reliable Smart City Applications€¦ · Reliable Smart City Application Travel Planner application Asks user for destination in Aarhus, selects near parking](https://reader033.vdocuments.us/reader033/viewer/2022043003/5f818ce4d1943e0191532b65/html5/thumbnails/3.jpg)
Hochschule OsnabrückUniversity of Applied Sciences
12.05.2016, 21. VDE/ITG Fachtagung Mobilkommunikation Osnabrück
Introduction
Smartphone distribution increasing
Germany > 50%
Denmark:
New applications to enhance life of citizens
Simple (weather or pollution forecasts), Complex (shopping planner with integrated user preferences,
e.g. “avoid pollution”, “use scenic routes”)
3
![Page 4: Monitoring and Testing for Reliable Smart City Applications€¦ · Reliable Smart City Application Travel Planner application Asks user for destination in Aarhus, selects near parking](https://reader033.vdocuments.us/reader033/viewer/2022043003/5f818ce4d1943e0191532b65/html5/thumbnails/4.jpg)
Hochschule OsnabrückUniversity of Applied Sciences
12.05.2016, 21. VDE/ITG Fachtagung Mobilkommunikation Osnabrück
CityPulse Framework
Architecture of CP-Framework
Virtualisation
Federation (Sensor Fusion)
Aggregation (Data Fusion)
Smart Adaptation
User Centric Decision Support
Reliable Information Processing
Smart City Applications
![Page 5: Monitoring and Testing for Reliable Smart City Applications€¦ · Reliable Smart City Application Travel Planner application Asks user for destination in Aarhus, selects near parking](https://reader033.vdocuments.us/reader033/viewer/2022043003/5f818ce4d1943e0191532b65/html5/thumbnails/5.jpg)
Hochschule OsnabrückUniversity of Applied Sciences
12.05.2016, 21. VDE/ITG Fachtagung Mobilkommunikation Osnabrück
Agenda
Introduction
• CityPulse Framework
Reliable Smart City Application
• Travel Planner
Testing Concept
Monitoring Concept
• Atomic Monitoring
• Composite Monitoring
Conclusion
5
![Page 6: Monitoring and Testing for Reliable Smart City Applications€¦ · Reliable Smart City Application Travel Planner application Asks user for destination in Aarhus, selects near parking](https://reader033.vdocuments.us/reader033/viewer/2022043003/5f818ce4d1943e0191532b65/html5/thumbnails/6.jpg)
Hochschule OsnabrückUniversity of Applied Sciences
12.05.2016, 21. VDE/ITG Fachtagung Mobilkommunikation Osnabrück
Reliable Smart City Application
Travel Planner application
Asks user for destination in Aarhus, selects near parking garage with free
parking slots
Contains features to constrain routing process by user preferences
On the fly route changes if events detected by CP-framework
Highly depended on used data sources
CP-QoI-Monitoring components allow real-time Quality Monitoring for
data streams
Data sources annotated with semantic annotation, allows reasoning
and conflict resolution
Prone to faulty data delivered by used data streams
Faulty data requires complex application testing before deployment
Applications may require certain QoI levels
Testing of applications with degenerated data streams will result in
minimum QoI requirements for applications
6
![Page 7: Monitoring and Testing for Reliable Smart City Applications€¦ · Reliable Smart City Application Travel Planner application Asks user for destination in Aarhus, selects near parking](https://reader033.vdocuments.us/reader033/viewer/2022043003/5f818ce4d1943e0191532b65/html5/thumbnails/7.jpg)
Hochschule OsnabrückUniversity of Applied Sciences
12.05.2016, 21. VDE/ITG Fachtagung Mobilkommunikation Osnabrück
Agenda
Introduction
• CityPulse Framework
Reliable Smart City Application
• Travel Planner
Testing Concept
Monitoring Concept
• Atomic Monitoring
• Composite Monitoring
Conclusion
7
![Page 8: Monitoring and Testing for Reliable Smart City Applications€¦ · Reliable Smart City Application Travel Planner application Asks user for destination in Aarhus, selects near parking](https://reader033.vdocuments.us/reader033/viewer/2022043003/5f818ce4d1943e0191532b65/html5/thumbnails/8.jpg)
Hochschule OsnabrückUniversity of Applied Sciences
12.05.2016, 21. VDE/ITG Fachtagung Mobilkommunikation Osnabrück
Testing
Goal: evaluate reliability of smart city applications
Problem: no ground truth for datasets available
Collected data for December 2015 used as reference dataset
Approach:
Determination of required data streams/sensors
First testcase T0
Output of CP-framework used as ground truth
Testcase Tn
Degeneration of input data
Output compared to output of further tests
Passed if distance between Tn and Tn-1 is
below a threshold
8
![Page 9: Monitoring and Testing for Reliable Smart City Applications€¦ · Reliable Smart City Application Travel Planner application Asks user for destination in Aarhus, selects near parking](https://reader033.vdocuments.us/reader033/viewer/2022043003/5f818ce4d1943e0191532b65/html5/thumbnails/9.jpg)
Hochschule OsnabrückUniversity of Applied Sciences
12.05.2016, 21. VDE/ITG Fachtagung Mobilkommunikation Osnabrück
Testing
Generation of test case stimuli:
v‘ substitutes historic values in H to form a new testcase
Process is repeated n times
Set of testcases with decreasing (more unreliable) stream data
10
1.While τ = Α + (r * tick) < Ω:
2. For each sensor s in S:
3. v = H(s, τ)
4. For each error e in E:
5. If (Pe,s = true and e not active)
6. activate e
7. v' += apply e on v if e is active
![Page 10: Monitoring and Testing for Reliable Smart City Applications€¦ · Reliable Smart City Application Travel Planner application Asks user for destination in Aarhus, selects near parking](https://reader033.vdocuments.us/reader033/viewer/2022043003/5f818ce4d1943e0191532b65/html5/thumbnails/10.jpg)
Hochschule OsnabrückUniversity of Applied Sciences
12.05.2016, 21. VDE/ITG Fachtagung Mobilkommunikation Osnabrück
Agenda
Introduction
• CityPulse Framework
Reliable Smart City Application
• Travel Planner
Testing Concept
Monitoring Concept
• Atomic Monitoring
• Composite Monitoring
Conclusion
11
![Page 11: Monitoring and Testing for Reliable Smart City Applications€¦ · Reliable Smart City Application Travel Planner application Asks user for destination in Aarhus, selects near parking](https://reader033.vdocuments.us/reader033/viewer/2022043003/5f818ce4d1943e0191532b65/html5/thumbnails/11.jpg)
Hochschule OsnabrückUniversity of Applied Sciences
12.05.2016, 21. VDE/ITG Fachtagung Mobilkommunikation Osnabrück
Monitoring Concept
Smart city applications depend on
appropriate,
accurate,
trustworthy
… data streams
Data stream reliability has to be monitored in real-time
Monitoring methods compare data stream QoI with required QoI level for application
To fulfil real-time requirements CP-Monitoring is split into two components
Atomic Monitoring
Rudimentary QoI check for single data streams
Real-time sanity checks
Composite Monitoring
Validation of detected events by investigating correlation between spatial-correlated streams
Computationally complex
12
![Page 12: Monitoring and Testing for Reliable Smart City Applications€¦ · Reliable Smart City Application Travel Planner application Asks user for destination in Aarhus, selects near parking](https://reader033.vdocuments.us/reader033/viewer/2022043003/5f818ce4d1943e0191532b65/html5/thumbnails/12.jpg)
Hochschule OsnabrückUniversity of Applied Sciences
12.05.2016, 21. VDE/ITG Fachtagung Mobilkommunikation Osnabrück
Atomic Monitoring
Real-time QoI annotation of incoming sensor observation from data streams
Directly integrated into Data Wrapper of stream
Includes only basic QoI checks based on a stream description
Age
Completeness
Correctness
Frequency
Latency
13
![Page 13: Monitoring and Testing for Reliable Smart City Applications€¦ · Reliable Smart City Application Travel Planner application Asks user for destination in Aarhus, selects near parking](https://reader033.vdocuments.us/reader033/viewer/2022043003/5f818ce4d1943e0191532b65/html5/thumbnails/13.jpg)
Hochschule OsnabrückUniversity of Applied Sciences
12.05.2016, 21. VDE/ITG Fachtagung Mobilkommunikation Osnabrück
Atomic Monitoring
Results
Experiments done with Parking and Traffic data stream for Aarhus (December 2015)
Parking
14
![Page 14: Monitoring and Testing for Reliable Smart City Applications€¦ · Reliable Smart City Application Travel Planner application Asks user for destination in Aarhus, selects near parking](https://reader033.vdocuments.us/reader033/viewer/2022043003/5f818ce4d1943e0191532b65/html5/thumbnails/14.jpg)
Hochschule OsnabrückUniversity of Applied Sciences
12.05.2016, 21. VDE/ITG Fachtagung Mobilkommunikation Osnabrück
Atomic Monitoring
Traffic Correctness
15
![Page 15: Monitoring and Testing for Reliable Smart City Applications€¦ · Reliable Smart City Application Travel Planner application Asks user for destination in Aarhus, selects near parking](https://reader033.vdocuments.us/reader033/viewer/2022043003/5f818ce4d1943e0191532b65/html5/thumbnails/15.jpg)
Hochschule OsnabrückUniversity of Applied Sciences
12.05.2016, 21. VDE/ITG Fachtagung Mobilkommunikation Osnabrück
Atomic Monitoring
Traffic Frequency
16
![Page 16: Monitoring and Testing for Reliable Smart City Applications€¦ · Reliable Smart City Application Travel Planner application Asks user for destination in Aarhus, selects near parking](https://reader033.vdocuments.us/reader033/viewer/2022043003/5f818ce4d1943e0191532b65/html5/thumbnails/16.jpg)
Hochschule OsnabrückUniversity of Applied Sciences
12.05.2016, 21. VDE/ITG Fachtagung Mobilkommunikation Osnabrück
Atomic Monitoring
Traffic Frequency
17
![Page 17: Monitoring and Testing for Reliable Smart City Applications€¦ · Reliable Smart City Application Travel Planner application Asks user for destination in Aarhus, selects near parking](https://reader033.vdocuments.us/reader033/viewer/2022043003/5f818ce4d1943e0191532b65/html5/thumbnails/17.jpg)
Hochschule OsnabrückUniversity of Applied Sciences
12.05.2016, 21. VDE/ITG Fachtagung Mobilkommunikation Osnabrück
Composite Monitoring
Main objective: prediction of errors and plausibility of events
Main challenge: no available ground truth
Composite Monitoring
Model based analysis of tempo-spatial related stream/sensor data
E.g. traffic sensor
A sensor reports slow traffic results in a detected traffic jam
Composite Monitoring validates event with the use of consecutive sensors on road
Usage of historic data (Atomic uses only current data of one sensor)
No real-time capabilities
Triggered by events or manual evaluations
18
![Page 18: Monitoring and Testing for Reliable Smart City Applications€¦ · Reliable Smart City Application Travel Planner application Asks user for destination in Aarhus, selects near parking](https://reader033.vdocuments.us/reader033/viewer/2022043003/5f818ce4d1943e0191532b65/html5/thumbnails/18.jpg)
Hochschule OsnabrückUniversity of Applied Sciences
12.05.2016, 21. VDE/ITG Fachtagung Mobilkommunikation Osnabrück
Composite Monitoring
Determine relevant sensors out of the set of all Streams (s)
Find spatially correlated streams by using a suitable distance model (dm), which describes the means
of propagation of the event (air/street)
![Page 19: Monitoring and Testing for Reliable Smart City Applications€¦ · Reliable Smart City Application Travel Planner application Asks user for destination in Aarhus, selects near parking](https://reader033.vdocuments.us/reader033/viewer/2022043003/5f818ce4d1943e0191532b65/html5/thumbnails/19.jpg)
Hochschule OsnabrückUniversity of Applied Sciences
12.05.2016, 21. VDE/ITG Fachtagung Mobilkommunikation Osnabrück
Composite Monitoring
Determine temporal distance by analysing the
Direction (d) of expansion
Propagation velocity (v)
Range (r) of impact as function of dm
![Page 20: Monitoring and Testing for Reliable Smart City Applications€¦ · Reliable Smart City Application Travel Planner application Asks user for destination in Aarhus, selects near parking](https://reader033.vdocuments.us/reader033/viewer/2022043003/5f818ce4d1943e0191532b65/html5/thumbnails/20.jpg)
Hochschule OsnabrückUniversity of Applied Sciences
12.05.2016, 21. VDE/ITG Fachtagung Mobilkommunikation Osnabrück
Composite Monitoring
Compute the correctness for each correlated stream by applying
Vs as the set of validator functions for e and each stream s ∈ Se
𝜏s as set of temporal direction (is the change in s a result of e, or is it a cause for e?)
![Page 21: Monitoring and Testing for Reliable Smart City Applications€¦ · Reliable Smart City Application Travel Planner application Asks user for destination in Aarhus, selects near parking](https://reader033.vdocuments.us/reader033/viewer/2022043003/5f818ce4d1943e0191532b65/html5/thumbnails/21.jpg)
Hochschule OsnabrückUniversity of Applied Sciences
12.05.2016, 21. VDE/ITG Fachtagung Mobilkommunikation Osnabrück
Composite Monitoring
Combine all partial correctness values by using the:
Set of weights (Ws) for each stream s ∈ Se
A combination function (Σ), e.g. min, mean
As a result, we get the combined correctness value (Ce)
Ce = (S, dm, d, v, r, Vs, 𝜏s, Ws, Σ).
![Page 22: Monitoring and Testing for Reliable Smart City Applications€¦ · Reliable Smart City Application Travel Planner application Asks user for destination in Aarhus, selects near parking](https://reader033.vdocuments.us/reader033/viewer/2022043003/5f818ce4d1943e0191532b65/html5/thumbnails/22.jpg)
Hochschule OsnabrückUniversity of Applied Sciences
12.05.2016, 21. VDE/ITG Fachtagung Mobilkommunikation Osnabrück
Composite Monitoring
Example: Traffic Jam event created by Event Detection
Detected by sensors
Composite Monitoring is triggered by the event
sao:a26db0a4-20ca-4f4c-b553-a799200d58ca a
ec:TrafficJam ;
ec:hasSource "SENSOR";
sao:hasLevel "1"^^xsd:long;
sao:hasLocation [ a geo:Instant;
geo:lat "56.18244908701999"^^xsd:double;
geo:lon "10.1972915214958"^^xsd:double
] ;
sao:hasType ec:TransportationEvent ;
tl:time "2016-02-12T13:57:07.001Z"^^xsd:dateTime
.
![Page 23: Monitoring and Testing for Reliable Smart City Applications€¦ · Reliable Smart City Application Travel Planner application Asks user for destination in Aarhus, selects near parking](https://reader033.vdocuments.us/reader033/viewer/2022043003/5f818ce4d1943e0191532b65/html5/thumbnails/23.jpg)
Hochschule OsnabrückUniversity of Applied Sciences
12.05.2016, 21. VDE/ITG Fachtagung Mobilkommunikation Osnabrück
Composite Monitoring
Example
Event location used to determine neighbouring
relevant sensors
No temporal distance effect as event is in
sensor measurement are
Analysis of time series for nearby sensors …
24
![Page 24: Monitoring and Testing for Reliable Smart City Applications€¦ · Reliable Smart City Application Travel Planner application Asks user for destination in Aarhus, selects near parking](https://reader033.vdocuments.us/reader033/viewer/2022043003/5f818ce4d1943e0191532b65/html5/thumbnails/24.jpg)
Hochschule OsnabrückUniversity of Applied Sciences
12.05.2016, 21. VDE/ITG Fachtagung Mobilkommunikation Osnabrück
Composite Monitoring
Example
Time series analysation
Sensors 179202 and 179228 detecting slow traffic at event time
assumption that event is plausible
25
![Page 25: Monitoring and Testing for Reliable Smart City Applications€¦ · Reliable Smart City Application Travel Planner application Asks user for destination in Aarhus, selects near parking](https://reader033.vdocuments.us/reader033/viewer/2022043003/5f818ce4d1943e0191532b65/html5/thumbnails/25.jpg)
Hochschule OsnabrückUniversity of Applied Sciences
12.05.2016, 21. VDE/ITG Fachtagung Mobilkommunikation Osnabrück
Agenda
Introduction
• CityPulse Framework
Reliable Smart City Application
• Travel Planner
Testing Concept
Monitoring Concept
• Atomic Monitoring
• Composite Monitoring
Conclusion
26
![Page 26: Monitoring and Testing for Reliable Smart City Applications€¦ · Reliable Smart City Application Travel Planner application Asks user for destination in Aarhus, selects near parking](https://reader033.vdocuments.us/reader033/viewer/2022043003/5f818ce4d1943e0191532b65/html5/thumbnails/26.jpg)
Hochschule OsnabrückUniversity of Applied Sciences
12.05.2016, 21. VDE/ITG Fachtagung Mobilkommunikation Osnabrück
Conclusion
Goal: Increased reliability of smart city applications
Measures:
Testing during design-time
Determination of required QoI for an application
Monitoring during run-time, separated for scalability reasons
Atomic Monitoring
- Basic QoI calculation, real-time capable
Composite Monitoring
- Enhanced QoI check for events, complex by using spatiotemporal related streams
CP-framework provides extensive methods to support reliable smart city applications
Coping error-prone and incorrect data streams
Future: Further investigation of Composite Monitoring, apply approach to different domains
(e.g. environment, noise pollution)
27
API
![Page 27: Monitoring and Testing for Reliable Smart City Applications€¦ · Reliable Smart City Application Travel Planner application Asks user for destination in Aarhus, selects near parking](https://reader033.vdocuments.us/reader033/viewer/2022043003/5f818ce4d1943e0191532b65/html5/thumbnails/27.jpg)
Hochschule OsnabrückUniversity of Applied Sciences
12.05.2016, 21. VDE/ITG Fachtagung Mobilkommunikation Osnabrück
End
Thank you for your attention!
Questions?
28
?