exploiting semantics for sensor re-calibration in event detection systems ronen vaisenberg, shengyue...
TRANSCRIPT
Exploiting Semantics for Sensor Re-Calibration in Event Detection Systems
Ronen Vaisenberg, Shengyue Ji, Bijit Hore, Sharad Mehrotra and Nalini Venkatasubramanian
School of Information and Computer SciencesUniversity of California - Irvine
Outline
Motivation – Self Managed Sensors
Event detection systems Task Challenge Our approach
Exploiting semantics in Event Detection Systems Algorithms Analysis Experiments and results
Motivation – Self Managed Sensors
Managing Networks of Sensors can be very Challenging when sensors are: Distant and isolated Deployed in Large numbers
Human intervention is not desirable.
Example 1 – Isolated/Distant Sensors
Microwave radio-meters are used to provide continuous measurements of integrated water vapor (IWV) and integrated liquid water (ILW).
Deployed in isolation. (Rural Oklahoma and Kansas, the north slope of Alaska).
Physical changes in the sensor: Operates Based on mirrors that tend to slip as much as 1 degree on their stepper motor shafts due to continuous use.
Human intervention is not desirable,
Too far for frequent visits
Example 3 – Network with Large Number of Video Sensors
Video surveillance systems are being used in a variety of public spaces such as metro stations, airports, shipping docks, etc.
Example 3 – Network with Large Number of Video Sensors (Cont.) Changes in the field of view
Human intervention is not desirable,
Too many cameras
Outline
Motivation – Self Managed Sensors
Event detection systems Task Challenge Our approach
Exploiting semantics in Event Detection Systems Algorithms Analysis Experiments and results
The System Monitored – The context: Event Detection Systems (EDS)
The fundamental concept underlying EDS’s is the gathering, managing, analyzing, and interpreting of different information streams in a timely manner to recognize potential incidents early enough to respond effectively.
In most cases, event detection systems gather streams from sensors. Each sensor monitors a system of interest.
This system can be modeled.
The System Monitored – The Modeling: Finite State Machine Many systems of interest and observed environments can be
modeled as Finite State Machines. For example: Level of pressure (“Critical", ”High", “Medium", “Low") on the
foundations of a bridge are of crucial importance to maintenance authorities.
Traffic light states (“Red”, “Orange”, ”Green”)
The Sensor’s Task Event Detection Task Given an FSM representation of a system,
containing states S={s1..sn} generate an accurate time-series:
Coffee-Level Example – A More Complicated System
Application: detect when there is fresh coffee. Can be modeled as a finite state machine
Challenge: changes in view.
System Evolution – Change of CameraExample of Small Change of View
State A
State B
State C
State D
Timeline
System EvolutionTime
State A
State D
State D
State D
The Challenge: Recover from “System Evolution”
State A
State B
State C
State D
Timeline
System EvolutionTime
State A
State B
State D
State C
Semantics to the rescue
Semantics (one of the definitions) The meanings assigned to symbols and sets of symbols in a language.
If a computer understands the semantics of a system, it understands the meaning, rather than just interpreting a set of features.
The Proposed Solution
State A
State B
State C
State D
Timeline
System EvolutionTime
State A
State B
State D
State C
Semantic Model States of the FSM: Red, Orange, Green) State transition: {G->O,O->R,R->G} Semantics used for re-calibration: transition times between states should be in [avg-2std, avg+2std].
Avg Std avg-2std avg+2std
G->O 14.75 6.185 2.380683123 27.11931688
O->R 3.25 0.957 1.335145784 5.164854216
R->G 15 4.761 5.478095429 24.52190457
Directory
Motivation – Self Managed Sensors
Event detection systems Task Challenge Our approach
Exploiting semantics in Event Detection Systems Algorithms Analysis Experiments and results
Information Flow – Regular Detection Process
Perform State Detection
User Specified Prediction
Parameters
Time Series State Data
Real TimeImage Input
Output SensingAuxiliary Services
Extract Features
Parameterized Prediction
State A
State B
State C
State D
…..…..…..…..ti=“Red”
Information Flow – Semantics Based Detection Process
Perform State Detection
User Specified Prediction
Parameters
Time Series State Data
Real TimeImage Input
Output SensingAuxiliary Services
Extract Features
Parameterized Prediction
State A
State B
State C
State D
Sliding Window of Images
…..…..ti=“Red”
Manage Semantic Module
System Evolution Occurred
Detect System
Evolution
No
Perform Recalibration
Yes
Auto GeneratedParameters
Semantic
State AState BState D State C
Parameter Recalibration
Search in the space of parameters for new parameters that optimize consistency of the stream generated with the model.
Forall p1,p2,..,pn do Use p1,p2,..,pn to generate a new stream based on features in the buffer Evaluate the previous stream against the semantic model return p1,p2,…,pn if consistent
DEMO: Searching for the correct field of viewUsing Semantics Only Search in the space of parameters for new parameters that
optimize consistency of the stream generated with the model.Forall p1,p2,..,pn do Use p1,p2,..,pn to generate a new stream based on features in the buffer Evaluate the previous stream against the semantic model return p1,p2,…,pn if consistent
Forall different fields of view do Extract features from the field of view from all images in buffer. Cluster features extracted to m clusters (m is the number of states). Forall possible m! state assignments to the different states do Use labeled centroids of clusters to generate a new state stream Evaluate the previous stream against the semantic model return p1,p2,…,pn if consistent
Detect System Evolution
Evaluate the consistency of the last M time series data elements generated using current prediction model.
The consistency value found in the previous step is compared to a predefined threshold.
Forall (M) State Transitions (si→sj) determined based on features in buffer do
avg ← get average transition time from model for (si→sj) stdv ← get standard deviation time from model for (si→sj)
If transition time for (si→sj) is not in [avg-2*stdv , avg+2*stdv] faults++
if faults/(number of State Transitions)>threshold declare “system evolution”
Probabilistic Analysis – Probability to avoid re-calibration under normal operation
(No System Evolution)
Probability of the observations collected in the WindowBuffer to have less than C=ThresholdConsistent percent of deviations observing a system with p = 4.4% probability to generate a transition outside the 2 stdv range.
Experimental Results - Coffee-Level Detection
Data Collection Two cameras: From June 11th to June 13th
Record an image when changes are detected anywhere in view 1174 images (60MB) were collected Number of actual transition transitions: 203
Bounding box adjusted correctly in both cameras when changed to make the prediction algorithm confuse two states
Empty↔Off
Empty↔Half-Full
Experimental Results - Coffee-Level Detection
After semantic based calibration, detection is much more accurate
Future Work
Multi sensor systems Incorporating better models for anomaly
detection Incorporating features in the process of re-
calibration
Questions/Comments?