![Page 1: Anomaly Detection in the WIPER System using A Markov Modulated Poisson Distribution](https://reader036.vdocuments.us/reader036/viewer/2022070410/56814659550346895db3752c/html5/thumbnails/1.jpg)
Anomaly Detection in the WIPER System using A Markov Modulated Poisson Distribution
Ping YanTim Schoenharl
Alec PawlingGreg Madey
![Page 2: Anomaly Detection in the WIPER System using A Markov Modulated Poisson Distribution](https://reader036.vdocuments.us/reader036/viewer/2022070410/56814659550346895db3752c/html5/thumbnails/2.jpg)
Outline
Background WIPER MMPP framework Application Experimental Results Conclusions and Future work
![Page 3: Anomaly Detection in the WIPER System using A Markov Modulated Poisson Distribution](https://reader036.vdocuments.us/reader036/viewer/2022070410/56814659550346895db3752c/html5/thumbnails/3.jpg)
Background Time Series
A sequence of observations measured on a continuous time period at time intervals
Example: economy (Stock, financial), weather, medical etc…
Characteristics Data are not independent Displays underlying trends
![Page 4: Anomaly Detection in the WIPER System using A Markov Modulated Poisson Distribution](https://reader036.vdocuments.us/reader036/viewer/2022070410/56814659550346895db3752c/html5/thumbnails/4.jpg)
WIPER System
Wireless Phone-based Emergency Response System
Functions Detect possible emergencies Improve situational awareness
Cell phone call activities reflect human behavior
![Page 5: Anomaly Detection in the WIPER System using A Markov Modulated Poisson Distribution](https://reader036.vdocuments.us/reader036/viewer/2022070410/56814659550346895db3752c/html5/thumbnails/5.jpg)
Data Characteristics
Two cities: Small city A:
Population – 20,000 Towers – 4 Large city B:
Population – 200,000 Towers – 31
Time Period Jan. 15 – Feb. 12, 2006
![Page 6: Anomaly Detection in the WIPER System using A Markov Modulated Poisson Distribution](https://reader036.vdocuments.us/reader036/viewer/2022070410/56814659550346895db3752c/html5/thumbnails/6.jpg)
Tower Activity
Small City (4 Towers)
![Page 7: Anomaly Detection in the WIPER System using A Markov Modulated Poisson Distribution](https://reader036.vdocuments.us/reader036/viewer/2022070410/56814659550346895db3752c/html5/thumbnails/7.jpg)
Tower Activity
Large City (31 Towers)
![Page 8: Anomaly Detection in the WIPER System using A Markov Modulated Poisson Distribution](https://reader036.vdocuments.us/reader036/viewer/2022070410/56814659550346895db3752c/html5/thumbnails/8.jpg)
15-Day Time Period Data
Small City
![Page 9: Anomaly Detection in the WIPER System using A Markov Modulated Poisson Distribution](https://reader036.vdocuments.us/reader036/viewer/2022070410/56814659550346895db3752c/html5/thumbnails/9.jpg)
15-Day Time Period Data
Large City
![Page 10: Anomaly Detection in the WIPER System using A Markov Modulated Poisson Distribution](https://reader036.vdocuments.us/reader036/viewer/2022070410/56814659550346895db3752c/html5/thumbnails/10.jpg)
Observations
Overall call activity of a city are more uniform than a single tower
Call activity for each day displays similar trend
Call activity for each day of the week shares similar behavior
![Page 11: Anomaly Detection in the WIPER System using A Markov Modulated Poisson Distribution](https://reader036.vdocuments.us/reader036/viewer/2022070410/56814659550346895db3752c/html5/thumbnails/11.jpg)
MMPP Modeling
: Observed Data
: Unobserved Data with normal behavior: Unobserved Data with abnormal behavior
Both and can be modulated as a Poisson Process.
![Page 12: Anomaly Detection in the WIPER System using A Markov Modulated Poisson Distribution](https://reader036.vdocuments.us/reader036/viewer/2022070410/56814659550346895db3752c/html5/thumbnails/12.jpg)
Modeling Normal Data Poisson distribution
Rate Parameter: a function of time
~
![Page 13: Anomaly Detection in the WIPER System using A Markov Modulated Poisson Distribution](https://reader036.vdocuments.us/reader036/viewer/2022070410/56814659550346895db3752c/html5/thumbnails/13.jpg)
Adding Day/hour effects
Associated with Monday, Tuesday … Sunday
: Average rate of the Poisson process over one week
: Time interval, such as minute, half hour, hour etc
![Page 14: Anomaly Detection in the WIPER System using A Markov Modulated Poisson Distribution](https://reader036.vdocuments.us/reader036/viewer/2022070410/56814659550346895db3752c/html5/thumbnails/14.jpg)
Requirements:
: Day effect, indicates the changes over the day of the week: Time of day effect, indicates the changes over the time period j on a given day of i
![Page 15: Anomaly Detection in the WIPER System using A Markov Modulated Poisson Distribution](https://reader036.vdocuments.us/reader036/viewer/2022070410/56814659550346895db3752c/html5/thumbnails/15.jpg)
Day Effect
![Page 16: Anomaly Detection in the WIPER System using A Markov Modulated Poisson Distribution](https://reader036.vdocuments.us/reader036/viewer/2022070410/56814659550346895db3752c/html5/thumbnails/16.jpg)
Time of Day Effect
![Page 17: Anomaly Detection in the WIPER System using A Markov Modulated Poisson Distribution](https://reader036.vdocuments.us/reader036/viewer/2022070410/56814659550346895db3752c/html5/thumbnails/17.jpg)
Prior Distributions for Parameters
![Page 18: Anomaly Detection in the WIPER System using A Markov Modulated Poisson Distribution](https://reader036.vdocuments.us/reader036/viewer/2022070410/56814659550346895db3752c/html5/thumbnails/18.jpg)
Modeling Anomalous Data
is also a Poisson process with rate
Markov process A(t) is used to determine the existence of anomalous events at time t
![Page 19: Anomaly Detection in the WIPER System using A Markov Modulated Poisson Distribution](https://reader036.vdocuments.us/reader036/viewer/2022070410/56814659550346895db3752c/html5/thumbnails/19.jpg)
Continued
Transition probabilities matrix
![Page 20: Anomaly Detection in the WIPER System using A Markov Modulated Poisson Distribution](https://reader036.vdocuments.us/reader036/viewer/2022070410/56814659550346895db3752c/html5/thumbnails/20.jpg)
MMPP ~ HMM
Typical HMM (Hidden Markov
Model)
MMPP(Markov Modulated Poisson Process)
![Page 21: Anomaly Detection in the WIPER System using A Markov Modulated Poisson Distribution](https://reader036.vdocuments.us/reader036/viewer/2022070410/56814659550346895db3752c/html5/thumbnails/21.jpg)
Apply MCMC
Forward Recursion Calculate conditional
distribution of P( A(t) | N(t) ) Backward Recursion
Draw sample of and Draw Transition Matrix from
Complete Data
![Page 22: Anomaly Detection in the WIPER System using A Markov Modulated Poisson Distribution](https://reader036.vdocuments.us/reader036/viewer/2022070410/56814659550346895db3752c/html5/thumbnails/22.jpg)
Anomaly Detection
Posterior probability of A(t) at each time t is an indicator of anomalies
Apply MCMC algorithm: 50 iterations
![Page 23: Anomaly Detection in the WIPER System using A Markov Modulated Poisson Distribution](https://reader036.vdocuments.us/reader036/viewer/2022070410/56814659550346895db3752c/html5/thumbnails/23.jpg)
Results
![Page 24: Anomaly Detection in the WIPER System using A Markov Modulated Poisson Distribution](https://reader036.vdocuments.us/reader036/viewer/2022070410/56814659550346895db3752c/html5/thumbnails/24.jpg)
Continued
![Page 25: Anomaly Detection in the WIPER System using A Markov Modulated Poisson Distribution](https://reader036.vdocuments.us/reader036/viewer/2022070410/56814659550346895db3752c/html5/thumbnails/25.jpg)
Conclusions
Cell phone data reflects human activities on hourly, daily scale
MMPP provides a method of modeling call activity, and detecting anomalous events
![Page 26: Anomaly Detection in the WIPER System using A Markov Modulated Poisson Distribution](https://reader036.vdocuments.us/reader036/viewer/2022070410/56814659550346895db3752c/html5/thumbnails/26.jpg)
Future Work
Apply on longer time period, and investigate monthly and seasonal behavior
Implement MMPP model as part of real time system on streaming data
Incorporate into WIPER system