local outlier factor
TRANSCRIPT
Local Outlier Factor
Lab Goal
Implement Local Outlier factory Batch Mode.
Implement Local Outlier factory Incremental Mode.
Compare two modes.
Integrate code into open source project
RealKD:https://bitbucket.org/realKD/
Motivation
http://www.dbs.ifi.lmu.de/Publikationen/Papers/LOF.pdf
Local Outlier Factor
https://en.wikipedia.org/wiki/Local_outlier_factor
Demo
Incremental Outlier Factor
Motivation:
- infinite stream makes memory constraints.
- computational constraint for processing each stream item.
Goal:
- Same performance as iterated static LOF algorithm.
- efficient algorithm: insertion/Deletion should effect only
limited number of nearest neighbours
http://www-ai.cs.uni-dortmund.de/LEHRE/FACHPROJEKT/SS12/paper/outlier/pokrajac2007.pdf
Incremental LOF Addition
Berlin;52.520;13.380Hamburg;53.550;10.000Munchen;48.140;11.580Bonn;50.730;7.100Koln;50.950;6.970Frankfurt;50.120;8.680Dortmund;51.510;7.480Stuttgart;48.790;9.190Essen;51.470;7.000Cairo;30.3;31.14Hurghada;27.15;33.50
Incremental LOF Addition
Incremental LOF Addition
1 1.19094756172923642 1.19568308563465563 0.96456311068508184 0.80296014778290055 0.75775401355993616 0.73774956443705167 0.75096085129748678 0.999561011381989 0.694331006095839610 3.7497548217312158 11 3.928514077815152 Now , lets add new Point="Alexandria;31.13;29.58"
Incremental LOF Addition
0 1 2 6 5 8 7 4 3 9 10 1 0 6 5 8 7 4 3 2 9 10 2 7 5 0 1 3 4 6 8 9 10 3 4 8 6 5 7 1 2 0 9 10 4 3 8 6 5 7 1 2 0 9 10 5 7 3 4 6 8 1 2 0 9 10 6 8 4 3 5 7 1 0 2 9 10 7 5 2 3 4 6 8 1 0 9 10 8 6 4 3 5 7 1 0 2 9 10 9 10 2 0 7 5 1 3 4 6 8 10 9 2 0 7 5 1 3 4 6 8
0 1 2 6 5 8 7 4 3 11 9 10
1 0 6 5 8 7 4 3 2 11 9 10
2 7 5 0 1 3 4 6 8 11 9 10
3 4 8 6 5 7 1 2 0 11 9 10
4 3 8 6 5 7 1 2 0 11 9 10
5 7 3 4 6 8 1 2 0 11 9 10
6 8 4 3 5 7 1 0 2 11 9 10
7 5 2 3 4 6 8 1 0 11 9 10
8 6 4 3 5 7 1 0 2 11 9 10
9 11 10 2 0 7 5 1 3 4 6 8
10 9 11 2 0 7 5 1 3 4 6 8 11 9 10 2 0 7 5 1 3 4 6 8
Incremental LOF Addition
Cities 9,10 has change in their K-distance.
According to:
The LRD for cities exists in K-NN of cities (9,10) should
updated
LRD List={9,10,2}
According to , all cites that has any of cities {9,10,2} in their new nearest neighbour should update thier LOF value. LOF List={9,10,2,0,7}
Comparison between static and incremental LOF
Running static LOF output:
1.1909475617292364 1.1956830856346556 0.9645631106850818
0.8029601477829005 0.7577540135599361 0.7377495644370516
0.7509608512974867 0.99956101138198 0.6943310060958396
2.3423102537190847 2.342310253719085 2.342310253719085
Running incremental LOF and addition output:1.1909475617292364 1.1956830856346556 0.9645631106850818 0.8029601477829005 0.7577540135599361 0.7377495644370516 0.7509608512974867 0.99956101138198 0.6943310060958396 2.3423102537190847 2.342310253719085 2.342310253719085
Conclusion
Implementation of Batch incremental mode has done.
Batch mode code is integrated into the project repository while pull request has made to integrate it.
Incremental LOF has equivalent detection performance as static LOF.
Incremental LOF requires less computation time than time.
Incremental LOF complexity is O(N log N)
Thank you
Click to edit the title
Click to edit the outline text formatSecond Outline Level Third Outline LevelFourth Outline LevelFifth Outline LevelSixth Outline LevelSeventh Outline LevelEighth Outline LevelNinth Outline Level
Data Mining Lab,Local Outlier FactorAmr Koura / Page Supervisor: Sebastian Bothe