surveillance system
TRANSCRIPT
![Page 1: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/1.jpg)
Surveillance System
![Page 2: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/2.jpg)
OutlineGray levelSome filtersBackground modelK-means algorithmDilation & erosionBoundary extractionConnected component labelingFeature extraction
![Page 3: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/3.jpg)
Gray levelGray = ( R + G + B ) / 3
![Page 4: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/4.jpg)
Gray levelGray = 0.299 *R + 0.587 * G + 0.114 B
![Page 5: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/5.jpg)
Median filter由小到大排序 0, 0, 0, 10, 10, 20, 30, 40, 50
![Page 6: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/6.jpg)
Median filter若在一張有 salt-and-pepper noise 的影像中,
加入 Median Filter 則可有效去除雜訊。
![Page 7: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/7.jpg)
Mean filter
Standard average
Weighted average
![Page 8: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/8.jpg)
Mean filterMask 尺寸愈大,糢糊效果愈強。
![Page 9: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/9.jpg)
Laplacian filter
![Page 10: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/10.jpg)
Laplacian filter銳化結果
![Page 11: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/11.jpg)
Laplacian filterLaplacian 濾波器對於雜訊非常敏感改善:先以
高斯濾波器衰減雜訊,再加以銳化: Laplacian of Gaussian (LOG) 濾波器
![Page 12: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/12.jpg)
Laplacian filter
![Page 13: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/13.jpg)
Surveillance systems
![Page 14: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/14.jpg)
背景差分影像找前景
![Page 15: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/15.jpg)
建立背景
無前景的固定背景
Gaussian mixture model (GMM)(Mixture of Gaussian, MoG)
Highest redundancy ratio (HRR)
![Page 16: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/16.jpg)
固定式背景Frami – Backgroundi > Th 則視為前景。此方法於背景物體與亮度不會隨時間而有所變動時
才可使用,當背景物體或亮度改變時則需更新背景才可正確找到目標物。
![Page 17: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/17.jpg)
Threshold 過大或過小都會影響到前景的判斷。Too high Too low
![Page 18: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/18.jpg)
直方圖均衡化增強對比Background Frame 12
![Page 19: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/19.jpg)
Background subtraction
Frame 12 – Background
![Page 20: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/20.jpg)
Thresholding
Th=20 Th=30Th=40
![Page 21: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/21.jpg)
GMM一個像素的 (R,G) 值歷史紀錄
![Page 22: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/22.jpg)
GMM高斯模型的數目通常取 3~5 個
![Page 23: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/23.jpg)
GMMModel example
![Page 24: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/24.jpg)
GMMUpdate model
![Page 25: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/25.jpg)
GMM當一個數值介於某一個 model 的 2.5 倍標準差以
內,此數值即屬於該 model ,而後進行前述之model 更新。
如何決定背景由哪些 model 所建立?
Model 的權重依大到小排序,若前 b 個權重合超過 threshold ,則 match 到前 b 個 model 的像素值都視為背景。
![Page 26: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/26.jpg)
GMM高斯函數取得影像中的特徵像素值做為背景圖影像,
較不易受到少數光線變化或事件的影響,進而破壞它背景圖的參考品質,因為方法較為強健適合戶外監視影像,但是缺點是計算量較大。
參考論文: Chris Stauffer and W.E.L. Grimson “Adaptive Background Mixture Models for Real-Time Tracking” 1999 IEEE computer society conference on computer vision and pattern recognition, volume 2, p.2246
![Page 27: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/27.jpg)
K-means algorithm主要目標是要在大量的資料點中找出具有代表性的
資料點,這些資料點可以稱為是群中心、代表點、 codewords 等,然後在根據這些群中心,進行後續處理。
k-means 分群法是屬於「向量量化」( Vector Quantization ,簡稱 VQ )的一個基本方法。
![Page 28: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/28.jpg)
K-means algorithm隨機選取 C 個資料點,將之分別視為 C 個群聚的群中
心,這就是 Y 。由固定的 Y ,產生最佳的 U 。換句話說,對每一個資
料點 x ,尋找與之最接近的群中心,並將 x 加入該群聚。
計算目標函數 J(X; Y, U) ,如果保持不變,代表分群結果已經穩定不變,所以可以結束此疊代方法。
再由固定的 U ,產生最佳的 Y 。跳回第 2 個步驟。
![Page 29: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/29.jpg)
K-means algorithm
![Page 30: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/30.jpg)
HRR利用統計方式建立背景圖以時間標示較久的像素值做為背景圖:它首先會建
立一個表格,用來記錄每張訓練影像的變動次數及時間長短,它盡量保留變動性少,且停留時間較長的像素值,用以排除一些少數的光影變化及特殊事件,會增加一些記憶體的容量,但改善前景影像留下殘影的缺陷。
![Page 31: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/31.jpg)
HRRh(x) = Highest_Redundant { N(Ft(x))}N(Ft(x)) 代表在一段時間裡 (F) 的同個位置 (x) ,
減化重覆後的數量,所以我們取重覆量最高的像素值當作背景影像。
在整個建立背景的計算中,會將影片裡的每一個像素值儲存在歷史記錄裡 (History Map) ,觀察與比較它的變化。
![Page 32: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/32.jpg)
HRR
![Page 33: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/33.jpg)
HRR1a 與 1b 分別紀錄著目前 pixel(x, y)位置的像素
值,以及 pixel(x, y)位置像素值的重覆量,也就是可冗餘減化量。
當所有的歷史記錄記載著連續影像中歷史記錄的像素值變化及可冗餘量,而我們依照它肌餘量的排序,可以得到一個最高肌餘量的比例 (Highest redundancy ratio, HRR) ,而每一個 HRR連結它相對的像素值,代表它在這段時間內停留最久的影像,我們便把這個資料建立成背景影像。
![Page 34: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/34.jpg)
HRR以平均法建立背景的方式,因為將雜訊部分平均,
所以取出的影像雜訊部分還是有出現,殘影物影像較模糊也較完整。
經中間值方式定義背景影像,相較之下容易受到前景物的顏色影像響,留下殘影在背景圖上,而且殘留影像成破裂狀,因為中間值未考慮平均因素,所以殘留影像較為單獨性。
利用 HRR 統計式背景建立法,經過時間性的統計方式,背景圖的顏色較不會被移動的影響,所以殘留雜訊會比較少。
![Page 35: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/35.jpg)
HRRExample
![Page 36: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/36.jpg)
Dilation集合 X 經由集合 B所做的澎漲為
X 為被操作的點集合B為結構元素
![Page 37: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/37.jpg)
DilationExample
![Page 38: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/38.jpg)
DilationX 集合中原始的像素不一定會包含於澎漲後的結果
之中。
![Page 39: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/39.jpg)
DilationFrame 30 經背景差分二值化影像,人體分裂為兩物件。
分裂
![Page 40: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/40.jpg)
Dilation澎漲後,使原本的兩物件有八鄰邊的關係,人體部位可形成單一物件。
八鄰邊
![Page 41: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/41.jpg)
DilationAn example
![Page 42: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/42.jpg)
DilationAn example
![Page 43: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/43.jpg)
Dilation基本特性
Commutative
Associative
A union of shifted point sets
Translation invariant
Increasing transformation
![Page 44: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/44.jpg)
Erosion集合 X 經由集合 B所做的浸蝕為
X 為被操作的點集合B為結構元素
![Page 45: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/45.jpg)
ErosionExample
![Page 46: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/46.jpg)
Erosion浸蝕的單步執行示意圖
![Page 47: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/47.jpg)
Erosion侵蝕後的結果不一定會包含於 X 集合中的原始像
素。
![Page 48: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/48.jpg)
Erosion主要可用來消除影像中的雜訊成分。
雜訊
原始影像
![Page 49: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/49.jpg)
Erosion同時可能造成影像的支離破碎。
過濾
支離破碎
![Page 50: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/50.jpg)
Erosion基本特性
Not commutative
Not associative
An intersection of shifted point sets
Translation invariant
Increasing transformation
![Page 51: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/51.jpg)
Boundary extraction集合 X 的邊界定義為:
以四鄰邊做為邊緣
以八鄰邊做為邊緣
![Page 52: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/52.jpg)
Boundary extraction4-border
8-border
![Page 53: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/53.jpg)
Boundary extraction4-border example
![Page 54: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/54.jpg)
Connected component labeling4-connectivity
8-connectivity
m-connectivity
![Page 55: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/55.jpg)
Connected component labeling
Frame 55 connected component
可對連通過後的物件面積取閥值,同時達到去除小面積雜訊的效果。
![Page 56: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/56.jpg)
Connected component labeling參考論文:
J. M. Park, C. G. Looney and H. C. Chen, “Fast connected component labeling algorithm using a divide and conquer technique”, university of Alabama, Tuscaloosa and university of Nevada, Reno.
V. Khama, P. Gupta and C. J. Hwang, “Finding connected components in digital images”, IEEE international conference on coding and computing, April 2001, p. 652-656.
![Page 57: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/57.jpg)
Feature extraction檢測特徵點
原始圖片
![Page 58: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/58.jpg)
Feature extraction利用檢測特徵點合成影像
![Page 59: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/59.jpg)
Feature extraction特徵點的匹配
![Page 60: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/60.jpg)
Feature extraction什麼特徵是我們常用的? Corners
Harris corner detector
![Page 61: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/61.jpg)
Harris feature我們可以在一個小視窗中辨識一個點的特徵。
向任何方向移動這個小視窗會得到一個相當程度的強度變化。
![Page 62: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/62.jpg)
Harris feature基本想法
![Page 63: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/63.jpg)
Harris featureWindow 水平、垂直移動的強度改變
![Page 64: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/64.jpg)
Harris feature
![Page 65: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/65.jpg)
Harris feature
![Page 66: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/66.jpg)
Harris feature
![Page 67: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/67.jpg)
Harris feature使用M 矩陣的 Eigenvalues 做分類。
![Page 68: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/68.jpg)
Harris featureMeasure of corner response :
K是經驗常數,通常為 0.04~0.06
![Page 69: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/69.jpg)
Harris featureR 只和M 矩陣的 eigenvalues 有關R 是正值即為 cornerR 是較大的負值即為邊 緣|R| 值很小即為平坦區 域
![Page 70: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/70.jpg)
Harris feature參考 paper :
C. Harris and M. Stephens, “A combined corner and edge detector”, Plessey Research Roke, United Kingdom, The Plessey Company plc. 1988.
![Page 71: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/71.jpg)
Modify Harris角隅反應函數 (corner response function,
CRF)
![Page 72: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/72.jpg)
Modify Harris
例 a:若把圓心與上圖圓心 C 重疊,計算其 CRF ,因不管是 A點、 B點、Q點、 C 點等,其灰階值皆相同,以二值化影像來說,這樣 CRFC 的值會是零。
例 b :若把圓心與上圖圓心 C 重疊,計算其 CRF ,因為正好有一條灰階值都相同的直線通過 A點,圓心 C 和 A‘點,以二值化影像來說,這樣 CRFC 算出來的值也會是零。
例 c:若把圓心與上圖圓心 C 重疊,計算其 CRF ,因為沒有任何直線會通過圓心 C ,以二值化影像來說,這樣CRFC 算出來的值將大於零。
![Page 73: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/73.jpg)
Modify Harris
![Page 74: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/74.jpg)
Modify Harris
![Page 75: Surveillance System](https://reader033.vdocuments.us/reader033/viewer/2022061613/558fd9ff1a28ab8d528b4621/html5/thumbnails/75.jpg)
Home workConnected component labeling