04/18/2023

Support Vector Machine

(Classification)DuongTB2

04/18/2023

Application

• Spam email filtering

04/18/2023

Application•optical character recognition

04/18/2023

Human Detection

04/18/2023

Basic IdeaDefine the margin of a linear classifier as the width that the boundary could be increased by before hitting a datapoint.

04/18/2023

Basic IdeaMaximizing the margin is good according to intuition.

f(x,w,b) = sign(w x + b)

04/18/2023

Non-linear SVM

BoostingKernel method

04/18/2023

Boosting

04/18/2023

Boosting

04/18/2023

Boosting

04/18/2023

Kernel Method

Φ (𝑥)

04/18/2023

The “Kernel trick”

• The Linear classifier relies on dot product between vectors

• => the dot product in high dimensional space becomes:

04/18/2023

Dual Problem

• )• Subject to (

• Apply dual theorem, we have:• • Subject to

04/18/2023

Solve quadratic programming problem• Gradient Descent

04/18/2023

References

• Christopher J.C. Burges. A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery, pages 121–167, 1998.• John C.Platt. Fast training of support vector machine using sequential minimal

optimization. Advances in Kernel Method-Support Vector Learning, pages 185–208. MIT Press, 1999.• Yoav Freund and Robert E. Schapire. A Decision-Theoretic Generalization of On-Line

Learning and an Application to Boosting. Journal of computer and system sciences, pages 119-139 , 1997.

04/18/2023

THANK YOU

04/18/2023

04/18/2023

04/18/2023

04/18/2023

Sequential Minimal Optimization