scholastic book supportvectorm part01 2014-01-26

8/9/2019 Scholastic Book SupportVectorM Part01 2014-01-26

1/17

Scholastic Video Book Series

Part 1

Linear Support Vector Machines (LSVM)

(with English Narrations)

http://scholastictutors.webs.com

(http://scholastictutors.webs.com/Scholastic-Book-SupportVectorM-Part01-2014-01-26.pdf)

1

Scholastic Tutors (Jan, 2014)ISVT 911-0-20-140126-1

SUPPORT VECTOR

MACHINES
http://scholastictutors.webs.com/http://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/


2/17

International Baccalaureate (IB)

2

Support Vector Machines - #1Liner Support Vector Machines (LSVM)


(SVM-001)

http://youtu.be/LXGaYVXkGtg

Click here to see the video
http://scholastictutors.webs.com/http://youtu.be/LXGaYVXkGtghttp://youtu.be/LXGaYVXkGtghttp://scholastictutors.webs.com/


3/17

Support Vector Machine (SVM) algorithms are used in

Classification.

Classification can be viewed as the task of separating classes

in feature space.


Support Vector Machines

x1

x2

0

-1

-2

1

2

1 2 3 4 5 6
http://scholastictutors.webs.com/http://scholastictutors.webs.com/


4/17

Here we select 3 Support Vectors to start with.

They are S1, S2and S3.


x1

x2

0

-1

-2

1

2

1 2 3 4 5 6

S1

S2

S3

=

2

1

= 21

= 40

http://youtu.be/LXGaYVXkGtg

Click here to see the video
http://youtu.be/LXGaYVXkGtghttp://youtu.be/LXGaYVXkGtg


5/17

Here we will use vectors augmented with a 1 as a bias input,

and for clarity we will differentiate these with an over-tilde.

That is:

http://scholastic-videos.com


=211

=

2

1 1

=40

1

= 21

= 2

1

= 40
http://scholastic-videos.com/http://scholastic-videos.com/http://scholastic-videos.com/http://scholastic-videos.com/http://scholastic-videos.com/


6/17

Now we need to find 3 parameters , ,and based onthe following 3 linear equations:



. + . + . = 1 ( )

. + . + . = 1 ( )

. + . + . = +1 (+ )
http://scholastic-videos.com/http://scholastic-videos.com/http://scholastic-videos.com/http://scholastic-videos.com/


7/17

Lets substitute the values for S1 , S2 and S3 in the aboveequations.



211

.211

+ 21 1

.211

+ 401

.211

=1

=211

= 21 1

=401

211 . 211 + 21 1 . 211 + 401 . 211 =1

211

.401

+ 21 1

.401

+ 401

.401

=+1
http://scholastictutors.webs.com/http://scholastictutors.webs.com/http://scholastictutors.webs.com/http://scholastictutors.webs.com/http://scholastictutors.webs.com/


8/17

After simplification we get:

Simplifying the above 3 simultaneous equations we

get: == -3.25 and = 3.5.http://scholastic-videos.com


6 + 4 + 9 =14 + 6 + 9 =1

9 + 9 +17 =+1


9/17

The hyper plane that discriminates the

positive class from the negative class is give

by:

Substituting the values we get:



=

= 2

11 + 2

1 1 + 4

01

= 3.25 .211

+ 3.25 . 21 1

+ 3.5 .401

=10

3


10/17

Our vectors are augmented with a bias.

Hence we can equate the entry in as thehyper plane with an offset b.

Therefore the separating hyper plane equation

= + with = 1

0and offset = 3.


http://youtu.be/LXGaYVXkGtgClick here to see the video


11/17http://scholastic-videos.com


x1

x2

0

-1

-2

1

2

1 2 3 4 5 6

S1

S2

S3

= + with = 10 and offset = 3 .

This is the expected decision surface of the LSVM.


12/17http://scholastic-videos.com


Programme Code Example 1

% 3 support vector version

s1 = [ 0 -1 1 ];

s2 = [ 0 1 1 ];

s3 = [ 2 0 1 ];

A = [ sum(s1.*s1) sum(s2.*s1) sum(s3.*s1) ;

sum(s1.*s2) sum(s2.*s2) sum(s3.*s2) ;

sum(s1.*s3) sum(s2.*s3) sum(s3.*s3) ]

Y = [ -1 -1 +1 ]

X = Y/A

p = X(1)

q = X(2)

r = X(3)

W = [ p*s1 + q*s2 + r*s3 ]

x1

x2

0

-1

-2

1

2

1 2 3 4 5 6

When you run you should get:

= [ 1 0 -1]. This is a vertical line passing through x1=1.


13/17


Support Vector Machines -

Programme Code Examples 2


s1 = [ 1 1 1 ];

s2 = [ 1 1 1 ];

s3 = [ 3 1 1 ];

s4 = [ 3 1 0 ];

A = [ sum(s1.s1) sum(s2.s1) sum(s3.s1) sum(s4.s1); sum(s1.s2) sum(s2.s2) sum(s3.s2) sum(s4.s1); sum(s1.s3) sum(s2.s3) sum(s3.s3) sum(s4.s3); sum(s1.s4) sum(s2.s4) sum(s3.s4) sum(s4.s4);] Y = [ 1 1 +1 +1 ]

X = Y/A

p = X(1) q = X(2)

r = X(3)

s = X(4)

W = [ ps1 + qs2 + rs3 + ss4 ]

x1

x2

0

-1

-2

1

2

1 2 3 4 5 6

When you run you should get:

= [ 1 0 -2]. This is a vertical line passing through x1=2.


14/17


Support Vector Machines -

Programme Code Example 3


s1 = [ 1 0 1 ];

s2 = [ 2 0 1 ];

s3 = [ 3 0 1 ];

s4 = [ 2 2 1 ];

s5 = [ 3 2 1 ];

A = [ sum(s1.s1) sum(s2.s1) sum(s3.s1) sum(s4.s1)sum(s5.s1);

sum(s1.s2) sum(s2.s2) sum(s3.s2) sum(s4.s2) sum(s5.s2); sum(s1.s3) sum(s2.s3) sum(s3.s3) sum(s4.s3) sum(s5.s3); sum(s1.s4) sum(s2.s4) sum(s3.s4) sum(s4.s4) sum(s5.s4); sum(s1.s5) sum(s2.s5) sum(s3.s5) sum(s4.s5) sum(s5.s5)] Y = [ 1 1 1 +1 +1 ]

X = Y/A

p = X(1) q = X(2)

r = X(3)

s = X(4)

t = X(5)

W = [ ps1 + qs2 + rs3 + ss4 + ts5 ]

x1

x2

0

-1

-2

1

2

1 2 3 4 5 6

When you run you should get: = [ 0 1 -1]. This is a horizontal line passing through x2=1.


15/17


Classification Examples

Lets take the 5 support vector version = [ 0 1 -1]. This is a horizontal line

passing through x2=1.

Lets classify the point (x1,x2)=(4,2).

. = 01 . 42 = 2 > 1 Hence this point belongs to the red

class

Lets classify the point (x1,x2)=(2,-2).

. = 01 . 22 = 2 < 1 Hence this point belongs to the blue

class

We can do the same for any new point.

x1

x2

0

-1

-2

1

2

1 2 3 4 5 6



16/17

International Baccalaureate (IB)

16

Support Vector Machines - #1Linear Support Vector Machines (LSVM)


(SVM-001)

END of the Book

If you like to see similar solutions to any Mathematics problems please

contact us at: [email protected] your request.

http://scholastic-videos.com/http://[email protected]/http://youtu.be/LXGaYVXkGtghttp://youtu.be/LXGaYVXkGtghttp://[email protected]/http://scholastic-videos.com/http://scholastic-videos.com/http://scholastic-videos.com/


17/17

Videos at: http://www.youtube.com/user/homevideotutor

17

(http://scholastictutors.webs.com/Scholastic-Book-SupportVectorM-Part01-2014-01-26.pdf)

Scholastic Video Book Series

Support Vector Machines (SVM)

Part 1

(LSVM)

(with English Narrations)

(END)

Scholastic Tutors (Jan, 2014)ISVT 911-0-20-140126-1
http://www.youtube.com/user/homevideotutorhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://www.youtube.com/user/homevideotutorhttp://www.youtube.com/user/homevideotutor

scholastic book supportvectorm part01 2014-01-26

Documents