sequential minimal optimization for svm contents 1 introduction to

42

Upload: others

Post on 10-Feb-2022

2 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Sequential Minimal Optimization for SVM Contents 1 Introduction to

����������� ������������������������ ��� ��!"�# ��$%�'&($*)+��,-�

.0/�1�243�1�245

687�9;:=<?>A@4B4CD:FEG>H9�:�>JIAB K4K>L<M:ON%PQCD:F>H<�RTSLC�U4EV94PJWXIYN�R[Z \](^_]a`Ab_ced�fDg"hQi�jk^%^l^%^%^l^%^�^%^l^%^%^l^%^l^%^l^%^%^�^%^l^%^%^l^%^l^%^l^%^ m](^nmporqed*sutHfDvYwegXx(yev_d=z{^%^l^%^�^%^l^%^%^l^%^l^%^l^%^%^�^%^l^%^%^l^%^l^%^l^%^ |](^ | }#x~cu��v�b_cedFf(g"hQi�j ^%^l^%^�^%^l^%^%^l^%^l^%^l^%^%^�^%^l^%^%^l^%^l^%^l^%^ �](^ � ��zOwHdFg���d����"��d=wLf(gXfD��b_x(c�^%^�^%^l^%^%^l^%^l^%^l^%^%^�^%^l^%^%^l^%^l^%^l^%^ �](^n�porqed*�%�%o���x(cHsub��Xb�x~cL��^%^�^%^l^%^%^l^%^l^%^l^%^%^�^%^l^%^%^l^%^l^%^l^%^ �](^ � ��qed�����b_ce���%�%o��=x(cLseb��Xb�x~c���b���qex~tu�#tL��b�cL���XqegXdF��qex(v�s���^%^l^%^l^%^l^%^  

\¡IAR0¢¤£�¥V¦§>L<?E�:�U ¨ ©mu^_]«ª�wu�Xb�zOb_¬=dl�­��xO®4¯�° �±^l^%^�^%^l^%^%^l^%^l^%^l^%^%^�^%^l^%^%^l^%^l^%^l^%^ ²mu^nm huj³ªµ´#v_�(x~g�b���qLz�¶¸·"w§sefD��b_ce��f?�¹�Xd=g#f���tL�=�=dF�X�­��tLvYx(wu�Xb�zOb_¬Ff?�Xb�x~c�����dFwº^»]�|mu^ | huj³ªµ´#v_�(x~g�b���qLz�¶¸¼b_�����­��x�® ¯ ° ����x(g�x(we��b_z�b_¬FfD��b_x(c�^l^%^%^l^%^l^%^l^%^»]F�

½p¾�¿�¿À7=¨ÁK4¥GPQ¨ÂPQ9;:FSu:�EV>§9 6uÃ|e^_]aorqed"Ä;ÅHÆ�Ç�g�x~tu��b_ced�^%^l^%^�^%^l^%^%^l^%^l^%^l^%^%^�^%^l^%^%^l^%^l^%^l^%^»]M�|e^nmporqedÉÈ~ÊeÅMÄËÆ�ÇLÈ~ÌQÊuÅMÄLÍHÎ�Èlg�x~tu��b_ced�^%^l^%^l^%^l^%^%^�^%^l^%^%^l^%^l^%^l^%^»]�Ï|e^ | orqed%ÐeÅ~ÑeÈ�Ò~ÐeÈ(Í�gXx(te��b_cedÓ^�^%^l^%^%^l^%^l^%^l^%^%^�^%^l^%^%^l^%^l^%^l^%^Ôm(Õ|e^ � ÖØ×?f(v�tLfD��b_ce����v�f(�X��b�ÙH�Ff?��b_x(c���tecL����b_x(cÚ^%^l^%^l^%^%^�^%^l^%^%^l^%^l^%^l^%^ÔmD�|e^n�pÛetLcL���Xb�x~cL�r��x��=x(zOwetu�Xd%suxD�#wegXxusutL���Ü^l^%^l^%^%^�^%^l^%^%^l^%^l^%^l^%^Ôm(�|e^ � �ld=gXcedFv;��tecL���Xb�x~cL��^%^l^%^�^%^l^%^%^l^%^l^%^l^%^%^�^%^l^%^%^l^%^l^%^l^%^Ôm(Ï|e^Ý «��cewete��fDcLs�x(te��wetu�Þ^%^l^%^�^%^l^%^%^l^%^l^%^l^%^%^�^%^l^%^%^l^%^l^%^l^%^Ôm(²|e^Ý Q^_] ß�d=�#wLfDg�fDzOd��Xd=g���y�à���x(zOz�fDcHs�v_b�cLdá^%^%^�^%^l^%^%^l^%^l^%^l^%^Ôm(²|e^Ý Q^nm â�d�f(s�b�cásefD�Xfã^%^�^%^l^%^%^l^%^l^%^l^%^%^�^%^l^%^%^l^%^l^%^l^%^ä|L]|e^Ý Q^ | hufM×Qb_ce�OfDcLs�v_x~f(seb�ce��zOxusud=vAwHfDg�fDzOd��Xd=g��Ü^%^l^%^%^l^%^l^%^l^%^ä|(�

|e^ Ï ��x(zOwete��d%d=gXgXx(grgXfD��d�^l^%^�^%^l^%^%^l^%^l^%^l^%^%^�^%^l^%^%^l^%^l^%^l^%^ä|~�|e^ ² játev���b���v�f(�X�å^l^%^l^%^%^l^%^�^%^l^%^%^l^%^l^%^l^%^%^�^%^l^%^%^l^%^l^%^l^%^ä|Q |e^_]FÕ�jæf(�(d�ÙHv�d���^l^%^l^%^%^l^%^�^%^l^%^%^l^%^l^%^l^%^%^�^%^l^%^%^l^%^l^%^l^%^Ü��Õ

£¡ç*U4PáèÉPQEV¦§U;:OéHPQCD:�>L<?ê�>§ër:FU4PáKìSu<?Se¥V¥GP�¥�ê�BËK4K>H<�:�EV94¦ÞK4¥GSe9 PQê íË6î ç*U4Pæ>LïYð�PQCD:FEVéHP³ëñB494CD:FEG>H9å>§ër:FU4Pæ@4B SL¥�K4<?>Lïì¥VPQ¨ íË6

]

Page 2: Sequential Minimal Optimization for SVM Contents 1 Introduction to

��������� ���������������

�������������

��

��

��

�����! !"$#&%' (*)&+-,.+/,1032 ¿D¿ +/4�576-8!4�8!9;:<0�:=+->'9?>�@BA'>')79C2EDGFH6/0�:I:KJ ,.,I8KL;M&8N9O:=+/0�6H4�+-97+/4�0�6

>'57:=+-4�+-PK0$:=+/>�9RQTS7UWV�XY@Z>�[�:I[<0�+-97+/97\]0�,IM&575^>�[I:�_'8!`N:=>�[�4�0�`<)&+-9&8aQTScb�UWXdDe(*)&+-,57[=>'\�[<0�4f+-,hgi0�,I8!j3>'9a:=)&8�5&,I8NMij7>;`N>Oj78�+-9]Fk6/0�:I:lQnmKo'o�pqXdD

(*)&+-,E+-,hg^>�:=)a:=)&8�jc>;`!M&4�8N9O:<0$:=+->'9]0�9ij�:=)78.2 ¿D¿ `!>Oj78�DsrT:h+-,E01t'u;vqwqx�jc>;`NyM&4�8!9;:z@{[=>'4}|h)7+/`<)3g^>�:=)a:=)78�~��N(c�&���i6-8�0�9ij�:=)&8.2 ¿D¿ �&6/8�`K0�9�g^8�\'8!978N[<0�:=8!jeD(*)&8hj7>;`NM&4�8!9;:<0�:=+->�9�+-,k8!,I,I8!9;:=+/0�6-6���41�.9&>�:=8N,k|h)78!9l[=8!0'j7+-97\�:=)&8*5&0�5^8N[=,hQ�4�>�,n:>�@�:=)&8N4}g^8!+-9&\W�N�O�������^���/�c���=���E@{[=>'4}:=)&8�5i0�5^8d[=,<XdD

� � 1�2*�¸/�������2*�=/�1 2ì/����C�C�Þ/a�42¡ µ3.��24/��£¢ ¤��E¥C�=1�3�¦e�� §¢ ¨orqeb��¸b_c~�Xg�xusutH����b_xl��xÉhutewew§x(g��i¸dF����x(gØj f(��qeb_cedr��x(gyeb�cHfDgXàÉ��v�f(�X��b�ÙH�Ff?��b_x(c�b��ØyLf~��d�sÉx~c© tegX�(dF�«ª�]F²(²~Ï7¬�^­B®<­ ¯a°d±.²k³*´¶µE· ¸Û4b�g�����v_d��%tL��v�xQx(�æfD�"�XqedOv�b_cedFf(g���tewew§x(g���×~dF����x(g�z�f(��qeb_ced~^����lb���yHf(��dFsæx~cá��qLd�b�sudFfxD�4qQà�w§d=gXwev�fDced%�=v_f~����b�ÙHd=g'¹�x~g�v�b_ced�fDgXv�à���d=wHfDg�fDyeb_v�b��­à(^hQtewew§x~��d ��d qHfM×(d»ºº�XgXf(b�cLb�ce��sef?��fÂw§x(b_c~����¼iª�½s¾;¿<À&¾�¬K¿�ªT½kÁc¿dÀcÁ�¬$¿�Â�Â�Â�¿'ª�½kÃ3¿<À7Ãl¬!Ä

��qedFg�d�½ ¯�Å�Æ�Ç f(cLsÈÀ ¯�Å ¼;É�]cÄ~^£ÊÞdá��x~tev_s v_b��~d���xÁv_dFfDgXcåfÞv�b_cedFf(g���d=wLf(gXfD��b_ce�qQàQwHdFg�wev�fDcLdl�=v_f~����b�ÙLd=g�¶

Ë ªT½H¬Y̶Í$ÎiÏhª èÑÐ ½�ÒÁ�$¬KÂÛeteg���qLd=gXz�x~g�d7¹M��dØ�rfDc�� ��qeb��ËqQàQw§d=gXwev_f(ced¸�Xx"qLfM×~d¸��qed�z�f;Óub�z�tez ��dFwLfDg�f?�Xb�cL��zOf(g��

�(b_cO��b��Xq�g�d���w§dF���¸��x%�Xqed��­��x*��v�f(�X��d��=^ìhQw§dF��b�ÙH�FfDv_v�à7¹(��d#��f(c��4�Xx%ÙLcLs��Xqeb��ØqQàQwHdFg�wev�fDcLdÔ ¶1ÀWÌ è¡Ð ½1Ò���ÌåÕ�fDcLsÉ�­��x�qQà�w§d=gXwev�fDced��4wLfDg�fDv_v�dFv(�Xx�b��ØfDcLs���b���q�d�Õ�tLfDvusub����XfDcH��dF���xOb���¹

Ô ¾�¶3À]Ì è�Ð ½�ÒÀ��̶Ö�]c¿

m

Page 3: Sequential Minimal Optimization for SVM Contents 1 Introduction to

Ô Á ¶3À]Ì è�Ð ½�ÒÀ��Ì�Ò%]c¿��b���qá�Xqed��=x(cLseb��Xb�x~c���qLfD�"��qLd=gXd�fDgXd*cex�sef?��f�w§x(b_c~���"y§d��­��d=dFc Ô ¾ f(cLs Ô Á ¹YfDcHs���qedsub��­��fDcL�=dlyHd=�­��dFd=c Ô ¾ fDcLs Ô Á b_�rz�f;Óub_zOb�¬FdFsY^Ûex~g4f(c�àl��d=wLf(gXfD��b_ce�#wev�fDced Ô fDcLs%��qedr��x~g�gXdF��wHx~cLsub_ce� Ô ¾ f(cLs Ô Á ¹?��d�FfDcÉf(v��rfMàu�

� cLx(gXzOf(v�b_¬=d����Xqed���xQd����=b�dFc~���l×(d�����x~g è ��x���qLfD� Ô ¾ ��b_v_v¸yHdWÀRÌ è Ð ½»Ò��3Ì Ö�]c¹fDcHs Ô Ál��b_v�vAy§d�À]Ì è�Ð ½�ÒÀ��ÌfÒ%](^ØhQd=d%´"wewHdFcLsub{Ó�´ ��x~g#sud���fDb_v_�F^

ʳd���f(c��A��x�zOfOÓub�zOb_¬=d¸�Xqed�sub����XfDcH��dy§d��­��d=dFc Ô ¾¸f(cLs Ô ÁD^¸hQx���qLd=gXd��b_v�v�yHd���x~zOdw§x~��b��Xb�×~dÉd$Óef(z�wLv�d��"x~c Ô ¾*f(cLs ��x(zOd�ced=��f?��b_×(d�d$ÓefDzOwev_dF��x(c Ô ÁD^*orqed���d�d$ÓefDzOwev_dF�fDgXd³�FfDv_v�d�s�������� ������������� ���y§dF�FfDtL��dÞx(cev_à���qedFàåwLfDg���b���b_wLfD��dÞb�c���qedÁsud=ÙLceb���b_x(c�x(���qLd���d=wLf(gXfD��b_ce��qQàQwHdFg�wev�fDcLdc¹;fDcLs xD��qLd=g"d�ÓefDzOwev_dF���=fDc yHd�g�dFzOx?×(dFs�f(cLs��Mx~g"zOx?×(d�sfDgXx(tLcLs�f~�rv�x~ce��f(����qLd=à�sex�cLxD�"��gXx~�X����qed%wLv_f(cedF� Ô ¾ fDcHs Ô Á ^â�dF�FfDv_vD�XqLf?�Ëb_c�m?����¹���qLd�sub��­��fDcL�=d¸��gXx(z�f#wHx~b�c��Eª����&¿<À��q¬§��x"f#v�b_ced�� ��Ö"!aÀ Ö$#fÌåÕ

b��&% ')(�*,+.-0/�*1+.23%4 ')51+6-75 ^�hQb_zOb�v�fDgXv�à7¹§��qed�sub_���Xf(cL��d�xD�f�wHx~b�c���x~c Ô ¾ �Xx Ô ¶ è Ð ½CÒJ��Ì Õb��8% 9;: <>=@?A%B 9 B Ì ¾B 9 B ¹Øf(cLsÞ��qLd�sub��­��fDcL�=d�y§d��­��d=dFc Ô ¾ fDcLs Ô Á b_�

ÁB 9 B ^ÞhQxe¹¸b_cÁx(g�sud=g��x�z�fOÓQb_zOb�¬Fdl��qed*seb_���Xf(cL��d7¹e��d*��qex~tev_s�zOb_ceb�zOb_¬=dDC è C.Ì èFEAè ��b��Xq���qed*�=x(cLseb��Xb�x~c

��qHf?����qLd=gXdlf(g�dlcex�sefD�Xf�w§x(b_c~����y§d��­��d=d=c Ô ¾�fDcLs Ô Á(¶è�Ð ½�ÒÁ�HG£Ö�]7¿»��x~g�wHx���b���b_×(dld�Óuf(zOwev�d���À ¯ Ì�Ö�]O¿è�Ð ½�ÒÁ�HI¶Ò%]7¿»��x~g�ced=��f?�Xb�×~d�d�Óuf(zOwev�d���À ¯ Ì�Ò%]OÂ

orqed���d��­��xO�=x(cLseb��Xb�x~cL���=f(c�y§d*��x~zÉyeb_cedFs�b_c~�XxÀ ¯ ª èÑÐ ½ ¯ ÒÁ�$¬JG�]7ÂhQx�x(tLg�wLg�x~yev�dFz+�=f(c�yHdl��x~g�z�tev_fD��d�s�f~�

zOb_c9;K ?]m è E è ��tLyML­dF������xWÀ(¯dª è�Ð ½A¯�ÒÁ�$¬JG ]OÂ

orqeb��rb_�"f��=x(cQ×(d�Ó�¹iÕ�tLf~sug�f?��b���wegXx(�(g�fDzOzOb_ce�ÉwegXx(yLv�dFz ª�b_c è ¿X�$¬$¹eb�cáfO��x~cQ×(d$Ó���d���^��c���gXxQsetL��b_ce��` fD�(g�fDcL�(d�zÉtev���b_wev_b�dFgX��® ¾ ¿�® Á ¿�Â�Â�Â�¿�® à G�Õe¹L��d*qLfM×(dl�Xqed%��x~v�v_x?��b�cL�

`Ëf(�(g�fDce�~b_f(cA¶

N ª è ¿X�q¿AO�¬QP ]m è E è ÒÃR¯TS ¾ ® ¯�À(¯Nª è�Ð ½A¯kÒÁ�$¬kÖ

ÃR¯TS ¾ ® ¯<Â

­B®VU WYX.²�Z [1³]\_^l´a`QbH\<²6cÊÞd��=f(c ��x(v_×(d%�Xqed�ÊÞx(v���d�setLfDvìb_cL�­�XdFf~sY¶$dfehgjikdlinm�� N ª è ¿X�q¿AO�¬"��b���q gXdF��w§dF������xoOa¹��teyML­dF���l��x���qed��=x(cL����g�fDb_c��X�"��qLfD���Xqed��(g�f(seb�dFc~��xD� N ª è ¿X�q¿AO�¬���b���qÞg�d���w§dF���l��x���qedwegXb�z�f(vY×Mf(g�b�fDyLv�d�� è fDcHs��"×?fDceb���qA¶

p Np è ÌrqH¿ ª­]'¬p Np � Ì�Õ ¿ ªGmc¬

|

Page 4: Sequential Minimal Optimization for SVM Contents 1 Introduction to

fDcHs��XqLf?�O G qHÂ

ÛegXx(z{ÖYÕ�tLf?�Xb�x~cL��]%fDcLsám^¹u��dlqLfM×(d

è ÌÃR¯nS ¾ ®4¯�À(¯�½A¯d¿

ÃR¯nS ¾ ®4¯TÀ~¯HÌ�ÕeÂ

hQteyH�­�Xb��Xtu��dl�Xqed=z b�c��Xx N ª è ¿��q¿ O«¬K¹H��dlqLfM×~d

N � PÃR¯TS ¾ ®4¯�Ò

]mR¯ K �®4¯G® � À(¯TÀ � ½A¯ Ð ½ � ¿

b_c���qLb_��q���qed%wLg�b_z�fDvY×?fDgXb_f(yev_dF��fDgXd�d=v_b�zOb_cLf?�XdFsY^Êåqed=c���d%��x~v�×~dl® ¯ ¹L��d%�=f(c��~d�� è Ì�� ïnS ¾ ® ¯ À ¯ ½ ¯ ¹HªV��dl��b_v�vËv_fD��d=g#��qex?��qLx?����x

��x~zOwetu��d��Xqedl��qegXdF��qex~v_s��$¬K¹HfDcLs���dl�FfDc���v�f(�X��b���à�f�cedF��x~yML­dF����½Þ��b���qË ªT½H¬ Ì Í�ÎiÏhª è�Ð ½?Ö��$¬

Ì Í�ÎiÏhª<ªÃR¯nS ¾ ® ¯ À ¯ ½ ¯ ¬ Ð ½?Ö��$¬

Ì Í�ÎiÏhªÃR¯TS ¾ ®4¯TÀ(¯dª�½A¯ Ð ½H¬HÖ��$¬KÂ

¼Øv_dFf~��dácex(��dá�XqLf?��b�c���qedæx~yML­dF����b_×(dá��tLcL���Xb�x~c�fDcLs �Xqed ��x(v_tu��b_x(cH¹r��qed ��g�fDb_ceb�cL�×(d����Xx(g��E½A¯ìxu�F��teg�x~cev�à�b�c���qLdl��x(gXz+xD�ìsuxD�#wLg�xusutL���F^

© d���x(gXd��~x(b_ce�æb_c���xæ�Xqed�sud���fDb_v_�É��xÞqex?�0��xÀ��x~v�×~d���qLb_�]Õ�tLf(segXfD��b���wegXx(�(g�fDzOzOb_ce�wegXx(yev_d=z�¹uv_d���° ��d�ÓQ��d=cHs�b��#b�c��­��x�sub�gXdF����b_x(cL�F^­B®�� �r`Y±��\<°<±1²k³*´ µ�· ¸ÊåqLfD�Ob��"�Xqedæ��teg��Vf~��d���dFwLfDg�f?�Xb�cL�æ��qLd��­��xÀ��v�f(�X��d���f(g�d�cex(�Ov_b_cedFf(g� ʳdFv�v ¹���d��=f(c��g�fDcH�­��x~g�zµ��qed�sef?��flwHx~b�c��X�Ø��x�fDcex(��qedFgØqeb_�(q�sub_zOd=cL��b�x~cLfDv§��wHf(��d#��tL��qO�XqLf?����qed�sef?��fw§x(b_c~���l��b�v_v¸y§d�v_b_cedFf(g�v_àæ��dFwLfDg�fDyLv�d~^É`Ad=���Xqed���g�fDcL����x(gXz�f?�Xb�x~cæyHd ��ª Ð ¬�^���c³��qLd�qLb��~qsub_zOd=cL��b�x~cLfDvA��wLf~��dc¹u��d%��x~v�×~d

N � PÃR¯TS ¾ ® ¯ Ò

]mR¯ K �® ¯ ® � À ¯ À � ��ª�½ ¯ ¬ Ð ��ªT½ � ¬

hQtewew§x~��dc¹Lb�cáf(sLsub��Xb�x~ck¹���ªT½ ¯ ¬ Ð ��ª�½ � ¬EÌ���ªT½ ¯ ¿d½ � ¬�^orqHf?�#b���¹u�Xqed*sux(�#weg�xusutH���#b_c��qHf?�¸qeb_�(qOsub�zOdFcL��b_x(cHfDvu��wHf(��d�b��ìd�Õ�teb_×Mf(v�dFc��4�Xx�f��j�� ��@���F��tecL����b_x(cOxD�H��qedrb_cewetu���wLf~��d(^hQx���d�cedFdFs³cexD�*y§dOd$Óuwev_b_�=b��*f(yHx~tu�l��qedO�XgXf(cL�­��x~g�z�fD��b_x(c���ª Ð ¬lf~��v_x(cL��f~����dO�Qcex?�

Page 5: Sequential Minimal Optimization for SVM Contents 1 Introduction to

��qHf?��Xqed#�~d=gXced=vL��tecL���Xb�x~c ��ªT½A¯<¿d½ � ¬ìb��d'Õ~tLb�×?fDv_d=c��Ø�Xx%��qLd�sux(��wegXxusutL����xD�Ë��x(zOd#xD�Xqed=gqeb_�(q�sub_z�dFcL��b�x~cLfDvY��wLf(�=d(^¸orqed=gXdlfDgXd�z�fDcQà��~d=gXced=v§��tecH����b_x(cH����qLfD���FfDc�yHdltL��dFs���qLb_��rfMàc¹���x(g�d�Óuf(zOwev�d7¹u��qLd%gXf~sub_f(v;yLf(��b_����tLcL���Xb�x~cªñßlfDtL�X��b_f(c��~d=gXced=v�¬

� ª�½ ¯ ¿<½ � ¬YÌ�� = B <���=�<�� B 5�� Á 5 Âorqed�jæd=g���d=g�° �"��x~cLsub���b_x(cæ�FfDcáy§d�tL��dFs��Xx�sud=��dFg�zOb_cedÉb��fO��tecL����b_x(cÞ�=fDcáy§dÉtH��d�s

f(��f��~d=gXced=v;��tecH����b_x(c˶orqLd=gXd�d�Óub_���X�#f�z�fDwLweb�cL� ��fDcHs�f(c�d�ÓQwHfDcL��b�x~c

� ªT½B¿��s¬EÌ R¯��ª�½H¬�¯ ��ª �s¬­¯N¿

b��¸fDcLs�x(cLv�à�b��=¹u��x~g#fDcQàWÎ�ª�½H¬r��tL��q���qLfD�� Î�ª�½H¬ Á�� ½Àb_��ÙLcLb��Xd7¿

�Xqed=c� � ª�½B¿�s¬nÎ�ª�½H¬IÎ ª �s¬ � ½ � � G�ÕeÂ

­B®�� �jc ^�²k´��N²������&².^1³*´ ³�� °,`Y±orqed�xD�Xqed=g¸sub_g�d����Xb�x~c*��x�d$ÓQ�Xd=cLsOhQi�j{b��Ë�Xx%fDv_v�x?�À��x~g4cLx(b���d7¹?x(gìb�zOw§d=g���dF���¸��d=wHfDg�f?��b_x(cË^orqLfD�b_��¹���d#suxÉcexD�r�­�Xg�b����Xv�à�d=cu��x~gX�=d���qLfD�Ø��qLd=gXd#yHd�cexÉsefD�Xflw§x(b_c��X��yHd=�­��dFd=c Ô ¾rfDcHsÔ ÁO¹DyLtu�¸��d�sud�ÙLcLb��Xd=v_à*��f(c�� �Xx�w§d=cLf(v�b_¬=d���qLdrsLf?�Xf�w§x(b_c~��� ��qLfD�ì�=g�x����Ë��qedry§x(tecHsefDgXb�d��=^orqed�wHdFcLfDv��­à #0��b_v_v y§d*ÙLcLb��Xd(^�ªV���J#�Ì��£¹§��d���x~z�dÉyLf(������x��Xqed�x(gXb��~b�cLf(vAw§d=g���dF�����d=wLf(gXfD��b_ce���Ff(��d(^�¬

ʳd%b_c���gXxusutL��dlcex~cu��ced=��f?��b_×(dl��v_f~����×?fDgXb_f(yev�d�� �=¯ G�Õ ¹e��x��XqLf?�è�Ð ½A¯�ÒÀ�HG£Ö�]�Ò!�=¯ ��x(g�À(¯HÌ�Ö�]O¿è�Ð ½ ¯ ÒÀ�HI¶Ò%]EÖ"� ¯ ��x(g�À ¯ ÌfÒ%]O¿�F¯ G Õe¿$#�%!Â

fDcHs���d%f(ses���x��Xqed%x(yML­d����Xb�×~d���tLcL���Xb�x~c�f�w§d=cLf(v�b_¬=b_ce�O�Xd=gXz�¶zOb_ceb_z�b_¬=d9;K ?1K &

]m è E è Ö #]ª R ¯ � ¯ ¬('

��qedFg�d*)�b���tL��tLfDv_v�à���d=����x�]c¹u��qeb���q��(b_×(d���tH�zOb�cLb�zOb_¬=d9;K ?1K + �

¾Á è E è Ö # � ïTS ¾ �=¯��teyML­d������Xx À ¯ ª è E ½ ¯ ÒÁ�$¬HÖ,� ¯ Ò ]$G�Õ ¿ ] I-%�IȺ

� ¯ G�Õ ¿ ] I-%�IȺ�

Page 6: Sequential Minimal Optimization for SVM Contents 1 Introduction to

��c���gXxusutL��b_ce��`Ëf(�(g�fDce�~d"z�tev��Xb�wLv�b_d=g�� Oa¹���¹u�Xqed*`Ëf(�(g�fDce�~b_f(c�b_�N ª è ¿X�q¿�=¯��AOa¿���¬ Ì ]

m è E è Ö #ÃR¯TS ¾ �=¯

ÒÃR¯nS ¾ ®4¯��-À~¯<ª è E ½A¯�ÒÁ�$¬HÖ,�=¯�Ò ]���Ò

ÃR¯nS ¾� ¯ �=¯

Ì ]m è E è Ö

ÃR¯TS ¾ ªV#¶ÒÀ® ¯ Ò � ¯ ¬ � ¯

Ò� ÃR¯nS ¾ ® ¯ À ¯ ½ E¯ � è Ò� ÃR

¯TS ¾ ® ¯ À ¯ � �*ÖÃR¯TS ¾ ® ¯

}#dFb��Xqed=g���qed �=¯�° ��¹ìcex(gÉ��qedFb�g�`ËfD�~gXf(ce�(d�z�tev���b_wev�b_d=g���¹ìf(wew§dFfDgÉb�cÁ�Xqed?ÊÞx(v���d�setLfDvwegXx(yev_d=z�¶

z�fOÓQb_zOb�¬Fd� N � P R ¯® ¯ Ò ]m R ¯ K �

® ¯ ® � À ¯ À � ½ ¯HÐ ½ ���teyML­dF���r��xL¶ÕfI ® ¯ I #�¿R¯®4¯TÀ~¯HÌ�ÕeÂ

orqed�x~cev�à�sub��;d=gXd=cH��dr��gXx(zT�Xqed�w§d=g���dF���Xv�à���dFwLfDg�f?�Xb�cL�l�Ff(��drb_�ì�XqLf?�®4¯Yb��ìcex?� yHx~tecLsud�sfDy§x?×(d�yQàD#µb�cL����d�f(s�xD� �å^ØÛLx(g#sud=�Xf(b�v���¹H��dFdl´"wew§d=cLsub_d$Ó © ^orqed*��x(v_tu��b_x(c�b_�"fD�~f(b�c��(b_×(dFc�yQà

è ÌÃR¯nS ¾ ®4¯�À(¯�½A¯

oËxl�XgXf(b�c��Xqed�hQi�j ¹(��d#��d�fDg���qÉ�XqegXx(te�~qÉ�Xqed���dFf~��b_yev_d�g�dF�(b_x(c�xD�;��qLd#sutLfDvHwegXx(yev_d=zfDcHs%z�f;Óub�zOb_¬=d���qed�x(yaL­dF���Xb�×~dØ��tecH����b_x(cË^4orqed�x(we��b_zOf(vu��x~v�tu�Xb�x~cÉ�=f(cÉy§d���qLdF���(d�sltL��b�ce���qLd*�%�%o���x~cLsub���b_x(cL�F^

­B®�� WYX.²���� W �@`Y±HZ1° � °1`Y± �orqedÀ�%�%o�x(we��b_zOf(v�b��­à���x~cLsub���b_x(cL��x(�*��qLdÞwLg�b_z�fDvlwegXx(yev_d=z f(g�d7¹"�XqedÀ�(g�f(sub_d=c���x(�N ª è ¿X�;¿ O�¿��.¬#��b���qÞg�d���w§dF�����Xx��XqedOweg�b_z�fDv4×?fDgXb_f(yev�d�� è ¹Ë�;¹�� ×?fDceb���qLdF�3ª¹��qLb_�lz�tL�­�fDv_�rfMàQ��y§d*��fD��b��­ÙLd�s�y�à���qed*sutHfDvYwegXx(yev_d=z�¬$¹LfDcLs���qHf?����x(g%]$I�%�I º ¹® ¯ ª�À ¯ ª è E ½ ¯ ÒÁ�$¬kÖ"� ¯ Ò ]'¬hÌåÕe¿ ªV|7¬

� ¯ � ¯ Ì�Õ Â ª��&¬��d=w§d=cLsub_ce��x~c��Xqed%×?fDv_ted%xD�ì® ¯ ¹e��d�qHfM×(d���qegXd=d*�Ff(��dF����x��=x(cL��b�dFgF¶

Page 7: Sequential Minimal Optimization for SVM Contents 1 Introduction to

]~^�����®4¯�Ì[Õ ¹A��qedFc � ¯�Ì #�ÒJ®4¯�Ì #��TÕe^�ÛLg�x~z¤ÖYÕ�tLfD��b_x(c � ¹ �=¯�Ì0Õe^���x���dqHfM×(d

À ¯ ª è E ½ ¯ ÒÁ�$¬zÒ�]$G Õ Âme^����¸Õ���® ¯ � #3¹u��gXx(z+ÖEÕ~tHf?��b_x(c�| ¹u��dlqLfM×(d

À ¯ ª è E ½ ¯ ÒÁ�$¬kÖ"� ¯ Ò�]lÌ�Õ ªG�c¬}"xD�Xd���qHf?� � ¯ Ì # Ò ® ¯ � Õ ¹¸��x � ¯ Ì+Õ ªVÖYÕ�tLfD��b_x(cÁ�i¬�^³hQtLyL�­�Xb��Xtu��b_ce�³b�c���xÖEÕ�tLf?�Xb�x~c�� ¹u��dlqLfM×~d

À ¯ ª è E ½ ¯ ÒÁ�$¬zÒ�]lÌ�Õ Â|L^����¸® ¯ Ì #3¹e��qedFc���gXx(z{ÖYÕ�tLf?�Xb�x~c�|e¹u��dlqLfM×~d

À(¯dª è E ½A¯kÒÁ�$¬kÖ"�F¯�Ò�]lÌ�Õ ªV�7¬}"xD�Xdl��qLfD� � ¯sÌ #¶ÒÀ® ¯sÌ�Õ ¹e��dlqLfM×(d �=¯ G�Õe^ØhQx

À(¯dª è E ½A¯kÒÁ�$¬zÒ�]$I Õ Âorqed�Õ�tLfDc��Xb��­à�À~¯<ª è E ½A¯kÒÁ�$¬zÒ ]%�=fDc�y§d*��x~z�wLtu��d�s�f~�� ¯ Ì À ¯ ª è E ½ ¯ ÒÀ�$¬BÒRÀ Á¯ Ì À ¯ ª è E ½ ¯ ÒÁ�EÒRÀ ¯ ¬hÌ À ¯��"¯

��qedFg�d �#¯ Ì è E ½ ¯ ÒÁ�EÒ À ¯ Ì�� ¯ Ò À ¯ b��r��qed%wLg�d�sub_����b_x(c�d=gXg�x~gF^oËx���tezOz�fDgXb�¬Fdc¹Q��qLd*�%�%o���x~cLsub���b_x(c�b�zOwev_b_dF�F¶

® ¯sÌ�Õ � ¯ G Õe¿Õ�� ® ¯� # � ¯���Õe¿® ¯ Ì # � ¯ I ÕeÂ

��c���qedl��x~v�v_x?��b�cL���­��xO�=f(��dF��¹Q��qLd*�%�%o���x~cLsub���b_x(c�b_�#×�b_x(v�f?�XdFsY¶ ®4¯�� #µf(cLs � ¯�� Õ ¹ ® ¯ ��ÕOfDcLs � ¯ � ÕL^

­B®�� �YX1² ����°<±�� � � W �@`Y± Zl° � °,`Y±��¡°(�MX `�[ �8[ �i°<±�� �aX.´ ²�� X `�\1Z��

´"���XqedásutLfDvrwegXx(yLv�dFz-suxQdF��cexD����x(v_×(d���x(g���qed���qLg�d���qex~v_s ��sub_gXdF���Xv�à7¹b�����x~tev_sJyHdy§d=ced=ÙH��b�fDv~��xl��qedF������qLd��%�%oÀ��x~cLsub���b_x(c���b��Xqex(tu�ìtL��b�cL�#��qLg�d���qex~v_s*�?^4orqeb��Ë��dF��qLceb Õ�tedb���sutedl��x��ldFd=g���qebYd=�#fDvñ^HªGm(Õ(Õe]q¬�^orqed.Õ�tLfDc��Xb��­à�À ¯ ª èFE ½ ¯ Ò��$¬&Ò�]�ª���qeb���q�zÉtL��� G�Õ���x(gØf(v�v %Ëb��H��qed#�%�%oÂ�=x(cLseb��Xb�x~c

b�����fD��b��­ÙLd�se¬r�=f(c�f(v_��x�yHd%��gXb�����dFc�f~�À(¯<ª è E ½A¯HÒÀ�$¬zÒ ]

Ì À ¯ ª è E ½ ¯ ÒÀ�$¬zÒ À Á¯Ò À ¯ ª è E ½ ¯ ÒRÀ ¯ ÒÁ�$¬Ì À ¯ ª�� ¯ ÒÀ�$¬$¿

 

Page 8: Sequential Minimal Optimization for SVM Contents 1 Introduction to

��qedFg�d �4¯7P è E ½A¯kÒ À(¯­^}#x(��d*��x~g � ¯YÌ � ¯sÒÂ�q¹H��dÉqHfM×(d � ¯sÒ � � Ì � ¯sÒ � � ^3ªVorqeb��"d'Õ�tLfDv_b��­à�b��"tL��d���tLv�¹f(�ɼØv�f?����° ��huj³ª{fDv_�(x~g�b���qLz tL��dF� �#¯ Ò � � ��qed=cÁx~wu��b_zOb�¬�f?��b_x(cÀ�Xqed��­��x ` fD�(g�fDcL�(dzÉtLv��Xb�wev_b_d=g���® ¯ ¹L® � ^ ¬orqeb���cexD��f?�Xb�x~c�b��rtL��d���tevAy§dF�FfDtL��dl��qLd*�%�%o���x~cLsub���b_x(cL�

® ¯sÌ�Õ À(¯Nª��4¯�ÒÁ�$¬JG ÕÕ�� ® ¯ � # À ¯ ª�� ¯ ÒÁ�$¬��Õ® ¯ Ì # À ¯ ª�� ¯ ÒÁ�$¬JI Õ

�=f(c�y§d%��gXb�����d=cáf~�% Å � ��� � ¾ � � Á �4¯ Gå�% Å � ��� ��� � ��� �4¯ Iå�q¿

��qedFg�d� � P ¼ %ض(Õ�� ® ¯�� #�Ä� ¾ P ¼ %ضOÀ(¯HÌ�Ö�]c¿X® ¯HÌ�Õ^Ä� Á P ¼ %ضOÀ(¯HÌfÒ%]c¿X® ¯HÌ #�Ä��� P ¼ %ضOÀ(¯HÌ�Ö�]c¿X® ¯HÌ #�Ä��� P ¼ %ضOÀ(¯HÌfÒ%]c¿X® ¯HÌ�Õ^Ä&Â

hQxÉ��qLfD� # % Å � �� � ¾ � � Á ¹ufDcHs #� Å � ��� ��� � ��� ¹u��d���qLx(tev�s�qLfM×~d �4¯ G � � ¹Qb��4�%�%o��x~cLsub���b_x(c�b_�"��fD��b��­ÙLd�sY^

I0 I0

I1

I2

I3

I4

oËx���qed�����b�� �Xqeb_�"��x(cHsub��Xb�x~c�qex(v�se��¹u��d%sud=ÙLced� tew Ì zOb�ck¼ �4¯¸¶�% Å � � � � ¾ � � Á Ä&¿� v�x?� Ì z�f;ÓG¼ �4¯¸¶�% Å � ��� ��� � ��� Ä&Â

orqed��%�%o ��x(cHsub��Xb�x~cæb_z�wLv�b_dF�%� tew G � v�x?� ¹YfDcLs³��b_zOb�v�fDgXv�à7¹ # % Å � ��� � ¾ � � Á ¹ �4¯;G� v_x?� ¹ef(cLs #�% Å � � � ��� � ��� ¹ �4¯ I�� tew ^orqedF��d*��x~zOwLfDgXb_��x(cH��suxOcex(�#tL��dl��qedl�Xqeg�d���qLx(v�s��M^´"��fDc�f(sLsudFs y§d=ced=Ùe��¹#�~b�×~d=c ��qLdáÙLg��­��®4¯<¹r�XqedF��dæ��x~zOwLfDgXb_��x(cH��fDtu�Xx(z�f?�Xb_�FfDv_v�à

ÙLcLsL�r��qed*��dF�=x(cLs�® ¯¸��x(g]L­x(b_c��#x(wu�Xb�zOb_¬FfD��b_x(c�b�cæhuj³ª�^

Ï

Page 9: Sequential Minimal Optimization for SVM Contents 1 Introduction to

� �?¢ � �����É/��h��2*¥��U�®<­ ^ � °1c�°��² � � ` ����� �orqed�hujÞª f(v��~x(gXb��Xqez ��dFfDg���qed�����qegXx(te�~q �Xqed���d�f(��b�yev_d�gXd=�(b_x(cÞxD����qLd�sutLfDv�wegXx(yev_d=zfDcHs�z�f;ÓuzOb_¬=dF���Xqed%x(yaL­dF���Xb�×~d���tecL���Xb�x~c

N � PÃR¯TS ¾ ®4¯�Ò

]mR¯ K �®4¯G® � À(¯TÀ � ½A¯ Ð ½ � ¿

ÕfI ® ¯QI #�¿ # %!Â���É��x(gX�u�ly�àÞx(we��b_z�¬Fb�cL���­��xæ® ¯ ° �ÉfD�Éf���b_zOd»ª���b���qÂ��qed�x(��qedFgÉ® ¯ ° �lÙ ÓudFs ¬�^����ÉtL��dF�qedFteg�b�����b��=���Xx���qexQx���d��Xqed �­��xJ® ¯ °���x(g�x(wu�Xb�zOb_¬FfD��b_x(cA^µorqLb_��b_��dF�X��d=c���b�fDv_v�à�fÁqeb_v_v����v_b_zÉyeb_ce�L^

Êåb��Xqex(te�Év�x����lx(���~d=cedFgXf(v�b��­àc¹Ë��tewew§x~��d���dOf(g�dOx~wu��b_zOb�¬Fb�ce�á®B¾;¹4®sÁc¹A��gXx(z fDcÞx(v�s��d��%xD����dFf(��b_yev�d���x(v_tu��b_x(c˶�®���� �¾ ¹ ®���� �Á ¹ ® � ¹hÂ�Â� ¹ ®zÃÉ^�ªVÛLx(g%b_ceb���b�fDv_b�¬�f?��b_x(cH¹A��dO�=f(cÞ��d��O���� �aÌ qË^ ¬

© dF�=f(tL��d�� ïnS ¾ À ¯ ® ¯ Ì�Õ ¹u��dlqLfM×(dÀ ¾ ® ¾ Ö À Á ® Á Ì£À ¾ ® ��� �¾ Ö À Á ® ��� �Á Â

orqeb��r��x(ceÙLcedF���Xqedlx(wu�Xb�zOb_¬Ff?�Xb�x~c��Xx�y§dlx(c�f�v_b�ced7¹ef(�r��qex?��c�b_c��Xqed���x(v_v�x?��b_ce��ÙL�(tegXd(¶

�������

�����!

�#"$��� �#"%�! �#"$��� �#"%�!

�������

�����!

&'")(�*&+�-,.�#"0/1�����32 &'"4�*&+�-,.�#"657�����32`Ad=�lÍlÌ À ¾ À Á ^¸játev���b_wev�àÀi¾�®B¾hÖ À7Á=®zÁ1Ì���x~cL�­��^

yQà�À ¾ ¹LfDcLs���d�qHfM×(d® ¾ Ì98?ÒÍ�® Á Â

��qedFg�d�8oP�®*¾hÖ¡Í�®sÁ1Ì�®���� �¾ Ö Í�®���� �Á ^Û4b/Óub_ce�O��qed%x(��qedFg#® ¯ ° ��¹u�Xqed%x(yML­d����Xb�×~d���tLcL���Xb�x~c��FfDc�yHd%��gXb�����dFcáf(�N � Ì ® ¾ Ö® Á Ö ��x(cL���F^

Ò ]m-: À ¾ À ¾ ½ E ¾ ½ ¾ ® Á ¾ Ö�À Á À Á ½ EÁ ½ Á ® ÁÁ Ö�m;À ¾ À Á ½ E ¾ ½ Á ® ¾ ® ÁÖlm ÃR¯TS �® ¯ À ¯ ½ E¯ � ª�À&¾$½s¾�®B¾hÖ�ÀcÁ$½kÁ�®zÁ�¬HÖ ��x(cL���F^ ;

²

Page 10: Sequential Minimal Optimization for SVM Contents 1 Introduction to

`Ad=� � ¾d¾ Ì ½ E ¾ ½ ¾ ¹ � ÁNÁ Ì ½ EÁ ½ Á ¹ � ¾<Á Ì ½ E ¾ ½ Á ¹ef(cLs� � P

ÃR¯nS �® ¯ À ¯ ½ E¯ ½ �

Ì ½ E� è ��� � ÒÀ® ��� �¾ À&¾K½ E ¾ ½ � ÒÀ® ��� �Á ÀcÁ�½ EÁ ½ �Ì ª�½ E� è ��� � ÒÁ� ��� � ¬HÖ�� ��� � ÒÀ® ��� �¾ À&¾K½ E ¾ ½ � ÒÀ® ��� �Á ÀcÁ$½ EÁ ½ �Ì � ��� �� Ö�� ��� � ÒÀ® ��� �¾ À&¾K½ E ¾ ½ � ÒÀ® ��� �Á ÀcÁ$½ EÁ ½ � ¿��qedFg�d � ��� �� Ì£½ E� è ��� ��ÒÁ� ��� ��b_���Xqed%x(te��wetu�#x(�z½ � tecLsed=g#x~v_s�wLfDg�fDzOd=��d=g��F^

N � Ì ® ¾ Ö® Á Ò ]m : � ¾N¾ ® Á ¾ Ö � ÁdÁ ® ÁÁ ÖJm7Í � ¾=Á ® ¾ ® ÁÖlm;À ¾ � ¾ ® ¾ ÖJmOÀ Á � Á ® Á ;]Ö ��x(cH�­��^

Ì 8?Ò Í�®sÁYÖJ®zÁ�Ò ]m�: � ¾d¾Oª 8�ÒÍ�®zÁ�¬ Á Ö � ÁNÁ=® ÁÁÖlmcÍ � ¾<Á ª 8�ÒÍ�® Á ¬­® ÁÖlm;À ¾ � ¾ ª 8�ÒÍ�® Á ¬HÖ�m;À Á � Á ® Á�; Ö ��x(cL���F^

Ì ª�]�ÒÍq¬­®zÁ�Ò ]m � ¾N¾;ª 8�Ò ÍF®zÁ�¬ Á Ò ]m � ÁNÁ=® ÁÁ ÒÍ � ¾<Ácª 8?Ò ÍF®zÁq¬­®zÁÒ�À ¾ � ¾ ª 8?Ò Í�® Á ¬*Ò À Á � Á ® Á Ö ��x(cH�­��^

Ì ª�]�ÒÍq¬­®zÁ�Ò ]m � ¾N¾�8 Á Ö¡Í � ¾d¾�8Y®zÁ�Ò ]m � ¾d¾�Í Á ® ÁÁ Ò ]m � ÁNÁ=® ÁÁÒ1Í � ¾=Á 8;® Á Ö Í Á � ¾<Á ® ÁÁ ÒRÀ ¾ � ¾ 8WÖ Í�À ¾ � ¾ ® Á ÒRÀ Á � Á ® ÁÖ%��x(cL���F^

Ì ª�]�ÒÍq¬­® Á Ö¡Í � ¾d¾ 8;® Á Ò ]m � ¾N¾ ® ÁÁ Ò ]m � ÁNÁ ® ÁÁÒ1Í � ¾=Á 8;® Á Ö � ¾=Á ® ÁÁ Ö�À Á � ¾ ® Á Ò À Á � Á ® ÁÖ%��x(cL���F^

Ì � Ò ]m � ¾d¾ Ò ]m � ÁdÁ Ö � ¾<Á�� ® ÁÁÖ¡ª�]�ÒÍ�Ö¡Í � ¾d¾ 8?ÒÍ � ¾<Á 8]Ö�À Á � ¾ ÒRÀ Á � Á ¬u® ÁÖ%��x(cL���F^

Ì ]m ªñm � ¾<Á Ò � ¾d¾ Ò � ÁNÁ ¬u® ÁÁÖ¡ª�]�ÒÍ�Ö¡Í � ¾d¾ 8?ÒÍ � ¾<Á 8]Ö�ÀcÁ � ¾�ÒRÀ7Á � Áq¬u®sÁÖ%��x(cL���F^

`Ad=����P�m � ¾<Á Ò � ¾d¾ Ò � ¾<Á ^Øorqed*�=x�d ����b_d=c���xD�¸® Á b��]�ÒÍ�Ö Í � ¾N¾�8?Ò Í � ¾<Á 8]Ö�ÀcÁ � ¾�ÒRÀcÁ � Á

Ì ]�ÒÍ�Ö Í � ¾N¾;ªG® ��� �¾ Ö Í�® ��� �Á ¬*Ò Í � ¾<ÁOªV® ��� �¾ Ö¡Í�® ��� �Á ¬ÖlÀcÁOª � ��� �¾ Ö�� ��� � ÒÀ® ��� �¾ À&¾ � ¾d¾�ÒÞ® ��� �Á À7Á � ¾<Á�¬

]FÕ

Page 11: Sequential Minimal Optimization for SVM Contents 1 Introduction to

Ò�À Á ª � ��� �Á Ö�� ��� � ÒÀ® ��� �¾ À ¾ � ¾=Á ÒÞ® ��� �Á À Á � ÁNÁ ¬Ì ]�ÒÍ�Ö Í � ¾N¾ ® ��� �¾ Ö � ¾N¾ ® ��� �Á ÒÍ � ¾<Á ® ��� �¾ Ò � ¾<Á ® ��� �Á

ÖlÀcÁ � ��� �¾ Ö À7ÁF� ��� � Ò Í � ¾N¾�® ��� �¾ Ò � ¾=ÁF® ��� �ÁÒ�ÀcÁ � ��� �Á ÒRÀ7ÁF� ��� � Ö¡Í � ¾<Á=® ��� �¾ Ö � ÁdÁF® ��� �Á

Ì ]�ÒÍ�Ö ª Í � ¾d¾�Ò Í � ¾<Á�ÒÍ � ¾N¾BÖ Í � ¾<Á�¬�® ��� �¾Ö3ª � ¾N¾EÒÀm � ¾<ÁYÖ � ÁNÁ�¬�® ��� �ÁÖlÀ Á ª � ��� �¾ Ò � ��� �Á ¬

Ì À ÁÁ Ò À ¾ À Á Ö ª � ¾N¾ ÒÀm � ¾<Á Ö � ÁNÁ ¬�® ��� �Á Ö À Á ª � ��� �¾ Ò � ��� �Á ¬Ì À Á ªTÀ Á Ò À ¾ Ö � ��� �¾ Ò � ��� �Á ¬*Ò �Q® ��� �ÁÌ À Á ªdª � ��� �¾ ÒRÀ ¾ ¬*Ò ª � ��� �Á Ò À Á ¬d¬*Ò �u® ��� �ÁÌ À Á ª � ��� �¾ Ò � ��� �Á ¬BÒ �u® ��� �Á Â

hQx���qLd%x(yML­d�����b_×(d���tecH����b_x(c�b��N � Ì ]m �u® ÁÁ Ö¶ª�À Á ª � ��� �¾ Ò � ��� �Á ¬*Ò �u® ��� �Á ¬­® Á Ö ��x(cL���F^

orqed�ÙLg��­�"f(cLs���d���x(cHs�sudFg�b_×?f?�Xb�×~dF��fDgXds N �sL® Á Ì �u® Á Ö ª�À Á ª � ��� �¾ Ò � ��� �Á ¬*Ò �Q® ��� �Á ¬K¿

s Á N �sL® ÁÁ Ì � Â

}#x(��d#�XqLf?� �WÌ�m � ¾<Á Ò � ¾N¾ Ò � ÁNÁ I�Õe^ì¼ØgXxQxD�­¶ì`Ad=� � ¾N¾ Ì£½ E ¾ ½ ¾ ¹ � ¾<Á Ì£½ E ¾ ½ Á ¹� ÁNÁ Ì ½ EÁ ½ Á ^ØorqLd=c �WÌ�Ò�ª�½ Á Ò ½ ¾ ¬ E ªT½ Á Ò ½ ¾ ¬YÌ�Ò CK½ Á Ò ½ ¾ C Á I ÕL^`Ad=� s ���s�� 5

Ì�Õ ¹ef(cLs���d%qLfM×~d

®����Á Ì Ò ÀcÁOª � ��� �¾ Ò � ��� �Á ¬zÒ �Q®���� �Á�Ì ® ��� �Á Ö À Á ª � ��� �Á Ò � ��� �¾ ¬�

��� � ��Õe¹?��qedrfDy§x?×(d�d�Õ�tLfD��b_x(cÉ�~b�×~dF� tL� ��qedrtecH��x(cH�­�XgXf(b�ced�s%z�f;Óub_zÉtez w§x(b_c��¸® ���Á ^ ���zÉtH�­�ryHdl��qed����(dFs�fD�~f(b�cH�­����qed���dFf~��b_yev_d�g�fDce�~d(^ì`Ad=��Í�Ì£À ¾ À Á ¹uf(cLs 8�Ì�®���� �¾ ÖÍ�®���� �Á ^orqedlg�fDce�~d�x(�4® Á b_�"sud��Xd=gXz�b_ced�s�f~����x(v_v�x?�#�F¶ ���hÍlÌT]c¹Q�Xqed=cá®B¾hÖJ®sÁlÌ984^

� ��� 8 � #3¹u�Xqed=c�z�f;Ó�®zÁ1Ì #3¹Lf(cLs�z�b_c#®zÁlÌ38�Ò #�^� ��� 8 � #3¹u�Xqed=c�zOb_c"® Á Ì�Õ ¹ef(cLs�zOfOÓr® Á Ì*84^

���hÍlÌfÒ%]c¹Q�Xqed=cá®B¾�ÒÀ®sÁ1Ì*84^� ��� 8 ��Õe¹Q��qedFcáz�b_c#®zÁ1Ì�Õe¹efDcLs�zOb�c#®zÁ1Ìr# Ò 8ì^

](]

Page 12: Sequential Minimal Optimization for SVM Contents 1 Introduction to

���

��� ����� ������ ��������

Û4b_�(tegXd�]~¶Ø® ¾ Ö® Á Ì98*¹efDcHs 8 � #�^

���

���

������������������

Û4b_�(tegXdlme¶Ø®*¾YÖ®zÁ1Ì98*¹efDcHs 8 � #�^� ��� 8 ��Õe¹Q��qedFcáz�b_c#®zÁ1Ì�Ò 8*¹efDcLs�z�f;Ó�®sÁ1Ìr#�^

`Ad=�r��qed%zOb_ceb�z�tez{��dFf(��b_yev�dl×?fDv_ted%x(�4®zÁlyHd! 1¹uzOfOÓub�z�tez+y§d Ô ^ØorqedFc® cedF��¹ �=v�b_wew§dFsÁ Ì

"# $ Ô ¿ b�� Ô � ® � ��Á ¿® � ��Á ¿¤b��% I�® � ��Á I Ô .¿ b��ì® ���Á �& .Â

oËx���tLz�z�f(g�b_¬=d7¹L�(b_×(dFcæ® ¾ ¹Y® Á ªVfDcHs���qed���x(gXgXdF��wHx~cLsub_ce�WÀ ¾ ¹GÀ Á ¹ � ¾d¾ ¹ � ¾<Á ¹ � ÁNÁ ¹� ��� �Á Ò � ��� �¾ ¬K¹e��d%�=f(c�x~wu��b_zOb�¬Fdl��qedl�­��x�®° ��yQà���qLdl��x(v_v�x?��b_ce��weg�xu�=dFsutegXd(¶]~^ ��Ì�m � ¾<Á Ò � ¾N¾ Ò � ÁdÁ ^me^���� � ��Õ ¹

' ®sÁ1Ì À Á ª � ��� �Á Ò � ��� �¾ ¬� ¿

f(cLs���v_b�w��Xqed*��x~v�te��b_x(c���b��Xqeb�c��Xqedl��dFf~��b_yev_d�gXd=�~b�x~cA^¸orqed=c

' ® ¾ ÌfÒ1Í ' ® Á Â

]�m

Page 13: Sequential Minimal Optimization for SVM Contents 1 Introduction to

���

���

�������

�� ��� ������

Û4b_�(tegXd%|e¶Ø®B¾�ÒÀ®sÁ1Ì*8z¹ef(cLs 8 � ÕL^

���

��� ��������

�!

�#"�%$&�

Û4b_�(tegXdl�L¶Ø® ¾ ÒÀ® Á Ì*8z¹ef(cLs 8 � ÕL^|L^���� �WÌ�Õ ¹u��d�cLd=dFs���xOd=×?f(v�tLfD��d���qedlx~yML­dF����b_×(d���tecL���Xb�x~c�fD���Xqed��­��x�d=cHsuwHx~b�c�����¹f(cLså��d���® ���Á ��xÂy§dæ�Xqed x~cedæ��b���qåv�fDgX�(d=g�x(yML­d�����b_×(d���tLcL���Xb�x~c�×?fDv_ted(^Torqedx~yML­dF����b_×(d���tecL����b_x(c�b��

N � Ì ]m �u® ÁÁ Ö ªTÀcÁcª � ��� �¾ Ò � ��� �Á ¬zÒ �u® ��� �Á ¬�®sÁ�Ö���x~cL�­��^ ªñ O¬

U�®VU µ�¸ ' \ ��`h´e° �aXHc)(+* ^ Z.³�� °<±�� ³ � �^²k´§³ �>[ � � ² � � ��[ \ `Q^ � ° �c�° �³�� °,`Y± � �i²6^

ÊåqedFc�®B¾q¹u®sÁ�fDgXd���qHfDce�~dFsOyQà ' ®B¾O¹ ' ®zÁc¹Q��d��FfDc�tew;sefD��d �#¯ ° ��¹ � ¯ ° ��¹ è ª¹��x~g�v_b�cLdFfDg�(dFg�cLd=v�¬K¹efDcHs��M^Ø`Ëd�� � ªT½B¿<À ¬�y§dl��qed%wegXdFseb_����b_x(c�d=gXg�x~g�x~c ªT½B¿<À ¬�¶

� ª�½B¿<À ¬YÌÃR¯TS ¾ ® ¯�À(¯�½ E¯ ½�ÒÀ�EÒ À ¿

orqed%��qLf(ce�(dlb_c � b��' � ªT½B¿<À ¬YÌ ' ®*¾KÀ&¾!½ E ¾ ½?Ö ' ®zÁ$ÀcÁ�½ EÁ ½�Ò ' �; ªVÏ7¬orqed���qLf(ce�(d�b�cÞ�Xqed���qegXdF��qex(v�sÞ�=fDcÞy§d���x(zOwete��dFsÞyQàá��x~gX�=b�ce� � � ��¾ Ì Õ�b��rÕ �

® ���¾ � # ªìx(g � ���Á ÌåÕOb��¸Õ ��® ���Á � #�¬�^ØÛegXx(zÕ Ì � ª�½B¿dÀ ¬ ���

]F|

Page 14: Sequential Minimal Optimization for SVM Contents 1 Introduction to

Ì � ª�½B¿dÀ ¬ ��� � Ö ' � ª�½B¿dÀ ¬Ì � ª�½B¿dÀ ¬ ��� � Ö ' ® ¾ À ¾ ½ E ¾ ½?Ö ' ® Á À Á ½ EÁ ½�Ò ' �

��d�qHfM×(d' ��Ì � ª�½B¿dÀ^¬ ��� � Ö ' ® ¾ À ¾ ½ E ¾ ½?Ö ' ® Á À Á ½ EÁ ½B ªV²7¬���H® ¾ Ì�Õe¹M��d��=fDc�x(cev_à%��fMà�À ¾ � ���¾ G Õ��(��b�zOb_v_f(g�v_àc¹Mb��H® ¾ Ì #3¹D��dqHfM×(dhÀ ¾ � ���Á I Õe^����yHx(��qJ®B¾�f(cLsÁ®zÁ��Xf(�(d�×?f(v�ted��ÉÕ�x~g_#3¹4��qed�x(gXb��~b�cHfDv�hujÞª+f(v��~x(gXb��Xqez��=x(zOwetu�XdF��­��x�×?fDv_ted���x(�Ë�Xqedlced=� �r��x(g�®B¾�fDcLs�®sÁ�tL��b_ce��ÖEÕ�tLf?�Xb�x~c�² ¹ef(cLs���fD�~dF����qed%fM×~d=g�fD�(d~^orqeb��rb_�#g�dF�~fDg�sud�s�f(��wegXx(yLv�dFzOfD��b���y�à��ld=dFg��XqebYd��"f(vG^kªñmDÕ(ÕL]'¬�^hQb_z�b_v�fDgXv�à7¹Q��g�x~z

�WªT½B¿<À ¬YÌÃR¯TS ¾ ®4¯�À(¯�½ E¯ ½�ÒRÀ

��d�qHfM×(d' �WªT½B¿<À ¬YÌ ' ®B¾!Ài¾!½ E ¾ ½�Ö ' ®sÁ�ÀcÁ$½ EÁ ½B ª�]FÕ7¬Ûex~gr��qed%��d=b_�(q���×(d�����x~grxD�ìv�b_cedFf(g��(dFg�cedFv_��¹

è ÌÃR¯nS ¾ ®4¯�À(¯�½A¯d¿

' è Ì ' ® ¾ À ¾ ½ ¾ Ö ' ® Á À Á ½ Á  ª�](]'¬U�®�� µ�¸ ' \ ��`h´e° �aXHc)( � °��� � � ` �4� � �,��`h´ `Q^ � ° cÑ°��³��^°1`Y±orqed*qLd=tegXb_����b��=�#��x(g#wLb_���Qb_ce���­��x�® ¯�° �#��x(g#x~wu��b_zOb�¬�f?�Xb�x~cáb�c��XqedÉx~g�b_�(b_cLfDv4huj³ªµwLfDw§d=gfDgXdlf(����x(v_v_x?�#�=¶ orqLd�x(te��d=g�v�xQx~wÂ��dFv�d����X�É��qLd�ÙLg��­��® ¯d¹4��qed�b�cecLd=g�v�xQx(wÂ��d=v_dF���X�*��qed���d���x(cHsÁ® ¯�XqLf?�#z�fOÓQb_zOb�¬FdF��� � Á Ò � ¾ �Ý^

orqLd"x(te��d=g�v�xQx~w�fDv���dFg�cLfD��d��Øy§d��­��d=d=c�x~ced�����d=d=wO�XqegXx(te�~q�fDv_vHd�Óuf(zOwev�d��f(cLs�f(�z�f(c�à�����d=d=wH��f~�w§x~�X��b_yev_d#�Xqeg�x~te�(q���qed�cex~cu�ñy§x(tLcLsefDgXàOd$ÓefDzOwev_dF��ª¹�Xqex~��d���b��XqÕ�� ® ¯ � #�¬$¹L��dFv�d����Xb�ce���Xqed%d$ÓefDzOwev_dl��qHf?��×Qb�x~v_fD��dF���Xqed*�%�%o��=x(cLseb��Xb�x~cA^

ß�b_×(dFcÉ�Xqed�ÙLg��­�Ø®4¯<¹?�Xqedrb�cLced=g¸v_xQx(wÉv_xQx(�u� ��x(g¸f�cex(ce�ñy§x(tecHsefDgXà���qLfD�ØzOfOÓub�zOb_¬=dF�� � Á Ò � ¾ ��^µ���%��qLb_��suxQdF��cexD��z�fD�(d weg�x~�(gXdF�X��¹rb������Xf(g�����fJ��d�Õ�tedFc���b�fDv��X�=f(c�XqegXx(te�~q���qedæcLx(cu��yHx~tecLsef(g�àJd$ÓefDzOwev_dF��¹#�­��fDg���b_ce�Âf?��fÀgXf(cLsux(zÜwHx���b���b_x(c��b���Xqeb��*�Vf(b�v��%�Xx�x ¹ b������Xf(g��Xb_�Éfæ��d'Õ�ted=c���b�fDv����FfDcÞ��qegXx(tL�(qÂfDv_vØ�Xqed�d�Óuf(zOwev�d���¹ìfDv���x���Xf(g��Xb�cL�OfD�#fOgXf(cLsux(z w§x~����b_x(cË^

© dF�FfDtL��d���qedÞfDv_�(x(gXb���qez ��w§d=cLse��zOx~����xD����qedæ��b_z�dÞf(s L­tH�­�Xb�ce�À�Xqedæcex~cu��yHx~tecLsef(g�àd$Óef(z�wLv�d���¹Q�Xqed � ¯�° �rx(� �XqedF��dld$ÓefDzOwev_dF��f(g�d%�=f~��qedFsA^orqed�b�zOwegXx?×(dFz�dFc���wegXx(w§x~��dFsÁb_c��ld=d=g���qLb�d=�Of(vG^�ªGm(Õ(ÕL]'¬*fM×~x(b�se�%�Xqed�tL��d�x(�#��qed

��qLg�d���qex~v_s��Øb_c���qedF���Qb_ce���%�%o���x~cLsub���b_x(ck¹(f(cLs���x(zOwLf(g�d��A�­��x � ¯ ° ��¹D��qLb_��qOfDv���xlfDte��xD�z�f?�Xb_�FfDv_v�àÉ��d=v_dF���X�ì�Xqed#��dF��x~cLsO® ¯ ��x(g@L­x(b_c��x(we��b_z�b_¬FfD��b_x(cA^ìorqedFg�d�fDgXd��­��x%×?f(g�b�fDc���b_x(cH���qedFc��Xqedlx(tu�Xd=g#v_xQx(w�sudFf(v_�rx~cev_à���b��Xq���qed%cex~cu��yHx~tecLsef(g�à�d�Óuf(zOwev�d��=¶

]=�

Page 15: Sequential Minimal Optimization for SVM Contents 1 Introduction to

orqLd�ÙHgX���*®�b_�%��d=v_dF����dFsÞ��d�Õ�ted=c��Xb_f(v�v_à���g�x~z¤fDv_vì��qedOcex~cu��yHx~tecLsef(g�àád�Óuf(zOwev�d��=^���r�XqedOÙLg��­�*®4¯�×Qb�x~v_fD��dF���Xqed��%�%oµ�=x(cLseb��Xb�x~cÞ��qLd=cÁ��x~z�wHfDgXdFsæ��b���q® � ��b��Xq� � Ì�� v_x?� ¹ex(g�� � Ì�� tew ¹e��qedFc���d=v_dF���#® � f(���Xqed*��d���x~cLs�®�^

orqLdl�­��x�®�° �#f(g�dlfDv���x��Xqex~��dl��b��Xq � ¯ Ì�� v_x?� x(g � ¯ Ì�� tew ^´��¹�Xd=g�f���tH�=��d�������tevË����dFwætL��b�cL��f�wLfDb_g#x(�ìb_cLsub���d���¹*ª�%IÁc¿�%<¾K¬K¹§v_d����� Ì � � �R¼ %<¾'¿�%IÁOÄ~^�ʳd��v�fDb_zÚ�XqLf?��d�f(��q xD����qedá�­��xÁ��d=�X��¹����� ª � � � � ¾ � � Á'¬Of(cLs����� ª � � � � � � � � ¬$¹rb_�cex~cu�ñdFzOwu�­àc¹uqedFcL��d%��d%�=f(c��=x(zOwetu�Xd%wLfDg���b�fDvA� v_x?� fDcHs�� tew ��gXx(z+��qedl�­��x���d����=^¼g�xQx(�­¶ÉorqedO�­��xæ��d=�X�*f(g�dOcex~cu�ñdFzOwu�­à b�� � ���Ìe^���� � � Ì�^¹Ë��qedFc®*¾;¹®zÁ��FfDcÀx(cev_àæ��fD�~d�×Mf(v�tLdF�l��gXx(z Õ�x(gl#�^�orqed=àÀ�=f(cecexD�%��fD�~d��Xqed��XfDzOd×?f(v�ted%f(cLs�À&¾�Ì ÀcÁlf?����qLdl�XfDzOd���b_zOdc¹ex(��qedFg���b���d���d�qLfM×~d�Õ�ÖÀÕ�Ì*8�x~g# Ö #}Ì98���x(g®B¾�Ö�Í�®zÁ.Ì98*¹~b_c���qeb���q�®*¾#fDcLs�®zÁ"�FfDcecLxD�y§d"��qLf(ce�(d�s�¹��qLb_��q �=x(c���g�f(seb_���X���Xqed*�Vf~������qLfD����d L­tL����qLf~sæf���tL�=�=dF�X�­��tLv4����dFwA^�hQx�b���Xqed=à���fD�(d#��qed"�XfDzOd#×?f(v�ted7¹~��b���q�sub��;d=gXd=c���À(¯­° ��¹~��qLd=à���b�v_vHy§d=v_x(cL�%��x%�­��xseb��;d=gXd=c��"��d��X�F^Ø���ì��qLd=à���fD�~dlseb��;d=gXd=c��#×?fDv_tedF��¹e��b���q���qLdÉ��f(zOd�À~¯­° ��¹u�Xqed=à��b_v_vËfDv���xOy§d=v_x(ce���Xx��­��x�sub �§dFg�dFc~�"��d����=^

� . ¿�¿ � � ���=3-� 3�1�2z¤�2B��/�1}#x?��v�d=�F° ����gXb��Xd��Xqed�� ¿?¿ �=xusudc¹uyLf~��d�s�x~c��Xqed%wL��d=tLsexQ�=xusud%b�c�¼Øv�f?���aª­]�²(²(Ï&¬�^ ����������������� ����! #"

$&% 8K0�j78N[h�i6-8!,B:=>«+-97`!6-Mij78 �('*)�+,+(+.-/ ��0�13241657�986��:653�*8;�=<�>�?$A@ 6/>�gi0�6G_�0�[=+/0�g76/8N, ��B�)�+(+,+.-$DC M&97`N:=+->'9&, ��E! �)9+(+,+.-$ U]0�+-9][=>'Mc:=+/978 ���7F6-G

��®<­ WYX.²IHKJML�Nf´a`3[ � °<±1²orqed Ä;ÅHÆ�Ç�gXx(tu�Xb�cLdØb�zOwev_d=zOdFc~���Y�Xqedx~tu��dFgËv_x�x~wl��qLfD� ��d=v_dF�����Y�XqedØÙLg���� ® ¯e��x(g x(wu�Xb�zOb_¬Ff?���b_x(cË^¸����f(v��Xd=gXcLf?�XdF�ry§d��­��d=dFc�fO����dFd=w���qegXx(te�~q�f(v�v;�Xqed%d$Óef(z�wLv�d���ªGÈ~ÊeÅ�Ä Æ�ÇLÈ�OeÎQÎ�P�PRQ'¬fDcHs�f~�Yz�fDcQà"����dFd=wL�Ëf(�Yw§x~�X��b_yev_dì��qegXx(tL�(q���qedØcLx(cu��yHx~tecLsef(g�à"d$ÓefDzOwev_dF�hªñÈ~ÊuÅMÄ Æ�ÇLÈSOLÎQÎ�P�P�Ti¬�^���efDcld$Óef(z�wLv�dØÑ�×Qb�x~v_fD��dF�Y��qed�%�%oÞ��x~cLsub���b_x(clzOx(gXdì��qLf(cÉÈ(ÍVU7ÌXW'¹�b��Ëb��Ë��d=v_dF���XdFs%f~�;��qedÙLg��­�"®4¯d¹§f(cLsæÈ~ÊuÅMÄ Æ�ÇLÈ(ÌQÊeÅMÄLÍLÎ�ÈMY�ÑVZ�b_�"�=fDv_v_dFs�¹§��qeb���q�gXd��XtegXcL�*]%b��ØwHx���b���b_×(d%wLg�x~�(gXdF�X�b��rzOf~sudl��xOb_zOweg�x?×~d"�Xqed%x(yaL­dF���Xb�×~d���tecL���Xb�x~c ª���b���q��­��x���qLf(ce�(d�s�®4¯�° �!¬�^$ U]0�+-93[=>�M7:=+-9&8 ���7F3-."

0�13<[��5�0�1]\,0�1�<;5*^�2S�`_a�!bS5*^;c�5=^�2�dfeAg�h�i$ bz0$[=+/0�g&6-8!,h6->;`K0�6 :=>«4�0�+/9�j �� �-0�13<k1 / �9l*bS5�13238�>R?0�13<�8*m657�n0�138�o6p�pV?

$A@ 8N:h+-9]5i0$[<0�4�8N:=8d[=,rq!s7t -$&u 8!0'j�+-9Wj70�:<0vj �,w�-

]��

Page 16: Sequential Minimal Optimization for SVM Contents 1 Introduction to

0�� \�� 0����*<�86�=<�����1Sp��9h4i5�p=:�bf� ^�86��0�38V\ 8*13>���� / :�:S�=^�<��30`_�R��h ?

��ck0�1�0�<S0�5�p30�38�<�b3^386�!bS��p*>�<6� 38�^6��c*���� R��?

��c w �30 � / �30������30 � �� ���30 � �����304c��8*^�^6�*^�����53��b68�� ^386��0�38�\ t hn?0�� \,0�����p30�168�5*^����S8*^�1683pSh� � ^�86��0�38V\ >R_�R�,h ?

$ r 97+-:=+/0�6/+-PK0$:=+->'9 q B! �)�+,+�+.-0�� \�� 0����*<�86�=<�����1Sp��9h4i

1 / �9l*bS5�13238�> � V?8*m657�n0�138�o6p�p ��� ?� b�0�p�8 \A1 / �9l*b65=13238�>�� ���� 8*m657�n0�138�o6p�pShki

1 / ��l*bS5=132�8�> � �?0�� \ 8�m657��0�168�o6p�pShki

�6�*^ \�0�13<�� � �? ��! t ? �#"�"�h1 / ��l*bS5=132�8�>$" � 8�m357�n0�168 w m657�S:6p�8 \%�nh ?

�83p3�=8�i

�6�*^ \�0�13<�� � �? ��! t ? �#"�"�h0�� \ 5�p=:�b.e&�6g�� � ('�'�5�p�:�b.e)�6g�� � l�h1 / �9l*bS5�13238�>*" � 8*m657�n0�138 w m657�6:Sp�8 \+�nh ?

�0�� \ 8�m657��0�168�o6p�p ����� h8*m657�n0�138�o6p�p � V?

83p3�=8 0�� \A1 / �9l*b65=13238�> ��� �h8*m657�n0�138�o6p�p ��� ?

���3�=8�^�^$!�!�8�^�^6�*^��*^65*<38V\(h�!�!�8*1�>6pV?$), +/0�\�9&>�,n:=+/`�+-9c@Z>�j B t -

�$&- [=+�:=8�4�>Oj78!6 5&0�[<0�4�8d:=8N[=, j B! �-�=8�^�^$!�!; �<�b3^�86��bS��p=> � !�! � !�!�8*13>3pV?

���� / <�!�!k8�^�^3�*^��*^65=<38�\(h�!�! 8*13>6pV?$&- [=+�:=8�`!6/0�,I,I+-�&`K0�:=+->�9?>�M7:=57M7: j B7w(-

�G

. 7w0/213/2465740/84:9�w:4 t<; 9>= w0/ @? ���! �+

`Ad=�F° ��sud�ÙHcedl��qed%�~v�x~yLfDvY×?f(g�b�fDyev_dF�F^$A@ 6->'g&0�6G_�0�[=+/0�g&6-8!, ��B�-."

]F�

Page 17: Sequential Minimal Optimization for SVM Contents 1 Introduction to

0�13< t � �? ��c t :S��0�13<S�`\ ^6� � �6h4c��0�13<4> � � � ? ��c[>4d65=^S0�5 � p*86��c���6p���5=<�l � R�8��V?�6p���5=<4<6��p�8*^65=1��*8 � R�2� � ?�6p���5=<�8*:�� � R�2� � ?�6p���5=<4< � ������0=2���5��3��� / 5=^38�> ��� ?

d386�=<3�*^�!��6p���5*<�� 5�p=:�b]? ��c��S5*2�^65=13238[� / p*<S0�:6p30*8�^S� c���6p���5=< � ? ��c[<�b3^�86��bS��p=> c*�d386�=<3�*^�!��6p���5*<�� � ? ��c � 860�2�b3<�d386��<6�*^�� �=16p����6�*^�p30�16835=^(�S8�^�1383p c��

d386�=<3�*^�!��6p���5*<���8�^�^3�*^��3��5���b68�?

�=<�^ / �=<;��:S5=^S�*8�� � 0�1S5*^����*d386�=<3�*^ id�86�=<6�*^�!30�13<�� 0�>R?

� ?�=<�^ / �=<;��:S5=^S�*8��*d�86�=<6�*^ i

d�86�=<6�*^�!30�13<�� 0�>R?d�86�=<6�*^�!��6p���5=<��4d65�pV?

� ?<���:68*>38���d386��<6�*^�!��3p���5*<��4>38=1��*8��*d�86�=<6�*^�?

� ����p40����3��:65*^S�*8��=>65*<65 � �65�p3�*8�?� ����p40���� � 0!1S5*^�� � �35�p3�*8�?�3c / �*8��=1Sp�� �=168����4<�b686�=8 c��d386�=<3�*^�!3��:65*^S�*8�� � 0�1S5*^����*d386��<6�*^�� �!:S5*^S�*8�� � 0�1S5=^����=:S��0�13<S�`?d386�=<3�*^�!3��:65*^S�*8��=d386�=<6�=^�� ��:65*^S�*8���:S�30�13<6�`?d386�=<3�*^�!*>38=1��*8��*d�86�=<6�*^��;>38*1S�*8��=:S��0�13<S�`?

d386�=<3�*^�!30�1�<��k<65*^�2�8�<R? ��ck��p�5����4p�5 � 8�p3�k���4<�^6530�1�0!132�>65*<65[:S�30�13<6��c��� ����p40����*<383�=<����=16p�� � �65�p��*8�?� ����p40�����p30!16835*^�� �S8�^�168�p � �65�p3�*8V?

�3c[>35*<65�:S�30!13<S� � 0�<�b 0�13>�8�m 0�1 e%�S0=^S�=<��*<386�=<��30 ��� t hc � 0�p�p � 8 <386�=<38*>�<6�����7�6: / <3848*^�^6�*^�^65=<38c��0�13<��60=^S�=<��=<386�=<���0 � �?

�3cck� / :�:S�*^�<�d386�=<6�=^S�45*^38 � 0=<�b�0�1 e+R��� 8*1�>��3� / :�:S�*^�<���06hc��0�13<�8=13>��3� / :�:S�*^�<��30 � � � ?

G. 7w0/21 t 4�9�4 t F�� = w0/ @?�= ��B�)�� s �) q �,F9) q!q w�) q!j F9) q B7F9) q!s w�+. 7w0/213/2465740/84:9�w:4 t<; 9>= w0/ @? ���! �+

$&% 8!0'j78d[B�i6-8!,B:=>«+-9&`!6-M&j78 �('!-]" 0�1���p / >38$!*d383�=<6�*^�� 0�1���p / >38$!�5�p=26�*^S0=<*b=� �

]M 

Page 18: Sequential Minimal Optimization for SVM Contents 1 Introduction to

0�1���p / >38$!�� / 1��=<S0���1S5�p��G

. 7w0/21 t 4�9�4 t F�� = w0/ @?�= �('*) q!s �) j �,F9) j!j +. 7w0/213/2465740/84:9�w:4 t<; 9>= w0/ @? ���! �+

��®VU WYX.²����.J H#L�N������ J H����f´a`�[ �^°d±.²ß�b_×(d=cÂ��qed�ÙLgX����®4¯3ª���b���qJb_cLsud�ÓåÆnQq¬$¹È(ÊeÅMÄ Æ�ÇeÈ~ÌQÊeÅMÄeÍHÎ�ÈMY�Æ Q�Z*ÙHgX������qed����u�*b��"b���×Qb��x(v�f?�XdF���Xqed��%�%o[�=x(cLseb��Xb�x~c³yQà zOx~g�dO�XqLfDcÀÐ��uÎ~È� eÅ(Ç��DÈ ¹;b���b���suxQdF��¹A�Xqed=cÀv_xQx(�u�l��x(g��qLdÀ��dF�=x(cLs�® ¯CªV��b��Xq�b_cLsud�ÓTÆ��i¬�fDcLs L­x(b_c���v_à�x(wu�Xb�zOb_¬=d³��qedÞ�­��x ®4¯­° ��yQà��=f(v�v_b�cL�ÐeÅ~ÑuÈQÒ~ÐeÈ(Í YFÆ Q��=Æ��`ZQ^$DC M79&`N:=+->'97, ��E! �-."

0�13<�8�m357�n0�168 w m657�S:6p�8�\,0�1�< 0 � hi�6p���5=<�� � _ 5�p=:�b � _ w � _�^ � ?� � � <65*^�238*<fe 0 � g�?5�p=:�b � � 5�p=:�b.e 0 � g�?

0�� \ 5�p=:�b � � �'�' 5�p=:�b � ! l�hw � � 8�^�^3�*^��3��5���b68]e 0 � gR?83p3�*8w � � p�835=^�168�>���� / 1��`\�0 � h��(� � ?^ � � � � c w � ?0�� \�\ ^ � !��=<6��p�8�^35=1��*8�'�'45�p=:�b � ! l�h���[\ ^ � �[<6��p�8*^65=1��*8�'�'45�p=:�b � �(�h�hi��c���^�� 0 � � �4<*b3^38�8 � 5��S�`? 0��;� / ���=86����� / pV_ <�b68*1 0(���98�>S0*5*<383p�� ^38*< / ^�1 � ? c��$ (e[I�30$[=\'4�0��aw � � w � ��E7F3-$ (e[I��+�:=8N[<0$:=+-9&\�:=)c[=>'M&\�)3:=)&8�9&>'9cyTg^>'M&9&ja8��&0�4�5&6-8!, � s F6-$ (e[I��+�:=8N[<0$:=+-9&\�:=)c[=>'M&\�)3:=)&8�8!9;:=+-[=8�:I[<0�+-9&+-9&\«,I8d: � s w(-

^38�< / ^�1$�?� G

. 7w0/21 t 4�9�4 t F�� = w0/ @?�= ��E! �) q�� ) q�� F9) q �! 7FSw!) q B t ) q '7 7F9) q E7F9) j!q ) j�� ) j ��) j B7F�+. 7w0/213/2465740/84:9�w:4 t<; 9>= w0/ @? ���! �+

·"��d���qed�qLd=tegXb_����b�����xæ��qexQx���d���qed���dF�=x(cLs d$ÓefDzOwev_d���g�x~z¤cex~cu�ñy§x(tLcLsÞd$ÓefDzOwev_dF��¹Ë��x��qHf?�#Ì]Q��DÌ���b���z�f;Óub�zOb_¬=d�sY^$ ( [I��0�[=\�4�0���w � � w � ��E7F6-f"

i0�13<(�]_ 0 � ?�6p���5=<4<���5*m�?

�6�*^ \�0 ��� \�� � h _�<!��5*m � V_ � � V? ��! 8*13>��3� / :�:S�*^�<��30`? �#"�"�h0�� \�5�p=:�bfe)�6g*�(('�' 5�p=:�b.e)�6g$!�l�h[i

]FÏ

Page 19: Sequential Minimal Optimization for SVM Contents 1 Introduction to

�6p���5=< w � _�<�8!�S:]?w ��� 8�^�^6�*^��3��53��b38]e)�6gR?<38!�S: � �65 � � \ w � � w � h ?0�� \ <�8!�S:*��<!��5*m9hi<!��5*m � <�8!�S:]?0 ��� �f?

��

0�� \�0 � � � Sh i0�� \ <65 �S8��*<38=: \�0 � _ 0 � h�h^38�< / ^�1 � ?

�� G

. 7w0/213/2465740/84:9�w:4 t<; 9>= w0/ @? ��E! �+

$A@ 6->'g&0�6G_�0�[=+/0�g&6-8!, � s �-."0�13<4<65 �S8��*<38=:]\,0�13< 0 � _ 0�13<;0 � h ?G

. 7w0/21 t 4�9�4 t F�� = w0/ @?�= ��B�)�� s �) q �,F9) q!q w�) q!j F9) q B7F9) q!s w�+. 7w0/213/2465740/84:9�w:4 t<; 9>= w0/ @? ���! �+

������dá�=f(cecexD�Oz�fD�~d�wLg�x~�(gXdF�X�*��b���q ��qed�y§dF����cex(ce�ñy§x(tecHsJd�Óuf(zOwev�d7¹Ø�Xqed=c���gXàJfDcQàcex~cu�ñy§x(tLcLs�d�ÓefDzOwev_dF�F^$ ( [I�a+�:=8N[<0�:=+-97\�:=)7[=>�M&\')�:=)&8�97>'97yTg^>�M&9ij�8 �70�4�576-8!, � s F6-f"

i0�13<(�]_ �#�?0�13<;0 � ?

�6�*^ \%�# � \�0�13<9h \ >�^65=13>���� \(h�c�8=13>��3� / :�:S�*^�<��306h _ � � �#�? �$!k8*13>���� / :�:S�=^�<��30�" �#V? �#"�"Sh�i0 �(� ���48*13>���� / :�:S�=^�<��30`?0�� \�5�p�:�b.e 0 � g�� �'�'45�p=:�b.e 0 � g*!kl�h i

0�� \ <65��S8��*<38*:]\�0 � _�0 � h�h^38�< / ^�1 � ?

��

� G. 7w0/213/2465740/84:9�w:4 t<; 9>= w0/ @? ��E! �+

���e��d��=f(cecex(� z�fD�~dØwegXx(�(gXdF�X�Y��b���qÉ��qed�cex(cu��y§x(tecLs*d$ÓefDzOwev_dF��¹���qedFc*��gXà%fDcQà�d$ÓefDzOwev_d(^$ ( [I�a+�:=8N[<0�:=+-97\�:=)7[=>�M&\')�:=)&8�8!9;:=+�[=8�:I[<0�+-97+/97\«,I8N: � s w(-."

i0�13<(���_ �f_�0 � ?

�6�*^ \%�# � \�0�13<9h \ >�^65=13>���� \(h�c 8=13>��3� / :�:S�*^�<��306h _ � � ���? �*!�8*13>��3� / :�:S�*^�<��30�"��#V? �#"�"�h[i0 �(� ���48*13>���� / :�:S�=^�<��30`?0�� \ <65 �S8��*<38=:]\,0 � _ 0 � h�h

]F²

Page 20: Sequential Minimal Optimization for SVM Contents 1 Introduction to

^38�< / ^�1 � ?�

� G. 7w0/213/2465740/84:9�w:4 t<; 9>= w0/ @? ��E! �+

��®�� WYX.²�� J�� ����� � � ´a`3[ � °<±1²}#x?��v_d���° ����g�b���d�ÐeÅ(ÑeÈQÒ~ÐeÈDÍ*��qLb_��q�x~wu��b_zOb�¬FdF��­��x�`ËfD�~gXf(ce�(drz�tev��Xb�wLv�b_d=g��=^ì���Ë��tL�=�=dF�X�­���tev ¹egXd���tLg�cÞ]c¹udFv_��d%g�d=��tegXc�Õe^$DC M79&`N:=+->'97, q�� -."

0�13<4<65 �S8��*<38=:]\,0�13< 0 � _ 0�13<;0 � h i0�13<�� � _ � � _��`?�6p���5=< 5�p=:�b � _ 5�p=:�b � ? �3ck��p*>��*d65�p / 86�4����5�p=:�bS5�� � _ 5�p=:�bS5�� � c���6p���5=< 5 � _ 5 � ? �3c[138 � d65�p / 86�k����5�p=:�bS5�� � _ 5�p=:�bS5�� � c���6p���5=< w � _ w � _ �]_��R_ � ��� _ � � � _ � ��� _�8�<35V_ �S� �� _��S� �� ?0�� \�0 � ��� 0 � h ^38�< / ^�1*�?

$ ~ >;>��M75 5�p=:�b � _ � � _ w � _ 5�p=:�b � _ � � _ w � q �� �-� � � � c � � ?

$ 2z>'4�57M7:=8�~ � % q!q �-0�� \ � ��� ��h^38*< / ^�1*�?

$ 2z>'4�57M7:=8 8*<65 q!q F6-0�� \ 8�<65�!(Sh i

5 �(� 5�p�:�b � "�� � c \ w � � w � h ��8�<35V?0�� \�5 � ! ��h5 ��� �]?

8�p3�*8 0�� \�5 � ����h5 ��� �]?

�83p3�*84i$ 2z>�4�5&M7:=8�~ >'g��� % >�g����k>'g�n8!`N:=+-_q8�@ M&9&`d:=+/>�9W0�:E0�����~��&0���� % q!q7t -0�� \ �S� ��� ���S� ��� "�8*:��6h5 �(� �]?

83p��*8 0�� \ �S� ��� !��S� ��� �*8*:��6h5 �(� �]?

83p��*85 �(� 5�p�:�b � ?

0�� \8�65 � �`\ 5 � ��5�p�:�b � h�!k8*:���c`\�5 � "�5�p=:�b � "�8*:��6h�h^38*< / ^�1*�?

5 � � 5�p=:�b � ���kc \ 5 � �k5�p=:�b � hn?0�� \ 5 � !��h i

5 � " � �4c�5 � ?

mDÕ

Page 21: Sequential Minimal Optimization for SVM Contents 1 Introduction to

5 � � �?�83p3�*8�0�� \�5 � �kl�h i

�6p���5=<4< � 5 � ��l�?5 � " � �4c[<�?5 � � l�?

$�� 5 j70�:=8�:=)c[=8!,I)&>�6{j�:=>�[=8��&8!`N:Y`d)&0�97\'8�+-9]~ 0�\�[<0�9&\�814lM&6�:=+-5&6-+-8N[=, q!j �-$�� 5 j70�:=8�|z8!+-\');:Y_q8!`N:=>�[B:=>�[=8��i8N`N:Y`d)&0�9&\�8�+-9]0cm10�9ij�0���&+�@�6-+/978K0�[�Scb�U q!j w(-$�� 5 j70�:=8�8N[I[=>�[Y`K0�`<)&8�M7,I+/97\«9&8d|È~ 0�\�[<0�9&\�8.4�M&6�:=+-5&6-+-8N[=, q�� �-5�p=:�bfe 0 � g � 5 � ? ��c �*<6�*^�8�5 � 0�14<�b68�5�p�:�bS545*^�^65��]��c*�5�p=:�bfe 0 � g � 5 � ? ��c �*<6�*^�8�5 � 0�14<�b68�5�p�:�bS545*^�^65��]��c*�

^38�< / ^�1 � ?� G

. 7w0/21 t 4�9�4 t F�� = w0/ @?�= ��E! �) q�� ) q�� F9) q �! 7FSw!) q B t ) q '7 7F9) q E7F9) j!q ) j�� ) j ��) j B7F�+. 7w0/213/2465740/84:9�w:4 t<; 9>= w0/ @? ���! �+

´"�Ø��qed�huj³ª�fDv_�(x~g�b���qez0��w§d=cHse�zOx��­�x(�Yb����¸��b_zOd"x~c�f(s L­tL����b_ce�%�Xqed"® ¯ ° �Øx(�;��qed�cex(cu�y§x(tecLsLfDgXà�d�ÓefDzOwev_dF��¹;f(c dFg�gXx(g��=f(��qLdÉb��"z�fDb_c��Xf(b�ced�sá��x~g"�Xqed=z�^ÉÖ�f(��q���b_zOd�f?�¹��dFg�f��tL�=�=dF�X�­��tevYx~wu��b_zOb�¬�f?�Xb�x~c�����dFwk¹u��x~gr��qed%�­��x�® ¯ ° ��¹ub��ØÕ�� ® ¯ � #3¹e��qLd*��x(gXgXdF��wHx~cLsub_ce��#¯ b_����d=�ɬ=dFg�xH^�orqLd �#¯ ° �*��x(gÉx(��qedFgÉ® ¯ ° �?ª���qLfD��qLfM×~d�yHdFd=cÂ�~d=wu�*ÙeÓQd�sÁsutegXb�cL�á��qedx(we��b_z�b_¬FfD��b_x(c��­�Xd=w ¬�b���tew;sefD��dFs�tL��b_ce��ÖYÕ�tLf?�Xb�x~c�ÏL^$ ~ >;>��M75 5�p�:�b � _ � � _ w � _�5�p=:�b � _ � � _ w � q �� �-f"

5�p=:�b � � 5�p=:�b.e 0 � g�?� � � <65*^�238*<fe 0 � g�?0�� \ 5�p=:�b � � �'�' 5�p=:�b � ! l�hw � � 8�^�^3�*^��3��5���b68]e 0 � gR?83p3�*8w � � p�835=^�168�>���� / 1��`\�0 � h��(� � ?5�p=:�b ��� 5�p=:�b.e 0 � g�?� �(� <65*^�238*<fe 0 � g�?0�� \ 5�p=:�b � � �'�' 5�p=:�b � ! l�hw ��� 8�^�^3�*^��3��5���b68]e 0 � gR?83p3�*8w ��� p�835=^�168�>���� / 1��`\�0 � h��(� � ?

G. 7w0/213/2465740/84:9�w:4 t<; 9>= w0/ @? q�� +

$A@ 6->'g&0�6G_�0�[=+/0�g&6-8!, q �,F6-f"�6p���5*< \,c�p�835=^�168�>���� / 1��6hn\,0�13<9h � t'u � �]?G

. 7w0/21 t 4�9�4 t F�� = w0/ @?�= ��B�)�� s �) q �,F9) q!q w�) q!j F9) q B7F9) q!s w�+. 7w0/213/2465740/84:9�w:4 t<; 9>= w0/ @? ���! �+

��x(zOwete��d���qedl��d�f(��b�yLv�d%g�fDce�~d�x(�ì® ���Á ^hQd=d��Xqed%�(g�fDweqH��x~cἸfD�(d�²e^

mu]

Page 22: Sequential Minimal Optimization for SVM Contents 1 Introduction to

$ 2z>'4�57M7:=8�~�� % q!q �-."

0�� \8� � ��� � � h i�3p���5*<�265�����5 � 5�p=:�b � "k5�p=:�b � ?0�� \ 2657����5���l�h[i

� � 2657����5���l�?� � l�?

�8�p3�*8 i

� � �?� � 2657����5`?

��83p3�=8�i

�6p���5*<k2657����5 � 5�p=:�b � �45�p=:�b � ?0�� \ 265�����5���Sh[i

� � �?� � l���2357����5V?

�83p3�*8�i

� � �*2657����5V?� � l�?

��

G. 7w0/213/2465740/84:9�w:4 t<; 9>= w0/ @? q�� +

$ 2z>'4�57M7:=8 8�<65 q!q F6-f"� ��� � �S8�^�1383p���� / 1��`\,0 � _ 0 � h ?� ����� �S8�^�1383p���� / 1��`\,0 � _ 0 � h ?� � ��� �S8�^�1383p���� / 1��`\,0 � _ 0 � h ?8�<65 � � c � ��� � � ��� ��� � � ?

G. 7w0/213/2465740/84:9�w:4 t<; 9>= w0/ @? q�� +

$A@ 6->'g&0�6G_�0�[=+/0�g&6-8!, q!q w(-."�6p���5*< \,c �S8�^*1683p���� / 1��6h \�0�13<R_,0!13<9h � t�u � �]?G

. 7w0/21 t 4�9�4 t F�� = w0/ @?�= ��B�)�� s �) q �,F9) q!q w�) q!j F9) q B7F9) q!s w�+. 7w0/213/2465740/84:9�w:4 t<; 9>= w0/ @? ���! �+

hQdFdlÖEÕ�tLf?�Xb�x~cá Éx(c�¼¸fD�~d�]�|���x(g�d=×?f(v�tLfD��b_ce� N � f?�"® Á ^$ 2z>'4�57M7:=8�~ >'g��� % >�g����s>�g�� 8!`N:=+-_q8�@ M&9&`d:=+/>�9W0�:E0�����~��&0���� % q!q7t -f"

i�6p���5*< � ��� 8�<65�� � ?�6p���5*< � � � � � cI\ w � � w � h��48�<654c�5�p=:�b � ?�S� ����� � � c �;c���" � � c �]?�S� ����� � � c��;c ��" � � c��]?

� G

m(m

Page 23: Sequential Minimal Optimization for SVM Contents 1 Introduction to

. 7w0/213/2465740/84:9�w:4 t<; 9>= w0/ @? q�� +

hQdFd*ÖYÕ�tLf?�Xb�x~c�²Ox~cἸfD�(dO]F����x~g#tew;sef?�Xb�cL����qed*��qegXdF��qex(v�s��"��qLd=cád=b���qLd=g#x(�ì® ¾ fDcHs® Á fDgXd�cex~cu��yHx~tecLsef(g�à~^$�� 5 j70�:=8�:=)c[=8!,I)&>�6/ja:=>�[=8��&8!`N:Y`<)i0�97\'8�+-9]~ 0�\�[<0�9&\�8.4�M&6�:=+-5&6-+-8N[=, q!j �-f"

i�6p���5=< � � _ � � _ � 168 � ?

0�� \ 5 � �� '�' 5 � ! l�h� 138 � �(� " w � "(� � cI\ 5 � �k5�p=:�b � hkc�� ��� "(� � c \ 5 � �k5�p=:�b � hkc�� ��� ?

83p3�*84i0�� \�5 � ��('�'�5 � !�lSh

� 168 � � � " w � " � � cI\ 5 � �45�p�:�b � h�c � ��� " � � cI\ 5 � �45�p�:�b � h�c � � � ?8�p3�*8�i

� ���(� " w � "�� � c \�5 � �k5�p=:�b � hkc � ��� "�� � c \�5 � �k5�p=:�b � hkc � ��� ?� � �(� " w � "�� � c \�5 � �k5�p=:�b � hkc � ��� "�� � c \�5 � �k5�p=:�b � hkc � � � ?� 168 � � \ � � " � � h � � ?

��

>383p*<35�� �*� � 168 � � � ?���(� 168 � ?

� G. 7w0/213/2465740/84:9�w:4 t<; 9>= w0/ @? q�� +

$A@ 6->'g&0�6G_�0�[=+/0�g&6-8!, q!j F6-f"�6p���5*<k>383p*<65�� � ?G

. 7w0/21 t 4�9�4 t F�� = w0/ @?�= ��B�)�� s �) q �,F9) q!q w�) q!j F9) q B7F9) q!s w�+. 7w0/213/2465740/84:9�w:4 t<; 9>= w0/ @? ���! �+

´�v_b�ced�fDghQi�j��=f(c�y§d#��wHd�s�tLw�y�àÉx(cev_àÉtL��b_ce�l��qed���d=b_�(q��¸×(d����Xx(g.ª�g�f?�Xqed=gì�XqLfDc�f(v�vex(���qLd%��g�fDb_ceb_ce��d$ÓefDzOwev_dF�#��qLfD����x~g�gXdF��w§x(cLs���x�cLx(cu��¬=dFg�x�`Ëf(�(g�fDce�~dlzÉtev���b_wev_b�dFgX�!¬r��qedFcd=×?f(v�tLfD��b_ce����qed%v_dFf(g�cLdFs��=v_f~����b�ÙH�=fD��b_x(c���tecL���Xb�x~cA^���H�Xqed0L­x(b_c��ìx~wu��b_zOb�¬�f?�Xb�x~c���tL�=�=d=d�se��¹D��qeb��¸�­�Xx(gXdFsÉ��dFb��~q~�ì×(d����Xx(gìzÉtL���¸yHdrtew;sefD��dFs

��xOgXd��Ld������Xqedlced=� `Ëf(�(g�fDce�~d"zÉtLv��Xb�wev_b_d=g#×?fDv_ted��=^hQdFd%ÖYÕ�tLf?�Xb�x~c³]~]�x~cἸfD�(d�]=�H^$�� 5 j70�:=8�|z8!+-\');:Y_q8!`N:=>�[B:=>�[=8��&8!`N:Y`d)&0�97\'8�+-9]0Om10�9ij�0�� �i+�@�6-+-9&8K0$[�SOb�U q!j w(-."

0�� \�0�����p30!16835*^�� �S8�^�168�pSh i�6p���5*<�< ��� � � cI\�5 � ��5�p=:�b � h ?�6p���5*<�< � � � � cI\�5 � ��5�p=:�b � h ?

0�� \,0����3�!:S5*^S�*8��*>65*<65�'�'�0���� � 0!1S5*^��9hki0�13<k: � _ 1 / � � _ : � _A1 / � � ?

1 / � � � ��:S5*^6�*8�� � 0!1S5*^�����:S�30�13<6��e 0 � gR��0=>]����0�38�\(h ?�6�*^ \A: � � �?�: � !=1 / � � ?�: � "�"Sh

� e ��:S5*^6�*8�� � 0!1S5*^�����:S�30�13<6��e 0 � gR��0=>.eD: � g3g " � < � ?

mD|

Page 24: Sequential Minimal Optimization for SVM Contents 1 Introduction to

1 / � �(� ��:S5*^6�*8�� � 0!1S5*^�����:S�30�13<6��e 0 � gR��0=>]����0�38�\(h ?�6�*^ \A: ��� �?�: � !=1 / � � ?�: � "�"Sh

� e ��:S5*^6�*8�� � 0!1S5*^�����:S�30�13<6��e 0 � gR��0=>.eD: � g3g " � < � ?�83p��*8 0�� \�0����3��:65*^S�*8��=>65*<65�'�' � 0���� � 0!1S5*^��9hki

0�13<k: � _ 1 / � � _ : � _A1 / � � ?

1 / � � � ��:S5*^6�*8��=:S��0�13<S��e 0 � g]��0�>]����0��8�\(h ?�6�*^ \A: � � �?�: � !=1 / � � ?�: � "�"Sh

� e ��:S5*^6�*8��=:S��0�13<S��e 0 � g]��0�>feD: � g�g " �< � c���:S5*^6�*8��=:S��0�13<S��e 0 � g]� d35�pReD: � gR?

1 / � �(� ��:S5*^6�*8��=:S��0�13<S��e 0 � g]��0�>]����0��8�\(h ?�6�*^ \A: ��� �?�: � !=1 / � � ?�: � "�"Sh

� e ��:S5*^6�*8��=:S��0�13<S��e 0 � g]��0�>feD: � g�g " �< � c���:S5*^6�*8��=:S��0�13<S��e 0 � g]� d35�pReD: � gR?

�83p��*8

�6�*^ \�0�13<;0 � �? 0�!*>�? 0�"�"�h� e 0=g�" � >38*1��*8��=:S�30�1�<S��e 0 � gfe 0=g c < � " >38*1��=8��=:S�30!13<S��e 0 � gfe 0=g c[< � ?

� G. 7w0/213/2465740/84:9�w:4 t<; 9>= w0/ @? q�� +

hQdFdlÖEÕ�tLf?�Xb�x~c�Ï�x(c�¼¸fD�~d�]�|e^$�� 5 j70�:=8�8d[I[=>�[Y`K0�`<)&8�M7,I+-9&\«978N|¡~G0�\�[<0�97\'8.4�M76-:=+-576/+-8N[=, q�� �-f"

i�6p���5*<�< ��� � � cI\�5 � ��5�p=:�b � hn?�6p���5*<�< � � � � cI\�5 � ��5�p=:�b � hn?

�6�=^ \,0�13< 0 � �? 0�!�8*13>���� / :�:S�=^�<��30`? 0�"�"Sh0�� \��!k5�p=:�b.e 0�g�'�' 5�p=:�b.e 0=g$!�l�h

8�^�^6�=^��3��53�!b68]e 0=g�" � < � c �68�^�1683p���� / 1�� \,0 � _,03h�" < � c �68�^�1683p���� / 1�� \,0 � _,03h��>383p*<35�� � ?

8�^�^6�*^��3�*53��b68]e 0 � g � R��?8�^�^6�*^��3�*53��b68]e 0 � g � R��?

� G. 7w0/213/2465740/84:9�w:4 t<; 9>= w0/ @? q�� +

��®�� ���z³ \ [.³�� °<±�� �)\=³ � �i°�� � ³�� °,`E± ��[1± ��� °,`Y±ÊÞdÁtL��dÂf ��tLcL���Xb�x~c wHx~b�c���dFg³Î�È�Å� �ÇLÈ�����QÇ��³�Xxåg�dFweg�d���dFc����XqedÁv_dFfDgXced�s���tecH����b_x(cË ª�½H¬hÌ è�Ð ½�ÒÀ�?^¸�����XfD�~dF����qLd%b�cLsed$Ó�x(�Ë�Xqed*sef?��fOwHx~b�c�� � ¹Lf(cLs���x(zOwete��dF� Ë ª�½ �O¬�^´"�F��x(g�sub_ce����x���qLd��(d=gXcedFv �­àQw§d�fDcLs³b_cewetu��sLf?�Xf��­àQw§dc¹Ë��d�sud�ÙHced���qedO��x~v�v_x?��b�cL�

��tecL����b_x(cL����x(g�d=×?f(v�tLfD��b_ce����qed%v_dFf(g�cLdFs��=v_f~����b�ÙH�=fD��b_x(c���tecL���Xb�x~cA^$DC M79&`N:=+->'97, q�� F6-f"

m?�

Page 25: Sequential Minimal Optimization for SVM Contents 1 Introduction to

�6p���5*<;p*835*^�168*>���� / 1S����p30�13835*^��3�!:S5*^S�*8�� � 0�1S5�^��R\,0=13<���h i�6p���5=<;� � ���?

�6�*^ \�0�13<;0 � �? 0�!3�!:S5*^S�*8�� � 0�1S5=^����=:S��0�13<S��e&�6g]��0=>R�,��0�38�\�hn?[0�"�"�h�(" � � e �!:S5*^S�*8�� � 0�1S5=^����=:S��0�13<S��e&�6g]��0=>.e 0=g�g]?

�(� � � ?^38�< / ^�1 �`?

� G. 7w0/21 t 4�9�4 t F�� = w0/ @?�= ��E! �) q�� ) q�� F9) q �! 7FSw!) q B t ) q '7 7F9) q E7F9) j!q ) j�� ) j ��) j B7F�+. 7w0/213/2465740/84:9�w:4 t<; 9>= w0/ @? ���! �+

$DC M79&`N:=+->'97, q �! �-."�6p���5*<;p*835*^�168*>���� / 1S����p30�13835*^��3�!:S5*^S�*8��=1S�=1 � 0!1S5*^��R\,0!16<��nh i�6p���5=<;� � ���?

�6�*^ \�0�13<;0 � �? 0�!3�!:S5*^S�*8��=:S�30�1�<S��e)�6gR��0=>]����0�38�\(hn? 0�"�"�hi

0!13< �(� ��:S5*^6�*8��=:S��0�13<S��e&�6g]��0=>]e 0=gR?�3p���5*<4d � ��:S5=^S�*8��=:6�30�13<S�Ve)�6g]� d35�pRe 0=g�?��" � � e � g;c[d�?

��(� � � ?^38�< / ^�1 �`?

� G. 7w0/21 t 4�9�4 t F�� = w0/ @?�= ��E! �) q�� ) q�� F9) q �! 7FSw!) q B t ) q '7 7F9) q E7F9) j!q ) j�� ) j ��) j B7F�+. 7w0/213/2465740/84:9�w:4 t<; 9>= w0/ @? ���! �+

$DC M79&`N:=+->'97, q �7F6-f"�6p���5*<;p*835*^�168*>���� / 1S����p30�13835*^��*>�8*1��*8�\�0�13<��nh[i�6p���5=<;� � ���?

�6�*^ \�0�13<;0 � �? 0�!*>�? 0�"�"�h�(" � � e 0�g;c >38*1S�*8��=:S��0�13<S��e&�6gfe 0=g�?

�(� � � ?^38�< / ^�1 �`?

� G. 7w0/21 t 4�9�4 t F�� = w0/ @?�= ��E! �) q�� ) q�� F9) q �! 7FSw!) q B t ) q '7 7F9) q E7F9) j!q ) j�� ) j ��) j B7F�+. 7w0/213/2465740/84:9�w:4 t<; 9>= w0/ @? ���! �+

$DC M79&`N:=+->'97, q �7w(-."�6p���5*<;p*835*^�168*>���� / 1S���=1S�=16p30�16835=^R\,0�13<*�nh[i�6p���5=<;� � ���?�6�*^ \�0�13<;0 � �? 0�!�8=13>��3� / :�:S�*^�<��30`? 0�"�"�h

0�� \�5�p�:�b.e 0=g*�(�h� " � 5�p=:�bfe 0=gSc=<35*^�238�<]e 0=gSc �68�^�1683p���� / 1�� \�0`_+�nh ?

�(� � � ?^38�< / ^�1 �`?

� G. 7w0/21 t 4�9�4 t F�� = w0/ @?�= ��E! �) q�� ) q�� F9) q �! 7FSw!) q B t ) q '7 7F9) q E7F9) j!q ) j�� ) j ��) j B7F�+

m(�

Page 26: Sequential Minimal Optimization for SVM Contents 1 Introduction to

. 7w0/213/2465740/84:9�w:4 t<; 9>= w0/ @? ���! �+

��teg�b_ce�Ob_ceb��Xb_f(v�b_¬FfD��b_x(ck¹L��dlw§x(b_c~��Î�ÈQÅ� �ÇeÈ�� ���Ç��"x~cedlxD�4��tecL����b_x(cL��y§d=v_x?�%^$ r 97+�:=+{0�6-+/P!0�:=+->'9 q B! �-f"

0�� \,0�����p�0�16835*^����S8�^�1383p$'�';0����3�!:S5*^S�*8��*>65*<65�'�' 0���� � 0!1S5*^��9hp�835*^*168�>���� / 1�� � p�835=^�168�>���� / 1�����p30�16835=^��3��:S5=^S�*8�� � 0�1S5�^��R?

0�� \,0�����p�0�16835*^����S8�^�1383p$'�';0����3�!:S5*^S�*8��*>65*<65�'�' � 0���� � 0�1S5*^���hp�835*^*168�>���� / 1�� � p�835=^�168�>���� / 1�����p30�16835=^��3��:S5=^S�*8��=16�=1 � 0�1S5=^��]?

0�� \,0�����p�0�16835*^����S8�^�1383p$'�' � 0�������:S5*^S�=8��*>65*<35Shp�835*^*168�>���� / 1�� � p�835=^�168�>���� / 1�����p30�16835=^��*>38*1S�*8�?

0�� \�� 0�����p30�16835=^����S8�^*1683pShp�835*^*168�>���� / 1�� � p�835=^�168�>���� / 1�����1S�=1Sp30!16835*^R?

G. 7w0/21 t 4�9�4 t F�� = w0/ @?�= q B! 7w!) q E! �) q!s F9+. 7w0/213/2465740/84:9�w:4 t<; 9>= w0/ @? ���7F9+

��®�� � [l± ��� °,`Y± � �M` �@`3c ^ [ �^² Z ` �8^l´a` Z [ ���´"�F��gXxusub�cL���Xx��XqedOb�cewLtu�Ésef?��f��­àQw§dc¹A��d�qLfM×(dOsub �§dFg�dFc~����tecL����b_x(cL���Xxá��x(zOwete��d���qedsux(�rwLg�xusutL���#xD� �­��x�sefD�Xf�wHx~b�c����=^$A@ 6->'g&0�6G_�0�[=+/0�g&6-8!, q B7F6-f"

�6p���5*< \,c=>6�*<��=:3^6�*> / �=<���� / 1��6h \�0�13<R_,0!13<9h � tqu � �]?G

. 7w0/21 t 4�9�4 t F�� = w0/ @?�= ��B�)�� s �) q �,F9) q!q w�) q!j F9) q B7F9) q!s w�+. 7w0/213/2465740/84:9�w:4 t<; 9>= w0/ @? ���! �+

$ r 97+�:=+{0�6-+/P!0�:=+->'9 q B7w(-f"0�� \,0����3�!:S5*^S�*8��*>65*<65�'�'�0���� � 0!1S5*^��9h

>6�*<���:3^6�*> / �=<���� / 1�� � >6�*<���:3^6�*> / �=<��3��:65*^S�*8�� � 0�1S5*^��R?0�� \,0����3�!:S5*^S�*8��*>65*<65�'�' � 0���� � 0�1S5*^���h

>6�*<���:3^6�*> / �=<���� / 1�� � >6�*<���:3^6�*> / �=<��3��:65*^S�*8���1S�=1 � 0!1S5*^��R?0�� \�� 0�������:S5*^S�=8��*>65*<35Sh

>6�*<���:3^6�*> / �=<���� / 1�� � >6�*<���:3^6�*> / �=<��*>38=1��*8�?G

. 7w0/21 t 4�9�4 t F�� = w0/ @?�= q B! 7w!) q E! �) q!s F9+. 7w0/213/2465740/84:9�w:4 t<; 9>= w0/ @? ���7F9+

$DC M79&`N:=+->'97, q B t -f"�6p���5*<�>3�*<��=:3^3�*> / �=<��3��:S5*^6�*8�� � 0!1S5*^��R\�0�13<I0 � _ 0�13<;0 � hi0�13<k: � � �_ : ��� �_�>3�*< � �?0�13<k1 / � � � ��:S5*^S�=8�� � 0�165*^����=:6�30�13<S�Ve 0 � g]� 0=>]�,��0��8R\(h ?0�13<k1 / � ��� ��:S5*^S�=8�� � 0�165*^����=:6�30�13<S�Ve 0 � g]� 0=>]�,��0��8R\(h ?

� b�0�p*8 \ : � ! 1 / � � '�'�: � ! 1 / � � h�i

mD�

Page 27: Sequential Minimal Optimization for SVM Contents 1 Introduction to

0!13< 5 ��� ��:S5*^6�*8�� � 0!1S5*^�����:S�30�13<6��e 0 � gR��0=>feD: � gR?0!13< 5 � � ��:S5*^6�*8�� � 0!1S5*^�����:S�30�13<6��e 0 � gR��0=>feD: � gR?0�� \�5 � ��� 5 � h[i

>6�*<�"�"�?: � "�"�?: � "�"�?

�8�p3�*8 0�� \�5 � �k5 � h: � "�"V?

8�p3�*8: � "�"V?

�^38�< / ^�1 \2�6p���5*<9h!>3�*<R?

� G. 7w0/21 t 4�9�4 t F�� = w0/ @?�= ��E! �) q�� ) q�� F9) q �! 7FSw!) q B t ) q '7 7F9) q E7F9) j!q ) j�� ) j ��) j B7F�+. 7w0/213/2465740/84:9�w:4 t<; 9>= w0/ @? ���! �+

$DC M79&`N:=+->'97, q '7 �-."�6p���5*<�>3�*<��=:3^3�*> / �=<��3��:S5*^6�*8��=1S��1 � 0�1S5=^��R\,0�16< 0 � _ 0�1�<;0 � hi0�13<k: � � �_ : ��� �?�6p���5=<4>6�*< � R��?0�13<k1 / � � � ��:S5*^S�=8��=:S�30!13<S��e 0 � g]��0=>R����0�38V\(h ?0�13<k1 / � ��� ��:S5*^S�=8��=:S�30!13<S��e 0 � g]��0=>R����0�38V\(h ?

� b�0�p*8 \ : � ! 1 / � � '�'�: � ! 1 / � � h�i0!13< 5 ��� ��:S5*^6�*8��=:S��0�13<S��e 0 � g]��0�>feD: � g�?0!13< 5 � � ��:S5*^6�*8��=:S��0�13<S��e 0 � g]��0�>feD: � g�?0�� \�5 � ��� 5 � h[i

>6�*<�" � ��:65*^S�*8���:S�30�13<6��e 0 � gR� d65�pRe&: � gIc4�!:S5*^S�*8��=:S�30�1�<S��e 0 � g]� d65�p�eD: � gR?: � "�"�?: � "�"�?

�8�p3�*8 0�� \�5 � �k5 � h: � "�"V?

8�p3�*8: � "�"V?

�^38�< / ^�1 \2�6p���5*<9h!>3�*<R?

� G. 7w0/21 t 4�9�4 t F�� = w0/ @?�= ��E! �) q�� ) q�� F9) q �! 7FSw!) q B t ) q '7 7F9) q E7F9) j!q ) j�� ) j ��) j B7F�+. 7w0/213/2465740/84:9�w:4 t<; 9>= w0/ @? ���! �+

$DC M79&`N:=+->'97, q '�F6-f"�6p���5*<�>3�*<��=:3^3�*> / �=<��*>38*1��=8�\,0�13< 0 � _ 0!13< 0 � hi�6p���5=<4>6�*< � R��?�6�*^ \�0�13<;0 � �? 0�!*>�? 0�"�"�h>6�=<�" � >38=1��*8��=:6�30�13<S�Ve 0 � gfe 0=g c[>38*1��*8��=:S�30�1�<S��e 0 � gfe 0=gR?

m~ 

Page 28: Sequential Minimal Optimization for SVM Contents 1 Introduction to

^38�< / ^�1�>6�*<�?�G

. 7w0/21 t 4�9�4 t F�� = w0/ @?�= ��E! �) q�� ) q�� F9) q �! 7FSw!) q B t ) q '7 7F9) q E7F9) j!q ) j�� ) j ��) j B7F�+. 7w0/213/2465740/84:9�w:4 t<; 9>= w0/ @? ���! �+

��®�� �¶²k´e±.²7\ ��[l± ��� °,`Y± �orqed*v_b_cedFf(g"�(dFg�cLd=vËb��"��b_z�wLv�à���qLd�sux(��wegXxusutL����^���tegXg�dFc~�Xv�à7¹H��d*qLfM×~d%x(cev_à�x(ced�cex(cu�v_b�ced�fDg��~d=gXced=vñ¶ìgXf~sub_f(v;yLf(��b_����tLcL���Xb�x~c��~d=gXced=vñ^$ r 97+�:=+{0�6-+/P!0�:=+->'9 q E! �-f"

0�� \,0�����p�0�16835*^����S8�^�1383pSh�S8�^�1383p���� / 1�� � >6�*<��=:3^6�*> / �=<���� / 1��`?

0�� \�� 0�����p30�16835=^����S8�^*1683pSh�S8�^�1383p���� / 1�� � ^ � �����S8�^�1383pV?

G. 7w0/21 t 4�9�4 t F�� = w0/ @?�= q B! 7w!) q E! �) q!s F9+. 7w0/213/2465740/84:9�w:4 t<; 9>= w0/ @? ���7F9+

orqed��FfDv���tev�f?�Xb�x~c³x(�$C$½s¾.Ò¡½kÁjC Á b�cÂfáßlfDtH����b_f(c³�~d=gXced=vØ�FfDcÀy§d���w§dFsÀtewÀtL��b_ce����qed��x(v_v_x?��b�ce��d�Õ�tLfD��b_x(cA¶

C$½s¾�Ò ½kÁ C Á Ì ª�½s¾�ÒR½kÁq¬ E ª�½s¾�Ò ½kÁq¬Ì ½ E ¾ ½s¾hÖ ½ EÁ ½kÁ�ÒÂm;½ E ¾ ½kÁO¿

��qedFg�d�½ E¯ ½ ¯ �FfDcáy§dÉwegXd�����x~z�wLtu��d�sY^�Ûex(g"dFf(��q�x(�¸��qed � sub_zOd=cL��b�x~cL��¹Hsub_g�d����Xv�à���x~z��wetu�Xb�cL� CK½ ¾ Ò ½ Á C Á cLd=dFsL�#|�x(w§d=g�f?��b_x(cH�=¶]~^��]Ì ��¾ � ÒY� Á �me^���Ì������|L^�Í�Ì Í�ÖJ�?^��cá��x~z�wHfDgXb_��x(ck¹~��qLd%ced=� zOd���qLxQs�cedFdFse�#x(cev_à�m�x(w§d=g�f?�Xb�x~cL�=¶]~^��]Ì ��¾ � �o� Á �me^�Í�Ì Í�Ö��§^

$DC M79&`N:=+->'97, q E7F6-f"�6p���5*<�^ � �����S8*^�1683pV\�0�13< 0 � _ 0!13<;0 � hi

�6p���5*< � � >6�*<���:3^6�*> / �=<���� / 1��`\,0 � _,0 � h ?�4c � � � ?�(" � :3^383���7�S: / <38�>��3�=83p����*>3�*<��=:3^3�*> / �=<.e 0 � g " :�^386���7�6: / <38�>��3�*83p����*>6�*<���:3^6�*> / �=<.e 0 � g]?^38*< / ^�1;8�m*:]\6�3���=< � ���3��0=2���5������ / 5*^�8�>9h ?

�G

mDÏ

Page 29: Sequential Minimal Optimization for SVM Contents 1 Introduction to

. 7w0/21 t 4�9�4 t F�� = w0/ @?�= ��E! �) q�� ) q�� F9) q �! 7FSw!) q B t ) q '7 7F9) q E7F9) j!q ) j�� ) j ��) j B7F�+. 7w0/213/2465740/84:9�w:4 t<; 9>= w0/ @? ���! �+

$&% 8!0'j78d[B�i6-8!,B:=>«+-9&`!6-M&j78 q!s �-f" 0�1���p / >38$!3���95*<�b#�G

. 7w0/21 t 4�9�4 t F�� = w0/ @?�= �('*) q!s �) j �,F9) j!j +. 7w0/213/2465740/84:9�w:4 t<; 9>= w0/ @? ���! �+

$ r 97+�:=+{0�6-+/P!0�:=+->'9 q!s F6-f"0�� \�� 0�����p30�16835=^����S8�^*1683pSh4i

:3^386�*�7�S: / <�8�>��3�*8�p����*>6�=<��=:3^6�=> / �=<]� ^�86��0�38�\ t h ?�6�*^ \�0�13<;0 � �? 0�! t ? 0�"�"�h

:�^386���7�6: / <38�>��3�*83p����*>6�*<���:3^6�*> / �=<fe�0=g � >6�*<��=:�^6�*> / ��<���� / 1S�`\,0`_,03h ?�

G. 7w0/21 t 4�9�4 t F�� = w0/ @?�= q B! 7w!) q E! �) q!s F9+. 7w0/213/2465740/84:9�w:4 t<; 9>= w0/ @? ���7F9+

$A@ 6->'g&0�6G_�0�[=+/0�g&6-8!, q!s w(-."d383�=<6�*^�!��6p���5*<��4:�^386���7�6: / <38�>��3�*83p����*>6�*<���:3^6�*> / �*<R?

G. 7w0/21 t 4�9�4 t F�� = w0/ @?�= ��B�)�� s �) q �,F9) q!q w�) q!j F9) q B7F9) q!s w�+. 7w0/213/2465740/84:9�w:4 t<; 9>= w0/ @? ���! �+

��®�� �7± ^H[ �¡³B± Z `�[ �M^H[ �½������V6 ��P�:OK4Se<MSL¨ÂP�:FP�<?ê�ï � C(>§¨Â¨ÂSL9 @ ¥VEG9 PÛ4b�cHfDv_v�à7¹#��d qLfM×(d��XxJ�(d=���Xqed wLf(gXf(zOd���dFgX�� �æÈDÍ�U ��Ð��uÎ�È eÅ(Ç��(Èe¹�d����D^ ªG´#cLs���qedb_cewetu���Mx(te��wetu��ÙHv�d%cLf(zOdF�F^ ¬�ÊÞd%tL��d��Xqed�eÈ(Ð��(ÍuÐ Y=Z�gXx(tu�Xb�cedl�XxOqLfDcLsev�dl�Xqeb_�F^$A@ 8N:h+-9]5&0�[<0�4�8d:=8N[=, q!s7t -."

i8�m�<38*^�1 ��bS5=^;c��=:3<35*^�2R?8�m�<38*^�1 0�13<��=:3<S0�1�>R?0�13<;� ?0�13<�8*^�^��6p*2 � �?

� b�0�p*8 \�\,� � 238�<6��:3< \�5*^�26�`_ 5*^�2�d�_ (1 � > ��� � <�� 8 �A: �8� � � � � �A^�� p3� � 59 *h�h � � w ��� h� � 0=<S��b \,�3hi�*53�*8�� 1����t � 5=<6�30`\���:3<65*^�2�h ?� ^3835 �f?

�*53�*8�� >����> � 5=<6�30`\���:3<65*^�2�h ?� ^3835 �f?

�*53�*8��,�����l � 5=<6��� \���:3<65*^�2�h ?

mD²

Page 30: Sequential Minimal Optimization for SVM Contents 1 Introduction to

� ^3835 �f?�*53�*8�� <����<6��p�8*^65=1��*8 � 5*<6���R\ �=:3<65*^�29h ?� ^3835 �f?

�*53�*8�� 8 ���8*:�� � 5*<6��� \ �=:3<65*^�29h ?� ^3835 �f?

�*53�*8�� :����< � ������0=2���5��3��� / 5=^38�> � 5*<6��� \��=:3<65=^�29h ?� ^3835 �f?

�*53�*8��2�����>65*<65����S0�p�8��=1S57�98 � ��:3<65*^�2�?� ^3835 �f?

�*53�*8��&� ����=d�� ���S0�p�8���1S57�98 � �=:�<65*^�2R?� ^3835 �f?

�*53�*8���� ���� / <�: / <����S0*p�8��=1S5��98 � �=:3<65=^�2R?� ^3835 �f?

�*53�*8�� ^�����=^65=1�>���� \�5*<3�30 \��=:�<65*^�29h�h ?� ^3835 �f?

�*53�*8���p ���0�����p�0�16835*^����S8�^�1383p � <�^ / 8V?� ^3835 �f?

�*53�*8��,�����0����3�!:S5*^S�*8��*>65*<65 � <�^ / 8�?� ^3835 �f?

�*53�*8�� � ���0���� � 0�1S5*^�� � <�^ / 8�?� ^3835 �f?

�*53�*8���5 ���0����*<�86�=<�����1Sp�� � <�^ / 8�?� ^3835 �f?

�*53�*8�� � ���8�^�^��3p*2�"�"�?

0�� \ 8�^�^��6p=2 ��� �=:3<60�13>*!k5=^�2S�6hi�=8�^�^$!�!; / ��5=238 � !�!�5*^�2�d]e8�g*!�! � !�!

��� >65*<65����S0�p�8��=1S57�98��=1 �7� �=d�� ���S0�p�8���1S57�98���1 ��� � / <�: / <����S0*p�8��=1S5��98��=1 �=1 t �=1 �*> >��=1 �3� l��=1 �*< <6��p�8*^65=1��*8��=1 ��8 8*:���0*p��=1��=1n �=: < � ������0=2���5��3��� / 5=^38�>��=1n

|(Õ

Page 31: Sequential Minimal Optimization for SVM Contents 1 Introduction to

�*^ ^65=13>3�7� �3�*8�8�>��=1 ��p \,0�����p30�16835=^����S8�^*1683pSh ��1 �3� \,0�������:S5*^S�=8��*>65*<35Sh �=1 � � \,0���� � 0�1S5*^��9h �=1 ��5 \,0����=<386�=<����=1Sp��9h �=1 ?

8*mS0=< \ � h ?�

� G. 7w0/213/2465740/84:9�w:4 t<; 9>= w0/ @? ���7F9+

$ bs0�[=+/0�g76/8N,Y6->;`K0�6 :=>«4�0�+-9�j �� �-f"��bS5*^;c�>65*<65����S0�p�8���1S57�98 � ��=d!�M� >65*<359 �?��bS5*^;c��=d�� ���60�p�8��=1657�98 � ��=d!�M� ���*>�83p9 �?��bS5*^;c*� / <�: / <����S0�p*8��=1S57��8 � !�=d��M� � / <�: / <� �?G

. 7w0/213/2465740/84:9�w:4 t<; 9>= w0/ @? ���7F9+

$&% 8!0'j78d[B�i6-8!,B:=>«+-9&`!6-M&j78 j �,F6-f" 0�1���p / >38$!30����=<�^3835�� � 0�1���p / >38$!3����<�>6p30 � � 0�1���p / >38$! / 1S0��=<�>]�Db#�G

. 7w0/21 t 4�9�4 t F�� = w0/ @?�= �('*) q!s �) j �,F9) j!j +. 7w0/213/2465740/84:9�w:4 t<; 9>= w0/ @? ���! �+

½�������\ � P�SL@åEV9�@ Se:FSorqedOsef?��f�ÙHv�d�b_�lf �Hf?����d�ÓQ��ÙHv�d7¹YdFf(��qÞsef?��f�w§x(b_c��lxu�F��teweb_dF��x~ced�v_b_ced�b_cÞ��qeb_��q ��qed��v�f(�X��v_f(yHdFvYª Ö�]lx~g�Ò%]'¬��x(v_v�x?�#���Xqed*f?����gXb�yete��d%×?fDv_tedF�F^�ª�g�sub�cHfDgXb�v_àc¹uf�v�b_ced%��b_v�vËyHdÅ~ÐQÐ ;Æ�� �uÐuÈ��VQ ���uÅuÎ �eÈÂÅ~Ð�Ð ;Æ�� �QÐeÈ�� �����eÅuÎ �LÈ�����Å(ÐQÐ ;Æ����uÐeÈ�������eÅuÎ �eÈÁÐuÅ� eÈ~Ð ���eÅuÎ �eÈÛex~g#��wHfDg���d���x(gXz�f?��¹ef�v_b�cLd%��b�v_vAy§dÆ � �`Q��uÅuÎ �VQ Æ ��� ���eÅuÎ � � ���³Æ � ��Ä��eÅuÎ �FÄ ÐeÅ� eÈ(Ð ���eÅuÎ �LÈ��qedFg�d�Æ � �����qex~tev�s�yHd�y§d��­��d=d=c ]�fDcHs � ^ìÛLx(gr��wHfDg���d#yLb�cLf(g�à���x~g�z�f?�'¹�fÉv�b_ced���b�v_v;yHdÆ � �`QåÆ � � � ��� Æ � �FÄ ÐuÅ� eÈ~Ð ���eÅuÎ �eÈqedFg�d7¹Q��xQxe¹YÆ � ��O��qex~tev�s�y§d%y§d��­��d=d=cÞ]%fDcLs � ^orqedsefD�Xfrb��YgXdFf(s�b_c~�Xx �uÈDÇ�U(È��DÍ��HÆ�ÇuÐVUi¹�x~gMU�ÍLÅ� VU(È �DÍ �HÆ�ÇQÐVU^¹Xx(gKUMÍLÅ� `U(È����YÆFÇLÅ� � �?Í �LÆ�Ç�ÐVU^¹

f(�F��x~gXsub_ce����x���qLd%b�cewLtu����x(gXz�f?�F^$&u 8K0'ja+/9]j&0�:<0vj �,w(- "

|e]

Page 32: Sequential Minimal Optimization for SVM Contents 1 Introduction to

i0�13<k1R?0�� \�0����*<383�=<����=16p��9h4i

0��S�=<�^38�57� �=d�� ���S0�p�8�\��=d�� ���60�p�8��=1657�98�h ?8=13>��3� / :�:S�*^�<��30 � �S0=^6�=<��*<383�=<��30 � 1 � ^3835*>��3�=d�� \��=d�� ���60�p�8�h ?t " � 1]?

�0�� \ t ���h i<65=^�238�<]�A^386�*8�^�d38�\ t hn?0�� \,0����3�!:S5*^S�*8��*>65*<65�'�'�0���� � 0!1S5*^��9h

��:S5*^6�*8�� � 0!1S5*^�����:S�30�13<6�V� ^386�=8�^�d38�\ t h ?83p��*8 0�� \�0����3��:65*^S�*8��=>65*<65�'�' � 0���� � 0!1S5*^��9h

��:S5*^6�*8��=:S��0�13<S�V�A^386�*8�^�d38�\ t hn?83p��*8

>38*1��=8��=:S�30!13<S�V� ^�86�*8�^�d�8�\ t h ?�0��S�=<�^38357� >65=<65����S0*p�8�\ >65=<65����S0*p�8��=1S5��98�h ?1 � ^�835*>��*>35*<65V\ >35*<65����60�p�8�h ?0�� \�0����*<383�=<����=16p��9h4it � �S0=^6�=<��*<383�=<��30�" 1R?�83p3�*84it � 1]?

�60=^S�=<��=<386�=<���0 � �?8=13>��3� / :�:S�*^�<��30 � t ?

�� G

. 7w0/213/2465740/84:9�w:4 t<; 9>= w0/ @? ���7F9+

orqed�f(���XtLfDv_v_à�gXdFf~sub�cL��xD�sefD�Xf�b_��qLfDcHsuv�d�sáyQà uÈQÅ�� ��QÅ~ÐeÅMYFÆ�U?Ð eÈQÅFÄ��VZ �Qb���f(wewHdFcLse�sefD�XfÀw§x(b_c~������gXx(z ��qedæb_cewete������gXdFfDz ��x �uÈ(Ç�U(È �DÍ �HÆ�ÇQÐVUª�x(g UMÍeÅ� VU(È��?Í �HÆ�ÇuÐ`U^¹ìx(gUMÍLÅ VU(È����;Æ�ÇLÅ� � �DÍ �LÆFÇQÐVU&¹Dsed=w§d=cLsub_ce��x~c�b_cewetu�"��x(gXzOfD�!¬�^orqed*��tLcL���Xb�x~c eÈQÅ� ��uÅ~ÐuÅMYFÆ9U?Ð eÈQÅ�Ä���Z�zOfMà�y§d��FfDv_v�d�sáyQà eÈQÅ�� �nU �DÄ#Y*Zl��x�gXdFf~s

b_cá��qedO��tewew§x(g���×(d����Xx(g���xD�ØwegXd=×Qb_x(tL�"��g�fDb_ced�s�z�xusudFv�ª�b��cex(ce�ñv_b�cLdFfDg"�(d=gXcedFv b��"tH��d�se¬�^orqed���dØ��tewew§x(g��A×(d�����x~gX�Yf(g�dìgXdFf~s�b_c���x �uÈ(Ç�UDÈ��DÍ �HÆFÇuÐVU�ªVx(gKUMÍLÅ� VUDÈ��DÍ �HÆFÇuÐVU^¹�x~gKUMÍLÅ VU(È����;Æ�ÇLÅ� � �DÍ��HÆ=ÇuÐVU&¹sudFwHdFcLsub_ce�%x(c�b_cewetu���x(gXzOfD�!¬4y§d���x~g�dr�Xqed"sefD�Xf�wHx~b�c����Øb_c��Xqed"sefD�Xf�ÙLv_d(^¸orqed#���Xf(g��Xb�cL�b_cLsud$Ó�xD�4��qed*sefD�Xf�w§x(b_c~����b_c��Xqed*sef?��f�ÙLv_d�b�� §Æ VUMÐ �(ÐeÈ UMÐ �LÆQ^$DC M79&`N:=+->'97, j!q -."

0�13<4^38�5*>��*>65=<65V\,0���<�^38357�#' 0��6hi�=<�^S0!132;�`?0�13<k1���p30�1683�`?

�6�*^ \A1#��p30�1386� � �? 238�<6p30!168�\,0�� _ � _ � �=1��(h ?�1#��p�0�1686��"�"�h�i0���<�^S�=<�^�8357� p30�168V\,�V�������=<�^R\(h�h ?d383�=<6�*^�!��6p���5*<���d�?�6p���5*<�<R?� bS0�p�8 \�p30!168�����<�h

|~m

Page 33: Sequential Minimal Optimization for SVM Contents 1 Introduction to

d]�A: / �!b#� � 53�@�f\ <9h ?<65=^�238�<]�D: / ��b#� � 53� �f\ d]� � 53�@�f\(h�h ?d]�D:S�=:#� � 53� �f\(hn?0�1�<k1 � dR����0�38V\(h ?0�� \,0����3�!:S5*^S�*8��*>65*<65�'�'�0���� � 0!1S5*^��9hki

��:S5*^6�*8�� � 0!1S5*^����=d386�=<6�=^ mR?�6�*^ \�0�13<;0 � �? 0�!=1R? 0�"�"�h�i

0�� \ dfe 0�g$! � ��� dfe 0=g�� >9h i�*8�^�^$!�! 78*^�^6�*^�� p�0�168 �!�! 1#��p�0�1686��" �

!�! � 5*<�<�^S0 � / <38;0�1�>38�m !�!�0�13<R\ dfe 0=g9h(!�! � / < ���4^65�13238R�� @!�!�8*13>3pV?8�mS0�<R\ � h ?

�mR��0=>]�A: / ��b#� � 53� �f\,0!13<R\ dfe 0=g9h�� � h ?

���:S5*^6�*8�� � 0!1S5*^�����:S�30�13<6�V�A: / �!b#� � 53���]\ m9h ?

�83p��*8 0�� \�0����3��:65*^S�*8��=>65*<65�'�' � 0���� � 0!1S5*^��9hki

��:S5*^6�*8��*d383�=<6�*^ mR?�6�*^ \�0�13<;0 � �? 0�!=1R? 0�" ��� h�i

0�� \ dfe 0�g$! � ��� dfe 0=g�� >9h i�*8�^�^$!�! �>35*<65(�S0*p�848�^�^3�*^�� p30�138 !�![1#��p30�1386��" �

!�! � 5*<�<�^60 � / <38 0�1�>38�m !�!40�13<�\ dfe 0=g�h(!�! � / < ���k^35=13238R�( !�!k8*1�>6pV?

8�mS0�<R\ � h ?�mR��0=>]�A: / ��b#� � 53� �f\,0!13<R\ dfe 0=g9h�� � h ?mR� d65�p��D: / ��b#� � 53� �f\ dfe 0�" � g9h ?

���:S5*^6�*8��=:S��0�13<S�V�D: / ��b#� � 53� �f\ m9h ?

�83p��*8�i

0�� \ dR����0�38V\(h � � >9h i�*8�^�^$!�! �>35*<65(�S0*p�848�^�^3�*^�� p30�138 !�![1#��p30�1386��" �

!�! bS53� !�! d]����0�38�\(h(!�! 5*<�<�^S0 � / <386�`?���bS� / p*> � 8 > � !�! >!�!�8*1�>6pV?

8�mS0�<R\ � h ?�>38*1��=8��=:S�30!13<S�V�A: / ��b#� � 53� �f\ d�h ?

��^38�< / ^�141#��p�0�1686�`?

� G. 7w0/21 t 4�9�4 t F�� = w0/ @?�= ��E! �) q�� ) q�� F9) q �! 7FSw!) q B t ) q '7 7F9) q E7F9) j!q ) j�� ) j ��) j B7F�+. 7w0/213/2465740/84:9�w:4 t<; 9>= w0/ @? ���! �+

$&% 8!0'j78d[B�i6-8!,B:=>«+-9&`!6-M&j78 j!j -]" 0�1���p / >38$!3�=<�^S0�132�� 0�1���p / >38$!*d383�=<6�*^�� 0�1���p / >38$!30����=<�^3835�� � 0�1���p / >38$!��S��<�^38357� �

|(|

Page 34: Sequential Minimal Optimization for SVM Contents 1 Introduction to

0�1���p / >38$!3�=<�^S�=<�^38�57� �G

. 7w0/21 t 4�9�4 t F�� = w0/ @?�= �('*) q!s �) j �,F9) j!j +. 7w0/213/2465740/84:9�w:4 t<; 9>= w0/ @? ���! �+

½������_½ IËS(éAEG9 ¦JSe9 @å¥G>HSL@ EG9 ¦À¨Â>A@4PQ¥¸KìSu<?Se¨ÂP�:FP�<?êorqedlx(te��wetu��x(g�sud=grxD� ��qed%zOxusudFvAwLfDg�fDzÉ��dFgX����b_v�vAy§d]~^rorqLd%c�tLzÉy§d=g�xD�¸f?����gXb�yLtu��d�� � ^me^rorqLd �HfD��Æ9U�nUMÍLÅ� VUDÈ���uÅ~ÐuÅ|L^rorqLd �HfD��Æ9U� �YÆ�ÇLÅ� ��H^rorqLd �HfD��Æ9U��ÎLÆ�ÇLÈQÅ �(ÑeÈ� ~ÇLÈuÎ�e^rorqLdl��qegXdF��qex(v�s���L^����4��qLd%v�b_cedFf(g��(dFg�cedFvYb_�rtH��d�sY¶

ªGf7¬�orqed%��d=b_�(q��r×(dF����x~g è u^����ìcex~cu�ñv_b_cedFf(g��(d=gXcedFv§b���tL��dFs

ªGf7¬��ld=gXcedFvYwLfDg�fDzÉ��d=g���ª�d(^ �L^ ¹Hm�� Á ��x(g�g�f(sub�fDv;yLf~��b�����tecL����b_x(c��(dFg�cedFvZ¬ª�y ¬�orqed%cQtez�yHdFg�xD�4��tewew§x(g��#×(d����Xx(g��ªG��¬�orqed*` fD�(g�fDcL�(d"z�tev���b_wev�b_d=g��rxD�4��qLd*��tewLwHx~g���×(d����Xx(g��ªVse¬�orqed*��tewew§x(g���×(dF����x~gX��¹Qx(cedlw§d=g#v_b�cLd

$DC M79&`N:=+->'97, j�� -."d6�30=> � ^S0=<38����=d�� \����=<�^3835�� ' �3�3h i

�3�(!�!�>�!�!k8*13>6p`?�3�(!�!�0�������:S5*^S�=8��*>65*<35*!�!k8*13>6p`?�3�(!�!�0���� � 0�1S5*^��*!�!k8*13>3pV?�3�(!�!�0�����p30�16835=^����S8�^*1683p$!�!k8*1�>6pV?�3�(!�! � !�!k8*13>6p`?0�� \,0�����p�0�16835*^����S8�^�1383pSh4i

�6�*^ \�0�13<;0 � �? 0�!*>�? 0�"�"�h����!�! � e 0=g*!�! 8*13>6pV?

�83p��*8�i

�3��!�![< � ������0=2���5��3��� / 5=^38�> !�!�8*13>3pV?0�13<k1��3� / :�:6�*^�<��*d�86�=<6�*^6� � �?�6�*^ \�0�13<;0 � �? 0�!�8=13>��3� / :�:S�*^�<��30`? 0�"�"�h

0�� \�5�p=:�b.e 0=g*� �h1#�3� / :�:S�*^�<��*d386��<6�*^S��"�"�?

�3��!�! 1#�3� / :�:S�*^�<��*d386�=<3�*^S��!�!�8*13>3pV?�6�*^ \�0�13<;0 � �? 0�!�8=13>��3� / :�:S�*^�<��30`? 0�"�"�h

0�� \�5�p=:�b.e 0=g*� �h

|D�

Page 35: Sequential Minimal Optimization for SVM Contents 1 Introduction to

�3��!�!k5�p=:�b.e 0=g*!�!�8*13>6pV?�6�*^ \�0�13<;0 � �? 0�!�8=13>��3� / :�:S�*^�<��30`? 0�"�"�h

0�� \�5�p=:�b.e 0=g*� �h i0�� \�0����3��:65*^S�*8��=>65*<65�'�';0���� � 0�165*^��9h�i

�6�*^ \,0!13< ��� V? � !3��:65*^S�*8�� � 0�1S5*^����=:S�30!13<S��e 0�g]��0=>]�,��0�38R\(h ? � "�"�h�3� !�! \,�!:S5*^S�*8�� � 0�1S5=^����=:S��0�13<S��e 0=g]��0=>]e � g�" � h�!�! � ��?

�83p3�=8 0�� \,0����3��:S5=^S�*8��*>35*<65�'�' � 0���� � 0�165*^��9hki

�6�*^ \,0!13< ��� V? � !3��:65*^S�*8���:S�30�13<6��e 0=g]� 0=>]����0�38�\(h ? � "�"�h�3� !�! \,�!:S5*^S�*8��=:S�30�1�<S��e 0=gR��0=>fe � g�" � h*!�! � �!�!���:65*^S�*8���:S�30�13<6��e 0=g]�Ad65�pRe � g !�! � ��?

�83p3�=8�i

�6�*^ \,0!13< ��� V? � !*>R? � "�"�h�3� !�!�>38=1��*8��=:6�30�13<S�Ve 0=gfe � g !�! � ��?

��3��!�! <65*^�238�<fe 0�gR?�3��!�!�8*13>3pV?

��

� G. 7w0/21 t 4�9�4 t F�� = w0/ @?�= ��E! �) q�� ) q�� F9) q �! 7FSw!) q B t ) q '7 7F9) q E7F9) j!q ) j�� ) j ��) j B7F�+. 7w0/213/2465740/84:9�w:4 t<; 9>= w0/ @? ���! �+

$DC M79&`N:=+->'97, j ��-."0�13<4^38�5*>��3�=d!� \,0��=<�^38357� ' 0��6h[i

0��(����>R?0��(����0�������:S5*^S�=8��*>65*<35V?0��(����0���� � 0�1S5*^��R?0��(����0�����p30�16835=^����S8�^*1683pV?0��(��� � ?0�� \,0�����p�0�16835*^����S8�^�1383pSh4i

� � ^383��0�38�\ >9h ?�6�*^ \�0�13<;0 � �? 0�!*>�? 0�"�"�h

0������ � e 0=gR?�83p��*8�i

0������[< � ������0=2���5��3��� / 5=^38�>R?0�13<k1��3� / :�:6�*^�<��*d�86�=<6�*^6�`?0������ 1#�3� / :�:S�*^�<��*d386�=<3�*^S�`?5�p=:�b]� ^386��0�38�\ 1#��� / :�:S�=^�<��*d383�=<6�*^S�V_�R��h ?�6�*^ \�0�13<;0 � �? 0�!=1��3� / :�:6�*^�<��*d�86�=<6�*^6�`? 0�"�"�h

0������k5�p=:�b.e 0=gR?�=<�^S0!132�> / ���#����p30�138��*<6����� �90�:#��168 � p30!168�?238�<6p�0�168�\,0��`_ > / ��� ����p30�168��*<6���3�@�90�:#�=138 � p30�138�_ � �=1��(h ?^38�< / ^�1 ^3835*>��*>65*<65`\,0��6h ?

�^38*< / ^�1*�?

� G

|~�

Page 36: Sequential Minimal Optimization for SVM Contents 1 Introduction to

. 7w0/21 t 4�9�4 t F�� = w0/ @?�= ��E! �) q�� ) q�� F9) q �! 7FSw!) q B t ) q '7 7F9) q E7F9) j!q ) j�� ) j ��) j B7F�+. 7w0/213/2465740/84:9�w:4 t<; 9>= w0/ @? ���! �+

$&- [=+�:=8�4�>Ojc8!6 5i0$[<0�4�8N:=8N[=, j B! �-f"i0�� \�� 0����*<386��<����=1Sp��*'�';�=d�� ���S0�p�8���1S57�98�� � t�u � ��h i

���6�=<�^3835�� �=d�� ���60�p�8�\,��d�� ���S0*p�8��=1S5��98�h ?� ^60=<38��3��d�� \,�=d!� ���S0�p*8�h ?

�� G

. 7w0/213/2465740/84:9�w:4 t<; 9>= w0/ @? ���7F9+

��®�� �&`�c ^ [ �^²¶²H´ ´a`h´È´ ³��^²$DC M79&`N:=+->'97, j B7F6-f"

�6p���5*<8�^�^6�*^��*^65*<38V\(hi

0�1�<k1#�*<6�=<65�p � V?0�1�<k1#��8�^�^6�*^ � V?�6�=^ \,0�13< 0 � �S0=^6�=<��*<383�=<��30`? 0�! t ? 0�"�"�h[i

0�� \�p�8�5*^�168�>���� / 1�� \,06h��� � � <65*^�2�8�<fe 0=g*�(�h1#�*8�^�^6�*^�"�"�?

1#�*<6�*<35�p�"�"�?�^38*< / ^�1$�6p���5*<R\ 1���8�^�^6�=^9h=���6p���5*<R\ 1��*<6�*<65�pSh ?

�G

. 7w0/21 t 4�9�4 t F�� = w0/ @?�= ��E! �) q�� ) q�� F9) q �! 7FSw!) q B t ) q '7 7F9) q E7F9) j!q ) j�� ) j ��) j B7F�+. 7w0/213/2465740/84:9�w:4 t<; 9>= w0/ @? ���! �+

orqed%��v�f(�X��b�ÙH�Ff?�Xb�x~c�x~tu��wete�#b_� è ½ ¯ ÒÁ�#��x(g�dFf~��q�sLf?�Xf�w§x(b_c��.½ ¯ ¹ex~cedlwHdFg�v�b_ced~^$&- [=+�:=8�`N6{0�,I,I+��i`K0$:=+/>�9�>'M7:=57M7: j B7w(-f"

i���6�=<�^3835�� � / <�: / <����S0�p*8�\�� / <*: / <����60�p�8��=1657�989h ?�6�=^ \,0�13< 0 � �S0=^6�=<��*<383�=<��30`? 0�! t ? 0�"�"�h

� / <�: / <����S0�p*8�!�!kp�835=^�168�>���� / 1��`\�06h�!�!k8*13>6p`?� G

. 7w0/213/2465740/84:9�w:4 t<; 9>= w0/ @? ���7F9+

$), +/0�\�9&>�,n:=+/`.+-97@ > j B t -f"�3c �#���;c��i 0����6p���5=<;� � ���?�6�*^ \�0�13<;0 � �? 0�! t ? 0�"�"�h

� " � 5�p=:�b.e 0=gR?�6p���5=<4< � ���?

|(�

Page 37: Sequential Minimal Optimization for SVM Contents 1 Introduction to

�6�*^ \�0�13<;0 � �? 0�! t ? 0�"�"�h�3�*^ \,0�1�< ��� �? � ! t ? � "�"�h

<�" � 5�p=:�bfe 0=gSc�5�p=:�b.e � gSc=<65*^�238�<fe 0�gSc=<S5*^�2�8�<fe � gSc��68�^�1S83p���� / 1��`\,0V_ � h ?�*8�^�^�!�! � �� 86�=<S0�d38�� / 1���<S0��=1 � (!�! \,���[<6� � �,h(!�!�8*1�>6pV?�6�*^ \�0�13<;0 � �? 0�! t ? 0�"�"�h

0�� \�5�p�:�b.e 0=g*!(�h�*8�^�^$!�! !5�p=:�b.e( �!�!�0(!�! �g � !�!45�p=:�bfe 0=g$!�! !(� !�!�8*1�>6pV?

� � ���?�6�*^ \�0�13<;0 � �? 0�! t ? 0�"�"�h�(" � 5�p=:�b.e 0=g c <35*^�238�<]e 0=gR?

�*8�^�^�!�! �� � !�!4�(!�!k8*13>3pV?�*8�^�^�!�! 78�^�^6�*^��*^35*<38 � !�! 8�^�^6�=^��*^65*<�8�\(h�!�! � �=<���?

8*13>S0��0�13<k16�=1#� � � / 13>��3� / :�:S�*^�< � V?0�13< � � / 13>���� / :�:S�=^�< � �?�6�*^ \�0�13<;0 � �? 0�! t ? 0�"�"�h0�� \�5�p=:�bfe 0=g*�(Sh i

0�� \�5�p�:�b.e 0=g*! l�h1S�=1�� � � / 1�>��3� / :�:S�*^�<�"�"�?

83p3�*8� � / 13>��3� / :�:S�*^�<�"�"�?

��*8�^�^�!�! (1S��1#� � � / 13> � !�! 1S�=1#� � � / 13>���� / :�:S�=^�< !�! � �*<���?�*8�^�^�!�! � � / 13>��3� / :�:S�*^�< � !�! � � / 13>��3� / :�:S�*^�<�!�!k8*13>6p`?

�G

. 7w0/213/2465740/84:9�w:4 t<; 9>= w0/ @? ���7F9+

$ r ,*:=)78�>'g�n8N`N:=+-_q8�@ZM&97`N:=+->'9]+-97`N[=8K0�,I+-97\�� j '!-f"i

�6p���5*< � ��� 8�<65�� � ?�6p���5*< � � � � � cI\ w � � w � h��48�<654c�5�p=:�b � ?�6p���5*<�< ��� � � c�5�p=:�b � ck5�p=:�b � "�� � ck5�p=:�b � ?�6p���5*<�< � � � � c�5 � c�5 � "�� � c�5 � ?0�� \ < � �*< � !��h

�*8�^�^�!�! ��!bS5=13238 � �!�!�< � � < � !�!k8*13>6p`?� G

. 7w0/2139�4���4:/ /2465 4:/8409�w04 t +

��®�� ¸ [H\ � °��)\<³ � �orqed�huj³ª���xusud�qLfDcLsev�d�� x~cev_à%yeb�cHfDgXà*��v�f(�X��b�ÙH�=fD��b_x(cA^ oËxlqLf(cLsuv_d�XqedrzÉtev���b���v�f(�X�ì�=f(��dc¹��dÉtL��d*�XqedÉ��x~v�v_x?��b_ce���X��gXb�we�F^%orqedÉb_cewete�%sLf?�Xf���x(gXz�f?��b�����b_zOb�v�fDg���x���qLfD��x(��huj³ª�¹d$Óe�=d=wu����qed%�=v_f~����v�fDy§d=v�����b���q�y§dlÕe¹;]c¹�Â�Â�Â/¹^Ï�ÒJ]c¹Qb�� ��qedFg�d%f(g�dlϳ��v�f(�X��dF�F^4orqedl�X��gXb�wu�U=Ä�� ��Ä �HÎ(Ð§Æ ���~Î�Å U�U�yeteb_v_se�3Ï�yeb_cLfDgXàÀ��v�f(�X��b�ÙLd=g���¹ Ë�� ªT½H¬3Ì Í�ÎiÏhª è � ½Òå� � ¬K¹¸x(cLd���x(gdFf~��q�x(�Ë�Xqed�l��v�f(�X��d��=^ìorqed*��v�f(�X��b�ÙH�Ff?�Xb�x~c�gXtev�dlx(� �Xqed%zÉtev���b���v�f(�X����v�f(�X��b�ÙLdFgrb_�

�.Ì�fDgX�¸z�f;Ó� è � ½�ÒÀ� � Â

|� 

Page 38: Sequential Minimal Optimization for SVM Contents 1 Introduction to

orqed%zOxusud=v��#fDgXd%��fM×~dFs�b�c���� U �?Ä �§Æ(Î~È��DÇLÅMÄ§È �DÍ QÈ�HÆ�Ê��L^ �O¹?��qed=gXd�«ÌµÕ ¹Ë]c¹Y^=^F^�¹Ï�Ò�]~^orqed %ñ��q�v�b_cedrxD�§��qLdr�=v_f~����b�ÙH�=fD��b_x(cÉx~tu��wLtu�¸ÙLv_dr�=x(c��Xf(b�cH�4��qed�Ï�×Mf(v�tLdF�4xD� è � ½ ¯ Ò�� �

xD� ��qed*sLf?�Xf�w§x(b_c��.½ ¯ ^ �����^S0�:3<S�*�3��������� / p*<S0��3��p�53���� j E "

� � � 0!1S�3��b �������7� / p*<60��3��p�5���� � � / p*<S0��3��p�53��� � ^65=:�:68*^��6�*^ ��� � u ��5*238 � �(�����7� / p*<S0��3�*p�53��� �=:3<S0*�=1������4����������:3<S0��=1S� �=:3<S0*�=1�� � / ��<;0�1���p / >38 � �3�[1 / � � 8�^��������3��p�5����*86� ���4>65*<35����S0�p*8��=1S57��8 ���4� / <*: / <����60�p�8��=1657�98 �7� �=d�� ���S0�p�8��=1S57�98��=:3^38��60=m ��b68������������:3<S0��=1S� �a5���<38�^����� � 5*^�8�:S53���=8�>4<6���(�����

0�� e�� ��p*<�� g<�b68*1

�*8*>$�=1 ���� �3�*�� ���=:��������' �8�m60=< �

�S0

1 / � � 8*^�����������p�53���=86� � >65*<65����S0�p�8���1S57�98 � t'u � �� / <�: / <����S0�p*8��=1S57��8 � t'u � ��=d�� ���60�p�8��=1657�98��=:�^38��S0=m � t'u � �� b�0�p�842�8�<6�=:3<6�4� �8��� � � � �a�>6�

��5��*8��3�40!1�6h 1 / � � 8�^��������3��p�53���*86� � � ��� ��o�����?�?�9h >65=<65����S0*p�8��=1S5��98 � � ��� ��o����V?�?�Sh � / <�: / <����S0�p�8���1S57�98 � � ��� ��o�����?�?� h��=d!� ���S0�p*8��=1S57��8��=:3^38��S0=m � � ��� ��o�����?�?� � h��*8*>$�=1 � �� ������ ����:��������' �

8�m60=< � ?�?86�*53�

>6�=168��b�0���<� 8�m�:3^�� ��� ��� t ��� � �0�� e���� / <�: / <����S0�p*8��=1S57��8 � t�u � ��g ��� e��3��d�� ���S0*p�8��=1S5��98��=:3^�8��S0=m � t'u � ��g<�b68*1

�*8�>$�=1����� �3���� ���=:��������' �8�mS0=< �

�S0

0�� e���1 / � � 8*^�����������p�53���=86���*2�8 � g<�b68*1

83p3�*8

|(Ï

Page 39: Sequential Minimal Optimization for SVM Contents 1 Introduction to

86�!bS� 78�^�^6�*^�� 0�1�d65�p30=>41 / � � 8�^;��� ��p�5����*86� \ ��1 / � � 8*^�����������p�53���=86�6h ? ��bS� / p=> � 8�� � � ��' �8�m60=< �

�S0

0�� e�� �����*>65=<65����S0*p�8��=1S5��98�g<�b68*1

86�!bS� 78�^�^6�*^�� ��5�1�1S�*<;�=:38*1�>65*<35(�S0�p�8 � �*>65*<35����S0�p*8��=1S57��8� ���' �8�m60=< �

�S0

<��S:#�*>35*<65����60�p�8��=1657�98 � ��� �*<��S:6�7� / p*<S0���p�53���=<=�6:��=d=�M� >35�<65<��S:#��� / <�: / <����S0�p�8��=1S57�98 � ��� �*<!�S:S�!� / p=<�0���p�53�3��<��S:9�=d�� ��� / <�: / <5�p�p��*<35*^�238�<����S0�p�8��=1S57�98 � ��� �*<!�S:S�!� / p=<�0���p�53�3��<��S:9�=d�� ��5�p�p��*<S5=^�238�<��5*<��*>35*<65����60�p�8��=1657�98 ��5 � � ��i :3^S0!13<�� t � � � � ��5�p�p��*<35*^�238�<����S0�p�8��=1S57�98:3^S0�13<�� � � ��� / <�: / <����S0�p*8��=1S57��80 � � b�0�p�8 e �30���p=<��=1 / � � 8�^��������3��p�5����*86�4g>6�

:3^60�13<�� ��*p�53��� ��0 � 0�1�>S0=dS0=> / 5�p��3��d�� ���S0*p�8��=1S5��98 � ��i��*d�� ���S0�p*8��=1�57�98��*:3^38��S0*m � � �60��5=<��*>65*<35����S0�p*8��=1S57��8 �5 � � ��i(�6�=^ \,0 � � ? 0�! t � ? 0�"�"�h

:3^S0�13<��R\7 �6� �_ �306h ?0�� \ � t � ��� � �30��(h:3^S0�13<��R\7 � ��1 *h ?

83p��*8:3^S0�13<��R\7 � � �=1 *h ?

� � ���*<��S:��*>65*<65����S0�p�8��=1S57�98��� �3���3���9� � � �

�����=<��S:#�*>35*<65����60�p�8��=1657�98 �

��� �=<��S:#��� / <�: / <����S0�p�8��=1S57�98 �

�7� ��0�13>S0=d60=> / 5�p��3�=d�� ���S0�p�8���1S57�98:S5��=<38���� / <�: / <����S0�p�8���1S57�98 �*<��6:#��� / <*: / <����60�p�8��=1657�98 ����i�� / <�: / <����S0�p�8���1S57�98 � � <��S:��d���i�� / <*: / <����60�p�8��=1657�98 � �A<��S: ��� / <�: / <����S0�p*8��=1S57��8^�� �*<��S:#�=>65*<65����S0�p�8���1S57�98 �*<��S:���� / <�: / <����S0*p�8��=1S5��980 � � 8�m�:3^ ��0(" � �

>6�=168:3^S0�13<�� �� / p*<60��3��p�5���� � :S53�=<38���� / <�: / <����S0�p*8��=1S57��8 ��5�p�p��*<65=^�238�<����S0�p�8���1S57�98 �5 � � � x'w ��� t i 1#�*<6�*<35�p � R�

1#��8�^�^3�*^ � R��

i� 86�=<��*d65�p � � �� 86�=<��30 ����6�*^ \�0 ��� ? 0�! t � ?a0�"�"�h

0�� \ �30�� � 86�=<��*d65�pSh�i� 86��<��*d65�p � �30� 86��<��30 � 0

|(²

Page 40: Sequential Minimal Optimization for SVM Contents 1 Introduction to

� 86�=<��30����0�� \ � 86�=<��30 � � � t � h

1���8�^�^6�=^�"�"1#�*<6�=<65�p�"�"

�w't �;i :3^60�13<k1#�*8�^�^6�*^3�=1#�*<6�=<65�p � �

^�� ��5�p�p��*<65*^�238�<����60�p�8��=1657�98G

��®<­�� ¸ ³ �h² � \=²�� �S5 �S8��S0�p*8� ��� #"

5�p�p � �(����� <38*m ������� >�dS0��������D:��������*>�8�����*>38 ���� �=>;�`?X��5��S8������� >�dS0 � �����V� <38�m �

:�0��*���S0=2 � � 8*:���:�0��*���S0=2 � � 8*:���:�0��*���S0=2���� 8*:�� �

:�0��*���S0=2��R� 8*:���:�0��*���S0=2 �V� 8*:���:�0��*���S0=2���� 8*:�� �

:�0��*�������� �]� 8*:��:�0������60=2 � � 8=:�� � :�0��*���S0=2 � �A:�0��:�0������60=2 � � 8=:�� � :�0��*���S0=2 � �A:�0��:�0������60=2��R� 8=:�� � :�0��*���S0=2����A:�0��:�0������60=2��]� 8=:�� � :�0��*���S0=2��R�A:�0��:�0������60=2 ��� 8=:�� � :�0��*���S0=2 �V�A:�0��:�0������60=2��R� 8=:�� � :�0��*���S0=2����A:�0��

0�1���p / >38��V\ � � � w h=�*>3�3���*^ / p�86�V� ���G

������S5��S8��60�p�8� ��� F "

5�p�p � �(���

l�� � �3o���� � �*2l�� � ��o���� � � � �G

� 3 �D31�¸3"1���3#5��qegXb_����x~weqedFg��L^¸��^ © tegX�(dF�F^�´ �Xtu��x~g�b�fDv�x(cÁ��tewew§x(g��É×(d�����x~g%z�f(��qeb_cedF�%��x(g*wLfD���Xd=gXcg�d���x(�~ceb���b_x(cA^��_e���e�� i ��i ��� e ����� � ����n�����j��� i � ������� ��;¹~m ªñmc¬�¶_]�me]� ;]F�� ^¹D]F²~²(ÏL^(·"â#`�QÆMÐeÈ U(È�È� ��Ç � ��ÇLÈ � ��� ��Ä�! � � eÈ U#"�$(Ð���Ð��� HÆDÅQÎ �&%uÐDħÎL^h;^~hY^(�ld=dFg��Xqeb�¹Qh;^(�O^~huqed=×?f(sedc¹Q��^ © qHf?����f(��qLf(g�àQà�f ¹DfDcHs��O^Dâ*^~�O^(játLg��Xq�à~^L��zOwegXx?×(d=�z�dFc��X�r�Xx�wLv_fD���F° �"hujÞªTf(v��~x(gXb��Xqez ��x(g"hQi�jÚ��v�f(�X��b�ÙLdFgrsedF��b��~cA^(' � �M e �() ��d @�M��e#*�Vi�� �e¹A]�|eªV|&¬�¶ �(|Q � Q�(�~²e¹(jæfDg���qámDÕ(ÕL](^

�H^ؼØv�f?����^�ÛLf~�­�O��g�fDb_ceb_ce�ÀxD�%��tLwewHx~g���×~dF���Xx(g�z�f(��qLb�ced��OtL��b�ce�Á��d�Õ�tedFc~�Xb_f(v�zOb�ceb_z�fDvx(wu�Xb�zOb_¬Ff?�Xb�x~cA^;��c © ^�hu��qLx(v_�(x(we�=¹u��^ © tLg��~dF��¹~fDcHs�´*^uhQzOx(v�f ¹~dFsub���x~gX��¹,+�����e �@����� i ��� �� � �.d����.-a�� �0/ ������� ��J� ��� ���� �n��e� ���i ���D^Hjá�­o�¼g�d�����¹;]F²~²(Ïe^

�~Õ

Page 41: Sequential Minimal Optimization for SVM Contents 1 Introduction to

� ��¥�3�� 31� �3¥�2���3.��24/��¸5[/ �À2*¥�3 ��¤��B¤ �$��3-�³5z�C�C�Þ/��42*�=1 �����¤*1�3#5

hQtewLwHx���d Ô ¾ ¶ S»Ð ½ Ò�� ¾ Ì Õe¹�fDcLs Ô Á ¶ S»Ð ½�Ò�� Á Ì ÕÁf(g�d���qLdá�­��xÀwLf(gXf(v�v_d=vwev�fDced��=^ © dF�FfDtL��d���qedFà�fDgXd�wLfDg�fDv_v�dFv�¹��Xqed=à��FfDc�qLfM×(d���qLd*��f(z�d���d=b_�(q���×(dF����x~g S ^Ø`Ad����� Ì ?��1+6? 5Á ¹Lf(cLs�Ì�� ¾ ÒÀ���V^rhQx�� ¾ Ì����&Ö��fDcLsá� Á Ì����^Ò Q^YÊÞd*�=fDc�gXd=��gXb��Xdl��qedd�Õ�tLfD��b_x(cL�rf~�

Ô ¾ ¶ S]Ð ½�Ò ªG� � Ö�O¬hÌ�ÕÔ Á ¶ S]Ð ½�Ò ªG� � Ò�O¬hÌ�Õ

x(gÔ ¾�¶ S]Ð ½�ÒÁ� � Ì�Ô Á%¶ S]Ð ½�ÒÁ� � ÌfÒ�

��b�×Qb�sudl��qed%d'Õ�tLf?�Xb�x~cL�ryQà�i¹e��d�qHfM×(dÔ ¾�¶ ] S]Ð ½�Ò

� � ÌT]

Ô Á%¶ ] S]Ð ½�Ò� � ÌfÒ%]

`Ad=� è�� Ì ¾� S fDcLsá��Ì ?��� ¹u��d%qHfM×(d

Ô ¾ ¶ èÑÐ ½�ÒÁ��̶Ö�]Ô Á ¶ èÑÐ ½�ÒÁ��Ì�Ò%]

� ��¥�3Ú/����Ë3.��2*����3 �c�á1���2*�=/�1 / ��2*¥�3 ����¤ �¡���¸/����=3-�Ûex~gr��qed*�=x(cQ×(d�Ó?Õ�tLf(sug�f?�Xb_��wegXb�z�f(vYweg�x~yev_d=z

zOb�cLb�zOb_¬=d9;K ?1K +(�

¾Á è E è Ö # � ïTS ¾ � ¯��teyML­d������Xx À(¯dª è E ½A¯�ÒÁ�$¬HÖ,�=¯�Ò ]$G�Õ ¿ ] I-%�IȺ

�=¯ G�Õ ¿ ] I-%�IȺC¿��qLd*`ËfD�~gXf(ce�(b�fDc�b_�

N ª è ¿X�q¿�=¯��AOa¿���¬ Ì ]m è E è Ö #

ÃR¯TS ¾ �=¯

ÒÃR¯nS ¾ ®4¯��-À~¯<ª è E ½A¯�ÒÁ�$¬HÖ,�=¯�Ò ]���Ò

ÃR¯nS ¾� ¯ �=¯

Ì ]m è E è Ö

ÃR¯TS ¾ ªV#¶ÒÀ® ¯�Ò � ¯n¬ �=¯

Ò ÃR¯nS ¾ ® ¯ À ¯ ½ E¯ � è Ò ÃR

¯TS ¾ ® ¯ À ¯ � �*ÖÃR¯TS ¾ ® ¯ ¿

�L]

Page 42: Sequential Minimal Optimization for SVM Contents 1 Introduction to

��qedFg�d$O�¹ ��fDgXd���qed*` fD�(g�fDcL�(d"z�tev���b_wev�b_d=g���¹Q��qed�ÊÞx(v���d*sutLf(vAwegXx(yev_d=z b��z�f;Óub�zOb_¬=d� K � N ª è ¿��q¿�� ¯ � O�¿��.¬��teyML­d������Xx � �

� 9 Ì q� �

� ? �� �

� +(� Ì�Õ ]"I�%3I¡ºO G q� G qHÂ

orqed%��x~cL����g�fDb_c~� � �� 9 Ìrq�b�zOwev_b_dF�

è E ÒÃR¯nS ¾ ®4¯TÀ(¯T½ E¯ Ìrqk¿

x(g'¹ud�Õ�teb_×?fDv_d=c���v_àc¹è Ì

ÃR¯nS ¾ ® ¯ À ¯ ½ ¯ Â

orqed%��x~cL����g�fDb_c~� � �� ? Ì�Õ�b�zOwev_b�d��Ã

R¯nS ¾ ® ¯ À ¯ Ì�ÕeÂ

orqed%��x~cL����g�fDb_c~��� � �� +(� ÌåÕOb_zOwev�à#¶ÒÀ® ¯ Ò � ¯ Ì�Õ ¿ ]$I %3IȺCÂ

}#x(��d���qHf?� O�G qk¹�� G qH¹efDcLs���dlqLfM×(dÕfI ® ¯QI #�Â

hQteyH�­�Xb��Xtu��b_ce�O��qLdF��d%g�d���tev��X�rb_c���x N ª è ¿��q¿��=¯ � O�¿��.¬�¶N ª è ¿X�q¿�=¯��AOa¿���¬ Ì ]

m è E è ÖÃR¯TS ¾ Õ � �F¯�Ò è E è ÒÞÕ �á�BÖ

ÃR¯nS ¾ ®4¯

Ì Ò ]m è E è ÖÃR¯TS ¾ ® ¯

ÌÃR¯TS ¾ ® ¯ Ò

]mÃR¯TS ¾

ÃR� S ¾

À ¯ À � ½ E¯ ½ � ® ¯ ® �o xO��tezOz�fDgXb�¬Fdc¹Q�Xqed*sutLf(vYweg�x~yev_d=z b��

z�f;Óub�zOb_¬=d� N � ÌÃ�¯nS ¾ ®4¯kÒ

¾ÁÃ�¯nS ¾

Ã�� S ¾

À(¯TÀ � ½ E¯ ½ � ®4¯V® ���teyML­d������Xx

Ã�¯TS ¾ À(¯V®4¯sÌ�ÕÕ_I ®4¯ I # ]"I�%QI ºCÂ

��m