efficient algorithms in computational learning theory a thesis presented by rocco anthony

199

Upload: others

Post on 11-Feb-2022

0 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

��� ������� � ������������� � ���� ��� ���� �!��"�����# %$&��'�(�)�*�+� , �-�.���0/

1 24365*798:7<;>=95*795*?@245*ABDC

EGFIHJHKFMLONQP*R�FSNUTWVYXKZ�[0XK\^]_F

24`a 365cbd8fe68g7h8g`i?-`Kjlkm?6n@8:?65�5�=48g?6n�oJ?>A-1;>;Sp:8g5*A)qsr�8g5*?>rt5u7

8:?-;UoK=v248woKp0jyx>p{zUp:pg|}5*?D2~`Jj(24365!=95u��x>8g=95*|}5*?D247jy`J=~24365cA65*nJ=h5�5�`Jj

bd`sr"24`J=.`Kj��m3>8:p:`J7h`K;S3 C8:?�2�365�79x B6� 5ur"2�`Jj� `J|};Sx�2h5*=�qsr*8{5u?>rt5

� o�=9eKo�=4A-��?>8fe@5*=h798{2 C� oJ| B =48gA6ni5J�>��oJ7974oKr�3sx>7v5t292�7

�)o C��K�i�>�

Page 2: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

��O�4���u�l�"����� ��� �~�^�"� �¡���"��¢*£¥¤§¦�£�¨¡©ª���«ª«i¤¬©ª­��"� ®I¤¬£¥®¬£¥¤§¦�£¥¨i¯

©ª©

Page 3: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

° ± ²�³'´Gµ%¶O³

· ¸º¹9»f¼¾½(¿�ÀGÁ"¸f¼�ÂK¹ÄÃ�»fÅ�ƾÇh¸º¹9ÈsÉ'Êm¸gÂÌËÍÆKÉÏÎ�Ð@¹vÑu¹9¸fÆ�Â�Ñ¡»0ÒS¼�ÑuÇ4Â�¸{Â�ÅG·ÍÈ�¼tƾÇvÓÃ�Ð@¹9È�ƾÇt½lÔ.Æ@Á"Á�ÆÏÃ�ÂK¹9ÈDÆ�ÂKÓ�ÕJ¼tÇvÖK¼�×�¸{ÆÃ�×@ÖD¸gÊ¥Æ�Çt½lÒS¼�ÊØ»g¸f¼�Ù�ÑÚ»{¸fÑÚÂK¹

�-Û~Ü9ݧ��¤U��Þ�£��.Þ¡¤§���¡«ª£�Ûß©à�Ä� �¡£�� �¡£¥��¤§�~��á@Û~Ü � �¡©ª�¡£�«à£ØÜ4¤¬�¡©ª�¡­Í©ª®Q�¬� ¨¡£â¦�£�«ª��Þ�Ü��<£¥ã � ©ª£��"�Ü�«ª­���¤¬©ä� �¡Ûåáf��¤ «à£ØÜ4¤¬�¡©ª�¡­!æ0����«à£�Ü��#áf��¤§Ûmç¡«èÜ4®m©à��é(©ª®fݧç¡� � � ©ä¦�£�ê���¤¬Û~Ü�«ìëÚ��¤§Û í{éîê�ë>ïð¯�ñI�¡©ª®� �u£�®¬©ª®î¤¬£¥ÞK��¤y� ®(Þu¤¬��­�¤§£�®§®(� �vòìÜ4¤¬¨¡®î®§ç � �'Ü4��Ü�«à­���¤¬©ª�¬�¡Û¯�óÏ£ô¨¡£�¤§©ª¦�£õÜ.�¡£¥òMÜ�«à­�£��¡¤¬Ü�© ��� �ÚÜ�¤_öÜ � � £�¤§©à÷�Ü4� ©ª���~�4á�éîê�ëcáf��¤¬Ûmç¡«àÜ�®UÜ��¡¨.çu®¬£��¬�¡©à® � �ÚÜ�¤ Ü � � £¥¤¬©ª÷ØÜ4�¬©à���Ä� � ���u�øÜ�©ª�.ÜÍéîê�ë'«ª£ØÜ�¤§�¡©à�¡­Ü�«ª­���¤¬©ä� �¡ÛùòY�¡© � �d©à®�®¬ç¡�¡®y�øÜ��"�¬©èÜ�«ª«ª�.á{Ü�®y� £¥¤��¬�ÚÜ��d�¬�¡£Í��£�®y�(Þ¡¤§£¥¦�©à��ç¡®(Ü�ÞuÞ¡¤¬�"Ü � �¡£�®¥¯^ñI�¡£Í¤¬ç¡��ö�¡©ª�¡­ � ©ªÛ~£���á���硤0Ü�«à­���¤¬©ä� �¡Ûß©ª®0£âú*Þ����¡£¥�t�¬©èÜ�«J©à�<�¬�¡£ � çu�K£(¤§�*�4�ì��á@�¬�¡£î�tç¡Ûm��£�¤ì��á@¦4Ü�¤§©èÜ��u«à£�®©à�!�¬�¡£Äéîê�ëOáf��¤¬Û ç¡«èÜuûDÜ4®���Þ¡ÞK��®¬£�¨�� �Þ¡¤§£¥¦�©à��çu®�����ç¡�u¨¡®�òY�u© � ��ò0£¥¤¬£Ä£âú*ÞK���¡£��"� ©àÜ�«>©à��� �u£®¬ütçÚÜ4¤¬£�¤¬�����I��á�� �¡£l�tç¡Ûm��£�¤Y��ás¦4Ü�¤¬©àÜ��¡«à£¥®�¯ñI�¡£(�ýòQ�ÄÛÄ��®y�ì©àÛÄÞ���¤§� Ü��"�0¤§£�®¬��硤 � £�®ìáf��¤IÜÍ«à£ØÜ4¤¬�¡©ª�¡­ôÜ�«ª­���¤¬©ä� �¡ÛþÜ�¤¬£ � ��ÛÄÞ¡çu�øÜh� ©à���~� ©ªÛ~£

Ü��¡¨G©ª�¡Þ¡çu�m¨ÚÜ4�øÜ*ÿ0Ü���©ª¨¡£ØÜ�«U«à£�Ü�¤¬�u©à�¡­!Ü4«à­���¤§©ª�¬�¡ÛWò0��ç¡«à¨G®§©àÛmç¡«ä�øÜ��u£���ç¡®§«ª�c��Þu�¬©àÛÄ©à÷¥£~©ä� ®mç¡®§£��áÍ�K�4� ��¤¬£�®§��硤 � £�®¥¯���®¬©à�u­��¬£ � �¡�u©àütç¡£�®áf¤¬��Û � ¤§��Þu�¬��­�¤¬Ü�Þ¡�"��û(òQ£�®¬�¡�vòW�¬�ÚÜ4�£¥¦�£��+¦�£�¤y�®¬©ªÛ~Þu«à£d«ª£ØÜ�¤§�¡©à�u­#Þ¡¤¬���¡«à£¥Û~® � Ü�� £âú*�¡©ª�¡©ª�<®§� ¤§���¡­�©à�u�¡£�¤§£��"��� ¤¬Ü�¨¡£����J®���£¥�ýòQ£�£¥���¬�¡£�®§£��ýòQ�¤¬£¥®¬��çu¤ � £¥®�¯>óÏ£�¨¡£¥® � ¤§©à��£(Ü � «èÜ4®¬®0�4á@¨¡£ � ©à®§©à���~«à©à®y� ®UÜ��¡¨~Þ¡¤¬�v¦�£�� �ÚÜh�Q�¬�¡£�®§£(®§�¬¤¬ç � � ç¡¤§£�® � Ü��.��£«à£�Ü�¤¬�u£�¨Ä©à�õÞK��«ª���¡��ÛÄ©èÜ�«�� ©ªÛ~£0áf¤§��ÛßÜl«èÜ�¤§­�£ì¨ÚÜh�øÜl®¬£¥�Q�¡çu� � Ü��¡�¡���U��£Y«à£ØÜ4¤¬�¡£¥¨~©ª�~Þ���«ª���¡��Û~©àÜ�«� ©ªÛ~£lç¡®§©à�¡­<Ü~ÛÄ©à�¡©ªÛ<Ü4«Sí � ���¡®§� Ü��"�(®§©à÷�£Øï�¨ÚÜh�øÜ~®§£¥��¯^ñI�¡£¥®¬£m¤§£�®§ç¡«ª�¬®�£�®y�øÜ��u«à©à®§���¬�ÚÜ4��áf��¤^®§��ÛÄ£�ÚÜ4�¬ç¡¤ Ü4«*«ª£ØÜ�¤§�¡©à�¡­�Þ¡¤§���¡«à£¥Û~®��§©ª¨¡£ØÜ�«��Y«à£ØÜ4¤¬�¡©ª�¡­�Ü4«à­���¤§©ª�¬�¡Û~®@®§©àÛmçu«ª�øÜ4�¡£���çu®¬«ª�l��Þu� ©ªÛ~©ª÷�©à�u­I�K���¬�¤¬£¥®¬��çu¤ � £¥® � Ü��¡�¡�4�^£ðú*©à®§��¯

� £¥¤ � £¥Þu� ¤§���õÜ��¡¨�ó ©à�¡�u�9òGÜ�¤¬£S�ýò0� � «èÜ4®¬®¬© � Ü�«�Ü�«ª­���¤§©ª� �uÛ~®Jáf��¤�«ª£ØÜ�¤§�¡©à�u­�ÜY«à©ª�¡£ØÜ�¤D®§£�ÞÚÜ�¤¬Ü4� ��¤òY�¡© � ���ÚÜئ�£î®¬��Û~£^¤¬£�Û~Ü�¤4Ü4�¡«à£�Þ¡¤¬��ÞK£¥¤§� ©ª£�®¥¯6ó'£(¨u£�ÛÄ���¡®§�¬¤ Ü4�¬£îÜ � «à��®§£ � ���¡�¡£ � � ©ª�����K£â�ýò0£¥£�����*��®y� ©ª�¡­¡ûKÜ<«à£�Ü�¤¬�¡©ª�¡­õ� £ � �u�¡©àütç¡£mòY�¡© � �!�ÚÜ�®(­�Ü�©à�¡£¥¨�òY©ª¨¡£ ç¡®§£ô©à��¤¬£ � £��"�î��£�Ü�¤¬®¥û@Ü��¡¨ �¬�¡£�®§£Ü�«ª­���¤¬©ä� �¡ÛÄ®�áf��¤S«ª£ØÜ�¤§�¡©à�¡­l«ª©à�¡£�Ü�¤ � «àÜ�®¬®§©��Ú£¥¤¬®�¯��^®¬©ª�¡­��K����®§�¬©à�¡­îòQ£ � ���¡®§�¬¤¬ç � �IÜl�¡£¥ò}á{Ü�Û~©ª«ª�Í��á

©ª©à©

Page 4: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

«à£�Ü�¤¬�u©à�¡­�Ü�«ª­���¤§©ª� �uÛ~®�áf��¤î«à©ª�¡£ØÜ�¤ � «èÜ4®¬®¬© �Ú£�¤§®(òY�¡© � � � «ª��®¬£¥«ª� Û~Ü4� � ���¬�¡£õ® Ü�ÛÄÞ¡«ª£ � ��ÛÄÞ¡«à£ðú*©ª�ý�Ü��¡¨Í�¡��©ª®¬£S� ��«ª£�¤ Ü4� � £U��áÚÜ4«à­���¤§©ª�¬�¡Û~®@®§ç � �ôÜ�® � £¥¤ � £�Þu�¬¤¬���õÜ��u¨Íó ©à�u�¡�vòl¯sóÏ£U�¬��çu®��¡£¥«àÞ çu�¡©ªáº�� �u£�®§£�£¥Û~©ª�¡­�«ª�<¨¡©à®§ÞÚÜ�¤¬Ü4� £(�¬��Þ¡© � ®I��ás�K����®§�¬©à�¡­ÄÜ��u¨��¬�¡£�®§£ � «èÜ�®§®¬© � Ü�«i«ª£ØÜ�¤§�¡©à�u­õÜ�«à­���¤¬©ª�¬�¡ÛÄ®�¯ó'£ Ü�«à®§�'Þ¡¤§£�®¬£¥�"���¡£¥ò Ü�«ª­���¤§©ª� �uÛ~®mòY�¡© � �Ì­�©ª¦�£dütçÚÜ��"�¬©ª�øÜh� ©ª¦�£�Þ�£�¤yáf��¤¬Û~Ü�� � £d©ªÛ~Þ¡¤§�v¦�£âö

ÛÄ£��"� ®õáf��¤~Üc¤¬Ü��¡­�£��á(òQ£�«ª«äöý®y� ç¡¨u©à£�¨�Þ¡¤§���¡«à£¥Û~®õ©ª��«ª£ØÜ�¤§�¡©à�¡­!�¬�¡£���¤y��¯OóÏ£dÞ¡¤§£�®§£��"��Üc�¡£¥ò���*��®y� ©ª�¡­ôÜ�«ª­���¤¬©ä� �¡ÛþòY�u© � �©à®ì­�ç¡Ü�¤ Ü��"�¬£�£�¨��¬�ôç¡®¬£l©à�"�¬£�¤¬ÛÄ£�¨u©èÜ4�¬£�Þ¡¤§���ÚÜ��¡©ª«à©ä�ý�õ¨¡©à®y� ¤¬©ª�¡çu�¬©à���¡®òY�¡© � �ÌÜ�¤¬£���Þu�¬©àÛ~Ü�«à«ä�#®¬ÛÄ����� �i¯ÌóÏ£ ç¡®§£�� �u©à®<®§Û~�����¬�}���*��®y� ©ª�¡­'Ü4«à­���¤§©ª�¬�¡Û � �'­�©ª¦�£�� �u£á{Ü�®y� £�®y����u�9òY�~Ü�«ª­���¤¬©ä� �¡Û áf��¤>«à£ØÜ4¤¬�¡©ª�¡­îéîê�ë çu�¡¨¡£�¤>�¬�¡£Iç¡�¡©äáf��¤¬Û�¨¡©à®y� ¤§©à�¡çu�¬©à��� ç¡®¬©ª�¡­lÛ~£¥Ûõö��£�¤¬®§�¡©àÞÍütç¡£�¤§©à£�®¥û"Ü��¡¨ Ü�«à®§�^�¬�^©ªÛ~Þ¡¤§�v¦�£U����t�¡�vòY�ô�ÚÜ�¤§¨*ö � ��¤¬£U®¬£â� � ���¡®§�¬¤¬ç � � ©ª���¡®�©ª� � ��ÛÄÞ¡«à£ðú�ö©ª�ý�õ�¬�¡£���¤y��¯>óÏ£îÜ�«à®§�m¨¡£¥® � ¤§©à��£îÜÍá{Ü�®y�Øû*�¡��©à®§£�� ��«à£�¤¬Ü��"�IÜ�«ª­���¤§©ª� �uÛMáf��¤0«ª£ØÜ�¤§�¡©à�u­ «à©ª�¡£ØÜ4¤ � «èÜ�®§®¬©äö�Ú£¥¤¬®Yç¡�¡¨¡£¥¤Y®§��Û~ÛÄ£¥�¬¤¬© � ¨¡©à®y� ¤§©à�¡çu�¬©à���u®�¯>ë�©ª�ÚÜ�«ª«ª��û"òQ£l­�©ª¦�£(� �¡£îá{Ü�®y� £�®y��t�¡�vòY� Ü�«ª­���¤¬©ä� �¡Ûþáf��¤«à£�Ü�¤¬�u©à�¡­îÛÄ���¡���¬���¡£Yéîê^ë�çu�¡¨¡£�¤S� �u£�çu�¡©ªáf��¤§ÛM¨¡©ª®§�¬¤¬©à�uçu� ©ª���mòY©ª�¬�¡��çu�SÛÄ£�Ûm��£�¤§®¬�¡©ªÞõü�çu£�¤¬©ª£�®¥¯

©ª¦

Page 5: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

° ¶���� ��� ����� ����� ��� ³#²

"!$#&%�')(+*-,/.103254768(+.:90; =<�>/6@?A6CB>/6@')DE( ,GFCDIH8',/(+,GFGJ$KLHM<N4"<O*P*P?A?7>G*QBRHTS/,U?"')#VWHXH8YX4PJ[Z\')6M#801./(+.]#8HTS^>G*-6@';?7>/6@?[*R_`?"')6@H8')./(+,a6@'7#]BRHXB)YUbcBR')HM<d?7>/(+4[DIH8'),a(+,GF�e:fg6@Y8*I6FTV+68474hH8%iDj#�47?"HTSG?PJ32

Gkal@m`no"eCpq(W')4"?hra6M#s#8HTS/'L.G*Pt/?74utv*P%wH8'7*x#8HTSyH@z{*P'�%wHTVWY`4x|Tl@m`nh4"?"HTSG?PJ32} KL*P')DE68,]!C*-VW~`(+V+VW*80i )�OH`B7Y�bc�ubc�[HXHX.aVW*Rbc�[H�HGJu���fu>G*��O')HM<�(+,/FOH8%X?7>G*��LH8taVW*���HXB)Y[�O*-,G*P~8*-,�?768,GHT2X�

9h68DN~8*P'7#QF8')6@?"*P%�S/V1?"H���*-4h�\68V (+68,U?P0�D�#y68.G~`(+4"H8'-0G<L>GHT4"*�BRHTSG')47*�HT,]BRHTDIraS/?76@?7(WHT,/68V�V+*-6@'),/(+,GF?7>G*PH8'7#s(+,/4"ra(+'7*-.EDI*[?"H�./Hj'7*-47*-6@')B)>y(+,�?7>/(+4��a*-V+.1�i��*-4O>a684�68VW<�6-#`4�t{*P*-,�F8*-,/*P'7HTS/4O<�(W?7>�>/(+4�?7(+DI*8068.G~`(+BR*801*-,/BRHTSG'6@F8*-DI*-,U?�68,a.54)SGr/r{H8'7?P�xKL( 4L47S/F8F8*-4"?7(WHT,/4x68,/.^(+,a47(WFT>�?74L>/6M~8*It{*P*-,5( ,U~@68V+S/6@taV+*�?"HDI*5(+,�Dj#�<OH8'7Y�0u68,a.�>/( 4I'7*-4"*-6@')B>�>/684�t{*P*-,�68,�( ,/4"ra(W'6@?7(WHT,1��9I?7>/68,GY���*-4s%wH8'�DE6@Y`(+,GF�D�#F8')68./Sa6@?"*�*R_Xr{*P')(+*-,/BR*�4"HI'7*P<�6@')./(+,/FE68,/.Q*-,��"HM#T6@taVW*8�

9�?7>/68,GY�Dj#�<�(+%�*x�8*-,/,�#E%wH8'u68V+VvH8%g>G*P'hVWH�~8*80/47SGr/r{H8'7?P0`68,/.�ra6@?7(W*-,aBR*[H�~8*P'u?7>G*;#8*-6@')4P0G<L(W?7>GHTSG?<�>/( B)>�?7>/( 4;?7>G*-47(+4�,/*P~8*P'�<OHTS/V+.�>/6M~8*�t{*P*-,�<u'(W?"?"*-,1�E9;?7>/68,GY^Dj#^47HT,&�[(+B)>GHTV 684P0�<�>/HT4"*E'7*-BR*-,�?6@'7')(+~�68V�DE68.G*A?7>/(+4Lra684"?;#8*-6@'�( ,`��,/(W?"*-V+#�DIH8')*A( ,U?"*P'7*-47?7(+,GFQ68,/.5*-,��=H�#T6@taVW*8��9L?7>/68,GY$Dj#CDEH8?7>G*P'-0!56@'7FT6@'7*P?��`*P'7~8*-./(WHG0U68,a.ED�#I47( 4"?"*P'-0T!56@')(+6j�X*P')~8*-./(WHG0T%wH8'\?7>G*-(W'�VWHM~8*�68,a.E47SGr/r{H8'7?qH�~8*P'�?7>G*u#8*-6@'4P�

�Xr{*-BP(+68VX?7>/68,/YX4qF8Hx?"H��[./68Dd �V+(+~�68,/4-0�<L>GHxBRHTV+V+6@t{H8')6@?"*-.A<�(W?7>ADI*�HT,A?7>G*h')*-4"*-6@')B)>�.G*-47BR')(Wt{*-.(+,E��>/6@ra?"*P')4i¡L68,a.A¢X�q9£?i>/684qt{*P*-,�6[raV+*-6847SG'7*O?"Hx<�H8')Y�<�(W?7>��[./68Dd68,/.�?"HxYX,/HM<�>/(+D¤684i6;%w')(W*-,/.1�

!568,U#yBRHTV+V+*-6@FTSG*-4�68,/.]%�')(+*-,/./4�>/6-~8*j>G*-VWr{*-.yDE6@Y8*�Dj#Q?7(+DI*�(+,QF8')68.aS/6@?"*�47B>GH�HTV�r/'7H`./S/BR?7(W~8*80DI*-DIH8'6@taVW*A68,/.5*-,��=H�#T6@taVW*8��f�>G*P'7*A6@'7*I?"H�HyDE68,�#$?"Hy,/68DE*801taSG?;68DIHT,GFQ?7>G*-D�9�<OHTS/V+.5V+(WY8*�?"H47(+,/FTVW*�HTSG?[�L./68D��[*-6@?"HT,�¥w%�H8'[?7>G*jV S/,/B)>/*-4¦0��[DIHT4[��*-( DI*-V�¥w%wH8'L?7>/*jBRH@z{*P*�¦0� xHT4"?7684[!C6@F8HTSG?7( 4¥w%wH8'�?7>G*h~8HTV+V+*P#�ta68V V�¦g68,/.s�X?"*P~8*-,Q§[H8'7?7VW*P';¥w%�H8'�?7>G*hta684"*Pta68V V�¦�g9i68V+47Hx?7>/68,GY���6@'7HTVGK[6@')VWH�<�0�¨�HT,/./6�`BRH8?"?P0`©�VW*-,a(/�[')(+,G*-6`0X��(+V+V (+68,A��*P*80`!C(+B>/6@*-Va�O*-,/./*P'-0�¨�HT,/(a �>a6@').GHT,10U�;68,s¨uH8?7>10Xª�>/*P*-VW*P'�¨LS/DEV«0��S/Y8*�K[S/,/4"t{*P'7F8*P'M0a¨u*Pt{*-BPBP6QK�<�6`0�¬�68,�­{HT,GF��;(+,/FG0{�THT>/,$�;S/,/6@FT68,�0v�i*-,/Y�6@?"*-4768,�§;S/')S/4"<�68DE(«0�8*Rz{'7*P#s�U68B7Y`4"HT,10/¨L(+B)>/6@'.s�O*-(WF8*-V«0`�L~�')( D®��V+S/D]0G�`68,�?"HT47>��i*-DIra68V 6`0`68,a.sDE68,�#EH8?7>G*P'4P�\fu>/68,GY`4?"Hs!5(+B>/6@*-Vg!5(W?"¯P*-,/D�68B)>G*P'-0{!5(+B)>/6@*-Vg¨�6@ta( ,10a68,/.^�`68V+(+V{�\68./>a68,]%wH8'[4"*P')~X(+,/FIHT,CDj#Q?7>G*-47(+4LBRHTDAbDE(W?"?"*P*8�

fu>a(+4�?7>G*-47(+4h( 4u.G*-./(+BP6@?"*-.]?"HE?7>G*�DI*-DEH8'7#sH8%qDj#�%�6@?7>G*P'-0�pG')68,GY]�X*P'7~8*-./(+HG�

fu>/( 4�')*-4"*-6@')B)>$<O684�47SGr/r{H8'7?"*-.�tX#�6E�[6@?7(WHT,/68V��GBP(W*-,/BR*�pGHTS/,a./6@?7(WHT,C§['68./S/6@?"*�¨u*-47*-6@')B)>CpG*-V+VWH�<hb47>/(+r�0gtX#:°;�[¨±F8')68,U?I�[²8²8²G³R´@b¶µ8·�b)³¸bc²T¹8¹@²`0utX#��;�`p�F8')68,�?I�u�h¨hb¶µ8¹�bc²@´8´U¡8·X0�68,a.&tX#&�x�`pºF8')68,�?�u��¨ub¶µ8¢�b£»8»�²@´UµX�¼8½7¾O¿vÀXÁ/ÂcÃ/¶Ä�¿v¿aÄ�Å�Æ8¶Ä�¿v¿�ÇÉÈ`¿v¿�ÇÉÈ`¿�¿gÇ ÈG¿�¿gÇ È`¿v¿gÇ3Ê

¦

Page 6: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

Ë Ì ± � Ì ��� ´GµÎÍ Ï Ì ¶ Ð � ³��

Ñ ��®y����ás�¬�¡£l¤¬£¥®¬£ØÜ4¤ � �dÞu¤¬£�®§£��"� £¥¨ ©à���¬�¡©à®ì�¬�¡£�®§©à®Y�ÚÜ�®�Ü�Þ¡Þ�£ØÜ4¤¬£�¨£�«ª®¬£âòY�¡£�¤§£Í©ª�®§��ÛÄ£láf��¤¬Û¯Ò �ÚÜ�Þu�¬£�¤�Ó~©à®��ÚÜ�®§£�¨ ����¬�¡£ ÞÚÜ4ÞK£¥¤^�¸Ô@£�Ü�¤¬�¡©ª�¡­~éîê�ë�©à�dñI©ªÛ~£Í��ÕÖg×ÙØ�Ú�ÛÝÜ7Þ �~òY�¡© � �d©à®6ݧ��©ª�t�òQ��¤¸ òY©ä� �c��¨ÚÜ�Ûàßl«ª©ª¦4Ü��¡®lÜ4�¡¨'Ü4Þ¡ÞK£�Ü�¤¬£¥¨Ï©ª�c�¬�¡£ � ¤¬� � £�£¥¨¡©à�u­�®Í�4á0� �u£<�4���u�~¢���Û~Þ���®§©àç¡Û����ñI�¡£¥��¤§����á Ò ��ÛÄÞ¡çu�¬©à�¡­�í¶ß©ª¦4Ü��¡®IÜ4�¡¨�¢*£�¤y¦�£�¨u©à�¡ûK�����*�9ïâ¯Ò �ÚÜ�Þu�¬£�¤\áI©à®@�¡Ü�®¬£¥¨����l�¬�¡£>ÞÚÜ�Þ�£�¤�� Ò ��Û~Þuçu�øÜ4�¬©à���¡Ü�«�¢*Ü�Û~Þu«à£ Ò ��Û~Þ¡«ª£âú*©ª�ý�(Ü��¡¨î�ì�§� ¤¬©ª�¡çu�¬£âö

â ã � ©à£¥�"��Ô@£ØÜ�¤§�¡©à�¡­X�ÍòY�¡© � ��Ü�Þ¡Þ�£ØÜ�¤§£�¨.©ª�<� �¡£�ã1äUå�æ-ç�èXé�ä7ê�ëLäUì�í�åaî7ïMæEèXç�ðQñ�ò8óMî7ïMìôñqõMö£ïMç�õPïPóí:¢�£�¤§¦�£�¨¡©ª�¡ûÚ�4������Ü"ïðÿuÜÍÞ¡¤¬£¥«à©àÛÄ©à�¡Ü�¤§�ͦ�£�¤§®¬©ª������á@� �u©à®UÞÚÜ�Þ�£�¤ìÜ4Þ¡ÞK£�Ü�¤¬£¥¨�©à�Ä� �¡£ � ¤§� � £�£�¨u©à�¡­�®Q��á� �u£Ä�@÷`÷`÷Ä¢���Û~Þ���®§©àç¡Û&����ñI�¡£���¤y����á Ò ��Û~Þuçu� ©ª�¡­�í:¢*£¥¤§¦�£¥¨¡©à�uû@�8÷`÷`÷�Ü�ïâ¯

Ò �ÚÜ�Þu�¬£�¤§®ùø#Ü��u¨®ú�Ü�¤§£!�ÚÜ�®§£�¨����Ì� �¡£!ÞÚÜ4ÞK£¥¤��Rû(� � � Ò Ô@£ØÜ4¤¬�¡©ª�¡­Gç¡®¬©ª�¡­Gó ©à�¡�u�9òlû� £¥¤ � £�Þu�¬¤¬���iûJÜ��¡¨�Ü � £�¤ � £�Þu�¬¤¬����öý«à©��£���«ª­���¤¬©ä� �¡Û$� òY�¡© � � Ü�ÞuÞK£�Ü�¤¬£¥¨d©à��� �u£ � ¤¬� � £¥£�¨¡©ª�¡­�®���á� �u£Ä�@÷`÷`÷ Ò ���uáf£¥¤¬£�� � £���� Ò ��Û~Þuçu�øÜ4�¬©à���¡Ü�«qÔ@£ØÜ4¤¬�¡©ª�¡­õñI�¡£���¤y�Ïí:¢*£¥¤§¦�£�¨¡©ª�¡û@�8÷`÷`÷4�¾ïâ¯Ò �ÚÜ�Þu�¬£�¤jü ©à®0�ÚÜ�®¬£¥¨������ �¡£îÞÚÜ4ÞK£¥¤]� � � Ò ���ÚÜ4«à��­�çu£�®ì��á � £�¤ � £�Þu�¬¤¬����Ü��¡¨�ó ©ª�¡�¡�vò+¦�©àÜæ0����®§�¬©à�¡­� �¡£ Ñ Ü�¤§­�©à�{�òY�¡© � �ÏòY©à«à«UÜ�ÞuÞK£�Ü�¤ ©ª��ýþèaõPÿaö«ç�ï��\ï¸èXæ-ç1ö«ç�� íg¢*£�¤y¦�£�¨u©à�¡ûS�4���u�Ø�¾ïðÿSÜÞ¡¤§£�«à©ªÛ~©ª�ÚÜ�¤y�ͦ�£�¤¬®§©à���~��áJ�¬�¡©à®UÞÚÜ�Þ�£�¤UÜ�ÞuÞK£�Ü�¤¬£¥¨<©à�õ� �u£ � ¤¬� � £¥£�¨¡©ª�¡­�®U��ái�¬�¡£(�4����� Ò ���uáf£¥¤¬£�� � £��� Ò ��ÛÄÞ¡çu� Ü4� ©ª���ÚÜ�«�ÔD£�Ü�¤¬�u©à�¡­õñI�¡£¥��¤§�Ïíg¢*£�¤y¦�£�¨u©à�¡û����������¾ïð¯

Ò �ÚÜ�Þu�¬£�¤��©à®��ÚÜ�®§£�¨G���'�¬�¡£<Þ¡Ü�ÞK£¥¤ù�§æ0�*��®§� ©ª�¡­ Ü4�¡¨��^Ü�¤¬¨*ö Ò ��¤¬£.¢*£¥�¬®¸� òY�¡© � �#©à®ìݧ��©ª�t�òQ��¤¸�òY©ª�¬�'��¨¡Ü�Û ßl«ª©ª¦4Ü��¡®�Ü��u¨GÜ�Þ¡Þ�£ØÜ�¤§£�¨#©ª�Ï�¬�¡£d�8÷X÷`÷ ¢���ÛÄÞK��®¬©àçuÛå���GëÚ��ç¡�¡¨¡Ü4� ©ª���¡®l��áÒ ��ÛÄÞ¡çu�¬£�¤^¢ � ©ª£�� � £~í¶ßî«à©ä¦�Ü4�¡®IÜ��¡¨d¢*£¥¤§¦�£¥¨¡©à�uû@�8÷`÷`÷"ïð¯Ò �ÚÜ�Þu�¬£�¤j÷Ä©ª®I�ÚÜ�®§£�¨d����¬�¡£lÞÚÜ�Þ�£�¤5� ¢*ÛÄ�*���¬��æ0����®§�¬©à�¡­ÄÜ��¡¨:Ô@£ØÜ�¤§�¡©à�u­õòY©ä� � Ñ Ü�«ª© � ©ª��ç¡®ê^��©à®¬£��mòY�¡© � ��Ü4Þ¡ÞK£�Ü�¤¬£¥¨�©à�<�¬�¡£ � ¤¬� � £¥£�¨¡©ª�¡­�®I��á@�¬�¡£������u� Ò ���uáf£�¤§£�� � £l��� Ò ��ÛÄÞ¡çu�øÜh� ©à���ÚÜ�«Ô@£ØÜ�¤§�¡©à�u­ÄñI�u£���¤y�Ïí:¢*£¥¤§¦�£¥¨¡©à�uû������u� � ïâ¯Ò �ÚÜ�Þu�¬£�¤õ�v��©à®î�¡Ü�®¬£¥¨Ï���!�¬�¡£ôÞÚÜ�Þ�£�¤&�Rû(��Ô@£ØÜ�¤§�¡©à�u­ Ñ ���¡���¬���¡£õéîê�ëÌç¡�u¨¡£�¤ � ¤¬��¨¡ç � �é(©à®y� ¤§©à�¡çu�¬©à���u®¸�.òY�¡© � �#Ü�Þ¡Þ�£ØÜ4¤¬£�¨c©à��� �u£ � ¤§� � £�£�¨u©à�¡­�®���áQ� �¡£<�����u� Ò ���*áf£�¤¬£¥� � £~��� Ò ��ÛõöÞ¡çu� Ü4� ©ª���ÚÜ�«�Ô@£ØÜ�¤§�¡©à�¡­õñI�¡£¥��¤§�Ïíg¢*£�¤y¦�£¥¨¡©à�¡û��4���u�vÜ"ïð¯

¦�©

Page 7: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

� � � ³�� � ³#²

Â�¹vÇ9Æ@×sÐ�Á�¹9¸fÆ� ��¯ä� Ñ ���¬©ª¦4Ü4� ©ª����¯Í¯�¯�¯Í¯l¯�¯Í¯�¯Í¯�¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ ���¯ � ¢*ç¡ÛÄÛ<Ü4¤§�~��á�ñI�u£�®¬©ª® Ò ���t�¬¤¬©ª�¡çu� ©ª���¡®�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ Ó��¯ÙÓ û(çu� «ª©à�¡£(�4ás�¬�¡£lñI�¡£�®§©à®�¯Í¯�¯Í¯�¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ ø

� � Æ@×�¼�»{ÊÄÑ¡Âs×� mÑÚÁ��ižÇ9Æ�ÐsÂ�× ���*¯ä� Ò ��� � £�Þu�¬®�Ü��u¨d�Y£�Þ¡¤§£�®§£��"�øÜ4�¬©à���u®Ì¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �v��*¯ � � � Ò ÔD£�Ü�¤¬�u©à�¡­ ¯�¯Í¯l¯�¯Í¯�¯Í¯�¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ ����*¯ÙÓ �UÜ�¤¬©àÜ4� ©ª���¡®0Ü��¡¨ â ú��¬£��¡®§©à���¡®���á�� �¡£ � � Ò Ñ ��¨¡£�«Q¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �8Ó�*¯ á ñI�¡£sû(�¡«à©ª�¡£EÔ@£ØÜ�¤§�¡©à�¡­ Ñ �*¨¡£¥«%¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �@á�*¯ ø Ñ Ü4�¬�¡£�Û~Ü4� © � Ü�«JæìÜ � t­�¤§��ç¡�¡¨ ¯�¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �Tø

� � ÑÚÊ¥¹v¼tÇ�ÒS¼tÑ¡ÇhÂs¸{Â�Å'Æ������ � ÖD¸fÑ���ÆK»ºÓDÂDÆ�É#¸{Ñ¡»Q·ÍÈ�Ç9¼�ÊØÈDÆK»{× � Ð�ÂsÁ�¹9¸fÆ�ÂsÊ ��Óu¯ä� ý�"� ¤¬��¨¡ç � � ©ª���+¯�¯�¯Í¯l¯�¯Í¯�¯Í¯�¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �Tü

Óu¯ä��¯ª� � ��«ª���¡��Û~©àÜ�«JñI�¡¤§£�®§�¡��«à¨�ëÚç¡� � � ©à���¡®î¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �TüÓu¯ä��¯ � Ô@£�Ü�¤¬�¡©ª�¡­Äéîê�ë�¯Í¯�¯Í¯�¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �!�Óu¯ä��¯ Ó �Wê^£¥ò ��ÞuÞ¡¤¬�"Ü � �#"ºÔ@£ØÜ�¤§�¡©à�u­�éîê�ë ¦�©èÜ � ��«ª���¡��Û~©àÜ�«^ñI�u¤¬£�®§�¡��«ª¨ëÚç¡� � � ©ª���¡® ¯l¯�¯Í¯�¯Í¯�¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �8÷Óu¯ä��¯Ùá û(硤Y�Y£�®§ç¡«ª�¬® ¯�¯Í¯�¯Í¯�¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ ���

Óu¯ � � ¤¬£�«ª©àÛÄ©à�ÚÜ4¤¬©à£¥® ¯�¯Í¯l¯�¯Í¯�¯Í¯�¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �*�Óu¯ �*¯ª� â ã � ©ª£��"����«ª­���¤¬©ä� �¡ÛÄ®Qáf��¤jÔ@£�Ü�¤¬�¡©ª�¡­]Ô@©à�¡£�Ü�¤IñI�¡¤§£�®¬�u��«à¨�ëÚç¡� � �¬©à���u® ���Óu¯ �*¯ � ñI�u£ Ñ ©à�¡®t�"ö � Ü�Þ�£�¤y�AÔ@�vò0£¥¤�æQ��ç¡�¡¨W¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �XÓ

Óu¯ÙÓ ����û(Þu�¬©àÛ~Ü�«0æQ��ç¡�¡¨Gáf��¤ô�Y£�Þu¤¬£�®§£��"� ©ª�¡­céîê�ë �"� � ��«ª���¡��Û~©àÜ�«QñI�u¤¬£�®§�¡��«ª¨ëÚç¡� � �¬©à���u® ¯Í¯�¯�¯Í¯l¯�¯Í¯�¯Í¯�¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ ��áÓu¯ÙÓu¯ª� Ô@�vòìö)û(¤§¨¡£�¤ � ��«ª���¡��ÛÄ©èÜ4«4ñI�¡¤§£�®§�¡��«à¨�ëÚç¡� � � ©ª���¡®iáf��¤@éîê^ë<òY©ª�¬��¢*Û<Ü4«à«ñ�£�¤§Û~®�¯�¯Í¯l¯�¯Í¯�¯Í¯�¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ ��áÓu¯ÙÓu¯ � ëÚ¤§��Û%éîê^ëG�¬�Äé(£ � ©ª®¬©ª����ñs¤§£�£¥®�¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �XúÓu¯ÙÓu¯ Ó �^�sûîÞ*� ©àÛ~Ü�«4æ0��ç¡�u¨láf��¤@��£¥Þ¡¤¬£¥®¬£¥�t�¬©à�¡­^éîê�ë.�"� � ��«ä�*�u��Û~©àÜ�«hñI�¡¤¬£¥®¬�*ö

��«à¨�ëÚç¡� � � ©à���¡® ¯Í¯�¯Í¯�¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �`üÓu¯ á é(©à® � ç¡®§®¬©à���%¯Í¯�¯�¯Í¯l¯�¯Í¯�¯Í¯�¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �X÷Óu¯ ø Ô@�9òìöýé(£�­�¤§£�£ � ��«ª���¡��ÛÄ©èÜ4«JñI�¡¤§£�®¬�u��«à¨�ëÚç¡� � �¬©à���u®Yáf��¤Y��£�Ü�¨*ö)û(� � £�éîê�ë ¯�¯ Óu�Óu¯Ùú Ô@�9òìö7û(¤¬¨¡£¥¤ � ��«ª���¡��Û~©àÜ�«�ñI�¡¤§£�®§�¡��«à¨'ëÚç¡� � �¬©à���u®láf��¤Í�Y£ØÜ4¨*ö)û(� � £ Ò ���¡®§� Ü��"�é(£�Þu�¬�ë¾��¤¬Ûmç¡«àÜ�£¯Í¯l¯�¯Í¯�¯Í¯�¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ Ó"�

¦*©ª©

Page 8: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

$ ËÍÆKÉÏÎ�Ð@¹vÑu¹9¸fÆ�Â�Ñ¡».ÕJÑÚÉ'Îs»f¼�ËÍÆ�É'Îs»{¼&%D¸º¹ðÓ�ÑÚÂs×&Ãõ¹�¹vÇh¸('sÐD¹Ø¼&)¬¿�À#Á"¸{¼�ÂK¹ÏÒS¼�ÑuÇ4Â#)¸gÂDÅ �*$ᡯä� ý�"� ¤¬��¨¡ç � � ©ª���+¯�¯�¯Í¯l¯�¯Í¯�¯Í¯�¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ ÓXá

ᡯä��¯ª� Ñ ��� ©ä¦�Üh� ©à����¯l¯�¯Í¯�¯Í¯�¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ ÓGøᡯä��¯ � Ò ��Û~Þ¡ç*�øÜ4�¬©à���ÚÜ4«@¢uÜ�ÛÄÞ¡«ª£ Ò ��ÛÄÞ¡«à£ðúu©ä�ý�<¦�£�¤¬®§ç¡®î¢*Ü�Û~Þu«à£ Ò ��ÛÄÞ¡«ª£âú*©ª�ý� Ó`úᡯä��¯ Ó û(硤I�^Þ¡Þ¡¤§�"Ü � ��¯Í¯�¯Í¯�¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ Ó+�

ᡯ � � ¤¬£�«ª©àÛÄ©à�ÚÜ4¤¬©à£¥® ¯�¯Í¯l¯�¯Í¯�¯Í¯�¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ Ó+�ᡯÙÓ � Ò ���¡®y� ¤¬ç � � ©ª���:�^®¬©ª�¡­ â ¤¬¤¬��¤yö Ò ��¤¬¤¬£ � �¬©à�¡­ Ò ��¨¡£�® ¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ Ó`÷

ᡯÙÓu¯ª� â ¤§¤¬��¤_ö Ò ��¤§¤¬£ � � ©ª�¡­ Ò �*¨¡£¥®l¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ á"�ᡯÙÓu¯ � ñI�u£ Ò ��� � £¥Þu� Ò «àÜ�®¬®�,.- ¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ á¡�ᡯÙÓu¯ Ó � ¤§�*��ás��ásñI�¡£���¤¬£�Û áu¯ ÓG¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ át�

ᡯ á ��¢�� ¤§���¡­�£¥¤0/�Ü�ÞM¯Í¯l¯�¯Í¯�¯Í¯�¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ á/øᡯ ᡯª� Ò ¤y��Þu� ��­�¤ Ü�Þ¡�u© � � ¤¬£¥«à©àÛÄ©à�¡Ü�¤¬©ª£�®}¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ á/øᡯ ᡯ � ñI�u£ Ò ��� � £¥Þu� Ò «àÜ�®¬®�,21 ¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ á&�ᡯ ᡯ Ó � ¤§�*��ás��ásñI�¡£���¤¬£�Û áu¯ ÷G¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ á&�ᡯ ᡯÙá �^�ùû(Þu�¬©àÛ~Ü�«3/�Ü�Þ ¯Í¯�¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ áG÷

ᡯ ø ��Ü�¤§¨¡�¡£¥®¬®���á��ì�§� ¤§©à�¡çu�¬£âö â ã � ©à£��"�jÔ@£ØÜ�¤§�¡©à�u­ ¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ ø��ᡯ ø*¯ª� � ¤§�*��ás��ásñI�¡£���¤¬£�Û áu¯ª�8ú ¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ ø��ᡯ ø*¯ � � «àÜ�ç¡®¬©ª�¡©à«ª©ª�ý�Ä��á�� �¡£ Ò ¤y�*Þ*� ��­�¤¬Ü�Þ¡�¡© � ��®§®¬ç¡ÛÄÞu� ©ª��� ¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ ø�á

4 �ìÃ.Ë ÒS¼tÑ¡ÇhÂs¸{ÂDÅ65~¸f¹9È7�(¼tÇhÁ�¼�ÎD¹vÇ9Æ�Â}ÑÚÂ�×98 ¸gÂ�Â�Æ�5 4;:ø*¯ä� ý�"� ¤¬��¨¡ç � � ©ª���+¯�¯�¯Í¯l¯�¯Í¯�¯Í¯�¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ øXú

ø*¯ä��¯ª� ñI�u£ � £�¤ � £�Þ*� ¤¬���!Ü��¡¨�ó ©à�u�¡�vò ��«ª­���¤§©ª� �uÛ~®�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ ø`üø*¯ä��¯ � � � Ò Ô@£ØÜ4¤¬�¡©ª�¡­ôòY©ª� � � £�¤ � £�Þu�¬¤¬��� Ü��¡¨�ó ©à�¡�u�9ò<" � ¤¬£â¦�©à��ç¡®IóÏ��¤¸-¯ øX÷ø*¯ä��¯ Ó � � Ò Ô@£ØÜ4¤¬�¡©ª�¡­ôòY©ª� � � £�¤ � £�Þu�¬¤¬��� Ü��¡¨�ó ©à�¡�u�9ò<"Lû(硤��Y£�®¬çu«ª� ® ¯�¯ ú��

ø*¯ � � ¤¬£�«ª©àÛÄ©à�ÚÜ4¤¬©à£¥® ¯�¯Í¯l¯�¯Í¯�¯Í¯�¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ ú��ø*¯ �*¯ª� � � Ò Ô@£ØÜ4¤¬�¡©ª�¡­$�^®¬©ª�¡­Cû(�¡«à©ª�¡£EÔ@£ØÜ�¤§�¡©à�¡­õ�^«à­���¤§©ª�¬�¡Û~® ¯l¯Í¯�¯Í¯�¯�¯ úu�ø*¯ �*¯ � ê�£�®y� £�¨dëÚç¡� � �¬©à���u®þ¯Í¯�¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ úu�

ø*¯ÙÓ ó ©à�¡�u�9ò Ò Ü��¡�u��� � � Ò Ô@£ØÜ4¤¬� � ��®§©ª� ©ä¦�£<�^Ü�«ªáf®§ÞÚÜ � £¥®�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ ú`Óø*¯ÙÓu¯ª� �=/î£��u£�¤ Ü4« � � Ò Ô@�vò0£¥¤�æ0��ç¡�¡¨�áf��¤ìó ©ª�¡�¡�vò ¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ ú+�

ø*¯ á � £¥¤ � £¥Þu� ¤§���d©ª®�¢*«à�vò+ç¡�¡¨¡£¥¤����¡©ªáf��¤¬Û&é(©à®§�¬¤¬©ª�¡çu� ©ª���¡®ù¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ ú`÷ø*¯ ø � £¥¤ � £¥Þu� ¤§���d©ª®Yë¾Ü�®§�Iáf��¤Yê�£�®y� £�¨dëÚç¡� � �¬©à���u®�çu�¡¨¡£�¤����¡©ªáf��¤§Û%é(©ª®§� ¤§©à�¡ç*� ©à���¡® ü��

: ÒU¼tÑ¡Çh¸{Â�Å�>õÇh¸{Å�¸{Â?)âËͼ�ÂK¹v¼tÇ9¼*×A@�Ñ¡»(�ýÊ¥ÎsÑÚÁ�¼�ÊIÐ�Âs×D¼tÇ^¹9ÈD¼CBÂ�¸D�wƾÇ4ÉE�¸{Ê¥¹vÇh¸D'�Ð@¹9¸fÆ� �F$úu¯ä� ý�"� ¤¬��¨¡ç � � ©ª���+¯�¯�¯Í¯l¯�¯Í¯�¯Í¯�¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ ü�á

úu¯ä��¯ª� � ¤§£¥¦�©à��çu®YóÏ��¤ ¯�¯Í¯�¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ ü�áúu¯ä��¯ � �)ê�£¥ò��^«à­���¤¬©ª�¬�¡Û ¯Í¯�¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ üXú

úu¯ � � ¤¬£�«ª©àÛÄ©à�ÚÜ4¤¬©à£¥® ¯�¯Í¯l¯�¯Í¯�¯Í¯�¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ üXúúu¯ÙÓ ñI�¡£l��«ª­���¤¬©ä� �¡Û ¯Í¯l¯�¯Í¯�¯Í¯�¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ üG�

úu¯ÙÓu¯ª� Ò ��Û~ÞÚÜ4¤¬©à®§������áIH&J*K+LFM+NFK�Ü4�¡¨ � £�¤ � £¥Þu� ¤§���d¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ üG�úu¯ÙÓu¯ � ó �"��H&J�K+L�M+NFK~óÏ��¤¸t®�¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ üX÷

úu¯ á ���ÚÜ4«ª��÷�©ª�¡­m� �¡£<H&J�K+L�M+NFK.��«ª­���¤¬©ä� �¡Û ¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ ���úu¯ ᡯª� � ÔDÜ�¤§­�£ � Ü�¤¬Ü�«à«ª£�« Ò ��Û~Þ����¡£¥�"�~¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ ���úu¯ ᡯ � ��¢�Û<Ü�«ª«iû(¤§�¬�¡��­����ÚÜ�« Ò ��ÛÄÞ����¡£��"�ͯͯ�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �"�

¦�©à©ª©

Page 9: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

úu¯ ᡯ Ó � çu�§� ©ª�¡­õ©ª�I�^«à«iñ���­�£â� �¡£¥¤�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �Xá� �ìÃ.Ë�Ã�ÂDÑÚ»fƾÅKÐD¼�ÊIÆ��O�(¼tÇhÁ�¼tÎ�¹ØÇhÆ�ÂÑÚÂs×P8 ¸{ÂsÂDÆF5 ÖD¸fÑQ mÆiÆ�Ê¥¹9¸{Â�Åõ¹9ÈD¼ � Ñ¡Ç9Å�¸gÂSR �

ü*¯ä� ý�"� ¤¬��¨¡ç � � ©ª���+¯�¯�¯Í¯l¯�¯Í¯�¯Í¯�¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �Güü*¯ä��¯ª� ñI�u£TH&J�K+LFM&NFK.��«ª­���¤§©ª� �uÛ&�Y£¥¦�©ª®¬©ª�¬£�¨O¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �Güü*¯ä��¯ � ñI�u£TH&J�K+LFM&NFK.��«ª­���¤§©ª� �uÛ&�Y£�¨u£�£�ÛÄ£�¨ ¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �`÷ü*¯ä��¯ Ó æQ�*��®y� ©ª�¡­4ö_æ0Ü�®¬£¥¨:Ô@©ª�¡£ØÜ�¤0ñI�¡¤¬£¥®¬�¡��«ª¨ùÔ@£ØÜ4¤¬�¡©ª�¡­ô��«ª­���¤¬©ä� �¡ÛÄ® ¯Í¯�¯�¯ �`÷ü*¯ä��¯Ùá �Y£�«àÜ4� £¥¨�ó'��¤ ¯Í¯�¯Í¯�¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ ÷��

ü*¯ � � ¤¬£�«ª©àÛÄ©à�ÚÜ4¤¬©à£¥® ¯�¯Í¯l¯�¯Í¯�¯Í¯�¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ ÷u�ü*¯ �*¯ª� /î£���Û~£â� ¤¬© � � ¤¬£¥«à©àÛÄ©à�¡Ü�¤¬©ª£�®W¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ ÷u�ü*¯ �*¯ � � � Ò Ô@£ØÜ4¤¬�¡©ª�¡­$ÔD©ª�¡£ØÜ4¤IñI�¡¤¬£¥®¬�¡��«ª¨dëÚç¡� � � ©ª���¡®IòY©ª�¬��¢*£�ÞÚÜ4¤ Ü4�¬©à��� ¯�¯ ÷"�ü*¯ �*¯ Ó ñI�u£�û(�¡«ª©à�¡£2U*öý�¡��¤§Û �^«à­���¤§©ª�¬�¡Û~® ¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ ÷`Óü*¯ �*¯Ùá ëÚ¤§��Û û(�¡«ª©à�¡£��¬� � � Ò Ô@£ØÜ4¤¬�¡©ª�¡­O¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ ÷Gø

ü*¯ÙÓ � � � Ò Ñ �*¨¡£¥«*Uuöý�u��¤¬Ûùó'£�ÜX�Ô@£ØÜ�¤§�¡©à�u­Ä�^«à­���¤¬©ª�¬�¡Û ¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ ÷`úü*¯ á ëÚ¤¬��Ûùó'£�ÜX<� �~¢��¬¤¬���u­5Ô@£ØÜ4¤¬�¡©ª�¡­ ¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ ÷+�

ü*¯ ᡯª� æQ�*��®y� ©ª�¡­m� �õ� � �u©à£¥¦�£V��©à­���� ��� çu¤ Ü � �þ¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ ÷+�ü*¯ ᡯ � æQ�*��®y� ©ª�¡­õ��£�Ü�«#�UÜ�«ªç¡£�¨A����Þ���� �u£�®¬£¥®��¬�~� � �¡©à£â¦�£ÍÜ]Ô�Ü4¤¬­�£ Ñ Ü4¤¬­�©ª�!¯ �v�*�ü*¯ ᡯ Ó ÔDÜ�¤§­�£ Ñ Ü�¤§­�©à�u®YÜ��¡¨P/l£¥�¡£�¤¬Ü�«à©ª÷ØÜ4�¬©à��� â ¤§¤¬��¤ ¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �v��áü*¯ ᡯÙá � çu�§� ©ª�¡­õ©ª�I�^«à«iñ���­�£â� �¡£¥¤�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �v�Xú

ü*¯ ø ��£¥«èÜ4�¬©à���u®¬�¡©ªÞ<òY©ª�¬���¬�¡£sû(�¡«à©ª�¡£2Uuöý�u��¤¬Û&��«à­���¤¬©ä� �¡ÛÄ®î¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �v�`üü*¯ ø*¯ª� UXW �ôÜ��u¨�¬�¡£ � £�¤ � £¥Þu� ¤§���d�^«à­���¤§©ª�¬�¡Û ¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �v�G�ü*¯ ø*¯ � UXWZY Ü��¡¨�� �u£V[�Ü � �®§���*ö Ò ¤ Üئ�£¥� ��«ª­���¤¬©ä� �¡Û ¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �v�G�

R mÆiÆ�Ê¥¹9¸{Â�ÅÏÑÚÂ�×�@�Ñ¡Çh×#)¥ËÍƾÇ9¼ÏÕi¼"¹�ËÍÆ�ÂsÊ¥¹ØÇ4Ð�Á�¹9¸fÆ� �;�u¯ä� ý�"� ¤¬��¨¡ç � � ©ª���+¯�¯�¯Í¯l¯�¯Í¯�¯Í¯�¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �����

�u¯ä��¯ª� æQ�*��®y� ©ª�¡­õÜ��¡¨\��Ü�¤§¨*ö Ò ��¤¬£�¢*£â� ® ¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ ������u¯ä��¯ � û(硤Y�Y£�®§ç¡«ª�¬® ¯�¯Í¯�¯Í¯�¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ ���@Ó�u¯ä��¯ Ó �Y£�«àÜ4� £¥¨�ó'��¤ ¯Í¯�¯Í¯�¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ ����á

�u¯ � ��Ü�¤§¨*ö Ò ��¤¬£�¢*£â� Ò ���¡®§�¬¤¬ç � � ©à���ùû�¦�£¥¤§¦�©à£âò ¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ ����á�u¯ �*¯ª� â ú*©à®y� £¥� � £���á]��Ü�¤§¨*ö Ò ��¤§£ Ñ £ØÜ�®§ç¡¤¬£¥® ¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ ���@ú

�u¯ÙÓ æ0����®§�¬©à�¡­$û�¦�£¥¤§¦�©à£âò ¯�¯Í¯�¯Í¯�¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ ���8ü�u¯ÙÓu¯ª� ¢t� ¤¬ç � � çu¤¬£l��á>æQ�*��®y� ©ª�¡­ô��«ª­���¤¬©ä� �¡ÛÄ®M¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ ���^�

�u¯ á ��Ü�¤§¨*ö Ò ��¤¬£�¢*£â� Ò ���¡®§�¬¤¬ç � � ©à���áf¤§��Û%æ0����®y� ©à�u­}¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ ���@÷�u¯ ᡯª� ��¢t� ¤¬ç � � çu¤ Ü�«s¢*©ªÛ~©ª«èÜ�¤§©ª�ý��¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ ���@÷�u¯ ᡯ � �=/î£��u£�¤ Ü4«O��Ü�¤§¨*ö Ò ��¤§£Í¢*£¥� Ò ���¡®y� ¤§ç � �¬©à��� ¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �9�4��u¯ ᡯ Ó ê�£¥òZ��Ü�¤§¨*ö Ò ��¤§£Í¢*£¥� Ò ���¡®y� ¤§ç � �¬©à���¡®ô¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �9����u¯ ᡯÙá ëÚ¤§��Û_�^Ü�¤¬¨*ö Ò ��¤¬£ Ñ £ØÜ�®§ç¡¤¬£¥®^� ���^Ü�¤¬¨*ö Ò ��¤¬£Í¢*£â� ® ¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �9�Uá�u¯ ᡯ ø ýÛÄÞ¡¤§�9¦�©ª�¡­ô� �¡£Í¢*£â�(¢�©à÷�£ � Ü�¤ Ü4Û~£â� £�¤!¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �9��ú�u¯ ᡯ ú û(Þu�¬©àÛ~Ü�«#�^Ü�¤¬¨�ö Ò ��¤§£�¢*£¥� Ò ���¡®y� ¤§ç � �¬©à��� ¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �9�`��u¯ ᡯ ü �)æQ�*��®y� ©ª�¡­õ��«à­���¤¬©ä� �¡Û áf¤¬��Û_acb�d ¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �9�`�

�u¯ ø ë¾Ü�®§�¬£�¤I��«ª­���¤§©ª� �uÛ~®Uáf��¤�Ô@£ØÜ�¤§�¡©à�¡­Äéîê^ë ¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �9��÷�u¯ ø*¯ª� ñI�u£Íéîê^ë ÔD£�Ü�¤¬�u©à�¡­ � ¤¬���u«à£�Û ¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �8Ó���u¯ ø*¯ � ñI�u£C�^Ü�¤¬ÛÄ���¡© � ¢*©ª£¥¦�£ ¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �8Ó���u¯ ø*¯ Ó �)ë¾Ü�®y� £¥¤e�S£¥¤¬®§©à������á�� �¡£T�^Ü�¤¬ÛÄ���¡© � ¢*©à£â¦�£.¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �8Ó��

©äú

Page 10: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

�u¯ ø*¯Ùá â ú�� £¥�¡®¬©ª���¡®ô¯l¯�¯Í¯�¯Í¯�¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �8Ó�áf Õ@ÉÏÆiÆÚ¹9Èg mÆiÆ�Ê¥¹9¸{ÂDÅ'ÑÚÂ�×ÌÒU¼tÑ¡ÇhÂ�¸gÂDÅ65~¸º¹9È � ÑÚ»{¸gÁ"¸fÆ�ÐsÊ��Æ�¸gÊ�¼ F�;4

÷u¯ä� ý�"� ¤¬��¨¡ç � � ©ª���+¯�¯�¯Í¯l¯�¯Í¯�¯Í¯�¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �8ÓXú÷u¯ä��¯ª� Ñ ��� ©ä¦�Üh� ©à���<áf��¤�¢*ÛÄ�*�4� �æ0����®§�¬©à�¡­¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �8ÓXú÷u¯ä��¯ � Ô@£�Ü�¤¬�¡©ª�¡­$Ô@©à�¡£�Ü�¤ìñI�¡¤§£�®¬�u��«à¨dë¾çu� � �¬©à���¡®ìòY©ä� � Ñ Ü�«ª© � ©à��ç¡®ìê^��©ª®¬£ß¯�¯ �8ÓG�

÷u¯ � ¢*ÛÄ�*���¬��æ0����®§�¬©à�¡­mòY©ä� �Ph`i�j�jlk&m�n*j�j�o�k ¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �8ÓG�÷u¯ �*¯ª� � ¤§£�«à©ªÛ~©ª�ÚÜ�¤§©à£�® ¯Í¯�¯Í¯�¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �8ÓX÷÷u¯ �*¯ � ñI�u£Vhli�j�jlk�m�p;j�j�olk ��«à­���¤¬©ä� �¡Û%¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �8ÓX÷÷u¯ �*¯ Ó � ¤§�*��ás��á Ò ��¤¬¤§£ � � �¡£¥®¬®��4áqhli�j�j`k&mFn*j�jFo�k ¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �@áu�÷u¯ �*¯Ùá Ò ��Û~ÞÚÜ4¤¬©à®§���.òY©ª�¬�ùû��¬�¡£�¤Yæ0����®y� ©à�u­Ä�^«à­���¤¬©ª�¬�¡ÛÄ®0¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �@á`ú

÷u¯ÙÓ Ô@£ØÜ�¤§�¡©à�¡­]Ô@©ª�¡£ØÜ�¤IñI�u¤¬£�®§�¡��«ª¨ ëÚç¡� � � ©à���¡®ìòY©ª�¬� Ñ Ü�«à© � ©à��ç¡®Iê^��©à®¬£ ¯Í¯�¯Í¯�¯�¯ �@á+�÷u¯ÙÓu¯ª� /î£���Û~£â� ¤¬© � � ¤¬£¥«à©àÛÄ©à�¡Ü�¤¬©ª£�®W¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �@á+�÷u¯ÙÓu¯ � � � Ò Ô@£ØÜ4¤¬�¡©ª�¡­ôòY©ª� � Ñ Ü4«à© � ©à��çu®Iê^��©ª®¬£�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �@á+�÷u¯ÙÓu¯ Ó �)ê���©à®§£lñs��«ª£�¤¬Ü��"�Ió'£�ÜX�Ô@£ØÜ�¤§�¡©à�u­Ä�^«à­���¤¬©ª�¬�¡Û ¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �@á`÷÷u¯ÙÓu¯Ùá � çu�§� ©ª�¡­õ©ª�I�^«à«iñ���­�£â� �¡£¥¤�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �Tø��

÷u¯ á é(©à® � ç¡®§®¬©à���%¯Í¯�¯�¯Í¯l¯�¯Í¯�¯Í¯�¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �Tø�Ó÷u¯ ᡯª� Ò ��Û~ÞÚÜ4¤¬©à®§���.òY©ª�¬�ùû(�¡«ª©à�¡£���«à­���¤¬©ä� �¡ÛÄ® ¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �Tø�Ó÷u¯ ᡯ � hli�j�jlk�mFn*j�j�olk©ª®�ûîÞ*� ©àÛ~Ü�«ª«ª�<¢*ÛÄ�*���¬� ¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �TøUá

�� ÒU¼tÑ¡ÇhÂ�¸gÂDÅ � Æ�Â�ÆÚ¹vÆ�ÂD¼r��� � ÐsÂ�×�¼tÇs��Ç9Æ@×�ÐsÁ�¹���¸{Ê¥¹vÇh¸('sÐ@¹9¸{Æ�Â�Ê F4t��v�u¯ä�u ý�"� ¤¬��¨¡ç � � ©ª���+¯�¯�¯Í¯l¯�¯Í¯�¯Í¯�¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �Tø`�

�v�u¯ä��¯ä��Ô@£�Ü�¤¬�¡©ª�¡­Äéîê�ë#©ª���Y£�®y� ¤§© � �¬£�¨ Ñ ��¨¡£�«ª® ¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �Tø`��v�u¯ä��¯ �ôÔ@£�Ü�¤¬�¡©ª�¡­$ÔDÜ�¤¬­�£¥¤ Ñ ���¡�4� ���¡£�éîê^ë}¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �8ú��

�v�u¯ � � ¤¬£�«ª©àÛÄ©à�ÚÜ4¤¬©à£¥® ¯�¯Í¯l¯�¯Í¯�¯Í¯�¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �8ú*��v�u¯ �*¯ä� é(©ª®§�¬¤¬©à�uçu� ©ª���*öy¢�ÞK£ � ©�� � � � Ò Ô@£ØÜ�¤§�¡©à�¡­ù¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �8ú���v�u¯ �*¯ �ßñI�u£Íé(©ª® � ¤¬£¥�¬£�ë¾��硤¬©ª£�¤ìñ�¤ Ü��u®§áf��¤§Û ¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �8ú��

�v�u¯ÙÓdÔ@£ØÜ�¤§�¡©à�¡­ôçu�¡¨¡£�¤j�^�u©ªáf��¤§Û%é(©à®y� ¤§©à�¡çu�¬©à���u®0¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �8úXÓ�v�u¯ÙÓu¯ä�= ý¨u£��"� ©äáº�*©ª�¡­Ä��£¥«à£â¦�Ü4�t�v�UÜ4¤¬©èÜ4�¡«à£¥®}¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �8úXÓ�v�u¯ÙÓu¯ �ßñI�u£sÔ@£�Ü�¤¬�¡©ª�¡­õ��«ª­���¤¬©ä� �¡Û ¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �8ú�á�v�u¯ÙÓu¯ÙÓ Ô@£�Ü�¤¬�¡©ª�¡­ Ñ ���¡�4� ���¡£Í� Ö�×xw y{z}|�Ø8Þ öw�¬£�¤¬Û éîê�ë ¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �8ú�á�v�u¯ÙÓu¯ á Ô@£�Ü�¤¬�¡©ª�¡­ Ñ ���¡�4� ���¡£ Ò ©ª¤ � çu©ª� ®�¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �8ú`ü

�v�u¯ á � ¤¬��¨¡ç � �^é(©ª®§� ¤§©à�¡ç*� ©à���¡®S¯Í¯�¯Í¯�¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �8úG��v�u¯ ᡯä�M¢���Û~£T~ æ0Ü�®¬©ª®^ëÚ��çu¤¬©à£¥¤xÔ@£�ÛÄÛ<Ü4®d¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �Tü4�

�v�u¯ ø Ô@£ØÜ�¤§�¡©à�¡­ôçu�¡¨¡£�¤ � ¤¬��¨¡ç � ��é(©ª®§�¬¤¬©à�uçu� ©ª���¡®î¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �Tü���v�u¯ ø*¯ä�= ý¨u£��"� ©äáº�*©ª�¡­Ä��£¥«à£â¦�Ü4�t�v�UÜ4¤¬©èÜ4�¡«à£¥®}¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �Tü���v�u¯ ø*¯ �ßñI�u£sÔ@£�Ü�¤¬�¡©ª�¡­õ��«ª­���¤¬©ä� �¡Û ¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �Tü���v�u¯ ø*¯ÙÓ Ô@£�Ü�¤¬�¡©ª�¡­ Ñ ���¡�4� ���¡£Í� Ö�×xw y{z}|�Ø8Þ öw�¬£�¤¬Û éîê�ë ¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �Tü�Ó�v�u¯ ø*¯ á Ô@£�Ü�¤¬�¡©ª�¡­ Ñ ���¡�4� ���¡£ Ò ©ª¤ � çu©ª� ®�¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �TüUá

;�� ÐD¹9ÐDÇ9¼r��¸{Ç9¼�Á�¹9¸fÆ�ÂsÊ *�*4

ú

Page 11: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

� Ì ²�³ ��� � Ì �E� ´�� ²

ᡯä� ��ç¡®¬£âáfç¡«D£âúuÜ�ÛÄÞ¡«à£��D�O���e�âí(�&� ×��MÞ ï(� ×��MÞ���� ¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ át�úu¯ä� ñI�¡£<H&J*K+LFM+NFK�Ü4«à­���¤§©ª�¬�¡Û�¯ ¯Í¯�¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ ü`üúu¯ � ���£âú*£ � çu� ©ª�����ás�¬�¡£<H&J*K+LFM+NFK.Ü�«à­���¤¬©ª�¬�¡Û¯�¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ üX÷úu¯ÙÓ ��ÞK£¥¤§áf��¤§Û<Ü4� � £ � ��Û~ÞÚÜ4¤¬©à®§������áIH&J*K+LFM+NFK.Ü��¡¨ � £�¤ � £¥Þu� ¤§���i¯ ¯�¯l¯Í¯�¯Í¯�¯�¯ �`úü*¯ä� � ò0��¤¬®§�yö � Ü�®¬£�¨ÚÜ4� Üõ®§£¥��áf��¤ì� �u£TH&J�K+LFM&NFK.Ü�«ª­���¤§©ª� �uÛ�¯W¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �+�ü*¯ � ñI�¡£l���¡«ª©à�¡£.U*öý�¡��¤§Û Ü�«ª­���¤¬©ä� �¡Û¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ ÷Xáü*¯ÙÓ ñI�¡£vUuöý�u��¤¬Ûùò0£�ÜX�«à£�Ü�¤¬�¡©ª�¡­õÜ�«ª­���¤¬©ä� �¡Û��F��H�¯Ú¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ ÷`úü*¯ á ñI�¡£<H���M+n*j�j�olk�Ü�«ª­���¤§©ª� �uÛ�¯�¯Í¯�¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ ÷`÷�u¯ä� ñI�¡£Q�!�FH�Ü4«à­���¤§©ª�¬�¡Û�¯Ä¯�¯Í¯�¯Í¯�¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ ���8ø÷u¯ä� ñI�¡£Chlitj�jlk&mFn�j�j�o�k Ü4«à­���¤§©ª�¬�¡Û�¯�¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �@á��÷u¯ � ��Þ¡«à�4�Y��ág�� òY©ä� ����W á � ¯Í¯�¯�¯Í¯�¯Í¯�¯l¯Í¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �@á`Ó÷u¯ÙÓ ñI�¡£vUuöý�u��¤¬Ûùò0£�ÜX�«à£�Ü�¤¬�¡©ª�¡­õÜ�«ª­���¤¬©ä� �¡Û��F��H�¯Ú¯�¯�¯Í¯�¯Í¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ �Tø4�

ú*©

Page 12: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

� Ì ²�³ ��� � µ ± ��� ²

�u¯ä� Ò ��ÛÄÞÚÜ�¤¬©ª®¬�����4áOt�¡�vòY�d�¡Ü�¤¬¨*ö � ��¤§£l®¬£¥� � ���¡®§�¬¤¬ç � � ©ª���¡®�¯ô¯�¯�¯Í¯�¯l¯Í¯�¯Í¯�¯�¯ ���v�

úu©ª©

Page 13: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

� Ï µÎÍ ³���´  

¡ � ³'´�� � � ¶O³ Ì � �

¢Q£!¢ ¤ ¥�¦q§©¨TªC¦«§¬¥�­

® ��¦���© � £ðö_Ü � � ©ä¦�Üh� £�¨�¨u��Û~£¥®§�¬© � ¤¬���K���YÞ�£�¤yáf��¤¬ÛÄ®YÜôòY©à¨¡£î¤¬Ü��¡­�£l��ás�¡��ç¡®§£��¡��«à¨ � �¡��¤¬£¥®�¯® �)¤¬£�Ü�«äö:� ©ªÛ~£�®§��®§�¬£�Û°¯Úç¡£¥�"� «ª�<� ¤ Ü4�¡®¬«àÜ4� £¥®���£¥�ýòQ£�£¥� â �¡­�«ª©à®§��Ü4�¡¨\ w� Ü�«à©àÜ��i¯® ��ÞK�vòQ£�¤yáfç¡« � ��ÛÄÞ¡çu� £¥¤Y©à�"� £¥¤¬Þ¡¤§£¥� ®^¤ Ü�¨u©à��­�¤¬Ü�Þ¡�¡© � ©àÛ~Ü�­�£¥®YÜ��¡¨dÛ~ÜX4£�®^Ü ��� 硤 Ü4�¬£ ¨¡©àÜ�­4ö�¡��®¬£�®¥¯

ó �¡©à«ª£îÜ�çu�¬��Û~Ü4� £¥¨�®§��®§�¬£�ÛÄ®�®§ç � ��Ü�®ì� �¡£¥®¬£(òQ��ç¡«ª¨��£î©à�u¨¡©à®§Þ¡çu�øÜ4�¡«ª�Äç¡®§£¥áfç¡«gûu� �¡£(� Ü�®¸~��áòY¤¬©ä� ©ª�¡­ £ðúuÞu«à© � ©ª� � ��Û~Þ¡ç*� £�¤QÞ¡¤¬��­�¤ Ü�ÛÄ®Q�¬� ®§��«ª¦�£(®¬ç � ���yÛ~£¥®¬®§�a�ô¤§£ØÜ�« öwòQ��¤¬«ª¨<Þ¡¤§���¡«à£¥Û~®U®¬£¥£�ÛÄ®� ���£~Þ¡¤§���¡©ª�¡©ª�¬©ª¦�£¥«ª��¨¡©ªã � ç¡«ª��¯<ëÚ��¤(�¬�¡£�®§£.Ü��¡¨cÛ~Ü��"����� �¡£¥¤lç¡®¬£âáfç¡«Q¤§£ØÜ�« öwòQ��¤¬«ª¨ �øÜ�®t®�û6Ü4�t�®¬ç ��� £¥®¬®yáfç¡«�®y�*®y� £¥Û Ûmç¡®§����£�Ü��¡«à£î�¬�Ä�¡Ü��¡¨¡«ª£ÍÜm¦4Ü�®y�^�tç¡Û �K£¥¤��4á6Þ���� £¥�t�¬©èÜ�«J©à�¡Þ¡ç*� ®Íí â �¡­�«ª©à®§�®¬£¥�"� £�� � £�®¥û ¤¬Ü�¨¡©à��­�¤ Ü�Þu�¡© � ©ªÛ<Ü4­�£�®¥û�£¥� � ¯ ïâÿô£â¦�£��-©ªá�� �¡£c®§��®§�¬£�ÛA± ®�¨¡£�®§©à­��u£�¤�ÚÜ�®£âú�� £¥�¡®¬©ä¦�£t�¡�vòY«à£¥¨¡­�£���á�� �¡£�®§ÞK£ � ©�� � Þu¤¬���¡«ª£�Û�¨¡��Û~Ü�©à�G�4á^©ª�"� £�¤§£�®y�ØûU©ª�ôÛ~ÜØ�#��£��¡£ØÜ�¤§«ª�'©àÛÄÞK��®§®¬©ª�¡«à£� ��£ðúuÞu«à© � ©ª�¬«ª� � ���¡®y� ¤§ç � �dÜ�Þu¤¬��­�¤¬Ü�Û òY�u© � ��£âú*�¡©à�u©ª� ®��¬�¡£�¨¡£�®§©à¤¬£¥¨-��£��ÚÜئ�©à��¤�¯ßñI�¡£��Ú£�«ª¨��á]ì$èaõPÿaö«ç�ï:éwï¸èXæ-ç1ö«ç�� Ü4�¬�¬£�ÛÄÞu� ®Ä� � � ©à¤ � ç¡Û ¦�£��"�.� �u©à®.¨¡©äã � ç¡«ä�ý�}�"���¡ç¡©à«ª¨¡©à�u­#Ü�çu�¬��Û~Ü4� £¥¨®§��®y� £�ÛÄ®lòY�¡© � � � Ü4�Ï«ª£ØÜ�¤§� áf¤¬��Û ¨ÚÜ4�øÜ*ûD�¬�tç¡®l���"¦*©àÜ4� ©ª�¡­.� �u£ô�¡£�£¥¨Ï� ��Þu¤¬��­�¤¬Ü�ÛWÜ4�c£ðú*Þ¡«à© � ©ª�Ü�«ª­���¤¬©ä� �¡Û¯õæ0çu��òY�ÚÜ4�ÍÞ¡¤§©à� � ©àÞ¡«ª£�®l®¬�¡��ç¡«à¨c­�ç¡©ª¨¡£ô� �¡£Ä¨¡£�®§©à­��Ï��á쫪£ØÜ�¤§�¡©à�u­4öý�ÚÜ4®¬£�¨�®y�*®y� £¥Û~®�²���u¨òY�ÚÜh��Ü4¤¬£(� �u£�©ª�¡�¡£¥¤¬£��"� � Ü�Þ¡Ü��¡©à«ª©ª�¬©à£�®0Ü��¡¨�«à©ªÛ~©ä�øÜ4�¬©à���u®Q��á � ��Û~Þ¡ç*� £�¤§®I� �ÚÜ4�Y«ª£ØÜ�¤§�;²��£¥®¬£�Ü�¤ � ��©à��� �¡£î�¬�¡£���¤y����ásÛ<Ü � �¡©à�¡£î«ª£ØÜ�¤§�¡©à�u­õÜ4�¬�¬£�ÛÄÞu� ®ì�¬�ÄÞu¤¬�v¦�©à¨¡£�Üô¤¬©ª­���¤¬��ç¡®ìÛ~Ü4� �*ö

£�Û~Ü4�¬© � Ü4«6áf��ç¡�¡¨ÚÜh� ©à���cáf��¤ÍÜ��¡®yò0£¥¤¬©ª�¡­ ®¬ç � �Gü�çu£�®§�¬©à���u®�¯ñI�¡£.�¡¤§�"Ü�¨#Ü4©àÛå��á0Û~��®y�ÍòQ��¤Ï©ª�� �u£���¤§£¥� © � Ü�«>Û~Ü � �¡©ª�¡£ô«à£�Ü�¤¬�u©à�¡­�©ª®(� �¨¡£��Ú�u£<Ü��u¨ÏÜ4�ÚÜ�«ª��÷¥£Ä¨u©��K£�¤¬£¥�"�Í«ª£ØÜ�¤§�¡©à�u­.áf¤ Ü�ÛÄ£¥òQ��¤¸t®¥û«à£�Ü�¤¬�u©à�¡­ Þu¤¬���¡«ª£�ÛÄ®�ûUÜ��¡¨�«ª£ØÜ�¤§�¡©à�¡­ Ü�«à­���¤¬©ª�¬�¡ÛÄ®�¯�ñs�vò0Ü�¤¬¨u®Ä�¬�¡©à®m£¥�¡¨�Ü ¤ Ü��¡­�£���á�¨u©��K£�¤¬£¥�"�

Page 14: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

Ü�Þ¡Þu¤¬�"Ü � �u£�®I�ÚÜئ�£l��£�£¥��� ÜX�£¥���t�<¤¬£¥®¬£ØÜ4¤ � �¡£¥¤¬®�©à�5�Ú£¥«à¨¡®I®§ç � ��Ü�®ìÞÚÜ4�§� £¥¤¬��¤¬£ � ��­��u©ª� ©ª���iû�£�®y� ©äöÛ~Ü4� ©ª����¬�¡£���¤§��ûKÜ��¡¨�� �¡£¥��¤¬£â� © � Ü�« � ��ÛÄÞ¡çu�¬£�¤^® � ©ª£�� � £�¯� ý�d� �u©à®Y� �¡£¥®¬©à®�ò0£ � ���¡®§©à¨¡£¥¤�«ª£ØÜ�¤§�¡©à�¡­áf¤¬��Ûþ� �¡£�ÞK£¥¤¬®§ÞK£ � � ©ä¦�£(�4á[õPäUì�í�å�î£èXîcö£äUç�èXé1éwï¸èXæMç1ö«ç��Qî«ÿ1ïPäUæ-ò�³¾�¬�¡£��¡¤¬Ü�� � �.�4áJ� �u£���¤§£¥� © � Ü�« � ��ÛõöÞ¡çu�¬£�¤î® � ©à£¥� � £mòY�u© � ��®§�¬ç¡¨¡©à£¥®(� �¡£õÜ4�¡©à«ª©ª� ©ª£�®�Ü��¡¨!«ª©àÛÄ©ª� Ü4� ©ª���¡®I��á � ��Û~Þ¡ç*�øÜ4�¬©à���ÚÜ4«à«ª�.£¥ã � ©ª£��"�«à£�Ü�¤¬�u©à�¡­õÜ�«ª­���¤§©ª� �uÛ~®¥¯¢*£¥¦�£�¤¬Ü�« � ��Û~Þ�£�«ª«à©ª�¡­ÄÜ�¤¬­�çuÛ~£¥�t�¬®^®§ç¡Þ¡Þ���¤§�^� �¡£Í®§�¬¤¬���¡­~£�ÛÄÞ¡�ÚÜ�®§©à®�òY�¡© � � � ��ÛÄÞ¡çu�øÜh� ©à���ÚÜ�«

«à£�Ü�¤¬�u©à�¡­�� �¡£¥��¤§� Þ¡«èÜ � £�®6���õÜ�«à­���¤¬©ª�¬�¡ÛÄ© � £¥ã � ©à£�� � �ô©ª®¬®¬çu£�®�¯>¢*©à� � £ � ��Û~Þ¡ç*�øÜ4�¬©à���ÚÜ4«à«ª�(£¥ã � ©ª£��"�«à£�Ü�¤¬�u©à�¡­õÜ�«ª­���¤§©ª� �uÛ~®QÜ�¤¬£î©ª�¡¨¡£¥£�¨�� �¡£î���u«ª��Ü4«à­���¤§©ª�¬�¡Û~®SòY�¡© � � � Ü����K£lç¡®§£�¨©à��Þ¡¤¬Ü � �¬© � £(áf��¤«èÜ4¤¬­�£Í¨ÚÜ4�øÜ�®§£¥� ®¥ûD©ª�(©ª®(��£¥�����¡¨Ï¨u©à®¬Þuçu� £m�¬�ÚÜ4�lÜ�«ª­���¤§©ª� �uÛ~© � £âã � ©à£¥� � � ©à®îÜ:óMö«ç�ïQ´�å{è:ç�äUç�©ª�¤¬£�Ü�«äö:ò0��¤¬«à¨Ä«èÜ4¤¬­�£ðöý® � Ü�«à£I«ª£ØÜ�¤§�¡©à�u­Í®§©ª� ç¡Ü4� ©ª���¡®�¯I ý��Ü�¨¡¨u©ª� ©ª���ô� ��� �¡©ª®QÞ¡¤¬Ü�­�Û~Ü4� © � Û~�4� ©ª¦4Ü4�¬©à���Jû"Ü®§�¬¤¬���u­ � Ü4®¬£ � Ü����K£�Û<Ü4¨¡£î� �ÚÜ4� � ��Û~Þ¡ç*�øÜ4�¬©à���ÚÜ4«i£¥ã � ©ª£�� � �®¬�¡��çu«à¨d��£l�øÜX4£��d©ª�"� �~Ü ��� ��ç¡�"�©à�<��¤¬¨¡£¥¤ì� �ôÜ � �¡©à£â¦�£�Ü���Ü�¨¡£¥ütçÚÜ4� £¥«ª�~¤¬© � ��Ü��¡¨��tçÚÜ�� � £�¨�Û<Üh� �¡£¥Û<Ü4�¬© � Ü�«Ú� �¡£¥��¤§�<��áD«à£�Ü�¤¬�¡©ª�¡­¡¯ w�m©ª®�òQ£�«ª«Lt�¡�vòY�'�¬�ÚÜ4� ©à�'Û~Ü��"��«à£�Ü�¤¬�u©à�¡­Û~��¨¡£¥«à®�û�©ªá0� �¡£ � ��ÛÄÞ¡çu� Ü4� ©ª���ÚÜ�« � ��Û~Þu«à£âú*©ä�ý�!��á«à£�Ü�¤¬�u©à�¡­^©à®�©ª­��¡��¤§£�¨ �¬�¡£��õ«ª£ØÜ�¤§�ÚÜ��¡©ª«à©ª�ý� � Ü��ô��£ � ��Û~Þ¡«ª£¥�¬£�«ª� � �¡Ü�¤ Ü � � £¥¤¬©à÷¥£�¨õ�"� ®§©àÛÄÞ¡«à£ � ��Ûm�¡©äö�ÚÜ4�¬��¤¬©àÜ�«JÞ¡Ü�¤ Ü�ÛÄ£¥�¬£�¤§®I®¬ç � �dÜ�®I� �¡£µ�UÜ�Þu�¡©w�ö Ò �u£�¤§¦����¡£¥�{t©à®�¨¡©àÛÄ£��¡®§©à���Jÿu®¬ç � � Ü � �"Ü4¤¬®¬£ � Ü4� £ðö­���¤§©à÷�Ü4� ©ª���ô®¬�¡£¥¨¡®0«ª©ª�§� «à£0«à©à­��t�6���Ä�¬�¡£^Ü � �¬çÚÜ�«Ú®¬©ªÛ~©ª«èÜ�¤§©ª�¬©à£�®sÜ��¡¨~¨¡©��K£�¤§£�� � £�®U�K£â�ýò0£¥£���«ª£ØÜ�¤§�¡©à�¡­Þ¡¤§���¡«à£¥Û~®¥¯ Ñ ��¤§£��v¦�£¥¤�û>áf¤¬��Û��¬�¡©à®ÍÞK£¥¤¬®¬Þ�£ � � ©ä¦�£���áY©ª�uáf��¤¬Û~Ü4�¬©à���*ö:� �¡£¥��¤¬£â� © � ígÜ�®m��Þ¡ÞK��®¬£�¨#�¬�� ��Û~Þ¡ç*�øÜ4�¬©à���ÚÜ4«ºïI«à£ØÜ4¤¬�ÚÜ��u©à«à©ä�ý��ûKÛ<Ü��"��«à£�Ü�¤¬�¡©ª�¡­�Þ¡¤§���¡«à£¥Û~® � Ü�����£��§®¬��«ä¦�£¥¨{���t� £âú*�ÚÜ�ç¡®y� ©ä¦�£®¬£�Ü�¤ � �J¯ùû�á � ��硤¬®§£<®§ç � �G®¬��«ªçu� ©ª���¡®�­�©ª¦�£�®�«à©ä�¬� «ª£ô©ªáìÜ��"�Ï©ª�¡®¬©ª­��"� ©ª�"� ��� �¡£~®§� ¤§ç � �¬ç¡¤¬£<��áì� �u£«à£�Ü�¤¬�u©à�¡­�Þ¡¤¬���u«à£�ÛûSÜ��¡¨�Ü4¤¬£ � ��Û~Þu«à£¥�¬£�«ä�Ï©ªÛ~Þu¤ Ü � � © � Ü�«>� � �������د�û(�#� �¡£<��� �¡£¥¤m�ÚÜ��u¨iûS�t��øÜ��©ª�¡­(� �u£ � ��ÛÄÞ¡çu�øÜh� ©à���ÚÜ�« � ��ÛÄÞ¡«à£ðúu©ä�ý� �4áJ«ª£ØÜ�¤§�¡©à�¡­î©ª�"� �lÜ �¥� ��ç¡�"�QÜî�¡¤¬��Ü�¨ÄÜ��¡¨õá{Ü�® � ©à�ÚÜ4�¬©à�¡­®¬Þ�£ � � ¤§ç¡Û �4áS¨¡©��K£�¤§£��"�(��£��ÚÜئ�©à��¤¬®(Ü4¤¬£láf��ç¡�¡¨d�¬�.£âú*©ª®§�îÜ4Û~���u­~«ª£ØÜ�¤§�¡©à�¡­~Þ¡¤¬���¡«à£¥Û~®¥¯� ý��� �¡©ª®� �u£�®¬©ª®�û¡áf��¤I£ðúuÜ�Û~Þu«à£�û¡òQ£ � ���u®¬©à¨u£�¤Y«à£�Ü�¤¬�¡©ª�¡­ôÞ¡¤§���¡«à£¥Û~®ìòY�u© � ��Ü�¤¬£lÞ¡¤§�9¦4Ü��u«ª��£�Ü�®§�cíf®¬��«ä¦4Ü��¡«à£�"��ÞK��«ä���¡��ÛÄ©èÜ�« öw� ©ªÛ~£sÜ�«ª­���¤¬©ä� �¡ÛÄ®øïðÿ9«à£ØÜ4¤¬�¡©ª�¡­^Þu¤¬���¡«ª£�ÛÄ®@òY�¡© � �ôÜ4¤¬£ � £�¤y� © �¾Ü��¡«ä�î�ÚÜ�¤§¨�ígÜ4��«ª£ØÜ�®y�Ü�®l¨¡©äã � ç¡«ª�ÍÜ�® � ¤§��Þu� ��­�¤ Ü�Þu�¡© � Þ¡¤§���¡«à£¥Û~®îòY©ª¨¡£�«ä�!�K£¥«à©ª£¥¦�£¥¨c�¬��¤§£�ütç¡©ª¤¬£Ä£âú*ÞK���¡£��"� ©àÜ�«6� ©ªÛ~£ØïâÿÜ��¡¨«à£�Ü�¤¬�u©à�¡­ôÞ¡¤§���¡«ª£�ÛÄ®0òY�¡��®§£ � ��Û~Þu«à£âú*©ä�ý�.©à®YÜ4®Y��£¥���øÜ��"� Ü�«à©ª÷�©à�u­�«ª�õç¡�u¤¬£�®§��«ª¦�£�¨i¯ ý� � ��Û~Þ¡ç*�øÜ4�¬©à���ÚÜ4«Ú«à£ØÜ4¤¬�¡©ª�¡­��¬�¡£���¤§�Ä� �¡£^����ݧ£ � �ì��£�©ª�¡­ «ª£ØÜ�¤§�¡£�¨.©ª®U�ý��Þ¡© � Ü�«ª«ª�ôÛÄ��¨¡£�«ª£�¨.Ü4®

Üæ0�*��«à£ØÜ4�cáfç¡� � � ©ª���c�9¦�£�¤�®¬��ÛģĨ¡��Û~Ü�©à�iûDÜ4�¡¨Ï�¬�¡£Ä� Ü�®¸!�4á0� �u£~«ª£ØÜ�¤§�¡©à�¡­Ü�«ª­���¤§©ª� �uÛ ©ª®l�¬�©à�*áf£�¤î� �¡©ª®lç¡�{t�¡�vòY�#�øÜ�¤§­�£¥�láfç¡� � � ©ª���c���Ï�¬�¡£~�¡Ü�®¬©ª®Í��á0«èÜ���£�«ª£�¨c£âúuÜ�ÛÄÞ¡«à£¥®�¯.ñI�u£~©ª�t�¬ç¡©ª�¬©à���©à®Í� �ÚÜ4� � �¡£�ç¡�v��u�9òY��áfç¡� � � ©à���G¤¬£�Þu¤¬£�®§£��"� ®~Ü õPäUç�õPï"í�îQòY�u© � �G� �¡£�«ª£ØÜ�¤§�¡©à�u­!Ü�«à­���¤¬©ä� �¡Û婪®� ¤y��©à�¡­ �¬�mÛ~Ü�®§�¬£�¤�ÿ*£�¯ ­¡¯>©à�.Üm®¬©ª�¬çÚÜ4�¬©à���ÄòY�¡£�¤§£lÜ ¤¬�������0©à®0«ª£ØÜ�¤§�¡©à�u­Í� � ¨¡�ô©à�"¦�£¥�"� ��¤y� � ���t�¬¤¬��«©à��Ü�«ª©à�¡¤¬Ü�¤§��ûJ� �¡£ � ��� � £�Þu�l� ����£ô«à£�Ü�¤¬�¡£¥¨�Û~©ª­��"�î�K£��y�K���`�òY¤§©ª�§� £���©ª� â �u­�«à©ª®¬�i¯ �¶ ý�!� �¡©ª®®¬£â�¬�¬©à�¡­l� �¡£Yç¡�v��u�9òY�.æ0�*��«à£ØÜ4�máfç¡� � �¬©à���õòQ��ç¡«à¨Ä«èÜ4�K£¥«¡£ØÜ � �~����ݧ£ � �U©à�ô� �u£^¤§���K�4�^± ®Uçu�¡©ª¦�£�¤¬®§£

Page 15: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

Ü�®U£�©ä� �¡£¥¤0Ü�ÞK��®§©ª�¬©ª¦�£Y��¤U�¡£¥­"Ü4� ©ä¦�£�©à�¡®y�øÜ�� � £���áDÜ�� â �u­�«à©ª®¬�*öw«èÜ��¡­�çÚÜ�­�£ì�����`¾ûuÜ��u¨~� �u£^¤§���K�4�^± ®�øÜ4®¸mòQ��ç¡«à¨~�K£Y� ��©à�*áf£�¤UÜ��.Ü ��� çu¤ Ü4�¬£ � «èÜ�®§®¬©�� � Ü4�¬©à���õ¤¬çu«à£�Ü4áº� £¥¤Q®§£�£¥©à�¡­�®¬��ÛÄ£�® Ü�ÛÄÞ¡«à£Y���tݧ£ � �¬®Ü��¡¨�� �¡£¥©à¤ � ��¤¬¤§£�®§ÞK���u¨¡©à�¡­<�¡©à�ÚÜ4¤§��«èÜ4�K£¥«à®�¯�ñI�¡£m®y�øÜ��u¨ÚÜ�¤¬¨dÞÚÜ�¤ Ü4¨¡©à­�Û ©ª�d«ª£ØÜ�¤§�¡©à�¡­Ä� �¡£¥��¤§�©ª®� �¡Ü4�î� �¡£õç¡�v��u�9òY� �y�øÜ4¤¬­�£â� � ��� � £�Þu�¸�d©ª®$è]í�æ-ö£äUæMö�t�¡�vòY�Ï�¬���£�«ª���¡­�� ��®§��Û~£Q�*úu£¥¨ � «èÜ�®§®��á>Þ���®§®¬©à�u«à£îáfç¡� � � ©à���¡®�û¾ç¡®§çÚÜ�«à«ä��¤¬£âáf£�¤¬¤§£�¨� �.Ü4®��¬�¡£&õPäUç�õPï"í�îxõMé èUóPóIòY�¡© � �d©à®���£�©à�u­~«ª£ØÜ�¤§�¡£�¨i¯ ý� ��çu¤^«ª©à�¡¤¬Ü�¤§��Ü4Þ¡Þ¡«à© � Ü4�¬©à�����¬�¡£ � ��� � £¥Þu� � «èÜ�®§®^ÛÄ©à­��"� � ���¡®¬©ª®§�(�4á � ��� � £�Þ*� ®(®§ç � ��Ü�®��y�K���`òY¤¬©ä�¬�¬£��Ä©à�Ä«èÜ��¡­�çÚÜ�­�£v·mû ���§�K���`mÞuç¡�¡«à©ª®¬�¡£¥¨~©ª� � ��ç¡�"� ¤y��¸ôû �ù�§�����`ôÜ�����çu�S� ��Þ¡© �0¹ �ÍÜ��¡¨<®§����i¯� ý� � ��ÛÄÞ¡çu�øÜh� ©à���ÚÜ�«�«ª£ØÜ�¤§�¡©à�u­��¬�¡£���¤§��¤¬£¥®¬£�Ü�¤ � �'�¬�¡£õÛ~��®§�láf¤¬£¥ü�çu£��"� «ä��®§� çu¨¡©à£¥¨ � ��� � £¥Þu�� «àÜ�®¬®§£�®ÍÜ�¤¬£Ä®§���"�øÜ � � © �~� «èÜ�®§®¬£¥®Í��áìæ0����«à£�Ü��!áf��¤¬Ûmçu«èÜ�£4ÿ��ýòQ�d®§ç � � � «àÜ�®¬®§£�®�òY�¡© � �c�ÚÜئ�£~�K£¥£��©à�"�¬£��¡®§©ª¦�£¥«ª�'®§� çu¨¡©à£¥¨-í{Ü��¡¨�Ü�¤§£�Ü�ÛÄ���¡­ � �u£�Û<Ü�©ª�'� ��Þ¡© � ®m��áY� �¡©ª®m� �¡£¥®¬©ª®øïmÜ�¤§£.� �¡£ � «èÜ�®§®m��áé(©à®ºÝ§ç¡� � � ©ª¦�£lê^��¤§Û<Ü4«@ëÚ��¤¬Ûùáf��¤¬Ûmçu«èÜ�£îÜ4�¡¨�¬�¡£ � «àÜ�®¬®I��ás«à©ª�¡£ØÜ4¤0� �¡¤§£�®§�¡��«à¨�áfç¡� � � ©ª���¡®�¯ w�Y©à®ì©ª�"� ç¡©ä� ©ª¦�£�«ä� � «à£�Ü�¤0� �¡Ü4�I®¬©ªÛ~Þu«à£ � ��� � £�Þu�¬®�®§�¡��ç¡«ª¨��K£î£�Ü�®¬©ª£�¤0�¬�ô«à£�Ü�¤¬�.�¬�ÚÜ�� � ��Û~Þ¡«ª£âú

���¡£¥®�ÿ�áf��¤�©ª�¡®§� Ü�� � £�û4�¬�¡£ � «èÜ4®¬®D��á¡®§©àÛÄÞ¡«à£>«ª©à�¡¤¬Ü�¤§� � ��� � £�Þ*� ®�¨u£�® � ¤¬©ª�K£¥¨mÜ����v¦�£U®¬�¡��çu«à¨��K£U£ØÜ�®§©à£¥¤� ��«à£�Ü�¤¬�õ� �¡Ü��<Ü � «àÜ�®¬®SòY�¡© � �~©ª� � «ªç¡¨¡£�® � ��� � £¥Þu� ®Q®¬ç � �<Ü�®Q�y­�¤¬©ªÞ¡Þ¡©à�u­î®¬ç¡®§ÞK£¥�¡®¬£��¬�¡¤¬©ª«à«ª£�¤¸�lÜ4�¡¨�§Þ���®§�¬®§�¬¤¬ç � � ç¡¤¬Ü�«à©ª®§� � ¤§©ª�¬©àütç¡£I��áK¨¡©èÜ4«à£ � � © � Ü�«*Û<Ü4�¬£�¤§©èÜ�«ª©à®¬Û¯ �� �¡¤§�"Ü�¨õ­��"Ü4«¡��á � ��ÛÄÞ¡çu�øÜh� ©à���ÚÜ�««à£�Ü�¤¬�u©à�¡­'�¬�¡£���¤§�O¤§£�®§£ØÜ�¤ � �iûîÜ��¡¨ ��ál�¬�¡©à®.�¬�¡£�®§©à®�û�©à®.�¬�G¨¡©à® � �v¦�£¥¤��¡�vò �¬�¡£ � ��ÛÄÞ¡«ª£âú*©ª�ý����á«à£�Ü�¤¬�u©à�¡­ô® � Ü�«ª£�®IòY©ª�¬���¬�¡£ � ��ÛÄÞ¡«ª£âú*©ª�ý�<��ás�¬�¡£ � ��� � £�Þ*� ®YòY�¡© � ��Ü�¤¬£l��£�©ª�¡­õ«à£ØÜ4¤¬�¡£¥¨i¯

¢Q£»º ¼�½r¾ ¾ ªV¿«À ¥QÁÃÂ�ÄÆÅÈÇ«§¬ÇÊÉ˥̭s¦«¿Í§¬Îr½P¦q§¬¥�­rÇ

ó �¡©ª«à£ô� �¡£Ä¤¬£¥®¬£�Ü�¤ � �G©ª�c�¬�¡©à®l� �¡£¥®¬©ª®Í®§ÞÚÜ��¡®Í®¬£¥¦�£�¤¬Ü�«0¨¡© �J£¥¤¬£¥�t��� ��Þ¡© � ®�ûs�¬�¡£ � ��ÛÄÛ~���c�¬�¡¤¬£�Ü�¨òY�¡© � �cçu�¡©ª�¬£�®l��硤(¤§£�®§ç¡«ª�¬®�©ª®îÜ � £¥�t�¬¤ Ü�«>£¥Û~Þu�ÚÜ�®¬©ª®(��� õPäUì�í�åaî=èXîcö£äUç�èXé«éWòùïDÏ õMö£ïMç1îxéwï¸èXæ-ç1ö«ç���¯óÏ£ � ���u®¬©à¨u£�¤(� �u£ áf��«à«ª�vòY©à�¡­Ä¤¬£¥®¬ç¡«ä� ®�ûJ¨¡£�® � ¤¬©ª�K£¥¨�©à� ÛÄ��¤¬£Í¨¡£¥� Ü�©à«s©à��¢�£ � �¬©à���'��¯ Ó*ûJ� �.��£m� �u£Þ¡¤§©àÛ~Ü�¤§� � ���"�¬¤¬©à�uçu� ©ª���¡®I��á�� �¡£î�¬�¡£�®§©à®¬"

® óÏ£(­�©ä¦�£��¬�¡£(á{Ü�®§�¬£�®y�xt�¡�vòY�dÜ�«ª­���¤§©ª� �uÛMáf��¤ì«ª£ØÜ�¤§�¡©à�¡­ Ü���Ü�¤§�¡©ª�¬¤ Ü�¤y�<é(©ª®fݧç¡� � � ©ä¦�£îê���¤yöÛ~Ü�«¡ëÚ��¤§Û í{éîê�ë>ï�áf��¤¬Ûmç¡«àÜ�©à�ô�¬�¡£ � ¤¬���ÚÜ4�¡«ª� ��Þ¡Þ¡¤§�Øúu©ªÛ<Üh� £�«ä� Ò ��¤§¤¬£ � �^í � � Ò ïSÛÄ��¨¡£�«��á0«à£�Ü�¤¬�¡©ª�¡­�òY©ª� �Ϥ¬£¥®¬Þ�£ � �Í� �dÜ��#Ü�¤§�¡©ª�¬¤ Ü�¤y�!Þ¡¤§���ÚÜ��¡©ª«à©ä�ý� ¨¡©ª®§� ¤§©à�¡ç*� ©à���c���'£âúuÜ�ÛÄÞ¡«ª£�®�¯ñI�¡£lÞ¡¤§���¡«ª£�Ûù��á � � Ò «ª£ØÜ�¤§�¡©à�¡­méîê�ë#©à®ìòY©ª¨¡£�«ä�~¦�©ª£¥òQ£�¨�Ü�®I���¡£î��á�� �u£îÛ~��®y�I©àÛÄÞ���¤yö� Ü��"�^Þu¤¬���¡«ª£�ÛÄ®ì©à� � ��Û~Þuçu�øÜ4�¬©à���¡Ü�«J«ª£ØÜ�¤§�¡©à�¡­m�¬�¡£���¤§��¯

® óÏ£l¨¡£�ÛÄ���¡®y� ¤ Üh� £î� �ÚÜh�Y£¥¦�£¥�dáf��¤ì¦�£�¤§��®§©àÛÄÞ¡«à£î«ª£ØÜ�¤§�¡©à�u­ôÞ¡¤¬���u«à£�ÛÄ®�û¡Üõ®y� ¤§���¡­õ� ¤ Ü4¨¡£��X�� Ü���£âú*©à®y�õ�K£â�ýò0£¥£����¬�¡£�¤¬çu�¡�¡©à�u­��¬©àÛÄ£.Ü��u¨��¬�¡£��tç¡Ûm��£�¤ô��á�£âúuÜ�ÛÄÞ¡«ª£�®ô¤¬£¥üt硩ध£�¨���áÜ��"�õ®§ç ��� £�®¬®yáfç¡«J«ª£ØÜ�¤§�¡©à�u­ÍÜ4«à­���¤§©ª�¬�¡Û�¯D�^®0Ü��~Ü�®§ÞK£ � �Q��áJ�¬�¡©à®>� ¤¬Ü�¨¡£¥�X��ò0£�­�©ª¦�£��¬�¡£x�Ú¤¬®y�

Ó

Page 16: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

£ðú¡Ü4Û~Þ¡«ª£~�4áYÜd®¬©ªÛ~Þu«à£õ«à£�Ü�¤¬�¡©ª�¡­Þ¡¤¬���u«à£�ÛWáf��¤�òY�¡© � � èXîcî¶æ-ö(Ð�åaî7ï�ѸïDÏdõMö£ïMç1îU«à£ØÜ4¤¬�¡©ª�¡­#í{Ü�ý��Þ�£��4á�«ª£ØÜ�¤§�¡©à�¡­máf¤§��Û áf£âò £âúuÜ�ÛÄÞ¡«ª£�®øï0©ª® � ��ÛÄÞ¡çu� Ü4� ©ª���ÚÜ�«ª«ª�Ä�ÚÜ�¤§¨i¯

® óÏ£ � ���¡®y� ¤¬ç � � Ü�¡£âòþá{Ü�ÛÄ©à«ä�d��á0£¥ã � ©ª£��"�Í�¡��©à®§£âö:� ��«ª£�¤ Ü4�t�l«ª©à�¡£�Ü�¤��¬�¡¤¬£¥®¬�¡��«ª¨'«ª£ØÜ�¤§�¡©à�¡­Ü�«ª­���¤§©ª� �uÛ~®¥¯>ñI�¡£�®§£ôÜ�«à­���¤¬©ä� �¡ÛÄ®�û�òY�¡© � �dÜ�¤¬£l�ÚÜ�®§£�¨ �����"��ÞK�4� �¡£¥®¬©à®^�K����®y� ©à�u­¡û � «ª��®¬£¥«ª�Û~Ü4� � � �¬�¡£ôÞ�£�¤§áf��¤¬Û~Ü�� � £ ��áS� �¡£ � £�¤ � £�Þu�¬¤¬���ÏÜ��¡¨ ó ©ª�¡�¡�vòMÜ�«à­���¤¬©ª�¬�¡ÛÄ®�û¾�ýò0����áS� �u£��£�®y�§ö"t�¡�vòY�dÜ�«à­���¤¬©ª�¬�¡ÛÄ®0©à��Û~Ü � �¡©ª�¡£î«à£�Ü�¤¬�¡©ª�¡­¡¯

® óÏ£ì£�®y�øÜ��u«à©à®§�ôÜ��m£¥ütç¡©ª¦4Ü�«ª£�� � £ì�K£â�ýò0£¥£��õ���*��®y� ©ª�¡­(Ü�«ª­���¤§©ª� �uÛ~®D©à� «à£ØÜ4¤¬�¡©ª�¡­^�¬�¡£���¤y�mÜ4�¡¨�ÚÜ4¤¬¨*ö � ��¤§£~®§£¥� � ���¡®y� ¤¬ç � � ©ª���¡®Í©à� � ��ÛÄÞ¡«à£ðúu©ä�ý�!� �¡£¥��¤§��¯�ñI�¡©à®�£�ütç¡©ä¦�Ü4«à£�� � £~��©ª£�«à¨u®Í©ªÛõöÞ¡¤§�v¦�£�¨ � ��Û~Þ¡«ª£âú*©ª�ý�Í� �¡£¥��¤¬£â� © � �ÚÜ�¤¬¨�ö � ��¤¬£I®¬£â� � ���u®§� ¤§ç � �¬©à���u®QÜ4®6ò0£¥«à«¾Ü�®>ÛÄ��¤¬£0£¥ã � ©ª£��"�Ü�«ª­���¤§©ª� �uÛ~®Q©à��«à£�Ü�¤¬�¡©ª�¡­ �¬�¡£���¤y��¯

® óÏ£d­�©ä¦�£dÜÏ�¡£âò ������®§�¬©à�¡­ÏÜ�«à­���¤¬©ä� �¡Û òY�u© � �O­�£¥�¡£�¤¬Ü4� £¥®.��Þu�¬©àÛ~Ü�«à«ä�'®§Û~�����¬�}¨¡©ª®§� ¤§©äö�¡ç*� ©à���¡®lÜ��¡¨�®¬�u�9ò � �ÚÜh�î� �¡©ª®î�K����®§�¬©à�¡­.Ü�«à­���¤¬©ª�¬�¡Û � Ü��c��£ôç¡®¬£¥¨c�¬� � ���¡®y� ¤¬ç � � � � Ò«ª£ØÜ�¤§�¡©à�¡­õÜ4«à­���¤§©ª�¬�¡Û~®UòY�¡© � ��� ��«à£�¤¬Ü4� £��¡©à­���«ª£¥¦�£¥«à®Y��ás�¡��©ª®§��¨ÚÜh�øÜu¯

ñI�¡£láf��«à«ª�vòY©à�¡­Í¤¬£�®§ç¡«ª�¬®�Ü�¤¬£l®¬£ � ���u¨ÚÜ�¤§� � ���"� ¤§©à�¡çu�¬©à���u®I��ás�¬�¡£î� �¡£¥®¬©ª®¬"® óÏ£î­�©ª¦�£lÜ���£¥ã � ©ª£��"�^Ü�«à­���¤¬©ä� �¡ÛßòY�¡© � �«ª£ØÜ�¤§�¡®�� Ö�× w y{z}|�Ø8Þ öw� £¥¤¬Û&Û~���u��� ���u£îéîê�ë#ç¡��ö¨¡£¥¤Q�¬�¡£(ç¡�¡©äáf��¤¬Û ¨¡©ª®§�¬¤¬©à�uçu� ©ª���i¯�ñI�¡©à®ì©ª®U�¬�¡£��Ú¤§®§�IÞ���«ä�*�u��Û~©àÜ�«äö:� ©ªÛ~£QÜ�«à­���¤¬©ä� �¡ÛMòY�u© � �«ª£ØÜ�¤§�¡® ÛÄ���¡���¬���¡£<éîê�ë+òY©ä� �#ÛÄ��¤¬£Ä� �ÚÜ4�GÜdÞ���«ª��«à��­"Ü�¤¬©ä� �¡ÛÄ© � ��çuÛm�K£¥¤m��áI�¬£�¤¬ÛÄ®m©ª�Ü��"��ÛÄ��¨¡£�«i�4á�«ª£ØÜ�¤§�¡©à�¡­máf¤§��Û ¤¬Ü��¡¨¡��Û%£âúuÜ�ÛÄÞ¡«à£¥®Y���¡«ª��¯

® óÏ£ ­�©ä¦�£l� �¡£�á{Ü�®§�¬£�®y�At�¡�vòY�d¦�£¥¤¬®§©à��� �4á6� �¡£ � £¥«à£��u¤ Ü4�¬£�¨Æ�^Ü�¤¬ÛÄ���¡© � ¢*©à£â¦�£mÜ4«à­���¤§©ª�¬�¡Ûáf��¤Í«à£�Ü�¤¬�¡©ª�¡­!éîê^ë+áf��¤¬Û ç¡«èÜ�£Äç¡�¡¨u£�¤m� �u£�ç¡�¡©ªáf��¤¬Û ¨¡©à®y� ¤§©à�¡çu�¬©à���Ïç¡®¬©ª�¡­!ÛÄ£�Ûm��£�¤§®¬�¡©ªÞütç¡£¥¤¬©à£¥®�¯

® óÏ£<­�©ä¦�£~�¡£¥ò&Ü��ÚÜ4«ª��®¬£¥®m��áì�¬�¡£ � £¥¤ � £�Þu�¬¤¬����Ü��¡¨Ïó ©à�¡�¡�vòùÜ�«ª­���¤§©ª� �uÛ~®î©ª�Ï� �u£ � � Ò«ª£ØÜ�¤§�¡©à�¡­<Û~��¨¡£�«gûi©à� � «àçu¨¡©à�¡­~� �¡£y�Ú¤§®§��Þ¡¤¬����áS� �¡Ü4�îó ©à�¡�u�9ò ©à®î�u����Ü���£¥ã � ©ª£��"� � � Ò«ª£ØÜ�¤§�¡©à�¡­õÜ4«à­���¤§©ª�¬�¡Ûßáf��¤ì� �¡£ � «èÜ�®§®��4á�«ª©à�¡£�Ü�¤0� �u¤¬£�®§�¡��«ª¨áfç¡� � � ©ª���¡®¥¯

® óÏ£�­�©ª¦�£�Üdá{Ü�®y�Øû>®¬©ªÛ~Þu«à£.Ü4«à­���¤§©ª�¬�¡ÛWáf��¤m«ª£ØÜ�¤§�¡©à�¡­d��¤¬©ª­�©à�*ö � £¥�t�¬£�¤§£�¨#«ª©à�¡£�Ü�¤Í�¬�¡¤¬£¥®¬�¡��«ª¨áfç¡� � � ©ª���¡®îç¡�u¨¡£�¤(�¬�¡£ôç¡�¡©äáf��¤¬Û ¨¡©à®y� ¤¬©ª�¡çu�¬©à���òY�¡© � �c©à®(¤§£�®§©à®§� Ü��"�î� ��®¬£â¦�£¥¤ Ü�«6�ý��Þ�£�®l��á�¡��©à®§�<¨ÚÜ4� Üu¯

ó ©ª�¬���¬�¡£Q£ðú � £�Þ*� ©à��� ��á Ò �ÚÜ�Þu�¬£�¤6�*ûvòY�¡© � � � ���"�øÜ4©à�¡®�Þ¡¤¬£¥«à©ªÛ~©ª�ÚÜ�¤§���ÚÜ � �­�¤¬��ç¡�u¨mÛ<Üh� £�¤§©èÜ�«gû£ØÜ � � � �ÚÜ�Þu�¬£�¤m©ª®Í®§£�«äá � ���"� Ü�©à�¡£¥¨GÜ��¡¨ � Ü��#��£Ä¤¬£ØÜ4¨#®¬£¥ÞÚÜ�¤ Üh� £�«ä��áf¤¬��ÛW�¬�¡£<¤§£�®§��¯dæQ£�«à�vòßòQ£­�©ä¦�£�Ü�����çu�¬«à©à�u£(��á�� �¡£î� �u£�®¬©ª® � �ÚÜ�Þu�¬£�¤¬®¥¯

á

Page 17: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

¢Q£cÒ Ó�½P¦«Ô^§^­ÆÅ ¥QÁ�¦qÄÆÅ ÂÕÄÆÅÈÇ]§^Ç

� Æ@×D¼*»{ÊõÑÚÂs×� mÑ¡Ál�ižÇ9Æ�ÐsÂ�× ý�Ì� �¡©ª® � �ÚÜ�Þ*� £�¤.òQ£!¨¡£M�Ú�¡£!� �¡£ � � Ò «ª£ØÜ�¤§�¡©à�u­'ÛÄ�*¨¡£¥«(Ü��u¨O� �¡£!���u«à©à�u£dÛĩ஧� ÜX�£d�K��çu�¡¨ÛÄ�*¨¡£¥«:ûu�¬�¡£l�ýò0�.Û~Ü�©ª�«ª£ØÜ�¤§�¡©à�u­ÄÛÄ��¨¡£�«ª®ìòY�¡© � �ò0£Íç¡®¬£4ûJÜ��u¨�Þ¡¤§£�®§£��"���Û~£lÛ~Ü4� �¡£¥Û<Üh� © � Ü�«�ÚÜ � t­�¤§��ç¡�¡¨�Û<Üh� £�¤§©èÜ�«g¯� ÑÚÊ¥¹v¼tÇ�ÒS¼tÑ¡ÇhÂs¸{ÂDÅ'Æ;�Ö��� � ÖD¸fÑ��(Æ�»fÓDÂDÆ�É#¸fÑÚ»S· ÈDÇh¼�ÊØÈDÆ�»g× � Ð�ÂsÁ�¹v¸{Æ�Â�ÊñI�¡£îÞ¡¤§���¡«ª£�Û ��áD£¥ã � ©à£��"�¬«ª�~«à£ØÜ4¤¬�¡©ª�¡­mÜ�¤§�¡©ª�¬¤ Ü�¤y�~éîê�ëcáf��¤¬Ûmç¡«àÜ�£�©à®0���¡£(��á@�¬�¡£(��çu�¬®§� Ü��¡¨¡©ª�¡­��Þ�£�� ütç¡£�®y� ©ª���¡®�©à� � ��Û~Þ¡ç*�øÜ4�¬©à���ÚÜ4«^«ª£ØÜ�¤§�¡©à�¡­#�¬�¡£���¤§��¯× ý���¬�¡©à® � �¡Ü�Þu� £¥¤òQ£�Þ¡¤¬£¥®¬£¥�t�� �u£á{Ü�®y� £�®y����u�9òY�dÜ�«à­���¤¬©ä� �¡Û áf��¤ì� �u©à®ìòQ£�«à« öý®y� ç¡¨¡©ª£�¨�Þ¡¤§���¡«à£¥Û�¯

û(硤mÜ�«ª­���¤§©ª� �uÛW©à®��ÚÜ�®§£�¨G���'Üd�¡£¥ò � ���"¦�£�¤§®¬©ª���'áf¤¬��Û éîê^ë��¬�&í1äUéWòUç�äUìyöcèXé�î«ÿaæRïPóPÿ1äUé ðê-åaç�õMî¶ö£äUçvóâ¯s�}Þ���«ª���¡��ÛÄ©èÜ4«�� �¡¤§£�®¬�u��«à¨ áfçu� � �¬©à���m�4ᾨ¡£�­�¤§£�£vØî�v¦�£¥¤«Ù<¦4Ü�¤§©èÜ��¡«ª£�®�©à®�� �¡£0æ0�*��«à£ØÜ4�áfç¡� � � ©ª���<���*�øÜ�©ª�¡£�¨.�"�Ä� �¡¤§£�®§�¡��«à¨u©à�¡­ Üͤ¬£�Ü�«äö:¦4Ü�«àç¡£¥¨ÌÙ¾öw¦4Ü�¤§©èÜ��¡«ª£YÞK��«ä���¡��ÛÄ©èÜ�«Ú��ái¨¡£�­�¤§£�£ÈØ � ý�� �u£�©à¤^�8÷`ú+�Í�K���`sÚ�ïMæPõPï"í�îcæRäUçvóâû Ñ ©à�¡®t�.Ü��¡¨ � Ü�Þ�£�¤§�ìÞ¡¤¬�v¦�£¥¨Üm«à�vò0£¥¤0����ç¡�¡¨��4áÜÛlí(Ù �ÞÝàß ïS���� �u£(¨¡£�­�¤¬£�£î�4áDÞK��«ä���¡��ÛÄ©èÜ�«¡�¬�¡¤¬£¥®¬�¡��«ª¨.áfç¡� � � ©à���¡®0òY�¡© � ��¤¬£�Þu¤¬£�®§£��"� � £¥¤§� Ü�©à��Þ���«ª���¡��Û~©àÜ�«äöw®¬©à÷¥£éîê�ëQ¯�óÏ£(­�©ä¦�£îÜ���çuÞ¡ÞK£¥¤ì����ç¡�¡¨�òY�¡© � ��Û<Ü4� � �¡£�®Q� �¡©ª®ì«à�vòQ£�¤ì����ç¡�¡¨.�¬�mòY©ª�¬�¡©à�<Üm«à��­"Ü4¤¬©ª�¬�*öÛÄ© � á{Ü � �¬��¤ì�"�<®§�¡�vòY©à�¡­m�¬�ÚÜ4�YÜ4�t�sá¥öw�¬£�¤¬Û&éîê�ëcáf��¤¬Ûmçu«èÜl����Ùd¦4Ü�¤¬©àÜ��¡«ª£�® � Ü�����£(£âú*Þ¡¤§£�®§®¬£�¨Ü�®0Ü ÞK��«ª���¡��ÛÄ©èÜ�«*� �¡¤§£�®¬�u��«à¨<áfçu� � �¬©à���.�4á@¨¡£�­�¤¬£�£µâ.í(Ù �ÞÝàß «à��­ãá9ï � ñI�¡©à®Qç¡Þ¡Þ�£�¤Q�K��çu�¡¨�£��ÚÜ4�¡«à£¥®ç¡®(�¬��­�©ª¦�£ôÜ���Ü�«ª­���¤¬©ä� �¡Û%òY�u© � �!«à£�Ü�¤¬�u®<á¥öw� £¥¤¬Û éîê�ë}áf��¤§Ûmç¡«àÜ�£Í�v¦�£�¤<Ù'¦4Ü�¤¬©àÜ��¡«ª£�®�©ª�d� ©ªÛ~£� Ö�×ÙØ Ú�ÛÝÜ y{z}|�Ø�y{z}|&ä¶Þ �*� �tç¡®�®¬ç¡�¡®y�øÜ��"�¬©èÜ�«ª«ª�~©àÛÄÞ¡¤¬�v¦�©à�¡­m�¬�¡£Í� ÕÖg×æå Ø8Þ �¬©àÛÄ£(����ç¡�¡¨¡®Y��á�� �u£���£�®y��Þ¡¤¬£ðö¦�©à��çu®(Ü�«ª­���¤¬©ä� �¡ÛÄ®�¯Qó'£ Ü�«à®§�~­�©ä¦�£Í©àÛÄÞ¡¤¬�v¦�£¥¨ ç¡Þ¡Þ�£�¤^�K��ç¡�¡¨¡®^���d� �u£ ¨¡£¥­�¤¬£¥£ ��áSÞ���«ª���¡��Û~©àÜ�«� �u¤¬£�®§�¡��«ª¨õáfçu� � �¬©à���¡®6áf��¤>¤¬£ØÜ4¨*öý��� � £Yéîê�ë!Ü4�¡¨Äáf��¤6¤¬£�Ü�¨*öý��� � £ � ���¡®§� Ü��"�Q¨u£�Þu�¬�õáf��¤¬Ûmç¡«àÜ�£�¯I ý�£ØÜ � � � Ü�®¬£I��硤6ç¡Þ¡Þ�£�¤>����ç¡�u¨Ä«ª£ØÜ�¨u®�� ���¬�¡£ìá{Ü�®y� £�®y�ut�¡�vòY�<Ü�«ª­���¤§©ª� �uÛ áf��¤s�¬�¡£ � ��¤§¤¬£¥®¬Þ����¡¨¡©ª�¡­«à£�Ü�¤¬�u©à�¡­ôÞ¡¤§���¡«ª£�Û¯ËÍÆ�É'Î�Ð@¹vÑu¹v¸{Æ�ÂDÑÚ»YÕJÑÚÉ'Îs»{¼�ËÍÆKÉÏÎ�»f¼&%D¸f¹ðÓGÑ¡Âs× Ãô¹Ø¹vÇh¸('sÐ@¹v¼�¿�ÀGÁ"¸f¼�ÂK¹ÄÒU¼tÑ¡ÇhÂ�¸gÂDÅñìòQ�m¤¬£¥®¬��硤 � £¥®ì¤¬£¥ü�çu©à¤¬£¥¨<�"�~Ü�«à«Ú«ª£ØÜ�¤§�¡©à�u­ÍÜ4«à­���¤§©ª�¬�¡Û~®SÜ4¤¬£�¨ÚÜ4�øÜ�Ü��¡¨ � ��ÛÄÞ¡çu� Ü4� ©ª���õ� ©àÛÄ£�¯I ý�� �u©à® � �¡Ü�Þu� £¥¤�òQ£I®¬�¡�vò�� �ÚÜh�>£¥¦�£��ôáf��¤�®¬©àÛÄÞ¡«ª£Q«ª£ØÜ�¤§�¡©à�¡­^Þ¡¤¬���¡«à£¥Û~®�Ü�®§�¬¤¬���¡­�©à�¡�¡£¥¤¬£¥�t��� ¤ Ü4¨¡£��X�� Ü4��£ðúu©ª®§����£¥�ýòQ£�£��� �¡£¥®¬£l�ýò0�Ĥ¬£¥®¬��çu¤ � £¥®�¯ ý�G¢*£ � �¬©à����ᡯ Ó�òQ£~ç¡®§£<£¥¤¬¤¬��¤yö � ��¤¬¤§£ � � ©à�u­ � �*¨u£�®mÜ4�¡¨Ï«ª£��¡­4� �*öwÞ¡¤¬£¥®¬£�¤y¦�©à�¡­����¡£âö:òìÜØ��Þ�£�¤_ö

Ûmçu� Ü4� ©ª���¡®>� � � ���¡®§�¬¤¬ç � �IÜ � «èÜ�®§®Q�4áJ¦�£¥¤§�Ä®¬©ªÛ~Þu«à£Yæ0����«à£�Ü��õáfç¡� � �¬©à���u®(í_�¥öý¨¡£ � ©ª®¬©à���Ä«ª©à®y� ®øïð¯�óÏ£Þ¡¤§�9¦�£(� �ÚÜh�0òY�¡©ª«à£(Ü � ��ÛÄÞ¡çu� Ü4� ©ª���ÚÜ�«ª«ª�ôç¡�t����ç¡�¡¨u£�¨�«à£�Ü�¤¬�¡£¥¤ � Ü���«à£�Ü�¤¬��Ü4�t�Äáfç¡� � � ©ª���.©à�~� �¡©ª®

ø

Page 18: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

� «àÜ�®¬®�áf¤¬��Ûçâ<íy�9ï(£âúuÜ�ÛÄÞ¡«ª£�®�û�ç¡�¡¨¡£¥¤ôÜ�­�£¥�¡£�¤¬Ü�« � ¤§��Þu�¬��­�¤ Ü4Þ¡�¡© � �ÚÜ�¤§¨¡�¡£�®§®ôÜ�®§®¬ç¡ÛÄÞu� ©ª���#Ü4�t�Þ���«ª���¡��ÛÄ©èÜ4«äöw�¬©àÛÄ£I«à£�Ü�¤¬�¡£¥¤IÛmç¡®y�^çu®¬£ÍÜ�«àÛÄ��®§�2Ûlí(ÙiïY£ðú¡Ü4Û~Þ¡«ª£�®¥¯

�^®§©à�¡­<ÜĨ¡© �J£¥¤¬£¥�t� � ¤y��Þu� ��­�¤ Ü�Þ¡�u© ��� ���¡®y� ¤§ç � �¬©à�����ÚÜ�®¬£¥¨ ���Þ¡®¬£¥ç¡¨¡��¤¬Ü��¡¨¡��Û ­�£��¡£¥¤ Ü4�¬��¤¬®¥û©à�}¢*£ � � ©ª����ᡯÙá�òQ£dÞ¡¤§£�®¬£¥�"��Ü�Û~��¤§£�­�£��¡£¥¤ Ü�« � «èÜ�®§®~�4á<ètöw¨¡£ � ©à®¬©ª����«à©à®y� ®mòY�¡© � �}£ðúu�u©à�¡©ä� ®ÄÜ®¬©ªÛ~©ª«èÜ�¤ �¡çu�~®§� ¤§���¡­�£¥¤~­�Ü�Þ í(â.í_�9ï ¦�£�¤§®¬ç¡®sÛlí(Ù 1 ï¬ïô©à��� �¡£�tç¡Ûm��£�¤Ä��áî£ðú¡Ü4Û~Þ¡«ª£�®õ¤¬£¥üt硩ध£�¨áf��¤ � ��Û~Þ¡ç*�øÜ4�¬©à���ÚÜ4«à«ª�~ç¡�t����ç¡�¡¨¡£¥¨d¦�£¥¤¬®§ç¡®(Þ���«ª���¡��ÛÄ©èÜ4«äöw�¬©àÛÄ£Y«à£ØÜ4¤¬�¡©ª�¡­¡¯QóÏ£�®¬�¡�vò�� �¡Ü4�Y� �¡©ª®� ���¡®§�¬¤¬ç � � ©à��� � ��Û~£¥®IòY©ª� �u©à�Üô«à��­"Ü4¤¬©ª�¬�¡ÛÄ© � á{Ü � �¬��¤Y��ás�¬�¡£l«èÜ�¤§­�£�®y�YÞK��®¬®¬©ª�¡«à£î­�Ü�Þáf��¤ � «èÜ4®¬®¬£¥®��á«ètöý¨u£ � ©ª®¬©ª����«à©à®y� ®¥¯

é�î¶îcæ-ö(Ð�å�î"ï�ïDÏdõMö£ïMç1î�«ª£ØÜ�¤§�¡©à�u­'©à®.Ü'�ý�*Þ�£!��á�«à£�Ü�¤¬�u©à�¡­'áf¤§��Û áf£¥ò £âúuÜ�ÛÄÞ¡«à£¥®.òY�¡© � � �ÚÜ4®��£�£�� òY©ª¨¡£�«ä�d®y� ç¡¨u©à£�¨J¯È ý�c¢*£ � � ©à���ùᡯ øÄò0£õÜ4Þ¡Þ¡«ª���çu¤�� £ � �¡�¡©àütç¡£¥®(� � � ���¡®§�¬¤¬ç � �lÜ � ��� � £¥Þu�� «àÜ�®¬®0��á�¨¡£ � ©à®¬©ª����«à©ª®§�¬®QòY�¡© � � � Ü�����£î«à£ØÜ4¤¬�¡£¥¨�Ü4�¬�¬¤¬©à�uçu� £ðöý£¥ã � ©à£��"�¬«ª�~Ü��¡¨ � Ü4���K£î«ª£ØÜ�¤§�¡£�¨�©ª�Þ���«ª���¡��ÛÄ©èÜ4«¡� ©àÛÄ£��¡çu� � Ü��¡�¡���ì�K£(«ª£ØÜ�¤§�¡£�¨.Ü4�¬�¬¤¬©à�uçu� £ðöý£¥ã � ©à£��"�¬«ª�Ä©à�.Þ���«ä�*�u��Û~©àÜ�«¡� ©ªÛ~£4¯6ñI�¡©ª®©à®~� �¡£ù�Ú¤§®§�¨¡£¥Û~���u®§� ¤¬Ü4� ©ª���}�¬�ÚÜ4�Ü4�§� ¤¬©ª�¡çu�¬£âöý£âã � ©à£¥�"��«à£�Ü�¤¬�¡©ª�¡­ � Ü�� �K£ � ��Û~Þuçu�øÜ4�¬©à���¡Ü�«à«ä��ÚÜ�¤§¨i¯�ìÃ.Ë%ÒU¼tÑ¡ÇhÂ�¸gÂDÅ65~¸f¹vÈê�(¼tÇhÁ�¼tÎ�¹vÇ9Æ�Â}ÑÚÂ�×98 ¸gÂ�ÂDÆF5Ò �ÚÜ�Þ*� £�¤�ø � ���"�øÜ�©ª�¡®�¨¡£¥� Ü�©à«ª£�¨Ü��ÚÜ�«ä��®¬£�®���á6�¬�¡£ � � Ò «ª£ØÜ�¤§�¡©à�¡­ÄÜ��u©à«à©ä� ©ª£�®ì��ás� �¡£ � £¥¤ � £�Þu�¬¤¬���Ü��¡¨�ó ©à�¡�¡�vò Ü4«à­���¤§©ª�¬�¡Û~®¥¯�ó �¡©ª«à£�� �¡£¥®¬£dÜ�«à­���¤¬©ª�¬�¡ÛÄ®ô�ÚÜئ�£ ��£�£¥�O®§�¬ç¡¨¡©à£¥¨O©à��� �u£d���¡«ª©à�¡£Ûĩ஧� ÜX�£ðöý����ç¡�¡¨�ÛÄ�*¨u£�«6�"�!Û~Ü��"�!¤¬£¥®¬£�Ü�¤ � �¡£¥¤¬®¥û�«à©ä�¬�¬«à£Íò0Ü�®lÞ¡¤§£¥¦�©à��çu®¬«ª�ù��u�9òY�ÏÜ�����çu�î�¬�¡£�©ª¤Þ�£�¤§áf��¤¬Û~Ü�� � £î©ª��� �¡£ � � Ò «à£ØÜ4¤¬�¡©ª�¡­ôÛ~��¨¡£¥«:¯ ý�O¢�£ � �¬©à���¤ø*¯ÙÓ ò0£�­�©ä¦�£Ü���£âú*Þ����¡£��"�¬©èÜ�«I«ª�9òQ£�¤õ����ç¡�¡¨����G� �u£¤§ç¡�¡�¡©ª�¡­!� ©ªÛ~£.�4á�� �u£

ó ©à�u�¡�vòþÜ�«ª­���¤§©ª� �uÛ òY�u£��cç¡®¬£¥¨'Ü�®lÜ � � Ò «à£�Ü�¤¬�u©à�¡­�Ü4«à­���¤§©ª�¬�¡Û áf��¤(� �¡£ � «èÜ�®§®î��áQÞK��®§©ª�¬©ª¦�£«à©ª�¡£ØÜ4¤ì� �¡¤§£�®¬�u��«à¨áfç¡� � �¬©à���u®^�v¦�£¥¤(æ0����«ª£ØÜ���£âúuÜ�ÛÄÞ¡«ª£�®�¯0ñI�¡©à®^Ü��¡®yò0£¥¤¬®îÜ����Þ�£���ü�çu£�®§�¬©à������ᢠ� �¡ÛÄ©ª�§��Ü��¡¨�©à®I�¬�¡£I�Ú¤¬®y�^�u£�­"Ü4�¬©ª¦�£î¤¬£�®§ç¡«ª�Yáf��¤ìó ©ª�¡�¡�vò ©ª��� �¡£ � � Ò «à£�Ü�¤¬�¡©ª�¡­ôÛÄ�*¨¡£¥«:¯ ý�+¢*£ � � ©à��� ø*¯Ùá#òQ£!Þ¡¤§�9¦�£cÜ#®§� ¤§���¡­G�¡£¥­"Ü4�¬©ª¦�£!¤§£�®§ç¡«ª�.áf��¤<� �¡£ � £�¤ � £¥Þu� ¤§���-Ü4«à­���¤§©ª�¬�¡Û

�"�c®¬�¡�vòY©à�u­�� �¡Ü4� � £�¤ � £¥Þu� ¤§��� � Ü��u�¡���Í£¥ã � ©ª£��"� «ä� � � Ò «à£�Ü�¤¬�c«à©ª�¡£ØÜ4¤(� �¡¤§£�®¬�u��«à¨cáfç¡� � �¬©à���u®£¥¦�£���ç¡�¡¨u£�¤^�¬�¡£mç¡�u©ªáf��¤§Û ¨¡©à®y� ¤¬©ª�¡çu�¬©à������!æ0����«ª£ØÜ��£âúuÜ�ÛÄÞ¡«à£¥®�¯Aûîçu¤�Þ¡¤§�*��á6®¬©àÛÄÞ¡«ª©��Ú£¥®^Ü4�¡¨®§�¬¤¬£¥�¡­�� �u£��¡®YÜ��~£ØÜ�¤§«à©ª£�¤S�¡£�­�Ü4� ©ä¦�£�¤§£�®§ç¡«ª�0¨¡ç¡£^�¬�õ¢ � �¡ÛÄ©ª�§�0���~� �u£ � � Ò öwÛ~��¨¡£�«¾Þ�£�¤yáf��¤¬Û~Ü�� � £��á�� �¡£ � £�¤ � £�Þu�¬¤¬���dÜ�«à­���¤¬©ä� �¡Û¯

û(��� �u£dÞ���®¬©ä� ©ª¦�£�®¬©à¨u£�û0òQ£d®§�¡�vò ©à�}¢*£ � � ©à��� ø*¯ ø��¬�ÚÜ4� � £¥¤ � £�Þu�¬¤¬���Ì©ª®~ÜcÞK��«ª���¡��ÛÄ©èÜ�« ö� ©ªÛ~£ � � Ò «à£ØÜ4¤¬�¡©ª�¡­GÜ�«ª­���¤¬©ä� �¡Û áf��¤<� �¡£ � «èÜ�®§®.��á��¡£�®y� £¥¨+áfç¡� � � ©ª���¡®.ç¡�u¨¡£�¤�� �u£!ç¡�¡©ªáf��¤¬Û¨¡©ª®§� ¤§©à�¡ç*� ©à������� æ0�*��«à£ØÜ4��£ðú¡Ü4Û~Þ¡«ª£�®¥¯^ñI�¡©ª® � «èÜ�®§®�©à®jt�¡�vòY� � �<��£ �¡Ü�¤¬¨dáf��¤ � £�¤ � £�Þu�¬¤¬����©ª�� �u£�­�£��¡£¥¤ Ü�« � � Ò Û~��¨¡£¥«JòY�¡£�¤§£î� �¡£l¨¡©ª®§� ¤§©à�¡ç*� ©à��� � Ü���K£�Ü�¤§�¡©ª�¬¤ Ü�¤y��¯

ú

Page 19: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

ÒS¼�ÑuÇ4Â�¸{Â�Åê>ôÇ4¸fÅ�¸gÂ#)âË ¼�ÂK¹Ø¼�Ç9¼�×9@ÑÚ»D�ýÊ�Î�ÑÚÁ�¼�Ê~Ð�Âs×D¼tÇ�B�Âs¸D�wƾÇhÉ �¸{Ê¥¹vÇh¸D'�Ð@¹9¸fÆ�ÂsÊ ý�#� �u©à® � �ÚÜ4Þu� £¥¤mò0£ � ���¡®¬©ª¨¡£�¤õÜ��ÚÜ4� çu¤ Ü�«Q«à£ØÜ4¤¬�¡©ª�¡­ Þ¡¤§���¡«à£¥Û�òY�u© � �G�ÚÜ�®ô��£�£¥��®§� çu¨¡©à£¥¨G�t�®¬£â¦�£¥¤ Ü�«6¤§£�®§£ØÜ�¤ � �¡£�¤§®¬"î«à£�Ü�¤¬�u©à�¡­<Ü�� ç¡�{t�¡�vòY�c��¤¬©ª­�©à��ö � £¥�"� £�¤§£�¨ �¡Ü�«ªáf®§ÞÚÜ � £�í{«à©ª�¡£ØÜ4¤��¬�¡¤¬£¥®¬�¡��«ª¨áfç¡� � � ©ª���¾ï�áf¤§��Û-¤¬Ü��¡¨¡��Û+£âúuÜ�ÛÄÞ¡«à£¥®i¨¡¤¬ÜvòY��ç¡�¡©äáf��¤¬ÛÄ«ª�Yáf¤§��Û��¬�¡£>��¤§©à­�©ª�*ö � £��"� £¥¤¬£�¨îçu�¡©ª�@®§Þ¡�¡£¥¤¬£©à�së Ø � óÏ£(¨¡£�® � ¤§©à��£lÜm�¡£¥ò-Ü�«à­���¤¬©ä� �¡Ûß�ÚÜ4®¬£�¨�����Üm­�£���ÛÄ£¥�¬¤¬© � Üئ�£�¤¬Ü�­�©ª�¡­m� £ � �¡�¡©àütç¡£(òY�u© � �� Ü4�-«à£�Ü�¤¬� £âã � ©à£¥�t�¬«ª� £â¦�£��-©à� � �¡£cÞ¡¤¬£¥®¬£¥� � £Ï��ámÛÄ���¡���¬���¡© � �¡��©à®§£�ígÜ�­�£��¡£¥¤ Ü�«ª©à÷�Ü4� ©ª���Ì��á� �u£còQ£�«ª«äöý®y� ç¡¨u©à£�¨ � «àÜ�®¬®§©�� � Ü4� ©ª��� �¡��©ª®¬£�Û~��¨¡£¥«ºï⯠óÏ£c®§�¡�vòå� �ÚÜh����硤��u£¥ò Ü�«ª­���¤§©ª� �uÛ ©ª®����� ��®¬©àÛÄÞ¡«ª£�¤YÜ��¡¨á{Ü�®§�¬£�¤Y� �ÚÜ4��Þ¡¤§£¥¦�©à��ç¡®(Ü�«ª­���¤§©ª� �uÛ~®0òY�u© � �d�¡Üv¦�£ ��£�£¥� Þ¡¤¬��ÞK��®§£�¨áf��¤I� �¡©ª®Þ¡¤§���¡«à£¥Û�¯�ìÃ.Ë%Ã�Â�ÑÚ»fƾÅ�Ð�¼�ÊõÆ��e�(¼tÇhÁ�¼tÎ�¹ØÇhÆ�Â}Ñ¡Âs×98 ¸{ÂsÂDÆF5 ÖD¸{Ñ� mÆiÆ�Ê¥¹v¸gÂDÅϹ9ÈD¼ � Ñ¡Ç9Å�¸{ ý�d�¬�¡©à® � �ÚÜ�Þu�¬£�¤�òQ£m¨¡£¥® � ¤§©à��£mÜ��u¨!Ü��ÚÜ�«ä��÷�£mÜ~�¡£¥ò)á{Ü�ÛÄ©à«ä�.��áS���*��®§� ©ª�¡­4öw�ÚÜ�®¬£¥¨ � � Ò ÛÄ��¨¡£�«Ü�«ª­���¤¬©ä� �¡ÛÄ®sáf��¤U«ª£ØÜ�¤§�¡©à�u­î«à©à�u£ØÜ�¤6�¬�¡¤¬£¥®¬�¡��«ª¨Äáfçu� � �¬©à���¡®¥¯�ó �¡©ª«à£I��硤S�¡£âò Ü�«ª­���¤¬©ä� �¡ÛÄ®6Ü4¤¬£ � ����ö� £¥Þu� ç¡Ü�«à«ä�.Ü��u¨�Ü�«à­���¤¬©ª�¬�¡ÛÄ© � Ü4«à«ª�Íütç¡©ª�¬£(¨¡© �J£¥¤¬£��"�Iáf¤§��Û � £�¤ � £¥Þu� ¤§���dÜ4�¡¨�ó ©à�¡�¡�vòlû�ò0£lÞ¡¤§�9¦�£Þ�£�¤§áf��¤¬Û~Ü�� � £^�K��ç¡�¡¨¡®ìáf��¤U� �¡£��¡£¥ò-Ü�«à­���¤¬©ä� �¡ÛÄ®>òY�¡© � ��Ü�¤§£(¤¬£¥Û<Ü4¤¸hÜ��¡«ä�õ®¬©àÛÄ©à«àÜ�¤>� � �¬�¡��®§£(��á� £¥¤ � £�Þu�¬¤¬����Ü4�¡¨~ó ©ª�¡�¡�vòlû��¬�tç¡®0®§ç¡­�­�£¥®§� ©ª�¡­l� �ÚÜ4�U� �u£�®¬£�ò0£¥«à« öý®§�¬ç¡¨¡©ª£�¨õ���¡«ª©à�¡£�Ü�«ª­���¤¬©ä� �¡ÛÄ®6©ª�®¬��Û~£î®§£��¡®§£ � ��¤¬¤¬£¥®¬Þ����¡¨�� �õ©à�u®§�øÜ4� � £¥®^��ás���*��®y� ©ª�¡­¡¯ ý��¢*£ � � ©ª��� ü*¯ÙÓÏòQ£ ¨¡£�® � ¤§©à��£!Ü'ÞÚÜ�¤¬Ü�ÛÄ£¥� £¥¤¬©ª÷�£�¨�¦�£¥¤¬®¬©ª���Ì��áî�¬�¡£d­�£¥��ÛÄ£¥� ¤§© � Üئ�£�¤¬Ü�­�©ª�¡­

Ü�«ª­���¤¬©ä� �¡Û&áf¤¬��Û Ò �ÚÜ4Þu� £¥¤Iú.Ü��¡¨d®¬�¡�vò)� �ÚÜh�^�¬�¡©à®�Ü�«à­���¤¬©ª�¬�¡Û Þu¤¬��¨¡ç � £¥®(�"��ÞK���¬�¡£�®§£�®îòY�¡��®§££�¤§¤¬��¤Ä¤ Üh� £©à®Ä�K��çu�¡¨¡£�¨OÜvò0ÜØ��áf¤¬��Û ��ì�� � ý�Ì¢*£ � � ©ª���dü�¯Ùácò0£�ç¡®¬£dÜÏ���*��®y� ©ª�¡­'Ü4«à­���¤§©ª�¬�¡Û� �d£¥ã � ©ª£��"� «ä� � ���"¦�£¥¤§�ô� �u©à®�­�£���ÛÄ£¥�¬¤¬© � òQ£ØÜXc«à£�Ü�¤¬�u©à�¡­dÜ�«à­���¤¬©ª�¬�¡ÛW©ª�t�¬� Üd®y� ¤§���¡­ «ª£ØÜ�¤§�¡©à�¡­Ü�«ª­���¤¬©ä� �¡ÛßòY�¡© � � � Ü��­�£��¡£¥¤ Ü4�¬£ Ü4¤¬�¡©ä� ¤ Ü4¤¬©à«ä�ÄÜ ��� 硤 Üh� £��t��Þ���� �u£�®¬£¥®�¯æQ��ç¡®¬©ª�¡­Ï�¬£ � �¡�u©àütç¡£�®.áf¤§��Û � �u£�� �u£���¤y���4áî«èÜ�¤§­�£�Û<Ü4¤¬­�©ª� � «èÜ�®§®¬©�� � Ü4�¬©à����� � � Ü�¤§£¥áfç¡«ª«ª�

����ç¡�¡¨ô� �u£I��çuÛm�K£¥¤6��á�£âúuÜ�ÛÄÞ¡«ª£�®>¤¬£¥üt硩ध£�¨õ�"� � �u£I�¡£¥òOÜ�«à­���¤¬©ª�¬�¡ÛÄ®�û9ò0£I®§�¡�vò}� �¡Ü4�6� �u£I�¡£¥òÜ�«ª­���¤¬©ä� �¡ÛÄ® � Ü���K£(¦�©ª£¥òQ£�¨ Ü4®I�ÚÜ4� çu¤ Ü�« � � Ò Ü��¡Ü�«à��­�ç¡£�®I��áD�¬�¡£l���¡«à©ª�¡£.Uuöw�¡��¤§Û%Ü�«ª­���¤¬©ä� �¡ÛÄ®òY�¡© � �Ï�ÚÜئ�£Ä¤§£ � £��"� «ä�!�K£¥£��c®§�¬ç¡¨¡©à£¥¨Ï�"��/�v¦�£�û�Ô@©ª�¬�¬«à£¥®§� ���¡£�û@Ü4�¡¨#¢ � ��çu硤¬Û~Ü��¡®~í�/�v¦�£þïMîèXéxíàû��@÷`÷Gü�ïmÜ��u¨g/î£��"� ©ª«à£Ü��¡¨ Ô@©ª�§� «à£¥®§�¬���¡£cí�/î£��"�¬©à«à£Ü��u¨¤Ô@©ª�§� «ª£�®§�¬���¡£4û^�8÷X÷`÷"ïâ¯���®Ä®¬Þ�£ � ©èÜ�«� Ü4®¬£�®I�4á��¬�¡£î�¡£âò Ü�«ª­���¤§©ª� �uÛ~®¥û"�t�~�øÜXt©ª�¡­µUXW �mÜ��¡¨îUsWïY òQ£î���u�øÜ4©à�.�ÚÜh� ç¡¤¬Ü�«J���*��®§� ©ª�¡­4ö�ÚÜ�®§£�¨ � � Ò Ü��ÚÜ�«ª��­�ç¡£¥®6��á � £�¤ � £¥Þu� ¤§���<Ü4�¡¨ôó ©à�¡�¡�vò}¤¬£�®§ÞK£ � �¬©ª¦�£¥«ª��¯6ñI�¡£qU�WðY � Ü�®¬£Y��á���çu¤Ü�«ª­���¤¬©ä� �¡Û � Ü�� Ü�«à®§�Y�K£S¦�©à£¥òQ£�¨mÜ�®�Ü�­�£��u£�¤ Ü4«à©à÷�Ü4� ©ª���ÄífòY©ä� �ÍÜ�� ©ªÛ~Þu¤¬�v¦�£¥¨ ® Ü4Û~Þ¡«ª£ � ��ÛÄÞ¡«à£ðú*©ª�ý�����ç¡�¡¨¾ï���á0[�Ü � t®¬����Ü��u¨ Ò ¤ Üئ�£¥�#± ® � � Ò öýÛÄ�*¨¡£¥«Q������®§�¬©à�¡­4öw�ÚÜ�®§£�¨GÜ�«ª­���¤§©ª� �uÛ áf��¤m«ª£ØÜ�¤§�¡©à�¡­�§®§ÞÚÜ�¤¬®§£ÍÞ�£�¤ � £¥Þu� ¤§���¡®�¯ �

ü

Page 20: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

mÆiÆ�Ê¥¹9¸{ÂDÅÏÑÚÂ�×ñ@�Ñ¡Çh×#)¥ËÍƾÇ9¼ÏÕJ¼t¹�ËÍÆ�ÂsÊâ¹vÇhÐsÁ�¹9¸fÆ� ý�m� �u©à® � �ÚÜ4Þu� £¥¤6ò0£Y®¬�u�9ò�� �¡Ü4�SÜ � «ª��®¬£ � ���¡�u£ � �¬©à���ô£ðú*©à®§�¬®>�K£â�ýò0£¥£��Ä�ÚÜ�¤§¨*ö � ��¤¬£ì®§£¥� � ���¡®§�¬¤¬ç � ö� ©ª���iûKÜ~�ý��ÞK£ô�4áS�ÚÜ�¤¬¨u�¡£�®§®îÜ�ÛÄÞ¡«à© � � Ü4�¬©à����áf¤¬��Û � ��ÛÄÞ¡çu�øÜh� ©à���ÚÜ�« � ��Û~Þ¡«ª£âú*©ª�ý��ûiÜ��u¨ �K����®y� ©à�u­Ü�«ª­���¤¬©ä� �¡ÛÄ®�©à� «ª£ØÜ�¤§�¡©à�¡­~� �¡£¥��¤§��¯�óÏ£ô£âú*Þ¡«ª��©ª�^� �¡©ª® � ���u�¡£ � � ©à���!�K���¬�!� ��©àÛÄÞ¡¤§�9¦�£ Þ¡¤§£¥¦�©à��ç¡®«à£�Ü�¤¬�u©à�¡­��¬�¡£���¤§�m¤§£�®§ç¡«ª�¬®6òY�¡© � �ÄÜ�¤§£ì�ÚÜ�®§£�¨õ���õ�K����®y� ©à�u­îÜ��¡¨m�¬�î���u�øÜ4©à�ô©àÛÄÞ¡¤§�9¦�£�¨ô�ÚÜ4¤¬¨*ö � ��¤§£®¬£â� � ���¡®y� ¤§ç � �¬©à���¡®�©à� � ��ÛÄÞ¡«à£ðúu©ä�ý�<�¬�¡£���¤y��¯ ý��¢*£ � � ©ª���u�u¯ á�òQ£.®§�¡�vòù�¬�ÚÜ4�Í� �¡£~�ÚÜ�¤¬¨�ö � ��¤¬£<®§£¥� � ���¡®y� ¤§ç � �¬©à���#�4áe ýÛ~ÞÚÜ4­�«à©àÜ�÷�÷¥�'íD ýÛõö

ÞÚÜ�­�«à©èÜ4÷�÷��uû*�8÷X÷Gø�ïâûhòY�¡© � �ô£¥®§� Ü��¡«à©ª®¬�¡£¥®�� �u£ì£âú*©à®y� £¥� � £I��ᾨu©à®§�¬¤¬©ª�¡çu� ©ª���¡®�ç¡�¡¨¡£¥¤�òY�¡© � �õæ0�*��«à£ØÜ4�áfç¡� � � ©ª���¡®îÜ�¤§£m�¡©ª­��¡«ª��©ª�ÚÜ�Þ¡Þu¤¬�Øú*©àÛ~Ü��¡«à£4ûKÛ~ÜØ��K£ ¦�©ª£¥òQ£�¨cÜ�®îÜ.���*��®§� ©ª�¡­.Ü4«à­���¤§©ª�¬�¡Û�¯ Ñ ��¤§£­�£¥�¡£�¤¬Ü�«à«ä��û*òQ£îÞ¡¤§�9¦�£î� �ÚÜ4�IÜ��"���K����®y� ©à�u­ôÜ�«à­���¤¬©ä� �¡Ûþ��©à£¥«à¨¡®ìÜ � ��¤§¤¬£�®§ÞK���¡¨¡©à�u­Ä�¡Ü�¤¬¨*ö � ��¤§£(®¬£â�� ���¡®§�¬¤¬ç � � ©à���i¯$�^®¬©ª�¡­�� �¡©ª® � ���¡�u£ � �¬©à���!òQ£<Ü4Þ¡Þ¡«ª��t�¡�vòY�'���*��®y� ©ª�¡­Ü�«à­���¤¬©ª�¬�¡ÛÄ®^� �����*�øÜ�©ª�©àÛÄÞ¡¤§�9¦�£�¨�����ç¡�¡¨¡®ôáf��¤ô�¡Ü�¤¬¨*ö � ��¤§£�®¬£â� � ���¡®y� ¤¬ç � � ©ª���GòY�¡© � ��Û~Ü4� � � t�¡�vòY�}«ª�9òQ£�¤ô����ç¡�u¨¡®áf¤¬��Û ���*��®y� ©ª�¡­õÜ��¡¨�� �tç¡®^Ü�¤¬£î��Þu� ©ªÛ<Ü�«�òY©ª�¬�¡©à��� �u©à® � «àÜ�®¬®I�4á�� £ � �¡�¡©ªü�çu£�®�¯ñI�¡£ ���*��®§� ©ª�¡­ÏÜ�«à­���¤¬©ª�¬�¡Û òY�¡© � � � ��¤¬¤¬£¥®¬Þ����¡¨¡®~� �� ýÛÄÞÚÜ�­�«ª©èÜ�÷¥÷��*± ®õ�ÚÜ�¤§¨*ö � ��¤¬£�®¬£¥� � ����ö

®§�¬¤¬ç � � ©ª�����ÚÜ�®m� �u£�Þ¡¤¬��Þ�£�¤y�ý�#� �¡Ü4�ô©ª�m­�£��¡£¥¤ Ü4�¬£�®õ���¡«ä�N�y®¬ÛÄ�*���¬�{� ¨¡©ª®§�¬¤¬©à�uçu� ©ª���¡® òY�¡© � ��¨¡��¡����Ü�®¬®§©à­��O�¬�*�#Û ç � �Ìò0£¥©à­��"��� ��Ü4�t�}®¬©ª�¡­�«à£�£âúuÜ�ÛÄÞ¡«à£4¯Ã ý�-¢�£ � �¬©à���ñ�*¯ øcòQ£�®¬�¡�vò � �ÚÜh�®¬ÛÄ����� �õ���*��®y� ©ª�¡­lÜ�«à­���¤¬©ä� �¡ÛÄ® � Ü��Ä�K£Yç¡®§£�¨~�¬�l­�©ª¦�£YÜl�u£¥òO¦�£�¤¬®§©à���Ä��á#[�Ü � t®¬���3± ® � £¥«à£��u¤ Ü4�¬£�¨��Ü4¤¬ÛÄ���¡© � ¢*©ª£¥¦�£lÜ4«à­���¤§©ª�¬�¡ÛMáf��¤ì«ª£ØÜ�¤§�¡©à�u­méîê�ëcáf��¤¬Ûmçu«èÜ�£�ç¡�¡¨¡£¥¤ì� �¡£�ç¡�¡©ªáf��¤¬Û ¨¡©à®y� ¤§©à�¡çu�¬©à���ç¡®§©à�¡­cÛ~£¥Ûm��£�¤¬®§�¡©àÞ�ütç¡£¥¤¬©à£¥®�¯¤û(硤õ�¡£¥ò%¦�£�¤§®¬©à������á�� �u£ ¢*©ª£¥¦�£�¡Ü�®ÄÜc®§©à­��¡© � � Ü4�t�ÄÜ4®§��Û~Þ�ö� �4� © � ©àÛÄÞ¡¤¬�v¦�£¥Û~£¥�"�õ©à�G¤¬çu�¡�¡©à�u­��¬©àÛÄ£�¯'æQ� � ��Ûm�¡©à�u©à�¡­ ��硤m� £ � �¡�¡©àütç¡£¥®ÄòY©ä� ��¤¬£ � £¥�t�Ä©ªÛõöÞ¡¤§�9¦�£�ÛÄ£��"� ®Y� �ô�4� �¡£¥¤^Ü4®¬Þ�£ � �¬®Y��áD� �¡£Í¢*©ª£¥¦�£4ûuòQ£����u�øÜ�©ª��Ü��_òâ.íDÙ?á¬ó�ìGôöõâïwöw�¬©àÛÄ£(Ü�«ª­���¤¬©ä� �¡Ûþáf��¤«à£�Ü�¤¬�u©à�¡­�á¥öw�¬£�¤§Û÷Ù�ö:¦4Ü�¤¬©àÜ��¡«à£(éîê�ëGç¡�¡¨¡£¥¤Yç¡�¡©ªáf��¤¬Û&òY©ä� ��ÛÄ£�Ûm��£�¤§®¬�¡©ªÞütç¡£¥¤¬©ª£�®�¯ÕiÉ'ÆiÆÚ¹9Èg mÆiÆ�Ê¥¹v¸gÂDÅ'ÑÚÂ�×ÌÒS¼�ÑuÇ4Â�¸{Â�Å65~¸º¹9È � ÑÚ»{¸{Á"¸{Æ�Ð�Ê���Æ�¸gÊ¥¼ñI�¡©ª® � �ÚÜ�Þ*� £�¤ � ���¡®§©à¨¡£¥¤¬®0�¬�¡£�Þu¤¬���¡«ª£�Û ��á@«ª£ØÜ�¤§�¡©à�u­ «à©ª�¡£ØÜ4¤S�¬�¡¤¬£¥®¬�¡��«à¨<áfç¡� � � ©ª���¡®Q©à�~�¬�¡£(Þ¡¤§£�®_ö£�� � £l��á�ì$èXéWö£õMö£äUå/óAç�äUöwó8ïâû¡ÜôÛÄ��¤¬£(¨u£�Û~Ü��¡¨¡©ª�¡­m�¡��©ª®¬£îÛÄ�*¨u£�«K�¬�ÚÜ���£�©ä� �¡£¥¤0� �u£ � «èÜ4®¬®¬© � � Üh� ©à����¡��©ª®¬£���¤.ÛÄ���¡���¬���¡© � �¡��©ª®¬£�Û~��¨¡£�«ª®ÄòY�u© � �Ìò0£ � ���¡®§©à¨¡£¥¤¬£¥¨�©ª� Ò �ÚÜ�Þ*� £�¤^úu¯�æQ� � ��Û �¡©à�*ö©à�u­.©à¨¡£�Ü�®^áf¤§��Û � �¡£mÞu¤¬£¥¦�©ª��ç¡®��ýòQ� � �ÚÜ�Þu�¬£�¤¬®¥ûiò0£ ¨¡£�¤§©ª¦�£õÜ~�¡£¥ò�á{Ü�Ûĩ૪�.��áSÛ<Ü4«à© � ©à��çu®��u��©à®§£� ��«à£�¤¬Ü��"��Ü�«à­���¤¬©ä� �¡ÛÄ®Qáf��¤I«ª£ØÜ�¤§�¡©à�u­ô«à©à�u£ØÜ�¤0�¬�¡¤¬£¥®¬�¡��«à¨�áfç¡� � � ©à���¡®�¯

ý� ¢*£ � � ©ª��� ÷*¯ �!ò0£ ¨u£�® � ¤¬©ª�K£dÜÏ�¡£âò �K����®§�¬©à�¡­ÏÜ�«ª­���¤¬©ä� �¡Û�òY�¡© � �JûI«à©w4£\ ýÛÄÞÚÜ�­�«à©èÜ4÷�÷���± ®Ü�«ª­���¤¬©ä� �¡Û&áf¤¬��Û Ò �ÚÜ4Þu� £¥¤È�uûJ­�£��¡£¥¤ Ü4�¬£�®(���u«ª��®¬ÛÄ����� ��¨¡©à®y� ¤¬©ª�¡çu�¬©à���¡®¥¯jûîçu¤��¡£âòMÜ4«à­���¤§©ª�¬�¡Û©à®ô®§©à­��u©�� � Ü��"� «ª�Ïá{Ü�®§�¬£�¤ô� �¡Ü��ø ýÛ~ÞÚÜ4­�«à©àÜ�÷�÷¥�*± ®mÜ�«à­���¤¬©ª�¬�¡Û Ü��¡¨��ÚÜ�®õ®¬£â¦�£¥¤ Ü�«���� �u£�¤õ¨¡£�®§©à¤¬Ü��¡«à£

Page 21: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

Þ¡¤§��ÞK£¥¤§�¬©à£�®Í®¬ç � ��Ü4®Í� �u£�Ü��¡©ª«à©ª�ý��� � ­�£��¡£¥¤ Ü4�¬£<«èÜ4¤¬­�£ðöýÛ~Ü�¤¬­�©ª���t��Þ���� �u£�®¬£¥®õÜ��¡¨'�¬�dç¡®§£.¤¬£�Ü�«äö¦4Ü�«àçu£�¨�ò0£�ÜX��"��ÞK���¬�¡£�®§£�®¥¯

ý�õ¢*£ � � ©ª���s÷u¯ ÓYò0£Q®¬�¡�vòG�¬�ÚÜ4����硤��¡£¥ò�������®§�¬©à�¡­�Ü�«ª­���¤¬©ä� �¡Û � Ü��Í�K£Uç¡®§£�¨m©ª� � ���hݧç¡� � � ©ª���òY©ª�¬���¬�¡£0Þ¡Ü�¤ Ü�ÛÄ£¥�¬£�¤§©à÷�£¥¨ Üئ�£�¤ Ü4­�©à�¡­^Ü�«ª­���¤¬©ä� �¡Û�áf¤¬��Û�¢*£ � � ©à���yü*¯ÙÓY�¬�����*�øÜ�©ª��Û~Ü�«ª© � ©ª��ç¡®i�u��©à®§£� ��«à£�¤¬Ü��"�ì¦�£¥¤¬®§©à���¡®Y��á�� �u£ � � Ò ÛÄ�*¨u£�«J«ª©à�¡£�Ü�¤0�¬�¡¤¬£¥®¬�¡��«ª¨�«à£ØÜ4¤¬�¡©ª�¡­ôÜ�«ª­���¤¬©ä� �¡ÛÄ®0¨u£�® � ¤¬©ª�K£¥¨�©ª�¢*£ � � ©ª���±ü*¯Ùáu¯%ñI�¡£c�K��çu�¡¨¡®���-Û~Ü�«à© � ©ª��ç¡®<�u��©à®§£c�¬��«à£¥¤ Ü�� � £cÜ��¡¨-®¬Ü�ÛÄÞ¡«à£ � ��Û~Þ¡«ª£âú*©ª�ý�}��á� �u£�®¬£^�¡£¥ò � � Ò Ü�«ª­���¤¬©ä� �¡ÛÄ® � «à��®§£�«ª� � ��¤§¤¬£�®§ÞK���¡¨~�¬�st�¡�vòY�.����ç¡�¡¨u®Uáf��¤>�¬�¡£^���¡«à©ª�¡£«Uuöý�u��¤¬ÛÜ�«ª­���¤¬©ä� �¡ÛÄ®QòY�u© � �©à� � «à硨¡£(�¬�¡£ � £¥¤ � £¥Þu� ¤§��� Ü4�¡¨ó ©ª�¡�¡�vò�Ü4«à­���¤§©ª�¬�¡Û~®QÜ�®Y®¬Þ�£ � ©èÜ�« � Ü�®¬£¥®�¯

û(硤6Ü��¡Ü�«ª��®¬©ª®�¤§£¥¦�£�Ü�«à®�Ü��m©ª�t�¬£�¤§£�®§�¬©à�¡­ � ���¡�u£ � �¬©à��� �K£â�ýò0£¥£��ô� �¡£Q®¬ÛÄ�*�4� �¡�¡£¥®¬®6�4áÚ�K����®y� ©à�u­Ü�«ª­���¤¬©ä� �¡ÛÄ®�Ü��¡¨'�¬�¡£.«ª£¥¦�£¥«0��áYÛ~Ü�«ª© � ©ª��ç¡®l�¡��©ª®¬£Ä� ��«ª£�¤ Ü4� � £<òY�u© � � � Ü4�#�K£.Ü � �¡©à£â¦�£¥¨�©ª�'� �u£� � Ò ÛÄ�*¨¡£¥«:¯r ý��¢*£ � � ©ª��� ÷u¯ á ò0£�çu®¬£�� �¡©ª® � ���¡�¡£ � � ©ª���#� �!Þ¡¤§�v¦�£�� �ÚÜh�m� �¡£<¨¡©à®y� ¤¬©ª�¡çu�¬©à���¡®­�£¥�¡£�¤¬Ü4� £¥¨��"����硤I�u£¥ò����*��®§� ©ª�¡­õÜ�«ª­���¤¬©ä� �¡ÛùÜ�¤§£l��Þu� ©ªÛ<Ü4«à«ª�õ®§Û~�����¬�i¯ÒS¼�ÑuÇ4Â�¸{Â�Å � Æ�ÂDƾ¹ØÆKÂD¼r��� � Ð�Âs×D¼tÇs�ÍÇ9Æ@×�Ð�Á�¹X��¸{Ê¥¹vÇh¸D'�Ð@¹9¸fÆKÂ�Êë�©ª�ÚÜ�«à«ä��û�©à� Ò �ÚÜ4Þu� £¥¤��Ø�ôòQ£l¤¬£â� ç¡¤§��¬�m� �¡£lÞ¡¤§���¡«à£¥Û ��ás«à£�Ü�¤¬�u©à�¡­ôéîê^ë0¯¡ñI�¡¤§��ç¡­���Ü � Ü4¤¬£¥áfçu«Ü��ÚÜ4«ª��®¬©ª®���á>�¬�¡£ôëÚ��硤§©à£�¤�®¬Þ�£ � � ¤§ç¡Û ��áUÛÄ���¡���¬���¡£Íéîê�ë0û¾òQ£ô­�©ª¦�£mÜ.Þ���«ª���¡��ÛÄ©èÜ4«i� ©àÛÄ£�ç¡�¡©äöáf��¤§ÛW¨¡©à®y� ¤§©à�¡çu�¬©à��� � � Ò «à£�Ü�¤¬�u©à�¡­Ü�«ª­���¤¬©ä� �¡Û áf��¤î� �u£ � «èÜ�®§®Í�4á0ÛÄ���¡���¬���¡£~� Ö�×xw y{z}|XØ�Þ öw� £¥¤¬Ûéîê�ë)áf��¤¬Û ç¡«èÜ�£����9Ù+¦4Ü�¤¬©àÜ��¡«ª£�®�¯ÌñI�u©à®<©ª®<Ü4�O£âú*Þ����¡£��"�¬©èÜ�«�©àÛÄÞ¡¤¬�v¦�£�ÛÄ£��"�.�v¦�£¥¤<Þ¡¤§£¥¦�©à��ç¡®Þ���«ª���¡��ÛÄ©èÜ4«äöw�¬©àÛÄ£�Ü�«à­���¤¬©ä� �¡ÛÄ®.©à�Ì�¬�¡©à®�ÛÄ�*¨u£�«:ûYòY�¡© � � � ��硫ਠ«à£�Ü�¤¬� ÛÄ���¡���¬���¡£úù*í{«ª��­ õ Ù@ïwö� £¥¤¬Û éîê�ëQûJÜ��u¨ ©à®Y� �¡£��Ú¤¬®y�ã � ©à£��"�(Ü�«à­���¤¬©ª�¬�¡Û áf��¤�ÛÄ���¡���¬���¡£<í{«à��­ÍÙiï}û × � Þ öw�¬£�¤§Û éîê^ë�©ª�Ü��"��Û~��¨¡£�«���á>«ª£ØÜ�¤§�¡©à�u­~áf¤¬��Û ¤¬Ü��¡¨¡��Û £ðúuÜ�Û~Þu«à£�®����¡«ª��¯^óÏ£ ®§�¡�vò �¬�ÚÜ4�(��硤�¤§£�®¬çu«ª� � Ü��!��££âú��¬£��¡¨¡£¥¨!� �.«ª£ØÜ�¤§�¡©à�¡­Ä¦4Ü�¤¬©ª��ç¡® � «èÜ�®§®¬£¥®(��áS®¬Û~Ü�«ª« � ���¡®y�øÜ��"�§öw¨¡£�Þ*� � � ©ª¤ � ç¡©ª�¬®^òY�¡© � � � ��ÛÄÞ¡çu�¬£ÛÄ���¡���¬���¡£ôáfç¡� � � ©à���¡®l���cáf£âòþ¦4Ü�¤¬©àÜ��¡«ª£�®lÜ��¡¨c� �«à£�Ü�¤¬�¡©ª�¡­�éîê�ë í{��¤ � ©à¤ � ç¡©ä� ®�Ü�®l¨¡£�® � ¤§©à��£�¨Ü����v¦�£vïìç¡�u¨¡£�¤^Ü��"� � ���¡®§� Ü��"�§öý����ç¡�u¨¡£�¨�Þ¡¤¬��¨¡ç � �^¨u©à®§�¬¤¬©ª�¡çu� ©ª���i¯� ÐD¹vÐ�Ç9¼r��¸fÇh¼�Á�¹9¸fÆ�ÂsÊóÏ£ � «à��®§£îòY©ª� �Üô�¡¤§©à£¥ás¨¡©ª® � çu®¬®¬©ª�����ásáfç*� ç¡¤§£�¤§£�®§£ØÜ�¤ � �d¨¡©ª¤¬£ � � ©ª���¡®�¯

÷

Page 22: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

� Ï µÎÍ ³���´ ü

� � ���6² µ � � Ë µ%¶ � � ´��E� � �

ý�G£ØÜ � �G��áI� �¡£<«à£ØÜ4¤¬�¡©ª�¡­dÛÄ�*¨u£�«à®lòY�¡© � �#ò0£ � ���¡®¬©ª¨¡£�¤�� �¡£<«à£ØÜ4¤¬�¡©ª�¡­ Ü�«ª­���¤§©ª� �uÛW�ÚÜ�®m®§��ÛÄ£�ý��ÞK£��4á(Ü �¥� £¥®¬®ô� � �¬�¡£�©à�¡Þuçu�§öw��çu� Þuçu�ô��£��ÚÜئ�©à��¤ ��á^Ü���ç¡�{t�¡�vòY�}æQ�*��«ª£ØÜ��Ïáfç¡� � �¬©à���JûQÜ4�¡¨� �u£î«à£ØÜ4¤¬�¡©ª�¡­õÜ�«à­���¤¬©ä� �¡ÛA± ®U­��"Ü4«i©à®0�¬� � ���¡®y� ¤§ç � �^Üô¤¬£¥Þ¡¤¬£¥®¬£��"� Ü4� ©ª���Ïí{£âúuÜ � ����¤YÜ�Þ¡Þ¡¤§�Øúu©ªÛ<Üh� £vï��ás� �¡©ª®^ç¡�v��u�9òY� áfçu� � �¬©à���i¯0ñI�¡£Í¦4Ü�¤§©à��ç¡®YÛÄ�*¨u£�«à®IòY�¡© � �dòQ£ ¨¡£¥® � ¤§©à��£Í��£�«ª�vò�¨¡©��K£�¤Y©ª��� �u£¨¡£â�øÜ�©ª«à®ì��á��¡�vò-�¬�¡£¥��©ª�¡®§� Ü��"� ©àÜ4� £(�¬�¡©à®I­�£¥�¡£�¤¬Ü�«@ÞÚÜ�¤¬Ü�¨¡©ª­�Û�¯

º�£!¢ ÉË¥�­rý�ŵþP¦qÇ_ªî­rÿ � ÅÈþr¿qÅÈÇ]ÅÈ­s¦]ªC¦q§©¥Ì­rÇÔ@£¥��� ��£îÜm®§£¥�ìòY�¡© � ��©à®Q�¬�¡£(¨¡��Û<Ü�©ª�<áf��¤U� �¡£(çu�{t�¡�vòY�áfç¡� � � ©ª���<�¬�ô�K£(«ª£ØÜ�¤§�¡£�¨Jÿuò0£î¤§£¥áf£¥¤� ����Ü4®0� �¡£yö«çvóMî£èXç�õPï�ðaäUì$èXö«çÚ¯sñU��Þ¡© � Ü�«ª«ª���åòY©à«ª«K��£(£�©ä� �¡£¥¤0� �u£læ0�*��«à£ØÜ4� � çu�K£��h������� Ø ��¤Ù�öw¨¡©àÛÄ£��u®¬©à���ÚÜ�« â ç � «ª©à¨¡£�Ü��.®¬Þ¡Ü � £<ë Ø � � õPäUç�õPï"í�î�©à®ìÜôæQ�*��«ª£ØÜ��Äáfç¡� � � ©à����µ"��� �� ��+�Ø��� �â ütç¡©ª¦4Ü�«ª£��"� «ä��ò0£�òY©à«à«@®§��ÛÄ£¥� ©ªÛ~£¥®I¦*©ª£¥ò�Ü � ��� � £�Þu���lÜ�®(ÜÄ®¬ç¡�¡®§£¥�î�4á��Æ�¾�ÚÜ�ÛÄ£�«ä���!�����S"�4íD�KïÈW ��� � � õPäUç�õPï"í�îAõMé èUóPó�©à®�Ü�®¬£¥���}��á � ��� � £�Þu�¬®�¯s ý�c£ØÜ � �'��áQ� �¡£õ«à£�Ü�¤¬�u©à�¡­�® � £��ÚÜ4¤¬©à��®òQ£ � ���¡®¬©ª¨¡£�¤î�¬�¡£�¤§£~©ª®lÜ��Ïçu�{t�¡�vòY� � ��� � £¥Þu������� � Ü�«ª«à£¥¨��¬�¡£�î=èXæÞ�aïMîIõPäUç�õPï"í�î�òY�¡© � �Ï� �u£Ü�«ª­���¤¬©ä� �¡Û&©à®I� ¤y��©à�¡­õ� �Ä«à£�Ü�¤¬��áf¤¬��Û «àÜ���£�«à£¥¨�£ðúuÜ�Û~Þu«à£�®¥¯ìñI�¡£�©à�"� çu©ª� ©ª����©à®I� �¡Ü4�^�¬�¡£ � «èÜ4®¬®��©à®xt�¡�vòY�� �ô� �¡£l«ª£ØÜ�¤§�¡©à�¡­õÜ4«à­���¤§©ª�¬�¡Û&�uçu�I� �¡£l©ª¨¡£��"� ©ä�ý����á������c©à®Y�¡�4�د¢*©à� � £Í�¬�¡£�¤§£ôÜ�¤§£ÍÛ~Ü��"�¨¡©��K£�¤§£��"��ò0ÜØ�*®^� �.¤§£�Þ¡¤§£�®¬£¥�"�lÜ��"�d­�©ª¦�£¥�!æ0����«à£�Ü���áfç¡� � �¬©à���Jû�òQ£

� ���¡®¬©ª¨¡£�¤:æPï"í�æRïPó8ïMç1î£èXî¶ö£äUç±õMé èUóPó8ïPóôÜ�®~òQ£�«ª«�Ü�® � ��� � £¥Þu� � «èÜ�®§®¬£�®¥¯+�^®.Ü���£âúuÜ�ÛÄÞ¡«à£�� �Ï�¡£�«ªÞ� «àÜ�¤¬©äáº�'�¬�¡£�¨¡©à®y� ©à� � � ©ª���G��£¥�ýòQ£�£��OÜ � ��� � £¥Þu� � «èÜ4®¬®~Ü��¡¨�Üc¤§£�Þ¡¤§£�®¬£¥�"�øÜ4�¬©à��� � «àÜ�®¬®¥ûQòQ£��¡�vò¨¡£M�Ú�¡£�� �¡£ � «èÜ�®§®ô��á�éWö«ç�ï¸èXæ5î«ÿaæRïPóPÿ1äUé ð]ê-åaç�õMî¶ö£äUçvó�òY�¡© � ��Þu«èÜØ��®õÜ��G©àÛÄÞK��¤§�øÜ4�t� ¤¬��«ª£<©ª�#� �¡©ª®� �u£�®¬©ª®�¯lëÚ��¤�� W��h���Ø��� Ø ��¤��SW×ë Ø Ü � ��� � £�Þ*���î"�� � �� Í�G�Ø���õ©à®�Ü<«à©ª�¡£ØÜ4¤��¬�¡¤¬£¥®¬�¡��«ª¨áfç¡� � � ©ª���'©äá0� �u£�¤¬£<£âú*©à®y�"!#��ë Ø ��$%��ëß®¬ç � �#�¬�ÚÜ4�&�4íD�KïTWW�<©äáIÜ��¡¨'���¡«ª��©äá�!('+�*)+$ �

�v�

Page 23: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

óÏ£�® ÜØ�Ä� �ÚÜh�Q�¬�¡£(ÞÚÜ�©ª¤^í,!V�-$�ï.�rë Ø0/ ë æRï"í�æRïPó8ïMç1îÝó>�¬�¡£�«ª©à�¡£�Ü�¤S� �¡¤§£�®¬�u��«à¨Äáfç¡� � �¬©à���1� � ñI�tç¡®� �u£ � ��� � £¥Þu� � «àÜ�®¬®Q��á@«ª©à�¡£�Ü�¤U�¬�¡¤¬£¥®¬�¡��«à¨<áfç¡� � � ©ª���¡®U�v¦�£�¤2�å©à®QÜ ®¬£â�ì��á@æ0�*��«à£ØÜ4�~áfç¡� � � ©ª���¡®ìÜ4®¨¡£¥® � ¤§©à��£�¨!Ü4�K�v¦�£4ûJÜ4�¡¨�¬�¡£ ¤§£�Þ¡¤§£�®§£��"�øÜ4�¬©à��� � «àÜ�®¬®���á6«ª©à�¡£�Ü�¤ì� �u¤¬£�®§�¡��«ª¨�áfç¡� � � ©ª���¡®��9¦�£�¤�� ©ª®� �u£�®§£¥�0ë Ø / ë ����"�Í«à£ØÜ4¤¬�¡©ª�¡­(Ü�«ª­���¤§©ª� �uÛ+òY�¡© � �ô��çu� Þuçu� ®s¤¬£�Þu¤¬£�®§£��"�øÜh� ©à���¡®>�K£¥«à���u­�©à�¡­Y� ��Ü � «àÜ�®¬®43 Ûmç¡®y�

¤¬£¥üt硩ध£ôÜ4�(«ª£ØÜ�®y�î£��u��ç¡­�� �¬©àÛÄ£��¬�.òY¤¬©ª�¬£ ¨¡�vòY�cÜ���Ü�ÞuÞ¡¤¬��Þu¤¬©èÜh� £�¤¬£�Þu¤¬£�®§£��"�øÜh� ©à���c©ª�53 � ëÚ��¤6 �53 ò0£�¨¡£��¡�¡£�®§©à÷�£�í 6 ï6� ���K£Y� �¡£�¨¡£¥® � ¤§©àÞu�¬©à���īࣥ�¡­��¬�~��á 6 ç¡�¡¨¡£¥¤Q®§��ÛÄ£��*úu£¥¨.¤¬£�Ü�®¬���ÚÜ��¡«ª££�� � ��¨¡©à�¡­Ï® � �¡£¥Û~£4¯ï wá�3 ©ª®<Üc¤¬£�Þu¤¬£�®§£��"�øÜh� ©à��� � «èÜ4®¬®~òY�u© � �O©ª®<Ü�®§®¬� � ©àÜ4� £¥¨}òY©ª�¬� � ��� � £¥Þu�� «àÜ�®¬®7�q�S� �¡£¥�}áf��¤8�9�:� òQ£d¨¡£M�Ú�¡£ ®§©à÷�£�í;�¥ïõ�¬�#�K£� �¡£�Û~©ª�¡©àÛ ç¡Û ��ál®§©à÷¥£"í 6 ïÄÜ � ¤§��®¬®<Ü�«à«¤¬£¥Þ¡¤¬£¥®¬£¥�t� Ü4� ©ª���¡® 6 �53 ��á�� �¡£ � ��� � £¥Þu��� �ñI�¡£��ýò0� � ��� � £�Þu� � «àÜ�®¬®§£�®õòY�¡© � ��òQ£d®§�¬ç¡¨u��ÛÄ��®§�ôáf¤§£�ütç¡£¥�t�¬«ª��©ª���¬�¡©à®ô�¬�¡£�®§©à®~Ü�¤¬£�� �u£

� «àÜ�®¬®S��áJ«à©ª�¡£ØÜ�¤s� �¡¤§£�®¬�u��«à¨õáfç¡� � � ©à���¡®Q¨u£�® � ¤¬©ª�K£¥¨<Ü����v¦�£^Ü��¡¨Ä� �¡£ � «àÜ�®¬®S��áhðXöwó=<-å�ç�õMîcö?>Xï�ç�äUæ-ì$èXéê@äUæ-ìMáf��¤§Ûmç¡«èÜ4£���¤(éîê�ëQ¯��Méîê�ë�áf��¤¬Ûmçu«èÜõ©à®^Ü<¨¡£�Þ*� ���~çu������ç¡�u¨¡£�¨ á{Ü��u©à� � ©à¤ � ç¡©ä��òY�u© � �� ��Û~Þ¡ç*� £�®�Ü���ûî�-��ás�(ê^él¢��4á6æ0����«ª£ØÜ���«à©ä� £¥¤ Ü�«ª®�¯sóÏ£l¤¬£âáf£�¤I� �ô� �u£���ê�é)­"Ü4�¬£�®Y��á�Ü~éîê^ëÜ�®jî7ïMæ-ì�óâ¯S¢��¬¤¬© � � «ä�õ®¬Þ�£ØÜXt©ª�¡­Í�¬�¡£ � «èÜ�®§®0��áDéîê^ëcáf��¤§Ûmç¡«àÜ�£Y©à®QÜͤ¬£¥Þ¡¤¬£¥®¬£¥�t� Ü4� ©ª��� � «èÜ�®§®0¤ Üh� �¡£¥¤� �¡Ü��GÜ � ��� � £�Þu� � «èÜ�®§®�ÿS�¡�vòQ£¥¦�£¥¤mò0£ÄòY©à«ª«U®§��ÛÄ£¥� ©ªÛ~£¥®�Ü��¡ç¡®§£.�¡��� Ü4� ©ª���'Ü4�¡¨#¤¬£âáf£�¤�� �!éîê^ëÜ�®�Ü � ��� � £�Þu� � «àÜ�®¬®¥¯Uê^���¬£l�¬�ÚÜ4��®¬©ª� � £�Ü��"��æQ�*��«ª£ØÜ���áfç¡� � � ©ª�����ÚÜ�®Y®¬��Û~£l¤¬£¥Þ¡¤¬£¥®¬£¥�t� Ü4� ©ª��� Ü4®Ü éîê^ë�áf��¤§Ûmç¡«èÜÏí{Þ���®§®¬©à�u«ª��¤¬£¥üt硩ध©à�¡­�£âú*ÞK���¡£��"� ©àÜ�«à«ä��Û<Ü4�t�!�¬£�¤§Û~® ïl� �u£ � ��� � £¥Þu� � «àÜ�®¬®Í��áéîê�ë#©ª®0� �u£ � «èÜ�®§®I��ásÜ�«à«Jæ0����«à£�Ü��<áfç¡� � � ©ª���¡®�¯q ý���¬�¡£ � ���"� £ðú��^�4á�éîê^ë'� �¡£(¦4Ü�«ªç¡£î��á�®¬©ª÷�£�í;�¥ï©à®S®y�øÜ��¡¨¡Ü�¤¬¨¡«ä�ô¨¡£M�Ú�¡£�¨~� ���K£I� �u£^ÛÄ©à�¡©ªÛmç¡Û��tç¡Ûm��£�¤U��áJ�¬£�¤¬ÛÄ®S©à�ÄÜ��"�Äéîê�ë!áf��¤§Ûmç¡«èÜ�òY�u© � �¤¬£¥Þ¡¤¬£¥®¬£¥�t�¬®^�¬�¡£ � ��� � £�Þu�@� � ó'£îòY©ª«à«J®§��ÛÄ£¥� ©ªÛ~£¥®ì¤¬£âáf£�¤I� �ô�¬�¡£ � «èÜ4®¬®Y��á«á&Ñ"î"ïMæ-ì+A�B�CSÿu�¬�¡©à®I©ª®� �u£(®¬£â�I��ásÜ�«à«�æ0����«ª£ØÜ��<áfçu� � �¬©à���¡®UòY�¡© � � � Ü�����£(£âú*Þ¡¤§£�®¬®§£�¨dÜ�®ìéîê^ëÏáf��¤¬Û ç¡«èÜ�£YòY�¡© � ��¡Üv¦�£Ü4�YÛÄ��®§�0ál� £�¤§Û~®¥¯

º�£»º DFEÊÉ GuŵªV¿ ­r§^­IHñI�¡£ � ¤¬���¡Ü��¡«ª�^��ÞuÞ¡¤¬�Øú*©àÛ~Ü4� £¥«ª� Ò ��¤¬¤§£ � �Qí � � Ò ïi«ª£ØÜ�¤§�¡©à�¡­0Û~��¨¡£�«hòìÜ4®�©ª�"� ¤¬��¨¡ç � £�¨l�"�Ö�UÜ�«ª©èÜ��"�©à�!íD�UÜ�«ª©èÜ��"�Øû¾�8÷G�XátïQÜ4�¡¨��ÚÜ�®ì®§©à� � £î�K£¥£��.òY©ª¨¡£�«ä�<®y� ç¡¨¡©ª£�¨i¯q ý�.�¬�¡£ � � Ò ÛÄ�*¨u£�«¾� �¡£î«ª£ØÜ�¤§�¡©à�¡­Ü�«ª­���¤¬©ä� �¡Û�©ª®�­�©ä¦�£¥�õÜ ��� £�®¬®>�¬�îÜ���ïKJ/èXì�í�éwï�äUæ¸èaõMéwï�L0��í;���NM~ïDÜ�®6©ª�¬®6®¬��硤 � £I��ᾫèÜ���£�«ª£�¨m£ðúuÜ�ÛõöÞ¡«ª£�®�¯2�^£�¤§£"�0�I�#©ª®I� �¡£l�øÜ�¤§­�£¥� � ��� � £¥Þu�îÜ4�¡¨�M ©à®�Ü��dçu�{t�¡�vòY� Þ¡¤¬���ÚÜ��¡©ª«à©ª�ý�<¨¡©à®y� ¤§©à�¡çu�¬©à����v¦�£¥¤(� �¡£m©ª�¡®§� Ü�� � £ô¨¡��Û~Ü�©à�O� � ñI�u£ô��¤ Ü � «à£"L���í,�l�NM~ïI�øÜX4£�®î�¡��©ª�¡Þ¡çu�¬®�ÿiòY�¡£¥��©à�"¦��`4£�¨�©ª�¤¬£â� ç¡¤§�¡®~ÜcÞÚÜ4©à¤\�(�O�P�4íD�Kï � òY�¡£¥¤¬£��:�Q� ©à® � �¡��®§£��ÌÜ �¥� ��¤§¨¡©à�u­c�¬��M � ñI�¡£�¦4Ü�«àçu£��4íD�Kïm©ª®¤¬£âáf£�¤§¤¬£�¨��¬�õÜ�®ì� �¡£yé è&ÐMïMé"�4ás©ª�¡®§� Ü�� � £µ�O�ÚÜ��¡¨.�¬�¡£îÞÚÜ�©ª¤È�(�O�P�4íD�Kï � Ü�®IÜ^é è&ÐMïMéwï¸ð�ïKJ/èXì�í�éwïâ¯�ñI�¡£

���

Page 24: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

� ��«à«à£ � �¬©à�����áS«èÜ4�K£¥«à£�¨�£âúuÜ�ÛÄÞ¡«à£¥®^òY�¡© � �!Ü<«ª£ØÜ�¤§�¡©à�¡­<Ü�«à­���¤¬©ä� �¡Û%¤§£ � £�©ª¦�£�®�áf¤§��Û#L���í,�l�KM~ïY©ª®��áº�¬£��¤¬£¥áf£¥¤¬¤§£�¨�� �ÄÜ�®�Ü5ó�èXì�í�éwïâ¯ëÚ��¤ 6 �53%òQ£^®¬ÜØ�Ä�¬�ÚÜ4�U� �¡£yïMæ-æRäUæjæèXî"ïS��á 6 ©ª®>� �¡£�Þ¡¤¬���ÚÜ��¡©ª«à©ª�ý��íf� ÜX�£¥�~òY©ª�¬�~¤§£�®§ÞK£ � �0�¬�M~ïQ� �¡Ü4�Y� �¡£îáfç¡� � � ©ª����¤¬£�Þu¤¬£�®§£��"� £¥¨ �"� 6 ¨¡©ª® Ü�­�¤§£�£¥®YòY©ä� �O���Ú©:¯ £�¯>£�¤§¤¬��¤Øí 6 ï W � ¤ �SRUT.V 6 íD�Kï�WW�4íD�KïYX � ê���� £�� �ÚÜh�ì� �¡£^£�¤§¤¬��¤ì¤¬Ü4� £^��á 6 ©à®QÛ~£�Ü�®¬ç¡¤§£�¨.òY©ä� �.¤§£�®¬Þ�£ � �I� � �¬�¡£(®¬Ü�Û~£^¨¡©à®y� ¤§©à�¡çu�¬©à���Mùáf¤¬��Û òY�u© � �«èÜ���£�«ª£�¨�£âúuÜ�ÛÄÞ¡«ª£�®�Ü�¤¬£l¨¡¤¬ÜØòY�iÿ¡� �tç¡®�©ªásÜõÞK��¤y� ©ª������á�� �¡£l©à�u®§�øÜ4� � £l®¬Þ¡Ü � £0�

¤¬£ � £¥©ª¦�£¥®^«ª©ª�§� «à£���¤I�¡�mòQ£�©ª­��"��ç¡�¡¨¡£¥¤�M��ÚÜô�t��Þ���� �u£�®¬©ª® 6 � Ü4�£¥¤¬¤I�����¬�¡©à®ìÞ���¤y� ©à������áZ� Ü4�¡¨®§�¬©à«ª«iÜ � �¡©ª£¥¦�£Í«ª�9ò+£�¤§¤¬��¤Y�v¦�£¥¤ Ü�«ª«:¯� � � Ò «à£�Ü�¤¬�u©à�¡­'Ü�«à­���¤¬©ª�¬�¡Û òQ��¤¸t®��"�O¨¡¤¬ÜØòY©à�¡­GÜ#®¬Ü�ÛÄÞ¡«à£áf¤¬��Û � �u£!£âúuÜ�ÛÄÞ¡«à£���¤ Ühö

� «ª£1L0��í;���NM~ï��DÞ�£�¤yáf��¤¬ÛÄ©à�¡­®¬��Û~£ � ��ÛÄÞ¡çu� Ü4� ©ª���'���Ï� �¡©ª® ® Ü4Û~Þ¡«ª£�û�Ü��¡¨'�¬�¡£��G��ç*� Þ¡çu�¬©à�¡­dÜ�"��ÞK���¬�¡£�®§©à® 6 òY�u© � �#��£�«ª���¡­�®î� ��®¬��ÛģĤ¬£�Þu¤¬£�®§£��"�øÜh� ©à��� � «àÜ�®¬®�3 � ñI�u£<­��"Ü4«U�4áYÜ�«ª£ØÜ�¤§�¡©à�¡­Ü�«ª­���¤¬©ä� �¡Û ©à®S� �m��çu� Þ¡ç*�ìÜ �"��Þ���� �¡£¥®¬©ª® 6 òY�¡© � �Jû*òY©ª�¬�<Þ¡¤§���ÚÜ��u©à«à©ä�ý�.�[ 5\�û*�ÚÜ�®0£¥¤¬¤§��¤ví 6 ï.]�ô �ñI�¡£S¦4Ü�«àç¡£^\I©à®D¤§£¥áf£�¤§¤¬£¥¨Í� �^Ü�®D� �¡£AõPäUçS_LðaïMç�õPïJÞÚÜ�¤¬Ü�Û~£â� £¥¤sÜ4�¡¨îôsÜ�®@� �¡£�èaõPõMå�æèaõMò>ÞÚÜ4¤ Ü�ÛÄ£¥�¬£�¤�¯��®ìÛÄ���¬©ª¦4Ü4� ©ª���ôáf��¤Q�¬�¡©à®Q�§Þ¡¤§���ÚÜ��¡«ä�ÄÜ�Þ¡Þ¡¤§�Øúu©ªÛ<Üh� £�«ä� � ��¤§¤¬£ � �R�ô®§ç ��� £�®§® � ¤¬©ä� £�¤§©à���JûtòQ£î���*ö

®¬£¥¤§¦�£ � �ÚÜ4�~� �¡£¥¤¬£dÜ�¤¬£�ýò0�G®§��硤 � £�®.�4ál¨¡©ªã � çu«ª�ý��òY�¡© � � � Ü�� � ���uáf��çu�¡¨ Ü � � Ò «ª£ØÜ�¤§�¡©à�¡­Ü�«ª­���¤¬©ä� �¡Û¯�¢*©ª� � £�� �¡£Í®¬��硤 � £&L0��í;���NM~ïI��áS«èÜ4�K£¥«à£�¨£âúuÜ�ÛÄÞ¡«ª£�®�©ª®�Þ¡¤§���ÚÜ��u©à«à©ª®§�¬© � ûÚ©ä��©ª®^Þ���®_ö®¬©ª�¡©à«ª£^�¬�ÚÜ4�YÜm«ª£ØÜ�¤§�¡£�¤©± ®I®¬Ü�Û~Þu«à£^òY©ª«à«��K£î�u©à­��¡«ä�Äç¡�¡¤¬£¥Þ¡¤¬£¥®¬£¥�t� Ü4� ©ä¦�£���áD� �¡£(� Ü�¤¬­�£¥� � ��� � £¥Þu�ØÿÚ©äá� �u©à®l£¥¦�£��"�íf�¬�¡£ÄÞ¡¤¬���¡Ü��¡©à«ª©ª�ý�d��á0òY�¡© � �'©ª® � ���"� ¤¬��«à«à£¥¨��t���¬�¡£8\�ÞÚÜ4¤ Ü�ÛÄ£¥�¬£�¤øï(®§�¡��ç¡«ª¨'� ��� çu¤� �u£��.� �u£(¤¬£¥®¬ç¡«ä� ©à�u­m�"�*Þ����¬�¡£�®§©à®IÛÄ©à­��"�Uò0£¥«à«J�¡Üv¦�£î«èÜ�¤§­�£�£¥¤¬¤§��¤�¯�û(�<�¬�¡£(���¬�¡£�¤ì�¡Ü��¡¨iû¡£â¦�£¥�©äá� �u£ «à£�Ü�¤¬�u©à�¡­~Ü�«à­���¤¬©ä� �¡Û%¨u¤ ÜØòY®(ܺ�_�ý�*Þu© � Ü4«w�Ä® Ü�ÛÄÞ¡«à£�©ª��Û~ÜØ���u������£ Þ���®§®¬©à�u«à£l� �.«ª£ØÜ�¤§��� �u£�øÜ4¤¬­�£â� � ��� � £�Þu�U£ðú¡Ü � � «ä��¯6ñI�tç¡®SòQ£�¤§£�ütç¡©ª¤¬£Y���¡«ä� � �¡Ü4�>� �¡£Y«ª£ØÜ�¤§�¡©à�¡­(Ü�«à­���¤¬©ª�¬�¡ÛA± ®s�t��Þ���� �u£�®¬©ª®��£ÍÜ4Þ¡Þ¡¤¬�Øú*©àÛ~Ü4�¬£�«ª� ígÜ4®�ÛÄ£ØÜ4®¬ç¡¤§£�¨�t��ôøï � ��¤§¤¬£ � �دó'£~�ÚÜئ�£Ä� �¡£õáf��«à«ª�vòY©à�¡­.áf��¤§Û<Ü4«>¨¡£��Ú�u©ª� ©ª���c��á � � Ò «à£ØÜ4¤¬�¡©ª�¡­�òY�¡© � �#©ª®�Ü�¨ÚÜ�Þu�¬£�¨cáf¤¬��Û

í¶ßî£�Ü�¤¬�¡®�Ü��u¨A�UÜ�÷¥©à¤¬Ü��¡©:ûK�8÷`÷Xá"ï�"��¼a`6Â�¸f¹v¸{Æ� �cb( �\ïMîc��ÐMï�è�õPäUç�õPï"í�î[õMé èUóPóQäd>XïMæ.� èXç�ð]éwïMî43 ÐMï�è�õPäUæ-æRïPó"í1äUç�ðXö«ç��]æRï"í�æRï�Ñó8ïMç1î£èXî¶ö£äUç õMé èUóPó¬íVé�çºèXé �aäUæ-ö«î«ÿaì ��öwósó�èXöcð�î"ä\ÐMïyè � � Ò «à£�Ü�¤¬�u©à�¡­�Ü4«à­���¤§©ª�¬�¡Û%áf��¤��Gç¡®¬©ª�¡­3 ö ê$îÝÿ1ï�ê@äUé«éwädehö«ç���ÿ1äUé ðUógf�ê@äUæ5èXé«és�%hÕô¬�P\�h �+��ê@äUæCèXé«éi��j�«�gê@äUæ5èXé«é[ðXöwóMîcæ-ö(Ð�å�îcö£äUçvó0Mäd>XïMæ.�P��ö ê2� öwó��Xö?>XïMçþèUóIö«çXí�åaîLèXçþèaõPõMåaæèaõMò�ívèXæ¸èXì5ïMî"ïMæeôQèXç�ðCè�õPäUçS_LðaïMç�õPï�ívèXæèXì5ïMî7ïMæ�\èXç�ðT� ÿvèUó�èaõPõPïPóPóAî7äyîÝÿ1ïsïKJ/èXì�í�éwï�äUæèaõMéwïkL0��í;���NM~ï���îÝÿ1ïMçIehö«î«ÿ�í�æRäGÐPè&Ð�ö«éWö«îcòCèXîhéwï¸èUóMîU�2 l\èXé �aäUæ-ö«îÝÿaì×� äUåaî+í�åaîÝóQè$ÿaòPí1äUîÝÿ1ïPóMöwó 6 �53�óMå1õPÿ�îÝÿvèXî � ¤ �SRUT2V 6 íD�Kï0WWm�4íD�KïnXZ]ñô � wá«�)¤§ç¡�¡®�©à��� ©àÛÄ£.U�íDÙ]� ®§©à÷¥£"í;�¥ï�����ìGô¬�Ø�!ì�\�ïQáf��¤�®§��ÛÄ£E�*úu£¥¨dÞ���«ª���¡��Û~©àÜ�«FU@û¡òQ£�® ÜØ��� �ÚÜh�0�)©ª®Üsí1äUéWòUç�äUìyöcèXégîcö«ì5ïµÚtéQë�éwï¸èXæMç1ö«ç��$èXé �aäUæ-ö«îÝÿaìô¯�� � ��� � £�Þu� � «èÜ�®§®2�c©à®Q® Ü�©ª¨.� �ô��£CïDÏdõMö£ïMç1î¶éWòÚtéQë¤éwï¸èXæ-ç�è&Ð�éwï0©ªáD� �u£�¤¬£l£ðúu©ª®§�¬®�ÜôÞ���«ª���¡��ÛÄ©èÜ4«��¬©àÛÄ£ � � Ò «ª£ØÜ�¤§�¡©à�u­õÜ�«à­���¤¬©ª�¬�¡Ûßáf��¤.� �

�9�

Page 25: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

ý�ôÜ�¨¡¨u©ª� ©ª���l� �^�¬�¡£Q�¬©àÛÄ£ � ��ÛÄÞ¡«à£ðú*©ª�ý�l��áÚ«ª£ØÜ�¤§�¡©à�u­�Ü�«ª­���¤§©ª� �uÛ~®JòQ£0òY©ª«à«tÜ�«ª®¬����£ � ��� � £�¤§�¡£�¨òY©ª�¬�.� �¡£î�tç¡Û �K£¥¤ì��á�£âúuÜ�ÛÄÞ¡«à£¥®0� �u£¥�.¤§£�ütç¡©ª¤¬£�¯�Ô@£¥��o¡íDÙ]�»á+�Ø��ì`ô¬�Ø��ì�\4ïU¨u£��¡���¬£(� �¡£�Û<Ühú*©ªÛmç¡Û�tç¡Ûm��£�¤>��á � Ü�«à«ª®s�¬�0L���í,�l�NM~ï@òY�¡© � �Q�O£â¦�£¥¤UÛ~ÜX4£�®6áf��¤>Ü4�t���øÜ4¤¬­�£â� � ��� � £�Þ*�.�@�����á�®¬©ª÷�£eáÜ��¡¨.Ü��"�<¨¡©ª®§�¬¤¬©à�uçu� ©ª����M � ñI�¡£^¦�Ü4«àç¡£@o¡íDÙ]�»á+�Ø��ì`ô¬�Ø��ì�\4ï6©à®�t�¡�vòY��Ü4®Q� �u£�ó�èXì�í�éwï�õPäUì�í�éwïKJ`ö«î¶ò��áO� � ñI�¡£î®¬Ü�Û~Þu«à£ � ��ÛÄÞ¡«à£ðú*©ª�ý�Ä��ásÜ � ��� � £¥Þu� � «àÜ�®¬®p��©à®U� �¡£îÛÄ©à�¡©ªÛmç¡Ûþ®¬Ü�ÛÄÞ¡«à£ � ��ÛÄÞ¡«à£ðú*©ª�ý�Ü � ¤¬��®§®^Ü4«à«i«ª£ØÜ�¤§�¡©à�¡­õÜ4«à­���¤§©ª�¬�¡Û~®Uáf��¤.� �º�£cÒ q ªî¿ §¬ªC¦«§¬¥�­rÇ ªV­rÿ rts�¦«ÅÈ­rÇ«§©¥Ì­rÇ°¥QÁð¦«ÄÆÅ DFEÊÉ ¤ ¥�ÿÆÅÈÔ

��«ª­���¤§©ª� �uÛ~®IòY�¡© � �!® Ü4�¬©à®§áº��é(£M�Ú�¡©ª�¬©à��� �*¯ä�ôÜ�¤¬£Í®¬��ÛÄ£¥�¬©àÛÄ£�®^¤¬£¥áf£¥¤¬¤§£�¨ � �.Ü�®sóMîcæPäUç���éwï¸èXæ-ç1ö«ç��èXé �aäUæ-ö«îÝÿaì�ó�®¬©ª� � £ì�¬�¡£¥� � Ü��~Ü � �¡©à£â¦�£�Ü��~Ü�¤¬�u©ª� ¤¬Ü�¤¬©ª«ª�l«ª�9ò�£�¤§¤¬��¤S¤¬Ü4� £eô>òY©ª�¬�Ä�u©à­��õÞ¡¤§���ÚÜ��¡©ª«à©ä�ý��¯� ÛÄ��¤¬£^¤¬£¥«èÜhú*£�¨��u��� ©ª���.��á�«à£�Ü�¤¬�u©à�¡­s��u�9òY�Ü�®ue[ï¸èdvyéwï¸èXæ-ç1ö«ç��^òìÜ4®ì©à�"� ¤§�*¨uç � £¥¨�"�5ßî£ØÜ�¤§�¡®Ü��¡¨<�UÜ�«ª©èÜ��"�Uícßl£�Ü�¤¬�u®�Ü��u¨T�UÜ�«ª©èÜ��"��û"�8÷`÷Xátïð¯D��òQ£ØÜ�î«à£ØÜ4¤¬�¡©ª�¡­�Ü�«à­���¤¬©ª�¬�¡Û � Ü��u�¡�����¡£ � £�®§® Ü�¤§©à«ª�Ü � �¡©ª£¥¦�£�Ü���ôyöýÜ ��� 硤 Ü4�¬£!�"�*Þ����¬�¡£�®§©à®<áf��¤<Ü�¤¬�u©ª� ¤¬Ü�¤¬©ª«ª�G®§Û<Ü4«à«0ô¬�0�¡ç*��©à®<Ü��¡«à£� �#­�£��¡£¥¤ Ü4�¬£!Ü�"��ÞK���¬�¡£�®§©à®(òY�¡��®¬£ô£�¤§¤¬��¤�¤ Ü4�¬£ ©à®��¡��� © � £�Ü��¡«ª�«ª£�®¬®^� �ÚÜ��G�!ì�� � Ñ ��¤¬£ Þ¡¤¬£ � ©à®§£�«ä��û#�þ©ª®(Ü<òQ£ØÜX«à£�Ü�¤¬�u©à�¡­�Ü�«à­���¤¬©ª�¬�¡Û ©äá¡� �¡£¥¤¬£I©ª®s®§��Û~£0ÞK��«ä���¡��ÛÄ©èÜ�«^U�íDÙ]� ®¬©ª÷�£"í,��ï§ï�®§ç � �ô� �ÚÜ4�sáf��¤6Ü4�t�"�����Ü4�¡¨Ü��"�~¨¡©à®y� ¤¬©ª�¡çu�¬©à���7M��t©ªá?�+©à®U­�©ª¦�£��\�w}�mÜ��¡¨�Ü ��� £¥®¬®0�¬�&L���í;���NM~ï��"� �u£��<òY©ä� �.Þ¡¤§���ÚÜ��u©à«à©ä�ý�Ü4�^«à£�Ü�®§� �@ �\ÄÜ�«à­���¤¬©ª�¬�¡Û_� ­�£��¡£¥¤ Ü4�¬£�®�Ü<�"�*Þ����¬�¡£�®§©à® 6 ®¬ç � � � �ÚÜ4� � ¤ �SRUT2V 6 íD�Kï�WWx��í �KïYX[]��ì��u Ì�!ì�U�í(ÙI� ®§©à÷¥£"í;�¥ï¬ï � w�m©ª® £ØÜ4®§�c� � ®§£�£<�¬�ÚÜ4�ôÜ4�t� � ��� � £¥Þu� � «àÜ�®¬®�òY�¡© � �'©à® ®y� ¤§���¡­�«ä� � � Ò «à£ØÜ4¤¬�ÚÜ��u«à£õ©à® Ü�«à®§�

òQ£ØÜXt«ª� � � Ò «à£ØÜ4¤¬�ÚÜ��u«à£�¯ã ý� Ü � £�«ª£��¡¤¬Ü4� £¥¨ ¤¬£¥®¬ç¡«ä��¢ � �ÚÜ�Þ¡©ª¤¬£.í:¢ � �ÚÜ4Þ¡©à¤§£�ûD�8÷`÷���ïI®§�¡�vò0£¥¨ � �ÚÜh�� �u£ � ���"¦�£�¤¬®§£I�¡��«ª¨¡®6Ü�®�ò0£¥«à«Þ"�Ü��"� � ��� � £¥Þu� � «àÜ�®¬®�òY�¡© � �ô©ª®s£âã � ©à£¥�t�¬«ª��ò0£�ÜXt«ª� � � Ò «à£�Ü�¤¬�¡Ü��¡«à£©à®0Ü�«à®§� £âã � ©à£¥�t�¬«ª�~®§�¬¤¬���¡­�«ª� � � Ò «à£�Ü�¤¬�ÚÜ4�¡«à£4¯>¢ � �ÚÜ4Þ¡©à¤§£G± ®IÞ¡¤¬����áiò0Ü�®I�¡©ª­��¡«ª� � ���¡®y� ¤¬ç � � ©ä¦�£î©ª�� �¡Ü4�l�¡£ô­"Üئ�£ÄÜ�ÐMä8ä8óMîcö«ç��õÜ4«à­���¤§©ª�¬�¡Û òY�¡© � � � Ü4�Ï��£ôç¡®¬£¥¨c�¬� � ���¡®y� ¤§ç � �ÍÜ�®§�¬¤¬���¡­�«ª£ØÜ�¤§�¡©à�¡­Ü�«ª­���¤¬©ä� �¡Ûåáf¤§��Û Ü��"�#òQ£ØÜ�#«à£�Ü�¤¬�¡©ª�¡­�Ü�«ª­���¤¬©ä� �¡Û ç¡®§©à�¡­!�¬�¡£�ò0£�ÜX#«à£�Ü�¤¬�u©à�¡­�Ü�«ª­���¤§©ª� �uÛ Ü4®Ü�®¬ç¡�u¤¬��çu�¬©à�¡£4¯$û�¦�£�¤l� �u£ôÞÚÜ�®§��¨¡£ � Ü�¨u£~���*��®§� ©ª�¡­��ÚÜ�®î��£ � ��ÛÄ£~Ü�Û~Ü9ݧ��¤(¤§£�®§£ØÜ�¤ � �Ï� ��Þ¡© � ©ª�� ��Û~Þ¡ç*�øÜ4�¬©à���ÚÜ4«�«ª£ØÜ�¤§�¡©à�u­��¬�¡£���¤y��¯�óÏ£ì®y� ç¡¨u� Ü��¡¨ � ���¡®y� ¤¬ç � �6òQ£ØÜXÍ«à£�Ü�¤¬�¡©ª�¡­^Ü�«à­���¤¬©ª�¬�¡ÛÄ®sÜ4�¡¨���*��®y� ©ª�¡­õÜ�«à­���¤¬©ä� �¡ÛÄ®0©ª� Ò �¡Ü�Þu� £¥¤¬®�ü*û��õÜ��¡¨:÷u¯ ý�'Ü4¨¡¨¡©ª�¬©à���!�¬���¬�¡£õ®§� Ü��¡¨ÚÜ�¤§¨ � � Ò �¡���¬©à���u®��4áQòQ£ØÜX�Ü4�¡¨Ï®y� ¤§���¡­«à£ØÜ4¤¬�¡©ª�¡­�¨¡£�® � ¤§©à��£�¨

Ü����v¦�£�û¡òQ£îòY©à«ª«iÜ�«ª®¬� � ���¡®¬©ª¨¡£�¤Y®¬£â¦�£¥¤ Ü�«@òQ£�«à« öý®y� ç¡¨¡©ª£�¨.¦4Ü�¤§©èÜ��"� ®I�4á��¬�¡£ � � Ò «à£ØÜ4¤¬�¡©ª�¡­mÛÄ�*¨¡£¥«:¯ ý� Ò �ÚÜ�Þ*� £�¤§®�ø*û{úôÜ4�¡¨!�v�ÍòQ£ � ���u®¬©à¨u£�¤IÜm¨¡©ª®§�¬¤¬©à�uçu� ©ª���*öý®§ÞK£ � © � � ¦�£�¤§®¬©à������á@� �¡£ � � Ò ÛÄ��¨¡£�«©à�mòY�u© � �õ� �¡£I«ª£ØÜ�¤§�¡©à�¡­(Ü�«à­���¤¬©ª�¬�¡Û)©à®6���¡«ª� ¤¬£�ütç¡©ª¤¬£¥¨õ�¬�l®¬ç �¥� £�£¥¨~áf��¤�®¬��ÛÄ£;�uú*£�¨Ä¨¡©à®y� ¤§©à�¡çu�¬©à������7� íf�ý��Þ¡© � Ü�«à«ä� �¬�¡£Yç¡�¡©ªáf��¤¬ÛM¨u©à®§�¬¤¬©ª�¡çu� ©ª���¾ïð¯I ý� Ò �ÚÜ�Þu�¬£�¤Í�îò0£Y®§�¬ç¡¨u�ÄÜ��~Ü�ç¡­�Û~£¥�t�¬£�¨ � � Ò

�8Ó

Page 26: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

ÛÄ�*¨¡£¥«^©ª��òY�¡© � �O� �¡£d«à£ØÜ4¤¬�¡©ª�¡­'Ü4«à­���¤§©ª�¬�¡Û � Ü4� Ü�¨¡¨¡©ä� ©à���ÚÜ�«à«ä�#Û~ÜX�£�ì5ïMì�ÐMïMæRóPÿaö íï´�å�ïMæ-ö£ïPóáf��¤(�¬�¡£Ä¦4Ü�«ªç¡£ô��áU� �¡£õ�øÜ4¤¬­�£â� � ��� � £�Þu�"�ôÜ4��¨¡£¥®¬©à­��ÚÜ4� £¥¨ÏÞ���©ª�t�¬®�¯Q ý� Ò �ÚÜ�Þu�¬£�¤¬®Qü�Ü��¡¨�÷�òQ£� ���¡®¬©ª¨¡£�¤ì¦�£�¤¬®§©à���u®��4ás�¬�¡£ � � Ò «à£ØÜ4¤¬�¡©ª�¡­mÛÄ�*¨¡£¥«J©ª�.òY�¡© � ��� �¡£î¨u©à®§�¬¤¬©ª�¡çu� ©ª���8M&¨¡£�Þ�£��u¨¡®Y���� �u£m�øÜ�¤§­�£¥� � ��� � £�Þu����£�©ª�¡­.«à£�Ü�¤¬�u£�¨i¯îóÏ£ô¨¡£�® � ¤¬©ª�K£ô£�Ü � ����áS� �¡£¥®¬£m¦4Ü�¤§©èÜ��"� ®�©à� ÛÄ��¤¬£ ¨¡£¥� Ü�©à«©à���¬�¡£ � ��¤§¤¬£�®§ÞK���¡¨¡©à�u­ � �¡Ü�Þu� £¥¤¬®¥¯ ý�<����� �õ� �u£^®y�øÜ��¡¨¡Ü�¤¬¨ � � Ò ÛÄ�*¨u£�«¾Ü��¡¨õ� �u£�¦4Ü�¤§©èÜ��"� ®SÛÄ£��"� ©ª���¡£¥¨<Ü����v¦�£4û*©ä�S©à®UÜ�®¬®§ç¡ÛÄ£�¨

� �¡Ü4�0�¬�¡£�£ðú¡Ü4Û~Þ¡«ª£^��¤ Ü � «à£@L���í,�l�NM~ïSÜ�«äòìÜØ��®0Þ¡¤§�v¦*©ª¨¡£�®Q¨ÚÜ4� Ü ÞK��©à�"� ®SòY�¡© � �.Ü�¤¬£^«èÜ���£�«ª£�¨ � ��¤_ö¤¬£ � �¬«ª��Ü �¥� ��¤§¨¡©à�u­õ�¬�8� � ¢*£â¦�£�¤¬Ü�«D¨¡©��K£�¤§£��"��Û~��¨¡£¥«à®I��á�«à£ØÜ4¤¬�¡©ª�¡­máf¤¬��Û �¡��©à®y�.¨ÚÜh�øÜõ�ÚÜئ�£��K£¥£��Þ¡¤§��ÞK��®¬£�¨d©à��Ü�� Ü4�§� £¥Û~Þu��� �~¤¬£¥«èÜhú��¬�¡©à®^Ü�®¬®§ç¡ÛÄÞu� ©ª���i¯IñI�¡£¥®¬£mÛÄ�*¨u£�«à®Y©ª� � «ªç¡¨¡£l� �¡£�õMé èUóPóMö _ Ñõ¸èXîcö£äUç&ç�äUöwó8ï�ÛÄ�*¨¡£¥«Ú©à�õòY�¡© � �Ä£ØÜ � � � Ü�«ª«u� �"L���í,�l�KM~ï�¤¬£¥�¬ç¡¤¬�u®0ÜlÛ~©ª®¬«àÜ��K£¥«à£¥¨ô£âúuÜ�ÛÄÞ¡«à£IòY©ª�¬�®¬��Û~£y�uú*£�¨'Þ¡¤¬���¡Ü��¡©à«ª©ª�ý��ÿD� �¡£&ì5äUç�äUî7äUç1ö£õyç�äUöwó8ï(ÛÄ�*¨u£�«>©à��òY�¡© � �N�§����¤¬¨u£�¤¬«ª©à�¡£���£âúuÜ�ÛÄÞ¡«à£¥®Ü�¤§£ÛÄ��¤§£�«à©w4£�«ä�'� �c��£Ûĩ஧«èÜ���£�«ª£�¨iÿ0Ü��¡¨�� �¡£�ì$èXéWö£õMö£äUåGó�ç�äUöwó8ï Û~��¨¡£�«I©ª�GòY�¡© � ��Ü�����Ûõö�¡©ª® � ©ª£��"� Ü�¨u¦�£¥¤¬®¬Ü�¤§� � Ü4� � ��¤¬¤¬çuÞu���K���¬�Ï�¬�¡£~«àÜ���£�«à®î�4áì£âúuÜ�ÛÄÞ¡«ª£�®ÍÜ��¡¨�� �¡£Ä£âúuÜ�ÛÄÞ¡«à£õÞ���©à�"�¬®� �u£�ÛÄ®¬£�«ä¦�£¥®�¯Só'£l¨¡£M�Ú�¡£ÍÜ��¡¨�®§�¬ç¡¨u��� �u£�®¬£�ÛÄ�*¨¡£¥«à®ì©ª� Ò �ÚÜ�Þu�¬£�¤¬®jÓuû1úÄÜ��¡¨&÷u¯

º�£zy Â�ÄÆÅ Ó ­rÔ¬§^­ÆÅ G�ŵªî¿ ­r§^­IH ¤ ¥�ÿÆÅÈÔ

óÏ£^òY©à«ª«Ú� �¥� Ü�®§©à���¡Ü�«à«ä� � ���u®¬©à¨u£�¤ìÜ�¨¡©��K£�¤§£��"�쫪£ØÜ�¤§�¡©à�u­ÍÛÄ�*¨u£�«�t�¡�vòY��Ü�®Q�¬�¡£$äUç1éWö«ç�ïjéwï¸èXæ-ç1ö«ç��ì5ä�ðaïMé9òY�¡© � �ÄòìÜ�®Q©à�"� ¤§�*¨uç � £¥¨~�"�Ä���¡­�«àç¡©ª�díf���¡­�«àç¡©ª�iûÚ�8÷+�+��ï6Ü��¡¨CÔD©ä�¬�¬«à£�®y� ���u£ ícÔ@©ª�¬�¬«à£¥®§� ���¡£�û�8÷+�G�"ïâ¯I ý�Ä�¬�¡©à®SÛÄ�*¨¡£¥«¡«à£�Ü�¤¬�u©à�¡­î©ª®6¦*©ª£¥òQ£�¨�Ü4®QÜ4�Ä©ª�"� £�¤¬Ü � �¬©ª¦�£�Þu¤¬� � £�®§®Q��£¥�ýòQ£�£¥�<� �u£^«ª£ØÜ�¤§�¡©à�¡­Ü�«ª­���¤¬©ä� �¡ÛùÜ��u¨�¬�¡£l��çu� ®§©à¨¡£îòQ��¤¬«ª¨i¯>ñI�¡¤§��ç¡­��¡��çu�Y� �¡£l©à�"�¬£�¤ Ü � � ©ª����� �¡£l«ª£ØÜ�¤§�¡©à�¡­õÜ4«à­���¤§©ª�¬�¡ÛÛ~Ü�©à�"�øÜ4©à�¡®�ÜG�"��ÞK�4� �¡£¥®¬©à® 6 "�� � �� Í�G�Ø���còY�¡© � � � Ü�����£cçuÞK¨ÚÜh� £�¨+Ü�®�� �u£Ï«ª£ØÜ�¤§�¡©à�¡­Ü�«ª­���¤¬©ä� �¡Û ¤¬£ � £�©ä¦�£�® Û~��¤§£<Ü4�¡¨#ÛÄ��¤¬£Ä©à�uáf��¤¬Û~Ü4� ©ª���cÜ��K��çu�Í�¬�¡£~� Ü�¤¬­�£¥� � ��� � £�Þu��¯ùÔ@£ØÜ�¤§�¡©à�u­Þ¡¤§� � £�£�¨u®©ª� Ü#®¬£¥ü�çu£�� � £'��á�®§� Ü�­�£¥®�òY�u£�¤¬£!£�Ü � ��®§�øÜ4­�£!©à®.®y� ¤§ç � �¬ç¡¤¬£¥¨-Ü�®.áf��«à«à�vòY®©"#ë�©ª¤¬®§�� �u£ «à£�Ü�¤¬�u©à�¡­<Ü�«à­���¤¬©ª�¬�¡Û%©ª®^Þ¡¤§£�®§£��"� £¥¨�òY©ä� �!Ü4� ç¡�¡«àÜ��K£¥«à£¥¨d£âúuÜ�ÛÄÞ¡«ª£C�{�{� � ñI�¡£m«ª£ØÜ�¤§�¡©à�¡­Ü�«ª­���¤¬©ä� �¡Û ç¡®¬£¥® ©ä� ® � 硤§¤¬£¥�t� �"�*Þ����¬�¡£�®§©à®Í�¬� � ��Û~Þuçu� £ 6 íD�KïlÜ��¡¨'��çu� Þ¡ç*� ®Í�¬�¡£õ¦�Ü4«àç¡£ 6 íD�Kï �ñI�¡£¥� � �¡£U«à£�Ü�¤¬�u©à�¡­^Ü�«à­���¤¬©ä� �¡Û�©à®�Þ¡¤¬£¥®¬£¥�t�¬£�¨mòY©ä� �l� �¡£U� ¤¬çu£U¦4Ü�«ªç¡£^��í �Kï���á*� �¡£S�øÜ�¤§­�£¥� � ��� � £¥Þu����Ä£âúuÜ�ÛÄÞ¡«à£v� � ñI�¡£�«à£�Ü�¤¬�¡©ª�¡­lÜ�«ª­���¤¬©ä� �¡Û�Û~Üv�mç¡ÞK¨¡Ü4� £�©ª�¬® � 硤¬¤§£��"�Q�t��Þ���� �u£�®¬©ª®Q���Ä�¬�¡£^�¡Ü�®¬©ª®��á�� �¡©ª®I�¡£¥ò ©à�uáf��¤§Û<Üh� ©à���<��£¥áf��¤§£l�¬�¡£l�¡£âú���®§� Ü�­�£l��£�­�©ª�¡®�¯� «à£�Ü�¤¬�u©à�¡­ÍÜ�«à­���¤¬©ä� �¡Ûþ©ª�<� �u©à®0ÛÄ��¨¡£�«¾©à®Q® Ü�©ª¨<� �5ì$èdvGïEèyìyöwóMî=èdvGïSòY�u£��¡£â¦�£�¤Y©ä� ®UÞ¡¤¬£¥¨¡© � ö

� ©ª��� ¨¡© �J£¥¤¬®�áf¤§��Û � �¡£m�¬¤¬ç¡£m¦4Ü�«ªç¡£ ��áS�¬�¡£ô«èÜ���£�«gûK©g¯ £4¯ 6 í �Kï�WW|�4íD�Kï � ñI�¡£õÞ�£�¤§áf��¤¬Û~Ü�� � £Í��áQÜ«à£�Ü�¤¬�u©à�¡­�Ü�«ª­���¤§©ª� �uÛ ©ª®�ÛÄ£ØÜ4®¬ç¡¤§£�¨'�"�!�¬�¡£õ��çuÛm�K£¥¤Í�4á0ÛÄ©à®y�øÜX4£�®�©ª�lÛ~ÜX�£¥®����'Ü��c£âúuÜ�ÛÄÞ¡«ª£®¬£¥ütç¡£�� � £�¯U�+«ª£ØÜ�¤§�¡©à�u­ Ü�«ª­���¤§©ª� �uÛ×� áf��¤0Ü � ��� � £¥Þu� � «àÜ�®¬®[�d©à®U® Ü�©ª¨~� ���ÚÜئ�£�Û~©ª®§�øÜ��£��K��çu�¡¨

�@á

Page 27: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

} ©ªá�áf��¤(Ü��"��®¬£¥ü�çu£�� � £Ä�4áS£âúuÜ�ÛÄÞ¡«à£¥®�©à��� Ü��¡¨!Ü4�t��øÜ4¤¬­�£â� � ��� � £¥Þu���"�{�«�iÜ�«ª­���¤¬©ä� �¡Û÷�Û~ÜX�£¥®^Üh�^ÛÄ��®y� } ÛÄ©à®y�øÜX4£�®�¯SñI�¡£$ìyöwóMî£èdvGïCÐMäUå�ç�ðî��á�Ü � ��� � £�Þu� � «àÜ�®¬®��Ï©à®ì�¬�¡£lÛ~©ª�¡©àÛ ç¡ÛÛĩ஧� ÜX�£î����ç¡�u¨���á�Ü��"�����u«à©à�u£î«à£ØÜ4¤¬�¡©ª�¡­õÜ�«à­���¤¬©ä� �¡Ûßáf��¤.� � ý����¤¬¨¡£¥¤m� � á{Ü � ©à«ª©ª�øÜh� £ � ��ÛÄÞÚÜ�¤§©à®¬���¡®m��£¥�ýòQ£�£������¡«ª©à�¡£<Ü��¡¨ � � Ò «à£ØÜ4¤¬�¡©ª�¡­!Ü�«ª­���¤¬©ä� �¡ÛÄ®

òQ£(òY©à«à«¾®§��Û~£â� ©ªÛ~£¥®Qò0Ü��"�I� �ôÜ��¡Ü�«ª��÷�£^� �¡£ � � Ò «ª£ØÜ�¤§�¡©à�u­mÜ��¡©ª«à©ä� ©à£¥®S��áD���¡«ª©à�¡£^«à£ØÜ4¤¬�¡©ª�¡­mÜ�«ª­��4ö¤¬©ä� �¡ÛÄ®�¯�¢*£â¦�£�¤¬Ü�«¡­�£¥�¡£�¤§© �Q� ���"¦�£¥¤¬®§©à���ô� £ � �¡�¡©ªü�çu£�®SÜ�¤§£[t�¡�vòY�ôòY�¡© � � � Ü��m��£ìç¡®¬£¥¨m� � � ���"¦�£¥¤§�Ü��"�l���¡«ª©à�¡£>«ª£ØÜ�¤§�¡©à�u­^Ü4«à­���¤§©ª�¬�¡Û+©ª�"� �^Ü � � Ò «à£ØÜ4¤¬�¡©ª�¡­�Ü�«à­���¤¬©ª�¬�¡ÛMí{���u­�«àç¡©ª�iût�8÷+�+�*ÿ!��Ü�çu®¬®¬«ª£�¤¥û�8÷+�G�uÿvßî£�Ü�¤¬�¡®�ïMî�èXéxíàûK�8÷G�Gü4�iÿ�Ô@©ª�¬�¬«à£¥®§� ���¡£�û��8÷+�`÷�Ü"ï⯫ ý��£�Ü � ����á@�¬�¡£�®§£ � ���"¦�£¥¤¬®§©à���¡®ì�¬�¡£(®¬Ü�ÛõöÞ¡«ª£ � ��ÛÄÞ¡«à£ðú*©ª�ý����á�� �u£�¤§£�®§ç¡«ª�¬©à�¡­ � � Ò Ü�«à­���¤¬©ª�¬�¡Ûù¨¡£�Þ�£��¡¨u®^����� �u£�ÛÄ©à®y�øÜX4£l�K��çu�¡¨��4á�� �u£���¡«ª©à�¡£�«à£�Ü�¤¬�u©à�¡­ Ü�«à­���¤¬©ª�¬�¡Û¯�ñI�u£ � ���"¦�£¥¤¬®§©à����Þ¡¤§� � £�¨¡çu¤¬£(¨¡çu£�� �yÔ@©ä�¬� «ª£�®y� ���¡£^©à®0Ü�®§��ÛÄÞu� ���¬©äö� Ü4«à«ª�mÛ~��®y�0£âã � ©à£¥�"�I©à�õ� £¥¤¬ÛÄ®0��ái�¬�¡£�®¬Ü�ÛÄÞ¡«à£ � ��Û~Þ¡«ª£âú*©ª�ý�õ�4ái� �¡£^¤¬£¥®¬ç¡«ä� ©à�u­ � � Ò Ü4«à­���¤§©ª�¬�¡ÛÜ��¡¨�òQ��¤¸t®YÜ�®ìáf��«à«à�vòY®©"6ñI�¡£îÞ¡¤§� � £�¨¡ç¡¤§£A�Ú¤¬®y�Y¤¬ç¡�u®Y�¬�¡£(���u«à©à�u£(Ü�«ª­���¤¬©ä� �¡Û ����Üm®§£�ütç¡£¥� � £���á«èÜ4�K£¥«à£�¨!£ðúuÜ�Û~Þu«à£�®l���*�øÜ�©ª�¡£�¨!áf¤¬��Û ®¬ç ��� £¥®¬®§©ª¦�£ � Ü�«ª«à®(�¬���¬�¡£ � � Ò ��¤ Ü � «à£õÜ��u¨Ï®y� ��¤§£�®�Ü�«à«s��á� �u£^�"��ÞK�4� �¡£¥®¬£�®UòY�¡© � �Ä� �¡£����¡«ª©à�¡£YÜ4«à­���¤§©ª�¬�¡Û ­�£��¡£¥¤ Ü4�¬£�®U���Ä�¬�¡©à®S£ðú¡Ü4Û~Þ¡«ª£Y®¬£�ütç¡£¥� � £4¯QñI�u£��� �u£�Þu¤¬� � £�¨u硤¬£l¨¡¤¬ÜvòY®�Ümáf¤¬£¥®¬���ÚÜ4� � ���4á�«àÜ��K£¥«à£¥¨�£âúuÜ�ÛÄÞ¡«à£¥®ìáf¤¬��Ûù� �¡£î��¤¬Ü � «ª£îÜ��¡¨ç¡®§£l�¬�¡£�®§£�¡£âò £ðúuÜ�Û~Þu«à£�®�� �.£¥®§�¬©àÛ~Ü4� £l� �¡£Í£�¤§¤¬��¤^¤ Ü4�¬£ ��áS£�Ü � �!��á6�¬�¡£m®§�¬��¤¬£¥¨ �"��ÞK���¬�¡£�®§£�®¥¯�ñI�u£��Ú�ÚÜ�«�"��ÞK���¬�¡£�®§©à®Í��çu� Þuçu� �"�!�¬�¡£<Þ¡¤§� � £�¨¡çu¤¬£Ä©à®l� �¡£Ä���¡£õòY©ª�¬�c�¬�¡£~«ª�vò0£¥®§�m���¡®¬£¥¤§¦�£¥¨#£�¤§¤¬��¤�¤ Üh� £�¯Ô@©ª�§� «à£¥®§�¬���¡£îÞ¡¤§�9¦�£�®Y� �¡£îáf��«ª«à�vòY©à�u­ � �¡£¥��¤¬£¥Û\"· ÈD¼tƾÇh¼�É �cb ��~ ÒQ¸º¹Ø¹9»f¼�Ê¥¹vÆ�ÂD¼�� �f R f Ñ�� ��ïMîã� ÐMï^èXç äUç1éWö«ç�ï$éwï¸èXæMç1ö«ç��þèXé �aäUæ-ö«î«ÿaì�eOÿaö£õPÿõPÿvèXç��aïPó�ö«î«ó�ÿaòPí1äUî«ÿ1ïPóMöwó$äUç1éWò�eOÿ1ïMçþö«î;ì$èdvGïPó�è^ìyöwóMî=èdvGï�èXç�ð1eOÿaö£õPÿ�ÿvèUóQè^ìyöwóMî=èdvGïVÐMäUåaç�ðä7ê } ê@äUæ�õPäUç�õPï"í�îLõMé èUóPóe, ��� ÿ1ïMç�î«ÿ1ïMæRïIöwósèîÚtéQë«Ñ"ì5ä�ðaïMé�éwï¸èXæMç1ö«ç��$èXé �aäUæ-ö«îÝÿaì �@�aê@äUæv,�èUóðaïPó8õMæ-ö(ÐMï¸ð^è&ÐMäd>Xï0eOÿaö£õPÿ^ÿvèUó�ó�èXì�í�éwïQõPäUì�í�éwïKJ`ö«î¶ò

â�� � ô �9«à��­ �\7� }t�4� �º�£�� ¤ ªC¦«ÄÆÅȾ ªC¦«§^ý�ªVÔ*� ªîý2�1H�¿ ¥�½r­rÿó{�Íç��K� Ü7� Ü�����0��á6Û~Ü��N����Û~Ü��-���ØÜ����-������������N�������N���g�K���g�. n� Ò �ÚÜ4Þ��-�U�jáõò.�Íç��N�0�-������ìá=�N��Û�g�z �Þ��-���� Ü�Þ��a ��N��Þ��K�¢¡�����vò.�U�0£���ç���¤�����'Ü����6Þ���� �����Û��àÜ��i�N�àÛ�������Ü��N�������Ü�������N�¥�-�¡Û���ÿ¦���Ò �ÚÜ�Þ§�-�g�K�Lø*û/úuû/üîÜ���¤$÷�ò2�Yç��N�k��U��Û��¨�-�K���ìÜ��N��ç¡Û��g�a�N���-�0¤��U¡��g����ÞÄÜ���¤ÄÜ��ÚÜ��¥ �÷g�YÜ������N�¥�-�¡Û��á=��7����Ü��N����������������Ü����-���K�g�K�����¤�áfç����¨�-�������ÿ�Ü���¤©��� Ò �ÚÜ4Þ��-�U��ª¢« ò2��ç��N� ë¬��ç��N���U�õÜ��ÚÜ��¥ ��N����¢¡�U�u�-���Íæ.�������Ü��5�¥ç�£��­�-���g����K�-�§ç��U�(Ü��!Ü������K���-�uÛ&á=�������Ü��N�������~éîê�ë.�; n�5��Ü��P����á��N���g�K��P�ÚÜ�Þ§�-�g�K��ò2�����a�-�K�§¤¡ç��g���-�����K��ütç����N�g¤O£ÚÜ��P���K��ç���¤�Û<Üd�-�g�K�èÜ��JÜ��k���g�U¤��g¤®�

ªTø

Page 28: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

û����@� Ü��¸mòk�����P�<ò2�^òk���������U�g¤8�N� Þ��g�§á=���¬Û���ÄÛ<Ü��� 7���g��Ü��N������^���p�-��£���ç���¤��N���u¤��U¡��èÜhö�-����1��á�Ü��§ç¡Û¯�4áZ� Ü���¤���Û°¡4Ü��K�èÜ�£����g�Uá=�N��Û°���N�.�ðúuÞ��g�¨�-�U¤O¡4Ü��ªç����6ñ����(á=������9òk�����lò.�U���q����vòk�á{Ü��¨�-�.�K���vò*�-�ÚÜ��Qá=��QÜ­�¬çuÛ+��áDÛ~Ü��a 7����¤���Þ��g��¤��g�a��� Ü���¤���Û�¡4Ü��N�àÜ�£����g�p�-���^Þ��N�£¡Ü�£��������n &�4áDÜ�èÜ��N���¤��U¡��èÜ��N����������âú*Þ������g�a�-�àÜ����¥ 8�§Û<Ü����±�6ê@���N���-�ÚÜ��������N���I�¬�N�z�Yá{Ü��U�k£��g���9òm�ØÜ��P�5� Ü���¤���Û¡4Ü��N�àÜ�£����"�øÜX��g�0¡4Ü��àç��U�����F�d«��Sª��<Ü���¤��N���7¤��U¡��àÜ��-����%���îÛ��ØÜ��§ç��N�U¤ÏÛmç����-�ªÞ������ØÜ��N��¡��g�� �û�òk����������7�-���@�K�g�g����¤<á{Ü��¨�.��Ü��P�8�¬Ü���¤���Û|¡4Ü��K�èÜ�£����������g�p���~ÜI�uú§�g¤1�����N�g�z¡�Ü�� V³² �P´-XKÜ���¤��N����¤��U¡��èÜd�-��������IÛ��ØÜ��¬ç��K�g¤�Ü�¤�¤��¥�-��¡��g�¥ �� Ѭµ�¶ �cb ��~P·�¸Z¹�ºd»4¼¬½ � �f�4;� � �\ïMî®� � � �^�^� �N�7¾.ÐMï�ö«ç�ðaï"í1ïMç�ðaïMç1î[¿PÀ®ÁGÑn>UèXéWå1ï¸ðyæèXç�ðaäUì�>UèXæ-ö Ñè&Ð�éwïPó&eOÿaö£õPÿ:ó�èXî¶öwóÝê-ò7Â[� V � � WêgX«WðU � èXç�ð�«OhñU � h�ªsê@äUæyèXé«é®Ä � �\ïMî[� W�Å ¾�ÇÆ � � � � C�äUæÈ W Å ¾�ÇÆ � U � èXç�ð^èXç1ò­«&h*É�hmª1e[ïAÿvè�>XïÂ[� V �Ê)xËzª � É�Ì È XZhj�¨Í§Î4Ëz È É õ ìXÓÌèXç�ð Â[� V �Ï]xËzªk �É�Ì È X¦hÐ��Í�Î4ËY È É õ ìÑÌ ��¦Ò µ�¶ �cbx$°~ @ ¼c¹a½^Ó4Ô;»ZÕ � �f;:;� � ��ïMîi��Öc� �^�^� �N�7¾0ÐMïQö«ç�ðaï"í1ïMç�ðaïMç1î�æèXç�ðaäUì×>UèXæMöcè&Ð�éwïPóIóMå1õPÿî«ÿvèXî ² ]Ð� � ]*´xê@äUæ�èXé«é¬Ä �®Ø ê�� W Ö ¾ Å ¾�ÙÆ Ö � � î«ÿ1ïMç�ê@äUæsèXç1ò"É�wÚ«Oe[ïIÿvè�>Xï

Â[� V �Ê)*L V �IX � É�XZ]t��ͧÎ5Û �ÑÝÜKÉ õË,´2 ² Ì õÞß�à�á Â[� V �â]*L V �IX� �É�XZ]Ú�¨Í§Î Û uÑdÜKÉ õË,´2 ² Ì õ Þ �ã �&�U����K�"�-�����u�P��ä�Î��-�U��åk���N���N�æ��������-ä��-����c��ç¬��u�N�Sä��¥èé¡�ä���ê��g¤Oë=ê����¨�-�������0ì9ä���¤Ií�å2�åk�N�¥�-��ì�Ë=î�Ì�ï òâ8Ë?í®Ë,î�ÌNÌ��¥ë2�N���g�K�����­äð�ͧ�g¤{�g����K�Pä����"�&�Nê��P���-�¬äd��ì�Ë,î�Ì0ï â8Ë=í®Ë,î�Ì������ñ�î�Ì �ò ���������Pä��N������ òÛ0Ë?ícË=î�ÌNÌkä���¤ òó Ë?ícË=î�ÌNÌkä��N�­¤��Uð����g¤%ä���ä�������ê��N�¥ � ã �"åk�N�¥�-��������-�8¤��g�����N������aä��N�¥�-��æm£¬ä��K��Ñ@ä���¤������-�@¤��g�����-�.��ä��-ê��Nä�������ä��N�¥�-��æO�Zç¬��¦ôI�rë�õ@å2�2åk�K���N�^ô¬ö§�-�u¤��g�����N��-���&Ä,èé�N�%�g����N¤����¬ä��N�"��ë^ô9ä���¤�å.��åk�K���-�I÷Pô[÷0�N�O¤��g�����N���-���7ø[ê��U����¤��Sä��%�����NæÊù Å õö Æ Ö ôûúöÝüý ë�ô%þ ÿd« � ª��¢õ"�-���U�5ô¬ö�¤��g�����N�g���-���­Ä,èé�N��£�������ë�ô ü ç¬����1þ�� �N���0¡�ä���ê�����ë��K����¦Ë��aÌ����0ª"�¥ë�m«7ä���¤������­ª0��ë ��Ú« ü

ª��

Page 29: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

� � ��� ����� �

� ������� � ! �����#" $%" & ')( * + � , $ �

- '/.10 " '/2 $ �3. 4 � �5� �6� '7.98 � : " ;<�=$>'7" �

ò �����i�P�¬ä�Î��-�U���g�z�Pä�£������K���g��ä@���¨å �g�æ�Î����¨Í§���n aèé�N���g���N�U�N���[�P�¬ä��-ä��¨�-�g�K�@?gä��-����"��ëBA�Cuçë=��Næ"ê�� ä��ä����-���K�g�N�����¤��U¤9Î���� ����æ�� ä����0ä���¤lê��N�U�"�-�����­�P��ä��-ä��¨�-�U�N�@?gä��-����{�N�I�£��-ä����{ä�ë,ä��z�����Sä��K�������ä������N�¥�-��æO�ED���N��Î��N�U�g���N�g�¥ GFi�ê���æ8ä����%�N�g�Kê����­���­ä�Î��N����ë.�N�¬ä��"ä��a 5HUè±�-�g�KæIA�Cuçt�Ý¡��g�KJ¡�ä��N��ä�£����g���gä��7£����U�æ�Î�ê��-�U¤7£� �ä�Î���¥ ����æ�� ä��a�-���N�g�K�����¤7ë=ê����U�N����&��ë�¤��U��N�U�ML1ËNJ Ö�OQP ����RHÝ̨�ò ������ê�Î�Î��g��£��ê���¤%æ�ä��-�P���U�SF¦ê�Î��N��ä1����aä��K���N��æ�����ë,ä��U�N��SF¦ä�������z�Pä���¤������1���¢å.�U��£��ê���¤ë=��MA�C@çFåk�����P��å�ä�����¥¡�g�O£a �D������UTa Oä���¤OÂ�ä�Î��U�K�@����-���U����ª�V��GW&£����GTYXMZ\[Rõ]Z_^a`b[]c à ó¨�%d@�ä7¤����K�g�U���U����N�fe�ê��g���U����ëi�-������ê�Î�Î��U�@£��ê���¤Oå2�"�£§�Pä�����ä�Ñ�gih õkjmlNnporq_s�õtorq_sGu�v?èxwzy�æ|{�äk}@~G�G��ymwU��æëN�G��}m{Sä����By@�B~�HUèxwz{S�Kæ�A�CuçRF�åM�By@�]�#y@��wU�B{ë,ä���wU{S��wET��B�¢åM�Ðäk}@~G�G��ymwU��æ�ëN�G��wz�Bym��y�æ�Î��G��wPä���wÎB���G�B}@{Uæ������� �����9�����������% f�|�

¡�¢�£¤¢�£ ¥§¦R¨ª©¬«­¦R®°¯ª±�¨�²5³­´tµ�¶·³­¦¸¨ª¹7º�»�«­¼i½B¯�¦R«­¶¾ {fw�ì¿�a{�ä�ÀR�Á�G}m{Sä��9ë=ê·�B�fwUy@�G�©ìÃÂkÿÅÄ �ÇÆ �¢õ�È ÿ�� ÆG�ÇÆ �Iä��·É#}m{fw­Ê#�a{OäËÉB{S~G��{S{YÌ{Îa�G}ÎÍ�è�B�æÏy äk}¤ym�5JÑÐ�ä���y ä��·}@{S��åMymwU�5�-äkwUy@�G��ä�}��S�Ò{fÓÏ�Sym{S��wz�f� ý ëMwU�B{��Uym~G�5��ë%Ê�Ô=ô�ÕK{fe�ê�ä�}@�7ì Ô=ô�Õ0ëN���{fÐ�{S��Í�ôÐþ ÿÅÄ �ÇÆ �¢õ � wU�B{S��å¸{|�-äÇÍ�wz�¬äÅw&ì=y@�Ö�S��æ|×�êÁwz{SÉ���Í�äØ^tcÅÙWò à cÅÚ�Û ß Ù%`NÜÒ[zZPó]ÜtcÅÙ áÞÝàß�à õ\á`bÛxc à ��ë¬ÉB{S~��U{S{�Ìt� ý �#wU�B{Sym�7åR{S}m}�T��B�¢åM� Æ VG��Wâ�a�Ò�GT#XMZ\[Rõ]Z_^a`�[zc à ó\F¸D�y@�·�UT�Í�ä��BÉÑã�ä�×a{S��w��wNêBÉBym{SÉ��U�æÏ{­�f�æ|×�ê·wPäkwUy@�G��ä�}®äk�U×a{S�fwU�M��ëä×��G}ÎÍÒ�B�æÏy ä�}twz�·�U{S���B�G}mÉ�ë=ê·�B�fwUy@�G�B��ëN���æ+ä��Ødå��wzy¥èðt�fy ä�} ý ��wz{S}m}@ym~G{S�B�f{­×a{S�U��×�{f�fwUymÐG{ÏÔ�D�y@�·�UT�Íä��·É�ã�ä�×a{S��wÇF Æ V��GW�Õ\� ò �B{fÍE×B�U��ÐG{fÉæFûä�æÏ�G�B~��kwz�B{f�wz�·y@�B~G�fF·wz��äkwM�B�×���}mÍÒ�B�æÏy ä�}çwz�·�U{S���B�G}mÉ�ë=ê·�B�fwUy@�G����ëäÉB{f~G�U{f{K}m{S�U�èwU�¬ä���Jâ�Sä����f�æ|×�ê·wz{�wz�·{

Æpé

Page 30: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

׬ä���ymw_Í�ë=ê·�B�fwUy@�G�Ï�G�EJ�Ð�ä��Uy�ä��B}m{S�SF�äk�BÉÏwU�¬äkw¤wz�B{f�U{�y@�^äK��{Sä�É�è_�G�B�f{åA�C@çIëN�G�Næ�ê·} ä�åM�By@�]�|�Sä���è�B��w­��{��f�æ|×�ê·wz{fÉ���Í�ä���ÍË×a�G}mÍÒ�B�æÏy äk}�wU�B�U{f�U�B��}@ÉIë=êB�B�\wzym�G����ë¤ÉB{S~��U{S{�}m{S���­wz�¬ä��âêÖÔNJ Ö�OQP Õ\�ë ym�B�S{�wz�·{S�æFÒ�S�æÏ×B}m{\ìÁymw_ÍKwz�B{f�G�Uym��wz���¬äÇÐ�{@êB��{SÉÏwz�B{f�U{�äk�BÉ|��{S}�äkwz{fÉÏ×·�U�G×a{S��wzy@{f����ëæ×a�G}mÍÒ�B��æ|y�ä�}wz�·�U{S���B�G}mÉ5ë=êB�B�\wzy@���B��wU��×B���pÐ�{��U{\ÐG{f�-ä�}>y�æÏ×��G��wPä���w��U{f�NêB}Îwz�¬y@�Y����wU�Ë�Sym�U�UêBymw@ä��Bɧ��wz�KêB�fwNêB�-äk}�S��æ|×B}m{\ìÁymw_Í�Ôbd¬�U×B�B{f�Z\` ß ÙÎí@F Æ V�V�îBïtÀ¸{fy@~G{f}xF Æ VGVGð·ï�çûêiF Æ VGV�ñGÕà�ý �ÏwU�B{��S�æÏ×�ê·w-äkwzym�G�¬ä�}Á}@{Säk�U�Bym�B~­wz�B{f�G��Í��S�æ�æ�êB�ByÎw_ÍGF�}m{Sä����By@�B~0äÖ×a�G}mÍÒ�B�æÏy äk}ÒwU�B�U{f�U�B�G}mÉë=êB�B�\wzym�G��ëN���æò}�ä���{f}@{fÉ#{àì�ä�æ|×·}@{S�Ï�¬ä��}m�G�B~���{f{S�Ðäâ�S{f��wz�-äk}�×·�U�G�B}m{gæ äk�BÉ<�f�G��wzy@��êB{f�ÏwU��a{�ä���äk�fwzyÎÐG{"ä��U{gäE��ë%��{S�U{gä��U�]�i�­dó��×�{f�Sy äk}4ëN�Ò�gêB�¬��ë^äkw�wz{S��wUy@�G���¬ä��å�a{S{f��ÉBy@��{S�\wz{SÉ�wz�¢å�äk�UÉ}@{gä��U�·y@�B~¬×��G}ÎÍÒ�B�æÏy ä�}GwU�B�U{f�U�B�G}mÉ"ë=êB�B�\wzym�G�B�9��ëçÉ·{S~G��{S{ Æ F�åM�By@�]�7ä���{¸T��B�¢åM��ä��èÙôÛ à Z ß [R`NÜÒ[zZPó]ÜÒácÅÙ áKÝàß�à õ\`�Ûxc à ó\� ò �B{|×B�U�G�·}@{gæ/��ë¸}m{Sä����By@�·~�ä�}@ym�B{Säk�åwz�B��{S���B�G}@É�ë=êB�B�\wzy@������Ð�{S��ÿÅÄ �ÇÆ �¢õØ�gä���a{�ëN�G�Kæ�êB}�äkwz{fÉ{äk�"äY}@y@�·{Sä���×B���G~G�Nä�æ�æÏy@�·~�×B�U���B}@{Uæ ä��BÉ�wU��êB�K�gä��6�a{Ï�U�G}ÎÐG{SÉ6y@��×��G}ÎÍiÔbJ1Õwzy�æ|{�ym������wU�YwU�B{Kã�d�õmæÏ�ÁÉ·{S}1��ë }@{Säk�U�Bym�B~&ëN�U�æ��Nä��BÉB��æ�{\ì�ä�æÏ×B}@{f��ä��Bɧym�Øwz�B{�æ|�ÒÉB{f}æ��ë{\ì�ä��\w%}@{gä��U�·y@�B~�ëN�U�æö{Se�êByÎÐ�ä�}@{f�B�S{èe�êB{f�Uym{S�MÔbÀ¸}�ê�æÏ{S��Z\` ß ÙÎí@F Æ V�WGV·ï�D5ää����[ä��BÉ ò êB�-äk�æF Æ VGVkî�Õ\�÷ {Uðt�B{Uæ|{f��wz�Ö��ë%wz�·{��¬ä��Uym��}@ym�B{Säk�å×B���G~G�-ä�æ�æÏy@�B~�ä�×B×·�U�aä��]�â�¬äÇÐ�{Ï}m{SɧwU�Ø×���}mÍÒ�B�æÏy ä�}ÙèQwUy�æÏ{ä�}m~G�G�UyÎwz��æÏ�¦ëN�G�9ãad�õ5}@{Säk�U�Bym�B~å}my@�B{gä��äwU�B�U{f�U�B��}@É"ë=êB�·�fwzym�G�B�>y@�øwU�B{è×B��{S�U{f�B�S{���ëç�S} äk�U�Uy¥ðt�SäÅwzy@����B�Gym�U{åÔ�À¸}�ê�æ°Z\` ß ÙÎí@F Æ VGV é ïÅõ¸�G�B{f�æF Æ VGV é Õ\�>D�êB�]�7äÅwUwz{f��wzy@���K�¬äk��ä�}m�U���U{f�S{S��wU}mÍÞ��{f{S�K~GymÐ�{S�KwU�ë,ä���wÇF·��y�æÏ×B}@{@ä�}m~G�G�UyÎwz��æÏ�SFaæÏ�G��wè�B��w-ä��B}ÎÍÏwU�B{ ã ym�B�B�¢åmä��BÉ�ã>{f�U�f{S×·wU�U�G�Oä�}m~G�G�UyÎwz��æÏ�SFåM�·y@�]�}@{gä��U�Ï}@ym�B{Säk�9wz�B��{S�U�·�G}@É�ë=êB�B�fwUy@�G�·�2êB�·ÉB{S�R�U{S��wz��y@�fwU{SÉ|�S�G�BÉ·ymwzym�G�B�¬Ô�À¤ÍÁ}�ä��BÉB{f�SF Æ VGV�W��æï�çt�U{gê·�BÉä��BÉ ë �]�¬ä�×Bym�U{�F Æ VGVGWÁïiù�ymÐÒym�B{S�ÑZ\` ß ÙÎí@F Æ VGV é ï ¾ yÎwUwz}m{S��wz�G�B{kF Æ VGWGW·ï ë �]��æÏymw�wÇF Æ VGVGW·ï ë {f��ÐG{fÉBy@�·FÆ VGV�V��çÕ\� ã {�~GyÎÐG{�ä�É·{fwPäky@}@{fÉä��¬äk}mÍÒ�Uym�è��ë�wz�B{f�U{0ä��·ÉØ�U{S}�äkwz{fÉ�ä�}@~��G�UyÎwz��æÏ�Rym��õ¸�¬ä�×Áwz{S���¬úÁFB�·Fé ä��BÉ�V·�¡�¢�£¤¢_û üµä±%´ç«�¯�«­ýÿþ��údå�·��wz�B{f�%y@��wz{f�B�UyÎÐG{f}mÍ���w-êBÉ·y@{SÉ×B���G�B}m{gæ�y@���S��æ|×�êÁwPäkwUy@�G�¬äk}Ò}m{Sä����By@�·~åwz�B{f�G��Í�FaåM�By@�]��¬ä���æ|{\wåMymwU��}@{f�U�è�KêB�S�f{S���SFçy@�¤wz�B{­×·�U�G�B}m{gæÿ��ë�}@{gä��U�·y@�B~øA�Cuç{ëN�G�Kæ�êB}�ä�{�� A�Cuç9ä���{�äkw�wz�-äk�fwzyÎÐG{�ëN�U�æäË}@{gä��U�·y@�B~§wU�B{S����Í5×a{S���U×a{S�\wzymÐ�{���{f�Sä�êB��{���ëMwz�B{fy@���Bym~G�={\ìÁ×B��{S���UymÐ�{�×a�¢å¸{f�YÔ;ä���Í5À¸�Á��}@{Säk�ë=êB�B�\wzym�G�§�gä��§�a{K��{S×B��{S��{S��wz{fÉ{äk�uä|A�Cuç%Õkä��BÉ���{f�Sä�êB��{KwU�B{fÍ��U{f{gæ3wz�|��{­ä|�¬äÅw-êB�Nä�}¦ëN�G�Næ��ë TÒ�·�ÝåM}m{SÉB~�{K��{S×B��{S�U{f��wPäkwUy@�G�OëN�G�M��ê�æ8ä��·�S���^äk}@y äk��w.ðt�U��wå×a�G�U{fÉ�wU�B{Ke�êB{f��wUy@�G����ë�åM�B{\wz�B{f�A�Cuç�ä���{¬{\Ó|�fy@{f��wU}mÍ}m{Sä����¬ä��B}m{�ym�|�·y@�¤�U{gæÏy@��ä�} Æ VGW�îÞ׬ä�×a{S�¤y@��wU�U�ÒÉ�êB�Sym�B~�wz�B{åã�d�õ<}m{Sä����By@�B~æÏ�ÁÉB{f}¤Ô��^ä�}my ä���wSF Æ VGW�î�Õàïûæ|�G��{�wz�¬ä��Oð�ë wz{f{S�§Í�{Sä����å}�äkwz{f�MwU�By@��e�êB{S��wzym�G�§ym��åMymÉB{S}ÎÍØ�U{f~aä��UÉ·{SÉä����G�B{Ö��ë>wz�B{�æ|����w�y�æÏ×�����wPäk��wM��×�{f��×B���G�B}m{gæÏ��ym��}m{Sä����By@�·~wU�B{S����ÍG� ã �By@}m{0æ8ä���ÍØ×�ä���wUy ä�}�U{f�NêB}Îwz�¬�¬äÇÐG{ø�a{S{f�Ë~GyÎÐG{S�OëN�G���U{S��wz��y@�fwU{SÉYÐG{S���Uym�G�B�å��ë9wz�B{øA�Cuç }@{gä��U�Bym�B~×B�U���B}@{Uæ Ôb�U{f{øwz�·{y@��wU�U�ÒÉ�êB�fwUy@�G�âwz��õ¸�¬ä�×·wU{S� Æ ÄOëN�G�­ä��5��ÐG{f��ÐÒy@{¨åÿ��ë��U��æ|{|��ëèwz�·{S�U{E�U{S�KêB}mwU�]ÕàF wz�B{|ÉBymÓÏ�UêB}mw_Í

Æ W

Page 31: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

��ë>wz�B{0êB�B��{S��wz�Uym�fwU{SÉËA�C@ç#}m{Sä����By@�·~�×B�U�G�·}@{gæ�y@�M{fÐÒy@É·{S�B�f{SÉË��ÍØwz�·{0ë,äk�fwMwz�¬äÅwÇFç×B�Uym�G�èwz�wz�·{�U{f�NêB}Îwz�¬��ë wU�By@���]�¬ä�×·wU{S�fFi�G�B}ÎÍ�wnåR�ä�}@~��G�UyÎwz��æÏ��å¸{f�U{KTÒ�·�ÝåM�IåM�·y@�]�§y�æ|×·�U��ÐG{Þ�G��wU�B{ø��ä�ymÐ�{ñ õ wUy�æÏ{­�a�êB�BÉ6Ô�À¸���B�ê·w_Í�F Æ VGVG�·ï ò ä��KêByZä��BÉ ò �KêBT�y���yxF Æ VGVGV�Õà�ò �B{uðt����w��KêB�a{\ìÁ×����B{S��wzy�ä�}çwzy�æÏ{@ä�}@~G���UymwU��æ:ëN�G�¤}@{gä��U�·y@�B~KA�Cuç�y@�¸É�êB{¬wU��À¸�U�·�ê·w_ÍGF§åM�B�~aäÇÐG{^ä��0äk}@~G�G��ymwU��æÚåM�By@�]�Ö}@{gä��U�·�Zä���ÍÖHUèQwU{S�Næ A�Cuç���ÐG{f�1J�Ð�ä��Uy�ä��B}@{f�æy@�­wUy�æÏ{%ñ gih�� õBorq_sGuSorq_s nNl õ�v üdèwMwz�B{��·{Sä���w¬��ë9À¸���B�ê·w_Í�� �uäk}@~G�G��ymwU��æ y@�kä���wU�NêB�\w-êB�Nä�}1�U{f�NêB}Îw�åM�By@�]�Y�U�B�¢åM�Mwz�¬äÅw�wz�¬äkw�äk��ÍHUèQwU{S�Næ/A�Cuç¿�gä����a{�{\ìÁ×B��{S���U{SÉ9ä��0ä� �ÉB{S�fy@��y@�G�â}@ym��w�����ëR�G�UÉB{f�ÖL�Ô � J�}m�G~�J�}@�G~¤HpÕ\ï4ä��Kæ|{fÉåMymwU�Øwz�Bym���U{S�KêB}mwSFtÀ¸���B�ê·w_Í�êB��{S�@ä���wPäk�Bɬä���É5ä�}m~G�G��ymwz��æ�ëN���æ Ô��¬{f}�æ��a�G}@É�Z\` ß ÙÎí@F Æ VGVGÄ�Õ^ëN���}@{gä��U�·y@�B~�ÉB{f�Sym�Uy@���Y}my@��wz�èwz��G�ÁwPä�ym�Ø�By@��A�Cuç5}m{Sä����By@�·~��U{S�KêB}mwS�ò äk�NêBy2ä��BÉ ò �NêBT�y���y¤~aäÇÐG{8ä��f�æ|×·}@{fwU{S}ÎÍ�ÉBy���{S��{S��w�×·�U�Ò��ëè��ë�ä��Uy�æ|ym} ä��¬wzy�æÏ{����êB�BÉ{ëN���}@{gä��U�·y@�B~�A�C@ç¸� ò �B{fÍ7äkɬä�×·wU{SÉ�wz�B{.æ8ä��]�·y@�B{f��Í���ë� Nä�×B×B���Çì·y�æ8äÅwz{èym�B�S}�êB�Uym�G���k{àì·�f}�êB��y@�G���åÉB{�èÐG{f}@�G×a{SÉK��Í ¾ ym�By ä�}�ä��BÉøC¬y@�Nä��EÔ ¾ y@�By�ä�}aä��·É�C¬ym�-ä��iF Æ V�VGÄ�Õ�wz�¬�U�B�¢å5wU�¬äkw4ëN�G�4ä���ÍKHUèQwz{f�NæöA�C@çìFä��BÉ{ä���ÍâÉBy@��wz��y@��ê·wUy@�G��� ��ÐG{f��ÿÅÄ �SÆ �¢õ � wz�B{f�U{y@�0ä��S�G���KêB�B�fwUy@�G������ëR�Uym?S{ÏL�Ô � J�}m�G~¤HpÕåM�Bym�]�Ï��ä�����ã�� �"!$#&%��|Ô=ô�Õpï ì9Ô?ô�Õ 'f� Öú ��ï ñ)(Ágih+* õtorq_s�õBorq_s�u�v ü-, �Uym�B~�wz�Bym�9�U{f�NêB}Îw%wz�G~�{fwz�·{S��åMymwU�çt�U{UêB�BÉ.� �/ ��a�Á����wKèQ��Í�èéæ8ä0���G�UyÎw_Í1��ä�}m~G�G��ymwz��æIÔ;çt�U{UêB�BÉæF Æ VGV�ú�Õ\F ò äk�NêByiä��BÉ ò �KêBT�y���yä�G�·w-ä�y@�B{fÉä��Oä�}m~G�G�UyÎwz��æ�ëN�G��}m{Sä����By@�B~ÏHUèxwz{S�Kæ�A�C@ç ym�Øwzy�æÏ{Öñ gih * õBorq_s�õtorq_s�uxv ü¡�¢�£¤¢Q¡ 2 �õ43 265�5­´t¦¤±%¼a³87Eüµ�±%´ç«�¯�«­ý#þ��ú:9¬¯ª±#¥�¦¸¨_©¬«­¦¸®ÿ¯ª±�¨¸²5³­´tµ�¶B³<;

¦¸¨ª¹ º�»�«­¼i½B¯�¦R«­¶ã {ä�×B×B���aä��]�6wU�B{ØA�C@ç }@{gä��U�Bym�B~§×·�U�G�B}m{gæI��Í6��{S×B��{S��{S��wzym�B~%äËA�CuçtëN�G�Næ�ê·} äOä��&ä§}m�Ýå�èÉB{f~G�U{f{�×��G}ÎÍÒ�B�æÏy ä�}BwU�B�U{f�U�B�G}mÉ1ë=êB�B�\wzy@���æ�9då�2å¸{��G�·�U{S��ÐG{Öy@� ë {S�\wzy@����ñÁF§åR{��Säk�êB�U{�T��B�¢åM�×a�G}mÍÒ�B�æÏy äk}¥èQwUy�æÏ{4ä�}@~G���UymwU��æÏ��ëN�G�1}m{Sä����By@�B~�}my@�B{gä��æwz�·�U{S���B�G}mÉ�ë=êB�·�fwzym�G�B�1wz�M}m{Sä����K×a�G}mÍÒ�B��æ|y�ä�}wz�·�U{S���B�G}mÉIë=êB�B�fwUy@�G�·����ë�ÉB{S~��U{S{Ì�y@�ËwUy�æÏ{øJigæh>=�v�� ò ��êB�fF4ê·×B×�{f������êB�BÉB�­���âwU�B{�ÉB{f~G�U{f{���ë×a�G}mÍÒ�B�æÏy äk} wz�B��{S���B�G}@É�ë=êB�B�\wzy@���B�­åM�Bym�]�5�S��æ|×�êÁwz{|A�Cuç wU�-ä��·�U} äÅwz{ÏÉBy@��{S�fwU}mÍâym��wU���a�êB�·ÉB��G�Øwz�·{Þ�KêB�B�Bym�B~wzy�æÏ{­��ë�äA�Cuç=}@{gä��U�·y@�B~7ä�}m~G�G��ymwz��æ��

�­ym{UåMy@�·~âA�Cuç ëN�G�Kæ�êB} äk{1ä���×a�G}ÎÍÁ�·�æ|y�ä�}%wz�B��{S���B�G}@ÉFë=êB�·�fwzym�G�B��y�æ�æÏ{SÉBy�äkwz{f}mÍËÍÒym{S}@É·�&ä�B{¨åöy@��wz{f�U×B��{fwPäÅwzy@������ë>wz�B{KA�Cuç }@{gä��U�·y@�B~�ä�}m~G�G�UyÎwz��æÏ�è��ë%À¸�U�B��ê·w_Í�ä��BÉ ò ä��KêBy4ä��BÉ ò �KêBTèy���y�� ë ym�B�S{�ä���Í@?¢è_ÉB{f�Sym�Uy@����}my@��w�y@��{fe�ê·ymÐ�ä�}@{f��wÞwU�Oä�×a�G}ÎÍÁ�·�æ|y�ä�}>wU�B�U{f�U�B��}@É�ë=êB�B�\wzy@������ëRÉB{�è~G��{S{A?ËÔ;ø%�B�U{f��ëN{gê·�]��wØZ\` ß ÙÎí@F Æ VGWGV�ÕàFiy@��wz�B{�}�ä��B~ê�ä�~G{ø��ëR×a�G}ÎÍÁ�·�æ|y�ä�}æwz�·�U{S���B�G}mÉIë=êB�B�fwUy@�G�·�À¸���B�ê·w_Í�� ����wz�KêB�fwNêB�-ä�}i�U{S�KêB}mwèy�æÏ×B}@ym{S�%wz�¬äÅwkä���Í�HUèQwU{S�Kæ�A�C@ç6�Sä��Ø�a{­{àì·×·�U{S���U{fÉIä���ä�×a�G}ÎÍ�è�B�æÏy äk}Bwz�B��{S�U�·�G}@É8ë=ê·�B�fwUy@�G�|��ë1ÉB{f~G�U{f{­L�Ô � J�}m�G~%J�}@�G~RHpÕ ü ý �EwU�B{å�gä��U{å��ë ò ä��NêByB� ò �Nê·TÒyC��yxF�ymw�Säk����{R�U�B�¢åM�7ä���ä¬�f�G�U�G}m} ä���Í���ëBwU�B{Sym�>��{S�KêB}mwU�>wz��äkw�äk��Í�HUèQwU{S�NæöA�C@ç��gä����a{¸{\ìÁ×B��{S���U{SÉ�ä���ä

Æ V

Page 32: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

×a�G}mÍÒ�B�æÏy äk}�wz�B��{S���B�G}@É­ë=êB�B�fwUy@�G�K��ëtÉB{f~G�U{f{�L�Ô � J�}m�G~RHpÕ ü ò ��êB�SF�{gä��]����ëBwU�B{S��{è{Sä���}@ym{S��}@{gä��U�§èy@�·~Iä�}m~G�G�UyÎwz��æÏ�Öy�æÏ×B}@ym{S��ä��=LEÔ � J�}@�G~�JæÕ�êB×B×a{S�ø���ê·�BÉ5�G�5wU�B{EÉB{f~G�U{f{Ø��ëkäË×a�G}mÍÒ�B��æ|y�ä�}wz�·�U{S���B�G}mÉ�ë=êB�·�fwzym�G�FëN�G�7ä���Í ×a�G}mÍÒ�B�æÏy äk}¥è_��y@?f{EA�C@ç¸�9d �NêB�·��wPäk��wUy ä�}�~ä�× ��wzym}@}è��{gæ�ä�y@�·{SÉæFwz�·�êB~G�æFi��{\wnå¸{f{S��wU�B{S��{L�Ô � J�}m�G~�J1Õ@êB×·×�{f�����ê·�BÉB�0ä��BÉËwU�B{êÖÔbJ4D OQP Õ¬}@�¢åR{S���a�êB�BÉ�É�êB{wz�|D�ym�B�UT�ÍOä��·É§ãiä�×�{f��wS�ED¡�¢�£¤¢�F G<»­´IH µ�¶B»�¨Q½B¶J êB��ðt�U��w��U{S�KêB}mwMym�èwz�B{�ëN�G}m}@�¢åMy@�·~øwz�B{f�G�U{Uæ�ÂKML4N)O�PQNSR T.U�VXW à ò�HGá_`_Z\[\ÚZY\[^] cQ_�Z\[�ÿÅÄ �ÇÆ �¢õYÛ à�` ÜÒÛ£õ]ÜËZ ß õ]Ü�õ]c à"aàߧà õ\`bÛxc à Ûwó�c Ý ó\Û�bÅZ ß `ÚEc8ó\`ZÜQõ ß�àdc Z�Zfef^a[zZPóPó�Z á8ß ó ß ^tcÅÙWò à cÅÚ�Û ß Ùæ` ÜÒ[]ZPó]ÜtcÅÙ áMÝàߧà õ\`bÛxc à c Ý�á Zhg�[zZ]Z¬L�Ô � ÜÁ}@�G~¤HpÕ üd�êB�U{¨ë=êB}�ëN{SäkwNêB�U{��ë ò �B{f�G�U{UæIð·� Æ ym�­wz�¬äkw�wU�B{ÉB{S~��U{S{Ï�a�êB�BÉâÉB{f×�{f�BÉB�Þ�G� � Ü@åM�·y@�]�

�Säk���a{7æ�êB�]�6�Kæ8äk}@}@{f�¬wU�¬ä�� � J ü õ¸}m�G�U{ym�B�U×a{S�\wzy@������ë¸wz�·{Ï��{S�KêB}mwU�ÞÉ�êB{ÏwU� ò ä��KêByi� ò �NêBT�y���y�U{\ÐG{gä�}@��wz��äkw�ä��Uy�æ|ym} ä��Mwz�B{f�G�U{Uæ7�Säk���a{ÉB{S��ymÐ�{SÉ�ëN�U�æ)wz�B{fy@�0ä���ä�}mÍÒ�Uym�S�Kdå��ä�É·Ð�ä���wPäk~G{���ë�êB�å×B�U�Ò��ë¬Ô=åM�·y@�]��y@�å�U{S}¥ë è_�f�G��wPä�ym�B{SÉ�äk�BÉ�ÉB�Ò{S�­�B�kw�êB�U{7äk×B×B�U�ÇìÁy�æ�äkwU{øy@�·�S}�ê·�Uy@���§è_{àì·�f}�êB��y@�G��G�K���Ò�G��wUy@�B~�ÕÖym�Kwz��äkw�yÎw��Bym~G�B}@ym~G��wz��wz�·y@�øÉ·{S×a{S�BÉB{f�B�S{åM�Bym�]�=×B} äÇÍÒ�&ä§�f�NêB�fy ä�}¤�U��}@{Ïy@�5�ê·�} äÅwz{S����{S�KêB}mwU�S�ã {�wU�B{S�OêB�U{ ò �B{f�G�U{Uæ3ð·� Æ wz�~GymÐ�{��U{\ÐG{S�Nä�}��B{Uå �U{f�NêB}Îwz�kä��a�ê·w�wU�B{ÖÉB{S~G��{S{Ö��ëä×a�G}mÍÒ�B��èæÏy ä�}awz�B��{S�U�·�G}@Éë=êB�B�\wzym�G�B��åM�By@�]�Y�S��æ|×�êÁwz{�Ð�ä��Uym�êB���S}�ä��U��{S�M��ë À¸�Ò�G}@{gä��1ëN�G�Næ"êB} ä��f�ÀRÍ��f�æ��Bym�By@�B~ ò �B{S�G��{gæÃðÁ� Æ åMyÎwz�0ä�ÉB{S�f�æÏ×��G��ymwUy@�G�åwU{S�]�B�·y@e�êB{�É�êB{9wU�¬ÀR�U�B�êÁw_ÍÏÔ�À¸���B�ê·w_Í�F

Æ VGV���Õ2å¸{Ö�G�ÁwPä�ym�Y��êB��æ8äky@�Ø�U{f�NêB}ÎwÇÂKML4N)O�PQNSR T.U�jkW à ò�H�á_`_Z\[àÚlYm[^] cQ_�Z\[2ÿÅÄ �ÇÆ � õ õ ß�ànc Z�Zfef^a[]ZPóPó�Z á�ß ó ß ^tcÅÙWò à cÅÚ�Û ß Ù�`NÜÒ[zZPó]ÜÒácÅÙ á�Ýàß�à õ\`�Ûxc à c Ý­á Zhg�[zZ]Z�LEÔbJ D OQP }m�G~¤HpÕ üò �B{f�G�U{Uæ�ð·� ñK{S���U{f��wUy ä�}m}mÍÏ�S}m�G�U{f�¸wz�·{å~aä�×1åM�·y@�]�1å.ä��è}@{¨ë w¸��×�{f�Ø��ÍÏwz�·{�L�Ô � J�}m�G~�J1Õ^ê·×B×�{f��a�êB�BÉB�äy�æÏ×B}my@�SyÎwæym�EÔbÀ¸�U�·�ê·w_ÍGF Æ VGVG�·ï ò ä��KêBy�ä��BÉ ò �NêBT�y���y�F Æ VGVGV�Õàï�ymw����B�¢åM�>wU�¬äkw�wz�B{¸D�ym�B�UT�Íaèã�äk×�{f��wè}m�¢å¸{f�R�a�êB�BÉ|y@��y@��ë,ä��fw�wUy@~G��wÇF�êB×|wz�­äK}@��~aä��UyÎwz��æÏy@�^ë,ä��fwU�G�SF�ëN�G� ß Ù ÙG×a�G}mÍÒ�B��æ|y�ä�}¥èQ�Uy@?f{A�CuçR� ò �B{f�G�U{Uæ�ð·� ð&ä�}@����ÍÒy@{S}mÉB��äñ�gih õ jmlNn orq_s õÇv?èxwzy�æÏ{@ä�}@~G���UymwU��æ|ëN�G�è}m{Sä����By@�·~�×���}mÍÒ�B�æÏy ä�}Ùè�Uym?S{¬A�Cuç¸FåM�·y@�]�Ey�æ|×·�U��ÐG{f�R�G�Ïwz�B{@ä�}m~G�G�UyÎwz��æÏ�9��ë1À¸�U�B��ê·w_Í�ä��BÉ ò ä��KêByi� ò �KêBT�y���yûä��·ÉEy@��wz�·{ë,ä���wz{S��w�TÒ�·�ÝåM�5ä�}@~��G�UyÎwz��æ�ëN�G�èwz�·{�ê·�B�U{f��wz��y@�\wz{SÉ�A�Cuç=}m{Sä����By@�B~�×·�U�G�B}m{gæY�ã {Ö�gä���y�æ|×·�U��ÐG{�êB×a�G��wz�B{Ö�a�êB�BÉ·����ë ò �B{f�G�U{Uæ ð·� ñ�ëN�����U{Säkɧè_�G�·�S{ÞA�Cuç¸Â

oqpBr�s+t�t�rhuvrhwfxys>z|{�}~w�s>��wf�1� �������>�������Q���Sr�Qu�r���s>zQt���yu�zQx���u���wf��r���u���w�u�z���{¡  ¢)�£�>¤v¥~w���¦-¤�§�zQxy¨E¤$��©«ª­¬¦�§0r�rh��s+t�¥~u$t.s�z\wf��h¤���{�}~w�s>��wf���Q®�¯�¯$¯$�q°ñ�Ä

Page 33: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

KML4N)O�PQNSR T.U�TkW à ò5[zZ ßaá áUc à õ]Z±Ym[^]3cQ_�Z\[OÿÅÄ �ÇÆ � õ õ ß�à²c ZâZfef^a[zZPóPó�Z á ß ó ß ^tcÅÙWò à cÅÚ�Û ß Ù` ÜÒ[]ZPó]ÜtcÅÙ áMÝàߧà õ\`bÛxc à c Ý�á Zhg�[zZ]Z¬L�ÔNJ­D OQP }@�G~ ú OQP JæÕ üç y@��ä�}@}ÎÍGF·��y@�B�f{"äÏA�Cuç9ëN�G�Kæ�êB}�ä�y@�M�Uy�æ|×B}ÎÍäÏÀ¸�Ò�G}@{gä����Sy@���gêByÎwM��ë9É·{S×·wU��ñÁFtyÎwMy@�M�¬äkwNêB�-ä�}

y@�#wz�Bym�E�f�G��wz{\ìÒw�wU�9ä�}m�U���S�G�B��y@ÉB{f�ØÀ¸�Ò�G}@{gä��#�Sy@���gêByÎwz�Ï��ë�ð·ìÁ{SÉ É·{S×·wU� Ì ð ü ã {�åR�êB}mÉêB}Îwzy�æ�äkwU{S}mÍ­}@y@Tk{¤wU�­×B���pÐ�{�êB×B×a{S�9�a�êB�·ÉB�9�G�øwU�B{èÉB{f~G�U{f{è��ëç×a�G}mÍÒ�B��æ|y�ä�}Gwz�B��{S���B�G}@É­ë=êB�B�fwUy@�G�·�ëN�G��äk�U�ByÎwz�-äk��Í��S�G�B��wPä���wÞÉB{f×·wz���Sym�U�gê·ymwz�fï��B�¢å¸{\ÐG{S�Öwz�·y@���U{f{gæÏ��wz����{�e�êByÎwz{�ÉByÎÓÏ�gêB}ÎwÇ�ød¬�0äðt����w���wU{Sקym��wz�Bym�MÉBy@��{S�fwUy@�G�iF§åR{K×B����ÐG{KML4N)O�PQNSR T.UE³ ]>cÅ[å̧�ð � ß�à òØ[zZ ßaá áUc à õ]Zµ´åc�cÅÙ@Z ß�àÏÝ cÅ[àÚ ß Ù ß c Ý&á Z_^a`NÜØÌ cQ_�Z\["ÿ�¶ ��·è��¸ �¹Nß�àdc Z ¹ cÅÚå^ ß `ªZ ánc�º1ß ^tcÅÙ ºdà cÅÚ�Û ß Ùæ` ÜÒ[]Zv»]ÜtcÅÙ áMÝàߧà~¹ `bÛxc à c Ý­á Zhg�[zZ]ZI¼L|ÔbJ Dq( jnh½ ¡¾ ¿ n Õ üò �B{S���U{gæ7ð·� î�y�æ|×B}my@{f��wU�¬äkw�wz�·{��S} äk�U����ë¤�U{gä�ɧèQ�G�B�S{��f�G�B��wPä���wÞÉB{S×Áwz��ëN�G�Næ"êB} ä��å�Sä��â�a{}@{gä��U�·{SÉ�y@���Nê·��{àì·×a�G�B{f��wzy äk}�wUy�æÏ{������À Áó�ÃÂ�ÄS "Å  Ç��Æ��� �Â\Ç÷ {S�Säk}@} wU�¬äkw­ä�È�Û¡» aàß1É~¹ `bÛi_�Z É cÅ[àÚËÊ�Ù Ý cÅ[àÚ°ëN���Næ�êB}�äâÔ�A�C@ç%Õåy@�0ä�ÉBy@�i�KêB�B�\wzy@���dÌ D ·ÎÍ0Í0ÍÏ· Ì u��ëÞ�S���Q�KêB�B�\wzy@���B�Ø��ëøÀ¸�Á��}@{Säk�¿}@yÎwz{S�Nä�}@�f�ödå� HUèQwz{f�Næ A�Cuç y@�Ø���B{IåM�By@�]� �¬äk��äkw�æ|����wYH�S���Q�KêB�B�\wzy@���B��Ô,ä�}@����T��B�¢åM�*ä��Ïwz{S�Kæ|�zÕ�ä��·ÉÐä�Ðéè_A�Cuç�y@�Ï�G�B{§ym�©åM�Bym�]�¿{Säk�]�<wz{f�Næ y@�Ï��ë�Uym?S{�äkw0æÏ�G��w�Ð ü dÿA�C@çöÔb�G�ÞÀ¸�Ò�G}@{gä��IëN�G�Kæ�êB} äGÕ�y@�[zZ�ÊÑÈ�áUc É~¹ Z¬y�ë¤ymwÖ�f�G��wPä�ym�B�0äkw�æÏ�G��wÖ�G�·{�Ò�S�gê·�U�U{f�B�S{Þ��ë>{Sä��]�YÐ�ä��Uy�ä��B}@{k�dlÒ�á�ÈÒZ ¹ Û¡»\Ûxc É ÙôÛ¡»\`�ym��ä�}@ym��wyÓFï Ô�Ì D � ì D Õ � ügüSü � Ô�Ì�Ô � ì0Ô¸Õ^åM�B{S��{�{Sä��]�ÕÌ�ö1y@��äøwz{S�Kæ°��ë>��y@?f{äkw�æÏ�G��wµÒ�ä��BÉâ{Sä��]�{ì¢ö�y@��äØÀ¸�Ò�G}m{Sä��Ië=êB�·�fwzym�G���G�9ÿÅÄ �SÆ � õ ü4Ö yÎÐG{f�%ä��ây@�B×�êÁw�ô þÚÿÅÄ �ÇÆ � õwz�·{­Ð�äk}�êB{­��ë«Ó�Ô?ô�Õ¸y@�kì�×kÔ?ô�Õ2åM�B{S��{ ƵØÚÙ y@�è�KêB�]��wz�¬äkwyÌ£×kÔ=ô�Õpï Æ äk�BÉÕÌ�ö_Ô=ô�Õpï Ä"ëN�G�&Û�� Ù üý ë«Ì�öªÔ?ô�Õpï Ä&ëN�G�kä�}@} ÆÜØ Û ØÞÝ wz�B{f�ßÓ�Ô=ô�Õpï Æ üdàÒ�á�ÈÒZ ¹ Û¡»\Ûxc É `�[zZ]Z�y@��äÏ�U�Ò��wz{fɧ�Bym�¬ä���ÍØwU�U{f{�åM�B{f�U{�{gä��]�Ëym��wU{S���¬ä�}��B�ÒÉB{K��ä��­ñ�]�By@}mÉB�U{f�ä��Bɧy@�M} ä��a{S}m{SÉOåMymwz�5äÏwU{S�Næ ��ë9�Uy@?f{"äÅwuæÏ�G��w\Ò5ä��·É§{Säk�]��}@{Sädë9y@��} äk��{f}@{SÉ�åMymwU�Iä|À¸�Á��}@{Säk�ë=êB�B�\wzym�G�æ�äd=ÉB{S�fy@�Uym�G��wz��{S{��U{f×B�U{f�U{S��wU��äMÀ¸�Ò�G}@{gä���ë=êB�B�\wzy@���­äk�ZëN��}@}@�¢åM�fÂæy�ë�wz�·{¤���Á�kw1y@�1}�ä���{f}@{fÉåMymwU�äøwz{f�NæáÌ wz�B{f��wz�B{f��wU���S��æ|×�êÁwz{¬wU�B{­Ð�ä�}�ê·{å��ë1wU�B{åwz��{S{­����ä���y@�·×�ê·w.ô5þlÿÅÄ �ÇÆ �¢õ­åR{~G�Þ}@{¨ë w^ëN���æ wz�·{¬���Á��w�y¥ë~ÌøÔ=ô�Õpï Ä"ä��BÉ|~G�Þ�Uy@~���w¤y¥ë�ÌøÔ=ô�Õpï Æ ü ã {��S�G��wUy@��êB{�y@�wz�Bym�^ë,ä����By@���êB��wzym}æ�U{gä��]�Bym�B~8ä�}m{Sä�ë4â�}�ä���{f}@{fÉ1åMymwz�Y�U��æ|{�ë=êB�·�fwzym�G��ì"ã�ä��BÉØwU�B{S���ê·wz×�ê·w@ì"ã\Ô=ô�Õ üò �B{Ï[�Ê É�ä ��ë�ä�ÉB{S�fy@�Uym�G�Øwz��{S{�ÌÃy@��É·{Uðt�B{fɧym�BÉ�êB�\wzymÐ�{S}ÎÍä���ëN�G}m}@�¢åM�SÂå ý ë­ÌÃy@�kä���y@�B~G}m{�}@{gä�ë�wz�B{f�æ?�ç�J�ÒæÔiÌ­Õpï Ä ü

ñ Æ

Page 34: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

å ý ë­ÌÃ�¬ä��M�KêB�·wz��{S{f�<Ì�èuäk�BÉ±Ì D wz�B{f�

?�ç�J�ÒæÔiÌ­Õpï éêë êìæ8äÅìÖÔ�?�ç�J�ÒæÔiÌ~èàÕ � ?�ç�J.ÒæÔ�Ì D ÕUÕ y�ëí?-ç�J�ÒiÔ�Ì�èàÕ�îï6?-ç�J.ÒæÔ�Ì D Õ?-ç�J�ÒiÔ�Ì�è\Õ�ï Æ ��wz�B{f�KåMym�U{��

ò �B{0ëN�G}@}m�ÝåMym�B~�}m{gæ�æ8ä­åMym}@}���{0êB��{Uë=êB}�Âð N1RñRñòóT�U�ôàõ�ö8÷�ø­Rúù�VSû�û�j~ü�ý Z\`~Ì c Z\Êÿþ�á�ÈÒZ ¹ Û¡»\Ûxc É `�[zZ]ZÖc Ý [fÊ É�ä ? ` Ütc0»�ZåÙ@Z�Ê-_�Zv»mÊ�[zZ¬Ù�Ê�ác Z\Ù@Z�È ` Û ` ÜÞ`NÜtZ ÝàßSÉ~¹ `bÛxc É »iì D � ügüSü � ì�� ü�� ÜtZ É ` ÜtZ\[zZ¤Û¡»yÊ É ?Áá�ÈÒZ ¹ Û¡»\Ûxc É ÙôÛ¡»\`æÔiÌ D � ì D Õ � üSüSü � Ô�Ì�� � ì��æÕ` ÜÒÛ ¹ Ü�Û¡»�Z�� ß Ûi_QÊ�Ù@Z É `�`ªc|Ì üd6^tcÅÙ º�É cÅÚ�Û�Ê�Ù�`NÜÒ[zZv»]ÜtcÅÙ�È ÝàßSÉ~¹ `�Ûxc É ym�1ÉB{Uðt�·{SÉÞ��Í�ä�æ�êB}mwUymÐ�ä���y äkwU{ä×a�G}ÎÍÁ�·�æ|y�ä�}àÊäÔ=ô D � üSüSü � ô õ Õ üò �B{�Ð�äk}�êB{M��ë�wz�B{�×���}mÍÒ�B�æÏy ä�}Òwz�B��{S���B�G}@É�ë=êB�B�\wzy@���|�G�|y@�B×�ê·wpôIþlÿÅÄ �SÆ � õ y@���Uym~G��ÔôÊäÔ=ô�Õ�Õ � yx� {��

Æ y�ë�ÊäÔ?ô D � üSüSü � ô õ Õ¸ Ä�äk�BÉy@�è� Æ ��wz�B{f�KåMym�U{�� ò �B{ÜÈÒZhg�[]Z]Z>��ë�ä�×a�G}mÍÒ�B�æÏy äk}�wU�B�U{f�U�B��}@É&ë=êB�B��èwzym�G�Eym�R��y�æÏ×B}mÍøwz�B{¬ÉB{S~G��{S{­��ëæwz�B{¬×���}mÍÒ�B�æÏy ä�}GÊæ� ý ë�{Sä��]�E�S�Ò{fÓÏ�Sym{S��w�ç����ëæwU�B{­×a�G}mÍÒ�B��æ|y�ä�}y@�käk��ym��wU{S~G{f�SF·wU�B{S��wz�·{ ` Z\Û�gÅÜÒ`i��ë>wU�B{Ö×���}mÍÒ�B�æÏy ä�}çwz�·�U{S���B�G}mÉOë=êB�·�fwzym�G��y@�� � ç���� ü¡�¢_ûè¢�£ ��°¼�¯�µä«¸½/2 ¨ªýR¦¤´ç¯_½B³�® ¶��à¦R´Øüµ�±%´ç«�¯�«­ý#üϯ�«­µ�±%´Y²5³­´tµ�¶B³­¦R¨ª¹ º�»�«­¼�;

½t¯ª¦R«­¶J êB�ä}@{Säk�U�Bym�B~M�U{S�KêB}mwU�iëN�G�äA�C@çY�B��}@Éøym�ÞwnåR�@åMymÉB{S}ÎÍÖ��wNêBÉBy@{fÉø}m{Sä����By@�B~�æÏ�ÁÉB{f}@�fÂæwz�B{�XR[]c c Ê c Ù ºW ^�^a[]c$eGÛ ÚËÊ�`_Z\Ù º�� cÅ[à[]Z ¹ `�Ôbã�d�õ�Õ�æ|�ÒÉB{f}9y@��wz���ÁÉ�êB�S{fÉ���Í@�^ä�}my ä���wÏÔi�^ä�}@y�ä���wÇF Æ V�W�î�Õ�ä��BÉâwz�·{æÏ�ÁÉB{f}���ë|Zfe)Ê ¹ `ÞÙ@Z�Ê�[ É Û É g Ý [zcÅÚ Z�� ß Ûi_QÊ�Ù@Z É~¹ Z�� ß Z\[àÛxZv»�ym��wz�U�ÒÉ�êB�f{SÉÑ��Í#d¬�B~G}�ê·y@� Ôbd¬�B~G}�ê·y@�æFÆ VGW�W�Õ�äk�BÉ ¾ ymw�wz}@{f��wU�G�B{§Ô ¾ ymw�wz}@{f��wU�G�B{�F Æ VGW�W�Õ\� ò �B{�ãad�õ }@{Säk�U�Bym�B~�æÏ�ÒÉB{S}på�äk�øÉB{¨ðt�B{SÉ ym�ë {f�fwzym�G�<ñÁ� ñÒïkåR{§�·�Ýå��B�Uym{��BÍ=É·{S�U�f�Uym��{�wz�B{OæÏ�ÁÉB{f}���ë­{àì�ä��fwE}@{Säk�U�Bym�B~IëN�U�æ�{fe�êBymÐ�ä�}m{S�B�f{e�êB{S��y@{f�S�ý �5wz�B{8æ|�ÒÉB{S}¤��ë�{àì�ä��fw}m{Sä����By@�B~�ëN�U��æ {fe�êBymÐ�ä�}m{S�B�f{�e�êB{S��y@{S�fF%}@{gä��U�Bym�B~§×·�U�Ò�S{S{fÉB��y@�lä�U{fe�êB{S�B�f{å��ëi��w-ä�~G{S�f� ý �Ï{Sä��]�|��wPäk~G{�wz�B{M}@{gä��U�·y@�B~0ä�}m~G�G��ymwz��æ �KêB��æÏymwz�päk��Z�� ß Ûi_QÊ�Ù@Z É~¹ Z�� ß Z\[ ºÔ;äÀ¸�Ò�G}m{Sä��8ë=ê·�B�fwUy@�G���çÕRwz��wU�B{­wz{gä��]�B{S�f� ý ë���y@��{Se�êByÎÐ�ä�}@{f��w¸wz��wz�B{­w-ä��U~�{fwM�S���B�S{f×·w���wz�·{S�wz�·{Ewz{gä��]�B{S�7äk�B�KåR{S���� ��0ø ë �%ä��·É=}@{Säk�U�Bym�B~§�¬äk}mwz�fï���wz�B{f�KåMym�U{EwU�B{Ewz{gä��]�B{f���U{f�BÉB��¬ä��]T{ä×a�Gy@��w�ô�þlÿÅÄ �ÇÆ � õ �KêB�]��wU�¬äkw���Ô=ô�Õ�îï��kÔ=ô�Õ ü dö}m{Sä����By@�B~7äk}@~G�G��ymwU��æ � Ù@Z�Ê�[ É » ¹ c É~¹ Z_^a` ¹ Ù�ÊQ»v»� Û É `bÛ ÚEZ<Ð�y�ë4ëN�G��!�}m}��0þ�� � !�}@~��G�UyÎwz�#"$� �%!��§{àì&!��fwU}mÍ�y@É·{S��wzy¥ë Í�wz�B{Öw'!k�U~G{\w��­ym�(!Åw�"Ï�G��wÐ�wzy)"|{Þ��wU{S×B�fF+*B��y@�B~,!kw-"Ï�G��w<ÐM{Se�*ByÎÐ�!�}@{f�B�S{Ke.*·{S�Uym{S�fF�/MymwU�§��ÍÒ×���wU�B{S��{S�0�(/M�Bym�]��{1!��]���%!���a{Þ��{S×B��{S��{S��wz{fÉ(/MymwU�/и�BymwU�2!��BÉY�1!��Y�a{�{fÐ�!�}3*#!kwz{fÉ��G��!���ÍØ×a�Gym��w�ô5þlÿÅÄ �ÇÆ � õ y@��wUy3"Ï{\Ð üò �B{54N�G}@}m�6/Mym�B~748!��\wMy@�9/R{S}m}1TÒ�·�6/M�iÂ

ñGñ

Page 35: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

: ò<;>=nT.U3?àõfö8÷ø­RñN)PA@B-C<DFE�ùíVSûHG�û+IHJlò�ò<K1Kmò<LNMdK&ø4PQò#L ùíVSû�û£³.ü9O Énc cÅ`NÜø` ÜtZ¸X W � ÚEc�ÈÒZ\ÙÊ É ÈE` ÜtZÞÚEc�ÈÒZ\Ù c Ý Zfe)Ê ¹ `RÙ@Z�Ê�[ É Û É g Ý [zcÅÚ/Z�� ß Ûi_QÊ�Ù@Z É~¹ ZP� ß Z\[àÛxZv»'Q¤` ÜtZ\[zZMÊ�[]ZMÊ�Ù gÒcÅ[àÛ ` ÜÒÚM» ` ÜÒÛ ¹ ÜÙ@Z�Ê�[ É ` ÜtZ ¹ Ù�ÊQ»v»�c Ý ÙôÛ É Z�Ê�[�`NÜÒ[zZv»]ÜtcÅÙ�È ÝàßSÉ~¹ `bÛxc É »�cQ_�Z\[�ÿÅÄ �ÇÆ � õ Û É `bÛ ÚEZ�^tcÅÙ º ÔNJ1Õ üò �B{,!�}@~G���UymwU�#"Ï�­�>4SRH!��\w�ðÁ� ��!��U{Ï�<!���{SÉ5���6×a�G}mÍÒ�B�T"ÏyU!k}>wzy)"|{}my@�B{%!���×B�U��~G�V!>"A"Ïy@�B~B�XW�{/My@}m}i�B{f{SÉYwz�B{Y4N�G}m}@�Z/My@�B~ø{\ìÒwz{f�B�Uym�G����4[RH!��\w�ð·� �ÁÂ: ò<;>=nT.U]\ký Z\`­� c Z|Ê ¹ Ù�ÊQ»v»Þc Ý ÝàßSÉ~¹ `�Ûxc É »ÞZ�Ê ¹ ÜYc Ý\` ÜÒÛ ¹ Ü ¹ Ê É�c ZKZfef^a[]Zv»v»�Z�ÈÿÊQ»|Ê É ÈÒZhg�[zZ]Z\áNÌ^tcÅÙ ºQÉ cÅÚ�Û�Ê�Ùa` ÜÒ[zZv»]ÜtcÅÙ�È Ýàß1É~¹ `bÛxc É cQ_�Z\[kÿÅÄ �ÇÆ �Z^ ü_� ÜtZ É Û Éßc cÅ` ÜE` ÜtZ�X W � Ù@Z�Ê�[ É Û É g�ÚEc�ÈÒZ\Ù�Ê É È` ÜtZåÚEc�ÈÒZ\Ùæc Ý Zfe)Ê ¹ ` Ù@Z�Ê�[ É Û É g Ý [zcÅÚ�Z�� ß Ûi_QÊ�Ù@Z É~¹ Z�� ß Z\[àÛxZv»'Q9` ÜtZ\[zZ¬Û¡»�Ê�Ù@Z�Ê�[ É Û É gÿÊ�Ù gÒcÅ[àÛ `NÜÒÚ Ý cÅ[� ` ÜÒÛ ¹ Ü�[ ß1É »ÞÛ É `bÛ ÚEZ�Jægih>=�v ü` PQO~Oba�c Rt�G��wz�B{�"|�ÒÉB{S}a�>41{àì&!��fw�}m{1!����By@�·~04N���T" {Se�*BymÐ�!�}m{S�B�f{åe�*B{f�Uy@{f�SF&/R{­�d*B�Ewz�·{å×a�G}mÍÒ�B��e"ÏyU!�}çwzy)"|{Y!k}@~G�G��ymwU�#"f4N�G�è}@{%!��U�·y@�B~�}my@�B{%!��RwU�B�U{f�U�B�G}mÉg4h*B�B�fwUy@�G�·����Ð�{S��!���{\ìÁ×<!��·ÉB{SÉ�ÐG{f�U�Uym�G��>4�wz�·{�y@�B×&*·w��U×#!��S{k� ë ym�B�S{�� ú5i ��4N�G�­�5þÐÿÅÄ �ÇÆ �j/R{Ï�%!�����*·×B×����U{j/MymwU�B�T*·wå}@�G���­�>4¤~G{S�.e{S��!�}@yÎw_ͧwU�<!kw�wz�·{wV!��U~�{fwÞ×���}mÍÒ�B�T"ÏyU!�}�wz�·�U{S���B�G}mÉ�4h*B�·�fwzym�G�ây@�0!g"k*B}Îwzy@}my@�B{%!���×��G}ÎÍÒ�B�T"ÏyU!�}ä�>4ÉB{f~G�U{f{|Ì ü ë *B�]�l!�×a�G}ÎÍÁ�·�T"|y3!�}1wz�B��{S���B�G}@Ém4h*B�·�fwzym�G���1!k���a{�ÐÒy@{n/¸{fÉo!��5!Ø}my@�B{%!��¬wU�B�U{f�U�B�G}mÉ4h*B�B�\wzym�G����ÐG{S��wU�B{��U×#!��S{�>4_!�}m}p"k*·}mwzym}@ym�B{1!��2"Ï�G�B�T"ÏyU!k}@�M�>4RÉB{S~G��{S{X!kw-"|�G��wÖÌ ü�q �B{f�U{A!���{r i =sut D[v ^ sxw Ø J = ��*B�]��"|���B�T"ÏyU!�}m�y!��·ÉY�·{S�B�f{Þ��Í�Rb!k�fwMð·� �7/R{Þ�%!���}@{%!��U�Ø�d*B�]�(!�×a�G}ÎÍ�e�B�T"ÏyU!k}Rwz�·�U{S���B�G}mÉz4h*B�·�fwzym�G�#y@�=wz�B{{"Ï�ÁÉ·{S}��>4¬{\ì&!��\wE}m{1!����By@�B~|4N���T"ò{fe.*·ymÐ�!�}@{f�B�S{�e�*B{f�Uy@{f���Í��d*B�B�Bym�B~X!�×���}mÍiÔ r ÕxeQwUy3"Ï{�!k}@~G�G��ymwU�#"}4N�G�è}m{1!����By@�B~K}@ym�B{1!��¤wz�B��{S���B�G}@É�4h*·�B�fwUy@�G�B�¸�pÐ�{S��wz�·{ÉB�T"~!�ym�IÿÅÄ �ÇÆ ����/M�B{S��{ r Ø J~= üRt�G��wz�B{âãad�õ)}m{1!����By@�B~�"Ï�ÒÉB{S}�F2/R{��1!���*B��{l!=��w'!��BÉ#!��UÉ wz��!��B�d4N����"~!kwzym�G��4N�U�>" wz�·{"Ï�ÁÉB{f}��>4ç{\ì&!��\w9}@{%!��U�·y@�B~�4N�U�T"Ã{fe.*·ymÐ�!�}@{f�B�S{Re�*B{S��y@{S�>wz�åwU�B{èãad�õ=}m{1!����By@�·~�"Ï�ÁÉ·{S}�Ôbd¬�B~G}3*·y@�æFÆ VGW�W�Õ\� ý wåym��{%!���Í�wU�ÏÐ�{S�Uy�4 Í�wz�<!kwMwU�By@��wz��!��B��4N�G��"~!kwzym�G��Ôbym��/M�Bym�]�§{fe.*·ymÐ�!�}@{f�B�S{Þe�*B{S��y@{f�-!���{�Uy)"k*B}3!kwz{fÉ���ͧwz{S��wzym�B~Øwz�B{���ÍÒ×��kwz�B{f�Uy@�0!�~T!�y@�B��w5!Ø�U{\wK��4¸{àì#!�"|×B}m{S��ÉB��!1/M��4N���T")wU�B{Ïã�d�õ�G��!��S}m{�Õ9×B�U{f�U{f��ÐG{f����*B�·�By@�B~Þwzy)"|{S*B×|wz�Þ×��G}ÎÍÒ�B�T"ÏyU!�}&48!k�fwz���U�S� då}mwU{S���<!kwzyÎÐG{f}mÍGF�*B��y@�B~Þwz�B{248!��\wÔbd¬��wz�B�G��Í�F Æ V�V�úGÕRwz�#!kwMwz�B{|��!�×B�BymTTe�õ¸�B{f��ÐG���B{S�BT�ym�åÉBy)"|{f�B�Uym�G���>4 wU�B{Ö�S}U!k�U���>49×a�G}mÍÒ�B�>"|y3!�}wz�·�U{S���B�G}mÉ�4h*·�B�fwUy@�G�B��>4�ÉB{f~G�U{f{§Ìâ��ÐG{f�|J¿Ð�!���yU!��B}m{S��ym� =sut è v ^ = wB� !�ÉBy@��{S�\w×B�U�Ò�>4��1!��#�a{~GyÎÐG{S��!�}m�G�B~�wz�·{Þ}my@�B{f���>4èÔ�À¸})*#"|{f�KZ\`yÊ�ÙÎí@F Æ VGWGVGÕ\�

¡�¢_ûè¢_û ²5³­µ�� ¯�«­¶&��©&;k¥§± 5Kµä´·½<ü¦í36µä´�� ¦R»�«­¹ý wÖy@�å�S}@{%!��åwU�<!kw0!���ÍËÉ·{S×·wU��e Æ �Sym�U��*Bymwå�pÐ�{S�&ÿ�¶ ��·è��¸ �E�%!��â��{�{àìÁ×B�U{f�U�U{fÉ�!��5!Ø×a�G}mÍÒ�B�>"|y3!�}wz�·�U{S���B�G}mÉ�4h*B�·�fwzym�G���>49É·{S~G��{S{ Æ � ý ���f�G��wz��!���wSFiDYy@�B��T�Í�!k�Bɧãp!�×a{S��w�~!ÇÐ�{7!êÖÔbJ D OQP Õ�}m�Z/¸{f�

ñ�ð

Page 36: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

�a�T*B�BÉÖ�G��wz�·{%ÉB{S~G��{S{��>4�!���Í­×a�G}ÎÍÁ�·�T"|y3!�}�wz�B��{S���B�G}@ÉY4h*B�·�fwzym�G�Y/M�By@�]�Ö�f�T"|×&*·wz{f��!M×#!���wUy@�%*·}U!���U{%!�É�eQ�G�B�S{ÞA�C�R¸�<Rt�G���S�>"|×B}m{fwU{S�B{f�U�S/¸{Þ~GyÎÐG{­wz�·{Sy@����y3"Ï×B}@{å×B�U�Ò�>4ª�KML4N)O�PQNSR T.U)Gàõ�J���LNK%���Þò#LpM ` ò#�ÃN)P6=ÏùmVSûH?HG~üÿý Z\`�� i Ì D · Í0Í0ÍQ· Ì�Ô c Z8Ê É Ý á_`ªZ\[àÚY\[<] cQ_�Z\[�ÿÅÄ �ÇÆ � ^ ` ÜtZ\[]ZÞZ�Ê ¹ ÜØ`_Z\[àÚ Ì s Û¡»µÊ ¹ c É"aàßSÉ~¹ `bÛxc É cQ_�Z\[Rî Ý ú _QÊ�[àÛ�Ê c Ù@Zv»'Q¤Z�Ê ¹ ÜÕ_QÊ�[àÛ áÊ c Ù@Z�Ê\^�^tZ�Ê�[�»YÛ É ^a[zZ ¹ Û¡»�Z\Ù º c É Z�`ªZ\[àÚ7Q Ê É È§J i î Ý P üA� ÜtZ É Ê É�º ^tcÅÙ º�É cÅÚ�Û�Ê�Ù�` ÜÒ[zZv»]ÜtcÅÙ�ÈÝàßSÉ~¹ `�Ûxc É@` ÜÒÛ ¹ Ü ¹ cÅÚå^ ß `_Zv»��ÑÚ ß »\`�Ü�Ê-_�ZÜÈÒZhg�[zZ]ZÜÊ�`¤Ù@Z�ÊQ»\` Ý ü` PQO~Oba�c ¾ {fw�ÊäÔ]� D � üSüSü � � ^ ÕÖ�a{,!Y×���}mÍÒ�B�T"ÏyU!�}>�>4èÉ·{S~G��{S{�ÌË�d*B�]�6wU�<!kw04N�G�0!�}@}[�jþ(ÿÅÄ �ÇÆ � ^/R{ø�<!ÇÐG{�ÊäÔh��Õ� ÄÏy��m����!kwzym�d�t{f�Y� ü Rt�G�<Û i ÆG� ügüSü ��Ý }@{\wY� s ��{Öwz�·{ø�U{\wå��4%î Ý ú Ð�!���yU!��·}@{S�/M�Bym�]�|!�×B×a{1!k�U��y@�Øwz{f��":Ì s ü ý wåym���S}m{1!���wz�#!kw�4N���2!���Í�×a{S�d"k*·wV!kwzym�G�B�9� D � üSügü � ��Ô ��ÐG{f��!�U{\w�>4¤�Uym?S{�î Ý ú � /R{��<!ÇÐG{�ÊäÔ�� D � üSüSü � ��Ô¸ÕÖ�Ä�y���ÊäÔ]� D Ô�� D Õ � üSüSü � ��Ô­Ô���Ô¸ÕUÕ��Ä ü õ¸���B�U{fe.*·{S��wz}ÎÍwz�·{Þ×a�G}ÎÍÁ�·�T"|y3!�} � Ô]� D � üSüSü � � ^ Õ i �� j�������� � �n� ÊäÔ]� D Ô�� D Õ � üSüSü � ��Ô�Ô���Ô¸ÕUÕy@���>4ÁÉB{f~G�U{f{y!kwN"Ï�G��wäÌ�!��BÉÞ�<!�� � Ô]� D � üSüSü � � ^ Õ� Ä�y��k���V!Åwzy@���t{S��� ü ë y@�B�f{ � Ôh��Õæym�ä��Í."~"Ï{fwU�Uy@�y@�Ïwz�B{å{S}@{�"|{f��wz�è�>4æ{1!��]���U{\w2� s � �G�B{¬�1!��Ø��wz��!�y@~G��wd4N�G�d/y!��UÉ·}mÍ�U�·�6/ wz�#!kw¸wU�B{S��{­y@�_!ø×a�G}mÍÒ�B��e"ÏyU!�}&?·Ô�{  j �S× �"Í0Í0Íf� �  � �S×fÕ�>4åÉB{S~��U{S{�!kwA"Ï�G��w|Ì6�d*B�]�<wz�#!kwË?BÔ��  j �S× � üSügü � �  � �S×fÕ i� Ôh� D � üSügü � � ^ Õ{4N�G�(!�}@}5��þâÿÅÄ �ÇÆ � ^ ü ý w|4N��}@}@�Z/M�{4N�U�T" wU�B{5ÉB{n�t�BymwUy@�G� �>4~� wz�<!Åw(4N�G�(!�}@}Ôç D � ügüSü � çÑÔèÕtþ ÿÅÄ �ÇÆ�� üSüSü � î Ý ú � Ô � /R{ �<!ÇÐG{Ú?·Ôç D � üSüSü � çÏÔ¸Õ Ä y�� �U�>"|{Îç s i î Ý ú ü¾ {fwYH�ÔiÐ�ÕÏ�a{�wz�·{|*B�ByÎÐ>!k�UyU!Åwz{Y×��G}ÎÍÒ�B�T"ÏyU!�}�?·Ôç D � ügüSü � çÏÔ¸ÕX/M�·{S�U{dç s i î Ý ú � ÔiЬ� Ôxñ�Ûè�Æ Õ�Õ ú 4N���8Û i ÆG� üSüSü ��Ý ü¡q �B{S�<wU�B{YÉB{S~G��{S{Y�>4ÖH§ym�A!kwA"|�G��wEñ�Ì � !��BÉz"|�G��{S��ÐG{f�A/R{§�#!�Ð�{H�Ô�Ä�Õ � H�ÔxñGÕ � H�Ôbî�Õ � üSüSü � H�Ô�ñ Ý Õ¬� Ä�!��BÉËHÒÔ Æ Õ � HÒÔbð�Õ � üSüSü � H�Ôxñ Ý � Æ Õ� Ä ü õ¸�G�B��{Se�*B{S��wU}mÍâH��<!k�!kwM}m{1!���w¬ñ Ý ��{1!�}1?f{S�U���SFt�U�Eñ�ÌÏ ÉB{S~�ÔxHpÕ¸ ñ Ý ü���]� ¢ � £¥¤§�% "Å ÆMÄ�¦°�Ï����� §G�Ï�©¨ ÂY¤��ÃÂ�Ç«Â����% S�|ª «©¬ ­ ®�¯

Á �ËÄ�¯Y�Ë�ËÅ  �ÆMÄ�°²±��ÃÂ�Ç�±Ë� ÄS� ­Y�������% f�|�@Çý �Ëwz�·y@���U{S�\wzym�G�|/¸{ø×B�U��ÐG{��>*B��"~!�y@�Y�U{f��*B}ÎwÇÂ�!���ÍËH%exwz{f��")A�C�R<��Ð�{S�"ÿÅÄ �SÆ � ^ �1!����a{ø�f�T"Xe×#*·wU{SÉY��Í�!×a�G}ÎÍÁ�·�T"|y3!�}çwz�B��{S�U�·�G}@É{4h*B�B�\wzy@���Y��4 ÉB{f~G�U{f{øLEÔbJ4D OQP }m�G~¤HpÕ ü¡�¢Q¡�¢�£ ü¦í3Î;)G<´t¹�µ�´ÿ¥§¦R¨ª©¬«­¦R®°¯ª±�¨=²5³­´tµ�¶B³­¦R¨ª¹ º�»�«­¼i½B¯ª¦¸«­¶³�\¦¤´ÿþ��ú

3¿¯Q½B³µ´¬®ÿ±�¨�¨E²Ïµä´ç® ¶W�{Þ��w'!���w���Í�×B�U��ÐÒy@�B~ q �B{f�G�U{�" ð·� Æ Â

ñkî

Page 37: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

KML4N)O�PQNSR T.U�VÜW É�º HGá_`_Z\[\Ú6ÐàáhY\[^] ¹ Ê ÉÕc Z­Zfef^a[zZv»v»�Z�ÈÜÊQ»^ÊM^tcÅÙ ºQÉ cÅÚ�Û�Ê�Ù·` ÜÒ[]Zv»]ÜtcÅÙ�È Ýàß1É~¹ `bÛxc Éc Ý ÈÒZhg�[zZ]Z�L�Ô � ÐÁ}@��~¤HÅÕ ü úq �Bym�øwz�B{f�G�U{�" ×·}U!ÇÍÒ�X!��5y3"Ï×a�G��wV!���wø���G}@{|y@�6wU�B{�×B���Á�>4M��4�wz�B{�"~!�ym�5�U{f��*B}ÎwÇ�|W�{�ÉBy@���%*B����U�>"|{���wU�B{S�M�f�G�B�U{fe�*B{S�B�f{S�­��4 q �B{f�G�U{�" ð·� Æ ym� ë {S�\wzy@���§ð·� îB�` PQO~Oba­OHa�KÜL­N)O�P�N1R T.U�V<c q �·{9",!ky@�Öwz�Ò�G}@��*B��{SÉ�y@�Þwz�B{R×B�U�Ò�>4H!k�U{�wz�B{èõ¸�B{S��ÍÒ�U�B{\Ð�×a�G}mÍÒ�B��e"ÏyU!�}m�æ�>4·wU�B{_�B�U��w>T�y@�BÉi� q �B{f�U{è×a�G}ÎÍÁ�·�T"|y3!�}@��×B}U!ÇÍ0!��Þy3"Ï×a�G��wV!���w��U�G}m{¤ym�Ö�.*&"|{f�Uy@�%!�}.!��<!�}ÎÍÒ�Uy@�!��BÉ�!�×B×·�U�ÇìÁy3"~!kwzym�G�Øwz�B{f�G��Íaïç�B{S��{0/R{P/Mym}@}æ�·{S{SÉ��G�B}ÎÍ{!X4N{n/Ã�Uy)"|×·}@{�×B���G×�{f��wUy@{S�f� q �·{�Ì>eQwU�õ¸�B{f��ÍÒ���B{fÐ=×a�G}ÎÍÁ�·�T"|y3!�}%�>4èwU�B{~�B�U��wøT�y@�BÉæF�� = Ôh��Õ � y@�7!�*B�·ymÐ�!��Uy3!kwz{ÏÉB{f~G�U{f{neªÌ�×a�G}mÍÒ�B�>"|y3!�}/M�Bym�]����!kwzym�d�t{f�MwU�B{Y4N�G}@}m�Z/My@�B~��f�G�BÉByÎwzym�G�B�ÞÔxõ¸�·{S�B{\ÍGF Æ VG�G�GÕ\Âð N1RñRñòóT�U�û � ÜtZè^tcÅÙ ºQÉ cÅÚ�Û�Ê�Ù�� = Ô]��Õ�»�Ê�`bÛ¡»h¶èZv»

å � � = Ôh��Õ�� ØöÆ Ý cÅ[ÿ� �Ã� ØöÆ ` Û ` Ü � = Ô Æ Õ i Æ ïå �Y·= Ô]��Õ� Ì ú Ý cÅ[S�(¸ Æ ` Û ` Ü �5·= Ô Æ Õ i Ì ú ü¾ {fw�� i Ì D · Ì ú ·dÍ0Í0Í"· Ì u ��{0!���H�eQwU{S�d" ÐFeªA�C�R¸� q �·{nÊ�[àÛ ` ÜÒÚEZ\`�Û�b0Ê�`bÛxc É �>4p!À¸�Á��}@{1!k�

}@yÎwz{f�V!�}^â�ym�g�S×ây�4Üâ i �S×¹!��BÉ ym� Æ ���S×�y)4|â i �S× ü ¾ {\wm� s ÉB{f�B��wz{âwz�·{��d*#" �>4�wz�·{!���ymwz�&"|{\wzy@?%!kwzym�G�B�>�>4çwz�B{M}@yÎwz{f�V!�}m�[!�×B×a{1!k�Uy@�·~Þym�AÌ s !��·ÉÏ}m{fwÃÐ s ÉB{f�B��wz{�wU�B{M�.*&"���{f�%�>4�}@ymwU{S��!�}@�y@�ÕÌ s ü W�{ÖÉB{��t�·{ÞwU�B{Ö×��G}ÎÍÒ�B�T"ÏyU!�}

º s Ô]��Õ i Ê�» � sÐ s�¼/M�B{f�U{ÊäÔ8½·Õ i � = »6½,» Æ ï Æ Ð ¼�¼ ü

�¬{f�U{m� = y@�%wz�·{åÌ>eQwU��õ¸�·{S��ÍÒ�U�B{\ÐØ×���}mÍÒ�B�T"ÏyU!�}Á�>4iwU�B{2�t�U��wèT�y@�BÉ~!��BÉEÌ i¿¾ � ÐxÀ ü ¾ {%"A",!Öð·� V�B�Z/ y)"|×·}@y@{f�RwU�<!kw�ÊäÔ Æ Õ¸ ñ��&*·wÿ� ÊäÔh½BÕ0� ØÃÆ 4N���S½1þÚ% Ä �ÇÆ � D Á ' üõ¸�G�B��y@ÉB{f�MwU�B{Ö×���}mÍÒ�B�T"ÏyU!�}çwz�·�U{S���B�G}mÉ�4h*B�·�fwzym�G� �Â|Ôh��Õ¸ H&ï Dú �X/M�·{S�U{

Â|Ôh��Õ i u� sut D º s Ô]��Õ orq_s ú u üë ym�B�S{Ë� = y@�5!�×��G}ÎÍÒ�B�T"ÏyU!�}ä�>4RÉB{S~G��{S{ÏÌ i � ÐY!��BÉl� s y@�5!Ø×a�G}mÍÒ�B�>"|y3!�}>��4¸ÉB{f~G�U{f{ Æ Fæwz�Bym�×a�G}mÍÒ�B�T"ÏyU!k}�wz�B��{S���B�G}@É04h*B�B�\wzy@���ø�<!��>ÉB{S~��U{S{ � ÐÁ}@�G~¤ñ�H ü W�{_/Mym}@}��U�·�6/=wU�<!kw>wz�Bym�>×a�G}mÍÒ�B�>"|y3!�}wz�·�U{S���B�G}mÉ�4h*B�·�fwzym�G���S�T"Ï×#*·wU{S��wz�·{KA�C�Rz��{àì&!��fwU}mÍG�Ã�ÄÜw u�zQu��>�1Å�w«r�Qw t�sÆÅ�w�¤�¨�r�QwÈÇ ¤"w�É2Ç s>w�z�r t.¤�¨Srh��wÃ�-¤��>��zQ¤nÊ&s>u��Ïr���hwft��¤$�>x<¨E§�z�Ç rs>¤�z�s>z-Ë"w�Çqrhs�¤$z-ÌQ° Í�°

ñGú

Page 38: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

R yÎìg!���ÍØ{S}m{%"Ï{S��w2�IþlÿÅÄ �ÇÆ �Z^Ò�å ý 4�� Ôh��Õ i Äwz�B{f�§ym��{%!��]�§wz{f��":Ì s !Åwå}m{1!���w¬���B{7!k�UymwU�#"Ï{fwzym?S{fÉ�}mymwz{f�V!�}�w'!�Tk{S�¬Ð�!�})*B{ÞÄ�G��� üÎq ��*B��4N�G��{1!��]��Û i Æ�� üSüSü � H0/R{��<!ÇÐG{k� shÏ Ð s Ø Ô�Ð s � Æ Õ Ï Ð s ØóÆ � D Á !��Bɧ�B{S�B�f{{%!��]��� º s Ôh��Õ0� ØÃÆ ü õ¸�G�B�U{fe�*B{S��wz}ÎÍ�ÂEÔ]��Õ Ø H ü

å ý 4Y�9Ôh��Õ i Æ wU�B{S�<�U�T"Ï{Øwz{S�d" Ì s "k*B��wE�a{Y�V!kwUy@�d�B{SÉ<��Í�� ����� s]Ï Ð s i Æ ü õ¸�G�B�U{Ðee�*B{f��wU}mÍ º s Ô]��ÕÞóñg!��BÉâ�B{S�·�S{ º s Ôh��Õ orq_s ú u �f�G��wz�Uym�#*·wU{S�0!kw�}m{1!���wÞñGHwU�{ÂEÔ]��Õ ü ë y@�B�f{º s Ô]��Õ orq_s ú u Ã� Æ 4N�G�2!�}@}~Û � /R{Þ�#!�Ð�{7Â|Ôh��Õè H�ï Æ ü¡�¢Q¡�¢_û º­´t¦R® þ6�ú ½B¦ÿþ µ�¼�¯ª¶B¯ª¦¸«I²Ï´tµäµ�¶¾ {fwS�Y�a{-!��g!k�U�ByÎwz�V!k��Í|H�eQwU{S��" A�C�R���ÐG{f�èJ�Ð�!���yU!��·}@{S�f� d¬�¸wz�B{��t�U��wè��wz{S×Øym�E�T*·�R�f�G�B��wU��*B�Ðewzym�G�Ï�>4Ñ!Ö×���}mÍÒ�B�T"ÏyU!�}Òwz�B��{S���B�G}@ÉA4h*B�B�\wzy@���A4N�G�y� � /R{�wU�V!��·�d4N�G�d"$�Yy@��wU�7! Æ e_É·{S�Sym�Uym�G��wz�U{f{åym�/M�Bym�]��{1!��]�â}@{%!�4%ym�-!ØA�C�Rz/Mymwz�§��"~!�}@}1wz{f��"Ï�SïiwU�By@�åy@�-!|�U{��B�B{%"Ï{S��wÖ�>4�!|wU�V!��·�d4N�G�d",!kwUy@�G�~GyÎÐG{S�|y@��Ô�ÀR�U�B�T*Áw_ÍGF Æ VGVG��Õà� J *B���G�Uym~Gy@�#!�}·×B���Á��4æ~!ÇÐG{�!K�a�T*B�·É|�G�Ïwz�B{M��y@?S{��>4�wz�B{��U{f��*B}Îwzy@�·~ÉB{f�Sy@��y@�G�ËwU�U{f{�� ë � ¾ �GT�!>"�Ô ¾ ��T�!�"�Fæñ�ÄGÄ Æ Õ��<!����G�B��{S��ÐG{SÉ�wz�<!Åw5!Ø�U}@ym~G��wz}ÎÍ���wz���G�B~G{f�����T*·�BÉ�1!k�E�a{����·w'!�ym�B{SÉÏ��Í�S�G�·�Uy@É·{S�Uym�B~Öwz�B{��V!��·T��>4iwU�B{�ÉB{S�fy@�Uym�G�wz�U{f{My@�B��wz{1!kÉæ�[W6{2*B��{ ¾ �GT�!>"±� �!�×B×·�U�!��]�Yy@�ØwU�B{Y4N�G}@}m�6/Mym�B~�}m{%"A",!·Âð N1RñRñòóT�U�V�Ò ý Z\`Î�âÂ�ÿÅÄ �SÆ � ^ È ÿ�� Æ��ÇÆ � c Z�Ê É HGá_`ªZ\[àÚ Y\[<]Rí£]>cÅ[�Ê É�º _QÊ�Ù ß Z Æ|Ø Ð Ø J �� ¹ Ê Édc ZøZfef^a[]Zv»v»�Z�ÈnÊQ»MÊ Æ á�ÈÒZ ¹ Û¡»\Ûxc É `b[zZ]Z�Ì ` ÜtZ\[]Z

å Z�Ê ¹ Ü�Ù@Z�Ê Ý c Ý Ì ¹ c É `hÊ�Û É »ÜÊ É HGá_`_Z\[àÚ ÐàáhY\[^]�Qå Ì Ü�ÊQ»Þ[fÊ É�ä Ê�`¤ÚEc0»\`èÔ�ñkJ Ï Ð�Õ·}@�MH�ï Æ í

` PQO~Oba|OHa ð NSR Róò�T�U�V�ÒÑc ¾ {fwmÌ D � ügüSü � Ì�Ó��{Kwz�B{�wU{S��"Ï���>4y��wz�#!kw��<!ÇÐG{��y@?S{X!Åw�}@{%!���w\Ð üë ym�B�S{Ï{1!��]��wz{S�d" Ì s �S�G��w'!ky@�B�P!kwK}@{1!k��w|ÐÖ}@yÎwz{S��!�}@�fFæwz�B{f�U{~"k*B��wø��{Ï�U�>"|{Ð�!��Uy3!��B}@{k� s wz�<!Åw�Ò�S�%*·�U�ÖÔN{SymwU�B{S�¤�B{f~!kwz{fÉØ�G�y*·�B�B{S~T!kwz{fÉçÕ¤ym�g!kwR}@{1!k��w Ê�Ð Ï J§�>4iwz�B{f�U{åwU{S��"Ï�S� q �·y@�RÐ�!���yU!��·}@{S� sy@��×·}U!��f{SÉYy@�Øwz�B{Ö���Á��wè�>4>wU�B{ÖÉB{S�fy@��y@�G��wz��{S{�FH!k�BÉYwU�B{Ö}@{n4 w2!��BÉY�Uym~G��wM�]�By@}mÉB�U{f�Y��4p� s /My@}m}æ�a{ÉB{f�Sy@��y@�G��wU�U{f{S�94N�G�¸wU�B{��U{f��wU�Uy@�\wzym�G�B�2� � �ÐÔuÕ�è2!��BÉ�� � �ÐÔuÕ D ��{S�U×a{S�\wzyÎÐG{S}ÎÍG� q �Bym���S�G�B��wz�d*B�fwUy@�G�Øym��U{f�%*B���UyÎÐG{S}ÎÍË�%!��U��y@{Sɧ�T*·w-4N�G�¬{1!��]�â�>4%wU�B{74h*B�·�fwzym�G�B�5� � �ÐÔuÕ�è5!��BÉ�� � �ÐÔuÕ D � ��wz�G×B×·y@�B~,/M�·{S��!A�C�R�/MyÎwz���B��wz{f��"Ï��}3!��U~�{S�¸wU�<!��/Ðèy@�����·w'!�ym�B{SÉi�ý w�y@�9�f}@{%!�� wz�#!kw9wz�·y@�%�U{f�%*B���UyÎÐG{M×B�U�Ò�S{fÉ#*B��{�~G{S�B{f�V!kwU{S���U�>"|{ Æ e_ÉB{f�Sy@��y@�G��wU�U{f{&Ì ü ë y@�·�S{èwz�·{4h*B�B�\wzym�G�Ë�G�·wV!�y@�·{SÉË��Í��·ìÁy@�B~����T"Ï{ø��*·�B�U{\wÖ�>4%Ð�!��Uy3!��B}@{f�å�>4¡!���H�eQwz{f��")A�C�RÑy@�-!k�âH�eQwz{f��"A�C�RRF&/¸{Þ�<!ÇÐG{�wU�<!kw�{1!��]�Y}@{%!�4>�>4íÌ �S�G��wV!�y@�B��!���H�eQwU{S�d" ÐFeªA�C�R¸�

ñ��

Page 39: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

¾ {fw8?·ÔbJ � ÊaÕÞ�a{EwU�B{�"~!ÅìÁy3"k*#" Ô w'!�Tk{S�=��ÐG{f�X!�}@}RA�C�R �K�G�=J<Ð�!���yU!��B}m{S�5/MymwU�§Ê=wz{f��"Ï��<!ÇÐÒy@�B~§�Uy@?f{g!kwø}@{1!k��wµÐ�ÕK�V!��·T���4�wz�B{�É·{S�Sym�Uym�G�6wz��{S{�~�{S�B{f�V!kwU{SÉ=��Í�wU�B{g!��a��ÐG{�×·�U�Ò�S{SÉ&*B�U{k�W�{����>*B�BÉ@?·ÔbJ � ÊaÕ�*B��y@�B~|wz�B{74N��}@}@�Z/Mym�B~|��y3"Ï×B}@{Þ�G�B��{S��Ð>!Åwzy@���æÂMy)4�ÌÎÖøym�-!|wU{S��")�>4_��/M�·y@�]��S����wV!�y@�·�S!���*B�·�B{S~!Åwz{SÉ�ÔN�B{S~!Åwz{SÉtÕ¤Ð�!��Uy3!��B}@{2� s Ô � s Õ\FÒwz�B{f��wU�B{­��{S��wU�Uym�fwzym�G�g� � �ÐÔ�Õ�èÞÔ�� � �ÐÔuÕ D Õ�1!�*B�U{f�ÏwU�B{YwU{S�d" Ì�Ö|wz�âÐ>!k�By@���æ� ë y@�·�S{Øwz�B{ØÐ�!���yU!��B}m{,� s !kw�wz�B{����Á��w��4^Ì3�Ò�S��*B�U�ym�×!Åw}@{%!���w¬Ê£Ð Ï J¿wU{S��"Ï�ø��4¬��y@?f{g!kw�}m{1!���wAÐ � 4N���k!kw�}m{1!���w��G�·{Ø�>4�wz�B{E�BymwKÐ>!k}3*B{f�jØ5þ�ÿÅÄ �ÇÆ �Øwz�·{�U{f��wU�Uy@�\wzym�G�l� � �ÐÔuÕSÙ2/My@}@}>��{A!�A�C�RÚ/M�Bym�]���#!��0!kw5"|����w�ÊäÔ Æ � Áú ^ Õ­wU{S�d"|���>4R�Uym?S{A!kwÖ}m{1!���wÐ ü ¾ {fw8Ì�è�Ô�Ì D ÕKÉB{f�B��wz{ØwU�B{Ø��*B�Áwz�U{f{��>4�Ì /M�Bym�]� �S�G���U{S��×����BÉB��wz��wz�·{Ø�U{S��wz��y@�fwUy@�G�z� � �ÐÔuÕ�èÔ�� � �ÐÔuÕ D Õ\F�!k�BÉ���*B×·×��G��{k/MymwU�B�T*·w�}m�G�U�å�>4¤~G{S�B{f�V!�}mymw_Í�wz�#!kw0� � �ÐÔ)Õ�è�ym�Y!�H�eQwz{f��" A�C�R×/M�·y@�]��<!���!kwÈ"Ï�G��wäÊ�Ô Æ � Áú ^ Õ wU{S��"Ï�%�>4��Uym?S{�!kw¤}@{%!���wÃÐ ü C���wz{�wz�#!kw ?-ç�J.ÒæÔ�Ì�è\Õ Ø ?BÔNJ�� ÆG� ÊäÔ Æ �

Áú ^ Õ�Õ!��Bɱ?�ç�J.ÒæÔ�Ì D Õ Ø ?·ÔbJØ� ÆG� ÊçÕ ü W�{Ö�S�G�B��y@ÉB{f�M�U{fÐ�{S��!�}��1!���{S�SÂå ý 4 ?�ç�J.ÒæÔ�Ì�è\Õ�� ?�ç�J�ÒæÔiÌ D Õ\Fæwz�·{S�d?-ç�J.ÒæÔ�ÌåÕ i ?-ç�J�ÒiÔ�Ì D Õ-!��BÉ��B{f�B�S{ÿ?·ÔbJ � ÊaÕ Ø ?·ÔbJ§�ÆG� ÊaÕ ü

å ý 4 ?�ç�J.ÒæÔ�Ì�è\ÕY¸ ?�ç�J�ÒæÔiÌ D Õ\Fæwz�·{S�d?-ç�J.ÒæÔ�ÌåÕ i ?-ç�J�ÒiÔ�Ì�è\Õ-!��BÉ��B{f�B�S{ÿ?·ÔbJ � ÊaÕ Ø ?·ÔbJ§�ÆG� ÊäÔ Æ �

Áú ^ ÕUÕ üå ý 4m?-ç�J�ÒiÔ�Ì�è\Õ i ?-ç�J.ÒæÔ�Ì D Õ\F¤wz�B{f� ?�ç�J�ÒæÔiÌ­Õ i ?�ç�J�ÒæÔiÌ~è\Õ ï Æ !��BÉ#�B{S�B�f{ß?·ÔbJ � ÊaÕ Ø?·ÔbJØ� Æ�� ÊäÔ Æ �

Áú ^ ÕUÕ�ï Æ üq �|{f��w'!k�B}@ym�U��y@�ByÎwzy3!�}i�f�G�BÉByÎwzy@���B�94N�G�èwz�·{Þ��{S��*B�U��{S�B�f{ø�U{f}U!kwUy@�G��/R{Þ�S�G�B��y@ÉB{f��wU�B{Ö�1!���{�Ê i Æ üý ��wz�·y@�¸�%!��U{­wU�B{S��{­y@�R�G�B{åwU{S�d"°ym�{��/M�Bym�]�Ø�S�G��w'!ky@�B�9"Ï�G��{¬wz�#!��nÐ�Ð�!��Uy3!��B}@{f�Sï�/MymwU�B�T*·w¸}@�G����>4>~G{S�·{S�V!k}@ymw_Í~/¸{Þ��*·×B×����U{�wz�<!Åw�wU�By@�èwU{S�d"�y@�SÛ D Û ú üSügü Û"ã ü<q �B{f�YwU�B{ Æ e_ÉB{f�Sym�Uy@���Y}my@��w

Ô Û D � � � Ü j Õ�èàÕ � üSüSü � Ô Û�ã � � � Ü�Ý�Õ�èàÕy@�¬{Se�*ByÎÐ>!k}@{S��w¬wz�g!E��!��BTTe Æ ÉB{S�fy@�Uym�G�§wU�U{S{Ky@�(/M�·y@�]��{1!k�]��}@{1!�4%�S�G��wV!�y@�B��!���H�eQwU{S��" ÐFe_A�C�R¸��¬{f�B�S{54N�G�2!���ÍØJm/R{K�#!�Ð�{|?BÔNJ �ÇÆ Õ i Æ üë �G}ÎÐÁym�B~¬wU�By@�ä{1!k��ÍK�U{S��*B�U��{S�B�f{è�U{f}U!kwUy@�G�54N�G�­?BÔNJ � ÊçÕ����B�Z/M� wz�<!Åw�?·ÔbJ � ÊaÕ Ø ÔxñkJ Ï Ð�Õ·}@�>Êíï Æ üë ym�B�S{�Ê Ø HÖwz�·{ÖwU�B{S���U{%"�y@��×B���pÐ�{SÉæ�¡�¢Q¡�¢Q¡ 2 « Gú5¬½B¯�® ±�¨�� ¦¸»�«­¹Þ�à¦R´ H µ­5­´tµ�¶·µä«¸½t¯�«­ý3þ��ú ßè©/¥�¦¸¨_©¬«­¦Ã;

®°¯ª±�¨E²6³­´tµä¶·³­¦¸¨ª¹7º�»�«­¼i½B¯ª¦¸«­¶KML4N)O�PQNSR T�U�j±ý Z\`N� c Z�Ê É HGá_`_Z\[\ÚàY\[<] cQ_�Z\[RJÎ_QÊ�[àÛ�Ê c Ù@Zv»Sí � ÜtZ É � ¹ Ê Éßc ZÞZfef^a[zZv»v»�Z�ÈÿÊQ»ÊÞ^tcÅÙ ºQÉ cÅÚ�Û�Ê�Ùi` ÜÒ[]Zv»]ÜtcÅÙ�È ÝàßSÉ~¹ `bÛxc É c Ý ÈÒZhg�[zZ]Z�L�ÔNJ­D OQP }@�G~¤HpÕfí

ñ é

Page 40: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

` PQO~Oba�c Rç���T" ¾ {%"A",!�ð·� Æ Ä|!��BÉ q �B{f�G�U{�" ðÁ� Æ F[/R{�T��B�Z/ÿwz�<!kwj�Ñ�1!k�=��{|{\ìÁ×B�U{f�U��{SÉÚ!k�! Æ eQÉB{S�fy@�Uym�G�¿wz��{S{�Ì �>4Þ�V!��·TÃÔxñÅJ Ï Ð�Õ·}m�MHMï Æ /M�B{S��{�{%!��]� }@{%!�4Þ�S�G��w'!ky@�B�g!=×a�G}mÍÒ�B�>"|y3!�}wz�·�U{S���B�G}mÉz4h*B�·�fwzym�G� �>4¬ÉB{S~G��{S{YL�Ô � ÐÁ}@��~¸HpÕØÔ wz�B{ØÐ�!�}3*·{Ø�>4^ÐP/My@}m}¸�a{��·ìÁ{SÉ<}3!kwz{f�]Õà��Rt�U�T"¾ {%"A",!|ð·� úA/¸{�T��B�Z/öwz�#!kw­wz�·y@�åÉ·{S�Sym�Uym�G�§wz��{S{MÌ��%!�����{ø{\ìÁ×B�U{f�U��{SÉl!��Y!���?Ze_É·{S�Sym�Uym�G�§}my@��w/M�B{f�U{�? i Ô�ñkJ Ï Ð�Õ·}@�MH£ï Æ !k�BÉ�{1!k�]�Ï�>*·wz×#*Áw%�>4çwz�B{¸ÉB{S�fy@�Uym�G��}my@��w y@��!­×��G}ÎÍÒ�B�T"ÏyU!�}GwU�B�U{f�U�B�G}mÉ4h*B�B�\wzym�G���>4 É·{S~G��{S{KL�Ô � ÐÁ}@�G~¤HpÕ ü õ9!k}@}iwU�By@��ÉB{f�Sym�Uy@���Y}my@��w�Ó ü¾ {fwË� D � ügüSü � �S�Ñ��{EwU�B{Ø�S���Q�d*B�B�\wzy@���B���S�G��wV!�y@�B{fÉ y@�5wz�·{���*B�f�S{f�U�UyÎÐG{��B�ÁÉ·{S���>4^Óà!k�BÉ

}@{\w| D Ôh��Õ � üSüSü � Â��>Ôh��Õ���{�wz�B{��S�G���U{f�U×a�G�BÉBym�B~<×a�G}mÍÒ�B�>"|y3!�}@�,4N���YwU�B{�!����U�Ò�Sy3!kwz{fÉ ×a�G}mÍÒ�B��e"ÏyU!�}Þwz�B��{S�U�·�G}@É�4h*B�B�\wzym�G�B��!kw�wz�·{=�T*·wz×&*·wz�fFÏy�� {k� wz�·{#×a�G}ÎÍÁ�·�T"|y3!�}Öwz�B��{S�U�·�G}@É�4h*B�B�\wzym�G��S���U�U{f�U×a�G�BÉBym�B~=wz�=wz�·{ Ù exwz� �S�G���d*B�B�fwUy@�G� �Ã×Y�S�T"Ï×#*·wU{S�Øwz�B{m4h*B�·�fwzym�G� �Â~×ÅÔ]��Õ= Ä ü � ý 4Â~×ÅÔ]��Õ i Ä|4N�G���U�>"|{��Qþ�ÿÅÄ �ÇÆ � ^ wz�·{S��/¸{��1!k�Ñ��{S×B}3!��S{{Â�×�Ô]��Õø��ÍzÂ�×ÅÔh��Õ ïâá Ï ñ � /M�B{S��{á i "Ïy@�ûÿ���Â~×kÔh��Õ Â¿�°þ ÿÅÄ �ÇÆ � ^ !��·É�Â�×�Ô]��Õ�� ÄÁ� � /MymwU�B�T*·wE�]�<!��B~Gym�B~6wU�B{(4h*B�B�\wzym�G��S�>"|×#*Áwz{SÉâ��ͧwU�B{�×a�G}mÍÒ�B�T"ÏyU!k}�wU�B�U{f�U�B��}@É|4h*B�B�\wzym�G�æ�KC¬�Z/ó��ͧ�U�1!k}@y@�·~Ø{1!��]��Â~×Þ��Ím!���!�×�e×B���G×B�Uy3!kwz{y"k*B}mwUy@×B}my@�%!kwzyÎÐG{È48!��fwU�G�[/¸{��1!k�Ï�d*B×B×a�G�U{2/MymwU�B�T*·w�}m�G�U�%��4i~G{f�B{S��!�}@yÎw_ÍKwU�<!kw¡4N�G�%{%!��]�Ù i ÆG� üSüSü �'ã /¸{Þ�<!ÇÐG{0"|ym� �"!Zähè � D�å�æ � Â~×ÅÔh��Õ0�Á Æ üõ¸�G�B��y@ÉB{f�MwU�B{Ö×���}mÍÒ�B�T"ÏyU!�}

º Ôh��Õ i � D ¼� D Ô]��Õd D Ô]��Õ�ïz� ú ¼� ú Ôh��Õ� ú Ôh��Õ�ï Í0Í0Í ïz��� ¼�S� Ôh��Õ�Â��9Ô]��Õ ü Ôbð·� Æ Õ

�¬{f�U{Z¼�Ã×y@�Þwz�B{E?S{S���)�k�G�·{EÐ�!�}3*B{fÉ=×���}mÍÒ�B�T"ÏyU!�}[/M�·y@�]�=�S���U�U{f�U×a�G�BÉB��wU�§wz�·{�"|���B�T"ÏyU!�} �Ã×Ôb{k� ~·�%y)44�Ã×My@���çP �<çV�Hè�wz�B{f� ¼�Ã×kÔ]��ÕRy@�È�çPÅÔ Æ �|�<çfÕ��Hè\Õà�¡éÈ!��]�ØÐ�!�}3*B{��&×My@�y!ø×a�G�UyÎwzymÐ�{å�f�G�B��wV!���w�]�B�G��{S���U�~!���wz��V!kwUy@��4 ÍEwz�·{04N�G}m}@�Z/My@�·~ø�S���BÉBymwUy@�G�·�SÂ

�-� i ÆG��-� (�D ¸ "~!Åì�"!Zähè � D�å æ � ��� ¼�S�>Ôh��Õ�Â��9Ô]��Õ�� �����&× ¸ "~!Åì�"!Zähè � D�å�æ � �y×�ê D ¼�Ã×dê D Ô]��ÕdÂ~×dê D Ôh��Õ4ï Í0Í0Í ïz��� ¼�S�>Ôh��Õ�Â��9Ô]��Õ�� ü���� D ¸ "~!Åì�"!Zähè � D�å æ � � ú ¼� ú Ôh��Õ� ú Ô]��Õ4ï Í0Í0Í ï��-� ¼�S�9Ôh��Õ�Â��>Ôh��Õ0� ü

q �B{f� wz�·{�×a�G}mÍÒ�B�>"|y3!�}�wU�B�U{f�U�B��}@É�4h*B�B�fwUy@�G� º Ôh��Õ= Ä��#�f�T"Ï×#*·wz{f��{àì&!��fwU}mÍ wz�·{���!>"Ï{4h*B�B�\wzym�G�A!��%wz�B{MÉB{f�Sy@��y@�G�}my@��wÃÓ üq �Þ�U{S{�wU�By@�fF��ÁìA!��Ïy@�B×#*Áw¡�IþlÿÅÄ �SÆ � ^ ü ý 4 Ù y@�%wz�B{Mym�BÉB{\ì��>4ñ�W

Page 41: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

wz�·{_�t����w>�S�G�Q�d*·�B�fwUy@�G�M�Ãס/M�·y@�]��ym�ä��!kwzym�d�t{fÉ���Í5� � wU�B{S� ¼� D Ôh��Õ i ¼� ú Ô]��Õ i Í0Í0Í i ¼�Ã× (�D Ôh��Õ iÄ � �U�­wU�B{è�G�B}ÎÍÖwz{S�d"|� �>4>Ô�ð·� Æ ÕÑ/M�By@�]�j"~!�T�{9!­�B�G�B?f{S�U���f�G��wz��y@�#*·wUy@�G�Þwz� º !��U{y� s ¼� s Ôh��Õ� s Ô]��Õ4N�G�<Û� Ù ü ë y@�B�f{ ¼�Ã×kÔh��Õ i Æ !k�BÉ � Â~×ÅÔh��Õ0�ç ÆG� wz�B{K�]�B�Gym�S{ø��4¡�&×�{S�·��*B��{S�­wz�#!kw¬wU�B{ø��y@~G���>4º Ôh��Õy/My@}m}i�a{ÖwU�B{Ö�V!>"Ï{0!��èwz�·{Þ��y@~G���>4�Â~×kÔh��Õ üq �B{ÞÉB{S~G��{S{Þ�>4 wU�B{Þ×��G}ÎÍÒ�B�T"ÏyU!�} º Ô]��ÕMym��!kw2"|����wøÔxñkJ Ï Ð�Õ·}@��H&ï Æ ïÑLEÔ � з}m�G~RHÅÕ ü ý 4�/R{w'!kT�{�Ð i J ú OQP wU�B{S��wz�Bym�èÐ�!�}3*B{�ym�ML�ÔNJ­D OQP }@�G~RHpÕ üdå×B×·}mÍÒy@�B~ARH!��\w�ð·� é ~GyÎÐG{f���>*B�2",!�ym��A�C�R5}m{1!����By@�B~���{S�d*B}mwSÂë O�PQO�÷�÷�ò�P6� T.U�V�V � ÜtZ ¹ Ù�ÊQ»v»|c Ý HGá_`ªZ\[àÚ Y\[<] Ý cÅ[\Ú ß Ù�ÊÒZEcQ_�Z\[�ÿÅÄ �ÇÆ � ^ ¹ Ê ÉÎc Z�Ù@Z�Ê�[ É Z�ÈlìQÛ É

c cÅ` ÜÏ`NÜtZ�X W � ÚEc�ÈÒZ\Ù�Ê É ÈK`NÜtZ�ÚEc�ÈÒZ\Ùic Ý Zfe)Ê ¹ `>Ù@Z�Ê�[ É Û É g Ý [zcÅÚ�Z�� ß Ûi_QÊ�Ù@Z É~¹ Z�� ß Z\[àÛxZv»xíÖÛ É `bÛ ÚEZñ gih ^ jmlNnporq_s ^ orq_s�u�v üî NSRñò£P6�[c ë {fÐ�{S�V!k}y!�}m~G�G��ymwz�&"|��!��U{T��B�Z/M� ÔbÀ¸}3*#" Z\`\Ê�ÙÎí@F Æ V�V é ï�õ¸�G�·{S�æF Æ VGV é Õ�4N�G�Öã�d�õ}@{%!��U�·y@�B~}@ym�B{1!k�èwz�B��{S�U�·�G}@É�4h*B�B�fwUy@�G�·����Ð�{S�­ÿÅÄ �ÇÆ � ^ ym�YwU�B{K×·�U{S��{S�B�f{��>4 �S}U!k�U�Uy��t�1!Åwzy@���Y�·�Gy@��{y@�ËwUy3"Ï{ø×a�G}mÍiÔNJ1Õ ü ý wY4N�G}m}@�Z/M�¬wU�<!kw��T*·�åwzy)"|{K���T*·�BÉB�-4N�G�­}m{1!����By@�·~ØA�C�R¿�f�G��wzy@��*B{�wU�Ø�B�G}mÉy@�ØwU�B{Ö×B�U{f�U{S�·�S{ø��4 �f}U!����Uy)�B�1!kwUy@�G�Ø�B��y@�U{k�ë O�PQO�÷�÷�ò�P6� T.U�VSj � ÜtZ¸êÖÔbJ­D OQP ÕåÙ@c ` Z\[ c c ß1É Èµg�Ûi_�Z Éæc�ºPï Û É » äQº Ê É ÈKX�Ê\^tZ\[à` Ý cÅ[¬`NÜtZ�ÈÒZhg�[zZ]Zc Ý Ê^tcÅÙ ºQÉ cÅÚ�Û�Ê�Ù9` ÜÒ[]Zv»]ÜtcÅÙ�È Ýàß1É~¹ `bÛxc É [zZ�� ß Û [zZ�ȧ`_c ¹ cÅÚå^ ß `_Z ÊÏ^tcÅÙ ºQÉ cÅÚ�Û�Ê�Ùí»\Û�bÅZAY\[<]ócQ_�Z\[ÿÅÄ �ÇÆ �Z^�Û¡»�`�Û�gÅÜÒ` ß ^�`_c Ê|Ù@c�gÑÊ�[àÛ `NÜÒÚ�Û ¹9Ý Ê ¹ `ªcÅ[fí

����ð «  "Ç9���@Ç�Ç9 S�Ï�ë ym�B�S{�Ð Ø J�y@� q �B{f�G�U{�"Ãð·� Æ F6RH!��fwäð·� é !�}m�U{1!kÉ·Íåy)"|×·}@y@{f��wU�<!kwæwU�B{S��{¤ym�Ñ!è}@ym�B{1!k��e_×·�U�G~G��!>"A"|ym�B~�<!���{SÉm!k}@~G�G��ymwU�#" 4N�G��ãad�õ }@{1!k�U�Bym�B~EA�C�R�/M�·y@�]�§wV!�T�{f��ñ�gih * ^ orq_s ^ orq_sGuxv wUy3"Ï{���wz{f×B�S� q !���*By!��BÉ q ��*BT�yC��yÁ~!ÇÐ�{S!��øymÉB{S��wUy@�1!k}�wUy3"Ï{����T*·�BÉP4N�G�p!¬É·y���{S�U{f��w¡!�}m~G�G�UyÎwz�#" �<!���{SÉ��G�K��ÍÒ×a��wz�·{S�Uym��a�Á�G��wzym�B~�*B��y@�B~��S���Q�d*B�B�\wzy@���B�S� ý ��wU�By@�­��{S�\wzy@���¹/R{�B��wU{�wz�<!kw�wU�B{�×B���Á�>4¤�>4 q �B{f�G�U{�" ð·� Æ~GyÎÐG{S�5!k�m*B×B×a{S�­�a�T*B�BÉâ�G�ËwU�B{7/R{Sym~G��wÖ�>4�wz�B{���{S�d*B}mwUy@�B~�×a�G}ÎÍÁ�·�T"|y3!�}1wz�B��{S���B�G}@É|4h*·�B�fwUy@�G�æ�q �Bym� �G�B��{S��Ð>!Åwzy@�����1!��ø��{9*·�U{SÉ�wU�­×B���pÐ�{è�S���U�U{f�fwU�B{S���¤��4Bwz�B{ q !��d*Byi� q ��*BT�y���yÒ�a�Á�G��wzym�B~�e_�#!��U{fÉ!�}m~G�G�UyÎwz�#"ñ!��BÉ6wU�§���B�Z/ wz�<!kwK�Uy)"|×·}@{S�0!�}m~G�G��ymwz�&"|���d*B�]��!��5W y@�B�B�Z/ �G�Kã>{f�U�f{S×·wU�U�G�=�%!���a{P*B��{SɧwU�|}m{1!���� ¼L|Ô � J1Õ�ÉB{S~��U{S{Þ×a�G}mÍÒ�B�T"ÏyU!k}�wz�·�U{S���B�G}mÉ�4h*B�B�\wzym�G�B�2/M�By@�]���S�T"Ï×#*·wU{øA�C�RÔbym�B��wU{1!�É��>4 ���Ò�G��wzy@�·~X!�}@~G���UymwU�#"Ï�¸�G�2!�}m~G�G��ymwz�&"|��4N�G���U�G}ÎÐÒy@�B~�}my@�B{%!��è×B���G~G�V!�"|�zÕ\�q �B{�Ì>eQwz��õ¸�B{S��ÍÒ�U�B{\ÐÏ×��G}ÎÍÒ�B�T"ÏyU!�}S� = Ôh��Õ i =sut è ç s � s �<!��[!�}@}Áy@��wz{f~G{S�9�f�Á{\ÓÏ�Sy@{f��wz�[/MymwU�{1!k�]��� ç s � Ø ñ�=�Ô�õ¸�B{S�B{\ÍGF Æ VG����Õ\��ÀRÍ�y@�B��×�{f�fwzym�G���>4äwz�B{Ö×B���Á�>4>�>4 q �B{S���U{%" ðÁ� Æ /R{Þ���·w'!�ym�

ñ�V

Page 42: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

ë O�PQO�÷�÷�ò�P6� T.U�VSTkW É�º HGá_`_Z\[\ÚlÐàáhY\[<] ¹ Ê Éæc ZÞZfef^a[zZv»v»�Z�ÈÿÊQ»\ʬ^tcÅÙ º�É cÅÚ�Û�Ê�Ùt` ÜÒ[zZv»]ÜtcÅÙ�È ÝàßSÉ~¹ á`bÛxc É c Ý ÈÒZhg�[zZ]Z�L�Ô � ÐÁ}@�G~¤HpÕMÊ É È ` Z\Û�gÅÜÒ`­Ð�gih * Á orq_sGuxv ü

, ��y@�B~�wz�Bym���S�G���G}@}3!���Í{/R{|���·w'!�ym��!���y3"Ï×B}@{ø×B�U�Ò�>4R�>4¸���B{��>4¸wU�B{X",!ky@�§wU�B{S�G��{%"Ï�Y4N�U�T"Ô q !��d*By�!k�BÉ q �d*BT�y���yxF Æ VGV�V�Õ\F�ÉB{f�U�S��y@�a{SÉ�ym� ë {f�fwUy@�G�Eð·� Æ � ñÁF�/M�Bym�]�g!��U��{S��wz�¸wU�<!kw�4N�G�y!k��Í|A�C�R�Ú!��BÉz!���Í6×B�U�G�#!��By@}mymw_ÍâÉBym��wU�Uy@�&*·wzym�G�ñ�Iwz�B{f�U{Ø{àì·ym��wU���U�T"Ï{��U�·�G��w�f�G�Q�d*B�B�\wzym�G��/M�By@�]� ym��B��wUy@�f{1!��B}ÎÍ5�f�G�U��{S}U!Åwz{SÉ�/MymwU�×��*B�·ÉB{S�ÿ� ü W�{�*B��{(!���y3"Ï×B}@{Ø}m{%"A",!ËÉ#*·{�wU� Ö �G}mÉ#",!k�B�æF��!k��w'!kÉ�!��BÉ ÷ !�?S�a�G���pÐ ÔUÔ Ö ��}@É#"~!��B�ÃZ\`ÿÊ�ÙÎí@F Æ VGV�ñ�Õ ¾ {%"A"~!�î�Õj/M�By@�]� ��w'!kwU{S��wz�#!kw�y�45!4h*B�B�\wzym�G�ò�Ã��Ð�{S�%ÿÅÄ �ÇÆ � ^ �1!�� ��{â{àì·×·�U{S���U{fÉ�!���!�",!������Uymw_ÍÑ�>47!kw{"Ï�G��w�ó ô Æ exÐ>!k}3*B{fÉ4h*B�B�\wzym�G�B�åÔb×a�G�U��y@�B}ÎÍk/MymwU�E�U{f×�{\wzyÎwzy@���B�]Õ ÉB��!1/M�,4N�U�T"²!Ö�U{\wSõ � wU�B{S�~4N�G��!���ÍÉBy@��wz�Uym�#*·wUy@�G�8���ÐG{f�kÿÅÄ �ÇÆ � ^ wU�B{S��{My@�%�U�>"|{94h*B�·�fwzym�G�~��þmõ3��*·�]�ÏwU�<!kw���ã%�f�"!$#�%ö��Ô]��Õ i �9Ôh��Õq'Ç� Dú �� D÷ ü ý ��T*B����{fwUwUy@�B~0/¸{Mw'!kT�{-õ3wz�Þ�a{�wU�B{¬��{fwR�>4Ñ!k}@}t�S���Q�d*B�B�\wzy@���B�R��4æ}@{f�B~�wU�ELEÔ � з}m�G~¤HpÕ�!��·ÉÏwU�B{Sym��B{f~!kwzym�G�B�f� q �B{S��{|ym�5!Y�f}@{1!k���S�G���U{S��×����BÉB{S�·�S{Ï��{\wx/¸{f{S�5×a�G}mÍÒ�B�>"|y3!�}äwU�B�U{f�U�B��}@ɹ4h*B�B�fwUy@�G�·�/MymwU�y@��wz{f~G{S�%�f�Á{\Ó|�fy@{f��wU�y!k�BÉÉB{S×·wU��e�ñ��Sym�U��*BymwU�[/Mymwz�A!ÞD�d-ø J ÷ký q � ~!kwU{2!kw�wz�B{����Á��w�!k�BÉÔb×a�G���Uy@�·}mÍË�·{S~!kwU{SÉçÕ�d­C¬Aÿ~T!kwz{f�P!ÅwKÉ·{S×·wU� Æ �|õ¸�G���G}@}3!���ÍËð·� Æ ðØ~GymÐ�{S��wz�B{��{Se�*By@��{SÉ����T*·�BÉ�G��ó � !��·É�/R{K���·w'!�ym�ë O�PQO�÷�÷�ò�P6� T.U�Vѳ¿ù Ûi_�Z É Ê É�º HGá_`ªZ\[àÚ ÐàáhY\[<]f�àÊ É ÈÚÊ É�º È�Û¡»\`b[àÛ cSß `bÛxc É � cQ_�Z\[�ÿÅÄ �ÇÆ � ^ �` ÜtZ\[]ZåÛ¡»�Ê ¹ c É"aàßSÉ~¹ `bÛxc É �3c Ý »\Û�bÅZ�Ê�`9ÚEc0»\`æLEÔ � ÐÁ}m�G~RHpÕ<» ß�¹ ÜE`NÜ�Ê�`���ã����"!�#�%��|Ôh��Õ i �9Ô]��Õ 'G�Dú � i ñ (Ágih * Á orq_s Á orq_s�uxv üq !�T�y@�B~ËÐ i J6~GyÎÐG{f� q !���*Byp!��BÉ q ��*BT�y���y� � q �B{S���U{%" Æ � Æ FH/M�By@�]��y)"~"Ï{SÉBy3!kwz{f}mÍ|y3"Ï×B}@ym{S��wz�·{{\ìÁym��wz{f�B�S{Þ�>4�!Ï�a�Ò�G��wUy@�B~�eQ�<!���{SÉ{!�}m~G�G�UyÎwz�#"²4N�G��}m{1!����By@�B~A�C�R5y@�Øwzy)"|{�ñ�úgih ^ jml_v üR y@�#!�}@}ÎÍGFT/R{­�G�B��{S��Ð�{åwz�#!kw¸wU�B{�/R{Sym~G��wè�a�T*B�BÉ�~�ymÐG{f��y@��õ¸�G�U��}@}U!k��Íð·� Æ ðÞy3"Ï×B}@ym{S�9wz�#!kwy/R{ÉB�M�B��w��·{S{SÉÖwz�M���G}mÐ�{%}@y@�·{1!���×B�U�G~��V!>"Ï��Ôb���1{fÐG{f�7*B��{����Ò�G��wUy@�B~9!�}@~G���UymwU�#"Ï�]Õaym���G�UÉ·{S�æwz�M}m{1!����×a�G}mÍÒ�B�T"ÏyU!k}�e_��y@?f{SÉ�A�C�R�y@�Öwzy)"|{¤ñ úgæh+* ^ v ü ý 4b�y@�N!¬×a�G}ÎÍÁ�·�T"|y3!�}kwU�B�U{f�U�B��}@ÉP4h*·�B�fwUy@�G�K�>4BÉB{f~G�U{f{Æ !��BÉX/¸{fy@~G��w_ó �pÐ�{S��wz�B{MÉB�>",!�ym�8ÿÅÄ �ÇÆ ��� � wU�B{S�Ï{SyÎwz�B{f�9wz�B{�ãä{S�U�f{S×·wU�U�G�,!�}@~��G�UyÎwz�#"ö�G�%wz�·{W y@�·�B�Z/�!k}@~G�G��ymwU�#"Ã�1!����a{9*B��{SÉ�wz�å}@{1!k�U�X�Eym��×���}mÍiÔ r � óöÕ1wUy3"Ï{¸��wU{S×B��ÔN�U{f{ q �B{S�G��{%"Ï��úÁ� ñ!��BÉYúÒ� ðøym��õ¸�<!�×Áwz{S��úGÕ\�%d¬�èy@�gRH!��\w�ð·� é F./¸{��%!��EÐÒy@{n/�!k�ØÉB{S~G��{S{ÐeªÌ×��G}ÎÍÒ�B�T"ÏyU!�}BwU�B�U{f�U�B�G}mÉ4h*B�B�\wzym�G�Ø��ÐG{S��ÿÅÄ �ÇÆ �Z^7!��S!��ØÉB{f~G�U{f{ne Æ ×a�G}ÎÍÁ�·�T"|y3!�}Bwz�·�U{S���B�G}mÉ{4h*·�B�fwUy@�G����Ð�{S��ÿÅÄ �ÇÆ �Z^ ¾ � !k�BÉwz��*B�2/R{K�%!���*·�U{Þ{SymwU�B{S��wU�B{Þã>{S���S{f×·wz���G�Ë�G�9W ym�B�B�Z/�!�}m~G�G��ymwz�&"ÿwU�|}m{1!����§H�exwz{S�d"3A�C�R=ym�wzy)"|{�ñ gih * ^ orq_s ^ orq_s�u�v ü

ðGÄ

Page 43: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

���Ðû ü5�5ý¥þ�« Â-ªÏ� Âm Á �ËÄ�¯Y�Ë�ËÅ  �ÆAÄÿ°²±��ÃÂ�Ç[±Ë�ËÄÇ� ­������M�9 S�Ï�@ǧ��|� ¨ Â\Æ���þ�£)���� « ¬ ­

då����{S{S�Ky@� ë {f�fwzym�G�Kð·� ñÒ� ñ¸wz�·{¸DYy@�B��T�Íe_ã�!�×a{S��w9êÖÔNJ­D OQP Õæ}@�Z/R{S�����>*B�BÉÞ�G�Ö×��G}ÎÍÒ�B�T"ÏyU!�}pwz�B��{S����e�G}mÉ�4h*B�B�fwUy@�G�ØÉ·{S~G��{S{-4N�G�è×a�G}ÎÍÁ�·�T"|y3!�}ç�Uym?S{ÖA�C�R�y@��×B���pÐ�{SÉ�*B�Uym�B~X!��U{%!�É�eQ�G�B�S{ÖA�C�R¸� ë y@�B�f{!���Í���{1!�É�eQ�G�B�f{|A�C�R �1!��â�<!ÇÐG{~!kw5"Ï�G��wÖJ=wU{S��"Ï�SF q �·{S�G��{%"Ið·� ñ�y3"Ï×B}@ym{S�¬wU�<!kw0!���Í���{1!�É.e�G�B�f{¸A�C�RØ�1!��K��{¤{\ìÁ×B��{S�U��{SÉX!��N!M×���}mÍÒ�B�T"ÏyU!�}Åwz�B��{S���B�G}@É54h*B�B�\wzym�G�K��4BÉB{f~G�U{f{¸LEÔbJ D OQP }m�G~�J1Õ ü�¬{f�U{Y/R{K~GyÎÐG{5!�U}my@~G��wz}ÎÍE�a{fw�wz{f���a�T*B�·ÉæÂKML4N)O�PQNSR T.U�T W É�º [zZ�ÊÑÈ�áUc É~¹ ZËY\[<]ÿcQ_�Z\[ _QÊ�[àÛ�Ê c Ù@Zv»P� D � üSüSü � � ^ ¹ Ê ÉIc Z�Zfef^a[zZv»v»�Z�È�ÊQ»ÕÊ^tcÅÙ ºQÉ cÅÚ�Û�Ê�Ùæ`NÜÒ[zZv»]ÜtcÅÙ�È ÝàßSÉ~¹ `bÛxc É c Ý ÈÒZhg�[zZ]Z�L�ÔNJ­D OQP }@�G~ ú OQP JæÕ üq �|×B����ÐG{ q �B{S���U{%" ðÁ� ðk/R{P*B��{ÞwU�B{Y4N�G}@}m�Z/My@�B~����<!���×�{f�MÐ�{S���Uy@�����>4 ¾ {%"A",!�ðÁ� Æ ÄÁÂð N1RñRñòóT�U�VSô ý Z\`[�ËÂ�ÿÅÄ �SÆ � ^ È ÿ�� ÆG�SÆ � c ZÜÊÏ[]Z�ÊÑÈ�áUc É~¹ Z^Y\[<]Rí4]äcÅ[|Ê É�º _QÊ�Ù ß Z Æ|Ø Ð ØJ � � ¹ Ê É±c ZÞZfef^a[zZv»v»�Z�È8ÊQ»\Ê Æ á�ÈÒZ ¹ Û¡»\Ûxc É `b[]Z]Z Ì ` ÜtZ\[zZÞZ�Ê ¹ ÜÏÙ@Z�Ê Ý c Ý Ì ¹ c É ` Ê�Û É »mÊ�[zZ�ÊÑÈ�áUc É~¹ ZÐàáhY\[<]6Ê É ÈµÌ Ü�ÊQ»Þ[fÊ É�ä Ê�`¤ÚEc0»\`>J Ï Ð ü` PQO~Oba|OHa ð NSR Róò�T�U�VSô+c ¾ {fwmÌ D � ügüSü � Ì�Ó��{Kwz�B{�wU{S��"Ï���>4y��wz�#!kw��<!ÇÐG{��y@?S{X!Åw�}@{%!���w\Ð üW�{j*B�U{�wz�·{��V!>"Ï{�ÉB{S�f�T"Ï×��G��ymwUy@�G�Ë×·�U�Ò�S{SÉ&*B�U{A!���ym� ¾ {�"~"~!�ð·� Æ Ä·F�!k�BÉm/¸{�}m{fw|?·ÔbJ � ÊaÕ¬�a{wz�·{g",!ÅìÁy)"k*#" ÔNw'!kT�{S� ��ÐG{f�A!�}@}R�U{1!kÉ�e_�G�·�S{�A�C�R>���G� J¿Ð>!k�UyU!k�B}@{f�P/MymwU��Ê=wz{S�d"|���#!�ÐÒym�B~�Uym?S{X!kwÖ}m{1!���w\Ð�Õå�V!k�BT§�>4¤wz�·{�ÉB{S�fy@�Uym�G��wz��{S{�~G{f�B{S��!kwz{fÉ���ͧwU�B{ÉB{S�f�T"Ï×��G��ymwUy@�G�Ë×·�U�Ò�S{SÉ&*B�U{k�ë ym�B�S{�{%!��]��Ð>!k�UyU!k�B}@{R�Á�f�%*B���%y@�X!kw¡"Ï�G��w%�G�B{èwU{S��"YF�wU�B{��U{f�%*B���U{S�·�S{S/M�By@�]�X/R{��G�·w'!ky@��y@�Kwz�Bym��U{\wUwUy@�B~Øym�m?·ÔbJ � ÊaÕ Ø ?·ÔbJY� ÆG� Ê�� Æ Õ­ï Æ ü d¬�­�a{�4N�G��{øwz�B{øy@�ByÎwzyU!k}ä�f�G�BÉByÎwzym�G�§ym�m?·ÔbJ �SÆ Õ i Æ4N�G�5!�}m} J � !��BÉâwz��*B�|?BÔNJ � ÊçÕ Ø Ê ü ë y@�B�f{A�=ym���U{%!�É�e_���B�S{j/R{|�<!ÇÐ�{wU�<!kwMÊ Ø J Ï Ð � !��BÉâwz�·{}@{�"~"~!øym��×B�U��ÐG{fÉæ� Ô ¾ {%"A",!�ðÁ� Æ ú�Õ` PQO~Oba­OHa�KÜL­N)O�P�N1R T.U�TÑc ¾ {fw¡�Ï�a{9!�H�exwz{S�d"Ã�U{%!�É�e_���B�S{�A�C�R���ÐG{f�2ÿÅÄ �SÆ �Z^ ü ¾ {%"A"~!¬ð·� Æ úÁFq �B{f�G�U{�"/ð·� Æ !k�BÉ ¾ {%"A"~!|ð·� úwU�G~G{\wz�B{f�¬y)"|×B}ÎÍ�wz�#!kwY�ây@���S�T"Ï×#*·wU{SÉË��Í�!ËÔNJ Ï Ð�ÕFe_ÉB{f�Sym�Uy@���}@ym��w[/M�·{S�U{�{1!k�]�|�T*Áwz×#*·w¤�>4awz�·{�É·{S�Sym�Uym�G�}@y@��w%y@�[!Þ×a�G}mÍÒ�B�>"|y3!�}ÒwU�B�U{f�U�B��}@Éj4h*B�B�\wzy@���Ï��4iÉB{f~G�U{f{L�Ô � ÐÁ}@��~¸HpÕ ü då��ym��wz�B{Ö×B���Á�>4ä�>4 q �B{f�G�U{�" ð·� ñKwz�B{f�U{Öy@�S!�×a�G}mÍÒ�B�>"|y3!�}twz�B��{S�U�·�G}@Ég4h*B�B�\wzym�G�4N�G�-��/M�By@�]��ym�­�>4�ÉB{f~G�U{f{�J Ï Ðíï¿L�Ô � ÐÁ}@�G~¤HpÕ ü ë y@�·�S{X��ym�å��{1!�É�eQ�G�B�f{H�%!����a{k!kwY"Ï�G��w­J �!��BÉ�w'!kTÒym�B~AÐ i J ú OQP Ï }@�G~ ú OQP Jâ×B����ÐG{S�Mwz�·{ÖwU�B{S���U{%"Y� Ô q �B{f�G�U{�" ð·� ðGÕÀRÍ�wU�B{0!��U~T*&"|{f��wU�M~GymÐ�{S�Yy@� ë {S�\wzym�G��ñÁF&/R{Ky)"~"Ï{SÉBy3!kwz{f}mÍÏ�<!ÇÐG{

ë O�PQO�÷�÷�ò�P6� T.U�V.? � ÜtZ ¹ Ù�ÊQ»v»Kc Ý [zZ�ÊÑÈ�áUc É~¹ Z^Y\[<] ¹ Ê É�c ZÖÙ@Z�Ê�[ É Z�È�ìQÛ É�c cÅ`NÜ�` ÜtZ­X W � ÚEc�ÈÒZ\ÙÊ É È|` ÜtZÞÚEc�ÈÒZ\Ù>c Ý Zfe)Ê ¹ `�Ù@Z�Ê�[ É Û É g Ý [zcÅÚ/Z�� ß Ûi_QÊ�Ù@Z É~¹ Z5� ß Z\[àÛxZv»xíÛ É `bÛ ÚEZåñ�gih ^ jmlbn orq_s � lNn ^ v ü

ð Æ

Page 44: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

î NSRñò£P6�[c ë w'!��·É<!��UÉ���{SÉ#*B�\wzym�G�B�2!���{�TÒ�·�6/M��y@�Ø}m{1!����By@�B~Kwz�B{f�G��Í�/M�·y@�]�Y�U{SÉ&*B�S{�wz�·{�×B�U�G�.e}@{�"I�>4¬ã�d�õ }@{%!��U�Bym�B~ËA�C�R wz��wz�<!kw��>4�ã�d�õ }@{%!��U�Bym�B~§��{1!�É.e_�G�B�f{ØA�C�R Ô�ù�{1!����B��Z\`|Ê�ÙÎí@FÆ VGW é !�Õ\�AW6{Ï�B��wU{|wU�<!kwP!�×B×B}ÎÍÒy@�B~�wz�B{f�U{|�U{SÉ&*B�fwUy@�G�B�Þ�B{f�U{EÉ·�Á{f� É cÅ`%ÍÒy@{f}@Él!Ëñ�úg�h ^ jmlNn v exwzy)"|{!�}m~G�G�UyÎwz�#" 4N���å}@{%!��U�·y@�B~,!��U�ByÎwz��!���ÍY×a�G}mÍÒ�B�T"ÏyU!k}�e_��y@?f{ÖA�C�R¸� q �B{��U{fÉ#*B�\wzy@���B�-/¸���UTY��ͧ�S�G�.eÐG{f��wUy@�B~j!KA�C�R¹/Mymwz�ÞÊäÔbJæÕ�wU��w'!�}ç�Ò�S��*B�U��{S�B�f{S���>4æÐ�!���yU!��·}@{S�%wU�j!K��{1!�É�eQ�G�B�f{­A�C�R���ÐG{S�>ÊäÔbJæÕÐ�!��Uy3!��B}m{S�SF#!��BÉ�wz��*B�åy�4�*·�U{SÉ�y@���S���Q�d*B�B�\wzy@����/MyÎwz�Y�T*B��wz�B{f�G�U{�" /¸�>*B}@É�ÍÒy@{S}mÉ�!Eñ�úg�h Ó h ^ v jmlNn v3ewzy)"|{-!�}@~G���UymwU�#"²4N�G��}@{%!��U�Bym�B~���*·�]�|!A�C�R¸������ ü5�5ý¥þ�£)����Â��XÁ �ËÄ�¯Y�Ë�ËÅ  �ÆMÄ�°f±��ÃÂ�Ç[±Ë�ËÄÇ� ­Y�������% f�|�@Çz§G�Ï�

¨ ÂmÆ���þ�£)���� �ÿ�Ï�@Ç>��Æø�Ø�¥« ÂY¤§�¡± ­��Ï� Å �@Ä�ÆMÂd��<!kwV*·�V!�}�~G{f�B{S��!�}@ym?1!kwUy@�G���>4%�>*B�¬��{S�d*B}mwU��4N�G�¬A�C�R�/R�T*B}@É���{Öwz�|×B�U��Ð�{k*B×B×a{S�����>*B�BÉB�¬�G�wz�·{YÉ·{S~G��{S{��>4¬×��G}ÎÍÒ�B�T"ÏyU!�}%wz�·�U{S���B�G}mÉz4h*B�·�fwzym�G�B�k/M�Bym�]�Ñ�f�T"Ï×#*·wz{Ø�fy@���%*ByÎwz���>4¬�Uym?S{�H�!k�BÉÉB{f×·wz��Ì{4N�����·ìÁ{SÉâÐ�!�}3*B{f���>4RÌ�óð ü J �·w'!�ym�By@�·~���*B�]��*B×B×a{S�Ö�a�T*B�BÉ·�SF1�B�Z/R{fÐG{f�SF>�U{S{�"|�­wU��a{¬e�*ByÎwz{MÉBymÓÏ��*B}mwS�pW�{¬�B�kwz{�wz�<!ÅwRwU�B{M×<!��UyÎw_Ík4h*B�B�\wzym�G�|���|J�Ð�!��Uy3!��B}@{f�%y@��T��B�Z/M�Ewz�Þ�U{fe�*By@��{×a�G}mÍÒ�B�T"ÏyU!k}çwU�B�U{f�U�B��}@Ég4h*B�B�\wzy@���B�è�>4äÉB{f~G�U{f{ÞJ=ÔxD�ym�B�UT�Í{!��·É�ãp!�×a{S��wSF Æ VG��W·ï·d¬�U×B�B{f��Z\`yÊ�ÙÎí@FÆ VGVkî�Õ\F�!��·É|�B{f�B�S{�!���Í�×B���Á��4i�>4�{fÐ�{S�g!k�ÏJ�� Æ *B×B×a{S���a�T*B�BÉÏ�G�wz�B{MÉB{f~G�U{f{M�>4æ×a�G}mÍÒ�B�>"|y3!�}wz�·�U{S���B�G}mÉz4h*B�·�fwzym�G�B�j4N�G���f�G�B��wV!���wEÉ·{S×·wU�<×a�G}mÍÒ�B�T"ÏyU!k}¸�Uym?S{��Sym�U��*BymwU�k/¸�T*·}@É#y3"A"|{fÉByU!kwU{S}ÎÍÍÒy@{f}@ɹ!��B{n/ ×B���Á�>4¤�>4�wU�B{k/¸{f}@}ue_T��B�Z/M�m48!��fw­wU�<!kwÖ×<!k�Uymw_Í�y@�å�B��wÖ�S�>"|×#*Áw'!��B}m{ø��Í��f�G�B��wV!���wÉB{f×·wz�Y×���}mÍÒ�B�T"ÏyU!�}a�Uym?S{Ö�Sym�U��*BymwU�ÞÔ�R<*B�U��w�Z\`&Ê�ÙÎí@F Æ VGW�îBï���!���wV!�ÉæF Æ V�WG��Õ\�då�7!{�t����wø��wz{f×�wU�6/y!���ÉB�ø×B����ÐÁym�B~(*·×B×�{f�K�a�T*B�·ÉB�ø�G��×a�G}mÍÒ�B�T"ÏyU!k} wz�B��{S���B�G}@ɹ4h*B�B�\wzym�G�ÉB{f~G�U{f{A4N�G�Ö�f�G�B��wV!���wøÉ·{S×·wU�5�Sym�U�%*·ymwz�fFÑ/¸{|�U�B�Z/ �B�Z/ wU�B{wz{S�]�·�By@e�*B{f�ø�>4Rwz�Bym�Þ�]�#!�×·wz{f�ø�%!���a{,*B��{SÉ=wz���G�·w'!ky@��!��B�G��wz��ymÐÒyU!�}È*B×·×�{f�����>*B�BÉ5�G��wz�B{�É·{S~G��{S{��>4è×��G}ÎÍÒ�B�T"ÏyU!�}9wU�B�U{f�U�B�G}mÉ4h*B�B�\wzym�G�B�S4N�G���U{%!�É�eQ�G�B�S{Ö�f�G�B��wV!���w�ÉB{S×·wU��4N����"k*B}3!�{��KML4N)O�PQNSR T.UE³ ]äcÅ[�ÌËóñ�� Ê É�º []Z�ÊÑÈ�áUc É~¹ Zµ´­c�cÅÙ@Z�Ê ÉÝ cÅ[àÚ ß Ù�ÊËc Ý ÈÒZ_^a` Ü�ÌÑcQ_�Z\[���¶� · � ¸�¹ Ê É@c Z ¹ cÅÚå^ ß `_Z�È c�º Ê­^tcÅÙ º�É cÅÚ�Û�Ê�Ù�`NÜÒ[zZv»]ÜtcÅÙ�È ÝàßSÉ~¹ `bÛxc É c Ý ÈÒZhg�[zZ]Z�L�ÔbJ Dq( jnh½ ¡¾ ¿ n }@�G~ jnh½ ¡¾�¿ n J1Õ ü` PQO~Oba�c q �B{Þ×B�U�Ò�>4äy@�M��ÍØy@�·É#*B�fwUy@�G���G�YÌ ü#q �B{Þ�<!���{Ö�1!���{øÌ i ñ�y@����*·×B×B}@ym{SÉ���Í q �B{S���U{%"ð·� ð·�[W�{Þ�d*B×B×a�G�U{Öwz�#!kw�wz�B{�wU�B{S�G��{%"3�B��}@ÉB�94N�G�MÌ i ñ�� ügüSü �vÒ � Æ !��·É�×B����ÐG{Þymw94N�G�MÌ i Ò ü¾ {fw9�Ø�a{�!ÖÉ·{S×·wU��e�Ò�U{1!kÉ�e_�G�·�S{S4N�G��"7*B}U!·�NW6{M��!�Í�wU�<!kw_!`ªZ\[àÚ ym�È!Þ~T!kwz{�!kw�wz�·{��a��w�wz�T"

}@{\ÐG{f} �>4_��wU�G~G{\wz�B{f��/MymwU�§wz�B{ø}@ymwU{S��!�}@�Mwz�<!Åw�4N{S{fÉ�y@��wz�EymwÇ� ë y@�·�S{X��ym�å��{1!�É�eQ�G�B�f{øwz�B{f�U{��%!���a{j!kwY"Ï�G��wåJ Ï ÐMwU{S��"Ï�­�>4���y@?f{�~G�U{%!kwz{f�¬wz�#!���Ð ü W�{j!�×B×B}ÎÍ�wU�B{�ÉB{f�S�T"Ï×a�G�UyÎwzy@���§×B���Á�f{SÉ#*B��{ÉB{f�U�S��y@�a{SÉ#y@�5wU�B{Ø×B�U�Ò�>4��>4 ¾ {%"A",!§ð·� Æ Ä�wU�Ëwz��!��B�d4N����" �<ym��wz�¹! Æ e_É·{S�Sym�Uym�G�6wz��{S{g/M�B�G��{

ð�ñ

Page 45: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

}@{%!ÇÐG{S�R{1!��]�Ø�f�G��w'!�ym�,!ÞÉB{S×Áwz��e Ò|�U{%!�É�e_���B�S{�4N�G��"k*·}U!�y@�A/M�By@�]��{%!��]�EwU{S��" ym�R��41�Uym?S{-!Åwy"Ï�G��wÐ ü då�Mym� ¾ {�"~"~!�ð·� Æ úøwz�Bym��ÉB{S�fy@�Uym�G�Øwz��{S{Öy@���>4>�V!k�BTg!kw�"Ï�G��wMJ Ï Ð ü� � {1!��]� }@{%!�4Ö�>4Öwz�·y@�|wU�U{f{�F�/R{��U{f×B}U!k�S{Ë{%!��]� wz{S�d" /MyÎwz� !5�B{n/ �É#*#"A"øÍS�5Ð>!k�UyU!k�B}@{wz�#!kw~!�×B×a{1!����|�G�·}mÍ=�G�·�S{��ÚW�{�wz��*B�Ï�G�·w'!ky@�×!�ÉB{f�Sym�Uy@���#wU�U{S{Y�>4å�V!��BT=J Ï Ðk/M�B���U{§}m{1!ÇÐG{f�{1!k�]�Ñ�f�G��w'!�ym��!Ë�U{1!kÉ�e_�G�·�S{�4N�G�d"k*B}U!��>4�ÉB{S×·wU�IÒ�� Æ �pÐ�{S��wU�B{S��{YÉ&*#"~"KÍ6Ð�!��Uy3!��B}m{S�S��ÀRÍwz�·{�y@�BÉ&*B�fwUy@�G����ÍÒ×a��wz�·{S�Uym�SFæ{%!��]�â��*B�]�|4N����"k*B}3!|ym�å{Se�*ByÎÐ�!�}@{f��w¬wz�g!|×��G}ÎÍÒ�B�T"ÏyU!�}æwU�B�U{f�U�B�G}mÉ4h*B�B�\wzym�G���>4 É·{S~G��{S{ÞL�ÔNJ Dq( jnh½ �� ¿�� }@�G~ jnh½ �� ¿�� JæÕ9/M�·y@�]�Yy@�èÉB{n�t�B{fɧ��ÐG{f��wU�B{�É#*#"A"øÍEÐ�!���yU!��·}@{S�ÉB{f�U�S��y@�a{SÉ�!��a�pÐ�{��� ��{1!k�]�Ø��*B�]�Ø×a�G}mÍÒ�B�>"|y3!�}·wz�·�U{S���B�G}mÉ,4h*B�B�\wzy@���æF/¸{­�·�6/ ��{S×B}3!��S{¬{1!��]�EÉ#*#"A"øÍÐ>!k�UyU!k�B}@{/MymwU�{!��U{%!�}�exÐ>!k}3*B{fÉE×a�G}mÍÒ�B�T"ÏyU!k}a��Ð�{S�èwz�·{��G�Uym~Gy@�<!k}çÐ�!���yU!��·}@{S��/M�By@�]�Øym��wz{S��×���}U!kwU{S�è×B��{S�fy@�U{f}mÍ

wz�·{�À¸�Á��}@{1!k��4h*B�B�fwUy@�G�=�f�T"Ï×#*·wz{fÉ ��Í6wz�B{��G��y@~Gym�<!�}%wz{f��"Y� ë ym�B�S{Ø{%!��]�#wU{S�d"µ/y!���>4å��y@?f{!kw7"|�G��wAÐ�� {%!��]�=��*·�]�Ñ×a�G}mÍÒ�B�>"|y3!�}%y@�K�>4�ÉB{S~G��{S{g!Åwj"|����wMÐ ü õ¸�G�B��{Se�*B{S��wU}mÍ5wU�B{,4h*B�B�\wzym�G��S�>"|×#*Áwz{SÉX!Åw%{1!��]�}m{1!�4ç�>4çwU�B{�ÉB{S�fy@��y@�G�øwU�U{f{�y@�¡!­×a�G}mÍÒ�B�>"|y3!�}Gwz�B��{S���B�G}@Éj4h*B�·�fwzym�G���>4aÉB{f~G�U{f{L�ÔiЪJ Dq( jnh½ � ¿�� }@�G~ jnh½ � ¿�� JæÕ üdå�¬y@��wU�B{ø×·�U�Ò�>49�>4 q �B{S�G��{%" ð·� ð·Fç�T*B���G�Uym~Gy@�<!k}�4h*B�B�\wzy@���m���%!��Ë�B�Z/ö�a{ø{àì·×·�U{S���U{fÉl!k�!�×���}mÍÒ�B�T"ÏyU!�}çwz�·�U{S���B�G}mÉ�4h*B�·�fwzym�G���>4>ÉB{S~G��{S{

JÐ ïÑLEÔ�ЪJ Dq( jnh½ � ¿�� }m�G~ jnh½ � ¿�� J1Õ ü

q !�T�y@�B~8Ð i J jnh½ � ¿ n Ï }@�G~ jnh½ � ¿ n J�×B����ÐG{S��wz�B{�wU�B{S�G��{%"Y�ë O�PQO�÷�÷�ò�P6� T.U�V&\ � ÜtZ ¹ Ù�ÊQ»v»§c Ý [zZ�ÊÑÈ�áUc É~¹ Z ¹ c É »\` Ê É `ÜÈÒZ_^a` Ü Ý cÅ[àÚ ß Ù�ÊÒZ ¹ Ê ÉÞc Z�Ù@Z�Ê�[ É Z�È�Û É» ß�c Zfef^tc É Z É `�Û�Ê�Ùæ`bÛ ÚEZSí

ðGð

Page 46: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

� � ��� ����� �

� '/2 � : �6� �=$>'/" ��. � ��2 � .>� � '/2 � .>��� $1�E0

�3" 8 � ���6�#$�� :�� ��!#" $ %'&(�*)+� , �.-0/1) &2) 3

465�7 8:9<;>=@?BADCDEF;2GH?I9J KMLON Ð K PQKRKHS'TUSWV�LXNZY�[\KR]_^`[\LXNa[bPQcO]dYe]dTUPQTUSOfZgihjKHk#lRTiKRSm[ngiK�NZ]QSOTUSefoNZgUfqpZ]dTi[dLOr#P<l�NZStsuKfqT Ð KRSwvxpq]2[\LeKysO]QpmNZzINZSOzIK|{�YO]dKHPdPQT Ð KylRgUNZPdP}pZv�~��y�6vxpq]dr�ceg�NZK�����S�lHpqSm[\]dNZPQ[}[\p�[\LOTiP@Y�pqPQTi[dT Ð K]dKHPdcOg�[��O��K�Sep���PQ[dcOzeh�[\LeK�TiSOLOKH]dKRSm[ygUTirIT�[�N�[dTUpqSOP@pZv(lRpZrIYOc�[�N�[dTUpqSXN�gUgih#K|kIlHTUKHS�[�NZgifqpq]dT�[\LOr#Pvxpq] Ð N�]dTUpZcOPygiK�NZ]QSOTUSef�YO]QpqsOgUKHrIPH��J K�PQLOp��.[dLXN�[�K Ð KHSDvxpq]ylHpqSOlHKRYe[�lHg�NZPQPdKRPy��LOTUl�L6lHpqSm[�NZTiSpqSOg�h Ð KR]�hnPQTUr#YOgUK�lRpZSOlRKHYe[\P�PQcOl�L�NZP�zOKHlRTiPdTUpZSngUTiPQ[dPR��N�PQ[d]dpqSOf�TiSOLOKH]dKRSm[y[\]\N�zOKRpZ� l�N�SDK|{�TUP�[suKH[���KRKRSM[\LOK ]QcOSOSOTiSOf�[\TirIK�NZSOz�P\NZr#YOgiK�lRpqr#YOgiK|{�Ti[�hbpZv�NZSmh6gUKRNZ]dSeTUSOf�N�gUfqpq]QTi[dLOrb���yP�NZSK|{eNZr#YOgiK#p�v�[\LeTUP�[\]dNZzOKRp����:��K#lRpZSOPQ[d]dcOl|[ NBlRpqSOlHKRYe[�lRgUNZPdP�pZv�zOKRlHTUPdTipqSMgUTUP�[\P���LOTil�L��(��LeTUgUKTi[�lRNZSnsuK�gUKRNZ]dSOKHz6N�[Q[\]QTUsOce[dK�KHk#lRTiKRSm[\g�h ��smh_NwSOpqS_Y�pqg�h�SOpqr#T�NZgu[\TUr#K�NZgifqpq]dT�[\LOr� �N�SOznlRNZSsuKbgiK�NZ]QSOKRzjTUS1Y�pqg�h�SOpqr#T�NZg�[dTUr#KM��smhjN SOpZS¡N�[Q[\]dTisOce[dK_K|k#lRTUKHSm[�NZgifqpq]dT�[\LOr� |�@lRNZSOSOpZ[IsuKgUKRNZ]dSeKRz�N�[Q[\]QTUsOce[dKyK|k#lRTUKHSm[\gihITUSIY�pZgih�SOpqr#T�NZgO[dTUr#KZ�(¢FLOTUPfZT Ð KHP�[dLOK�£X]QPQ[zeKRr#pqSOPQ[d]\N�[dTUpqSIpZvN gUKRNZ]dSOTiSOf YO]QpqsOgUKHr0vxpq]��LOTil�LnNa[d[\]QTUsOc�[\K�KHk#lHTUKRSm[�gUKRNZ]dSeTUSOf TiP�¤�¥a¦�§u¨�©«ªZ©­¬«¥a®uªZ¯�¯±°�LXNZ]dz²�³ TUSelRK�cOSelRpqSOzeTi[\TipqSXNZg´YO]Qp�p�vxP�pZv:lHpqrIYece[�N�[dTUpqSONZgXLXNZ]QzOSOKRPQPFPdKHKRrµ[dp�gUTiK���KRgig�suKHhZpqSOz�[\LeK

lRce]d]dKHSm[�P�[�N�[dK�pZv�[dLOK�NZ]Q[@TiS�[\LOKHpq]dK|[\Til�NZgXlHpqrIYece[\KH]2PdlRTiKRSOlHKZ��[\LOKFLONZ]dzOSeKRPdP}]dKHPdcOg�[\P2��LOTil�L#��KKRP�[�NZsegUTUPQL�r�cOP�[>]QKRgih�pqS�cOSOYO]Qp Ð KRS#NZPQPdcOr#Ye[\TipqSOPH�2¶�cO]�]QKRPQcOgi[dP>TUS�[dLOTUP·l�LXNZY�[\KR]@N�]dK�sONZPdKHz pqSfqKHSOKR]dNZgXNZPQPdcOr#Ye[\TipqSOP���[\LOKFK�{�TUPQ[dKRSOlHKyp�v�giKRSOf�[\L�¸¹YO]dKHPdKR] Ð TUSOf�pqSeK|¸¹��N�h YuKR]dr�ce[�N�[dTUpqSeP� :��LeTUl�LNZ]QK���TUzOKHgihILOKRgiz�TUS�[dLOKRpZ]dKH[dTUlRNZg²lH]Qh�Ye[\pZfq]\NZYeL�h�NZSOz�zOp SOp�[F]dKHgihIPdpqgiKRgihIpqS<[dLOK�LXNZ]QzOSOKRPQPFpZvPdYuKRlHTi£Xl�lHpqrIYece[�N�[dTUpqSONZg²YO]QpqsOgiKRr#PFPdcOl�LnNZPFv�NZlH[dpq]dTiSOf pq]FzOTiPdlR]QKH[dK�gipqfmNZ]QTi[dLOrIPH�

^Zº

Page 47: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

»y¼Q½}¼Q½ ¾ ¿2ÀXÁ�Â2Ã�ÀOÁ�¿�Ä��Sw[\LOK�Å2]dpqsONZsOgih���YOYe]dp�{�TUrIN�[\KHgihwV�pZ]d]dKHlH[��­Åu��VF >rIp�zOKRgepZv²lHpqSOlHKRYe[�gUKRNZ]dSOTiSOfO�Z[dLOK�ÆRªZ¦�§u¯UǤ�¥a¦�§u¯UÇQÈq¬�©�°2pZv´NylHpqSOlHKRYe[�lRg�N�PdP>ÉÊTiP·[dLOK�r#TUSeTUr�cOrWS�cOr�suKR]�p�vOg�N�s�KHgUKRz�K|{eNZr#YOgiKRP(��LeTUl�L�N�S�hPdcelRlRKHPdP�vxcOg´gUK�N�]dSOTiSOf�NZgifqpq]dT�[\LOrËvxpq]>ÉWr�cOP�[@]dKHÌ�ceTU]dK��>¢FLOK�P\N�rIYOgiK�lHpqrIYegUK|{�T�[�h�pZv�N�lRpZSOlRKHYe[lRgUNZPdPITUP#pqSOKbpZv�Ti[dPIr#pqPQ[ITUr#Yupq]Q[\NZSm[IYONZ]\NZr#KH[dKR]QPIvx]dpZr NÍgUKRNZ]dSOTiSOf�[\LOKHpq]Qh1YuKR]dPQY�KHlH[dT Ð K�Îvxpq]�TiSOPQ[\NZSOlRK�T�[�TUP�lHgUK�N�]<[\LXN�[bNZSmhÊlHpqSOlHKRYe[_lRgUNZPdP���LOTil�LWLONZPBPdcOYuKR]QY�pqg�h�SOpqr#T�NZgyP\NZr#YOgUKlRpZrIYOgiK|{�Ti[�h�l�NZSeSOpZ[·s�K@giK�NZ]QSOKRz�TUS�Yupqgih�SOpqr#T�N�g�[\TirIK��:��S�NZS�TUr#Yupq]Q[\NZSm[�]dKHPdcOg�[���Ï@Le]dKRS�vxKRcOl�Lm[Ç|©·ªZ¯�Ð�fmN Ð KNyfqKHSOKR]dNZg�gip���KH]�supqcOSOz�pZS�P\NZr#YOgiK}lHpqr#YOgUK�{eT�[�h�smh�YO]Qp Ð TiSOfy[dLXN�[>NZSmh�N�gUfqpq]QTi[dLOr��LOTil�L�Åu��VÑgUKRNZ]dSOP_N`lRpZSOlRKHYe[nlHg�NZPQPBpZv�Ò}NZYeSOTUÓq¸�V�LeKR] Ð pqSOKHSOÓ�TUP_zOTUr#KRSOPQTUpqStÔ1r�cOPQ[_cOPQKÕ �­Ô�ÖZ×\ �K|{eNZr#YOgUKHP�TUSØ[dLOK���pq]dP�[wl�NZPQKM�­Ï2LO]QKRSevxKHcOl�Lm[MÇ|©�ªZ¯�ÐU�FÙ�ÚZÛqÚm |� ³ TUSOlHK�Ü�gUcerIKH]_Ç|©�ªZ¯�ÐLXN Ð KnPdLOp���SÊ[\LXNa[ÞÝß �­Ô�ÖZ×� wK|{eNZr#YOgUKHP<NZ]QKnTUS�vxpq]drIN�[\TipqS�¸«[\LOKHpq]dK|[\TUlRNZgUg�hMPdcek#lRTiKRSm[<vxpq]�Å���VgUKRNZ]dSeTUSOfO��[\LOK6Ò}NZYeSOTUÓq¸�V�LeKR] Ð pqSOKHSOÓ�TUP�zOTirIKHSOPdTipqSàTiP�Ó�SOp���SÊ[\p`lRpZrIYOgiKH[dKRgihol�LONZ]\NZl|[\KH]dTUáHK[\LeK�PdNZr#YOgUK�lHpqrIYegUK|{�T�[�h<pZv�NZSmh�lRpZSOlRKHYe[�lHg�NZPQP��­Ü�gicOr#KR]�Ç|©�ªZ¯�ÐU��Ù�ÚqÛqÚq |�JâLOTUgiK [\LOTiP�TiSevxpq]Qr�N�[dTUpqS�¸¹[\LeKRpq]QKH[\Til�l�LONZ]\NZl|[\KH]dTUáRN�[\TipqSMpZv�PdNZr#YOgUKwlRpZrIYOgiK|{�Ti[�hnTUS�[\KH]dr#P

pZv�ÒyVjzOTUr#KRSePdTUpZS#TiP@s�pZ[dL�TiS�[dKR]QKRPQ[dTUSOf�NZSOz#TUr#Yupq]Q[\NZSm[��qvx]dpZrãN�lHpqr#YOce[�Na[\TUpZSXNZgOP�[�NZSOzeY�pqTiSm[[\LeK<YOTUl|[\cO]QK�TiP�PQ[dTUgig@v�N�]�vx]dpZrälHpqr#YOgUK|[\KZ�n¢FLOTUP�TUP�suKRlRNZcOPQK<[\LOKHPdK�]dKHPdcOg�[\P�p�v�Ï@Le]dKRS�vxKRcOl�Lm[Ç|©IªZ¯�еNZSez�Ü�gicOr#KR] Ç|©IªZ¯�Ðåzep1SepZ[bNZzOzO]QKRPdPB[\LeKMÌ�cOKHPQ[dTUpqStpZv�LOp��ærINZSmhÊK|{eNZr#YOgUKHPBN§X¥a¯±°a®²¥a¦ ¬�ªZ¯�©�¬�¦�ÇIgUKRNZ]dSOTiSOf`NZgifqpq]QTi[\Lerçr�N�hj]dKRÌ�cOTi]dKnvxpZ]�NZSmh¡fqT Ð KHSÊlRpqSOlHKRYe[_lHg�NZPQPR�µ�«¶�vlRpZcO]dPQKZ�PQTUSOlHK_ze]\N���TUSOf NZSoK|{eNZr#YOgiKB[\NZÓZKHP�Na[IgiK�NZP�[wpqSOK�[\TirIK�P�[\KHY��N�Y�pZgih�SOpqr#T�NZg@[\TirIKgUKRNZ]dSeTUSOf�NZgUfZpq]dT�[\LOrèl�NZSI]dKRÌ�cOTi]dK�N�[�r#pqP�[�Yupqgih�SOpZrITUNZgUg�h�rINZSmhwK|{eNZr#YOgiKRPR�é  ³ TUSelRK�TUPdPQcOKRP�pZvlRpZrIYOc�[�N�[dTUpqSXN�g(K|kIlHTUKHSOlHhDLON Ð KIp�lRlHcOYOTiKRzMN<lRKHS�[d]\NZg2Yeg�NZlHK TUSDgUKRNZ]dSeTUSOf�[dLOKRpZ]Qh6K Ð KH]�PQTUSOlHKÒ}NZgiT�NZSm[�êëPPQKRr#TUSXNZg�YXNZYuKR]���Ò}NZgUTUNZSm[��uÙ�ÚqÛZº� ��OTi[TUPpZv(lRpZSOPdTizOKR]dNZsOgUK�TiSm[\KR]QKRP�[F[\pwfmNZTiSBNwsuKH[Q[\KH]cOSOzeKR]dP�[�NZSezOTUSOf�pZv�[dLOK�S�cOr�suKR]}pZv�K�{eNZrIYegUKRP@��LOTil�L�Yupqgih�SOpqr#T�N�ge[\TirIKFgiK�NZ]QSOTUSOf�NZgifqpq]dT�[\LOr#P]dKHÌ�cOTU]QKZ�¢FLOK�£X]QPQ[�TUSOzOTil�N�[dTUpqSB[\LON�[�Yupqgih�SOpZrITUNZg²[\TirIK�giK�NZ]QSOTUSOf#rIN�h_suK�lHpqrIYece[�N�[dTUpqSONZgUg�h<LXNZ]Qz

vxpq]:lRKH]Q[\NZTUS�lRpqSOlHKRYe[�lRgUNZPdPQKRP(��NZP·fqT Ð KRS�s�h�Ò}NZgiT�NZSm[���Ò}NZgUTUNZSm[���Ù�ÚqÛZºm |����LOp�pqsePdKR] Ð KHz�[\LXN�[·[\LeKK|{�TiPQ[\KHSOlRK p�v@Y�pqg�h�SOpqr#T�NZg�[dTUr#K�lHpqrIYece[�NZsegUK�YePdKRcezOpq]\N�SOzOpqrìvxcOSOlH[dTUpqSeP �«í�pqgizO]dKHTUl�L1Ç|©�ªZ¯�ÐU�Ù�ÚqÛZîm @TirIYegUTUKHP>[\LXNa[�[\LeK�lRgUNZPdP�pZv�Yupqgih�SOpZrITUNZgOPdTiáRK�Ü�p�pZgUK�N�SIlHTU]dlHcOTi[dP�TiP�SOp�[Åu��VÊgiK�NZ]QSXNZsOgiKZ�ï�K�NZ]QSOP�NZSOzDÒ}NZgiT�NZSm[�PQcOsOPdKHÌ�cOKRSm[\g�h�YO]dp Ð KRz [dLXN�[�cOSezOKR]�lRKH]Q[\NZTUS�lR]�h�Ye[\pqfZ]\NZYOLeTUl LXNZ]QzOSOKHPdPNZPQPdcOr#Ye[\TipqSOP2[\LeKylHpqSOlRKHYe[�lRg�N�PdP}pZv�Yupqgih�SOpZrITUNZgePQTUáRK�Ü�p�pqgiK�NZS vxpq]Qr�cOg�N�KFl�NZSOSepZ[�suK�gUKRNZ]dSOKHzTUS_Yupqgih�SOpqr#T�N�g²[\TirIKI�­ï�KRNZ]dSOPyNZSOz_Ò}NZgUTUNZSm[���Ù�ÚZÚZº� |��¢FLOTUPy]dKRPQcOgi[���NZP�PQTUfqSOT�£Xl�N�S�[dgih�]QKH£XSeKRzNZSOz�K|{�[\KHSOzOKRzBsmh�ï�LONZ]dT�[\pqSOp Ð �­ï�LXN�]dTi[dpqSOp Ð �´Ù�ÚqÚmðq �����Lep fmN Ð K�N�lR]Qh�Ye[dpqfq]dNZYOLOTil�LXNZ]QzOSOKHPdP]dKHPdcOg�[(vxpZ](N�r#pq]dK@Yup���KR]�vxcOg�gUKRNZ]dSOTiSOf�rIp�zOKHgu��gUKRNZ]dSeTUSOf�cOPQTUSOf�rIKHr�suKR]dPQLOTUY�Ì�cOKR]QTUKHP� |�ZNyr#pq]dK

^mð

Page 48: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

]dKHPQ[d]dTUl|[\KHzØlRpqSelRKRY�[ lRgUNZPdP<�x[\LeK�lRgUNZPdP�ñ�É�ò�pZvFlHpqSOP�[�NZSm[ zOKHYe[\LÍY�pqg�h�SOpqr#T�NZg2PQTUáRKIlRTi]dlRceTi[\P\ |�NZSOz6NI��KRNZÓZKH]�lR]dT�[\KH]dTUpZSbvxpq]�PdcOlHlRKHPdPQvxceg2gUK�N�]dSOTiSOfD�xgUK�N�]dSOTiSOfI[dp�NZlRlHcO]\N�lHh Ù�Öqó�ôâÙ�ÖaY�pqg�h²��õ� cOSOzeKR]F[\LOK�cOSeTivxpq]Qr0zOTiPQ[\]QTUsOc�[\TUpZS´ |�~�KRlRN�[\cO]H�·í�pZgUzO]QKRTUl�L6NZSOz�ö�pqS6��KH]dK [\LeK�£X]dP�[�[\p�P�[\cOzeh6gUK�N�]dSOTiSOf�YO]QpqsOgiKRr#P�vxpq]y��LeTUl�L

Yupqgih�SOpqr#T�N�g²[\TirIK�gUKRNZ]dSeTUSOf#TUP�vxK�NZPQTUsOgiK�sece[�]QKRÌ�cOTU]QKRP�r#pq]QK�K�{ON�rIYOgiKRP�[\LXN�S6gUKRNZ]dSOTiSOf#cOPdTiSOfN�lHpqrIYece[�N�[dTUpqSONZgUg�h cOS�s�pZcOSOzOKHzbN�gUfqpq]QTi[dLOr÷�­~�KHl�N�[dcO]�Ç|©�ªZ¯�ÐU�uÙ�ÚqÚqÚq |�>¢FLOK|h<TiS�[d]dp�zOcOlHKRz<[\LeKSOpZ[dTUpqSwpZv�[\LOK�¤�¥a¦�§u¨�©¹ªZ©�¬«¥a®uªZ¯eÆRªZ¦�§u¯UÇ�¤�¥a¦�§u¯UÇQÈq¬�©�°pZv�N�lHpqSOlRKHYe[�lRg�N�PdP�É�øZ��LeTUl�L�TiP>[\LOK�r#TUS�¸TUr�cOrÞS�cer�s�KH]pZv·g�N�s�KHgUKRzIK|{eNZr#YOgUKHP���LOTil�LbN�S�hIYupqgih�SOpqr#T�N�g´[dTUr#KygiK�NZ]QSOTUSef NZgUfZpq]dT�[\LOr*vxpZ]ÉWr�cOPQ[@]QKRÌ�cOTU]QKZ�>�yrIpZSOf�pZ[dLOKR]2]QKRPdcegi[\P>[dLOKHh�KRP�[�NZsOgiTUPQLOKRz [\LeKK|{�TUP�[\KHSOlRK�pZvulRpZSOlRKHYe[}lRg�N�PdPdKHP��LOTil�L LXN Ð K�NZ]dseTi[\]dNZ]dTigih�gUNZ]dfZK�fmN�YOP>suKH[���KRKRSw[\LeKRTU]�P\NZr#YOgUK�lRpqr#YOgiK|{�Ti[�h�NZSOz lHpqr#YOce[�Na[\TUpZSXNZgP\N�rIYOgiK�lRpqr#YOgiK|{�Ti[�huùú�û:ü�ý´þaü�ÿ ³���������ü���� ·þ�������������������� �"! Ç|©�#:��õ� %$|Ç�ªZ®X°§X¥a¯±°�®²¥a¦ ¬�ªZ¯�Æ|¨´¤'&w©(&eªZ©)#·��õ� +*õ ü-,�. ªZ®X°b¥a®²Ç0/21}ªZ° . ¨�®²¤|©�¬«¥a®`ÇQÈq¬UÆ|©�Æ'3�©�&XÇ|®M©�&XÇ04\Ç�¬UÆ�ª6¤�¥a®²¤�Ç�§u©�¤|¯�ªaÆ�Æ�É ¥ . §X¥a¯±°a®²¥a¦ ¬�ªZ¯�Æ|¬65aǤ|¬(4\¤|¨�¬�©�Æ�Æ|¨X¤'&_©�&eªZ©

7 ªZ®X°�§X¥a¯±°a®²¥a¦ ¬�ªZ¯:©­¬�¦�Ç�8 9;:â¯UÇdª<4�®X¬�®�=_ªZ¯>=�¥?4�¬�©(&�¦ . ¥?4�É+¦ ¨mÆ|©F¨�Æ�Ç Õ �@#·�xõ� QÖZ×� IÇQÈ�ªZ¦A/§u¯UÇ�Æ'3

7 ©(&XÇ04\Ç�¬UÆ�ª<¤�¥a¦�§u¨�©¹ªZ©�¬«¥a®uªZ¯�¯±°�¨�®B$|¥a¨�®BC�Ç�CD8 9;:o¯UÇdª<4�®X¬�®�=IªZ¯>=�¥?4�¬�©(&�¦ . ¥?4�ÉE1F&�¬«¤'&<¨mÆ�Ç�Æß ��õ�Ö�×� �ÇQÈ�ªZ¦�§u¯UÇ�ÆRÐ

¢FLOKMYO]dp�pZv�pZv�¢FLeKRpq]QKRr ºO�éóØ]dKRgiTUKHP�KRPdPQKRSm[\TUNZgUg�h¡pqSâ[\LeKMTizOK�NØ[dLXN�[_NoYOPdKHcOzOpq]dNZSOzOpqrfqKHSOKR]dN�[\pq]�lRNZS�s�K cePdKRz�[\pBLOTizOK�TiSevxpq]drIN�[dTUpqS_vx]dpqrÑN�lRpqr#YOce[\N�[\TipqSXNZgigihBsupqcOSOzOKHz�gUK�N�]dSOKH]sOce[�SOp�[Fvx]dpqr N lHpqrIYece[�N�[dTUpqSONZgUg�hwcOS�s�pqceSOzOKRzbgUKRNZ]dSOKH]R�

»y¼Q½}¼2G HØ¿JILKNMyÀOÃ�ÀOÁ�¿�Ä�ÃPO%Q�ÃPILKNOSRTHÍ¿UILKNOVR-W�Á¹À�XWÂUR�Y[Z\M]Z^Q�ÃPI_KNOVR`HØ¿JIbaKNOSRcW�Á¹À�X

J K<PQ[d]dKRSefZ[\LOKHS`[\LOK<]QKRPQcOgi[dP�pZvy~�KRlRN�[\cO]BÇ|©�ªZ¯�Ð�smhÍKRPQ[\NZsOgiTUPdLeTUSOfbPQ[\]QpqSOfqKH]�fmNZYeP�s�K|[���KHKRSP\N�rIYOgiK�lHpqr#YOgUK�{eT�[�hbNZSOz6lHpqrIYece[�N�[dTUpqSONZg:P\NZr#YOgUK�lRpZrIYOgiK|{�Ti[�hbNZSez6smhnPQLOp���TUSOf�[dLXN�[�PdcOl�LfmNZYePFl�NZSBK|{�TiPQ[�K Ð KRSBvxpq]lHg�NZPQPdKRPF��LOpZPdK�lRpqSelRKRY�[\P�LXN Ð K�NZSBK|{�[d]dKRr#KRg�h<PQTUr#YOgUK�NZSOzBSXN�[dcO]\NZg]dKHYO]dKHPdKHS�[\N�[\TipqSØN�P�zOKHlRTiPdTUpZS giTUPQ[dPR��¶�cO]�£O]dPQ[�lRpZSOPQ[d]dcOl|[\TUpZSMh�TUKHgUzOP�N_lRpqSOlHKRYe[�lRgUNZPdP���LOpqPQKlRpZSOlRKHYe[\P�NZ]dK�ÙH¸�zeKRlRTiPdTipqSwgUTUP�[\P}NZSOzw��LOTil�L<LXNZP@[\LOK�vxpqgigUp���TUSOf�YO]dpqYuKR]�[�huù@N�lHpqrIYece[�N�[dTUpqSONZgUg�hcOS�supqcOSOzOKHz�gUK�N�]dSOKH]�l�NZSDgiK�NZ]QSn[\LeK�lRgUNZPdP�vx]Qpqr ß ��Ù�ÖZ×� �K�{eNZrIYegUKRPH�²sOc�[�cOSOzOKH]�N<PQ[\NZSOzXNZ]Qz

^qî

Page 49: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

lR]�h�Ye[\pqfZ]\NZYOLeTUlBNZPQPdcOr#Ye[\TipqS1NZSmh`Yupqgih�SOpqr#T�N�g}[dTUr#K<gUKRNZ]dSOKH] ]dKHÌ�ceTU]dKHP#NZgUr#pqPQ[ed#�xõ�ÖZ×\  K�{�¸NZr#YOgUKHPR�b¢FLOTUP�lHpqSOP�[\]dcelH[\TipqS`cOPQKRP KR]Q]dpq] ¸�lHpq]d]QKRlH[dTUSOfblRp�zOKRP�NZSOz`]dKHÌ�cOTU]QKRP�pqSegih Ð KH]QhØsONZPdTillR]�h�Ye[\pqfZ]\NZYOLeTUl�TUzOKRNZP��x[dLOK�SOpZ[dTUpqS�pZv�NwpqSOK|¸«�N�h<vxceSOlH[dTUpqS´ ��¶�cO] PdKHlRpqSOz1lRpZSOPQ[d]dcOl|[\TUpZS1rINZÓ�KRP r#pq]dK�K�{�[dKRSOPQT Ð K_cePdKBpZv�lR]�h�Y�[\pqfq]dNZYOLOTil�r�N�l�LOTUSOKH]Qh

NZSOz�h�TUKHgUzOP�[dLOK#vxpZgUgUp���TiSOf<]dKHPdcOg�[�ù�vxpq]�NZSmhgfT* Ù [dLOKR]QKITiP�N_lHpqSOlRKHYe[�lRg�N�PdP�pZvhf�¸�zOKHlRTiPdTUpZSgUTiPQ[dP���LOTUl�LMl�NZSMsuK#giK�NZ]QSOKRzMsmh�NBlRpqr#YOce[\N�[\TipqSXNZgigih_cOS�s�pZcOSOzOKHzØNZgifqpq]dT�[\LOr vx]Qpqr ß � Ù�ÖZ×� K|{eNZr#YOgiKRPR�´sOc�[ylRNZSOSOp�[�suK�gUK�N�]dSOKHzbsmhbNZSmhBY�pZgih�SOpqr#T�NZg´[\TirIK�N�gUfqpq]QTi[dLOråvx]Qpqr0vxK|��KH]y[\LONZSÕ ��õ i�ÖZ×\ IK|{eNZr#YOgUKHP���ceSOzOKR]BNÍ��TizOKRg�hàLeKRgUzÊlH]Qh�Ye[dpqfq]\N�YOLOTUl6NZPdPQcOrIY�[\TUpZS´ |�Ë�yP<��KDPQLOp����[\LeTUP�TUP���Ti[dLOTUSÍN_gipqfmNZ]QTi[\LerITil�v�NZlH[dpq]�pZv[dLOK�gUNZ]dfqKHPQ[�YupqPdPQTUsOgiKIfmN�YMvxpq]�lRpZSOlRKHYe[�lHg�NZPQPdKRP�pZvf�¸¹zOKRlHTUPdTipqSBgUTiPQ[\PH�JÍK�NZgiPdpwlRpqSePdTUzeKR]�[\LOK�TUr#YOgiTUl�Na[\TUpZSOP�pZv�pqcO]�]dKHPdcOg�[\PFvxpq]F[\LeK�YOLeKRSOpqr#KRSepqSbp�v�ªZ©�©j4�¬k$R¨�© Ç0/

Ç�lʤ|¬«Ç|®X©�giK�NZ]QSOTUSefO� ö�pqcOfqLOg�htPdYuK�NZÓ�TUSefO��NàlHpqSOlHKRYe[�lHg�NZPQPDÉ TiPnP\N�TUz'[\pàs�K`Na[d[\]QTUsOc�[\K|¸KHk#lHTUKRSm[dgih�giK�NZ]QSXNZsOgiK>Tivm[dLOKR]QK@TUP�NgiK�NZ]QSOTUSOfNZgUfqpZ]dTi[dLOrÊvxpq]�Éj��LeTUl�L�]dKHÌ�ceTU]dKHP:pqSOgihyYupqgih²��PQTUáHKm�jmH |øgUpZf}õ� �K|{eNZr#YOgUKHP�[\pBgUKRNZ]dSMNZSmhDlHpqSOlHKRYe[nmpojÉ p Ð KH]�õ Ð NZ]QT�NZsegUKRP �x��KIfqT Ð KwN�YO]QKRlRTiPdKwzOK|v�¸TUSeTi[\TipqSØTiS ³ KRlH[dTUpqSoºO�éðq ��`�[d[\]QTUsOc�[\K|¸¹KHk#lRTiKRSm[#gUKRNZ]dSOTiSOf�NZgifqpq]dT�[\LOr#P�N�]dK�cOPQKHvxcOgF��LOKHSj[\LeK[�N�]dfqK|[BlRpqSOlHKRYe[bzOKHY�KHSOzOPnpZSâvxK|� Ð NZ]dTUNZsOgiKRP<sOc�[bõ(ø[\LOKD[dpZ[�NZg�S�cOr�s�KH]_p�v Ð NZ]QT�NZsOgiKRPH�yTiPg�N�]dfqK���ö�KRPQcOgi[dP�pZvJq�N�cOPdPQgUKR]#�kq�NZcePdPdgiKR]H�2Ù�ÚqÛqÛm �NZSOzgr�Ti[Q[\gUKHPQ[dpqSOK��kr�Ti[d[dgUKHPQ[\pZSOKZ�(Ù�ÚqÛqÛm �h�TiKRgizN�[Q[\]dTisOce[dK|¸�K|k#lRTUKHSm[�gUK�N�]dSOTiSOfBNZgUfZpq]dT�[\LOr#Pyvxpq]Nf�¸�V�y�WNZSezsfm¸ ~����àvxpZ]dr�cOgUNZKZÎ:r#pq]dK ]QKRlHKRSm[]dKHPdcOg�[\P@pqSIN�[d[d]dTisOce[\K�¸�K|kIlHTUKHSOlHh lRNZSws�KFvxpqceSOzITiSb�­Ü�gicOrìÇ|©>ªZ¯�ÐU�XÙ�ÚqÚmð�ÎqÜ�gUcOr_�OÙ�ÚZÚqîeÎ�t�KRLXNZ]dNÇ|©�ªZ¯�ÐU��Ù�ÚqÚ�u�ÎeÒ}NZgUTUNZSm[���Ù�ÚqÚqÚm ��

r�KH[wv i suKy[dLOK�lHg�NZPQP�pZv@ÙH¸¹zOKRlHTUPQTUpqSIgUTUP�[\P}p Ð KR]yx?zeø�Ù<{}|���LOTil�LBNZ]dKypZv�gUKHSOfZ[dL^f ü ³ TUSOlHK�[\LeKÒ}NZYOSeTUÓq¸�V�LOKH] Ð pqSOKRSeÓ�TiPwzOTUr#KRSOPQTUpqS`p�v]v i TUP�K�NZPQTUgih PdLep���S1[\pDsuK ß �jf�gUpqf@õ� �ø�[\LeK_]QKRPQcOgi[dPpZv�Ü�gicOr#KR]<Ç|©�ªZ¯�Ð6TUr#YOgihn[\LXN�[�[\LeKR]dK�TUP�N_sO]dc�[\K|¸«vxpq]dlHK<gUKRNZ]dSeTUSOfbNZgUfZpq]dT�[\LOr+vxpZ]Dv i ��LeTUl�LcOPQKRP ß �jf�gUpqf@õ� �K�{eNZrIYegUKRPH�2¢FLOTUP�N�gUfqpq]QTi[dLOrb�m��LOTigUK�T�[FTUP�Na[d[\]QTUsOc�[\K|¸¹KHk#lRTiKRSm[��ezOp�KHPFSOpZ[�]QcOSTUS`Yupqg�h��xõ� �[dTUr#KZ� Ü�gUcer ��Ü�gUcOr_�Ù�ÚZÚqîm �NZSezoÒ}N�gUT�N�S�[��xÒ}NZgiT�NZSm[R�Ù�ÚqÚqÚq �LXN Ð KBK�N�l�L1YupqPQKRz[\LeKnÌ�cOKRP�[\TipqS¡pZv���LeKH[\LeKR]�[dLOKR]QKnK|{�TiPQ[\P�N Yupqgih�SOpZrITUNZg�[\TirIK_N�[Q[\]QTUsOce[dK|¸¹KHk#lRTiKRSm[<giK�NZ]QSOTUSOfNZgifqpq]dT�[\LOrWvxpq]~v i ü ³ cOl�L#NZSwNZgifqpq]QTi[\Ler'lRpqcegUz�YupZ[\KHSm[\T�N�gUgih�suKFN Ð KR]�h�cOPQKHvxcOge[dp�pZg�TiS�rINZl�LOTUSeKgUKRNZ]dSeTUSOfO�JÍK [\NZÓZK#N�PQ[dKRY [dp��NZ]QzÍN�SOPQ��KR]QTUSOf_Ü�gicOræNZSezMÒ}NZgiT�NZSm[Rê P�Ì�cOKHPQ[dTUpqSMsmhDYO]dp Ð TUzeTUSOf�[\LeK

£X]QPQ[�YO]Qp�p�v2[\LXN�[�N�[d[d]dTUsece[\K�¸�KHk#lHTUKRSm[�giK�NZ]QSOTUSef�l�N�SDs�Kb¤�¥a¦�§u¨�©«ªZ©­¬«¥a®uªZ¯�¯±°�LXNZ]dz²��JÍK�zep<[\LOTiPsmhoK|{�LOTisOTi[dTUSOfMN�lRpqSOlHKRYe[IlRgUNZPdP#pZv�zOKRlHTUPdTipqSogUTiPQ[\P ��LOTil�LàlRNZSjsuK_gUK�N�]dSOKHzjTiSjYupqgih�SOpZrITUNZg[\TirIKyNZSOz�l�NZS�suK�giK�NZ]QSOKRz<smh<N�lRpqr#YOce[\N�[\TipqSXNZgigih�cOS�s�pqceSOzOKRz_N�[Q[\]QTUsOce[dK|¸¹KHk#lRTiKRSm[giK�NZ]QSOTUSOfNZgifqpq]dT�[\LOr�sOce[<lRNZSOSOp�[���ceSOzOKR]<NÍYOg�N�cOPdTisOgUKBlR]�h�Ye[\pqfZ]\NZYOLeTUl6N�PdPdcerIYe[dTUpqSX #suKngUKRNZ]dSeKRz¡TiS

^�u

Page 50: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

Yupqgih�SOpqr#T�N�gu[dTUr#K�smh_NZSmhBN�[d[d]dTisOce[\K�¸�K|kIlHTUKHS�[�N�gUfqpq]QTi[dLOrb�

»y¼Q½}¼�� ��M]Y���KNK]YX¿}Ã~�B��ÞlRpqr#r#pqSDYXNZ]dNZzOTUfZrìvxpq]�lHpqSOlHKRYe[dP�NZSez K�{ON�rIYOgiKRP�TUP�cePdKRz�vxpq]�NZgig>pqcO]�]QKRPQcOgi[dPR�w��S KRNZl�LpZv�[dLOK�lHpqSOlHKRYe[FlHg�NZPQPdKHP���LOTUl�LI��K�lHpqSOPQTUzOKH]�K�N�l�L<lRpqSelRKRY�[FTUP�NZPdPQp�lHT�N�[dKRz���T�[\L�N�PdKHlR]QKH[FÓ�KHhuÎTi[�TiP�KRNZPQh�[\p_K|{eNZl|[\g�h TizOKRSm[dTiv�h6[dLOKI[\NZ]dfqK|[�lRpqSOlHKRYe[�Tiv}[\LOTiP�Ó�KHh TUP�Ó�SOp���S��b�ygUPQpO�(TiS KRNZl�LpZv�pqcO]�lRpqSOP�[\]QcOlH[dTUpqSOP�K|{eNZr#YOgUKHP�lHpqr#K#TiSM[���pn[�h�YuKRPR�(��LOTUl�LM��KIl�NZgig�¨mÆ�Ç . ¨�¯�NZSOzj¨�Æ�Ç|¯UÇ�Æ�Æ|�tyPQKHvxcOg´K�{ON�rIYOgiKRP2KRNZl�L�lHpqSm[�NZTiSIN�S#KHSOlR]�h�Ye[\KHz Ð KH]dPdTipqS#pZv�[dLOK�PdKHlR]dK|[�Ó�KHh#NZP>��KRgig´NZP}N�PdrINZgUgNZr#pqcOSm[�pZv}cOSOKHSOlR]�h�Ye[\KHzØTUSevxpZ]drIN�[\TipqS6NZsupqce[�[dLOK [�NZ]QfqKH[�lHpqSOlHKRYe[R�etyPdKHgUKHPdP�K|{eNZr#YOgUKHP�NZgUgLXN Ð K�g�N�s�KHg-zwNZSOz_lRpqSm[�N�TUSBSOpwTUSevxpZ]drIN�[\TipqS�NZsupqce[F[dLOK�[�NZ]QfqKH[�lHpqSOlRKHYe[��¶�cO]ylHpqSOP�[\]dcelH[\TipqSOPyNZ]dK�sXNZPQKRzbpqSb[dLOK�vxpZgUgUp���TiSOf PdTirIYOgiK�TUzOKRNeù�NIlRpZrIYOc�[�N�[dTUpqSXN�gUgih#cOS�¸

supqcOSOzOKHzwgUK�N�]dSOTiSOf�NZgifqpq]dT�[\LOr lRNZSwzOKRlH]Qh�Ye[@[\LOKFPQKRlH]dKH[}Ó�KHhwNZSez LOKRSOlHK�l�NZSwgUKRNZ]dS�[dLOK[�N�]dfqK|[lRpZSOlRKHYe[>K�{ON�lH[\g�h�vx]dpqr N�PdTiSOfqgUK@cOPQKHvxcOg�K|{eNZr#YOgiKZ��V�pqSOPdKHÌ�cOKRSm[\g�hq�mPQcOl�L NygiK�NZ]QSOTUSefyN�gUfqpq]QTi[dLOr]dKHÌ�cOTU]QKRP�vxK|�tK|{eNZr#YOgUKHPR�@¶�S�[dLOK�pZ[dLOKR]�LXNZSOz²�ON�Yupqg�h�SepqrITUNZge[dTUr#K�giK�NZ]QSOKR]�l�NZSeSOpZ[zeKRlR]�h�Ye[[\LeK�PdKRlH]dK|[�ÓZKHhuδTUSePQ[\KRNZz���Ti[lRNZSBpqSOg�hIcePdK�[\LOK�PdrINZgUg�NZr#pqcOSm[FpZv·cOSOKHSOlR]�h�Ye[\KHzbTiSevxpq]drIN�[dTUpqSTUSbK�NZl�LDcePdKHvxceg�K|{eNZr#YOgiKZ�yqyKRSelRK TUSm[dcOTi[dT Ð KHgihBNIY�pqg�h�SOpqr#T�NZg²[dTUr#K�giK�NZ]QSOKR]���TUgig:SOKRKHzDr�NZSmhcOPQKHvxcOg�K|{eNZr#YOgiKRP TUSopq]QzOKR]�[dpÍN�lRÌ�cOTU]QKBNDPdTUfZSOTi£XlRNZSm[#NZr#pqcOSm[wpZvyTUSevxpq]Qr�Na[\TUpZSØNZsupqce[w[\LeK[�N�]dfqK|[�lRpqSOlHKRYe[R�

465�� �*=~�N�RG�� G�9���=@G��]�¢FLO]QpqcOfqLOpZce[�[\LeTUP�l�LONZYe[\KH]���K���TUgig(pqSegihBlRpZSOPdTizOKR]�lHg�NZPQPdKHP�pZv@lRpZSOlRKHYe[\P���LOTil�L�NZ]dK�zOKH£XSeKRzp Ð KH]�[\LeKIÜ�p�pqgiK�NZSMlRces�K^x?zeø�Ù<{ | ü ö�KRlRNZgUg>[dLXN�[�TUS�[\LOKIÜ�p�pqgUKRNZSMÅ���V*giK�NZ]QSOTUSOfBr#p�zeKRg>[\LeKgUKRNZ]dSeKR]LXN�P�NZlRlHKRPdPF[dpwNZSØÇQÈ�ªZ¦�§u¯UÇ�¥?4Qª�¤|¯UÇJ���1�kmaø�� |  }��LOTil�L_pZS_KRNZl�LBl�NZgig�[\NZÓZKHP�pZSOK�[\TirIKPQ[dKRYâNZSez¡pqce[dYOce[\P<NÍg�NZsuKRgiKRzjÜ�p�pqgUKRNZSàK�{ON�rIYOgiK`�k�:ø�mZ�(�� ��#��LOKH]dK��âTiP�zO]dN���S¡vx]dpqrç[\LeKzOTiPQ[\]QTUsOc�[\TUpZS�� | p Ð KR]Nx?zeø�Ù�{}| ü í�T Ð KRS<[���pIÜ�p�pZgUK�N�S<vxcOSOlH[dTUpqSeP%��ø�m�NZSOz_N zOTiPQ[\]QTUsOc�[\TUpZS�� |p Ð KH]�x?zeøRÙ<{}|�øO��K�P\N�h�[\LON�["�o¬UÆy×�/�ª�¤�¤|¨�4QªZ© Ç�¨�®BC�Ç04w� | 12¬�©�&�4\Ç�Æ�§XÇ�¤|©}© ¥Dm�T�v>Å2]��� ¢¡�£¤¥�:�(�� �¦§m����� V¨�©¡×RÎ�NZgi[dKR]QSXN�[\T Ð KHgihq�ZPdcOl�L�N�vxcOSelH[\TipqSp�ITUP2PdNZTUzw[\p�×�/�ª|§m§B4\¥HÈq¬�¦#ªZ©�Ç�©(&XÇ�¤�¥a®²¤�Ç�§u© m�¨�®BC�Ç04� | ü ��SBNZgifqpq]QTi[\Ler�v1TiP�PdNZTUz�[dp�suK�Nª8 9;:o¯UÇdª<4�®X¬�®�=IªZ¯>=�¥?4�¬�©(&�¦ . ¥?4�¤�¥a®²¤�Ç�§u©�¤|¯�ªaÆ�ÆÉ Tiv�[\LeKvxpqgigUp���TUSOfBlHpqSOzOT�[\TUpZS LOpZgUzOPHù�vxpq]�N�gUg2õ«*ìÙqø:vxpq]�K Ð KH]Qh zOTUP�[\]QTUsOce[dTUpqS¬� | p Ð KR]�x?zeøRÙ<{}|�ø:vxpZ]K Ð KR]�h¬m;o`É0N�SOz6vxpq]�K Ð KH]Qh¬z^©Ë×Rø'­�©*Ùqø²T�v®vtTiP�fqT Ð KRS�N�lRlRKHPdP�[dp^���1�kmaø�� |  �[\LeKRS6��Ti[dLYO]QpqsXNZsOTigUT�[�h<N�[�giK�NZP�[�Ù�ô¯­�øuNZgUfqpZ]dTi[dLOr_v¡pqce[\Yece[\PyN#Lmh�YupZ[\LOKHPdTiP]�b��LOTil�L6×�¸�NZYOYO]Qp�{eTir�Na[\KRPm�cOSezOKR]w� | ü

^qÛ

Page 51: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

¢FLOK�vxpqgUgip���TUSOfbzOKH£XSeTi[\TipqSOPwNZ]QK<vx]dpqr �­~�KHl�N�[dcO]nÇ|©�ªZ¯�ÐU��Ù�ÚqÚqÚm �ù�¢FLOK°CZ¬UÆ|©k4�¬k$R¨�©�¬«¥a® . 4\Ç�Ǭ�® . ¥?4�¦#ªZ©­¬«¥a®�©(&XÇ�¥?4\Ç|©­¬«¤�ÆRªZ¦�§u¯UÇ�¤�¥a¦�§u¯UÇQÈq¬�©�°FpZv�N�lHpqSOlRKHYe[�lRg�N�PdP�É�ømzOKHSOpZ[dKRzn±J²D³P´��­ÉwÎ\õ(ø�×� |øTUP�[dLOK2r#TUSOTir�cOrâP\NZr#YOgiK>PdTiáRK��­N�P:NFvxceSOlH[dTUpqS�pZv�õwNZSOz�×� �SOKHKRzOKHz�vxpZ]:Å���VMgiK�NZ]QSOTUSef�[\LOK@lHg�NZPQPÉ���Ti[dL�N�lRlRce]\NZl|h#×�NZSezIlHpqSe£XzOKHSOlRK"­ § ÚmÖ�Ù}zeøZ��LOKH]dKySOp�lHpqr#YOce[�Na[\TUpZSXNZgegiTUr#Ti[\N�[\TipqSOP·K|{�TUP�[pqSD[\LeK gUK�N�]dSOTiSOf�NZgifqpq]dT�[\LOr#P���LOTUl�L�rIN�hbsuK cOPdKHz���¢FLOKµCZ¬UÆ|©k4|¬k$R¨�©�¬«¥a® . 4\Ç�Ç<¤�¥a¦�§u¨�©¹ªZ©�¬«¥a®uªZ¯ÆRªZ¦�§u¯UÇ�¤�¥a¦�§u¯UÇQÈq¬�©�°pZv²N�lHpqSOlRKHYe[�lRg�N�PdP}É�øZzOKRSOp�[\KRze´c³P´��­ÉwÎ\õ�ø\×� |øqTiP2[\LOK�r#TUSOTir�cOr.P\NZr#YOgUKPdTiáRK<�­NZP�NIvxcOSOl|[\TUpZS6pZv2õÍNZSOzD×� �SOKRKHzOKRz6vxpq]�Å���V'gUKRNZ]dSeTUSOfw[\LOK�lRg�N�PdP�Éã��Ti[dL6NZlHlRcO]dNZlHh6×NZSOz�lHpqSe£XzOKHSOlRKp­ § ÚmÖ�Ù}zeø���LeKR]dKw[\LeK gUK�N�]dSOTiSOfBNZgUfZpq]dT�[\LOr÷r�cOP�[�pqYuKR]\Na[\K TUS�Yupqgih�SOpZrITUNZg��TiSBõ�NZSOzMÙ�ÖZ×� �[\TirIK���¶f /SC�Ç�¤|¬UÆ|¬«¥a®j¯±¬UÆ|©�¥ . ¯UÇ|®�=Z©�&�·<p Ð KR] [dLOK_Ü�p�pZgUK�N�S Ð N�]dT�N�sOgUKHPD�c¸�ø üSügü ø�� | TiP N�Ü�p�pZgUK�N�SvxcOSOl|[\TipqS¹v0��LOTUl�LtTUP�]QKRYO]QKRPdKHSm[\KRzËsmhâNogUTiPQ[�p�vn·�YXN�TU]dPM��º�¸|ø'»�¸Q |ø��kº½¼�ø'»'¼� �ø üSüSü ø��kºp¾�ø'»�¾Q |ø

��LOKH]dK�K�N�l�L¿º�À�TiP�N6lHpqS?ÁQcOSOl|[\TipqS`pZv�N�[ r#pqP�[efÍgUT�[\KH]\NZgiP�p}ÂqKH];�c¸|ø üSüSü ø�� | N�SOz`K�NZl�L¿»�ÀFTiPKRT�[\LOKH]%zIpq]�Ùq�í�T6ÂqKRSnNZSmh½�¬o`x?zeø�Ù<{}|�øO[\LeK�Â�NZgicOK�pZvPv����� FTUP%»�À�T�v�Ã@TUP�[\LeK�PQr�NZgigUKHPQ[�TUSezOK|{Pdcel�Lb[\LON�[wº�À·TUPFP\Na[\TUP�£XKRz²ÎuT�v·SOppº�À·TiPFP\N�[dTUPQ£OKRzB[\LOKHS�v�����  § z üJÍK���]dT�[\Kw�%ÄÆÅ�[dp�zOKHSOpZ[dKy[\LeKylHpqSOl�Na[\KRSON�[\TipqSIp�v�sOTiSXNZ]Qh�PQ[\]QTUSOfZP®�:ø�ÅwN�SOz°Ç �~Ça[dp�zeKRSOpZ[dK

[\LeK�giKRSOfZ[dLbp�vF� ü JÍK�P\N�h�[dLXN�[yN#YuKR]Qr�ce[\N�[\TipqS�ÈDù�x?zeø�Ù<{?ÉJÊ x?zeø�Ù�{?É�TiP�¯UÇ|®�=Z©(&�/U§B4\Ç�Æ�Ç04�Ë�¬�®�=TivwÇÌÈ>���� }Ç § Ç �~Çmvxpq]�N�gUgÍ��osx?zeø�Ù<{ É ü�ç¯UÇ|®�=Z©�&�/U§B4\Ç�Æ�Ç04�Ë�¬�®�=1¥a®²Ç0/21}ªZ°b§XÇ04�¦ ¨�©¹ªZ©�¬«¥a®oTUP�NÍgiKRSOf�[\L�¸¹YO]dKHPdKR]SÂ�TUSOf`YuKR]Qr�ce[�Na[\TUpZSÎÈ

��LOTil�L�LXNZP·[\LOK�vxpZgUgUp���TiSOf�YO]dpqYuKR]�[\TiKRPRù�[dLOKR]QKTUP(NyzOK|[\KH]dr#TUSOTiPQ[\Til�Yupqg�h�SepqrITUNZgZ[dTUr#K�N�gUfqpq]QTi[dLOr��LOTil�L�lRpqr#YOce[dKRP�Èuø�sOce[(vxpq](PQcek#lRTiKRSm[\g�h�gUNZ]dfZK}õ#[\LOKH]dK�TUP·SOp�YO]QpqsXNZseTUgUTiPQ[dTUl2Yupqgih�SOpqr#T�N�g�[\TirIKNZgifqpq]dT�[\LOrã��LOTil�LbTiS�ÂqKR]�[\P"È6pqSnN�Ù�ÖaY�pqg�h²��õ� �vx]dNZlH[dTUpqS�p�vJx?zeøRÙ<{}| ü465'Ï Ð Ñµ?#9Ò�(;>=}CDEF;2GH?I9 ÓÔ�>G�9�Õ Ö'=@=2?#=�×-ѵ?#=@=~��E�;>G�9�Õ Ñµ?BA��]�

��SW[dLOTUPBPQKRlH[dTUpqSt��K YO]dp}ÂZK [\LeKMvxpqgigUp���TUSefØ[\LOKHpq]dKHrb����LeTUl�L�KHPQ[\NZsOgUTiPdLOKHPnN1fmNZYts�K|[���KHKRSlRpZrIYOc�[�N�[dTUpqSXN�g>P\NZr#YOgiK lRpqr#YOgUK�{�Ti[�hDN�SOzÍPdNZrIYegUK lRpZrIYOgiK|{�Ti[�hnvxpq]�N_lHg�NZPQP�pZvYXN�]Q[\TilRcOgUNZ]dg�hPdTirIYegUK�lHpqSOlRKHYe[\PH�ú�û:ü�ý´þaü�ÿ ³���ØÙ! Ç|©Pz^©�Ú°©*Ù�$|Ç ªZ®X°n¤�¥a®eÆ|©¹ªZ®X©«Ð ,(. ¯UÇ|®�=Z©(&�/U§B4\Ç�Æ�Ç040Ë�¬�®�=b¥a®²Ç0/21}ªZ°�§XÇ04|¦ ¨�/©«ªZ©­¬«¥a®eÆ ÇQÈq¬UÆ|©(3@©(&XÇ|®D©�&XÇ04\Ç�¬UÆ�ª_¤�¥a®²¤�Ç�§u©¤|¯�ªaÆ�Æ�ÉUÛ�1F&�¬«¤'&�&eªaÆ

±J²D³P´��­ÉUÛZÎ\õ�ø�×\  § ß � Ù�ÖZ×� 

^qÚ

Page 52: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

ªZ®BCÕ ��õ ¸2Ü)Û ÖZ×�  § ´Í³®´���ÉUÛZÎ\õ(ø�×�  § ß ��õ�Ö�×� 

1F&XÇ04\Ç�Çdª�¤'&6¤�¥a®²¤�Ç�§u©}¬�®�ÉUÛ<¬UÆ�ª#Ù�/SC�Ç�¤|¬UÆ|¬«¥a®n¯±¬UÆ|©F¥?ËZÇ04Nx?zeø�Ù<{}| ü»y¼��F¼Q½ Ý�Y[YX¿®Ya)HØ¿®Y[Y[R��²ÀXÁ�Ä]ÞßHÍ¿wàNR�ZJ K SOKRKHz PQpqrIK sONZPdTil�[\KR]QrITiSOpqgipqfZh�vx]dpZrÑ[\LeK�[\LOKHpq]QhDp�v�KH]d]Qpq]�¸¹lRpq]Q]dKRl|[\TiSOf�lRp�zOKRPHÎ(LeKR]dK ��KvxpqgigUp��W[dLOK�[\KH]dr#TUSOpqgipqfZhIpZv�� ³ TUYePdKR]yNZSOz ³ YOTiKRgir�NZS²��Ù�ÚqÚqî�Î ³ YOTiKRgUrINZS��²Ù�ÚqÚqîm ��}J K�P\N�h�[\LXNa[N`$R¬�®uª<4�°M¤�¥�C�ÇB¥ . $R¯U¥�¤'áb¯UÇ|®�=Z©�&^·DªZ®BC�4dªZ©�ÇDâ0¾�TiP�NnlHp�zeK�TUSM��LOTUl�LÍlRp�zOKH��pq]dzePwNZ]dKe·#sOTi[dPgUpZSOfO�q��LOKH]dK"â�¾[ã2·FYupqPdT�[\TipqSOP}NZ]dKpäQr#KRPQP\NZfZKysOT�[\P�å�[dLXN�[}l�NZSIs�KF£XgigUKHz#��T�[\LINZSmhwlRpqr�seTUSXN�[dTUpqSpZvczeê P�NZSOz_ÙqêëP�N�SOzI[dLOKy]QKRrINZTUSeTUSOf#� Ù�ô^â0¾  2·�YupqPdT�[\TUpZSOP2LXN�ÂqK�[\LOKHTU]2lRpZS�[dKRSm[\P�zOKH[dKR]dr#TUSeKRz#s�h[\LeK�r#KRPQP\NZfZK�sOT�[\PH�Jr�K|[yñ+¾�ù\x?zeø�Ù<{}æ�ç�è ¾ Ê x?zeøRÙ<{ ¾ suK�NwsOTiSXNZ]Qh�lRp�zOK�pZv�segUp�l�Ó<gUKHSOfZ[\L�·�N�SOz]\Na[\Kéâ0¾\ÎOvxpq]h��osx?z�ø�Ù<{ æ ç è ¾ øe[dLOK%êZ¸¹[\LBsOT�[�pZv·[\LOK]·�¸�sOT�[FPQ[d]dTiSOfwñw¾H�(�� TUPFzOKHSOpZ[\KHznsmh�ñ+¾H���� kë üJÍK�P\N�h_[\LXN�[�[\LeK�lRp�zOK�ñw¾LXNZP�¦ ¬�®X¬�¦ ¨�¦ì4\Ç|¯�ªZ©­¬(ËZÇ;CZ¬UÆ|©«ªZ®²¤�Ç%­'¾FTiv2N�S�h_YXN�TU]�pZv2zOTiPQ[\TiSOlH[

lRp�zOK|��pq]QzOP]x�ñw¾|���� �ø\ñw¾|�kÅe �{�LXNZPhq�NZr#rITiSOf�zOTUP�[�NZSOlHK�N�[�gUK�N�PQ[+­�¾Íã'· ü �´pZ]hí;J©�­'¾�Öqó�ø���K�PdN�h[\LON�[�NZS6NZgUfqpZ]dTi[dLOrïî+TiP�NZS�íc¾'/SC�Ç�¤�¥�CZ¬�®�=_ªZ¯>=�¥?4�¬�©(&�¦ . ¥?4yñw¾FTiv �´��LOKRS¬î+TiP�fqT6ÂqKHS�N�PQ[d]dTUSefð oÎx?zeø�Ù<{ ¾ ��LeTUl�L�LXNZP�q�N�rIr#TUSOfIzOTUP�[�NZSelRK#N�[�r#pqPQ[�í;Pã�·�vx]Qpqr+PQpqrIK�lRp�zOKH��pq]dzMñw¾H���� |ø[\LeK�N�gUfqpq]QTi[dLOrLîìpqc�[\YOce[dP+� ü¢FLOK�vxpqgUgip���TiSOf�TirIYupq]Q[\NZSm[[\LeKRpq]QKRr TiP���KRgig�Ó�SOp���SnTUSBlHp�zOTiSOf [\LOKHpq]QhuÎXP�[\]dpZSOfªÂZKR]QPdTUpZSOP

l�N�SbsuK�vxpqcOSOzBTUS²�OKZ�ëfO���·� ³ TiYOPdKH]yN�SOz ³ YOTUKHgUrINZS��uÙ�ÚqÚqîeÎ ³ YOTUKHgUrINZS���Ù�ÚqÚqîm ��ú�û:ü�ý´þaü�ÿ ³��E³òñ &XÇ04�Ç�ÇQÈq¬UÆ|©�Æ�ª�§X¥a¯±°�®²¥a¦ ¬�ªZ¯´©�¬�¦�Ç0/d¤�¥a®eÆ|©j4�¨X¤|©­¬k$R¯UÇ . ªZ¦ ¬�¯±°Dx�ñ+¾'{?ó¾jô ¸ ¥ . $R¬�®uª<4|°Ç04�4\¥?40/d¤�¥?4�4\Ç�¤|©�¬�®�=B¤�¥�C�Ç�Æ'3P1F&XÇ04\Ç�Çdª�¤'&wñ+¾�¬UÆ�ª . ¨�®²¤|©­¬«¥a® . 4\¥a¦Ex?z�ø�Ù<{ æ ç è ¾ ©�¥ªx?zeø�Ù<{ ¾ øõ12¬�©�&_©(&XÇ. ¥a¯�¯U¥?12¬�®�=�§B4\¥�§XÇ04�©�¬«Ç�Æ�ö

7 giTUr�¾j÷ ó â0¾Uøùz)3·gUTUré¾j÷ ó ­�¾UøùzDªZ®BC�giTUrn¾j÷ ó í;Jø�zeø7Îú ¥?4�Çdª�¤'&�·�ø�©(&XÇ04�Ç�¬UÆ�ªZ®ªí;�/SC�Ç�¤�¥�CZ¬�®�=wªZ¯>=�¥?4�¬�©�&�¦ . ¥?4�ñ+¾y1F&�¬«¤'&�4�¨�®eÆ�¬�®_©�¬�¦�Ç�§X¥a¯±°Z�(·�  üö�KHl�NZgigq[\LXNa[:TUS�[\LOK�Å���VDvx]\NZr#KH��pq]QÓu��N�gUK�N�]dSOTiSOf�NZgUfZpq]dT�[\LOrtPdcOlHlRKHKRzOP(Tiv�Ti[:l�N�S�lHpqSOPQ[d]dcOl|[

N�Lmh�Y�p�[\LOKHPdTUP(��LOTUl�L N�fq]dKHKRP���T�[\L�[dLOK�[\NZ]dfZKH[>lHpqSOlHKRYe[2pZS NZgUg�sOc�[2NyPQr�N�gUgmvx]\NZl|[\TipqS�pZvOY�pqTiSm[\PR���Sb[dLOK�lRpqSOP�[\]QcOlH[dTUpqSB��LOTUl�L_��K�cOPdK�[\p#YO]Qp�ÂZK�¢FLOKHpq]dKHr÷ºO�é^��XPdcOl�LDNwLmh�Y�pZ[dLOKRPQTUP���TUgUg�h�TiKRgizNØPQ[d]dTiSOf ð ��LOTUl�LâTUP�lRgUpZPdKn[dpoNØlRp�zOKH��pq]dztñw¾|����  ü Ü�hà[dLOK6Yupqgih�SOpZrITUNZgF[\TUr#KnzeKRlRp�zOTiSOfNZgifqpq]dT�[\LOrìpZv�¢FLOKRpq]QKRr ºO�ëºO��[\LOKINZsOTigUT�[�hb[dp_£XSezØNZS NZlRlHcO]\Na[\K#L�h�YupZ[\LeKRPdTiP�TiS Yupqgih�SOpZrITUNZg[\TirIK#��pZcOgUzØ[dL�ceP TUr#YOgihD[dLOK�NZsOTigUT�[�hD[\p6£XSOz`�1TUSÍY�pqg�h�SOpqr#T�NZg2[\TirIK��¬qyp���K¢ÂqKH]R�@��K<��TigUg

º�z

Page 53: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

PdLep��j[\LXNa[2[\LOTiP2TUP2TUr#YupqPdPQTUsOgiK���ceSOzOKR]@N�lH]Qh�Ye[\pZfq]\NZYeLOTUl�NZPQPdcOr#Ye[dTUpqS´ �T�v´vxKH�jK|{eNZr#YOgUKHP}LON}ÂZKsuKRKRS_PdKHKRS��

»y¼��F¼2G ûT�]RüHÍ¿�Ä]�-RõKyÀ«HTO Ã~ZýZÎþ;ÿ

Ü�K|vxpq]dKwfqT6Â�TiSOf_NBvxpZ]drINZg�zOKHPdlH]dTUY�[\TUpZS pZv�[\LOKwlHpqSOlRKHYe[�lRg�N�PdP�ÉUÛqø���KIr#KRSm[\TipqSD[\LXNa[�TiSD[\LOTiPlRpZSOlRKHYe[@lRgUNZPdP>[dLOKFPdKHlR]dK|[@ÓZK|h�vxpq]>KRNZl�LwlRpqSOlHKRYe[@TiP>lRpZrIYupqPdKHz pZvur�N�S�h�PdrINZgUg�PQcOsOÓZK|h�PR�eKRNZl�LpZv���LOTil�L�TUP�KRSOlH]Qh�Ye[\KHzbPQKRYXN�]\N�[dKRgihZ�2¢FLOK�]QK�NZPQpqS�TUP}[\LXN�[�K�NZl�LBcOPQKHvxcOg�K|{eNZr#YOgUK���TUgigulHpqSm[�NZTiSN�PdrINZgUgXN�rIpqceS�[}pZv�cOSOKHSOlR]�h�Ye[\KHz�TUSevxpq]Qr�Na[\TUpZS NZsupqce[}K|{eNZl|[\gihwpZSOK�pZv²[dLOKyPQcOsOÓ�KHh�PR�JqyKHSOlRK��cOSOgiKRPQPr�N�S�hwcePdKHvxceg²K|{eNZr#YOgiKRP�LON}ÂZK�suKRKRS�PQKRKHS��e[\LeKR]dK���TUgiguK�{�TUPQ[�PdcOseÓZKHh�P�NZsupqce[���LOTUl�L�SOpcOSOKHSOlR]�h�Ye[\KHz6TUSevxpZ]drIN�[\TipqS�LXN�P�suKRKHSn]QK¢ÂqKRNZgUKHz��Ü�KHvxpZ]dK���KlRNZS�zOKHPdlR]QTUsuK�[dLOK�lHpqSOlRKHYe[2lHg�NZPQP�ÉUÛ���Kr�cOPQ[·£X]dP�[>PQY�KHlRTiv�h�PQpqrIK�YXNZ]dNZrIK|[\KH]dPR�

r�KH[nx�ñw¾'{?ó¾kô ¸ suK�N#£�{eKHznv�NZr#TUg�h<pZv2KH]d]Qpq]�¸¹lRpq]Q]dKRl|[\TiSOf�lHp�zOKHPy��T�[\Lb[dLOK�Ye]dpqYuKR]�[\TUKHP�P�[�N�[dKRz6TiS¢FLOKHpq]dKHr ºO�ëºo�xPdpn[dLOK<]\Na[\K�TiPnâ�¾�ø:[\LOK<r#TUSeTUr�cOrÑ]dKHg�N�[dT�ÂqKIzOTUP�[�NZSelRK�TiP;­�¾�ø(NZSOzÍ[dLOKR]QK<TUP�NYupqgih²�(·� ¹¸¹[\TirIKJí;¹¸�zeKRlRp�zOTiSOf�NZgUfZpq]dT�[\LOr'vxpq]�ñw¾Q ��@í�T�ÂqKHSwN�Y�pqPQTi[dT�ÂqK�TUSm[\KHfqKR]�ºDøZgUKH[�� § º������� øgUK|[U·�suK�[\LOK�PQr�N�gUgUKHPQ[FTiSm[\KRfZKR]PQcOl�Lb[dLXN�[wâ0¾õã0·N*�ºDø´NZSOzBgiKH[Fõ § º����}· ü ¢FLOK�vxpqgUgip���TUSOfv�NZl|[\P�l�NZS_suK�K�NZPQTUgihpÂqKH]dTi£OKRz�ù� [�<�˳��� í; § dw��Ù� �ø�â0¾ § dw��Ù� �� . ¥a¯�¯U¥?1�Æ . 4\¥a¦ ñ &XÇ�¥?4\Ç|¦��uÐ����ZÐ� [�<�˳���� · § dw�kº_ �� . ¥a¯�¯U¥?1�Æ . 4\¥a¦ìC�Ç��@®X¬�©­¬«¥a®M¥ . ·�ªZ®BC ú ª�¤|©��uÐ����ZÐ� [�<�˳���� õ § dw��º��q �� . ¥a¯�¯U¥?1�Æ . 4\¥a¦ßC�Ç��@®X¬�©­¬«¥a®�¥ . õʪZ®BC ú ª�¤|©��uÐ����ZÐ� [�<�˳��� º § dw��õ Û  |ø!� § dw�xõ ¸2Ü)Û  �� . ¥a¯�¯U¥?1�Æ . 4\¥a¦ìC�Ç��@®X¬�©�¬«¥a® ¥ . �bªZ®BC ú ª�¤|©��uÐ#"$�ZÐ

r�KH[�È suK NI£�{eKHz�gUKRSefZ[\L�¸¹YO]dKHPdKH]�Â�TUSOf�pZSOK|¸«�N�hbY�KH]dr�ce[\N�[\TipqS���¢FLeK�PdK|[#�Vx?zeøRÙ<{�%� '&y��TigUgsuK�pqcO]PQKH[�pZv·PQKRlR]QKH[�ÓZK|h�PRÎXK�N�l�LbPQKRlR]QKH[�Ó�KHh�( § ��( ¸ ø ügüSü ø)(*&d hoo�Vx?zeøRÙ<{�%� '&TiPlRpZrIYupqPdKHz�pZv��PQcOsOÓ�KHh�PyKRNZl�L_pZv���LOTUl�LBTiPhºÑseTi[\P�gUpqSOfe��¢FLOK�lRgUNZPdPÉUÛ�LXNZP�N�lHpqSOlRKHYe[%m�+�vxpq]�K�NZl�L_PdKHlR]dK|[ÓZK|h,( üJÍK�SOp��µzeKRPdlH]dTis�K�NblRpqSOlHKRYe[ªm�+#p}ÂqKR]px?zeø�Ù<{ | ü �¹v"m�+#TUP�[\LOK�[\NZ]dfZKH[�lHpqSOlHKRYe[R�>[\LOKHSoNZS

K|{eNZr#YOgiKw��osx?zeø�Ù�{ | TUP2PdNZTUzw[\p�suK�¨mÆ�Ç . ¨�¯ZTivB�c¸Íã}ã}ã�� %-& § È>��( ¸  )Ä®ã}ã}ã�ÄFÈ>��( &  2NZSOz#TUP�¨mÆ�Ç|¯UÇ�Æ�ÆpZ[dLOKR]���TUPdK��Wí�T6ÂqKRSàNZSàK�{eNZrIYegUK^�boEx?zeøRÙ<{ | ø�giKH[ªÃ��(�� �oEx�Ùqø üSügü ø)�){�ø2êu���� µo_x�Ùqø üSüSü ø�·?{suK PdcOl�L�[\LXNa[]� %.&#/10 À 0 ��2 Ü[¸32 ¾ / ë 0 ��2 TiP�[\LeK�£X]dP�[�sOTi[�p�v®� %-&4/ ¸Íã}ã}ã�� %-&4/5& ¾���LOpZPdKAÂ�NZgUcOK�TUP�Ùq���¹v� %-&4/ ¸ § ã}ã}ã § � %-&4/5& ¾ § z�[dLOKRS���K�P\N�h#[\LXNa[hÃ��(��  § êu����  § z ü ��TUfZcO]dK�ºO��Ù�TUgigUcOP�[\]\Na[\KRP@[\LeKPQ[d]dcOl|[\cO]QK�p�v>N cOPQKHvxcOg:K|{eNZr#YOgiKZ�2¢FLOK�lRpZSOlRKHYe[]m�+�TiP�zOKH£XSeKRz6N�PvxpqgUgip���PHù

ºOÙ

Page 54: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

È>��( ¸  ��� ������ ������ ������ ������ ������ ������ ���È>��(6&d 7 849 :ºÑsOTi[dP

;<<<<<<<<<<<<<<<= <<<<<<<<<<<<<<<>�

�S �

lRpqgicOrISêu���� 

zªzªz¹ã}ã}ã ã}ã}ãwzªzªzzªzªz¹ã}ã}ã ã}ã}ãwzªzªz��� ������ ���zªzªz¹ã}ã}ã zpzez � Ù�z�Ù ã}ã}ã�ÙwÙ�z]dp��«Ã�����  ��� ������ ������ ���ÙnzBÙ ã}ã}ã ã}ã}ãwzBÙ#Ù7 8?9 :·�seTi[\P

@ <<<<<<<<<<<<<<<A<<<<<<<<<<<<<<<B�

�$C~���TifqcO]dK�ºO�iÙZù2�.cePdKHvxceg·K�{eNZrIYegUKª�k�:ø\ñ+¾H��( À 0 ��2  jë 0 ��2k� üÅ�N�]Q[ ��B� zOKRYeTUlH[dP�[\LeK+º���¸¹sOTi[(YO]dK|£e{ p�v[�:ÎZPdTiSOlRKJ��TiP�cOPQKHvxcOg�[\LOTiP>r�cOP�[2s�K+È>��( ¸  �Ä�ã}ã}ãkÄÍÈ>��(6&d  üÅ�N�]Q[ �'CP� zOKRYOTilH[dPy[\LeKD�}·|¸¹sOTi[�PQcek {�� %-&4/ ¸ üSüSü � | ø´��LOKR]QK�[\LOK�sOTi[%� %-&4/10 æ Ü[¸32 ¾ /5E TiPyTiSn]Qp�� âNZSOzblRpqgicOr#Sµm�vxpZ]�ÙGFÎâ�FH����ÙDF�mIFù· ü ��P�PdLOp���SbTUS �$CP� �O[\LOKéÂ�NZgUcOKHP�pZv�Ã\���� �NZSOz�êu�(�� NZ]QK�zeKH[\KH]dr#TUSOKHzBs�h�[\LOK�gUp�l�Na[\TUpZS�pZv([dLOK�£X]dP�[�Ù�TiS�[\LOKI�}·|¸¹sOTi[Pdcek {_pZvõ� ü

7 m�+Z����  § zwTiv��6TiPFcOPdKHgUKHPdPR�7 m�+Z����  § ñ+¾H��( À 0 ��2  jë 0 ��2�øq[\LOK®êu���� ¹¸¹[dL#seTi[@pZv�ñw¾|��( À 0 ��2  �ømTiv��<TiP2cOPdK|vxcOguNZSOzªÃ��(�� |ø2êu���� h*ËÙ ü�¹võ�6TiPFcOPdK|vxcOg·NZSOz�Ã�����  § êu�(��  § z�[\LOKHS�m�+�����  § z ü

»y¼��F¼�� J°YX¿�¿LKB¿MK�ûT�]R:¿®Y[RÆI »y¼����Ti]dPQ[���K<KRP�[�NZsOgiTUPQL [\LON�[Am�+wTUP�N ÙH¸¹zOKRlHTUPdTipqSÍgiTUP�[��_�Xpq]�KRNZl�LÊÙNF fOFEº���ø·gUKH[é· i zeKRSOpZ[dK[\LeK�gUTi[dKR]dNZg � i Tiv:[\LOK�fm¸¹[\L�seTi[FpZvFÈ>��( ¸  ÍÄ"ã}ã}ã¢Ä+È>��(6&\ �TUP�ÙqøXNZSOz�giKH[U· i zeKRSOpZ[dKN� i pZ[\LeKR]Q��TiPdK��¢FLOKHSb[\LeK�vxpZgUgUp���TiSOf�TUPFPQKRKHSb[\pwsuK�NBÙ|¸�zOKHlRTUPQTUpqS�giTUP�[��LOTUl�L_lRpZrIYOc�[\KRPym�+�ù

� ·?¸|ø�zm |ø ügüSü ø��(· %.& ø'zm �ø���� %-&4/ ¸�ø\ñ+¾H��( ¸  '¸Q �ø���� %-&4/ ¼�ø\ñ+¾H��( ¸  V¼� |ø üSüSü ø�(� %.&4/10 À Ü[¸32 ¾ / ë ødñw¾H��( À  jë| �ø üSüSü ø���� %-&4/5& ¾�ø\ñw¾|��( &   ¾Q  ü

¢FLOTiP�TUP�suKRl�N�cOPdK�[dLOK�£O]dPQ["º��IYONZTU]QPyKHSOPdcO]QK�[\LXNa[�NZgig:cOPdKHgUKRPQP�K|{eNZr#YOgiKRP���TUgUg:suK�gUNZs�KHgUKHz�zeøNZSOzw[\LeKypq]QzOKR]QTUSOf�pZv�[\LOK�g�NZP�[P��·FYXNZTU]QP@KRSOPQcO]dKHP�[\LON�[@[\LOK�g�NZsuKRgXp�v�K�N�l�L�cOPQKHvxcOg´K|{eNZr#YOgiK���TigUgsuK�N�P�zeKRPdlH]dTis�KHzbTiS ³ KRlH[dTUpqSBºO� ^e�éó��

º�ó

Page 55: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

¢(p�YO]dp}ÂqK�[\LOK�TUSevxpq]Qr�Na[\TUpZS�¸¹[dLOKRpq]QKH[dTUl>PdNZrIYegUK@lRpqr#YOgiK|{�Ti[�h�cOYeY�KH]�supqcOSOz�����K�r�cOPQ[�PdLOp��[\LON�[�ceSOzOKR]>NZSmh�zeTUPQ[d]dTisOce[\TipqS�N�[�r#pqP�[ ß ��Ù�ÖZ×\ :K�{ON�rIYOgiKRP(NZ]dK�]dKHÌ�ceTU]dKHz�� ³ TUSOlHK�K�N�l�L Y�pqPQTi[dT�ÂqKK|{eNZr#YOgiK�lRpqSm[�N�TUSOPyÈ>��( ¸  -Ä üSügü Ä%È>��( &  �øXNwlRpqr#YOce[\N�[\TipqSXNZgigihwcOS�supqcOSOzOKHzngiK�NZ]QSOKR]�l�N�SbgiK�NZ]QS[\LeK#[\NZ]dfqK|[�lRpqSOlHKRYe[�K|{eNZl|[\g�hDvx]dpqr NBPdTiSOfqgiK Y�pqPQTi[dT�ÂqK K�{eNZrIYegUKwsmh�TUS�ÂqKH]Q[\TiSOfB[\LOK#pqSOK�¸¹��N�hYuKR]dr�ce[�N�[dTUpqS°ÈØ[\pB£XSOz K�NZl�LQ( À NZSOzM[\LOKHSÍlHpqrIYece[\TiSOfBK�NZl�L ñw¾H��( À   ü ³ cel�LØNbgiK�NZ]QSOKR]�lRNZS[\L�cOP�r�NZÓ�KZ�2PdN�hZ�}ó<zmÖZ×�l�NZgigUP�[\pn[dLOK<pq]dNZlRgiK��é�1�jm�ø�� |  �NZSOzØpqc�[\YOce[�[\LOKITUzOKHS�[dTUlRNZgUg�h�áRKR]QpLmh�Y�pZ[dLOKRPQTUP�T�v�NZgig}K�{eNZrIYegUKRP�NZ]dKISOKRfqN�[\T6ÂqKZ�2p�[\LOKH]Q��TUPQK�pqc�[\YOce[�[\LOK�lRpq]Q]dKRl|[ Lmh�Y�pZ[dLOKRPQTUP N�PzOKHPdlR]QTUsuKRzDN�s�p}ÂqK����èPdTUr#YOgiK�lRNZgUlHcOg�Na[\TUpZS_PQLOp���P�[\LON�[�[dLOTUPyNZgUfZpq]dT�[\LOrå£XSOzOP�N�S6×�¸�NZlRlHcO]\N�[dKLmh�Y�pZ[dLOKRPQTUP���Ti[dL_LeTUfqLBYO]QpqsXNZsOTigUT�[�hq��NZSOz_LOKRSOlHK%±J²D³P´��­ÉUÛZÎ\õ�ø\×�  § ß � Ù�ÖZ×�  ü�¹[�]dKHr�NZTiSOP�[dp�s�pqceSOzD[\LOK lHpqrIYece[�N�[dTUpqSONZg�PdNZrIYegUK�lHpqr#YOgUK�{eT�[�h6pZv�ÉUÛZβ��K suKRfZTUS6��Ti[dL

[\LeK�PdTUr#YOgiKR]}cOYOYuKR]FsupqcOSOz��>J K�P\N�hI[\LXN�[FN�ÙH¸�zeKRlRTiPdTipqS<gUTiPQ[�p�ÂZKR]]x?zeøRÙ<{}|�TiP]1�Ç|¯�¯ /«Æ|©k4|¨X¤|©�¨4\Ç�CTiv>Ti[\PygUKHSOfZ[\LnTUP�K�{eNZlH[dgihbõØNZSOzDT�[�LONZP�[\LeK�vxpqgigUp���TUSOf#PQ[d]dcOl|[\cO]QKZù�vxpq]�ÙRFTSUF º��#[dLOKDS¹¸¹[dLYXNZTi]2pZv²[dLOK�zOKRlHTUPdTipqSwgUTiPQ[@LXNZPP�WV�pZ] �XV²NZP}Ti[\P@lHpqS?ÁQcOSOl|[\TipqS�N�SOzILONZPUz�NZP@Ti[dP@pqce[\Yece[}sOTi[R��N�SOzvxpq]+º��YâÙGFZS[Fκ��Y\�}·�[dLOK]S¹¸¹[\LB[\KH]dr p�v([dLOK�zOKRlHTUPdTipqSBgUTiPQ[FLXN�Pw�WV>NZP�Ti[dPFlRpqS?ÁQceSOlH[dTUpqS��í�T�ÂqKHS`NnPdNZrIYegUK�^'pZv�K�{eNZrIYegUKRP���LOTUl�LoN�]dK�gUNZsuKRgUKHzØNZlHlRpq]QzOTUSefn[\pn[dLOK�lHpqSOlHKRYe[ªm�+aø(T�[�TiPK�N�PQhDvxpq]�NBYupqgih�SOpqr#T�N�g·[dTUr#K NZgUfZpq]dT�[\LOr [\p�£XSezÍN<��KRgUg ¸�P�[\]dcelH[\ce]dKRz1ÙH¸¹zOKRlHTUPdTipqSDgUTiPQ[���LeTUl�LTUP�lRpZSOPdTiPQ[\KHSm[���Ti[dL_^ ü �ySmhnYupqPdT�[\T6ÂqK�K|{eNZr#YOgUK�pZvL^¡TUzOKHSm[\Ti£OKRPy[dLOK�£O]dPQ[]º��#giTi[dKR]\N�gUP�pZv2[\LeK��KRgUg ¸�P�[\]dcelH[\ce]dKRzàÙH¸�zOKHlRTiPdTUpZS giTUPQ[R��N�SOzÍKRNZl�L`cOPdK|vxcOg}K|{eNZr#YOgUK#YO]dp}Â�TUzeKRP�[dLOK�pZce[\YOc�[�sOT�[�vxpZ]pqSOK�pZvX[\LOKFgUNZPQ[.��·�YXN�TU]dP��SOp�[\K�[dLXN�[2T�[>TUP(Y�pZPdPdTisOgUK}[\p�TizOKRSm[\T�v�h�cOPdK|vxcOgOK|{eNZr#YOgiKRP2NZP�gUpqSOf�NZPP^lRpZS�[\NZTUSeP�N�[FgUKRNZPQ[pqSOK�YupqPdT�[\T�ÂZK�K|{eNZr#YOgiK� |� ³ TUSOlHK�[\LOKH]dK�NZ]QK�ó | ��KRgig�¸¹PQ[\]QcOlH[dcO]dKHz�ÙH¸�zOKHlRTiPdTUpZSgUTiPQ[dPR��¶�lRlRNZrbêëP�öyNZáHpq]��­Ü�gUcOr#KR]�Ç|©>ªZ¯�ÐU�uÙ�ÚqÛ�uq �TirIr#KRzOTUN�[\KHgih�TUr#YOgUTiKRP·[\LXN�[ ß ��õ�Ö�×� >K|{eNZr#YOgUKHPPdc�kIlHK�vxpZ][\LOTiPFY�pqg�h�SOpqr#T�NZg´[\TirIK�giK�NZ]QSOTUSOfwN�gUfqpq]QTi[dLOrb��yp��1��KPQLOp��j[dLOKgip���KR]�s�pqceSOz pqSn´Í³®´���ÉUÛqÎ\õ(ø�×�  ü ¢FLOKFTizOK�N�p�vX[\LOKYe]dp�pZv´TUP�NZP�vxpqgigUp���PRù

��K���TUgig�K�{�LOTUsOT�[2N�YONZ]Q[dTUlRceg�NZ]:zOTiPQ[\]QTUsOc�[\TUpZS�pqS�x?z�ø�Ù<{}|yNZSOz�PQLOp��j[dLXN�[>NZSmh�Yupqgih�SOpZrITUNZgZ[\TirIKgUKRNZ]dSeTUSOf�NZgUfqpZ]dTi[dLOr0vxpq]yÉUÛ���LOTil�L6gUKRNZ]dSePy[\p<NZlRlHcO]\NZl|hD×�cePdTUSef`�<í;QÖ�Ù�Ûq×�K�{ON�rIYOgiKRPyzO]\N���Svx]dpZrè[\LeTUP@zOTUP�[\]QTUsOce[dTUpqS�l�NZS#s�K�cOPQKRz#[\p�TUS�ÂqKH]Q[@[\LeK�pZSOK|¸«�N�hwYuKR]dr�ce[�N�[dTUpqSªÈ�TUSwYupqgih�SOpZrITUNZg[\TirIK<��T�[\L1SOpqSOSeKRfqgiTUfqTisOgUK<PQcOlRlHKRPQP<YO]dpZsXNZsOTigUTi[�hZ�j¢FLeTUPwlRpqSm[d]\NZzOTilH[dTUpqS1TUr#YOgUTiKRP�[dLXN�[#K¢ÂZKR]�hYupqgih�SOpqr#T�N�g�[\TirIK�giK�NZ]QSOTUSef N�gUfqpq]QTi[dLOr�r�cePQ[IcOPdK_rIpZ]dK�[\LXN�S���íc¾�Ö�Ù�Ûq×wK|{eNZr#YOgUKHPR� ³ TUSOlHK�´NZlH[dP�ºO�éðØNZSOzâºO� ÛÍ[\pqfqK|[\LOKH]�TUr#YOgiho[dLXN�[���íc¾�Ö�Ù�Ûq× § dw��õ ¸2Ü)Û ÖZ×� |ø}[\LOTiP<��TUgigyYO]Qp}ÂqKD[\LXNa[´c³P´���ÉUÛqÎ\õ(ø�×�  § Õ �xõ ¸2Ü)Û ÖZ×\ �N�P�zeKRPdTi]dKHz��

r�KH[w� | suK�[\LOK�zOTiPQ[d]dTUsece[\TipqS�pqSÒx?zeø�Ù<{}|���LOTUl�LbNZPdPQTUfqSeP���KRTifqLm[y^q×�ÖO�kíc¾Æã��}·� �[\p#K�NZl�L_pZv

ºm^

Page 56: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

[\LeKa�}·�cePdKHvxceg·K�{eNZrIYegUKRPx�È>��( ¸  �Ä]ã}ã}ã'È>��( &  -Ä%z i Ù}z & ¾2Ü i Ü[¸ { & ¾2Ü[¸i ô ò ü

NZSOz�N�PdPdTifqSOP�[dLOK�]QKRrINZTUSOTiSOfnÙyô1^q×�Ö<í;���KRTUfZL�[�[\p<[\LeK�PdTiSOfqgUK�cOPdKHgUKHPdP�K|{eNZr#YOgiK;z�| ü ö�KRlRNZgUgvx]dpZr ³ KHlH[\TipqSBºO� ^e�iÙ�[dLXN�[%í;�TUP�[dLOK�vx]dKHÌ�ceKRSOl|hBpZv·KR]d]Qpq]dP�cOY�[dpw��LeTUl�L�[\LOK�KH]d]dpZ]�¸�lHpq]d]QKRl|[\TUSeflRp�zOKHP�p�v�ºO�ëºBl�NZSMsuK#PQcOlRlHKRPQPQvxcOgigih�zOKHlRp�zOKRzMcOPdTiSOf_N_Yupqg�h�� ·� �¸¹[dTUr#K�N�gUfqpq]QTi[dLOrb���ypZ[dK�[\LXNa[cOSOzeKR]�[\LOTiPyzOTiPQ[d]dTUsece[\TipqS��OKRNZl�LnsOT�[�pZv>KRNZl�Lnñ+¾H��( À  �TiP�KHÌ�cXNZgUg�h�gUTiÓZKHgih�[dpIp�lRlHcO]yNZP�[dLOK�g�NZsuKRgpZv�N cOPdK|vxcOg:K|{eNZr#YOgUK��

r�KH[G^Ês�K N<PdNZr#YOgUK�p�vP�<í;QÖ�Ù�Ûq×�K|{eNZr#YOgUKHP���LOTil�L N�]dK�zO]dN���S�vx]Qpqr¶���1�kmaø�� |   ü ³ TUSOlHK[\LeK�K|{�YuKRlH[dKRz_S�cOr�suKR]FpZv:cOPQKHvxcOg�K�{eNZrIYegUKRP�TUS,^`TUPM�qÖZî�øON�PdTirIYegUKyNZYOYOgiTUl�Na[\TUpZS�pZv(V�LOKH]dSOp��supqcOSOzOP�PQLOp���P�[dLXN�[���T�[\L�p}ÂqKR]���LOKRgirITiSOfqg�hnLOTifqL�YO]dpZsXNZsOTigUTi[�hB[dLOK P\NZr#YOgiK�^â��TUgig(lHpqSm[�NZTiSN�[@giK�NZP�[�pZSOK�cOPdK|vxcOgXK|{eNZr#YOgUK�� ³ TiSOlRK�K�N�l�LIcOPQKHvxcOg´K�{ON�rIYOgiKFlRpqSm[\NZTUSOPJÈ>��( ¸  �ÄPã}ã}ã�È>��(*&d 2NZP2Ti[dPº���¸�sOT�[@YO]dK|£e{���T�[2vxpqgUgip���P>[\LXN�[2��T�[\Lwp}ÂqKH]Q��LOKHgUr#TUSOfqg�h�LOTifqLwYO]dpZsXNZsOTigUTi[�h�N�Y�pZgih�SOpqr#T�NZg�[\TirIKgUKRNZ]dSeTUSOfwNZgifqpq]QTi[\LerÞ��LeTUl�L_LXNZP�NZlRlHKRPQPy[\pb^ol�NZS_TUzOKHSm[\Tiv�hI[\LOK�P�[\]dTiSOfqP%È>��( ¸  |ø ügüSü ø�È���(*&\  ü�yp��çPQcOYOYupqPdKo[\LXNa[MNâYupqg�h�SepqrITUNZg�[dTUr#KØgUKRNZ]dSeTUSOfâNZgifqpq]QTi[\Ler lHpqcOgiz*NZl�LeTUK¢ÂZKàN�Sè× ¸

NZlHlRcO]dN�[\KLmh�YupZ[\LOKHPdTiP>vx]dpqr [\LOK�P\NZr#YOgUK[^ ü ³ TiSOlRK}[\LOK�gUK�N�]dSOTiSOf�NZgifqpq]dT�[\LOr�Ó�Sep���P®È���( ¸  |ø üSüSü øÈ>��(6&d |�u[\LOK�NZgUfZpq]dT�[\LOr lRNZSDNZYeYOgih_Ti[dP�× ¸ NZlHlRcO]dN�[\K�Lmh�YupZ[\LOKHPdTiP�[\pIK�NZl�LDpZv2[dLOKD�}·�cOPdK|vxcOg�K�{�¸NZr#YOgUKHPzOKHPdlR]QTUsuKRz_NZsup}ÂqKZ�@¢FLOK�N�gUfqpq]QTi[dLOrãl�NZS�[dL�ceP�lHpqSOP�[\]dcelH[dc ¸ ø ügüSü ø�c & TUS�Yupqgih�SOpZrITUNZg[\TirIK��·��LeKR]dK�KRNZl�Lec À TUP�NZS°·�¸�sOT�[�PQ[\]QTUSOfb��LOTUl�LØTiP�[dLOK�giK�NZ]QSOTUSOfbNZgifqpq]dT�[\LOr_êëP�ä�fqcOKRPQP�åDNa[[\LeK�P�[\]QTUSOf�ñw¾|��( À   ü ³ TiSOlRK�smhwNZPQPdcOr#Ye[\TipqS [\LOK�Lmh�Y�p�[\LOKHPdTUP}TUP@× ¸ NZlHlRcO]dN�[\K�cOSOzeKR]~� | ømN�[}r#pqPQ[NZS°íc¾�ÖZ^�vx]\NZl|[\TUpZSMp�v�[\LeK��}· [dpZ[�NZg2seTi[\P�TiS�[dLOK#PQ[\]QTUSOfZPGc ¸ ø üSüSü ø�cD&�l�N�SÍsuK#TiSOlRpZ]d]dKHlH[R��Ü�hfnNZ]QÓZp}Âuê PITUSOKHÌ�cXNZgUT�[�hq��N�[#gUKRNZPQ[�óqÖZ^�pZvy[\LOKgc À êëP#r�cOPQ[#KRNZl�L¡LXN�ÂqKbN�[#r#pqPQ[pí;wã�·BTUSOlHpq]�¸]dKHlH[�sOT�[\PRÎXlHpqSOPQKRÌ�cOKRSm[dgihq�usmh�cOPQTUSOfw[\LOK�Yupqgih�SOpZrITUNZgu[dTUr#K�zOKRlHp�zeTUSOf#NZgifqpq]dT�[\LOrãvxpq]�ñ+¾�øO[\LeKgUKRNZ]dSeTUSOf�NZgifqpq]QTi[\Ler lRNZSj£XSezàN�[#giK�NZP�[<óZÖZ^DpZv�[dLOKBPdcOsOÓ�KHh�P�xh( ¸ ø ügüSü ø)( & {6TiSjYupqgih�SOpZrITUNZg[\TirIK��;q�p���K¢ÂZKR]H��PdTiSOlRK#NZP�SepZ[\KHz KRNZ]dgiTUKR]y[\LOK K�{eYuKRl|[\KHzØS�cOr�suKR]�pZv�cOPdK|vxcOg2K|{eNZr#YOgiKRP�TUSi^TUP]�ZÖZîeø�smh�NBPQ[d]\NZTifqLm[dvxpq]��N�]dzMNZYOYOgiTUlRN�[\TipqSDpZvV�LOKR]QSOpZ�1s�pZcOSOzOP�Ti[�TiP�K|{�[\]QKRr#KRgih6cOSOgUTiÓZKHgih[\LON�[j^.lRpZS�[\NZTUSeKRz1rIpq]QK�[\LXNZS\�ZÖZ^DcOPdK|vxcOg�K|{eNZr#YOgUKHPR�1��P#NM]QKRPQcOgi[R�}��KbLON}ÂZKb[dLXN�[ ��Ti[dLÂqKH]QhbLOTUfZLbYO]QpqsXNZseTUgUT�[�h�[dLOK�Yupqg�h�SepqrITUNZg²[dTUr#K�gUK�N�]dSOKH]yLXN�Py]QKRlRKHT�ÂZKRzDSOpITUSevxpq]Qr�Na[\TUpZS�N�[�NZgUg��p�[\LOKH]�[dLXNZSsÈ>��( À  Q �vxpq]�N�[�giK�NZP�[�óqÖZ^BpZv�[dLOKIPQcOsOÓ�KHh�PR�B�¹[�vxpqgUgip���P�[dLXN�[�[\LeKIYupqgih²�xõ� �¸«[\TirIKgUKRNZ]dSeKR]I��NZP�N�sOgUK_[\pØTiS)ÂZKR]�[�ÈWpqSÊNa[<gUKRNZPQ[nÙ�ÖZ^Íp�v�[\LOK¬È���( À  �ê P`ä�vx]dpqr PQlR]\Na[\l�L��>å ³ TUSOlHKK�N�l�LWPQcOsOÓZK|hk( À TUP�º § dw��õ Û  �sOT�[\P<gipqSOfO��[\LepqcOfqL���pqcO]�Yupqg�h��xõ� ¹¸¹[\TirIK_gUKRNZ]dSeKR]�TUP<NZgUPQpN�Y�pqg�h²�kº_ �¸«[\TirIK�NZgUfqpZ]dTi[dLOr_Î�sOc�[I[dLOTUP lHpqSm[\]\N�zOTUl|[\Pw[\LOK�v�NZl|[#[dLXN�[�È¡TUPwpqSOK�¸¹��N�hq�ùq�KRSOlHK´c³P´���ÉUÛqÎ\õ(ø�×� hø���íc¾�Ö�Ù�Ûq× § Õ �xõ ¸2Ü)Û Ö�×�  ü ��¢FLOKHpq]dKHr ºO�é^q 

ºqº

Page 57: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

465�4 Ð l<;>=2?I9�Õe��=nm �Do�yp��t��K�rINZÓZK�rIpq]QK�K|{�[dKRSOPQT�ÂqK�cOPQK�pZv·lR]Qh�Ye[dpqfq]dNZYOLOTil�r�N�l�LOTUSOKH]QhI[\p YO]Qp�ÂZK�[\LOK�vxpqgigUp���TUSefOùú�û:ü�ý´þaü�ÿ ³����Ù! Ç|©®fÒ*ãÙ�$|ÇwªZ®X°�¬�®X©�Ç =�Ç04HÐ ,(. ¯UÇ|®�=Z©(&�/U§B4\Ç�Æ�Ç040Ë�¬�®�=b¥a®²Ç0/21}ªZ°�§XÇ04�¦ ¨�©¹ªZ©�¬«¥a®eÆÇQÈq¬UÆ|©(3}©�&XÇ|®6©(&XÇ04\Ç�¬UÆ�ªB¤�¥a®²¤�Ç�§u©F¤|¯�ªaÆ�Æ�É i 1F&�¬«¤'&�&eªaÆ

±J²D³P´��­É i Î\õ�ø�×\  § ß � Ù�ÖZ×� ªZ®BC

´c³P´���É i Î\õ(ø�×�  § dw��õ i ÖZ×\ 1F&XÇ04\Ç�Çdª�¤'&6¤�¥a®²¤�Ç�§u©}¬�®�É i ¬UÆ�ªAf /SC�Ç�¤|¬UÆ|¬«¥a®n¯±¬UÆ|©F¥?ËZÇ04Nx?zeø�Ù<{}| ü¢FLOTUPyPQ[d]dKRSefZ[\LOKHSOPy[dLOK�]QKRPQcOgi[ypZv2~�KHl�N�[dcO] Ç|©¹Ð�ªZ¯�Ð�pqSnzeTUPQ[d]dTisOce[\TipqS�¸«vx]dKRK�lHpqr#YOce[�Na[\TUpZSXNZg

ÂqKH]dPQcOP�TiSevxpq]Qr�N�[dTUpqS�¸¹[\LeKRpq]QKH[\Til�P\NZr#YOgUK�lRpqr#YOgUK�{�Ti[�h6TiS6[���p���N�h�PHù���KwTUr#YO]Qp�ÂZK�[\LOK ceYOY�KH]supqcOSOzØpZSØTUSevxpZ]drIN�[\TipqS�¸¹[dLOKRpZ]dKH[dTUl PdNZrIYegUK#lRpqr#YOgUK�{�Ti[�hDvx]dpZr ß ��õ�ÖZ×� �[dp ß ��Ù�ÖZ×\ |ø(NZSOzÍ��KYO]Qp�ÂZK<[\LOTiP�PQ[d]dpqSefqKR] fmN�YÍvxpq] N6lHg�NZPQP�lRpqSePdTUP�[\TiSOf6pZv�r�cOl�LoPQTUr#YOgUKH]�lHpqSOlRKHYe[\Pb�jf�¸�zOKHlRTiPdTUpZSgUTiPQ[dPF]\N�[dLOKR][dLXNZS_Y�pqg�h�SOpqr#T�NZguPdTiáRK�lRTi]dlRceTi[\P\ |�

»y¼�»y¼Q½ HTY\XyKyÀe¿JÞ®YXÃPKN��ÁV�pJ°Y[RõO�Á�IåÁQÄ�Ã~Y´ÁVR�Z¢FLOK6lH]Qh�Ye[dpqfq]\N�YOLOTUlnzOKH£XSeTi[\TipqSOP#��K6Ye]dKRPQKRSm[_TUS1[\LOTiP<PdKHlH[\TipqSÊNZ]QK6PdgiTUfqLm[dgihor#pq]dKbfqKRSOKH]\NZg[\LONZSD[\LOK�P�[�NZSezXNZ]dz�zOK|£XSOT�[\TUpZSOP���LOTil�LMlRNZS�s�K�vxpZcOSOzDTUS²��KZ�ëfO���}�«í�pqgUze��NZPQPdKR]�N�SOz�Ü�KRgig�NZ]QKZ�Ù�ÚqÚZîm |Î���Kn��TigUg�SOKHKRz¡[\LeTUP�K�{�[\]\NØfZKRSOKH]\NZgiTi[�hoTUS ³ KRl|[\TUpZS¡ºO�éð���¢FLO]QpqcOfqLepqce[<[dLOTUPIPdKRl|[\TipqS[\LeK<vxcOSOlH[dTUpqSq�O�Vã± �zOKHSOpZ[dKRPwNZSoNZ]QsOTi[d]\NZ]�hÍSepqSOzOKHlR]dKRNZPdTiSOfDTUSm[dKRfqKH]�¸�Â�NZgicOKRzØvxceSOlH[dTUpqSÍ��LeTUl�LP\Na[\TUP�£XKRP-�O�xõ� h*Êõ ü ¢FLOK�PQ[�N�SOzXNZ]Qz�lR]�h�Y�[\pqfq]dNZYOLOTil�zOK|£XSOT�[\TUpZSOP�N�]dK}pqse[�N�TUSOKHz�Tiv5�O�xõ� �TUP:[�NZÓ�KRS[\p�suK�N�Yupqgih�SOpqr#T�N�gZTUS�õb�x[dLOK�]QK�NZzeKR]·TUP�KRSelRpqcO]dNZfqKHz�[\pwÂqKH]dTiv�h�[\LOTiP:vxpZ]·LeKR]dPQKRg�v� ��(��Sm[\ceTi[\T6ÂqKHgihq�[\LeK�v�NZPQ[dKR]r�e��õ� �fZ]dp���PR�O[dLOK�gUKHPdPFYOgUNZcOPdTisOgUK�NZ]dK�[dLOK�]dKHPdcOg�[\TiSOf lR]�h�Y�[\pqfq]dNZYOLOTil�NZPdPQcOr#Ye[\TipqSOPR���S ³ KRlH[dTUpqSIºO�éð���Ky��TigUge[\NZÓZKs�O�xõ� �[\p�s�KyN�vxceSOlH[dTUpqSw��LOTUl�LIfq]dp���P®ÂqKR]�h#PdgUTifqLm[\g�h�v�NZP�[\KR]@[\LONZSNZSmh<Yupqg�h�SepqrITUNZgXTUS�õ(Îe[\LeTUP�TiPFN�PQ[d]dpqSefqKR] ¸¹[\LONZS�¸�P�[�NZSezXNZ]dz�lH]Qh�Ye[dpqfq]\N�YOLOTUl�NZPdPQcOrIY�[\TUpZS��esOc�[NZPF��K�zeTUPdlHcOPdP�TUS ³ KHlH[dTUpqSBºO�éð ��K�s�KHgUTiK¢ÂqK�[dLXN�[�Ti[�TiPFPQ[dTUgUg�Ì�cOTi[dK�N#]QK�NZPQpqSXNZsOgiK�NZPdPQcOr#Ye[\TipqS��¢FLOK�SOpZ[\N�[\TipqS`äV��o�� | å�rIKRNZSOP�[dLXN�[h�_TUP�PdKHgUKRl|[\KHzBvx]Qpqrã[\LOK�PdKH[Nx?z�ø�Ù<{}|�NZlRlHpq]dzeTUSOf�[dpzOTiPQ[\]QTUsOc�[\TUpZS�� | ü J K���]QTi[dKYt | [dp#zeKRSOpZ[dK�[dLOK�cOSOT�vxpq]drµzOTUP�[\]dTisOce[dTUpqS<p}ÂZKR]éx?zeø�Ù<{}| ü

��üvu�wyx���x�ý!w ³��k�*z 9è¯UÇ|®�=Z©(&�/U§B4�Ç�Æ�Ç04�Ë�¬�®�=�§XÇ04�¦ ¨�©¹ªZ©�¬«¥a®�ÈجUÆ�ÆRªZ¬kC�© ¥A$|Ç[�O�xõ� �¸¹pqSOK�¸¹��N�hn¬ . ©(&XÇ. ¥a¯�¯U¥?12¬�®�=�¤�¥a®BCZ¬�©�¬«¥a®eÆ]&X¥a¯6C�ö

º�ð

Page 58: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

7 ©(&XÇ04\Ç�¬UÆ_ªsC�Ç|© Ç04�¦ ¬�®X¬UÆ|©�¬«¤BªZ¯>=�¥?4�¬�©(&�¦ 1F&�¬«¤'&s4|¨�®eÆB¬�®j©­¬�¦�Ç�§X¥a¯±°q�'Ç �~Ç  BªZ®BCͤ�¥a¦�§u¨�©�Ç�ÆÈ>�(�� |ø

7 . ¥?4�ªZ¯�¯e§B4\¥<$�ª�$R¬�¯±¬UÆ|©­¬«¤�§X¥a¯±°q���O�xõ� d �/�©­¬�¦�Ç�ªZ¯>=�¥?4|¬�©(&�¦�Æ�ñ�ø . ¥?4�ªZ¯�¯e§X¥a¯±°�®²¥a¦ ¬�ªZ¯éÆY{A3 . ¥?4�ªZ¯�¯Æ|¨�lʤ|¬«Ç|®X©­¯±°#¯�ª<4j=�Ç�õ�øõ1�ÇN&eª<ËZÇ

Å2]�� )|<£ ¤ ñ�� È>�(�� d  § �[¨�© Ù{I���e��õ� d  ü��üvu�wyx���x�ý!w ³��k����! Ç|©�Èù$|Ç�ª#¯UÇ|®�=Z©�&�/U§B4\Ç�Æ�Ç04�Ë�¬�®�=�§XÇ04�¦ ¨�©¹ªZ©�¬«¥a®uÐ+9'§X¥a¯±°a®²¥a¦ ¬�ªZ¯�©�¬�¦�Ç ¤�¥a¦A/§u¨�©¹ª�$R¯UÇ�§B4\Ç�CZ¬«¤dªZ© ÇscµùBx?zeøRÙ<{?ÉhÊ x?z�ø�Ù<{B¬UÆ�ÆRªZ¬kC�© ¥½$|Ç�ª}�O�xõ� ¹¸�LXNZ]Qz�¸�lHpq]dK�YO]QKRzOTil�N�[dK�p�v®Èj¬ .©(&XÇ . ¥a¯�¯U¥?12¬�®�=I¤�¥a®BCZ¬�©­¬«¥a®½&X¥a¯6CaÆ�ö . ¥?4�ªZ¯�¯�§B4\¥<$�ª�$R¬�¯±¬UÆ|©�¬«¤}§X¥a¯±°Z���e��õ� d �/�©­¬�¦�ÇNC�Ç�¤|¬UÆ|¬«¥a®6ªZ¯>=�¥?4�¬�©(&�¦�Æñ�ø . ¥?4�ªZ¯�¯�§X¥a¯±°a®²¥a¦ ¬�ªZ¯éÆd{#ø . ¥?4�ªZ¯�¯�Æ|¨�lâ¤|¬«Ç|®X©�¯±°#¯�ª<4 =�ÇFõ�øF1�Ç"&eª<ËZÇ

Å2]�� )|<£ ¤ ñ �jÈ>���� Q  § c����� 2¨�© Ùó Ù{I���e��õ� d  ü³ cOYOYupqPQKy[\LON�[P~ TUP}N�giKRSOf�[\L�¸¹YO]dKHPdKR]SÂ�TUSOf�Y�pZgih²��õ� �¸�pZSOK|¸«�N�hwYuKR]dr�ce[�N�[dTUpqS²��r�KH[P� § #wÄÆÅ

��LOKH]dK�Ç #FÇ § Ç Å�Ç § õ�ø²NZSOzDgiKH[éÈÍsuK�[dLOK�vxcOSOl|[\TipqSDzOKH£OSOKRzMNZPéÈ�����  § #eÄs~²��ÅO  ü �¹[�TUP�KRNZPQh[\p�l�LeKRl�Ón[dLXN�[éÈÍTiP�NZgiPdp�N<giKRSOf�[\L�¸¹YO]dKHPdKR]SÂ�TUSOf<Yupqg�h��xõ� ¹¸�pqSOK�¸¹��N�hnYuKR]Qr�ce[�Na[\TUpZS���í�pqgUzO]QKRTil�LNZSOz°r�K¢Â�TUS¡�«í�pqgUze]dKRTil�L�NZSOzsr:K0Â�TUS��}Ù�ÚqÛqÚq �LXN�ÂqK#PdLOp���SÍ[dLXN�[Gc�����  §�� |À@ô ¸ #\À Å�À���r#p�zÍóZ TUP�NBYupqgih²��õ� �¸¹LXNZ]dz�¸�lRpZ]dK YO]QKRzOTil�N�[dKwvxpZ]�Èt�xPdKHKI�yYOY�KHSOzOT {ÍV�� ó�pZv��«í�pqgUzO]QKRTil�L��2Ù�ÚqÚqÛq yvxpq]�NÂqKH]Qhb]dKRNZzXNZsOgiK�Ye]dp�pZv>pZv�[\LOTiPy]QKRPdcegi[� ��F��SbKRSm[\Ti]dKRg�hBPQ[d]\NZTifqLm[dvxpq]��NZ]Qznr#p�zeTi£XlRN�[\TipqSBpZv�[dLOKRTi]YO]Qp�pZvPdLOp���P�[\LON�[�T�v~�TUP�NngUKHSOfZ[dL�¸�YO]QKRPQKR]�Â�TiSOf��e��õ� ¹¸�pqSeK|¸¹��N�hÍYuKR]Qr�ce[\N�[\TipqS��:[\LOKHS¿ÈjTUP�NgUKHSOfZ[dL�¸�YO]QKRPQKR]�Â�TiSOf,�O�xõ� �¸¹pqSOK�¸¹��N�hnY�KH]dr�c�[�N�[dTUpqS�NZSez�c<���� �TUP�N`�e��õ� �¸¹LXNZ]dz�¸�lRpZ]dK YO]QKRzOTil�N�[dKvxpq]%È ü��üvu�wyx���x�ý!w ³��k��� 9 . ªZ¦ ¬�¯±°j¥ . §B4�¥<$�ª�$R¬�¯±¬�©�°¯CZ¬UÆ|©k4�¬k$R¨�©�¬«¥a®eÆ�x�� &$0¥| 2 {â¥a®¹x?zeøRÙ<{h&$0¥| 2 ¬UÆbÆRªZ¬kC©�¥ª$|Çr�e��õ� �¸¹YOPdKHcOzOpq]dNZSOzOpqr ¬ . x�� &$0 | 2�{<¬UÆ�§X¥a¯±°Z���e��õ� d �/�©­¬�¦�Ç�¬�®BCZ¬UÆ|©­¬�®�=Z¨�¬UÆ'&eª�$R¯UÇ . 4�¥a¦ x�t &$0 | 2V{ üñ &eªZ©:¬UÆ'3 . ¥?4�ªZ¯�¯�§B4\¥<$�ª�$R¬�¯±¬UÆ|©­¬«¤(§X¥a¯±°q���O�xõ� Q 0/�©�¬�¦�ÇyC�Ç�¤|¬UÆ|¬«¥a®�ªZ¯>=�¥?4�¬�©(&�¦�Æ@ñ�ø . ¥?4�ªZ¯�¯�§X¥a¯±°a®²¥a¦ ¬�ªZ¯éÆ{wø . ¥?4�ªZ¯�¯�Æ|¨�lʤ|¬«Ç|®X©­¯±°#¯�ª<4j=�Ç�õ�øõ1�Ç]&eª<ËZÇ����� Å2]�  ����#� £)� ¤ ñ � ð   § Ù�¨Xô Å2]�  )|��#� £�� ¤ ñ � ð   § Ù�¨ ����� © Ù{����O�xõ� d  ü��üvu�wyx���x�ý!w ³��k��Ø 9è§X¥a¯±°q���O�xõ� d �/�©­¬�¦�ÇAC�Ç|©�Ç04|¦ ¬�®X¬UÆ|©�¬«¤ ªZ¯>=�¥?4|¬�©(&�¦�� ù-x?zeøRÙ<{}|ªÊ x?z�ø�Ù<{h&$0¥| 2¬UÆÆRªZ¬kC�©�¥n$|Ç�ªs�e��õ� �¸�YePdKRcezOpq]\N�SOzOpqr fZKRSOKH]\N�[dpq]�¬ . x�� &$0 | 2V{w¬UÆ�ªs�O�xõ� �/U§XÆ�Ç|¨\C�¥?4QªZ®BC�¥a¦ . ªZ¦ ¬�¯±°¥ . CZ¬UÆ|©j4�¬k$R¨�©­¬«¥a®eÆ'3]1F&XÇ04�ÇD� &'0 | 2�¬UÆ<©(&XǽCZ¬UÆ|©j4�¬k$R¨�©­¬«¥a®¡¥a®¯x?zeø�Ù<{h&$0¥| 2 ¥<$R©«ªZ¬�®²Ç�C ªaÆ . ¥a¯�¯U¥?1�Æ�ö_©�¥

ºmî

Page 59: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

Æ�Ç|¯UÇ�¤|© ð oi� &$0¥| 2 ø´§u¬«¤'áé�¬o,t | ªZ®BC#Æ�Ç|© ð § ���(��  ü�� Ç�1~4�¬�©�Ç��I� ð  VÀF©�¥�C�Ç|®²¥a© Ç�©(&XÇyÃ�/�©(&Ò$R¬�©¥ . �I� ð   ü�yp�����K�l�NZS_PQ[\N�[\K�[\LeK�vxpZgUgUp���TiSOf�cOPQKHvxcOg�[dLOKRpq]QKRr_ù

ú�û:ü�ý´þaü�ÿ ³��k�)³ü! Ç|©ÆÈ�$|Ç�ªI¯UÇ|®�=Z©(&�/U§B4�Ç�Æ�Ç04�Ë�¬�®�=]�O�xõ� �/d¥a®²Ç0/21}ªZ°�§XÇ04|¦ ¨�©«ªZ©­¬«¥a®�ªZ®BCw¯UÇ|©ycò$|ǪD�e��õ� �/ &eª<4�C</d¤�¥?4�Ç�§B4\Ç�CZ¬«¤dªZ© Ç ¥ . È ü ! Ç|©.�*ù\x?zeø�Ù<{}|éÊ x?zeøRÙ<{h&$0¥| 2 $|ÇDC�Ç��@®²Ç�C<ªaÆ . ¥a¯�¯U¥?1�Æ�ö

���(��  § c����� -ÄYc�� È����� d �ÄYc�� È�� È>�(�� d d ÍÄéã}ã}ã�ÄYc��jÈ &$0¥| 2 Ü[¸ ���� Q  üñ &XÇ|®��÷¬UÆ�ªD�e��õ� �/U§XÆ�Ç|¨\C�¥?4QªZ®BC�¥a¦b=�Ç|®²Ç04dªZ©�¥?4HÐd� ¥?4\Ç�¥?ËZÇ04�3@©(&XÇDCZ¬UÆ|©j4�¬k$R¨�©­¬«¥a®eÆ

x��I� ð  �Ä%È &$0¥| 2 � ð  '{ �  �|<£ªZ®BC

xh�¿Ä%È &$0¥| 2 � ð  �{��B �|��#� £)��� �  )|<£ª<4\ÇF§X¥a¯±°q���O�xõ� Q 0/�©�¬�¦�Ç�¬�®BCZ¬UÆ|©­¬�®�=Z¨�¬UÆ'&eª�$R¯UÇRÐ��SÊ[\LOKDlRNZPdK6��LOKH]dK��e��õ� �TUP<NØYupqg�h�SepqrITUNZgF[\LOTiP�[\LeKRpq]QKRr TiP�NØPQ[\NZSOzXN�]dzâ]dKHPdcOg�[�Î�PdKHK

KZ�ëfO����Å2]dpqYupqPdT�[\TipqSà^��iÙ�uDpZvI�­í�pqgizO]dKHTUl�L��yÙ�ÚqÚmðZ |�Ê¢FLOTiP�lRpZSOPQ[d]dcOl|[\TUpZSàpZv�N YOPdKHcOzOpq]dNZSOzOpqrfqKHSOKR]dN�[\pq]H�²N�gUpqSOf#��Ti[dLb[\LeK�zOK|£XSOTi[dTUpqSbpZv@N�YePdKRcezOpq]\N�SOzOpqrìfqKRSeKR]\Na[\pq]H��TiPypq]QTUfqTiSXNZgigih#vx]dpqr�­Ü�gUcOr*NZSOzbfbTUlRNZgUT­�eÙ�ÚZÛZº� |�·¢FLOK�YO]Qp�pZvupZvu[\LeKFrIpq]QKfqKHSOKR]dNZgO[\LOKHpq]dKHr*P�[�N�[dKRzINZs�p}ÂZK�����LOKR]QK�e��õ� �SeKRKRz�SOp�[�s�KwN<Yupqgih�SOpZrITUNZg� TUP�N<P�[\]dNZTUfqL�vxpq]Q��NZ]dzDr#p�zOTi£XlRN�[\TipqSnp�v}[dLOK YO]Qp�pZv}pZv}[\LeKPQ[\NZSOzXN�]dz�]dKRPQcOgi[R��KHSm[\TU]QKRg�h�NZSXNZgipqfqpqceP�[\pF[\LOK}r#p�zOT�£Xl�Na[\TUpZS�pZv�[\LOK�í�pZgUzO]QKRTUl�L�¸Vr�K¢Â�TUS�[dLOKRpZ]dKRrr#KRSm[\TipqSOKRz_NZsup}ÂqK��� C-�Rü�þ��\ý�hxxý!wú³��k�� � Ç�®²¥a© Ç�©�&eªZ©N$R° ñ &XÇ�¥?4\Ç|¦��uÐ����ý3�Ç0ËZÇ|®o¬ . ª<§X¥a¯±°Z���e��õ� d 0/�©�¬�¦�Ç<ªZ¯>=�¥?/4�¬�©�&�¦Þ¬UÆh=Z¬(ËZÇ|®ªÈ &$0¥| 2 � ð  �ªZ¯U¥a®�=D12¬�©�&wÆ�¥a¦�Ç"$R¬�©�Æ�¥ . �I� ð  �ø�©(&XÇ�ªZ¯>=�¥?4�¬�©(&�¦ Æ|©­¬�¯�¯�¤dªZ®X®²¥a©X§B4�Ç�CZ¬«¤|©©(&XÇ<¨�®eÆ�Ç�Ç|®ù$R¬�©�Æn¥ . �I� ð  �12¬�©(&oª�¤�¤|¨�4Qª�¤|°bÆ|¬6=Z®X¬ �¤dªZ®X©­¯±°Ò$|Ç|©­©�Ç04<©(&eªZ®jÙ�ÖZó ü ñ &�¬UÆI¬UÆ�$|Ç�¤dªZ¨mÆ�Ç©(&XÇ�ª�$R¬�¯±¬�©­°w©�¥pC�¥�Æ|¨X¤'&�§B4\Ç�CZ¬«¤|©­¬«¥a®µ1�¥a¨�¯6CªË�¬«¥a¯�ªZ© Ç�©�&XÇF§X¥a¯±°Z���e��õ� d 0/�©�¬�¦�Ç�¬�®BCZ¬UÆ|©�¬�®�=Z¨�¬UÆ'&eª�$R¬�¯±¬�©­°1F&�¬«¤'& ¬UÆ�ªaÆ�Æ�Ç04�© Ç�C�¬�® ñ &XÇ�¥?4\Ç|¦��uÐ����ý3�Æ|¬�®²¤�Çy¤|¯UÇdª<4�¯±°�®²¥�§X¥a¯±°Z���e��õ� Q 0/�©�¬�¦�Ç�ªZ¯>=�¥?4�¬�©(&�¦��¹¬�® . ª�¤|©(3®²¥6ªZ¯>=�¥?4�¬�©�&�¦ ªZ©�ªZ¯�¯ ��¤dªZ®`Æ|¨´¤�¤�Ç�Æ�Æ . ¨�¯�¯±°#§B4\Ç�CZ¬«¤|©�©(&XÇI¨�®eÆ�Ç�Ç|®�$R¬�©xÆB¥ . ª6¨�®X¬ . ¥?4�¦ ¯±°bÆ�Ç|¯UÇ�¤|©�Ç�C4QªZ®BC�¥a¦µÆ|©k4|¬�®�=mÐ

º)u

Page 60: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

»y¼�»y¼2G ûT�]RüHÍ¿�Ä]�-RõKyÀ«HTO Ã~ZýZÎþR�r�KH[+Èbs�K�N�gUKHSOfZ[dL�¸�YO]QKRPQKR]�Â�TiSOf�pqSOK�¸¹��N�h�YuKR]Qr�ce[�Na[\TUpZS���¢FLOK�PdKH["x?zeø�Ù<{�%`��TigUg´s�KypqcO]�PdKH[�pZvPdKHlR]QKH[�ÓZK|h�PR����P�zOTiPdlHcOPdPQKRzMTUS ³ KHlH[dTUpqSDºO�ëºO��Ù���K l�NZSDPQcOYOYupqPdK���T�[\LOpqc�[�gUpZPdP�p�v�fZKRSOKH]\NZgiTi[�h[\LON�[NÈ�LXNZP�NILXNZ]Qz�¸�lHpq]dK�YO]QKRzOTil�N�[dKZ�%r�KH[U�ÞsuK�[dLOKN  | i¢¡ ¸¹YOPdKHcOzOpq]dNZSOzOpqrìfqKHSOKR]dN�[\pq]yNZPQPdp�lRT�¸N�[dKRz���T�[\L½È���LOpqPQK�K|{�TiPQ[\KHSOlRK�TUP�NZPdPQKR]Q[dKRz�smhI¢FLOKRpq]QKRrÞºO��Ù�ºO��Pdp£�ËrINZYOP}TUSOYece[\P}pZv�giKRSOfZ[dLº [\pbpqce[dYOce[\P�pZvFgUKHSOfZ[dLp  % iv¡ ü r�KH[�õ § ó�º ü �Xpq]#Ù�F_ÃDF¤  % i*¡ ø·gUKH[a¥BÀzOKHSOpZ[\KI[\LOKpÃ�¸¹[dLf�¸¹KRgUKHrIKHSm[PdcesOPdK|[yp�v�[\LOK�PQKH[]x�º¦àÙZø üSüSü ø�ó�º�{�cOSOzeKR]FPdpZrIK�£�{eKHz_NZSOz�K�N�PdTUg�h#lRpqr#YOce[\NZsOgUKpq]QzOKR]QTUSOf���K�� fO���egiK|{�TUlHpqfq]dNZYOLOTil� |��N�SOz�giKH[ ð À²suKy[dLOKylHpqS?ÁQcOSelH[\TipqS�§ ë� �¨�© ��ë ü í�T6ÂqKRS<NZSmhwTUSOYOc�[�Toùx?z�ø�Ù<{ | ø�giKH["Ã����� �suK�[dLOK�PdrINZgigUKRP�[�TiSOzOK�{6TiSsx�Ùqø üSügü ø   % i*¡ {#Pdcel�L�[dLXN�[ ð À 0 ��2@TUPyP\N�[dTUPQ£OKRzsmh�� ü �¹v�Sep ð À 0 ��2 TiPFP\N�[dTUPQ£OKRz_s�h½�bvxpq]�ÙIF�Ã��(�� YF   % i ¡ [dLOKRS_gUK|[wÃ��(��  § ze�¢FLOK lRgUNZPdP�É i LXN�P�N�lRpZSOlRKHYe[�m¢+�vxpq]yK�NZl�LDPQKRlH]dKH[�Ó�KHhN(�o�x?zeø�Ù�{ % ü �¹v®m�+�TiPy[dLOK�[�N�]dfqK|[

lRpZSOlRKHYe[��²[dLOKRS6NZS6K|{eNZr#YOgiKn��TUP�¨�Æ�Ç . ¨�¯XTivF�c¸Íã}ã}ãS� % § È ��ª «�  ��(� �NZSOz6TiP�¨mÆ�Ç|¯UÇ�Æ�Æ�pZ[\LeKR]Q��TiPdK����P�TUS ³ KHlH[dTUpqS_ºO�ëºO�iÙZ�BÈ � ª «�  ��(� �zOKHSOpZ[dKRPy[dLOK�]dKHPdcOg�[yp�v2NZYOYegih�TUSOfpÈ6K|{eNZl|[\g�h   % i*¡ [\TirIKHP[\p�( ü¢FLOK�lRpqSelRKRY�["m�+�TiPFzOKH£XSeKRz6N�PvxpqgUgip���PHù

7 m�+Z����  § zwTiv��6TiPFcOPdKHgUKHPdPR�7 m�+Z����  § �I��(�  À 0 ��2 øZ[\LeKyÃ\���� �¸«[\L#sOTi[>pZv¬�I��(� |øZTivB�<TUP2cePdKHvxceg�N�SOzpÃ\���� h*ËÙ ü �¹vB��TUP2cePdKHvxcegNZSezµÃ\����  § z [\LOKHS�m�+Z����  § z ü

»y¼�»y¼�� J°YX¿�¿LKB¿MK�ûT�]R:¿®Y[RÆI »y¼#­��Ti]dPQ[���K�PQLOp��0[\LON�[em�+<TUP�N¬f�¸�zOKHlRTiPdTUpZS`gUTUP�[��M�Xpq]�KRNZl�LtÙ�F ê�FïºDø�gUKH[n·�ë#zOKRSepZ[\K�[\LeKgUT�[\KH]\NZg ��ëyT�v�[\LOKwêZ¸¹[dL�sOTi[�pZvFÈ � ª «   ��(e }TUP�ÙqøeNZSOzBgUK|[U·�ë�zeKRSOpZ[dK]��ë�pZ[dLOKR]���TUPdK��2¢FLOK�vxpqgUgip���TUSOff�¸¹zOKRlHTUPdTipqSBgUTiPQ[FpZv(gUKHSOfZ[dL�º��  % i ¡ lHpqr#YOce[\KHP%m¢+�ù

�(·�¸|ø'zm �ø üSüSü ø�� · % ø'zq |ø�� ð ¸�ø��I��(� '¸Q �ø üSüSü ø�� ð � ª «�  ø�����(�  � ª «�    ü¢(p�s�pqceSOz�±J²D³®´���É i Îdõ�ø�×� �ø�SepZ[\K�[\LXN�[�cOYupqS6]dKHlRKHT�Â�TUSOf<NIPdTiSOfqgiK�YupqPdT�[\T6ÂqK�K�{eNZrIYegUK�NZScOS�supqcOSOzOKHz giK�NZ]QSOKR]�l�NZSDTiS)ÂZKR]�[nÈ ��ª «�  ��(� �[dp<£XSOz_(Í�x[\LeTUP�TiP�Y�pZPdPdTisOgUK�PdTUSelRKªÈÍTUP�N�Y�KH]dr�c�¸

[�Na[\TUpZS´ �NZSez<[\L�cOPFgUKRNZ]dSI[\LOK�[\NZ]dfZKH[FlHpqSOlRKHYe[%m�+�K�{ON�lH[\g�hq�2�yPFTUS�[dLOK�YO]Qp�pZv:p�v(¢FLeKRpq]QKRr0ºe�é^e�Ti[FvxpZgUgUp���P�[dLXN�[~±J²D³P´��­É i Î\õ�ø�×\  § ß � Ù�ÖZ×�  ü��SÊN�SXNZgUpZfqpqcOP#NZ]QfqcOr#KRSm[I[dp [dLOKnlHpqrIYece[�N�[dTUpqSONZgFP\NZr#YOgiK_lHpqr#YOgUK�{eT�[�hocOYOYuKR]Is�pqceSOzYO]Qp�pZv(pZv(¢FLOKRpZ]dKRr ºe�é^ KRP�[�NZsegUTUPQLOKRPF[dLXN�[h´c³P´��­É i Î\õ(ø�×�  § ß �   % iv¡ ÖZ×�  § ß ��õ i ÖZ×\  ü

ºmÛ

Page 61: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

�Xpq]<[\LOK�lHpqrIYece[�N�[dTUpqSONZgygip���KH]_supqcOSez���lHpqSOPdTizOKR]�[dLOKDzOTUP�[\]QTUsOce[dTUpqSù� | p}ÂqKH]Òx?zeø�Ù<{ |��LOTil�LwNZPdPQTUfqSOP(��KHTUfqLm[�Ùuôwîq×>[\p�[dLOKPdTiSOfqgiK�cOPQKRgUKHPdP2K�{ON�rIYOgiK+z�|�NZSOzwNZPQPdTifqSOP���KRTifqLm[@îq×�Ö®  % i ¡[\pwKRNZl�LnpZv·[\LeK   % i*¡ cOPdK|vxcOg:K|{eNZr#YOgUKHP

x�È � ª «   ��(� -Ä¥ Àj{ � ª «  À@ô ¸

��LeKR]dK���KnN�]dK�Â�TUK|��TUSOf�K�N�l�L\¥BÀyNZP#NZS¯º�¸�sOT�[ PQ[d]dTUSefMTiS`[\LOKBpqs�Â�TUpqceP��N�he |� r�KH[b^èsuKBNP\N�rIYOgiK�pZvY  % iv¡ Öqóaºm×�K|{eNZr#YOgiKRP���LOTil�L�NZ]QK�zO]\N���S<vx]QpqrL���1�kmaø�� |   ü Ü�h�¢FLeKRpq]QKRr0ºe�iÙ�ºe����KLXN�ÂqK�[\LON�[F[\LOK�P�[\]dTiSOf�¸ ÂZN�gUcOKHz�]\NZSOzepqrEÂ�NZ]dTUNZsOgiKRP

x��I� ð  -ÄyÈ � ª «   � ð  '{ �  )| ªNZSOz

xh�¯Ä%È � ª «   � ð  �{��B �| � ª «   � �  )| ªNZ]QKyYupqgih�SOpZrITUNZge[\TirIK�TiSOzOTUP�[\TiSOfqcOTiPdLXNZsegUKZ��V�pqSOPQKRÌ�cOKHS�[dgihZ�OK¢ÂZKRS�[dLOpqcOfqL<N�Yupqgih�SOpqr#T�N�g�[\TirIKgUKRNZ]dSeKR]���LeTUl�LØLXN�P�zO]dN���SØ[dLOK<PdNZrIYegUKj^�r�N�hMzOTUPQlRp}ÂqKH]eÈ � ª «   ��(� �vx]dpZr N�S�hMYupqPdT�[\T6ÂqK�K�{�¸NZr#YOgUK���smhM¶�sOPdKH]�Â�N�[\TipqSMºO�iÙ�ð�PQcOl�LØN�giK�NZ]QSOKR]�lRNZSOSOpZ[�YO]QKRzOTilH[�[\LeK#seTi[\P�p�vr�I��(� y��LOTUl�LMTi[LXNZP�SOp�[�PdKRKHSb��T�[\L_NZlRlHcO]\N�lHh�PQTUfqSOT�£Xl�N�S�[dgihIs�K|[d[dKR]F[\LXN�SMÙ�Öqó ü ³ TiSOlRK�[dLOK�K|{�YuKRlH[dKRz6S�cOr�s�KH]pZv�cOPdK|vxcOg�K�{eNZrIYegUKRP�TiSQ^WTiP   % i*¡ Ö�ºeø:NbPQ[d]\NZTifqLm[dvxpq]��N�]dz NZYOYOgiTUl�Na[\TUpZSDpZv�V�LOKH]dSOp��jsupqcOSezOPPdLep���Py[\LXN�[���T�[\LµÂZKR]�hnLOTifqL_YO]dpqsONZsOTUgiTi[�h�^j��TUgUg²lRpqSm[�N�TUSbvxK|��KH]y[dLXNZS¯  % i ¡ Öqó cOPdK|vxcOg�K�{eNZrw¸YOgiKRPR�uNZSezb[\L�cOP���Ti[dLµÂZKR]Qh_LOTifqLBYO]dpZsXNZsOTigUTi[�hI[\LOK�Yupqgih�SOpqr#T�N�g�[\TirIK�giK�NZ]QSOKR]F��TigUg�LXN�ÂqK�PdKHKRSN�[yrIpZPQ[yLXNZgiv�pZv>[dLOK�  % i ¡ sOT�[\P�pZvP�I��(�  ü ³ TiSOlRK�cOPdK|vxcOg·K|{eNZr#YOgUKHP���LOTil�L6lRpZ]d]dKHPdYupqSOz6[dpI[\LeKcOSOPQKRKHS`sOTi[dP�pZvr����(e �LXN�ÂZK���KHTUfqLm[�N�[�gUK�N�PQ[�^q×�cOSezOKR]�[dLOK#zOTUP�[\]dTisOce[dTUpqSÒ� | ø:[\LOK#Yupqgih�SOp�¸r#T�NZg²[dTUr#K�gUKRNZ]dSOKH]RêëP�p�ÂZKR]dNZgUg:KR]Q]dpq]�]dN�[\K���TUgig�K|{�lRKHKRzD×���Ti[dL�ÂZKR]Qh_LOTifqL_YO]dpqsONZsOTUgiTi[�hZ�Jq�KRSOlHK  % iv¡ Öqó�ºm×�K�{ON�rIYOgiKRP�zOp�SepZ[�Pdcek#lHKwvxpZ]�Y�pqg�h�SOpqr#T�NZg�[\TirIK�gUKRNZ]dSXN�sOTUgiTi[�hq��NZSOzD��KwLXN�ÂqK [\LXNa[´c³P´���É i Î\õ(ø�×� U*   % i ¡ Öqó�ºm× § dw��õ i�ÖZ×\  ü ��¢FLOKHpq]dKHr ºO�éÚq 

»y¼�»y¼�» �WÄ �ùKyÀOÁSIåÃPO�°�ÃPK�¹[:TUP�TiSm[\KR]QKRP�[\TUSef�[dp�lRpqSm[\]dNZPQ[:[dLOK2s�pZcOSOzOP:fqT6ÂqKRS�TUS�¢FLeKRpq]QKRr˺O� Ú���Ti[dL�p�[\LOKH]·Ó�SOp���S�s�pZcOSOzOPH�¢FLOK�cOYeY�KH]s�pZcOSOz�pqS�TiSevxpq]drIN�[dTUpqS�¸«[\LOKHpq]dK|[\TilFP\NZr#YOgUKylRpqr#YOgiK|{�Ti[�h#��LOTUl�L�TiP�fqT6ÂqKHS�TUS<¢FLOK�¸pq]QKRr ºO� Ú TUP[dLOK�s�KHPQ[�YupqPdPQTUsOgiK�vxpq]FSOpqSm[\]QT�Â�T�N�g²lRpZSOlRKHYe[�lHg�NZPQPdKHPR�}ö�T6ÂqKHPQ[�êëP�Y�pZgih�SOpqr#T�NZg´[\TirIKNZgifqpq]dT�[\LOrÞvxpq]FgiK�NZ]QSOTUSOfpf�¸¹zOKRlHTUPQTUpqSBgUTiPQ[dP��­ö�T�ÂqKHPQ[R��Ù�ÚqÛ�uq �]QKRÌ�cOTi]dKRP ß � | «± r#TUS�x�gUpZf}õ(ø\gUpZf ¸± {� K|{eNZr#YOgiKRPRÎ([\L�cOP�pZcO]�gip���KH]�supqcOSOz pqSÍlHpqr#YOce[�Na[\TUpZSXNZg>PdNZrIYegUKwlRpqr#YOgiK|{�Ti[�hDlHpqcOgizÍsuKITirw¸

ºmÚ

Page 62: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

YO]Qp�ÂZKRzbs�h_Na[�rIpqP�[�N#gipqfmNZ]QTi[dLOrITil�v�N�lH[\pZ]vxpq]FlRpqSelRKRY�[�lHg�NZPQPdKRP�pZv�f�¸�zeKRlRTiPdTipqSBgUTUP�[\PH�2Ï2LO]dKHS�¸vxKRcel�L�[BÇ|©«ÐDªZ¯�Ðn�­Ï2LO]QKRSevxKHcOl�Lm[nÇ|©�ªZ¯�ÐU�2Ù�ÚqÛqÚm �LON}ÂZKIPQLOp���SÍ[\LON�[�[\LOK Ò}NZYeSOTUÓq¸�V�LeKR]�ÂZpqSOKHSOÓ�TUPzOTirIKHSOPdTipqSbp�v>[\LOK�lRgUNZPdP�pZv®f�¸¹zOKRlHTUPQTUpqSbgUTUP�[\P�pqS6õ¬Â�NZ]dTUNZsOgiKRP�TUP"dw��õÍi� |øuNZSOznLOKRSOlHK Õ ��õ i�ÖZ×\ K|{eNZr#YOgiKRP�NZ]QK ]dKHÌ�ceTU]dKHz6vxpq]�TiSevxpq]drIN�[dTUpqS�¸«[\LOKHpq]dK|[\Til�]QK�NZPQpqSOP�vxpq]�gUK�N�]dSOTiSOf½f�¸¹zOKRlHTUPQTUpqSDgiTUPQ[dPR�¶�cO]�¢FLeKRpq]QKRr.ºO�éÚ�PdLOp���P�[dLXN�[ Õ ��õ i ÖZ×� :K|{eNZr#YOgUKHP�l�N�S�s�K�]dKHÌ�ceTU]dKHz�vxpq]·gUKRNZ]dSOTiSOf�PdcOsulRgUNZPdPQKRPpZv�f�¸�zeKRlRTiPdTipqSBgUTUP�[\P�vxpZ]�¤�¥a¦�§u¨�©«ªZ©­¬«¥a®uªZ¯�]dKRNZPdpZSOPR�XK0ÂqKRSnTUS<[\LeK�N�sOPdKHSOlRK�pZv�NZSmhBTUSevxpq]Qr�Na[\TUpZS�¸[\LeKRpq]QKH[\Til�sXNZ]Q]dTUKH]dPF[\pwgiK�NZ]QSOTUSOf�vx]Qpqrµv�N�]vxKH��KR]�K|{eNZr#YOgUKHPR�

465¢² ³ ��=}AD9��]��� ?�´«Ðì;(;2=@G¶µDCn;��%×ÍÖ¸· EyG¢��9<;º¹`�]��=@9DG�9�Õ

J K6SOp�� [\cO]QSÊpqcO]�N�[d[dKRSm[\TipqSà[\p¡ªZ©­©k4�¬k$R¨�©�Ç0/dÇ�lʤ|¬«Ç|®X©�giK�NZ]QSOTUSOf NZgifqpq]dT�[\LOr#PR�â¢FLOKHPdK�NZgifqp�¸]dT�[\LOr#P}]dKRÌ�cOTi]dK%ÂqKR]�hIvxK|�tK|{eNZr#YOgUKHP�]QKRg�Na[\T�ÂZKy[dp�[\LeKy[\p�[�NZguS�cOr�suKR]�p�v�TUSOYece[UÂ�NZ]QT�NZsOgiKRPy��T­� K��i�N�[Q[\]dTisOce[dKRP� ���N�SOzoLOKRSOlHKbLON}ÂZKbK�{�lRKRY�[\TUpZSXNZgUg�hØfqp�p�z`Y�KH]Qvxpq]Qr�NZSelRK�p}ÂqKH]ILOTifqL�¸¹zOTUr#KRSOPQTUpqSONZgTUSeYOce[<PQYXNZlHKRP<��LOTil�L¡lRpqSm[\NZTUS¡rINZSmh1TU]Q]dKRgiK¢Â�NZSm[�N�[d[d]dTUsece[\KHPR�'¢FLOTiP�YO]QpqYuKR]Q[�hjLXNZPIgUKHz¡]dK�¸PdKRNZ]dl�LeKR]dPF[dpwNZYOYOg�h�Na[d[\]QTUsOc�[\K|¸¹KHk#lRTiKRSm[FgiK�NZ]QSOTUSOf�NZgUfZpq]dT�[\LOr#P2[\p ]dKRNZg�¸«��pZ]dgUzIYO]dpZsOgUKHrIP�PdcOl�LNZP�l�N�gUKRSezXNZ]�Pdl�LeKRzOcOgiTUSOf ��Ü�gUcOr_�>Ù�ÚZÚ�uq |��[dK|{�[�l�N�[dKRfqpZ]dTUáRN�[\TipqS1�­~�N�fmNZSàÇ|©yªZ¯�ÐU�@Ù�ÚqÚ�uq ���fqKHSOKPQ[d]dcOl|[\cO]QK�Ye]dKRzeTUlH[dTUpqSM��~�K�N�[dpqS_NZSOz ³ KH]�ÂqKHzOTUpe�uó<z�zeÙ� ��XNZSOzBlHpqSm[\K|{�[�¸�PdKHSOPdT�[\T6ÂqK�PdYuKRgUgiTUSOf�lRpq] ¸]dKHlH[dTUpqS �«í�pqgizOTUSOf�NZSOzbö�pZ[dL���Ù�ÚZÚqÚm |��[d[d]dTUsece[\K�¸�KHk#lHTUKRSelHhILXNZP�l�LeTUK¼»eh�suKRKHS�PQ[\cezOTUKHz<TUS#[\LOKI¥a®X¯±¬�®²Çy¦ ¬UÆ|©«ª?ámÇé$|¥a¨�®BCyr#p�zeKRg´pZv

lRpZSOlRKHYe[�giK�NZ]QSOTUSOfe�F��Sb[\LeTUP�rIp�zOKRg:giK�NZ]QSOTUSOf#YO]Qp�lHKRKRzeP�TUSnNIPQKR]dTiKRPypZv>[\]QT�NZgiPR�O��LOKH]dK�TUSnKRNZl�L[\]QT�NZg´[\LeK�gUKRNZ]dSOKH]TUP�fqT�ÂZKRSBNZS�ceSOg�NZsuKRgiKRz�Ü�p�pqgiK�NZS�K|{eNZr#YOgUK"��osx?zeø�Ù�{}|�NZSez�r�cOPQ[YO]dKHzOTUl|[[\LeK�Â�NZgUcOK�mZ�(��  ü �yv�[\KH]�K�N�l�L`YO]dKHzOTUl|[\TUpZSÍ[\LOK�gUK�N�]dSOKH]�TUP�[\pqgizÍ[\LeK�[\]QcOK�Â�NZgUceK�pZv%m����� �N�SOzl�N�SÍcOYuzXN�[dK�Ti[dP�L�h�YupZ[\LeKRPdTiPR�B¢FLOKIrITiPQ[�N�ÓZK#s�pqceSOzÍpZvFN_giK�NZ]QSOTUSOfbNZgifqpq]dT�[\LOrÑpqSØN�[�N�]dfqK|[lRpZSOlRKHYe[®m�TiP>rIKRNZPdce]dKRz smh�[\LOK��pq]dP�[Q¸¹l�NZPQKFS�cer�s�KH]>pZvur#TUPQ[\NZÓZKHP�[\LON�[>[\LeKFNZgUfZpq]dT�[\LOrËrINZÓZKHPp}ÂqKH]�NZgUg�PQKRÌ�cOKHSOlRKHP�p�v2K|{eNZr#YOgUKHPR��NZSOzn[dLOK�r#TUP�[�NZÓ�K�supqcOSez6pZv@N�giK�NZ]QSOTUSef�NZgifqpq]QTi[\Lerµvxpq]yNlRpZSOlRKHYe[lHg�NZPQP�ÉËTiP@[dLOK���pq]QPQ[�¸�l�N�PdK�rITiPQ[�N�ÓZK�s�pqceSOz�N�lR]dpZPdPN�gUgOlHpqSOlRKHYe[\PUm]oDÉ ü �WgiK�NZ]QSOTUSOfNZgifqpq]dT�[\LOrüvÍvxpq]�N�lHpqSOlRKHYe[lHg�NZPQPÉ'p}ÂqKH]"x?zeø�Ù�{ | TUP}P\NZTiz#[dp�]QcOS�TiS�Yupqgih�SOpqr#T�N�g�[\TirIK�T�v²[\LeKr#TUPQ[\NZÓZK�s�pqceSOzIp�vÍvMpqS#É�TUP�Y�pqg�h²��õ� (NZSOz [\LOK[dTUr#K]dKHÌ�cOTU]QKRzwsmhevM[dp�rINZÓ�KFTi[\P�YO]dKHzOTUl|[\TUpZSNZSOz_cOYuzXN�[dK�T�[\PFLmh�Y�p�[\LOKHPdTUP�pqS_K�N�l�LbK�{ON�rIYOgiK�TUPFYupqgih²�xõ�  ü�'Ü�p�pqgiK�NZSwvxcOSOl|[\TUpZS�m�p�ÂZKR]®�c¸|ø üSüSü ø�� | TiP�PdNZTUz#[\p�C�Ç�§XÇ|®BCypqS<NéÂ�NZ]dTUNZsOgUK+�[À²Tiv�[dLOKR]QK�NZ]QK[���p_P�[\]QTUSOfqP"Å�ø ð oùx?zeø�Ù�{ | ��LOTil�L�LXN�ÂqKeÅ�ë § ð ë�vxpq]�NZgigÍê`¦§ ÃdøcÅ?À"¦§ ð À«ø�NZSOz¬m��kÅe A¦§ m�� ð   ü

r�KH[�É s�KnNMlHg�NZPQP�pZv�Ü�p�pZgUK�N�S1vxcOSelH[\TipqSOP#pqS �c¸|ø üSüSü ø�� | KRNZl�L¡pZv���LOTUl�L¡zeKRYuKRSOzOPIpqS¡Na[r#pqPQ[+ânÂZN�]dT�N�sOgUKHPFNZSOzBK�N�l�L_p�v(��LOTil�LBLXNZP�N�zOKRPQlR]QTUYe[dTUpqSBpZv·gUKHSOfZ[dLbNa[FrIpZPQ[s½�cOSOzOKH]�PQpqr#Kð<z

Page 63: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

]dKRNZPdpZSXNZsOgiK#KHSOlRp�zOTiSOfnPQl�LOKRr#KZ���XpqgigUp���TUSefM�­Ü�gicOr Ç|©�ªZ¯�ÐU�}Ù�ÚqÚqðq |����KIPdN�h�[dLXN�[�NbgiK�NZ]QSOTUSOfNZgifqpq]dT�[\LOr vèvxpq]wÉÑTUS`[\LOKBr#TUP�[�NZÓ�K|¸�supqcOSez1r#p�zOKRgFTiP_ªZ©­©k4�¬k$R¨�©�Ç0/dÇ�lʤ|¬«Ç|®X©FTiv�[\LOKBr#TUP�[�NZÓ�KsupqcOSOzâpZvDvµpqStNZSmhÊlHpqSOlHKRYe[µm�o É TUP�Yupqgih²�kâ�ø¢½qø\gipqf>õ�  ü ¢FL�cOPbNZStN�[d[d]dTisOce[\KnKHk#lRTiKRSm[gUKRNZ]dSeTUSOf�NZgUfqpZ]dTi[dLOr lRNZS<gUKRNZ]dS�PdTUr#YOgiKylHpqSOlRKHYe[\PFzeKH£XSOKHzBp�ÂZKR]�vxK|�t]dKRgiK¢Â�NZSm[UÂ�NZ]QT�NZsegUKRP@��Ti[dLN�rITiPQ[�N�ÓZK@s�pZcOSOz���LOTUl�L�l�NZS�s�K@lHpqSOPdTizOKR]dNZsOg�h�PdrINZgUgiKR]�[dLXNZS�[\LOK2[\p�[�NZgmS�cOr�suKR]:pZvÂ�NZ]dTUNZsOgUKHPR�K¢ÂZKRS_��LOKRS_[\LOK�TUzeKRSm[\T�[�h<pZv·[\LOK�]QKRgUK0Â�NZSm[%Â�NZ]QT�NZsegUKRP�TUPFcOSOÓ�SOp���S����S�[\LeTUP@PdKHlH[\TipqS���K�YO]Qp}Â�TizOK�PQ[\]QpqSOf�K¢Â�TUzOKHSOlRK�[\LXNa[�[dLOKR]QK�NZ]QKylHpqSOlRKHYe[lHg�NZPQPdKHP�gUKRNZ]dSONZsOgUK

TUSDYupqg�h�SepqrITUNZg�[\TirIK�vxpq]y��LOTil�LMN�[d[d]dTUsece[\K�¸�KHk#lHTUKRSm[�giK�NZ]QSOTUSef<TUP�TiSevxpq]Qr�N�[dTUpqS�¸¹[\LeKRpq]QKH[\Til�NZgigihYupqPdPQTUsOgiK�sOce[�lRpqr#YOce[\N�[\TipqSXNZgigihwLXN�]dz��2J K�zepw[dLOTUPFsmhBYO]Qp}Â�TiSOf [\LOK�vxpqgigUp���TUSef�[\LOKHpq]dKHrbùú�û:ü�ý´þaü�ÿ ¾��k��� ú ¥?4�ªZ®X°#¬�®X© Ç =�Ç04wm]*tó�ø·¯UÇ|©·gUpqf´�kmaødõ� DC�Ç|®²¥a©�ÇE9 :?7 8gUpqf@gUpZf+ã}ã}ãdgUpZf}õ ü! Ç|©G�e�jm�ø\õ�  § õ!¿ÁÀ4 0�E � | 2 üy,�. ©(&XÇ04\Ç�¬UÆ6Æ�¥a¦�Ç�¬�®X©�Ç =�Ç04^m¹* óÊÆ|¨X¤'&â©(&eªZ©I¯UÇ|®�=Z©(&�/U§B4\Ç�Æ�Ç04�Ë�¬�®�=�e�jm�ø\õ� �/d¥a®²Ç0/21}ªZ°}§XÇ04�¦ ¨�©«ªZ©�¬«¥a®eÆ�ÇQÈq¬UÆ|©(3:©�&XÇ|®#©(&XÇ04�ÇyÇQÈq¬UÆ|©xÆFª ¤�¥a®²¤�Ç�§u©·¤|¯�ªaÆ�Æ2ɵ¥ . ß �xgUpqfO�jm�ø\õ� Q 0/C�Ç�¤|¬UÆ|¬«¥a®6¯±¬UÆ|©�Æé1F&�¬«¤'&�&eªaÆ�©(&XÇ . ¥a¯�¯U¥?12¬�®�=�§B4\¥�§XÇ04�©­¬«Ç�Æ�¬�®D©�&XÇ�¦ ¬UÆ|©«ª?ámÇ0/S$|¥a¨�®BCw¦�¥�C�Ç|¯6ö

7 90¤�¥a¦�§u¨�©«ªZ©�¬«¥a®uªZ¯�¯±°I¨�®B$|¥a¨�®BC�Ç�C#¯UÇdª<4|®²Ç04�¤dªZ®n¯UÇdª<4|®BÉï12¬�©(&nªZ©�¦�¥�Æ|©[��¦ ¬UÆ|©«ª?ámÇ'37 Éæ¤dªZ®¬$|Ç�¯UÇdª<4�®²Ç�CI¬�®I§X¥a¯±°a®²¥a¦ ¬�ªZ¯�©�¬�¦�Ç'37 Éä¤dªZ®X®²¥a©+$|Ç�¯UÇdª<4�®²Ç�C�¬�®B§X¥a¯±°a®²¥a¦ ¬�ªZ¯(©�¬�¦�ÇA$R°BªZ®`ªZ©�©j4�¬k$R¨�© Ç0/dÇ�lʤ|¬«Ç|®X©¯UÇdª<4�®X¬�®�=_ªZ¯>=�¥?/4�¬�©(&�¦#Ð

JâLOTUgiKb[dLOKnK�{eTiPQ[dKRSOlHKDpZv�gUKHSOfZ[\L�¸�YO]QKRPdKH]�Â�TUSefQ�O�kmaødõ� �¸¹pqSOK�¸¹��N�hjYuKR]dr�ce[�N�[dTUpqSePITiP�SOpZ[<NPQ[\NZSOzXN�]dzÍlH]Qh�Ye[dpqfq]\N�YOLOTUlINZPQPdcOr#Ye[\TipqS�����K�suKRgUTiK¢ÂZKw[dLXN�[�Ti[�TiP�P�[\TigUg2N^ÂqKR]�hM]dK�N�PdpqSXN�sOgUK#NZP ¸PdcerIYe[dTUpqS²���PyK¢Â�TUzOKHSOlRK�vxpq]�[\LOTiPysuKRgiTUKHv �XTiS ³ KRlH[dTUpqSbºO�éð�� ó ��K�PdÓ�KH[\l�LMNZSnNZ]dfZcOrIKHSm[���LeTUl�LPdLep���P�[\LXN�[�Tiv�[\LOKH]dKyzOp�KHP�SOp�[�K�{eTiPQ[�N�lHpqgUgiKRlH[dTUpqSwpZv®�O�kmaø\õ� �¸¹pqSOK|¸«�N�hwYuKR]Qr�ce[�Na[\TUpZSOP�����LOKR]QKK�N�l�LnYuKR]dr�ce[�N�[dTUpqS�TiS�[\LOK�lRpZgUgUKHlH[dTUpqS�TiPFzOKH£XSeKRznp}ÂZKR]yN £OSOTi[dK�zepqr�N�TUS´  ¸ �O[dLOKRSB[\LOKH]dK�r�cOP�[K|{�TiPQ[wNZgifqpq]QTi[\LerIP�vxpq]�v�NZlH[dpq]dTiSOf6Ü�gUcOr TUSm[dKRfqKH]dP���LOTil�LoNZ]dK#v�NZ]�r#pq]QKIYup���KH]QvxcOg�[dLXNZS`N�S�hlRce]d]dKHSm[\gih<Ó�Sep���SnNZgUfZpq]dT�[\LOr#P�vxpq][dLOTUP��KRgig�¸¹PQ[\cezOTUKHz_Ye]dpqsOgiKRr_�Ã?Ä®ÅhÆÈÇLÆÈÇLÉIÇ3Ê�ÆÌ˼żÍÎÊÌÏ}Ð�ÆÌÑ�Ò$Ó3Ò$Ô�ÍLÔhÕ¢ÍÎÆÌÕ¼Ô£Õ¢Ö�É]Õ¼ÔhÒ?×ÙØÚÉ)ÏIÖÜÛhÔ�Ý4ÍÎÆÌÕ¼Ô£Í3Å�É¢Ô£ÍÎÅhÒYÔhÕ�Í3Æ�Õ�ԣجÅhÆÈÝ#ÅRØÚÒYÅ�É)Þ�Òrß�Ò'Ò$ÔÛ�Ç3ÆÌÔ�ËsÍÎŶÛ�Ç�ÖàÉ¢Ó�árâ¶Ò'Òdâ¶Ò�Ý4ÍÎÆÌÕ¼Ô}ãhá ä�á ãdÕ¢Öæåèç.Õ¼Ê�Ð�ÓÎÒ'ÆÈÝ#Å*éXê�ë�ë�ì�í�ÖÜÕ�ÓPÉ�Ð�ÆÈÇ3Ý'Û�Ç3Ç3Æ�Õ�ÔaÕ�Ö1ÍÎÅhÒrÓÎÒ'ÊÈÉ�Í3Æ�Õ�Ô�Ç�ÅhÆ�îaß�Ò?Í�ØÚÒ'Ò$ÔÍ3Å�Ò$Ç3Ò�Í�Ø1Õ[ÔhÕ¢ÍÎÆÌÕ¼Ô�Ç$á

ð�Ù

Page 64: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

»y¼4ï¼Q½ J°YX¿�¿LKB¿MK�ûT�]R:¿®Y[RÆI »y¼Q½.ð��Ti]dPQ[y��K�zOKH£XSeK�[\LOK�lHpqSOlHKRYe[�lHg�NZPQP�É ü ¢FLOTiP�lRpZSOPQ[d]dcOl|[\TUpZS6TUPyPdTUr#TUgUNZ]F[\p�[dLOK�lHpqSOPQ[d]dcOl|[\TipqSpZv ³ KRl|[\TipqSbºe� ºO�éó�sOce[F��T�[\L_Pdpqr#K�zOTi��KR]QKRSm[�YXNZ]dNZrIK|[\KH]dPR�

r�KH[sÈÞsuK1NÊgUKRSefZ[\L�¸¹YO]dKHPdKH]�Â�TUSOfH�e�jm�ø\õ� ¹¸�pqSOK�¸¹��N�hËY�KH]dr�c�[�N�[dTUpqS�Î<NZP�s�K|vxpq]dK���[\LOKoPdK|[x?zeø�Ù�{�%t��TigUg:suK�pqcO]yPdK|[�pZv@PdKHlR]QKH[�ÓZK|h�PR�ér�KH[I�ÞsuK�[\LOKG�O�kmaødõ� �¸¹YOPdKHcOzOpq]dNZSOzOpZrÑfqKHSOKR]dN�[\pZ]��LOpqPQK K|{�TUP�[\KHSOlRK TiP�fqcXN�]\NZSm[\KHKRzDsmhb¢FLOKRpq]QKRrѺO�iÙRºO�]r�KH[�õÍsuK�PdcOl�LD[dLXN�[]º § õ ¸3ñ ¿ÁÀ4 0ÈE � | 2 øNZSOzàgUKH[½f���õ� #zOKHSOpZ[dKn[\LOKngUK�N�PQ[ITUSm[\KHfqKR]IPdcOl�LÊ[\LON�[   %i 0¥| 2 ¡ *n�e�jm�ø�º_  ü JÍKb��TUgUgFcePdK6[\LeKvxpqgigUp���TUSOf�lRg�N�TUr_ùòDó 5x­ÿ ¾��k�ô� ú ¥?4r��ø�ºDø�ªZ®BCAfͪaÆDC�Ç��@®²Ç�C<ª�$|¥?ËZÇ'3~1�Ç]&eª<ËZÇ�öÙ ü �O�kmaø�º_  § õ ¸2Ü6õ 0 ¸32 üó ü f���õ�  § ß �xgUpqfO�jm�ø\õ� Q  üö þaý²ý!÷�ø ö�KRl�N�gUgX[\LXNa[�gipqf´�«ó�ø\õ�  § gipqf2gUpZf}õnNZSOzIgUpqf´�kmaødõ�  § gipqf´�jm·ôÍÙqødgUpqf2õ� @vxpq]Jm]øÊó ü J K

£X]QPQ[�SOpZ[dK�[dLXN�[gipqf´�jm�ø�º_  § gipqf´�kmaø\õ ¸3ñ ¿ÁÀ4 0�E � | 2  

§ gipqf´�km�ôÊÙqø\gipqf´��õ ¸3ñ ¿ÁÀ4 0�E � | 2  d § gipqf´�km�ôÊÙqø\gipqf2õ�Ö>gUpZf´�jm�ø\õ� Q § gipqf´�km�ô1ó�ø\gipqf´��gipqf2õ�Ö>gipqf´�jm�ø\õ� d d § gipqf´�km�ô1ó�ø\gipqf2gUpqf@õ�ô`gipqf´�jm�tÙqø\õ� d 

�¹[�vxpqgUgip���P�[\LXN�[�O�kmaø�º_  § º ¿ÁÀ4 0�E � % 2

§  dõ ¸3ñ ¿ÁÀ4 0�E � | 2 ¡ ¿ÁÀ4 0�E Ü�¼ � ¿ÁÀ4Â�¿ÁÀ4 | Ü ¿ÁÀ4 0�E�/ ¸ � | 2à2§ õ ¸2Ü6õ 0 ¸32

��LOTil�LbYO]Qp}ÂqKRP�[\LeK�£O]dPQ[�YXN�]Q[�pZv·[\LOK�lRgUNZTUr_��Xpq]�[dLOK<PQKRlRpZSOz`YXNZ]�[�pZvF[dLOK�lHg�NZTirb�:]QKRl�N�gUg@[\LXN�[;f��xõ� �TiP�[dLOK�giK�NZP�[�TiS�[dKRfqKH]�PQcOl�LØ[\LXNa[  %i 0¥| 2 ¡ *¯�e�jm�ø�º_  ü JÍK2Ye]dp}ÂqK2[\LON�[Æf���õ� Ftó2gipqfX�kmaødõ� ²smh�PQLOp���TUSOf�[\LXNa[   %¼ ¿ÁÀ4 0�E � | 2 ¡ *¯�e�jm�ø�º_  ü

ðqó

Page 65: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

¢·pIPQKRK�[\LOTiPR�XSOp�[\K�[\LON�[�s�h�[\LOK�PQ[\NZSOzXN�]dz_TUSOKHÌ�cONZgUT�[�h   � ù ¡ *è����Ö�Åe  ù øO��K�LXN�ÂqKú ºó>gipqfX�kmaø\õ� ?û §

ú õ ¸3ñ ¿ÁÀ4Â 0ÈE � | 2ó2gipqfX�kmaødõ� ?û * ú õ ¸3ñ ¿ÁÀ4Â 0�E � | 2

ó2gipqf´�kmaø\õ� �û ¼ ¿ÁÀ4 0ÈE � | 2 § õ ¼�«ó>gipqf´�jm�ø\õ� d  ¼ ¿ÁÀ4 0ÈE � | 2 § õ ¼�Ü6õ 0 ¸32 ü

³ TiSOlRKI�e�jm�ø�º_  § õ ¸2Ü6õ 0 ¸32 øO[\LOK�lHg�NZTirµTiPFYO]dp}ÂqKHz���Xpq]"à § ÙZø üSüSü ø)�e�jm�ø�º_ �giKH[U¥BÀ>zeKRSOpZ[dK�[\LeK;Ã�¸«[\L¬f��xõ� ¹¸�KRgiKRr#KRSm[�PQcOsOPQKH[�pZv2[dLOK�PdK|[;x�ºü

Ùqø ügüSü ø�ó?º�{�NZSOzMgUK|[ ð À�suK#[dLOK#lRpqS?ÁQceSOlH[dTUpqSQ§ ë� �¨¼© ��ë ü í�T�ÂqKHSØNZSmh�TiSOYOce[é��o«x?z�ø�Ù<{}|�ø:gUKH[Ã��(�� �s�K�[dLOK [\LOK PQr�NZgigUKHPQ[�TiSOzOK|{6TUS`x�Ùqø ügüSü ø��O�kmaø�º_ '{#PdcOl�LM[\LXN�[ ð À 0 ��2�TiP�P\N�[dTUP�£XKRz�smh�� ü �¹vSOp ð À·TUP�P\Na[\TUP�£XKRz_smh��n[dLOKRS�Ã��(��  § ze��Xpq]�K�NZl�LMPdKHlR]QKH[�ÓZK|h_(goÎx?zeø�Ù�{ % ø�[\LOKH]dKwK|{�TUP�[\P�N�lRpq]Q]dKHPdYupqSOzOTiSOfBlRpqSOlHKRYe[nm�+ªoàÉ ü �¹v

m�+�TUPy[\LOK [�N�]dfqK|[�lRpqSOlHKRYe[R��[\LOKHS N�SMK�{eNZrIYegUK;� TUP ¨mÆ�Ç . ¨�¯uTivP�c¸ ã}ã}ã�� % § È1&$0�E � % 2 ��(� �NZSOz�TiP¨mÆ�Ç|¯UÇ�Æ�ÆpZ[dLOKR]���TUPQKZ�@¢FLOK�lRpqSelRKRY�[]m�+�TUPFzeKH£XSOKHz6NZPvxpZgUgUp���PHù

7 m�+Z����  § zwTiv��6TiPFcOPdKHgUKHPdPR�7 m�+Z����  § �I��(� VÀ 0 ��2 øZ[\LeKyÃ\���� �¸«[\L#sOTi[>pZv¬�I��(� |øZTivB�<TUP2cePdKHvxceg�N�SOzpÃ\���� h*ËÙ ü �¹vB��TUP2cePdKHvxcegNZSezµÃ\����  § z [\LOKHS�m�+Z����  § z ü�yp��o��KFYe]dp}ÂqK[dLXN�[2ÉtLXNZP�[dLOK�Ye]dpqYuKR]�[\TUKHP>gUTiPQ[dKRz�TiS�¢FLOKHpq]dKHr*ºO��Ù�îe�(¢FLOK�£X]QPQ[2YO]QpqYuKR]Q[�h

TUP�KRNZPQhuù�N�lRpZrIYOc�[�N�[dTUpqSXN�gUgihBceS�supqcOSezOKRzMgUKRNZ]dSOKH]�l�NZSMNZl�LeTUK¢ÂZK#NBr#TUP�[�NZÓ�K s�pZcOSOz�pZv�Ùws�hYO]QKRzOTilH[\TiSOf¬zDcOSm[\Tig�Ti[�r�NZÓ�KRP#N6r#TUPQ[\NZÓZK��`�X]dpqr [\LeTUP�YupqPdT�[\T6ÂqK�K|{eNZr#YOgiK�[\LOKBcOS�supqcOSOzeKRzgUKRNZ]dSeKR]�lRNZSolRpqr#YOce[dK`(Ê�xs�hÍTUS�ÂqKH]Q[dTUSOfgÈ1&'0ÈE � % 2 ��(� d �NZSez1LeKRSOlHK_lRNZSoK|{eNZl|[\gihÍTUzOKHSm[\Tiv�hM[\LeK[�N�]dfqK|[�lRpqSOlHKRYe[R��Xpq]²[\LeK2PdKRlHpqSOz�YO]dpZY�KH]Q[�hq�aSOpZ[dK>[\LXNa[�[\LOK2lHpqSOlRKHYe[Æm¢+(l�N�S�suK2]dKHYO]dKHPdKHS�[dKRz�NZP:N ß ��gipqfX�kmaødõ� d ¹¸

zOKHlRTUPQTUpqS�gUTUP�[·pZv´gUKHSOfZ[\L�N�[>r#pqPQ[ƺq��e�jm�ø�º_  ü ��P>TiS�[\LeKlRpqr#YOce[\N�[\TipqSXNZgmPdNZr#YOgUK�lRpqr#YOgUK�{�Ti[�hcOYOYuKR]}supqcOSOzIpZv�¢FLOKHpq]dKHrÞºO�é^���N�Yupqgih�SOpqr#T�N�g�[\TirIK�N�gUfqpq]QTi[dLOr*lRNZS�giK�NZ]QS#[dLOK�£O]dPQ[Jº YXN�TU]dPpZv([\LOK�[�NZ]QfqKH[�zOKHlRTiPdTUpZSbgiTUPQ[Fvx]Qpqr÷NwPdTUSefqgUK�YupqPdT�[\T�ÂZK�K|{eNZr#YOgUK��uNZSOzb��TigUg�rINZÓ�K�Na[�r#pqPQ[�pqSeKr#TUPQ[\NZÓZK�vxpq]FKRNZl�L6pZv·[\LeK�gUNZPQ[Y�e�jm�ø�º_ YXNZTi]dP�pZv�[dLOK�zOKRlHTUPQTUpqS_gUTiPQ[�� ³ TiSOlRKI�e�jm�ø�º_  § õ ¸2Ü6õ 0 ¸32 øPdcel�L N�SDNZgUfZpq]dT�[\LOr0��TigUg:rINZÓZK�Y�pqg�h²��õ� �rITiPQ[\NZÓZKHPR��N�SOz6Ti[�vxpqgigUp���P�[dLXN�[�ÉålRNZSDs�K�gUKRNZ]dSOKHzTUSBYupqgih�SOpZrITUNZgu[dTUr#KZ��yp��¡PQcOYOYupqPdK[dLXN�[2[dLOKR]QKFTUP2N�Yupqgih�SOpqr#T�N�g�[dTUr#KN�[Q[\]QTUsOce[dK|¸¹KHk#lRTiKRSm[2gUKRNZ]dSeTUSOf�N�gUfqpq]QTi[dLOr

vxpq]2[\LeKylHpqSOlRKHYe[lHg�NZPQP�É ü ³ TiSOlRK�K�NZl�LIlRpqSelRKRY�[+m�+�LXNZP�NZSpº�¸¹sOTi[}zOKHPdlH]dTUY�[\TUpZS6��[\LOK�PQ[d]dTUSefG(� |���K�LXN�ÂZK�[\LON�[s½ § ß ��º_  ü Ï@NZl�L_vxcOSOlH[dTUpqSµm�+�zOKRYuKRSezOPypZSOgih�pZS_[dLOKNÂZN�]dT�N�sOgUKHPh�c¸|ø üSüSü ø���¼ % øPdp^âITUP�NZgiPdp ß �kº_  ü qyKHSOlRK#NZSmhDNa[d[\]QTUsOc�[\K|¸¹KHk#lRTiKRSm[�giK�NZ]QSOTUSOf<NZgUfqpZ]dTi[dLOr vxpq]�ɵr�cOPQ[�LON}ÂZKr#TUPQ[\NZÓZK�supqcOSezbYupqgih²��ºDø\gUpqf2õ�  § Yupqgih²�kº_  ü

ðZ^

Page 66: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

V�pqSOPQTUzOKH]�[\LOKP�e�jm�ø�º_ �¸¹gUpqSOf�PdKHÌ�ceKRSOlHKr^�pZvecePdKHvxcegqK|{eNZr#YOgUKHP�x�È1&$0�E � % 2 ��(� �Ä�¥ À(Ä�z�| Ü�¼ %w{ &$0�E � % 2À@ô ¸ ü�X]dpqr0¢FLOKRpq]QKRr÷ºO��Ù�ºO�O��K�LON}ÂZK�[dLXN�[�SOp�Yupqg�h����O�kmaø�º_ d �¸«[\TirIKygUK�N�]dSOTiSOf#NZgUfqpZ]dTi[dLOrål�NZSbYO]dK�¸zOTilH[2N�S�cOSOPQKRKRSwsOT�[>pZv!�I��(� ���Ti[\L�NZlRlHcO]\NZl|h�PQTUfqSeTi£XlRNZSm[\gih�suKH[Q[\KH]�[\LXN�S�Ù�Öqó ü ³ TiSOlRK�e�jm�ø�º_  §õ ¸2Ü6õ 0 ¸32 ø���K�LON}ÂZK�[\LXN�[Y�pqg�h²���e�jm�ø�º_ d  § Y�pZgih²��õ�  ü V�pZSOPdKHÌ�ceKRSm[\g�hq�uNZSmh<Yupqgih²�xõ� �¸«[\TirIK�gUKRNZ]dS�¸TUSefoNZgUfqpZ]dTi[dLOr ��TUgUgyLXN�ÂqKMYO]dpZsXNZsOTigUTi[�htÙ�ÖZó`p�v�rINZÓ�TUSOfoN`r#TUP�[�NZÓ�KDpqSâK�NZl�LWK|{eNZr#YOgUKDTiS[\LeKbPQKRÌ�cOKRSelRK_^Î�Ti[ vxpqgigUp���P [\LXNa[#��T�[\L¯ÂqKH]Qh1LOTUfZLjYO]QpqsXNZseTUgUT�[�hq�}NZSmhoYupqgih²��õ� �¸«[\TUr#K�NZgifqp�¸]dT�[\LOr ��TigUg:r�N�ÓZKAdw���O�kmaø�º_ Q �r#TUP�[�NZÓ�KRP�pqS�^ ü Ü�ce[y[\LOTiP�r#K�NZSOP�[\LXNa[�SOp<Yupqg�h�SepqrITUNZg�[\TirIKN�[Q[\]dTisOce[dK|¸�K|k#lRTUKHSm[FgUKRNZ]dSOTiSOf�N�gUfqpq]QTi[dLOr l�N�S�K|{�TUP�[Fvxpq]�[dLOK�lRpZSOlRKHYe[�lRgUNZPdPFÉ�ø�PdTUSelRK�Yupqgih²�kº_ §�ý ���e�jm�ø�º_ d  ü �x¢FLOKRpZ]dKRr ºe�iÙ�îq 

»y¼4ï¼2G JsO ÃPM]ZOÁ'þ�ÁSOQÁ¹À�X+¿MK<À\�]R�HsY\XyKyÀO¿®Þ®YXÃPKN��ÁS�Ù�ÎZýZ\MNILKyÀOÁ�¿�Är�KH[]ÿ | s�K�[dLOK�PdK|[�pZv(NZgUguõu¸¹sOTi[�YO]dTirIKHP���LOTil�LbN�]dK�lRpZSOfq]dceKRSm[F[\pw^�r#p�z�ºe�2��S�õF/S$R¬�©���¯±¨�¦¬�®X©�Ç =�Ç042TUP�NZS�TiS�[dKRfqKH]�p�v�[\LOK�vxpq]Qr�� § #͸Bãj#[¼}��LOKR]QKJ#͸N¦§ #[¼NZSOzn# ¸�ø #[¼%oqÿ | ñ�¼ ü í�T�ÂZKRS�NZSõu¸¹sOTi[�Ü�gicOrÞTUSm[\KHfqKR]��_ø�giKH[{��1zOKRSOp�[\K�[dLOK�PdK|[FpZv:Ì�cXNZze]\N�[dTUl�]QKRPdTizOcOKHPrIp�zOcOgip��_øONZSez�gUKH[È��'ù5{��ÎÊ {��ts�K�[\LOK�vxceSOlH[dTUpqS¬È��� ð   § ð ¼ r#p�z� ü Ü�gUcOrìNZSeznJâTUgigUT�N�rIPFLXN�ÂZK SOpZ[\KHz[\LON�[JÈ��ÍTiP@N�Y�KH]dr�c�[�N�[dTUpqS pqS�{��Ê��PQKRK%r�KRr#r�N�ó��é^e�éóZÚ�pZv>�«í�pqgize�N�PdPdKH]@NZSOz#Ü�KHgUg�N�]dKZ�eÙ�ÚqÚqîm vxpq]�N YO]Qp�pZv\ |���P�zOTiPdlRcePdPdKHzÍTiS ³ KRl|[\TUpZSÍó�� ºO� ^<pZv��«í�pqgUze]dKRTil�L��>Ù�ÚZÚmðq |�²Ti[�TiP���TUzOKHgih6suKRgiTUK0ÂqKRzD[dLXN�[�[\LeK

lRpZgUgUKHlH[dTUpqSx�È��+{�� �� %�� x�È��¡ù�� TiPyNwÜ�gicOr0TiSm[\KRfZKR]¢{

LXNZPF[dLOK�vxpqgUgip���TiSOf�YO]QpqYuKR]Q[dTUKHPRùÙq�yí�T�ÂZKRSÍõ(ø:Ti[�TUP�lRpqr#YOce[\N�[\TipqSXNZgigihbK�NZP�h [dpbceSOTivxpq]QrIg�h6PQKRgUKHlH[�Nb]\N�SOzOpqr õ´¸�sOT�[�Ü�gUcOrTiS�[dKRfqKH]��ç����Ti[\L�SeKRfqgiTUfqTisOgUK�KR]Q]dpq]�YO]dpZsXNZsOTigUTi[�he 2smh�[�N�Ó�TiSOf�� § #͸cãS#\¼�ø���LOKH]dK+#͸�øj#[¼NZ]QKyceSOTivxpq]QrIg�h�PQKRgiKRlH[dKRz<õ�Öqó�¸¹sOTi[@YO]QTUr#KRP2��Ti[dLn#͸U©°#\¼NZSOz�#͸ � #\¼ � ^�rIp�z#º<��[\LOTiPNZPQPdcOr#KRP�[\LXNa[�[dLOKIPQKH[bÿ | ñ�¼�pZvFPdcel�L`YO]dTirIKHP�TiP�SOpqS�¸¹SOKRfqgiTUfqTisOgihbzOKHSOPdK<TiSM[dLOK�PQKH[�pZvõ�Öqó�¸�seTi[FTiS�[dKRfqKH]dP\ |�

ó��yí�T�ÂZKRSwNZS õ´¸�sOT�[2Ü�gicOrèTiSm[\KRfZKR]��_øZTi[�TUP>KRNZPQh�[\p�cOSeTivxpq]QrIg�h�PdKRgiKRl|[}N�Ì�cONZzO]\Na[\TUl�]dKHPdTUzecOKâ�r#p�z�� ��[\LOTiPFl�NZS_s�K�zepqSOK�smhBPdÌ�cXNZ]QTUSOf#Nw]\NZSOzepqrIg�h�l�LepqPdKHSnKRgiKRr#KRSm[�pZv��"É�  |�

^e���Xpq]}K¢ÂZKR]Qh�Pdcek#lHTUKRSm[dgih#g�NZ]QfqK�õ�ø�vxpq]�K0ÂqKR]�h�Ye]dpqsXN�sOTUgiTUPQ[dTUlFYupqgih²�xõ� �¸«[\TirIK�NZgifqpq]dT�[\LOrèñ���ø

ð�º

Page 67: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

vxpq]FK0ÂqKH]QhBYupqgih�SOpqr#T�N�gÚ{wøXfqT�ÂZKRS�� NZSOz�â�PdKHgUKRl|[\KHznNZP�zOKHPdlH]dTUsuKRznNZsup�ÂZKZ�O��K�LON}ÂZKÅ2]�¤ ñ � � È����âq �ø��6  § â�¨-© Ù{I��õ�  ü

��P(TUS��«í�pqgUze]dKRTil�L���Ù�ÚqÚmðq ��a��KPdN�h�[dLXN�[®x�È��+{�� �� % TUP(Nw¤�¥a¯�¯UÇ�¤|©�¬«¥a®�¥ . ¥a®²Ç0/21}ªZ°}§XÇ04�¦ ¨�©¹ªZ©�¬«¥a®eÆRÐ�yp��'lHpqSOPQTUzOKH]�[dLOK�vxpqgUgip���TUSOfDÂ�NZ]QT�NZSm[�pZv�Å2]dpqYuKR]�[�hB^eù^�� ü �Xpq]FK0ÂqKR]�hnPQcek#lRTiKRSm[\g�hBg�NZ]QfqK�õ�øOvxpq]�K0ÂqKR]�h_YO]dpqsONZsOTUgiTUP�[\TUl�Y�pqg�h²���e��õ� Q �¸«[\TUr#K�N�gUfqpq]QTi[dLOrñ � ø�vxpq]@K¢ÂqKH]QhIY�pZgih�SOpqr#T�NZg5{#ø�fqT�ÂZKRS��+NZSOzpâ�PdKRgiKRl|[\KRz<NZP�zeKRPdlH]dTis�KHzBNZs�p}ÂZKZ����K�LON}ÂZK

Å2]�¤ ñ � �jÈ��y�kâZ |ø��n  § â?¨�© Ù{����O�xõ� Q  ü�¹vÍx�È�w{�� �� % P\N�[dTUPQ£OKRP(Å@]QpqYuKR]Q[dTUKHP}ÙZ�Zó�NZSOz�^���[dLOKRS���K}P\N�h�[dLXN�[Px�È��+{�� �� % TiP(N�¤�¥a¯�¯UÇ�¤|©�¬«¥a®¥ . �e��õ� �/d¥a®²Ç0/21}ªZ°�§XÇ04�¦ ¨�©«ªZ©­¬«¥a®eÆRÐöyN�sOTUS�êëP�]dKRPQcOgi[dP�TiSØ��öyNZseTUS��:Ù�Ú�u�Úm �h�TiKRgizb[\LeK�vxpqgigUp���TUSOfeù ³ cOYOYupqPdK�[\LXN�[yñ � TUPyNIYO]QpqsXNa¸

sOTigUTUP�[\Til2Y�pqg�h²���e��õ� Q �¸«[\TUr#K2NZgifqpq]QTi[\Ler���LOTUl�L²�Z��LOKRS�fqT�ÂZKRS NZP(TUSOYece[([dLOK}YXNZTU]�� È����âq �ø��6 ���Ti[dL� NZSOzAâyPdKHgUKRl|[\KHz�NZP�zOKRPQlR]dTis�KHz#NZs�p}ÂZKZ��pqce[dYOce[\P~â���Ti[dL YO]dpqsONZsOTUgiTi[�h�N�[>gUK�N�PQ[FÙ�ÖaY�pqg�h²���e��õ� Q  ü¢FLOKHSb[\LeKR]dK�TiP�NwYupqg�h����O�xõ� Q �¸¹[dTUr#K�N�gUfqpq]QTi[dLOrµñ'��LeTUl�L��O��LOKHSnfqT6ÂqKHSbNwcOSOT�vxpq]dr#gih#PQKRgUKHlH[dKRzõu¸¹sOTi[FÜ�gUcOr0TUSm[\KHfqKR]��_øOv�NZlH[dpq]dP��æ��Ti[dLbPQcOlRlHKRPQP�YO]QpqsXNZsOTigUT�[�h�Na[ygiK�NZP�[�Ù�Ö�Yupqgih²���O�xõ� d  ü¢FL�cOPR��Tiv-x�È�w{�� �� % TUP(SOpZ[@NylRpqgigUKRl|[\TipqS�pZv!�e�jm�ø\õ� ¹¸�pqSeK|¸¹��N�h�YuKR]Qr�ce[\N�[\TipqSOPR�a[\LOKHS [\LOKH]dKTiPNR�O�kmaø\õ� �¸«[\TUr#K�NZgifqpq]QTi[\Lerµ��LOTil�Lbv�NZl|[\pq]QP�]dNZSOzOpZrIg�h�PdKHgUKRl|[\KHz6Ü�gicOr TUSm[dKRfqKH]dP���Ti[\L_PdcelRlRKHPdP

YO]QpqsXNZsOTigUT�[�h1N�[<giK�NZP�[6Ù�Ö�Yupqg�h����O�kmaødõ� d  ü ¢FLOTUP���pqcOgiz¡suK6Ì�cOTi[dK6NØPdce]dYO]QTUPdTiSOfØ]QKRPdcegi[���PQTUSOlHK[\LeK�v�NZPQ[dKRP�[�Ó�SOp���S N�gUfqpq]QTi[dLOrIPFvxpq]�v�NZl|[\pq]QTUSOfIõu¸�seTi[�Ü�gUcOr÷TUSm[dKRfqKH]dP�]QKRÌ�cOTU]QK�[dTUr#K�ó��50¥| � �"! 2 ø��LOTil�L TUP>N�r�cOl�L�v�NZPQ[dKR] ¸�fq]Qp���TiSOf�vxcOSelH[\TipqS�[dLXNZS£�e�jm�ø\õ� �vxpq]>NZgigým]*tó���]dKHl�NZgig�[dLXN�[��e�«ó�ø\õ�  §õ ¿ÁÀ4Â�¿ÁÀ4 |�øW�O��^eø\õ�  § õ ¿ÁÀ4Â�¿ÁÀ4Â�¿ÁÀ4 |eøXK|[\lZ�é |�

ðqð

Page 68: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

# $ -&% ')(./ *

+ , # , (.-0/1) -2) 3 . -/' $ + (./102(3% 'Í/54+)

- ) 6 -2) ) 47.

��S�[\LOK�gUNZPQ[l�LXNZYe[dKR]��K�cePdKRzB[\p�pqgiP�vx]dpZrµlH]Qh�Ye[\pZfq]\NZYeL�h�[\p pqse[\NZTUS�gip���KH]s�pZcOSOzOP�pqS�lHpqrw¸YOce[\N�[\TipqSXNZgXPdNZr#YOgUK�lRpZrIYOgiK|{�Ti[�h���LOTUl�LILOpqgUzwvxpq]}NZSmhwgUKRNZ]dSeTUSOf�NZgifqpq]dT�[\LOr_�:JÍK�SOp��Êl�LXNZSefqKpqcO]�vxp�lRceP�NZSOzD[�NZÓ�K#NBlHgUpqPQK gUp�pqÓbN�[�[\LOKwN�sOTUgiTi[\TiKRPyNZSOz�gUTirIT�[�N�[dTUpqSeP�p�v�[���pBYXNZ]Q[dTUlHcOg�NZ]�NZg�¸fqpq]QTi[dLOrIP�vxpq]�pqSOKBpZv�[\LeK�rIpqP�[ vxcOSOzXN�rIKHS�[\NZgYO]QpqsOgiKRr#P TUSor�N�l�LOTUSOK�gUK�N�]dSOTiSOfOù<giK�NZ]QSOTUSOfNZS�cOSeÓ�Sep���S_gUTiSOK�N�]�[\Le]dKRPQLOpqgiz<vxcOSOlH[dTUpqS<vx]Qpqråg�NZsuKRgiKRz<K�{ON�rIYOgiKRPH�2¢FLOK�£X]dP�[�NZgUfZpq]dT�[\LOr ��KlRpZSOPdTizOKR]H��[\LOKwÅ·KR]dlHKRYe[d]dpqS NZgifqpq]dT�[\LOr_�´TUP�YuKR]QLXNZYOP�[dLOK r#pqPQ[���TizOKRg�hnÓ�SOp���SMNZgifqpq]dT�[\LOr÷TiSrINZl�LOTUSeKIgiK�NZ]QSOTUSOfe��¢FLOKIPdKRlHpqSOzoNZgifqpq]QTi[\LerÑ��K�lRpZSOPdTizOKR]�TUPnr�Ti[d[dgUKHPQ[\pZSOKZêëP�JâTiSOSOp��ÞNZgifqp�¸]dT�[\LOr_�ONDÂ�NZ]QT�NZSm[Fp�v([dLOK�Å(KH]dlRKHYe[\]QpqS6N�gUfqpq]QTi[dLOrã��LOTUl�LBLXNZP�]QKRlRKHT�ÂZKRz_TUSm[\KHSOPdT6ÂqK�PQ[dcOzeh�p}ÂqKH][\LeK�YXN�PQ[�zepqáRKHSnpq]�Pdp#hqK�N�]dPR�J K�fZT�ÂqK�supZ[\L_Y�pZPdTi[dT�ÂZK�N�SOznSOKHfmN�[dT�ÂqK�]dKHPdcOg�[\P�vxpq]F[\LOKHPdK�[���pNZgifqpq]dT�[\LOr#P�TiS�[\LOK�Å���VâgUKRNZ]dSOTiSOf r#p�zOKHg«�

²�5�7 8:9<;>=@?BADCDEF;2GH?I9¢FLOK<lHg�NZPQPdTil�NZg}Å(KH]dlHKRYe[d]dpqS1NZgifqpq]dT�[\LOrç�­Ü�gip�l�Ó´��Ù�Úqîmó�η�yp}Â�TUÓ�pZ�(�Ù�Úqîmó�Î(ö�pqPQKRS�sOg�Na[d[���Ù�Úqîmóq NZSOzTr:T�[d[dgUKRP�[\pqSeKZêëP�JâTiSOSOp��ÞNZgifqpq]dT�[\LOrç�jr�Ti[Q[\gUKHPQ[dpqSOKZ�}Ù�ÚqÛZÛeÎõr:T�[d[dgUKRP�[\pqSeKZ�}Ù�ÚqÛqÚZsX �NZ]QK#[���pNZgifqpq]dT�[\LOr#P�vxpq]�giK�NZ]QSOTUSef�gUTiSOK�N�]·[dLO]dKHPdLOpqgiz�vxcOSOl|[\TUpZSOP���LOTil�L LXN�ÂqKFsuKRKRS P�[\cOzeTUKRz K�{�[\KRSePdT�ÂZKRg�hTUS�[\LOK�pqSOgiTUSOK}rITiPQ[\NZÓZK}s�pZcOSOz�r#p�zeKRg�TUS�]dKRlHKRSm[>hZK�NZ]QPRÎ�PdKHKKZ�ëfO�F�x��cOKH]>NZSez�JØN�]dr�ce[dL��eÙ�ÚqÚmð�ÎÜ�h�g�N�SOzOKR]H�(Ù�ÚqÚqÛqNeδ�X]QKRcOSOznNZSOz ³ l�LXNZYOTi]dKZ�:Ù�ÚZÚqÛeÎuï�T�Â�TUSOKHSoÇ|©FªZ¯�ÐU�·Ù�ÚqÚ�u�Î�r�Ti[Q[\giKRPQ[dpqSOK��·Ù�ÚqÚeÙqÎfnNqN�PdP�NZSOz JÍNZ]Qr�ce[\L²��Ù�ÚZÚqÛeÎ�Ò}NZgiT�NZSm[��2Ù�ÚqÚqÚm ���~�KHPdYOT�[\Kw[\LOTiP���TizOKRPQYO]dKRNZzØTiS�[dKR]QKRPQ[�TUS�[\LeKÅ(KH]dlHKRYe[d]dpqSwNZSez�JâTUSeSOp��oNZgUfZpq]dT�[\LOr#PR��]QKRg�Na[\T�ÂZKRg�h�gUT�[d[dgUK2TUP:Ó�SOp���SwNZsupqce[:[\LeKRTU]:YuKR]�vxpq]drINZSOlHK

ðZî

Page 69: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

TUSbÒ}NZgUTUNZSm[�êëP�Åu��VËgiK�NZ]QSOTUSef�r#p�zOKHg«����Sn[\LeTUP�l�LONZYe[\KH]���K KRPQ[\NZsOgiTUPdLnY�pqPQTi[dT�ÂqK�NZSOzDSeKRfmN�[dT�ÂZK]dKHPdcOg�[\P<pZS¡[\LOK�Åu��V÷giK�NZ]QSOTUSOfÍNZsOTigUTi[dTUKHP#p�v�JâTiSOSOp��ÑNZSOzâÅ·KR]dlHKRYe[d]dpqS²�y[\L�cOP�LeKRgUYeTUSOfÍ[dplRgUNZ]dT�v�hI[dLOKRTi]FPQ[\N�[\cOP�TiSB[dLOK�Å���VWr#p�zOKRg­�

ï¼Q½}¼Q½ ûT�]RpJ�R�Y[�-RõKyÀýYX¿�Ä Ã@Ä]à98 Á�Ä�Ä�¿;: ��OVÞ}¿JY´Á¹À[�NI Z

Ü�pZ[dL�Å·KR]QlRKRY�[\]dpZSBNZSOzwJâTUSOSOp��âN�]dK�pqSOgUTiSOKFr#TUPQ[\NZÓZK�supqcOSOzIgUKRNZ]dSOTiSOf�NZgUfZpq]dT�[\LOr#PRÎZ��K�]QKHvxKH][\LeK�]QK�NZzeKR]F[\p ³ KHlH[\TipqSnó�� º vxpq]�N zOKHPdlH]dTUY�[\TUpZS_p�v([dLOTUP�r#p�zOKHg«�ú�û:ü ö ü�þ?�qü=<õ��þaý!w�> ó@? ý´þ�x���û·ÿ¢FLO]QpqcOfqLOpZce[2Ti[dP�K�{eKHlRce[dTUpqS [dLOKFÅ(KH]dlHKRYe[d]dpqSINZgUfZpq]dT�[\LOr'rINZTiS�[\NZTUSeP�N���KRTifqLm[�ÂqKHlH[\pZ]æ� oBAw|NZSOzMN�[dLO]dKHPdLOpqgizDCµoEA*NZP�T�[\P�lHcO]d]QKRSm[�Lmh�YupZ[dLOKRPQTUPR����SOTi[dT�NZgigih�[\LeKwÅ(KR]QlRKHYe[\]QpqSÍN�gUfqpq]QTi[dLOrPQ[\NZ]Q[dP���T�[\L�� § �jzeø ügüSü ø'zq �N�SOz5C § z ü tyYupqS ]dKRlHKRT6Â�TiSOfbNZSÍK�{ON�rIYOgiKA�:ø·[dLOK�N�gUfqpq]QTi[dLOrYO]QKRzOTilH[\P�NZlRlHpq]dzeTUSOfM[\pÍ[dLOKngiTUSOKRNZ] [\LO]QKRPdLepqgUzàvxceSOlH[dTUpqSO�üã�ü*FC ü �¹v�[\LOKbYO]dKHzOTUl|[\TUpZSjTiPTUSelRpq]Q]dKRl|[F[\LOK�Lmh�Y�p�[\LOKHPdTUP�TUPFceY�zXNa[\KRzbNZlHlRpq]QzOTUSOf [dp#[dLOK�vxpqgUgip���TiSOf�]QcOgUK�ù

7 �¹v�[\LOK@YO]QKRzOTilH[\TipqS�TUP2Ù�NZSOz�[\LOK@gUNZs�KHgqTUP@ô�Ù���v�NZgUPQK2Y�pZPdTi[dT�ÂZK2YO]dKHzOTUl|[\TUpZS´ |��PdK|[.�2G � ô"�NZSezC § CYâÙ ü

7 �¹v}[\LOKwYO]QKRzOTilH[dTUpqS�TUP ô�ÙINZSOz�[\LOKwg�N�s�KHg�TUP Ùn�xv�N�gUPdK SeKRfmN�[dT�ÂZK#Ye]dKRzeTUlH[dTUpqSX |��PQKH[G�HG��`�DNZSezC § C�ô¡Ù ü�ypØl�LONZSOfqKbTUP#r�NZzeKnTivy[\LOKbLmh�YupZ[dLOKRPQTUP���NZP�lHpq]d]QKRl|[�pqS � ü �¹[<TiPIlHgUKRNZ]#[dLXN�[�Tiv�KRNZl�L

K|{eNZr#YOgiK%��suKRgipqSOfqP@[\pªx?zeø�Ù<{ | [dLOKRSIK�NZl�Lj�hÀ�NZSezIC���TigUg´NZg��N�h�P}s�KyTUSm[\KHfqKR]QPRÎ�[dLOTUP2v�NZl|[���TigUgYO]Qp�ÂZK�cePdKHvxceg�g�N�[dKR]R�¢FLOK ��KHgUg ¸�Ó�SOp���SÍÅ·KR]QlRKRY�[\]dpZSØV�pqS�ÂZKR]dfZKRSOlHK�¢FLOKHpq]dKHr fqT�ÂZKRP�NZS�cOYOYuKR]�supqcOSOz�pqS�[\LeK

S�cOr�suKR]�pZv(rITiPQ[\NZÓZKHP���LeTUl�LB[\LOK�Å(KH]dlHKRYe[d]dpqSnNZgUfqpZ]dTi[dLOrÞl�NZS_rINZÓZK��ú�û:ü�ý´þaü�ÿ c�����KJ ó ýc�L�� ���X�B�NMPObý6��x"Luý�Qw�Í��X���NM=R<ý!�Rü6wÚC ó ý�����c���X�B� �h! Ç|©Í�k� ¸ ø�Å)¸��Hø ügüSü ø�k� % ø�Å % �½$|Ç<ªnÆ�ÇKSR¨´Ç|®²¤�ÇB¥ . ¯�ª�$|Ç|¯UÇ�C ÇQÈ�ªZ¦�§u¯UÇ�ƪ12¬�©�&�T'� À T,F&U�ªZ®BC�Å�À"oÙx�ô�Ùqø�Ù�{ . ¥?4�ªZ¯�¯Ã ü ! Ç|©WV $|Ç<ª¬ËZÇ�¤|©�¥?4�ªZ®BCXC�øKY $|ÇwÆ|¨´¤'&Í©(&eªZ©hÅ�À��@V�ã�� À ôZCZ ª*[Y . ¥?4<ªZ¯�¯õÃdøh1F&XÇ04\Ç�YTøLz üñ &XÇ|®B©�&XÇ�©�¥a©¹ªZ¯u®X¨�¦ª$|Ç04�¥ . ¦ ¬UÆ|©«ª?ámÇ�Æy¦#ª�C�Ç]$R°�©�&XÇ%8�Ç04\¤�Ç�§u©j4\¥a®nªZ¯>=�¥?4|¬�©(&�¦ ¥a®B©�&�¬UÆ�ÇQÈ�ªZ¦�§u¯UÇÆ�ÇKSR¨XÇ|®²¤�Ç�Æ�¬UÆ�ªZ©�¦�¥�Æ|© �@U ¼ tÙ� H�\T\V�T ¼ �C ¼  

Y ¼ üð�u

Page 70: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

ú�û:ü] x�w-w:ý_^9> ó@? ý´þ�x���û(ÿ¢FLOK JâTUSeSOp�� NZgifqpq]QTi[\LerìTiP�KRPQPdKRSm[dT�NZgigihDN<r#p�zOTi£XKHz�Å(KH]dlRKHYe[\]QpqSØN�gUfqpq]QTi[dLOr÷TUS6��LOTil�L cOY�¸zXN�[dKRP�NZ]dKyY�KH]Qvxpq]QrIKHz�r�cOg�[\TiYOgUTil�N�[dT�ÂqKHgih�]\N�[dLOKR]}[\LXN�S�NZzOzOT�[\T6ÂqKRg�h pqS<KRNZl�L�lRp�pq]QzOTUSXNa[\K�pZv�[\LeKLmh�Y�pZ[dLOKRPQTUPéÂqKHlH[\pZ]R�#��P�zOKRPQlR]dTis�KHz TiS¡�kr�Ti[d[dgUKHPQ[\pZSOKZ�2Ù�ÚZÚeÙ� y[\LeKwJâTiSOSOp��ãN�gUfqpq]QTi[dLOrì[\NZÓZKHPNZPTiSOYOce[FNZS�TUSOT�[\TUNZg\ÂqKHlH[\pZ]r�a`%oBAw|��XN�YO]Qpqr#pZ[\TipqSIv�NZl|[\pq]UíTobA��eNZSOzBN�[dLO]dKHPdLOpqgizXCAoBA ü¢FLOK�NZgUfqpZ]dTi[dLOr*]QKRÌ�cOTi]dKRP}[\LXNa[�[dLOK%ÂZKRl|[\pq]M�a`suKyYupqPQTi[\T6ÂqK���T­� K��>K�N�l�L<lRp�pq]dzeTUSXN�[dKY�a`À TiP}YupqP ¸Ti[dT�ÂZK� |�X[dLXN�["í ø Ùq��NZSOzb[dLXN�[aC�ø¹ze�%r�TUÓZK�[\LeK�Å(KH]dlHKRYe[d]dpqSDN�gUfqpq]QTi[dLOrb��JâTUSOSOp��ËYe]dKRzeTUlH[dPTUSwKRNZl�L�P�[�NZfqKyNZlHlRpq]QzOTUSOf�[dp�[dLOKF[\LO]QKRPdLepqgUzwvxcOSOl|[\TUpZSb�¬ã2��*ZC ü �¹v²[dLOKyYe]dKRzeTUlH[dTUpqSwTiP2lRpq]Q]dKRl|[[\LeKRS_SOpIceY�zXNa[\K�TUPFYuKR]�vxpq]dr#KRz��epZ[\LOKH]Q��TiPdK�[\LOK���KRTifqLm[\P�NZ]dK�cOYuzXN�[dKRzbNZPFvxpqgUgip���PRù

7 ¶�S_Nwv�N�gUPdK�YupqPQTi[\T6ÂqK�YO]QKRzOTilH[dTUpqS���vxpq]�NZgigcÃ@PdKH[d�hÀcG í Ü)�¢© �hÀ¹Î7 ¶�S_Nwv�N�gUPdK�SeKRfmN�[dT�ÂZK�Ye]dKRzeTUlH[dTUpqS²��vxpq]�NZgigcÃ@PdK|[s�hÀdG í �¢© �hÀ«��¹[}PdLOpqcegUz#s�KFSOp�[\KRzw[\LON�[}TUS [\LOTiP>vxpq]dr.JâTUSOSep��àTUP2pqSegih�lRNZYXNZsegUK�pZv�K|{�YO]dKHPdPQTUSOf�Y�pqPQTi[dT�ÂqK

[\Le]dKRPQLOpqgiz�vxcOSOl|[\TUpZSOPNZP�Ti[dP�Lmh�Y�pZ[dLOKRPQKRPH���yPzOKRPQlR]QTUsuKRz�TUS6�jr�Ti[Q[\giKRPQ[dpqSOK���Ù�ÚZÛqÛeÎ�r�Ti[d[dgUKHPQ[\pZSOKZ�Ù�ÚqÛZÚZs´ ·[dLOTUP>giTUr#Ti[\N�[\TipqS�l�NZSwsuKFK�NZPQTUgih�p}ÂqKH]dlRpZrIK�smh�cOPQTUSOf]Â�NZ]QTUpqceP>PdTirIYOgiK�[\]dNZSOP�vxpq]drIN�[\TipqSOPpqS�[\LeK�TiSOYOce[�K�{ON�rIYOgiKRPH�

r�Ti[d[dgUKHPQ[\pZSOK LXNZP�YO]Qp�ÂZKRzÍ[dLOK vxpqgUgip���TUSOf<]QKRPQcOgi[R�(N�SXNZgUpZfqpqcOP�[dpB[dLOK#Å(KH]dlRKHYe[\]QpqSØlHpqS�ÂqKR] ¸fqKHSOlRK�[\LOKHpq]dKHrb�es�pqceSOzOTUSefw[dLOK�S�cOr�suKR]�pZv(rITiPQ[�N�ÓZKRPF��LOTil�L_JâTiSOSOp��Wr�NZÓ�KRPHùú�û:ü�ý´þaü�ÿ c��Ø��KeMx(��� ó ü6�¢��ý!w:üB� �������ý�J! Ç|©c�k� ¸ ø�Å)¸S�Hø ügüSü ø����W%yø�Å % �h$|Ç�ª�Æ�ÇKSR¨XÇ|®²¤�Ç¥ . ¯�ª�$|Ç|¯UÇ�CÇQÈ�ªZ¦�§u¯UÇ�ÆP12¬�©(&%� À o`¤>zeøRÙ�¨�|�ªZ®BChÅ�À�osx�ô�Ùqø�Ù�{ . ¥?4ªZ¯�¯}à ü ! Ç|©fVµ$|Ǫ"ËZÇ�¤|© ¥?4�ªZ®BCh­;øùze$|Ç2Æ|¨X¤'&©(&eªZ©U1F&XÇ|®²Ç0ËZÇ04+Å�À § Ù�1�ÇN&eª<ËZÇgV�ã���* ÙBªZ®BC�1F&XÇ|®²Ç0ËZÇ04+Å?À § ô�Ù�1�ÇN&eª<ËZÇ�V�ã}��F Ù�ô¯­ ü,�.�� ¬�®X®²¥?1µ¬UÆ�4�¨�®Ë¥a®â©�&�¬UÆMÇQÈ�ªZ¦�§u¯UÇBÆ�ÇKSR¨´Ç|®²¤�ǵ12¬�©(&¡¬�®X¬�©�¬�ªZ¯2§eª<4dªZ¦�Ç|©�Ç04\Æb�a`�ø'í}ø�Cù1F&XÇ04\ÇÙ�© í © ¸¸2ÜPh ø©(&XÇ|®o©�&XÇI©�¥a©¹ªZ¯@®X¨�¦ª$|Ç04B¥ . ¦ ¬UÆ|©«ª?ámÇ�Æ#¦#ª�C�Ç�$R° � ¬�®X®²¥?1*¥a®1©(&�¬UÆBÇQÈ�ªZ¦�§u¯UÇÆ�ÇKSR¨XÇ|®²¤�Ç�¬UÆ�ªZ©�¦�¥�Æ|©

i 0 ¼�ÜPh#2 ¿kj ii Ü[¸ � |À@ô ¸ �ml©n ��kí,tÙ�  � |À@ô ¸ V[À�gUS �¼© no 0 ¸2ÜPhÎ2Ü�fl©��ÙFô¯í� Ù�ô`­Z Q egUSwí ü

,�. í § ÙP h¼ ªZ®BCG� ` § � Ùqø üSüSü ø�Ù� �©(&XÇ|®6©�&XÇ�®X¨�¦ª$|Ç04�¥ . ¦ ¬UÆ|©«ª?ámÇ�Æ�¦#ª�C�Ç�$R° � ¬�®X®²¥?1¡¬UÆ�ªZ©¦�¥�Æ|© ÛZõ

­ ¼ C or�Na{Bx?zeø Ù�º­ ¼ |pÀ ô ¸ V[À�gUS��"V[ÀqCq '{ ü

ðZÛ

Page 71: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

ï¼Q½}¼2G Jª�ÎH reR:Ã~Y´Ä�Á�Ä]Þs:àÁ�À\� JµR�Y[�-RÆKyÀýYX¿�ÄÞÃ@Ä]àH8 Á�Ä�Ä�¿;:utgJgY[R�ÂyÁ�¿JM]Z8 ¿®Y�v

qyKH]dK���KysO]dTiK¼»Oh�PdcOr#r�N�]dTUáHK�]dKRgiK¢Â�NZSm[}YO]dK0Â�TUpqcOP}]dKHPdKRNZ]dl�L<pZS�Åu��VàgUK�N�]dSOTiSOf�cOPdTiSOf�JâTiSOSOp��NZSOz1Å·KR]dlHKRYe[d]dpqS²�jö�KRlRNZgUg�[\LXN�[wTiSØ[\LOKBÅu��VµgUKRNZ]dSeTUSOf6r#p�zOKRg�[dLOKR]QK�TUPwN6£e{�KRz1cOSOÓ�SOp���SzOTiPQ[\]QTUsOc�[\TUpZSp�ãvx]dpZrå��LOTil�L�g�NZsuKRgiKRzIK|{eNZr#YOgUKHPFNZ]dK�zO]\N���S��XN�SOz�[\LeK�fqpmNZgupZv�[\LeK�gUKRNZ]dSOKH]�TiP[\py£XSOz N�Lmh�YupZ[\LOKHPdTiP���LOTUl�L lHgUpqPQKRg�h�N�YOYO]dp�{�TUrIN�[dKRP:[\LOK�[\NZ]dfZKH[·vxcOSOlH[dTUpqS�cOSOzOKH]>zOTUP�[\]QTUsOce[dTUpqS� ü��PFzOKHPdlH]dTUsuKRz�TiSBV�LXNZY�[\KR]Fó��efqKHSOKR]QTUl�[dKRl�LOSeTUÌ�cOKRPFK�{�TUPQ[�[\p�lHpqS�ÂqKR]�[yNZSmh�pqSegUTUSeKyr#TUP�[�NZÓ�K

supqcOSOzDgiK�NZ]QSOTUSOf�NZgUfZpq]dT�[\LOr0[dpBN<Å���V NZgifqpq]dT�[\LOr_���¹[�TUPyK�NZP�hn[dp<PdKHK�[\LXN�[�[\LOK�lHpqS�ÂqKH]dPdTipqSYO]Qp�lHKRzOcO]QK�zecOK�[\ppr:T�[d[dgUKRP�[\pqSeK���LOTUl�L_TUPFzOKHPdlH]dTUsuKRzbTUSbV�LXNZY�[\KR]�óB��¢FLOKHpq]dKHrìó��éóq �h�TiKRgizOP�NÅ���V gUKRNZ]dSOTiSOfBNZgifqpq]dT�[\LOr ��LOpZPdK#]dcOSOSeTUSOf�[\TirIK�TUP�Yupqgih�SOpqr#T�N�gUgih<]dKRgUN�[\KHzD[\pB[\LOK ]QcOSOSOTiSOf[\TirIK�p�v�[\LOK_pq]QTUfqTiSXNZg�pZSOgUTiSOK_NZgUfqpZ]dTi[dLOr_�1¢FLeKRpq]QKRr#P<ó��éó���ð�� ó�NZSOz¡ð��é^D[\L�cOPITUr#YOgih [\LXNa[Å(KH]dlHKRYe[d]dpqSBNZSezIJâTUSeSOp��Êh�TiKRgiz�KHk#lHTUKRSm[Å���VàgUKRNZ]dSeTUSOf�NZgUfZpq]dT�[\LOr#P>vxpq]@lRKH]Q[�N�TUS�]QKRP�[\]dTilH[dKRzgUTiSOK�N�]�[dLO]dKHPdLOpZgUzDgUKRNZ]dSeTUSOf#YO]dpZsOgUKHrIPH���Xpq]�K|{eNZr#YOgUK��u[dLOKRPQK�[\LOKHpq]dKHrIPyl�NZSns�K�P�[\]dNZTUfqLm[Qvxpq]�¸��NZ]dzOg�h�lRpZr�sOTUSeKRz�[\pwYO]Qp�ÂZKò ý´þaý ó�ó eþxwO c�ܾü! Ç|©´É�yï$|ÇF©(&XÇ�¤|¯�ªaÆ�Æ�¥ . ¯±¬�®²Çdª<4�©�&�4\Ç�Æ'&X¥a¯6C . ¨�®²¤|©�¬«¥a®eÆæ�½ã ��*ZC<¥?ËZÇ04Jx?zeø�Ù<{}|Æ|¨X¤'& ©(&eªZ©2Çdª�¤'&U�hÀ·¬UÆ�ªZ®I¬�®X© Ç =�Ç04�ªZ®BC � |À ô ¸ Ç �hÀ�Ç6F2z ü ñ &XÇ|®BÇ|¬�©�&XÇ04®8�Ç04\¤�Ç�§u©k4\¥a®_¥?4 � ¬�®X®²¥?1¤dªZ®�$|ǨmÆ�Ç�C�©�¥�¥<$R©¹ªZ¬�®�ª]8B9;:�¯UÇdª<4�®X¬�®�=�ªZ¯>=�¥?4�¬�©�&�¦ . ¥?4>É�y 1F&�¬«¤'&n4�¨�®eƬ�®I©�¬�¦�Ç(§X¥a¯±°Z��õ(ø\z   ü¢FLOTUP�lRpq]QpqgUgUNZ]Qh�TirIYOgiTUKHP·Yupqg�h�SepqrITUNZg�¸«[\TirIK�Å���V`gUK�N�]dSXNZseTUgUT�[�h�cOPQTUSOf�Å·KR]QlRKRY�[\]dpZSIpq]·JâTUS�¸

SOp��*��LeKRS{z § Y�pZgih²��õ�  ü �¹[�TiP���KHgUg>Ó�SOp���S��:[\LepqcOfqL���[dLXN�[�[dLOKR]QK#K�{�TUPQ[�giTUSOKRNZ]�[dLO]dKHPdLOpqgizvxcOSOl|[\TipqSOP�p}ÂqKH];x?zeø�Ù<{}|w��LOTil�LD]dKHÌ�ceTU]dK lHp�K|k#lRTUKHSm[\P�pZv}PQTUáRK ó��50 | 2 ����K fqT6ÂqK�NZSDK|{eNZr#YOgUK�pZvPdcel�LnN�giTUSOKRNZ]�[dLO]dKHPdLOpqgiz�vxcOSOlH[dTUpqS�TiS ³ KRl|[\TipqS_ð�� ^m |�@�Xpq]}vxcOSOlH[dTUpqSePFPdcOl�LbNZP[\LeKRPdK�[dLOK�[\TirIKsupqcOSOz pZv�V�pq]QpqgUgUNZ]QhDð��ëºbTiP�K�{�Y�pqSeKRSm[\TUNZgUg�hDg�NZ]QfqKwNZSOz LOKRSelRK�SepZ[nÂZKR]�h TiSevxpq]Qr�N�[dT�ÂZKZ��VNZSPQ[d]dpqSefqKR]�s�pqceSOzOP�suK�KHPQ[\NZsOgUTiPdLOKHz�pqSI[\LOK�Å���V¡gUKRNZ]dSOTiSOf�NZsOTigUTi[dTUKHP2pZv:Å(KH]dlHKRYe[d]dpqSBNZSez�JâTUS�¸SOp���|bJâLXN�[�supqcOSezOP�l�NZS_s�K�pZse[�NZTiSOKRz�vxpq]FgiK�NZ]QSOTUSef�cOSOzOKH]�PdYuKRlRT�£Xl�SXN�[dcO]\NZg²zOTUP�[\]dTisOce[dTUpqSOPPdcel�L6NZPF[\LeK�ceSOTivxpq]Qr0zOTiPQ[d]dTUsece[\TipqS�|¢FLOKRPQKBÌ�ceKRPQ[dTUpqSeP LXN�ÂqK�suKRKRS`lHpqSOPQTUzOKH]dKRzosmh PdK¢ÂZKR]dNZgF]dKHPdK�N�]dl�LOKH]dPR� ÜNZcer LXN�P PdLOp���S

[\LON�[@TivB�*TiP2]dKHPQ[\]QTUl|[\KRzw[\p�suK[\LOK�cOSeTivxpq]Qr*zOTiPQ[\]QTUsOc�[\TUpZS pqS [\LOK�õu¸¹zOTUr#KRSePdTUpZSXNZgeceSOTi[2PQYOLOKH]dK^ | Ü[¸ TUS�A | ø�[\LOKHS�Å·KR]dlHKRYe[d]dpqS�TUP·NZS�KHk#lRTiKRSm[>Åu��VÍgUKRNZ]dSOTiSOf�NZgUfZpq]dT�[\LOrâvxpq]�[\LOK}cOSO]QKRP�[\]dTilH[dKRzlRgUNZPdP�pZv:gUTiSOK�N�]@[dLO]dKHPdLOpZgUz�vxcOSelH[\TipqSOP��­ÜNZcerb�uÙ�ÚqÚ�zq |� ³ l�LerIT�[d[�LXNZP�PQLOp���S�[\LXN�[�Å(KH]dlHKRYe[d]dpqSTUPySOpZ[�NZSDKHk#lRTiKRSm[�Åu��V giK�NZ]QSOTUSOf�NZgUfZpq]dT�[\LOr0vxpZ]y[\LeK�lRgUNZPdPypZv@gUTiSOK�NZ]�[\Le]dKRPQLOpqgiz6vxcOSOlH[dTUpqSePp}ÂqKH]wx?z�ø�Ù<{}|w� ³ l�LOr#Ti[Q[��XÙ�ÚqÚZÛm |�FqyTUP�YO]dp�pZvu��pq]QÓ�P}smh K�{�LOTUsOT�[\TiSOf�N ®²Ç�Æ|© Ç�C�Ü�p�pqgUKRNZS�vxcOSOl|[\TipqS

ðZÚ

Page 72: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

��zeKH£XSOKHz�TUS ³ KHlH[\TipqS ð�� ó��éóq �NZSOz�N�zeTUPQ[d]dTisOce[\TipqS���LOTUl�L�TUP·lRpqSelRKRSm[d]\N�[dKRz�pqS�äQLXNZ]Qz\å�K|{eNZr#YOgUKHPvxpq][dLXN�[FvxcOSOl|[\TipqS��Å2]dTUpZ]@[dp�[dLOK�]dKRPQcOgi[dP@pZv²[dLOTUP@l�LXNZY�[\KR]H�eSOKHTi[dLOKR]@YupqPdT�[\T�ÂZK�SOpq]@SOKRfqN�[\T6ÂqK�]dKHPdcOg�[\P�NZSXNZgipqfqpqcOP

[\p�[dLOK�N�s�p}ÂqK#Åu��VèrIp�zOKHg>]dKHPdcOg�[\P�vxpq]�Å(KH]dlRKHYe[\]QpqSÍLONZz suKRKHSÍpqs�[�NZTiSOKRzDvxpq]�[\LeKwJâTiSOSOp��NZgifqpq]dT�[\LOr_�:��P ³ l�LOr#Ti[Q[@SOpZ[\KHz���Ti[2��NZP�SOp�[�Ó�SOp���STä���LOKH[dLOKR]Ur�Ti[d[dgUKHPQ[\pZSOKZêëP@]dcOgiKRP@l�NZS�Å���VTUzeKRSm[\T�v�h<TUSBYupqgih�SOpZrITUNZgu[dTUr#K�å_� ³ l�LOr#Ti[d[R��Ù�ÚqÚZÛm |�

ï¼Q½}¼�� Jª�ÎH reR:Ã~Y´Ä�ÁQÄ]Þ}:¡Á¹À[� J�R�Y[�-RõKyÀ\YX¿�Ä÷Ã@Ä]à~8 Á�Ä�Ä�¿;:uts��M]YE��RÍaZ\MNO¹À\Z

J K�£X]dP�[FfqT6ÂqK�N�SOKRfmNa[\T�ÂZK�NZSOP���KH][\p ³ l�LOrIT�[d[Rê P�Ì�ceKRPQ[dTUpqSBN�s�pqc�[�JâTUSeSOp��WsmhIYO]dp}Â�TUSef�[\LXNa[[\LeKyJâTUSeSOp��tNZgUfqpZ]dTi[dLOr TiPy®²¥a©�N�Yupqgih�SOpqr#T�N�ge[\TirIK�Å���V¡giK�NZ]QSOTUSOf�NZgUfZpq]dT�[\LOr.vxpq]�[dLOK�lHg�NZPQPpZv}YupqPdT�[\T�ÂZK gUTiSOK�NZ]�[\LO]QKRPQLOpqgUzDvxcOSelH[\TipqSOPH��¢·pB[dLOK suKRPQ[�pZv}pqce]�Ó�Sep���giKRzOfZK�[\LOTiP�TUP�[dLOK�£X]dP�[SOKHfmN�[\T6ÂqK�]dKHPdcOg�[Fvxpq]JâTUSeSOp��'TUS�[dLOK�Å���VWr#p�zeKRg«�JÍK#[\LOKHSÍP�[\]dKHSOfZ[dLOKRS ³ l�LOrIT�[d[Rê P�SOKRfqN�[\T6ÂqK#]dKRPQcOgi[�vxpq]�Å(KH]dlHKRYe[d]dpqSØsmhMfqT6Â�TiSOf_N_PQTUr#YOgUK

YO]Qp�pZv�[dLXN�[ [\LeKBÅ(KR]QlRKHYe[\]QpqSjTiP SOpZ[#NZS`K|k#lRTUKHSm[�Åu��VµNZgifqpq]QTi[\Ler vxpZ]�[\LOK�lHg�NZPQP pZvygUTUSeK�NZ][\Le]dKRPQLOpqgiz¡vxcOSOlH[dTUpqSeP�K¢ÂZKRSâcOSOzeKR]�[dLOK6cOSOT�vxpq]dr zOTiPQ[d]dTUsece[\TipqSjpZS x?zeø�Ù<{}| ü ¢FLOTUP<SeKRfmN�[dT�ÂZK]dKHPdcOg�[�Ye]dp}Â�TUzOKHP�NZSITUSm[\KH]dKHPQ[\TiSOf�lRpqSm[\]dNZPQ[}[\p�ÜNZcerbêëP�YupqPQTi[\T6ÂqK�]dKHPdcOg�[�vxpZ]�Å·KR]QlRKRY�[\]dpZS<cOSOzOKH][\LeK�ceSOTivxpq]Qr0zOTiPQ[d]dTUsece[\TipqS<pqSB[\LOK�cOSeTi[�PdYeLOKR]QKZ���TUSONZgUg�hq����K�PdLep��*[\LXN�[�cOSOzeKR]�[\LOKIcOSOTivxpZ]dr+zeTUPQ[d]dTisOce[\TipqS�pqSØÜ�p�pqgiK�NZS�K�{ON�rIYOgiKRPH�([\LeK

Å(KH]dlHKRYe[d]dpqSoNZgifqpq]QTi[\LeræTUP�NZSÍK|k#lRTUKHSm[wÅ���VãNZgifqpq]dT�[\LOrìvxpq]�[\LOKIlRg�N�PdP�p�vFSOKRP�[\KHzoÜ�p�pZgUK�N�SvxcOSOl|[\TipqSOPR�å¢FLOTiP_PQcOfqfqKHPQ[dPb[\LON�[ ³ l�LOr#Ti[Q[�êëP_SeKRfmN�[dT�ÂZKM]QKRPQcOgi[Bvxpq]_Å(KH]dlHKRYe[d]dpqS'pZStSeKRPQ[dKRzvxcOSOl|[\TipqSOP�zOKRYuKRSezOPypZS_l�Lep�pqPQTUSOf#NZSbNZzýÂZKR]QP\NZ]QT�NZg�zOTiPQ[d]dTUsece[\TipqS��

²�5�� �*=~�N�RG�� G�9���=@G��]�ö�KRl�N�gUgy[dLXN�[_N`lRpZSOlRKHYe[_lRg�N�PdPBÉ�pqStNZSÊK�{eNZrIYegUKDPdYONZlRKÒ� TUP�N`lHpqgUgiKRlH[dTUpqSàpZv�Ü�p�pZgUK�N�SvxcOSOl|[\TipqSOP:pqSN� ü ��S�[\LOTiP�l�LXNZYe[dKR]-�å��TUgUg�s�K2[dLOK}Ü�p�pZgUK�N�S�lRcOsuK+x?zeø�Ù<{ | ü �oÜ�p�pqgUKRNZS�vxcOSOl|[\TipqSÈWùõ� Ê x�ô�ÙqøRÙ<{BTUP�NngUTiSOK�N�]�[dLO]dKHPdLOpqgizÍvxcOSOl|[\TipqSØTiv�[\LOKH]dK<TiP�Nb��KRTifqLm[DÂqKHlH[dpq]£� osAw|NZSOz6N#[\Le]dKRPQLOpqgizBC�o5AËPdcel�L�[dLXN�[]È>����  § Ù�T��_�¹ã��°*�C ü ³ cOl�L�N#YXNZTU]�����ø�Cq FTUP�P\N�TUzb[dp4\Ç�§B4\Ç�Æ�Ç|®X©PÈ ü ³ KHKb�­~�KR]�[\pqceáRpqPH�}Ù�Úqîmð�ήfbcO]dpZfmNe�2Ù�Ú�u�Ù� yvxpq]�K|{�[dKRSOPQT�ÂqKw[\]QK�N�[drIKHSm[\P�pZv�gUTUSeK�NZ][\Le]dKRPQLOpqgiz#vxcOSelH[\TipqSOP@p}ÂqKH]%x?zeøRÙ<{}| ü JÍK�P\N�h [\LON�[P�¬ãV�µ*2C�TUP@N�§X¥�Æ|¬�©­¬(ËZÇ([\LO]QKRPQLOpqgUzwvxcOSOl|[\TipqSTiv·KRNZl�LN�hÀ:TiPFY�pqPQTi[dT�ÂqK��

î�z

Page 73: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

ï¼2G¼Q½ Jª�ÎH reR·Ã~Y´Ä�Á�Ä]Þ&��ZOÁ�Ä]Þì�àÄNO�ÁQÄ]R3reR:Ã~Y´Ä�ÁQÄ]Þ_��OVÞ}¿JY´Á¹À[�NI Z

��SÍ[dLOK�Åu��V gUK�N�]dSOTiSOfbr#p�zOKRg>[dLOK�giK�NZ]QSOTUSOfbNZgifqpq]dT�[\LOrÑLXNZP�NZlRlHKRPdP�[\p6N�StÇQÈ�ªZ¦�§u¯UÇ�¥?4Qª�¤|¯UÇ���o�jm�ø��I ���LeTUl�L��2TiSØpqSOK#[\TUr#K#PQ[\KHY��@YO]dp}Â�TUzeKRP NngUNZsuKRgUKHzÍK�{ON�rIYOgiKÒ�k�:ø�mZ�(�� �����LOKH]dK��oTiPzO]dN���S�vx]Qpqrã[\LOK�zeTUPQ[d]dTisOce[\TipqSp�åpqSI[\LOK�K�{ON�rIYOgiK�PQYXNZlRK]� ü ¢FLOK�vxceSOlH[dTUpqS½m]o6ÉèTUP�l�NZgigUKHz[\LeK�©«ª<4j=�Ç|©�¤�¥a®²¤�Ç�§u©­��NZSezI[\LeKyfqpqNZg´pZv²[dLOKygiK�NZ]QSOTUSef�NZgifqpq]QTi[\LerèTiP}[dp�lHpqSOPQ[d]dcOl|[FN�L�h�YupZ[\LeKRPdTiP�o��LeTUl�L��2��Ti[dL`LOTUfZL`YO]dpqsONZsOTUgiTi[�hZ�>LXNZP gip��åKH]d]dpZ]���Ti[\L`]QKRPQY�KHlH[w[\p¬m ü J K<[\L�cOPwLXN�ÂqK�[\LeKvxpqgigUp���TUSOf�SXN�[\ce]\NZg�zeKH£XSOT�[\TipqSBpZv>Åu��VWgUKRNZ]dSeTUSOf cOPQTUSOf#NZSBpqSegUTUSeK�gUKRNZ]dSOTiSOfwNZgifqpq]dT�[\LOr_ù��üvu�wyx���x�ý!wZ c�è � Ç�ÆRªZ°#©(&eªZ©FªZ®Í¥a®X¯±¬�®²Ç�¯UÇdª<4�®X¬�®�=<ªZ¯>=�¥?4�¬�©�&�¦ �­Æ|¨X¤'&nªaÆ � ¬�®X®²¥?1�¥?4N8�Ç04�/¤�Ç�§u©k4�¥a®��_¬UÆ�ªZ®DK|k#lRTUKHSm[ Å���V*giK�NZ]QSOTUSefbN�gUfqpq]QTi[dLOrìvxpZ]�lHpqSOlHKRYe[�lRg�N�PdP�É÷p}ÂZKR]é� ¬ . ©�&XÇ04\ǬUÆ�ª<§X¥a¯±°a®²¥a¦ ¬�ªZ¯[#·�Sãiø�ãUø�ã   Æ|¨X¤'&Ø©(&eªZ©�©(&XÇ . ¥a¯�¯U¥?12¬�®�= ¤�¥a®BCZ¬�©�¬«¥a®eƪ&X¥a¯6C . ¥?4�ªZ®X°�õ *æÙqø�ªZ®X°CZ¬UÆ|©k4|¬k$R¨�©�¬«¥a®���¥?ËZÇ04+�6ø�ªZ®X°Dm]oDÉ�ø(ªZ®BC<ªZ®X°nzª©à×Rø'­;©�Ù�ù

77� ¬(ËZÇ|®¡ªZ®X° ÇQÈ�ªZ¦�§u¯UÇD�«o«�6øyªZ¯>=�¥?4�¬�©(&�¦Ññ ªZ¯ 1}ªZ°�ÆbÇ0ËaªZ¯±¨eªZ© Ç�Æ<¬�©xÆp&�°�§X¥a©(&XÇ�Æ|¬UÆ_¥a®°�ªZ®BCN�Q¥a®²¤�Ç�§B4�¥?Ë�¬kC�Ç�Cª12¬�©�&pm����� #�#¨�§ýCmªZ©�Ç�Æ�¬�©�ÆN&�°�§X¥a©(&XÇ�Æ|¬UÆ�¬�®I§X¥a¯±°Z��õ� �©­¬�¦�Ç��

7 ¬ . ªZ¯>=�¥?4�¬�©�&�¦÷ñ ¬UÆp4�¨�®¡¥a®àªbÆ�ÇKSR¨´Ç|®²¤�ÇB¥ . ÇQÈ�ªZ¦�§u¯UÇ�ÆD=�Ç|®²Ç04QªZ© Ç�C¬$R°_Æ|¨X¤�¤�Ç�Æ�Æ|¬(ËZÇB¤dªZ¯�¯éÆ©�¥^�é�1�jm�ø��I |øF©(&XÇ|®�12¬�©(&n§B4\¥<$�ª�$R¬�¯±¬�©�°MªZ©�¯UÇdªaÆ|©�Ù�ô¹­oªZ¯>=�¥?4�¬�©�&�¦ìñÔ12¬�¯�¯�=�Ç|®²Ç04QªZ©�Ç_ª&�°�§X¥a©�&XÇ�Æ|¬UÆé�jÆ|¨´¤'&�©(&eªZ©}Å2]��� ¢¡h¤>�:���� �¦§ m����� 2¨h©è×<ª . ©�Ç04#ªZ©y¦�¥�Æ|©c#·�xõ�ø ¸± ø ¸h  <¤dªZ¯�¯éÆw©�¥�é�1�jm�ø��I  ü

��S�pZSOgUTiSOK�NZgifqpq]QTi[\LerãñËTUPN�S�KHk#lRTiKRSm[�Å���VâgUK�N�]dSOTiSOf�NZgifqpq]QTi[\LerãcOSOzOKH]FzOTUP�[\]dTisOce[dTUpqSª�µT�vTi[�PdN�[\TiPQ£XKHPF[\LOK�NZsup}ÂqK�zOKH£OSOTi[dTUpqS�vxpq]�N�£e{�KRzbzOTUP�[\]QTUsOce[dTUpqS�� üï¼2G¼2G � R�Z�ÀýRõà��NM�Ä]�²ÀOÁ ¿�Ä]Z³ K0ÂqKR]dNZg�pZv·pqcO]�]dKRPQcOgi[dPFTUS�ÂqpZg�ÂqK�[dLOK�lRgUNZPdPFp�v�®²Ç�Æ|©�Ç�C�vxcOSOl|[\TUpZSOPR�2¢FLOTiPFlRgUNZPdP���NZP�TUSm[d]dp�zOcOlRKHzsmh���Sm[dLOpqSmhq�uÜ�]QTUfqLm[���KRgUg:N�SOz ³ LXN���K�¸�¢(N�h�gipq]�TUS ���yS�[dLOpqSmhØÇ|©�ªZ¯�ÐU��Ù�ÚqÚmðZ |���üvu�wyx���x�ý!wZ c�è�¶ñ &XÇF¤|¯�ªaÆ�ÆF¥ . ®²Ç�Æ|© Ç�C . ¨�®²¤|©�¬«¥a®eÆF¥?ËZÇ04��c¸|ø üSüSü ø�� | ø\C�Ç|®²¥a© Ç�C���� | ø�¬UÆUC�Ç��@®²Ç�CªaÆ . ¥a¯�¯U¥?1�Æ�ö�mÐ . ¥?4�õ § Ù�3}©(&XÇ . ¨�®²¤|©­¬«¥a®eÆw�c¸�ªZ®BC �c¸�ª<4�Ç�®²Ç�Æ|© Ç�CmÐ

� Ð . ¥?4�õ¬øËÙ�3ÆÈ>�(�c¸|ø üSüSü ø�� |  �¬UÆ�®²Ç�Æ|©�Ç�C�¬ . È § ~���� | 3®1F&XÇ04\Çr~M¬UÆ�ª�®²Ç�Æ|©�Ç�C . ¨�®²¤|©�¬«¥a® ¥a®�c¸|ø ügüSü ø�� | Ü[¸�3��B¬UÆ Ç|¬�©(&XÇ04��*¥?4��J3�ªZ®BC�� | ¬UÆ Ç|¬�©(&XÇ04h� | ¥?4 � | Ð

îeÙ

Page 74: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

�¹[�TiP@K�NZP�h#[dp�ÂZKR]QTiv�h [\LON�[@[\LOK�lRgUNZPdP}pZv�SeKRPQ[dKRz�vxceSOlH[dTUpqSOP@TiP}KHÌ�cOT�Â�NZgiKRSm[�[dp�[dLOK�lRg�N�PdP}pZv>ÙH¸zOKHlRTUPQTUpqS#gUTiPQ[dP2pZv²giKRSOf�[\L#õ_TiS ��LOTUl�L#[\LOK+Â�NZ]dTUNZsOgiKRP@NZYOYuK�N�]}TiS [\LOK�]dK0ÂqKR]QPdKypq]dzOKH]~� | ø üSügü ø��c¸ ü¢FLOK_vxpqgUgip���TiSOf�gUKHrIrINMKHPQ[\NZsOgUTiPdLOKHP<NMl�NZSOpZSOTUlRNZg�]dKRYe]dKRPQKRSm[�Na[\TUpZS¡pZv�SOKHPQ[dKRz¡vxcOSOl|[\TipqSOP<N�PgUTiSOK�N�]�[\LO]QKRPQLOpqgUzBvxcOSOl|[\TipqSOPRùe@ü�ÿÍÿs� -�Ù� 9�®X°DÈ�ob�� | ¤dªZ®¬$|Ç�4\Ç�§B4\Ç�Æ�Ç|®X© Ç�C�$R°<ª�¯±¬�®²Çdª<4�©(&�4\Ç�Æ'&X¥a¯6C . ¨�®²¤|©�¬«¥a®

�"¸2�c¸¬¹ã}ã}ã�\� | � | *ZC | ø12¬�©(&�C | § fa ¸¼ . ¥?4�Æ�¥a¦�Ç�¬�®X©�Ç =�Ç04yf�ø)�hÀ §�� ó À Ü[¸ ø�ªZ®BC � ��©�� ò �hÀõ©�C | © � �6©q� ò �hÀ üö þaý²ý!÷�ø ¢FLOK�YO]Qp�pZv²TUP@smhwTUSOzecOlH[dTUpqSwpqSIõ��2�Xpq]2õ § Ù�[\LOKyNZYOYe]dpqYO]QT�N�[dK[\LO]QKRPQLOpqgUzwvxcOSOl|[\TipqSTUPA�c¸�* ¸¼ pq]Bô%�c¸µ* ô ¸¼ �j�´pZ]#õüøäÙZ�+ÈÊr�cOPQ[#suKbp�vy[dLOK_vxpZ]dr ~���� | �}��LOKR]QK ~`TiPwNSOKHPQ[\KHznvxcOSOl|[\TipqSnpqS��c¸|ø üSüSü ø�� | Ü[¸���Ü�h�[\LOK�TUSOzecOlH[dTUpqS_Lmh�YupZ[dLOKRPQTUPR�X~�lRNZS6suK�K�{eYe]dKRPQPdKHz N�PN�[dLO]dKHPdLOpZgUz�vxcOSelH[\TipqS��"¸V�c¸ô¯ã}ã}ã$�� | Ü[¸2� | Ü[¸h*ZC | Ü[¸|����T�[\Lj�"¸�ø üSüSü ø)� | Ü[¸\ø�C | Ü[¸@PdN�[\TiPQv�h�TUSef[\LeK�lHpqSOzOT�[\TipqSOPFpZv·[\LOK�giKRr#r�Ne�(¢FLOKR]QK�N�]dK�vxpqcO]�YupqPdPQTUsOTigUT�[\TUKHPRùÙq�yÈ § ~���� | ù@[dLOKRS�È6TUP

�"¸2�c¸¬ üSüSü \� | Ü[¸�� | Ü[¸®àó | Ü[¸ � | *2C | § C | Ü[¸�jó | Ü[¸ó��yÈ § ~�� � | ù@[dLOKRS�È6TUP

�"¸��c¸� üSüSü �� | Ü[¸2� | Ü[¸>ôoó | Ü[¸ � | *ZC | § C | Ü[¸

^e�yÈ § ~���� | ù@[dLOKRS�È6TUP�"¸2�c¸¬ ügüSü \� | Ü[¸2� | Ü[¸®jó | Ü[¸ � | *2C | § C | Ü[¸

ºO�yÈ § ~�� � | ù@[dLOKRS�È6TUP�"¸2�c¸¬ üSüSü \� | Ü[¸�� | Ü[¸(ô1ó | Ü[¸ � | *ZC | § C | Ü[¸(ô1ó | Ü[¸ ü

��SnKRNZl�L6lRNZPdK�Ti[�l�NZSns�K�K�NZPQTUg�h�ÂqKH]dTi£OKRzn[dLXN�[�[\LOK�PQ[\N�[\KHzn[\LO]QKRPQLOpqgUz_vxcOSOl|[\TUpZSnTUP�KHÌ�ceT�Â�NZgUKHSm[[\ppÈDNZSOzB[\LON�[d�"¸�ø üSüSü ø)� | ø�C | P\Na[\TUP�v�h�[dLOK�lRpqSOzeTi[\TipqSOPFpZv·[\LeK�giKRr#r�N��

îmó

Page 75: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

²�5'Ï � G�9D96?�� Ñ_��9D96?w;Î�¯ÐÔÑ ¹`�]��=@9 ��?ª�>GH;>G��ª� ³ �;��´��.o���Ew�N���g�[\LOpZcOfqLn[dLOK�JâTUSeSOp��.NZgUfqpZ]dTi[dLOr LXNZPys�KHKRS�PQ[dcOzOTiKRzDK|{�[dKRSOPQT�ÂqKHgihnTUSb[dLOK�pqSegUTUSeK�r#TUP�[�NZÓ�KsupqcOSOzwgUKRNZ]dSeTUSOf�r#p�zOKRg­�ZgUT�[d[dgUK}TUP>Ó�SOp���SINZs�pZce[2Ti[dP>YuKR]QvxpZ]drINZSOlRK�TUS pZ[dLOKR]2giK�NZ]QSOTUSOf�rIp�zOKRgiPR���S�[\LOTiP�PdKRl|[\TipqSD��KwYO]Qp�ÂZK#[dLOK vxpqgUgip���TUSOf#[\LOKHpq]dKHræ��LOTUl�L�PdLep���P�[\LON�[�JâTUSOSOp��èTUP�SOp�[�NZSKHk#lHTUKRSm[_Åu��V÷giK�NZ]QSOTUSOfÍNZgUfZpq]dT�[\LOr vxpq]#[\LOKnlRg�N�PdP<pZv�YupqPdT�[\T6ÂqKb[dLO]dKHPdLOpZgUz¡vxcOSOl|[\TipqSOP<p}ÂqKH]x?zeø�Ù�{ | üú�û:ü�ý´þaü�ÿ c�è � ¬(ËZÇ|®`ªZ®X°�§X¥�Æ|¬�©­¬(ËZÇ�Æ|©«ª<4|©wËZÇ�¤|©�¥?4]�a`�ø�ªZ®X°�§B4\¥a¦�¥a©�¬«¥a® . ª�¤|©�¥?4Ní¹øãÙ6ªZ®BCªZ®X°Ø©(&�4\Ç�Æ'&X¥a¯6CbCÎø zeø�©(&XÇ04\Ç6¬UÆ6ªb§X¥�Æ|¬�©�¬(ËZÇn©(&�4\Ç�Æ'&X¥a¯6C . ¨�®²¤|©�¬«¥a®�m�3Iª¿CZ¬UÆ|©j4�¬k$R¨�©­¬«¥a®`� ¥a®x?zeø�Ù�{}|)3}ªZ®BC�ªªËaªZ¯±¨XÇ�×wø�z . ¥?4N1F&�¬«¤'&�z ëõ�õ ý � ���a`�ø'í}ø�Cq "12¬�¯�¯�®²¥a©Æ=�Ç|®²Ç04dªZ©�Ç�ªA&�°�§X¥a©(&XÇ�Æ|¬UÆ� Æ|¨´¤'&_©(&eªZ©(Å@]S�� �¡U¤>�:���� N¦§ m����� 2¨-Fâ×�¬�®�§X¥a¯±°q�xõ�ø ¸±  �Æ|© Ç�§XÆRÐ��P#N6lRpZSOPdKHÌ�ceKRSOlHKnpZv�[\LOK�Ye]dp�pZv�pZv�[\LOTiP�[\LOKHpq]dKHrb�2��KB��TigUg�pqse[�N�TUS`NZSoK�{eYegUTUlHTi[�v�N�rw¸

TUg�hBpZvnäQLXN�]dz\å�[dLO]dKHPdLOpZgUznvxcOSelH[\TipqSOP�N�SOz6lRpZ]d]dKHPdYupqSOzOTiSOf<K�{ON�rIYOgiK�PQKRÌ�cOKRSelRKRP���LOTil�LDl�NZcOPQKJâTUSeSOp��à[\p�rINZÓZK�K|{�YupqSOKRSm[dT�NZgigih�rINZSmh r#TUP�[�NZÓ�KRPR�æfnNqNZPQP�NZSez�¢·cO]\NZSn�ÎfbNqNZPdP�NZSOz#¢·cO]\NZS²�Ù�ÚqÚ�º� �LXN�ÂqK�cePdKRz�N�lHpqcOSm[\TiSOf�NZ]QfqcOr#KRSm[:[\pyPdLOp��`[\LON�[·fqT�ÂZKRS�N�S�h�[d]dTUYegUK�����`�ø'í}ø�Cq �ø�[\LOKH]dK�K�{�¸TUP�[\P�N�[\NZ]dfqK|[F[\LO]QKRPQLOpqgUz<vxceSOlH[dTUpqS_NZSOz�K�{eNZrIYegUK�PdKHÌ�cOKRSOlHK���LeTUl�LBl�NZcePdK�JâTUSOSep������a`�ø'í}ø�Cq [\pwrINZÓZK�K�{eYupqSOKHSm[\T�N�gUgihIrINZSmh�rITiPQ[\NZÓZKHPR�XsOce[�SOpwK�{eYegUTUlHTi[FlHpqSOP�[\]dcelH[\TipqSB�NZP�Ó�SOp���S��¢FLOK�YO]dp�pZvupZvu¢FLOKHpq]dKHrÞð�� Û�TUP2sXNZPQKRzwpqSwPdK0ÂqKH]\NZgXgiKRr#r�NZPH�(��S [dLOK£X]dP�[@gUKHrIrINe����K�PdLOp��

[\LON�[wNnSOKHPQ[dKRz1Ü�p�pqgUKRNZSÍvxcOSOl|[\TipqSÍ��Ti[dLoNZgi[dKR]QSXN�[\TiSOfnlHpqSOSOKHlH[dT�ÂqKHP ]dKHÌ�ceTU]dKHP K|{�YupqSOKRSm[dT�NZgigihg�N�]dfqK�lHp�K|k#lRTUKHSm[\PR� ³ TirITig�NZ]}]dKHPdcOg�[\P�l�NZS_suK�vxpZcOSOzBTUS �Îf_cO]dpqfqNe��Ù�Ú�u�ÙZÎOÅ�N�]dsuKR]d]�hq�:Ù�ÚqÚ�º� |�e@ü�ÿÍÿs� -���Ù! Ç|©�õ¿$|Ç�¥�C�CwªZ®BC ¯UÇ|©cV�ãS�Ò*ZC^$|Ç�ª�§X¥�Æ|¬�©�¬(ËZÇy©�&�4\Ç�Æ'&X¥a¯6C . ¨�®²¤|©­¬«¥a®D¥a®½x?zeø�Ù<{}|1F&�¬«¤'&^4\Ç�§B4\Ç�Æ�Ç|®X©xÆ�©�&XÇ�®²Ç�Æ|©�Ç�C . ¨�®²¤|©�¬«¥a®

È | § � üSüSü ���c¸/�^��¼| /�����| /�½���H  üSüSü  /��� | Ü[¸Q /��� | üú ¥?4�^�F_ÃGF0õ 1�Çe&eª<ËZÇ�V[À]*��õÀ ÜP� V���øU1F&XÇ04�Ç���À�¬UÆI©�&XÇDÃ�/�©(& ú ¬k$|¥a®uª�¤�¤|¬®X¨�¦ª$|Ç04¢ö�� ò §�P¸ § �õ¼ § ÙZø��;� § ó�ø �¡� § ^eø2Ç|© ¤RÐ�Ðö þaý²ý!÷�ø ¢FLOK<Ye]dp�pZvTiP�smh TUSOzOcelH[\TipqSMpqS¯f�ø���LOKH]dK<õ § ó�fR.Ù ü �Xpq]�lRg�N�]dTi[�hn��K<cePdKI[���psXNZPQK<l�N�PdKRPH�6¢FLOK<lRNZPdK½f § Ù�TUP�[\]dT6Â�T�NZg­�_�¹vyf § ó�ø·[dLOKRS`PQTUSOlHK½È¢a�kzeø'zeø�zeø�Ùqø�Ù�  § Ù�N�SOzÈ¢a�kzeø'zeøRÙqø'zeø�Ù�  § ze����K�£XSezn[\LXNa[�V���øsV�� ü ³ TUr#TUgUNZ]dg�hq�OPQTUSOlHKAÈx¢a�jzeø'z�ø�Ùqø�Ùqø�zm  § zeøX��K�£XSOz[\LON�[�V�¢yø�V�� ü

îq^

Page 76: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

JÍK�SOp�� PdceYOY�pZPdK�[\LXN�[�[\LeK�giKRr#r�N�TiP�[d]dcOK�vxpq]F[\LeK�Â�NZgicOKRP"f § Ùqø�ó�ø üSüSü ø2ê�ôâÙ�NZSOzbgUKH[õ § ó�ê�âÙq��Ü}h_NZPdPQcOrIY�[\TUpZS��·�@Vc¸�ø ügüSü ø V |  FNZSOz�CwNZ]dK�PdcOl�L_[\LXNa[�Vpã}�Ò*2Cw]QKRYO]QKRPQKRSm[\PéÈ | ü�¹v���Ky£e{ª� | § ÙyNZSOzp� | Ü[¸ § ze��[\LOKHS<Ti[@vxpqgigUp���P2[\LXNa[£Vc¸2�c¸ôTã}ã}ã$¤V | Ü�¼�� | Ü�¼y*ZC�ô¥V | TUP}N[\Le]dKRPQLOpqgizbvxcOSOl|[\TipqSb��LOTil�L6]dKHYO]dKHPdKHS�[dP�È | Ü�¼���PQp�smhB[\LOK�TUSOzecOlH[dTUpqS_Lmh�YupZ[dLOKRPQTUP�[dLOK�giKRr#r�NLOpqgizOPvxpq]�V���ø ügüSü ø V | Ü�¼Rø´NZSOzB��K�SeKRKRzbpqSOg�h<PdLep���[\LXN�[�T�[�LOpqgizOPvxpq]�V | Ü[¸�NZSOz¥V | ü³ TUSelRKeÈ | � Ùqø�Ùqø ügüSü øRÙqø'zeø'z�ø�Ù�  § zeø���K�LXN�ÂqK�[dLXN�[¦Vc¸�§VB¼�«ã}ã}ã�§V | ÜP�æ§V | ©uC���¶�S[\LeK�pZ[dLOKR]}LXNZSOz²��PQTUSOlHKNÈ | �jz�ø'zeø üSügü ø'z�ø�Ùqø�Ù�  § Ùqøm��KyLXN�ÂqK�[dLXN�[£V | Ü[¸�¤V | *ZC ü �X]dpqr [dLOKRPQK[���pITiSOKRÌ�cXNZgiTi[dTUKRP�Ti[vxpqgigUp���P[\LON�[

V | Ü[¸Jø�Vc¸®EV�¼-¹ã}ã}ã¶EV | ÜP� ü³ TiSOlRK�VnTiPFY�pZPdTi[dT�ÂZKZ�e[dLOTUPFTiSOKRÌ�cXNZgiTi[�h�TUr#YOgUTiKRP�[dLXN�[

V | Ü[¸Jø�V��-EV��.¹ã}ã}ã¶EV | ÜP� ütyPQTUSOf [dLOK�TUSOzOcelH[\TipqSBLmh�YupZ[dLOKRPQTUPF��K�pqse[\NZTUS�[dLOK�TUSOKHÌ�cONZgUT�[�h

V | Ü[¸Jø ��ÙLE�P¸¬¹ã}ã}ãhE� | ÜP¨� ©V�� § � | Ü=��V�� ü³ TirITig�NZ]Qgihq��PQTUSOlHKDÈ | � Ùqø�Ùqø üSüSü ø�Ùqø'zm  § zeøe��K�LON}ÂZK�[dLXN�[�Vc¸®EV�¼-¹ã}ã}ãh1V | Ü[¸J©�C�øXPdp

V | ø§Vc¸¬EV�¼-¹ã}ã}ã¶EV | Ü�¼wø�V��-EV��.¹ã}ã}ã¶�V | Ü�¼ üÜ�h<[dLOK�TUSOzecOlH[dTUpqSBLmh�Y�pZ[dLOKRPQTUPH�O��K�£XSOzB[\LXNa[

V | øË��ÙME�P¸¬¹ã}ã}ã�E� | ÜP¢| ©V�� § � | ÜP� V���øNZSOzB[\LeK�giKRr#r�N�TUPFYO]Qp�ÂZKRz��³ TUSelRK¦� | § Õ �«ª[|m �ø���LOKH]dK�ª § ¸ /�¬ ¢¼ ø���K�LON}ÂZK�PdLOp���S�[\LON�[%È | r�cOP�[FLXN�ÂqK�lHp�K|kIlHTUKHS�[dP��LOpqPQK�]dN�[\Tip TUPFK�{eYupqSOKHSm[\T�N�gUgihIg�N�]dfqK��JÍKF]QKRÌ�cOTi]dKFN�zOK|£XSOT�[\TUpZS�vx]dpqr0���ySm[\LOpqSmh_Ç|©�ªZ¯�ÐU�eÙ�ÚqÚmðq :s�K|vxpq]dK�PQ[�Na[\TUSef�[\LOKSeK|{�[@gUKHrIrINe�

��üvu�wyx���x�ý!wZ c�k�*z ! Ç|©ém¯$|Ç�ª`¯±¬�®²Çdª<4n©(&�4\Ç�Æ'&X¥a¯6C . ¨�®²¤|©­¬«¥a® ¥?ËZÇ04µx?zeøRÙ<{}| ü � Ç_ÆRªZ°`©�&eªZ©ém¬UÆ�lRpqSePdTUP�[\KHS�[^12¬�©(&`ªnÆ�Ç|©^ § x�k� ¸ ø'»�¸��Høh�k� ¼ ø'»'¼��Hø üSüSü ø+�k�W%�ø�» % �0{`¥ . ¯�ª�$|Ç|¯UÇ�CMÇQÈ�ªZ¦�§u¯UÇ�Æ#¬ .m���� À   § »�À . ¥?4BªZ¯�¯ÆÃ\Ð ñ &XÇ#Æ�Ç|©Y^Þ¬UÆIÆRªZ¬kCD© ¥<PdYuKRlHTiv�h`m6¬ . mn¬UÆ�©�&XÇb¥a®X¯±°6¯±¬�®²Çdª<4�©(&�4\Ç�Æ'&X¥a¯6C

îZº

Page 77: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

. ¨�®²¤|©­¬«¥a®n¥?ËZÇ04hx?zeøRÙ<{ | 1F&�¬«¤'&#¬UÆ�¤�¥a®eÆ|¬UÆ|© Ç|®X©õ12¬�©(&D^FÎB1�ÇÆRªZ°�©�&eªZ©:Æ|¨´¤'&�ª�Æ�Ç|©·¬UÆ�ªyPQY�KHlRTiv�h�TiSOfPdK|[ . ¥?4Am ü ñ &XÇ�PdYuKRlHTi£XlRN�[\TipqSØS�cOr�suKR]6¥ . maø%C�Ç|®²¥a©�Ç�CX­ | �jmH |øF¬UÆ�©(&XÇ�Æ|¦#ªZ¯�¯UÇ�Æ|©yÆ|¬65aÇ_¥ . ªZ®X°Æ�§XÇ�¤|¬ . °a¬�®�=wÆ�Ç|© . ¥?4wm ü¢FLOK�vxpqgUgip���TiSOf�]QKRPdcegi[\P�NZ]QK�YO]dp}ÂZKRznTiS ����Sm[dLOpqSmhØÇ|©ªZ¯�ÐU�²Ù�ÚqÚmðq �ù

e@ü�ÿÍÿs� -�k����mÐ�®JËZÇ04�°�¯±¬�®²Çdª<4�©�&�4\Ç�Æ'&X¥a¯6C . ¨�®²¤|©­¬«¥a®pmw¥?ËZÇ04wx?zeø�Ù<{}|e&eªaÆ�ªw¨�®X¬@SR¨XÇ�Æ�§XÇ�¤|¬ . °a¬�®�=�Æ�Ç|©}¥ . Æ|¬65aÇ

­ | �jmH RÐ� Ð ,(. m]oB�� | ©(&XÇ|®I­ | �kmR  § õ�tÙ ü¯ Ð ! Ç|©®mN°����c¸|ø ügüSü ø�� | Ü[¸Q ªC�Ç|®²¥a© Ç�m����c¸�ø üSüSü ø�� | Ü[¸|ø�Ù� #ªZ®BC_¯UÇ|©JmN±��(�c¸�ø üSügü ø�� | Ü[¸Q �C�Ç|®²¥a©�Ç

m����c¸|ø ügüSü ø�� | Ü[¸|ø'zm  ü ñ &XÇ|®­ | �kmR [FZ­ | Ü[¸H�jm=°� ¬E­ | Ü[¸R�kmP±�  ü

�uÐ ,(. m�¬UÆyª�¯±¬�®²Çdª<4©�&�4\Ç�Æ'&X¥a¯6C . ¨�®²¤|©�¬«¥a®_¥?ËZÇ04Ux?z�ø�Ù<{}|e1F&�¬«¤'&AC�Ç�§XÇ|®BCaÆy¥a®X¯±°�¥a®_¤�¥�¥?4�CZ¬�®uªZ©�Ç�ÆÙqø�ó�ø üSüSü ø�f�ø·©(&XÇ|®6©�&XÇ�Æ�§XÇ�¤|¬ �¤dªZ©�¬«¥a®6®X¨�¦ª$|Ç04�¥ . m#¬UÆ

­ | �kmR  § ó | Ü i ­ i �kmR  üJ KFcOPdK%r�KRr#r�N�ð��iÙqÙF[dp�PdLOp��¡[dLXN�[>[dLOKFvxcOSOlH[dTUpqS}~ | zOK|£XSOKHz�suKRgUp��1LXNZP}N�PdrINZgig�PQY�KHlRTiv�h�TiSOfPdK|[��e@ü�ÿÍÿs� -�k���Ù! Ç|©�õù$|Ç�Ç0ËZÇ|®DªZ®BC ¯UÇ|©®~ | $|Ç�©(&XÇa��¥�¥a¯UÇdªZ®_©(&�4�Ç�Æ'&X¥a¯6C . ¨�®²¤|©­¬«¥a®µ4�Ç�§B4\Ç�Æ�Ç|®X©�Ç�C$R°�c¸¬àó?��¼.1º<���.¹ã}ã}ã¶àó | Ü�¼ � | Ü[¸¬��­ó | Ü�¼ âÙ� V� | *âºdWÙ�îY¹ã}ã}ãhjó | Ü=� àó | Ü�¼ Ùó üñ &XÇ|®�­ | ��~ |  Ftð�õ�ôoÛ üö þaý²ý!÷�ø ¢FLOK�vxcOSOlH[dTUpqS�~ | ±<TiPF]dKHYO]dKHPdKRSm[dKRz6smh

�c¸�jó?��¼�oº<���.¹ã}ã}ã�jó | Ü�¼ � | Ü[¸+*ʺsâÙ�îY¹ã}ã}ãhjó | Ü�¼ Ùó üîmð

Page 78: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

�¹[�TUPFP�[\]\N�TUfqLm[dvxpZ]Q��NZ]dz�[\pAÂZKR]dT�v�h<[\LON�[F[\LOTiP�TUPFYO]QKRlRTiPdKHgih<[dLOK�SOKRP�[\KRz_vxcOSOl|[\TUpZS� üSüSü ���c¸/�^��¼| /�����| /�½���H  üSüSü  /��� | Ü[¸

pqSBõBô¡ÙéÂZN�]dT�N�sOgUKHPR�2Ü�h�Å�NZ]�[�ó�pZvFr�KRr#r�Nwð��iÙqÙZ�O��K�LXN�ÂqK�[\LON�[�­ | Ü[¸R��~ | ±�  § õ ü¢FLOK�vxcOSOlH[dTUpqS�~ | °ITUPF]QKRYO]QKRPdKHSm[\KRzns�h

�c¸�jó?��¼-1º<���.¹ã}ã}ãhjó | Ü�¼ � | Ü[¸U*ʺUâÙ�îd¹ã}ã}ã¶àó | Ü=� ô Ùó ü��fqNZTUS��epqSOK�l�N�SbKRNZPdTigih�ÂZKR]QTiv�h<[dLXN�[F[\LeTUP�TUPFYO]QKRlHTUPdKHgih<[dLOK�SOKRP�[\KHzbvxcOSOl|[\TipqS

� üSügü �����;�����H /����¢� /����¨|  üSüSü  /��� | ÜP�� /��� | Ü�¼� /��� | Ü[¸

pqS6[dLOK õ6ôj^�ÂZN�]dT�N�sOgUKHPy����ø üSügü ø�� | Ü[¸�Î�TUS6[dLOTUP�SOKRP�[\KRz�vxcOSOl|[\TipqS6[\LOKwÜ�p�pqgiK�NZSnlRpqSOSeKRlH[dT�ÂZKRPNZg�[\KR]QSXN�[dKFs�K|[���KHKRSI� NZSOz�� cOSm[\Tige[\LeKwÂZKR]�hwKRSOz��m��LOKH]dKF[���p�lHpqSOPdKHlRce[dT�ÂZK��êëP�p�lRlHcO]R� ³ TUSOlHK~ | °#zOp�KRP�SOpZ[FzeKRYuKRSOz_pqS�[\LOK�[���pAÂ�NZ]dTUNZsOgUKHPU�c¸|ø���¼�øeYXNZ]Q[dPFº#NZSOzbó�pZvÆr�KRr#r�Nwð���ÙqÙ�TirIYOg�h[\LON�[a­ | Ü[¸H��~ | °�  § º²­ | ÜP����~ | °�  § º´��õBô1óq  ü �¹[�vxpqgigUp���PFvx]dpqr0YXNZ]Q[�^wpZvPr�KRr#r�N#ð���ÙqÙ�[\LXNa[­ | ��~ |  Ftð�õ�ôoÛ ü¶�SOKyr#pq]QK�giKRr#r�N�TiP@]dKRÌ�cOTi]dKHz��>JÍK�YO]Qp�ÂZK�[dLXN�[}[\LOK�lRp�KHk#lHTUKRSm[dP�pZv � | Ü[¸2NZSOzp� | TiSIN�S�h]dKHYO]dKHPdKHS�[\N�[\TipqS_pZv-~ | r�cOP�[�s�K�NZgirIpZPQ[���sOce[�SOpZ[�K�{eNZlH[dgihq�XKHÌ�cXNZg«�

e@ü�ÿÍÿs� -�k��ØÙ! Ç|©@õ $|ÇIÇ0ËZÇ|®`ªZ®BC�¯UÇ|©P~ | $|ǪC�Ç��@®²Ç�CbªaÆ ¬�® ! Ç|¦ ¦#ªg�mÐ�� � Ð ! Ç|©P(ªã��¯*}C4\Ç�§B4\Ç�Æ�Ç|®X©�~ | Ð ñ &XÇ|®`( | Ü[¸h©O( | ©�( | Ü[¸¬�(�� üö þaý²ý!÷�ø r�KH[�³'ë�zOKRSepZ[\K�[dLOK�Ü�p�pqgiK�NZS�ÂZKRl|[\pq]���LepqPdKwêZ¸¹[dL�lRp�pq]dzeTUSXN�[dKyTiP�Ù�NZSezBNZgUgupZv���LOpqPQKpZ[dLOKR]FlHp�pq]QzOTUSON�[\KHPFNZ]dKéz��Pr:K|[�´ § ³µ�y1³µ¢-1³·¶yùã}ã}ã�E³ | Ü[¸�� ³ TUSOlHK�~ | �@´�  § z�øeTi[�vxpqgUgip���P[\LON�[�(�æ�(¢P«ã}ã}ã��( | Ü[¸y©�C���¶�S6[\LOK�pZ[\LeKR]�LXN�SOz��²giKH["» § ³µ�槳µ¢P«ã}ã}ã��³ | ÜP���³ | �³ TiSOlRKD~ | �j»|  § Ùqø�Ti[�vxpqgUgip���P�[\LXNa[�(�P�(�¢æ«ã}ã}ã��( | ÜP���( | *3C���V�pqr�sOTiSOTUSefI[\LeKRPdK�[���pTUSeKRÌ�cXNZgiTi[\TiKRP���K�£XSOzB[\LXNa[s( | ø�( | Ü[¸ ü¢(p PQKRK�[dLXN�[r( | lRNZSOSOpZ[s�K�r�cOl�LBfq]dKRN�[\KH]�[\LXN�S�( | Ü[¸�øegiKH[+m § ³�¸!5³µ¢¬5³·¶Ú5³µ¸¬�ã}ã}ã�³ | ÜP�®{³ | � ³ TUSOlHKU~ | �kmR  § z�ø���K�LXN�ÂqK](�¸!�(¢¬�ã}ã}ã��( | ÜP�®�( | ©¹C ü ¶�SI[\LOK�pZ[dLOKR]�LXNZSOz��gUK|[�Ô § ³�¸y�³µ�æ¹ã}ã}ã�§³ | Ü[¸ ü ³ TiSOlRKa~ | �­Ô�  § ÙqøX��K�LXN�ÂqK�[dLXN�[s()¸��(����ã}ã}ã��( | Ü[¸%*�C��V�pqr�seTUSOTiSOf6[\LOKHPdK_[���pMTUSOKHÌ�cXNZgUT�[\TiKRPR�2��K_£OSOz1[dLXN�[�( | © ( | Ü[¸M�(��ø@NZSOzo[\LOK_gUKHrIrINnTiPYO]Qp�ÂZKRz��

îqî

Page 79: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

ö þaý²ý!÷�ý!÷ú�û:ü�ý´þ�ü�ÿ c�è ¬ø JÍK�£O]dPQ[YO]dp}ÂqK�[dLOK�[dLOKRpq]QKRråvxpq]�[dLOK�]dKHPQ[d]dTUl|[\KHz�l�NZPQK�TUSI��LeTUl�L��K�NZPdPQcOr#K�[\LXNa[}�a` § ��ÙZø üSüSü ø�Ù�  ü ��v�[\KR]�YO]dp}Â�TUSOfb[\LOTiP�lRNZPdK<��K���TUgig@PdLOp��åLOp��Þ[\LOTiP�NZP ¸PdcerIYe[dTUpqSBlRNZSbsuK�KRgiTUr#TUSXNa[\KRz²���T�{�í�øãÙ NZSOzDC^ø z ü r:K|[a^ÊzOKRSOp�[\K�[\LeK PdYuKRlRT�v�h�TUSOf<PQKH[�vxpq]y[\LOK�[dLO]dKHPdLOpZgUz6vxcOSOl|[\TipqS~ | Îq��KyÓ�SOp��¡vx]Qpqr�r�KRr#rIN�ð��iÙ�ó�[dLXN�[éÇ ^"Ç*Ftð�õ�ôBÛ ü r�KH[®�ès�KF[\LeKyzOTiPQ[d]dTUsece[\TipqS pqS^x?zeø�Ù<{}|

��LOTil�L#TiP�cOSeTivxpq]Qr.pZS�^MNZSez fqT�ÂZKRP2áRKH]dp���KRTifqLm[2[\p"ÂZKRlH[dpq]dP@pqc�[\PdTizOKpZv1^ ü JÍK���TigUg�PdLOp��1[\LXNa[��Ti[dLg~ | NZPy[dLOK�[\NZ]dfqK|[�lHpqSOlHKRYe[R�Í�÷NZP�[\LOK�zOTUP�[\]dTisOce[dTUpqSBp}ÂqKH]�K|{eNZr#YOgUKHPR��NZSOzD× § ¸¢ | Ü_¶ N�P[\LeKKR]Q]dpq]>YONZ]\NZr#KH[dKR]H�=z ëõ²õ ý ���d� Ùqø üSüSü ø�Ù� �ø'í�ø Cq �l�NZSOSepZ[@NZl�LOTiK¢ÂZK�N�Lmh�YupZ[dLOKRPQTUP®�:���� ·��LeTUl�LP\Na[\TUP�£XKRP�Å2]�¡U¤>�·�(�� ½¦§ m����� V¨YFÞ×�TUS�Yupqgih²��õ(ø ¸±   § Yupqgih²��õ� �PQ[\KHYOPR��¢(p_PdKHK#[dLOTUPH�(SOp�[\K £X]dP�[[\LON�[�smh6pqce]�l�LOpqTilRK pZv}×�N�SOz��Bø�NZSmh_[\LO]QKRPQLOpqgUzDvxcOSelH[\TipqS��¹ã?�T*ºC���LOTUl�LDTiP�×�¸�NZlRlHcO]\N�[dK��Ti[dL�]dKHPdYuKRlH[F[dp}~ | r�cOP�[Fs�K�lHpqSOPQTUPQ[dKRSm[F��Ti[dL ^����¢FLOTiP[\KHl�LOSOTUÌ�cOK���NZP£X]QPQ[�cOPQKRzBTUSM��Å2Ti[d[NZSOz�Ò}N�gUT�N�S�[R�eÙ�ÚZÛqÛm |�é  ³ TUSOlHKY^�TiP2N�PdYuKRlHTiv�h�TUSef�PdK|[2vxpq].~ | øa[\LOpZcOfqL��mT�vX�µã���*ZCyTUP>lHpqSOPdTiPQ[dKRSm[��Ti[dL�^ [dLOKRSITi[}r�cOP�[TUSwv�NZl|[NZfZ]dKRK�K|{eNZl|[\g�h#��T�[\L�~ | ü JÍK���TUgigXPdLOp��Ê[\LON�[�[dLOKR]QK�TiP}SepDÂ�NZgUcOKpZv í_��LeTUl�L ��pZcOgUz KHSXNZsOgiK�JâTUSOSep��o[dp�fqKRSOKH]\N�[dK�NyÂqKHlH[dpq]æ�¡Pdcel�Lw[dLXN�[æ��ã���*ZC�]QKRYO]QKRPQKRSm[\P~ | �(�� FTiS�Y�pZgih²��õ� �PQ[\KHYOPR�

r�KH[�����ø�CZ @suK�PQcOl�L<[\LON�[�JâTiSOSOp��âfqKRSeKR]\Na[\KRP[��NZSez`�Tã���*ZC�]dKHYO]dKHPdKHS�[dP~ | ����  ü ³ TUSOlHK~ | ±#TUPYO]QKRlHTUPdKHgih#[\LOK�SeKRPQ[dKRzBvxcOSOl|[\TUpZS^È | Ü[¸�p�võr:KHrIrIN ð�� Ú NZSOz���TUP�YupqPQTi[\T6ÂqK��esmh½r�KRr#rINð�� Ú���K�LXN�ÂqK�[\LXNa[Y� | Ü[¸U*Z� | Ü=�)��� ü V�pqr�sOTiSOTUSef�[\LOTiP��Ti[\L�r�KRr#rINwð��iÙ�^e�e��K�pqs�[�NZTiSÙn© � |� | Ü[¸ ©�ÙM Ù

� | Ü=� § ÙM ÙÕ �«ª |   ü

³ TiSOlRKF��K�NZPQPdcOr#KRz#[\LXNa[P�a` § ��Ùqø üSüSü ø�Ù� |ø�N�SOzIK0ÂqKH]QhwK|{eNZr#YOgiK�vxpZ]>JâTUSOSOp��àgUTUKHP2TUS½x?zeø�Ù�{}|�øTi[yvxpqgUgip���Py[\LON�[ � £� £ ��� § í ë vxpZ]�Pdpqr#K�YupqPQTi[\T6ÂqK�TiSm[\KRfZKR]hêqø�NZSOzDLOKHSOlRK�[dLXN�[éí § Ù ¸

�50¼» £ 2 üÜ�ce[F[dLOKRS Õ ��õNª |  �cOYuzXN�[\K�P�[\KHYOP�NZ]dK�]QKRÌ�cOTU]QKRzB[\p#NZl�LOTiK¢ÂZKa� | Ü[¸J*Z� | Ü=�¢���RÎOlRpqSePdKRÌ�cOKHSm[\gihZ�SOpwLmh�Y�pZ[dLOKRPQTUP�lRpqSePdTUP�[\KHS�[���Ti[dL�~ | lRNZS_s�K�NZl�LOTiK¢ÂZKRznTiSBY�pqg�h²��õ� }PQ[\KHYOPR��yp�� ��KblHpqSOPQTUzOKH]w[dLOKBl�NZPQKbp�v�NZSjNZ]QsOTi[d]\NZ]�hØYupqPdT�[\T�ÂZK�PQ[\NZ]Q[eÂqKHlH[dpq]R� ` Î2Pdp�£e{oPQpqr#KYupqPdT�[\T�ÂZK]�a`�ø�íTø�ÙqøXNZSOz¥CAø�z ü JÍK�NZPQPdcOr#K���Ti[\Lepqce[�gUpZPdPFpZv(fqKRSeKR]\N�gUTi[�hI[\LXNa[Y�a` ¸ F¯�a`¼ FüSüSü F¯� `| øqPQTUSOlHKFTiv´[\LeTUP2TUP�SOpZ[}NZgi]dK�N�zeh�[dLOK�l�NZPQKF]dKHSXNZr#TUSOf]Â�NZ]QT�NZsOgiKRP·��TUgUg�r�N�ÓZKFTi[>PdpO� ³ TUSOlHKNZgig>K|{eNZr#YOgUKHP�vxpq]�JâTUSOSep�� NZ]dKwTiS¯x?z�ø�Ù<{}|�ø�N�[�K¢ÂZKR]�h YupqTiS�[�TiSD[\LOKwK|{�KHlRce[dTUpqSMpZv�JâTiSOSOp��[\LeKy]dN�[\Tip�p�v²��KRTUfZL�[dPM�hÀ²NZSOz��Për�cOP�[}s�K � l©� l½ ã�í-E:vxpq]@PdpZrIK�TUSm[dKRfqKH]Jm ü �¹v²[dLOKR]QK�TUP@SOp�TUSm[dKRfqKH]ÃS¸�Pdcel�Lb[\LON�[

Ù�© ��`|� `| Ü[¸ã�í À � ©'ÙM Ù

� | Ü=� ø

î�u

Page 80: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

[\LeKRS�JâTiSOSOp��âl�NZS�SOK¢ÂZKR]�NZl�LOTiK¢ÂZK�N�Lmh�Y�p�[\LOKHPdTUP���LOTUl�L<]QKRYO]QKRPQKRSm[\P[dLOK�[\Le]dKRPQLOpqgizIvxcOSOl|[\TipqS~ | �(�� |ø·NZSOzM[\L�cOP�lRNZSØSOK0ÂqKH]�NZl�LOTiK¢ÂZK�×�¸ N�lRlRce]\NZl|h�Î2PQpb��K<NZPQPdcOr#KI[dLXN�[�PdcOl�LØNZSsÃV¸�K|{�TUP�[\PH�³ TirITig�NZ]Qgihq��T�v·[dLOKR]QK�TUPFSOp#TUSm[\KHfqKR]+Ãj¼�PQcOl�Lb[dLXN�[

Ù�© � `| Ü[¸� `|ã�í À¿¾ ©'ÙM Ù

� | Ü=� ø[\LeKRSw[\LOKH]dK�TUP@N�[\LO]QKRPdLepqgUzwvxcOSOl|[\TUpZSw��LeTUl�LwJâTUSOSOp��àl�NZS#SOK¢ÂZKR]}K|{�YO]QKRPdP���~ | ��Ti[dLeÂ�NZ]QT�NZsegUKRPYuKR]dr�ce[\KHz´ |�OPQpwPdcOl�LbNZS^à ¼�r�cOP�[�K|{�TUP�[�NZP���KRgig«�2¢(NZÓ�TUSOf�[dLOK�YO]dp�zOcOl|[�pZv:[\LOKHPdK�TUSeKRÌ�cXNZgiTi[\TiKRP��K�£XSez_[dLXN�[

Ùn©�í À � / À¿¾ ©ü |Ù[ Ù� | Ü=� ¡ ¼ ü

³ TiSOlRKµÃS¸[ à ¼�r�cePQ[<suK6NMY�pZPdTi[dT�ÂZKbTiS�[dKRfqKH]R�[dLOTUPITUr#YOgUTiKRP�[\LON�[�íì©  HÙa ¸À £ �·Á ¡ ¼ ü �yp��lRpZSOPdTizOKR]IN�[dLO]dKHPdLOpZgUzovxcOSOlH[dTUpqSo��LOTil�Là]dKHÌ�cOTU]QKRPw[\LXN�[��"¸�*Â� | Ü=��� | ��NZfmNZTiS��æ~ | ��T�[\LjNYuKR]dr�ce[�N�[dTUpqS pZSw[dLOK+Â�NZ]dTUNZsOgUKHP>TUP2PQcOl�LIN�vxceSOlH[dTUpqS´ �� ³ TiSOlRKr� ` ¸ F¯� `| N�SOz�íT©   Ù! ¸À £ �·Á ¡ ¼ øTi[�vxpqgUgip���P�[\LXN�[ Õ �xõ/ª[|m �cOYuzXN�[dK�PQ[\KHYOPF��TUgig²suK�]dKRÌ�cOTi]dKHz��XPdpwSOpwlRpZSOPdTiPQ[\KHSm[yLmh�YupZ[\LOKHPdTiPylRNZSsuK�N�l�LOTUK0ÂqKRzbTUS�Yupqgih�SOpZrITUNZg�[dTUr#KZ� ��¢FLOKHpq]dKHrìð��éÛq ��P�NZS�KRNZPQhDlHpqSOPQKRÌ�cOKRSelRK#pZv@[\LOTiP�YO]dp�pZv �²��KwSOpZ[dK�[\LXN�[�[\LOK K�{ON�rIYOgiK�PdKHÌ�cOKRSOlHK#��LeTUl�L

PdTirIYegihIlHh�lRgiKRPF[\LO]QpqcOfqLg^`��TUgUguvxpq]dlHK�JâTUSOSOp��t[\pwrINZÓZK�K�{eYupqSOKHSm[\T�N�gUgihIrINZSmh�rITiPQ[\NZÓZKHPFpqS~ | üï¼��F¼Q½ � °«R·Ä]R�YXÃPObJª��H rw¿¡:TR�YZÃ1¿JM�Ä]à K�¿®Y�8 ÁQÄ�Ä�¿¡:¢FLOK�lRpqSOP�[\]QcOlH[dTUpqS_pZv>¢FLeKRpq]QKRrÑð�� ÛIl�N�SbsuK�cePdKRzb[\p#PdLOp��Ë[\LON�[�V�pq]dpqgig�NZ]�hBð��ëº�TiPF[\TUfZL�[FvxpZ]JâTUSeSOp��*cOY�[\pBYupqgih�SOpZrITUNZg·v�N�lH[\pZ]dPR���´pZ]�K¢ÂqKHS°Â�NZgicOKRP�fqFèõ`gUKH[]~ i| suK�[\LOK vxcOSelH[\TipqS�~ ip}ÂqKH]FÂ�NZ]dTUNZsOgUKHP-�c¸|ø üSüSü ø�� i ��LeTUl�L���KUÂ�TUK|�1NZP(suKRgipqSOfqTiSOf�[\p�Nyg�NZ]QfqKR]-Â�NZ]dTUNZsOgiK}PQKH[Æ�c¸|ø üSüSü ø�� | üV�pqSOPQTUzOKH]>NZfqNZTUS�[\LOK�lRpqSePQ[\]QcOlH[dTUpqS�pZvX¢FLOKHpq]dKHr*ð��éÛe��sOce[�cOPdTiSOfs~ i| TUS�Yeg�NZlHK�pZvô~ | NZSOz�[�NZÓ�TUSefK¢ÂZKR]�hwK|{eNZr#YOgUKh�I��LOTUl�L#LXNZP@SOpqSeáRKR]Qp���KRTifqLm[@cOSOzOKH]~�è[dp�LON}ÂZK%� i / ¸ § ã}ã}ã § � | § z ü ¢FLOKlRpZSOPQ[d]dcOl|[\TUpZS�fqp�KHP�[\Le]dpqcOfZLBNZPFs�K|vxpq]dK��ONZSez<��K�£XSezB[dLXN�[JâTiSOSOp��t]dKHÌ�ceTU]dKHP Õ � fmª�i� �P�[\KHYOP[\p�Å���VØgUKRNZ]dSa~ i| ü �¹[>TUP·K�N�PdTUg�h�PdKHKRS�[\LON�[�[dLOK�vxpqgigUp���TUSef�giKRr#r�NFTirIYOgiTUKHP:[dLXN�[·[\LOKH]dKK�{�TUPQ[.��ø�CPdcel�Lb[\LON�[Y��ã���*ZC ]dKHYO]dKHPdKHS�[dPs~ i| NZSOz_K�N�l�Lg�hÀ·TUP�N�S_TiSm[\KRfZKR]F��Ti[dL � |À@ô ¸ Ç �hÀ�Ç § dw�«ª i  ùe@ü�ÿÍÿs� -�k�v¾ü! Ç|©æ~ | $|Ç#ªaÆeC�Ç��@®²Ç�C_¬�® ! Ç|¦ ¦#ªg�mÐ�� � Ð ñ &XÇ|®Í©(&XÇ04�Ç�ÇQÈq¬UÆ|©æ��ø�CnÆ|¨X¤'&�©�&eªZ©��ã���*sC�4\Ç�§B4�Ç�Æ�Ç|®X©�Ær~ | ªZ®BC�Çdª�¤'&R�hÀ¬UÆ�ªZ®D¬�®X©�Ç =�Ç04N12¬�©(& � |À@ô ¸ Ç �hÀSÇ § dw�«ª[|m  üö þaý²ý!÷�ø r�KH[d��ø�C�suK�zOKH£OSOKRznNZPvxpqgUgip���PRù

îqÛ

Page 81: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

7 �"¸ § Ù�ÖaºIN�SOzg�+¼ § Ù�Ö�ºOÎ7 vxpq]y^`F ÃF�õbôâÙD�hÀ § �õÀ�Ü[¸F��LOKR]QK��Íë�TiP�[dLOK]êZ¸¹[dLD��Tis�pqSONZlRlHT·S�cOr�s�KH]�NZPyzOKH£XSeKRzTiSµr�KHrIrINwð�� ÚeÎ

7 � | § � | Ü�¼-âÙ�Ö�º#NZSOz�C § � | Ü[¸>ôØ^mÖ�º ür�KH[n� | suK�[\LeK gUTUSeK�NZ]�[\LO]QKRPdLepqgUz6vxcOSelH[\TipqSg� | ùõx?zeøRÙ<{}|pÊ x�ô�Ùqø�Ù<{ ��LOTil�L�TUP�]QKRYO]QKRPdKHSm[\KRzsmh [dLOK�NZsup}ÂqK`��ø�C��b�¹[�TiP�PQ[d]\NZTifqLm[dvxpq]��N�]dzÍ[dp�ÂqKH]dTiv�hM[\LON�[e� | ±�lRpqr#YOce[dKRP�Ye]dKRlHTUPQKRgihM[\LeKvxcOSOl|[\TipqS

� üSüSü �(�c¸Ä�^��¼| /�^���| /�����H  üSügü  /�½� | Ü[¸ ü�¹[�TUP�NZgiPdp P�[\]\N�TUfqLm[dvxpZ]Q��NZ]dz�[\peÂqKH]dT�v�h<[\LXNa[y� | °<lHpqrIYece[\KHP�Ye]dKRlHTUPQKRgih�[\LOK�vxcOSOl|[\TipqS

� üSüSü �����Å�^���H /����¢| /�½��¨|  üSüSü  /��� | ÜP�| /�½� | Ü�¼| /�½� | Ü[¸ üÜ�h¬r�KHrIrINBð���Ù�ó�[\LOTiP�TUr#YOgUTiKRPy[dLXN�[n� | NZSOz�~ | NZ]QK#TizOKRSm[dTUl�N�g>pqSMNZgUg(pZv%x?zeø�Ù<{ | ü Ü�hDr�cOg�¸[\TiYOgih�TiSOf6K�N�l�LO�hÀ�NZSez§C6smh`ºD��K_pZse[�NZTiSoNDgUTUSeK�NZ]�[dLO]dKHPdLOpZgUz`vxcOSOl|[\TUpZSo��T�[\L1NZgig�TUSm[dKRfqKH]lRp�KHk#lHTUKRSm[dP���LOTil�LbPdN�[\TiPQ£XKHP � |À@ô ¸ Ç �hÀSÇ § dw�«ª |   üJÍK�[dL�cePyLXN�ÂZK�[dLOK�vxpqgUgip���TiSOf�]QKRPdcegi[F��LOTil�LbTiPFlRpqr#YOgUKHrIKHSm[�NZ]�h�[\pIV�pq]QpqgUgUNZ]Qh<ð��ëºO�ò ý´þaý ó�ó eþxwO c�k�6 ¶ñ &XÇ � ¬�®X®²¥?1 ªZ¯>=�¥?4�¬�©�&�¦D3�¨mÆ�Ç�C6ªaÆIª^8B9;:�¯UÇdª<4�®X¬�®�=6ªZ¯>=�¥?4�¬�©�&�¦ . ¥?4 ©(&XÇ

¤|¯�ªaÆ�Æ�É�y<ø·¦ ¨�Æ|©J4�¨�® . ¥?4 Õ �ÆzägUpZfÇz. �©­¬�¦�Ç�Æ|©�Ç�§XÆRÐ

²�5�4 �«��=@Ew�]on;2=2?#9 G�� l^�H?�� CD9DA���= Ó 9DG�´q?#=P� È G��(;>=@G¶µDCµ×

;>GR?#9Ò�

��S�[\LeKlRpqSePQ[\]QcOlH[dTUpqSwÁQcOPQ[2Ye]dKRPQKRSm[\KHz#[dLOKeäQLONZ]dz\å�zOTUP�[\]dTisOce[dTUpqS�vxpq](JâTiSOSOp��1zOKRYuKRSezOP2pqS�[\LeK[�N�]dfqK|[ygiTUSOKRNZ]�[dLO]dKHPdLOpqgizbvxcOSelH[\TipqS��}��S_[dLOTUP�PQKRlH[dTUpqSB��K�PQLOp��Ë[\LXN�[yNwPdTiSOfqgUK�SON�[\cO]dNZg�zOTiPQ[\]QT�¸sOce[dTUpqSØ�x[dLOK�cOSOT�vxpq]dr zeTUPQ[d]dTisOce[\TipqS_p�ÂZKR]Dx?z�ø�Ù<{}|� �lRNZSn[dL���NZ]�[�[\LOK�Å(KH]dlRKHYe[\]QpqSMNZgUfZpq]dT�[\LOr_��ïÂqKH]QhoPdTirIYegUKBNZ]QfqcOr#KRSm[wPdcek#lHKRPI[dp KHPQ[\NZsOgUTiPdL`[\LeTUPw]dKHPdcOg�[��`JÍKB]QKRÌ�cOTU]QK�[\LOKBvxpqgUgip���TUSOfzOK|£XSOTi[dTUpqS²ù��üvu�wyx���x�ý!wZ c�k�6�¶ñ &XÇ@��KHTUfqLm[}lRpZrIYOgiK|{�Ti[�hD¥ . ª�¯±¬�®²Çdª<4�©(&�4�Ç�Æ'&X¥a¯6C . ¨�®²¤|©�¬«¥a®ªÈo¥?ËZÇ04+x?zeø�Ù<{}|¬UÆ�©�&XÇ�Æ|¦#ªZ¯�¯UÇ�Æ|©�®uªZ©�¨4QªZ¯�®X¨�¦ª$|Ç04PSyÆ|¨X¤'&<©(&eªZ©�Èj¤dªZ®�$|Ç]4\Ç�§B4\Ç�Æ�Ç|®X© Ç�CwªaÆ[�sã���*ZC½12¬�©�&_Çdª�¤'&�hÀ�ªZ®BC�C6ªZ®D¬�®X©�Ç =�Ç04�ªZ®BCI¦#ª�È<x[Ç �"¸�Ç ø üSügü ø<Ç � | Çéø<ÇkC�Ç {aF¯SHÐ

îqÚ

Page 82: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

ú�û:ü�ý´þaü�ÿ c�k�5�ìñ &XÇU8�Ç04\¤�Ç�§u©k4\¥a®BªZ¯>=�¥?4�¬�©�&�¦ ¬UÆF®²¥a©>ªZ®bÇ�lʤ|¬«Ç|®X©c8 9;:ͯUÇdª<4�®X¬�®�=�ªZ¯>=�¥?4�¬�©(&�¦. ¥?4�©�&XÇ ¤|¯�ªaÆ�Æ�¥ . ¯±¬�®²Çdª<4�©�&�4\Ç�Æ'&X¥a¯6C . ¨�®²¤|©­¬«¥a®eÆ�¨�®BC�Ç04�©(&XÇ�¨�®X¬ . ¥?4�¦ìCZ¬UÆ|©k4�¬k$R¨�©�¬«¥a®Í¥a®�x?zeø�Ù�{}| üö þaý²ý!÷�ø J Kb[�N�ÓZK6× § ¸¼ £ / ¸ ��Pdp`NZSmhj×�¸�NZlRlHcO]\N�[dKnLmh�Y�p�[\LOKHPdTUP�r�cOPQ[<NZfq]QKRK_K|{eNZlH[dgiho��Ti[dL[\LeKI[�N�]dfqK|[�lHpqSOlRKHYe[ PQTUSOlHK�r#TUPdlHg�NZPQPdT�£Xl�N�[dTUpqSMpZv�N_PdTiSOfqgUK#K|{eNZr#YOgiK#��pqcOgizØTUr#YOgih6NZSØKH]d]dpZ]]\Na[\K#cOSOzOKH]�[\LOK#cOSOT�vxpq]dr zeTUPQ[d]dTisOce[\TipqSMpZv�N�[�gUKRNZPQ[ ¸¼ £ øÞ×H�µqyNZPQ[\NZzâ�kq�NZP�[�NZz���Ù�ÚqÚZºm �LXN�PlRpZSOPQ[d]dcOl|[\KRzwN�Ü�p�pqgUKRNZS�[\LO]QKRPdLepqgUz�vxcOSOlH[dTUpqS���LOTil�L�LXNZP:��KHTUfqLm[(lRpqr#YOgiK|{�Ti[�h�ó�W0¥| ¿ÁÀ4 | 2 ���¹vO��K[�N�ÓZK@[\LOTiP(N�P·pZcO]�[�NZ]QfqKH[·lRpZSOlRKHYe[��a[\LOKHS�Ti[�vxpZgUgUp���P�[dLXN�[(N�[·gUKRNZPQ[�ó��50¥| ¿ÁÀ4 | 2 cOY�zON�[\K@PQ[dKRYOP(r�cOP�[suK}YuKR]�vxpq]dr#KRz�smh�[dLOK}Å(KR]QlRKHYe[\]QpqS�NZgifqpq]QTi[\LertTiS�pq]dzeKR]�[\p�NZl�LOTiK¢ÂZK}K�{eNZlH[(TUzOKHSm[\Ti£Ol�N�[dTUpqS�PQTUSOlHKÅ(KH]dlHKRYe[d]dpqS_Lmh�YupZ[dLOKRPQTUP��KRTUfZL�[dP�NZ]dK�NZgi��N�h�PFTiSm[\KRfZ]\NZg²NZSOz�K�N�l�LB��KRTifqLm[�TUP�TUSOlH]dK�N�PdKRzBsmh�Na[r#pqPQ[IÙ�zOce]dTUSefnK�N�l�L`Å(KH]dlRKHYe[\]QpqS`cOYuzXN�[dK�PQ[dKRY��_Ü�c�[�[\LeK<NZr#pqcOSm[�pZvlHpqrIYece[�N�[dTUpqS�[\TirIK��LOTil�L�N�Åu��VÊgiK�NZ]QSOTUSef�NZgifqpq]QTi[\Ler*TUP�NZgigUp���KHz�TiP�pqSegihwYupqgih²��õ(ø ¸±   § Yupqgih²��õ(ø�ó?|Z  § ó²ÉÚ0¥| 2 ü²�5¢² �«��=@Ew�]on;2=2?#9 G��&ʽ�;�(; ´q?#=ÌË �N�(;F��A Ê_CD9DEF;2GH?I9Ò�èCD9DA���=

Óä9DG�´q?#=P� È G��·;2=@G¶µDCn;2GH?I9Ò�

��S�[\LOTiPFPdKHlH[\TipqS���K�KHPQ[�N�sOgUTiPdLBN Pdcek#lHTUKRSm[�lRpZSOzOTi[dTUpqS<vxpZ]FN lRg�N�PdPFpZv:[\Le]dKRPQLOpqgizBvxceSOlH[dTUpqSOP[dpsuK�K|kIlHTUKHS�[dgih_gUKRNZ]dSONZsOgUK�smhbÅ(KH]dlRKHYe[\]QpqSDcOSOzeKR]y[dLOK�ceSOTivxpq]Qr zOTiPQ[\]QTUsOc�[\TUpZS_pZS°x?zeø�Ù�{}| ü J KYO]Qp�ÂZK�[dLXN�[ySOKRP�[\KHzbvxcOSOl|[\TipqSOP�P\N�[dTUP�v�h�[\LOTiP�lHpqSOzOT�[\TipqSBNZSOz_[\L�cOP�pqse[�N�TUS�[\LOK�rINZTUS�]QKRPdcegi[�pZv[\LeTUP�PQKRl|[\TUpZS���¢FLOTiP�lHpqrIYegUKRr#KRSm[dP ³ l�LerIT�[d[�êëP�]QKRPQcOgi[([\LXN�[([\LOK�Å(KH]dlRKHYe[\]QpqS#NZgUfZpq]dT�[\LOr'lRNZSOSOp�[KHk#lHTUKRSm[dgihBÅu��VWgUKRNZ]dS�SOKHPQ[dKRznvxcOSelH[\TipqSOPFcOSezOKR]�NZ]QsOTi[d]\NZ]�h<zOTiPQ[\]QTUsOc�[\TUpZSOPR���üvu�wyx���x�ý!wZ c�k�6 Ù! Ç|©-� | $|Ç�ª<¤�¥a¯�¯UÇ�¤|©­¬«¥a®�¥ . &�°�§XÇ04�§u¯�ªZ®²Ç�Æ�¬�®IA | ü 9 . ªZ¦ ¬�¯±°I� §�Í |Î ¸'� |¥ . &�°�§XÇ04¹§u¯�ªZ®²Ç�Æ�¬UÆ<ÆRªZ¬kC�© ¥s$|ÇwfZ]\NZzOcONZg�¬ . ©�&XÇ04\ÇB¬UÆ�Æ�¥a¦�Çn¤�¥a®eÆ|©«ªZ®X©ym¬øìzoÆ|¨X¤'&o©(&eªZ©�©(&XÇ. ¥a¯�¯U¥?12¬�®�=_¤�¥a®BCZ¬�©�¬«¥a®�&X¥a¯6CaÆ�ö . ¥?4 Ç0ËZÇ04�°nÚ�* zeø2Ç0ËZÇ04�°�õT*.Ù_ªZ®BC_Ç0ËZÇ04�°ª&�°�§XÇ04�§u¯�ªZ®²Ç�¬�® � | øªZ©�¦�¥�Æ|©ym'Ú´ó�|E��¥�¥a¯UÇdªZ®tÇQÈ�ªZ¦�§u¯UÇ�Æ;� oLx?zeøRÙ<{}|1¯±¬«Ç^12¬�©�&�¬�®E®�¨X¤|¯±¬kC�ÇdªZ®ÎCZ¬UÆ|©«ªZ®²¤�ÇAÚ*¥ . ©(&XÇ&�°�§XÇ04�§u¯�ªZ®²ÇRÐU9 ¤|¯�ªaÆ�Æ�¥ . ¯±¬�®²Çdª<4�©(&�4�Ç�Æ'&X¥a¯6C . ¨�®²¤|©�¬«¥a®eÆ��+¥?ËZÇ04éx?z�ø�Ù<{}|_¬UÆ�ÆRªZ¬kCI©�¥½$|Ç%=<4dª�CZ¨eªZ¯¬ . ©(&XÇ04\Çw¬UÆ#ªb¦#ª|§m§u¬�®�=�ÏWùN�_Ê �wø®1F&XÇ04\Ç]� ¬UÆ#ª�=<4Qª�CZ¨OªZ¯ . ªZ¦ ¬�¯±°D¥ . &�°�§XÇ04¹§u¯�ªZ®²Ç�Æ'3Æ|¨X¤'&©(&eªZ© . ¥?4�ªZ¯�¯�ÈÒo¤��ø�¬ . Ï� Ȳ �¬UÆy©�&XÇw&�°�§XÇ04¹§u¯�ªZ®²ÇP�Òã2� § C�ø²©(&XÇ|®_����ø Cq %4\Ç�§B4\Ç�Æ�Ç|®X©�Æ�©�&XÇy¯±¬�®²Çdª<4©(&�4�Ç�Æ'&X¥a¯6C . ¨�®²¤|©�¬«¥a®�È üe@ü�ÿÍÿs� -�k���3Ð Ç�Æ|© Ç�C . ¨�®²¤|©�¬«¥a®eÆ<¤dªZ®`$|Çe4\Ç�§B4\Ç�Æ�Ç|®X©�Ç�C�$R°p=<4Qª�CZ¨OªZ¯>¯±¬�®²Çdª<4�©�&�4\Ç�Æ'&X¥a¯6C . ¨�®²¤0/©�¬«¥a®eÆ ¥?ËZÇ04]x?zeø�Ù<{}| ü

u<z

Page 83: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

ö þaý²ý!÷�ø J K cOPdK�[dLOK ]dKHYO]dKHPdKHS�[\N�[\TipqS�KRP�[�NZsOgiTUPQLOKRz�TUS�r:KHrIrIN�ð�� u��nr�KH[nÈ`o���� | N�SOz�gUKH[� ã�� *ÑC6s�KBNDgiTUSOKRNZ]�[dLO]dKHPdLOpqgiz`vxcOSOlH[dTUpqSØ��LeTUl�Lo]dKHYO]dKHPdKRSm[dP�Èà��Ti[dLe�hÀ §Â� ó À�Ü[¸ N�SOzC § fG ¸¼ vxpq]�Pdpqr#K�TUSm[\KHfqKR]yf ü �Xpq]h�¬o`x?zeø�Ù<{ | øXT�v.�Îã�� § S[dLOKRS Sr�cOP�[�suK�NZSbTiSm[\KRfZKR]R�sOce[�PdTUSelRK K¢ÂZKR]�hDTUSm[\KHfqKR]�LXNZP�N<cOSOTiÌ�cOK sOTUSONZ]Qh_]dKHYO]dKHPdKHS�[\N�[\TipqS��:N�[�rIpqP�[�pqSOK;�Toùx?zeø�Ù<{}|l�N�SDP\N�[dTUPQv�hg�¹ã?� § S�vxpq]�NZSmhnfqT�ÂZKRS�Â�NZgUceK�pZvæS ü V�pqSOPdKHÌ�cOKRSm[\g�hq�:SOp�K�{ON�rIYOgiKn�`o x?zeø�Ù<{}|l�N�SbLXN�ÂZK�Ç �ùã���ô1C�Ç�© ¸¼ ø´NZSOz

Ç¥x}��osx?zeøRÙ<{ | ùcÇ �ùã��<ôÒC�Ç6F�º�{[Ç�Ftó?º âÙvxpq]wº * ¸¼ ü ³ TUSOlHK�[dLOK�zOTUP�[�NZSOlHK�vx]Qpqr÷NwY�pqTiSm[%� � [\pw[\LeK�Lmh�Y�KH]dYOgUNZSOKa�Îã�� § C#TiPÓT¢��T Ü[¸ ãÇ ��ã�����ô1C�Ç øe[dLOK�gUKHrIrIN�vxpqgigUp���PFsmh�SOpZ[dTUSOf [\LON�[F��K�LXN�ÂqKIT¢��T § � � £ Ü[¸�   ¸3ñ�¼ § dw�­ó?|Z  ü¢FLOTUP�gUKHrIrIN�KRSOPQcO]dKHP�[dLXN�[�]dKRgUN�[\T6ÂqKHgihnvxK|�ÞY�pZTUSm[\P�l�N�SÍgUTiK lRgipqPdKw[\pB[\pB[\LOK#PdKHYXNZ]dN�[\TiSOf

Lmh�Y�KH]dYOgUNZSOK�vxpq]�NnSOKRP�[\KHzovxceSOlH[dTUpqS�Î}lRpZSOPdKHÌ�ceKRSm[\g�hq��NZP���K<]dcOS1Å·KR]dlHKRYe[d]dpqS²�}r#pqP�[�pZv�[\LeKcOYuzXN�[dKRP���TigUg�l�NZcePdKn[dLOK6NZgifqpq]QTi[\Ler�[\pØrINZÓ�KnPdTifqSOT�£Xl�NZSm[�YO]dpqfZ]dKRPQPR��NZSOzàTi[#��TUgUg�NZl�LeTUK¢ÂZK×�¸�NZlRlHcO]\NZl|hBTUS�Yupqgih�SOpZrITUNZgu[dTUr#KZ�2¢FLOK�vxpqgigUp���TUSef�[\LOKHpq]dKHr0vxpq]Qr�NZgiTUáHKRP�[dLOTUPFTiS�[dcOTi[dTUpqS²ùú�û:ü�ý´þaü�ÿ c���Xz ,(. ɵ¬UÆ�ªN=<4Qª�CZ¨OªZ¯�¤|¯�ªaÆ�Æ�¥ . ¯±¬�®²Çdª<4�©�&�4\Ç�Æ'&X¥a¯6C . ¨�®²¤|©­¬«¥a®eÆ�¥?ËZÇ04wx?zeø�Ù�{}|I©(&XÇ|®©(&XÇ�8�Ç04\¤�Ç�§u©k4�¥a®D¬UÆ�ªª8B9;:¡¯UÇdª<4|®X¬�®�=�ªZ¯>=�¥?4�¬�©(&�¦ . ¥?4yÉì¨�®BC�Ç04�©�&XÇ�¨�®X¬ . ¥?4|¦ßCZ¬UÆ|©k4�¬k$R¨�©�¬«¥a®Í¥a®x?zeø�Ù�{}| üö þaý²ý!÷�ø ¢FLOK�YO]Qp�p�v²TiP2PdTUr#TUgUNZ]:[\p�[\LOK�YO]Qp�p�v²pZv�¢FLOKHpq]dKHr Ù�TiSb�­ÜNZcerb�OÙ�ÚqÚ�zm ���r�KH[L��ã���*ZCsuK�N�gUTiSOK�NZ]�[dLO]dKHPdLOpZgUz�vxceSOlH[dTUpqS���LOTUl�L�]dKHYO]dKHPdKRSm[dP~m ü J K�NZPdPQcOrIK@��Ti[dLOpqce[·gipqPdP·pZvOfZKRSOKH]\NZgiTi[�h[\LON�[���ø�C�LON}ÂZKFs�KHKRS SOpZ]drINZgUTiáRKRz²�aT­� K���T¢��T § Ùq�ZPQp�Ç �½ã ��ô�CBÇHTUP:[\LOK�zOTUP�[�NZSOlHK�vx]Qpqr �#[dp�[\LeKLmh�Y�KH]dYOgUNZSOKZ��Ü�hB~�K|£XSOT�[\TUpZS_ð��iÙ�Ûe�O[\LOKH]dK�TUPFPQpqr#K�lHpqSOP�[�NZSm[]fBPQcOl�Lb[dLXN�[Fvxpq]�NZgig Úµø�zeøO[\LeKYO]QpqsXNZsOTigUT�[�h<[\LXNa[�N�]dNZSOzOpqrìK|{eNZr#YOgiK�zO]dN���SDcOSOTivxpZ]dr#gih�vx]dpZr x?z�ø�Ù<{}|#TUP���Ti[dLOTUS6zeTUPQ[\NZSOlHKÚØp�v[\LOKILmh�YuKR]QYOg�N�SOKj� ã�� § CbTUP�N�[�r#pqPQ[�Ú´Öqó<f ü r�K|[d[\TiSOf^Ú § fe×Rø:��K�LXN�ÂqKI[\LXN�[���Ti[dLYO]QpqsXNZsOTigUT�[�hBN�[�r#pqP�[�×�ÖZó�N�]\N�SOzOpqr÷K|{eNZr#YOgiK�zO]dN���S6vx]dpqr x?zeø�Ù�{}|ITiP���T�[\LOTiSnzOTiPQ[\NZSOlRKAfe×pZv�[dLOK�L�h�YuKR]QYOg�NZSeKZ�Pr�KH[[cLo°x?zeø�Ù<{}|�suK�[dLOK�PQKH[�pZv�K�{ON�rIYOgiKRPP����LOTUl�LIgUTiK���T�[\LOTiSIzeTUPQ[\NZSOlHKfe×�pZv·[\LOK�Lmh�Y�KH]dYOgUNZSOKZ�XPQp�Å2]�¤ ��o�cn¨�Fâ×�Öqó ü

r�KH[w���V�ø�C�V­ F]dKHYO]dKHPdKRSm[�[dLOK�Å·KR]dlHKRYe[d]dpqS�N�gUfqpq]QTi[dLOrbêëPFLmh�Y�pZ[dLOKRPQTUP�N�v�[\KH]sSFcOYuzXN�[dKRP�LON}ÂZKsuKRKRSÍr�NZzeKZ���¹v����V ø�C�V� �TiP�SOpZ[�hqKH[ ×�¸ NZlHlRcO]dN�[\K��([dLOKRSÍ��T�[\L YO]dpqsONZsOTUgiTi[�h6N�[�r#pqP�[�Ù�ÖZóB[\LeKSOK�{�[yK|{eNZr#YOgiK���LeTUl�LnlRNZcOPQKRP�N�SbcOYuzXN�[dK���TUgUg²s�K�TUS c��V�pqSOPdTizOKR]F[dLOK�vxpZgUgUp���TiSOf Y�p�[\KRSm[dT�NZgvxcOSOl|[\TipqS�ù

�IVQ�jí�  § T0í-�¡ôe�VKT ¼ W�jí;C�ô1C�V­  ¼ üu�Ù

Page 84: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

ö�KRl�N�gUgUTiSOf�[dLOK�Å(KR]QlRKHYe[\]QpqSncOYuzXN�[dK�]dcOgiKZ�e��K�PQKRK�[\LXNa[g�aV / ¸R�jí� �ôÒ�aVQ�kí> �TiPÔ �Ø�kí>  § �aV / ¸�ôÒ�aV

§ T0í-�¡ôq�V / ¸xT ¼ W�jí;C�ô1C�V / ¸d  ¼ ôuT0í-�¡ôe�VKT ¼ ôâ�kíÅC�ô1C�V­  ¼§ Õ ó�í-��ã�� � ó�íÅC � ó��V�ã�� Õ ó�C�VW�T'��T ¼ âÙF ó<í(ñ � óO���V�ã �#ô1C�V­ ¬oõjâÙ

qyKH]dK�ñ §�Õ ����ã}�<ô1CZ �N�SOz_[dLOK�[\pqYØ��supZ[d[dpqr< �PQTUfqS_TUSBKRNZl�L � NZSOz Õ TUP�TUS_KH��KRl|[�T�v�[\LeK��S¶_Ù� �¸�P�[2cOYuzXN�[dK�TUP(zOcOK�[dp�Nyv�NZgiPdK�Y�pqPQTi[dT�ÂqK��xv�N�gUPdK�SOKRfqN�[\T6ÂqK� ·K|{eNZr#YOgUK��(¢FLOK�g�NZP�[>TUSeKRÌ�cXNZgiTi[�hNZsup}ÂqK�LepqgUzOP}suKRl�N�cOPdK�T'��T�FuÖ õBvxpq]�NZgUg\��osx?zeø�Ù�{ | ü ³ TiSOlRKw����NZP�r#TUPQlRg�N�PdPdT�£XKRz²����K�Ó�SOp��[\LON�[ � ���Vkã ��ô�C�V­ +©�zeø�NZSOz#LOKHSOlRK Ô �Ø�jí� h©Êó<í·ñ _õdMÙ ü �¹[�TiP2K�NZP�h [\p�PdKHK�[dLXN�[@T�v ��o�c[\LeKRSnñ F¹zIN�SOzbT�vÆ�àÖo_cÞ[dLOKRSbñ F ô"fe× ü ��PyN#]QKRPQcOgi[R� Ô �Í�jí� w©Êõ�WÙ�vxpq]h�Òo_cI��N�SOzÔ �Ø�kí> h©¡õ�tÙ�ô1ó�f�×�íÍvxpq]h�jÖo�c ü³ cOYOYupqPQK�[dLXN�[�zecO]dTiSOfÍ[\LeKDlRpqcO]QPdKDpZv�T�[\P�K�{�KRlRc�[\TUpZSâ[dLOK�Å(KH]dlHKRYe[d]dpqSWNZgUfZpq]dT�[\LOrçLXN�P

rINZzOK%â�ceY�zXNa[\KRP�pqS�K�{eNZrIYegUKRP�TUSbc NZSOz`½�cOY�zON�[\KHP�pqSIK|{eNZr#YOgUKHP�pqc�[\PdTizOK�c ü ³ TUSOlHK ����ø�Cq LXN�ÂqK�s�KHKRS6Sepq]drINZgUTiáRKHzB��K�rIN�hBNZPQPdcOr#K�[dLXN�[eÇkC�Ç5F Ö õ�øuNZSeznLOKHSOlRK�� ò �jí� dF�í ¼ �xõbWÙ�  ü³ TiSOlRK��aV �jí> r�cePQ[�NZg��N�h�P�s�K�SOpqSeSOKRfmNa[\T�ÂZKZ�XT�[FvxpqgUgip���P�[\LXN�[

zRF�âO�xõ�tÙ� ¬O½��xõ�tÙ�ô1ó�fe×�í> ¬ í ¼ ��õ�âÙ�  ü�¹v���K�PdK|["í §

¸k¼ 0 |¶/ ¸32¢ i ± �e[\LeKRSnPQTUr#YOgUT�v�h�TUSOf�[\LOK�NZsup}ÂqK�TUSOKHÌ�cXNZgUT�[�h���K�pqse[�N�TUSzRF�â�ô Ù�Úð ½M Ù�ºZº´��õRtÙ�  ¼óqðO�jfe×�  ¼ ø

vx]dpZrÞ��LeTUl�L�Ti[�vxpqgUgip���P}[dLXN�[FT�v-º�¸ §¸"�×� 0 |¶/ ¸32 ¾¼×¢ 0 i ± 2 ¾ cOYuzXN�[\KHPLXN�ÂqK�suKRKHSBr�NZzeKZ��N�[FgUKRNZPQ[+uqÖ�Ù�ó�pZv

[\LeK�ceY�zXNa[\KRP�r�cePQ[�LXN�ÂqK�s�KHKRS_pqS_K|{eNZr#YOgUKHPFTUS c ür�KH[eº § rINa{Bx�Ù�ºqº@giS h¼ ø�º�¸'{nNZSOzolRpZSOPdTizOKR] ]QcOSOSOTiSOf6[\LOKBÅ·KR]QlRKRY�[\]dpZS¡NZgifqpq]dT�[\LOrævxpZ]

ó�º_ÖZ×#K|{eNZr#YOgUKHPR�Îr�KH[½� ¸�ø��\¼�ø üSüSü zeKRSOpZ[dKb[dLOKbLmh�YupZ[\LOKHPdKHP<��LOTUl�LàNZ]dKbfqKRSOKH]\N�[dKRzjs�h1[\LeKÅ(KH]dlHKRYe[d]dpqSBNZgifqpq]QTi[\Ler zOcO]QTUSOf�[\LOKylRpqce]dPdK�pZv�Ti[dP�K�{�KRlRc�[\TUpZS<pqS�[dLOKRPQK�ó�º_ÖZ×�K|{eNZr#YOgUKHPR�>J KLXN�ÂqK�[\LON�[

Å2]�¤ K�NZl�L��ýÀ(LONZP�KR]d]Qpq]«ø¡×�¨ § Å@]�¤i�xK�NZl�L��ýÀ:LXNZP�KR]Q]dpq] øà×� �Ø�xvxKH��KR]F[dLXNZS�º+cOYuzXN�[dKRPF[�NZÓ�K�Yeg�NZlHK� 2¨�uqó

Page 85: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

Å@]�¤i�xK�NZl�L��ýÀ:LXNZP�KR]Q]dpq] øà×� �Ø�­N�[�giK�NZP�[wº+cOYuzXN�[dKRPF[�NZÓ�K�Yeg�NZlHK� 2¨F Å@]�¤i��vxKH��KR]�[\LXN�S�ºÑcOYuzXN�[\KHP�[\NZÓZK�YOgUNZlRK� nÇ��K�N�l�L��ýÀ·LXNZP�KR]Q]dpq]wøà×� V¨�

Å@]�¤i��N�[�gUKRNZPQ[wºÑcOYuzXN�[\KHP�[\NZÓZK�YOgUNZlRK� nÇ��K�N�l�L��ýÀ·LXNZP�KR]Q]dpq]éøà×� V¨ ü

¢·p�s�pZcOSOz�[dLOK�£X]QPQ[p�v�[\LOKHPdK�lHpqSOzOT�[\TUpZSXNZg´YO]dpZsXNZsOTigUTi[dTUKHPR�Z��K�SOpZ[dK�[\LON�[�lHpqSOzOT�[\TipqSOKRz�pqS�[\LeKK¢ÂZKRSm[ [\LON�[�KRNZl�L¿�ýÀLXN�P�KR]Q]dpq]�N�[�giK�NZP�[�×Hø·[dLOK<K|{�YuKRl|[\KRz`S�cOr�s�KH]�pZvFcOYuzXN�[dKRP�TiP�Na[�giK�NZP�[ó�º ü �ËP�[\]\N�TUfqLevxpZ]Q��NZ]dzbV�LOKH]dSOpZ��supqcOSezbPQLOp���P�[dLXN�[�[\LOTiP£X]dP�[�lRpqSOzeTi[\TipqSXNZg�YO]dpZsXNZsOTigUTi[�h#TiPN�[�r#pqPQ[%­�Öqó ü ¢(p#s�pZcOSOzB[\LOK�PdKHlRpqSeznlRpZSOzOTi[dTUpqSONZg²YO]QpqsXNZseTUgUT�[�hq����K�SOpZ[dK�[\LXN�[

Å@]�¤i��N�[�gUKRNZPQ[wºÑcOYuzXN�[\KHP�[\NZÓZK�YOgUNZlRK� nÇ���KRNZl�LÒ�ýÀ·LXNZP�KR]Q]dpq]Nø¡×� V¨F Å2]�¤i��N�[�gUK�N�PQ[yuqÖ�Ù�ó�pZv�[dLOKéºÑcOYuzXN�[dKRPyNZ]dK�pZSbK�{ON�rIYOgiKRPFTiSgc� ;Ç�xK�NZl�L��ýÀ:LXNZP�KR]Q]dpq]éø¡×� 2¨ ü

qyp���K¢ÂZKR]R�NZP SOpZ[dKRzoK�NZ]QgUTiKR]�T�v�K�NZl�L��ýÀyLONZP KR]Q]dpq]wN�[�gUK�N�PQ[#×Rø�[dLOKRSovxpq]�KRNZl�L1cOY�zON�[\K�[\LeKYO]QpqsXNZsOTigUT�[�h�[\LXN�[y[\LOK ceY�zXNa[\K�TiP�TUS�c TUP�Na[�rIpZPQ[�Ù�Öqó ü ��SOpZ[dLOKR]�N�YOYOgUTil�N�[dTUpqS_pZv�V�LOKH]dSOp��supqcOSOzOP�PdLOp���P�[dLXN�[�[\LOKINZsup}ÂqK�Ye]dpqsXN�sOTUgiTi[�h6TiP�Na[�r#pqPQ[n­�Öqó�ø·NZSOz LOKRSelRK�[dLOK#[dLOKRpZ]dKRr TiPYO]Qp�ÂZKRz����P@NZS TirIr#KRzeT�N�[dK�lHpq]dpqgig�NZ]�h�pZv´¢FLOKRpZ]dKRrãð�� ó�ze�Z��KFLXN�ÂqKF[\LXNa[>[\LOK�Å·KR]QlRKRY�[\]dpZSITiP2N�Å���V

gUKRNZ]dSeTUSOf�NZgifqpq]dT�[\LOr.vxpq]2[\LOKylRgUNZPdP}pZv�SeKRPQ[dKRz�vxceSOlH[dTUpqSOP}cOSOzeKR]�[dLOK�cOSOTivxpZ]dr zOTiPQ[d]dTUsece[\TipqSwpqSx?zeø�Ù�{}| ü

uZ^

Page 86: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

# $ -&% ')(./ Ù

, (.-0/E-�Ú 3 Û ÜE-(39-�ÚÞÝ # (7Ú9')(3Ü5(}6

ß àâáÅãgä % à 02( ä å Ú 6 (3Ü 'Òæ ç è é ê ã�ë~ìEí

î ê ä�ï{ì êÅð åñï ê ë é

ò×óõô öf÷Wø�ù�úûôýü�ömù²þfô ÷�ÿ��£÷��W÷�ÿK÷�ü�ö�� ÷�����ü�²ómü ÷�ÿKóm÷����� ô ö�ôKöm÷� mÿ���ù���� úûô�� ómü ô���� ó� ÷�ô��£÷ ÷�ó�þ�� ø����ó�������ù�ø_ù�ó��ú��mþ ÷ ÿûþ��²ø��Pó�������ù²ø=ÿ��mómó�� ó� aô�����÷"!#� ÿ¡ô öm÷�$¡÷ ÿKü�÷ þfôKÿ�� ó�ù²ó��%&� ómó��'��ù²ø� �� ÿ�� ô ö���ú� ón²ù�ÿ��(���mú)$�*,+-���.f÷ øPø ÷µù�ÿKó�� ó� aúûü ÷ ómù²ÿ���� ú/10,�'�2�£÷�ó�ù²ÿûÿ��'�2���mÿ3!#�_ü��mú�� ÷ ôÇù� �ù�� óõù²ó�4 ���²÷ù5m÷�ô ù�� ø ÷Óÿ��fómó�� ó� �ô��(��÷�ù²ó�ù�ø��Pú�� ú1��!Nù¦óm÷6�¹ù²ø� �� ÿ�� ô ö��7!#� ÿ ù¦ÿû÷ úûôKÿ�� ü�ô ÷��ömù²ø�!"úûþ�ù²ü ÷�ø ÷µù²ÿûó�� ó� þmÿ��� mø ÷��8/"9�öm÷�þmÿ��� mø ÷�:�£÷�ü�� ómú��(m÷�ÿ;� ú�ø ÷µù�ÿKó�� ó� Óù²ó<�mó�==ó��'��ó¤ø(� óm÷µù�ÿWô ömÿû÷ úûö�� ø(>!#�mómü ô���� ó��� ôKö¥ôKömÿK÷�úKö��²ø(8?A@CB<D@ù²ø ú���==ó��'��óBù�ú�ù�ó-EGFIH�J�HLKNM�OQP6KNR�P6FQP�S4T�U�V W�X�Y�UZOQP�[\�mó�m÷�ÿgôKöm÷]�mó���!#�²ÿ���� úûô ÿ��( ��_ô��(�²ó^�²ó{ô öm÷_�mó�� ô�úûþmöm÷�ÿK÷`� óbadc�e/ô���ü����þ ÷ ómúKù�ô ÷�!#� ÿ�ôKöm÷�ÿKù�ô öf÷ ÿ�úûþd÷�ü��f�ü�ó�ùô��mÿû÷��!Çô ö�� ú�þmÿ��� fø ÷�g�W÷�ÿK÷�h���� ÿû÷����mÿ�ø ÷µù�ÿKó�� ó� Xù²ø( �� ÿ�� ô ö��Ñô��Xú��fü ü ÷�÷b� óDù>!@ù�� ÿKø��im÷���ù�ó��� ó� ó���� úK÷<�_�Zm÷�øj/k%)÷úûö��'�âô ö�ù�ô_���mÿ�óm÷6�~ù²ø( ��²ÿ�� ôKö��l� ú�ú��( ó���f�ü ù²ó�ô ø��m!@ù²úûôK÷ ÿ�ôKö�ù²ó�úK÷¢Â²÷ ÿKù²øþmÿû÷¢ÂZ�(���fú�ù²ø( ��²ÿ�� ôKö���ú,��ö�� ü�öbö�ù� ÷� d÷�÷ óDü� ófú��(f÷ ÿK÷�n!#� ÿ�ô ö�� ú�þmÿ��� mø ÷��8/�op�mÿ¦ù²þfþmÿ���ù�ü�öbô��ô ö�� ú�þmÿ��� mø ÷��8q���ö�� ü�ö<� ú�÷6rPô ÿû÷��÷ ø��Iú�����þmø ÷�q�� ó�ô ÿ��Z��mü�÷ úgù� ÷����÷�ôKÿ�� ü�ù}²÷ ÿKù� �� ó� �ôK÷ ü�ömó���h��m÷��ö�� ü�ö<�£÷A ²÷ óm÷�ÿ ù²ø��(s ÷�ù�ó�X÷6rPô ÷�ó�ú��( ó���f�üµù�ó=ôKø��t� óIô öm÷�óm÷IrPô�ü�ömù²þfô ÷�ÿ/

u>v'w xzy`{)|"}>~n�n�d{1��}_y

�d������� ���N���;�����p�2� ���N�

9�öm÷�þmÿ��� mø ÷��:��!Åø ÷µù�ÿKó�� ó� Óù²ó<�mó�==ó��'��ói� ÿ��( �� ó.�×ü�÷ ó�ô ÷�ÿK÷�¥ömù²ø�!"úûþ�ù²ü ÷�� ó8a c ��6 ÷ ó�ù²ü�ü ÷ úûúgô��÷6rfù���þmø ÷ údmÿKù'��ó>�mó���!#� ÿ���ø��4!"ÿ����[ô öm÷5�mó�� ô�úûþmöm÷ ÿû÷�ö�ù²úd ÷ ÷ óIôKöm÷¦ú��� )Áû÷ ü�ô,��!¡ü�� ómú���m÷ ÿKù� mø ÷

���

Page 87: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

ÿK÷�úK÷ ù²ÿKü�ö&D���ù����<q"�'����B.e���ù�����ù�ó�b���Z����q��'������e3��ù²ÿ�móm÷�ÿ�ù�ó�m��÷�ÿKÿ��(�ù�q"�'�� ���e�¡¦÷µù�ÿKómú�q�'���� �eN��� ó� �q��¢��� � eN��� ó� �q��'���¤£Ze¥o�þmþd÷�ÿgù²ó�8¦aù��múûúKø ÷ ÿqz�'������e¥§_÷��mó� nP6RdU�V�¨(q��¢���¤©�[I/"%Zö�� ø ÷ô ö�� úWþfÿ��� mø ÷�ª��ù¢��ù�þmþd÷ ù²ÿ£ô��� ÷aú��( ó���f�üµù�ó=ôKø��Ó÷ ù²ú�� ÷ ÿ£ôKö�ù²ó�ôKöm÷« ²÷ óm÷�ÿ ù²ø þmÿ��� mø ÷��¬��!/ø ÷ ù²ÿKó.�� ó� �ù�óù²ÿ� �� ôKÿ ù²ÿ­��ömù²ø�!"úûþ�ù²ü ÷��mó�m÷ ÿ�ù²ó�ù²ÿ� �� ô ÿKù²ÿ��t�� ú©ô ÿ��� ��fô��(� ó�q���� ó� Óö�ù²ú�úKö��®��ó�ô ö�ù�ôgù�ó��ù²ø� �� ÿ�� ô ö��g!#� ÿ�ô ö�� ú�þfÿ��� mø ÷�¯���fúûôA�múK÷Iùô�ø ÷µù²ú©ô�°�D c ± [¦÷6rfù���þmø ÷�ú�ô��ø ÷µù²ÿûó)ô���ù²ü�ü�mÿKù²ü��m²D³��� ó� �q��'����£�[6/�§.� ófü ÷WôKöm÷� ÷�óm÷ ÿKù²øfö�ù�ø�!"úKþmù²ü ÷�ø ÷µù²ÿûó�� ó� aþmÿ��� mø ÷��ºüµù²ó4 ÷�÷�´�ü�� ÷ ó�ôKø���ú�� ø6 ÷õô��ù²ü�ü�mÿKù²ü��>²��mú�� ó� ¶µ· D c ± [W÷Irfù���þfø ÷ ú]D��Wø(����÷�ÿAP6R�U�V�¨(q¸�'�� ��¤[Iq_ôKöm÷¦ô����Óþmÿ��� mø ÷���ú�ù�ÿK÷¦ó��²ô�ú��( ��ó���f�üµù²ó�ôKø��_���¹d÷ ÿû÷ ó�ôd� ó�ôK÷ ÿ���ú���!;úKù���þmø ÷�ü����þmø ÷Ir.� ô���/)�z�²ó� �ö�ù²ú�ù²ø ú��õúûö��'��óbD���� ó� �q��'��� � [ô ömù�ô� ¤�{ù²þmþmø��Z� ó� >ºÇù��(.��ù�»�ú�þ�� ø��Pó����_� ù²øÅô�����÷Óø�� óm÷ ù²ÿ�þmÿ��� ²ÿ ù����_� ó� ¥ù²ø( �� ÿ�� ô ö��¼D#ºÇù������ù�q�'�� ��¤[��fú�� ó� · D#½¸¾�¿ À�Á6[�ô��(��÷_��ù�ôKÿ���r^���mø ô�� þmø�� üµùô��(�²óBù²ú�ùú��� mÿ����fô�� óm÷ÂD³+�� þmþ ÷ ÿKú��_� ô öÒù�ó�%&� ó��� ÿ ù��q¥�'��  � [)� ô\� ú£þ�� úûú��( fø ÷�ô���ø ÷µù�ÿKó�ô��õù²ü�ü�mÿKù²ü��_²�� óõ· D c¢Ä±�Å ½¸À�¿ À�Á6[ýô��(��÷�úûôK÷ þmú\�mú�� ó� µ· D c ± [�÷Irmù���þmø ÷ ú�/ÇÆ��Wù����Ìö�ù�úgúûö��'��ó�ôKö�ù�ô�ô öm÷]$¡÷ ÿKü�÷ þfôKÿ�� ó¤ù²ø( ��²ÿ�� ôKö��[ø ÷µù�ÿKómú�ô���ù²ü�ü�mÿKù²ü��²��mú�� ó� ȵ· D c ± [�÷6rfù���þmø ÷�ú�ù²ó�5ÿ��mófó�� ó� 8� óDô��(��÷ · D c ı(É [_D³��ù����8q\�'����B�[6/Ê9�öm÷�úK÷�ÿû÷ ú��fø ô úA��!��� ó� �ù²ó�i��ù����âö��²ø(<� ó¥ù�ó���� úK÷I�Ë!"ÿû÷ ÷�úK÷ ôKô�� ó� �� ó>��ö�� ü�ö8� ô,� úgù²úûú�����÷¥ô ö�ù�ôgô öf÷�ø ÷µù²ÿûó�� ó� ù²ø� �� ÿ�� ô ö���óm÷0 ÷�ÿaÿK÷�ü ÷��� ÷�úaù�ó¥÷Irfù���þfø ÷p��ö�� ü�ö�ö�ù�ú, ÷ ÷�óø ù� d÷�ø ÷�Ê� ómü�� ÿKÿû÷ ü�ôKø���/

*aó� ²ø(��� óÒù²ó�2�Äù�� ÿ�b� ó�ôKÿ��Z��mü ÷�5ô öm÷mO6V�UGXQX6H Ì�O�U�R�HjEGK-K¸EGH(X'P]�_�Zm÷�ø�� ó���ö�� ü�öÒô öf÷Iø�ù� ÷ ø��!�÷ ù²ü�ö¹÷6rfù���þmø ÷8 ���²÷ ó§ô��{ô öf÷¤ø ÷ ù²ÿKóm÷�ÿt� ú�ÿ ù�ó������ø��Eù²ó�Í� ó�f÷ þ ÷ ó�m÷�ó�ô ø��2�N� þmþd÷�k��� ôKöþmÿ��� �ù� �� ø(� ô��kÎ7D#*aó� ø���� ó2ù²ó�Ï�Äù�� ÿ��qA�'�� � ¤[I/¬¡�÷ ù²ÿKófú8 �ù�²÷{ù1ú�����þmø ÷¤úûô ù�ô�� úûô�� üµù�ø5hZ�f÷ ÿ�� �ù²úû÷1ù²ø� �� ÿ�� ô ö��Ð!#� ÿ�ø ÷ ù²ÿKó�� ó� <� ÿ��� �� ó.� ü ÷�ó=ôK÷ ÿû÷5ö�ù�ø�!"úKþmù²ü ÷�ú��mó�f÷ ÿ�ôKöm÷t�fó���!#� ÿ��¯�� ú©ô ÿ��( ��.�ô���� ó2� ó1ô öm÷�þmÿK÷�úK÷ ófü ÷Ñ��!�ü ø�ù�úKú���f�üµùô��(�²óEó���� úû÷bD�¡�÷ ù²ÿKófúq5�'���� ¤[6/Ò¦,� ú�ù²ø� �� ÿ�� ô ö�� ÿK÷�hZ��� ÿK÷�úµ· D c ı Ä­Ó Æ�ÔZ¾ËÕ�Ö Ä [�÷Irfù���þfø ÷ úaù²ó�¤ÿ��mómú;� ó�ô��(��÷Cµ· D c ɱ Ä­Ó Æ�ÔZ¾ËÕ�Ö Ä [I׸�£ø(���â÷ ô¦ù²ø³/]D³�Wø����lP6R�U�V�¨(q3�'��� � [ù²ó��+��²öm÷ ó<D³+�� öm÷ ó¸qN�'��� � [Äö�ù� ÷d ��6 ÷�ó�þ�� ø��Pó�������ù²øÇ� ô�����÷Åù²ø( �� ÿ�� ô ö���ú�!#� ÿÅø ÷µù²ÿûó�� ó� aù²ó�ù²ÿ� ����ô ÿKù²ÿ���ö�ù²ø�!"úKþ�ù�ü ÷p� ó�ô öm÷aþmÿK÷�úK÷ ófü ÷���!¡ü ø ù²úKú���f�ü ù�ô���� ó�ó���� úK÷«�mó�m÷�ÿ�ù²óXù²ÿ� �� ô ÿKù²ÿ����� úûôKÿ��( ��fô���� ó�/9�öm÷�� ÿ�ù²ø( �� ÿ�� ô ö���ú�ö�ù� ÷Óô�����÷�ü�����þmø ÷Ir�� ô��Bù�ô�ø ÷µù²ú©ô · D c�Ø�Ù± Ä­Ó Æ�ÔZ¾ËÕ�Ö Ä [IqÄôKö����� ²ö�qÅù�ó�)öf÷ ómü�÷�ù²ÿû÷ó��²ô�þmù²ÿûô�� ü�fø�ù²ÿûø���÷�´�ü� ÷ ó�ô�!#� ÿ�ô öm÷��mó���!#� ÿ��:�� úûô ÿ��( ��_ô��(�²óIþmÿ��� fø ÷�</

*aó�÷¢Â²÷ ót��� ÿû÷�ü�ö�ù²ø ø ÷ ó� �� ó� aó���� úû÷d�_�Zm÷�ø���ù�úÇþfÿ�� þ�� úK÷�_ ¤�����_ø ù²ó�m÷�ÿq���ö��]� ó�ôKÿ��Z��mü ÷�ô öf÷�ó��²ô��(� ó���!5ÚtEGK¸EGR�EGKNHjO_K¸EGH(X'P5!#� ÿ�ö�ù²ø�!"úûþ�ù²ü�÷ ú>D��\�Pø�ù²ó�f÷ ÿq\�'���� ²ù¤[6/¥ò×ó{ô ö�� ú]�_�Zm÷ ø ô öf÷þmÿ��� �ù� �� ø(� ô��EôKö�ù�ôXù�ó2÷Irfù���þfø ÷Ñ� úIø�ù� ÷ ø ø ÷Û� ómü�� ÿKÿû÷ ü ô ø��km÷ ü�ÿK÷ ù²úK÷�ú<��� ô ö�ô öf÷n�� ú©ô�ù²ómü�÷n��!ô öf÷õ÷6rfù���þmø ÷]!"ÿ����Ñô öf÷õúK÷�þ�ù²ÿ ùô�� ó� �ö¤�_þ ÷ ÿûþmø�ù�óm÷�eNôKö�� ú�üµù²þfô��mÿK÷�ú�ô öm÷�� ó�ô���� ô���� ó¥ôKö�ù�ô¦÷Irfù����þmø ÷ ú�ü ø(�²úK÷ ÿ¦ô���ô öf÷_m÷�ü� ú��(�²ó� ����mó�mù²ÿ��Dù²ÿû÷Óö�ù²ÿ�f÷ ÿ�ô��ø ù� ÷ øýü�� ÿKÿû÷ ü�ôKø���/Ê�\�Pø�ù²ó�m÷ ÿ] �ù� ÷�ùÜ�ÝLÞ�ßIàáà®âIã�äjå®æzçjæ�è³é®àêäËèNë�å®ìîíËå�ë�æzï'ìîèjí�éGèjèNå®æ�ç³æ�ð�ß6è¥ë�æ�àñàGß�èNìáÞdâ6é®ç¥âQë�Þ�çjæ�è³é®àñäjè�ðIë�æòß6ï'â�ó'äNä³å®æòí�â6Þ¢ô�æ­Þä³ìáâ6Þ

ä³å�ßQä�ó�æ�ç�ãÇâ�ç³õ�ìáÞ®ö\ß�è�ìáÞ®ö�àñæ�ßIçjìêäjå®õ�æ�ä³ìîíòâ6ó�æ­çËßQä³ìáâ6Þ;è�éGíËå«ß6è�õ�é®àñä³ìáó®àáìáí�ßQäjìñâ�Þ,â6ç�í�â�õ�óGßIçjìáè³â6Þ,â6Þ,ß\óGßIìáç¸âIãZçjæ�ß6àÞ¢é®õ�÷�æ­çËè¸äjß6ø6æ�è�â6Þ®æ�ä³ìáõ�æ"è#äjæ­ó�ù

� £

Page 88: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

Â�ù²ÿ�� ù²ó�ô4��!gô öm÷<$¡÷ ÿKü�÷ þfôKÿ�� ó§ù²ø( �� ÿ�� ô ö��¯��ö�� ü�ö�q\!#� ÿ�ü ÷ ÿ©ô�ù�� ó-�� ú©ô ÿ��� ��fô��(� ómú�q ø ÷µù�ÿKómúÓù²ó-�móZ�==ó��'��ó¤ö�ù²ø�!"úKþ�ù�ü ÷�÷¢Â ÷�ón� óXô öm÷�þmÿK÷�úK÷�ómü ÷���!1�_�²ó��²ô��²ó�� ü¦ó���� úK÷�/�ú�ó�f÷ ÿ�ôKöm÷]�mó���!#�²ÿ��û�� úûô ÿ���� ��fô��(� ó�ö�� ú���� ó���ô�� ó�� ü�ó���� úK÷gù²ø( �� ÿ�� ô ö��3ÿK÷�hZ��� ÿK÷�ú µ· D c ɱ Ù Ó Æ�ÔZ¾ËÕ�Ö Ù [�÷6rfù���þmø ÷�úÇù²ó��ÿ��mómú�� ó�ô�����÷µ· D c¢ü±þý Ó Æ�ÔZ¾ËÕ�Ö ý [6×

�d������ÿ � � ��� �������\�¥��� �%)÷4 ���²÷�ùIóm÷��7ù²ø( �� ÿ�� ô ö�� ��ö�� ü�öbø ÷µù�ÿKómú5� ÿ��( �� ó.�×ü�÷ ó�ô ÷�ÿK÷�¤ö�ù²ø�!"úKþ�ù²ü�÷ ú5�mó�m÷ ÿ¦ô öf÷4�mó���!#�²ÿ���� úûô ÿ��( ��_ô��(�²ói� ó¤ôKöm÷��mó�� ô¦úKþföm÷ ÿû÷�/�ò×ó¤ô öm÷õþfÿK÷ úû÷ ómü�÷���!��_� ó��²ô�� ó�� ü]� ÿ�ü ø ù²úKú���f�ü ù�ô���� óó���� úû÷���mÿWù²ø( �� ÿ�� ô ö�� ÿû÷h���� ÿK÷ ú2µ· D c± Ä­Ó Æ�ÔZ¾ËÕ�Ö Ä [ ÷6rfù���þmø ÷�ú�ù²ó�Iÿ��mómú�� ó�ô��(��÷Òµ· D c ı Ä­Ó Æ�ÔZ¾ËÕ�Ö Ä [6×�op�mÿ�ù²ø� ����ÿ�� ô ö��3ô ö��mú��(��þmÿ��}²÷ ú"� ó�ô öm÷�ô��(��÷�ù²ó�Óú ù���þmø ÷�ü����þmø ÷Ir.� ô�����!cþmÿû÷¢ÂZ�(���múWù�ø( �� ÿ�� ôKö���úò� ó�ô öf÷ü ø ù²úKú���f�ü ù�ô���� óXó���� úK÷�ù²ó�8�_� ó��²ô�� ó�� ü�ó���� úK÷]���.m÷�ø ú�q��(��þmÿ��}²÷ ú,� ó¥ôKöm÷�ô��(��÷¦ü����þmø ÷6r.� ô��t��!ô öf÷5$;÷�ÿKü ÷�þfô ÿ�� ó¥ù²ø( �� ÿ�� ô ö��ª� ó�ô öm÷aó���� úû÷6�j!"ÿK÷ ÷�úK÷ ôKô�� ó� �qfù²ó�Ê� ú\� ómü�����þ�ù²ÿ ù� mø ÷�ô��<D# ��fô����mü�öú�����þfø ÷ ÿ£ô ö�ù²óN[���� ó� �» ú�ù�ø( �� ÿ�� ôKö�� � óIô öm÷�ó���� úK÷p!"ÿû÷ ÷�úK÷ ôKô�� ó� �/

9�öm÷�óm÷6� ù²ø� �� ÿ�� ô ö��û� úa÷IrPô ÿK÷���÷�ø��ú��(��þmø ÷��d� ô�ù� ÷ ÿKù� ÷�ú�ù�úK÷�ô���!£ø ù� ÷ ø ÷�÷Irmù���þmø ÷ úaô���� fô ù�� ó�ùaö¤�Pþ��²ôKöm÷ ú�� ú�²÷ ü�ô�� ÿ/�9�öm÷�ù²ø� �� ÿ�� ô ö��sô ö��múýø ÷µù²ÿûómú)�mú�� ó� aôKöm÷WúKù���÷Wö¤�Pþ��²ôKöm÷ ú�� ú�ü�ø�ù²úûúD@ø�� óm÷ ù²ÿWôKömÿK÷�úKö�� ø�<!#�fómü�ô��(� ómú�[�ù²ú�ô öm÷�÷ ù²ÿKø�� ÷ ÿ�ù²ø( ��²ÿ�� ôKö���ú���÷ ó�ô���� óm÷¥ù� ��} ÷�/

u>v�� � |�� ���� �¢y���|"�����¦�÷�ÿK÷Ñ�£÷¤þmÿ���ÂZ��m÷¤ú�����÷Ñ�fúK÷�!#�føpm÷�fmó�� ô��(� ómú�ù²ó�¹þmÿ��� �ù� �� ø(� ô�� !@ù²ü�ôKú/C%)÷Ñm÷�ó��²ô ÷¤ôKöm÷ѽ¥������÷�ómú���� ó�ù²ø��mó�� ô�úûþmöm÷�ÿK÷] ��

� c�ÔNÆ�� �"!�# a c �%$ ! $�@¶��&D@ó��²ô ÷�ô ö�ù�ô � c�ÔNÆ ø(� ÷�ú;� óia c ù²ó�ó��²ôp� óia c�ÔNÆ [I/�*aóÛEGFIH�J�HLKNM�OQP6KNR�P6FQP�S�T�U�V W�X�Y�UZOQPd��²÷ ÿ � cGÔNÆ� úgù�!#�mómü�ô��(� ó<!"ÿ���� � cGÔNÆ�ô�� �(' �*)¢��&A��ö�� ü�öÑ� ú�m÷�f�óf÷Ñ ¤��ù�ó��²ó�s ÷�ÿ���²÷ ü�ô�� ÿ,+ # adc�e�ô öf÷Â�ù²ø(�f÷5��!-+òD ! [�� ú�����!.+0/ !21 B�ù�ó�8� ú ' �]��!.+3/ !�4 B.×

ò×ó<�\�Pø�ù²ó�f÷ ÿ»�ú�m÷���ù�ó��� ó� `ÚtEGK¸EGR�EGKNHjO«K¸EGH(X'P����.m÷�øjqP÷6rfù���þmø ÷�ú�ü ø�� úK÷�ÿWô���ô öm÷pf÷ ü� ú���� ó ����mó��ù�ÿ���ù²ÿK÷8�_�²ÿK÷¥ø(�(=�÷ ø��Òô��m ÷i��� úûø�ù� ÷ ø ÷�/¹ò×ó�ô ö�� út���.m÷�ø�ô öm÷�ø ÷µù�ÿKó�� ó� {ù�ø( �� ÿ�� ôKö��üµù�ómó��²ô�ù²ü�ü ÷ úûú�ô öf÷¥ó���� úû÷6�Ë!"ÿû÷ ÷>� ÿKù²ü ø ÷6587 ��fôt� ómúûôK÷µù�2���mú©ô��múK÷¥ô öm÷>��� ó��²ô�� ó�� üIó���� úû÷� ÿKù²ü ø ÷95:7 Õ; < ��öm÷ ÿû÷>B>=:Î 4 �@?�©8� ú�ùif�r_÷1ó���� úK÷�ÿKù�ô ÷�/^���6 ÷ ó1ù¤ömù²ø�!"úûþ�ù²ü ÷tm÷�f�óf÷ ¤�Eù^�fó�� ôe ÷�ü�ô�� ÿ3+ # � c�ÔNÆ�)�ô öf÷<��� ó���ô�� ó�� üIó���� úK÷>� ÿ ù�ü ø ÷A587 Õ; < DB+.)DC_[�� ú�m÷�fmóm÷¹ù�ú!#� ø ø(�'��ú��gô öm÷4� ÿKù²ü ø ÷�f�ÿûúûô]mÿ ù¢��ú�ù²ób÷6rfù���þmø ÷ ! ù�ü ü�²ÿ��� ó� Iô��9C9ù²ó�Bô öm÷�ón�N� þmú�ùÊ �� ù²úK÷�

��E

Page 89: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

FHG.IKJ�L*M ÷6rfù���þmø ÷5�²ÿ ù²ü�ø ÷:NO7� ó�ô ÷� ÷ ÿQPSRÍBT JUL"IKJ�LVMh²÷ ü ô�� ÿXW

��/ZY[]\;Ãò@ �:L"[^P`_�[©./ ��ÿKù'�ba ! À )dc'Àfed!"ÿ����gNh7i /Z\@jVL@J�\kGlW�@ Æ m%n mÀpo Æ c�À ! À

qò�( ��mÿû÷ E /��*�19�öm÷8rVsut*vxw*yxtXù�ø( �� ÿ�� ôKö��8/ü��� ói��� ô ön$�ÿ"zköm÷ ù�mú|{ò@~}Î�D�� +9/ ! � [I×�9�öm÷�� ÿ ù²ü�ø ÷A���fô þ��fô ú�a ! ) ' +òD ! [|e;��!ýô öm÷�ü��� ó¤ü����÷ ú«�mþöm÷ ù�mú£ù²ó�����fô þ��fô ú,a ! )d+òD ! [De"��!�� ôÇü����÷ ú"�mþÓô�ù�� ø ú�/�9�öm÷�!#�mómü ô���� ó�}Î��]zêBx)¢��{�� zêB�)��{�üµù�ót ÷ù²ó¤�¤ó�� ó�� ófü ÿK÷ ù²ú�� ó� Ê!#�mómü ô��(�²óBú��mü�öbô ö�ùô¦ô öm÷4�"� ÷�ÿ ù²ø øýþmÿ��� �ù� �� ø(� ô��Iô ö�ù�ô¦ôKöm÷4� ÿKù²ü ø ÷õÿK÷ ô��mÿûómúù���� úKø ù� ÷ ø ø ÷Ó÷6rfù���þmø ÷p� ú\Î�×�0;�²ôK÷aô ö�ùô�ô öm÷aü ø�ù�úKú���f�üµùô��(�²ó�ó���� úK÷p���.f÷ ø�� ú��� fô�ù�� óm÷`��öf÷ ó}θD�/ [«� ú�ÿK÷ ú©ô ÿ�� ü�ôK÷bô��< d÷�ôKöm÷�ü�� ómú©ô�ù²ó�ô]!#�mómü�ô��(� óÂÎ�)Äù²ó�bô öm÷�� ÿ��( �� ó�ù²ø;ó���� úû÷6�j!"ÿK÷ ÷����.f÷ ø)� ú�� fô ù�� óf÷8��öf÷ óÑÎt@7B.×�*�ú«� ó¥ôKöm÷�ú©ô�ù²ó�mù²ÿ�i$�*,+C�_�Zm÷�øNô öm÷�ø ÷µù�ÿKóm÷�ÿ»�ú; �� ù²øz� ú�ô��t���fô þ��_ôù�ö¤�_þ��²ôKöm÷ ú�� ú��>��ö�� ü�ö>��� ô öIþfÿ��� �ù� �� ø�� ô���ù�ô�ø ÷µù²ú©ô�� 'l� ömù²ú�$�ÿD���H�`z��zD ! [ �@�+òD ! [�{U=&²�)N ��_ôó��'�¹ô öf÷�ø ÷µù²ÿûóm÷ ÿ�� ú" ���� ÷ ó�ù²ü ü�÷ úKúÇô���5:7 Õ; < DB+�)|C_[�)�²�) � )_ù²ó�_Î�)Pù²ó��ô öf÷�ø ÷µù²ÿûóm÷ ÿ�� ú£ù²ø ø��'�W÷�ô�����÷Óþ�� ø��Pó����_� ù²ø�� óm½-) Æ ± ) Æ� )¡ù²ó� ÆÆ�ÔZ¾ËÕ ×ò9�öf÷tm÷�þd÷�ó�m÷ ófü ÷`� ó ÆÆ�ÔZ¾ËÕ � ú�óf÷ ü ÷�úKúKù²ÿ��m d÷�üµù��múû÷ô öf÷ ÿK÷,� ú ø ÷ úKúÇù²ó��ø ÷�úKú"� ó.!#� ÿ���ù�ô���� ó�� ó�ôKöm÷�ø ù� ÷ ø(� ó� p��!c÷µù²ü�ö�÷6rfù���þmø ÷gù²ú"ÎÓù²þfþmÿ���ù�ü�öm÷ ú;�@?�©.×

0;�'�Ò�£÷; ���� ÷gú�����÷��múK÷6!#�mø þmÿ��� mù� �� ø�� ô��A!@ù²ü�ôKú/�%)÷�ø ÷�ôK� c�ÔNÆ f÷ ó��²ôK÷�ôKöm÷;�fó���!#� ÿ��Ã�� úûô ÿ���� ��fô��(� ó<� ó � cGÔNÆI×�òË!�+ # � cGÔNÆ�ÿû÷ þmÿû÷ úû÷ ó�ô ú�ôKöm÷�ô ù²ÿ� ²÷�ô�ö�ù²ø�!"úûþ�ù²ü�÷�ù²ó�2W # � cGÔNÆ�ÿû÷ þmÿû÷ úû÷ ó�ô úù�ö¤�Pþ��²ôKöm÷ ú�� ú�ö�ù²ø�!"úKþ�ù�ü ÷�qNôKöm÷ ó^$�ÿD���d����� Ø z�W�D ! [3�@�+òD ! [�{�)dô öf÷�÷ ÿKÿ�� ÿ5��!�W>��� ô öBÿû÷ úûþd÷�ü�ô�ô���+�mó�f÷ ÿ`� c�ÔNÆ )�� úgôKöm÷�!"ÿ ù²ü ô��(�²ó8��! � c�ÔNÆ ��ö�� ü�öø�� ÷�ú,� ó¥ôKöm÷�ú­�Z�_��÷�ôKÿ�� ü����¹c÷�ÿK÷�ómü ÷]��!ýôKöm÷�ô��\�ö�ù²ø�!"úKþ�ù�ü ÷ ú�/n9�ö�� úA� ú�÷µù²ú�� ø��Dúû÷ ÷ óÒô��Ñ d÷`?�D�+�)dW.[D?¡ �)���öm÷ ÿû÷Ê?�DB+�)dW.[�@~ù�ÿKü ü�� ú'DB+2/VW.[A� ú�ô öf÷ù²ó� ²ø ÷5 ÷�ô��£÷ ÷ ó6+¤ù²ó�6W¥×

�/÷�ô8¢ cGÔNÆ m÷ ó��²ô ÷õôKöm÷õú��fÿ�!@ù²ü�÷Óù²ÿK÷ ù`��! ô öm÷��fó�� ô¦úûþmöm÷�ÿK÷ � c�ÔNÆI×dò ô5� ú«=Pó��®��óÛD"úK÷ ÷�qÄ÷�/ê �/�qD³�Wù����<qA�'����B¤[�[�ô ö�ùôA¢ c�ÔNÆ @ ©¡ �c"£Ë¾�?�¤�D�½U?�©�[�)���öf÷ ÿK÷¥¤ � ú�ôKöm÷Bü ø ù²úKú�� üµù�ø« �ù�����ù�!#�mómüI�ô���� ó�/�ú�ú�� ó� � �ù²ú�� ü�þmÿ�� þd÷�ÿûô�� ÷ ú,��!¡ô öm÷] �ù�����ù�!#�mómü�ô��(� ómD�*�ÿûô�� ó�q��'� E�� [�� óm÷�ü ù²óúûö��'��ô ö�ùô¢ cGÔZ¾ ?H¢ c�ÔNÆ @§¦�D�½zÆ�£Ë¾6[6×uqN� ÿ\½ 1 i ).��!�+ # � cGÔNÆ"� ú�f�r_÷�qmôKöm÷ óXù²úWó��²ô ÷�Ê� óÂD���ù����<q��'����B¤[�£÷�ö�ù��²÷

$�ÿ���d����� Ø z�¨©=ª+«/ ! =ª¬U{�@ ¢ c�ÔZ¾¢ c�ÔNÆ /¡­�®¯  �° � '©± ¾�² cGÔZÀ�³ ± ×���

Page 90: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

u>v�´ µb¶�� ·¸��¹�}�|"��{�¶º�9�ömÿ����� ö����fô�ô öf÷�ÿK÷�úûôd��!/ôKö�� ú£ü�ö�ù²þfôK÷ ÿ`+>m÷�ó��²ôK÷ ú�ô öf÷p�mó�� ôS� ÷�ü�ô�� ÿ�� ó`a�c]��ö�� ü�öIÿû÷ þmÿû÷ úû÷ ó�ô úô öf÷�ô�ù²ÿ� ÷�ô ö�ù�ø�!"úKþmù²ü ÷�ù²ó�0Nh7 m÷�ó��²ôK÷ ú�ô öm÷d��� ó���ô�� ó�� ü�÷6rfù���þmø ÷�� ÿ ù�ü ø ÷`5:7 Õ; < DB+�)�� cGÔNÆ [6×9�öm÷�ø ÷µù²ÿûó�� ó� �ù²ø� �� ÿ�� ô ö��¶�£÷�ü� ófú��(f÷ ÿqPüµù�ø ø ÷�0rVs�t*v�w*yxt�q�� ú� ���� ÷ ót� ó9qò�( ��mÿû÷ E /���/�ò ô�� ú£ü�ø ÷µù�ÿô ömù�ôXrVsut*vxw*yxt�D�NO7�)DP�[;�múK÷�ú�P£÷6rfù���þmø ÷ úgù²ó�¥ÿ��mófú,� óIô��(��÷8¦�D�½�P�[6×�d�»d��� ¼b�½�¿¾�À1�¥�­�.�SÁ¯��ÂAÃ.ÄÆÅ%Ç�ÈÊÉÆŪÀÆÁ�˯�8�z��Ì��K¾Í���N�½Áò ôd� ú�� ó�ô ÷�ÿK÷�úûô�� ó� õô���ó��²ô ÷aôKö�ù�ô�ôKöm÷ rVs�tVvxw*yxt�ù²ø� �� ÿ�� ô ö��ª� ú½�²÷ ÿ���ú��(��� ø ù²ÿ�ô��õôKöm÷5$;÷�ÿKü�÷ þfôKÿ�� óù²ø� �� ÿ�� ô ö��</z9�öm÷«�²ómø������¹c÷�ÿK÷�ómü ÷; d÷ ô��W÷�ó�ô öm÷gô�����ù�ø( �� ÿ�� ôKö���ú�q�� ó�!@ù²ü�ôq�� ú ô ö�ù�ô�$;÷�ÿKü�÷ þfôKÿ�� ó� úbOQEGK�X'P6F�ÎGU�R�HfÎ�P6q«�j/�÷�/�q5� ô>�²ómø��Ò�fþ��ùô ÷ ú<� ôKúXü�mÿûÿK÷ ó�ôö��Pþ��²ô öf÷ ú�� ú2WÍ�²óuù²ós÷6rfù���þmø ÷Â��!�Wþmÿû÷�� ü�ô ú4� ómü�²ÿKÿK÷�ü�ôKø���� óÒô ö�ù�ôõ÷6rfù���þmø ÷�/�9�öm÷�rVs�t*vxwVyxt5ù²ø� �� ÿ�� ô ö��8q3� óÒô öm÷Ê��ô öm÷�ÿ�ö�ù²ó��qüµù�ó2 ÷9�Z� ÷6�W÷�§ù²ú�þ ÷ ÿ­!#� ÿ���� ó� Bù²ó-�mþ��ù�ô ÷Ê�²óE÷�� ÷�ÿ��1÷6rfù���þmø ÷�/b�Wù����Þömù²ú�úKö��'��ó1ô ö�ùô� óbô öf÷�ù� múû÷ ómü�÷`��!Wó���� úû÷�ôKöm÷`$¡÷ ÿûü ÷ þ_ô ÿ��²ó1ù²ø( ��²ÿ�� ôKö�� ù²ü�ö�� ÷�� ÷�ú�ù²ób²�� ù²ü�ü�mÿKù�ô ÷�ö��Pþ��²ô öf÷ ú�� ú��� ôKö�ö��� ö�þmÿ��� �ù� �� ø(� ô��1ù�!qô ÷�ÿ · D�½U?�²�¾I[_�mþ��ù�ôK÷ úÂD���ù����<q5�'����B¤[6/Co¦ófü ÷ôKöm÷n$;÷�ÿKü�÷ þfôKÿ�� óö¤�Pþ��²ôKöm÷ ú�� úIö�ù�ú�ù²ü�ö�� ÷H�²÷�¦�D³²Q[�ù�ü ü�fÿ ù²ü6��q�ô ö����� ö�qgù²ó�÷Ir_þd÷�ü�ô ÷�Ϧ�D��@?�²Q[Ó÷6rfù���þmø ÷�ú�ù²ÿû÷ÿK÷�h���� ÿû÷�ô��� ÷�óm÷ ÿKù�ô ÷aù�ú�� ó� ø ÷g÷Irfù���þfø ÷���ö�� ü�öt��� ø ø�ü ù��múK÷�ù²ót�mþ��ù�ôK÷�qfù²ó��öf÷ ómü�÷ · D#½U?�²�ÀQ[÷6rfù���þmø ÷ ú�ù²ÿû÷�ÿK÷h���� ÿK÷�^�"�²÷ ÿ ù�ø øj/�%)÷�þmÿ��"� ÷�� ó^§_÷ ü ô��(�²ó E / � ôKö�ù�ô�� ó¤ôKöm÷Óù� múû÷ ómü�÷_��!£ó���� úû÷ô öf÷ÐrVsut*vxw*yxt�ù²ø� �� ÿ�� ô ö��&ÿK÷h���� ÿK÷�ú µ· D#½U?�² ¾ [,�mþ��ù�ôK÷ ú/p¦;�'�W÷�� ÷�ÿq/ú�� ómü ÷�r(s�t*vxw*y�tXþd÷�ÿ�!#� ÿ���úù²ó_�mþ�mù�ô ÷,� ó�÷H� ÷�ÿ���÷6rfù���þmø ÷�q�� ôÇóm÷ ÷�mú��²ómø��¶µ· D�½%?�²�¾I[�÷6rfù���þmø ÷�ú\� ó�ô��²ô�ù�øj/39�ö�� ú����¹d÷ ÿû÷ ómü�÷ ÷�ô��£÷ ÷ óBôKöm÷�ô��\�Xù²ø( ��²ÿ�� ôKö���ú�ù�þmþd÷ ù²ÿKúgô��` ÷�ù�ÿû÷µù²ø3� óm÷õù²ó�¤ó���ô�Ñ��mú©ô�ù²óBù²ÿ©ô���!@ù²ü�ô;��!�ô öf÷ù²ó�ù�ø��Pú�� ú/Íq��( ��fÿK÷ E / i DÆù¤[d ÿ ù�þmömú�ôKöm÷�÷�ÿKÿ�� ÿaÿKù�ô ÷�úp��!ýôKöm÷�ô��\�Xù²ø( �� ÿ�� ô ö���ú�ù²ú�ù�!#�mómü�ô��(� ó8��!ô öf÷aó����� ÷ ÿ\��!/÷6rfù���þmø ÷ ú��múK÷��/ ò ô�� úÇü ø ÷µù²ÿ"!"ÿ���� ô ö�� ú\ ÿKù²þmö�ôKö�ù�ô�$;÷�ÿKü ÷�þfô ÿ�� óIÿK÷�h���� ÿû÷ ú\!@ù�ÿ��� ÿK÷¦÷Irmù���þmø ÷ ú�ôKö�ù²ó^r(s�t*vxw*y�t�ô���ù�ü�ö�� ÷�� ÷�ù� ��Ò� ÷�óX÷ ÿKÿ�� ÿ�ÿ ù�ôK÷�/

*aó���ô öm÷�ÿò���¹c÷�ÿK÷�ómü ÷� d÷ ô��W÷�÷ ó�ôKöm÷ ô����aù²ø( ��²ÿ�� ôKö���ú¸� ú/ôKöm÷� ÿ;ù� �� ø�� ô��gô���ô�� ø ÷ ÿ ùô ÷�ó���� úK÷�/ò%)÷þmÿ��@�²÷;� ó_§_÷ ü ô��(�²ó E / � ô ömù�ôÆrVsut*vxw*yxt�ü ù²ó�ù²ü�ö�� ÷�� ÷gù²ó�ù²ÿ� �� ô ÿKù²ÿ�� ø���ù²ü�ü�mÿKù�ô ÷gö¤�Pþ��²ôKöm÷ ú�� ú ÷H�²÷ ó� óbô öf÷�þmÿû÷ úû÷ ómü�÷`��!���� ó���ô�� ó�� ü�ó���� úK÷�/t0;��ú��mü�öb ���ù²ÿ ù�ó=ôK÷ ÷�úõù²ÿû÷�==ó��'��óm!#�²ÿA$¡÷ ÿûü ÷ þ_ô ÿ��²ó�qù²ó�Ò� ôIúK÷�÷��úIø(��=²÷ ø��1ô ö�ù�ô�ó�� óm÷Bü ù²ó& d÷Ñ ��Ò� ÷�ó%�©q��� ��mÿû÷ E / i D� N[_ ÿ ù�þmömú�ô öf÷¤÷ ÿûÿ�� ÿIÿKù�ô ÷�ú��!�ô öm÷¥ô����{ù²ø( ��²ÿ�� ôKö���ú���öm÷ ó�ÿ��mók�fú�� ó� )÷Irmù���þmø ÷ ú���� ôKö�ùbü ø�ù�úKú���f�üµùô��(�²ó�ó���� úK÷¥ÿ ùô ÷>��!Îi@ B.×��'B.×zo�óm÷�þ��²úKú��� mø ÷�÷Ir_þmø�ù²ómù�ô���� óÑ!#� ÿ5$¡÷ ÿKü�÷ þfôKÿ�� ó¸» ú]� ó�ù� �� ø�� ô��Xô��Xù²ü�ö�� ÷H�²÷�ù�ø��'�7÷�ÿKÿ��²ÿÿ ùô ÷�� ú¦ô öf÷�!#�²ø ø(�'��� ó� u��ò×ó�� ô�� ù²ø ø��i$¡÷ ÿûü ÷ þ_ô ÿ��²ó�» ú�÷ ÿKÿ�� ÿ�ÿ ù�ôK÷�m÷ ü�ÿK÷µù�úK÷ ú`D@úû÷ ÷3q��( ��fÿK÷ E / i D� N[�[6/o�ómü ÷�ô öm÷�÷ ÿûÿ�� ÿ�ÿ ù�ôK÷A�� þmú, d÷�ø(�'�ºù�ü ÷ ÿ©ô�ù�� ó�ø ÷H�²÷ øjq�ô ö����� ö�q�ôKöm÷õù²ø� �� ÿ�� ô ö��&þ ÷ ÿ�!#�²ÿ���ú;��� úûô��!�� ô ú��mþ��ù�ôK÷ ú;� ói��� úûø�ù� ÷ ø ÷I÷6rfù���þmø ÷ ú/�$;÷ ÿ­!#� ÿ���� ó� ÓôKöm÷ úû÷ºÓ� mù�uÔt�fþ��ùô ÷ ú�ô öf÷ óü ù��múû÷ ú

�  

Page 91: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

Õ¸Ö× Æ�Ø Æ × m Ø m

q��( ��mÿK÷ E /î©x�"*�ó¥÷Ir_÷ ü�_ô��(�²ói��!¡ô öm÷8rVs�t*v�w*yxt�ù²ø� �� ÿ�� ô ö��</ò×óXô öm÷�ó���� úK÷I�Ë!"ÿK÷�÷�üµù²úû÷�÷ ù²ü�öþ���� ó�ô�cdÙ ! Ù ø(� ÷ ú�ù� ��"� ÷�ô öf÷�ô ù²ÿ� ²÷�ôgö¤�_þ ÷ ÿûþmø�ù�óm÷�m÷�f�óf÷Ñ ¤�Iô öf÷� ÷�ü�ô�� ÿÍ+z×N¦g÷ ÿû÷:W_� ú�ôKöm÷ � ÷ ü ô�� ÿ�ù�� ÷�ÿ ù� ÷]��!-c Æ ! Æ�)¢×¢×¢×�)�c m ! m ×ô öf÷�ö¤�Pþ���ô öm÷�ú�� ú�ô��^ d÷�ü����÷¥ø ÷ úûú�ù�ü ü�fÿ ù�ôK÷�qWôKö����� ö¸q�ú��bô öm÷�÷ ÿûÿ�� ÿÓÿ ùô ÷<� ómü�ÿK÷µù�úK÷ ú�ù� �ù�� ó�/ò×ó1ü� ó�ôKÿ ù²ú©ô¢qÇú�� ófü ÷�rVs�tVvxw*yxt��mþ��ù�ôK÷ ú�� óÒ÷H�²÷ ÿ­�1÷6rfù���þmø ÷ÂD@ù²ó�-��� úûô�÷Irfù���þfø ÷ úõù²ÿK÷Ió���ô��� úKø ù� ÷ ø ÷��ú�� ómü ÷�Î 4 �@?�©�[Iq�� ôKú�÷ ÿKÿ�� ÿ�ÿ ùô ÷�ü� ó�ô�� ó��m÷ ú�ô���m÷�ü ÿû÷µù²úû÷�/

�d�»d��ÿ � `ÚÛÃKÄÆÅ%Ç�È�É�Åk� ���N�,�

9�öm÷Ê��m÷µùÑ�mó�m÷ ÿûø��Z� ó� ¤ôKöm÷�rVs�tVvxw*yxt{ù²ø� �� ÿ�� ô ö��¯� ú�� ÷�ÿ��Òú��(��þmø ÷�/BòË!�ô öm÷�ÿK÷`�£÷ ÿK÷Xó��Bó���� úû÷� óXô öf÷�ø ù� d÷�ø(� ó� 4��!ýôKöm÷�÷Irfù���þfø ÷ ú�q�ô öm÷�ó8!#� ÿg÷µù²ü�ö�ø�ù� ÷ ø ÷¥÷6rfù���þmø ÷3a ! )dcHe�� ô��\���mø(8 d÷�ô öf÷üµù�úK÷�ô ömù�ô`+Ð/�D�c ! [ 1 B�×�q¥�fÿûô öf÷ ÿ���� ÿû÷�qPú�� ófü ÷�ô öf÷p�� ú©ô ÿ��( ��fô��(� ó_��! ! � ú��fó���!#� ÿ�� � ó�ô öf÷5�mó�� ôúKþföm÷ ÿû÷�qcôKöm÷��� úûôKÿ��( ��fô���� ó>��!Æc ! �����fø(i ÷�ú­�Z�_��÷�ôKÿ�� ü ù²ø¡ù²ÿ����mó��ôKöm÷Ü�²÷ ü ô�� ÿ�+�) ù²ó�öm÷ ómü�÷ô öf÷�÷Ir_þd÷�ü�ô ÷�2��ù²ø(�f÷5��!�c ! �\���mø�8 ÷�ù��²÷ ü ô�� ÿd��ö�� ü�öþ���� ó�ô ú�þfÿK÷ ü�� úû÷ ø��`� óIô öm÷]�� ÿK÷ ü ô���� óÊ��!+mD@úK÷�÷:q��( ��fÿK÷ E /î©�[I/�ò×óÊ!@ù²ü�ôqf÷H�²÷ ó<� ó�ôKöm÷¦þmÿK÷�úK÷�ómü ÷]��!3�_�²ó��²ô��²ó�� ü�ó���� úû÷¦ù�ô�ù�ó���� úK÷aÿ ù�ôK÷p��!Î 4 �"?�©x)®� ôò� ú;úûô�� ø ø ôKöm÷�üµù�úK÷WôKö�ù�ô�50z�c ! {�� ú;ùÍ� ÷�ü�ô��²ÿ���ö�� ü�ö�þ���� ó�ô úÅþfÿK÷ ü�� úû÷ ø��]� ó�ô öm÷\�� ÿû÷ ü ô��(�²ó��!�+ D@ÿK÷�üµù²ø ø/ô ö�ù�ô�!#� ÿ�ù²ó�÷6rfù���þmø ÷ ! )�ô öm÷�þmÿ��� �ù� �� ø(� ô���ô ö�ùô ! � ú�ü�� ÿKÿ��mþfôK÷Ñ ¤�<�_�²ó��²ô��²ó�� üó���� úK÷`f÷ þ ÷ ó�mú�� ófø��^� óÏ� +6/ ! � [6/Â9�ö��mú�q\��!d�W÷Iü ù²óEù²þmþmÿ��'r.����ù�ôK÷950z�c ! {��£÷Iüµù²óbfmó�Eù²óù²ü�ü�mÿKù�ô ÷�ö¤�Pþ��²ôKöm÷ ú�� ú�ePôKöm÷Xr(s�t*vxw*y�tõù�ø( �� ÿ�� ôKö��ºú��(��þmø����múû÷ ú ú ù���þmø�� ó� �ô���ù²þmþmÿ��¢r�����ùô ÷�ô öf÷÷6r_þ ÷ ü ô ÷^��ù²ø(�m÷5��!�c ! ×

9�öm÷Xù²ó�ù�ø��Pú�� úA��!�ôKöm÷Iù²ø� �� ÿ�� ô ö�� ü� ófú�� ú©ô ú���!�þmÿ��"�Z� ó� �ô ö�ù�ô�!#� ÿ�ù¤ú���� ô ù� mø ÷«��ù²ø��m÷t��!QP�)ô öf÷�ù²ó� ²ø ÷, d÷ ô��W÷�÷ óA+Xù²ó�9W<D@ù²ó��öf÷ ómü�÷aô öm÷�÷ ÿûÿ�� ÿ\��!Nô öf÷aö¤�Pþ��²ôKöm÷ ú�� ú`W.[1��P÷ úWó��²ô�m÷��Z��ù�ôK÷� ÷�ÿ��8���mü�öi!"ÿ����ûs ÷�ÿ���/�%)÷����ôKö�� ú,� ó¥ô�����þmö�ù²úû÷ ú���� óѧ_÷�ü�ô��(� ó E / � /��A�£÷�úûö��'�ºô ö�ù�ô,��� ôKöö��� ö{þmÿ��� �ù� �� ø(� ô��¤ô öf÷0� ÷�ü�ô��²ÿÐWn��� ø ø ö�ù��²÷Iùø�ù�ÿ� ÷�ü����þ�� óm÷ ó�ô�ø��Z� ó� 8� óDô öm÷_�� ÿû÷ ü�ô��(� ó���!

� �

Page 92: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

+�)cù�ó�Ñ� óѧ_÷ ü ô���� ó E / � /á©��W÷�þmÿ��"� ÷�ô ö�ù�ô;��� ôKö¤ö��( ²ö¤þmÿ��� �ù� �� ø(� ô��9W`��� ø øÄö�ù��²÷4� ómø��¥ù�ú���ù²ø øü����þ�� óm÷�ó=ôz��ö�� ü�ö�� ú3�²ÿûô ö��� �� ó�ù�ø²ô���+z×@q�� ó�ù�ø ø���q'� óA§_÷�ü�ô���� ó E / � / i q®�£÷£ü����� �� óm÷�ô öm÷�úK÷Wÿû÷ ú��mø ôKúô��Óü�����þfø ÷�ôK÷�ô öm÷�þmÿ��.��!�/

u>v�Ý · y��Ð��ÞAß«�¢y�¹ {à¶�� áÍâ�ã�äÜåQæ:ãZ·~��¹�}�|��6{�¶º��d��ç;��� � è«À1�����7�2À1��ÀÆ���­�.�^¼m�S�¿¾��½Áp�.Á`�op�mÿ«f�ÿûúûô¦ø ÷����ù�� úaù�ø(�'�W÷�ÿp ����mó�n� ó¤ôKöm÷õ÷Irfþ ÷ ü ô�ù�ô��(� ón��!Æ+�/�D�c ! [I׸%)÷��múK÷õôKö�� ú« ����fó�ø�ùô ÷ ÿ�ô��Óþmÿ��"� ÷�ô öf÷]��ù�� óIÿK÷ ú��mø ô���!;ôKö�� ú�ú��� múK÷�ü�ô��(� ó�/éÆjxêëêëìîíUïÒðòñzP6RSa ! )�c�eÐó6P4E�óRjU�HLK¸P�S9ó�ô9õ�öNP6F�ô�HLKZJ÷Nh7Ñ×Íø�TNP6K2ù�PpT�U�Î�P�RLT�U�R

59z�+0/ZD�c ! [�{�@Ï°hú � ' ©�ν Æ�£Ë¾Oû ×ü:\¡[Ê[�Y|M��/÷ ôý}Î ÷Xô öm÷<�_�²ó��²ô��²ó�� ü�ó���� úû÷>!#�fómü�ô��(� ó�/2%{÷>f�ÿûúûô�úKö��'�âôKö�ù�ô359z�+º/�D�c ! [�{]@°�D Æ�ÔZ¾ËÕc ØÒþ Ä [���!Ð}Î>� ú�ô öm÷�ü� ófúûô�ù�ó=ô;!#�mómü ô��(�²óiÎ�× òË!Åô ö�� ú;� úgôKöm÷�ü ù²úK÷�ô öm÷�ói!#� ÿg÷µù²ü�ö !î# � cGÔNÆ��£÷ö�ù�� ÷�ô ömù�ô,$�ÿ@z�c�@O+òD ! [�{�@¶� ' ÎIù²ó�8$�ÿ"zÿc÷�@O+òD ! [�{�@ Î�)�ú��

59z�+0/ZD�c ! [�{ @ ©Ê/ ¢ cGÔZ¾¢ c�ÔNÆ / ­ Æ� D­� ' ©�Î�[ ±  �° � '>± ¾ ² c�ÔZÀ ³ ±@ ° � D­� ' ©�Î�[­½ Æ�£Ë¾ ­ c � ØÒþ Ä� ±  �° � '>± ¾ ² cGÔZÀ ³ ±��@ ° � D­� ' ©GÎ�[�½ Æ�£Ë¾ ­ c � ØÒþ Ä� ± ³ ±��@ ° ú � ' ©�ν Æ�£Ë¾ û ×

9�öm÷¤úû÷ ü�� ó�¹÷h���ù�ø(� ô��Eö��²ø(mú_ d÷�üµù��múû÷¤ô öm÷8� ó�ô ÷� ÿ ù�ó�Í� ú�óm÷H�²÷ ÿ�óf÷ �ù�ô��Ò�²÷�qgù²ó�§ôKöm÷ôKö�� ÿ�÷h���ù�ø(� ô���ö�� ø�mú�ú�� ómü ÷_D ° � '>± ¾ [�cGÔZÀ�@�¦�D��®[�!#� ÿ�B÷= ± =Ò½zÔNÆ�£Ë¾'×

%{÷�ó��'�ºúKö��'�ºô ö�ùô�50zÿ+�/.D�c ! [�{)� ú,��� ó����_��s ÷Ê��öm÷�ó }Î�D�/ [�� ú�ô öm÷�ü� ófúûô�ù�ó=ô«!#�mómü�ô��(� ó8Î�ק.� ómü ÷

50zÿ+3/�D�c ! [�{¸@ © / ¢ c�ÔZ¾¢ c�ÔNÆ /¡­ Æ� ± /ZD­� ' ©K}Î�D ± [�[æ �° � '©± ¾ ² c�ÔZÀ�³ ± �B

Page 93: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

� ô�ú���´�ü ÷ úgô��Óþmÿ��@�²÷�ôKö�ù�ô­ Æ� }θD ± [ ± /   ° � '©± ¾ ² cGÔZÀ�³ ± =O­ Æ� Î ± /   ° � '>± ¾ ² c�ÔZÀ]³ ± D E /��'[

!#� ÿÇù²ó¤��!#�mómü ô��(�²óý}Î4��ö�� ü�ö�ú ù�ô�� ú­f�÷ ú ô öm÷�ü� ó��� ô��(� ómú"��!¸�_�²ó��²ô��²ó�� ü�ó���� úK÷�/"§.� ómü ÷gô öm÷,�"� ÷ ÿKù²ø øþmÿ��� �ù� �� ø(� ô���ôKö�ù�ô�cÐ�@�+òD ! [�� údÎ�)��W÷�ö�ù�� ÷

Î @ ©�ÎÐ/ ¢ cGÔZ¾¢ cGÔNÆ / ­ Æ�  �° � '>± ¾ ² cGÔZÀ�³ ±@ © / ¢ c�ÔZ¾¢ c�ÔNÆ /¡­ Æ� }θD ± [æ �° � '©± ¾ ² c�ÔZÀ�³ ± ×

9�ö�� ú,����þfø(� ÷�ú£ôKö�ù�ôd!#� ÿ�ù²ø ø�B÷=ª¬l= �*)­ ®� D¡}θD ± [ ' Î�[æ �° � '©± ¾ ² c�ÔZÀ ³ ± @ ­ Æ® D#Î ' }θD ± [�[æ �° � '©± ¾ ² cGÔZÀ ³ ± ×

0;�'�Zø ÷ ô��mú£ü�ö��.� úû÷ͬBú��mü�ö�ôKö�ù�ô3}Î�D ± [ 1 Î4!#� ÿ�B�= ±«4 ¬Bù�ó� }θD ± [`=ÒÎ4!#� ÿƬ 4 ± = ��ק.�mü�öXùܬn���múûôW÷6r.� ú©ô� ÷ ü ù��múK÷�}Î�� úWù�ó�� ó�� ófü ÿK÷ ù²ú�� ó� A!#�fómü�ô��(� óIù�ó��ô öf÷«�"� ÷�ÿ ù²ø ødþmÿ��� �ù� �� ø(� ô����!ò�_� úKü�ø�ù²úûú���fmüµù�ô��(� ó`� údÎ�/)%)÷¦ô öm÷�óö�ù��²÷

­�®� ± /ZD¡}Î�D ± [ ' Î�[  �° � '>± ¾�² cGÔZÀ ³ ± = ­�®� ¬6/�D¡}θD ± [ ' Î�[  �° � '>± ¾�² c�ÔZÀ ³ ±@ ­ Æ® ¬2/ZD#Î ' }θD ± [�[æ �° � '>± ¾ ² cGÔZÀ ³ ±= ­ Æ® ± /ZD#Î ' }θD ± [�[  �° � '©± ¾�² cGÔZÀ�³ ± ×

9�ö�� ú��(��þmø�� ÷ ú5� óm÷h���ù�ø(� ô��-D E /��®[6q;ú��i�£÷�ömù"�²÷�úKö��®��ó5ôKö�ù�ô�59z�+2/�D�c ! [�{d@ °�D Æ�ÔZ¾ËÕc ØÒþ Ä [p!#� ÿ�ù�ó����� ó��²ô�� ó�� üaó���� úû÷p!#�mómü ô��(�²óO}Î�×

0;�'� �£÷Óþmÿ��"�²÷ÓôKöm÷4��ù�� óBÿû÷ ú��fø ô���! ô ö�� ú¦úK÷�ü�ô��(� ó�q���ö�� ü�öb÷ ú©ô�ù� mø�� úûöm÷ ú¦ô ömù�ô�ô öf÷����fô þ��_ô��!XrVs�t*v�w*yxt¥D NO72)|P�[A��� ø ø1��� ô ö{ö��( ö)þmÿ��� �ù� �� ø(� ô��¤ö�ù�� ÷Iùú��� mú©ô�ù²ó�ô�� ù²øÇü����þ��²óm÷ ó�ô�� ó{ô öf÷�� ÿK÷ ü ô���� óÊ��!à+z×éÆjxêëêëìîíUï��òñzP6RUW4@ rVsut*vxw*yxt�D�NO7�)DP�[6×Xø�TNP6K<RLTNP6F�PpH(X,X'EGÚtPÍÎGU�V ö¥P½P"@ µ¦�D cÓ Æ�ÔZ¾ËÕ�Ö Ä [,X�öNOQTRLT�U�R +0/"W�@Ï°hú � ' ©Gν Æ�£Ë¾Oû

 ��

Page 94: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

ù1HLRLT4Y�FQE�óQUVóHLV HLR�ô_U�R\V(P�UGX6R�� '©� ?�©.×ü:\¡[Ê[�Y|Md§.� ómü ÷Í+�/�W�� ú Æ m ô��(��÷ ú£ù�ú���� ��!UP"� ó�m÷�þd÷�ó�m÷�ó=ô�ÿKù²ó����� ��ù²ÿ���ù� fø ÷ ú�q_÷ ù²ü�öÊ��!���ö�� ü�öø(� ÷ úd� ó¥z ' �*)¢��{�)��£÷�ü ù²ó>�múK÷A¦;�P÷�¹��� ó� �» ú�ô ù�� ø¸ ����fó�Iô����� fô�ù�� ó

$�ÿ��p+3/�W9= �© 50zÿ+3/�WV{�= ©�/·÷6r_þ � ' P�D�50zÿ+3/"WV{L[ ¾

  � ×�\�Iø�� óm÷ ù²ÿ�� ô��t��!Å÷Irfþ ÷ ü ô�ù�ô��(� ó�ù²ó�<�/÷����ù E /î©��£÷�ömù"�²÷�ôKö�ù�ô

59z�+0/"WV{�@Ï° ú � ' ©Gν Æ�£Ë¾Oû צ�÷�ómü ÷]� ó>� ÿ�m÷ ÿ�ô��_ ����mó�XôKöm÷�ù� ��@�²÷�þfÿ��� �ù� �� ø�� ô��� ¤� � ?�© )�� ô�ú���´�ü ÷ ú�ô���ô ù�=²÷

P1@Ϧ � ½¦ø(�� Æ�D�� ' ©�Î�[ ¾ � @ µ¦ � ½D­� ' ©�Î�[ ¾ � ×

�d��ç;��ÿ � �Í�¿ÀÆ��� �Û���up�½���SÁ�ÀÆ��¼b�½�¿¾��SÁp�.ÁS�0;�'�2�W÷d���múûô�úûö��'��ôKö�ù�ô)��� ô ö�ö��� öõþmÿ��� �ù� �� ø(� ô��¦ô öm÷Wü����þ��²óm÷ ó�ô1��!�W���ö�� ü�ö4� ú)� ÿ©ô ö��� �� ó�ù²øô��©+ � ú�ó��²ô9� ÷�ÿ��Zø ù²ÿ� ÷�/ 9�öm÷n!#� ø ø��®��� ó� {ø ÷����ùDúKö��'��úXô ö�ùô`��� ôKö2ö��( ö¹þmÿ��� �ù� �� ø(� ô��Eùÿ ù�ó�����¼�N�� ö�ô�� ó��«cÂ��ö�� ü�ö�þmÿ��Pü ÷�÷mú�!#� ÿ÷PõúûôK÷ þmú���� ø ø�÷ ó�Û�mþ�ùô_��� ú©ô�ù^�� ú©ô�ù²ófü ÷>��!DÆù�þmþmÿ��¢r.�(��ù�ôK÷ ø���[QP�Æ�£Ë¾�ù¢�Wù'�i!"ÿ����Ð� ô úaúûô�ù�ÿûô�� ó� Iþ���� ó�ô¢/�%)÷���� ø øò�múK÷õôKö�� ú¦ø ÷����ùÓô��Xþmÿ��@�²÷���mÿdf÷ ú�� ÿK÷�i�mþmþ ÷ ÿ� ����mó�¸/éÆjxêëêëìîíUï���ñzP6R ! Æ�)¢×¢×¢×�) ! m ó6P�HLK�SZP�YNP6K�SZP6KNR�V ô_S�F�U�ù1KAWIF�EGÚ � c�ÔNÆ ×Xø�TNP6K2ù1HLRLT�Y�F�E�óQUVóHLV HLR�ôU�R�V(P�UGX6R\� '©� ?�©6ù�PpT�U�Î�P ����� m�Ùpo Æ ! Ù

����� @ µ· DBP Æ�£Ë¾ [I×ü:\¡[Ê[�Y|M�+�� ómú���m÷ ÿ�ùÊf�r_÷)ü�Z� ÿ��� ó�ù�ôK÷�� #�� ��)¢×¢×¢×�)�½K&�×3§Z� ómü�÷ n mÙpo Æ ! Ù� � ú�ùIú����¯��!�� ó�m÷6�þ ÷ ó�m÷�ó�ô�ÿKù²ó�����¸��ù²ÿ�� ù� mø ÷ úõ÷ ù²ü�öÛ��!;��ö�� ü�ö1ø(� ÷�ú�� ó5ôKöm÷Xÿ ù²ó� ÷¥z ' �*)¢�H{ )��£÷Xü���fø(Òú�����þfø��ù²þmþfø��i¦;�P÷�¹��� ó� �» ú�ô ù�� øò ����fó�Bô��Ê�� fô ù�� óbù` ����mó�^�²óbôKöm÷�þmÿ��� �ù� �� ø(� ô��¥ôKö�ù�ô n mÙpo Æ ! Ù� m÷I��Z��ù�ôK÷ ú�ú��� ó���fmüµù²ó�ô ø��Ñ!"ÿ���� � ôKú�÷Ir_þd÷�ü�ô ÷�l��ù²ø(�m÷���!�s ÷ ÿ���/`9�ö�� ú�ú©ô ÿKù��( ö�ô�!#� ÿ��Wù²ÿ�)ù²þmþfÿ���ù²ü�ö¸qö��'�£÷H� ÷�ÿq¸�Z� ÷ ø(fú�ù��W÷ ù�=²÷�ÿ«f�ó�ù�ø3 ����mó��ô ömù²ói� óf÷]��ö�� ü�ö8�£÷�ü ù²ói�� _ô�ù�� ó<��� ô ö¤ùÓø(� ôKôKø ÷�ù�.��� ô��(�²ó�ù²ø)�\� ÿ�=^ ¤���fú�� ó� ¤ü� ó��� ô��(� ó�ù�ø�þmÿ��� mù� �� ø�� ô�� ÷ ú�ù�ó�5÷6r_þ ÷ ü ô�ù�ô��(� ómú�/Ñ�\�Dü�� ó��� ô��(�²ó�� ó�

 ¤©

Page 95: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

� ó�ôKöm÷p��ù� ó�� ô���f÷;��!N÷µù²ü�ö ! Ù� ).�£÷�üµù�ó�ÿK÷�þmø�ù²ü�÷÷z ' ��)¢��{���� ô ö�ù����mü�öIú���ù²ø ø ÷�ÿ�� ó�ô ÷ ÿ���ù²øcù�ó�ô öf÷ ÿK÷� ¤�>�� fô�ù�� ó¥ùõô��( ²ö=ôK÷ ÿd ����mó�¸/qN� ÿ�ù²ó¤�`f�r_÷�ü�Z� ÿ��� ó�ù�ôK÷�� #ë� ��)¢×¢×¢×H)�½K&�ù²ó��ù²ó¤�A¨ 1 B�)��£÷�ömù"�²÷¦ô ö�ù�ô

$�ÿ���d� ��� Ø zf� ! � � 1 ¨�{ @ © / ¢ c�ÔZ¾¢ c�ÔNÆ /¡­ Ư�� ° � 'l± ¾ ² c�ÔZÀ ³ ±= © / ¢ c�ÔZ¾¢ c�ÔNÆ / � ° � ' ¨ ¾d² c�ÔZÀ ×+�� ómúû÷h��m÷�ó=ôKø��>!#� ÿX¨Â@��«D"ø(�� Æ�£Ë¾ DBP�½%? � [D?�½3Æ�£Ë¾6[�)��W÷�ö�ù��²÷

$�ÿ"z � ! � � 1 ¨.{ 4 �� P�½ ×�/÷�ô��^m÷�ó��²ôK÷�ôKöm÷�÷H�²÷ ó�ô�ô ö�ù�ô÷� ! Ù� � 4 ¨�!#� ÿ5�8=��Æ= P�D@ú���$ ÿ�z �Ê{ 1 � '©� ? � ½Ñ ¤��ôKöm÷��mó��(�²ó ����mó�¥[I/>§.� ómü ÷� ¤�bú��Z�_��÷�ôKÿ��n�£÷�ö�ù��²÷959z n mÙ o Æ ! Ù� � �Ê{�@ B�)��W÷�üµù²ó{ù²þmþmø��¦;�P÷�¹��� ó� �»�úô�ù�� ø¸ ����mó�¥ô����� fô�ù�� óIô öm÷5!#� ø ø(�'��� ó� ��ò!#� ÿ�ù²ó¤�"! 1 B�)

$�ÿ$#&%%%%%� P m� Ù o Æ ! Ù� %%%%% R'!©�(�*)0= ©�/·÷6r_þ � ' P&!Z¾

©�¨ ¾ � ×9;ù�=�� ó� �¨Â@¯µ¦�D#½3ÔNÆ�£Ë¾6[Wù²ó�"!_@ µ· D�D#½�P�[�ÔNÆ�£Ë¾6[�)fôKö�� ú�þfÿ��� �ù� �� ø�� ô��Óüµù�óÊ d÷5 ����mó�f÷> ¤� � ? � ½�ק.� ómü ÷]$�ÿ"z �%{ 1 � '>� ? � ½-)_ô ö�� úd�(��þmø(� ÷ úWôKö�ù�ôd��� ôKöXþmÿ��� mù� �� ø�� ô���ù�ô�ø ÷ ù²úûô�� '>� ?�©�½Ñ�£÷�ö�ù�� ÷� m� Ùpo Æ ! Ù� � ¾ @ µ· ú P½ û ×ú�ú�� ó� �ô öm÷5�mó��(� ó` ����mó�Ê�@�²÷ ÿ ! Æ )×¢×¢×�) ! c )Z�W÷pf�ó��ô ö�ù�ô���� ô ö�þmÿ��� mù� �� ø�� ô���ù�ôWø ÷ ù²úûô«� '¥� ?�©�£÷�ö�ù��²÷ � m�Ùpo Æ ! Ù Æ � ¾ Å /"/"/ Å � m� Ù o Æ ! Ùc � ¾ @ µ· DfP�[��ö�� ü�ö�þmÿ��"� ÷ ú�ô öf÷�ø ÷����ù./

0;�'�m�£÷��múK÷��/÷����ù E / � ô��� ���� ÷Çù²ó��mþmþ ÷ ÿz ����mó�]� ó¦ô öf÷���ù� ó�� ô���m÷)��!Pô öm÷ ü�����þ�� óm÷�ó�ô��!KW_��ö�� ü�ö8� ú�� ÿûôKö��� �� ómù²ødô��3+z×éÆjxêëêëìîíUï�+òñzP6R W @~rVs�t*v�w*yxt¥D NO72)|P�[mU�K�S V(P6R�W(,Üó6PiR#TNP^OQEGÚ«YNEGK¸P6KNR�E�W3W§ù�TZHjOQTÛH(XEGFIR#TNE�JZEGK�U�V¥R�E +.-ËH�¨�P¨�W , @OW ' D�+:/dW.[�+0/�¨÷ø�TNP6K6ù1HLRLTAY�F�E�óQUVóHLV HLR�ô�U�R)V(P�UGX6R)� '2� ?�©Aù�P,T�U�Î�PRLT�U�R $�P�W , $ ¾ @ µ· DfP�[6×

  i

Page 96: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

ü:\¡[Ê[�Y|M�%&� ôKö����fô/ø�� úKú¸��!� ²÷ óm÷�ÿ ù²ø�� ô��,�W÷�ü ù²ó�ú��mþmþ�� úû÷ ôKö�ù�ô/ôKöm÷ýô�ù²ÿ� ÷�ôU�²÷ ü ô�� ÿU+A� ú"D��*)QB�)¢××¢×H)�B¤[�)ú������mÿd ���ù²ø�� ú�ô��� ����mó� P ¾ D�W ¾¾ Å /"/"/ Å W ¾c [I×1�÷ üµù�ø ø�ô ö�ù�ô½P�W�@ n mÙpo Æ cdÙ ! Ù ×�§.� ófü ÷�÷ ù²ü�ö ! Ù � ú\mÿ ù¢��ó`!"ÿ����ò� cGÔNÆ )PôKöm÷p�� úûô ÿ��( ��_ô��(�²ó_��!Ä÷ ù²ü�ö! Ù� � úgú��Z����÷ ô ÿ�� ü�ù�ÿ����mó�ÑB�×�qN� ÿ�ù²ó¤�2� #ý� �*)¢××¢×H)DP�&Óù²ó�3� #�� ©x)¢×¢×¢×�)�½.&õô öm÷A�� ú©ô ÿ��( ��fô��(� ó��!-cdÙz� úd� ó�m÷ þ ÷ ó�f÷ ó�ô,��!Äô öm÷��� úûô ÿ��( ��_ô��(�²ó`��! ! Ù� )fú��4�W÷�ö�ù��²÷�ôKö�ù�ô�ôKöm÷��� ú©ô ÿ��� ��fô��(� ó`��!-cdÙ ! Ù�� ú;�(m÷�ó�ô�� ü ù²ø/ô���ôKö�ù�ô«��! ! Ù� )Nù²ó�öm÷ ómü�÷�ô öf÷��� ú©ô ÿ��( ��fô��(� ó<��! n mÙpo Æ c�Ù ! Ù� � ú«��m÷ ó�ô�� üµù²ø/ô���ô ö�ùô��! n mÙpo Æ ! Ù� !#�²ÿ4�4@Ï©x)×¢×¢×�)�½�×¥§.� ómü ÷

P ¾ W ¾� @ � m�Ùpo Æ cdÙ ! Ù� � ¾!#� ÿ4�4@ ©x)¢×¢××�)�½K)fô öm÷��� úûôKÿ��( ��fô���� óÊ��!.P�¾GD�W�¾¾ Å /"/"/ Å W�¾c [�� úd��m÷ ó�ô�� üµù²ødô���ô ö�ùô;��!� m�Ùpo Æ ! Ù ¾ � ¾ Å /"/"/ Å � m�Ùpo Æ ! Ùc � ¾ ×�/÷����ù E / � �(��þmø�� ÷ úÇô ö�ùô � m�Ùpo Æ ! Ù ¾ � ¾ Å /"/"/ Å � m� Ù o Æ ! Ùc � ¾ @ µ· DfP�[��� ôKöXþmÿ��� mù� �� ø�� ô���ù�ô�ø ÷ ù²úûô�� '>� ?�©x)mú���ôKöm÷�ø ÷��_��ù�� ú�þfÿ��"� ÷��/�d��ç;�» �m�Í� ���DÁ�� ���ª�����65_���\�%�up�3�%)÷�ö�ù�� ÷�ù²ø��_� ú©ôWÿû÷µù²ü�öf÷<���mÿ� �� ù²øj/��\�`�/÷��_��ù E / i q���!��W÷�ô ù�=²÷�PÇú���´�ü� ÷�ó�ô ø���ø�ù²ÿ� ÷�q=ô öf÷ ó��� ôKöXþmÿ��� mù� �� ø�� ô���ù�ô�ø ÷ ù²úûô�� '>� ?�©��W÷�ö�ù��²÷

+3/"W4@Ï°hú � ' ©�ν Æ�£Ë¾hû ×o�ó¥ôKöm÷��²ô öf÷ ÿ�ö�ù²ó��q¥�/÷����ù E /á£õô ÷ ø ø ú��fú�ôKö�ù�ô

$�P�W , $�@ µ· DfP Æ�£Ë¾ [

  �

Page 97: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

��� ôKöXþmÿ��� mù� �� ø�� ô���ù�ô�ø ÷ ù²úûô�� '>� ?�©.×¥+��²ómúK÷�hZ�f÷ ó�ô ø���q¥��� ô öXþmÿ��� �ù� �� ø(� ô��<� '>� �W÷�ö�ù�� ÷$�W , $+«/"W @ µ· � ½ Æ�£Ë¾P Æ�£Ë¾ D�� ' ©�Î�[ � ×0;��ô ÷�ôKö�ù�ô9$�W , $H?�D�+9/¡W.[;� úgô öm÷õô ù²ó� ÷�ó=ôp��!\?�D�+�)dW.[�) ôKöm÷�ù²ó� ²ø ÷� ÷�ô��£÷ ÷�óî+)ù²ó�ºW¥×�*aø ú���q/ù�ú��÷ ó�ô���� óm÷>� ó<§_÷ ü ô��(�²ó E /î©.q_ô öm÷�÷ ÿûÿ�� ÿd��!KW_�mó�m÷�ÿ�� cGÔNÆ )���ö�� ü�öÊ�£÷Af÷ ó��²ôK÷5 ¤�"798�8�:;8�D�W.[�).� ú?�DB+�)dW.[|?¡ 1×�+�� ómúû÷h��m÷ ó�ôKø���q¥��� ôKöXþmÿ��� mù� �� ø�� ô��8� '©� �W÷�ö�ù��²÷

798�8�:;8�D�W.[¯@ �  /xù²ÿûü�ô�ù�ó � $�W , $+3/�W �= $�W , $DB+3/"W.[-/� 

@ µ· � ½ Æ�£Ë¾P Æ�£Ë¾ D�� ' ©�Î�[ �D"ôKöm÷]� óm÷�hZ�mù²ø(� ô���ö�� ø�mú� d÷�üµù��fúK÷õù�ÿKü�ô ù²ózD ! [X= ! !#� ÿ ! 1 B¤[I×¥�\�Xü�ö��Z� ú�� ó� «P\@ µ¦�D c± Ä Ó Æ�ÔZ¾ËÕ�Ö Ä [�£÷�üµù�óú ùô�� ú­!L�Xù²ø ø���!ÅôKöm÷�ÿû÷h���� ÿK÷�ü� ó��� ô��(� ómúgù²ó�8�� _ô�ù�� ó<798;8=:>8�D�W.[ 4 ²×¥9�öZ�fúq��£÷�ömù"�²÷þmÿ��@�²÷¥ô öm÷5!#� ø ø(�'��� ó� �ô öm÷�� ÿK÷��6�?A@ j([]\¡j ê í%ïÒíCBòEGF`X'EGÚtP^ÎGU�V öNP^E�W�PÊ@ µ¦�D c± Ä Ó Æ�ÔZ¾ËÕ�Ö Ä [�);R#TNPÑU�VêJZEGFIHLR#TZÚ rVs�t*vxwVyxt¥D Nh7�)|P�[H(X8UEDGFIH¶V(P�U�FIKNHLKZJmU�VêJZEGFIHLRLTZÚ:W¢EGFÊR#TNPiO6V�UGXQXiE�WiEGFIH�J�HLKNM�OQP6KNR�P6FQP�SÑT�U�V W�X�Y�UZOQPQX�öZK�SZP6F8� cGÔNÆù�TZHjOQT6ö�X'PQX^µ¦�D c± Ä Ó Æ�ÔZ¾ËÕ�Ö Ä [�P&J�U�Ú«Y�V(PQX�U�K�S`F�ö.K�X�HLK µ¦�D c ı Ä Ó Æ�ÔZ¾ËÕ�Ö Ä [�R³HLÚtP]X6R�P�YNX�HLKÂR#TNP,Y�F�PQX'P6K¸OQPE�W�ÚtEGK¸EGR�EGKNHjO5K¸EGH(X'P¨

 ¤£

Page 98: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000

hypo

thes

is e

rror

rat

eK

number of examples

PERCEPTRONAVERAGE

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000

hypo

thes

is e

rror

rat

eK

number of examples

PERCEPTRONAVERAGE

L ì*M

L&N Mq��( ��fÿK÷ E / i �1*�þ ÷ ÿ�!#�²ÿ���ù²ómü ÷�ü�����þmù²ÿ�� ú�� óÊ��!KrVs�tVvxw*yxt�ù²ó�8$;÷�ÿKü ÷�þfô ÿ�� ó�/

$Åù�ÿûôÂD@ù¤[�úûö��'��ú�ô öm÷÷ ÿKÿ�� ÿ�ÿKù�ô ÷�ú`��! rVs�t*vxwVyxt§ù�ó�Ò$;÷�ÿKü�÷ þfôKÿ�� ó&� ó�ó���� úû÷6�Ë!"ÿû÷ ÷�÷6rfù���þmø ÷�úmÿKù'��ó !"ÿ���� � cGÔNÆ ��� ôKö&½û@ �'B�B.×d$Åù²ÿ©ô D# ¥[�úKö��'��ú¥ô öm÷b÷ ÿûÿ�� ÿ¥ÿ ùô ÷ ú>��!�rVsut*vxw*yxtZù�ó�$;÷�ÿKü�÷ þfôKÿ�� óÛ� ó�÷Irfù���þfø ÷ ú�mÿKù¢��ó2!"ÿ���� � c�ÔNÆ ��� ô ö-½C@ �¢B�Bbù²ó�EùDü�ø�ù²úûú���f�üµù�ô��(� óÒó���� úû÷ÿ ùô ÷t��!�Î @ûB�×��'B�×POÇù�ü�öEþfø(�²ô�ÿK÷�þmÿK÷�úK÷ ó�ôKú�ù�óÒù�� ÷ ÿKù� ÷`��!��'B�Bi� ó�m÷ þ ÷ ó�m÷ ó�ô�ÿ��mómú���!�÷ ù²ü�öù²ø� �� ÿ�� ô ö��</

  E

Page 99: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

Q æ RTS ï ç ì UV W Q W é RYX ë[Z]\ ç_^ ëa` V b3ìdceb S ï{ì5ë é Râé f

ê�é é ëhg i êjR k ëFë ^ ï ê�é Z ïml b R ìnZ ê�é

ò×óÒ+Wömù²þfô ÷�ÿ E �£÷úKù'�&ô ö�ù�ô�ùbú�����þmø ÷` ÷�����÷ ô ÿ�� ü¥ù�� ÷ ÿKù� �� ó� Bô ÷ ü�öfó��(h��m÷�üµù²ó2 d÷<�múû÷�ô���� fô ù�� ó�ù�ú��mÿKþmÿ�� ú�� ó� ø��Ó÷�¹d÷ ü ô��Ò�²÷¦ù²ø� �� ÿ�� ô ö��7!#� ÿWø ÷µù²ÿûó�� ó� A� ÿ��( �� ó.�×ü�÷ ó�ô ÷�ÿK÷��ö�ù²ø�!"úKþ�ù�ü ÷ ú\�mó�m÷�ÿô öf÷_�mó���!#� ÿ��¯�� ú©ô ÿ��� ��fô��(� ó�/�%)÷�ó��'��÷6rPôK÷ ó�{ôKö�� ú�ù�� ÷�ÿ ù� �� ó� �ù²þmþmÿ���ù²ü�ö{ô��8�� _ô�ù�� ó{ù¥óm÷��!@ù���� ø��4��!;ù²ø( �� ÿ�� ô ö���ú1!#� ÿWø ÷µù²ÿûó�� ó� �ø(� óm÷µù²ÿÇô ömÿû÷ úûö�� ø(`!#�fómü�ô��(� ómú\�mó�m÷�ÿd���mü�öÊ��� ÿK÷« ÷ óm÷�ÿ ù²øü�²ó��� ô��(� ófú/¬9�öm÷)óf÷��Fù²ø� �� ÿ�� ô ö���ú_�\� ÿ�= ¤�Zü����� �� ó�� ó� Eù- ÷ óf÷ ÿ ù�ø(�(s�÷h� ÷�ÿKú���� óÏ��!�ô öf÷rVs�tVvxw*yxt�ù²ø� �� ÿ�� ô ö�� ��� ô öÓù�þ��®�£÷ ÿ­!#�mø�ù²ó�4�£÷ ø ø��×ú©ô����� ÷�ø ÷µù�ÿKó�� ó� aô öf÷� ÿ­�õô ÷�ü�ömó��(h��m÷�==ó��'��óù²ú¥ó6E'E'X6R�HLKZJ�/7%)÷Bþmÿ��@�²÷bþd÷�ÿ�!#� ÿ���ù²ófü ÷n ���ù�ÿ ù²ó�ô ÷�÷ ú`!#� ÿt���mÿIóm÷6�Âù²ø� �� ÿ�� ô ö���ú���ö�� ü�ö2ù²ÿû÷ÿK÷���ù�ÿ�=ù� mø��ú��(��� ø�ù�ÿ�ô��>=Pó��®��ó^ ����mó�múp!#�²ÿ5� ómø(� óm÷�ø�� óm÷ ù²ÿ�ôKömÿK÷�úKö��²ø(bø ÷ ù²ÿKó�� ó� Xù²ø� �� ÿ�� ô ö���ú� ófü ø(���� ó� �$;÷ ÿûü ÷�þfô ÿ�� ó¤ù²ó�>%&� ómó��'��/19�ö�� ú�ú��(��� ø ù²ÿ�� ô��ÓúûôKÿ�� ó� ²ø���ú��� � ²÷ úûôKú�ô ö�ù�ô�ôKöm÷ úû÷5�W÷�ø ø��úûô����� ÷<� ómø(� óm÷¦ù²ø� �� ÿ�� ô ö���ú\� óXú�����÷¦úK÷�ómúK÷�ü� ÿûÿK÷�úKþ�� ó�Xô���� ómú©ô�ù²ófü ÷ ú,��!ò ��Z� ú©ô�� ó� �/

o v'w xzy`{)|"}>~n�n�d{1��}_y

p ������� 5lp�ÏÃ.Ä�ÅÊÇÆÈ�ÉÆÅO�������\�¥��� � qÒ���«��������.Ë1�÷ üµù�ø ø_ôKö�ù�ôýôKöm÷`rVs�t*v�w*yxt�ù²ø� �� ÿ�� ô ö�� m÷ úûü ÿ��� d÷��� ó�+Wö�ù²þ_ô ÷ ÿ E �\� ÿ�=Pú1� ó�ù,� ÷�ÿ���ú��(��þmø ÷���ù¢��/9�öm÷�fmÿKúûô ú©ô ÷�þ4��!�ô öm÷�ù²ø( ��²ÿ�� ôKö��C� ú;ô��5fÿ ù¢��ùaúK÷ ô1��!�ø�ù� d÷�ø ÷�÷6rfù���þmø ÷�úýù²ó�ÊK¸EGFIÚ�U�V HrGPÄ÷ ù²ü�öóm÷� �ù�ô���� ÷�÷Irmù���þmø ÷4 ¤�BÿK÷6��÷ ü ô�� ó� <� ô�ô öfÿ����� ²öbôKöm÷4� ÿ��( �� ó�/A9�öf÷�ù�ø( �� ÿ�� ôKö��âô öm÷�óDü�����þ��fô ÷�úô öf÷_ ÷�����÷�ô ÿ�� ü�ù��²÷ ÿ ù� ÷t��!Çô öf÷�þ�� ú�� ô���� ÷�÷6rfù���þmø ÷ ú�ù²ó�^�múK÷�ú�ôKö�ù�ô�ù�� ÷ ÿKù� ÷�� ÷�ü�ô��²ÿ�ù²ú�ô öf÷ö¤�Pþ��²ôKöm÷ ú�� ú�ø�� óm÷ ù²ÿ£ôKömÿK÷�úKö�� ø�`!#�mómü ô��(�²ó�/)%{÷aú ù¢�sôKö�ù�ô�ôKö�� úWú��(��þmø ÷�ù²ø� �� ÿ�� ô ö��ª� úWú��mü ü�÷ úKú­!#�mø��öm÷�ó1ôKöm÷�ô ù²ÿ� ÷ ôõü�� ómü ÷�þfô�� ú�ù²ó � ÿ��( �� ó.�×ü�÷ ó�ô ÷�ÿK÷{ö�ù²ø�!"úûþ�ù²ü�÷�ù²ó�5ôKöm÷�÷Irmù���þmø ÷ ú��fúK÷b!#�²ÿ

  �

Page 100: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

Ø¥Æ

sutwvyx{z

sutwv}|~x{z�� sut�v}|�����x{z

Ø.¾

q��� ��mÿK÷ � /��*�1*C�\� ÿûúûô�� üµù²úû÷Amù�ô�ù�úK÷ ô;!#�²ÿ�ô öm÷8rVs�tVvxw*yxt�ù²ø( �� ÿ�� ô ö��</9�öm÷Xô�ù²ÿ� ÷�ô�ü�� ómü ÷�þfô_� ú4m÷6f�óm÷k ��ÒôKöm÷Xö�ù²ø�!"úKþ�ù²ü�÷ ! Æ 1 B�×19�öm÷<�� úûô ÿ��( ��_ô��(�²ólCñþmø ù²ü ÷�ú�£÷�( ²ö=ô ƾ 'd� � ó¤ôKöm÷õþ�� ú�� ô��Ò�²÷�÷Irmù���þmø ÷ÊD � )&�¥[�ù²ó�þmø�ù²ü�÷ ú«�£÷�� ö�ô ƾ Å � � óô öf÷�þ�� ú�� ô��Ò� ÷÷6rfù���þmø ÷bD � ) ' �N[_��öm÷ ÿû÷ � � ú�ù{þ��²ú�� ô��Ò�²÷^��ù²ø��m÷úKù�ô�� ú�!L�Z� ó� � R � ¾ ?�Dj©;� ¾ [6×�9�öm÷Bù��²÷ ÿ ù� ÷� ÷�ü�ô�� ÿ;�fó�m÷ ÿ�ôKö�� úd�� úûôKÿ��( ��fô���� óÊ� úQW�@ÃD � ) ' © � �¥[���ö�� ü�ö�ö�ù�úg÷�ÿKÿ�� ÿ ƾ 'n� �mó�m÷�ÿ�C>×ø ÷ ù²ÿKó�� ó� �ù²ÿK÷�fÿ ù¢��ón�mó���!#� ÿ���ø��`!"ÿ����Ñô öm÷��²ÿ��( �� ó.� ü ÷ ó�ôK÷ ÿK÷�Ñ�mó�� ô�úûþmöm÷ ÿû÷4� ónadc.×�9�öm÷�þfÿ��Z��!��!�ü�� ÿKÿû÷ ü ô óm÷�úKú�� ó +Wö�ù²þ_ô ÷ ÿ E m÷�þd÷�ó�múõü�ÿ��mü���ù²ø ø��n�²ó)ô öf÷�ú��Z����÷ ô ÿ­�Â��!�ô öf÷t�mó���!#� ÿ��¯�� ú­�ô ÿ��( ��_ô��(�²ó>��ö�� ü�ö÷�ómú��fÿK÷ úgô ö�ùô�ôKöm÷�ù��²÷ ÿ ù� ÷:�²÷ ü ô�� ÿ,��� ø ødô ÷ ó��ô���þ���� ó�ô,� óXô öm÷�÷6rfù²ü ôaúKù���÷�� ÿK÷ ü ô���� óXù²ú�ô öm÷�ó��²ÿ���ù²øÊ� ÷�ü�ô�� ÿ�ô��Óô öm÷�úK÷�þ�ù²ÿKù�ô�� ó� Óö��Pþ ÷ ÿûþmø�ù²óf÷�/

*§ó�ùô��mÿKù²ø.h��m÷�úûô��(� ó�� ú¡ôKöm÷\!#�²ø ø(�'��� ó� u�¸��ö�ùôýüµù�ó� d÷£ú ù���õù� ����fô;ô öm÷£þd÷�ÿ�!#� ÿ���ù�ómü ÷���!�ô öf÷rVs�tVvxw*yxt�ù²ø( ��²ÿ�� ôKö��7��!�� ô"� ú ÿ��fót�mó�f÷ ÿ£ù�ó��²óZ�fó���!#� ÿ��Ã�� úûô ÿ��( ��_ô��(�²ó��`* ������÷ ó�ô¢»�ú�ô ö����� ö�ôúKö��®��ú�ôKö�ù�ô�ôKöm÷�ù�� ÷ ÿKù� ÷��²÷ ü�ô�� ÿd�mó�f÷ ÿgùõó�� ó��mó���!#� ÿ��¬�� úûô ÿ��( ��_ô��(�²ó�óm÷�÷Xó��²ô�þ���� ó�ô�� ó�ô öf÷ú ù���÷��� ÿK÷ ü ô���� óXù²ú�ô öm÷ �²÷ ü�ô�� ÿgü� ÿûÿK÷ úûþ��²ó��� ó� �ô��ÓôKöm÷¦ô�ù²ÿ� ÷�ôgö�ù�ø�!"úKþmù²ü ÷�eN�_� ÿû÷�"� ÷�ÿq�÷�� ÷ óùúKø��( ö�ô;m÷H�Z� ù�ô���� ó<!"ÿ����âô ö�� úgô ù²ÿ� ²÷�ô�� ÷ ü ô�� ÿaüµù²ó¤ü ù��múû÷�ô öm÷�ÿK÷�ú��mø ô�� ó� �ö¤�Pþ��²ôKöm÷ ú�� úaô���ö�ù��²÷�ùö��� ö÷�ÿKÿ��²ÿ�ÿ ù�ôK÷�/,q��( ��fÿK÷ � /��� ��Ò�²÷ ú�ù�óB÷6rfù���þmø ÷]��!Çùt�� úûôKÿ��( ��fô���� ó8�"�²÷ ÿ�a ¾ !#� ÿ;��ö�� ü�ö¤ô öf÷ù�� ÷�ÿ ù� ÷ �²÷ ü�ô�� ÿ�þ���� ó�ô ú�� óIóm÷µù�ÿKø���ô öm÷¦úKù���÷p�� ÿû÷ ü ô��(�²óXù²ú�ô öm÷�ô ù²ÿ� ²÷�ôQ� ÷�ü�ô�� ÿqN ��_ô�ô öm÷¦÷�ÿKÿ��²ÿÿ ùô ÷���!¡ô öm÷�ù�� ÷�ÿ ù� ÷ �²÷ ü�ô�� ÿgö��Pþ��²ô öf÷ ú�� ú;� ú�ù²ø(��� ú©ô�£�B(�`/

 � 

Page 101: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

p ������ÿ 5lp�ÏÃ.Ä�ÅÊÇÆÈ�ÉÆÅO�������\�¥��� � qÒ��Ë5�3�.� ��Ë9�öm÷�÷Irfù���þfø ÷���!Xq��� ��mÿK÷ � /���ú��� � ÷�úûôKú�ô ö�ùô�ô öf÷0rVs�tVvxw*yxtDù�ø( �� ÿ�� ôKö�� üµù�ó{þ ÷ ÿ�!#�²ÿ�� h���� ôK÷þ��.� ÿûø���!#� ÿýó�� ó��mó���!#�²ÿ��7�� ú©ô ÿ��� ��fô��(� ómú�/�¦,�'�W÷�� ÷ ÿ�qZ�£÷d��� ø ø_úû÷ ÷�ô ömù�ô"� ó�!@ù�ü�ôýôKöm÷QrVsut*vxw*yxtõù²ø�� �� ÿ�� ôKö�� O�U�K� ÷"�múK÷��ù²úNôKöm÷"=²÷6�¦ü����þ�� óm÷ ó�ôz� ó�ù�ó�ù�ø( �� ÿ�� ôKö��Í!#� ÿ/ø ÷ ù²ÿKó�� ó� �ø�� óm÷ ù²ÿcôKömÿK÷�úKö�� ø�!#�mómü ô���� ómú,�mó�m÷�ÿaó�� ó��mó���!#� ÿ��û�� ú©ô ÿ��� ��fô��(� ómú�/�9�öm÷õ÷Irfù���þfø ÷� ��Ò�²÷ óÑ� ó2qò�( ��mÿû÷ � /���ù²ü�ô���ù²ø ø��ÿK÷�þmÿK÷�úK÷�ó=ôKúÓù8�\� ÿKú©ô��×ü ù²úK÷Iúûü ÷�ó�ù²ÿ����8!#� ÿ�ô öm÷3rVsut*vxw*yxt)ù²ø� �� ÿ�� ô ö��^��ù²ú��£÷�úKö��'�¬� ób§_÷ ü ô���� ó� / i q3�mó�m÷�ÿ�ú���� ô�ù� mø ÷Óü�²ó��� ô��(� ófú�ôKöm÷«r(s�t*vxw*y�tbù²ø� �� ÿ�� ô ö�� ö�ù²ú�ô öm÷�ü ÿ��mü� ù²ø�þmÿ��²þd÷�ÿûô��bô ö�ùôô öf÷�÷�ÿKÿ�� ÿ�ÿ ù�ôK÷���!�� ô ú�ö¤�Pþ���ô öm÷�ú�� ú���� ø ø¸ ÷�ø ÷ úûú�ô ö�ù²óÂ�@?�©�!#�²ÿ]U�K�ô¦þmÿ��� mù� �� ø�� ô����� úûô ÿ��( ��_ô��(�²ó�/

*aø� �� ÿ�� ô ö���ú���ö�� ü�ö¹ö�ù��²÷¤ô ö�� ú` ���ù²ÿKù²ó�ô ÷�÷Ñ��!����fôKþd÷�ÿ�!#� ÿ��_� ó� 5ùÏÓûÿ ù²ó����� ��m÷ úûú|ÔEù²ÿû÷==ó��'��ó{ù�úÐù�P�U;�`V(P�U�FIKNHLKZJ8U�VêJZEGF6HLRLTZÚ�X6/]§.�mü�ö)ù²ø� �� ÿ�� ô ö���úd�W÷�ÿK÷4f�ÿûúûô5�� úKü��múKúû÷^ ��Ñ¡¦÷µù²ÿûómúù²ó��ºÇù²ø(� ù²ó�ô`D�¡¦÷µù²ÿûómú�ù²ó��ºÇù²ø(� ù²ó�ô¢q��'��� � [�ù²ó�{ö�ù�� ÷Iú�� ómü ÷_ d÷�÷ óÒú©ô����� ÷b� ó�ô ÷ ófú��Ò�²÷ ø��m� óø ÷ ù²ÿKó�� ó� �ô öf÷� ÿ­��/\%)÷�úKö��'� � ói§P÷ ü�ô��(� ó � / � ôKö�ù�ô, ¤��ù²þmþmø��.� ó� ���÷ ô ö��Zmúd!"ÿ���� ��Z� ú©ô�� ó� �q�� ô� ú�þ�� úKú��( mø ÷¦ô���ü� óV� ÷�ÿûôgô öm÷:rVs�tVvxw*yxtIù²ø( ��²ÿ�� ôKö��:� ó�ô���ù²ó¥÷�¹d÷ ü�ô��Ò�²÷�q ÷�´�ü� ÷ ó�ô�ù�ø( �� ÿ�� ôKö��¬!#�²ÿø ÷ ù²ÿKó�� ó� �ø�� óm÷ ù²ÿWô öfÿK÷ úûö�� ø�<!#�mófü�ô���� ómú�ô���ù�ÿ� �� ô ÿ ù�ÿ��Iù�ü ü�fÿ ù²ü6��/

p �����» �-�,��� ����Á������>À1����Ë è���Áp��À1��5lp�N�z�up�½��Ë�è���À1�]Á5��Á��O�����\���¥��� � �%Zö�� ø ÷Xô ö�� ú�ù²þfþmÿ���ù�ü�öÍ��!� ��.� ú©ô�� ó� DôKöm÷2rVs�t*vxwVyxtEù²ø( ��²ÿ�� ôKö��l� ú�ü�� ómü ÷�þfô��mù²ø ø��Eù²ó�¹ù²ø� ����ÿ�� ô ö���� ü ù²ø ø��3�²÷ ÿ­�i���¹c÷�ÿK÷�ó=ô;!"ÿ����ÌôKöm÷���t*v ��t=�{� v0�>�bù²ø� �� ÿ�� ô ö��<q��£÷�÷ ú©ô�ù� fø(� úûöþ ÷ ÿ­!#� ÿ���ù²ómü�÷ ����mó�múp!#�²ÿ�ô öf÷õóm÷��7ù²ø� �� ÿ�� ô ö�� ��ö�� ü�öbù²ÿû÷õÿK÷���ù²ÿ�=ù� mø��¥ú��(��� ø�ù�ÿ�ô���ô öf÷� ����mó�mú5 ���� ÷�ón ��ô öf÷;$¡÷ ÿûü ÷ þ_ô ÿ��²ó`+�� óV�²÷ ÿ� ²÷ ómü�÷«9�öm÷�� ÿK÷��8/�%)÷�ô ö��mú ÿK÷6!"÷ ÿ�ô������fÿ�óm÷��k ��Z� úûô�� ó� G�� �ù²úû÷Óù²ø� ����ÿ�� ô ö��[ù²ú�ù�D*FIHkU�K�U�V(E�J�öNP���!3$¡÷ ÿKü�÷ þfôKÿ�� ó¸/SqN�mÿ©ô öm÷�ÿ���� ÿK÷�qZ�£÷¦úKö��'�2ô ö�ùô�ùõú�����þfø ÷X�²ù�ÿ���ù�ó=ô��!�ô öf÷�rVsut*vxw*yxt§ù²ø( �� ÿ�� ô ö�� ü ù²ó2ù²ø ú��b d÷n�fúK÷2ù²ú�ô öm÷Ñ�£÷µù�=§ø ÷µù�ÿKó�� ó� {ü�����þ�� óm÷�ó�ôÊ��!�ù ��.� ú©ô�� ó� ��� mù²úK÷�bù²ø( ��²ÿ�� ôKö�� !#� ÿ�ø�� óm÷ ù²ÿ�ôKömÿK÷�úKö��²ø(n!#�mómü ô��(�²ómúqÄù²ó�Ñ�W÷Óþmÿ��@�²÷�ô ö�ù�ôaô öm÷�þ ÷ ÿ��!#� ÿ���ù²ófü ÷���!NôKö�� úÇóm÷��¹ù²ø( ��²ÿ�� ôKö�� � ú£ÿû÷��ù²ÿ�=ù� fø���ú��(��� ø�ù�ÿ;ô���ô ö�ùô���!���� ôKôKø ÷�úûô��²óm÷�»�ú�%&� ómó��'�ù²ø� �� ÿ�� ô ö��</�9�öm÷�úK÷�ú��(��� ø�ù�ÿ�� ô�� ÷�úNú��� � ÷ ú©ô�ùaü�ø(� úû÷£ÿû÷ ø�ùô��(�²ómúKö�� þA ÷�ô��£÷ ÷�ó� ��Z� úûô�� ó� �ô ÷�ü�ömó���hZ�f÷ úù²ó�Xô öf÷]�²ómø(� óm÷5$;÷�ÿKü�÷ þfôKÿ�� óù²ó�>%&� ómó��'��ù²ø( ��²ÿ�� ôKö���ú/

%{÷\ ��Ò�²÷Çù;�mó���f�÷�ù²ó�ù²ø��Pú�� úz��!����mÿò$;÷�ÿKü ÷�þfô ÿ�� óÓù²ó��%&� ófó��'�1ù²ó�ù²ø��� ��m÷�ú3��ö�� ü�ö�� ómü ø���m÷�ú��ù²ó¤� �²ô öf÷ ÿ�ù²ø( �� ÿ�� ô ö���úõù²ú4�£÷ ø øj/2�¦ÿ��"� ÷�q���� ôûô ø ÷�úûô�� óm÷Xù²ó�k§_ü�ö����mÿ���ù�ómúÑD³��ÿ��"� ÷ P6R�U�V�¨(q�'��� � [�ömù"�²÷�úKö��'��ó¤ôKö�ù�ô5$¡÷ ÿûü ÷ þ_ô ÿ��²ó)ù�ó�2DÆù3�²÷ ÿûú��(�²ón��!Q[�%&� ófó��'�}ü ù²óÑ ÷��Z� ÷6�W÷�bù²úaô öf÷� @Ï©�ù�ó� � � � ü ù²úK÷�ú)��!�ù, ÷ óf÷ ÿ ù�ø.� ófø(� óf÷ � �×ó�� ÿ��ºø(� óf÷µù²ÿ/ô öfÿK÷ úûö�� ø��ø ÷ ù²ÿKó�� ó� �ù²ø( �� ÿ�� ô ö��<q��öm÷�ÿK÷ � 1 ©]� úWù²ó¤��ÿK÷ ù²ø�óZ���� d÷�ÿ/)%)÷�þmÿû÷ úK÷�ó�ô�$�*,+��Ë�_�Zm÷ ø� ��.� ú©ô�� ó� ��� mù²úK÷��ù²ómù²ø(�� ��m÷ ú���!ô öf÷ úK÷,� ómø�� óm÷ � �×ó��²ÿ�� ù�ø( �� ÿ�� ôKö���ú3!#� ÿ ù²ó¤�Ð��ù²ø��m÷,©Ð= � =e�Ï×�9�öf÷«$�*,+������.m÷�øN$;÷�ÿKü�÷ þfôKÿ�� ó

 ��

Page 102: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

ù²ó�Ê%&� ómó��'��ù�ó�ù²ø(�� ��m÷ ú���÷�ó=ô��(� óf÷¥ù� ��@�²÷�ù²ÿK÷¦ÿû÷ úûþd÷�ü�ô���� ÷�ø���ôKöm÷ � @ ©�ù²ó� � @�� ü ù²úK÷�ú��!¡ô ö�� úd ÷ óf÷ ÿ ù�øÄù²ø( ��²ÿ�� ôKö��</

9�öm÷ � @�� üµù�úK÷���!)���mÿ�ù²ø� �� ÿ�� ô ö��[üµù�ó¤ù²ø ú��� d÷8�Z� ÷��£÷¤ù²úgù_ ÷�óm÷ ÿKù²ø(��sµù�ô��(� óÊ��!�� ù²üQ=��ú��²ó5ù²ó� +Wÿ ù�� ÷�ó�» ú�$�*,+������Zm÷ øÇù²ø� �� ÿ�� ô ö��g!#� ÿ�ø ÷ ù²ÿKó�� ó� ÓûúKþ�ù�ÿKúK÷�þ ÷ ÿKü�÷ þfôKÿ�� ófú|Ô2D�� ù�üQ=Pú�� óù²ó��+WÿKù"�²÷ ó�q"�'��� E [6/]9�öm÷�� ÿ�ù²ø� �� ÿ�� ô ö��È ��.�²úûô ú«�mú�� ó� Ê�W÷ ù�=¤ö¤�Pþ��²ôKöm÷ úû÷ ú���ö�� ü�öDù²ÿû÷�ú�� ó� ø ÷���Z� ø ÷ ù²óø�� ôK÷ ÿ ù�ø úe ô ö�� ú«� úaú����_� ø�ù²ÿ�ô��t��ö�ù�ô�ôKöm÷ � @�� üµù²úû÷4��!"���mÿ�ù²ø( ��²ÿ�� ôKö�� ��P÷ ú�/Ao5�fÿù²ó�ù�ø��Pú�� ú\��!cô öm÷ � @�� üµù²úû÷p ÷ óf÷ ÿ ù�ø(�(s�÷ ú ô öm÷�� ÿ£ù²ø( �� ÿ�� ô ö��7ô���m÷ ù²ø¥��� ô ö�ÿK÷µù�ø����ù²ø��m÷�ÿ ùô öm÷�ÿô ömù²óm�\�.� ø ÷µù²ón� ómþ��fô8��ù²ÿ���ù� mø ÷ ú�qNô ö��mú�ù²ü�ö�� ÷��Z� ó� ¥ù> ���ù²øýú©ô�ù�ôK÷^ ¤��� ù²üQ==ú�� ó{ù²ó�m+Wÿ ù�� ÷�ó�qù²ó��ù²ø ú��d�.� ÷ ø�mú¡ùgú��� múûô ù²ó�ô�� ù²ø ø��¦úûôKÿ�� ó� ²÷ ÿÄú ù���þmø ÷Çü�����þmø ÷Ir�� ô��p ����mó��ô ö�ù�óA�Wù²ú;÷�úûô ù� mø(� úKöm÷� ¤�"� ù²üQ==ú��²ó¤ù²ó�i+Wÿ ù��²÷ ó�/

p ������ç qÒ�.��À-����Ë � �\�N�

§_÷�� ÷ ÿKù²ø�ù��_ô ö�� ÿûúõö�ù�� ÷Xþmÿû÷H�Z�(���múKø��{ú©ô����� ÷Òø�� óm÷ ù²ÿ�ôKömÿK÷�úKö��²ø(Òø ÷ ù²ÿKó�� ó� ù²ø( ��²ÿ�� ôKö���ú���ö�� ü�ö�\� ÿ�=` ¤��ü���� �� ó�� ó� ��£÷µù�=�þfÿK÷�� ü�ô�� ÿKú�/�q�ÿû÷�mó��ù�ó�<§_ü�ö�ù²þ�� ÿK÷¦ömù"�²÷�úûô����� ÷�ù²óXù�ø( �� ÿ�� ôKö����ö�� ü�ö{þmÿû÷�� ü�ô ú]�mú�� ó� �ù>�£÷�� ö�ô ÷�ë���²ôK÷_��!Wô öf÷�ö¤�Pþ���ô öm÷�úK÷ úA��ö�� ü�öDô öm÷_$;÷�ÿKü ÷�þfô ÿ�� óÒù²ø� ����ÿ�� ô ö��g ÷�óm÷ ÿKù�ô ÷�ú]��mÿ�� ó� Ê� ôKúaô ÿKù�� ó�� ó� �þmömù²úK÷8D�q�ÿû÷�mó�Dù²ó�m§Pü�ö�ù²þ�� ÿK÷�q1�¢���� ¤[I/A9�öm÷4�£÷�� ö�ô��!¦÷ ù²ü�ö�ö¤�Pþ���ô öm÷�ú�� ú_� óEô ö�� ú«���²ô ÷8� ú�þmÿ�� þ�� ÿ©ô��(�²ó�ù²ø�ô��^� ôKú�ú��mÿD�Z���²ù�øgô��(��÷�q��³/ ÷�/�ô öm÷�ó������ ÷ ÿ���!�÷6rfù���þmø ÷�úA��ö�� ü�ö5÷ ø ù²þmúK÷t d÷6!#� ÿK÷_� ô�ü ø�ù�úKú���f�÷ ú�ù²ó5÷Irmù���þmø ÷t� ómü� ÿûÿK÷�ü�ô ø��Dù²ó�5ü ù��múû÷ úô öf÷4$;÷�ÿKü ÷�þfô ÿ�� óDù²ø( �� ÿ�� ô ö��&ô��` ÷�óm÷ ÿKù�ô ÷�ù�óm÷��3ö¤�Pþ��²ôKöm÷ ú�� ú�/8q�ÿû÷�mó�bù²ó�§_ü�ömù²þ�� ÿû÷õþmÿ��@�²÷ ÷�óm÷ ÿKù²ø(��sµù�ô��(� ó¤÷�ÿKÿ�� ÿ� ����fó�mú�� óBô öm÷�ÿK÷�ú��mø ô�� ó� Iü ø ù²úKú���f�÷�ÿp��ö�� ü�ö{ù²ÿK÷�ú��(��� ø�ù�ÿgô��<ºÇù²þfó��(=¥»�ú ÷�óm÷ ÿKù²ø(��sµù�ô��(� óÒ÷�ÿKÿ��²ÿ4 ����fó�mú�!#� ÿõô öm÷ Ó���ùGr.�(��ù²ø���ù²ÿ� �� ó�ÔBö��Pþ ÷ ÿûþmø�ù²óf÷bDLºÇù²þmó���=�q,�'���� ¤[6/9�öm÷^q�ÿK÷��mó�.��§_ü�ö�ù²þ�� ÿK÷Xù²ø� �� ÿ�� ô ö�� ���¹d÷ ÿKú�!"ÿ���� ���mÿÓù²þmþfÿ���ù²ü�ö2� óÒúû÷H�²÷ ÿ ù�ø���ù¢�Pú��Ê!#� ÿ4� óf÷ô ö�� ó� �q�ôKöm÷� ÿaù²ø� �� ÿ�� ô ö��:� ú,�mómø��(=�÷]���mÿKú;� ó�ôKö�ù�ô;� ô,��P÷ ú�ó��²ô;�múK÷A ��Z� úûô�� ó� Óô���ü���� �� óm÷�ô öf÷�£÷µù�=õþfÿK÷�� ü�ô�� ÿKú�/�*;��� ô��(� ómù²ø ø���q®��öm÷ ÿû÷µù²ú"���mÿýù²ø( ��²ÿ�� ôKö��<» úzf�ó�ù²øfö��Pþ��²ô öf÷ ú�� ú"� ú�ù�ú�� ó� ø ÷Wø�� ó.�÷µù�ÿ;ô öfÿK÷ úûö�� ø��!#�mómü�ô��(� ó¸q�ô öf÷� ÿÅù²ø( �� ÿ�� ô ö��<» ú�f�ómù²øfö¤�Pþ���ô öm÷�ú�� ú�� úýù;m÷ þfôKö.�­©�ôKömÿK÷�úKö��²ø(õü�� ÿKü���� ôDÆùÊ�£÷�( ²ö=ôK÷ë���²ô ÷_�"� ÷ ÿ�$;÷�ÿKü ÷�þfô ÿ�� ó5ö¤�Pþ���ô öm÷�úK÷ úA��ö�� ü�öÒù�ÿK÷Óô öm÷���úû÷ øÒ�²÷ ú�ø(� óf÷µù²ÿ�ôKömÿK÷�úKö�� ø�!#�mómü ô���� ómúQ[I/��� ù²ó�3 �ù�ö�ù��²÷aú��� � ÷ ú©ô ÷��ô ö�ùôWù�ÿ ù²ó�����4�×úK÷ ù²ÿKü�öZ� ù²ó�Z� ô ÷�úûô£ù²þmþfÿ���ù²ü�ö�üµù²ó_ d÷,�múû÷�ô��f�ó�`�£÷µù�=�ü ø ù²úKú���f�÷�ÿWø�� óm÷ ù²ÿÇôKömÿK÷�úKö��²ø(`!#�mómü ô���� ómú\!#�²ÿWü ÷�ÿûô ù�� ó�ÿû÷ ú©ô ÿ�� ü�ôK÷Iö�ù²ø�!"úKþ�ù�ü ÷�ø ÷µù²ÿûó�� ó� þmÿ��� mø ÷���ú<D¡����ù²ó�' ù�q,�'��� � [I/-9�öm÷��Eþmÿ�� þ��²úK÷Xü���� �� ó�� ó� ¤ô öf÷ úK÷>�£÷µù�=1ü ø ù²úKú���f�÷�ÿÓø(� óf÷µù²ÿô öfÿK÷ úûö�� ø�i!#�mómü ô��(�²ómú,��� ô öù�ú�����þfø ÷���ù"Ñ�� ÿ�� ô������²ôK÷�edôKö��múq ô öm÷�� ÿ�ù²þmþmÿ���ù²ü�öBù²ø ú���ÿK÷ ú��mø ôKú«� óù�f�ó�ù²ø/ö¤�Pþ��²ô öm÷�ú�� ú,��ö�� ü�ö8� ú�ù4m÷ þfôKön©�ô öfÿK÷ úûö�� ø��ü�� ÿKü���� ô/

op�mÿù�þmþmÿ�� ù²ü�öÏ� ú¥ü�ø(� úû÷ ú©ô�ô��EôKö�ù�ô<��!¢� ù�üQ=Pú�� ó�ù²ó� +Wÿ ù��²÷ ó�q5��ö��Û�múK÷^ ��Z� ú©ô�� ó� 1ô����B

Page 103: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

ü���� �� óf÷gú�� ó� ²ø ÷�ø(� ô ÷�ÿ ù²ø ú)� ó�ô���ù�úûôKÿ�� ó� �ö¤�_þ��²ôKöm÷ ú�� úÇø(� óf÷µù²ÿýôKömÿK÷�úKö�� ø��!#�fómü�ô��(� ó�/�*aú"�W÷gúKö��'�� ó2§P÷ ü�ô��(� ó � /á£.q�ô öf÷ � @£� üµù�úK÷>��!,���mÿÓù²ø� �� ÿ�� ô ö��Âúûô ÿû÷ ó� �ô öm÷�ómú�ù�ó�- ÷ óm÷�ÿ ù²ø��(s�÷ ú�ôKöm÷� ÿÿK÷�ú��mø ô ú�/

9�öm÷ ü ø�� úK÷ ú����_� ø�ù²ÿ�� ô��d� ó�þd÷�ÿ�!#� ÿ���ù²ófü ÷) ����mó�fú3 ÷�ô��£÷ ÷�ó����fÿ� ��Z� úûô�� ó� G�� �ù²úû÷�ù²ø� �� ÿ�� ô ö���úù²ó�õôKöm÷�� ófø(� óf÷ � � ó�� ÿ��7ù²ø� �� ÿ�� ô ö���úÄú��� � ÷ ú©ô ú ùaÿû÷ ø ù�ô���� ómúKö�� þ� ÷�ô��£÷ ÷�ó_ ��Z� úûô�� ó� aù²ó�4� ómø�� óm÷ø ÷ ù²ÿKó�� ó� �/`q�ÿû÷�mó�¤ù²ó�n§_ü�ö�ù²þ�� ÿK÷tD�q�ÿK÷��mó�Bù�ó�n§_ü�ömù²þ�� ÿû÷�q3�'��� E [gù²ó�n§_ü�ö�ù�þ�� ÿû÷ÊDj§Pü�ö�ù²þ�� ÿK÷�q�'����� ù¤[£ö�ù��²÷A� ó(�²÷ ú©ô��( ù�ô ÷�¥ôKö�� ú�ÿû÷ ø�ùô��(�²ómúKö�� þÊ� óIô öf÷�ü�� ó�ô ÷Ir_ô;��!ò �ù���÷aô öf÷� ÿ­��/

o v�� � |�� ���� �¢y���|"�����p ��ÿ���� ¤ �3�½� �U���¥��Ì7�^�N�.�����¬�DÁ�À1�¥���z�qN� ÿ�ù�þ���� ó�ô ! @¬D ! Æ )¢××¢×H) ! c [ # a�c�ù²ó� � 1 �A�£÷���ÿ�� ô ÷A$ ! $y¥�ô��tm÷ ó���ô ÷�ôKöm÷ � �×ó�� ÿ��û��!! )�ó�ù���÷ ø�� $ ! $y¥p@ � c�Ùpo Æ � ! ÙD� ¥ � Æ�£

¥×

9�öm÷¦�2�×ó��²ÿ�� ��! ! � ú9$ ! $w§:@ª��ùGr Ùpo Æ©¨ ¿ ¿ ¿ ¨ c � ! Ù|�á×�q¥�²ÿ � )«ª 1 ��ôKöm÷¢ª®� ó�� ÿ�� � ú�S�ö�U�Vdô��Xô öf÷� � ó�� ÿ��È��! Æ¥ ŬƬ @ ��ecöm÷�ómü ÷�ô öm÷`�6�×ó�� ÿ��~ù²ó�¤ô öf÷¢�2�×ó��²ÿ��~ù²ÿK÷A���ù²ø¡ô���÷ ù²ü�ön�²ôKöm÷ ÿ¦ù�ó�ô öf÷Ê©®� ó�� ÿ��¯� úA���ù²ø ô��i� ô úû÷ ø�!�/`%)÷t�fúK÷ � ù²ó�nªIô��8m÷ ó��²ô ÷_���ù²øÇó�� ÿ���ú�/<9�öf÷t!#� ø ø��'��� ó� !@ù²ü ô úgù²ÿK÷p�£÷ ø ø�==ó��'��ó D@÷�/ñ �/«D#9Åù¢�Pø(�²ÿ�ù²ó�< ù²ómó�q��¢� � ©�[£þfþ�/\©�B i ��©�B � [��­n®[G¯�_.j(\«F�G�j{°%J�ì ¯¡±fL�²�M�� +3/"WU�(= $�+�$y¥�$�WU$ ¬ !#� ÿ�ù²ø øÊ+�)dW # a�c�ù²ó�^�:= � =e�Ï׳´±�G¶µ][¸·�¹ºµ»±�F�G�j{°%J�ì ¯¡±fL�²�M�$�+ Å WU$y¥:= $�+Æ$y¥ Å $�W�$y¥�!#� ÿ�ù²ø øÊ+�)dW # a�c�ù�ó�Â�Ü= � =e�Ï×

9�ömÿ����� ö����_ô�ô ö�� ú�ü�ö�ù²þ_ô ÷ ÿ�ô öm÷õ÷Irfù���þfø ÷�úûþ�ù²ü�÷Ð7 � ú¦ù�ú��� múû÷�ô���!�a c ×cò×ó¤ô ö�� úaü�ömù²þfô ÷�ÿ�£÷¥ü�� ómú��(m÷�ÿ4� ómø���� ÿ��( �� ó.�×ü�÷ ó�ô ÷�ÿK÷Òø�� óm÷ ù²ÿ�ôKömÿK÷�úKö��²ø( !#�mómü ô��(�²ómúq"�j/�÷�/½¼)D ! [�@âú��� ózDB+�/ ! [!#� ÿgú�����÷:+ # a c ×N%Zö�� ø ÷¦ô öm÷�úûô ù²ó��ù²ÿ�im÷6f�ó�� ô��(�²ói��!ýù�ø(� óm÷µù�ÿ�ôKömÿK÷�úKö��²ø(8!#�fómü�ô��(� óù²ø ø��®��úù�ó�� ó�s ÷�ÿ��ôKömÿK÷�úKö��²ø(&D#�j/�÷�/2¼)D ! [A@&ú��� ózD�+^/ !6' ?�[p��öm÷ ÿû÷Ê?�ü ù²ób ÷�ù²ó¤�DÿK÷ ù²ø£ó����� d÷�ÿQ[6q� ô�� ú�÷µù�ú��Òô��búû÷ ÷Iô ö�ùô�ù�ó��5ø(� óf÷µù²ÿ�ôKömÿK÷�úKö��²ø( !#�mómü ô��(�²ó2��!�ôKö�� ú4��� ÿK÷Ê ²÷ óm÷�ÿ ù²ø�!#� ÿ��l�"� ÷�ÿ½>�²ù�ÿ���ù� mø ÷�ú]� ú�÷�h����Ò��ù²ø ÷ ó�ô�ô��¤ù¥ø(� óm÷µù²ÿ�ôKömÿK÷�úKö��²ø(m!#�mófü�ô���� óÂ��� ô öDô ömÿû÷ úûö�� ø(mB8�"� ÷�ÿ�½ Å ���ù²ÿ�� ù� mø ÷ úq_ú������mÿ�óm÷6�Cm÷6f�ó�� ô��(� ó>� ómü�mÿûú�ó���ÿK÷µù�ø/ø(� úûúd��!� ²÷ óm÷�ÿ ù²ø�� ô���/

���

Page 104: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

p ��ÿ���ÿ �«�h¼ è���À1�]Á5��Á��gè_��Áp��À1�¾5ëp�N�3��p�S��ËÀ¿5� Á�ÌÊ�N���½Áp� �Í��uÁ�,�.¾�ÀP���À-�N���½Á

9�öm÷Â��� úûô ù�=²÷n ����mó�mú`!#� ÿÊ ��²ô ö�ôKöm÷^$;÷ ÿûü ÷�þfô ÿ�� óuù�ó�Ò%&� ómó��'� ù²ø( ��²ÿ�� ôKö���ú_m÷�þd÷�ó� � óùOÓûúK÷�þ�ù²ÿKù�ô���� óÒþ�ù²ÿKù���÷ ô ÷�ÿ|Ôn��ö�� ü�ö2 ���� ÷ ú�ùBø(�'�£÷ ÿ4 ����mó�-� ó1ô öf÷Ê�� ú©ô�ù²ómü�÷> d÷ ô��W÷�÷ ó�ù�ó��÷6rfù���þmø ÷Óù²ó�bô öm÷�ö¤�_þ ÷ ÿûþmø�ù�óm÷4��ö�� ü�ö�m÷�f�óf÷ ú�ôKöm÷�ô�ù²ÿ� ÷�ô�ömù²ø�!"úûþ�ù²ü ÷�/`§Z�(��� ø�ù�ÿKø���q����mÿ�óm÷��ù²ø� �� ÿ�� ô ö���ú�ù²ø ú��ÓÿK÷�h���� ÿû÷�ôKö�ù�ô�ô öm÷�÷6rfù���þmø ÷ ú��múK÷�i!#� ÿ�ø ÷ ù²ÿKó�� ó� ����mú©ôgó��²ôgø(� ÷�ô��Z��ü�ø(� úû÷�ô��ô öf÷�úK÷�þ�ù²ÿKù�ô�� ó� �ö¤�Pþd÷�ÿKþmø ù²óm÷�/«9�ö��múp�£÷A��� ø øÄü� ómú��(m÷�ÿ�ù�úKø(�� ö�ô ø��XÿK÷ ú©ô ÿ�� ü�ôK÷2�²÷ ÿûú��(�²óÑ��!�ô öf÷$�*,+�ø ÷µù²ÿûó�� ó� ]���.m÷�ø¥� ó_��ö�� ü�ö�ôKöm÷«�����ù�� ó_��!/þ��²úKú��� mø ÷�÷6rfù���þmø ÷�ú\f÷ þ ÷ ó�múd�²ó�ô öf÷�ô�ù�ÿ� ÷ ôü�²ómü ÷�þfô« ÷� ó� Óø ÷ ù²ÿKóm÷��/qN� ÿ � 1 ©Óù�ó��7ÃÂ&adc��£÷5��ÿ�� ôK÷9$�7 $y¥�ô���m÷�ó��²ôK÷�ú��mþ ����Ä $ ! $y¥¤×�qN� ÿ�ù�ô ù²ÿ� ²÷�ôQ� ÷�ü�ô�� ÿ+ # a�c4�£÷��múK÷�ô öm÷�ú­�.�� �� ø ��Å ¨ Ä�ô���m÷ ó��²ô ÷¦ô öf÷]h���ù�ó=ô�� ô��

��Å ¨ ĽƫÇuÈ@ � ó�!���ºÄ D�+0/ ! [�D@ú��( ózDB+3/ ! [�[�)��ö�� ü�öÍ� ú�ù^��÷µù�ú��mÿû÷i��!�ôKöm÷úû÷ þ�ù�ÿ ù�ô��(� ó2 d÷ ô��W÷�÷ óZ÷6rfù���þmø ÷ ú�� ó>7 ù²ó��ô öm÷¥ö¤�_þ ÷ ÿûþmø�ù�óm÷��ö�� úû÷aó��²ÿ���ù²øu� ÷�ü�ô��²ÿ�� úÆ+zפ%{÷�ù²úKú�����÷�ô ömù�ôÜ$�7 $y¥ 4 �§)Z�³/ ÷�/Åô öf÷�úû÷�ôS7 � ú" ����mó�m÷��qmù�ó�ô ömù�ô �wÅ ¨ ÄORÒB�)¥�j/�÷�/�ô öf÷ ÿK÷]� ú�ú�����÷�ó��²ó�s ÷�ÿ���ø��'�W÷�ÿ, ����mó�i� óXô öm÷�úK÷�þ�ù²ÿKù�ô���� ó> d÷ ô��W÷�÷ ó¤ô öf÷ö¤�Pþd÷�ÿKþmø ù²óm÷�m÷�fmóm÷8 ��A+ù²ó�Xô öm÷�÷Irmù���þmø ÷ úd� óA7b/

�/÷�ô,5:72DB+.)DC_[�m÷ ó���ô ÷õù�ó�÷Irmù���þmø ÷�� ÿ ù²ü�ø ÷5��ö�� ü�ö�qN��öm÷ óÑh��m÷�ÿ�� ÷�q þmÿ��"�Z�(f÷ ú�ù�ø ù� d÷�ø ÷�÷6rfù���þmø ÷Ða ! )Pú��� ózDB+Ü/ ! [De1��öf÷ ÿK÷ ! � ú�mÿ ù¢��óIù²ü�ü� ÿ��� ó� �ô���ô öm÷;�� ú©ô ÿ��( ��fô��(� ó�Cª�"� ÷�ÿS7Ñ×�%)÷ú ù¢��ô ö�ù�ô¦ù²óDù²ø( ��²ÿ�� ôKö��g¢Ã� ú¦ù8X6R³F�EGKZJ>V(P�U�F6KNHLKZJ<U�VêJZEGFIHLR#TZÚ7W¢EGF�+ EGK^7 ��!"� ôaú ù�ô�� ú­f�÷ ú¦ô öf÷!#� ø ø(�'��� ó� �ü� ó��� ô��(� ó%�¡ô öm÷�ÿK÷5� ú�ù�!#�mómü ô��(�²ó3É^D³²�) � )d+�)|7Â[Çú��fü�ö¥ôKö�ù�ô�!#� ÿ�ù²ó¤�`�� úûô ÿ��( ��_ô��(�²ó3C�"� ÷�ÿ.72)®!#�²ÿÄù²ø ø¤B 4 ²H) �Ð4 �*)xù²ø� �� ÿ�� ô ö���¢-��ù�=²÷�úÅù�ôz��� ú©ôPÉ^D�²�) � )d+�)|7^[Nü ù²ø ø úNô���5:72DB+�)|C_[�)ù²ó�4��� ô öÓþmÿ��� �ù� �� ø(� ô���ù�ô ø ÷µù²ú©ô�� '9� ù²ø� �� ÿ�� ô ö�� ¢&���fôKþ��fôKú£ù¦ö¤�Pþ���ô öm÷�ú�� ú½�6�*7 � �(' �*)��&ú��fü�ö�ô ö�ù�ô`$�ÿD���H�½z �zD ! [>�@ ú��( ²ózD�+º/ ! [�{�= ²×�%)÷úKù¢��ôKö�ù�ô�ú��mü�öZù)ö¤�Pþ��²ô öm÷�ú�� úA�Í� ú�ù²ó²M­UZOQO�öZF�U�R�PpT ôQYNEGRLTNPQX6H(X)W¢EGF`+löZK�SZP6FSC�ù²ó��ôKö�ù�ô�ôKöm÷p!#�mómü ô���� óÊÉ^D³²H) � )d+�)|7^[\� ú£ô öf÷¦ú ù���þmø ÷ü����þmø ÷6r.� ô��t��!Åù²ø( ��²ÿ�� ôKö�� ¢�×

*aú����mÿ���ù�� óDÿK÷�ú��mø ô�� ó)ô ö�� ú�ü�ömù²þfô ÷�ÿ��£÷t��� ø ø1m÷ úûü ÿ��� d÷�ùúûôKÿ�� ó� ø ÷µù�ÿKó�� ó� ù�ø( �� ÿ�� ôKö��ù²ó�Xü ù²ÿK÷6!#�mø ø���ù²ómù²ø��Zs ÷5� ô ú�úKù���þmø ÷�ü�����þmø ÷Ir�� ô���/193�����ô ö�� ú��W÷����fúûô�ü�²ómú���m÷ ÿgù²ø� �� ÿ�� ô ö���ú��ö�� ü�öÑ���ó���ô�ú ùô�� ú­!L�Xô öm÷�úûô ÿ�� ó� �ø ÷µù²ÿûó�� ó� �þmÿ�� þ ÷ ÿûô��< ��_ô¦ù²ÿû÷�úûô�� ø ø¡ü ù²þ�ù� fø ÷]��!" ÷ óf÷ ÿ ùô�� ó� ö¤�Pþ��²ôKöm÷ úû÷ ú�ô ö�ù�ô�ömù"�²÷Óú�����÷�úKø��( ö�ô�ù�x�²ù�ó=ô ù� ÷4�"� ÷�ÿ�ÿKù²ó����� ��m÷�úKú�� ó� �e¡ú��fü�ö5ù²ø� �� ÿ�� ô ö���ú

�¤©

Page 105: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

ù²ÿû÷]==ó��'��ó�ù²ú8ù�P�U;��V(P�U�FIKNHLKZJ`U�VêJZEGFIHLR#TZÚ�X¨��/÷�ô� @�a ! Æ )Kú��( ²ó�D�+3/ ! Æ [|e�)¢×¢×¢×H)@a ! Ë ) ú��� ózDB+3/ ! Ë [|e

÷õù�fmó�� ôK÷�úû÷h��m÷ ófü ÷4��! ø ù� d÷�ø ÷��÷Irmù���þmø ÷ ú�!"ÿ����g7ñù²ó�ø ÷�ô�Cû ÷õù_�� úûôKÿ��� ��fô���� ó<�@�²÷ ÿ � ×qN� ÿ�B 4.��4 �"?�©x)��£÷�úKù'�ÓôKö�ù�ôS�6�*7 � z ' ��)¢��{¥� úÇù_D­�@?�© '3� [6M­U6Y¤Y�F�E�J�HLÚ�U�R�EGF�W¢EGFÆ+¥ö.K�SZP6FC ��!�©Ë� Ùpo Æ C8D ! Ù [-/u�ÿ�zD ! Ù [ ' ú��( ²ózD�+3/ ! Ù [@�(= �

© 'n� × D � /��'[

%)÷�ú ù¢�bô ömù�ô�ù²ó{ù²ø( �� ÿ�� ô ö�� ¢:� ú�ùbD��@?�© ''� [6M�ù�P�U;�<V(P�U�FIKNHLKZJnU�VêJZEGFIHLRLTZÚ W¢EGF:+hö.K�SZP6F8C��!¦ôKöm÷Ñ!#� ø ø(�'��� ó� )ü� ó��� ô��(� ó�ö�� ø(múH��!#� ÿ�ù²ó¤�kf�ó�� ô ÷Búû÷�ô � ù²ú`m÷�úKü�ÿ��( ÷sù� ��"� ÷bù²ó�Zù�ó���� úûô ÿ��( ��_ô��(�²óëC � ó � ))��!�¢û� ú� ��Ò�²÷ ó©Cñù�ó� � ù²ú4� ómþ��fô�ô öm÷�óý¢û���_ô þ��fôKúÓùBö��Pþ��²ô öf÷ ú�� ú� �S7 � z ' �*)¢��{���ö�� ü�öÍ� ú�ùÍD��@?�© '�� [��×ù²þmþmÿ��'r.����ù�ô�� ÿ�!#� ÿ«+Í�mó�f÷ ÿ0C>×"9�ö��mú_!#�²ÿ_���fÿþ��mÿûþ��²úK÷ ú�ùÊ�W÷ ù�=Bø ÷ ù²ÿKó�� ó� Xù²ø� �� ÿ�� ô ö��g� ú5�²óm÷4��ö�� ü�ö)üµù²ó)ù²ø��Wù¢�Pú�fmó�5ùIö¤�Pþ��²ôKöm÷ ú�� ú�ô ö�ùô���fôKþd÷�ÿ�!#� ÿ���ú�ÿKù²ó����� ��m÷�úKú�� ó� �� ó�ù4f.rf÷�úKù���þfø ÷�/

p ��ÿ��» 5lp�h� Á ����Áp�'ÌÍ��Áp�\�]� �O�������¥���u �¬�

ò×ó�ôKöm÷«� ófø(� óf÷gø ÷µù²ÿûó�� ó� A���.f÷ øjq=ø ÷µù²ÿûó�� ó� �ô�ù�=²÷ úWþmø�ù²ü�÷;�"�²÷ ÿ�ù�úû÷h��m÷�ómü ÷5��!cô ÿ�� ù²ø ú�/�9�ömÿ����� öZ����fô¥ô öf÷)ø ÷µù²ÿûó�� ó� 1þmÿ��_ü�÷ úûú�ô öf÷)ø ÷µù²ÿûóm÷ ÿ<��ù�� ó�ô�ù�� ómúXù1ö��Pþ��²ô öf÷ ú�� úº� ��ö�� ü�öÏ��ù²þmúA7 ô���(' �*)¢��&�× O ù²ü�ö¥ô ÿ���ù²øNþfÿ��Pü ÷ ÷�mú�ù�ú�!#�²ø ø(�'��úH�1�mþ��²ó�ÿû÷ ü ÷��Ò�Z� ó� �ù²ó¥÷6rfù���þmø ÷ !º# 7 ôKöm÷�ø ÷ ù²ÿKó.�� ó� �ù�ø( �� ÿ�� ôKö�� ���fô þ��fô ú,� ôKú�þmÿK÷��� ü ô��(�²óO}Î @Ï�zD ! [d��!;ôKöm÷�ù²úKú��_ü���ù�ôK÷¥ø�ù� ÷ ø Î ×�9�öf÷�ø ÷µù²ÿûó�� ó� ù²ø� �� ÿ�� ô ö��:� ú�ô öm÷�óÑ ��Ò�²÷ óôKöm÷�ôKÿ��m÷�ø�ù� ÷ ø Î #ý�(' �*)¢��&�ù²ó�ôKöm÷õù²ø� �� ÿ�� ô ö��Ìüµù²ó8�mþ��ù�ô ÷A� ôKúö¤�Pþ��²ôKöm÷ ú�� úÍ�Ê �ù²úK÷�>� óIô ö�� ú�óm÷�� � ó.!#� ÿ���ù�ô���� ót ÷�!#�²ÿK÷aôKöm÷¦óm÷6rPô�ôKÿ���ù�ø¸ ÷ �� ómú/)9�öf÷�þ ÷ ÿ­!#� ÿ­���ù²ómü ÷���!�ù²óm� ófø(� óf÷�ø ÷µù²ÿûó�� ó� ù²ø� �� ÿ�� ô ö��Ð� ó{ù²ó{÷Irfù���þfø ÷�úK÷h��m÷�ómü ÷t� ú]��÷ ù²ú��mÿû÷b ¤�bô öf÷ó����� ÷ ÿ���!;þmÿK÷��� ü ô���� ó>�_� úûô�ù�=²÷ ú���ö�� ü�ö�ôKöm÷�ù²ø( �� ÿ�� ô ö�� ��ù�=²÷�ú/

��ÿ��"� ÷�q���� ôKôKø ÷�úûô��²óm÷Óù²ó�m§_ü�ö����mÿ���ù²ófú`D³��ÿ��@�²÷ÂP6R«U�V�¨(q��'��� � [�ù�ó�b�¦÷ ó�ô�� ø ÷Óù�ó�m��� ôKô ø ÷6�úûô�� óm÷<D³��÷�ó=ô�� ø ÷�ù²ó�^��� ôûô ø ÷�úûô�� óm÷�q��'�����¤[gö�ù�� ÷Óúûô����� ÷�Dùt!@ù��_� ø��<��!�� ómø�� óm÷�ù²ø( �� ÿ�� ô ö���ú,!#�²ÿø ÷ ù²ÿKó�� ó� �ø(� óm÷µù²ÿ�ôKömÿK÷�úKö��²ø(i!#�mófü�ô���� ómú�D@úû÷ ÷÷q��� ��mÿû÷��'[6/�%)÷�ÿK÷6!"÷ ÿgô���ôKö�� ú�ù²ø� �� ÿ�� ô ö��<q.��ö�� ü�ö� ú�þmù²ÿ ù���÷�ôK÷ ÿ��(s ÷�_ ¤��ù�ÿK÷ ù²øu��ù²ø(�m÷ � 1 ©x)=ù²ú�ôKöm÷�EGKNV HLK¸P � M�K¸EGFIÚ¬U�VêJZEGF6HLRLTZÚ�¨1���(=²÷�ôKöm÷d�W÷�ø ø��==ó��'��óÑ$¡÷ ÿKü�÷ þfôKÿ�� óù²ø( �� ÿ�� ô ö��<qPôKöm÷�� ómø�� óm÷ � �×ó�� ÿ��Ìù²ø� �� ÿ�� ô ö�� �mþ��ù�ôK÷ ú�� ôKú�ö¤�_þ��²ôKöm÷ ú�� ú, ����ù�=�� ó� Xù²ó)ù���� ô���� ÷�ü�ö�ù�ó� ÷�ô��¥ùÊ�W÷��( ö�ô8� ÷�ü�ô��²ÿ ± ×ò¦,�'�W÷�� ÷ ÿ�qýù²ú�úûö��'��óm� óbú©ô ÷ þfú � ��£t��!q��� ��mÿK÷���q�ôKöm÷ � �×ó�� ÿ��Ñù²ø( �� ÿ�� ô ö�� ��_÷�ú�ó��²ô«�fúK÷�ôKöm÷ ± � ÷�ü�ô��²ÿ«�� ÿK÷ ü ô ø��`!#� ÿgþmÿK÷��� ü ô��(�²ó8 ��_ô

� i

Page 106: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

FHG.IKJ�L*M ÿK÷ ù²ø/ó����� ÷ ÿ � 1 ©� ó�� ô�� ù²ø��£÷�� ö�ôX� ÷ ü ô�� ÿ ± � @ÃD ± �Æ )¢×¢×¢×H) ± �c [ # adcÿK÷ ù²ø/ó����� ÷ ÿÐÏ0RÍBT JUL"IKJ�LVM9ö��Pþ��²ô öf÷ ú�� ú��zD ! [

��/ ¹�jVL:P"@ B©./Ñ· @ ±¡¯Bj�÷Irfù���þfø ÷ úgù²ÿû÷�ù��²ù�� ø�ù� mø ÷ _�[i / Ò]jVL��fómø�ù� ÷ ø ÷I÷Irmù���þmø ÷ ! m� / Y[]\9ì0¯¡¯Ó��@ �*)¢×¢××�)�½E¹�jVL�Ô mÙ @3ú��( ózD ± mÙ [@� ± mÙ � ¥ ÔNÆ£./ I.\@jx_Ó±�Õ�L¥}Î m @ºú��� ózD�Ô m / ! m [E / Ò]jVL�ø ù� ÷ ø Î m #î�(' �*) Å �k&� / Y[]\9ì0¯¡¯Ó��@ �*)¢×¢××�)�½E¹�jVL ± mÖ ÆÙ @ ± mÙ Å Ï�D Î m ' }Î m [ ! mÙ �/ ¹�jVL:P"@ªP Å ���/Zj G._K_�[

q��( ��fÿK÷ � /á©x�19�öm÷�� ómø�� óm÷ � �×ó�� ÿ��Ìù²ø� �� ÿ�� ô ö��8/ÿ ùô öm÷�ÿÅþmÿû÷�� ü�ô úò�mú�� ó� �ùX� ÷�ü�ô��²ÿÍÔ2��ö�� ü�ö�� úÅù�ôKÿ ù²ófú�!#� ÿ���÷�Ü� ÷�ÿKú��(� ó���!mô öm÷ ± � ÷�ü�ô�� ÿq�ó�ù���÷�ø��Ô`Ù�@&ú��� ózD ± ÙL[@� ± Ù|� ¥ ÔNÆ;!#� ÿ�ù²ø ø��;@Ð��)¢×¢×¢×H)�½�×30;��ô ÷Óô ö�ù�ô���öm÷�ó � @ ©>�£÷�ö�ù��²÷ ± @×Ô�ù�ó�öm÷�ómü ÷�ô öm÷_� ómø(� óm÷�©®� ó�� ÿ���ù�ø( �� ÿ�� ôKö�� � ú]��m÷ ó�ô�� üµù�øýô���ôKöm÷_$;÷ ÿûü ÷�þfô ÿ�� ó1ù²ø( �� ÿ�� ô ö��</_��ÿ��@�²÷P6R�U�V�¨�úKö��'�ºô ö�ùô�ù²ú � � � ôKöm÷�� ófø(� óf÷ � �×ó�� ÿ��Ñù²ø( �� ÿ�� ô ö��&ù²þmþmÿ���ù²ü�öm÷�ú¦ù«�²÷ ÿKú��(� ói��!ýô öf÷%&� ófó��'� ù²ø� �� ÿ�� ô ö��8/Ø 8�²ÿK÷�þmÿû÷ ü� úK÷�ø���qNôKöm÷�!#� ø ø(�'��� ó� �ô öm÷�� ÿK÷�� ���� ÷ ú5��� ú©ô�ù�=�÷� ����mó�múp!#�²ÿô öf÷]�²ómø(� óm÷ � � ó�� ÿ��&ù²ø( �� ÿ�� ô ö���ú��?A@ j([]\¡j ê Ù�ïÒð L�Ú \@[¸Û�j�Ü(ÝAÞ�ßáàãâ~ä¸å�å*Ù�M ñzP6R � @�a ! Æ ) Î Æ e�)¢×¢×¢×�)@a ! Ë ) Î Ë e ó6P5UAX'P|õ�ö¥P6K¸OQP�E�WV�UVó6P6V(P�S>P&J�U�Ú«Y�V(PQX�ù�TNP6FQP !2# 7 U�K�S Î @ X6H�J�K�D�+«/ ! ["W¢EGF�P�Î�P6F�ô>P&J�U�Ú«Y�V(P8a ! ) Î e # � ×-�Uw/½B3EGF4U�K�ô�©�= � 4 � U�K�SiU�K�ôæÏ�RCB�)1H W�RLTNPtEGKNV HLK¸P � M�K¸EGFIÚ U�VêJZEGFIHLRLTZÚÐH(X�HLK�Î�Eç�¤P�Sù1HLRLTÒHLK�Y�öZRpY�U�F�U�ÚtP6R�P6F�XÂD � ) ± � @ D�B�)¢×¢×¢×H)QB¤[�)}ÏZ[�)5RLTNP6KÏRLTNPnÚ4H(X6RjU;�¤P¥ó6E¡öZK�SÒEGKÏRLTNP

P&J�U�Ú«Y�V(P;X'P|õ�ö¥P6K¸OQP � H(X�U�R�ÚtE'X6RD � ' �'[@$�+�$I¾ ¬ $�7 $I¾¥� ¾Å ¨ Ä ×

-�óè/½B3EGF_U�K�ôÊ©î= � 4 �§)�H W ± � XU�R�H(X#Ì�PQXÜ+6/ ± � R B-U�K�S3Ï^@ �ãé�ê ëjìãíyîïì ÄðÓ ¥ ÔNÆ³Ö Åçñ í î ì Ä ì Äð )�RLTNP6K2RLTNP� �

Page 107: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

Ú4H(X6RjU;�¤PÜó6E¡öZK�S>EGK � H(XAU�R\ÚtE'X6RD � ' �®[@$�+�$I¾ ¬ $�7 $I¾¥� ¾Å ¨ Ä òó � ' � +«/ ± �$�+Æ$ ¬ $ ± � $y¥ � ¾«ôõ ×

-�O©/ëñzP6R ± � @ÃD��*)¢×¢××�)¢�®[]U�K�S�X�ö'Y¤YNE'X'P�R#T�U�R.+�Ù.RÒBAW¢EGFö��@7�*)¢××¢×�)�½ò×ø÷LW � � � U�K�SAÏnH(XUGXASZPQX'O6FIH�ó6P�S_HLKtY�U�FIRù-�óè/çú"RLTNP6KnR#TNP]Ú4H(X6RjU;�¤PÜó6E¡öZK�S�J�HfÎ�P6KÑHLKd-�óè/`OQEGK�Î�P6F³JZPQX�R�E

©�$�+�$I¾ Æ $�7 $I¾ §� ¾Å ¨ Ä � ø��� 1½ Å c� Ù o Æ +]Ù$�+�$ Æ ø(�� +�Ù$�+�$ Æ � ×%Zöm÷ ó � @C©õþ�ù²ÿûôAD@ù¤[���!¡ô ö�� ú�ô öm÷�� ÿK÷��Ìü�²ÿKÿK÷�úKþ�� ó�múg÷6rfù²ü ô ø���ô���ôKöm÷]$;÷ ÿûü ÷�þfô ÿ�� ón+�� óZ�� ÷�ÿ� ÷�ómü ÷t9�öm÷� ÿû÷� D�9�öm÷�� ÿK÷�� £./î©>��!�+Wömù²þfô ÷�ÿ4£�[I/AqN�mÿûôKöm÷ ÿ��_� ÿû÷�q¡þ�ù²ÿ©ôÊD"ü'[�ü�� ÿKÿû÷ úûþ�� ó�mú

ü ø�� úK÷�ø���ô���ôKöm÷A��� ú©ô�ù�=�÷A ����mó�þmÿ��@�²÷Ñ ¤�8��� ôûô ø ÷�úûô�� óm÷p!#� ÿ�ô öm÷�%&� ófó��'�ºù²ø( �� ÿ�� ô ö�� D�9�öm÷I�� ÿû÷�È£./ i ��!)+Wö�ù²þfôK÷ ÿ;£�[6/

p ��ÿ���ç ¿p�N�½� � Á �­�DÁp� ��� �«�h¼ è���À1�]Á5�DÁ��*aúAm÷ úûü ÿ��( ÷b� ó +Wö�ù²þ_ô ÷ ÿ�©.qÅúû÷H�²÷ ÿ ù�ø� ÷ óf÷ ÿ�� ü�þfÿ��Pü ÷��mÿK÷�ú�ù²ÿK÷_==ó��'��óm!#� ÿ�ù��fô�����ù�ô�� ü ù²ø ø��ü�²ó(�²÷ ÿ©ô�� ó� p� ómø�� óm÷Çø ÷ ù²ÿKó�� ó� �ù²ø( ��²ÿ�� ôKö���ú�� ó�ô��«$�*,+������Zm÷ øPù²ø� �� ÿ�� ô ö���ú/�%)÷Wÿû÷ ü ù²ø øZ9�öm÷�²ÿK÷�©./á©.q���f÷�ô��_��� ôKô ø ÷ ú©ô�� óm÷�q.��ö�� ü�öi ���� ÷�ú�ù�ú©ô ÿ��²ó� � ����fó�8�²ó8� óf÷�ú��mü�öü�� óV� ÷�ÿKú���� ó%�?A@ j([]\¡j ê~ð%ïÒð«ñ3P6R�¢òó6P�U�K`EGKNV HLK¸P\V(P�U�FIKNHLKZJ]U�VêJZEGFIHLRLTZÚ ù�TZHjOQT�OQT�U�KZJZPQX\HLRLX\T ôQYNEGRLTNPQX6H(X,EGKNV ôù�TNP6KmHLR,Ú�U;�¤PQX�U<Ú4H(X6RËU;�¤P_U�K�S^ù�TZHjOQTiT�UGX_U8Ú4H(X6RjU;�¤P«ó6E¡öZK�SÂE�W:N W¢EGFtOQEGK¸OQP�Y�R5O6V�UGXQXüû]×ø�TNP6K<R#TNP6F�P5H(X�UæD*FIH)M�ÚtESZP6V¥V(P�U�FIKNHLKZJ�U�VêJZEGF6HLRLTZÚò¢ , W¢EGFöû ù�TZHjOQT�T�UGX«XU�Ú«Y�V(P]OQEGÚ«Y�V(P&J�HLR�ô

· ú � ² ú ø(�� �� Å N û.û ×�\�uù²þmþmø��.� ó� k9�öf÷� ÿû÷� ©Z/î©Eô��Í9�öm÷�� ÿK÷�� � /á©Û� óf÷{üµù�ó �� fô�ù�� óuúKù���þmø ÷)ü����þmø ÷Ir.� ô��

����mó�mú�� ó¥ù� ÷�óm÷ ÿ�� ü�$�*,+������.m÷�øNü�²ó(�²÷ ÿûú��(�²ói��!Äô öf÷5� ómø(� óm÷ � � ó�� ÿ��Ìù²ø� �� ÿ�� ô ö��</z%{÷¦ó��'�m÷�úKü ÿ��( ÷�ùÓü�����þmø ÷ ô ÷ ø��t���¹d÷ ÿK÷�ó�ô,$�*,+������Zm÷ øNø ÷µù²ÿûó�� ó� Óù²ø( ��²ÿ�� ôKö��ª��ö�� ü�ö¥ö�ù²ú�ÿû÷��ù²ÿ�=ù� fø��ú����_� ø�ù²ÿÇú ù���þmø ÷¦ü����þmø ÷6r.� ô��t ����fó�mú/

�¤£

Page 108: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

FHG.IKJ�L*M ÿK÷ ù²ø/ó����� ÷ ÿ � 1 ©úK÷�hZ�f÷ ómü�÷ � @ a ! Æ�) Î Æ eH)¢×¢×¢×�)@a ! Ë ) Î Ë e���!;ø�ù� d÷�ø ÷I÷Irfù���þfø ÷ úd��� ô ö ! Ù # a�cx)Î Ù #ë�(' �*)¢��&�� ú©ô ÿ��( ��fô��(� ó9C �"� ÷�ÿ �T JUL"IKJ�LVM9ÿK÷µù�ø%�²ù�ø(�m÷�¥ö¤�Pþ��²ô öm÷�ú�� ú��zD ! [

��/ ¹�jVL ± @ n Ë� o Æ C8D ! � [ Î � ! �©./ZY[]\9ì0¯¡¯Ó��@ �*)¢×¢××�)�½E¹�jVL�Ô`Ù�@}ú��� ózD ± Ù#[@� ± Ù|� ¥ ÔNÆi /Z\@jVL@J�\kG5ö¤�Pþ��²ô öm÷�ú�� ú,�zD ! [ � ý ñ �ì ý ìuþ«ì Ä ì ðq��� ��mÿû÷ � / i �19�öm÷ � �×ó��²ÿ�� �£÷µù�=Iø ÷µù�ÿKó�� ó� Óù²ø( �� ÿ�� ô ö���ÿ��(r�/o v�´ · �©·�� � }>~�� ������yÑ}�|Æ� �,�� �l����|"yn�¢y�¹ ·~��¹�}��

|"��{ය�op�mÿ � � ó�� ÿ��¬�£÷µù�=�ø ÷µù�ÿKó�� ó� �ù²ø� �� ÿ�� ô ö��ª� ú����²ô����²ùô ÷t ���ô öm÷p!#�²ø ø(�'��� ó� �ú��(��þmø ÷;�(m÷ ù��"§.�mþZ�þ�� úK÷�ôKö�ù�ô � @~a ! Æ ) Î Æ eH)¢×¢××�)@a ! Ë�) Î Ë eA� ú�ù¤ü� ø ø ÷ ü ô���� ó���!�ø�ù� ÷ ø ÷5÷Irmù���þmø ÷ ú]��öm÷ ÿû÷ Î Ù,@ú��� ózDB+ë/ ! Ù [t!#� ÿI÷ ù²ü�ö¾�i@ �*)¢××¢×H)ïÉn×�0;�'�¯�(��ù� �� óm÷ÿû÷ þmø ù²ü� ó� 1÷µù²ü�ö2óm÷� �ù�ô���� ÷b÷6rfù���þmø ÷a ! Ù ) ' �"e«� ó � ¤�ô öf÷�÷h����Ò��ù²ø ÷ ó�ô�þ�� ú�� ô���� ÷õ÷Irfù���þfø ÷�a '�! Ù )�@egô��Ê�� fô�ù�� óBù�óm÷6�7ü�� ø ø ÷ ü�ô��(� ó� , ��!�ó�� ÿ���ù²ø��(s ÷�Ò÷6rfù���þmø ÷�ú/-�/÷�ô ± # adc ÷�ô öm÷¥ù�� ÷�ÿ ù� ÷Xø��_ü ù�ô���� ó ��!�ù²óE÷6rfù���þmø ÷Ê� ó� , )��³/ ÷�/ ± � ú�ô öm÷îÓûü�÷ ó�ô ÷�ÿA��!\��ù²úûú|Ô<��!£ôKöm÷�þ���� ó�ô�ü ø����� � , ×3§.� ómü ÷�÷ ù²ü�öD÷6rfù���þmø ÷4� ó � , � úþ�� ú�� ô��Ò�²÷�q¡÷µù²ü�ö)÷6rfù���þmø ÷�� ó � , ���múûô�ø(� ÷4� ó)ôKöm÷�ú ù���÷Óú��(f÷_��!£ô öm÷�ö¤�_þ ÷ ÿûþmø�ù�óm÷3+^/ ! @:Bù²ú�ô öm÷:� ÷�ü�ô��²ÿ,+�)�ú���ü�ø ÷µù�ÿKø�� ± ���mú©ô�ù²ø ú���ø(� ÷]�²ó�ô ö�� úgú��(f÷A��!Åô öm÷�ö��Pþ ÷ ÿûþmø�ù²óf÷�/;o�óm÷A���( ²ö=ô÷H�²÷ óXö�� þ ÷aôKö�ù�ô ± )�� ÿWú�����÷aÿK÷ ø ù�ô ÷�0� ÷�ü�ô�� ÿqmþ���� ó=ôKú�� ó�ù²þmþmÿ��'r.����ù�ôK÷ ø���ô öm÷�úKù���÷p�� ÿû÷ ü ô��(�²óù²ú�ô öf÷8�²÷ ü ô�� ÿX+z×

op�mÿ � � ó�� ÿ�� �£÷µù�=)ø ÷µù²ÿûó�� ó� �ù²ø� �� ÿ�� ô ö��<q���ö�� ü�öb�W÷�ü ù²ø øÍÿ��(r¥qò� ú�þmÿû÷ úK÷�ó�ô ÷-� ólq��� ��mÿK÷� /á©./)9�öm÷�� ÷�ü�ô��²ÿ ± � úÇôKöm÷0Óûü ÷�ó=ôK÷ ÿ\��!���ù²úKúDÔ���!cô öm÷gó�� ÿ���ù²ø��(s�÷Óþ���� ó�ô ú1��� ôKö�ÿû÷ úKþ ÷ ü ôWô���ô öf÷þmÿ��� �ù� �� ø(� ô��p�� úûô ÿ��( ��_ô��(�²ó�CÏ��ö�� ü�ö�� úÄþ�ù�ÿûô3��!fôKöm÷�� ómþ��_ô¡ô��Ðÿ��(r�D@ú���ÿ��mómó�� ó� �ÿ��(r�ÿK÷�þd÷ ù�ô ÷�mø��� óõôKöm÷�ú ù���÷��ùô�ù�úK÷ ô � ��fô)��� ôKö4���¹d÷ ÿû÷ ó�ô1�� úûôKÿ��( ��fô���� ómúKC3ü ù²ó4�.� ÷ ø�����¹d÷ ÿû÷ ó�ôà��ù²ø��m÷ ú�!#�²ÿ± [I/"�z��=²÷aôKöm÷p� ófø(� óf÷ � � ó�� ÿ���ù�ø( �� ÿ�� ôKö��8q�ô öm÷ùÿ��(r�ù²ø( ��²ÿ�� ôKö��7ôKÿ ù²ómú­!#� ÿ���ú£ôKöm÷,�²÷ ü�ô�� ÿ ± ô���ù� ÷�ü�ô�� ÿ�Ô �mú�� ó� �ô öm÷���ù²þmþ�� ó� æÔ`Ù�@}ú��� ózD ± Ù#[@� ± Ù|� ¥ ÔNÆ�×N9�öm÷�ÿK÷ ù²ø�� �²ù�ø(�m÷�¦ÿ �(r�ö¤�Pþ��²ô öm÷�ú�� ú,� ú�ùúKü ù²ø ÷�3�²÷ ÿKú��(� ó_��!côKöm÷aø�� óm÷ ù²ÿ)!#�mómü ô���� ó�ù²øNm÷6f�óm÷�` ¤��ô öm÷�� ÷ ü ô�� ÿ~Ô�×�9�öm÷�!#� ø ø(�'��� ó� �ôKöm÷�²ÿK÷�÷ ú©ô�ù� fø(� úûöm÷ ú�ôKö�ù�ô�ô ö�� ú�ú�����þmø ÷¦ù²ø� �� ÿ�� ô ö�� � úd� óÊ!@ù²ü�ôgù4�£÷µù�=�ø ÷µù²ÿûóm÷ ÿH�

� E

Page 109: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

?A@ j([]\¡j ê Ù�ï��eÿ �(rÒH(X>UmD­�"?�© '�� [6M�ù�P�U;�iV(P�U�FIKNHLKZJ�U�VêJZEGFIHLRLTZÚ W¢EGFÐ+§ö.K�SZP6F�CÐW¢EGF � @�ãé}ê ë¾ ì Ä ì ð ì Å ì þ ×ü:\¡[Ê[�Y|M\�/÷ ô � @�a ! Æ ) Î Æ e�)¢×¢×¢×�)@a ! ËÍ) Î Ë e� ÷�ù�úK÷�h��m÷ ómü�÷;��!�ø�ù� ÷ ø ÷õ÷Irmù���þmø ÷ ú)��öm÷�ÿK÷ !2# 7ù²ó� Î @�ú��( ²ózD�+½/ ! [�!#� ÿ¡÷H�²÷ ÿ­�õþ�ù�� ÿ`a ! ) Î e # � )�ù²ó��ø ÷�ô.C d÷Wù«�� úûô ÿ��( ��_ô��(�²óA�"� ÷�ÿ � ×�%)÷\��� ø øúKö��®�HôKö�ù�ô�ô öf÷�ö¤�Pþ���ô öm÷�ú�� úÐ����ö�� ü�öEÿ�� r�D � ) � )|C_[¦ÿK÷ ô��mÿûómúA� ú�ùbD­�"?�© '.� [��×ù²þmþmÿ��¢r�����ùô�� ÿ!#� ÿQ+8�mó�m÷�ÿXC>×

9ò�¥úK÷�÷�ô ö�ù�ôÜ�^��ù²þfú�7 � ó�ô��¥z ' �*)��{ )/ó��²ô ÷õôKö�ù�ô5!#� ÿ�ù²ó¤� !�# 7 ¦��� ø(f÷ ÿ»�ú5� óf÷h���ù²ø�� ô���(��þmø�� ÷ ú ���3D ! [@��@ � Ô / ! �$�Ô«$ ¬ $�7 $y¥ = $�Ô3$ ¬ $ ! $y¥$�Ô«$ ¬ $�7 $y¥ = $�Ô3$ ¬ $�7 $y¥$�Ô3$ ¬ $�7 $y¥ @¶��×

0;�'�&�W÷¦úûö��'�Zô ö�ù�ô�� óm÷�hZ�mù²ø(� ô��8D � /��®["!"ÿ���� §_÷ ü ô���� ó � /á©./á©�ö�� ø�mú/�§.� ómü ÷��zD ! � [ # z ' �*)¢��{ù²ó� Î � #l�(' ��)¢��&��W÷�ö�ù�� ÷�ô ömù�ô

���3D ! � [ ' Î � ��@7� ' Î � �zD ! � [�)ù²ó�Xô ö��mú

�©Ë�� o Æ C8D ! � ["�ÿ�zD ! � [ ' Î � �Ï@ �

©Ë�� o Æ C8D ! � [D­� ' Î � �3D ! � [�[

@ �© ' �

©]$�7 $y¥ � n Ë� o Æ C8D ! � [ Î � D�Ô / ! � [$�Ô3$ ¬ � ×9�ö��mú,� ô�ú��.´�ü�÷ ú�ô���úKö��'�sô ö�ù�ôn Ë� o Æ C8D ! � [ Î � DãÔ / ! � [$�Ô«$ ¬ 1 �wÅ ¨ Ä$�+�$ ¬ ×%)÷�fmÿKúûôgó���ô ÷¦ô ö�ùô Ë�� o Æ C8D ! � [ Î � DãÔ / ! � [ @ Ô / òó Ë�� o Æ C8D ! � [ Î � ! � ôõ@ Ô / ±

@ Ë�� o Æ � ± � � ¥@ $ ± $ ¥¥� �

Page 110: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

ù²ó��öm÷�ómü ÷�ô öm÷gø ÷6!qô�� ö�ù²ó��ú��(m÷���!dô öm÷�m÷�ú�� ÿû÷_� óm÷�hZ�mù²ø(� ô���÷�hZ�mù²ø ú8$ ± $ ¥¥ ?�$�Ô«$ ¬ ×�%)÷�ù²ø ú���ömù"�²÷

$�Ô«$ ¬ @ � c� Ùpo Æ � � ± Ù|� ¥ ÔNÆ ² ¬ � Æ�£ ¬@ � c� Ùpo Æ � ± Ù|� ¥ � Æ�£ ¬@ $ ± $ ¥ £ ¬¥ )

��öm÷�ÿK÷�� óBô öm÷�úK÷�ü� ó�{÷h���ù�ø(� ô��i�W÷��múK÷�)ôKöm÷4!@ù²ü�ô�ô ömù�ô_D � ' �®[èª>@ � ×3+�� ófúK÷h��m÷�ó�ô ø��Dô öf÷ø ÷6!qô�� ö�ù²ó�)ú��(f÷�ü ù²ób ÷_!#�mÿ©ô öm÷�ÿ�ú�����þfø(��fm÷bô��ý$ ± $ ¥¥ ?�$�Ô3$ ¬ @¸$ ± $ ¥ Ô ¥ £ ¬¥ @¸$ ± $y¥V);ù²ó�{ôKö��mú���mÿ« ���ù²øò� ú�ô��IúKö��'�}ô ö�ù�ô0$ ± $y¥ 1ò�wÅ ¨ Ä`?�$�+�$ ¬ ×z§Z� ómü�÷ �wÅ ¨ Äò= +�/ND Î � ! � [,!#�²ÿö�Ê@:�*)¢××¢×�)«Éº)�£÷�ö�ù��²÷

�wÅ ¨ Äh= Ë�� o Æ C8D ! � [�+«/ZD Î � ! � [ @ +3/ òó Ë�� o Æ C8D ! � [ Î � ! � ôõ@ +3/ ±= $�+�$ ¬ $ ± $y¥V)��öm÷�ÿK÷�ô öm÷�ø�ù�úûô�ø(� óm÷«!#� ø ø(�'��ú�!"ÿ�����ôKöm÷]¦��� ø�m÷ ÿd� óm÷h���ù²ø�� ô���q�ù²ó�Xô öm÷¦ôKöm÷� ÿû÷� � ú�þmÿ��"�²÷�/

9�ö��múq ô öm÷�ú�����þmø ÷6ÿ��(r{ù�ø( �� ÿ�� ôKö�� üµù�óÒúK÷ ÿ�� ÷Xù²úõù8�W÷ ù�={ø ÷µù²ÿûó�� ó� ¤ù²ø� �� ÿ�� ô ö�� !#�²ÿ�ô öf÷ö�ù²ø�!"úKþ�ù�ü ÷¤ø ÷µù²ÿûó�� ó� )þmÿ��� mø ÷��8/sò×ó�ô öm÷¤óf÷6rPôXúK÷�ü�ô���� ók�£÷n�múK÷�ô ÷ ü�öfó��(h��m÷�ú>!"ÿ���� ��Z� ú©ô�� ó� ù²ó�Iø ù²ÿ� ²÷;��ù²ÿ� �� ó�ü ø�ù�úKú���f�üµùô��(�²ó�ô����� fô�ù�� óIù�ú©ô ÿ�� ó� õø ÷µù²ÿûó�� ó� �ù²ø( �� ÿ�� ô ö�� ��ö�� ü�öIö�ù²úWú���ù²ø øú ù���þmø ÷¦ü����þmø ÷6r.� ô���/

o v�Ý �>|1}3� ���� {)} �`{1|1}�y�¹ �l����|"yn�¢y�¹p ��ç;��� �-�,��� ����Á�� ��� �hÌ�5�­���\��� ���½~�hÌUÌz�p��À�Ì%Úò×óÏD³¡¦÷µù²ÿûómú�ù²ó�-ºÇù²ø(� ù²ó�ô¢q��'��� � [�ô öm÷>!#� ø ø��'��� ó� Ñh��m÷�úûô��(� ó2��ù²ú�þ�� úK÷�%�8� úÓ÷H� ÷�ÿ��Eü�²ómü ÷�þfôü ø ù²úKúA��ö�� ü�ö � ú�÷�´�ü�� ÷ ó�ôKø��^�W÷ ù�==ø��5ø ÷µù²ÿûó�ù� mø ÷�ù�ø ú��÷�´�ü� ÷ ó�ô ø��{ú©ô ÿ��²ó� ø��Dø ÷µù²ÿûó�ù� mø ÷ç�Hò×ó1ù²ó�(��þ�� ÿûô ù²ó�ô ÿK÷ ú��mø ôÇô ö�� ú"h��m÷ ú©ô��(�²ó_�Wù²ú�ù²ófú��£÷ ÿK÷�Ê� óÓô öm÷�ù�´�ÿ���ù�ô��Ò� ÷� ¤�_§_ü�ö�ù�þ�� ÿû÷4Dj§Pü�ö�ù²þ�� ÿK÷�q�'����B¤[6q���ö��_ �ù�� ÷�ù�ó6E'E'X6R�HLKZJ�ù²ø� �� ÿ�� ô ö�� ��ö�� ü�ö¤üµù²ó8 d÷]�múû÷ô���÷6´�ü�� ÷�ó=ôKø��IôKÿ ù²ómú­!#� ÿ��Ñù�ó���£÷µù�=Zø ÷ ù²ÿKó�� ó� Eù²ø� �� ÿ�� ô ö�� � ó�ô���ù1úûô ÿ�� ó� �ø ÷µù�ÿKó�� ó� Eù²ø( ��²ÿ�� ôKö��</[ò×ósúK÷�� ÷ ÿKù²ø�ú��� múû÷h��m÷ ó�ôþ�ù²þ ÷ ÿûú`��ù²ÿ������múd�(��þmÿ��@�²÷Ê ��.�²úûô�� ó� Óù²ø� �� ÿ�� ô ö���ú"�W÷�ÿK÷�m÷H�²÷ ø�� þd÷�bD�q�ÿK÷�fó��q��'����B�euq�ÿû÷�mó�¸q�'����©.e*q�ÿK÷��mó��qN�¢���¤£.e�q�ÿû÷�mó��ù�ó��§_ü�ö�ù²þ�� ÿK÷�qN�'��� � [6e��£÷,�fúK÷�ô öf÷Qr��xw��0� ���>��ù²ø� �� ÿ�� ô ö��C!"ÿ����

�� 

Page 111: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

FHG.IKJ�L*M ÿK÷ ù²ø/ó����� ÷ ÿKú,B 4 � )�� 4 ƾúK÷�hZ�f÷ ómü�÷ � @ a ! Æ ) Î Æ eH)¢×¢×¢×�)@a ! ËÍ) Î Ë e���!;ø�ù� d÷�ø ÷I÷Irfù���þfø ÷ ú�W÷ ù�=Iø ÷ ù²ÿKó�� ó� Óù²ø� �� ÿ�� ô ö�� ÿ��]� � � z ' �*)¢�H{T JUL"IKJ�LVM9ö��Pþ��²ô öf÷ ú�� ú��zD ! [

��/ ¹�jVL��&@ ƾ�� Ä ø��� Æ�©./ZY[]\9ì0¯¡¯Ó��@ �*)¢×¢××�)«É]¹�jVL:C Æ D ! Ù [1@ ÆËi /ZY[]\�P1@7�*)×¢×¢×�)�� _�[� / ø ÷�ôÍ� m ÷¦ô öm÷����_ô þ��fô,��!¶ÿ���DBC m ) � [£./ ¹�jVL�² m @ ƾ n ËÙpo Æ C m D ! Ù [@��� m D ! Ù [ ' Î Ù|�E / ¹�jVL�¨ m @ ƾ ø ó�D�D­� ' ² m [|?�² m [� / Y[]\9ì0¯¡¯Ó��@ �*)¢×¢××�)«É]¹�jVL

C mÖ Æ D ! Ù [1@ C m D ! Ù [f÷6r_þzD ' Î Ùf¨ m � m D ! Ù [�[ m��öm÷ ÿû÷� m @ n ËÙpo Æ C m D ! Ù [f÷6r_þzD ' Î ÙB¨ m � m D ! Ù [�[�� ú�ù�ó�� ÿ���ù²ø��(s�� ó� ]!@ù²ü�ô�� ÿd��ö�� ü�ö÷ ómú��mÿK÷�ú�ô ömù�ôQC mÖ Æ�� ú�ù4�� ú©ô ÿ��� ��fô��(� ó

��/Zj G._K_�[�'B�/3[�J�L"IKJULõù²ú�fmó�ù²ø/ö¤�Pþ��²ôKöm÷ ú�� ú,�zD ! [ � ú��( ó�D�¼)D ! [�[���öm÷ ÿû÷

¼)D ! ["@ n"! m o Æ ¨ m � m D ! [n ! m o Æ ¨ m ×

qò�( ��mÿû÷ � / � �19�öm÷:r#��w��0� ���;�¥ù�ø( �� ÿ�� ôKö��8/

���

Page 112: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

D�q�ÿû÷�mó��ù²ó�i§_ü�ö�ù�þ�� ÿû÷�q��'��� � [���ö�� ü�ö8� ú�úKö��'��ói� ó^q��( ��fÿK÷ i /op�mÿ�ó��²ô ù�ô���� ó�!#� ÿÅôKöm÷gù²ø( �� ÿ�� ô ö�� � úýú����_� ø�ù²ÿÄô���ôKö�ù�ô1��!)Dj§Pü�ö�ù²þ�� ÿK÷�P6RòU�V�¨(qN�'���� .e¤§_ü�ö�ù�þ�� ÿû÷

ù²ó�ʧ.� ó� ²÷ ÿq��'���� ¤[6/)9�öf÷p� ómþ��fôWô��Ðr#��w��0� ���;�t� ú£ù�úK÷�h��m÷ ómü�÷ � @ a ! Æ ) Î Æ e�)¢×¢×¢×H)%a ! ËÍ) Î Ë e"��!ÉFø�ù� ÷ ø ÷B÷6rfù���þmø ÷ úq¡ù`�£÷µù�=¤ø ÷µù²ÿûó�� ó� Xù²ø( ��²ÿ�� ôKö��Yÿ���qNù²ó�bô����Xþ�ù²ÿKù���÷�ô ÷�ÿKú]B 4�� )�� 4�@?�©Z×N�5�Ò� ÷�ó¥ù��� ú©ô ÿ��� ��fô��(� ó0C m �"� ÷�ÿgù��ù�ô�ùõúû÷�ô � )�ù²ø� �� ÿ�� ô ö��hÿ �t���fô þ��fô ú�ùõö¤�Pþ���ô öm÷�ú�� úÍ� m��ö�� ü�ö8��ù²þmú � ô��6z ' �*)��{j×�r��xw��0� ���>�Ê�\� ÿ�==ú�� ó¥ùõúû÷h��m÷ ófü ÷���!Åú©ô�ù� ÷�úq���öm÷�ÿK÷�� óIú©ô�ù� ²÷8P\� ô ÷�óm÷ ÿKù�ô ÷�ú£ùp�� úûôKÿ��� ��fô���� ó�C m ù²ó��ÿ��mómúÍÿ���ô��5�� fô�ù�� óÓù�ö¤�Pþ��²ôKöm÷ ú�� ú½� m ×�9�öm÷df�ómù²øur#�xw$�0� ���>�ö¤�Pþ��²ôKöm÷ ú�� ú,� ú�ù�ø(� óm÷µù�ÿWô ömÿû÷ úûö�� ø(>!#�mómü ô���� ó>�@�²÷ ÿ�ô öf÷�ö¤�Pþ��²ô öm÷�úK÷�ú � Æ )¢×¢×¢×H)�� ! ×q�ÿK÷�fó��ù²ó�Ò§_ü�ö�ù�þ�� ÿû÷�ö�ù��²÷úKö��®��ó�ô ö�ù�ô���!�ôKöm÷ù�ø( �� ÿ�� ôKö��Àÿ�� � ú�ùÛD­�@?�© '�� [Ë�Ë�£÷µù�=ø ÷ ù²ÿKó�� ó� ¥ù²ø� �� ÿ�� ô ö��<q��³/ ÷�/�÷ ù²ü�ö{ü ù²ø ø1��!~ÿ��n� ó¥r��xw��0� ���>�^ ÷�óm÷ ÿKù�ô ÷�úõùXö��Pþ��²ô öf÷ ú�� úÐ� m ú��mü�öô ömù�ô�² m DÆù²ú5f÷�f�óm÷��� ó¤ø(� óm÷�£�[,� ú¦ù�ôp�_� ú©ô4�@?�© ' � qcôKöm÷ óBô öf÷�!"ÿ ù²ü ô��(�²ón��!Ç÷6rfù���þmø ÷�úp� ó ���ö�� ü�öBù²ÿK÷]��� úKü�ø�ù²úûú���f�÷> ¤�Xô öm÷]f�ó�ù²øÄö¤�Pþ��²ôKöm÷ ú�� ú��i� ú�ù�ô,�_� ú©ô%��׸�5�Ò�²÷ ó¥ô ö�� úgÿû÷ ú��mø ôq�� óf÷úûôKÿ ù��� ö�ô�!#� ÿ­��ù�ÿ�>��ù¢�Xô����� fô ù�� ó�ù�ú©ô ÿ�� ó� �ø ÷µù�ÿKó�� ó� �ù²ø( ��²ÿ�� ôKö�� !#�²ÿ,���mÿ�ö�ù²ø�!"úûþ�ù²ü�÷�ø ÷µù²ÿûó�� ó� þmÿ��� mø ÷��¶� úýô��5mÿKù¢��ù�ú���´�ü� ÷�ó�ô ø��õø�ù²ÿ� ÷5D@ù²ú�úKþ ÷ ü���f�÷�_ ÷ ø��®�;[¡ú ù���þmø ÷ � !"ÿ����3ôKöm÷�÷6rfù���þmø ÷� ÿKù²ü ø ÷Í5872D�+�)|C_[ýù�ó��ÿ��mó0r#�xw��0�{���>�_� ó � �fú�� ó� �ÿ��(rÓù²ú ô öm÷��W÷ ù�=Óø ÷ ù²ÿKó�� ó� �ù²ø( �� ÿ�� ô ö��<q �ù²úò ��Ò�²÷ ó�� ó�9�öm÷�²ÿK÷� � / i q ù�ó�&� 4 �@?�� � �î×®9�ö�� ú;ü�ö���� ü ÷���!'��÷ ómú��mÿK÷�úýô ö�ùôàr#�xw$�0� ���>��»�ú1f�ó�ù²øö¤�Pþ��²ôKöm÷ ú�� ú���ù�=²÷ ú ó���÷�ÿKÿ��²ÿKú"� ó � e¤�_� ÿû÷�"� ÷�ÿq=ú�� ómü ÷�÷ ù²ü�ö�ö¤�Pþ��²ô öm÷�ú�� ú\ ÷�óm÷ ÿKù�ô ÷�_ ¤�Aÿ��(r�� ú��!/ôKöm÷;!#� ÿ�� � m D ! ["@OW m / ! !#�²ÿWú�����÷ÍW m # a c ) r#�xw$�0� ���>��»�údf�ó�ù²ø ö¤�Pþ��²ôKöm÷ ú�� ú���� ø øN ÷p��!/ô öf÷!#� ÿ��b�zD ! [�@2ú��( ózDBWS/ ! [�!#� ÿ�ú�����÷QW # a c ×�ú�ú�� ó� ¦ôKöm÷d!@ù²ü�ô ô ömù�ô�ô öm÷dºÇù²þfó��(=��­+Wöf÷ ÿD��� óm÷�ó�=�� ú�����÷�ómú���� óÑ��!Çô öm÷�ü�ø�ù²úûúp��!\s ÷�ÿ����«ô ömÿû÷ úKö�� ø(Bø�� óm÷ ù²ÿaô öfÿK÷ úûö�� ø�Ñ!#�mómü�ô��(� ófú5�@�²÷ ÿ]a c � ú5½K)cô öf÷�£÷ ø øÇ��==ó��'��ó�ôKöm÷�²ÿK÷�û��!1�Wø�����÷�ÿq�O�ömÿû÷ ó�!"÷��mü�ö�ô¢q�¦�ù��múûúKø ÷�ÿaù�ó�8%{ù²ÿ����fô ömD³�Wø�����÷ ÿ�P6RdU�V�¨(q�'�� ��¤[z����þfø(� ÷�úNô ö�ùôò��� ô ö�þfÿ��� �ù� �� ø�� ô���ù�ô;ø ÷ ù²úûô�� 'Ð� ô öf÷\f�ó�ù�ø_ö¤�Pþ���ô öm÷�ú�� ú��4� úÅù�ó�²­�×ù²ü ü��mÿ ù�ôK÷ö¤�Pþ��²ôKöm÷ ú�� ú«!#�²ÿÍ+Â�mó�f÷ ÿ,Câþfÿ��"�Z�(m÷�¤ô ö�ù�ô«� � � 1)( D�²6ÔNÆD�½¦ø��� ¥D³²IÔNÆ�[ Å ø��� ¥D � ÔNÆ�[�[�[d!#� ÿ�ú�����÷ü�²ómúûô ù²ó�ô ( RÍB�×

9�ö�� ú�ù²ó�ù�ø��Pú�� úqmô ö����� ö�ù�ôKôKÿ ù²ü ô���� ÷ ø��Iú�����þfø ÷�q.�.� ÷ ø�múgù�ÿKù�ô öm÷�ÿ�ü�ÿ���f÷A ����mó�8� ó¥ú ù���þmø ÷ü����þmø ÷6r.� ô��_��ö�� ü�ö8��P÷ ú�ó��²ô,m÷�þd÷�ó�Ñ� óXô öm÷�þ�ù�ÿûô�� ü�mø ù²ÿKúd��!;ôKöm÷�ø ÷µù²ÿûó�� ó� �þmÿ��� fø ÷�¯D��j/�÷�/Æ+ù²ó�«7^[6/Åò×ó�ô öm÷�ÿK÷�úûô\��!Nô ö�� úÇúK÷ ü ô���� ó_�£÷p�múû÷aÿK÷�ü ÷�ó=ôWÿK÷�ú��mø ô ú��²ó0r#�xw$�0� ���>��»�ú�ù� �� ø(� ô���ô��A ÷ óZ�÷ ÿKù�ô ÷gù�ø�ù²ÿ� ÷6�Ë��ù�ÿ� �� ó�ü ø ù²úKú���f�÷�ÿÇù²ó�Óô öm÷� ÷�óm÷ ÿKù²ø(��sµù�ô��(� ó�ù� �� ø�� ô��A��!cø ù²ÿ� ²÷6����ù²ÿ� �� óõü�ø�ù²úûú���f�÷ ÿûúô��� ���� ÷�ù4���mü�öXô��� ö�ô ÷ ÿd ����mó�8� óXúKù���þfø ÷¦ü����þmø ÷6r.� ô��_!#� ÿ�ô ö�� ú�ø ÷ ù²ÿKó�� ó� Óù²ø� �� ÿ�� ô ö��8/

�'B�B

Page 113: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

p ��ç;��ÿ �-�,��� ����Á��]qÒ��ÀÆ�+*¥ÀÆ�­�p�.Ë,�ÏÚ;����up�3�.�z����� �hÌ�5�������¿Àgè�À1���\�- À1���\�DÁ§.�mþfþ�� úû÷�ô ömù�ô«�ª�K7 � �(' �*)¢��&>� úõùü�ø�ù²úûú���fm÷ ÿ���!�ô öm÷t!#�²ÿ��¸�zD ! [4@Ìú��� ózD�¼�D ! [�[5��öm÷ ÿû÷¼ ��ù²þfú87 � ó�ô��îz ' ��)¢��{³×z%)÷�ú ù¢�Bô ö�ùô�ôKöm÷<Ú�U�FjJ�HLK8��!X�^� ó5ùXø�ù� ÷ ø ÷b÷6rfù���þmø ÷2a ! ) Î ep� úÎ ¼)D ! [IeNó��²ôK÷ÓôKö�ù�ô¦ôKö�� ú5h���ù²ó�ô�� ô��Â� ú¦ó�� ómóm÷� �ù�ô���� ÷���!£ù²ó��� ófø��Ñ��!Q�Dü� ÿûÿK÷ ü ô ø��¤þmÿû÷�� ü�ô ú�ô öf÷ø�ù� d÷�ø Î ù²úûú��Pü��ùô ÷>��� ôKö ! ×N9�öf÷A��ù� ó�� ô���f÷5��!ÅôKöm÷���ù²ÿ� �� óIü ù²ó< d÷ �Z� ÷6�W÷�Bù²úgù���÷µù�ú��mÿû÷��!¡ô öm÷�ü�� ó�fNm÷�ómü ÷]��� ôKö>��ö�� ü�öXô öm÷�ü ø ù²úKú���f�÷�ÿd��ù�=²÷ ú�� ô ú�þmÿû÷�� ü�ô���� ó>� ó ! ×

9�öm÷i!#� ø ø(�'��� ó� bôKöm÷�²ÿK÷�<q���ö�� ü�öÒ� ú�ù²ó§÷IrPô ÷ ófú��(�²óÒ��!59�öm÷�� ÿK÷�� £n!"ÿ���� Dj§Pü�ö�ù²þ�� ÿK÷2P6RU�V�¨(qz�'���� �[6q�úKö��'��úaôKö�ù�ôÍr#��w��0� ���;��üµù�ói ÷��múû÷Ñ� ó�ü�� ó¡Ñ��mómü ô���� ó8��� ô ö¤ù�ÿK÷ ù²ø�� �²ù�ø(�m÷�<�£÷µù�=ø ÷ ù²ÿKóf÷ ÿ�ô����� _ô�ù�� ó¥ø ù²ÿ� ²÷6����ù²ÿ� �� ó�ö��Pþ��²ô öf÷ úK÷�ú/?A@ j([]\¡j ê Ù�ï��/.]ö®Y¤YNE'X'P"RLT�U�R r��xw��0� ���>�ÊH(X1F�öZKtEGK�U�K_P&J�U�Ú«Y�V(P)X'P|õ�öNP6K¸OQP � @ a ! Æ�) Î Æ eH)¢××¢×�)a ! Ë ) Î Ë e�ö¤X6HLKZJ>U�ù�P�U;��V(P�U�FIKNHLKZJ>U�VêJZEGFIHLR#TZÚhÿ��]� � � z ' �*)��{j×�ø�TNP6K�W¢EGF4U�K�ô3ÎGU�V ö¥P«? 1 Bù�PpT�U�Î�P � � � #î� �*)I©x)¢××¢×H)ïÉ�&�� Î Ùã¼)D ! Ù [`=&? &��É = © ! !0m o Æ 1 ² Æ�Ô�2m D�� ' ² m [ Æ Ö 2 ×ü:\¡[Ê[�Y|M�9�öm÷�þmÿ��Z��!aü���� �� óm÷�ú���m÷µù�ú�!"ÿ���� Dj§_ü�ömù²þ�� ÿû÷-P6R�U�V�¨(qp�'���� ¤[6q"��öm÷�ÿK÷i� ô�� ú�úKö��'��óô ömù�ôSr#�xw��0�{���>����� ô ö� �� ó�ù²ÿ��÷��ù²ø��m÷�ö¤�Pþ��²ôKöm÷ úû÷ ú� ²÷ óm÷�ÿ ù�ôK÷ ú�ù�ø�ù²ÿ� ÷���ù�ÿ� �� ó�ü ø�ù�úKú���f�÷ ÿ�qfù�ó�Dj§Pü�ö�ù²þ�� ÿK÷õù�ó�^§.� ó� ÷ ÿ�qò�'���� ¤[6q¥��öm÷�ÿK÷�ù²ó¤ù²ó�ù²ø��_ú�� ú;� ú« ��Ò� ÷�ói!#� ÿÍr��xw��0� ���>��»�ú¦ü ø�ù�úKú���f�üµùô��(�²ó÷ ÿûÿ�� ÿ���� ôKö§ÿû÷µù²øQ��ù²ø(�f÷§ö¤�Pþ��²ô öm÷�úK÷�ú/ *aút� óÛ9�öf÷� ÿû÷� £^��!4Dj§_ü�ö�ù�þ�� ÿû÷2P6R�U�V�¨(qp�'���� �[6q���!Î Ùu¼)D ! Ù [`=&?�ô öm÷�ó Î Ù !�m o Æ ¨ m � m D ! Ù [Q=&? !�m o Æ ¨ m��ö�� ü�ö8����þmø�� ÷�ú£ô ömù�ô

÷Irfþ � ' Î Ù !�m o Æ ¨ m � m D ! Ù [ Å ? !�m o Æ ¨ m � 1 ��×qN� ø ø��'��� ó� >D³§_ü�ö�ù²þ�� ÿK÷tP6R�U�V�¨(q��'���� ¤[6qZ�W÷�ô ö��múgö�ù�� ÷� � � #ë� �*)Q©x)¢×¢×¢×�)«É�&�� Î Ùu¼)D ! Ù [S=&?(&��É = Ë� Ùpo Æ �É /Í#q÷6r_þ � ' Î Ù !�m o Æ ¨ m � m D ! Ù [ Å ? !�m o Æ ¨ m � )

@ ÷6r_þ � ? n3! m o Æ ¨ m ²É Ë�Ùpo Æ ÷6r_þ � ' Î Ù !�m o Æ ¨ m � m D ! Ù [ ��'B��

Page 114: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

@ ÷6r_þ � ? !�m o Æ ¨ m � � !0m o Æ m � Ë� Ùpo Æ C ! Ö Æ D ! Ù [@ ÷6r_þ � ? !�m o Æ ¨ m � � !0m o Æ m � D � /î©�[

��öm÷�ÿK÷aôKöm÷aúû÷ ü�²ó�I÷h���ù²ø�� ô��4!#� ø ø(�'��ú1!"ÿ���� ôKöm÷«f÷�f�ó�� ô���� ót��!ÊC mÖ Æ ù²ó��ô öm÷,f�ó�ù²ø ÷h���ù²ø�� ô���� ú ÷ üµù��múK÷ÐC ! Ö Æ�� ú�ùt�� úûôKÿ��( ��fô���� ó¤ù²ó�¤öm÷ ófü ÷�ú�����úaô��Ñ��/�op�mÿ; ���ù²ø3� úgô ö��mú�ô��` ����mó�¤ô öf÷ÿ��� ö�ô�ú���m÷5��!�D � /á©�[6/

òË!��£÷�ø ÷�ô 8 m @ Ë�Ùpo Æ C m D ! Ù [ Î Ù � m D ! Ù [ô öf÷ ó<�mú�� ó� �ô öm÷5!@ù²ü ôgôKö�ù�ô ���zD ! � [ ' Î � ��@¶� ' Î � �zD ! � [�£÷]f�ó�¥ôKö�ù�ô,² m @ Æ�Ô#465¾ ×�§.�� fúûô�� ô��fô�� ó� �� ó=ô��õô öm÷�m÷6f�ó�� ô��(�²ó<��!ਠm �£÷]�� fô�ù�� ó

¨ m @ �© ø ó ú � Å 8 m

� ' 8 m û ×qN� ø ø��®��� ó� >D³§_ü�ö�ù²þ�� ÿK÷�ù²ó�8§.� ó� ÷�ÿq��'���� ¤[�!#� ÿ�ú��(��þmø(� ü� ô�����!;ó��²ô ù�ô���� ó`�£÷�ó��'� f�r�P�ù�ó�ø ÷ ô8+�Ù\@ Î Ùf� m D ! Ù [�)7 @8 m )%C¯@bC m )z²A@¬² m )»8`@×8 m )Äù²ó�ë¨&@¿¨ m ×�*�ú��(��þmø ÷�ü�� óV� ÷6r.� ô��ù²ÿ� �����÷ ó�ô�úKö��'��ú�ô ö�ùô 7 Ô ¯ Å = � Å +

© 7 Ô ¯ Å � ' +© 7 ¯

!#� ÿ�ù�ó��9¨ # auù²ó��ù�ó��0+ # z ' ��)¢��{³×N§.� ómü ÷ +�ÙÄù²ø���ù¢�Pú�ø(� ÷ ú�� ó�ô öm÷5� ó�ô ÷ ÿ���ù²ø�z ' ��)¢��{�)��W÷�ü ù²óù²þmþfø���ôKö�� úd� óm÷h���ù�ø(� ô���ô����� fô ù�� ó

@ Ë�Ùpo Æ C8D ! Ù [&7 Ô ¯ Å:9= Ë�Ùpo Æ C8D ! Ù [Sú � Å +�Ù© 7 Ô ¯«Å � ' +�Ù© 7 ¯ û × D � / i [

*aú�� óѧ_÷�ü�ô��(� ó i /á£���!dD³§_ü�ö�ù²þ�� ÿK÷�ù²ó�i§.� ó� ÷ ÿ�q��'���� �[6qmú��� múûô�� ô��fô�� ó� 0¨b� ó�ô���� óm÷�hZ�mù²ø(� ô��nD � / i [�Z� ÷�ø(mú

m = 1 � ' 8 ¾m@ 1 � ' D­� ' ©�² m [ ¾@ © 1 ² m D­� ' ² m [6× D � / � [

�'B¤©

Page 115: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

§.�� fúûô�� ô��fô�� ó� �� óm÷h���ù²ø�� ô��<D � / � [)� ó�ô���� óf÷h���ù²ø�� ô��>D � /á©�[�ù²ó�t�fú�� ó� �ôKöm÷,m÷�f�ó�� ô���� ó���!%¨ m �Z� ÷ ø(fúô öf÷]f÷ ú�� ÿK÷�8 ����mó�<��!¡ô öm÷¦ôKöm÷� ÿû÷�</

9�öm÷gÿK÷ ú��mø ôKú\��!z§_÷�ü�ô���� ó � / i �(��þmø���ôKö�ù�ô���!�ÿ�� r4� ú"�múK÷��ù²ú�ô öf÷,�£÷µù�=Óø ÷µù²ÿûó�� ó� �ù�ø( �� ÿ�� ôKö��� óAr#�xw�� � ���>��q�ôKöm÷ óXô öf÷8��ù²ø��m÷�² m ��� ø øNù²ø���ù¢�Pú� d÷�ù�ô���� úûô��"?�© '½� )�ù²ó�Iô öf÷]�fþmþd÷�ÿ� ����fó���!�9�öm÷� ÿû÷� � / � ÷ ü�����÷ ú�D�D­� ' © � [QÆ�Ô�2'D­� Å © � [QÆ Ö 2Q[ ! £Ë¾¢×¥9�öm÷�!#� ø ø(�'��� ó� õôK÷ ü�ömó�� üµù�ø¡ø ÷����ù� úd�fúK÷�!#�fø �éÆjxêëêëìnÙÊï�+ªD�� ' � ! [QÆ�Ô � D�� Å � ! [QÆ Ö � = � ' � ! ¾�W¢EGF�B�= ! = �@? � ×ü:\¡[Ê[�Y|M\úgú�� ó� �ùaú��(��þmø ÷Çü�²ó(�²÷6r.� ô��õù²ÿ� �����÷�ó�ô)� ô;üµù�ó� d÷½� ÷ ÿ���f�÷�õô ö�ùôਠ4 = � ' D�� ' ¨)[y8�!#�²ÿù²ó¤�3¨ 1 B�ù²ó��ù²ó¤��B�=�8Ð= ��×�9�ö�� ú�� óf÷h���ù²ø�� ô��4����þmø�� ÷�úÅô ö�ùô5D�� ' � ! [QÆ�Ô � = � ' � ! Å � ! ¾ù²ó�mD­� Å � ! [ � = � Å � ! ¾")�ú��Óü� ómúû÷h��m÷�ó=ôKø��

D­� ' � ! [ Æ�Ô � D­� Å � ! [ Æ Ö � =7D­� ' � ! Å � ! ¾ [D­� Å � ! [D�� Å � ! ¾ [�)��ö�� ü�ö8� ú�ù�ô��_�²úûô�� ' � ! ¾�!#� ÿ�B«= ! = �@? � ×

òË!z�W÷¦úû÷�ô,?]@ � ?�©õù²ó�Xù²þmþmø��t�/÷��_��ù � /á£]��� ôKö ! @ ?x)_ôKöm÷5�mþmþ ÷ ÿ� ����fó�Ê��!39�öm÷�²ÿK÷�� / � d÷�ü����÷ úAD­� 'n� ¾ [ ! £Ë¾ ù�ó�<�£÷]�� fô�ù�� ó�ô öf÷�!#�²ø ø(�'��� ó� u�; []\¡[G¯�¯�ì�\>²'Ù�ïÒí�÷#WÍr#�xw$�0� ���>�-H(X4F�ö.K EGK UtX'P|õ�öNP6K¸OQP � E�W�V�UVó6P6V(P�SiP&J�U�Ú«Y�V(PQX�S�F�U�ù1K4WIF�EGÚ5:7-D�+�)|C_[`ö�X6HLKZJ�ÿ�� r8UGX�RLTNPXù�P�U;�pV(P�U�F6K¸P6F«ú � UGX�SZP#Ì"K¸P�S]HLKëø�TNPQEGF�P6Ú=<®¨?>�U�K�S@� 4 �@?�� � � A�)RLTNP6K_RLTNP\T ôQYNEGRLTNPQX6H(XÆ�Aù�TZHjOQT�r#�xw�� � ���>�_JZP6K¸P6F�U�R�PQX½ù1HLVLV¤T�U�Î�P�Ú�U�F³J�HLK_U�R�V(P�UGX6R � ?�©_EGKÊP�Î�P6F�ôP&J�U�Ú«Y�V(P�HLK � ×ü:\¡[Ê[�Y|M�9�öm÷� ����mó�>� óB�Büµù��fúK÷ úC�2ô��� d÷5 ²ÿK÷µùô ÷ ÿ�ôKö�ù²ó ¾� Ä ø(�� ÆD E#D )mù�ó�Xü� ómúû÷h��m÷�ó=ôKø��Xô öf÷�mþmþ ÷ ÿ� ����mó�<��!ò9�öm÷�²ÿK÷� � / � � ú�ø ÷ úûú�ôKö�ù²ó^�@?�� � �á×

+�� ÿ��²ø ø�ù�ÿ�� � / E úûö��'��ú�ô ömù�ô�ùÐÑ����� ü��(���fú�ü�ö���� ü ÷���!�þmù²ÿ ù���÷�ôK÷ ÿûúA!#� ÿ:r#��w��0� ���;�{÷ ó�ù� mø ÷�ú���mÿ, ��.� ú©ô�� ó� ��� mù²úK÷�¤ù²ø� �� ÿ�� ô ö��[ô��_ ��²ôKö¤ÿ��fó÷�´�ü�� ÷ ó�ôKø���ù²ó�Ñ ²÷ óm÷�ÿ ù�ôK÷õù�fmó�ù²ø¡ö��Pþ��²ô öf÷ ú�� ú��ö�� ü�ö�ö�ù²ú�ùm��ù�ÿ� �� ó2��!�ù�ô�ø ÷µù²ú©ô � ?�©Â� ó�÷�� ÷ ÿ­�§÷Irfù���þfø ÷<� ó�ôKöm÷�ôKÿ ù�� ó�� ó� )úû÷�ô¢/sò×ó§ô öf÷óm÷Ir_ôgú��� múK÷�ü�ô��(� óÊ�£÷]�fúK÷A+�� ÿ��²ø ø�ù�ÿ�� � / E ù²ó�Iô ÷�ü�ömó��(h��m÷�úd!"ÿ����[ô öm÷�ôKöm÷� ÿ­�`��!¡ø ù²ÿ� ÷«��ù²ÿ� �� óü ø ù²úKú���f�ü ù�ô���� ó)ô��B÷ ú©ô�ù� mø�� úûöÒùi ����mó� � ó5ôKöm÷` ÷�óm÷ ÿKù²ø(��sµù�ô��(� ó)÷ ÿûÿ�� ÿA��!�ô ö�� úõö¤�Pþ��²ôKöm÷ ú�� ú4� óô ÷�ÿ���ú���!;ôKöm÷�ú ù���þmø ÷�ú���s ÷�Én×

�'B i

Page 116: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

p ��ç;�» è�À1���\� - À1���\�DÁp�ýÀÆÁ�Ëä �.Áp�3��ÀÆ�­�GFUÀ-�N���SÁIHi�N�N����/÷�ôKJ¶ ÷�ù�ü� ø ø ÷ ü ô���� óA��!�ÿK÷ ù²ø�� �²ù�ø(�m÷�]!#�mófü�ô���� ómúò� ó�ù�úû÷�ôà7i×�*2f�ó�� ô ÷Wúû÷�ô �"! Æ�)¢×¢×¢×H) !ML &ON 7� ú�ú ù���{ô��i d÷QP�MjXQT�U�R³R�P6F�P�Sîó�ôRJl��!�ô öm÷�ÿK÷Xù²ÿû÷�ÿû÷µù²ø£óZ���� d÷�ÿKúæ8 Æ )¢×¢×¢×H)«8 L ú��mü�öÒô ö�ùô�!#� ÿ�ù²ø øcd@ÃD�c Æ )¢×¢×¢×�)�c L [ #ë�(' �*)¢��& L )mô öf÷ ÿK÷�� ú�ù4!#�fómü�ô��(� ó2¼TS # Jâú��mü�öôKö�ù�ôd!#� ÿ�ù²ø ø �ò@¶�*)×¢×¢×�)VU

¼TS6D ! Ù [XW 1 8�Ù Å P ��!>cdÙ¸@¶�=�8�Ù ' P ��!>cdÙ¸@ ' ��×qN� ÿ@P 1 B�)dôKöm÷pW�U�R�MjXQT�U�R³R�P6FIHLKZJiS�HLÚtP6K�X6HjEGKbE�W%J U�R�X'O�U�V(P@Pu)¸f÷ ó��²ôK÷n!@ù�ôZYdD[P�[�)�� úaô öm÷�ú��(s�÷��!£ô öm÷Óø�ù�ÿ� ÷�úûô�úK÷ ôA��ö�� ü�öm� ú\P�� úKö�ù�ôûô ÷�ÿK÷^ ¤�]J2)���! ô ö�� ú]� ú5f�ó�� ôK÷�q¡ù²ó�^� ó.f�ó�� ô��i�²ô öf÷ ÿ���� úK÷�/9�öm÷�!@ù�ô­�×úûö�ù�ôKôK÷ ÿ�� ó� Ê�����÷�ómú���� óÑ� úp�múK÷6!#�mø�!#� ÿ«�mú5 d÷�üµù��fúK÷���! ô öm÷�!#�²ø ø(�'��� ó� �ôKöm÷� ÿû÷�g!"ÿ����D³�Wù²ÿûôKø ÷�ôûôgù²ó�8§_ö�ù¢�£÷6�Ë9Åù¢�Pø(� ÿ�qz�'�����¤[��?A@ j([]\¡j ê Ù�ï Ù ñ3P6R^J ó6P>U OQEGVLV(PQO6R�HjEGK E�W`F�P�U�V M�ÎGU�V öNP�S4W�öZK¸O6R�HjEGK�XiEGKî7 U�K�SÂV(P6RQC ó6P<US�H(X6R�F6H�ó�öZR�HjEGKmE¡Î�P6FX7�_ �(' �*)¢��&�×%ñzP6R � @ a ! Æ ) Î Æ eH)¢××¢×�)@a ! ËÍ) Î Ë eÐó6P�U�X'P|õ�öNP6K¸OQP�E�W�V�UVó6P6V(P�SP&J�U�Ú«Y�V(PQX8S�F�U�ù1KiWIF�EGÚ C>×Q`>HLRLTÂY�F�E�óQUVóHLV HLR�ô�U�R]V(P�UGX6RA� '�� E¡Î�P6FÊR#TNPnOQTNEGHjOQPÑE�W � ),H W8UO6V�UGXQX6H Ì�P6F,�3D ! [ � X6H�J�K�D�¼)D ! [�[Üù1HLRLT3¼ # JlT�UGXAÚ�U�F³J�HLKmU�RdV(P�UGX6RKP�RCBmEGKbP�Î�P6F�ô8P&J�U�Ú«Y�V(PHLK � )3R#TNP6K

$�ÿÓ � ¨ a�Ö ��� z��3D ! [ �@ Î {%= ©É ú ³ ø(��  =79ɳ ø��� ¥D i ©�É<[ Å ø���  =É� û )ù�TNP6F�P ³ @ W�U�RbY�D[P(?.� E [6×

*aúýó��²ô ÷��� ó�§_÷ ü ô��(�²ó � / � /���q�ô öf÷\f�ó�ù�ø_ö¤�Pþ���ô öm÷�ú�� ú�����ö�� ü�öÐr#�xw�� � ���>�����_ô þ��fôKú���� ø ø¤ d÷\��!ô öf÷�!#� ÿ�� �zD ! ["@¹ú��� ózD�¼�D ! [�[���� ôKö6¼)D ! [1@OW-/ ! !#� ÿÅú�����÷`W # a c ×*qN�mÿûôKöm÷ ÿ��_�²ÿK÷�q�ú�� ómü ÷�÷ ù²ü�ö� óV���_ü ù�ô���� ó_��! ÿ�� r� ÷�óm÷ ÿKù�ô ÷�ú�ù�ö¤�Pþ��²ô öm÷�ú�� ú���!NôKöm÷,!#�²ÿ�� � m D ! ["@�W m / ! ��� ô ö¥$�W m $ ¬ = Æì Ä ì ð ) 8� ó�=��'��ú�=��³» ú5� óm÷h���ù²ø�� ô��<�(��þmø�� ÷ ú�ôKö�ù�ôaôKöm÷��²÷ ü�ô�� ÿ W>���mú©ô¦ú ù�ô�� ú­!L��$�WU$ ¬ = Æì Ä ì ð ×�%{÷�ôKö��múü�²ómú���m÷ ÿ�ô öf÷�ü�ø�ù²úûú���!ò!#�fómü�ô��(� ómú

J¬@ W !Bc�ZW�/ ! �%$�WU$ ¬ = �$�7 $y¥ )�$ ! $y¥Ü= $�7 $y¥ d4× e ��f £hgikjmlon%prqhs"tvu�w's xQjyq{z�Y|e[P#g�)$z~}'n�s"�h�Ò��n�s3qhs$����qh���'��n@������nùɺ) �^}'n�u��~n��=� f �������Rn�x'�?q{z~n��������n���x'�^q�p�u��~��n�����u�s x'��s'��t�u�w s'xBu�sB���D���H�`z���e ! g �� �~����smeB+÷/ ! g�{�j�u��|uhw'�|}�qh��j��~��q{p�n\��nrqh��s'��s'��'��u�t'��n�� f �^}'n^j�u�����u�lC��s'�Cz�} n�u���n��+��lC}'��p�}�����qhs&n:��z�n�s'�~��u�s�uhjv�^}'n�uh�~n��8� f E j���u��8e6 |qh�Zz���n�z~zqhs'x¢¡�}�qrl£n:¤k�¥qr����u����m��¦�¦�¦�g:�'������n��Cz~}'n�x'n�������n�x+t�u�w s'x§uhsBjyqTz�Y|e�P$g:¨?A@�©#ªM«T© ê Ù�ï�¬òñ7­:®�¯ ó:­�°6ó:±¡ö�²v³�­~³B´�­�µh¶�±T²·¶[²¢¸^¹º°h²v³3»�­:®¥J ó:­&®[¼�­�½:»�°T¾�¾R±G¿À¿�ö�²�½:®y¶�±T² ¾±T²Q¯ ³�­�Á²�­~³�¶[²m-�<ÄÃ[Åw/Q°$Æ:±TÇh­�Ã�Èɼ�­:²�¿�°h® Y e�P#goÊÌËÄÍÏÎÑÐ A ¹ÒÑÓ ¨

��ÔhÕ

Page 117: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

�^}'n�u��~n��Ö� fØ× j�uh����u�lC�C������n�x'��q{z�n����"w ��u�s§p�u���t'��s'��s'��z�}'nO��s'n�Ù�w'qh����z���n��C�'�~u�Ú�n�x]��s+z�}'n�j�u���¤��u�lC��s'��zÑlou���n��R�Rqh� fÛ ©�ÜÝÜÝÞ Ù�ß�å�÷[¿�®�¼�­�¾�­:®£à�áÉâ�ãr¨r¨r¨�ã~áMä�åB¶�¾@P æ�¾�¼ °h®6®Ñ­:´�­~³"Æ�çOJ�®[¼�­:²è­:Çh­:´Vç�é � e6é â ãr¨r¨r¨�ã�é ä gëêà#ì���ãr�håÄä�¾�°h®y¶�¾�Á|­�¾ ííííí äî ïñð â é ï áï ííííí ¥�ò P$óÉô�¯õôy¥{¨

ö «Tª�ªv÷~ø ¡�w'�'�vu���n�z~}�q{z\à�áÉâ:ã�¨r¨r¨�ã�áMärå����oP{¤k�~}�q{z�z�n��~n�xBt$�QJùqh�|lC��z~s'n����~n�x+t$�Qz~}'n��~nrq{�És#w'��¤tvn��~�ûú â ãr¨r¨r¨�ã�ú ä ¨��^}'n�s]j�u��\n:Ú�n����üé � eyé â ãr¨r¨r¨�ã�é ä g�êõà#ì���ãr�hå ä ãvz�}'n��~n&���\qQÚ�n�p�z~u��ûý{S@êþ¸C¹lC��z~}ÿôVýTSrô ¬ Ê âì Ä ì ð �~w p�}þz~}�q{z�é ï eyýTS���á ï ì ú ï g ò P§j�u���� � ��ãr¨r¨r¨:ã�ó�¨¥¡�w'���R��s'�3z�}'n��~n3ó��s n�Ù#w�qh����z���n��^q{s'xB�~nrq{�~��q{s'����s �'� l£n�uht z�qh��sý{S���� äîïbð â é ï á

ï��ò P�ó� äî ïñð â é ï ú ï ¨ e6� f� g

�^}'n��~nOqh�~n\zÑl£uQp�qh�~n��ëz~u�p�u�s'����x'n�� f � qh�~n������^��j�� äïñð â é ï ú ï ò Ô�����j7z~}'���^���|z���w'nh�'l£nO}'q�Úhn�ô�¯õôy¥ �

ííííí äî ïñð â é ï áï ííííí ¥ ò ôVý{S�ô ¬ ííííí äî ïñð â é ï á

ï ííííí ¥ò ý{S���� äî ïbð â é ï á

ï��ò P$ó�¨

�@n��~nÂz�}'n�������zK��s'n�Ù#w�qh����zÑ�\���7t$�ûz~}'nÂx'n���s ��z���u�s�uhj�J§ãrz�} n£��n�p�uhs'x���s'n�Ù#w�qh����zÑ�%�������u���x'n����Ø���hq{s'xz�} n�z�}'���~x]���@j��~uh� ��s'n�Ù#w�qh����zÑ�Ýe�� f� g f �^}'���\����n���x �@z�} n&x'n������~n�x]��s'n�Ù#w�qh����zÑ�þô � äïñð â é ï á ï ôy¥ òP$óÉô�¯õôy¥{¨iÑsXz�}'nR�~n�p�u�s'xèp�qh�~n{� � äïñð â é ï ú ï�� Ô ¨�ikj^z�}'��������z�} n"prqh��nQz�} n�sX��n�z ( � e ( â ãr¨r¨�¨�ã ( ä g �eZì@é â ãr¨r¨r¨�ãrì@é ä gV¨��ºn�z�}'n�sX}�qrÚ�n�� äïñð â ( ï ú ï�� Ô ã���u � qh��n¢�������'����n��%z~}�q{z+ô�� äïñð â ( ï á ï ôy¥ òP$óÉô�¯õôy¥{ãMqhs'x3z�} n���n����Qq�j�uh����u�lC�|����s'p�nííííí äî ïñð â ( ï áï ííííí ¥ �

ííííí ì äîïñð â é ï áï ííííí ¥ �

ííííí äî ïñð â é ï áï ííííí ¥ ¨

Û ©�ÜÝÜÝÞ Ù�ß�ä! #"7±T´�°h²�ç�¾�­:®oà�áÉâVãr¨r¨�¨�ã~áMä�å%$�¶[®�¼·­~°�½�¼üô�á ï ôy¥&Ê ô�¯õôy¥�ã�¶ ¿ � ò'& ®[¼�­:²ü®�¼�­:´�­¶�¾û¾�±)(Q­ëé � e6é â ãr¨r¨�¨�ã�é ä goêÝà#ì���ãr�hå ä ¾+*�½�¼+®�¼ °h® ííí � äïñð â é ï á ï ííí ¥ Ê-, & óÀ��u��¥Õ/.0�'ô�¯õôy¥�¨��Ô21

Page 118: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

ö «Tª�ªv÷~ø �^}'nü�'�~u�uhj�w'��n��Bz�}'nü�'��u�t�qht'��������z���p§��n�z~}'u�x f �ºn·p�u�s'����x'n��Bz~}'n]��q{s'x'u�� Ú{qh����¤qht'��n43 � � äïbð â é ï á ï lC}'n���n e6é â ãr¨r¨r¨:ã�é ä g+���¢w's'��j�u��~����� x ����z~�~��t'w z�n�x)u�Ú�n��Ýà#ìO�hãr�håÄä�¨65�u��qhs$�õp�u�uh�~x'��s�q{z�n87 ê à#��ã�¨r¨r¨:ã9.Kåül£n }�qrÚ�n:3<; � � äïñð â é ï á ï; q{s'x }'n�s p�n>=�?@3<;<A � Ô ¨CBmn:zDE; �GF áÉâ; F Ë �H�I�I�I� F áMä; F Ë �J�ëu�n+K�x'��s'�L� �^t�uhw's'x+u�sB�~w'���Cuhj¥��s'x'n���n�s'x'n�s$z@��qhs x'u�� Ú{qh���?qht ��n�������'����n��Àz�}�qTz^j�u��ëqhs$��M � Ô�l£n�}�qrÚ�n���N? F 3<; F � MOA�Ê & nV� � � ìPM Ë& DE;

� ¨Q �£qûp�u�s �~n�Ù#w'n�s'p�n{� z�qSR���s'�TM � 1 & DE;���uh��Õ/.3lonë}�qrÚ�nëz�}�q{z£���I? F 3<; F ò MOA�Ê �NU & .¥¨!V@����s'��z�} nw's'��u�s¢tvu�w's x]qhp���u��~�C7 � ��ã & ã�¨r¨r¨:ã9.¥ãMlon�}�qrÚ�n�z~}�q{zëlC��z�}¢�'�~u�t'qht'������zÑ�"q{z\��nrqh�Zz��NU & n�Úhn��Z�p�u�u���x'��s�qTz�nT3<;%uhjW3���q{z~���X�'n�� F 3<; F � 1 & DY;v��u���Õ/.KãMqhs'xB}'n�s'p�n

ô+3�ôy¥ � òó ¹î; ð â F 3<; F ¥ ôõ â[Z¥

Ê òó ¹î; ð â]\ 1 & DY;v��u���Õ/._^I`bac â[Z `� 1 & ��u���Õ/.0��def df ¹î; ð âhg F á â; F Ë �i�I�I�j� F á ä; F ËXk ` Z Ë ac Ë Z ` amlc â[Z Ë e6� f �hg

¡���s'p�non òp& ã l£nO}�qrÚhnPnqU &�ò ��qhs x§}'n�s'p�nsr§��sLRhu�lC�tR#�[�Ø�ë��s'n�Ù#w�qh����zÑ�Q���R� ����n��£z~}�q{zdf ¹î; ð âhg F á â; F Ë �H�I�I�I� F á ä; F ËXk ` Z Ë ac Ë Z ` Ê uv ¹î; ð â F á â; F Ë ` Z Ëmwx Ë

Z ` �i�I�I�I�yuv ¹î; ð â F á ä; F Ë ` Z ËXwx ËZ `

� ô�á â ô Ë` �i�I�I�I� ô�á ä ô Ë`Ê óÉô�¯õô Ë` ¨ e6� f × g�^}'n���n��R�RqOj�u�����u�lC�|t$�Bp�u���t'��s'��s'����s'n�Ù#w�qh����z~��n��ûe�� f ��g|qhs'xÝe�� f × g fzC{}|~{}| ���~�E�L�X�6���O���p�m������� �]�L�6���� u���t ��s'��s'�R�^} n�u���n�� � f�� � � u���u�����qh���§� f� ��qhs'x]�^}'n�u��~n��R�û� f ��qhs'x·� fØ× �v��zëj�u�����u�lC�ëz~}�q{z\��ju�w'�Cq{����u�����z~}'� w �~n��@q&��qh���'��nûuhj¥������n F@�F{�p� ã'z�} n�sBlC��z�}B�'��u�t�qht ������zÑ�Rq{zë��nrq{��z��Cì���z�} n

��Ô

Page 119: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

}$����uhz~}'n�������BlC}'��p�}§���C��n�s'n���q{z~n�xBlC��������qTz����Zj[�����j��� ?���e�á�g��� �~����sme��8�rá�g A �i¡ � �� � ô¢��ô Ë£ ô�¯ ô Ë`� ˤN¥ ¦ ��u���.û��uh��Ë � � ��u�� � � ��� ¨

�^}#w'�Clon�}�qrÚ�n�n��Zz�qht'������}'n�x3z�} n�j�uh����u�lC��s'�Be�lC} n��~n�z�} n¨§¡ ¤Ñs uhz�q{z~��u�s3} ��x'n��ë��u���jyqhp:z�u�����g+©ªs«�©#ªM«T©�Ü ¬ ß}­®­8Èɼ�­�°h» µ�±T´:¶[®[¼E( ±hÆ�®�°h¶[²�­~³¢Æ�碰+¯2¯v»bçT¶[²�µ�°!±Y²2³L´!´EµN¶ ®Ñ±¸·Y¹E°º*$¾:¶[²�µ+®[¼�­6¯ °hæ´�°»(Q­:®G­:´ë¾�­:®6®y¶[²�µT¾O³�­�¾�½:´V¶yÆ:­~³�¶[²4¼£±T´�±T»[»�°h´Vç¾½ÄÃ}¿�¶�¾�°�¾:®y´�±T²�µ�»�­~°h´V²�¶[²�µQ°h» µ�±T´V¶[®[¼E( ¿r±T´o� ±T²�¯$�¶[®[¼3¾�°»(¯v»�­�½�±)(¯v»�­XÀ�¶[®yç � e}Á�ã<��ã9��ã~¯·g � §¡Ã âÄ �]Å ¤ Å ÓÆ Å ¦ Å ÓÇÈ ÓÉ¢Ê Ë Ì ¨Í�Î¢Ï ÐÒÑ6ÓbÔ�Õ�Öb×�ØÚÙ�ÛÚÖjÜÞÝyÖ�Õ�ÛßÕ�ÛàÑâáãØÚÓjÖbØàÑ-äæå®Øà×�ç�è éêÓbëì×�ç�Ö+Õ�ÛÚè Ù�^}Lí��tîh���'��íæïbð����'�ñí:����zÑ�%ð»ò�ð»ó'�ô®ðYðh��z��ñõ'�{¤öôJîh�tí�÷�n ¤öõLð���� � Q � ��íjî{�tõ'�ñõ'��îh���/ð�����z�} �=���ë�tíV¤��îh�tR)î»ô'��������������îh�mz9ðûz�}JîTz�ð»òMz�}Líë� Q � ¤�z��tî»õ'�Xò�ðh�~��íb÷ìð/õ'����õLíWn�¤OõLð����Ãî{���/ð�����z~}'�R�hð»ò�¡Yíbï:z��ñð/õ�Yø & ø & ø�V@�Rz9ð��ñð���ò}î»ï:z9ð����ùô®ðhz�}R�tí:z��oð»ò�ô®ð/óLõ�÷'�C÷Lí��_í�õL÷3���ñõLíjîh�����¸ð»õ0Á+ú�â î»õL÷QÙEóLî»÷'�9îTz���ïbîh�����ð/õ�ô¢��ô £ ô�¯õô ` U»� ¤I¥ ¦ ¨ � ð��R�Lîh�~�ñõ'�Xz�}Lí:ô®ð/óLõL÷'�B�ñõÿ��ð��tíÚ÷�í�z<î{�����Pû íº�Xíbíüz�}Jî{z¾ûC}'����íÚð/ó �ô®ð/óLõL÷'�~ïbð/õ$z9îh��õ+�tð���í���ð��2î{�~��z~}'����ï~ò}îSï�z9ðh�~�PûC}'��ï�}Úî{�tíTõLðhz@�'�tí��tí�õ#z\��õBz�}Líæô_ð»óLõL÷'�~ð»ò�z�}�í� Q � ¤�z��tî»õ'�Xò�ðh�~��íb÷àð/õ ����õ�íPn�¤OõLð����ãîh���/ðh�~��z~}'�����'z�}�ísð/õ'����õLíûÚSîh����î»õ$z~÷Lí��Xï��~�ñô_í�÷]��õ§�Jîh��zýü[î/þð»ò��^}�íbð��Xí�� �Yø & }Jî{�6î»õÿí:��z��tî�ò}î»ï:z9ð���ð»ò nRì ���ñõ§��z��Pô_ð»óLõL÷ ûC}'�ñï�}¢���~õLð{z%�'�Xí��Xíbõ$z\�ñõ§z�}�í�9î{�R�'�ñíýïbðh�R�'�ñí:����zÑ�ÿð»ò�ð/ó'�6îh���/ðh�~��z~}'�%ø���îh�~��î»õ$zìü[î/þð»K í��~�\z~}Líìî»÷ ÚSî»õ$z<î{�/íh��z~}Lð/ó'�h}É�Éz�}JîTzz�}�íæó �tí��P÷LðEí��PõLð{z~õ�íbíb÷Bz9ð�R!õLðIû z~}LíûÚ»î{��óLí���ðSò�î»õ$�3Ù!óJîSõ#z~��z~��í��^�XóLï�}àî{��ô�¯ ô ` ð���ô¢��ô £ �ñõî»÷ ÚSî»õ�ïbí&��õÚðh�t÷Lí��\z9ðB�tóLõ]z~}Líìîh���»ð��~��z�}'�%ø\��ó'�tõ'�ñõ'�3z9ð3�Jî{��z�ü}ôJþ~ð»ò£�^}Líbð��Xí��I�Eø & �_ûùí��tí�íz�}Lî{z��ñòÀz�}Lí&�Lîh�9îh��í�ztí����¢���û�tí:zsî{�'�'�tðh�'�~��î{z9í����§��õ]z�}Líìð/õ ����õ�í¸îh���»ð��~��z�}'�ùz�}�íbõüz�}Líìð/õ'����õLíô®ð/óLõL÷%÷'�ñK_í����Cò��tð�� ð/ó'�C� Q � îh���/ðh�~��z~}'�Gô_ð»óLõL÷%ð/õ'���8ô$�¾î»õ%í:��z��tîìò}î»ï:z9ð��oðSò

ü n�ì �Nþ df �Cì � ���N3��ô¢��ô £ ô+3 � ô `� Ë ac

ü[î{�2îh��õ&���/õLð�����õ'�û�ñð��6ò}î»ï:z9ð����<þ+ø��Wîh�Zz6ü�ïIþ�ð»ò��^}Líbðh�tí�� �Eø & �~}�ðNûC�Âz�}Lî{z îh��n�� � ������ò}îSï�9ðh�ô®íbïbð���í��"Ù!ó���tí�� ��îh���Pí:Ú/íbõHû��Lí�õp3�������ï��Lð��tíbõ�tð4ô_í�üZ��ãr¨�¨r¨�ãr�Iþ:¨����� �Lð»ó'��÷�ô®í õLð�9íb÷�����ð/ó�� �É�! �Jî�"��Líÿô_ð»óLõL÷#�ñõ%$Jî�&'Úü�ïIþìïjî»õ ô®íÿî�$�$�(���íb÷ ð/õ�(*)+�ñò6íbî»ï���ïbðEð &X÷���õJî,9í¾ð»ò-���í<î�& �/í.^Ú/íbï.9ð &��/���0$_ð�� �� ��Ú»íh��û�����(�í�ð/ó1&Pî�(��/ð�& �� ���2�Jî��PõLð3�tóLï��4&tí��5�&5��ï.���ð»õ%ð»õ%��ø��í î�(��tðPõLð�tí6 �Jî��û��Lí�õn �87 ü9(ñð ��.�þ;:�íbõ<���(�í�î»õ�÷sB=��5�(ñí>�5tð/õLí?�JîrÚ»í@� ��Ú/íbõ�î�(*9í>&XõJî� ��Ú/íí.AB$�&Xí>�5� ��ð»õ���ò�ð &;��LíCð»õ�(��ñõLí�n ¤OõLð�& � ô_ð»óLõL÷��C��õDtí>& �"��ð»ò�ô�¯õô.E î»õ�÷¢ô¢��ô â ¨»5Jð &;���í>�tí£Ú»î�(�óLí��F Ô<G

Page 120: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

ð»ò2nmãSó�� �ñõ��Tî»õ¸í�õH ��&Xí>(�)I�5���"��(�î�&hîSõJî�(�)J�5���C �LíCô®ð/óLõL÷1��ð»òqð/ó�&�î�(��/ð &5�� ���-ïjîSõ�ô®íoî»õLî�(�ð �»ð/ó�� (*)&tí�$���&tî��tí�÷/��õ�tí>& �"�oð»òëô�¯õô.E î»õL÷þô¢�Âô â î��oû í>(�( øzC{LKC{5M NPORQTS���UÒ�L�6� �ÿ���WV��CX~���J� �ê�i� � ���q�O�L�-Y¡B�ñõLïbíZ���í¾n\[ & ïjî��Xí ðSò] �Lí>ð/õ�(��ñõLí8n ¤öõLð &5� î�(��»ð & �*���� ����$1&tíbï����Xí>(�)%��Lí_^hí�&tï�í>$1 &tð/õî�(��/ð & �*����a`b �Líænc[ & ïjî��Xí8ð»òùð/ó�&æî�(��/ð & �*����Òïjî»õ ô_í�ÚJ��í+ûùí�÷4î���î%õLî�9ó�&tî�(�^ Q � ¤���ðE÷Lí>(î»õJî�(�ð �/ó�í�ðSò!��Lí�ð/õ�(���õLíd^hí�&tïbí�$1�&Xð/õºî�(��/ð & �*����%ø�� í8õ�ð�9í3��Jî,sû��Lí�õ ne[ & �Lí8ó1$�$_í�&ô®ð/óLõL÷f� ��Ú/í�õf��õ�B�í>�"��îZGYø F Ô ïjîSõ�ô_í��'�&Xíbõ��� �Líbõ�íb÷+9ð , ó��mô�¯õô Ë ü9�Xíbí¾B�í>�"��î F ø � ð»òü6 ùî�&5 (�í�5CîSõL÷B¡B�Jîjû í:¤�gWîh)J(�ð &�` F ¦h¦�¦2þ�ð &ig0��íbð &Xí>�8Õ�ø F ð»ò�ü Q (�ð/õü­:®À°h»�Ã�` F ¦�¦ & þ�ò�ð & îD$�&tðEð»ò9þ+øg0�����D��íjî»õ��I��Jî,D���í�ò}î�Z¤L� �Lî� tí>& �ñõ�� ÷1����íbõ��5��ð/õ ó�$�$®í>&sô®ð/óLõ�÷4ð»ò�g0�Lí�ð &tí��jGYø × ïbî»õ4ô®í���"$�&XðÄÚ»íb÷39ð âÒGÓ ã2û����ñï��k&tí���ð�Ú»í>� îI(ñð ��ò}î»ï�tð &�òl&tð �m �Lí�ô_ð/ó�õL÷8ð»òbg0�Líbð�&tí>�2GEø F F �B�LðIû í�Ú»í>&��1����ô_ð/ó�õL÷ÿû���(�(=�5 ��(�(]ïbð/õ<9î���õ+Ú»î�& ��ð»ó���(�ð �ìò}î»ï�tð & ��ô®íbïjîSó��tí�ð»òn �LíD(ñð �o9í>&5�d�p��õ4g0�Líbð�&tí>�GYøqGYøzC{LKC{Lr NPOjs S���UÒ�J�6�ut0S?Vwv�x�� �pyHz��WS;{ ��� �i� � ���q�O�L�-YQ � �Lísð� �Lí>&í.AJ�&Xí>��í�` û íýïjîSõ>î�(��Xð8÷Lí+�JõLí¸î�õJî,9ó�&tî�(®n/[| Ú/í>&5� �ñð/õÚð»ò�ð/ó�&î�(��»ð & �*����%ø� ð/õ��5��÷Lí�&] �LíRÚ»íbï�tð & �ý3Úî»õL÷c}Gû�����ï��4î�&tí�ïbð �"$Ló1tíb÷�ô<)_��Lí�ûùíbî»R_(�íjî�&tõ��ñõ�� î�(��/ð &5�� ���·Y¹!°®ø~�öò�û íD(ñí�ëú¸ô_í����ísõ!ó��ýô®í>&�ð»ò�ï�ðYð�&t÷��ñõJî�9í��3 ï ð»ò�3"�tóLï��� �Jî� F 3 ï F [8ô+3�ô.E�ãW���íbõÿò�ð�&î»õ<)0��û í]�JîrÚ»í

(����`.� E � } ïô�}�ô £ � [ (����`.� E � � ���/õ�ü}3 ï þ F 3 ï F ` ú�âü � ¹ïñð â F 3 ï F�� ` ú�â�� £ þ â[Z £�

[ ��� ��� ���/õ�ü[3 ï þtU{ú � ò F 3 ï F [ ô+3�ô.EÔ ð���Lí�&Xû����Xí»ø

�~í�õLïbí0�*����WõJî�tó�&9î�(Jtð�ïbð/õ��5��÷Lí�&�î n�[8�Ú»í>& �5��ð/õ¸ðSòq·Y¹!°w`Sû����ñï��ýûùío÷�íbõLð�tí ·Y¹!°H��`<��õ�û��1��ï�����íOÚ/í�ï�tð &�}�����÷Lí��Jõ�íb÷Úô<)49î»RH��õ��d} ï [m� ���/õ�ü}3 ï þp� ò F 3 ï F [=ô+3�ô.E î»õL÷/} ï [�Ô�ð����í>&Xû����tíSøQ (�(Jð»ò_ð/ó�&�î»õJî�(*)J� ����ï�ð/õ<���õ!óLí���tð]��ð (�÷ìò�ð &?��LíC·Y¹!°<�Jî�(��/ð & �*���� ü�û��*��"�d�ñõLð &���ðE÷��ñ�Jïbî���ñð/õ��î����tRSí�tï��Líb÷àô®í>(ñðIû~þPîSõL÷0ûùíæð/ô19î���õ¾î�nk[� �'�&Xð/õ��"(�íjî�&tõ��ñõ��¸î�(��/ð & �*����%©�D� Þ��6Ü ¬ ß}­J� Èɼ�­�±T´�­+( ½ÄÃ��<��¼�±T»�³T¾�¿r±T´�n�[8 $�¶[®[¼¸·�¹!°<��¶[²8¯v»�°�½�­�±G¿C·�¹!°�Ã

F Ô ×

Page 121: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

ö «Tª�ªv÷~ø g0�Lí!$�&XðYð»òqðSòWg0�Líbð�&tí>�mGYø � ü�û��*���·Y¹!° � ��õ]$�(�îSïbíùðSò�·Y¹E°YþC���hó�õLï��Jî»õ��»íb÷ìó�$D ��&tð/ó1� ����íI$®ð �ñõHoû���í>&tíTû íI�só��5p� ��ðNû8��Jî����ïñð âJ� ü�á ï þ'� ï ü9} �rá ï þô�}Rô â ò � ¤I¥ ¦ô¢��ô â ¨

g0�LíI(�í+ò��¤L�Lî»õL÷�� �ñ÷LíTð»ò�������0��õLí�ÙEóLî�(��*L)�ïjîSõ%ô®í�&tí+û�& ��59í�õ î��} �N3ô�}Rô â [ �Z� ���l� ð Å � Å9� � ���»õ�ü[3 ï þX3 ï� � ���l� ð Å � Å � F

[ � � ���l� ð Å � Å � ô+3�ô.E� � ���l� ð Å � Å � F[ ô+3�ô.E�ãî»õL÷4�Líbõ�ïbí��*��Xó1 �ïbí��¡9ðd$�&tð�Ú/í]��Jî��ô+3�ô.E ò � ¤I¥ ¦ UMô¢�Âô â�¢ g0�1���p����í>�59î»ô�(���� ��íb÷àî,� �Lí�í�õL÷ð»ò���Lí�$�&XðYðSòWð»òng0�Líbð�&tí>�£GEø � `1�tð"g0�Líbð &Xí>�£GYø � �Lð (ñ÷��oû�����¸n�[� î»õL÷0·�¹!° � �tóLô��'��*9ó1tíb÷ò�ð &ù·Y¹!°qøg0�Lí�&tí��5�ð»ò;��LíTîSõJî�(�)J�5���!�/ðEí>�! ��&tð»ó�� �¾óLõLï��JîSõ��/íb÷%í.AYï�í>$1Pò�ð &0�ñõLí�Ù!óJî�(��� ��í>�6ü�GYøqG/þùîSõL÷ü�GEø × þ8ðSò¸B�í>�"�8î+GEø F ÔYø¥¤J��õLï�í ô�¯ ô.E [ �Xó�$ �j�b¦ �8î,A»; ð â ¥§¦§¦§¦ ¥ ¹ F áE; F ãCû í¨�JîrÚ»í_ �Jî�ÿDY;õÊóÉô�¯õô Ë E ò�ð &Pî�(�(J7�ãqî»õL÷a�Líbõ�ïbíI��õ�$�(�î»ïbíTð»òC��õLí�Ù!óJî�(��*��ñí>�Tü�GYø©G»þCî»õL÷ ü�GYø × þ û í]�Lî�Ú»í

ô+3�ô.E [ ��î,A; F 3<; FÊ ��î,A; ª & DE;~(ñð �ÂÕ/.Ê ª & ói(ñð ��Õ/.0�'ô�¯õô.EOãû����ñï��4$�&Xð�Ú/í>�p(�í��d��î3GYø F Ô�øg0�Lí>&Xíi����îoï>(ñð �tí?&Xí>(�î,���ð»õ�� ����$6ô®í�Oû íbíbõ]�������n�[8 î�(��/ð &5�� ���iî»õL÷-��Lí�û ð &tRð»òW«/îSï<RJ�Xð/õî»õL÷ � &9îrÚ»íbõ4ð/õ_(�íbî�&tõ1��õ���� $Lî�& �Xíd$®í>&Xïbí>$B�&tð»õ��0ü�«/î»ï<RH�tð/õ�î»õL÷ � &9îrÚ/í�õ¬` F®­ ­ þ+ø"¯~ð�9ío��Jî,�ñò£ú�[ F ã=� ø�í»øìð/õ�(*)Úð/õLíìï�ðYð &X÷���õLî�9íìð»òù34�Jî�� F 3 ï F [ ô+3�ô.E&ã¬��Lí�õ_ �Líì·Y¹!° � �<)J$®ð���Lí�� ���]�����ü�á_þ�[ °Å ¦ Å9� û��Lí>&XíI±����!��LíI� ���/õLíb÷BÚSî�& ��î»ô�(ñíòl&tð � à�á â ã ¢h¢h¢ ã~á ¹ ã�ìëá â ã ¢>¢h¢ ã�ìëá ¹ åsû����ñï��4�����ð �5I�5 &tð/õ1� (�)Úï�ð & &Xí>(�î�9í�÷ óLõ�÷Lí>&�÷����'�& �ñôLó1 ��ð/õ � û�����_��Lí�ÚSî�(�ó�í�ðSò!� ���/õ�ü��%�{á_þ ¢ g0���������Ú/í�&5)"� ���d��(�î�&�9ð���LíPû íjî»R3(�íbî�&tõ1��õ��Tî�(��/ð &5����1�Ãó��Xíb÷8ô<)d«/î»ï<RH�Xð/õ0î»õL÷ � &9îrÚ/í�õ¬`Yû��1��ï��d9î»R»í�����íI�5��õ���(�í�ô®í>�5'²Oï�ð & &Xí>(�î,9íb÷�(���9í�&9î�( î����� �0�H)J$®ð����í>� ����ü}ô�&Xíjî»RH��õ1�" ��í>�Pî�&tô��*�&9î�& ��(*)�þ+øg0�Lío$�&XðYð»ò6 �Jî�� ������³Xô_í��5'²L� �ñõ�� (ñí.²L(���tí>&9î�(�´�î�(��»ð & �*���� ó��Xíb÷µ��õ�ü�«/î»ï<RH�tð»õ�î»õ�÷ � &9îrÚ/í�õ¬`F®­ ­ þ�����îTû íjîSR"(ñíjî�&Xõ���õ1��î�(��/ð &5�� ���|����÷LóLí�9ðD:Tð (ñ÷���î»õLõ¬`!�~î��59î»÷�î»õ�÷k¶�î�·bô®ð &XðÄÚÿü�:Tð�(�÷B²

F Ô ­

Page 122: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

��î»õLõ�¸.¹�°h»�Ã�` F®­�­ & þ+øW�~ðIû í�Ú/í�&>`J��Lí���&?$�&XðYð»ò î��5�tó���í>�?��Lî�6��LíPí.A�î��"$�(ñí0� $Jî»ï�í믣���|àTÔ ã F å�¹î»õL÷o �Jî�?���í�9î�& �»í��Ú»íbï�tð &��k�Jî���î�(�(B��õ<9í��/í>&�ï�ðYí. �ï���í�õH �b�H��!ó���`�î���õ�ð�9í�÷�ô<)"«/îSï<RJ�Xð/õ�îSõL÷� &9îrÚ»íbõ¬`J��Lí���&�î�(��»ð & �*���� ò�ð &6(�íbî�&tõ��ñõ���� $Jî�& �tí�$_í�&tï�í>$1 &tð/õ�� ð/õ�(*)ìî�$�$�(���í���9ðD(ñíjî�&Xõ���õ1�-$�&tð/ôJ²(�í��d��û����ñï��ýî�&Xíù÷Lí+�JõLíb÷¸ð�Ú»í>&�º ðYð (ñíjî»õI��õ1$Ló1h÷Lð ��î���õ���ø;�Oõýïbð/õ< &9î��'h`/ð/ó1&®n�[8ßî�(��/ð &5�� ���ïjîSõ:ô®í¸î�$1$�(��ñíb÷Úð/õÚï�ð/õ<���õ!óLð»ó��-��õ�$�ó1�÷Lð ��î��ñõ��p»a���í¸ð/õ�(�)4&tí��5 & ��ï.��ñð/õ���&Xí�Ù!ó���&Xíb÷>ô<)àð/ó1&î�(��/ð & �*���� î�&tí���Lî�� �Lísí�A�î��d$1(�í�� $JîSïbí]¼ î»õL÷a��LíI<î�&5�/í�ëÚ/í�ï�9ð�&�Z�9î,����mò�)èô�¼õô.E � î»õL÷ÿ� ¤I¥ ¦ � Ô ¢��í�î�(��tð�ð/ô1�tí>&ZÚ/í���Jî,6g0�Líbð &Xí>�2GYø F F í>�'<î»ô1(����5�Lí>��î� ��� �<9í�&?�9î��"$�(ñíCïbð �"$�(ñí.AB��L)sô_ð/ó�õL÷ò�ð &oð/ó1&�n�[� �5�&Xð/õ��"(�íbî�&tõ��ñõ��ìî�(��/ð & �*����½��Lî»õ%ûùî��p� ��Ú»íbõ%ô<)a«/î»ï<RH�tð/õàî»õL÷ � &9îrÚ/í�õ]ø@g�ð�tí�í" ������`=(�í.-¼ [ùàTÔ ã F å ¹ î»õ�÷¨�Xó�$�$®ð �tí3��Jî,-��Lío<î�& �/í.ûÚ/íbï.9ð &æ� ê ¸ ¹ �Jî��æî�(�(n��õ<tí>�/í�&ïbðEí� �ï���íbõ< �C�Xðp �Líi«/î»ï<RH�tð/õB² � &9îrÚ/í�õìî�(��»ð & �*����HïjîSõ�ô®í�î�$�$�(���íb÷ øW5Jð &¬ �����=(ñíjî�&Xõ���õ��0$1&tð/ô�(ñí>�û í��JîrÚ/í�� ¤I¥ ¦ [ 7 ü F þ î»õL÷·ô�¼õô.EP[ F �H(�í. ��ñõ��D¾�[�ô¢��ô â ãJg0�Líbð�&tí>�¿GYø F F ���"$�(��ñí>�?��Jî,ùð/ó1&nÀ[¥ �5 &tð/õ��a(�íbî�&tõ1��õ��¾î�(��/ð &5����1�Á�Jî��I�tî��"$�(�íìïbð��d$�(ñí.AB��L)/&tð/ó1� ��(�)Z¾ Ë U»Á8ül���/õ�ð & �ñõ���(�ð��ò}î»ï.9ð &5�<þ+ø?g0�1�������oîo�XóLô��59î»õ<���î�(=���"$�&tð�Ú»í>��íbõ<oð�Ú/í�&p �Lí-&tð/ó1� ��(�)�¾>Â+U»Á0�9î��"$�(ñí�ïbð �"$�(�í�AB��L)ô®ð/óLõL÷Z� ��Ú/í�õàô<)µ«/î»ï<RH�tð»õ îSõL÷ � &9îrÚ/í�õ]ø�rÿð &Xí��/íbõ�í>&9î�(�(�)�`~ �Lí��tî��d$1(�ísï�ð �"$�(�í�A1�*L)%ô_ð/ó�õL÷� ��Ú/íbõ%ô<)4«/î»ï<RH�tð/õàî»õL÷ � &9îrÚ/í�õ ò�ð &0(�íbî�&tõ1��õ��¨³�¾�²L�5$Jî�&5�tí]ÃH²L$®í>&Xïbí�$1�&Xð/õ�� ´d����&tð»ó�� ��(*)�Ã�¾  U»Áb����íýî»õJî�(�)J� ���~ð»òn������ï��Jî�$B9í>&~ïjî»õ íjî��5��(�)¾ô®ísí�AJ9íbõ�÷Líb÷Z9ð�í>�'<î»ô�(����5�àîk�tî��d$1(�íæïbð �"$�(�í�AB��L)ô®ð/óLõL÷%ð»òC&tð/ó�����(�)�Ã�¾ Ë U»Áoò�ð &0(�íbî�&tõ��ñõ��"¾�²L�5$Jî�& �Xí]ÃH²L$®í>&Xïbí>$B�&tð»õ��bø

F F Ô

Page 123: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

Ä Å Æ¥Ç È¨É�Ê Ë

Ì ÍÎÍÐÏ ÈfÑ?Ò Ó ÆÔÒ Õ Ö Æ¿Ê+Õj×4Ä Í ÊcÉ ØÙÉ8ÈÄ Í Ò Ï ÈÀÊeÚ Û%ÈfÑ Í Ò

�Oõ4 �Lí�(�î��'�ï��Jî�$1tí>&PûùíI�9îjûu�LðIû'°!±Y²/³L´E´EµN¶%ïjî»õÿô_íæî�$1$�(��ñíb÷�9ð�ð/ô19î���õa^ Q � (�íjî�&tõ��ñõ���î�(*²�/ð &5�� ���d�Cû��1��ï��àî�&tíI&tí>��î�&XRSîSô�(�)��5���"��(�î�&itð�^hí�&tï�í>$1 &tð/õÚîSõL÷ÿ�Ü�ñõLõLðIûTøù� í�õLðIû í�A1$1(�ð &Xíî0÷��ñK_í>&Xíbõ<æî�$�$1(���ïbî���ñð/õ ðSò ô®ðEð �5 ��õ���`Ý������¡�����í��ñõ���Líìî�&tíjî�ðSòùï�ð �d$�ó1<î� ��ð/õLî�(hï�ð �"$�(�í�AJ²��L)4��Lí�ð &5)»ø�� í3� �LðIû|��Jî��î0ï>(ñð �tíìï�ð/õLõLí�ï���ñð/õ:í.AB���'��Tô_í.Oûùí�íbõ4ô®ðEð �5 ��õ��¾î»õ�÷+Þ1ß�à5áhæ â�ã,à�¸ä ¸.¹]â�ã,å ä ¹9à¢*wâ.¹9æ�ã,åW`�î�L)B$®í�ð»ò!�Jî�&t÷�õLí>�5�ìî��d$1(��ñ�Lïjî� ��ð/õàòl&Xð � ïbð �"$�(�í�AB��L)Z���íbð &')/ø�º�) í��3²$�(ñðç)J�ñõ���ô®ðYð �'��ñõ��k9í�ï��Lõ���èEó�í>�6òl&tð��Á(ñíjî�&Xõ���õ��d��Lí�ð &5)�` ûùí¸ó1�tí���1���6ï�ð/õLõLí�ï���ñð/õ�9ð�����é/íýî»õ���"$�&Xðçé»íb÷:ô_ð»óLõL÷>ò�ð &-�Jî�&X÷B²Oï�ð &tí3�tí.sï�ð/õ��'�&tó�ï���ñð/õ>û�����ï��_�8î,9ï��Lí���R!õLðIûPõÀ(ñðIûùí�&æô®ð/óLõ�÷��òl&tð���ô®ðYð��5��ñõ��ìî»õL÷� �Eó1�0���oð $1 ����î�(Jû��� ����õ� �����Cï�(�î��5�oð»ò;9í�ï��Lõ���è!óLí��bø���íTî�(��tð��5�LðIû��LðIû9ðýî�$�$�(*)k�ñ÷Líjî���òl&tð �½�Jî�&t÷B²öïbð &Xí��tí.oïbð/õ1�5�&XóLï� ��ð/õ�tð�� �*é/í�îsõ�í�ûTé/í>&5� �ñð/õ0ð»òC«/î»ï<RH�tð»õ]���Pï�í>(*²íbô1&9î�tíb÷Úô®ðYð �'��ñõ���²OôLî��tí�÷Ú�î�&5��ð»õ���ïD¤B��í.é/íìî�(��/ð &5����1� ò�ð &¡(ñíjî�&Xõ���õ1��ê-¯5 ò�ð & �ýó1(�î»íæóLõL÷Lí�&���íÿóLõ1�ñò�ð &5�ß÷����5�&5��ôLóB���ð»õ�ó�� �ñõ��¨��í>�sô_í�& � �1��$+è!óLí>&5��í>��ø�ë6ó�&ìõLí+ûÔé/í�& � �ñð/õ ðSò¡���í/¤J��í�é»í�Jî��ùî]�5���/õ�� �JïjîSõHùî��')B�"$1tð���ñï����d$�&Xð®é/í>��íbõ<@��õ"&tóLõLõ1��õ��D ����í»ø � & �� ��ïbî�(�9ð�ð/ó�&Cî�& �/ó���íbõ< ����ýîàïjî�&Xí�ò�ó�(Pî»õLî�(�)J� ���sð»òp���í0÷����'�& �ñôLó1 ��ð/õ��æû�����ï�� î�&Xí0í>�"$�(�ð®)»íb÷f��õ�ô®ð� � ô®ðEð �5 ��õ��ÚîSõL÷�Jî�&X÷B²Oï�ð &tí��Xí��ïbð/õ1�5�&XóLï� ��ð/õ1�bøì ήí î�Ø�Õ�ç�×�ïZðZñoÕ�Ö�×�Øò {5M�{5M óº����xE�L�m�6�¿S���UõôuS��WU�yHz�� �J�÷ö��]��x� í0&tí�ò�í�&?9ðTîDÞ1ß�à5á�åb¸ ä�ä ß»(¯~øùæ ú!â ß�¹9æ�ã,å�î���î�&tí>�Xó�(��ðSòÝ �LíCò�ð (�(ñðIû���õ���ò�ð & �%©C����é/í�õ�î-ºùðYð�(�íjîSõò�óLõLï.��ñð/õ3û" �Jî�����C�"��(ñ÷�(�)���õJî�$1$�&tðhAB����î»ô�(ñí�ô<)�ï>��&tïbó1����hð»òW�Xð ��í ô_ð»óLõL÷Lí�÷o� ��·�í@ü~`»ï�ð/õ��5 &tóLï.

F F F

Page 124: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

¶�í+ò�í>&tí�õLïbíS© ¤Eí�p� ��·bí�$Jî�&tî���í.9í�&b© � ��&tï�ó���!� ��·�í�$Jî�&9î���í.9í>&�©�L�d$Jî�� (���î�·>·�ð%ü9�L�"$Jî�� (���î�·�·bð�` F®­ ­ 1»þ Á ¡ ülý~þtÁ þ+þW�hü¯����tî»õ�ü9�L�"$Jî�� (���î�·�·bð�` F®­ ­ 1»þ Á ¡ ülý þ ü9(�ð��qü F Uçý Á9þtþ úWÿ þW�hüë�ó1&�¶Pí>�Xó�(� � Á<U & ¡ ülý þ ü9(�ð��qü F USÁ<þtþ úWÿ þW�hüghî»ô�(ñí���ø F © � ð �d$Lî�& ���tð/õ0ð»òhR!õLðIûPõ/�Jî�&t÷B²öïbð &XíI�Xí��ïbð»õ��5 &tóLï.���ð»õ��bø

òl&tð��£û:î�õLí�û-ò�ó�õLï� ��ð/õ�û~�w��Jî����������� ��(*)���õJî�$�$�&tðhAB����î»ô�(ñí6ô<)ÿî�(�(�ï���&tï�ó��� �Pð»ò��5��·bíæï>(ñð �tí�(�)&tí�(�î�tíb÷�9ð3ü®øC�~í�&tí/³5�"��(�÷1(�)"��õJî�$�$�&tðhAB����î»ô�(ñí>´o��íjî»õ1��&Xð/ó�� ��(*)���Lî��õLð8ï���&tï�ó���Pïbî»õàî�� &tí�íû��� �Zû>ð/õàîìòl&9î»ï.���ð»õÿð»òn��õ�$�ó1��pé/í>&')%ï>(�ð��tí�9ð F�� û��1��(�í/³5����� ��(*)���õLî�$�$�&Xð®AB���8î»ô1(�í>´3��íjî»õ1���Lî��õLð ï���&Xïbó��*ìïjî»õ î�� &Xíbí¾û����� û�ð/õ î>òl&tî»ï� ��ð/õºð»ò¡��õ�$LóB��3� ���/õ��ñ�Lïjî»õ<�(*)c� &Xíjî�tí>&o��Lî»õF U & ¢ �î�&X÷LõLí>�5��î��"$�(��ñ�Jïbî���ñð/õc&tí��tó�(*���î�&tíÿî>ï>&tó�ï>��î�(oïbð �"$_ð»õLíbõ<�ð»ò�&Xíbïbí�õ<0î�59í>�"$1 �3tð÷Lí�&9î»õL÷�ð �d��·bí]º!^6^ ü�ºCî»ôLî��!¸.¹�ß�ø���` F®­ ­ � �W�L�"$Jî���(���î�·>·bð�î»õ�÷ �Ü��÷��»í>& �Xð/õ¬` F®­ ­ GY�~¯����tî»õàîSõL÷�Ü���»÷Lí>&5�tð/õ¬` F®­ ­�� þ+øa^hí�& �Jî�$1�D��Lí"��ð �5æò}î���ð/ó��I�Jî�&t÷�õLí>�5�¸î��"$�(�� �Jïjî� ��ð/õZ&Xí>�Xó�(�D����� î»ðL� ��ë-¶0²�(�í��d��î�ü�:�ð (ñ÷�&tí���ï��8¸.¹]ß�ø���` Fh­ ­ 1/þ�`�û��1��ï��e�5<î,9í>�o �Jî�"�ñò~ùðEð (�íbî»õ4ò�óLõLï.���ð»õ%û#����"��(�÷1(�)f�ñõJî�$�$�&XðhA1���8îSô�(�í%ô<) ï>��&tï�ó��� ��ð»ò�� ��·bí�ü+���íbõ���Lí�ë�¶ãð»òI�tí.é/í>&tî�(-��õ�÷Lí>$®íbõL÷�íbõ<ïbð�$���í���ðSò?û����0�1��� ��(*)��ñõJî�$�$�&XðhA1���8îSô�(�í�ò�ð &oï>��&tïbó1����oðSòn�5��·bíTï�(�ð �Xí>(�)k&tí�(�î�tíb÷�9ðoü®ø¤Yó�$®í>&m�Jï>��î�(�(*) `�ô_ðEð �'���õ1�>î»õL÷e�Jî�&t÷�õLí>�5�8î��"$�(��ñ�Jïbî���ñð/õc�tí�í>�jtð_�Lî®é»íÿð�$�$_ð�� ��tí��/ð2î�(��bø¶Píbïjî�(�(Ý �Jî��î¸ô®ðYð �'��ñõ��¸î�(��/ð & �*����÷9î»R»í��î��0�ñõ�$Ló1oî¾$@¸ ß��oø�¸ ß�à.åWæ�å� dß�ø� Jã,à�æ�¹lÞE('û����ñï��ÿïbî»õ�/í�õLí>&tî�9íI�<)B$®ð� �Lí>�Xí>�û����ñï��/�Jîhé»íýî»ï�ïbó�&tî»ï�)%ð/õ�(*)�� (���� �< (�)�ô_í. 9í�&p �Jî»õ F U & � î»õ�÷ÿð/óB�$Ló1 �î��LõJî�(;�<)B$®ð� �Lí>�5���~û��1��ï��/���î"��������(�)�î»ïbï�ó�&9î�tíD$�&Xíb÷��ñï�tð &�ò�ð�&� �Lí]9î�& �»í�Pò�óLõLï.���ð»õ]ø!g0�!ó��ô®ðYð �'��ñõ���ïbð/õ1�5�&XóLï� �ùî-�<)J$_ð���Lí�� ����û����ñï���ï�(�ð �Xí>(�)ýî�$�$�&XðhA1���8î,9í>��î�ò�óLõLï.��ñð/õdû�û��1��(�í@�Jî�&t÷J²õLí�� �~î��d$1(��ñ�Lïjî� ��ð/õd&tí>�Xó�(� ��$�&tð®é/í���Lî�Pïbí�&5<î���õ0ò�óLõLï.���ð»õ��Pî�&tí��Jî�&t÷�9ð�î�$�$1&tðhAB����î�9íSø?g0�Lí$�&XðYð»òJ9í�ï��Lõ���èEó�í>�Wí��d$1(�ð®)/í�÷D��õTô®ð����î�&Xíjî���`��LðIû í�é»í>&>`<�Lî®é»íùî�� ���d��(�î�&Ý�5 &tóLï.9ó�&Xí»ø Q (�(2R!õLðIûPõ�Jî�&X÷LõLí�� �~î��d$1(��ñ�Lïjî� ��ð/õ�&Xí>�Xó�(� �p�»ð�ô<)¾ï�ð/õ<�&9îS÷���ï.���ð»õ]©�î��5�tó��"��õ��o �Lí>&Xí�í�A1���5 �~î¸ï���&tï�ó�����ïjî�$Jî»ô�(ñí ð»òw�"��(ñ÷�(�)î�$�$1&tðhAB����î���ñõ���ûw��`»ð»õLí@$�&Xð®é/í>�� �Líùí�AB���5tíbõLï�íCð»òqîp� (���� �<�(*)�(�î�&5�/í>&�ï���&tï�ó���û����ñï��ÿï�(�ð �Xí>(�)0î�$�$�&Xð®AB���8î�tí>�pûhø�5W&tð��¿��1����$_í�& �5$_í�ï���*é/í�`_î3�Jî�&X÷LõLí�� �~î��d$1(��ñ�Lïjî� ��ð/õ�$1&tðEð»ò&tí��tí��ýô�(�í���î�L)J$®í�ðSò ô®ðYð��5��ñõ���$�&tðEïbí�÷Ló�&Xí»©�ï���&tï�ó��� ��û�����ï��"�"��(�÷1(�)æî�$�$1&tðhAB����î�9íPî6ò�óLõLï.��ñð/õû � ül �Lí>�Xí�ï�ð & &Xí>�5$_ð/õ�÷atð3 �Lí��<)B$®ð� �Lí>�Xí>�6ð/óB�$Ló1Pô<)� �Lí�ûùíbî»R�(�íbî�&tõLí�&<þCî�&XíTïbð �ýô1��õLí�÷�tðò�ð &5�âîìõLí�û¨ï>��&Xïbó��*Pïbð �"$Ló1 ��õ��"ûàð/õÿîo(�î�&5�/íòl&tî»ï� ��ð/õ¾ð»òC��õ�$�ó1���ø�Oõ�îSõ¨���d$®ð &59î»õ<�$Jî�$_í�&>`;�L�d$Lî�� (���î�·>·�ð:ü9�L�"$Jî�� (���î�·�·bð�` F®­ ­ 1»þ¡&tíb÷�óLïbí��I �Lí3$�&tð/ô1(�í>�Òð»òî��"$�(�� ò�)J��õ��-���í¡�Jî�&X÷LõLí�� �Cð»ò�îTò�óLõLï.���ð»õ�û�tðD��Lí�$�&tð»ô�(�í�� ðSò�ïbð/õ1�5�&XóLï� ��õ��ýîs÷����'�&5��ôLó1 ��ð/õ� �tóLï�����Lî�pû/���@����� �1(�)"��õJî�$�$�&tðhAB����î»ô�(ñí~ô<)k� ��î�(�(_ï>��&Xïbó��*���ò�ð &!��õ1$Ló1��oï���ð �tí�õ î»ï�ïbð &X÷���õ1�9ð � ¢ �~í3��Lí�õ4ïbð/õ1�5�&XóLï� ���tó�ï���îÿ÷����5 & ��ô�ó1��ñð/õ�îSõL÷�ó��Xí>�]��I9ða$�&tð®é/í�î»õ��ë-¶÷(�í��d��î�ø

F F &

Page 125: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

�L�"$Jî�� (���î�·�·bðæî�(��XðD�5�LðIû��@��Jî,@���íí.AB���59í�õLïbíð»ò=�tó�ï��¾î�÷����5�&5��ôLóB���ð»õd���d$�(���í�����Lí~í.AB���'9íbõ�ïbíð»òqî�³5�Jî�&X÷B²Oï�ð &tíi�tí� ´�î��h÷Lí��Jõ�íb÷3��õ�¤Yíbï.���ð»õ���ø & �»û í@��!ó���&Xí�ò�í�&C9ð¡�L�d$Jî�� (���î�·>·�ðL���;��í� �LðY÷�ð»òïbð»õ��5 &tóLï.���õ1�k�tó�ï��>î�÷����5�&5��ôLóB���ð»õÿî��î�Þ1ß�à á�� â�ã,à�¸ ä ¸.¹pâ�ã,å ä ¹9à¢*wâ.¹9æ�ã,åJøp¤Yï��Jî�$���&Xí0ü�¤Eï��Jî�$���&tí�`F®­ ­�� þhûCî��i���íP��& �5�9ðI$®ð ��õ<�ð/ó16��Lî�6��Lí�í.AB���'9í�õLïbí~ð»ò]îæô_ðEð �'���õ1�Tî�(��/ð�& �� ���m���d$1(���í��n���íí.AB���59í�õLïbíæð»òC�tóLï�� î�÷1���5 & �ñôLó1��ñð/õ]øò {5M�{Lr � �6�����;xL���ö��x� ío� ��é»íìî»õ>í.AB$�(��ñï>�*�ïbð &5&tí�� $®ð/õL÷Lí�õLïbí�ô_í.Oûùí�íbõ_��Lí¸÷����5 & ��ô�ó1��ñð/õ��¡ �Jî�Tî�&5���Xío��õZ�L�"$Jî�� (��*²î�·�·bðL� ���Jî�&X÷B²Oï�ð &tík�tí�ýïbð/õ��'�&XóLï� ��ð/õºî»õ�÷À��Lí8÷����'�& �ñôLó1 ��ð/õ��æïbð»õ��5 &tóLï.9íb÷ºô<)4ô®ðEð �5 ��õ�� î�(*²�/ð &5�� ���d��øÀg0�����¸ð/ô��Xí>&5éSî� ��ð/õ î�(�(�ðIû��ýó��otð_$1&tð®é/í���Lî�o��Lí��Jî�&t÷B²öïbð &Xí��tí��ï�ð/õ��5 &tóLï.��ñð/õð»ò@�L�d$Jî�� (���î�·>·�ð�æ ä î¾ô_ðEð �'���õ1�¾î�(��/ð�& �� ��� û��Líbõ_��Lí"��õ��*���î�(h÷1���5 & �ñôLó1��ñð/õZ����ó�õ��ñò�ð &5�ÿø Q �û ísû���(�(=� �LðIû�`Ý �Lí>&Xíýî�&Xí]Oû ð����"$®ð &59î»õ<�$Lî�&9î���í�tí>&5��û����ñï��àô®ðYð �'��ñõ��8îSõL÷��Jî�&X÷B²Oï�ð &tíI�tí.ïbð»õ��5 &tóLï.���ð»õ��D�5�Jî�&tíS©o��Lí�õEó1�ýô_í�&sð»òd³'�5<î��/í>�5´�&tí�èEó1��&tí�÷�î»õL÷¨��Líc³5� ��ðYð���LõLí�� �5´àð»ò0���í÷����5�&5��ôLóB���ð»õ���û�����ï���î�&tí�ïbð/õ��'�&XóLï�tíb÷]ø4�OõHtí>&Xí>�5 ��õ���(�) `C��Líd$�&tðEïbí�÷Ló�&tí���û����ñï��c�Jîhé/í8ô_í�íbõó��Xíb÷ ò�ð�&-�Jî�&t÷J²Oïbð�&tío�tí.sï�ð/õ��'�&tó�ï���ñð/õ_�Jîhé/íìô_í. tí>&�� ��ðYð���LõLí�� ��î»õL÷>ïjîSõ ô®íìó��tí�÷¨9ð����3²$�&Xðçé»í6î�(��/ð &5����1�d���ñõ�ï�ð �"$Ló1<î,���ð»õJî�(W(�íbî�&tõ1��õ��-���íbð &') `!û�����(ñí�ô®ðEð �5 ��õ��æî�(��/ð & �*����"�?&tí�è!ó���&Xíò�í�û í>&p�59î��/í��~îSõL÷ÿïbî»õ%ô_í�ó��Xíb÷a9ð3���"$�&tð®é»íI�Lî�&t÷B²öïbð &Xí-�tí.~ï�ð/õ��'�&tó�ï���ñð/õ]ø��í6��& �5��5�LðIû��LðIûT9ðìó��Xí�REõ�ðNûPõ%ô®ðYð �'��ñõ��¸î�(��/ð & �*����"�69ðìð/ô19î���õ�õ�í�û��Lî�&t÷B²öïbð &Xí��tí.ïbð»õ��5 &tóLï.���ð»õ��bø+�L�"$Jî�� (���î�·�·bð�$�&Xð®é/í>�"��Lí0ò�ð (�(�ðIû���õ1�/��Lí�ð &tí��ÿ©�����é/í�õ îàò�óLõ�ï���ñð/õ û#�tóLï����Lî��õLð�ï���&Xïbó��* ðSò¬� ��·bí�(�í�� �6��JîSõ"ü¸ï�ð & &Xíbï.�(�)ìï�ð �"$Ló19í��!û0ð/õ"��ð�&tí0��JîSõàü F�� Á<þ &�� ��õ�$LóB��>`���íbõ ò�ð�&8î»õ<)fý � F U & ���í>&tíÿí.AB���'���î¨�Xí� � ð»ò-� ��·bíÿÁ & � �tó�ï��%��Jî,8õLð ï>��&tïbó1��ìð»ò-�5��·�í¡ ü�ý~þ<Á þ¢þLü8ïbî»õ¾ïbð &5&tíbï.�(*)�ïbð��d$LóB9í�ûàð/õ���ð &tí���Jî»õ%î¾ü F U & �Àý þ òl&9îSï���ñð/õ�ð»ò; �Lí-��õ�$�ó1��0�ñõ� ø�ºi)�(�í�5��ñõ���R!õLðIûPõ�ô®ðYð��5��ñõ��Tî�(��/ð & �*����"�W÷1��ï�9î�9í! �Líoïbð»õ��5 &tóLï.���ð»õ�ð»ò~���ío÷����'�& �ñôLó1 ��ð/õ����õe�L�"$Jî�� (���î�·�·bðL� �o$�&tðEð»ò `�ûùí4���"$�&tð®é/í¾ð/õ $�&tí.éJ��ð/ó��"&tí��tó�(*��ìû��� � &Xí>� $®íbï.ktð¨���íÿï���&tï�ó���� ��·bí3$Jî�&tî���í�9í�&6û��� � ð/õ1(�)Úî�� ��î�(�(Wïbð»õ��59î»õ<�ò}î»ï.9ð &-(ñð � ���ñõµ �Lí3�tí.D�5��·bí3$Jî�&9î���í�tí>&bø Q �í.AB$�(�î���õ�íb÷_��õ¨¤Yíbï.��ñð/õ��Yø � ø `�û í�ô®í>(���í�é»íìð/ó�&�$Jî�&tî���í.9í�& �-9ð%ô_í�ð $1 ����î�(Wó1$_tðÿï�ð/õ��59î»õ<ò}î»ï.9ð &5��û��*��k&Xí>�5$_í�ï�!tðI ������ï>(�î�� � ð»òbtíbï��Lõ���è!óLí>��øighî»ô�(ñí F �tó��"�8î�& ��·�í>��ð/ó�&��Jî�&X÷B²Oï�ð &típ�tí.ïbð»õ��5 &tóLï.���ð»õ4&Xí>�tó1(����ø��í� �Líbõe� ��ðNû¥�LðIû¿tðÚó��tí��L�"$Jî�� (���î�·�·bðL� �D�Lî�&t÷B²öïbð &Xí��tí.ìïbð/õ��'�&XóLï� ��ð/õf9ð>ð/ôB<î��ñõ�î��ð &tí~í� �ï>�ñíbõ<!é/í�& �5��ð/õ0ð»òC«/îSï<RJ�Xð/õ]� �oô�&tíbî»R<���&Xð/ó�� �¾�~î�& ��ð/õ��ñï�¤J��í�é»í�î�(��/ð & �*����ãü�«/î»ï<RH�tð/õb`F®­ ­ G/þ~ò�ð &-(�íbî�&tõ1��õ��aê-¯5 ò�ð &5�ýó�(�î»íýóLõL÷�í>&���Lí�óLõ�� ò�ð & �Ò÷����'�& �ñôLó1 ��ð/õÚó��5��õ��a��í>�ýô®í>&5� ����$è!óLí>&5��í��bø3«/î»ï<RH�Xð/õ]� ��ð &5��� �ñõJî�(hî�(��»ð & �*���� (ñíjî�&Xõ���ó�� �ñõ�����Lío�<)J$®ð���Lí�� ���æï�(�î��5��ð»ò6���&Xí>�5�Lð (�÷J²ð»ò�²�$Jî�& �*L)Úò�óLõLï.���ð»õ��ýî»õ�÷c&tóLõ1�D��õ¨�����íìí>�5�tíbõ< ��î�(�(�) §¡ ü�.=¾��+U»Á ÿ9þ þ�`hû��Lí�&tí0. ������Lí�õ!ó��sô_í�&

F F �

Page 126: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

ð»òpéSî�& ��î»ô�(�í��o��õc �Líaê-¯5-ò�ð & �só�(�î1`i¾��������í%õ!ó��sô_í�&�ðSò¡9í�& �"�>` î»õL÷ Á"������Lí%î»ï�ïbó�&tî»ï�)$Jî�&tî���í�9í�&bø�ë6ó�&péSî�& ��î»õ<6ó��Xí>�����í��9î���íD�<)J$®ð���Lí�� ����ï>(�î�� ��î»õ�÷�&tóLõ1�-��õ4 ����í'§¡ ü}.¬¾ � U»Á � þ+ø� ísïjîSõ ò�ó�&5 �Lí>&����"$�&tð®é»íI �LíD&XóLõLõ��ñõ��d ����íIt𠧡 ü}.=¾��+U»Á! ¢þ~î�p���ísïbð��56ðSò?(�íbî�&tõ��ñõ���ó�� �ñõ��îo��ð�&tí�ïbð��d$�(ñí.A�ï>(�î�� �Pð»òC�H)J$®ð����í>�tí��bø�Oõd&Xíbï�íbõ<�ûùð &XRoº@�5�Lð/ó1L)�`�«/î»ï<RH�Xð/õ8îSõL÷3gWî���ð/õÿü�ºi� �Lð/óBL)/¸.¹�ß�ø���` Fh­ ­ ­ þC�Lî®é»í����"$�&tð®é/í�÷���í�&XóLõLõ��ñõ��0�����í�ð»òJ���í �~î�& ��ð/õ��ñï�¤B��í.é/í?9𠧡 ü}.=¾>Â+USÁ5Â�þ.`Iû��Lí�&tíÀú0���¬���í�õ!ó��sô_í�&�ðSòL÷����5��ñõLï�éSî�& ��î»ô�(ñí>�¸û�����ï��Hî�$�$_íbî�&"��õ �Lí/�"��õ1����î�(0ê-¯5�&tí�$�&tí��tíbõ<9î���ñð/õ ð»ò� �Lí49î�& �»í�8ò�ð�& �ýó�(�î�øë6ó�&6&tí��tó�(*��6���"$�&tð®é»í!��Líp&tóLõ�õ���õ��- ����íCð»òb �Lí>��&�õLí�û�î�(��/ð�& �� ���u9ð-§¡ ü�.=¾  U»Á þ þ� ����í!�5tí>$���`û����ñï��"���n���íPò}î��59í��5�R!õLðIûPõ�î�(��/ð�& �� ���'ò�ð &?^ Q � (�íbî�&tõ1��õ��-ê-¯~5 û��� �o��í��ýô_í�& �5����$oè!óLí�& ��í��óLõL÷�í>&0��LíTóLõ1�ñò�ð &5� ÷����5�&5��ôLóB���ð»õ]øë6ó�&��8î���õ%tíbï��Lõ��ñïjî�(ïbð/õ<�&5��ôLóB���ð»õ#���0î4ïbî�&tí+ò�ó�(�î»õLî�(�)J� ����ð»ò����íÚ÷����'�& �ñôLó1 ��ð/õ���ïbð/õJ²�5 &tóLï.9í�÷ ÷Ló�&5��õ��o �Lí�ô®ðEð �5 ��õ��o$�&XðYï�í>�5�bø���íI� �LðIû� �Jî�Pô®ðYð��5��ñõ��3$�&tðEïbí�÷Ló�&Xí>�Pû�����ï��%ïbð/õJ²�5 &tóLï.÷1���5 & �ñôLó1��ñð/õ��Cû��*��a����� ���"��õ����ýó�� íbõ<�&Xð $<)%î�&tíæ÷Lí�� ��&tî»ô�(ñí6ò�ð &��/ðYðE÷��Jî�&X÷B²Oï�ð &tí��tí.ïbð»õ��5 &tóLï.���ð»õ��bøò {5M�{#" �����'SW���;U%$ � �Wvºùð/õ�í>�ìî»õL÷'&;��$19ð»õ0ü�º ð/õLí>�ìîSõL÷�&=��$1tð/õ¬` F®­ ­ � þ=�Jîhé/íPî�$�$�(��ñíb÷(� î»ð*) �+�ë-¶0²�(�í��d��î�tð�$�&Xðçé»í���í8í>è!ó��*éSî�(�í�õLïbí�ð»òCû íjî�, î»õL÷¨�5 &tð/õ��4(�íbî�&tõJîSô���(���L) ò�ð�&æï�í>&'<î��ñõ¨L)J$_í��ýð»òoï�ð/õLï�í>$1sï>(�î�� �tí��óLõL÷�í>&¡ �LíýóLõ�� ò�ð & � ÷����5 & ��ô�ó1��ñð/õ]øpg0�Lí>��&�&tí��tó�(*�î�$�$1(���í���tð�ïbð/õ�ïbí>$BTï>(�î�� �tí��6ï>(ñð �tí�÷ÚóLõL÷Lí�&îo$_ð�(�)EõLð �"��î�(®õ!ó��ýô®í>&Pð»ò-�ë-¶-ð $_í�&9î� ��ð/õ1�bøì Î/. 0ßÔsç�ïÿå21�×�ç�Ñ 30ÑÕ%1�×�ØÚÙhÕ�ç�ðZñoÕ�Ö�×�Ø á54�Ñ�ç64%ÖbÑÝë6ó�&87W& �'÷�í97Jõ��*��ñð/õ¬`19î�,»í�õ òl&tð � ü9�L�"$Jî�� (���î�·�·bð�` F®­ ­;: þ�`�ò�ð &5�8î�(���·�í>�@ �Lí�õ�ð���ñð/õ¾ð»ò�î¸ò�óLõLï�²��ñð/õ û��1��ï��f���]�Jî�&X÷À9ðÚî�$�$�&tðhAB����î�tí»ø�¶PíjîS÷Lí>&5�sû��Lð>î�&Xí�ò}î��"��(���î�&û��*��À �Lí�õLð�<î� ��ð/õ:ð»òü9�L�"$Jî���(���î�·>·bð1` Fh­ ­;: þoû���(�(�õLð� ��ï�íD��Jî,û íìî�&tíýó1� ��õ1�0÷��=<_í>&Xíbõ<�éSî�& ��î»ô�(ñí>�?>~��Lío&Xíjî��Xð/õ��ò�ð�&��1���Cû���(�( ô®íbï�ð ��íTï�(�íjî�&0��õ4¤Yíbï.��ñð/õ@��ø � øACB;D6E �GF®�IH E�JLK��NM;¸.¹oû5O.¸ÀßQP�ã®ã,ø�¸ ß�å�R/SJåbâ.¹9æ�ã,å÷ã,åUT �V�hF�W � ß�å~á � ßTá�æ ä ¹9à�æIO?SB¹9æ�ã,åmã,åT �X�hFYW � ¢ M�¸.¹ �[Z Á Z�F]\_^ ß�å~á�ø�¸.¹a`cbÜüdb ^ � \ ` ¢fe ¸ ä ß�g�¹lÞ1ß�¹WûZæ ä Á'²��Jî�&X÷�ò�ð &�� ��·bí6ü�óLõL÷Lí�&� æ RhRhã,à0ß�åag�P¡ã®ã,ø�¸ ß�åkâ.æ�à�âiSJæ�¹a�Nj?æ�¹lÞoß�¹lkkã ä ¹1üm <ß�¹L¸ äon j@¸6Þ1ß�p�¸n^?& �rq ûWüGs_þ6[t��üGs_þvuwb FX� Á]�

�Oõ�ð���Lí�&�û ð &X÷��>`Yî»õ<)¸ï>��&tï�ó����ð»òÝ� ��·bí�î�6��ð �56üD�ýó��5�÷1���9î�� &tí�íPû��*��dû�û�����3$�&Xð/ôJî»ô1��(��*L)î�@(�íjî��5oÁ�ò�ð &xs%÷�&tîIûPõ¾îSïbïbð�&t÷��ñõ��]tð � ¢�y íû�&5��9íxz�tðý÷Lí�õLð�9í���Lí~óLõ��ñò�ð�& �G÷����'�&5��ôLó1 ��ð/õF F��

Page 127: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

{9E-|+} F_~ &tíbî�(�õ!ó��ýô®í>& � ��Z Á Z�F_�a��Z ý Z�F�\_^ºùðEð (�íbî»õ�ò�óLõLï.���ð»õ/û� } F |+} F;~ ï>��&Xïbó��*~�4�tó�ï��4��Lî�p^?&#� q ��ü�s_þ6[�û�üGs_þvuw� F�� Á

F ø �?B F��+� �^ ø �?B F'� � ü�s_þ�� F� ø }-E Fç� �+� ü ��� þ Z Á�� H� ø (ñí�x� � ô_í~î�ï���&Xïbó��*�ð»ò¬�5��·bí~î����ð �56ü � û��*��d^?& ��� � q ��üGs_þ6[�û�ü�s_þ�uw� F�\�^�� ý: ø �?B Fm��� � üGs_þf� F �ñò�û�üGs_þ?[t� � üGs_þ � ��� � üGs_þf� �IF ð� �Lí>&mû����Xí� ø �?B Fm��� üGs_þf��� ���Y�o� � ���]  üGs_þGYø �?B F'���¢¡ ÿ ü�s_þf� ��� �� F �ñò ��� ü�s_þ Z��F�� Á'ý �(� üGs_þ �ñò � b �(� ü�s_þ8b F]\ Á5ý

� �ñò ��� ü�s_þ£� F�\ Á'ý��ø �?B F��+� � �ÜF­ ø¥¤�B F }w¤YEº�¦� �t§�¨ ü©� � � � ÿ � ¢h¢h¢ � � � úWÿ þ

ªn���»ó�&tí��Yø F ©?g0��í¦«]¬E°0î�(��/ð & �*����%øð®é/í�&�T �X�>F�W � ¢ACB;D6E �GF®�IH E�JLK=­�® ��íjî��Xó�&tíIã,å�T �V�hF�W � æ ä ß6R/SJåbâ.¹9æ�ã,å � ©XT �X�hFYW � � q �X�hF u ¢-¯ ÞW¸hî»ô��tð�(�ó1tí� ��·bí�ã�R-ß[kk¸ ß ä SBà�¸ � æ ä áJ¸.åbã,¹L¸ ádO?g F � F ß�å~á�¸�°?S1ß�ø ä � � � üGs_þ/>�¹�ÞW¸�&Xí>(�î,���é»í¡�5��·bí�ã�R � æ äáJ¸.åbã,¹L¸ á � ü � þDß�å~áa¸�°?S1ß�ø ä F � F \_^ � ¢ACB;D6E �GF®�IH E�JLK¢±³²�ã,àIß�åagdà�¸ ß�øLp,ß�ø´SW¸ á�R/SBåbâ.¹9æ�ã,ådµ n-¶ E�ü�µ!þIáJ¸.åbã,¹ ¸ ä �8î,A � F µqüGs_þ F �

g0�LíaèEóLî»õ<���L)c(�ð �qü ¶ Eìü � þ<úWÿmþD���¸ð»ò�tíbõe&tí�ò�í�& &Xíb÷+9ð î��o��Lí·koæ�åWæ�k�S�k ¸.åW¹9à�ã<¯�g8ð»ò � øg0�Lí�&tíf���àî õJî�9ó1&9î�(�ïbð &5&tí>�5$_ð»õL÷Líbõ�ïbíºô_í.Oûùí�íbõ���íbî��tó1&tí>�>îSõL÷'÷����'�&5��ôLó1 ��ð/õ1�b©���Lí�÷1���'²�&5��ôLóB���ð»õ �'¸ �ñõL÷LóLï�íb÷Hô<)Hîc��íbî��tó1&tí � ����÷Líi7JõLí�÷pô<) �m¸ üGs_þZ[ � ü�s_þ \ F � F ¢�¹ ð/õJ²é/í�& �Xí>(�)�`��ñò � ���¾î ÷����'�&5��ôLó1 ��ð/õ# �LíbõT��Lí_��íjî��Xó�&tí � � �ñõL÷LóLï�íb÷iôH) � ���¾÷Líi7JõLíb÷¨ôH)� � üGs_þ6[ � ü�s_þ \ ¶ E¸ü � þ+ø�g0�!ó�� � � ������Líp(�î�&5�/í>�'6��íbî��tó1&tíPû�����ï��"����îTïbð/õ��'<î»õ<'²L�ýó�(*���$�(�í&tí��tïbî�(��ñõ��¸ð»ò � ü}õLð�tí���Jî, � ����Xí>( òC���oî���íjî��Xó�&tí�`1��Lð»ó�� ��L)J$���ïbî�(�(*)�ð»õLí6û�����ï����Jî��0�ýóLï��� ��î�(�(�í>&��5��·bíI��JîSõ � � þ+øp������ï>(ñíjî�&���Lî� F � � F [ F]\ ¶ Eìü � þPî»õL÷ � ü � � þ0[ F]\ ¶ Eìü ^ � � þ+øg0�!ó��>`1(�î�&5�/í¡��íjî��Xó�&tí��Cïbð�& &tí�� $®ð/õL÷�tðý÷����'�&5��ôLó1 ��ð/õ1��û��1��ï��0÷Lðýõ�ð�Pî��5� ���»õ�(�î�&5�/íPûùí���� �<@tðî»õ<)�$®ð ��õ<�ül� ø�í»ø*`��Jîhé/íI�������a�"��õ1���ýó��Gí�õH &tð $<)�þ+øg0�LíæõLí.AJ�÷Lí97Lõ��� ��ð/õ����Pî�(��tð¸òl&Xð � ül�L�d$Jî�� (���î�·>·�ð�` Fh­ ­;: þ¢©

F F�:

Page 128: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

ACB;D6E �GF®�IH E�JLK=º e ¸ ä ß�g-¹�Þ1ß�¹~ûZæ ä ýW²L�Jî�&X÷B²Oï�ð &tí�ð/õ � ò�ð &C�5��·�íiü�æ R�^?& ����q ûWüGs_þ6[t��üGs_þvu+bF�\_^�� ý�Rhã,à�¸ip�¸.à/g_â.æ�à�âiSBæ�¹£� ã�R ä æ�»,¸�ß�¹¼kkã ä ¹@ü ¢ ²Cã,à �³½ T �X�>F�W � j@¸ ä ß�g4¹lÞ1ß�¹�û+����ýW²�Jî�&X÷B²Oï�ð &tí�ð/õ � ò�ð &�� ��·�í"üeæ RDû8æ ä ý-��Þ1ß�à5á�� â�ã,à�¸�ã,å ��¾ Rhã,à ä æ�»,¸�ü � jnÞW¸.à�¸ ��¾ üGs_þ"æ ä ¹�ÞW¸â�Þ1ß�à5ßJâ.¹ ¸.à�æ ä ¹9æ�â¿R/SJåbâ.¹�æ�ã,åÀã�R � ¢ò {LrC{5M À@Á� xE���h��V�� ��ÂaôuS��WU�yHz����J�Äà ��S?x��6�J�;xg0�LíTò�ð (�(ñðIû���õ��I��Lí�ð &tí�� ÷LóLí-9ð3�L�"$Jî�� (���î�·�·bð����� �Lí��59î�&5 ��õ��o$®ð ��õ<Pð»òWî�(�( ð/ó�&�&Xí>�tó1(����©ªs« B H ¤�B�Å JLK=ÆÈÇ!{?Å�|+É*Ê � � ÉXË;Ë H-Ì ­�Í�ÍhºÏÎ M�¸.¹0ûNO.¸ýÁ?��Þ1ß�à5á'Rhã,à�â.æ�à�âiSJæ�¹ ä ã�R ä æ�»,¸]üQSBå~áJ¸.àzjß�å~á/ø�¸.¹ �·Z ý ZÔF�\�^ ¢�¯ ÞW¸.åf¹lÞW¸.à�¸dæ ä ßÐkk¸ ß ä SBà�¸ � j?æ�¹�Þ � ü � þ��GÁ ä SWâ�Þ_¹lÞ1ß�¹!ûPæ äý-��Þ1ß�à5á�� â�ã,à�¸Dã,å � Rhã,à ä æ�»,¸0üH�B[ ¡ ü}Á þ ý~þ¢þ ü ¢Ñ�¤ HlH�ÒÔÓlÕ B F]ÖS«f~Ø× �5�tó���íÚô<) ûùîh) ð»òTï�ð/õ<�&9îS÷���ï.���ð»õ%��Jî,8ò�ð &�í�é/í�&5)%��íbî��tó�&Xí � û��� �� ü � þ�� Á¡��Lí�&tí����6î8ï���&tï�ó���¼� ¸ ð»ò6�5��·�í¸î����ð �'¡ü � �XóLï��Z��Jî��^?& ����q ûWüGs_þ�[Ä� ¸ ü�s_þ�uf�F�\_^+� ý ¢ ¯~ðIû�ïbð»õ�� �ñ÷Lí>&6��Lí~î�(��/ð & �*����Ù«]¬E°ýû�����ï������6� ��é»íbõ"��õ¦ªn���/ó1&tí���ø ^ øng0�����ùî�(��/ð &5�� �����tí>&tî���*é/í>(*)o��ðY÷���7Jí>� � óLõ< ��(w�� �i&Xí>(�î,���é»í�� ��·�í¡���i(ñí>� �6��Lî»õ0Ábø × ò�9í�&ùíjîSï�����ðE÷���7Jïjî� ��ð/õ¸û íð/ô19î���õsîï>��&tï�ó���6� ¸ î���îSô_ð®é/íSø6ë6õLïbíi��Lí!&Xí>(�î���*é/í@�5��·�íùð»ò � ô®íbï�ð ��í���(�í�� �n �Jî»õ¸ÁWû íCï�ð �3²ô��ñõLí���Lí¸ï���&Xïbó��*��6ð/ôB<î��ñõLíb÷>÷Ló�&5��õ��� �Lío$�&XðYï�í>� �-tð%ï�ð/õ<�&tî»÷���ï.-��Lí¸ð�& ������õJî�(�î��5�tó��"$1 ��ð/õ]øg0�LíTò�ð (�(ñðIû���õ���íjî��5��(�)dé/í�& �=7Jî»ô�(�í�ï�(�î����d�Cî�&tíTó��Xí�ò�ó�(]ò�ð &oó�õL÷Lí>&5�59î»õL÷��ñõ��d��ðNûÚ«�¬E°8û ð &!,J��©

Û �(� üGs_þ@���@���í-�8î�&5� ��õ�ô<)�û�����ï��� �Lí-�8î]ÜXð & �*L)"é»ð�9íTð»ò+� � � ¢h¢h¢ � � � ïbð &5&tí�ï��(*)k$1&tíb÷1��ï� � �Lí-é»î�(�óLí�ð»ò�û�ü�s_þ ¢Û g0�Líp��íjî��Xó�&tí ���¢¡ ÿ î�� �5���/õ���û í>��� �< � 9ðI$_ð���õ<���û��Lí�&tí���Líp�8î�& � �ñõ¸ð»ò]ï�ð & &Xíbï�tõLí>�5�@���(�î�& �/í�`2ûùí���� �< F 9ð]$_ð �ñõ<�� û���í>&típ��Lí��8î�& � �ñõd��� õLí��2î� ��é/í�`Lî»õ�÷k��õ<tí>& ��íb÷1��î�tí�û í>��� �<tð"$®ð �ñõH �Cû��Lí>&XíI �Lí��8î�& � �ñõ����0$®ð � �*���é»í�ôLó1p� ��î�(�( ø�L�d$Lî�� (���î�·>·�ðk$�&Xð®é/í>�- �Jî�æîSò�tí>&�î����ð �' � � [ ¡ ü F]\ ü[Á þ'ý~þ+þtþ~ï.)Yï�(�í������&Xð/ó�� �Z���í�(�ðEð $¬`

� ü ��� þ0�ýó��'~ô®íI(�í>�5�p �Jî»õ Á ¢ ë6õLïbíI������p�Lî�$�$®íbõ��î»õL÷¾ûùí�í.AB������í](ñðYð $b`���p���Píbî��5)�9ð"�tí�í��Lî����� �t§�¨ üI� � � ¢h¢h¢ � � � úWÿ þýî�� &tí�í>�ýû��� �+û ð/õ î�(�(i�ñõ�$Ló1 �ýí.AYïbí�$13��Lð��tí�û����ñï��e�Lî®é»í��� üGs_þÔb � î»õL÷%��íbõLï�í ��� ü�s_þ�[ F ¢ ¤B�ñõLïbí � ü �·� þ Z Á � �����3���"$�(���í>�� �Jî�k^?&#� q û�üGs_þ�[��üGs_þvuf� F¼� � ü �·� þ�� F�� Á ¢ ºùó1��µ���6îk��î�ÜXð &5��L)¾ï���&tï�ó���~ð®é/í�&Tî����ð �5 � � ï���&Xïbó��*��íbî»ï��ð»ò�� ��·bíìî����ð �'�üH� � î»õL÷µ�8î]ÜXð & �*L)ÿð®é»í>& � � ��õ1$Ló1��ïjî»õ>ô®í¸ïbð �"$Ló1tíb÷>ô<)Úî�ï>��&tï�ó����ðSòi�5��·�í¡ ü � � þ ¢ ��ýò�ð (�(�ðIû��D �Jî���+�Jî��ìî�o��ð �'oü � � � � ¡ ü � � þÝb ü¨�2î�9í��>`�û����ñï�� ï�ð/õ<�&9îS÷���ï.������íð &5��� �ñõJî�(_î�� �Xó��d$B���ð»õ� �Jî�¡û����PÁ'²��Jî�&X÷¾ò�ð�&oï>��&Xïbó��*��oð»òn� ��·bí�ü�ó�õL÷Lí>&�z ¢

F F]�

Page 129: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

Þ��5��õ��ÿî0õLð/õB²öïbð/õ��'�&XóLï� ��é/í3$�&XðYðSòitíbï��Lõ���è!óLí�`n¯����tî»õµ�Jî��Tí>�'<î»ô1(����5�Líb÷:î�� ���d��(�î�&p&tí��tó�(*û����ñï��#���d&tí>$®ð &'9íb÷%��õ'ül�L�d$Lî�� (���î�·>·�ð�` Fh­ ­;: þ¢øT�Oõ#¯����tî»õL) �k���íbð &Xí>����Lí ï>��&Xïbó��*"� ��·�í�$Jî,²&9î���í.9í>&����p� (���� �<�(*)0ûùð�& �tíýî���î�ò�óLõ�ï���ñð/õ ðSò?ý ô�ó1-�tó�ô��59î»õ<���î�(�(�)¾ô®í�59í�&6î���î�ò�óLõLï� ��ð/õÿð»òÁP©ªs« B H ¤�B�Å JLK ¬ Ç!{?Å�|+É*Ê � � ÉXË;Ë H-Ì ­VÍhÍhºÏÎ�M�¸.¹?ûßO.¸6Á9��Þ1ß�à á�Rhã,àoâ.æ�à�âiSJæ�¹ ä ã�R ä æ�»,¸pü�SJå~áJ¸.à£zß�å~á¨ø�¸.¹ ��Z ý ZÙF]\_^ ¢�¯ ÞW¸.å#¹lÞW¸.à�¸aæ ä ß�kk¸ ß ä SJà�¸ � j?æ�¹�Þ � ü � þ@� Á ä Swâ�Þf¹lÞ1ß�¹�û�æ äý-��Þ1ß�à5á�� â�ã,à�¸Dã,å � Rhã,à ä æ�»,¸0üH�B[ ¡ ü�ýÝþjü9(ñð �qü ^�\ ý_Á<þtþ9úWÿXþ ü ¢

�Oõ¨¤Yíbï.���ð»õ·��ø � ø ^ û í¸û���(�(hí��5<îSô�(���� ��&Xí>�tó1(����ðSò� �����pL)J$_íýû����ñï��µ�Jîhé/í�î8ô®í� tí>&�ï���&tï�ó���� ��·bí�$Jî�&tî���í�9í�&!��JîSõ%í��� �Lí>&�g0�Lí�ð &tí��à��ø � ð &0g0��íbð &Xí>�à��øqGYøy í�õ�ð�9í� �Jî�0g0�Lí�ð &tí��d���Yø � î»õ�÷�Yø©Gýî��5�tí�&50���í�í.AB���'9íbõ�ïbíTð»òhî�(�î�&5�/í¡��íjî��tó�&Xí�`LõLð�Pî(�î�& �/íp�tí.Pî�� ûùî��0$�&Xð �"���tí�÷k�ñõ�¤Yí�ï���ñð/õ F ø¿Þp� �ñõ��¸îD$�&Xð/ôJî»ô1��(����5 ��ïPî�&5�/ó���íbõ<ùû����ñï���ûùí¡� ��é»í��õ_¤Eíbï� ��ð/õ·�Yø � ø � �L�d$Lî�� (���î�·>·�ðk�Jî��-� �LðIûPõµ��Jî�¡��Lí¸í�A1���5tíbõLï�í�ð»ò î�(�î�& �/íD��íbî��tó�&Xí � ð/õû����ñï��kû������Jî�&X÷B²Oï�ð &tí0���d$�(���í��� �Líoí.AB���'9í�õLïbí�ð»ò_î-(�î�&5�/í@�Xí� � ð»õìû��1��ï��dû�����î�(��tð-�Jî�&t÷B²öïbð &Xíü�û��*���� (���� �< (�)�÷��=<_í>&Xíbõ<p$Jî�&9î���í.9í>&5�<þ¢ø

ì Îoá â�×0×�ÙhÕ�ÖjØàë á54�Ñ�ç64%ÖbÑÝ�Oõ3��������Xíbï.���ð»õ�û í�&Xí�éJ��í+û% �Lí~õ�ð���ñð/õ���ð»ò_ûùíbî�,ìî»õL÷"�5 &tð/õ��I(�íbî�&tõ1��õ��-�ñõ3 �Lí�^ × ¹ (ñíjî�&Xõ���õ����ðY÷Lí�(�î»õL÷Ü� �*é/í>îf�/í�õLí>&tî�(òl&9î���í+ûùð &!, ò�ð &0ô®ðEð �5 ��õ��ºî�(��/ð &5����1�d�ìû�����ï��iïbð»õHé»í>&'ÿû íjî�,(�íbî�&tõ�í>& ���ñõHtðo�5 &tð/õ���ð/õLí��bø¶�í�ïjî�(�(� �Jî�6î�î8ïbð»õLïbí�$1�ï>(�î�� �~ð®é/í>&�T �X�hF�W � ���î8ïbð�(�(�í�ï� ��ð/õ�÷[³ã

��ä ÿ � � ð»ò?ºùðYð�(�íjîSõò�óLõLï.��ñð/õ��sû��Lí�&tí�íbî»ï�� ûæåU�����sî�ºùðEð (�íbî»õ�ò�óLõLï.��ñð/õ4ð/õçT �V�hF�W � ¢ �öò�û îSõL÷ � î�&tídOû ðºùðEð (�íbî»õÿò�ó�õLï� ��ð/õ��~ð/õ�T �X�>F�W � î»õL÷ � ���6î8÷1���5 & �ñôLó1��ñð/õ%ð/õ�T �X�hFYW � � û ío�9îh)a��Jî�6�����î»õÁ?�'ß+¯2¯~à�ã�À æ�k"ß�¹Lã,à£Rhã,àIûèSJå~áJ¸.à � � ò@^?& �xq û�ü�s_þ-[���üGs_þvu£� F¼� Á ¢ g0�Lí3(ñíjî�&Xõ���õ��¾î�(��/ð &5�� ����Jî��ýî»ïbï�í>� �otðÚî»õ8¸XÀHß�k¯~ø�¸�ã,à5ßJâ.ø�¸�鿸ü�û � � þæû����ñï��¬`�û��Líbõ+è!óLí>&5��í�÷¬`?$�&tð®éJ��÷�í>�ìî/(�î»ô_í�(�í�÷í.A�î��"$�(ñí�êIs � ûWüGs_þ�ë û���í>&tí�sa���C÷�&9îjûPõ�òl&Xð �ìT �V�hF�W � î»ïbï�ð &t÷1��õ���tðD��Lí÷����'�& �ñôLó1 ��ð/õ � îSõL÷ûèåè�

��������í8óLõ*,!õLðIûPõc9î�& �/í.sï�ð/õLï�í>$1�û�����ï��À �Lí�î�(��/ð &5�� ���Ð���I &5)J��õ��a9ð4(�íjî�&tõ]ø y í�Jîhé/í����íTò�ð�(�(�ðIû��ñõ��sò�ð &5�8î�(_÷Lí97Lõ��� ��ð/õ]©

ACB;D6E �GF®�IH E�JLK=J�®¡åÜß�ø� Jã,à�æ�¹�Þ�ky°�æ ä ß4�'�&tð»õ��À^ × ¹ (�íbî�&tõ1��õ��>î�(��/ð &5����1�êò�ð &ìî>ïbð»õLïbí�$1ï>(�î�� ���£æ R�¹lÞW¸fRhã,ø�ø�ã�j?æ�å� �â�ã,å~á�æ�¹9æ�ã,å4ÞWã,ø*á ä?í Rhã,àDß�åag�`�� F_� ß�åag�û·å·��� ß�åag�á�æ ä ¹9à�æIO?SB¹9æ�ã,å

� ã,åcT �X�hF�W � � ß�å~á�ß�åag �dZ Á � � Z F n æ R~°_æ ä �æ�p�¸.å¨ßJâ�â�¸ ä�ä ¹ ã�` � Á � ��ß�å~á¦î�ï�ü�û � � þ � ¹�ÞW¸.åF F G

Page 130: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

°eàiSJå ä æ�åf¹�æ�kk¸�¯Wã,ø´g,åbã�koæ9ß�ø?æ�åc` � Á úWÿ � � úWÿ � ß�å~á�¾ �#ð_ñ ü�û]þ � ß�å~áÐj?æ�¹�Þ0¯~à�ã�O�ß;O>æ�øùæ�¹©g/ß�¹¡ø�¸ ß ä ¹F�� �aß�ø� Jã,à.æ�¹�Þ�k °fã�SB¹ ¯�SJ¹ ä ß�å¾Á?�'ß+¯2¯~à�ã�À æ�k"ß�¹Lã,à6Rhã,àpûQSJå~áJ¸.à � ¢ò~í�&tí]¾ �#ð_ñ ü�û]þ@��íjî��Xó�&tí��� �Lí�ïbð��d$�(ñí.AB��L)8ð»ò;��Lí6ò�ó�õLï� ��ð/õ4û óLõL÷�í>&��tð ��í�71AYíb÷a&tíbî��tð»õJî»ô�(ñííbõ�ïbðE÷���õ����Xï��Lí>��í»ø�ªJð &� �Lí¸ïbð»õLïbí�$1Tï>(�î�� �-ê-¯�ª û����ñï��>û íýû���(�(hï�ð/õ��5��÷Lí�&���õµ¤Eíbï� ��ð/õ·�Yø : `¾ �#ð_ñ ü�û]þ!���! �Lí��d�ñõ����ýó1� õ!ó��sô_í�&�ðSòCtí>& �"����õ%î»õ<)0÷����GÜXóLõ�ï���*é/íTõLð &5�8î�(_ò�ð &5�Ô&tí�$�&tí��tíbõ<9î,²��ñð/õ¾ð»ò�û ¢�öòi �Lí¸î�(��/ð &5����1�ã°a���ð/õ�(�)4�/óJî�&9î»õ<9í�íb÷Z9ðÝ7LõL÷:î>ü F�\_^(� ý þ�² î�$�$1&tðhAB����î�9ð &Pò�ð &��Xð ��íýó� �X� �Líbõºû ía�tîh)f��Jî�ì°¨���ìî ü F]\_^�� ý þi�'ß+¯2¯~à�ã�À æ�k"ß�¹L¸](�íjî�&tõ��ñõ��Úî�(��/ð & �*����%ø_�öò�ýP[7 ü F]\ n�üI` � ¾ �#ð_ñ ü�û]þtþXþ�ò�ð�&I�Xð ��íd$®ð (�)EõLð��d��î�(_n¨��Lí�õ û ík�9îh)Z �Jî��°Z���æî�j@¸ ß���(�íjî�&tõ��ñõ��%î�(*²�/ð &5�� ���ÿø y íTû���(�(]î»ôLó��Xí�õLð�<î� ��ð/õ%î»õL÷a�9îh)� �Jî��°�����îÿü F�\�^�� ý þ�² î�$�$1&tðhAB����î�9í-(ñíjî�&Xõ���õ��î�(��/ð & �*����Òò�ð &oû � òP°¨���ýî�ü F�\_^'� ý þL²Oî�$�$�&Xð®AB���8î�tíd(ñíjî�&Xõ���õ1�àî�(��/ð & �*����Òò�ð &] �Lí�ïbð»õLïbí�$1ï>(�î�� ���#û��1��ï��%ïbð/õ��5���'���ð»òC �Lí�� �ñõ�� (�í~ò�óLõLï� ��ð/õ4û ¢¤Yï��Jî�$1��&tí�ü�¤Yï��Jî�$1��&tí�` F®­ ­_� þùî»õL÷��tó�ô��tí�èEó�íbõ<�(*)@ªw&XíbóLõ�÷�üIªW&tíbó�õL÷¬` F®­ ­�� >*ªW&tí�óLõL÷¬` F®­ ­;^ þ�Jîhé/ío����é/í�õÚí.AB$�(���ï>�*�O.ã®ã ä ¹9æ�å� ýî�(��»ð & �*����"�oû�����ï��Úí. �ï���í�õH (�)a�&tî»õ��Xò�ð�& � ûùíbî�,/(ñíjî�&Xõ���õ���î�(*²�/ð &5�� ���d�]��õ<9ð/�5�&Xð/õ��Úð/õ�í>�bø y í8õLðIû�ò�ð &5�8î�(�(�)>÷�í97JõLí0ô®ðEð �5 ��õ��Úî�(��/ð &5�� ���d��ü}î�&tí>(�î�9í�÷÷Líi7Jõ��� ��ð/õ¾ïbî»õ¾ô_í�ò�ð/óLõ�÷4�ñõ�üIªW&tíbó�õL÷¬` F®­ ­_: þtþ¢©ACB;D6E �GF®�IH E�JLK=Í�®¡åfß�ø� Jã,à�æ�¹�Þ�k ³cæ ä]ä ß�æ9áa¹Lã@O.¸3ß�ô®ðYð �'��ñõ��0î�(��/ð & �*����Îæ R�æ�¹ ä ß�¹9æ ä ú!¸ ä ¹�ÞW¸Rhã,ø�ø�ã�j?æ�å� �â�ã,å~á�æ�¹9æ�ã,å í Rhã,àIß�åag'P�ã®ã,ø�¸ ß�å�R/SBåbâ.¹9æ�ã,å�û ß�å~á�ß�åagdá�æ ä ¹�à�æIO?SJ¹�æ�ã,å � � æ RC³Zæ ä �æ�p�¸.å��Z Á � � ZPF_�Ï��Z ýcb F�\_^�� ß�å_¸XÀHß�k¯~ø�¸Dã,à5ßJâ.ø�¸�î�ïsü�û � � þ � ß�å~á�ß�ü F]\_^¼� ý þi�'ß+¯2¯~à�ã�À æ�k"ß�¹L¸ø�¸ ß�à�åWæ�å� fß�ø� Jã,à�æ�¹lÞ�k ·Y¹�Rhã,àkûÄjnÞJæ�â�Þeà/SJå ä æ�åܹ9æ�kk¸[ô � ¹�ÞW¸.å8ß�ø� Jã,à�æ�¹�Þ�k ³Pà/SJå ä æ�å�¹9æ�kk¸¯Wã,ø´g,åbã�koæ9ß�ø0æ�åõô � ý�úWÿ � Á+úWÿ � ß�å~áà�2úWÿ � ß�å~á�j?æ�¹lÞ>¯~à�ã�O�ß;O>æ�øùæ�¹©gÀß�¹Dø�¸ ß ä ¹ F'� �%ß�ø� Jã,à�æ�¹lÞ�k ³ã�SJ¹ ¯�SJ¹ ä ß�å0Á9�'ß+¯2¯~à�ã�À æ�k"ß�¹Lã,à6Rhã,à�ûöSJå~áJ¸.à � ¢ò {#"o{5M öo�L�q��V �L�6�J� �xÂ�óº�P�ix!�J�m�6���p� � ���q�ö�J�-Y½x× (�(h,Eõ�ðNûPõ¾ô®ðYð��5��ñõ��ýî�(��/ð &5�� ���d�?&Xí>(*)�ï�&tóLï���î�(�(�)¸ð»õk��Lí~ò}î»ï�i��Jî�i��Lí�ûùíbî�,k(ñíjî�&Xõ���õ1�sî�(��/ð�²& �*���� ·Y¹ÿïbî»õc7JõL÷>î>ü F�\_^�� ý þL²Oî�$�$�&Xð®AB���8î�tð &Pò�ð &�û�óLõL÷�í>& � �]ò�ð &oß�åags÷����5�&5��ôLóB���ð»õ � � �î���(�ð/õ���î��P·�¹������ �*é/í�õÚî»ïbï�í>�5��9ðd �Lísí�A�î��d$1(�íæð &9î»ï�(�í�é6ìü�û � � � þ ¢�y íD����é/íI��Líæò�ð (�(ñðIû���õ������� �B²L(ñí�é»í>(]÷Líi7Jõ��*���ð»õ]©ACB;D6E �GF®�IH E�JLK}­ø÷ó®Gïbî»õLð/õ1��ïjî�(�ô_ðEð �'���õ1�0î�(��»ð & �*����Ùæ ä ßÔO.ã®ã ä ¹9æ�å� �ß�ø� Jã,à�æ�¹lÞ�kùjnÞJæ�â�Þ�Þ1ß ä¹�ÞW¸fRhã,ø�ø�ã�j?æ�å� "æ�¹L¸.à ß�¹9æ�p�¸ ä ¹9àiSWâ.¹ISBà�¸ í

F F �

Page 131: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

Û ®¡¹ ä ¹�ß] J¸¼ú-¹�ÞW¸�ß�ø� Jã,à�æ�¹�Þ�k ä ¹�ß�à�¹ ä j?æ�¹�Þ � � [ � ß�å~á(S ä ¸ ä ·Y¹�¹Lã� J¸.åb¸.à5ß�¹L¸pßsü F�\_^2� ý þi�ß+¯2¯~à�ã�À æ�k"ß�¹Lã,à�� � Rhã,à¡ûûSBå~áJ¸.à � �h¢Û ®¡¹ ä ¹�ß] J¸ � ¹lÞW¸ÔO.ã®ã ä ¹9æ�å� +ß�ø� Jã,à.æ�¹�Þ�k áJ㮸 ä ¹©j@㨹�ÞJæ�å� ä?ítü �iýcáJ¸lú6åb¸ ä ßeá�æ ä ¹9à�æIO?SB¹9æ�ã,å� � jnÞJæ�â�Þ'R�ß�p�ã,à ä ¯Wã,æ�åW¹ ä jnÞW¸.à�¸o¹�ÞW¸¯~à�¸ip®æ�ã�S ä Þ�g<¯Wã,¹lÞW¸ ä ¸ ä � � � ¢h¢h¢ � � � úWÿ áJã�¯Wã®ã,à�ø´g/ß�¹¯~à�¸ á�æ�â.¹9æ�å� 4¹lÞW¸�p,ß�ø´Sw¸�ã�RIû � ß�å~á ü#þ ý ä æ�k�SBø*ß�¹L¸ ä ¹�ÞW¸�¸XÀHß�k¯~ø�¸�ã,à5ßJâ.ø�¸'î�ïsü�û � � � þdß�å~áø�¸.¹ ä ·Y¹¨ßJâ�â�¸ ä�ä ¹lÞJæ ä¡ä æ�k�SBø*ß�¹L¸ á�¸XÀHß�k¯~ø�¸Dã,à5ßJâ.ø�¸�¹Lã6¯~à�ã>á�SWâ�¸]ß3Þ�g<¯Wã,¹lÞW¸ ä æ ä � � jnÞJæ�â�Þ�æ äß0ü F�\_^¼� ý þi�'ß+¯2¯~à�ã�À æ�k"ß�¹Lã,à�Rhã,à�ûöSJå~áJ¸.à � � ¢

Û ®6R�¹ ¸.àkáJã,æ�å� 4¹�ÞJæ ä à�¸O¯W¸ ß�¹ ¸ á1ø´g(Rhã,à ä ¸ip�¸.à ß�ø ä ¹�ß] J¸ äon ¹�ÞW¸CO.ã®ã ä ¹9æ�å� �ß�ø� Jã,à�æ�¹�Þ�kÙâ�ã�k[O>æ�åb¸ ä¹�ÞW¸�Þ�g¢¯Wã,¹�ÞW¸ ä ¸ ä � � � ¢h¢h¢ � � � úWÿ æ�å ä ã�kk¸�j�ß�gd¹Lã�ã�O>¹�ß�æ�å¨ß�ú6å~ß�øbÞ�g<¯Wã,¹lÞW¸ ä æ ä �ØjnÞJæ�â�Þaæ äß�å0Á9�'ß+¯2¯~à�ã�À æ�k"ß�¹Lã,à6Rhã,à�ûQSJå~áJ¸.à � ¢ì Î ÿ 0ßÔsç�ïÿå21�×�ç�Ñ 30ÑÕ%1�×�ØÚÙhÕ�ç�ðZñoÕ�Ö�×�Ø � ç�×�è â�×0×�ÙhÕ�ÖjØàëò������ M � ö�����V ���WS���ö��'Y���� S�������ªW&tð �\ �Lí6÷Lí��tï�& ��$B���ð»õ��ùð»òb��Lí��Jî�&t÷J²Oïbð�&tí¡�Xí�oï�ð/õ��'�&tó�ï���ñð/õ�ð»òC¤Eíbï� ��ð/õd��ø ^ î»õL÷k �Líïbî»õLð/õJ²��ïbî�(Pô_ðEð �'���õ1�4î�(��/ð & �*���� ð»ò]¤Yí�ï���ñð/õ��Yø��1`oð»õLí ïbî»õ#�tí�íÚî�ï�(�ð �Xí/�5 &tóLï.9ó�&tî�(p&tí>�Xí>�ýô1(�î»õLï�íô®í�Oû íbíbõa��Lí�«]¬��0î�(��/ð &5����1��îSõL÷���Lí�ïbî»õLð/õ��ñïjî�( ô_ðEð �5 ��õ��ýî�(��/ð�& �� ���Gð/ó1 (���õ�íb÷¾î»ô®ð®é/í»ø6g�ðô®íp��ð &Xíp�5$_í�ï>��7Jï�`?ÜXó��'Cî���«]¬��ìî�� �Xó���í>�6��Jî� î��íbî»ï��k�'<î��»íp �Lí>&Xí�����îæï���&tï�ó����� � ò�ð &�û��1��ï��^?&�� � � q ûWüGs_þ��[ � � üGs_þvu¼b F�\�^m� ý � �Lí8ïbî»õLð/õ��ñïjî�(�ô®ðEð �5 ��õ��ÿî�(��»ð & �*���� î�� �Xó���í��] �Jî� �"!ïjîSõ_�»íbõLí�&9î�tíìî��íjîSï��_�'<î��»íìî��H)J$®ð����í>� ���$# � ò�ð�&û�����ï��_^?&�� � q û�ü�s_þ%�[&# � ü�s_þ�uxb F]\_^�� ý ¢g0�Lí���õ�÷LóLïbí�÷¾÷����'�& �ñôLó1 ��ð/õ�� �'¸ � ð»ò�«]¬���ïbð�& &tí�� $®ð/õL÷a$�&tí�ï>���tí>(*)ktð3 �Lí�÷����5�&5��ôLóB���ð»õ�� � � ð»ò���í�ïbî»õLð/õ��ñïjî�(�ô®ðEð �5 ��õ���î�(��»ð & �*���� ü}õLð�9í���Jî��«]¬����5<î�&5���ð�<Úû��*�����Lí]��íbî��tó�&Xí � � [ Fû����ñï��8ïbð�& &tí�� $®ð/õL÷���9ð���Lí�óLõ�� ò�ð & �Ã÷����'�& �ñôLó1 ��ð/õ�z [ � � þ ¢ ªC��õJî�(�(�)�`iÜXó��5ùî��?���í~ïbî»õLð/õ��ñïjî�(ô®ðYð �'��ñõ�� î�(��/ð &5�� ���yïbð��ýô���õ�í>����Líd�H)J$®ð����í>�tí��%# � � ¢h¢h¢ � # � úWÿ ��õ_�tð���í�ò}î�� ���ñð/õ¨tð ð/ôB<î��ñõîÐ7JõJî�(@�<)J$_ð� �Lí>�5���%#4û�����ï��f�Lî��o^?&#� q û�üGs_þo['#�üGs_þ�u�� F��)(9� ��LíЫ�¬�� î�(��»ð & �*���� ï�ð �3²ô��ñõLí>�"��Líàï���&Xïbó��*��¦� � � ¢h¢h¢ � � � úWÿ ô<)e<î�,H�ñõ��À��î�ÜXð &5��L)f9ð�ð/ô1<î���õ î ï>��&Xïbó��**##�XóLï��#��Jî,^?&#� q û�üGs_þ6[+#�ü�s_þ�u-� F��,( ¢

y í�ïbð/õ�ï>(�ó�÷Líd �Jî�¦«�¬��µ���æî»õ4î�(��/ð�& �� ���Òû��1��ï��À�XóLïbï�íbí�÷��o��õ ô_ðEð �5 ��õ��*-~à�ã�p®æ9áJ¸ á/¹lÞ1ß�¹¹�ÞW¸ ä ¹�ß�à�¹�æ�å� eá�æ ä ¹�à�æIO?SJ¹�æ�ã,åTæ ä ¹lÞW¸ÐSBåWæ Rhã,à/k á�æ ä ¹�à�æIO?SJ¹�æ�ã,å�zìø�¤B��õLï�íàô®ðYð �'��ñõ���î�(��/ð & �*����"�òl&tð��Òï�ð �d$�ó1<î� ��ð/õLî�(;(�íjî�&tõ��ñõ��k���íbð &') û���(�(�ûùð &!,ÿò�ð &oß�åagD�59î�&5 ��õ��8÷����'�& �ñôLó1 ��ð/õ¬`nß.-~à�æ�ã,à�æ��@�tí�í>�"�~î��!�ñò;�*0� �Lð»ó�(�÷0ô®í�$®ð � �5��ô�(ñí¡9ð¸ó��XíTî»õ<)�ô®ðYð��5��ñõ��¸î�(��/ð &5����1�÷��õ�$�(�î»ïbí6ðSòx«]¬���îSõL÷ð/ô19î���õ:î��Jî�&X÷B²Oï�ð &tí3�tí.sï�ð/õ��'�&tó�ï���ñð/õ]ø"�Oõ¨ �Lí�õ�í.AJD�Xíbï.���ð»õ:û íd$1&tð®é/í8î� �Líbð�&tí>�yû��1��ï��F F®­

Page 132: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

ò�ð &5�8î�(���·�í>�@ �����0�ñ÷Líjî�î»õL÷%í>�"$��Jî��5��·�í>�0��LíI$Jî�&tî���í�9í�& �Cû����ñï��àî�&Xí����"$_ð &'<î»õ<!tðìð/ô1<î���õ%î�/ðEðY÷��Lî�&t÷B²öïbð &Xí��tí��ï�ð/õ��5 &tóLï.��ñð/õ]øò������ r � /1032�04WS��dôuS�WU�yHz65780½ö90:�Tz;5<2�x��"���V �8�=5<2ACB;D6E �GF®�IH E�JLK�>�>tM;¸.¹ � O.¸kß_á�æ ä ¹9à�æIO?SB¹9æ�ã,å�ã�p�¸.à�T �V�hF�W � ¢ ²�ã,à$?õ� F j@¸ ä ß�g/¹�Þ1ß�¹ � æ ä?�²L�5��ðEð���_æ R ¶ Eìü ^ � � þrb)? ¢× ��î»õa���"��íb÷���î�9í~ïbð/õ��Xí>è!óLí�õLïbí�ð»ò�ê6íi7Jõ��*���ð»õ�����ø@��îSõL÷@��ø F�F `�û í]�Jîhé»í�BA �?Bø¤DC*É Fç�GH EßJLK�>B�FE�R]á�æ ä ¹�à�æIO?SJ¹�æ�ã,å � æ ä ?h� ä kkã®ã,¹�Þ�¹�ÞW¸.å � ü � �Wþr� F]\ ? ¢ACB;D6E �GF®�IH E�JLK�>�­NM;¸.¹3GcO.¸Iß�â ß�åbã,åWæ�â ß�øÏO.ã®ã ä ¹9æ�å� kß�ø� Jã,à�æ�¹lÞ�k jnÞJæ�â�Þ�¹�ß��<¸ ä ß ä æ�å�-�SB¹ (?�IH�� ý �ß�åc¸�JHß�kK-~ø�¸oã,à5ßJâ.ø�¸�î�ï�ü�û � � þ � ß�å~á�ß%ü F]\_^�� ý þ/�'ßL-M-~à�ãNJ æ�k"ß�¹L¸Iø�¸ ß�à�åWæ�å� �ß�ø� Jã,à.æ�¹�Þ�kO�P!¦Rhã,àû ¢

�;� e ¸ ä ß�gI¹lÞ1ß�¹�G�æ ä ß�Ã]ü (?� ý þL²��5IQ��SRmO.ã®ã ä ¹�æ�å� �ß�ø� Jã,à�æ�¹�Þ�k\æ R0¹�ÞW¸2Rhã,ø�ø�ã�j?æ�å� -ÞWã,ø*á ä?í ²�ã,à�ß�ø�ø¸�JHß�kK-~ø�¸!ã,à5ßJâ.ø�¸ ä î�ï$T�û � �VU ß�å~áWT F]\_^X� ý U �'ßL-M-~à�ãNJ æ�k"ß�¹L¸�ø�¸ ß�à�åb¸.à ä �P!�Rhã,àCû � ß�ø� Jã,à�æ�¹�Þ�kG ä æ�k�SJø*ß�¹L¸ ä ß�¹8kkã ä ¹6Ãa[\Ã:T (?� ý U á�æ ä ¹�à�æIO?SJ¹�æ�ã,å ä �$X � � ÿ � ¢h¢h¢ � �$YNZ ÿ Rhã,à��P!+ß�å~á S ä ¸ ä�P!µ¹ ãm J¸.åb¸.à5ß�¹L¸Dß�¹xkkã ä ¹CõÞ�g[-Wã,¹�ÞW¸ ä ¸ ä # X � ¢h¢h¢ � # YNZ ÿ�¢

þ � e ¸ ä ß�gf¹�Þ1ß�¹\Guæ ä ß]?^T (?� ý U ²L�5�V_�_�`��ÄO.ã®ã ä ¹9æ�å� %ß�ø� Jã,à�æ�¹�Þ�k æ R�¹�ÞW¸�Rhã,ø�ø�ã�j?æ�å� ÀÞWã,ø*á ä?í²�ã,à4ß�ø�øwR/SJåbâ.¹9æ�ã,å ä û½ß�å~á,T F�\_^¦� ý U �'ßL-M-~à�ãNJ æ�k"ß�¹L¸aø�¸ ß�à�åb¸.à ä �"! n jnÞW¸.åaG8æ ä �æ�p�¸.åî�ï$T�û � z U ß�å~á*�P! n j?æ�¹�Þfåbã,å�»,¸.à�ã%-~à�ã�O�ß;O>æ�øùæ�¹©g�O.ã,¹�Þ ã�Rk¹lÞW¸¼Rhã,ø�ø�ã�j?æ�å� c¸ip�¸.åW¹ ä ã®â�âiSBà íü æ ý_¹�ÞW¸ ä æ�k�SBø*ß�¹L¸ áfá�æ ä ¹�à�æIO?SJ¹�æ�ã,å ä �$X � ¢h¢>¢ � �$YNZ ÿ ß�à�¸Z¸ ßJâ�Þ]?^T (?� ý U � ä kkã®ã,¹lÞ n ß�å~á ü æ�æ ý¹�ÞW¸-Þ�gb-Wã,¹�ÞW¸ ä æ ä #õjnÞJæ�â�ÞBGcã�SJ¹c-�SJ¹ ä�ä ß�¹9æ ä ú!¸ ä ^?&#� q ûdTGs U [+#3T�s U u-� F��,( ¢

g0�R-$�&�_ $eR�&�`L)f_�gh`��RKi����j`�& �lkm"`onp_Sqr � � iRsr�t>&�npk^Rsiunpq�$Q�&�` ^ _�g4`IvRWQwke_®éSRWi"R978qnx`Inl_SqfnlrronlyVn�(zQ�&�`I_¦&hR�é�npql)@r{q_�`onp_Sq*_wg-³�i"_SyVnlq8Qw`oRsi�´.inlr�` &onpk"m"`Inl_Sqr TI&4R.é�npq¬` F®­ � � UL|¯�_D}~}�R{t�Qwq.r�`[Q�`IR�`Iv"R�g�_S�p�l_�}{nlq�{`IvRN_S�oRNyO}{vnpt[v.��RsqRN�IQ��ln�·sRNrd��y%�8Q��S�lnzQ�·�·s_*)�r�v8Qw�oiB²�ts_S��RroRL`9ts_Sqrj`I��mtN`onp_Sq |��� B H ¤�B�Å JLK�>ø±³M;¸.¹^GO.¸¡ß�Ã:T (9� ý U � ä ¹�ß] J¸ n ?eT (?� ý U � ä kkã®ã,¹�Þ¦O.ã®ã ä ¹9æ�å� �ß�ø� Jã,à�æ�¹lÞ�k³jnÞJæ�â�Þ�ã�SB¹I�-�SJ¹ ä ß ä æ�¹ ä ú6å~ß�ø�Þ�gb-Wã,¹�ÞW¸ ä æ ä ß�â.æ�à�âiSJæ�¹�ã�R ä æ�»,¸*�8ã�p�¸.àZæ�å�-�SJ¹ ä # X � ¢h¢h¢ � # YNZ ÿ � M;¸.¹DûÙO.¸( ��Þ1ß�à5á¦Rhã,àZâ.æ�à�âiSJæ�¹ ä ã�R ä æ�»,¸3ütSJå~áJ¸.à�z ß�å~áÀø�¸.¹ � Z ý Z��]\_^ ¢�¯ ÞW¸.å�¹lÞW¸.à�¸aæ ä ßõkk¸ ß��ä SJà�¸ � ã,å�T �X����W � j?æ�¹�Þ � T � U � ��\ ?^T (?� ý U ä Swâ�Þ_¹�Þ1ß�¹!ûÜæ ä ý-��Þ1ß�à5á�� â�ã,à�¸kã,å � Rhã,à ä æ�»,¸ü��P��Tlü � � U \ Ã:T (?� ý U ¢

��^��

Page 133: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

Ñ�¤ HlH�Ò�~,� vR����_P_�g�npr�Qwq8Q��p_��S_SmrB`I_6`ovR����_P_wg _�g � vRN_S�oRNy � | � | × ror�myVR�kM�~}�Q���_�gts_�q�`o�IQ�i"nptN`onp_Sq�`ov8Qw`{g�_S��R.éSRs�j�uyVRsQ�rom"�oR � }{nx`Iv � T � U � �]\ ?^T (?� ý U `ovRs��R\nlr{Q%tNnp��tsmnx`�� ¸_�g�ron�·NRfQ�`$y%_Srj`Iü � rom"t[v6`Iv8Q�`����o� ��q ûdTGs U � � ¸ TGs U u�� �]\_^�� ý3�;º<�¨ë�kr�Rs�'é�Q�`Inp_�q�� | ��^P�`Iv"npr{npy%��pnlRsr�`ov8Qw`{g�_S�{R�é�Rs��� ?^T (?� ý U ²�r�yV_�_�`Ivuinlr�`o�onpk"m"`Inl_Sq�¡¢`Iv"Rs�oR\npr9Q%tsnp��tsmnx`����,_�g�r�n�·NRQw`{y%_Sr�`0ü���r�mt[v�`ov8Qw`������ q û�TGs U �t����TGs U uw� ��\_^8� ý£�¯�_D}¤��mq+`ovR¥ke_�_Sr�`onpq�~Q��l�S_S��nl`Iv"y G¦_Sq§npq"�m"`Ir (?�IH�� ý � Q�q"iUé6BT�û � z U ��¨PnlqtsR¥G©nlr?^T (?� ý U ²�r�yV_�_�`Iv � }{nl`Iv%q_Sq1·sRs��_����_Sk8Q�k"np�pnx`���}<RKv8Qhé�R�`IvQw`KT�n U R.éSRs�j�%inprj`I�onlkm"`onp_Sq.¡ � }{v"npt[vG¥ronlyªm�zQ�`IRsr }{np�l��keRf?eT (?� ý U ²�roy%_P_w`Iv � Q�qi)T�npn U `ovRd78qQ��<vM���^_�`IvRNronlr�}{vnpt[v,G¥_SmP`I�m"`or�nlrQ�q ( ²=Q��"��o_�«PnpyVQw`I_��{`o_�û¬m"qiRs�9`IvR$_S�onl�SnpqQ��hinprj`I��npkm"`onp_Sq[zB�bº7�;T�n U � `ovRs��Rªyªmrj`KRb«Pnpr�`�Qtsnl�otNmnl`f� � _�geQw`�yV_�r�`�ü��M�MQ�`IRsrd}{v"npt[vBnpr�QBT ��\_^+� ý U ²=Qw���o_�«PnpyVQw`o_S�£g�_S�?ûfmqiRN��¡ � �S­ n*éSR�G`Iv"nprdtNnp�otNmnl`�}{vRsqBnl`dt�Q��l�pr3�P!$_Sq.inprj`I��npkm"`onp_Sq$¡ � �M®�_D}�k��*T�nln U � `IvR£7q8Q��vM���e_w`IvRNronprd}{v"npt[vG\_Sm"`I�"m"`Ir£yªmr�`dkeR�Q�q (j¯ Qw���o_�«PnpyVQw`o_S�h`I_�ûVm"qiRs�£`IvR�_��onp��npq8Q��Mi"npr�`o�onlkm"`Inl_Sq�z%��º�m"`�r�npqtNRG�nlrIÃ:T (?� ý U ¯ rj`[Q��SR � `ovnpr�7q8Q���vM���e_�`ovRsr�npr\npr\Q*tsnl�otsm"nl` _�g<ronl°sR%Qw`\yV_�r�` � � ü�� à T (?� ý U b\ü �}{vnlt[v*ts_SqM`I�oQ�inltN`Ir<`IvR�_S�onl�Snpq8Qw��Q�ror�myV�P`Inp_�qf`Iv8Q�`�û�nlr (j¯ v8Q��oiVg�_S��tNnp�otNmnl`or�_�g4r�np°NR¡üBmqiRN�zB�

ò������ " ±F0h² ôF³�8´�µ�¶65<80÷ö90:�+¶65<2�·��"���¸ �8�=5�2�·ò�RN�oR�}<RªQw���l� � vRN_S�oRNy � | ��� `I_�_�k"`[Q�nlq�q"RN}¹v8Q���i ¯ tN_S�oR$roRN`�ts_Sqrj`I��mtN`onp_Sqr9g��o_Syà,�q_D}{qk^_P_Srj`Inlq��Q��l�S_S��nl`Iv"yVr | y R����_PtNRsRNi%nlqªr�`IQ��SRNr | ª�nl�orj` � }<R�rov"_�}�v_D}a`�}�_Wi"n=<eRs�oRNqM`�ke_�_Srj`Inpq"�Q��l�S_S�onx`Ivy%r���npRN�piVin=<eRs��RsqM`�vQ��oi ¯ ts_S��R�r�RN`�ts_Sqrj`I��mtN`onp_Sqr | ®�Rb«P` � }<RKtN_SyªknlqR9`ovRsr�RKk^_P_�r�` ¯npq"�ªQ��l�S_S�onx`Ivy%rd`I_ªQ�t[vnpR.éSRWQ$qRN})v8Qw�oi ¯ ts_S��RKroRL`�ts_�qr�`o�omtL`Inp_�qf}{vnpt[v*nlyV�"�o_®éSRNr�_Sq���Rsr�m�l`or_�g���y%�8Q��S�lnzQ�°N°s_fQ�qi¥®�nlrIQ�q npq `ovRªtsnl�otNmnl`Kronp°NRª�8Q��oQ�y%RN`IRN�KkmP`Wv8Q�rWQ�r�RN`Wr�np°NRª�8Q��oQ�yVRL`IRN�}<_S�or�R$`IvQ�qº`ovRsnl�{�oRsr�m�l`or�kM�ºQB�p_S�SQ��onx`Ivy%npt9g�Q�tN`o_S� | ��q�¨PRNtN`onp_Sq� | � | : }<R$nlyV���_®éSRW`IvR\roRL`ronl°sR �8Q��oQ�y%RN`IRN��`I_.}{nl`ovnpqºQ.ts_�qr�`IQ�qM`9g�QwtN`I_���_�g�`ovR ��yV�8Qw�S�pnpQ�°s°N_�»�®�nlrIQ�qV�oRNrom�x`Ir |

y R67�or�`3tN_SqronliRs�wª8��Rsmqil)@rhk^_P_�r�` ¯ kM� ¯ yfQ]Ü�_S�onx`��KQw�p�S_S��nl`ovy¦g��o_Sy�T©ª8�oRNmqi ���D¼S¼�� U }{vnlt[v �g�_S�l�p_D}{npq�ºT¾½SQ�to,�ro_Sq ���D¼S¼_: U � }<R���RNg�Rs�9`I_VQ�rK¿eÀ | × �l�S_S��nl`Iv"yÁ¿eÀ�npr�Q�Â��FÃfTlý Z þ �p_S�8T ��\�( UoU ¯r�`IQ��SR�k^_P_Srj`Inlq�{Q��p��_S�onx`Ivy)}{vnpt[v\ts_�yªknpq"Rsr4nl`or3 vM���e_w`IvRNroRsr£mronlq�{`IvR�yVQ�Ü�_S�onx`���g�m"qtN`onp_Sq |½SQ�to,�ro_�qL)@rKQ�q8Q��x��ronprWToTĽSQwto,�r�_Sq �4�D¼S¼_: U � �"� | :SÅDÆX:�¼ U ��npRs�lir�`Iv"R g�_��p�p_D}{nlq��g�Q�tL`KQwke_SmP`{¿ ÀSÇÈ2É Ö�F JLK¾>�ºFE�R�¿eÀ/æ ä �æ�p�É.å_æ�å�-�SPÊ ä (?�[H]� ý � î�ï�T�û � ¡ U ß�å~á4ß T �]\_^�� ý U �'ßL-M-~à�ãNJ æ�k"ß�Ê=Émj�É ß��øpÉ ß�à�å É.à��"!CRhã,à]û � ÊlÞ8É.åØj?æËÊ�ÞZÞJæ� ,Þ*-~à�ã�O�ß;O>æ�øùæËÊ©g�É ßJâ�Þ¨á�æ ä Ê9à�æIO?SPÊ9æ�ã,å�¡ � jnÞJæ�â�Þº¿eÀ ä æ�k�SBø*ß�Ê�É äRhã,à��P! ä ß�Ê9æ ä ú�É ä

¶7Ì TË¡ � U �1ÃfT ��\w(oÍ U�Î ¶�Ì T�¡ U ���^P�

Page 134: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

� vnpr�npy%yVRNinzQw`oRs�x�.npy%��pnlRsr�`IvQw`�¿eÀWnpr�ÃfT ��\w( Í U ¯ roy%_P_w`Iv | ÿ y RK`ov�mr�_Sk"`IQ�npq%`IvRKg�_��p�p_D} ¯npq"�%vQ��oi ¯ ts_S��RWroRL`�tN_Sqrj`I�om"tN`Inl_Sq:Ç��� B H ¤�B�Å JLK�>VÆNM4ÉLÊ�ûèOLÉ ( ��Þ1ß�à á�Rhã,à"â.æ�à�âiSJæËÊ ä ã�R ä æ�»�É�üõSJå~á�É.à8z ß�å~á�øpÉLÊ � Z ý ZO��\_^ �¯ Þ8É.å Ê�Þ8É.àIÉIæ ä ߦkfÉ ß ä SJàIÉ � ã,åÐT �V����W � j?æËÊ�Þ � T � U �+Ï T ( Í U ä Swâ�Þ*ÊlÞ1ß�Ênûeæ ä ý-��Þ1ß�à5á�� â�ã,àIÉã,å � Rhã,à ä æ�»�É0ü��P�1ÃfTlý~þ�T��l_S�8T ��\�( U�U Z ÿ ü U �®�Rb«P` � }�RBtN_SqronliRs��ª���Rsmq"iL)@rW�pQw`IRN�KG�Ð:ÑcÒÔÓ{Q��l�S_S�onx`Ivy¢g���_Sy�T©ª8�oRNmqi ���D¼�¼;: U TË`IvR�q8Q�y%Rts_�yVRNr�g��o_SyÕ`Iv"R�g�Q�tN`ª`ov8Qw`$`IvRuQ��l�S_S��nl`Iv"y Ö 78�l`oRs��ro×�RL«"Q�y%��lRsr�g���_Sy¤`IvR*_S��np�Snlq8Q���inlr�`I��n ¯

km"`onp_Sq*`o_Vr�npyªm�pQw`IRWq"RN}Finlr�`o�onpk"m"`Inl_Sqr Ub| &4n=,�R ¿eÀ � Qw�p�S_S��nl`ovyØG Ð:ÑcÒÔÓ nlr{Qf ¯ r�`IQ��SR ke_�_Srj`Inpq"�Q��l�S_S�onx`Ivy�g�_���Â%�1Ã�T�ý Z þ�T��l_S� ��\�( U�U �"G Ð:ÑcÒÔÓ tN_SyªknlqRsr�nl`IrWT ��\_^8� ý U ¯ Qw���o_�«PnpyVQw`o_S�or7`I_._Sk ¯`[Qwnpq,Q�q (j¯ Q��"��o_�«PnpyVQw`I_��\g�_S��ûÙkM�6mr�npq�¥Q�yfQ�Ü�_��onl`��¥g�mq"tN`Inl_Sq;_Sq,Â6npq�mP`Ir�}{vnlt[v,yfQ��v8QhéSRfro_Sy%RV�oQ�qi_�yÚnlq�m"`or | × rj`I�oQ�np�SvM`�g�_S��}�Q��oi6Q��o��myVRNqM`�r�v_D}{r�`Iv8Q�`ªro_�yVR%tsnl�otsm"nl`\_�gronl°sR$Ã�T¾Â U nlrKQ (j¯ Qw���o_�«PnpyVQw`o_S��g�_S�¡û | � _fQ�q8Q��x��°sR$`ovR\roy%_P_�`ovqRNrorK_wg�G ª�np�x` � }<R�mr�R�`Iv"Rg�_S�l�p_D}{npq�$g�Q�tN`�}{vnlt[vºg�_��p�p_D}{r�g���_Sy &hRsy%yVQ.� | � Q�q"iÐ&hRsy%yVQ.� | ¼ _�g�T©ª8��Rsmqi �h�D¼S¼;: U ÇÈ2É Ö�F JLK¾>"Û1E�R<GPÜ3ÝlÞàßiæ ä �æ�p�É.åaæ�å�-�S�Ê ä (9�[H]� ý � î�ï�T�û � ¡ U ß�å~áoßBT ��\_^�� ý U �'ßL-M-~à�ãNJ æ�k"ß�Ê�É�j�É ß��øpÉ ß�à�å É.à7�P!'Rhã,à�û � ÊlÞ8É.åj?æËÊlÞdÞJæ� ,Þ�-~à�ã�O�ß;O>æ�øùæËÊ©g%É ßJâ�Þaá�æ ä Ê9à.æIO?S�Ê9æ�ã,åV¡.�-jnÞJæ�â�ÞªGPÜ3ÝlÞàß ä æ�k�SBø*ß�Ê�É äRhã,à��P! ä ß�Ê9æ ä ú�É ä

¶�Ì T�¡ � U �+Ã�T��l_S�8T ��\�( U \ T ( ý U�U�Î ¶�Ì T�¡ U �¨PnlqtsR�ª�Q�tN`�� | ��Å nlyV��lnpRNr<`IvQw`{G Ð:ÑcÒcÓ npr{Ã�T��p_S�T ��\w( U \ T ( ý UoU ¯ roy%_�_�`Iv � }�R _Sk"`IQ�npq��� B H ¤�B�Å JLK�>VJNM4ÉLÊ�ûèOLÉ ( ��Þ1ß�à á�Rhã,à"â.æ�à�âiSJæËÊ ä ã�R ä æ�»�É�üõSJå~á�É.à8z ß�å~á�øpÉLÊ � Z ý ZO��\_^ �¯ Þ8É.å Ê�Þ8É.àIÉ-æ ä ߦkfÉ ß ä SBàIÉ � ã,åT �V����W � j?æËÊ�Þ � T � U �1Ï T ( ý�T��p_S��T ��\�( U�U Z ÿ U ä SWâ�Þ*Ê�Þ1ß�Ênûfæ äý-��Þ1ß�à5á�� â�ã,à[ÉDã,å � Rhã,à ä æ�»�É0ü � �1Ã�T�ý þ T��l_S��T �]\�( UoU Z ÿ ü U �ª�npqQ��p�x��}�R�Rsr�`IQ�k�lnprov�Qfr�`o�o_Sq��Rs��v8Qw�oi ¯ ts_S��R�roRL`Wts_�qr�`o�omtL`Inp_�q�k���ts_�yªknpq"npq�B`IvR���oR ¯é�np_Sm"r{`�}<_VQ�����_MQ�t[vRNr | ��q;TIª���Rsmq"i �h�D¼S¼;^ U ª���Rsmq"i�iRNrotN�onpk^Rsr�Qª`�}�_ ¯ �pR.éSRN�3k^_�_Sr�`onpq�BQ��l�S_ ¯�onx`Ivyá}{vnlt[v*}�_S�!,�r�Qwr�g�_S�p�l_D}{rsÇdQw�p�S_S��nl`ovyá¿eÀ\npr<mroRNiº`o_ªk^_P_Srj`�g��o_�yÁQ�tNtsm�oQ�tN��T ��\�^�� ý U`I_KQ�tstNm�IQwtN� ��\Y�*� Q�qiªQ��p�S_��onl`ovy1G�Ð:ÑcÒÔÓdk^_�_Sr�`or£g��o_�yâQ�tstNm�IQ�tL� �]\Y� `o_WQ�tNtsm�oQ�tN� ( kM�\`[Q�,�npq"�

¿eÀ�Q�r{nl`or�}�RsQ�,*�pRsQ��oq"Rs� | y R t�Q��l� `Iv"npr�ts_�yªknpq"RsiºQ��p��_S�onx`Ivy�GPãdä�å9æ |ç�èËé�êlë�ìÄíjî�é�ï�ðsñoòLòNóNôMë�ì¾í�î�é�ï�õ�öÄ÷bö¾í�õ�øhù�öÄú�ûLî�ö"ü�ìÄû�ûbý�ö¾úw÷[ö�öÄú�í^þ�ÿe÷����LûNì¾ù@ö¾ú����j÷Lé��íeõ¾ú�û[øhédö¾û�wí���ê�ñ� �����ô��õ���û�ûLö¾ú����s÷����Dõ¾ûLéKü�ì¾û��Ní�õ:öÄú�÷[ö£þMÿ7ùàõ��ê�ñ� �����ô��zõ���û�ûbö¾ú ÷Lé�ïWõ�öÄ÷[öÄí�õ4ö¾ú�÷bö�[÷bìÄù�÷LésöÄõhûLýþ�ÿ��j÷Lé �wí�õ�ú�û[øhéKöÄû��í!��ê�ñ� �����"�#Iô��Ëõ���û�ûLö¾úKýcûLì4÷Lì���ù@ö¾ì�÷bìÄù��%$�õ���÷��%�&�I÷��@î�í�õ�')(fó�*

��^_^

Page 135: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

+fBVÅ�Å�É�J2K�>�Í ¯ Þ8ɼO.ã®ã ä Ê9æ�å� �ß�ø� Jã,à�æËÊlÞ�k G-,/.1032Dæ ä ß�å.ÃfTlý Z þ �p_S�8T ��\�( UoU � ä Ê�ß] �É�O.ã®ã ä Ê9æ�å� �ß�ø´� Jã,à�æËÊlÞ�k[�Ñ�¤ HlH�Ò�~<� vR9`I_S�V�pR.éSRs��_�g:G ¹ _�yªk � }{vnpt[vVmroRNr�Q��l�S_S��nl`Iv"yOG�Ð:ÑcÒÔÓ � `[Q�,wRsr�ÃfT��l_S��T �]\�( UoU rj`[Q��SRNrronlqtsR�`ovRd}�RsQ�,W�pRsQ��oq"Rs� }{vnlt[v$nx`hmr�Rsr3npr:¿eÀ7}{vnlt[v$�"�o_®é�npiRNr�T ��\_^X� ý^� U ¯ Q�tstNm�IQ�`IR�vM�P�^_�`ovRsr�Rsr}{nl`ov�ý^�"� ��\Y� � � v"R\k^_�`�`I_Sy��pR.éSRs� � }{vnlt[v�mr�Rsr9Q��l�S_S�onx`Ivy&¿eÀ � `[Q�,wRsr9Ã�T�ý Z þ U r�`IQ��SRsr{r�npqtNRnl`£k^_P_Srj`Ir£Q.T �]\_^l� ý U ¯ Q�����_�«"nlyfQ�`IR��pRsQ��oqRN�£`o_�Q�tNtsm�oQ�tN� ��\Y� � ¹ _Sq"roR>è�mRNqM`I�l�$`IvR<ts_Sy�knpqRNiQ��l�S_S�onx`Ivy&GPã�ä�å�æ mroRNr9`ovR ts�zQwnpy%Rsi*q�myªk^Rs�{_�g£r�`IQ��SRsr |+fBVÅ�Å�É�J2K54a÷ G ,/.1032 æ ä ß�åuÃ�T��p_S�T ��\w( U \�( U � ä kkã®ã�Ê�ÞO.ã®ã ä Ê�æ�å� kß�ø� Jã,à.æËÊ�Þ�k[�Ñ�¤ HlH�Ò�~ ¨PnpqtNR$G Ð:ÑcÒÔÓ nlr�ke_�_Sr�`onpq�.g���_Sy Q�tNtsm�oQ�tN� ��\�� `o_VQwtstsm"�IQ�tL� ( mr�npq�%¿eÀªQ�r{nl`Ir�}<R�Q�,�pRsQ��oq"Rs� � ª^QwtN`�� | ��Å npy%��pnlRsr7`Iv8Qw`9RsQ�t[v�i"npr�`o�onlkm"`Inl_Sq�¡.�8}{vnlt[v�G Ð:ÑcÒÔÓ �8Q�r�roRNr�`I_B¿eÀ$roQw`Inlr!78RNr

¶7Ì TË¡ � U �1ÃfT��l_S�8T ��\�( U \�( U�Î ¶�Ì T�¡ U �

¨PnlqtsRW¿eÀ\nlr�k^_P_Srj`Inlq�ªg��o_SyØQ�tNtsm�oQ�tN�¬T ��\_^�� ý U `I_BQ�tNtsm�oQ�tN� �]\Y�*� ª�Q�tL`�� | ��: npy%��pnlRsr�`Iv8Q�`nlg4¡.� �^nlr�`IvR inlr�`I��npkmP`Inp_�q�}{vnpt[vº¿eÀ$�8Q�r�roRsr{`I_ª�P! � `IvRNq

¶�Ì T�¡ � � U �1Ã�T � U£Î ¶�Ì T�¡ � U �

¹ _Syªk"npqnlq�ª`IvRNroR `�}�_%R>è�m8Q�`Inp_�qr � }<R�78qi�`ov8Qw`¶7Ì TË¡ � � U �§Ã�T��l_S�8T ��\�( U \�( U�Î ¶�Ì T�¡ U �

ª�npqQ��p�x� � }�R¬q_�`oR�`Iv8Qw`*`ovR·78q8Q��WvM���e_w`IvRNronpr*}{vnlt[v1G ã�ä�å�æ _Sm"`I�"m"`IrunpruQaiRN�"`Iv ^yVQ�Ü�_S�onx`��*tsnl�otNmnl`�_®éSRs��`IvR\}�RsQ�,�vM���^_�`IvRNroRNr\# � � r�npqtNR�k^_�`Iv�¿eÀBQwqi�G�Ð:ÑcÒÔÓ�ts_�yªknpq"R\`ovRsnl�vM���e_�`ovRsr�Rsr mr�npq��`ovRªyVQ�Ü�_S�onx`��ug�mqtN`onp_Sq | × r�`o�IQ�nl�Sv"g�_S�j}�Q���i�k^_Smqi�_Sq�`ovR.ronl°sRª_wg�`IvnlryVQ�Ü�_S�onx`��ftNnp��tsmnx`��PnlRs�lir�`IvRWg�_S�l�p_D}{npq"�ªv8Q���i ¯ tN_S�oRWr�RN`�ts_Sqrj`I��mtN`onp_Sq:Ç��� B H ¤�B�Å JLK544>tM4ÉLÊ�ûèOLÉ ( ��Þ1ß�à á�Rhã,à"â.æ�à�âiSJæËÊ ä ã�R ä æ�»�É�üõSJå~á�É.à8z ß�å~á�øpÉLÊ � Z ý ZO��\_^ �¯ Þ8É.å¥ÊlÞ8É.àIÉ�æ ä ßCkfÉ ß ä SJà[É � ã,å�T �X�s��W � j?æËÊ�Þ � T � U �¹Ï T ( T��p_S�^T ��\�( U�U Z ÿ U ä SWâ�Þ�Ê�Þ1ß�Ê6û#æ äý-��Þ1ß�à5á�� â�ã,à[ÉDã,å � Rhã,à ä æ�»�É0ü � �1Ã�T�ý þ T��l_S��T �]\�( UoU Z ÿ ü U �

��^ �

Page 136: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

ò�������� 6 85�7 ôF³�8´�µ�¶65780Äà 03³�·"��804·~�"5¥ôF³�8´�µ�¶65<80\ö90:�·y vnl�pR�}<RBv8QhéSR%keRNRsq¬�oRNg�RN�o��npq��`o_º_�m�W�oRNrom�x`IrW`Iv�mrWg�Q�� Q�rWvQ��oi ¯ ts_S��R.roRL`$tN_Sqrj`I�om"tN`Inl_Sqr �npq¬g�Q�tN` � vRs_��oRsy%r�� | �]�"� � | � � Q�qiû� | ^P� Rsrj`[Q�k�lnpr�v6`IvRfRL«Pnprj`IRsq"tsR�_�g{vQ��oi ¯ ts_S��Rfy%R�Q�r�m�oRNr�IQ�`IvRN�{`ov8Q�qºv8Q���i ¯ tN_S�oR roRL`Ir | y R q_D}Frov_D}Fv"_�}1v8Q���i ¯ tN_S�oR y%R�Q�r�m�oR ts_�qr�`o�omtL`Inp_�qr{romt[vQ�r � vRs_��oRsy%rÝ� | �]�P� � | � �6Qwqi�� | ^P� npy%��l�6ts_S���oRNro�^_Sqinlq�6v8Qw�oi ¯ ts_S��R�r�RN`�tN_Sqrj`I�om"tN`Inl_Sqr |� vnlrKts_�qHé�Rs��ronp_�q¥g���_Sy v8Q���i ¯ tN_S�oR$yVRsQ�rom��RsrK`o_�v8Q���i ¯ tN_S�oR$roRL`IrWnpr�k8Q�r�Rsi�_Sq�Qwq�Q��o��myVRNqM`g��o_�y¤T���y%�8Q����pnzQw°s°s_ � �D¼S¼_: UL|

y RW}{nl�p�:mr�RW`IvRWg�_S�l�p_D}{npq��ts�om"iR$�lRsy%yfQPÇ+fBVÅ�Å�É�J2K5484 ¯ Þ8É�åaS�k[OLÉ.à�ã�R¼P�ã®ã,øpÉ ß�å¨â.æ�à�âiSJæËÊ ä ã�R ä æ�»�É�üµæ ä ß�Êrkkã ä Ê<ToTG` �ç: U üBþ U:9 �Ñ�¤ HlH�Ò�~W� _*r��eRNtsnlgË� Q�tsnl�otNmnl`�nl`Krom<;%tsRNrW`I_�r��eRNtsnlgË� � g�_S��R�Q�t[v¥_�g?üº�MQw`oRsr � `Iv"Rª`�}�_*nlq�m"`orQ�qiV`IvR��zQ�k^Rs��_�g:`ovR��MQ�`IR | ª�_���Q���n�éSRNq��MQw`oR9`ovRs��RWQ���R�Qw`<yV_�r�`�üBþ�t[v_SnptNRsr7g�_S�<`ovRKnlq�m"`orQ�qi�Qw`{y%_Sr�`�` �Q: t[v"_SnptNRsr$TGs ÿ � �s��� � s � ��=£��>£��?£� ô �A@CB ��D § ¶!E B U g�_��{`ovR �zQ�k^Rs� |� vR g�_S�p�l_�}{nlq�ªRsQ�r���g�QwtN`{g�_S�p�l_D}{r�g��o_Sy�`IvR iRi78qnx`Inp_�qº_wg F � F �È2É Ö�F JLKF4h­NM4ÉLÊ£� OLÉkߨâ.æ�à�âiSJæËÊ-ß�å~áÐp®æÄÉij �Ùß�å~á�ûuß ä Ê�ß��,æ�å� Ðp,ß�ø´S�É ä æ�åØT �\�_����W � ¯ Þ8É.å����� ��q �fT�s U �8ûdTGs U ue� ÿþ �HG æ R]ß�å~á�ã,åWø´g"æ R��JI1KML XON ÿQPQR � T�s U �VTGs U û�TGs U � ^SG F � F �®�_D}~}�R{r�`IQw`IR{QwqiB��o_®éSR{_Sm"�dts_Sq<é�Rs�or�np_SqBg��o_�y v8Qw�oi ¯ ts_S��R�yVRsQ�rom��Rsrd`o_Wv8Q��oi ¯ ts_��oR�roRL`Ir |

¹ _Sqrj`[Q�qM`9g�Q�tN`o_S�or�vQ®é�R$q"_�`{keRNRsq�_S�"`InlyVnl°sRNi*npq�`Iv"R g�_��p�p_D}{nlq�ª�pRNyVyVQ |+fBVÅ�Å�É�J2K54*±³M4ÉLÊ;ûØOLÉ-ßmP¡ã®ã,øpÉ ß�å�R/SBåbâLÊ9æ�ã,åµã,åCT �X�s��W � ß�å~á � ß'kfÉ ß ä SBàIÉ ä Swâ�ÞVÊ�Þ1ß�Ê ü æ ý� T � U �UT_ß�å~á ü æ�æ ý�û¨æ ä ý-��Þ1ß�à5á�� â�ã,àIÉ-ã,å � Rhã,à ä æ�»�É�üjnÞ8É.àIÉ?üdb ÿþ � Î ^ �

ýÝþVT�þD� ¯ Þ8É.åfÊlÞ8É.àIÉÉ�J æ ä Ê ä ß ä ÉLÊ E j?æËÊ�Þ F E F �XW þ ^ � ä SWâ�ÞºÊlÞ1ß�Ênû æ ä � ý+��Þ1ß�à5á�� â�ã,àIÉ�ã,å E Rhã,à ä æ�»�É�ü^�Ñ�¤ HlH�Ò�~ &hRN`'� k^R�Qwq��¬tNnp�otNmnl`$_�g�r�np°NR�Qw`$yV_�r�`Iüe� ¹ _Sq"ronpi"Rs�$`ovRVg�_��p�p_D}{nlq�u�IQ�qi"_SyVnl°sRNits_�qr�`o�omtL`Inp_�q¬_wg�Q*r�RN` E ½ T �V����W � Ç g�_S�KR�Q�t[v·sûåßT �X����W � �m"`�s]npq E }{nx`Iv����_Sk8Q�k"np�pnx`��� T�s U ��&hRN` ��¾ k^RW`IvR t[v8Q��oQ�tN`oRs��npr�`onptWg�mqtL`Inl_Squ_�g E �hª8_S��RsQ�t[vscå�T �X����W � `ovR RL«P�^RstN`oRsiéwQ��pm"RW_�g ��¾ T�s U npr � T�s U � Q�qiu`Iv�mr{kM�u�pnlqR�Q���nl`��%_�g�RL«P�eRNtN`IQw`Inl_Sq*}�R\v8QhéSR

B YZ\[I1KML XON^] P R ��¾ TGs U �fT�s UO_ T�s U�`a � [I1K�L XON^] P R � TGs UVb TGs UV_ TGs U b ^dc F � F}{vRN�oR�`Iv"R�npqRfe�m8Q��pnx`��Bg�_S�p�l_�}{r�g��o_�y ª�Q�tL`hg | ^ �ªQ�q"i _ keRNnpq� c8¯ v8Q��oi ¯ ts_��oRK_�q � g�_S��ronl°sRCi^�ª8_S��RsQ�t[vkj�Qw�pmR�_wg-sº`Iv"R e�mQ�qM`Inl`�� ��¾ TGs Ulb T�s UO_ T�s U npr�Qª�oQ�qi_SymjwQ��onpQ�k�pR�npqf`ovRWnpqM`oRs�VjwQ��

��^Sn

Page 137: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

qpo �rqs� uÄ�hò9_PRi< i"npq�*)�r�`[Qwnp� k^_Smq"i�q_D}1npy%��pnlRsr<`ov8Qw`��� YZ �^ds [I1K�L XON^] P R ��¾ T�s UVb TGs UO_ TGs U � ntc � T � U�`a b Rb«"�vu o ^ Î ^ s T ^�c � T � U�Ulwn x

b Rb«"�3T o ^ Î ^ s c w T w U}{vRN�oR�`Iv"R�roRNts_SqiunlqRye�m8Q��lnl`��%npr<keRNt�Q�m"roR � T � U �UT�{z7�Ý&hRsy%yVQ|g | ^_^ Q�q"i�`IvR�k^_Smq"i*_Sqi�}<Rfv8Q1jSRV`Iv8Qw`\`Iv"Rfq�myªk^Rs�$_�g�tsnl�otNmnl`or$_�g�ronp°NR�Qw`$y%_Sr�`3i¥npr$Qw`$yV_Srj`*T�TI` �è: U i w U}9�~]]�X RL«P�3T ^ Î ^ s c w T w U � � v�mr�`ovR ��o_SkQ�knp�lnl`��B`IvQw`{`IvRN�oR RL«Pnprj`Ir9Q b }{nl`ov F b F bUifromt[vº`Iv8Q�`� I1KML XON^] P R ��¾ T�s UVb TGs UO_ TGs U � ntc F � F nlr��pRNror�`Iv8Qwq ]]�X ���R�Qwq�}{v"np�pR � }<R�Qw�pro_uv8Q1j�RV`IvQw` B q F E F u7� F � F Q�qi F E F npr\QuromyÕ_�g ^ s npq"iRs�^Rsqi"RsqM`�IQwqi_Sy�jwQ���nzQ�k"�pRsr�R�Q�t[vu}{nl`ov��oQ�q�SR'Td� qs��W � � v�mr�ò�_�R9<einpq�X)@r�k^_Smqiºnpy%��lnpRsr7`Iv8Q�`

����� F E F^ds Z � T � U^ � b RL«P�3T o ^ Î ^ s T � T � U \_^ U w Ub RL«P�3T o ^ s T w \_^ U �

��g�T Z ]w RO��� `IvRNqV`Iv"R�k^_Smqi%_Sq�iªnpqB`Iv"R�rj`[Qw`oRsy%RsqM`�_wg `Iv"R��lRsy%yfQ�nlr7�lRsr�r7`ov8Q�q ]w s Q�q"iV`Iv"R�pRNyVyVQ$nlr<`o�on5j�npQ��p�x�ª`I��mR | � v�mr�}<R Q�ror�my%RK`Iv8Q�`�T@� ]w R���� }{v"npt[v*kM�f`ovRWQ�k^_MjSRWnpq"Rye�m8Q��lnl`��npy%��lnpRsr7`Iv8Q�`9��� q F E F ��� ¸ �w uw� ]]�X �� vRsr�R�`�}�_.�"�o_Sk8Qwknp�lnl`��.k^_Smqi"r�`I_S��RN`Iv"Rs��nlyV�"�l�ª`ov8Qw`�`ovRs��R�RL«Pnlr�`Ir<ro_Sy%R E }{nx`Iv F E F �� ¸ �w � W w ^ s r�mt[v�`ov8Qw`�g�_S��R�jSRs�j�ºtsnp��tsmnx` b }{nl`ovºQw`{yV_�r�`hi��SQw`IRNr[I1KML XON^] P R ��¾ TGs Ulb T�s UO_ T�s U b nSc F � F b�g c F E F ��g c F ��¾ F �z<�¦ª�Q�tN`�g | ^ �\}<R�v8Q1j�RK`Iv8Q�`���� I1K ¾ q b TGs U � _ TGs U uwb ]w ��ntc g�_S�7RfjSRN���fromt[v*tNnp�otNmnl` b q Qwqi`Iv�mr _ npr ntc8¯ v8Q��oi ¯ ts_��oRW_Sq E �

¹ _Syªknlqnpq"�V`Iv"nprK�lRsy%yfQB}{nl`ov � vRs_��oRsy�g | ^P� }<Rª_Sk"`IQ�npq `ovR�g�_S�l�p_D}{npq"�Vv8Qw�oi ¯ ts_S��R�roRL`ts_�qr�`o�omtL`Inp_�q:Ç��� B H ¤�B�Å JLK54�ºNM4ÉLÊ _ OLÉ (y�Q�<�t�V� R1� ��������� S � Ê��h��R�� � »�É�i¦S&� � É � zNj � É � É�i.�+Ã�� w Rs Π� �¡^¢}£ ]�¤ ��¥� � �v¦ ÉLÊ�� Z§c Z¢�]\_^ � ¯ � É��,Ê � É � É � � � �DÉLÊ E ½ Td� qs��W s j � Ê � F E F �ÁÏ T ( T��l_S� ��\�( U Z¨] U ^ s�iS ��� Ê �<� Ê _ � � ntc-�Q�<�t�V�t�O� � � É©�d� E R1� � � � »�Éhi��P�1Ã�T c w T��l_S��T ��\�( U�U Z¨] i U �

��^_:

Page 138: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

ª������}« ¬ 7®­�85-¯���2�°á�¨±�0³²90:��²��:´h0¶µ ³�8³7&0:�04��q�`Iv"npr�r�RstL`Inp_�q�}�R�r�v_D}1v"_�})`ovR�roRL`�ronl°sR��Q��IQ�y%RN`oRs�<_�g � vRs_��oRsy�g | ^_: t�Qwq�keRKnpy%��o_MjSRNig��o_�yáÏ T ( T��p_S� ��\w( U Z¨] U `o_$Ï T ( U � � vR{k8Qwronpt{nliR�QWnpre�mnx`IR�ronlyV�"�pR{Q�qiB}�Q�r�rom�S��Rsr�`oRsi�kM� × j�ny np��iRs��ro_SqaT y np�Si"Rs�or�_Sq �3�D¼S¼S¼ UL| ��g�`Iv"R�v8Q���i ¯ tN_S�oR\roRL` E npr9yªmt[v royVQ��p�lRs��`Iv8Qwq (o^ s `Iv"Rsq}<Rªt�Q�q�`IQ�,�RªQD}�Q�� `ovR��^_SnpqM`orKnpq�`IvnlrKv8Q���i ¯ tN_S�oR$roRL` Q�qi _ yªmr�`�r�`Inl�p�4k^RªQ��lyV_Srj` (=¯ v8Q���ig�_S��tNnp��tsmnx`Ir�_�g3r�np°sR·ifmq"iRs��`ovRWmqnxg�_S�oy&inlr�`I��npkmP`Inp_�q�_Sq�`Iv"RW�oRsyVQ�nlqnpq�$�e_�npqM`Ir | y RWtsQ�q`Iv"Rsqu�oRsQ����x� � vRN_S�oRNy�g | ^_: _Sq�`Iv"R\��RsyVQ�npq"npq�$�e_SnlqM`Ir�mr�npq�¬Ö�Q��py%_Sr�` ( ×Bnpq*��pQ�tsRK_�g ( `o__Sk"`IQ�npq�Q�qRN}+v8Q���i ¯ tN_S�oRKroRN` | � vnpr��"�o_�tsRsi"m�oRWtsQ�q*keRK�oRs�^R�Q�`IRsi*m"q�`onp�^`IvRK`o_�`[Qw� ronl°sR�_wg3Q��p�`Iv"R\vQ��oi ¯ ts_S��R roRN`or{npr{Ï T ( ^ s U �� _KyfQ�,wR7`ovnpr£Q����Smy%RsqM`3��oRNtsnpr�R<}<R<}{nl�p�MqRNRsi�`o_Kts_�qronliRs�£yVRsQ�rom��Rsr3}{vnpt[v�Q��oR<iR978q"Rsi_MjSRN�$���_S�^Rs�\romkr�RN`or�_�g�Td� q���W s �4��g�¸ ½ Td� qs��W s Q�qi � Ç�¸ ¹ q � q�� u�nlr Qºy%R�Qwrom��R%}<R}{�onx`IR»º � º ¼,`I_\iRNq_�`IR��kI1K ¼ � TGs U Q�qi � ¼WT � U `I_ iRsq"_�`IR½� ¸ � ¾� ¼ � � y R�}{�onx`IR�z¼a`o_\i"Rsq_�`oR`Iv"R\m"qnlg�_S��yÁinlr�`o�onpk"m"`Inl_Sq�_SqJ¸��� vR g�_S�l�p_D}{npq"�¬��RsqRN�IQ��lnp°�Q�`Inp_�q~_wg � vRs_��oRsy g | ^_: nlrVRsQ�ronl�l�,r�RsRNq©`I_]g�_��p�p_D}Øg��o_Sy `Iv"R

Q����Smy%RsqM`Ir{_�gd¨PRstL`Inl_Sq½g | n | � |��� B H ¤�B�Å JLK54�ÆN² � J�¸ ½ Td� q��YW s �iS ��� Ê �<� Ê º ¸¿ºl� ^ s \_^ �6M4ÉLÊ _ OLÉ (y�Q�<�t�V� R1� �À������� S � Ê����R©� � »�ÉÁiöSÂ� � É � zü j � É � ÉÁi¬�ÁÃ�� w Rs ÎÄ� �¡^¢}£ ]�¤ ��¥ � � �v¦ ÉLÊ�� ZXcßZÁ��\_^ � ¯ � É��,Ê � É � É � � ��DÉLÊ EÆÅ ¸ j � Ê � º E ºe�ØÏ T ( T��l_S�^T �]\�( UoU Z¨] U º ¸Uº��iS ��� Ê �<� Ê _ � � c�Q�<�t�V�t�O� � � ÉÇ�d� E R1� � � � »�Éi��P�1Ã�T c w T��l_S��T ��\�( U�U Z¨] i U �Ñ�¤ HlH�Ò�~º� v"R*_Sq�l�¬��pQ�tsRV}{vRs��R�`IvR�i"_SyfQwnpqQTd� q���W s nprªmr�Rsianpq]`IvR��"�o_�_�g{_�g � vRs_��oRsyg | ^_: npr7npqB`IvRÈj�Qw�pmR ^ s }{vnlt[v�_PtNtsm��r�nlq%`ovR¼ò9_PRi< i"npq�\`[Q�nl�8ke_Sm"qir7_�gw&hRsy%yfQ3g | ^Sn | ��`�nlrr�`o�IQ�nl�SvM`og�_S�j}�Qw�oiu`I_ÉjSRs��nlgË�*`ov8Qw`{`IvRN�oR\npr{Rsq"_Sm�Sv�ro�zQwto,*npq*`IvRNroR$ke_Sm"qir{g�_S��`IvRNy¢`I_%r�`Inl�p��S_.`Iv"�o_Sm��vº}{nx`Ivk¸Únpqu��zQwtsRW_�g�Td� q���W s Q�r��p_Sq"�VQwrɺ ¸Uº;� ^ s \_^ �

y R qRNRsiº`IvRWg�_S�l�p_D}{npq��R�Q�rj�*�pRNyVyVQ"Ç+fBVÅ�Å�É�J2K54eÛ1EGR!¸ ½ Td� q���W s � � Ê � �ËÊ�É��Aº ¸Uºø�¹T � o �w U ^ s � � � _ � � (y�Q�<�t�V� R1� �Á������� S � Ê��|��R� � »�Éhi·S&� � É � z Ê � É�� _ � � �w �Q�<�t�V� R1� �|������� S � Ê��»��R�� � »�Éhi·S&� � É � z¼K�Ñ�¤ HlH�Ò�~ ª8_S�<Q�qM�ftNnp�otNmnl` b }{nx`Iv̺ b ºøb�i.}<R�v8Q1j�R�`Iv8Qw`Aº T]s�Ç b T�s U � _ TGs U W ºøb¹T � o ( U ^ s �� v�mr���� � ¾ q b TGs U � _ T�s U uwb ]�Z �]�Z � ¤ w Z1� o �w �®�_D}1}<R\��o_MjSR\_Sm��rj`I��_Sq�SRNr�`�v8Q��oi ¯ ts_��oRWroRL`KtN_Sqrj`I�om"tN`Inl_Sq |

��^��

Page 139: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

��� B H ¤�B�Å JLK54�JNM4ÉLÊ _ OLÉ (y�Q�<�t�V� R1� ��������� S � Ê��h��R�� � »�É�i¦S&� � É � zNj � É � É�i.�+à � w Rs Î � �¡^¢}£ ]�¤ ��¥� � �½¦ ÉLÊ� Z�cûZá��\_^ � ¯ � É��]Ê � É � É � � � �DÉLÊ E ½ Td� q���W s j � Ê � º E ºl� �w ^ s �iS ��� Ê �<� Ê _ � �c-�Q�<�t�V�t�O� � � É|�d� E R1� � � � »�É�i � �+Ã�T c w T��p_���T ��\w( UoU Z¨] i U �Ñ�¤ HlH�Ò�~7� v"Rs_S��RsyXg | ^_: npy%��pnlRsrh`Iv8Q�`d`IvRN�oR�nlrdQ�r�RN` E X }{nx`Iv�º E X ºw�1Ï T ( T��p_S� ��\w( U Z¨] U ^ s romt[v`IvQw` _ npr c�¯ v8Q���i ¯ tN_S�oR9_Sq E X g�_S�<r�np°NR·i��P�1Ã�T c w T��l_S� ��\�( U Z¨] i U ����g�º E X º�� �w ^ s }�RWQw�oR9i_SqR �ro_V}�R�Q�ror�myVR\`Iv8Q�`Àº E X º Z �w Q�qi��pRN`ȸ ] �³Td� q���W s·Í E X ��ÎhRNyVyVQÀg | ^SÅ npy%��lnpRsr�`ov8Qw` _ nlr�w ¯ vQ��oiºg�_S��tNnp��tsmnx`Ir{_�g�r�np°sRÁiumqi"Rs�£züÏ[� � v"Rs_S��RsyÐg | ^Y� q_D}ânlyV��lnpRNr�`IvR$RL«Pnprj`IRsq"tsR�_wg�QroRL` E ] Å ¸ ] }{nl`IvѺ E ] ºw�+Ï T ( T��p_�� ��\w( U Z¨] U º ¸ ] º�r�mt[v�`ov8Qw` _ npr c8¯ vQ��oi ¯ ts_S��RW_Sq E ] g�_S��r�np°NRi�� �OÃ�T c w T��p_���T ��\w( UoU Z¨] i U �e®�_D}O�lRN`·¸ w �Ò¸ ] Í E ] � ¹ _SqM`Inpq�mR.nlq `Iv"nprKg�Q�r�vnp_�q � _SkP`[Q�nlqnpq�E � g��o_Sym¸ � Qwr{Q�ke_Mj�R � mqM`Inl� � º E � º�� �w ^ s �� _VroRNR\`ov8Qw`{`Ivnlr�}�_S�VÓ�r � q_�`InltsRW`IvQw`9mqM`Inp�-�Ôº E � ºV� �w ^ s R�Qwt[v roRL`·¸ � rIQw`onpr 78Rsrɺ ¸ � º��T � o �w U ^ s � � v�m"r%kM�ÕÎ4RNyVyVQ�g | ^SÅ Q�`.R�Q�t[varj`[Q��SR�}<R*v8Q1jSR�`ov8Qw` _ nlr �w ¯ vQ��oi;g�_S�$ronl°sRÇimqi"Rs��zü�Ö � ro_�}<R�tsQ�q;Q��"��l� � vRN_S�oRNy×g | ^�� R�Qwt[v;`Inpy%Rª}{nl`ov6v8Qw�oiqRNrorª�Q��IQ�y%RN`oRs� (o\_^ �ª8m��`ovRs��yV_��oR � }<R.v8Q1jSR�`ov8Qw`�RsQ�t[vUº E � ºP� Ï T ( T��l_S��T �]\�( UoU Z¨] U º ¸ � º"}{v"Rs�oRغ ¸ � ºh��T � o �w U ^ s �ro_B}<R\y�mr�`�Q�t[vnlRfj�R'�mº E � ºø� �w ^ s QwgË`IRN�9Qw`{yV_�r�`9Ã�T��p_S� �]\�( U r�`IQ��SRNr |®�_D}F�lRN` E ��ã � E � � ¨�npqtNR `ovR E � Qw�oR inpr�Ü�_SnpqM`�}<R$vQMj�Rغ E ºø� �w ^ s �Ù��_S��Rs_MjSRN� � ronpq"tsR _npr c8¯ v8Q��oi ¯ ts_��oRW_SqºR�Q�t[vHº E � ºMg�_S��ronl°sR i�� q g�_��{Q�qM�utsnp��tsmnx` b _�g�ronl°sR Qw`9y%_Sr�`hi��8}<R\vQMj�R

����rÚ q b T�s U � _ TGs U uØ� [� u º E � ºº E º ����tÚ Ö q b T�s U � _ TGs U u x b �

^ �Ñc �� v�mr _ nlr c8¯ v8Q���i ¯ tN_S�oR _Sq E g�_S��ronl°sR i��^Q�qiu`IvRW`Iv"Rs_S��Rsy¢nlr���o_MjSRNi |� vRu�oRsQ�iRs�.yVQ��;v8Q1j�Rºq"_�`InltsRsi;`ov8Qw`.`ovR�`�}�_¬Q����Smy%RsqM`Ir.m"roRsia`I_���_¥g���_Sy Q�v8Q��oi ¯

ts_��oRªy%R�Q�r�m�oRª`o_uQfv8Q���i ¯ tN_S�oR�roRN`\Qwqi�`I_*nlqts��R�Q�r�R�`IvR.r�np°NRª_�g�`IvR�v8Q��oi ¯ ts_��oRªr�RN` i_*q_w`iRN�eRNqiÙnlq�Qwq��;Rsror�RsqM`InpQ���}�QD�a_�q~`ovR*g�Q�tN`.`ov8Qw`B`IvRunlqnl`onzQ���v8Q���i ¯ tN_S�oRuy%R�Q�r�m�oR*}�Q�rB_�gronl°sR � T � U �áÏ T ( T��p_�� �]\�( U Z¨] U � y R.ts_Sm�li¬Q��pr�_*v8Q1jSRB_Sk"`IQ�npqRNi�`IvR.roQ�yVR'7q8Q��d��Rsrom"�l` Æ Qv8Q���i ¯ tN_S�oRVroRL`.ts_Sq"r�`I��mtN`onp_Sq;}{nx`Iv ( \_^ Q�r\`IvR�r�RN`.r�np°NRf�8Q��oQ�y%RN`IRN�ªQ�q"iaÃ�T c w T��p_�� �]\�( U Z¨] UQ�r�`Iv"RWtsnp��tsmnx`{ronl°sRW�8Q��oQ�y%RN`IRN� Æ k���mronlq�.`IvR k^_P_Srj`Inlq�BQ��p��_S�onx`Ivy&¿eÀ\}{vnlt[v�nlyV��lnpRNr<`Iv"RRL«Pnlr�`IRNqtsR9_�g:Q\y%R�Q�r�m�oR{_�geronl°sR9Ï T ( Í U T � vRs_��oRsymg | �]� Ub| y R{npqM`I��_Pim"tsRsiVQ�qiVQ�q8Qw�l��°sRNiV`Iv"RGPã�ä�å�æºQ��l�S_S�onx`IvyOnpqf�Q���`<keRNt�Q�m"roR�`Iv"npr7ke_�_Sr�`onpq�$Q��l�S_S�onx`Ivy }{np�p����pQ��%Q�q�npy%�^_S��`IQ�qM`<��_S�pRnpq*`ovR �pR�Qw�oqnlq�ª`IvRN_S�����oRsr�m�l`or{_�g�¨�RstN`onp_Sq½g | : |

��^SÅ

Page 140: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

ª�������Û Ü ­��8�l7�³���ÝF³�8´�µ�¶;5780³²90h�§¶65�2�·�����¸ ��j5<2Þ8�oRNmqi;v8Qwrªrov_D}{q;`ov8Qw`.Qwq��6Q��l�S_S�onx`Ivy'}{vnpt[v,k^_P_�r�`Ir�Q,T �Mßrà o c U ¯ Qw���o_�«PnpyVQw`oR%}<R�QtÓ�pRsQ��oq"Rs�V`o_,Q�tNtsm�oQ�tN� � o ( yªm"r�`*tN_SyªknlqR�Qw`f�pRsQ�r�`fÏ T c Z w �p_S��T ��ß�( U�U }<R�QtÓ�vM�P�^_�`ovRsr�Rsrnpq;`ovR�}<_S�orj`VtsQ�roR¬T�Þ8�oRsm"qi �9�D¼S¼âá Ub| � v�mr � g�_��BQ�qM�;v8Q���i ¯ tN_S�oR*r�RN`BtN_Sqr�`o�omtL`Inl_Sq,g�Q��p�lnpq�}{nl`ovnpq¥`Iv"npr g��IQ�y%RN}<_S�OÓ�_�m�$tNnp��tsmnx` ronp°NRf�8Qw�IQ�y%RN`oRs� npr\_S�"`onpyVQ��dm�¬`I_ tN_Sqrj`[Q�qM`�g�QwtN`I_��or |Þ8m��`ovRs��yV_��oR � nl`Bnlr.R�Q�rj�,`I_¬roRNRº`ov8Qw`VQ�qM�,�SRNqRs�oQ��{v8Q��oi ¯ ts_��oRuroRL`VtN_Sqrj`I�om"tN`Inl_Sq~romt[v¦Qwr`Iv"_SroR�}<R�v8Q1j�R��SnãjSRNqVyªm"r�`7v8Q1jSR�QWroRN`7r�np°sR{�8Qw�IQ�y%RN`oRs��_�g:Qw`�yV_�r�`7Ã�T ( U � r�npqtNR�`IvR{_S��np�Snlq8Q��(j¯ v8Q��oi$g�mqtN`onp_Sq _ yVnl�SvM`�k^R�j�Rs�j�ªR�Qwr��\`I_�tN_SyV�"m"`IR�T�R | � | tN_Sqrj`[Q�qM` U _Sq.Q � o ÃfT ( U g��IQ�tL`Inp_�q_�genpq�"m"`Ir | � v�mr7}<R9keRN�pnpR�jSR�`IvQw`�`IvR{v8Qw�oi ¯ ts_S��R9r�RN`7ts_�qr�`o�omtL`Inp_�qV�SnãjSRNqVkM� � vRN_S�oRNyÔg | à gnpr$_S�"`onpyVQ���m�;`I_ ts_Sqrj`[Q�qM`ªg�QwtN`I_��or�}{nx`Iv;��Rsro�^RstL`ª`I_ ke_�`ov6`Iv"R�tsnl�otsm"nl`ªr�np°NR�Q�qi,r�RN`.r�np°NR�8Q��oQ�y%RN`IRN�or |ª������}ä � å~5{57·����2�°Á�+�:°75<����¨± 7 æb85�7 çtèHéê]R�v8Q1jSR�q_�`f�SRN`ºiRNrotN�onpk^Rsiìë�mrj`*v_D}'ke_�_Sr�`onpq�;Q��p�S_��onl`ovy%rVyVQ�q8Q���R�`I_,r�npyªm"�zQw`oR�`Iv"Rinãí RN�oRsqM`%inlr�`I��npkmP`Inp_�qr�¡3î9g�_��ª`IvRuRL«"Q�y%��lR*_S�IQ�tN�pRNrÀï�ð.T _ q ¡Aî U }{v"npt[v�Q��oRu�oRfe�mnp��Rsi~k��`Iv"R }<R�QSÓ��pRsQ��oqnlq�BQ��l�S_S�onx`Ivy�Qw`{RsQ�t[v k^_�_Sr�`onpq�ªrj`[Q���R | � vRs��R�Qw�oR�`�}<_Vinãí RN�oRNq�`{`����^Rsr{_�gk^_P_Srj`Inlq�uQ��p��_S�onx`Ivy%r � Ó�q_D}{q6Q�rÀñ��M�M�LÊ � �&ò � ñyó � Ê ¦ Ê=É ��� �&òBQ�qiôñ��M�M�LÊ � �&ò � ñyó � � �tõ - ¦p� �&ò � }{v"npt[vv8Q�q"i�pRº`Ivnlr�npr�romR�nlq©inãí RN�oRsqM`V}�Q���r | ��q©k^_P_Srj`Inlq� ¯ kM� ¯�ö �l`IRN�onlq�¬`ovR�inlr�`o�onpk"m"`Inl_Sq�¡3î�nlrronlyªm�pQw`IRNi\g���_SyF¡FkM� ö �l`IRN�onlq��Rb«"Q�yV�"�pRsr£�oRNtsRsnãjSRNiªg��o_Sy¶ï�ðBT _ q ¡ U �w��g8`ovR ö �l`IRN�onlq���"�o_�tsRsr�rQ�tNtsRs�P`IrBRL«"Q�y%��pRÀ÷�}{nx`Iva��o_�k8Q�knl�pnl`��Ìø�TË÷ U Q�q"i~��R:ë�RNtN`Ir»÷ T�n | R | inlrotsQ��oir©÷©Q�qi,yVQtÓ�RNrQ�q_w`IvRN��t�Q��l�£`o_JïÃð.T _ q ¡ UoU }{nx`Iv����_Sk8Q�k"np�pnx`�� � o ø�TË÷ U � `Iv"Rsq¥nl`WnlrKRsQ�r���`o_*roRNRª`Iv8Qw`K`Ivnlrö �x`IRs��npq�.�"�o_�tsRsr�r�i"R ö qRNrKQ.qRL}Finlr�`I��npkmP`Inp_�q�¡.�8}{vRs��R

¡ � T�÷ U � ø�TË÷ U ¡�T�÷ UùJú ø�TËû U ¡�T�û U � T�g | � U� vR7ke_�_Srj`Inpq"�{Q��p�S_��onl`ovy%r:¿eÀ � G Ð:ÑcÒÔÓ Q�qi\GÂü _Syªk Q��p�SQ��oR�RL«"Q�y%��lRsr4_�gk^_�_Sr�`onpq� ¯ kM� ¯Qö �l`oRs��npq�Q��l�S_S�onx`Ivy%r |��q6k^_P_�r�`Inlq� ¯ k�� ¯ rIQ�y%��pnlq� � _Sq¬`ovR%_�`IvRN� v8Q�qi � QuroRN` E _�g�RL«"Q�y%��pRNr npr i�IQ�}{q¬g��o_Syï�ðBT _ q ¡ U _�qtsRuQ�q"iag�_��ªQ��l��Qw`ª`ovR�k^Rs�Snlqqnlq��_�g{`Iv"R*ke_�_Sr�`onpq� ��o_�tsRNrorBQ�qi,`ovR*npqnx`InpQ��inlr�`I��npkmP`Inp_�qº¡ X npr{`[QtÓwRsq�`o_�keR\`IvR�mqnlg�_��oy inlr�`I��npkmP`Inp_�q _MjSRN� E �:¨�mkroRfe�mRsqM`\inprj`I��npkm ¯`Inl_Sqr ¡Aî�Q���RVq_�q°sRN�o_�_Sq"�l�¥_Sq¬`IvRV�e_SnlqM`Ir _�g E q Q�qi¬`IvR ö q8Q���vM���e_�`ovRsr�nprB#6��RsqRN�IQw`oRsikM��k^_P_�r�`Inlq�ªv8Qwr�vnp�Sv�Q�tstNm�IQ�tL�f}{nx`Iv*�oRNro�^RstL`9`o_ª`IvRKmqnlg�_��oy�inprj`I�onlkm"`onp_SqV_�j�Rs� E �<ê]Rs�l� ¯

��à g

Page 141: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

Ó�q_D}{q*�oRNrom�x`Ir�_�qf�SRNqRs�oQ��pnl°�Qw`onp_Sq%Rs���o_S�KT�z��lmy%Rs��ÉLÊ �t¦ãýp�e��¼ g ¼ U npy%��x�ª`Iv8Q�`�nxg3#fkeRN�p_Sq"�Sr7`o_Q�tN_SqtNRs�"`�ts�zQwror }{nl`Iv¬k^_SmqiRNiìþ7Q��q"nFÓ ¯ ü vRN�Vj�_SqRsq<Ó�nlr�inlyVRNqronl_Sq � `Iv"Rsq6g�_S� r�m<;%tsnpRNqM`I�l��zQw�o�SR E Qwq��uvM���^_�`IvRNronlrW#º}{vnlt[v nlr9tN_S�o��RstL`K_Sq�Q��l�h_�g E }{nl�p� }{nx`Ivºvnp��vº�"�o_Sk8Qwknp�lnl`���vQMj�R�p_D}ØRs���o_S�ªm"qiRs�.¡u� � v"Rº��ÿ��SG��������~Q��l�S_S�onx`IvyÕ_�g�T�Þ���Rsmq"iÙQ�q"iÙ¨�t[v8Q��nl�oR �9�D¼S¼SÅ U nprªQ�qRL«"Q�y%��lRW_�gdQ.k^_P_�r�`Inlq� ¯ k�� ¯ rIQ�y%��pnlq�.Q��l�S_S��nl`Iv"y |��yV�Q��S�pnpQ�°s°N_���rK���_P_�g7r�v_D}{r `ov8Qw` nx`�nprK�e_Sr�ronlk�pR�`o_*mroR���� inl�oRNtN`I�x��Qwr QºÃfT ��ßw( w c w U ¯r�`IQ��SR �$��ßw(j¯ r�yV_�_�`IvÙke_�_Sr�`onpq� ¯ kM� ¯ rIQwyV��lnpq�]Q��l�S_S�onx`Ivy | ��q ü v8Qw�"`IRN� ¼ }�R�}{np�l����n5jSR�Q�qnpy%���_�j�Rsi)k^_P_Srj`Inlq� ¯ kM� ¯ roQ�y%��pnlq�;Q��l�S_S��nl`Iv"y }{v"npt[v§npr*Q��lro_ ��ß�(=¯ roy%_�_�`Iv©k"m"`ºmroRNrº_�q�l�Ã�T ��ß�(Vc w U r�`IQ��SRsr | ��q�`IvR.qRb«�`WroRNtN`Inl_Sq�}<R.inpr�tsmr�rWmronlq� ��P��Q�rWQ�k^_�_Sr�`onpq� ¯ kM� ¯Qö �l`oRs��npq�Q��l�S_S�onx`Ivy&nlqu`ovR t�Q�r�R\}{v"Rs�oRW`ovR npqnx`InzQw�einlr�`o�onpk"m"`Inl_Sq�¡Ánlr�mqnxg�_S�oyØ_�j�Rs���d� q���� s ����� ��������� �"!$#&%(')�+*,�.- / � � ')�$01�2�3�54 *647% 8:9 �

ê]R�v8Q1jSR�roRNRsq$`ov8Qw`hk^_�_Sr�`onpq�{Q��l�S_S��nl`Iv"yVr^t�Q�q k^R�mroRNi\`o_{npy%��o_MjSR�_Sq ��oR�jPnl_SmrhtN_SyV�"�pRL«Pnx`�� ¯`Iv"Rs_S��RN`InltBv8Q��oi ¯ ts_��oRBroRL`$tN_Sqrj`I�om"tN`Inl_Sqr | ®9_�}O}�R%�S_unpq�`Iv"RV_S�"�e_Sr�nl`oR.inp��RstN`onp_Sq¬Q�q"i]mr�Rnpi"R�Q�r�g���_SyÁvQ��oi ¯ ts_S��RWroRL`{ts_Sqrj`I��mtN`onp_Sq*`I_.RNr�`IQ�k�pnlrov*qRL} ��Rsrom"�l`Ir�nlq*�pRsQ��oqnlq��`ovRs_S�j� | ê]Rrov"_�}�`Iv8Q�`�`IvR�mq"nlg�_S��y inlr�`o�onpk"m"`Inl_Sqªk^_P_Srj`Inlq� Q��p��_S�onx`IvyF}{vnlt[v%nlr�npy%��pnltsnx`£nlq����;�t�Q�qBk^Rmr�Rsif`o_�npy%���_�j�R9`ovRK��mqqnlq�$`onpy%R{_�g3½SQ�t�Ó�ro_�q<�@r>=KQ���yV_Sq"npt�¨PnpR�jSRKQ��p�S_��onl`ovy g�_��<�lR�Q���qnpq�?W®CÞ m"qiRs�.`ovR*mqnxg�_S�oyÚinprj`I��npkm"`onp_Sq6mronlq�¥yVRNyªkeRN�or�vnp�Ñe�mRN�onpRNr | � vnlrBQ��p��_S�onx`Ivy¤nlr}{npi"Rs�l�»j�nlRN}<RsiuQ�r7_SqR�_�g:`Iv"R�y%_Sr�`<npy%�^_S��`IQ�qM`<��Rsr�m�l`or<nlqfts_�yV�mP`[Qw`onp_Sq8Qw�8�pR�Qw�oqnlq�W`IvRN_S��� |ê]RVQw�pro_*r�v_D}áv_D}áQ*inãí RN�oRNq�`\y%_Pi"n ö tsQw`Inl_Sq�npq"ro�nl�oRsi¥kM��_Sm"�$Qwq8Q��l��r�npr�nlq6¨�RstN`onp_SqÌg | n |A@t�Qwq%nlyV�"�o_MjSR�`ovR{�omqq"npq�W`InlyVR<Rfj�RsqVg�m"��`Iv"Rs�7Qw`�`Iv"R�ts_Srj`<_wg �lR�Q���qnpq�Kmronlq� yV_S��R�ts_�yV��lRL«vM���e_�`ovRsr�Rsr |z�rov"_Sm"`�� � ½SQwt�Ó�r�_SqºQ�qi � Q�y%_Sq¥T�z�rov_�m"`��]ÉLÊ �t¦ãýp�e�D¼S¼S¼ U v8Q1jSRW��RstNRsqM`I�x���SnãjSRsquQ3j�Qw�onzQwq�`_�g{`Iv"R=KQ���yV_Sq"npt*¨�npRfj�R�}{vnpt[va��mqr.rom"kr�`IQ�qM`InzQw�p�l�¥g�Q�rj`IRN�ª`Iv8Q�q;`ovR*_S��np�Snlq8Q��7Q��p��_S�onx`Ivy |� vRNnp��nlyV�"�o_MjSRNyVRNq�`$npr$_Sk"`IQ�npqRNi6kM�]ro�^RsRsi"npq��m"�,Qº}<R�QtÓ¬�pRsQ��oq"npq� Q��p�S_��onl`ovy'}{v"npt[v;nlrQuts_Sy%�e_�qRsqM`$_�g�`ovRB=�Q��oy%_Sqnlt%¨PnpR�jSR � Q�qi¬npr _S�j`Iv_��S_Sq8Q��d`o_�_�m�$nlyV�"�o_MjSRNyVRNq�`or _�g�`Iv"Rk^_P_Srj`Inlq�ftN_SyV�^_SqRNqM`�_�g�`ovRB¨PnlRfjSR | C r�i"RsrotN�onlkeRNi�keRN�p_D} � kM��ts_Syªk"npqnlq�f_Sm"��`IRNt[vqn5e�m"Rsr}{nl`ov,`IvRNnp�.npy%���_�j�Rsy%RsqM`Ir.}<R�_�k"`[Q�nlq,`IvR*g�Q�rj`IRsrj`�Ó�q"_�}{qÙQ��p��_S�onx`IvyÕg�_S�.�pRsQ��oq"npq�D?W®·Þmqi"Rs��`IvR mq"nlg�_S��yÁinlr�`I��npkmP`Inp_�q�}{nl`Ivuy%Rsyªk^Rs��rovnl�Je�m"Rs�onlRsr |

���

Page 142: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

ªFE}«>EHG I ±2JLK1± 6 M J4³.NPO QRO�°®µSNUTWV XYJ-7Z\[&]6^`_a_FbdcU^ebB^ �t� �gf�hÂ� � Ê �ji ɽ��� ��õ��t¦<k � ��õml ?W®·Þ.n�[L«�o�pH[&qrqHsatvuwsxq)^zy�sxqËëH{�u�],bdsxtvu|t`}2bd[~pr�)q� c�[~pr[)[6^�]�cSbd[&pH��saq�^]&tvu�ëH{�u�]~brsatvuSt�}�zWt�tv_a[&^�uD_asxbr[&pr^�_aq~�%¨�sau�]~[([fjv[~p�� zWt�tv_a[&^�u7}g{�u�],bdsxtvu]6^`u���[�[L«�o�pH[&qrqH[&y�^�q�^�?W®CÞ � bdc�[W]&t�u�]&[~o�b.]&_a^�qrq�?W®CÞ�saq�bdc�[�]&_�^`qrq�t`}�^�_a_�zWt�tv_x[6^�u�}g{�u�]~brsatvu�qtMjv[~p��d� q��`� s � � c�[(�2��� � � ��� ÉFt�}.^)}g{�u�],bdsxtvu _ saq\bdc�[���sxu�sa��{���u�{�����[&p�t�}�br[&pr�)q�sxu�^`u��?W®CÞD[L«�o�pH[&qHqrsat�uB}gtvp _ � � c�{�q>^`u�[�;�]~sa[~u�bW_a[&^�pru�sxu���^�_a��tvprs�bdc���}gtvp.bdc�[�]&t�u�]&[~o�b>]~_�^�qHqW?W®·Þ��{�qRb���[F^���_x[+brt2_a[&^�pru3^�uM�©z�t�tv_x[6^�u�}g{�u�],bdsat�u3sau�bdsa�)[5o�t�_l��u�tv�)s�^�_�sxu�brc�[>u�{�����[&p�t�}Ubd[~pr�)qsaus�bdqFqr��^�_x_a[&qRb\?W®CÞ1pH[&o�pH[&qr[~u�b�^`brsatvu<���u�c�saq qr[~��sxuU^�_vo�^�o�[~p�þ+^�_asa^�u�b<o�tvqr[~y�brc�[e�{�[&qRbdsat�u�t�} � c�[,bdc�[~p¡brc�[&pH[+sxq ^�u�[f;)]&sx[&u�b£� C ü_a[&^�pru�sau�� ^`_a�vtvpHsxbrc��¢}gtvp.?W®·Þ l þ+^�_xs�^�u�b �"�D¼ g n n,�-ê£[>qr^ � sau ü cU^�o�br[&p @ bdcU^eb�bdc�[~pr[Fsxq.^W� C ü_a[&^�pru�sau��B^�_a�vt�prsxbrc�� � c�sx]�c _a[6^`pru�q�bdt^�]~]&{�pr^�]N��¤ ^�u�y pH{�u�q sau¥bdsa�)[ à�¦¨§ s Ï �ª© ¡^¢}£ s ¡^¢}£�«­¬ ß ¤�}gt�p

b�^`pr�v[,bF]&tvu�]&[&o�bdq\t�}�?W®CÞ ¯ qrsl°&[�® q ��{�b\sxbWsaq>u�t`b��v[~b)Ó�u�t � u � c�[~brc�[&pFbrc�[&pH[ saqF^�u^`_a�vtvpHsxbrc��� c�sx]�c�pH{�u�q¡sau bdsx��[�o�tv_x� lg¯ q ® q��Mß ¤�nL�&�°}�bdc�[+_x[6^�pHu�sau��Fqr]&[~uU^�pHsat�saq qr{�sxb�^`��_l����t�y�s ö [&y � bdc�tv{��vc �bdc�[&u±[f;)]&sx[&u�b)_a[6^`pru�sxu�� t�}�?W®CÞL��[~]&tv�)[&q(o�tvqrqHsa��_x[��1��u±^D��pr[&^tÓ�bdc�pHtv{��vc±pr[~qr{�_�b(qr[fj�[&pr^�_�v[&^�prqW^��vt�½v^�]�Ó�qrtvu��v^Mj�[�brc�[�=�^�pr�)tvu�sx]9¨�sa[�jv[�^�_a�vt�prsxbrc�� � c�sa]�cB{�qr[&q õ É õ ñLÉ � � �&�³²�´ h�É ��� É��bdt_x[6^�pHuD?W®CÞµbdt�^�]~]&{�pr^�]N�S¤ {�u�y�[&p bdc�[¶{�u�sx}gt�pr�·y�saqHbrprsx��{�bdsxtvu7sxu prtv{���c�_l� ¯ ®6¸ ß ¤ ] w bdsx��[qHbr[&o�q l ½v^�]�Ó�qrtvu ����¼S¼MÅ n�� C �)[&�3��[~prqrc�sao�e�{�[~p�� sxq ^�u tvpr^�]&_x[|e�{�[&pj� sau � c�sa]�c bdc�[3_a[6^`pru�[~pqro�[&]~s ö [~q�^¶o�tvsau�bh÷7^�u�y�bdc�[��)[&����[&pHqrc�sxotvpd^�]~_a[3�vï!� l _ n>pr[,bd{�pHu�q�brc�[ j`^�_a{�[ _ l ÷�nL�C _�bdc�tv{��vc|brc�[�=�^�pr�)tvu�sx]�¨�sa[�jv[�pr{�u�q�sau|o�tv_x��u�tv�)s�^�_+bdsa�)[ � sxb)saq(u�t`bB]~tvu�qrsxy�[&pH[&yµbrt��[¶]&tv�)o�{�bd^`bdsxtvuU^�_x_l�o�pr^�]~brsa]6^`_�y�{�[�brt�bdc�[¶c�sx�vc y�[~�vpr[~[)t`}+brc�[¶o�tv_l��u�tv�)s�^`_�brsa�)[���tv{�u�y<�êz[ qrc�t � c�t � bdt¶qH{���qRb�^�u�bdsa^�_a_x�Bsx��o�prtMjv[�brc�[�^�_x�vtvprs�bdc����¹q+brsa�)[�y�[~o�[~u�y�[&u�]&[�tvu�bdc�[ [~prprt�p

oU^�pr^��)[~bd[~p\¤ q bdc�{�q���^tÓ�sau��¶o�pHtv�vpH[&qrqFbrt � ^�pHy�q�^¶�)tvpr[ [�;)]&sa[~u�b�sx��o�_a[&�)[&u�bd^`bdsxtvu<�ªFE}«>E�º I ±2J�ÝF³.N87»TWO Q=¸X²�QRJ/¯�J� c�[��B^`sau¼pH[&qr{�_xb�t�} l ½v^�]�Ó�qrt�u ��½D¼S¼MÅ nWsxq>bdc�[ }gtv_x_at � sau��3bdc�[~tvpr[~��Ǿ3¿�À�ÁPÂeÀ�à Ä<Å 4�ÆÈÇRɨÊUË�Ì Í Á�Î5Ï�Ð ÆPÆeÛPÑ�Ò4ÉLÊ _ ñLÉ ��Ó �M� ¦ É � � k h&� � Ê � �d�ô�d�S�d� q6½�� s � k �2��� �� ��� É�® � � ��¦ ÉLÊh�SÔÕ¤ q�Ö Ô ½v× �-� � � �¨ó � �d�<�LÊ � �8Ê G1Ø � q ò �ji É�� �&��� É����VÊ:� ��õ É õ ñLÉ � � �&�³²� �V�&��¦ É�Ù B Ù l _ nBÊ � É3Ú �t��õ �d� �Q�3Û8� É i É �t¦ ò&� ��� Ê �&õ � hÂ�<� � �]Ê ��õ ÉÝÜà lª¯ ® ¸ ß ¤ ] wYÞ�ß n � � �àÃ� Ê �²Ù� �tñ � ñ ��¦p� Ê�ó ½ o Ö �ehPÊ ² hPÊ�� �É� ó ² ��Ê � É�� � ��á �,h ��� Ê �<� Ê�â.p�ã5ä¹á l ÷�n åæ _ l ÷�nYç�èé¤ ×C b�bdc�[Wc�[&^�pHb.t`}�êv^�]�Ó�qrtvu<�¹q5=�^`pr�)tvu�sa]�ë�sx[fj�[F^�_a��tvprs�bdc��¢sxq�^�o�prt�]&[&y�{�pr[�ì�í�î�ï � c�sa]�c � ^�qö pHqHb�qHbr{�y�sx[&yDsau l zW_a{��mð,ñ �t¦ãýaò.½�óvóSn n,��ôFc�[�ì�í�î�ï�^�_a��tvprs�bdc��õbd^tÓ�[~q�^`q sau�o�{�b�^�u [,ö�^��)o�_x[

½ @ �

Page 143: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

tvpr^�]&_x[·ï�ð l _ qH÷ n q ^��)[&����[&pHqrc�sxo�t�pd^�]~_a[C�vï!� l _ n q ^�y�sxqHbdpHsa��{�bdsat�u)t�pd^�]~_a[�?2�Rë�ô lg÷ n5^�u�yB^j`^�_a{�[ Ö(Ø � × C y�sxqHbdpHsa��{�bdsat�u¥tvpr^�]&_x[�?2�Rë�ô lg÷ nFsaq�^�u�t�pd^�]~_a[ � c�sa]�c ò � c�[~u�e�{�[&pHsa[&y � sxbdc�^o�tvsau�b÷�sau(bdc�[�y�t��B^�sxu�t`} ÷Jq pr[,bd{�pHu�q�brc�[)j�^`_a{�[�t�} ÷¥l ÷�n × êésxbrc�o�prtv�U^`��sa_xsxb�ø¶^eb�_x[6^�qRb ½ o Öbdc�[\ì�í�î�ï(^�_a�vt�prsxbrc��ùt�{�bdo�{�bdq�^�oU^`prsxb�ø3}g{�u�]~brsatvu � c�sx]�cBsaq+^ lR½�ßtà o�ú lR½�ß ®enHn�û�^�o�o�pHt�ö�sx�B^`brtvp}gtvp _ {�u�y�[&p ÷Jq � c�[&pH[�®�saq>brc�[�? üCÞ¡û°qrsaý~[ t�} _ ×ôFc�[�=�^�pH��t�u�sa]�ë�sa[fj�[�^�_a�vt�prsxbrc�� � t�pOÓ�q���ø�pH{�u�u�sxu���ÞUpr[&{�u�y<�¹q���t�tvqHbrsau��¶^`_a�vtvpHsxbrc��»ï¨þ� sxbrcµì�í�î�ïw^`q¼bdc�[ � [6^tÓµ_a[6^`pru�sxu��1^�_x�vtvprs�bdc���� C b�brc�[ ÿªû­bdcéqHbd^��v[ t�}���t�tvqRbdsxu��£bdc�[ ï¨þ��t�tvqRbdsxu���^�_a�vt�prsxbrc��·qHsa��{�_�^`br[&q�qrtv�)[)y�saqRbdprsx��{�brsatvu ÷ îW^`u�y£{�qH[&q3brc�[)qrsa�3{�_�^`br[&yD[,ö�^��)o�_x[tvpr^�]&_x[��3¸ l _ qr÷ îgn�}gt�p�ì�í�î�ïP��êv^�]�Ó�qrt�u¥qHc�t � q�brcU^`bFbdc�[ ï¨þ���t�tvqRbdsau��(^�_a�vt�prsxbrc��È]&t�u�qHbrpr{�],bdq[6^`]�c3y�sxqHbdpHsa��{�bdsat�u ÷ î�sau�qr{�]�c�^ � ^6ø�brcU^`b�^`}jbr[&p¡^��Ht�u�[,û­bdsa�)[��Fsxu�sxbrs�^�_`]&tvqRb�t`} Ü� l��(lQà s ÷ îgn���nbdsx��[+qHbr[&o�q l bdt [~qHbdsx�B^ebd[5bdc�[>qH]6^�_xsau���}ª^�]~brtvp�sxu3bdc�[Wy�[&u�t���sxuU^`bdt�p�t�}ÙïÃe�{U^`brsatvuÉg�� ½ n }gtvp�[&^�]�c÷ î q sxb�saq5o�tvqrqHsa��_x[�brt3qrsx��{�_a^`bd[�^�]~tvu�qRb�^�u�bHû­}ª^�]~brtvp�^�o�o�pHt�ö�sx�B^`brsatvu)?2�Rë�ô� lg÷ îgn.t�}�?2�Rë�ô lj÷ îjnsau � lª¯ n�brsa�)[�qRbd[&o�q.o�[~p�]6^`_a_­� w ôFc�[>}gt�_a_at � sxu��\_a[&�)��^�saq�^�y�sxpr[&],b�]~tvu�qr[fe�{�[&u�]~[�t`}�êv^�]�Ó�qrtvu<�¹qΠ[&�)�B^ ó ^�u�y�brc�[�^�uU^�_�ø�qHsaqF{�qH[&y¥sausxbrqFo�prt�t�}��� À�Ã£Ã Ê Ä Å���� ÇHÉ�ÊUËeÌ<Í Á�Î+Ï�Ð ÆPÆ��PÑ(Ò¡ð,ñ _ ñ,ð � �¨ó Ó �M� ¦ ð � � k hÂ� � ñ � �d�Ñ� k �2��� � � ��� ð2®�� i ð ��d� q6½`� s � � �̦ ð,ñ ÷ ñ,ð � �¨ó �t� �,ñ ��� ñ&h�ñ � �d� � i ð � �d� q6½�� s ×��gk ì�í�î�ï � � � h&�whâ� � �&ò���� l _ qr÷ n q��� � l _ n � � � � ��Û"! lj÷ n � � � ñË� � �V�&��¦ ð��$# à � ð � ðzð �&��� �O�t¦�¦ ñ}�%�&� l _ qH÷ nDñ �(' ð���) ñ ��õ ð�,ñ�ð ² �$#+ð �&��� �O�t¦�¦ ñ:������ l _ n�ñ �(' ð��+*3ñ ��õ ðh�,ñYð ² # � � � ñ � ð�ñ ��õ ð � ð ´ h ��� ð õ ð��UñË� � k � ��Û"! lg÷ n�t� ð � � � ð�� ����� ñ,ð �Ø� ñ�� i ð$#Wñ � ð��ì�í�î�ï � h&�<� � �zñ ��õ ð Ü� l ®�, l-� (l�à s ÷ nrn w )/. ¯ ®0� l-� (l�à s ÷ nrn���nñ ��õ ðk�,ñYð ² � � � �£àÃ� ñ �7²Ù� �tñ � ñ ��¦p� ñ�ó � ñ ¦ ð � �,ñ ½ o Ö �eh�ñ ² h�ñË� ��²<�t��� ñ�ó k h&� � ñ � �d� à �&�Q���H� � �lR½Mßrà o ú lR½�ß ®enHn �l�,²�²Ù� �21 ��õ�� ñ}� � ñ:� _ hÂ� � ð �F÷×êv^�]�Ó�qrtvu)o�pHt�j�[&q5bdcU^eb.[6^�]�c)]6^`_a_�brtÁï�ð l _ qr÷ îªn���^�y�[F��ø�ì�í�î�ï�]6^�u(��[Fqrsx��{�_�^ebd[&y3sau�bdsx��[Ü� lª¯3�)lQà s ÷ îgn�4,n � sxbrc¥c�sx�vco�prt��U^���sx_asxb�ø�� ü tvu�qH[ye�{�[~u�br_xøbdc�[�brsa�)[ ��t�{�u�y�t�}�Π[~����^Ég�� @ �saq3y�tv�)sauU^`br[&y1��ø£brc�[ ¯ ®0� l-� (l�à s ÷ nrn���br[&pH� ò ^�u�y1brc�{�q�bdc�[¼brsa�)[¼pr[fe�{�sapr[~y1}gtvp3bdc�[�ÿªû°brc

[,ö�[~]&{�brsatvu7t`}�ì�í�î�ïtvu¥y�saqRbdpHsa��{�brsatvu ÷ î�sxq���tv{�u�y�[&y7��ø Ü� lª¯ ®0� l-� (l�à s ÷ nrn���n × C q�u�t�br[&y7sxuë�[~]~bdsxtvuHg�� n � @ ò }gtvp3^�u�ø%5 Ø � ^�_x�vtvpHsxbdc��"ï�þ]&^�u1��[Bqrc�t � u bdt7��[�^�u � lËc76 w _atv� lR½�ß ¤dnrn�ûqHbd^��v[ ò � lY½�ß ¤ wYÞ�ß n°û�qH��t�t�bdcw��t�tvqHbrsau�� ^�_x�vtvpHsxbdc�� ò qHt �(lQà s ÷ îgn æ � lR½�ß ¤ wYÞ�ß n�}gtvp�[fj�[&pRøy�sxqHbdpHsa��{�bdsat�u ÷ î � c�sa]�c ì�í�î�ï {�qr[&q~� ë�sxu�]&[ c æ ú lR½�ß ®�n�}gtvp�bdc�[Bì�í�î�ï1^�_a��tvprs�bdc�� ò ^�_x_Wsxu^�_x_�bdc�[z=�^�pr�)tvu�sx]Dë�sx[fjv[S^�_x�vtvpHsxbdc�� pH{�u�qsauwbrsa�)[ Ü� lg¯ ® ¸ ß ¤ ] wYÞ�ß n�}gtvp^`u�ø85 Ø � × ôFc�[c�ø�o�t�brc�[&qH[&q�t�{�bdo�{�b���ø brc�[B=�^�pr�)tvu�sx](ë�sx[fjv[�^�pH[���^�ëHtvpHsxb�ø�û�t�}�û°oU^�pHsxb�ø�]&sxpr]&{�sxbdq qHsau�]~[�[&^�]�c9;:=<?>A@CB�<EDF@HGIDF@;J�KML�@EN O�PI@C<EDRQEJ�ST@HUWV-XY:7Z\[^]_\`aKbLcL�d(JRPeOFP(Q$Ogf9 UWV-XY:"[h]"_\`�[jik`mlnUWV-XY:7Z\[^]_\`�[jik`olp9 U/V-XY:"[h]_q`�[hik`�r

½ @ ½

Page 144: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

� [6^tÓ¶c�ø�o�t`bdc�[~qrsaq+saq5^�oU^�pHsxb�ø3]&sapH]&{�s�b+^�u�y)ï�þ��¹q+c�ø�o�t�bdc�[&qrsxq�sxq5^���^�ëHtvpHsxb�ø�]&sxpr]&{�sxb5tMjv[~p � [6^tÓc�ø�o�t�brc�[&qH[&q~�

ªFE}«>ERs t uwv"x=y J�N{zSJ�N x QYT�O·T�æ y}| JÆÝ v N�~»T�O Q�����QYJ7�WJ

C qFy�[&qr]~prsx��[~y�^���tY�v[ ò bdc�[�=�^�pr�)tvu�sx]�ë�sa[E�v[�^�_x�vtvpHsxbdc�� � tvp;��q\��ø���t�t�qHbdsxu���{�u�y�[&pFbdc�[ {�u�s�û}gtvpH� y�sxqHbrprsa��{�bdsxtvu ^�u�y1sxbdq3pr{�u�u�sau��7brsa�)[BqHbrprtvu��v_xøSy�[~o�[~u�y�q(tvu bdc�[�qr�)t�t�brc�u�[~qrq¶t�}\bdc�[��t�tvqRbdsxu���^�_x�vtvpHsxbdc��¥�¡ôFc�[2}gtv_x_at � sau���tv��qr[&p��`^`bdsxtvu¼}gtv_a_xt � q�y�sxpr[~]~bd_�ø(}gprtv�Èbdc�[2y�saqH]&{�qHqrsxtvu�t�}���;¼sau¥ë�[~]~bdsxtvu?��� à ��H� Í À�ÂY� Ê��I� Á�ÎwÄ<Å���Ð �/�(�zð��=�$� õ ð��(�,h��dð�Ù î?�$�(���,ñ-�,h��,ñ�ð������¢ñ��Uð ð;1�ð$�,h�ñ��F�(�n� k ���; ñ��Uð�k�q�,ñ-�E��ñ&h�ñ-�F�(� ÷�� � �q� ½�ß ¤��F� õ �Y�eñ�� ýôFc�saq¶sxq�qr{���qHbd^�u�bds�^`_a_xøS��[~bHbd[&p�bdcU^�u1brc�[¼y�sxqHbdpHsa��{�bdsat�u�q3]~tvu�qHbrpr{�],bd[~y|��ø ï�þ � c�sa]�c±^�pH[

�v{U^`pd^�u�bd[~[&y¥tvu�_xøBbdt)��[ lR½�ß ¤�� Þ�ß n�û°qr�)t�t�brc<�+ôFc�{�q ò sxb�^�o�o�[6^�pHq�brcU^`b � [�]&^�u�{�qH[�brc�[���[~bHbd[~p��tv{�u�y�[~y�u�[~qrq�t�}5bdc�[B���;�^�_a�vt�prsxbrc���brt�tv��b�^`sau ^�}ª^`qHbd[~p��v[~prqHsatvuDt�}.brc�[(=�^�pH��t�u�sa]3ë�sa[E�v[ ò^�u�y�sau�y�[&[~y¥bdc�saq>bd{�pru�q�tv{�b\bdt)��[�brc�[�]6^�qH[k Pc�t � [E�v[~p2qrtv�)[�y�[~bd^�sa_xqFu�[&[~y¥brt���[�^�y�y�pH[&qrqH[&y^�q\y�[~qr]&pHsa��[&y���[~_at � �

¡ u�[.y�[~bd^�sa_�saq¨brcU^`b qHsau�]~[�bdc�[5pr{�u�u�sau��>bdsa�)[�t`}�ì�í�î�ï�y�[~o�[~u�y�q¡tvu2bdc�["¢�{U^`u�brsxb�ø �(l�à(£`÷ îgn¥¤� [3u�[~[&y ^H�v[&pHqrsxtvu�t�}>���; � c�sx]�c � tvp���q�[E¦)]&sa[~u�bd_xø�tY�v[~p�brc�[�[~u�bdsapH[�y�tv�B^`sau �(§�¤ ½�� £ × C u�ût�brc�[&p � ^6ø¼t�}¡qr^�ø�sxu���bdc�sxq>saq+bdcU^eb � [ u�[&[&y�^���t�t�qHbHû°��ø�û ö _xbr[&pHsau��¨�v[&pHqrsxtvu�t�}+���;P�U?2t�sau��3^�u[,ö�^�],b�]~tv�)o�{�b�^ebdsat�u�t�}&© l ÙÌîjnFsxu_asau�[ @ t`}W���; � tv{�_ayb�^A��[�[,ö�o�t�u�[&u�bdsa^�_<bdsx��[�}gtvp>bdc�[�y�t`û��^�sau �(§�¤ ½`� £ ò qHt�t�{�p���t�tvqHbRû���ø�û�qd^`��o�_xsau��c�v[&pHqrsxtvu�t�}W���;�sxu�qHbr[6^�y�[&qRbdsx�B^`br[&qFbdc�[¨�`^�_x{�[ t�}© l ÙÌîgn5��ø�{�qHsau��¶^�qd^��)o�_a[�^0�v[~pd^��v[`�«ª¥tvpr[�o�pr[~]&sxqr[&_�ø ò bdc�[�^�_a�vt�prsxbrc�� y�pr^ � q\^�]&t�_a_a[~]~brsatvu)t�}{�u�s�}gtvpr�)_xø�y�sxqHbdpHsa��{�bd[&y3÷��Aq�}gprt�� �(§�¤ ½�� £ ^`u�y3{�qH[&q¡bdc�[+tv��qH[&p;��[&y(^0�v[~pd^���[ �`^�_x{�[5t�}�ÙÌî l ÷�n tvubdc�saq�qr^��)o�_a[W^�q�sxbrq�[~qHbdsx�B^ebd[�}gt�p&© l ÙÌîªn × �°b.sxq�[6^`qHø3brt2qr[~[>bdcU^eb"© l Ù îgn�sxq�bdc�[>[,ö�o�[&]~br[&yH�`^�_a{�[t�}�ÙÌî l ÷�n�}gtvpF{�u�s�}gtvpr�)_xø)]�c�t�qr[&uJ÷¬ UqHbd^�u�yU^`pry��t�{�u�y�q\tvuqd^`��o�_xsau�� l [��¹��� ü tvpHtv_a_a^�pHø à � à t�}l êv^`]$��qHtvu ò�½�óvóâá nHn\qHc�t � bdcU^eb � s�bdc?��[&pRø�c�sx�vc�o�prtv�U^`��sa_xsxb�ø�^`u7[&qRbdsa��^`br[¨©� l Ù îªn\qr^`bdsxqH}jø�sau��­� © l ÙÌîgn�è�©® l ÙÌîªn�è 4� © l ÙÌîªn�]6^�uS��[�}gt�{�u�yD{�qrsxu�� Ü� lY½�ß © l ÙÌî�n � n�qd^`��o�_x[&q~�3ôFc�{�q�sau bdc�[��t�tvqRbHû°��ø�û°qd^��)o�_asxu��H�v[&pHqrsxtvu t�}F���;�brc�[3br[&qHb2sau7_xsau�[ @ ]&^�u ��[�^�o�o�pHt�ö�sx�B^`br[&_�ø�o�[~pH}gtvpH��[~ysau Ü� lR½Mß ¤��,n�bdsx��[�qHbr[&o�q ò ^�u�y(brc�[Fpr[~qr{�_�bdsxu���^�_a��tvprs�bdc�� � sa_x_���[ àrß ¤Rû°qr�)t�t`bdc � sxbrc¶c�sa��c¶o�prtv��û^���sx_as�b�øv��ê£[\u�t�br[�brcU^`b5bdc�[ Ü� lR½�ß © l ÙÌî�n � n�brsa�)[FqHbd[~o�q+pr[�¢�{�sxpr[~y�bdt�o�[&pH}gt�pr�ùbrc�saq5[&qRbdsx�B^`brsatvu}gtvp+[6^�]�c��)[6^`qr{�pH[ ÙÌî ]~tvprpH[&qHo�tvu�y�q>[,ö�^�],bd_�ø�brt�bdc�[¯�Htvu�[,û°brsa�)[�]~tvqHb���t�} Ü� l��(lQà £ ÷ îgn��,n.}gt�p[&qRbdsx�B^`brsau���bdc�[�y�[&u�t���sxuU^`bdt�p�t�}-ï"¢�{U^`brsatvu°��� ½ � c�sa]�c � ^�qFpr[2¢�{�sapr[~y¼}gtvpW[&^�]�cy�saqRbdpHsa��{�brsatvu÷ î�qHsa��{�_�^`br[&y���ø�ï�þv�

½ @ à

Page 145: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

C u�t`bdc�[~pFy�[~bd^�sa_ � c�sa]�c��{�qRbF��[2b�^k�`[&usau�bdt�^�]&]~tv{�u�b�saq�brc�[ qr_xt � y�t � u�sxu�]&{�prpr[~ysau¼bdc�[o�pHt�]~[&qrq�t�} ö _�bd[&pHsau��)[,ö�^��)o�_x[&q�}gpHtv� ï�ð l�± ¤³²�n�bdt�qrsx��{�_�^ebd[�y�pd^ � q�}gpHtv�´�¨µ lF± ¤ ÷ îjn × �°b�sxqu�t�b�y�s�¦)]~{�_xb�bdt qr[~[Wbdc�^`b.sx}U^2��[&^�qr{�pH[\ÙÌîUqr^`bdsxq ö [~q�© l ÙÌîgn�¶é¤ ßrà ¤`bdc�[&u¶tvu(^0�v[~pd^���[F^`b.�)tvqHbàrß ¤>y�pd^ � qW}gprt��C�aµ lF± ¤³²3n+^�pr[�pr[2¢�{�sapr[~y�bdt�qHsa��{�_a^`bd[�^�y�pr^ � }gpHtv�mïÃð lF± ¤ ÷�� � nE¤�^`u�y � sxbrc�v[~pHø�c�sx�vco�prtv��^���sa_xsxb�ø�u�t)��tvpH[2bdcU^`u Ü� lR½�ß ¤dnFy�pd^ � q�^`pr[�pr[2¢�{�sapr[~y<�WôFc�{�q\bdc�saq\qr_xt � y�t � u ò� c�sx]�c¥sxqFo�pr[~]&saqH[&_�ø¼bdc�[ }ª^�],bdtvp�)Wsau�· [&�)�B^¸��� @ § ò saq\^`b\�)tvqRb Ü� lR½Mß ¤�n ×ôFc�{�q1���;Ý]6^`uL��[£brpd^�u�qr_�^ebd[&y sau�bdtµ^ � lR½Mß(¹ � ¤ � n°û�qHbd^��v[ ò � lR½�ß ¤�n�û°qr�)t�t�brc ��t�tvqHbRû���ø�ûö _�bd[&pHsau���^�_x�vtvpHsxbdc�� {�u�y�[&p�bdc�[){�u�s�}gtvpr�·y�saqRbdprsx��{�brsatvu<�(ôFc�sxq���t�tvqHbrsau���^�_x�vtvprs�bdc�� ]6^`u£��[{�qH[&y£sxu�qHbr[6^�yDt�}�ï�þ�sau brc�[¶bdtvoD_�^6ø�[&p�t�}+bdc�[)=�^`pr�)tvu�sa]¶ë�sa[E�v[k  � [(]&^�_a_�brc�saq �)t�y�s ö [~yS^�_�û�vtvpHsxbrc�� �}ºk a�¨ª¥tvpr[~tY�v[&p ò ^�q�y�saqr]~{�qrqH[&yz^���tI��[¸· [&�)�B^»��� @ §Bsa�)o�_xsa[&qFbrcU^`b [&^�]�c [,ö�[&]~{�bdsxtvut�}�ì�í�î�ï¥sxu¥��ºk � sa_x_�bd^k��[�^`b2�)tvqHb Ü� lª¯ ®0� ß ¤;�,n\bdsx��[�qRbd[&o�q&� ôFc�{�q ò ^�_xbrc�tv{��vc���º¼ <pr[�¢�{�sxpr[~q ^}ª^�],bdtvp t�} Üú lR½�ß ¤ � n2��tvpH[¶��t�tvqRbdsau���qHb�^`�v[&q�bdcU^`uSbrc�[(tvprsx�vsauU^`_.ë�sx[2�v[ ò brc�saq�y�sxqd^�y���^`u�bd^��v[¶sxq�)tvpr[2brcU^�uttí�qr[~b���ø�bdc�[�sx��o�prtY�v[~y�pH{�u�bdsx��[ t�}�ì�í�î�ïP�¨êz[�t���b�^�sxu¼bdc�[ }gtv_x_at � sau��}�¾3¿�À�ÁPÂeÀ�à Ä<Å��¾½ ! �UðE�dðw�q�c� õ ð õ ñ,ðE�¿�$�=� ² � ´ hPðE��óÀ�kÁ ò=�(�¥�jñ�� õ ��ºk k �(��Áað��k�¥�����&ò¶�2���µh��¾��ðE�ñ��Uð3h=��� k �(� õ �k�q�,ñ��¥��ñ&h�ñ��F�(� à �=�F�$����h=���Â���Dñ-� õ ð Ü� lg¯ ® ¸ ß ¤ ¸ n ×+! �UðÂ�kÁ ò=�(�¥�jñ�� õ �eh�ñ ² h�ñ����(�Â�jñ��à �¾�kÁ��&ó ² �eñÄ�Uð$�E�q�¨� õ �Hf0�(�E�jñ�ó(��� k � ² �k�E�jñ�ó¯�E���$�,h=�jñ ýê£[D]&^�uÝ^�]�c�sa[E�v[S^�ué[2��[&ué}ª^�qHbr[&p��`^�pHs�^�u�b�t�}�bdc�[D=�^�pH��tvu�sa] ë�sa[2��[ ò ^eb¼bdc�[Do�pHsa]~[7t�}{�qHsau�� �)tvpH[z]~tv��o�_a[,öÝc�ø�o�t�bdc�[~qr[~q ò ��ø {�qrsxu��µbdc�[ÆÅ ]&tv�3� ��t�tvqRbdsau��µ^�_x�vtvpHsxbdc�� sau�qRbd[&^�yt�}¶brc�[ ��t�tvqRbHû°��ø�û ö _xbr[&prsxu��Ý���; ^�_a��tvprs�bdc���� C q7u�t`bd[&y sau�ë�[~]~brsatvuÇ���TÈ�� à�ò Å�É�ÊIËÍÌsaq�^�u� l�¹76 � _atv� lY½�ß ¤�nHn�û°qHb�^`�v[ ò � l _atv� lR½�ß ¤�n ß ¤dn�û�qH��t�t�brc2��t�tvqRbHû���ø�û ö _xbd[~prsxu��F^�_x�vtvprs�bdc����<ôFc�[.qH��t�t�brc�ûu�[~qrq3t`}�Å�É&ÊIË+Ì�sa�)o�_asx[&q�brcU^`b�bdc�[Æ�Htvu�[,û°brsa�)[¶]&tvqRb¿�¥t�}>]~tv��o�{�bdsxu���brc�[)qr]6^`_asau��}ª^`]~bdt�p�}gt�p[6^`]�c3y�sxqHbdpHsa��{�bdsat�u ÷ î�saq�u�t � Ü� l��(lQà(£e÷ îgn � n æ Ü� lY½�ß ¤ � n × ÞU{�pHbrc�[&pH��tvpH[ ò bdc�[5bdsx��[.tY��[&prc�[6^�y}ª^�],bdtvp.}gtvp5qrsx��{�_a^`bdsxu�� [6^�]�c�]6^�_x_�bdtc�¨µ l�± ¤ ÷ îgn�sxq+u�t � ) æ Ü� l��(lQà £ ÷ îgnHn æ Ü� lY½�ß ¤�n × ÞUprtv�· [&�)�B^?��� @ § � [ ö u�y bdc�^`b�s�}ÎÅ É�ÊIËÍÌ saq3{�qr[&y ^`q3bdc�[B��t�tvqHbrsau���^�_a��tvprs�bdc��"sxu£bdc�[ë�sa[E�v[ òbdc�[&u1bdc�[�pH{�u�u�sxu��7bdsx��[�t�}\[6^`]�c|��t�t�qHbdsxu��7qRb�^��v[ � sa_a_+��[^`b¶�)tvqHb Ü� lª¯ ®0� ß ¤��,n × ë�sxu�]&[ � [��t�tvqRbW}gtvpW^`b>�)tvqRb � l ® � _atv� lR½�ß ¤dnrn+qRb�^���[&q>{�u�y�[&pÎÅ ]&t���� ò � [2c�^Y��[2bdc�[�}gtv_a_xt � sau���bdc�[~tvpr[~�?�¾3¿�À�ÁPÂeÀ�à Ä<Å��¾� ! �UðE�dðg�q��� õ ð õ ñ,ðE�¿�$�=� ² � ´ hPðE��óÏ�kÁ ò=�(�¥�jñÄ� õ k �(��Áað��k�¥�����&òz®Ð��ñ�ðE� õ �2���k �(� õ h�Áj��ð�h��¾��ðE��ñ��Uð�h���� k �(� õ �k�q�,ñ-�¥��ñ&h�ñ-�F�(�Ñ�(�µ�(§�¤ ½�� £ à �=�F�$�Ò��h��������µñ�� õ ð Ü� lª¯ ®�¸ ß ¤��,n ×! �Uð¨�kÁ ò=�(�¥�jñ�� õ �eh�ñ ² h�ñÄ�¨�(�a�jñÄ� à �¾�kÁ��&ó ² �eñ��Uð$�E�q�¨� õ �Hf0�(�E�jñ�ó(��� k � õ �Hf0�(�E�jñ�ó(��� k � ² �k�E�jñ�óC�E���¿�,h��jñ ýôFc�[�^�y�y�sxbrsatvuU^`_�]~sapH]&{�s�bW]~tv��o�_a[,ö�s�b�ø�]~tv�)[&qW}gpHtv�Õbrc�[�}ª^�],bWbrcU^`b>brc�[2c�ø�o�t�brc�[&qHsaqFtv{�bdo�{�b>��øÅ�É&ÊIË+Ì�saq\^¶y�[~o�bdc à ��^�ëHtvpHsxb�ø�]&sxpr]&{�sxbFtY�v[~p�s�bdqFsxu�o�{�brq&�

½ @v@

Page 146: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

ÓFE³Ô>E-Õ Ö�×Øy J�O x QRT�O xôFc�pHtv{��vc�t�{�b3brc�saq�qr[~]~bdsxtvu � [¼cU^0�v[Btvu�_xøDy�saqr]~{�qrqH[&y1{�qHsau��¥bdc�[B=�^�pH��tvu�sa]�ë�sa[E�v[)bdt7_x[6^�pHu? üCÞ }gtvpr�3{�_�^�[�{�u�y�[&p�bdc�[�{�u�sx}gt�pr� y�sxqHbdpHsa��{�bdsat�u<�\êv^�]$��qrt�u �v[&u�[~pd^�_xsaý~[&q\bdc�[3^�_a�vt�prsxbrc�� brtqr[E�v[~pd^�_�t�bdc�[~p�]~tvu�]~[&o�b�]&_�^`qrqr[~q�sxu�]&_x{�y�sau��)ô ¡ â l o�tv_�ø�u�tv��sa^�_�û°qrsxý&[�y�[&o�brc�û à ]&sapH]&{�s�bdq � c�[&pH[bdc�[�brtvo���^ebd[�]&tv�)o�{�br[&q��B^�ëHt�prsxb�ø�^�u�y¥brc�[���t�bHbdtv����^`bd[~q�]~tv��o�{�bd[�oU^�pHsxbrsa[&qdnF^�u�y�{�u�satvu�qt�}<^eö�saqRû°oU^�pr^�_a_x[&_�pr[&],b�^�u��v_a[~q�tY�v[~p��(§�¤ ½ ¤ ×6×&× ¤$Ù � £ }gtvp.]~tvu�qRb�^�u�b�Ù × ��u)[6^�]�cB]6^�qH[�tv{�p�^`o�o�prtv^�]�c]6^`u¥��[�{�qr[~y�brt�sx��o�pHtY�v[2bdc�[�pH{�u�u�sxu��¶bdsx��[2t`}.êv^�]$��qHtvu<�¹q2^�_x�vtvprs�bdc��)q&���u7pr[~]&[~u�b � tvp;��ÚWqrc�t�{�b�ø ò êv^`]$��qHtvuD^�u�y¥ô�^`��tvu l ÚWqHc�tv{�b�ø ð,ñØ�kÁ ýaò�½�óvó�ó n\c�^Y��[��vsj�v[&u�^

u�[ � �v[~prqHsatvut�}�bdc�[�=�^�pH��tvu�sa]�ë�sx[2��[2}gtvp>_a[&^�pru�sau��¶? ü·Þ {�u�y�[~pFbdc�[ {�u�sx}gtvpH� y�sxqHbrprsa��{�bdsxtvu<�ôFc�[)u�[ � ^�_a��tvprs�bdc�� y�sãí¨[~prq }gprtv� bdc�[(tvpHsa�vsxuU^�_�=�^�pH��t�u�sa](ë�sa[2��[�sxu bdcU^`b�s�b {�qH[&q�^}ª^�qRbd[~p�v[~prqHsatvu�t�}¡bdc�[2ì�í�î�ï^`_a�vtvpHsxbrc��¥�¡���)o�_asx]&sxbWsau�bdc�[&sap\^`uU^�_xø�qHsaqFsaq>brc�[ }gtv_a_xt � sau���_x[&�)�B^��

� À�Ã£Ã Ê Ä Å��}Û Ç�Ü¶Í ¿�Á¾Ý �¥ÞnßáàCâ�ãRä ÏFÐ ÆPÆPÆ�Ѷҡð,ñ ± ñ,ð��k�¨ó Ó �Y�(Áað��k� k h��®�,ñ-�F�(�%� k �2����F�E� � ð®¯� i ðE���(§�¤ ½��Y£ �k�¾�¸Áað,ñ ÷ ñ,ðw�k�¨óH�k�q�,ñ-�¥��ñ&h�ñ-�F�(�e� i ðE���(§�¤ ½`�Y£�׫! �UðE��ð+�q���k�?�kÁ ò=�(�E�jñ�� õ ì�í�î�ï à �=�F�$�DñF� ' ð$�»�(�À��� ² h�ñw�k�|ð;1á� õ�² Áað»�(�;�=�EÁað��&� l�± ¤ ÷ n¥¤�� õ ð õ ñ,ðE�¿�$�=� ² �(�;�=�EÁað����� l�± nE¤���k�q�,ñ-�E��ñ&h�ñ-�F�(�Ñ�(���=�EÁað�� ��Û"! lj÷ nE¤+�k�¾�å� i �kÁ hUð Ö|Ø § ×�� �µñÄ�Uðm�$�(�Uñ�ð;1vñ¸� k ñÄ�Uð�Ú��k� õ �(���F�Û �­ð i ð$#+ð��=�$�¯��� i �Y����ñ-�F�(�g� k ì�í�î�ïÐ à �jñÄ�¼ñÄ�Uð>ÿ���ñ��?�k�q�,ñ��¥��ñ&h�ñ��F�(� ÷ î�ò�ðE�¨ðE�;��ñ�ð�� ñyó¶ñ��Uð3ñE�Y�Y�,ñ-���&ò�kÁ ò=�(�¥�jñÄ� õ ñF� ' ð$� Ü� lª¯ ® � l-� (l�à £ ÷ îjnrn � n ñ-� õ ð��,ñ�ð ² �¨�k�¾�À�k�¾� à �jñ�� ² �¿�tñ$�âñ���Á^�jñ�óH��ñÁað��(�,ñ ½ æzÖ�eh�ñ ² h�ñÄ��� ² �k�¥�jñ�ó k h=�®�,ñ��F�(� à �=�F�$�°�q�Â� lR½Mßràwæ ú lR½�ß ®enHnE��� ²�² �¿�21¼� õ ��ñ³�(� ñ�� ± h=�¾��ðE� ÷ î ×ôFc�[�=�^�pr�)tvu�sx]�ë�sa[E�v[a�`^�prsa^�u�b�y�[&qr]~prsx��[~y�sau l Ú�qrc�tv{�b�ø ð,ñ��kÁ ýaò ½�óvóvó nFpH{�u�q\bdc�[�tvpHsa�vsxuU^�_

� lY½�ß ¤ � Þ�ß n°û�qH��t�t�bdc�ï�þ���t�tvqHbrsau��(^�_x�vtvpHsxbdc��Õ}gtvp Ü� l ® � n>qRb�^��v[~q ò {�qHsau��(bdc�[�u�[ � ì�í�î�ïÐ <^�_x�vt`ûprs�bdc��»^`q�bdc�[ � [6^k�)_a[&^�pru�[&p ò bdt�tv��bd^�sauB^�uBtY�v[~pd^�_x_Ppr{�u�u�sau���bdsa�)[�t�} [&qHqr[~u�brs�^�_x_xø Ü� lª¯ ®0, ß ¤;,~n ×Ú�ø)sau�qHbd[&^�yB{�qrsau���brc�[ � l _at�� lR½Mß ¤�n ßI¹ � n�û�qRb�^���[ ò � l _xtv� lY½�ß ¤�n ß ¤�n°û�qr�)t�t�bdc(��t�tvqRbdsxu��3^`_a�vtvpHsxbrc��Å ]&tv�3� ^`qFsau¥ë�[&],bdsxtvu���� á � @ ò � [�tv��bd^�sau¼brc�[ }gtv_a_xt � sxu��3pH[&qH{�_xb ò � c�sa]�c�saq>brc�[ }ª^�qHbr[&qHbÍ��u�t � u^�_x�vtvprs�bdc��Ý}gtvp�_x[6^�pHu�sau���? ü·Þ�{�u�y�[&p�brc�[W{�u�sx}gtvpH�Ly�saqRbdpHsa��{�brsatvu�{�qHsau�����[~����[~prqHc�saoc¢�{�[&prsx[&q2�¾3¿�À�ÁPÂeÀ�à Ä<Å��¾ç ! �UðE�dðg�q��� õ ð õ ñ,ðE�¿�$�=� ² � ´ hPðE��óÏ�kÁ ò=�(�¥�jñÄ� õ k �(��Áað��k�¥�����&òz®Ð��ñ�ðE� õ �2���k �(� õ h�Áj��ðÀ� i ðE���(§�¤ ½�� £ h=�¾��ðE�3ñ��Uð¶h=��� k �(� õ �k�q�,ñ-�¥��ñ&h�ñ-�F�(� à �=�F�$�o��h��������zñ-� õ ð Ü� lª¯ ®0, ß ¤ � n ×! �Uð¨�kÁ ò=�(�¥�jñ�� õ �eh�ñ ² h�ñÄ�¨�(�a�jñÄ� à �¾�kÁ��&ó ² �eñ��Uð$�E�q�¨� õ �Hf0�(�E�jñ�ó(��� k � õ �Hf0�(�E�jñ�ó(��� k � ² �k�E�jñ�óC�E���¿�,h��jñ ý

½ @ È

Page 147: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

è é êìë íïîñð ò

óõô ö÷öìíåé ø ö÷öúùoíåû"ü ý êþü ÿ � îñêìð�ü û"ü ý

� û7íåé ê��"û�� ûWö�� ù � ö û�ùgî

��u bdc�saqB]�c�^�o�bd[~p � [ ]&tv����sau�[¥say�[&^�q�}gpHtv� WcU^�o�br[&prq�z^�u�y{�zbrt tv��bd^�sau|u�tvsaqH[,û­bdtv_x[&pd^`u�b��t�tvqRbdsxu��`û���^�qr[~y¥^`_a�vtvpHsxbrc���q+}gtvp\â C é_a[&^�pru�sxu��¶_asxu�[6^`pWbdc�pH[&qHc�tv_ay}g{�u�],bdsxtvu�q&�

ª¥t�brs��`^`bd[~y ��ø�tv{�p2sauÐ�v[~qHbrsa��^`brsatvu�t�}5��t�tvqRbdsau��¼^�_x�vtvprs�bdc��)q�sau�WcU^`o�bd[~p�� ò � [� UprqRb y�[�ûqr]~prsx��[+^Fu�[ � ��t�tvqHbrsau���^�_x�vtvpHsxbdc��Ý]&^�_a_x[&ycº�����������Å����������· sq��[5���)oU^��v_xs�^�ý~ý&t��¹q ^�_x�vtvpHsxbdc��é}gt�pcU^�pHy�û�]~tvpr[3qr[~b�]~tvu�qRbdpr{�]~bdsxtvu ò º��������������������7�v[&u�[~pd^`br[&q tvu�_xø¥qr�)t�t�brc7y�sxqHbdpHsa��{�bdsat�u�q � c�sa]�cy�tu�t�b�^`qrqrsx�vu bdt�t��{�]�c � [~sa�vc�b bdt�^�u�ø qHsau���_a[�[�ö�^`��o�_x[k ¡c�t � [E�v[~p�º�����������Å��������Ssaq qHsa�`ûu�s U]6^�u�br_xø }ª^�qRbd[&p3^�u�y �)tvpH["!�[,ö�sa��_x[�brcU^�uzbrc�[B��t�tvqHbrsau��¥^�_a��tvprs�bdc�� � c�sa]�c sxq�sx��o�_xsa]~sxb sxu���)oU^��v_xs�^�ý~ý&t��¹qF^�_x�vtvprs�bdc����

#£[¼brc�[&u {�qH[�º��$��������Å%�������1bdt ]~tvu�qHbrpr{�],b(�B^`_asa]~satv{�q�u�t�saqr[�bdtv_x[&pd^`u�bc�v[~prqrsxtvu�q(t�}\bdc�[â C ���t�y�[~_$&�û°u�tvpr� _asxu�[6^`p�bdc�pr[&qHc�tv_xyS_x[6^�pHu�sau���^�_a��tvprs�bdc��)q�y�[&qr]~prsx��[~ySsxu'WcU^�o�br[&p(�����u^¼u�tvsxqr[�û°}gpr[~[�qr[,brbrsau��¼brc�[¶qd^��)o�_x[3]&t���o�_x[,ö�sxb�ø¥��tv{�u�y�q t`}+brc�[¶u�[ � ^�_x�vtvpHsxbdc���q\��^`bd]�c bdc�[��tv{�u�y�qt�}�bdc�[z^`_a�vtvpHsxbrc���q��vs���[&uÝsau)WcU^�o�bd[&p*��»=�t � [E�v[&p ò bdc�[Du�[ � ^�_a��tvprs�bdc��)qB]&^�ubdt�_a[&pr^`bd[�pr[~_�^`brs��v[~_xø�c�sx�vc7_a[E�v[~_aq2t�}5��^�_xsa]&sxtv{�q\u�tvsxqr[ ò ^Bo�pHtvo�[~pHb�ø � c�sa]�c7brc�[¶^�_x�vtvprs�bdc��)q\t�}WcU^�o�bd[&p+ y�tSu�t�b)^�o�o�[6^`p(brtDqrcU^�pH[��,#£[qHc�t � bdc�^`b(bdc�[u�tvsxqr[�bdt�_a[&pr^�u�]~[^�u�y±qd^��)o�_a[]&t���o�_x[,ö�sxb�ø t`} bdc�[~qr[7u�[ � â C ^�_a��tvprs�bdc��)q¶]&_xtvqr[~_xø ]&t�prpr[~qro�tvu�y brt%��u�t � uw��tv{�u�y�q�}gt�pbdc�[�t�u�_asxu�[-&�û°u�tvpr� ^�_x�vtvprs�bdc��)q3t`}/. prtY�v[ ò · s�brbd_x[&qRbdtvu�[^`u�y|ë�]�c�{�{�pr��^�u�q l .�pHtI��[ ð,ñÂ�kÁ 0 ò½�óvó vn�^�u�y1. [&u�bdsx_a[¶^�u�yÆ· sxbHbd_x[&qHbrtvu�[ l .�[~u�bdsa_x[(^�u�y�· sxbHbd_a[~qHbrtvu�[ ò.½6óvóvó n�� C q�qro�[&]~s�^�_.]&^�qr[~qt�}\tv{�p3u�[ � ^�_x�vtvpHsxbdc���q � [�tv��bd^�sauz_asau�[6^�p�bdc�pH[&qHc�tv_ay1_x[6^�pHu�sau���^�_x�vtvpHsxbdc���q � c�sx]�c1�B^`br]�cbdc�[�qr^��)o�_a[�]~tv��o�_a[,ö�s�b�ø{�k�¾���B^�_xsa]~satv{�q�u�tvsxqr[�bdtv_x[&pr^�u�]&[�t`}�bdc�[�t�u�_asxu�[�â�[~pr]~[&o�brprtvuw^`u�y#ésau�u�t � ^�_x�vtvprs�bdc��)q&�

½ @�2

Page 148: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

35476 8:9<;�=?>5@BABCD;FEG>+9

C u�ø¥pr[&^�_asxqHbdsx]���t�y�[&_�t�}�_a[6^`pru�sxu���}gpHtv� [,ö�^��)o�_a[~q���{�qHb ^`y�y�pr[~qrq�brc�[�sxqrqr{�[3t�}.u�tvsaqRø¥y�^`b�^����u ½�ó � 2IH ^�_xs�^�u�b5[�ö�br[&u�y�[&y�bdc�[�tvprsx�vsau�^�_Uâ C _x[6^�pHu�sau����)t�y�[~_Ubdt�^`_a_at � }gtvp+brc�[�o�tvqHqrsa��sa_as�b�øt�}?JÀ�kÁ^�F�E�F�ehÐ�w�®�(�q��ð,��ôFc�saq�saq>^ � tvpHqHbRû�]6^`qr[ �)t�y�[~_�t`}�[&prpHtvprqWsau � c�sa]�c�qrt���[�}gpd^�],bdsat�u¼t�} bdc�[_�^`��[~_a[&y£[,ö�^��)o�_a[~q��vs���[&u brtD^7_a[6^`pru�sxu�� ^�_x�vtvprs�bdc�����^6ø£��[¼]~tvprpH{�o�bd[~y ��ø ^�u ^�y���[&pHqd^�pRø� c�t�]6^�uS�)t�y�s�}jø���t�brcS[�ö�^���o�_a[¶o�tvsau�bdq�^`u�yz_a^���[&_aq�sauS^�uS^�pr��sxbdpr^�pHø¥}ª^�qrc�satvu l ^�y�[,b�^�sx_a[~yy�[~qr]&pHsao�brsatvuzt�}>brc�[B�)t�y�[&_5saq��vs���[&u sxu1ë�[~]~brsatvu ó � @ n��ôFc�[�}gpH[�¢�{�[&u�]~ø t`}Fqr{�]�c1]~tvprpH{�o�br[&y[,ö�^��)o�_x[&q\saqÎ��u�t � u7^`q>bdc�["JÀ�kÁ^�F�E�F�ehÐ�a�®�(�q��ða�;��ñYð,�

· [6^�pHu�sau��¶sxu¼bdc�[ o�pr[&qH[&u�]~[�t`}���^�_xsa]&sxtv{�q+u�tvsxqr[ saq���u�t � ubdt¶��[�¢�{�sxbr[ y�s�¦)]~{�_xb&�FK [6^�pHu�q^�u�y¨· s l K�[&^�pru�q�^�u�y¨· s ò�½�óvó @ n�cU^0��[+qHc�t � u�brcU^`b }gtvp¨�B^�u�ø�]&tvu�]~[&o�b¡]~_�^�qHqr[&q s�b saq¨sau�}gtvpH�B^ebdsat�u�ûbdc�[&tvpH[~bdsx]6^�_x_xø¶sx��o�tvqHqrsa��_a[>bdt�_a[&^�pru)bdt�^`]&]&{�pd^�],øB¤�sx}¨brc�[���^�_xsa]&sxtv{�q.u�t�saqr[�pd^ebd[�[,ö�]&[~[&y�qML­ Þ L ×��u|}ª^`]~b ò }gtvp¶��^�u�ø sau�bd[~pr[&qRbdsxu��z]~tvu�]~[&o�b�]&_a^�qrqH[&q l qr{�]�cµ^�q(bdc�[�]&_a^�qrq(t�}�_asau�[6^�p�brc�pr[~qrc�tv_xy}g{�u�],bdsxtvu�q�n ò bdc�[���[~qHb�[2¦)]&sx[&u�bW^�_a�vt�prsxbrc��)q���u�t � u¼]6^�u)tvu�_�ø�bdt�_a[&pr^`bd[\��^�_asx]&sxtv{�q�u�tvsaqH[�pr^`bd[~q� c�sx]�c£^�pH[(qrsa��u�sN U]&^�u�bd_�ø7_at � [&p bdc�^�uSbrc�saq ��[&u�[~pd^�_5{�o�o�[&p���tv{�u�y¨�+O2[~qro�s�bd[¶bdc�[&qr[)y�sj¦�]~{�_�ûbdsx[&q ò brc�[�o�pr^�]~brsa]6^`_Wsa�)o�tvpHbd^�u�]&[�t�}���[~sau��7^���_a[�bdt ]&t�o�[ � s�bdc1u�tvsxqHø£yU^`b�^7c�^�q¶_a[~y1�B^�u�øpr[~qr[&^�pr]�c�[~prqFbdt�qHbr{�y�ø�â C _a[6^`pru�sxu��3sxu�bdc�[2o�pr[~qr[~u�]&[ t�}���^�_xsa]&sxtv{�q.u�t�saqr[A �qH[&[2[`�A��� l C qH_�^��^�u�y,O2[~]6^`br{�p ò\½�ó�ó ��  C {�[&p ò\½�óvó �  C {�[&p(^�u�yPW[~qd^eû³ÚWs�^�u�]�c�s òF½�ó�ó �� QW[&qd^�û�ÚWsa^�u�]�c�s�ð,ñ¨�kÁ 0 ò½�óvó�ó  %O2[&]&^`bd{�p ò<½�ó�óSR  �ª�^`u�qrtv{�p�^`u�y�â�^�pru�^�q ò ½�ó�ó ��n,���u�bdc�sxq>]�cU^�o�br[&p � [�y�[&qH]&prsx��[�^�u�[ � �TJC�Y�eñÄ�5UE�Y�Y�,ñ�����V ^�_x�vtvprs�bdc��»^`u�y�{�qr[ brc�saq>qH��t�t�brc

��t�tvqRbdsxu��2^�_x�vtvpHsxbdc�� bdt2]~tvu�qRbdpr{�]~b+^�}ª^��)sa_�ø t�}�â�WX ^`_a�vtvpHsxbrc���q }gtvp�_a[&^�pru�sxu���_xsau�[&^�p bdc�pH[&qHc�ûtv_xy}g{�u�]~brsatvu�qFsxu�bdc�[�o�pH[&qr[~u�]&[�t�}��B^�_xsa]~satv{�q+u�tvsxqr[`�

YDZ\[]Z\[ ^ T y Q³� v&y QRT�O`_�T�NÑ�+~»T\T y}|ba T�T x=y QROdc¡ {�p+�U^�qrsx]�^`o�o�prtv^�]�c¼saq ¢�{�sxbr[�qHsa�)o�_a[ ò ^�qWsx_a_x{�qHbrpd^`br[&y(��ø)brc�[�}gt�_a_at � sxu���[,ö�^��)o�_a[`�?Wtvu�qHsay�[~p^D_a[6^`pru�sxu��Dqr]&[~uU^�pHsatSsau � c�sa]�c � [¥cU^0�v[¥^ � [6^A�±_x[6^�pHu�sau��z^`_a�vtvpHsxbrc�� � � c�sa]�c±b�^k�`[&q¼^`qsau�o�{�b�^e Uu�s�bd[�qd^��)o�_a[gf t�}Fh�_a^���[&_a[~y¥[�ö�^���o�_a[&q~�iW�_a��tvprs�bdc�� � sxq+��u�t � u7bdt�cU^0�v[�qHtv�)[bdt�_a[&pr^�u�]~[¼bdtD�B^`_asa]~satv{�q�u�t�saqr[A >qro�[&]~sN U]&^�_a_�ø ò � sxq¶�v{U^�pr^�u�bd[~[&y bdtD�v[&u�[&pd^ebd[�^ c�ø�o�t�bdc�[&qrsxq� sxbrc u�tvu�u�[~�v_asx�vsa��_a[�^�y��`^�u�b�^���[�o�prtY��say�[~y1bdcU^`b¶brc�[¼}gpr[2¢�{�[&u�],ø t�}�u�tvsaqRø£[�ö�^���o�_a[&q¶sxu1sxbrqqd^`��o�_x[(saq�^`b���t�qHb ½ §�j × #z[ � tv{�_xyz_xsq��[¶brt¥_x[6^�pHuSbrt¥c�sx�vcz^�]&]~{�pd^�],øDsauDbdc�[)o�pH[&qr[~u�]&[�t�}��^�_asx]&sat�{�qWu�t�saqr[�^eb�^)pr^`bd[ t�} ½ j¼�ôFc�[¼tv�Ð��sat�{�q3^`o�o�prtv^�]�c sauSbdc�sxq�qH[~bHbdsau���saq�bdt¥{�qr[B^¥��t�t�qHbdsxu��¥^�_a�vt�prsxbrc�� ò � c�sa]�c � sx_a_

�v[~u�[&pr^`bd[qrt���[¼qH[�¢�{�[~u�]&[¥t�}�y�saqRbdprsx��{�brsatvu�q ÷ ­ ¤ ÷ � ¤ ×6×6× tY��[&p+f × ôFc�saq(^`o�o�prtv^�]�c ]6^�u1}ª^�sx_ ò

½7RSk

Page 149: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

bdc�tv{��vc ò sx}�bdc�[���t�tvqRbdsau���^`_a�vtvpHsxbrc��ù�v[~u�[&pr^`bd[~q>y�sxqHbdpHsa��{�bdsat�u�q � c�sa]�cB^�pr[Ø�v[~pHø�q���[ � [~y�}gprtv�bdc�[){�u�sx}gtvpH� y�saqHbrprsx��{�bdsxtvu tvulf� <sx}+y�sxqHbrprsa��{�bdsxtvu ÷(m ^`qrqrsx�vu�q o�pHtv�U^���sa_as�bdsx[&q�^�q�_�^�pH�v[¶^�q �onpbdt(sxu�y�s���sxy�{U^�_�o�t�sau�bdqFsxuqf&¤�}gtvpFsau�qRb�^�u�]&[ ò brc�[&u�brc�[ }gpr[�¢�{�[~u�]~øt`}�u�t�saqHøB[,ö�^��)o�_a[~qF}gtvp � sxuqHbd^��v[Fÿ<]~tv{�_xy¶��[F^`q.c�sa�vc�^�q]rk§�j × #éc�^`b � [Fu�[&[~y�sxu�qHbr[6^�y¶sxq.^c�TJC�Y�eñÄ����t�tvqHbrsau�� ^`_a�vtvpHsxbrc��� c�sx]�c¼tvu�_�ø(]&tvu�qRbdpH{�]~brq>y�saqRbdpHsa��{�brsatvu�q ÷(m tY��[&pDf � c�sa]�c¼u�[E�v[~pF^�qrqHsa�vu � [~sa�vc�b��vpr[&^`bd[~p�bdc�^�u­ np brt�^�u�ø7qrsxu��v_x[¶[,ö�^��)o�_a[`�¸Ú�ø7{�qrsau���qH{�]�c ^�qH��t�t�bdc ��t�tvqHbr[&p � [B^�pH[(^�qrqH{�pr[~ySbrcU^`b�bdc�[� [6^k��_a[&^�pru�[~p � sx_a_<}g{�u�]~bdsxtvu�qH{�]&]~[&qHqH}g{�_x_xø�^`b�[&^�]�c qHbd^��v[ ò qHtBbdc�[�tY�v[&pr^�_a_¡��t�tvqRbdsxu���o�prt�]&[&qHq� sa_x_ � tvp���]~tvprpH[&],bd_xø��

#éc�sa_x[Wbdc�[Fqr[~bHbdsxu���y�[&qH]&prsx��[~y)^���tY��[�sxq.^�pRbdsN �]&s�^`_ ò � [\u�t�br[Fc�[&pH[>bdcU^`b5sxu�y�sxpr[&],b.[&�)o�sxprsa]&^�_[2��sxy�[&u�]~[2cU^�qW��[~[&u��vsj�v[&uBqr{�o�o�tvpRbdsau��3bdc�[�qr�)t�t`bdc¼��t�t�qHbdsxu���^�o�o�pHt�^�]�c¼}gtvp�u�tvsaqRø)qr[~bHbdsxu��vq&��°b(saq � [~_a_���u�t � u l O2sa[,brbd[~prsx]�c ò rk§¼§¼§� .ë�]�cU^`o�sapH[ ò�½�óvóvó �Un�bdcU^eb¶]&tv�)�)tvu�_xøS{�qH[&y±��t�tvqRbdsau��^�_x�vtvprs�bdc��)qFqr{�]�cD^�qts�u�v¼Å%��������]&^�u7o�[&pR}gtvpr� o�t�tvpH_xø�t�u7u�tvsxqHøyU^`bd^��XO2sa[,brbd[~prsx]�c�cU^�q�qr{��eû�v[~qHbd[~y1bdcU^`b�bdc�sxq�o�t�tvp�o�[~pH}gtvpH�B^`u�]&[�saq3y�{�[¼bdtws�u�v¼Å����������Aq(bd[~u�y�[~u�]~ø brt7�v[~u�[&pr^`bd[C��[&pRøq��`[ � [&y7y�saqHbrprsx��{�bdsxtvu�q � c�sa]�c�o�{�b�^¶�vpH[6^`b\y�[&^�_<t�} � [&sa��c�b\t�u¥^�}g[ � u�t�saqHøB[,ö�^��)o�_a[~q&�5ôFc�sxqtY�v[~p � [&sa��c�brsau��)t�}�u�tvsxqHø�[�ö�^`��o�_x[&qF]&^�u�u�t�b�t�]&]~{�p\{�u�y�[~p�^(qr�)t�t�bdc���t�t�qHbdsxu��¶pr[~�vsa�)[&u¨���u1ë�[~]~brsatvu ó �xr � [)�vs���[�^qrsx��o�_a[�u�[ � ��t�tvqRbdsxu��^�_a��tvprs�bdc�� ò º�����������Å�������� ò � c�sx]�czsxq

�v{U^`pd^�u�bd[~[&y±bdtS�v[&u�[~pd^`br[¥t�u�_xø1qH��t�t�brc±y�saqHbrprsx��{�bdsxtvu�q(^�q)y�[~qr]&pHsa��[&yµ^���tY��[��µ��u±}ª^�]~b ò � [qrc�t � sau¼ë�[&],bdsxtvu ó � È��yr�bdcU^`b�bdc�[�y�saqRbdprsx��{�brsatvu�q+�v[~u�[&pr^`bd[~y���ø�º��$��������Å%��������^�pr[�tvo�brsa��^�_a_�øqr�)t�t�bdc<�

º��$��������Å%������� sxqu�t�bbdc�[� UpHqHb¥��t�tvqHbrsau��±^�_x�vtvpHsxbdc�� � c�sx]�c ^ebrbd[~��o�bdqbdt±^0�vtvsxy bdc�[q��`[ � [&ySy�saqRbdpHsa��{�brsatvu�q�t�}?s�u�v¼Å�������������uD^�y�y�s�bdsat�u�brtBbdc�[3��t�tvqHbrsau��B^�_a��tvprs�bdc��õsx��o�_asa]~sxb\sxu���)oU^��v_xs�^�ý~ý&t��¹q�cU^�pry�û�]&t�pr[BqH[~b¶]~tvu�qRbdpr{�]~bdsxtvu ^�_x�vtvprs�bdc�� l ���)oU^��v_xs�^�ý~ý&t ò5½�óvó 2 n ò ^���t�tvqRbdsau��^�_x�vtvprs�bdc�� � sxbrc�^�qH��t�t�brc�u�[&qHq��v{�^�pd^�u�br[&[WqHsa�)sa_a^�p¨bdt�bdcU^eb�t�}�º�����������Å���������cU^�q���[~[&u���s��v[~u��ø�O2tv�)sau���t¼^�u�yB#£^`b�^`uU^���[ l O2tv�)sau��vt�^�u�yB#£^`b�^`uU^���[ ò rk§k§¼§�n,�(#£[¶qr^ � sau'Wc�^�o�bd[~pa�bdc�^`bDzUpr[~{�u�y{�¹qÎÅ Wtv��� ��t�tvqHbrsau��3^�_a�vt�prsxbrc�� l zPpH[&{�u�y ò¨½�ó�ó rvn+��[&u�[~pd^`br[&q>pH[&_�^ebds���[&_�ø�qH��t�t�brcy�sxqHbdpHsa��{�bdsat�u�q ò ^�u�yw��tvpH[7pr[~]&[~u�br_xø|zPpH[&{�u�y l zUpr[~{�u�y ò�½�óvóvó n)cU^�q^`_aqrt£y�[&qH]&prsx��[~yÝ^ u�[ ���t�tvqRbdsxu��7^�_x�vtvpHsxbdc�� � c�sa]�c {�qH[&q(^��)tvpH[B�)t�y�[~pd^`br[ � [&sx�vc�bdsxu��7qr]�c�[&�)[¼bdcU^�u}s�u�v¼Å������������uµë�[&],bdsat�u ó �xr�� R � [�qrc�t � bdc�^`b(sau|^�y�y�s�bdsxtvu bdtS�v[&u�[&pd^ebdsau��SqH��t�t�brc|y�sxqHbrprsa��{�bdsxtvu�q ò tv{�pº�����������Å��������£^�_a�vt�prsxbrc�� cU^�q�qr[2��[&pr^�_5t�bdc�[~p�y�[&qrsxpd^���_a[¶o�pHtvo�[~pHbrsa[&q ò qH{�]�c ^`q�]~tvu�qHbrpr{�],bdsxu��^z_a^�pr��[¥��^�pr��sau} UuU^�_�c�ø�o�t�brc�[&qHsaqB^�u�y {�qHsau��zpr[&^�_�ûF��^`_a{�[~y � [6^k�±c�ø�o�t�brc�[&qH[&q ò � c�sx]�cw^�pH[[&qHqr[~u�brs�^�_P}gtvp.bdc�[�u�t�saqHø�_asau�[6^�p�brc�pr[~qrc�tv_xyB_x[6^�pHu�sau���^�o�o�_asx]6^`brsatvu¶t�} ë�[&],bdsxtvu ó � R �~#£[�y�saqH]&{�qHqbdc�[�pH[&_�^ebdsat�u�qrc�sxo���[,b � [~[&uÀº�����������Å��������¶^`u�y�brc�[>^`_a�vtvpHsxbrc���q¨t�}%O2tv�)sau��vt\^�u�y(# ^`bd^�uU^���[ òzUpr[~{�u�y ò ^�u�y¥���)oU^��v_xs�^�ý~ý&t�sxu¥ë�[&],bdsat�u ó �yr�� È��

½7R

Page 150: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

YDZ\[]Z�º �"�/vF���/��� c �e�\�d�¬vF�`�%|d�%�¬x�|d����� u/�/� � y������wx����³y}| ^ v?��� � �����wx� ����x��

#l�������������ͺ�����������Å��������e���N�S�S��� � ��¡¢�N£(ë��G¤G� �N�S£ ó�¥¦R � �/¤G�S£���� ����¤G�]�t§¨��¡e�©��ø/��§�â$WX+ûª¡+��«��¬�¡+���©�N¤¬�©�����D£��S�N���g���S�©�G� ��£��t���©���S��� � ��¡e��§­�S�t�N�®���\£��©£��+�N�©£��¬���D� ���\�¬�����S�©«q§­��£�¤G���©�S£�� ¥ ¡ ���¯£��G°§¨��¡e�©��ø5��§����©�S�S�\�N����¡+�X�©��¢����N� �±�\�©¡e�©�©���D� �²� ����§¨��¡+�N�xø5¤¬��£��\������¤T� �¬«l�©£'��%��³�� �G�- ò �,ö�¤¬�G³��� �����d£��7°¢°������\�¸º�����������Å��������B�N£��\���®��«w��§Fs�u�v¼Å��������w���¯�����-´����S�\���©£��e¤¬�S¡e³��S£��¬£�� ¥ #l�������°��¬«+�©£+��%��³�� �¬��t� �����?§­�S���©�N£��®�����xøg���G³%������´��©�(µ­£��S�N���,û¶§­���G�7·Q«%��� � ò � ���Ds�u�v¼Å���������ûo´%�����¬«���N�S�S��� � ��¡e�(�%�0�S�²� ��¡+³��N�<¤¬��¡+³��N�,ö��N��ø'´��S��£�«���°i���©¤¸�¹���\�²�¬���\�¬£�� �©���©��ø1�N«��¬£����©¤®���]���w� ���S�\���§�� ���+�S£��©�N£��d&�ûª£��S��¡`�N�©£��¬���d� �����¬�\���S�N«1�©�¬����£��©£������N�S�S��� � ��¡e�d��§i.I�\�I��� ò ·:� �����©�¬��� �S£��+��£�«ë�¤¸�������\¡º��£��(µ¶./���Y�S�Bð,ñ«�kÁ 0 ò$»�óvó S· ¥ Wt�D«��¬��¤G���N´$�G«��N£5��%��³�� �¬�X ò � ���¬���I�S£��©�©£��d���N�S�S��� � ��¡e��©£�¤¬�©��«��º���-��³��¬¤G�¼���]¤®�����¬�²µ½& æ r���£�«q& æ�¾ ·d� ���+°��¬�©� û³��£��7°i£¿�S£��©�©£��BâÀ�¬��¤G�¬³������S£,��£�«#Á�©£�£��7°¢���N�S�S��� � ��¡e� ¥

.I�G£�� �©�N�D��£�«H·Â� ��� �N�¬��� �S£��/µ¶./�¬£�� �N�©����£�«À·:� �����©�¬��� �S£�� ò�»�ó�óvó ·À���Y���i�S�����¬£"¡+�N�\� �k����´��S��£�«��§­�S�Q�������S£��©�©£��~&�ûª£��S��¡Ã���N�S�S��� � ��¡e�:°i���¬£"����£e��£±�,ö���¡e³��©�G�Q°i���©¤¸�e���\�D£����F�N�©£��¬������øI���G³%��� ��û´��N� ò � ���������¬£��G� ���N�aý¬�N£��(³����E���N�S����´$�S��£�«�����£º£��S�N���X� �S�N�¬� ��£�¤¬�i§­�S��â~�¬�\¤¬�G³�� �\�S£'µÄz%���G��£�«²��£�«ë�¤¸�%��³��©�\� òÅ»6óvó ��·F��£�«+#Á�©£�£��7°Ãµ-·:� �����©�¬��� �S£�� ò�»�óvó�» · ¥ WÆ«�� �®°i´%��¤$�º��§Â������s�u�v¼Å���������ûo´%�����¬«â$WX+ûª¡+��«��G��&�ûª£��S��¡Ç���N�S�S�\�N� ��¡+�?��§~��%��³�� �G�Xg�©�]� �%����� ���,ø<«���£����i��³�³��®����� �±�\��¤¬¤G�¬�G«*�N£� ���g³��\�¬�\�¬£�¤G����§F¡+���©�N¤¬�N�S����£����©���/§­�S�i� ���g�\�®���\�S£��t«��¬��¤G���N´$�G«B��´��I��� ¥ #l�g�\���7°È�©£�ë��¬¤T� �N�S£ó�¥ È+���%���X§­�S�t���©�3�����©���¬�/r)èP&Dè ¾ ¤��S���¯£��T° â$WXÉ���N�S�S��� � ��¡e��°i���N¤¸�w������º��$��������Å%�������¡+��� ¤¸�}´���� �¿�����²� ��¡+³��N�<¤¬��¡+³��N�,ö��N��ø �k�¾�º� ���²¡º���N�©¤G�©�S���I£����©���+� �S�N�¬� ��£�¤¬�²��§D�����²�S£��N�©£��&�ûª£��S��¡ ���N�S�S�\�N� ��¡+� ¥ ¡ ���<¤G�S£��\������¤T� �N�S£|� �����²³��\�Y���N«��¬�º¡º���N�©¤G�©�S���"£����©���*� �S�N�¬� ��£��¼â$WX��£%���©�S�S���¬�D��§�â~�G��¤¬�G³�� �\�S£w��£�«�#Á�N£�£��7°Êµ¨��£�«5¡+��£�ø²��� ���¬�X���©���S��� � ��¡e�����i°��¬�N�Ë· ¥

354ÍÌ Î5Ï >²>±;�Ð ÑÒ>²>eÓ~;�E®9wÔ Õ�EG;FÐ ÖF׺ØdØ�Ù¯ÚIÛ-ØdØIÜ�Ù

Ýo£�� ���N�/���¬¤T� �N�S£B°��e«��¬�\¤¬�\�©´��+��£�«'��£%����ø�ýG��� ���»º�����������Å��������l���N�S�S��� � ��¡ ¥ #1�"������°Ã� �%���º�����������Å����������%���i���E�S�G� ���À�����T§­���³��\�S³$�G�\���©�¬�2�

Þ � �/�S£���øq¤G�S£��\������¤T� �(��¡e����� �B«��N�\� �\�©´���� �©��£��t°i���N¤¸�'«���£����I³����/�����²¡���¤¸�'°��¬�N�S���I�S£��£�ø²�\�©£����©�/�,ö���¡e³��N�k 

Þ � �D¤®��£�´��(�����¬«�� �"�S�G£��¬����� �(�±�¼���\�S�/¡+�����S�N£< %£%���Å��ø�³���� ���¬���N�� Þ � �-¤¬��£¿´��º���\�¬«}°i�N���¿�*°��®�k�l�©�¬����£��N£��*���©�S�����N����¡�°i���N¤¸�¿�S����³����������¬����ûF�����©���G«1��ø�û

»7R �

Page 151: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

³���� ���¬���G�� Þ � �g¤G�S£���� ����¤G� �-�N� �/ %£�������ø�³���� ���G���N�-´�ø�� ���\�¬�����S�©«��N£��*��°��¬�©�������¬«¿�\��¡ß��§�°��®�k�'��ø�û

³���� ���¬���G� ¥Wt�N�F��§������¬�\�<³�����³$�G�\� �N�¬�g�����e�¬�\���G£����¼���?§­�S�I� ���e£��S�N�Hø'�©�N£��®���d� ���\�¬�����S�©«¿�N�®���\£��©£���³����S´��N�¬¡°��g¤G�S£����N«��¬�i�N£�ë��¬¤T� �©��£ ó�¥¦R�¥

YDZoà�Z\[ ál�%�À��� ~ ���wvF�����¬xzQ�N���\�?°��¯�\�¬¤¬���©������¡+�i���\�G§­����£���� ��� �N�S£±§­���S¡Ê�����³�� �G����g��£�«»� ¥ WâJBð��(�Tã=��ð��S£<�/ %£�� � �i���T��©���(§­��£�¤G���©�S£<ä ��f¿å æ §�¤ » çÄç�#1�X°i���N���eèéäÇè�� �-«��¬£������Iê<ë®ì7í�äîµ­ï$·���£�«C©Fµ¶äÉ·F� �-«��¬£������èðäÊèðñ�èéfdèxç�.I�j�S�G£w�"¡+�¬�������\��ä ¤������¬�\�-�N�¯�e£%������� ���Â�N£�«���¤G�¬«*«��©��� ���N´������©�S£ ÷Â� «��T %£��¬«q´�ø÷Â� µ­ï$· æ äòµ­ï$·�ñ�èðäÇèxç�ôD���©�i«��G �£��N���©�S£¼ø��©�G�©«���H�~ó¬ô�õ ��ö �I�­÷�øúù{û ½ � µ ÷�� ·Fè ­ü � ü æ ­ý�þ �/ÿ ü í ü ç

·Â�G� ÷ ´��±�º«��©�\�����N´���� �N�S£��I���¬�/�+���G�If æ�� ï ­ ¤�� ­�� ¤®ç®ç®çE¤ � ï p ¤�� p � ��§?�¼��´��¬�N�¬«*�,ö���¡e³��©�G���£�«��©�T����´��(�"�\�®��� ûR�����N���¬«<§­��£�¤G� �N�S£²°i���©¤¸��¡+��³���7ï ­ ¤®ç®ç¬ç�¤�ï p� �N£�� �*æ æ(» ¤ » ç¶ç%ݪ§

»r

p� �� ­ ÷ µ���·7è��:µ­ï · æ � è�è »r æƹ

� ���¬£�°��"� �6øw���%���/�������Ð���(�k�UñR�7V�ð¯��§��'��£�«��¬� ÷ �©� ¹ ç{W¯�I�©£'��%��³�� �¬�gº°��e���6øw� �%���(��£���N�S�S��� � ��¡�°i���N¤¸�'� �k���G��fÆ��£�« ÷ ���(�N£�³����(��£�«l����� ³���� �(��£��'°i���N¤¸�l�%���g��«�����£��¸���S�"����©�¬���\� ¹�� §"�©�i���Wð�� ' Áað��k�¥������V��kÁ V=�(�¥�jñÄ��Je0

zQ�©£����©��ø ò �©�G���ŵËï$· æ ���N�S£:µ ± µ­ï$·\·�°i���¬�\� ± �¼µ�å æ æg» ¤ » çÂ�N�i�"�\�®��� ûR�����N���¬«º§­��£�¤G���©�S£ ¥ W¯��©£)ë��¬¤G���©�S£e ¥ È ¥ r¯°��D� �6ø-���%���F� ���dJÀ�k�ÄVk���(��§��(�S£e�d�©��´$�G�©�G«��,ö���¡e³��©� � ﬤ�� ��� µ! "� æg» ¤ »��©�#� ± µ­ï$·¥ ��N£�� ��� � �����¬��ø�� ���N���N�]�����d��¡+�S��£���´�øe°i���©¤¸�$�"³��\�¬«��N¤G�����"¤¬�S�\���G¤G� ��ø ¥ üX��� �X� �%����� ���¡+�����S�N£²��§%�º�S£ � ﬤ�� � �©�D£��S£�£��G����� �j�S�I�N§Q��£�«5�S£��xø<� §&�º³����G«��©¤T� �'�º¤G�S���\�¬¤T� �xø ¥

YDZoà�Zoà �%|d�)(+*�,%,.-0/%1�,%,32+-{tÆ� c �]���³y}| ~ôD���Hº��������������������B���©�S�����N����¡ �N�t�S�j�S�G£w�©£qzQ�©�S����� ó�¥ r ¥N» ôD����³%������¡+�T� �G�54w�©�X� ���g«��¬�\�©�\�¬«�¬�\���S�±����� �²��§¯�����² %£%������ø�³���� ���G���N� ò �����5³%������¡+�T� �G� ¹ �©�g� ���5�S�%��� ��£�� �G�¬«P��«�����£��¸���S�²��§� ���B��ø�³���� ���¬���G�5���G������£��¬«Æ´�øP� ���w°��®�A���©�¬����£��¬� ò ��£�«76l�©�º�����B«��¬�\�©�\�¬«ú¡+�����S�N£|��§I� ��� %£%���¯��ø�³$��� ���G���©� ¥ º��������������������B�\��£��e� ���q°��®�A�P�N�®���\£��©£��l���N�S�S��� � ��¡ ���2���¬�����¯���©¡e�¬�e�S£

»7Rvó

Page 152: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

8Gø%9 Ý �;: ���¬���£���¡�´��¬���ͧ=<>4?< » ¤�§A@B6"@ ¹ < ­�� ��¡e³��N�(fDC � ï ­ ¤�� ­�� ¤®ç®ç®ç2¤ � ï p ¤�� p � °i���G���(�®��¤¸�?� m � � æ(» ¤ »�°��¬�k�²�©�¬����£��¬��EGFº°i���N¤¸��� �k���G�¯�N£�³����gµÄf&¤�HJIÄ·���£�«*����� ³���� ��KI&LM�7ï ­ ¤®ç®ç®çE¤�ï p� å æ æ(» ¤ »ON

� Ý �P9 Ý �Q:��SR�³���� ���¬���N�5�:µËï$·#Câ���©��£:µ ± µËï$·�·»S¥UT ÷ õ öWVXV �"C » ¤®ç®ç®çE¤ h ó¬ô ��ä ­ µY��·ZC »r ¥UT ÷ õ öWVXV �"C » ¤®ç®ç®çE¤ h ó¬ô �\[ n µY��·]CѧR�¥ ó¬ô �_^ZC »È ¥ Ý ø��I� V ©?µÄäÉ·�<`4?a:÷2 ¥ T ÷ õ öWVXV �"C » ¤®ç®ç®çE¤ h ó¬ô �_H_I\µY��·]CÉäbI\µY��·�ñ�èéäcI èk�¥ �\��£dEGF$µ¶f&¤�HJIÄ·�� �"�S�T���KI�����¤¸�*� ����� ­� ê p �� ­ H_I�µY��·7è��KI\µ­ï · æ � èe@ ­� æ%¹ ¥ T ÷ õ öWVXV �"C » ¤®ç®ç®çE¤ h ó¬ô �\[fI\µY��·ZC7[fIhg ­ µY��·0ij� �KI\µËï · æ 6� ¥ T ÷ õ öWVXV �"C » ¤®ç®ç®çE¤ h ó¬ô ��äbI�k ­ µY��·ZCml » � §&[_I\µY��·�<8§

µ »Íæï¹ ·onWp þ ÿ�q � �N§r[_I\µY��·�¶{§s�¥ ó¬ô �_^ZC>^0i »» § ¥ ó¬ô �ftBC`^ æ|»»S»S¥ õ�ô � Ý õ øu�vCâ���©��£:µ ± µËï$·�·�°i���G��� ± µËï$·#C ­w ê w m � ­ � m µ­ï$·

zQ�©������� s�¥N» L]xD����º�����������Å��������*���N�S�S��� � ��¡ ¥

» ÈЧ

Page 153: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

�*�\��¢����G£�¤¬�<��§D¤®���\�G§­���N�YRB¤G�S£���� ����¤G� �G«}«��©��� �\�©´������©�S£��I��£�«1����� ³���� ���5� ���\�¬�\���S�©«��¬«1�\��¡ß��§� ���e�QR�³���� ���G���G�-� �����(�S�¬£��¬� ��� �¬« ¥ xD���À¢��%��£�� � �yRc[_I\µ���·d�N£'�N�©£��±5¡+�zRB´$�¸���©�T°��G«¿���I� ���¤¬��¡����¼��� �����±��¡e�S��£��I´QRq°i���N¤¸�'�����±�QR�³$�������¬�\�¬�\� ­ ¤®ç®ç®ç2¤{�KI��{|�¤¬�¬�G«l�����±«��¬�\�©�\�¬«'¡º���\�S�©£�6�S£�� ���(�¼��´��¬�N�¬«5�}|���¡e³��N� � ï ¤�� � çWxD���-¡e�®�������\��äcI�k ­ �����\�©�S£��X�}|�³��S£��G£����¼���N�YR²�N�¬�\�X°��¬�N�S���� �g�}|���¡e³��©�G�]°i���¬���~[_IÅ�©�]�¼���\�S����� �����]§­�S��¤G�©£��/�����¯°��®�k�"�N�®���\£��¬�?� �I§­��¤G���]�©£º���¸������^Gi » �S£�}|���¡e³��N�¬�D°i���¬�\�g³����2���N�S���i�QR�³��������¬�\�¬�t���Y���g«��S£��I³$���S�\�YR ¥3� �����I���%���i�\�©£�¤G�g��£SR5¡e�®�������\�¡+��³��F�©£����eæ §�¤ »ON �����¬�\�X�N�?�/�\� �\�©¤T�F´$����£�«e�S£±� ���i��¡e�S��£��?��§�°��¬�N�S���F°i���©¤¸�e¤®��£"´��X�����\�©�S£��G«� �e��£QR²�{|���¡+³��©� ¥

YDZoà�Z�� ál�%�i� _ � _�� ���%�%� � y��d�/x�x|� _ (+*�,&,�-�/r�Z,&,r2�-� �E�S�¬�����:�����T§­���:³�����³$�G�\� �N�¬�D��§À� ���º��������������������B���©�S�����N����¡Ç�����I�¬����R<���À���¬�\�N§�R ¥ xD���(���N�S������ � ��¡Ç�©�D¤¬���©�N�¬«°º��������������������w´$�G¤®�����\�I�®��¤¸��«��©��� �\�©´������©�S£º�N�D¤G�S£��\������¤T� �i�©�D���%��� ��£����¬�G«5���´�������¡e�������M�±�©£²�����I���¬£����I� �%���X£��"�\�©£��S�N�/³$���©£��D���G¤¬�G���S�G�X�����"¡���¤¸��°��¬�N�S���zL� V¨ö ��� ù{ûY�����=�$�À�k�q�}�-�¥�¨U¬ãG�-�F�(�\H_I�e� à �.�������À�}�y�y�?�¸���"º��������������������Æ���(�Z�Z�"µ�H_I¨·�@ ­�¸p ç� õ ÷Å÷ T :Dz%�S�©�N��°i��«��©�\�¬¤G���YR+§­���S¡ ¡ ´��\�¬������� �©��£ s�¥ r±��£�«5� ���(¤G�S£�«�� � �N�S£²�©£5�©�N£��wÈ ¥

Wt£���� ���G�������T§­���ų����S³��¬���yR+�N�]���%����� ���X %£%�����QR�³��������¬�\�©�'�<�%����¡º���\�S�©£e�����©�¬���\��6g�S£<���©�´����¯��4�§­����¤G���©�S£²��§À� ���(³��S�N£����D�©£*f L¡\¢ ô ÷ õ�ô � ù{û Û�£ �ͺ��������������������u�y�E�ÍJ¸���¾�����$��Ä�W�E� ± �����-�q� à �$� ü ¤ 5¥Q¦X§�¨ þ ë § ÿh©eªo« üp <>4Âç� õ ÷Å÷ T : � �N£�¤¬�d[ w µY��·¬C­t�µ�� ± µËï · æ 6S·¥¤��N§�� ± µËï ·v@®6q� ���¬£ [ w µY��·�@ §B��£�«¹���¬£�¤G�ä w k ­ µY��·]C » ç���S£��\��¢����¬£����YR5°��I�%�0���

è¯�}�°L�� ± µ­ï ·�@`6 èh @ ê<p �� ­ ä w k ­ µ���·

h C{©?µ¶ä w k ­ ·�<`4´QR<� ���(¤G�S£�«�� � �N�S£5�©£²�©�N£���È ¥

� �����X���%�������N£�¤¬��6¸¶{§JxD���G�S���G¡ s�¥ È(�©¡e³��©�N�¬�Q���%���]� ���i %£%���®º��������������������w�SR�³���� ���¬���N��©�D¤G�S���\�¬¤T�¯��£q���N�Å´����t��4�§­����¤G���©�S£²��§?fQç

zQ�©£����©��Rq°��<¡������-�\���7°Ç���%���(� ���º���©���S��� � ��¡ � �¬�\¡+�N£%�����¬�(�©£l�*���¬������£%��´��N�<��¡e�S��£��(��§� �N¡+� ¥ xD���*§­���©�©�7°i�N£��B� ���G�S���G¡ ´��S��£�«��+�����q£���¡�´��¬�+��§d���©¡e�¬�"� �%����º�����������Å���������°i�N�©��}|��G¤¬�����-�N���D¡º���©£<�N���S³.L

» È »

Page 154: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

¡\¢ ô ÷ õ�ô � ù{û ç±£ ����=�$�»�e²³���´���W�$�E�q���KI«���}�Äã=�¥�.���+UO²�EGF?���?Á^���.�fµ¨���(���Ð���(�k�W�F�7Ve�w����Á¶���(�}� ¹ã=�¾�e�E�#H_I�·R�¨0X�¬0Î�����-�q� à �$���Ä�W�a�$�(�¾�k���-�F�(�o���ÍÁ^���.�_µ³¸Â�k�¾�6C�q�Ø�P�}�&��� ¹� k ¹ ¤.���W�E�»º��$��������Å%��������y�E�TJ¸���¾���y�$�f�"�����=t±< �� ¹»ºy¼ ­ g ¹ ç

Wt�À°i�©�©��´����E���©«��G£��~§­���S¡É������³�������§����\�©�©�������YRt«��Y½$�¬�\�¬£���´��S��£�«��Q�S£_tP¤¬��£-´����G�\� ��´��©�N�����G«´QR�¤¸���������©£��+«���½Å�G���G£��Í�����©���¬�X��§]6"�©£5� ���(� ��£��S�ºæ §�¤ ¹�N ç�#l�(� �k���\6\C ¹� k ¹ �N£������(�����¬�����¬¡��´��Y�S�t´������e���-�S´�� ���©£+�I¡º���\�S�©£�6_C¿¾Iµ�À{·?��£�«e� �(�S´�� ���©£+�g¤G�©�®��£+´��S��£�«º�©£"� ���i� ���G�S���G¡ ¥xD���G�S���G¡ s�¥ 2 §­�S�N�©�7°i��§­�\�S¡ � ���(´��S��£�«��i�¬�\� ��´��N�©�����¬«��©£<� ���I§­���©�©�7°i�N£��-�o°��º�©�G¡+¡+���»LÁ ô �D� ö ùÂûY ê²p ���à ê w I ��à äcI\µ���·o� �KI\µ­ï ·�ÄÆr´À ê w I ��à èðäcI¸èxçÁ ô �D� ö ùÂûhÅ £ �'6_C ¹Æ k ¹ ��ÇW�}È�ê p ���à ê w I ��à äbI\µY��·o� �KI�µËï ·�< Æ p¹»¼ à g ¹ iuÀIê w I ��à èéäcI èyç��S¡�´��©£��N£���� ���G���(´$����£�«��D°��g�S´��¸���N£

r�hÀ�É »'Ê À � À w�I ��à èéäcI èeÄËÀ.4�hvt°i���G���I� ���I�¼���\�i�©£��OÌ��%���N�N�yR+�©�D´��¬¤®�������5èðäcI¸è�CBÍ?µ¶äcI­·�hÎÄB4�h §­�S��^]C »�Ï ç®ç®ç Ï tiç� õ ÷Å÷ T ÷ T Á ô �D� ö ù{ûYÂÐ: � �N£�¤¬�\�KI\µ­ï · � æ Ê(»;Ï®»»N ��£�«�� � � Ê(»;Ï®»�SÏ °��-�%�zÑS�J� �KI\µ­ï ·�C»'Ê è��KI�µËï · Ê � è Ï ��£�«5� �����

p� ���à H_I\µY��·o� �MI�µ­ï ·#C p� ���à HJI�µY��·¬µ »�Ê è��KI\µ­ï · Ê � èð·�ÄÆrÒÀ~ç

xD���N�X�N¡+³��©�©�G�����%���p� ���Ã

w�I ��à äbI\µY��·o� �KI\µËï ·ZC w�I ��à èðäbI è p� ���à H_I�µY��·o� �MI\µËï ·�Ä w�I ��à r´À?èðäbI¸èyç� õ ÷Å÷ T ÷ T Á ô �Ó� ö ù{ûhÅ�:'ÔÕR<� ���(«��G %£�� � �N�S£²��§r[_I\µY��·�°��g���PÑ��

p� ���Ãw� I ��à äcI\µY��·o� �MI\µËï ·ÖC p� ���Ã

w�I ��à äbI\µY��·¬µX[_I\µY��· Ê [fIhg à µY��·0i 6�·

C 6 w�I ��à èéäcI è»i w�I ��à p� ���à äcI\µ���·¬µ�[fI�µY��· Ê [fIhg à µY��·�·Íç µ s�¥N» ·

»z× r

Page 155: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

N(t)

0 1 2 3 4 t

Ø n Ø ÃËÙ Æ ÙzÚDÙOÛ ÙzÜ ÙzÝ ÙPÞ

zQ�N�S����� s�¥ reLFWâ³��©���D��§Ëß[b°i�N���àt>C × ç� ��� �t���%��� ß[ �©��³��©�¬¤G�G°i�N���d�N�©£��¬���]°i� � �_á\���©£��������N£����¬�S�G�ÕÑ����N���¬����§0^Íç�W ³��S�\���©´��©�t³����©�\�©£��(��§���G�S¡e�¬£�� �d¡+����¤¸���¬��æ Ù Æ Ï ÙzÚ N °i� � �}æ ÙzÜ Ï ÙzÝ N ��£�«¿æ ÙzÚ Ï ÙOÛ N °i� � �¿æ ÙOÛ Ï ÙzÜ N�Ï �©�¬�PÑ��N£��Bæ Ù n Ï Ù Ã N�Ï æ Ù Ã Ï Ù Æ N��£�«1æ ÙzÝ Ï ÙPÞ N ��£�³%���©�\�¬« ¥ Ýo£5� ���N�D�}|���¡e³��©�!ß[`�N�D�©£�¤G���®�����©£��e��£��¬��¤¸�q��£�³%���N���¬«5�\�¬�S¡e�¬£�� ¥

ݪ�i� �����i���Gâ+¤G�¬�X� �"�����7°É���%���i� §36fC ¹Æ k ¹ Ï �����¬£5§­�S�D�¬��¤¸�v�\C »;Ï ç®ç¬ç Ï h °��(�%�zÑS�w�I ��à äcI\µ���·Gµ�[fI\µY��· Ê [_Ihg à µ���·�·�< rÀ É »'Ê À i µ�À Ê 6S· w�I ��à äcI\µ���· µ s�¥ r�·

���N£�¤¬�+����¡e¡e�©£��5� ���N�-�N£��OÌ��%���N�N�yR��PÑS�G�5�cC »;Ï ç®ç¬ç Ï h ��£�«¿����´��\� � � �����©£��*�©£�� �¿µ s�¥ » ·/³����PÑ��¬�� ���d�©�G¡+¡+� ¥ z~��|º��£QR"� � � »;Ï ç®ç¬ç Ï h Sã §­�S���¬�����d��§~£����¸�����©�S£e°��d°i�\�N���5[_I~��£�«�äcI:�©£<³��¼��¤G���§r[_I\µ���·���£�«wäbI\µY��·�§­�S��� ���(�\�¬���¯��§Q� ���(³�������§ ¥

ݪ§�[_IfC®[_Ihg à §­�S���\�S¡e�<�©£�� �G�S�¬�J^(�����¬£}�����<� �G��¡ äcI\µ�[fI Ê [_Ihg à ·(¤¬�S£�� �\�©´���� �¬�=äq���� ���q����¡ �N£¢µ s�¥ r�·}���\�l°i�N�����S���+�©�����e��§/�S�G£��¬�����©� �yR}°��q�����\��¡+��� �%����[_I�åCæ[fIhg à §­�S�+���©��©£����¬�S�G����^Íç�#l�(�{|�� �¬£�«*� ���(�\�OÌ����G£�¤¬�<µ�[ n Ï [ Ã Ï ç®ç®ç Ï [ w ·�� �e�±¤¬��£����©£����S���¯³��N�¬¤G�G°i�©�\�I�©�©£��®���§­��£�¤T� �N�S£ ß[ �S£Ææ�ä Ï t N �N£1� ���²�S´QÑ��N�S���-°��zR;�]� ¥ � ¥ §­�S�_^���£}�N£�� �¬���¬�±��£�«èç � æ�ä Ï®»»N °��5�©�G�ß[}µ�^0ij縷ZC7[_IMi ç®µ�[_I�k Ã Ê [_IÄ·Íç+éÂ�G�

êëC�� Ù � æ�ä Ï t N Ldß[1µ Ù ·ZC7[fIQ§­�S�D�\�S¡e�/�©£�� �G�S�¬��^3C¿ä Ï®»;Ï ç®ç®ç Ï t ç

xD���"���T�Jê��©�t %£��N� �±�\�²°��e�%�zÑS��ä�C Ù n < Ù ÃÐìPìPì < ÙOí Cîtò°i� � �bê°C�� Ù n Ï ç®ç®ç Ï ÙOí µ¨���G�zQ�N�S����� s�¥ rS· ¥ � �©£�¤G�(§­���¯�¬��¤¸�w�©£����¬�S�G��^'Ä » � ���-�©£����¬��Ñ�����µ�^ ÊÁ»;Ï ^ N ¡������t¤G�S£��¸���N£q�\�S¡+� Ù m �°��g¤¬��£5���¬�{|�³����G���t�����I����¡ ê w I ��à äcI\µ�[fI Ê [fIhg à ·D���

í� m ��à äBïñðóòõô#ö~ß[¿µ Ù m · Ê ß[1µ Ù m g à ·�÷iç µ s�¥xR ·

»z×�R

Page 156: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

#l�B� �zR|� �����<�o°��P�\�¬�S¡e�¬£�� �læ Ùzø g Ã Ï Ùzø N ��£�«Ãæ Ùzù g Ã Ï Ùzù N J�ú��yû³Ç1� § ß[}µ Ùzø g à ·üC ß[}µ Ùzù ·<��£�«ß[}µ Ùzù g à ·ZC ß[}µ Ùzø ·Íç%z%�S���}|���¡e³��N�����N£�zQ�©�S����� s�¥ r-� ���/���¬��¡+�G£��-æ Ù Æ Ï ÙzÚ N ¡+����¤¸���¬�gæ ÙzÜ Ï ÙzÝ N ´����«����¬�(£����(¡+��� ¤¸�|æ ÙzÝ Ï ÙPÞ N¶¥ #1�e³%���N�(��³l¡+��� ¤¸���©£������G�S¡+�G£�� �(��£�� �©�?£���¡e�S���"³%���N���I¤®��£1´��§­�S�\¡+�G« ¥~� ��� �(� �%���t� §?��£QRq��£�³%���N���¬«*���G�S¡+�G£�� �d�\�¬¡+���©£Ð��� �t¡����\�d´��g������¤¬�����(� �%���t�¬� � ���G�ß[ �N���©£�¤G���¬�����N£��q�S£}�¬��¤¸�,��£�³%���©���G«}���G�S¡+�G£���µ¨� §'[ w � ä�·I�S�°ß[ �N�-«��G¤¬�\�®���\�©£��B�S£}�¬��¤¸���£�³����©�\�¬«����G�S¡+�G£��²µ­�N§Õ[ w <)ä�· ¥=� ��°Ã°��e���G³%������� �±� ���e�\��¡ µ s�¥xR ·t�N£�� �<�o°��q³��N�¬¤¬�G�O�:� ¥ � ¥ê ím ��à ä ïýð�òõô µÕß[¿µ Ù m · Ê ß[}µ Ù m g à ·�·'CÿþBi�� Ï °i���G���=þÇ�©�¯�������\��¡��PÑS�G�I���©�À³%���N���G«B���¬��¡+�G£������£�«��)�©�?� ���¯����¡Ê�PÑS�G�����N�%��£�³%���N���G«º���G�S¡e�¬£�� � ¥ #l�i°i�©�©�%�\���7°Á� �%����þî<ɵ�À Ê 6�· ê w I ��à äcI��£�«��ë< ƹ ¼ à g ¹ ��� �����i³����PÑ��©£��"�����I�©�G¡+¡+� ¥

zQ�©�\�\�±°��*´��S��£�« þ ¥ éÂ�G�qæ Ùzø g Ã Ï Ùzø N ��£�« æ Ùzù g Ã Ï Ùzù N ´��*�B³%���©�±��§¯¡º��� ¤¸���N£������¬��¡+�G£����°i���G��� ß[ �©�(�©£�¤¬���¬�����N£����S£|æ Ùzø g Ã Ï Ùzø N ��£�«1«��G¤¬���¬�����N£��q��£úæ Ùzù g Ã Ï Ùzù N ç%xD���º¤¬��£������N´���� �N�S£���§� ���¬���I�o°��+�\�¬�S¡e�¬£�� �i� ��þò�N�

ä ïñð���ô]ö ß[}µ Ùzø · Ê ß[¿µ Ùzø g à · ÷ i�ä ïñð���ô]ö ß[¿µ Ùzù · Ê ß[¿µ Ùzù g à · ÷CòµÄä ïñð��oô Ê ä ïñð��hô · ö ß[}µ Ùzø · Ê ß[}µ Ùzø g à · ÷ ç µ s�¥ý× ·� �N£�¤¬�d�¬��¤¸�5���G�S¡+�G£��gæ Ùzø g Ã Ï Ùzø N �N��¤¬�S£�� ���©£��G«²�©£�æ ^ ʹ»;Ï ^ N §­�S���\�S¡+�t�©£�� �G�S�¬��^ Ï °��/���PÑ��d� �%���� Ùzø Ê�» @ Ùzø g à < Ùzø @ � Ùzø� çWxD���I�©�©£��®���\�N�yR+��§Ëß[`�S£1æ � Ùzø� Êú»;Ï � Ùzø� N �©¡e³��©�N�¬�����%���

[ ïñð � ô�g à @ ß[}µ Ùzø g à ·�< ß[}µ Ùzø ·�@`[ ïñð � ô @`[ ïñð � ô�g à i »�Ê 6 µ s�¥ 2 ·

°i���G���i� ���D�©���\�?�N£��OÌ��%���N�N�yRg�©�F´��¬¤¬��������� �KI\µ­ï ·�@ » �©£±�©�N£��Xd��§ ��������������������� ¥ � �N¡+�N�¼���\�YR��°��g�%�zÑ��(���%��� � Ùzù� Êú» @ Ùzù g à < Ùzù @ � Ùzù� Ï ��£�«����¬£�¤G�

[ ïñð���ô�g Ã Ä ß[1µ Ùzù g à · � ß[1µ Ùzù ·�Ä`[ ïñð���ô Ä`[ ïñð��hô�g à Êú»'Ê 6�ç µ s�¥xk ·

� �N£�¤¬� ß[}µ Ùzø ·#C ß[}µ Ùzù g à ·��©£��»Ì��%���©� � �N�¬�Iµ s�¥ 2 ·i��£�«¿µ s�¥¦k ·��©¡e³��YR+� �%����[àïñð � ôog à ÄB[àïñð � ô�g Ã Ê r�çxD���]«��G �£��N���©�S£(��§�ä £��7°}�©¡e³��N�©�¬�$� �%���Qä ïñð � ô Äòµ »KÊ ÀÅ·�ä ïñð � ô ç � �N£�¤¬�¿ß[¿µ Ùzø · Ê ß[¿µ Ùzø g à · � ä Ï°��(�������X���PÑ��(���%���gµ s�¥ý× ·D�N�i���i¡e�S�\�

ÀÅäBïñð � ô ö ß[1µ Ùzø · Ê ß[¿µ Ùzø g à · ÷ @ËÀQµ »'Ê 6S·\äBïñð � ô7µ Ùzø Ê Ùzø g à · µ s�¥ �·

»z×;×

Page 157: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

°i���G���q�����q�N£��OÌ��%���N�N�yR}§­�S�N�©�7°i�"§­���S¡ µ s�¥ 2 ·"��£�«P� ���*�©�©£��®���\�N�yR}��§ ß[ �S£Èæ Ùzø g Ã Ï Ùzø N ç � �©£�¤G�ß[}µ Ùzø · Ê ß[¿µ Ùzø g à ·ZC ß[¿µ Ùzù g à · Ê ß[¿µ Ùzù · Ï °��g�\�©¡e�©�¼�����YR"���PÑ��/� �%���-µ s�¥ý× ·��©�i���i¡e�S�\�

ÀÅä ïñð���ôZö ß[¿µ Ùzù g à · Ê ß[}µ Ùzù · ÷ @ À»'Ê À ä ïýð���ô]ö ß[}µ Ùzù g à · Ê ß[}µ Ùzù · ÷@ À»'Ê À µ » i 6S·\äBïñð���ô¬µ Ùzù g Ã Ê Ùzù ·Tç µ s�¥�� ·

�¯�\�©£��g� ���X§¨��¤G��� �%����6_C ¹Æ k ¹ ��£�«<�\�S¡e�d���N�S�¬´����K���©£��»Ì��%���©� � �N�¬�tµ s�¥ S·?��£�«�µ s�¥�� ·]�©¡e³���R�� �%���µ s�¥ý× ·��N�i���i¡e�S�\�

ÀQµ » ièÀÅ·r�iuÀ ö äBïñð � ô®µ Ùzø Ê Ùzø g à ·0iPäBïñð � ô7µ Ùzù g Ã Ê Ùzù · ÷ ç µ s�¥¯s ·

ݪ§{°��t�\��¡ µ s�¥ýs ·F�PÑS�¬�����©��³%���©�\����§{¡+��� ¤¸���N£��(���¬��¡+�G£����������¯�\�¬�\���N���©£��_Ì��%��£�� � �yR"�©����£+��³�³$�G�´��S��£�«*�S£�þDç�Ýo£*�����©�X����¡à�%§­�S�i�®��¤¸�?Ñ����©���I��§3^ÕC »;Ï ç®ç®ç Ï t Ï �����(¤¬���»âe¤¬�N�¬£��d��§FäcI�°i�N�©�´�����¯¡+���\� ¹ þ à k ¹ ÿÆ k ¹ C À Ê 6�ç�µ�xD���©�¯´$����£�«w�S£�� ���-¤¬���»âe¤¬�N�¬£��d��§?äcI?���S�N«��t´��¬¤¬�������g§­�S�¯�¬��¤¸�^ Ï � ���I� ��� ���:�©�G£���� �*��§?���N�³%���N���¬«����G�S¡e�¬£�� �d�N£¿æ ^ ÊÁ»;Ï ^ N �N�t���X¡e�S�\� » · ¥ ��S£����»Ì����¬£�� ��Rq°���%�zÑS�Jþî<ȵ�À Ê 6�·%ê w I ��à äcI����D«��¬�\�©�\�¬« ¥� �7°¢°��-������°È���%����� Ï �����-�\��¡ �ÒÑ��¬�¯��£�³%���N���G«q���G�S¡e�¬£�� �»�Å�©�¯���¯¡+���\� ƹ ¼ à g ¹ ç�ݪ§ ß[ �N�«��G¤¬���¬�����N£��±�S£+�®��¤¸�²��£�³%���©�\�¬«<�\�¬�S¡e�¬£��������¬£²¤G�©�¬������R��ë<>ä Ï ���-°��d�\��³�³��S���t���%���±ß[��©�]�©£G�¤¬�\�®���\�©£��-�S£<�®��¤¸�²��£�³%���N���G«º���G�S¡e�¬£�� ¥ éÂ�G�/æ Ù���� g Ã Ï Ù���� NóÏ ç¬ç®ç Ï æ Ù���� g Ã Ï Ù���� N ´��d���©�%� ���¯��£�³%���©���G«���G�S¡e�¬£�� � ¥ Wt�I�©£�zQ�N�S���\� s�¥ rº�N�/¡����\�(´$�������±¤®�����±� �%���/� ���"�©£����¬��Ñ����N�"æ�ß[¿µ Ù�� òhg à · Ï ß[¿µ Ù�� ò¨·\·���\�g���©�Å«��©��á\���©£��i��£�«5� ���¬�©�D��£��©�S£5�N�(æñä Ï [ w ·Tç+Ô#R<� ���I«��T %£�� � �©��£²��§Fä Ï °��(�%�zÑS�

�7C �� m ��à µ »'Ê ÀÅ· þ n������ ò� �! � ÿ�q Æ ö ß[}µ Ù�� ò¨· Ê ß[1µ Ù�� òhg à · ÷ ç

Wt�i�N£5�����I´$����£�«5§­�S�'þ Ï °��(�%�zÑS�

[àïñð � ò ô�g à @ ß[¿µ Ù�� òhg à ·�< ß[1µ Ù�� ò¨·�@`[àïñð � ò ô�@`[àïñð � ò ôog à i »�Ê 6A< [àïñð � ò ô�g à i »

��£�«����G£�¤¬�

� @ ��m ��à µ »�Ê À{· þ#"n þ ð � ò ÿ g à ÿYq Æ ö ß[}µ Ù�� ò¨· Ê ß[1µ Ù�� òhg à · ÷C µ »�Ê À{· g à q Æ ��m ��à µ »�Ê ÀÅ· "n þ ð � ò ÿYq Æ öß[¿µ Ù�� ò¶· Ê ß[}µ Ù�� òhg à ·o÷iç

»z× 2

Page 158: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

� �N£�¤¬� ß[`�©�D�N£�¤¬�\�®���\�©£��M��§­�S�D�®��¤¸�%$F°��(�%�zÑS�

µ »'Ê ÀÅ· "n þ ð � ò ÿYq Æ ö ß[¿µ Ù�� ò¶· Ê ß[}µ Ù�� òhg à · ÷ <'& "n þ ð � ò ÿ( � "n þ ð � ò ! � ÿ µ »'Ê ÀÅ· ( q Æ*),+ ç

� �N£�¤¬�/� ���(«��©��á\���©£��D�©£����¬��Ñ����N�(æ�ß[1µ Ù�� ò�g à · Ï ß[1µ Ù�� ò¨·�·�¤G�ÒÑ��¬�-æñä Ï [ w ·�°��/� �����X�%�zÑS�

� < µ »'Ê ÀÅ· g à q Æ & n.-( � nµ »�Ê ÀÅ· ( q Æ ),+

< µ »'Ê ÀÅ· g à q Æ & �( � nµ »'Ê ÀÅ· ( q Æ),+

C Ê rÉ »'Ê Àt�©£{µ »'Ê ÀÅ· < rÀ0É »'Ê À §­�S�'ä=< À�< » ñSr�ç

µ�éÂ�¬¡e¡+� s�¥ �·

YDZoà�Z0/ � �214365F���87����b���:9<;>=?9@;d�:�úa,�i�27,9%��� cBA � c �]���:9<;C147xD���D�������������������������©���S��� � ��¡Á°����Q�©£���³��N���¬«g´QR-��£����N�S�S��� � ��¡)�S�YÑ��¬£g´SR(Ýo¡e³%���S�N�¼�FEGE¬�D�N£g� ���¤¬��£����}|��¯��§Q������«G�ª¤¬�S�\�/���T�t¤G�S£��\������¤T� �N�S£��i�©£5¤¬��¡+³��N�}|��N�yR+� ���G�S��R�µ¨Ýo¡e³%���S�N�¼�FEGE¬�M� »Ps;s 2 · ¥ #l��S´��\�¬�oÑS�¬«g�©£(��%��³�� �G� � ���%���:Ýo¡e³%���S�N�¼�FEGE¬��� �{���©�S�S�\�N����¡Á¤®��£I´��F���¬�N£�� �¬�\³����T� �G«-���:�D´������\� �N£����´QRQ�o� ��¡+³��N�©£��"���©�S�S�\�N����¡ °i���©¤¸���S�¬£��¬� ��� �¬�X«��N�\� �\�©´���� �©��£���H_I~°i���N¤¸�Ð���©�IH��t�����I«��©��� ���N´������©�S£���S�G£��¬����� �G«g´QR������������KJ����������S����� �N�\§�R5�#�"µ�HJI­·�@ Ã�¸p çML¯�7°��»Ñ��¬�O���S�������������������������w���©�S�����N����¡«���½Å�G���:§­���S¡¢Ýo¡e³%���S�N�¼�FEGE¬��� �À���©���S��� � ��¡)�©£-�\�»Ñ��¬� �����©¡e³��S�\� ��£��°��zR�� ¥ xD�������©���S��� � ��¡)�©£²µ­Ýo¡=�³%�����©�¼�NEOE¬�K� »Ps;s 2 ·D���\�¬�/��«�«��N���YÑS�(� ��� ���G�X���%��£w¡����N� �N³��©�N¤®�����YÑ��d��³�«%��� �G�¯§­�S�täcI\µ���·{�Å��£�«q� ���´��S��£�«5�S£vtÆ°i���©¤¸�5�N���S�YÑ��¬£<§­�S��� ���I���N�S�S�\�N� ��¡ �©£�µ­Ýo¡+³%���S�©�©�FEOEG�M� »zs;s 2 ·?�©�QPºµ Ã� º�¹»º ·]°i���©¤¸��©�d°��S�\���+���%��£'�S���/´��S��£�«'´QR'�5§¨��¤G� ���/��§ Ã� çÂW¯£���� ���¬�(�©¡e³$�S���¸��£��/«��Y½$�¬���G£�¤¬�"�N�I���%���I� ������N�S�S��� � ��¡Ê�N£'µ¨Ýo¡e³%���S�N�¼�FEGE¬�M� »Ps;s 2 ·��%���D£��"¡+�������©£<³%��� ��¡e�G���¬�'6±��£�«5«����¬�i£����i��³�³��®��������S����³����(�²�¼�����S�g¡º���\�S�©£* %£%�����QR�³���� ���G���N� ¥ zQ�N£%���©��R;��� ���"��£%���YR��\�©�d�N£¹µ¨Ýo¡e³%���S�N�¼�FEGE¬�M� »Ps;s 2 ·�S£���Rq¤G�PÑS�¬�\�/� ���±¤¬������°i���G���±� ����°��¬�FHq�QR�³$��� ���G���¬�g���\��´��©£�����RQ��Ñ����N���¬«q� �������¬�d���%��£B���¬�����Ñ����©���¬« ¥

z%���¬��£�«î��£�« � ¤¸����³��©�\��� �B°��G�©����H�£��7°i£Ã´������\� �N£��Æ���©���S��� � ��¡ s�u�vRJ��������Ã�N�����S¡e�G°i�%���§¨����� �¬�����%��£S�����������KJ������������\�OÌ����©�\�©£��/��£��YRJt`CTPºµUWV:X þ à q � ÿ¹ º ·Q�\� ���S�¬�¯µ¨z��\�¬��£�«+��£�« � ¤¸�%��³��N�����»Ps;s S· ¥ é:�IH��S�����������KJ����������Qs�u�v#J%��������¤®��£'´��±�����G«l°i�N�������®������Ñ����N���¬«*°��¬�FHB�QR�³��������¬�\�¬���£�«}¤¬��£,´$�º�����G«,���B�S����³����e�w�¼�����S�+¡º���\�S�©£' %£%�����QR�³��������¬�\�©��µ � ¤¸�%��³��N���D�}�_úFY 0¶� »Ps;sF� · ¥L¯�7°��»Ñ��¬�O�ts�u�v#J��������Á�N��È[Z´�t�S����� ��£����¬�¬«ú���¹���¬£��G� �����B�S£��YRP��¡e����� �|«��©�\�����N´���� �N�S£��»�¯��£�«� �����X«����¬�i£����X��³�³$�¬���D� �"´��(�����G§­���Â�©£��±¡º���N�©¤G�©�S����£��S�N���/¤¬��£����}|�� ¥

»z×�k

Page 159: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

xD���'���¬�\���N���²��§���%��³����¬� � �©¡e³��YR¹� �%����z��\�¬��£�«{�¦��J ��S¡�´ ���N�S�S�\�N� ��¡ °i�N�©�¯�S�¬£��G� �����«��N�\� �\�©´���� �©��£��d°i���©¤¸�l� �����©��§�Rc�#�"µ�HJIÄ·"@\Pºµ UWV:X þ à q � ÿ� ì Ãp · Ï °i���©¤¸�1�N�(��°��®�FH��¬�-�\¡+��������£��¬�\��S�%��� ��£�� �G��� �%��£]�����������^J���������´SRw�<�©�S�S����� � ��¡e�©¤d§¨��¤T� �S� ¥ Wt£��������¬�/«�� �®°i´%��¤*Hw��§2J ��S¡-´§­�S�<�������©�©£��®���º�������G�������©«Á�©�¬����£��N£��,��³�³��©�©¤¬��� �N�S£�°i���N¤¸�Æ°��B°i�©�N�t�\����«KRÁ�N�<� �����\������¤T� ���\���§I� ���q %£%���t�SR�³���� ���¬���N�<°i���©¤¸�Á� �<�S�G£��¬����� �G� ¥ Wt�<°��B� �®°��N£Æ��%��³����¬� � �i� §/� ���w°��®�FH�©�¬����£��©£��±���©�S�S�\�N����¡ ���T� ���\£��i�SR�³���� ���¬���G�5� Ã Ï ç®ç®ç Ï � w � ���¬£²� ���d �£%���Å�QR�³���� ���G���N�X���¬£��G� �����¬«´QR�J ��S¡�´ °i�N�©��´$�d��«��¬³����²�o°��"¡+�Òá\�S��� �yRe¤¬�©�\¤¬��� ��°i� � �<� ���d�QR�³��������¬�\�¬�~� m ����� ���/�©£�³���� � ¥Ýo£q¤G�S£�� �����\�O�������_�����������KJ��������* %£%���:�QR�³$��� ���G���©�¯�©�i���N¡+³��YR²�"� ���\�¬�\���S�©«��¬«*°��¬�N�S��� �G«w����¡��§~� ���(�QR�³��������¬�\�¬�f� m�Ï � ¥ � ¥ � �¯�N�X�e«��G³�� �*�S£��(¡º�Pá\�S��� �yR<¤¬�N��¤G���N� ¥ xD���N�¯�\�©¡e³��©�/��� �\��¤G�������(��§� ���� %£%���`�����������^J��������l�QR�³���� ���G���N�d°i�©�©�À³��©�PRw�<¤¬�\��¤¬�©���Q�\�S�©�g�©£w� ���±�S�¬£��G� ���N��E¬��� �N�S£q�G�������´��S��£�«���§À� ���I£��{|��X�\�¬¤G���©�S£ ¥

z%���¬��£�«"�%�����\�¬¤¬�G£�� �YR-�\����«��©�G«"�d���S³����©��� �©¤¬��� �G«�´����S��� �N£��/���N�S�S��� � ��¡¢¤¬���©�N�¬«aJcb��edRfKJ%�������µÄz%�\�¬��£�«.� »zs;s;s ·X°i���©¤¸�'�����G�g�²�S�G£�� �©�G�d°��G�©�S�����©£��5��¤¸���¬¡e�"���%��£�s�u�vRJ�������� ¥ z%���G��£�«������´��S�G�\� �"� �����gJ,b%�edRfKJ��������¹���������©«¹´$�²°��¬�N������� � �G«¹§­�S�±«��®���N�©£��w°i� � �¹£��S�©�oR¿«%���¸� ã ���7°��»Ñ��¬��N�e�N�e£����e¤¬�©�¬���"§­���S¡ �����*��£%����R��\�©�"�©£ µÄz%�\�¬��£�«.� »Ps;s;s ·�°i���G� ���¬�hJ,b��edRf^J��������S�ª�S�¬£��¬� ��� �¬««��N�\� �\�©´���� �©��£���� ��� �©��§�R}�B��¡e����� ��£��¬���"³��\�S³��¬�\�yR¿����¤¸������� ���à�Z�"µ�H_I¨·d@ Ã�¸p ³����S³��¬���yR}��§�����������KJ������������S�d°i���T� ���G�aJ,b��edRf^J��������l¤¬��£�´$�"���\�¬«'� �5�S�¬£��¬� ��� �e�²�¼���\�S��¡+�����S�N£w %£%����QR�³$�������¬�\�©� ¥ #1�+£������º� �%���g������J,b��NdRfKJ��������,���©�S�S�\�N����¡��N�-¡���¤¸�}¡e�S�\�+¤G�S¡+³��©�©¤¬��� �G«'�������£q� �%��£������������KJ����������:���X�N�¯�©£QÑS�S��ÑS�G�t�����YÑ��©£��+�+«��Y½$�¬���G£�� �¼���Â�OÌ��%�����©�S£B���t�¬��¤¸�w�\� ���S�(��§´����S��� �N£�� ¥

���$�����������������B�©�D³��¬�\�%��³��X¡e�S�\�i�\�©¡e�©�¼����� �"� ���(¡e��«��N %�G«5Wt«%�;Ô����S���X���©���S��� � ��¡Ç¤®���N�©�G«i vSu�v#J��������È°i���N¤¸�â°����'���G¤¬�¬£����YR¢�S��ÑS�G£Ã´QRâOd�S¡e�©£����ú��£�«¢#1���¸��£���´$�)µ¨Od�S¡+�N£��S�ú��£�«#1���¸��£���´$���gr�ä;ä;ä�· ¥ éÂ��H��j�����������KJ���������� i vSu�vRJ��������¢�����G�w¡���� � �N³��©�N¤®�����YÑS�B��³$«%��� �¬�q�S£°��¬�©�������q��£�«Á£��»Ñ��¬�w���©�©�7°i�º°��¬�N�S��� �5� �,�}|�¤G�¬�¬« » �N£`Ñ����N��� ¥ Od��¡+�N£��S�}��£�«Æ#1���¸��£%��´��³��\�ÒÑ��¬« ���%��� i v�u�v#J��������Æ�¸�FH��¬�q����¡+�S���dt @ Æ� ¹»º ���¸���S�G�O�/°i���N¤¸� �©�dÌ���� � �'���©¡e�©�©���+����S���+´$����£�«ú�©£ xD���¬�����¬¡ s�¥ 2 ¥ µ­Ýª§(°��w�\�G�d6 C ä1�©£k�����������^J����������d�1���N�©�S���+¡+��«��N �¤®�´�� �N�S£Á��§(� ����³��\����§g��§JxD���¬�����¬¡ s�¥ 2 �S��ÑS�¬���,´$����£�«Æ��§-�\�S���S���YR ÛÚ � ¹»º Ï °i���N¤¸�)�N¡+³����PÑS�G�� ��� i v�u�v����������}´��S��£�«}´SR1�w¤¬��£��\� ��£���§¨��¤G���S� ¥ ·lL¯�7°��»Ñ��¬�O�?� ���5��£%����R��\�©�(§­�S� i vSu�vRJ���������S��ÑS�¬£*�©£1µÄOd�S¡e�©£��S�e��£�«*#1��� ��£%��´����$r�ä;ä�ä�·��S£��YR5¤G�PÑS�¬�\�t� ���g¤¬�����g��§F´��N£%����RQ�óÑ����©���G«�°��®�FH�QR�³$�������¬�\�¬�»�¯��£�«P«����G�+£����+�G�\� ��´��©�N���,� �%��� i vSu�vRJ��������P���¬£��G� �����¬�<���¼�����S�²¡+�������©£} %£%����QR�³$�������¬�\�©� ¥ #1�t���©�\�g£���� �i� �%���]�S����³��\����§Å� �G¤¸��£��YÌ����t��§Â�\�©¡����N�¸��£��¬�S�����YR-��³�³��¬����£�«º�N�7°��G�´��S��£�«��N£���ê p ���à ê w I ��à äcI\µ���·�� �KI\µ­ï ·t�©�d«��Y½$�¬���G£��/§­���S¡ � ���"��³�³�������¤¸�'�����¬«l�©£¹µ¨Od�S¡+�N£��S���£�«5#1���¸��£���´$����r�ä;ä�ä�· ¥

»z×

Page 160: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

354*m npo6q-=]9BE¬9wÔrn}E®9so6q-=ut ÐB=vo/Ó�Ðw>xw®@ y�AB9BCD;FEG>+9BÓ,ÕbET;FÐ z q�{w¬E®C¯EG>+ABÓ}|�~x�®Ó�o

Ýo£������©�e���G¤G� �N�S£�°��w������° ����°��������e���,�N���KJ<�,�����|���©�S�����N����¡ ¤¬��£|´��q���\�¬«ú�N£�¤¬��£´á\��£�¤{�� �N�S£q°i� � �B�<�\�©¡e³��©�(£��S�N���-���S�©�G� ��£��t°��®�NHq�©�¬����£��©£��º���©�S�����N����¡�� �º�S´��¸���©£B�������È�N�®���\£��©£�����N�S�S��� � ��¡ §­�S�D�N�®���\£��©£��±�©�©£��®�����������G�������©«5§­��£�¤G���©�S£��i°i� � �5¡+���©�N¤¬�©������£��S�©�\� ¥

�Q���Q��� ���À�21\��9^�.���âá]�@�À���814���65v�.����7�l��´��\�©�G�KR-�\�»Ñ��©�T°}� �������¬�S¡e�G�����©¤]³��\�¬�©�N¡+�N£%���\�©�¬�{§­�\�S¡ � �G¤G� �N�S£g� ¥���¥ � ���Àï ��¡£¢ ��£�«¥¤�Ä »;Ϧ ï ¦�§ «��G£������¬�-�����a¤K�ª£��S�\¡ ��§�ï Ï £���¡+�G�YR ¦ ï ¦�§ CßµÄê ¢¨ ��à è ï ¨ è § · à q § ç%xD��� ¾ �o£��S�\¡`��§�ï,�N�¦ ï ¦ �mCÊ¡+�´| ¨ ��Ã:©WªWªWª © ¢ è ï ¨ èyç[�l�±°i���N���g« § µ�¬I·t����«��¬£������±� ���C¤K�ª£��S��¡�´%���©�~��§�� ��«��N���­¬ Ï � ¥ � ¥« § µ0¬I·]C��7ï ��¡Q¢ L ¦ ï ¦�§ @'¬ ç

� �S��¤ ϯ® Ä » � ��� ® �o£�����¡`�©�S°�ãKúFYÅ� ��� ����¤K�ª£��S��¡`� § ç i ñ C »;ã §­�S�(����¤¿��£�« ® °i�N�©���� °��zR��i«��¬£���� �I«��%���£��S��¡e� ¥ xD���I§­�S�N�©�7°i�©£��g§¨��¤G���¯�����/���\�G§­����L²p³÷ V a ô�õ 8¬ø ôc´ µ öWV�¶�·¸ :-è ¹ ìMº èS@ ¦ ¹ ¦�§^¦ º ¦ ± §­�S�i���N�[¹ Ï º �s¡ ¢ ��£�« » @p¤?@ ¾ ç» ¶ ø�¼�÷K½ ó ¼ ¶ 8¬ø ôc´ µ öWV�¶�·¸ : ¦ ¹\i º ¦�§ @ ¦ ¹ ¦�§ i ¦ º ¦�§ §­�S�i���N�[¹ Ï º ��¡ ¢ ��£�« » @p¤?@ ¾ ç¾i�¬¤®���©�D� �%���²�¿YÁÀ�È.��úFÂd��Çc³�*óÇ@ZeYÄ°$�ÍãeÈ.û}��À�Z´È}�N�º�'§­��£�¤G���©�S£kÅmL ¡ ¢ å � Ê(»;Ï®»� �\��¤¸�|� �%���Å�µËï$·#CÈ���N�S£:µ0¹ ì ï$·�§­�S�D����¡+�C¹ �s¡ ¢ ç

�Q���Q�oà á A`� Æ ��5v�.�C�8�6ÇÒ�¿��9@;rÈ 5?�����������67 � �2�87K�éÂ�G�\ê­ÉsÊË6Ì^Í µ0¹ Ï H+·d´��º�]Î�úFYÁÀóû�À�Z�ã,Ã$��ÏSúFÎ~��Y¶��ZeÂ�úeû�Y¶���vÀ���ÇcÈ[ZeÀ�ÃP�xÂ�ú����aÐB� �����-´��¬���PÑ��¬�±���§­�S�N�©�7°i�X°i���¬£B�©£QÑ��#H��G«ÐL¯°i� � �w³��\�S´%��´��©�©� �yR »~Ê Ð5� ���-�S� ��¤G�©�(���T� ���\£��I�cû�Y¶��ú�È<�{|���¡+³��N� � ï Ï���N�S£:µÑ¹ ì ï$· � °i���¬�\��ïl�N�d«����7°i£w§­�\�S¡ � ����³����S´%��´��©�N�N�yR5«��N�\� �\�©´���� �©��£?H �PÑS�¬��« § µ�¬I·Tç��Á�N���³��\�S´%��´��N�©� �yR�Ð Ï � �������S�Ð�Sê�É ÊË6ÌKÍ µ0¹ Ï H+·À�\�G������£��?�_°FÀ�Â}�X²��{|���¡+³��©� � ï Ï � ��� « § µ�¬I·W \� Êg»�Ï®»���´��S���F°i���N¤¸�"£���� ���©£��/¤¬��£"´$�X���\����¡e�¬« ¥ � ��¤¸�+�I¡+���N�©¤¬�N�S���À�}|���¡e³��©� � ï Ï � � ¡º�zR�´��D¤¸���S�\�¬£´QRB�²¤¬��¡+³����¸�����©�S£%���©�YR<��£�´$�S��£�«��¬«l��«KÑ��¬�\� ���oRB°i���©¤¸���%���/¤G�S¡e³��©�T� �_H�£��7°i�©�G«��S�±��§Ò¹ Ï H Ï��£�«5� ���(���¸�����I��§~�����I�©�¬����£��N£��"���N�S�S��� � ��¡ °i���G£*�����I�S� ��¤G�©�/�N�D�©£QÑS�#H��¬« ¥xD���±�S�����~��§��º�N�®���\£��©£��²���©���S��� � ��¡��N£w� ���N�t¡e��«��¬�À�©�t���<¤¬�S£��\� �\��¤G�(��£���³�³��\�P|��N¡º�����©�S£� �}�����w�¸���\�S�G�5¤G�S£�¤G�¬³������N�S£:µÑ¹ ì ï$·Íç£Ó*�S�\�B§­�S��¡+���©��R;��°���� �zR����%�����èÔ����S�©�¬��£�§­��£�¤T� �N�S£�?L ¡Q¢ å � Êg»;Ϭ»� �©�]��£$çOÔoú}�Q��¯ZGÏ#À�Î�ú���ZeÂ���ZeÂÒ¹lãeÈ�°e��Â#Hî�N§��F� ë®ìOÕ æñ�:µ­ï$·fåC)���N�S£:µ0¹ ì ï$· N @Bç¬çxD���]�©�¬����£��N£��i���N�S�S��� � ��¡Æ�©�Â���YÑS�G£-��£-��¤G¤¬������¤»RI³���� ��¡e�G���¬��ç~��£�«g�i¤G�S£^Ö%«��¬£�¤G��³%������¡e�G� �G��× Ï

»z×R�

Page 161: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

�%������¤¬¤G�¬�\��� �Aê�ÉsÊË6ÌKÍ µ0¹ Ï H+· Ï ��£�«<¡��������S����³������-�QR�³��������¬�\�©�'�º°i���N¤¸�Ð��°i� � �<³��\�S´%��´��©�©� �yR���F�N�®����� ».Ê × Ï �©����£=ço����³�³����z|��©¡+�����S�~§­�S� ¹e��£�«��¬�]H5ç�xD���ØÃOúFÎ~��Y¶�~û*ZeÎ~��Y¶��Ï#À��X²���§$�d�N�®���\£��©£�����N�S�S��� � ��¡)�©£(� ���N�~¡e��«��¬���©�:� ����£���¡�´��¬�Q��§�� �N¡+�G�~� �rÌ����G���©�G�~������¡+���©�N¤¬�N�S���Â�}|���¡e³��N���S����¤¬�N� ¥

µ0� ���N�©�S�����YRl�\� �\�S£��S�G�±¡+��«��G����§Ø�Ù���Ò�N�®���\£��©£��w°i� � �,¡+���©�N¤¬�©�����g£��S�©�\�������e���©�\�B´$�G�¬£³��\�S³$�����¬«lµÑ�t���©��¡Ç��£�«%Úd�¬¤¬��� ���O� »zs;s#�Kã ���¬���´��Ô��©��£�¤¸���#�}��úFYÄÛ¶� »Ps;s;s · ¥ Ýo£<�����©��¡e��«��¬��Ö%�����i�¤¬�N�®��£I����¡+³��©����§�� ���]«��¬���N���G«g�\��E¬�F�N�«�� �®°i£I§­���S¡ �D£��S�©�\�}�¶§­���¬�F��� ��¤G�©� ã � ���¬£(�®��¤¸�g³$�S�N£��Â�©£/� ���� ��¡+³��N�d�©�D�N£�«��G³$�G£�«��¬£����YR<�\�¬�N�¬¤G���¬«5°i�N����³����S´%��´��©�N�N�yRxÐ ã � ���G£*��£*��«KÑS�G��� ����R5���G³��¼��¤¬�¬�D�¬��¤¸����G�©�G¤G� �G«q³��S�©£��D°i� � �*�"«��©�\�yR<�}|���¡e³��©�/��§Q�N���D¤¸���S�©¤G� ã ��£�«�Ö%£%���©�YRe� ���g¤G�S���\��³�� �G«*����¡e³��©�/�N�³��\�ÒÑ��N«��¬«����t� �����©�®����£��N£��d���N�S�S�\�N� ��¡ ¥ xD���N��¡e��«��G���N�Q��� ����£��S�¬�~� �%��£��������S�\�©�S�N£%����¡+���©�N¤¬�N�S���£��S�N���]¡+��«��G���\�©£�¤G���¬��¤¸�±«��©���yRI�}|���¡e³��©�]�©�À¤¸���S�\�¬£±´QR(� ������«KÑS�G��������R(°i�N���g§­���N�cH�£��7°i�N�¬«��S����§� ���t�¬£����©���X� ��¡+³��N�i� �������¬�]� �%��£SH�£��7°i�©�¬«��S�t�S£��YR"��§{�����t�¬�����N�©�G�F�}|���¡e³��©�G��� �����������X�©�®����£��G��%���Q«�� �®°i£±� ������§¨���Q�©£-������� ��¡+³��N���\�OÌ����¬£�¤¬� ¥ �¯�©����§��S�����\�¬�����N� �����©�\�/���S�©«-�©£-�����©����� �\�S£��S�G�¡e��«��G� ¥ ·

�ÜÖ%£����¯£���� ��Lw�©�IH��5� ���Ý�~�¬�\¤¬�G³�� �\�S£Á���©���S��� � ��¡à�F� ���*�©�®����£��N£��1���©�S�S�\�N����¡+��°i���©¤¸��°��¤¬��£����N«��¬�F°i�©�N�����»Ì����©�\�X���%���?� ���'Ì��%��£����N�yRg¹ ì ï<´��i´$����£�«��G«<�®°��zR"§­����¡ÞE¬�G���ºµÄ���]�©�¬���\�]¡e�S�\���§~� ���/� �N¡+�®· ¥ �l�I�������¯���zR²� �%���¯�"«��©��� �\�©´������©�S£vHÒ�©�Qß^Ô0àcZMZO°���Ze£¹q�N§Xè ¹ ì ïFèKÄáß±§­�S�X���N�Åï°i���N¤¸�q�%�zÑS�(£��S£^E¬�¬�\�+°��¬�N�S���¯��£�«��¬��H Ï ��£�«�°��(���G�\�����©¤T�X�S���X�������¬£�� �N�S£5� �e�©�®����£��N£��±��£�«��G�ß��ª�S����«�«��©��� �\�©´������©�S£�� ¥¥â §?¤¬�S���\�����{«��©���yR��}|���¡e³��©�G�t«����7°i£w§­�\�S¡­ê­ÉÝÊË6ÌKÍ µÑ¹ Ï H+·i£��G�¬«B£����� ��� �©��§�R¿è ¹ ì ïFèeÄkß�ç

�Q���Q��� A �uã ��7^�åä ã2æ ���@5ç�è9ké ��5vê Æ ��5v�.�C�8�6Ç A æ Ç ã �.��9<;C1�@ë#ìÒí�îFï�íØëFð�ì�íGðÑíOì�íOñ<òGíØó2í6ìôíGõÙíOîNö2öô÷<í ¤K� ñ<ë#ì�øÞóèíOî HSù í�îFì�ñ<ú�ñ^ûüî ù û#ë#ì�úIö¯÷^ø EGF,ý ðÑì�ë#ø � íOò�ö¯úIë#ñ�Kþ�ÿ úIñ_� úIû��<ìôí#s þ�ÿ^þ ¾ íGò�î ùIù öô÷@îNö EGFcý ö¯î H íGïvîFï úIñ<õ��^ö î��<îNö*î�ïôíGö��sîFñ�� î��<ú�ï8ö¯ì�ú���^öôú�ë#ñ H ë Ñ íGì� ç.x ÷^í î ù ûFë#ìôúÄö¯÷<øBòGë#øxõ��^ö¯íGïØö¯÷^í Ñ íGòGö¯ëFì + ó£÷<ú�ò*÷súIï¥öô÷<í î Ñ íGì¯îFû#í ù ëKòOîNö¯úIë#ñ%ëFðvö¯÷<í� ù î��Ùí ù��ñ<ë#ì�øhî ù ú E í ���Cõ�ë#ú�ñRö¯ïüú�ñ����<ñ��<íGì H Ï�öôì¯îFñ<ï8ðÑë#ìôøxï + öôë�ë��^ö*îNú�ñjî Ñ íOòGöôë#ì��\Ï�îFñ��jõ^ìôí �^ú�òGöôï�<ï�ú�ñ<û ö¯÷^í ù ú�ñ<íOîFìèð��<ñ^òGö¯úIë#ñ@î ù �<í Ö ñ^í ��� R � ç

� í ïô÷<ëMó2í �'ú�ñ � ÷@îFõ^öôíOì � ö¯÷<îNöSöô÷<í EGF,ý î ù û#ëFìôúIöô÷<ø úIïhî�óèíOî H ù í�îNìôñ<úIñ<û î ù û#ë#ì�úIöô÷<øðÑë#ì ù ú�ñ^í�îFì�ö¯÷<ì�íOï�÷<ë ù �pð��^ñ<òGöôú�ë#ñ<ïüú�ñjîsñ<ëFú�ïôí � ðÑì�íOíhï�íGö�ö¯ú�ñ^û þüx ÷<í�ðÑë ùIù ëMó£ú�ñ<û�ö¯÷^íOë#ì�íOø ïô÷<ëMó£ïö¯÷<îNöhúÄðCî]ïôøSî ù�ù ðÑìôîFòGöôú�ë#ñ ëFð6öô÷<íÝí | îNøSõ ù íOïxú�ñ��ÞîFìôíÝî ½ íOòGöôí ��� R øSî ù úIòOú�ë��<ï_ñ<ë#úIïôí �ÕEGFcýó£ú ùIù ï�öôú ùIù û#íOñ^íOì¯îeö¯íOï2î�÷ R õ�ëFö¯÷^íOïôúIï2ó£úÄö¯÷xñ<ë#ñ<ñ<íGû ù úIû#ú� ù íDî�� Ñ îFñRö*îFû#í�õ<ì�ë Ñ ú��<í �xö¯÷@îNöçöô÷<í£ú�ñ<õ��Kö�<úIï�ö¯ì�ú���Kö¯ú�ëFñ H úIïQï�� â òOúIíOñRö ù�R ïôøxëcëFö¯÷ þ¡\¢ ô ÷ õ�ô � ù{û����çÀ Ï � @�¤u@�� ú�È�°%Y¶�}� � C � ï Ã Ï � à � Ï ç®ç¬ç Ï � ï! Ï � �#" �¬ú�ÃP�}�ØZ�� Y�ú " ��Y¶�ô°

$z×Qs

Page 162: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

8 %�9 µ · : ìôí�î ù ñ&�^ø'�ÙíGì ¤?Ä �ï¯îFøxõ ù í(� C*),+ à Ï�- Ã/. Ï 01010}Ï ),+! Ï2- . ëFð ù î���í ù í ��í | îFøxõ ù íOïQó£úIöô÷ + ¨43 ¡ ¢ Ï- ¨ 365 Ê($;Ï1$�7�<ú�ï8ö¯ìôú����^öôú�ë#ñ H ë Ñ íOì��8xµ ·:9 µ ·<; ì�í�î ùÐÑ î ù �<í ��÷ R õÙëNö¯÷<íGïôú�ï>=? + �$ þ @ A · + CCB D�E�à H GF&�H- D + D� þ TJI�KML V�V $&C $;Ï101010»Ï2N @ A · � ¨ C ïôúIû#ñ? + ¨ �PO + ¨ O § g Ãÿ^þ K A · µ K % ÷ R õ�ëFö¯÷<íGïôúIïQ=? + �SR ºJì + ó£÷<íGìôí º C TU T UWVYX� úIû��<ì�ís þ�ÿKL]x ÷<í ¤K� ñ<ëFìôø ó2í�î H�ù í�îNìôñ<úIñ<ûgî ù ûFë#ìôúÄö¯÷<ø EGF,ý�þ

��ÏSúFÎ~��Y¶�*Ã[ZvÀ���Ç���úeû³Ç�+ D 3 « § ¬ �\0^] �}�KH " ��ú�°FÀ�Ã}�0ÂÀ " _ �0À�Z´È�Za`��� � à _ û³Ç\��ÇKú��M�Z� H � @ Ã� 0b _ �Q�@ZMÃP�"��ÇKú�� ßdc äcú�È�° ¹ 3 ¡Q¢ úFÂ��­Ã _ û³Ç���ÇKú��`ßd@ ¬ ¦ ¹ ¦ ± ú�È�°àú��QÎhZMÃ}�`Ðfehg ��ÏSúFÎ~��Y¶�*ÃÀ�È � °cZüÈ[Z´�ØÃOú��0À�Ã��{² - D ¹ ì + D � ÄBß Ï ZrÇW��Â�� Ð e @ �jiÛ X�UWkaU V 0�l ÇW�}ÈüEGFcý ¤ Ï\�rÏ H � Â��}� _ Â{È^ÃàúÇe²³�@Z´��ÇW�*Ã�À�à = L,« § ¬ ��m n Ê($;Ï1$ON ZrÇcÀóû³ÇvÇKúeÃ\úR°�`´ú�ÈW�óúMàe�_ú��2Y¶��úeÃ}� iÛ XoUWkPUWV _ È�°e��Â�H 0� KaIpI�T ; ÔÕR�Lrqë ù �<íGì � ï£ú�ñ<í Ì �<î ù úÄö R;� ðÑëFì£îFñ Rs+ 3 « § ¬ �2óèí­÷@î Ñ í

Ot=? + �:O C O � ì + O¦ � ¦ ± ¬ @ ¦ � ¦ ± ¦ + ¦�§¦ � ¦ ± ¬ @ $;ÏîFñ���ö¯÷f�<ï�= úIñ��<íGí �%øhîFõ^ï « § ¬ �èúIñ,öôëun Ê($;Ï1$ONv0�¥ëMó óèígïô÷^ë�ó ö¯÷<îNö(=�÷@îFï6ö¯÷^í��<íOï�ú�ìôíw�]î�� Ñ îFñRö*îFûFí þ � ú�ñ^òOíx= I + D � 3 n Ê($;Ï1$ON`îFñ��y- D 35 Ê($;Ï1$�7SÏ^ó2í­÷@î Ñ ídOt=? + D �rÊz- D O C $'Ê{- D =? + D �{Ï<ïôë$� |D�E�à H �F&�:Ot=? + D �rÊ{- D O C $� |D/E�à H GF&� H$�Êz- D =4 + D �/� C $� Ê~} B D/E�à H �F&�H- D ,� ì + D �� ¦ � ¦ ± ¬ � 0x ë¿õ<ì�ë Ñ í]öô÷<í�ö¯÷<íGë#ìôíGø úÄö�ö¯÷f�<ï ï�� â òOíGï öôë ï�÷<ëMó}ö¯÷@îNö Bd�§�� � Õ?� D�� ¦X§ � T�� � § �U T U V Ä iÆ UWkPU V 0 x ÷<íñf�<øxíOì¯îeö¯ë#ì¥ëFðvö¯÷<í ù íGð�öØïôú��<í_ú�ï�� ì ö B D�E�à H �F&�Y- D + D ÷ C � ì + C*B ¢¨ E�à O + ¨ O § C ¦ + ¦ §§<0 � ïôúIñ<ûö¯÷^í�ð0îNòGöQö¯÷<îNör ¤ Ê�$P��® C?¤ Ï<öô÷<í��<íOñ<ëFøSúIñ@îNö¯ëFìDú�ï

¦ � ¦ ± C } ¢|¨ E�à ö O + ¨ O § g à ÷ ± � Ãq ± C } ¢| ¨ E�à O + ¨ O § � Ã

q ± C ¦ + ¦ § q ±§ 0� í�ò�îFñpöô÷<íOì�íGðÑë#ì�í�ìôí�ó£ìôúIöôíSöô÷<í ù í�ð�ö_ï�ú�<í�îNï ¦ + ¦ §§a� ¦ + ¦ § q ±§ C ¦ + ¦�§ Ï`îNñ��pö¯÷f�<ï ë��^ìüû#ëRî ù úIïö¯ëhïô÷<ëMó ö¯÷@îNö ¦ + ¦�§ Ä iÆ UWkPU V 0 Ô#RsLrqë ù �<íGì � ï6ú�ñ^í Ì �@î ù úIö R úIö¥ï�� â òGíOïCö¯ë�ï�÷<ëMóåöô÷@îNö + ì ¹�Ä i Æ Ï

$P� ä

Page 163: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

ó£÷<úIò*÷%ó2íañ^ë�óTõ<ìôë Ñ í þé íGö�� à C 5 ),+ D Ï2- D�. 3 � L - D ¹ ì + D � ÄTß 7gîFñ�� ù íGöQ� Æ C �y��� à 0 x ÷^í'�<í Ö ñ<úIöôú�ë#ñ ëFð�� Ãú�øxøxí �<ú�îNö¯í ùYR~R úIí ù �^ï B D\�1� � H GF^�Y- D ¹ ì + D � Ä>H �� à � ß 0 Ó ë#ì�íOë Ñ íGì � ï�ú�ñ<òGíDí�îNò*÷ ¦ + D ¦�§ @'¬ Ïa� RL'që ù �<íGì � ï ú�ñ^í Ì �@î ù úIö R ó2íD÷@î Ñ í�- D ¹ ì + D � Ä Ê ¦ + D ¦�§ ì ¦ ¹ ¦ ± Ä Ê ¬ ¦ ¹ ¦ ± ðÑë#ì íOîFò*÷ ),+ D Ï2- Dw. 3 � Æ 0� úIñ<òOí­í�îFò*÷�í | îFøSõ ù í­ú�ñ�� Æ ÷@îFï�óèíGú�û#÷RöØîNöØøxë#ï8ö Ã� �<ñ��<íGì H Ï@óèía÷@î Ñ í H v� Æ � @ Ê/�� Ï[îNñ��÷<íGñ<òOí+ ì ¹�C |D/E�à H �F&�H- D ¹ ì + D � C |D\�1� � H �F&�Y- D ¹ ì + D � i |D\�1� º H �F&�Y- D ¹ ì + D �Ä H �� à � ß Ê H �� Æ � ¬ ¦ ¹ ¦ ±Ä }�$�Ê Ðfe� � ß Ê ÐfeÁ¬ ¦ ¹ ¦ ±�Ä ÿNß× Ê ß× C ß � Ï

ó£÷<íGìôí�ö¯÷<í�úIñ<í Ì �@î ù úIö R Y$JÊ ÊH�� � Ä ÚÛ ðÑë ù�ù ëMó£ï­ðÑìôë#ø ö¯÷<ís��ë��<ñ��jëFñ Ðfe îFñ��?öô÷<í�ð0îFò�ö_ö¯÷@îeöß=@'¬ ¦ ¹ ¦ ± 0�Q���Q�0/ �6�¥9c9<���6Ç �:9 A æ8æ ä ã Ç2� 9<;6���� ñaöô÷<ú�ï�ï�íOòGöôú�ë#ñ­óèíÒïô÷^ë�ó?÷^ë�ó �N���,�e���^J<�,����� òOîFñr��í��^ïôí �aó£úIö¯÷ EGF,ý ö¯ë�ë��^ö*îNú�ñaîFñüî ù û#ë#ì�úIöô÷<øðÑë#ì ù í�îFì�ñ<ú�ñ<û ù ú�ñ<íOîFìèöô÷<ìôíGïô÷<ë ù �%ð��^ñ<òGöôú�ë#ñ<ïQúIñ�ö¯÷<í�õ<ì�íOïôíGñ<òOíaëFð�øhî ù ú�òGú�ë��<ï2ñ<ë#úIïôí þx ÷<í6î ù û#ë#ì�úIöô÷<ø ðÑëFì ù í�îNìôñ<úIñ<û�ïôú�ûFñ? ¹ ì + � ó£úIöô÷Sì�íOï�õÙíGòGöèöôëüî ß�� û#ëcë^���<úIï�öôìôú���^ö¯úIë#ñ H ë Ñ íGì« § ¬ �èú�ï£îFïQðÑë ùIù ëMó£ï L

� Ú ìôîMó îDï¯îNøSõ ù íS� C�),+ à Ï2- Ã/. Ï1010 0OÏ ),+! Ï2- . ëFð g ù î��Ùí ù í �Øí | îFøxõ ù íGï ðÑìôë#ø ê­É ÊË6Ì^Í ¹ Ï H �\0� ¾ �<ñ �N���,�e���&�.�,����� ë#ñ��?ó£úIöô÷ õ<îFì¯îFøxíGöôíOì�ï � C��Û Ï À?C iÛ XoUWkPUWV Ï�� C ¹Æ k ¹ �<ï�ú�ñ<û EGFcýîFïQöô÷<íCóèíOî H�ù íOîFìôñ<úIñ<û î ù û#ë#ì�úIö¯÷^ø þ� í ñ<ëMó*�<í�ö¯íOì�øSúIñ<í òOëFñ<ï�öôì¯îFúIñ,öôïCë#ñ�ö¯÷<í ïôîFøxõ ù í_ï�ú E í g îFñ���öô÷<í øhî ù ú�òGú�ë��^ïØñ<ë#úIïôí_ìôîNö¯í Ð�<ñ��^íOìQó£÷<ú�ò*÷�ö¯÷^ú�ï£ú�ï£î ï��^òOòOíGïôï8ð�� ù îNñ�� í â òOú�íGñRö ù í�îFì�ñ<ú�ñ^ûgî ù û#ëFìôúIöô÷<ø þ

� í Ö ìôï�öDñ<ëFö¯í¥ö¯÷@îNöDïôú�ñ^òOí H ú�ï ß�� û#ëKë&�SðÑëFì ¹ ÏKó2íC÷@î Ñ í6öô÷@îNö ß=@'¬ ¦ ¹ ¦ ± 0<�!�<ì�öô÷<íOì�øSëFìôí �ïôúIñ<òOí � C �Û Ï �èù îNú�ø s þ�ÿ ú�øxõ ù úIíOïCö¯÷<îNö_íOîFò*÷��<ú�ï8ö¯ìôú����^öôú�ë#ñ HJI ó£÷<úIò*÷jú�ïaû#ú Ñ íOñpö¯ë EGF,ý � R�e���c�e���KJ<�c����� ÷@îFï �Z� HJI � @ Û� 0 é íGö���� �¡�z�Ùí6öô÷<íCò ù í�îFñ�í | îFøSõ ù íOïDîFñ��u��¢ C �s�£���ö¯÷^í¤�<ú�ì8ö R í | îFøxõ ù íOïDúIñ¥�¦0 � ð Ð�@ � iÚ Æ X�UWkPUWV îFñ�� g Ĩ§ Ý XoUWkPU V� i ù ë#û Æ © Ï^öô÷<íOñ î_ïôúIøSõ ù í � ÷<íGìôñ<ë ½��ë��<ñ���ú�øxõ ù ú�íOï�ö¯÷@îeöCó£úIö¯÷�õ<ì�ë��@î��^ú ù úÄö R îNö ù íOîFï�öª$5Ê ©Æ ó2íg÷@î Ñ í«Ot��¢�O @ � iÃ Ý X�UWkaUWV g 0 x ÷f�<ï �

$P�^$

Page 164: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

ó2í_ò�îFñsîFõ<õ ù�Rdx ÷<íOë#ì�íOø s þ �Só£úIöô÷ Ðfe�C � iÃ Ý X�UWkPU V ãÙïôë�íOîFò*÷só2í�î H ÷ R õ�ëFöô÷<íOï�ú�ï�= I + � C º I ì +û#íGñ<íOìôîNö¯íw�M� R"EGFcý ÷@îFï2î�� Ñ îFñRö*îFûFí iÛ X�UWkPU V �<ñ��^íOì HJI 0 � ë#ñ^ïôí Ì �<íGñRö ùYR�� � R=x ÷<íGë#ìôíGøSï#s þ ×üîNñ��s þ � �2�e�Ù�,�e���f���,����� í â òGú�íOñRö ùYR ë��Kö¯õ��^öôïxî Ö ñ@î ù ÷ R õÙëNö¯÷<íGïôú�ï#=? + � C ï�ú�û#ñ Å + �aó£÷<úIò*÷ ÷@îNïøSîFìôû#úIñ ù íGïôï6ö¯÷<îFñ6��ë#ñ]îNö�øxë#ï8ö­îFñ �Û ðÑì¯îNòGö¯úIë#ñ�ëFð��¦0 � ú�ñ^òOíyOt����O.úIïCí�îFï�ú ù�R ïôíGíOñ�ö¯ë¥�Ùígîeöù íOîFï�ö Æ ÏOó2í2÷<î Ñ ívö¯÷@îNö�öô÷<íçøhîNìôû#úIñ6ëFð�=­ú�ï ù íOï�ï�öô÷@îFñ(�Dë#ñaîNö�øxë#ï�ö�îNñ �Æ ðÑì¯îFò�ö¯úIë#ñCëFð����¬0 � ï�úIñ� ÷@îFõKö¯íOì �G� ö¯÷<úIïçøSíOîFñ<ïçö¯÷@îNöÒó2í¥òOîFñ�îFõ^õ ùYR øxíGöô÷<ë^�^ïçðÑì�ë#øÞö¯÷^í�öô÷<íOë#ì R ëFð��@îeö*î � �<íOõ�íOñ��^íOñRöï�öôì��<ò�ö2�<ìôî ù ì�ú�ï H øSúIñ<ú�øxú E îNö¯úIë#ñ­ Ô îNì�ö ù íGö�öCîFñ�� � ÷<îMó2í ��x î Rcù ëFì � $Ps;s�sKã � ÷<îMó2í ��x î Rcù ëFì �}�~úFYÄÛ¶�$Ps;sF�<�2ö¯ëª��ë��<ñ���ö¯÷<í�íGìôìôëFìQëFð�=¥�<ñ��<íGì H 0

¾ íGò�î ùIù ö¯÷@îNö�ö¯÷^í Ö ñ@î ù �e���,�e���f�.�,���e� ÷ R õ�ëFö¯÷^íOïôúIïØúIïQ=? + � C ïôú�ûFñ? Å + ���èó£÷<íGìôí Å + � Cº�ì + ú�ïçîCòOë#ñ Ñ í | òOë#ør�<ú�ñ@îeö¯ú�ëFñ ëFð ÷ R õÙëNö¯÷<íGïôíOï�= I + � C º I ì + 0 � ú�ñ<òGí£í�îFò*÷ Ñ íOò�ö¯ë#ì º I ïôîNö¯úIï Ö íGï¦ º I ¦ ± @ ÃX Ï�� R%Ó ú�ñ H ë�ó£ï H ú � ï¥úIñ<í Ì �@î ù úIö R óèí�÷@î Ñ í�öô÷@îNö ¦ º ¦ ± @ ÃX îFïDó2í ù�ù�þ`� íCõ^ìôë Ñ íw� ö¯÷^íðÑë ùIù ëMó£ú�ñ<ûaö¯÷<íGë#ìôíGøBú�ñ � ÷@îFõKö¯íOì � x ÷<íGë#ìôíGø �Kþ �<� L¡\¢ A I�K Ae� ®�¯�®��çÀ Ïüú�ÈW²d`´úFY _ � � @�¤è@¨� ú�È�° Y¶�}�[° " �A��ÇW��û�Y�úeÃ*ÃSZ���� _ È.û}�0À�Z´È^à 5 +6±mº�ì + L ¦ º ¦ ± @ ÃX Ï + 3 « § ¬ �\7f0�l ÇW�}Èb�»ú��³² Í � @ Æ UWV:X Û ¢´ º Ï ZrÇW��Â��\�»ú��µ² Í � À�Ã���ÇW� ð0îNö �ïô÷<îNöôöôíOìôúIñ<û¤�<úIøSíGñ<ïôúIë#ñ_ëFð ° îeövïôòOî ù í ÍÓúeÃ¥°e��¶ZÈ.�ô°­À�È�·]�OÛÁàRÛG·Õ·v¸�úFÂ{�0Y¶�}��� í�öÒî ù�þ³·º¹<»<»eµ:¼�¸�úFÂ{�0Y¶�}���ú�È�° b ÇKú�Z���Ô l ú�²NY�ZeÂ2·[¹<»<»<»a¼ b ÇKú�Z���Ô l ú�²eY�Ze íGö¥î ù�þµ·�¹<»<»�½»¸FÛ

¾ ÷<í�ðÑë ù�ù ë�ó£úIñ<ûüöô÷<íOë#ì�íOøBúIïDðÑìôë#ø¿ Ô îNì�ö ù íGö�ö�îFñ���ÀK÷@î�óèí\Á ¾ î1 ù ë#ì ÃÄ$1Å�Å�Å<�\ÆÇrÈ A I�K A&É ®�¯,Ê&Ë~Ì/Í L�K ·PÎ A ·�· L %4Ï�Ð È L ½rA<Ñ Ç L ¸?Î I�K&Ò Ê^®�®�®�Ó ]?Ô�Õ ° " Ô ú=û*ZeY�Y Ô û Õ À�Z´È%ZYÖQÂ Ô úFYÁÔ`´úFY _ Ô °¤Ö _ È.û Õ À�Z´È^ÃxZa` Ô Â­ÃMZeÎ Ô °cZeÎ�úFÀ�È�É Ï Y Ô�Õ¬× " Ô ú °FÀ�Ã Õ ÂÀ " _ Õ À�Z´È?Za` Ô Â�ÉÙØ 5 Ê($;Ï1$�7SÏ Y Ô�Õ�ÛÚ ),+ à Ï2- Ã/. Ï1010 0OÏ ),+! Ï2- . " Ô ú à Ô�Ü _ Ô È.û Ô ZYÖxY�ú " Ô Y Ô ° Ô ÏSúFÎ>Ý�Y Ô Ã�°FÂ�ú�Z]ÈsÖÂ*ZeÎ ×dÏ ú�È�°�Y Ô�Õ=? + �SÚ Ã�ÀÄà�È Å + ��� Ö�ZeÂQÃMZeÎ Ô Å 3 ° 0�Þ Ö = ÇKúeÃ�Î�úFÂ�àFÀ�È�Y Ô Ã*Ã Õ ÇKú�È � Z´Èàú Õ ÎhZMà Õpß«Ô ÏSúFÎ>Ý�Y Ô ÃÀ�È �rÏ4Õ Ç Ô ÈyZvÀ Õ Ç'Ý�¯Z " ú " À�YÁÀ Õvà ú Õ Y Ô úeà Õ�$�Ê ×dZ Ô ÇKú�` Ô�Õ ÇKú Õ � ì�á � © â�ãW� Õ nä=? + � åÚC-få À�Ã\ú Õ ÎhZMà Õ

ßg¿æèç �g ) ù ñp ÿ × Ù � g � ù ë#û�v� � � g � æ ù ñÄX× � × ���³Ï vÅ þ $ ä �ZrÇ Ô Â Ô ) Ú Ö»ú Õ ² ,� � $1é<��0� í¥÷@î Ñ í�öô÷@îNöê=S÷@îFïÒøSîFìôû#úIñ ù íGïôïvö¯÷@îNñ��Cë#ñSîNöÒøSë#ï8öÒîFñ �Æ ðÑì¯îFò�ö¯úIë#ñgëFðÙöô÷<í�ò ù íOîFñxí�ë^îFøxõ ù íGï���rÏ,ïôëaóèí¥øhî1Â_ö¯î H íQß � g ö¯ë(��í �Æ úIñxöô÷<í6î���ë Ñ í£ö¯÷<íGë#ìôíGø þSì ëMóáúIðÙóèíØîFõ<õ ù  ¾ ÷<íOë#ì�íOøíÅ þ ÅîFñ���ï�ë ùYÑ íØðÑë#ì g öô÷<íCú�ñ<í Ì �@î ù úÄöîÂgë��^ö¯îFú�ñ<íw�s�<Â�ïôí�öôöôú�ñ<ûdY$P�Òöôë��ÙíCîeöDøxë#ï�ö ç ÏKó2íCë��^ö*îNú�ñ�ë��^ìøSîFú�ñ�ì�íOï�� ù ö1Æ

$P�#�

Page 165: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

ÇrÈ A I�K A&É ®�¯,ÊoÊC�vÀ Ï ��ï ¤ ï � ú�È�° Y Ô�Õ�× " Ô úp°FÀ�Ã Õ Â�À " _ Õ À�Z´ÈTZa` Ô Âg« § ¬ � ZrÇcÀóû³Ç?À�Ãß<Ô0àcZMZO°rÖ�ZeÂ_¹ 0rl Ç Ô úFY àcZeÂÀ Õ ÇcÎ>° Ô ÃPû�ÂÀ " Ô °cú " Za` Ô _ Ã Ô Ã'g Ú ðPòñ ö XoUWkPUWVi � ÷ Æ2ó Ô ÏSúFÎ>Ý�Y Ô Ã�ú�È�°Z _ Õ Ý _ Õ Ãàú�ÈÓçOÔoú�Ý<Ý�¯ZGÏ#À�Î�ú Õ ZeÂ�Ö�Ze ¹ _ È�° Ô Â × ZvÀ Õ ÇôÝ�¯Z " ú " À�YÁÀ Õ,ày$JÊ ×pÀ�È Õ Ç Ô Ý�Â Ô Ã Ô È.û Ô ZYÖÎ�úFYÁÀóû�À�Z _ ÃfÈ[ZeÀ�Ã Ô ú Õ úhÂ�ú ÕîÔ Ð Ú ¾ ö ç ì iXoUWkPUWV ÷ 0õdöH÷ ø �1ù¬ú�ûyù¬ù`�O~xü�Q�0/¥��� ý ã 14365v�.�87 ã �¿þ¿�:9<;>=¿� æ �8�6� A æ Ç ã �.��9@;C1\7¾ ÷<í���ë��<ñ��<ïçë#ñgïôîFøSõ ù íQòOëFøSõ ù í�ëKúIöîÂ_îFñ��gøSî ù úIòOú�ë��<ï�ñ^ë#ú�ï�í£öôë ù íGì¯îFñ^òOíQëFð�ë��<ì �e���,�N���KJ<�,����� Á�@îFï�í ��î ù û#ë#ì�úIö¯÷^øSïçû#ú Ñ íOñM�f ¾ ÷<íGë#ìôíGøÿÅ þ $�$CîFì�íØì�íOøSîFì H î�� ù ÂgïôúIøSú ù îFì`öôëüö¯÷<í��Ùë��<ñ��<ïèó£÷^ú�ò*÷ò�îNñy�Ùí_ë��^ö*îFúIñ<í � ö¯÷<ì�ë��<û#÷�î�ñ<îNö2�<ìôî ù��Ù��� òOë#ñ Ñ íGìôï�ú�ë#ñ�ëFðvöô÷<í_ë#ñ ù ú�ñ<í ¤ Á�ñ<ë#ìôøÜî ù û#ë#ìôúÄö¯÷<øxïú�ñRöôìôë&���<òOíw���<Â��Cìôë Ñ í�à é úIö�ö ù íOï�öôë#ñ<í¥îFñ��dÀcò*÷&���<ìôøSîFñ<ï£îFñ���ï�ö����<ú�íw�s�<Â��CíOñRö¯ú ù í¥îFñ�� é úÄöôö ù í�Áï�öôë#ñ<í þh� ï­ï�ö¯îNö¯íw� úIñ ¾ ÷^íOë#ì�íOø �Kþ ��ëNð � ÷@îFõKö¯íOì � Ã��Cìôë Ñ í�à é úIö�ö ù íOï�öôë#ñ<ígîFñ��zÀKò*÷f���<ì�øhîNñ<ïõ<ì�ë Ñ í �pöô÷@îNöüöô÷<íhëFñ ù úIñ<í ¤ Á:ñ<ëFìôø î ù û#ë#ìôúÄö¯÷<ø øhî H íOïüîNö_øxë#ï�ö P ñ ö X�UWkaUWVi ÷ Æ/ó øxú�ï8ö*î H íOïaë#ñù úIñ<í�îNì ù Â�ïôíOõ<îFì¯î�� ù í'�@îeö*î þ À&���<ïôí Ì �<íOñRö ù Â��CíOñRö¯ú ù íaîFñ�� é úIö�ö ù íGï�öôë#ñ<í­í�ëcö¯íGñ��<í ��ö¯÷^í îFñ@î ù Âcïôú�ïðÑìôëFø ��Cìôë Ñ ízÔ�Õ úFYÄÛ Ã�$:Å�Å � �_îFñ�� òOë#ñ^ïôú�^íOìôíw�kî�ïôí�öôö¯úIñ<û]ú�ñjó£÷<ú�ò*÷ ö¯÷<í%í�ë^îFøxõ ù íOïxîFìôí�ñ<ëNöù úIñ<í�îNì ù Â%ï�íOõ@îFìôî�� ù í þ�¾ ÷<íGú�ì�îFñ@î ù ÂcïôúIïx ¾ ÷<íGë#ìôíGø é<�Øú�øxõ ù ú�íOï�ö¯÷@îeö�úIð2îFñ�í�ë^îFøxõ ù í ï�í Ì �<íGñ<òOíòOëFñ,ö¯îFú�ñ^ú�ñ<û�� øhî ù ú�òGú�ë��^ïØíOì�ìôë#ì�ï�ú�ï6õ<ì�ë Ñ ú��<í ��ö¯ë�öô÷<ígë#ñ ù úIñ<í ¤ Á�ñ<ë#ì�ørî ù û#ë#ìôúÄö¯÷<ø¥ÃÙöô÷<íOñ�ö¯÷^íî ù û#ë#ìôúÄö¯÷<ø ó£ú ù�ù øSî H í�îeö¥øxë#ï8ö

P �� } ¬ ¦ ¹ ¦ ±ß �¦æ � ì ¬ ¦ ¹ ¦ ±ß ��øxú�ï�ö¯î H íGï þ~¾ ë ë��Kö*îFúIñ ����� Á:øxë^�^í ù ��ë��<ñ��<ï�ë#ñ'ö¯÷^í]ëFñ ù úIñ<í ¤ Á:ñ^ë#ìôø î ù û#ëFìôúIöô÷<øxïhú�ñ'ö¯÷^íõ<ì�íOïôíGñ<òOí�ëNðCøhî ù ú�òGú�ë��^ï_ñ<ë#úIïôí�ÃDó2íô�<ï�í öô÷<í ðÑë ùIù ëMó£ú�ñ^û�öô÷<íOëFìôíOø ���^í ö¯ë � �<íGì�îFñ�� � íOïôîaÁ ú�îFñ<ò*÷<ú[ � �<íGì¥îNñ�� � íOï¯îPÁ ú�îFñ<ò*÷<úvÃÄ$:Å�Å#�<�> ¾ ÷<íGë#ìôíGø�é þ ����ÆÇrÈ A I�K A&É ®�¯,Ê����vÀ Ï\ú�� à Ý@Z Õ � Ô Ã�À�Ã~û�Y�úeÃ*Ã��>ZYÖ��Wú�Ý��@À��eÔ���� Ô Â�`FZ�� Ô ���eÀ�Ãè°FÀ�Î Ô �^Ã�À�Z�� ) 0^]?Ô�Õ��" Ô ú��%Z��@YÁÀ � Ô Y Ô úFÂ��@À �càJúFY àcZeÂÀ Õ �cÎòZvÀ Õ � Õ � Ô Ö�ZeY�Y�ZaZvÀ �càêÝ�¯ZjÝ Ô Â Õ À Ô Ã"!�#Y¹%$ � Z��@Y à _ Ã Ô Ã� à Ý@Z Õ � Ô ÔÃ Ô Ã(Z&�cÀóû'� " Ô Y�Z��cà Õ Z(�*)+#-,"$xÀ Ö � À�ÃØàFÀW` Ô �bú.�[ZeÀ�Ã Ô ÔhÖ ÔjÔ'Ô ÏSúFÎ>Ý�Y Ô Ã Ô�Ü _ Ô �.û Ô(Õ � Ô � � Î�ú�� Ô Ãú Õ ÎhZMÃ Õ g�/%Î À�Ã Õ ú�� Ô Ãj·ú���°0#213$%À Ö � À�Ã_àFÀW` Ô � ú�� Ô ÏSúFÎ>Ý�Y Ô Ã Ô�Ü _ Ô �.û Ô ZvÀ Õ � � Î�úFYÁÀóû�À�Z _ ÃÔ Â¯ZeÂ*Ã Õ � Ô � � Î�ú�� Ô Ã_ú Õ ÎhZMÃ Õ g�/ æ « � Î À�Ã Õ ú�� Ô ÃOÛ l � Ô � Õ � Ô Â Ô À�ÃJú54�67�>úFY àcZeÂÀ Õ �cÎ � eZ&�cÀóû'��Y Ô úFÂ��^Ã Õ Z¬úeû³û _ Â�úeû à ç\ú���°dû*Z��1¶2° Ô �.û Ô ×8) _ Ã Ô Ã ðP 39�:� : æ &;� æ � � �'Ô ÏSúFÎ>Ý�Y Ô Ãj·Zú���°dû�ú��Õ ZeY Ô Â�ú ÕîÔ úhÎ�úFYÁÀóû�À�Z _ Ã<�[ZeÀ�Ã Ô Â�ú ÕYÔ Ð Ú � 9 0 $P� ÿ

Page 166: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

� õ<õ ù Âcú�ñ<ûxö¯÷<úIïQö¯÷<íGë#ìôíGøuÃ.óèí Ö ñ��Ýöô÷@îNö�ö¯÷<íGïôí �Ù��� òOëFñ>=FíOì�ïôú�ëFñ<ï6ëFð`öô÷<íüëFñ ù úIñ<í ¤ Á:ñ^ë#ìôøî ù û#ë#ìôúÄö¯÷<øxï�÷@î?=#ígï¯îFøxõ ù í_òOë#øxõ ù í�ëKúIöîÂÝîFñ���øSî ù úIòOú�ë��<ï¥ñ^ë#ú�ï�í_ö¯ë ù íOì¯îNñ<òOí'��ë��<ñ��<ïCó£÷<ú�ò*÷]îFì�ííOï�ïôíGñ,öôú î ùIù  ú�<íGñRö¯ú�òOî ù öôë�ö¯÷<í���ë��<ñ��^ï­ûFú@=#íGñ�ðÑë#ì6ë��<ì �N���,�e���^J<�,����� ÁJ�@îFï�í �?î ù û#ë#ìôúÄö¯÷<ø þ6� íñ<ëFöôíCö¯÷@îNö�ö¯÷^íOïôíaî ù û#ë#ìôúÄö¯÷<øxïD÷@î?=Fí­öô÷<í ��íOï8ö>��ë��<ñ��<ï¥ëFñ øSî ù úIòOúIë��<ïèñ^ë#ú�ï�í�öôë ù íGì¯îFñ^òOí�ëFðvîNñfÂH ñ<ëMó£ñÝòGë#øSõ��^ö*îNöôú�ë#ñ<î ù�ù Âgí â òOúIíOñRö6î ù û#ë#ì�úIöô÷<øSïÒðÑë#ì ù íOîFìôñ^ú�ñ<û ù ú�ñ<íOîFìèö¯÷^ìôíOï�÷<ë ù ��ð��^ñ<òGöôú�ë#ñ<ï þ�Q�0/¥�2A B�C+DEDGFIHKJLDEDKM�Fl��7�=¿3¥9<��1\5 æ8æONQP 1 ã£ã 9@;� íCë��<ïôíGìR=Fí �ÝúIñdÀKíGòGö¯úIë#ñ%� þTS<þ éüöô÷@îNö£îNñfÂ�÷@îFì��&Á:òOëFìôí6ï�íGö�òOë#ñ^ï�ö¯ì/�<òGöôú�ë#ñ�öô÷<íOëFìôíOø ø'�<ï�ö£÷<î8=FíîSï�íGöØïôú E íaõ<îFì¯îFøxíGöôíOì£ó£÷<ú�ò*÷ ú�ï¥îNö¥øSë#ï8ö P ç ��0 � ï�ú�ñ<ûxö¯÷<í­òOë#ñ^ñ<íOò�ö¯ú�ëFñô�Ùí�ö:óèíGíOñ�÷@îNì��^Á�òOë#ì�íïôí�ö6òGë#ñ<ï�öôì��<ò�ö¯úIë#ñ<ï¥îFñ�����ëcë#ï�öôú�ñ<ûSî ù ûFë#ìôúÄö¯÷<øxï��<íGïôòOì�ú��í �súIñ � ÷<îFõ^ö¯íGì6��Ã.úÄö�ðÑë ù�ù ëMó£ïDöô÷@îNö¥ñ<ëî ù û#ë#ìôúÄö¯÷<ø�ó£÷<ú�ò*÷r�Ùëcë#ï�öôï�öôëØîNòOò �^ì¯îFò� ç ëU=FíOì î��<ñ<úV=#íGìôïôíèëFð g õ�ë#ú�ñRöôï�òOîFñ_ïôîNö¯úIï�ðW­î¥ï8ö¯ì�ë#ñ<û#íGìW ï�øSëcëFö¯÷^ñ<íOï�ï�û��@îFìôîFñRö¯íOí3Xjë#ñáúIöôï��<úIï�ö¯ì�ú���Kö¯ú�ëFñ<ïxöô÷@îFñáö¯÷^íô�Ùë��<ñ��áëFðZYL[ª�×(\v��Ú P ^]� �ó£÷<úIò*÷]úIï6îFò*÷^ú�í3=Fí � �< � øxõ@îFû ù ú î EGE ë � ïØî ù û#ë#ì�úIöô÷<ø þ ÀKú�ñ<òGí �e���c�e���KJ<�c����� î ù ïôë�ïôîNö¯úIï Ö íGï6ö¯÷<úIï��ë��<ñ���Ã�úIö�ö¯ëcë%û#íOñ<íGì¯îNöôíOï'�^ú�ï�öôìôú����^ö¯úIë#ñ<ï6ó£÷<úIò*÷jîFì�íSë#õKö¯ú�øSî ùIù  ïôøxëKëFöô÷­�^õ ö¯ëÝî òGë#ñ<ï�ö¯îFñRöð0îFò�ö¯ë#ì þ�� ívñ<ëMópû#úV=#íÒîFñ­î ù ö¯íOì�ñ@îNöôú@=#ívõ^ìôëcëFðcëFð �e���,�e���KJ<�,���e��� ï`ë#õ^öôú�øSî ù úÄöîÂQó£÷<ú�ò*÷­÷<úIû#÷ ù úIû#÷Rö¯ïîFñxú�ñRö¯íGìôíGï�ö¯úIñ<û­òOë#ñ<ñ^íOòGöôú�ë#ñ#�Ùí�ö:óèíGíOñhöô÷<í ù ú�øxúIöôï ëFðp�Ùëcë#ï8ö¯ú�ñ^û�î ù û#ëFìôúIöô÷<øxïvîFñ�� H ñ<ëMó£ñ ù úIøSúÄö¯ïë#ñ�ñ<ë#úIïôí\Á�ö¯ë ù íOìôîFñRö ù íOîFìôñ<úIñ<û þ� ö£ú�ïDí%=cú�<íGñ,öQðÑì�ë#ø öô÷<íCõ<ì�ëKëFð�ëNð ¾ ÷^íOë#ì�íOø Å þ $�$6öô÷@îNöDöô÷<íCïôøxëKëNö¯÷<ñ<íGïôïQëFð�öô÷<í¤�<ú�ï8ö¯ì�ú���^Áö¯úIë#ñ<ï2û#íOñ<íGì¯îNöôí �d�< �e���,�N���KJ<�,����� ì�í ù îeö¯íOï��<ú�ì�íOò�ö ù Âxö¯ëaö¯÷<í ù í3=#í ù ëFð�øSî ù úIòOúIë��<ïçñ<ë#úIïôí¥ó£÷^ú�ò*÷ë��<ì ù ú�ñ<íOîFìüö¯÷^ìôíOï�÷<ë ù � ù í�îFì�ñ<ú�ñ<û�î ù û#ë#ì�úIöô÷<ø>òOîFñ?ö¯ë ù íOì¯îeö¯í þ â6ñjö¯÷<í%ëFöô÷<íOìg÷@îFñ���ÃDîFïgøxíOñ^Áö¯úIë#ñ<íw�]úIñ­ÀKíOò�ö¯ú�ëFñ6Å þ $�ÃE_Cí�îFì�ñ<ï­îFñ�� é úv÷@î?=#íxïô÷<ëMó£ñ öô÷@îNö�ðÑë#ì�îd�<ìôë#î�� ìôîFñ<û#í ëNðDòOëFñ<òOíGõ^öò ù îFïôï�íOï�ñ<ë%î ù û#ë#ìôúÄö¯÷<ø òOîFñ ù í�îNìôñ�ö¯ë�îFòGò �<ìôîFòw ç úIñsö¯÷<ígõ<ì�íOïôíGñ<òOíxëFðÒøSî ù úIòOú�ë��<ïØñ<ëFú�ïôígîeö�îì¯îeö¯í Ð�c �]a` � 0 � ïôúIñ<ûgö¯÷<í(_Cí�îFì�ñ<ïHÁ é ú��<õ<õ�íOì��Ùë��<ñ��%ë#ñ�øhî ù ú�òGú�ë��<ï2ñ<ë#úIïôí6ö¯ë ù íOìôîFñ<òGí�Ã<ó2íaòOîFñõ<ì�ëU=Fí¥öô÷@îNö �N���,�e���^J<�,����� úIï2ë#õKö¯ú�øSî ù �^õSö¯ë­òOëFñ<ï�ö¯îFñRöèð0îFò�ö¯ë#ì�ï2úIñxöôíOìôøxïÒëFð[öô÷<í¥ï�øSëcëFöô÷<ñ<íOï�ïëFð�ö¯÷<í��<úIï�öôìôú���^ö¯úIë#ñ<ïèó£÷<úIò*÷ úÄö£û#íOñ^íOì¯îeö¯íOï þ

¾ íGò�î ùIù öô÷@îNö`úÄð �e�Ù�,�e���KJ@�,����� ú�ï`ì/�<ñ ë#ñgîØï�íGöçëFð g í�ë^îFøxõ ù íOï ó£úÄö¯÷ îFòGò �<ìôîFòwÂüõ@îFìôîFøSí�ö¯íGì� ) öô÷<íOñgí�îNò*÷x�^ú�ï�öôìôú����^ö¯úIë#ñ ×(\@ó£÷<úIò*÷ �e���,�e���KJ<�,���e� òOë#ñ<ï8ö¯ì/�<òGöôïçó£ú ù�ù ï¯îNöôú�ï8ðWÂ7YL[ª�×(\,��ï ]b 0¾ ÷<í�õ<ì�ëKëFðaú�ïs�<ÂáòGë#ñRö¯ìôî��<ú�ò�ö¯úIë#ñÐã�ïôë?ï��^õ<õÙëFïôí�ö¯÷@îNö�öô÷<íOì�í�í�ëKú�ï8ö¯ï%îz�Ùëcë#ï�öôú�ñ<û?î ù û#ë#ì�úIöô÷<øò�î ù�ù íw� �dcfehg#bK�e�Ù�,�e���KJ@�,����� ó£÷<ú�ò*÷ ú�ï_ï�ú�øxú ù îFìCö¯ë �e�Ù�,�e���KJ@�,����� ���^ö ó£÷<úIò*÷ ÷<îFïgîFñjí3=FíOñï�öôìôë#ñ^û#íOì6ï�øSëcëFöô÷<ñ<íOï�ï6û��@îNì¯îFñRö¯íGí_ë#ñsúÄö¯ï��<ú�ï8ö¯ìôú����^öôú�ë#ñ<ï þ6Ó ë#ìôí_õ^ìôíOòGú�ï�í ù Â%ó2í_ï��<õ^õÙë#ï�í_ö¯÷@îeö�dc>eig#bK�e���c�e���KJ<�c����� ö¯î H íGï�îFïxú�ñ<õ��^öxõ@îFì¯îNøSí�ö¯íOì�ï � )Rj îFñ��kî ù î��Ùí ù í �?ïôîFøSõ ù íu� ëFð6ï�ú E ígk) ÷<îFïgîFòOòGíOï�ï_ö¯ë�î ó2í�î H ù íOîFìôñ<úIñ<û�î ù û#ë#ì�úIö¯÷^øml�noàû#íOñ<íGì¯îNöôíOïgîsï�í"of�<íOñ^òOís× ] ) × ) 01010�ëFð�<úIï�ö¯ì�ú���Kö¯ú�ëFñ<ïQë8=#íOìQ� ) îFñ�� ë��^ö¯õ��^ö¯ï¥î ëKë ù í�îFñ&Á-=Nî ù �^í � Ö ñ@î ù ÷<Âcõ�ëFö¯÷<íGïôúIï¤=�0 � ï�ú�ñ�ÀKíOò�ö¯úIë#ñ

$P� S

Page 167: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

Å þ � þ�ÿ ÃKó2í�ï��<õ<õ�ë#ï�í6ö¯÷@îNö£úÄð�ö¯÷<í6ó2í�î Hhù íOîFìôñ^ú�ñ<û î ù û#ë#ì�úIöô÷<øpl�n�î ù óDî1ÂcïQìôí�ö2�<ì�ñ<ï�î_÷fÂcõ�ëFö¯÷^íOïôúIï=�\�ó£÷<ú�ò*÷�÷@îFï�î���=FîNñ,ö¯îFû#í j �<ñ��<íGìs×Z\ ) ö¯÷<íGñ �dc>ehg#b^�e���,�e���f�.�,���e� ó£ú ù�ù í3=#íGñRö2�@î ùIù ¿÷@î ù öîFñ��?ö¯÷^í Ö ñ@î ù ÷<ÂcõÙëFöô÷<íOï�ú�ïM= ó£ú ùIù îFûFìôíOí�ó£úÄö¯÷ îNö ù íOîFï�öxî�$rq � ðÑìôîFòGöôú�ë#ñ?ëFð�ö¯÷<í ù î��Ùí ù íw�í�ë^îFøxõ ù íOïQú�ñ¥�¦0<� ú�ñ@î ù�ù Â�Ãcó2í�ï��<õ^õÙë#ï�íCö¯÷@îNö£íOîFò*÷¥�<ú�ï8ö¯ìôú����^öôú�ë#ñM×(\�ú�ïQû��<îFì¯îFñRöôíOí ��öôëxïôîNö¯úIï�ðWÂYL[ª�×(\v�êï ]Ý�Û b 0� ë#ñ<ï�ú�<íGì6ö¯÷<í_ðÑë ùIù ëMó£ú�ñ^ûhïôí%=#íGìôí ù Â�ìôíGï�ö¯ì�ú�ò�ö¯í � ù ú�ñ<íOîFì¥öô÷<ìôíGïô÷<ë ù � ù íOîFìôñ<úIñ<û�õ<ì�ë�� ù íOø¥Æ¥ö¯÷^í�<ë#øSîFúIñ%úIï 5 q $ ) $�7 ts ¡ ) ï�ëhîNñfÂ¥�<úIï�öôìôú���^ö¯úIë#ñ�×BòOîFñsîFï�ïôúIû#ñ%ó2íOúIû#÷RöØëFñ ù Â�öôëxöô÷<íOï�íaðÑë��^ìõ�ë#ú�ñRö¯ï þSÓ ë#ìôíGë8=#íOìwÃ�ó2íSëFñ ù Â�î ù�ù ëMó ö:ó2ëÝõ�ë#ïôï�ú�<ú ù úIöôú�íGï¥ðÑë#ì6öô÷<íxö¯îFìôûFíGö�òOëFñ<òOíGõ^öüï�ú�û#ñÄ ¹ ì + ��Æö¯÷^ír=#íGòGö¯ëFì ¹ ú�ïØíGúIöô÷<íOì'H$ )³ä �£ë#ì' ä�) $P��0 ¾ ÷<í_ðÑë��^ìØõ�ë#ú�ñRö¯ï�ú�ñ 5 q $ ) $�7 îFì�íüò ù îFïôï�ú Ö íw��ú�ñÝî ù�ùðÑë��<ìØõÙë#ï�ïôú�� ù í_óèî1ÂKï��<Âsöô÷<íOï�ígö:ó2ë�òOë#ñ^òOíOõKö¯ï Ã�îFñ���÷<íGñ<òOí öô÷<í òOë#ñ^òOíOõKöaò ù îFï�ïCòOë#ñ^ïôú�ï8ö¯úIñ<û�ëFðö¯÷^íOïôíxö:óèëÝòOë#ñ<òGíOõ^öôïüúIïüî °FÀ�Ã Õ À �.û ÕçòGë#ñ<òOíGõ^ö_ò ù îFï�ïüîNï �<í Ö ñ<íw�{�<Âu_CíOîFìôñ<ïüîFñ�� é ú�v_Cí�îFì�ñ<ïîFñ�� é úvÃê$:Å�Å ÿ � þd� öüú�ï­ò ù íOîFì�ö¯÷@îNöüí3=FíOì/ í�ë^îFøxõ ù íx��í ù ëFñ<û#ï�ö¯ë « [ªY$P�x0ú þ í þ ¬ Ú¿$P��Ã�ö¯÷@îeö¦ ¹ ¦ ] Úò$ ) îFñ��%öô÷@îNö�îFñ<Âs�<úIï�öôìôú���^ö¯úIë#ñ�× ë8=FíOì 5 q $ ) $�7 ú�ï¤$wÁ�û#ëKë&��ðÑë#ì ¹ 0ú þ í þ`ß Ú�$P� þ

� ë#ñ<ï�ú�<íGì£öô÷<íCðÑë ù�ù ëMó£ú�ñ<û î ù û#ë#ì�úIöô÷<ø4ðÑëFìDö¯÷<úIïQìôíOï8ö¯ì�ú�òGöôí � ù íOîFìôñ<úIñ<û õ<ì�ë�� ù íGøuÆ� Ú ìôîMó îDï¯îNøSõ ù íS�6Ú ),+ ] ) - ] . ) 010 0 )P),+! ^) - . ëFð g ù î��Ùí ù í �Øí�ë^îFøxõ ù íGï ðÑìôë#ø ê­É ÊË6Ì^Í ¹w) ×#�{ã� ¾ �<ñ ��c>ehg#bK�N���,�e���^J<�,����� ë#ñM�]ó£úIöô÷gõ@îFì¯îNøSí�ö¯íOì�ï � Ú �Û )xj Ú iÛ XoUWkPUWV Ú ]Û �<ïôúIñ<û�l�n ýó£úÄö¯÷ ¤ Ú � îNï£öô÷<íCóèíOî H�ù íOîFìôñ^ú�ñ<ûgî ù û#ë#ì�úIö¯÷^ø þÀ^�<õ<õ�ë#ï�íèö¯÷@îeö�öô÷<íèøSî ù úIòOúIë��<ï�ñ<ë#úIïôíÒì¯îNöôí Ð ú�ï � ç 0 � ï ú�ñ'ÀKíOò�ö¯úIë#ñ�Å þ�ÿ^þTS ÃFî � ÷<íOì�ñ<ë�yM�Ùë��^ñ��

ïô÷^ë�ó£ïQöô÷@îNöQðÑë#ì g Ú P ?] � ù ë#ûr]© � ) ó£úIöô÷�õ<ìôë��<î��<ú ù úIöîÂSîNö ù í�îFï8ö�$zq © ó2íC÷@î?=#íCö¯÷@îeöQö¯÷<íCïôîFøªÁõ ù í��áòOë#ñRö*îNú�ñ<ïØîNöCøxë#ï�ö S|{�g �<ú�ì8öîÂ%í�ë^îFøxõ ù íGï þ} Â%ö¯÷<í ��c>ehg#bK�N���,�e���^J<�,����� ï�øSëcëFöô÷<ñ<íOï�ï

õ<ì�ë#õÙíGì�öîÂhîFñ���ë��^ìèò*÷<ë#úIòOí6ëNð � ) óèíC÷<î8=Fí6ö¯÷@îeö~YL[ª�×(\,��ï ]]v� � 0 ¾ ÷<íGë#ìôíGøÿÅ þ �üñ<ëMó�ú�øxõ ù úIíOïö¯÷<îNö_íOîFò*÷�l�n ý ÷<ÂcõÙëNö¯÷<íGïôú�ï�=h\£÷<îFï_î���=FîNñ,ö¯îFû#íhîNö ù í�îFï8ö i� X�UWkaU V Ú ]� ó£úÄö¯÷pì�íOïôõ�íOò�ö_ö¯ë¥×Z\Y0� ï£úIñ�ÀcíOòGöôú�ë#ñuÅ þ�ÿ^þTS Ã^ó2ía÷@î?=Fí­öô÷@îNöQó£úIöô÷�õ<ìôë��<î��<ú ù úIöîÂhîNö ù íOîFï�ö�$�q ©

ö¯÷<í Ö ñ@î ù ÷fÂcõ�ëFö¯÷^íOïôúIï= ë��Kö¯õ��^öª�< �dc>ehgRbK�e���,�N���KJ<�,����� �<ú�ïôîFû#ìôíGíOï ó£úIöô÷ îNögøSëFï�öxîFñ � ðÑìôîFòGöôú�ë#ñjëFð�ö¯÷^í%ò ù íOîFñí�ë^îFøxõ ù íOï����¦0ÀKú�ñ^òOíCö¯÷<í��<ëFøhîFúIñ�ú�ï Ö ñ<úÄö¯íxÑú�ñ�ð0îFòGö�ëFð�ïôú E íCðÑë��<ì2�èóèí­ò�îFñ¥��ë��<ñ��%û#íOñ^íOì¯î ù ú E îNö¯úIë#ñ�íGìôìôëFì�<úIìôíOò�ö ù  þ � ï�ú�øxõ ù í � ÷<íGìôñ<ëdyC��ë��<ñ��kîFì�û��<øxíOñRöxïô÷<ëMó£ïSöô÷@îNöxúIð g úIïxï/���xòOúIíOñRö ù  ù îFì�û#í�Ãö¯÷^íOñ�ó£úIö¯÷]õ<ì�ë��@î��<ú ù úÄöîÂsîNö ù í�îFï8öª$<q × öô÷<íx÷fÂcõ�ëFö¯÷^íOïôúIï'=�ó£ú ùIù �ÙíxîFñ { Á:îFõ<õ<ì�ë1ë^úIøhîeö¯ë#ìØðÑëFìïôúIû#ñ? ¹k�^+ �x�^ñ��<íOì#×d0 L ëMóèí%=#íOìwÃt_CíOîFìôñ<ïhîFñ�����ú£÷@î?=#íÝïô÷<ëMó£ñ� ¾ ÷^íOë#ì�íOø $ÝëFðxv_Cí�îFì�ñ<ïîFñ��0��úvÃ�$:Å�Å ÿ ���6ö¯÷@îNö_ñ<ë ù íOîFìôñ^ú�ñ<ûÝî ù û#ëFìôúIöô÷<ørðÑë#ìüî��<ú�ï8ö¯úIñ<òGöüòOë#ñ<òGíOõ^ögò ù îFï�ïüòOîFñ ù íOîFìôñ öôëîFòGò �<ìôîFòw { ó£úÄö¯÷%õ<ìôë��<î��<ú ù úIöîÂ�$�q × ú�ñ�ö¯÷<íaõ<ìôíGïôíGñ<òOí_ëNðvøhî ù ú�òGú�ë��^ïDñ<ë#úIïôíaîNöØìôîNö¯í Ð�� �]a` � 0¾ ÷<úIïüòOëFñ,öôì¯î��^ú�òGöôú�ë#ñ õ<ìôë8=#íGï_ö¯÷@îeöüö¯÷^í�� c>eig#b � ���c�e���KJ<�c����� î ù ûFë#ìôúÄö¯÷<øròOîFñ<ñ<ëNö_í�ëKúIï�ö1ÃçîNñ��

$P���

Page 168: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

÷<íGñ<òOíDöô÷<í£�<ú�ï8ö¯ì�ú���^öôú�ë#ñ^ï û#íGñ<íOìôîNö¯íw�x�<Â�� ���,�e���^J<�,����� îFìôíQëFõ^ö¯úIøhî ù �<õ_öôëCòOë#ñ^ï�ö*îNñ,övð0îNòGö¯ëFìôï þ

$P��é

Page 169: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

� � ��� ����� ���� ������� ��� � ��� ������� � �   ¡ ¢ � £ ���¤ �¥�¦£ ¢ §¨� � �K©ª�u���&« ¢¬�­����� ©�2®�¯±°h²@³.´i®iµd¶�·'°iµ�¸�¯º¹"»�¼½¹�µd¾|µ�²�®¨·"¿x®h³R²�Àh¹3»Á¯±°h¹*¤ì�ÃĶ�¹"µ�»º®h²@®h¾¥¸h»R¿xÅh¶@¹"Æ Ã Å�Ç�¯�¯±°h²@³.¯±²@ÆÈ¹É »º¿�ÆʵZ®h¹3¼Ë¸Ì¹3»º³R¸Ì¹3·3¯±² = ¹dÍLÎÏ°�²�¶�¹tÐ+°iµ�¸�¯±¹"»�ÑZ¾|µ = ¹<µr³RÇh¸�¹"»º¸�¿x¶  ®�¿xÆȲ�µ�¶i¯±²@Æȹ�µ�¶�¾�¿x»º²V¯±°hÆ É ¿�»¶�¹"µ�»º®�²�®h¾tµ�®�µ�»RÅh²@¯º»±µ�»  ¤ìtêÇ�®hÀh¹"»Kµ�®Òµ�»ºÅ�²@¯±»ºµ�»  À�²�³R¯º»º²@ÅhÇ�¯±²@¿x® à ¼+¹+®h¿8¼ É ¿f·"Ç�³&¿x®7»R¹"³Ó¯±»º²@·3¯º¹"À= ¹3»º³R²�¿x®h³^¿ É ¯±°h¹(¤ì�Ã0¶@¹?µ�»R®h²�®�¾7¸h»º¿�Åh¶�¹3Æ�¼�°h²�·'°�·?µd®ªÅ�¹}³º¿x¶ = ¹"À�²�®�¸�¿x¶  ®h¿�ÆȲ�µ�¶�¯º²�Æ5¹�;~°h¹�Æ.µd²�®7»R¹"³ºÇ�¶@¯�¿ É ¯º°h²�³�·'°iµd¸�¯±¹3»�²�³�µ�®5µ�¶@¾x¿x»R²@¯±°�ÆÔ¯±°iµd¯Õ�Ö^Ð0¶@¹?µ�»R®h³K¯±°�¹~·"¶×µd³º³K¿ É Æ5¿x®h¿ Á¯±¿�®h¹ÈØ�Ù �@Ú ÛÝÜ2Þ�ß � Á ¯±¹"»RÆà¤ì�Ãϲ@®*¸Ì¿x¶  ®h¿xÆ5²×µ�¶I¯±²@Æȹ�Ç�®hÀh¹"»<¯º°h¹ÒÇh®h² É ¿�»ºÆáÀh²�³Ó¯±»º²@ÅhÇ�¯º²�¿x® à Çh³º²@®h¾»±µd®hÀh¿xÆà¹ ë µ�Æȸ�¶�¹"³}¿x®�¶  Íx¾~°h²@³(²@³(µ�®�¹ ë ¸Ì¿x®�¹"®|¯±²�µ�¶�²�Æ5¸h»º¿ = ¹"Æ5¹"®|¯(¿ = ¹"»}¯±°h¹5Å�¹"³R¯(¸h»R¹ = ²�¿�Çh³¸�¿x¶  ®h¿xÆ5²×µd¶ Á ¯º²�Æ5¹.µd¶�¾x¿x»R²@¯º°hÆȳ�²@®â¯º°h²�³Ò¼+¹3¶�¶ Á ³R¯ºÇhÀh²@¹"À¨Æ5¿fÀh¹"¶ à ¼�°h²@·'°ã·?µ�®¨¶�¹?µd»º®¨Æ5¿x®h¿d¯±¿x®h¹ä�å ¶�¿�¾ �æIç Á ¯±¹3»ºÆè¤ì�à à µ�®hÀ�²�³+¯º°h¹�´i»R³R¯�¹ � ·3²�¹"®|¯�µ�¶@¾x¿x»º²V¯±°hÆ É ¿�»zÆÈ¿�®h¿�¯±¿�®h¹ å ¶@¿x¾ æIç-é � ] � Á ¯±¹3»ºÆ¤ìtÃ�²�®êµ�®  Æ5¿�À�¹"¶w¿ É ¶�¹"µ�»º®�²�®h¾ É »º¿xÆ뻺µ�®hÀh¿�Æá¹ ë µ�Æȸ�¶�¹"³3Í åvì ¹3·?µ�¶@¶ É »R¿xÆmÐ+°iµ�¸�¯º¹"»<íÒ¯±°iµ�¯îxµ�·'ï>³º¿�®{�ð³+ñtµ�»RÆÈ¿�®h²�· À ²�¹ = ¹tµd¶�¾x¿x»R²@¯º°hÆ É ¿x»L¶�¹?µd»º®h²@®h¾}¤ìtÃkÇh®hÀ�¹"»L¯±°h¹�Çh®h² É ¿x»ºÆ�Àh²�³Ó¯±»R²�ÅhÇ�¯º²�¿x®Æȵ�ï�¹3³+¹"³R³º¹3®>¯º²×µ�¶ÌÇh³º¹t¿ É Æȹ3Æ7Å�¹"»º³R°h²�¸ o Çh¹"»R²�¹"³+µ�®hÀ.¯º°>Çh³zÀh¿f¹"³z®�¿�¯½´�¯z²�®|¯º¿Z¯±°�¹�¸iµ�»ºµ�Àh²�¾�ÆÄ¿ ɶ�¹"µ�»º®�²�®h¾ É »R¿xÆò»±µ�®hÀ�¿xÆʹ ë µ�Æȸ�¶�¹"³~¿�®h¶ Â Í ç Î�¹}µ�¶�³R¿�³º°h¿8¼Ë°h¿8¼Ë¯º¿�¹ ë ¯±¹3®hÀ�¿xÇh»�µ�¶@¾x¿x»º²V¯±°hÆó³R¿¯±°hµd¯�²V¯t¶@¹?µ�»R®h³t²@®ê¸�¿x¶  ®h¿xÆ5²×µd¶G¯±²@Æȹ(Çh®hÀh¹3»<µ�®  ·"¿x®�³R¯'µd®>¯ Á Å�¿xÇh®hÀ�¹"À*¸h»º¿fÀhÇh·%¯�Àh²@³R¯º»º²�Å�Ç�¯±²@¿x® õ�®hÀ*³R°h¿8¼p¯±°hµd¯�¯±°�¹Èµd¶�¾x¿x»R²@¯º°hÆë·?µ�®*Å�¹�Çh³º¹3À�¯º¿�¶�¹?µd»º® = µ�»º²@¿xÇh³�·3¶×µ�³R³º¹3³<¿ É ·3¿x®h³Ó¯'µ�®|¯ Á Àh¹"¸�¯±°Æ5¿x®h¿�¯º¿x®h¹<·"²@»º·3Çh²@¯º³z¼�²@¯±° É ¹%¼Ô»º¹3¶�¹ = µd®>¯ = µ�»R²×µ�Åh¶@¹"³3Í

ôUõxö

Page 170: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

÷<ø¥ö8÷ ù?üÁúKûLü�ýyûyú~ú�þ3ü#üÿ����Óÿ��Óÿ ����� ������������� ���������� �"!#�$�%'& ()%���*��+2®ã°h²@³È³R¹"Æ5²�®iµd¶ ô-, í�S�¸iµ�¸�¹"» å/. µ�¶�²�µ�®|¯-0 ô1, í�S ç . µ�¶@²×µ�®|¯Ò²@®>¯º»º¿fÀhÇh·3¹"À¨¯º°h¹êÀh²@³R¯º»º²�Å�Ç�¯±²@¿x®32 É »R¹"¹Æ5¿�Àh¹3¶~¿ É Õ�»º¿�Åiµ�Åh¶54­Öt¸h¸�»º¿ ë ²�Æȵd¯±¹3¶64�Ð+¿x»R»º¹3·3¯ å ÕÌÖ^Ð ç ¶�¹"µ�»º®h²@®h¾ É »º¿xÆ »ºµ�®hÀh¿�Æ ¹ ë µ�Æ5¸h¶�¹3³µ�®hÀ­¸�¿x³R¹"À­¯±°h¹ É ¿�¶�¶�¿8¼�²@®h¾ o Çh¹3³R¯º²�¿x® Æ ·?µ�®­¸�¿x¶54�®�¿xÆȲ�µ�¶L³º²67"¹�¤ì�ÃÔÅ̹Á¶@¹?µ�»R®h¹"À­²@®¥¸�¿x¶64f®h¿82Æ5²×µ�¶K¯º²�Æ5¹5Ç�³º²�®�¾ê»±µd®hÀh¿xÆ ¹ ë µdÆȸh¶@¹"³ZÀh»±µ?¼�® É »R¿xÆQµ�®­µ�»RÅh²@¯º»±µ�»�4kÀh²@³R¯±»R²�ÅhÇ�¯±²�¿�® 9;: = ¹3»r¯±°�¹³ºÇ�Åh³º¹ o Ç�¹"®|¯)4x¹?µd»º³<0f¯±°h¹tµ�¸h¸iµd»±µ�®|¯½Àh² � ·"Ç�¶@¯=4Ò¿ É À�¹"³º²@¾x®h²@®h¾r¸�¿x¶64f®h¿�ÆȲ�µ�¶�¯±²@Æȹ^µ�¶@¾x¿x»R²@¯±°�Æȳ É ¿�»¯±°�²�³r¾�¹"®h¹3»±µ�¶½¸h»º¿xÅ�¶�¹"ÆQ¶@¹"À¥Æȵ�®>4�»º¹"³R¹?µ�»R·'°h¹"»R³�¯±¿ ³Ó¯±ÇhÀ 4�»º¹3³R¯º»º²�·%¯±¹3À = ¹"»R³º²@¿x®h³7¿ É ¯º°h¹�¤ì�ö�¹"µ�»º®�²�®h¾5¸h»º¿�Åh¶�¹3Æè¼�°h²�·'°kµdÀhÆȲV¯^¸Ì¿x¶54f®h¿xÆ5²×µ�¶G¯º²�Æ5¹(µ�¶�¾x¿�»º²@¯º°hÆ5³"ÍzÖ�³tÀh¹3³º·"»R²�Å�¹"ÀkÅ�¹"¶@¿U¼?0�¯±°�¹»º¹3³R¯º»º²�·%¯±²@¿x®h³z¼�°h²@·'°ª°iµ = ¹ZÅ�¹"¹3®ª·3¿x®h³º²@Àh¹"»R¹"Àª²�®h·3¶�ÇhÀh¹

@ µ�¶@¶�¿8¼�²�®�¾*¯º°h¹�¶�¹?µd»º®h¹3»5¯º¿uÆȵ�ï�¹BA Ô ADC ÔFEHG ��I Ý Ü<J KFE I KHG É ¿x»�¯º°h¹ = µ�¶@Çh¹ª¿ É ¯±°h¹�¯'µd»º¾x¹%¯É Çh®�·3¯±²@¿x®�µd¯�¸�¿x²�®|¯±³~³R¹"¶@¹"·3¯º¹"ÀªÅL4.¯±°h¹}¶�¹"µ�»º®�¹"»<M@ »R¹ o Çh²�»R²�®h¾7¯º°iµd¯~¯º°h¹}¶�¹?µd»º®h¹3»~³ºÇh·3·"¹"¹3Àê¿x®�¶64ÁÇ�®hÀh¹"»�»R¹"³R¯º»º²@·3¯±¹3ÀªÀ�²�³R¯º»º²@ÅhÇ�¯±²@¿x®h³½¿x®ª¹ ë µ�Æ�2¸h¶@¹"³N0i³ºÇh·'°êµ�³z¯±°h¹}Çh®�² É ¿x»RÆ�Àh²@³R¯±»R²�ÅhÇ�¯±²�¿�®O0�»ºµd¯±°h¹3»~¯±°iµ�®�µ�¶@¶ Àh²@³R¯±»R²�ÅhÇ�¯±²�¿�®h³<M@ »R¹ o Çh²�»R²�®h¾�¯±°hµd¯w¯±°h¹¶�¹"µ�»º®h¹3»�³RÇh·"·3¹"¹3À�¿x®h¶64 É ¿x»w»º¹"³Ó¯±»R²�·3¯º¹"À7³RÇhÅ�·"¶×µd³º³º¹3³&¿ É Â¤ìtÃ É ¿x»ºÆrÇh¶×µ�¹³RÇh·'° µ�³�Æ5¿x®h¿�¯º¿x®h¹<¤ì�Ã¥¼�²@¯±°�µÒÅÌ¿�Çh®hÀh¹3Àª®>ÇhÆ7Å�¹"»�¿ É ¯±¹3»ºÆ5³"ÍÖQP�Ö�RS2�T5¤ìtÃk²�³µ}¤ìtÃ*²�®�¼�°h²@·'°.¹?µd·'°È¯±»RÇ�¯±°Èµ�³R³º²�¾�®hÆȹ3®|¯½³ºµd¯±²@³R´i¹3³+µ�¯LÆÈ¿x³Ó¯ST�¯º¹"»ºÆ5³"Í_ °iµ�»RÀh¿x® å _ °iµ�»RÀh¿x®O0 ô1,U, S ç ¾|µ = ¹.µ�¸�¿x¶64f®h¿xÆ5²×µd¶E¯º²�Æ5¹7Æȹ3Æ7Å�¹"»º³R°h²�¸ o Çh¹3»�4�µ�¶@¾x¿x»R²@¯±°�Æ É ¿�»¶�¹"µ�»º®�²�®h¾�¸�¿x¶64f®h¿�ÆȲ�µ�¶52-³º²V73¹�P�Ö�RS2 ô ¤ìtÃ0Çh®hÀh¹3»�¯º°h¹(Çh®h² É ¿�»ºÆ�À�²�³R¯º»º²@ÅhÇ�¯±²@¿x®OMf¯±°h²@³^»R¹"³RÇh¶@¯�¼+µ�³¶×µ�¯±¹"»}³Ó¯±»º¹3®h¾�¯º°h¹"®h¹3À¥Å>4� +¶�ÇhÆ�¹3¯rµ�¶�Í å +¶@ÇhÆ KFWYXUZ5[ 0 ô1,U, S ç ¯±¿\P�Ö�RS2�T�Â�]tÃ É ¿x»(µ�®>4*·"¿x®�2³R¯±µ�®|¯^T`_ +¹3¶�¶�µ�»º¹�¾|µ = ¹ µ�¸�¿x¶64f®h¿xÆ5²×µd¶½¯±²@Æȹ�Æ5¹"Æ7Å�¹"»R³º°h²@¸ o Çh¹3»�4¨µ�¶�¾x¿�»º²@¯º°hÆ É ¿�»È¶@¹?µ�»R®h²�®h¾a å ¶�¿x¾ æ ç 2�¯±¹"»RÆ�Â�]�ÃãÇh®hÀh¹3»�¯±°h¹�Ç�®h² É ¿x»RÆmÀ�²�³R¯º»º²@ÅhÇ�¯±²@¿x® å +¹"¶@¶×µ�»R¹U0 ô-,b, Ø ç ÍcR~°�²�³<»º¹3³ºÇh¶V¯�¼+µ�³³R¯º»º¹3®h¾�¯±°�¹"®h¹3À Å>4ã +¶�ÇhÆ µ�®�À ì Ç�Àh²�·'°�¾|µ = ¹kµu¸�¿x¶64f®h¿xÆ5²×µd¶z¯±²�Æ5¹�Æȹ3Æ7Å̹3»º³R°h²�¸ o Çh¹3»�4ãµ�¶52¾x¿x»R²@¯º°hÆ å +¶�ÇhÆ�µ�®hÀ ì ÇhÀh²�·'°#0 ô-,b,|õ ç ¼�°h¿�¾|µ = ¹5µ�¸�¿x¶64f®h¿xÆ5²×µd¶E¯º²�Æ5¹Òµ�¶�¾x¿�»º²@¯º°hÆ É ¿x»<¹Fd�µ�·%¯¶�¹"µ�»º®�²�®h¾ a å ¶�¿�¾ æIç 2�¯±¹3»ºÆóÂ�]�à Çh³º²@®h¾ZÆ5¹"ÆrÅ̹3»º³º°�²�¸5µ�®hÀÒ¹ o Çh² = µ�¶�¹3®h·"¹ o Çh¹"»R²�¹3³<Mf³º¹ = ¹"»ºµ�¶Ì¿d¯±°h¹3»¸�¿x¶64f®h¿xÆ5²×µd¶52-¯º²�Æ5¹�µ�¶�¾x¿�»º²@¯º°hÆ5³ É ¿x» a å ¶�¿x¾ æ ç 2�¯±¹"»RÆ Â�]tÃó°iµ = ¹�³º²@®h·"¹�Å̹3¹"®Ë¾x² = ¹"® ²@®�¯±°h²@³Æ5¿�Àh¹3¶ å +¹"²@Æȹ3¶ KFWcXUZ5[ 0 ô1,b,be M� +³R°h¿xÇ�¯=4U0 ô1,U,|õ ME +³º°�¿xÇ�¯=4b0 ô1,b,xö M _ Çh³R°h²�¶@¹ = ²@¯f7U0 ô-,b,|ö ç Íhg�³"2²�®�¾ê¯±¹3·'°h®h² o Çh¹3³ É »º¿xÆ Ãi¿xÇh»R²�¹"»rµ�®iµ�¶54f³º²�³_Óêµ�®h³R¿xÇh»�¾|µ = ¹�µ�® æ Ù#i ÛÝÜ2Þ�ÛÝÜ2Þdß1j 2-¯±²@Æȹ5Æ5¹"Æ7Å�¹"»R³º°h²@¸o Çh¹"»�40µ�¶@¾x¿x»º²V¯±°hÆà¼�°�²�·'°�¶�¹?µd»º®h³7¸�¿x¶54�®�¿xÆȲ�µ�¶52-³º²67"¹5Â�]tÃÔÇh®�Àh¹"»7¯º°h¹�Çh®h² É ¿x»ºÆ Àh²�³Ó¯±»R²�ÅhÇ�¯º²�¿x®å Óªµ�®h³º¿�Çh»<0 ô1,U,|õ ç ÍâÖt³ÈÀh¹"³R·"»º²@Å̹3Àã²�®¨Ð+°iµ�¸�¯±¹"».í 0+²@®ãµ*·"¹3¶�¹"Å�»±µd¯º¹"Àã»R¹"³ºÇ�¶@¯Áîxµ�·'ï>³º¿x®ã¾xµ = ¹

ôUõ í

Page 171: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

¯±°�¹Òñtµ�»RÆÈ¿x®�²�·kPf²�¹ = ¹Òµ�¶�¾�¿x»º²V¯±°hÆè¼�°h²@·'°*»ºÇh®h³<²@®k¸Ì¿�¶64f®h¿xÆ5²×µ�¶I¯º²�Æ5¹7µ�®hÀ*Çh³R¹"³}Æ5¹"Æ7Å�¹"»R³º°h²@¸o Çh¹"»R²�¹3³½¯±¿Z¶�¹"µ�»º®È¸Ì¿x¶54f®h¿xÆ5²×µ�¶h³R²V73¹tÂ�]�Ã�²�®È¸Ì¿�¶64f®h¿xÆ5²×µ�¶�¯±²�Æ5¹�Çh®hÀh¹3»½¯º°h¹tÇh®�² É ¿x»RÆÄÀh²�³Ó¯±»R²�ÅhÇ32¯±²@¿x® å îxµ�·'ï>³º¿x®O0 ô-,b,|ö ç Í�+2® É µ�·3¯<0wîxµ�·'ï>³º¿�®�¸�»º¿ = ¹3³}¯º°iµd¯�¯º°h¹�P�²@¹ = ¹7·?µ�®*Å�¹7Æ5¿�Àh²V´i¹"À�¯±¿�»RÇh®²�®�¸�¿x¶64f®h¿�ÆȲ�µ�¶�¯º²�Æ5¹�Çh®hÀ�¹"»^µ�®>4�·"¿x®�³R¯'µd®>¯�22Å�¿xÇh®hÀ�¹"À�¸h»º¿fÀhÇh·%¯�À�²�³R¯º»º²@ÅhÇ�¯±²@¿x® Í+2®(¯º°h¹�³R¯±µ�®hÀiµd»ºÀZÕÌÖ^Ð�Æ5¿fÀh¹"¶�¼�²V¯±°h¿�Ç�¯wÆȹ3Æ7Å�¹"»º³R°h²�¸ o Çh¹3»º²@¹"³w¸Ì¿�³º²@¯º² = ¹�»º¹"³RÇh¶@¯º³wµ�»º¹Lï>®h¿8¼�®É ¿x» = µd»º²�¿�Çh³+³RÇhÅÌ·3¶×µ�³R³º¹3³^¿ É Â�]tÃuÇh®�Àh¹"»~»R¹"³R¯º»º²@·3¯±¹3À�Àh²�³Ó¯±»R²�ÅhÇ�¯º²�¿x®�³"ÍKÖ ElKfX>m ÔnT.Â�]�Ãu²�³+¿x®h¹�²@®¼�°h²@·'°�¹"µ�·'° = µ�»R²×µ�Åh¶@¹�µ�¸h¸�¹?µd»º³(µd¯}Æ5¿x³R¯oT ¯±²@Æȹ3³"Í _ ¹?µd»º®h³ KFW�XUZ5[ ³º°h¿8¼½¹"À�¯±°iµd¯(»R¹?µ�À32-¿x®h·3¹Â�]tÃâµ�»º¹rÕÌÖ^ÐÔ¶�¹"µ�»º®hµ�Åh¶�¹(Çh®hÀ�¹"»t¯º°h¹rÇh®�² É ¿x»RÆëÀh²�³Ó¯±»º²@ÅhÇ�¯º²�¿x®�²�®ª¸Ì¿x¶54f®h¿xÆ5²×µ�¶ ¯º²�Æ5¹ å _ ¹?µ�»R®h³KFWpXUZ5[ 0 ô1,U, S ç Ízñtµ�®h·3¿�·'ï�¹qd�¯º¹"®hÀ�¹"À ¯±°�²�³�»º¹3³ºÇh¶V¯�¯º¿.»º¹"µ�À32"T�Â�]�Ã É ¿x»^µ�®>4�·"¿x®�³R¯'µd®>¯�T å ñtµ�®�2·"¿f·'ï 0 ô-,b, Ø ç Í . ¹3»ºÅ�¹"Çh»R¾�¯¾|µ = ¹�µ�®.µd¶�¾x¿x»R²@¯º°hÆ É ¿x»�¶�¹"µ�»º®h²@®h¾}µ�»RÅh²@¯º»±µ�»�47¸�¿x¶64f®h¿xÆ5²×µd¶522³R²V73¹+Â�]�ÃÇh®hÀ�¹"»<¯±°h¹�Ç�®h² É ¿x»RÆ�Àh²@³R¯º»º²�Å�Ç�¯±²@¿x® ²�® ¯º²�Æ5¹ æ Ù#i ÛÝÜ2Þ�ß-j år. ¹3»ºÅ�¹"Çh»R¾�¯-0 ô1,U,bs ç Ío+2®�µ�®*²�Æ5¸Ì¿x»Ó¯'µ�®|¯»º¹3³ºÇh¶V¯ � ²�®h²�µ�¶ KFWtXUZ5[ ¾|µ = ¹Òµ�®*µ�¶@¾x¿x»R²@¯±°�Æ É ¿x»�¶�¹"µ�»º®�²�®h¾�µd®L4 ��u / ·"²�»R·"Çh²V¯ å ·"¿x®h³Ó¯'µ�®|¯}Àh¹3¸�¯±°#0¸�¿x¶64f®h¿xÆ5²×µd¶h³º²67"¹U0�Çh®>ÅÌ¿�Çh®hÀh¹3À É µ�®h²�®ÈÖ�]�Â?vb: ì ¾xµd¯±¹3³ ç Ç�®hÀh¹"»½¯±°h¹�Çh®h² É ¿�»ºÆóÀh²@³R¯±»R²�ÅhÇ�¯±²�¿�®È²@®æ wqxzy|{ i ÛÝÜ2Þdß1j ¯º²�Æ5¹ å � ²�®�²×µ�¶ KFW}XUZ5[ 0 ô-,b, Ñ ç ÍÖ~A����#� W ��� K Â�]�Ãã²�³<µ�Â�]�è¼�²@¯º°k®h¿�®h¹"¾xµd¯±¹3À = µ�»R²×µ�Å�¶�¹"³3Í?+-¯}²�³t¼½¹"¶@¶Kï>®h¿8¼�®*¯º°iµd¯}²@®¯±°�¹ZÀh²@³R¯º»º²�Å�Ç�¯±²@¿x®322²@®hÀh¹3¸Ì¹3®hÀh¹"®|¯�³R¹3¯º¯º²�®h¾ 0�¶@¹?µ�»R®h²�®�¾�ÆÈ¿x®�¿�¯±¿x®�¹ZÂ�]�Ã�²@³�¹ o Çh² = µ�¶@¹"®|¯�¯±¿5¶�¹"µ�»º®32²�®�¾.¾x¹3®h¹"»ºµ�¶&Â�]�à å _ ¹?µ�»R®h³ KFW)XUZ5[ 0 ô1, í ö Å ç Í�ñ^¿8¼+¹ = ¹"»�¯º°h²�³�¹ o Çh² = µd¶�¹"®�·"¹(Àh¿�¹3³t®h¿d¯�°h¿x¶@À É ¿�»¶�¹"µ�»º®�²�®h¾ Ç�®hÀh¹"»�»R¹"³Ó¯±»º²@·3¯º¹"À0Àh²�³Ó¯±»R²�ÅhÇ�¯º²�¿x®�³r³ºÇ�·'°¨µ�³Z¯±°h¹�Çh®�² É ¿x»RÆ Àh²@³R¯º»º²�Å�Ç�¯±²@¿x®O0Kµd®hÀ0Æ.µ�®>4»º¹3³º¹"µ�»º·'°h¹3»º³�°iµ = ¹k³Ó¯±ÇhÀ�²�¹"À㯺°h¹k¸h»R¿xÅh¶�¹3Æ ¿ É ¶�¹?µd»º®h²@®h¾uÆ5¿x®h¿�¯º¿x®h¹êÂ�]tÃpÇh®�Àh¹"»Á»R¹"³Ó¯±»º²@·3¯º¹"ÀÀh²@³R¯±»R²�ÅhÇ�¯±²�¿�®h³"Í<ñ�µ�®h·3¿�·'ïêµ�®hÀ]Óêµ�®�³º¿xÇh»<¾xµ = ¹Òµ.¸�¿x¶64f®h¿xÆ5²×µd¶w¯º²�Æ5¹rµd¶�¾x¿x»R²@¯º°hÆ É ¿x»�¶@¹?µ�»R®h²�®h¾Æ5¿x®h¿�¯º¿x®h¹}»º¹"µ�À32�T�Â�]tÃ0Çh®hÀ�¹"»^·"¿x®h³Ó¯'µ�®|¯�22Å�¿xÇh®hÀh¹3À ¸h»R¿�ÀhÇ�·3¯�Àh²�³Ó¯±»º²@ÅhÇ�¯º²�¿x®h³ å ñtµ�®�·"¿f·'蝹d®hÀÓêµ�®�³º¿xÇh»N0 ô1,b,�ô ç Í . ¹"»RÅ̹3Çh»º¾d¯K¾|µ = ¹½µ�¸�¿x¶54�®�¿xÆȲ�µ�¶�¯º²�Æ5¹�Çh®h² É ¿�»ºÆ Àh²@³R¯±»R²�ÅhÇ�¯±²�¿�®Zµd¶�¾x¿x»R²@¯º°hÆ É ¿�»¶�¹"µ�»º®�²�®h¾�¸�¿x¶64>2-Àh²�³/�R¿x²�®|¯~¿x®�¹F22»R¹?µ�À�22¿x®h·3¹}ÆÈ¿x®�¿�¯±¿x®�¹ZÂ�]�Ã0µ�®hÀ�»º¹?µdÀ322¿x®�·"¹ É µ�·3¯º¿x»±µdÅh¶�¹<Æ5¿x®h¿82¯±¿�®h¹(Â�]tà år. ¹3»ºÅ�¹"Çh»R¾�¯-0 ô-,b, í ç Í _ Ç�·"¹"»ºµ KFWSXUZ5[ ¾|µ = ¹ZµÒ¸�¿x¶54�®�¿xÆȲ�µ�¶52�¯±²@Æȹ�µ�¶@¾x¿x»R²@¯±°�Æ ¼�°�²�·'°¶�¹"µ�»º®�³½Æ5¿x®h¿�¯º¿x®h¹�TL2�¯±¹"»RÆ Â�]tÃ*Çh®hÀh¹3»½¯º°h¹�Ç�®h² É ¿x»RÆÄÀh²@³R¯±»R²�ÅhÇ�¯±²�¿�®5Ç�³º²�®�¾r°>4f¸Ì¿�¯º°h¹"³R¹"³+¼�°�²�·'°µ�»R¹5Æ5¿x®h¿d¯±¿x®h¹�TL2�¯±¹3»ºÆ Â�]�à å _ Çh·"¹3»±µ KFWYXUZ5[ 0 ô1,b, S ç Í�R~°h²�³}¼+µ�³Z²@Æȸ�»º¿ = ¹3À�Å>4BP�µ�ï�µ�²Lµd®hÀÓêµ�»RÇh¿xï�µ<¼�°h¿Z¾xµ = ¹^µ(¸�¿x¶64f®h¿�ÆȲ�µ�¶52�¯±²�Æ5¹Lµ�¶@¾x¿x»º²V¯±°hÆ É ¿x»�¶@¹?µ�»R®h²�®h¾<Æ5¿x®h¿d¯±¿x®h¹ a å ¶@¿x¾ æIç 2-¯±¹3»ºÆÂ�]tÃ�Çh®hÀh¹3»z¯±°h¹�Ç�®h² É ¿x»RÆÊÀ�²�³R¯º»º²@ÅhÇ�¯±²@¿x®.Ç�³º²�®�¾7°>4�¸�¿�¯º°h¹"³R¹"³�¼�°h²�·'°�µ�»R¹�Æ5¿x®h¿�¯º¿x®h¹ a å ¶�¿�¾ æIç 2¯±¹3»ºÆ Â�]tà å P�µdïdµd²xµ�®hÀaÓêµd»ºÇh¿xï�µ 0dØ sUsbs ç Í+2® å +³R°h¿xÇ�¯=4U0 ô-,b,|õ ç ½³º°h¿xÇ�¯=4<¾|µ = ¹µ~¸Ì¿�¶64f®h¿xÆ5²×µ�¶�2¯±²@ÆȹêÇh®h² É ¿�»ºÆk22Àh²@³R¯±»R²�ÅhÇ�¯±²�¿�®¨µ�¶@¾x¿x»R²@¯±°�Æ É ¿�»�¶�¹?µd»º®h²@®h¾­µ­·"¶�µ�³º³�¼�°h²@·'°Ë²@®h·"¶@ÇhÀh¹"³�Æ5¿x®h¿d¯±¿x®h¹a å ¶�¿x¾ æ ç 2�¯±¹"»RÆ Â�]tÃ+Í � µd¯º¹"»Á ½³º°h¿xÇ�¯=4ãµ�®�À�R�µ�ÆÈ¿�®¨¾|µ = ¹ µ�¸�¿x¶64f®h¿xÆ5²×µd¶52-¯º²�Æ5¹.µd¶�¾x¿x»R²@¯º°hÆÉ ¿x»�¶�¹"µ�»º®�²�®h¾5µÒ·"¶×µd³º³�¼�°h²�·'°ª²�®h·3¶�ÇhÀh¹3³�Æ5¿x®h¿d¯±¿x®h¹ a å ¶�¿�¾ �æ � å ¶�¿x¾L¶@¿x¾ æIç � ç 2-¯º¹"»RÆëÂ�]tÃ0Çh®hÀh¹3»·"¿�®h³R¯±µ�®|¯�22Å�¿xÇh®�Àh¹"Àª¸h»º¿fÀhÇh·%¯�À�²�³R¯º»º²@ÅhÇ�¯±²@¿x®h³ å +³º°�¿xÇ�¯=4�µ�®hÀ^R�µ�Æ5¿x®O0 ô1,U,be ç ÍôUõU,

Page 172: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

ÿ����Óÿ��-A ����� �����������������& (S��(��$(����������Î�¹<¾x² = ¹}µ�®�µ�¶�¾�¿x»º²V¯±°hÆ É ¿x»z¶@¹?µ�»R®h²�®�¾7ÆÈ¿�®h¿�¯±¿�®h¹<Â�]tÃ¥Ç�®hÀh¹"»~¯º°h¹<Çh®h² É ¿x»ºÆòÀh²�³Ó¯±»R²�ÅhÇ�¯º²�¿x®GÍ�+ ɯ±°�¹ZÀh¹3³º²@»º¹"Àêµ�·"·3Çh»±µd·N4�¶�¹ = ¹"¶�{�²@³�·"¿x®h³Ó¯'µ�®|¯�µ�³tµ É Çh®h·%¯±²@¿x®ª¿ É æ å ¯±°h¹(®>ÇhÆ7Å�¹"»^¿ É = µ�»R²×µ�Å�¶�¹"³ ç 0¯±°�¹"®�¯±°h¹}µ�¶@¾x¿x»R²@¯±°�Æò¶@¹?µ�»R®h³^Ø Ù#i Ú ÛÝÜ2Þ�ß-j 2�¯±¹3»ºÆ�Æ5¿x®h¿d¯±¿x®h¹�Â�]tí¿ = ¹"» æ = µ�»R²×µ�Åh¶@¹"³z²@®�¸Ì¿x¶54 å æI篱²@ÆȹdÍ å Îu¹�®h¿�¯±¹�¯±°hµd¯+¯º°h¹�µ�¶@¾x¿x»R²@¯±°�Æp¿ É å +³º°�¿xÇ�¯=4ȵ�®hÀ�R�µ�Æ5¿x®O0 ô1,b,Ue ç É ¿�»½¶@¹?µ�»R®h²�®h¾}Æ5¿x®h¿82¯±¿�®h¹�Â�]tÃê¼�²V¯±° a åRå ¶@¿x¾ æIç � å ¶@¿x¾L¶�¿x¾ æ ç � ç ¯º¹"»RÆȳLµ�¶@³º¿<»R¹ o Çh²�»R¹"³K¯º°iµd¯{�Å̹+·"¿x®h³Ó¯'µ�®|¯²�®�¿�»ºÀh¹3»¯±¿Áµ�·'°h²�¹ = ¹7¸�¿x¶54 å æIç »ºÇh®|¯º²�Æ5¹�Í ç R~°h²�³�²�³^¯±°h¹(´i»º³Ó¯<¸Ì¿�¶64f®h¿xÆ5²×µ�¶Ì¯±²�Æ5¹Zµ�¶�¾x¿�»º²@¯º°hÆ�¼�°h²@·'° Çh³R¹"³¿x®h¶54r»ºµ�®hÀh¿�Æp¹Fd�µ�Æ5¸h¶@¹"³�µ�®�À5³RÇh·"·3¹"³R³ É Çh¶@¶64�¶@¹?µ�»R®h³KÆ5¿x®h¿�¯º¿x®h¹~Â�]�Ãê¼�²@¯±°�Æ5¿x»R¹+¯±°iµd®Èµ�¸�¿x¶54L2¶�¿�¾|µ�»º²V¯±°hÆ5²�·�®>ÇhÆrÅ̹3»K¿ É ¯±¹3»ºÆ5³"ÍEÎu¹+µ�¶�³R¿�³º°h¿8¼�¯±°iµ�¯K¹"³R³º¹3®>¯º²×µ�¶@¶64}¯±°�¹z³±µ�Æ5¹zµd¶�¾x¿x»R²@¯º°hÆÔ¶�¹"µ�»º®�³= µ�»º²@¿xÇh³�·"¶×µd³º³º¹3³K¿ É ³RÆ.µ�¶@¶>·"¿x®h³Ó¯'µ�®|¯�22Àh¹3¸�¯±°Ò·"²@»º·3Çh²@¯º³�¼�°h²@·'°�·"¿xÆ5¸hÇ�¯º¹zÆÈ¿�®h¿�¯±¿�®h¹ É Çh®�·3¯±²@¿x®h³�¿x®É ¹3¼ = µ�»º²�µ�Åh¶@¹"³"Í�R~°h¹"³R¹Z»R¹"³ºÇ�¶@¯±³~¹qdf¯±¹"®�Àª¯±¿�¶@¹?µ�»R®h²�®�¾7Çh®hÀh¹3»^µ�®>4�·"¿x®�³R¯'µd®>¯�22Å�¿xÇh®hÀ�¹"À�¸h»º¿fÀhÇh·%¯Àh²@³R¯±»R²�ÅhÇ�¯±²�¿�® Í:�Çh»�µ�¶@¾x¿x»R²@¯±°�Æè·"¿xÆrÅh²�®h¹3³t²@Àh¹?µ�³ É »º¿xÆ � ²�®�²×µ�¶ KFWtXUZ5[ �T³t²@® �iÇh¹3®>¯º²×µ�¶w¸iµd¸Ì¹3» å � ²�®�²×µ�¶ KFW)XUZ5[ 0ô1,b, Ñ ç ¿x®�¶�¹?µd»º®h²@®h¾ ��u / É Çh®�·3¯±²@¿x®h³(Çh³º²@®h¾�¯±°h¹ÈÃi¿xÇh»R²�¹"»�¯º»±µ�®�³ É ¿x»RÆ µ�®hÀu +³º°h¿�Ç�¯=4�µ�®hÀ�R&µ�2Æ5¿x®{�T³�¸iµ�¸�¹"» å +³º°�¿xÇ�¯=4*µ�®�ÀhR&µ�Æ5¿x®O0 ô1,b,be ç ¿x®k¶�¹"µ�»º®�²�®h¾ÁÆ5¿x®h¿d¯±¿x®h¹ É Çh®h·%¯±²�¿�®h³<Çh³º²@®h¾Á¯±°�¹Ãi¿xÇh»R²�¹"»}¯±»ºµ�®h³ É ¿x»ºÆ�Íê �4uµd®iµ�¶64�73²�®h¾�¯±°h¹�Ãi¿xÇ�»º²�¹3»<¯±»±µd®h³ É ¿x»RÆ ¿ É ��u / É Çh®h·%¯±²�¿�®h³<0 � ²@®h²×µ�¶ KFWXUZ5[ ³º°�¿U¼½¹"À㯺°iµd¯Èµ�¶@ÆÈ¿x³Ó¯5µ�¶�¶+¿ É ¯º°h¹êÃi¿xÇ�»º²�¹3» W ¸�¿U¼½¹"»5³º¸�¹"·%¯±»RÇhÆ X ¿ É µd®L4 �Yu / É Çh®h·%¯±²@¿x®²�³�·"¿x®|¯±µ�²�®h¹3À ²�® W ¶�¿8¼ X Ãi¿xÇ�»º²�¹3»�·3¿�¹ � ·"²�¹3®|¯±³<0G²�ÍT¹�Ít·3¿�¹ � ·"²�¹3®|¯±³�¼�°h²@·'° ·"¿x»R»º¹3³º¸�¿x®hÀ ¯±¿.³ºÆȵ�¶�¶³ºÇ�Åh³º¹%¯±³^¿ É = µd»º²×µdÅh¶�¹3³"Í å Îu¹<¾x² = ¹}µ7Àh¹%¯'µ�²@¶�¹3À�¿ = ¹"» = ²@¹3¼ ¿ É Ãi¿xÇ�»º²�¹3»zµ�®hµ�¶64f³º²@³z¿x®�¯±°�¹} +¿�¿�¶�¹?µd®·"Ç�Å̹Ҳ�®�Pf¹"·3¯º²�¿x® ô1s Í Ø�Í Ø�Í ç R~°h¹3²�»<¶�¹"µ�»º®�²�®h¾�µ�¶@¾x¿x»R²@¯±°�Æá¹3³R¯º²�Æȵd¯±¹3³�¹?µ�·'°�¶�¿8¼pÃi¿xÇh»º²@¹"»�·3¿�¹ � 2·"²@¹"®|¯KÅ>4Z³±µ�Æ5¸h¶@²�®h¾�µ�®hÀ7·3¿x®h³Ó¯±»ºÇ�·3¯±³Kµ�®�µ�¸h¸�»º¿-d�²�Æȵd¯±²@¿x®(¯º¿��ÈÇ�³º²�®�¾t¯±°h¹3³º¹z¹3³R¯º²�Æȵd¯±¹3À7Ãi¿xÇh»R²�¹"»·"¿f¹ � ·3²�¹"®|¯º³"Í�+ Ét� ²�³}¯±°h¹.³º²V73¹.Å�¿xÇh®hÀ É ¿x»r¶@¿U¼óÃ�¿�Çh»º²@¹"»(·"¿f¹ � ·"²@¹"®|¯±³N0K¯±°h¹3®­³º²�®�·"¹È¯º°h¹"»R¹�µ�»R¹� ß �"� Ã�¿�Çh»º²@¹"»·"¿f¹ � ·"²@¹"®|¯±³~·3¿x»º»R¹"³º¸�¿x®hÀ�²�®h¾r¯º¿7³ºÇ�Åh³º¹%¯±³~¿ É�� = µ�»º²�µ�Åh¶�¹3³¯º°h¹<µ�¶@¾x¿x»R²@¯±°�ÆÄ»º¹ o Ç�²�»º¹3³»º¿�Çh¾x°h¶54 æ� ¯º²�Æ5¹�³Ó¯±¹3¸h³"Í � ²�®h²�µ�¶ KFW�XUZ5[ ³º°�¿U¼½¹"À�¯º°iµd¯ É ¿x» ��u / ·"²@»º·"Ç�²@¯±³ � ²�³Ò¹"³R³º¹3®>¯º²×µ�¶@¶64¸�¿x¶64 å ¶@¿x¾ æIç Mf¯º°h²�³»º¹3³ºÇh¶V¯+¼+µ�³+¶�µd¯±¹3»½³R°iµ�»R¸Ì¹3®h¹"À É ¿x»½Â�]tÃ É ¿x»ºÆrÇh¶×µ�¹�Å>4�Óêµ�®h³R¿xÇh» å Óêµ�®h³R¿xÇh»N0ô1,b,xõ ç Í:�Çh»7µ�¶@¾x¿x»R²@¯±°�Æ ¹Fdf¯±¹3®hÀh³7¯±°�²�³7µd¸h¸h»º¿xµ�·'°¥²�®�¯±°h¹ É ¿x¶�¶@¿U¼�²@®h¾�¼zµ-4 Æ.� ¹%¯ u s ��u / Å̹.µ·"¶�µ�³º³Ò¿ É +¿f¿x¶�¹"µ�® É Çh®h·%¯±²�¿�®h³�¼�°h²�·'°0¼+¹�¼½¿xÇh¶�À�¶�²@ï�¹Á¯º¿�¶@¹?µ�»R® Í�P�Çh¸h¸�¿x³R¹�¯º°iµd¯ u °iµ�³Ò¯±°�¹É ¿x¶@¶�¿8¼�²�®h¾r¸h»º¿x¸�¹"»Ó¯±²@¹"³ Æ

ô Í^Ãi¿x»z¹ = ¹3»�4���� u ¯º°h¹"»R¹(²@³�µ7³º¹%¯��O�}¿ É W ²�Æ5¸�¿x»R¯±µ�®|¯ Xr= µ�»R²×µ�Åh¶@¹"³½³ºÇh·'°�¯±°iµd¯^µd¶�Æ5¿x³R¯zµ�¶�¶¿ É ¯±°h¹�¸Ì¿8¼½¹"»z³R¸Ì¹3·3¯±»RÇhÆò¿ É ��²@³½·3¿x®|¯'µ�²@®h¹"ÀÁ²@®ÁÃi¿xÇh»R²�¹3»½·3¿�¹ � ·3²�¹3®>¯º³z·"¿�»º»º¹3³º¸�¿x®hÀh²@®h¾Z¯º¿³RÇhÅh³º¹%¯±³�¿ É �O�b_ô1ebs

Page 173: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

Ø�Í)R~°h¹3»º¹²�³Eµ�®(¹ � ·"²@¹"®|¯&µ�¶�¾x¿�»º²@¯º°hÆ�¼�°h²�·'°Z²�Àh¹3®|¯±²@´h¹"³I¯±°h¹³º¹%¯��O� É »º¿xÆ »±µ�®hÀ�¿xÆÔ¹Fd�µ�Æ5¸h¶@¹"³"Íå PfÇh·'° µ�®Ïµ�¶�¾�¿x»º²V¯±°hÆ�0+¼�°h²@·'°ã¼+¹*¾x² = ¹ ²@®�P�¹"·%¯±²@¿x® ô1s ÍðÑ�Í ô 0�²�³Á²@Æȸh¶@²�·3²@¯Ò²�® å +³R°h¿xÇ�¯=4ϵd®hÀR�µ�ÆÈ¿�®O0 ô1,b,be ç µ�®hÀ­»R¹ o Çh²�»R¹"³Ò¿x®h¶54�¯º°iµd¯D�ãÅ�¹ÁÆ5¿x®h¿�¯º¿x®h¹�Í ç Î�¹�·?µd®�¶@¹?µ�»R®0µ�®­Çh®hï>®h¿8¼�®É Çh®h·%¯±²@¿x®�� É »º¿xÆp³ºÇh·'°�µ(·"¶�µ�³º³ u Å>45´h»º³R¯½²�À�¹"®|¯±² É 4�²@®h¾<¯±°h¹�³º¹%¯p�O�x)|¯±°�¹"®Á¹3³R¯±²@Æ.µ�¯±²�®�¾�¯±°h¹^¶�¿8¼Ãi¿xÇh»R²�¹"»&·"¿f¹ � ·"²@¹"®|¯±³L¼�°h²�·'°5·"¿�»º»º¹3³º¸�¿x®hÀ5¯±¿}³ºÆȵ�¶�¶�³ºÇhÅ�³º¹3¯º³+¿ É �O�tµd®hÀÈÇ�³º²�®�¾}¯º°h¹"³R¹�¹3³R¯º²�Æȵd¯±¹3³¯±¿�·"¿�®h³R¯º»ºÇh·%¯�µ�®�µ�¸h¸h»R¿-d�²@Æ.µ�¯±²�¿�® ¯±¿\��_R�¿�³º¹3¹È¼�°>4*¯±°h²@³<¼+¿x»Rï>³<0&®�¿�¯±¹Ò¯±°iµ�¯Z³R²�®h·3¹��­²�³}²@®��u / µ�¶�Æ5¿x³Ó¯Kµ�¶@¶�¿ É ¯º°h¹~¸�¿U¼½¹"»�³R¸Ì¹3·3¯º»ºÇhÆÄ¿ É �.²�³K²@®r¯±°h¹~¶@¿8¼¨Ãi¿xÇ�»º²�¹3»&·3¿�¹ � ·3²�¹3®>¯º³<M>Æ5¿x»º¹3¿ = ¹"»N0¸h»R¿x¸Ì¹3»R¯=4 å ô ç ²@Æȸh¶@²�¹3³7¯±°iµd¯.µ�¶@ÆÈ¿x³Ó¯5µ�¶�¶~¿ É ¯±°h¹ª¸Ì¿8¼½¹"»È³º¸�¹"·3¯º»ºÇhÆ ¿ É � ²�³5²�®�¯±°h¹ªÃi¿xÇh»R²�¹"»·"¿f¹ � ·3²�¹"®|¯º³+¼�°h²�·'°�·3¿x»º»R¹"³R¸Ì¿x®�ÀÁ¯±¿7³RÇhÅh³º¹%¯±³~¿ É �O�L_�Ð+¿x®h³R¹ o Çh¹3®>¯º¶64.²@¯½Æ7Çh³R¯+Å̹�¯±°h¹�·?µ�³R¹t¯±°iµ�¯µ�¶@ÆÈ¿x³Ó¯�µ�¶�¶|¿ É ¯±°h¹¸Ì¿8¼½¹"»K³R¸Ì¹3·3¯º»ºÇhÆ�¿ É �Ò²�³E²�®Z¶�¿8¼0Ãi¿xÇh»R²�¹"»w·3¿�¹ � ·3²�¹3®>¯º³�¼�°h²@·'°7·"¿�»º»º¹3³º¸�¿x®hÀr¯º¿³ºÇ�Åh³º¹%¯±³+¿ É �O�b_>R~°>Çh³²�®5¿xÇh»�³º¹%¯º¯º²�®h¾}¼+¹^®h¹3¹"À.¿�®h¶64�¹3³R¯º²�Æȵd¯±¹z¯º°h¹ �<� �< <��o� Ãi¿xÇh»º²@¹"»�·3¿�¹ � ·3²�¹3®>¯º³¼�°h²@·'°�·3¿x»º»R¹"³R¸Ì¿x®�ÀÁ¯±¿ W ³RÆ.µ�¶@¶ X ³ºÇhÅ�³º¹3¯º³�¿ É = µd»º²×µdÅh¶�¹3³+²�®¡�O�b_$+ É)¢ �O� ¢8£ æ ¯±°h¹3®�¯±°h²@³z²�³½Æ7Çh·'°Æ5¿x»º¹<¹ � ·3²�¹3®>¯�¯º°iµ�®�¹"³R¯º²�Æȵd¯±²@®h¾�µ�¶@¶ � ß � � ¶@¿U¼ Ãi¿xÇh»R²�¹"»~·3¿�¹ � ·"²�¹3®|¯±³"Í÷<ø�¤q¥ ¦pû�§�¨"þ-© þ-ª�«7ûLþ<§�¬Î�¹r¼�»º²V¯±¹^­ æ`® ¯±¿�Àh¹3®h¿�¯±¹Z¯±°h¹�³R¹3¯°¯ ôb± _-_-_ ± æ³² µ�®hÀkÇ�³º¹�·?µd¸h²@¯±µ�¶�¶@¹3¯R¯±¹"»R³�´ ±lµ�±Hu É ¿�»t³ºÇ�Åh³º¹%¯±³¿ É ­ æ�® _ Î�¹�¼�»º²V¯±¹ ¢ ´ ¢ ¯±¿�Àh¹"®�¿�¯±¹�¯±°�¹5®>ÇhÆrÅ̹3»(¿ É ¹"¶@¹"Æ5¹"®|¯±³<²@®¶´�_·R~°h¹5¶�¿8¼½¹"»º·"µ�³º¹Ò¶@¹3¯º¯º¹"»�¸µ�¶V¼zµ-4f³�Àh¹"®h¿d¯±¹"³�µ�® æ 2-Åh²@¯�³Ó¯±»R²�®h¾k¸^¹�¸Oº#_<_-_�¸ ß �»¯ s3±?ô ² ß _Îu¹½¼f²@¹3¼¿¾+¿f¿x¶�¹"µ�®Ô·"²@»º·3Çh²@¯º³kµ�³ Å�¹"²@®h¾ã·"¿�Æȸ�¿x³º¹3À ¿ É Ö�]�Â?vb: ì vU]Y:�R ¾|µd¯±¹3³ ¼�°h¹"»R¹Ö�]�Âóµ�®hÀ�: ì ¾|µd¯±¹3³�°iµ-¼x¹rÇh®>ÅÌ¿xÇ�®hÀh¹"À É µ�®�²�®kµ�®�À ®h¹"¾xµd¯±²@¿x®h³t¿f·"·3Çh»�¿x®�¶64�¿x® ²@®h¸hÇ�¯º³"Í�Î�¹¼f²�¹%¼�¾+¿f¿x¶�¹"µ�® É Çh®h·3¯º²�¿x®�³^¿�® æ ¼dµ�»º²�µ�Åh¶�¹3³�µ�³�»º¹"µ�¶·¼�µd¶�Çh¹3À É Çh®�·3¯±²@¿x®h³~¼�°h²@·'°êÆȵ�¸h¯ s ±?ô ² ß ¯º¿¯LÀ ôb±?ô ² _�Ö�¾+¿�¿�¶�¹?µd® É Çh®h·%¯±²�¿�®\� Æ ¯ s ±"ô ² ß?Á ¯LÀ ôb±"ô ² ²�³�A��ÃÂ#� W �àK ² É ·'°iµ�®�¾x²�®h¾7¯º°h¹�¼dµ�¶�Çh¹¿ É µ�®�²�®h¸hÇ�¯�Åh²@¯ É »R¿xÆ s ¯±¿ ô ®h¹N¼�¹"»^·?µ�Ç�³º¹"³�¯±°�¹?¼dµ�¶@Çh¹<¿ É �꯱¿Ò·'°iµ�®h¾�¹ É »R¿xÆ ô ¯±¿^À ô _+ É³Ä ²@³�µ7Àh²�³Ó¯±»º²@ÅhÇ�¯º²�¿x®�µd®hÀ��ê²�³�µk¾+¿f¿x¶�¹"µ�® É Çh®h·%¯±²@¿x®�¿x®h¯ s ±?ô ² ß ± ¯±°h¹3®êµ�³~²@® å ¾+³R°h¿xÇ�¯=4µ�®hÀ¶R�µ�Æ5¿x®O0 ô1,U,be M ñ�µ�®h·3¿�·'ïkµd®hÀ Óêµ�®h³R¿xÇh»<0 ô1,U,�ô MOÅ}µ�°�® KFW�XUZ5[ 0 ô-, íxí ç ¼½¹Ò³±µ-4 ¯º°iµd¯<¯±°�¹I/Â-Æ J K Â#Ç K �"Èt¸�É��ÃÂ\��Ê�I W/Ë\EHKHG=Ì�K Ç W�W � Ä ²@³^¯º°h¹r¸h»R¿xÅiµ�Å�²�¶�²V¯=4Á¯±°hµd¯�� å ¸ ç Àh²6Í̹"»R³ É »º¿�ÆÎ� ånÏ ç ±¼�°h¹3»º¹ Ï ²�³�¸u¼�²V¯±°*¯±°h¹DÐn2�¯±°�Åh²@¯?�i²@¸h¸Ì¹3Àuµd®hÀ�¸¥²�³}Àh»±µ?¼�® É »º¿xÆ Ä _wÃi¿x»<¹"µ�³º¹5¿ É ®h¿�¯'µ�¯±²�¿�®¶�¹%¯?�1ÉVÑ Ò<Àh¹"®�¿�¯±¹7¯º°h¹ É Çh®h·%¯±²@¿x®k¿xÅ�¯±µ�²�®h¹3À É »º¿�Æ��uÅ>4ê´3d�²@®h¾�¸�É�¯º¿ s µ�®hÀk¶@¹3¯?�1ÉVÑÓº^Å�¹�Àh¹3´i®�¹"À³º²@ÆȲ@¶×µ�»R¶64�ÍEÎ�¹<¯±°>Çh³�°iµ-¼x¹

Ô Õ Ñ É å � ç ¹ Õ�»Õ ­Õ�1ÉVÑ Ò å ¸ ç�Ö¹×�1ÉVÑÓº å ¸ ç"® ¹ ôØ`Ø Õ ­ ¢ ��É6ÑÓº�ÀÙ�1ÉVÑ Ò ¢ ® _ô1e�ô

Page 174: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

Ãi¿x»~Æ5¿x®h¿�¯º¿x®h¹c�꯱°h²@³~·?µ�®�Å�¹ É Ç�»R¯±°�¹"»�³º²@Æȸh¶@²@´i¹3À.¯±¿Ô Õ Ñ É å � ç ¹ ôØ Ø Õ ­Ú��É6ÑÓº�ÀÙ�1ÉVÑ Ò ® ¹

ôØ å Ø Õ ­Ú��ÉVÑÓº ® À Ø Õ ­Ú��ÉVÑ Ò ® ç _ å ô1s Í ô çÿ����zÛp�Óÿ �����L�$ ��Ü�Ý��$��(��tÞLß�à��!�"á?!�âDã�ä ����� �������R~°h¹�¶@¹?µ�»R®h²�®h¾ªÆÈ¿fÀh¹3¶¼½¹�·"¿x®�³º²�À�¹"»7²@®u¯±°h²@³r·'°iµd¸�¯±¹3»�²�³7µ À�²�³R¯º»º²@ÅhÇ�¯±²@¿x®32-³º¸�¹"·"²V´i·�¼x¹"»R³º²@¿x®­¿ ɯ±°�¹�ÕÌÖ^Ðó¶�¹?µd»º®h²@®h¾ Æ5¿�Àh¹3¶½¼�°h²@·'°0°iµ�³�Å�¹"¹"®0³Ó¯±ÇhÀh²@¹"À0Å>4¥Æȵ�®>4¥»R¹"³º¹"µ�»º·'°�¹"»º³N0+¹dÍð¾hÍ å ¾½¶�ÇhÆKFWoXUZ5[ 0 ô-,b,Uå M�¾+¹"¶@¶×µ�»R¹U0 ô1,b, Ø�Mæ¾+³R°h¿xÇ�¯=4 KFWoXUZ5[ 0 ô1,b,b, M�¾+³º°�¿xÇ�¯=4�µ�®hÀçR�µ�Æ5¿x®O0 ô1,b,be M�ÃiÇh»º³Ó¯KFW�XUZ5[ 0 ô1,b,�ô MGñ�µ�®h·3¿�·'ïkµd®hÀ]Óêµ�®h³R¿xÇh»N0 ô1,b,�ô M îxµ�·'ï>³º¿x®#0 ô-,b,|ö M`Å<°iµ�»ºÀ�¿x®O0 ô1,U,Uå MOÅ<Çh·3¹"»ºµ KFWXUZ5[ 0 ô1,b,Uå M�èI²�®h²�µ�¶ KFWpXUZ5[ 0 ô1,U, Ñ M.Óêµd®h³º¿xÇ�»<0 ô1,b,|õ M . ¹3»ºÅ�¹"Çh»R¾�¯-0 ô1,U,bs M . ¹"»RÅ̹3Çh»º¾d¯-0 ô1,b, í ç Í}èI¹%¯u Å�¹ªµk·3¶×µ�³R³5¿ É ¾+¿f¿x¶�¹"µ�®­·"¿�®h·"¹3¸�¯±³È¿�¼�¹"»^¯ s ±?ô ² ß ± ¶@¹3¯ Ä Å̹�µ*¸h»º¿xÅhµ�Åh²�¶@²@¯=4�Àh²�³Ó¯±»R²�ÅhÇ�¯º²�¿x®¿1¼x¹3»o¯ s ±?ô ² ß ± µ�®hÀª¶�¹%¯��B� u Å̹rµ�®êÇh®�ïf®�¿U¼�® ¯±µ�»º¾�¹3¯ É Çh®h·3¯º²�¿x®GÍ~Öp¶�¹"µ�»º®�²�®h¾Èµ�¶�¾�¿x»º²V¯±°hÆé´É ¿x» u ¯±µ�ï�¹3³rµ�³}²@®h¸hÇ�¯(µ�®�µ�·3·"Çh»ºµ�·N4*¸iµd»±µ�Æ5¹3¯º¹"» shê;ëkêÊô µd®hÀuµ�·"¿�®�´iÀh¹3®h·"¹5¸iµ�»ºµ�Æȹ%¯±¹3»sìê í�ê ô _�Â�Çh»R²�®h¾0²V¯±³�¹Fd�¹"·3Ç�¯±²@¿x®Ë¯º°h¹�µ�¶�¾x¿�»º²@¯º°hÆ °iµ�³�µ�·"·3¹"³R³ª¯±¿âµ�® K Ï X A Ì�ZVK � E�X Ç ZVKØcî å � ± Ä ç ¼�°h²@·'°O0�¼�°h¹3®�ï>Çh¹3»º²@¹"ÀO0h¾�¹"®h¹3»±µd¯º¹"³�µ7»±µd®hÀh¿xÆò¶×µ�Å�¹"¶@¹"ÀÁ¹qd�µ�Æȸ�¶�¹kðr¸ ± � å ¸ ç�ñ ¼�°h¹"»R¹¸�²�³½Àh»±µ?¼�®�µ�·3·"¿x»RÀh²�®h¾Z¯±¿ Ä _�R~°h¹�¶@¹?µ�»R®h²�®h¾rµ�¶�¾�¿x»º²V¯±°hÆÄ¿�Ç�¯±¸hÇ�¯±³~µr°>4f¸�¿�¯±°h¹3³º²@³tòÁ¼�°�²�·'°�²�³+µ¾+¿f¿x¶�¹"µ�® É Ç�®h·3¯º²�¿x®Á¿1¼�¹"»Y¯ s ±"ô ² ß M�¯±°�¹�¹"»R»º¿x»½¿ É ¯±°�²�³+°>4f¸�¿�¯±°h¹3³º²@³~²�³Àh¹3´i®�¹"À�¯±¿7Å�¹t¹3»º»R¿x» å ò ç ¹Õ�» Õ ­Úò å ¸ ç½Ö¹¿� å ¸ ç"® _�Îu¹�³ºµ140¯º°iµd¯D´ ZVKfXUE  GDu J  m�KFE Ä ² É^É ¿x»Ò¹N¼x¹3»�4ó�;� u µ�®�À sóêë<±Hí°êËôb± ¼�²@¯º°È¸h»R¿xÅiµ�Å�²�¶�²V¯=4�µd¯¶@¹?µ�³Ó¯ ô À í µd¶�¾x¿x»R²@¯º°hÆ;´Ï¿xÇ�¯º¸hÇ�¯º³zµ}°L4f¸�¿�¯±°�¹"³º²@³pò5¼�°�²�·'°Á°iµd³¹"»R»º¿x» å ò ç}ô ë _ÿ����zÛp�zÛ õ»ö��÷�����3!·��O�$�÷��(�Ý� ����õD��ø��� ùq(� úèI¹3¯ûÏÀh¹3®h¿�¯±¹z¯º°h¹zÇh®�² É ¿x»RÆ Àh²�³Ó¯±»º²@ÅhÇ�¯º²�¿x®Z¿�¼�¹"»}¯ s ±"ô ² ß _UR~°h¹+³º¹3¯L¿ É µ�¶�¶>»º¹"µ�¶3¼dµ�¶@Çh¹"À É Çh®h·3¯º²�¿x®�³¿x®�¯ s ±"ô ² ß Æȵ-4rÅ�¹S¼f²�¹%¼+¹3À.µ�³KµZØ ß 2-Àh²�Æ5¹"®�³º²�¿�®iµ�¶L¼x¹"·%¯±¿x»�³R¸iµ�·3¹�¼�²V¯±°�²@®h®h¹"»&¸h»º¿fÀhÇh·%¯LÀh¹3´i®�¹"ÀÅ>4 ðz� ±fü ñ ¹ÔØLý ß þ

ÿ���� ÒfÑÓº���� � å ¸ ç ü å ¸ ç ¹ Ø�� ­Ú� ü ®µ�®hÀ0®h¿�»ºÆ¦À�¹3´i®h¹3ÀâÅ>4N��^¹ � ðz� ± � ñ _��<²6¼�¹"®�µ�®>4¥³RÇhÅh³R¹3¯D´ � ­ æ`® ± ¯º°h¹�Ã�¿�Çh»º²@¹"»rÅhµ�³º²@³É Çh®h·%¯±²@¿x®���� Æ ¯ s ±?ô ² ߶Á ¯LÀ ôb±?ô ² ²@³rÀh¹%´i®h¹"À�Å>4���� å ¸ ç ¹ å À ô ç � ����� � ± ¼�°h¹"»R¹ î ²@³r¯±°�¹³ºÇ�Åh³º¹%¯Ò¿ É ­ æ�® Àh¹%´i®h¹"À0Å>4BÐk� î ²6Íç¸�É�¹ ô _³+-¯7²�³Z¼½¹"¶@¶+ï>®h¿8¼�®­¯±°iµ�¯7¯±°h¹�Ø ß Åhµ�³º²@³ É Çh®h·q2¯±²@¿x®h³���� É ¿x»ºÆmµ�®k¿x»Ó¯±°h¿�®h¿x»ºÆȵ�¶wÅiµ�³R²�³ É ¿�»�¯±°�¹�¼x¹3·3¯±¿�»<³º¸iµ�·3¹�¿ É »º¹"µ�¶³¼dµ�¶@Çh¹"À É Ç�®h·3¯º²�¿x®h³t¿x®¯ s ±?ô ² ß ME¼+¹�»R¹ É ¹3»r¯±¿ ¯º°h²�³7Åhµ�³º²@³7µ�³ W/Ë�K � C XÃG I G Íh+2®­¸iµ�»R¯º²�·3Çh¶×µ�»N0�µ�®>4 É Çh®h·3¯º²�¿x®Ù��·?µ�®0Å�¹ô1e Ø

Page 175: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

Çh®h²6ï>Çh¹"¶54u¹qd�¸�»º¹"³R³º¹3À¨µ�³k� å ¸ ç ¹�� ��� å ´ ç ��� å ¸ ç ± ¼�°h¹"»R¹Á¯±°h¹�¼dµ�¶@Çh¹"³ �� å ´ ç µ�»R¹Áï>®h¿8¼�®âµd³¯±°�¹}Ã�¿�Çh»º²@¹"»z·3¿�¹ �È·3²�¹3®>¯º³�¿ É �ê¼�²V¯±°�»º¹3³º¸�¹"·%¯�¯º¿7¯±°h¹!�uÅhµ�³º²@³"Í�P�²@®h·"¹<¯±°�¹ É Çh®h·3¯º²�¿x®�³"��� É ¿x»ºÆµ�®­¿x»Ó¯±°h¿�®h¿x»ºÆȵ�¶Åiµ�³R²�³<0�¯º°h¹�¼dµ�¶�Çh¹.¿ É �� å ´ ç ²�³¡ðz� ± ��� ñ MKµd¶�³º¿ 0�ÅL4u¶�²@®h¹?µ�»R²@¯=4*¼½¹�°iµ-¼x¹�¯±°iµ�¯� å ¸ ç$# ü å ¸ ç ¹ � � å �� å ´ ç�# �ü å ´ çºç ��� å ¸ ç _ÌÖ�®h¿�¯±°�¹"»t¹"µ�³�4ª·"¿�®h³º¹NïfÇ�¹"®h·3¹�¿ É ¿x»R¯º°h¿x®h¿�»ºÆȵ�¶�²V¯=4²�³�Õ�µ�»º³R¹N¼dµ�¶Ä�ð³�²�À�¹"®|¯±²V¯=4Ø�� ­Õ�&% ® ¹'N��(%S¹ þ�*)*+ ß-, �� å ´ ç %-_+ É �˲�³Áµ½¾+¿f¿x¶�¹"µ�® É Çh®h·%¯±²@¿x®¨¯±°�¹"®�¯±°h²@³�¼dµ�¶@Çh¹ ²�³È¹Fd�µ�·%¯±¶64 ô Í Ã&²@®iµ�¶�¶54b0 É ¿x»�µ�®>4�¾+¿�¿�¶�¹?µd®É Çh®h·%¯±²@¿x®��*µd®hÀ�»º¹?µd¶52z¼dµ�¶@Çh¹"À É Ç�®h·3¯º²�¿x® ü ¼½¹}°iµ-¼x¹}¯±°h¹ É ¿�¶�¶�¿8¼�²@®h¾7¹?µd³º²�¶54�¼�¹"»º²V´i¹"À�²�®�¹<ï>Çiµ�¶@²@¯=4å ¾½³º°h¿xÇ�¯=4�µ�®hÀ�R&µ�Æ5¿x®O0 ô1,b,be M èI²�®h²�µ�¶ KFWSXUZ5[ 0 ô1,b, Ñ ç ÆÕ�»� ­Ú� Ö¹ ³º²�¾�® å ü ç=®·ô Ø�� ­ å ��À ü ç % ® å ô1s Í Ø ç¼�°h¹3»º¹(³º²�¾�® å ü ç ¯±µ�ï�¹3³t¼dµ�¶@Çh¹ ô ² É ü/.�s µ�®hÀ�¯'µ�ïd¹"³)¼dµ�¶�Çh¹kÀ ô ² É ü�ê�s _

÷<ø�¤10 2½§�«rû�ªkþ<ª43 5hªký�§<û76¿ªkþ-8xü5û�© 9 þ-¬�úKûþ;:<5êú�þ"üDªh¬ÿ�����=)�Óÿ >>%����S�$��ù1?������é����*��A@��ø�S�CB��� �"�øÜ�*���R~°h¹ É ¿x¶�¶@¿8¼�²�®h¾¥¶@¹"Æ5Æ.µ30^¼�°h²@·'°Ë²@³�²�Æ5¸h¶�²@·"²@¯È²�® å ¾+³º°h¿�Ç�¯=4 µ�®hÀQR&µ�Æ5¿x®O0 ô1,b,be ç 0t¾�²6¼x¹3³�µ�®¹-�5·3²�¹"®|¯ªµ�¶@¾x¿x»º²V¯±°hÆ É ¿�»�²�Àh¹3®>¯º² É 4f²�®�¾u¯±°h¹k²@Æȸ�¿x»Ó¯'µ�®|¯�¼dµ�»R²×µ�Å�¶�¹"³.¿ É µ0Æ5¿x®h¿d¯±¿x®h¹�¾+¿�¿�¶�¹?µd®É Çh®h·%¯±²@¿x® ÍKÎ�¹(»R¹ É ¹3»�¯º¿�¯±°h²@³�µ�¶�¾x¿�»º²@¯º°hÆʵ�³"D*E�FHGJILKNMOEPKPQ*RNS�TfÍU�VHWXWXY[Z]\A^`_ba KFW � Æ ¯ s ±"ô ² ß Á ¯LÀ ôb±?ô ² C KkX A��ÃÂ#� W �àKdc �1� ZVKfX ÂkÈ J Â#Ç W I �à[feOË�KFEHK I GX  XUZhg � E I WrË A Ê Ë I Ç Ë¶Ë XÃG¡X ÇHÇ KHGHG�W � Ø?î å � ± û ç ±}EqJ  G I/Â Ì � Zji å æ ±"ô � ë<± ¶@¿x¾ ô � í ç W I/A KDGFW=K=Ì�GÈ-� EoXUZ/Z�ë<±Hílk�s ±�X  m Ê�I WrËkÌ�E �UC X C<I Z I Wmi�XUW�ZVKfXÃGFWLô À í � J�W�Ì�J�W/G�XDG1KFW �O�n�÷­ æ�® GFJ Ç Ë^WrË XUWÐø�¶�O��I/A Ì�Z I KHG þ�Oo É � � �� å ´ ç % . ë � Ø X  m Ð ��h�O��I/A Ì�Z I KHG þ�*o É � � �� å ´ ç % ô ë _pnqsrtrOuwvpÅ}µ�°h® KFW}XUZ5[ åºå Å}µ�°�® KFWSXUZ5[ 0 ô1, í�í ç P�¹3·3¯±²@¿x®ªÑ ç °iµ-¼x¹(³º°h¿8¼�®�¯±°iµd¯

Ô � Ñ É å � ç ¹ þ�Oo É � � �� å ´ ç %<_ å ô1s Í Ñ çRE¿Á¸h»º¿1¼�¹7¯±°h¹Z¶�¹"Æ5Æȵ5²V¯t¯±°>Çh³t³ºÇL�5·"¹3³�¯±¿Á³R°h¿8¼ ¯±°iµd¯YÔ � Ñ É å � ç ·"µ�® Å�¹7¹"³Ó¯±²@Æ.µd¯º¹"À�¯±¿.¼�²V¯±°h²@®µ�·3·"Çh»ºµ�·N4 ë � å ¼�²@¯º°.°h²@¾x°È¸�»º¿xÅiµdÅh²�¶@²@¯=4xͳ¾�4yx�ï>Çiµd¯±²@¿x® å ô1s Í ô ç É »º¿xÆ Pf¹"·3¯º²�¿x® ô1s Í Ø}¯º°h²�³L·"µ�®ÁÅ�¹ô1e Ñ

Page 176: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

Àh¿x®�¹5Å>4*¹3³R¯±²@Æ.µ�¯±²�®�¾ Ø�� ­Ú��ÉVÑÓº ® µ�®hÀ Ø�� ­Ú��ÉVÑ Ò ® _g^³º²�®�¾êÐ+°�¹"»º®�¿UÍ�ÅÌ¿xÇ�®hÀh³}²@¯}²@³<¹?µ�³R²�¶64�¼x¹3»º²@´h¹"À¯±°hµd¯Z¼�²V¯±°¥¸�»º¿xÅiµdÅh²�¶@²@¯=4kµ�¯r¶@¹?µ�³Ó¯ ô À í � Ø æ µª³±µdÆȸh¶@¹È¿ É ³º²67"¹.¸�¿x¶54 å æ ±?ô � ë<± ¶@¿x¾ ô � í ç Àh»±µ?¼�®É »º¿�Æ Øcî å � ± û ç ¼�²�¶�¶G·"¿x®|¯'µd²�®ê¸�¿x¶54 å æ ±?ô � ë<± ¶@¿x¾ ô � í ç ¹Fd�µ�Æ5¸h¶�¹3³^¼�²V¯±°^¸�ɹ{z É ¿x»�¹?µd·'°\¼dµ�¶�Çh¹¿ É zh¹ s3±?ô _LÖ ³º¹"·3¿x®hÀϵ�¸�¸h¶�²@·?µd¯º²�¿x®­¿ É Ð+°h¹"»R®h¿UÍËÅÌ¿�Çh®hÀh³5³º°h¿8¼�³.¯º°iµd¯Ò¼�²@¯º°¨¸h»R¿xÅiµ�Å�²�¶�²V¯=4µd¯^¶�¹"µ�³R¯ ô À í � Ø æ µÒ³±µ�Æ5¸h¶�¹<¿ É ¸�¿x¶64 å æ ±?ô � ëN± ¶�¿x¾ ô � í ç ¶�µ�Å�¹"¶�¹3À�¹Fd�µ�Æ5¸h¶�¹3³�Àh»±µ?¼�®êÇ�®h² É ¿x»RÆȶ54É »º¿�Æ ¯1¸ Æ ¸�É#¹|z ² 4�²@¹"¶@Àh³�µ�® ë � å 22µ�·"·3Çh»±µ�¯±¹}¹"³Ó¯±²�Æȵd¯º¹<¿ É Ø�� ­Ú��ÉVÑ } ® _ÿ�����=)�zÛ õ»ö��÷����� ��������ã *���(� �=�$ö�ú:�Çh»�¶�¹"µ�»º®h²@®h¾�µ�¶@¾x¿x»R²@¯±°�Æ�0>¼�°h²�·'°�¼½¹Z·"µ�¶�¶�~HS�KNMNF]�*��F*�s�*��F�S$0G²@³~¾x²6¼�¹"®�Å̹3¶�¿8¼ Æ

@ g^³º¹!D*E�FHGJILKNMOEPKPQ*RNS�T5¯±¿5²�À�¹"®|¯±² É 4�µ�³º¹%¯��O�(¿ É ²�Æ5¸Ì¿x»Ó¯'µ�®|¯}¼dµ�»R²×µ�Å�¶�¹"³3Í@ Â�»ºµ8¼X�p¶�µ�Å̹3¶�¹3À(¹qd�µ�Æȸ�¶�¹"³Sðn¸ º ± � å ¸ º ç�ñ ± _-_<_ ± ðn¸�� ± � å ¸�� çfñ É »º¿xÆ Øcî å � ± û ç _�Ãi¿x»I¹N¼�¹"»�4´�� �O�¼�²V¯±° ¢ ´ ¢ ô � ³º¹3¯����^¹ º� � �É��`º � å ¸ É ç ��� å ¸ É ç _UÃ�¿�»I¹N¼x¹3»�4c´0³ºÇh·'°Z¯±°iµd¯ ¢ ´ ¢ k �¿x»p´ Ö� �O�(³º¹3¯�����¹ s _@ :�Ç�¯º¸hÇ�¯�¯±°h¹}°>4f¸Ì¿d¯±°h¹3³º²�³�³R²�¾x® å ü å ¸ çºç ± ¼�°h¹3»º¹ ü å ¸ ç ¹�� � ���O��� å ¸ ç _R~°h¹<µd¶�¾x¿x»R²@¯º°hÆį±°>Çh³z¹3³R¯º²�Æȵd¯±¹3³ �� å ´ ç É ¿x»�´�� �O� ± ¢ ´ ¢ ô � Å>4.³ºµ�Æ5¸h¶�²@®h¾rµd®hÀ�·"¿x®�³R¯±»RÇh·3¯º³�µ°>4f¸Ì¿�¯º°h¹"³R²�³<Çh³R²�®h¾È¯±°h¹3³º¹�µ�¸�¸h»º¿-d�²�Æȵd¯±¹rÃi¿xÇh»º²@¹"»^·"¿f¹-�5·"²@¹"®|¯±³3Í�R~°h¹°¼dµ�¶�Ç�¹"³t¿ É � µd®hÀ � µd®hÀ¯±°�¹(¸hµ�»±µ�Æ5¹3¯º¹"»~³R¹3¯º¯º²�®h¾�³ É ¿�»"DOE�FHGJILKJM*EPKPQ*RJSLT.µ�»º¹<³R¸Ì¹3·"²V´i¹"ÀªÅ̹3¶�¿8¼}Í

ÿ�����=)��= ����� ��������& (���(��$(������O����� ��� �&���zÞ�� �� ú �����èI¹3¯�� Æ ¯ s ±"ô ² ß»Á ¯LÀ ôU±?ô ² Å̹�µkÆ5¿x®h¿�¯º¿x®h¹��N2-¯º¹"»ºÆ¦Â�]tÃ+Í�R~°h¹�¸h»º¿f¿ É ¯±°iµd¯5µd¶�¾x¿x»R²@¯º°hÆ~HS�KJMNF��*��FO�s����F�SÒ¶@¹?µ�»R®h³���Ç�³º¹"³¿x®h¹^Â�]�Ãk¼�°�²�·'°È¼½¹�·?µ�¶@¶ �>º&¯±¿}³º°�¿U¼¨¯±°iµd¯�D*E�FHG�ILKNM*EPK�Q*RNSLT²�À�¹"®|¯±²V´i¹"³^µ�³ºÆȵ�¶�¶Ì³º¹%¯t¿ É ¼dµ�»º²�µ�Åh¶@¹"³)�O�Zµ�®hÀ�Çh³º¹3³�µ�®�¿�¯±°h¹3»�Â�]tÃ¥¼�°h²@·'°�¼½¹Z·"µ�¶�¶·� % ¯±¿Ò³º°h¿8¼¯±°hµd¯Y�*·"µ�®êÅ�¹(µ�¸h¸h»R¿1d�²@Æ.µd¯º¹"À�Å>4�µ�¸h¸h»R¿1d�²@Æ.µd¯º²�®h¾5Ãi¿xÇh»R²�¹3»~·"¿f¹-�5·"²@¹"®|¯±³�¼�°h²�·'°ª·"¿x»R»º¹3³º¸�¿x®hÀ¯±¿Ò³RÆ.µ�¶@¶G³RÇhÅh³º¹%¯±³�¿ É �O�>_èI¹3¯��>º.Å̹*¯±°h¹�Â�]tÃļ�°h²@·'° ²@³�¿�Å�¯'µ�²@®h¹"À É »R¿xÆ � ÅL4Ï»R¹"Æ5¿1¼�²@®h¾�¹N¼x¹3»�4ϯ±¹3»ºÆ ¼�°�²�·'°·"¿�®>¯±µ�²�®�³<ÆÈ¿x»R¹r¯±°hµ�®*¶�¿x¾ � %�� ß� ¼dµ�»R²×µ�Å�¶�¹"³3Í°Pf²�®h·3¹7¯±°h¹3»º¹5µ�»º¹Òµ�¯<ÆÈ¿�³R¯n�Ò³ºÇh·'°�¯±¹"»RÆȳ<¹"µ�·'°*¿ ɼ�°h²@·'°u²�³}³ºµd¯±²@³R´i¹3ÀuÅ>4�µ�»ºµ�®hÀh¿xÆ�¹Fd�µ�Æ5¸h¶�¹�¼�²@¯º°u¸h»R¿xÅiµ�Å�²�¶�²V¯=4ê¶@¹"³R³(¯º°iµ�® ë � Ñ|ØN� æ ± ¼½¹.°hµ1¼�¹Õ�» � ­Ú� å ¸ ç?Ö¹×�bº å ¸ ç"® ê �� % ß å ¯±°h²@³�¯=4f¸�¹Z¿ É µ�»R¾xÇhÆ5¹"®|¯~¼zµd³^´i»R³R¯�Çh³º¹3Àê²@® å/. ¹"»RÅ̹3Çh»º¾�¯<0 ô1,b,bs çRç ÍèI¹3¯$ ¡�÷­ æ`® Å�¹�¯±°h¹³º¹%¯&¿ É ¼dµ�»º²�µ�Åh¶�¹3³ ¼�°h²�·'°°�>ºGÀh¹"¸�¹"®�Àh³&¿x®#Mx²@¯w²@³E·3¶�¹"µ�» ¯±°iµ�¯ ¢   ¢ ô �E¶�¿�¾ � %�� ß� _

ô1eUå

Page 177: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

Óª¿x»R¹"¿1¼x¹3»L³º²�®�·"¹pÔ � Ñ É å �bº ç ¹ s É ¿x»æÐ ��<  ± ¹Nï>Çiµd¯±²@¿x® å ô-s Í Ñ ç É »º¿xÆ;P�¹3·3¯±²@¿x® ô1s ÍðÑ�Í ô ²�Æ5¸h¶�²@¹"³w¯±°iµ�¯��>º å ´ ç ¹ s É ¿x»p´ Ö�¢ °_P�²�®�·"¹k��µ�®hÀh�bº�µ�»º¹o¾+¿f¿x¶�¹"µ�® É Ç�®h·3¯º²�¿x®h³N0O�¡À��>º�²@³�¹3²@¯±°�¹"» s ¿x»tØ30̳R¿ Ø�� ­ å ��À��>º ç % ® ¹å Õ�» � ­Ú� Ö¹��>º ® ê�ë � í æ _ ¾�4�Õ&µd»º³º¹F¼dµ�¶¶�T³�²�Àh¹3®>¯º²@¯=4Á¼½¹}°iµ-¼x¹

Ø�� ­ å ��ÀÙ�>º ç % ® ¹ þ � å �� å ´ ç À ��bº å ´ çºç %¹ þ�O)H£ å �� å ´ ç À ��bº å ´ çRç % # þ��¤)H£ å �� å ´ ç À ��>º å ´ çºç %¹ þ�O)H£ å �� å ´ ç À ��bº å ´ çRç % # þ��¤)H£ å �� å ´ çºç %ê ë � í æ _R~°>Çh³ � �A¤)H£ �� å ´ ç % ê �¥ ß ± µd®hÀª·3¿x®h³R¹<ï>Çh¹"®|¯º¶64�¼½¹Z°iµ-¼�¹Ð ��< �²�Æ5¸h¶�²@¹"³ þ�*o É � � �� å ´ ç % ê ëí æ _ å ô1s Í å çÎ�¹(³R¹3¯�¯±°�¹(¸hµ�»±µ�Æ5¹3¯º¹"»R³^¿ É D*E�FHG�ILKNM*EPK�Q*RNSLT.³R¿5¯º°iµd¯~¼�²V¯±°�°h²�¾�°�¸�»º¿xÅiµdÅh²�¶@²@¯=4Ð��h�O� ²@Æȸ�¶�²�¹3³ þ�Oo É � � �� å ´ ç % .�ë � í æ å ô1s Í õ çÐ ��h�O� ²@Æȸ�¶�²�¹3³ þ�Oo É � � �� å ´ ç % ô ë � å æ _ å ô1s Í e ç+2®h¹NïfÇhµ�¶�²V¯±²�¹3³ å ô-s Í å ç µ�®hÀ å ô1s Í õ ç ²�Æ5¸h¶54�¯º°iµd¯o�O�/�¡  ± ³R¿ ¢ �O� ¢ ô �E¶@¿x¾ � %�� ß� _ ÃiÇh»R¯º°h¹"»RÆÈ¿�»º¹U0³º²@®h·"¹?´ Ö� �O�Z²@Æȸ�¶�²�¹3³SÐø��´ É ¿x»~³R¿xÆȹ�Ð ��¶�O� ± ²@®h¹<ï>Çiµd¶�²@¯=4 å ô1s Í e ç ²�Æ5¸h¶�²@¹"³

þ�A¤) �   �� å ´ ç % ô ë � å _ å ô1s Í ö çR~°h¹ É ¿x¶�¶@¿U¼�²@®h¾7¶@¹"Æ5Æ.µr²@³~ÀhÇh¹<¯±¿hÓêµ�®�³º¿xÇh» ÆU�VHWXWXY[Z]\A^`¦¨§w©{Y�ª�«;rO¬�q�­®Z]¯*¯*°&±²a KFW �BC KYX!³µ´·¶ Ê�I W/Ë�W=KFE A G �"È G I¹¸ K�XUW A�� GFWtº _ eOË�K ÂÈ-� EoXUZ/Z�ë»k�s þ� � � ¼ % Ò¾½ ÛÝÜ2Þ i %�¿ � j �� å ´ ç % ô ë � Ø3_:�®h¹<µ�¸�¸h»º¿|µd·'°ªµ�¯+¯±°�²�³z¸�¿x²�®|¯+²�³+¯º¿7Çh³º¹­Óªµ�®h³º¿�Çh»¬�T³�¶�¹"Æ5Æȵ}¯º¿Òµ�¸h¸h»R¿1d�²@Æ.µd¯º¹��êÅ>4�µd¸h¸h»º¿-d�²52Æȵd¯±²@®h¾^¯º°h¹zÃi¿xÇ�»º²�¹3»�·3¿�¹ �5·"²�¹3®|¯±³K¿ É µ�¶�¶>³ºÇ�Åh³º¹%¯±³�¿ É �O�+¼�°h²@·'°�µ�»R¹+³ºÆȵ�¶@¶�¹"»I¯º°iµ�®ÒØ s�º ¶�¿�¾ å Ø � ë ç ±¼�°h¹3»º¹ º ¹Ê¶�¿x¾ � %�� ß� ²@³<¯±°h¹5Æ.µ�d�²@Æ7ÇhÆà³R²V73¹È¿ É µ�®>4k¯±¹3»ºÆ ²�®»�>ºF_Eñ^¿U¼½¹N¼�¹"»N0&¯±°h²@³rµd¸h¸h»º¿xµ�·'°

ô1e|õ

Page 178: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

Àh¿f¹"³z®�¿�¯+¾�²6¼x¹�µZ¾x¿f¿�ÀÈ¿1¼x¹"»ºµ�¶�¶�»ºÇ�®h®h²�®�¾r¯±²@Æȹ^Å̹3·?µ�Ç�³º¹ º ²�³¯±¿f¿r¶�µ�»º¾x¹dÍ�+2®h³Ó¯±¹"µ�ÀÁ¼+¹<·3¿x®h³R²�Àh¹3»µ�®h¿d¯±°h¹3»½Â�]�Ã*¼�²V¯±°Ò³ºÆȵ�¶@¶�¹"»K¯º¹"»RÆȳ�¯±°hµ�®��bºK¼�°h²@·'°.µd¶�³º¿(·"¶@¿x³º¹3¶647µ�¸h¸h»R¿1d�²@Æ.µd¯º¹"³���_�¾�47Çh³º²@®h¾¯±°�²�³~³R¯º»º¿x®�¾x¹"»�Å�¿xÇh®hÀ�¿x®�¯±¹3»ºÆè³R²V7"¹}²@®ÝÓªµ�®h³º¿�Çh»¬�T³�¶@¹"Æ5Æ.µ(¼+¹(¾x¹3¯^µÒÅ�¹3¯º¯º¹"»~´i®iµd¶ »º¹3³ºÇh¶V¯?ÍÓª¿x»R¹t¸h»R¹"·"²@³º¹3¶64b0�¶�¹%¯p� % Å̹^¯±°h¹tÂ�]tÃ�¿xÅ�¯'µd²�®h¹3À É »º¿�Æ���Å>4.»R¹"Æ5¿�¼f²@®h¾r¹F¼x¹"»�4ȯº¹"»RÆ ¼�°�²�·'°·"¿�®>¯±µ�²�®�³�µ�¯�¶@¹?µ�³Ó¯^¶�¿x¾ � %��� ¼dµ�»º²�µ�Åh¶�¹3³"Í�èI¹3¯ � ¹ Ø s ¶�¿x¾ º % ¥ �� ¶@¿x¾ ¥ � _�Óêµ�®h³R¿xÇh»G�ð³^¶�¹3ÆÈÆȵ7²�Æ5¸h¶�²@¹"³¯±°hµd¯

þ� � � ¼ � �� % å ´ ç % ô ë � í _ å ô1s Í í çÓª¿x»R¹"¿1¼x¹3»<0i¼½¹Z°iµ-¼�¹rÕ�» � ­Ú� Ö¹×� % ®ô ë � Ñ|ØÒµ�®hÀ�°h¹3®h·"¹

å Õ�»� ­Ú� Ö¹�� % ® ¹ Ø�� ­ å ��ÀÙ� % ç % ® ¹ þ � å �� å ´ ç À �� % å ´ çRç % ô ë � í _ å ô1s Í , çèI¹3¯$���Òµ�®hÀ ü å ¸ ç Å̹µ�³EÀh¹3´h®h¹"ÀZ²�®!~HS]KNMNF��*�sF*�s����F�ShÍ�g�³R²�®h¾^²�®�¹<ï>Çiµ�¶@²@¯=4 å ô1s Í Ø ç É »º¿�Æ�P�¹"·%¯±²@¿x®ô1s Í Ø�Í Ø30�¼½¹Z°hµ1¼�¹Õ�»1­Ý³º²@¾x® å ü ç�Ö¹Q� ®ô Ø�� ­ å ü ÀÙ� ç % ® ¹ þ � å ����À �� å ´ çºç % ¹ î #[À#ÂÁ ±¼�°h¹3»º¹

î ¹ þ� � � à � Ñ �A¤) �   å ����À �� å ´ çºç % ± À ¹ þ� � � ¼ � å ���¡À �� å ´ çRç % ± Á ¹ þ� � � à � Ñ �O) �   å ����À �� å ´ çºç % _R�¿5Å�¿xÇh®hÀ î ± ¼½¹}¿xÅh³º¹3»�¼�¹Z¯±°hµd¯·����¹ s É ¿�»p´ Ö���O� ± ³R¿ÒÅL4 å ô1s Í ö ç ¼½¹(°hµ1¼�¹

î ¹ þ� � � à � Ñ �A¤) �N  �� å ´ ç % ô þ�A¤) �   �� å ´ ç % ô ë � å _R�¿ÈÅÌ¿�Çh®hÀ À ± ¼½¹Z®h¿d¯±¹<¯±°iµ�¯²����¹ s É ¿�» ¢ ´ ¢ k � µ�®hÀª°h¹"®�·"¹ À ¹ � � � � ¼ � �� å ´ ç % _#Pf²�®h·3¹

�� å ´ ç % ô Ø å �� å ´ ç À �� % å ´ çºç % # Ø �� % å ´ ç % 0h¼½¹(°hµ1¼�¹À ô Ø þ� � � ¼ � å �� å ´ ç À �� % å ´ çºç % # Ø þ� � � ¼ �Ä�� % å ´ ç %

ô Ø þ � å �� å ´ ç À �� % å ´ çºç % # ë � åô ë � Ø

ô1ebe

Page 179: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

Å>4�²�®�¹<ï>Çiµ�¶@²@¯±²@¹"³ å ô1s Íðí ç µ�®hÀ å ô-s Í , ç »º¹3³º¸�¹"·3¯º²6¼�¹"¶64�Í+-¯Á»R¹"Æȵ�²�®h³�¯º¿uÅ�¿xÇh®hÀ Á ¹ � � � � à � Ñ �*) �   å ���BÀ �� å ´ çºç % _LÖt³.²@® å èI²�®h²�µ�¶ KFWkXUZ5[ 0 ô1,b, Ñ ç¯±°�²�³^³ºÇhÆá·?µ�®êÅ̹ZÆ.µ�À�¹r¶�¹3³º³^¯±°iµ�® ë � å Å>4�¯'µ�ï>²@®h¾Å� ³RÇL�5·"²�¹3®|¯±¶64ª¶×µd»º¾x¹(³º¿.¯º°iµd¯^¼�²@¯º° °h²�¾�°¸h»R¿xÅiµ�Åh²@¶�²V¯=4Á¹?µd·'° ¹"³Ó¯±²�Æȵd¯º¹d���kÀh²5ÍG¹3»º³ É »º¿xÆ诺°h¹(¯º»ºÇh¹?¼dµ�¶�Ç�¹ �� å ´ ç Å>4ªµd¯^ÆÈ¿x³Ó¯ � ë � å ¢ �O� ¢ � _Öó³R¯º»±µ�²@¾x°|¯ É ¿x»Ó¼zµd»ºÀ*Çh³º¹5¿ É Ð+°h¹"»R®h¿UÍ0Å�¿xÇh®hÀ�³(³R°h¿8¼�³(¯º°iµd¯!�¿¹Ê¸�¿x¶64 å ¢ �O� ¢ � ±?ô � ëN± ¶�¿x¾ å ô � í çºç³ºÇH�È·3¹"³^¯±¿Ò¹"®h³RÇh»º¹}¯±°hµd¯ Á�ô ë � å _R~°>Çh³<0|¼½¹~°iµ-¼x¹ î #/À�#ÆÁ×ô ë _ ì ¹"·"µ�¶�¶@²�®h¾^¿xÇh»&ÅÌ¿xÇ�®hÀh³�¿x® ¢ �O� ¢ µ�®hÀ � ± ¼½¹~°iµ-¼x¹~¸�»º¿1¼x¹3À ÆÇÉÈ V�r*qsV]W Z�\A^¹ÊÌË m�KFE�WrË�KDJ Â�I È-� E A m I GFWnE InC J�W I �ÃÂ��û¯ s ±?ô ² ß ± È-� EDX  iÎÍÃXUZ J�KHGoë<±Hí�k s ±XUZhg � E I WrË AÏ~HS�KJMNF��*��FO�s����F�S ZVKfXUE  G �#Ô W=KFE A A��ÃÂ#� W �àK�³²´�¶ I/ W I/A K�Ì � Zji Â#�ÃAkI XUZ I/Â æ ±å �w¶�¿x¾ � ß� ç ÛÝÜ2Þ�ÐÑ�ÛÝÜ2Þ$ÒÑ X  m ¶�¿x¾ å ô � í ç _R�µ�ï>²�®h¾y��¹ËØ�Ù#i Ú ÛÝÜ2Þ�ß1j ¼+¹(¿xÅ�¯±µ�²�®Á¯º°h¹ É ¿x¶�¶@¿U¼�²@®h¾7·3¿x»º¿x¶@¶×µ�»�4 ÆÓ r*qsrOÔmÔmYLq�ÕÖZ�\A^`° ¶ � E�X  i ÇH�àGFWzX  W#ëoXUZhg � E I W/Ë A{~HS]KNMNF��*�sF*�s����F�S ZVKfXUE  G Ø�Ù#i Ú ÛÝÜ2Þ�ß-j¾× W"KFE AA��ÃÂ#� W �àKµ³µ´·¶ I/Â Ì � Zji å æ ± ¶�¿x¾ å ô � í çºç W I/A K�J  m�KFE?W/Ë�KcJ Â�I È-� E A m I GFWnE InC J3W I �à[Öt³È®h¿�¯±¹3À¨¹"µ�»º¶@²�¹"»N0}¾+³R°h¿xÇ�¯=4âµ�®�ÀóR&µdÆÈ¿x®Å�ð³Èµ�¶�¾�¿x»º²V¯±°hÆ É ¿x»5¶�¹?µd»º®h²@®h¾*ÆÈ¿x®�¿�¯±¿x®�¹êÂ�]�ü�²@¯º° a åRå ¶�¿�¾ æIç % � å ¶�¿�¾�¶�¿x¾ æIç � ç ¯±¹3»ºÆ5³Lµ�¶�³R¿<»º¹NïfÇ�²�»º¹3³K¯±°iµ�¯ ë Å�¹~·"¿�®h³R¯±µ�®|¯½²@®�¿x»ºÀ�¹"»K¯±¿(µ�·'°�²�¹N¼�¹¸�¿x¶64 å æ ç »ºÇh®|¯º²�Æ5¹�Íÿ�����=)�ÙØ ����� ��������& (���(��$(���� ä �"�!Ý��=� �èI¹3¯ u Å�¹.¯±°h¹.·"¶×µd³º³7¿ É À�¹"¸�¯º° º ± ³º²67"¹ÛÚ ·3²�»º·3Çh²@¯º³(¼�°�²�·'°0·"¿xÆ5¸hÇ�¯º¹.Æ5¿x®h¿�¯º¿x®h¹ É Çh®h·3¯º²�¿x®�³¿x®fÜ.¿�Ç�¯<¿ É æ ¼dµ�»º²�µ�Åh¶�¹3³"Í}Öt®�µ�®hµ�¶64f³º²@³�³º²@ÆȲ@¶×µ�»�¯±¿Á¯º°iµd¯<¿ É ¯º°h¹�¶×µ�³Ó¯<³º¹3·3¯º²�¿x® å Å�Ç�¯}³º²@Æȸh¶@¹"»³º²@®h·"¹~¼½¹�À�¿(®�¿�¯®h¹"¹3À5¯º¿(²@®|¯±»º¿fÀhÇh·3¹^µ�Ç3d�²�¶@²×µ�»�4 É Ç�®h·3¯º²�¿x®h³��bº�µ�®�À�� % ç ³º°h¿8¼�³L¯±°iµd¯µd¶�¾x¿x»R²@¯º°hÆ~HS�KJMNF��*��FO�s����F�SÏ·"µ�®ËÅ�¹*Çh³º¹3À˯º¿�¶@¹?µ�»R® u _p+ É � ²@³Á¯±°h¹k¯±µ�»º¾x¹%¯ É Çh®h·%¯±²�¿�® µ�®�À ü ²�³�¯±°�¹°>4f¸Ì¿�¯º°h¹"³R²�³5¾x¹"®�¹"»±µ�¯±¹"À¨ÅL4Ý~HS�KNMNF]�*��F*�s�*��F�S$0~¯±°h¹3®ãµ�³5²�®­¯±°�¹ª¶�µ�³R¯5³º¹3·3¯º²�¿x®0¼+¹ª°iµ-¼x¹�¯±°iµ�¯Õ�»1­Ý³º²@¾x® å ü ç�Ö¹Q� ®ô î #ÂÀ¢#ÞÁ ¼�²@¯º°

î ¹ þ� � � à � Ñ �A¤) �N  �� å ´ ç % ± À ¹ þ� � � ¼ � �� å ´ ç % ± Á ¹ þ� � � à � Ñ �O) �N  å ����À �� å ´ çºç %<_¾�4�Çh³º²@®h¾XD*E�FHGJI�KNM*EPKPQORNSLT¥¯º¿u²�À�¹"®|¯±² É 4­¯±°h¹àßR²�Æ5¸Ì¿�»R¯'µd®>¯wá*»º¹"¶@¹N¼dµ�®|¯�¼dµ�»R²×µ�Å�¶�¹"³��O� å ¿ ɼ�°h²@·'°¥¯±°h¹3»º¹�µ�»R¹.®h¿8¼�µd¯7Æ5¿x³R¯ÉÜ ç ¼+¹�·"µ�®­¹"®�³ºÇh»R¹Á¯±°iµd¯ � � � � ¤) �<  �� å ´ ç % ô ë � å µ�®hÀ¥¯º°>Çh³Å�¿xÇh®hÀ î _3RE¿rÅ�¿xÇh®�À À ± ²@®h³R¯º¹?µ�ÀÁ¿ É Çh³º²@®h¾ã⪵�®h³º¿�Çh»¬�T³z¶�¹3ÆÈÆȵ�¼+¹�¼+¹�Çh³º¹^¯±°h¹�Æ.µ�²@®È¶@¹"Æ5Æ.µ¿ É å èI²@®h²×µ�¶ KFW�XUZ5[ 0 ô1,b, Ñ ç ¼�°h²@·'°5Å�¿xÇh®�Àh³K¯±°h¹z¯º¿�¯'µd¶�¼½¹"²�¾�°>¯K¿ É °h²�¾x°�22¿x»RÀh¹"»�Ãi¿xÇ�»º²�¹3»&·3¿�¹ �È·3²�¹3®>¯º³É ¿x»~·3¿x®h³R¯±µ�®|¯�2-Àh¹"¸�¯º°ê·3²�»º·3Çh²@¯º³ Æ

ô1e|ö

Page 180: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

U�VHWXWXY[Z]\A^`äà§wU�åmª$åÙY�Ôµæ�çyè*éëê�±ìa KFW ��C K�Xyc �1� ZVKfX Â°È J Â#Ç W I �àÇH�ÃA Ì�J�W=Kfm C i¡X ÇFI E Ç J I W �"Èm�K=Ì�W/Ëyº¶X  m�G I¹¸ K Ú X  m�ZVKFW � C K�X  i I/ W=K�g�KFE<[íeOË�K Âþ� � � ¼ � �� å ´ ç % ô ØNÚÔØ>ý � Ò`îðï ¿ë% Ò _

Ã&²@®iµ�¶�¶54b0>µ�³+²�®^P�¹3·3¯º²�¿x® ô-s Í Ñ�Í ÑZ¯º°h¹�Å�¿xÇh®hÀ�¿�® Á ²�³½¿xÅ�¯'µd²�®h¹3ÀÁÅ>4.¯'µdïf²@®h¾7µ7³RÇL�5·"²@¹"®|¯±¶54.¶�µ�»º¾x¹³±µdÆȸh¶@¹Á¯±¿*¹"³Ó¯±²@Æ.µd¯º¹�¹?µ�·'°¨Ãi¿xÇh»º²@¹"»7·3¿�¹ �5·"²�¹3®|¯ �� å ´ ç ¼�²V¯±°ç´ñ� �O� ± ¢ ´ ¢ ô � ¼�²@¯±°0¹3»º»º¿�»µd¯5Æ5¿x³R¯ � ë � å ¢ �O� ¢ � _øR&µdïf²@®h¾ì� ¹á¸Ì¿x¶54 å Ü � ±?ô � ëN± ¶�¿x¾ å ô � í çRç µd®hÀ � ¹ ò åRå ¶�¿�¾ å Ú � ë çºç ½ ç ²@®~HS�KJMNF��*��FO�s����F�S�¼½¹Z¿�Å�¯'µ�²@® ÆÇÉÈ V�r*qsV]W Z�\A^ðó ¶ I�ô ºõ.�ôoX  m�ZVKFW u ½lÑ ö�Ñ ÷�C KSW/Ë�K Ç Z5XÃGHG �"È m�K=Ì�W/Ë!ºLø#G I¹¸ K Ú ÇFI E Ç J I WrG Ê Ë I Ç ËÇH�ÃA Ì�J�W"K A��ÃÂ#� W �àK È J Â#Ç W I �àG Ê�I WrË Ü ElKFZVK ÍÃX  W·ÍÃXUE I X C ZVKHG � J3W �"È æ _lË m�KFE�W/Ë�K�J Â�I È-� E Am I GFWnE InC J�W I �àø È-� E�X  i�ëN±Hí4k s ±SXUZhg � E I W/Ë Aù~HS�KJMNF��*��FO�s����F�S ZVKfXUE  G Ç Z5XÃGHGou ½lÑ öYÑ ÷�I/ W I/A KÌ � Zji Â#�ÃAkI XUZ I/Â æ ± Ü i ÛÝÜ2Þ i ö ¿ � j5j ï X  m ¶�¿x¾ å ô � í ç _:�®h¹}²�®|¯±¹3»º¹"³Ó¯±²@®h¾Ò·"¿x»R¿x¶�¶�µ�»�4Ȳ�³z¯º°h¹ É ¿x¶�¶@¿U¼�²@®h¾ ÆÓ r*qsrOÔmÔmYLq�ÕÖZ�\A^`ú ¶ I�ô ºÅ. ô�X  mkZVKFWu ½kC K�WrË�K Ç Z5XÃGHG �"È m�K=Ì�WrËãº�±#G I¹¸ K Ø Ù#i5i ÛÝÜ2Þdß1j Ò`î¾û ïëüNÒ`ý j ÇFI E ×Ç J I WrG Ê Ë I Ç Ë ÇH�ÃA Ì�J�W"K A��ÃÂ#� W �àK È J Â#Ç W I �àG Ê�I W/Ë Ø Ù#i5i ÛÝÜ2Þdß1j Ò`î¾ûþï�üNÒ`ý j ElKFZVK ÍÃX  WÿÍÃXUE I X C ZVKHG � J�W �"Èæ _ eOË�K ÂSÈ-� E�X  i ÇH�àGFWzX  W�ë)XUZhg � E I WrË AÞ~LS�KNMNF��O��F*�s���sF�S ZVKfXUE  G Ç Z5XÃGHG³u ½pI/Â Ì � Zji å æ ± ¶@¿x¾ å ô � í çRçW I/A K<[

ÎÏ°h²�¶@¹�¯º°h²�³z·3¶×µ�³R³ u ½�²�³z»ºµd¯±°h¹3»z¶�²@ÆȲV¯±¹3À É »R¿xÆò¯º°h¹<¸�¹"»º³R¸Ì¹3·3¯º²6¼x¹<¿ É ¾½¿�¿x¶@¹?µ�®.·"²�»R·"Çh²V¯z·3¿xÆ�2¸h¶@¹Fd�²@¯=4U0 É »R¿xÆáµÈ¶�¹"µ�»º®�²�®h¾Ò¯±°h¹3¿x»�4ª¸�¹"»º³R¸Ì¹3·3¯º²6¼x¹r²@¯�²�³ É µ�²�»R¶64�»R²�·'° Í�Îu¹Z®h¿�¯º¹Z¯±°hµd¯ u ½<³Ó¯±»º²@·3¯º¶64²�®�·"¶�Ç�Àh¹"³<¯±°�¹5·3¶×µ�³R³}¿ É À�¹"¸�¯º° º�± ³º²67"¹ÈØ Ù#i5i ÛÝÜ2Þdß1j Ò`î¾ûþï�üNÒ`ý j ·"²@»º·3Çh²@¯º³<¿x®¥Ø ÙOi5i ÛÝÜ2Þ�ß1j Ò`îëûþïëüNÒ`ý j ¼dµ�»R²×µ�Å�¶�¹"³¼�°h²@·'°È·3¿x®|¯'µ�²@®5¿�®h¶64�Ç�®fÅ�¿xÇh®�Àh¹"À É µ�®h²�®�ÖY]tÂ˵�®hÀ�: ì ¾|µd¯±¹3³"ÍæR~°�²�³ É ¿x¶�¶@¿8¼�³ É »º¿xÆ�»º¹3³ºÇh¶V¯±³L¿ É:�ï�¿x¶Ä�ð®�²�³º°�®h²�ïd¿�¼dµ å :�ï�¿�¶¶�T®h²�³R°h®h²�ïd¿1¼�µ30 ô1, í|Ø ç µ�®hÀZÖø�Ó¯'µd²�µd®hÀã�<Çh»º¹F¼f²�·'° å Öø�Ó¯±µ�²>µ�®hÀl�}Çh»R¹N¼f²�·'°#0ô1, í ö ç å ³R¹"¹Áµd¶�³º¿ å ¾+¿x¸h¸iµd®iµªµd®hÀ P�²@¸h³º¹3»<0 ô1,b,bs ç P�¹"·%¯±²�¿�®uÑ�Í e ç ¼�°h²@·'°u³º°�¿U¼ó¯º°iµd¯}¯±°�¹"»º¹Èµ�»R¹Æ5¿x®h¿�¯º¿x®h¹ É Ç�®h·3¯º²�¿x®h³½¼�°h²�·'°�·?µd®�Å̹�·3¿xÆ5¸hÇ�¯±¹3À�ÅL4�´ u Ò ·"²@»º·"Ç�²@¯±³+ÅhÇ�¯�µ�»R¹�®h¿�¯z·"¿xÆ5¸hÇ�¯±µ�Åh¶�¹Å>4¡´ u Ò ·3²�»R·"Çh²V¯±³+¼�°h²@·'°ê°iµ-¼�¹(®�¿È®h¹3¾|µd¯º²�¿x®h³3Í÷<ø�¤�� ¦pû�ü�ý<5��~ú 9 þ-¬�úKûþ;:<5êú�þ"üDªh¬Ö Ì�E � mUJ Ç W�m I GFWnE InC J3W I �à¿1¼x¹3»�¯ s ±"ô ² ß ²�³(·'°iµ�»ºµ�·3¯º¹"»º²67"¹3À¥Å>4�¸hµ�»±µ�Æ5¹3¯º¹"»R³��³º ± _-_-_ ± � ß ¼�°h¹"»R¹�#É#¹ËÕ�»-­Ó¸�ÉO¹ ô ® _ P�Ç�·'°êµ7Àh²@³R¯º»º²�Å�Ç�¯±²@¿x® Ä µ�³º³R²�¾x®�³}¼�µd¶�Çh¹3³~²�®hÀh¹3¸Ì¹3®hÀh¹3®>¯º¶64Á¯º¿�¹?µ�·'°^¼dµ�»º²�µ�Åh¶@¹U0

ô1e í

Page 181: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

³º¿ É ¿x»����B¯ s ±?ô ² ß ¼+¹(°iµ-¼x¹ Ä å � ç ¹ ��� � �`º �#É � ����� �$Ò å ô À��#É ç � _�R~°h¹(Çh®h² É ¿x»ºÆòÀh²�³Ó¯±»R²�ÅhÇ32¯±²@¿x®5²�³µZ¸h»R¿�À�Çh·3¯½Àh²�³Ó¯±»R²�ÅhÇ�¯º²�¿x®�¼�²V¯±°.¹"µ�·'°��#É`¹ ô � Ø3_>R~°h¹�³R¯±µ�®hÀiµ�»RÀ.Àh¹F¼f²×µd¯º²�¿x®5¿ É ¸�ÉÌÇh®hÀh¹3»µ*¸h»º¿fÀhÇh·%¯.Àh²@³R¯º»º²�Å�Ç�¯±²@¿x®­²�³��$Éc¹ � �#É å ô À��#É ç _LÖ ¸h»º¿fÀhÇh·%¯5Àh²�³Ó¯±»º²@ÅhÇ�¯º²�¿x® Ä ²@³\ÇH�àGFWzX  W ×CF� J  m�Kfm ² É ¯º°h¹"»R¹}²�³~³º¿�Æȹ<·3¿x®h³R¯±µ�®|¯ � � å s ±"ô ç ²�®�Àh¹"¸�¹"®hÀ�¹"®|¯t¿ É æ ³RÇh·'°ª¯º°iµd¯��#É�Ù­ � ±"ô À � ®É ¿x»µ�¶@¶�Ð�¹ ôb± _-_<_ ± æ _>Î�¹^¶�¹3¯�� À�¹"®h¿�¯º¹tÆȵÃd�É��`º=Ñ������ Ñ ß å ô � �#É ±?ô � å ô À��#É çRç _LR~°h»R¿xÇh¾x°h¿�Ç�¯½¯º°h¹^»º¹"³Ó¯¿ É ¯±°h²@³~¸iµ�¸�¹"» Ä Àh¹"®h¿d¯±¹"³�µ�¸h»º¿fÀhÇh·%¯^Àh²�³Ó¯±»º²@ÅhÇ�¯º²�¿x® Í�}²5¼x¹3®âµk¸h»R¿�À�Çh·3¯ÒÀh²@³R¯º»º²�Å�Ç�¯±²@¿x® Ä ¼+¹�Àh¹%´i®h¹�µk®h¹3¼�²�®h®�¹"»7¸h»R¿�À�Çh·3¯5¿1¼x¹3»Ò¯º°h¹�¼x¹3·3¯º¿x»³º¸hµ�·"¹(¿ É »º¹?µd¶#¼dµ�¶�Çh¹3À É Ç�®h·3¯º²�¿x®h³~¿�®�¯ s ±"ô ² ß µ�³ð � ±fü ñ�� ¹ þ

ÿ���� ÒfÑÓº���� Ä å ¸ ç � å ¸ ç ü å ¸ ç ¹ Ø � ­Ú� ü ®µ�®hÀ(µ~·"¿�»º»º¹3³º¸�¿x®hÀh²@®h¾�®h¿x»RÆ N���¹¡� ð � ± � ñ�� _"Îu¹�»R¹ É ¹3» ¯±¿~¯±°h²@³I®h¿x»RÆÔµ�³ W/Ë�K Ä × Â#� E A�Í�Ãi¿x»Ð³¹ ôb± _-_-_ ± æ ¶@¹3¯��<É#¹ å ¸�É8À��`É ç � �$Éz_]�<²6¼x¹3®�´{�ì­ æ`® ± ¶�¹%¯��O��Å�¹^À�¹3´i®h¹3ÀÁµ�³��*� å ¸ ç ¹ � É � � �<É=_Öt³}®h¿d¯±¹"À�Å>4¶¾zµ�°hµ�ÀhÇh» å ¾+µ�°iµ�ÀhÇ�»<0 ô1,be�ô ç µ�®hÀ�ÃiÇh»º³Ó¯ KFW�XUZ5[ å ÃiÇh»R³R¯ KFW�XUZ5[ 0 ô-,b,�ô ç 0G¯±°h¹ÈØ ßÉ Çh®h·%¯±²@¿x®h³��O� É ¿�»ºÆëµ�®�¿x»Ó¯±°h¿x®�¿x»ºÆȵ�¶GÅhµ�³º²@³ É ¿�»z¯±°h¹�¼x¹3·3¯º¿x»�³R¸iµ�·3¹Z¿ É »º¹"µ�¶O¼dµ�¶�Çh¹3À É Çh®h·3¯º²�¿x®�³¿x®�¯ s ±"ô ² ß ¼�²V¯±°�»º¹3³º¸�¹"·%¯¯º¿�¯±°h¹ Ä 2-®h¿x»RÆ�0d²�ÍT¹�Ípð��O� ± � � ñ�� ²�³ ô ² É ´Q¹ µ µ�®hÀÒ²@³ s ¿�¯±°�¹"»R¼�²@³º¹dÍÎ�¹(»R¹ É ¹3»�¯º¿�¯±°h²@³~Åiµ�³º²@³^µ�³ WrË�K �ÙC XÃG I G Í�R~°h¹ É ¿x¶�¶@¿8¼�²�®h¾ É µd·3¯^²�³~Çh³R¹ É Çh¶ Æ

!AY#"%$ÎZ�\�^`¯à§ &lY È Y('$¬�q*)+!�¬�qs«,$fæ�ç(êÎè*é ± eOË�K � C XÃG I G I G�WrË�K C XÃG I G Ê Ë I Ç Ë ÊS� J3Z5m C K �UC ×W X I/ Kfm C i.-}E�X A ×0/ Ç Ë AkI mUW � EqW/Ë �ÃÂ#� E A XUZ I¹¸ XUW I �ÃÂ21zÊ�I WrË»ElKHG=Ì�K Ç W�W � W/Ë�K Ä × Â#� E A43¶�"È W/Ë�K �C XÃG I G}Ì�KFE È-� E A Kfm I/ÂB� E�m�KFE �"È?I/Â#Ç ElKfXÃG I/ g ¢ ´ ¢ _¾�4k¯º°h¹È¿�»R¯±°�¿x®h¿x»RÆ.µ�¶@²@¯=4ª¿ É ¯º°h¹5�­Åhµ�³º²@³<0�µ�®>4*»º¹?µd¶ É Ç�®h·3¯º²�¿x®�¿x®ç¯ s ±?ô ² ß ·"µ�®uÅ�¹5Çh®h²Vï>Çh¹3¶64¹Fd�¸h»R¹"³R³º¹"À�µ�³?� å ¸ ç ¹ � �76� å ´ ç �*� å ¸ ç ¼�°h¹3»º¹ 6� å ´ ç ¹�ðz� ± �O� ñ�� ²�³^¯±°h¹7Ã�¿�Çh»º²@¹"»^·"¿f¹-�5·"²@¹"®|¯¿ É ´Ê¼�²@¯±°�»º¹3³º¸�¹"·3¯r¯±¿ê¯º°h¹��0Åiµ�³º²@³"Í\]�¿�¯º¹È¯±°hµd¯Z¼½¹.¼�»R²@¯±¹ 6� å ´ ç É ¿x»}¯±°�¹��0Åhµ�³º²@³7Ãi¿xÇh»R²�¹"»·"¿f¹-�5·3²�¹"®|¯7µ�®hÀ �� å ´ ç É ¿x»<¯±°h¹y�ÏÅiµ�³º²@³rÃi¿xÇ�»º²�¹3»}·"¿f¹-�5·"²@¹"®|¯?Í�Öt¶@³º¿êÅ>4�¿x»Ó¯±°h¿x®�¿x»ºÆȵ�¶�²V¯=4ê¼½¹°iµ-¼x¹ZÕ&µd»º³º¹F¼dµ�¶¶�T³�²�Àh¹3®>¯º²@¯=4

Ø � ­Ú�&% ® ¹¨N��(%�¹ þ�O)*+ ß-, 6� å ´ ç %¼�°h²@·'°ª²@³ ô É ¿x»p¾+¿f¿x¶�¹"µ�®���_hÃ&²�®iµd¶�¶64U0 É ¿�»p¾+¿f¿x¶�¹"µ�®��kµ�®hÀ�»º¹"µ�¶52 ¼�µd¶�Çh¹3À ü ¼½¹(°iµ-¼x¹ åºå Ã�Ç�»º³R¯ KFWXUZ5[ 0 ô1,U,�ô ç èI¹"Æ5Æ.µ ô-s ç Õ�»

�­Ú� Ö¹ ³º²@¾x® å ü ç=®·ô Ø � ­ å ��À ü ç % ® _ å ô1s Í ô1s çô1eb,

Page 182: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

ÃiÇh»º³Ó¯ KFWcXUZ5[ å Ã�Ç�»º³R¯ KFWcXUZ5[ 0 ô1,b,�ô ç µ�®iµ�¶54�7"¹"À¥¯º°h¹��âÅhµ�³º²@³�Ãi¿xÇh»º²@¹"»Z³º¸�¹"·3¯º»ºÇhÆ ¿ É ´ u ÒÉ Çh®h·%¯±²@¿x®h³Lµ�®hÀ5¾|µ-¼x¹�¸h»R¿�À�Çh·3¯Àh²@³R¯±»R²�ÅhÇ�¯±²�¿�®�µ�®iµ�¶@¿x¾xÇh¹3³�¿ É èI²�®h²�µ�¶ KFW�XUZ5[ �ð³L»R¹"³ºÇ�¶@¯±³�¿�®È¶@¹?µ�»R®h²�®h¾´ u Ò ·"²@»º·"Ç�²@¯±³�Çh®hÀh¹3»K¯±°h¹~Ç�®h² É ¿x»RÆ Àh²�³Ó¯±»º²@ÅhÇ�¯º²�¿x® Í+2®�P�¹3·3¯±²@¿x® ô1s Í å Í ô ¼½¹~³º°iµ�»R¸Ì¹3®5µ�®hÀ�¹Fdf¯º¹"®hÀ³º¿�Æȹ�»º¹"³RÇh¶@¯º³ É »R¿xÆ å ÃiÇh»º³Ó¯ KFWøXUZ5[ 0 ô1,b,�ô ç 0�µ�®hÀÁ²@®¡P�¹3·3¯º²�¿x® ô-s Í õ ¼+¹tÇh³º¹t¯º°h¹"³R¹�³º°hµ�»º¸�¹"®h¹3À�»º¹q2³ºÇ�¶@¯±³K¯º¿x¾x¹%¯±°h¹3»K¼�²@¯±°7¯º¹"·'°h®�²Vï>Çh¹"³ É »º¿�Æ å ÃiÇh»R³R¯ KFW�XUZ5[ 0 ô1,b,�ô ç ¯±¿<¿�Å�¯'µ�²@®�¸h»º¿fÀhÇh·%¯LÀh²�³Ó¯±»R²�ÅhÇ�¯º²�¿x®µ�®iµd¶�¿x¾xÇ�¹"³~¿ É ¿xÇh»�µ�¶@¾x¿x»R²@¯±°�Æȳ É »º¿xÆ Pf¹"·3¯º²�¿x®�Ñ�Íÿ����ÙØY�Óÿ ßt(�ú �98;:ç��� ������(SÝ� �"�����³ú ú ���Ö E�X  m �ÃA ElKHGFW E I Ç W I �ÃÂ=< w Ñ � ²�³Áµ�Æ.µ�¸�¸h²�®h¾ É »º¿xÆ ¯1¸Oº ± _<_-_ ± ¸ ß ² ¯±¿Ù¯ s ±?ôb±?> ² ¼�°h¹"»R¹�¸�É�²@³Æȵ�¸h¸�¹"À�¯±¿ > ¼�²@¯±°�¸h»R¿xÅiµ�Å�²�¶�²V¯=4A@ ± ¯±¿ ô ¼�²@¯º°�¸h»º¿xÅhµ�Åh²�¶@²@¯=4 å ô ÀB@ ç �#É ± µ�®hÀ�¯±¿ s ¼�²@¯º°�¸h»º¿xÅ�2µ�Åh²@¶�²V¯=4 å ô ÀC@ ç å ô ÀD�#É ç _�+ É ��²�³Zµ\¾½¿�¿x¶@¹?µ�® É Çh®h·%¯±²@¿x®�¯º°h¹"®ç��EF< »º¹3¸h»º¹3³º¹3®>¯º³7¯±°h¹ É Çh®h·%¯±²@¿x®� å < w Ñ � å ¸ çºç ¼�°h¿x³R¹�¼dµ�»R²×µ�Åh¶@¹"³}µ�»R¹�¯±°h¿x³R¹D¸�ɼ�°h²@·'°uµ�»R¹ÈÆȵ�¸h¸�¹"Àk¯º¿ > µ�®hÀ�¼�°�¿x³º¹5¿�¯±°�¹"»�¸�ɵ�»R¹(²@®h³R¯±µ�®|¯±²�µd¯±¹3À�µ�³ s ¿�» ô µ�·3·"¿x»RÀh²�®�¾Ò¯º¿�< w Ñ � _R~°h¹ É ¿x¶@¶�¿8¼�²�®�¾^²@³Kµ)¼�µd»º²×µd®>¯�¿ É ñ�µ�³R¯±µ�À{�T³�¼½¹"¶�¶�ï>®h¿8¼�®�³R¼�²@¯º·'°h²�®�¾�¶�¹3ÆÈÆȵ å ñtµd³R¯'µdÀO0 ô1, í e ç ÆU�VHWXWXY[Z]\A^mZ�\¡a KFW Ä C K�XoÌ�E � mUJ Ç WSm I GFW E InC J�W I �öÊ�I W/ËDÌ XUE�X A KFW=KFElG �#É X  m � XÃG�m�KHG  KfmX CF� ÍUK1ø�ZVKFW ��C K°X�I$´�¶ È-� E A J3Z5X Ê Ë�KFElK�KfX Ç Ë Ç Z5XUJLG1KcË XÃG°XUW A�� GFW®º¶Z I W=KFEfXUZ G1ø}X  m�ZVKFW < w Ñ �C K�X�E�X  m �ÃA ElKHGFWnE I Ç W I �à[yeOË�K Â�Ê�I W/ËkÌ�E �UC X C<I Z I W i�XUW�ZVKfXÃGFW½ô À å å �#@ º ç J ±K>[oW/Ë�K È J Â#Ç W I �û��EF< Ç X Â�C K�K ô Ì�EHKHGHG1Kfm¶XÃG�X ³µ´�¶ È-� E A J�Z5X Ê Ë�KFElK^KfX Ç ËBW"KFE A Ë XÃG^XUWA�� GFWMLcZ I W"KFE�XUZ GONP�[oW/Ë�KcW"KFE A G �"È GFJ Ç Ë\Xí³µ´·¶QXUZ/ZX ÇHÇ K=Ì�WSm I GHQ �ÃI/ WæG1KFWrG �"È�I/ Ì�J3W/G<[

pnqsrtrOu�«SR*V*$T" È v}R~°h¹<¸h»R¿�¿ É ²�³~µ7Æ5²�®h¿�»½Æ5¿�À�²@´i·"µd¯±²@¿x®È¿ É µd»º¾xÇhÆ5¹"®|¯º³z¾x²6¼�¹"®�²@®�P�¹"·%¯±²�¿�® å ¿ Éå ¾½¹?µ�Æ5¹U0 ô-,b,Uå ç ÍR~°h¹ É ¿x¶@¶�¿8¼�²�®h¾(·"¿�»º¿x¶@¶×µ�»�4Ò²�³~µr¸�»º¿fÀhÇh·3¯�À�²�³R¯º»º²@ÅhÇ�¯±²@¿x®Èµ�®iµ�¶@¿x¾xÇh¹�¿ É åºå èI²@®h²×µ�¶ KFW�XUZ5[ 0 ô1,b, Ñ çÐ+¿x»R¿x¶�¶�µ�»�4 ô ç ÆÓ r*qsrOÔmÔmYLq�ÕÖZ�\A^mZ*Za KFW Ä C KoX�Ì�E � mUJ Ç WSm I GFWnE InC J3W I �ÃÂ�Ê�I W/Ë°Ì XUE�X A KFW"KFElG �#É X  m � ±ZVKFW �ÙC KX.I$´�¶ È-� E A J�Z5X Ê Ë�KFElK^KfX Ç Ë Ç Z5XUJLG1KDË XÃG�XUW A�� GFW"ºçZ I W"KFE�XUZ G1øcX  mhZVKFW < w Ñ � C K¡XhE�X  m �ÃAElKHGFW E I Ç W I �à[õeOË�K ¡Ê�I W/ËoÌ�E �UC X C<I Z I WmikXUWæZVKfXÃGFWKô À å å �#@ º ç J Ê KtË X�ÍUKYW/Ë XUW4U�VEW< å ´ ç ¹ s È-� EYXUZ/Z¢ ´ ¢ kDL _pnqsrtrOuwvh+-¯Ò²�³Ò³º°h¿8¼�®â²�® å èI²�®�²×µ�¶ KFW°XUZ5[ 0 ô1,b, Ñ ç ¯±°hµd¯5² É �VEF<�³±µd¯º²�³R´h¹"³5¸h»º¿x¸�¹"»Ó¯±²@¹"³ å ô ç µd®hÀå Ø ç ¿ É èI¹"Æ5Æȵ ô-s Í ô-s ¯º°h¹"®YX�VEF< å ´ ç ¹ s É ¿x»Òµ�¶�¶ ¢ ´ ¢ kZL _½ñ^¹"®h·3¹ª³RÇh·'°�µ��VEF<u²�³�²�®0¯±°�¹

ôUöUs

Page 183: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

³º¸hµ�·"¹�³º¸iµd®h®h¹"ÀâÅ>4�¯���� Æ ¢ ´ ¢ ô L ² _ø¾�4�Ã�µ�·%¯ ô1s Í , µ�®�À�¯º°h¹�®iµd¯±Ç�»º¹�¿ É �<»±µdÆ�2"P�·'°hÆ5²�À�¯¿x»Ó¯±°h¿x®�¿x»ºÆȵ�¶�²67?µd¯º²�¿x®#0h¯±°h²@³�²�³^¯±°h¹�³ºµ�Æȹr³º¸iµ�·3¹7¼�°h²�·'°k²@³�³º¸hµ�®h®h¹3À�Å>4B¯[�O� Æ ¢ ´ ¢ ô L ² ± µd®hÀ¯±°�¹(·3¿x»º¿�¶�¶×µd»�4 É ¿x¶�¶@¿U¼�³3ÍÐ+¿x»º¿�¶�¶×µd»�4 ô1s Í ôxô ²�³rµª³º°hµ�»º¸�¹"®h¹3À ¼x¹"»R³º²@¿x®¥¿ É µª³º²�Æ5²�¶�µ�»�¶@¹"Æ5Æ.µ 0E²�Æ5¸h¶@²�·"²V¯<²�® å ÃiÇh»R³R¯ KFWXUZ5[ 0 ô-,b,�ô ç 0^¼�°h²@·'°Ë³Ó¯'µd¯º¹"³�¯±°hµd¯�Çh®hÀh¹3»�¯±°h¹*³±µdÆȹk·3¿x®hÀh²V¯±²�¿�®h³.¼�²@¯º°Ï¸h»º¿xÅhµ�Åh²�¶@²@¯=4âµd¯�¶@¹?µ�³Ó¯ô À å õ �\@ º � Ø ç J ¼+¹�°hµ1¼�¹ U�VEF< å ´ ç ¹ s É ¿�»zµ�¶@¶ ¢ ´ ¢ k]L % _�Ö�»ºÆ5¹"À.¼�²V¯±°.¯º°h¹t³R°iµ�»º¸�¹"»~Ð+¿x»º¿x¶@¶×µ�»�4ô1s Í ôxô 0�ÅL4Á»º¹3¯º»±µ�·3²�®h¾5µ�»R¾xÇhÆ5¹"®|¯±³ É »R¿xÆ å Ã�Ç�»º³R¯ KFWSXUZ5[ 0 ô1,U,�ô ç ²V¯�²�³~³R¯º»±µ�²@¾x°|¯ É ¿x»Ó¼zµd»ºÀ�¯±¿Ò¸h»R¿�¼�¹U�VHWXWXY[Z]\A^mZ]_ ¶ � EcX  iãc �1� ZVKfX ÂcÈ J Â#Ç W I �ÃÂ�� ± È-� EoX  i I/ W=K�g�KFE_^8±

þ� � � ¼a` 6� å ´ ç % ô ØÁÕL»b0c d e ­ U�VEF< å ´ ç�Ö¹ s È-� E�G �ÃA K ¢ ´ ¢ kf^ @ � Ø ® _¾+¿f¿x¶�¹"µ�®*ÀhÇiµ�¶@²@¯=4 ²@Æȸ�¶�²�¹3³t¯±°hµd¯}¯±°h¹Ò·"¿�®h·"¶@Çh³º²@¿x®�¿ É Ð+¿x»R¿x¶�¶�µ�»�4 ô1s Í ôxô µ�¶�³R¿�°h¿x¶�À�³(² É �¥²@³µÁÂ�]�Ãâ¼�²@¯º°k¹?µ�·'° ¯º¹"»ºÆm¿ É ¶�¹3®h¾�¯º°kµd¯<Æ5¿x³R¯ º _`R�µ�ï>²�®h¾g@�¹ ô � í%� º µ�®hÀ L ¹�¶�¿x¾ih � ²�®ª¯±°h²@³Â�]tÃó¼�¹"»º³R²�¿x®�¿ É Ð+¿x»º¿x¶@¶×µ�»�4 ô-s Í ô�ô µ�®�À ^ ¹ ô1e � º ¶�¿x¾�h � ²�®�èI¹"Æ5Æȵ ô-s Í ô Ø30I¼+¹5¿xÅ�¯±µ�²�®k¯±°�¹É ¿x¶@¶�¿8¼�²�®h¾7µ�®iµ�¶@¿x¾xÇh¹�¿ É âêµd®h³º¿xÇ�»¬�T³�¶@¹"Æ5Æ.µ å èw¹3ÆÈÆȵ ô1s ÍðÑ ç É ¿x»z¯º°h¹��êÅiµ�³R²�³ ÆU�VHWXWXY[Z]\A^mZ]¦ba KFW �óC K�Xã³µ´�¶ Ê�I WrË^W=KFE A G �"È G I¹¸ KoXUW A�� GFW�º _ eOË�K Â�È-� EoXUZ/Z ë·k�s

þ� � � ¼ ºkj0l�½ ÛÝÜ2Þ i h ¿ � j 6� å ´ ç % ô ë � Ø3_Öt¾|µd²�®.Ç�³º²�®�¾7µ�»R¾xÇhÆ5¹"®|¯±³ É »º¿xÆ å Ã�Ç�»º³R¯ KFWøXUZ5[ 0 ô1,b,�ô ç 0�Ð+¿x»º¿x¶@¶×µ�»�4 ô1s Í ôxô ·?µ�®.µ�¶�³R¿rÅ�¹tÇh³R¹"À¯±¿Ò¸h»R¿1¼x¹}¯±°h¹ É ¿x¶@¶�¿8¼�²�®h¾o¼x¹"»R³º²@¿x®ª¿ É ¯º°h¹}Æ.µ�²@®�¶�¹3ÆÈÆȵ É »º¿�Æ å ÃiÇh»R³R¯ KFWSXUZ5[ 0 ô1,b,�ô ç ÆU�VHWXWXY[Z]\A^mZJÊ�a KFW �½C KYXnc �1� ZVKfX Â�È J Â#Ç W I �ÃÂ\ÇH�ÃA Ì�J�W"Kfm C i°X ÇFI E Ç J I W �"È m�K=Ì�W/Ëdº^X  mcG I¹¸ KÚ X  m�ZVKFW � C KcX  i I/ W=K�g�KFEN[yeOË�K Â

þ� � � ¼ � 6� å ´ ç % ô ØNÚÔØ ý � Ò`îðï ¿ ¥ l _R~°h¹t¼x¹"»R³º²@¿x®Á¿ É ¯º°h²�³½¶�¹3ÆÈÆȵ<¾x²6¼�¹"®.²@® å Ã�Ç�»º³R¯ KFWøXUZ5[ 0 ô1,b,�ô ç °iµd³ ô � å º # Ø ç ²�®�³R¯±¹"µ�À.¿ É ô � º ²@®¯±°�¹<¹Fd�¸Ì¿�®h¹"®|¯�¿ Éæ� _$R~°h²@³~®h¹3¼Ë¯±²@¾x°|¯±¹"»+ÅÌ¿xÇ�®hÀ�¼�²@¶�¶�¹"®iµdÅh¶�¹<Çh³~¯º¿5¾�²6¼x¹<³Ó¯±»R¿x®h¾x¹3»~¾xÇiµ�»ºµ�®|¯±¹"¹3³¿x®k¿xÇ�»�¶�¹"µ�»º®h²@®h¾�µ�¶@¾x¿x»R²@¯±°�Æ*�T³�¸�¹"» É ¿x»ºÆȵ�®h·3¹7Çh®hÀ�¹"»<¸h»R¿�ÀhÇ�·3¯}Àh²@³R¯±»R²�ÅhÇ�¯±²�¿�®h³�¯º°iµ�® ¼½¹�·"¿xÇh¶@À°iµ-¼x¹(¿xÅ�¯±µ�²�®�¹"À�Å>4�³º²@Æȸh¶54ÁÇh³R²�®h¾�¯º°h¹}¶�¹"Æ5Æȵ É »º¿xÆ å ÃiÇh»º³Ó¯ KFW}XUZ5[ 0 ô1,b,�ô ç Í

ôUö�ô

Page 184: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

÷<ø�¤nm 2½§�«rû�ªkþ<ª43 5hªký�§<û ¦pû�ü�ý<5��~ú79 þ-¬�ú�ûLþ�:<5êúKþ"üDªh¬ÿ����0op�Óÿ >>%����S�$��ù1?������é����*��A@��ø�S�CB��� �"�øÜ�*���Î�¹(°hµ1¼�¹(¯º°h¹ É ¿x¶�¶@¿8¼�²�®h¾�µd®iµ�¶�¿�¾xÇh¹t¯º¿�èI¹3ÆÈÆȵ ô1s Í Ø É ¿�»~¸h»º¿fÀhÇh·%¯�Àh²�³Ó¯±»º²@ÅhÇ�¯º²�¿x®h³ ÆU�VHWXWXY[Z]\A^mZ]°ba KFW � Æ ¯ s ±?ô ² ßcÁ ¯LÀ ôb±?ô ² C KoX A��ÃÂ#� W �àKµc �1� ZVKfX ÂcÈ J Â#Ç W I �à[õeOË�KFElK I GX  XUZhg � E I WrË A Ê Ë I Ç Ë�Ë XÃG�X ÇHÇ KHGHGYW � Øcî å � ± Ä ç ±#EqJ  G I/Â Ì � Zji å æ ± � ±?ô � ë<± ¶@¿x¾ ô � í ç W I/A KtGFW=K=Ì�GÈ-� EoXUZ/Z�ë<±Hílk�s ±�X  m Ê�I WrËkÌ�E �UC X C<I Z I Wmi�XUW�ZVKfXÃGFWLô À í � J�W�Ì�J�W/G�XDG1KFW �O�n�÷­ æ�® GFJ Ç Ë^WrË XUWÐø�¶�O��I/A Ì�Z I KHG þ�Oo É � � 6� å ´ ç % . ë � Ø X  m Ð ��h�O��I/A Ì�Z I KHG þ�*o É � � 6� å ´ ç % ô ë _pnqsrtrOuwv+Îu¹}Çh³R¹(¯º°h¹ É ¿x¶�¶@¿8¼�²�®h¾ É µd·3¯ Æ!AY#"%$ÎZ�\�^mZ]äà§�§ &ã« È rO¬M$(ÕÂY�ªp' Ç Y�WXrOª�­ ZH¯*¯*ä&±ÉU�V]W�WXYìÊ�^mZ�± ¶ � E�X  iÎc �1� ZVKfX Â^È J Â#Ç ×W I �ÃÂ�� X  m¡X  i�Ì�E � mUJ Ç WSm I GFW E InC J�W I �ÃÂ Ä ±

å ��%É Ô�Ñ É å � ç ¹ þ�Oo É � � 6� å ´ ç %1_g�³R²�®h¾<Ð+°h¹"»R®h¿UÍ�Å�¿xÇh®hÀ�³K¿x®h¹z·"µ�®�¹?µd³º²�¶54(³º°h¿8¼�¯±°iµd¯&¼�²@¯±°r¸Ì¿x¶54 å æ ± � ±"ô � ë<± ¶@¿x¾ ô � í ç ¹Fd�µ�Æ5¸h¶�¹3³²@¯&²�³�¸Ì¿x³R³º²@Åh¶�¹¯±¿�¹3³R¯±²@Æ.µ�¯±¹+¹"µ�·'°k¼dµ�¶�Çh¹��#É å µ�®hÀ7¯º°fÇ�³+�$É ç ¼�²V¯±°7°h²@¾x°�µ�·3·"Çh»ºµ�·N4�ÍKÖt³&²�®kèI¹"Æ5Æȵô1s Í Ø30xÇ�³º²�®�¾(Ð+°h¹"»R®h¿UÍÁÅ�¿xÇh®�Àh³�²@¯&²�³K¸�¿x³º³R²�Åh¶@¹½¯º¿<¹"³R¯º²�Æȵd¯±¹�Ô

�Ñ É å � ç ¼�²V¯±°�°h²@¾x°�µ�·3·"Çh»ºµ�·N47µ�¾|µ�²@®Çh³R²�®h¾5¸Ì¿�¶64 å æ ± � ±?ô � ë<± ¶�¿x¾ ô � í ç ¹qd�µ�Æȸ�¶�¹"³3Í~Ð+¿xÆrÅh²�®h²@®h¾�¯±°h¹3³º¹(¹"³Ó¯±²�Æȵd¯º¹"³ É ¿x»��$ÉKµd®hÀ�Ô

�Ñ É å � ç¸h»R¿�¼�¹"³�¯±°h¹}¶@¹"Æ5Æ.µ�Í�Îu¹}·?µd¶�¶Ì¯º°h²�³^µ�¶@¾x¿x»R²@¯±°�Æ D*E�FHGJI�KNM*EPKPQORNSLT%qhÍ

ÿ����0op�zÛ õ»ö��÷����� ��������ã *���(� �=�$ö�úÎ�¹Z¼½¿xÇh¶�Àê¶�²�ïd¹(¯º¿.Æ5¿�À�² É 4Û~HS�KJMNF��*��FO�s����F�Sª³º¿.¯º°iµd¯�²@¯�Çh³º¹3³t¯±°�¹��*Åhµ�³º²@³t»±µ�¯±°h¹3»�¯±°hµ�®ê¯±°�¹�ãÅiµd³º²�³3ÍÒñ�¿8¼½¹N¼x¹3»<0&µ�³}²�® å ÃiÇh»R³R¯ KFW�XUZ5[ 0 ô1,b,�ô ç ¯±°h¹5µ�¶@¾x¿x»º²V¯±°hÆ Àh¿f¹"³}®h¿d¯Zï>®h¿8¼�¯±°h¹Ò¹Fd�µ�·%¯¼dµ�¶�Ç�¹"³t¿ É �#É�³º¿Á²V¯t·"µ�®h®h¿�¯tÇh³º¹Z¹Fd�µ�·%¯±¶64�¯º°h¹r�*Åiµ�³º²@³<M̲�®h³Ó¯±¹?µdÀê²@¯tµ�¸h¸�»º¿-d�²�Æȵd¯±¹3³�¹"µ�·'°��#É�ÅL4µ�³±µ�Æ5¸h¶@¹°¼�µd¶�Çh¹��ts É µ�®�À�Çh³R¹"³}¯±°h¹Ò»R¹"³ºÇ�¶@¯±²@®h¾�Åiµ�³R²�³<0I¼�°h²@·'°�¼½¹Ò·?µ�¶@¶K¯±°h¹u�\swÅhµ�³º²@³"ÍD+2®�Æ5¿x»º¹Àh¹%¯'µ�²@¶ 0�¯º°h¹(µ�¶�¾�¿x»º²V¯±°hÆ ²�³^µ�³ É ¿�¶�¶�¿8¼�³ Æ@ g^³º¹!D*E�FHGJILKNMOEPKPQ*RNS�T%q�¯±¿Ò²�Àh¹3®|¯±² É 4Áµ�³º¹3¯Y�O�(¿ É ²@Æȸ�¿x»Ó¯'µ�®|¯}¼dµ�»R²×µ�Åh¶@¹"³3Í

ôUö Ø

Page 185: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

@ Â�»ºµ8¼ ��¶×µ�Å�¹"¶@¹"À<¹Fd�µ�Æ5¸h¶@¹"³�ðn¸ º ± � å ¸ º ç�ñ ± _-_-_ ± ðn¸ � ± � å ¸ � ç�ñ É »º¿xÆ Øcî å � ± Ä ç _UÐ+¿xÆ5¸hÇ�¯º¹�tsÉ ¹ º� � �v �`º ¸ v É É ¿x» ô ô Ð ô æ _KÂ�¹%´i®h¹��[sÉ ¹ å ¸�É�Àw�tsÉ ç � � � sÉ å ô À=� sÉ ç µd®hÀf�\s � ¹� É � � � sÉ _

@ Ãi¿x»<¹F¼x¹"»�4¶´ù� �O�5¼�²V¯±° ¢ ´ ¢ ô � ³R¹3¯d�xs� ¹ º� � �v �`º � å ¸ v ç �\s � å ¸ v ç _·+ Éo¢ �ys� ¢ k�ô ³º¹%¯�ys� ¹�³R²�¾x® å �xs� ç _hÃ�¿�»~¹N¼x¹3»�4^´ ³ºÇ�·'°ª¯±°hµd¯ ¢ ´ ¢ k � ¿x»)´ Ö� �O�Z³R¹3¯��xs� ¹ s _

@ :�Ç�¯º¸hÇ�¯�¯±°h¹}°>4f¸Ì¿d¯±°h¹3³º²�³�³R²�¾x® å ü å ¸ çºç ± ¼�°h¹3»º¹ ü å ¸ ç ¹ � � � s� ��� å ¸ ç _Î�¹ ·?µ�¶@¶^¯º°h²�³.µ�¶�¾�¿x»º²V¯±°hÆñ~HS�KNMNF]�*��F*�s�*��F�S*qhÍpÖt³�²@® å Ã�Ç�»º³R¯ KFW�XUZ5[ 0 ô1,b,�ô ç ¼½¹ ®h¿�¯º¹ê¯±°iµ�¯³º¹%¯º¯º²�®h¾ �xs� ¯±¿{z ô ² É�¢ �xs� ¢ k ô ·?µ�®¥¿�®h¶64*Åh»R²�®h¾�¯±°h¹È¹"³Ó¯±²�Æȵd¯º¹"ÀB¼dµ�¶@Çh¹5·"¶�¿�³º¹"»}¯±¿ª¯±°h¹5¯±»ºÇ�¹¼dµ�¶�Ç�¹<¿ É 6� å ´ ç _ÿ����0op��= ����� ��������& (���(��$(������ ��� � ��� �&��� Þ�� �� ú �����Ãi¿x»~¯±°h¹ZÆÈ¿�³R¯�¸iµ�»R¯�¿x®h¶54�Æ5²�®h¿x»�·'°iµd®h¾x¹"³^¯±¿5¯±°h¹rµ�®iµ�¶54�³R²�³�¿ É Pf¹"·3¯º²�¿x® ô1s ÍðÑ�ÍðÑȵ�»R¹Z»R¹<ï>Çh²@»º¹"ÀGÍÖt³5Å�¹ É ¿x»R¹�¼½¹ª³RÇh¸h¸�¿x³º¹�¯±°iµ�¯È¯º°h¹�¯±µ�»º¾�¹3¯5·"¿x®�·"¹"¸�¯.²@³.µ�®Þ�N2-¯º¹"»ºÆ Â�]�Ã+ÍøP�²@®h·"¹ªµk¯±¹"»RÆ ¿ ɳº²67"¹�¾x»R¹?µd¯º¹"»7¯º°iµ�® º ²@³�³±µd¯º²�³R´h¹"À�ÅL4­µ »±µ�®hÀ�¿xƬ¹Fd�µ�Æ5¸h¶@¹ É »º¿xÆ Ä ¼�²V¯±°­¸h»R¿xÅiµ�Å�²�¶�²V¯=4u¶@¹"³R³¯±°hµ�® å l ý ºl ç ½ ± ¼½¹�®h¿8¼Ê¯±µ�ï�¹�¶@¿x¾f||,} Ò � %�� ß� ¹ ò å �(¶�¿�¾ � ß� ç µ�³7¯±°h¹È¯±¹"»RƦ³R²V7"¹�Å�¿xÇh®�À É ¿�»k�>ºF_Õ�»º¿f·"¹3¹"Àh²@®h¾(µ�³K²�®DPf¹"·3¯º²�¿x® ô-s Í Ñ�Í Ñt¼½¹~¿xÅ�¯±µ�²�® ¢ �O� ¢ ¹ a å ���w¶�¿x¾ � ß� ç _dÎ�¹~³º²@ÆȲ@¶×µ�»R¶64<³R¹3¯µt¯º¹"»ºÆ³º²67"¹�Å�¿xÇh®hÀ*¿ É ò å �(¶�¿�¾ � � ç É ¿x»c� % _GÎ�¹�Çh³º¹�¯±°�¹~�uÅiµ�³º²@³}Õ&µ�»R³º¹F¼�µd¶K²�Àh¹3®|¯±²@¯=4 µ�®hÀ*²�®�¹<ï>Çiµ�¶@²@¯=4å ô-s Í ô-s ç ²�®­¸�¶×µ�·3¹�¿ É ¯º°h¹Æ� Åhµ�³º²@³�²�Àh¹3®>¯º²@¯=4¥µ�®hÀ0²�®h¹Nï>Çiµ�¶�²V¯=4 å ô1s Í Ø ç »R¹"³º¸�¹"·%¯±²5¼x¹"¶54xÍçèI¹"Æ5Æȵô1s Í ô Ñ.¸�»º¿1¼f²�Àh¹3³t¯±°h¹r»º¹<ï>Çh²@»º¹3À*µ�®iµ�¶@¿x¾xÇh¹(¿ É âêµ�®�³º¿xÇh»G�ð³<¶@¹"Æ5Æ.µ É ¿x»�¸h»º¿fÀhÇh·%¯�Àh²@³R¯±»R²�ÅhÇ�¯±²�¿�®h³<MÇh³R²�®h¾�¯±°�¹(®�¹3¼Ô¯º¹"»ºÆ�³º²V73¹}ÅÌ¿xÇ�®hÀ�¿x®�� % ¼½¹(¿xÅ�¯'µd²�® � ¹|ò å � % ¶@¿x¾ � � ¶�¿�¾ º� ç _R~°h¹^¿x®h¹~®h¹%¼�²�®h¾�»º¹"À�²�¹"®|¯�²@®�¯±°h¹^µ�®iµd¶64f³º²@³�¿ É ~HS�KNMNF]�*��F*�s�*��F�S*qÒ·3¿xÆ5¹"³�²�®ÒÅ�¿xÇh®hÀ�²�®h¾<¯±°�¹ï>Çiµ�®|¯±²V¯=4 Á ¹�� � � � à � Ñ �O) �   å � s� À 6� å ´ çRç % _b+2®5µ�ÀhÀ�²@¯±²@¿x®r¯º¿}¯º°h¹~³±µdÆȸh¶@²�®h¾t¹"»º»R¿x»K¼�°h²@·'°Ò¼½¿xÇh¶@ÀÅ�¹È¸h»R¹"³R¹"®|¯�¹N¼�¹"®­² É �tsÉ ¼+¹3»º¹È¹Fd�µ�·3¯º¶64��#É ± ¼½¹.ÆrÇh³R¯rµ�¶�³R¿ªÀ�¹?µ�¶�¼�²@¯º°u¹3»º»º¿�»ZÀhÇ�¹È¯±¿�¯±°h¹ É µ�·%¯¯±°hµd¯»� s� ²�³+µ�®�¹"³Ó¯±²@Æ.µd¯º¹t¿ É ¯º°h¹�� s Åiµ�³º²@³z·"¿f¹-�5·3²�¹"®|¯~»ºµd¯±°�¹"»+¯º°iµ�®�¿ É ¯º°h¹���Åiµd³º²�³+·"¿f¹-�5·"²@¹"®|¯6� å ´ ç _�Öt®ªµ�®iµ�¶54�³R²�³~¹3®>¯º²�»R¹"¶64.³º²@ÆȲ@¶×µ�»L¯±¿�¯±°hµd¯�¿ É P�¹"·%¯±²�¿�® õ Í Ø�¿ É å ÃiÇh»R³R¯ KFW}XUZ5[ 0 ô1,b,�ô ç ³º°h¿8¼�³¯±°hµd¯}¯'µ�ï>²�®�¾�� ¹Ê¸�¿x¶64 å � ± ¢ �O� ¢ � ± � � ±?ô � ë<± ¶�¿x¾ å ô � í çºç ³RÇL�5·"¹"³(¯±¿�Å�¿xÇh®hÀ Á~ô ë � å _ Î�¹5¯º°>Çh³°iµ-¼x¹ÇÉÈ V�r*qsV]W Z�\A^ ZLó Ë m�KFEBX  ihÌ�E � mUJ Ç W�m I GFWnE InC J3W I �ÃÂ Ä �ÃÂQ¯ s ±?ô ² ß ± È-� E�X  i»ë<±lí�k s ±XUZhg � E I WrË A�~HS�KJMNF��*��FO�s����F�S�q ZVKfXUE  G � × W=KFE A A��ÃÂ#� W �àKy³µ´·¶ I/ W I/A K�Ì � Zji Â#�ÃAkI XUZ I/Â æ ±å �$�w¶�¿x¾ � ß� ç l : ÛÝÜ2Þ�ÐÑ�ÛÝÜ2Þ$ÒÑ ±³X  m ¶@¿x¾ å ô � í ç _P�²@®h·"¹(µ�·"¿x®�³R¯'µd®>¯�22Å�¿xÇh®hÀ�¹"Àê¸�»º¿fÀhÇh·3¯^Àh²@³R¯º»º²�Å�Ç�¯±²@¿x® Ä °iµ�³��¶¹|ò å ô ç ± ¼½¹(¿xÅ�¯'µd²�®

ôUö Ñ

Page 186: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

Ó r*qsrOÔmÔmYLq�ÕÖZ�\A^mZ]ú ¶ � EcX  i ÇH�àGFW X  W�ëkX  m�X  i ÇH�àGFW X  W × CF� J  m�Kfm�Ì�E � mUJ Ç W}m I GFWnE InC J3W I �ÃÂÄ ± XUZhg � E I W/Ë Aà~HS�KNMJF��*��F*�P����F�S*q ZVKfXUE  G Ø�Ù#i Ú ÛÝÜ2Þ�ß-j × W=KFE A×A��ÃÂ#� W �àK�³µ´�¶ I/Â Ì � Zji å æ ± ¶@¿x¾ å ô � í çºçW I/A K<[ÿ����0op�ÙØ ����� ��������& (���(��$(���� ä �"�!Ý��=� �g�³R²�®h¾�èI¹"Æ5Æȵ ô1s Í ô-å µ�®hÀâµ�®�µ�®iµd¶64f³º²@³7³º²@ÆȲ@¶×µ�»}¯±¿ ¯º°h¹�µ�ÅÌ¿1¼�¹U0L¼+¹�¿xÅ�¯±µ�²�®¥¯º°h¹ É ¿x¶�¶@¿8¼�²�®h¾»º¹3³ºÇh¶V¯±³3ÍÇÉÈ V�r*qsV]W Z�\A^ Z]¯ ¶ I�ô º/.�ô�X  m�ZVKFWæu C KoW/Ë�K Ç Z5XÃGHG �"È m�K=Ì�W/ËyºLø�G I¹¸ K Ú ÇFI E Ç J I W/G Ê Ë I Ç ËÇH�ÃA Ì�J�W"K A��ÃÂ#� W �àK È J Â#Ç W I �àG �ÃÂyÜ\� J3W �"È æ ÍÃXUE I X C ZVKHG<[ Ë m�KFE?X  i�Ì�E � mUJ Ç W�m I GFWnE InC J3W I �ÃÂÄ ± È-� E�X  ioë<±Híãk�s3±æXUZhg � E I W/Ë A¨~LS�KNMNF��O��F*�s���sF�S*q ZVKfXUE  G Ç Z5XÃGHG�u I/ W I/A KpÌ � Zji Â#�ÃAkI XUZ I/Âæ ± Ü i l ÛÝÜ2Þ�� Ñ�j ï X  m ¶�¿x¾ å ô � í ç _Ó r*qsrOÔmÔmYLq�ÕÖZ�\A^`_�\ ¶ I�ô ºà.Qô X  m¶ZVKFWtu C K^WrË�K Ç Z5XÃGHG �"È m�K=Ì�W/Ëfº�±}G I¹¸ K Ø ÙOi5i ÛÝÜ2Þ�ß1j Ò`îëûþïëüNÒ`ý jÇFI E Ç J I W/G Ê Ë I Ç Ë ÇH�ÃA Ì�J�W"K A��ÃÂ#� W �àK È J Â#Ç W I �àG �ÃÂ�Ø Ù#i6i ÛÝÜ2Þxß1j Ò`îëûþïëüNÒ`ý j ÍÃXUE I X C ZVKHG<[ãeOË�K Â�È-� EYX  iÇH�àGFW X  W ëpX  mYX  i ÇH�àGFW X  W × CF� J  m�KfmøÌ�E � mUJ Ç WOm I GFWnE InC J3W I �ÃÂ Ä ±3XUZhg � E I W/Ë Aà~HS�KNMJF��*��F*�P����F�S*qZVKfXUE  G Ç Z5XÃGHGtu I/Â Ì � Zji å æ ± ¶@¿x¾ å ô � í çºç W I/A K<[

ôUö8å

Page 187: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

� � ��� ����� ���

� � � � ��� � �������D�=�p��� �

�w�(��� ��¡a¢(£~£¤�,¥¤¢(�§¦¥~¥¤�(�,¨ª©(�,«���� ¬a�­¦~¡a©]¥¤�F¼a�S£¤®%�4�@Æu¯°¡a£�¦O®%©­¦~®%¥ ¯ �,±,¦¥5¡%²i±,¡xÆu¯³¢³¦O®[¦¤� ¡a©(®%� �§´�-�~±,� �S©­¦V� �S®%£¤©³� ©(¬(µ�Æ5®[©*´�� ©­¦�,£¤�S¥�¦��©(¬�ï�¢³�S¥ ¦¤� ¡a©³¥�£ �"Æu®%� ©�¦¤¡~¶°��®%©(¥�«_�,£¤�S¨t·¸�¹B! U�V�Y�qsª$åmªyº�vtR»�(�g¼�½¾ iÀ¿ Ò`îkÁ j ¦¤��Æ~�±S¡xÆ~¯(� �qdÃ��¦Ä´�¡%²Å¡a¢(£VÆ�]4Ç.� �S®%£¤©(��©(¬u®%� ¬a¡%£¤��¦¤�hÆY²H£¤¡xÆÈ �#®%¯Ã¦�S£�É�� ¥i®�¥ � ¬a©³��Ê#±S®%©­¦Â��Æ~¯(£¤¡1¼%�"Æ~�S©­¦A¡1¼a�S£A¯(£ �N¼�� ¡%¢(¥A£¤�S¥ ¢(��¦¤¥Sµy¶(¢³¦Â¦¤�(� ¥Â£ ¢(©(©(��©(¬�¦�@Æu�� ¥�¥�¦� ���˲k®%£r²H£¤¡xÆ̯°¡a�§´Ã©³¡xÆu� ®%��· È ®[©D¯°¡a��´�©(¡xÆ~�W®[�_¦�@Æu��®%��¬a¡a£ ��¦��Æu¥g¶°�Í¡a¶³¦O®[� ©(�,¨]²H¡a£�¦�³�¬a�,©(�S£¤®%��¯(£¤¡a¶³� �"ÆΡ%²AÏ Ð È � �Ñ®[£¤©(��©(¬�Æ�]4ÇÒ²H£¤¡xÆÌ£¤®%©(¨(¡xÆÎ�Fd³®�Æ~¯(� �,¥u¢(©³¨(�S£u®%©ª®[£¤¶(�§¦£®[£ ´¨(��¥ ¦£ � ¶(¢Ã¦� ¡%©\ÓÕÔ���Æ~� �W®[£¤��´%µÅ«��(� ���Ö¡%¢(£5®[� ¬a¡a£ ��¦¤�hÆÎ��© È �#®[¯³¦�,£B× s ²H¡a£�� �Ñ®[£¤©(��©(¬�Æ~¡a©(¡[¦¡a©(�ÆÂØ4ÇÙ¢(©(¨³�S£Ú¦�(�V¢(©(�§²H¡a£ºÆÛ¨(� ¥�¦£¤��¶(¢³¦¤� ¡a©�£¤¢³©(¥���©u¯°¡a�§´Ã©³¡xÆu� ®%�*¦��Æ~�˲H¡a£�ÆÂØ4Ç�«���¦¤��¼ ¾ i§Ü Ý�ÞÄß[¿Ñà¦�,£ºÆ~¥Sµ­�§¦Å£¤�3Æ5®[� ©(¥�®%©�¡a¯ �,©kï*¢(�S¥�¦� ¡%©�¦¤¡Â¨(�S¥ � ¬a©~®i¯°¡a��´�©(¡�Æu� ®%�­¦��Æ~�Ë®[� ¬a¡a£ ��¦¤�hÆá²H¡a£Å���Ñ®%£ ©(� ©(¬®%£ ¶(��¦¤£®%£�´�¯°¡a��´�©(¡xÆ~�W®[� ¥ � âS�<Æ~¡a©(¡[¦¡a©(�AÆÂØ�Ç=¢(©(¨(�,£�¦¤�(�A¢(©(�§²H¡a£ºÆ;¨(� ¥�¦£¤��¶(¢³¦¤� ¡a©�·Ó rOWCã�¬M$�Y³$�åmrOª�Y�Ô»ätY�WCã$ÔÙV Ó rOWCã$ÔmV­å�åF$(Õ®vÚÐÒ¬a¡­®%�°²H¡a£_²H¢³¦¤¢(£¤�£ �S¥¤�S®%£¤±O�Í� ¥�¦¡r£¤�,¯(�W®[±S�i¦�³�©(¡a©³¥ ¦O®[©(¨#®%£ ¨7±S£ ´�¯³¦¤¡a¬a£¤®%¯(�(��±B�#®%£ ¨(©(�,¥¤¥�®%¥ ¥¤¢hÆ~¯³¦¤� ¡a©ª¡[²�ÔÃ�S±?¦��¡a© å ·Àæõ˫��(��±O�ª«_�{¢³¥¤�S¨ª¦¤¡¯(£ ¡�¼%��¦¤�#®[¦�®[¦¤¦¤£¤��¶(¢³¦�A� �u±,� �,©*¦4� �S®%£¤©(��©(¬�±Ñ®%©Ö¶ �A±,¡xÆu¯³¢³¦O®[¦¤� ¡a©(®%� �§´5�#®[£¤¨�µ#«���¦¤�B®~¥ ¦®%©(¨#®%£ ¨®%¥ ¥¤¢hÆ~¯³¦��¡a©�·Åç0¦�«_¡%¢(� ¨�®%��¥¤¡�¶ ��� ©­¦�,£¤�S¥�¦��©(¬è¦¡è�?é�¦�,©(¨�¡a¢(£_±,£ ´�¯³¦¤¡a¬a£®[¯(�(� ±��#®%£ ¨(©(�S¥ ¥»£¤�,¥¤¢(�§¦²H¡a£è®[¦¤¦¤£¤� ¶³¢³¦��� �~±S� �,©­¦r� �S®%£¤©(��©(¬B¦¤¡�® Æu¡%£¤�5©(®[¦¢(£¤®%�_±,¡a©(±,�S¯³¦�±,�W®%¥ ¥g¥¤¢(±O�f®%¥è¦¤�(��±,�W®%¥ ¥g¡%²×,ê0¨(�S±,� ¥¤��¡a©�� ��¥ ¦¤¥»¡%²����S©(¬[¦�Öë ì&lr�rO«,$�åmªyº�v�í»�(�詳�,«�îðï&���s�­ñÃò����]T���®%��¬a¡a£ ��¦��Æó� ©­¦£ ¡Ã¨³¢(±S�,¨{� © È �#®%¯³¦¤�S£Âô~� ¥V¬a¢#®%£¤®%©­¦�,�S¨¦¡u¬a�S©³�S£®ð¦��¡a¯³¦�@Æ5®[� ��´u¥ºÆ~¡�¡%¦�ͨ(� ¥�¦£ � ¶(¢³¦¤� ¡a©³¥Sµ(¶(¢Ã¦i��¦�£ �<ï*¢(��£¤�S¥�×öõ ë ÷ % ¥ ¦O®[¬a�S¥�¡%²Å¶°¡Ã¡%¥ ¦��©(¬(·Ð4¥_¨(�S¥ ±S£¤��¶ �,¨�� © È �#®[¯³¦�,£¤¥�øg®%©³¨�ô³µ³¡%¦¤�(�S£_¶ ¡�¡a¥ ¦¤� ©(¬è®%� ¬%¡a£¤�§¦�hÆ~¥Ú¥¤¢(±O�Í®%¥»Ç#£ �S¢(©(¨Å�ù¥»¶°¡Ã¡%¥ ¦ 궭´­ê2Æu®öú ¡a£¤�§¦Ä´Í®%� ¬%¡a£¤�§¦�hÆ�®%©(¨�Ç#£¤�,¢(©(¨B®%©(¨ÙÔñO�(®%¯(� £ ���û¥4ü�GHKaò����]T���®%£ �èý*©(¡T«�©B¦¤¡�£¤¢(©Ö� ©B®%©¡a¯³¦¤��Æu®%�(��¡a¬\þ�×Tõ ëOÿ õ ÷ % ¥ ¦®%¬a�,¥Sµ�¶(¢³¦�¦�(�,¥¤��®%� ¬a¡%£¤��¦¤�hÆ~¥Å¨(¡è©³¡%¦�®%¯(¯°�Ñ®%£+¦¡�¬a�,©(�S£¤®[¦�V¡a¯³¦¤��Æu®%� �§´

×��aæ

Page 188: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

¥ºÆ~¡�¡%¦��¨(� ¥�¦£ � ¶(¢³¦¤� ¡a©³¥S·�çÄ¥A¦�(�,£¤��®Í¥¤��©(¬a� �~¶°¡Ã¡a¥�¦��©(¬B®[� ¬a¡a£ ��¦¤�hÆ «��(��±O�C��¥�¶°¡%¦��¡a¯³¦¤��Æu®%� �§´¥ºÆ~¡�¡%¦�Í®%©(¨�¡a¯Ã¦��Æu®%����´�²k®%¥ ¦OÓU�å ª�V�Y�q ÇdÈ q�V]« È r&Ôk'¨U�V]YLqPª�åmªxº�v��¢³£r®%� ¬%¡a£¤�§¦�hÆ~¥���© È �#®%¯Ã¦�S£�ô Æu®[¦¤±O��¦�(��Æu®%� ��±S��¡a¢(¥©(¡a��¥¤��¦¡a���S£¤®%©(±S��¡%²t¦�(�V¶ �,¥ ¦�¯(£¤�F¼�� ¡a¢(¥_®%� ¬%¡a£¤�§¦�hÆ~¥Sµa¶³¢³¦�¦¤�(�V� �N¼%�S�\¡%²IÆu®%��� ±S��¡a¢(¥Å©³¡a� ¥ ��«��³� ±O�¦�³�,´7±S®%©w�#®%©(¨(����� ¥�¥�¦��� �V¥¤� ¬%©(��Ê#±S®%©­¦�§´D¥RÆ5®%��� �,£u¦�(®%©7¦�(����©³²H¡a£ºÆu®[¦¤� ¡a©Ãê�¦�(�,¡a£¤�?¦��±Í¢³¯(¯ �,£¶°¡a¢(©(¨�· È ®[©ª�-�~±S���S©­¦���� ©(�S®%£g¦�³£¤�S¥ �(¡a��¨D� �S®%£¤©³� ©(¬�®%� ¬%¡a£¤�§¦�hÆ~¥g¶°�Ö¨(�,¥¤��¬a©(�S¨f«��(� ±O�ª�(®1¼%���Æ~¯(£ ¡�¼%�S¨�¦¡%� �S£¤®%©(±,�A¦¤¡ÒÆ5®%��� ±,� ¡a¢(¥�©(¡a��¥¤�TÓ

����

Page 189: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

� §µ8b§��ø§�ª��t§�¬â�·xÐÚú�¦®%��®%©(¨g·$�A¢(£ �N¼�� ±O�t·�×Ñôaø��÷�âÖ¡%©(¡%¦¡%©(�D¼%�S£ ¥¤¢(¥r¯°¡a¥¤�§¦�5¼a�%· �������qÂ������"È�� Ë����gI�� µÉ å þ å*ÿ! ×#"$" å�% ×#"³×öæ�·Ø�·�Ð4��¡a©�µ'&³·èÔï°�S©(±,�S£,µ�®%©(¨YÏp·Éx+£¤¨(¡a¥,· ×Tôaô­¼�· eOË��)( �*��+*��+-,.�/,10!�2, Ç �� � Ë �-3%· �w��� �?´*êçÄ©­¦�S£ ¥¤±,� �S©³±S�%µ Ø��?«4�¡a£ ý\·Æg·�Ð�©(¬a� ¢³� ©Ù®%©(¨�Ïx·èx®[� £¤¨t·5×Tôaø%ø³·°èM�Ñ®[£¤©(��©(¬�²H£¤¡xÆ ©(¡a� ¥�´{�né(®dÆu¯(���S¥,· � ��Ç Ë ,/ � a � ���qÂ�,/ g µ¼ É å É % É$��"³·Æg·(Ð�©(¬a� ¢³� ©�·Ë×Ñôaøaø³·65i¢(�,£¤� �,¥�®%©(¨Í±S¡a©³±S�S¯Ã¦4���Ñ®%£ ©(� ©(¬³· � ��Ç Ë ,/ � a � ���qÂ�,/ g µ\¼ ɳ×Ñô % É å ¼�·â�·�Ð4©­¦¤�(¡a©­´aµÉ��·)¾�£¤� ¬%�*¦Ä«��S����µÂ®[©(¨7&³·�ÔÃ�#®Ñ«��?ê0í�®Ñ´�� ¡a£,· ×Tôaôaæ÷8�i© ¥ ¯ �,±S�§²F´Ã��©(¬�¾_¡Ã¡%� �Ñ®[©²H¢(©(±?¦� ¡%©(¥�¢(¥¤��©(¬��W®%¶°�S��� �S¨��né³®�Æu¯³� �S¥,· ³ ,101Ç!� � � �9�}Ì>Ì �/, � 3 � ��� Ë��!: ���;, Ç*0?µ��³× × % ¼aæ÷â�·yÐ4©­¦¤�(¡a©­´a·Ö×Tôaô­æ÷ È � ®%¥¤¥ ��Ê#±S®[¦��¡a©�¶­´Ù¯ ¡a�§´�©(¡xÆ~�W®%�p¥¤¢(£�²k®%±S�,¥S· ³ ,101Ç!� � � �<�SÌ>Ì �/, � 3 � ��� Ë�� ×

: ���;, Ç*0?µ=�³× ô³× % ×#"%ɳ·x»·#Ð4£�¦� ©t·»×Tô$� å · eOË��~- � :>: � ¶ ��Â#Ç!�2,?�ÃÂ#·A@�¡a�§¦ÑµCBV��©(�S�#®[£ ¦V®%©(¨��w� ©³¥ ¦¡%©�µ#Ø��?«D�¡a£ ý°·&³·�Ð4¥¤� ®�Æ ®%©(¨ªÔ ·�Æi�S±Ñ®ð¦¢(£,·�×Ñôaôaø³·2Ôï°�S±,��Ê#±S®[¦��¡a©ª®[©(¨D¥ ��Æg¢(� ®[¦��¡a©.¡%²Â¥�¦O®[¦¤� ¥ ¦¤� ±S®%�)ï�¢³�S£ ´®%� ¬%¡a£¤�§¦�hÆ~¥Â²H¡a£g�-�~±,� �S©³±,´=®[©(¨=©(¡a��¥¤�5¦¤¡a� �,£®%©(±,�%·E�������FÂ����F�"È I � :YÌ �G� � �'�UÂ�3 / i 0!� �!:/ Ç!, � Â#Ç � 0?µ\æ�� ×Tô³× % ¼�"aø³·&³·³Ð�¥¤¯(©³�S¥Sµ�BA·$¾_�,� ¬a�,��µLâ�·#Ç#¢(£ ¥ ¦Ñµ(®%©(¨ÍÔ ·�B�¢(¨(��±O��·»×Tôaô å ·�í»�(�A�n鳯³£¤�S¥ ¥¤�5¼a�A¯ ¡T«��S£�¡%²·¼%¡%¦��©(¬¯ ¡%��´�©(¡xÆ~�W®%��¥S· I � : +-,/Â����H���I, ÇJ�%µt× å þ�¼ ÿI × % × å ·Ïp·MÐ�¢(�S£�®%©(¨�Ø�· È �,¥®ðê=¾_�W®[©(±O�(��·�×Tô%ôaø³·��i©ÃêÄ��� ©(�g���Ñ®%£ ©(� ©³¬�«��§¦�*Æu®%� ��±S� ¡%¢(¥i©(¡a��¥¤��®%©³¨�¦�³�±S��¡a¥¤¢(£ ��®%� ¬a¡%£¤��¦¤�hÆÍ· � Â�Â����K0��"È � ��� Ë��!: ���;, Ç*0<�UÂ�3 � �I�2, G Ç!,2����LFÂ�� � �.�/, g�� Â#Ç � µ#¼[É ø%É % ôaô³·Ïp·*Ð�¢(�S£�®%©(¨/â�·­��®%£RÆg¢³¦¤��·Ë×Tôaôaæ÷�íx£®%±Oý*��©(¬�¦¤�(��¶°�S¥�¦_¨³� ¥Hú ¢³©(±,¦¤� ¡a©�·�çÄ© ( �M�1Ç �*� 3�,/ g 0N�"ÈO� Ë��eOË ,.�I� i ×0/ ,�ô$� Ë / i�:YÌ �#0!,.� : �ච����Â�3����2,?�ÃÂC0P�"È I � :YÌ ��� � � / Ç!, � Â#Ç � µÃ¯#®%¬a�,¥�ÉÃ×ö¼ % É­¼�×a·Ïp·\Ð�¢(�S£,·4×Tôaô$�÷�èy�S®%£¤©³� ©(¬~©(�S¥�¦�,¨B¨(�5Ít�,£¤�S©³±S�S¥�� ©�¦�³��¯(£ �S¥ �S©(±,�è¡%²&Æ5®%��� ±,� ¡a¢(¥_©(¡a� ¥ �%· eOË�� � ×

� � �;, ÇJ��� I � :YÌ ��� � � / Ç!, � Â#Ç � µ ×Tø­æ(þ�× ÿ! ×öæ%ô % ×��aæ÷è+·�¾Ë®%¶#®%� µpè+·�Ç\¡%£ ¦©³¡ö«AµËØ�·�Ø�� ¥¤®%©�µ»®[©(¨DÐ�·û�w��¬a¨(�S£ ¥¤¡a©t·�×Ñôaôaɳ·�¾ËÏÚÏ �#®%¥u¥¤¢(¶°�?éï°¡a©(�,©Ãê¦� ®%�˦�@Æu��¥ ��Æg¢(� ®[¦��¡a©(¥r¢(©³� �S¥ ¥ÅxRQiÏÚí»ç âÎx �#®[¥u¯(¢³¶(� ��¥¤�#®%¶³� ��¯(£¤¡�¡%²H¥,· I � :YÌ ���S���2,?�ÃÂ����I � :YÌ � � ô$,.� i µ#É É$"�� % ɳ×Tø³·BA·�¾Ë®%�#®[¨(¢(£Sµ�×Tô��³×a· � � �=Ì � � 0 � Â��S���2,?�ÃÂT�"ÈU� Ë��ÂQ ��,/Â��V3�,10!�;�I,2+-���;,?�ÃÂW�"ÈU� � 0 Ì �ÃÂC0 � 0U�X�ZY[3�, ×Ç Ë ���H� : ����0\,.� �!: 0?µ*çÄ©'@�·(Ôá%� ¡xÆ~¡a©�µ­�,¨(��¦¤¡a£Sµ / �;�C3�, � 0O,/ÂULI� �!:]� Â���� i 0!,10N�UÂ�3 ( � � 3�, Ç!�;,?�ÃÂ#µ¯#®%¬a�,¥A×öæ%ø % ×^�aø³·ÚÔ�¦®%©³²H¡a£ ¨�_V©(�6¼%�S£ ¥¤��¦Ä´�ÏÚ£ �S¥ ¥S·Ïp·$¾Ë®%£ ¦¤� �?¦¤¦�®%©³¨�&³·#ÔÃ�#®Ñ«_�nêÄíp®Ñ´Ã��¡a£SµM×Ñôaôaô³· -O�  � �`���/,¹¸#���2,?�àÌ�� �/È^��� : �UÂ#Ç � �"È\0!� Ì>Ì ���I� Í�� Ç!�H���

: ��Ç Ë ,/ � 0a�UÂ�3���� Ë�� � Ì ���;� � �qÂÙÇ!�b��0*0!, G�� �M0?µtçÄ©¶¾�·MÔñO�(¡%� ý%¡%¯³²�µ È ·c&³· È ··¾_¢³£¤¬a�,¥SµM®%©³¨�Ð�·c&³·Ô�Æ~¡a�W®³µ��S¨(�§¦¡a£ ¥Sµ � 3 Í �UÂ#Ç � 0d,/Âfe � �q � � �g� � Ë �-3�0�h / � Ì>Ì ���I�ji � Ç!�H���Æa � ���FÂ�,/ g µ�¯#®%¬a�,¥å É % æ å ·�âÖçÄíáÏ+£ �S¥¤¥,·×��$�

Page 190: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

Ïp·3¾Ë®%£�¦���,¦¤¦Sµ�Ïp·3èy¡%©(¬(µÃ®%©(¨UBA·*�w� � ���W®�Æ~¥¤¡%©�·+×Tôaô$�÷ÅÇ\®[¦ ê0¥¤�#®ð¦¤¦�,£¤��©(¬è®[©(¨5¦�³�4� �S®%£¤©(®%¶(� ����¦Ä´�¡%²£¤�S®%�§ê ¼%®[� ¢(�,¨�²H¢(©(±,¦¤� ¡a©³¥S·F�����G�qÂ������"È I � :YÌ ��� � �V�UÂ�3 / i 0!� �!: / Ç!, � Â#Ç � 0?µ\æa¼(þkÉ ÿ! Úå É å�%�å æa¼Ã·x»··¾4·¾Ë®[¢hÆ ®%©³¨k�ê�Æg·#èM´�¢(¢t·�×Tôaô³×a·�í»�(�r¦£¤®%©(¥¤�§¦��¡a©B¦¡�¯°�S£�²H�S±,¦A¬%�S©(�,£®%��� âÑ®ð¦� ¡%©{� ©�¯°�S£�ê±S�,¯³¦£ ¡a©(¥S· ´V� �G�`��� I � :YÌ ���S���2,?�ÃÂ#µ#É Éaø$� %Lå "³×a·x»·�¾Ë®%¢hÆÍ·á×Tôaô�"³·Dí»�(�ÖÏp�,£¤±,�S¯³¦¤£¤¡a©w®%��¬a¡a£ ��¦��ÆÎ� ¥r²k®%¥�¦u²H¡%£~©³¡a©hÆu®%� ��±S� ¡%¢(¥g¨(��¥ ¦£ � ¶(¢Ã¦� ¡%©(¥S·´<� ���`��� I � :YÌ �G�?���;,?�ÃÂ#µ#¼ ¼ å ø % ¼���"³·Ïp·¾��Ñ®�Æ~�%·�×Tôaô å ·�Ð ¥�«���¦¤±O�(� ©(¬Í� �3ÆÈÆu®5¯(£ ��Æ~�S£,·ríp�,±O�(©(� ±S®%�6BV�,¯ ¡%£ ¦<_V�ªê È ÔLx�ê�ô­æöêX"��öêX"³×aµ

_�©(�5¼a�,£¤¥¤�§¦Ä´�¡%²x��®%¥ �(� ©³¬%¦¡a©t·BA·�¾_�S��¬a�S� µ�Ø�·lB��S� ©³¬a¡a� ¨tµÚ®%©(¨fÆg·ÚÔï(���S��Æu®%©�·f×Tôaô³×%·Cí»�(�ͯ �,£¤±,�S¯³¦¤£¤¡a©]¥ ¦¤£¤� ý[�S¥~¶#®%±Oý\·.çÄ©

( �*�1Ç �*� 3�,/ g 0m�"Èd� Ë�� / ,�ô$� Ë I �ÃÂ-È � � � Â#Ç � �à/ �2�I� Ç!�2�G� � ,/ I � :YÌ � � ô$,.� ieOË�� ��� i µ»¯#®%¬a�,¥¼%ø$� % ¼%ô³×a·BA·$¾_�,� ¬a�,��·Ë×Tô%ôaɳ·Åí»�(�¯°¡a�§´Ã©³¡xÆu� ®%��Æu�?¦�(¡�¨�� ©5±S� £ ±S¢(�§¦Ë±,¡xÆu¯³� �?éÃ�§¦Ä´a·ÅçÄ© ( �M�1Ç �*� 3�,/ g 0P�"È�� Ë��

no, g � Ë�I �ÃÂ-È � � � Â#Ç � �à/ �2�I� Ç!�2��� � ,/ I � :YÌ � � ô$,.� i<eOË�� ��� i µ(¯#®[¬a�S¥Vø­¼ % ô­æ÷BA·�¾_�S��¬a�S� ·�¼�"$"$"÷+Ïp�,£¤¥¤¡%©#®%�y±S¡xÆ5Æg¢(©³� ±Ñ®ð¦� ¡%©�·Ðè·O¾_�S�@Æu�,��µMÇ_·O¾_�S£ ¬­®%¨#®%©³¡(µyØ�··¾�¥¤�(¡a¢Ã¦Ä´aµ$x�·Å¢(¥ �(� ���N¼���¦¤â%µM®[©(¨�Ô ·qp�®%£¤£ � ±,±O�(� ¡(·u×Tôaô$�÷>�i©¦�(�_®%¯(¯(��� ±S®[¦��¡a©(¥y¡%²iÆg¢³��¦��¯(� ��±S�§¦Ä´i®%¢Ã¦¡xÆu®[¦O®V� ©�� �Ñ®[£¤©(��©(¬(·MçÄ© ( �M�1Ç �*� 3�,/ g 0O�"Èo� Ë��leOË ,.�I� i ×/ � Í�� Â�� Ë�� Â�Â��C��� / ir:YÌ �#0!,.� : �ච����Â�3����;,?�ÃÂC0\�"È I � :YÌ ��� � � / Ç!, � Â#Ç � µ­¯#®[¬a�S¥�É å ô % É­æ%ø³·â�··¾��S� � ®%£¤�[·u×Ñôaô­¼Ã·�Ð9¦�,±O�(©(�Vï*¢(�r²H¡a£4¢(¯(¯°�S£A¶ ¡%¢(©(¨(��©(¬�¦�(�r¥ ¯ �,±,¦¤£®%�+©(¡%£ºÆ «��§¦�Ù®%¯³¯(� ��±Ñ®ðê¦��¡a©(¥+¦¡�� �Ñ®[£¤©(��©(¬(·xçÄ© ( �M�1Ç �*� 3�,/ g 0N�"ÈO� Ë��»¶ , ÈI� Ë>� Â�Â��C���6s����*t#0 Ë � Ì �àI � :YÌ ���S���2,?�ÃÂ����a � ���FÂ�,/ gfeOË�� ��� i µ#¯#®%¬%�S¥9�­¼ % �r"³·@�·�¾_��¡Ã±Oý\·i×Ñô$�­¼Ã·�í»�(�gÏx�S£¤±,�S¯³¦¤£¤¡a© ®5Æ~¡Ã¨³�S��²H¡a£V¶(£®[� ©�²H¢(©(±,¦¤� ¡a©³� ©(¬(·Fu � Í , � Êv0a�"È � �-3 � �qÂ

(øË]i 0!, Ç*0?µ\É åC ×ö¼[É % ×TÉ­æ�·â�·¾_��¢hÆ ®%©(¨CÔ ·�âÖ��±Ñ®%����·~×Tôaø å ·P@�¡T«�¦¤¡�¬a�S©(�,£®[¦¤��±S£ ´�¯³¦¤¡a¬a£¤®%¯(�(��±Ñ®%����´Ö¥ ¦£ ¡a©(¬�¥¤�Nï*¢(�S©(±,�S¥¡%²p¯(¥¤�,¢(¨(¡[ê0£®%©(¨³¡xÆZ¶(��¦¤¥S· / L ��� �����G�qÂ����v�àI � :YÌ ���;,/ g µt×TÉ ø­æ�" % ø�� å ·Ðè·#¾_� ¢�Æ�®%©³¨�Ô ·qB�¢(¨(��±O��·�×Tôaô­æ÷AÇ\®%¥�¦i���Ñ®%£ ©(� ©(¬5¡%²�ë*ê0¦¤�S£RÆ�ÆÂØ�ÇD²H¡a£ºÆg¢³�W®%¥�«���¦¤�hï�¢³�S£¤���S¥,·

�����G�qÂ����v�"È I � :YÌ ��� � �V�UÂ�3 / i 0!� �!: / Ç!, � Â#Ç � 0?µ\æÃ×*þkÉ ÿ! É$��� % É��%ɳ·Ðè·8¾_��¢hÆÖµNâ�·aÇ#¢(£ ¥ ¦Sµ�&³·�&a®[±Oý�¥ ¡a©�µ]â�·UÅÂ�Ñ®%£ ©(¥Sµrg·JâB®[©(¥¤¡a¢³£Sµ­®%©³¨~Ô ·�B�¢(¨(��±O��·_×Ñôaô å ·y�C�S®%ý*��´� �S®%£¤©(��©(¬gÆÂØ�ÇC®%©³¨�±O�#®%£¤®%±,¦¤�S£¤��âS��©(¬è¥�¦O®[¦¤� ¥ ¦¤� ±S®%� ï�¢³�S£ ´u� �S®%£¤©³� ©(¬�¢(¥¤��©(¬rÇ#¡a¢(£ � �,£_®%©(®%��´�¥¤��¥S·çÄ© ( �M�1Ç �*� 3�,/ g 0>�"Èw� Ë���e Ê � Â�� i ×0/ ,�ô$� Ëx� Â�Â��C��� / ir:YÌ �#0!,.� : �àeOË�� ��� i �"È I � :YÌ ���;,/ g µ¯#®%¬a�,¥�¼aæ%É % ¼r�­¼Ã·Ðè·L¾�� ¢hÆ͵>è+·�@��,� ���S£¤¥�¦�,� ©�µ­®[©(¨5Ø�·>èM��¦ ¦� �,¥ ¦¤¡a©(�%·�×Tôaô­æ�·�èM�Ñ®[£¤©(��©(¬A� ©r¦�(�»¯³£¤�S¥ �S©(±,�4¡%²°Ê#©(�§¦�,��´¡a£M� ©ÃÊ#©(��¦¤�S�§´tÆu®%©­´i��£¤£¤�,� �F¼%®[©*¦M®[¦¤¦¤£¤� ¶³¢³¦�,¥S·A�����G�qÂ����C�"È I � :YÌ ��� � �o�UÂ�3 / i 0!� �!: / Ç!, � Â#Ç � 0?µæ�" É­¼ %�å "³·Ðè·Ã¾_��¢hÆ͵ðÐ�·ðÇ\£ � �,âS�%µrBA·8ÅA®%©³©#®%©�µ�µa®%©³¨rÔ ·#pÚ�"Æ~¯#®%� ®³·+×Tôaô$�÷pÐ]¯°¡a��´�©(¡xÆ~�W®[�[¦�@Æu�Ú®[� ¬a¡a£ ��¦¤�hƲH¡a£»� �S®%£¤©³� ©(¬r©(¡%� ¥ ´�� � ©³�Ñ®%£_¦¤�(£¤�,¥¤�(¡%� ¨�²H¢(©(±?¦� ¡%©(¥S· � � g ���I,.� Ë�: , ÇJ�%µ(¼a¼(þ�×�ya¼ ÿI É­æ % æa¼Ã·

×��%ø

Page 191: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

Ðè·�¾_��¢hÆÍ·i×Tôaô­¼�·FB�®[©(ýaê¾Ü�¨(�S±,� ¥ � ¡a©Í¦£¤�,�S¥4®%£¤��®u¥ ¢(¶ ±,�W®%¥ ¥4¡%²�Üöê0¨(�S±,� ¥ � ¡a©Í� ��¥ ¦¥,·zLFÂ-È^��� : ���2,?�ÃÂ( �*�1Ç � 0*0!,/ g a � �2� � �*0?µ å ¼(þ å*ÿI ×ÑøaÉ % ×Tøaæ÷

Ðè·�¾�� ¢hÆÍ· ×Tô%ô$�³· �i©ÃêÄ��� ©(� ®%� ¬a¡%£¤��¦¤�hÆ~¥�� © Æu®%±O�(��©(� � �S®%£¤©(��©(¬(· ®1¼[®%���W®%¶³� � ®ð¦ñ]�]��{�|`}~}r�~�~���J�]Tv�J�Ñï��F��SNG$��}C�PK$�]MOEÑï�}���K�{�SNM*T~}�{~�]Q&Tv� ñL�aïOR÷Ðè·�¾_��¢hÆÍ·�×Tôaô��÷/xLÆ~¯(��£¤� ±S®%�+¥¤¢³¯(¯ ¡%£ ¦�²H¡a£��w��©(©(¡T« ®%©(¨�«_�,� ¬a�­¦¤�S¨Ãê-Æ5®öú ¡%£¤��¦Ä´�®[� ¬a¡a£ ��¦¤�hÆu¥ £¤�,¥¤¢(�§¦¥�¡a©Ö®�±S®%� �,©(¨#®%£»¥ ±O�(�S¨³¢(� ��©(¬u¨³¡xÆ5®[� ©�· � ��Ç Ë ,/ � a � ���qÂ�,/ g µ\¼�� æ % ¼[ɳ·Ðè·³¾_��¢hÆu�,£SµÅÐ�·�x+�(£ �S©³²H�,¢(±O�­¦Ñµ�Æg·v@�®%¢(¥ ¥¤� �,£Sµ+®[©(¨[â�·y��®%£RÆg¢³¦¤��·C×Tô%ø��÷k�i±,±Ñ®�Æ��ù¥�£®%â,¡a£S·

LFÂ-È^��� : ���2,?�à( �*�1Ç � 0*0!,/ g a � �2� � �*0?µ°¼ å= É$�$� % Éaø�"³·Ðè·#¾_� ¢�Æu�,£Sµ°Ðè·�x+�(£ �S©³²H�,¢(±O�­¦ÑµxÆg·q@�®%¢(¥ ¥¤� �,£SµM®[©(¨fâ�·t�C®%£RÆg¢³¦�t·g×Tôaøaô÷�èM�Ñ®%£ ©#®%¶(��� ��¦Ä´Í®[©(¨¦�(�Np�®%¯³©(� ýaê È �³�S£�¼%¡a©(�,©(ý*� ¥V¨(��Æ~�S©(¥ � ¡a©t·������G�qÂ����v�"Èw� Ë��\�gI�� µ#É��#þ å*ÿI ôa¼%ô % ô$�aæ÷Æg·ø¾_¡a©³�S�ª®[©(¨�BA·øèM� ¯³¦¤¡a©�·2×Tôaôaɳ·fÐtÆ~¯(� �§Ê#±Ñ®ð¦� ¡%©�¡%²4«��Ñ®[ý]���Ñ®%£ ©(� ©³¬�¡1¼%�S£u¦¤�(�Í¢(©(��²H¡%£ºÆ¨(� ¥�¦£ � ¶(¢³¦¤� ¡a©t·�çÄ© ( �M�1Ç �*� 3�,/ g 0�"Èk� Ë�� / ,�ô$� Ë�� Â�Â��=���Usj���Mt#0 Ë � Ì �àI � :YÌ ���S���2,?�ÃÂ����a � ���FÂ�,/ gfeOË�� ��� i µ#¯#®%¬%�S¥VÉ å � % É­æÃ×a·BA·Y¾_¡%¯(¯#®%©#®D®%©(¨�â�·iÔÃ� ¯³¥¤�S£,µu×Ñôaô$"³· eOË�� Ç*� :YÌ � � ô$,.� i �"È G Â�,.� � ÈI��Â#Ç!�;,?�ÃÂC0?µAçÄ©�&³·)¼[®%©èM�S�S¢­«��S©�µÂ�,¨(��¦¤¡a£SµN�N����3�+!�#�#tf�"È eOË�� ��� � �2, ÇJ��� I � :YÌ ��� � � / Ç!, � �#Ç �1øõÍ ���/� :x�d� µË¯#®%¬a�,¥

�aæ$� % ø$" å ·�âÍçÄíÒÏ+£¤�S¥ ¥S·Ø�·q�_¥¤�(¡%¢³¦Ä´Ù®%©(¨ È ·tí�®�Æ~¡a©�·�×Tô%ô$�³·P�©{¦¤�(��Ç#¡a¢(£ � �S£�¥¤¯°�S±?¦£¤¢�Æ ¡%²Æu¡%©(¡%¦¡%©(�è²H¢(©(±?¦��¡a©(¥S·

�����G�I�����v�"Èw� Ë��\�gI�� µ å É#þ å*ÿI � å � % �$�r"³·Ø�·q�_¥¤�(¡%¢³¦Ä´aµ�&³·�&a®%±Oý*¥ ¡a©�µx®%©(¨ È ·�íp®�Æ~¡a©�·~×Tôaôaô³·dâÖ¡%£¤�r�-�~±,� �S©­¦�Ï Ð È � �S®%£¤©(��©(¬�¡%²�ÆÂØ�Ç«���¦¤�¥Æu�3Æg¶°�S£¤¥ �(� ¯»ï�¢³�S£¤���S¥g¢³©(¨(�S£�¦�(��¢³©(��²H¡a£RÆ ¨(� ¥�¦£ � ¶(¢³¦¤� ¡a©t·ÖçÄ© ( �M�1Ç �*� 3�,.� g 0'�"Èx� Ë��e Ê � � ÈI� Ëx� �����C��� I ���-È � � � �#Ç � ��� I � :YÌ ���?���;,?�������&a � ���I��,.� gìeOË�� ��� i µ(¯#®%¬a�,¥�¼%ø$� % ¼[ô­æ÷Ø�·��_¥¤�³¡a¢³¦Ä´a·�×Tô%ô­æ÷Äx�é³®%±,¦i� �Ñ®[£¤©(��©(¬����W®~¦�(�7Æ~¡a©(¡[¦¡a©(��¦�(�,¡a£ ´%·VL!�-È^��� : ���2,?���E����3 I � : ×Ì �G�?���2,?���#µM×ö¼%É#þ�× ÿ! × å � % ×öæ%ɳ·Ø�·C�_¥¤�³¡a¢³¦Ä´a·�×Tô%ô$�³·ÅÐᥠ¢(¶°�?éï ¡%©(�S©­¦� ®%�t�né³®%±,¦����Ñ®%£ ©(� ©(¬è®%� ¬%¡a£¤�§¦�hÆ9²H¡a£_ÆÂØ4ÇC¢³¥¤� ©³¬r�<ï*¢(���[®ðê� �,©(±S�?ï*¢(�S£ � �,¥S·oL!�-È^��� : ���2,?��� ( �*�1Ç � 0*0!,.� g a � �2� � �*0?µ°æ[ô É$� % Éaô³·Ø�·��_¥¤�³¡a¢³¦Ä´a·~×Tôaô$�÷�ÔÃ��Æ~¯(�������Ñ®%£ ©(� ©(¬5®%� ¬%¡a£¤�§¦�hÆ~¥�¢³¥¤� ©³¬�¨(������¨(�r®%©(¨�±,¡a©$ï*¢(�,£S· I � :YÌ �G�?� ×

�;,?������� I � :YÌ � � ô$,.� i µ�� ×�� å�% ×Ñô å ·í· ��´Ã� ®%©(¨(�,£S· ×Tô%ôaøa®³· èM�S®%£¤©(��©(¬ ©³¡a� ¥�´ ��� ©(�S®%£ ¦�(£ �S¥ �Ãê¡a� ¨ ²H¢(©³±,¦��¡a©(¥,· ¥¤¢(¶hÆ~��¦ ¦�,¨ ²H¡a£ ¯³¢(¶(� ��±Ñ®[¦¤� ¡a©tµ ®#�[®%���W®%¶³� � ®ð¦ñ]�]��{�|`}~}�M*E�FG�HSJMo�J��Tv�?�]�*T�Kl� SNG���}�MLSLT�S]KJM��Tñ�}%ü��$}sQC��RNKNFHGHSNM�}�{~��QtT�}�{~�HQOTv� ñL�aïOR(·í·$��´Ã� ®%©(¨(�,£S·�×Tôaôaø%¶t·x��¡a£¤¥�¦ ê0±Ñ®%¥ ��®%©#®%�§´Ã¥ � ¥p¡%²#¦�(�ËÏp�S£ ±S�,¯³¦£ ¡a©~®%©(¨r�?éï°¡a©(�,©*¦¤�W®[¦¤�S¨�¢(¯°¨#®[¦¤�®%� ¬%¡a£¤�§¦�hÆ~¥S· � �!�2, G�� ,2����LI��� � �.�/, g�� � �*� µ ×#"$�³·Ø�· È �,¥®ðêX�_�W®[©(±O�(��µµx�·VÆi��±O�*¦¤�S£RÆ5®%©tµiÏp·�Çp� ¥¤±O�³�S£Sµ!x»·VÔÃ�(®�Æu��£Sµ4®%©(¨4@�·�_�·�ÔÃ��Æ~¡a©�·Z×Ñôaôaô³·Ô³®�Æ~¯(� �nêÄ� �u±,� �,©*¦�¥�¦£¤®[¦�,¬a� �,¥�²H¡%£r� �Ñ®[£¤©(��©(¬�� ©�¦¤�(��¯(£¤�,¥¤�S©³±S�B¡[²4©(¡%� ¥¤�[·������G�I�����O�H�'�.� �

�gI�� µ å �#þ�æ ÿI �%ø å�% �Ã×Ñô³·×��%ô

Page 192: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

x»· È �(�,©(�,´%·D×Tô��$�³·mLI���;�M�-3�� � �;,?�����H�m�!�����M�-ô$, : ���;,?���)�.� � ��� i · âÖ±��£®Ñ«ËêX@�� ����µ�Ø��,«��¡a£ ý°µØ��?«D�¡a£¤ý\·@�· È �(�,£¤©(¡r��·»×Tôaæa¼Ã·�Ð Æ~�Ñ®%¥ ¢(£¤�V¡%²y®%¥�´fÆu¯³¦¤¡%¦��±��-�~±S���S©(±?´~²H¡%£�¦¤�S¥ ¦¤¥_¡%²�®��­´�¯ ¡[¦�(�,¥¤� ¥�¶#®%¥ �S¨¡a©�¦�(�A¥¤¢�Æó¡%²p¡a¶(¥¤�,£`�[®[¦¤� ¡a©(¥,· � ����� � ���.�C� / �?���;,10!�?� µ°¼%É Úå ôaÉ % æ�"��÷x»· È ¡a�(�,©�·�×Ñôaô��÷øèM�Ñ®[£¤©(��©(¬�©(¡a� ¥�´�¯°�S£¤±,�S¯³¦¤£¤¡a©³¥�¶­´Í®r¯ �,£¤±,�S¯³¦¤£¤¡a©Ö� ©�¯ ¡%��´�©(¡xÆ~�W®%�\¦�@Æu�[·+çÄ©

( �*� �*�*� 3�,.� g 0z�H�6�.� �Ée ��,.�I� i × no, g ���.� / i�: ���#0!,.� : ��� ¶ ���G��3����2,?���C0F�H� I � : ����� � � / � , � � �*� µ¯#®%¬a�,¥�æÃ× å�% æ%¼Ã×a·Æg· È ¡a¯³¯ �,£¤¥ºÆ~��¦¤��®%©(¨�Ô ·��w� ©³¡a¬a£®[¨�·�×Tôaø$�÷íâB®[¦¤£¤� ékÆg¢(�§¦��¯(� ��±Ñ®[¦¤� ¡a©Z���W®�®%£ ��¦��Æu�?¦� ±è¯(£¤¡[ê¬a£¤�,¥¤¥ � ¡a©(¥,·_çÄ© ( �M� �*�*� 3�,.� g 0��H�w�.� �!´ ,.� � � �*� ���.� / ir: ���#0!,.� : ��� e � � ��� i �H� I � : �����;,.� g µ¯#®%¬a�,¥A× % �³·çO· Æ®%¬­®%©�µ=g·��A®%£ ¡#� µ ®%©(¨{Æg·�BV¡%¦¤��·A×Tôaô���··âÖ� ¥�¦O®%ý[�?êĨ³£¤���%�S©{���Ñ®%£ ©(� ©³¬u��©Í¦¤�?é�¦i±S®[¦�,¬a¡a£¤��âÑ®ðê¦��¡a©�·rçÄ© ( �M� �*�*� 3�,.� g 0��H�>��� � / �*� ����3 I ���^� � � � � �*� ���gn : ��,.�I, � ��� �� �����-3�0a,.� ´ ���2�G�`���aq��� g �=� g��N( �M� �*� 0*0!,.� g µ³¯#®[¬a�S¥�æaæ % �aɳ·Ðè·tÆi�Ñ®[¦¤¡a©�®%©³¨kB�·tÔÃ�,£`�a�,¨(� ¡³·�¼�"$"³×a·µ�A�,©(�g¥�¦£¤¢³±,¦¢³£¤�r¯(£ �S¨(��±,¦��¡a©Ö²H£ ¡xÆáÆ5®%©­´Ö®[¦ ¦£¤��¶(¢³¦¤�S¥S·íp¡u®%¯(¯°�Ñ®[£»� ©k�����G�I�����v�H� I � : �����S���2,?������� c ,?���1� g�i ·Ô ·�Æi�S±S®[¦¢³£Sµz�g· �A¡a��¨(£¤�,� ±O��µ�®%©(¨ªÆg·lBV¡a©t·Ò×Tôaôaô³· È ¡xÆu¯³¢³¦O®[¦¤� ¡a©(®%�Ë¥®dÆu¯(���Ö±,¡xÆ~¯(� �né³�§¦Ä´a·/ L �\� �������I�����v��� I � : ���G�2,.� g µ#¼%ô(þ É ÿI ø­æ å�% ø��%ôÃ·Ô ·MÆi�,±Ñ®[¦¤¢(£S·5×Tôaôaɳ·èÔ�¦®[¦��¥ ¦��±Ñ®%�³ï*¢(�S£ � �S¥�®%©³¨Ù²k®[¢(��¦Ä´BÏ Ð È ¡a£®%±,� �,¥S·èçÄ© ( �M� �*�*� 3�,.� g 0��H�>�.� �/ ,�ô$���fs����Mt#0*���*�E��� I � : �����S���2,?�������&a � ���I��,.� gìe � � ��� i µ(¯(®%¬a�S¥�¼��a¼ % ¼��aø÷â�· Æi�S£�¦¡a¢(â,¡a¥S·Â×Ñô$�­æ÷ e ��� � 0*�����b3x�1� g , �-  �U0 i ����� � 0!,10��!�����M�-� � ��·»âÖçÄí Ï+£ �S¥¤¥,µ È ®�Æg¶³£¤� ¨³¬a�%µâÖÐè·í·���·°Æi� �?¦¤¦¤�S£¤��±O��·»¼r"$"$"³·�Ð4©B�néï �,£¤��Æ~�S©­¦®%�y±S¡xÆ~¯#®%£ � ¥¤¡%©Ö¡%²p¦�(£ �S�(Æu�?¦�(¡�¨(¥�²H¡a£�±S¡a©³¥ ¦£ ¢(±,¦�ê� ©(¬5�S©(¥ �"Æg¶(���S¥Â¡%²Ú¨³�S±S��¥¤��¡a©B¦£ �S�,¥ ¶(®%¬a¬a��©(¬(µ°¶ ¡�¡a¥ ¦¤� ©(¬³µt®%©³¨�£®%©(¨³¡xÆu��âÑ®[¦¤� ¡a©t· � � � ��,.� �a � ���!��,.� g µ å "#þ�¼ ÿ! ×TÉaô % ×Tæ%ø³·È · Æi¡xÆu��©(¬a¡~®%©(¨k�r·\�C®[¦O®%©(®%¶ �[·�¼�"$"$"³·�âB®[¨#®$�_¡�¡a¥ ¦ ®ÈÆ~¡Ã¨³��Ê#�,¨ �a�,£¤¥ � ¡a©{¡[²+Ð4¨#®��_¡Ã¡%¥ ¦Ñ·çÄ© ( �M� �*�*� 3�,.� g 0x�H����� �<e ��,.�I� �*� ����� � �����=��� I ���^� � � � � �*� ��� I � : �����S���2,?��������a � ���I��,.� ge � � ��� i µ#¯#®[¬a�S¥A×Tø$" % ×Tøaô³·Ðè·Px+�(£¤�,©³²H�S¢³±O�*¦Sµ�Æg·�@�®%¢(¥¤¥ � �,£Sµ�â�·r�Â�Ñ®%£ ©(¥Sµa®%©(¨kè+·�p�®%� � ®%©­¦Ñ·Ú×Tôaøaô÷xÐf¬a�S©³�S£®[�³� ¡T«��S£p¶ ¡a¢³©(¨¡a©Ù¦�(�~©*¢hÆg¶°�S£�¡%²_�né³®�Æu¯³� �S¥�©(�,�S¨(�,¨�²H¡a£A� �S®%£¤©³� ©(¬(·¡LI�^�^��� : ���;,?���)����3 I � : ���G�?���;,?���#µø­¼(þkÉ ÿ! ¼ å � % ¼aæÃ×a·r·°Ç#£¤�,¢(©(¨B®%©(¨ B�·°ÔñO�(®%¯(� £ �%·i×Ñôaô$�³· ��®dÆu�¦�³�S¡a£�´aµ\¡a©Ãê0� � ©³�¯(£¤�,¨(� ±?¦� ¡%©Ö®[©(¨B¶°¡Ã¡%¥ ¦��©(¬(·�çÄ©

( �*� �*�*� 3�,.� g 0>�H�w�.� �!´ ,.���.� � �����C��� I ���^� � � � � �*� ��� I � : �����?���;,?�������&a � ���I��,.� g e � � ��� i µ¯#®%¬a�,¥VÉ­¼aæ % É%É­¼Ã·r·%Ç#£¤�S¢³©(¨~®%©(¨¡B�·aÔñO�#®%¯³� £¤�[·_×Tô%ô��÷xÐ7¨(�S±,� ¥¤��¡a©Ãê�¦�(�,¡a£¤�?¦� ±�¬a�S©³�S£®[� � âS®[¦��¡a©è¡[² ¡%©ÃêÄ��� ©(��� �S®%£¤©Ãê� ©(¬Ö®%©(¨C®%©�®%¯(¯(��� ±Ñ®ð¦� ¡%©Ù¦¤¡Ö¶°¡�¡a¥ ¦¤� ©(¬(·Z�����G�I�������H� I � : ����� � ������3 / i 0!� �!: / � , � � �*� 0?µæaæ(þ�× ÿ! ×a×Tô % ×TÉaô³·

×Tø$"

Page 193: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

r·�Ç\£ �S¢(©³¨Ù®%©(¨B�·�ÔñO�#®%¯³� £¤�[·u×Ñôaôaø³·cèy®%£¤¬%�7Æ5®[£¤¬a��©B±S� ®%¥¤¥ ��Ê#±S®[¦��¡a©{¢(¥ � ©(¬�¦¤�(��Ïp�,£¤±,�S¯³¦¤£¤¡a©®%� ¬%¡a£¤�§¦�hÆÍ·ÂçÄ© ( �M� �*�*� 3�,.� g 0¡�H�>��� � no� � Í�� ����� � �����C��� I ���^� � � � � �*� ��� I � : �����S���2,?�������a � ���!��,.� gfe � � ��� i µ#¯#®%¬%�S¥�¼�"aô % ¼Ã×��÷r·(Ç\£ �S¢(©³¨�·�×Tôaô$"³·R�_¡Ã¡%¥ ¦��©(¬�®g«_�S®%ý����Ñ®%£ ©(� ©(¬g®%� ¬a¡%£¤��¦¤�hÆY¶­´�Æ5®Tú ¡a£¤�§¦Ä´a·ÅçÄ© ( �M� �*�*� 3�,.� g 0P�H�

��� �Ûe ��,.�`3 � �����C���9s����*t#0*���*�¢��� I � : ���G�?���;,?�������Oa � ���I��,.� gìe � � ��� i µ(¯#®%¬%�S¥�¼�"­¼ % ¼Ã×#�³·r·ÚÇ#£ �S¢(©(¨t·2×Ñôaô­¼Ã·=Ð�©f�@Æu¯(£ ¡#�a�S¨]¶°¡Ã¡a¥�¦��©(¬Ù®%� ¬%¡a£¤�§¦�hÆÕ®%©(¨]��¦¤¥r��Æ~¯(� ��±Ñ®[¦¤� ¡a©(¥�¡a©]� �S®%£¤©Ãê� ©(¬�±,¡xÆ~¯(� �né³�§¦Ä´a·ÂçÄ© ( �M� �*�*� 3�,.� g 0��H�P��� �l¶ , �I��� � �����=����sj���Mt#0*���*����� I � : �����S���2,?�������a � ���!��,.� gfe � � ��� i µ#¯#®%¬%�S¥VÉaô³× % Éaôaø³·r·°Ç#£¤�,¢(©(¨�·4×Tôaô­æ÷o�_¡�¡a¥ ¦¤� ©(¬u®r«_�S®%ý�� �Ñ®[£¤©(��©(¬~®%� ¬a¡%£¤��¦¤�hÆ ¶*´�Æu®öú ¡a£ ��¦Ä´a·�L!�^�^��� : ���2,?�������3I � : ���G�?���2,?���#µ�×ö¼Ã×*þ ¼ ÿ! ¼aæ�� % ¼%ø­æ÷r·�Ç\£ �S¢(©³¨�·�×Tôaô%ô³·èÐ�©�®[¨#®%¯³¦¤���a���%�S£ ¥¤� ¡%©�¡%²Ú¦¤�(��¶ ¡�¡a¥ ¦�êĶ­´­ê2Æu®öú ¡a£ ��¦Ä´Ö®%� ¬%¡a£¤�§¦�hÆÍ·AçÄ© ( �M� ×

�*�*� 3�,.� g 0 �H�x��� ��e�£�� � �I��� � �����=��� I ���^� � � � � �*� ��� I � : ���G�?���;,?��������a � ���!��,.� gàe � � ��� i µ¯#®%¬a�,¥A×#"­¼ % ×%×Tɳ·�4·ÃÇ#¢�·Ë×Tôaôa¼Ã·ÚÔÃ�,¯#®%£®ð¦� ©³¬gÏl@ ²H£¤¡�Æ;ÏÚÏ=¶­´5£ �S� ®[¦�b�Ã��âÑ®[¦¤� ¡a©t· �\� �S� � ���.�C� / ,.��, � �%µ�ø#þkÉ ÿ! É­¼%ô %ÉaÉ$�³·â�·�Ç\¢³£¤¥ ¦Sµ6&³·ÅÔîðéÃ�%µÚ®%©(¨[â�·+ÔÃ��¯(¥¤�,£S·�×Tô%ø å ·{Ïp®%£¤�§¦Ä´aµ+±,� £¤±,¢(��¦¤¥Sµ+®[©(¨C¦�³��¯ ¡%��´�©(¡xÆ~�W®%� ê0¦¤��Æ~��(� �,£®%£ ±O�­´a· � ���.� �!: ���2, � ��� / i 0!� �!: 0 e � � ��� i µM×#�(þ�× ÿI ×ÑÉ % ¼$�÷â�·MÇ#¢(£ ¥ ¦Sµ¤&³·�&a®%±Oý*¥¤¡a©tµ�®%©³¨CÔ°·yÔ�Æ~��¦�t·u×Tô%ô³×a·�ç2Æu¯(£ ¡#�a�S¨�� �S®%£¤©³� ©(¬�¡[²o¥\¦V§Â²H¢(©³±,¦��¡a©(¥,·gçÄ©

( �*� �*�*� 3�,.� g 0>�H�V�.� �!¶ �����I��� � �����C���9sj���Mt#0*���*�)��� I � : ���G�?���;,?�������ta � ���I��,.� g e � � ��� i µ¯#®%¬a�,¥Vɳ×�� % Éa¼aæ÷x»·®�A®%£¤¨³©(�S£�®%©(¨)�4·pÆi�S£ £¤��¨#®³·Ù×Tôaøaô÷Öí»�(£ �S��¢(©ÃÊ#©(� ¥ �(�S¨�«�¡a£ ý�¥è¡a©C¦�³��¡a¯³¦¤��Æu®%�+¥ ¦¤¡a£®%¬%�±Ñ®%¯(®%±S�§¦Ä´�¡%²p©(�,¦Ä«�¡a£¤ý*¥,·O�C� ( � i 0-� �V �� ���.�C� -O� ��� µ\¼a¼ ×Tô%øaÉ % ×Tô%ô å ·È ·$�A�,©­¦� ���~®%©(¨�Ø�·�èy�§¦¤¦¤� �S¥�¦¡a©³�%·Í×Tôaôaô³·uí»�(�5£ ¡a¶(¢(¥�¦©(�,¥¤¥g¡[²_¦�³��@³ê0©(¡a£RÆ ®%� ¬a¡%£¤��¦¤�hÆ~¥S·�çÄ©

( �*� �*�*� 3�,.� g 0Z�H����� ��K%P �.� � �����C��� I ���^� � � � � �*� ��� I � : �����S���2,?�������ÿa � ���I��,.� gÞe � � ��� i µ¯#®%¬a�,¥A× % ×a×a·Ðè·�BA·t�¡a��¨(� ©(¬~®%©³¨{Æg·�B�¡%¦��·4×Tôaôaô³·_Ð��w��©(©(¡T«ËêĶ#®[¥¤�S¨B®%¯(¯(£ ¡­®%±O�B¦¤¡u¥ ¯ �,� ��� ©(¬�±S¡%£¤£¤�,±,¦¤� ¡a©�·

� � � ��,.� � a � ���I��,.� g µ(É å= ×^"�� % ×TÉ�"³·â�·s�¡a� ¨�Æ5®%©³©�µ�&³·#@4®[¥ ¦O®[¨�µ%®[©(¨�Ð�·�B�®[âS¶°¡a£¤¡#�°·Ú×Tôaô­¼Ã·$âB®Tú ¡a£¤�§¦Ä´Â¬­®[¦¤�S¥q��¥S·ö¬a�,©(�S£¤®%�a«_�,� ¬a�­¦¤�S¨¦�(£ �S¥ �(¡a� ¨Í¬­®[¦¤�S¥,· I � : �����?���;,?������� I � : ��� � ô$,.� i µ\¼ ¼$�$� % É�"$"³·�g·$�A¡a��¨(£¤�,� ±O��®%©(¨»è+·³èM�!�Ã��©�·�×Tôaø%ô³·5Ð �#®%£ ¨Ãêı,¡a£¤�~¯(£¤�,¨(� ±S®[¦��²H¡a£�®%���+¡a©(�?ê�«Ë®Ñ´Ù²H¢(©(±?¦��¡a©(¥S·çÄ© ( �M� �*�*� 3�,.� g 0>�H�w��� �Îe�£�� ��� i × ¶ ,.�M0!� � �����=��� / i�: ���#0!,.� : ��� e � � ��� i �H� I � : �����;,.� g µ¯#®%¬a�,¥�¼aæ % É­¼�·�g·��¡a� ¨(£ �S��±O��µ[Ô ·��¡a� ¨³«_®%¥ ¥¤�S£,µ%®[©(¨gÔ°·sâÖ� ±S®%� � ·Å×Tôaø$�÷�@V¡T«C¦¡�±,¡a©(¥ ¦¤£¤¢(±?¦x£®%©(¨³¡xÆ ²H¢(©(±?¦��¡a©(¥S·

�����G�I�����v�H�w�.� �\� 0*0#� � ,2���2,?���P�^��� I � : �����;,.� g¡� � � ��,.� � � i µ#É%É#þ å*ÿI �[ô­¼ % ø$"$�÷×Tø³×

Page 194: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

�g·]�A¡%� ¨(£ �S� ±O�tµ*Ø�·*Ø���¥®%©tµÃ®[©(¨uÐ�·a�w� ¬a¨³�S£¤¥ ¡a©�·Ë×Ñôaô­æ÷v�i©a�®%¡��ù¥lQO�NB»ê=èy�3ÆÈÆu®³·$x+� �S±?¦£ ¡a©(� ±È ¡a� ��¡3ï*¢(��¢hÆ ¡a© È ¡xÆu¯³¢³¦O®[¦¤� ¡a©(®%� È ¡xÆ~¯(� �néÃ��¦Ä´uízB�ô­æTêH"­æ�"÷�g·&�A¡a��¨(£¤�,� ±O��·�×Tôaô­æ�·»Ç#¡a¢(©(¨#®ð¦� ¡%©(¥�¡%²+±S£�´�¯³¦¡a¬%£®%¯(�­´Cþ Ç#£¤®%¬xÆ~�S©­¦¥V¡%²+®��_¡�¡aý ÿ ·ËÐ9�%®[� �W®[¶(� �®[¦»ñ]�]��{�|`}~}r�~�~���?�&EJTsG��Tïz�?�LS�E�¨aï�KPF�F�� KC�v�(ENR~}G���PGHSNG�}r©�MHK~�o� ñH�­ïOR÷�g· �A¡a��¨(£¤�,� ±O��· ×Tôaô%ø³· � �-3 � �I� � � i ���H� g �`�!��� i�ø ���M��+*��+-,.�/,10!�2, � ���M�#�H�M0ª����3ª��0 � �C3�� ×

�`����3�� : � � 0*0?·ÚÔï(£ � ©(¬a�,£�ê?pÚ�S£¤� ®%¬(µ³Ø��,«W�¡a£¤ý\·Ô ·��A¡%� ¨³«_®%¥¤¥ �S£�®%©(¨�â�·v���S� � ®%£¤�[·{×Tôaô��³·Z«M�S±,¦¤¢(£¤�u©(¡%¦¤�S¥è¡%©C±S£�´�¯³¦¡a¬%£®%¯(�­´%·ÍÐ9�[®%� � ®%¶(� ��®ð¦ñ]�]��{�|`}~}r�~�~�C¬C�]T�Sl�;���]TsGo��SNG���}��OTNSJM�T�}öï�EÑñ&EsM�}�{�K�{�SNM�T�}r�NQ��0ñ]�aïOR(·Ðè·O�£¤¡#�%�%µ\Ø�·�«M�§¦¤¦���S¥�¦¡a©(�[µ#®%©(¨ÖÆg·\ÔñO�*¢(¢³£ºÆu®%©(¥S·V×Tôaô$�÷��Â�S©(�,£®%�M±,¡a©��a�S£ ¬a�S©³±S�è£ �S¥ ¢(��¦¤¥�²H¡%£� ��©(�Ñ®%£Â¨³� ¥¤±,£¤�@Æu��©#®%©­¦A¢(¯°¨#®[¦�,¥S·�çÄ© ( �*� �*�*� 3�,.� g 0x�H�a��� �fe�� ����� � �����C��� I ���^� � � � � �*� ���I � : ���G�?���2,?�������&a � ���!��,.� gfe � � ��� i µ#¯#®%¬%�S¥A×��Ã× % ×Tøaɳ·í·�@4®%©³±S¡�±OýA®%©(¨Nr·PâB®%©(¥ ¡a¢(£,·_×TôaôÃ×a·¤«y�S®%£¤©³� ©(¬�Æ~¡a©(¡%¦¤¡a©(��ë*êÄ�~ÆÂØ�Ç�²H¡a£RÆg¢(� ®%¥M¡a©�¯(£¤¡�¨(¢(±?¦¨(� ¥�¦£ � ¶(¢³¦¤� ¡a©³¥S·ÅçÄ© ( �*� �*�*� 3�,.� g 0P�H�N��� �Ķ �����!�.� � �����C��� I ���^� � � � � �*� ��� I � : �����S���2,?�������a � ���!��,.� gfe � � ��� i µ#¯#®%¬%�S¥A×��%ô % ×Tôaɳ·í·�@4®%©(±,¡Ã±Oý\·Â×Tô%ô­¼Ã· e � �a� � : ��� � ô$,.� i �H�V� � ���I��,.� g �^��� : ���b��0P����3Z3 �*� ,10!,?���k�;� �*� 0<�.�C���o�C� Í��

� � 0!�;�I, � � � 3x� � ��3�0?·ÚÏ+��·ùÆg·(¦�³�S¥¤��¥Sµ�@�®%£`�[®%£ ¨ _V©(���%�S£ ¥¤��¦Ä´%·&³·­@�®%¥ ¦®%¨�·�×Tô%ø$�³· I � : �����?���;,?��������av, : ,.�?���2,?���C0��^��� / : ���.� ³P� ����� I ,.� � ��,.��0?·lâÖçÄí9Ï+£ �S¥¤¥,µÈ ®�Æg¶(£ � ¨(¬%�%µ*âÖÐ�·&³·F@�®%¥ ¦®%¨�· ×Tôaô�®(·]�i©ª¦�(��¥ � â,��¡%²A«_�,� ¬a�­¦¥�²H¡a£u¦�(£ �S¥¤�³¡a� ¨7¬­®ð¦�S¥,· / L ��� �������!�����N���³ ,10 � � � � �\� ���.� �!: ���;, � 0?µ��³þ É ÿI ®­ør® % ®­ô­¼�·Æg·6@4®%¢³¥¤¥¤���S£,·Ò×Tôaøaø³·7Ôï#®%±,�B�-�~±,� �S©­¦5� �Ñ®[£¤©(��©(¬C®[� ¬a¡a£ ��¦¤�hÆu¥,·fíx�S±O�³©(� ±S®%�\B��S¯°¡a£�¦U_ È Ô È êÈ BO«têÄøaøðê�¼�µ=_�©³���a�,£¤¥ ��¦Ä´�¡%² È ®%� �§²H¡a£¤©³�W®è®ð¦iÔî%©­¦O® È £¤¢(â[·Æg·F@V�S��Æè¶ ¡a��¨�µ9B�·VÔ�� ¡­®%©tµ4®%©³¨|â�·Ë��®%£RÆg¢³¦¤��·Z×Tôaô$"÷¯«M�Ñ®%£ ©(� ©³¬]©³�S¥ ¦¤�S¨ ¨(��� �S£ �S©(±,�S¥�¡%²� ©­¦�,£¤¥ �S±,¦¤� ¡a©�êıS��¡a¥¤�,¨Ö±,¡a©(±,�S¯³¦V±S� ®%¥¤¥ �S¥,· � � � ��,.� � a � ���I��,.� g µ\æ ×#�­æ % ×Ñô$�³·� ·�@�¡��°� ¨(� ©³¬(·�×Ñô$�aɳ·�Ï+£¤¡a¶#®[¶(� ����¦Ä´u� ©(�°±�¢(®%� �§¦� �,¥Ë²H¡a£�¥¤¢hÆ~¥�¡%²Å¶°¡a¢(©³¨(�S¨{£¤®%©(¨(¡�Ʋ�[®%£¤� ®%¶(� �,¥S·

�����G�I�����v�H�w�.� �\�\:x� �!, � ��� / �S���2,10!�2, � ��� � 0*0#� � ,2���2,?���#µ\æ%ø ×TÉ % É$"³·BA·+ç2Æu¯#®[¬a� � ®%âSâ,¡�®[©(¨fÐ�·+�w� ¨(¬a�,£¤¥ ¡a©�· ×Ñôaô��÷fÏ�³´�ËÏÚÏ9¢(©(���S¥¤¥yx9�#®%¥~¥¤¢³¶ �n鳯°¡a©(�,©­¦�W®[�±S��£¤±S¢³��¦¥ ¨(�,£®%©(¨³¡xÆu��âS��©(¬�¦¤�(� QO�NBZ� �3ÆÈÆu®³·ªçÄ© ( �M� �*�*� 3�,.� g 0k�H�Z�.� �[e�£�� ��� i × ´ ,.���.�

� �����=��� / i�: ���#0!,.� : ��� e � � ��� i �H� I � : ���G�2,.� g µ(¯#®%¬%�S¥�¼a¼�" % ¼a¼%ô³·BA·»ç2Æu¯#®[¬a� � ®%âSâ,¡(·�×Tô%ô­æ÷ª@�®%£¤¨Ãê0±S¡a£ �Ù¨(� ¥�¦£¤��¶(¢³¦¤� ¡a©(¥u²H¡a£�¥¤¡xÆ~�,«��(®[¦Í�#®%£¤¨2¯(£ ¡a¶(� �3Æu¥,· çÄ©

( �*� �*�*� 3�,.� g 0a�H�P��� �Îe ��,.�I� i ×0/ ,�ô$��� � �����C��� / i�: ���#0!,.� : ��� ¶ ������3����;,?���C0>�H� I � : ����� � �/ � , � � �*� µ(¯#®%¬%�S¥�æ%Éaø % ær®*æ÷&³·�&a®[±Oý�¥ ¡a©~®%©(¨õâ�· È £®^�a�,©�·Ë×Tôaô��³·v«M�S®%£¤©(��©(¬Â¥¤¯(®%£¤¥ �»¯ �,£¤±S�,¯³¦£ ¡a©(¥,·ÅçÄ© � 3 Í ��� �*� 0�,.� ´V� �G�`���

L!�^�^��� : ���2,?��� ( �*� �*� 0*0!,.� g / i 0!� �!: 0Vµaµ#¯#®%¬%�S¥9�­ær® % �$�$"³·×Tø­¼

Page 195: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

&³·A&a®[±Oý�¥ ¡a©�·]×Tôaô­æ�· e � � �N��� : ����, � 0!, � Í��-  �m��� Í�� ���!�����/, � ���;,?�����H� ¶ �����I, � �'������� i 0!,10U�X�: � � ��,.� � � � ���I��,.� g ��� � ��� i ����3����`� � �2, �*� ·èÏ+��·ùÆg·t¦¤�(�S¥ � ¥Sµ È ®[£¤©(�,¬a� �íâÍ�S� ��¡a©k_V©(���%�S£¤¥ ��¦Ä´%µÐ4¢(¬%¢(¥ ¦S·

&³·�&a®%±Oý*¥¤¡a©t·{×Tôaô���·uÐ�©C�-�~±,� �S©­¦ZÆu�3Æg¶°�S£¤¥ �(� ¯�êX±*¢(�S£�´�®%� ¬%¡a£¤�§¦�hÆ�²H¡a£A� �S®%£¤©(��©(¬ÍÆÂØ4Ǫ«���¦¤�£¤�,¥¤¯°�S±,¦�¦¡�¦�(��¢(©³��²H¡a£RÆ ¨(� ¥�¦£ � ¶(¢³¦¤� ¡a©t·U�������I�������H� I � : ���G� � ������3 / i 0!� �!: / � , � � �*� 0?µæaæ ®(×^® % ®$®�"³·È ·6&%�V®%©(¨DÔ ·�âB®³·w×Tô%ô��÷ È ¡xÆg¶(��©#®[¦¤� ¡a©(¥è¡%²4«��Ñ®%ý=±,�W®%¥ ¥¤�§Ê#�S£ ¥S·fL*nlnln e �`���C0-� � �;,?���C0d���´<� ���`��� ´<� � £ ���Mt#0?µ#ø#þ�× ÿ! É­¼ % ®*¼�·&³·��A®%�(©�µ&�r·��A®%� ®%��µ³®%©(¨BØ�·�«M� ©(� ®%��·V×Tôaøaø÷_í»�(��� ©C¶#¢³�S©(±,�è¡%²��%®[£¤�W®[¶(� �,¥V¡%©��_¡�¡a���Ñ®%©�²H¢(©(±nê¦��¡a©(¥S·�çÄ© ( �M� �*�*� 3�,.� g 0N�H�9��� �VP�· �.� � �����=��� / i�: ���#0!,.� : ��� ¶ ���G��3����2,?���C0N�H� I � : ����� � �/ � , � � �*� µ(¯#®%¬%�S¥9�aø % ø�"³·â�·A�Â�S®%£¤©(¥�®[©(¨Ââ�·A«M��·.×Tôaô%ɳ·«y�S®%£¤©³� ©(¬���©�¦�(��¯(£ �S¥ �S©(±,�Ö¡%²^Æu®%� ��±S��¡a¢(¥A�S£ £¤¡a£ ¥S· / L ���

�����G�I�����v��� I � : �����;,.� g µ(¼a¼(þ2® ÿ! ø$"�� % ø%É��÷â�·��A�S®%£¤©³¥�®%©(¨T«+·6p�®%� � ®%©­¦Ñ·�×Tôaô�®(· È £ ´�¯³¦¤¡a¬a£¤®%¯(�(��±{� �@Æu�§¦O®[¦¤� ¡a©(¥è¡a©ª� �S®%£¤©³� ©(¬¢�_¡Ã¡%� �Ñ®[©²H¡a£ºÆè¢(�W®%�i®%©(¨�Ê#©(��¦¤�A®%¢³¦¤¡xÆ5®ð¦O®³·��������I�����v�H�N��� �\�gI�� µ�®(×*þ�× ÿ! ��� % ô­æ÷â�·��Â�S®%£¤©(¥V®%©³¨ _�·Cp�®%â,� £®[©(��·�×Tôaô�®(· � �j,.���;�M�-3�� � �;,?�����H� � � : �����S���2,?��������� � ���I��,.� g �.� � ��� i ·âÖçÄíÒÏ+£¤�,¥¤¥,µ È ®dÆg¶(£¤��¨(¬a�[µ*âÖÐè·â�·v�A�S®%£¤©³¥Sµ�â�·v«y� µR«+·pÏÚ�§¦¤¦Sµ+®%©(¨)«+·�p�®%� � ®%©­¦Ñ·�×Tôaø��%®Ã·j�i©�¦�³��� �S®%£¤©#®[¶(� ����¦Ä´{¡[²9�_¡Ã¡%� �Ñ®[©²H¡a£ºÆè¢(�W®%�[·yçÄ© ( �M� �*�*� 3�,.� g 09�H���.� � ´ ,.� � � �*� ���.� � �����C��� / i�: ���#0!,.� : ��� e � � ��� i �H� I � : ×

���G�2,.� g µ#¯#®%¬%�S¥�¼%ø­æ % ¼%ô­æ÷â�·¤�Â�Ñ®%£ ©(¥Sµÿâ�·¤«M��µ�«Ú·xÏ+��¦ ¦Ñµp®%©(¨g«+·�p�®[� �W®[©*¦S·Ö×Ñôaø��[¶�·UBV�,±S�S©­¦è£¤�S¥ ¢(��¦¤¥�¡%©E�_¡�¡a� �S®%©Ù±S¡a©�ê±S�,¯³¦�� �Ñ®[£¤©(��©(¬(·ÂçÄ© ( �*� �*�*� 3�,.� g 0¡�H�>�.� �ɶ ���G�I�.�dLI��� � �I�����2,?�������Ns����*t#0*���*����� � � � ��,.� �a � ���!��,.� g µ³¯#®%¬a�,¥�É%É�� % É­æ%¼Ã·â�·��Â�Ñ®%£ ©(¥Sµtâ�·�«M��µ\®%©(¨�«+·�p�®[� �W®[©*¦S·4×Tô%ô�®(·�«M�S®%£¤©(��©(¬��_¡Ã¡%� �Ñ®[©Í²H¡a£RÆg¢(� ®%¥S·9�������I�����R�H�<�.� �

�gI�� µ�®³×*þ;� ÿI ×T¼%ôaø % ×ÑÉ­¼%ø³·â�·��A�S®%£¤©³¥S·�×Tôaôaø÷Äxÿ�u±,� �,©*¦A©³¡a� ¥ �?ê0¦¤¡a� �,£®%©­¦�«y�S®%£¤©³� ©(¬5²H£ ¡xÆ�¥ ¦®[¦��¥ ¦��±Ñ®%�¤±*¢(�S£ � �,¥S·>�����G�I�����

�H�w��� �\�gI�� µ=®*æ³þ;� ÿI ôaø%É % ×#"$"��³·BA·��A�(®%£¤¨(¡%©�·i×Tôaôr®(·��i©Ö¢(¥¤��©(¬~¦¤�(��Ç\¡%¢(£¤���S£»¦£¤®%©(¥ ²H¡%£ºÆ�¦¤¡u���Ñ®%£ ©Ö¨(��¥Hú ¡a��©­¦4ÆÂØ�Ç_·�L!�^�^��� : � ×

�;,?��� ( �M� �*� 0*0!,.� g a � �2� � �M0?µ�®­ô ¼Ã×Tô % ¼a¼a¼Ã·â�·��Â�#®%£¤�§¦¡a©³¡��°·Â×Tô%ô­æ÷ È £ ´�¯³¦¤¡a¬a£®[¯(�(� ±�� ¡T«��S£�¶ ¡a¢³©(¨(¥�²H¡%£����Ñ®%£ ©#®%¶(��� ��¦Ä´5¡%²l�_¡�¡a���Ñ®%©�²H¢(©(±nê¦��¡a©(¥5¡%©D¦�(�Ö¢(©(�§²H¡a£ºÆ̨(��¥ ¦¤£¤� ¶³¢³¦��¡a©�·7�����G�I�������H� I � : ���G� � �j����3 / i 0!� �!: / � , � � �*� 0?µæ�" �$"$" % �³×#"³·&³·��A�b��� ©(�,©�µ�â�·x��®%£RÆg¢³¦¤��µ+®%©(¨�Ïp·�Ð�¢(�S£,·�×Tôaô$�÷Öí»�³��Ïp�S£ ±S�,¯³¦£ ¡a©]®%� ¬a¡%£¤��¦¤�hƸ��¥S·x�w� ©Ãê©(¡T« � ��©(�Ñ®%£R��¥,·���¡a¬­®%£ ��¦¤�hÆu��±+Æ~� ¥�¦O®%ý[�V¶°¡a¢(©³¨(¥�«��(�,©5²H�,«ª� ©(¯(¢Ã¦6�[®%£ �W®%¶³� �S¥Ú®[£¤�V£¤�S���°�[®%©­¦Ñ·

� �I�;, G�� ,2����LI��� � �.�/, g�� � �*� µ³ô��(þ�×?ê�¼ ÿI É­¼%æ % É�®­É÷×TøaÉ

Page 196: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

Ðè·��Â� �b�[®%©(¥Ú®%©(¨xB�·ÃÔÃ�,£`�%�S¨(��¡(·Ë×Tôaô%ô³·R�_¡�¡a¥�¦� ©³¬è®%©³¨5�#®%£ ¨Ãêı,¡a£¤��¥ �,¦¥,·ÚçÄ© ( �M� �*�*� 3�,.� g 0N�H�O�.� �¶ ���I�2, � ��� � �����C��� / i�: ���#0!,.� : ��� ¶ ������3����;,?���C0\�H� I � : ���G� � � / � , � � �*� µ­¯#®%¬a�,¥l�­¼r® % �aÉaɳ·Ðè·l�Â�����[®%©(¥r®[©(¨fBA·�ÔÃ�,£`�a�,¨(� ¡³·ª¼�"�"³×a·)«M�Ñ®%£ ©(� ©³¬�ÆÂØ�ÇÒ� ©=¦�@Æu��¼�½¾�¹ ¿ Ò`îkÁ à ·.íp¡�®%¯(¯°�Ñ®%£���©

( �*� �*�*� 3�,.� g 0P�H�w��� �Æe ��,.�I� i × e ��,.�`3 � �����C��� / i�: ���#0!,.� : ��� e � � ��� i �H� I � : �����;,.� g ·«+·6�A¢³±S�S£¤®³µËÐ�·�âB®%£¤±O�³�,¦¤¦¤�§ê�Ôï#®%±,±Ñ®�Æ~�S� ®³µË®%©³¨Câ�·�Ï+£¤¡%¦®%¥¤¥ ��· ×Tôaô�®(·��i©f���Ñ®%£ ©(� ©(¬*Æ~¡a©(¡[ꦡa©³�rÆÂØ4Çf²H¡%£ºÆg¢(� ®%��¢(©(¨(�,£Â¢(©(��²H¡%£ºÆ�¨(� ¥�¦£ � ¶(¢³¦¤� ¡a©³¥S·NLI�^�^��� : ���;,?���g����3 I � : ���G�?���;,?���#µ×a×#" ø�® % ô­æ÷x»·��¢(¥¤�(��� �!�Ã�§¦â[·»×Tôaô��÷+Ð ¥ ��Æ~¯(� �®[� ¬a¡a£ ��¦¤�hÆ ²H¡a£�� �Ñ®[£¤©(��©(¬aº�þH� ¡a¬AY ÿ ê0¦¤�S£RÆóÆÂØ4Ç�·6L!�^�^��� : � ×

�;,?��� ( �M� �*� 0*0!,.� g a � �2� � �M0?µ��³×*þ2� ÿ! ¼%øaô % ¼%ô­¼Ã·«+·6«y�!��� ©�·Ò×Tô%ø$�³·DÐ9�a�,£®%¬a�B±Ñ®%¥ �Ö±,¡xÆu¯³� �,¦¤�Ö¯³£¤¡a¶(���"Æ~¥S· / L ��� �����G�I��������� I � : �����;,.� g µ×öæ(þ�× ÿ! ¼%ø­æ % ¼%ø$�³·Ø�·�«M� ©(� ®%�V®%©(¨2Ø�·_Ø�� ¥¤®%©�· ×Tô%ô$"³·ÒÐ4¯³¯(£¤¡ÑéÃ��Æu®[¦�Ö� ©(±,� ¢(¥ � ¡a©�êÄ�?éñ,� ¢(¥ � ¡a©�· I � : +-,.�����H���I, � �%µ×#"#þ�® ÿ! É�®­ô % É$�­æ÷Ø�·�«M� ©(� ®%��µ=g·�âB®[©(¥¤¡a¢³£SµM®%©³¨�Ø�·tØ�� ¥¤®%©�·�×Tôaôaɳ· È ¡a©(¥�¦O®%©­¦A¨(�,¯³¦��±,� £ ±S¢(�§¦¥Sµ Ç#¡a¢(£ � �S£�¦£¤®%©(¥�ê²H¡a£ºÆó®%©(¨�� �S®%£¤©#®[¶(� ����¦Ä´a·6�����G�I�����v�H�N�.� �\�gI�� µ�®$"#þ É ÿI ��"�� % �­¼r"³·Ø�·l«M�§¦¤¦���S¥�¦¡a©(�[·7×Tôaøaø÷�«M�S®%£¤©(��©(¬m±*¢(� ±Oý*��´C«��(�S©f� £¤£ �S���°�[®%©­¦~®[¦¤¦¤£¤��¶(¢³¦�,¥~®%¶ ¡%¢(©(¨ ®Ù©(�,«� ��©(�Ñ®%£�ê0¦¤�(£¤�,¥¤�(¡a��¨�®%� ¬%¡a£¤�§¦�hÆÍ· � � � ��,.� � a � ���I��,.� g µ\¼ ¼%ø­æ % ɳ×Tø³·Ø�·q«y�§¦¤¦¤� �S¥�¦¡a©³�%·~×Tôaøaôa®Ã·èÇ#£¤¡�Æ ¡a©³� � ©³�覡�¶#®ð¦±O��� �Ñ®[£¤©(��©(¬(·AçÄ© ( �M� �*�*� 3�,.� g 0��H���.� � / �*� ����3

� �����=���9s����Mt#0*���*�E��� I � : �����S���2,?�������&a � ���I��,.� g e � � ��� i µ(¯(®%¬a�S¥�¼��%ô % ¼%ø�®³·Ø�·�«M��¦ ¦���S¥ ¦¤¡a©(�[·_×Tôaø%ô%¶�· � ,10!�?��t � +!������3�0\����3��1� g ���!,.�.� : , � �/,.� � ��� × ����� � 0*�����b3P� � ���I��,.� g ��� g � ×

�I,.��� : 0?·ÚÏ+��·ùÆg·(¦�(�,¥¤��¥Sµ�_�©³���a�,£¤¥ ��¦Ä´�¡%² È ®%� �§²H¡a£¤©³�W®è®ð¦�Ô³®%©­¦O® È £¤¢(â[·Ø�·�«y�§¦¤¦¤� �S¥�¦¡a©³�%· ×Tô%ô³×a·»B��S¨³¢(©(¨#®%©­¦�©(¡a��¥ ´á®[¦¤¦¤£¤� ¶³¢³¦�,¥Sµr®[¦ ¦£ � ¶(¢³¦¤�=�S£ £¤¡a£ ¥Sµ�®%©(¨ ��� ©(�S®%£�ê¦�(£ �S¥ �(¡a� ¨5� �Ñ®[£¤©(��©(¬�¢³¥¤� ©³¬è�w� ©(©³¡ö«A·�çÄ© ( �*� �*�*� 3�,.� g 0w�H�O��� �·¶ ���G�I�.� � �����C����s����Mt#0*���*�

��� I � : ���G�?���2,?�������&a � ���!��,.� gfe � � ��� i µ(¯#®[¬a�S¥A×^®~� % ×öæ��³·Ô ·�«M¡aýð®�ÆÍ·Ú¼�"$"Ã×a·+Ïp�,£¤¥ ¡a©#®%�y±S¡�ÆÈÆg¢³©(� ±S®[¦��¡a©�·Ïp·q«y¡%©(¬(·~×Tôaô�®(·V@�®%��²H¥ ¯#®%±S�r���Ñ®%£ ©(� ©³¬(µt��� ©(�S®%£4¯(£ ¡a¬a£¤®�ÆÈÆ~� ©³¬(µt®[©(¨�©(¡a©hÆu®%��� ±S��¡a¢(¥V¨(� ¥�¦£ � ¶(¢Ãê¦��¡a©(¥S·6LI�^�^��� : ���;,?��� ( �M� �*� 0*0!,.� g a � �2� � �M0?µ°æÃ× ¼r®­æ % ¼aæ�"÷Ïp·v«M¡a©(¬(·=×Tô%ô­æ÷�i©�¦�(�5¥®�Æ~¯(� �u±S¡xÆ~¯(���?éÃ��¦Ä´�¡%²�Ï Ð È � �S®%£¤©³� ©(¬Ö�#®%��²H¥ ¯#®%±,�S¥�®%¬­®[� ©(¥�¦è¦�³�¢(©(�§²H¡a£ºÆ;¨(� ¥�¦£¤��¶(¢³¦¤� ¡a©�·AL*nlnln e �`���C0-� � �;,?���C0<��� ´<� ���`��� ´<� � £ ���Mt#0?µ��#þ;� ÿ! ×öæaæ�� % ×öæaæ%ô³·� ·tâB®a®%¥ ¥Â®%©(¨ì�r·\íp¢(£¤®%©�µÅ×Ñôaô�®(·N�<� £ �°��0!� � ���¢�Z�.��� � 0*�����b3 g ��� � � � ���I�l¼µtçÄ©ÙÔ ·�@4®[©(¥¤¡a©tµ��·�Æi£®%¥�¦O®%� µ�®%©(¨BA·qB����%�S¥ ¦Sµy�S¨(�§¦¡a£ ¥Sµ I � : �����S���2,?��������a � ���I��,.� gÝe � � ��� i ����3 ´ ���2�G�`���a � ���!��,.� g / i 0!� �!: 0   i����/� :x� L  gI ���C0!�2�J��,.���.0�����3 ( �*�#0X� �*� �.0?µË¯#®[¬a�S¥5Éaø³× % ®³×^®(·ÖâÖçÄíÏ+£¤�S¥ ¥S·� ·$âB®a®%¥ ¥g®%©³¨�â�·y��®[£ºÆg¢³¦¤��·{×Tô%ôaø³·/xÿ�~±S���S©­¦g���Ñ®%£ ©(� ©³¬�«��§¦��Ã��£ ¦¤¢#®%�Ŧ�³£¤�S¥ �(¡a��¨�¬­®[¦¤�S¥S·

L!�^�^��� : ���2,?���m����3 I � : �����S���2,?���#µ�×^®(×­þ�× ÿI É��[ø % Éaø$�÷×Tø�®

Page 197: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

r·�âB®%©(¥ ¡a¢(£Â®[©(¨ìâ�·�Ïp®%£¤©(®%¥S·r×Tô%ôaø³·N«M�Ñ®[£¤©(��©(¬�±S¡%©ðú ¢(©(±?¦� ¡%©(¥4«���¦¤�{©(¡a��¥¤��¢(©(¨(�,£Â¯(£¤¡�¨(¢(±?¦¨(� ¥�¦£ � ¶(¢³¦¤� ¡a©³¥S·lLI�^�^��� : ���2,?��� ( �M� �*� 0*0!,.� g a � �2� � �M0?µ��aø#þ�® ÿ! ×Tøaô % ×Tô$�³·r·�âB®%©³¥¤¡a¢(£,·�×Tôaôaæ÷ÖÐ�©)º�þ�Y Ý�ÞÄßðÝ�ÞÄß%¿ ÿ � �S®%£¤©³� ©(¬B®%� ¬a¡%£¤��¦¤�hÆ�²H¡a£èÆÂØ4Ç2¢(©(¨(�,£è¦�(�5¢(©(��²H¡%£ºÆ¨(� ¥�¦£ � ¶(¢³¦¤� ¡a©t·������G�I�����v�H� I � : ����� � �<����3 / i 0!� �!: / � , � � �*� 0?µ#æ�" ær®­É % æaæ�"³·â�·»âÍ� ©(¥ ý*´2®[©(¨ Ô ·»Ï�®[¯ �,£ ¦Ñ·�×Tô$�aø³· (z� � �*� ���2�M���C0   ���4,.���2�M�-3�� � �2,?���D�X� � � : �����S���2,?�������g�� � :x� �;� i ·�âÖçÄíÒÏ+£¤�,¥¤¥,µ È ®dÆg¶(£¤��¨(¬a�[µ*âÖÐè·Ô ·]âÖ¢³£¤¡a¬­®Ã·_×Tô���×a· e ��� � 0*�����b3V�1� g , � ����3P,.�.0\�!�����/, � ���2,?���C0?·x�w��� �,´­ê0çÄ©*¦¤�S£ ¥¤±S���S©(±,�%µ�Ø��?«f�¡a£ ý°·Ø�·ÑØ�� ¥¤®%©A®%©(¨AÐ�·S�w� ¬a¨³�S£¤¥ ¡a©�·�×Tôaô�®³·�@4®%£ ¨(©(�S¥ ¥¤�%�S£ ¥¤¢(¥x£®%©³¨(¡xÆ~©(�S¥ ¥S·l�������I�����C�H� I � : ����� � �

����3 / i 0!� �!: / � , � � �*� 0?µ=®aô ×-®­ô % ×#�$�÷Ðè·�Ø�¡#��� ý[¡��p·Ò×Tô$�­¼Ã·W�i©f±S¡a©��a�,£¤¬a�,©(±S��¯³£¤¡�¡%²H¥u¡%©D¯°�S£¤±,�S¯³¦¤£¤¡a©³¥S·2çÄ© ( �*� �*�*� 3�,.� g 0k�H�Z�.� �/ ir: ���#0!,.� : ��� � ����� �!: ���2, � ��� e � � ��� i �H� � �G�X� : ���?�%µ=�a¡a��¢hÆu��Q4ç¤çOµ#¯#®[¬a�S¥9�³×öæ % �­¼a¼Ã·x»·��iý%¡a�Ä�ù©(��¥¤�(©³� ý%¡#�[®³·»×Tô%ø­¼Ã·6�i©Í¦¤�(�A� ©C¶(¢(�S©(±,��¡[²�©(�,¬­®[¦¤� ¡a©(¥»¡%©Í¦¤�(�A±S¡xÆ~¯(� �néÃ��¦Ä´�¡[²Å®r£¤�S®%�§ê� âS®[¦��¡a©�¡[²iÆu¡%©(¡%¦¡%©(�6�_¡Ã¡%� �Ñ®[©�²H¢(©(±?¦��¡a©(¥x¶*´A²H¡a£RÆg¢(� ®%¥y¡%²#¶°¡a¢(©(¨(�,¨g¨(�,¯³¦��þH� ©PBV¢(¥ ¥¤� ®%© ÿ ·

�g� �X�-3 iõ³ ,10*t�� � �?� � �����/,¹¸#� µ#É%ø ��® % ø$"³·â�·=�¯³¯ �,£�®%©(¨�Æg·G@4®%¢(¥ ¥¤���S£S·»×Ñôaô³×a· È ®[� ±S¢³�W®[¦¤� ¡a©u¡%²M¦�(�Â���Ñ®%£ ©(� ©³¬è±S¢³£`�a�A¡%²¤�_®T´%�S¥�¡a¯³¦¤��Æu®%�±S� ®%¥¤¥ ��Ê#±S®[¦��¡a©~®%� ¬a¡%£¤��¦¤�hÆ ²H¡a£Ú� �Ñ®[£¤©(��©(¬�®A¯°�S£ ±S�,¯³¦£ ¡a©5«���¦¤��©(¡a� ¥ �%·ÅçÄ© ( �*� �*�*� 3�,.� g 0\�H�O�.� �¶ �����I��� � �����C���zs����Mt#0*���*�E��� I � : ���G�?���2,?�������&a � ���!��,.� gfe � � ��� i µ(¯#®[¬a�S¥O�aæ % ø��÷çO·MÏp®%£¤¶°�S£ £ ´a·~×Tôaôr®(· I ,.� � ��,.� � � : ��� � ô$,.� i ����3Z� � ���`���v� � � £ ���*t#0?·dâÖçÄí�Ï+£¤�S¥ ¥Sµ È ®�Æg¶³£¤� ¨³¬a�%µâÖÐè·«+·\Ï+��¦¤¦�®%©³¨�«+·=p�®%���W®%©­¦Ñ·�×Tô%øaø³· È ¡xÆ~¯(¢³¦O®ð¦� ¡%©#®%������Æ~��¦O®ð¦� ¡%©(¥�¡%©Ö���Ñ®%£ ©(� ©(¬r²H£ ¡xÆó�?é³®�Æ~¯(���S¥S·

�����G�I�����v�H�w�.� �\�gI�� µ#Éaæ(þ2® ÿI ô��­æ % ôaør®(·â�·6BV®%¶(� ©t·á×Tô��%ô÷ªÆi� ¬%��¦O®[� � â,�S¨�¥ � ¬a©(®[¦¢(£ �S¥5®%¥u��©­¦£®[±,¦O®[¶(� �Í®%¥�²k®%±,¦¤¡a£¤��âÑ®[¦¤� ¡a©t·fíx�S±O�(©³� ±Ñ®[�

BV�,¯ ¡%£ ¦²âÖçÄí9y�« È Ô=y[ízBËê�¼�×ö¼Ãµ*âB®[¥¤¥®[±O��¢³¥¤�,¦ ¦¥içÄ©³¥ ¦�§¦¢³¦¤�A¡%²�íx�S±O�³©(¡a� ¡%¬%´a·BA·�BV�b�a�,¥ ¦Ñ·�×Tô%ø��÷l«M�Ñ®%£ ©(� ©³¬r¨(�S±,� ¥¤��¡a©�� ��¥ ¦¥,· � � � ��,.� � a � ���I��,.� g µ\¼³þ É ÿI ¼a¼[ô % ¼r®��÷Ç_·�B�¡a¥ �S©*¶(�W®ð¦¤¦Ñ·»×Ñô$�­¼Ã· ( �I,.� � ,K��� � 0��H�N� � ���M�-3 i ��� : , � 0?·�Ôï³£¤� ©³¬a�S£�êSpÚ�S£ �W®%¬³µ³ØV�,«D�¡a£ ý\·r·�Ô³®%ýð®%�p®%©(¨�Ð�·AâB®%£ ¢(¡aý𮳷A¼r"$"$"³·N«M�S®%£¤©(��©(¬ÁÆ~¡a©(¡%¦¤¡a©(��� ¡a¬[ê�¦�,£ºÆ�ÆÂØ4Ç]²H¡a£ºÆg¢³�W®%¥V¢(©(¨(�,£¦�(�A¢(©³��²H¡a£RÆ�¨(��¥ ¦£ � ¶(¢Ã¦� ¡%©�· e � � ��� i �H� I � : ���G�2,.� g / i 0!� �!: 0?µ#É%É ×#� % Éaɳ·BA·ÅÔñO�(®%¯(� £ ��®%©(¨¢g·+ÔÃ��©(¬a�,£S·�×Tô%ôaø³·Bç2Æ~¯(£ ¡��%�S¨.¶°¡Ã¡a¥�¦��©(¬{®%��¬a¡a£ ��¦��Æu¥A¢(¥ � ©(¬�±,¡a©³Ê#¨(�,©(±S�n꣮[¦¤�S¨{¯³£¤�S¨³� ±,¦¤� ¡a©³¥S·�çÄ© ( �M� �*�*� 3�,.� g 0¡�H�<�.� � no� � Í�� ���.� � �����C��� I ���^� � � � � �*� ��� I � : ��� ×

�S���2,?�������ta � ���I��,.� gìe � � ��� i µ(¯(®%¬a�S¥Vø$" % ô³×a·BA·_Ô�±O�#®%¯(��£¤�%µ6r·�Ç#£¤�S¢³©(¨�µ»Ïx·��Ë®[£ ¦���,¦ ¦Ñµ�®%©(¨f� ·o«M�S�[·Ò×Tôaôaø³·f�_¡�¡a¥ ¦¤� ©(¬Ù¦�(��Æu®%£¤¬a��© ®©(�,« �?éï(� ®%©#®[¦¤� ¡a©f²H¡a£5¦�³�{�°� �S±?¦���%�S©(�,¥¤¥Í¡%²N�a¡[¦� ©³¬­Æu�?¦�(¡�¨(¥S· � �������K0m�H� / �S���2,10!�;, � 0?µ¼��#þ æ ÿ! ×#�­æÃ× % ×#�aø$�÷

×Tø­æ

Page 198: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

BA·MÔñO�#®[¯(� £ �%·5×Ñôaô$"³·Aí»�(��¥ ¦¤£¤�,©(¬%¦�Ù¡%²Ú«��Ñ®%ýB� �Ñ®[£¤©#®%¶³� � �§¦Ä´a· � � � ��,.� � a � ���I��,.� g µMæ(þ ¼ ÿ! ×Tô�� %¼a¼$�÷BA·­ÔñO�#®[¯(� £ �%·_×Ñôaôaôa®³·pÆi£¤�§²F¦��©(¬�¬a®�Æu�,¥S·xçÄ© ( �M� �*�*� 3�,.� g 09�H���.� �ãe�£�� � �I�.� � �����C��� I ���^� � � � � �*�

��� I � : ���G�?���2,?�������&a � ���!��,.� gfe � � ��� i µ(¯#®[¬a�S¥A×a×^® % ×ö¼r®(·BA·»ÔñO�#®[¯(� £ �%· ×Tôaôaô%¶t·�í»�³�S¡a£ �,¦��±Ñ®%�O�Ã���,«�¥�¡%²�¶°¡Ã¡%¥ ¦��©(¬(· çÄ© ( �*� �*�*� 3�,.� g 0m�H� ��� �Þe�� ���.�

L!��� � �I�����;,?������� I ���^� � � � � �*� ��� � � g ���!,.�.� : , � a � ���I��,.� gìe � � ��� i µ³¯#®%¬a�,¥�×T¼ % ¼r®(·â�·°ÔñO��Æu�§¦¤¦Ñ·_×Tôaôaø³·+çĨ³�S©­¦�§Ê#±Ñ®[¦¤� ¡a©�±,£¤�§¦�S£ �W®r®%©³¨Í��¡T«_�,£V¶°¡a¢(©³¨(¥»²H¡a£»Ïp�,£¤±,�S¯³¦¤£¤¡a©Ãê0� ��ý%�A� �S®%£¤©Ãê� ©(¬�£ ¢(� �,¥S· ´V� �G�`��� I � : ���G�?���2,?���#µ�×#" ¼%É­æ % ¼aæ�"³·BA·�ÔÃ�,£`�%�S¨(��¡(·�×Ñôaôaôa®³· È ¡xÆ~¯(¢³¦®[¦��¡a©#®%�y¥¤®�Æu¯³� ��±S¡xÆ~¯(� �néÃ��¦Ä´Í®%©(¨{®[¦¤¦¤£¤� ¶³¢³¦�nêÄ�-�~±,� �S©­¦�� �S®%£¤©Ãê� ©(¬³·ÅçÄ© ( �M� �*�*� 3�,.� g 0<�H�O��� �Åe ��,.�I� i × ¶ ,.�M0!� � �����=��� / i�: ���#0!,.� : ��� e � � ��� i �H� I � : ���G� ×

,.� g µ(¯#®%¬%�S¥\��"³× % ��×#"³·BA·\ÔÃ�S£½�a�,¨(� ¡(·»×Ñôaôaô%¶�·6�i©ÍÏ Ð È � �S®%£¤©(��©(¬g¢(¥ � ©(¬g�w��©(©(¡T«Aµ\Ïp�,£¤±,�S¯³¦¤£¤¡a©�µ\®[©(¨Í®~Ïx�S£ ±S�S¯Ã¦£¤¡%©Ãê� ��ý%�Ú®%��¬a¡a£¤�§¦�hÆÍ·�çÄ© ( �*� �*�*� 3�,.� g 0��H�l��� �ne�£�� � �I�.� � �����=��� I ���^� � � � � �*� ��� I � : �����S���2,?�������a � ���!��,.� gfe � � ��� i µ#¯#®%¬%�S¥�¼%ô$� % É$"��÷BA·�ÔÃ�,£`�%�S¨(��¡(·i¼r"$"$"a®³· È ¡xÆ~¯(¢³¦®[¦��¡a©#®%�y¥¤®�Æu¯³� ��±S¡xÆ~¯(� �néÃ��¦Ä´Í®%©(¨{®[¦¤¦¤£¤� ¶³¢³¦�nêÄ�-�~±,� �S©­¦�� �S®%£¤©Ãê� ©(¬³·��������I�������H� I � : ����� � �<����3 / i 0!� �!: / � , � � �*� 0?µC�$"#þ�× ÿ! ×#�³× % ×��%ø³·BA·_Ô��S£`�%�S¨(��¡(·2¼�"$"�"%¶�·ªÏ°Ð È ®%©#®%��¡a¬a¢(�,¥u¡[²ÂÏp�,£¤±,�S¯³¦¤£¤¡a©w®%©³¨f�w� ©³©(¡T«¾���W®Ù¶ ¡�¡a¥ ¦¤� ©(¬Ù¦�³�Æ5®[£¤¬a��©�·�çÄ© ( �*� �*�*� 3�,.� g 0'�H���.� � e ��,.�I� �*� ����� � �����C��� I ���^� � � � � �*� ��� I � : �����S���2,?�������a � ���!��,.� gfe � � ��� i µ#¯#®%¬%�S¥A×^®­ø % ×öæ$�÷BA·+ÔÃ�,£`�a�,¨(� ¡³·C¼r"$"³×T®³·�i©=���Ñ®%£ ©(� ©(¬ªÆu¡%©(¡%¦¡%©(��ÆÂØ�Çá¢(©(¨(�,£g¯(£¤¡�¨(¢(±?¦�¨(� ¥�¦£ � ¶(¢³¦¤� ¡a©³¥S·{íx¡®%¯(¯°�Ñ®%£���© ( �*� �*�*� 3�,.� g 0�H�j�.� � ¶ �����!� �*� ���.� � �����C��� I ���^� � � � � �*� ��� I � : �����S���2,?�������a � ���!��,.� gfe � � ��� i ·BA·_Ô��S£`�%�S¨(��¡(·2¼�"$"Ã×Ѷ�·ªÏ°Ð È ®%©#®%��¡a¬a¢(�,¥u¡[²ÂÏp�,£¤±,�S¯³¦¤£¤¡a©w®%©³¨f�w� ©³©(¡T«¾���W®Ù¶ ¡�¡a¥ ¦¤� ©(¬Ù¦�³�Æ5®[£¤¬a��©�·Åíx¡u®[¯(¯ �S®%£�� © � � � ��,.� � a � ���!��,.� g ·BA·�ÔÃ�,£`�%�S¨(��¡(·A¼�"$"Ã×ѱ%·iÔ�Æu¡�¡%¦¤�B¶°¡Ã¡a¥�¦��©(¬�®%©³¨{� �S®%£¤©(��©(¬�«���¦�êÆ5®%��� ±,� ¡a¢(¥�©(¡%� ¥¤�[·4íx¡�®%¯(¯°�Ñ®[£� © ( �M� �*�*� 3�,.� g 0x�H���.� �l¶ ���G�I� �*� ����� � �����=��� I ���^� � � � � �*� ��� I � : �����S���2,?��������a � ���I��,.� ge � � ��� i ·@�·�ÔÃ�,¢(©(¬(µA@�·ÅÔ�¡xÆu¯°¡a��� ©(¥ ý*´%µ�®[©(¨=Ø�·xí»� ¥ �*¶*´%·C×Tô%ô­¼Ã·ÖÔ�¦O®ð¦� ¥�¦��±Ñ®%��Æ~�S±O�#®[©(� ±,¥è¡%²����Ñ®%£ ©(� ©(¬²H£¤¡xÆ��?é³®�Æ~¯(� �,¥S· ( � i 0!, � ����u � Í , �!£�� µ°¼aæ(þkø ÿ! �$"­ær� % �$"aôÃ×a·&³· ÔÃ�#®Ñ«��?êÄíp®Ñ´�� ¡a£,µ Ïx·��Ë®%£ ¦¤� �?¦¤¦Ñµ�BA·(�w� ��� � ®�Æu¥ ¡a©�µÃ®%©(¨4â�·#Ð4©­¦�³¡a©­´a·Â×Tôaô%ø³·ËÔ*¦£¤¢³±,¦¢³£®%�y£¤��¥¤ýÆu��©(��Æ~� âS®[¦��¡a©V¡��%�S£x¨#®[¦®ðêĨ(�,¯ �,©(¨(�,©*¦x�(���S£¤®%£¤±O�(���S¥,·RLMn6nln e �`���C0-� � �2,?���C0����VL!�^�^��� : ���2,?���e � � ��� i µ�®$®#þ�æ ÿI ×Tôa¼�� % ×Tôr®�"³·â�·°ÔÃ��¯(¥¤�,£V®%©(¨ÖÆg·\Ôï(���S��Æu®%©�·_×Tôaô$�³·�x�éï#®%©³¨(�S£V±S¡�¨(�S¥,·�LMnln6n e �J���C0-� � �2,?���C0P���'L!�^�^��� : � ×

�;,?��� e � � ��� i µ=®*¼(þ2� ÿ! ×��Ã×^" % ×��a¼%¼Ã·

×Tø$�

Page 199: Efficient Algorithms in Computational Learning Theory A thesis presented by Rocco Anthony

Æg·»Ô�¯(� �,��Æu®%©�· ×Tôaô��³·�«y��©(�Ñ®[£�ê0¦¤��Æ~�Ö�,©(±S¡�¨#®%¶³� ��®%©³¨2¨³�S±S¡�¨#®%¶³� �{�,£¤£¤¡%£�êı,¡a£¤£ �S±?¦� ©³¬=±S¡�¨(�S¥,·L*nlnln e �`���C0-� � �;,?���C0<���dL!�^�^��� : ���2,?��� e � � ��� i µC®*¼(þ;� ÿ! ×��a¼%É % ×��%ɳ×a·

&³·típ®%£ ¢(��®[©(¨�í·tí»¥ ¢(ý*� ú ��·è×Tôaôaô³·\«M�Ñ®%£ ©(� ©(¬�ÆÂØ4Çf¶­´{®%¯(¯(£ ¡TéÃ�@Æ5®[¦¤� ©(¬~� ©³±S� ¢³¥¤� ¡%©ÃêÄ�né³±,� ¢(¥ � ¡a©²H¡a£ºÆè¢(�W®%�[·�çÄ© ( �M� �*�*� 3�,.� g 0U�H�¡��� �í¶ ���G�I� �*� ����� I ���^� � � � � �*� ��� I � : �����S���2,?������� I � : ×��� � ô$,.� i µ#¯#®%¬%�S¥�¼Ã×öæ % ¼a¼�"³·

Ðè·#íp®Ñ´�� ¡a£�®%©³¨Í� ·*âÖ®%©(©�·�×Tô��%¼Ã· � 3 Í ��� �*� 3 I ��� � �G�/��0?·Å�w� ���,´'¿�Ôáa©(¥,·BA·�_V�S�#®%£¤®³µ$�r·ðí»¥¤¢(±O�³� ¨#®³µa®%©(¨gçO·ö�C�,¬a�S©³�S£S·�×Tôaô���·R�i¯³¦¤��Æu®%�*®[¦¤¦¤£¤��¶(¢³¦�nêÄ� �u±,� �,©*¦p� �S®%£¤©³� ©(¬V¡%²¨(� ¥Fú ¢(©(±?¦� ¡%©�µ­¯#®%£ ��¦Ä´aµ�®%©³¨~¦¤�(£¤�,¥¤�(¡a��¨~²H¢³©(±,¦¤� ¡a©(¥,·�çÄ© ( �*� �*�*� 3�,.� g 0N�H�z�.� �õe ��,.�`3wno�G�M�*� � ���I ���^� � � � � �*� ��� I � : �����S���2,?�������&a � ���I��,.� gìe � � ��� i µ#¯#®[¬a�S¥A×��Ã× % ×Tø�®(·Ïp·Gp�®%� ¨³´a®³·»×Tôaø%ô³·+ÐÒ©(�,«á®%� ¬%¡a£¤�§¦�hÆ ²H¡a£~Æ~� ©(�@Æu��âS��©(¬A±S¡a©��a�néͲH¢(©(±,¦¤� ¡a©³¥»¡��%�S£�±S¡%©~�%�?éÖ¥¤�?¦¥,·çÄ© ( �*� �*�*� 3�,.� g 0V�H���.� �Æe ��,.�I�.� � ��� / ir: ���#0!,.� : ��� ¶ ������3����2,?���C0V�H� I � : ���G� � � / � , � � �*� µ¯#®%¬a�,¥VÉaÉaø % Ér®­É³·«+·Pp�®%���W®%©­¦Ñ· ×Tôaø�®³· Ð ¦�³�S¡a£�´ ¡%²Í¦�(�w� �S®%£¤©(®%¶(� �[· I � :>: ����, � ���;,?���C0À�H����� �)�gI�� µ¼$�(þ�×a× ÿ! ×a×TÉr® % ×a×^®­¼Ã·«+·=p�®%� � ®%©­¦Ñ·�×Tôaôaô³·ÚÏ+£ ¡ðú �S±?¦� ¡%©Í���Ñ®%£ ©(� ©³¬(· � � � ��,.� � a � ���!��,.� g µ(É��(þ�¼ ÿ! ×a×öæ % ×ÑÉ$"³·pg·=p�®[¯(©(� ý\·»×Tô%ôaø³· / �?���;,10!�2, � ���ta � ���I��,.� gìe � � ��� i ·Å�w��� �,´­ê0çÄ©*¦¤�S£ ¥¤±S���S©(±,�%µ°Ø��?«4�¡a£ ý\·�r·PpÚ�S£ ¶ �,¢(£¤¬[¦Ñ· ×Tô%ô$"³· «M�S®%£¤©(��©(¬ ÆÂØ4Ç ¢(©³¨(�S£=¦�³�7¢(©(��²H¡%£ºÆ ¨³� ¥ ¦¤£¤��¶(¢³¦��¡a©9��©ª±�¢(®%¥¤� ê¯ ¡%��´�©(¡xÆ~�W®%�a¦¤��Æ~�%·xçÄ© ( �M� �*�*� 3�,.� g 0O�H����� �ãe ��,.�`3 � �����=���Rsj���Mt#0*���*� ��� I � : �����S���2,?�������a � ���!��,.� gfe � � ��� i µ#¯#®%¬%�S¥Vɳ×^® % É­¼��³·�r·qpÚ�S£ ¶ �,¢(£¤¬%¦S·�×Tôaôaø³·�«y�S®%£¤©³� ©(¬�¥¤¢³¶Ãêı,�W®%¥ ¥¤�S¥�¡%²½Æu¡%©(¡%¦¡%©(��ÆÂØ4Ç7¡a©Ù¦�(��¢(©³��²H¡a£RÆ�¨(��¥ ¦£ �§ê¶(¢³¦¤� ¡a©�·�çÄ© ( �M� �*�*� 3�,.� g 0>�H�w�.� �Ä´ ,.���.� I ���^� � � � � �*� ��� � � g ���I,.��� : , � a � ���I��,.� gìe � � ��� i µ¯#®%¬a�,¥VÉaø­æ % É%ôaô³·Ðè·(�w��¬a¨(�,£¤¥¤¡%©�·»×Tôaôaô÷ÚÏp�S£ ¥¤¡a©(®%�M±S¡xÆ5Æg¢(©³� ±Ñ®ð¦� ¡%©�·

×Tø��