Upload File

compare linear classifiers lda logistic regression linear svm

  • Home
  • Documents
  • Compare Linear Classifiers LDA Logistic Regression Linear SVM
5
Compare Linear Classifiers LDA Linear SVM Estimate mu1, mu0, Sigma then we have P(y) and P(x | y). For binary classification, decision boundary is determined by P(Y=1 | x)/P(Y=0 | x) > 0, which corresponds to the following linear function x t ⌃ -1 (μ 1 - μ 2 )+ ↵ 0 Directly estimate log of the following ratio by a linear function, without any assumptions on P(x) P(Y=1 | x)/P(Y=0 | x) > 0. So if data are generated from a mixture of two normals following the assumption of LDA, then LDA and Logistic should return the same linear function, of course, asymptotically. Estimates from finites samples may differ. arg min f E h 1 - Yf (X )] + = sign ⇣ ⌘ (x) - 1 2 ⌘

Upload: others

Post on 12-Apr-2022

18 views

Category:

Documents


0 download

Report

TRANSCRIPT

Page 1: Compare Linear Classifiers LDA Logistic Regression Linear SVM

Compare Linear Classifiers

LDA Logistic Regression Linear SVM

Estimate mu1, mu0, Sigma then we have P(y) and P(x | y).

For binary classification, decision boundary is determined by

P(Y=1 | x)/P(Y=0 | x) > 0,

which corresponds to the following linear function

xt⌃�1(µ1 � µ2) + ↵0<latexit sha1_base64="zVuYcXluvDypRbh24fxdkO5mUaM=">AAACNnicbVBNS8NAEN3U7/oV9ehlsQiKWJIi6FHQgxeholWhacNku2mX7iZhdyOWkF/lxd/hrRcPinj1J7ipPWjrg2Uf780wMy9IOFPacYZWaWZ2bn5hcam8vLK6tm5vbN6qOJWENkjMY3kfgKKcRbShmeb0PpEURMDpXdA/K/y7ByoVi6MbPUhoS0A3YiEjoI3k25eeAN0Lwuwxb2vsXbOugHZ26OZ4zwti3lEDYb7ME2nuu/gQT4m1fXyAPeBJD3zHtytO1RkBTxN3TCpojLpvv3idmKSCRppwUKrpOoluZSA1I5zmZS9VNAHShy5tGhqBoKqVjc7O8a5ROjiMpXmRxiP1d0cGQhWrmsriSDXpFeJ/XjPV4UkrY1GSahqRn0FhyrGOcZEh7jBJieYDQ4BIZnbFpAcSiDZJl00I7uTJ0+S2VnUNvzqqnJ6P41hE22gH7SEXHaNTdIHqqIEIekJD9IberWfr1fqwPn9KS9a4Zwv9gfX1DaXIq80=</latexit><latexit sha1_base64="zVuYcXluvDypRbh24fxdkO5mUaM=">AAACNnicbVBNS8NAEN3U7/oV9ehlsQiKWJIi6FHQgxeholWhacNku2mX7iZhdyOWkF/lxd/hrRcPinj1J7ipPWjrg2Uf780wMy9IOFPacYZWaWZ2bn5hcam8vLK6tm5vbN6qOJWENkjMY3kfgKKcRbShmeb0PpEURMDpXdA/K/y7ByoVi6MbPUhoS0A3YiEjoI3k25eeAN0Lwuwxb2vsXbOugHZ26OZ4zwti3lEDYb7ME2nuu/gQT4m1fXyAPeBJD3zHtytO1RkBTxN3TCpojLpvv3idmKSCRppwUKrpOoluZSA1I5zmZS9VNAHShy5tGhqBoKqVjc7O8a5ROjiMpXmRxiP1d0cGQhWrmsriSDXpFeJ/XjPV4UkrY1GSahqRn0FhyrGOcZEh7jBJieYDQ4BIZnbFpAcSiDZJl00I7uTJ0+S2VnUNvzqqnJ6P41hE22gH7SEXHaNTdIHqqIEIekJD9IberWfr1fqwPn9KS9a4Zwv9gfX1DaXIq80=</latexit><latexit sha1_base64="zVuYcXluvDypRbh24fxdkO5mUaM=">AAACNnicbVBNS8NAEN3U7/oV9ehlsQiKWJIi6FHQgxeholWhacNku2mX7iZhdyOWkF/lxd/hrRcPinj1J7ipPWjrg2Uf780wMy9IOFPacYZWaWZ2bn5hcam8vLK6tm5vbN6qOJWENkjMY3kfgKKcRbShmeb0PpEURMDpXdA/K/y7ByoVi6MbPUhoS0A3YiEjoI3k25eeAN0Lwuwxb2vsXbOugHZ26OZ4zwti3lEDYb7ME2nuu/gQT4m1fXyAPeBJD3zHtytO1RkBTxN3TCpojLpvv3idmKSCRppwUKrpOoluZSA1I5zmZS9VNAHShy5tGhqBoKqVjc7O8a5ROjiMpXmRxiP1d0cGQhWrmsriSDXpFeJ/XjPV4UkrY1GSahqRn0FhyrGOcZEh7jBJieYDQ4BIZnbFpAcSiDZJl00I7uTJ0+S2VnUNvzqqnJ6P41hE22gH7SEXHaNTdIHqqIEIekJD9IberWfr1fqwPn9KS9a4Zwv9gfX1DaXIq80=</latexit><latexit sha1_base64="ck8pdC+ekZH4nUmSP+ZG7r8lEyk=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odOn4MoA7ncAFXEMIN3MEDdKALAhJ4hXdv4r15H6uuat66tDP4I+/zBzjGijg=</latexit><latexit sha1_base64="UoU/kohMI0eLrJmMtXZwH2Bl6uI=">AAACK3icbVBNSwMxFHzrZ61f1auXoAgVUXZ70aPgxYtQ0arQbZe3abYNJrtLkhXLsr/Ki7/DWy8eFPFvmG17UOtAyDCTR95MmAqujeuOnLn5hcWl5cpKdXVtfWOztrV2q5NMUdaiiUjUfYiaCR6zluFGsPtUMZShYHfhw3np3z0ypXkS35hhyjoS+zGPOEVjpaB26Us0gzDKn4quIf4170vs5kdeQep+mIieHkp75b7MisAjR2RGbByQQ+KjSAcYuEFtzz12xyCzxJuSPZiiGdRe/V5CM8liQwVq3fbc1HRyVIZTwYqqn2mWIn3APmtbGqNkupOPYxdk3yo9EiXKntiQsfpzIkepy1XtyzKk/uuV4n9eOzPRaSfncZoZFtPJR1EmiElI2SHpccWoEUNLkCpudyV0gAqpsU1XbQne38iz5LZx7Fl+5UIFdmAX6uDBCZzBBTShBRSeYQTv8OG8OG/O56SuOWfa2zb8gvP1DUuYqio=</latexit><latexit sha1_base64="UoU/kohMI0eLrJmMtXZwH2Bl6uI=">AAACK3icbVBNSwMxFHzrZ61f1auXoAgVUXZ70aPgxYtQ0arQbZe3abYNJrtLkhXLsr/Ki7/DWy8eFPFvmG17UOtAyDCTR95MmAqujeuOnLn5hcWl5cpKdXVtfWOztrV2q5NMUdaiiUjUfYiaCR6zluFGsPtUMZShYHfhw3np3z0ypXkS35hhyjoS+zGPOEVjpaB26Us0gzDKn4quIf4170vs5kdeQep+mIieHkp75b7MisAjR2RGbByQQ+KjSAcYuEFtzz12xyCzxJuSPZiiGdRe/V5CM8liQwVq3fbc1HRyVIZTwYqqn2mWIn3APmtbGqNkupOPYxdk3yo9EiXKntiQsfpzIkepy1XtyzKk/uuV4n9eOzPRaSfncZoZFtPJR1EmiElI2SHpccWoEUNLkCpudyV0gAqpsU1XbQne38iz5LZx7Fl+5UIFdmAX6uDBCZzBBTShBRSeYQTv8OG8OG/O56SuOWfa2zb8gvP1DUuYqio=</latexit><latexit sha1_base64="HyZRT/1YJy8Ubx5qfy5P3JsqXiI=">AAACNnicbVBNS8NAEN34bf2qevSyWARFLEkvehT04EWoaFVo0jDZbNqlu0nY3Ygl5Fd58Xd468WDIl79CW5qD9r6YNnHezPMzAtSzpS27aE1Mzs3v7C4tFxZWV1b36hubt2qJJOEtkjCE3kfgKKcxbSlmeb0PpUURMDpXdA/K/27ByoVS+IbPUipJ6Abs4gR0Ebyq5euAN0Lovyx6GjsXrOugE5+5BR43w0SHqqBMF/uiqzwHXyEp8TGAT7ELvC0B77tV2t23R4BTxNnTGpojKZffXHDhGSCxppwUKrt2Kn2cpCaEU6LipspmgLpQ5e2DY1BUOXlo7MLvGeUEEeJNC/WeKT+7shBqHJVU1keqSa9UvzPa2c6OvFyFqeZpjH5GRRlHOsElxnikElKNB8YAkQysysmPZBAtEm6YkJwJk+eJreNumP4lV07PR/HsYR20C7aRw46RqfoAjVRCxH0hIboDb1bz9ar9WF9/pTOWOOebfQH1tc3pIiryQ==</latexit><latexit sha1_base64="zVuYcXluvDypRbh24fxdkO5mUaM=">AAACNnicbVBNS8NAEN3U7/oV9ehlsQiKWJIi6FHQgxeholWhacNku2mX7iZhdyOWkF/lxd/hrRcPinj1J7ipPWjrg2Uf780wMy9IOFPacYZWaWZ2bn5hcam8vLK6tm5vbN6qOJWENkjMY3kfgKKcRbShmeb0PpEURMDpXdA/K/y7ByoVi6MbPUhoS0A3YiEjoI3k25eeAN0Lwuwxb2vsXbOugHZ26OZ4zwti3lEDYb7ME2nuu/gQT4m1fXyAPeBJD3zHtytO1RkBTxN3TCpojLpvv3idmKSCRppwUKrpOoluZSA1I5zmZS9VNAHShy5tGhqBoKqVjc7O8a5ROjiMpXmRxiP1d0cGQhWrmsriSDXpFeJ/XjPV4UkrY1GSahqRn0FhyrGOcZEh7jBJieYDQ4BIZnbFpAcSiDZJl00I7uTJ0+S2VnUNvzqqnJ6P41hE22gH7SEXHaNTdIHqqIEIekJD9IberWfr1fqwPn9KS9a4Zwv9gfX1DaXIq80=</latexit><latexit sha1_base64="zVuYcXluvDypRbh24fxdkO5mUaM=">AAACNnicbVBNS8NAEN3U7/oV9ehlsQiKWJIi6FHQgxeholWhacNku2mX7iZhdyOWkF/lxd/hrRcPinj1J7ipPWjrg2Uf780wMy9IOFPacYZWaWZ2bn5hcam8vLK6tm5vbN6qOJWENkjMY3kfgKKcRbShmeb0PpEURMDpXdA/K/y7ByoVi6MbPUhoS0A3YiEjoI3k25eeAN0Lwuwxb2vsXbOugHZ26OZ4zwti3lEDYb7ME2nuu/gQT4m1fXyAPeBJD3zHtytO1RkBTxN3TCpojLpvv3idmKSCRppwUKrpOoluZSA1I5zmZS9VNAHShy5tGhqBoKqVjc7O8a5ROjiMpXmRxiP1d0cGQhWrmsriSDXpFeJ/XjPV4UkrY1GSahqRn0FhyrGOcZEh7jBJieYDQ4BIZnbFpAcSiDZJl00I7uTJ0+S2VnUNvzqqnJ6P41hE22gH7SEXHaNTdIHqqIEIekJD9IberWfr1fqwPn9KS9a4Zwv9gfX1DaXIq80=</latexit><latexit sha1_base64="zVuYcXluvDypRbh24fxdkO5mUaM=">AAACNnicbVBNS8NAEN3U7/oV9ehlsQiKWJIi6FHQgxeholWhacNku2mX7iZhdyOWkF/lxd/hrRcPinj1J7ipPWjrg2Uf780wMy9IOFPacYZWaWZ2bn5hcam8vLK6tm5vbN6qOJWENkjMY3kfgKKcRbShmeb0PpEURMDpXdA/K/y7ByoVi6MbPUhoS0A3YiEjoI3k25eeAN0Lwuwxb2vsXbOugHZ26OZ4zwti3lEDYb7ME2nuu/gQT4m1fXyAPeBJD3zHtytO1RkBTxN3TCpojLpvv3idmKSCRppwUKrpOoluZSA1I5zmZS9VNAHShy5tGhqBoKqVjc7O8a5ROjiMpXmRxiP1d0cGQhWrmsriSDXpFeJ/XjPV4UkrY1GSahqRn0FhyrGOcZEh7jBJieYDQ4BIZnbFpAcSiDZJl00I7uTJ0+S2VnUNvzqqnJ6P41hE22gH7SEXHaNTdIHqqIEIekJD9IberWfr1fqwPn9KS9a4Zwv9gfX1DaXIq80=</latexit><latexit sha1_base64="zVuYcXluvDypRbh24fxdkO5mUaM=">AAACNnicbVBNS8NAEN3U7/oV9ehlsQiKWJIi6FHQgxeholWhacNku2mX7iZhdyOWkF/lxd/hrRcPinj1J7ipPWjrg2Uf780wMy9IOFPacYZWaWZ2bn5hcam8vLK6tm5vbN6qOJWENkjMY3kfgKKcRbShmeb0PpEURMDpXdA/K/y7ByoVi6MbPUhoS0A3YiEjoI3k25eeAN0Lwuwxb2vsXbOugHZ26OZ4zwti3lEDYb7ME2nuu/gQT4m1fXyAPeBJD3zHtytO1RkBTxN3TCpojLpvv3idmKSCRppwUKrpOoluZSA1I5zmZS9VNAHShy5tGhqBoKqVjc7O8a5ROjiMpXmRxiP1d0cGQhWrmsriSDXpFeJ/XjPV4UkrY1GSahqRn0FhyrGOcZEh7jBJieYDQ4BIZnbFpAcSiDZJl00I7uTJ0+S2VnUNvzqqnJ6P41hE22gH7SEXHaNTdIHqqIEIekJD9IberWfr1fqwPn9KS9a4Zwv9gfX1DaXIq80=</latexit><latexit sha1_base64="zVuYcXluvDypRbh24fxdkO5mUaM=">AAACNnicbVBNS8NAEN3U7/oV9ehlsQiKWJIi6FHQgxeholWhacNku2mX7iZhdyOWkF/lxd/hrRcPinj1J7ipPWjrg2Uf780wMy9IOFPacYZWaWZ2bn5hcam8vLK6tm5vbN6qOJWENkjMY3kfgKKcRbShmeb0PpEURMDpXdA/K/y7ByoVi6MbPUhoS0A3YiEjoI3k25eeAN0Lwuwxb2vsXbOugHZ26OZ4zwti3lEDYb7ME2nuu/gQT4m1fXyAPeBJD3zHtytO1RkBTxN3TCpojLpvv3idmKSCRppwUKrpOoluZSA1I5zmZS9VNAHShy5tGhqBoKqVjc7O8a5ROjiMpXmRxiP1d0cGQhWrmsriSDXpFeJ/XjPV4UkrY1GSahqRn0FhyrGOcZEh7jBJieYDQ4BIZnbFpAcSiDZJl00I7uTJ0+S2VnUNvzqqnJ6P41hE22gH7SEXHaNTdIHqqIEIekJD9IberWfr1fqwPn9KS9a4Zwv9gfX1DaXIq80=</latexit><latexit sha1_base64="zVuYcXluvDypRbh24fxdkO5mUaM=">AAACNnicbVBNS8NAEN3U7/oV9ehlsQiKWJIi6FHQgxeholWhacNku2mX7iZhdyOWkF/lxd/hrRcPinj1J7ipPWjrg2Uf780wMy9IOFPacYZWaWZ2bn5hcam8vLK6tm5vbN6qOJWENkjMY3kfgKKcRbShmeb0PpEURMDpXdA/K/y7ByoVi6MbPUhoS0A3YiEjoI3k25eeAN0Lwuwxb2vsXbOugHZ26OZ4zwti3lEDYb7ME2nuu/gQT4m1fXyAPeBJD3zHtytO1RkBTxN3TCpojLpvv3idmKSCRppwUKrpOoluZSA1I5zmZS9VNAHShy5tGhqBoKqVjc7O8a5ROjiMpXmRxiP1d0cGQhWrmsriSDXpFeJ/XjPV4UkrY1GSahqRn0FhyrGOcZEh7jBJieYDQ4BIZnbFpAcSiDZJl00I7uTJ0+S2VnUNvzqqnJ6P41hE22gH7SEXHaNTdIHqqIEIekJD9IberWfr1fqwPn9KS9a4Zwv9gfX1DaXIq80=</latexit>

Directly estimate log of the following ratio by a linear function, without any assumptions on P(x)

P(Y=1 | x)/P(Y=0 | x) > 0.

So if data are generated from a mixture of two normals following the assumption of LDA, then LDA and Logistic should return the same linear function, of course, asymptotically. Estimates from finites samples may differ.

argminf

Eh1� Y f(X)]+

= sign⇣⌘(x)� 1

2

⌘

<latexit sha1_base64="3uBbVqjnJUt9t3137oYx3FKtpUU=">AAACTXicbVFNaxsxFNS6H0ndj7jtsZdHTINNadgNhfYSCGkDPSYQJy7WsmjltxsRrXaR3oaYxX8wl0Jv/Re99JAQSmRnD23SAcEwMw/pjdJKK0dh+DPoPHj46PHK6pPu02fPX6z1Xr46cmVtJY5kqUs7ToVDrQyOSJHGcWVRFKnG4/T088I/PkPrVGkOaVZhXIjcqExJQV5KetMN4MLmvFAmyYAXgk7StNmbA99VOUwggvfwDbLBeAhx8g44727ANnDCc2qcyk0bHHAkMTgf+jTPrJBNNG+2Wm8ISa8fboZLwH0StaTPWuwnvR98Wsq6QENSC+cmUVhR3AhLSmqcd3ntsBLyVOQ48dSIAl3cLNuYw1uvTCErrT+GYKn+PdGIwrlZkfrkYl1311uI//MmNWWf4kaZqiY08vairNZAJSyqhamyKEnPPBHSKv9WkCfCt0H+A7q+hOjuyvfJ0dZm5PnBh/7Ol7aOVfaGrbMBi9hHtsO+sn02YpJdsF/skl0F34PfwXXw5zbaCdqZ1+wfdFZuABG8roc=</latexit><latexit sha1_base64="3uBbVqjnJUt9t3137oYx3FKtpUU=">AAACTXicbVFNaxsxFNS6H0ndj7jtsZdHTINNadgNhfYSCGkDPSYQJy7WsmjltxsRrXaR3oaYxX8wl0Jv/Re99JAQSmRnD23SAcEwMw/pjdJKK0dh+DPoPHj46PHK6pPu02fPX6z1Xr46cmVtJY5kqUs7ToVDrQyOSJHGcWVRFKnG4/T088I/PkPrVGkOaVZhXIjcqExJQV5KetMN4MLmvFAmyYAXgk7StNmbA99VOUwggvfwDbLBeAhx8g44727ANnDCc2qcyk0bHHAkMTgf+jTPrJBNNG+2Wm8ISa8fboZLwH0StaTPWuwnvR98Wsq6QENSC+cmUVhR3AhLSmqcd3ntsBLyVOQ48dSIAl3cLNuYw1uvTCErrT+GYKn+PdGIwrlZkfrkYl1311uI//MmNWWf4kaZqiY08vairNZAJSyqhamyKEnPPBHSKv9WkCfCt0H+A7q+hOjuyvfJ0dZm5PnBh/7Ol7aOVfaGrbMBi9hHtsO+sn02YpJdsF/skl0F34PfwXXw5zbaCdqZ1+wfdFZuABG8roc=</latexit><latexit sha1_base64="3uBbVqjnJUt9t3137oYx3FKtpUU=">AAACTXicbVFNaxsxFNS6H0ndj7jtsZdHTINNadgNhfYSCGkDPSYQJy7WsmjltxsRrXaR3oaYxX8wl0Jv/Re99JAQSmRnD23SAcEwMw/pjdJKK0dh+DPoPHj46PHK6pPu02fPX6z1Xr46cmVtJY5kqUs7ToVDrQyOSJHGcWVRFKnG4/T088I/PkPrVGkOaVZhXIjcqExJQV5KetMN4MLmvFAmyYAXgk7StNmbA99VOUwggvfwDbLBeAhx8g44727ANnDCc2qcyk0bHHAkMTgf+jTPrJBNNG+2Wm8ISa8fboZLwH0StaTPWuwnvR98Wsq6QENSC+cmUVhR3AhLSmqcd3ntsBLyVOQ48dSIAl3cLNuYw1uvTCErrT+GYKn+PdGIwrlZkfrkYl1311uI//MmNWWf4kaZqiY08vairNZAJSyqhamyKEnPPBHSKv9WkCfCt0H+A7q+hOjuyvfJ0dZm5PnBh/7Ol7aOVfaGrbMBi9hHtsO+sn02YpJdsF/skl0F34PfwXXw5zbaCdqZ1+wfdFZuABG8roc=</latexit><latexit sha1_base64="3uBbVqjnJUt9t3137oYx3FKtpUU=">AAACTXicbVFNaxsxFNS6H0ndj7jtsZdHTINNadgNhfYSCGkDPSYQJy7WsmjltxsRrXaR3oaYxX8wl0Jv/Re99JAQSmRnD23SAcEwMw/pjdJKK0dh+DPoPHj46PHK6pPu02fPX6z1Xr46cmVtJY5kqUs7ToVDrQyOSJHGcWVRFKnG4/T088I/PkPrVGkOaVZhXIjcqExJQV5KetMN4MLmvFAmyYAXgk7StNmbA99VOUwggvfwDbLBeAhx8g44727ANnDCc2qcyk0bHHAkMTgf+jTPrJBNNG+2Wm8ISa8fboZLwH0StaTPWuwnvR98Wsq6QENSC+cmUVhR3AhLSmqcd3ntsBLyVOQ48dSIAl3cLNuYw1uvTCErrT+GYKn+PdGIwrlZkfrkYl1311uI//MmNWWf4kaZqiY08vairNZAJSyqhamyKEnPPBHSKv9WkCfCt0H+A7q+hOjuyvfJ0dZm5PnBh/7Ol7aOVfaGrbMBi9hHtsO+sn02YpJdsF/skl0F34PfwXXw5zbaCdqZ1+wfdFZuABG8roc=</latexit>

Page 2: Compare Linear Classifiers LDA Logistic Regression Linear SVM

Compare Linear Classifiers

Linear SVM

argminf

Eh1� Y f(X)]+

= sign⇣⌘(x)� 1

2

⌘

<latexit sha1_base64="3uBbVqjnJUt9t3137oYx3FKtpUU=">AAACTXicbVFNaxsxFNS6H0ndj7jtsZdHTINNadgNhfYSCGkDPSYQJy7WsmjltxsRrXaR3oaYxX8wl0Jv/Re99JAQSmRnD23SAcEwMw/pjdJKK0dh+DPoPHj46PHK6pPu02fPX6z1Xr46cmVtJY5kqUs7ToVDrQyOSJHGcWVRFKnG4/T088I/PkPrVGkOaVZhXIjcqExJQV5KetMN4MLmvFAmyYAXgk7StNmbA99VOUwggvfwDbLBeAhx8g44727ANnDCc2qcyk0bHHAkMTgf+jTPrJBNNG+2Wm8ISa8fboZLwH0StaTPWuwnvR98Wsq6QENSC+cmUVhR3AhLSmqcd3ntsBLyVOQ48dSIAl3cLNuYw1uvTCErrT+GYKn+PdGIwrlZkfrkYl1311uI//MmNWWf4kaZqiY08vairNZAJSyqhamyKEnPPBHSKv9WkCfCt0H+A7q+hOjuyvfJ0dZm5PnBh/7Ol7aOVfaGrbMBi9hHtsO+sn02YpJdsF/skl0F34PfwXXw5zbaCdqZ1+wfdFZuABG8roc=</latexit><latexit sha1_base64="3uBbVqjnJUt9t3137oYx3FKtpUU=">AAACTXicbVFNaxsxFNS6H0ndj7jtsZdHTINNadgNhfYSCGkDPSYQJy7WsmjltxsRrXaR3oaYxX8wl0Jv/Re99JAQSmRnD23SAcEwMw/pjdJKK0dh+DPoPHj46PHK6pPu02fPX6z1Xr46cmVtJY5kqUs7ToVDrQyOSJHGcWVRFKnG4/T088I/PkPrVGkOaVZhXIjcqExJQV5KetMN4MLmvFAmyYAXgk7StNmbA99VOUwggvfwDbLBeAhx8g44727ANnDCc2qcyk0bHHAkMTgf+jTPrJBNNG+2Wm8ISa8fboZLwH0StaTPWuwnvR98Wsq6QENSC+cmUVhR3AhLSmqcd3ntsBLyVOQ48dSIAl3cLNuYw1uvTCErrT+GYKn+PdGIwrlZkfrkYl1311uI//MmNWWf4kaZqiY08vairNZAJSyqhamyKEnPPBHSKv9WkCfCt0H+A7q+hOjuyvfJ0dZm5PnBh/7Ol7aOVfaGrbMBi9hHtsO+sn02YpJdsF/skl0F34PfwXXw5zbaCdqZ1+wfdFZuABG8roc=</latexit><latexit sha1_base64="3uBbVqjnJUt9t3137oYx3FKtpUU=">AAACTXicbVFNaxsxFNS6H0ndj7jtsZdHTINNadgNhfYSCGkDPSYQJy7WsmjltxsRrXaR3oaYxX8wl0Jv/Re99JAQSmRnD23SAcEwMw/pjdJKK0dh+DPoPHj46PHK6pPu02fPX6z1Xr46cmVtJY5kqUs7ToVDrQyOSJHGcWVRFKnG4/T088I/PkPrVGkOaVZhXIjcqExJQV5KetMN4MLmvFAmyYAXgk7StNmbA99VOUwggvfwDbLBeAhx8g44727ANnDCc2qcyk0bHHAkMTgf+jTPrJBNNG+2Wm8ISa8fboZLwH0StaTPWuwnvR98Wsq6QENSC+cmUVhR3AhLSmqcd3ntsBLyVOQ48dSIAl3cLNuYw1uvTCErrT+GYKn+PdGIwrlZkfrkYl1311uI//MmNWWf4kaZqiY08vairNZAJSyqhamyKEnPPBHSKv9WkCfCt0H+A7q+hOjuyvfJ0dZm5PnBh/7Ol7aOVfaGrbMBi9hHtsO+sn02YpJdsF/skl0F34PfwXXw5zbaCdqZ1+wfdFZuABG8roc=</latexit><latexit sha1_base64="3uBbVqjnJUt9t3137oYx3FKtpUU=">AAACTXicbVFNaxsxFNS6H0ndj7jtsZdHTINNadgNhfYSCGkDPSYQJy7WsmjltxsRrXaR3oaYxX8wl0Jv/Re99JAQSmRnD23SAcEwMw/pjdJKK0dh+DPoPHj46PHK6pPu02fPX6z1Xr46cmVtJY5kqUs7ToVDrQyOSJHGcWVRFKnG4/T088I/PkPrVGkOaVZhXIjcqExJQV5KetMN4MLmvFAmyYAXgk7StNmbA99VOUwggvfwDbLBeAhx8g44727ANnDCc2qcyk0bHHAkMTgf+jTPrJBNNG+2Wm8ISa8fboZLwH0StaTPWuwnvR98Wsq6QENSC+cmUVhR3AhLSmqcd3ntsBLyVOQ48dSIAl3cLNuYw1uvTCErrT+GYKn+PdGIwrlZkfrkYl1311uI//MmNWWf4kaZqiY08vairNZAJSyqhamyKEnPPBHSKv9WkCfCt0H+A7q+hOjuyvfJ0dZm5PnBh/7Ol7aOVfaGrbMBi9hHtsO+sn02YpJdsF/skl0F34PfwXXw5zbaCdqZ1+wfdFZuABG8roc=</latexit>

⌘(x) = P (Y = 1|X = x)<latexit sha1_base64="8/y/Su435pxxBxCvxcA4Bvx2vfs=">AAACAHicbZDLSgMxFIbP1Futt1EXLtwEi1A3ZUYE3RQKunBZwV6kHUomzbShmcyQZMQyduOruHGhiFsfw51vY9rOQlt/CHz5zzkk5/djzpR2nG8rt7S8srqWXy9sbG5t79i7ew0VJZLQOol4JFs+VpQzQeuaaU5bsaQ49Dlt+sPLSb15T6VikbjVo5h6Ie4LFjCCtbG69kGHalx6OEEVVCvdVVz0iFoVc+3aRafsTIUWwc2gCJlqXfur04tIElKhCcdKtV0n1l6KpWaE03GhkygaYzLEfdo2KHBIlZdOFxijY+P0UBBJc4RGU/f3RIpDpUahbzpDrAdqvjYx/6u1Ex1ceCkTcaKpILOHgoQjHaFJGqjHJCWajwxgIpn5KyIDLDHRJrOCCcGdX3kRGqdl1/DNWbF6lcWRh0M4ghK4cA5VuIYa1IHAGJ7hFd6sJ+vFerc+Zq05K5vZhz+yPn8AfNiTwA==</latexit><latexit sha1_base64="8/y/Su435pxxBxCvxcA4Bvx2vfs=">AAACAHicbZDLSgMxFIbP1Futt1EXLtwEi1A3ZUYE3RQKunBZwV6kHUomzbShmcyQZMQyduOruHGhiFsfw51vY9rOQlt/CHz5zzkk5/djzpR2nG8rt7S8srqWXy9sbG5t79i7ew0VJZLQOol4JFs+VpQzQeuaaU5bsaQ49Dlt+sPLSb15T6VikbjVo5h6Ie4LFjCCtbG69kGHalx6OEEVVCvdVVz0iFoVc+3aRafsTIUWwc2gCJlqXfur04tIElKhCcdKtV0n1l6KpWaE03GhkygaYzLEfdo2KHBIlZdOFxijY+P0UBBJc4RGU/f3RIpDpUahbzpDrAdqvjYx/6u1Ex1ceCkTcaKpILOHgoQjHaFJGqjHJCWajwxgIpn5KyIDLDHRJrOCCcGdX3kRGqdl1/DNWbF6lcWRh0M4ghK4cA5VuIYa1IHAGJ7hFd6sJ+vFerc+Zq05K5vZhz+yPn8AfNiTwA==</latexit><latexit sha1_base64="8/y/Su435pxxBxCvxcA4Bvx2vfs=">AAACAHicbZDLSgMxFIbP1Futt1EXLtwEi1A3ZUYE3RQKunBZwV6kHUomzbShmcyQZMQyduOruHGhiFsfw51vY9rOQlt/CHz5zzkk5/djzpR2nG8rt7S8srqWXy9sbG5t79i7ew0VJZLQOol4JFs+VpQzQeuaaU5bsaQ49Dlt+sPLSb15T6VikbjVo5h6Ie4LFjCCtbG69kGHalx6OEEVVCvdVVz0iFoVc+3aRafsTIUWwc2gCJlqXfur04tIElKhCcdKtV0n1l6KpWaE03GhkygaYzLEfdo2KHBIlZdOFxijY+P0UBBJc4RGU/f3RIpDpUahbzpDrAdqvjYx/6u1Ex1ceCkTcaKpILOHgoQjHaFJGqjHJCWajwxgIpn5KyIDLDHRJrOCCcGdX3kRGqdl1/DNWbF6lcWRh0M4ghK4cA5VuIYa1IHAGJ7hFd6sJ+vFerc+Zq05K5vZhz+yPn8AfNiTwA==</latexit><latexit sha1_base64="8/y/Su435pxxBxCvxcA4Bvx2vfs=">AAACAHicbZDLSgMxFIbP1Futt1EXLtwEi1A3ZUYE3RQKunBZwV6kHUomzbShmcyQZMQyduOruHGhiFsfw51vY9rOQlt/CHz5zzkk5/djzpR2nG8rt7S8srqWXy9sbG5t79i7ew0VJZLQOol4JFs+VpQzQeuaaU5bsaQ49Dlt+sPLSb15T6VikbjVo5h6Ie4LFjCCtbG69kGHalx6OEEVVCvdVVz0iFoVc+3aRafsTIUWwc2gCJlqXfur04tIElKhCcdKtV0n1l6KpWaE03GhkygaYzLEfdo2KHBIlZdOFxijY+P0UBBJc4RGU/f3RIpDpUahbzpDrAdqvjYx/6u1Ex1ceCkTcaKpILOHgoQjHaFJGqjHJCWajwxgIpn5KyIDLDHRJrOCCcGdX3kRGqdl1/DNWbF6lcWRh0M4ghK4cA5VuIYa1IHAGJ7hFd6sJ+vFerc+Zq05K5vZhz+yPn8AfNiTwA==</latexit>

EY |x⇥1� Y · a

⇤+=

⇥1� a

⇤+· ⌘(x) +

⇥1 + a

⇤+·�1� ⌘(x)

�<latexit sha1_base64="0Sh4VsLuQMAuBfq+EIyruqdcZQo=">AAACfXicbVFdSxtBFJ3d+rlaG+tjXy4GIWnasFuE9KUgaMFHCyYq2WWZnUySIbMfzNwVwzb/or/MN/+KLzq7STFGLwycOefc+Tg3yqTQ6LoPlv1hbX1jc2vb2dn9uPeptv+5p9NcMd5lqUzVdUQ1lyLhXRQo+XWmOI0jya+iyWmpX91ypUWaXOI040FMR4kYCkbRUGHtH/gxxXEUFb9nYXEDf+EOZuBHYgR98OA73IDPBikCnZNB2IJfy/oSPzf6HGnjrgmtF1frHVe5bVQn/G+oqCY4jhPW6m7brQreAm8B6mRRF2Ht3h+kLI95gkxSrfuem2FQUIWCST5z/FzzjLIJHfG+gQmNuQ6KKr0ZHBlmAMNUmZUgVOxyR0FjradxZJxlVHpVK8n3tH6Ow59BIZIsR56w+UXDXAKmUI4CBkJxhnJqAGVKmLcCG1NFGZqBlSF4q19+C3o/2p7Bf47rJ2eLOLbIF3JIGsQjHXJCzskF6RJGHi2wmtZX68k+sr/Z7bnVthY9B+RV2Z1nhaa19g==</latexit><latexit sha1_base64="0Sh4VsLuQMAuBfq+EIyruqdcZQo=">AAACfXicbVFdSxtBFJ3d+rlaG+tjXy4GIWnasFuE9KUgaMFHCyYq2WWZnUySIbMfzNwVwzb/or/MN/+KLzq7STFGLwycOefc+Tg3yqTQ6LoPlv1hbX1jc2vb2dn9uPeptv+5p9NcMd5lqUzVdUQ1lyLhXRQo+XWmOI0jya+iyWmpX91ypUWaXOI040FMR4kYCkbRUGHtH/gxxXEUFb9nYXEDf+EOZuBHYgR98OA73IDPBikCnZNB2IJfy/oSPzf6HGnjrgmtF1frHVe5bVQn/G+oqCY4jhPW6m7brQreAm8B6mRRF2Ht3h+kLI95gkxSrfuem2FQUIWCST5z/FzzjLIJHfG+gQmNuQ6KKr0ZHBlmAMNUmZUgVOxyR0FjradxZJxlVHpVK8n3tH6Ow59BIZIsR56w+UXDXAKmUI4CBkJxhnJqAGVKmLcCG1NFGZqBlSF4q19+C3o/2p7Bf47rJ2eLOLbIF3JIGsQjHXJCzskF6RJGHi2wmtZX68k+sr/Z7bnVthY9B+RV2Z1nhaa19g==</latexit><latexit sha1_base64="0Sh4VsLuQMAuBfq+EIyruqdcZQo=">AAACfXicbVFdSxtBFJ3d+rlaG+tjXy4GIWnasFuE9KUgaMFHCyYq2WWZnUySIbMfzNwVwzb/or/MN/+KLzq7STFGLwycOefc+Tg3yqTQ6LoPlv1hbX1jc2vb2dn9uPeptv+5p9NcMd5lqUzVdUQ1lyLhXRQo+XWmOI0jya+iyWmpX91ypUWaXOI040FMR4kYCkbRUGHtH/gxxXEUFb9nYXEDf+EOZuBHYgR98OA73IDPBikCnZNB2IJfy/oSPzf6HGnjrgmtF1frHVe5bVQn/G+oqCY4jhPW6m7brQreAm8B6mRRF2Ht3h+kLI95gkxSrfuem2FQUIWCST5z/FzzjLIJHfG+gQmNuQ6KKr0ZHBlmAMNUmZUgVOxyR0FjradxZJxlVHpVK8n3tH6Ow59BIZIsR56w+UXDXAKmUI4CBkJxhnJqAGVKmLcCG1NFGZqBlSF4q19+C3o/2p7Bf47rJ2eLOLbIF3JIGsQjHXJCzskF6RJGHi2wmtZX68k+sr/Z7bnVthY9B+RV2Z1nhaa19g==</latexit><latexit sha1_base64="0Sh4VsLuQMAuBfq+EIyruqdcZQo=">AAACfXicbVFdSxtBFJ3d+rlaG+tjXy4GIWnasFuE9KUgaMFHCyYq2WWZnUySIbMfzNwVwzb/or/MN/+KLzq7STFGLwycOefc+Tg3yqTQ6LoPlv1hbX1jc2vb2dn9uPeptv+5p9NcMd5lqUzVdUQ1lyLhXRQo+XWmOI0jya+iyWmpX91ypUWaXOI040FMR4kYCkbRUGHtH/gxxXEUFb9nYXEDf+EOZuBHYgR98OA73IDPBikCnZNB2IJfy/oSPzf6HGnjrgmtF1frHVe5bVQn/G+oqCY4jhPW6m7brQreAm8B6mRRF2Ht3h+kLI95gkxSrfuem2FQUIWCST5z/FzzjLIJHfG+gQmNuQ6KKr0ZHBlmAMNUmZUgVOxyR0FjradxZJxlVHpVK8n3tH6Ow59BIZIsR56w+UXDXAKmUI4CBkJxhnJqAGVKmLcCG1NFGZqBlSF4q19+C3o/2p7Bf47rJ2eLOLbIF3JIGsQjHXJCzskF6RJGHi2wmtZX68k+sr/Z7bnVthY9B+RV2Z1nhaa19g==</latexit>

f(x) = a =?<latexit sha1_base64="qt6x/zSqQGVSHxBW/LGUHnY6sj0=">AAAB9HicbZDLSsNAFIZP6q3WW9Slm8Ei1E1JRNCNWNCFywr2Am0ok+mkHTqZxJlJsYQ+hxsXirj1Ydz5Nk7aLLT1hwMf/zmHOfP7MWdKO863VVhZXVvfKG6WtrZ3dvfs/YOmihJJaINEPJJtHyvKmaANzTSn7VhSHPqctvzRTdZvjalULBIPehJTL8QDwQJGsDaWF1SeTtEVwqauSz277FSdmdAyuDmUIVe9Z391+xFJQio04VipjuvE2kux1IxwOi11E0VjTEZ4QDsGBQ6p8tLZ0VN0Ypw+CiJpSmg0c39vpDhUahL6ZjLEeqgWe5n5X6+T6ODSS5mIE00FmT8UJBzpCGUJoD6TlGg+MYCJZOZWRIZYYqJNTlkI7uKXl6F5VnUN35+Xa7d5HEU4gmOogAsXUIM7qEMDCDzCM7zCmzW2Xqx362M+WrDynUP4I+vzB0f2j9E=</latexit><latexit sha1_base64="qt6x/zSqQGVSHxBW/LGUHnY6sj0=">AAAB9HicbZDLSsNAFIZP6q3WW9Slm8Ei1E1JRNCNWNCFywr2Am0ok+mkHTqZxJlJsYQ+hxsXirj1Ydz5Nk7aLLT1hwMf/zmHOfP7MWdKO863VVhZXVvfKG6WtrZ3dvfs/YOmihJJaINEPJJtHyvKmaANzTSn7VhSHPqctvzRTdZvjalULBIPehJTL8QDwQJGsDaWF1SeTtEVwqauSz277FSdmdAyuDmUIVe9Z391+xFJQio04VipjuvE2kux1IxwOi11E0VjTEZ4QDsGBQ6p8tLZ0VN0Ypw+CiJpSmg0c39vpDhUahL6ZjLEeqgWe5n5X6+T6ODSS5mIE00FmT8UJBzpCGUJoD6TlGg+MYCJZOZWRIZYYqJNTlkI7uKXl6F5VnUN35+Xa7d5HEU4gmOogAsXUIM7qEMDCDzCM7zCmzW2Xqx362M+WrDynUP4I+vzB0f2j9E=</latexit><latexit sha1_base64="qt6x/zSqQGVSHxBW/LGUHnY6sj0=">AAAB9HicbZDLSsNAFIZP6q3WW9Slm8Ei1E1JRNCNWNCFywr2Am0ok+mkHTqZxJlJsYQ+hxsXirj1Ydz5Nk7aLLT1hwMf/zmHOfP7MWdKO863VVhZXVvfKG6WtrZ3dvfs/YOmihJJaINEPJJtHyvKmaANzTSn7VhSHPqctvzRTdZvjalULBIPehJTL8QDwQJGsDaWF1SeTtEVwqauSz277FSdmdAyuDmUIVe9Z391+xFJQio04VipjuvE2kux1IxwOi11E0VjTEZ4QDsGBQ6p8tLZ0VN0Ypw+CiJpSmg0c39vpDhUahL6ZjLEeqgWe5n5X6+T6ODSS5mIE00FmT8UJBzpCGUJoD6TlGg+MYCJZOZWRIZYYqJNTlkI7uKXl6F5VnUN35+Xa7d5HEU4gmOogAsXUIM7qEMDCDzCM7zCmzW2Xqx362M+WrDynUP4I+vzB0f2j9E=</latexit><latexit sha1_base64="qt6x/zSqQGVSHxBW/LGUHnY6sj0=">AAAB9HicbZDLSsNAFIZP6q3WW9Slm8Ei1E1JRNCNWNCFywr2Am0ok+mkHTqZxJlJsYQ+hxsXirj1Ydz5Nk7aLLT1hwMf/zmHOfP7MWdKO863VVhZXVvfKG6WtrZ3dvfs/YOmihJJaINEPJJtHyvKmaANzTSn7VhSHPqctvzRTdZvjalULBIPehJTL8QDwQJGsDaWF1SeTtEVwqauSz277FSdmdAyuDmUIVe9Z391+xFJQio04VipjuvE2kux1IxwOi11E0VjTEZ4QDsGBQ6p8tLZ0VN0Ypw+CiJpSmg0c39vpDhUahL6ZjLEeqgWe5n5X6+T6ODSS5mIE00FmT8UJBzpCGUJoD6TlGg+MYCJZOZWRIZYYqJNTlkI7uKXl6F5VnUN35+Xa7d5HEU4gmOogAsXUIM7qEMDCDzCM7zCmzW2Xqx362M+WrDynUP4I+vzB0f2j9E=</latexit>

a 2 [�1, 1]<latexit sha1_base64="TuCMOp0n0uCA/IV7vaAXK5FcP6o=">AAAB9XicbZDLSgMxFIbPeK31VnXpJlgEF1pmRNBlQRcuK9gLTMeSSTNtaCYZkoxShr6HGxeKuPVd3Pk2ZtpZaOsPgY//nMM5+cOEM21c99tZWl5ZXVsvbZQ3t7Z3dit7+y0tU0Vok0guVSfEmnImaNMww2knURTHIaftcHSd19uPVGkmxb0ZJzSI8UCwiBFsrPWAUZcJ5J95p8gLyr1K1a25U6FF8AqoQqFGr/LV7UuSxlQYwrHWvucmJsiwMoxwOil3U00TTEZ4QH2LAsdUB9n06gk6tk4fRVLZJwyaur8nMhxrPY5D2xljM9Tztdz8r+anJroKMiaS1FBBZouilCMjUR4B6jNFieFjC5goZm9FZIgVJsYGlYfgzX95EVrnNc/y3UW1flPEUYJDOIIT8OAS6nALDWgCAQXP8ApvzpPz4rw7H7PWJaeYOYA/cj5/AGJ6kHk=</latexit><latexit sha1_base64="TuCMOp0n0uCA/IV7vaAXK5FcP6o=">AAAB9XicbZDLSgMxFIbPeK31VnXpJlgEF1pmRNBlQRcuK9gLTMeSSTNtaCYZkoxShr6HGxeKuPVd3Pk2ZtpZaOsPgY//nMM5+cOEM21c99tZWl5ZXVsvbZQ3t7Z3dit7+y0tU0Vok0guVSfEmnImaNMww2knURTHIaftcHSd19uPVGkmxb0ZJzSI8UCwiBFsrPWAUZcJ5J95p8gLyr1K1a25U6FF8AqoQqFGr/LV7UuSxlQYwrHWvucmJsiwMoxwOil3U00TTEZ4QH2LAsdUB9n06gk6tk4fRVLZJwyaur8nMhxrPY5D2xljM9Tztdz8r+anJroKMiaS1FBBZouilCMjUR4B6jNFieFjC5goZm9FZIgVJsYGlYfgzX95EVrnNc/y3UW1flPEUYJDOIIT8OAS6nALDWgCAQXP8ApvzpPz4rw7H7PWJaeYOYA/cj5/AGJ6kHk=</latexit><latexit sha1_base64="TuCMOp0n0uCA/IV7vaAXK5FcP6o=">AAAB9XicbZDLSgMxFIbPeK31VnXpJlgEF1pmRNBlQRcuK9gLTMeSSTNtaCYZkoxShr6HGxeKuPVd3Pk2ZtpZaOsPgY//nMM5+cOEM21c99tZWl5ZXVsvbZQ3t7Z3dit7+y0tU0Vok0guVSfEmnImaNMww2knURTHIaftcHSd19uPVGkmxb0ZJzSI8UCwiBFsrPWAUZcJ5J95p8gLyr1K1a25U6FF8AqoQqFGr/LV7UuSxlQYwrHWvucmJsiwMoxwOil3U00TTEZ4QH2LAsdUB9n06gk6tk4fRVLZJwyaur8nMhxrPY5D2xljM9Tztdz8r+anJroKMiaS1FBBZouilCMjUR4B6jNFieFjC5goZm9FZIgVJsYGlYfgzX95EVrnNc/y3UW1flPEUYJDOIIT8OAS6nALDWgCAQXP8ApvzpPz4rw7H7PWJaeYOYA/cj5/AGJ6kHk=</latexit><latexit sha1_base64="TuCMOp0n0uCA/IV7vaAXK5FcP6o=">AAAB9XicbZDLSgMxFIbPeK31VnXpJlgEF1pmRNBlQRcuK9gLTMeSSTNtaCYZkoxShr6HGxeKuPVd3Pk2ZtpZaOsPgY//nMM5+cOEM21c99tZWl5ZXVsvbZQ3t7Z3dit7+y0tU0Vok0guVSfEmnImaNMww2knURTHIaftcHSd19uPVGkmxb0ZJzSI8UCwiBFsrPWAUZcJ5J95p8gLyr1K1a25U6FF8AqoQqFGr/LV7UuSxlQYwrHWvucmJsiwMoxwOil3U00TTEZ4QH2LAsdUB9n06gk6tk4fRVLZJwyaur8nMhxrPY5D2xljM9Tztdz8r+anJroKMiaS1FBBZouilCMjUR4B6jNFieFjC5goZm9FZIgVJsYGlYfgzX95EVrnNc/y3UW1flPEUYJDOIIT8OAS6nALDWgCAQXP8ApvzpPz4rw7H7PWJaeYOYA/cj5/AGJ6kHk=</latexit>

⇥1� a

⇤· ⌘(x) +

⇥1 + a

⇤·�1� ⌘(x)

�

=1� a ·�1� 2⌘(x))

<latexit sha1_base64="N10+RuKVvuQex2jy+VPMUTnFnkc=">AAACa3icbVFNSwMxEM2uX7V+VT0o6mGwKBWx7BZBL4KgB48KVoXuUrJpWoPZD5JZsZRe/Ine/Ade/A9mty3W1oGQN2/eTJKXIJFCo+N8WvbM7Nz8QmGxuLS8srpWWt940HGqGK+zWMbqKaCaSxHxOgqU/ClRnIaB5I/By1VWf3zlSos4usduwv2QdiLRFoyioZql90PwAtGBBrhwAnSQ+OCxVozgcaSVtyM4/tUcT2mypJJ3j+Q5ZTaveAGHo7lj4oypjdRHxWap7FSdPGAauENQJsO4bZY+vFbM0pBHyCTVuuE6Cfo9qlAwyftFL9U8oeyFdnjDwIiGXPu93Ks+HBimBe1YmRUh5Ox4R4+GWnfDwChDis96spaR/9UaKbbP/Z6IkhR5xAYHtVMJGENmPLSE4gxl1wDKlDB3BfZMFWVoviczwZ188jR4qFVdg+9Oy5fXQzsKZJfskwpxyRm5JDfkltQJI1/WqrVlbVvf9qa9Y+8NpLY17Nkkf8I++AFCL64U</latexit><latexit sha1_base64="N10+RuKVvuQex2jy+VPMUTnFnkc=">AAACa3icbVFNSwMxEM2uX7V+VT0o6mGwKBWx7BZBL4KgB48KVoXuUrJpWoPZD5JZsZRe/Ine/Ade/A9mty3W1oGQN2/eTJKXIJFCo+N8WvbM7Nz8QmGxuLS8srpWWt940HGqGK+zWMbqKaCaSxHxOgqU/ClRnIaB5I/By1VWf3zlSos4usduwv2QdiLRFoyioZql90PwAtGBBrhwAnSQ+OCxVozgcaSVtyM4/tUcT2mypJJ3j+Q5ZTaveAGHo7lj4oypjdRHxWap7FSdPGAauENQJsO4bZY+vFbM0pBHyCTVuuE6Cfo9qlAwyftFL9U8oeyFdnjDwIiGXPu93Ks+HBimBe1YmRUh5Ox4R4+GWnfDwChDis96spaR/9UaKbbP/Z6IkhR5xAYHtVMJGENmPLSE4gxl1wDKlDB3BfZMFWVoviczwZ188jR4qFVdg+9Oy5fXQzsKZJfskwpxyRm5JDfkltQJI1/WqrVlbVvf9qa9Y+8NpLY17Nkkf8I++AFCL64U</latexit><latexit sha1_base64="N10+RuKVvuQex2jy+VPMUTnFnkc=">AAACa3icbVFNSwMxEM2uX7V+VT0o6mGwKBWx7BZBL4KgB48KVoXuUrJpWoPZD5JZsZRe/Ine/Ade/A9mty3W1oGQN2/eTJKXIJFCo+N8WvbM7Nz8QmGxuLS8srpWWt940HGqGK+zWMbqKaCaSxHxOgqU/ClRnIaB5I/By1VWf3zlSos4usduwv2QdiLRFoyioZql90PwAtGBBrhwAnSQ+OCxVozgcaSVtyM4/tUcT2mypJJ3j+Q5ZTaveAGHo7lj4oypjdRHxWap7FSdPGAauENQJsO4bZY+vFbM0pBHyCTVuuE6Cfo9qlAwyftFL9U8oeyFdnjDwIiGXPu93Ks+HBimBe1YmRUh5Ox4R4+GWnfDwChDis96spaR/9UaKbbP/Z6IkhR5xAYHtVMJGENmPLSE4gxl1wDKlDB3BfZMFWVoviczwZ188jR4qFVdg+9Oy5fXQzsKZJfskwpxyRm5JDfkltQJI1/WqrVlbVvf9qa9Y+8NpLY17Nkkf8I++AFCL64U</latexit><latexit sha1_base64="N10+RuKVvuQex2jy+VPMUTnFnkc=">AAACa3icbVFNSwMxEM2uX7V+VT0o6mGwKBWx7BZBL4KgB48KVoXuUrJpWoPZD5JZsZRe/Ine/Ade/A9mty3W1oGQN2/eTJKXIJFCo+N8WvbM7Nz8QmGxuLS8srpWWt940HGqGK+zWMbqKaCaSxHxOgqU/ClRnIaB5I/By1VWf3zlSos4usduwv2QdiLRFoyioZql90PwAtGBBrhwAnSQ+OCxVozgcaSVtyM4/tUcT2mypJJ3j+Q5ZTaveAGHo7lj4oypjdRHxWap7FSdPGAauENQJsO4bZY+vFbM0pBHyCTVuuE6Cfo9qlAwyftFL9U8oeyFdnjDwIiGXPu93Ks+HBimBe1YmRUh5Ox4R4+GWnfDwChDis96spaR/9UaKbbP/Z6IkhR5xAYHtVMJGENmPLSE4gxl1wDKlDB3BfZMFWVoviczwZ188jR4qFVdg+9Oy5fXQzsKZJfskwpxyRm5JDfkltQJI1/WqrVlbVvf9qa9Y+8NpLY17Nkkf8I++AFCL64U</latexit>

Page 3: Compare Linear Classifiers LDA Logistic Regression Linear SVM

Imbalanced/Unbalanced Data

1. If mis-classification rate is the goal, then go with that one-class classification rule.

2. If the two errors, classifying Y=1 to be 0 or classifying Y=0 to be 1, have different consequences, then use asymmetric classification error, which will lead to a prob cut-off value different from the usual 0.5.

3. Use other loss functions (for evaluation) that fit the underlying application. For example, if ranking is of interest, use AUC.

4. Down-sampling or up-sampling or re-weighting. Don’t forget to calibrate your model at the end. 1. Platt’s scaling2. Isotonic Regression

Data: many Y=0, very few Y=1

Problem: cannot beat the rule that predicts everything to be class 0

Page 4: Compare Linear Classifiers LDA Logistic Regression Linear SVM

Infinitely Imbalanced Logistic Regression(by Art Owen, see paper link on Piazza)

Suppose data arey=1, x_{1i}, i=1, …, n_1y=0, x_{0i}, i=1, …., N

Fit a logistic regression model on this data, then consider an extreme situation, N —> infinity. Do the logistic coefficients have a meaningful limit?

To understand the exponential tilt, assume LDA assumption holds.

The intercept —> (-infinity)The slope beta converges to the following, under some mild conditions on F0

Page 5: Compare Linear Classifiers LDA Logistic Regression Linear SVM

Learning Decision Trees with Stochastic Linear Classifiersmansour/students/Tom_Jorgenson_thesis.pdf · Learning Decision Trees with Stochastic Linear Classifiers ... derive efficient

Ensemble-based classifiers

Multiclass Classifiers Based on Dimension Reduction with ...hpark/papers/multiclasslda.pdfMulticlass Classifiers Based on Dimension Reduction with Generalized LDA Hyunsoo Kima Barry

Linear Methods for Classificationkti.mff.cuni.cz/~marta/su3a.pdf · 1 Linear Methods for Classification Linear and Logistic Regression, LDA, QDA, k-NN (k Nearest Neighbors) optimal

Learning Kernel Classifiers

7 Gaussian Discriminant Analysis, including QDA and LDAjrs/189/lec/07.pdfGaussian Discriminant Analysis, including QDA and LDA 37 Linear Discriminant Analysis (LDA) [LDA is a variant

2 Linear Classifiers and Perceptrons - Peoplejrs/189/lec/02.pdfLinear Classifiers and Perceptrons 9 Math Review [I will write vectors in matrix notation.] Vectors: x = 2 666 666

Numerical Linear algebra methods for data miningsaad/PDF/WM_Oct31_2014.pdf · Supervised learning: Linear classification Linear classifiers: Find a hy-perplane which best separates

Face Recognition Using LDA Based Algorithms · 2002. 9. 26. · Face Recognition, Linear Discriminant Analysis (LDA), direct LDA, fractional-step LDA, principle component analysis

From LDA+U to LDA+DMFT

Using LDA - Rutgers Universityyhung/HW586/LDA/Lecture LDA and R.pdf · 1 Using LDA Randy Julian Lilly Research Laboratories Linear Discriminant Analysis Used in Supervised Learning!

The World of ALDA - European Committee of the Regions · lda prijedor (bih) lda zavidoviĆi (bih) lda montenegro (mne) lda central and southern serbia (rs) lda subotica (rs) lda kosovo

Linear combinations of simple classifiers for the PASCAL …dclee/pascal_report.pdf · 2006-06-01 · Linear combinations of simple classifiers for the PASCAL challenge Nik A. Melchior

Bayesian Classifiers - INAOE - Ciencias Computacionalesesucar/Clases-mgp/Notes/c4-clasif.pdf · Semi-Naive Bayesian Classifiers Multidimen. Bayesian Classifiers Bayesian Chain Classifiers

Lecture 9: Large Margin Classifiers. Linear Support Vector

A Family of Large Margin Linear Classifiers and Its …web.engr.oregonstate.edu/~tgd/publications/sadm2010...A Family of Large Margin Linear Classifiers and Its Application in Dynamic

Towards Robust Interpretability with Self-Explaining Neural ......self-explaining models in stages, progressively generalizing linear classifiers to complex yet architecturally explicit

Generalized Correspondence-LDA Models (GC-LDA) for ... · The GC-LDA and Correspondence-LDA models are extensions of Latent Dirichlet Allocation (LDA) [3]. Several Bayesian methods

Linear discriminant analysis, two classes Linear ...gn.dronacharya.info/CSEDept/Downloads/Question... · LDA vs. PCA • Limitations of LDA • Variants of LDA • Other dimensionality

Probabilistic Linear Discriminant Analysis · recognition [1,2]. Most notably, these include Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA). While PCA identifies

Generalized Correspondence-LDA Models (GC-LDA) for Identifying ...clcl/Papers/Neurotopics_NIPS_2016_Final_Plus... · Generalized Correspondence-LDA Models (GC-LDA) for Identifying

lda Documentation - media.readthedocs.org · lda Documentation lda implements latent Dirichlet allocation (LDA) using collapsed Gibbs sampling. lda is fast and can be installed without

Using LDA - Rutgers University LDA and R.pdfUsing LDA Randy Julian Lilly Research Laboratories Linear Discriminant Analysis Used in Supervised Learning! Must know some class information

Applied Machine Learning - Oregon State Universityclasses.engr.oregonstate.edu/eecs/spring2020/cs519-400/...linear classifiers: perceptron, logistic regression, (linear) SVMs, etc

7.1. Maximum Margin Classifiers

Dimensionality Reduction: Linear Discriminant Analysis and ... · Linear Discriminant Analysis (LDA, LDiscA) and Principal Component Analysis (PCA) Summarize D-dimensional input data

Lecture 5: LDA and Logistic Regressionhzhang/math574m/2020... · Both LDA and Logistic regression models rely on the linear-odd assumption, indirectly or directly. However, they estimate

Printed Machine Learning Classifiers

A New Face Recognition Method using PCA, LDA and Neural ......PCA (principal Component Analysis), LDA (Linear Discriminant Analysis) and neural networks is proposed. This method consists

Linear discriminant analysis, two classes Linear ...research.cs.tamu.edu/prism/lectures/pr/pr_l10.pdf · Linear discriminant analysis, ... –LDA can be derived as the Maximum Likelihood

Basis Expansion and Nonlinear SVM - Tsinghua Universitybigeye.au.tsinghua.edu.cn/.../dragonstar_lecture5_nonlinear_svm.pdf · Basis Expansion and Nonlinear SVM! Kai Yu! Linear Classifiers!

Adapting a Pedestrian Detector by ... - Sebastian Ramossebastian-ramos.net/wp-content/uploads/2015/02/cvprw13a.pdf · Adapting a Pedestrian Detector by Boosting LDA Exemplar Classifiers

Linear Discriminant Analysis(LDA)syllabus.cs.manchester.ac.uk/pgt/COMP61021/lectures/LDA.pdf• Case Study: PCA vs. LDA • Relevant Issues ... • Linear discriminant analysis (LDA)

Linear Discriminant Analysis (LDA) for selection cuts :

goalparadistjudiciary.gov.ingoalparadistjudiciary.gov.in/stat/sscjm.pdfUDA UDA UDA UDA LDA LDA LDA LDA LDA LDA LDA 21-02-1986 29-05-2015 29-05-2015 19-05-1998 21-02-1986 29-05-2015