distance metric learning for large margin nearest neighbor classification (lmnn) nips 2006 kilian q....
TRANSCRIPT
![Page 1: Distance Metric Learning for Large Margin Nearest Neighbor Classification (LMNN) NIPS 2006 Kilian Q. Weinberger, John Blitzer and Lawrence K. Saul](https://reader036.vdocuments.us/reader036/viewer/2022082422/56649d6f5503460f94a50543/html5/thumbnails/1.jpg)
Distance Metric Learning for Large Margin Nearest Neighbor Classification (LMNN)
NIPS 2006Kilian Q. Weinberger, John Blitzer and Lawrence K. Saul
![Page 2: Distance Metric Learning for Large Margin Nearest Neighbor Classification (LMNN) NIPS 2006 Kilian Q. Weinberger, John Blitzer and Lawrence K. Saul](https://reader036.vdocuments.us/reader036/viewer/2022082422/56649d6f5503460f94a50543/html5/thumbnails/2.jpg)
Problem
• Distance metric learning for kNN
Goal:k-nearest neighbors belong to same classExample from different classes are separated by a large margin
kNN LMNN
linear classification Linear SVM
counterpart
![Page 3: Distance Metric Learning for Large Margin Nearest Neighbor Classification (LMNN) NIPS 2006 Kilian Q. Weinberger, John Blitzer and Lawrence K. Saul](https://reader036.vdocuments.us/reader036/viewer/2022082422/56649d6f5503460f94a50543/html5/thumbnails/3.jpg)
Model
• Mahalanobis distance for metric learning
![Page 4: Distance Metric Learning for Large Margin Nearest Neighbor Classification (LMNN) NIPS 2006 Kilian Q. Weinberger, John Blitzer and Lawrence K. Saul](https://reader036.vdocuments.us/reader036/viewer/2022082422/56649d6f5503460f94a50543/html5/thumbnails/4.jpg)
Model
• Goal: learn the linear transformation L
• Cost function: Indicates if a target neighbor
SVM:Hinge loss
![Page 5: Distance Metric Learning for Large Margin Nearest Neighbor Classification (LMNN) NIPS 2006 Kilian Q. Weinberger, John Blitzer and Lawrence K. Saul](https://reader036.vdocuments.us/reader036/viewer/2022082422/56649d6f5503460f94a50543/html5/thumbnails/5.jpg)
How to solve it?
• Convex optimization with semidefinite programming– The matrix whose elements are linear with the
unknown variables is required to be positive semidefinite.
Slack variable
![Page 6: Distance Metric Learning for Large Margin Nearest Neighbor Classification (LMNN) NIPS 2006 Kilian Q. Weinberger, John Blitzer and Lawrence K. Saul](https://reader036.vdocuments.us/reader036/viewer/2022082422/56649d6f5503460f94a50543/html5/thumbnails/6.jpg)
Classification
• kNN
• Energy-based classification
![Page 7: Distance Metric Learning for Large Margin Nearest Neighbor Classification (LMNN) NIPS 2006 Kilian Q. Weinberger, John Blitzer and Lawrence K. Saul](https://reader036.vdocuments.us/reader036/viewer/2022082422/56649d6f5503460f94a50543/html5/thumbnails/7.jpg)
Results
![Page 8: Distance Metric Learning for Large Margin Nearest Neighbor Classification (LMNN) NIPS 2006 Kilian Q. Weinberger, John Blitzer and Lawrence K. Saul](https://reader036.vdocuments.us/reader036/viewer/2022082422/56649d6f5503460f94a50543/html5/thumbnails/8.jpg)
Results
![Page 9: Distance Metric Learning for Large Margin Nearest Neighbor Classification (LMNN) NIPS 2006 Kilian Q. Weinberger, John Blitzer and Lawrence K. Saul](https://reader036.vdocuments.us/reader036/viewer/2022082422/56649d6f5503460f94a50543/html5/thumbnails/9.jpg)
Results
![Page 10: Distance Metric Learning for Large Margin Nearest Neighbor Classification (LMNN) NIPS 2006 Kilian Q. Weinberger, John Blitzer and Lawrence K. Saul](https://reader036.vdocuments.us/reader036/viewer/2022082422/56649d6f5503460f94a50543/html5/thumbnails/10.jpg)