Upload File

lecture 17 ml supp - github pages · lecture 17: linear classifiers and support vector machines...

  • Home
  • Documents
  • Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s
26
CS639: Data Management for Data Science Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by Ankur Goswami many slides from David Sontag) 1

Upload: others

Post on 14-Aug-2020

5 views

Category:

Documents


0 download

Report

TRANSCRIPT

Page 1: Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s

CS639:DataManagementfor

DataScienceLecture17:LinearClassifiersandSupportVectorMachines

TheodorosRekatsinas(lecturebyAnkur Goswami manyslidesfromDavidSontag)

1

Page 2: Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s

Today’sLecture

1. Linearclassifiers

2. ThePerceptronAlgorithm

3. SupportVectorMachines

2

Page 3: Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s

Linearclassifier• Let’ssimplifylifebyassuming:

• Everyinstanceisavectorofrealnumbers,x=(x1,…,xn).(Notation:boldfacexisavector.)

• Thereareonlytwoclasses,y=(+1) andy=(-1)

• Alinearclassifier isvectorw ofthesamedimensionasx thatisusedtomakethisprediction:

)(sign)...sign(ˆ 2211 xw ×=+++= nnxwxwxwy

þýü

îíì

<-³+

=0if10 if1

)(signx

x

Page 4: Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s

Example:Linearclassifier

Page 5: Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s

ThePerceptronAlgorithmtolearnaLinearClassifier

Page 6: Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s

Definition:Linearlyseparabledata

Page 7: Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s

Doestheperceptronalgorithmwork?

Page 8: Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s

Propertiesoftheperceptronalgorithm

Page 9: Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s

Problemswiththeperceptronalgorithm

[Wewillseenextweek]

Page 10: Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s

Linearseparators

Page 11: Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s

SupportVectorMachines

Page 12: Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s

Normaltoaplane

Page 13: Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s

Scaleinvariance

Page 14: Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s

Scaleinvariance

Page 15: Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s

Whatisγasafunctionofw?

Page 16: Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s

SupportVectorMachines(SVMs)

Page 17: Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s

Whatifthedataisnotseparable?

Page 18: Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s

Allowingforslack:“Softmargin”SVM

Page 19: Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s

Allowingforslack:“Softmargin”SVM

Page 20: Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s

EquivalentHingeLossFormulation

Page 21: Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s

HingeLossvs0-1Loss

Page 22: Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s

MulticlassSVM

Page 23: Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s

Oneversusallclassification

Page 24: Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s

MulticlassSVM

Page 25: Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s

MulticlassSVM

Page 26: Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s

Whatyouneedtoknow


TEISIAS AND THEODOROS: EAST BOIOTIAN POTTERS · TEISIAS AND THEODOROS: EAST BOIOTIAN POTTERS ... Dionysian Imagery in Archaic Greek Art, ... The Local Scripts of Archaic Greece,

Lecture 16 ML DT - GitHub Pages · Lecture 16: Intro to ML and Decision Trees Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s Lecture 1. Intro

Deep Learning for Entity Matching: A Design Space Exploration...Deep Learning for Entity Matching: A Design Space Exploration Sidharth Mudgal 1, Han Li , Theodoros Rekatsinas , AnHai

Learning Functional Dependencies with Sparse Regression · 2019. 5. 7. · Learning Functional Dependencies with Sparse Regression Zhihan Guo, Theodoros Rekatsinas ABSTRACT We study

How to Escape the Capitalist Axiomatic? - by Theodoros Kyriakides

1 Neural Networks- a brief intro Dr Theodoros Manavis [email protected]

Theodoros Kitsakos: The Midstream Challenge in Eastern Mediterranean

Lecture 15 BM - GitHub Pages · Lecture 14: Bayesian Methods Theodoros Rekatsinas 1. Announcements •We will release grades of Midterm by the end of day today. ... Lecture_15_BM

Lecture 1 Intro - GitHub PagesLecture 1: Intro to Data Science and Course Overview Theodoros Rekatsinas 1 2 3 Big science is data driven. 4 Increasingly many companies see themselves

1 Advanced Databases Data Mining Dr Theodoros Manavis [email protected]

Fonduer: Knowledge Base Constructionfrom Richly ...Fonduer: Knowledge Base Construction from Richly Formatted Data Sen Wu Luke Hsiao Xiao Cheng Braden Hancock Theodoros Rekatsinas

Lecture 22: The Attention Mechanism - GitHub Pages€¦ · Lecture 22: The Attention Mechanism Theo Rekatsinas 1. Why Attention? 2 •Consider machine translation: •We need to pay

SLiMFast: Guaranteed Results for Data Fusion and Source ...SLiMFast: Guaranteed Results for Data Fusion and Source Reliability Theodoros Rekatsinas, Manas Joglekar, Hector Garcia-Molina,

Theodoros Aristodemou Business Profile

THEODOROS ALBANIS AND EVDOXIA KLADOPOULOU, HELLENIC

Lecture 2 Stats - GitHub Pages · Lecture 2: Statistical Inference and Exploratory Data Analysis Theodoros Rekatsinas 1. 2 Announcements • Waiting list: you receive invitations

Agios Theodoros Stoudiths

Theodoros Arvanitis IoT Midlands Presentation Oct 2014

Lecture 5: Message Passing/Belief Propagation · Lecture 5: Message Passing/Belief Propagation Theo Rekatsinas 1. Junction Tree 2 •A clique tree for a triangulatedgraph is referred

HoloClean: Holistic Data Repairs with Probabilistic Inference · 2019-07-12 · HoloClean: Holistic Data Repairs with Probabilistic Inference Theodoros Rekatsinas*, Xu Chu y, Ihab

Theodoros Axiotis- Argo

Maria Kamargianni Prof. Nikitas Nikitakos Dr. Theodoros Lilas · Maria Kamargianni Prof. Nikitas Nikitakos Dr. Theodoros Lilas 3rd International Scientific Conference “Energy and

COM211 Communications and Networks CDA College Theodoros Christophides Email: [email protected]

Theodoros - Photopets

Ioannis S. Arvanitoyannis and Theodoros H. Varzakas

Professor Emeritus Theodoros D. Tsiboukisee.auth.gr/wp-content/uploads/2013/11/Tsiboukis_cv_en.pdfProfessor Emeritus Theodoros D. Tsiboukis Last update on July , [email protected] +

Theodoros Chailis Sample Portfolio

Lecture 18 Eval ML - GitHub Pages · 2019-05-08 · Lecture 17: Evaluating Machine Learning Methods Theodoros Rekatsinas 1. Announcements 1. You can see your exams during office hours

Lecture 25 EDA - GitHub Pages · Lecture 25: EDA Theodoros Rekatsinas 1. Data Visualizations Today 2. Data Visualizations Today 3. 4 Standard Data Visualization Recipe: 1.Loaddataset

Dimitrios Kateris, Ioannis Gravalos, Theodoros Gialamas ......Dimitrios Kateris, Ioannis Gravalos, Theodoros Gialamas, Panagiotis Xyradakis, Anastasios Georgiadis and Roxana Agarici,

Theodoros Chiotis is a poet and literary theorist. He

THEODOROS KITSOS

Oracle Technology Overview Theodoros Demosthenous Principal Consultant [email protected]

Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms

1 Advanced Database Dr Theodoros Manavis [email protected]