CS#189#Final#Review#Session#–#Part#1#

University#of#California,#Berkeley#

CS#189/289#IntroducBon#to#Machine#Learning#

Advice#

1.  Make#a#table#

•  Rows:#ML#techniques#(SVM,#logisBc#regression,#etc.)#

•  Columns:#

•  Sources#of#overfiQng#

•  Parameters#

•  HyperTparameters#

•  Algorithm#convergence#

•  Common#opBons…regularizaBon,#kernelizaBon,#

etc.#

Advice#

2.  Be#very#comfortable#taking#a#gradient#of#linear#algebra#

to#minimize#a#loss#funcBon#(including#taking#logs#and#

manipulaBng#exponenBal#funcBons)#

3.  Be#very#comfortable#converBng#from#least#squares#to#

the#normal#equaBons#(ideally,#you#can#easily#do#this#

with#an#added#L2#regularizaBon#term)#

4.  Know#how#to#manipulate#expectaBons#(mean,#

variance,#biased,#risk)#

5.  Know#the#effect#of#parameters/hyperTparameters#on#

algorithms#(what#happens#when#lambda/k/beta/etc#

goes#to#infinity)#

Advice#

6.  Know#the#differences/tradeToffs#between#different#techniques#within#a#topic#(SVM#vs#logisBc#regression,#

kTmeans#vs#PCA)#

7.  Review#decision#theory.#It#was#a#sizable#part#of#the#course,#but#didn’t#make#it#into#the#midterm.#

True/False#

c)  The#maximum#likelihood#esBmator#for#the#parameter#θ#

of#a#uniform#distribuBons#[0,#θ]#is#unbiased.#

#False##

f)  There#exists#a#oneTtoTone#feature#mapping#φ#for#every#

valid#kernel#

#False##

g)  For#highTdimensional#data,#kTd#trees#can#be#slower#than#

brute#force#nearest#neighbor#search.#

#True##

h)  If#we#had#infinite#data#and#infinitely#fast#computers,#

kNN#would#be#the#only#algorithm#we#would#study#in#189#

#True#

MulBple#Choice#

a)  You#had#a#very#good#score#on#the#Kaggle#public#test#set,#but#did#poorly#on#the#private#test#set.#This#is#likely#

because#you#overfihed#by#submiQng#mulBple#Bmes#and#

changing#the#following#between#submissions:#

!  λ,#your#penalty#term#

!  η,#your#step#size#!  ε,#your#convergence#criterion#!  Fixing#a#random#bug#

MulBple#Choice#

c)  PuQng#a#standard#Gaussian#prior#on#the#weights#for#

linear#regression#(w#~#N(0,1))#will#result#in#what#type#of#

posterior#distribuBon#on#the#weights?#

!  Laplace#!  Poisson#!  Uniform#

!  None#of#the#above#

MulBple#Choice#

e)  Which#of#these#classifiers#could#have#generated#this#

decision#boundary?#

##

!  Linear#SVM#

!  LogisBc#Regression#!  1TNN#!  None#of#the#above#

MulBple#Choice#

e)  Which#of#these#classifiers#could#have#generated#this#

decision#boundary?#

##

!  Linear#SVM#

!  LogisBc#Regression#!  1TNN#!  None#of#the#above#

MulBple#Choice#

e)  Which#of#these#classifiers#could#have#generated#this#

decision#boundary?#

##

!  Linear#SVM#

!  LogisBc#Regression#!  1TNN#!  None#of#the#above#

b)  Is#there#a#relaBonship#between#this#type#of#input#perturbaBon#and#some#type#of#regularizer?#

#Yes,#L2#regularizer#